Avisynth: Bad Interlaced Source?

tengo6dedos · #1 03-23-2009, 01:38 PM

hello

this is a vhs rip with really bad interlacing, i tried the scripts here for interlaced sources but the problem remains

this was a very old ntsc vhs that i wanted to fix so i went to a store for the rip, the output was this 352x480 crappy video and kinda of a repent feeling

can this source be fixed or half-fixed? anything better than this would be great, i dont really mind speed

here is a 20mb sample:
http://www.mediafire.com/?nqy11ezmnym

many thanks in advance
saluts

Dialhot · #2 03-23-2009, 05:05 PM

The sample is from the source, or something you already encoded ?
Seems to be hard combed, thus I think it's already encoded.
If you want some help, post a part of the source.

Anyway, did you try TDeint ?

tengo6dedos · #3 03-23-2009, 08:20 PM

hello, is from the dvd the store gave me after rippin vhs, maybe they reencoded or something, it doesnt looks that bad in the standalone but i thought i could try to fix it, specialy because of the hard subs, i cant read them in some scenes, i blame the interlacing is that correct? i know many people try to trick you but im sayin the truth

i tried TDeint but seems to still have some interlacing

no way of fixing it?

saluts

Dialhot · #4 03-24-2009, 04:25 AM

Unfortunately then, the crap is done. As I said your image is hard combed now, and removing it will be hard. You can try to proceed with IVTC or something else in the Decomb package:
http://neuron2.net/decomb/decombnew.html

Or the one done by Tritical:
http://forum.doom9.org/showthread.php?t=82264

tengo6dedos · #5 03-24-2009, 04:09 PM

hello, thanks,

ive been playin for a while with IVTC filters and i think im into something, i have few questions i want to ask you, here some caps of what i have so far, can you tell me what you think?

do you think this will look to blurred when playin on standalone?
im using only DGDecode's CPU=4 as a denoiser, i think thats enough

is there a way to get rid of the yellow spot on the second cap?

the output is a 23.976 video, with the frame reduction the audio will be in-sync if i only pulldown the video in HCenc?

thanks
saluts

Dialhot · #6 03-24-2009, 05:54 PM

Result looks good to me (what did you use ?) but need to be checked on part where there are subtitles, because they may be hard to read.

For the spot know as I don't see it

But may be you have chroma issue frequent on VHS. I think cnr2 can help to remove that.

And for the sync, if you play correctly wih IVTC, the frame number changed together with framerate, thus the playing length should be the same. Else the audio won't be in sync, even with soft pulldown.

tengo6dedos · #7 03-25-2009, 07:08 PM

I used AnimeIVTC and treated it with double-hard telecine medicine

The subs checked out O.K. for me

The yellow spot cant see? my eyes failling? or maybe yours

, more than a spot is a fat degraded line beside the top border

[bad french spelling]
l'architecture de Le Corbusier est l'une des choses que j'admire de la france
[/bad french spelling]

saluts

Dialhot · #8 03-26-2009, 08:16 AM

Quote:

Originally Posted by tengo6dedos

more than a spot is a fat degraded line beside the top border

That makes me think about a permanent defect in the magnetic tape (or the VCR used for reading). I don't think it can be fixed.

Quote:

[bad french spelling]
l'architecture de Le Corbusier est l'une des choses que j'admire de la france
[/bad french spelling]

Not bad except that France takes an upper caps !

tengo6dedos · #9 03-26-2009, 07:46 PM

Cnr2 made the white building look whiter, it does have some yellow tint, but lines keep on

I'm happy with the result that i got so far, im just tryin to get optiomal n to learn a bit more about avisynth while i still have some free time

Quote:

Not bad except that France takes an upper caps ! Laughing

je, all the worrying about the complicated stuff made me forget about the basics

saluts

tengo6dedos · #10 04-16-2009, 10:49 AM

Hello again

The intelacing part is solved but now I encountered a new problem, is a very ugly noise in black areas, I cant clean it with nothing, when I try converting it becomes in av ery ugly sequence of blocks. The normal colour areas dont have this, does this noise have a specific name?

here some caps from source:

here is when the shot goes black and the blocks of the compressed mpg completely ruins my viewing experience

here in LC's body

Is there a way of fixing or make it look better?
thanks n saluts

Dialhot · #11 04-16-2009, 04:07 PM

Denoising that would be tricky. All you can do is avoiding as much as possible macroblocks while encoding to MPEG2. For that just that line at the end of you script:

Code:

Blockbuster(method="noise",detail_min=1,detail_max=10,variance=0.3,seed=5623)

tengo6dedos · #12 04-16-2009, 06:00 PM

hey! I noticed something, for a wierd reason Virtualdub decrease the bright when im doin something else with other program, when VD is in second hand, and I noticed the noise was completely gone !
I did this manually with a simple:

Code:

Tweak(bright=-20)

and the result is as follows:

Its like it made the noise invisible but is still there?, is wierd
saluts

Dialhot · #13 04-16-2009, 06:02 PM

What is so suprising ? The video is played darker, so you see less artifacts.
But you should not see a lot of light in the other images too

tengo6dedos · #14 04-16-2009, 08:51 PM

Well I disagree, it helps very much, is surprising. Maybe not using something strong as "Bright=-20" but does save you a lot of filtering of this ugly noise, noise that I could not delete with nothing, plus the .png is more than half size! and is very fast

and is not that strong, this is with still tweak(bright=-20)

and the noise well..

Is like the same noise causing this problem hides in the bright and enhace it, and by reducing the bright you also kill the noise?

saluts

p.s.: "helps very much" is that bad english grammar? sounds wrong

Dialhot · #15 04-17-2009, 07:45 AM

Actually, looking at "before/after" for both dark screen an normal picture, it seems that your brightness was way to high originally (look at the green of the leaves: more natural on the right).

It's possible that a bad mapping 0-255 -> 16-235 applied during VHS transfer. Better then would be to use "ColorYUV(levels="TV->PC")". Try this instead of Tweak if the tweak on the bright makes the dark scenes too dark.

Reducing the bright does not kill the noise, it lowers it below visible level.

tengo6dedos · #16 04-29-2009, 08:09 PM

Hello there Dialhot, I think i found the solution for this source, I used a script made by videoFred in doom9 forum in The power of Avisynth: restoring old 8mm films thread, also very good for VHS

I still need to tweak some things specially sharpening, the script is a bit complex and im understanding it little by little. Is not that slow, i got 6 fps on my dual core thanks to the multicore optimizations that the script has and thanks to the half resolution size that vhs had. Even this oversharped .png have less weight than the original. I also added aWarpsharp to remove a blue shadow that the subs and some parts of the image had and chroma shift:

Code:

Vshift=2
Hshift=2
mergechroma(last.crop(Hshift,Vshift,0,0).addborders(0,0,Hshift,Vshift))
c=aWarpSharp(depth=20.0, thresh=0.75, blurlevel=2, cm=1)
mergechroma(c)

and here the original script by videoFred directly:

Code:

# film restoring script by videoFred.
# denoising, resizing, stabilising, sharpening, auto-levels and auto-white balance.


film="F:\002_dodcaps_tebewaren\privé 64\privé_64T_0014.avi" # source clip, you must specify the full path here

short="L"  # L=long clip  S=short clip try it!

result="resultS3" # specify the wanted output here 

trim_begin=6  trim_end=6   play_speed=18.75   #trim frames and play speed (PAL: 16.6666 or 18.75)

saturation=1.0   #saturation

X=4 gamma=0.7  # X is a special parameter for reducing the autolevels effect

black_level=0  white_level=255 output_black=0  output_white=255 # manual levels, when returning result4

AGC_max_sat=2  AGC_max_gain=1.0  #parameters of HDRAGC filter, improves colors and shadows
 
blue=-4  red=2  #manual color adjustment, when returning result2. Values can be positive or negative

denoising_strenght=800  #denoising level of first denoiser: MVDegrainMulti() 
denoising_frames= 4  #number of frames for averaging (forwards and backwards) 3 is a good start value
block_size= 16  #block size of MVDegrainMulti()
block_over= 4  #block overlapping of MVDegrainMulti()

temp_radius=20  temp_luma=6  temp_chroma=6 #second denoiser: TemporalSoften

grain_luma=10  grain_chroma=10
# this will add some digital grain to the final result, set it to zero if you do not want it.

LSF_sharp_ness=250 LSF_radi_us=3  LSF_sub=1.5 #first sharpening parameters (LimitedSharpenFaster) sub=subsampling

USM_sharp_ness=40    USM_radi_us=2  USM_thres_hold=0  #second sharpening parameters (UnsharpMask)

USM_sharp_ness2=20    USM_radi_us2=1  USM_thres_hold2=0  #third sharpening parameters (UnsharpMask)

maxstab=60  #maximum values for the stabiliser (in pixels) 20 is a good start value

est_left=20   est_top=20  est_right=20  est_bottom=20  est_cont=0.8  #crop values for special Estimate clip
CLeft=30  CTop=30  CRight=30  CBottom=30  #crop values after Depan and before final resizing (40,30,40,30)

W=720  H=576 #final size from the returned clip

bord_left=0  bord_top=0  bord_right=0  bord_bot=0
#you can add black borders after resizing, final size is then size + borders!!


# End variables, begin script
#====================================================================================================


SetMemoryMax(1024)  #set this to 1/3 of the available memory


Loadplugin("Depan.dll")
LoadPlugin("DepanEstimate.dll")
Loadplugin("removegrain.dll")
LoadPlugin("AGC.dll")
LoadPlugin("MVTools.dll")
Loadplugin("mt_masktools.dll")
LoadPlugin("MaskTools.dll")
Loadplugin("warpsharp.dll")
LoadPlugIn("LimitedSupport_09Jan06B.dll")
LoadPlugin("MT.dll")
LoadPlugin("autolevels.dll")
LoadPlugin("AddGrainC.dll")
Import("LimitedSharpenFaster.avs")


SetMTMode(5)


source1= Avisource(film).assumefps(play_speed).trim(trim_begin,0).converttoYV12()
end= source1.framecount()
end2= end-trim_end
frames=end+trim_begin

skip= end2/5
skip0=skip+3
skipend= 3

skipend1= skip0+3

skip2= skipend1+skip
skipend2= skip2+3

skip3= skipend2+skip
skipend3= skip3+3

skip4= skipend3+skip
skipend4= skip4+3

skip5= end2-3

L= trim(source1,0,end2)
LS= trim(source1,0,end2).scriptclip("""subtitle("frame  "+string(trim_begin+current_frame)\
+"  from  "+string(frames)+"   trim_begin-"+string(trim_begin)+"   trim_end-"\
+string(trim_end),x=100,y=60,size=32)""")

sourceT1= trim(source1,0,skipend).scriptclip("""subtitle("frame  "+string(trim_begin+current_frame)\
+"  from  "+string(frames)+"   trim_begin-"+string(trim_begin)+"   trim_end-"+string(trim_end)\
+"  steps: "+string(skip),x=100,y=60,size=32)""")

sourceT2= trim(source1,skip0,skipend1).scriptclip("""subtitle("frame  "+string(trim_begin+skip0+current_frame)\
+"  from  "+string(frames)+"   trim_begin-"+string(trim_begin)+"   trim_end-"+string(trim_end)+"  steps: "\
+string(skip),x=100,y=60,size=32)""")

sourceT3= trim(source1,skip2,skipend2).scriptclip("""subtitle("frame  "+string(trim_begin+skip2+current_frame)\
+"  from  "+string(frames)+"   trim_begin-"+string(trim_begin)+"   trim_end-"+string(trim_end)+"  steps:\
 "+string(skip),x=100,y=60,size=32)""")

sourceT4= trim(source1,skip3,skipend3).scriptclip("""subtitle("frame  "+string(trim_begin+skip3+current_frame)\
+"  from  "+string(frames)+"   trim_begin-"+string(trim_begin)+"   trim_end-"+string(trim_end)+\
"  steps: "+string(skip),x=100,y=60,size=32)""")

sourceT5= trim(source1,skip4,skipend4).scriptclip("""subtitle("frame  "+string(trim_begin+skip4+current_frame)\
+"  from  "+string(frames)+"   trim_begin-"+string(trim_begin)+"   trim_end-"+string(trim_end)+\
"  steps: "+string(skip),x=100,y=60,size=32)""")

sourceT6= trim(source1,skip5,end2).scriptclip("""subtitle("frame  "+string(trim_begin+skip5+current_frame)\
+"  from  "+string(frames)+"   trim_begin-"+string(trim_begin)+"   trim_end-"+string(trim_end)+\
"  steps: "+string(skip),x=100,y=60,size=32)""")

SS= sourceT1+sourceT2+sourceT3+sourceT4+sourceT5+sourceT6


sourceT10= trim(source1,0,skipend)
sourceT20= trim(source1,skip0,skipend1)
sourceT30= trim(source1,skip2,skipend2)
sourceT40= trim(source1,skip3,skipend3)
sourceT50= trim(source1,skip4,skipend4)
sourceT60= trim(source1,skip5,end2)

S= sourceT10+sourceT20+sourceT30+sourceT40+sourceT50+sourceT60


stab_reference= eval(short).crop(est_left,est_top,-est_right,-est_bottom)\
.tweak(cont=est_cont).binarize(threshold=80).greyscale().invert()

mdata=DePanEstimate(stab_reference,trust=1.0,dxmax=maxstab,dymax=maxstab)
stab=DePanStabilize(eval(short),data=mdata,cutoff=0.5,dxmax=maxstab,dymax=maxstab,method=1,mirror=15)
stab2= stab.crop(CLeft,CTop,-CRight,-CBottom).tweak(sat=saturation)
stab3=DePanStabilize(eval(short),data=mdata,cutoff=0.5,dxmax=maxstab,dymax=maxstab,method=1,info=true)

WS= width(stab)
HS= height(stab)
stab4= stab3.addborders(10,10,10,10,$B1B1B1).Lanczos4Resize(WS,HS)


vectors= stab2.MVAnalyseMulti(refframes=denoising_frames, pel=2, blksize=block_size, overlap=block_over, idx=1)
denoised= stab2.MVDegrainMulti(vectors, thSAD=denoising_strenght, SadMode=1, idx=1).tweak(sat=saturation)




leveled= denoised.HDRAGC(coef_gain=2.0,max_gain=AGC_max_gain,min_gain=0.5,max_sat=AGC_max_sat,shadows=true)

sharp1=limitedSharpenFaster(leveled,smode=1,strength=LSF_sharp_ness,overshoot=50,\
radius=LSF_radi_us, ss_X=LSF_sub, SS_Y=LSF_sub, dest_x=W,dest_y=H)

sharp2= unsharpmask(sharp1,USM_sharp_ness,USM_radi_us,USM_thres_hold)
sharpX= unsharpmask(sharp2,USM_sharp_ness2,USM_radi_us2,USM_thres_hold2)

sharp3= sharpX.TemporalSoften(temp_radius,temp_luma,temp_chroma,15,2).addgrainC(grain_luma,grain_chroma,0.2,0.2,5)




#backward_vectors = sharp3.MVAnalyse(isb = true,truemotion=true,idx=2)
#forward_vectors = sharp3.MVAnalyse(isb = false,truemotion=true,idx=2)
#frameclip=MVFlowFps(sharp3,backward_vectors, forward_vectors, num=25, den=1, ml=100, idx=2)



result1= sharp3.addborders(X,0,0,0,$FFFFFF).levels(0,gamma,255,0,255).autolevels().coloryuv(autowhite=true)\
.crop(X,0,-0,-0).addborders(bord_left, bord_top, bord_right, bord_bot)

result2= sharp3.levels(black_level,gamma,white_level,0,255).coloryuv(autowhite=true)\
.addborders(bord_left, bord_top, bord_right, bord_bot)

result3= sharp3.coloryuv(off_U=blue,off_V=red).levels(0,gamma,255,0,255).addborders(X,0,0,0,$FFFFFF)\
.autolevels().crop(X,0,-0,-0).addborders(bord_left, bord_top, bord_right, bord_bot)

result4= sharp3.coloryuv(off_U=blue,off_V=red).levels(black_level,gamma,white_level,0,255)\
.addborders(bord_left, bord_top, bord_right, bord_bot)

result5= overlay(eval(short),stab_reference,x=est_left,y=est_top).addborders(2,2,2,2,$FFFFFF).Lanczos4Resize(WS,HS)


W2= W+bord_left+bord_right
H2= H+bord_top+bord_bot

short2=short+"S"

source2=Lanczos4Resize(eval(short2),W2,H2)
source3=Lanczos4Resize(eval(short2),W,H)

resultS1= stackhorizontal(subtitle(source2,"original",size=32,align=2)\
,subtitle(result1,"autolevels, autowhite",size=28,align=2))

resultS2= stackhorizontal(subtitle(source2,"original",size=32,align=2)\
,subtitle(result2,"autowhite, manual levels correction",size=28,align=2))

resultS3= stackhorizontal(subtitle(source2,"original",size=32,align=2)\
,subtitle(result3,"autolevels + manual color correction",size=28,align=2))

resultS4= stackhorizontal(subtitle(source2,"original",size=32,align=2)\
,subtitle(result4,"manual colors and levels correction",size=28,align=2))

resultS2H= stackhorizontal(subtitle(source2,"original",size=32,align=2)\
,subtitle(result2,"autowhite, manual levels\
 correction",size=28,align=2).histogram(mode="levels"))

resultS3H= stackhorizontal(subtitle(source2,"original",size=32,align=2),subtitle(result3,"autolevels + manual color\ 
correction",size=28,align=2).histogram(mode="levels"))

resultS4H= stackhorizontal(subtitle(source2,"original",size=32,align=2),subtitle(result4,"manual colors and levels\ 
correction",size=28,align=2).histogram(mode="levels"))

result6= stackhorizontal(subtitle(result5,"baseclip for stabiliser -only the B/W clip is used",size=32,align=2)\
,subtitle(stab4,"test stabiliser: dx=horizontal, dy=vertical",size=32,align=5))


Eval(result)

I thought it might interest you or someone in the forum

saluts

Dialhot · #17 04-30-2009, 04:32 AM

I did not read all the script but I see that it uses an "autolevel" filter.
Can you tell me if it is efficient ?

All things I've tested (including the autolevel included in the avisynth command "ColorYUV") has the big default to turn a fade out to a white screen !
(explanation: as the luma goes down for the fade out, the autolevel filter tries to compensate it by rising the brightness).

tengo6dedos · #18 04-30-2009, 01:52 PM

The autolevel was very good for this source, but even if your not satisfied you can use the "manual level" output that the script has

The stabiliser also is very good, the typical vhs shake is completely gone

Saluts

Dialhot · #19 05-01-2009, 04:28 AM

You're right. I went on the author's topic and it's impressive how he manage to remove the hand shaking on his 8mm film:
http://www.vimeo.com/2827387

Can you make me save some time telling me where I can find the autolevel plugin ?

tengo6dedos · #20 05-01-2009, 02:49 PM

"This filter is an improvement of the ColorYUV filter's autogain feature. Basically,
it stretches the luma histogram to use the entire valid range.
Unlike ColorYUV, this filter averages the amount of "gain" over consecutive frames
to better handle flashes and to avoid flickering.
No averaging is done when a scene change is detected so no artificial fade effect
is introduced."

Autolevels 0.3

I think is just what you were looking for..

Saluts