This is just an example script which fails to execute, I am not actually thinking of putting FixRips for whole video..
Something haywire about that copy of FixRipsP2. Either the text is actually corrupt, or it has somehow been changed and there is a weird piece of punctuation, a letter out in a margin somewhere -- or something odd about it. It didn't work on my system, either.
Attached is a new copy of fixRipsP2, the same copy I have in my Avisynth plugins. Works for me.
Meanwhile, the script you posted above has logic errors. For example, you don't run vInverse before deinterlacing, you ruin it after deinterlacing. You're also making the mistake of applying Santiag to an interlaced video. Santiag is an anti-aliasing plugin, it requires non-interlaced video. And so on.....
Here is a new copy of your test script. It has a new copy of FixRipsP2 in it:
Code:
AviSource("D:\test2.avi")
ConvertToYV12(interlaced=true,ChromaOutPlacement="MPEG2")
AssumeTFF()
SeparateFields()
e=SelectEven().FixRipsP2()
o=SelectOdd().FixRipsP2()
Interleave(e,o)
Weave()
QTGMC(preset="very slow",EZDenoise=2,denoiser="dfttest",ChromaMotion=true, border=true,ChromaNoise=true,DenoiseMC=true,GrainRestore=0.3,TR2=2)
vInverse2()
Santiag(2,2)
MergeChroma(aWarpSharp2(Depth=20))
GradFun2DBmod(thr=1.8)
AddGrainC(1.75,1.75)
Crop(2,0,-28,-12)
AddBorders(16,6,14,6, color=$000000)
nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=960, fheight=720)
ConvertToRGB32(interlaced=false,matrix="Rec601",ChromaInPlacement="MPEG2")
return last
# ################ FixRipsP function, modified from lordsmurf median filter ########################
# http://www.digitalfaq.com/forum/video-restore/8048-identifying-artifacts-fixed.html#post49760
# ###################--- adapted from poisondeathray original in Videdohelp ---#####################
# https://forum.videohelp.com/threads/343296-Transfer-of-real-bad-tape-to-pc-and-restore#post2139402
#
# ######################### for progressive video or SeparateFields() ##############################
function FixRipsP2(clip a)
{
a
clense(reduceflicker=false).merge(last,0.5).clense(reduceflicker=false)
mot=removegrain(11,0).removegrain(20,0).DepanEstimate(range=2)
take2=a.depaninterleave(mot,prev=2,next=2,subpixel=2)
clean1=take2.TMedian2().selectevery(5,2)
sup1 = clean1.minblur(1).removegrain(11,0).removegrain(11,0)
\ .mt_lutxy(clean1,"x 1 + y < x 2 + x 1 - y > x 2 - y ? ?",U=2,V=2)
\ .msuper(pel=2,sharp=0)
sup2 = a.msuper(pel=2,levels=1,sharp=2)
bv22=sup1.manalyse(isb=true, truemotion=false,global=true,delta=2,blksize=16,overlap=8,search=5,searchparam=4,DCT=5)
bv21=sup1.manalyse(isb=true, truemotion=false,global=true,delta=1,blksize=16,overlap=8,search=5,searchparam=4,DCT=5)
fv21=sup1.manalyse(isb=false,truemotion=false,global=true,delta=1,blksize=16,overlap=8,search=5,searchparam=4,DCT=5)
fv22=sup1.manalyse(isb=false,truemotion=false,global=true,delta=2,blksize=16,overlap=8,search=5,searchparam=4,DCT=5)
interleave(a.mcompensate(sup2,fv22),a.mcompensate(sup2,fv21),a,a.mcompensate(sup2,bv21),a.mcompensate(sup2,bv22))
TMedian2().selectevery(5,2)
sup3 = last.msuper(pel=2,sharp=2)
bv33=sup3.manalyse(isb=true, truemotion=false,global=true,delta=3,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
bv32=sup3.manalyse(isb=true, truemotion=false,global=true,delta=2,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
bv31=sup3.manalyse(isb=true, truemotion=false,global=true,delta=1,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
fv31=sup3.manalyse(isb=false,truemotion=false,global=true,delta=1,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
fv32=sup3.manalyse(isb=false,truemotion=false,global=true,delta=2,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
fv33=sup3.manalyse(isb=false,truemotion=false,global=true,delta=3,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
last.mdegrain3(sup3,bv31,fv31,bv32,fv32,bv33,fv33,thSAD=499)
interleave()
return last
}
function MinBlur(clip clp, int r, int "uv")
{
uv = default(uv,3)
uv2 = (uv==2) ? 1 : uv
rg4 = (uv==3) ? 4 : -1
rg11 = (uv==3) ? 11 : -1
rg20 = (uv==3) ? 20 : -1
medf = (uv==3) ? 1 : -200
RG11D = (r==0) ? mt_makediff(clp,clp.sbr(),U=uv2,V=uv2)
\ : (r==1) ? mt_makediff(clp,clp.removegrain(11,rg11),U=uv2,V=uv2)
\ : (r==2) ? mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20),U=uv2,V=uv2)
\ : mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20).removegrain(20,rg20),U=uv2,V=uv2)
RG4D = (r<=1) ? mt_makediff(clp,clp.removegrain(4,rg4),U=uv2,V=uv2)
\ : (r==2) ? mt_makediff(clp,clp.medianblur(2,2*medf,2*medf),U=uv2,V=uv2)
\ : mt_makediff(clp,clp.medianblur(3,3*medf,3*medf),U=uv2,V=uv2)
DD = mt_lutxy(RG11D,RG4D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
clp.mt_makediff(DD,U=uv,V=uv)
return(last)
}
# median of 5 clips from helpers.avs by G-force
function Median2(clip "input_1", clip "input_2", clip "input_3", clip "input_4", clip "input_5", string "chroma")
{
chroma = default(chroma,"process") #default is "process". Alternates: "copy first" or "copy second"
#MEDIAN(i1,i3,i5)
Interleave(input_1,input_3,input_5)
chroma == "process" ? Clense(reduceflicker=false) : Clense(reduceflicker=false,grey=true)
m1 = selectevery(3,1)
#MAX(MIN(i1,i3,i5),i2)
m2 = input_1.MT_Logic(input_3,"min",chroma=chroma).MT_Logic(input_5,"min",chroma=chroma).MT_Logic(input_2,"max",chroma=chroma)
#MIN(MAX(i1,i3,i5),i4)
m3 = input_1.MT_Logic(input_3,"max",chroma=chroma).MT_Logic(input_5,"max",chroma=chroma).MT_Logic(input_4,"min",chroma=chroma)
Interleave(m1,m2,m3)
chroma == "process" ? Clense(reduceflicker=false) : Clense(reduceflicker=false,grey=true)
selectevery(3,1)
chroma == "copy first" ? last.MergeChroma(input_1) : chroma == "copy second" ? last.MergeChroma(input_2) : last
Return(last)
}
function TMedian2(clip c) {
Median2(c.selectevery(1,-2), c.selectevery(1,-1), c, c.selectevery(1,1), c.selectevery(1,2) ) }
\
Back to work on the project. But people are arriving to take me out for my birthday dinner, so I'll be interrupted again. Shucks. Oh, well, I'll be back later. I'm getting pretty hungry anyway.
Despite all the interruptions, I finally managed to get a filtered and color-corrected version of Test2Proper.avi, attached as "Test2Proper_All_Scenes_720p50.mp4". I think it looks better , but it will never be fixed 100%. I tried a number of tricks with Avisynth and VirtualDub, and since you have NeatVideo i used it in low-power tweak mode on the last two shots in the clip. I found NV inappropriate in most scenes, where it made trees look like giant sponges. I will have to clean up my messy copies of the 10 scripts I used for the 10 scenes. You might not want to use everything I tried, but when I post the scripts later they might give you some useful ideas. I wish I'd had time tom do more with the second sample. Perhaps I can get to it.
It's likely not possible for all readers to download all 535MB of the huffyuv capture I worked with, so I have attached a smaller mop4 encoded version of it (of course the mp4 isn't lossless but it might give readers a better idea of what the troubles are). Attached as "Test2Proper_encoded_capture.mp4". I wish it had been captured with better control of input levels.
A few days ago, Turner Classic Movies showed some restorations of formerly lost classic films. You should see the shambles and damage in those sources: images with half the frames torn away, long segments with still photos where frames were missing, etc.
Filtering details to come later.
Quote:
Originally Posted by glazuna
About that function, I still get the same error with the attached FixRipsP2 aswell as with the one in the code you pasted
I find this upsetting, because instead of using my own copy of the .avs filter I made an exact copy of the script you posted yesterday ran it on my system with no errors. One possibiolity is tyhat you might be missing something like a VisualC++ runtime .dll. Sometimes you can get strange error messages if something like that is missing. Avisynth plugins require VC runtimes from from 2008 on up to 2015 and probably more. there is an entire page of those runtimes and a package of several bundled togther at Test2Proper_All_Scenes_720p50.mp4. There is another list at https://www.microsoft.com/en-us/down...table+package.
.
Last edited by sanlyn; 05-15-2019 at 03:57 PM.
The following users thank sanlyn for this useful post:
glazuna (05-16-2019)
Hopefully this demo post will give you a few ideas.
The original lossless ProperTest.avi sample tghat was attached to post #10 (http://www.digitalfaq.com/forum/vide...html#post61393) has 15 camera shots which I broke down into 10 segments. Because each segment had different filter needs, I processed the segments separately. To do that I used 2 scripts for each segment: Script "A" was mostly experimentation or trial and error, followed by Script "B" that tweaked the results and applied VirtualDub filters if needed. The scripts have similarities, but none is exactly like the others.
The result on my PC was 20 small scripts and 10 small video segments, which I assembled into a single clip and upsampled to 960x720. The finished video was attached earlier in Post #23 as Test2Proper_All_Scenes_720p50.mp4. An h264 lossy encode of the original lossless sample was attached as Test2Proper_encoded_capture.mp4.
There's nothing unusual about this workflow. VHS can be maddeningly variable second by second -- and a duped tape like this one is even worse. So segment-by-segment is rather routine for damaged media. If the way highly paid restoration shops do it is OK with pros and owners, it should be OK with us.
It wouldn't be practical to post the text of 20 scripts and several VirtualDub settings. So I pasted all of the scripts in their proper order into an attached text file called "Scripts.txt". Each segment is identified in the scripts with a unique number and name. Virtualdub .vcf files and NeatVideo settings also use names that match the segments -- these are attached as "VirtualDub_vcf.zip" and "NeatVideo_dnp_nfp.zip". Unfortunately there is no real NeatVideo equivalent in VirtualDub but some people have used Smart Smoother HiQuality 2.11 by Klaus Post (SmoothHiQ.vdf, which comes with its own html help file from http://www.digitalfaq.com/forum/atta...smootherhiqzip).
Thumbnails of the 15 camera shots are shown below with the segment numbers 1 thru 10 in which they appear in the final video:
One coding trick that I found years ago but which I haven't used in a while was used here. Users might notice with very noisy video that the first and last frames in a cut don't get all of the filtration that other frames get. This is common with temporal filters, which work on multiple frames to analyze data and recognize noise. The first few or last few frames get short-changed, sometimes looking untouched. The old trick used in some of the attached scripts borrows a few leading and trailing frames and adds them to the start or end of the clip, thus "priming" temporal filters for better cleaning of first and last frames. At the end of the script, the added frames are removed.
Here is a portion of the script for Segment #06 that shows how it can be done with a short clip that has 32 frames numbered 0 thru 31:
Code:
AviSource("D:\forum\faq\glazuna\F2\Test2Proper.avi")
Trim(1051,1082)
V1=last
First2frames=V1.Trim(0,1)
Last2Frames=V1.Trim(30,31)
V2=First2Frames + V1 + Last2Frames #<- Add 2 dupe frames to start and end. --#
V2
AssumeTFF()
ConvertToYV12(interlaced=true,ChromaOutPlacement="MPEG2")
ContrastMask(enhance=6.5)
ColorYUV(Off_y=-5,off_u=8,off_v=5)
Levels(12,0.95,255,16,255,dither=true,coring=false)
ConvertToYV12(interlaced=true,ChromaOutPlacement="MPEG2")
SeparateFields()
e=SelectEven().FixRipsP2()
o=SelectOdd().FixRipsP2()
Interleave(e,o)
Weave()
Trim(2,33) #<- Remove the first 2 and last 2 duplicate frames --#
return last
The effect can be shown in some images from segment Number 06.
(Below) The noisy image below is frame 0 of the segment without adding any duplicate or priming frames to the beginning of the clip:
(Above) The same frame 0 with better filtering after adding and then removing "priming' frames.
Hope this helps users who think using one filter for an entire video in one shot is the way advanced owners do it. Advanced users have a very different workflow. Sometimes it's more difficult than shown, sometimes it's easier. It depends on the source and your expectations.
How to keep the audio from "blanking out" on replaced frames:
The secret is saving the audio as a separate clip and re-sync at the end of the script. Example:
Thank you so much for your help sanlyn, I also found out the reason I recieved the colorspace error with FixRips. It was due to mismatch in versions of DePan.dll and DePanEstimate.dll.
Since this was the most problematic part of the tape, I will continue using your technique for that music video, which is 4 minutes long. For others I will try to minimize the effort as it is after all around 13 of them and it is literally impossible for me to invest so much time especially as other parts aren't that bad to need special treatments. Color correction will be done in AE as it is much faster to cut scenes and adjust saturations, ligthness and contrast with instant visual checks.
I will let you know If I stumble on another difficulty, but I think I have it mostly covered now. The first music video will premiere sunday 7PM CEST, so check it out then
It was due to mismatch in versions of DePan.dll and DePanEstimate.dll.
I had trouble with those two myself. I'm still using older versions because new ones won't work in many of my old scripts. To use the new versions in new scripts, I have to explicitly load them from a different plugins folder using the LoadPlugin() function.
Quote:
Originally Posted by glazuna
Color correction will be done in AE as it is much faster to cut scenes and adjust saturations, ligthness and contrast with instant visual checks.
I use AE for color work often myself. But rememebr that it works in RGB. Your capture has illegal values that clip darks and brights. Once you go into RGB you can no longer retrieve detail from clipped areas.
Quote:
Originally Posted by glazuna
The first music video will premiere sunday 7PM CEST, so check it out then
https://www.youtube.com/watch?v=FiHv2ZJDxuw
Here is the first video, I think apart from some scenes having a bit of too much blue, it turned out really well. I tried to compensate the blue color a bit, but it would show up as white spots instantly when trying to desaturize and lighten it.
I had to compensate for cyan a lot as it was all purple-ish on the same scenes where there was still some blue left (especially where guys stand in formation)
https://www.youtube.com/watch?v=FiHv2ZJDxuw
I had to compensate for cyan a lot as it was all purple-ish on the same scenes where there was still some blue left (especially where guys stand in formation)
Sorry to see you had such a difficult time with AfterEffects and NeatVideo. Bad capture levels don't help the case, and from your description above it appears you need some help in basic color theory. Crushed darks and clipped brights are evident in all but a few seconds, which effects both dynamic range and the ability to control many color elements so that some scenes look underexposed and others have over saturated color that blooms into hot spots without detail. I do wish you had the time to cut the processing into individual segments. The video is over filtered and some scenes look eviscerated, which is why most forums don't recommend NeatVideo. Would like to help with the details, but there's not enough video left on YouTube to work with.
The following users thank sanlyn for this useful post:
glazuna (05-22-2019)
Sorry to see you had such a difficult time with AfterEffects and NeatVideo. Bad capture levels don't help the case, and from your description above it appears you need some help in basic color theory. Crushed darks and clipped brights are evident in all but a few seconds, which effects both dynamic range and the ability to control many color elements so that some scenes look underexposed and others have over saturated color that blooms into hot spots without detail. I do wish you had the time to cut the processing into individual segments. The video is over filtered and some scenes look eviscerated, which is why most forums don't recommend NeatVideo. Would like to help with the details, but there's not enough video left on YouTube to work with.
I could technically try to re-capture it with better capture settings for colors, but I've always used the virtual dub's defaults. However it is the tape that has weird colors, not due to my capture card since some scenes look completely natural, while others have strong colors, so I'm sure that altering the settings would only ruin the scenes that were normal for now.
That's not the way complex color correction works. With your composite video it can't be done during capture anyway. Colors and varying noise conditions have to be corrected segment by segment in post processing. It does make me feel that I wasted time with detailed explanations of how each segment of your sample video was corrected. There can't be individual segment processing during capture, and you can't dump an entire video of widely varying segments into AfterEffects and denoise with powerful NeatVideo settings that eviscerate the entire movie. What you should correct during capture is the illegal levels that cause clipping, which can't be corrected after capture. It's a waste of time to try to correct bad video levels later, which also restricts many color corrections. Adjusting for signal levels during capture can be done once for the entire video, and individual segments can be tweaked later. Levels adjustment is described and illustrated in the VirtualDubcapture settings guide, and the same material has appeared in many other posts. No one wants to take on a project with bad levels.
It does make me feel that I wasted time with detailed explanations of how each segment of your sample video was corrected. There can't be individual segment processing during capture, and you can't dump an entire video of widely varying segments into AfterEffects and denoise with powerful NeatVideo settings that eviscerate the entire movie.
sanlyn, I very much appreciate your detailed explanations not only in this thread but also throughout the site. Many thanks! Your time is not wasted.
sanlyn, I very much appreciate your detailed explanations not only in this thread but also throughout the site. Many thanks! Your time is not wasted.
Thank you. I can appreciate that sometimes people find it difficult to understand some of those detailed descriptions. Again, people don't always have time to follow up on them. I hope my explanations were not confusing for the owner. If so, I'm open to more questions as well as suggestions from others. I'm sure I'm not the only reader who comes up with ways to process something, and I can't say I'm totally satisfied with my results.
Another video, this time scene by scene was adjusted in avisynth.
Firstly with yuv's autowhite, then the levels of saturation,contrast and brightness were tweaked to get more detail in shadows but not to overbrighten the surrounding areas.
I think it turned out quite nicely as before it was all greenish and yellow, it is much more natural looking.
