Go Back    Forum > Digital Video > Video Project Help > Restore, Filter, Improve Quality

Reply
 
LinkBack Thread Tools
  #1  
05-11-2019, 06:53 AM
glazuna glazuna is offline
Free Member
 
Join Date: Mar 2017
Location: Slovenia
Posts: 37
Thanked 1 Time in 1 Post
Hello dear friends,

I have stumbled on a very rare tape, which causes me trouble..
Tested it in three different VCRs, (2 x HR 7600S and 1x RTV 950).
All of them have the same issue, so it is tape related.

I did try to clean the tape itself with a cloth for glasses wrapped on a stick and lightly moist with alcohol, rewound it a few times, but that did not solve it either.

My setup consists of:

- AIW 7500 PCI capture card
- The above mentioned VCRs
- DataVideo TBC - 1000

So I guess the only solution is software correction with avisynth.

Here is the untouched sample of the recorded video. The transparent lines seem to be present all the time, but are not visible depending on the scene, some scenes (specificly where the guy stands in front of the grassy field and blue sky, makes it stand out) Some scenes don't have the lines visible at all or are overpowered by the background itself. (this example is the most extreme case).

https://youtu.be/KqPau5UPr8I

I've tried:
- DeScratch which seems to be the most effective against it, but I have to set mindif=1, otherwise it does not have an effect. However that makes every distinctive line in the video pulse like crazy and it just ruins the experience.

DeVCR isn't that effective and it causes black artifacts appearing in the video.

DePan also seems to help a tiny bit, but since these lines aren't appearing for just a few frames, it's use is also very limited.

Does anyone have a solid solution to this problem? I am looking for something, that would at least make the errors look more appealing and not distracting the viewer as much.

This is what I've tried so far, but it does not really deal with it very good, just cripples the lines a bit..
The biggest problem I think is detecting the lines, not the removal itself..

Code:
 

function DetectVCRLines(clip c,int threshold)
{
  spacial_data = GeneralConvolution(ConvertToRGB(c),0,"0 -1 0 0 2 0 0 -1 0")
  bar_data = ConvertToRGB(BilinearResize(spacial_data,16,c.height))
  st_data = Overlay(bar_data,Trim(bar_data,1,0),mode = "subtract")
  st_data2 = Greyscale(Levels(st_data,threshold,10.0,threshold+1,0,255,coring = false))
  st_data3 = Greyscale(Levels(st_data2,127,10.0,128,0,255,coring = false))
  st_data4 = Overlay(st_data3,st_data3,y = -1, mode = "add")
  return PointResize(st_data4,c.width,c.height)
}

function deVCR(clip c,int threshold)
{
  mybars = DetectVCRLines(c,threshold)
  return Overlay(c,Trim(c,1,0), mask = mybars,greymask = true)  
}

 
 AviSource("E:\Test.avi")
 Crop(4,0, -24, -12)
 ConvertToYV12()
 #DeVCR(30)
 
SeparateFields() # for analog interlaced source
TurnLeft()
DeScratch(mindif=1,modeV=0,modeU=3,mindifUV=1)
TurnRight()
Weave() # restore fields
DeSpot(p1=30, p2=12, mthres=20, dilate=2, fitluma=true, blur=2, seg=2,interlaced=true)
#
# Assume Bottom Field First
AssumeTFF()
# ConvertToYV12(interlaced=true)
v=ConvertToYV12(interlaced=true)
#
# Separate Fields from Frame
SeparateFields()
#
# Use to show before/after comparisons in VirtualDub
i = last
#
# Makes 3 copies of each frame for motion compensation
d = DePanEstimate(v,range=1, trust=3, log="depan.log")
DePanInterleave(v,data=d, prev=1, next=1, matchfields=false)
#
# Despot - remove noise
DeSpot(p1=24, p2=12, pwidth=20, pheight=4, mthres=20, motpn=true, dilate=1, seg=2)
#
# Recover original frame
SelectEvery(3, 1)
#
# Place additional filters here after SelectEvery() function
# convolution3D Low Quality
odd=SelectOdd.Convolution3D (0, 6, 10, 6, 8, 2.8, 0)
evn=SelectEven.Convolution3D (0, 6, 10, 6, 8, 2.8, 0)
#
#
# convert to yuy2 for cinema craft encoder
converttoyuy2()
#
# Interleave for Convolution3D
Interleave(evn,odd)
## Separate Fields from Frame
SeparateFields()
# Combine both fields back into a frame
weave()
#
# Use to show before/after comparisons in VirtualDub

QTGMC(Preset="Fast",EdiThreads=0,EZDenoise=1.0, SourceMatch=1, Sharpness=1, TR2=3, Lossless=2)



#video2=Sharpen(video2,0.2)
nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=960, fheight=720)
Reply With Quote
Someday, 12:01 PM
admin's Avatar
Ads / Sponsors
 
Join Date: ∞
Posts: 42
Thanks: ∞
Thanked 42 Times in 42 Posts
  #2  
05-11-2019, 09:58 AM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,307 Times in 982 Posts
Sorry to see you're having difficulty. The disturbance that you see isn't caused by dirty tape. The tape is physically damaged. It might be possible to make a few improvements, but the outlook isn't good. There don't appear to be enough good frames from which to interpolate new ones.In any case, your sample can't be used.

Quote:
Originally Posted by glazuna View Post

Here is the untouched sample of the recorded video.

https://youtu.be/KqPau5UPr8I
Your sample is not unaltered. It as been deinterlaced, alternate frames were discarded, the colorspace is changed, it has been resized, and it has been re-encoded with a bitrate that is too low for further processing. We cannot work with YouTube re-processed videos. Furthermore, when you posted "ADP" processing, so the video has been watermarked by YouTube.

Please post directly to the forum. When your sample disappears from YouTube, your question will be useless. If you need to post a long video, edit it into short segments and post it in parts. If you cut edits in VirtualDub, click "Video"...-> "direct stream processing' before saving the file, so that the colorspace and compression will remain intact. http://www.digitalfaq.com/forum/news...ly-upload.html
Reply With Quote
The following users thank sanlyn for this useful post: Mikey32 (05-14-2019)
  #3  
05-11-2019, 10:48 AM
glazuna glazuna is offline
Free Member
 
Join Date: Mar 2017
Location: Slovenia
Posts: 37
Thanked 1 Time in 1 Post
Quote:
Originally Posted by sanlyn View Post
Please post directly to the forum. When your sample disappears from YouTube, your question will be useless. If you need to post a long video, edit it into short segments and post it in parts. If you cut edits in VirtualDub, click "Video"...-> "direct stream processing' before saving the file, so that the colorspace and compression will remain intact. http://www.digitalfaq.com/forum/news...ly-upload.html
Thank you for a quick reply!
The previously sent file was not altered on my side (only used avisynth to trim the video, but did not due anything to it), it was by youtube however.

I am working with lossless AVI with Huffyuv codec so the size unfortunately is a bit bigger (1.5GB) and I kept it that way so it is easier for you to work with.

Rather than creating hundreds of rar files, I decided to upload it to my GoogleDrive and the before and after shots will be posted here with pictures if we manage to beautify these artifacts.

If there is another option to upload it without the loss of quality, let me know.

The original with the worst artifacts:
https://drive.google.com/open?id=1sB...EdIquHbXF5B_8V

And another unedited example of a good image on which lines aren't visible, especially after de-interlacing and de-noising.
https://drive.google.com/open?id=1Iz...luhyrdDTylYwg8

Last edited by glazuna; 05-11-2019 at 11:40 AM.
Reply With Quote
  #4  
05-11-2019, 05:01 PM
themaster1 themaster1 is offline
Free Member
 
Join Date: Feb 2011
Location: France
Posts: 488
Thanked 96 Times in 80 Posts
I see so you're in a quest for the magic scipt, i think i got most of that:
https://drive.google.com/drive/folde...ZvW8ul0BPLUlLs


Attached Files
File Type: avs Test2-MagicScript.avs (9.1 KB, 61 downloads)
Reply With Quote
The following users thank themaster1 for this useful post: glazuna (05-11-2019)
  #5  
05-11-2019, 05:41 PM
glazuna glazuna is offline
Free Member
 
Join Date: Mar 2017
Location: Slovenia
Posts: 37
Thanked 1 Time in 1 Post
Quote:
Originally Posted by themaster1 View Post
I see so you're in a quest for the magic scipt, i think i got most of that:
https://drive.google.com/drive/folde...ZvW8ul0BPLUlLs
That is exactly the result I was hoping for, much better visuals, thank you so much!


I plan to upscale it and deinterlace it by appending the following commands at the end of yours:


Code:
QTGMC(Preset="Very Slow",EdiThreads=0, SourceMatch=1, Sharpness=1, TR2=3, Lossless=2)
nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=960, fheight=720)
Reason being that I wish to have my videos in 50fps and uncompressed by youtube. And the only way to avoid it is to upload it in HD resolution.

However I did notice that you removed a lot of grain, thus making the video look a bit soft when I add the following commands. Which values of RemoveGrain should I reduce to not disturb the so perfect code?
Reply With Quote
  #6  
05-11-2019, 05:43 PM
hodgey hodgey is offline
Free Member
 
Join Date: Dec 2017
Location: Norway
Posts: 1,676
Thanked 445 Times in 382 Posts
Quote:
The tape is physically damaged.
To expand, the lines you see are the result of the VCR compensating for tape damage by "copying" the video the previous line, also known as drop-out compensation. The JVC VCRs do a pretty good job of this, on a less capable VCR you may see some of it as noise lines instead. It will inevitably start getting noticeable though when there are lot of video missing like on this tape. Though, for whatever reason, at least from what I've seen, the TBC in the VCR doesn't fully (or at all?) correct the "copied" lines, so if there is a large dropout spanning many lines I've found using a panasonic ES10 or similar as the line-tbc can be helpful as it does straighten things up a bit more, and avoid some of the vertical jumping.

EDIT:
Also note with that script, you may want to tweak the video levels before the converttorgb call, as if there are values outside 16-235 they will be clipped. Granted, I've had issues with it clipping in the color channels even with legal levels, don't know if anyone else have encountered that.
Reply With Quote
  #7  
05-11-2019, 11:58 PM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,307 Times in 982 Posts
Quote:
Originally Posted by hodgey View Post
EDIT:
Also note with that script, you may want to tweak the video levels before the converttorgb call, as if there are values outside 16-235 they will be clipped. Granted, I've had issues with it clipping in the color channels even with legal levels, don't know if anyone else have encountered that.
The videos are already clipped, especially in the blacks, because they were converted to RGB when edited. Thanks to that conversion, clipped detail can't be recovered. We would need samples in the original YUY2 without being converted to RGB. Glazuna was already told how to save the samples properly.
Reply With Quote
The following users thank sanlyn for this useful post: Mikey32 (05-14-2019)
  #8  
05-12-2019, 12:25 AM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,307 Times in 982 Posts
Thank you for re-posting. Let me remind you again: when your offline post disappears your question will be useless.

Besides being much longer than needed, your video is large because you didn't save the sample as instructed. It has been converted to uncompressed RGB24. Because the capture has low black levels to begin with, shadow detail was crushed and therefore destroyed by converting to RGB. The destruction can't be repaired. The only thing in the dark shadow areas is plain black noise, which makes the video more difficult to flter. As uncompressed RGB your video was 1.6 GB in size. When losslessly compressed to YUY2 with Lagarith its size is only 498MB. Unfortunately the recompression can't retrieve the detail that was lost by converting to RGB with incorrect black levels. You should learn how to manage colorspaces, how to use VirtualDub, and how to control input levels during capture. This has all been explained in several guides and many posts, so there's no reason to repeat it here.

I have taken the liberty of posting some short edits from your Test2.Avi, losslessly recompressed as YUY2 using Lagarith. You can get Lagarith at https://lags.leetcode.net/codec.html, although VLC Player can decode Lagarith if it's not installed on your system. I suggest that you get Lagarith for lossless intermediate work files.

I'll continue with more detail in the next post.


Attached Files
File Type: avi Test2L_editA.avi (81.51 MB, 30 downloads)
File Type: avi Test2L_EditB.avi (84.96 MB, 13 downloads)
File Type: avi Test2L_EditC.avi (72.60 MB, 12 downloads)
Reply With Quote
The following users thank sanlyn for this useful post: Mikey32 (05-14-2019)
  #9  
05-12-2019, 12:48 AM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,307 Times in 982 Posts
Quote:
Originally Posted by glazuna View Post
That is exactly the result I was hoping for, much better visuals
Really? You want your video to look that dark and noisy?

In addition to the distortion you refer to, several areas of the video have been damaged beyond repair because of the colorspace issue. Shadows are overly dense blocks of pure black. It looks quite grim and artificial. Look at the singer and you'll see what I mean: his eyes and mouth look like black holes most of the time. In processing, I used a contrast mask filter to brighten shadows, but very little detail could be recovered.

In the image below, the top white band of the histogram shows crushed blacks. Shadows are dense, without detail. It's possible that the original video looks like this, but after RGB conversion the blacks can't be repaired.


53 seconds of video is far more than is needed for a sample. 8 to 10 seconds would suffice. I've taken the liberty of posting several seconds cut from your RGB sample Test2.avi and recompressed it to YUY2 with the Lagarith lossless codec (in the previous post, for the benefit of our readers). you can download the free Lagarith installer at https://lags.leetcode.net/codec.html. The TestGood.avi has also been damaged by converting illegal black levels to uncompressed RGB. Hopefully the original doesn't look as badly damaged and could possibly be repaired in its original YUY2 color.

It's not possible to clean all of the distortion. The principle denoiser I used is a filter called FixRipsP2, which lordsmurf adapted from an earlier median filter from doom9. It often works well, but it makes mistakes with frantic motion and camera jitter, causing more distortion. In fast-moving scenes I used RemoveDirtMC at very high settings, and used FioxRipsP2 on slower-moving shots. Other noise was cleaned with QTGMC. Because there is a lot of noise and the filters used are very slow, I ran the FixRipsP2 filter in a script called PartA and saved that output with Lagarith in YV12. Then I cleaned PartA with QTGMC in a PartB script. Because different camera shots required different filters, I had to break the PartA script into 4 sections, then joined them on output. I applied ColorCamcorderDenoise to the output of PartB in VirtualDub.

In two segments of the TestGood.avi I was able to create interpolated replacement frames using a plugin called ReplaceFramesMC2.avs. Fortunately there wasn't much movement in the bad frames, so the interpolation of severasl new frames worked well and without distortion. Things seldom go that well if there is too much motion.

original frames from TestGood.avi:


Interpolated frames using ReplaceFramesMC2:

You might want to take a closer look at how the previously posted script handled those frames in TestGood.avi.

Resizing always has a cost. Upsampling low resolution video makes absolutely no improvement, especially with damaged video. It's a total waste of time. HD video comes from high resolution sources, not from low-resolution sources blown up into big blurry frames. It's like shooting yourself in the foot: you resize once to 720p in software, then your display resizes it again to 1080p, creating a double layer of interpolation errors. If you must have 4:3 square-pixel progressive output, resize to a more standard 640x480.

I could not get your script to run due to colorspace errors, so the script you posted isn't exactly like the one you used. There are also logic errors -- for instance, when you change colorspace you must tell Avisynth whether or not the video is interlaced. Also, I had to recompress your posted avi's to YUY2 using Lagarith to conserve space.

Because the filters I used are so slooooow when all of them are run in the same script, I divided the processing of both avi's into PartA and PartB. In particular, FixRipsP2 is a median averaging filter and is exceptionally slow. PartA is designed to smooth the distorted scanlines as much as is practical. PartB is for general cleanup with QTGMC and a few other filters.

For Test2.avi I had to cut the video into 4 parts for different filtering of each part. Segments without much motion used FixRipsP2 for the ripples and tearing. Segments with too much motion used RemoveDirtMC. Here is the script for PARTA that I used for Test2.avi:

Code:
AviSource("D:\forum\faq\glazuna\A\Test2.avi")
ConvertToYV12(interlaced=true,ChromaOutPlacement="MPEG2")
ContrastMask(enhance=6.5)
ColorYUV(off_y=-8)
srce=last

Part1=srce.trim(0,817)
Part1
SeparateFields()
e=SelectEven().RemoveDirtMC(100,false)
o=SelectOdd().RemoveDirtMC(100,false)
Interleave(e,o)
Weave()
Part1_fix=last   #-> save this result as Part1

Part2=srce.trim(818,1033)
Part2
SeparateFields()
e=SelectEven().FixRipsP2() 
o=SelectOdd().FixRipsP2()
Interleave(e,o)
Weave()
Part2_fix=last   #-> save this result as Part2

Part3=srce.trim(1034,1143)
Part3
SeparateFields()
e=SelectEven().RemoveDirtMC(100,false)
o=SelectOdd().RemoveDirtMC(100,false)
Interleave(e,o)
Weave()
Part3_fix=last   #-> save this result as Part3

Part4=srce.Trim(1144,0)
Part4
SeparateFields()
e=SelectEven().FixRipsP2()
o=SelectOdd().FixRipsP2()
Interleave(e,o)
Weave()
Part4_fix=last   #-> save this result as Part4

# --- Join all the part5s and save them as PartA.avi.  ---
# --- In VirtualDub, save the output as Lagarith YV12. ---
PartA=Part1_Fix + Part2_Fix + Part3_Fix + Part4_Fix
Return PartA
Here is the PartB script I used for Test2.avi. The input to PartB is the saved PartA.avi:

Code:
AviSource("D:\forum\faq\glazuna\A\PartA.avi")
AssumeTFF()
QTGMC(preset="very fast",EZDenoise=4,denoiser="dfttest",ChromaMotion=true,\
   border=true,ChromaNoise=true,DenoiseMC=true,GrainRestore=0.3,TR2=3)
vInverse2()
ChromaShift(C=2)
Santiag(2,2)
MergeChroma(aWarpSharp2(Depth=20))   
GradFun2DBmod(thr=1.8)
AddGrainC(1.75,1.75)

### --- ConvertToRGB32 for the ColorCamcorderDenoise VirtualDub filter --- ###
ConvertToRGB32(interlaced=false,matrix="Rec601",ChromaInPlacement="MPEG2")
Crop(2,0,-28,-12).AddBorders(14,6,16,6)
### --- Note that this video is 50p progressive and is YV12 color. --- ###
return last
For TestGood.avi, I found that I didn't need the FixRipsP2 filter. RemoveDirtMC at high power removed most of the distorted lines in PartA. QTGMC completerd the cleanup in PartB. The script below PartA for TestGood.avi:

Code:
AviSource("D:\forum\faq\glazuna\B\TestGood.avi")
ConvertToYV12(interlaced=true,ChromaOutPlacement="MPEG2")
ContrastMask(enhance=6.5)
ColorYUV(off_y=-8)
Levels(12,0.95,255,16,255,dither=true,coring=false)
SeparateFields()
e=SelectEven().RemoveDirtMC(80,false)
o=SelectOdd().RemoveDirtMC(80,false)
Interleave(e,o)
Weave()
return last
# ###--- Save the output as Lagarith YV12 ---###
Here is the PartB script I used for for TestGood.avi:

Code:
AviSource("D:\forum\faq\glazuna\B\TestGoodA.avi")
AssumeTFF()
QTGMC(preset="very fast",EZDenoise=2,denoiser="dfttest",ChromaMotion=true,\
   border=true,ChromaNoise=true,DenoiseMC=true,GrainRestore=0.3,TR2=2)
vInverse2()

# ###--- Use ReplaceFramesMC2.avs to replace some bad frames. ---###
# ###--- The frame numbers shown are for deinterlaced frames. ---###
ReplaceFramesMC2(942,6)
ReplaceFramesMC2(1094,7)

ChromaShift(C=2)
Santiag(2,2)
MergeChroma(aWarpSharp2(Depth=20))
GradFun2DBmod(thr=1.8)
AddGrainC(1.75,1.75)
### --- ConvertToRGB32 for the ColorCamcorderDenoise VirtualDub filter --- ###
ConvertToRGB32(interlaced=false,matrix="Rec601",ChromaInPlacement="MPEG2")
Crop(2,0,-28,-12).AddBorders(14,6,16,6)
### --- Note that this video is 50p progressive and is YV12 color. --- ###
return last
For resizing progressive video to square-pixel for the internet, I used the code below to resize the clips (the input file is non-interlaced and was previously saved as Lagarith YV12)::

Code:
AviSource("Drive:\path\filename.avi")  #<- change the path and filename.
AssumeTFF()
ConvertToYUY2(interlaced=false,ChromaInPlacement="MPEG2")
SPline36Resize(640,480)
# --- Downsample chroma for MPEG or h.264 encoding.---###
# --- YUY2 will be needed for CCE encoding, which  ---###
# --- is strange because CCE encodes to YV12.      ---###
ConvertToYV12(interlaced=false,ChromaOutPlacement="MPEG2")
return last
For re-interlacing progressive video for DVD or standard-def BluRay, I used the code below (the input file is non-interlaced and was saved as Lagarith YV12):

Code:
AviSource("Drive:\path\filename.avi")  #<- change the path and filename.
AssumeTFF()
SeparateFields().SelectEvery(4,0,3).Weave()
# --- Downsample chroma for MPEG or h.264 encoding.---###
# --- YUY2 will be needed for CCE encoding, which  ---###
# --- is strange because CCE encodes to YV12.      ---###
ConvertToYV12(interlaced=true,ChromaOutPlacement="MPEG2")
return last
I've attached three sample encodes made from the above scripts:
- Test2_25i_ forDVD.mpg is interlaced and encoded for DVD.
- Test2_4x3_50p.mp4 is 50fps progressive in square-pixel 640x480.
- TestGood_4x3_50p.mp4 is 50fps progressive in square-pixel 640x480.
You're wasting your time upsampling bad video.

If you need more details, I'll be able to post more later.


Attached Images
File Type: jpg crushed darks YUV.jpg (106.3 KB, 354 downloads)
File Type: jpg TestGood_Original frames.jpg (93.5 KB, 351 downloads)
File Type: jpg TestGood_Replaced frames.jpg (84.2 KB, 347 downloads)
Attached Files
File Type: mpg Test2_25i_ forDVD.mpg (42.21 MB, 7 downloads)
File Type: mp4 Test2_4x3_50p.mp4 (30.02 MB, 8 downloads)
File Type: mp4 TestGood_4x3_50p.mp4 (22.94 MB, 8 downloads)
Reply With Quote
The following users thank sanlyn for this useful post: bar72 (08-21-2023), captainvic (05-17-2019), glazuna (05-12-2019), Mikey32 (05-14-2019)
  #10  
05-12-2019, 03:30 AM
glazuna glazuna is offline
Free Member
 
Join Date: Mar 2017
Location: Slovenia
Posts: 37
Thanked 1 Time in 1 Post
Sorry for the head aches of my lack of knowledge
I did now select the direct stream copy in virtualdub and saved it that way, it is much smaller now so I uploaded it in .rar

Also I need to upload it in the resolution of 960 x 720 because youtube compresses all videos and does not allow higher than 30fps if under 720p.


The only problem is, I have a 50 minute long recording of music videos by this singer, however the worst part is in the Test2, the rest of the video looks like TestGood more or less. I guess the only way is to check how you chose your script parts for particular scene noises and I continue that pattern for the rest of the tape and in some way devide the parts depending on the tape problems / noise.

Thanks for putting so much effort, it means a lot to me and my community, especially as this is a really rare tape.


Attached Files
File Type: rar ProperTest.part01.rar (98.00 MB, 6 downloads)
File Type: rar ProperTest.part02.rar (98.00 MB, 2 downloads)
File Type: rar ProperTest.part03.rar (98.00 MB, 2 downloads)
File Type: rar ProperTest.part04.rar (98.00 MB, 2 downloads)
File Type: rar ProperTest.part05.rar (98.00 MB, 2 downloads)
File Type: rar ProperTest.part06.rar (98.00 MB, 2 downloads)
File Type: rar ProperTest.part07.rar (98.00 MB, 2 downloads)
File Type: rar ProperTest.part08.rar (98.00 MB, 3 downloads)
File Type: rar ProperTest.part09.rar (56.02 MB, 2 downloads)

Last edited by glazuna; 05-12-2019 at 03:50 AM.
Reply With Quote
  #11  
05-12-2019, 11:09 AM
glazuna glazuna is offline
Free Member
 
Join Date: Mar 2017
Location: Slovenia
Posts: 37
Thanked 1 Time in 1 Post
I'm also having difficulties with FixRipsP2, constantly saying "DePanInterleave: input must be planar YUV or YUY2".
As soon as a I remove the part with FixRipsP2, the video appears again.

I installed the required plugins and also all the missing functions. This is what I found for FoxRipsP2()


Code:
function FixRipsP2(clip a)
{
a
clense(reduceflicker=false).merge(last,0.5).clense(reduceflicker=false)
mot=removegrain(11,0).removegrain(20,0).DepanEstimate(range=2)
take2=a.depaninterleave(mot,prev=2,next=2,subpixel=2)
clean1=take2.TMedian2().selectevery(5,2)

sup1 = clean1.minblur(1).removegrain(11,0).removegrain(11,0)
 \           .mt_lutxy(clean1,"x 1 + y < x 2 + x 1 - y > x 2 - y ? ?",U=2,V=2)
 \           .msuper(pel=2,sharp=0)
sup2 = a.msuper(pel=2,levels=1,sharp=2)

bv22=sup1.manalyse(isb=true, truemotion=false,global=true,delta=2,blksize=16,overlap=8,search=5,searchparam=4,DCT=5)
bv21=sup1.manalyse(isb=true, truemotion=false,global=true,delta=1,blksize=16,overlap=8,search=5,searchparam=4,DCT=5)
fv21=sup1.manalyse(isb=false,truemotion=false,global=true,delta=1,blksize=16,overlap=8,search=5,searchparam=4,DCT=5)
fv22=sup1.manalyse(isb=false,truemotion=false,global=true,delta=2,blksize=16,overlap=8,search=5,searchparam=4,DCT=5)

interleave(a.mcompensate(sup2,fv22),a.mcompensate(sup2,fv21),a,a.mcompensate(sup2,bv21),a.mcompensate(sup2,bv22))
TMedian2().selectevery(5,2)

sup3 = last.msuper(pel=2,sharp=2)
bv33=sup3.manalyse(isb=true, truemotion=false,global=true,delta=3,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
bv32=sup3.manalyse(isb=true, truemotion=false,global=true,delta=2,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
bv31=sup3.manalyse(isb=true, truemotion=false,global=true,delta=1,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
fv31=sup3.manalyse(isb=false,truemotion=false,global=true,delta=1,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
fv32=sup3.manalyse(isb=false,truemotion=false,global=true,delta=2,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
fv33=sup3.manalyse(isb=false,truemotion=false,global=true,delta=3,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)

last.mdegrain3(sup3,bv31,fv31,bv32,fv32,bv33,fv33,thSAD=499)

interleave()
return last
}


function MinBlur(clip clp, int r, int "uv")
{
uv   = default(uv,3)
uv2  = (uv==2) ? 1 : uv
rg4  = (uv==3) ? 4 : -1
rg11 = (uv==3) ? 11 : -1
rg20 = (uv==3) ? 20 : -1
medf = (uv==3) ? 1 : -200

RG11D = (r==0) ? mt_makediff(clp,clp.sbr(),U=uv2,V=uv2)
 \    : (r==1) ? mt_makediff(clp,clp.removegrain(11,rg11),U=uv2,V=uv2)
 \    : (r==2) ? mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20),U=uv2,V=uv2)
 \    :          mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20).removegrain(20,rg20),U=uv2,V=uv2)
RG4D  = (r<=1) ? mt_makediff(clp,clp.removegrain(4,rg4),U=uv2,V=uv2)
 \    : (r==2) ? mt_makediff(clp,clp.medianblur(2,2*medf,2*medf),U=uv2,V=uv2)
 \    :          mt_makediff(clp,clp.medianblur(3,3*medf,3*medf),U=uv2,V=uv2)
DD    = mt_lutxy(RG11D,RG4D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
clp.mt_makediff(DD,U=uv,V=uv)
return(last)
}

# median of 5 clips from helpers.avs by G-force
function Median2(clip "input_1", clip "input_2", clip "input_3", clip "input_4", clip "input_5", string "chroma")
{
chroma = default(chroma,"process") #default is "process". Alternates: "copy first" or "copy second"
#MEDIAN(i1,i3,i5)
Interleave(input_1,input_3,input_5)
chroma == "process" ? Clense(reduceflicker=false) : Clense(reduceflicker=false,grey=true)
m1 = selectevery(3,1)
#MAX(MIN(i1,i3,i5),i2)
m2  = input_1.MT_Logic(input_3,"min",chroma=chroma).MT_Logic(input_5,"min",chroma=chroma).MT_Logic(input_2,"max",chroma=chroma)
#MIN(MAX(i1,i3,i5),i4)
m3  = input_1.MT_Logic(input_3,"max",chroma=chroma).MT_Logic(input_5,"max",chroma=chroma).MT_Logic(input_4,"min",chroma=chroma)
Interleave(m1,m2,m3)
chroma == "process" ? Clense(reduceflicker=false) : Clense(reduceflicker=false,grey=true)
selectevery(3,1)
chroma == "copy first" ? last.MergeChroma(input_1) : chroma == "copy second" ? last.MergeChroma(input_2) : last
Return(last)
}

function TMedian2(clip c) {
Median2(c.selectevery(1,-2), c.selectevery(1,-1), c, c.selectevery(1,1), c.selectevery(1,2) ) }
Reply With Quote
  #12  
05-13-2019, 06:41 AM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,307 Times in 982 Posts
Sorry for the delay, yesterday was very busy.

FixRips2P uses plugins that require require YV12. You are apparently working with an RGB or YUY2 version of a sample. There is a version of YUY2 that is planar like YUV, but because some plugins used here require plain YV12 it's safer to convert to YV12. When converting colorspace you must tell Avisynth whether or not the video is interlaced (also, telecine is treated as a form of interlace). The code would be
Code:
ConvertToYV12(interlaced=-true)
or
Code:
ConvertToYV12(interlaced = false)
It is required for all colorspace conversions, not just YV12.

I'll be able to post more later, with some images.
Reply With Quote
The following users thank sanlyn for this useful post: bar72 (08-21-2023), captainvic (05-17-2019), Mikey32 (05-14-2019)
  #13  
05-13-2019, 12:01 PM
glazuna glazuna is offline
Free Member
 
Join Date: Mar 2017
Location: Slovenia
Posts: 37
Thanked 1 Time in 1 Post
Here is the real kicker, I did change colorspace to YV12, I've tried many others, without input parameters aswell but I can't get it to work..


Code:
AviSource("D:\test.avi")
ConvertToYV12(interlaced=true,ChromaOutPlacement="MPEG2")
AssumeTFF()
QTGMC(preset="very slow",EZDenoise=2,denoiser="dfttest",ChromaMotion=true, border=true,ChromaNoise=true,DenoiseMC=true,GrainRestore=0.3,TR2=2)
vInverse2() 
# ###--- Use ReplaceFramesMC2.avs to replace some bad frames. ---### 
# ###--- The frame numbers shown are for deinterlaced frames. ---### 
ReplaceFramesMC2(1719,3) 
ReplaceFramesMC2(1815,4) 
ReplaceFramesMC2(2007,1) 


srce=last
Part2=srce.trim(818,1033)
Part2 
SeparateFields()
e=SelectEven().FixRipsP2()
o=SelectOdd().FixRipsP2()
Interleave(e,o)
Weave()
Part2_fix=last

ChromaShift(C=2) 
Santiag(2,2)
MergeChroma(aWarpSharp2(Depth=20))
GradFun2DBmod(thr=1.8) 
AddGrainC(1.75,1.75) 
Crop(2,0,-28,-12)
### --- ConvertToRGB32 for the ColorCamcorderDenoise VirtualDub filter --- ###

nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=960, fheight=720)
#LimitedSharpenFaster(ss_x=1.25,ss_y=1.25,Smode=4,strength=5000,soft=30)
 ConvertToRGB32(interlaced=false,matrix="Rec601",ChromaInPlacement="MPEG2") 
 #Crop(2,0,-28,-12).AddBorders(14,6,16,6) 
 ### --- Note that this video is 50p progressive and is YV12 color. --- ### 
 return Part2_fix
Reply With Quote
  #14  
05-13-2019, 04:59 PM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,307 Times in 982 Posts
Quote:
Originally Posted by glazuna View Post
Here is the real kicker, I did change colorspace to YV12, I've tried many others, without input parameters aswell but I can't get it to work..


Code:
AviSource("D:\test.avi")
ConvertToYV12(interlaced=true,ChromaOutPlacement="MPEG2")
AssumeTFF()
QTGMC(preset="very slow",EZDenoise=2,denoiser="dfttest",ChromaMotion=true, border=true,ChromaNoise=true,DenoiseMC=true,GrainRestore=0.3,TR2=2)
vInverse2() 
# ###--- Use ReplaceFramesMC2.avs to replace some bad frames. ---### 
# ###--- The frame numbers shown are for deinterlaced frames. ---### 
ReplaceFramesMC2(1719,3) 
ReplaceFramesMC2(1815,4) 
ReplaceFramesMC2(2007,1) 


srce=last
Part2=srce.trim(818,1033)
Part2 
SeparateFields()
e=SelectEven().FixRipsP2()
o=SelectOdd().FixRipsP2()
Interleave(e,o)
Weave()
Part2_fix=last

ChromaShift(C=2) 
Santiag(2,2)
MergeChroma(aWarpSharp2(Depth=20))
GradFun2DBmod(thr=1.8) 
AddGrainC(1.75,1.75) 
Crop(2,0,-28,-12)
### --- ConvertToRGB32 for the ColorCamcorderDenoise VirtualDub filter --- ###

nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=960, fheight=720)
#LimitedSharpenFaster(ss_x=1.25,ss_y=1.25,Smode=4,strength=5000,soft=30)
 ConvertToRGB32(interlaced=false,matrix="Rec601",ChromaInPlacement="MPEG2") 
 #Crop(2,0,-28,-12).AddBorders(14,6,16,6) 
 ### --- Note that this video is 50p progressive and is YV12 color. --- ### 
 return Part2_fix
The error that I get using the script above is not a YV12 error. The error is:
Quote:
The message states that you have a frame size error. For NNEDI3 an image has to be mod4 in all dimensions. For some other filters, it must be mod 8. Some filters even require mod 16. "Mod 4" means evenly divided by 4 with no remainder.

If you are removing 30 side border pixels and intend to later upscale to 960 width, you will have distorted the original aspect ratio. Your final image will be stretched (distorted) by being 30 pixels too wide.

Still working on a brief demo to give you more details. Will return later (soon, I hope, but pausing for dinner).


Attached Images
File Type: png Frame size error.png (9.4 KB, 329 downloads)
Reply With Quote
  #15  
05-13-2019, 08:35 PM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,307 Times in 982 Posts
Thanks for posting edits from the original YUY2 captures. This will the first of two consecutive posts.

Without an unwanted RGB covcersion, some of the filtering work can have visibly cleaner results. I say "visibly" because, as I suspected at first, the tape you are using appears to be a tape dubbed from another damaged original. If the tape you are using does not appear to have wrinkles or other physical damage, then the original source was damaged. There is also over-saturation, "soiled" colors, varying color casts and levels changes, and exaggerated high contrast that looks like a tape-to-tape copy. You can also see definite changes in color balance from scene to scene, which is common with VHS.

In looking over the details of glitches and defects that are discussed below, remember that it's common with analog sources to cut video into individual segments for special treatment. Segment processing is done all the time. But you don't have to go crazy with this method. Many defects are marginal or too slight for special treatment. Often you can configure a color or noise filter that will work well on many other that have the same problems. After processing, segments are saved as lossless work files and can be rejoined in Avisynth or in VirtualDub. Be sure that your segments are saved using the same colorspace and frame size so that they can be joined again. This is why I suggest Lagarith for intermediate working files, because it works with YV12, YUY2, and RGB, and it makes slightly smaller files than huffYUV or default UT Video.

Another clue about dubbed or reworked tape is the absence of fine detail. Notice in most of the images that edges look thick and overly dark, but there is not much defined detailed or subtle textures between edges. This is most noticeable in facial contours or in grainy objects like brick walls, helmets, uniforms, trees, and so forth. Of course, the newsreel shots are not expected to look pristine.

The opening frames in TestGoodProper.avi show two problems with input levels. The first problem in shadow detail is apparently due to the high contrast of the original source. Contrast effects are made a little worse by the capture device, which is clipping darks below y=16 so that no detail below that point can be recovered. Many capture devices do this, including the ATI 600 USB, The Hauppauge USB Live-2, and some of their cheaper clones. Because dark clipping occurs early, before the signal reaches the capture software, it can be prevented with an external proc amp. Good proc amps for analog use are very expensive and hard to find in good condition. But you can still make some improvements and recover some dark details for a more natural look by using a contrast mask filter designed to recover lost detail from clipped blacks and clipped brights.

The filter I used was ContrastMask.avs. If you don't have it, or if you lack other plugins, let us know -- some of them can be very difficult to find.

The image below is a Virtualdub capture of the first frame in TestGoodProper.avi. This image is unfiltered but is slightly resized, with the original black borders removed to prevent them from affecting the value measurements in the histogram.

In this image notice the YUV histogram on the right-hand side. The white horizontal band at the top of the graph shows luminance values across the top (the "y" channel in "YUV"), with darks at the left and brights at the right. Note that the horizontal bands include a shaded border along each side. Values that lie inside the unshaded portion are values that lie within the safe video range of y=16-235. Values that lie inside the shaded portions indicates values that are outside the legal video range; values darker than y=16 are shown the left border, values brighter than y=235 are shown at the right border. When YUV is displayed in RGB, the safe video range Y=16-235 is expanded into RGB=0-255. If the video already has levels that are darker than y=16 or brighter than y=235, there is no room for that unsafe data in RGB 0-255. those out of bounds values are clipped in RGB either by deleting them or by converting them to solid colors with no detail. When data values are clipped, the original details can't be recovered.

In the horizontal white band of the YUV histogram below, notice the thin white "spike" at the left shaded border. The white spike with no data to its left indicates clipping at y=16. Below y=16, there is no data that can be recovered. The shadows behind the singer and inn the tree are an almost solid greenish black with no detail. Also notice the green color cast: the bricks in the wall are too green, and so are the man's boots and uniform and his shadow on the bricks.




(Above) TestGood Frame 0 corrected with levels adjustmemnts and a VirtualDub curves filter. The Avisynth code for the corrections was:
Code:
ContrastMask(enhance=7.5)
ColorYUV(off_y=-3,off_u=5,off_v=3)
Levels(12,0.95,255,16,230,dither=true,coring=false)
Code for the VirtualDub filters isd attached as a .vcf file, which is a simnple text file that saveS virtualDub settings. To create a .vcf file click "File.." -> "Save processing savings". To load filter settings that are saved with a .vcf file, downlaod the .vcf and save it in the same folder with your project files (don't save itm in VirtualDub plugins. It should be used there). Click "File..." -> "Load processing settings...", then locate the .vcf file, select6 it and click "Open". You must have the VirtuaLDub filters in your plugins folder or the .vcf won't work. The VDub filter used was gradation curves v1.45 (curves.vdf). The .vcf for the VDub filters used here are attached as "TestGood Frame 0 original.vcf".

(Below) The original frame 332 with no levels or color corrections. In the white band of the histogram below, white values overflow into the unsafe banded area at the right-hand border (yellow arrow in the corner). The tiny right-hand "spike" at the right edge indicates additional bright clipping during capture. Those Y values are be brighter than y=235, so brights will be clipped in RGB. Notice how this makes bright highlights on clothing, helmets, etc. look like "hot spots". Over-saturation makes yellow and green colors "bloom" unnaturally. There is also a y=16 clipping "spike" at the left side. These spikes are indications that levels were not controlled during capture. The frame is so saturated that it looks almost liker a cartoon.




(Above) C Frame 332 after corrections. Slightly different Avisynth corrections were used for shadows, color balance, overall levels, and saturation (reduced by about 20%):
Code:
ContrastMask(enhance=6.5)
ColorYUV(off_y=3,off_u=12,off_v=3)
Levels(12,0.90,255,16,230,dither=true,coring=false)
Tweak(sat=0.8,dither=true,coring=false)
The VirtualDub filters used were curves 1.45 and Hue/Saturation/intensity (hue.vdf). The .vcf is attached as "Test GoodFrame 332 corrections.vcf".

(Below)Here is another sequence of badly discolored and oversaturated frames, with the original frame 445 (top image) and the corrected frame 445 (lower image). The original has crushed blacks and clipped brights like many other shots. But color correction was more difficult and is mainly a matter of lowering saturation and adding a little blue. Otherwise it looks as if this sequence of newsfilm has been manipulated so many times that natural color restoration is probably impossible -- but the original is definitely oversaturated and looks like anime.


The Avisynth code I used for basic corrections:
Code:
ContrastMask(enhance=7.5)
ColorYUV(gain_y=15,off_u=15,off_v=3)
Levels(12,1.0,255,16,230,dither=true,coring=false)
Tweak(sat=0.7,dither=true,coring=false)
The only VirtualDub filter was gradation curves 1.45.m The settings are attached as "TestGood Frame 445 corrections.vcf".

(Below) Frame 1162 of Test2Proper.avi before corrections (upper image(, and Frame 1162 after corrections (lower image). he major difference here with Avisynth was setting the ContrastMap filtyer's "enhance" paramter to its default value of 10.0 and raising image gamma slightly. As soon as these take effect you see a major problem, which is that the image is badly oversaturated with dark blue. Along with clipping below y=16, this limits how much shadow detail can be retrieved. I added VirtualDub filters to enliven the image as much as I could. and to take some of the heavy blue out of the dark areas.

The Avisynth code for fixing this image was:
Code:
ContrastMask(enhance=10.0)
Levels(16,1.2,255,16,255,dither=true,coring=false)
The VirtualDub filters used were gradation curves 1.45 and the built-in VirtualDub "Levels" filter that comes with VirtualDub.. The settings are attached as "Test2 Frame 1162 corrections.vcf".

We can't cover every camera shot in the entire movie. Some scenes would require very minor correction, some not at all except for black levels. But I think you get the general idea by looking at these images and by viewing VirtualDub filters after loading them with the .vcf files to see how the filters were configured.


Attached Images
File Type: jpg A TestGood Frame 0 original YUV histogram.jpg (116.1 KB, 325 downloads)
File Type: jpg B TestGood Frame 0 after corrections.jpg (124.7 KB, 321 downloads)
File Type: jpg C TesttGood Frame 332 original YUV histogram.jpg (119.1 KB, 327 downloads)
File Type: jpg D TesttGood Frame 332 after corrections.jpg (118.1 KB, 327 downloads)
File Type: jpg E TestGood Frame 445 berfore and after.jpg (111.9 KB, 322 downloads)
File Type: jpg F Test2 Frame 1162 berfore and after.jpg (160.5 KB, 323 downloads)
Attached Files
File Type: vcf TestGood Frame 0 corrections.vcf (3.4 KB, 3 downloads)
File Type: vcf TestGood Frame 332 corrections.vcf (3.5 KB, 3 downloads)
File Type: vcf TestGood Frame 445 corrections.vcf (3.4 KB, 4 downloads)
File Type: vcf Test2 Frame 1162 corrections.vcf (3.5 KB, 2 downloads)
Reply With Quote
The following users thank sanlyn for this useful post: bar72 (08-21-2023), captainvic (05-17-2019), Delta (06-19-2021), Mikey32 (05-14-2019)
  #16  
05-13-2019, 08:59 PM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,307 Times in 982 Posts
Noise reduction and cleaning of horizontal rips gets complicated due to motion and camera movement. When motion is too erratic or fitful, median filters like FixRipsP2 create lots of distortion. A filter like RemoveDirtMC can smooth many types of horizontal rips and clean many spots, but it's not as effective as a median filter in addressing rips -- on the other hand, RemoveDirtMC is less destructive. So I decided to apply FixRipsP2 in scenes where distortion wouldn't be so great, and apply RemoveDirtMC where a few remaining rips wouldn't be too difficult to live with.

The images below show the kind of ugly distortion that occurs when FixRipsP2 is used with too much motion involved:

Distorted hands with FixRips and other averaging median filters


Hands are not distorted with RemoveDirtMC


Distorted camera motion with FixRips and other averaging median filters:


Motion not distorted with RemoveDirtMC


Before deciding on which camera shots to filter with FixRipsP2 and which shots tom fi9lter with RemoveDirtMC, I browsed the downloaded samples and wrote down the ranges of frame numbers that required eityher FixRipsP2 or RemoveDirtMC. You can write a very long script to process desired segments using trim() statements. But if there are a great many segments, it's probably easier to make the segments with Avisynth, save them with VirtualDub, filter each saved file as desired, then join them again in Avisynth into one long file.

The other advantage to making a workfile for each segment is that you can assign consecutive segment numbers in the filenames and you can keep text6 files of scripts and notes on each file -- this is especially useful for segments that will require different color adjustments.

While that sounds complicated, it's a fact that with troublesome VHS files such as these (and we've seen worse), this is the way most advanced users break down their workflow and reassemble segments at the end.

You don't have to save all of the intermediate working files, but I suggest that you save the scripts and any notes you want to keep as plain text files.

Something that I noticed earlier with the latest scripts you used. I see that you ran QTGMC before running the RFixripsP2 and RemoveDirtMC routines. You should run the FixRips amnd RemoveDirtMc routines first, using SeparateFields. If you run QTGMC,m uyo0u give the other routines larger images to work on, which slows them down and makes them less effective. In many cases, running QTGMC first can carry defects over to adjacent frames and makes them tougher to remove.

I also noticed that you might have been removing 30 side border pixels and then upsampling to 960x720. Removing border pixels and not replacing border parts to restore the original frame dimensions is distorting the original image aspect ratio. In the scripts that I posted earlier, border pixels were removed and then replaced with new ones as follows:

Code:
Crop(2,0,-28,-12).AddBorders(14,6,16,6)
If you do that at the end of your scripts and then upscale to 960x720, you'll preserve the original horizontal aspect ratio of the image. Otherwise, your image will look stretched horizontally.

Tomorrow I'll come up with some new scripts for your new samples.


Attached Images
File Type: jpg A1 Distortedod hands with FixRips.jpg (88.5 KB, 321 downloads)
File Type: jpg A2 Hands with RemovedirtMC.jpg (95.5 KB, 319 downloads)
File Type: jpg B1 Distorted motion with FixRiPs.jpg (33.1 KB, 317 downloads)
File Type: jpg B2 Motion with RemoveDirtMC.jpg (49.6 KB, 316 downloads)
Reply With Quote
The following users thank sanlyn for this useful post: bar72 (08-21-2023), captainvic (05-17-2019), glazuna (05-14-2019), Mikey32 (05-14-2019)
  #17  
05-14-2019, 01:33 PM
glazuna glazuna is offline
Free Member
 
Join Date: Mar 2017
Location: Slovenia
Posts: 37
Thanked 1 Time in 1 Post
Thank you for writing such a detailed post. However I still cannot get the fixRips to work. It still says "DePanInterleave: input must be planar YUV or YUY2". Even though I had put ConvertToYV12() command and haven't changed the colorspace before or after that. I've also added black borders to match the original size before cropping. Could you provide your own FixRips function?

Regarding the saturation, hue, contrast.. Would it be a big no no if I use avisynth just to interpolate a few bad frames and get rid of the lines but do the color part in Adobe's After Effects? I also use that NeatVideo plugin to polish it off and exporting it in H264 quicktime codec (mov).

RemoveDirtMC kills a lot of detail, especially at shots where singer is further away and you can no longer see his mouth moving (such scenes might not be included in the test samples), while you could before. But I guess that is just a work of segmentation on the video for with and without this function.
Reply With Quote
  #18  
05-14-2019, 02:43 PM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,307 Times in 982 Posts
Quote:
Originally Posted by glazuna View Post
Thank you for writing such a detailed post. However I still cannot get the fixRips to work. It still says "DePanInterleave: input must be planar YUV or YUY2". Even though I had put ConvertToYV12() command and haven't changed the colorspace before or after that. I've also added black borders to match the original size before cropping. Could you provide your own FixRips function?

Regarding the saturation, hue, contrast.. Would it be a big no no if I use avisynth just to interpolate a few bad frames and get rid of the lines but do the color part in Adobe's After Effects? I also use that NeatVideo plugin to polish it off and exporting it in H264 quicktime codec (mov).

RemoveDirtMC kills a lot of detail, especially at shots where singer is further away and you can no longer see his mouth moving (such scenes might not be included in the test samples), while you could before. But I guess that is just a work of segmentation on the video for with and without this function.
I'm still working this afternoon on several ideas for fixes and will post later today, but have been interrupted at home several times. Phooey.

Post a sample script with FixRipsP2, it might just be a logic or naming problem. The version of FixRips that I've been using on the forum is from lordsmurf's post of a couple of years back, which can be downloaded from http://www.digitalfaq.com/forum/atta...d-fixripsp2avs.

I've used AfterEffects/ColorFinesse for color in the past. I didn't find much use for it with this project, but you can use whatever you want. The main problem to start with color is that levels were not well controlled during the capture and crushed darks and high contrast have to be calmed first in YUY2. Levels have to be fixed in YUV before you convert to RGB color apps. You can't retrieve clipped details after going to RGB. Those are the scripts I'm working on at the moment. The images I posted earlier show why YUV levels come before color.

Will return later today.
Reply With Quote
  #19  
05-14-2019, 03:28 PM
glazuna glazuna is offline
Free Member
 
Join Date: Mar 2017
Location: Slovenia
Posts: 37
Thanked 1 Time in 1 Post
Code:
AviSource("D:\test2.avi")
ConvertToYV12(interlaced=true,ChromaOutPlacement="MPEG2")
AssumeTFF()

SeparateFields()
e=SelectEven().FixRipsP2()
o=SelectOdd().FixRipsP2()
Interleave(e,o)
Weave()

vInverse2() 
Santiag(2,2)
MergeChroma(aWarpSharp2(Depth=20))
GradFun2DBmod(thr=1.8) 
AddGrainC(1.75,1.75) 

Crop(2,0,-28,-12)
AddBorders(16,6,14,6, color=$000000)
 
QTGMC(preset="very slow",EZDenoise=2,denoiser="dfttest",ChromaMotion=true, border=true,ChromaNoise=true,DenoiseMC=true,GrainRestore=0.3,TR2=2)
nnedi3_rpow2(2, cshift="Spline36Resize", fwidth=960, fheight=720)
ConvertToRGB32(interlaced=false,matrix="Rec601",ChromaInPlacement="MPEG2")

function FixRipsP2(clip a)
{
a
clense(reduceflicker=false).merge(last,0.5).clense(reduceflicker=false)
mot=removegrain(11,0).removegrain(20,0).DepanEstimate(range=2)
take2=a.depaninterleave(mot,prev=2,next=2,subpixel=2)
clean1=take2.TMedian2().selectevery(5,2)

sup1 = clean1.MiniBlur(1).removegrain(11,0).removegrain(11,0)
 \           .mt_lutxy(clean1,"x 1 + y < x 2 + x 1 - y > x 2 - y ? ?",U=2,V=2)
 \           .msuper(pel=2,sharp=0)
sup2 = a.msuper(pel=2,levels=1,sharp=2)

bv22=sup1.manalyse(isb=true, truemotion=false,global=true,delta=2,blksize=16,overlap=8,search=5,searchparam=4,DCT=5)
bv21=sup1.manalyse(isb=true, truemotion=false,global=true,delta=1,blksize=16,overlap=8,search=5,searchparam=4,DCT=5)
fv21=sup1.manalyse(isb=false,truemotion=false,global=true,delta=1,blksize=16,overlap=8,search=5,searchparam=4,DCT=5)
fv22=sup1.manalyse(isb=false,truemotion=false,global=true,delta=2,blksize=16,overlap=8,search=5,searchparam=4,DCT=5)

interleave(a.mcompensate(sup2,fv22),a.mcompensate(sup2,fv21),a,a.mcompensate(sup2,bv21),a.mcompensate(sup2,bv22))
TMedian2().selectevery(5,2)

sup3 = last.msuper(pel=2,sharp=2)
bv33=sup3.manalyse(isb=true, truemotion=false,global=true,delta=3,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
bv32=sup3.manalyse(isb=true, truemotion=false,global=true,delta=2,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
bv31=sup3.manalyse(isb=true, truemotion=false,global=true,delta=1,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
fv31=sup3.manalyse(isb=false,truemotion=false,global=true,delta=1,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
fv32=sup3.manalyse(isb=false,truemotion=false,global=true,delta=2,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)
fv33=sup3.manalyse(isb=false,truemotion=false,global=true,delta=3,blksize=8,overlap=4,search=5,searchparam=4,DCT=5)

last.mdegrain3(sup3,bv31,fv31,bv32,fv32,bv33,fv33,thSAD=499)

interleave()
return last
}


function MinBlur(clip clp, int r, int "uv")
{
uv   = default(uv,3)
uv2  = (uv==2) ? 1 : uv
rg4  = (uv==3) ? 4 : -1
rg11 = (uv==3) ? 11 : -1
rg20 = (uv==3) ? 20 : -1
medf = (uv==3) ? 1 : -200

RG11D = (r==0) ? mt_makediff(clp,clp.sbr(),U=uv2,V=uv2)
 \    : (r==1) ? mt_makediff(clp,clp.removegrain(11,rg11),U=uv2,V=uv2)
 \    : (r==2) ? mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20),U=uv2,V=uv2)
 \    :          mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20).removegrain(20,rg20),U=uv2,V=uv2)
RG4D  = (r<=1) ? mt_makediff(clp,clp.removegrain(4,rg4),U=uv2,V=uv2)
 \    : (r==2) ? mt_makediff(clp,clp.medianblur(2,2*medf,2*medf),U=uv2,V=uv2)
 \    :          mt_makediff(clp,clp.medianblur(3,3*medf,3*medf),U=uv2,V=uv2)
DD    = mt_lutxy(RG11D,RG4D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
clp.mt_makediff(DD,U=uv,V=uv)
return(last)
}

# median of 5 clips from helpers.avs by G-force
function Median2(clip "input_1", clip "input_2", clip "input_3", clip "input_4", clip "input_5", string "chroma")
{
chroma = default(chroma,"process") #default is "process". Alternates: "copy first" or "copy second"
#MEDIAN(i1,i3,i5)
Interleave(input_1,input_3,input_5)
chroma == "process" ? Clense(reduceflicker=false) : Clense(reduceflicker=false,grey=true)
m1 = selectevery(3,1)
#MAX(MIN(i1,i3,i5),i2)
m2  = input_1.MT_Logic(input_3,"min",chroma=chroma).MT_Logic(input_5,"min",chroma=chroma).MT_Logic(input_2,"max",chroma=chroma)
#MIN(MAX(i1,i3,i5),i4)
m3  = input_1.MT_Logic(input_3,"max",chroma=chroma).MT_Logic(input_5,"max",chroma=chroma).MT_Logic(input_4,"min",chroma=chroma)
Interleave(m1,m2,m3)
chroma == "process" ? Clense(reduceflicker=false) : Clense(reduceflicker=false,grey=true)
selectevery(3,1)
chroma == "copy first" ? last.MergeChroma(input_1) : chroma == "copy second" ? last.MergeChroma(input_2) : last
Return(last)
}

function TMedian2(clip c) {
Median2(c.selectevery(1,-2), c.selectevery(1,-1), c, c.selectevery(1,1), c.selectevery(1,2) ) }
This is just an example script which fails to execute, I am not actually thinking of putting FixRips for whole video..
Reply With Quote
  #20  
05-14-2019, 04:14 PM
lordsmurf's Avatar
lordsmurf lordsmurf is offline
Site Staff | Video
 
Join Date: Dec 2002
Posts: 13,443
Thanked 2,442 Times in 2,075 Posts
Quote:
Originally Posted by sanlyn View Post
I'm still working this afternoon on several ideas for fixes and will post later today, but have been interrupted at home several times. Phooey.
Same here. I really, really want to sit in my recliner with a tablet, and try to digest this thread.

Very Avisynth-y discussion here.

At a cursory glance, I'm lost as to what's going on here. Good job. I don't confuse easy.

- Did my advice help you? Then become a Premium Member and support this site.
- For sale in the marketplace: TBCs, workflows, capture cards, VCRs
Reply With Quote
Reply




Similar Threads
Thread Thread Starter Forum Replies Last Post
Problems with Elite Video BVP-4 Plus, lines on side of video? Dallas Video Hardware Repair 6 06-13-2021 06:11 PM
Remove wavy lines going down video? esotech Restore, Filter, Improve Quality 3 02-24-2017 12:51 PM
Anti-Aliasing Jaggies on chalk lines of race track with Avisynth? VideoFanatic Restore, Filter, Improve Quality 1 01-23-2014 11:02 PM
JVC HR-D470UM help - slow speed video, lines in video WestroCezarf Video Hardware Repair 1 09-05-2013 02:57 PM
Remove extra lines (empty lines) from copied/pasted text with Notepad++ kpmedia Computers 1 10-07-2011 05:32 PM

Thread Tools



 
All times are GMT -5. The time now is 04:42 AM