digitalFAQ.com Forum - Removing black/white spots from VHS capture?

Page 1 of 2

Show 40 post(s) from this thread on one page

digitalFAQ.com Forum (https://www.digitalfaq.com/forum/)

- Restore, Filter, Improve Quality (https://www.digitalfaq.com/forum/video-restore/)

- - Removing black/white spots from VHS capture? (https://www.digitalfaq.com/forum/video-restore/6010-removing-black-white.html)

Bobban

07-30-2014 03:59 AM

Removing black/white spots from VHS capture?

Hi!

Have been checking out this great site and have been getting a lot of helpful tips while planning my project, and decided to register now and see if I can get some additional help here.

The basics: I am converting PAL VHS tapes to digital files, using a JVC HR-S8500 (with TBC) hooked up with an S-video cable, converting them to AVI files which I then process through color correction etc. in Vegas Pro 11.0.

Now, here's my question. Some of my tapes were recorded back in the mid-90's here in Sweden off a satellite (if I remember correctly), and there seems to have been some disturbance in the signal. The result of this is that in bright areas there are black spots popping up and in dark areas there are white spots popping up. This is inherit to the quality of the satellite signal I believe (sometimes no spots at all and sometimes a lot of them) and the tapes always looked like that when you viewed them, regardless of VCR player. The spots move and never stay still for more than a frame at a time. Because of that they need to be removed after the capture process, and there are too many to do it manually frame by frame with masking.

I did a search and only came up with one solution that seemed to address this issue: The Dirt Remover FX, which is included in a plugin package for Final Cut from CHV-plugins called RepairFX. It seems like this would be the perfect tool, but I neither have a Mac or Final Cut...

So, then: Anyone have any tips or alternative tools I could use? I have access to Vegas Pro, Virtualdub, AviSynth and After Effects.

Thank you in advance! Here are a couple of screens showing what I mean (the white background has black spots and the black background has white spots):

http://www.digitalfaq.com/forum/imag...7/Image3-1.png
http://www.digitalfaq.com/forum/imag...7/Image1-1.png

dpalomaki

07-30-2014 06:53 AM

Neatvideo can help remove noise, but might not work for what you have. You could try the demo download. The is a plug-in version for Vegas and several other programs..

sanlyn

07-30-2014 07:22 AM

There is no effective VirtualDub or NLE filter for those kinds of spots. Avisynth has a handful of fairly effective de-spot plugins.

themaster1

07-30-2014 08:46 AM

Try this avs script:
You need removespotmc, removedirtmc

Code:

SetMemoryMax(900)

SetMTMode(3,4)

MPEG2Source("MyVideo.d2v", cpu=0,info=3)

SetMTMode(4) # for 4 cores



c=last

 #even fields

# Motion compensation on frames -2...+2

even = c.SeparateFields().SelectEven()

super_even=even.MSuper()

vf2 = super_even.MAnalyse (isb=false, delta=2, overlap=4)

vf1 = super_even.MAnalyse (isb=false, delta=1, overlap=4)

vb1 = super_even.MAnalyse (isb=true,  delta=1, overlap=4)

vb2 = super_even.MAnalyse (isb=true,  delta=2, overlap=4)

cf2 = MCompensate (even, super_even, vf2, thSAD=400)

cf1 = MCompensate (even, super_even, vf1, thSAD=400)

cb1 = MCompensate (even, super_even, vb1, thSAD=400)

cb2 = MCompensate (even, super_even, vb2, thSAD=400)

 # dfttest on motion-compensated clip

Interleave (cf2, cf1, even, cb1, cb2)

SetMTMode(2)

RemoveSpotsMC() # 2 threads only

#dfttest (sigma=16, sigma2=16, tbsize=3, lsb=true)

#RemoveDust(last, 2) 

#RemoveSpotsMC()

#dfttest (sigma=4, sigma2=4, tbsize=1, lsb=true) # param pour 2Pac SNL

SetMTMode(4)

SelectEvery (5, 2)

 # Gradient smoothing

presmooth = last

#SmoothGrad (radius=20, thr=0.15) #

#SmoothGrad (radius=12, thr=0.25, ref=presmooth)# 

 # Back to 8 bits, ordered dithering

#DitherPost () #

filtered_even=last

 #odd fields

# Motion compensation on frames -2...+2

odd = c.SeparateFields().SelectOdd()

super_odd=odd.MSuper()

vf2 = super_odd.MAnalyse (isb=false, delta=2, overlap=4)

vf1 = super_odd.MAnalyse (isb=false, delta=1, overlap=4)

vb1 = super_odd.MAnalyse (isb=true,  delta=1, overlap=4)

vb2 = super_odd.MAnalyse (isb=true,  delta=2, overlap=4)

cf2 = MCompensate (odd, super_odd, vf2, thSAD=400)

cf1 = MCompensate (odd, super_odd, vf1, thSAD=400)

cb1 = MCompensate (odd, super_odd, vb1, thSAD=400)

cb2 = MCompensate (odd, super_odd, vb2, thSAD=400)

# dfttest on motion-compensated clip

Interleave (cf2, cf1, odd, cb1, cb2)

SetMTMode(2)

RemoveSpotsMC()

SetMTMode(4)

SelectEvery (5, 2)

# Gradient smoothing

presmooth = last

#SmoothGrad (radius=20, thr=0.15) #

#SmoothGrad (radius=12, thr=0.25, ref=presmooth)#SNL VHS

# Back to 8 bits, ordered dithering

#DitherPost ()# 

filtered_odd=last

Interleave(filtered_even, filtered_odd)

###########

# remove black lines (again) / not necessary:

noisy   = last

nonoise =RemoveDirtMC

diff    = 10                         # Difference between denoised and noisy clips

m       = mt_lutxy(noisy,nonoise,"y x - "+string(diff)+" - 255 *")

mt_merge(noisy,nonoise,m)

weave

lordsmurf

07-30-2014 09:03 AM

I'd need to see a sample video clip. It's probably a simple depulse() fix in Avisynth.
This error is not as hard is it once was.

Bobban

07-30-2014 02:43 PM

1 Attachment(s)

Thank you so much everybody for quick replys! I'm feeling hopeful now :)

@themaster1: I can't seem to find the "removespotmc" plugin anywhere... Also, thanks for the script. I'm not very experienced with these things - is that text ready to just be copied into an avs file and loaded or do I have to modify it (i.e. have you written explanations or such in the script that I have to remove first?).

@Lordsmurf: Here's a short clip of the spots. Also, having trouble finding the depulse plugin...:smack:

lordsmurf

07-30-2014 05:45 PM

3 Attachment(s)

Easy. :)

Code:

AVISource("c:\Spots.avi")

ConvertToYUY2 

ChromaShift(C=-2, L=0) # align chroma over luma

Cnr2("xoo",4,2,64) # remove chroma banding noise, wide UV setting

DePulse(h=70, l=1, d=15, debug=false) # white -max

DePulse(h=10, l=800, d=5, debug=false) # black

and add a few filters in VirtualDub for added clean-up.

This was really easy to fix.

This was all on my MultiScript that I'm building right now. I'm slowly working on the Avisynth guides. :2cents:

Bobban

07-31-2014 03:11 AM

2 Attachment(s)

Wow, thank you so much Lordsmurf!!! Looks like it removes all the spots. However, when I tried it on a another clip I noticed that it seems to leave some artefacts around the edges in scenes with movement (could this be due to interlacing???). It creates new noise, small black/white spots around the edges of the animation where the interlaced fields "overlap", that were not there before. See the attached pics of before/after.

themaster1

07-31-2014 03:27 AM

Depulse ain't that great according to my tests that's why i like removespotsmc

download it from here: http://www.mediafire.com/view/oy9k0a...moveSpots.avsi

lordsmurf

07-31-2014 04:00 AM

I was never a fan of RemoveSpotsMC -- it also left interlace artifacts, and also left motion issues.

With DePulse, all you need to do is tweak the value. The sample I uploaded was not perfect either -- it had some small interlace boogers. But this was a just a quick proof-of-concept post. A sample script for the sample clip. The full video may pose other issues. It will require some testing and tweaking on your part.

@Bobban, if you need more help, ask more questions.

@themaster1: I can't download from MediaFire. I wish you'd use Dropbox.

Bobban

07-31-2014 05:34 AM

1 Attachment(s)

Okay then, so which part of the code would cause these "interlace boogers" then? I.e. which values should I tweak in order to find a good script?

From what I can find it controls the brightness of the pixels, where h is the high value and l is the low value. What is d? Debug can be used to see which pixels are depulsed, no?

Here is a short raw clip from that animation, without anything applied (spots still in it).

Bobban

07-31-2014 10:28 AM

So I have had time to play around with the settings, but to no avail I'm afraid... I think I know too little about the parameters to understand how to change them. It seems that either it takes away no spots at all or way too much, leaving those ugly artefacts in the interlacing overlaps. I tried de-interlacing it with QTGMC, getting rid of the overlap areas (50 fps now instead) and then applying your script, but then it didn't seem to remove anything at all???

themaster1

07-31-2014 01:45 PM

Why is the video sample in RGB to begin with ?

Bobban

07-31-2014 02:26 PM

RGB? All of the clips are YUV 4:2:0.

themaster1

07-31-2014 03:16 PM

Vdub says it's Dv but on AVSP it report rgb24

Bobban

07-31-2014 03:46 PM

Hmm, I just checked them with Media Info and it says YUV.

sanlyn

07-31-2014 04:35 PM

The animation clip attached to post #11 is in YUV 4:2:0. As VHS it played as Top Field First inerlaced/telecined video. The attached sample was DV encoded with the SONY DVSD codec, Bottom Field First. Analog tape encoded to DV with this change in field structure and a different color matrix and storage algorithm has always struck me as having noisy, ragged edges with interlace "boogers" as you call them.

The spot removers used and suggested don't work well with interlace or telecine, and they neglect the edge noise and interlace problems anyway. The video has to be inverse telecined for most of Avisynth's filters to prove their mettle. They were designed for progressive or deinterlaced\de-telecined video. As you've already incurred some problems and quality loss with a lossy DV encode, and as the proposed color correction and denoising methods will throw smart-rendering out the window anyway and incur more lossy re-encoding, I'd suggest that you avoid DV in the first place. Learn to work with lossless media and the original source field priorities when trying to fixup bad analog video. IMO you can only go downhill from where you are now.

Bobban

08-01-2014 02:24 AM

@Sanlyn Originally I looked into capturing in uncompressed format, but couldn't get it to work. My capture device is a Pinnacle Moviebox USB-510, which can do either MPEG-2 or AVI - but the AVI only has DV or the even more lossy MJPEG codecs. Unless you can somehow mod/force it to capture with a different codec? I tried using AMcap and virtualdub as well, but they didn't work, so I was forced to use the Pinnacle Studio software...

Anyway, still trying to get the Depulse script to work, playing with the parameters using debug=true to see if it affects the wrong areas (see image), but failing... Tried inverse telecide and de-interlace first, but after that it won't remove any spots at all? If someone could aid in this I would be very greatful!

http://www.digitalfaq.com/forum/imag.../Depulse-1.png

sanlyn

08-01-2014 08:56 AM

The animation avi appears to have been around the block a few times, possibly originally NTSC with field decimation used to get a slower frame rate and then telecined or interlaced for 25FPS PAL. Difficult to say exactly, as I'm working on an ancient laptop right now -- no one but a masochist would try processing difficult video on a generic laptop -- and this machine doesn't have much of my usual software installed. The telltale signs of fields or frames removed are the stalled/jerky motion, rough edges and aliasing, and objects changing shape during motion. Unfortunately you can't re-create elements that have been discarded or destroyed.

The following script removes almost all of the spots and small comets. You could replace RemoveSpotsMC with lordsmurf's version of DePulse and get pretty much the same results. I added Santiag for some anti-alias; it doesn't really help that much, nor would most other anti-alias filters because the fields/frames that helped define edges and motion are gone forever. The script gives 20FPS progressive with duplicate frames that could use some pulldown flags to get speed up to 25FPS (try DGPulldown).

Code:

AssumeBFF().TFM().TDecimate()

a=last

e=a.SelectEven().RemoveSpotsMC()

o=a.SelectOdd().RemoveSpotsMC()

Interleave(e,o)

Spline36Resize(360,576).Santiag().Spline64Resize(720,576) # ##<- optional, for slight edge and line smoothing

return last

Bobban

08-02-2014 08:39 AM

Thank you so much Sanlyn! By combining your script with a personal tweak of Lordsmurf's DePulse (After hours of trying I managed to find a good balance between removal of dots and not destroying animation lines, mostly with the threshold value) I have finally gotten a good result!

Again, thank you both for this wonderful help I've gotten.:congrats:

All times are GMT -5. The time now is 08:11 AM

Page 1 of 2

Show 40 post(s) from this thread on one page