Go Back    Forum > Digital Video > Video Project Help > Restore, Filter, Improve Quality

Reply
 
LinkBack Thread Tools
  #1  
07-30-2014, 03:59 AM
Bobban Bobban is offline
Free Member
 
Join Date: Jul 2014
Posts: 27
Thanked 0 Times in 0 Posts
Hi!

Have been checking out this great site and have been getting a lot of helpful tips while planning my project, and decided to register now and see if I can get some additional help here.

The basics: I am converting PAL VHS tapes to digital files, using a JVC HR-S8500 (with TBC) hooked up with an S-video cable, converting them to AVI files which I then process through color correction etc. in Vegas Pro 11.0.

Now, here's my question. Some of my tapes were recorded back in the mid-90's here in Sweden off a satellite (if I remember correctly), and there seems to have been some disturbance in the signal. The result of this is that in bright areas there are black spots popping up and in dark areas there are white spots popping up. This is inherit to the quality of the satellite signal I believe (sometimes no spots at all and sometimes a lot of them) and the tapes always looked like that when you viewed them, regardless of VCR player. The spots move and never stay still for more than a frame at a time. Because of that they need to be removed after the capture process, and there are too many to do it manually frame by frame with masking.

I did a search and only came up with one solution that seemed to address this issue: The Dirt Remover FX, which is included in a plugin package for Final Cut from CHV-plugins called RepairFX. It seems like this would be the perfect tool, but I neither have a Mac or Final Cut...

So, then: Anyone have any tips or alternative tools I could use? I have access to Vegas Pro, Virtualdub, AviSynth and After Effects.

Thank you in advance! Here are a couple of screens showing what I mean (the white background has black spots and the black background has white spots):


Reply With Quote
Someday, 12:01 PM
admin's Avatar
Ads / Sponsors
 
Join Date: ∞
Posts: 42
Thanks: ∞
Thanked 42 Times in 42 Posts
  #2  
07-30-2014, 06:53 AM
dpalomaki dpalomaki is offline
Free Member
 
Join Date: Feb 2014
Location: VA
Posts: 1,700
Thanked 370 Times in 326 Posts
Neatvideo can help remove noise, but might not work for what you have. You could try the demo download. The is a plug-in version for Vegas and several other programs..
Reply With Quote
  #3  
07-30-2014, 07:22 AM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,308 Times in 982 Posts
There is no effective VirtualDub or NLE filter for those kinds of spots. Avisynth has a handful of fairly effective de-spot plugins.
Reply With Quote
  #4  
07-30-2014, 08:46 AM
themaster1 themaster1 is offline
Free Member
 
Join Date: Feb 2011
Location: France
Posts: 509
Thanked 103 Times in 87 Posts
Try this avs script:
You need removespotmc, removedirtmc

Code:
SetMemoryMax(900)
SetMTMode(3,4)
MPEG2Source("MyVideo.d2v", cpu=0,info=3)
SetMTMode(4) # for 4 cores

c=last
 #even fields
# Motion compensation on frames -2...+2
even = c.SeparateFields().SelectEven()
super_even=even.MSuper()
vf2 = super_even.MAnalyse (isb=false, delta=2, overlap=4)
vf1 = super_even.MAnalyse (isb=false, delta=1, overlap=4)
vb1 = super_even.MAnalyse (isb=true,  delta=1, overlap=4)
vb2 = super_even.MAnalyse (isb=true,  delta=2, overlap=4)
cf2 = MCompensate (even, super_even, vf2, thSAD=400)
cf1 = MCompensate (even, super_even, vf1, thSAD=400)
cb1 = MCompensate (even, super_even, vb1, thSAD=400)
cb2 = MCompensate (even, super_even, vb2, thSAD=400)
 # dfttest on motion-compensated clip
Interleave (cf2, cf1, even, cb1, cb2)
SetMTMode(2)
RemoveSpotsMC() # 2 threads only
#dfttest (sigma=16, sigma2=16, tbsize=3, lsb=true)
#RemoveDust(last, 2) 
#RemoveSpotsMC()
#dfttest (sigma=4, sigma2=4, tbsize=1, lsb=true) # param pour 2Pac SNL
SetMTMode(4)
SelectEvery (5, 2)
 # Gradient smoothing
presmooth = last
#SmoothGrad (radius=20, thr=0.15) #
#SmoothGrad (radius=12, thr=0.25, ref=presmooth)# 
 # Back to 8 bits, ordered dithering
#DitherPost () #
filtered_even=last
 #odd fields
# Motion compensation on frames -2...+2
odd = c.SeparateFields().SelectOdd()
super_odd=odd.MSuper()
vf2 = super_odd.MAnalyse (isb=false, delta=2, overlap=4)
vf1 = super_odd.MAnalyse (isb=false, delta=1, overlap=4)
vb1 = super_odd.MAnalyse (isb=true,  delta=1, overlap=4)
vb2 = super_odd.MAnalyse (isb=true,  delta=2, overlap=4)
cf2 = MCompensate (odd, super_odd, vf2, thSAD=400)
cf1 = MCompensate (odd, super_odd, vf1, thSAD=400)
cb1 = MCompensate (odd, super_odd, vb1, thSAD=400)
cb2 = MCompensate (odd, super_odd, vb2, thSAD=400)
# dfttest on motion-compensated clip
Interleave (cf2, cf1, odd, cb1, cb2)
SetMTMode(2)
RemoveSpotsMC()
SetMTMode(4)
SelectEvery (5, 2)
# Gradient smoothing
presmooth = last
#SmoothGrad (radius=20, thr=0.15) #
#SmoothGrad (radius=12, thr=0.25, ref=presmooth)#SNL VHS
# Back to 8 bits, ordered dithering
#DitherPost ()# 
filtered_odd=last
Interleave(filtered_even, filtered_odd)
###########
# remove black lines (again) / not necessary:
noisy   = last
nonoise =RemoveDirtMC
diff    = 10                         # Difference between denoised and noisy clips
m       = mt_lutxy(noisy,nonoise,"y x - "+string(diff)+" - 255 *")
mt_merge(noisy,nonoise,m)
weave
Reply With Quote
  #5  
07-30-2014, 09:03 AM
lordsmurf's Avatar
lordsmurf lordsmurf is online now
Site Staff | Video
 
Join Date: Dec 2002
Posts: 13,654
Thanked 2,461 Times in 2,093 Posts
I'd need to see a sample video clip. It's probably a simple depulse() fix in Avisynth.
This error is not as hard is it once was.

- Did my advice help you? Then become a Premium Member and support this site.
- For sale in the marketplace: TBCs, workflows, capture cards, VCRs
Reply With Quote
  #6  
07-30-2014, 02:43 PM
Bobban Bobban is offline
Free Member
 
Join Date: Jul 2014
Posts: 27
Thanked 0 Times in 0 Posts
Thank you so much everybody for quick replys! I'm feeling hopeful now

@themaster1: I can't seem to find the "removespotmc" plugin anywhere... Also, thanks for the script. I'm not very experienced with these things - is that text ready to just be copied into an avs file and loaded or do I have to modify it (i.e. have you written explanations or such in the script that I have to remove first?).

@Lordsmurf: Here's a short clip of the spots. Also, having trouble finding the depulse plugin...


Attached Files
File Type: avi Spots.avi (13.48 MB, 51 downloads)
Reply With Quote
  #7  
07-30-2014, 05:45 PM
lordsmurf's Avatar
lordsmurf lordsmurf is online now
Site Staff | Video
 
Join Date: Dec 2002
Posts: 13,654
Thanked 2,461 Times in 2,093 Posts
Easy.

Code:
AVISource("c:\Spots.avi")
ConvertToYUY2 
ChromaShift(C=-2, L=0) # align chroma over luma
Cnr2("xoo",4,2,64) # remove chroma banding noise, wide UV setting
DePulse(h=70, l=1, d=15, debug=false) # white -max
DePulse(h=10, l=800, d=5, debug=false) # black
and add a few filters in VirtualDub for added clean-up.

This was really easy to fix.

This was all on my MultiScript that I'm building right now. I'm slowly working on the Avisynth guides.


Attached Files
File Type: m2v spots-restored_out.m2v (6.55 MB, 49 downloads)
File Type: vcf spots.vcf (983 Bytes, 25 downloads)
File Type: rar DePulse.rar (17.7 KB, 40 downloads)

- Did my advice help you? Then become a Premium Member and support this site.
- For sale in the marketplace: TBCs, workflows, capture cards, VCRs
Reply With Quote
The following users thank lordsmurf for this useful post: Bobban (07-31-2014)
  #8  
07-31-2014, 03:11 AM
Bobban Bobban is offline
Free Member
 
Join Date: Jul 2014
Posts: 27
Thanked 0 Times in 0 Posts
Wow, thank you so much Lordsmurf!!! Looks like it removes all the spots. However, when I tried it on a another clip I noticed that it seems to leave some artefacts around the edges in scenes with movement (could this be due to interlacing???). It creates new noise, small black/white spots around the edges of the animation where the interlaced fields "overlap", that were not there before. See the attached pics of before/after.


Attached Images
File Type: jpg After.jpg (30.3 KB, 36 downloads)
File Type: jpg Before.jpg (33.4 KB, 35 downloads)
Reply With Quote
  #9  
07-31-2014, 03:27 AM
themaster1 themaster1 is offline
Free Member
 
Join Date: Feb 2011
Location: France
Posts: 509
Thanked 103 Times in 87 Posts
Depulse ain't that great according to my tests that's why i like removespotsmc

download it from here: http://www.mediafire.com/view/oy9k0a...moveSpots.avsi
Reply With Quote
  #10  
07-31-2014, 04:00 AM
lordsmurf's Avatar
lordsmurf lordsmurf is online now
Site Staff | Video
 
Join Date: Dec 2002
Posts: 13,654
Thanked 2,461 Times in 2,093 Posts
I was never a fan of RemoveSpotsMC -- it also left interlace artifacts, and also left motion issues.

With DePulse, all you need to do is tweak the value. The sample I uploaded was not perfect either -- it had some small interlace boogers. But this was a just a quick proof-of-concept post. A sample script for the sample clip. The full video may pose other issues. It will require some testing and tweaking on your part.

@Bobban, if you need more help, ask more questions.

@themaster1: I can't download from MediaFire. I wish you'd use Dropbox.

- Did my advice help you? Then become a Premium Member and support this site.
- For sale in the marketplace: TBCs, workflows, capture cards, VCRs
Reply With Quote
The following users thank lordsmurf for this useful post: Bobban (08-02-2014)
  #11  
07-31-2014, 05:34 AM
Bobban Bobban is offline
Free Member
 
Join Date: Jul 2014
Posts: 27
Thanked 0 Times in 0 Posts
Okay then, so which part of the code would cause these "interlace boogers" then? I.e. which values should I tweak in order to find a good script?

From what I can find it controls the brightness of the pixels, where h is the high value and l is the low value. What is d? Debug can be used to see which pixels are depulsed, no?

Here is a short raw clip from that animation, without anything applied (spots still in it).


Attached Files
File Type: avi Animation clip.avi (31.76 MB, 11 downloads)

Last edited by Bobban; 07-31-2014 at 06:16 AM.
Reply With Quote
  #12  
07-31-2014, 10:28 AM
Bobban Bobban is offline
Free Member
 
Join Date: Jul 2014
Posts: 27
Thanked 0 Times in 0 Posts
So I have had time to play around with the settings, but to no avail I'm afraid... I think I know too little about the parameters to understand how to change them. It seems that either it takes away no spots at all or way too much, leaving those ugly artefacts in the interlacing overlaps. I tried de-interlacing it with QTGMC, getting rid of the overlap areas (50 fps now instead) and then applying your script, but then it didn't seem to remove anything at all???
Reply With Quote
  #13  
07-31-2014, 01:45 PM
themaster1 themaster1 is offline
Free Member
 
Join Date: Feb 2011
Location: France
Posts: 509
Thanked 103 Times in 87 Posts
Why is the video sample in RGB to begin with ?
Reply With Quote
  #14  
07-31-2014, 02:26 PM
Bobban Bobban is offline
Free Member
 
Join Date: Jul 2014
Posts: 27
Thanked 0 Times in 0 Posts
RGB? All of the clips are YUV 4:2:0.
Reply With Quote
  #15  
07-31-2014, 03:16 PM
themaster1 themaster1 is offline
Free Member
 
Join Date: Feb 2011
Location: France
Posts: 509
Thanked 103 Times in 87 Posts
Vdub says it's Dv but on AVSP it report rgb24
Reply With Quote
  #16  
07-31-2014, 03:46 PM
Bobban Bobban is offline
Free Member
 
Join Date: Jul 2014
Posts: 27
Thanked 0 Times in 0 Posts
Hmm, I just checked them with Media Info and it says YUV.
Reply With Quote
  #17  
07-31-2014, 04:35 PM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,308 Times in 982 Posts
The animation clip attached to post #11 is in YUV 4:2:0. As VHS it played as Top Field First inerlaced/telecined video. The attached sample was DV encoded with the SONY DVSD codec, Bottom Field First. Analog tape encoded to DV with this change in field structure and a different color matrix and storage algorithm has always struck me as having noisy, ragged edges with interlace "boogers" as you call them.

The spot removers used and suggested don't work well with interlace or telecine, and they neglect the edge noise and interlace problems anyway. The video has to be inverse telecined for most of Avisynth's filters to prove their mettle. They were designed for progressive or deinterlaced\de-telecined video. As you've already incurred some problems and quality loss with a lossy DV encode, and as the proposed color correction and denoising methods will throw smart-rendering out the window anyway and incur more lossy re-encoding, I'd suggest that you avoid DV in the first place. Learn to work with lossless media and the original source field priorities when trying to fixup bad analog video. IMO you can only go downhill from where you are now.

Last edited by sanlyn; 07-31-2014 at 05:17 PM.
Reply With Quote
  #18  
08-01-2014, 02:24 AM
Bobban Bobban is offline
Free Member
 
Join Date: Jul 2014
Posts: 27
Thanked 0 Times in 0 Posts
@Sanlyn Originally I looked into capturing in uncompressed format, but couldn't get it to work. My capture device is a Pinnacle Moviebox USB-510, which can do either MPEG-2 or AVI - but the AVI only has DV or the even more lossy MJPEG codecs. Unless you can somehow mod/force it to capture with a different codec? I tried using AMcap and virtualdub as well, but they didn't work, so I was forced to use the Pinnacle Studio software...

Anyway, still trying to get the Depulse script to work, playing with the parameters using debug=true to see if it affects the wrong areas (see image), but failing... Tried inverse telecide and de-interlace first, but after that it won't remove any spots at all? If someone could aid in this I would be very greatful!

Reply With Quote
  #19  
08-01-2014, 08:56 AM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,308 Times in 982 Posts
The animation avi appears to have been around the block a few times, possibly originally NTSC with field decimation used to get a slower frame rate and then telecined or interlaced for 25FPS PAL. Difficult to say exactly, as I'm working on an ancient laptop right now -- no one but a masochist would try processing difficult video on a generic laptop -- and this machine doesn't have much of my usual software installed. The telltale signs of fields or frames removed are the stalled/jerky motion, rough edges and aliasing, and objects changing shape during motion. Unfortunately you can't re-create elements that have been discarded or destroyed.

The following script removes almost all of the spots and small comets. You could replace RemoveSpotsMC with lordsmurf's version of DePulse and get pretty much the same results. I added Santiag for some anti-alias; it doesn't really help that much, nor would most other anti-alias filters because the fields/frames that helped define edges and motion are gone forever. The script gives 20FPS progressive with duplicate frames that could use some pulldown flags to get speed up to 25FPS (try DGPulldown).

Code:
AssumeBFF().TFM().TDecimate()
a=last
e=a.SelectEven().RemoveSpotsMC()
o=a.SelectOdd().RemoveSpotsMC()
Interleave(e,o)
Spline36Resize(360,576).Santiag().Spline64Resize(720,576) # ##<- optional, for slight edge and line smoothing
return last
Reply With Quote
The following users thank sanlyn for this useful post: Bobban (08-02-2014), lordsmurf (08-22-2014)
  #20  
08-02-2014, 08:39 AM
Bobban Bobban is offline
Free Member
 
Join Date: Jul 2014
Posts: 27
Thanked 0 Times in 0 Posts
Thank you so much Sanlyn! By combining your script with a personal tweak of Lordsmurf's DePulse (After hours of trying I managed to find a good balance between removal of dots and not destroying animation lines, mostly with the threshold value) I have finally gotten a good result!

Again, thank you both for this wonderful help I've gotten.
Reply With Quote
Reply




Similar Threads
Thread Thread Starter Forum Replies Last Post
Removing black bars on TV shows for DVD? Superstar Edit Video, Audio 12 03-12-2014 11:55 PM
DVD-R with black spots? Neves Blank Media 3 03-02-2014 11:13 PM
What settings to scan old photos in black & white ? Sossity Photo Processing, Scanning & Printing 7 02-20-2012 02:34 PM
9800 AIW black & white prob manthing Capture, Record, Transfer 6 06-26-2005 06:48 PM
Why Black & White capture ? Sujit Capture, Record, Transfer 4 05-23-2005 08:27 AM




 
All times are GMT -5. The time now is 06:49 PM