Horizontal dropout/pulses in VHS capture?
 

In my VHS capture's I'm seeing horizontal noise lines such as in the screenshot attached below (also a lagarith encoded video sample is attached).

The lines will appear at random heights, Each time a line such as above appears, it will only appear for a very short amount of time, they come 'flashing' by.

My question is threefold,

What am I seeing?
Are these the VHS dropouts which are commonly referred to? Most screenshots of those I've seen on forums are much smaller horizontal lines than these.

Why am I seeing these lines?
At first I thought this was due to old/bad VHS tapes, but I'm seeing it in pretty much all of the material I'm archiving. Is this a VCR issue, is it in my tapes, or is it related to my BT878 capture card (using iuvcr drivers)?

Can I filter it?
The only filter with which I've managed to configure to 'correct' these lines is the DePulse() avisynth filter. However this filter also introduces quite heavy speckled noise on fast high-contrast moving objects. (It seems to trigger on any fast moving contrast rather than horizontal lines only)

In the still, it appears to be a future/past portion of a frame. If so, that looks more like a capture card error than anything else. Or a hard drive related issue, probably fragmentation.

But in the video clip, it's so fast that it appears to be standard dropouts.

It indeed seems to be capturecard or driver related.
If I change the VCR for a Wii with composite output I get the same distortions.

I tried changing to the btwincap drivers instead of the iuvcr Bt878 drivers, but I can't set the 720*576 resolution on that driver and still see result in VirtualDub capturing. The btwincap drivers seem to be quite buggy on the XP SP3 machine I'm using for this.

I have never like the BT8x8 capture chips, as they have too many issues. And the quality is subpar anyway. For example, did you know that it is distorting the aspect ratio?

This is why ATI AIW and others are suggesting. I'd even take DV over BT8x8!

I agree strongly with lordsmurf's reply. You need a better capture device. I'd also suggest that your VCR, which you didn't mention, has severe oversharpening and halo effects. There are also some bad color problems, common with the capture card you mentioned and discussed in greater detail, below.

Thanks for the .avi sample. The sample has two visible glitches. If you didn't capture in RGB, then it's been converted to RGB in VirtualDub. Noise and chroma problems are best fixed in the original YUV color space, before any other color conversions are made. The other glitch is a dropped field and a duplicated field that results in visible stutter at frame 10 in the original. If you view the video as fields instead of frames (you can do this in Virtualdub), you'll see that the bottom field of frame 9 and the bottom field of frame 10 are duplicated due to a dropped field in frame 10. I fixed this in Avisynth with the ReplaceFramesMC filter, creating new fields for the top and bottom fields of frame 10. Otherwise, when the former bottom fields repeats, the fields move backward for one frame when viewed as fields, and they display as a still image that stutters when viewed as interlaced.

Those horizontal lines (sometimes called rips or ripples, a form of dropout) are usually caused by scratched, damaged tape, made worse by playing in a VCR with a soiled tape path (dirt on rollers, audio and video heads). That kind of magnetic oxide layer damage is usually permanent. But as lordsmurf mentioned the disturbances can often be repaired to a great extent, if not always completely.

Below are portions from two frames. The top image is from the original interlaced frame 54, showing a horizontal rip across the lower middle of the frame. The bottom frame is the same interlaced fame after filtering and some color correction with Avisynth and VirtualDub. The images still show interlace combing and severe motion blur -- and for reasons described below, the bad color can be improved but not fully corrected.

Interlaced frame 54 original (showing horizontal streak):
http://www.digitalfaq.com/forum/atta...1&d=1453130584

Interlaced frame 54 after corrections, streak repaired with RemoveSpotsMC in Avisynth:
http://www.digitalfaq.com/forum/atta...1&d=1453130644


Some of the orange stain along the top and right border were cleaned, but it would have been more effective if that video had not been converted to RGB. A small white bit of the horixontal line is still visisble above the front railing of the bullding's porch. Cleanup of rainbows and chroma smear and blotches would also have been more effective in the original YUV. YUV and RGB histograms show that darks are slightly crushed yet elevated above RGB30 or so, and YUV and RGB both show severe bright clipping "spikes", which destroys bright detail and limits the dynamic range of levels and brilliance that one can get in an image.

YUV (left) and RGB (right) unfiltered histograms, frame 41 original):
http://www.digitalfaq.com/forum/atta...1&d=1453131147

The histograms show a crippled V channel (red) and elevated blue, resulting in a strong bluish-cyan color cast that creates a blue "fog" in the image and mutes yellows. Two chroma cleaners used for the attached mpg were BiFrost and SmoothUV in Avisynth, and CamcorderColorDenosie in VirtualDub. Color correction, such as was possible, was with ColorMill and gradation curves in VirtualDub. I didn't try to work a complete fix here, which is really difficult with a poor capture and damaged tape.

I'd suggest that you learn to repack old tapes, which can help get smoother tape flow. Repacking means to fast-forward the tape all the way to the end (without playing it, and without stopping), then fast rewind to the beginning the same way. You might have to repeat the process. The purpose is to smooth out bumps and ledges in the tape winding. Do not play the tape while doing this: use fast forward and fast rewind by themselves. Let the tape rest for a day or so so that the elastic can have time to return to a more natural state.

The audio frequency in your sample is invalid for DVD, BluRay, or AVCHD. The audio frequency should be 48KHz for those formats, not 44.1KHz.

Thank you for your very detailed reply and extensive effort you have put into your post. I really appreciate it and have taken time to study every detail you wrote about.

Firstly, addressing the main issue; the dropout lines, it's nice to see they can in fact be filtered, however as I understand the ReplaceFramesMC() function would need to be used to manually specify which frames to be fixed. That's no option considering the dropouts are occuring every few seconds over hours of material.

Mostly I gather the capture hardware and possibly VCR are the main issues.
Proper hardware is a problem for me, I'm using a rather old Philips VR6490 VCR, and I don't have the budget to buy both professional canopus-grade capture hardware and the type of VCR that would get the most out of such capture hardware.

However, when I connect the VCR directly to the TV the image quality seems to be quite acceptable (for me at least, I'm sure both lordsmurf and sanlyn would disagree). So the question for me is how to get the best out of the situation within a small budget.

I have got my hands on a USB PCTV Quattrostick 521e (Hauppage Cx23100) and will try to see what results that will get me. I don't have my hopes up as i tried this card before and got some sort of grain over the captured image. I'll post a sample as soon as the tape currently being captured is finished.

Another option I'm considering is buying a MiniDV camcorder with composite-in functionality and firewire-out. Would this be likely to bring me any quality improvements? I'm seeing this option being mentioned quite a lot, and I think it will be easy to get a second hand sony miniDV cam for a few bucks. Which I could then hook up to my VCR.

By the way, I did clean my VCR heads already. If there's any quality loss resulting from head oxidation that would probably be indeed of a permanent nature.

PS. I also went to find where the RGB conversion happened, I thought I had everything set to YUY2. Turned out it was a setting in the lagarith capture codec settings. Thanks for pointing that out to me.

Here's the repacked capture from my Hauppage Cx23100 USB card. To me it looks worse than the the Bt878 capture.

I've, again, attached a lagarith avi sample and a snapshot.

Considering this result with my current hardware the Bt878 still looks like the better option. Unless this Hauppauge capture can be filtered to look better. (Which my attempts have failed)

ReplaceFramesMC wasn't used to clean up the horizontal streaks. It was used to interpolate a new field for for the dropped field. You can always live with a stutter or two if necessary. The streaks were handled with RemoveSpotsMC, which effectively is an "auto" filter that finds such junk for you and tries to interpolate clear areas. It isn't 100% effective with such fast motion, but it worked fairly well here. To use either of those filters one must separate fields (which I did for RemoveSpotsMC) or deinterlace into full frames (for ReplaceFramesMC) and then re-weave or re-interlace. The deinterlacer was QTGMC, which also denoised quite a bit. The results were re-interlaced with Avisynth.

Nothing made by Canopus would be appropriate for VHS, much less for damaged VHS. We wouldn't recommend it, so save your money. It doesn't capture to lossless video anyway.

Well yeah, we'd disagree, but we'd also agree that it looks "better" directly to TV because the signal stays in the analog domain. Going from analog to digital is a different story.

Not familiar with that item, but a sample would yield much information.

I don't think you'll care for the digital nosie that results from analog-to-DV unless the camera has analog pass-thru. You cannot capture lossless video via Firewire. Compsoite from a VCR is another step down: if you've ever tried to clean compsoite dot crawl and chroma problems by nearly ruining a video to get rid of it, you'll see what we mean.

Heads don't oxidize. Tape layers do to an extent, with time. At least a cleaner VCR will prevent further damage.

Likely you saved the sample in "full processing mode". To prevent a colorspace change use "direct stream copy".

DV-AVI devices are not lossless. They make VHS defects extremely difficult and often impossible to repair. Like lossless media, DV is PC-only playback. Denoisng, repairing, or color correcting DV means another lossy re-encode every time you process the video, plus the last encode for your intended final output format. What gets lost thru lossy re-encoding can't be retrieved. The only way to clean up DV other than simple cut-and-join is to decode it to lossless media anyway.

The idea behind capturing analog to lossless media is to get the best quality you can with post-processing and for use with better software encoders than are normally available with something like DVD recorders or lossy encoding capture devices. Some first-class post-process encoders are free. The devices you mention will make little or no improvement over what you already have. It's true that direct-to-TV sometimes looks, well, relatively "better" -- but as mentioned, analog to digital is another story. You wouldn't get the same results by capturing directly to a lossy digital format like DV, MPEG, or h264.

Many of the hardware items recommended here are not very expensive, as a used ATI USB card or Diamond VC500 is often sold at low prices. Quality begins with the player, as you probably know. Each link in the circuit affects the overall result. But as you say, most people just do the best they can with what they have.
:)

Thanks for the Hauppauge cap sample. Getting late where I am, so I'll post tomorrow A.M. with details.

That's good news, if I can't get better results with another capture device that would probably be a great improvement. I'll try to figure out where to get this plugin and test it next time I have some time on my hands. (RemoveSpotsMC() isn't listed on avisynth.org, is this the script you're using? )

My wallet thinks that's good to hear. :) Thanks for pointing it out, having no to little experience with different kinds of capture hardware it's difficult to estimate what would get me better results cost efficiently.Learned something new again :) Didn't know that modern LCD/LED TV's dont convert the analog signal to digital in order to display it.

Again, thanks for saving me my money on something that wouldn't yield better results. I don't mind the extra work with lossless capturing, I like figuring stuff like that out.


If the QuattroStick won't get me decent results I'll try getting one of those. There seem to be some availability problems with those products in the Netherlands. They aren't listed on any of the tech comparison sites I'm using. However I probably can order them from the german amazon site. They often, but not always, ship products to the Netherlands as well. That's something I'll figure out.

PS.
The QuattroStick 521e USB stick I'm using is an equivalent of the Hauppage USB-Live2 / WinTV-USB2-FM chipset. Maybe you're familiar with those.

Thanks for your second capture.

That's the script. It originally showed up in Doom9 but was modified in the post your reference.

The script contains multiple functions: RemoveSpotsMC (a single-pass filter), RemoveSpotsMC2X (two passes of RemoveSpots), and RemoveSpots3X (3 passes of remove spots, and greatly softens the video). I used RemoveSpotsMC2X. You can call any of the three in your script.

In the case of your latest capture, RemoveSpotsMC thins the ripple visibly, but it's still there. I tried another Avisynth filter, DeVCR.avs It removed much of the rip but left a slightly gray line. That particular rip isn't a single-frame or double-frame dropout. It's a ripple that extends for 5 fields and is "rolling" vertically, deforming everything it touches. Unfortunately that rolling ripple doesn't look like an elongated "spot", it just looks like part of the image. Rather than a dropout of displaced or missing oxide pieces, it's a wrinkle or "cinch" in the tape. Because it's multiple fields in duration, you might just have to live with what you get. I get them on old tapes often.

The front display panel of every PC monitor or TV, even the pro stuff, is a digital-to-analog converter (DAC). Analog passes thru as-is, but digital input is converted to analog RGB. Human beings can't see 0's and 1's. All such "digital" devices are analog in the end, including audio CD players.

I've seen many posts with that device. Seems to be of decent but variable quality, although ATI and Diamond make better units. Some of those USB-Live2 devices work, well, tolerably...but quality control is a big problem, just as it is with the EZ-Cap EasyCap ripoffs where you never know what you're getting.

Some notes on your latest capture. It's sharper and interlaces better, cleaner in that it doesn't have as much chroma blotching, smear, or rainbows. But remember that you're comparing two different tapes with the same capture device. I'd still suggest that you try to get better. MAny members here have found other suitable units in Europe.

Some notes on your latest capture, which I found very revealing of your VCR. First, the color in the video is green because of the lighting. This is easy enough to fix. I used gradation ciurves in Virtualdub, but you can do the same thing with ColorMill. The trick is to lower green about 20% in nthe midtones, and maybe just 5% in the darks and brights. Counter that by adding about 20% midtone blue, and a little less blue in the darks and brights. Red is about right.

The original, frame 81 (green):
http://www.digitalfaq.com/forum/atta...1&d=1453217953

Frame 81 after color correction as described:
http://www.digitalfaq.com/forum/atta...1&d=1453218007

But color casts are a relatively minor problem here. Did you notice in the above images that the side borders are leaning to the left at the top? That's a scanline sync problem. Some scanlines in the image are reaching the capture device at a different point in time than other scanlines. Take a look at the image below of a blown-up section of the right-hand side of frame 25. The window frames in the mirror image are distorted--some are bowed inward, some outward. Lest you think that this is a problem with the mirror, look closer. The frame of the mirror itself is also warped, and so is the right-hand window and brown panel that aren't in the mirror.

http://www.digitalfaq.com/forum/atta...1&d=1453218306

In the above image you'll also see a fine cross-hatch pattern (herringbone) and dot crawl, as well as bright oversharpening halos on vertical edges. There are filters that clear some of this stuff, but they're aggressive and will destroy fine detail. Meanwhile, the warping and wiggles and vertical edges will move and change shape during play -- you'll see this on TV, not just on your PC. The wiggles can't be fixed in software. They can be avoided only by using a line-level tbc, which is not the same thing as a frame-level tbc for fps timing sync. The same wiggles are also in your original capture of the dock scene -- more difficult to see because of the noise and blurring, but still present. Note that the earlier capture didn't have black side borders, but the time/date characters wiggled a bit as in the above video.

The attached Lagarith .avi was deinterlaced with QTGMC to show how the edges morph and warp from field to field. The .avi was slowed to 8fps. As you watch the video (play it in VirtualDub), keep an eye on vertical frames and on the shifting borders to see what's happening. The oversharpening is from the VCR, the wiggles are worsened by tape slippage on a worn capstan in your VCR as well as by the condition of the tape. Only a line-level tbc can fix this problem, and only during capture. It can't be repaired later. I suspect that the herringbone is from the capture card. I don't recall seeing it in the earlier capture; it might be on the tape.

If you don't have an external line tbc (they're too expensive anyway) or a built-in line tbc in a VCR (they're expensive too, but not as much as an external line unit), you can get line sync and decent frame sync together by using a Panasonic ES10 or ES15 as a pass-tru tbc. You connect the VCR to the Panny, then connect the Panny's output to a capture card. These units also have decent y/c comb filters to avoid dot crawl and are almost always found used on auction sites at decent prices -- a lot cheaper than a prosumer VCR or external line unit. Very few similar units can be used for pass-thru.

Tracking in older badly aligned players, and even in good players, is a serious issue. In the post that follows this I'll show you some Dropouts From Hell, if you think you're having problems.

IF you think you have some annoying dropouts and ripples, see below. The images are unretouched from the original huffyuv captures, although of course I rezised them to fit them into a post. No other processing. They show how a better and more robust VCR, even if it isn't a fancy prosumer unit, can make a vast difference in tracking.

The particular tape in question was damaged course rubbing in a JVC VCR. There are several sequences, but I'm showing only two samples. You can blame the dropouts on dirty tape, but in this case it's due to damage rather than dirt. The same tape was used in two different players: a problematic and lightweight Panasonic PV-9668 from 1999, and later with better results in an older, cleaned up Panasonic PV-S4670 SVHS VCR from 1996. Neither VCR has built-in tbc; the tbc was a Toshiba RD-XS34 or Panny ES10 used as pass-thru. The capture card was an ATI AIW 7500 Radeon AGP. The first sequence was capped in 2007. The cleaner sequences were capped in 2011.

Sequence 1A: three consecutive frames (PV-9668, RD-XS34 pass-thru):
http://www.digitalfaq.com/forum/atta...1&d=1453220486

Sequence A2: the same three consecutive frames (PV-S4670, ES10 pass-thru):
http://www.digitalfaq.com/forum/atta...1&d=1453220575

I should mention that curvy "trees" in the images were designed that way. It's a surrealistic dream set. The trees don't "wiggle", either. Side borders are solid and straight.

Sequence B1: non-consecutive frames (captures 6 frames apart): PV-9668, RD-XS34 pass-thru:
http://www.digitalfaq.com/forum/atta...1&d=1453220644

Sequence B2: the same non-consecutive frames (captures 6 frames apart): PV-S4670, ES10 pass-thru:
http://www.digitalfaq.com/forum/atta...1&d=1453220739

The biggest improvement was switching players. The different pass-thru's were effectively identical. I think you can see why certain hardware and procedures are recommended in these tech sites, and why some components are not. BTW, I also own a Panasonic AG-1980, but this tape did not track quite as well in that player and the old tape's heavy noise level resulted in some lightly smeared motion from the VCR's aggressive dnr. Sometimes a pricey prosumer player isn't the right tool.

Again, thank you for taking the time to explain all this to me.

What a difference a good VCR can make. I checked the local ebays but, good VCR's do seem to come by but theyre quickly sold (same day) for around $65. As for the Diamond and ATI USB capture cards; those don't seem to be available in the Netherlands. Ebay germany won't ship them to The Netherlands.

Anyway, I ended up with a horizontally stacked sample. Left side bt878, Right side Hauppauge.

Not sure yet which one I like more. It's sharpness vs. herringbone now :)

Here's the AVS script I used, using samples from lordsmurf's AVS script and additions of my own.
(Hope I did everything right.)

And the unfiltered AVI (attachment, stacked only, no NR or deinterlacing)

Looking at the frames side by side closely it looks like the herringbone is also in the Bt878 capture, but blurred out.

If you watch the top left corner of the left Bt878 recording you also see the scan line sync problem playing up. The Bt878 just seems to crop differently.

Would you agree that the source for most of the distortion and imperfection therefore is the VCR and tapes I'm using?
(Not the hauppauge capture card)

The USB-Live2 seems to be roughly the same as the VC500, so if this device is the same as the USB-Live2, don't bother trying to acquire the VC500.

Does this VCR have an RCA composite output, or are you using a SCART adapter?

These two statements contradict one another: how would you send an analog input to a DAC?

Analog sources are converted to digital, deinterlaced if necessary, and scaled to fit the digital pixel grid before being sent to the display. If analog was simply passed through, you could show "any" resolution natively via VGA on an LCD monitor like with CRT, but this isn't possible.

Yeah the VCR seems to be the main issue now that would need to be tackled to see if I can get more out of my tape material.
It's a SCART adapter to RCA, I'm by the way still trying to borrow a S-VHS deck with a proper S-Video/Composite output somewhere.

@JasperE: Thanks again for working up these samples.

I don't see herringbone in the bt878 samples. I see a mild excess of interlace combing compared to the hauppauge, but that's no surprise and can be fixed. Tape noise seems more evident, but it would take more than one tape to form conclusions about that. Neither capture device is anywhere near perfect, but you certainly don't need the Hauppauge's RF hash/herringbone which is clearly visible in both the processed and unprocessed samples.

Yes, capture cards do use different areas of the frame. Some use borders, some don't, and borders differ in size.
You don't have to watch the corners of the bt878 sample to see line sync errors; they're visible all over the frame, but not nearly as severe as with the Hauppauge. Lordsmurf is pretty well informed about most capture cards, maybe he or someone can chime in with detail about why the two devices don't show sync errors the same way. Both devices show slight vignetting in the camera lens and geometric distortion at frame edges, especially during zoom-in -- not unusual in most cameras.

Yes. But the Hauppauge card does have serious problems of its own. Better cards are available, but I'd say that if you don't expect more than the BT878 can deliver, then the BT would suffice for your needs. If you find a good buy in a used Panasonic ES10 or ES15 for pass-thru (the same models were sold in Europe and the USA), you could improve performance from the BT card.

Good show, and it does the job. There are a few things you don't need here, a caution or two, and some tips on how to tighten that script a little.

Your input is already YUY2. So this statement isn't needed.

SelectEven() as used here discards 50% of the video's temporal resolution. Motion often won't be as smooth as interlaced, and panning or tracking shots will display judder especially on big screens or slow displays. Interlaced 25fps video updates the displayed image 50 times per second; progressive 25fps video displays each image for twice as long, changing 25 times per second.

PAL DVD is usually interlaced, but it doesn't have to be. Some DVD authoring programs will balk at progressive input insist on re-encoding the video in its entirety. Standard definition BluRay for BD disc or most AVCHD disc/HDD standard-def formats must be interlaced. Progressive standard-def BluRay disc is not allowed.

If you have other formats in mind such as mp4 MPEG2/AVC.h264 for USB or hard drive playback, you have more leeway but you'll need a BluRay player or smart TV.

The stab() filter has its uses, but it won't help here. For one thing, look at the date/time stamp in the lower left of the processed videos where you can see Stab() shifting the characters from side to side.

This conversion isn't needed unless you have specific work to be done in YUY2. I would avoid randomly toggling back and forth between colorspaces when not needed. Each conversion accumulates loss through rounding and interpolation errors. Having gone from YUY2 to YV12 and then to RGB, I'd suggest two choices here: either RGB for VirtualDub ot NLE filtering (in which case you don't need a conversion, since MyColorMill converted to RGB already) -- or return to YV12 for encoding. VHS uses a form of YUY2 or YUV, RGB is often used for color work and many filtering apps, but MPEG and h264 encode to YV12.

Your script works as-is. But you know how people always have their own coding tweaks, so I cane up with the following. It ran correctly in my test, using different AVi videos. This mnethod might save some typing:

I suggested the Diamond VC500 because over the years I've seen halfway decent performance from them in forums. Never saw herringbone or other oddities in a VC500 the way they appar in the O.P.'s Hauppauge. The Live-2 could just be a problematic sample of the product.

My apologies for the contradiction, I used an oversimplified explanation in the same way it was presented to me. Thanks for posting clarification. It would be a great boon to digital video if it could be as free from processing artifacts as the analog-to-digital conversion on my HDTVs, even if visibly imperfect to my eyes. But the point I wanted to make is that any device that sends light waves (analog) or sound waves (analog) via vibrations (analog) in the atmosphere between a TV/loudspeaker setup and the viewer's eyeballs or listener's eardrums is sending an analog signal. No human eyeball or eardrum can detect or decipher a stream of 0's and 1's. So at the output stage there is a DAC somewhere. I appreciate learning about the internal analog conversion to digital, which makes sense because LCDs can't change pixel size but CRTs can....but disappointed to find that analog input is subject to the same loss through digital rounding as noticed in audio CD's and in movie houses that have downgraded from film to digital. Poor LCD-TV motion handling aside, no wonder I kept feeling something was not quite right with straight analog into digital TV.

But that does tend to pull the rug from under pat explanations of the claim that analog cameras and vcr's "play better" directly into digital TVs that can accept the sources. Frankly I never shared that opinion myself. So I stand corrected for accepting a popular story at face value.

Thanks for the scripting example. Nicer that way.

I saw horizontal 'bands' of varying unsharpness on the bt878 capture when I zoomed in.
A herringbone angle would show up at the same height as one of these bands on the Bt878 capture.

If I add a VDub Internal Smoother filter with threshold 30 to the Hauppauge capture, it looks like the Bt878 capture (sort of). Check out the attachment to see what I mean. Right is still the Hauppauge capture, but smoothed.

Added it as x264 this time to save some space on the forum server :)

I'll try a capture with another VCR when I get my hands on one to be sure.

Herringbone or RF noise is a crosshatch or serrated pattern but comes in other forms, like radiating vertical bars. You shouldn't have to deal with a capture device that has noise that obvious.

So I connected my Wii to the Hauppauge, not sure why I didn't think of that earlier, not much heringbone to see now :laugh:

I think the VCR probably needs replacing.