For VHS to DVD, mask (crop) before other restore filters?
 

I am capturing and restoring VHS for archiving, transfer to DVD and streaming purposes. I'm using Virtualdub for my work so far. I think I understand how to mask (crop) my captured Huffyuv compressed AVI files based on directions and advice from http://www.digitalfaq.com/forum/vide...erly-crop.html . However, I am not sure what stage of the restoration workflow is best for doing the masking task. I suspect that, because of the way some of the filters must work, the masking should be done at the beginning. Is this true or does it depend on what the other restoration steps (filters) are? Is masking the VHS tape artifacts the natural 1st step after capture? Thanks!

You crop at the end of the filters chain, always (especially if it's interlaced footage) and add black bars if it's for dvd
ex: NTSC Video resolution: 720x480 > filters > crop -16 (resolution 720x474) > addblackbars 720x480

When you can crop depends on the filter chain. But to be safe, do it last.

Masking techniques vary
- Avisynth = crop + add black = mask
- VirtualDub = resize, letterbox/crop to size + crop = mask
- Premiere = clip = mask

Different steps and names to accomplish same end result: black covering of noise.

When interlaced, always do in 2-pixel increments.

Thanks for the help.

I am just getting my feet wet and I have gained a great appreciation for the depth and breadth of the knowledge and experience of the people on this site.

I have pulled together a JVC SR-V10U, TBC-100 followed by a Toshiba DVR670 (pass thru) and ending up in an AIW9600 in an XP based AMD 3200+ computer. (I also have a Panasonic AG-DS555 editing deck with DNR and TBC. Only 300 hours on the drum but I am going to sell it since my JVC SR-V10U works well and can play SP and LP tapes.)

Capturing with Virtualdub is working nicely with few, if any, dropped or inserted frames even from a really bad VHS tape in LP mode. This is an amazing improvement over just using a bare VCR that would give me 10's of drops in just a few minutes not to mention the improvement in picture quality/stability. Just now I am working on adjusting the capture levels using the Virtualdub histogram.

Next I am working up to using Avisynth with Virtualdub for more picture enhancements. Sounds like fun.

Besides a quantity VHS tapes I have quite a few MiniDV's and some 8mm film to convert.

As a retired engineer I am enjoying the steep learning curve for all of this. Great stuff.

Thanks again and I'm sure I will be back.

Why are you using a pass-thru if you already have a line-tbc in your JVC player and a frame-tbc in your TBC-100? The pass-thru sees no line or frame errors and adds nothing to your capture chain except an extra connection.

Also, the TBC-100 is a frame tbc that precedes your pass=-thru unit, usually used as a line tbc and whose first function is line correction followed by frame sync correction. A line-tbc should always precede a frame-tbc, not the other way around. If a frame tbc precedes a line tbc, the line tbc sees no line error and thus corrects nothing.

You will get better results from LP tapes with the Panasonic than with the JVC. JVC never catered to extra performance processing for LP.

Thank you for your reply.

You are right about the Toshiba DVR670 as a pass-thru especially since it doesn't have TBC etc. As you probably know the DVR670 is a combi VCR/DVD unit. My original plan was to use this as a DVD burner benefiting from the upstream JVC/DNR/TBC and the TBC100. I have used it to burn a disk while I am doing captures via my AIW9600 just to compare results but you are right and I may take it out if there is no other benefit.

I am pretty sure the Panasonic AG-DS555 only plays SP recordings. Nice deck though. It is very tempting to keep it since it has low hours (c=900hrs, d=300hrs, which is low isn't it?). I even bought the XLR connectors for the audio. Unfortunately my wife is not so inclined:wink2:.

I was just goofing around adjusting the capture levels of the AIW9600 via Virtualdub. The histogram showed a lot of red on the tails. I am just adjusting the Brightness and Contrast to fix that. Is that correct?

Thanks again.

Perhaps you should ask your wife (as I did mine, a while ago) what she would suggest doing with your remaining tapes if your JVC should develop problems.

You should adjust the histogram after temporarily cropping off any black borders as well as head-switching noise using the "Crop" window, to prevent affecting the histogram with non-content elements. Don't forget to turn off cropping (set all crop values back to zero) before starting the capture. And yes, the idea is to keep most content out of the red zone.

So far the VHS tapes I am capturing are in great condition. I can clearly see the benefit of the DNR/TBC from the JVC deck and the TBC-100 but beyond that I am not sure what else I can or should do to improve the quality.

Are there some filters or other restoration steps I should take as a matter of course for VHS tapes to improve the quality?

VHS captures always need extra work, even for minor problems, since VHS is inherently imperfect and has many common defects even in good condition. But without specific problems or questions, it's impossible to give details.

It's best to get advice if an unprocessed sample is posted. The sample should include motion of some kind (someone walking, moving, gesturing, etc.) and the sample should not be re-encoded. Short samples in YUV of 8 or 10 seconds are well within the file size limit for posting samples directly to the forum. To avoid any processing, color conversion, or recompression, edit your cut in VirtualDub and save it using "direct stream copy" mode (found under the "Video" dropdown menu). Please do not post to YouTube or other download sites.

Thanks in advance for taking a look at this sample. I hope I sent this sample correctly per your instructions.

This sample2 may be more representative of how I set the brightness and contrast using the histogram in VirtualDub.

Nice sample. Thanks for posting. Looks as if Ernie's getting the worst of it.

A few minor issues and tape noise, but nothing difficult. Let me have a look overnight and report later.

Great! Thanks!!

Thanks for your patience. Time now to see how picky you are, and suggestions in various categories. The sample looks a little soft -- no surprise, considering interior light and wide-open lens apertures, but nothing to worry about. Some subtle midtone changes can improve perceived "sharpness" if it's important. In any case, avoid trying to oversharpen these types of soft-focus images -- oversharpening can only make it look worse and really ugly.

Your sample is a very clean capture and well handled, even if the brightest specular highlights are about 5 or 10 RGB points off spec -- something that's tricky to control during capture, especially with typical home lighting. But it's so easy to fix I mention it mainly because its fixed all the time for occasional mild overshoot. It's easiest and simple to fix in Avisynth and YUV, a little more complicated in VirtualDub. Vdub's gradation curve filter is good at this sort of thing because it can target very precise luma areas. In Avisynth I did it with Avisynth's "Levels" function to gently roll off values above y=235.

Other problems were mild, but of course they do add up. It depends on how "digitally clean" you want for results. There is minor but visible tape noise, more obvious in the red chair in the lower right and in the dark upper right and left shadows (it looks like subtle "horizontal rain" and a little chroma flicker). it can be fixed in either Avisynth or Virtualdub with temporal smoothers. Resharpening would be needed, but most sharpeners maked edge artifacts and will amplify grain. I chose to resharpen with Avisynth's carefully designed plugin, LimitedSharpenFaster, which avoids many unwanted effects. More annoying is excessive combing artifacts during motion, which required Avisynth for repair and is a common problem with home video cameras. There's some chroma shift and mild bleed, seen in the red chair -- easiest to fix in Avisynth but not likely to be 100% perfect, considering the way VHS usually mishandles red. At this point I'm bypassing Avisynth code details unless you want to get into it. Some fixes are best done with Avisynth, others in VirtualDub. Each app has its limits and advantages.

There is a DeShaker filter in VirtualDub that stabilizes most forms of camera tilt and jiggle. I didn't use it here. DeShaker works well and is highly adjustable but can cost some image real estate after new borders are rearranged and cleaned up. There are 4 pixels of head switching noise along the bottom border. You can mask 4 pixels in VirtualDub; I elected to use Avisynth to crop off the bottom 4 scanlines, then added two black lines each to top and bottom borders to center the frame vertically without affecting the original image.

Easiest to deal with is a slightly dim appearance because of the lighting. This was fixed in VirtuaLDub by raising targeted midtones with VDub's colorMill and gradation curve filters. Note that "brightness" won't work here because it would wash out the darks. "Contrast" won't work, either, because brights are already at peak levels. So raising midtones and saturation a bit did the trick.

The attached mpg tweak sample was encoded for DVD with MPEG2, which IMO handles interlacing and subtle textures more accurately than h.264. Also attached is an annotated Avisynth .avs script I used for cleanup and running in VirtualDub. An .avs file is a simple text file you can read in Windows Notepad or any word processor.

I wish I got to work with source files this good...

When you talk about combing, and fixing it with Vinverse, are you saying the QTGMC deinterlace isn't fully working?

So do I. :depressed:

QTGMC isn't used to fix residual combing artifacts, but it can be helpful for some aliasing and mice teeth edge problems.

Since vInverse deals with smoothing out luminance fluctuations in some manner, and because these are part of sloppy interlacing and the cause of residual combing problems, I used the Inverse2() function of the plugin which is sourced from some code by didee at doom9. If you want to know exactly what didee was doing his original script is here: https://forum.doom9.org/showthread.php?p=841641. The source code that comes with the vInverse plugin would tell you how it's implemented in the vInverse .dll. I don't design filters, I just use them according to the ways designers tell us to. For example, I wouldn't use vInverse to repair aliasing or mosquito noise. You have to experiment with its settings at times, even though the defaults usually work, because stronger settings can cause more obvious blurring.

Wow! I am very impressed! Your insight into the shortcomings of this sample is amazing and a lot for me to absorb. Thank you.

I may be more curious than picky but the result is the same. I am going try to duplicate your work as an example to help me move up the learning curve for the mechanics of using Avisynth with Virtualdub.

My other tapes may not be so good. This one was probably the first recording with my new VHS camera 25+ years ago. I'll bet things degraded over the next 10 years or so.

What tool did you use to measure the "5 or 10 RGB points off spec"?

Thanks again.

My understanding is that 'combing' is what happens when you view an interlaced video on a progressive display. Deinterlacing should eliminate this. Am I misunderstanding?

I eyeballed the frames using a Levels histogram in Avisynth and an RGB histogram in Virtualdub. It was just a mild overshoot, common and easily corrected. If you wanted an exact measure in YUV you would use an Avisynth function called Analyze, which overlays a table of numbers onto the frame. The coded line in a script would be would be:
If there are borders or switching noise in the frame, they should be cropped to avoid throwing off the numbers. But in this case the levels were well controlled so this wasn't necessary. Most of the overshoot was the lamp in the background.

Avisynth looks intimidating at first, but you tend to use the same filters and techniques most of the time. If the tape is in bad shape, as most of my 1990's tapes were (nightmares, really) you'd use heavier filters.

Let us know if you have problems with Avisynth or VirtualDub plugins. Fortunately many are built-in or easy to find. Now and then you get the maverick that doesn't seem to exist anywhere. I hate those.
:mad4:

What do you use to play and view files when you are making critical assessments about restoration steps etc. Do you just open up VirtualDub in full screen on a computer monitor or something like that?

I am working on the installation of the plugins for QTGMC. I think I am running into version issues since my AMD 3200+ does not support SSE2. I am getting Kernel32 issues when trying to open an .avs script in Virtualdub. I noticed that several of the plugins are ____SSE2.dll

I just use VirtualDub as-is. I seldom have to magnify the input or output pane (right-click on the pane you want to adjust to get a menu). Resizing the panels results in distortion beyond a certain point, for example aliasing. Also, remember that Virtualdub is an editor and frame server client, not a media player, so video won't be as smooth as it is in Media Player Classic. I run Virtualdub resized to almost fill the 16:9 screen, but the top and sides have a few pixels clearance in case I need something just behind the window. My desktop itself is a solid panel, not fancy wallpaper, in a neutral color to prevent affecting color balance (in this case my desktop's solid color is set at RGB 32, which isn't quite black). Lighter shades pf gray were too distracting. The dark color also helps 8 rows and 5 columns of left-hand desktop icons to stand out visually. To prevent overpopulating the desktop I have many icons categorized inside desktop folders.

When running a script I have two running program icons available in the bottom taskbar: VirtualDub and the current script. I can click on either to change the focus back and forth. You can observe changes in a script by stopping it in VDub, changing the script, and then clicking "Reopen video file" or pressing F2 -- either action will repaint the current frame. You probably notice that a script runs slowly with heavy-duty filters like QTGMC or many others; you can speed things by commenting-out the line(s) with the slow filters that you don't need at the moment. An example of some original script passages that have been commented-out:

The code portion above is of course a complete mess, but you can see that "#" has been used to disable some lines at certain points. Also, note how many times "return last" appears at certain points. When not not disables by a comment mark, "Return last" stops a script at that point and returns the last step that was executed. In the case of the script above, the script will stop just above the line that would have executed QTGMC.

I could also have allowed the script to continue, but without QTGMC running, as shown:

Normally, of course,m all these extra lines get cleaned up for the final script.

All AMD CPU's since Athlon64 support SSE2, which goes back quite a long way.

When you see Avisynth or Virtualdub errors running scripts, post the entire text of the error message or make a screen capture and crop the image down to the message panel. Often it's also necessary to post the script.

Another suggestion: if you have a filter package that includes two versions of a plugin (an SSE2 version and a non-SSE2 version of the same name), don't install both dll's. Don't install SSE3 versions at all. Note that some filters come in two versions, one for AMD and one for Intel (although that involves optimization only. It's unlikely that an AMD version would fail on an Intel machine). Often the complete error message gives further clues about which plugin is involved, in which case it might be necessary to check versions or creation dates. Many kernel errors are due to missing Microsoft VisualC++ runtime files or the ubiquitous fft3d system library errors -- if those are needed it's usually indicated in the documentation for the filter.

Sometimes you hit a quality ceiling. I need to look at your samples...

... and done. You mostly have some loss from the camera, which can't be fixed. A proc amp could help, but this is probably a compilation/multiuse tape, meaning the values probably change every 15 minutes. I never mess with footage like that, unless isolating a specific clip for a specific purpose.

A detailer may help as well. The sharpness loss is correctable in analog domain.

Oh, I don't know. Sometimes you hit diminishing returns, where attempts to further fix it being to degrade it. I find that to especially be true with my own SP recordings in the JVC deck to JVC recorder (which corrects chroma noise, and mild grain, leaving almost zero flaws). The JVC files could be further "fixed", but it would be a nitcpick.

Thanks once again for the great help and advice!

I am continuing to deal with preliminary setup issues. I can open a simple Avisynth script (Version, Stacking video...) in Vdub with no problem. I have downloaded QTGMC and the required plugins and installed per the instructions contained therein. When I attempt to open the scripts that you have provided (thank you) I get a Vdub error: "Avisynth open failure: LoadPlugin: unable to load C:\Program Files (x86)\AviSynth\plugins\RemoveGrainSSE2.dll, Module not found. Install missing library?" This is a call from QTGMC line 393 so I am inside that function.

I know that I have this dll in my "C:\Program Files (x86)\AviSynth\plugins" directory. Do I need to have some kind of LoadPlugin statement in my script or a "Path" statement somewhere?
Kinda stalled out on this.

BTW My XP machine really does have a really old AMD Athlon XP 3200+ processor that only uses SSE instructions. The problem I describe above is the same on my Windows 10 machine as well though.

There are a couple of things you can try (although I find it curious that your Windows 10 OS gives the same error). Attached are the non-SEE versions of RemoveGrain and associated dll's. Download the .zip file into itsown folder, unzip, and copy the dll's into your plugins folder. These are non-AVS-26 filters, by the way, for Avisynth 2.5 ()I'm using them with Avisynth 2.6). They should still work, but if they don't you can download and install the newer RGTools here: http://avisynth.nl/index.php/RgTools. You can have the RemoveGrain and the newer RGTools.dll in your plugins at the same time. Note on the RGTools download page that the package requires MSVC++ runtime for 2015, with a link for the installer.

If you still have a problem, it's likely related to MSVC runtime files. Microsoft has versions of these from 2010 on. Normally they come installed with SvcPak updates for XP, but if you're using SVCPak2 you might not have them. I don't agree with this preference for SVCPak2. Many filters require MSVC support files that won't install in SvcPack2.

Let us know how these fixes go.

Runtime packages helped thanks.

vInverse2()?

Vinverse and vInverss2 are functions in the latest version of vInverse.dll. They have very simialr effects. I thought vInverse2 looked a little smoother than vInverse in this case. Both are masking/blurring/overlay functions normally used to reduce visible remnants of excessive combing artifacts, especially from home cameras with their sloppy interlacing.

http://avisynth.nl/index.php/Vinverse

Ooops, in my rush the other day I forgot to include some notes on the VirtualDub filters I used for the posted
Sample2_Tweak.mpg in post #13 (http://www.digitalfaq.com/forum/vide...html#post48061). I posted the .avs script but forgot the VirtualDub filter settings.
:smack:

VirtualDub filter settings can be saved in a .vcf file. To save the settings, click "File..." -> "Save processing settings", give the file a name and location, and click ":save" or OK. To load the .vcf and exactly the filter settings used, click "File..." -> "Load processing settings", and navigate to the .vcf file location. Note that loading a .vcf file will overwrite any VDub filters you've already loaded. A .vcf is a simple text file that you can open in Notepad for a look. It can be edited, but you'd better know know what you're doing. Save the original copy if you try making any changes. With the filters loaded you can open the filters dialog and see the settings.

The .vcf file I used is attached as Sample2_tweak.vcf. You must have the .vdf filters installed in your VirtualDub plugins. The three filters were:
- temporal smoother (no problem -- it's a VirtualDub built-in filter, so you already have it)
- ColorMill 2.1 (download here: http://www.digitalfaq.com/forum/atta...colormill21zip)
- Gradation Curves 1.45 (.zip'd .vdf filter and html attached)

Some time ago in an earlier post I wrote up a brief tutorial with graphics on how to use some of the filters. The filters covered in the post were CamcorderColorDenoise (aka "CCD"), Levels, ColorMill, and Gradation Curves (aka "curves", similar to Photoshop Pro and others). The post is here: http://www.digitalfaq.com/forum/vide...html#post42315.

To apply the filters on the output of an .avs script while the script runs, you must have "Video" set to "full processing mode". This mode automatically uses RGB and by default saves the output as uncompressed RGB. You can save the output file in your preferred colorspace and compressor by setting "color depth..." and "Compression" to your preference. I saved the output as YV12 with Lagarith compression. Note that huffyuv can't compress YV12. So if you want the popular lossless Lagarith codec you can get it here: https://lags.leetcode.net/codec.html.

Lagarith makes slightly smaller lossless files than huffyuv. It can also be used for capture, but it does eat up more CPU than huffyuv, so I use huffyuv for capture but I usually use Lagarith for lossless intermediate working files.

Good work on getting those Avisynth plugins. If you have any problems, just ask.

Probably better here than bumping that other thread: when using ColorTools, do you check '16-235', or '0-255'? I read the help page and I think I'm more confused by it now than I was before.

ColorTools always reads 0-255. Checking 16-235 is info only, it puts markers at 16 and 235. Remember that YUV input is being expanded from 16-235 to 0-255 in RGB. The setting is used lower in the dialog box where you enable setting hot pixels to black (always use IRE 100, which in RGB is 255 on TV) -- but I don't think I've used that hot pixel display for years. You can tell when luma or chroma is "hot" anyway when the horizontal bars collide against the sides of the histogram.

The markers are there regardless of which is checked, and changing it alters the shape of the luma histogram and waveform monitor.

Yes, that's correct. I should have clarified: The 16-235/0-255 and the iRE settings at the bottom refer to turning "hot" pixelks black, depending on which range you want to check. It has no effect on the markers. I forget that those markers are always there because I'm using 0-255 all the time anyway. I haven't used the "hot pixels" feature in years.

Thank you for the follow up. I now have Avisynth working on two machines. My old XP that I use for captures requires some of the older filters etc in order to get things to run. My newer computer works much better and runs all of the newer stuff so I guess I will mainly use my XP machine for capture. It works fine for that.

I have a tape that is SLP and is most certainly a copy. It is the worst I have to work with I think. I am curious what can be done to improve the quality of this recording if anything. I have attached two files just for grins: one is played/captured with a JVC sr-v10U and the other with a Panasonic ag-1980.

I am more interested in the restoration process/filters but curious if the two decks make any difference.

I feel like I am asking a lot but if you can help me with this it will be greatly appreciated.

Thanks for the samples. There are obvious differences between the two.

I'd prefer to work with the AG1980 sample. Its play of the tape is a more honest performance with visually less noise, less distortion, and less highlight burnout than the JVC sample. One problem with the JVC version that you likely won't be able to fix without massive detail destruction are the blackish vertical lines and comets that spark off the edges of bright objects. The effect is known as "bearding" and is usually due to video heads in poor shape. The AG1980 has better noise reduction, but I doubt that even that would be good enough to clean bearding artifacts if they exist on the tape itself, so I'd say the JVC's heads are the problem..

The JVC has more obvious oversharpening artifacts, less stability in thin edges, and brighter and sharper edge halos than the AG1980. Both samples have levels problems, notably with the JVC pumping contrast beyond legal values. The JVC version looks oversharpened and rather "etched'. The AG1980 version has legal levels but black levels are a bit high and make the image look a little washed out. Lowering black levels will fix that. The JVC version requires a considerable reduction in contrast and a lowering of the resulting black levels to avoid most of the artificially enhanced appearance. But sadly the JVC is also stuck with the annoying edge noise mentioned.

Both samples have their faults and virtues. I feel that the AG1980 has fewer problems.

Of course, a lot depends on the tapes and how they play nice (or not) with a particular player. This is why most advanced members here maintain more than one VCR (I have 4). I'd try cleaning the heads on the JVC to see if comets and bearding can be prevented.

Thank you for taking the time to analyze these samples. I am not sure I knew how to use Vdub histogram to set the levels for the JVC capture including cropping before doing so. The AG-1980 I tried to set the levels at least nominally.

This is an SLP recording. Is it possible that the difference between the two samples is the deck performance for the slow speed of the tapes (or the level set?)? Note that the JVC is the same deck that was used to capture the samples in the early part of this thread ( again I am extremely grateful for your help and guidance for processing these).

I am most interested in using the Ag-1980 going forward. What would prescribe to improve the quality of that sample?

The VDub capture histogram and pictures of what it looks like appear in post #3 of the later VirtualDub capture settings guide (http://www.digitalfaq.com/forum/vide...html#post45238). A pictures and notes on the levels dialog and the crop window are in post #4 (http://www.digitalfaq.com/forum/vide...html#post45239).

A tape will usually require different level settings when played in a different VCR. Different tapes played in the same VCR will usually require different levels settings. The same tape usually will not play exactly the same way in different VCRs.

Yes. Excellent guides. Thanks.
I usually adjust levels for each tape also.