First of all let me say thank you to this forum for all the knowledge it contains, I've been reading it for the past month and it was invaluable. I'm making this post after realizing that my almost 15 hours of captures were wrong because of an incorrect capture resolution, as per sanlyn's post here (thank you mate, I made a mess but now I have some time to recapture those cassettes).
Since I'm starting capturing again, I wanted to share my process with you and ask for your comments in order to avoid huge mistakes on my part.
Context:
After much searching, a friend lent me a camcorder to digitalize my family's 25 years old "8" and "Hi-8" cassettes. I must give the camcorder back before christmas, and given that I work full time, there is not much time. I have 27 cassettes to digitalize, mostly SP but some LP, so about 40 hours of videos.
Setup:
Camcorder: Sony Handicam Digital8 DCR-TRV230E. It has an integrated TBC that I have set ON. Cassettes should be in PAL format (southern Europe)
Grabber: EZGrabber2 USB
Connections: pinjack for audio, S-Video for video
PC: 2009 PC, but pretty powerful (2.5GHz quad core CPU, 12 GB RAM). The PC is connected to an LG 47 inch TV, not a monitor, so I don't know if it has square or rectangular pixels.
SO: Recently installed Windows 7 64 bit with pretty much nothing installed on
VDub: VirtualDub 1.10 64 bit with Huffyuv 64 bit codec (non multithreaded), x264, x265, xvid
Objective:
I want to archive all the cassettes in a lossless format (Huffyuv), one file per cassette, without dropped or inserted frames (aka without missing pieces). I will make 2 copies of each cassette for backup purposes.
For every cassette I want to extract all the different clips contained (for instance: birthday 1992, christmas 1992, summer 1993, etc.), save them in Huffyuv, and make a second copy for backup. Since Huffyuv stores all frames independently, I should be able to extract the clips just by cutting in VDub and saving with Direct Stream Copy.
For every clip I want to create a compressed version to watch on PC, and for every cassette (or clip collection) I want to produce DVDs so that my parents can easily watch them.
Lossless capture process:
Unplug LAN cable, close firewall and antivirus software
As per sanlyn's post I will capture at 720x576, 25 fps, YUY2 colorspace
Disable the preview pane and the audio playback in order to minimize dropped\inserted frames
I have set the recording volume input to 5% in the Windows recording options because otherwise the sound would be distorted
Timing options attached
Compression options attached
Sample capture attached
Lossy compression process:
I still don't have a process in mind for lossy compression. I know how to compress a file in VDub but I don't know how to make a playable DVD. I would like to mainly use open and widely used formats. I would prefer to use VDub to do everything, but I can use other software if necessary (Handbrake for instance); I have a linux machine too if needed.
What container should I use for the PC-readable clips? MP4 \ MKV? What resolution? What compression algorithm (I was thinking x264)?
How could I create a DVD? Is it sufficient to compress the files in MPEG2 and burn them on a DVD?
Problems so far:
All of my videos have a small green vertical line on the right, and an horizontal interference line at the bottom (see sample attached). Is it because the camcorder and the cassettes are old? Is there something I can do to eliminate those in capture?
Depending on the cassette, I can have some dropped frames and a lot of inserted frames. In some cassettes it's around 25 per hour, which could be acceptable for single frames, but in others they are in the hundreds, with the substitution of entire scenes with blue screens. I was thinking about making 2 copies of each cassette and then substitute the missed frames in one copy with the corresponding frames in the other by cutting and pasting in VDub, but it's a tedious process. What do you think?
Capturing at 720x576 stretches the images; sanlyn's explained how that is changed in playback by setting the DAR (Display Aspect Ratio) to 4:3 or 16:9. I guess that since these videos are so old I should see them in 4:3, right? Is there a way to tell VDub to embed the correct DAR in the video file, so that I do not have to manually set it in VLC?
I'm using YUY2 colorspace, but I had to set it in the custom format options because in the normal format options panel I could only set UYVY colorspace without the preview pane freezing. Am I doing it wrong? Is there a regional difference like with NTSC and PAL?
Since I have an external grabber I tried disabling the "Correct video timing for fewer frame drops\inserts" option in the Timing options panel for a test. In that way VDub doesn't compress\extend the audio to match the video. In the capture the two streams are synchronized, but I have noticed heavy audio distortion in some parts of the capture (2-3 seconds of audio very slowed down). Maybe it's better to let VDub handle a\v sync?
Thank you very much for any insight you would like to provide
First of all let me say thank you to this forum for all the knowledge it contains, I've been reading it for the past month and it was invaluable.
Most members here learned a great deal in the capture forum, where they usu8ally start. But most skill with post-capture improvements are learned in the restoration forum.
Your capture sample had good levels control and color. There is some tape noise to consider. High-end VCRs clean some of the noise but none of them clean it all. Still, it's a better sample than those we usual see, some of which are nightmares.
Quote:
Originally Posted by cicaesar
I must give the camcorder back before Christmas, and given that I work full time, there is not much time. I have 27 cassettes to digitalize, mostly SP but some LP, so about 40 hours of videos.
Consider yourself fortunate. I started several years ago with 350-plus hours of tapes, and collected another 100 before I went to DV-only recording. Considering the time you have available I'd suggest capturing all the tapes first. Give yourself some time to load those captures into VirtualDub where you can very quickly scan them lickety-split to make sure there are no major errors. I would then work on cleaning and encoding only the most important ones first, or just the older Christmas tapes. Remember that most viewers get a little tired of home tapes after the first 20 minutes, so a couple of short encoded features will probably suffice at first. That will leave time later for overhauling the remaining captures.
Quote:
Originally Posted by cicaesar
PC: 2009 PC, but pretty powerful (2.5GHz quad core CPU, 12 GB RAM). The PC is connected to an LG 47 inch TV, not a monitor, so I don't know if it has square or rectangular pixels.
Today's PC monitors and TV's have square pixels. That's why captured NTSC and PAL 720-width lossless/unencoded AVI's look misshapen -- lossless AVI's have no display aspect ratio flags, so they don't appear as true 4:3 or 16:9 images. After they are encoded at the proper display aspect ratio (DAR), they will display normally. Meanwyhile, if you want to see them as 4:3 is VirtualDub, right-click anywhere on VDub's right-hand output panel and select 4:3 from the choices in the menu that appears. Some, but not all, editor apps will also allow this.
Quote:
Originally Posted by cicaesar
VDub: VirtualDub 1.10 64 bit with Huffyuv 64 bit codec (non multithreaded), x264, x265, xvid
Are you certain you installed 64-bit? Your sample is in 32-bit huffyuv. If the sample was encoded at 64-bits I would not be able to play it on my 32-bit software. If you did install 64-bit VirtualDub it was not a good idea, as version 1.10.x is buggy, especially for capture. Also, there are hundreds of VirtualDub filters but very few of them will run in 64-bit VDub, so your choice of VDub filters is seriously limited.
I suggest that you uninstall 64-bit VirtualDub and install 32-bit VirtualDub 1.9.11 instead. You can uninstall VirtualDub 1.10.x by going to its program folder in "C;\Program Files\VirtualDub" and look for auxsetup.exe. Double-click auxsetup to run it and follow the screen prompts to remove VirtualDub and its registry entries. Then change the name of the Virtualdub program folder to "VirtualDub_old10x". The plugins and filters will still be there.
Make a new program folder for "C:\Program Files (x86)\VirtualDub". You can get 32-bit VirtualDub 1.9.11 at https://www.videohelp.com/download/V...r=SJLcpxpkGCQl. Download the .zip file into the new VirtualDub folder and unzip it there with Win7's unzipper by right-clicking on the .zip file. When the .zip file decompresses double-click on "VirtualDub.exe". This will install 32-bit VirtualDub on your system. From now on, use only 32-bit plugins. 32-bit VirtualDub also has MPEG2 and other format input plugins available. Later you will likely have to install 32-bit Avisynth and 32-bit plugins.
The versions of huffyuv you used are likely 64-bit and 32-bit installs. In your post-capture processing you will also need the 32-bit and 64-bit versions of the Lagarith lossless codec. You can get Lagarith's handy all-in-one simple installer at https://lags.leetcode.net/codec.html.
Quote:
Originally Posted by cicaesar
Since Huffyuv stores all frames independently, I should be able to extract the clips just by cutting in VDub and saving with Direct Stream Copy.
Correct.
Quote:
Originally Posted by cicaesar
For every clip I want to create a compressed version to watch on PC, and for every cassette (or clip collection) I want to produce DVDs so that my parents can easily watch them.
DVD's are universally playable and so is the MPEG format. You can use your MPEG encodes to play on your PC and to author DVD discs. Note that encoding and authoring are two different steps. MPEG can be played on PC's, smart TV's, video servers, etc. The same MPEG videos are also input to DVD authoring programs, which rearrange MPEG's into MPEG-encoded VOB containers for authoring and burning to DVD disc.
Quote:
Originally Posted by cicaesar
Depending on the cassette, I can have some dropped frames and a lot of inserted frames. In some cassettes it's around 25 per hour, which could be acceptable for single frames, but in others they are in the hundreds, with the substitution of entire scenes with blue screens. I was thinking about making 2 copies of each cassette and then substitute the missed frames in one copy with the corresponding frames in the other by cutting and pasting in VDub, but it's a tedious process. What do you think?
You shouldn't encounter many dropped/inserted frames. While you do have a line-level tbc in your camera, you apparently don't have frame-level sync tbc and that's why you have dropped frames. There are pass-thru units that solve that problem, but you won't have time between now and Christmas to set one up. Unfortunately a tbc can't do anything about blue screens -- those are a camera or player fault and result from badly damaged tape.
At this point your best defense against dropped frames is to use a different Timing Option setup. The Timing Options image that you posted earlier doesn't show the options recommended for USB capture devices in the newer Settings Guide. The settings that you used inn your posted image tell VirtualDub to manage frame timing, which isn't always a good idea. Below, this is what is usually recommended for USB capture dongles:
Timing Options for USB:
Turn off Audio playback, but you should be able to use Preview rather than Overlay for viewing during capture. Preview might skip a frame now and then but it doesn't affect the capture. If you want to verify that audio is actually going through the system, click "Audio" -> "Volume meter".
Quote:
Originally Posted by cicaesar
Maybe it's better to let VDub handle a\v sync?
Almost all USB capture devices handle their own a/v sync. Usually, letting VDub do it just interferes with the device.
Quote:
Originally Posted by cicaesar
How could I create a DVD? Is it sufficient to compress the files in MPEG2 and burn them on a DVD?
Although MPEG is one of three codecs allowed for BluRay, MPEG is the only codec allowed for DVD. No, you can't just burn MPEG files to a DVD disc. A DVD-video disc has to be authored. that is, authoring programs accept MPEG files and add menu information, all of it rearranged into VOB containers for burning to DVD disc, which is a specialized disc format. You can also play unauthored MPEG on PC media players or burn them to optical disc as "data" discs and play them on many set top media players, although most such players would rather see MPEG data files on USB flash drives or external hard drives.
There are free and paid encoding and authoring programs. One decent budget encoding amp for DVD,BluRay,AVCHD and several other formats is TMPGEnc Video Mastering Works. They also make a decent budget authoring app, TMPGenc Authoring Works. TAW6, as it is sometimes called, will also take lossless AVi as input and encode to MPEG2 for DVD, set up interactive menus, burn to single-layer or double-layer discs, and do the same for Bluray and AVCHD. But if you want more flexibility for various output formats, you're better off with separate encoding and authoring apps, because Authoring Works only outputs for authored discs. TMPGenc's deal is that if you buy theb encoding app you get a discount on the authoring app.
Another route that people take is an all-in-one budget MPEG encoder, authoring, and burning app such as Corel VideoStudio Ultimate X10 (2017 edition for Windows 7, 2018 only works on Win8 and 10). We would not recommend Cyberlink software, Adobe Premiere Elements, or off-brands.
There are free apps around, usually involving a learning curve and rather basic features. Handbrake is not an especially favored app and it doesn't encode MPEG. A favorite freebie is AVStoDVD (https://www.videohelp.com/software/AVStoDVD) which uses the well regarded HCenc MPEG2 encoder for both DVD disc and MPEG "data" files. Reading the online Help is mandatory for all software, but especially for freebies. Most of them don't follow Windows design protocol and can be somewhat less than intuitive to use. You can explore more options later, but most of the time you have more flexibility with separate encoders and authoring apps. A favorite free DVD authoring-only app is DVDStyler (https://www.videohelp.com/software/DVDStyler) -- get the 32-bit version.
Quote:
Originally Posted by cicaesar
All of my videos have a small green vertical line on the right, and an horizontal interference line at the bottom (see sample attached). Is it because the camcorder and the cassettes are old? Is there something I can do to eliminate those in capture?
The bottom border noise isn't "interference", it's head-switching noise common to all VCRs. It's usually masked by overscan, especially on CRT's. It's later masked (using VirtualDub) or cropped off and replaced by new black pixels (using Avisynth). As for border stains: they're usually magenta, blue, or cyan, and can appear on any border. Filtering is most effective if applied early in their original YUV colorspace. It takes a lot of filtering and they never go away entirely. VirtualDubb has no effective filter for them. For stains I've heard a dozen explanations, none of which make sense. In the attached MPEG samples you can still see remnants of the green stain.
I'll be posting some methods for cleaning and encoding clips. Below, the first effort at color-correcting your sample. The top image is an original frame (the color balance is far too red, likely due to the lighting), the lower image is the corrected version:
Original frame3, bottom field, 4x3
Filtered frame:
The attached "Sample_DVD_4x3.mpg" video is your original sample, denoised in Avisynth with new borders, then color corrected in VirtualDub, and then encoded to MPEG2 for 4:3 DVD with TMPGenc Video Mastering Works. The MPEG's demonstrate how denoisers have a lot of trouble with frantic camera motion, which eats up bitrate and makes for rather fuzzy encoding.
Some time tomorrow I can post a version that uses only VirtualDub for cleanup. I don't think you'll be able to handle Avisynth before Christmas with all that you have to do.
The other video attachment is "Sample_stabi9lized_4x3.mpg". It was stabilized at about 50% power with the DeShaker filter in VirtualDub -- but look at the odd motion! The reason you won't be able to stabilize these frantic camera motions for better denoising is because of the date stamp in the lower right corner. When the overall image is stabilized by moving it around in the frame, the date stamp is also moved around. That's OK for demonstrating how far and fast the image moves, but it's annoying to see the data stamp jiggling around.
Will post more details later.
And by the way -- No, there is no way to correct conditions where the camera goes out-of-focus.
The following users thank sanlyn for this useful post:
cicaesar (11-30-2018),
lordsmurf (12-01-2018),
videosean (05-06-2020)
All of my videos have a small green vertical line on the right, and an horizontal interference line at the bottom (see sample attached). Is it because the camcorder and the cassettes are old? Is there something I can do to eliminate those in capture?
The green line on the right is an artifact/design flaw in the newer Sony camcorders when playing back PAL video1. It's not present on NTSC, when playing a NTSC tape on a PAL camera. As far as I know it's present on most or all the the newer (late 90s and later) Sony Hi8 and Digital8 camcorders. Haven't seen any TBC-equipped ones that doesn't have it, don't know for sure about the earliest ones. The really old cameras don't have this issue, but the image quality is worse and they also lack the TBC and dropout compensation of the newer ones.
It would not have been visible when using them back in the day since it would be in the overscan area of the image that would normally be cropped off by displays, but it does show up when capturing as you then get the whole image.
There is a thread somewhere either here or on videohelp discussing it but I can't find it right now. I think it's also present when using the av in -> dv out function that some of the D8 cameras have, but I don't remember for sure.
There is an avisynth script in this thread that can mask it to an extent.
The EZGrabber 2 seems to be nearly identical to the Diamond VC500 which both me and Sanlyn use regularly, so the image quality should be good otherwise.
Sanlyn explained the noise at the bottom.
The following users thank hodgey for this useful post:
cicaesar (11-30-2018),
sanlyn (11-30-2018)
Some additional suggestions for home VHS restoration:
With a deadline looming, you likely won't have time to get accustomed to using Avisynth. If you've never used it you would need a full afternoon just to download the Avisynth filters I used for the mpg in the earlier post. Although VirtualDub doesn't have industrial-strength cleaning plugins like Avisynth, it nevertheless has useful filters that can still make quality improvements and has very good color controls that Avisynth doesn't have. So here are some VirtualDub suggestions that don't require Avisynth.
Below, an image of the VDub filters I used for the attached "Sample_VDub_DVD_4x3.mpg", which is encoded to DVD spec. The filters are purposely loaded in the order shown, with color correction followed by denoising and then border repair:
I saved the filter sequence and their settings in a VirtualDub .vcf file. A .vcf is a plain text file that you can download and use in your own copy of VirtualDub. In order for the .vcf to work, you must have the named filters present in your VDub plugins folder. The filters I used are all 32-bit (there are no 64-bit versions of these filters). To open a .vcf file and load the filters and settings, in VirtualDub click "File...", then click "Load processing settings...", then locate and select the .vcf file. The filters will load into VirtualDub's filter chain as shown in the image above. The .vcf file is attached as "VirtualDub_Settings.vcf".
Some details and download links for the filters:
Color Camcorder Denoise (aka "CCD") is a favorite for cleaning rainbows, which are vague color streaks and blotches. Although it isn't the perfect fix for bad border stains -- most of the time you'll need Avisynth for those -- it does make them less noticeable with help from other chroma cleaners. For your sample I used CCD at full power, which can sometimes introduce its own artifacts if you're not careful with some videos. It seems OK here. You can get CCD version 1.7 at http://www.digitalfaq.com/forum/atta...ove-ccd_v17zip.
Chroma Noise Reduction is an old standby that goes by the nickname "cnr". It tends to smooth color differences in large areas such as background walls and sky. Users say that in general it just makes colors look "cleaner". There is also an Avisynth version of this filter. The attached "cnr11.zip" contains the VirtualDub filter cnr.vdf plus a Rich text File (.rtf) version of the original html page, which has long since disappeared from the internet. I used mild settings that cleared the green stain a little more. This is another filter that has undesirable side effects if set too high.
The brightness/contrast filter like most bright/contrast filters -- "brightness" actually controls black levels, while "contrast" controls brights. In this case I gave the image a mild 118% boost. This is a built-in VDub filter.
Color Mill and gradation curves are two excellent color adjustment filters that mimic those found in more expensive editors. Color Mill differs from many color controls in that it uses sliders instead of circles, but many find circular controls difficult to use. Color Mill's interface is shown below:
The left-hand side of the control allows selection of the category of image properties that you want to adjust. The spectrum can be adjusted in dark, middle, and bright ranges separately. How do you get to understand color adjustment? Unfortunately that's a subject that can't be covered in a single post. Most people learn about color work through the hundreds of free tutorials found on photography sites all over the internet. The principles of color for photos and movies are similar. After all, a video is really a stream of still photos.
In the case of your sample, the images have a definite yellow-red color cast, most of which is due to strong reddish side lighting. Most people incorrectly assume that the color opposite to red is blue. But it doesn't work that way. We humans see all colors as a mix of Red, Green, and Blue (RGB). If you have too much of any one of those three colors, it means you need more of the other two colors to balance things. So, if there's too much Red the balance calls for more of Blue and Green, the combination of which is Cyan. I used Color Mill to correct the over balance of Red by raising the values of Blue and Green in the middle tones and brights.
The darker colors suffered from too little Blue, as was evident from the blacks in the floating cork rings in the water, so Blue was raised in the "dark" zone. Also, the "Levels" category was raised in the "Middle" zone to highlight the water and skin tones and to keep the image from looking so dimly lighted. The best way to get accustomed to these controls is to simply use them -- push and pull some sliders, watch the preview pane, and see what happens. It doesn't take long. You can get Color Mill at http://www.digitalfaq.com/forum/atta...colormill21zip.
The gradation curves filter does take some getting used to. It's called a "curve" filter because the diagonal line that bisects the interface can be fashioned into curves and tilts that adjust a more specific range of colors and brightness in very specific ways. The image below shows how that diagonal line is used to make adjustments:
Curves was introduced in two video forums, but it has its own website. There's a nice tutorial at http://members.chello.at/nagiller/vd.../tutorial.html. Without adjustments, the default diagonal line is straight from the bottom left corner to the upper right corner. Changing the shape of that line by adding those little round "control buttons" forms the shape you want. The top of the graph is the brights, the middle is the midtones (which includes most skin tones), the bottom is darks. In the image I added some RGB values along the left-hand side to show which parts of that grid affect which values.
You can even use the curves filter to work in color systems other than RGB, although RGB is the three-color system most people find easiest. Curves has a separate adjustment pane for 4 areas of correction: "RGB" (which adjusts all 3 colors at once for "bright" and "dark"), a Red pane, a Green pane, and a Blue pane. Each can be adjusted independently. The 4 panels of 4 images below show how the initial Color Mill corrections were tweaked by using the curves filter:
RGB (brightness) and Red panels:
Green and Blue panels:
The top RGB left panel shows a darkening of the darker colors to give the image some "snap" and dimension. Red, Green, and Blue have slight adjustments in the brightest portions to keep the date-stamp figures from turning the wrong color. Date/time stamps in videos are big problems; corrections to the image will also apply to the date stamp, which isn't always what is wanted.
A little color correction goes a long way to mask other visual faults and makes a video more interesting to watch. Get the curves filter and its main Help and tutorial page at http://www.digitalfaq.com/forum/atta...1&d=1489408797.
Deinterlacing came next. Temporal denoise filters are more effective when video isn't interlaced -- although you have to reinterlace before encoding for DVD. Virtualdub doesn't have a deinterlacer as cool as Q@TGMC, but it does have an implentation of the "yadif" filter used in many media players. Shown below, yadif is activated by selecting "Interpolate using Yadif algorithm", and the field order column is set to "Double frame rate, top field first". It's important that you use the correct field order. The image below with orange arrows shows the setup:
After denoising, the video will be re-interlaced and the original frame rate will be restored.
I used two popular VDub denoisers, SmartSmootherHiQ (SmoothHiQ.vdf) and VDub's built-in "temporal smoother". Both are set at moderate strength. Usually it's cleaner and sharper to use two mild temporal denoisers instead of one overly strong one that blurs too aggressively or creates ghosts during motion. Get SmartSmootherHiQ at http://www.digitalfaq.com/forum/atta...smootherhiqzip. "Temporal smoother" comes with VirtualDub. The denoisers are followed by the built-in "sharpen" filter at a moderate value of 8.
VirtualDub's built-in "resize" filter is used to crop off the old borders and remove the bottom-border head switching noise, then to build new borders and center the image in the frame. However, the image itself isn't resized (resizing always has a quality cost). it's the new borders that are actually "resized", Using the method shown below with the filter dialog open and Preview enabled:
A thread that shows the step by step method for cropping and bordering in this way is in a post at How to Properly Crop the Overscan in VirtualDub [GUIDE]. The article begins in an odd way by warning the user to never crop (!), but in fact that's exactly what you'll be doing. Of course you'll just crop off unwanted or dirty borders, not big chunks of the original image content. The post also refers to the image's "overscan". Actually images don't have overscan; displays have overscan, but images don't.
The last step is to restore the original interlacing and frame rate, using the built-in "interlace" filter:
Now save the video by clicking "File..." -> "Save as avi...", and give the new file a location and name.
The attached .vcf file, when opened in VirtualDub, will load the filters in the proper sequence using the same settings I used. The resulting .mpg is attached as "Sample_VDub_DVD_4x3.mpg" and was encoded with TMPGenc Video Mastering Works.
I guess it depends on what you'd think the strong sidelighting would be like. In the top picture the water is gray, the overhead banners are reddish, and the splashes around the swimmer are yellow. Then again, it could be made to look like anything.
The following users thank sanlyn for this useful post:
cicaesar (12-03-2018)
Sorry for the late response guys, and thank you very much for all your input, it’s very very appreciated. I don't want to jinx it but so far my new round of recordings is going pretty well.
To add to the list of capture settings I used: I've started capturing on an external drive as suggested in another forum post. I was a little skeptic at first because it's a USB 2.0 HDD (worse than my SATA III internal HDD) but it's working wonders. I have almost always 0 dropped frames and very few inserted frames now -- I suspect that the random batch of 100ish inserted frames could be caused by Windows writing something on the internal HDD. There are still captures that suffer with many inserted frames, but they are the exception now.
@hodgey:
Quote:
The green line on the right is an artifact/design flaw in the newer Sony camcorders when playing back PAL video1. It's not present on NTSC, when playing a NTSC tape on a PAL camera. As far as I know it's present on most or all the the newer (late 90s and later) Sony Hi8 and Digital8 camcorders. [...] There is an avisynth script in this thread that can mask it to an extent.
That is actually good news, at least I know it's not something I could avoid. I've read the proposed script and I'm not really sure if it crops the line or if it tries to "restore" it by working with the colors, but I will definitely tinker with it later on in the process; same goes with Sanlyn’s resize method. I’ve not quite understood whether the green line should be somewhat smoothed or cropped out entirely though.
Quote:
The EZGrabber 2 seems to be nearly identical to the Diamond VC500 which both me and Sanlyn use regularly, so the image quality should be good otherwise.
This made my day, thanks, I was worried that I was using a low quality equipment since the grabbers cited here are not available online for me -- maybe it's because I'm in the EU market zone.
@sanlyn
Quote:
Your capture sample had good levels control and color.
Yay. I definitely don't have the skill to tell this, it's great to have confirmation of good source quality from you.
Quote:
Consider yourself fortunate. I started several years ago with 350-plus hours of tapes, and collected another 100 before I went to DV-only recording. Considering the time you have available I'd suggest capturing all the tapes first. Give yourself some time to load those captures into VirtualDub where you can very quickly scan them lickety-split to make sure there are no major errors
Thank you for your encouragement (I need it!); I'm doing as advised. After a capture I take 10 mins to check for audio sync and inserted frames: usually I find single dubbed frames which I leave alone (it would take too much time to recapture every single frame), and batch of inserted frames when there is a change of "event", mainly because my father skipped some seconds ahead as a means for cutting between events. In the case of many inserted frames I do a full recapture, which usually fixes things. Other than that quality check, I capture constantly.
Quote:
Meanwyhile, if you want to see them as 4:3 is VirtualDub, right-click anywhere on VDub's right-hand output panel and select 4:3 from the choices in the menu that appears.
Great, didn't know that. Is there a way to embed the DAR information in the video file so that a player like VLC can apply it automatically? Or is that only done in the compression phase?
Quote:
Are you certain you installed 64-bit? Your sample is in 32-bit huffyuv.
The installation package is named "huffyuv64-2.1.1.zip" so I guess it's 64 bit, although it could have installed both versions, I'm not really sure. For VirtualDub, I checked the “About” page and I can confirm that it’s the 64 bit version.
I haven’t mentioned that I first tried to install VirtualDub 1.9.11 following Lordsmurf’s guide, but it didn’t work for me: VDub couldn’t recognize Huffyuv. I’ve tried installing and reinstalling many times but I only managed to work with Huffyuv multithreaded: given that it produces videos with a different 4CC than standard Huffyuv I’ve decided against using it, for compatibility’s sake. I’ve tried VDub 1.10 as a last resort and it recognized Huffyuv immediately, that’s why I sticked with it.
Given the situation and the limitation of 64 bit you mentioned, I guess that my only option is to capture with VDub 1.10 64 bit and use your procedure to revert to VDub 1.09 32 bit for post processing. I am under the assumption though that a video encoded in Huffyuv format with a 64 bit encoder can be read by a 32 bit encoder: is it correct?
Quote:
If you did install 64-bit VirtualDub it was not a good idea, as version 1.10.x is buggy, especially for capture
This preoccupies me A LOT, since as described before VDub 1.10 is the only way I managed to have a functioning Huffyuv codec. Can you elaborate on what part of the capture process is impacted by the bugs? I’ve looked around the internet but I couldn’t find additional information. What I mean is: do the bugs affect the quality of the capture? The recordings I made are somewhat compromised, and is there a way to check if they are? As of now I’m pretty happy with my setup and I wouldn’t want to start reinstalling everything again – I’m not even sure it would change anything since I’ve already tried the 1.09 route unsuccessfully.
Quote:
At this point your best defense against dropped frames is to use a different Timing Option setup. The Timing Options image that you posted earlier doesn't show the options recommended for USB capture devices in the newer Settings Guide. The settings that you used inn your posted image tell VirtualDub to manage frame timing, which isn't always a good idea. Below, this is what is usually recommended for USB capture dongles:
I’ve tried again with your options but maybe there is something wrong with my dongle because it introduces random audio distortions (slowing the sound). Luckily though leaving the sync job to VDub is producing videos with consistent a\v sync, with audio resamples around the 1.0x range during capture usually.
Quote:
A favorite free DVD authoring-only app is DVDStyler (https://www.videohelp.com/software/DVDStyler) -- get the 32-bit version.
Ok perfect, so I guess the DVD creation process would be:
- I convert the videos to MPEG (by the way, MPEG 1 or MPEG 2?)
- I use DVDStyler to author them and create .vob files
- I burn the resulting files to disk with some burning program (long time since I did this )
Quote:
Below, the first effort at color-correcting your sample. The top image is an original frame (the color balance is far too red, likely due to the lighting), the lower image is the corrected version
Well, what to say, I really didn’t think about touching up the videos but after seeing your incredible results I guess I will take a shot at it after the capture rounds!
Quote:
The other video attachment is "Sample_stabi9lized_4x3.mpg". It was stabilized at about 50% power with the DeShaker filter in VirtualDub -- but look at the odd motion! The reason you won't be able to stabilize these frantic camera motions for better denoising is because of the date stamp in the lower right corner. When the overall image is stabilized by moving it around in the frame, the date stamp is also moved around. That's OK for demonstrating how far and fast the image moves, but it's annoying to see the data stamp jiggling around.
That’s unfortunate, I actually appreciate the date stamp because I couldn’t know the event dates otherwise, but with the stabilizer filter the image is so much better, I’m actually surprised it works so well. Those frantic movements come from a less than steady hand from my father (I also have the same problem) combined with the zoom in, which amplifies them. I have a (very limited) amount of clips without date stamps though: I will definitely apply the filter to them.
Quote:
With a deadline looming, you likely won't have time to get accustomed to using Avisynth. If you've never used it you would need a full afternoon just to download the Avisynth filters I used for the mpg in the earlier post. Although VirtualDub doesn't have industrial-strength cleaning plugins like Avisynth, it nevertheless has useful filters that can still make quality improvements and has very good color controls that Avisynth doesn't have. So here are some VirtualDub suggestions that don't require Avisynth.
Thank you VERY much for this in depth crash course on filters, I find it really useful and interesting and I see how this could change from a chore to a new hobby of mine. Your resulting video is way better than the original.
The real obstacle for me would be color correction. I feel like I don’t have enough skill to look at an image and tell if it’s “correct” in terms of brightness, contrast, levels of RGB. I couldn’t even tell if that scene was too red, although looking at your final image it seems pretty clear now that it was. I fear like I would do more harm than good venturing there.
I’d feel more comfortable using a generic, low profile and conservative approach, which maybe would not yield optimal results but which would make the images surely better (and surely not worse) than the original. A sort of “set and forget” filter chain that I would apply to all capturings without changing it on a scene to scene basis.
Basing on your explanation I was thinking about this:
- Color Camcorder Denoise
- Chroma Noise Reduction
- Deinterlace mode
- Smart Smoother HiQuality
- Temporal Smoother
- Sharpen (not entirely sure about this)
- Resize
- Interlace
What do you think? I’m open to investing time to learn Avisynth if needed \ better.
Quote:
VirtualDub's built-in "resize" filter is used to crop off the old borders and remove the bottom-border head switching noise, then to build new borders and center the image in the frame. However, the image itself isn't resized (resizing always has a quality cost). it's the new borders that are actually "resized"
I’m wondering: since you explained that every filter has an associated cost in terms of quality lost, why is the resize one excluded – is its cost so much higher than the others?
Goodbye and thanks again for this very informative discussion.
That is actually good news, at least I know it's not something I could avoid. I've read the proposed script and I'm not really sure if it crops the line or if it tries to "restore" it by working with the colors, but I will definitely tinker with it later on in the process; same goes with Sanlyn’s resize method. I’ve not quite understood whether the green line should be somewhat smoothed or cropped out entirely though.
The idea in that script is to extend the last good pixels in th color channel to the edge of the image. It won't look perfect, especially if there is a , but makes it much less noticeable.
The script in that thread was to fix an error at the top though, which was present on the specific Hi8 VCR. I ended up making a different script based on it to fix the camcorder specific issue:
Code:
# Dumb interpolation of right edge of v channel to mask chroma issue on sony Hi8 cameras.
# usage:
# The line number only works in steps of four at the moment.
# Start low and increase until it looks right.
#
# fixsonychroma(16)
#
function fixsonychroma_b1(clip c,int goodline)
{
origW = c.width()
origH = c.height()
badcolor=c.vtoy()
fillcolor=badcolor.crop((origW - goodline) / 2,0,2,0).pointresize(goodline / 2,origH)
fixcolor=stackhorizontal(badcolor.crop(0,0,-(goodline / 2),0),fillcolor)
ytouv(c.utoy,fixcolor,c)
}
You can look into it later if you start looking into avisynth. Alternatively you could crop it.
The following users thank hodgey for this useful post:
cicaesar (12-07-2018),
sanlyn (12-04-2018)
Your capture sample had good levels control and color.
Yay. I definitely don't have the skill to tell this, it's great to have confirmation of good source quality from you.
Sometimes proper input signal levels fit into the groove by accident. Sometimes not. When we look at Mother Nature or watch TV, our eyeballs play with only 3 colors, Red, Green, and Blue -- which we refer to in graphics as RGB. Color science described human vision in terms of numbers (as usual) and used a value system of RGB 0 to 255 from dark-to-bright.
On the other hand, digital video as most of us know it is usually stored in a YUV data storage system: Y = luminance (brightness), U=Blue (actually blue-yellow), and V=red (actually red-green). It's been that way since early broadcast days when engineers sought to save transmission cost by reducing video data, which was all-analog waveforms back then, to the smallest number of components.
Today, engineers limit the darkest and brightest transmission and storage values to 16-235 for various reasons, most of which have to do with the cost of bandwidth and not the least of which has to do with not blowing out TV sets trying to reproduce unreasonable signal and contrast levels. When color TV came along, video systems had to work with the nifty new color tubes as well as with b&w TV, surveillance systems, and other display technology that didn't want or use color. By storing and transmitting brightness separately from color, displays with no color capability could do the B&W thing while color displays could use it all. And although the YUV spectrum range was limited to y=16-235, RGB systems were so designed that when they interpreted the typical YUV signal they expanded YUV 16-235 to RGB 0-255.
Most YUV systems can contain signal levels that are far wider than 16-235. Film and digital cameras can record colors darker and brighter than we humans can see. Now, picture this: If RGB 0-255 is all that a PC or TV display can handle, and if a YUV signal is expanded from 16-235 to RGB 0-255, what happens in RGB if the incoming YUV signal is already darker than 16 and brighter than 235? If you think that the typical RGB setup will simply clip (destroy or ignore) super-blacks and super-brights, you're right. Many digital recording systems will do the same thing when their incoming signal levels are more than a receiving or transmitting system is designed to handle. When that dark or bright data is destroyed (clipped), it never comes back. It's gone forever, lost to the ether. Not only is out-of-spec data often clipped inside cameras and in capture drivers, but when YUV exceeds 16-235 an RGB translator won't expand beyond 0-255. Super-black shadows become grim, dark blobs without detail (aka crushed blacks) and super-brights become pale colorless bright blobs without detail (aka clipped brights). If you try to brighten the black blobs, all you find is grainy blobs of pure noisy mush. All you get from darkening clipped super-brights is colorless gray.
During capture, incoming signal levels are controlled by adjusting brightness (which controls black levels) and contrast (which controls brights) while observing the results with a histogram that cleverly contains markers telling you when signals are out of whack. VirtualDub's YUV capture histogram turns red at the left and right sides when the signal overflows the safe 16-235 zone. Some users have an external PA-100 proc amp with LED panels that show pretty much the same thing, but with cute blue panel lights (cost: USD $400 used, and rising). What most users do is randomly sample a sequence of incoming video, adjust for worst-case scenarios allowing for a slight overrun that can be corrected later, and proceed to capture with safe levels.
What that histogram looks like and what it tells you is graphically explained in post #3 of Capturing with VirtualDub [Settings Guide], and setting it up with the proc amp and temporary crop controls is illustrated in post #4 of the same Guide. This takes less than a minute or two. If you're trying to eyeball it, save yourself some grief: it's impossible to do accurately by eyeball even with a monitor calibrated to D6500 standards -- which, I'm pretty sure, you're not using at this point.
Quote:
Originally Posted by cicaesar
(by the way, MPEG 1 or MPEG 2?)
Standard DVD-disc is MPEG2 only. BluRay can use MPEG2, h.264, or VC-1.
Quote:
Originally Posted by cicaesar
Is there a way to embed the DAR information in the video file so that a player like VLC can apply it automatically? Or is that only done in the compression phase?
DAR is imbedded by encoders. But you can right-click on most media player screens and adjust the playback's aspect ratio.
Quote:
Originally Posted by cicaesar
I am under the assumption though that a video encoded in Huffyuv format with a 64 bit encoder can be read by a 32 bit encoder: is it correct?
I use Windows XP for most of my work, so everything at this end is 32-bit. Your sample played Ok here. I have Win7 and Win10 64-bit mnachines also, but frankly all the media players installed on them are 32-bit except one. I never tried mixing 32-bit and 64-bit software, so someone familiar with what can go wrong might advise all of us on this point.
Quote:
Originally Posted by cicaesar
I’ve tried again with your options but maybe there is something wrong with my dongle because it introduces random audio distortions (slowing the sound). Luckily though leaving the sync job to VDub is producing videos with consistent a\v sync, with audio resamples around the 1.0x range during capture usually.
To tell the truth, in my expertience I've encountered just about every variation one can think of. But my own setups are strictly by the book and never gave me problems. How all this other stuff happens is a mystery to me, but the universal rule seems to be "Whatever works."
Quote:
Originally Posted by cicaesar
with the stabilizer filter the image is so much better, I’m actually surprised it works so well. Those frantic movements come from a less than steady hand from my father (I also have the same problem) combined with the zoom in, which amplifies them. I have a (very limited) amount of clips without date stamps though: I will definitely apply the filter to them.
The deshaker filter works most of the time, but it takes getting used to and it is time-consuming. Note that when the image moves in the frame, so do the side borders. Cropping and cleanup afterward can sometimes cost a lot of screen real estate. Usually a partial correction works better than 100%. Deshaker's home page has an essential guide written by the author (https://www.guthspot.se/video/deshaker.htm). Also has tutorial video links, one of which is http://www.sheldonbrown.com/deshaker.html. And there's a short introduction at https://www.tipard.com/video/video-deshaker.html.
Quote:
Originally Posted by cicaesar
The real obstacle for me would be color correction.
It was obstacle for me, too, at first. Regardless of whether one works in YUV or in RGB, at least there are only 3 colors to learn. But of course it does get more complicated. I'd suggest that with the time you have available, the only color work to worry about is something that looks like a disaster which you really want to include for now. Meanwhile, most of what the experts know about color gradingn is learned from example, from tons of free internet websites, and from one or two Amazon books on the subject. Most of the time I think you could use the VirtualDub noise and crop filters alone, for now. The color control that's easiest to learn if you want to give it a try is Color Mill. Meanwhile, go easy with sharpeners.
Quote:
Originally Posted by cicaesar
I’m wondering: since you explained that every filter has an associated cost in terms of quality lost, why is the resize one excluded – is its cost so much higher than the others?
The way the resize filter is actually used, it doesn't resize the core image content. It simply adds black border pixels to restore the original frame dimensions. If you wanted to do something like resize the video to square-pixel dimensions for internet posting, you'll find better resizers in Avisynth, and they are very simple to use. Interlaced or telecined video should be deinterlaced or inverse-telecined before resizing. But that's another story.
The following users thank sanlyn for this useful post:
cicaesar (12-07-2018)
I’m open to investing time to learn Avisynth if needed \ better.
Avisynth does take time and patience, but it's not rocket science. In order to actually program an Avisynth plugin you have to get pretty deep into geekiness. But simply using Avisynth and its plugins is typically a case of following examples, doing a bit of reading to get the general idea, developing an eye for clean video, and remembering that everything is a compromise -- some things just can't be fixed.
Quote:
Originally Posted by cicaesar
@hodgey:
......I've read the proposed script and I'm not really sure if it crops the line or if it tries to "restore" it by working with the colors, but I will definitely tinker with it later on in the process
It's a useful script and similar to many that accomplish the same thing. Basically is "borrows" color pixels from an adjacent portion of the image and overlays them onto the stain. It's not perfect but I've had to use that method on some really ugly videos.
Another method is to overlay the stained area with a cleaned version that consists only of the cleaned border itself. This can often take a lot of filtering and is about as imperfect as anything else. It's the method I used to get the video later encoded as Sample_VDub_DVD_4x3.mpg and attached earlier in post #4 .
For your information and for other readers, below is the Avisynth script I used:
Code:
Import("D:\Avisynth 2.5\plugins\chubbyrain2.avs")
Import("D:\Avisynth 2.5\plugins\MDG2.avs")
AviSource("J:\forum\faq\ciceasar\Sample capture.avi")
Tweak(cont=1.3, sat=1.2, Hue=-15, dither=true, coring=false)
Levels(20, 0.95, 255, 16, 235, dither=true, coring=false)
AssumeTFF()
ConvertToYV12(interlaced=true)
# -------- chubbyrain2 for interlaced right border cleanup -------
separatefields
a=last
a
Cnr2(mode="ooo", scdthr=255.0, ln=255, lm=222, un=255, um=255, vn=255, vm=255)
chubbyrain2()
smoothuv(radius=7)
BiFrost(interlaced=false)
Tweak(Hue=4, dither=true, coring=false)
crop(692,0,-0,-0,true)
b=last
# -- overlay version "b" over original version "a" -- #
overlay(a,b,x=692)
weave()
# -------- end of chubbyrain2 routine --------
QTGMC(preset="very fast",border=true,ChromaNoise=true)
vInverse2()
Stab(range=4)
MDG2()
SeparateFields().SelectEvery(4,0,3).Weave()
ConvertToRGB32(interlaced=true,matrix="Rec601")
Crop(10,2,-16,-10).AddBorders(12,6,14,6)
return last
I opened the above.avs script in VirtualDub using "File.." -> "Open video file." VirtualDub considers an .avs file to be a video stream -- which it would be, actually, once Avisynth is opened. As I scrolled back and forth to examine the frames I added some virtualdub filters to the VDub filter chain. VirtualDub applies those filters to Avisynth's output. The VDub filters I used were arranged as shown below:
You'll notice that I didn't deinterlace, crop, or reinterlace in VirtualDub's filters, because I had already done all that in Avisynth. I saved all of the total output in YV12 color (because the next step would be MPEG encoding elsewhere) using lagarith lossless compression. Note that huffyuv can't compress YV12. Lagarith handles YV12, YUY2, and several flavors of RGB.
An .avs script is really a list of sequential operations. While only a handful of plugins appear in the above script, they require about two dozen support files in the background. Fortunately those support files are popular filters used everywhere. The initial setup is still a chore, but once you have those filters they can be re-used by other plugins as well as on their own.
Some script details might remove much of the mystery:
Import("D:\Avisynth 2.5\plugins\chubbyrain2.avs")
Import("D:\Avisynth 2.5\plugins\MDG2.avs")
The builtin Import() function at runtime imports the text code from two other filters that happen to be published as .avs scripts. There are many reasons why some plugins appear as coded text files instead of as machine-language dll's. At any rate, some of these .avs plugins have hundreds of lines of code, so the Import() function saves a lot of cut and paste and duplication. The code being imported is for two filters, chubbyrain2.avs and MDG2.avs, explained later.
Avisynth built-in functions can be found in the Avisynth wiki. Just go to Google and enter "Avisynth" followed by the name of a function or plugin, and you'll get a page like http://avisynth.nl/index.php?title=Import&redirect=no. Click on that additional link on that page and you'll see all the specs on that function and many others.
AviSource("J:\forum\faq\ciceasar\Sample capture.avi") AviSource() is Avisynth's all-purpose file opener and decoder, one of many. Once opened, the video stream is decoded and decompressed. If you want an encoded format for output, save the file as a lossless AVI and send it to an encoder.
Tweak(cont=1.3, sat=1.2, Hue=-15, dither=true, coring=false)
Levels(20, 0.95, 255, 16, 235, dither=true, coring=false)
The Tweak() builtin increases the contrast slightly ("cont"), increases saturation slightly ("sat"), adds some blue-green to the overall red color balance ("Hue"). Dithering:
Quote:
Dithering is used in computer graphics to create the illusion of "color depth" in images with a limited color palette - a technique also known as color quantization. In a dithered image, colors that are not available in the palette are approximated by a diffusion of colored pixels from within the available palette.
Coring is turned off to avoid clipping values that might lie outside the range y=16-235.
The Levels() function is used to control black levels and bright levels and to maintain the safe y=16-235 corridor. Gamma at 0.95 reduces midtone response just a tad to add some dimensional snap to darker colors.
AssumeTFF()
ConvertToYV12(interlaced=true)
Avisynth assumes that the incoming video, if interlaced, has a Bottom Filed First (BFF) field priority. But for your sample, that assumption is incorrect. AssumeTFF() tells Avisynth that your video sample has Top Field First (TFF) field priority.
This conversion from YUY2 to YV12 is necessary for the plugins that follow. Avisynth performs these conversions with great precision and very little distortion. it's much better at it than many very expensive editors, including those from Adobe. note that for any conversion you must specify whether or not the video is interlaced (telecine or pulldown is usually considered a form of interlace).
The chubbyrain2 routine is in wide use. No one really knows where the name came from. It is classed as an anti-rainbow (chroma noise) cleaner. You can read a whole lot about it at http://avisynth.nl/index.php/ChubbyRain2, which is a typical plugin download page. The first thing this routine does is use SeparateFields() to create a new video stream that breaks each interlaced frame into its two separate interlaced fields -- it is a form of non-destructive deinterlacing. Often it's sufficient for temporal denoising, sometimes it's not.
After creating this stream of separated fields, the code gives the name "a" to that new video, which is the "last" thing that was executed. You can assign names to new creations, as long as the name doesn't duplicate a function or plugin.
Next, the new video "a" is given focus by simply mentioning its name. Then some filters are applied to video "a", namely chubbyrain2(), three other rainbow cleaners (cnr2, SmoothUV, and Bifrost). Then good old Tweak() is called to add some blue to the result. Next, Crop() is used to cut off the left-hand 692 pixels. You now have a new video that is 576 vertical lines high but only 28 pixels wide -- which happens to be the same width as the stained border. That crop operation becomes the "last" thing the script does before naming the newly cleaned video as "b".
The next thing to do is to overlay() the original, stained "a" with the newly cleaned and created "b". The 28-pixel wide "b" is superimposed onto "a" beginning at 692 pixels to the right, thus overlaying the original border stain. The last step is to Weave() the separated fields back together again as interlaced frames. This ends the chubbyrain2 routine. All subsequent operations will be performed on the new, filtered video.
QTGMC(preset="very fast",border=true,ChromaNoise=true)
vInverse2() QTGMC is the prime deinterlacer from Avisynth, as good as or better than anything your TV or set top player can do and superior to deinterlacers from popular NLE's. Here, it specifies a preset that includes dozens of pre-built operational settings. ChromaNoise=true is added for extra chroma cleaning, which also helps to minimize stain remnants. This deinterlacing uses motion analysis for cleaner results in creating full-sized frames from half-sized fields. QTGMC isn't always necessary and is sometimes overkill; it's used here at mild settings in order to subdue excess interlace combing that is present in the source. The plugin that aids this smoothing is vInverse2(), which can be used only with fully deinterlaced video. QTGMC is a heavy-hitter plugin that requires a dozen support filters. These are available in a .zip package that contains several subfolders and complete documentation on every component. It's all explained in several previous threads, including post #26 in this thread: http://www.digitalfaq.com/forum/vide...html#post57401.
A demo video has been posted that shows the difference between some commonly used deinterlacers. The demo video compares other deinterlacers with TempGaussMC, the direct ancestor of QTGMC and from which QTGMC gets its acronym. http://www.digitalfaq.com/forum/atta...gmca4_tdtmmavi is a real eye-opener.
Stab() is a mild stabilizer plugin that is set with range=4 to study motion for several consecutive frames. It very slightly calms some of the camera motion to help the denoisers work a little more efficiently. This is a slow filter that requires motion interpolation support filters.
MDG2()
This is an enhanced version of a filter called MDegrain2 that is builtin to a bigger plugin called MVTools2, which is available with QTGMC. MDG2 itself is a degrainer and is pretty good at cleaning what many call "floating VHS grunge". It is posted as an .avs script and requires non-interlaced video. It, too, is mentioned in post #26 in the previously linked http://www.digitalfaq.com/forum/vide...html#post57401.
SeparateFields().SelectEvery(4,0,3).Weave() is a common means for re-interlacing video. SeparateFierlds() breaks full-sized progressive frames into top and bottom half-height fields (because the frames are progressive, the two half-height fields are exactly the same image). Then for every 4 separated fields, Select() fields 0 and 3 (field and frame numbers begin with 0). Finally, weave() the selected fields into fully re-interlaced frames.
ConvertToRGB32(interlaced=true,matrix="Rec601") converts the processed current video to RGB32 for use by VirtualDub's filters, which work in RGB color. Again, the interlace state must be specified. The conversion uses the RGB color matrix "Rec601" for standard definition video.
Then Crop(10,2,-16,-10).AddBorders(12,6,14,6) performs the same border cropping seen earlier in the VirtualDub sample, but with Avisynth it's more straightforward. Crop() removes specified border pixels in the order of 10 pixels from the left, 2 pixels from the top, 16 pixels from the right, 10 pixels from the bottom. Then AddBorders() restores the original frame size, centers the image, and replaces the cropped pixels with clean new ones in the same order: 12 on the left, 6 on the top, 14 on the right, 6 on the bottom. This cropping is slightly thicker than shown in the VDub example because the VirtualDub-only sample didn't use the Stab() stablizer.
Finally, return last outputs the "last" thing the script performed, which was the Crop and AddBorders statement.
The following users thank sanlyn for this useful post:
cicaesar (12-07-2018),
Delta (07-11-2022)
Avisynth does take time and patience, but it's not rocket science.
Some of those filters get really far advanced, straight up computer programming code, and my eye gloss over. I can handle a lot of Avisynth, but at some point even I get lost. Some of these filters have steep learning curves.
To make matters worse, remember that Avisynth is open source with contributors, and techies sometimes aren't the most communicative. Meaning their documentation is terrible, if any exists at all. So it's not just a learning curve in those instances, but trying to decipher code, reverse engineer, figure out what it's doing.
Quite a few filters also make false promises, give newbies false hope.
That recent post of yours is excellent, really takes the confusion our of some of those filters.
Hi everyone. I'm replying to my old thread for consistency (this is really the continuation of that same work) but if moderators prefer I can create a new thread. I've seen many replies to old posts throughout the forum so maybe this is ok.
First of all I want to thank the people in this forum, especially Sanlyn and Lordsmurf, not only for your help on my thread but for all the useful information that you put in this forum.
Since I asked here about how to capture my video8 cassettes more than a year has passed. I did complete my capture in time, thanks to you. Now I have some time to spare and I'd like to proceed with restoring and converting to DVD and x264, by picking up the avisynth script Sanlyn gave me and starting from there.
Before posting here I prepared myself the best I could: I've put in almost a month reading dozens of forum posts, installing the needed software, experimenting by myself and trying to find a process to do this restoration. I went through avisynth's wiki, color theory, video formats, etc., and with every guide I came out with 2 answers and 5 new questions... some of them are still needing answers, hence this post.
Please bear with me for the incoming wall of text: I try to be thorough, so that while I'm asking for help I can contribute what little progress I have made to others who will approach my same task in the future.
Context
I have many Video8\Hi8 25 fps PAL cassettes that I captured in Huffyuv. The cassettes were originally taped with a Sony Handycam Video8, and were digitalized with a Sony Handicam Digital8 DCR-TRV230E with integrated TBC, and an EZGrabber2 USB converter. I'm working on a 10 year old 4-core i5 CPU with 12 GB of RAM, so not the fastest machine on Earth; nevertheless, I don't mind waiting. It is not connected to a monitor but to a LG TV (model 57LM620S-ZE). Progressing from my first post, I managed to install both Huffyuv and Lagarith in their 32 bit versions, so for the editing I will be using the recommended virtualdub 1.9.11 (32 bit) and avisynth 2.6.0 (32 bit).
What I want to do
I want to restore this videos with avisynth and virtualdub, and in particular:
Make them "cleaner" (at least reasonably); for instance, reducing noise or eliminating jagged lines
Crop the headswitching noise
Minimize the visual impact of the right green border that pesters every one of my videos
Stabilize the image, and wherever there isn't the timestamp overlayed, deshaken it
Adjust color levels and remove color casts
Restoration is the main focus of this post, after I'm done restoring I'll think about the conversion to DVD and x264 (maybe this time I'll manage not to wait an another year for the next step...).
I know there isn't a one-size-fits-all solution, but I guess that since all my cassettes were taped with the same camera I can separate the activity in 2 main steps: one to apply filters that only depend on the camera and can be replicated for every video (eg: cropping) and one with filters that depend on the scene and thus must be tailored every time (eg: color levels).
Regarding colors, I just want to make slight adjustments. I'm not a professional unfortunately and I do not have a professionally calibrated monitor. In many a post people strongly advised not to eyeball colors, so I'm going to rely on histograms to align the luma\chroma spectrum and I will make only slight adjustments regarding saturation. I've tried dabbling with virtualdub's graduation curves like Sanlyn explained but I'm always concerned that I'm overdoing things... I'd prefer not to use it if possibile.
I did however adjust my TV colors with simple visual calibration (twice actually, the first time I messed it up): I set Windows' and my video card's color adjustments as neutral, I have removed all kinds of dynamic color regulation from my TV, and I have followed the Lagom calibrating guide to manually set the TV color parameters. I can't pass the sharpness and the gradient test though (albeit the latter by a small margin).
So, I will describe the process I've come up with to have your input. I will provide attachments. I will list my questions in the text.
The sample
Attached is an Huffyuv video I've cut from a cassette clip in YUY format: 01 - Restoration sample - Cut Huffyuv.avi.
I've excluded the sound. It's shot indoors at night. Aside from the headswitching noise and the godforsaken right green vertical band, it looks to me to have a red cast.
At frame 141, Csample gives me a value of R:230 G:160 B:166 from the front of the man's sleeve, which should be white.
At frame 309, Csample gives me a value of R:255 G:162 B:144 inside of the lower half of the "eight" candle for a color that should be whitish.
Setting input levels
This is the first roadblock I am encountering. I couldn't find any post online that settled the problem of regulating the input level of the video. What I did find out is that is very important to set levels before using avisynth plugins, and that is better to set levels first and regulate contrast \ brightness after.
So my first step is to open the video with virtualdub, load a crop filter to remove the right green edge and the black borders, load the levels filter and try to eyeball the correct input levels that I will use later in avisynth. I noticed that the midpoint value doesn't change when I move the left and right sliders, so I think I should leave it at 1.0. I reeeeeeally don't know if all this is correct to be honest.
I attach the virtualdub filter chain: 02 - Restoration sample - Levels initial reading.vcf.
Preprocessing with avisynth
I have separated avisynth processing in 2 scripts: the first is a preprocess that should be camera-dependent and thus applies filters that should be useful for all my videos without the need of changing it on a per-video basis. It's the very same avisynth script that Sanlyn suggested (I definitely am not able to identify image defects and choose which plugins to use).
Separating image quality processing from color correction has also the benefit of making faster adjustments in the color correction phase, because this processing is extremely slow on my machine.
I compress the video in Lagarith YV12. In the video color depth options I specify Autoselect for the input and 4:2:0 planar YbCbCr (YV12) for the output, in case it's not automatically setted just by choosing Lagarith YV12 in the compressing options. Seems to me that this settings gets resetted every time.
This is my Lagarith configuration:
You must be logged in to view this content; either login or register for the forum. The attached screen shots, before/after images, photos and graphics are created/posted for the benefit of site members. And you are invited to join our digital media community.
Many questions here:
Is my Lagarith configuration correct?
The process runs at 1.5 fps, it's very very slow. As I said, I don't mind waiting if the quality gets better, but I do wonder if I'm doing something wrong to be honest.
In the script I tried adding an antialiasing filter (Santiag()) because I have very bad jagged lines, especially with subjects in motion, and I thought the antialiasing could help. I still see jagged lines though, I don't know how to remove them, and they really (REALLY) bother me. What can I use?
Maybe it's not very apparent here, but in other longer clips the processed video looks clunkier than the original one, as if motion were not as smooth. Am I doing something wrong?
Is it normal that the Lagarith file occupies way less space than the Huffyuv one (75 MB vs 100 MB)?
Is it normal that the Lagarith video has way less bitrate than the Huffyuv one? GSpot gives me 50 Mbps for Lagarith and 65 Mbps for Huffyuv. I mean these are still very high bitrate values but I wonder if I'm doing something wrong and if this can have an impact on the smoothness of the video in motion.
I've seen many times here the "align chroma" suggestion (ChromaShift(C=2, L=-8)): should I add this too?
I've also seen a lot this line: MergeChroma(awarpsharp2(depth=20)). I understand it sharpens the chroma channel. Should I use it?
Is there a way to clean the right border a little more, with a more aggressive "de-greener"? It's still too green after the chubbyrain routine; for now I tweak a little of its brightness down in the color adjustment step (more on this later), but maybe there's a better way?
I've seen that Lordsmurf made a version of Stab() for VHS and tapes (name is Stabmod()): should I use that with Video8 cassettes?
Here is the script, which I also attach (03 - Restoration sample - AviSynth preprocessing.avs):
Code:
Import("C:\Program Files (x86)\AviSynth\plugins\ChubbyRain2.avsi")
Import("C:\Program Files (x86)\AviSynth\plugins\MDG2.avs")
/* Variables */
green_border_offset=690 # The green border offset
dark_input_level=0 # The dark input level
bright_input_level=215 # The bright input level
AviSource("01 - Restoration sample - Cut Huffyuv.avi") # Source file
Levels(dark_input_level, 1.0, bright_input_level, 16, 235, dither=true, coring=false) # Map input levels to output levels [how to set input parameters though?]
Tweak(cont=1.3, sat=1.2, dither=true, coring=false) # Initial bump in contrast and saturation for filters, to be corrected after
AssumeTFF() # Top field first
ConvertToYV12(interlaced=true) # YV12 conversion to work on filters
# -- chubbyrain2 for interlaced right border cleanup -- #
Separatefields # Deinterlace
a=last # Set variable "a" as the full frame
a # Use stream a for creating the clean right border
Cnr2(mode="ooo", scdthr=255.0, ln=255, lm=222, un=255, um=255, vn=255, vm=255) # Chroma noise reduction (should luma be 222 or 255?)
chubbyrain2() # De-rainbow
Smoothuv(radius=7) # Smoother
BiFrost(interlaced=false) # De-rainbow
Tweak(Hue=4, dither=true, coring=false) # Smooth green color
Crop(green_border_offset,0,-0,-0,true) # Crop out the image, selecting only the green band
b=last # Set variable "b" as the clean right border
Overlay(a,b,x=green_border_offset) # Overlap the clean right border "b" over the full frame "a"
Weave() # Reinterlace
# -- end of chubbyrain2 routine -- #
QTGMC(preset="very fast",border=true,ChromaNoise=true) # Deinterlace and clean
vInverse2() # Smoothing
#Stab(range=4) # Mild stabilizer
MDG2() # De-grain
Santiag(strh=2,strv=2) # Anti aliasing
SeparateFields().SelectEvery(4,0,3).Weave() # Reinterlace
Crop(10,2,-16,-10).AddBorders(12,6,14,6) # Crop out bottom head switching noise
return last
Attached is the resulting video: 04 - Restoration sample - Preprocessed Lagarith.avi
Analysis for color adjustment
For color correction I use an avisynth script to apply changes and analyze results, so that I can work on colors faster. As I mentioned I don't really want to do color correction, for which maybe virtualdub with colormill and curves would be more appropriate, I just want to "balance" the histograms. I don't feel confident eyeballing it with my limited skills and uncalibrated monitor. It seems though that I do need to apply saturation if I don't want everything to look washed out.
This is the script I use (05 - Restoration sample - AviSynth analysis.avs):
Code:
# Source file
AviSource("04 - Restoration sample - Preprocessed Lagarith.avi")
green_border_width=690 # The green border width
crop(720-green_border_width,10,green_border_width-720,-10) # Crop to exclude borders from histogram analysis
original=last # Set "original" as the original video
Tweak(bright=1.0, cont=0.85, dither=true, coring=false) # Luma spectrum alignment with brightness and contrast
Tweak(sat=1.8, hue=1.0) # Saturation and hue contol
ColorYUV(off_y=-10, gain_y=10) # Luma spectrum shift and amplitude gain
ColorYUV(gain_v=-40, gain_u=15) # Color alignment
Histogram("levels") # Histogram YUV analysis
StackHorizontal (original, last) # Horizontal comparison
ConvertToRGB32(interlaced=true,matrix="Rec601") # RGB conversion for final VirtualDub processing
RGBAdjust(rb=-20, gb=10, bb=10) # RGB spectrum alignment
HistogramRGBLevels(factor=1.5) # Histogram RGB analysis
return last
I'll explain my reasoning for you to critique (some more details are in the script's comments):
I start with the luma histogram: usually the Levels() function in preprocessing makes it stay inside the the allowed range, but I try to "center" it a bit more by regulating contrast\brightness and offset and amplitude with ColorYUV.
I try to center a little bit more the u and v channels to eliminate color casts (red in this case).
In RGB, I try to shift every channel so that it stays inside the allowed values
Finally, I change the saturation level. This is the only change that I apply by eyeball, I would have avoided it but without this change everything looks washed out.
You must be logged in to view this content; either login or register for the forum. The attached screen shots, before/after images, photos and graphics are created/posted for the benefit of site members. And you are invited to join our digital media community.
Questions:
Should I have the luma histogram expand a little more to the sides? How would I do it? Would that make colors clip?
Even after centering the RGB channel, their spectrums are still outside of the allowed range, so I guess I am losing details there. Is there a way to "shrink" those spectrums in order to make them all stay inside the range?
If you happen to have a calibrated monitor, do you think the resulting colors are decent? Are they oversaturated maybe?
Applying color adjustment and virtualdub filters
After the analysis I copy the results in another script.
This script doesn't include histograms and the StackHorizontal function.
This script includes a section to try and minimize the impact of the right green border, by not including it in the saturation bump, removing some of its brightness and centering a little more the v channel.
After this script I apply virtualdub filters and I save the video as a Lagarith YV12.
This is the script (also attached: 06 - Restoration sample - AviSynth color correction.avs)
Code:
# Source file
AviSource("04 - Restoration sample - Preprocessed Lagarith.avi")
green_border_width=690 # The green border width
original=last # Set "original" as the original video
#-- Green edge processing --#
crop(green_border_width,0,-0,-0,true) # Exclude the green edge from color adjustment
Tweak(bright=0.1) # Reduce the brightness of the edge
ColorYUV(gain_v=3) # Move the green-red spectrum towards the center
green_edge=last # Set "green_edge" as the green edge
#-- Green edge processing end --#
original # Work on the original video
/* Edit here: */
Tweak(bright=1.0, cont=0.85, dither=true, coring=false) # Luma spectrum alignment with brightness and contrast
Tweak(sat=1.8, hue=1.0) # Saturation and hue contol
ColorYUV(off_y=-10, gain_y=10) # Luma spectrum shift and amplitude gain
ColorYUV(gain_v=-40, gain_u=15) # Color alignment
overlay(last,green_edge,x=green_border_width) # Map the unmodified green edge on the color corrected video
ConvertToRGB32(interlaced=true,matrix="Rec601") # RGB conversion for final VirtualDub processing
/* Edit here: */
RGBAdjust(rb=-20, gb=10, bb=10) # RGB spectrum alignment
return last
I open this script with avisynth and I apply the filters which Sanlyn suggested, minus the color correction bits. This is the filter chain (also attached: 07 - Restoration sample - Virtualdub filters.vcf): 05 - Restoration sample - AviSynth analysis - Before.jpg
Attached is the resulting video (08 - Restoration sample - Postprocessed Lagarith.avi)
Questions:
Are the virtualdub filters too strong? For instance, at frame 238 if you look at the right part of the cake, isn't it too blurred?
Is it correct to save the video in YV12 at this point, being that I just converted in RGB32 in avisynth? Shouldn't I just save it in RGB32?
The right border bothers me SO MUCH, it's still too visible and I really hate it. Moreover I'm not sure that the alterations I made to it are producing a better or an even worse result. Is there something different that I can do about it? Should I crop it entirely?
Deshaken
I loved what Sanlyn showed me about the Deshaker plugin in virtualdub, so I've decided to apply it whenever the videos didn't have a timestamp (not many of them unfortunately). The fluttering borders distract me a lot, so I'm going with the fixed zoom option, but I have to say in many occasions it feels like it applies a bit too much zoom. I've gone through Deshaker's documentation and tried out the other options but I'm not able to reduce the zoom without having (albeit slightly) moving borders. I have configured the Deshaker plugin using this Lordsmurf's guide, but some options are different in my version so I'm not sure it is correct.
Questions:
There is a huge problem here, which I have to admit is the first time I see happening: as soon as the scene change (from the one with the presents to the one with the cake, frame 145), the borders start to shake A LOT. The first scene is perfect but the second sucks. I've tried splitting the 2 scenes and deshaken them separately but nothing changed. I fiddled with options to no avail. Maybe I should apply some sort of zoom filter manually to the second scene? How to do that while maintaining the same resolution?
The file size of the final Lagarith video is 50 MB, half of the original Huffyuv video (100 MB). Is this correct or should I worry?
Here is my configuration.
Pass 1:
You must be logged in to view this content; either login or register for the forum. The attached screen shots, before/after images, photos and graphics are created/posted for the benefit of site members. And you are invited to join our digital media community.
For anyone interested, I have continued this conversation in a new thread in the Restoration area of the forum.
I have tried to edit\delete the above post since it is duplicated now, but it looks like I'm not able to? Maybe a moderator can help? Thank you and sorry for the inconvenience.
