Interlacing - can anyone help clarify? - digitalFAQ Forum
 Forum Interlacing - can anyone help clarify?
 User Name Remember Me? Password
 Ask Question Join / Register FAQ Search Today's Posts Mark Forums Read

#1
08-17-2018, 05:45 PM
 discmeister Free Member Join Date: Apr 2018 Posts: 31 Thanks: 5 Thanked 0 Times in 0 Posts
Hi all,

Through a process of trial and error, I'm about ready (I think) to start my workflow in earnest. But there's one last thing that's bugging me.

I'm currently sampling my video from a VHS deck with built-in TBC, and through another TBC supplied by LordSmurf, to an ATI 600 USB. I'm using VirtualDub for Capture. I'm using Lagarith Lossless compression because I discovered during experiments that Huffyuv glitches in Premiere Pro when I'm doing my final edit (fade ins/outs etc - stuff that VirtualDub can't do).

Anyway, I'm wondering if anyone can supply clarification on interlacing? If I watch my raw AVI in 1:1 in VLC and go through it frame by frame, I don't see any combing. And if I bring up the file in MediaInfo, it doesn't have an entry for 'Scan Type' (whereas an MPEG2 file has 'Lower field first' and an H264 has 'Progressive').

Thing is, I can't see where any conversion to a deinterlaced format would be taking place - unless the ATI is doing it on the fly?

Assuming, then, that I have deinterlaced files at my raw stage, can someone explain why I'd then RE-interlace them for inclusion on DVD-Video discs? Couldn't I just keep them Progressive - especially since the final movies will be played on a DVD player but watched on LCD TVs?

Regards to all,

Discy
Someday, 12:01 PM
 Ads / Sponsors Join Date: ∞ Posts: 42 Thanks: ∞ Thanked 42 Times in 42 Posts
#2
08-17-2018, 09:50 PM
 sanlyn Premium Member Join Date: Aug 2009 Location: N. Carolina and NY, USA Posts: 3,190 Thanks: 107 Thanked 957 Times in 820 Posts
You can't always go by MediaInfo in this regard. It often displays interlaced factors incorrectly. It just does the best it can when what it gets, especially since many videos have incomplete metadata. It's possible that you have a physically progressive video that has been encoded or recorded with fake interlace flags, or a video that uses 3:2 pulldown flags (a mix of progressive and interlaced frames if telecine is hard-coded, or a video of progressive frames with soft-coded pulldown flags). It's difficult to say because we can't see your video from here. But if the source is VHS, it's physically interlaced.

Quote:
 Originally Posted by discmeister Thing is, I can't see where any conversion to a deinterlaced format would be taking place - unless the ATI is doing it on the fly?
The ATI 600 doesn't deinterlace. Neither does VirtualDub capture. If anything is deinterlacing it would be the player, but I figure that's unlikely. You don't say what frame size you captured to, but if it's height is less than 288 lines it won't capture as interlaced.

Quote:
 Originally Posted by discmeister Assuming, then, that I have deinterlaced files at my raw stage, can someone explain why I'd then RE-interlace them for inclusion on DVD-Video discs? Couldn't I just keep them Progressive - especially since the final movies will be played on a DVD player but watched on LCD TVs?
If they don't appear to be interlaced, why would you have to re-interlace? We can't really say because we don't known how you're post-processing. It's true that you can encode and author progressive standard definition for DVD with some software app or other, but video for standard definition BluRay must be interlaced or telecined, while video for web posting must be progressive.

Quote:
 Originally Posted by discmeister If I watch my raw AVI in 1:1 in VLC and go through it frame by frame,
I don't know why people use VLC for video work. It is notoriously unsuitable for anything other than normal playback. Virtualdub and Adobe are so much easier and more accurate for what you describe.

Neuron2_How To Analyze Video Frame Structure.zip

Be careful about using then term "raw" when referring to video. "Raw" doesn't mean what you think it does. But in this case we know what you mean to say, so take that advice with a grain of salt.
#3
08-18-2018, 03:14 PM
 discmeister Free Member Join Date: Apr 2018 Posts: 31 Thanks: 5 Thanked 0 Times in 0 Posts
Quote:
 Originally Posted by sanlyn You can't always go by MediaInfo in this regard. It often displays interlaced factors incorrectly. It just does the best it can when what it gets, especially since many videos have incomplete metadata. It's possible that you have a physically progressive video that has been encoded or recorded with fake interlace flags, or a video that uses 3:2 pulldown flags (a mix of progressive and interlaced frames if telecine is hard-coded, or a video of progressive frames with soft-coded pulldown flags). It's difficult to say because we can't see your video from here. But if the source is VHS, it's physically interlaced. The ATI 600 doesn't deinterlace. Neither does VirtualDub capture. If anything is deinterlacing it would be the player, but I figure that's unlikely. You don't say what frame size you captured to, but if it's height is less than 288 lines it won't capture as interlaced.
It's more than 288 lines. VirtualDub File Infomation says it's 720x576. PAL VHS, right?

Quote:
 Originally Posted by sanlyn If they don't appear to be interlaced, why would you have to re-interlace? We can't really say because we don't known how you're post-processing. It's true that you can encode and author progressive standard definition for DVD with some software app or other, but video for standard definition BluRay must be interlaced or telecined, while video for web posting must be progressive.
I have these files as AVI with Lagarith Lossless Compression. I need to do basic processing - fade in/out is pretty much the height of it - and then output them into two formats. First is H264 for my home Kodi player/media server set-up. Second is MPEG2 and separate audio files for import into Encore to make DVDs for friends. I'm using Adobe Premiere Pro CC to do the basic processing and then Adobe Media Encoder for output.

I know that H264 has to be Progressive - that's how Adobe Premiere Pro outputs it anyway. My query was whether, if for some reason the initial import files are not interlaced, I should still interlace the DVD Video MPEG 2 files or not?

Quote:
 Originally Posted by sanlyn I don't know why people use VLC for video work. It is notoriously unsuitable for anything other than normal playback. Virtualdub and Adobe are so much easier and more accurate for what you describe.
I use VLC because, well, it's how I've watched videos over the years. And countless websites say that if you're not sure if a file is interlaced or not, the easiest way to find out if it is is to go through on a frame by frame basis and look for 'combing' lines. It's a doddle to go through a file frame by frame in VLC, and you can turn DeInterlacing off.

Neuron2_How To Analyze Video Frame Structure.zip

Be careful about using then term "raw" when referring to video. "Raw" doesn't mean what you think it does. But in this case we know what you mean to say, so take that advice with a grain of salt.[/quote]

Thanks for your reply. But I'm not really any the wiser on whether there's a concrete, easy way to find out whether or not my files are interlaced or not?

-- merged --

Incidentally, if I import one of the initial capture files into Premiere Pro and ask it to Interpret Footage, its analysis comes back saying 'Progressive Scan' so No Field Order.

I can't quite work out how VHS footage going from the SVHS socket of my JVC deck to the ATI USB 600 and then through VirtualDub would not be interlaced, I guess...?
#4
08-18-2018, 04:23 PM
 sanlyn Premium Member Join Date: Aug 2009 Location: N. Carolina and NY, USA Posts: 3,190 Thanks: 107 Thanked 957 Times in 820 Posts
Since your mind is decided on all of the above, it has me wondering what your question is. Not much more that anyone can advise without a short sample of what you're working with. A sample is worth much more than a thousand words. It seems a bit sad that you're encoding before doing any real cleanup of VHS, which you can't do with Adobe except for color correction. Adobe doesn't resize or deinterlace well anyway, so it's probably a plus that you don't have to do any of that. BTW, h.264 can be interlaced or progressive, and can be telecined. I'm surprised that at Adobe's price point you have so little flexibility with h.264. After all, many h.264/AVC BluRay formats are interlaced or telecined as required by official BluRay spec. Not all BluRay formats are progressive. For example 29.97 NTSC and 25fps PAL at 1920x1080 or SD 720x480 or 720x576 are required in BluRay encoding to be interlaced or telecined. At BluRay 1920x1080 only film speeds (23.97 or 24 fps) can be progressive.

Quote:
 Originally Posted by discmeister I can't quite work out how VHS footage going from the SVHS socket of my JVC deck to the ATI USB 600 and then through VirtualDub would not be interlaced, I guess...?
There is no such thing as an "SVHS" outlet or "SVHS" cable. The output is an s-video out for s-video cables. I think you're confusing SVHS with s-video. SVHS is a tape format. S-video is a type of cable and signal transmission. s-video can transmit SVHS, VHS, VHS-C, Hi8, Video8, DVD, and cable box output (analog only, standard definition only). On some SVHS players, however, you do have to turn on an "SVHS" output indicator switch if you're playing SVHS tape, but the playback can be output via s-video, composite, or even component or HDMI cable if the player allows it.

You're also might be confusing interlaced playback flags with physical interlacing. A frame can be described as interlaced or as having interlace playback flags, but if both fields appear to be identical images, the frame is not physically interlaced. "Interlace" infers that the frame consists of two interwoven fields from two different instances in time, those instances being 1/50 (PAL) or 1/60 second (NTSC) apart. Truly interlaced 25fps or 30fps frames play 50 or 60 different field images per second. Physically progressive non-interlaced video when played on interlace media players (such as a VCR with its video heads) will play 50/60 fields per second but every two fields will be duplicates, so the same image is captured as two duplicate fields in one frame. If you deinterlace non-interlaced video in an editor you'll see two identical images with every deinterlaced frame. If you deinterlace interlaced video in an editor you'll see two different images with every deinterlaced frame.

Last edited by sanlyn; 08-18-2018 at 04:46 PM.
#5
08-18-2018, 04:50 PM
 hodgey Free Member Join Date: Dec 2017 Location: Norway Posts: 272 Thanks: 15 Thanked 74 Times in 63 Posts
Quote:
 Originally Posted by discmeister I can't quite work out how VHS footage going from the SVHS socket of my JVC deck to the ATI USB 600 and then through VirtualDub would not be interlaced, I guess...?
If the recording is of a movie that was originally shot on film, which is not interlaced footage in the first place, these were usually just recorded in PAL land by speeding the film up to 25 fps to match the PAL frame rate, and capturing such that each field is simply half of a film frame. Thus, when the fields are weaved back by the capture cards, this process is undone and you simply get the original frames back, provided that the fields are in the right order, even though the analog data is transmitted as separate fields. Maybe that's what you are seeing? If that is the case the footage obviously won't look interlaced.

(24fps film on NTSC is a bit more complicated).
#6
08-19-2018, 04:30 AM
 discmeister Free Member Join Date: Apr 2018 Posts: 31 Thanks: 5 Thanked 0 Times in 0 Posts
Thanks for the advice. My mind isn't decided upon anything, beyond the end goal - an H264 file that plays nicely on my Kodi box via my LCD TV, and the best-quality DVDs that I can provide to the original tape suppliers. Totally open to advice and suggestions from those with vastly more experience here. I've just become confused because I thought that the mix of VHS --> TBC --> ATI USB 600 --> VirtualDub AVI would give me an obviously interlaced file, and it hasn't.

How's best to supply a sample here? If I do so, perhaps someone might be able to suggest some of the more obvious cleanup processes that I could apply to it.

Regards to all,

Discy
#7
08-19-2018, 07:36 AM
 sanlyn Premium Member Join Date: Aug 2009 Location: N. Carolina and NY, USA Posts: 3,190 Thanks: 107 Thanked 957 Times in 820 Posts
Quote:
 Originally Posted by discmeister Totally open to advice and suggestions from those with vastly more experience here. I've just become confused because I thought that the mix of VHS --> TBC --> ATI USB 600 --> VirtualDub AVI would give me an obviously interlaced file, and it hasn't.
Normally it would be interlaced, but you have encountered a case of progressive video transferred to tape as progressive. This happens, especially with PAL. But it could easily go the other way, with interlaced or even telecined shows on tape.

Quote:
 Originally Posted by discmeister How's best to supply a sample here? If I do so, perhaps someone might be able to suggest some of the more obvious cleanup processes that I could apply to it.
The idea behind submitting samples is to create a short edit of a few seconds that includes motion of some kind (people moving, walking, gesturing, or slow camera pans, etc). Then reasoning behind motion is to allow checking for frame structure, noise characteristics, and other elements. Cuts from AVI captures are usually made in VirtualDub. Such captures are usually made as 720x576 framews in YUY2 color. From such an AVI a scene selection of 8 to 10 seconds in the original unaltered colorspace would fall well within the 99MB file size limit for uploads.

Open the AVI in VirtualDub and use its scroll and edit controls to create your selection. In order to prevent unwanted processing or colorspace conversion when saving the file, click the top main-menu "Video" item, and in the drop-down menu select "direct stream copy". Then click "File.." --> "Save AVI", give your sample a name and location, and save the output.

This procedure is general SOP for most forums of this type. Don't forget to enter a message or remarks you want to make in the reply window. Also, this forum runs a virus check on submissions, so give your sample plenty of time to upload. When upload is complete, close the dialog window, complete your reply, and click "Submit".

Wishing you the best in your projects....
 The following users thank sanlyn for this useful post: SuperS (08-19-2018)
#8
08-19-2018, 08:32 AM
 discmeister Free Member Join Date: Apr 2018 Posts: 31 Thanks: 5 Thanked 0 Times in 0 Posts
Thanks again for all your help. I've attached a couple of Direct Stream Copy from my Lagarith VirtualDub Capture files, as suggested. All advice gratefully received!

Kind regards,

Discy

Attached Files
 1000 Lakes snippet for DigitalFAQ.avi (43.04 MB, 7 downloads) 1000 Lakes snippet 2 for DigitalFAQ.avi (18.61 MB, 7 downloads)
#9
08-19-2018, 10:15 AM
 sanlyn Premium Member Join Date: Aug 2009 Location: N. Carolina and NY, USA Posts: 3,190 Thanks: 107 Thanked 957 Times in 820 Posts
Thank you for the two samples. Well done.
I'll give them a closer look and see what can be done smooth out some rather ragged lines and edges. Meanwhile, I can say for certain that the two samples are interlaced.
You can verify this at home by opening the video in VirtualDub and loading VirtualDub's built-in deinterlace filter. Configure the deinterlace for "yadif" on the left-hand side of the dialog, and on the right-hand side for "Double frame rate, top field first" (which means that no fields will be discarded). You can then cruise through the video using the frame-by-frame scroll icon in the lower left panel. As you scroll one frame at a time, the left-hand "Input" window will show the same image twice but the right-hand "output" panel will show two different images in succession with each click. This demonstrates that each frame contains two distinct images from different instants in time.You can also note along the bottom of the VirtualDub window that when the deinterlace filter is activated the frame count in the first snippet sample changes from 142 original frames to 284 deinterlaced images.

The image below shows how I set up the VirtualDub yadif deinterlace for this test:

Attached Images
 deinterlace setup VDub.png (25.1 KB, 79 downloads)
#10
08-19-2018, 11:15 AM
 discmeister Free Member Join Date: Apr 2018 Posts: 31 Thanks: 5 Thanked 0 Times in 0 Posts
That's really useful info - thanks. And yes, I've now performed that test in VirtualDub and seen the every-other-frame movement and the increased count.

Interested to hear how you think I can clean things up.

Kind regards,
Discy

-- merged --

Incidentally, one initial improvement that I may be able to make is within Premiere Pro itself. There is a setting for each clip you import, under Clip>Modify>Interpret Footage, and the resulting dialog box has a 'Field Order' setting. It has been set to 'Use Field Order from File', which Premiere Pro inteprets as 'No Fields (Progressive Scan)'.

Am I right in thinking that interlaced footage would mean I should change that to either Lower Field First or Upper Field First? I'm wondering now if Premiere Pro has been starting from a poor base point, given that it doesn't appear to think the raw footage is interlaced and I've allowed it to assume that...

Kind regards,

Discy
#11
08-19-2018, 12:31 PM
 sanlyn Premium Member Join Date: Aug 2009 Location: N. Carolina and NY, USA Posts: 3,190 Thanks: 107 Thanked 957 Times in 820 Posts
Your video samples are interlaced, top field first (TFF). It's immediately obvious when you work the file in VirtualDub and Avisynth.

You've just encountered one of many reasons why this forum and others don't recommend Adobe Premiere as a restoration and repair platform.
#12
08-19-2018, 01:16 PM
 discmeister Free Member Join Date: Apr 2018 Posts: 31 Thanks: 5 Thanked 0 Times in 0 Posts
Yeah, I'm aware it's not popular around these parts...

How is it obvious immediately, by the way? A technique that I could use on other files?

Regards,

Discy
#13
08-19-2018, 02:39 PM
 lordsmurf Site Staff | Video Join Date: Dec 2002 Posts: 7,882 Thanks: 834 Thanked 1,225 Times in 1,093 Posts
Quote:
 Originally Posted by discmeister Yeah, I'm aware it's not popular around these parts...
I've been an Adobe Premiere user since the 90s, and anytime I need non-basic editing, it's my tool of choice. It's an excellent editor (NLE), one of the best, and has always been more popular than any other out there (Vegas, Final Cut, etc).

But it's an NLE, not a restoration tool.

It, like all other NLEs, has a poor understanding of interlace, and the "restoration" aspects are very rudimentary at best.

Besides color correction. It does that very well, especially in the past decade where Premiere got more Photoshop-like in that area.

- Did my advice help you? Then become a Premium Member and support this site.
- Find television shows, cartoons, DVDs and Blu-ray releases at the TVPast forums.
#14
08-19-2018, 03:08 PM
 discmeister Free Member Join Date: Apr 2018 Posts: 31 Thanks: 5 Thanked 0 Times in 0 Posts
Yeah, the workflow is so straightforward in it that the basic stuff that I need to do - fade in/out (vid and audio), and chapter markers for Encore, can be completed in barely a minute. Easy to save presets for output into MPEG2 and H264, too.

I am curious about the restoration aspect and what can be achieved, but I saw somewhere else how someone on this forum had spent 90 hours retoring 90 minutes of video. I simply have to be realistic and say that's not going to happen. Because if that was the time involved in each recording (I have at least 50 to get through), I simply wouldn't have considered doing it in the first place.

I am interested in how avisynth (or virtualdub) can do a resizing job, mind you. I hate VHS noise around the edge of the frame, but I also hate having black lines anywhere around the picture on my television...

Regards to all,

Discy
#15
08-19-2018, 11:20 PM
 sanlyn Premium Member Join Date: Aug 2009 Location: N. Carolina and NY, USA Posts: 3,190 Thanks: 107 Thanked 957 Times in 820 Posts
Thanks again for the samples.

Quote:
 Originally Posted by discmeister I am curious about the restoration aspect and what can be achieved, but I saw somewhere else how someone on this forum had spent 90 hours retoring 90 minutes of video.
Only 90 hours for 90 minutes? LOL! Sounds like a fairly easy project. I'm envious. I wish my tapes were so clean that I could get 9 good minutes with 1 hour's work. But I tend to tackle nightmare videos -- and my crazy sister's home camera keeps me well supplied with those! @#$# Some nominal cleanup in Avisynth or VirtualDub can get visibly better results without too much trouble, using a good VCR and capture setup. It's slow at first because it's not as automated as, say, a typical NLE. And no NLE has the kind of controls you get with restoration apps (at least, no NLE that you and I can afford!). For your samples I used a fairly common set of repair tools, although I could have used perhaps one or two less, and it did take some extra time because of faults in the tape sources. The faults are not related to your capture process but are apparently embedded in the source. Overall on your end the captures were well done. Quote: Originally Posted by discmeister Quote:  Originally Posted by sanlyn Your video samples are interlaced, top field first (TFF). It's immediately obvious when you work the file in VirtualDub and Avisynth. How is it obvious immediately, by the way? A technique that I could use on other files? It was obvious on playback (sawtooth edges from bad interlacing). The image below is interlaced frame 32 of your Sample #1, opened with an Avisynth script in VirtualDub. Of course the interlace combing is easy to spot, but you could see it just as easily if you opened the file directly into VDub, without a script. In my opinion the combing is excessive and sloppy, and there is also plenty of aliasing (sawtooth and jaggies) on angular lines. This is sloppy tape mastering. It almost looks like 2nd generation work. One sees annoying "buzzy" edges and lots of motion noise. Borders were cropped off to keep them from affecting the numbers. The numbers overlaying the above frame instantly analyze luma and chroma factors. Here, two white arrows point to luminance that exceeds a value of y=235. The max value here and in other frames from these samples is y=255, which is as high as the graph can read. Y=255 is too hot for TV. YUV luminance at safe levels would be in a range of y=16 at the dark end, to 235 at the bright end. Why? Because the YUV video storage range of y=16-235 gets expanded in RGB to RGB=0-255. If YUV brights are already 255, those values will be clipped (ignored) by a system that can't display anything brighter than RGB 255. In NLE processing, if y is already 255, the NLE will clip many bright values beyond recovery when it expands YUV to RGB for the usual NLE processing. Out of range darks and brights get destroyed, and details are lost. Another thing the numbers show is that there are no dark values below y=16. That's OK, but it does illustrate that the ATI 600 clips all blacks lower than y=16 (aka "super-blacks). Testing shows that it does so before the signal reaches the capture software. If you did have darks below y=16 it would be possible to recover some of the details using tricky YUV processing. But once darks are clipped they're like clipped brights -- no recovery possible. In this case, clipping was mild and easily corrected in Avisynth. Histograms in Avisynth and VirtualDub tell a story about how colors are behaving and what can be done about it. The image below contains three graphic analysis tools that are similar to those found in Adobe Premiere. I'm sure you're familiar with similar Adobe tools considering how much you paid for them. Below, there are three analysis graphs: an Avisynth YUV Levels histogram (left), an Avisynth color vectorscope (middle), and a VirtualDub RGB parade-style histogram (right). These were taken from your Snippet #1 avi. In the left-hand YUV histogram, yellow arrows point to the left and right edges that mark the "safe zone" for YUV luminance between 16 and 235. At the left end, darks are sharply cut off at y=16. The right arrow shows that the white luma band flows into the unsafe zone. The middle graphic is a vectorscope showing that yellow is slightly oversaturated. This, too, was easily corrected. The Snippet #1 shots had a visible yellow color cast. The yellow cast can also be seen in the right-side RGB histogram, where red and green arrows point to color lines climbing up the right-hand wall (note that red + green = yellow). You can also see in the RGB histogram that blue is shoved toward the left (dark) end, so there's a deficit of brighter blues that makes things look too yellow, but it's easy to correct. Then there was some odd garbage in Snippet #1 that didn't appear in Snippet #$2:

Notice the odd cross=hatch and dot pattern on the auto surface as well as bright "sparkles" and other detritus on the side mirror and on other edges and contours. Welcome to the world of mosquito noise and edge ringing, which are strictly digital thingies, not analog. So this segment of your VHS has a digital mastering past, not to mention some sloppy interlacing.

Yes, there are filters that can clean this stuff. But in this case you'd have to use settings so strong, there wouldn't be much video left to watch. It wouldn't be worth the effort. Besides, it goes by so fast in playback that no one would notice. The QTGMC filter that I used did manage to smooth some of it. This problem junk didn't occur in Snippet #2, but Snippet #2 did have some aliasing.

Below, I couldn't help myself. The greenish color cast in Snippet #2 was kinda ugly, really, so I did some corrections with RGB colors in VirtualDub. One of the color tools was gradation curves, just like the expensive one in Adobe Premeiere, Photoshop, and After Effects.

original color (top image) and color fix (bottom):

Quote:
 Originally Posted by discmeister I am interested in how avisynth (or virtualdub) can do a resizing job, mind you. I hate VHS noise around the edge of the frame, but I also hate having black lines anywhere around the picture on my television...
The only video frame that can exactly fill a 16x9 TV screen is a 16x9 frame. TV shows from the golden age of tv are 4:3 tape and interlaced or telecined film. Hollywood made 1:37:1 movies before CinemaScope came along, then 2.4:1, 2.35:1, 1.86:1, 1.6:1 VistaVision, and a host of other formats appear on TV and retail disc, not to forget 1280x740 TV. None of those formats will fill a 16x9 screen. Hopefully all of your time isn't spent playing with a tv remote to distort and blur images to force-fill a 16x9 screen. Of course, it's your tv.

As for resizing, Avisynth can resize cleaner and sharper than Virtualdub, and both can do it better than Adobe and with far less damage. With your two AVi samples, a 4:3 SMPTE image from tape fills only 75% of the actual frame (16:9 and DV video is different). The actual picture area from VHS is almost always 704 pixels wide on a 720-wide frame in both PAL and NTSC. Capturing at 720-wide ensures that you'll get the full image with a workable aspect ratio. You could capture at 704-width if you want, but you'll still have side border pixels and will end up with a smaller image if you remove them.

So what the experts and purists normally do is dispose of unwanted side pixels (note that side borders rarely contain "noise", unless something's wrong with your playback. It's the bottom borders that have head-switching noise). In Avisynth you remove unwanted border pixels, then replace them with new black ones configured to center the image in the frame. This maintains the original image, the original frame dimensions, and the original (i.e, correct) image aspect ratio.

In the case of your two samples, I removed unwanted pixels from all four sides. This left us with a 688x568 frame in YUY2. That frame size isn't exactly the 12:11 pixel ratio to get a 4:3 display on playback. I then resized the height to 576 and resized the width to 704 instead of 720. It happens that 704x576 is valid for PAL 4:3 video. You won't get exactly 4:3 on playback, it will be about 0.2% off -- but who can notice that? There won't be black lines around the physical frame. But, then, there are always black lines in frames that don't fill wide screens, but they blend in with the blacks that the tv puts there to fill the frame. So resizing to replace border pixels is something rather pointless with modern tv's, and is also a lossy proposition. Resizing always costs. Always, no matter how it's done. In this case the resize was small and Spline36Resize is about as clean as you get these days without spending seven figures for software. It's always better to let your hardware players (disc player, TV, etc) do upsampling. They're better at it than you'll ever be with PC software.

In making the attached mp4's I joined Snippets #1 and #2 together as one video. There are two versions, a "25i" interlaced and a "50p" progressive. The 25i still shows remnants of sloppy mastering and grubby edges in the originals, but much less so after filtering, and the progressive version obviously has a smoother look but doesn't have quite the temporal pace of the interlaced. Both play pretty well and will look brighter on tv than on a PC. You'll notice that the motion noise and buzzing edges of the originals are just about gone.

Posted below is the Avisynth script I used for both videos. I can also post a .vcf file for the VirtualDub filter settings used to correct the color cast in Snippet #2, if you wish.

Code:
vidpath="E:\forum\faq\discmeister\"

AviSource(vidpath + "1000 Lakes snippet 2 for DigitalFAQ.avi")

### --- Remove and configure new borders ----#

### --- contrast, gamma, hue, saturation issues ----#
ColorYUV(cont_y=-20,off_y=-8)
Levels(20, 1.2, 255, 16, 250, dither=true, coring=false)
Tweak(Sat=0.90, starthue=165, endhue=180, dither=true, coring=false)
ConvertToYV24(interlaced=true)
SmoothTweak(hue1=5)

### --- deinterlace setup and noise reduction ----#
ConvertToYV12(interlaced=true)
AssumeTFF()
QTGMC(preset="medium",EZDenoise=8,denoiser="dfttest",ChromaMotion=true,border=true,\
ChromaNoise=true,DenoiseMC=true,GrainRestore=0.3,StabilizeNoise=true)

### --- smooth excessive combing ----#
Vinverse2()

### --- smooth aliasing and jaggies ----#
ConvertToYUY2(interlaced=false)
Santiag(2,2)

### --- add film-like fine grain to mask over-filtering ----#

###---Remove borders, resize to fill frame ---###
Crop(16,4,-14,-4)
Spline36Resize(704,576)

###---To RGB for VirtualDub filters ---###
ConvertToRGB32(interlaced=false,matrix="Rec601")

Attached Images
 snippet1 f32 Analyze A.jpg (113.1 KB, 63 downloads) snippet1 f32 YUV Vector RGB panels A.png (57.2 KB, 63 downloads) crosshatch_mosquito noise.jpg (102.9 KB, 63 downloads) Snippet2 Before_After.jpg (86.2 KB, 65 downloads)
Attached Files

Last edited by sanlyn; 08-19-2018 at 11:32 PM.
#16
08-20-2018, 03:18 AM
 discmeister Free Member Join Date: Apr 2018 Posts: 31 Thanks: 5 Thanked 0 Times in 0 Posts
Wow, lots to digest here. Many, many thanks.

It looks like at the very least I should be doing a colour check on all of my capture AVIs to make sure they're within the boundaries you specify. How do you get up the luma and chroma overlays like you've illustrated? Also, is this something I could have avoided in the first place by adjusting the Levels in VirtualDub for capture?

You're crediting me with too much knowledge of Premiere Pro, I'm afraid. I buy the whole CC suite on a monthly sub for lots of other reasons - Photoshop and InDesign among them - so I'm not that experienced beyond the basics that I've already outlined in this thread.

I need to get my head around the resizing stuff. I can work out how many pixels to remove by doing a 'dummy' crop in VirtualDub - then get AviSynth to do it for me, right? Long ago, at the very start of this process, I did experiment with stretching captures to fill a 16:9 box in Premiere Pro (I know, I know...). It looked awful and something at the back of my mind told me I was wrong for adjusting the aspect ratio - so the plan is for the output files to be 4:3, yes. If someone (me, probably) wants to get their media player or DVD player to bodge the stretch to fill a TV screen, then that's fine, because it's on the fly and doesn't hurt the original output file.

Am I right in thinking, though, that the workflow would be something like the following? Capture to AVI with Lossless Compression --> Colour-correct and resize via AviSynth, outputting back to AVI with Losseless Compression --> Do basic fade in/out and chapter marking in Premiere Pro before sending to... --> Adobe Media Encoder for encoding to MPEG2 and H264 files.

Thanks again. Hugely impressive work and has definitely persuaded me that it's worth trying to make the best of these old tapes before they get committed to H264 and MPEG2. Incidentally, they are pre-broadcast tapes provided to one of the production team from within the BBC back in the nineties. I'm sure he'll be interested to hear your remarks on mastering!

Kind regards,

Discy
#17
08-20-2018, 09:57 AM
 sanlyn Premium Member Join Date: Aug 2009 Location: N. Carolina and NY, USA Posts: 3,190 Thanks: 107 Thanked 957 Times in 820 Posts
Quote:
 Originally Posted by discmeister Wow, lots to digest here.
I'm the first to admit that a learning curve is involved. It would be spectacular if there were a single user guide like Adobe's online help (but After Effects' guide took me months to digest). Not everyone is up to complete restoral, and perfection isn't really attainable anyway, so most users tackle only the worst defects and leave the rest to the playback gods. Knowledge and tips about particular problems is learned from browsing restoration forums and asking questions. Fortunately, once you understand some basics about video, little of it changes.

Avisynth and VirtualDub install help files. The internet has tons more on Avisynth via its wiki. Ask Google about "Avisynth" plus the name of a function such as "histogram", and you get this page: http://avisynth.nl/index.php/Histogram. In learning to read histograms as a guide to photo and video correction, I encountered many free internet tutorials about Adobe's tools. For instance, these two guides apply to photo and video alike:
Understanding histograms Part 1 and Part 2
http://www.cambridgeincolour.com/tut...istograms1.htm
http://www.cambridgeincolour.com/tut...istograms2.htm
VirtualDub has several filters that mimic fancier tools in Adobe and Vegas.

Quote:
 Originally Posted by discmeister How do you get up the luma and chroma overlays like you've illustrated?
The images were composed in Photoshop. But there are free apps around for assembling such demos. The numeric overlay uses a built-in Avisynth function:
Code:
ColorYUV(Analyze=true)
and the resulting frame is copied from VirtualDub while viewing the script's output. Panels for individual histograms were made with direct Windows clipboard copies from VirtualDub, then cropped and assembled in Photoshop.

Quote:
 Originally Posted by discmeister It looks like at the very least I should be doing a colour check on all of my capture AVIs to make sure they're within the boundaries you specify. ... Also, is this something I could have avoided in the first place by adjusting the Levels in VirtualDub for capture?
Checking incoming signal levels in VirtualdDub is something of a clumsy chore. Different capture devices behave differently. But once you've done it, things fall into place. Analog source levels are so unpredictable they're infuriating, but usually a brief run-thru of a section of tape lets you set up for worst-case scenarios, and you're allowed some occasional overrun. The rest is in post processing. Trying for color correction during capture is an exercise in futility -- analog colour will change minute to minute, even with retail tapes.

There is an updated and advanced VirtualDub capture guide that discusses setting levels, using the histogram, and other frustrating details: Capturing with VirtualDub [Settings Guide]. Posts #3 and #4 have sample photos and other details about reading the histogram, proc amp settings, temporary crop, etc.

Quote:
 Originally Posted by discmeister Am I right in thinking, though, that the workflow would be something like the following? Capture to AVI with Lossless Compression --> Colour-correct and resize via AviSynth, outputting back to AVI with Losseless Compression --> Do basic fade in/out and chapter marking in Premiere Pro before sending to... --> Adobe Media Encoder for encoding to MPEG2 and H264 files.
That workflow is commonly used. At the end of a project I don't keep the intermediate files, but I save the scripts and VirtualDub settings in case I decide to re-visit later. Most of the time investment is filtering. Avisynth and VirtualDub have hundreds of filters. Most are special-purpose for specific tasks and are fast filters. But for videos with many problems there are the heavy-hitters that incorporate many support filters and take their sweet time executing all the work they do. QTGMC is one such heavy-hitter and is the best deinterlacer around, good enough to be used often as a denoiser. It has many default presets from very fast to very slow -- the settings I used on the two troublesome sample were very slow, running at about 4 to 5 frames per second. The second-best deinterlacer that doesn't work any particular cleanup but gives decent results is yadif, used in Avisynth and VirtualDub and by several media players.

For setting up border work I use a handy VirtualDub GUI filter called BorderControl, then adjust borders in Avisynth. You can get BorderControl here: BorderControl235.zip. The only downside to that filter is that you can't save its settings with a VirtualDub .vcf file. Well, you can't have everything!
 The following users thank sanlyn for this useful post: discmeister (08-20-2018)
#18
08-20-2018, 11:20 AM
 discmeister Free Member Join Date: Apr 2018 Posts: 31 Thanks: 5 Thanked 0 Times in 0 Posts
Yet more to digest here!

One question (well, lots of questions, but I'll get this one out of the way). You keep mentioning DeInterlacing. Would the file in the workflow I described be transformed from an interlaced one into a deinterlaced one during the process 'Colour-correct and resize via AviSynth, outputting back to AVI with Lossless Compression'? If so, I then have a deinterlaced AVI file that I can import into Premiere Pro - for outputting as a deinterlaced H264 file but, potentially, a REinterlaced MPEG2 file for DVD?

Sorry, I'm being a pain. But you've really got me thinking here and I'm going to start playing with AviSynth to do some basic stuff, at least.

Discy

Oh, and if you do have that vcf file on the color cast correction, that'd be very useful to see...
#19
08-20-2018, 12:07 PM
 sanlyn Premium Member Join Date: Aug 2009 Location: N. Carolina and NY, USA Posts: 3,190 Thanks: 107 Thanked 957 Times in 820 Posts
The 50fps deinterlaced video was re-interlaced this way in Avisynth:

Code:
AviSource("drive:\path\to\progressive_vid.avi")
AssumeTFF()
SeparateFields().SelectEvery(4,0,3).Weave()
The output would be 25fps interlaced. The code above is a standard way of doing it. It is a bit tricky because progressed video doesn't really have two different "fields", it's just treated as if it does. SeparateFields() produces two identical half-height images from each frame. Two frames will produce 4 half-height fields. SelectEvery (4,0,3) takes images 0 and 3 from each set of 4 images (frames and fields start counting from 0), then weaves them together ("Weave()") to form one interlaced frame.

How to use a VirtualDub .vcf file:
A .vcf contains filter and settings as well as any compression you've set up. The attached .vcf sets filters only, so unless you change something the output will be the default uncompressed RGB. Set color depth and compressor before you save the output. I used Lagarith YV12.

Save the .vcf file in a folder by itself, not in the plugins folder. When you want to use it in VirtualDub, Click "File.." -> "Load processing settings", locate and select the .vcf, and click OK or "Open". The filters will load with the settings I used. Note that if you have other filters loaded in VirtualDub, the .vcf will overwrite them.

The two filters I used were ColorMill and gradation curves. If you don't have them, they've been attached in earlier posts here: ColorMill21.zip and GradationCurves.zip. Place the .vdf filters in your VirtualDub plugins folder. If I recall, the help files that came with those filters are also included. If they are placed in the plugins folder as .txt or .html files, clicking "Help" in the filter's GUI will show the help content.

Attached Files

Last edited by sanlyn; 08-20-2018 at 12:20 PM.
#20
08-22-2018, 09:05 AM
 discmeister Free Member Join Date: Apr 2018 Posts: 31 Thanks: 5 Thanked 0 Times in 0 Posts
Okay, so I've decided to be realistic about this. I have to offset the desire for perfection with a) the likelihood of it being achieved and b) the amount of time it would take to even try. So I'm going to go for the following...

AVI Lagarith files captured in VirtualDub via ATI USB 600.
Check for contrast, gamma, saturation and hues - fix as required, in AviSynth
Crop to remove black bars at either side of AVI file, and head-switching noise, in AviSynth
Resize that to 704x576, in AviSynth.
Output back to an AVI Lagarith file.

What I'm not sure on is whether I output an interlaced file or not. I have to deinterlace it at the start for AviSynth to do its magic, right? If that's the case, I then have to decide whether to interlace it again or whether to drop it into Premiere Pro for final edits as a 50fps AVI Lagarith file...

Regards to all,

Discy

-- merged --

I'm beginning to work my way through this. I've set aside all of Monday to just play with AviSynth and see if I can do the basic colour/gamma/contrast check and cropping that I want to do.

Looking at your script, I understand a fair amount of it (!) - the likes of Crop and AddBorders will just be adjusted for my own files, depending on what the original AVI is like. It would seem to make sense to smooth excessive combing where required, too.

I'm confused, though, on why you keep changing color modes - first to YV24, then to YV12, then back to YUY2 (like the original AVI) and then RGB32 (for VirtualDub filters? But haven't you adjusted the levels already in the script?).

Also, do you think the final output file that I should aim for should be interlaced or not? If it's been deinterlaced properly by AviSynth (because that is seemingly required for some of the functions anyway?) then wouldn't it be better to output it as a deinterlaced AVI? I could then import this into Premiere Pro for the final fade ins/outs and it'd hopefully result in a better export to H264 - which is, after all, the file that I'll be watching myself. The MPEG2 DVD file could then be TFF via Adobe Media Encoder (MPEG2 DVD has to be interlaced, right?)?

Just checking. Sorry for so many questions.

Kind regards to all,

Discy

 Similar Threads Thread Thread Starter Forum Replies Last Post TerryJW Restore, Filter, Improve Quality 1 06-21-2016 04:25 PM Tytanic Capture, Record, Transfer 12 05-10-2016 04:03 PM unclescoob Restore, Filter, Improve Quality 6 10-25-2011 11:12 AM admin Capture, Record, Transfer 2 12-02-2009 12:42 PM admin Videography: Cameras, TVs and Players 0 01-22-2009 07:47 PM