Filters/scripts for US broadcast TV VHS conversions?
 

Hi All;

Short Version: Would anyone be willing to share examples of their most used, go to filters and scripts for making adjustments to VHS captures of 1980's and 1990's era US broadcast TV.

Long Version: 2 years ago I made my first attempt (with some great help from this forum) at building a reliable workflow and understanding what to do post capture. Other priorities caused me to put things on hold. But now I'm back and ready to dive in. My priorities are to start with the 40 or so VHS broadcast TV SLP tapes that I have from the late 80's and early 90's and extract the unique things such as commercials, news breaks, station ID's. I like that kind of thing. The shows might get captured but unless they aren't available elsewhere will likely not get processed. Final format will likely remain on computer disk or maybe get posted online for others to enjoy. I don't expect to burn anything to DVD but you never know.

Thanks to help on this forum I feel like my workflow is quite good.
Panasonic AG-1980 - from T.Grant - I love this machine!
JVC HR-S9800U - from eBay and not as good as the AG-1980, not even plugged in.
Data Video TBC-3000
ATI 600 USB
S-Video between all devices
Lenovo T500 laptop with the discrete graphics card, Windows XP SP3
OS and Capture Disks are SSD
Sony TV for monitoring the VCR Video Out and Audio to allow Virtual Dub to capture without Preview/Overlay Video and Audio Monitor.

I bought a SpyderXPro to calibrate my monitor but that was before reverting to XP and it doesn't work on XP. Any recommendations for calibrating a modern LED monitor on XP?

Captures that I've done have been very good, although I'm still experimenting how best to adjust the TBC and how to use the histograms but I'm getting there. Dropped frames is single digits or 0, audio is in sync and I've had no issues using HuffyUV or Lagarith to reduce disk space by 2-3x.

2 years ago lordsmurf and sanlyn were kind enough to analyze a capture or two of mine. The explanations were excellent and I intend to re-review those at length before I get too far along. The video I had uploaded was from a Star Trek TV episode. I've since decided that I want to process the commercials, news clips, etc... and not the TV show.

One thing that has been puzzling me from their analysis was the use of 3:2 Pull Down. I fully understand the why - the TV show was shot on film at 24 fps and broadcast at 29.97 fps so there was a conversion. What I don't understand is how to know when this is done and when to undo it in Avisynth. I did just Google this explanation from forum post 8562 - "Open the AVI in VirtualDub, play frame by frame and look for horizontal motion. You will see 3 progressive frames and 2 interlaced frames in every group of 5 frames. This is 3:2 pulldown, which is a form of telecine." Is that really the best way to know? And what about the commercials in those shows? I remember in the late 80's when I worked in broadcast TV and we shot local commercials we used 2 inch video tape. So how best to know when to 3:2?

Also a lot of what I've seen on this forum, understandably, is VHS conversion of home movies. So I feel that maybe some of the recommendations in many posts may not apply to broadcast TV VHS.

I guess what I'm asking is this. There is a ton of info on this forum, I've read a lot, I've played with maybe a dozen scripts one piece at a time and I've gotten a bit overloaded. If someone could share with me a best-of set of their most commonly used tools (with an emphasis on broadcast TV) I'd be very grateful and it might get me heading in the right direction a bit faster. OK, that seems pretty lazy on my part but I'm asking because I'm spinning my wheels a bit and could really use a good kick start.

Thanks,
Bill

This is one of the scripts i used the most (on many vhs sources over the years) if that can help you.

Some things I recognize, some I do not. Lots to play with. Thank you, Bill

There's nothing wrong with that explanation for checking 3:2 pulldown.

But most commercials from the 90s and some from earlier will be a mix of underlying framerates due to different sources, graphical elements, and edits: 24, 30Hz progressive, and 60Hz interlaced. I wouldn't bother trying to process these separately for an ad. Just deinterlace them.

Why do you think there is some major difference between the filters you would use on noise from broadcast TV tapes and the noise you encounter on other home-made and retail tapes? Are these tapes recorded from TV broadcasts (I'd guess so, if they have commercials). Do those tapes have that grungy floating tape noise that you see on all tapes? Why would you use a different filter to clean it ?

Don't imagine that tapes recorded from TV broadcasts don't have signal level problems. They do. The steps you take to resolve crushed blacks and/or blown-out highlights because of bad signal levels are no different for tapes from home cameras or retail VHS tapes from the CRT era. Don't nurse the myth that later digital TV broadcasts have no noise. Digital broadcasts have defects that didn't exist in analogue or film; those defects are compression artifacts. I have lots of recordings off HD digital TV that are rife with compression artifacts, and a slew of old VHS recordings off cable TV from the 1990's that have digital "mastering" garbage I seldom saw on film or on good retail VHS.

Suit the filter to the problem, whatever the video's source. If you're dealing with only commercials and news footage, you'll have mostly telecine from the 80's and 90's, although digital video was being used more often during the late 90's -- and that digital video was shot as both telecined and interlaced. After you solve the frame structure differences, guess what you're left with: good ol' VHS noise, the same VHS noise, chroma junk, dirty color, edge halos, dropouts, spots, and all the other booboos that come with all VHS tapes, differing tape by tape and with every TV program, every TV commercial, and every TV news clip. Two problems unique to 80's and 90's broadcast TV are transmission noise and signal level problems. Commercials had annoyances such as wildly varying color balance -- and anyway, since when was VHS color all that consistent to begin with?

Give me a day or so and I'll try to get up a demo of some early 90's TV captures I archived somewhere on external drives.

Maybe someone would be kind enough to take a look at the first commercial that I'd like to work on and let me know what you think. A full blown filtering & fix with explanations (sanlyn style) would be very much appreciated if someone has the time. But anything you can do to point me in the right direction is very much appreciated.

When this was captured I didn't take advantage of the TBC-3000 and adjust for 16-235. As I look at them they don't seem to get too bright. Maybe I will re-capture and fix that. On capture and for these shorter avi I compressed with Lagarith. For the shorter segments I used Fast Recompress. Let's hope I upload the videos correctly. Apologies if I don't.

One of the things I'd like to know is how to handle multiple things that need fixing in a 30 second spot?

In Firestone-1-lag.avi, the very first frame has all kinds of rainbow crud below the license plate. This was just after a scene change - this isn't the start of the commercial. And in the first 5 frames, even though the camera is steady, when you look at the "Levels" histogram, it's bouncing all over the place. Fascinating!

Next, the man in the red shirt and the yellow car. "Levels" shows a lot of yellow and a lot of red. That seems to make sense. Although looking at luma I'm guessing we need to push everything a little to the right - maybe get some better details from the dark car grill.

The Firestone sign against the sky seems about right but notice how the left side that previously had a really wide gap (that needs a Crop and Border added) is now much thinner in this scene. Better yet in the next scene with the red stoplight the left border junk is wide again. Is this the original spot or the VHS?

Next around frame 115 the red T-Bird. Why is this so heavily blue? I suppose it could be it's a rainy day?

And finally the two men and little boy. "Levels" says this is heavy on the red and yellow. Luma seems a little low again, so maybe proc-amp and recapture?

Firestone-2-lag.avi, the first scene with the $49.95 the numbers all have a halo outside of the white border. What filter fixes this?

The drivers arm shifting the car - this looks terribly red to me. "Levels" agrees but again, is it the original or something that needs to be fixed? And if I lower the red here how do I do that without affecting the rest of the commercial?

Next the 3 happy Firestone guys. That doesn't look too bad to me or "Levels" although again, I think it might need more brightness during capture.

Then the 2 guys looking at a wall of tires - lots of red again and a little weak on brightness.

The superimposed Radials 4/$100 again has the outside halos like earlier and the final scene with Firestone letters on a black background also seems to need some brightness?

Funny, now that I wrote this all out it seems like in almost every case I'm in need of lowering red a bit.

I humbly ask that someone take a look, crucify me for what I misunderstand and help me take a step or two forward :o

Thanks for all the work you did on the samples, but.....
Why is the frame size a non-standard 708x472? What frame content did you remove? Some frames look as if something was removed or changed, but it's hard to tell. What do you intend to do with this frame size?

Overall noise isn't too bad, should be the easier part. Color is pretty messy, which will be difficult -- not a surprise with many old analog broadcast sources. The toughest part will be the oversharpening halos and edge ghosts. Have you increased sharpening on the AG-1980 ? (the sharpener is disabled when the slider clicks into the center indent in the middle of the scale).

Can't work on this with a mod-4 frame dimension, as many filters especially in Avisynth require at least mod-8 in all dimensions. A 708 pixel width is not mod-8. It's mod-4. The preferred frame size for NTSC VHS is 720x480, or at least 704x480 or 640x480.

Yikes, I captured this in 2018 and assumed in 2020 that I did it right. That might explain why there wasn't any crud along the bottom edge as I cropped and didn't add a border to bring it back to 720x480. The AG-1980 sharpener is centered today but again, in 2018 I really had no clue. I will get that tape back out this weekend, recapture and repost. Thank you for taking a look!

I understand, it takes a time or two to work out everything. You'd be surprised at the omissions even the old timers make (yours truly included). As they say, the devil is in the details.
:wink2:


In the meantime I'm putting together a demo of 3 or 4 1990's commercials with lots of problems to consider. Sorry for the delay, this COVID19 rigmarole makes every daily chore take 3 times longer than it should.

If that means that you will be posting before and after samples that includes your amazing explanations of what was done and why that will be VERY much appreciated. I'm happy to wait for that for as long as it takes!

In the meantime I have recaptured the Firestone commercial from 1986 if anyone might care to take a look. Nothing has been done besides compressing with Lagarith. I didn't even crop and border but this time we do have 720x480.

Interesting that for the first time I noticed a ticking noise in the audio. It is there throughout the entire recording (of Miami Vice) but doesn't happen on the next show that was recorded in 1992 with a better VCR. So it seems my old VCR is at fault. I will either have to live with that or do some more repair work. I have an older copy of Adobe Audition and could record the audio, isolate and remove the noise and then put the clean audio back with the yet to be filtered video. But let's take baby steps before we go there.

BAD TAPE CHRONICLES: TAPES FROM HELL, CONT'D:
Working with bad cable signals and bad tape from the 1990's, #1

Thanks for the new captures. I'll be look them over tonight.


The attached "Colgate_8661_sample.mpg" is a high bitrate encode of an otherwise unfiltered lossless capture from a TV broadcast. The attached "Colgate_ pv_s4670_sample.mpg" is a high bitrate encode of another otherwise unfiltered lossless capture from a TV broadcast, using a different VCR. The original lossless YUY2 samples were too large for posting here, but if you want a lossless copy request in this thread. I can mount one in my pro account at Mediafire (no popups, no ads).

The specs below will leave no doubt that in the early 90's I had no idea what the heck I was doing with video gear. A lousy crummy underpowered cable signal in my apartment building in NYC didn't help. My habit of capturing at 640x480 stemmed from the irrational idea that 720x480 wouldn't "look right" in my editors. But you can tell my editors which display ratio to use, silly boy. Live and learn.

Date recorded: October 1990 (based on other ads on the tape promoting cable programs and dates).
Original recorder: Mainstream JVC 4-head onto Maxell medium-grade tape at 6-hour slow speed.
Original broadcast: RF cable line -> 10w cable signal amp -> cheap RF cable -> analog cable box -> cheap RF wire -> cheap Radio Shack 4-way amplified RF splitter with 4 composite outputs -> cheap composite wire to VCR.

Captures (2008): Rebuilt Panasonic PV-8661 VCR (circa 1988), and a second capture using a rebuilt Panasonic PV-S4670 SVHS player (circa 1986)
-> AR blue-jacket composite -> Panasonic DMR-ES15 tbc passthru
-> s-video -> AVT-8710 TBC (circa 2004)
-> s-video -> ATI AIW 9600XT AGP card
-> VirtualDub/huffyuv YUY2, 640x480.
Original capture reviewed in VDub and cut-edited into lossless sections, archived to external HDD as Lagarith YUY2. The capture version chosen for final output was the capture played with the PV-8661. I felt the 8661 had the same defects but retained more detail after processing.

From the very start, the first thing one sees is the horrible noise. The opening black frames are almost gray with snowy grain. The apartment building's cable circuitry, really cheap wiring, multiple splitters, and less than perfect VCRs delivered exactly the noisy junk you would expect, and then some. There is so much floating gunk, fuzzy edges and streaking that you think your eyes are going bad. There is visible chroma bleed and chroma shift, over saturated reds, some mild edge ghosting, cable noise that plays as "horizontal rain" and flicker that's most visible in bright solid areas like sky and walls. The noise level and so-so tape quality didn't make for the world's sharpest picture.

How to clean this without destroying everything in sight? I turned to some of my favorite plugins. First, the video was inverse telecined in a separate step and saved as input for further processing. Why? It's difficult scrolling video back and forth in VDub to check filter changes and color if you're trying to run ivtc at same time. The inverse telecine script for all the demo videos that follow was short and simple:

Someone will ask right away: why didn't you use your AG-1980 player? I don't always use it. Some tapes just don't look so great on denoising VCRs, which often cause posterizing effects. Denoisers in premium VCRs were relatively primitive; today's Avisynth is more sophisticated. And some tapes just won't track nicely on some machines.

After inverse telecine, the script below employs several of my most-used VHS filters for very noisy or damaged tapes. I had to use two scripts, versions A and B. Version A was for the full length of the short commercial. Version B had script changes specifically for the really bad noise in the first 63 opening frames and the fadeout in the last 7 frames. Then B and A results were edited and joined in an output file script.

The code for version A:

Output from Script A was saved in Virtualdub as Lagarith YUY2 using "fast recompress" mode. Below is the Version B script for the very bad noise sections. It;'s the same as script A, but some lines at the very end use 16-bit processing for the worst noise:

The Script for version B:

The script below trims 2 parts for the leading and trailing frames, then trims a 3rd section dfor the bulk of the commercial. Then the script joins all three parts for encoding:

Some detailed notes on filters used in the A and B scripts:

AVISource("I:\forum\faq\billct97\Colgate\Colgate2_ ivtc.avi")
ColorYUV(off_y=-5)
ConvertToYV12(interlaced=false)
### --- Next 3 lines are activated for testing only ### ---
## Crop(2,4,-22,-4)
## Histogram("Levels")
## return last
The file is opened with AviSource(). ColorYUV(off_y=-5) lowers the black levels slightly by lowering all values in the video's luma and chroma by 5 negative points. This brought the black start and end frames down to exactly y=16, which would equal RGB=0 in RGB display. But the ColorYUV() line wasn't added until after the video's borders were cropped off and levels were analyzed with the Histogram() function. When the 3 test lines are not commented-out with the "#" character, the frame's borders won't affect the histogram readings and the "return last" statement ends the program and returns the results. After the desired settings were determined, the ColorYUV() line was added, the three test lines were disabled with "#" characters, and the program resumed normally. Note that the 3 test lines are not needed in script B because the desired ColorfYUV values were already analyzed in script A.

Stab()
FixChromaBleeding()
ChromaShift(l=-2,c=-4)
There is some slight vertical and horizontal frame hop which is stabilized with the Stab() filter. FixChromaBleeding is a handy plugin that works with chroma only, mainly by reducing over saturation that causes bleeding. It's not perfect but it helps visibly where high saturation is the problem. ChromaShift() works on U and V chroma tracks only, shifting colors upward by 2 pixels and leftward by 4 pixels.

qtgmc(InputType=2,preset="medium",EZDenoise=4,deno iser="dfttest",GrainRestore=0.3)
QTGMC is a deinterlacer but here it's used on slightluy damaged progressive video (InputType=2). Rather than deinterlace, QTGMC is used here for its denoising (with EZDenoise and dfttest), and its motion-compensated anti-shimmer and edge and motion smoothing properties. As used with its "medium" preset parameter it's only mildly destructive of detail, some of which is restored internally and refreshed by re-inserting a small amount of the original grain to prevent an over-filtered look. There are stronger denoising values available, but this video is already a bit soft.

source=last
super = source.MSuper(pel=2, sharp=1)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, blksize=8, overlap=4, dct=0)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, blksize=8, overlap=4, dct=0)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, blksize=8, overlap=4, dct=0)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, blksize=8, overlap=4, dct=0)
MDegrain2(source,super, backward_vec1,forward_vec1,backward_vec2,forward_v ec2,thSAD=400)
In case no one noticed, this routine is copied almost verbatim from the MVTools2.dll motion vector tools documentation. The denoiser is MDegrain2 in the last line of this routine. There are dozens of variations of this routine in several big plugins. The code analyzes noise and motion in 2 frames in each direction. It's good for excess temporal grain and for what are called the "fuzzies" in edges, textures, and other noise patterns. Be careful though, as this filter can sometimes make motion-blurred moving objects disappear for a frame or two.

### --- 16bit dithering filter for noise problem --- ###
Dither_convert_8_to_16()
GradFun3(thr=0.8,mask=0,lsb=false,lsb_In=true,smod e=1)
GradFun3() (aka "gradation function" 3) is from the dither tools plugin, which contains dozens of 16-bit conversion and filtering routines. It's basically a smoother that helps clear color banding, hard transition edges and tough grain in areas of fine gradation between contrasting or color regions and in large areas such as sky, walls, shadows, etc. It can soften the entire video if improperly used, so it's used here only in Script B (it's not in Script A). It also helps avoid hard edges in facial tones. The conversion from normal 8-bit video data to 16-bit is done with standard dithering functions. If you need a couple of evenings' reading time, you can read about the filter in its original appearance at doom9 (Color banding and noise removal) while dithering is covered in many internet posts such as https://en.wikipedia.org/wiki/Dither and hundreds of others. GradFun3 and other features in DitherTools are covered briefly in online docs at http://avisynth.nl/index.php/Dither_tools#GradFun3.

grainfactory3(g1str=1, g2str=1, g3str=1)
AddGrainC(2.0,2.0)
Both functions add very fine, very ordered film-like grain to prevent an overly smooth appearance and create the illusion of optical "detail" with highly filtered media. Don't overdo it or it will be worse than the dirty grainy stuff that was removed.

MergeChroma(aWarpSharp2(depth=30).aWarpSharp2(dept h=10))
LimitedSharpenFaster(edgemode=2)
Merge Chroma() limits processing to only the chroma channels, where aWarpSharp2 literally "warps" color to cling more closely to defined edges -- so it is a sort of anti-bleed and anti-smear function. LimitedSharpenFaster() is an old-time favorite sharpener that doesn't overdo edge sharpening with artifacts, and "edgemode=2" sharpens only areas between edges, not edges themselves, to prevent a phony oversharpened look on soft video.

Crop(2,4,-22,-4).AddBorders(12,4,12,4)
Spline36Resize(720,height)
Crop() removes dirty or discolored border pixels and AddBorders creates new border pixels (the default color is black). Crop() removes 2 pixels from the left border, 4 pixels from the top, 22 pixels from the right, and 4 pixels from the bottom. AddBorders() adds new pixels to the left, top, right, and bottom in a way that centers the frame inside the new borders. The image is restored to its original dimensions without affecting the core image content itself. Finally, Spline36 resizes the 640x480 capture for DVD at 720x480. It would have been better if I had captured 720x480 to begin with, but years ago I didn't know right from left about capturing.

The final encode "Colgate_8661_cleanup_480p.mp4" at original film speed 23.976 fps is attached. Not perfect, but one can do only so much with bad video. Looks cleaner and brighter on TV than on a PC monitor (no surprise).

(continued, next post)

BAD TAPE CHRONICLES: TAPES FROM HELL, CONT'D:
Working with bad cable signals and bad tape from the 1990's, #2


The attached "Dentist_AG1980_sample.mpg" is a high bitrate encode of an otherwise unfiltered lossless capture from a TV broadcast. The original lossless YUY2 sample was too large for posting here, but if you want a lossless copy let me know. I can mount one in my pro account at Mediafire (no popups, no ads).


Date recorded: July 1991 (based on other ads on the tape promoting cable programs and dates).
Original recorder: Mainstream JVC 4-head onto Sony "HI-FI" tape at 6-hour slow speed.
Original broadcast: RF cable line -> 20w no-name RF cable amp -> cheap RF cable -> analog cable box -> cheap RF wire -> cheap Radio Shack 4-way amplified RF splitter with 4 composite outputs -> cheap composite wire to VCR.

Capture (2010): Panasonic AG-1980
-> s-video -> AVT-8710 (circa 2004)
-> s-video -> SignVideo PA-100 proc amp
-> s-video -> ATI AIW 9600XT AGP card
-> VirtualDub/huffyuv YUY2, 640x480. Original capture reviewed in VDub and cut-edited into lossless sections, archived to external HDD as Lagarith YUY2.

Despite my using the AG-1980 for this capture, the noise level is still atrocious -- thanks to the bad cable line, inferior connections, and also to SONY "HI-FI" VHS tape, one of the grittiest, grainiest tapes I've ever seen. Years later I'm sorry I used that tape.

Because the sample's problems are similar to those in the previous example, I used a very similar script and filters. But there are additions: the first half of the sample is supposed to be mostly grayscale and the small amount of color looks ok. But the second half has atrocious skin tones thanks to the no-name cable amp I used and which simply wrecked the colors. It also over-amplified the signal, which gave problems with other VCRs. So the script was run 2 times: once with less color correction for the sake of the more neutral first half, then again with more RGB color correction to get less sickening skin tones in the latter half. At encode time I cut and joined the two results using the TMPGenc encoder app.

As usual, ivtc came first as a previous file used for input here. The script below is like the earlier post but with a few additions, which I'll note below:
Lines that make this script different from those in previous first:

hqdering()
qtgmc(InputType=2,preset="medium",EZDenoise=2,deno iser="dfttest",GrainRestore=0.3,\
border=true,ChromaNoise=true)
the hqdering() filter works on mildly discolored and noisy edge artifacts that are rather common in VHS, even with high end players. Again, QTGMC is used as a smoother and denoiser, not to deinterlace; its "border" parameter avoids border flutter when resizing new frames. ChromaNoise=true activates chroma cleaning.

Dither_convert_8_to_16 ()
Dither_resize16 (720, 480, kernel="spline36", cplace="MPEG2")
Dither_convert_yuv_to_rgb(matrix="601",interlaced= false,tv_range=true,cplace="MPEG2",\
lsb_in=true, output="rgb32")
These lines add 16-bit dithered processing to the resize and the conversion from YV12 to RGB, the latter used for the VirtualDub filters. Greater 16-bit density helps make resizing and color reinterpretation cleaner (but check the results first, as sometimes it can overly smoothed and phoney). The functions are described in the Dither Tools Avisynth Wiki documentation at http://avisynth.nl/index.php/Dither_tools. See the table of contents list for details on filter functions.

The VirtualDub filters that were used are CamcorderColorDenoise, ColorMill, gradation curves, and Donald Graft's hue/saturation/intensity. These filters were used for the color-corrected portion of the clip. The RGB filter that was used for the first neutral-colored half was NeatVideo v3.0 at very conservative settings and low power, basically used as a mild tweak. You can eliminate Neatvideo (it's a paid retail filter) and substitute EZDenoise=6 in the QTGMC line.

The VDub .vcf settings file for the RGB correction filters is attached as "VirtualDub_RGB_color_settings.vcf". The NeatViedo .dnp and .nfp processing files are attached as "NeatVideo_Sample_and_Filter_Settings.zip".

The final 23.976 fps 4:3 encoded results are attached as "Dentist_AG1980_cleanup_480p.mp4".

BAD TAPE CHRONICLES: TAPES FROM HELL, CONT'D:
Working with bad cable signals and bad tape from the 1990's, #3

The attached "WheatThins AG1980 sample.mpg" is a high bitrate encode of an otherwise unfiltered lossless capture from a TV broadcast. The original lossless YUY2 sample was too large for posting here, but if you want a lossless copy post a request here. I can mount one in my pro account at Mediafire (no popups, no ads).


This time I used slightly better tape and a better VCR for its era, a pricey SONY SLV-695HF, although at slow 6-hour mode. Noise was about the same but overall accutance and color were better than most recordings under my stupid cable setup. The opening and closing black frames are a cable noise blizzard. And you can easily hear the low-frequency buzz that was coming over the USA cable channel during the broadcast. This time there were a few complications in the form of spots. This clip is an opportunity to demo a couple more of my favorite VHS filters.

The composite image below displays cropped pics of the kind of spots and freaky edge noise in this clip:

http://www.digitalfaq.com/forum/atta...1&d=1585503647

(Above) Upper left, note the white speck to the right of Sandy's face. Shows up like a lighthouse as the camera pans. Upper right, a broken black line and a white "blink" in the extreme upper right corner of the top border. Bottom left, another stupid white speck on the lower left edge of the yellow box. Bottom right, yellow bleed and ghost image on right side of yellow box, and note all the ugly streaky noise on the white sweater along the edges of the outer blue sweater and elsewhere in the image. These cropped images are from /mpg sample frames 40, 101, 277 and 323.

Date recorded: January 1991 (based on other ads on the tape promoting cable programs and dates).
Original recorder: SONY SLV-695HF on Maxell premium grade tape at 6-hour slow speed.
Original broadcast: RF cable line -> 10w Archer cable RF amp -> cheap RF cable -> analog cable box -> cheap composite wire -> cheap Radio Shack 4-way amplified RF splitter with 4 composite outputs -> cheap composite wire to VCR.

Capture (2009): Panasonic AG-1980 VCR
-> svideo -> AVT-8710 (circa 2004)
-> SignVideo PA-100 proc amp
-> s-video -> ATI AIW 9600XT AGP card
-> VirtualDub/huffyuv YUY2, 640x480.
Original capture reviewed in VDub and cut-edited into lossless sections, archived to external HDD as Lagarith YUY2.

Again, the script is similar to those in preceding posts. I'll give details about the differences below:

AVISource("I:\forum\faq\billct97\WheatThins\WheatT hins_ivtc.avi")
a=last
save_audio = a
a
the AVISource() function opens and decodes the video, then immediately an arbitrary value "a" is assigned to the input. "a" is a value in memory that I invented, which works as long as no other entity in the program is also called "a". In the third line I create a new copy of "a" and call it "save_audio", which saves in memory a copy of the clip with its original audio stream. Frame replacement later in the script will create audio breaks, so Save_audio will be used later to restore the audio. In the last line, clip "a" is named explicitly to restore processing focus to the "a" clip.

ColorYUV(off_y=-8)
Tweak(cont=1.05,dither=true,coring=false)
Levels(16,1.0,255,16,250,dither=true,coring=false)
#Crop(4,0,-20,-8)
#ConvertToYV12(interlaced=false)
#Histogram("Levels")
#return last
ColorYUV is used to lower all luminance values in the clip by 8 negative points for a more realistic black level. Then Tweak() is used to visibly raise contrast to restore the original "snap" of a brighter high end. Because this creates an occasional highlight overshoot past y=235, the Levels() function slightly lowers highlight extension using a value below 255. One arrives at these values by examining the clip in Avisynth's histogram function; the coded lines that create and test the histogram are disabled with "#" characters after they are used. The Tweak() and Levels() functions are documented in hundreds of scripts and in Avisynth's built-in Help documents. Online, one can find the details at http://avisynth.nl/index.php/Tweak and http://avisynth.nl/index.php/Levels. Using the Tweak() and Levels() functions is very basic Avisynth.

FixVHSOversharp(20,16,12)
This is a YUY2-only filter that helps alleviate bright or dark edge halos and sharpening effects. Often these effects originate with the broadcast and are sometimes smeared edges caused by transmission noise. They're also caused by "echoes' from the multitude of RF splitters I was using. FixVHSOversharp() can be used for right-hand or left-hand halo and mild ghost-like effects. You can also download it from the forum as fixvhsoversharp_25_dll_20030723.zip.

ReplaceFramesMC(221,1)
b=last
b0=b
b01=b.ReplaceFramesMC(32,1).Crop(248,114,-382,-356)
b02=Overlay(b0,b01,x=248,y=114)
b1=ReplaceFramesSimple(b,b02,mappings="32")

b0=b1
b01=b1.ReplaceFramesMC(81,1).Crop(120,434,-508,-38)
b02=Overlay(b0,b01,x=120,y=434)
b2=ReplaceFramesSimple(b1,b02,mappings="81")

b2
These lines eliminate the spots and white flecks shown in the composite image posted earlier. Note that the frame numbers here don't refer to the original sample; rather the frame numbers in these lines are from the ivtc'd version, which has fewer frames than the original because duplicte telecine frames are removed by IVCT.

The ReplaceFramesMC(221,1) statement is the lazy way of replacing frame 221 with a new one: it simply interpolates a new frame 221 based on data and motion in the previous and following frame. It works OK here because there isn't much motion. However, with more motion some bizarre distortions would result, so the statements that follow create small "patches" in the video that cover the undesirable defects rather than re-interpret the entire frame.

First, a variable "b" is created to contain a copy of the clip ("last") that has been processed so far. Then another variable "b0" is created to point to "b" for later use. In the next line, "b01" is code that creates a copy of the clip in which a complete new frame 32 is created, and then that frame is cropped down to a small 10x10-pixel patch that contains clean video. This patch will be used to overlay the 10x10-pixel area in the original clip that contains the annoying white fleck. Using this small speck eliminates processing the other parts of the frame where excess motion would cause weird distortions.

The line b02=Overlay(b0,b01,x=120,y=434) creates a new video clip in which the clean new 10x10 patch is overlaid onto the unwanted 10x10 area. This covers the white spot in frame 32 with a new patch of cleaned data -- but it covers the 10x10 patch in every frame of clip B02. That's not what we want. The only cleaned frame we want from clip b02 is frame 32. Therefore, the following statement b1=ReplaceFramesSimple(b,b02,mappings="32") takes only the cleaned frame 32 from b02 and uses ReplaceFramesSimple() to replace the old frame 32 in the original "b". The new video is named "b1". ReplaceFramesSimple is one of three functions is the plugin Remapframes.dll.

The routine that follows this 4-line cleanup creates a different patch from a different part of the new "b1" video and creates a new cleaned version called "b2". The last statement of the routines explicitly names "b2" in order to keep processing focus on the newest b2 version of the video.

santiag()
SmoothUV()
santiag (http://avisynth.nl/index.php/Santiag) is an anti-alias filter used here to do a little more edge and line smoothing. Use it sparingly because strength values stronger than the defaults can seriously soften the video. Here, default values are used because other values are not stipulated in the command line. SmoothUV (http://avisynth.nl/index.php/SmoothUV) is a classic old plugin used as a mild chroma cleaner to smooth away blotchy discolorations. It helps smooth skin tones.

AudioDub(last,save_audio)
return last
The AudioDub() built-in function takes the video that is the "last" thing done in the previous statement and dubs in the audio stream that was saved earlier in Save_audio. This replaces any damage done to the audio by the frame replacement routines. Finally, the statement "return last" ends the program by returning the "last" thing that was executed, which was AudioDub() in the previous statement. Why do we need "return last" here? Because in the course of processing, the script created several video clips (with names like names a, b, b0, b1, b2, etc.) and Avisynth has to know which clip we want returned.

The conversion to RGB is done with 16-bit dithering as described in the earlier post. RGB is used by the VirtualDub filters that are loaded when VDub is executing the avisynth script. The filters used are ColorMill, gradation curves, and the exorcist anti-ghost filter. Exorcist alleviates (but doesn't completely remove) the edge ghosting in the original, but in doing so it darkens the image. Gradation curves is used to restore the original luminance levels. There are also 2 instances of the FadeFX filter at the end of the VDub filter chain (https://web.archive.org/web/20180627...deo/fadefx.vdf). The first instance of FadeFX replaces the dirty fade-from-black frames at the start of the clip. The second copy of fadeFX replaces the last few fade-to-black frames at the end of the clip.

The VDub filter chain's order and filter settings are attached in the .vcf file WheatThins_VDub_Settings.vcf.

The results of the scripts and filters were encoded as the attached "WheatThins_AG1980_DVD pulldown.mpg". The mpg is encoded and configured for DVD with 3:2 pulldown restored by the TMPGenc encoder for 29.97fps DVD playback.

First, thank you so much for giving me more examples than I could get through on a rainy cold Spring day with nowhere else to be. I did have some old notes and HDD cleanup to do first so I was only able to get through the first example but it all made sense. I would like to ask what will likely be very basic questions instead of just accepting what I read and moving on.

1. Why did you encode the final mp4 at 23.976 fps? It would seem to me if you did the 3:2 conversion you would leave it there at 29.97 fps.

2. Virtual Dub gives me an error when I try to open Colgate_8661_cleanup_480p.mp4. The pop up error says "Unable to initialize Quicktime: InitializeQTML". The .mpg files open and play just fine in Virtual Dub. The .mp4 opens and plays ok in VLC. I'd assume I need to install a codec in Virtual Dub. Just not sure what codec and how it is done.

3. What do you use to convert to MPEG for the final copy? My plan is to archive everything as lossless compressed with Lagarith but I may also want to upload to YouTube or share a copy with friends. So a balance of portability and quality is what I would think I would want for a final format?

4. Since you offered I would like to have a copy of the original lossless file(s). Whatever you are willing to share. That way I can work with your scripts and the original avi until I have succeeded in replicating your results.

Thank You!
Bill

By the term "3:2 conversion" I think you meant inverse telecine. Inverse telecine removes pulldown fields, so the video no longer plays at 29.97. When pulldown is removed, the video will play at its original frame rate which in this case is 23.976fps. If you want it to play AT 29.97 again, pulldown must be restored when the video is encoded.

If you want to mount the video on the internet, it can't be telecined. So you will need a progressive 23.976 version for the 'net. Also, the last time I heard about YouTube submissions, video intended for 4:3 display aspect ratio has to be submitted in a certain frame size, or YouTube will reformat it for you using the quickest and dirtiest means possible. They prefer submissions in 720p format. 720x480 can't be used on YouTube. The 4:3 video has to be resized to 960x720, then 160 black border pixels are added to each side to make a 4:3 video enclosed in a 1280x720 frame. Anamorphic frames (720x480) can't be used.

In Avisynth you can do it this way:
Don't try this with interlaced video. If it's interlaced, it must be deinterlaced for resizing or for posting to the internet. If you submit interlaced video it will be deinterlaced using crappy fast and dirty software that you'd never use yourself.

If you want the 720x480 progressive 23.976 video to be on DVD or standard def BluRay, it must be encoded for a 4:3 display aspect ratio (DAR) and 3:2 pulldown must be added. When authored to DVD or BluRay disc, the authoring program will treat the telecined video as "interlaced". If you want to stream it as 23.976 or place it on a hard drive or USB stick as "data", encode it as-is for a 4:3 DAR and play it as 23.976.

However -- if you want to combine your progressive 720X480 23.976 video with interlaced 29.97 video, you can't do it. You have to encode the 23.976 video with pulldown for 29.97fps playback, and treat it as interlaced. Then you can combine it with 29.97fps interlaced sections.

You need the ffmpeg plugin. Attached is a copy of 32-bit plugins for VirtualDub. Download it to a separate area on a hard drive and unzip it. It will unzip into a folder called "plugins32". You should also have a "plugins32" folder in your VirtualDub program folder. if you don't, create it. Then look at the .vdf plugins inside the unzipped Plugins32 folder and add the .vdf's that you need to your VirtualDub "plugins32". The particular .vdf in question is the ffmpeg input plugin.

I used TMPGenc Mastering Works for both the mpg and the mp4/h.264 encodes. Sometimes I use Avidemux for h.264 (it's not a very good MPEG encoder).

Colgate_Number2 8661.avi ~190m https://www.mediafire.com/file/nymgv..._8661.avi/file
Colgate_tune pv-s4670.avi ~189mb https://www.mediafire.com/file/iudpe...s4670.avi/file
GI1_e_dentist Ag1980.avi ~182mb https://www.mediafire.com/file/jogh2...g1980.avi/file
Living4 - WheatThins AG1980.avi ~97mb, but the forum says >99 ???
https://www.mediafire.com/file/a7guj...G1980.avi/file

I added the ffmpeg plugin (thank you for that). The video now plays in Virtual Dub (if I check No Audio) but I get this error: "No audio decompressor could be found to decompress the source audio format. (source format tag: 00ff)". I also added lordsmurfs bundle from post 5895 but I believe those are all Avisynth filters.

Install the system-wide AAC/ACM codec. https://www.videohelp.com/software/AAC-ACM-Codec

Got it working. I also needed ffdshow or I got a "missing codec AVC1" on a black screen in Virtual Dub.
1. ffmpeg (sanlyn posted the plugin earlier)
2. AACACM19 (sanlyn posted the link)
3. ffdshow - https://www.videohelp.com/software/ffdshow

Yes, there is always something else! I've had ffdshow for so many years I keep forgetting it's there.

Taking things very slowly as I had to work today. Can you please fix the IVTC line in Post #11 as it tripped me up for a few minutes. If someone who is new finds this later on I suppose it would be good to have it corrected.

TFM().(Tdecimate) should be TFM().Tdecimate()

Holy smokes! How did that typo sneak by? Anyway, your version is correct. Unfortunately posts can't be edited 1 hour after they post. Shucks. I'll try sending a note to a moderator. The other scripts shouldn't have this problem, they were all cut and paste from the originals.



Meanwhile I tried to post some pics and results for Firestone_1, but the cable company is working in my neighborhood and the internet keeps popping on and off. I'll wait until they're finished and post later tonight.

This post on Firestone_1.avi introduces a couple of filters.

There's a thread from a few years back that goes into detail on scripts, histograms, and VirtualDub controls. A post entitled "Information Overload #3" demonstrates the VirtualDub Levels, ColorMill, and gradation curves adjustment controls, which I used extensively working with the Firestone caps.

Another essential tool is the CSamp pixel reader (http://www.digitalfaq.com/forum/atta...-time-csampzip) -- a free desktop standalone gizmo that comes in very handy. Using it is covered with pictures in post #39 in another, older thread. The later thread also discusses the very important ColorTools histogram. There\'s a link to it, but it\'s the old version that won\'t work in Win7 or later. The updated ColorTools is at http://www.digitalfaq.com/forum/atta...1&d=1487006540.

I had a fairly tough time with Firestone_1.avi. Every camera shot in that ad has different problems. Some of the glitches are common to every shot, such as the bad edge ghosts. All have pretty much the same moderate noise signature which isn't nearly as noisy as my own caps posted earlier. You controlled signal levels pretty well. Because each shot was so different I had to split the video into separate shots; the scripts for each are similar but with minor differences, and the color was -- well, it's no surprise from the days of analog broadcasting that colors were a mess. Your decisions about the color might differ from mine, but hopefully the details I post will be helpful.

The results of the work were encoded with restored 3:2 pulldown for DVD and is attached as "Firestone1_cleanup_DVD_pulldown.mpg". It was encoded with my old standby, TMPGenc 2.5 plus, and video and audio were remuxed with Dolby Ac3 in TMPGenc Smart Renderer (the old 2.5-Plus is so old it was never updated for Dolby audio!).

The first step was to remove telecine from the entire video, then work with individual scenes. First, I inverse telecined Firestone_1.avi and saved it as Firestone1_ivtc.avi:

Firestone1_ivtc.avi was saved out of VirtualDub as Lagarith YUY2. I then used 5 different scripts to cut Firestone1_ivtc into 5 different scenes, or working files, named sc1 thru sc5. This is not unusual especially for analog broadcasts, which contain many wild variations.
http://www.digitalfaq.com/forum/atta...1&d=1585739477
The image above contains two color schematics based on the original unfiltered frame 32 of scene 1 in inverse telecined Firestone1_ivtc. The chart on the left is a 4-band RGB histogram of color values in the frame, showing brightness (white), Red, Green, and Blue. The chart to the right is an RGB vectorscope which is basically a picture of color saturation in the frame.

The pic below is unfiltered frame 32 of scene 1. In the RGB histogram above you'd expect lots of yellow, but the strongest color is the Red band with slightly less Green. Blue is easily the shortest band, indicating a deficit of bright Blue. In the guy's hand and face, skin looks a little dim and pumpkin-colored. Most of the "black" in the car's front grill is actually blackish green. In the vectorscope, the biggest color blob is red. The vertical line that extends upward and slanted to the left is where flesh tones ideally belong but there's just a muddy mix on that line. The other dots asnd squiggles are disorganized remnants of green, cyan, and blue. In the frame you might also note dot crawl on the right-hand border.
http://www.digitalfaq.com/forum/atta...1&d=1585739559

http://www.digitalfaq.com/forum/atta...1&d=1585739610
The image above is frame 32, filtered and color-corrected. The script and the settings for the VDub color controls are described below. You've seen most of the Avisynth filters in previous scripts:

AVISource("I:\forum\faq\billct97\X1\F1\Firestone1_ ivtc.avi")
Trim(0,36)
AviSource() opens and decodes Firestone1_ivtc.avi and uses only the first 37 frames, which is Scene 1.

Import("D:\Avisynth 2.5\plugins\ContrastMask.avs")
contrastmask(enhance=2.0)
ContrastMask.avs is a specialized filter for correcting poor luminance in shadow areas without affecting the rest of the spectrum. In practise it needs some fixing in the upper midrange around RGB 64 to 128, which it sometimes over-brightens. Generally this video seems to have overly dense darks. Because contrast mask is an .avs scripted filter, it has to be added to the script at runtime with the Import() command. The default "enhance" value is 10, which is almost always too bright. The .avs is at http://www.digitalfaq.com/forum/atta...ontrastmaskavs. It requires the MaskTools plugin that comes with QTGMC. It also requires VariableBlur.dll, which in turn requires a Microsoft VC+ runtime and the fftw3 library, which are linked on Variable Blur's web page at http://avisynth.nl/index.php/VariableBlur. The fftw3 library also comes with the QTGMC package.

The conversion to RGB32 is for VirtualDub's filters. The filters used were ColorCamcorderDenoise, ColorMill, exorcist, and gradation curves. The main idea behind color adjustment was to add a visible amount of midrange blue for the skin tones, then reduce dark blue and dark green for the front grill. A small amount of dark red was needed, too. ColorMill made further tweaks in Blue, Green, and saturation. The color settings for Scene 1 are in "scene1.vcf", which will be inside the attached "Firestone1_vcf_settings.zip".

Don't download .vfc's into VirtualDub plugins. Rather, keep .vcf's in the same area as your source videos or other project files.

Frame 28 of the final Scene 1 has a "hiccup" in the broadcast, looking like a break in the telecine stream that has sawtooth edges on objects. It appears to be a fault in the original broadcast, and the man's arm movement is distorted. In the original telecined clip, the frame is #35. Unfortunately there is too much motion to allow frame interpolation for ReplaceFramesMC; the resulting distortion would look worse than the problem.

Scene 2 consists of the Firestone outdoor signage and the red-yellow-green stoplight sequence. Color balance seems convincing enough as-is but it could use a little more "snap" with brighter highlights. It has edge ghosts and halos along with chroma displacement and smearing.

AVISource("I:\forum\faq\billct97\X1\F1\Firestone1_ ivtc.avi")
Trim(37,89)
AviSource opens and decodes Firestone1_ivtc, using only frames 37 to 89 for Scene 2.

dfttest(sigma=8)
DeHalo_Alpha()
dfttest() is a denoiser noted for smoothing noise rather than removing grain. It's one of the support files that comes with QTGMC. It's used here at half power. DeHalo_Alpha() attacks either bright or dark edge halos (bright halos by default). It's pixel width can be adjusted, but wider pixel width softens video more. Use sparingly (http://www.digitalfaq.com/forum/atta...dehaloalphazip). Its support files MVTools2 and RGTools are part of QTGMC's package.

VirtualDub filters used were ColorCamcorderDenoiose, exorcist, and the built-in VDub Levels control. VDub Levels is used to restore brightness and contrast after using the exorcist filter. The settings are in "Scene2.vcf" which is inside "Firestone1_vcf_settings.zip".

The images below are from scene 3. Top image = original unfiltered frame #3 with YUV histogram showing dominance of Blue (U channel) and deficit of Red (V channel). Bottom image = frame 3 after YUV adjustment with Blue reduceed and red added. Did it matter? Too much Blue looked overdone to me.
http://www.digitalfaq.com/forum/atta...1&d=1585740045http://www.digitalfaq.com/forum/atta...1&d=1585740045


AVISource("I:\forum\faq\billct97\X1\F1")
Trim(90,105)
AviSource() opens and decodes Firestone1_ivtc.avi, and Trim(90,105) uses only frames 90 thru 105 for Scene 3.

No VirtualDub filters were used on output, so there is no .vcf file for Scene 3.

Scene 4: color and gamma levels were a pain in the neck. The scene looks dim, with lame color. The original histogram is all twisted up: midtones are too red and are missing blue, darks are bluish, other colors look too green or too red, the salesman's skin tone looks putrid and heaven knows what shade the shirt is. Shadow areas look underexposed and there's an overall gritty look to dark details; the young kid's face is obscured and the car batteries on display are the wrong shade of white. These problems look like sloppy film mastering. It took a while of making RGB adjustments and then studying the effect for a long time, then coming back later to check it again. I must have changed everything 40 times. Some colors are like that, especially with commercials.

AVISource("I:\forum\faq\billct97\X1\F1\Firestone1_ ivtc.avi")
Trim(106,155)
AviSource() opens and decodes Firestone1_ivtc.avi, and Trim() uses only frames 106 thru 155 for Scene 4.

ColorYUV(off_u=9,off_v=-5)
Levels(12,1.0,255,16,250,dither=true,coring=false)
ColorYUV() increases Blue and decreases Red. Avisynth's Levels() looks at values starting with "12" and raises them to 16, to get some of the dark grime out of deep shadows. The value "250" slightly tames the brightest specular highlights. Look at the Levels function in online Help to see what those first 5 numbers mean (http://avisynth.nl/index.php/Levels). You can adjust those values to massage the behavior of dark and bright extremes. Use the Avisynth YUV histogram to check the results.

While scrolling in VirtualDub frame by frame when checking a script, you can turn script or filter changes on or off in the script, then check results instantly by clicking "File..." -> "Reopen video file" (or press F2), which reopens the video to the current frame. Sometimes there is a slight pause while Avisynth repeats the processing. You can repeat that again and again until memory clogs up and VirtualDub freezes, in which case you just have to reopen the script from scratch.

ReplaceFramesMC(30,1)
There is a very dark horizontal ripple in frame #30 that flashes across the screen just above the customer's forearm, in the horizontal stripes of his shirt. Because there's very little motion, ReplaceFramesMC interpolates a new frame successfully. In the final version of the DVD with pulldown, the black ripple is in frame 170 and is also partly visible in the stripes on the customer's tummy.

VirtualDub filters used were ColorCamcorderDenoiose, exorcist, gradation curves, and ColorMill. Gradation curves added brighter values of Red and Green, which brightened the top end of the spectrum, while Blue was added mostly to the midrange. After using exorcist, the overall brightness values were restored with the gradation curves "RGB" setting. The image below has a pink upward arrow that shows how the diagonal "RGB" line was altered to boost the brights:
http://www.digitalfaq.com/forum/atta...1&d=1585740287

Be careful when adding contrast or highlight brightening. When bright detail starts to fog out or disappear, the brightening is too strong.

The VDub filter settings are in "Scene4.vcf" which is inside "Firestone1_vcf_settings.zip".

Scene 5:

AVISource("I:\forum\faq\billct97\X1\F1\Firestone1_ ivtc.avi")
Trim(156,0)
AviSource opens and decodes Firestone1_ivtc.avi and uses only frames 156 to the end for Scene 5.

Levels(16,1.0,255,16,245,dither=true,coring=false)
The blue lettering is just a tad overly bright at greater than y=235. Setting the bright limit to "245" tames that problem and makes it possible to spiff up the bright end in RGB.

checkmate(thr=10, max=20, tthr2=2)
Checkmate() is a dot crawl filter. Its wiki download page describes it and its settings (http://avisynth.nl/index.php/Checkmate). If you play your original Firestone_1.avi, look at the edges of the big Blue $20.00 characters. You'll see a noisy edge "buzz" that looks like bad blue screening in the mastering lab. Checkmate is used to calm that buzz, using settings widely recommended for this sort of thing. You can also use the filter at its weaker default settings by just typing "Checkmate()". Stronger settings create softer effects.

The VDub filters used were CamcorderColortDenoise, two (2) instances of the exorcist filter, and gradation curves to tweak color and restore highlights. The exorcist filter is run twice because of the very heavy edge ghosting on those blue letters. The VDub filter settings are in "Scene5.vcf" which is inside "Firestone1_vcf_settings.zip".

The 5 intermediate files were all saved out of Virtualdub as Lagarith YV12. I joined them in Avisynth and encoded them with 3:2 pulldown in TMPGenc as "Firestone1_cleanup_DVD_pulldown.mpg" (attached).

That was a toughie. Most of the problems appeared to be in the original broadcast.
On to Firestone_2....

sanlyn, just a quick reply to say thank you. So much in this first clip that looks like I knew it should, wasn't sure how to make it happen and now it does. The car looking... yellow, the face tones, the detail in the black grille, the burgundy car, the battery which now looks white and the character edges that look so much tighter. I look forward to part 2 if you have time but more so look forward to playing with all of the examples in this forum thread and actually learning all I can. I expect this will be slow going as I have a lot to learn. Again, a quick thank you in case I don't post anything while I'm learning.

A slowdown problem here, too. Doing COVID19 chores in our gated neighborhood, lots of houses and apts. Helping to check on the elderly and make sure they're OK. Had a pulmonologist appointment today at a hospital only 10 minutes from here and we live in a small town, but there was so much chaos at the clinic I ended up waiting for 2.5 hours, ate up the rest of the afternoon. Bummer.

Oh, well. Tomorrow is another day. I think that's a famous line I heard in a movie somewhere. :laugh:

I think you got your wires crossed from too many crappy tapes! :eek::D This one was shot on film, but at 30fps. Your decimated version is choppy since you took out every 5th frame.

After the dissolve at the 0:11, the rest of the ad is 29.97p already. No deinterlacing required.

The "black & white talking kids" segment that precedes this looks interlaced but is really just phase-shifted. SeparateFields().Trim(1,0).Weave() turns the shots themselves into proper progressive, but the edits are out-of-phase, so now those few frames are interlaced (i.e. one frame at each scene change).

TFM works "okay" here, but picks a lot of wrong matches because of the small amount of motion and high amount of noise. I created a filter years ago to automatically handle content like this with better results, but I couldn't nail down why it would crash sometimes, so I never released it.

Yes, I noticed those shortcomings on working with newer filters for the clips I wasn't satisfied with, but I've not had time to address it and post corrections. However, I didn't catch the phase shifted part. Thanks for catching it. I'll have to take a closer look tonight.
:salute:

I get most of what you're saying. But SeparateFields().Trim(1,0).Weave didn't make anything look progressive. However the script I used below did so. Since there is such a mix of frame structures I decided to make it all progressive.

The video in question was a sample capture encoded to high bitrate MPEG and attached to post #12 as "Dentist_AG1980_sample.mpg". It's original title as lossless Lagarith YUY2 was "GI1_e_dentist_Ag1980.avi". It's linked to a pro MediaFire account at https://www.mediafire.com/file/jogh2...g1980.avi/file. Pro accounts do not show popup ads or web pages.

So the "dentist" clip has both interlaced and progressive sequences. It's not telecined. That's what I get for making assumptions and rushing in without looking around first. (wagging finger at myself) Let that be a lesson to me! And it's not the first time, either.
:smack:
The content of the original clip has two parts, beginning with a black and white sequence that dissolves into two or three panning shots of dental tools, toothpaste, and a model tooth. It looks interlaced but, according to msgohan (who happens to be correct), it's really fields out of phase. The second part begins at frame 652. It is progressive and in color.

I remade this clip in three parts. Part 1 assumes that the first part up to frame 651 looks interlaced ut has out of phase fields. The script for Part 1 discards the first field in the clip, moveing the others one field forward. Then QTGMC deinterlaces the result and discards alternate frames, which are duplicates. Part 2 takes the results of part 1 and applies the same denoising and filters that were used in the first version of this clip in post #12.

Part 3 then takes the clip from the original's progressive sequence in frame 652 to the end. It denoises and corrects color, mainly skin tones. The beginning and end sequences are then joined together and encoded.

The attached "Dentist_Remake_29_97_480i.mpg" is encopded as interlaced, although physically it's progressive 29.97fps. Why encode it that wa7? if you wanted to join this clip into a series of other videos that are either interlaced or telecined (the latter is treated as interlaced in authoring programs), you would have to encode the dentist clip as interlaced. And for standard definition BluRay, progressive isn't allowed. If you wantesd to mount itg on the internet, you would encode he progressive version as progressive in something like h.264/mp4. But remember that you can't mix progressive and interlaced or mix different frame rates in the same video.

Here's the script I used (Part 1) to make the front end of the dentist clip entirely progressive:

SeparateFields().Trim(1,0)
Weave()
SeparateFields() breaks full-sized interlaced frames into half-size fields, using alternating scanlines to build in field (http://avisynth.nl/index.php/SeparateFields). Trim(1,0) starts with takes the second field in the stream of fields (field numbers start with zero) and makes a new stream of fields using only field 1 to the end of the clip (http://avisynth.nl/index.php/Trim). Weave() re-weaves the stream of fields into top and bottom fields in interlaced frames (http://avisynth.nl/index.php/Weave).

"FPSdivisore=2" in the QTGMC line drops every other field in the deinterlaced video, thus keeping the frame rate at 29.97fps instead of 59.94. The fields being dropped are out of phase duplicates.

Below is the script for Part 2, which denoises Part 1. It's mostly grayscale but with some color that already looks OK, so no complicated color correction is needed.

The conversion to RGB32 is for the NeatVideo filter and ColorCamcorderDenoise for the color frames of this sequence. If you don't have NeatVideo, increase the value of sigma in the dfttest line to "dfttest(sigma=16)". The VDub filter settings are in "Dentist_remake_Part2.vcf" inside the attached "Dentist_Remake_VDub_Settings.zip". The NeatVideo noise test patches are attached in post #12.

Also, If you don't have NeatVideo the .vcf file will give an error. In that case, make a backup copy of the .vcf. Open the vcf in Notepad or other plain text editor. Delete the next to last 2 lines in the .vcf that begin with these words:
Leave the last line intact in the .vcf. The last line will read:

Don't leave Word Wrap turned on in Notepad when editing scripts.

Part 3 is only one script. It takes line 652 to the end of the clip and works with the color sequence that is already progressive. Again, I'm using QTGMC in progressive mode ("InputType=2") for smoothing and denoising of this crummy video, along with MGrain2.

RGB32 conversion is for NeatVideo and the color correction filters, which were ColorCamcorderDenoise v1.7, ColorMill, gradation curves, and hue/saturation/intensity (download Hue.vdf at http://www.digitalfaq.com/forum/atta...9&d=1540679970). The latter filter was used to reduce saturation in Red and Magenta. VDub settings are in "Dentist_remake_Part3.vcf" inside of the attached "Dentist_Remake_VDub_Settings.zip".

As mentioned earlier, if you don't have NeatVideo use "dfttest(sigma=16)" in the line following the MDegrain2 routine and before the "MergeChroma" line. And again, if you don't have NeatVideo use the .vcf file changes suggested earlier for Script 2.

The finished and attached .mpg "Dentist_Remake_29_97_480i.mpg" is encoded for DVD with 3:2 pulldown restored.

You have to use AssumeTFF() before any field-based operation in Avisynth, like SeparateFields, Bob, etc., since the default operation of AVISource is to assume BFF.

When I was testing it I didn't have to do this since the MPEG-2 you posted is flagged TFF, and I hadn't downloaded the lossless version earlier.

Here:
Not 3:2 pulldown!

I did TFF and BFF. Finally took the default and most looked interlaced anyway. Also tried fancy tricks around the dissolves, but it always looked clunky at those points so I just kept it consistent all the way through to the color sequence. When I run into clips with oddball mastering it's always a hassle and never comes off smoothly. I end up keeping it simple or consistent.

I tried something similar and gave up. I'll try your exact wording and see what happens, but I hate it when mixed-bag dissolves come along. I have a long history of just eventually plowing through that stuff.

No, it doesn't have 3:2 pulldown but when I caught the typo it was too late to change it. One hour isn't enough lag time IMO.

Thanks again for the better samples. I worked over Firestone-1.avi (or it worked me over!) in an earlier post.

The Firestone-2.avi sample demonstrates how VHS and old analog commercials demand patience. Every analog commercial and scene has different requirements. It seems the farther back you go in time with analog TV, the more things differ scene by scene. I cut the sample into 6 scenes this time. Filters were similar to those used in Firestone-1, with exorcist and gradation curves as usual to handle those edge ghosts.

The first script removes telecine from the incoming Firestone-2.avi and was saved as Lagarith YUY2.
Scene 1 is certainly dark with very little shadow detail, and overly red. Brightening doesn't help (nothing but noise and distortion down there). Correcting color for real-life accuracy isn't possible -- the colors really turn to garbage if you try for realism. But it does look a little better with whiter whites on those knobs and the gear shift numerals. That's as far as one can get. It's obvious that the creators wanted macho darks and red-hot color, for whatever reason. The scene does need smoothing: the grain on the exposed parts of the hand could turn to schmutz when encoded.
http://www.digitalfaq.com/forum/atta...1&d=1586134151
scene 1 color/ Before (left), after (right)

dfttest() is used at its default sigma value of 16 for a little more smoothing of the remaining grain. The conversion to RGB32 is for the VirtualDub filters loaded on output, which were ColorCamcorderDenoise, exorcist, and gradation curves. Frankly the original color is probably closer to what the creators wanted. To keep it, just don't use the VirtualDub filters. The VDub settings are in "scene1.vcf" inside the attached zip file "Firestone2_VDub_Settings.zip".

"Firestone2_VDub_Settings.zip"

A note about gradation curves in this project: VDub's gradation curve filter isn't used only for color correction. It's also used to blacken the final frame borders, which often take on a color cast after correction work. If you look at some of the curves applied you'll see some with a short corrective "hook" at the lower left zero point. If you don't use curves then you can't use this method, so you would have to replace the frame's borders with new black ones by using VDub's "Border Control.vdf" filter (https://sourceforge.net/projects/bor...atest/download). Unfortunately this .vdf can't be saved in a .vcf. A vcf that contains text for the filter will not load it properly.

http://www.digitalfaq.com/forum/atta...1&d=1586134403
The image above is the RGB panel of a curves control that adjusts brightness after the exorcist filter darkens the image. At the lower left corner of the graph the slanted line has several "o"-rings to hold the curve in place, then along the lower border the lowest o-ring on the slanted line is placed last and slid slightly to the right. Just below the arrow, the caption over the "Input Value" reads "4", while the caption over the "Output Value" reads "0". This indicates that the bottom o-ring has set all values 4 and darker to 0. The other o-rings keep the setting from affecting other parts of the image. This image of a curves control is from Scene 6.

Scene 2 with the three guys doesn't look bad, but it's rather greenish. There is also some blue lighting off and dark blue in the "black:" background>

The VirtualDub filters used are ColorCamcorderDenoise, exorcist, and gradation curves. The VDub settings are in "scene2.vcf" inside the attached zip file "Firestone2_VDub_Settings.zip".

Scene 3 script:

The VirtualDub filters used are ColorCamcorderDenoise, exorcist, and gradation curves. The VDub settings are in "scene3.vcf" inside the attached zip file "Firestone2_VDub_Settings.zip".

Scene 3 has the same color problems as a previous salesroom scene in Firestone-1. The yellowish problem seems to be from interior lighting. Fortunately there is white, gray and black in the tires on display, which is helpful even if the skin tones are still a little yellow and the whitewalls are not pure white. I didn't try corrections in YUV, which turned everything blue, especially in the blacks. Its another case of trying an RGB adjustment, letting it bake for a few hours, and checking later for another round of adjustments.

How do you know when the whites are white, and so forth? Basic color correction attempts to find a white balance and, if available, a gray balance and a black balance. All shades of white from pure white thru gray to black are formed with RGB colors Red, Green, and Blue (which is where wwe get the acronym "RGB"). The RGB numbers for those colors are often stated as RGB RRR-GGG-BBB, as below:

super white = Red 255, green 255, B 255, or = RGB 255-255-255
video white = RGB 235-235-235
light gray = RGB 192-192-192
middle gray = RGB 128-128-128
dark gray = RGB 64-64-64
digital black = RGB 16-16-16
super black = RGB 0-0-0

You'll see that each shade of white from super white to super black consists of equal chunks of each color, R, G, and B. If you can find a white or gray object in an image and make it look correct for that shade, then all RGB colors are in balance at that point and the other mixed colors will fall into place. Achieving that balance is called finding the white point, or the gray point or, when needed, a proper black point.

A pure red will have only Red, with no Green or Blue in the mix. But yolu're not likely to fin d pure color in nature, mnot even in a "blue" sky (which will likely contain some slight red and green).,We would say that the RGB colors in pure primary RGB terms are:
Red = RGB 255-000-000
Green = RGB 000-255-000
Blue = RGB 000-000-255

The secondary colors mix 2 basic RGB colors together:
Yellow = RGB 255-255-000
Cyan = RGB 000-255-255
Magenta = RGB 255-000-255

All other colors such as orange, hot pink, scarlet, brown, indigo, etc., are just various mixes of R, G, and B.

How do you measure those pixel values? With pixel readers like those provided with big-name video and photo editors, or with freebies like the CSamp pixel reader control mentioned earlier or the ColorPic at
http://www.iconico.com/colorpic/. Take a reading and then use ColorMill, gradation curves or other RGB controls to set those colors to the ideal combo. It will seldom be perfect but at least you'll get close. And you'll also find out just how bad VHS color really is.

Scene 4 script:

FixChromaBleed() helps tame the magenta lighting from the side and in the background. But practically speaking the VirtualDub RGB color changes don't help much. The black tire reflects colors from everywhere and will never be consistently black everywhere. Nevertheless the filters used were ColorCamcorderDenoise, exorcist, and gradation curves. The only part of the curves filter you really need is the "RGB" panel to correct luminance after exorcist is applied. The VDub settings are in "scene4.vcf" inside the attached zip file "Firestone2_VDub_Settings.zip".

Scene 5 with the bold blue characters had the usual problems, like the similar shot in Firestone-1:

ppath=Import(ppath+"ContrastMask.avs")"
Import(ppath+"ContrastMask.avs")
Import(ppath+"ReplaceFramesMC.avs")
These lines are really tricks to help avoid some typing. "ppath" is a name I invented that means "plugin path", which becomes a user variable that contains a string with the name of the Avisynth plugin path in my system, which is "D:\Avisynth 2.5\plugins". Don't forget the dividing "" on the end of the string. Import(ppath+"ContrastMask.avs") uses the concatenate character "+" to join the ppath string "D:\Avisynth 2.5\plugins" to the filter name string, "ContrastMask.avs". So the line Import(ppath+"ContrastMask.avs") actually gets read as Import"D:\Avisynth 2.5\plugins\ContrastMask.avs"). You don't have to use that trick if you don't want it, otherwise you have to type the path entirely from scratch on each Import line.

a=last
Save_audio=a
a
You've seen this before. A frame replacement routine cause a break in the audio, so a copy of the incoming clip called "a" and another copy is saved as "Save_Audio" for later use to restore the original audio near the end of the script. The clip "a" is named explicitly to return processing focus to the incoming clip. At the end of the script, Audio-Dub() uses Save-Audio to restore the original sound.

contrastmask(enhance=2.0)
ReplaceFramesMC(43,1)
ContrastMask() at a low enhance value is used to bring up some deep shadow detail. ReplaceFramesMC fixes a thin horizontal ripple (dropout) in frame #43 that obscures some of the white lettering under the blue characters. In the original Firestone-1.avi the frame is #186.

DeHalo_Alpha(brightstr=2.5)
The brightstring parameter in DeHalo_Alpha is strengthened a bit to calm the bright right-edge halo on the pile of tiles. VDub's exorcist filter later helps with the edge ghosts.

Spline36Resize(width/2,height)
Spline36resize(width*2,height)
The firt line resizes the frames to 360x480, the second line returns the frame to 720x480. This reduction in horizontal size helps to thin some of the heavy whitish ghost lines in the blue letters and also helps thin out the white characters a little. This routine affects sharpness only slightly.

The VDub filters used were ColorCamcorderDenoise, two (2) consecutive copies of exorcist for the heavy ghosting, gradation curves, and the built-in Levels to give specular highlights a little more snap after using exorcist. The VDub settings are in "scene5.vcf" inside the attached zip file "Firestone2_VDub_Settings.zip".

Scene 6 script, with a few more problems:

RemoveDirtMC(40,false) is used to quell some grainy noise as wwell as to clean up 3 horizontal rips in frames #12, 20, and 30 (in the original Firestone-2.avi the frames are #249, 259, and 269). These are short ripples across the white letters at the bottom of the frame. They're easier to see full screen or on TV. RemoveDirtMC(40,false) is used here at strength 40 which can clean up many forms of small dropouts and white streaks, Stronger settings work but can sometimes make moving objects disappear. The usual strength for denoising is 20. The "false" parameter means you're not working with monochrome.

The VirtualDub filters used were ColorCamcorderDenoise, exorcist, gradation curves, built-in Levels, and FadeFX to mask the very dirty fade-out in frames 31 to 35. The Levels filter and curves were used to clean up all the colorful noise in the so-called "black" background which isn't a very clean black without some help. Remember to use controls like FadeFX at the end of the filter chain, or your other filters will affect FadeFX's action. The VDub settings are in "scene6.vcf" inside the attached zip file "Firestone2_VDub_Settings.zip".

Hope there weren't too many weird typos this time. The result of these scripts and filters is attached as "Firestone2_cleanup_DVD_pulldown.mpg".

This is a fantastic education! I spent some time this weekend comparing scripts in this thread as well as a few older threads and built a script to play with. First to prove that I can get Virtual Dub and Avisynth loaded with all the plugins that seem to be commonly used here (and I did!) and second to see the effects of using different plugins. Entirely useless for actual repair work without logical selection but confidence building just the same. This is so much farther than I got 2 years ago before giving up.

I also have come to the realization that doing this kind of repair for everything I have will need to be prioritized. I mapped out in a one hour TV show about 25 commercials, news breaks, etc... quite likely an impossible task to take on given that I have 40 tapes each with 6 hours of TV shows. More or less. The plan for now is to capture everything with lossless compression (lagarith has worked well so far) while being careful to watch for proper levels and any cases where the capture itself can be improved (AG-1980 vs. JVC HR-S9800U, TBC-3000 adjustments, tape problems, etc...). The ones that are worth the time and effort of post-process will get it as time permits while the others are at least available to enjoy in their raw form.

One other thing that I've come to realize (and I believe Lord Smurf has extolled) is that the imperfections in some cases might be worth leaving as-is. This one hour that I mysteriously have from 1986 (the rest are later 80's and 90's) is so much fun to step through frame by frame. I worked at a CBS affiliate for a short time around this time and seeing "the crud" in the capture brings back memories. Particularly seeing the loss of sync between commercials - I would guess because, as we used, the stations TBC had to switch between two commercial cart machines. Technology was good then but not perfect. And video levels at the start of a commercial fluctuate, I'd guess, because that 2" Ampex cart had to go from zero to full speed pretty quick. Also the same video level fluctuation in the Firestone spot when the price is superimposed likely showing that two video sources again had to be combined and the technology to do that, well... it was 1986. The one that surprised me was the show (a Miami Vice episode) was obviously on film as you've told me many times. I recorded it on the air date and I assume it was satellite fed (that's what we did for network shows) yet the film copy, especially the show open is horribly laden with all kinds of black spots you can see quite clearly frame by frame. Best of all, the first frame when the camera is pointing at the sun, the sun itself is magenta-purple-orange another sign, I'd assume that it's film. Things I never would have seen playing back this VHS tape on an old TV. That frame is attached for your enjoyment.

Sorry, got a bit off topic... To wrap up, again I so very much appreciate everyone's help - sanlyn, lordsmurf, themaster, msgohan - and if I don't post anything for a bit it is because I'm deep into learning mode.

I second billct97's praise for Sanlyn's tutorials here and elsewhere on this site--they are possibly the best we have for newcomers on all of the interwebs. Sanlyn wrote this a few posts ago:

However, Youtube notes that it will also accept 480p at 854x480 (https://support.google.com/youtube/answer/6375112). Is it better to avoid the resizing and add a border to make the width 854 (66 and 68 on the sides, due to the mod-2)?

No. YouTube uploads look the best the more you upscale them, because they use higher quality compression/processing for the larger frame sizes. The other day I resized a video I was posting to YT from 352x240 to 2880x2160 ("4K" but in 4:3 aspect ratio) because their processing is so awful at 240p.

Sanlyn is incorrect when it comes to YouTube's current aspect ratio requirements. Nowadays, you can upload 4:3, 16:9, 2.35:1, etc. and it handles them appropriately without baking in black bars.

Thanks. Do you think it is a safer compromise to upscale to 1080p because of the effect of upscaling on video that is not lossless, assuming noise removal has been attempted? 4K is twice the vertical resolution, but is YouTube's 4K codec really twice as good as it's 2K version?

My familiarity with YouTube policy is obviously dated but I never post there. Thanks for the corrections.

I haven't compared directly between the two myself, but I did do a test a few years ago of 720p video game footage where I uploaded original, upscaled to 1080p, and upscaled to 1440p. (Because 2160p wasn't an option back then.) Note that this game applies an artificial "grain" filter over the rendered image, which makes it much harder to compress than a clean-looking video game.

https://forum.videohelp.com/threads/...be#post2353139

Upscaling to 2160p is closer to an integer scale of 480 and 576 than 1080p is, so that is an added benefit.

In that videohelp thread there are conflicting opinions as to whether upscaling does help. If it does help, the difference may not be worth the time.

In this more recent thread, the OP's analysis concludes that it is a waste of time, although down the page I did see more quality in a 4K upscale from the 1080p version. The thread starter counters that the difference in quality is just due to sharpening filters in the browser.

https://www.magiclantern.fm/forum/in...?topic=22681.0

I'm feeling exceptionally lazy today... 2 years ago my Dad wanted a VHS tape put on DVD. My usual workflow captured at 720x480 with lagarith lossless compression has given me a 2 hour 50GB AVI file. The original video is from 1993, is in 4:3 format and except for some B&W footage at the beginning that is 3:2 telecine (easily seen when stepping frame by frame) the rest is not telecine, just interlaced. I don't intend to do anything besides trimming the warped tape garbage at the start and cropping out the edge noise and adding borders back to 720x480.

My question, can someone share with me a high level plan for taking this video and getting it onto DVD? I have a DVD burner, plenty of writable DVD's and probably software that will do the burning. I don't care about menus or anything more than a DVD that can be put in a stand-alone DVD player (not BluRay) and plays with decent quality. If breaking the video into 2 or more DVD's will yield better quality I'm all for that.

Pointing me to a past forum discussion and RTFM is also acceptable :)