Quantcast Filters/scripts for US broadcast TV VHS conversions? - digitalFAQ Forum
Go Back    Forum > Digital Video > Video Project Help > Restore, Filter, Improve Quality

Reply
 
LinkBack Thread Tools
  #1  
03-19-2020, 05:44 PM
billct97 billct97 is online now
Premium Member
 
Join Date: Jan 2018
Posts: 36
Thanked 1 Time in 1 Post
Hi All;

Short Version: Would anyone be willing to share examples of their most used, go to filters and scripts for making adjustments to VHS captures of 1980's and 1990's era US broadcast TV.

Long Version: 2 years ago I made my first attempt (with some great help from this forum) at building a reliable workflow and understanding what to do post capture. Other priorities caused me to put things on hold. But now I'm back and ready to dive in. My priorities are to start with the 40 or so VHS broadcast TV SLP tapes that I have from the late 80's and early 90's and extract the unique things such as commercials, news breaks, station ID's. I like that kind of thing. The shows might get captured but unless they aren't available elsewhere will likely not get processed. Final format will likely remain on computer disk or maybe get posted online for others to enjoy. I don't expect to burn anything to DVD but you never know.

Thanks to help on this forum I feel like my workflow is quite good.
Panasonic AG-1980 - from T.Grant - I love this machine!
JVC HR-S9800U - from eBay and not as good as the AG-1980, not even plugged in.
Data Video TBC-3000
ATI 600 USB
S-Video between all devices
Lenovo T500 laptop with the discrete graphics card, Windows XP SP3
OS and Capture Disks are SSD
Sony TV for monitoring the VCR Video Out and Audio to allow Virtual Dub to capture without Preview/Overlay Video and Audio Monitor.

I bought a SpyderXPro to calibrate my monitor but that was before reverting to XP and it doesn't work on XP. Any recommendations for calibrating a modern LED monitor on XP?

Captures that I've done have been very good, although I'm still experimenting how best to adjust the TBC and how to use the histograms but I'm getting there. Dropped frames is single digits or 0, audio is in sync and I've had no issues using HuffyUV or Lagarith to reduce disk space by 2-3x.

2 years ago lordsmurf and sanlyn were kind enough to analyze a capture or two of mine. The explanations were excellent and I intend to re-review those at length before I get too far along. The video I had uploaded was from a Star Trek TV episode. I've since decided that I want to process the commercials, news clips, etc... and not the TV show.

One thing that has been puzzling me from their analysis was the use of 3:2 Pull Down. I fully understand the why - the TV show was shot on film at 24 fps and broadcast at 29.97 fps so there was a conversion. What I don't understand is how to know when this is done and when to undo it in Avisynth. I did just Google this explanation from forum post 8562 - "Open the AVI in VirtualDub, play frame by frame and look for horizontal motion. You will see 3 progressive frames and 2 interlaced frames in every group of 5 frames. This is 3:2 pulldown, which is a form of telecine." Is that really the best way to know? And what about the commercials in those shows? I remember in the late 80's when I worked in broadcast TV and we shot local commercials we used 2 inch video tape. So how best to know when to 3:2?

Also a lot of what I've seen on this forum, understandably, is VHS conversion of home movies. So I feel that maybe some of the recommendations in many posts may not apply to broadcast TV VHS.

I guess what I'm asking is this. There is a ton of info on this forum, I've read a lot, I've played with maybe a dozen scripts one piece at a time and I've gotten a bit overloaded. If someone could share with me a best-of set of their most commonly used tools (with an emphasis on broadcast TV) I'd be very grateful and it might get me heading in the right direction a bit faster. OK, that seems pretty lazy on my part but I'm asking because I'm spinning my wheels a bit and could really use a good kick start.

Thanks,
Bill
Reply With Quote
Someday, 12:01 PM
admin's Avatar
Ads / Sponsors
 
Join Date: ∞
Posts: 42
Thanks: ∞
Thanked 42 Times in 42 Posts
  #2  
03-20-2020, 03:11 AM
themaster1 themaster1 is offline
Free Member
 
Join Date: Feb 2011
Posts: 213
Thanked 37 Times in 32 Posts
This is one of the scripts i used the most (on many vhs sources over the years) if that can help you.

Quote:
SetMemoryMax(1200) # 1200 MB allocated (depends on Ram you have at your disposal and Config)
SetMTMode(3,4) # for multi-threading processing of the Video Source!
AVISource("MyVideo.avi")
converttoyv12(interlaced=true)
Assumetff() # for topfield interlaced source

SetMTMode(3,4) # for multi-threading processing of the Filters following !

#1) Fix Video Levels, Saturation
separatefields()
ylevels(0,1.000,255,16,235)
tweak(sat=1.3,coring=false)
#2) Fix Chromatic misalignement (it happen a lot on Vhs tapes)
A=Last
B=A.Greyscale()
Overlay(B,A,X=0,Y=-2,Mode="Chroma")
weave()

#3) Fix Chromatic Noise:

ConverttoRGB32(matrix="rec601",interlaced=true)
separatefields()
#### CCD COLOR DENOISING :
LoadVirtualDubPlugin("C:\Program Files (x86)\virtualdubmod1.5\plugins\Camcorder_Color_Den oise_sse2.vdf", "CCD", 1)
CCD(10,1) # Range: 0-100 / default is 30 (too strong for commercial tapes) / good range is 7-15 in practice, higher for deteriorated tapes
# Sharpening (optional)
#LoadVirtualDubPlugin("C:\Program Files (x86)\VirtualDub\plugins\msu_sharpen.vdf", "MSUSmartSharpen", 1)
#MSUSmartSharpen(1) # Range 0 (small overall) to 5 (mad, not recommanded!)
weave()
converttoyv12(matrix="Rec601",interlaced=true)

# 4) Noise Removal (Spatio-Temporal, With Dfftest, RemoveSpotsMc)

c=last
#even fields
# Motion compensation on frames -2...+2
even = c.SeparateFields().SelectEven()
super_even=even.MSuper()
vf2 = super_even.MAnalyse (isb=false, delta=2, overlap=4)
vf1 = super_even.MAnalyse (isb=false, delta=1, overlap=4)
vb1 = super_even.MAnalyse (isb=true, delta=1, overlap=4)
vb2 = super_even.MAnalyse (isb=true, delta=2, overlap=4)
cf2 = MCompensate (even, super_even, vf2, thSAD=400)
cf1 = MCompensate (even, super_even, vf1, thSAD=400)
cb1 = MCompensate (even, super_even, vb1, thSAD=400)
cb2 = MCompensate (even, super_even, vb2, thSAD=400)
# dfttest on motion-compensated clip
Interleave (cf2, cf1, even, cb1, cb2)
SetMTMode(2)
RemoveSpotsMC()
SetMTMode(4)
dfttest (sigma=4, sigma2=4, tbsize=5, lsb=false)
SelectEvery (5, 2)
# Gradient smoothing
presmooth = last
#SmoothGrad (radius=20, thr=0.15) #
#SmoothGrad (radius=12, thr=0.25, ref=presmooth)#
# Back to 8 bits, ordered dithering
#DitherPost () #
filtered_even=last
#odd fields
# Motion compensation on frames -2...+2
odd = c.SeparateFields().SelectOdd()
super_odd=odd.MSuper()
vf2 = super_odd.MAnalyse (isb=false, delta=2, overlap=4)
vf1 = super_odd.MAnalyse (isb=false, delta=1, overlap=4)
vb1 = super_odd.MAnalyse (isb=true, delta=1, overlap=4)
vb2 = super_odd.MAnalyse (isb=true, delta=2, overlap=4)
cf2 = MCompensate (odd, super_odd, vf2, thSAD=400)
cf1 = MCompensate (odd, super_odd, vf1, thSAD=400)
cb1 = MCompensate (odd, super_odd, vb1, thSAD=400)
cb2 = MCompensate (odd, super_odd, vb2, thSAD=400)
# dfttest on motion-compensated clip
Interleave (cf2, cf1, odd, cb1, cb2)
SetMTMode(2)
RemoveSpotsMC()
SetMTMode(4)
dfttest (sigma=4, sigma2=4, tbsize=5, lsb=false) #
SelectEvery (5, 2)
# Gradient smoothing
presmooth = last
#SmoothGrad (radius=20, thr=0.15) #
#SmoothGrad (radius=12, thr=0.25, ref=presmooth)#
# Back to 8 bits, ordered dithering
#DitherPost ()#
filtered_odd=last
Interleave(filtered_even, filtered_odd)
weave()
############ ANALYSE WITH HISTOGRAMS #######################
#Histogram(int2mode(2)) # with Slider for HISTOGRAM
#ColorYUV(analyze=true)# Enable to have precise statistics written onscreen
#Histogram(int2mode(1)) # with Slider for HISTOGRAM
#Histogram(mode="luma") # Luma Mode
#HistogramRGBParade()
#HistogramRGBLevels # rgb mode
#HistogramCMYLevels(range=true) #<< Good to Analyse Secondary colors (purple,yellow, cyan)!! (many people forget)

### # ANALYSIS OF YUV ###
Y = GreyScale() # analyse Y only
U = UtoY() # analyse U only
V = VtoY() # analyse V only
#StackHorizontal(Y, StackVertical(U,V))# <<< TO ANALYSE YUV

# To Analyse U and V precisely:
Stackhorizontal(U,V).Histogram(mode="luma") # <<< ONLY U and V and enhanced for better viewing with the luma mode of histogram

Last edited by themaster1; 03-20-2020 at 03:23 AM.
Reply With Quote
  #3  
03-20-2020, 07:16 AM
billct97 billct97 is online now
Premium Member
 
Join Date: Jan 2018
Posts: 36
Thanked 1 Time in 1 Post
Some things I recognize, some I do not. Lots to play with. Thank you, Bill
Reply With Quote
  #4  
03-23-2020, 10:29 AM
msgohan msgohan is offline
Free Member
 
Join Date: Feb 2011
Location: Vancouver, Canada
Posts: 1,084
Thanked 235 Times in 207 Posts
There's nothing wrong with that explanation for checking 3:2 pulldown.

But most commercials from the 90s and some from earlier will be a mix of underlying framerates due to different sources, graphical elements, and edits: 24, 30Hz progressive, and 60Hz interlaced. I wouldn't bother trying to process these separately for an ad. Just deinterlace them.
Reply With Quote
  #5  
03-25-2020, 09:27 AM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,564
Thanked 1,134 Times in 925 Posts
Quote:
Originally Posted by billct97 View Post
I guess what I'm asking is this. There is a ton of info on this forum, I've read a lot, I've played with maybe a dozen scripts one piece at a time and I've gotten a bit overloaded. If someone could share with me a best-of set of their most commonly used tools (with an emphasis on broadcast TV)
Why do you think there is some major difference between the filters you would use on noise from broadcast TV tapes and the noise you encounter on other home-made and retail tapes? Are these tapes recorded from TV broadcasts (I'd guess so, if they have commercials). Do those tapes have that grungy floating tape noise that you see on all tapes? Why would you use a different filter to clean it ?

Don't imagine that tapes recorded from TV broadcasts don't have signal level problems. They do. The steps you take to resolve crushed blacks and/or blown-out highlights because of bad signal levels are no different for tapes from home cameras or retail VHS tapes from the CRT era. Don't nurse the myth that later digital TV broadcasts have no noise. Digital broadcasts have defects that didn't exist in analogue or film; those defects are compression artifacts. I have lots of recordings off HD digital TV that are rife with compression artifacts, and a slew of old VHS recordings off cable TV from the 1990's that have digital "mastering" garbage I seldom saw on film or on good retail VHS.

Suit the filter to the problem, whatever the video's source. If you're dealing with only commercials and news footage, you'll have mostly telecine from the 80's and 90's, although digital video was being used more often during the late 90's -- and that digital video was shot as both telecined and interlaced. After you solve the frame structure differences, guess what you're left with: good ol' VHS noise, the same VHS noise, chroma junk, dirty color, edge halos, dropouts, spots, and all the other booboos that come with all VHS tapes, differing tape by tape and with every TV program, every TV commercial, and every TV news clip. Two problems unique to 80's and 90's broadcast TV are transmission noise and signal level problems. Commercials had annoyances such as wildly varying color balance -- and anyway, since when was VHS color all that consistent to begin with?

Give me a day or so and I'll try to get up a demo of some early 90's TV captures I archived somewhere on external drives.
Reply With Quote
The following users thank sanlyn for this useful post: billct97 (03-25-2020)
  #6  
03-26-2020, 08:23 PM
billct97 billct97 is online now
Premium Member
 
Join Date: Jan 2018
Posts: 36
Thanked 1 Time in 1 Post
Maybe someone would be kind enough to take a look at the first commercial that I'd like to work on and let me know what you think. A full blown filtering & fix with explanations (sanlyn style) would be very much appreciated if someone has the time. But anything you can do to point me in the right direction is very much appreciated.

When this was captured I didn't take advantage of the TBC-3000 and adjust for 16-235. As I look at them they don't seem to get too bright. Maybe I will re-capture and fix that. On capture and for these shorter avi I compressed with Lagarith. For the shorter segments I used Fast Recompress. Let's hope I upload the videos correctly. Apologies if I don't.

One of the things I'd like to know is how to handle multiple things that need fixing in a 30 second spot?

In Firestone-1-lag.avi, the very first frame has all kinds of rainbow crud below the license plate. This was just after a scene change - this isn't the start of the commercial. And in the first 5 frames, even though the camera is steady, when you look at the "Levels" histogram, it's bouncing all over the place. Fascinating!

Next, the man in the red shirt and the yellow car. "Levels" shows a lot of yellow and a lot of red. That seems to make sense. Although looking at luma I'm guessing we need to push everything a little to the right - maybe get some better details from the dark car grill.

The Firestone sign against the sky seems about right but notice how the left side that previously had a really wide gap (that needs a Crop and Border added) is now much thinner in this scene. Better yet in the next scene with the red stoplight the left border junk is wide again. Is this the original spot or the VHS?

Next around frame 115 the red T-Bird. Why is this so heavily blue? I suppose it could be it's a rainy day?

And finally the two men and little boy. "Levels" says this is heavy on the red and yellow. Luma seems a little low again, so maybe proc-amp and recapture?

Firestone-2-lag.avi, the first scene with the $49.95 the numbers all have a halo outside of the white border. What filter fixes this?

The drivers arm shifting the car - this looks terribly red to me. "Levels" agrees but again, is it the original or something that needs to be fixed? And if I lower the red here how do I do that without affecting the rest of the commercial?

Next the 3 happy Firestone guys. That doesn't look too bad to me or "Levels" although again, I think it might need more brightness during capture.

Then the 2 guys looking at a wall of tires - lots of red again and a little weak on brightness.

The superimposed Radials 4/$100 again has the outside halos like earlier and the final scene with Firestone letters on a black background also seems to need some brightness?

Funny, now that I wrote this all out it seems like in almost every case I'm in need of lowering red a bit.

I humbly ask that someone take a look, crucify me for what I misunderstand and help me take a step or two forward


Attached Files
File Type: avi Firestone-1-lag.avi (47.28 MB, 5 downloads)
File Type: avi Firestone-2-lag.avi (81.84 MB, 3 downloads)
Reply With Quote
  #7  
03-27-2020, 01:05 PM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,564
Thanked 1,134 Times in 925 Posts
Thanks for all the work you did on the samples, but.....
Why is the frame size a non-standard 708x472? What frame content did you remove? Some frames look as if something was removed or changed, but it's hard to tell. What do you intend to do with this frame size?

Overall noise isn't too bad, should be the easier part. Color is pretty messy, which will be difficult -- not a surprise with many old analog broadcast sources. The toughest part will be the oversharpening halos and edge ghosts. Have you increased sharpening on the AG-1980 ? (the sharpener is disabled when the slider clicks into the center indent in the middle of the scale).

Can't work on this with a mod-4 frame dimension, as many filters especially in Avisynth require at least mod-8 in all dimensions. A 708 pixel width is not mod-8. It's mod-4. The preferred frame size for NTSC VHS is 720x480, or at least 704x480 or 640x480.
Reply With Quote
  #8  
03-27-2020, 01:13 PM
billct97 billct97 is online now
Premium Member
 
Join Date: Jan 2018
Posts: 36
Thanked 1 Time in 1 Post
Yikes, I captured this in 2018 and assumed in 2020 that I did it right. That might explain why there wasn't any crud along the bottom edge as I cropped and didn't add a border to bring it back to 720x480. The AG-1980 sharpener is centered today but again, in 2018 I really had no clue. I will get that tape back out this weekend, recapture and repost. Thank you for taking a look!
Reply With Quote
  #9  
03-27-2020, 05:23 PM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,564
Thanked 1,134 Times in 925 Posts
I understand, it takes a time or two to work out everything. You'd be surprised at the omissions even the old timers make (yours truly included). As they say, the devil is in the details.



In the meantime I'm putting together a demo of 3 or 4 1990's commercials with lots of problems to consider. Sorry for the delay, this COVID19 rigmarole makes every daily chore take 3 times longer than it should.
Reply With Quote
  #10  
03-29-2020, 08:40 AM
billct97 billct97 is online now
Premium Member
 
Join Date: Jan 2018
Posts: 36
Thanked 1 Time in 1 Post
Quote:
Originally Posted by sanlyn View Post
In the meantime I'm putting together a demo of 3 or 4 1990's commercials with lots of problems to consider.
If that means that you will be posting before and after samples that includes your amazing explanations of what was done and why that will be VERY much appreciated. I'm happy to wait for that for as long as it takes!

In the meantime I have recaptured the Firestone commercial from 1986 if anyone might care to take a look. Nothing has been done besides compressing with Lagarith. I didn't even crop and border but this time we do have 720x480.

Interesting that for the first time I noticed a ticking noise in the audio. It is there throughout the entire recording (of Miami Vice) but doesn't happen on the next show that was recorded in 1992 with a better VCR. So it seems my old VCR is at fault. I will either have to live with that or do some more repair work. I have an older copy of Adobe Audition and could record the audio, isolate and remove the noise and then put the clean audio back with the yet to be filtered video. But let's take baby steps before we go there.


Attached Files
File Type: avi Firestone-1.avi (70.21 MB, 4 downloads)
File Type: avi Firestone-2.avi (71.65 MB, 2 downloads)
Reply With Quote
  #11  
03-29-2020, 12:28 PM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,564
Thanked 1,134 Times in 925 Posts
BAD TAPE CHRONICLES: TAPES FROM HELL, CONT'D:
Working with bad cable signals and bad tape from the 1990's, #1

Thanks for the new captures. I'll be look them over tonight.


The attached "Colgate_8661_sample.mpg" is a high bitrate encode of an otherwise unfiltered lossless capture from a TV broadcast. The attached "Colgate_ pv_s4670_sample.mpg" is a high bitrate encode of another otherwise unfiltered lossless capture from a TV broadcast, using a different VCR. The original lossless YUY2 samples were too large for posting here, but if you want a lossless copy request in this thread. I can mount one in my pro account at Mediafire (no popups, no ads).

The specs below will leave no doubt that in the early 90's I had no idea what the heck I was doing with video gear. A lousy crummy underpowered cable signal in my apartment building in NYC didn't help. My habit of capturing at 640x480 stemmed from the irrational idea that 720x480 wouldn't "look right" in my editors. But you can tell my editors which display ratio to use, silly boy. Live and learn.

Date recorded: October 1990 (based on other ads on the tape promoting cable programs and dates).
Original recorder: Mainstream JVC 4-head onto Maxell medium-grade tape at 6-hour slow speed.
Original broadcast: RF cable line -> 10w cable signal amp -> cheap RF cable -> analog cable box -> cheap RF wire -> cheap Radio Shack 4-way amplified RF splitter with 4 composite outputs -> cheap composite wire to VCR.

Captures (2008): Rebuilt Panasonic PV-8661 VCR (circa 1988), and a second capture using a rebuilt Panasonic PV-S4670 SVHS player (circa 1986)
-> AR blue-jacket composite -> Panasonic DMR-ES15 tbc passthru
-> s-video -> AVT-8710 TBC (circa 2004)
-> s-video -> ATI AIW 9600XT AGP card
-> VirtualDub/huffyuv YUY2, 640x480.
Original capture reviewed in VDub and cut-edited into lossless sections, archived to external HDD as Lagarith YUY2. The capture version chosen for final output was the capture played with the PV-8661. I felt the 8661 had the same defects but retained more detail after processing.

From the very start, the first thing one sees is the horrible noise. The opening black frames are almost gray with snowy grain. The apartment building's cable circuitry, really cheap wiring, multiple splitters, and less than perfect VCRs delivered exactly the noisy junk you would expect, and then some. There is so much floating gunk, fuzzy edges and streaking that you think your eyes are going bad. There is visible chroma bleed and chroma shift, over saturated reds, some mild edge ghosting, cable noise that plays as "horizontal rain" and flicker that's most visible in bright solid areas like sky and walls. The noise level and so-so tape quality didn't make for the world's sharpest picture.

How to clean this without destroying everything in sight? I turned to some of my favorite plugins. First, the video was inverse telecined in a separate step and saved as input for further processing. Why? It's difficult scrolling video back and forth in VDub to check filter changes and color if you're trying to run ivtc at same time. The inverse telecine script for all the demo videos that follow was short and simple:

Code:
### --- This IVTC works with YUY2 and YV12. --- ###
### --- Specify your input file in the PATH.--- ###
AVISource("drive:\path\to\capture file.avi"
AssumeTFF()
TFM().(Tdecimate)
Someone will ask right away: why didn't you use your AG-1980 player? I don't always use it. Some tapes just don't look so great on denoising VCRs, which often cause posterizing effects. Denoisers in premium VCRs were relatively primitive; today's Avisynth is more sophisticated. And some tapes just won't track nicely on some machines.

After inverse telecine, the script below employs several of my most-used VHS filters for very noisy or damaged tapes. I had to use two scripts, versions A and B. Version A was for the full length of the short commercial. Version B had script changes specifically for the really bad noise in the first 63 opening frames and the fadeout in the last 7 frames. Then B and A results were edited and joined in an output file script.

The code for version A:

Code:
AVISource("I:\forum\faq\billct97\Colgate\Colgate2_ivtc.avi")
ColorYUV(off_y=-5)
ConvertToYV12(interlaced=false)
### --- Next 3 lines are activated for testing only ### ---
## Crop(2,4,-22,-4)
## Histogram("Levels")
## return last

Stab()
FixChromaBleeding()
ChromaShift(l=-2,c=-4)
qtgmc(InputType=2,preset="medium",EZDenoise=4,denoiser="dfttest",GrainRestore=0.3)

source=last
  super = source.MSuper(pel=2, sharp=1)
  backward_vec2 = MAnalyse(super, isb = true, delta = 2, blksize=8, overlap=4, dct=0)
  backward_vec1 = MAnalyse(super, isb = true, delta = 1, blksize=8, overlap=4, dct=0)
  forward_vec1 = MAnalyse(super, isb = false, delta = 1, blksize=8, overlap=4, dct=0)
  forward_vec2 = MAnalyse(super, isb = false, delta = 2, blksize=8, overlap=4, dct=0)
  MDegrain2(source,super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400) 

grainfactory3(g1str=1, g2str=1, g3str=1)
AddGrainC(2.0,2.0)
MergeChroma(aWarpSharp2(depth=30).aWarpSharp2(depth=10))
LimitedSharpenFaster(edgemode=2)
Crop(2,4,-22,-4).AddBorders(12,4,12,4)

###--- Resize 640x480 to 720x480 for DVD --- ###
Spline36Resize(720,height)
Output from Script A was saved in Virtualdub as Lagarith YUY2 using "fast recompress" mode. Below is the Version B script for the very bad noise sections. It;'s the same as script A, but some lines at the very end use 16-bit processing for the worst noise:

The Script for version B:

Code:
AVISource("I:\forum\faq\billct97\Colgate\Colgate2_ivtc.avi")
ColorYUV(off_y=-5)
ConvertToYV12(interlaced=false)

Stab()
FixChromaBleeding()
ChromaShift(l=-2,c=-4)
qtgmc(InputType=2,preset="medium",EZDenoise=4,denoiser="dfttest",GrainRestore=0.3)

source=last
  super = source.MSuper(pel=2, sharp=1)
  backward_vec2 = MAnalyse(super, isb = true, delta = 2, blksize=8, overlap=4, dct=0)
  backward_vec1 = MAnalyse(super, isb = true, delta = 1, blksize=8, overlap=4, dct=0)
  forward_vec1 = MAnalyse(super, isb = false, delta = 1, blksize=8, overlap=4, dct=0)
  forward_vec2 = MAnalyse(super, isb = false, delta = 2, blksize=8, overlap=4, dct=0)
  MDegrain2(source,super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400) 

### --- 16bit dithering filter for noise problem --- ###
Dither_convert_8_to_16()
GradFun3(thr=0.8,mask=0,lsb=false,lsb_In=true,smode=1)

grainfactory3(g1str=1, g2str=1, g3str=1)
AddGrainC(2.0,2.0)
MergeChroma(aWarpSharp2(depth=30).aWarpSharp2(depth=10))
LimitedSharpenFaster(edgemode=2)
Crop(2,4,-22,-4).AddBorders(12,4,12,4)

###--- Resize 640x480 to 720x480 for DVD --- ###
Spline36Resize(720,height)
The script below trims 2 parts for the leading and trailing frames, then trims a 3rd section dfor the bulk of the commercial. Then the script joins all three parts for encoding:

Code:
VersionA = AVISource("I:\forum\faq\billct97\Colgate\Colgate2A_ivtc_01Qmd2.avi")
VersionB = AVISource("I:\forum\faq\billct97\Colgate\Colgate2B_ivtc_01Qmd2.avi")

VB1=VersionB.Trim(0,63)    #<- lead-in frames
VB2=VersionB.Trim(718,0)   #<- fade-out frames
VA=VersionA.Trim(64,717)   #<- main video
ColgateOut=VB1+VA+VB2
Return ColgateOut
Some detailed notes on filters used in the A and B scripts:

AVISource("I:\forum\faq\billct97\Colgate\Colgate2_ ivtc.avi")
ColorYUV(off_y=-5)
ConvertToYV12(interlaced=false)
### --- Next 3 lines are activated for testing only ### ---
## Crop(2,4,-22,-4)
## Histogram("Levels")
## return last

The file is opened with AviSource(). ColorYUV(off_y=-5) lowers the black levels slightly by lowering all values in the video's luma and chroma by 5 negative points. This brought the black start and end frames down to exactly y=16, which would equal RGB=0 in RGB display. But the ColorYUV() line wasn't added until after the video's borders were cropped off and levels were analyzed with the Histogram() function. When the 3 test lines are not commented-out with the "#" character, the frame's borders won't affect the histogram readings and the "return last" statement ends the program and returns the results. After the desired settings were determined, the ColorYUV() line was added, the three test lines were disabled with "#" characters, and the program resumed normally. Note that the 3 test lines are not needed in script B because the desired ColorfYUV values were already analyzed in script A.

Stab()
FixChromaBleeding()
ChromaShift(l=-2,c=-4)

There is some slight vertical and horizontal frame hop which is stabilized with the Stab() filter. FixChromaBleeding is a handy plugin that works with chroma only, mainly by reducing over saturation that causes bleeding. It's not perfect but it helps visibly where high saturation is the problem. ChromaShift() works on U and V chroma tracks only, shifting colors upward by 2 pixels and leftward by 4 pixels.

qtgmc(InputType=2,preset="medium",EZDenoise=4,deno iser="dfttest",GrainRestore=0.3)
QTGMC is a deinterlacer but here it's used on slightluy damaged progressive video (InputType=2). Rather than deinterlace, QTGMC is used here for its denoising (with EZDenoise and dfttest), and its motion-compensated anti-shimmer and edge and motion smoothing properties. As used with its "medium" preset parameter it's only mildly destructive of detail, some of which is restored internally and refreshed by re-inserting a small amount of the original grain to prevent an over-filtered look. There are stronger denoising values available, but this video is already a bit soft.

source=last
super = source.MSuper(pel=2, sharp=1)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, blksize=8, overlap=4, dct=0)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, blksize=8, overlap=4, dct=0)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, blksize=8, overlap=4, dct=0)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, blksize=8, overlap=4, dct=0)
MDegrain2(source,super, backward_vec1,forward_vec1,backward_vec2,forward_v ec2,thSAD=400)
In case no one noticed, this routine is copied almost verbatim from the MVTools2.dll motion vector tools documentation. The denoiser is MDegrain2 in the last line of this routine. There are dozens of variations of this routine in several big plugins. The code analyzes noise and motion in 2 frames in each direction. It's good for excess temporal grain and for what are called the "fuzzies" in edges, textures, and other noise patterns. Be careful though, as this filter can sometimes make motion-blurred moving objects disappear for a frame or two.

### --- 16bit dithering filter for noise problem --- ###
Dither_convert_8_to_16()
GradFun3(thr=0.8,mask=0,lsb=false,lsb_In=true,smod e=1)

GradFun3() (aka "gradation function" 3) is from the dither tools plugin, which contains dozens of 16-bit conversion and filtering routines. It's basically a smoother that helps clear color banding, hard transition edges and tough grain in areas of fine gradation between contrasting or color regions and in large areas such as sky, walls, shadows, etc. It can soften the entire video if improperly used, so it's used here only in Script B (it's not in Script A). It also helps avoid hard edges in facial tones. The conversion from normal 8-bit video data to 16-bit is done with standard dithering functions. If you need a couple of evenings' reading time, you can read about the filter in its original appearance at doom9 (Color banding and noise removal) while dithering is covered in many internet posts such as https://en.wikipedia.org/wiki/Dither and hundreds of others. GradFun3 and other features in DitherTools are covered briefly in online docs at http://avisynth.nl/index.php/Dither_tools#GradFun3.

grainfactory3(g1str=1, g2str=1, g3str=1)
AddGrainC(2.0,2.0)

Both functions add very fine, very ordered film-like grain to prevent an overly smooth appearance and create the illusion of optical "detail" with highly filtered media. Don't overdo it or it will be worse than the dirty grainy stuff that was removed.

MergeChroma(aWarpSharp2(depth=30).aWarpSharp2(dept h=10))
LimitedSharpenFaster(edgemode=2)

Merge Chroma() limits processing to only the chroma channels, where aWarpSharp2 literally "warps" color to cling more closely to defined edges -- so it is a sort of anti-bleed and anti-smear function. LimitedSharpenFaster() is an old-time favorite sharpener that doesn't overdo edge sharpening with artifacts, and "edgemode=2" sharpens only areas between edges, not edges themselves, to prevent a phony oversharpened look on soft video.

Crop(2,4,-22,-4).AddBorders(12,4,12,4)
Spline36Resize(720,height)

Crop() removes dirty or discolored border pixels and AddBorders creates new border pixels (the default color is black). Crop() removes 2 pixels from the left border, 4 pixels from the top, 22 pixels from the right, and 4 pixels from the bottom. AddBorders() adds new pixels to the left, top, right, and bottom in a way that centers the frame inside the new borders. The image is restored to its original dimensions without affecting the core image content itself. Finally, Spline36 resizes the 640x480 capture for DVD at 720x480. It would have been better if I had captured 720x480 to begin with, but years ago I didn't know right from left about capturing.

The final encode "Colgate_8661_cleanup_480p.mp4" at original film speed 23.976 fps is attached. Not perfect, but one can do only so much with bad video. Looks cleaner and brighter on TV than on a PC monitor (no surprise).

(continued, next post)


Attached Files
File Type: mpg Colgate_ 8661_sample.mpg (34.94 MB, 5 downloads)
File Type: mpg Colgate_ pv_s4670_sample.mpg (34.76 MB, 4 downloads)
File Type: mp4 Colgate_8661_cleanup_480p.mp4 (20.78 MB, 5 downloads)
Reply With Quote
The following users thank sanlyn for this useful post: captainvic (Yesterday), keaton (03-30-2020)
  #12  
03-29-2020, 12:36 PM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,564
Thanked 1,134 Times in 925 Posts
BAD TAPE CHRONICLES: TAPES FROM HELL, CONT'D:
Working with bad cable signals and bad tape from the 1990's, #2


The attached "Dentist_AG1980_sample.mpg" is a high bitrate encode of an otherwise unfiltered lossless capture from a TV broadcast. The original lossless YUY2 sample was too large for posting here, but if you want a lossless copy let me know. I can mount one in my pro account at Mediafire (no popups, no ads).


Date recorded: July 1991 (based on other ads on the tape promoting cable programs and dates).
Original recorder: Mainstream JVC 4-head onto Sony "HI-FI" tape at 6-hour slow speed.
Original broadcast: RF cable line -> 20w no-name RF cable amp -> cheap RF cable -> analog cable box -> cheap RF wire -> cheap Radio Shack 4-way amplified RF splitter with 4 composite outputs -> cheap composite wire to VCR.

Capture (2010): Panasonic AG-1980
-> s-video -> AVT-8710 (circa 2004)
-> s-video -> SignVideo PA-100 proc amp
-> s-video -> ATI AIW 9600XT AGP card
-> VirtualDub/huffyuv YUY2, 640x480. Original capture reviewed in VDub and cut-edited into lossless sections, archived to external HDD as Lagarith YUY2.

Despite my using the AG-1980 for this capture, the noise level is still atrocious -- thanks to the bad cable line, inferior connections, and also to SONY "HI-FI" VHS tape, one of the grittiest, grainiest tapes I've ever seen. Years later I'm sorry I used that tape.

Because the sample's problems are similar to those in the previous example, I used a very similar script and filters. But there are additions: the first half of the sample is supposed to be mostly grayscale and the small amount of color looks ok. But the second half has atrocious skin tones thanks to the no-name cable amp I used and which simply wrecked the colors. It also over-amplified the signal, which gave problems with other VCRs. So the script was run 2 times: once with less color correction for the sake of the more neutral first half, then again with more RGB color correction to get less sickening skin tones in the latter half. At encode time I cut and joined the two results using the TMPGenc encoder app.

As usual, ivtc came first as a previous file used for input here. The script below is like the earlier post but with a few additions, which I'll note below:
Code:
AVISource("I:\forum\faq\billct97\dentist\dentist_ivtc.avi")

### --- the filters below work in YUY2 --- ###
ColorYUV(off_y=-5)
FixChromaBleeding()
ChromaShift(l=-2,c=-6)

### --- processing below requires YV12 --- ###
ConvertToYV12(interlaced=false)
hqdering()
qtgmc(InputType=2,preset="medium",EZDenoise=2,denoiser="dfttest",GrainRestore=0.3,\
  border=true,ChromaNoise=true)

source=last
super = source.MSuper(pel=2, sharp=1)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, blksize=8, overlap=4, dct=0)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, blksize=8, overlap=4, dct=0)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, blksize=8, overlap=4, dct=0)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, blksize=8, overlap=4, dct=0)
MDegrain2(source,super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400) 

grainfactory3(g1str=1, g2str=1, g3str=1)
AddGrainC(2.0,2.0)
MergeChroma(aWarpSharp2(depth=30).aWarpSharp2(depth=10))
LimitedSharpenFaster(edgemode=2)
Crop(4,6,-20,-4).AddBorders(12,4,12,6)

Dither_convert_8_to_16 ()
Dither_resize16 (720, 480, kernel="spline36", cplace="MPEG2")
Dither_convert_yuv_to_rgb(matrix="601",interlaced=false,tv_range=true,cplace="MPEG2",\
  lsb_in=true, output="rgb32")

return last
Lines that make this script different from those in previous first:

hqdering()
qtgmc(InputType=2,preset="medium",EZDenoise=2,deno iser="dfttest",GrainRestore=0.3,\
border=true,ChromaNoise=true)

the hqdering() filter works on mildly discolored and noisy edge artifacts that are rather common in VHS, even with high end players. Again, QTGMC is used as a smoother and denoiser, not to deinterlace; its "border" parameter avoids border flutter when resizing new frames. ChromaNoise=true activates chroma cleaning.

Dither_convert_8_to_16 ()
Dither_resize16 (720, 480, kernel="spline36", cplace="MPEG2")
Dither_convert_yuv_to_rgb(matrix="601",interlaced= false,tv_range=true,cplace="MPEG2",\
lsb_in=true, output="rgb32")

These lines add 16-bit dithered processing to the resize and the conversion from YV12 to RGB, the latter used for the VirtualDub filters. Greater 16-bit density helps make resizing and color reinterpretation cleaner (but check the results first, as sometimes it can overly smoothed and phoney). The functions are described in the Dither Tools Avisynth Wiki documentation at http://avisynth.nl/index.php/Dither_tools. See the table of contents list for details on filter functions.

The VirtualDub filters that were used are CamcorderColorDenoise, ColorMill, gradation curves, and Donald Graft's hue/saturation/intensity. These filters were used for the color-corrected portion of the clip. The RGB filter that was used for the first neutral-colored half was NeatVideo v3.0 at very conservative settings and low power, basically used as a mild tweak. You can eliminate Neatvideo (it's a paid retail filter) and substitute EZDenoise=6 in the QTGMC line.

The VDub .vcf settings file for the RGB correction filters is attached as "VirtualDub_RGB_color_settings.vcf". The NeatViedo .dnp and .nfp processing files are attached as "NeatVideo_Sample_and_Filter_Settings.zip".

The final 23.976 fps 4:3 encoded results are attached as "Dentist_AG1980_cleanup_480p.mp4".


Attached Files
File Type: mpg Dentist_AG1980_sample.mpg (34.79 MB, 2 downloads)
File Type: mp4 Dentist_AG1980_cleanup_480p.mp4 (26.02 MB, 1 downloads)
File Type: vcf VirtualDub_RGB_color_settings.vcf (3.8 KB, 1 downloads)
File Type: zip NeatVideo_Sample_and_Filter_Settings.zip (25.4 KB, 2 downloads)
Reply With Quote
The following users thank sanlyn for this useful post: keaton (03-30-2020)
  #13  
03-29-2020, 01:00 PM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,564
Thanked 1,134 Times in 925 Posts
BAD TAPE CHRONICLES: TAPES FROM HELL, CONT'D:
Working with bad cable signals and bad tape from the 1990's, #3

The attached "WheatThins AG1980 sample.mpg" is a high bitrate encode of an otherwise unfiltered lossless capture from a TV broadcast. The original lossless YUY2 sample was too large for posting here, but if you want a lossless copy post a request here. I can mount one in my pro account at Mediafire (no popups, no ads).


This time I used slightly better tape and a better VCR for its era, a pricey SONY SLV-695HF, although at slow 6-hour mode. Noise was about the same but overall accutance and color were better than most recordings under my stupid cable setup. The opening and closing black frames are a cable noise blizzard. And you can easily hear the low-frequency buzz that was coming over the USA cable channel during the broadcast. This time there were a few complications in the form of spots. This clip is an opportunity to demo a couple more of my favorite VHS filters.

The composite image below displays cropped pics of the kind of spots and freaky edge noise in this clip:



(Above) Upper left, note the white speck to the right of Sandy's face. Shows up like a lighthouse as the camera pans. Upper right, a broken black line and a white "blink" in the extreme upper right corner of the top border. Bottom left, another stupid white speck on the lower left edge of the yellow box. Bottom right, yellow bleed and ghost image on right side of yellow box, and note all the ugly streaky noise on the white sweater along the edges of the outer blue sweater and elsewhere in the image. These cropped images are from /mpg sample frames 40, 101, 277 and 323.

Date recorded: January 1991 (based on other ads on the tape promoting cable programs and dates).
Original recorder: SONY SLV-695HF on Maxell premium grade tape at 6-hour slow speed.
Original broadcast: RF cable line -> 10w Archer cable RF amp -> cheap RF cable -> analog cable box -> cheap composite wire -> cheap Radio Shack 4-way amplified RF splitter with 4 composite outputs -> cheap composite wire to VCR.

Capture (2009): Panasonic AG-1980 VCR
-> svideo -> AVT-8710 (circa 2004)
-> SignVideo PA-100 proc amp
-> s-video -> ATI AIW 9600XT AGP card
-> VirtualDub/huffyuv YUY2, 640x480.
Original capture reviewed in VDub and cut-edited into lossless sections, archived to external HDD as Lagarith YUY2.

Again, the script is similar to those in preceding posts. I'll give details about the differences below:

Code:
AVISource("I:\forum\faq\billct97\WheatThins\WheatThins_ivtc.avi")
a=last
save_audio = a
a

ColorYUV(off_y=-8)
Tweak(cont=1.05,dither=true,coring=false)
Levels(16,1.0,255,16,250,dither=true,coring=false)
#Crop(4,0,-20,-8)
#ConvertToYV12(interlaced=false)
#Histogram("Levels")
#return last

FixChromaBleeding()
ChromaShift(c=-4)
FixVHSOversharp(20,16,12)

ConvertToYV12(interlaced=false)
ReplaceFramesMC(221,1)
b=last
b0=b
b01=b.ReplaceFramesMC(32,1).Crop(248,114,-382,-356)
b02=Overlay(b0,b01,x=248,y=114)
b1=ReplaceFramesSimple(b,b02,mappings="32")

b0=b1
b01=b1.ReplaceFramesMC(81,1).Crop(120,434,-508,-38)
b02=Overlay(b0,b01,x=120,y=434)
b2=ReplaceFramesSimple(b1,b02,mappings="81")

b2
hqdering()
qtgmc(InputType=2,preset="medium",EZDenoise=4,denoiser="dfttest",GrainRestore=0.3,\
  border=true,ChromaNoise=true)
source=last
super = source.MSuper(pel=2, sharp=1)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, blksize=8, overlap=4, dct=0)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, blksize=8, overlap=4, dct=0)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, blksize=8, overlap=4, dct=0)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, blksize=8, overlap=4, dct=0)
MDegrain2(source,super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400) 

santiag()
SmoothUV()
AddGrainC(2.0,2.0)
MergeChroma(aWarpSharp2(depth=30).aWarpSharp2(depth=10))
LimitedSharpenFaster(edgemode=2)
Crop(4,0,-20,-8).AddBorders(12,4,12,4)

Dither_convert_8_to_16 ()
gradfun3(thr=0.3,lsb_In=true,lsb=true)
Dither_resize16 (720, 480, kernel="spline36", cplace="MPEG2")
GradFun3(thr=0.3)
Dither_convert_yuv_to_rgb(matrix="601",interlaced=false,tv_range=true,cplace="MPEG2",\
   lsb_in=true,output="rgb32")
AudioDub(last,save_audio)

return last
AVISource("I:\forum\faq\billct97\WheatThins\WheatT hins_ivtc.avi")
a=last
save_audio = a
a

the AVISource() function opens and decodes the video, then immediately an arbitrary value "a" is assigned to the input. "a" is a value in memory that I invented, which works as long as no other entity in the program is also called "a". In the third line I create a new copy of "a" and call it "save_audio", which saves in memory a copy of the clip with its original audio stream. Frame replacement later in the script will create audio breaks, so Save_audio will be used later to restore the audio. In the last line, clip "a" is named explicitly to restore processing focus to the "a" clip.

ColorYUV(off_y=-8)
Tweak(cont=1.05,dither=true,coring=false)
Levels(16,1.0,255,16,250,dither=true,coring=false)
#Crop(4,0,-20,-8)
#ConvertToYV12(interlaced=false)
#Histogram("Levels")
#return last

ColorYUV is used to lower all luminance values in the clip by 8 negative points for a more realistic black level. Then Tweak() is used to visibly raise contrast to restore the original "snap" of a brighter high end. Because this creates an occasional highlight overshoot past y=235, the Levels() function slightly lowers highlight extension using a value below 255. One arrives at these values by examining the clip in Avisynth's histogram function; the coded lines that create and test the histogram are disabled with "#" characters after they are used. The Tweak() and Levels() functions are documented in hundreds of scripts and in Avisynth's built-in Help documents. Online, one can find the details at http://avisynth.nl/index.php/Tweak and http://avisynth.nl/index.php/Levels. Using the Tweak() and Levels() functions is very basic Avisynth.

FixVHSOversharp(20,16,12)
This is a YUY2-only filter that helps alleviate bright or dark edge halos and sharpening effects. Often these effects originate with the broadcast and are sometimes smeared edges caused by transmission noise. They're also caused by "echoes' from the multitude of RF splitters I was using. FixVHSOversharp() can be used for right-hand or left-hand halo and mild ghost-like effects. You can also download it from the forum as fixvhsoversharp_25_dll_20030723.zip.

ReplaceFramesMC(221,1)
b=last
b0=b
b01=b.ReplaceFramesMC(32,1).Crop(248,114,-382,-356)
b02=Overlay(b0,b01,x=248,y=114)
b1=ReplaceFramesSimple(b,b02,mappings="32")

b0=b1
b01=b1.ReplaceFramesMC(81,1).Crop(120,434,-508,-38)
b02=Overlay(b0,b01,x=120,y=434)
b2=ReplaceFramesSimple(b1,b02,mappings="81")

b2

These lines eliminate the spots and white flecks shown in the composite image posted earlier. Note that the frame numbers here don't refer to the original sample; rather the frame numbers in these lines are from the ivtc'd version, which has fewer frames than the original because duplicte telecine frames are removed by IVCT.

The ReplaceFramesMC(221,1) statement is the lazy way of replacing frame 221 with a new one: it simply interpolates a new frame 221 based on data and motion in the previous and following frame. It works OK here because there isn't much motion. However, with more motion some bizarre distortions would result, so the statements that follow create small "patches" in the video that cover the undesirable defects rather than re-interpret the entire frame.

First, a variable "b" is created to contain a copy of the clip ("last") that has been processed so far. Then another variable "b0" is created to point to "b" for later use. In the next line, "b01" is code that creates a copy of the clip in which a complete new frame 32 is created, and then that frame is cropped down to a small 10x10-pixel patch that contains clean video. This patch will be used to overlay the 10x10-pixel area in the original clip that contains the annoying white fleck. Using this small speck eliminates processing the other parts of the frame where excess motion would cause weird distortions.

The line b02=Overlay(b0,b01,x=120,y=434) creates a new video clip in which the clean new 10x10 patch is overlaid onto the unwanted 10x10 area. This covers the white spot in frame 32 with a new patch of cleaned data -- but it covers the 10x10 patch in every frame of clip B02. That's not what we want. The only cleaned frame we want from clip b02 is frame 32. Therefore, the following statement b1=ReplaceFramesSimple(b,b02,mappings="32") takes only the cleaned frame 32 from b02 and uses ReplaceFramesSimple() to replace the old frame 32 in the original "b". The new video is named "b1". ReplaceFramesSimple is one of three functions is the plugin Remapframes.dll.

The routine that follows this 4-line cleanup creates a different patch from a different part of the new "b1" video and creates a new cleaned version called "b2". The last statement of the routines explicitly names "b2" in order to keep processing focus on the newest b2 version of the video.

santiag()
SmoothUV()

santiag (http://avisynth.nl/index.php/Santiag) is an anti-alias filter used here to do a little more edge and line smoothing. Use it sparingly because strength values stronger than the defaults can seriously soften the video. Here, default values are used because other values are not stipulated in the command line. SmoothUV (http://avisynth.nl/index.php/SmoothUV) is a classic old plugin used as a mild chroma cleaner to smooth away blotchy discolorations. It helps smooth skin tones.

AudioDub(last,save_audio)
return last

The AudioDub() built-in function takes the video that is the "last" thing done in the previous statement and dubs in the audio stream that was saved earlier in Save_audio. This replaces any damage done to the audio by the frame replacement routines. Finally, the statement "return last" ends the program by returning the "last" thing that was executed, which was AudioDub() in the previous statement. Why do we need "return last" here? Because in the course of processing, the script created several video clips (with names like names a, b, b0, b1, b2, etc.) and Avisynth has to know which clip we want returned.

The conversion to RGB is done with 16-bit dithering as described in the earlier post. RGB is used by the VirtualDub filters that are loaded when VDub is executing the avisynth script. The filters used are ColorMill, gradation curves, and the exorcist anti-ghost filter. Exorcist alleviates (but doesn't completely remove) the edge ghosting in the original, but in doing so it darkens the image. Gradation curves is used to restore the original luminance levels. There are also 2 instances of the FadeFX filter at the end of the VDub filter chain (https://web.archive.org/web/20180627...deo/fadefx.vdf). The first instance of FadeFX replaces the dirty fade-from-black frames at the start of the clip. The second copy of fadeFX replaces the last few fade-to-black frames at the end of the clip.

The VDub filter chain's order and filter settings are attached in the .vcf file WheatThins_VDub_Settings.vcf.

The results of the scripts and filters were encoded as the attached "WheatThins_AG1980_DVD pulldown.mpg". The mpg is encoded and configured for DVD with 3:2 pulldown restored by the TMPGenc encoder for 29.97fps DVD playback.


Attached Images
File Type: jpg Annoying Glitches.jpg (44.1 KB, 18 downloads)
Attached Files
File Type: mpg WheatThins AG1980 sample.mpg (17.38 MB, 1 downloads)
File Type: mpg WheatThins_AG1980_DVD pulldown.mpg (13.51 MB, 1 downloads)
File Type: vcf WheatThins_VDub_Settings.vcf (3.8 KB, 1 downloads)
Reply With Quote
The following users thank sanlyn for this useful post: keaton (03-30-2020)
  #14  
03-29-2020, 09:12 PM
billct97 billct97 is online now
Premium Member
 
Join Date: Jan 2018
Posts: 36
Thanked 1 Time in 1 Post
First, thank you so much for giving me more examples than I could get through on a rainy cold Spring day with nowhere else to be. I did have some old notes and HDD cleanup to do first so I was only able to get through the first example but it all made sense. I would like to ask what will likely be very basic questions instead of just accepting what I read and moving on.

1. Why did you encode the final mp4 at 23.976 fps? It would seem to me if you did the 3:2 conversion you would leave it there at 29.97 fps.

2. Virtual Dub gives me an error when I try to open Colgate_8661_cleanup_480p.mp4. The pop up error says "Unable to initialize Quicktime: InitializeQTML". The .mpg files open and play just fine in Virtual Dub. The .mp4 opens and plays ok in VLC. I'd assume I need to install a codec in Virtual Dub. Just not sure what codec and how it is done.

3. What do you use to convert to MPEG for the final copy? My plan is to archive everything as lossless compressed with Lagarith but I may also want to upload to YouTube or share a copy with friends. So a balance of portability and quality is what I would think I would want for a final format?

4. Since you offered I would like to have a copy of the original lossless file(s). Whatever you are willing to share. That way I can work with your scripts and the original avi until I have succeeded in replicating your results.

Thank You!
Bill
Reply With Quote
  #15  
03-30-2020, 06:16 AM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,564
Thanked 1,134 Times in 925 Posts
Quote:
Originally Posted by billct97 View Post
Why did you encode the final mp4 at 23.976 fps? It would seem to me if you did the 3:2 conversion you would leave it there at 29.97 fps.
By the term "3:2 conversion" I think you meant inverse telecine. Inverse telecine removes pulldown fields, so the video no longer plays at 29.97. When pulldown is removed, the video will play at its original frame rate which in this case is 23.976fps. If you want it to play AT 29.97 again, pulldown must be restored when the video is encoded.

If you want to mount the video on the internet, it can't be telecined. So you will need a progressive 23.976 version for the 'net. Also, the last time I heard about YouTube submissions, video intended for 4:3 display aspect ratio has to be submitted in a certain frame size, or YouTube will reformat it for you using the quickest and dirtiest means possible. They prefer submissions in 720p format. 720x480 can't be used on YouTube. The 4:3 video has to be resized to 960x720, then 160 black border pixels are added to each side to make a 4:3 video enclosed in a 1280x720 frame. Anamorphic frames (720x480) can't be used.

In Avisynth you can do it this way:
Code:
AviSource("path to 23.976 720x480 video")
Spline36Resize(960,720)
AddBorders(160,0,160,0)
Don't try this with interlaced video. If it's interlaced, it must be deinterlaced for resizing or for posting to the internet. If you submit interlaced video it will be deinterlaced using crappy fast and dirty software that you'd never use yourself.

If you want the 720x480 progressive 23.976 video to be on DVD or standard def BluRay, it must be encoded for a 4:3 display aspect ratio (DAR) and 3:2 pulldown must be added. When authored to DVD or BluRay disc, the authoring program will treat the telecined video as "interlaced". If you want to stream it as 23.976 or place it on a hard drive or USB stick as "data", encode it as-is for a 4:3 DAR and play it as 23.976.

However -- if you want to combine your progressive 720X480 23.976 video with interlaced 29.97 video, you can't do it. You have to encode the 23.976 video with pulldown for 29.97fps playback, and treat it as interlaced. Then you can combine it with 29.97fps interlaced sections.

Quote:
Originally Posted by billct97 View Post
Virtual Dub gives me an error when I try to open Colgate_8661_cleanup_480p.mp4. The pop up error says "Unable to initialize Quicktime: InitializeQTML". The .mpg files open and play just fine in Virtual Dub. The .mp4 opens and plays ok in VLC. I'd assume I need to install a codec in Virtual Dub. Just not sure what codec and how it is done.
You need the ffmpeg plugin. Attached is a copy of 32-bit plugins for VirtualDub. Download it to a separate area on a hard drive and unzip it. It will unzip into a folder called "plugins32". You should also have a "plugins32" folder in your VirtualDub program folder. if you don't, create it. Then look at the .vdf plugins inside the unzipped Plugins32 folder and add the .vdf's that you need to your VirtualDub "plugins32". The particular .vdf in question is the ffmpeg input plugin.

Quote:
Originally Posted by billct97 View Post
What do you use to convert to MPEG for the final copy?
I used TMPGenc Mastering Works for both the mpg and the mp4/h.264 encodes. Sometimes I use Avidemux for h.264 (it's not a very good MPEG encoder).

Quote:
Originally Posted by billct97 View Post
Since you offered I would like to have a copy of the original lossless file(s).
Colgate_Number2 8661.avi ~190m https://www.mediafire.com/file/nymgv..._8661.avi/file
Colgate_tune pv-s4670.avi ~189mb https://www.mediafire.com/file/iudpe...s4670.avi/file
GI1_e_dentist Ag1980.avi ~182mb https://www.mediafire.com/file/jogh2...g1980.avi/file
Living4 - WheatThins AG1980.avi ~97mb, but the forum says >99 ???
https://www.mediafire.com/file/a7guj...G1980.avi/file


Attached Files
File Type: zip plugins32.zip (581.2 KB, 2 downloads)
Reply With Quote
  #16  
03-30-2020, 06:49 AM
billct97 billct97 is online now
Premium Member
 
Join Date: Jan 2018
Posts: 36
Thanked 1 Time in 1 Post
Quote:
Originally Posted by sanlyn View Post
You need the ffmpeg plugin. Attached is a copy of 32-bit plugins for VirtualDub. Download it to a separate area on a hard drive and unzip it. It will unzip into a folder called "plugins32". You should also have a "plugins32" folder in your VirtualDub program folder. if you don't, create it. Then look at the .vdf plugins inside the unzipped Plugins32 folder and add the .vdf's that you need to your VirtualDub "plugins32". The particular .vdf in question is the ffmpeg input plugin.
I added the ffmpeg plugin (thank you for that). The video now plays in Virtual Dub (if I check No Audio) but I get this error: "No audio decompressor could be found to decompress the source audio format. (source format tag: 00ff)". I also added lordsmurfs bundle from post 5895 but I believe those are all Avisynth filters.
Reply With Quote
  #17  
03-30-2020, 10:02 AM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,564
Thanked 1,134 Times in 925 Posts
Quote:
Originally Posted by billct97 View Post
I added the ffmpeg plugin (thank you for that). The video now plays in Virtual Dub (if I check No Audio) but I get this error: "No audio decompressor could be found to decompress the source audio format. (source format tag: 00ff)". I also added lordsmurfs bundle from post 5895 but I believe those are all Avisynth filters.
Install the system-wide AAC/ACM codec. https://www.videohelp.com/software/AAC-ACM-Codec

Reply With Quote
  #18  
03-30-2020, 11:49 AM
billct97 billct97 is online now
Premium Member
 
Join Date: Jan 2018
Posts: 36
Thanked 1 Time in 1 Post
Quote:
Originally Posted by sanlyn View Post
Install the system-wide AAC/ACM codec. https://www.videohelp.com/software/AAC-ACM-Codec
Got it working. I also needed ffdshow or I got a "missing codec AVC1" on a black screen in Virtual Dub.
1. ffmpeg (sanlyn posted the plugin earlier)
2. AACACM19 (sanlyn posted the link)
3. ffdshow - https://www.videohelp.com/software/ffdshow
Reply With Quote
  #19  
03-30-2020, 08:57 PM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,564
Thanked 1,134 Times in 925 Posts
Yes, there is always something else! I've had ffdshow for so many years I keep forgetting it's there.
Reply With Quote
  #20  
03-30-2020, 09:09 PM
billct97 billct97 is online now
Premium Member
 
Join Date: Jan 2018
Posts: 36
Thanked 1 Time in 1 Post
Taking things very slowly as I had to work today. Can you please fix the IVTC line in Post #11 as it tripped me up for a few minutes. If someone who is new finds this later on I suppose it would be good to have it corrected.

TFM().(Tdecimate) should be TFM().Tdecimate()
Reply With Quote
Reply




Similar Threads
Thread Thread Starter Forum Replies Last Post
AviSynth Scripts/Guide dnppro Restore, Filter, Improve Quality 5 09-05-2019 12:15 PM
Are AviSynth scripts, VirtualDub filters run as separate processes? stevevid Restore, Filter, Improve Quality 7 09-25-2018 04:35 PM
Best scripts for de-telecine ragu0012 Restore, Filter, Improve Quality 8 03-17-2018 08:36 PM
VCD to DVD conversions-question? MagnificentMarcus Encode, Convert for discs 11 09-05-2006 02:38 PM

Thread Tools



 
All times are GMT -5. The time now is 10:48 AM