Restoration of VCD

[naripeddi]

I have a precious wedding video of which I have only a VCD as source (PAL). The original tape was long gone unfortunately. It was never converted to DVD.

The VCD, obviously, looks absolutely crap with lots of compression blocks, mosquito noise when played on even a 19 inch computer monitor.

Are there any generic AviSynth scripts or VDub filters that can attack typical VCD problems? This video is very important to me.

Should I post sample video of the VCD? If yes, would the MPEG file be fine or I need to convert it to Huffyuv?

NOTE: I am aware of Garbage In Garbage Out, VCD can never be made to look like DVD quality...etc. My intention is to do whatever little corrections that can be done to make the VCD look slightly better. :-)

-- merged --

Ok now I realized I asked this same question back in 2011 in the below thread. I am sure the suggestions still apply.

http://www.digitalfaq.com/forum/vide...-workflow.html

-- merged --

I couldn't make any meaningful progress in restoring this VCD.

Could you suggest me the Avisynth or any other filters to correct this VCD? Unfortunately, I do not have the original tape and this VCD is the only source there is.

A short unaltered representative clip is attached. The clip is extracted using Dgindex.

Appreciate your help.

Regards

[themaster1]

Try this (it's a beginning, it may need a good antialiasing filter that's not smoothing everything...)

MPEG2Source("VCD Source Clip.d2v", cpu=0, info=3)
SetMTMode(2,4)

super = MSuper(pel=2, sharp=1)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400)
SetMTMode(2,4)
c=last 
#even fields
# Motion compensation on frames -2...+2
even = c.SelectEven()
super_even=even.MSuper()
vf2 = super_even.MAnalyse (isb=false, delta=2, overlap=4)
vf1 = super_even.MAnalyse (isb=false, delta=1, overlap=4)
vb1 = super_even.MAnalyse (isb=true,  delta=1, overlap=4)
vb2 = super_even.MAnalyse (isb=true,  delta=2, overlap=4)
cf2 = MCompensate (even, super_even, vf2, thSAD=400)
cf1 = MCompensate (even, super_even, vf1, thSAD=400)
cb1 = MCompensate (even, super_even, vb1, thSAD=400)
cb2 = MCompensate (even, super_even, vb2, thSAD=400)
 # dfttest on motion-compensated clip
Interleave (cf2, cf1, even, cb1, cb2)
SetMTMode(2,4)
dfttest (sigma=4, sigma2=4, tbsize=3, lsb=true)
SelectEvery (5, 2)
 # Gradient smoothing
presmooth = last
SmoothGrad (radius=20, thr=0.15)
SmoothGrad (radius=12, thr=0.25, ref=presmooth)
 # Back to 8 bits, ordered dithering
DitherPost ()
filtered_even=last
 #odd fields
# Motion compensation on frames -2...+2
odd = c.SelectOdd()
super_odd=odd.MSuper()
vf2 = super_odd.MAnalyse (isb=false, delta=2, overlap=4)
vf1 = super_odd.MAnalyse (isb=false, delta=1, overlap=4)
vb1 = super_odd.MAnalyse (isb=true,  delta=1, overlap=4)
vb2 = super_odd.MAnalyse (isb=true,  delta=2, overlap=4)
cf2 = MCompensate (odd, super_odd, vf2, thSAD=400)
cf1 = MCompensate (odd, super_odd, vf1, thSAD=400)
cb1 = MCompensate (odd, super_odd, vb1, thSAD=400)
cb2 = MCompensate (odd, super_odd, vb2, thSAD=400)
# dfttest on motion-compensated clip
Interleave (cf2, cf1, odd, cb1, cb2)
SetMTMode(2,4)
dfttest (sigma=4, sigma2=4, tbsize=3, lsb=true)
SelectEvery (5, 2)
# Gradient smoothing
presmooth = last
SmoothGrad (radius=20, thr=0.15)
SmoothGrad (radius=12, thr=0.25, ref=presmooth)
# Back to 8 bits, ordered dithering
DitherPost ()
filtered_odd=last
Interleave(filtered_even, filtered_odd)
###############
SetMTMode(2,4)
# Sharpening on non edge
LimitedSharpenFaster(ss_x=1.3,ss_y=1.5,Smode=3,strength=100,soft=0,edgemode=2,wide=false)  # YV12 & progressive only
ConverttoRGB32(matrix="rec601",interlaced=false)
LoadVirtualDubPlugin("C:\Program Files (x86)\VirtualDub\plugins\msu_sharpen.vdf", "MSUSmartSharpen", 1)
MSUSmartSharpen(0) # 0 (small) to 5 (mad)
converttoyv12(matrix="Rec601",interlaced=false)

original vs processed:

[sanlyn]

The original sample has choppy motion because alternate fields in the interlaced original were discarded when creating the 352x288 VCD, which is not an interlaced format. Dropping alternate fields reduced the original temporal resolution from 50 fields per second to 25 fields per second. The sample also shows the effects of low bitrate and poor deinterlacing (aliasing, broken lines, shimmer, macroblocks, detail loss, etc.). There are also obvious oversharpening edge halos and some mild but visible chroma shift and bleed. The sample also has unsafe luma levels that exceed y=16-235 -- in fact the levels even exceed y=0-255, with strong bright clipping in parts of the image. Gamma levels also change considerably during the scene due to the way the camera reacted to light changes. There is not much grain in the sample, so I didn't use specific degrainers.

I used the AutoAdjust plugin to help keep overall levels and gamma a little more calm. The Tweak() and Levels() built-ins were used to maintain levels closer to y=16-235. Note that black borders are y=0 and will affect the levels filters so I cropped off the black pixels but restored them later to center the image vertically. Nothing can be done to fix the camera lens flare in the second half of the clip or to fix the sharp clipping of brights, which occurred partially in-camera and partially in earlier processing.

I used DeBlock_QED to smooth microblocks. FixVHSOversharp helped to clean right and left edge halos. I used QTGMC in progressive mode to reduce shimmer, repair broken lines and aliasing, and to clean up some of the chroma blotches. The old cnr2 plugin also reduced more of the chroma noise. aWarpSharp2 in merge-chroma mode helped tighten blurred chroma around edges.

The original sample has 8 pixels of black border across the bottom, which was apparently a rather basic method of covering bottom-border head switching noise but doesn't center the image vertically. I removed that bottom border and replaced it with 4 pixels of top border and 4 of bottom border, which centers the image. I used nnedi3 to resize the image 2X to 704x576. This size is legal for DVD but won't alwys be acceptable if you intend to go to standard definition BluRay. Also 704x576 can't be joined to other videos if others are 720x576. I added 8 black p[ixels to each side of the image to make the frame 720x576. The final result is progressive video but is encoded as interlaced to match other 576i videos you might be working with.

VCD is a low-resoltion, low detail format that was never intendedm for large-screen display, so you can't get the detail you see in larger frames. Detail can't be "created", so I used only mild sharpening to avoid the usual artifacts.

MPEG2Source("J:\forum\faq\naripeddi\VCD Source Clip.d2v")

#--- temporarily remove border mask that affects levels filters ---#
Crop(0,0,0,-8)

#--- Clean up unsafe and inconsistent yuv levels a little ---#
AutoAdjust(auto_gain=true,gain_mode=1,dark_limit=5.0,gamma_limit=7.0,bright_limit=1.0)
Tweak(cont=0.92,dither=true,coring=false)
Levels(12, 1.05, 255, 16, 250,dither=true,coring=false)

#--- restore original frame height & center image vertically ---#
AddBorders(0,4,0,4)

DeBlock_QED()
#--- clean up right and left edge oversharpen halos ---#
ConvertToYUY2(interlaced=false)
FixVHSOversharp(20,16,12)
FixVHSOversharpL(20,12,8)

#--- clean up shimmer, bad deinterlace edges, and chroma noise ---#
ConvertToYV12(interlaced=false)
QTGMC(InputType=3,preset="slower",EZDenoise=8,denoiser="dfttest",\
 ChromaMotion=true,border=true,ChromaNoise=true,DenoiseMC=true,GrainRestore=0.3)
Cnr2(mode="ooo", scdthr=255.0, ln=255, lm=222, un=255,\
 um=255, vn=255, vm=255)

#--- clean up some chroma shift and chroma bleed ---#
MergeChroma(aWarpSharp2(depth=30).aWarpSharp2(depth=15))
ChromaShift(c=-2)

#--- resample to full D1 DVD image, sharpen a little ---#
nnedi3_rpow2(rfactor=2,cshift="spline36resize")
LimitedSharpenFaster(strength=400,edgemode=2)
AddBorders(8,0,8,0)
return last

If you don't want a frame resize specifically for DVD or if you want something square-pixel for web mounting, replace the last 5 lines in the script with the following 4 lines:

#--- resample for web posting, sharpen a little ---#
spline36Resize(640,480)
LimitedSharpenFaster(strength=400,edgemode=2)
return last

I used DGIndex to create a .d2v project file for decoding the m1v with Avisynth. When creating a .d2v file, DGindex also creates a demuxed audio file. If you wish, you can attached the demuxed audio file and I can demonstrate how to open the video using MPEG2Source along with another filter that can open and remux the audio for you.

@theMaster1; I didn't see much grain in the sample but saw macroblocks and shimmer, so I'm wondering why you used Mdegrain. Thanks for the script, however. Those MV routines look interesting.

[naripeddi]

Many thanks to themaster1 and sanlyn for the scripts. @sanlyn, thanks for the education too.

The video was shot by professional videographers on DV tapes in wedding (may be better DV than consumer DV), and at the time DVD was not offered as an option as delivery format, neither was I appreciative of the DVD vs VCD quality difference. So, we went with the default option, which was VCD. Those days CRT ruled the TV space and the VCD looked perfectly fine on those TV's.

I will apply these scripts to my entire video and post the results back.

[lordsmurf]

I don't have access to my main video computer this week, special project, but do plan to look at this. And I have experienced with Indian VCDs.

A VCD mostly has issues due to bitrate (macroblocks, mosquito noise) and deinterlacing. While some VCDs had good deinterlacing, or even IVTC, most did not. Thankfully, methods to improve such sources have themselves improved in recent years (past decade or less). QTGMC modes are one example.

I agree, both themaster1 and sanlyn have given you nice scripts to start with.

[naripeddi]

Pardon my ignorance, after saving these scripts in notepad as AVS file, and opening in VirtualDub, how do I get my final output file in MPG?

[naripeddi]

Quote:

Originally Posted by lordsmurf

I don't have access to my main video computer this week, special project, but do plan to look at this. And I have experienced with Indian VCDs.

A VCD mostly has issues due to bitrate (macroblocks, mosquito noise) and deinterlacing. While some VCDs had good deinterlacing, or even IVTC, most did not. Thankfully, methods to improve such sources have themselves improved in recent years (past decade or less). QTGMC modes are one example.

I agree, both themaster1 and sanlyn have given you nice scripts to start with.

Much appreciated LS.

[sanlyn]

Quote:

Originally Posted by naripeddi

Pardon my ignorance, after saving these scripts in notepad as AVS file, and opening in VirtualDub, how do I get my final output file in MPG?

Most users save the output of Avisynth scripts as lossless AVI, then they combine their various lossless AVI working files into encoding editors such as Corel Video Studio or a more expensive Adobe editor (I would go for a decent but more affordable Corel product, which has fairly good encoding and DVD authoring), or you can use TMPGenc Mastering Works or TMPGenc Authoring Works. These apps are designed for cutting/joining/encoding, they can't perform restoration tasks like Avisynth or VirtualDub. Or you can use a free but competent app for encoding and burning such as AVStoDVD. Obviously, retail software has more features.

I don't know what you used to demux and create your .m1v sample but it would also have created a matching audio file. The video and audio can be joined using Avisynth. If you post a sample of .m1v and its matching audio output file, we can post a script showing how it's done.

[naripeddi]

I used Dgindex.exe to cut the sample portion of the VCD. Yes, it has created an audio file in .mp2 format (attached). Bear in mind that there is nothing but a lot of noise in the audio, I am not worried about the audio part restoration for now, since it will be replaced with some sort of music.

So you are saying that the output of this Avisynth script will be a lossless clip (though we started off with VCD to begin with), which again needs to be encoded either to VCD or DVD as per my choice. I would prefer to encode to DVD, I see you have resized it to 720X576. I will do DVD encoding later in Avidemux since I do not have paid software for it.

As for my cutting joining needs, I do not plan to do it. The entire video file will be processed with the script and that's it. Even in case I need cutting, joining, I have bought TMPGEnc Smart Renderer 5 which is serving me very well.

[naripeddi]

Sorry, tried to open the avs file in VirtualDub and I am facing several errors stating that certain plug-ins are not loaded, "/" can only be used at the beginning or end of a line...etc. I guess I do not have a proper set-up for Avisynth to work.

I started downloading (from avisynth.nl) and copying whatever plugin it said it didn't find into Avisynth plugins folder, but facing some new errors such as "There is no function named mt_LutSpa"....Some of the plugins I found as .dll and some are available as download only as an avsi script, which I assume I need to copy to pluin folder as well.

I cannot fully utilize both of your scripts until I sort out my setup I guess. Is there an Avisynth rar file that I can download which contains all these plug-ins already (like LordSmurf's VirtualDub with filters)?

I am using Windows 10 Home 64 bit.

[sanlyn]

Quote:

Originally Posted by naripeddi

I cannot fully utilize both of your scripts until I sort out my setup I guess. Is there an Avisynth rar file that I can download which contains all these plug-ins already (like LordSmurf's VirtualDub with filters)?

There have been attempts at building such a package, but with so many different versions of Avisynth it didn't prove practical. However, many of the plugins used in the posted scripts have been posted in various forum threads recently. I'll look them up and post back sometime tonight or tomorrow a.m.

Quote:

Originally Posted by naripeddi

I am using Windows 10 Home 64 bit.

I suggest using 32-bit Avisynth version 2.6 and 32-bit Virtualdub v.1.9.11. I would avoid the VirtualDub filter collection, as many are obsolete and many will never be used. You're better off finding 32-bit versions and installing them as needed. The same holds true for lossless codecs, although with Windows64-bit you can install 32-bit and 64-bit codecs in the same machine -- note that they are not installed in the same system folders. 32-bit codec dll's usually get installed in the SYSWOW64 folder, while 64-but codecs get installed in System32 (yes, I know it sounds haywire, but there's little that's been very logical about 64-bit Windows ever since Vista). When you install Avisynth 32-bit the default Avisynth program folder in Win64 is in ProgramFiles(x86).

You should use 32-bit all the way for Avisynth and VirtualDub. When you start mixing 32-bit and 64-bit components, you get errors.

[sanlyn]

Quote:

Originally Posted by naripeddi

I used Dgindex.exe to cut the sample portion of the VCD. Yes, it has created an audio file in .mp2 format (attached). Bear in mind that there is nothing but a lot of noise in the audio, I am not worried about the audio part restoration for now, since it will be replaced with some sort of music.

I understand. Meanwhile the audio .rtar sample gives ann error that says bit contains no archive. Indeed, at 378kb it's a rather small audio file if it's supposed to match the video file. In the future if you want to know how to join audio and video from d2v project files, it can be demonstrated later.

Here are downloads and links to the filters and plugins used in the avs script I posted earlier. The best way to manage them ism to create a separate folder for plugin downloads then create subfoldedrs for specific plugins or zip files,m and unzip the compressed files into their own subfolders. Do not unzip Avisynth plugin downloads directly into your Avisynth program's main plugins folder. Many plugin downloads contain additional files that don't belong in your main folder and can cause errors if they are in there.

Avisynth plugins use files names that end in .dll, .avsi, and .avs.
.dll's and .avsi files load automatically when an Avisynth script calls them in the text of the script.
.avsi files are in plain text format and can be read using Notepad or any other word processor.
.avs plugins are also plain text files but they don't load automatically when used in an Avisynth script. Instead, .avs plugins haven to loaded explicitly using the Import() function (http://avisynth.nl/index.php/Internal_functions#Import). .avs scripts are often used for plugins that have many different versions but contain common blocks of coded text that can be confusing if all of them are auto-loaded into a running script.

MPEG2Source is a function in the free DGMPGDec utility (https://www.videohelp.com/download/dgmpgdec158.zip). DGMPGDec also contains the DGIndex.exe utility to create demuxed d2v project files and demuxed audio files. A d2v project file can be created from VOB, MPG, MPEG2 and MPEG1 video files. Create a folder on your PC and name it "DGMPGDec", then download the DMPGDec .zip file into that folder. Unzip the file, and all the utilities and guides you need are in those unzipped files.

There is no install required, but you must place a copy of DGDecode.dll in your Avisynth plugins. DGDecode.dll is the only plugin thye utility needs in the Avisyth plugins folder.

The scriptm functions Crop(), Tweak(), AddBorders(), ConvertToYV12/YUY2(), MergeChroma(), and return() are built-in Avisynth functions. They are documented with examples in Avisynth's installed help files in the Avisynth program folder and are available on the 'net by searching Google for "Avisynth" followed by a function name.

The AutoAdjust plugin is in a previous forum post attachment as AutoAdjust260.zip

SmoothLevels is a function in the SmoothAdjust plugin. It is contained with 11 other plugins in the previously attached avisynth_plugins.zip, which has a total of 12 subfolders with very popular Avisynth plugins. Some of the included plugins in the script posted earlier are SmoothAdjust280, FixVHSOversharp, QTGMC_Plugins_Nov_2017, ChromaShift27, FixChromaBleeding, and aWarpSharp2, and any support files required by those filters. Note the instructions that contain important details about how the files are installed and used, especially for the QTGMC package. Other plugins contained in the .zip are ContrastMask, DePan_1_13_1, FixRips, RemoveDirtMC, SmoothUV, and vInverse. You will see these plugins mentioned in many restoration threads and posts, one of which is here: http://www.digitalfaq.com/forum/vide...html#post51575.

DeBlock_QED is designed to smooth macroblocks without destroying too much detail. It is attached here as DeBlock_QED.zip. DeBlock_QED requires three support plugins, all of which can be used as standalone filters in their own right:
- DCTFilter_0150 (attached as DCT_Filter_0150.zip)
- DeBlock_13 (attached as DeBlock_13.zip)
- MaskTools2 (version 2.2.1x supplied with the QTGMC plugin package, listed above)

Cnr2 is an old standby usedc to help clean chroma noise and ugly color blotches. It is attached here as Cnr2_v261.zip.

nnedi3_rpow2 is a resizing function in the nnedi3.dll plugin (included inn the QTGMC plugin package, listed above).

LimitedSharpenFaster.avs is contained in LimitedSharpenFaster.zip.

Many Avisynth plugins require the Microsoft Visual C++ 2015 runtime Redistributable Package (vcredist_x86.exe). Microsoft's download link is http://www.microsoft.com/en-us/downl....aspx?id=52982 . You might already have this installed in your Win10 setup, but check Microsoft Visual C++ installs in your Control Panel. You'll need it sooner or later.

After you install these filters you'll have a handy store of very popular Avisynth plugins that are in use everywhere. If any problems, post questions.

[sanlyn]

Quoting myself here:

Quote:

Originally Posted by sanlyn

Many Avisynth plugins require the Microsoft Visual C++ 2015 runtime Redistributable Package (vcredist_x86.exe). Microsoft's download link is http://www.microsoft.com/en-us/downl....aspx?id=52982 . You might already have this installed in your Win10 setup, but check Microsoft Visual C++ installs in your Control Panel. You'll need it sooner or later.

Microsoft rears its ugly head again! And they used to be so friendly and accommodating just a few years back. So here's a new link for the Visual C++ Redistributable for Visual Studio 2015: https://www.microsoft.com/en-us/down....aspx?id=48145.

[naripeddi]

Wow sanlyn, that's so generous of you to provide all those plugins together. I will create my setup with them and attempt the scripts. Point noted on 32-bit.

I had to create RAR file for the audio since the forum did not allow upload of file type .mp2. This was the same reason why I had to create RAR file for the video as well, the forum did not allow files of extension .m1v (only m2v was allowed). Again, I will look at why the audio archive was invalid.

Will be a day or two.

[admin]

Quote:

Originally Posted by naripeddi

the forum did not allow upload of file type .mp2.
the forum did not allow files of extension .m1v (only m2v was allowed)

.mp2 has been allowed for at least 10 years now.
.m1v has just been added.

99mb limit on both.

[naripeddi]

Thank you admin for adding support for .m1v.

Sorry for my oversight on .mp2.

[lordsmurf]

Within sensible limits, we'll add whatever is needed to make this site work better.
m1v was a good addition.