digitalFAQ.com Forum - First captures: Understanding histograms, filters, cropping?

Thanks for your capture sample. As far as I can tell, it's a decent capture of a really bad original recording. Let me ask, did you apply filters during this capture? It looks heavily over sharpened, and the noise especially with bright objects is severe. If your Panasonic is set to increase sharpening, you should restore that sharpen slider to the middle notched position (disabled). Killing the sharpened noise in post-processing destroyed a lot of fine detail.

Quote:

Originally Posted by billct97 (Post 53184)

I have finally got a reliable workflow with the addition of the AG-1980, TBC-3000, ATI USB 600 and VirtualDub 1.10.4.

I would have used v1.9.11 for capture. It's less buggy. Whatever....

Quote:

Originally Posted by billct97 (Post 53184)

I want to make use of filters as needed and so am trying to pick the right ones and set realistic settings. So far using VDub as I figure I should understand that before I attempt avisynth.

For the cleanup to get the sample mpg attached to this post, the only VDub filters I could use were color filters (Color Mill and gradation curves). There was a slight initial levels correction required in Avisynth in the original YUY2 colorspace. Otherwise I had to use Avisynth filters for cleanup. For example, there is no effective VirtualDub filter that addresses DCT ringing effects, sharpening, or aliasing. One speciaized tweak that I applied was a mild NeatVideo VirtualDub plugin.

Quote:

Originally Posted by billct97 (Post 53184)

I'm working on a quality Windows 10 laptop (S-Video into the ATI 600, screen is not calibrated...yet) and also viewing the VCR output on a Sony TV with composite video and audio inputs for comparison. Both look relatively close but overall it seems to me that the color saturation could be improved.

Working with uncalibrated monitors and TVs is a quick trip to the looney bin.

Quote:

Originally Posted by billct97 (Post 53184)

It seems to me that the first place to start would be a reliable histogram. I was able to get that working in VDub and also captured this short video. I cropped the edges by 20 on all sides for the histogram but removed the crop for the video capture. The short video has 3 sections, the space scene, the scene with the bright lights and the scene where they are sitting. Hopefully this is a good sample. I am confused by the histograms and instead of wasting your time explaining what I think is happening can someone kindly tell me what they think of the video and what the histograms are telling me.

First of all, you should be looking at histograms in the original colorspace first, not in RGB. Many levels problems and (and should) be solved in YUV first. In YUV the 16 to 235 luminance range is expanded in RGB to 0-255. In RGB you can't any darker than zero or any brighter than 255. If you are in YUV and are already darker than 15 or brighten than 255, what do you think will happen when that overrun is expanded in RGB? Answer: the out of bounds values will be clipped (destroyed).

You likely will never see values darker than y=16 using the ATI 600. It internally clips values below 16. If you look at the true black and darkest areas of your capture, you'll see that at a certain dark point there is no more detail, just a black mass. The commander's uniform and the woman's eyes are examples of dark areas in which detail has been clipped and can't be recovered. Avisynth does have a few filters that can retrieve detail from dark or bright areas where there is slight overrun into the non-safe areas during capture.

In your sample a YUV histogram and detailed analysis of the unaltered capture looks like this:

http://www.digitalfaq.com/forum/atta...1&d=1520229072

In the histogram, the top band (white) is the luma (Y) channel. At the left-hand border (darks, where "black" is y = y=16), you;'ll see a sharp "spike" right at the y=16 border and nothing beyond it, no rolloff or extension at all, just a sharp cutoff. This indicates clipping with the ATI 600. If you've made your darkest values bright enough, it's not a problem. Note that if you had included your original black borders vwhen reading the histogram, they would be y=zero originally but the ATI 600 would change that superblack to y=16. In the image above, you;'ll note that I cropped off the side borders and head switching noise before making then histogram and taking then Analyze readings.

In the luma and chroma analysis numbers you"ll see a column labeled "Luma Y". Under that heading are the luma values for this scene. The "minimum" luma value is 16 (with the AI 600, it will never be lower than 16). The "maximum" luma value does indeed exceed 235 (it's actually 253). By viewing this scene in VirtualDub using a pixel reader on this scene, it appears that the brightest values in the non-safe zone beyond 235 are in the blue tail. There is also some visible TV transmission noise along the top border.

The luminance band is concentrated mostly at the dark left side because, of course,this is a very dark scene with very little midrange or bright value.

Here is a YUV histogram and analysis readout for the "lights" scene (frame 209 in the original):

http://www.digitalfaq.com/forum/atta...1&d=1520229072

The histogram in this scene has the expected dark-end cutoff at y-16. Although the scene appears dim (there is very little output in the midrange or bright portion of the white band). there are specular highlights somewhere in this image that exceed y-235. When expanded in RGB, the dark cutoff will display as RGB=0 and the bright highlights will appear as 255 in either luma or chroma that starts climbing up the right-hand side of the RGB histogram, even though there is not much output in that bright region.

I worked on this sample with some strong denoisers. To use them I had to return the video to its original state, which was progressive film-based media at 23.976 fps and telecined with 3:2 pulldown for 23.976 fps playback. It was broadcast and captured as interlaced, so the telecine frames were hard-coded (embedded) in the video and had to be removed with an Avisynth inverse telecine filter (TIVTC.dll). Avisynth's Level() filter was used to correct the slight y=255 overflow, and VDub color filters expanded the midrange, the brights somewhat, and saturation. The results were encoded with the HCenc MPEG encoder to get the DVD-compliant MPG sample attached.

Quote:

Originally Posted by billct97 (Post 53184)

If filtering would help I've read that ColorMill is well regarded. Is there any way to adjust levels while watching the results? If so I haven't been able to figure out how to do this. I'm actually pretty frustrated with having to have Preview and Preview Acceleration enabled for seeing the effects of the filters and then having to go back to Overlay to see video when filters are disabled. Does avisynth make this any easier? Should I just give in now and go with something simpler like avidemux (it looks pretty).

Sorry, I don't understand this paragraph. What does Preview and Overlay have to do with VirtualDub filters during capture? Are you using VDub filters during capture? If so, yikes! Don' t ever do that. The proc amp controls are all you need to set during capture.

There's a good free histogram and levels tutorial at Cambridge Photo. It's for still cameras, but the principles for video are exactly the same. The article uses parade-style histograms like those in Photoshop, Premiere, Vegas and VDub's ColorTools.
Understanding histograms Part 1 and Part 2
http://www.cambridgeincolour.com/tut...istograms1.htm
http://www.cambridgeincolour.com/tut...istograms2.htm

There's also a handy video that demonstrates the principles: https://www.youtube.com/watch?v=htqrTTSZp-M.

Quote:

Originally Posted by billct97 (Post 53184)

And final question for the day (I've been at this for over 6 hours so forgive me if I'm not making complete sense). I understand why cropping is bad - you lose the 720x480 format - and I could swear I had read that you should matte instead. I can't figure out how to best accomplish that.

It's easier in Avisynth, and you don't lose any of the image. The way I cropped the mpg is shown in the explanation below of the Avisynth script I used:

I worked on this sample with some strong denoisers. To use them I hadtoreturn the video to its original state,which was progressive film-based media at 23.976 fps and telecined with 3:2 pulldown for 23.976 fps playback. It was broadcast and captured as interlaced, so the telecine frames were hard-coded (embedded) in the video and had to be removed with an Avisynth inverse telecine filter (TIVTC.dll). Avisynth's Level() filter was used to correct the slight y=255 overflow, and VDub color filters expanded the midrange, the brights somewhat, and saturation. The results were encoded with the HCenc MPEG encoder to get the DVD-compliant MPG sample attached.

Quote:

Originally Posted by billct97 (Post 53184)

Quote:

Originally Posted by billct97 (Post 53184)

It's easier in Avisynth, and you don't lose any of the image. The way I cropped the mpg is shown in the explanation below of the Avisynth script I used:

Code:

Import("D:\Avisynth 2.6\plugins\Santiag_v16.avs")

Import("D:\Avisynth 2.6\plugins\RemoveDirtMC.avs")

Import("D:\Avisynth 2.6\plugins\TemporalDeGrain.avs")



AviSource(E:\forum\faq\billct97\trek3-short.avi")

Levels(16,1.0,255,16,235,dither=true,coring=false)

AssumeTFF()

ConvertToYV12(interlaced=true)

HqDering()

TFM().TDecimate()

MergeChroma(aWarpSharp2(depth=20).aWarpSharp2(depth=10))

Santiag(2,2)

DeHalo_Alpha()

TemporalDegrain(SAD1=200, SAD2=100, Sigma=6)

RemoveDirtMC(40,false)

GradFun2Dbmod(thr=1.8,mask=false)

grainfactory3(g1str=1, g2str=1, g3str=1)

AddGrainC(2.5,2.5)

Crop(10,4,-10,-2).AddBorders(2,2,2,4)

ConvertToRGB32(interlaced=false)

Return last

Looking at it line by line:

Import("D:\Avisynth 2.6\plugins\Santiag_v16.avs")
Import("D:\Avisynth 2.6\plugins\RemoveDirtMC.avs")
Import("D:\Avisynth 2.6\plugins\TemporalDeGrain.avs")
These three lines import the coded lines of an Avisynth filter that is in script form instead of a compiled .dll. This saves me thr trouble of typing dozens of lines of lines of code. The filters are maintained as scripts because they exist in several versions,and each version has to be imported explicitly for specific purposes.

AviSource(E:\forum\faq\billct97\trek3-short.avi")
This statement opens and decodes your AVI sample into uncomprfessed video. It also decodes the audio. you want to process a video then, of course,you have to open it first. The name of the built-in function that opens this type of file is AviSource(). The path to the video goes inside the parenethesesd in double quotes.

Levels(16,1.0,255,16,235,dither=true,coring=false) .
the Levels() function manipulates darks, brights, and gamma according to values set in its parameters. The numbered parameters indicate, in the order shown, the following values: value of darks coming in, value of desired gamma (midrange) adjustment, value of brightest elements coming in, darkest value of darks to be output, brightest value of brights to be output. notice that for incoming values of y=255, the function willperforma gradual contraction of 255 values down to 235 to avoid clipping effects. Output will be dithered (transitions between colors and shades in large areas will be smoothed, such as in the sky). Also there should be no coring of output (another way of saying, don't clip).

ConvertToYV12(interlaced=true) performs a high-precision conversion from YUY2 to YV12. Most of the filters to be used after this conversion will work only in YV12. Note that the video's interlaced condition must be stated for proper conversion -- something that many programs don't bother with, with clumsy results and many interpolation errors.

HqDering() is Avisynth add-in that cleans DCT ringing artifacts (edge ghosting and halos),which is a frequemnt villain in tape playback. This filter is usually loaded before deinterlacing or inverse telecine.

TFM().TDecimate()
These two functions are partners in the inverse telecine process.This removes the 2-in5 interlaced frames of 2:3 pulldown and the dup0licate images that are involved. There are two functions separated by a period in the text to effectively form two different statements. These functions are members of the TIVTC Avisynth plugin. After inverse telecine, the video will be pure progressive running at the original films peed of 23.976 fps. When processing is complete, pulldown will be restored during the final encode step.

MergeChroma(aWarpSharp2(depth=20).aWarpSharp2(dept h=10))
This statement runs a sharpener called aWarpSharp2 that tightens colors along edges to prevent chroma shift and bleed effects. MergeChroma effectively means that the filter sharpens chroma only, not luma, to avoid edge halo effects. In the case the sharpener is run twice in succession because there is visible chroma edge overrun, especially with red. The "depth" value is a strength setting.

Santiag(2,2)
DeHalo_Alpha()
Santiag() is an anti-alias filter. Edge aliasing is most noticeable inn the overhead lamps in the lights scene. DeHalo_ALpha helps clear up bright oversharpening edge halos.

TemporalDegrain(SAD1=200, SAD2=100, Sigma=6)
This is a strong degraining filter. It makes at least a nominal effort to preserve some of the fine detail that exists under the grain. It's being used here with light to medium values to avoid too much damage. It can result in clay-face and posterizing effects, which is always a problem when cleaning really noisy video.

RemoveDirtMC(40,false) is another filter aimed more at removing general grunge, tape noise, and blemishes. It's also good at removing many spots and dropouts, but it can also remove fine lines and similar objects. The value 40 is a strength setting and the "false" value means that this video isn't grayscale.

GradFun2Dbmod(thr=1.8,mask=false) is a smoother that helps smooth and mask hard contour edges in gradients, such as walls and the sofa cushions, and helpsm avoid hard facial contours after all the previous denosing.

grainfactory3(g1str=1, g2str=1, g3str=1)
AddGrainC(2.5,2.5)
Removing grain removes a lot of noise butgives a plastic look. The look of film can be restored and many plastic areas can look more detailed by adding very fine film-like grain. grainfactory() is a little more course than AddGrainC().

Crop(10,4,-10,-2).AddBorders(2,2,2,4)
This statement is two commands in one line.Crop() removes pixels,in this case grungy or unwanted border pixels that have been changed or discolored by processing. This video sample has eight black pixels in the borders at each side, plus 4 pixels of transmission noise along the top and 2 pixels of discoloration at the bottom. However, each side border also has a noisy whitish area running along the interior side of both edges. So the Crop() function removes 10 pixels from the left side, 4 pixels from the top, 10 pixels from the right side, and 2 pixels from the bottom. AddBorders() will then replace the removed pixels with brand new black ones (or the color of your choice. Black is the default. Notice that when the new borders are added, the frame size is now 704x480, which is valid for DVD and for 4:3 BluRay

If you wanted 720x480 for 16Z:9 BluRay (required) or jus wanted to restore the original frame size, you would use AddBorders(10,2,10,4) -- in this case 4 pixels were originally removed from the top and 2 from the bottom, for a total reduction of 6 pixels of vertical height. Because of the colorspace used we can't restore odd numbers of pixels, so the coded restores 4 pixels to the bottom and 2 to the top because thicker bottom borders are usually prettier.

ConvertToRGB32(interlaced=false,matrix="rec601") is a high-precision conversion from the current colorspace to RGB32, which will be used for the VirtualDuib filters. Rec601 is the color matrix for standard definition video.

Return last outputs the results of the last statement in the sequential list of instructions. This statement can be inserted at any line to stop the program where desired and return the results up to that point.

The VirtualDub filters used were ColorMill and gradation curves. Their settings are contained in a text file called a .vcf file. You can load a .vcf file in VirtualDub and it will automatically load the filters involved with their proper settings. It won't work if you don't have ColorMill and gradation curves in your VDub plugins. One other filter used was a tweak with NeatVideo, but I have a different version than the one that comes with the forum's VirtuaklDub 1.9.11 download. I used the ColorMill histogram to check luma and chroma ranges between filter settings.

The attached mpg has 3:2 pulldown (telecine) restored during the encode. It is DVD compliant and plays at 29.97 fps.