The sample avi's physical frames have these border characteristics: 10 pixels of left border, a 1-pixel broken border across the top (the top border pixels extend only 65% across the length of the top border), 12 pixels of right border, and 8 pixels of bottom-border head switching noise. For purposes of analyzing the core image content , border pixels were temporarily removed so that zero-black pixels would not affect histograms. The Avisynth command for removing these border pixels was Crop(10,2,-12,-10). Notice that 10 pixels are temporarily removed from the bottom rather than 8, because the analysis tools require a mod4 height.
In the final output workfile, the borders are removed with Crop(10,2,-12,-8). When the frames were reassembled later, new black pixels were added and the image content was more vertically centered using the command AddBorders(10,4,12,6). YUY2 requires pixels in groups of Mod2. Odd-numbered pixel groups cannot be used. This and other limitations are due to the way luma and chroma information is stored in pixels. Some highly regarded NLE's ignore these rules, often with oddball results that belie the cost of the software.
One of my first steps in checking captures is to open an Avi in VirtualDub
directly or by using a simple Avisynth script that allows the use of fairly direct and simple tools. The first direct and simple set of tools is eyeballs, but they're enhanced with more objectivity and precision by the tools shown below.
Above, the YUV levels histogram is an Avisynth builtin feature that displays horizontal bands describing specific luminance and chroma channels. Luminance (the Y channel) is shown in the white band across the top of the graph. The blue-yellow chroma (U channel) is the middle band, and red-green chroma (V channel) is the lower band. Darker (lower) values are on the left, brighter (higher) values are on the right. The thin vertical line down the center represents the middle of the spectrum between dark and bright.
The two shaded vertical borders along the left and right represent pixel values that lie outside the legal video range of y=16-235. Because YUV 16-235 is expanded in RGB to 0-255, elements inside the shaded borders would be values that lie outside the range of RGB=0 to 255 and thus would display incorrectly or would be clipped (destroyed). In the far upper right-hand corner of the histogram shown above, the bright yellow "peak" in the unsafe right-hand border would expand to values greater than RGB 255, which could not be accepted by most displays and would normally be clipped (rejected) by broadcast equipment. Besides, unsafe video values don't look so great and often have a freaky appearance. Plenty of PoohTube examples abound.
The "parade" formatted histogram above is a popular means of displaying RGB values for average sum brightness (the white band), Red chroma (the red band), Green chroma (the green band), and Blue chroma (the blue band). Dark colors are at the left, bright colors at the right. The height of the peaks in the bands indicate intensity and/or the number of pixels in different segments of the spectrum. The RGB histogram above shows somewhat elevated black levels, since the horizontal bands don't extend to the left (dark) border wall. On the right border wall, the amount of Green and Blue slightly surpasses RGB 255 and exceeds the ability of most RGB devices to display properly. The positions of the bands also show a deficit of Red and a predominance of Green and Blue.
Vectorscopes exist in both RGB (above) and YUV color. The center of the circle represents lower values of the colors shown, while higher values extend to the outer perimeter. Center values indicate lower saturation (grays), outer values indicate greater saturation. The letters along the outer edge of the circle indicate regions of primary and secondary colors: M=Magenta, B=Blue, C=Cyan, G=Green, Y=Yellow, and R=Red. The extended off-center line indicates the usual area for skin tones. Color pixels that extend beyond the small rectangles along the outer perimeter would be oversaturated values that distort and clip in RGB displays. The vectorscope above shows a dominance of magenta and a stronger dominance of bluish Cyan. There is a deficit of Red, Yellow, and pure Green. In the images generated by this vectorscope there would be no clean whites, grays, or blacks. The overall imbalance would be heavily tinted toward Cyan.
Using Avisynth and VirtualDub
, I prepared a demo video that shows your sample video (with black borders removed) and simultaneously shows all three histograms during play. They show the color imbalance as well as the way the camera's AGC (Auto Gain) and autowhite act more like defects than "features", seldom doing what they're supposed to do. The AGC does some visible gamma "pumping" at frames 9 to 32, changes three times again shortly thereafter, and undergoes more gamma pumping and shifting at frames 406, 416, 435, and 488. There's really not much you can do about this except live with it. We've seen much worse. In more disruptive scenarios it's sometimes possible to smooth matters to a slight degree using one of several Avisynth configurable autobalance filters, but this involves a great deal of additional manipulation -- and it still doesn't totally solve the problem. You Often need the help of something like AfterEffects to address it.
Below is a reduced-size frame from the histogram video:
The original frames are unaltered except for removing the side borders. At this stage I ignored the right-hand magenta stain. Note how blue the image looks, with no clean whites or grays. The street surface and sunlit sand are all blue, ground surfaces are grayed-out tan, foliage has suppressed green, the surf's foam in the distance don't look white, and the overall scene looks dim rather than lighted by a bright midday sun. Shadow details of people under the trees looks murky. Note that the excessive camera jerking is mercilessly demanding and wasteful of encoding bitrate. It takes higher than average bitrate to properly describe hectic motion. Deinterlacers in software or hardware have to be on their toes to handle such scenes properly, whether it plays as interlaced or progressive. In this case the camera at times recorded a little motion smear and ghosting or shimmer. Or the subtle effect could be coming from the player.
The demo video is attached as Histogram-demo_25i.mp4
Play it several times and note how the histograms changes fitfully with the camera's AGC, often going into unsafe territory in the two middle histograms. It's normal for histograms to change as scene elements change, but not to this degree if a constant manual exposure had been used. Color balance also changes, especially in the beach and surf elements. The color of the foreground roadway changes several times.
Histograms don't reveal much about noise, except some types of chroma noise. I noticed some tape noise in darker areas of shadows and trees, but most bright daylight scenes don't display as much typical noisy junk as do darker videos. I also note that interlace activity looks smooth here with less buzzy edges, aliasing, and/or excess combing than seen with videos from most consumer cameras.
Below, I've selected three frames to illustrate the changes that occurred from the original starting point to the final output mp4's. Each frame is resized to 4:3 proportions. The original frame is on the left, the frame from the 25i mp4 is on the right. From top to bottom the frame numbers are 86, 305, and 493. By comparing frames in this manner you can also see how the camera's autowhite changes colors in many elements from beginning to end.
I prepared three versions of the final output, two of which are presented here.
Each version is characterized by a certain "look" that comes from the camera, which attempted to compress extremely high contrast into the limits of the recording cells and media. The results are a few burned highlights (same of which can be rescued) and a loss of shadow detail in many areas. Most of the people are silhouettes. But within the limits of the gear and media, it's a decent recording and capture. It simply doesn't have the latitude or dynamic range of film. In processing I used contrast masking filters to bring out shadow detail.
The first output version attempted to correct the rightmost border stain using conventional mask and overlay methods. Those methods work often, but sometimes not so well. The problem in this sample is that the magenta stain isn't a tint; it actually replaces or destroys original colors. Anti-rainbow filters can work with tints, but they can't restore colors that no longer exist. The result is that a lot of experimentation and tweaking is involved, and many of the corrections have different results against different backgrounds. I find it barely acceptable and a headache to work with but you might have a better opinion. The pile of filters used slowed the processing to a measly 2.5 fps. This version of the video runs at 25i interlaced and is attached as Out_25i.mp4
The second version took a more consistent approach, even it does cost a small bit of screen real estate. 26 pixels are removed from the right border, eliminating the stain and about 16 pixels of actual image. Then, 2 pixels from the top and 8 pixels from the bottom are removed, none of which is actual image content. This allows resizing the image to fill the frame vertically. But can one extend the image horizontally to fill the missing right border? Sorry, but definitely not. The result would be a horizontally distorted aspect ratio. Mom and Aunt Margaret would gain 30 pounds each and look weird. Doorknobs, wheels, clocks, O's and other rounds would morph into ovals, and squares would be rectangles, all of which looks like foolish mistakes. I did cheat a bit by extending the image horizontally an extra 4 pixels; that's easy enough to get away with. But going wider simply would not work. Using this method, I produced a 25i interlaced video that looks pretty much like the first version above, but without the right-border stain. The left border stain is mild and isn't worth the bother. It would be hidden by overscan on most TV's (yes, folks, today's HDTV's do use overscan, and on many it can't be disabled). Fewer filters were used here, so processing was a decent 5 to 6 fps.
The third version is the same as the above but is 50p progressive. It's not quite ready for web mounting -- you'd have to resize the 50p to square-pixel 4:3, something like 640x480. Processing ran at 8 to 9 fps. The 50p version is attached as Out_50p.mp4
The denoisers used are Avisynth plugins, the deinterlacer is QTGMC, the resizer was Spline36resize. Color filters initially used Avisynth functions and the SmoothAdjust plugin in YUV, but these were very basic corrections that were later tweaked in RGB using VirtualDub plugins that mimic expensive NLE features. RGB correction required different color adjustments for darks, midtones, and brights. One advantage of color correction is that it can mask a multitude of sins.
I can quote all of the Avisynth and VirtualDub details if you wish, but I'll caution that if you've never used these tools you have a learning curve ahead -- not that it's anything like rocket science (if a klutz like myself can do it, any other monkey can do it even better), but it won't happen in an hour. This forum's restoration section is filled with hundreds of project examples if you want to first take a browse.
I'll also caution that if you think you can get the same deinterlace and denoise results with NLE's, you won't. Big name NLE's are editors and encoders and are great as such, but they're not restoration and repair apps. They can be upgraded with plugins if you have a few tens of thousands of euros to spare and 6 months to get the hang of using them. Avisynth is free and you can get there in a weekend.
Originally Posted by MindrustUK
Originally Posted by sanlyn
One thing one can tell from the photos is that there is very poor or nonexistent input level control, and that there are illegal or unsuitabe luminance levels resulting in unrecoverable blown out highlight detail in some pics, and unrecoverable clipped blacks in others, and some rather grim looking dark chroma density in one of the others.
I'll do some reading and try and understand what that all means, I'm guessing these are hardware limitations down to the combination of card and player? What would be the remedy, things that can be fixed in post with software or additional hardware signal filtering before capture?
Input signal levels are not often controlled by players, but most often by the capture software or with the use of an external proc amp. Good proc asmps ain't cheap. My favorite is the SignVideo PA-100
. It's no longer made but you can find info in forum posts at digitalfaq. Most hobbyists and many pros use VirtualDub for lossless capture, which "hooks in" to the capture device's drivers and software proc amp. VDub also has a built-n capture histogram to measure what you think your eyes are seeing. The only required adjustments are brightness and contrast. But many like to torture themselves trying to get "perfect" color during VHS capture, which is simply masochistic. Using VDub's capture histogram is discussed in post #3 in Capturing with VirtualDub [Settings Guide]