Levels OK, but RGB illegal? Need education...

[ragu0012]

OK I am getting a pretty good hang of making sure the levels are capturing without clipping, then slightly tweaking with "levels" in AVIsynth to make sure they are between 16 and 235.

However, when I look at colortools this is what I see.

colortools.png

Does this mean the colors are illegal? What is the best way to fix this if it needs fixing, and what could I have done to prevent it?

Here is the corresponding frame:

You must be logged in to view this content; either login or register for the forum. The attached screen shots, before/after images, photos and graphics are created/posted for the benefit of site members. And you are invited to join our digital media community.

And here is the script I used:

#####-----LOAD A FILE
AviSource("H:\Lou VHS project\Raw tapes\Mary Poppins.avi")
AssumeTFF()

### / YUY2 -----------------------------
ConvertToYV12(interlaced=true) # script below this line requires YV12

### ------Deinterlacers
QTGMC(Preset="Very Fast")

Levels(16, 1.1, 255, 0, 235,dither=true,coring=false)

### / YV12 ----------------------------------------------------------------------------------------

### Final resizers for destination format
Crop(18,0,-14,-8)#.AddBorders(16,4,16,4) 
Spline36Resize(640,height) 

# ---- Convert 480i to RGB for VirtualDub filters ----- # 
ConvertToRGB32(interlaced=false,matrix="Rec601") 

return last

[juhok]

Your video is not RGB. Check YUV with "Color" or "Color2".

http://avisynth.nl/index.php/Histogram#Color_mode

Quote:

The U component is displayed on the horizontal (X) axis, with the leftmost side being U=0 and the rightmost side being U=255. The V component is displayed on the vertical (Y) axis, with the top representing V=0 and the bottom representing V=255. The white square denotes the valid CCIR-601 range.

[hodgey]

Anything from 0-255 is valid in RGB.

[sanlyn]

That RGB histogram has luma and chroma values between 0 and 255, which is valid for RGB. However, the black levels and brightness are a bit high. Darkest luminance values that would naturally be black from about RGB 8 to 32 are 16 to 50 in the histogram, and your image looks just slightly washed out at the low end but the darkest blacks looks rather dense at the same time. Most of the darkest luminance values don't go below RGB 50 or so, while chroma values go darker (not unusual).

The RGB histogram doesn't show quite the same thing your code is doing.

Levels(16, 1.1, 255, 0, 235,dither=true,coring=false)

The first number in the above sequence defines the input luma value, which is 16. The 4th number in that sequence defines where you want the value of 16 to be upon output, which here is 0. This changes all dark values in the area of about y=16 to y=0, which is outside the range of 16-235 and will crush the darkest details in RGB, which can't expand y=0 below the value of y=zero. But that's not what your RGB histogram shows -- rather, it shows slightly elevated dark luma values and crushed blacks at the same time. So you're doing something in RGB you haven't told us about.

The second number in that series defines the gamma (midtone) value, which here raises the midtones. I wouldn't think that's a good choice with elevated blacks in the standard interior scene shown, so I really can't tell what's going on. The RGB histogram describes your image, but the code for YUV levels describes something different.

That image suggests to me that your video is film-based and telecined, not interlaced. But I might be wrong about that, hard to tell from a single image. As`far as color balance goes it's fairly close to natural but whites are too cyan (blue+green).

[juhok]

Indeed. I didn't realize this was VDub meters after RGB conversion.

[sanlyn]

VirtualDub works in RGB only.

Any histogram that displays Red, Green, and Blue channels is working in RGB. YUV color doesn't have R, G, or B channels. YUV has Y, U , and V only. Y contains luminance only and displays like a black and white movie, with no color. the U channel is called "blue" for short but contains part of the cyan (blue+green) spectrum. the V or "Red" channel contains red and part of the yellow (red+green) spectrum.

RGB does not store a separate luminance channel. RGB Luminance values are stored in the same pixel as chroma values. When you see a white "luminance" or "brightness" band in RGB histograms, brightness is derived as the average brightness contained in the chroma values. Adding or reducing the amount of any RGB color increases or decreases overall brightness in the RGB image. In YUV, adding or reducing the U or V chroma channel has no effect on YUV luma values, while adjusting luma doesn't increase or reduce YUV chroma (however, because you are almost always viewing YUV "results" in RGB, the effects you see reflect the way RGB values are being changed after translation from YUV). The human eye can't "see" YUV. Eyeballs are analog RGB only. They're not digital and can't do a damn thing with 0's and 1's.

[ragu0012]

OK I understand most of what you're saying. Now let me throw you one more example on a mundane clip. Levels seem to be OK (less than perfect, but legal at least) using Histogram in Avisynth. Then when I look at colortools in RGB, my colors look to be in illegal ranges? Please let me know one last time what I should do differently here.

Here is what the frame looks like:test03.png

I adjusted levels with the script below. And here is what the Histogram in Avisynth says

You must be logged in to view this content; either login or register for the forum. The attached screen shots, before/after images, photos and graphics are created/posted for the benefit of site members. And you are invited to join our digital media community.

Back in Virtualdub, I check the RGB with colortools and this is what i see: test01.jpg

Here is the script:

#####-----LOAD A FILE
AviSource("H:\Lou VHS project\Raw tapes\Christmas 1990.avi")
AssumeTFF()

### / YUY2 -----------------------------
ConvertToYV12(interlaced=true) # script below this line requires YV12

### ------Deinterlacers
QTGMC(Preset="Very Fast")


### YV12 Color correction
ColorYUV(autogain=false, autowhite=true)

Levels(-10, 0.9, 255, 0, 240,dither=true,coring=false)

### / YV12 ----------------------------------------------------------------------------------------

### Final resizers for destination format
Crop(18,0,-14,-8)#.AddBorders(16,4,16,4) 
Spline36Resize(640,height) 


######------TEST LEVELS
ConvertToYV12(interlaced=false)
Histogram("levels")

# ---------------- reinterlace for DVD ---------------- # 
#SeparateFields().SelectEvery(4,0,3).Weave() 

# ---- Convert 480i to RGB for VirtualDub filters ----- # 
#ConvertToRGB32(interlaced=true,matrix="Rec601") 
ConvertToRGB32(interlaced=false,matrix="Rec601") 

return last

[juhok]

In VirtualDub you're working with RGB.

RGB is 0-255 so your levels are "legal".

[ragu0012]

OooooK got it

[sanlyn]

Quote:

Originally Posted by ragu0012

OooooK got it

You do understand, i hope -- after having it repeated many times in many posts -- that y=16-235 as stored in YUV is expanded to 0-255 in RGB for display. The same goes for chroma values, whose allowed YUV range is 16-240 (which means that it is possible to have some slight overflow in chroma values, although it doesn't always look so great in display when values exceed chroma=235.

So, you see, if YUV has values darker than y=16 and/or brights greater than y=235, an RGB medium has no way to expand those values beyond 0-255. So it is that beyond-spec values in YUV will get clipped in RGB.

[ragu0012]

Ha, yes. Sometimes it takes a sledgehammer to make it sink in but I got it and thank you all very much for your help.

Quote:

Originally Posted by sanlyn

You do understand, i hope -- after having it repeated many times in many posts -- that y=16-235 as stored in YUV is expanded to 0-255 in RGB for display. The same goes for chroma values, whose allowed YUV range is 16-240 (which means that it is possible to have some slight overflow in chroma values, although it doesn't always look so great in display when values exceed chroma=235.

So, you see, if YUV has values darker than y=16 and/or brights greater than y=235, an RGB medium has no way to expand those values beyond 0-255. So it is that beyond-spec values in YUV will get clipped in RGB.