digitalFAQ.com Forums [Archives] - SansGrip Filters: LegalClip() is not clipping?

digitalFAQ.com Forums [Archives] (http://www.digitalfaq.com/archives/)

- Avisynth Scripting (http://www.digitalfaq.com/archives/avisynth/)

- - SansGrip Filters: LegalClip() is not clipping? (http://www.digitalfaq.com/archives/avisynth/2949-sansgrip-filters-legalclip.html)

SansGrip Filters: LegalClip() is not clipping?

:!: In making a number of scripts for a number of AviSynth versions, I just cannot seem to get LegalClip() to work. That is, after I apply it, I still get blacks at $000000 and whites at $ffffff.

:arrow: I made a test script for AviSynth v.2.0.7 that should, when run, produce a checkered pattern due to selective application and non-application of the LegalClip filter:

Code:

clip1 = BlankClip(width=150,height=100,color=$000000)

clip2 = BlankClip(width=150,height=100,color=$000000)

clip3 = BlankClip(width=150,height=100,color=$ffffff)

clip4 = BlankClip(width=150,height=100,color=$ffffff)



clip1 = ConvertToYUY2(clip1)

clip2 = ConvertToYUY2(clip2)

clip3 = ConvertToYUY2(clip3)

clip4 = ConvertToYUY2(clip4)



clip1 = Subtitle(clip1,"black")



clip2 = Subtitle(clip2,"black-legalclip")

clip2 = LegalClip(clip2)



clip3 = Subtitle(clip3,"white")



clip4 = Subtitle(clip4,"white-legalclip")

clip4 = LegalClip(clip4)



clips12 = StackHorizontal(clip1,clip2)

clips21 = StackHorizontal(clip2,clip1)

clips34 = StackHorizontal(clip3,clip4)

clips43 = StackHorizontal(clip4,clip3)

StackVertical(clips12,clips21,clips34,clips43)

:!: My end result is solid black and solid white (not counting the text, of course). As I understand how LegalClip() is supposed to work, I should be getting a checkered pattern of black $000000 and clipped-black $101010, and white $ffffff and clipped-white $f0f0f0. (Of course, this test reflects my real-world video processing -- that is, no apparent clipping.)

:?: Can anyone help with this?

Hi easymenu,

You can make an easier test. Encode one minute clip of some material with LegalClip(), and encode the same one minute without LegalClip(). The one minute clip without LegalClip() should be larger. At least for me, as I did try that about a month ago :roll:

Edited

-kwag

:) Thanks for the suggestion.

:arrow: Okay, so I made two very simple AVS 2.0.7 scripts for a short MPEG2 352x480 clip (BTW, this testing is with a WinNT4 system, which uses DirectX3 generally and DirectX6 for graphics -- so no problems with the infamous DirectX8.1) :

Code:

MPEG2Source("video.d2v")

and

Code:

MPEG2Source("video.d2v")

LegalClip()

I just did plain processing in TMPGEnc 2.5.10, except to use your KDVD-352 template. The resulting file sizes were different -- 3,833,105 w/o LegalClip() and 3,833,192 with LegalClip(). LegalClip() is doing something. Is it doing CCIR601 clipping though?

:arrow: So I made another test script. This one contained a number of test areas around the CCIR601 clipping points (both low and high). To prevent Windows display / monitor settings from corrupting a visual examination, I included Histogram() for graphical representation of the actual output:

Code:

# legalclip() in AVS 2.0.x

#-------------------------



clipblack0 = BlankClip(width=132,height=40,color=$000000)

clipblack0 = ConvertToYUY2(clipblack0)

clipblack0 = Subtitle(clipblack0,"$000000")

clipblack0 = Histogram(clipblack0)



clipblack1 = BlankClip(width=132,height=40,color=$000000)

clipblack1 = ConvertToYUY2(clipblack1)

clipblack1 = Subtitle(clipblack1,"$000000 clipped")

clipblack1 = LegalClip(clipblack1)

clipblack1 = Histogram(clipblack1)



clipblack2 = BlankClip(width=132,height=40,color=$0f0f0f)

clipblack2 = ConvertToYUY2(clipblack2)

clipblack2 = Subtitle(clipblack2,"$0f0f0f clipped")

clipblack2 = LegalClip(clipblack2)

clipblack2 = Histogram(clipblack2)



clipblack3 = BlankClip(width=132,height=40,color=$101010)

clipblack3 = ConvertToYUY2(clipblack3)

clipblack3 = Subtitle(clipblack3,"$101010 clipped")

clipblack3 = LegalClip(clipblack3)

clipblack3 = Histogram(clipblack3)



clipblack4 = BlankClip(width=132,height=40,color=$202020)

clipblack4 = ConvertToYUY2(clipblack4)

clipblack4 = Subtitle(clipblack4,"$202020 clipped")

clipblack4 = LegalClip(clipblack4)

clipblack4 = Histogram(clipblack4)





clipwhite0 = BlankClip(width=132,height=40,color=$ffffff)

clipwhite0 = ConvertToYUY2(clipwhite0)

clipwhite0 = Subtitle(clipwhite0,"$ffffff")

clipwhite0 = Histogram(clipwhite0)



clipwhite1 = BlankClip(width=132,height=40,color=$ffffff)

clipwhite1 = ConvertToYUY2(clipwhite1)

clipwhite1 = Subtitle(clipwhite1,"$ffffff clipped")

clipwhite1 = LegalClip(clipwhite1)

clipwhite1 = Histogram(clipwhite1)



clipwhite2 = BlankClip(width=132,height=40,color=$ececec)

clipwhite2 = ConvertToYUY2(clipwhite2)

clipwhite2 = Subtitle(clipwhite2,"$ececec clipped")

clipwhite2 = LegalClip(clipwhite2)

clipwhite2 = Histogram(clipwhite2)



clipwhite3 = BlankClip(width=132,height=40,color=$ebebeb)

clipwhite3 = ConvertToYUY2(clipwhite3)

clipwhite3 = Subtitle(clipwhite3,"$ebebeb clipped")

clipwhite3 = LegalClip(clipwhite3)

clipwhite3 = Histogram(clipwhite3)



clipwhite4 = BlankClip(width=132,height=40,color=$e0e0e0)

clipwhite4 = ConvertToYUY2(clipwhite4)

clipwhite4 = Subtitle(clipwhite4,"$e0e0e0 clipped")

clipwhite4 = LegalClip(clipwhite4)

clipwhite4 = Histogram(clipwhite4)





StackVertical(clipblack0, clipblack1, clipblack2, clipblack3, clipblack4, clipwhite0, clipwhite1, clipwhite2, clipwhite3, clipwhite4)

:!: The result? As Doctor McCoy would've put it, "Well, either our scouting probes and detectors are malfunctioning and all us scouts careless and beauty intoxicated, or I must report myself unfit for duty." As I understand CCIR601 clipping, no CCIR601 clipping happened!

:arrow: First, a couple of "granted's". ConvertToYUY2() must be used before using LegalClip(). Is there a problem in ConvertToYUY2()? Also, the test colors are set in RBG but the clipping is implemented in YUV. While they don't correspond exactly, there will still be obvious jumps of brightness on the processed, out-of-CCIR601-range colors.

:?: I looked quickly at the LegalClip() source code. In the C++ section, the code that sets up the "clipping table" looks correct. The actual pixel replacement code is in Assembly code (and that is not for casual reading). If there is any problem in LegalClip(), I would suspect it would be in the Assembly section.

As if that is not enough, here's something more.

:!: I similarly tested AVS 2.5b (with a Win98 system w/DirectX9) and it's built-in, more flexible, clipping function "Limiter()". Guess what? Same results (that is, no apparent clipping)!

:?: So am I a bug-finder-genius or just "unfit for CCIR601 duty"?

Let me ask you something easymenu, when you did your .d2v project, did you use "PC Scale" or "TV Scale" :idea: :idea:
Because if you used TV scale, DVD2AVI already produced a clipped file :idea:
So it shouldn't make any difference if you use LegalClip() or Limiter(), and that is what you are seeing :!:

-kwag

:!: I just checked the DVD2AVI 1.76 settings. I did have it set to "TV", not "PC". I'm rerunning the test to get new filesizes with the "PC" setting. It should be done by the time I'm finished writing this.

:arrow: Of course, this wouldn't explain my AVS-generated-video test results. If I have the CCIR601 theory correct, those generated blacks and whites should be changing (and they're not). Alternately, when I do limiting with TMPGEnc's filters (one for low cutoff and one for high cutoff), the proper result is instant and obvious. (Try it and let me know what you think.)

:!: While the file is generating in TMPGEnc, I'm sampling the output screen with EyeDropper. A presently displayed title on black-background has a black reading of RGB=0,0,0 for the LegalClip() script. This actual result on a real video source shows it's not being CCIR601-clipped.

:!: Okay, both files are finished with the "PC" setting -- 3,833,105 w/o LegalClip(), 3,833,192 w/LegalClip(). Interesting ... no different than the "TV" setting file sizes. I guess that's not a factor here.

:?: Regarding AVS 2.5b, any idea where they got their code for Limiter() (LegalClip() maybe? -- same results make me wonder)? The docs don't detail such credits.

SansGrip should probably take a look at this 8O

@kwag:

wow, I just understood 75% of this but... I always thought LegalClip() would reduce file size, by clipping frequencies that can not be displayed anyway... did I miss out on that one? :roll:

Quote:

Originally Posted by Jellygoose

@kwag:

wow, I just understood 75% of this but... I always thought LegalClip() would reduce file size, by clipping frequencies that can not be displayed anyway... did I miss out on that one? :roll:

Yes, and my post was incorrect :!:. I'll fix it now. It should say that using legalclip, the file size should be smaller. So obviously something is wrong :roll:

-kwag

I just did a little test by adding SansGrip's RangeInfo filter to my script and comparing the high and low pixel values with and without LegalClip(). After it was through evaluating the end credits, the results were;
without LegalClip() low-0, high-255
with LegalClip() low-16 and high-235

Quote:

Originally Posted by Jellygoose

@kwag:

wow, I just understood 75% of this but... I always thought LegalClip() would reduce file size, by clipping frequencies that can not be displayed anyway... did I miss out on that one? :roll:

It doesn't clip any frequencies IIRC, it just clamps the values to the limits of the standard CCIR-601.

:) Thanks for checking on this (and for referencing something that can verify CCIR601 values). Of course I'm relieved to know others can and have verified it's operation (and a little embarrassed at my panic, too :oops: ).

Anyway, I'm off to more testing on my original clips, which didn't seem to be altered by LegalClip() (and for that matter Limiter() ). Now, with RangeInfo(), hopefully I can get (self) satisfactory answers to my original questions.

:arrow: I've preformed some tests based on the math of RGB to YUV conversion. This is a formula from one site:

Quote:

RGB <--> YUV Conversion Formulas
[Charles B. Owen hompage]
Y = (R * .299) + (G * .587) + (B * .114)

:arrow: For the purposes of this test, I'm only concerned with the luminance in RGB and YUV. Here's the math for various test background "colors" (which are entered in RGB):

RGB = $000000 [black ... grey value of 0]
Y = (0 * .299) + (0 * .587) + (0 * .114) = 0

RGB = $0f0f0f [grey value of 15]
Y = (15 * .299) + (15 * .587) + (15 * .114) = 15

RGB = $101010 [grey value of 16]
Y = (16 * .299) + (16 * .587) + (16 * .114) = 16

RGB = $111111 [grey value of 17]
Y = (17 * .299) + (17 * .587) + (17 * .114) = 17

Notice that for equal-RGB (shades of gray) values, the YUV luminance values are the same (as one would intuitively expect).

:arrow: If the following AVS code is run (I test in MPLAYER2.EXE), it generates assorted black backgrounds. RangeInfo() then reads out statistics on what it believes are the YUV values that LegalClip() uses to "clamp" to CCIR601 values. RangeInfo() also allows for a green mask of pixels within it's set limits.

Code:

# RangeInfo() in AVS 2.0.x

#-------------------------



clip1=BlankClip(width=400,height=184,color=$000000)

clip1=ConvertToYUY2(clip1)

clip1=RangeInfo(clip1,ylow=16,yhigh=235)



clip1a=BlankClip(width=400,height=32,color=$000000)

clip1a=ConvertToYUY2(clip1a)

clip1a=Subtitle(clip1a,"RGB=$000000, Y range=16-235, green=within range")



clip2=BlankClip(width=400,height=184,color=$101010)

clip2=ConvertToYUY2(clip2)

clip2=RangeInfo(clip2,ylow=16,yhigh=235)



clip2a=BlankClip(width=400,height=32,color=$101010)

clip2a=ConvertToYUY2(clip2a)

clip2a=Subtitle(clip2a,"RGB=$101010, Y range=16-235, green=within range")



StackVertical(clip1a,clip1,clip2a,clip2)

:!: What follows, for the too-lazy-to-run it :), are the readouts of RangeInfo(). Notice that for a "0" black color, RangeInfo() reads a luminance of "16". For a "16" black color, RangeInfo() reads a luminance of "30". This is counter to the math (above). It is significant that I get corresponding results when using LegalClip(), both by the same author.

RangeInfo for black $000000 screen:

Quote:

Frame Y 16 - 16 (mean 16, most 16)
Frame U 128 - 128 (mean 128, most 128)
Frame V 128 - 128 (mean 128, most 128)

RangeInfo for near-black $101010 screen:

Quote:

Frame Y 30 - 30 (mean 30, most 30)
Frame U 128 - 128 (mean 128, most 128)
Frame V 128 - 128 (mean 128, most 128)

:?: Anyone knowledgeable care to look into this and critique these worrisome findings?

There is nothing worrysome about your findings, except that you use the wrong algorithm for calculating YUV. These are the specs defined as CCIR601.

This is the code actually used:

Code:

inline int RGB2YUV(int rgb) 

{

  const int cyb = int(0.114*219/255*65536+0.5);

  const int cyg = int(0.587*219/255*65536+0.5);

  const int cyr = int(0.299*219/255*65536+0.5);



  // y can't overflow

  int y = (cyb*(rgb&255) + cyg*((rgb>>8)&255) + cyr*((rgb>>16)&255) + 0x108000) >> 16;

[...]

BLACK = RGB(0,0,0) = YCrCb(16,80,80)
WHITE = RGB(255,255,255) = YCrCb(235,80,80)

You should read up on your colorspaces and get your facts straight - you using assumptions from RGB colorspace and wrongly applying them to the YUV colorspace.

Both Legalclip (2.0) and Limiter (2.5) are cropping correctly - as are the RGB <-> YUV routines correct.

I found a nice link http://www.animemusicvideos.org/guid...olorspace.html with the help of Dr. Google that explains this stuff pretty good :D Well- it sure helped me :wink:

ren

Very good page! Recommended reading :)

Quote:

:!: First, thanks for looking into this. It's not a matter of getting my facts straight, as it is just getting them. :) My noticing that an image, filtered by LegalClip(), and by Limiter(), came out with no apparent change, prompted my inquiry into this. I think I'm learning more about colorspaces than I really want ... but what else is life for if not to explore life's mysteries? :D

:?: So, from your example, quoted above, are you saying that the full range of RGB brightness values (from 0 to 255) is, or as translated, CCIR601 legal? And does this, by obvious extension, apply to all RGB colors as well?

While I was waiting for a reply with, perhaps, definitive information, I thought to expand my script displays to include color as well. The AVS ColorBars() command produces a final RGB image (at least, on screen) with a "16" value for the low (actually, the lowest is "7", but that color strip is put there as an "illegal color" for eyeball brightness adjustment) and a "235" value at the high. That is what I expect for an RGB image within CCIR601 "legal colors".

:arrow: So I made a script that displays colorbars with maximum color brightness levels from 0 to 255 -- one plain, one filtered with LegalClip():

Code:

# AVS 2.0.7 script

#------------------------- 



cliptitle = BlankClip(width = 280,height = 32,color = $000000)

cliptitle = ConvertToYUY2(cliptitle)

testtitle = Subtitle(cliptitle,"AVS 2.0.7 script")

clip1title = Subtitle(cliptitle,"SMPTE colorbars w/o LegalClip()")

clip2title = Subtitle(cliptitle,"SMPTE colorbars with LegalClip()")



clipR1C1 = BlankClip(width = 40,height = 80,color = $cccccc)

clipR1C2 = BlankClip(width = 40,height = 80,color = $ffff00)

clipR1C3 = BlankClip(width = 40,height = 80,color = $00ffff)

clipR1C4 = BlankClip(width = 40,height = 80,color = $00ff00)

clipR1C5 = BlankClip(width = 40,height = 80,color = $ff00ff)

clipR1C6 = BlankClip(width = 40,height = 80,color = $ff0000)

clipR1C7 = BlankClip(width = 40,height = 80,color = $0000ff)

clipR1 = stackhorizontal(clipR1C1, clipR1C2, clipR1C3, clipR1C4, clipR1C5, clipR1C6, clipR1C7)



clipR2C1 = BlankClip(width = 40,height = 16,color = $0000ff)

clipR2C2 = BlankClip(width = 40,height = 16,color = $131313)

clipR2C3 = BlankClip(width = 40,height = 16,color = $ff00ff)

clipR2C4 = BlankClip(width = 40,height = 16,color = $131313)

clipR2C5 = BlankClip(width = 40,height = 16,color = $00ffff)

clipR2C6 = BlankClip(width = 40,height = 16,color = $131313)

clipR2C7 = BlankClip(width = 40,height = 16,color = $cccccc)

clipR2 = stackhorizontal(clipR2C1, clipR2C2, clipR2C3, clipR2C4, clipR2C5, clipR2C6, clipR2C7)



clipR3C1 = BlankClip(width = 50,height = 32,color = $083e59)

clipR3C2 = BlankClip(width = 50,height = 32,color = $ffffff)

clipR3C3 = BlankClip(width = 50,height = 32,color = $3a007e)

clipR3C4 = BlankClip(width = 50,height = 32,color = $131313)

clipR3C5 = BlankClip(width = 15,height = 32,color = $000000)

clipR3C6 = BlankClip(width = 15,height = 32,color = $131313)

clipR3C7 = BlankClip(width = 15,height = 32,color = $262626)

clipR3C8 = BlankClip(width = 35,height = 32,color = $131313)

clipR3 = stackhorizontal(clipR3C1, clipR3C2, clipR3C3, clipR3C4, clipR3C5, clipR3C6, clipR3C7, clipR3C8)



colorbar1 = stackvertical(clipR1,clipR2,clipR3)

colorbar1 = ConvertToYUY2(colorbar1)

colorbar2 = LegalClip(colorbar1)

stackvertical(testtitle,clip1title,colorbar1,clip2title,colorbar2)

:!: In my estimation, if LegalClip() (and Limiter()) were working, I would end up with a duplicate image of your built-in AVS-generate colorbar image. I don't, but I think I should. Otherwise, how is it that AVS comes up with it's own colorbars that way?

:?: Please, any comments about this "colorbar dilemma"?

RGB cannot be CCIR601. RGB is ALWAYS 0-255 in all definitions. RGB is only black when RGB is 0.

YUV can be "fullrange" (seldom seen) or it can be CCIR601 as it is in all MPEG formats, the DV format, etc. So in short terms:

RGB is ALWAYS 0-255.
YUV is MOSTLY 16-236, and should ALWAYS be so for AviSynth processing.

Quote:

RGB is ALWAYS 0-255.
YUV is MOSTLY 16-236, and should ALWAYS be so for AviSynth processing.

Hmm ... interesting. As far as conversion is concerned, does that make RGB a colorspace subset of YUV? (Sorry if that opens up a new can of worms. :) )

:arrow: I have two practical questions as relates to recording broadcasts and storing on digital media (CD's for present, DVD's in future).

:?: QUESTION 1

My (previously posted) script's "0-255 RGB colorbars" are not altered by LegalClip() (0-255 goes in, 0-255 comes out) -- meaning they are within YUV CCIR601 in AviSynth. Yet, AVS ColorBars() generates "16-235 RGB" colors. Why does AVS's ColorBars() generate 16-235 RGB colorbars rather than 0-255 RGB colorbars?

:?: QUESTION 2

When I grab a frame (in, for example, PowerDVD) from a DVD movie, it's RGB range isn't 0-255, rather it is 16-235. With a DVD (for television viewing) being within YUV CCIR601, and with that equivalent to 0-255 RGB, why do I get 16-235 RGB frame?

Quote:

Why does AVS's ColorBars() generate 16-235 RGB colorbars rather than 0-255 RGB colorbars?

Don't know - it has done so forever. The source links to this site.

Quote:

Don't know - it has done so forever. The source links to this site. ( http://www.greatdv.com/video/smptebars3.htm )

:!: I remember running across that site, once, some time ago. To me, it further indicates that clipping, by what-ever filter, should result in a similar color range of the final RGB-displayed image. But thanks anyway for addressing my concerns on this. I'll keep digging (for my own satisfaction) and will post again if I find any crucial information.