Hi SansGrip,

SansGrip wrote:LegalClip is unrelated to DCT blocks. Without going into too much detail, there's a standard called CCIR-601 which defines various elements of video signals destined for TV (all formats: NTSC, PAL, SECAM etc.). It specifies that luma pixel values must be in the range of 16-235 and chroma pixel values must be in the range of 16-240. Colours outside this range are known as "superblack" and "superwhite" and can cause problems on some TVs.

Since Avisynth can produce pixels outside these ranges, I wrote this filter to clamp to the legal ranges. In other words, luma pixels in the range 0-15 become 16 and 236-255 become 235. The same is done in chroma except the upper limit is 240.

If you look at the output of mpeg2dec you'll notice a lot of out-of-range pixels, since they seem to be a side-effect of MPEG compression. In this case these pixels convey no actual information -- they are just artifacts -- so removing them is both The Right Thing To Do For TVs (tm) and doesn't force the encoder to use up bits describing useless information that will get removed by the DVD player anyway.

Edit: When encoding from an MPEG-2 source I put one instance of LegalClip directly after Mpeg2Source so that the out-of-range artifacts don't interfer with the results of other filters in the script, and then one at the very end to remove any that might have been generated during processing.

Thanks for explaination. I and others at KVCD are constantly look-
ing for ways to improve picture quality even more. The old "AddNoise"
was a start for me and "BlockBuster" gave me problems also until you
and Kwag stepped in. My major interest is "mosquito noise" at low
CQ settings. I did look at trbarry's "UnDot", but it must be use with
avisynth 2.5 alpha. The source seemed to be available and wondered
if something could be written for avisynth 2.05 or 2.06? This type of
pratical solution solving is what I like. Keep up your great work.

-black prince

can you put a script here to test it;please?

thanks.

@SansGrip,
Thanks for joining and thanks for the explanation. I'll be experimenting more with Blockbuster filter

-kwag

black prince: My major interest is "mosquito noise" at low CQ settings.

As soon as I have some time I'm going to sit down and read this forum, since my main interest is MPEG-1 with TMPGEnc. I've never tried CQ mode at anything less than 100, so I still have a lot to learn .

I did look at trbarry's "UnDot", but it must be use with avisynth 2.5 alpha. The source seemed to be available and wondered if something could be written for avisynth 2.05 or 2.06?

I'll find the source and take a look at it. I can't think of any reason it need be 2.5-specific, since it does YUY2 already, so "porting" it to 2.0 should be fairly simple.

Keep up your great work.

I decided it would be best to balance my cosmic karma by writing a smoother too. You can check that out at my web site .

kwag: Thanks for joining and thanks for the explanation

No problem. I'll be happy to try to answer any questions you might have about Blockbuster (or any of my other filters for that matter). Suggestions and feature requests are always welcome, too .

Hi SansGrip,

I am about to start looking at the source of UnDot. I assume that he uses some API calls, specific to AviSynth 2.5. So it might be a matter of rewriting the API calls. I've been programming in C for some time ( for too long ), but I have no experience with the AviSynth API and hooks, and very little C++ experience. So it will take me some time to study the mechanism, before I can actually do something with AviSynth plugins. Also, I don't use MS VC++ ( ) I use GCC on Unix and recently started to use DEV-C++ on Windows for plain "LEAN_AND_MEAN" ( you should know what that is ) console applications. So I have A LOT of studying to do
I mention that, because I see that UnDOt was written with ( ) Microsoft Developer Studio.

-kwag

@SansGrip,

What can I say I'm drooling
Here's my first try, using the KVCD LBR template, with a CQ_VBR of 27 ( the value I used originally to put this particular movie on one CD-R )

Here is the .m1v without any filters:
http://www.kvcd.net/test-nofilter.m1v
And here's the same sample, but with the line: Blockbuster(method="sharpen", detail_threshold=0, lv=20) added to my test .avs script:
http://www.kvcd.net/test-sharpen.m1v

AwWwWewsome

I'll be doing many more test strips tomorrow

Thanks,
-kwag

any help please,i can't see a thing.

Hi jorel,

Here's a script you could try.

# ------- Beginning of Script --------
LoadPlugin("E:\DVD Backup\2 - DVD2SVCD\MPEG2DEC\MPEG2DEC2.dll")
LoadPlugin("E:\DVD Backup\2 - DVD2SVCD\BlockBuster\BlockBuster.dll")
LoadPlugin("E:\DVD Backup\2 - DVD2SVCD\LegalClip\LegalClip.dll")
mpeg2source("D:\Temp\movie.d2v")
#
LanczosResize(336,224)
#
opening=Trim(0,550)
opening=opening.TemporalSoften(3,8,30)
#
movie=Trim(551,158936)
movie=movie.Blockbuster(method="noise", detail_threshold=0, lv=1)
#
credits=Trim(158937,0)
credits=credits.Greyscale().TemporalSoften(3,8,30)
#
last=opening+movie+credits
#
AddBorders(8,8,8,8 )
#
LegalClip()
ConvertToRGB()
# ------End of Script ------

@Kwag

Downloaded your test scripts and ditto. It's awesome!!!

-black prince

Sorry for not posting my script jorel,
Here it is:

LoadPlugin("C:\encoding\MPEG2DEC.dll")
LoadPlugin("C:\encoding\blockbuster.dll")
mpeg2source("K:\UNDEFINED\VIDEO_TS\movie.d2v")
BilinearResize(336,224,0,0,720,480)
Blockbuster(method="sharpen", detail_threshold=0, lv=20)
#TemporalSmoother(2,2)
AddBorders( 8,8,8,8 )


That's the script I used for the samples I posted.

-kwag

Just finished encoding the complete movie with the blockbuster filter. Originally, I had used a CQ_VBR of 27. With the blockbuster filter, I had to change to 24.5 to maintain the same target file size. But the difference in quality is excelent! Still, the file prediction works as expected. +-1% even with the filter

-kwag

thanks!
sometimes i read the faqs or help of a prog or "how to use it"
and it's not clear for my poor english,than i use a sample,
test it and learn something!

@kwag
Your test downloads aren't working for me. But why did you choose sharpen over noise ? Are there certain settings for anime than for live movies ?

That was just a test. I still have to experiment with noise and blur.
Play the samples with WMP. If you want to see them with WinDVD, rename them to .mpg. They don't have any sound.

-kwag

Hello SansGrip,

Here is my first in depth test with your filters. Please let me know if what I describe below is correct, or is bologna
Here is the latest combination I have tried, with a detailed explanation followed:

Blockbuster(method="sharpen", detail_threshold=20, Invert=True, lv=20) # Only sharpen material above 20%.
Blockbuster(method="noise", detail_threshold=20, lv=2) # Only apply noise below 20% complexity.

Both lines were added to my test .avs script.
This is what I have done, after reading your documentation, and understanding how the filter works.
The "sharpen" line on the script, works as a high pass filter by selecting "Invert=True", so only the content above the treshold of 20 is processed by the filter. That means that low lit and low to medium details won't be sharpened at all. Which makes sense, because we don't want to sharpen what we can't see, including DCT blocks.

Then the "noise" line, works the opossite way. It works as a low pass filter, because everything below the treshold of 20 is injected with noise. So only low lit scenes see noise. Thus reducing DCT blocks.

The result is this sample: http://mpeg.0catch.com/test-sharp-noise.m1v ( Very small, 948KB )

Now I believe that what we need next, would be a variable frequency notch filter, targeted at the "Mosquito Effect" frequency domain. That would complete your filter set as the "Killer Filters Set"

-kwag

kwag: What can I say I'm drooling

hehehe

Here's my first try, using the KVCD LBR template, with a CQ_VBR of 27 ( the value I used originally to put this particular movie on one CD-R )

Looks good, though possibly a little over-sharp in places. Or that might just be my imagination, since I knew you used the sharpen method on it .

Blockbuster(method="sharpen", detail_threshold=0, lv=20)

Obviously the docs aren't very clear on which parameters go with which method. The lv (luma variance) parameter is specific to the "noise" method, and will have no effect with the others.

I'm going to fix the docs today, but in the meantime here's the valid parameters for each method:

method="noise": detail_threshold, block_size, invert, lm, lv, cm, cv, cache

method="sharpen": detail_threshold, block_size, invert, strength

method="blur": detail_threshold, block_size, invert, strength

method="show": detail_threshold, block_size, invert

I'll be doing many more test strips tomorrow

Glad to hear that, it'll be nice to put some "recommended ranges" on the various parameters .

kwag: The "sharpen" line on the script, works as a high pass filter by selecting "Invert=True", so only the content above the treshold of 20 is processed by the filter.

Well, technically with invert it's a low-pass filter, because only the high-frequency information is being targeted. "Low-pass" means the low end gets passed through unaltered while the high end gets processed. At least, that's my understanding of the term and I could easily be proven wrong .

Bear in mind that increasing the frequency of already high-frequency information will probably result in more mosquito noise. It's not evident in that clip because of the overall low luma levels, but you might want to look for it in day-time sections of the movie.

The clip looks excellent -- definitely the best of the three you posted.

Now I believe that what we need next, would be a variable frequency notch filter, targeted at the "Mosquito Effect" frequency domain.

I've yet to do any analysis on exactly what frequencies are involved in the various types of artifacts. That would be interesting, and would mean I'd know roughly what ranges of detail_threshold to recommend for each. Perhaps what might help is to remove invert and split detail_threshold into two parameters, detail_threshold_min and detail_threshold_max, so that users could target a specific frequency range?

(Now this is the kind of productive discussion I've been wanting since I released the filter... )

Hi SansGrip,

Ok, it was my mistake!. I should have specified more clearly what was passed to the filter. My explanation refering to what was "passed" to the filter, is what was processed by the filter. That's why I used the "Invert" on the sharpen line. This way it's a high pass filter, meaning that above 20, sharpenning is applied. But on the noise filter, below 20 is applied. So the correct meaning I wanted to express is that the "Active" part of the filters is what I described as low pass and high pass. Sorry for the confussion

Exactly. On high luma, the noise filter will not be injecting noise, because the scenes will be above the treshold of 20, so the noise filter will be inactive. But the sharpen filter will be active. I know I set the "cut off/turn on" point the same, thus when the luma increases from below 20 to 20 and above, theoretically that should trigger the sharpen filter and stop the noise filter. Maybe I could widen the sharpen to turn on at a higher point than 20. Maybe around 50. So that would create a "Bandpass Filter" area, where neither sharpen nor noise would have effect. So >=20 OR <=50 is the bandpass window area.

Thanks to you
I made some small changes.
I changed:
Blockbuster(method="sharpen", detail_threshold=20, Invert=True, lv=10) # Only sharpen material above 20%.
Blockbuster(method="noise", detail_threshold=20, lv=1.5) # Only apply noise below 20% complexity.

The lv=10 on the sharpen line, and lv=1.5 on the noise line. It was a little strong before. Here are two samples of dark screen with fog on the scene. Look at the DCT macroblock dance on the sample without filter, and look at the sample with the combination of both filters above:
http://mpeg.0catch.com/test2-sharp-noise.m1v
http://mpeg.0catch.com/test2-nofilter.m1v

This is a great discussion , and what you just described above, basically a bandstop filter, is exactly what's needed for the "Mosquito effect". Your idea of a MIN and MAX treshold specifier is great, because that would allow a "Dynamic Window" to be adjusted to the frequency domain of the "Mosquito Effect" I just hope the frequency domain falls in a very small segment, because if it has separate frequency blocks, then the only way to filter that would be to have a king of "Parametric Equalizer" ( similar to what's used on audio equipment ) kind of filter, in order to notch out each block of noise area. What do you think of that

Regards,
-kwag

Edit:
I changed the script as follows:
Blockbuster(method="sharpen", detail_threshold=20, Invert=True, lv=2) # Only sharpen material above 20%
Blockbuster(method="noise", detail_threshold=20, lv=1.5) # Only apply noise below 20% complexity

You're right on target SansGrip . On bright scenes, the value for lv on sharpen was too high. At lv=2, it's enough to sharpen the image, while not increasing the "Mosquito Effect" that much. I'll post samples in a while. The previous samples are not affected, because it only affected bright scenes.

As promised. Here are the two new samples, with the filter settings I posted before:

Blockbuster(method="sharpen", detail_threshold=20, Invert=True, lv=2) # Only sharpen material above 20%.
Blockbuster(method="noise", detail_threshold=20, lv=1.5) # Only apply noise below 20% complexity.

http://mpeg.0catch.com/test3-sharp-noise.m1v
http://mpeg.0catch.com/test3-nofilter.m1v

And please, don't judge the quality of these samples. They are meant only to test the filters. With those settings, the movie does fit on a single CD-R. And because it's a Full Screen film, it won't look nearly as good as if it was the Wide Screen Version.

-kwag

A high contrast scene, made with KVCDx3, using a CQ_VBR value of 15.5 to save space, using the same filter parameters of the previous post:

http://mpeg.0catch.com/test4-sharp-noise.m1v
http://mpeg.0catch.com/test4-nofilter.m1v

-kwag