Hi Kwag,

Kwag wrote:
I never created a KVCDx3 using CQ_VBR below 25. Your test clips
looked great for CQ = 15.5. I was hoping that by targeting certain
problems using noise, the CQ could be lowered enough to have the
file size still remain small. When there is some layman's explainations
during discussions between you and SansGrip, it's very exciting. :D
The sample with sharp-noise filters are still hard for me to believe
you did this with CQ=15.5. 8O I just hope you have as much success
with mosquito noise as the others. :D

-black prince

Corrections.
 

Correction on the filter syntax I was using:

Blockbuster(method="sharpen", detail_threshold=20, Invert=True, strength=20) # Only sharpen material above 20%.
Blockbuster(method="noise", detail_threshold=20, lv=1.5) # Only apply noise below 20% complexity.

I had the "sharpen" line calling a "lv=2" which is an incorrect parameter name for the "sharpen" method.

Anyone trying out the filters, please make the corrections above.

@black prince,
The cleanup of the mosquitoes and flies is the only thing really missing to make MPEG-1 better than what it is now :D

-kwag

Re: Blockbuster sharpen + noise ( Low pass, high pass filter
 

And the sharpening filter will still be increasing the frequency, by amplifying existing detail instead of adding noise. Hence you might see more mosquito noise.

The lv parameter has no effect with the sharpen method -- it controls the variance of the random number generator when using the noise method. To adjust the degree of sharpening, use the "strength" parameter.

I'm about to release a new version with a couple of added features, and it will also check the parameters are correct for the current method.

Ok, I'll implement that in the new version before I release it.

I think that sounds complicated ;). We'll see whether the min/max idea will be able to target it first...

New version
 

I just released 0.4 (and source) with the following changes:

* Added luma_offset and luma_threshold parameters
* Split detail_threshold into detail_min and detail_max
* Removed redundant invert mode

See the documentation for more info.

Have fun :).

Gee thanks SansGrip :D

-kwag

So now it would read something like this:

Blockbuster(method="sharpen", detail_min=60, detail_max=99, strength=20) # Only sharpen material above 60%.
Blockbuster(method="noise", detail_min=1, detail_max=20, lv=1.5) # Only apply noise below 20% complexity.

And that would apply noise to material below ~20%, leave the mid frequencies from ~20% to ~60% unsharpened, and apply sharpenning from ~60% to the highest frequencies :)

And for what I can see, if we can find the frequency of the "Mosquito Effect" maybe a method=blur with a tight min,max focusing on the noise, would blend only the mosquito frequencies :idea:

Excelent work SansGrip :)

-kwag

Yep :).

Bear in mind that, while it's a close (and fast) approximation, counting the number of unique luma levels in a block is not quite the same as determining the block's frequency. That would require more complicated code (I'd have to do a Fast Fourier Transform on the data just like an encoder does) and is something I'm trying to avoid if this method works just as well.

WRT mosquito noise (otherwise known as the Gibbs Effect, which I learned thanks to the link on artifacts from LadyMiles), it tends to occur at the boundaries between low- and high-frequency areas rather than be associated with a particular level of detail, so my hopes aren't high that this filter will be able to combat it in any targeted manner.

I think the main result of blurring high-detail areas would be, well, loss of detail. If it does reduce mosquito noise that would in my opinion be a side-effect. To carry on the mosquito analogy, it's akin to throwing a bucket of pesticide at a swarm of them. Sure, it'll kill a few (or even most) of them, but has very undesirable consequences for whatever the excess lands on ;).

What I think would be most effective is a seperate filter to blur high-frequency parts only around edges. I'm going to think about it for a while and might add that to my TODO list :).

Incidentally, do you have any clips (say, <20mb) that demonstrate bad mosquito noise? If I can see the worst-case scenario I might be able to pin down a solution more easily.

You've got a strong point there :)
I will make one specifically for you and give you a download link :wink:

-kwag

@SansGrip,

Please check your private messages :!:

-kwag

Hi Kwag and SansGrip,

With regards to Gibbs effect will UnDot have this effect:

-black prince

Well, the author (Tom Barry) says it does and I respect his opinion greatly. He's been writing filters and working with video far longer than I.

However, the question is does he mean existing mosquito noise, or areas of potential mosquito noise? I would imagine he means the former, whereas we want a proactive filter to somehow stop them happening in known trouble spots.

@SansGrip,

SansGrip wrote:
Yes your right. He mentions in some of his posts about the source
being Divx (avi). I have a feeling this problem is the encoder formulas
that fail to detect this between scene changes. Writting a new encoder is
work. My test so far with the Blockbuster filter settings made by you and
Kwag have produced excellent mpeg-1(LBR) video using only cq = 16. It looks
better than my video without filters at cq = 25. They both suffer from the
same amount of Gibbs noise. Once a solution is found for Gibbs, I can
drop cq even lower. Thanks for your hard work, Kwag and SansGrip.
Your work with this problem is so exciting, I visit this forum 4 and 5
times a day. :D

-black prince

Hey black prince,

Don't thank me :D , it's all the work of SansGrip. I only follow his documents, and put them on practice after analyzing and testing. Send him a virtual beer :lol: ( But just one, because too many will get us a blockbuster filter creating Moire patterns and inverted videos on our clean mpegs :lol: )

-kwag

@SansGrip,

This one's for you guy!!! :D Here's a tall imaginary virtual beer coming up.
Ahhhhhh!! that's was smooth. :P And here's another toast for the queen!!
Ooops, I think I am virtually drunk. Let me get out of here
before the virtual cops catch me. :D and put me in virtual jail.

Thankx SansGrip,

-black prince

Once a solution is found for Gibbs, I can drop cq even lower.

I've not done any tests, but my gut tells me this won't be as easy as getting rid of DCT blocks. I'm gonna try, though ;).

hehehe thanks, but really I wrote this for myself. I didn't realize there were other people out there looking for a solution too ;).

Yes. That would be dealing at the macro block level, because low CQ is actually bit rate starvation.
Maybe what is needed is something like a "pre motion estimation filter", sort of, for the encoder :idea: If that makes any sense at all. Just a wild guess :roll: Sort of a software Time Base Corrector ahead of the stream :idea: ( Maybe I'm virtually drunk now :lol: )

-kwag

I think the first thing I'm going to try is edge-constrained blurring, sort of like Donald Graft's MSmooth but in reverse :).

What about 180 degree phase shifting on macro block boundaries? Thus, cancelling the block edges. Is that a crazy idea 8O
Like duplicate the frame, then phase shift the block edges on one copy, and join the two frames again. The result :?: Don't know. Just another crazy idea :idea:
I know in the analog world that would work fine ( phase cancellation, etc. ). Not sure if it would apply to mpeg encoding :roll:

-kwag

I'm not quite sure how a phase shift would be accomplished. Do you know any good links to information?