Motion adaptive filtering now possible?
 

Source: http://forum.doom9.org/showthread.php?s=&threadid=52989

All I can say is 8O

Has anyone here seen this yet? I'm currently working with this script to try to figure out just exactly how it works, but it looks pretty good. Could motion adaptive filtering be the next step in the evolution of KVCD?

From the NoMoSmooth README:
So it now seems that if this script works like it appears to, we can do precisely that, and more. I like to use a temporal smoother at a setting of around 2 for my VHS captures to stabilize solid areas like blue sky backgrounds, but I find that the detail is so poor to begin with in VHS sources that a temporal smoother totally kills any motion scenes. So now apparently I can use a temporal smoother on low motion scenes, and leave the rest alone since I don't want any smoothing (temporal or spatial) on motion scenes. :lol: :lol: :lol:

Ideas, anyone?

This is an excelent idea :!: We can have control, and maybe even using heavy filtering on high motion parts by adding stronger filters on action parts. Even increasing the mergeluma value on motion, because on motion there's not much detail visible anyway, we can blur much stronger :idea:
Even the actual "current script" can be used in two parts or three parts. I mean a low to medium action bracket values and a medium to high action bracket values :idea:
Actually, if it's possible, the best (and ideal!) way would be to change the filter values dynamically on a scale, where the higher the action, the higher the filter values are automatically applied :idea:
This way we get a dynamic range of values applied to the movie being encoded automatically :idea: :idea: :wink:

-kwag

Well, it's still a very cool filter, but there are some downsides..... :?

As near as I can tell, it depends on the new conditional filter function in the latest AVISynth 2.51 binaries, which according to sh0dan...
http://forum.doom9.org/showthread.php?s=&threadid=50105

It requires YV12 input, which I suppose is OK if you're ripping DVDs and aren't attached to any old YUY2 filters like dust. But I do mostly analog captures, which means that I capture in HuffYUV and that dust is pretty much a requirement for me, so I'm looking at a lot of ugly colorspace conversions.

HuffYUV -> YV12 -> Filter_Motion function -> YUY2 -> Dust -> YV12. :roll:

The script seems to work great as long as you use it as-is with all YV12 filters. But for some reason the colorspace conversions (or dust, or both) are introducing some REALLY weird stuff. Every few seconds it will insert one or two frames from a previous scene and then return to the proper point in the sequence. :?

I also get VDub crashes if I use anything other than 0 for the cropping values.

I've been looking at it all day, and I'm only now beginning to understand what it does a little bit. If this turns out to be the Next Big Thing(TM), it pretty much means a wholesale move to 2.51 and pure YV12 for everybody, and I don't think we're ready for that until some filter developers get on the ball with YV12 support.

[edit] I figured it out. I was using Pixiedust for high motion and FaeryDust for low and medium motion. There can only be 1 instance of dust in a script, and apparently the 3 functions were treating it like 3 instances of dust. So I guess dust can't be one of the conditional filters. And moving it up to the main script cut out a lot of those conversions.

I'm still keeping away from avisynth 2.5x :!:
It's just not stable enough :?
I would really like to see this filter working with the "current script", which is just mergechroma and mergeluma. Simply applying a higher value on mergeluma on high action scenes. This will lower the file size because it will really blurr the image, but the image wouldn't have any detail anyway on a high speed scene :!: So maybe the script simplified a little, might just get us started playing with this :D

-kwag

@ KWAG !
I just found this article .
Sounds very promissing , hah...
Did u tryed it already.
If u tryed what was a result ? Is it working as have to ???
I'll try it when I'll come back from work until then whanted some more info if it's possible :wink:
bman

No, and I guess someone else will have to try it, because I'm not running avisynth 2.5x :twisted:
Until I see less problem reports on newer versions, I'll stick to the tried and tested 2.08 :!:

-kwag

Guys !
We have new challenge in here !
Who is running Avisynth 2.5 ? Could u try this script and see if it's working as promissed ?
Cuz there is french forum
http://atlas2.tgv.net/~media-video/f...pic.php?t=2267
and there is ( I don't speack French but assume ) more advanced script that looks interesting too .
Someone with AVS v2.5 :!: :?: :!:
bman

Ok, I have tried it with dust and C3D on analog captures, and it seems to work exactly as advertised.

However, I'm having some trouble implementing the KVCD script because 2.5 stmedianfilter has different paramaters than the 2.0x version, there is no temporal smoother in 2.5, and I can't get the syntax right to put mergechroma and mergeluma in the functions. If you guys will help me get the YV12 equivalent of the current script, I will be the guinea pig. :lol:

This is what I'm using right now:

As you can see, I have the same mergeluma and mergechroma values in all the functions. I want to get it working before I start playing with the values. Using this script, I get an error about illegal options in the blur functions. I have also tried it with the following syntaxes:

Medium = mergechroma(Medium, blur(Medium, 1.50))

Medium = mergechroma(blur(Medium, 1.50))

So what's wrong with my mergeluma and mergechroma functions?

Hi sbin,

Maybe you have to change all the functions to read like this:

:idea:

-kwag

Oh yeah... I guess I should have mentioned that was the first way I tried it. :oops:

No matter how I set it, I get

On lines 67, 87, and 37.

Man, where is sh0dan when you need him. :wink: I think I will post this over at Doom9 since most of the heavy-duty AVIsynth people are there. I just registered an account, but I have to wait 3 days to post. :twisted:

Send him a PM here :!: :)

-kwag

After analyzing this filtering logic, this is going to be a SLOOOOOOOOOOOOOOOWWWWW process :!:
This scheme works on a frame-by-frame basis, and in theory it will work, but it's going the slowest encode of all times :!:
I'm not so sure if it's worth it :roll:

-kwag

I had thought so too, but it's actually surprisingly quick. Just for comparison purposes, I did a little test with Planet of the Apes TV series at 544x480 full frame. I created an intermediate AVI file with VDub using fast recompress to HuffYUV just to see how fast the script is. When I comment out the line

clip = Filter_Motion( clip, Width, Height, threshold_sm, threshold_hm)

in the main script above and use no conditional filtering at all, it runs at about 19/20 fps on my Athlon T-bird 1.3.

For the conditional filtering test, since I can't get the blurring syntax right, I substituted the C3D values given in the script posted at Doom9. I figure C3D is a reasonably fair comparison for speed purposes. With conditional C3D filtering, it runs at 13/14 fps. Using conditional filtering with empty motion filter functions, it runs at about 17fps. So the actual conditional filtering hit appears to be pretty small.

Using dust in an old-style script gives me about 5/6 fps. So I think this is certainly within the realm of acceptability. At least for me. :lol:

Yes it is :) I thought it was going to be slower. I guess the problem with the mergeX functions is that you can't have multiple instances running :!:
Similar to the xDust filters. I guess I should read more on the AviSynth's web site :wink:
I guess it all boils down to the function family "XDifferenceFromPrevious" that is available on AviSynth 2.5x :!:
If it was available on 2.0x, we could use it just by adding it to the current script. I guess I'll take a look at this today, by switching to avisynth 2.51 and trying this out. This really opens up a world of possibilities :idea:
Just like I had conditional filter strength calculated for BlockBuster filter depending on the resolution to be used, this could also be automated to apply BlockBuster "noise" to low lit scenes, but lower the noise value if there's movement :idea:. Really a "Dynamic" avisynth script :D

-kwag

OK , Now u are talking !!! :lol:
That's the spirit and bacause of that we are here master KWAG ! :D :D
bman

Ok, gang. After a false start earlier today, I now have a working script that does motion adaptive luma and chroma blurring and some preliminary results to report. It's not the prettiest script, but it does work.

My sample was The Spy Who Loved Me DVD. First, for a control sample, I commented out all conditional filtering and used mergechroma and mergeluma in the main body of the script with the current script values(1.50 and .2, respectively). File size was 15,217k.

Then I removed luma and chroma blurring from the main script and used motion adaptive luma and chroma filtering with the current script values for all three levels to test that the script still worked exactly the same. It did. File size was 15,217k. So I know the script is right and it works properly. Time to play with the values.

Going with Kwag's suggestion earlier, the only value I changed was luma blurring. At high .40, medium .30, and low .20, the file size was 14,979k. Not spectatular, but at least it establishes that it does work.

At higher values of high .60, medium .40, and low .20, file size was 14,808k. Again, not spectacular, but significant. All samples are visually indistinguishable to my eye. 8)

That's perfect :!: :D
Now try a value for luma of 0.2 for LOW, 0.75 for MEDIUM, and 1.5 for HIGH.
See how the file shrinks 8)

-kwag

Well, I changed some values in my script and a couple of minor things in TMPGEng, so I can't compare these samples to the ones I did earlier. I changed from Lanczos to Bilinear, did some different cropping, and used a different GOP (I was inadvertently using the KDVD template before).

The control sample with current script values was 13,406k.

The sample with your suggested values was 12,651k. 8)

Thanks sbin :)
Here's a cleaned up (with more to be cleaned!) working script, with the suggested values.

I changed some values in the width an height, so that it matches the values of BilinearResize, or whatever resizer is being used. Also, I removed the top, bottom, left and right variables, because we can use the resize, addborders and letterbox for that. Same for the "start" and "end" frame. Those variables and the Crop(clip, Left, Top, -Right, -Bottom) are really verbose. The original script was working on the full frame, so it was actually working on black bars the way it was :wink:
Now I'm going to optimize this thing, and see how to automate the blur strength in a linear fashion, instead of three steps (Low, Medium, High). This way, there won't be sharp cut-off transitions between the three areas being worked on. So the "blur" value will work on a linear range between 0.2 to 1.50, depending on the frame activity :D

Edit: This was tested with AviSynth version 2.51 Build May 15, 2003

-kwag

Yeah, that was the cropping issue I mentioned that I changed. :D

Ok, this looks very promising :wink:

-kwag

If you use MarcFD's mpeg2dec3, there is no need for this line because it supports native YV12.

http://ziquash.chez.tiscali.fr/

Thanks :mrgreen: , I was beginning to wonder if there was a workaround :D

-kwag

Not that it's really relevant to the thread, but MarcFD never ceases to amaze me. He may not be the absolute best AVS filter author, but for a 17-year-old kid he writes some pretty darned amazing stuff. This guy is going to be bringing us a LOT of good stuff in the future.

8O 8O Damn, I wish I had a computer when I was 17 :cry:
If I had, then probably KVCD would have been a standard in every home on every DVD player :mrgreen:

-kwag

Ok, there is definitely a problem with the stmedianfilter and temporalsoften settings. Using the straight current script, I get a file size of 12,434k, which is actually smaller than what I get with the best motion adaptive settings in the new script. Here are some file sizes. When I say "motion adaptive" I mean I used the luma blurring values of 1.5/.75/.2. In the case of the current script, that means some colorspace conversions. :?

current script = 12,434k
current script + motion adaptive = 11,867k

new script = 13,404k
new script + motion adaptive = 12,598k

Although there is no doubt that motion adaptive filtering reduces file size, the new script overall is not as good as the current script. Then I started wondering which new filter was the culprit. I replaced temporalsoften with the old temporalsmoother(1,2) (and, of course, the accompanying colorspace conversion) and got a file size of 13,054k - somewhere in the middle. Replacing stmedianfilter with the old 2.0 version and current script values got me down to the same size as the current script.

So while the principle is sound, the script needs a lot of tweaking to be as good as the current script. The following script gave me my best file size to date (11,867k):

That's very interesting .
According your test it's worth to switch to AVS v2.51 !!???
This Filter combination is my favorite for almoast every my encodes .
What I'm interesting is If u can compare resulted clip quality to quality without Filter_Motion . To make it easy to understand let's say : with Motion CQ 60 and to get the same visual quality clip without Motion - CQ 65 .
It's easyer to see difference ( if there is any ) when we express it in numbers .
Please report if u have new results about Filter_Motion .
bman

you're right sbin!
:)

MarcFD is fantastic.
:!:

I just noticed that I can't use this script with ToK 8O :?
The current version of ToK sets the file prediction at the end of the script, and it doesn't work :roll:
I also tried to move it to the top, after the "clip = Mpeg2source( Source )" line, and it doesn't work either :cry:
I believe that Sampler.dll has been compiled for avisynth 2.5x :?:
I guess in order to use this method, we must use manual file size prediction :x until we can get a fix for using this with ToK :!:

-kwag

amazing work, all of you! 8O
i hope this thread leads us to another big improvement in quality...
hopefully SansGrip can compile the sampler.dll to AviSynth2.51.

Isn't there some kind of integrated filter in the newest AviSynth version that does the same thing the sampler.dll?

Keep up the good work! I'll try this new stuff out tomorrow!
btw: can you run AviSynth2.0x and AviSynth2.5x at the same time?

I am the author of this intelligent script. The objective of this intelligent script is of detecter the various types of motion and to apply filters specific. Here some French addresses to use this script correctly.

Downlaod
http://jfl1974.free.fr/HTM/Download.htm

FAQ filters avisynth
http://jfl1974.free.fr/HTM/21_Test_Filtres.htm

Comparative of the video codecs
http://jfl1974.free.fr/HTM/22_Test_Codec_Video.htm

FAQ ffdshow
http://jfl1974.free.fr/HTM/24_ffdshow.htm

And here the last version of this script. I added "subtitle" so that you observe how script works. You can modify the threshold to make script more or less senssible to the scene fast and slow ... :wink:

Welcome Sagittaire :)
We've been playing and modifying your script :wink:

-kwag

Hi Jellygoose,

The script will work only on AviSynth 2.5x, because the heart of the script is the "Conditionalfilter" function :)
It won't be long before we convert this to a linear strength filter instead of three filter bracket ranges (Low, Medium, High), and integrate the new algorithm into MovieStacker ( Hello muaddib :mrgreen: ) :wink:

-kwag

OK Sagittaire is this going to work with our KVCDs, cause that looks like a Divx site and is like in frensh language :?: I don't understand frensh :?: :wink:

Hi ovg64,

Check page 2 of this thread. It's been translated to english.

-kwag

Hey kwag,

Did you try to move ALL functions to the top of the script?
I did not try the script yet, but it "should" work... :roll:

Hi K man! We will sure do that! 8) :mrgreen:
I didn't have time to put my hands in this great script (thanks Sagittaire), but don't stop... It's getting exciting! :wink:

You know, this thread reminds me something I exposed several months ago, read here: http://www.kvcd.net/forum/viewtopic.php?t=2834
And instead of using a long script to do all of this, if one of SansGrip's filters could be modified ( NoMoSmooth :idea: ) to include "attached" avisynth functions like mergechroma, mergeluma, etc, then we could use the temporal motion engine built-in this filter :!:
This way, it would work just like any other avisynth filters. For example, BlockBuster has tresholds to apply noise, depending on the darkness of the picture. NoMo applies filtering, depending on difference between adjacent frames. So this engine can be applied to "call" mergeluma or other filters, depending on selected activity "trigger" points.
SansGrip, where are you :D
The result: "Automatic Linear Motion Adaptive Filtering"
And that is what we need 8)

-kwag

Greetings all. I haven't had much time to play with this today, because I had to do some work at my paying job. :( But hopefully I can spend some quality time with this now that the weekend is here. :drink:

You can, but there's not much need to since loadpluginex.dll will allow you to run 2.0 filters in 2.5x.

But if you want to run both versions, you can make 2 virtualdub folders and put the different avisynth.dll versions in each folder. Then just start whichever Virtualdub version you want to use. Not sure how that works with frameserving directly to TMPGEnc, though.... never tried it.

:)
Sagittaire
welcome! :wink:

great pages and great job.
the result will be amazing

:!: