Avisynth: Ads() a Function Using a Masked Denoising etc - Page 2

J-Wo · #21 09-20-2004, 11:38 AM

Quote:

Originally Posted by Dialhot

Try to use "RemoveDirt(mthreshold=50)" insteed of just "RemoveDirt()".
EclCCE answers : not abble to load the script. (CCE 2.67)

I do not get that error message. I'm using EclCCE 1.81 with CCE SP 2.67.00.27. Whenever I've had problems with with plugins I always uninstall avisynth and install the latest version(v2.5.5 final just came out btw). My project files were created using DGIndex 1.0.12 by neuron2. Here is the script I used:

Code:

loadCplugin("D:\Program Files\AviSynth 2.5\plugins\SmartDecimate.dll")
mpeg2source("F:\Alias\5 bonus\b-jen garner.d2v")
b=dgbob(order=1)
smartdecimate(bob=b)
RemoveDirt(mthreshold=50)

Dialhot · #22 09-20-2004, 11:57 AM

Quote:

Originally Posted by J-Wo

DGIndex 1.0.12 by neuron2.

This is the only diff between your situation and mine but I don't think this can be the guilty. I will try to check.

audioslave · #23 09-20-2004, 03:31 PM

Hi guys!
I just made a test with FluxSmooth (the new version) and UnDot instead of the Remove-filters, and I really liked what I saw!

This is what I used:

Code:

function ADS (clip c, int "To", int "Bo", Int "Ov", Bool "Lb" ) {
To=default(To,0)
Bo=default(Bo,0)
Ov=default(Ov,0)
Lb=default(Lb,false)
Ov=Ov*8
W=width(c)
H=height(c)
cr=(W==720) ? 0 : 0
c= (To <= Ov && Bo <= Ov) ? crop(c,Ov+cr,Ov,-(Ov+cr),-Ov) : crop(c,Ov+cr,To,-(Ov+cr),-Bo)
c= ScriptClip(c,"nf = YDifferenceToNext()"+chr(13)+\
"m=masktools_EdgeMask(thY1 = 2, thY2 = 5, thC1 = 1, thC2 = 5, type= "+chr(34)+\
"roberts"+chr(34)+")"+chr(13)+\
"m=m.blur(0.2)"+chr(13)+\
"f1=FluxSmoothT().UnDot()"+chr(13)+\
"f2=UnFilter(0,80)"+chr(13)+\
"masktools_MaskedMerge (f1,last,m)"+chr(13)+\
"masktools_MaskedMerge (last,f2,m) ")
c= (Lb == true) ? Lanczosresize(c,width(c)-cr*2,round(height(c)*0.75)) : c
Addborders(c,(round(W-width(c)-cr*2)/2),(round(H-height(c))/2), \
(round(W-width(c)-cr*2)/2),(round(H-height(c))/2))
}

And this is my script:

Code:

LoadPlugin("G:\DVD\Filter\ADS\FluxSmooth.dll")
LoadPlugin("G:\DVD\Filter\ADS\MaskTools.dll")
LoadPlugin("G:\DVD\Filter\ADS\MPEG2Dec3.dll")
LoadPlugin("G:\DVD\Filter\ADS\UnDot.dll")
LoadPlugin("G:\DVD\Filter\ADS\UnFilter.dll")

Import("G:\DVD\Filter\ADS\ADS-HeaMod.avsi")

Mpeg2Source("G:\DVD\Endast film\The Passion Of The Christ\VIDEO_TS\New\Christ.d2v")
ADS(72, 72, 0, False)

I only wanted to inform you...

BTW Is there a way of using, not overlay overscan, but the other kind where you shrink the size of the frame? With ADS that is...

incredible · #24 09-20-2004, 04:42 PM

Try setting Blur(0.2) to Blur(1.5) and unfilter(0,80) to unfilter(80,80) .... Im testing right now and it seems that when doing this the details do even more come out .

Also I did find out that removedirt at default gots HEAVY settings according to motionthreshold! Do lower it to approx 50 if using it:

f1=removegrain(8).RemoveDirt(mthreshold=50,cthreshold=30,pthreshold=20)

I changed the "roberts" Edgemask architecture to "sobel" as it detects better the edges in a way I want.

incredible · #25 09-20-2004, 04:56 PM

With the rise of blurring the mask using (1.5) and rising the sharpener to 80,80 I got nice results!

Two Commands are compared

Code:

MPEG2Source("G:\KILLBILL_VOL1\killbill.d2v")
#
#
#### Testing in a horiz.stacked state "Ads(0,0,0,false)" and "undot.deen()"
#
Crop(round(width/2),0,0,0)
v1=ADS(0,0,0,false).subtitle("ads",0,20)
v2=undot().Deen().subtitle("undot/deen",0,20)
stackhorizontal(v1,v2)
#
#
### The function state of 9/20/2004 23.40h
#
function ADS (clip c, int "To", int "Bo", Int "Ov", Bool "Lb" ) {
To=default(To,0)
Bo=default(Bo,0)
Ov=default(Ov,0)
Lb=default(Lb,false)
Ov=Ov*8
W=width(c)
H=height(c)
cr=(W==720) ? 8 : 0
c= (To <= Ov && Bo <= Ov) ? crop(c,Ov+cr,Ov,-(Ov+cr),-Ov) : crop(c,Ov+cr,To,-(Ov+cr),-Bo)
c= ScriptClip(c,"nf = YDifferenceToNext()"+chr(13)+\
"m=masktools_EdgeMask(thY1 = 2, thY2 = 6, thC1 = 2, thC2 = 4, type= "+chr(34)+"sobel"+chr(34)+")"+chr(13)+\
"m=m.blur(1.5)"+chr(13)+\
"f1=removegrain(8).RemoveDirt(mthreshold=50,cthreshold=30,pthreshold=20)"+chr(13)+\
"f2=unfilter(80,80)"+chr(13)+\
"masktools_MaskedMerge (f1,last,m)"+chr(13)+\
"masktools_MaskedMerge (last,f2,m) ")
c= (Lb == true) ? Lanczosresize(c,width(c)-cr*2,round(height(c)*0.75)) : c
Addborders(c,(round(W-width(c)-cr*2)/2),(round(H-height(c))/2), \
(round(W-width(c)-cr*2)/2),(round(H-height(c))/2))
}

The pics:

J-Wo · #26 09-20-2004, 08:57 PM

wow incredible, that ads filter is quite something! I gave it a small test and I have to say that I like what I see. I do think I'm getting larger file sizes with it, and encoding takes much longer, but the results are impressive. I have two comments right now:

1) you mention to make sure the top and bottom borders (To and Bo) are mod8 because RemoveGrain() requires that. But I just finished encoding 22 episdoes of Alias where I removed 20 pixels from the top and bottom, and used RemoveGrain(mode=2) without any problems!

2) I have only one problem with RemoveDirt, and I've mentioned it many times. It can cause "twinkling" of small objects such as stars, background lights, etc. during camera pan scenes. To see what I mean, encode a scene where you get a pan of a night sky, or a pan of a skyscraper at night. With removedirt(edited) you WILL see this twinkling, but without it you won't. I really wish someone else could verify this for me cause I think I'm going crazy!

audioslave · #27 09-21-2004, 12:52 AM

Hmm, another thing I noticed about the Remove-filters -> During panning they (or one of them) creates stuttering in the movie. In other words, the panning is far from smooth. Seems like one (or more) of those filters isn't adapted to motion...

incredible · #28 09-21-2004, 02:47 AM

@ J-Wo & Audioslave

In the latest version of ADS() I used ...

RemoveDirt(mthreshold=50,cthreshold=30,pthreshold=20)

As the motionthreshold at higher settings than 50 does "just" kill details and some kind of a bad looking motionblur happens!

So the latest setting maybe solved that? Test are welcome.

At your point 2) you mention the problem of "twinkling". But one time related to RemoveDirt and at the end the other time related to Removegrain

Could it be you mixed up both filter names (like I did in the past) ?

Related to filesizes:
Yep, as said, this funktion gots its focus primary on detailenhance and quality. And as we gain from a luxury MB space on a DVD-R Media this doesent matter in case of DVDrebuilder, you agree?

But as also said, you can easely lower the unfilter(80,80) to some lower values like (30,30).

J-Wo, a cropping of 20 & 20 will result in a total cropping of 40 which is devidable by 8, thats why it works, so it was my mistake to say "cropping has to be mod8" as it should be "the final cropped movie area should be mod8". So as you live in NTSC land you got a height of 480. So 480-20-20 = 440 and thats MOD8 and thats the area processed by RemoveX()

Do a test by cropping for instance 26 and 26 and the error will occur.

incredible · #29 09-21-2004, 03:32 AM

Maybe this filter is something to test?
http://forum.doom9.org/showthread.php?s=&threadid=80834

J-Wo · #30 09-21-2004, 04:15 AM

Quote:

Originally Posted by incredible

As the motionthreshold at higher settings than 50 does "just" kill details and some kind of a bad looking motionblur happens!

So the latest setting maybe solved that? Test are welcome.

At your point 2) you mention the problem of "twinkling". But one time related to RemoveDirt and at the end the other time related to Removegrain

Could it be you mixed up both filter names (like I did in the past) ?

haha sorry for the confusion, I corrected my above post. I did always mean RemoveDirt, as I remember it is the 2nd filter in the combo RemoveGrain.removedirt.

I did do an encode with your newest settings of mthreshold=50. You are right that edges no longer move but I did see a brief scene in which some slight twinkling was noticed. I do believe it was less pronounced than removedirt at default settings. I compared ADS to my simple script of removegrain(mode=2).temporalSoften(2,4,6,6,2). My source was anime, but I'll try and find one that is live action to post.

Since I see you are testing with Kill Bill, maybe you could try the end of the movie which is a nighttime snow scene. I believe there are some shots which look up into the sky and pan the stars. Please test to see if you see the same problem as me!

Quote:

J-Wo, a cropping of 20 & 20 will result in a total cropping of 40 which is devidable by 8, thats why it works, so it was my mistake to say "cropping has to be mod8" as it should be "the final cropped movie area should be mod8". So as you live in NTSC land you got a height of 480. So 480-20-20 = 440 and thats MOD8 and thats the area processed by RemoveX()

Ahhhh thanks for that, makes a lot of sense. Is that ALWAYS the case when people say filter x needs mod8 height/width, such as BlindPP?

Dialhot · #31 09-21-2004, 04:23 AM

Quote:

Originally Posted by audioslave

Hi guys!
I just made a test with FluxSmooth (the new version) and UnDot

Be carrefull : this is what I used to use month ago and I dropped fluxsmooth because it is too heavy on the smoothing (it is at the origin of a lot of DCT blocks in the plain areas).

Quote:

Originally Posted by incredible

With the rise of blurring the mask using (1.5) and rising the sharpener to 80,80 I got nice results!

For sure the result are good. But can you tell us a little about file size ? Because in my only test (for the moment) of your function, the file was 18% bigger

incredible · #32 09-21-2004, 06:07 AM

I did an encoding test on that KillBill1 last night and got approx 5-7% bigger filesize

Encoded in 704width anamorph and I added 16px overscan to both sides and kept the orig size of the top/bottom borders (added afterwards again in that script).

More details do need more bits, so someone has to do his own compromise.

But shure, every movie is different.

Well we'll see, .... also it depends on HOW long is the movie and how much bits in avg are needed. Cause when doing DVDrebuilder workouts with the usage of ShrinkDVD (menue shrinkage) and Vobblanker (geting rid of Trailers) before I get that low Q values in CCE that this little rise IMHO you wont notice, like a jump from Q10 to Q 15. But this is just the beginning of the tests with this script. It was just the beginning of the usage of masktools, .... and I got some other Ideas totally different than this script.

Dialhot · #33 09-21-2004, 06:36 AM

Quote:

Originally Posted by incredible

I did an encoding test on that KillBill1 last night and got approx 5-7% bigger filesize

[...]
But shure, every movie is different.

To clarify the things, I was speaking of myt test on your very first version of the fucntion (the "old" one, that have 2 days now

). I did not try yet the last enhancement.

incredible · #34 09-21-2004, 04:46 PM

Why didnt someone advise me that a conditional/scriptclip environment isnt needed in my function :banghead: :banghead: :banghead:

Not needed that even more power consumpting scriptclip routine.

So I rewrote the function.

AND! A new Idea

Why do we denoise an area at full resolution IF there are (cause its masked!) NO necessary details ????

No reason.
So I pulled out my old idea of ONE-HALF-ONE, where I recognised that denoising is MUCH more efficient if the filter gots LESS pixels to handle.
http://www.kvcd.net/forum/viewtopic.php?t=6863

So AS I did mask that area to treat by a denoiser I easely can reduce its resolution to half, denoise it better and bring it back to its origin resolution WHERE I use bicubicresize using the parameters 1,0, which means an extra spatial softening, so no extra spatial softener is needed!!
Almost every important detail will be kept as the details have been masked before and will be added back to the image finally!

Second... IF reducing the to-denoise-resolution, I choose reducing the HEIGHT by half! Why? Because IF I do denoise temporal a noisy area at half WIDTH resolution and scale it UP again, I would suffer from enlarged temporal denoise artefacts when the camera moves sidewards in a scene and if a camera moves its the case that almost 95% of all movie pannings are done horizontally. So keeping the horizontal resolution when doing temporal jobs would be inevitable.
ALSO .... most Details seen by our eyes are vertically as we do see horizontal sharpeness more sensitive: Best Example Hairs! Mostly, hair does got horizontal details (hair falls down) and it doesnt suffer from blur if we do interpolate vertical as the details of hairs are separated by our eyes in their width.

So heres the new function and tests are welcome.

EDIT:
Latest version and explanation can be found here:
http://www.kvcd.net/forum/viewtopic....r=asc&start=52

....

The syntax is the same:

ADS (clip c, int "To", int "Bo", Int "Ov", Bool "Lb", int "sh", int "th")

You see the parameter "sh" is added which stand for sharpening, means "0" does not apply any sharpening/detail enhance and 10 is the most (you shouldnt go above 8 ).
And the detail threshold "th", that one stands for the mask/detail threshold

So ...

ADS(72,72,2,true,6,8 )

Means ..

- do pre-crop the original black borders at Top and Bottom by 72px each,
- and do apply and overscan of 2 (=16px on each side)
- true means that a Letterboxing from anamorph 16:9 to 4:3 will be performed
- 6 stands for the sharpening factor (max 10)
- the 8 stands for the detail threshold, means the more big grain is present, the bigger thresh should be set. On DVD clean materail like 4 and on trashy old VHS captures maybe about 12

I also tested it on a shitty old VHS capture using a "th" of 12 and it did wonders. On almost clean DVDs 4 - 6 is enough.

And its quite fast as temporalsoften is performing on the half of pixel information (well the speedgain could suffer on the other hand cause of the masking job)

rds_correia · #35 09-21-2004, 05:33 PM

Quote:

Originally Posted by incredible

Why didnt someone advise me that a conditional/scriptclip environment isnt needed in my function :banghead: :banghead: :banghead:

Damn it Inc.

You know I can't script otherwise I would have told you

Great findings buddy.
Keep those nice ideas coming

Cheers

J-Wo · #36 09-21-2004, 10:17 PM

holy crap inc! You are a scripting wonder. Gonna give it a try today, along with DVD-RB which I have yet to give a shot. Do you think this script is too complex to be used with that program? I thought I read somewhere that it doesn't like removegrain/removedirt...

Peter1234 · #37 09-22-2004, 02:12 AM

incredible,
I tried your new ADS function and it seems to work well. Only a quick test.

incredible · #38 09-22-2004, 02:41 AM

@ Rui

@ J-Wo

As you can see in the function the removedirt is NO longer present as its now temporalsoften which does the denoising and the parameters 1,0 in bicubicresize which do the spatial softening.

So the filter which are NON avisynth native are masktools, unfilter and undot.
I do think there will be no Problem with Rebuilder (well in my cases not)

@ Peter

What hasAnamorphic to do with "processing pixels in a frame"?

No Problem as I did tests on anamorphic "KillBill1" and non anamorphic captures.
If you want to keep it anamorphic just set the "lb" boolean to false.
The pre-cropping is up to you, no matter if its anamorphic or not.

Peter1234 · #39 09-22-2004, 02:47 AM

incredible,
I already edited comments out. I did not read your instructions carefully enough.

Peter1234 · #40 09-22-2004, 01:36 PM

incredible,
I did a few more tests on ADS and noticed that an anamorphic 16:9 720x480 input is changed to a 704x480 output by ADS. Since this function is designed to get the best possible image quailty, I would think that it would be best to keep all of the image information that is available. Perhaps you could add an option to not reduce 720x480 image width.