Avisynth: Removedirt Stopped?

Dialhot · #1 09-20-2004, 03:36 AM

Hi all,

I continued to test Removedirt filter this we. And I stopped to use it. The following picture will tell you all.

These are the frames 518, 519 and 520 of a sample I used. First 3 are done with RemoveGrain().Deen(), the 3 others with RemoveGrain().RemoveDirt(). Look at the ark above Gibson's head.

I think I do not need to tell you how awfull was the watching of the sequence.

Zyphon · #2 09-20-2004, 06:11 AM

I agree it does look rough around the edges above Gibson's head.

Thanks for the info Phil.

jorel · #3 09-20-2004, 06:35 AM

Quote:

Originally Posted by Dialhot

Hi all,

I continued to test Removedirt filter this we. And I stopped to use it. The following picture will tell you all.

These are the frames 518, 519 and 520 of a sample I used. First 3 are done with RemoveGrain().Deen(), the 3 others with RemoveGrain().RemoveDirt(). Look at the ark above Gibson's head.

I think I do not need to tell you how awfull was the watching of the sequence.

can you post the links to open each picture?
this way is better Phil,
we can open each picture in a new page and it's easy to compare!

danpos · #4 09-20-2004, 07:05 AM

@Phil

It´s very good your tests about Remover´s filters. It´s clear in the sequence (frames) that roughness effect on the edges (ark) resulting of RemoveDirt filter application is more strong than Deen one. Here in Brazil we be accustomed to say that you "matou a cobra e mostrou o pau"

(I don´t to say this stuff in english)

Cheers,

Dialhot · #5 09-20-2004, 07:25 AM

Quote:

Originally Posted by jorel

can you post the links to open each picture?

1/ when you anwser to my post by clicking on the "quote" button, you have all the links that are in the quoted message.
2/ by doing a right click on each image, the context menu allow you to open the image in a new widow (or new tab).

BTW, if it is easier for you :
http://pagesperso.laposte.net/philippeperez/Deen1.jpg
http://pagesperso.laposte.net/philippeperez/Deen2.jpg
http://pagesperso.laposte.net/philippeperez/Deen3.jpg
http://pagesperso.laposte.net/philip...emoveDirt1.jpg
http://pagesperso.laposte.net/philip...emoveDirt2.jpg
http://pagesperso.laposte.net/philip...emoveDirt3.jpg

incredible · #6 09-20-2004, 07:27 AM

The pics "still" just say what happens if just removedirt() is used.

Do we just use "convolution3d()" or just "Temporalsoften()" ??? I dont think so.
So at least we should test diff. settings, thats all I wanna say.
And if that doesnt work, Im the last one who doesnt say that we can use other temporal filters

Dialhot · #7 09-20-2004, 07:31 AM

Quote:

Originally Posted by incredible

The pics "still" just say what happens if just removedirt() is used. Do we just use "convolution3d()" or just "Temporalsoften()" ??? I dont think so.

???
What do you mean by that ???

Removedirt removed TOO MUCH pixels. I don't see how using other filters before or after it can change this behaviour.

BTW; using "just" C3D or deen is the same (Deen is a C3D).

Major · #8 09-20-2004, 07:33 AM

So what is the best filter to remove the dirts (dots or streaks).
Yesterday i saw other filter that do the same that removedirt, is the filter Despot based in Conditional Temporal Median Filter v.0.93

http://bag.hotmail.ru/despot/despot.dhtml

Dialhot · #9 09-20-2004, 07:35 AM

Quote:

Originally Posted by Major

So what is the best filter to remove the dirts (dots or streaks).

The answer change month after month

And that is a good news indeed.

Note: Despot and RemoveDirt do not do the same things. Despot literraly "restore" pictures that have big defaults due to age. Removedirt is more focussed on "normal" video noise you often find in digital sources, DVD included.

incredible · #10 09-20-2004, 08:00 AM

Quote:

Originally Posted by Dialhot

Quote:

Originally Posted by incredible

The pics "still" just say what happens if just removedirt() is used. Do we just use "convolution3d()" or just "Temporalsoften()" ??? I dont think so.

???
What do you mean by that ???

Removedirt removed TOO MUCH pixels. I don't see how using other filters before or after it can change this behaviour.

BTW; using "just" C3D or deen is the same (Deen is a C3D).

I meant that we at least should try some parameters

Deen? C3d?? I know, but I refered to convolution3d as we also do use that filter using specific settings.

Dialhot · #11 09-20-2004, 11:05 AM

New tests with the ADS function developped by Inc (from which I remove the resize part as it it already done by other parts of my testing script).

Code:

function ADS (clip c) {
c= ScriptClip(c,"nf = YDifferenceToNext()"+chr(13)+\
"m=masktools_EdgeMask(thY1 = 2, thY2 = 5, thC1 = 1, thC2 = 5, type= "+chr(34)+\
"roberts"+chr(34)+")"+chr(13)+\
"m=m.blur(0.2)"+chr(13)+\
"f1=dremovegrain(8).RemoveDirt(mthreshold=125,cthreshold=30,pthreshold=20)"+chr(13)+\
"f2=unfilter(0,80)"+chr(13)+\
"masktools_MaskedMerge (f1,last,m)"+chr(13)+\
"masktools_MaskedMerge (last,f2,m) ")
}

I just removed the "RemoveGrain().Deen()" and put ADS().

No problem so far but the drawbacks is :
- with removedirt : sample size = 18001 KB
- with deen : 17568 KB
- with ADS : 21744

Things are under developement.
http://pagesperso.laposte.net/philippeperez/ACS1.jpg
http://pagesperso.laposte.net/philippeperez/ACS2.jpg
http://pagesperso.laposte.net/philippeperez/ACS3.jpg

jorel · #12 09-20-2004, 12:05 PM

Phil,
only an idea.

why don't use scenes seamless like i use to see more differences...

from vob:

from script:

don't show more details for comparisons

Dialhot · #13 09-20-2004, 12:14 PM

Quote:

Originally Posted by jorel

don't show more details for comparisons

I did not peek a scene from nowhere; I took that scene because it gave me problem.
The idea is not to compare the details of the outputs, but to show that removedirt has serious problems and can lead to disaster if we don't care.

I took 3 frames that are following in the scene to let you imagine how it can be to see all the action that last for 4 or 5 seconds on the screen ! All the frames are like this : the ark seems to blink !

incredible · #14 09-20-2004, 12:28 PM

Phil I do see you called all your 3 pics ADSx.jpg ....

But on the other hand you talk about

1 - Removedirt (just removedirt() right??)
2 - Deen (just Deen() right??)
3 - ADS (not modded except the resizing right??)

Only just in case I dont get confused.

Anyway - at least one of the Pictures uses ADS incl. Removegrain, right?
So that Detail killing at the "arc" above Mel's head is now solved?

Quote:

I just removed the "RemoveGrain().Deen()" and put ADS()

Ok that means to the other 2 comparisons beside ADS() you didnt apply any sharpener? (btw. all do look sharp the same but in my cases not when I did testings

) And sharpening (in one dirction only) is done via ADS() to enhance details (as you shurely know), which results for shure in a loss of compression. So if you dont need a sharpener, just put a "#" before the SECOND "masktools_Maskedmerge......" line and no sharpener will be applied, as f2 is the sharpening processor in ADS().

I for my personal taste like to bring out more details as for instance the usage of DVDrebuilder offers you that much Diskspace for your encode so we "could" focus more on quality as "compression" will be a bit less important in case of DVD-Rebuilder?

Also I still will test some other combinations of spatial and temporal denoiser combinations (I would apriciate your participating) as this fuction is still mega beta

The main idea of that function is just the simple use of edge masking, the further choice of filters we can easely find out in testings

(keep in mind my advice of the usage of Deen() in a conditional environment as I figured out that this should be avoided)

I think Ill apply this evening a masking edge threshold so you can simply determine the threshold of the masked edges.
Also a nice "showmask" boolean would be nice where you can see HOW the mask is builded ?

EDIT: Quoting you from the other thread:

Quote:

Originally Posted by Dialhot

Note : I used ECLCEE with no problem even with the parameters used in RemoveDirt.

So that means the Scriptclip environment solved the problem with EclCCE??

[/quote]

Major · #15 09-20-2004, 01:50 PM

I have a question, I have applied removedirt for a clip black and white and I have noticed a slight cleaning, I would like to clean it more

I have put on the script: Removedirt (grey= true, pthreshold= 10, mthreshold=180)

1. must I put Removegrain before RemoveDirt???

2. my values are well or not?? or i must put the default values ??

Dialhot · #16 09-21-2004, 04:42 AM

Quote:

Originally Posted by incredible

1 - Removedirt (just removedirt() right??)
2 - Deen (just Deen() right??)

WIth a RemoveGrain just before, as already said

Quote:

3 - ADS (not modded except the resizing right??)

Yep.

Quote:

Ok that means to the other 2 comparisons beside ADS() you didnt apply any sharpener?

I NEVER use any sharpener. NEVER like in N.E.V.E.R.
I tested your function just because you asked me too.

Quote:

So if you dont need a sharpener, just put a "#" before the SECOND "masktools_Maskedmerge......" line and no sharpener will be applied, as f2 is the sharpening processor in ADS().

I will try.

Quote:

I for my personal taste like to bring out more details as for instance the usage of DVDrebuilder offers you that much Diskspace for your encode so we "could" focus more on quality as "compression" will be a bit less important in case of DVD-Rebuilder?

Yes it is. But I fear that with such a diff in file size, the Q factor that will be used for the job will be increased by too much points to obtain a good result.

Quote:

The main idea of that function is just the simple use of edge masking, the further choice of filters we can easely find out in testings

(keep in mind my advice of the usage of Deen() in a conditional environment as I figured out that this should be avoided)

As I said, I just tested in because you asked me too. I didn't intent to start a real competition between it and Deen now. This will come later

Quote:

So that means the Scriptclip environment solved the problem with EclCCE??

[/quote]
It seems so !

Boulder · #17 09-22-2004, 03:19 PM

Phil,

did you try lowering the thresholds? I think that the key lies there. See the 'official' RemoveDirt thread:

http://forum.doom9.org/showthread.php?s=&threadid=70856

EDIT: You could also replace RemoveDirt by ReduceFluctuations(limit=5). It's undocumented but it's there, works a bit like the Dust family. Lower the value for limit for less denoising.