digitalFAQ.com Forums [Archives] - Avisynth: Filter to clean up a noisy source with MA script?

digitalFAQ.com Forums [Archives] (http://www.digitalfaq.com/archives/)

- Avisynth Scripting (http://www.digitalfaq.com/archives/avisynth/)

- - Avisynth: Filter to clean up a noisy source with MA script? (http://www.digitalfaq.com/archives/avisynth/6022-avisynth-filter-clean.html)

audioslave

10-09-2003 05:03 PM

Filter to clean up a noisy source with MA script?

I'm trying to encode "Deliverance" and since it's an old movie (1972) it's quite noisy. Hence I'm having trouble with getting a high CQ value. What filter can I use to clean up the movie? Please, let me know what filter to use, settings and placement in the script.

incredible

10-10-2003 04:23 AM

In the MA Script you can find a Line including STMedianFilter.
Just change it to Deen()
(download Deen() via Warpenterprises)

or change to
TemporalSoften(2, 10, 10, 15, 2)
(No Filter download necessary, included in MPEG2Dec3.dll)

Its heavy but you will kill noise. On the other Hand many details will be lost, for exampe killed structures in Hairs or flat surfaces in Skins.

TemporalSoften for me is the best compromise cause
- Its fast
- Its the Filter which keeps the most details when doing a "normal, not to heavy" denoise

But if set to heavy, the Picture in Surfaces will look like viewed through a Glass with "static" Dots. So watch out!

So use MA like this:

Code:

## Defined Variables and Constants ## 

# 

MaxTreshold = 1.50 

nf =* 0 # Current frame. 

# 

#### 



## Main section and static filters ### 

# 

Mpeg2Source("Your_D2V_Source_Here") 

#  

Limiter() 

asharp(1, 2) 

GripCrop(Your_GripCrop_Parameters_Here) 

GripSize(resizer="BicubicResize") 

TemporalSoften(2, 10, 10, 15, 2) 

undot()

MergeChroma(blur(MaxTreshold)) 

MergeLuma(blur(0.1)) 

# 

# 



## Linear Motion Adaptive Filtering ## 

# 

# ( Portions from AviSynth's manual ) - This will apply temporalsoften to 

# very static scenes, and apply variable blur on moving scenes. 

# We also assign a variable - and this is why a line break is inserted: 



SwitchThreshold = (Width<=352) ? 4 : (Width<=480) ? 3 : 2 

ScriptClip("nf = YDifferenceToNext()"+chr(13)+ "nf >= SwitchThreshold ? \ 

unfilter( -(fmin(round(nf)*2, 100)), -(fmin(round(nf)*2, 100)) ) : \ 

TemporalSoften( fmin( round(2/nf), 6), round(1/nf) , round(3/nf) , 1, 1) ") 



# 

# 

# 



GripBorders() 

#LetterBox( Your_Values_Here ) # Depends on situation. Use MovieStacker! 

Limiter() 



# 

# 

## Functions ### 



function fmin( int f1, int f2) { 

* return ( f1<f2 ) ? f1 : f2 

} 

* 

# 

####

Well I tried a lot on "old" sources an even if TemporalSoften is already used in the MA Routine it gave me a very good quality Stream with more CQ an keeped Details when also placed in front of the MA routine too.

If you lost to much details just change to
TemporalSoften(1, 8, 8, 15, 2)

The undot() is put behind TemporalSoften to eliminate a little bit the mentioned "viewed through a Glass with static dots" effect.
The Threshold of Asharp is set to (1, 2) to avoid sharpen more the noisy surfaces in the movie.

BUT!

If someone can tell me how to configure TemporalSoften in the MA Routine that it handles the "static" Scenes more heavy with individual parameters ... let me know! Makes more sense.
Just to avoid the double filtering.

Like this?

Code:

TemporalSoften( fmin( round(3/nf), 6), round(2/nf) , round(3/nf) , 1, 2) ")

Can someone explain detailed how the mathematical calculation routine of the MA Part works.

Well I'll try some settings at Weekend.

kwag	10-10-2003 06:38 AM

I would leave the "temporal" values out completely in STMedian and increase the "spatial" values :!:
Use only the "spatial" values, and not "a-la-adaptive", but use it statically.
You want to kill noise spots, which are (ideally) different from frame to frame.
So what is really needed is strong "spatial" filtering. Not "temporal" filtering.

-kwag

audioslave

10-10-2003 07:34 AM

Thanks guys!

@kwag
Could you please point me int the right direction on the spatial filtering? Shall I keep the temporal values, and what are good starting values for the spatial filtering? :roll:
This movie (Deliverance) is in bad need of a clean up. It's very noisy/pixelated.

incredible

10-10-2003 07:45 AM

Quote:

Originally Posted by kwag

I would leave the "temporal" values out completely in STMedian and increase the "spatial" values :!:

You're exactly right, like mentioned above it's the way it has to be.
:banghead:

kwag	10-10-2003 08:20 AM

Quote:

Originally Posted by audioslave

@kwag
Could you please point me int the right direction on the spatial filtering?

Look at the MA script. The line: STMedianFilter(8, 32, 0, 0 ) has the temporal values set to 0. So you can play with the "spatial" values ( 8, 32 ) by increasing them, and test your results.

Quote:

This movie (Deliverance) is in bad need of a clean up. It's very noisy/pixelated.

Noisy or pixelated :?:
It's two completely different things :!:

-kwag

kwag	10-10-2003 08:22 AM

Quote:

Originally Posted by incredible

You're exactly right, like mentioned above it's the way it has to be.
:banghead:

;)

audioslave

10-10-2003 09:44 AM

Oh, sorry. I ment to say noisy. :oops:

audioslave

10-10-2003 09:46 AM

What are my options on STMedian filter? I mean what's the range of values for parameter 3 and 4, and what do they do?

incredible

10-10-2003 10:07 AM

STMedianFilter(S_FilterThreshHold, S_EdgeThreshHold, T_FilterThreshHold, T_MotionThreshHold)

Tom Barry:

Quote:

Larger values cause more filtering but more artifacts. Parm descriptions:

S_FilterThreshHold - don't spatial filter pixel if it would change it more than this
Larger values may cause loss of fine line detail but it's not extremely sensitive.

S_EdgeThreshHold - don't spatial filter pixel if on an edge where vals change more
than this. It seems fairly forgiving, so it can be raised a lot if desired.

T_FilterThreshHold -don't Temporal filter pixel if it would change it more than this
Large values will cause ghosting. Don't set over 5-10 to avoid ghosting.

T_MotionThreshHold don't Temporal filter pixel if uncompensated motion vals change more
than this. Don't set over about 10 to avoid ghosting.

muhali3

10-11-2003 03:13 PM

does deen act as a spatial-temporal smoother. or just spatial?
what exactly does it do?

i've read the documentation that came along with it, but it doesn't exactly say what it does.

Thanks
:twisted:

Dialhot

10-11-2003 04:04 PM

Deen is a Spatial only filter.

audioslave

10-13-2003 11:11 AM

Well, now I've tried encoding with Deen() but I only gain 2-3 in CQ. "Deliverance" is about 1:45 long and I usually get a CQ around 70 for this kind of movies. In this case the CQ stops at around 56 :!:
Can someone please post good settings for STMedian (spatial values) to use for low, medium, and high noise sources?

Dialhot

10-13-2003 11:17 AM

Are you sure your problem doesn't come from a source too sharpen ?

It's strange that you gain so little CQ on the video. Deen is quite heavy in noise removing. Try this : deen("a2d",2,10,12)
(note: oops, I said an error last time, by default deen is a Spatio-Temporal filter using a 3d matrix. For Spatial only filtering you have to use the line suggested above).

Finally, I can suggest you eDeen that is heavier than deen.

Ps:For the values of StMEdian, why don't you do some test and give results there ?

audioslave

10-13-2003 11:22 AM

@Dialhot
I don't know. I'm using the MAscript and it has always worked before, on other movies. Maybe I could try increasing the LumaBlur? Or do you have any other suggestions?

Dialhot

10-13-2003 11:24 AM

I edited my post while you were answering. See the suggestion I do in it :-)

audioslave

10-13-2003 11:26 AM

Where can I find eDeen?

EDIT: Now that I think of it the DVD seems like it is over sharpened. Almost like when you apply too much sharpen filter in Photoshop, if you know what I mean?

Dialhot

10-13-2003 11:29 AM

http://ziquash.chez.tiscali.fr/

audioslave

10-13-2003 11:33 AM

Thanx :wink: :D

Dialhot

10-13-2003 11:34 AM

You can soften it with unfilter (that is its original purpose). Or as you suggested raise a little the blur on the luma.

muhali3

10-13-2003 03:46 PM

i would also recommend Unfilter, on the website it shows sample pictures of terminator 2 with unfilter, so you can check that out. :wink:

audioslave

10-13-2003 04:55 PM

Grateful for all help but what I really need is a filter for removing noise. As I said before, I've tried Deen and now also eDeen but none of them where any good on my noisy DVD source. Is there any other filter I could try for cleaning up my source?

muhali3

10-13-2003 07:10 PM

VagueDenoiser or Wavelet Noise Reducer have very good results, except those take a long time.

Try the new Mipsmooth, its in the avisynth forum if you want.

If you're running AVS2.0, pixiedust is good.

J-Wo	10-24-2003 01:18 AM

Hey guys. I tried out some of the tips here and found that Deen() worked best as a replacement for stmedianfilter. My DVD source is from the Alias TV series, and certain scenes appear grainy/pixelated/noisy (I dont' know how to define it!). But here are some screen caps to show you:

original 720x480 screen cap from DVD
http://www.digitalfaq.com/archives/error.gif

704x480 KDVD, no filtering, encoded with CCE @ Q 30
http://www.digitalfaq.com/archives/error.gif

704x480 KDVD, original MA script
http://www.digitalfaq.com/archives/error.gif

704x480 KDVD, MA script with Deen()
http://www.digitalfaq.com/archives/error.gif

So as you can see, my source is grainy to begin with, and then encoding at Q 30 introduces some more artifacting. It's hard to tell from the screen caps but the artifacting looks really ugly when the original MA script (without Deen) is applied. The artifacts dance around and really stick out at you. Deen helps to smooth it all out, but again I don't think you can see that well from these caps. Deen has the added benefit of really compressing this noisy sample. Encoding is slowed down a bit, but not as much as VagueDenoiser would.

I tried fooling around with the first two values in Stmedianfilter but wasn't able to achieve the desired effect. Any suggestions?

incredible

10-24-2003 03:04 AM

Nice :wink:

But it seems you used a small jpeg compression quality to obtain small files to post in here, by this its not easy to see an objective result cause of added jpeg artefacts :cry:

Could you change the pics by the same ones but less compressed therefore less artefacts? :)

I tried also a lot by just changing the Stmedian Line to Deen().
But you have to set the Deen() to spatial settings if you follow Kwags recommendations. For example Deen("a2d",2,12,15).
Well also try to set deen() optimal for the video stream individually.
I'm still not happy by using Deen(xxxx) in MA cause the last two days I was armed with some MA spatial mods to attack bad DVD Sources like my "Black Rain" Sample in the "How Funny" Thread. I also Tried VagueDenoiser using diff. settings. We'll see ... :D

kwag	10-24-2003 09:35 AM

Quote:

Originally Posted by J-Wo

I tried fooling around with the first two values in Stmedianfilter but wasn't able to achieve the desired effect. Any suggestions?

STMedianFilter(3, 3, 1, 1 )
Let me know how it compares, instead of deen.

-kwag

ovg64

10-24-2003 09:39 AM

Deen is a very good spatial filter, i have been using it since the begining we started using Avs 2.5, i wonder Y it took you guys so long to find out about it :twisted: BTW you can also use it as a smoother just can not remember setting since Im not in my PC. :roll:

ovg64

10-24-2003 09:41 AM

Hey Karl what are you doing sneaking in front of me :mrgreen:

incredible

10-24-2003 10:18 AM

Be careful "Deen()" set to default like this is not spatial-only!
If you want spatial only choose "a2d" method!
Example: Deen("a2d",2,10,12)
This is mentioned in Deen's Readme doc.

J-Wo	10-24-2003 11:14 AM

Wow, okay so this is a tough choice for me. There wasn't a big difference between Deen() and Deen("a2d",2,10,12) in my test clips. They both did a good job of smoothing out and blending the grainy footage. However upon closer inspection I realized that the grain is in both the background AND the foreground, so both versions of Deen lose detail in the foreground images. On the other hand, STMedianFilter(3, 3, 1, 1 ) did a good job of cleaning up the background, making the artifacts not dance around as much as with the original StMedianFilter(8,.24,0,0). The other deciding factor is encoding time and amount of compression. Here are my results for a 2:30 min clip:

Deen()
encoding time = 4:07 min, size = 43,509 kb

Deen("a2d",2,10,12)
encoding time = 4:16 min, size = 43,130 kb

STMedianFilter(3, 3, 1, 1 )
encoding time = 3:18 min, size = 48,010 kb

StMedianFilter(8, 24, 0, 0)
encoding time = 3:29 min, size = 48,156 kb

On my mediocre 20" TV, Deen() actually looks fine to me but I'm sure I'd notice the difference in detail on my parents 47" widescreen HDTV. So my choice might be STMedianFilter(3, 3, 1, 1 ) in the end. However since Deen() does a better job at compressing maybe I can lower the Q factor in CCE (n.b. inverse relationship from CQ in tmpgenc) to compensate.

Any thoughts?

P.S. I've recompressed my jpegs from above to be higher quality. Still not sure if you can tell a difference or not...

kwag	10-24-2003 01:07 PM

Quote:

Originally Posted by J-Wo

On the other hand, STMedianFilter(3, 3, 1, 1 ) did a good job of cleaning up the background, making the artifacts not dance around as much as with the original StMedianFilter(8,.24,0,0).

I know :cool:
That's why I gave you those values :)
I just haven't updated the optimal script, but those are the new values for the script.
A smaller radius for spatial filtering, and the minimal value for temporal.
STMedianFilter(3, 3, 1, 1 )
Does wonders on background noise, without speed penalties.

-kwag

kwag	10-24-2003 01:08 PM

Quote:

Originally Posted by ovg64

Hey Karl what are you doing sneaking in front of me :mrgreen:

Long time no see ovg :D
Welcome back :cool:

-kwag