Avisynth: funny denoising routines
 

This evening happend something strange!

I got an analog capture from tv. Well it was just horrible (for me) full over with noise as you can see here:
(captured at 704x576 PAL using PicVideo YUY2 and safed as jpeg pic at 70% quality and 100% size)

http://www.digitalfaq.com/archives/error.gif

And as I everytime do, I tried a lot of combinated denoising routines and so on just to figure out something new for me.

So I tried one at 352x288 resized (bicubic) handled by pixiedust() using the loadpluginex.dll for AVS 2.5.2.
ok. at 352x288 pixiedust is fast enough and just for fun I added a scaler 352x576 and to scale up again to 704x576 .... "what???" do you think ... I too ;-)

Oh, I saw the image isn't that bad up sized as I though before.
Now ... even it was up sized using Lanczos I sharpened a little using asharp ... "Laczos & Asharp afterwards, ... WOW that will look funny oversharpen" but .... it wasn't such oversharpened.

8O 8O 8O 8O 8O

Well thats the pic after the downscale, pixiedust and upscale process;
http://www.digitalfaq.com/archives/error.gif

To me it does not seem like really down - upscaled and oversharpened, still details enough preserved (exept her "pullover") and no more noise when watching on a pc screen.


And heres the script:

#############################
LoadPlugin("C:\Programme\AviSynth 2.5\LoadPluginEx.dll")
LoadPlugin("C:\Programme\AviSynth 2.5\pllugins20\DustV5.dll")
# Still no MJPEGcorrect used
Avisource("H:\capture\capture.avi")
Limiter()
Bicubicresize(352,576)
Pixiedust()
Lanczosresize(704,576)
ConverttoYV12() # Stream was not interlaced
Asharp(1,4) ######### Must be crazy doing an asharp after Lanczos
Limiter()
#############################

I dont want to say its a perfect denoised 704x576 capture but
to show how funny the result comes out by using such a crazy script and how fast it is when pixiedusting @ 352x576.

EDIT: Sorry for late correcting ... the Pic above is denoised using the downsize to 352x576 and not 352x288 ! :hammer:

for me seems amazing!
8O
:!:

8O 8O 8O
Hell, might as well try the new VagueDenoiser instead of PixieDust :idea:

-kwag

I typed an error in the script when posting in here!

....... Edit: Sorry Mistake .. wrong size of high mentioned! Use as shown above in the script .............
Pic is real and an output of exactly that script.



If someone wants to try another Filter or something else, let me know, I keep the capture to be able to post results of other combinations here too.
So Kwag what are your recommended settings vor Vaguedenoiser???
For me personally I didn't come to a good solution using Vagueden.
But Ill try again and we'll SEE!

Now try the same thing with a DVD Source... This might be getting interesting! :roll:

Theoretically: As Captures are not that sharpen like DVD Sources it wouldn't be such an amazing effect like above when using this Routine on a d2v project from DVD.

BUT! What's theoretical??? I think at least in this case of a capture I only belive in practical outputs and how they do look like.
So stay tuned JellyGoose, this evening I will perform this "OneQuarterOne" script even on DVD d2v sources, interlaced DVD sources and also interlaced captures ... so that we'll see what's behind all this used on all sources. Cause I'm also like you more than interested what this will bring.

In case of interlaced captures I think it would be the best not to resize the orig 704x576 capture just to by bicubicresize(352,288). Ill try it also by eliminating the BottomFields and afterwards only resizing the width to 352.
And shure Ill do a test using Tv sources directly captured at 352x288.

wow....better for me incredible,...i have a old ati all in wonder and
it's good for captures in resolutions like 352x240(ntsc/pal-m)!
:D

I just tried reducing-to-VCD-resolution-and-then-upsizing-back on a TV cap..too smooth for me even without any filters :cry: Maybe an extra sharp capture would give better results or oversharpening before downsizing would help, but the detail loss was just too big.

When I compare those screenshots to the capture I've got, there's really not much difference in sharpness or detail so I don't know how the details were lost 8O

Same here. I can't duplicate those screenshots.

@incredible,
Are you SURE (positive!) you processed those screenshots with the posted script :?:
I just cannot see how a downsize to 352x288 and then upsize to 704x576 ( where details will be lost ) can look like that :!:
You can't sharpen what doesn't exist :!:

-kwag

During making some tests using other denoisers and sources I recognised an error and I hope my friends you didn't waste too much time.

Well as my mouth was too wide open when beeing amazed
there was a little confusion whe I did a wrong size correction today in this Thread I posted yesterday containing the correct sizes in the script!!!!
The picture above is right!! and made using a down-filter-upsizing BUT it was made with the script including a resize to 352x576!!! so its not a OneQuaterOne its a OneHalfOne, ... thats why its still so sharp by maintaining the vertical size and its sharpen factor in comparison to a horizontal size of a videostream. I was just shure I leaved the 352x288 I don't know why, maybe it was yesterday a little bit too late, sorry

Sorry for this 352x288 mistake Gentlemen!!!, .... so you have to resize to 1/2 DVD size NOT VCD size!



But as you still see the resizing from 768x576 to 352x576,adding the pixiedust and bring it back to 704x576 gave me a good denoised picture as shown above, thats fact.

And I imagine Jorel & Kwag when you posted your feedback you still read the correct script, when it was still at 352x576 posted. I hope so!!

Kwag ... I used VagueDenoiser instead of pixiedust() just to figure out.
VagueDenoiser also did make the picture less noisy thats true but the picture especially the surfaces still seem a little "disturbed" and therefore the picture isn't that calm as when pixiedust is used.
I also tried Golddust but Golddust seems to make the picture more smoother also the details.

And Jorel as you ended up with a "sonrisa" in your face by imagine that you can also receive that quality with your ATI All in Wonder, which now its not possible I can still say thats a big advantage to do the pixiedust and scale it up and sharpen it. I did a lot of tests the last hours to give a little 352x288 example to you ;-) using this kind of also a nice script.
I did encode a 352x288 MJPEG to 704x576 mpeg2 using the following script and burned it togehther with a mpeg1 352x288 version of the same source to a DVD-RW.

##############################
LoadPlugin("C:\Programme\AviSynth 2.5\LoadPluginEx.dll")
LoadPlugin("C:\Programme\AviSynth 2.5\pllugins20\DustV5.dll")
LoadPlugin("C:\Programme\AviSynth 2.5\pllugins20\MJPEGcorrect.dll")
#
Avisource("H:\capture\capture352x288.avi")
MJPEGcorrect()
Limiter()
#### no resizing down when already captured at 352x288
Pixiedust()
ConverttoYV12()
Bicubicresize(704, 576) # in this case of 288 high to 576 bicubicresize gives a better picture
Asharp(1.5,4)
Unfilter(60,60)
Limiter()
##############################

Original 352x288 upscaled on a Tv:
http://www.digitalfaq.com/archives/error.gif

An handled by the script "352x288to704x576"
http://www.digitalfaq.com/archives/error.gif

When watching on a TV there is an advantage in comparison to when the player resizes the stream to 1:1 PAL Screen size which is done by the mpeg1 352x288 sample.
Even with the lower CQ of the 704x576 or 352x576 (pixiedusted, upscaled and sharpened 352x288 origin.) it makes sense if your player accepts 352x576 or 704x576 KVCDs or if you want to burn your ATI All in Wonder Captures to KDVD.

Now Im testing the d2v Streams using the 720x576-352x576-filtering-704x576 script as shown and corrected again in my first posting of this thread.

:wink:
hey incredible,
for me "still" amazing.
in the pictures i can see big differences!

please post more results!
:)

ps:
my preference is for pictures with "girls"!
:lol:

OH now I recognize!! Ähm I really prefer Girls too!
This just was a fast screenie where you can see a lot of different things like flowers, textiles and skin ... it was not my basic intention to hardly find a man from the "blue oyster bar" ;-)

Well Jorel, I handled now a very bad orig. DVD - a nice price edition of "Black Rain" - nice price - nice noise!!! 8O Even not anamorph :evil:
I couldn't find a DVD containing a nice Girl - in a bad condition like this :D - to show the effect.
But maybe in a next "sharpen" thread we can do some tests on a Girls Lipgloss! Ill be there!

The orig "nice price, nice noisy" DVD:
(looks and moves like treated using Photoshops "sand" structure)
http://www.digitalfaq.com/archives/error.gif

And the "Wellness" ....
http://www.digitalfaq.com/archives/error.gif
Look at his face and the collar ... no noise (calm picture) and still details preserved.

On DVD Sources which do not contain such a heavy noise we should not perform this script. Cause this script here is for bad sources like captures and bad DVDs only. Its faster than performing the pixiedust on a 720x576 size but still not realtime. But I think in case of bad sources we can live with that if it ends up with such an effect.

Which script did you use on that one? Downscale to 352x576 then PixieDust() then upscale to 704x576 again?
How about the given compression by performing this compared to the current MA script ?

Try to use PixieDust directly on the 704*576 picture and I'm sure you will have the same results.

You are all stuck to this "downscaling/upscaling" fantastic results where all the benefits has to be to pixiedust.

My two cents.

@ Jellygoose
Exact!
As the script is shown above in the beginning of this thread.
You can also fine-tune a bit by adjusting the sharpen factor.

When downscaling a 720x576 DVD source you should first crop 8px at each side! to get to right 704x576 PAL, then you can perform the downscaling to 352x576. In this case you preserve the aspect ratio.
Well JellyGoose this filtering is still not Motion adaptive cause it was a result of a lot of tryings to really get rid of a noisy capture or a very bad produced commercial DVD. This script definitive gives me a CQ advantage when used on captures as the picture will be less complex after performing the pixiedust() at the 352x576 step.
BUT! If you got a good quality DVD I stll would perform the MA on such a source cause this OneHalfOne script maybe performs to heavy on well looking d2v streams.
I did a test on a perfect detail quality DVD source including a little noise and after this the picture looks too clean ;-) this sounds funny but in my eyes it appears to overtreated.
So its up to you what you choose to perform on diverse sources.
Just do a preview test by adding a Sampler() at the end of the script. ;-)


@ Dialhot
Two reasons why I do performe such a OneHalfOne scaling Ping-Pong:
I love Pixiedust(). for me its even very much better than Vague or others.
It cleans more, preserve more details, and keeps the picture more sharp than other deinoisers during its performing. I did a lot of tests with many denoisers, using this OneHalfOne technique and for shure the static-size conventional way.
The main reason is SPEED we obtain by downscaling so the filter performs faster.
The second reason is that pixiedust works even more effective (in my opinion) when using half DVD size, .. I don't know for shure but is does not seem that this pixiedust denoiser is resolution adaptive!

Yeah, I have already understood that. But it seems that other people have complete forgotten this point in your first post ! They all discuss about your resizing parameters and how you DS/US the image and they never focus on the pixiedust that is done in the meantime :-)

Oh ! If that's true, that's an other story then ! Can you post a comparison between full and half resolution result for pixiedust ?

@ Dialhot

First again the 704-352-704 using pixiedust (OneHalfOne Method):

(jorel this babe is again for you, but still without lipgloss)

http://www.digitalfaq.com/archives/error.gif


Here's the result when "very slow" pixiedusting WITHOUT downscaling to 352x576 before so just doing his job on real 704:

http://www.digitalfaq.com/archives/error.gif

Shure, .. even more details .. a bit too sharp cause of lanczos and ... much more "Disco" in the noise ... as I said ;-).
And .... at 704x576 it needs much more time to render!!!


Now lets have a look when our all friend Vaguedenoiser does his job on the "OneHalfOne" (704-352-704) script.

http://www.digitalfaq.com/archives/error.gif

The picture looks very "nervous" at the edges and also at the cleaned surfaces. It seems that he likes a little "artefacting" the cleaned surfaces 8O

I used the latest version I have: VagueDenoiser0.28.0.zip

Set to
VagueDenoiser(threshold=3,method=1,nsteps=6,chroma = true)

If I rise the Thresh it will shurely clean more but the picture gets more and more "uneasy"... well lowering the Thresh .. as we can assume just is for nothing ...
Vague at 704x576 working directly ... can't defenitely NOT handle such a noisy picture as my sample, but maybe there's a setings trick with vaguesenoiser ... if I change nextsteps ... Vdub quits emediately.


And here one shot used teh golddust() at 352x576 ... nice denoising as we know but .... image too soft :cry:

http://www.digitalfaq.com/archives/error.gif


And lets see when we all will reach the traffic limit of this free lycos webspace whre the pics are hosted :lol:

One thing! These outputs are only refering to the script I posted above!
This does not say anything about the "real" quality of these denoisers!
The pics handled by the denoisers which are shown here as you can see are from ONE source .. and as we know every source is different ...

One interesting filter might be MipSmooth. To kiss your bandwidth limit goodbye, try that one too. Make sure you have the latest Avisynth CVS binary and MipSmooth 1.0.

(as before still the script in the first post of this thread used, just converttoVY12() added if needed)

So Boulder ...

OneHalfOne using Mipsmooth "HQ" and "LQ", ... "VHS" was unviewable on that source using this script.
Maybe I set something wrong, cause its a new one from shodan.

http://www.digitalfaq.com/archives/error.gif


The Peachsmoother set to Peachsmoother():
(ca. 200 frames pre-roll)
http://www.digitalfaq.com/archives/error.gif

Very very interesting! Still very much noise after 200 frames BUT even its much noise, the noise seems and behaves very different than the noise kept by the others exept pixiedust. So I want to try some combinations.

One fast breaking da rules "heavy-shot"
Peachsmoother().Deen()

http://www.digitalfaq.com/archives/error.gif

Noise is "DEAD!" but shure in this fast combination much details are lost .. look at her collar ... and much more faster cause of AVS 2.5 plugIns in comparision to pixiedust!

I've been using Mipsmooth(preset="MovieHQ") and I'm very satisfied with the results! Don't ever use the "UGLY" preset though! :lol:

http://www.digitalfaq.com/archives/i...2003/10/10.jpg

:lol:
What the hell is that ren :mrgreen:

-kwag

hehehe, just kidding! (I'll remove the distraction :lol: )

Seriously though I haven't tried the dust filters since using Avisynth 2.5x- that GoldDust sample didn't look half bad to me

ren

8O 8O

hey friendal,
the dentifrice.dll filter clean "too much" the "smile"!
i can't see the tooth.
:rotf:

While exchanging PMs with incredible, I did a *lot* of encoding yesterday. I finished one big VHS restoration and encoded two episodes of different series. I used the 704 - 352 - 704 method on both, the scripts are at the bottom of this post.

Here are the screenshots:

TV capture, original
http://www.digitalfaq.com/archives/error.gif

TV capture, filtered and encoded, avg bitrate was 1820kbps.
http://www.digitalfaq.com/archives/error.gif

VHS tape, original (horrible colours, I know :wink: )
http://www.digitalfaq.com/archives/error.gif

VHS tape, filtered
http://www.digitalfaq.com/archives/error.gif

I should mention that the bloke in that picture is not me. It's an old friend who has a dual personality - that's his Billy Gibbons (ZZ Top) character.

Despite the age of the tape and the fact that it's a second generation copy, the VHS restoration looks 8O after the filters have done their job.

Here are the scripts (some color related tweaks were applied to the VHS one along with the one shown):

TV capture
I encoded this one as progressive, thus saving some bits, since interlaced encoding didn't give any smoother motion.

VHS
The VHS cap was encoded as interlaced.

Hey Boulder!

First what capture Card are u using?
It looks very amazing even unfiltered!

And by using the OneHalfOne Routine it looks even more phantastic.

And the interlaced one ....
Did you also tried the last version of the script I send to you by using peachsmoother instead of Deen()?
Its even better and you don't have to change the colorspace:

The peachsmoother engine gives to me AMAZING results on analog captured interlaced material even using default Peachsmoother().
(here still without macroblock optimal based letterboxing)

http://www.digitalfaq.com/archives/error.gif

I used this on a 704x576 capture to do an example encoding/mpeg2/interlaced and burned it in 480x576 svcd mode.
The interlacing was very well preserved not mention the quality on tv screen by the benefit of peachsmoothers adaptive noise routine. 8O
It looked almost the same when using pixiedust() but MUCH more faster!
Depending on the source like using this gives me about 10-20% more CQ than just only resizing the capture in its original noisy condition!


So we should do a test and change on the "frame-based"-script in the first post of this thread ... or in th first TV-Capture script of boulder

pixiedust()

to

TemporalSoften(2,5,8,15,2).peachsmoother().Unfilte r(60,0)


@rendalunit

I can't see your MipSmooth result .. would be very interesting to me to see it, cause everybody says good things about it.

so does someone have a overall script for tv capture as I'm readin through all these posts with a blank exprection on me face wonderin what u lot are on about hehehehe.

The OneHalfOne Routine as shown in the first post of this thread is for 704x576 DVD encodings.

If you just want to encode to a format which is not fullscreen like 352x576 or 480x576 you can choose theese PAL scripts as i figured out yesterday together with Boulder:

On progressive Captures:

Avisource("D:\Avisource.avi")
Limiter()
GripCrop(480, 576, source_anamorphic=false, dest_anamorphic=false) # your destination values here
GripSize(resizer="BicubicResize")
TemporalSoften(1,5,5,5,2).peachsmoother().Unfilter (60,0)
GripBorders()
Letterbox(16,16,16,16)
Limiter()


On Interlaced captures:


Avisource("D:\captures\capture_test.avi")
Limiter()
video = SeparateFields().BicubicResize(480, 288) # calculate your size parameters in Moviestacker using INTERLACED settings!! = 1/2 vertcal size!!
evenfield = SelectEven(video).TemporalSoften(1,5,5,5,2).peachs moother().Unfilter(60,0)
oddfield = SelectOdd(video).TemporalSoften(1,5,5,5,2).peachsm oother().Unfilter(60,0)
Interleave(evenfield,oddfield)
Weave()
Letterbox(16,16,16,16)
Limiter()

In case of heavy noise change the Temp.Soften Values to
TemporalSoften(2,5,8,5,2)

well I captured 352x576 pal and below is what the film looks like after capture.

http://www.digitalfaq.com/archives/error.gif

when playin normally in mediaplayer looks excellent quality, I played it in powerdvd and there is noise as seen below.

http://www.digitalfaq.com/archives/error.gif

First.. look at my last posting which I edited including now the latest scripts for destination encodings BELOW 704 PAL width size.

Well to me it seems that your capture is really done using 352x288!
Are you shure you captured using 352x576???

When not capturing at full size like 704x576 and scaling it afterwards shure the noise without filtering will be "scaled" too ... and that its how it seems to me when looking at your samples. Shure a capture looks good when viewn at 352x288 as in your first sample.
(By the way your captures are good .... you already put a filter on that??)

I see what u mean about the size pf capture, I checked settings it has 352 576 selected but for some reason its 352x288 I tried a full pal format caputre test but I lost loads of frames I closed all progs that I could close before hand, I used to capture 640x480 for little clips I used to capture from a programme but I was told for pal encodes its not advised. So any help is welcomed.

Some advices to avoid dropped frames:

- kick out ALL windows tweaks!
- Defragment your capture HD
- use a not CPU intensive codec like huffyuv or PicVideo-mjpeg! NOT DVIX!!
- find out how your capture appl. accepts finally 352x576
or
- buy a faster CPU :oops:

well I use morgan mjpeg and my cpu speed is 1.1ghz guess its just a case of tweekin about maybe who knows.

It's a Hauppauge WinTV Theatre, BT878 chip. I use the Hauppauge WDM drivers since the BTWinCap drivers don't produce true stereo sound. The quality is really not that good when you play the clip. On a single frame it may look good but there's a good amount of analog noise there.

I didn't run that on the VHS since it produced that view-from-behind-a-dirty-windshield phenomenon. When there's lots of noise such as in VHS sources, it often makes the noise static so that you see the specks and the noise but it doesn't move from frame to frame. The filter thinks that the noise is part of the image itself. I think that this happens because the filter has been designed to keep the details - sometimes it just gets noise and details mixed up :wink:


I agree. The Dust filters are too slow for any longer capture.

Boulder, before we'll get to a actual final recommendation (cause Newbies will ask for it - and the whole thread as Dialhot saw it right, is getting a bit cunfused when someone starts reading it now - we should also test the prediction capabilities when using Peachsmoother().
It seems to me if Peachsmoother needs at least 200 frames preroll that would cause wrong predictions when for example using Cq-Matic ... If I'm right CQmatic takes 3000 Frames per prediction turn and "incorrect" denoised 200 frames at beginning mens percentual a lot within 3000 Frames in a prediction :?:
Well we'll figure out this, or maybe you already got experiences on this.

I haven't tried any predictions, all the encodes I've done have been as multipass VBR with CCE.

PeachSmoother shouldn't need that big a preroll, and it's even less noticable when you set the noiselevel and baseline settings manually.

@Incredible,

Would you happen to have the link for the LoadPluginEx.dll? I can't find it anywhere :roll:

thnx

Here it comes:
http://www.incredible.de.tf/Downloads/LoadPluginEx.dll

:wink:

Everytime I try to load this LOADPLUGINEX.DLL that I downloaded from your link, I get an error message that it is a bad image and check my installation diskette 8O :?: :roll:

Well i did a test and the download works well :!: :?
Did you try using "save target as..." (right mouse click)???

Anybody else got the same problem??
(Cause the link there is already three weeks old)

But just in case I stuffed it as a .rar archive which you also can try...
www.incredible.de.tf/Downloads/LoadPluginEx.rar

Incredible, is that a WinRAR archive? I get an error that it is unrecognisable or corrupt with WinRAR.