Applying AviSynth filters to capture, FPS cost?

[sprotte]

Hello everyone

I've been following lollo2's great advice on how to proceed with a first ever video capture. Many thanks (!) for your script and its implicit hints towards deinterlacing, denoising, sharpening.
To be honest, I'm absolutely new to this field and this has helped me no end after having read just some other posts on which device to use for capturing.

(Long story short - I was asked to convert VHS to digital video, for pure preservation purposes, and lordsmurf's comment on the "The Best Easy Way to Capture Analog Video" Youtube video has directly brought me here. )

Quote:

Originally Posted by lollo2

Open the script in VirtualDub and save it with "Fast compress" HufYUV, or feed ffmpeg with the avs script to create a h264/aac compressed file in a mp4 container:

ffmpeg.exe -i input.avs -c:v libx264 -crf 17 -preset slow -aspect 4:3 -c:a aac -b:a 128k output.mp4

The plugins will evolve during time, so if needed upgrade to last version of them once available

Of course this is just a basic/quick restoration, playing with the filters and their parameters the result can be improved a lot!

So using ffmpeg with settings as above, converting my raw video by just sending it back to ffmpeg, i.e. by

video_in=AviSource("D:\raw_capture.avi")
return(video_in)

is totally alright speed-wise, at ~15 fps. I thought of this also as some kind of sanity check for my AviSynth installation, luckily it gave the same speed as for "-i raw_capture.avi"

• Including the QTGMC deinterlacing, speed drops to 6 fps, so it's just half as fast
• The sharpening block, i.e. conversion to YV12, applying LSFmod, and converting back to YUY2, does not cost much, adding it to deinterlacing still results in 5.4 fps
• The denoising block at TR=3 just kills performance (1 or 2 frames per minute). For now, I think of TR as some kind of quality parameter, TR=1 gives 1.2 fps

If you don't mind, I may ask three things:

• Why are color models changed before applying the denoising and sharpening filter, especially conversion to YV12? Or is it just a requirement by the filters?
• Are my fps numbers somehow similar to what one would expect? I'm using a five year old mobile i5, but I could also try a recent i7.
• What would be interesting to look at next, especially regarding color saturation and contrast, from your more experienced point of view?

Thanks again.

[hodgey]

Quote:

Originally Posted by sprotte

Why are color models changed before applying the denoising and sharpening filter, especially conversion to YV12? Or is it just a requirement by the filters?

You are right, some filters don't support yv16 and/or yuy2. (Using YV12 over YV16/YUY2 can also have some degree of impact on speed if there is chroma filtering happening since the chroma resolution is halved vertically which may or may not be an acceptable tradeoff).

Quote:

Originally Posted by sprotte

Are my fps numbers somehow similar to what one would expect? I'm using a five year old mobile i5, but I could also try a recent i7.

Yeah looks normal. It's possible you can squeeze out more performance by using 64-bit avisynth+ (which is the most up to date variant, there are plugins that will only work on 32-bit, mainly old ones that haven't been updated).

There are also some quality/speed/look setting tradeoffs in e.g QTGMC you can play with. Look at the QTGMC avisynth wiki page for details.

I haven't used temporaldegrain much, tend to use smdegrain instead, but it seems it is able to make use of the GPU (requires FFT3DGPU plugin to be available) which may or may not help.

[lordsmurf]

Quote:

Originally Posted by sprotte

I've been following lollo2's great advice

I've not followed all posts in that thread in detail, but he does give good Avisynth advice. (Even if I don't like his coding style, tad complicated, preference.) Hopefully he can chime in on this thread as well!

Quote:

To be honest, I'm absolutely new to this field and this has helped me no end after having read just some other posts on which device to use for capturing.

When it comes to Avisynth, Hybrid can ease you into it. It's a swiss knife GUI, lots of options, Avisynth, Vapoursynth, and more. Most basic operations, like deinterlace, are there.

Quote:

(Long story short - I was asked to convert VHS to digital video, for pure preservation purposes, and lordsmurf's comment on the "The Best Easy Way to Capture Analog Video" Youtube video has directly brought me here. )

Quote:

speed-wise

Newbie mistake #1 = trying to cram everything into a single script. Do not do this. Get your order of processing down, then break it into logical operations. For example, very often (not always), QTGMC deinterlace (interpolate) from i29.97 fields to p59.94 frames is a single process. And best run in x64 Avisynth. Others may require x86/32bit.

Quote:

applying LSFmod

Be very careful using this. It's rarely needed as much as it is. Learn when NOT to sharpen.

Quote:

TR=3 just kills performance

It's a consequence of depth of processing. More needs more, sometimes more than you may realize.

Quote:

conversion to YV12? Or is it just a requirement by the filters?

Sometimes, yes.

Quote:

Originally Posted by hodgey

You are right, some filters don't support yv16 and/or yuy2. (Using YV12 over YV16/YUY2 can also have some degree of impact on speed if there is chroma filtering happening since the chroma resolution is halved vertically which may or may not be an acceptable tradeoff).

Net effect matters. If the end result is improved, minor colorspace conversion losses are just acceptable casualties. Obviously don't lose quality when possible (ie, don't be lazy, ignorant), but don't beat yourself up for minor incursions.

Quote:

There are also some quality/speed/look setting tradeoffs in e.g QTGMC you can play with. Look at the QTGMC avisynth wiki page for details.

Note: DO NOT USE MT 32-bit! There are glitch issues, especially with QTGMC. Either use x64 for speed, or learn to go slow with official 32-bit.

Quote:

I haven't used temporaldegrain much, tend to use smdegrain instead, but it seems it is able to make use of the GPU (requires FFT3DGPU plugin to be available) which may or may not help.

Even my onboard Intel GPU allows for GPU accelerations. It's not just for fancy expensive graphics cards anymore, and not for years.

[lollo2]

Quote:

Why are color models changed before applying the denoising and sharpening filter, especially conversion to YV12? Or is it just a requirement by the filters?

LSFmod only accepts YV12 color format. In general, apply a convertions only when is needed. Be careful with the convertions on interlaced material when, for whatever good reason, no deinterlacing is done inside the script.

Quote:

Are my fps numbers somehow similar to what one would expect? I'm using a five year old mobile i5, but I could also try a recent i7.

Yes. I experiment slightly higher fps than you with my Asus i7, but the same order of magnitude.
The best AviSynth filter are the slowest, which is normal.
TemporalDegrain2 has a specific TR parameter (temporal radius) to analyze the frames across their temporal distribution. For example, with TR=3 the previous 3 frames and the next 3 frames respect to the current frame are analyzed, to determine how to act for noise reductions (discriminate noise versus real details, because noise by its nature does not stay contant for long time). Higher the TR, better the final results, slower the processing. I have seen TR used at 9/12/16, producing insanely slow processing.

To accelerate the operations, as suggested by hodgey, for post processing you can move to AviSynth+ 64-bit Multi-Threading.

In alternative you can split the script in 3 smaller scripts: the first for deinterlacing, the second for denoise and the third for sharpening and eventually upscale. The sum of the processing times for the 3 scripts will be much lower than the required time for the whole script, but the negative point is that you'll be creating 2 additional intermediate large files (the output of the first and the second scripts)

Quote:

What would be interesting to look at next, especially regarding color saturation and contrast, from your more experienced point of view?

Prior to any processing you should check the hystograms to understand the levels of the captured video. I generally perform a check and level corrections at the beginning to stay inside 16-235 range if later I have a filter requiring a RGB colorspace, or if my final display option strictly requires RGB colorspace.

At the end of the filtering procedures you can add whatever color/saturation/gamma correction you think is appropriate or you like more. This is more an "art" than a procedure. Have a look here for a basic understanding:
https://www.cambridgeincolour.com/tu...istograms1.htm
https://www.cambridgeincolour.com/tu...istograms2.htm