Upscaling DV still shows aliasing?

[iseevisions]

I was deinterlacing+upscaling analog PAL captures in vapoursynth with following commands, which works well:

clip = haf.QTGMC(clip, Preset='Slower', TFF=True)
clip = core.resize.Lanczos(clip, 1440, 1080 ,matrix_in="6",matrix="1")

The stairstepping/jagged lines/aliasing go away which is nice.


But now when I do it on PAL DV, as follows:

clip = haf.QTGMC(clip, Preset='Slower', TFF=False)
clip = core.resize.Lanczos(clip, 1440, 1080 ,matrix_in="6",matrix="1")

there's still some aliasing visible, see attached zoomed example:

vlcsnap-2024-11-19-14h16m40s986.png

It is also present in the DV source so probably due to the DV codec itself.

Is there a way to fix this without blurring?

[latreche34]

I would input the DV file into vdub2 and output as lossless AVI or compressed lossless HuffYUV AVI yuv2 or similar codec to get rid of the DV codec first, then proceed with your normal script as if it was an analog capture, Also when you resize to 1440x1080 make sure to switch to rec.709.

[iseevisions]

Do you think there's something going wrong when feeding DV directly to vapoursynth?
I'll try the preconversion to lossless and let you know...

matrix 1 is already rec.709, according to http://www.vapoursynth.com/doc/funct...resize.Lanczos
When I compare the colors of the source/input and the output, it's the same, so that should be already correct.

[themaster1]

try this:

main=last
main
Santiag() # reduce aliasing
filtered=last
main
mt_edge().mt_expand
mymask=last
mt_merge(main, filtered, mymask)

[iseevisions]

I did the conversion to lossless first and then vapoursynth, same result.

Quote:

Originally Posted by themaster1

try this:

This looks something for avisynth, but I'm using vapoursynth.
I'll have to lookup and install the equivalent for that...

-- merged --

I did it with this and it looks better now: clip = xaa(clip,mode="di2 znedi3", mask=0, chroma=1)

But should I do it before or after de-interlacing(QTGMC) ?

[latreche34]

I would investigate the root cause first, I see this in forums all the time when someone presents a problem everyone jumps in with a patch solution instead of focusing on addressing the root cause of the problem. Maybe post few seconds sample of the raw DV so we can examine.

[iseevisions]

OK here it is attached: test.avi

I was briefly thinking it could be something to do with the camera'a digital zoom, but it's also visible in wide/zoomed out shots.

[traal]

PAL DV uses 4:2:0 chroma subsampling and so the reds and blues will have less resolution than the greens.

In my experience, Topaz AI works well on NTSC DV, you might try it with your PAL DV.

-- merged --

I tried to upscale the video in Topaz AI but it did not satisfactorily resolve the aliasing issues. It's a tough problem.

[iseevisions]

Thanks for the input. I'll do it with the xaa antialiasing in vapoursynth.
Rests the question, is it best before or after deinterlacing?

[lordsmurf]

AA must come after, but noting QTGMC has some built in.

[latreche34]

I just looked at the sample, The problems you are facing are baked in the original DV file, They are not caused by Vapoursynth, Thick vertical lines, image overexposed, not sure what happens during the shooting of the footage. One thing for sure, you can't bring back the over exposed areas like her face, Also it's hard to fix those vertical lines in post.

[iseevisions]

Quote:

Originally Posted by latreche34

They are not caused by Vapoursynth

Of course not, you can also see it in the source DV I posted... I just mentioned Vapoursynth/QTGMC because in analog footage it also removes a lot of aliasing/stairstepping, but not here in this DV.

Quote:

Originally Posted by latreche34

you can't bring back the over exposed areas

I know, and I'm not trying to, it's just a part of the recording. Only the aliasing bothers me.

Quote:

Originally Posted by latreche34

Also it's hard to fix those vertical lines in post.

The "mice teeth" in the curtain or what do you mean, the aliasing itself?


I did the AA after QTGMC + resizing and it looks better, see test.avi.mp4

So if the camera(Sony TRV-620E) recorded the aliasing, is there a problem with it, or is it just a DV codec problem due to low resolution etc?

I also noticed that the DV rec has only 32Khz audio, which is a bit crappy... Should I convert it to 48Khz AAC in MP4 or leave it 32Khz?

[Selur]

After QTGMC, I would try fixing the chroma shift, apply Santiag with in a vertical flipped video, adjust the colors then try denoising,... https://imgsli.com/MzIzMTE0/0/3 and only after that think about upscaling.

[iseevisions]

Looks better indeed, for this clip at least. I see you also did a lower gamma. What syntax did you use for these?

[timtape]

Quote:

Originally Posted by latreche34

I just looked at the sample, The problems you are facing are baked in the original DV file,... Thick vertical lines, image overexposed, not sure what happens during the shooting of the footage.

Yes, this isnt cine film where we didnt see over or under exposure until the film came back from lab processing and was projected. That wildly overexposed face would have looked like that in the DV camera's viewfinder at the time of the event. The camera operator either didnt notice or for whatever reason didnt do anything about it. Some things, many things, we must get right at time of shooting.

[timtape]

Quote:

Originally Posted by iseevisions

..I also noticed that the DV rec has only 32Khz audio, which is a bit crappy... Should I convert it to 48Khz AAC in MP4 or leave it 32Khz?

It helps to relate these abstract numbers to actual sound in the real world. The camera's original audio sample rate determines only the upper frequency limit which at 32 kHz sample rate is around 15 kHz, higher than many people can hear, and probably higher than anybody over 50 will ever hear. Professional FM radio has the same upper limit of around 15 kHz.

As to the audio recording of the speaker onstage it's typical of an amateur recording with a "from the audience" camera and onboard mic. That's the basic limitation here, not the audio sample rate.

Converting to 48 kHz sample rate wont bring back audio between 15 kHz and 23 kHz which was never recorded, and even if it could, it would make essentially no difference.

[Selur]

Quote:

What syntax did you use for these?

I used 0.85
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="470bg", range_s="limited")
clip = core.std.Levels(clip=clip, min_in=16, max_in=235, min_out=16, max_out=235, gamma=0.85)[/code]

[iseevisions]

Quote:

Originally Posted by timtape

The camera operator either didnt notice or for whatever reason didnt do anything about it.

I did not shoot this myself but I assume the camera was just set to "AUTO" and the operator didn't know better or did not bother.
The overexposed face part is also a small portion of the full recording, I just chose this part deliberately to upload here because the face isn't recognizable

Quote:

Originally Posted by timtape

Converting to 48 kHz sample rate wont bring back

I don't mind the 15Khz limit, only am worried about possible player compatibility issues. Is 32Khz (instead of the "standard" 48Khz) AAC with H264 inside .mp4 playable everywhere?

Quote:

Originally Posted by Selur

clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="470bg", range_s="limited")

Converting to RGB? I thought that was a "no-no" for these types of video?
Also then I have to do something like this now to convert it back to YUV for the next processes in line:

clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="470bg", range_s="limited")
clip = core.std.Levels(clip=clip, min_in=16, max_in=235, min_out=16, max_out=235, gamma=0.85)
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P8, matrix_s="709", range_s="limited")

Can you also share the others because did you use avisynth or vapoursynth, these often have different plugins?

[Selur]

Without BasicVSR++ on the chroma:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import ctypes
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'F:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("F:/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
# loading plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/libsangnom.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/EEDI2.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/GrainFilter/RemoveGrain/RemoveGrainVS.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/EEDI3m_opencl.dll")# vsQTGMC
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/NNEDI3CL.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DeinterlaceFilter/Bwdif/Bwdif.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")
# Import scripts
import mvsfunc
import chromashift
import havsfunc
import validate
# Source: 'C:\Users\Selur\Desktop\test.avi'
# Current color space: YUV420P8, bit depth: 8, resolution: 720x576, frame rate: 25fps, scanorder: bottom field first, yuv luminance scale: limited, matrix: 470bg, format: DV
# Loading C:\Users\Selur\Desktop\test.avi using LWLibavSource
clip = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/test.avi", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
frame = clip.get_frame(0)
# setting color matrix to 470bg.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG)
# setting color transfer (vs.TRANSFER_BT601), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601)
# setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)
# making sure frame rate is set to 25fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# making sure the detected scan type is set (detected: bottom field first)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_BOTTOM) # bff
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=False, opencl=True) # new fps: 50
# Making sure content is preceived as frame based
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
# adjusting color space from YUV420P8 to RGB24 for vsLevels
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="470bg", range_s="limited")
# adjusting color using Levels on RGB24 (8 bit)
clip = core.std.Levels(clip=clip, min_in=16, max_in=235, min_out=16, max_out=235, gamma=0.85)
# adjusting color space from RGB24 to YUV444P16 for vsChromaShiftSP
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P16, matrix_s="470bg", range_s="limited")
# Chroma adjustment using ChromaShiftSP
clip = chromashift.ChromaShiftSP(clip=clip, X=-1.50, jeh=False)
# applying anti aliasing using santiag
clip = core.std.FlipVertical(clip)
clip = havsfunc.santiag(c=clip, nns=2, qual=2, pscrn=2, opencl=True)
clip = core.std.FlipVertical(clip)
# adjusting output color from: YUV444P16 to YUV420P10 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited", dither_type="error_diffusion")
# set output frame rate to 50fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=50, fpsden=1)
# output
clip.set_output()

with BasicVSR++ applied on the chroma:

# Imports
import vapoursynth as vs
# getting Vapoursynth core
import ctypes
import sys
import os
core = vs.core
# Import scripts folder
scriptPath = 'F:/Hybrid/64bit/vsscripts'
sys.path.insert(0, os.path.abspath(scriptPath))
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("F:/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
# loading plugins
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/libsangnom.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/EEDI2.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/GrainFilter/RemoveGrain/RemoveGrainVS.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DenoiseFilter/NEO_FFT3DFilter/neo-fft3d.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/EEDI3m_opencl.dll")# vsQTGMC
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/NNEDI3CL.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/MiscFilter/MiscFilters/MiscFilters.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/DeinterlaceFilter/Bwdif/Bwdif.dll")
core.std.LoadPlugin(path="F:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/LSMASHSource.dll")
# Import scripts
import mvsfunc
import chromashift
import havsfunc
import validate
# Source: 'C:\Users\Selur\Desktop\test.avi'
# Current color space: YUV420P8, bit depth: 8, resolution: 720x576, frame rate: 25fps, scanorder: bottom field first, yuv luminance scale: limited, matrix: 470bg, format: DV
# Loading C:\Users\Selur\Desktop\test.avi using LWLibavSource
clip = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/test.avi", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
frame = clip.get_frame(0)
# setting color matrix to 470bg.
clip = core.std.SetFrameProps(clip, _Matrix=vs.MATRIX_BT470_BG)
# setting color transfer (vs.TRANSFER_BT601), if it is not set.
if validate.transferIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Transfer=vs.TRANSFER_BT601)
# setting color primaries info (to vs.PRIMARIES_BT470_BG), if it is not set.
if validate.primariesIsInvalid(clip):
  clip = core.std.SetFrameProps(clip=clip, _Primaries=vs.PRIMARIES_BT470_BG)
# setting color range to TV (limited) range.
clip = core.std.SetFrameProps(clip=clip, _ColorRange=vs.RANGE_LIMITED)
# making sure frame rate is set to 25fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=25, fpsden=1)
# making sure the detected scan type is set (detected: bottom field first)
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_BOTTOM) # bff
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=False, opencl=True) # new fps: 50
# Making sure content is preceived as frame based
clip = core.std.SetFrameProps(clip=clip, _FieldBased=vs.FIELD_PROGRESSIVE) # progressive
# adjusting color space from YUV420P8 to RGB24 for vsLevels
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="470bg", range_s="limited")
# adjusting color using Levels on RGB24 (8 bit)
clip = core.std.Levels(clip=clip, min_in=16, max_in=235, min_out=16, max_out=235, gamma=0.85)
# adjusting color space from RGB24 to YUV444P16 for vsChromaShiftSP
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P16, matrix_s="470bg", range_s="limited")
# Chroma adjustment using ChromaShiftSP
clip = chromashift.ChromaShiftSP(clip=clip, X=-1.50, jeh=False)
clipMerge = clip
# adjusting color space from YUV444P16 to RGBH for vsBasicVSRPPFilter
clipMerge = core.resize.Bicubic(clip=clipMerge, format=vs.RGBH, matrix_in_s="470bg", range_s="limited")
# Quality enhancement using BasicVSR++
from vsbasicvsrpp import basicvsrpp as BasicVSRPP
clipMerge = BasicVSRPP(clip=clipMerge, model=4)
clipMerge = core.resize.Bicubic(clip=clipMerge, format=vs.YUV444P16, matrix_s="470bg", range_s="limited")
clip = core.std.Merge(clip,clipMerge,[0, 1, 1]) # merge filtered and unfiltered by plane
# applying anti aliasing using santiag
clip = core.std.FlipVertical(clip)
clip = havsfunc.santiag(c=clip, nns=2, qual=2, pscrn=2, opencl=True)
clip = core.std.FlipVertical(clip)
# adjusting output color from: YUV444P16 to YUV420P10 for NVEncModel
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited", dither_type="error_diffusion")
# set output frame rate to 50fps (progressive)
clip = core.std.AssumeFPS(clip=clip, fpsnum=50, fpsden=1)
# output
clip.set_output()

in both cases I did not apply any denoising.

Cu Selur

[iseevisions]

Quote:

Originally Posted by Selur

BasicVSR

Is BasicVSR (which uses GPU cuda/opencl) also very slow in your setup?

I'm playing with your config and using BasicVSR (this line to be specific: clipMerge = BasicVSRPP(clip=clipMerge, model=4) ), it runs 1 full minute on the same test.avi, versus only 6 seconds if disabled...

My GPU = nvidia 1080ti

For other comparison, my own old config including upscaling + encoding is only 2 seconds.