digitalFAQ.com Forum

digitalFAQ.com Forum (https://www.digitalfaq.com/forum/)
-   Restore, Filter, Improve Quality (https://www.digitalfaq.com/forum/video-restore/)
-   -   Levels before and after filtering (https://www.digitalfaq.com/forum/video-restore/10776-levels-before-after.html)

lollo2 07-04-2020 03:03 PM

Levels before and after filtering
 
I have written an avs script trying to "improve" the look of my captures: following most of the recommendations about correct levels in yuy2, before filtering I fix the luma in the range 16-235 and then apply TemporalDegrain2 and LimitedSharpenFaster, and I stay in the yuy2 color space without any conversion.

If I check the levels after the filtering, they are in the range 0-255!

Is it a normal behaviour for the filters to expand the yuy2 luma to 0-255?

Do I have to re-establish 16-235 correct range at the end of my script?

Winsordawson 07-04-2020 11:14 PM

Can you post your script?

keaton 07-04-2020 11:24 PM

Yes, some plugins do expand levels so the output min/max is smaller/larger than the input min/max. I guess you have 2 options: 1.) apply levels again after all plugins that effect levels are used, or 2.) reduce your min/max input levels further in from 16/235 before using plugins that effect levels.

I don't have a lot of experience here. However, if I were in this situation, I would choose option 2. My current view is option 1 sort of seems like undoing some of what the plugins are trying to do. So it may be preferred to give yourself more headroom before using such plugins. Trial and error would be needed to find how much further in from 16 and 235 to go to prevent exceeding those levels after using those plugins. Other threads talk about avisynth or virtualdub utilities that can be used to help find your min and max values. I suppose the best approach would be to trim the entire clip down to one or more sample sections that reach your input level min and max levels, thus saving you the time of iterating through script changes on the entire video.

lollo2 07-05-2020 04:53 AM

3 Attachment(s)
Here is my script:

Code:

# "progressive" fields
# interlaced  fields TFF
# TemporalDegrain2: Denoiser, Temporal filter with motion compensation
# modeA for progressive fields
# modeC_Nnedi3 for interlaced fields

video_org=AviSource("C:\Users\giuse\Videos\Acquisizioni\ufo_sII2a_spot_amtv_2_cut.avi")

# trimming
#        trim_start=528
#        trim_end=1232
#video_org_trim=video_org.trim(trim_start,trim_end)
video_org_trim=video_org

# cropping
        crop_left=8        # | removing black borders on left, top and righ; removal "heads switching noise" on bottom       
        crop_top=2        # | 720-(8+24)x576-(2+10)=688x564
        crop_right=24
        crop_bottom=10
video_org_trim_crop=video_org_trim.crop(crop_left,crop_top,-crop_right,-crop_bottom)

# plugins directory
plugins_dir="C:\Users\giuse\Documents\VideoSoft\MPEG\AviSynth\extFilters\"

        # LevelsLumaOnly_modGMa2
Import(plugins_dir + "LevelsLumaOnly_modGMa2.avsi")
        # Levels_GMa
Import(plugins_dir + "Levels_Gma.avsi")

        # TemporalDegrain2
Import(plugins_dir + "TemporalDegrain-v2.1.2_modGMa.avsi")
        # RgTools
loadPlugin(plugins_dir + "RgTools-v1.0\x86\RgTools.dll")
        # MaskTools2
loadPlugin(plugins_dir + "masktools2-v2.2.23\x86\masktools2.dll")
        # MVTools
loadPlugin(plugins_dir + "mvtools-2.7.41-with-depans20200430\x86\mvtools2.dll")
        # FFT3DFilter
loadPlugin(plugins_dir + "FFT3dFilter-v2.6\x86\fft3dfilter.dll")
        # FFTW
loadPlugin(plugins_dir + "LoadDll\LoadDll.dll")
loadDll(plugins_dir + "fftw-3.3.5-dll32\libfftw3f-3.dll")
        # Dfttest
loadPlugin(plugins_dir + "dfttest-v1.9.6\msvc\x86\dfttest.dll")

        # LimitedSharpen
Import(plugins_dir + "LimitedSharpenFaster.avsi")

        # Nnedi3
loadPlugin(plugins_dir + "NNEDI3_v0_9_4_55\x86\Release_W7\nnedi3.dll")

# parameters
# ColorYUV
        cont_y=-20
        off_y=-9
        off_u=5
        off_v=-5
# Tweak       
        sat=1.3
        cont=1.1
# Levels
        # first step
        input_low_1=17
        gamma_1=1.05
        input_high_1=254
        output_low_1=16
        output_high_1=235
        # second step
        input_low_2=5
        gamma_2=1
        input_high_2=255
        output_low_2=0
        output_high_2=255
# denoiser
        degrainTR=3
        postFFT=0
        postSigma=1
# sharpener
        strength=250
        overshoot=50

### levels and colors correction before filtering
# ColorYUV used to adjust luma contrast to limit and center luma hystogram, and to correct colors
# Tweak used to adjust luma contrast and colors saturation (hue and brightness not used)
# LevelsLumaOnly used to adjusts luma brightness, luma contrast, and luma gamma
noise_baseclip=video_org_trim_crop.\
                        ColorYUV(cont_y=cont_y, off_y=off_y, off_u=off_u, off_v=off_v).\
                        Tweak(sat=sat, cont=cont, coring=false, dither=true).\
                        LevelsLumaOnly(input_low=input_low_1, gamma=gamma_1, input_high=input_high_1, output_low=output_low_1, output_high=output_high_1, coring=false, dither=true)
                        # or ColorYUV(gain_y=-20, off_y=1, off_v=-5, off_u=5)

### Conditional filtering, audio is taken from source 1 (filterChainProgressive)
ConditionalFilter(noise_baseclip,\
filterChainProgressive(noise_baseclip, degrainTR=degrainTR, postFFT=postFFT, postSigma=postSigma, strength=strength, overshoot=overshoot),\
filterChainInterlaced(noise_baseclip, degrainTR=degrainTR, postFFT=postFFT, postSigma=postSigma, strength=strength, overshoot=overshoot),\
"progressive", "equals", "true", false)
ConditionalReader("C:\Users\giuse\Videos\Acquisizioni\avs_scripts\gma\14- restore\ufo_sII2a_spot_amtv_2_cut.txt", "progressive", false)

### levels correction after filtering
# LevelsLumaOnly used to adjusts luma brightness and luma contrast
LevelsLumaOnly(input_low=input_low_2, gamma=gamma_2, input_high=input_high_2, output_low=output_low_2, output_high=output_high_2, coring=false, dither=true)

### add borders
addborders(crop_left,crop_top,crop_right,crop_bottom)

return(last)


# filterChainProgressive modeA for progressive fields
function filterChainProgressive(clip c, int "degrainTR", int "postFFT", int "postSigma", int "strength", int "overshoot")
{
### denoising
denoised=c.TemporalDegrain2(degrainTR=degrainTR, postFFT=postFFT, postSigma=postSigma)
### sharpening
sharpened=denoised.LimitedSharpenFaster(strength=strength, overshoot=overshoot)
return(sharpened)
}

# filterChainInterlaced modeC_Nnedi3 for interlaced fields
function filterChainInterlaced(clip c, int "degrainTR", int "postFFT", int "postSigma", int "strength", int "overshoot")
{
### de-interlacing
deinterlaced=c.AssumeTFF().nnedi3(field=-2)
### denoising
denoised=deinterlaced.TemporalDegrain2(degrainTR=degrainTR, postFFT=postFFT, postSigma=postSigma)
### sharpening
sharpened=denoised.LimitedSharpenFaster(strength=strength, overshoot=overshoot)
### interlacing
interlaced=sharpened.AssumeTFF().SeparateFields().SelectEvery(4,0,3).Weave()
return(interlaced)
}

My video has segments that are interlaced (I call it PAL50Ii) and segments showing 2 consecutive frames identical (I call it PAL50Ip), so I de-interlace the first for the denoiser to be more effective, and then interlace it back.

I use ColorYUV() to reduce the luma range to 16-235 range and some color adjustement to my personal taste, Tweak() to increase contrast and saturation, which increase the levels to 17-254, and then LevelsLumaOnly to shrink back the levels at 16-235 (and a small gamma correction).

I simply apply TemporalDegrain2 and LimitedSharpenFast and after the processing the luma levels are now 0-255.

At the end I slightly adjust the "black" levels to my taste with LevelsLumaOnly again (still 0-255 range).

I am also attaching some picture to show an example of the levels before and after and the results of the script.

Levels of original frame:
http://www.digitalfaq.com/forum/atta...er-originaljpg

Levels of final frame (the "dark distibution" and the "bright distribution" seem ok, but some pixels are out of range)
http://www.digitalfaq.com/forum/atta...after-finaljpg

Comparison original/restored
http://www.digitalfaq.com/forum/atta...-comparisonjpg

lollo2 07-05-2020 05:17 AM

1 Attachment(s)
Thanks Keaton for your reply.

I already tried apply at the end a 16-255 shrink with ColorYUV(levels="PC->TV"), but the result is bad (see picture).

http://www.digitalfaq.com/forum/atta...1&d=1593944156


About option 2 I think that the filter may expand anyhow the final levels at 0-255 even with a small correction before (but I have not experiment it). And I think is not recommended to shrink too much the input levels.
Your approach to separate the video in several portion is convincing!

My concern is that there is a certain effort to work with the proper yuy2 levels before filtering, but finally the levels are 0-255. I also understand that the filters work better with correct input range, so the fix is appropriate.
I can leave the final level as they are, because I will play the files only on my PC, but I am not sure that a player will not clip or clamp the queues. And more, if I wish to rework again the colors, for example in VirtualDub, I may have a problem...

lollo2 07-05-2020 06:06 AM

Quote:

My video has segments that are interlaced (I call it PAL50Ii) and segments showing 2 consecutive frames identical
My video has segments that are interlaced (I call it PAL50Ii) and segments showing 2 consecutive fields identical

Winsordawson 07-05-2020 08:14 PM

Are the loose minimum and loose maximum also out of range? It is normal for a few pixels to be out of range in some scenes, such as on the nose of a person. This is done to ensure the rest of the face is properly exposed.

lollo2 07-06-2020 02:54 AM

Good point!

What I did was to run a small code writing in a file the luma parameters of all frames:
Code:

# convert to YV12 (required for Y statistics)
video_org_trim_crop_yv12=video_org_trim_crop.ConvertToYV12(interlaced=false)

# analysis
video_coloryuv=video_org_trim_crop_yv12.\
WriteFile("ufo_sII2a_spot_amtv_2_cut_step1a.txt", "current_frame", """ "  " """,\
"AverageLuma()", """ "  " """, "YPlaneMin()", """ "  " """, "YPlaneMax()", """ "  " """,\
"YPlaneMin(threshold=0.4)", """ "  " """, "YPlaneMax(threshold=0.4)", append=false).\

ColorYUV(analyze=true)

and then check the values overall and for the significant individual frames.

What I had was (video --> Luma Min-Max / Luma Loose Min-Loose Max (-4%):

org --> 16-254 / 16-250
ColorYUV(cont_y=-20, off_y=-9, off_u=5, off_v=-5) --> 16-235 / 16-231
Tweak(sat=1.3, cont=1.1, coring=false, dither=true) --> 17-254 / 17-250
#LevelsLumaOnly(17, 1.05, 254, 16, 235, coring=false, dither=true) -> 16-235 / 16-231

Loose Max was out of range also, and I checked some significant frame with
Code:

Limiter(16, 235, 16, 240, "luma_grey")
to be sure that I was not introducing useless filters only for a few pixels in a logo or in very limited areas, but it was not the case; I added then LevelsLumaOnly

sanlyn 07-06-2020 07:00 AM

It's not possible to give accurate advice without an unfiltered sample of the original video.
Processing in RGB will expand YUV levels from 16-235 to RGB 0-255.
A better analysis would be available with a piece of the original video.

lollo2 07-06-2020 07:55 AM

1 Attachment(s)
Attached a portion of the original video (frame 0 to 220).
It has been captured in HuffYUV with AmarecTV from a S-VHS tape recorded 20 years ago from a DVB-S broadcast (bad transmission, bitrate was low).
The VCR is a JVC S-VHS HR-S9500MS, used in play with TBC/NR=on, B.E.S.T.=on, edit=off, D3R=on (setting that in my case gives the best results imo).
No frame TBC.
The capture card is a Hauppauge USB-Live2.
PC is an ASUS laptop running Windows 10.

lordsmurf 07-20-2020 12:16 AM

I've always been more fond of GUI color grading in VirtualDub ColorMill or Premiere.
I've toyed with DaVinci recently, not fond of the experience.

Avisynth is possible, but I generally do all color grading type scripting separate from any sort of corrective or NR filters. Sometimes Avisynth doesn't play nice, doesn't make sense, conflicts.

lollo2 07-20-2020 03:39 AM

Thanks for your reply.
I am a beginner and just wanted to do some basic color correction (in my captures there is too much yellow for my taste), and did it with simple avisynth functions.

I will upgrade to ColorMill and VirtualDub for a more advanced correction.

I did not know that AviSynth can be problematic mixing color corrections and filtering. I will reduce the avs script to only denoise and sharpening, but at some point in time I have to combine it with the color correction. Do you suggest to include the color correction before, in the middle, or after the denoise+sharpening? (in the meantime I will run some experiment)


All times are GMT -5. The time now is 04:20 PM

Site design, images and content © 2002-2024 The Digital FAQ, www.digitalFAQ.com
Forum Software by vBulletin · Copyright © 2024 Jelsoft Enterprises Ltd.