DV AVI files, next steps to restore video?

[nickytbhp]

Hello,
This forum helped me capture my miniDV again in raw format. My professional converter used Scenalyzer to get DV Avi type 2 file (single file). 720x576, 25fps, PAL.
Now, my next steps are:
1. De-interlacing with QTGMC along with 25fps to 50fps conversion (Hybrid).
2. Need to reduce Noise but not sure which filter and what settings?
3. Would like to refine more but not sure which filters are good. I have Topaz Video AI and DAVINCI Resolve.
4. Will Hybrid and Resolve take this avi file or need to re encode to another codec and Container?
5. What is the best software to change container only? Avidemux?
6. What is the best software to re encode in case needed? Avidemux?

Looking for info from experts and if possible link to guides which has settings for specific tasks.

[latreche34]

I know for sure other members here have more experience than me in DV post processing but I would import the file into vdub2 and output as AVI 4:2:2 (yuv2) HuffYUV, so like this it becomes a lossless like AVI file and you can do what ever you want with it, Though it is not really lossless since it was converted from 4:2:0 to 4:2:2. The files will be huge even with HufYUV lossless compression but they should be just temporary files for post processing purposes.

[BarryTheCrab]

I recently got a batch of DV tapes and had not actually done DV for quite some time, and only my own tapes.
My big issue was, or is...as I am still tweaking my workflow...DV1 does not seem to play well with my VirtualDub or Hybrid, or Avidemux. Freeze crash error, no ingest.
I have been forced to take my WinDV transfers, as well as my PowerDirector transfers, and run them through PowerDirector AGAIN to render as DV type 2.
Only THEN can I get success with those programs and if needed QTGMC.
I think Scenalyzer will give me the correct file type and I can skip a boatload of processing.
QUESTION: How destructive is a convert from DV1 to DV2?
I hope this question, and my comments get some input that helps the topic.

[nickytbhp]

Quote:

Originally Posted by BarryTheCrab

I recently got a batch of DV tapes and had not actually done DV for quite some time, and only my own tapes.
My big issue was, or is...as I am still tweaking my workflow...DV1 does not seem to play well with my VirtualDub or Hybrid, or Avidemux. Freeze crash error, no ingest.
I have been forced to take my WinDV transfers, as well as my PowerDirector transfers, and run them through PowerDirector AGAIN to render as DV type 2.
Only THEN can I get success with those programs and if needed QTGMC.
I think Scenalyzer will give me the correct file type and I can skip a boatload of processing.
QUESTION: How destructive is a convert from DV1 to DV2?
I hope this question, and my comments get some input that helps the topic.

That’s critical info. Thanks. Luckily I asked for type 2 and scenalyzer from professional converter after lots of research here.
Sorry, but I am no expert to answer your question. Hope experts will pitch in.

[Hushpower]

Quote:

I think Scenalyzer will give me the correct file type and I can skip a boatload of processing.

You can choose either type 1 or 2 for Scenalyzer capture.

Quote:

QUESTION: How destructive is a convert from DV1 to DV2?

I think, not much. I use DVDate to change from type 1 to 2:

https://paulglagla.com/en/dvdate-2/

While it does take a little while to convert, there's hardly any CPU activity so I doubt that there's any encoding going on.

[Hushpower]

Quote:

Originally Posted by Latreche

I would import the file into vdub2 and output as AVI 4:2:2 (yuv2) HuffYUV, so like this it becomes a lossless like AVI file and you can do what ever you want with it

Why can't you just process it in Virtual Dub and/or AVISynth as a DV AVI? Is the 420>422 change for processing actually necessary?

[dpalomaki]

The main reason for converting 420 to 422 would be for subsequent processing such as color correction, filtering, and graphics effects. Processing in 422 reduces artifacts resulting from the lower color resolution. Note that these artifacts may or may not be significant for your purposes. If converting directly to a SD 422 distribution format with cuts only editing it may not be worth it.

[BarryTheCrab]

I can confirm the DVtype2 files open in both Avidemux and VirtualDub.
Staxrip recode also ingested the files, and it has QTGMC native!

[lollo2]

Quote:

The main reason for converting 420 to 422 would be for subsequent processing such as color correction, filtering, and graphics effects. Processing in 422 reduces artifacts resulting from the lower color resolution.

A conversion from 4:2:0 to 4:2:2 will not bring back any additional chroma lost details (and many AviSynth filters works on YV12 colorspace in any case). A more sofisticated processing like "chroma reconstruction based on luma" would be necessary. An example here (not really appropriate, but just as indication): https://forum.doom9.org/showthread.php?t=173005

Quote:

I would import the file into vdub2 and output as AVI 4:2:2 (yuv2) HuffYUV, so like this it becomes a lossless like AVI file and you can do what ever you want with it

There is no advantage doing that, and a useless additional big file is created. A DV video can be directly imported in AviSynth.

Quote:

I can confirm the DVtype2 files open in both Avidemux and VirtualDub.
Staxrip recode also ingested the files, and it has QTGMC native!

It may be a little be more difficult if you are not familiar, but you do not need any GUI like Avidemux or Staxrip for restoration. Only Avisynth (script) and ffmpeg (command line)

Quote:

Now, my next steps are:
1. De-interlacing with QTGMC along with 25fps to 50fps conversion (Hybrid).
2. Need to reduce Noise but not sure which filter and what settings?
3. Would like to refine more but not sure which filters are good. I have Topaz Video AI and DAVINCI Resolve.
4. Will Hybrid and Resolve take this avi file or need to re encode to another codec and Container?
5. What is the best software to change container only? Avidemux?
6. What is the best software to re encode in case needed? Avidemux?

1. OK. Choose the best options in QTGMC for an optimal result. Hybrid will help you, but you can build your own specific and optimized script.
2. The best denoiser are the temporal or spatial/temporal filters. TemporalDegrain2, MCTemporalDenoise, SMDegrain, KNLMeansCL. QTGMC itself is a (less accurate) denoiser.
3. Topaz VEAI is not really appropriate for SD DV videos, but you can give it a try using it just as an upscaler (no deinterlacing, no denoising). DAVINCI Resolve is excellent for color correction.
4. Yes, as long as the right codecs are installed
5. There is no sense in changing container unrelated to a compression (i.e. to h264 or h265)
6. All the existing GUI do not reencode on their own, but use ffmpeg. If you are not comfortable with a command line, choose whatever you like more

[latreche34]

Quote:

Originally Posted by Hushpower

Why can't you just process it in Virtual Dub and/or AVISynth as a DV AVI? Is the 420>422 change for processing actually necessary?

Yes, Not my word but it was suggested to me few years ago that the coding works more efficiently out of 4:2:2, Not that it brings any additional chroma info, The math just works better. Dpalomaki worded it better than me. If in doubt ask some coding members over at videohelp, There are quite few bright folks out there.

[nickytbhp]

Quote:

Originally Posted by lollo2

It may be a little be more difficult if you are not familiar, but you do not need any GUI like Avidemux or Staxrip for restoration. Only Avisynth (script) and ffmpeg (command line)

1. OK. Choose the best options in QTGMC for an optimal result. Hybrid will help you, but you can build your own specific and optimized script.
2. The best denoiser are the temporal or spatial/temporal filters. TemporalDegrain2, MCTemporalDenoise, SMDegrain, KNLMeansCL. QTGMC itself is a (less accurate) denoiser.
3. Topaz VEAI is not really appropriate for SD DV videos, but you can give it a try using it just as an upscaler (no deinterlacing, no denoising). DAVINCI Resolve is excellent for color correction.
4. Yes, as long as the right codecs are installed
5. There is no sense in changing container unrelated to a compression (i.e. to h264 or h265)
6. All the existing GUI do not reencode on their own, but use ffmpeg. If you are not comfortable with a command line, choose whatever you like more

Thanks for the response.

1. QTGMC (Vapoursynth) in Hybrid, I normally use bob (which doubles fps). That's it. Should I use other settings also which might result in better output? Any link to settings guide?
2. Are there any optimal settings guide for these filters?
3. I somewhere read in this forum that right sequence is colour correction->denoise, degrain ->deinterlace -> encode to final format. is this correct?

[lollo2]

Quote:

1. QTGMC (Vapoursynth) in Hybrid, I normally use bob (which doubles fps). That's it. Should I use other settings also which might result in better output? Any link to settings guide?

The guide for QTGMC parameters is inside its doc: http://avisynth.nl/index.php/QTGMC.

The setting for a better output does not exist, it depends on your source. For a "natural" look I often use

QTGMC(preset="slow", matchpreset="slow", matchpreset2="slow", sourcematch=3, tr1=2, tr2=1, NoiseTR=2, sharpness=0.1)

Quote:

2. Are there any optimal settings guide for these filters?

No. For temporal filtering the most important parameter is the temporal radius, i.e. the number of frames across which the image is analysed to remove non constant/random noise. Higher is better, but more time consuming.
Be careful when denoising after QTGMC to do not introduce a "plastic look" by removing too much noise and details

In addition you can also do some sharpening after a denoise, but again be careful to do not introduce/enhace the halos.

Quote:

3. I somewhere read in this forum that right sequence is colour correction->denoise, degrain ->deinterlace -> encode to final format. is this correct?

No. The correct sequence is deinterlace -> denoise/degrain -> color correction -> encode. Color correction can be the first operation, according to your source, the filtering you use and your preference. I often do so. Do not forget the level correction, needed if at some point you use a filtering working on the RGB colorspace and maybe needed at the end accordingto your display options.

[lordsmurf]

Quote:

Originally Posted by nickytbhp

HNow, my next steps are:
1. De-interlacing with QTGMC along with 25fps to 50fps conversion (Hybrid).
2. Need to reduce Noise but not sure which filter and what settings?
3. Would like to refine more but not sure which filters are good. I have Topaz Video AI and DAVINCI Resolve.
4. Will Hybrid and Resolve take this avi file or need to re encode to another codec and Container?
5. What is the best software to change container only? Avidemux?
6. What is the best software to re encode in case needed? Avidemux?

Looking for info from experts and if possible link to guides which has settings for specific tasks.

1. Yes, QTGMC best, and Hybrid makes it easier than Avisynth or Vapoursynth directly. But note that 50p vs 25p is not merely something you choose on preference, nor even quality, but output factors. Contrary to popular misconception, 50p is not better than 25p, and it does not lose 50% of the date. Some temporal data is lost (not half), and 50p can incur artifacts. So you need to test, do not assume.

2. Be careful. Do not turn video plastic from aggressive filtering. Remove noise, not detail.

3. Filters are based on needs, not random non-reasons. DaVinci is for color, worthless otherwise. Topaz is worthless altogether, and is wholly inferior (especially for SD sources, and even more especially for VHS) to Avisynth. Do not use Topaz if you seek quality, it adds artifacts, oversharpens, messy stuff.

5. You often don't just "change container". From what to what? Source, destination. And no, usually not Avidemux.

6. Again, what to what? Source, destination? For some things, yes Avidemux. For others, Hybrid. For some, MainConcept or other payware. It just depends.

Also, adding to above replies...

Filter 4:2:0 to 4:2:2, when possible, hoping it 's not too damaged. There are filters for Avisynth and VirtualDub both. Hybrid may even has some integrated, don't recall offhand.

Correct, needs to be stated again: deinterlace > NR > color correction > encode
In rarer situations, order may change. But usually those who think they found the exception really have not.

For QTGMC, preset=Slow softens. I usually start with Faster, and slow down if needed. Rarely do I exceed Medium. I think lollo2 here is overriding Slow with switches, but I'd still want to doublecheck the syntax. I'd rather switch everything, not use a preset, if switching.

@Hushpower, the variable is the DV codec, and all of the software codecs massively suck. Even bad hardware codecs are often better.

Avidemux really often very little for restoring. VirtualDub and Hybrid are the GUI tools for this. Even more detailed is from scripting. Sometimes scripting is the only way to restore, as Hybrid doesn't include everything, just popular options. And Hybrid doesn't necessarily order the commands as needed, just started ordering.

[lollo2]

Quote:

Contrary to popular misconception, 50p is not better than 25p, and it does not lose 50% of the date. Some temporal data is lost (not half), and 50p can incur artifacts.

No way. We discussed about it several times. In general, if you deinterlace, there is little sense to throw away half of the frames, losing half of the motion (when there is motion). Of course, everything must be experimented with own material, rather then rely on generic guidelines.

Quote:

I think lollo2 here is overriding Slow with switches,

The opposite. I do a reinforcement of the "slow preset" behavior to a more intense processing, to limit noise reduction and sharpening and to match the original look of the source (difficult to achieve). The "slow preset" alone tends to denoise and sharpen too much for my taste, especially because I generally use a denoiser and a sharpener right after.

[lordsmurf]

It just fully depends on multiple factors. I'm just stating that the dogma of "throwing away 50% of motion data" is not accurate, and 50/59.94 is not always better. To get there, data is interpolated. (This isn't at all the same as 4:1:1 DV tossing out 50% of color data, by contrast.)

The Slow just softens too much. I don't want false soft, nor false sharp. But trying to get there can be source based. It's why start with Faster, and go from there.

[nickytbhp]

Quote:

Originally Posted by lollo2

The guide for QTGMC parameters is inside its doc: http://avisynth.nl/index.php/QTGMC.

The setting for a better output does not exist, it depends on your source. For a "natural" look I often use

QTGMC(preset="slow", matchpreset="slow", matchpreset2="slow", sourcematch=3, tr1=2, tr2=1, NoiseTR=2, sharpness=0.1)

No. For temporal filtering the most important parameter is the temporal radius, i.e. the number of frames across which the image is analysed to remove non constant/random noise. Higher is better, but more time consuming.
Be careful when denoising after QTGMC to do not introduce a "plastic look" by removing too much noise and details

In addition you can also do some sharpening after a denoise, but again be careful to do not introduce/enhace the halos.



No. The correct sequence is deinterlace -> denoise/degrain -> color correction -> encode. Color correction can be the first operation, according to your source, the filtering you use and your preference. I often do so. Do not forget the level correction, needed if at some point you use a filtering working on the RGB colorspace and maybe needed at the end accordingto your display options.

Now, I have DV Avi type 2. When we do deinterlace (QTGMC), Hybrid re encodes to mp4 container with selected codec (x264, x265, prores etc. are listed in Hybrid). I guess this does some compression (quality of Video goes down to some extent) and might affect next operation like color correction (Davinci Resolve).

If this is true, which codec should I select so that quality is not compromised? It would be great if this codec is accepted in Resolve and Final Cut pro like softwares.

[lollo2]

Have a look here for the supported codecs in Resolve:
https://www.digitalfaq.com/forum/vid...i-resolve.html

You need to export the AviSynth processing and send them to Resolve in uncompressed avi format or a lossless format, to avoid degradation. Prores (almost lossless under certain conditions) or h264 with very low CRF (<10/14) can be used.

Selur can help you through the outputs of his tool

[Selur]

In regard to (near-)lossless formats. Hybrid supports:

• lossless H.264
• lossless H.265
• lossless VP9
• lossless AV1
• FFvHuff (lossless)
• UT Video (lossless)
• ProRes
• DNxHD
• PNG image sequence

at least one of these should be supported by any modern NLE.

Cu Selur