Best way to sync audio to VEAI output?

Razings · #1 10-10-2022, 05:36 PM

I am working on making some old home video DV tapes cleaner using Topaz VEAI. Overall I was able to import them losslessly using an old xp machine with Firewire ports to AVI and there isn't any noticeable degradation to be fixed up, so I am just trying to use VEAI to touch up some of the inherent blurriness of a 20 year old handheld camcorder so it looks appreciable blown up on a modern TV. After trying practically every preset I found Dione Interlaced TV achieved my desired balance best of not over sharpening or unnecessary upscaling while still noticeably improving some lost details.

My issue is that none of the output options retain the audio, even the options meant to retain audio(MP4) have the keep audio option grayed out even though the tooltip implies it shouldn't be. Even more problematically the output video doesn't always match the length of the original input clips(most likely to do with how it doubles frame rate). I attempted to use DaVinci Resolve to manually separate the original audio and align it to with the new video, but I struggled heavily with the way Resolve would try to snap clips to the start of the timeline, especially when there was only a frame or two of difference between the two. And with well over 40 tapes I think I would have lost my mind or developed carpal tunnel at the pace I was able to achieve.

So my question is if anyone knows a better way to sync the audio to the new improved video or if this is a know bug in VEAI that can be fixed? Has anyone made a script in AVISynth to do something similar? Any advice is welcome but I am extremely new to all this and learning as I go.

lollo2 · #2 10-11-2022, 06:05 AM

Are you using a licensed version of Topaz VEAI?

dpalomaki · #3 10-11-2022, 06:46 AM

A simple fix: if the audio is off by a fixed amount and you are dealing with separate audio and video files - add (or trim) the necessary number of frames at the start of the audio. (Any decent NLE should allow you the unlink the audio and video to allow frame level timing adjustments.)

Audio normally will not lead video, it will lag by an amount that corresponds to the subject distance from the camera. However, in playback systems that are not appropriately coordinated sound heard by the viewer may lead because the video signal often takes more processing time to display than audio.

But make sure the audio shift is not an artifact of the viewing system rather than the output file. I often see this with audio from a motherboard sound output vs video on a TV from a HDMI output from the same PC, while the same sound and video on the TV will be OK.

Razings · #4 10-13-2022, 03:45 PM

I am attempting this with their Demo version before buying. It appears to be fully featured other than the watermark.

lollo2 · #5 10-13-2022, 04:15 PM

The deinterlace engine in Topaz VEAI is poor. Deinterlace with AviSynth QTGMC and use Topaz VEAI only as upscaler. Then we can check audio/video synch and we'll see about denoise...

Razings · #6 10-13-2022, 04:34 PM

Quote:

Originally Posted by dpalomaki

But make sure the audio shift is not an artifact of the viewing system rather than the output file. I often see this with audio from a motherboard sound output vs video on a TV from a HDMI output from the same PC, while the same sound and video on the TV will be OK.

I just double checked but it's not an audio shift but an entire video shift. The output appears to be 3 seconds longer over the course of an hour. Reading through some of the Topaz forums it appears it's a problem with their deinterlacing model(Dione). It seems like it was much worse originally and that they had improved it quite a bit but I guess its still an issue if I am doing large clips at once. I think I am going to have to abandon this method for the time being and instead do the deinterlacing separately with QTGMC and then try one of the different models.

Razings · #7 10-13-2022, 04:39 PM

Quote:

Originally Posted by lollo2

The deinterlace engine in Topaz VEAI is poor. Deinterlace with AviSynth QTGMC and use Topaz VEAI only as upscaler. Then we can check audio/video synch and we'll see about denoise...

Lol I just read the same thing in the VEAI forums and ended up writing my response above before refreshing to see your advice. Good to see I'm on the right track, thanks for the response.

lollo2 · #8 10-13-2022, 04:53 PM

Consider also that with your SD DV material, you can obtain better results with a full AviSynth or VapourSynth flow rather that Topaz. But it's nice if you experiment yourself

Razings · #9 10-13-2022, 05:18 PM

I've read that and tried looking into AviSynth and VapourSynth's tutorials/wiki pages but the amount of options and relative lack of visual examples made it extremely intimidating. What processes would you suggest I attempt first to compare against? Whenever I looked in the past on here the tutorials all seem to stop before filters or just say go try everything and find out what works best leaving me in the same place as before, just trolling through the wiki.

lollo2 · #10 10-14-2022, 03:39 AM

Quote:

What processes would you suggest I attempt first to compare against?

Yes, AviSynth/VapourSynth are intimidating at the beginning. Just start with a simple deinterlace/denoise/sharpening like in this example: https://www.digitalfaq.com/forum/vid...html#post85085

Then add a basic AviSynth upscale:

Code:

nnedi3_rpow2(rfactor=2, nns=4, qual=2, cshift="Spline36Resize", fwidth=1440, fheight=1080)

You may also consider to sharpen after upscaling rather then before for a better result.

Then you can look at how VapurSynth basicvsr++ (for example) can be more performant: https://forum.videohelp.com/threads/...0p#post2666502 and try to implement this workflow (diffult and long).

And then compare with the Topaz VEAI approach.

Hushpower · #11 10-14-2022, 08:47 PM

A comparison deinterlacing between QTGMC mind-bender and Virtual Dub's couple-of-clicks Muksun de-interlacing filter (not necessarily in that order!

) I also applied the CCD denoise filter in VDub, because QTGMC, I believe, does some light denoising by default.

I repeat, I have deliberately not nominated which is which.

labcoats QTGMC vs Muskun.jpg

lollo2 · #12 10-15-2022, 04:32 AM

Quote:

I repeat, I have deliberately not nominated which is which.

None of them make sense. The source is telecine 3:2 pull down at 29.79fps, it should not be deinterlaced, but inverse telecined.

In addition, when comparing deinterlacers use a video segment with fast moving objects, not static images, or at least static images derived from fields (not frames) with a large difference in temporal data.

And the superiority of QTGMC does not need to be reassessed, there are tons of examples proving it.

lordsmurf · #13 10-17-2022, 10:49 AM

Quote:

Originally Posted by Hushpower

I repeat, I have deliberately not nominated which is which.

The stair steps and artifacts betray the non-QTGMC clip. (You can get stair steps with the rougher not-suggested presets of QTGMC, but not the artifact.)

Quote:

Originally Posted by lollo2

In addition, when comparing deinterlacers use a video segment with fast moving objects, not static images, or at least static images derived from fields (not frames) with a large difference in temporal data.

Yep.