Super resolution equivalent for VHS or video?

[ErikCalifornia]

I just recently discovered a neat technique in digital photography called superresolution. Perhaps there are some who have heard of this before? Maybe there are some who have not too.

In digital photography, by stacking multiple similar photos on each other, you can actually remove noise and double the resolution. For example and in general with superresolution, a digital image that was originally 1920 x 1080 now results in 3840 x 2160. I've played around with it a bit, and it's pretty nifty for digital photos in bringing out extra detail that otherwise may be hidden in lower resolution images.

If anyone is interested, I'd suggest looking at a neat software package that does this:

PhotoAcute Studio 3:
http://www.photoacute.com/studio/index.html

The question I have is what would be the equivalent to supperresolution in VHS or video? Or, would supperesolution even really apply to VHS and video?

One of my favorite threads on here is:

http://www.digitalfaq.com/forum/vide...-captures.html

How are the techniques of averaging multiple captures similar to that of superresolution? Or, how are they different? And, if you can use superresolution on digital images through stacking, then couldn't the same technique be applied through multiple copies of VHS or video in general? Or, is averaging multiple captures (as described in the above thread) actually close to what supperresolution accomplishes?

The defacto standard resolution for transferring VHS to digital is 720 x 480. Through superresoltuion (or similar technique), you would in theory end up with a resolution of 1440 x 960. If applicable, I would imagine that superresolution applied to up-scaling would look better on the more common HD displays that everyone seems to have today as opposed to just not applying any techniques at all and doing a straight upscale from 720 x 480 to a full HD resolution. Or?

My point is, I just thought supperresoultion is pretty neat for Digital Images and photography. I was wondering are their any similar techniques available to us VHS and video people too? Perhaps, techniques that can be performed through Avisynth?

Thoughts, comments, ideas?

Thanks!

Erik

[jmac698]

Hi,
There's a couple misconceptions here. Multiple copies of the same thing reduces noise. Superresolution requires slightly different views of the same thing. So stacking VHS copies won't give supperresolution.

How superresolution works

Imagine you have a 100x100 image. Take out every odd column and now it's a 50x100 image. Call this picture A. Now take out every even column from the original and call it B. It's clear that you could combine A & B to get the original image. How this relates to a camera is, say the camera only has 50x100 resolution. It can take pic A. Then nudge the camera slightly to the left, now it's taking pic B. I'm not explaining some details here, but in reality, due to camera shake, every picture is slightly different. The more random offsets, the better the resolution you can reconstruct. It works by figuring out the offset of each image and then recreates the in-between values. Another requirement is that there must be some aliasing in the image. I won't go into it further, hope you get the idea.

That won't work of VHS copies of the same movie because it's the exact same image that was recorded to each copy.

Averaging multiple copies of the same image reduces noise. But which noise does it reduce? In the case of multiple VHS copies of the same movie, it's reducing all noise added to the system beyond the original copy. That would be tape noise, VCR noise and capture noise. In the case of multiple captures of the same tape, only VCR noise and capture noise is reduced. The tape noise is there to begin with.

I should add an exception here. You can still do superresolution on a VHS, but it requires motion in the movie. That would work by the slightly different view idea. But any static objects won't have superresolution. I was stuck in thinking in photo mode for some reason

Also, the resolution I have measured in VHS was barely 240 pixels across.

[themaster1]

You'd have to deinterlace your video first, remove noise and such and finally use this software: http://www.infognition.com/articles/...esolution.html

Note that by deinterlacing (with qtgmc (high end deinterlacer) or similar) you'd loose some infos regardless

[msgohan]

Quote:

Originally Posted by jmac698

Superresolution requires slightly different views of the same thing.

In the case of movies, a user on OriginalTrilogy.com has suggested that an NTSC and PAL release based on the same master could be combined to improve resolution: http://originaltrilogy.com/forum/top...y/topic/15928/

What do you think? And if that could work, what about if in theory there were two releases of the same content based on the same higher-resolution master but released twice, where the second just happens to have a slight horizontal offset induced prior to the final output?

Quote:

In the case of multiple VHS copies of the same movie, it's reducing all noise added to the system beyond the original copy. That would be tape noise, VCR noise and capture noise. In the case of multiple captures of the same tape, only VCR noise and capture noise is reduced.

Have you tried averaging two copies of the same VHS retail release to reduce noise? I have two copies of a movie but I don't have them both here to try... I suspect the benefits would be equalled by downsides.

[jmac698]

Yes, most definitely! Not by a lot though. I liked those original trilogy threads, have that focus has spurred a lot of creative thinking. I've wanted to add to the discussion, but I thought the project was over.

There's a simple improvement in the NTSC/PAL case however. A PAL color represents two lines, but NTSC is better and represents one. You could shift the tint/saturation of the NTSC version averaged over two lines to match the PAL version, so get the PAL colour accuracy, then double the colour resolution by mapping the NTSC colour to the PAL luma.

As for combining resolutions, it's a pretty simple formula. Let's use an easy case, two sources at once and 1.5 times the resolution, you want to combine them.

Say the pixels are 7 9 2 8 7 3 originally, the NTSC version is half-rez so 8 5 5, and the PAL version 2/3 so 6 6.

Now to work backwards you have a formula,
x1+x2=2*8
x1+x2+x3=3*6
So x3=3*6-2*8=2

What you've found is one more new pixel for every PAL pixel. It's not an even increase in resolution and may give an aliased look to it. The formulas also imply that this will increase noise, because the moment the source pixels are averaged, there's roundoff error. Any new pixels are going to be noisier.

I could simulate all this in Avisynth in a few hours but my main computer is down atm. I'd love to work on it and show you the actual difference it makes though. Hmm maybe if I hook up my main monitor to this one, it would be comfortable enough to deal with...

Also, I bought 3 brand new copies of an animated VHS just to test the averaging idea, but it didn't work out because one of the copies has some problems with it. I saw 4 copies of an old movie just a few days ago, but I didn't get it because it wasn't the ideal movie to test with. I wanted something low noise like animation so you could see the difference from a noiseless source to VHS.

I've also experimented with new formats for storing video on VHS. IMHO the best way to store a 2.40:1 ratio movie is to rotate it 90 and make it anamorphic. I've also separated a movie into Y, U, and V and stored them as luma for increased and more accurate colour.

[lordsmurf]

Some of the threads there are indeed interesting, and several regular/long-time users there have contacted me through the years.

The main issue I see there are gamma, luma and IRE errors on the clips. While the combined resolution may be better, the combined color vales look like crap. Good theory, but difficult to use in practice.

[jmac698]

Just an update, I tried the superrsolution plugin you talked about on VHS, it didn't really look better to me.

[Lolmen]

Is anyone found something this days?
I heard about yandex deep hd.
https://yandex.ru/promo/deephd