Enhance uncompressed 320x240 digital video

[admin]

from email...

Quote:

I have a uncompressed digital video file made using this card: http://hothardware.com/Articles/Hauppauge-USB-WinTV.
The resolution is 320 x 240.
I need to enhance it to enable it to be reliably lipread as there is no audio.
I have used DeeMon's Video enhancer, vReveal, Motion Perfect and VirtualDub but I am starting from scratch in terms of knowledge and experience and am finding the whole thing frustrating (to say the least!)
I have had this video "professionally" enhanced but was not pleased with the results.
Any help would be gratefully received

320x240 is highly compressed in size. What is the video source? VHS tapes?

[1Ross]

The recording was made of a live TV show as it was broadcast.
The resolution is the best quality the card could capture unfortunately.

[1Ross]

Shown in the attached pic is shown is the result of using DeeMon's Video Enhancer to increase the resolution by 4 times (716x536).
I have cropped the original video to isolate the area of interest which resulted in a video sized 179x134.
I am unsure what effect the order in which I process the video will have if any.
i.e. should I denoise & deblock before or after increasing the resolution.
Any Virtualdub filters you could recommend would also be helpful.

Res x 4.jpg

[1Ross]

Don't know if this is helpful but here is a list of the filters I currently have.

Vdub Filters.JPG

[admin]

What resolution boosting generally does -- for digital methods, at least -- is just take low resolution video and put it into more pixels. Detail is still missing. In many cases, just adds re-compression noise.

For 320x240 video, I would suggest conversion to 352x480 resolution. I would also look into using TMPGEnc Plus for this, not VirtualDub. I had a lot of success converting VCDs (352x240) to DVD in years past, restoring with TMPGEnc's built-in "sharpen edge" filter. It works different (and arguably better) than what is available for VirtualDub. Vdub filter/plugin programmers are seemingly too afraid to really apply harsh sharpening methods, so results tend to be mediocre or unnoticeable.

Buy TMPGEnc Plus for $37 at http://edge.affiliateshop.com/public...7389&BID=12418
Guide for TMPGEnc at http://www.digitalfaq.com/guides/video/convert-tmpg.htm

You can also frameserve out of TMPGEnc, and into VirtualDub, if you want, to further process the video. I did that a few times. Back then, it was slow, but dual-core and quad-core computers would probably breeze through it.

Like VirtualDub, TMPGEnc is one of those tools nobody should be without. And a bargain at $37.

Frameserving from TMPGEnc to VirtualDub is not obvious method, and a number of sites online mess up the instructions. If this is something you'd like to look into, let me know, I can create a new post with instructions and downloads (excluding the payware TMPGEnc Plus, which you'll need to buy or at very least try).

I'd also ask that you upgrade to Premium Member status, just a few bucks (less than the price of a book!) You could also submit a test clip to me, and I'd run some tests, give you an exact layout of what filters would work best.

I just ran a test here on my system, still works great.

I really don't think this will be a problem to improve. It's won't be magically high res or anything, but it can look more like a VHS tape, and less like a low-res computer clip, with the right work and filter chain.

News I'm sure you were hoping to hear!

[1Ross]

Thank you for your reply, I have been waiting for a very long time for this to be handled properly.

By the way the company I originally used are here http://www.videoit.co.uk/index.htm.
I asked these guys to specifically enhance my video to be lipread but the lip the lipreader said that my enhancement was better!?! (Iv'e included a screenshot)

I shall become a premium member in the next couple of days (Paypal transaction ref:65B75536TU0635448) and then upload a clip for you to look at, I am going to use the Lagarith lossless codec to compress the raw video. Is this ok or is there a better option?

[admin]

manchestervideo.com a.k.a. videoit.co.uk isn't a site that inspires confidence.
For example, the information quoted below from http://www.manchestervideo.com/conversion_info.htm is mostly horsecrap:

Quote:

A question often asked is how good will the copy be ?

The simple answer is that it's never perfect and mostly depends on how good the original footage is.

As a rule if you have a good quality source tape you can expect a fairly good quality copy. However, if your source tape has a crackling soundtrack and a wobbly picture then so will the copy!

If you would like to know more detailed information about the conversion process please read-on.

Copying analogue video footage is a lossy process. What this means is that you will never get a perfect copy of a VHS tape. When you copy and convert analogue video from one tape to another some information such as colour saturation or image detail is lost.

When transferring analogue video from one colour standard to another the quality drop issue is even greater. VHS is prone to poor colour reproduction (especially NTSC) with drop-out, colour bleed and tracking errors.

With proper equipment, and some effort -- and apparently knowledge they don't have -- a DVD can easily look better than the VHS tape did. Crackling audio can be fixed, wobbly picture can be fixed.

With proper settings, detail and color saturation is not lost. That's just lack of knowledge on their part, and nothing more.

You'd think they would know this, supposedly having a forensic analysis "division" at http://www.avive.co.uk -- and now I wonder what sort of work they do. I've consulted police departments and private investigators with what is possible, and how to go about getting desired results.

This is also BS:

Quote:

NTSC standard definition video is 720 x 480 pixels per frame at 29.97 frames per second. PAL standard definition video is 720 x 576 pixels per frame running at 25 frames per second. To convert between these two formats, depending on which way we are converting (to or from NTSC/PAL) we need to either squash or stretch the picture size and either reduce or increase the frame rate plus sort out the differences between the colour signals.

The video signal is passed through a digital conversion unit. As you might expect during this stretching/squashing frame padding/frame dropping process something has to give!

That's nothing more than a lack of understanding of digital formats. The palette (pixels) and image (video) is separate from one another. A 4:3 PAL will look the same as a 4:3 NTSC if done correctly -- nothing has to be stretched or squished. That's just silly.

DVD's don't have "colour signals" like VHS tapes did either, it's all YUV data these days. This is why PAL DVDs work in the USA, and why NTSC DVDs work in Europe.


Anyway....

Lipreading is mostly about size and motion. By having a deinterlaced 320x240 source, you lost half of your motion, and about half of your size.

I'd have to see more to experiment with it. Your problem may sound simple, but it's actually one of the more difficult things to filter and correct.

I do have some forensic-type methods that could be used and taught. But it takes time, so it's best use on short clips of under 10 minutes. And I mean tiiiiiiiiiiiiiiiiiiiiiiiiiime. Lots of it. Nothing hard, just repetitive as can be.

Nice sample clip, by the way. (NSFW, that's for sure!)

Definitely upload a clip, in some decent format. To attach here, it needs to be under 8MB. If it's larger, use one of these options: http://www.digitalfaq.com/forum/show...arge-1834.html

[1Ross]

Wow these guys at Manchester video sound like a bunch of fools.....just my luck

Going back to the process I've used to increase the resolution and your comments regarding it's detrimental effects I just wanted to be sure that your opinion applied to DeeMon's Super Resolution (http://www.thedeemon.com/articles/wh...esolution.html)

I have sent a PM to you with the FTP details in order for you to download the clip you requested. The clip has been compressed using the afore mentioned Lagarith lossless codec, please let me know if you need me to use MSU lossless codec as it seems to be quite old (maybe it is old due to it's near perfect design, I don't know) and having tested it though it will take significantly longer.

My system spec is:
OS: XP 32bit SP3
CPU: Core2 Quad Q6600 @ 2.4GHz (will oc to 3GHz)
RAM: 6GB DDR2 800 Dual channel (I use PAE and Superspeed RAMdisk in conjunction with eBoostr to use the excess 2.5GB above system managed RAM)
HDD: 3 x 500GB Sata in RAID0 + Single backup drive

Please believe me when I say that I realize that this problem isn't simple!!
In terms of the forensic-type methods you mention I am more than willing to learn and implement them should it be necessary.

In the above mentioned PM I will go into brief details as to why it is important that I am able to enhance this video in the best way possible.

[admin]

Got the PM, read it over, downloading the sample now.
Interesting details.

It will take several days to look this over. I'll get back to you in about a week on it.

[1Ross]

To say the least!

many thanks for your time.

Please let me know should any part of this require payment beyond the membership fee.

Kind regards

R

P.S. any results of your work can be uploaded via the same FTP account. Again many thanks.

[admin]

QUESTION 1. Is this sample video smaller than the 320x240 source? Or was this even-smaller video the source for your 320x240 version?

My goal would be to create a 352x480 DVD. This is maybe doable.

QUESTION 2. How long is the full video?

QUESTION 3. How big is the full source file?

[1Ross]

Answer 1: the source for this recording was a live UK PAL TV broadcast captured at 320x240 which was the best resolution the card could capture.
The original video was cropped to isolate the area of interest which resulted in the 179x134 video.
I have included a screenshot of the original 320x240 video

Answer 2: 20min 42 sec
I have trimmed it down to 16min 11sec, again to focus on what I need to be lip-read

Answer 3: 6.87 GB (7,381,190,144 bytes)

[admin]

Ah yes, the infamous European ad-crap on screen. They do that a lot on music channels, too, from what I've seen and heard. You've cropped down to the non-ad content. Gotcha.

[admin]

I used VirtualDub to decompile the video into a series of still PNG images.

Next, I performed a batch process in Photoshop CS3 -- with the Genuine Fractals Print Pro plugin -- to resize all frames to 720x576. This took about an hour of processing.

The current version of VirtualDub will reload the full series of PNGs when I open the first image in the series. (Images were in a separate folder, with nothing else in it, to be safe.)

The video was set to a proper framerate, and then encoded to a new AVI file.

It looks at least as good as the Deemon super resolution option, possible better.

I can convert to high bitrate MPEG-2 and upload to FTP.

[1Ross]

I didn't realize the video could be broken down like that, must of left you with thousands of images!

Sounds good, can't wait to see the results!

[admin]

I know women tend to talk a bit fast, but your sample video looks to have been accidentally sped up from 25fps to 30fps. I slowed it back down to 25fps on the new file. Their mouths are moving at a more human rate now. I'm sure a lipreader would appreciate it.

I saved it as an H.264 MP4 file, with a fairly generous bitrate to avoid digital artifacts. File is 20MB. Watch with VLC on your computer.
I've uploaded it to http://www.digitalfaq.com/videos/1Ro...40x480.mp4.flv (right-click, save as)
If you don't have VLC, get it at http://www.videolan.org/vlc/

For anybody else reading this post, know that the sample file has boobs in it. "Not safe for work" as some might say. It's actually quite boring, but I'm sure some prude would get offended by it.

Quality would probably be higher if it were converted to 4x precise resolution. The odd resize to 640x480 (not 720x576 as mentioned earlier) can cause the aliasing on the lips. However, the original deinterlace on the 320x240 capture has already done major damage to quality -- aliasing will still be present to a degree.

[1Ross]

Ok I now have the following:

TMPEG Plus 2.5
Photoshop CS4
Genuine Fractals Print Pro plug-in

So I am now hopefully ready to duplicate the process you used on my video (which looks so much better btw).
And your right it does seem so much better at the lower frame rate, I can't explain what happened there.

Could your please detail exactly the process you would recommend to enhance the video including all settings for each program.
Could you also perhaps recommend any filters that I might use to reduce the saturation from the lighting maybe?

[admin]

Open the video in VirtualDub. Use this one: http://www.digitalfaq.com/forum/show...-pre-1727.html

Go to File > Queue Batch Operation > Save Image Sequence
Save as Targa.


Do that for now.
... sorry, have to come back and edit this, back in a bit...

[1Ross]

Ok I am now the proud owner of 29,101 Targa images

[admin]

I have to re-create this scenario and document it for you. Back with that info soon.