digitalFAQ.com Forum

digitalFAQ.com Forum (https://www.digitalfaq.com/forum/)
-   Restore, Filter, Improve Quality (https://www.digitalfaq.com/forum/video-restore/)
-   -   Restoring vs. distorting, audio and video? (https://www.digitalfaq.com/forum/video-restore/2773-restoring-vs-distorting.html)

magillagorilla 01-05-2011 09:43 AM

Restoring vs. distorting, audio and video?
 
Hi everyone,
I've been lurking for a while and reading a lot. I've been hanging out at videohelp for a while too. I got an account here yesterday and downloaded the Lord Smurf VDUB. There is some good stuff in there. Sorry long first post...............

Anyhow, some background:
I have 12 plus years experience as an audio restoration hobbyist. That is, taking analog audio from cassette, vinyl, VHS, and Beta and cleaning it up for digital archiving. I am in the process of restoring family videos from VHS, VHS-C and mini-DV. I have already captured all of my footage in DV and HuffyUV AVI (VDUB 720x480).

Now, in my school of audio archiving some concepts carry over to video but some do not. I believe in creating a final product that represents the original as faithfully as possible. In audio archiving, noise reduction is the work of the devil. Audio enthusiasts will come after you with pitchforks for using it. NR kills detail in audio and I am learning through experimenting it does the same for video. In video it creates splotchy patches and blurs out detail. Like in audio, in most cases, I'd rather deal with the noise and retain the detail. Some enlightenment on how to properly use video NR is welcome, or I may just skip it all together.

Fair play in audio restoration is the use of EQs and notch filters, de-clicking/clipping (done properly) and the occasional compressor/limiter. These tools can be used to shape the sound to make it properly represent the original event. I am light handed when it comes to these tools. I often reduce undesired frequencies rather than boosting desired frequencies (boosting being an armature mistake). How this translates to video I'm not sure and I am looking for guidance. When I am correcting color is it better to reduce the undesired color like reducing unwanted frequencies in audio? There are obviously more "knobs" to tweak in video and I find myself getting carried away. After hours of tweaking color, contrast, and white balance on a clip I get lost. I think it looks better but maybe it's too artificial and I am punching settings too high. How do I benchmark and determine if I am helping the video rather than just repainting each frame with a digital facsimile?

I guess I am looking for advise on the best do's and don'ts using the LS compiled VDUB toolbox. Specifically for VHS. My delivery format will me MPEG2 as I have found it to be the friendliest codec for VHS (what I use to encode will be another thread :) )

THANKS!

jmac698 01-05-2011 02:17 PM

To restore is to return something to it's original state; in this case it would be to view a VHS as if it just came out of the camera.
Without the original camera to compare it too, it's impossible to know to restore the colors perfectly.
Why do we restore? My guess is to preserve a feeling of authenticity, and details of techniques used at the time.
You can also enhance. In this era of HDTV, we don't expect noise and blur. The choice is up to you.
You can also fudge the definition of restore to say, you want it to look as realistic as possible; as if you could go there again with modern equipment.
That means making skin tones look proper even if "authenticly" the camera always had bad white balance.

Theoratically, except for TBC and repairing dropouts, you wouldn't do anything to a VHS, as it probably doesn't look that much different than originally. VHS doesn't fade colors over time AFAIK. Even TBC is cheating; unless it was part of the camera playback, it's not authentic.

The equivalent of EQ is sharpness. There are some light methods of denoising that barely degrade the picture. Clicks would be dropouts or white lines. As far as boosting vs damping, the order of operations will affect numerical accuracy, and dealing with 8bit video, there's not much accuracy to go around.

Another point is that all your work on color grading will be for nothing unless you are using a calibrated monitor. When you've made it look perfect on your equipment, you may be disappointed to discover that it doesn't look the same on another TV. Most TV's these days vary quite a bit in their look and it takes a lot of work to calibrate everything.

I'll let the experts address the rest :)

magillagorilla 01-05-2011 03:43 PM

Enhance it the word
 
Great point about monitor calibration. Any suggestions on a reasonably priced calibration package? Seeing that my other hobby is digital photography this would be handy. I don't want to derail the topic though.

I like your point about restore verses enhancement. I think those terms get minced in these forums. To me a restore would be getting the tape to play and nothing else. Kind of like fixing up an old car with only stock parts. Where enhancing the video would be like putting some chrome wheels and a flame paint job on the car or variations thereof.

To that, I suppose I have already restored the video. I would like to enhance it tastefully by possibly removing noise and adjusting color to something that looks natural. I guess I want it to look better without looking like I did something to it, if that makes sense (no overworking or smudginess). It seems like some people want to take their VHS and make it look like it was never VHS, this is not my intent. In fact I don't mind it looking like VHS, it dates the footage perfectly. However, some of the footage is full of chroma noise and bad white balance. Some has blown out exposure from 2nd and 3rd gen SLP copies, just awful looking but I'm sure it can be spruced up a bit.

I thought I'd ask you all for advise before I go bushwhacking away on filters.

lordsmurf 01-05-2011 03:49 PM

Yep, I'm replying backward to this one. jmac first, then magilla (the OP).
Here goes...

Quote:

Originally Posted by jmac698 (Post 14054)
To restore is to return something to it's original state;

Well, actually, no -- that's not the definition used by this site, or even by most in the field (be it forensic or entertainment), because that's not really the goal. Sometimes it's not even possible. The single goal of restoration is to make it better. Or to be more accurate, to make it the best quality possible, given the contraints of technology. And to a lesser extent, sometimes the constraints of time and budget.

This is, of course, a definition limited to video -- not films, fine art paintings, etc -- just video. The "original state" of video is very often not all that good, and in many cases restoration is being sought to improve it greater than it's original condition ever was (or in some cases, could ever have been, given limits of how/when it was shot or recorded).

Quote:

in this case it would be to view a VHS as if it just came out of the camera.
Nope. Not for video. The goal would be to make it appear as good as it was seen through the lens -- not necessarily how it was recorded onto the medium in the camera. The recording media, such as VHS or 8mm, was the weak link. Or the camera's recording ability.

Same for VCRs that record off TV. The goal would to be to make it look as good as the original broadcast signal it was recording, not the recording itself which is for whatever reason degraded.

Quote:

Why do we restore? My guess is to preserve a feeling of authenticity, and details of techniques used at the time.
Nope, again, that's for fine art paintings or film -- not video. With audio, it can go both ways. Sometimes the original recording needs to be restored to a good original condition. Other times audio needs to be restored to overcome the recording medium, to re-create the original sound before it was butchered by the device.

For example, tape hiss and consumer audio cassettes. The original tape was a dull sound, and added noise. Restoring this tape would remove the hiss, and calibrate the audio back to normal levels and EQ -- maybe even removing audio distortion (red-lining of values, which can cause crackles/etc).

Quote:

You can also enhance.
Enhancement needs to be considered by the same ethics and values used by darkroom photojournalists. Anything that improves the quality without changing the content is acceptable. Adding contrast is fine, adding a person is not. The same is true of audio -- removing hiss is fine, removing vocals is not. Restoration is not alteration, and alteration is not restoration.

Archiving "as is" or "true to the format" is also not restoration -- there is a distinct difference in restored and archived, and a librarian at an archival facility can probably (hopefully) give a good insight on that.

Quote:

You can also fudge the definition of restore to say, you want it to look as realistic as possible; as if you could go there again with modern equipment.
That's not fudged -- that is the definition. Forensic restoration is built on this very endeavor.

Quote:

That means making skin tones look proper even if "authenticly" the camera always had bad white balance.
And that's a great example of an "entertainment" restoration. (Skin tones don't really matter all that much to forensic work, where the goal is to get a good ID, not to correct for his/her exact level of sun tan.)

Quote:

VHS doesn't fade colors over time AFAIK.
This is true. So very true. I beat this drum last year in a thread at VH, because of all the ridiculous dumb "professional" (not!) services that advertise with things like "tape fade." Based on the most hardcore broadcast/analog engineering docs I could fine, it's complete BS. Tapes don't fade, as the signal is magnetic and not optical.

Quote:

Theoratically, except for TBC and repairing dropouts, you wouldn't do anything to a VHS, as it probably doesn't look that much different than originally.... Even TBC is cheating; unless it was part of the camera playback, it's not authentic.
This doesn't matter. You're trying to preserve the content, not the medium. In that process, it's often a desire to improve the content. As stated above, this is often to override the damage done by the medium itself, to the scene that was recorded.

Consider a nice B&W photo.
-- Preserving it means to retain all the original sepia tones caused by the chemical development process, possibly even the various imperfections (hairs, grains, dust/dirt specs and spots, etc).
-- Restoring it involves removing the hair/yuck, cracks, and balancing the sepia. Sometimes completely removal of sepia is desired, as that can be a purely aging issue, and not how the original looked. (I have prints that are starting to approach 20 years old, and the day those came from the wash bath, I assure you the images were black and gray and white -- and that's it. Now they have red/yellow tints.)

Quote:

The equivalent of EQ is sharpness.
I don't know off-hand that I could compare audio EQ to anything visual. Maybe NR.

Quote:

Another point is that all your work on color grading will be for nothing unless you are using a calibrated monitor. When you've made it look perfect on your equipment, you may be disappointed to discover that it doesn't look the same on another TV. Most TV's these days vary quite a bit in their look and it takes a lot of work to calibrate everything.
And some it simply can't be calibrated because it's designed so poorly. Many consumer TVs are this way -- especially plasma sets. Terrible contrast.

Quote:

Originally Posted by magillagorilla (Post 14053)
In audio archiving, noise reduction is the work of the devil. Audio enthusiasts will come after you with pitchforks for using it.

I think audio enthusiasts that enjoy hearing hiss are certifiably insane. Some of the work I do with music distributors completely goes against some of the nonsense I see in "audiophile" forums. I can understand not altering the audio, or even erring on the side of leaving noise so as not to harm the audio's fidelity -- but to leave noise that was caused by the recording medium and can silently be removed? That's just nuts. I want to hear the audio -- it's why I'm listening. There's nothing enjoyable about "ssssssssssssssss" while I try to do it. I refuse to listen to hissy audio. Anyway...

Quote:

NR kills detail in audio and I am learning through experimenting it does the same for video. In video it creates splotchy patches and blurs out detail.
You'll find there's no way to avoid this, if you hope to save the video out to a watchable format. Compression schemes by their very nature soften some parts of a picture. MPEG-2 and H.264 will both make some decisions on what's "alike enough" to make the encode.

Failure to pre-filter will result in blocky/ugly video at most "format" bitrates (DVD, Blu-ray, web streams) -- and this is where audio and video differ. You could leave all the noise in an audio file, but you can't do that for video. You will create FURTHER damage to the signal clarity by NOT filtering any. This is why I got so involved in filtering and restoration in 2001, because I saw how badly DVDs would look when no effort was put forth. It's worse on homemade sources. (Note that I had already been doing restore/filter work on tapes, back in the 1990s, but it was more by choice then. With DVD, it became necessity.)

Quote:

Fair play in audio restoration is the use of EQs and notch filters, de-clicking/clipping (done properly) and the occasional compressor/limiter. These tools can be used to shape the sound to make it properly represent the original event. I am light handed when it comes to these tools. I often reduce undesired frequencies rather than boosting desired frequencies (boosting being an armature mistake).
Okay, so you're not one of those that like to hear his hiss. That's good. :)

We're really not too different on our approach to filtering. Reduction of undesired portion of the audio (frequency carving, as I call it) is how most of my audio filter work is done. Sometimes cancellations are possible. Boosting is only acceptable when the original so far far screwed up that it's the only way to pull out what little good audio is left -- when the source is more noise than audio. I run into this quite a bit on OTR (old time radio shows) that were recorded with wire, cones, vinyl and others.

Quote:

How this translates to video I'm not sure and I am looking for guidance. When I am correcting color is it better to reduce the undesired color like reducing unwanted frequencies in audio?
Maybe. Sometimes it's an issue of temporal compression to average colors. Sometimes it needs further chroma reduction works. Sometimes it really is as easy as altering the white balance or reducing a single color.

Quote:

After hours of tweaking color, contrast, and white balance on a clip I get lost. I think it looks better but maybe it's too artificial and I am punching settings too high. How do I benchmark and determine if I am helping the video rather than just repainting each frame with a digital facsimile?
I'd have to see samples, both as stills AND as moving video. (You can attach 8MB clips here to the forum, so as MPEG-2 I-frame files, not AVI. Premium Members get temp FTP access, as needed, for larger test files -- preferably under 64MB each so an admin can attach them to the forum later on.)

Quote:

what I use to encode will be another thread
Excellent. Look forward to it. :)

kpmedia 01-05-2011 03:57 PM

Quote:

Any suggestions on a reasonably priced calibration package?
Get a Spyder: http://www.amazon.com/gp/redirect.ht...reative=390957

Quote:

I like your point about restore verses enhancement. I think those terms get minced in these forums. To me a restore would be getting the tape to play and nothing else. Kind of like fixing up an old car with only stock parts. Where enhancing the video would be like putting some chrome wheels and a flame paint job on the car or variations thereof.
Archive/Preserve - leave it as is, noise and all
Restore - improve it without changing the actual content (the noise added by the recording medium is not content)
Enhance - to alter the content

In photography terms:
Archive/Preserve - print/scan, no corrections (this is a "picture", not a photo) -- like a JPEG from the camera
Restore - process photo in darkroom, Photoshop
Enhance - Photochop, makes an "illustration" (no longer a photo) -- HDR goes here, too. Overused gimmick.

NOTE: You can archive restored versions. There is some contention here amongst galleries and libraries, and I think it's stubborn and bullheaded for "uncorrected" to be the mantra espoused by some places. Archive them both. Some people want the best version available to see/hear, not the one that's falling apart.

Quote:

It seems like some people want to take their VHS and make it look like it was never VHS, this is not my intent. In fact I don't mind it looking like VHS, it dates the footage perfectly. However, some of the footage is full of chroma noise and bad white balance. Some has blown out exposure from 2nd and 3rd gen SLP copies, just awful looking but I'm sure it can be spruced up a bit.
Even my best "magic" won't make a VHS tape look like anything other than a VHS tape -- some things are inherent, such as resolution. But noise created from the medium itself is unacceptable and should not be retained from some false idea that it belongs. So kudos to you for removing chroma noise, bad WB, etc -- there's zero reason to retain problems.

To me, it's like keeping a broken leg instead of using a cast. Fix it.

magillagorilla 01-05-2011 09:04 PM

2 Attachment(s)
sample1
This is badly shot 2nd or 3rd generation SLP. The color is blown out and there is plenty of chroma noise. This is one of the worst clips I have.

sample2
This one is pretty bad too. Most of my clips are in pretty good shape. If I can get a grip on how to clean the really bad ones I think I'll be headed down the right path.

Thanks for any guidance!

jmac698 01-05-2011 10:03 PM

Hi,
Interesting discussion! I didn't know that the restoration definition differed in the field of video; for the record I did check on the definition before I replied. Here is what I found:
Quote:

From Dictionary.com:
restoration:
3. a return of something to a former, original, normal, or unimpaired condition.

From UCLA Film & Television Archive
http://www.cinema.ucla.edu/preservat...servation.html

In general, the term "preservation" refers to the process of gathering the best surviving materials from a film or television program and transferring them to the most stable format possible. "Restoration" usually refers to even more time-consuming and complicated projects in which altered or missing material is restored to the film, bringing it as close as possible to its initial release form.

http://www.lowrydigital.com/restoration.html
From Lowry Digital, restorers of famous films:
Our goal is to return catalog movies and television shows to their original quality, delivering the same experience as a show print from the premiere or the initial broadcast.

from a commerical restoration company:
http://www.dcvideo.com/video-service...storation.html

What is Videotape Restoration?

Videotape restoration is a more advanced process in which an unplayable tape is made playable with methods that are outside of the normal re-mastering methods.
To me, mistakenly or not, all the defintions seem to agree; to return to an original state. There is no clear resolution over what techniques exactly would qualify for this term.
Obviously in film there is some controversy; do colorized versions count as restorations, even if the original director supervised it, even if he meant to shoot it in color? Does "Han Shot first" qualify?
I can see leeway for the argument that re-creating the original broadcast regardless of the method used to record it would be a restoration.

As far as restoration vs forensic, it seems by your own admission, the purpose of forensics is to answer a question about a particular fact or detail regardless of the means necessary to obtain it; in your very own example, to see the structure of a face regardless of it's skin tone, where restoration would include seeing the skin tone as faithfully as possible.

In terms of film there is another important distinction; there is considered to be an original artistic intent relevant to the processes in use at the time; film grain can be used *on purpose* to achieve a specific look. To "restore" such a film by removing the grain would be destoying the artistic integrity of the film.
Though I accept as respect your opinion, I'm still not convinced there isn't a *controversy* over the details of the definition.

Anyhow, my understanding of the original poster is that he is seeking "natural enhancement" (no pun intended :), that is, improving quality without introducing a subjective perception of un-naturalness. Certainly, this everyone's goal.

I would improve the definition of archive/conserve/preserve as "prevent the occurence of age-related detoriation; to maintain extant quality without change", restore might be "to remove medium related defects which weren't present in the original", and enhance as "to alter the original form to add qualities not present in the original". These definitions might overlap.

EQ vs Sharpness: actually these are exactly the same. If you took a video signal and put it into a WAV file, then put EQ on it, then converted back to a video, you would have sharpness. I've experimented with this; treating video as audio to see what would happen. For example, the simple formula for average (pixel N+1 added to pixel N divided by 2) has an exact translation into the frequency domain as a low pass filter; see the blue line in this diagram:
http://ptolemy.eecs.berkeley.edu/eec...esponseRA.html
A sharpness filter might subtract the original samples from the low-passed samples and be the same graph "upside down"/mirrored (?), or high pass filter.

VHS colors don't fade - from my technical perspective, the reason would be that color is stored as an FM signal. In this modulation, a certain frequency represents a voltage value for the color signal. See
http://books.google.com/books?id=NXE...%20vhs&f=false
A frequency of 629KHz represents 0, a higher frequency represents a higher voltage, respective for a lower voltage.
The frequency of the voltage going up and down (and the frequency of the 629KHz) gives the tint, and the amplitude gives the saturation. A wild swing from (say) 600-700Khz means a deeply saturated color.
This is an indirect way of storing color. If the tape "fades", the carrier (629KHz) is less. but as long as it can be detected, the color it represents doesn't change at all.
I'm not sure how well I explained that but I won't write an engineering course here. I'm not sure what color a dropout will turn out to be; but some specifc color for the duration of that dropout, perhaps greyscale.

I agree with your example of the photo; to me that's just restoration.

You talk alot about compression artefacts, but he was referring to VHS, VHS-C and some mini-DV. I think he talks about NR being too unnatural in terms of cartoonish, or blurry. That's definitely a tradeoff in all denoising methods I've seen.

Interesting link, upsizing methods:
http://www.general-cathexis.com/inte...ion/index.html

Quote:

simply can't be calibrated because it's designed so poorly
this is a deep subject but you can do the basics with the Digital Video Essentials or Avia disks, but what might be missing is Primaries selection, color standard (rec709 vs rec601) gamma, and some other controls - for example, you can correct for the actual monitors that color grading was performed on as a sort of re-correction... long story. See yCMS
http://forum.doom9.org/showthread.php?t=154719

I tend to ignore audio and focus of video; but it would be interesting to hear what you guys have done for audio restoration/enhancement whatever. One neat program I found is:

http://www.winamp.com/plugin/tape-restore-live/154246

- Software Dolby B remover
Approximates a Dolby B decoder in software. If needed, tape bias settings can be fixed first. This is especially useful if the sound of tapes has become dull due to tape wear and age. Also, sound card noise is deminished by the remover.

- Automatic AZIMUTH correction
If the tape head is not set exactly equal to the setting during recording, playback in mono or through a surround system will cause very ugly artifacts. This filter corrects for this on the fly.

- FM Stereo hiss removal
Removes the FM stereo hiss without damaging the real sound. (Not only useful for tapes!)

ps I have an experiment where I have the original digital file which was recorded to tape and then the tape version;
I onced owned this cassette deck -
http://www.generalmanual.com/Audio/P...sette-Deck.htm
Results - cassettes degrade severely! The high end was really lost, the Dolby tracking was lost, the tape sounded just horrible and good riddance to cassettes. I'd rather have open reel for an analog recording.

kpmedia 01-05-2011 10:26 PM

Quote:

film grain can be used *on purpose* to achieve a specific look.
This tends to be revisionist BS more than anything else. (Referring to claims by other people here, to which you likely refer, and not to you.) The idea that anybody intended for film grain to exist is silly -- it's not like they had a choice in the matter. If you used film, it had grain. If you didn't use film, you weren't shooting anything. The few experiments in tape (Twilight Zone, for example) were spectacular failures in quality and long-terms costs. Not to mention a lack of archival condition.

Quote:

natural enhancement
All restoration should be as such -- otherwise it's not restoration, it's alteration and transformation. George Lucas didn't "restore" the scene where Han shot first -- he outright altered it. (Referring to 1997 edition where Greedo shot first.) That's not a restoration. Great example.

Indeed, there are arguments to be made for color vs B&W, pre-CGI vs post-CGI versioning (example: Star Wars), and both sides have valid points.

What Lucas did NOT do, however, was leave the film deteriorated and transfer as-is, nor did he leave in obvious mistakes from the first edition (matte outlines around X-wings). Those aspects do qualify as restoration. Indeed, there have been some fans that actually restored it better than Lucas did, by hand-painting thousands of frames, using every available source as the starting point (including some of the restore work done by Lucas, but excluding his alterations).

...

magillagorilla, I'll be sure to look at your samples here in the next couple of days. :)

jmac698 01-05-2011 10:55 PM

1 Attachment(s)
Some movies have grain on purpose like the opening scene of Casino Royale. I think that example speaks for itself; why else would a modern film be shot in B/W? On the other hand, you have a point - you didn't have a choice to avoid grain completely. Also I have a great example of a bad remaster, dunno if you've heard of it but Krafterwerk Home Computer remastered was horrible. Perfect example of the unnatural effect. The audio has a "drowned" effect, to me it sounds like mistracking Dbx, or a compander set wrong. It cuts off the natural attack and decay of sounds. I'd rather the noisy version.
Another thing, the so hyped Beatles remasters, don't sound any different to me, I can't see what the fuss is about.

Anyhow.

I looked at the sample. A lot can be fixed with white balance correction. I can correct the color stripes, in fact I did this already for another poster. The sample2 has some weird color lines that I'll have to think about. Give me some time.

-- merged --

Here ya go.
The colored lines are almost all gone, the big purple haze on the left is gone, the color is somewhat corrected. The comets are gone. You should try my favorite technique, capture the same section 3 times and post.

magillagorilla 01-06-2011 09:25 AM

Quote:

Originally Posted by jmac698 (Post 14068)
Here ya go.
The colored lines are almost all gone, the big purple haze on the left is gone, the color is somewhat corrected. The comets are gone. You should try my favorite technique, capture the same section 3 times and post.


Nice, was that just white balance or did you punch down the blue on a color filter as well?

jmac698 01-06-2011 05:05 PM

I tried a number of combinations, autowhitebalance seemed to work best.
Your other sample is a bit harder, for example I can't use autowhite balance because it leaves a correction tinge on the blowouts.
The flashing colored lines at top might be harder too.

magillagorilla 01-07-2011 09:07 AM

Any hints on what specific filters and settings were used. I know this site offers a service and I don't expect anyone to give away the farm but the LS VDUB has a lot of filters. Just looking for pointers on which ones may best suit my clips. Maybe some hints on settings so I don't "over do" it.

Is there any method other than eyeballing it which I should consider?

jmac698 01-07-2011 12:55 PM

Actually I used Avisynth. I don't have time to completely clean it up, it's just a start. The scripting is really messy too :)
Code:

vid2=MPEG2Source("G:\project001a\sampleproblem\sample2.d2v", cpu=0)
vid2=vid2.crop(0,0,0,-8)
vid2
assumetff
#QTGMC( Preset="Slow" )
l=utoy
l=l.Crop(0, 0, -342, -0).tweak(bright=-10)
overlay(vid2.utoy,l,opacity=1)
u2=last
r=vid2.vtoy
r=r.Crop(0, 0, -342, -0).tweak(bright=-6)
overlay(vid2.vtoy,r,opacity=1)
v2=last
ytouv(u2,v2,vid2)

ColorYUV(autowhite=true)
tweak(cont=1.3)
separatefields
selecteven
bob
t=last
vtoy
temporalsoften(2,55,5,mode=2)
ytouv(t.utoy,last,t)
Crop(20, 0, -0,0)

Install Avisynth, save the text above as tape.avs, load the file into vdub, enjoy.
I hope you have something better than the mpg version, it was really blocky.

magillagorilla 01-07-2011 01:47 PM

avisynth
 
Thanks, I'll try it out. I don't really know how to use AVIsynth but I'm no stranger to codeing. It does not look much more difficult then editing an INI or a JAVA .config file. I just don't know the toolset or parameters at all. I guess I could read the manual.

My source is HuffyUV played back from a Pany AG-1980P. It isn't going to cap much better unless someone has a Datavideo TBC they are selling for $40. I doubt I can afford any more hardware. The huffy files are not blocky at all.

The sample was coded on a crapy 1pass low bitrate. Hopefully when I do MPEG2 2pass VBR 6000-9000mbps it will look good. Or I may go with h.264, I havent decided.

jmac698 01-07-2011 02:07 PM

Ah, that explains it.
What I meant was, there's a technique where you cap the same thing 3 times in a row. This seems redundant, but a lot of types of noise only appear the same place once.
I'm also curious if that's the case with your particular problems.
As far as the script, it will take you so far, but you'd have to learn the scripting to make any improvements, that is, unless you pay these guys to do what they do.
I did write a quick tutorial elsewhere here.

In your case, you should write vid2=avisource("") for your original file. assumetff can be removed (it only applies to mpeg2). The d2v file you don't have, I created it with a program called dgindex, but that doesn't matter now.
Understanding my script is going to be a little hard as it's messy.
If you want to code with it, get a program called avspmod. It's a nice, code completion ide and video preview.

Now in general terms, what I found is one of your heads is bad (I believe?), the comets are little white streaks that appear, they only appeared in one particular field. The color lines were also worse in that field. So I just used the good field.
Also I found that the color lines were just in one color channel, so I heavily denoised that channel.
As for the purple haze, that is transparent, so I just color-corrected that section and it comes out fine. Ironically you could also use a good transparent logo remover to do an even better job.

magillagorilla 01-07-2011 02:25 PM

hm, I hope I can get some of this done in VDUB since the filters are GUI. I'll try the logo remover. Also, I concure with your theory of a bad head. The bad head was, however, on X generation and not the final playback deck. I have cleaner tapes with no comets on playback using the same deck.

jmac698 01-07-2011 02:51 PM

Actually, you can use the script itself as input, and then add vdub filters on top of that, so it's no problem - just keep going!

lordsmurf 01-07-2011 11:57 PM

sample1 is too far gone. The video signal appears to be pure luma with blue chroma and no red (in other words, just the YCb -- no Cr -- as expected in YCrCb signal). The image is incomplete, and there's really no way to fix it. There's also therefore no reference point upon with to restore any kind of white balance. At best, we could shift colors, but recreation/restoration is essentially impossible.

Footage this badly destroyed should be converted to grayscale (B&W video).

sample2 is not quite a badly damaged, but still appears to be missing some portion of the full signal. At best guess, the original camera was damaged when this was shot, because tapes don't "fade" color-wise. The only other option is this is a copy of a copy of a copy (etc etc etc), without a TBC, at such an nth generation that it's impossible to recover a proper signal.

jmac gave it a good try, but essentially just altered the remaining signal colors to a sepia-shaded tone.

Again, conversion to B&W would be the ideal solution. A B&W signal is more acceptable and watchable than a severely color-shifted version. (Feel free to archive or "preserve" the original, but create the B&W for pleasant viewing conditions.)

Some things can't be restored, unfortunately. This is a good example of that.

Wish I had better news for you. :(

admin 01-08-2011 01:59 AM

Quote:

Originally Posted by jmac698 (Post 14071)
The flashing colored lines at top might be harder too.

That's overscan, anyway -- no point. Just mask with black.

jmac698 01-08-2011 04:23 AM

1 Attachment(s)
Well I have better news :)

The colored lines at the top are fixed, this is easy and automatic.
The color haze at the left can be fixed; there's still some left only because I didn't finish tweaking it.
The comets are gone, that's standard.
The colors are corrected; except for that yellow on highlights - I'll get to that in a bit.
It's been denoised and deinterlaced as well.
Never say never :)

lordsmurf 01-08-2011 04:37 AM

4 Attachment(s)
I spent a few minutes on this for you, seeing what might be possible.
See the attached Matrox MPEG-2 (10-bit I-frame only in AVI wrapper).

Here's what I did:

Step 1: Deinterlace with Avisynth, output to AVS file for further VirtualDub processing. Attached is my installed Avisynth plugins folder from "C:\Program Files\AviSynth 2.5\plugins" (which includes the needed plugins). You'll want to install Avisynth on your own, then "steal" the added contents from my folder into your own. And then use this script:
Code:

ffvideosource("d:\sample2.mpg")
yadifmod(order=1, mode=0, edeint=NNEDI2(field=1))

or
Code:

avisource("d:\sample2.avi")
ConvertToYUY2(interlaced=true)
yadifmod(order=1, mode=0, edeint=NNEDI2(field=1))

The first script is for MPEG-2 input, the second for HuffYUV input.

Note: VirtualDub deinterlacing is inferior to what Avisynth can do, hence this step. Everything else is fine in Vdub.

Step 2: First pass with VirtualDub, to corral some of that harsh color, and provide most of the noise reduction (NR). Median filter is what knocks out most/all of the dropouts ("comets" as some have called it). Output to lossless HuffYUV. FYI, YES those filters are in a specific order -- it is NOT random.

Step 3: Second pass with VirtualDub, to refine some of the color quality in ColorMill and add another NR pass.

Attached are my VCF settings files for both Vdub passes.

This is just a quick proof-of-concept type correction. Don't mistake this as the best I can do. It would take a good bit of tweaking with color and NR filters, to get that oh-so-perfect balance of removal vs loss, and good vs bad coloration.

My choice here is to reduce, but not entirely remove, the color. Remember that a whole channel is missing (1 of the 3 pieces, the YCb with no Cr), so you're looking to reduce that obvious problem. It cannot be corrected, but it can be minimized for your viewing pleasure. Nobody wants to watch a severely off-color video. Think of it as a color-tinted silent movie (but with sound, in your cases) similar to the famous Metropolis flick, which used gold and other additives to give pigments to otherwise B&W scenes.

Because I know you err on the side of "leave it alone", my approach to NR was very light-handed.

jmac698 01-08-2011 04:39 AM

I don't see any attachments...
Ok, that took a while.
Nice job! I admittedly suck at color grading, no practice.

lordsmurf 01-08-2011 05:02 AM

Sometimes posts take 3-4 steps to fully get up. This was one.
You saw it after my first save, before it was completely uploaded and finished for viewing.

Thanks. :)

magillagorilla 01-08-2011 01:43 PM

Indulge me a little more
 
1 Attachment(s)
Great idea, B&W, it hadn't occured to me. Thanks!

OK please bare with me.
Here is another sample of some of the old VHS. This is in better shape, it's meduim quality in my set. Most of my clips look better some look like this and very few are really bad like the previous samples.

jmac698 01-08-2011 01:48 PM

No B&W! I fixed it.

magillagorilla 01-08-2011 01:50 PM

2 Attachment(s)
sample4
EDIT: sorry I posted more junk before I realized you guys were still working on the last samples. Let me look at your results on sample2 LS. Thanks!

OK, my restoration project involves 3 different types of media VHS. VHS-C and MiniDV.

sampel5
This is a sample which is typical of my VHS-C recordings. The fat guy is me 7 years ago. I could use a bit of cleaning as could most of my VHS-C. Should I just denoise a bit.

Here is a MiniDV sample. I had the camera set to crop for 16:9 for some reason. The interlace lines seem harsh for some reason.

Should I even mess with my miniDV footage? Most of it is really clean. I guess I'm asking if there is anything to gain from filtering?

LS, Thanks for the example. I'm going to mess with those settings tonight.

Just curious, what is the advantage of deinterlacing? I'd prefer to keep my temporal resolution rather than smudging it, though I have not tried Avisynth's deinterlace. I know that VLC and my TV will handel interlace.

admin 01-10-2011 11:54 AM

Quote:

Originally Posted by magillagorilla (Post 14155)
Just curious, what is the advantage of deinterlacing?

It is required in order to run several of those NR filters -- especially the median filter.

Quote:

I'd prefer to keep my temporal resolution rather than smudging it, though I have not tried Avisynth's deinterlace.
You're not really smudging anything. It's not like VHS tapes had a lot of temporal or in-frame resolution anyway. Not that I'd want to reduce it further, ala VCD or drop-field deinterlacers, but this one is non-destructive.

The NNEDI2+Yadifmod combination works very well, mostly working to rebuild combed areas as progressive, but without simply throwing away data like most other methods will do.

A number of us have been working towards a reliable method to fully or near-fully remove magnetic tape drop-outs, and this is probably the first example I've seen where one of the proposed/theorized methods has succeeded in near-complete removal of this awful analog noise. Some of these filters and filter combos did not exist even 5 years ago. I have some client project files here that are the basis for "worst case" tests on dropouts, and need to try this script/filter combo on it.

So you're getting VERY current advice. :)

Steve(MS) 01-11-2011 10:09 PM

Well I must admit so far my adventures in noise reduction using vdub has been less than spectacular.
When correcting color, etc it has been somewhat better provided the original wasn't too bad.
No doubt most of my lack of success has been due to my ignorance of video computer software.
Further since I am on dialup, I haven't been able to look at the
images on this thread but I would like to add my novice opinion
to the dialog from the beginning,
First get the best possible VCR that you can, like some
of the JVCs recommended on this site.
Sometimes it helps to have more than one model so if one tape doesn't play on one, it may do better on another.
Next and this is important, get a hardware proc amp and do
your color,brightness, contrast adjustments before they go
to be recorded.
And I decided to bypass the computer capture and got some
really great JVC recorders.
The only video I run through the computer are the recordings
that need some extra little bit of work.
Once again for me the hardware approach for color, brightness, contrast along with several of those great JVC VCRs work best.
Unless you know what you are doing with the computer approach, you can spend a ton of time, which is fine for those so inclined to do so.
BTW, Admin I hope one of these days you can put up a guide
for basic NR and so forth using Vdub filters,maybe a set of your favorite ones, for us bunch that are computer and video computer restoration challenged.

magillagorilla 01-12-2011 12:07 AM

Steve,
What do you mean by JVC recorder? If you are talking about DVD recorders, they won't suit my needs. I feel that transferring the video to DVD just creates more work for me in the future for archiving. Plus, as individual computer files, I can create an unlimited combination of productions. DVD is rigid and I don't trust DVDr media. Hard drives are cheap and can be easily duplicated.

I agree a video processor would be ideal but it's not really in the budget. My playback decks are a Panasonic AG-1980P and AG1960. The AG-1980P does a great job.

Yes the software learning curve is high but if you can't spend money, you have to spend time.

Steve(MS) 01-12-2011 01:14 AM

No problem with how you want to approach it.
However I have seen firsthand where a tape will look not so good on one
model vcr and on another will look better, sometimes much better.
And it seems to me that proc amps do a better job with adjusting
color errors, etc.
I am using the k.i.s.s. method and thought I would mention it as an
alternative.

magillagorilla 01-12-2011 07:12 PM

I agree with your method. I just don't have the budget for multiple VCRs and processing equipment. Unfortunately in this case, k.i.s.s. equals expensive.

LS,
I finally got around to really playing with the example files you posted. You are correct, the AVISynth deinterlace looks really good. Is it recomended to deinterlace all of my VHS video. I mean, it looks pretty good to me (especially on a progressive display), any reason I shouldn't?

Your VHS YCB noCr Test Pass1.vcf settings for VDUB produce a file that crashes my media players (VLC, WMP, and VDUB). It's really weird. When I run it on a longer clip the file it outputs is un-playable. VDUB steps through the clip and applies the filters without errors but as soon as I try to open the resulting AVI, in anything, it crashes. The file gets about two frames in where 2/3 of the top of the frame is bright colored rainbow snow (digital noise) no picture at all. The bottom 1/3 looks like my video. Then all players dump memory errors. (sorry dont have those right now).

I thought my machine was freaking out or I had bad RAM or something so I ran another pass (on the same clip) using a different filter set and it plays fine. Reboot..... same problem.

I didn't have time to troubleshoot the filter set. I started to remove filters 1 at a time to see of there was an offending filter but I ran out of time to get through the all. (it was 2am and way time for bed).

My first suspest was the resize filter. Could it be resizing the video so that the AVI wrapper says the video is 720x480 but the actual size of the data is different, causing players to crash? Or maybe some kind of "out of range" setting for the AVI format? I dunno, any thoughts?

kpmedia 01-24-2011 07:55 PM

Quote:

Is it recomended to deinterlace all of my VHS video
No. Only deinterlace when required. Although the method being used here is pretty good, it's not perfect. None of the available methods are. Someday, far off, maybe we'll see one happen. It does get better every year. The only way to use a better deinterlacer later is to keep interlaced footage as-is for now.

Unless, of course, it needs to be done in order to allow for restoration, such as the median filter usage that was done here in these samples. This is an exception to the "never deinterlace" rules.

Quote:

Your VHS YCB noCr Test Pass1.vcf settings for VDUB produce a file that crashes my media players (VLC, WMP, and VDUB). It's really weird. When I run it on a longer clip the file it outputs is un-playable. VDUB steps through the clip and applies the filters without errors but as soon as I try to open the resulting AVI, in anything, it crashes. The file gets about two frames in where 2/3 of the top of the frame is bright colored rainbow snow (digital noise) no picture at all. The bottom 1/3 looks like my video. Then all players dump memory errors. (sorry dont have those right now)
Players generally don't like to play lossless video, like HuffYUV. This format was intended for editors, not players. If I try to play the "MT" (multi-threaded) variant of HuffYUV, it will crash most of my players, too. PowerDVD (or maybe it's WinDVD?) tend to work pretty well with HuffYUV codec -- go figure. A DVD player software works better than a general-use media player for that format, in my experience.

Quote:

Could it be resizing the video so that the AVI wrapper says the video is 720x480 but the actual size of the data is different, causing players to crash?
The AVI wrapper doesn't contain resolution data.

magillagorilla 01-27-2011 03:59 PM

Quote:

Players generally don't like to play lossless video, like HuffYUV
The codec is not the problem. The original HUFFYUV file plays fine. When I process the same clip using a different filter set the final HUFFYUV file also plays fine. The filter set LS posted produces an AVI file which crashes even VDUB. ....weird

Anyhow, I will have to deinterlace a lot of my footage because it's in bad shape. I am keeping the original HUFFYUV caps archived though to future proof for restoration techniques.


All times are GMT -5. The time now is 10:01 AM

Site design, images and content © 2002-2024 The Digital FAQ, www.digitalFAQ.com
Forum Software by vBulletin · Copyright © 2024 Jelsoft Enterprises Ltd.