A tape that can't be digitized? (Dropped frames?)

[moxiecat]

I'm going nuts here with a VHS tape. First, it's a tape that's in good condition--a wedding tape from a videographer. SP, Hi-Fi. About 10 years old. No physical problems.

When I play it from the VCR straight to the TV (either composite or S-video), everything is fine. Nice clear stable picture. This is true on many VCRs I've tried, from my good JVC units to the cheapie Sony in my den.

The problem arises whenever any digitizing device is thrown into the mix.

For example, if you play the tape on the JVC and output it to the DVD recorder (instead of straight to the TV), the picture begins jittering and dropping frames. It's a complete mess. If you run the JVC to an external TBC and then into the TV, same thing. The ADVC-100 converter does the same thing. Anytime any digital device is introduced that gets in the way of the straight VCR-TV or VCR-VCR-TV run, the picture begins skipping and jittering.

Using the internal TBC on the Panasonic AG-1980 has a similar effect: the picture begins flipping uncontrollably when the TBC switch is flipped on--and this rarely happens on the AG-1980. So I knew right off the bat that this is some kind of signal problem.

I have even tried dubbing the tape from a JVC to JVC (which works fine), and then playing back the dubbed tape through the DVD recorder. But even though the dubbed tape plays fine when hooked directly to the TV (just like the master does), when it is played through the DVD recorder, the problem is there too. This shows that it's not the tape itself, but something in the video that's on it.

At this point, I have no idea how to get the video onto a DVD (or even into the computer) since all digital converters seem to go haywire with the video signal. Any ideas? (I am officially out of them and desperate. Thanks.

[jmac698]

I bet I can reconstruct anything

Don't know much of the problem, but could you record 3 short clips of the same scene, only a few seconds needed, I can analyze the videos and make a theory of what's happening, but also if I can piece together stable frames.

[moxiecat]

I recorded a clip from the videotape. I hope it is helpful.
http://www.timelessdvd.com/jvc-with-ext-tbc.mov

The tape is being played on a JVC HR-S9911U with the TBC on, through a Datavideo TBC-3000, and finally through a Canopus ADVC-100.

If more are needed, let me know. I'm not sure why you need 3 short clips of the same scene though? This clips is illustrative of the problem. Jitter and frame skips. The frame skips seem worse when the video is fed through the DVD recorder. If I play this same video straight from the JVC to the TV set (no ext. TBC or Canopus), it is perfectly stable and consistent.

Thanks for your help.

[jmac698]

I do need more independant captures of the same scene. That's the whole point, if frames are missing in one, they won't be missing in the other. I've had a problem before where my capture card dropped frames, and I had to reconstruct recordings.

I even made a tool to mesaure this, http://forum.doom9.org/showthread.php?p=1462931

[jmac698]

If anyone is wondering, the problem is like slow motion/freeze frame/time lapse effect.
I found the problem. It's the playback you are using

The video is fine. Try another playback program.

[jmac698]

Ok, having got past that first problem I see a second problem. I notice a slight pause in the motion in the fence as it pans past the guests. It's also very combed at that point.

There seems to be dupped fields. There's a plugin that can fix that.
Having another copy of the same scene will still be interesting though.

Try this for yourself:

DirectShowSource("C:\Documents and Settings\My Documents\Downloads\jvc-with-ext-tbc.mov")
SeparateFields

Go to frame 430.

My first problem was that VLC doesn't play it properly (nor did MPC), but it plays in media player with ffdshow. It's in DV format.

[moxiecat]

OK--sorry, I did not understand your reasoning.

Here are two more runs of the closeup scene with the bridesmaids:
http://www.timelessdvd.com/jvc-with-ext-tbc2.mov
http://www.timelessdvd.com/jvc-with-ext-tbc3.mov

Keep in mind, though: the problem happens when a digitizer (either the external TBC, DVD recorder, or Canopus capture card) encounters the video. When the video is entirely analog (VCR to TV), it is fine.

So there must be a specific, technical reason why the signal from this tape cannot be digitized in a stable fashion. I've never seen instability that is just caused by the A-D conversion. If I could figure out why this is happening, and a workaround, that would fix the problem.

Thanks again for your help.

[moxiecat]

I didn't see your earlier posts before posting the two new links.

Yes, I'm working in Mac. These are DV files in a MOV/Quicktime container.

This issue does not have anything to do with Mac or MOV files, or whatever software player is used to play the files. Before I captured the clips for you, I was working entirely with my external TBC and my DVD recorders--not on the computer at all. The problem showed up in the hardware path. I only tried the Canopus as a third device to see if the problem reacted there as well, and it did.

Trust me: the issue is something internal in the video signal that makes a digitizing device react badly. Analog hookup, it's fine. Put any digital converter in the mix, and it jumps and skips.

[jmac698]

Oh, I agree. I have this problem with my capture cards, and I investigated it thoroughly, resulting in my test suite glitch analyzer:
http://forum.doom9.org/showthread.php?p=1462931

I don't know how Mac works internally, but in Windows it's something like this:
The timing of the VHS signal is variable, of course, due to being mechanical and not perfect. The way the drivers and programming setup of windows is, you have to declare a framerate *before* you start the capture. This leads to dupping some frames. After a dupped frame there is usually a skipped frame, as it catches up. I found a definite pattern to this in windows.

Anyhow your problem seems simpler, in windows a whole frame is dupped but you only dupe a field. There's a script with dedupe plugin and avisynth, problem is that works for PC/Wine. I don't know anything about Mac, but I could fix it. I'm sure you have a version of Wine for the Mac.

There's probably a way in virtualdub too, but I'm not too familiar with that program. And I'm not familiar with any other tools like After Effects etc.

Anyhow I'll make a fix later and show you the result.

[moxiecat]

The thing is, I've NEVER run into this issue before. That is what makes me think it is something specific to this tape (or more specifically, its video signal).

You're losing me with your software talk, to be honest.

Most of my capturing is done directly to the DVD recorder using a good VCR/TBC/proc amp/detailer setup. When I capture to the computer it's usually on a Mac with Final Cut Studio. But I'm looking forward to your work. Thank you for your efforts.

I was ideally hoping to find a method of altering the signal so the DVD recorder (or external TBC) will accept it and ignore whatever is buried in the video that it currently dislikes. Or maybe there is a device or unit that can alter a signal enough to allow the video to pass through correctly. I thought I was on the right track with making a dub of the tape, but whatever is wonky in the signal got transferred right into the dub as well. So short of shooting the "good" video signal from the TV with a video camera, I'm at a loss. (And yes, I'm joking about doing that--it's just the only thing I can think of to separate the video from the original signal, if that makes any sense!)

LordSmurf....are you out there? I am curious as to your thoughts on this.

[lordsmurf]

There are dropped frames, but I can't think of an immediate cause.

Some tapes are so screwed up that a TBC actually makes them worse. This is not very common, and I encounter this maybe a dozen times per year at most. In these cases, the TBC has to be bypassed -- both the internal VCR TBC, and the external TBC, and then you have to take your chances with the recording device. (It usually fails.)

Sometimes the Panasonic ES10 as passthrough can save you. Do you have one of those available?

jmac is going over some of the more advanced ways to addressed digitized damage. (Much of which I enjoy reading, too.)

[moxiecat]

Man, I was sure that there was a specific reason as to why the signal becomes erratic when it's processed digitally, and I just didn't know enough "tech" to know why! I guess there isn't.

I was thinking of the ES10. I don't have one, but it's something I should have so I may as well just track one down and try it. I was under the impression that that unit was mainly for top-screen tearing though--has it been useful to you in other situations too?

The closest I came to fixing this situation was turning off the TBC on the JVC and using the image stabilization instead. The problem was still there (especially in fast-motion scenes) but it was minimized to a degree. If I can't figure this out, I think we'll just have to go with that solution.

Thanks for the help.

[jmac698]

Don't worry I figured out the problem and it can be fixed, just testing the solution now.

[jmac698]

http://www.sendspace.com/file/50w09b

dir="D:\project001a\dedup fields\"
f1=dir+"jvc-with-ext-tbc.mov"
FFIndex(f1)
AudioDub(FFVideoSource(f1), FFAudioSource(f1)).converttoyv12
qtgmc
selectodd

Very simple, I was trying to fix the top fields too but still having troubles.

[lordsmurf]

jmac, I think we're approaching this from two different starting points.

I'm assuming the video was never fully captured, and that many frames are missing.
You're assuming the video is there, but with lots of extra frames.

I think I'm right, however -- the video was not completely captured, so there's no way to fix the flawed digital file.

[jmac698]

Well the problem is, there's missing fields, top fields only actually. For example, capture 1 is missing fields 20,120,132,150 and 208. Capture 2 is missing 2,6,130,140,164, and 222. You can easily merge the missing fields from each file, as they are distinct. I have a big scipt to do this. It's getting out of sync at some point, just have to fix that.

Look at this labelled video to see what I mean (single step it):
http://www.sendspace.com/file/n55b91

In this case, it doesn't matter than top fields are missing, if you just deinterlace.. yes, deinterlacing is evil So I'm working to fix the top fields too. I can do this by substitution from the other copy, or by motion compensating. Even with interpolation, it's only 1 field every few seconds, you'll probably never see it.

[moxiecat]

Wow--that's quite impressive. It's interesting that the two (or three) clips do have missing frames.

Here's the thing, though. This is a video that is almost two hours long. You are working on a five-second clip. Is this process feasible when expanded to two hours of footage? To be candid, this is a simple transfer of a tape for a customer. There is no extra funding for spending additional time restoring it (especially since the customer would legitimately believe there is nothing wrong with it)...even if I understood and had the software that you're using, which I don't. I don't mind putting some extra effort into a customer's job for free (and I've already spent a lot of time trying the tape in different machines, with different settings), but significant additional software/editing time is just not going to be cost-effective.

Yes, I know that sounds cold, especially in light of the work you are doing, for which I AM very appreciative. But unfortunately, I am forced to consider other projects that are waiting here as well. That is why I was ideally hoping to find a method of capture that would circumvent or suppress whatever problem is buried in the signal that upsets the digitizer. (My video camera idea is a bad technique, but does express a legitimate solution.)

My problem is that this customer probably does not think there is anything wrong with the tape, and he would be right. If you just watch it via a VCR hooked up to a TV (which is probably how they do), it plays just fine. It's only when a digital device or capture device is added that everything goes kerflooey. So this is very frustrating.

Let me say again: I appreciate your efforts very much. But I would also appreciate your take on how involved the restoration solution you are working on will be. Because if it's quite long and involved, I'm not sure there is a point in either of us continuing to spend time on it. I hope you understand where I am coming from... thank you again for your time and work!

[jmac698]

Oh,
You didn't explain your circumstances. Well, I have these problems myself so I'd like to solve them. Anyhow, there's no need to go further for your project, because the script I posted works fine. I don't know what the original problem is; no one else has stepped up with a solution, so as it stands, processing is the only solution.

Since you have a Mac and it would take a bit of work to get these PC solutions working for you, tell you what - if you wanna upload the whole thing I can process it for you.

We gotta agree on codecs though, I can send it back in DV is you like. It will be 29.97fps progressive - as -interlaced.

[moxiecat]

Oh my goodness....you are so generous!!!

This is going to be a huge file, though, for a 2-hour video in DV. Are you sure? (I'll have to check on my server space as well before saying yes.)

Alternately, I could easily give you an MPEG-2 file of the whole thing, if that would be an option, since I already have the whole thing in that format. Let me know.

Thanks again, really, I can't thank you enough for your help with this. This issue must really be a rare thing, since no one else has any ideas. I was sure someone would have an idea for stripping some internal component out of the video signal in order to get this to work, but I guess it really is a random event.

[jmac698]

Well, I'm thinking of these poor people that want to watch their wedding. I would go to any reasonable effort to make a customer's tape of acceptable quality.

Speaking of which, I wouldn't want to deal with mpeg2, I don't want it to end up being blocky. It's best if we deal with high quality codecs and then you can take responsiblity of the mpeg2 encoding yourself.

You can save space if you give me the video only. You can put the files in parts too. It will take me overnight to process so it would be best if you could get started now. In fact splitting it into parts would streamline our workflow.

Probably best to split at scene changes to avoid risk; in case there's stuttering in the audio eventually etc.

And hit that thank you button

I can d/l 2MB/s from a good server, so 100G at a time is no problem.