Remove vertical jitter from interlaced footage?

[hysteriah]

I guess the title says it all. I have a few irreplaceable videofiles with vertical jitter (fields bouncing up and down). I guess I must fix them manually, frame by frame?

I know it will be possible in VirtualDub to add deinterlace filter and export the Whole video as image sequence... and then open frame by frame in Photoshop (or other image editor) to move the frame up and down... and then import the image sequence into VirtualDub, add deinterlace filter to make it interlaced video again...

It's just a few minutes of video that I have to fix, so I know I will make it if I have to, BUT! ...I'm sure there MUST be an easier way to do it?

[sanlyn]

It might seem seem a little fussy here, but what do you mean by "video files"? .mp4, .mkv, .flv, .mov, wmv, mpeg, mJpeg, .ts, m2t .....? maybe .AVI encoded with QuickTime, h264, DVCPRO, uncompressed, DivX, XVid, ProRes?

How much jitter? 2 pixels in each direction? 4 pixels? 7 pixels? Every other frame? Every 5 frames? Every 10th frame? Are they videos that you recorded or captured from another source? If so, does the original source exist? What hardware and software were used to create the video files, or were they created by another party?

The correction you suggest will result in ruined video, not to mention a huge waste of time. Avisynth and VirtualDub have anti-jitter and anti-shake filters. It depends on the video. If you post a short, unprocessed sample in the forum, someone here could take a look and offer more detailed suggestions. Otherwise, one can only guess.

If you need to know how to make and post a sample, ask. We can't even tell you how to do it at this point, because "video files" covers a lot of territory.

[lordsmurf]

For something like this, samples are needed.

[hysteriah]

Okey.
First, these attachments are just examples. I know they're garbage quality videofiles, captured from 2nd gen tapes recorded from TV broadcast. But I think they show the jitter I'm struggeling with quite well... and I guess it's easier to track the jitter because of that TV logo in the upper right corner. I converted them to MPEG-2 without sound to save upload/download time ;-)

My videofiles are mainly AVI DV PAL format (that means 4:2:0, not 4:1:1)... but I may also have to do it on a few M-JPEG or high quality MPEG-2 files (at 15000KBPS). I guess I have 5-10 files with 1-5 minutes duration that I have to fix this way, because the source footage simply doesn't exist anymore

Some files are recorded with my JVC-HR-s7600 and a DataVideo TBC-1000 to Canopus ADVC-300. These footage is bouncing mostly 2-3 pixels on regular/irregular basis (see jitter 1 example).
Others are recorded the same way but without the DataVideo TBC-1000 (and yes, they suffer from dropped frames also), then the bouncing varies more, usually 1-2-3 or 4 pixels at totally irregular intervals (see jitter 2 example).

sanlyn: Can you please explain to me why the correction I'm suggesting will result in ruined video? I've actually done it this way on a few videos (See "RUINED before" and "RUINED after" for example). It doesn't exactly look ruined to me

[themaster1]

With stab maybe http://avisynth.nl/index.php/Stab

[sanlyn]

Thanks for posting the samples. Unprocessed originals were requested, but these are processed and re-encoded. It's trouble enough that your originals are VHS-to-DV (too late to fix that now), but the usual DV artifacts are all over the place -- ragged and broken edges, motion noise, compression noise, sloppy interlacing -- and another lossy encode to m2v loses more of the slim detail that you started with.

As it is, the "ruined" version was ruined before you started. It's improperly interlaced, with a couple of frames that have reversed fields. jitter1 is unworkable, looking like telecined video with blended fields encoded as interlaced. Motion noise is worse than in the first version. Is your original DV in this same condition?

The one possibility for attempting suggestions is jitter2. Unfortunately it's been lossy re-encoded, so expect a very blah image after processing again. Some frames have reversed or displaced fields. In particular, the first few frames of Jitter2 appear as fields 0,1,2,3,4,5. The correct field order should be 4, 2, 0, 3, 5, 1. In AVisynth the fields were remapped this way:

V1=MPEG2Source("path\to\video\Jitter2.d2v")
v1=V1.QTGMC(preset="fast",NoiseProcess=2, GrainRestore=0.4, NoiseRestore=0.2, NoiseTR=2,\
  NoiseDeint="Generate", StabilizeNoise=true)
V2=MPEG2Source("path\to\video\Jitter2.d2v")
v2=V2.QTGMC(preset="faster",NoiseProcess=2, GrainRestore=0.4, NoiseRestore=0.2, NoiseTR=2,\
  NoiseDeint="Generate", StabilizeNoise=true)

v1=v1.RemapFrames(mappings="0 4
1 2
3 0
4 5
5 1",SourceClip=v2)

In deinterlaced form, some basic stabuilization was generated with stab(), and a few mnore bad frames were cleaned up using this:

v1
Stab(range=3)
Stab(range=2)
ReplaceFramesMC(284,1)
ReplaceFramesMC(286,1)
ReplaceFramesMC(351,1)
ReplaceFramesMC(353,1)
ReplaceFramesMC(355,1)
ReplaceFramesMC(369,3)
return last

The file from this first processing step was saved as Lagarith RGB32. Then the first shot in the clip (the cute gal typing) was processed in Avisynth with DeShaker v3.1. This doesn't work well with subsequent scenes because DeShaker tried to steady the singer in the frame and moved the logo all over the place. Some smoother motion in the rest of the clip was done with RemoveDirt(). Because most of the filters used are temporal, the first 1 or 2 frames of the result are kinda crappy.

What's with the changing borders in jitter2? The first few shots have 28 pixels of right border, but the last shot is obviously a cut that has 40 pixels on the right. It also looks as if some of the borders were changed around, but it's difficult to say -- we don't have your original DV's. Centering all the shots took an effort.

Unfortunately re-encoding twice from noisy DV captures makes for lower quality video. But it has a lot less jitter. Next time, please make cuts of unprocessed originals.

[sanlyn]

Y'know it must be weak coffee that makes people step on themselves needlessly....

If I haven't frightened hysteriah away at this point, I should mention that there's no easy filter for doing lots of this frame manipulation in VirtualDub. But I'm open for suggestions.

Meanwhile I have to correct myself. The code above is ridiculous:

Quote:

Originally Posted by sanlyn

V1=MPEG2Source("path\to\video\Jitter2.d2v")
v1=V1.QTGMC(preset="fast",NoiseProcess=2, GrainRestore=0.4, NoiseRestore=0.2, NoiseTR=2,\
  NoiseDeint="Generate", StabilizeNoise=true)
V2=MPEG2Source("path\to\video\Jitter2.d2v")
v2=V2.QTGMC(preset="faster",NoiseProcess=2, GrainRestore=0.4, NoiseRestore=0.2, NoiseTR=2,\
  NoiseDeint="Generate", StabilizeNoise=true)
.........

All that was really needed was this:

V1=MPEG2Source("path\to\video\Jitter2.d2v")
v1=V1.QTGMC(preset="fast",NoiseProcess=2, GrainRestore=0.4, NoiseRestore=0.2, NoiseTR=2,\
  NoiseDeint="Generate", StabilizeNoise=true)

v2=V1
.........

Being stubborn, I made more notes on the "RUINED" samples to show a faster way of fixing some frames. Have to take care of Valentine's Day first. Back later.

[hysteriah]

First of all, thank you very much for the effort you put in this.

I'm sorry for not posting the unprosessed DV files. I just thought it would be to large filesize to upload. I actually had trouble uploading the "RUINED before" and "RUINED after" files that isn't even 40MB. I lost connection 3 times while uploading them. AVI DV is BIG compared to MPEG. And I just wanted you to see the jitter that I'm struggeling with and maybe give me some advices. I didn't expect you to actually work on the files the way you do ;-) But thanks a lot :-)

When it comes to the changing borders in jitter2, I don't want you to mind them at all. I have no idea why they're changing like that, but it's a part of the TV broadcast here in europe back in the late 80's... I know it's not right, but it is outside the overscan area and I think it's a part of the "charm" with old analog recordings from that time... so I just want to leaving them "as is". I think it's more anoying to see the TV logo jumping around on screen than the boarders outside the overscan area changing a bit ;-)

All I actually want to do is making the TV logo in the upper right corner to stand perfectly still, not bouncing up and down like it does now.

As you say, this is already ruined footage. But I'm not so sure if t's because of the DV format or if it's the VHS tapes they come from. You must remember that this is garbage quality video from 2nd gen VHS copies. I have no ambitions to make them perfect quality. That's simply not possible. I'm satisfied with them as long as the TV logo is not bouncing up and Down, cos that's really anoying!

When that's said, I must admit that I'm really not into this "AVI Synth script" thing. I understand that there's no easy filter to apply that can just fix my videos.... but I was hoping for something more like a "WYSIWIG" way to fix this. Isn't there a video editor that let me play around with the frames/fields, frame by frame... so I can move my frames around almost like a Puzzle?
I mean; this actually IS possible with the combination of VirtualDub and Photoshop and it's not really that hard at all... but it would've been SO MUCH EASIER if it could've been all done inside one Application. I mean, what about Adobe Premiere or After FX?

[sanlyn]

Yes, those connection problems. Admin reports they fixed a few things last couple of days.

Quote:

Originally Posted by hysteriah

When it comes to the changing borders in jitter2, I don't want you to mind them at all. I have no idea why they're changing like that, but it's a part of the TV broadcast here in europe back in the late 80's...

Crazy yanks do the same thing here. Have to live with it, I guess. Could always leave the maverick borders as-is.

Quote:

Originally Posted by hysteriah

All I actually want to do is making the TV logo in the upper right corner to stand perfectly still, not bouncing up and down like it does now.

Some of that is avoidable, some isn't. If you move the frame, the logo moves with it. Biggest problem is when the image hops behind overlaid logos. Avisynth's Stab() stabilizer does some motion interpolation a bit, but it's not always perfect. And as they, you can't always have everything.

Quote:

Originally Posted by hysteriah

As you say, this is already ruined footage. But I'm not so sure if t's because of the DV format or if it's the VHS tapes they come from.

Both, but mostly the tape and some obvious playback problems (e.g., displaced frames).

Quote:

Originally Posted by hysteriah

When that's said, I must admit that I'm really not into this "AVI Synth script" thing. I understand that there's no easy filter to apply that can just fix my videos.... but I was hoping for something more like a "WYSIWIG" way to fix this.

I used VirtualDub for WYSIWYG monitoring of Avisynth's output. There's also AvsPmod GUI, which is buggy, and I'm not afraid to admit that I hate avsPmod.

Quote:

Originally Posted by hysteriah

Isn't there a video editor that let me play around with the frames/fields, frame by frame... so I can move my frames around almost like a Puzzle?

Not that I know of, although you can move images around within a frame in AfterEffects, which I often use for color work and special effects. But moving one frame to another...not that I know of.

Quote:

Originally Posted by hysteriah

I mean; this actually IS possible with the combination of VirtualDub and Photoshop and it's not really that hard at all... but it would've been SO MUCH EASIER if it could've been all done inside one Application. I mean, what about Adobe Premiere or After FX?

Neither. The Photoshop method has its cost, and the effects are often more annoying than damage in the original. How would you do the following in an "editor"?

v1=v1.RemapFrames(mappings="54 53
53 54
756 755
755 756
761 760
760 761
1012 1011
1011 1012
1302 1301
1301 1302
1623 1624",SourceClip=v2)

This code works with two newly created (in memory) versions of the same video, each of which contains the same frames. The lines replace frame 54 in the first clip with frame 53 in the second. Then the original frame 53 in clip #1 is replaced by frame 54 of clip #2. And so on. Reason: frames 53 and 54 in the original video are in reverse order.

Maybe someone out there knows of an "editor" that can interpolate a new, good frame from a bad one:

ReplaceFramesMC(506,1)

This code uses motion compensation to create a new frame using the data in frames 505 and 507, and places a new frame in "bad" frame 506. In a different context, this can often be done with After Effects motion tracking.

The attached Photoshop_Method_4fps.mp4 shows how the interlaced fields look individually during playback in the your "after" clip. Objects change shape in alternating fields, which are also darker, distorted, and more noisy. This is annoying and wiggly on playback. It also illustrates that the "interlaced" original is really progressive, and badly interlaced. The frames play at 4fps.

I prepared a new video based on your "before" sample. The attached before_via_Avisynth_4fps.mp4 plays each progressive frame twice at 4fps from the new version of the "before" clip that I made. Successive frames have the same luma and chroma levels, etc. The frames are duplicated here for comparison, but they're not duplicated in the new video that I made.

[sanlyn]

New version of the "before" sample, made using methods similar to the earlier clip. The new version saved useable frames from the original, then discarded all the odd numbered fields, which had a obvious flicker and were badly distorted. The mpg is PAL 704x576, 25fps progressive. 4:3 standard def video doesn't usually have any data in the 8 pixels of each side border. This varies, and some players place the 4:3 frame on one side of the frame with a 16-pixel border on one side.

Could be tweaked, but the color loss would still be horrible.

[hysteriah]

I'm sorry for not answering before. I've been very busy this weekend.
I see. Really nice work you've done with motion compensation. It's MUCH smoother motion now :-) I understand not that I really must learn this

But I don't have time for that right now. I've gotten myself a "New" VCR now (Panasonic NV-HS960) that seems to work well... so I will try to focus on re-capturing all footage that is replaceable first, and then I have to focus on these irreplaceable footage in the end. Maybe I don't need all of them when I've captured all the other stuff ;-)

Anyway. THANKS YOU VERY MUCH for wonderful help!

[lordsmurf]

I wish I could help more in topics like this. Stupid slow internet.

I may be able to do better than sanlyn here (or maybe slightly worse!), but that's about as good as it will get. I want to try some of my custom Avisynth work on it. One of the more interesting things I've done in the past is to pre-stabilize (custom filters), discard half the frame rate, then stabilize again (more custom filters). It's actually quite decent. No, losing half the fields in not ideal, but the net effect is better video. And that's what restoration is about!

[sanlyn]

True, the last result could be tweaked. I used QTGMC and discarded odd fields, which were distorted. If you use QTGMC and view all fields, you see that the "original" 'way-back-when was progressive. Main object was cleaner frames and the correct frame sequence. Have a feeling that at some time in its dark past this video was telecined. Would've been a good idea to let QTGMC retain some noise, though.

[lordsmurf]

Now I'm curious what's better: drop a frame, or drop a field. Interesting! Added to my testing docket.
I'm currently sorting my hard drives.
FYI: I really want your input here: http://www.digitalfaq.com/forum/vide...html#post36694

I try not to be too picky when it comes to advanced VHS restoration work. Trade-offs happen. That's just a limitation of QTGMC.