Software TBC started, progress update?
 

I have been working on various TBC techniques for sometime, mostly workarounds in rare cases. Finally I know a good technique to do software TBC that is fully general, with good results, and is simple to implement. Nothing written yet, but after about a year of thinking about this, there's progress :)

(admin might be interested in this).

Could be something in a few weeks, never know :)

the title restriction is really annoying.

Well, titles are an important part of SEO, which in turn is important for helping people find accurate results when they search the web. When some goober puts down "help pls" as a title, that helps nobody. The title should describe the discussion that is about to take place, not reflect the txt message of a 14-year-old girl. So that's why that had to be put in place.

As far as the virtual TBC software goes...

I have some ugly footage, if you'd like to test with it. It's not horrible timebase errors, but just lots of little small ones that make the video crappy. If you're interested, I can send that to you. In fact, if you're able to find a corrective method sometime soon, I may even be able to pay you a few dollars for your assistance, since it would be something use on a for-pay project. A donation for your efforts. (The sample clip is from a non-private segment, a commercial from the recording.)

I have an update. I finally managed to tweak a capture card chip to give me the raw TV signal. That means I have access to the HSYNC just like a hardware TBC does, except now I can make a TBC purely in software, and it should work with most capture cards. Not only that, but I can get much higher resolution (1700 pixels) and all the lines (full 486 and also entire VBI). I can see closed captioning, VITC, teletext, laser disc frame/chapters, XDS, and all the other hidden data there.
I'm still working on it, but here's a picture:
http://screenshotcomparison.com/comparison/86066
Obviously I'm lining up that bright line, but I have a bad feeling that's not actually my HSYNC. I'm expecting more like a thick black line instead. That's probably why it's not working yet.
Anyone know what that bright line would be? I think the green line after it is the colorburst.

There's a chance I used to know what that was. You'd really have to research analog broadcasting and VCR/recording technology. Or find somebody that can remember everything off the top of their head in the post-analog era. This is all interesting, the attempt to create a software-based TBC, and I'm rooting for you, but I don't see it happening anymore than I foresee a cure to AIDS or cancer. It's an extremely complicated problem.

Please do keep reporting your status on this.

It would be awesome if you're someday able to create a filter that can resolve timing errors, even when original sync information has been discarded. Even more impressive will be if it's able to correct videos that have been abused a few times with re-encoding (Youtube uploads, for example).

It's getting much better,
http://www.sendspace.com/file/jdh9j9
That bright green line is actually not part of the video, so I was lining up to the wrong thing.
I now think that jitter has little to do with mechanical speed; it seems more related to video content.

Ok, I've finished my software TBC and it works. It was so simple, why didn't anyone do this before?
http://screenshotcomparison.com/comparison/88691

Next up is a TBC that works with 'baked in' jitter.

oh ya, lordsmurf should buy me lunch or something while he eats his words :)

Thats because the "software" TBC is the same as a hardware line TBC, if you have access to the horizontal sync data its easy to correct. The 'baked in' jitter will be difficult to correct however since that horizontal sync data will have already been lost.

I don't think you need access to the hsync after all, it's just a matter of lining up two edges.
I realize baked in jitter is different, I have an approach for that that's known to work as well. Of course, if statement A is true then I don't need to use it anyway.

Proving me wrong would be most excellent.

I have some homemade barbecue sauce here, if needed. Maybe I can cut a steak to read "TBC". :P

Make a working software TBC that fixes some videos I have, and I'll definitely look at sending you something.
Take the whole family out to a small feast.

But I did - did you see the screenshot? I guess you mean the baked in kind then?

I need something for already-existing digital videos. We can even say it's an interlaced MPEG-2 from a DVD recorder, to keep it easy.

Youtube videos may be too many generations removed from the source to even be fixed again. There's a limit to bake-in, before the error starts to replace the actual image. I would imagine your typical poorly-converted progressive MP4/FLV files are not going to make for good candidates of repair. Would that be an accurate assessment?

Is this dedicated software, or leveraging Avisynth?

I have a lot of ideas floating in my head so let's be clear.

The method to work on digital/baked jitter I'll call 'intrinsic dejitter'. I have the algorithm worked out but it would take a week to implement. I don't think there can be too much jitter, but working with compressed progressive digital video I think does impact the algorithm, but it's too far to speculate yet. This method has *no* requirements and lines up structures in the video. It's the holy grail.

What I've been doing in the last week is replace a hardware TBC with a capturecard+software. It's nothing exciting to professionals as they already have this ability but - it might work in these cases:

-Black borders visible in digital version
-Analog tape available but capture didn't have complete black borders or none - if you could re-capture with tweaked drivers, you can access black borders and then it can be processed.
-It's an avisynth solution, I could make it standalone if need be, possibly integrated in a capture program/utility.

There is something innovative I've done however, as far as I can tell no one has gotten my approach to work this well so far, and also I found a way to tweak drivers to view the hsync and vbi, which has it's own possibilities. I also think my software TBC can be slightly better quality than a hardware box in terms of it's processing precision, since I can use high quality algorithms for the line shifting (which implies interpolation). Finally there's loads of possibilities for further basic research, which I think will lead to something.

I think it just takes R&D to beat this thing, it's hard work and not for everyone, and I think others would just give up unless it showed immediate practical results. For the potential benefits I think it's worth it.

Fixed the bugs.
http://screenshotcomparison.com/comparison/88810

New comparison http://screenshotcomparison.com/comparison/88873
It shows to me that hsync does give better result.

I'm watching with great interest, even if I don't post any feedback. :)

Me too! This is all very cool. I'm wondering if it would be possible to apply these same methods to fixing the issue caused by JVC VCR TBCs where a tape playing back with the TBC engaged occasionally causes the picture to shift up and then down (usually about four frames, first shifted +2 to +4 pixels above the X axis, down -1 or -2 below it, and then back up to 0) and the interlaced frames are smudged together and/or "mangled". Once I get my new computer up and running, I'll see about posting an example.

I often find myself going back and identifying these errors later, and then re-doing all of those individual parts with the TBC off (and possible with an ES-10 hooked up, since having the TBC off may introduce other issues), while matching the color with the original with my proc-amp.

At worst, it ends up being about 50 to 100 frames of video over the course of say a two-hour tape, but the time to fix that and edit them back in is monumental. If it could be fixed in software after the fact (with just the slight penalty to quality due to having to re-encode it that frame) then it would save me lots and lots of time. If it could analyze a video stream and correctly detect AND correct those sorts of errors, it would be a miracle, but just even being able to correct them as long as I know where they are would be a godsend. That's certainly something I'd be willing to pay for.

That shouldn't be too hard. I'm in the middle of some other things though. Let me know when you got a sample.

Sure! I need to go snatch my other hard drive from my broken computer as well. Probably be about a week, so take your time. I'll get a really representative sample.

Time for another update.

This is an experiment to answer the questions, what would a perfectly calibrated VHS look like? What are the effects of tape, VCR, and capture card on noise, jitter, linearity, and frequency response?

To answer the questions, custom software was designed and test recordings carefully measured. The resulting technologies advance calibration far beyond the test patterns on typical test DVD's. In essence, practically every measurable aspect of analog capturing can be optimized to the fullest. These experiments are just beginning, but included is one example of it's use.

How this video was created

First, the video was separated into it's component values (Y, Pb, Pr). Each image was recorded separately as a frame. It was also anamorphically squeezed.

Next, a special alignment signal was added to the video.

Upon playback, custom software aligned the video to within .1 pixel. While theoretically perfect, the processing was affected by noise and distortion in the video.

Then the component frames were merged into a color composite. The anamorphic video was unsqueezed. Finally the levels were perfectly stetched between the resulting black/white points.

The resulting video has only these distortions remaining:
-noise
-frequency response
-levels non-linearity
-some banding was introduced by the levels adjustment
-various other minor effects

Commentary on the Video
The video is fairly blurry as can be expected from VHS. The image is quite stable, but one can see a slight movement in the edges of the colorbars. This can probably be eliminated by further work in the software. The noise level is quite good for VHS. Sometimes there is line jitter as the video exceeded the range of the software (this can be corrected too).
Finally, the bottom lines of the video look normal. In fact in the orignal, it was badly slanted and had head switching noise. This part is normally cutoff, but now we don't need to!

Further possibilities

HD
With this technique, pixel-perfect video can be recorded. This brings the possibility of recording HD resolution. To do so, we would separate the HD image into 3x2 tiles or more. Then each tile is recorded as a separate image. The tiles are then stacked back together upon playback, increasing resolution 6 times. The only downside, is that the video takes 6 times as much tape!

The perfect alignment enables this technique because otherwise, it would be difficult to piece the tiles together, as the borders would be wiggling into each other (due to jitter) resulting in obvious artefacts.

Noise reduction
We could also record the video twice, and average the noise upon reconstruction. If this were done without alignment, the result would get much blurrier, as the two copies would be overlapping each other.

Perfect linearity
Currently when you record brightness levels of 128,129,130 etc. we get back 128, 130, 130 for example, as the mid-tones are a little brighter than they should be. Though these levels can be adjusted, there will be some banding as half of the mid-tone values have disappeared. There could be a scheme of HDR where the shadows and highlights are recorded as separate frames.

Frequency Response
Another test signal (frequency sweep) could be recorded to determine frequency response. This can easily be undone with an FFT.

Other methods of encoding video
The alignment would be critical for other methods of recording video. For example the video could be recorded as it's 2d FFT. In this case, mis-alignment would ruin the results, possibly resulting in severe ringing effects.

Measuring VHS Parameters
Further test signals could determine the ideal noise profile of each VCR/Tape combination, the jitter performance of a VCR/TBC/Capture card, frequency response, linearity, etc.

Using the VCR model to improve normal captures
Once we have an ideal model of these parameters, it can be used to tune normal VHS captures to the highest quality. For example, a plain grey image with ideal noise can be provided for use with the well-known Neat Video noise reduction program. The frequency response profile can be used for sharpening. The linearity profile can be used, with debanding, to improve the brightness curve (although this is usually adapted to user preference anyhow).

Further work
There are many aspects to VHS recordings. We can further examine tint/saturation errors, contamination between luma/chroma levels, chroma placement, and other various linear/non-linear shifts within and between signals.

Before anyone says so, yes it's completely useless to come up with better ways of recording to VHS. However if I were working in a TV studio 30 years ago I'm sure I would've come up with a scheme to preserve old TV Shows and today I'd be recovering them ;)

Anyhow, it's all part of the basic research. The practical aspect that could result from this is a measured, scientifically ideal way of restoring VHS recordings, in new ways. It can also lead to a better explanation/theories of various problems like color distortions, head switching noise, jitter, etc. Finally it can be used to rate the performance of tapes/VCR's/capture cards etc.

Example of my ideal TBC performance
http://screenshotcomparison.com/comparison/91487
You'll see some aliasing, that's due to a simple resizer to fix the jitter. On VHS you'd never notice.

This is pretty neat, and probably the first time anyone has seen such a comparison (well, rare anyhow).
It's color bars in normal VHS vs a component format I made up, on the same tape. Basically it's an excellent comparison in apples vs apples of the color resolution of VHS. The luma looks exactly the same between screenshots. You can see the color resolution is something like 20 pixels!
http://screenshotcomparison.com/comparison/91499

WARNING: The Sendspace.com link was displaying a false "download here" ad that would download malware.

Please attach files to the forum, do not use "file sharing" services because this is a known problem on those shady sites. I downloaded what I thought was your "setup.exe" for a TBC program, but it was malware caught by MalwareBytes. Even the best of us can be tricked by some of this malicious crap.

All images/etc related to the thread are archived long-term this way, too. Those silly "sharing" sites delete data after a few months.

Thanks for understanding. :thumb:

The intended download "ideal TBC.zip" has been added to this post.

Sorry about that :) I have noscript and noflash installed and don't even see what you're talking about. Though I didn't give it much thought, I had assumed that my file would exceed the size restrictions. Most videos are too large to put on a forum.
Anyhow, if I can find a simple and malware free site, would that be acceptable should I have a larger video? If it's not much different than a plugin download link, that should be acceptable.

About the file, I've been getting some comments and non-experts are a little confused about what I'm doing. To be clear, the video is a capture from a VHS. The capture has been processed in 3 ways; 1 levels adjustment to perfect black/white 2 dejitter *based on an embedded test signal* 3 decoding "component mode"
-the dejitter here only works with the special test signal embedded during recording, and is not in any way a general TBC
-the component mode is something I made up, I just record 3 b/w frames for each original frame; the frames represent luma, pb, and pr. This completely bypasses the color of VHS and has the resolution of VHS luma for each color channel.

Again, the point of the video is just a fun experiment *however*, it's also a demonstration of the embedded dejitter algorithm. The practical use of this algorithm is to measure the jitter of VCR's in various conditions, and to rate the performance of real TBC's or capture cards etc. I plan to make a 2nd generation copy of this tape and list statistcs of actual jitter, i.e. exactly how many pixels it moves and any patterns. This is just basic research to help me make a better *real, practical software TBC*

You can attach files to the forum: up to 8MB RAR or ZIP files for Free Members. So just do that.
Dropbox is good for larger files.

Premium Members can attach 16MB.
Site Staff 32MB. We can also add files to FTP, if needed.

And then I'd like to see the before/after images side by side, not just as rollover. Maybe do your rollover, but also attach both images to posts, so they're archived here as well. I was looking into some rollover before/after scripting for the site last night. I found something, but it has to be manually implemented into site pages or forum posts. I'm working on it (won't be a fast addition).

Well for now here's a screenshot of my latest script on a user supplied video, showing that it actually does work, at least in some cases:
http://screenshotcomparison.com/comparison/91853

This is my dejitter script - this one lines up images with black borders in them. My "perfect TBC" is something totally different, and my "intrinsic dejitter" is again totally different (it doesn't need the black borders).

jittered.jpg - artificially jittered test image, with embedded test signal (not shown)
dejittered.jpg - dejittered back to original by using embedded test signal

composite.jpg - a real VHS recording of a test image, that was dejittered perfectly by using an embedded test signal (test signal removed)
component.jpg - a real VHS recording of a test image, that used an embedded test signal, and then perfectly dejittered, and also using a component video - on VHS scheme

original.jpg - a user supplied, badly jittered video for which the original tape is not available
processed.jpg - dejittered with my dejitter script, which simply lines up the black borders

For a gallery, this isn't mouseover but the changing images is pretty quick when cached
http://www.cpurigs.com/forums/showthread.php?t=2056
http://www.cpurigs.com/forums/vbpicg...ig&p=514#photo

Albums are native to the forum software.
Here's yours: http://www.digitalFAQ.com/forum/memb...98-albums.html

You can add images there, too.

I spy something new that you've not shared here yet. Saw this at VH:

Definitely go over your Avisynth scripts when you get a chance.
This may be something that I can incorporate into an existing and ongoing guide series involving Avisynth and VHS problem scenarios.

Ya, that's pretty immature version. I think I released it too soon, a lot of people are trying to use it and I'm spending more time answering their questions than actually finishing it :)
Problem is, it only works on the bright scenes, so it's probably not practical to anyone yet.

I didn't think avisynth was very big here, you seem more hardware/virtualdub oriented as opposed to programming new scripts for each video.

I'd suggest this, to make your life easier:

1. Give a brief and quick overview of whatever everything does. Literally one sentence per function, maybe a few extra sentences to describe a key variable and what a change does. Just do this once, and never again until you're ready to write a full manual in detail.
2. Release each version as attached txt/avs files.
3. If somebody doesn't understand it, then unfortunately for them they're just out of luck for the moment. They'll have to wait until the project is closer to beta or done entirely.
4. Keep releasing versions. I've seen many projects disappear entirely because the developer lost interest and nothing was ever released. Great ideas, nice samples, but no clue on how to recreate what was done. This way, if you give up, others can continue on the work.
5. In fact, it may even strike enough interest in others to be forked.

I've been very vocal against sucky Avisynth documentation for years, but something like this, where you're constantly developing it, there's good reason to not sidetrack yourself with Q&A's, how-to docs, etc. That can wait. Just don't never do it, like many of the past developers have done. The brief overview and some notes is good enough for now.

For each new release, just give a couple of quick mentions on what was changed/tweaked, without writing an explanatory novel. We'll either pick up what's going on, or we won't.

I don't want to see you overwhelmed by people --especially stupid/lazy ones that just want a big "Go!" button to "fix video" -- as opposed to keeping this an interesting and enjoyable experience for tackling this video problem. That's a big problem I have to deal with, often spending vast amounts of times documenting and answering questions, instead of doing the research, dev and application.

I've uploaded a small clip from an old Cable TV airing of BeetleJuice to demonstrate this error. Note that my definition of what the "error" is (see above) certainly does not cover all of the problems with the video and in fact, both clips demonstrate visual problems in the same set of frames, but I'm looking to specifically correct the error as described above that occurs in the TBC ON version of the clip, in some consecutive and non consecutive frames between frame 131 and frame 144.

I actually have (somewhere) a file with about 40 examples of this error that I've come across over various tapes, which I'll try to find later, although this is pretty representative of the typical case. Note that the amount of frame displacement vertically can vary by maybe +/- 3 frames. The length of time each frame is displaced from it's original position can vary a bit too. Typically these sorts of instances are brief though and scattered, never lasting more a second at a time. As LS has outlined previously, errors with these characteristics tend to occur on tapes with other problems, when the TBC frame buffer memory is unable to compensate. As a note, my particular JVC deck has a 4MB frame buffer.

Ultimately I'm wondering if this kind of error is addressable once in the digital domain. If I could re-encode (obviously with a slight penalty to image quality) small batches of frames (hidden between cuts typically) without doing separate non-TBC or ES-10 encodes it would surely save me a lot of time. Even better would be an automated tool could scan the video stream and reliably identify instances of such an error. I've played around a bit myself with log files of the AviSynth DeJitter filter to see if I could identify them based on the shifts of the vertical axis pixel values, but I could not reliably identify them based on that data alone and the correction was spotty as well.

Both clips are accessible here as TBC Clip.MPG and No TBC Clip.MPG

http://www.wou.edu/~rvitolo06/test/

Ugh... the vertical jitter (image bounce) vs horizontal jitter (time base instability) issue.
It's more pronounced on JVC S-VHS VCRs, but Panasonic is far from immune.

Also, your clips are small enough to attach to forum posts (under 16MB each). :)

Indeed! In my case, it seems to mostly impact home recorded tapes the of a particular generation in my collection, although I see the occasional issue on commercial tapes. Typically when I run into this scenario, if it's pretty bad I'll just use the JVC with the TBC off with ES10, which aside from a slightly detrimental impact on some of the qualities of the picture, fixes it up fine to an acceptable level. Some darker tapes just don't look good with the ES-10 though and so I get into mixing TBC footage with color-matched footage from the ES-10 in areas where this error occurs. On tapes where there are maybe 50 or 60 instances like this, it can take many hours to identify and correct all of them. It'd be nice to be able to do it after the fact from the TBC footage, when I actually get around to watching it! :p

Yep, I think I know what that is, and there's a fix for it ;)

Awesome, that is very exciting! I think we're at the point in all this VHS to DVD stuff that we're not going to see much more innovation and tweaking hardware wise, but you're making some impressive inroads using software. What is your background in programming by the way?

At any rate, I'd definitely be willing to make some sort of a donation towards the work and towards the site as well.

Organization
Could you move post#32 on to a new thread, "Dropped/Duped Fields (Plays as Vertical Jitter)"

The Problem
Video looks like it's jumping when passed through TBC. Applicable only to this particular sample.

The Analysis
The pattern is this, with field numbers:
0/1, 2/3, 2/5, 6/7, 8/7, ... and repeating.
In other terms it's, ok/ok, ok/ok, dup/ok, ok/ok, ok/dup, ...or you're missing 2 of every 10 fields.

The Repair
It's easy and high quality interpolation, hardly noticable.

Fixing existing files
I can create a text file with ranges of frames to fix plus the corrected video. Let me know if this works for you; to have the entire corrected video. Then you could encode it and splice bits of mpg together, replacing each range specified. Easiest option is to just re-encode it all, of course it might get blocky...

Note to self: value of thresh for this repair was 2.4-2.8.

Good idea -- I suppose this is a separate issue. Maybe LS can move all of them so I don't start a new topic and start duplicating posts. Whatever is easier.

Also -- regarding the video clip itself, this particular video isn't one I have a need to correct, although it would be fun to see the before and after for this particular example. It was just intended as an example of the general problem that I find on other video footage on my tapes.

I'm not sure I understand how I'd apply the fix to a general video -- this is a representative example, but (I think) some videos do behave slightly differently in terms of how long it lasts, but I suppose it's possible that the same pattern is just repeating over and over as you identified. If I did run into a variation on the problem, would I need to edit the text values to handle it as well?

In general, I don't mind re-encoding the videos and storing them with the original until I can splice in the problem frames from the re-encode. It'll save me time in the long-run and I have lots of space. For me the idea is to use the re-encoded version only as an alternative version for fixes and not rely on it as the base version of the footage.

Ok, for your purposes, I could encode the frame ranges as: no problems surround 15 frames around each problem. That would be the size of cutting an mpg anyhow. So if the problem was like this:
ok, bad, ok, bad, ok, ok, ok, ok, ok, bad...
It would say replace 9 frames. That way, it doesn't matter if the patterns are quite different.
The pattern doesn't affect the repair but, if there's a full frame drop I'd like to test it first.
So I'll prepare a fixed version for you with range text file and see how that works for you.
Check another 12 hours though, my day is done here.

Meanwhile if you find another sample just to ensure we're seeing the full variety here. Especially something that seems any different to you.,

Another problem, I assumed it was only movies that were 25p (encoded as 50i), could there be some tapes with 50i video too?

Sounds good! I'll see if I can find some other examples -- I have a file of a bunch of them somewhere. I'll have some time to work through some sources I've had problems previously over the next month or so, so I'll have lots of opportunity to test it out.

By the way, as far as the footage I would use this with -- good question! For the most part it's actually not movies or film sources, although there are some of course. Most of the sources I'd be using are actually mostly television shows taped on video and not film and also some sports, which I assume are also taped on video.

I'll have to use a different repair method, it's a bit worse. Do you normally keep your encodes as fully interlaced? I should have a video sample for testing.

Good to know, glad you asked, I didn't even think to mention it.

Yes, I leave the interlacing alone, as my destination is typically DVD-R for playback on a standard DVD player. The sample tape was from a cable TV recording from about 1990, playback on an NTSC JVC SR-W5U deck fed directly into my JVC DR-M100S DVD recorder (top frame first). I just did a straight rip from the DVD recorder and then edited it together an MPEG from the VOB without re-encoding it -- that's the typical process I use for just about everything. I'll see about getting another clip tomorrow for you.

-- merged --

Here are a couple of more TBC on examples for ya -- from a basketball game. I doubt it's of any consequence , but just as a note, these are from an EP source, were the movie was from an SP one.

Problems with about frames 72 - 78 on the first one and 95 to 101 on the second. (Interesting that both are about 6 frames worth). Not sure if these differ at all from the first ones -- although perhaps the 'variety' I'm seeing is just perceived and not actual.

I have a few tapes I can use to test this thoroughly that have a high number of these sorts of instances, so I'll give the algorithm a workout for sure!

Good catch, those are different. 2nd clip is really weird, half the frame is ok the other half is a repeat of 1 frame back. It would need a special repair technique. Does the effect occur a lot?
Your clip1 is different as well. It's a missing field with no dupe. Detection is harder but repair is the same.
So these are all 3 different problems. I can't solve them all right away, but just try one and go from there. And give me a few days :)

Interesting though, things I never saw before... ultimately could find a way to fix all these types of problems.

Absolutely! Take your time.

Yeah, after looking at it, I'd say the version in clip 2 occurs fairly often as well. I guess I generalized the problem to "jittery/jumpy frame" but I guess it's much more complex in terms of how it manifests itself specifically. Some are more noticeable than others. I'll see if I can pull out another tape that displays some more -- hopefully there is a small set of different errors we're working with here! If it can be made into a workable solution it would pretty much eliminate the one major shortcoming of the TBC correction in these SVHS decks, which is pretty astonishing to think about in 2011!

I'm still looking forward to seeing what you were able to do with the messed-up clips I gave you. :)

Alright jmac, this was created because of you: http://www.digitalfaq.com/forum/news...er-images.html
This wasn't some off-the-shelf mod.
I spent several hours this morning writing out new vBulletin code just for this.

It should come in useful for us advanced users. :)