Creating time base errors for testing TBCs?

[aramkolt]

Wondering if there's a process that produces the "bad" timebase errors for purposes of testing various TBCs and capture cards to see how likely they are to drop frames or lose sync altogether. My guess is that this would involve tapping or slowing down different parts of the transport mechanism of the recording VCR, or possibly even the use of magnets somehow, but I'd like to get more specifics on how that would be done.

At the same time, I'm not trying to create alignment issues haha.

Also, what's the best way to test for AGC issues that some cards are probe to have (automatic gain control)?

[Gary34]

I was just wondering about your testing. Out of curiosity.

1. What all is being tested?
2. What all equipment are you going to have to purchase?
3. Is it just you doing this testing?
4. What kind of timeline are you thinking of?
5. How’s the testing going so far?

[themaster1]

Mess with the A/c head and you'll have plenty of fun

[Gary34]

I had just heard testing of soo many things and I didn’t want to comment on another thread that had a different title because then it extends the thread by a page of something that is off topic so I thought I’d ask here on a thread about testing.

[BW37]

@aramkolt Here are a few ideas on creating time base errors on tapes to use as test samples:

1 - Record a tape with a camcorder while moving it around as you record. Pan way too rapidly side to side and up and down, maybe even shake it a bit. This should cause the drum speed to vary some due to the gyroscopic forces of the spinning drum. These should be close to real world TB errors.
2 - Create “nth” generation tapes, maybe using lesser quality tape decks. Errors should compound as the generations increase. This might give you some gradation in the size and numbers of TB errors that could be useful. Maybe start with one of the tapes made as in 1 above.
3 - Along the lines of themaster1’s suggestion but maybe a bit more real world, make some recordings on a deck with an intentionally dirty control head. I don’t know how you could “dirty” it intentionally and reversibly but maybe it could be done. Also, with your electronics background maybe you could somehow weaken the control signal with a mod of some kind.

My 2 cents,
BW

[aramkolt]

The tests I propose are:

1. Differentiate capture quality of a large variety of capture cards (I have access to over 20 different cards) comparing sharpness, automatic gain control, color accuracy, white/black levels, etc. This will be done with digitally produced test patterns and/or other short video clips with a wide dynamic range and movement while using vectorscrope/waveform monitor to make sure the signals going into the cards are at appropriate levels. I may also use a proc amp to artificially boost luma and chroma levels to something like 20% higher than they should be to see if the automatic gain control scales the brightness back down, or if the card under test just clips levels past 100IRE. Each test would be done both with composite-in and S-Video-in. Depending on the comb filters present in each card, S-Video may not be visually significantly different than composite which is useful information to know.

2. Test a variety of frame TBCs for any image degradation introduced by passthrough alone from test pattern generators and/or digital starting material for reproducibility and the ability to compare to the capture to the actual starting source. Will also capture via SDI to show if there is any less image degradation when using SDI output from TBCs that have it (SDI out should aways be superior, theoretically, but is it visually significant?).

3. Record test material to an SVHS tape and then introduce severe timebase errors either during recording or playback and see how each TBC handles those errors - If timebase errors are introduced during playback, I may do simultaneous captures for head-to-head comparisons of the best ones using an S-Video distribution amplifier so that they all see an identical live signal. I will do a test where frames will have burnt-in unique timecode stamps on each frame and LTC audio to verify whether the TBC itself is duplicating or dropping frames. Using linear timecode audio and burnt-in visible timecode will therefore not rely on drop/insert statistics and doesn't limit doing captures to Virtualdub alone.

4. Test capture methods outside outside of the recommended AIW AGP Virtualdub XP workflow to include modern computers with OBS, SDI capture with Blackmagic Media Express, SDI capture with AJA Ki Pro to ProRes, AJA Kona LHi, Blackmagic Intensity Shuttle Thunderbolt, Blackmagic Hyperdeck (will test both going to ProRes and H.264), Canopus ADVC-110 (DV), Digital8 camcorder to DV stream, DVD recorders DR-M10 and VRD-MC5 to DVD, ATI MMC high bitrate mpeg2 capture with AIW card, Retrotink devices to HDMI capture card, VHS_Decode, and European DMR-EH77 passthrough (which is supposed to have ES10-like line TBC effects, but also preserves 480i output over HDMI). The point is to see how they handle an identical starting VHS source test material. Elgato will be included since that is probably what most average users will use and some of those users are probably wondering how much better quality they should expect if they upgrade their setup. I'd consider throwing a ClearClick and Cloner Alliance box in there as well if someone has them I can borrow (I spent enough on this project already and I don't believe that they'll be the winners). Showing a variety of capture methods with the same source material will let people decide if it's worth the extra time or expense in using one capture setup over another.

5. Testing hardware upscalers/deinterlacers to see how they compare to software deinterlacing. I realize this isn't something you'd use for archival quality masters, but if any turn out to give similar quality in fewer steps, might be worth considering for a non-archival quality workflow.

6. Test the most commonly recommended Line-TBC containing SVHS machines and identify how they reproduce chroma/luma levels/sharpness compared to each other with the same source VHS tape with lossless capture samples posted. Test will include JVC 9600 series, JVC VS30 (equivalent to 7600 series), JVC SR-MV series, Panasonic AG1980, Panasonic NV-SV1, HS-HD2000U, and MD3000U (SP only). Will also compare a basic Panasonic Omnivision plus ES10 for line-TBC effect for a low budget comparison.

As far as current progress - it has mostly been hardware acquisition up until now, but I think I have just about everything to do all of the above. As for timeline, I'd like to have some results posted within the next couple of months.

The test patterns and audio sync/dropped frame tests I'll plan on posting a burnable DVD ISO so that others can run the same tests on their own setups to see how results compare. The limitations of that is that chroma subsampling for DVD is 4:2:0, so any test that is looking at chroma subsampling probably won't be valid. For my own testing, I'll primarily be doing it with ProRes444 SDI captures from test pattern generators with SDI out or PNG files played back on Davinci Resolve via an Ultrastudio Monitor 3G with the SDI output from that going through a 12 bit Brighteye SDI to Analog converter with S-Video output and would verify appropriate output levels with a waveform monitor/vectorscope.

I might actually put a DVD ISO file out before I do my own testing for users to suggest any patterns I should be adding or removing. I don't really want to test all of these devices multiple times since there are so many of them haha.

[lordsmurf]

@aram
1. Nothing to do with TBCs.
2. Must be very wide sample size to form any conclusions.
3. No. Faking errors never works the same as actual errors.
4. Nothing to do with TBCs.
5. Nothing to do with TBCs.
6. Be very aware that many models vary greatly, and condition is huge factor here. Test will be inconclusive beyond loose results. This is a yes/no test, not graded test.

[Gary34]

The sample size is a good point. They mention on this site that bad methods are really dependent on sources if they well give decent results. The testing that was done a long time ago was done when mechanical issues weren’t as big of a factor and it was collaboration among professionals over years during a time when there were more professionals and a lot higher volume.

The testing will be scrutinized by this site but if you post it on YouTube it will be accepted. There are some really bad video comparisons that get accepted on YouTube. The Decode comparisons I see are really awful. Its a decode video that is always way over sharpened to the point where people that don’t know what over sharpening halos are point it out. Then the traditional capture is done on bad gear over composite in random shape with no TBCs.

It’s a lot to test. It seems like you are trying to test everything. I wonder about how thoroughly each thing will be tested.

[aramkolt]

Correct, I am not just testing TBCs. The post was more to outline all of the testing I plan to do.

I am doing what I thought would have been 20 years ago by this or a similar site - actual capture samples provided from a variety of capture cards, capture methods, and TBCs using the same source material to show (and not just tell) which devices or methods give better results and to show (visually) the improvement is over the alternatives. If you show a user a capture from a $100 chain and one from a $2000 chain done from the same source material and they can’t personally tell the difference in quality/stability and neither is shown to have audio sync issues or dropped frames, why should that user be persuaded that the $2000 chain is always the “better” option?

What is different about my testing is that I’ll be putting the test material for free that should expose different potential flaws of different capture cards and TBCs in a matter of seconds so that users can test their own setups with it to decide if it is worth upgrading their hardware chain as well as know in advance what results they should expect with a given chain that they do decide to invest in.

It is also true that these will just be the results of one test, hence why it will be useful for others to do their own tests with the source DVD and upload their samples if they get a significantly better or worse result visually. In many cases, I have multiple units (often as many as 3-5 each) of the items to be tested which should give an idea of the variability to expect from device to device of the same model. Has anyone ever posted a direct comparison between 10 different refurbished AG1980's using the same test tape and capture chain? If you did see that comparison, would you question what sort of results you should expect to get from a refurbished AG1980? One could easily counter argue and say that test is completely useless because all tapes are different and maybe that test tape is a really good tape or something. It is still worth putting the test out there in my opinion. Pooling results of people testing their favorite chains and posting results should be useful in a similar way, despite the actual test tapes themselves being a little different as they were recorded in different VHS machines, are different brands of tapes, and the source DVDs were played by different DVD players.

@Lordsmurf - what testing do you recommend that I do to differentiate these TBCs that I have not already proposed other than testing specific difficult tapes that I am unable to source? If it’s that hard to get a hold of such misbehaving tapes in the wild and they can’t be created by agitating the mechanism during recording, I don’t think it’s likely that the average user is likely to run into many tapes like that and therefore it probably won’t be applicable to most.

I have asked in the past if anyone will donate original tapes with significant timebase errors that drop a lot of frames, but haven’t had any responses.

[Gary34]

Quote:

I am doing what I thought would have been 20 years ago by this or a similar site

I can see why professionals 20 years ago didn’t save losslessly compressed files of clients personal footage given the price of storage back then.

Quote:

If you show a user a capture from a $100 chain and one from a $2000 chain done from the same source material and they can’t personally tell the difference in quality/stability and neither is shown to have audio sync issues or dropped frames, why should that user be persuaded that the $2000 chain is always the “better” option?

There’s a huge difference once you look at a compressed MP4 done through composite and a recommended workflow processed in Hybrid. That 100 dollar chain is going to be bad. I’ve seen seen enough with my own tapes personally to know that isn’t going to happen.

Some cheap devices like Clearclicks are really dependent on sources to get it to where it is remotely watchable. The deinterlacing can look way worse on some sources https://youtu.be/LuVTx_tOfvw?si=Bic1HFsrqrZM2DqL. Not as bad on others. https://youtu.be/lyWyKvU2XF8?si=dhKr7YOpFyJ8rJQG. One person having source A can tell person having source B this is what you can expect and source B person doesn’t get the same result. I really don’t see the point in testing those devices.

Quote:

What is different about my testing is that I’ll be putting the test material for free that should expose different potential flaws of different capture cards and TBCs in a matter of seconds so that users can test their own setups with it to decide if it is worth upgrading their hardware chain as well as know in advance what results they should expect with a given chain that they do decide to invest in.

What exactly is your test material that can expose different flaws in different capture cards and TBCs?

[Gary34]

I just don’t see the DVD idea that you described where you’re introducing timebase errors being able to help people differentiate between what TBCs they should get.

I do agree with you that it would be really good to see more visually if you could show someone what there tapes would look like with different gear. I just don’t see how to do that.

I’m not trying to be a Debbie downer. I can see you are definitely trying with this.

[lordsmurf]

Quote:

Originally Posted by aramkolt

I am doing what I thought would have been 20 years ago by this or a similar site
- actual capture samples provided from a variety of capture cards, capture methods, and TBCs using the same source material to show (and not just tell) which devices or methods give better results and to show (visually) the improvement is over the alternatives.

With this statement, you're being blinded by modern conveniences. It's also somewhat 20/20 hindsight or "Monday morning quarterbacking". I don't have the time to dissect this now, but I'll try to remember to circle back to it.

Quote:

If you show a user

The Youtube era has ruined quality research.

Quote:

If it’s that hard to get a hold of such misbehaving tapes in the wild and they can’t be created by agitating the mechanism during recording, I don’t think it’s likely that the average user is likely to run into many tapes like that and therefore it probably won’t be applicable to most.

No, false logic. "YOU" cannot always easily get good "TESTING" tapes, but the errors are widespread and common. A "test tape" is not just a tape with the error, but a tape that easily shows the error. I don't know if I'm explaining this well here, but I'm just pressed for time. (It's amazing that I'm replying at all right now.)

Quote:

I have asked in the past if anyone will donate original tapes with significant timebase errors that drop a lot of frames, but haven’t had any responses.

That's because most people don't want to part with their tapes.

When it comes to "home movies" (not tapes of mere recorded TV), there's also a ridiculous idea of "privacy". As if anybody will know (or care) about people/kids at a stranger's family event 25-40 years ago. Heck, I barely know who some of the people are in our family videos -- and I really don't care, as my focus is not whoever they were. I've asked individuals if I could use a clip from their tapes, to showcase an error, and they'd get bent out of shape. "OMG! NO! I don't want people to see me when I was 10 years old! I don't want to share a video clip of my long-since-passed grandma!" You'd sometimes think half of the population was in witness protection. Seriously, who cares? You were a dorky kid. Every kid was a dorky kid. Your grandma would probably be amazed by this whole "internet thing", and find it neat to share online. So there's that standing in your way, too.

Quote:

Originally Posted by Gary34

I just don’t see the DVD idea that you described where you’re introducing timebase errors being able to help people differentiate between what TBCs they should get.

Nope, it will not.

Quote:

I’m not trying to be a Debbie downer. I can see you are definitely trying with this.

That's called being a realist. Not everything is sunshine and candy.