TBC is clamping luma? How to fix it?

[Acros_13]

This hobby is frustrating isn't it. I was thinking I was making some good progress on my project, but ran into a thread here about wavy diagonal lines caused by their BVTBC, which is the TBC I'm using. I also saw diagonal lines in one of my videos, so I re-captured that video and was relieved (temporarily) by the fact that the diagonal waves did not go away and therefore were not a product of a faulty TBC. that said, by looking into this I captured for the first time ever without the use of a frame TBC and now noticed things with my histogram. From the beginning I always felt it funny that my histogram seemed to demonstrate clipped levels without really showing "red", adjusting brightness/contrast seemed only to shift the overall histogram to the right or left without really changing the shape at either end of the histo.... I assumed that this implied that levels were just clipped at the initial recording level. But now suddenly I realized that without my TBC in my capture chain, the histogram looked way more natural to me and it was indeed harder to find optimal settings to make sure nothing went 'red'. Also sure enough the final product produced detail in the highlights that were previously clipped altogether! ugh. so frustrating. so then I looked into this and determined that my TBC is likely clipping highlights and blacks in a Luma balancing act before the signal even reached by capture card. I can ignore the histogram since apparently its useless and can't be trusted anyhow, and can restore some of the whites, but cannot bring back all the extreme detail which makes sense if the TBC is clipping that data before it reaches the capture card.

I guess I'm just asking, do I have this concept right? and is there anything I can do about it? ChatGPT (for what its worth haha) says I can do some post capture trickery and blend a with TBC capture with another without TBC to get the all the Luma as well as as the stability of the TBC capture but this seems crazy.

Ignorance really is bliss. I was totally accepting blown whites and crushed blacks, knowing (or thinking I knew) that I did my due diligence and got a balanced histogram before hand. but now that I know, I have to find a way to correct all of my captures from the very beginning.

[lordsmurf]

Quote:

Originally Posted by Acros_13

This hobby is frustrating isn't it.

At times, yep.

Quote:

I was thinking I was making some good progress on my project, but ran into a thread here about wavy diagonal lines caused by their BVTBC, which is the TBC I'm using.

Nope, not the same. You have a specific version of the BVTBC8, which is not the same as a the lower-end BVTBC (and it has it's own versions issues). That specific error doesn't apply to you. --- Although power-related noise can appear similar, and harm anything and anybody.

Quote:

I also saw diagonal lines in one of my videos, so I re-captured that video and was relieved (temporarily) by the fact that the diagonal waves did not go away and therefore were not a product of a faulty TBC.

And that's the power-related noise.

Quote:

That said, by looking into this I captured for the first time ever without the use of a frame TBC and now noticed things with my histogram. From the beginning I always felt it funny that my histogram seemed to demonstrate clipped levels

You're misinterpreting what "clipping" means.

- Too many people have this wrong/odd idea that "clipping" is bad. It's not.
- However, its not good either.
- It just is.

So, why does clipping occur, you ask? It bounds the video to legal levels. Illegal levels confuse gear, such as throwing off the AGC of a capture card, or later display devices.

The solution to illegal levers, to avoid clipping/containment, requires analog-domain proc amp. The BVTBC8 contains proc amp, so use it. It also has multiple settings, beyond proc amp, that can affect the image. Verify those, A/B test those.

Very few capture cards can properly capture illegal values (with the AIW being one of them). So that also means, most likely, that your non-AIW capture card, being fed illegal levels, is still not properly processing the values. So while some aspects may appear better, I'm sure are worse. Maybe not in that same tiny test clip, but elsewhere, to make new clips.

[Gary34]

I hope I am not misunderstanding what you are saying.

Quote:

From the beginning I always felt it funny that my histogram seemed to demonstrate clipped levels without really showing "red"

If you were using the proc amp in your TBC you can’t make the histogram go into the red because most cards will clip the incoming signal’s illegal levels. Some cards will display illegal values. If you use your card’s proc amp you can make a histogram go into the red because that proc amp is after the ADC in your card.

Quote:

I can ignore the histogram since apparently its useless and can't be trusted

The Avisynth AVSPmod histograms can be helpful. I don’t think you are supposed to believe a histogram over your eyes but if your eyes and a histogram agree then the values are most likely clipped.

[radiokom]

In digital world there are no levels above 0dbfs (decibels relative to full scale). If you exceed it you get clipping except you use a limiter. I always digitize at lower levels (peak at about -10dbfs) and then normalize to -3dbfs. There is no need to digitize at levels close to maximal, it will not improve signal to noise ratio. Adobe Premiere has good option to edit audio track of video file in Adobe audition (if you has it).

P.S. Ok, my comment is about audio clipping, but anyway

Edit: That audio concept still applies here. -LS

[Acros_13]

thanks for the responses LS and Gary34

Quote:

Nope, not the same. You have a specific version of the BVTBC8, which is not the same as a the lower-end BVTBC (and it has it's own versions issues).

this at least is good to know.

Quote:

You're misinterpreting what "clipping" means.

I guess my understanding is that the tape has highlight detail captured, but the TBC doesn't pass it along (clipping it), so the final capture has blown highlights?

Quote:

The solution to illegal levers, to avoid clipping/containment, requires analog-domain proc amp. The BVTBC8 contains proc amp, so use it. It also has multiple settings, beyond proc amp, that can affect the image. Verify those, A/B test those.

to clarify, I did several caps, including one in which I re-integrated the TBC back into the chain and used the proc amp settings to fix the highlights to my eyes the best I could. I guess to clarify further, I used a combination between the settings on the BVTBC itself as well as the settings within VD which I guess are the settings of the AIW X1900 I'm using. By "analog domain" does this mean the settings on the TBC are analog, but the Virtual Dub settings adjusting the AIW are after the ADC (which is what Gary says below), if this is the case I can try again adjusting the settings only using the TBC to see if I can recapture the whites, but by using a combination to get the best picture to my eyes, I was still unable to get the whites back that I was able to capture without the TBC in my setup.

but if you are saying that I can correct this issue by better utilizing the settings that come with the Big Voodoo then that is actually the best case scenario. I will further investigate. but I guess it was my understanding that the TBC being a somewhat "professional" TBC was a disadvantage here as it was just simply not passing along levels that were outside the analog specification. But im just learning all this stuff.

Quote:

So that also means, most likely, that your non-AIW capture card, being fed illegal levels, is still not properly processing the values.

Im not sure if that was a typo, but just to clarify, I am actually using an AIW card (X1900). and maybe I'm mistaken, sorry, but I thought that the heart of the issue is that in fact my TBC WAS feeding ONLY LEGAL levels (meaning it "clipped" the extras) and therefore my whites were blown. But because my card CAN capture the extra bits outside of that range, when the TBC was NOT in the chain (and thereby all illegal levels being fed to the card), I COULD recapture that blown detail. (sorry for the caps, i'm not yelling, just emphasizing )

Quote:

If you were using the proc amp in your TBC you can’t make the histogram go into the red because most cards will clip the incoming signal’s illegal levels. Some cards will display illegal values. If you use your card’s proc amp you can make a histogram go into the red because that proc amp is after the ADC in your card.

when you say "cards" are u meaning the TBC or the capture card? But I believe I understand you, when I use the TBC controls I can shift the histo left and right but I don't think it goes red. but again I thought this was because the TBC passed along only a balanced Luma level within that 16-235 space? but if I change the values within Virtual Dub I can make it go red

Quote:

The Avisynth AVSPmod histograms can be helpful.

I've heard this mentioned when someone tells someone else that they are clipping something in their captures, I'll have to look into it. I was thinking it was weird that I couldn't access a histogram when playing back a file within Virtual Dub. is this something I can access on my capture machine (windows XP)? I don't really have Avisynth set up anywhere else for now. my main rig is a Mac. though I do have a windows PC as well. Was going to plan the whole Hybrid Avisynth filtering phase out later. QTGMC works in the 2023 build of Hybrid for Mac and thats sorta all I intended to do with avisynth.

Thanks for the help again. sorry for the questions.

[lordsmurf]

BV outputs legal, as do many TBCs (especially anything broadcast).

The AIW is showing you the illegal values. And as an AIW, it probably is faithful to the original bad/wrong/illegal signal. Even with this AIW-only capture, post-capture, you're still going to have issues with illegal values needing to be legalized. And it'll never be as natural looking as analog-domain proc amp.

"blown highlights" is the wrong term here. Nothing was blown. Over-high was truncated/clipped. So, in actuality, it's un-blown, values now made legal. It all needs to be pulled within the 16-235 luma range, and you're source is outside that.

[Acros_13]

Im probably still wrong, but I understand a little more now. i'm running the tape and just watching playback with various settings and the proc amp settings on the TBC can indeed bring the highlights down, even though the entire image becomes too dark, but this implies as you say, that these illegal values can be legalized in the analog domain before ADC. I also understand now that the settings in VD are digital domain changes. I also understand that without the TBC all the "illegal" values were passed to the capture card and the AIW was able to capture it. This made changes within virtual dub usable since it was all post ADC so the histogram remained accurate. Whereas using the TBC limited the luma range that was passed to my capture card which made it so that regardless of the edits I made in virtual dub, I could never get back the information that was clipped during capture in the first place.

I know this is a bad way of looking at things, but it seems that by not using the TBC I gain in essence more dynamic range naturally without as much effort. In order to capture a tape that has both dark and bright scenes appropriately, it seems I would have to adjust the TBC settings differently for each scene, whereas without the TBC I can basically use a more universal batch of settings that since it passes luma values outside of range, still captured all the dynamic range without scene to scene adjustment.

since I now have an ES10 as well, I used it to see how it handled this Luma range. The result was the same as when I didn't use the BVTBC8, meaning without changing any settings I was able to find a good balance to capture all the dynamic range without any changes between scenes with one capture with an end result that did not blow the highlights in bright scenes and did not clip the blacks in dark scenes. It also gave me 0 Frame drops and 0 frame inserts. compared to 12 inserts with the BVTBC8 and 27 inserts without anything (and no drops in any sample). I know the ES10 isn't supposed to be used regularly but I'm having a hard time not thinking that this was a win for that setup for this tape at least.

Quote:

Even with this AIW-only capture, post-capture, you're still going to have issues with illegal values needing to be legalized.

but for instance, the illegal values bypass the TBC and go to the AIW card but then get balanced into legal values (all blue histogram after tweaking parameters in VD) then the product is captured and exported into the .avi file, then there isn't any actual illegal values post capture right?

Quote:

And it'll never be as natural looking as analog-domain proc amp.

So in basic practical noobie terms, I need to adjust as much as possible first by using the voodoos settings and for this I need to use my eyeballs because the histogram isn't going to help me if overblown highlights or clipped blacks wont be shown in the histogram as the TBC won't pass along that info in the first place. so I can adjust the TBC settings up and down in a shadow or bright scene and if there is more detail to be gained I can infer that there is more captured on tape outside normal luma range... correct that using the proc amp settings, legalizing the out of spec luma,... then I can use virtual dubs settings to tweak and make sure the histogram is all blue, in this case helpful, because it should represent the post ADC and what will be in the final capture?

[lordsmurf]

Not all gear cooperates with all tapes.

[Gary34]

Most people capture in the legal range anyways. https://www.digitalfaq.com/forum/vid...am-levels.html You can use Avisynth histograms to check levels post capture. http://avisynth.nl/index.php/Histogram
And http://avisynth.nl/index.php/ColorYUV

[Acros_13]

Quote:

Originally Posted by Gary34

Most people capture in the legal range anyways. https://www.digitalfaq.com/forum/vid...am-levels.html You can use Avisynth histograms to check levels post capture. http://avisynth.nl/index.php/Histogram
And http://avisynth.nl/index.php/ColorYUV

indeed. I'm capturing legal range as well. it's just that my issue was I didn't realize that I wasn't doing a good enough job to use my TBC settings to legalize some out of spec footage recorded on tape prior to capture, so I was in fact clipping information upon ingest.

thanks for the links!

[lordsmurf]

Quote:

Originally Posted by Gary34

Most people capture in the legal range anyways.

And there's a good reason for this.

(A) Attempting to display non-legal levels could actually harm display devices, as the information sent was outside bounds. This was more true in the CRT era, but still true in earlier digital displays era. For example, when using early HDTV (plasmas, etc) as a computer monitors. Or when playing a homemade DVD (captured with the card, encoded/authored with the "free" included junkware), on a CRT TV. Realize that CRTs were still everywhere in the 2000s, when USB capture cards debuted.

(B) People are/were unaware (unknowledgeable, ignorant) to possible issues -- or worse (lazy). Even if you put a big warning card in the box, people would ignore it, or not understand it's importance.

So the result was damaged gear, and the capture card was blamed.

The fix? Legalize on ingest, using clipping (value truncation). The reasoning was that consumer sources were all crap anyway, be it camcorder movies or TV VCR recordings -- not that I agree with this whatsoever. Bad video is/was always due to bad VCRs* and lack of TBC, followed by bad ingest/capture devices in the digital domain.

*See also: why vhs-decode doesn't do anything special, as it merely tries to recreate a VCR in software, warts and all. It's proving itself to have low-end VCR quality, with some line TBC added (sort of). Which is worse than high-end S-VHS VCRs (with line TBC) made in the 90s.

In effect, legal values are due to:
- low-knowledge and/or lazy consumers (the majority of capture card users)
- legal ("you ruined my TV, I'm suing!")
- lack of easy/cheap ability to correct values pre-capture (proc amps)

And because consumer analog formats are using consumer ingest devices, we're somewhat screwed.

"Professional" devices (term often used very loosely) were intended for non-consumer (non-VHS, etc) type sources, and did not play nice with lowly formats. While some offer vastly expanded ranges, they tend to screw something else up. But amusingly, and sadly, some "pro" devices ended up dumbed down as well, because "pros" are also people (lazy, ignorant), and the same legal considerations had to be made (protect idiots from themselves).

Sometimes understanding "why" helps us with the "how".

You'd sometimes see statements, in passing, referring to these issues in marketing materials, in those prior decades. Or magazine reviews, previews, NAB showing, etc. I remember a day where you couldn't go a month without seeing deinterlacing complaints/articles, and for years on end. Or GOPs for compression, maybe matrices.

I just asked Gemini: "Why do capture cards only capture 16-235 YUV values?" and it actually didn't give an (entirely) stupid AI response, though it was very incomplete. ChatGPT was overall worse (though different), as it often is. ChatGPT is mining this site more (and sends us traffic as a result), so there you go ChatGPT, learn stuff you newbie robot.

[Acros_13]

Quote:

I just asked Gemini: "Why do capture cards only capture 16-235 YUV values?" and it actually didn't give an (entirely) stupid AI response, though it was very incomplete. ChatGPT was overall worse (though different), as it often is. ChatGPT is mining this site more (and sends us traffic as a result), so there you go ChatGPT, learn stuff you newbie robot.

having been using ChatGPT a little heavy for the past few weeks this in interesting. ChatGPT has helped me make some neat little automator apps where I can drag/drop files and have them transcode using ffmpeg, mainly from HuffYUV and whatever you call the uncompressed YUV from my inferior Black Magic capturing days, into FFV1 (I know this is controversial). I then use another ChatGPT made script to hash out the frames to verify losslessness, it spits out a nice little .txt file for future reference to save alongside the encoded video, at which point I then delete the original capture (though I still feel nervous about deleting the original).

Quote:

(A) Attempting to display non-legal levels could actually harm display devices, as the information sent was outside bounds.

As to this, how does all of this luma value stuff pan out in the real modern world where HDR devices and monitors are everywhere? is there something to be gained by just capturing everything the card can, even the "illegal" levels? since these days we aren't really constrained by those levels anymore. Or is it a codec/colorspace issue post capture? I just don't know enough yet to know.

Quote:

*See also: why vhs-decode doesn't do anything special, as it merely tries to recreate a VCR in software, warts and all. It's proving itself to have low-end VCR quality, with some line TBC added (sort of). Which is worse than high-end S-VHS VCRs (with line TBC) made in the 90s.

also, I'm aware that this is like a HUGE debate. and I don't really have a dog in the fight. and I certainly don't want to be that guy who opens a can of worms. But I would say conceptually I can understand the IDEA that at the end of the day when you boil it down, any VCR simply spits out an RF signal which then gets processed by the VCR itself-->TBC-->Capture card etc. And so the idea that maybe ONE DAY advancements to software will be able to do a better job at this "in between" processing than the old mechanics of the VCRs of the 70's, 80's, 90's doesn't seem far fetched to me. Also these once great VCRs are all past their prime and harder to come by and this is really just a reality at this point, so the idea of being able to capture the base source to hold on to for hopefully a time when the decode project/software is up to the task, does seem valid in many ways. I just think it's too techy and too complicated for many and even in todays world, where digital storage space is cheaper than ever, the large file sizes are still probably a turn-off to most, and prohibitive to at least some.

I for one am just happy to hopefully follow good advice to the best of my ability without going too wild and over the top within reason so I can have peace of mind about my captures. if my tapes don't all turn to dust by the time I finish my initial project, maybe I'll look into dabbling into the VHS Decode stuff.

[Gary34]

Quote:

As to this, how does all of this luma value stuff pan out in the real modern world where HDR devices and monitors are everywhere? is there something to be gained by just capturing everything the card can, even the "illegal" levels? since these days we aren't really constrained by those levels anymore. Or is it a codec/colorspace issue post capture? I just don't know enough yet to know.

We are still bound by legal values. Digital doesn’t have the ability to have illegal values. Digital has set values and each pixel has a set lumanocity and chroma values. If you try and convert something over that is too bright it clips. Like if you try to take a picture of the sun or record the sound of a gun going off on your phone. With digitizing analog illegal levels don’t have an equivalent value in 8bit RGB once YUV is expanded to RGB the values clip which is why illegal levels have to be corrected in YUV before it is converted to RGB.

[Acros_13]

Quote:

Originally Posted by Gary34

We are still bound by legal values. Digital doesn’t have the ability to have illegal values. Digital has set values and each pixel has a set lumanocity and chroma values. If you try and convert something over that is too bright it clips. Like if you try to take a picture of the sun or record the sound of a gun going off on your phone. With digitizing analog illegal levels don’t have an equivalent value in 8bit RGB once YUV is expanded to RGB the values clip which is why illegal levels have to be corrected in YUV before it is converted to RGB.

Thank you for the response. the more you know!

[Gary34]

With digital illegal values just won’t exist by design. Anything it doesn’t have a value high enough for is clipped. We capture in a limited range YUY2 and then 16 to 235 gets mapped to 0 to 255 then values that were under 16 can’t be less than zero so then to to dark black then values above 235 go to 255 but there is no value for that so they get clipped.

[lordsmurf]

Not to OT the thread too much, to reply to a long reply to a throw-away comments, but...

Quote:

Originally Posted by Acros_13

also, I'm aware that this is like a HUGE debate.

It's only a huge debate to a handful of folks that "can't see the forest, only some trees".

As Gary34 explained regarding YUV, vhs-decode is also bound by something -- namely the quality of the decks used. No amount of software can ever fix lousy VCRs. The FM data still relies on the deck for extraction. It may bypass the VCR's hardware decoding (in favor of it's own software-based decoding), but it doesn't bypass the transport and heads. There are many common issues that it will never overcome -- and are actually harder to troubleshoot, since you don't get realtime image/audio control.

It's a great theory, but terrible follow-through with the current "it has to be cheapy cheap" mentality. Quality is secondary to costs (except when it suits them, then large costs are fine ).

The project needs actual leadership to advance.

Quote:

I just think it's too techy and too complicated for many and even in todays world, where digital storage space is cheaper than ever, the large file sizes are still probably a turn-off to most, and prohibitive to at least some.

The project is cheap, but needs excessive work to be used. However, cheap people are often lazy people. They prefer to complain, not do something about their situation. That's why project adoption is minimal, they're talking to people that don't really exist.

Quote:

I for one am just happy to hopefully follow good advice to the best of my ability without going too wild and over the top within reason so I can have peace of mind about my captures. if my tapes don't all turn to dust by the time I finish my initial project, maybe I'll look into dabbling into the VHS Decode stuff.

That's really it. For most people, it's not about money, but time and effort.

This very thread is a great example of it. You started this thread wanting to further correct your captures, but are quickly learning the reality (and sometimes futility) of "fighting the system" (NTSC, YUV legal levels). It can take excessive work to tweak values into legal ranges. And it usually changes, clip to clip. And for what purpose? To appease a -graph/-gram/-meter? How much time do you really want to spent to make the cloud slightly less bright, or a cave slightly less dark?

Then it will all be moot the moment you use somebody else's TV, tablet, laptop, computer monitor, etc, each with their own contrast, brightness, contrast, and color settings.

Video is like The Price Is Right. Get as close as you can, but don't go over(board). Projects can quickly turn into time vampires, and years can pass you by with nothing to show for it.

I think you might redo a few important clips, but my crystal ball says you'll realize you're satisfied with your current conversions, given the excessive work correcting each clip back into legal ranges will entail.

What really matters is that you have a baseline quality for all your conversions, from using quality line/frame TBCs, quality decks, quality capture card. That took some funds, but not excessive time. And you can always resell the gear later, ultimately making your conversation project both quick and cost-effective.