From what we've documented in other threads, the AVT uses
SAA7114 video decoder chips for decoding the analog video, the TBC-3000 (and possibly the TBC-1000 as well) use the older SAA7111. They're both off-the-shelf video decoders that were used in tons of stuff, but are now discontinued, though it seems they're still available second hand. It seems NXP, who last had the license for them stopped making video decoder chips in general.
The video chips output decoded video data in YUV 422 format with 720 pixels per line (or RGB but I don't think that's used here, and possibly also raw 8-bit CVBS, but that may or may not be only for VBI). They also have some additional outputs for sync signals (can also be embedded in the video output as described in
ITU-R_BT.656). Most analog video decoder chips work in a similar manner.
In the TBCs this seems to be hooked up to some memory chips, which are then hooked up to a FGPA chip. This is where any custom processing would happen (if any). The AVTs also have a separate ROM chip hooked up to it, which presumably contain code to run on the FPGA. The FPGA presumably also translates key presses to commands to the video chips. The TBC-3000 seemed to have some FPGA chip or microcontroller with embedded ROM + some simple logic chips as well.
This is then hooked up on the other side to another memory buffer, and then to an off-the-shelf video encoder chip that converts digital component data to an analog signal. On a modern alternative one would probably want to output to HDMI, SDI or over USB as a capture device instead, no reason to turn it back to analog.
There isn't really so much magic going on here, if there are custom settings on the video decoder, it would be possible to read it off the serial connection pins of the chip. The FPGA may have some logic to decide on how to deal with data that comes in too fast or slow, but I don't see a whole lot that could be done with it. It's the video decoder chip that's doing the brunt of the work, and these chips originally made by philips are quite good at that. It's possible to dump ROM chips as well, but that gets into possible copyright infringement territory.
There are some other companies making analog video decoder chips still, Analog devices even has
chips with built in line and frame TBC functions (provided it's hooked up to some memory.) Whether it's as robust as the Philips/NXP chips I don't know, it may be something that could work as a "modern" solution. I know some people on videohelp (not sure if they're here as well), have used their Eval boards as a TBC/Capture device of sorts.
Renesas and TI also still makes video decoder chips, but I don't know if they're as advanced.