Quantcast Software TBC that doesn't need the frame edges? - Page 2 - digitalFAQ Forum
Go Back    Forum > Digital Video > Video Project Help > Restore, Filter, Improve Quality

Reply
 
LinkBack Thread Tools
  #21  
09-23-2020, 12:15 PM
imgkd imgkd is offline
Premium Member
 
Join Date: Jan 2019
Location: Yes.
Posts: 17
Thanked 7 Times in 5 Posts
Quote:
Originally Posted by hodgey View Post
Have you looked into AI algorithms for doing this? People are accomplishing some pretty crazy things on images and video using AI and Machine learning these days. Maybe you could train one of these neural networks designed for video using both originals, good captures and bad captures with lots of wiggling of the same material to try to transform the bad captures into something closer to the original.
Spatial TBC 1 and Temporal TBC 1 utilize something called "stochastic optimization", which is indeed a component used in training neural networks. However, I don't think it's enough to call either TBC "AI".

With a standard supervised neural network system you need to have many training samples of TBCed vs non-TBCed clips, which are quite difficult to obtain, especially for a very wide range of types of videos (cartoon, live-action, high detail, low detail, etc.) and jitter (smooth wobble, jagged wobble, etc.) that a practical set of software TBC tools need to cover. The other alternatives are unsupervised neural networks and "untrained" neural networks (which is an unsupervised neural network being trained on-the-spot). Unsupervised neural networks may be more robust to many examples but aren't going to perform as well in general. Untrained neural networks can visually output superior results but are very very memory- and time-hungry.

I will consider exploring neural networks for TBCs, but I don't expect them to be the de facto TBCs for the reasons mentioned above.
Reply With Quote
The following users thank imgkd for this useful post: archivarious (01-12-2021)
Someday, 12:01 PM
admin's Avatar
Ads / Sponsors
 
Join Date: ∞
Posts: 42
Thanks: ∞
Thanked 42 Times in 42 Posts
  #22  
10-06-2020, 12:41 AM
jjdd jjdd is offline
Free Member
 
Join Date: Jul 2012
Posts: 137
Thanked 6 Times in 6 Posts
Hi did see this about paper-shredder machines how to reconstruction of strip-shredded text documents i was thinking if there is some help or tips how to do software TBC

http://sibgrapi.sid.inpe.br/col/sid....%20ID%2088.pdf

and

https://github.com/thiagopx/deeprec-sib18

are you using OpenCV library in python for this ?

i have tested it a little in c++ language but im not a programmer
vhs maybe to bad quality to use Edge Detection like this https://medium.com/sicara/opencv-edg...l-7c3303f10788 to align the fields
just thinking out loud

Last edited by jjdd; 10-06-2020 at 01:13 AM.
Reply With Quote
  #23  
10-06-2020, 02:35 AM
imgkd imgkd is offline
Premium Member
 
Join Date: Jan 2019
Location: Yes.
Posts: 17
Thanked 7 Times in 5 Posts
Quote:
Originally Posted by jjdd View Post
Hi did see this about paper-shredder machines how to reconstruction of strip-shredded text documents i was thinking if there is some help or tips how to do software TBC

http://sibgrapi.sid.inpe.br/col/sid....%20ID%2088.pdf

and

https://github.com/thiagopx/deeprec-sib18

are you using OpenCV library in python for this ?

i have tested it a little in c++ language but im not a programmer
Strip-shedded text document reconstruction appears to be a very different problem from software TBC. In software TBC we already know the orders of the lines and we can rely on the overall picture details to a limited extent (the extent depends on the amount of jitter). In strip-shredded text document reconstruction the order of the pieces is completely unknown.

For Spatial TBC 1 and Temporal TBC 1 I'm just using PyTorch. I'm going to explore a few more ideas, some of which will use stuff from OpenCV.

Quote:
Originally Posted by jjdd View Post
vhs maybe to bad quality to use Edge Detection like this https://medium.com/sicara/opencv-edg...l-7c3303f10788 to align the fields
just thinking out loud
That's just Canny edge detection, which in fact I might end up using for some of my ideas. However it's going to detect a lot of vertical edges that come from the jitter rather than the edges that are part of the "non-jittered" image (at least in lower settings). In some ways this could be used to an advantage.

Last edited by imgkd; 10-06-2020 at 02:39 AM. Reason: forgot thing from jjdd
Reply With Quote
  #24  
10-06-2020, 05:05 AM
jjdd jjdd is offline
Free Member
 
Join Date: Jul 2012
Posts: 137
Thanked 6 Times in 6 Posts
ok interesting to see how it progress
Reply With Quote
  #25  
10-15-2020, 03:10 PM
jjdd jjdd is offline
Free Member
 
Join Date: Jul 2012
Posts: 137
Thanked 6 Times in 6 Posts
imgkd any progress on the TBC software

i myself have been testing jmac698 TBC V0.61 and trying to understand how it works
if i understand it use a gradient mask to detect better the left and right side edges like this Picture i Attached it

and then use minmax() function to detect the lowest pixel value at every line here is that function POST Nr:3 https://forum.doom9.org/showthread.php?t=162790

and with that it creates a clip that is 2 pixel wide and that have the inputs clips hight

2 pixel wide because Left half side lowest pixel value and Right half side lowest pixel value

and that creats the offset clip that goes then in to the Rescale() function and that use the dejitter() function to offset the line by the lowest pixel value per line

example if the luma value is 3 it shift it 3 pixels then to the right or left

but this is my best guess about jmac698 TBC how it works

i did use RemoveGrain(11) on the clip i did input to the findpos_h() function that creats the offset clip and that did help little

i was thinking if it's better to resize the video only vertical to get something to work better in the tbc software

i did test this function to RemoveDeadPixels(x, 1, 7, 255) to change the lowest pixel to white X in the parameter is the horizontal coordinate to the pixel to change to white and then use a align function to align

more info here about RemoveDeadPixels function POST Nr:11 https://forum.doom9.org/showthread.p...915#post699915

a good old app to pixel peeping i use is Mouse zoom 1.5


Attached Images
File Type: png Gradient Mask.png (61.4 KB, 2 downloads)
File Type: jpg Mousezoom.jpg (24.3 KB, 4 downloads)

Last edited by jjdd; 10-15-2020 at 04:01 PM.
Reply With Quote
  #26  
10-15-2020, 04:00 PM
imgkd imgkd is offline
Premium Member
 
Join Date: Jan 2019
Location: Yes.
Posts: 17
Thanked 7 Times in 5 Posts
I've written down a couple new ideas to try out, and have been developing one idea in particular, but that's all. No new TBCs to show for now, though.

I've mostly been waiting for LS to give me some more samples so I can test out these TBCs on a wider variety of content.

Given the sheer novelty of this stuff (as something that actually works), I have been entertaining the thought of developing this into proprietary software. If I start being less specific in future posts, this is why.

jmac's software TBC project is mainly considered a failure outside of a few toy samples*.

The main problem is that if you're producing a piece of software that relies purely on the image content, such as software TBC, it is by definition a computer vision problem. You must treat it like a computer vision problem. Jmac didn't treat it like one. (He proposed later ideas that were closer, but he never actually prototyped them.)

One may argue that jmac's TBC relies on the black borders. But what are "black borders"? In real-world cases they're almost never black, the borders are not always reliable, and regardless it's still part of the image content. And often times you don't have the borders at all (a good software TBC shouldn't need the borders).

Computer vision is a major (and very broad) field consisting of tons of tools, various algorithms, and various high-level methodologies. So to write a good software TBC (or in reality, multiple software TBCs), you need to study lots of techniques in the computer vision field and have a basic understanding of how they work, their pros and cons and what they're best for. Followed by lots of trial and error.

This is what makes my suite of software TBCs distinct from Jmac's prototype.

--------

Also, I believe Jmac details this algorithm in the first post of the filter:

Quote:
Usage
Quite simple, there is an autothresh which looks for black pixels in the left border. A small amount is added to this to form the real thresh. Adjust the added amount to tweak, until the picture lines up. I used AvsPmod to look at the raw video, and moving my cursor over the black area, I saw it was a noisy 15-25 (in a user test clip). I decided to just set 32. Values slightly lower would leave a few lines wrong, seen as little black stripes at the left edge of the video. For that particular video, only the bright scenes worked correctly. Having it work for dark scenes is being experimented with.

Advanced Usage
The script first makes a mask, in this case every pixel at thresh or above is marked to luma=255 in the mask or 0 elsewhere.
Next I pass my mask to findpos_h which searches for the first 255 value on each line, within the searchwindow. It also simultaneously searches from right to left by searchwindow pixels. It saves the results in a special clip, which records the number of pixels before the mask was found. For example if the HSYNC line occurs at x=4, the luma of the shift clip contains luma=4 for that line, at the left hand side. The right hand side contains the offset from the right of where the video ended, for example if it ended at x=710 and the video is 720 pixels wide, the luma of shift is 10 on the right hand side.
alignbyluma now reads the original video and the shift video and resizes to a standard size. Hopefully this is enough information to do anything else you want here.
My interpretation of this is it creates a video with only completely white and completely black pixels, where the white ones are pixels that are brighter than the threshold, and the black ones otherwise. Then the left-most and right-most white pixels are used to determine where the jitter is.

Quote:
i was thinking if it's better to resize the video only vertical to get something to work better in the tbc software
There are many different types of jitter. That idea may work for more "jagged" sources, but that, or any sort of purely spatial techniques for that matter, won't work at all for smooth wobble-type jitter.

--------

*To contrast this, the only thing I'm using to test my software TBCs are real-world examples. Therefore, if the TBCs work, they work on real-world samples.

Last edited by imgkd; 10-15-2020 at 04:28 PM.
Reply With Quote
  #27  
10-15-2020, 04:30 PM
jjdd jjdd is offline
Free Member
 
Join Date: Jul 2012
Posts: 137
Thanked 6 Times in 6 Posts
ok thanks for the update and information
Reply With Quote
  #28  
01-11-2021, 07:48 PM
Daegalus Daegalus is offline
Free Member
 
Join Date: Oct 2016
Posts: 13
Thanked 0 Times in 0 Posts
Any progress on this? this looks like something I could greatly use as I am having a hard time finding a Field TBC tht isnt an arm and a leg in price, and my VCR only has a line tbc.

As a dev, I think python is the better choice, especially with all the integrations you can do in the future with AI/ML. Even if we dont train our own, existing stuff can just improve image quality overall. I have seen some crazy AI enhancement of very old footage.

Youtube has a lot of examples like https://www.youtube.com/watch?v=EjVzjxihGvU

AS for proprietary. I encourage keeping it open source and allowing the community to work on, but I know everyone deserves to make money how they can. I just hope you keep the pricing reasonable if you do monetize.
Reply With Quote
  #29  
01-11-2021, 08:48 PM
lordsmurf's Avatar
lordsmurf lordsmurf is offline
Site Staff | Video
 
Join Date: Dec 2002
Posts: 10,326
Thanked 1,782 Times in 1,547 Posts
Quote:
Originally Posted by Daegalus View Post
Any progress on this? this looks like something I could greatly use as I am having a hard time finding a Field TBC tht isnt an arm and a leg in price, and my VCR only has a line tbc.
You're not understanding what this is for. This isn't something to avoided costs of an external frame TBC, nor line TBC. It's a post-capture line TBC, for restoration, when sources are gone. If you have the original tapes, recapture with a proper workflow. This will never supplant a proper hardware workflow, merely complement it.

Quote:
Youtube has a lot of examples like https://www.youtube.com/watch?v=EjVzjxihGvU
That has nothing to do with this project.

Quote:
AS for proprietary. I encourage keeping it open source and allowing the community to work on
This sort of specialized video tasks is more likely to be harmed by community support, not aided. I've seen this happen before, and it didn't end well. I'm all for open-source, but it's not always the best choice for everything.

- Did my advice help you? Then become a Premium Member and support this site.
- Find television shows, cartoons, DVDs and Blu-ray releases at the TVPast forums.
Reply With Quote
  #30  
01-11-2021, 09:49 PM
Daegalus Daegalus is offline
Free Member
 
Join Date: Oct 2016
Posts: 13
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by lordsmurf View Post
You're not understanding what this is for. This isn't something to avoided costs of an external frame TBC, nor line TBC. It's a post-capture line TBC, for restoration, when sources are gone. If you have the original tapes, recapture with a proper workflow. This will never supplant a proper hardware workflow, merely complement it.

That has nothing to do with this project.

This sort of specialized video tasks is more likely to be harmed by community support, not aided. I've seen this happen before, and it didn't end well. I'm all for open-source, but it's not always the best choice for everything.
Hmm, understandable, but at the same time. Unless there is ever an affordable TBC option (sub-$100) then external TBC isn't an option for many (for many financial reasons). Maybe Ill dabble in AI restoration or AI post-capture field TBC myself, but thats a long term idea.

And I know you say many times it can be purchased and then sold, but you still need to fork over the money, and life gets in the way and 2-3 years later, its still sitting in the closet. So it's not a great option for many people, because that's just how people work.

I posted here a couple years ago. Got a VS30U, and the ATI Wonder 600 USB. But an external TBC has always alluded me due to cost (I managed to get a KeyWest Voodoo for $70, but it caused audio sync issues and died about 20 mins in). I got the VCR for like $50 working, and the 600 USB for something similar if not cheaper. But then the TBC prices re $200+ and rising. Just a balance of quality and money.

So if we can use software to augment that, it could save people money. I don't expect it to be as good as external, but even if its 30-50% as good, it's better than 0. And even just that could be good enough for some to ignore the external TBC.

Because at this point, I am about to just give up and capture VCR > PC directly, and call it a day and just hope I get decent quality without an external TBC

Unfortunately I am not great at AI or Video, even if I am a dev, I specialized in a different direction for my career, so it would take me good while to ramp up. I would need to dig deeper than my surface level understanding of all the technical aspects of analog video capture.

So I get it LordSmurf, and I know you are right and where you are coming from, and I respect your experience, but as hardware becomes increasingly harder and harder to get, some of this stuff needs to move to software, or we need to start designing an open source board to do all this. Maybe push into the maker scene and program a bunch of arduinos to do what 1 board does, or design a custom board. Either way, we can't just rely on old hardware forever. We might need to program an arduino to do what 1 tiny chip did on the original board. But we have to start trying stuff and pushing the envelope (i wish i was more into hardware too at this point).
Reply With Quote
  #31  
01-12-2021, 01:09 AM
lordsmurf's Avatar
lordsmurf lordsmurf is offline
Site Staff | Video
 
Join Date: Dec 2002
Posts: 10,326
Thanked 1,782 Times in 1,547 Posts
Quote:
Originally Posted by Daegalus View Post
external TBC isn't an option for many
Software TBC cannot recreate frame TBC.

Quote:
(for many financial reasons).
Yes, but:
- If hobby -- All hobbies have costs (DIY video = hobby, even if short-term hobby). Buy it, use it, resell it.
- If business -- All businesses require capex. Quit being a cheapskate, it will only result in horrible quality that should never be foisted upon paying customers of services.

Quote:
Maybe Ill dabble in AI restoration or AI post-capture field TBC myself
Please use "AI" correctly, as it's gotten to be ridiculous in recent years. NN alone is not AI.

Quote:
And I know you say many times it can be purchased and then sold, but you still need to fork over the money, and life gets in the way and 2-3 years later, its still sitting in the closet. So it's not a great option for many people, because that's just how people work.
Nod of understanding. Yep.

Quote:
I posted here a couple years ago. Got a VS30U, and the ATI Wonder 600 USB.
Good stuff.

Quote:
(I managed to get a KeyWest Voodoo for $70, but it caused audio sync issues and died about 20 mins in).
Yep, almost always a bad TBC, not unexpected. (Exception = early production models, not commonly found.)

Quote:
I got the VCR for like $50 working, and the 600 USB for something similar if not cheaper.
Nice. Some years ago, that was possible. Not common, rare actually, but possible. But not now.

Quote:
But then the TBC prices re $200+ and rising.
I don't know that anything was ever $200 -- at least not a good model of anything.

Quote:
So if we can use software to augment that, it could save people money. I don't expect it to be as good as external, but even if its 30-50% as good, it's better than 0. And even just that could be good enough for some to ignore the external TBC.
It's not about "quality" or "being good". It does not work, at all. Software is an attempt to untangle bad timing -- line TBC functionality. Frame TBC isn't for that, it's for frame, most notably dropped frames. Once a frame is lost, you can't get it back. It's not about quality, it's about not being a function that can exist. You cannot create the missing data. It's missing.

Quote:
Because at this point, I am about to just give up and capture VCR > PC directly, and call it a day and just hope I get decent quality without an external TBC
You need a budget option. That entails (at minimum_ the ES10/15, ideally add the DataVideo DVK to create a TBC(ish). What you suggest, VCR>card, will almost always result in dropped frames, because you have no framesync TBC (the ES10/15 sort-of privode line TBC + frame sync, but not frame sync TBC). The DVK is a weak frame TBC, which is why you add it -- the video must be pre-processed, and VCR line TBC sometimes isn't enough of a purifier to appease the DVK.

Quote:
but as hardware becomes increasingly harder and harder to get, some of this stuff needs to move to software,
You must understand that some functionality cannot be recreated in software, and probably never will. It's not that easy, otherwise it would have been done long ago.

Quote:
or we need to start designing an open source board to do all this.
It's not that easy. The cost to simply develop a board is more than the $200 that you (seemingly?) want to spend on a TBC.

Quote:
Either way, we can't just rely on old hardware forever.
I agree. It's the main reason I got into hardware refurb work 5 years ago. If I can at least restore gear, it gives the community more to work with.

Quote:
But we have to start trying stuff and pushing the envelope (i wish i was more into hardware too at this point).
Some of us already are. It's not that easy.

- Did my advice help you? Then become a Premium Member and support this site.
- Find television shows, cartoons, DVDs and Blu-ray releases at the TVPast forums.
Reply With Quote
  #32  
01-12-2021, 05:56 PM
Daegalus Daegalus is offline
Free Member
 
Join Date: Oct 2016
Posts: 13
Thanked 0 Times in 0 Posts
After reading up on how a frame TBC worked, I was like "You can do that in code" but something clicked finally about analog signals. I was completely forgetting the part that in order for any software to process that signal, it needs to be converted to a digital signal. and the Analog to digital conversion is how you lose the data needed to create a frame. Now I realize where i was making an error in my logic.

I also have kind of derailed this topic, though I do want to add a last thing. Now that I realize what you are getting at, maybe the key isnt a TBC per say, but the Analog to Digital conversion. We do various conversions to an image. But maybe we need to take the analog signal and convert it to just labeled data? take each line of video, or whatever we can get, label it up with all the meta data, and send it on. Then recreate images from the data + labels and then use the label information to do things normally you couldn't in software? Just spitballing, I know its probably been considered already, but I feel like there is something there.

Thank you for the ES15/DVK suggestion. I might go this route, because even if it not super cheap, its multiple parts that can be purchased over time and better budgeted for.

And you mentioned hobby or business, but im not really either. THis is a 1 time project for me. The only reason i am going this far, is these tapes are the only remnant of memories of my wife's father, who she lost in her early 20s. Due to fires, and time , pictures, and many things are lost. One of those is voice, which is only on these tapes. So I want to do everything I can to squeeze as much as possible out, without breaking the bank on top tier equipment.
Reply With Quote
  #33  
01-12-2021, 06:11 PM
lordsmurf's Avatar
lordsmurf lordsmurf is offline
Site Staff | Video
 
Join Date: Dec 2002
Posts: 10,326
Thanked 1,782 Times in 1,547 Posts
Quote:
Originally Posted by Daegalus View Post
I also have kind of derailed this topic,
That was my thought at first as well, but it is also important to state what a software TBC "is not". The intention is line TBC, on already-digitized captures, where the original source tapes are no longer available. The intention is not to be cheaper, lazier, recreate lost data, etc.

This is not likely to be an "easy button" newbie/dummy-friendly approach, but I imagine it to be more like barrel/pincushion distortion correction in camera lenses. Yes, it will have a set of profiles for common situations, but it will have some manual correction ability as well.

Quote:
but the Analog to Digital conversion. We do various conversions to an image. But maybe we need to take the analog signal and convert it to just labeled data? take each line of video,
That's where you're still not understanding the role of a TBC. Analog video data isn't always "whole", but partials. This is a simple way to put it, but it suffices for this conversation. A capture device requires a "whole" frame, otherwise it has to be dumped. The TBC "makes it whole" for capture. Capturing partials/pieces is not possible because of how the incoming data is timed. The ingest device would need a TBC instead -- and to be honest, that's how it should have been! But it costs more $$, more R&D, so capture card makers didn't do it.

Arguably, line TBC should have been inclusive as well. All DVD recorders (not just the rare ES10/15), all capture cards. But that's just not what happened.

So what we have now are lots of wiggly crappy videos that need to be saved with a software TBC -- if it can exist. And I think we're getting close now, with IMGKD leading the development.

Quote:
And you mentioned hobby or business, but im not really either. THis is a 1 time project for me.
I figured you weren't in it for business, and that makes it hobby. Yes, short-term hobby is still hobby. Video isn't like home DIY (not really hobby) were you decide to change out your own sink plumbing connections. I would argue that plumbing is a need, while video is a want (but an important want, ie family history).

Quote:
The only reason i am going this far, is these tapes are the only remnant of memories of my wife's father, who she lost in her early 20s. Due to fires, and time , pictures, and many things are lost. One of those is voice, which is only on these tapes. So I want to do everything I can to squeeze as much as possible out, without breaking the bank on top tier equipment.
Thank you for the ES15/DVK suggestion. I might go this route, because even if it not super cheap, its multiple parts that can be purchased over time and better budgeted for.
I think you'll do fine, you're on the right path here. You have the original tapes, you're planning to use good gear (JVC with line TBC), so you don't need software TBC.

... and I think enough derail for now.

- Did my advice help you? Then become a Premium Member and support this site.
- Find television shows, cartoons, DVDs and Blu-ray releases at the TVPast forums.
Reply With Quote
Reply




Similar Threads
Thread Thread Starter Forum Replies Last Post
Video8 deinterlace has jagged edges? FreeLemons Restore, Filter, Improve Quality 35 01-22-2020 11:53 AM
Strange edges on Verbatim discs HeyMcfly Blank Media 2 09-22-2019 04:25 PM
Software that does/doesn't run on Wine (run Windows apps in Mac OS X) kpmedia Computers 5 04-08-2013 07:58 PM
Errors on the edges of converted VHS tapes? xyciana Capture, Record, Transfer 3 09-14-2012 12:44 PM
Software recommendations for frame replacement ? lordsmurf Restore, Filter, Improve Quality 6 12-23-2010 04:58 AM

Thread Tools



 
All times are GMT -5. The time now is 08:38 PM