Quantcast Software TBC that doesn't need the frame edges? - Page 2 - digitalFAQ Forum
Go Back    Forum > Digital Video > Video Project Help > Restore, Filter, Improve Quality

Reply
 
LinkBack Thread Tools
  #21  
09-23-2020, 12:15 PM
imgkd imgkd is offline
Premium Member
 
Join Date: Jan 2019
Location: Yes.
Posts: 16
Thanked 4 Times in 4 Posts
Quote:
Originally Posted by hodgey View Post
Have you looked into AI algorithms for doing this? People are accomplishing some pretty crazy things on images and video using AI and Machine learning these days. Maybe you could train one of these neural networks designed for video using both originals, good captures and bad captures with lots of wiggling of the same material to try to transform the bad captures into something closer to the original.
Spatial TBC 1 and Temporal TBC 1 utilize something called "stochastic optimization", which is indeed a component used in training neural networks. However, I don't think it's enough to call either TBC "AI".

With a standard supervised neural network system you need to have many training samples of TBCed vs non-TBCed clips, which are quite difficult to obtain, especially for a very wide range of types of videos (cartoon, live-action, high detail, low detail, etc.) and jitter (smooth wobble, jagged wobble, etc.) that a practical set of software TBC tools need to cover. The other alternatives are unsupervised neural networks and "untrained" neural networks (which is an unsupervised neural network being trained on-the-spot). Unsupervised neural networks may be more robust to many examples but aren't going to perform as well in general. Untrained neural networks can visually output superior results but are very very memory- and time-hungry.

I will consider exploring neural networks for TBCs, but I don't expect them to be the de facto TBCs for the reasons mentioned above.
Reply With Quote
Someday, 12:01 PM
admin's Avatar
Ads / Sponsors
 
Join Date: ∞
Posts: 42
Thanks: ∞
Thanked 42 Times in 42 Posts
  #22  
10-06-2020, 12:41 AM
jjdd jjdd is offline
Free Member
 
Join Date: Jul 2012
Posts: 137
Thanked 6 Times in 6 Posts
Hi did see this about paper-shredder machines how to reconstruction of strip-shredded text documents i was thinking if there is some help or tips how to do software TBC

http://sibgrapi.sid.inpe.br/col/sid....%20ID%2088.pdf

and

https://github.com/thiagopx/deeprec-sib18

are you using OpenCV library in python for this ?

i have tested it a little in c++ language but im not a programmer
vhs maybe to bad quality to use Edge Detection like this https://medium.com/sicara/opencv-edg...l-7c3303f10788 to align the fields
just thinking out loud

Last edited by jjdd; 10-06-2020 at 01:13 AM.
Reply With Quote
  #23  
10-06-2020, 02:35 AM
imgkd imgkd is offline
Premium Member
 
Join Date: Jan 2019
Location: Yes.
Posts: 16
Thanked 4 Times in 4 Posts
Quote:
Originally Posted by jjdd View Post
Hi did see this about paper-shredder machines how to reconstruction of strip-shredded text documents i was thinking if there is some help or tips how to do software TBC

http://sibgrapi.sid.inpe.br/col/sid....%20ID%2088.pdf

and

https://github.com/thiagopx/deeprec-sib18

are you using OpenCV library in python for this ?

i have tested it a little in c++ language but im not a programmer
Strip-shedded text document reconstruction appears to be a very different problem from software TBC. In software TBC we already know the orders of the lines and we can rely on the overall picture details to a limited extent (the extent depends on the amount of jitter). In strip-shredded text document reconstruction the order of the pieces is completely unknown.

For Spatial TBC 1 and Temporal TBC 1 I'm just using PyTorch. I'm going to explore a few more ideas, some of which will use stuff from OpenCV.

Quote:
Originally Posted by jjdd View Post
vhs maybe to bad quality to use Edge Detection like this https://medium.com/sicara/opencv-edg...l-7c3303f10788 to align the fields
just thinking out loud
That's just Canny edge detection, which in fact I might end up using for some of my ideas. However it's going to detect a lot of vertical edges that come from the jitter rather than the edges that are part of the "non-jittered" image (at least in lower settings). In some ways this could be used to an advantage.

Last edited by imgkd; 10-06-2020 at 02:39 AM. Reason: forgot thing from jjdd
Reply With Quote
  #24  
10-06-2020, 05:05 AM
jjdd jjdd is offline
Free Member
 
Join Date: Jul 2012
Posts: 137
Thanked 6 Times in 6 Posts
ok interesting to see how it progress
Reply With Quote
  #25  
10-15-2020, 03:10 PM
jjdd jjdd is offline
Free Member
 
Join Date: Jul 2012
Posts: 137
Thanked 6 Times in 6 Posts
imgkd any progress on the TBC software

i myself have been testing jmac698 TBC V0.61 and trying to understand how it works
if i understand it use a gradient mask to detect better the left and right side edges like this Picture i Attached it

and then use minmax() function to detect the lowest pixel value at every line here is that function POST Nr:3 https://forum.doom9.org/showthread.php?t=162790

and with that it creates a clip that is 2 pixel wide and that have the inputs clips hight

2 pixel wide because Left half side lowest pixel value and Right half side lowest pixel value

and that creats the offset clip that goes then in to the Rescale() function and that use the dejitter() function to offset the line by the lowest pixel value per line

example if the luma value is 3 it shift it 3 pixels then to the right or left

but this is my best guess about jmac698 TBC how it works

i did use RemoveGrain(11) on the clip i did input to the findpos_h() function that creats the offset clip and that did help little

i was thinking if it's better to resize the video only vertical to get something to work better in the tbc software

i did test this function to RemoveDeadPixels(x, 1, 7, 255) to change the lowest pixel to white X in the parameter is the horizontal coordinate to the pixel to change to white and then use a align function to align

more info here about RemoveDeadPixels function POST Nr:11 https://forum.doom9.org/showthread.p...915#post699915

a good old app to pixel peeping i use is Mouse zoom 1.5


Attached Images
File Type: png Gradient Mask.png (61.4 KB, 2 downloads)
File Type: jpg Mousezoom.jpg (24.3 KB, 3 downloads)

Last edited by jjdd; 10-15-2020 at 04:01 PM.
Reply With Quote
  #26  
10-15-2020, 04:00 PM
imgkd imgkd is offline
Premium Member
 
Join Date: Jan 2019
Location: Yes.
Posts: 16
Thanked 4 Times in 4 Posts
I've written down a couple new ideas to try out, and have been developing one idea in particular, but that's all. No new TBCs to show for now, though.

I've mostly been waiting for LS to give me some more samples so I can test out these TBCs on a wider variety of content.

Given the sheer novelty of this stuff (as something that actually works), I have been entertaining the thought of developing this into proprietary software. If I start being less specific in future posts, this is why.

jmac's software TBC project is mainly considered a failure outside of a few toy samples*.

The main problem is that if you're producing a piece of software that relies purely on the image content, such as software TBC, it is by definition a computer vision problem. You must treat it like a computer vision problem. Jmac didn't treat it like one. (He proposed later ideas that were closer, but he never actually prototyped them.)

One may argue that jmac's TBC relies on the black borders. But what are "black borders"? In real-world cases they're almost never black, the borders are not always reliable, and regardless it's still part of the image content. And often times you don't have the borders at all (a good software TBC shouldn't need the borders).

Computer vision is a major (and very broad) field consisting of tons of tools, various algorithms, and various high-level methodologies. So to write a good software TBC (or in reality, multiple software TBCs), you need to study lots of techniques in the computer vision field and have a basic understanding of how they work, their pros and cons and what they're best for. Followed by lots of trial and error.

This is what makes my suite of software TBCs distinct from Jmac's prototype.

--------

Also, I believe Jmac details this algorithm in the first post of the filter:

Quote:
Usage
Quite simple, there is an autothresh which looks for black pixels in the left border. A small amount is added to this to form the real thresh. Adjust the added amount to tweak, until the picture lines up. I used AvsPmod to look at the raw video, and moving my cursor over the black area, I saw it was a noisy 15-25 (in a user test clip). I decided to just set 32. Values slightly lower would leave a few lines wrong, seen as little black stripes at the left edge of the video. For that particular video, only the bright scenes worked correctly. Having it work for dark scenes is being experimented with.

Advanced Usage
The script first makes a mask, in this case every pixel at thresh or above is marked to luma=255 in the mask or 0 elsewhere.
Next I pass my mask to findpos_h which searches for the first 255 value on each line, within the searchwindow. It also simultaneously searches from right to left by searchwindow pixels. It saves the results in a special clip, which records the number of pixels before the mask was found. For example if the HSYNC line occurs at x=4, the luma of the shift clip contains luma=4 for that line, at the left hand side. The right hand side contains the offset from the right of where the video ended, for example if it ended at x=710 and the video is 720 pixels wide, the luma of shift is 10 on the right hand side.
alignbyluma now reads the original video and the shift video and resizes to a standard size. Hopefully this is enough information to do anything else you want here.
My interpretation of this is it creates a video with only completely white and completely black pixels, where the white ones are pixels that are brighter than the threshold, and the black ones otherwise. Then the left-most and right-most white pixels are used to determine where the jitter is.

Quote:
i was thinking if it's better to resize the video only vertical to get something to work better in the tbc software
There are many different types of jitter. That idea may work for more "jagged" sources, but that, or any sort of purely spatial techniques for that matter, won't work at all for smooth wobble-type jitter.

--------

*To contrast this, the only thing I'm using to test my software TBCs are real-world examples. Therefore, if the TBCs work, they work on real-world samples.

Last edited by imgkd; 10-15-2020 at 04:28 PM.
Reply With Quote
  #27  
10-15-2020, 04:30 PM
jjdd jjdd is offline
Free Member
 
Join Date: Jul 2012
Posts: 137
Thanked 6 Times in 6 Posts
ok thanks for the update and information
Reply With Quote
Reply




Similar Threads
Thread Thread Starter Forum Replies Last Post
Video8 deinterlace has jagged edges? FreeLemons Restore, Filter, Improve Quality 35 01-22-2020 11:53 AM
Strange edges on Verbatim discs HeyMcfly Blank Media 2 09-22-2019 04:25 PM
Software that does/doesn't run on Wine (run Windows apps in Mac OS X) kpmedia Computers 5 04-08-2013 07:58 PM
Errors on the edges of converted VHS tapes? xyciana Capture, Record, Transfer 3 09-14-2012 12:44 PM
Software recommendations for frame replacement ? lordsmurf Restore, Filter, Improve Quality 6 12-23-2010 04:58 AM

Thread Tools



 
All times are GMT -5. The time now is 05:34 AM