Software TBC started, progress update?

[jmac698]

Well for now here's a screenshot of my latest script on a user supplied video, showing that it actually does work, at least in some cases:
http://screenshotcomparison.com/comparison/91853

This is my dejitter script - this one lines up images with black borders in them. My "perfect TBC" is something totally different, and my "intrinsic dejitter" is again totally different (it doesn't need the black borders).

jittered.jpg - artificially jittered test image, with embedded test signal (not shown)
dejittered.jpg - dejittered back to original by using embedded test signal

composite.jpg - a real VHS recording of a test image, that was dejittered perfectly by using an embedded test signal (test signal removed)
component.jpg - a real VHS recording of a test image, that used an embedded test signal, and then perfectly dejittered, and also using a component video - on VHS scheme

original.jpg - a user supplied, badly jittered video for which the original tape is not available
processed.jpg - dejittered with my dejitter script, which simply lines up the black borders

For a gallery, this isn't mouseover but the changing images is pretty quick when cached
http://www.cpurigs.com/forums/showthread.php?t=2056
http://www.cpurigs.com/forums/vbpicg...ig&p=514#photo

[lordsmurf]

Albums are native to the forum software.
Here's yours: http://www.digitalFAQ.com/forum/memb...98-albums.html

You can add images there, too.

[kpmedia]

I spy something new that you've not shared here yet. Saw this at VH:

#Fast line shifter Ver 0.52 by jmac698
#Lines up either or both edges of a video.  Can also be used as displacement for 3d scripts.
#Requires Masktools v2a45+ (mt_lutspa mode), GScript http://forum.doom9.org/showthread.php?t=147846
#MinMax http://forum.doom9.org/showthread.php?p=1532124#post1532124


#Modified to work with Cherbette's sample (also required dgdecode and a dgindex *.d2v for the source video)
#warning this is extremely slow

src=Mpeg2Source("D:\project001a\sampleproblem5\Untitled.d2v").converttoyuy2.crop(8,0,0,0).trim(555,555+117)

thresh=32
ScriptClip(src, """
#Mark video edges
converttoyv12
m=mt_binarize(thresh)
#Line up video
findpos_h(m, searchwidth=22)
alignbyluma(src,last)
""")
converttoyv12
#I used the lines below in another script for a 2nd pass at the video.  Don't enable here they are ridiculously slow and likely to crash.
#MCTemporalDenoise(settings="high")
#QTGMC( Preset="Slow" , Sharpness=1.2, SLMode=1).selecteven

function findpos_h(clip m, int "searchwidth", int "x1", int "x2"){
    #Searches m from left to right in the range x1 to x1+searchwidth-1 and right to left in the range width-1-x2 to x2-searchwidth-1
    #for the first luma=255 pixel, then colors the output line with the offset from x
    #for example m is 0 0 255 255 255 0 0 0, width=8, x1=0, x2=0, searchwidth=4 becomes 4 4 2 3 3 4 4 4, then 2 2 2 2 3 3 3 3 
    #Can only search for 255 pixels (as the luma output is only 8 bit)
    #c and m should have the same clip properties (same size)
    #searchwidth should be <=width/2
    searchwidth=default(searchwidth,32)
    x1=default(x1,0)
    x2=default(x2,0)
    rampexpr="x "+string(m.width/2)+" < x "+string(m.width-1)+" x - ?"#x w/2 < x w-1 x - ?
    ramp=mt_lutspa(m, mode="absolute",expr=rampexpr)
    notfound=searchwidth#Value to return if no mask on this line, should be >=searchwidth or you'll find the wrong minimum later
    maskmarker=255#The luma value in the mask which indicates a detected pixel
    #(if m=maskmarker return ramp else notfound), 255 means x>=255, x<searchwidth or notfound
    mt_lutxy(m,ramp,yexpr="x "+string(maskmarker)+" = y "+string(notfound)+" ?")
    #now make solid lines based on min luma found in each line
    l=crop(0,0,-width/2,0)
    r=crop(width/2,0,0,0)
    l=l.minmax(0,0)
    r=r.minmax(0,0)
    StackHorizontal(l,r)
}

function alignbyluma(clip src, clip shift, int "mode"){
    #Shift/scale each line of clip src by the x offset defined by the luma of shift
    #for example if shift were all luma=8, the entire src clip would move 8 pixels to the (dir)
    #This works on a pixel basis, so solid horizontal lines in shift can shift src by variable amounts per line
    #It uses a simple replacement strategy, where each pixel in shift is tested and replaced by the same pixel in a shifted copy
    #Currently handles only 0-15 shifts
    #Magnify everything to get full color resolution
    mode=default(mode, 2)
    shiftuv=shift
    shift=shift.pointresize(shift.width*2,shift.height*2)
    shift=ytouv(shiftuv,shiftuv,shift)
    src=src.pointresize(src.width*2,src.height*2)#We double here to preserve chroma rez
    GScript("
        for (y=0, src.height/2-1, 1) {
            l=int(getpixel(shift,0,y).YPlaneMin)
            r=int(getpixel(shift,shift.width/2-2,y).YPlaneMin)
            getline(src, y*2)
            align(l*2, r*2, 12, 4)
            out=y==0?last:stackvertical(out,last)
        }#for y
    ")#GScript
    out
    converttoyuy2
    bilinearresize(src.width/2,src.height/2)
}

function align(clip v, int xl, int xr, int lb, int rb, int "mode") {
    v#shift an image, x>0 shifts left, xl is amount to shift left, xr is amount to shift right
    #mode 0 is shift left only, 1 shift right, 2 scale to shift left and right
    mode=default(mode, 2)
    offx=mode==0?xl:-xr
    mode<2?pointresize(last.width, last.height, offx, 0, last.width, last.height):crop(xl,0,-xr,0).addborders(lb, 0, rb, 0).Spline36Resize(last.width,last.height)
}

function getpixel(clip v, int x, int y) {
    #get color of a single pixel and return as a fat 2x2 yv12 pixel
    v
    #pointresize(last.width*2,last.height*2)
    crop(x>0?x*2:0,y>0?y*2:0,-(last.width-x*2-2),-(last.height-y*2-2))
}

function getline(clip v, int y) {
    v#return a line of height 2 from y to y+1
    crop(0,y,0,-(last.height-y-2))
}

function findthresh(clip v){
    #Find a resonable starting point for thresh by searching border
    current_frame=0
    v.converttoyv12
    crop(0,16,-last.width+2,-16)
    AverageLuma
}

Definitely go over your Avisynth scripts when you get a chance.
This may be something that I can incorporate into an existing and ongoing guide series involving Avisynth and VHS problem scenarios.

[jmac698]

Ya, that's pretty immature version. I think I released it too soon, a lot of people are trying to use it and I'm spending more time answering their questions than actually finishing it
Problem is, it only works on the bright scenes, so it's probably not practical to anyone yet.

I didn't think avisynth was very big here, you seem more hardware/virtualdub oriented as opposed to programming new scripts for each video.

[lordsmurf]

Quote:

a lot of people are trying to use it and I'm spending more time answering their questions than actually finishing it

I'd suggest this, to make your life easier:

1. Give a brief and quick overview of whatever everything does. Literally one sentence per function, maybe a few extra sentences to describe a key variable and what a change does. Just do this once, and never again until you're ready to write a full manual in detail.
2. Release each version as attached txt/avs files.
3. If somebody doesn't understand it, then unfortunately for them they're just out of luck for the moment. They'll have to wait until the project is closer to beta or done entirely.
4. Keep releasing versions. I've seen many projects disappear entirely because the developer lost interest and nothing was ever released. Great ideas, nice samples, but no clue on how to recreate what was done. This way, if you give up, others can continue on the work.
5. In fact, it may even strike enough interest in others to be forked.

I've been very vocal against sucky Avisynth documentation for years, but something like this, where you're constantly developing it, there's good reason to not sidetrack yourself with Q&A's, how-to docs, etc. That can wait. Just don't never do it, like many of the past developers have done. The brief overview and some notes is good enough for now.

For each new release, just give a couple of quick mentions on what was changed/tweaked, without writing an explanatory novel. We'll either pick up what's going on, or we won't.

I don't want to see you overwhelmed by people --especially stupid/lazy ones that just want a big "Go!" button to "fix video" -- as opposed to keeping this an interesting and enjoyable experience for tackling this video problem. That's a big problem I have to deal with, often spending vast amounts of times documenting and answering questions, instead of doing the research, dev and application.

[robjv1]

Quote:

Originally Posted by robjv1

Me too! This is all very cool. I'm wondering if it would be possible to apply these same methods to fixing the issue caused by JVC VCR TBCs where a tape playing back with the TBC engaged occasionally causes the picture to shift up and then down (usually about four frames, first shifted +2 to +4 pixels above the X axis, down -1 or -2 below it, and then back up to 0) and the interlaced frames are smudged together and/or "mangled". Once I get my new computer up and running, I'll see about posting an example.

Quote:

Originally Posted by jmac698

That shouldn't be too hard. I'm in the middle of some other things though. Let me know when you got a sample.

I've uploaded a small clip from an old Cable TV airing of BeetleJuice to demonstrate this error. Note that my definition of what the "error" is (see above) certainly does not cover all of the problems with the video and in fact, both clips demonstrate visual problems in the same set of frames, but I'm looking to specifically correct the error as described above that occurs in the TBC ON version of the clip, in some consecutive and non consecutive frames between frame 131 and frame 144.

I actually have (somewhere) a file with about 40 examples of this error that I've come across over various tapes, which I'll try to find later, although this is pretty representative of the typical case. Note that the amount of frame displacement vertically can vary by maybe +/- 3 frames. The length of time each frame is displaced from it's original position can vary a bit too. Typically these sorts of instances are brief though and scattered, never lasting more a second at a time. As LS has outlined previously, errors with these characteristics tend to occur on tapes with other problems, when the TBC frame buffer memory is unable to compensate. As a note, my particular JVC deck has a 4MB frame buffer.

Ultimately I'm wondering if this kind of error is addressable once in the digital domain. If I could re-encode (obviously with a slight penalty to image quality) small batches of frames (hidden between cuts typically) without doing separate non-TBC or ES-10 encodes it would surely save me a lot of time. Even better would be an automated tool could scan the video stream and reliably identify instances of such an error. I've played around a bit myself with log files of the AviSynth DeJitter filter to see if I could identify them based on the shifts of the vertical axis pixel values, but I could not reliably identify them based on that data alone and the correction was spotty as well.

Both clips are accessible here as TBC Clip.MPG and No TBC Clip.MPG

http://www.wou.edu/~rvitolo06/test/

[admin]

Ugh... the vertical jitter (image bounce) vs horizontal jitter (time base instability) issue.
It's more pronounced on JVC S-VHS VCRs, but Panasonic is far from immune.

Also, your clips are small enough to attach to forum posts (under 16MB each).

[robjv1]

Quote:

Originally Posted by admin

Ugh... the vertical jitter (image bounce) vs horizontal jitter (time base instability) issue.
It's more pronounced on JVC S-VHS VCRs, but Panasonic is far from immune.

Also, your clips are small enough to attach to forum posts (under 16MB each).

Indeed! In my case, it seems to mostly impact home recorded tapes the of a particular generation in my collection, although I see the occasional issue on commercial tapes. Typically when I run into this scenario, if it's pretty bad I'll just use the JVC with the TBC off with ES10, which aside from a slightly detrimental impact on some of the qualities of the picture, fixes it up fine to an acceptable level. Some darker tapes just don't look good with the ES-10 though and so I get into mixing TBC footage with color-matched footage from the ES-10 in areas where this error occurs. On tapes where there are maybe 50 or 60 instances like this, it can take many hours to identify and correct all of them. It'd be nice to be able to do it after the fact from the TBC footage, when I actually get around to watching it!

[jmac698]

Yep, I think I know what that is, and there's a fix for it

[robjv1]

Quote:

Originally Posted by jmac698

Yep, I think I know what that is, and there's a fix for it

Awesome, that is very exciting! I think we're at the point in all this VHS to DVD stuff that we're not going to see much more innovation and tweaking hardware wise, but you're making some impressive inroads using software. What is your background in programming by the way?

At any rate, I'd definitely be willing to make some sort of a donation towards the work and towards the site as well.

[jmac698]

Organization
Could you move post#32 on to a new thread, "Dropped/Duped Fields (Plays as Vertical Jitter)"

The Problem
Video looks like it's jumping when passed through TBC. Applicable only to this particular sample.

The Analysis
The pattern is this, with field numbers:
0/1, 2/3, 2/5, 6/7, 8/7, ... and repeating.
In other terms it's, ok/ok, ok/ok, dup/ok, ok/ok, ok/dup, ...or you're missing 2 of every 10 fields.

The Repair
It's easy and high quality interpolation, hardly noticable.

Fixing existing files
I can create a text file with ranges of frames to fix plus the corrected video. Let me know if this works for you; to have the entire corrected video. Then you could encode it and splice bits of mpg together, replacing each range specified. Easiest option is to just re-encode it all, of course it might get blocky...

Note to self: value of thresh for this repair was 2.4-2.8.

[robjv1]

Quote:

Originally Posted by jmac698

Organization
Could you move post#32 on to a new thread, "Dropped/Duped Fields (Plays as Vertical Jitter)"

Good idea -- I suppose this is a separate issue. Maybe LS can move all of them so I don't start a new topic and start duplicating posts. Whatever is easier.

Also -- regarding the video clip itself, this particular video isn't one I have a need to correct, although it would be fun to see the before and after for this particular example. It was just intended as an example of the general problem that I find on other video footage on my tapes.

I'm not sure I understand how I'd apply the fix to a general video -- this is a representative example, but (I think) some videos do behave slightly differently in terms of how long it lasts, but I suppose it's possible that the same pattern is just repeating over and over as you identified. If I did run into a variation on the problem, would I need to edit the text values to handle it as well?

In general, I don't mind re-encoding the videos and storing them with the original until I can splice in the problem frames from the re-encode. It'll save me time in the long-run and I have lots of space. For me the idea is to use the re-encoded version only as an alternative version for fixes and not rely on it as the base version of the footage.

[jmac698]

Ok, for your purposes, I could encode the frame ranges as: no problems surround 15 frames around each problem. That would be the size of cutting an mpg anyhow. So if the problem was like this:
ok, bad, ok, bad, ok, ok, ok, ok, ok, bad...
It would say replace 9 frames. That way, it doesn't matter if the patterns are quite different.
The pattern doesn't affect the repair but, if there's a full frame drop I'd like to test it first.
So I'll prepare a fixed version for you with range text file and see how that works for you.
Check another 12 hours though, my day is done here.

Meanwhile if you find another sample just to ensure we're seeing the full variety here. Especially something that seems any different to you.,

Another problem, I assumed it was only movies that were 25p (encoded as 50i), could there be some tapes with 50i video too?

[robjv1]

Sounds good! I'll see if I can find some other examples -- I have a file of a bunch of them somewhere. I'll have some time to work through some sources I've had problems previously over the next month or so, so I'll have lots of opportunity to test it out.

By the way, as far as the footage I would use this with -- good question! For the most part it's actually not movies or film sources, although there are some of course. Most of the sources I'd be using are actually mostly television shows taped on video and not film and also some sports, which I assume are also taped on video.

[jmac698]

I'll have to use a different repair method, it's a bit worse. Do you normally keep your encodes as fully interlaced? I should have a video sample for testing.

[robjv1]

Good to know, glad you asked, I didn't even think to mention it.

Yes, I leave the interlacing alone, as my destination is typically DVD-R for playback on a standard DVD player. The sample tape was from a cable TV recording from about 1990, playback on an NTSC JVC SR-W5U deck fed directly into my JVC DR-M100S DVD recorder (top frame first). I just did a straight rip from the DVD recorder and then edited it together an MPEG from the VOB without re-encoding it -- that's the typical process I use for just about everything. I'll see about getting another clip tomorrow for you.

-- merged --

Here are a couple of more TBC on examples for ya -- from a basketball game. I doubt it's of any consequence , but just as a note, these are from an EP source, were the movie was from an SP one.

Problems with about frames 72 - 78 on the first one and 95 to 101 on the second. (Interesting that both are about 6 frames worth). Not sure if these differ at all from the first ones -- although perhaps the 'variety' I'm seeing is just perceived and not actual.

I have a few tapes I can use to test this thoroughly that have a high number of these sorts of instances, so I'll give the algorithm a workout for sure!

[jmac698]

Good catch, those are different. 2nd clip is really weird, half the frame is ok the other half is a repeat of 1 frame back. It would need a special repair technique. Does the effect occur a lot?
Your clip1 is different as well. It's a missing field with no dupe. Detection is harder but repair is the same.
So these are all 3 different problems. I can't solve them all right away, but just try one and go from there. And give me a few days

Interesting though, things I never saw before... ultimately could find a way to fix all these types of problems.

[robjv1]

Quote:

Originally Posted by jmac698

Good catch, those are different. 2nd clip is really weird, half the frame is ok the other half is a repeat of 1 frame back. It would need a special repair technique. Does the effect occur a lot?
Your clip1 is different as well. It's a missing field with no dupe. Detection is harder but repair is the same.
So these are all 3 different problems. I can't solve them all right away, but just try one and go from there. And give me a few days

Interesting though, things I never saw before... ultimately could find a way to fix all these types of problems.

Absolutely! Take your time.

Yeah, after looking at it, I'd say the version in clip 2 occurs fairly often as well. I guess I generalized the problem to "jittery/jumpy frame" but I guess it's much more complex in terms of how it manifests itself specifically. Some are more noticeable than others. I'll see if I can pull out another tape that displays some more -- hopefully there is a small set of different errors we're working with here! If it can be made into a workable solution it would pretty much eliminate the one major shortcoming of the TBC correction in these SVHS decks, which is pretty astonishing to think about in 2011!

[lordsmurf]

I'm still looking forward to seeing what you were able to do with the messed-up clips I gave you.

[admin]

Alright jmac, this was created because of you: http://www.digitalfaq.com/forum/news...er-images.html
This wasn't some off-the-shelf mod.
I spent several hours this morning writing out new vBulletin code just for this.

It should come in useful for us advanced users.