Quantcast Averaging multiple captures? (test samples included) - digitalFAQ Forum
Go Back    Forum > Digital Video > Video Project Help > Restore, Filter, Improve Quality

Reply
 
LinkBack Thread Tools
  #1  
12-27-2010, 10:03 AM
juhok juhok is offline
Free Member
 
Join Date: Sep 2009
Posts: 379
Thanked 105 Times in 87 Posts
Inspired by jmac698's VHS averaging I made some averaging tests with BetacamSP material. I have many SP decks with worn heads which provide good samples. This tape/deck combo played out pretty good and I didn't have the huge amount of dropouts I hoped for, but there's something. And it's also interesting to see if it helps with noise.

Tools: Avisynth, Average (newer versions towards the end of the thread).

Samples (ffdshow huffyuv codec):
Capture 1 - straight from deck
3 cap average
Difference between the above (Avisynth, Subtract)

As anticipated, there's a bit less noise. Nothing spectacular. I guess the format's pretty noise free already. A bit of softening can be seen. What I didn't expect is that horizontal line 1/3 down from the top which is visible in the subtracted video. Looks like a mild physical damage in the tape, maybe.
Reply With Quote
Someday, 12:01 PM
admin's Avatar
Ads / Sponsors
 
Join Date: ∞
Posts: 42
Thanks: ∞
Thanked 42 Times in 42 Posts
  #2  
12-27-2010, 02:42 PM
admin's Avatar
admin admin is offline
Site Staff | Web Development
 
Join Date: Jul 2003
Posts: 4,333
Thanked 635 Times in 452 Posts
That source is really clean -- I see nothing wrong in either version.
Maybe I blinked?

I'm also not entirely sure I understand what this filter is supposed to be doing here. Is this an automatic method to blend several captures, and take the best version from each? The usage notes in the thread mean nothing to me -- no idea what "alpha" and "gamma", etc is supposed to mean in plain words. My problem with Avisynth is those who write instructions for it often assume you already know lots of other things.

Avisynth plugin filter "Average" attached below...


Attached Files
File Type: rar Average_21Oct05.rar (31.2 KB, 19 downloads)

- Did this site help you? Then upgrade to Premium Member and show your support!
- Also: Like Us on Facebook for special DVD/Blu-ray news and deals!
Reply With Quote
  #3  
12-27-2010, 03:30 PM
juhok juhok is offline
Free Member
 
Join Date: Sep 2009
Posts: 379
Thanked 105 Times in 87 Posts
The source is almost perfect. It's a bad example in this regard but I didn't have patience to dig another 30kg machine for redoing it. I'll try it on some VHS tapes later.

Alfa, Beta etc. are your input. They could be called clip1, clip2, etc. Here's my script if it helps to understand the principle:

Code:
v1 = AVISource("E:\avetest\1.avi").trim(16, 1016)
v2 = AVISource("E:\avetest\2.avi").trim(24, 1024)
v3 = AVISource("E:\avetest\3.avi").trim(20, 1020)

Average(v1, 1.0/3, v2, 1.0/3, v3, 1.0/3)
If I were to have 4 input clips, it'd look like:
Code:
v1 = AVISource("E:\avetest\1.avi").trim(16, 1016)
v2 = AVISource("E:\avetest\2.avi").trim(24, 1024)
v3 = AVISource("E:\avetest\3.avi").trim(20, 1020)
v4 = AVISource("E:\avetest\4.avi").trim(20, 1020)

Average(v1, 0.25, v2, 0.25, v3, 0.25, v4, 0.25)
So it's "input clip", then it's "weight", next input and so on. Input must be trimmed so that they match temporally on every frame.

I'm in a habit of "using, not thinking about how it works", but definition for weighted average is:
Quote:
If not mistaken the principle is that majority pixels rule. Odd errors get "voted out".

Sample of an error frame(number 27) included. I'd have missed this one in normal workflow. It makes me laugh a bit to think about the "sales talk" of offering this kinda service to people. I have very hard time in the present just presenting that quality hardware and postprocessing is worth the tiny difference in costs Vs some of my "competitors" who use VHS-combo decks and such.


Attached Images
File Type: jpg 1227_sample_subtract.jpg (13.0 KB, 21 downloads)
Reply With Quote
  #4  
12-28-2010, 04:41 PM
jmac698 jmac698 is offline
Free Member
 
Join Date: Dec 2010
Posts: 387
Thanked 73 Times in 56 Posts
I think I can explain this another way. Your original recording is x. The tape playback added some noise, n. So you have x+n in each capture. Let's say your original pixel was 10. Your noise is random, let's say -2,3,-1 each time you capture it. So let's just average them together:
10-2=8 capture 1
10+3=13 capture 2
10-1=9 capture 3
-------
(8+13+9)=30
30/3=10 average of 3 captures, in this case exactly the original.

This works because noise always averages to 0. If you added a huge amount of noise, with enough captures and averaging, it has to come out less noisy.

Neat story: The amazing thing is, Shannon's theorem says that you can always find the tiniest signal is the biggest noise. We've come very close to this with cellphones which perform close to the theoretical limits. Instead of averaging, we use Viterbi decoding - but that's another story

Another technique is called median. This is even simpler! Take the pixel values, sort them, and pick the middle of the list. That will also be close to the original - and better yet, it automatically throws out any extreme values like black or white lines.

Example: 8,13,9 -> sorted to 8,9,13 -> middle is 9.

I think this is superior because, in his capture above, one of the caps had a big line in the middle. Instead of includiing this line in the average, with median, it's completely ignored, because it will be like the '8' above, and thrown out as an extreme value.

Where does the blurring come from in averaging? I believe this is because of slight jitter left in the video. Here's why:

Let's say my original pixels are 0,50,90 ... this is a sharp edge going from black to white. Now lets say in capture 2 it's shifted 1 pixel to the left: 50,90.... now look what happens when I average the two captures: (0+50)/2,(50+90)/2,(90+?)/2 = 25,70.... my edge isn't so sharp anymore! It also starts sooner. So the 'energy' is being spread out over more horizontal area.

Median avoids this problem as well, it will be less blurry.

I'd like to see the original 3 captures for download.
Reply With Quote
  #5  
12-28-2010, 05:00 PM
jmac698 jmac698 is offline
Free Member
 
Join Date: Dec 2010
Posts: 387
Thanked 73 Times in 56 Posts
Basics of scripting: If you learn just these two commands, it's all you need to use a lot of plugins.
v1 = AVISource("E:\avetest\1.avi").trim(16, 1016)
AVISource opens a video file. We're going to refer to that file by some name, so we'll call it v1 (for video 1). The period is just a shortcut to put another command on the same line. Trim is an video edit command. It means to keep (inclusively) frames 16 through 1016. The use here is to edit all 3 videos so that they start on the same frame. They start on different frames depending on the moment you clicked record in your capture software.
So we end up with 3 videos, edited and named, v1-v3. The next command comes from the average plugin. You will find the details in it's manual. It takes two or more videos and averages them.
Average(v1, 1.0/3, v2, 1.0/3, v3, 1.0/3)
Is like saying: v1/3+v2/3+v3/3 which is mathematically the same as: (v1+v2+v3)/3, which is an average.

There is no = sign here... that's because there is always a 'floating' value called last which sticks around after each line. What is really happening is: last=Average(v1, 1.0/3, v2, 1.0/3, v3, 1.0/3) and when I end the script suddenly, the result returned is the value of last, which is the averaged video.

Now I can explain the other way of writing this. Instead of using period to put things on one line, I'm going to use last and put things on two lines:
AVISource("E:\avetest\1.avi")
trim(16, 1016)
v1 = last
After the first line, last is my (unnamed) video. Then last gets edited. Finally, I assign v1 to last and it's now the equivalent of the all in one version.

Only one more thing I can mention here. There's limits to using 8 bit video. Let's say I had 3 pixels of 100 each and I want to average them. If I do it this way: (100+100+100)/3, look what we get: 300/3. But you can't store 300 in a video, it only goes up to 255 (and 235 is supposed to be white). That's why average does it a different way: 100/3+100/3+100/3=33+33+33=99. Notice we got a roundoff error, because we can't store .333 or 1/3 as part of a pixel either.
It would be possible to write a better average plugin but it would have to deal with fractional arithmetic and do things like reduce fractions, use GCD, and so on.

Neat story: Avisynth *does* use fractional arithmetic in one case, for storing the framerate of a video. It has to! There's annoying problems that occur when 29.970... is not the same as 29.97003... anyhow, the exact fraction is 30000/1001. This is the only way to compare framerates that are exactly the same, you need it to join two videos together, or else you'd get a "frame rates don't match" error.
Reply With Quote
  #6  
12-28-2010, 06:30 PM
juhok juhok is offline
Free Member
 
Join Date: Sep 2009
Posts: 379
Thanked 105 Times in 87 Posts
Quote:
Originally Posted by jmac698 View Post
I'd like to see the original 3 captures for download.
I don't remember the cut points and don't feel spending time to find them, so here's completely new set of caps:

http://www.siluriformes.net/mm/1229_cap1.avi
http://www.siluriformes.net/mm/1229_cap2.avi
http://www.siluriformes.net/mm/1229_cap3.avi
Reply With Quote
  #7  
12-29-2010, 04:24 AM
jmac698 jmac698 is offline
Free Member
 
Join Date: Dec 2010
Posts: 387
Thanked 73 Times in 56 Posts
cap1 and cap2 seem to be exactly the same.
Very good sample, but cap3 has one moment of noise.
There's some chroma noise.
Reply With Quote
  #8  
12-29-2010, 04:28 AM
juhok juhok is offline
Free Member
 
Join Date: Sep 2009
Posts: 379
Thanked 105 Times in 87 Posts
Oh, then I've saved the same file twice. Maybe new batch of files later again
Reply With Quote
  #9  
01-13-2011, 11:06 AM
magillagorilla magillagorilla is offline
Free Member
 
Join Date: Jan 2011
Posts: 17
Thanked 0 Times in 0 Posts
I haven't tried this so I'm operating on the theory only. First off, isn't a pain to try and synch the frames?

My thoughts are this:
If I have 3 versions of a video, each have a bad pixel (this is our noise) in a different place on the same frame. When I average the frames together haven't I effectively created a composite with 3 incorrect pixels where any of the individual frams only had 1. Shure I have diluted the 3 error pixels, but did 1frame's error make another frame's non-error 33% incorrect?

This method seems odd to me. It's like adding noise to waterdown other noise isn't it?

It would seem more effective if you could use a regular noise filter to hihglight what is considers noise and use this as a mask. You could then use the mask to replace the detected noise using pixels from another clip. Replacing rather than blending could correct pixels 100%.

.... I'm sure someone already tried this. I'm pretty late to the video restoration party.
Reply With Quote
  #10  
01-13-2011, 05:37 PM
jmac698 jmac698 is offline
Free Member
 
Join Date: Dec 2010
Posts: 387
Thanked 73 Times in 56 Posts
Good point! And you are right. That's why I use median. If a dropout is in one of the videos, you get pixels like this: 45, 52, 200 and median will take the middle value, 52. It works very well!
Random noise is different. It is always there and everywhere on the frame, so average will work in that case.
I have recently wrote a script to simulate a bad video tape, complete with noise, jitter, and dropouts, and I can simulate 3 captures and show that it works too.

As for syncing frames, theoretically you just trim the start of each capture until they start on the same frame. In reality I found VirtudalDub drops frames by default and shows no indication of it. Then I did tests with every setting until I found one that works. To prove I fixed it I had to write a script to generate my own timecode and then automatically analyze that for missing frames. I'm also working on a way to automatically detect missing frames. There's a 3rd problem, where I don't have a TBC so the videos don't merge well, but I've almost got that solved too.
Reply With Quote
  #11  
03-01-2011, 12:03 PM
Bugs.Bunny Bugs.Bunny is offline
Free Member
 
Join Date: Feb 2011
Posts: 13
Thanked 5 Times in 4 Posts
Hi jmac698, what avisynth filter do you use (instead of average) to apply median to a number of input clips?

Link to the average filter (those who want to try the average version)
http://forum.doom9.org/showthread.ph...19#post1129919

Got a reply at doom9 - for this type there is no filter but rather avisynth scripts. If anyone's interested look here:
http://forum.doom9.org/showthread.ph...90#post1169990
http://forum.doom9.org/showthread.ph...76#post1331776

Tried it a bit with the 3 captures version - seems to give very good results.
Reply With Quote
  #12  
03-01-2011, 09:20 PM
jmac698 jmac698 is offline
Free Member
 
Join Date: Dec 2010
Posts: 387
Thanked 73 Times in 56 Posts
Code:
Function Median1(clip input_1, clip input_2, clip input_3, string "chroma")
{# median of 3 clips from Helpers.avs by G-force

chroma = Default(chroma,"process") #default is "process". Alternates: "copy first" or "copy second"

Interleave(input_1,input_2,input_3)
chroma == "process" ? Clense(reduceflicker=false) : Clense(reduceflicker=false,grey=true)
SelectEvery(3,1)

chroma == "copy first" ? last.MergeChroma(input_1) : chroma == "copy second" ? last.MergeChroma(input_2) : last

Return(last)
}
Clense is a temporal median, so we interleave the clips to do a spatial median. It's from RemoveGrain I think. Another way to do this is with an inline sort, min(max(a,b),max(b,c)) I think
Reply With Quote
The following users thank jmac698 for this useful post: lordsmurf (03-04-2011)
  #13  
03-04-2011, 05:32 PM
lordsmurf's Avatar
lordsmurf lordsmurf is offline
Site Staff | Video
 
Join Date: Dec 2002
Posts: 11,922
Thanked 2,170 Times in 1,870 Posts
I'd like to see a median filter that works on interlaced video without messing it up.

Or even something that can average two interlaced sources.

Or let's go one further -- averaging a PAL and an NTSC, to take the best of each. I know this is a severe issue among fan restorers of rare movies, which often involves worldwide sources. The Superman II and Superman IV movies come to mind.

- Did my advice help you? Then become a Premium Member and support this site.
- For sale in the marketplace: TBCs, workflows, capture cards, VCRs
Reply With Quote
  #14  
03-05-2011, 08:12 AM
Bugs.Bunny Bugs.Bunny is offline
Free Member
 
Join Date: Feb 2011
Posts: 13
Thanked 5 Times in 4 Posts
Quote:
Originally Posted by lordsmurf View Post
I'd like to see a median filter that works on interlaced video without messing it up.

Or even something that can average two interlaced sources.
I've recorded in YUV2 where the vertical chroma resolution is the same as the luma resolution. I had no problems averaging / median my interlaced sources.

I'm not 100% sure but shouldn't the average filter / median script work on a per pixel basis?

What I would be interested in, is a median script / filter for an even number of sources.
http://en.wikipedia.org/wiki/Median
Because it would perform an average of the two "middle" values.
The result would be a combination of median and average.
The median eliminates spikes and the likes that are only present in one or two of the sources while the averaging afterwards reduces temporary playback noise. Maybe Fizick can adapt the average filter he posted?
Reply With Quote
  #15  
03-05-2011, 08:56 AM
admin's Avatar
admin admin is offline
Site Staff | Web Development
 
Join Date: Jul 2003
Posts: 4,333
Thanked 635 Times in 452 Posts
Median chews up the interlacing, both VirtualDub and Avisynth versions.
I think I tried Avidemux, too -- same thing (and likely the same filter from Vdub or Avs anyway).

- Did this site help you? Then upgrade to Premium Member and show your support!
- Also: Like Us on Facebook for special DVD/Blu-ray news and deals!
Reply With Quote
  #16  
03-05-2011, 09:03 AM
juhok juhok is offline
Free Member
 
Join Date: Sep 2009
Posts: 379
Thanked 105 Times in 87 Posts
How about

Code:
SeparateFields()
Your average filter here
Weave()
Reply With Quote
  #17  
03-05-2011, 09:13 AM
admin's Avatar
admin admin is offline
Site Staff | Web Development
 
Join Date: Jul 2003
Posts: 4,333
Thanked 635 Times in 452 Posts
First thing I tried. No effect. Same mess.

I've been saving all the clips for a future guide, post or editorial (something that appears on the new not-yet-live site). Just backed up with projects at the moment, can't breathe long enough to write it up -- it's long and ugly.

With 3-4 people now interested, I'll have to carve out time next week for it.

- Did this site help you? Then upgrade to Premium Member and show your support!
- Also: Like Us on Facebook for special DVD/Blu-ray news and deals!
Reply With Quote
  #18  
03-05-2011, 09:46 AM
Bugs.Bunny Bugs.Bunny is offline
Free Member
 
Join Date: Feb 2011
Posts: 13
Thanked 5 Times in 4 Posts
Quote:
Originally Posted by jmac698 View Post
Another way to do this is with an inline sort, min(max(a,b),max(b,c)) I think
It probably has to be something like that:
min(min(max(a,b),max(b,c)),max(a,c))

@ admin: does the average filter work for you, or does it mess up as well?

With the median filters one has to take care a lot. When I was searching for median filters, these are build to do a median on a single frame, but not a median of three frames that come from three different captures of the same video. At least that's what I think.
I did not find a median filter that can process three or more sources. Thanks to forum members I was pointed to scripts that should do the trick.
I'm not very into avisynth scripts. Maybe the script is not perfect? Maybe it could be enhanced. Or someone could make a dedicated filter for it.
At least my idea (median+average with 4 sources) does not seem very complex. I do not know anything about programming an avisynth filter, but code one like this should be rather easy.
Reply With Quote
  #19  
03-05-2011, 10:50 AM
jmac698 jmac698 is offline
Free Member
 
Join Date: Dec 2010
Posts: 387
Thanked 73 Times in 56 Posts
What you're saying doesn't make sense. I can see a spatial median on each frame causing problems, but the median we are talking about is completely different. Show me the exact script that screwed up your interlacing.
As far as averaging ntsc and pal, that's easy:
Code:
ntsc=avisource("...").telecide.decimate.assumefps(25)
pal=avisource("..")
average(pal,.5,ntsc,.5)
Of course it depends how your sources were converted, if there's been film->pal->ntsc blending, you would need srestore.

bugs: I have a script for median5.
Reply With Quote
  #20  
03-05-2011, 10:56 AM
admin's Avatar
admin admin is offline
Site Staff | Web Development
 
Join Date: Jul 2003
Posts: 4,333
Thanked 635 Times in 452 Posts
Quote:
Originally Posted by jmac698 View Post
spatial median
This is what I was doing. Maybe I'm off-topic for this conversation?

- Did this site help you? Then upgrade to Premium Member and show your support!
- Also: Like Us on Facebook for special DVD/Blu-ray news and deals!
Reply With Quote
Reply




Similar Threads
Thread Thread Starter Forum Replies Last Post
Evaluate my setup and test captures? kooz Capture, Record, Transfer 9 03-21-2017 10:51 AM
Is DVD Decryptor included in GSpot? (or Imgburn?) rocko General Discussion 7 07-12-2015 04:43 PM
Best way to merge segments from multiple captures? premiumcapture Restore, Filter, Improve Quality 6 05-05-2015 12:03 PM
Test captures of VHS tapes with JVC HR-S9500 knumag Capture, Record, Transfer 17 07-20-2014 10:00 PM
Averaging multiple VHS captures technique JasonCA Capture, Record, Transfer 9 12-31-2012 11:35 AM

Thread Tools



 
All times are GMT -5. The time now is 02:37 AM