#1  
06-24-2017, 06:10 AM
koberulz koberulz is offline
Premium Member
 
Join Date: Feb 2016
Location: Perth, Australia
Posts: 453
Thanked 3 Times in 2 Posts
When I look at the ColorTools histograms, everything is climbing the wall on the left. I can fix the RGB clipping by desaturating, but I have to go so far it looks awful. No matter what values I throw into the 'levels' line of the script, it clips low.

What's the solution here? Here's my script:
Code:
AVISource("Capture1.avi").Trim(36000,183000)
SmoothLevels(0,1,255,16,250)
ColorYUV(off_v=1, off_u=1, cont_u=-90, cont_v=-90)
MergeChroma(awarpsharp(depth=20))
ConvertToYUY2(interlaced=false)
FixVHSOversharp(40,12,8)
FixVHSOversharp(40,14,10)
FixVHSOversharp(40,14,10)
ConvertToYV12(interlaced=false)
FixChromaBleeding()
StabMod()
StabMod()
#Histogram(mode="levels")
LimitedSharpenFaster()
AddGrainC(1,1)
ConvertToRGB(interlaced=true)


Attached Files
File Type: avi Capture2_2.avi (3.18 MB, 3 downloads)
Reply With Quote
Someday, 12:01 PM
admin's Avatar
Ads / Sponsors
 
Join Date: ∞
Posts: 42
Thanks: ∞
Thanked 42 Times in 42 Posts
  #2  
06-24-2017, 06:36 AM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,307 Times in 982 Posts
Can't help with a script. Your sample isn't from the original YUV capture. It's processed uncompressed RGB, so anything YUY2 or YV12 looked like originally can't be revived.

The Hauppauge USB Live-2, just like the ATi 600 USB, clips at y=16. You'll almost always have a left-hand y=16 spike with those capture cards, especially if the source has a lot of darker superblacks.
Reply With Quote
  #3  
06-24-2017, 06:48 AM
koberulz koberulz is offline
Premium Member
 
Join Date: Feb 2016
Location: Perth, Australia
Posts: 453
Thanked 3 Times in 2 Posts
Oops, accidentally attached the output of the script rather than the input. Don't have time to find that frame in the original right now, however: even if the card clips at 16, SmoothLevels() should shift the spike at a certain value, right? I was able to wash out the entire thing and it still spiked at 16, rather than retaining the spike but shifting it rightward.
Reply With Quote
  #4  
06-24-2017, 07:43 AM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,307 Times in 982 Posts
Shifting the spike doesn't recover anything from clipped data, it just makes it brighter (more gray than black, looking washed out). For cards that clip superblacks you have to compensate during capture to bring some superblacks above y=16 (at the same time you'll likely have to control brights by tamping down contrast a bit). You can tweak later, which you usually do anyway. Once hard clipping occurs, there's no detail to recover.
Reply With Quote
  #5  
06-24-2017, 11:27 AM
koberulz koberulz is offline
Premium Member
 
Join Date: Feb 2016
Location: Perth, Australia
Posts: 453
Thanked 3 Times in 2 Posts
Yes, once the spike starts shifting all you can do is leave it at 16. But if it doesn't shift, something's gone wonky. AFAIK.

Attached what is hopefully the correct file this time.


Attached Files
File Type: avi Capture2_2.avi (365.5 KB, 8 downloads)
Reply With Quote
  #6  
06-24-2017, 04:36 PM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,307 Times in 982 Posts
The only clipping I see, and it's not really hard clipping, is data at y=16 due to the original black borders. You should crop off borders and head switching noise or be aware of them when using histograms and whatnot. Dark borders and that sort of thing will always peak at the left.

Below, the YUV histogram of your sample. The histogram with with borders (left) -vs- borders cropped (right):



The colors aren't too great in this dubbed tape, and the detail is prety7 bad (as if don't already know this), but the saturation histogram shows low saturation in some areas. Red hardly shows up here except as browns in the image, yellow is a bit hot. Needs a little red in the Mitsubishi logo -- at least I think so, but I could be wrong.



Personally I think the levels look OK. There's no lighting on the crowd itself, so you're not going to get a lot of bright data out of those areas. But if you insist, I used a ContrastMask.avsi filter to bring up the darks a bit (I used a setting of 5.0, which i think is a little high and looks foggy). I used Donald Graft's hue-saturation-intensity Virtualdub plugin to increase red saturation and subdue the hot yellow a bit, and used ColorMill to tame the reddish color balance in the darks (a .vcf file with settings is attached). You can do the same thing in Premiere with its saturation and other color filters.

I think we mentioned ContrastMask.avsi previously, but if not it is attached. The avsi requires VariableBlur, also attached.

I used this script:
Code:
ContrastMask(enhance=5.0)
SmoothLevels(16,0.90,255,16,255)
SmoothTweak(contrast=1.1)
Crop(8,0,-12,-8).AddBorders(10,4,10,4)
ConvertToRGB32(interlaced=false)      #<- for Virtualduib color tweak.
The resulting Capture2_2_Levels_Colors.avi is Lagarith YV12, and is attached.

original:


after:


Attached Images
File Type: png with boders -vs- borders cropped.png (16.0 KB, 36 downloads)
File Type: png original - color2 YUV.png (28.3 KB, 36 downloads)
File Type: jpg original.jpg (81.0 KB, 36 downloads)
File Type: jpg contrast mask.jpg (98.2 KB, 36 downloads)
Attached Files
File Type: vcf capture2_2.vcf (1.1 KB, 0 downloads)
File Type: avsi ContrastMask.avsi (919 Bytes, 1 downloads)
File Type: zip VariableBlur0_4_0.zip (37.6 KB, 0 downloads)
File Type: avi Capture2_2_Levels_Colors.avi (442.0 KB, 0 downloads)
Reply With Quote
  #7  
06-25-2017, 12:03 AM
koberulz koberulz is offline
Premium Member
 
Join Date: Feb 2016
Location: Perth, Australia
Posts: 453
Thanked 3 Times in 2 Posts
This is what it looks like in ColorTools' histograms, though:
histograms.png

Everything spikes at either 0 or 16, with blue even flying all the way off the top of the graph.

EDIT: Here's what my script looks like:
You must be logged in to view this content; either login or register for the forum. The attached screen shots, before/after images, photos and graphics are created/posted for the benefit of site members. And you are invited to join our digital media community.


Reply With Quote
  #8  
06-25-2017, 12:16 AM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,307 Times in 982 Posts
Then use yours. Are you familiar with curves filters? I didn't use one to tweak further at the extremes for the demo, but thought you might think of that since you paid all that money to Adobe to get those fancy features. Your earlier processed sample referred to by your histogram looks washed out and has really bad posterization effects that are even worse than the original. But everyone has their viewing preferences.

[EDIT Oops, I leaned on the keyboard and logged myself off, LOL! Holy smokes.

Anyway, I don't envy you. The samples you're working with are some of the worst videos I've ever seen, and they present no good choices. Otherwise, viewing preferences aside, there isn't any "YRGB" clipping in the image content of the sources you presented. That should have been obvious from the start, because the dark areas didn't look like crushed blacks, they just look like really murky video. I don't really know what to advise in detail about these sources, so many aspects just can't be repaired to make them look like "real" representations of events. Just do the best you can, I guess. Sorry I left you with you with a rushed and unfinished sample fix in the earlier post. Getting too hectic around here.

Last edited by sanlyn; 06-25-2017 at 01:00 AM.
Reply With Quote
  #9  
06-25-2017, 12:58 AM
koberulz koberulz is offline
Premium Member
 
Join Date: Feb 2016
Location: Perth, Australia
Posts: 453
Thanked 3 Times in 2 Posts
Curves won't recover anything that's already been clipped, though...and what's causing it? With blue being higher at 0, does that mean blue is lacking overall? It's also the only color that hits 255, though...

Where are you seeing posterization?
Reply With Quote
  #10  
06-25-2017, 01:51 AM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,307 Times in 982 Posts
What you said about curves tells me that you don't appreciate its use. But you're right, it won't recover clipped data. But it can create a gradual, imperceptible, and precisely targeted falloff or rise in discreet or overall color and brightness ranges.

If you have zero pixels of certain colors in certain parts of a histogram, it means there are zero or close to zero pixels of that color value in that part of the image. Normally it means "very few", because you can't create new pixel values from nothing. What you can do is take a value range and spread it out or narrow its range (which can cause hard gradient edges and posterizing, blocky effects). Curves filters easily qualify for that use.

Sorry you don't see the posterizing loss of detail, which is in the original and only looks worse the more it's filtered and worked on. Posterization effects are due to fine detail removal and merging/unwanted blending of adjacent detail or color areas that should be distinct and well defined. There's purposeful, heavy posterizing such as line art and poster art, but in video it's an unwanted effect unless you're doing it on purpose. In this case it's by mistake or carelessness, with tape dupes made on lord knows what low level of equipment and abused in storage and playback by its owners. Sometimes it can be partially corrected by blurring and resharpening, and adding dithered, ordered noise to simulate "detail". But it can never be truly restored, especially with samples that are already murky. An example is look at faces and contours, where it appears that people are wearing hosiery over their heads, giving the clay-face effect or carved-from-soap appearance. Look at the carved-in-plastic appearance of clothing contours and textures.

None of this damage is your doing, of course. I had a few tapes that I stupidly stored in the trunk of a car for several years. When I finally played them, the choice was clear: they were irreparable, so they were discarded. Fortunately there were no important recorded memories that didn't exist elsewhere. But that choice isn't always available.
Reply With Quote
  #11  
06-25-2017, 02:05 AM
koberulz koberulz is offline
Premium Member
 
Join Date: Feb 2016
Location: Perth, Australia
Posts: 453
Thanked 3 Times in 2 Posts
Quote:
Originally Posted by sanlyn View Post
What you said about curves tells me that you don't appreciate its use. But you're right, it won't recover clipped data. But it can create a gradual, imperceptible, and precisely targeted falloff or rise in discreet or overall color and brightness ranges.
But it's better to correct in YUV, if possible, before clipping.

My comment about blue being higher was referring to the histogram, not to using the curves filter. Not sure if that's what you thought I meant.

Quote:
If you have zero pixels of certain colors in certain parts of a histogram, it means there are zero or close to zero pixels of that color value in that part of the image.
But if I'm reading the histogram correctly, there are a lot of pixels with no blue in them, causing the spike on the left. Correct?

I'm just not really sure how RGB shifts around on a histogram when adjusting UV contrast and offset, so I'm not sure how to tackle it.

Quote:
Sorry you don't see the posterizing loss of detail, which is in the original and only looks worse the more it's filtered and worked on.
Can you point out some examples of it? Could it be caused by excessive denoising? The original was pretty noisy, what I've posted here was post-QTGMC as I'm just worried about colors. It did seem to posterize a little, so I backed off...might not have been enough though.

I'm working with the only known footage for all of these. Gotta do the best I can.
Reply With Quote
  #12  
06-25-2017, 02:19 AM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,307 Times in 982 Posts
I think thiu was covered many times about histograms. A lot of data in a histogram means a lot of pixels with bright or dark values in those areas. Less data in a histogram means less pixel values in those areas. Hard clipping at the dark or bright end is usually a tall, thin spike that contains a great many pixels all having the same values at a point where all former data was compressed into a single availalble value.

If you want to see examples of posterization, look at the videos and images posted in this thread. You see example on the internet, but they're usually extremes that you don't often see in real video.

Yes,it was obvious that your samples had already gone through some processing, which almost made me bypass them and ask for a real original. But two frames isn't really much to work with it, is it? With temporal filters like QTGMC, two frames isn't enough to process. Anyway, this was about clipped blacks, but there weren't any except for the black borders.
Reply With Quote
  #13  
06-25-2017, 02:37 AM
koberulz koberulz is offline
Premium Member
 
Join Date: Feb 2016
Location: Perth, Australia
Posts: 453
Thanked 3 Times in 2 Posts
Quote:
Originally Posted by sanlyn View Post
I think thiu was covered many times about histograms. A lot of data in a histogram means a lot of pixels with bright or dark values in those areas. Less data in a histogram means less pixel values in those areas. Hard clipping at the dark or bright end is usually a tall, thin spike that contains a great many pixels all having the same values at a point where all former data was compressed into a single availalble value.
In luma, yes. Levels manipulates the luma in the exact terms a histogram expresses it.

In chroma...what is 'dark' and 'light'? Saturation levels, or something else? How are these affected by ColorYUV adjustments?

Quote:
If you want to see examples of posterization, look at the videos and images posted in this thread. You see example on the internet, but they're usually extremes that you don't often see in real video.
Right, I meant specific spots on the frame in question where it's evident (or most evident). I can see the frame grabs, obviously. But it's not jumping out, and I'm not quite sure what I'm looking for because as you say, it's not going to be as blatant as if done intentionally for artistic purposes.

Quote:
Yes,it was obvious that your samples had already gone through some processing, which almost made me bypass them and ask for a real original. But two frames isn't really much to work with it, is it? With temporal filters like QTGMC, two frames isn't enough to process. Anyway, this was about clipped blacks, but there weren't any except for the black borders.
As I said, colors were my main concern and the histogram may have behaved differently on other frames, so it made most sense to go with a super-short sample. QTGMC was run on the whole thing, which is around an hour.
Reply With Quote
  #14  
06-25-2017, 03:33 AM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,307 Times in 982 Posts
Quote:
Originally Posted by koberulz View Post
My comment about blue being higher was referring to the histogram, not to using the curves filter. Not sure if that's what you thought I meant.
High spikes in a histogrm mean many pixels having that valuie. Depressions mean fewer pixels having that value. If you have one or more color climbing the walls at eithe extreme, you can use individual curves to crate a gradual rolloff in that particular area. Try it in YUV, where it's not possible to target very small, discrete areas for change.

Quote:
Originally Posted by koberulz View Post
But if I'm reading the histogram correctly, there are a lot of pixels with no blue in them, causing the spike on the left. Correct?
If there's a blue spike on the left, that means a lot of blue pixels with dark blue color values.

Quote:
Originally Posted by koberulz View Post
I'm just not really sure how RGB shifts around on a histogram when adjusting UV contrast and offset, so I'm not sure how to tackle it.
If you want to see how changes with ColorYUV, Tweak, etc., are reflected in RGB, mount a ColorTools histogram in VirtualDub while you make changes in a script, then hit F2 or click "Reopen video file" to repaint the frame you're looking at.

Yes, posterizing is caused by excessive or improper denoisers. But with tape dupes, posterizing comes with the original. That's why tape dupes are so godawful, they need cleaning but the more you clean the more denuded they look.

Can you see that people in the samples look as if they're wearing sheer stockings over their heads and have missing/smeared features in their face and clothing? Cartoon-like contours are also a sign of posterizing, along with such effects as objects blurred to the point of being unrecognizable because so much detail has been blurred and blended. Posterization also takes the form of hard-edged gradients, as in banding effects where there should be smooth gradations of color from one shade to the next. When these blurred and softened blobs are sharpened during tape dub playback, you get some very strange "sharpened blurs" as it were, which adds no real detail whatsoever.

Quote:
Originally Posted by koberulz View Post
In chroma...what is 'dark' and 'light'? Saturation levels, or something else? How are these affected by ColorYUV adjustments?
Dark and light in chroma refer to shades of a hue. I suppose you know the difference in brightness value (shade) between a bright blue sky and a dark navy blue automobile. The hue is blue, the shade is dark or light. Saturation has to do with the amount of white and of light reflection in a color. Unfortunately a parade histogram doesn't measure saturation. You nead a vectorscope. Low saturation means that some portion of white (amounts of the other two colors) have been added to the basic hue. Low-saturation colors tend to have a brighter or even foggy (grayed) look. With deeper saturation, whiteness is removed and you have more of the pure color and fewer traces of the the other two. So a histogram with a high horizontal level of peaks and a band that sits high in the saddle, so to speak, is more thinly saturated overall and is reflecting more white. A histogram with a more low-rising horizontal band has deeper saturation and is refelecting less white. (And remember that whites, along with grays and blacks, are a mix of equal proportions of all three colors. If you take one color and keep adding more and more equal portions of the other two colors, you'll achieve a totally unsaturated color that has finally become white).

Mount a frame in VirtualDub, then load a ColorMill filter followed by a ColorTools histogram at the bottom of the filter chain. When you change saturation in ColorMill, watch what happens to the histogram in the preview window. Do the same with gamma, hue, middle point, and other settings.

The amount of whiteness is often tied in with contrast values, which many also like to refer to as "saturation", but that's not really the case. The effects of high and low contrast can make a color appear to be more or less prominent, respectively. Technically a color with extremes of contrast at the low and/or bright end isn't "saturated', it's just over extended. Contrast is a brightness value range, not saturation. To say that lowering contrast of a color channel with ColorYUV will lower saturation is almost correct, because of the way ColorYUV's contrast filter works: it tends to shrink pixel values toward the middle from each end, or to exted valuies from the middle outward in both directions, in effect giving the opposing channel more or less prominence, which really means adding more white effect to the overall picture, which in term means lowering a color's saturation. But not all contrast filters work that way. SmoothTweak's contrast filter extends or shrinks mostly the bright end, with less effect on the lower mids and almost none on the darks. So SmoothTweaks' contrast works the way most contrast filters are supposed to work, and ColorYUV's contrast filter is a hybrid.

But people don't normally use such precise terms unless they're colorists who get far more technical. We mere hobbyists tend to speak in more generic terms.

...oh my, this post took a while because of all the typos. Must be getting sleepy.
Reply With Quote
  #15  
06-25-2017, 06:14 PM
msgohan msgohan is offline
Free Member
 
Join Date: Feb 2011
Location: Vancouver, Canada
Posts: 1,323
Thanked 334 Times in 276 Posts
Quote:
Originally Posted by koberulz View Post
This is what it looks like in ColorTools' histograms, though:
Attachment 7657

Everything spikes at either 0 or 16, with blue even flying all the way off the top of the graph.
I don't have this filter, but those are the results after cropping all borders?
Reply With Quote
  #16  
06-25-2017, 08:12 PM
sanlyn sanlyn is offline
Premium Member
 
Join Date: Aug 2009
Location: N. Carolina and NY, USA
Posts: 3,648
Thanked 1,307 Times in 982 Posts
I get the same histogram. It includes black borders, no head switching noise.
Reply With Quote
Reply




Similar Threads
Thread Thread Starter Forum Replies Last Post
ATI 600 USB audio clipping? someperson Capture, Record, Transfer 2 08-04-2016 11:07 PM
Fixing clipping audio in MP4 video? summergleason Restore, Filter, Improve Quality 5 12-04-2013 04:32 PM
How to adjust very loud audio clipping during capture? moxiecat Capture, Record, Transfer 6 03-07-2012 05:37 AM
My Other High-End VCR, the MITSUBISHI HS-U70 (with audio output clipping problem) Tasuke Videography: Cameras, TVs and Players 0 05-13-2011 01:56 PM

Thread Tools



 
All times are GMT -5. The time now is 09:38 AM