Will do. Tomorrow a.m. after I download you AVSMeter posted and other materials.

Wow, I forgot about that! Will include source code from now on. My bad.

Done. ZIP attached with 4 logs:

Intro_Script1.log
Intro_AVISource_x10.log
First_Script.log
Second_Script.log

I just remembered, the only background app running was the usual antivirus (Kaspersky Antivirus 2016) , and CPU was checked at 0% at the start of each run.

Not entirely sure what you were after with power saving - I already had both of those settings set to 'never', but I was on the 'balanced' profile. Set it to 'high performance' and ran all four scripts.

No, I was just saying that because I got the 0.04fps encoding issue with the Studio clip on your first script, I never went through your second script in detail.

What's the difference between them? I had a look on the Wiki and it seems the only different parameter is 'dither', which only exists in Tweak. 'Smooth' and 'interp' seem to do roughly the same thing.

Neither do I.

If I drag the input high down, things get lighter and the wave form moves up. If I drag the input high up, things get darker and the wave form moves down.

If I drag the output high down, things get darker and the wave form moves down. If I drag the output high up, things get lighter and the wave form moves up.

I'm just not sure what the difference is. Doing it in VDub, everything has a max of 255 and a min of 0, so in cases where things need to be higher or lower than those on the input I use the output in the opposite direction.

In AviSynth, there doesn't appear to be a minimum or maximum, so I can set input_low to -30 and input_high to 280 and it's all fine...although I'm not sure how much I'm changing that I shouldn't just be doing with gamma anyway.

The 'how do I figure out how to use it?' was in reference to RemoveSpots.

While we're on the subject, the color wheel in the TweakColor documentation doesn't match the color wheel on the Tweak Wiki page. But using Tweak's color wheel with TweakColor seemed to get better results than using TweakColor's color wheel with TweakColor. Am I going crazy or is the supplied color wheel wrong?

Thanks for the logs, both.

Choose "change plan settings" next to "High Performance", then "Change advanced power settings", set Min & Max Processor State to 100% and the cooling policy to Active. Don't know whether this will help, but it's worth a try.

Min and Max Processor State are already 100% (minimum was 5 when I had it under 'balanced'), I don't see a cooling policy option.

Ah, okay. I did a quick search and it seemed as though Min was always 5 on default plans. :o

I did see one post in a WoW thread that said a Hyper-Threaded quad-core would show 12% CPU usage for single-threaded apps (100% / 8 logical cores = 12.5%). But that was really the only search result I could find referring to this. Changing it to 13% does find a few more, such as this. Meanwhile, the number of results discussing why quad-cores get "stuck" at 25% CPU usage is innumerable. I would have expected it to be easy to find results talking about the 12% cap if it's common to 8-core systems.

But let's go back and compare your AVSMeter results to your VirtualDub tests.

Balanced
High Performance
Well within the margin of error here, I'm sure. There's no real difference between those two power plans with your system while running scripts. Feel free to use Balanced, after all. Sorry for that little diversion. (I can't use it on the system I'm typing to you from, personally: whether due to this AMD CPU's own power saving settings, the mobo, or something else, allowing it to throttle down the clock speed slows down my entire system. It isn't smart enough to kick the speed back up when I actually need it. Like yours though, my Intel build performs perfectly as expected on Balanced.)

But the real point here:
50 seconds is a lot shorter than 90-94. You can double-check that VDub can also do ~50 sec by using File -> Run video analysis pass. You'll have to actually watch the window the entire time, as it closes once completed.

I believe your ~90-sec result is bottlenecked by HDD I/O. If you're reading to the same drive as you're writing to, it needs to thrash back and forth the entire time when both the input & output are a lossless file. Have you tried reading from internal and writing to external?

sanlyn's results:

94 seconds again.

Screenshots of Perf and Log tabs of that VDub window may indicate something.

BTW, I think you all won. I had to install Lagarith.
See the messy ordeal post here: http://www.digitalfaq.com/forum/vide...ed-messed.html
Huffyuv 32-bit in 64-bit ffdshow is borked for PAL.

I need to run some speed tests. I may switch from Huffyuv to Lagarith for all encoding. Not sure yet.

So, Lagarith test files are now fine for me. :o

This might be relevant, but it just ran in 47 seconds.

Yeah, the top summary figures are an estimate from eyeballing VirtualDub's output window which disappears just as it makes its last update. I'm surprised my estimate was that close.

Well, there's a slowdown in koberulz' system somewhere. For some info on what I'm using most of the time, the PC I'm on right now has 4 SATA hard drives in IDE mode. The C: drive is a 250GB that has only the OS and installed program files, except for Avisynth, VirtualDub, and freebie standalone utilities that are on the 320GB Drive D: (which is also used for storage, pictures, documents, saved forum posts, plugin downloads, Wiki pages, web articles, PDF's etc., periodically backed up on an external USB drive). Drives E: and F: are 500GB each for intermediate working files, scripts for SD and HD work, and processing. I ran AVSmeter on Drive E:. A DVD reader and a BluRay/DVD burner are mapped as Drives G: and H:. USB and a.c. powered external drives are for additional storage and backups only, no processing. All original lossless captures underway and archived are on external USB's -- I pull off portions of them as needed for processing.

Kaspersky Antivirus is a resource hog that slows things down tortuously when it does auto hard drive scans, so I turned scanning off and do it manually overnight. Windows Automatic updates has been turned off for years (big deal, there are no XP updates anyway). Any other software that uses auto update is disabled when possible. As an estimate of my old XP/AMD cheapo Biostar mobo machine used for capture, it would take about twice as long to run any script. I have a newer small Win7 Pro x64 refurb PC but noticed no outstanding speed increase running Avisynth -- I just don't have to reboot as often as with XP when working heavy processing for long periods.

I've done a little work with high definition on the win7 PC, but I guess I'm just a holdout. I do almost everything on XP, being a bah-humbug renegade who thinks mostly that what Microsoft did to everything post-XP is near criminal. HD hasn't yet given me the buzz I expected from it, even on my big plasma TV.

Maybe others can explain the slowdown, I'm running out of ideas.

In the meantime, I still had some AviSynth questions in this post:

...now I can't even get a standard AVI - no AviSynth or VirtualDub processing at all - to run at more than 2fps.

EDIT: Closed VDub, reopened it, tried to open the video file:
---------------------------
ffdshow error
---------------------------
AVISource autodetect: couldn't open file 'Captures\Capture 1st AVT.avi'
Error code: 3
(ffdshow_dec_avisynth_script, line 3)
---------------------------
OK
---------------------------


I don't get it. It's just an AVI file?

Sorry, I had something typed but it got lost in the AVSmeter activity.

With all versions of Avisynth later than 2005 or so, the color functions of Tweak and TweakColor are essentially the same. Tweak has additional paramrters for adjusting luminance factors (Brightness and Contrast). The old and new versions of Tweak have "Hue" and "coring" functions that TweakColor doesn't have.

Yes, "Smooth" in TweakColor and "interp" in Tweak are similar. TweakColor has no "dither" function. Minsat and Maxsat are also similar. The complaint about the older Tweak versions was that the effect of adjusting color ranges was not as precise as it is these days or in TweakColor. Tweakcolor was a 2004 modification of the older Tweak. If you like, you can use Tweak in the same way that TweakColor was used. Bit if you use Tweak, set "coring" to false.

You don't say what function you refer to but I'm guessing you refer to "Levels" and to "dragging" as the way the controls are pictured in AvsPmod, which I don't use. Avisynth's Levels is a command-line version of the Levels filters found in GUI's. It works pretty much the same way the GUI versions do. Input and output settings refer to the left-hand and right-hand pointers in GUI Levels controls. Gamma refers to the middle pointer. Most levels filters adjust luminance, not chroma. In RGB advanced controls such as Photoshop Pro or After Effects you'll find Levels controls for all colors as well as luminance. YUV levels controls work on luma only.

No, numerically there's no min or max value specified that I know of, but you can use a "Levels" control to either adjust for valid levels or to completely crush darks and blow out highlights, as you wish.

There are 3 different functions in RemoveSpotsMC. The syntax for the three functions is:
function RemoveSpotsMC(clip, int "limit", bool "_grey")
function RemoveSpotsMC2X(clip, int "limit", bool "_grey")
function RemoveSpots(clip, int "limit", bool "_grey")

For each of those functions you can use code that calls them using default values:
RemoveSpotsMC()
or
RemoveSpotsMC2X()
or
RemoveSpots()

If you look at the script you'll see that the default values for "limit" are defined in the script, It wouldn't be as good idea to change the default, since the script itself changes them. "Limit" is a threshold or strength/mode value.

The three functions are arranged in such a way that you can call any of them separately.
- If you specify RemoveSpots() alone, there is no motion vector search to help determne more precisely what is a spot and what isn't. The spot remover action is perfomed only once, on one image.
- If you use RemoveSpotsMC() a motion vector search is performed -- basically meaning that image motion is examined to help determine whether something that looks like a spot exists in more than one image. This version then calls the spot remover activity only once.
- If you use RemoveSpotsMC2X(), the same motion vector search is performed but cleaners are performed twice.

The only syntax value people change is "_grey", which refers to grayscale images. "_grey" default is false for color video), set it to true for monochrome. For monochrome _grey to be truly effective, however, insure that your image is truly greyscale by using Avisynth's grayscale() filter before calling these functions.

The motion vector operations in the script are from MVTools, while other operators are in the original RmoveDirt() plugin. MVTools and RemoveDirt are required for Remove Spots, but if you have something like QTGMC you already have those support plugins.

You see a lot of YUV color wheels in lots of places. You even see squares and triangles. The wheel in the .rtf document was prepared by someone else (I don't even recall where I found it) and assumes that the zero-position of the wheel is at the center top of the wheel. The YUV wheel in the Tweak Wiki is more common, where the "start" position of the wheel is at the right-hand horizontal corner. This position in the image is considered as 0 degrees. The way it's pictured makes no difference except to help those who think a wheel should "start" at a certain physical location. In the inner coded workings of YUV, there is no such visual distinction. The center degree location for a pure YUV red is always at "110" as far as the coding is concerned, and likewise the accepted range for the "reddishness" of a color extends for 60 degrees from 80 to 140 around the wheel, regardless of where the 0 position is physically pictured. YUV's internal coding doesn't use a physical picture. Use degree numbers as your reference, despite the way different pictures show the positions.

:huh1:

What does Line 3 say?

VirtualDub is where I've done all my levels adjustments in the past, along with Photoshop. But AvsPmod's work basically the same way.

Right, I get that. It's the interaction between input, output and gamma that I'm not quite across. If I want to make something darker, for example, I can decrease gamma, increase input low, or decrease output high. They all seem to do pretty much the same thing.

Well, for example, I have this piece of garbage:
Attachment 6647

Which I adjusted with Levels(-30,0.7,280,16,235).
Attachment 6646

But I'm not sure if I'm doing that right, or if I should be keeping input and output within the 0-255 range and just playing with the gamma, or what. I mean, I seem to be getting more detail than I started with. Certainly more than my previous effort where I didn't adjust the levels until the VDub stage.


What does this all mean, in practical terms? What is it even doing?

Well, I was trying to use the wheel to guesstimate the correct number. I ended up plugging in 0-359, just to make sure it worked, and then bringing them closer together until approximately the correct range was affected (in this case, the green basketball court above).


There is no line 3. It's an AVI file. Just an AVI file. Not even an AVS that merely opens an AVI file, it's literally just an AVI file.

I also had that file open via an AVS in AvsPmod and another instance of VirtualDub, after closing everything it now works fine, including playback speed. Bizarre.


EDIT: I also keep getting this when I open AvsPmod:
Attachment 6648

Not quite. If you have out of range darks or brights in YUV, as soon as they go to RGB they're cllipped. You can't retrieve all of it, you can only brighten or darken what's left. If you brighten or darken those areas, you're working mostly with noise after clipping.

Depends on what part of the spectrum you want adjusted. Gamma works mostly on the midtones, say about RGB 64 to RGB 180, the range being centered at RGB 128. Input low works from RGB 0 upward to the midrange, input high works in the opposite direction and down toward the midtones.

The preferred range in YUV is y=16-235. If you can post a short piece of that video segment I'll try to show how to handle it. The "fixed" image posted is washed out and has a strong color cast.

In short, the functions look for spots, dropouts, and other garbage and repair them by comparing "good" data in adjacent areas and/or frames. Each function is more thorough than the other, with the MC2X version the most thorough. How it decides what's a spot and what isn't is in the programmer's source code. You''ll need the source code and your C++ and assembler language manuals to read through that. I couldn't help you there.

Specifying 0-359 selects and desaturates all colors, not just one. If that's what you wanted to do you could have just used the "sat" parameter in Tweak and not specified any StartHue or EndHue. But I don't think that's how you should do it. Judging from the original image posted, the scene already appears to have low saturation.

Don't open the same file in two different apps at the same time. The two apps are competing for access.

Another reason why I don't use AvsPmod. I have the same Autolevels dll with the sme name, so I don't know what AvsPmod's problem is. But you seem to have some junk in your plugins folder. Renaming one version of a dll and adding another dll with similar internal code will be a problem. You also have two versions of mvtools v2 in your plugins, not good because both have similar internal code and function names. If you needed a different version for one project and another version for another project, keep one of them separately in another folder and import the one you want explicitly with LoadPlugin().

I run projects using AVisynth 2.6 most of the time, but I have older projects that require v2.5 and 2.5 plugins. I can switch versions of Avisynth.dll in the System folder, but I have to explicitly import some 2.5 dll's in the script from another folder. That's not as often as it seems, as most 2.5 plugins also work in v2.6. But a few oldies don't.

Most people don't download a zip'd or compressed plugin package into the plugins folder. They download zips into a separate folder somewhere, unzip it there, look over the specs, and move only the needed dll's or avsi into the plugins folder. Otherwise you have a plugins folder full of cpp's, htm's, readme's, and other junk that shouldn't be there.

I just meant in terms of dragging sliders to adjust numbers.


If you think that's washed out you should see the version made with a DVD recorder/VHS combo unit. Or even the version where I adjusted levels in VDub instead of AviSynth.

What I posted also isn't finished, it was just the AviSynth step (and possibly not all of that, I don't exactly recall). Further color work is occurring in VirtualDub, although it's still complicated by the fact that there's really not much in the original anyway.

Irritatingly, it's an overcompensation made at half time as a response to the first half being underlit, but that didn't clip any details so I've been able to actually get that one looking okay. I'm running the first part of that script through VDub as we speak, so no samples until tomorrow.


So it basically works like noise removal?

I set it to 0-359 just because I couldn't see any effect when I had it around what I thought should have been 'green'. I then gradually increased startHue and decreased endHue until it was doing what I needed. And I was working with hue, not saturation (basketball courts aren't green).

With my computer being so slow, having just the AVI open is the only way I can scan through the footage, and I've done that no worries in the past. Might just have been AvsPmod.

That's how they are in lordsmurf's plugin pack; I just downloaded it and copied everything across; from the post to which he attached it I get the impression he just zipped his own plugins folder.

Samples of that other game.

Thanks for the new captures.

These were made with defective cameras? Or something just as disatrous, whatever it was. 1stHalf isn't just underexposed, it's worse than that. Both captures have crippled output in the blue, magenta, and red range. The best you can do is to try and stertch those valus into something visiblem but the results will look prtty bad.

This "levels" statemnet needs a mention:
With this type of YUV there are no values of "-30" or "280". The capture card clipped darks at y=16 and brights can't go higher than y=-255 with this type of video. so the dark and bright input levels you mention don't exist. The "Levels" and "Color2" YUV histograms below can illustrate those points, where you can also see that color values barely exist in the lower half of the vectorscope except as a few dimly colored dots:

http://www.digitalfaq.com/forum/atta...1&d=1476539821

Basically what that levels statement does is to drag some of the brightst values into the y=235 range and make some burned-out brights a little darker. The other color work raised what little red and magenta was available, but because so many color values are missing in the lower half of the vectorsope it affected the court colors adversely (the markings on the floor aren't white, for one thing, and shadows elsewhere are green). But you're correct in that the best you can do is to stretch the avaialbe data to look like something else, even if few of the colors look correct. Another problem is that the capture itself apparently tried to include values that are outside of the video's capabilities.

1stHalf,avi is simply a dark capture where levels and color values hover around the dark-to-midtone range, and most color values approach gray. The originals are simply too corrupt for anyone to expect very much. Even if you made 1stHalf a grayscale image, it has no values beyond the midrange. You can make the darks and mids brighter, but there's no color detail to be retrieved. What remains of the video is mostly noise.

Not sure if it's just my imagination, but there does seem to be more detail when I have -30 and 280 there than when I have 0 and 255.

And yeah, I wasn't expecting much from either of them. Although I've got the first half looking pretty decent, actually. Good enough that it's overtaken the first half in terms of watchability, at least.

And yeah, I have no idea what was going on with the camera or its settings.

Attachment 6663Attachment 6664

There in;t more detail, there are just brighter darks. I don't think you applied the -30 or 280 values to 1stHalf.avi because those values don't exist in that video.

http://www.digitalfaq.com/forum/atta...1&d=1476573000

Even if lower and higher values did exist, the coded statement "Levels(-30,0.7,280,16,235)" wouldn't brighten anything. All it would do is compress existing values into a more narrow low midrange:

http://www.digitalfaq.com/forum/atta...1&d=1476573270

You might have used that Levels statement on the far brighter 2ndHalf.avi, but you would just be making tough work for yourself if you used it on the darker video.

The coection image you posted earlier from 1stHalf.avi is a decent workable effort, even if it doesn't look as if it resulted from the Levels code posted earlier:

The image is certainly and improvement. I'd suggest that the background bleechers appear to be covered by a subtle orange haze, and contrast overall is low with no clean blacks in the image.

I had similar results straight from Avisynth, but I used this code:
http://www.digitalfaq.com/forum/atta...1&d=1476590370

The code resulted in a rd haze in the darks and reddish blacks. So after some Virtualdub tweaks with two gradation curve, the darker bleachers look more detailed (even though they aren't) by throwing a little more contrast into black levels with the curves filters, which I exaggerated slightly here to show the effect.:

http://www.digitalfaq.com/forum/atta...1&d=1476590532

Any way you look at it, it's still pretty awful but you're correct to do what you can with what you have. I don't think recapturing this video would help -- color is wiped out in the original, so a brighter capture won't help.

Attached is my own YV12 Lagarith correction. Some of the noise is tamed, but underexposure will get you little more than noise and ghosting when brightened.

The levels I mentioned earlier was for the second half, not the first half.

That capture for the first half was done through an AVT-8710 with brightness all the way up. A capture through the TBC-1000 was even darker.

For the first half:
What does SmoothLevels actually do? I just have that sitting there from when we were working on the other game a while back, and you had that in your script.

EDIT: Going back over your second "Studio" script, you've got "RemoveSpotsMC().RemoveSpotsMC3()". I'm getting an error citing no function named RemoveSpotsMC3.

I still don't really understand the difference between RemoveSpots, RemoveSpotsMC, RemoveSpotsMC2x either, in terms of what happens visually or what visual issues each is best for or whatever.

EDIT 2: I like the color of the court better on my version, but you've got the jerseys to stand out a bit more. Yours also looks quite a bit noisier to me. I'm not really too fussed about the crowd as long as what's on-court looks...I was going to say 'good', but that's probably not the right word under the circumstances.

In VDub, I ran hue/saturation/intensity with hue -14, sat 0.84, intensity 1.09, then ran it again for cyan and blue with saturation 0.63. Gradation curves moving the bottom left corner up a bit, and Camcorder Color Denoise at 30.

A more advanced version of "Levels", it applies stronger dithering that seems to produce a smoother transition between the darks and brights that it works on. It's one of several functions in SmoothADjust, which has more than 20 paragraphs explaining its settings. XP version and ReadMe documentation attached, for all versions of Windows. Later versions are for 64-bit WIn7/8/10.. I never found a link for the source code. Some complain that the author doesn't bother to clarify some obscure features, which is probably why many people use only selected parts of the filter and ignore others. Most use it as a smoother-looking version of Avisynth's v2.6 "Levels". SmoothAdjust2.62-XP.zip. Original discussion thread, http://forum.doom9.org/showthread.php?t=154971.

That's because the RemoveSpotsMC avs plugin has no RemoveSpotsMC3 function. It's found in RemoveSpotsMC5, which you don't have and don't need. It's overkill, destroys details, and crashes if you try to use it like RemoveSpotsMC. Stick with RenoveSpotsMC2x in RemoveSpotsMC.avsi.

Motion compensation (MC to you, or motion analysis) is used to make RemoveSpots more effective during shots with motion involved. The visual issues it's designed for are spots, comets, horizontal rips and tears, and many forms of scratches -- all of which are classed as defects known as dropouts or loss of signal in areas of the frame.

In the image below from post #44, there are two horizontal dropouts: a horizontal "rip" across the man's jacket and another short, thin black line in the upper right that passeds thru the edges of the logo. There are many such dropouts in that capture and others.

http://www.digitalfaq.com/forum/atta...riginal-640jpg

If you don't know what these bad guys look like or if they don't matter, don't use RemoveSpots, RemoveSpotsMC, DeSpot, DeVCR, or any such plugin, they will just be confusing.

Those are all effective techniques, and of course color balance is a personal preference and up to the user. It's a shame the original recording is such a mess, but there's only so much one can do with any available tool, even very expensive or sophisticated ones. For the YV12 avi I posted, I didn't use QTGMC or Hue/saturation/intensity, only dfttest alone and then gradation curves and Levels in VirtuaDub. I figured you would apply more adjustments than I did anyway and this was mostly a gain_y sample for ColorYUV. As for denoising, there's too little detail for extensive or aggressive denoising without ending up with a plastic look and losing more definition. Ghosting during motion is too dmaging already in that video, besides other problems.

What are you actually doing with it, though? Gamma's still 1, input and output are the same...and why is it now 16-255 instead of 16-235? I'm still not entirely sure I understand the 'protect' thing either.

So what would I use in place of MC5 in your sample script? Just leave that off entirely, since it's also running RemoveSpots, or add something else?

Motion compensation (MC to you, or motion analysis) is used to make RemoveSpots more effective during shots with motion involved. The visual issues it's designed for are spots, comets, horizontal rips and tears, and many forms of scratches -- all of which are classed as defects known as dropouts or loss of signal in areas of the frame.

Yay, more options!

I still don't really understand which to use when. If I've got footage with those sorts of issues...I assume there's more to it than just randomly picking one of those plugins.

EDIT: VDub crashed again. "out-of-bounds memory access (access violation) occurred in module 'ntdll'...writing address [bunch of zeroes]...while stopping filter "Color Tools [1.4]" (FilterInstance.cpp:1214)."

No, that line isn't my code. It's yours, posted here:

My code was different, posted here:
My SmoothLevels line is dithering the output to smooth gaps between luma and chroma values that were stretched and compressed by gain and saturation, as well as taming the y=255 brights by intetpolating them down gradually to y=235. What your version of the line is doing is dithering or smoothing output from the previous filters, which won't hurt but is unnecessary because you have already used the regular "Levels" filter and activated its own dithering. You're using versions of two filters designed to have the same or similar functionality.

quote=koberulz;46069]What are you actually doing with it, though? Gamma's still 1, input and output are the same...and why is it now 16-255 instead of 16-235? I'm still not entirely sure I understand the 'protect' thing either.[/quote]The two versions as posted above. and they are not identical, as a I've shown. You're definitely unsure about them, as evidenced by using both of them together in succession in your own script for no apparent reason.

The logic behind the "protect" parameter is in the documentation below, where I placed the explanation for "protect" in bold and quote a general description of the SMoothadjust plugin package. This documentation was attached to the SmoothAdjust plugin that was posted.

Use any of the 3 RemoveSpots functions in RemovespotsMC.avsi. RemoveSpots() is the weaker function, but it doesn't harm fine detail very much. RemovespotsMC() is more effective for videos with motion. RemoveSpotsMC2x is the strongest of the three and required for fast motion. Use it the same way you saw it used in earlier scripts. An example was previously posted in "Script 3 (Optional)" near the bottom of post 71 at http://www.digitalfaq.com/forum/vide...html#post45935

This is the way most people do it when designing a script. Use one, and if it doesn't work, use the other. If neither of them work. use a third version if available. If none of them work, then (a) you're using the wrong filter for the job, or (b) the video is so bad nothi9ng will fix it, or (c) you should research to see if any other method is available for the specific problem. Meanwhile I can save you some trouble beforehand and advise that RemoveSpotsMC and RemoveSpotsMC2x are the functions to use to reduce the number of dropouts in action videos. Don't run these filters while a video is still interlaced. First, deinterlace or use SeparateFields.

ntdll is a Vista systems file. Sounds as if you need to reboot and clear some memory.

Great scot, I just re-read the previous post and was shocked at all my typo's. So why don'[t I see that junk before I hit Submit after reading it 5 or 6 times?

@koberules, everybody learns this stuff piecemeal. Years back a member started a thread on video restoration and repair but lasted barely four posts before everyone realized that a 6-volume book was the only way anyone would be able to cover the subject. Remember, some of us have been at this madness for quite a spell. They say you can tell by the weird look in our eyes. LOL!

So SmoothLevels is used in place of Levels...when? What's the case for using one or the other (or SmoothTweak vs Tweak vs Tweak Color)?

I read that, it still didn't really click at the time. Occurred to me later on, though...if it's bringing the blacks closer to grey, it stops black borders from becoming greyer?

Okay, so I'm using this:
But in VirtualDub, it does absolutely nothing except screw with the colors:
Attachment 6673

Whereas in AvsPmod, I actually get some spot removal (with the same color issue):
Attachment 6674

Using MC2x gets me the VirtualDub result in both VDub and AvsPmod.

It's the second of two scripts, so I ran QTGMC in the first script and left it deinterlaced, with the re-interlacing done in the second script after RemoveSpots(). Commented that line out so the frames would match for the StackVertical().

The biggest problem is the concern of patching a video up, and then weeks/months/years later finding out that some problems that remained could have been fixed, or problems that were fixed could have been fixed better. For example, I'd just assumed I was stuck with those dropouts. Which I might still be if RemoveSpots() insists on continuing to screw with me.

TweakColor and the newer version of Tweak have simlar functions. I think TweakColor is more precise with the hue ranges and looks smoother. Use the one you like.

You don't have any superblacks (darker than y=16) because your capture card is clipping a y=16. I've been using SMoothLevels as well a Levels after using other filters that massage existing levels first (ColorYUV). Eitrher way, if you include black borders in the statement, Levels will make the video brighter than it should be at the dark end, so it looks washed out.

The pics don't look like the same processing stage to me, and I'm not sure what your two scripts are doing. How did you get your "Intermediate" again? Was Intermediate.avi the result of the first script, is that what you're saying?

All versions of RemoveSpots do some smoothing, even of chroma and can smooth some elements of color flicker, but they don't affect saturation and flicker the way your images show. The two sets of imagrs have different contrast levels, which I've not seen with RemoveSpots. You should show us what the first script is doing. Also, if you have filters mounted in VirtualDub, the results will be different. Again, U'm not using AvsPmod because I found no particular use for it, but many like it.

I still don't understand what's going on with the slow Vista machine. Your i7 CPU was benchmarked as only about 20% slower overall than my Intel i5 Ivy Bridge, so something else is wrong somewhere.

Thank you for the first script you used. "Very low" for MCTD won't stop the flicker, but I realize this is a demo only. I used the same script to get the attached Studio_intermediate.avi, but I omitted the Trim statement because I'm using the original Studio.avi submitted earlier and don't need to trim().

The result is attached as Studio_intermediate.avi (losless Lagarith YV12, deinterlaced per the above script).

I used the script below which I copied from your own Script#2, but I used StackHorizontal to place the fames side by side for playback. Original on the right ("y"), RemoveSpotsMC version on the left6 ("X"). The comparison clip is attached as Studio_Intermediate2.mp4. Left and right look pretty similar to me. Although RemoveSpotsMC cleaned a little of it, that version of RemoveSpots isn't as strong as RemoveSpots2X and doesn't accomplish much here, but it doesn't do anything to the colors. The script:

However, that's not the way I used any version of RemoveSpots. The scripts I posted show that the function was applied to re-interlaced video and used SeparateFields(), which breaks the frames and motion into smaller and dissimilar pieces. I also used RemoveSpots2X, shown below (attached as comparison clip Studio_Intermediate2X.mp4, interlaced), which looks slightly cleaner as far as dropouts go. The script I used:

Note that "y=a" in the above script refers to the re-interlaced version of the input. You might remember that the original filtered version of Studio.avi posted a while back used RemoveSpotsMC3, not RemoveSpots2x, and I even threw in RemoveSpotsMC for good measure (that was little much, too much softening). Considering all the damage in Studio.avi, RemoveSpots3 did more cleanup of dropouts. I ran another script on Studio_intermediate avi, this time using RmeoveSpots3. I also cleaned up the borders that were changed bv stabmod in Studio_intermediate.avi::

The side by side comparison in the attached Studio_intermediate3.mp4 shows the same color as on both sides of the comparison frames, with the left side cleaned up more than with the otehr versions of RemoveSpots.

I've also attached the avs script for RemoveSpots4, which contains five functions that called be called separately:
RemoveSpots
RemoveSpotsMC
RemoveSpotsMC2
RemoveSpotsMC3
RemoveSpotsMC4

It took all morning to clean up that original avs file so that all functions worked properly. RemoveSpotsMC4.avs is far more complicated than the versions in RemoveSpotsMC.avsi. Caution: RemoveSpotsMC3 is very slow and RemoveSpotsMC4 is even slower.

Also, you can write the code above:
with this structure, which means the same thing:
But the lower version would be a little slower because it performs SeparateFields twice.

What's the difference between running RemoveSpots() before reinterlacing and running it after reinterlacing and separating fields?

Do I need to do anything special with the two version of RemoveSpots(), or can they both sit in the plugins folder?

Must be one of those days. Multitasking: not my thing. Quoting myself, from earlier:

And then I forgot to attach Studio_Intermediate2.mp4 !! :rolleyes: :o
Attached herewith.

Not sure if you missed this post while sorting that out:

As noted earlier:
Meaning that if you look at the pattern of those dropouts streaks and rips in VirtualDub frame by frame, you'll see that some streaks stretch over multiple fields, some over multiple frames, some are thicker and some thinner. A temporal filter like RemoveSpots/RemoveDirt is based on time, that is, on the way images change over multiple fields or frames. How the filter decides what is noise and what isn't is known to the filter authors rather than to us mere mortals. But if noise remains the same and in the same place over multiple consecutive instants, the filter doesn't recognize it as noise. Breaking the motion into smaller and more disparate pieces makes the noise pattern look more intermittent and thus is more easily recognized as noise. Then again, a filter might be more effective with things like horizontal dropouts because a dropout that looks four pixels thick in a full frame may be only 2 pixels thick in one field but physically lower in one field and higher in another -- or in one field the dropout could thin out to 2 pixels in a slightly different position later. In this case it's an effort to break certain noise patterns into smaller, less distinct pieces. What I did was to observe a few frames using one method and observe the same frames using another method. You don't have to review an entire video to see what's happening with noise patterns.

Also many of these filters work faster on half-sized fields (SeparateFields) than on full-sized frames (full-frame deinterlaced). Some filters work best with real deinterlace (such as dfttest and chroma cleaners), others work just as well using SeparateFields. It take some experimentation while learning what certain filters do and don't do. MCTemporalDenoise, when used for chroma cleaning to reduce flicker from oversaturation, seems to me to be more effective with deinterlaced frames. I suppose people who can read the filter's source code could tell us why in greater detail.

Users tend to build up a small battery of frequently used filters and soon learn their quirks with different kinds of problems. You don't have to know about every filter in the world -- none of would live long enough for that, anyway.

You can keep both of the plugins as .avsi files in your plugins folder. I have 3 versions of RemoveSpots scripts in my plugins, all kept as .avs files and imported explicitly, but that's just my choice because the 3rd version, RemoveSpots5, has never worked properly (I just just get rid of it). Here is a table of the two plugins as .avsi files and the internal function you can casll from each:

You can see that two internal functions have identical names in both .avsi's. It doesn't matter which of those identical functions Avisynth decides to call, because they both are coded alike. No problem. The functions that don't have identical names operate differently internally, which is why the plugin designers gave them different names. in RemoveSpotsMC4, the #2, #3, and #4 functions internally invoke additional coded routines and work more slowly. They do more work, so it depends on how slow you want to run, how much you want cleaned, and how much fine detail you're willing to sacrifice. I found that RemoveSpots2X was good enough for some of the clips, but clips like Studio.avi had drastic dropout problems that needed RemoveSpotsMC3 for better cleaning. RemoveSpotsMC4 was slower overkill with no appreciable improvement.

Note that any RemoveSpots function can sometimes obscure very fine lines or remove spots that aren't really spots (using them in a rainy or water sports scene can have strange results, with lots of water removed! I hate that). Sometimes you have to try another filter such as dfttest or RemoveDirt, or just live with some noise.

Right, but what does SeparateFields() do that QTGMC() isn't doing? They seem like the same thing. Obviously I'm missing something.

You should ask the question in reverse: what QTGMC does that SeparateFields doesn't do. By now you know the difference between deinterlacing (reforming two separated half-field scanlines into two full-sized frames) and SeparateFields (using alternating scanlines as half-height video images with no interpolation or resizing into full-sized frames). If you'd rather use the same slow RemoveSpots filter on 1152 horizontal scanlines in 2 full-sized reinterpolated frames instead of 576 horizontal scanlines in two separated but unaltered image fields, it's up to you. You can run any of the scripts shown so far, including your own, using QTGMC or using SeparateFields, and check the results. Cleanup will look similar, but not quite the same.

With SeparateFields, the even lines begin at scanline 0, odd lines begin at scanline 1 (one line lower). Each new half-sized "frame" of even lines begins scanline 0. But the odd fields, in order to populate 288 lines in the new separated "frame", have one black line of pixels added at the top, so that the odd-line image has data beginning at line 2. A speck of noise in even-lined fields will be 1 scanline higher than in the odd-lined fields -- thus, the noise in the even fields is in a different position than in the odd fields. Further, a line of noise in even fields may be 1/4 line shorter in the odd fields or might not even appear in some alternate fields.

Remember that an interlaced frame has two images. When you see a line of black or white dropout pixels in an interlaced frame, that line of dropout might only be in one of the fields, with the other field clean. Or the line of dropout might be 3 pixels high with only one line of dropout pixels in the even field and the other two dropout lines in the odd field. And again, a line of dropout in the odd field of one frame might be different in the even field of the next frame.

And yet again, when a time and motion based intepolator like QTGMC starts rearranging data to make separate half-height fields into full frames, some noise such as dropouts can be carried over into subsequent frames, so that dropout won't look the same in all frames. Some advanced users who get really picky will perform some "prep" noise reduction before using QTGMC to prevent this carryover of artifacts. It's not always safe to do so, but you can try it using SeparateFields, then reweave the fields into interlaced frames before using QTGMC for other problems such as smoothing line shimmer and aliasing.

As an experiment I looked at the original studio,avi and the same clip processed as Studio_intermediate.avi, which is what you get after running QTGMC on Studio.avi. I noted that the thick horizontal dropout in frames 23 and 24 (or 4 interlaced fields) of Studio.avi appeared in 5 fields of the intermediate avi after QTGMC, and when reinterlaced that dropout and a remnant appeared in frames 23, 24, and 25 of the reintelaced video. So it becomes obvious that motion compensated filters like QTGMC, MCTD, etc., etc., can indeed create their own problems (you decide what's more important in any particular video, some subtle interpolation errors and less than perfect cleanup, or no cleanup at all). These interpolations don't occur with SeparateFields.

So, with my experiment I first ran RemoveSpotsMC3 on Studio.avi using SeparateFields, then reweaved the clip and ran the rest of the script on it including the QTGMC line. It took 15 minutes at less than 0.3 fps on my machine, so be prepared to sit for a good spell waiting for it on yours. The results look like those of previous running methods but took far longer, mainly because RemoveSpotsMC3 had a lot more original noise to contend with, which took forever.

Oops, here there I go again. Quoting myself:

Nope, the first odd line begins at line 1 in the SeparateFields view. I guess 2 cups of coffee weren't enough. I'd better upgrade to 3 cups.
:o

I'll go through that in more detail later, but quick question: can I split out the MergeChroma(MCTemporalDenoise(settings="very high") line? ie have a script that goes up to the line before that, create the AVI, then run the MCTemporalDenoise on that AVI to create a third AVI, then use MergeChroma("thirdavi.avi")? Would that have the same effect? That line seems to be the point at which my computer decides flailing its arms in the air and storming off is its only option; I'm thinking separating it out like that might help.

Of course you can split your processing. That's been discussed since post #71 and earlier.