Avisynth: Optimal script now fully adaptive
 

@All,

Here it is for you to try it before I post it on the optimal script section :)

The original script would blur as action increases after a treshold, but the TemporalSoften had static values. So now this change makes the TemporalSoften function to operate at very high values on minimal movement, and as movement starts to increase, it decrease the values linearly. So now we have full linearity on both functions :D
This should buy us even higher CQ values ;)

-kwag

@kwag,

That's fantastic :D ! I'll try it out right away!

hi Kwag.. :)

Great work on everything here..

Question...

* Have you tested this latest "code fragment" out on Animation ie, like
movies such as "Shrek" or "Ice Age" (<-- you have?? )

Unfortunately, I don't have any Animation movies like these on DVD, but
shortly, I'm going out and make a purchase on "Ice Age" - spooze to be
a good movie.. and looks good when I pass it by while playing on a 42"
wideScreem. Anyways..

I'm testing your latest "snipplet" out on the lostest, vcd4ever's posted
file (hope he doesn't mind) until I get an actual DVD shortly (above)

Oops.. just heard TMPG's ding. let me see how it looks and filesize. I'll get
back ta you shortly. But, this is not the best test I have.

Thanks for any input.
-vhelp

@ all..

Ok, just finished prepping my sample test file.

* Unfortunatly, I got some color washout cause of the re-encode and the
...added ConvertToYV12 conversion that the MA script requires, else
...I'd get the YV12 error (per vdub) ***
...But, this is not due to the MA script, just the conversion to YV12 :!:

* Still need to "nail" that bug when BfDfAf.. in any case, it still works
...great, and you don't notice this issue anyways when you play it.. just
...that it would be nice to "nail" it once and for all. But I like :)

* final size: I saved an extra 600kb !! on a 26 second test clip

*** we gotta try and put our heads together and find ONLY the YV12 filters
to use in our MA scripting or what-have-you, and "pefect" them and scripts.
These color conversions are killing our encodes (at least mine)

Well, I'm off to the store for "Ice Age".. see you in a bit.

So, good work - - success thus far :wink:
-vhelp

No, I'm trying it out on a same movie (K-Pax) for consistency checks. The stills and almost still scenes look even better now :mrgreen:

-kwag

You definitely have a CODEC problem :!:
I don't have to use ConvertToYV12() with the current MA script :!:

Let us know how your "Ice Age" encode comes out 8)

-kwag

Great kwag awesome job. I just hope prediction isn't off again.

It's not, I tried it :mrgreen:
However, and I warn you :!:, there are some small discrepancies between using the fast prediction and the normal prediction. I'm settling for the normal (longer) prediction, because it's much more accurate.

-kwag

hi Kwag pal..

I'm back. poowy.. I walked in looking for one Movie, and here's what I
walked out with:

* Ice Age (yeah)
* Monsters, Inc. (another yeah, cause it's a great movie)
* Toy Story (came w/ BOTH movies, price cut)

So, I guess I walked out w/ 4 DVD titles, sheesh!!

You bet!!

Hay, funny short story.. K? .. .. ..
--- --- --- --- --- --- --- --- --- --- --- --- --- --- ---
I was at the counter, with my back turned, watching the TV (Dune was
playing at the time)...
And, this cute Spanish cashier was checking me out while I wasn't looking or
noticing. So she thought hehe.. She didn't know that I could see her eyeing
my BUTT while the TV screen was reflecting the darker scenes hehe..
I played a nasty joke, and quickly turned around "..ain't that a great movie??"
she was stunned!! Busted!! hehe.. but I was nice enough, and played it off
like I didn't know.. "cought in the cookee-jar" hehe..
--- --- --- --- --- --- --- --- --- --- --- --- --- --- ---

Take it easy.
-vhelp

@ Kwag..

Yes, Yes, I agree. Good show.

I can't wait to mess around even more now.

But, Kwag.. I'm still haing problems w/ setting the resolution to
704x480. I need this res set in TMPGnec.. NOT in my scripts!! It's my personal
opinion that I have "better" cuts and accurate (AR and tighter compression)
when I go for the 704x480 and 16:9 encodes for MPEG-2. I'm trying even
for MPEG-1 (even though 16:9 arent supported) but for some reason, even
so, I ToK keeps re-writing an NEW video.tpr file. I noticed this because it
always as the same date 8O What the heck am I doing wrong ??

* Is there a way to change this, so that I can have TMPG set my final resolution
...to 704x480 (or any other resolution) ??
Or.... ....
* Do I have to resort to trickery (prob. won't work anyways, and might end
...up requiring "manual" process instead of ToK) Hmmm... that would mean
...I would have to come up w/ a NEW app (of my creation) to pretty much
...do basically the same thing that ToK does.. hmm...

Looking forward to your (or anyone elses) assistance on the issue, cause I
really need it !!

So, what-da-ya think about my problems.. can help me ??

Well, Spaggetti is for dinner hehe.. My favorite for Weenend meals, w/ a nice
Coca-Cola and some burps)
-vhelp

@ all..

YV12 issues...
Here's what I've got, in hopes of fixing my ealier color washout experience.

The following two lines are producing errors:

* TemporalSoften(2,7,7,3,2) - - - give me an "YUV" error message
* unfilter(1,100) - - - - - - - - - - - give me an "YV12" error message

Please forget about the values I used inside the filters. I just needed them
to work and produce the error messages.

Below is some code showing How I'm arriving at the color washout ??
I don't see any temporalSoften() .DLL's anywheres.

Thanks a bunch all.
-vhelp

TemporalSoften is built in AviSynth 2.52. Could your problem be that you are using an older version of MPEG2DEC3.dll :idea:

-kwag

@ Kwag..

Your script must be working, cause my test encode of "Ice Age" is really
slowing up my system 8O on this DVD. I must be doing something wrong.
There's a pause in every keystroke I make... Most be something else..

Anyways.. as far as MPEG3Dec.DLL .. .. ..

Here is my current specs:
* Size: 100KB (102,400 bytes), 114,688 bytes uses
* Dated: Sun, Janury 15, 2003 - 5:46:20 PM

-vhelp
PS: The movie is GREAT!! Go get it!! Hurry!!

Kwag it looks like you might be right, I think the faster prediction leads to sligggghtly higher CQs than full prediction, at least in the case of one test. However the latest change in your script seems to be really slowing down my encodes! And by a good 50% even! Can you confirm this? I'm not certain if it's worth the speed decrease if I don't see any big visual difference, cause the way things were my encodes looked pretty damn good.

Not at all :!:
At least on my P4 @1.6Ghz, encoding speed is almost identical.
I'm almost finished with K-Pax right now ( 89% done ), and the total encoding time will be about the same as my prior encode. ~5 hours, 30 minutes.

-kwag

@ Kwag.. and J-Wo..

hmm... interesting.. Maybe it as something to do w/ the type of Movie it
is. At least I'm not the only one experiencing this "slow up".

But, I think that if this latest MA script (snipplet part) is as good as it gets,
then it may be worth it after all. I mean, how many of you are actally
Ripping a DVD and encoding them every single day ?? I might do a few
a week or none at all.. just depends.. but once is usually enought unless
you find an NEW encoding technique that changes everthing (quality)

Well, that's my two cents worth, but J-Wo is right, it's deffinately slowing
down the encoding time (and my keyboard) Anyways..

My "Ice Age" encoding is going great. That is, the test clips I've ben doing
so far. I'm currently testing a few things out. Kwag I remember
you saying something about changing the GOP to 12. Now, I don't know
where in the line are you suppose to, so, below is what I have:
I also found an issue w/ some artifacts using the above GOP setting.
I noticed that the color will bleed on some STILL shots. When I changed
the MAX (last GOP setting above) it went away.

Can anyone explain this "bleeding" thing ta me ??

Other than that, the Movie "Ice Age" is fantastic !!

Thanks for any response.
-vhelp

ya I'm encoding the Dune miniseries, and each disc is about 1hr 30min long. On my Athlon XP 2500+ Barton, previous encodes would finish in about 2hrs, but with the latest change the encode is now taking 3hrs. I can see it chugging away too in the preview screen. I have it on preview with thinning, and before I'd see the picture change a steady, fast rate. But now I can see on some scenes it's fast but on others it slows down right away. I suspect it has to do with the "full linearity" aspect of MA.

@ J-Wo..

Come to think of it..

THAT'S HOW IT'S SUPPOSE TA WORK !! :) :) !!

This is what I've ben trying to describe (mechanism'ly speaking) for a very
long time. And, it looks like Kwag finally got it !! And, it's working !!

So, the only thing left to do now, is fune-tune this "mechanism" for speed,
and the other issue I mentioned above.

@ Kwag..
Looks like you got it !!

@ anybody.. Hay, what about my GOP ??
-vhelp

Hi vhelp,

Ice Age is a computer generated movie. It's not like old cartoons or anime movies that have very sharp edges and contrast. For that movie, you use the same GOP that you would use with a regular movie.

-kwag

Being that you use a different gop for cartoons, what kind of GOP do you use for a movie like Space Jam :twisted:

That would fall in the regular movie category. Because the toons are also computer generated :)
But if you go and encode "Felix el gato", "AstroBoy", "The Jetsons", or something like that :lol: then you REALLY need the shorter GOP :mrgreen:

-kwag

Here's a VERY critical sample for MPEG encoding. It's very dark, and the colors and tones are poison for MPEG encoding. Scenes like this is where the full adaptive script shines :)
http://www.kvcd.net/k-pax-full-adaptive.mpg

Note that there are almost no visible artifacts :D

-kwag

@ Kwag..

Hi, I D/L'ed your sample clip.

I did that scene, and I agree w/ ya about it's scene. It's murder !!
Took me a while to get it right, but there were other darkly lite scenes
that drove me crazy (and you) !!

Hay, it's time for a fresh cup of coffee hehe..

Hay Kwag.. I can't seem to get anything better beyond the 24 for the
GOP param in the last entry part. Seems to be the sweet-spot of all :wink:
.
.
But, I think I may have the answer to the "bleeding" in my test clip of
"Ice Age".. you have to play w/ the colors unfortunately. That is what I'm
doing now, and it seems to making the issue go away slightly.

I'm wondering if an U/L is ready for all ta see, thus far..

Thanks for your help all.
-vhelp

Final tuneup before I commit the script :) :

Changed the chroma portion of the filter [ round(2/nf)*50 ] to vary 25 times more that the luma part [ round(2/nf)*2 ]. We all know how much we can blur the chroma channel, but not the luma channel ;)
Also, changed the boundary treshold from 2.5 to 2. This means that the unfilter line will kick in at a slight lower value, and again, we gain compression without any visual loss in quality.
Just finished prediction on "K-Pax", and I set the same CQ to start, as the CQ I encoded. ToK exited on the first prediction, meaning that the current CQ is still within the .5% I set for precision :)

Now, let's play a little game: Can you find the dancing DCT blocks in the black and dark areas :mrgreen: www.kvcd.net/k-pax-very-dark.mpg

Edit: Sample above done with the script changes mentioned here.
Edit #2: Also, bye bye mosquitoes. The radius of "4" ( first parameter on TemporalSmoother ), blends them really good. Can you find the flying mosquitoes :mrgreen:

-kwag

@ Kwag..

I think I figure out my issue w/ YV12 (hence, color washout) !!

* When your source is a .D2V file, and you use AVIsynth in between,
...YV12 support is built-in, and filters for YV12 will work w/out issues.

* But, if you use VFAPI and make a psuedo .AVI file, then YV12 is not
...built-in, and hence, you have to use ConvertToYV12.. 8O hence
...the color washout.

Ahaaa!!

Now, I may have a work around this delima because of the way I frame
serve my source files into TMPG, for my WideScreen 16:9 projects to be
successful in quality.

-vhelp

The only thing you have to do is use GripFit. It will make your aspect correct every time :)

-kwag

@ Kwag..

Ok, I decided to give GripFit() a try, but I keep getting an error msg
that there's "no function.."

and I'm making the call:
Over..
-vhelp

Your script is completely wrong 8O
Get the latest script here: http://www.kvcd.net/forum/viewtopic.php?t=3483

-kwag

@ Kwag..

8O ..I know.. I knew that.., but I'm trying finish w/ my older one first,
than I was gonna try out the NEW one hehe..

Listen, I think I figured out how to use GripFit() (see above)
but it's not working. Instead, i have to use the snipplet below, and then
it works, but w/out GripFit() 8O So, i'm confused:

But, now, what I need, is to narrow it down to the cropped areas. The
above snip seems to crop off too much of what I need.

* The right and left sides are snipped, but I want to keep them!! How do I
...preseve them ??

* And, how can I fine-tune the cropped areas to whatever pixels I need ??

Thanks for accepting my headackes hehe.. :screwy:

-vhelp

You got to do some reading vhelp :)
First of all, why are you encoding 720x480 :?:
You might as well encode 704x480, and save some bitrate.
The 2 blocks overscan cut off what your TV will stretch and crop out, and you wouldn't see it anyway :!:. So you encode less pixels, and the result is a higher CQ and quality.
That's why we use 2 blocks overscan on resolutions >= 528x
It you want to get your correct GripFit lines, run MovieStacker, and check "Use GripFit".
Then when you change the resolutions, etc, you'll see how the script automatically changes the corresponding Grip(X) lines.
Then you can copy & paste those lines to your optimal script in Notepad or any editor.

Try it ;)

-kwag

@ Kwag..

Touchy!!

I do LOTS of reading. Anyways.. I'll continue, but to be quite honest w/
you, it's just too confusing to just crop and resize and whatnot w/ that
filter GripFit() which is a bit misleading anyways..

I think I'll play around w/ it a little while longer, and if I can't figure it out
enough, I'll have to resort to other means. I'm trying to eliminate the
vdub step if I can, but you're not making easy for me :)

In any case, those number you saw were posted too quickly. Again, I'm
testing numbers (params) and as I post, whatever happended to be in my
sample test to you were shown, which led you to believe either I didn't
read anything (not true) 8O or I'm too deep into what I'm aiming for.

Gosh.. I'm only trying to get a decent clip posted, but it's not ben easy for
me this whole afternoon and evening, and all in the name of ridding the
YV12 bug I've ben hit with lately.

But, I need to get things to work in AVIsynth the way I have them when
I use vdub. If I'm to rid vdub al-to-gether, I have to figure out HOW to
replace the same principles that I currently use in vdub, over in AVIsynth !!
And, it's not easy as I thought.

Well, I can see I'm wearing you out, so I'll stop w/ the questions and try
and figure things out on my own :)

Have a good night 8)
-vhelp

Ok, hi again.. hehe..

Here's what I've come up w/ so far, and it seems to work pretty decently.

Assuming my source is DVD rip..

So, why the above, and bicubicResize "size" ???
Well, that's because after the Cropping of the areas, I have to resize a
certain amount (after the cropping) ie, if the cropping from GripCrop()
produced a final size of 260, and my encode required an x272, I would
have to resize to that x272.
So, basically, if your size is smaller than your "required" size, as in my
example above (x272) just bicubicResize(720,nnn) and feed it to
TMPG for encoding. I can't think of an easier way, to quickly CROP your
boarders.

This works for me, and I'm happy again :) :)

I'm a little confused about what source_anamorphic=true does for me.
But, since there were no documentation on GripFit() when I D/L'ed
the file, and it's other siblings, GripBoarder() and GripSize() etc., I had to
do some hunting around for sample snipplets and whatever docs I could find.

Here's a great link by sansGrip (creator of GripFit():
* GripFit - dahh !! :screwy: me..

@ Kwag..
The above works great for me.. and I'm happy now :)
Thanks for staying up late w/ me.
-vhelp

@ Kwag..

I'm finally trying out your latest "snipplet" (still using my older script) but
changed to your 2nd latest:

To the newer...

I've tried it, and so far, it's as good as the above, but maybe for computer
generated sources, it makes filesize just slitely larger ie,

* 20 second clip w/ older = 1,460kb
* 20 second clip w/ NEWER = 1,499kb

Hay Kwag..
I'm thinking about maybe U/L'ing a sample VOB of "Ice Age" for you to
play with. I feel it's not fair that you don't have something to test with.

Do you want me to try and U/L a small piece to you. If so, please tell
me how to cut a small section from a VOB, and I'll do so. I don't want
to encode and make a VOB (actually, I don't know how hehe) I want to
send an orginal. Up to you.
Sorry, it's almost 4:30am here in NY, and my eyes are falling out :confused:

-vhelp

ok, update..

From your: http://www.kvcd.net/forum/viewtopic.php?t=3483 thread,
made my prevous encodes smaller again ie,
* 20 second clip w/ older = 1,460kb
* 20 second clip w/ NEWER(link above) = 1,467kb - - better :)

I think that this one (snipplet above) was better in the details, and a bit
more smoother quality. But, then again, it's late (4:37am) and my eyes
are pretty shot. Sorry. But, it looks great to me so far.

Thanks, and keep up the great work.
-vhelp

@ Kwag..

After all the headackes, and 40+ test encodes later, I'm finally getting their.

made my prevous encodes smaller again ie,
* 20 second clip w/ older = 1,460kb
* 20 second clip w/ NEWER(link above) minue asharp(2,4) = 1,444kb - - better

Since my source seems to be sharp enough, I felt that it didn't need the
added sharpness filter above.

Now, it's really late, and I have ta get ta bed. (4:47am)

Thanks again.
-vhelp

I'm encoding disc 3 now of Dune and the 90 min movie is again taking only 2 hr to encode! I find it unlikely that disc 2 would take significantly longer to encode than the other discs so it looks like the latest change has gone back to the old speed of MA encoding but with full linearity! Of course I haven't been able to SEE any visible difference between all your changes cause frankly, each of my encodes looks amazing on my tv. Well done Kwag! :D

Hey Jwo, i think it has more to do with the Res. n the type of movie you are encoding more so than the adaptive script, but i guess when you r overclocking a Barton 2500+ on an abit board ( that sims like the combo this days) you dont want to go back to the 6 hrs encodes that u had in the past. :wink:

I'm not sure if this is a good this or bad thing, but with this new snippet it only took tok one attempt to predict a CQ value of 70.

Log File:

andybno1,

i think that CQ70 is perfect for your movie cos ToK
found 11,809,622 for desired sample size: 11,838,816 .
less then 0.500 % !
seems ok.

:idea:
do the prediction again,if you got the same result... :wink:

8O

please,someone answer me:

did you see the image(colors) "breathing" in each second
after the changes to "adaptive in the TemporalSmoother filter" :?:

the colors seems that encrease and decrease each second...
like beating!
:?