digitalFAQ.com Forums [Archives]

digitalFAQ.com Forums [Archives] (http://www.digitalfaq.com/archives/)
-   Avisynth Scripting (http://www.digitalfaq.com/archives/avisynth/)
-   -   Time to revise file prediction! (http://www.digitalfaq.com/archives/avisynth/4101-time-revise-file.html)

kwag 06-22-2003 06:39 PM

@All,

Well here I go again.
Hopefully we start zeroing in on this inconsistency.
I just finished encoding "The Boondock Saints" with the following changes:

(1) Turned ON Scene change detect on TMPEG. Yes :!:, it seems to be working, and for some "I_DON'T_KNOW_WHY_REASON" the file size is smaller 8O with the option turned on. Don't ask me why! Seems TMPEG is playing tricks on us 8)
Apparently, it optimizes the compression much better with variable GOP, which is the case with this option turned on. Yep, you heard it right: The file size is smaller with scene change detection on. Maybe one reason for the erratic prediction :idea: BTW, I'm using TMPGEnc 2.513.

(2) Changed the GOP back to 24, because 12 was just as unpredictable as 24. So I'll leave it at 24, for a smoother picture, plus now it's variable anyway with the change described above.

(3) Enabled the scene change detection in ToK's "Video.en1" file. Here it is, so you can copy it to your OPT directory: http://www.kvcd.net/video.en1

(4) As posted on the current optimal script page, I removed the line:
conditionalfilter( last, asharp(-1,0), last, "nf", ">", "scd_trigger" )
Read the change log for the explanation.

So here are my "The Boondock Saints" results this time:

Wanted video size: 714,303.19
Final encoded video size: 690.997 :)
For a difrerence of 3.3%, which I can tolerate as long as is negative :!:
The CQ was 63.02, and my manual CQ calculation was 63 :!: So ToK is doing the job correctly. This was using first and second pass as full sampler. Not the fast 10% sampler. That will come later. I'm just trying to get this back on a consistent state.
Now back to encode "K-Pax", and see what comes out, as that was the other extreme yesterday.

-kwag

vhelp 06-22-2003 06:45 PM

@ vico1..

>> To bad we can`t use some kinda` "transcode" technology with it...

Well, if we could only just get the "bitrate" accross the time-line, much
like what BV.EXE does, we're in for go, and so much for MA, as we know it.

Yes, I believe that Kwag made mention of these in previous topics
elsewhere's. I've ben wondering about this myself. It's a matter of time
before someone (here or doom9) finds the technique and shares it (we hope) !!

Then, finally we'll be ride of those items that plauged us in past.. some ie's:
* poor quality
* poor predictions
* better CDr/rw - DVDr/rw handling (on a disk per bases)
* anything else to ad here

-vhelp

vhelp 06-22-2003 06:49 PM

.
.

And, as for the MA script, I've ben messing around w/ some of the code,
and I've ben playing w/ the:
* scd_trigger = 60, and
* ScriptClip().. w/ asharp( -(fmin((nf/60), 1)) uses

Note the 60 instead of 30.
I think I've had better results, but I'm still debating the above changes I've
ben toying around with.

One thing I noticed still, is the shifting up/dn of the video on the first Bf.
If you remember my Bf/Df/Af analigy in my previous post.

To better analigies it:
CL....FRAME...Fltr.....STRENGTH...Notes
----------------------------------------------------------
Bf - 69379 - blur - 2 - noChange - person sitting
Df - 69380 - blur - 1 - noChange - person sitting
Af - 69381 - clear - 1 - sceneChange - person is walking
...
Bf - 69806 - blur - 2 - noChange - person sitting
Df - 69807 - clear - 1 - sceneChange - person sitting
Af - 69808 - clear - 1 - noChange - person is walking

Note 1,
The blur/blur/clear, and blur/clear/clear (for Bf/Df/Af)

Note 2,
The above is an example of trying to describe the MA process
a little better, I think. A way to visualize in a descriptive manor.
I guess I would catagorize it as a "tool"

Note 3,
The 2 and 1 are the Strength of the filter being applied. ie, I
noticed that on first frame BF, it would be strong hence the 2,
and on the Df frame, it would sometimes be less strength, or 1.

If the above help some, when working out this MA script :wink:
I hope this "tool" can be useful.

-vhelp

vhelp 06-22-2003 08:49 PM

@ all..

Can we try a different Filter in place of TemporalSoften(2,7,7,3,2)
I'm not liking all that much in my detais when I run it through vdub on
some items. it's smudging the letters too distortedly like.

And, I'm having difficulty adentifying those filters that will work under
YV12 color space. I'm still getting that noticable color washout look when
I compare two vdub's together, I can see one has richer colors. Anyways..

Can someone throw up a few Filters in place of the above ??
(but that will work w/out error showing up in vdub's screen.. YV12 arrr!!)

Thanks.
-vhelp

ovg64 06-22-2003 09:21 PM

Hi Vhelp im currently using the act filter which is a temporal Smoother
n to me it does a pretty good job at smoothing and blending detail, kind of take away the fliks, i use atc(3,6,9,0.5,false) in place of the TemporalSoften(2,7,7,3,2) .

BTW: act is a YV12 filter too

Quote:

MaxTreshold = 1.50
nf = 0 # Current frame.
Mpeg2Source("C:\Documents and Settings\Osvaldo\Desktop\DVD\NS.d2v")
Limiter()
asharp(2,4)
GripCrop(528, 480)
GripSize(resizer="BicubicResize")
undot()
STMedianFilter(8, 32, 0, 0 )
Convolution3D(preset="movieLQ")
atc(3,6,9,0.5,false)
MergeChroma(blur(1.50))
MergeLuma(blur(0.2))
ScriptClip("nf = YDifferenceToNext()"+chr(13)+ "nf > 2.5 ? asharp( -(fmin((nf/30), 1)), 0 ) : \
TemporalSoften(2,7,7,3,2) ")
GripBorders()
LetterBox(0, 0, 16, 16)
Limiter()
function fmin(float f1, float f2) {
return (f1<f2) ? f1 : f2
}


vhelp 06-22-2003 09:30 PM

@ ovg64..

Great. I'll look up that filter in my repository hehe..
Ohoh.. looks like I GOT IT. yippy. I'm gonna try. Just finished one of
my test encodes w/ MA script. Looks good too. Hay, now I'll do a test
compar w/ Temporal(2,7,7,3,2) vs. your act(3,6,9,0.5,false)

Thanks.

Any more suggests ?? i'm gain to give them a shot. I'm having fun now :)
-vhelp

ovg64 06-22-2003 09:40 PM

Yes Vhelp Convolution works like the old conv. and the new version
of STMedianFilter is 0.1.0.1 :wink: they r also YV12.

BTW: who's osv64 :?: :lol:

kwag 06-23-2003 12:11 AM

Here are the results and analisys on my K-Pax encode:

Wanted video file size: 714,303KB
Encoded video file size: 741,735KB

A difference of ~+3.69%

And this is GOOD :!:
Here's the pattern I see. On action movies, file size will be ~3% smaller MAX.
On low action movies (like K-Pax), file size will be ~3% larger MAX.
So normal "average" films should target ~100% accuracy :!:
There's something I did notice, and that is that just before the end credits, the file size was just at about 714MB, which was the wanted size. So the end credits is REALLY what's throwing off the calculations :)
I was watching TMPEG as it was encoding the end credits, and the file size was soaring high on the vertical scroll of the credits. "THAT'S OUR MAIN PROBLEM" The function "YDifferenceToNext()" reports a VERY LOW value on the end credits (verified with Vdub adding "nf" variable from script as subtitle), even though the scroll is a very high difference to next frame :!: But because it's mainly a black screen with text scrolling vertically, the returned value is very small, and we are loosing A LOT of compression there :!:
So my suggestions are as follow:

(1) Cut off end credits with AviSynth's function "Trim", and encode with a factor of 1.0. OR take into consideration the type of movie you're about to encode. If it's a low action film, subtract 3% (multiply by 0.97) the wanted sampler size.
If it's an action film, add 3% (multiply by 1.03) the wanted sampler size.
If it's an average film, don't add or subtract anything :)
(2) If using ToK, the same applies, BUT:
(a) Use a factor of 0.97 for low action movies.
(b) Use a factor of 1.03 for action movies.
(c) Use a factor of 1.0 for regular (average movies)

*** PAL people, could you please test this please ***
Please use the "Video.en1" file I uploaded earlier ( www.kvcd.net/video.en1 ) and copy it to your ToK OPT directory.
*** PLEASE, let us know your results :!: ***


-kwag

audi2honda 06-23-2003 12:56 AM

kwag it's funny you mention the end credits messing up the movie. I've been having fairly accurate results using the old style prediction with a high sample length of 48 and prediction factor of 1.00.

Precition was erratic but kind of on target. Well I allways cut the end credits except for this one movie where I wanted to keep them. That movie has given me so much trouble over the last few days. I've encoded it 4 times lowering the CQ tons and it allways over shoots the mark by several MB. Based on what I've seen in my last few encodes it is pretty much consistant with your results.

kwag 06-23-2003 01:41 AM

Well, now we just need some feedback from PAL land, and see how the results match :)

-kwag

Krassi 06-23-2003 04:21 AM

Quote:

Originally Posted by kwag
Well, now we just need some feedback from PAL land, and see how the results match :)

-kwag

I'm currently @work :roll: .. I'll start the encoding this evening. I think i will encode "The Fast and the Furious". What factor should i use, what do you think?
Mabye another PAL-User can give any feedback? Jorel :D :?: .

EDIT: Encoding "The Score" with factor 1.0.

ovg64 06-23-2003 08:23 AM

Quote:

Originally Posted by kwag
Here are the results and analisys on my K-Pax encode:

Wanted video file size: 714,303KB
Encoded video file size: 741,735KB

A difference of ~+3.69%

And this is GOOD :!:
Here's the pattern I see. On action movies, file size will be ~3% smaller MAX.
On low action movies (like K-Pax), file size will be ~3% larger MAX.
So normal "average" films should target ~100% accuracy :!:
There's something I did notice, and that is that just before the end credits, the file size was just at about 714MB, which was the wanted size. So the end credits is REALLY what's throwing off the calculations :)
I was watching TMPEG as it was encoding the end credits, and the file size was soaring high on the vertical scroll of the credits. "THAT'S OUR MAIN PROBLEM" The function "YDifferenceToNext()" reports a VERY LOW value on the end credits (verified with Vdub adding "nf" variable from script as subtitle), even though the scroll is a very high difference to next frame :!: But because it's mainly a black screen with text scrolling vertically, the returned value is very small :!:
So my suggestions are as follow:

(1) Cut off end credits with AviSynth's function "Trim", and encode with a factor of 1.0. OR take into consideration the type of movie you're about to encode. If it's a low action film, add 6% (multiply by 1.06) the wanted sampler size.
If it's an action film, subtract 6% (multiply by 0.94) the wanted sampler size.
If it's an average film, don't add or subtract anything :)
(2) If using ToK, the same applies, BUT:
(a) Use a factor of 1.06 for low action movies.
(b) Use a factor of 0.94 for action movies.
(c) Use a factor of 1.0 for regular (average movies)

*** PAL people, could you please test this please ***
Please use the "Video.en1" file I uploaded earlier ( www.kvcd.net/video.en1 ) and copy it to your ToK OPT directory.
*** PLEASE, let us know your results :!: ***

-kwag

Well the thing i see here is that you already had rich file size
before the end credits, so again that shows the inability of tok
predicting when the AScript is turning on n off (blur), as for the
for the end credits shooting up the bitrate i seen that even before
the adaptive script while using 2.08 so i not sure kwag but to me that just
the way Tmpeg encodes. Maybe by lowing the max bitrate from 2500 to
2300 stabilize prediction :idea: just a tough.

jorel 06-23-2003 09:06 AM

Quote:

Originally Posted by Krassi
Quote:

Originally Posted by kwag
Well, now we just need some feedback from PAL land, and see how the results match :)

-kwag

I'm currently @work :roll: .. I'll start the encoding this evening. I think i will encode "The Fast and the Furious". What factor should i use, what do you think?
Mabye another PAL-User can give any feedback? Jorel :D :?: .

EDIT: Encoding "The Score" with factor 1.0.

hi Krassi!

in Brasil is pal-m, different from europe pal-g.
then here we encode ntsc-m, the tvs, vk7 and dvds have "dual systems".
ntsc-m is the normal ntsc (like in USA)
the letters "g" and "m" in the final of the system
have a big difference between them,
not only the sizes but all the transmition and reception too.
see, a pal-g tv in Brasil don't work,don't show the image or sound.
the ntsc tv in Brasil stay "black&white" and we need to
make some "transcodifications" as we call,in the color system.
pal and ntsc are the names of the color systems
and "g" and "m" are the "standard" for the remainder signals.
is a big difference.
:wink:

Jellygoose 06-23-2003 09:27 AM

8O
And I always thought PAL=PAL...
Another contribution to my knowledge! Thanks Jorge! :D

GFR 06-23-2003 09:29 AM

As jorel said, pal and ntsc tell how color info is multiplexed into the analog signal. NTSC is one of the first color muxing standards. PAL is more sophisticated and has more stable colors.

the G and M refer to the vertical scanning frequencies, and set the number of lines. As a rule of thumb this is 1/2 the AC power frequency. There are many standards, like M, N, G.

NTSC-M (Japan, USA) is ~30fps. In these countries AC power is 60Hz. You have 480 lines.

In countries where the AC power is 50Hz (most of Europe?) the video standard is 25fps, 576 lines.

In Brazil we have PAL color encoding, but AC power is 60Hz, so we have PAL-M, ~30fps, 480 lines.

So, when you associate PAL=576 lines, NTSC=480 lines, that's not exact.

PS.: It is a hell to find capture cards that support this (since Brazil is the only country that has this standard). The only one I know is ATI AIW, but it works OK with PAL-M broadcast signals but not with prerecorded PAL-M tapes.

PS2: jorel, eu conheci o cara que ajudou a implantar a TV colorida no Brasil, e que definiu o sistema PAL-M. Era o professor Alcione, do IME. MUIIITO fera, mas doidinho :)

Krassi 06-23-2003 09:38 AM

Hi Jorel,
sorry, didn't know about that. Seems to be very complicated. Just saw in a list that Brazil has PAL-Standard :oops: .
However, about ~ 3 hours to go...

jorel 06-23-2003 10:19 AM

yes Krassi,
big complicated.

jell and GFR:
the differences are more than vertical and number of the lines.
i try to explain..if my english help me:

frequences of color transmitions
ntsc-m: 3.579.545
pal-n:3.579.545
pal-m:3.575.561.149
pal-g:4.5...?(don't remember the rest)

the sizes of the "channels bands" and the audio band
is different between "m" and "g".

ntsc have in vector:
zero>the burst signal
+90>the red signal
+180>the blue signal

pal have in vector:
zero>the burst signal
+90>the red signal(one phase)
-90>the red signal(another phase)
180>the blue signal

pal-m is the same like pal-g but
in Brasil we have a circuit called
"pal key" that find what is +90 red and what is -90 red
and turn all red in +90 like in ntsc,without invertion of colors
like sometimes the ntsc turn the red to green and the blue to purple.
this circuit find and turn the lines to the original poinr in vector.

the reference for the colors in vector is the "zero" burst.
in ntsc if the +90 turn to -90,the burst see the red
in 90 degrees and "think" that he is correct in the vector but
the red is 180 degress from the original point and show as green.
than we use the "tint" or "hue" to correct this.
the pal system is to correct the original
-90 red automatically to +90 and turn the system stable.
this is the reason of the name "pal":
Phase Alternating Line!

well this is just only for the color...the tuner band,the audio band,
and lots more things are differents between "m" and "g".

i will that my poor english don't turn it all more confused,
i can't explain better than this.
:oops:

GFR:
"Era o professor Alcione, do IME."
já ouvir falar muito bem dele... :wink:

me pareço bastante com ele na parte do "doidin".
:lol:

GFR 06-23-2003 10:51 AM

>>this is the reason of the name "pal":
Phase Alternating Line!

Or as people say "Peace At Last"! against "Never Twice the Same Color". :)

I forgot the funny acronym for SECAM.

Krassi 06-23-2003 12:47 PM

Quote:

Originally Posted by GFR
Or as people say "Peace At Last"!

Yepp, thats right :lol: ... Here are the results :roll: :
Code:

=============================================================
ToK Log: THE_SCORE.avs
=============================================================
 
Resolution (fps):704x576 (25,000 fps)
Total Frames: 170405
Total Time  : 01:53:36
-------------------------------------------------------------
 
Audio Size: 95.428.389
Required Video Size: 723.771.611
 
Factor: 60,000
Desired Sample Size: 12.062.860
 
-------------------------------------------------------------
New Faster Prediction
-------------------------------------------------------------
 
Full Sample
Next CQ: 60,000. Sample Size: 18.108.725
Small Sample
Next CQ: 60,000. Sample Size: 1.528.880
Predicting...
Next CQ: 39,968. Sample Size: 12.650.243
Next CQ: 29,321. Sample Size: 11.162.712
Next CQ: 35,204. Sample Size: 11.853.128
Next CQ: 37,022. Sample Size: 12.429.518
Next CQ: 35,989. Sample Size: 12.411.949
Next CQ: 35,548. Sample Size: 12.126.407
Next CQ: 35,422. Sample Size: 12.031.024

Exit Condition: 0,500 % reached ! yahoo !
Tries  : 8
 

Final CQ: 35,422
Total Time For Predicition: 00:29:21

 
-------------------------------------------------------------
Encoding F:\Recorder\THE_SCORE.avs
-------------------------------------------------------------
 
Encoding... CQ : 35,422
Final Encoded Size: 691.823.124

Factor was 1.0, New Prediction Method in ToK and 3 Samples/Minute. Credits were cut off.

kwag 06-23-2003 01:34 PM

Quote:

Originally Posted by Krassi

Factor was 1.0, New Prediction Method in ToK and 3 Samples/Minute. Credits were cut off.

3 samples a minute, of how many frames per sample :?:
I'm using one sample per minute of 24 frames each :!:

-kwag


All times are GMT -5. The time now is 12:44 AM  —  vBulletin © Jelsoft Enterprises Ltd

Site design, images and content © 2002-2024 The Digital FAQ, www.digitalFAQ.com
Forum Software by vBulletin · Copyright © 2024 Jelsoft Enterprises Ltd.