digitalFAQ.com Forums [Archives]

digitalFAQ.com Forums [Archives] (http://www.digitalfaq.com/archives/)
-   Avisynth Scripting (http://www.digitalfaq.com/archives/avisynth/)
-   -   File prediction experiment #1,000,000 (http://www.digitalfaq.com/archives/avisynth/4430-file-prediction-experiment.html)

GFR 07-11-2003 08:12 AM

Quote:

Originally Posted by kwag
So maybe after refinement, the formula simplifies to:
CompressionFactor = (MovieStacker's Elementary Video Stream / Total Minutes of Movie / MovieStacker's Average Bitrate) / ( Demuxed .m2v file size / Average Bitrate of demuxed.m2v)

And finally:

Sample size = (MovieStacker's Elementary Video Stream / Total Minutes of Movie ) * CompressionFactor :idea:

Edit: I believe this is now correct :!:

-kwag

kwag:

I think you're dealing with a placebo formula :(

size/time=bitrate
size/bitrate=length

so,

MovieStacker's Elementary Video Stream / Total Minutes of Movie = MovieStacker's Average Bitrate (in kilobytes/minute)


MovieStacker's Average Bitrate (in kilobytes/minute) / MovieStacker's Average Bitrate (in kilobits/second) = 60/8.

Demuxed .m2v file size (in kilobytes)/ Average Bitrate of demuxed.m2v (in kilobits/second) = total seconds of Demuxed .m2v/8 =60/8

As total seconds of Demuxed .m2v is 60s (since you chose a 1 min sample)

CompressionFactor = 60/8 / 60/8 = 1.

Any deviance from 1.0000 is because:

1) The length of the movie in minutes is truncated (you discard the final seconds)
2) The length of the sample ain't exactly 60 seconds.

And not because of the compressability of the movie.


Sorry.

Fiz 07-11-2003 09:28 AM

Kwag / GFR,

May be this has been done, suggested and tried before but would another way of getting the compressability of the movie be possible if we did this:

Take a sample with TMPeg with no filters or MA (just resize) using a base CQ of 70.

Then doing the sample again with TMPeg and CQ 70 using the final script (with the filters + MA). The compressability could be defined as

Sample1 Size (only resize) / Sample2 Size (Full MA script with resize)

We then use this factor in Tok to get the correct CQ? I know the CQ factor is not linear and it sounds too simplistic (not for my simple brain though :!: )

I hear what you say GFR and it makes sense, though you and kwag seem far more in tune with this stuff. I'm just bouncing some ideas so please feel free to laugh at my kindergarten suggestion. :wink:

Fiz 07-11-2003 10:34 AM

Ignore me... :imstupid:

My "theory" has no basis. My test results on a 2.5 hour NTSC rip:

Code:

Sample1 = 352x240 no filters, no MA. Use Sampler(length=24)
Sample2 = 352x240 filters + MA.  Use Sampler(length=24)

CQ 70
=====
Sample 1 = 9,622.253Kbs
Sample 2 = 9,303.696Kbs

Factor = 9,622.253Kbs / 9,303.696Kbs = 1.0342398

CQ 50
=====
Sample 1 = 7,178.153Kbs
Sample 2 = 7,060.089Kbs

Factor = 7,178.153Kbs / 7,060.089Kbs = 1.0167227

Guess I forgot the non linear behaviour of CQ.

Abond 07-12-2003 10:58 AM

Hmm, I have a feeling :D we should orient to the average bitrate. If the average bitrate of encoded sample = average bitrate reported for full video file that is the CQ value.

kwag 07-12-2003 11:13 AM

I'm heavily working on this issue ... :cool:

-kwag

kwag 07-17-2003 09:12 PM

Hi to all,

After doing several tests on file prediction for the past 4 days, I believe I found the problems related to the innacurate prediction.
It's all related to the dynamic range of the encoder, when using a wide range as we normally do ( 300 MIN, 2,500 MAX )
Right now, with KVCDx3 528x480(576), there's really no need to go beyond 2,000Kbps, when using the MA script.
So I have found that using a MAX of 2,000Kbps and a MIN of (averageBitrate - 43%), produces extremely accurate results.
Here are the numbers on 4 different movies:

Code:

Movie              Wanted Samp.  Encoded Samp.  Final Size  Moviestacker Wanted Size

The Boondock Saints    12,046KB      12,075KB      706,475KB      722,779KB
Count of Monte Cristo  11,797KB      11,756KB      703,355KB      707,835KB
K-Pax                  11,905KB      11,928KB      726,282KB      714,303KB
K19                    --------    ---------      685,181KB      699,875KB



So you see, the worst case is about ~16MB, which is really negligible in visual quality.

The method is really simple:
Set your MAX bitrate to 2,000Kbps.
Set your MIN bitrate to Average bitrate (data from Moviestacker) - 43%.
Example: My K-Pax movie average bitrate is 800Kbps. So 800 - 43% = 456Kbps
That's it :!:
Now the encoder's dynamic range will be 2000 - 456 = 1,544Kbps, instead of 2,500 - 300 = 2,200Kbps, and the results are far more accurate.
This is probably the reason why the MainConcept encoder is very poor when doing one pass encoding on resolutions greater than 352x480(576), because it's dynamic range is very narow ( way smaller than TMPEG ), but at 352x240(288) it works extremely well with a bitrate range of ~300 to ~1,500.
The file prediction data above was done manually with "Sampler()" at the end of the script. I didn't use ToK, because I wanted as consistent results as I could, so I did them with the manual method. ToK should behave the same with the changes too. Maybe now the "Fast" prediction will be as accurate as the "Full" prediction :idea:
So that has to be tested.
So there you have it :) I hope this works as good for PAL people, as it works for us here in NTSC land :)

----------------------------
@ovg64,
Thanks for your tests ;)
----------------------------

-kwag

kwag 07-17-2003 10:42 PM

And here are the results of the same 4 movies, but using ToK to calculate CQ in "Fast" mode with the new method described above:

Code:

The Boondock Saints:  Manual CQ: 63.2  ToK CQ: 63.713
Count of Monte Cristo: Manual CQ: 65.9  ToK CQ: 64.642
K-Pax:                Manual CQ: 63.9  ToK CQ: 62.661
K19:                  Manual CQ: 62.6  ToK CQ: 63.313

All calculations were done using: one sample per minute, 24 frames per GOP (Length of sample (Auto)), factor of 1.0 and speedup of 10 (Tenra's "Fast" method)

-kwag

J-Wo 07-17-2003 11:18 PM

Just wondering... I've never looked for average bitrate in MStacker before, but I just checked out a couple of my existing d2v files and most are around 1800-2200 kbps. So why is yours for K-Pax only 800??? Am I looking in the right place?

kwag 07-17-2003 11:24 PM

Quote:

Originally Posted by J-Wo
Just wondering... I've never looked for average bitrate in MStacker before, but I just checked out a couple of my existing d2v files and most are around 1800-2200 kbps. So why is yours for K-Pax only 800??? Am I looking in the right place?

My K-Pax movie is ~120 minutes. When I load the .d2v in MovieStacker, the suggested average bitrate is exactly 800Kbps. That's for a target of one 80 minute CD-R.
You probably have MovieStacker set to 2 CD's ;)
Quote:


Oh and after we finish with predicting, do you change your min/max bitrate back to 300 and 2500 or leave them as calculated?
You leave the MIN set exactly as you did with prediction. That is, -43% of average bitrate. And leave the MAX at 2,000.

-kwag

ovg64 07-17-2003 11:30 PM

Hi jwo 800 is the average now take 800 - 43% = 456Kbps and that is the min. you are going to use, 2000 is the max. and you ll encode with those numbers too.

vico1 07-18-2003 12:01 AM

@Kwag,
Your prediction results look pretty good.

I`m wondering if we can get away with that low of a max bitrate,
on Mpeg2 vids... :?: (visual quality wise)

MMMMM...more tests...more tests... :)



*******************************
The Devil`s always.....in the Details!

kwag 07-18-2003 12:06 AM

Quote:

Originally Posted by vico1

I`m wondering if we can get away with that low of a max bitrate,
on Mpeg2 vids... :?: (visual quality wise)

With the MA script, yes :)

-kwag

vico1 07-18-2003 12:30 AM

Quote:

Originally Posted by kwag
Quote:

Originally Posted by vico1

I`m wondering if we can get away with that low of a max bitrate,
on Mpeg2 vids... :?: (visual quality wise)

With the MA script, yes :)

-kwag


http://www.digitalfaq.com/archives/error.gif


*******************************
The Devil`s always.....in the Details!

kwag 07-18-2003 12:31 AM

It's beer time :mrgreen:
http://www.kvcd.net/boon-action.mpg

Edit: The sample is 1,115Kbps average bitrate :mrgreen:

-kwag

jorel 07-18-2003 12:50 AM

Quote:

Originally Posted by kwag
It's beer time :mrgreen:
http://www.kvcd.net/boon-action.mpg

Edit: The sample is 1,115Kbps average bitrate :mrgreen:

-kwag

means:
downstairs to the kitchen again!
:lol:

kwag 07-18-2003 12:54 AM

Quote:

Originally Posted by jorel

means:
downstairs to the kitchen again!
:lol:

I'm on my 4'th trip :mrgreen:

audioslave 07-18-2003 03:26 AM

Wow! You guys really are amazing! :D
Could someone please post a screenshot of where to find the average bitrate in MovieStacker? And how to set up MovieStacker correctly?
I've never really used this software before... :oops:
Thanks!

jorel 07-18-2003 06:39 AM

audioslave,
:)

without screenshot, it's easy!

run MovieStacker and if you are using avisynth2.5x
just click "ignore" in the warning advice.

:arrow: in the "Resize & Video Stream" tab,
in "Source" under "MPEG Resizing" frame,
load your project file(*.d2v or *.avi).

:arrow: the average bitrate(kbps) is under
"Video Stream" frame in the bottom of this tab!

:wink:

audioslave 07-18-2003 07:48 AM

Of course! Thank you very much for you help jorel! :D
Easy and simple. Like I said, I haven't been using MovieStacker that much before... :wink:

bigggt 07-18-2003 11:42 AM

HI,does anyone know if this works with avi's because when i load them into moviestacker the average bitrate is like 1000-1300 which in turn would make the min bitrate around 600 or so.Is 600 min going to make the encode like like crap.

Thanx

kwag 07-18-2003 11:51 AM

Quote:

Originally Posted by bigggt
HI,does anyone know if this works with avi's because when i load them into moviestacker the average bitrate is like 1000-1300 which in turn would make the min bitrate around 600 or so.Is 600 min going to make the encode like like crap.

Thanx

It should work the same with AVIs. Same principles apply.

-kwag

break 07-18-2003 06:06 PM

It seems not to work with PAL system. :cry:
I try prediction with "The Matrix" for one CD at 704x576 (I have recently encoded it at that resolution with CQ 56):
- average bitrate 755 (cutting end credits)
- set min=430 / max 2000 (using ToK 0.0.5.2, Avisynth 2.5x and last script)
- factor for prediction 1,02 (overburning)
- new faster prediction (Tenra) with precision 0,5%
- lenght of sample 24 / nr. of sample 1
- audio 128 and GOP 25

Ended prediction with factor of apprx. 34... :oops:

Any idea? 8O

I rest with my method, working for me (PAL): standard setting in ToK (v0.0.5.2), but with nr. of sample set to "2", GOP 25 and lenght of sample 24 (works also well with fast prediction). :wink:

kwag 07-18-2003 06:24 PM

Quote:

Originally Posted by break

Any idea? 8O

Yes. Do a manual prediction with "Sampler()" ;)

-kwag

break 07-19-2003 07:18 AM

kwag wrote:
Quote:

Yes. Do a manual prediction with "Sampler()"

-kwag
With manual prediction:
frames 186314
time 02:04:13
PAL 25 frames
186314/124/25 = 60,10
800 MB - 115 MB (audio) = 685 MB (for video)
685/60,10 = 11,39 MB (size for sample)

found CQ 57,3 (at 11,4 MB) :P

Well, but now the most important thing: encoding! :D

Thanks kwag, as soon as possible I post my result... :!:

EDIT:
Well, encoding finished, final file size is 686 MB! 8O

-break

vhelp 07-19-2003 11:27 AM

sampler prediction "script" samples..
 
Hello peoples :)

Does anyone have any good .AVS samples of the "sampler" for the
manual prediction method ??

All I have (after a careful search) is what I got (below) but I know there
are some missing pieces before and after hte Sampler() ...
Note, I'm not using MStacker (nor do I have it installed) I'm simply
trying to understand the "prediction" process :)

Code:

Sampler(length=24)
Thank you all,
-vhelp

break 07-19-2003 02:50 PM

Re: sampler prediction "script" samples..
 
Quote:

Originally Posted by vhelp
Hello peoples :)

Does anyone have any good .AVS samples of the "sampler" for the
manual prediction method ??

All I have (after a careful search) is what I got (below) but I know there
are some missing pieces before and after hte Sampler() ...
Note, I'm not using MStacker (nor do I have it installed) I'm simply
trying to understand the "prediction" process :)

Code:

Sampler(length=24)
Thank you all,
-vhelp

The only thing I do, is to add as last line to my AVS script the command "Sampler(lenght=24).
Then I launch TMPGEnc to create sample video, after I found exact size I want, comment last line (sampler command) with "#", and encode the full movie.
I remember you the great Kwag guide for understand manual prediction
here: http://www.kvcd.net/forum/viewtopic.php?t=2102

I hope this can help you. :wink:

-break

vhelp 07-19-2003 02:55 PM

hi break..

Thanks for your response. Actaully, I have read that link - many times, and
am still a bit dazed at it. But, in any case, I don't think it is used any longer,
per Kwag's recommend (elsewheres) where he (in so many words)
said, this is the new prefered method or something like that. I can't remember.

But, I read that that link has some issues w/ which page has the actual
how-to preduct process.

Sooner or later, I'm bound to get it. At the moment, I'm resarching one
too many things :)

Thanks again pal,
-vhelp


All times are GMT -5. The time now is 04:46 PM  —  vBulletin © Jelsoft Enterprises Ltd

Site design, images and content © 2002-2024 The Digital FAQ, www.digitalFAQ.com
Forum Software by vBulletin · Copyright © 2024 Jelsoft Enterprises Ltd.