Predicting File Size for VBR .....
 

Hi All,

Here's an interesting script that I have tried and the margin of error
for predicting file size was approx. 5%. I made some minor changes.
AM1=72 (23.976 x 3 sec. ~ 72), AM2=(23.976 x 42 sec ~1007),etc.
Here's the link:
http://www.vcdhelp.com/forum/userguides/114551.php
With headac3he I can predict the audio file size from the extracted
.wav. The test runs 10 to 15 minutes depending on your cpu. 5% is
very good, but I'd like to reduce this further. :D

-black prince

RE: TMPGEnc VBR. Accurate file size prediction.
 

Thanks black prince,
I'm glad you found this script interesting. I wrote this script with two
objectives in mind. To predict file size, and to test various filters throughout
the movie. Since writing this script, I have tried to optimise it further.

Taking smaller samples at shorter intervals, may be more accurate.

Note (1a):
Delete my old script, and use my new one instead.
(See new script and new instructions below) :D



TMPGEnc VBR. Accurate file size prediction.

Most people encode a sample of a movie to ascertain
visual quality and to estimate final file size.
The problem is where to take this sample from, taking
only one sample does not give a good overall view of the
movie and does not give reliable size prediction.


Simple explanation of the script:
The script takes a x second sample every
x second of the movie, then splices them together.

Brief, how to:
(a) Use DVD2AVI and FitCD to get your basic script.

(b) Edit, copy and past your script and mine,(See note 1.)
save as whatever.avs.

(c) Open the avs file in TMPGEnc.

(d) Check and view the source range in TMPGEnc, hit the
default button, move to end frame, adjust the slider
to an even minute mark, then set end frame.(See note 2.)

(e) Set TMPGEnc to encode - System (video only).

(f) Encode test file.

(g) Divide, encoded test file size by source range time,
then multiply by full movie time.

(h) Use a data rate calculator to establish audio only file size.

(i) Add calculated video and audio files sizes together for full
movie file size. (See note 3.)

Notes:
1. Anything after a # is ignored by the program.

2. Occasionally you may have to adjust the IL variable,
save avs file, then re-view source range in TMPGEnc.

3. On an average, the final full movie file size is 5%
smaller then estimated.

4. SelectRangeEvery() is a function of mpeg2dec.dll.


Any comments, questions or suggestions are welcomed.
Please feel free to test, modify and post your results.


# Start of Script
#
# Paste your script in here.
LoadPlugin("D:\Program files\CDR\MPEG2DEC.dll")
Mpeg2Source("D:\Trains_Plains\movie.d2v")
Crop(11,0,704,576)
TemporalSmoother(2,2)
BicubicResize(320,224,0.333,0.333,0,0,704,576)
AddBorders(16,32,16,32)
FadeOut(75)
#
# IL Depending on length of movie, you may have to adjust this.
# SL See later posts on this forum for details.
#
IL = 710 # interval length in frames.
SL = 49 # sample length in frames.
SelectRangeEvery(IL,SL)
#
# End of script.

Tip for the process above:

IL = Total Frames of movie / 100
SL = 24 for NTCSFilm, 30 for NTSC, 25 for PAL

So now we have a "granularity" of 100 samples. That is, every movie will be sampled 100 times, with a total of 100 seconds per movie. That granularity level should be appropiate for just about any movie.
Updated:
Now divide total film time ( in seconds ) by 100, and multiply the resulting mpeg file size times this number, and that would be your file size for the video stream of the complete movie. Adjust your CQ_VBR or CQ value accordingly.

Formula: ~Total MPEG size = (Film play time in seconds / 100) * MPEG sample file size
This should provide around +- 2% accuracy :wink:

Thanks for the original script Ozzi :wink:

-kwag

Script update: TMPGEnc VBR. Accurate file size prediction.
 

Brief, how to:
(a) Use DVD2AVI to generate your d2v file.

(b) Open the d2v file in FitCD, note the amount of frames,
transfer this to the FC variable in my script, then hit
save in FitCD to get your basic script.

(c) Edit, copy and past your script and mine,(See note 1.)
save as whatever.avs.

(d) Open the avs file in TMPGEnc.

(e) Check and view the source range in TMPGEnc, hit the
default button, move to end frame, adjust the slider
to an even minute mark, then set end frame.

(f) Set TMPGEnc to encode - System (video only).

(g) Encode test file.

(h) Divide, encoded test file size by source range time,
then multiply by full movie time.

(i) Use a data rate calculator to establish audio only file size.

(j) Add calculated video and audio files sizes together for full
movie file size. (See note 2.)


Notes:
1. Anything after a # is ignored by the program.

2. On an average, the final full movie file size is 5%
smaller then estimated.

3. SelectRangeEvery() is a function of mpeg2dec.dll.


Any comments, questions or suggestions are welcomed.
Please feel free to test, modify and post your results.


# Start of Script
#
# Paste your script in here.
LoadPlugin("D:\Program files\CDR\MPEG2DEC.dll")
Mpeg2Source("D:\Trains_Plains\movie.d2v")
Crop(11,0,704,576)
TemporalSmoother(2,2)
BicubicResize(320,224,0.333,0.333,0,0,704,576)
AddBorders(16,32,16,32)
FadeOut(75)
#
# FR --Adjust this to your framerate.
# FC --Adjust this to you total frame count.
# SL --See later posts on this forum for details.
#
PT = 5 # higher number=greater accuracy.
FR = 25 # frames per second (PAL).
FC = 125269 # framecount
IL = (FC / FR) /PT # interval length in frames.
SL = 51 # sample length in frames.
SelectRangeEvery(IL,SL)
#
# End of script. :)

Hi Kwag,

Tried your formula for predicting file size, but the results don’t
seem right. :?

Movie is Omega Code2, full length is 106 min 26 sec (6386 seconds),
IL = total frames of movie / 100 = 152775 / 100 = 152778, SL = 24
For NTCSFilm. The sample file size created is 26,838,046.

~Total MPEG size = (6386 / 100) * 26,838,046 = 1,713,877,618 (1.7GB)

Actual encoded file size is 1,366,330,588 (1.4GB)

Where did I go wrong. :roll: Ozzie’s method came a whole closer to the
Actual file size.

-black prince

Hey Ozzi,

Great predicting script, I've been trying it since your first post.
It gives a pretty accurate predict size.
I tried it in 3 films, a disney cartoon, a music show, and a movie.
All tree gave me a 2% smaller file. Great! :lol:

You have an error in your math :wink:
Your IL should be IL = (total frames of movie / 100) = 152775 / 100 = 1527
So IL=1527

I just finished KPAX using the LBR, and the formula above. Predicted file size was 668,644KB. Actual final file size 630,464KB :wink:
Now I'm encoding it again with the KVCDx3. Done 50%, and size is right on target too :D

-kwag

Hi Kwag,

oops, I was so excited I over looked that mistake :oops:

Thanks :D

-black prince

@Hi Again,

Ozzi's avs calc filesize VBR is the greatest. Give him an award! :)
This time I got it right!!! :wink: The interesting thing was increasing
CQ from 25 to 30 did not increase the file size as great
as increasing from lower CQ. Seem the higher the cq the file
increase keeps getting less. :D Tried CQ = 30 and still had room.

GENERAL

Movie Title...................The Omega Code2
Length (seconds)..........6386

DVD2AVI
Aspect Ratio..................Full Screen (4:3)
Frame Rate...................29.970fps
Video Type....................Film 86%
Frame Type...................Progressive/Interlaced
Field Operation...............None
Audio File.......................None

AVISYNTH
telecide().decimate(5)......de-interlace
BilinearResize..................(496,448,6,0,708,4 80)
TemporalSmoother ..........(2,2)
AddBorders.....................(16,16,16,16)

TMPGENC
Template..........................KVCDx3-MPEG1
cq_vbr..............................25

TEST
Test Length (seconds)..........100
Test Movie File Size..............21,027,309
Est. Movie File Size...............1,342,803,953
Act. Movie File Size...............1,366,330,588
Diff. of Act vs Est...................23,526,635
Diff in Percent........................2%
CD Size (800MB)....................1,600,000,000
Est Space for Audio.................257,196,047

HEADAC3HE
Audio Size (224kbps)..............175,000,000
Audio Size (192kbps)..............149,000,000
Audio Size (160kbps)..............124,000,000
Audio Size (128kbps)..............100,000,000
Audio Size (112kbps)..............87,000,000
Audio Size (96kbps)................75,000,000
Audio Size (80kbps)................62,000,000
Audio Size (64kbps)................50,000,000

-black prince 8)

Thanks,
black prince, kwag and muaddib for your comments.
I'm pleased to read that you are having success with this script.
Nothing gets me more motivated than a challenge.
After being told, on another forum, that it is impossible to predict
file size with any degree of accuracy, I just had to try.
I have limited mathematic knowledge; I failed maths at school,
I could not get my head around, how a letter could be a number.- :?
Therefore, my method is based on an idea and lots of trial and error.
I'm sure that the script can be tweaked even further.
Give it a go. Read the AviSynth documentation, write a script and run it.
TMPGEnc will let you know if there is a problem and where it is.

Good luck.
Ozzie.

In seconds
 

When you dived bt time is this in total seconds (i.e 1 min =60 seconds) and where can you get a bit rate calc. I am ussing 224 sound. Oh and sometimes my GOP is blan (0) when I open up templates. And you are using CQ_VBR right ?

Hi Ozzi,
I wonder where that "other forum" was :lol:
That reminds me when "some other forum" people would tell me that it was IMPOSSIBLE to get 120 minutes of video with "VIEWABLE" quality :lol:
Now we get 180 minutes with good quality 8)

Now if I just could get the "ClipProperties" to work with this script, we could get the number of frames and frame rate automatically in the script, without having to edit. But sadly, I can't get it to work with "mpeg2source". I guess it works with "AviSource" :cry:
More to investigate..... :wink:
Keep up your great work :D

-kwag

Hi kwag,

Updated the script again.
The script now automatically gets number of frames.
It uses the AviSynth function, Framecount. :D

Still working on frame rate part.
There is a AviSynth function, Framerate,
just can't seem to make it work, :( -it's probably something simple.

#Update 1 oct 02:
#
LoadPlugin("D:\CDR\AviSynth2\plugins\MPEG2DEC2.dll ")
mpeg2source("D:\Trains_Plains\movie.d2v")
Crop(11,0,704,576)
TemporalSmoother(2,2)
BilinearResize(320,416,0,0,704,576)
AddBorders(16,80,16,80)
#
# PT - Adjust this from 5 for a 80min movie, to 10 for a 3 hour movie.
# FR - Adjust this to your framerate.
# SL - Should be one frame greater than GOP length.
#
PT = 5 # theoretically a higher number=greater accuracy.
FR = 25 # frames per second (PAL).
IL = (Framecount / FR) /PT # interval length.
SL = 51 # sample length in frames.
SelectRangeEvery(IL,SL)
#
# End of script.

Hi Ozzie,

I couldn't make Framerate or Framecount work 8O
At least with AviSynth 2.06, the current version.
My script was to read:

IL = Framecount / 100
SL = Framerate
SelectRangeEvery(IL,SL)

That's all!
Then:
MPEG size = (Film play time in seconds / 100) * MPEG sample file size
Or:
MPEG size = ( (Framecount / Framerate) / 100) * MPEG sample file size

-kwag

UPDATE: I got it working, and fully automatic.
Add this to the end of your script:

IL = Framecount / 100 # interval length in frames.
SL = round(Framerate) # sample length in frames.
SelectRangeEvery(IL,SL)

The problem was that Framerate is a floating point value ex: 23.976, 29.97, etc. and it doesn't work in the "SelectRangeEvery()" function. So the function "round()" rounds the value. So 23.976 becomes 24, 29.97 becomes 30, etc. The round() function returns the nearest integer to the number parameter.



Enjoy :wink:
-kwag

Changes: Please use round() function. I edited this post. I was using the ceil() function before.

Do you use the same How to for this script ?
And how do you calculate the audio file ?

Hi LadyMiles,

It's simple.
Say your target CD-R is 800MB. You open your AC3 audio file with Headac3he, which you have already "demuxed" with DVD2AVI.
Select 128Kbps in Headac3he for this example, and note the file size that will be created. If you change the audio destination bit rate, you'll see that the file size will change too. So Headac3he tells you immediately what the final audio file size will be. Then subtract this size from 800, and that would be your MPEG video stream target.

-kwag

Way to go Kwag,

This is getting better and better. :D The script to predict file size
is fantastic. I can now make better decisions about which
KVCD templates to use, whether to encode credits at a lower
CQ or cut credits or change movie CQ or change KVCD templates, etc.
Before I had to make crude guesses and re-do encodes. Not any more. :wink:

Great work Ozzi and Kwag

-black prince :D :D :D :D

Hey kwag
Are there any special instructions for using your avs script ?
i.e.
1) fitcd paste script
2)load into tmepg
3)source range to one minute ?
2) Set TMPGEnc to encode - System (video only).
3) Encode test file.
4) Divide, encoded test file size by source range time,
then multiply by full movie time.

Right on the target, black prince!
You take the words out of my mouth... :lol:

:!: Great work Ozzi and Kwag :!:

Thanks black prince,

But greater thanks must go to Ozzie :lol:
I had been trying to find a method to do this for over a month now, and I didn't know that the "SelectRangeEvery( X, Y ) function existed 8O
I was trying to find out the structure of a TMPEG project file, so that I could generate a .prj file with to "100" granularity sampling that I had in mind. So when Ozzie came out with the script, this immediately popped into my head :lol:
Yes, you're right, it works super good!. I just did my first 2 hour film with the KVCDx3 :lol: and it looks damn good 8O
The file size came out 20MB smaller than the predicted file size ( good insurance 8) ) with the formula above to calculate the file size with an audio of 192Kbps. It looks and sound excelent :lol: So no more worries from now on thinking if it will fit or not. Also, to Ozzie, I must say this: You just killed multipass encoding :wink:

Enjoy, play, and encode :lol:
-kwag

This file size prediction script is quite awesome! :D My latest encode came out to 70 mb less than the sample predicted so I raised the audio bitrate all the way up. The 90 minute full screen 30fps movie I did came out to 530 mb with the LBR template and it looks great!

I was also wondering if you want the file size a little bigger and better quality do you raise the cq and max bitrate or just the cq?

thanks,
ren

You are right!
No more need to multipass encode to predict the file size!
Just great :!: :lol:

Let me just ask one thing.
If I understand correctly, with kwag's version you are taking 100 samples of one second each. This will give you a total of 100 seconds sample.
Is that enough to get good accuracy to a 3hour movie?
Lets say I want to increase the accuracy, what would be better... more samples or more sec/samples.

These two examples will give a total 5 minutes (300sec) of "predict sample".
1) 150 samples of 2 seconds each:
IL = Framecount / 150
SL = round(Framerate*2)

2) 300 samples of 1 second each:
IL = Framecount / 300
SL = round(Framerate)

Then, MPEG size = (Film play time in seconds / 300) * MPEG sample file size

Witch one is best for accuracy?
Or better, will I get an accuracy increase doing that?

Thanks,

@rendalunit,

Increase your CQ value only. If you increase your MAX and CQ at the same time, the internal CQ encoding algorithm scale changes. So it's better to maintain MIN and MAX constant, and adjust the CQ value until you get the mpeg size you want.


@muaddib,

That's what I mean for "granularity". It's better to take many small samples than less large samples. 100 one second snapshots are enough, and makes encoding the test strip very fast. 100 samples for a 2 hour movie will give you a very accurate prediction. If your movie is 3 hours or more, increase to 150 samples. That should just about cover all your bases :wink:

Later!,
-kwag

Any special instructions
 

Are there any special instructions for using your avs script ?
i.e.
1) fitcd paste script
2)load into tmepg
3)source range to one minute ?
2) Set TMPGEnc to encode - System (video only).
3) Encode test file.
4) Divide, encoded test file size by source range time,
then multiply by full movie time.

Re: Any special instructions
 

No. Range is already selected by SelectRangeEvery(IL,SL)
Try this:
Final MPEG size = (Film play time in seconds / 100) * MPEG sample file size
Or:
Final MPEG size = ( ( Total frames / Framerate) / 100 ) * MPEG sample file size

-kwag

Dear Kwag
Thank you for being so patient with me. Just one more question fitcd seems to show the wrong time length. But when I loda the d2v in Tmpeg it shows the correct length. Is this a bug in fot cd or is there somethingwrong with the d2v

hey Kwag,

can you use this script when frameserving with Mediator? I tried to do this and it encoded only the first 5 seconds of the movie then ended. 8O

-ren

Hey Kwag
I think you template is great and you are so helpful and patient with people
If you can tell me how to calculate an audio file based on an mpa or an mpeg2. I will be gald to use visual basics to create a little program where people can input the info your script produces.
I.e.
input time
Input sample file size
Input audio file size
Output estimated file size

Hey LadyMiles,

No need to write a VB program to calculate audio file size prediction.
We use Headac3he to input (wav, mpa, ac3, etc.) and choose
Destination File as .mp2. Click twice on OPTIONS button, select
dual channel and as you change audio bitrate from 224, 192, etc.
you can see the Destination file size change in MB's. It's fairly
accurate. 8)

-black prince

Haven't tried that ren 8O
I'll let you know :roll:

-kwag

Hey Kwag,

I was saying I would write a VBS for predicting file size. where people could input the sample file size that your script would create and the audio file size instead of using a calculator

Bascially a VBS just doing what you told me
Final MPEG size = (Film play time in seconds / 100) * MPEG sample file size

Hi LadyMiles,

As black prince said, there's no need, because Headac3he already has an automatic calculator. When you open a AC3, WAV, etc file, it automatically tells you the file size for the audio, depending on the bit rate you choose. So you can dynamically play with the destination bit rate, and immediately see the file size that will be created.

-kwag

filesize prediction
 

hi there,

I've tried this (or similar) methode several
times b4 to predict the filesize based on
extrapolation of a sample (independant of
the scripts).

The majority of the predictions did not
succeed :-(
Errors of up to 20% came up.

My explanation/guess:
the scenery of a movie varies pretty
much along the film. It all depends on
a similar mix of action scenes compared
to the sample sequence. You can't
expect this to be the same mix. So the
results of a one-sample-extrapolation
might be good but it isn't necessarily
good.

A better way could be a strategy that
takes different samples along the film
(say 30 sec samples every 5 or 10 min).
This average might bring up a better
estimation of the final result but I
couldn't find a way to automatize this.

Obrigado

Re: filesize prediction
 

Hi obrigado,

IL = round(Framerate*600) # interval length = 10min.
SL = round(Framerate*30) # sample length = 30seg.
SelectRangeEvery(IL,SL)

The script above takes 30 sec samples every 10 min.
But I think that what kwag say about the 100 sample granularity is a better way.
It's giving me great results, about 2 or 3% error. And with a smaller file size. :lol:
If you want a better accuracy, you can just increase the granularity (as kwag said). You can doble it (200samples / 1sec each) and still have a small prediction file size.

Here's an update on the formula:

Final MPEG size = (Film play time in seconds / 100) * (MPEG sample file size * .95)

Or:

Final MPEG size = ( ( Total frames / Framerate) / 100 ) * (MPEG sample file size * .95)

I'm getting consistent result between 2% to 4 % lower than predicted file size every time with every movie. So if you use the above formula, multiplying the resulting mpeg sample by .95, your estimation will be almost the final file size. The .95 value is just a little insurance margin :wink:

-kwag

Hi All,

I decided to try predicting file size using KVCD-LBR for the same movie,
Omega Code2 (106 minutes, 26 seconds). Normally I would use a CQ of
30 recommended by Kwag prior to testing file size. So, I tested CQ’s from
30 to 65 in increments of 5. The test allowed me to use CQ to 50 and
still have room for audio file of 100MB at 128Kbps. The final A/V file
is ~ 783MB. The picture quality is excellent. The actual video file size
is 689,983,797. This is –5% smaller estimated size I chose at CQ 50.
Check out my stats below:

CQ…………….Test File Size…………….Est. Total File Size………
30……………..10,861,976……………….693,645,787……
35……………..11,225,219……………….716,842,485……
40……………..11,345,128……………….724,499,874……
45……………..11,375,721……………….726,453,543……
50……………..11,378,161……………….726,609,361……
55……………..11,378,161……………….726,609,361……
60……………..11,378,161……………….726,609,361……
65……………..11,378,161……………….726,609,361……

Note: from CQ = 50 to 65 the file size did not change in my tests.
I chose CQ=50 because there was no difference in picture quality.
The difference between total file size previous and present
resulted in a dramatically smaller difference for each pair.
(i.e. 716,842,485 – 693,645,787 = 2,319,670MB)

This process helped to use the maximum space on my CD (800MB)
with higher quality video and even audio. I will adjust my total
file size formula to Kwag's revised computations. 8)

-black prince

Hey Kwag,

Is it possible to test MT in bold below and check the total movie
time "MT = round(Framecount / round(Framerate)) " for say 10800
seconds or 3 hours and select a value of 150 else 100. :?:
This assumes movies 3 hours or more would use "Framecount / 150
else 100". Will avisynth allow condition tests, say if (movie time) >= 10800
seconds then 150 else 100? I tried, but couldn't find any refrence.

LoadPlugin("E:\DVD Backup\2 - DVD2SVCD\MPEG2DEC\MPEG2DEC2.dll")
LoadPlugin("E:\DVD Backup\2 - DVD2SVCD\INVERS~1\DONALD~1\DECOMB.DLL")
Mpeg2Source("D:\Temp\Omega_Code2.d2v")
Telecide()
Decimate(5)
#BilinearResize(496,448,6,0,708,480)
#TemporalSmoother(2,2)
#AddBorders(16,16,16,16)
#
BilinearResize(336,224,0,0,720,480)
TemporalSmoother(2,2)
AddBorders(8,8,8,8 )
#
#MT = round(Framecount / round(Framerate)) # movie time in seconds
#IL = Framecount / 150 # If movie is 3 hours or more
IL = Framecount / 100 # If movie is less than 3 hours
SL = round(Framerate) # 24 for NTCSFilm, 30 for NTCS, 25 for PAL
#
SelectRangeEvery(IL,SL)
#
# End of script.

-black prince

Hi black prince,

I'm not sure about if/else/then conditions in AviSynth. I'll have to check the reference manual. It does allow user defined functions ( a la C Language ). I'll have to check on that.

But I have something to report :lol:
I just finished encoding "The Green Mile" with the LBR for testing the formula and the script ( 100 samples ), and I am baffled at the results. 8O
Using the formula "Final MPEG size = ( ( Total frames / Framerate) / 100 ) * (MPEG sample file size * .95)", the predicted file size target was 643,377KB. The actual file size after encoding is 639,352KB :lol: So it's only a difference of 4.025MB in a 3 hour movie.
Now I'll say it again" The hell with multipass encoding" :lol:
And that would give the formula a -0.62% accuracy, right :?: 8)

-kwag

Hey Kwag,

What CQ did you use for "Green Mile" KVCD-LBR? You
have recommended 20 for 3+ hours movies in the past.
What did your tests tell you about increasing CQ?. 8)

-black prince