File size prediction formula
 

Can I check a couple of things please?

1) The latest file prediction fomula, for use with the Sampler plugin and default parameters, is:

PredictedSize = ((Framecount/TimeInMinutes)/24) * SampleSize * Factor


2) The previous file prediction fomula, for use with SelectRangeEvery 100 x 1-second samples, was:

PredictedSize = ((Framecount/FrameRate)/100) * SampleSize * Factor


Is that right?

/girv

Hi girv,

This is the current formula: MPEG size = ((Total frames/MovieTimeInMinutes)/24) * MPEG sample file size

-kwag

Hold on to your hats.

I think the "old" and "new" formula are just different forms of the same fundamental (and more intutive IMHO) formula. Dust of the algebra books, take a deep breath and follow me...

DEFINITIONS:
Sm: movie size
Ss: sample size
Tm: move time in seconds (= 60*MovieTimeInMinutes)
Ts: sample time in seconds (100 for old method, Tm/60 for new)
Ts: sample time in seconds
FC: frame count
FR: frame rate (frames per second)
F: fudge factor (0.96-ish)

OLD:
Sm = ((FC/FR)/100) * Ss * F

rewriting:
Sm = (FC/(FR * 100)) * Ss * F

now FC = Tm * FR and Ts = 100 so:
Sm = ((Tm * FR)/(FR * Ts)) * Ss * F

cancelling FR:
Sm = (Tm/Ts) * Ss * F

Lets call this equation (1)


NEW:
Sm = ((FC/(Tm/60))/24) * Ss * F

rewriting:
Sm = ((FC * (60/Tm)) / 24) * Ss * F

and again:
Sm = ((60*FC)/(24*Tm)) * Ss * F

now FC = Tm * FR and lets say FR = 24, so:
Sm = ((60 * Tm * FR) / (FR * Tm)) * Ss * F

cancelling Tm and FR:
Sm = 60 * Ss * F

but Tm / Ts = 60 in this case so:
Sm = (Tm / Ts) * Ss * F

...which is equation (1) again.

Notes:
1. I know the "new" formula doesnt include the fudge factor, but in all my tests (using the new GOP of course ;) I still found I needed it just the same as with the "old" formula. Why was it dropped anyway?

2. I know its a bit of a leap of faith to say "FR=24" in step 4 of the "new" proof, but IMHO its more reasonable to say 24 is the framerate rather than the GOP length. Besides, it makes the proof work ;)

3. Could the fudge factor be due to extra I frames being inserted into the sample clip as the "scene" changes very rapidly (> once per second) compared to the movie, thereby artificially inflating the sample file size? Might be useful to switch off "detect scene change" when encoding the sample. Need to test that.


What equation (1) says is: to get the predicted movie size you just scale up the sample file size by the ratio of the sample time to the movie time. So if you sample 1/60 of the movie (one second for every minute, just like Sampler does by default) you just multiply the sample file size by 60 to get the movie size. Seems a lot more intuitive to me.

I've tested this formula on a variety of clips and the results are identical
to the "old" and "new" ones, which would lead me to believe I'm right :) I'd appreciate it if the good folk of kvcd.net could do some independent testing :) Different framerates and GOP lengths would be important to test (see note 2 above) as would low and high action clips. Also cast your eye over the math above and check I've got it right in the first place!

So what do you think?

girv,

I think that the old and new formulas have to be different. The old formula was for 100 samples per movie and the new formula is for one sample per minute.

As a side note, I think that if your movie is 24 frames per second then the formula is simply:

Predicted MPEG size = 60 * MPEG sample file size

-ten

Girv,

If you are interested...

I've created a small Microsoft Excell file that you can use for your file prediction. It is based on Kwags formula plus a little tweak here and there.

You just typed in the variables and the formula are already built-in so you will have answers too on that page at the same time.

But its kind of low tech but I think it is good.

the last movie I did has a 0.02 percent efficiency...

"If you are interested..."

yes,i am .....say more ! :)
or post it! :wink:
thanks.

I dont know how... it wont accept attachments....
email will do...

any suggestions?

my e-mail is working now,
please send it!
:D

On Kwags formula, you are looking for SM which is movie size.

What if you already know the movie size. So we are going to jumble the formula and instead look for SS which is sample size.

I also need to add that I just used Kwags formula to know the SS...
PredictedSize = ((Framecount/FrameRate)/100) * SampleSize * Factor

I must admit KWAG that the 0.95 factor is accurate enough to make a 0.02 percent difference on two movies I've made...

Spiderman and Jimmy Neutron...

Ex: Spiderman
Predicted Size = 651,801
Actual Size =651,699

pretty close huh...

But, of course this might not work on other movies...

Hi katchupoy,

That's with the old formula. It's not used anymore since we changed the GOP to 1-12-2-1-24.

The new formula is Predicted MPEG size = (( Total frames/MovieTimeInMinutes) / 24 ) * MPEG sample file size

You don't need offset factor with the new formula, because it's much more accurate. Specially with CQ. I recall problems with the old formula and CQ mode.

-kwag

Ok, I am using this...

KVCD-CQ-352x240-_NTSCFilm_-PLUS__120MIN.mcf
GOP = 1-5-2-1-24

am i going to change the GOP as per your instruction?

I will give it a try...

So I just need to make my factor = 1 so I dont have to rewrite my small excel file formula

:lol: :lol: :lol:


Thanks

Jorel,

did you received the file?

yeah :wink:
read this:

don't thank me yet...

It might not work for you... but I am crossing my fingers...
:D

thank you!! :yippie:

@ARnet_tenRA:
Check through the "working out" in my post. The sample length (Ts) is accounted for in both cases. The old and new formulae are different only because the old method used 100s of sample and the new uses 1s per minute of movie. If you replace these numbers with variables both formulae can be proved to be the same thing.

Thats correct. I believe the "24" in the new formula is framerate rather than GOP length, so I also believe that the formula you give above will hold so long as you sample one second per minute of movie (Tm/Ts = 60) independent of framerate. More tests are required though.

@katchupoy:
Thanks, but I've got my own :)

With the new formula one uses minutes-in-movie samples of max-gop-length frames. At the moment that's 24 regardless of frame rate :).

This is correct. But to simplify the formula you could use the following.

If frames/second: 23.976
Then formula: Predicted Size = 59.94 * sample size

If frames/second: 25
Then formula: Predicted Size = 62.5 * sample size

If frames/second: 29.97
Then formula: Predicted Size = 74.925 * sample size

These calculations are true as long as you sample 24 frames for every minute of your video (this matches the current GOP length). If you sample 1 second for every minute of your video (ie. 25 samples for 25fps or 30 samples for 29.97fps) then the formula for all framerates would be:
Predicted Size = 60 * sample size

-ARnet_tenRA

You are correct :encore: :mrgreen:
And as we are always sampling 24 frames per minute, the simplified formula would be Predicted Size = 60 * sample size as you just wrote it above.

That means that all the bull of frames / minutes / 24 is over :mrgreen:
Just 800 - audio size / 60 will give you predicted sample size :mrgreen:

THANKS :!: :D
-kwag

I believe that's true only when the frame rate is 23.976.

SansGrip is right. The simplest formula is only correct when the number of frames samples per minute equals the frame rate.

What I do is always sample one second per minute of video. This makes the formula so much easier.

Also, Predicted = (800 - audio size) / 60 You have to remember the parenthesis when subtracting. :D

-tenra

Of course :D, that happens when I'm trying to do about four things at the same time :lol:

-kwag

I was wondering how accurate the prediction is for people that sample the framerate rather than the GOP rate of 24. If it is negligible then we could switch to the simpler formula. Of course this is most important for PAL,25 or NTSC,29.97 fps.

Let me know.

-tenra

It's most accurate when the sample length matches the GOP length. We ran both formulas (along with about a dozen others) already :).

Thats what I said :wink: Sm = (Tm/Ts) * Ss. If you adjust this equation
for taking one GOP-length sample for every 60 seconds of movie you get:

• Sm = 60 * (FR/Lg) * Ss
  Sm = movie file size
  FR = frame rate in frames per second
  Lg = GOP Length in frames
  Ss = sample file size

If you plug in the numbers for each frame rate you get the
constants ARnet_tenRA posted (59.94, 62.5, 74.925).

More generally this becomes:

• Sm = (Tm/Cs) * (FR/Ls) * Ss
  Sm = movie file size
  Tm = movie length in seconds
  Cs = number of samples taken
  FR = frame rate in frames per second
  Ls = length of each sample in frames
  Ss = sample file size

Why 800? ISTR something about VCD's being written in "MODE2" (or
something) so you could fit more data on the disk, but does this mean
that I can fit an 800Mb .mpg file on to a 700Mb disk if its used as a
VCD / SVCD?!

I've been using an upper limit of 700Mb until now, but an extra 100Mb
would be very nice :)

/girv

Just a thought, but I wonder if the prediction would be even more
accurate if sample strips were started on frame numbers that were
multiples of GOP length instead of multiples of one second?
eg: sample 24 frames every 24*60 frames instead of every
framerate*60.

Im thinking that this way you would be creating the sample with
GOPs that would actually be in the final encode. Mad?

Then today's your lucky day, because you can indeed get a maximum of 800mb on a VCD. To be exact, 813,019,155 bytes :).

They should always be multiples of GOP length. If your frame rate is not 23.976, you should specify the sample length, i.e. Sampler(length=24). If your frame rate is 23.976 you can just use Sampler() since the rounded frame rate happens to match the GOP length.

I was referring to the start frame of the sample strip not the number
of frames in it, which as you say should be equal to the GOP length.

e.g.: if your frame rate is 25fps then Sampler(length=24) will take
sample strips starting at frame 0, 1500 (25*60*1), 3000 (25*60*2),
4500 (25*60*3) ... correct? What I am suggesting is to instead align
the start of the sample strip to the multiple of GOP length closest to
these numbers i.e.: 0, 1488, 3000, 4488 ...

fps: 23.976
current sample start: 0,1437,2877,4316,...
proposed: 0,1440,2880,4320,...

fps: 25
current: 0,1500,3000,4500,...
proposed: 0,1488,3000,4488,...

fps: 29.97
current: 0,1798,3596,5395,...
proposed: 0,1800,3600,5400,...

The differences aren't much (+- 12 frames at most) but I just wondered
if it could give a little extra accuracy :)

Sorry to be dumb :oops: but if I have a .mpg file on my hard drive that
is 810,000,000 bytes then it can be burned on to a standard 700Mb
CD-R as a VCD? Happy day :lol: !

What about SVCD? Is that the same?

What is the overhead for VCD/SVCD ie: how big can a .mpg file
be on my hard drive and still (just) fit on to a 700Mb CD-R ?

Ah, I see. It might make it more accurate, yes, but my gut says not a lot. I'll have to modify Sampler slightly and try it :).

Roughly, though I believe the SVCD filesystem is slightly different, so will have slightly different overhead.

The figure I gave (813,019,155 bytes) is compensated for filesystem overhead and the system stream. In other words, if you subtract from that the size of your audio (which I always encode first), you'll get the maximum number of bytes for your video stream.

If you have a .mpg file then it's already got the system stream in it, so the maximum byte count will be 825,105,664.

(By the way, this is how I always do my prediction:

813,019,155 - audio_bytes = max_video_bytes
max_video_bytes / frames_in_movie = bytes_per_frame
bytes_per_frame * frame_count_with_sampler = sample_bytes

It's almost always accurate within 0.5% or so. It's more involved than the regular formula, but I'm testing it for the next release of KVCDP :).)

How about this, always sample (24*60*n) as the starting frame for each sample. Don't worry about the frame rate at all. ie. 0, 1440, 2880, 4320, . . . When you do it this way all sample MPEGs produced will be exactly 1/60th of the final movie.

fps: 23.976
current sample start: 0,1437,2877,4316,...
proposed: 0,1440,2880,4320,...

fps: 25
current: 0,1500,3000,4500,...
proposed: 0,1440,2880,4320,...

fps: 29.97
current: 0,1798,3596,5395,...
proposed: 0,1440,2880,4320,...

This will have the benefit of aligning with the GOP like girv suggested and having the simplest formula no matter the length of movie or framerate:
Predicted Size = 60 * sample size

-ARnet_tenRA

The main requirement of the formula is to be accurate, not necessarily simple ;). A great deal of testing indicates that the most accurate formula for all kinds of sources and all resolutions is minutes-in-movie samples, each max-gop-size frames in length. That said, we should obviously test this method out against the current one :).

Sampler uses a pretty simple algorithm to decide which frames to select. The curious can take a look at the source code here.

I'm not sure that aligning with the GOP will make things any more accurate, but I can certainly build a test version of Sampler with that modification and try it out.

The frame rate is only used by Sampler as a default sample length if none is specified -- it's not a part of the formula. I figured that movie-length-in-minutes samples of one second each would be a good generic default, and happened to correspond to the current file prediction formula for KVCD.

Hi all,

I just ran a test last night using my suggested formula, and I got some pretty exciting results. .0001% accuracy!!!

AVISynth script:

• Sample file size = 12,091,021 bytes
  Predicted movie size = 60 * 12,091,021 = 725,461,260 bytes
  Actual final Movie size = 725,551,421 bytes
  Error = 90,161 bytes
  % error = 0.0001

Based on these results I would like to try some more movies and see what I get.

Note that SelectRangeEvery doesn't seem to be quite accurate. The old prediction method, which used SelectRangeEvery, produced sample strips that had a frame count fairly significantly different from what should have been produced.

That's one of the reasons I wrote Sampler :).

I knew that you could get up to one sample too many, but I was unaware of any more offset than that. Let me know it this is not the case.

I used SelectRangeEvery because I knew exactly where the frames would be captured from in the video (every 1440 frames). Maybe this is not the case. :oops:

Anyways, let me know if sampling 24 frames every (24*60*n) frames gives you as accurate results as I got. Whether you use Sampler or SelectRangeEvery. :)

-ARnet_tenRA

Using the old prediction method I would get mismatches of a dozen or two frames. Not normally a big deal, but it is for our purposes.

I will try that out once GripFit is released. I don't want to get distracted by anything right now so I can get it out as quickly as possible ;).

AHA!, will that be the official name, GripFit 8)

-kwag

I haven't got round to deciding yet ;).