digitalFAQ.com Forums [Archives]

Page 2 of 5

Show 40 post(s) from this thread on one page

digitalFAQ.com Forums [Archives] (http://www.digitalfaq.com/archives/)

- Avisynth Scripting (http://www.digitalfaq.com/archives/avisynth/)

- - KVCD Predictor (http://www.digitalfaq.com/archives/avisynth/1626-kvcd-predictor.html)

SansGrip

11-19-2002 09:00 AM

Quote:

Originally Posted by ozjeff99

I get my movie length from Fitcd which nicely shows it in total seconds. Is it possible to have the program accept this single value rather than have to make several entries.

FitCD (at least the version I'm running) also gives the length in frames as well as the FPS. You can hit the "Enter length in frames and frames per second..." link and enter these instead of breaking up the seconds into hours, mins, and secs.

HTH.

SansGrip

11-19-2002 05:31 PM

@kwag

I tried (manually, with a calculator) your method for CQ prediction and I got the perfect result of 19.9 with just three encodes. It would've taken me at least double that by guesswork alone :!:.

Since I've finished porting Blockbuster to YV12, I think I'm going to start implementing the wizard tonight...

SansGrip

11-19-2002 07:02 PM

@kwag:

I just did a one-disc of American Pie using CQ 19.9 and a scale factor of 0.95. Sample size was 13mb, audio is 87mb, but the final muxed encode came out at 761mb. That's out by 36mb 8O. I'm trying again with a scale factor of 0.93...

What do you think may have gone wrong?

Script:

Code:

AudioDub(Mpeg2Source("ap.d2v"), WavSource("ap.wav"))

LegalClip()

BilinearResize(352, 352, 11, 8, 700, 462)

FluxSmooth()

Blockbuster(method="noise", detail_min=1, detail_max=10, variance=1.5)

AddBorders(0, 64, 0, 64)

#IL = Framecount / 100

#SL = round(Framerate)

#SelectRangeEvery(IL, SL)

LegalClip()

kwag	11-19-2002 08:17 PM

Quote:

Originally Posted by SansGrip

Nothing. That's within 5% and that's good. I usually get between 2% to 5%. Maybe the filters change the factor offset a little. If your factor was .95, try .98. Not 0.93, because that will make the file smaller.

-kwag

SansGrip

11-19-2002 08:35 PM

Quote:

Originally Posted by kwag

Maybe the filters change the factor offset a little. If your factor was .95, try .98. Not 0.93, because that will make the file smaller.

See, this is the kind of thing that makes me thing I should be multiplying by the scale factor!

If you took a desired encode size of, say, 797mb, but wanted to leave a little room for insurance, multiplying it by 0.95 would make it smaller. Dividing it by 0.95 would make it bigger.

Try it in KVCD Predictor. When you lower the scale factor, the target size increases. This is because you're dividing by a number less than 1. If we multiplied by the scale factor, then lowering it would cause a corresponding decrease in the target size.

I'm going to go revisit that formula and do a proper rearrangement and see what I come up with.

SansGrip

11-19-2002 09:02 PM

Houston, we have a problem -- or, at least, I do ;).

The original formula, slightly simplified, is:

final_size = secs / 100 * scale_factor * sample_size

Let's assume for the moment that the scale factor is 1. This gives us:

final_size = secs / 100 * sample_size

Now, since the sample_size represents the encoded size of 100 clips 1 second in length, we could say that:

sec_size = sample_size / 100

where sec_size means "average size of 1 second of the movie".

We can then represent the formula as:

final_size = secs * sec_size

Which is to say, the final size will be the size of an average second times the number of seconds in the movie.

Now, the reason for the scale factor was to provide for some insurance, a little headroom in the calculation. To this end, and assuming we want to use 0.95 to mean 95% of full capacity, we would multiply the left-hand side of the equation by the scale factor:

final_size * scale_factor = secs * sec_size

Or, rearranged:

final_size = secs * sec_size / scale_factor

And, expanded:

final_size = secs * (sample_size / 100) / scale_factor

Thus when we decrease the scale factor, the final size also decreases. When we increase the scale factor, the final size also increases.

In that case the formula KVCD Predictor should be using is:

sample_size = final_size * scale_factor / secs / 100

This is the formula used in 0.1a.

Thoughts?

kwag	11-19-2002 10:28 PM

Quote:

Originally Posted by SansGrip

Now, the reason for the scale factor was to provide for some insurance, a little headroom in the calculation. To this end, and assuming we want to use 0.95 to mean 95% of full capacity, we would multiply the left-hand side of the equation by the scale factor:

final_size * scale_factor = secs * sec_size

Or, rearranged:

final_size = secs * sec_size / scale_factor

And, expanded:

final_size = secs * (sample_size / 100) / scale_factor

Thus when we decrease the scale factor, the final size also decreases. When we increase the scale factor, the final size also increases.

In that case the formula KVCD Predictor should be using is:

sample_size = final_size * scale_factor / secs / 100

This is the formula used in 0.1a.

Thoughts?

Or manually calculated:
Sample Size = ((Total frames/Framerate)/100) * (MPEG sample file size * .95)
Where increasing the factor, in this case .95, increases file size.

Edit: ( And I'm not drunk, yet! :P )
Final MPEG Predicted Size = ((Total frames/Framerate)/100) * (MPEG sample file size * .95)

-kwag

muaddib

11-19-2002 11:38 PM

Hi SansGrip!
I want to thank you for all the GREAT filters and also for the KVCD Predictor!

But I just got confused now...

Quote:

Originally Posted by SansGrip

And, expanded:

final_size = secs * (sample_size / 100) / scale_factor

Thus when we decrease the scale factor, the final size also decreases. When we increase the scale factor, the final size also increases.

With this formula, I think that's the opposite...
As the scale factor is dividing the whole thing, when we decrease the scale factor, the final size increases, because we are dividing by a smaller value. And that's not the way the formula should behave.

I think that the error is here:

Quote:

Originally Posted by SansGrip

As we are calculating what would be the final size, and if we want to leave a little headroom in the calculation (95% of the full file size), we have to multiply the right-hand side of the equation by the scale factor (95%), not the left side.

Just to illustrate:
If we want 95% of X and X=100, we have to multiply 100 (that's the X value) by 95%, in order to get 95. If you divide by 95% you will get ~105.26, and that’s not the 95% that we want.

Well, please excuse me for the bad english.
And excuse me if I'm completely wrong... because it's 3:30AM here, and according to kwag, I could be drunk by now. :wink: :lol:

muaddib

11-19-2002 11:40 PM

Quote:

Originally Posted by kwag

Or manually calculated:
Sample Size = ((Total frames/Framerate)/100) * (MPEG sample file size * .95)

Oh Man, I really should be drunk... I'm seeing too much "sample size" in this formula. :wink: :lol:

kwag	11-19-2002 11:46 PM

Quote:

Originally Posted by muaddib

Quote:

Originally Posted by kwag

Or manually calculated:
Sample Size = ((Total frames/Framerate)/100) * (MPEG sample file size * .95)

Oh Man, I really should be drunk... I'm seeing too much "sample size" in this formula. :wink: :lol:

I'm the one drunk 8O . It should be MPEG final size. Not Sample size :lol:

Correction here:
Final MPEG Predicted Size = ((Total frames/Framerate)/100) * (MPEG sample file size * .95)

( And I haven't had a beer yet. Wonder what I would write with a couple of beers :P )

Thanks muaddib,
-kwag

muaddib

11-19-2002 11:59 PM

BTW, I think that the scale factor (.95) is just a correction for an "almost" constant accuracy error in the formula. The aim here is to get a perfect file to fill the CD completely. Isn't it, kwag?

kwag	11-20-2002 12:23 AM

Quote:

Originally Posted by muaddib

Yeah, but given the nature of CQ encoding and different encoding materials, there's no way to get 100% accuracy. It's pretty close though :wink:

-kwag

SansGrip

11-20-2002 01:38 AM

Nope, I'm the one who's drunk :D. I should've run some real numbers through and I would've spotted the error.

I think what's confusing me is that when you adjust the scale factor down in KVCD Predictor, the sample target size goes up, and vice versa.

Is that the correct behaviour?

I've been thinking way too much about noise and averages today to think properly about anything else ;).

ozjeff99

11-20-2002 06:26 AM

Quote:

Originally Posted by SansGrip

Quote:

Originally Posted by ozjeff99

I get my movie length from Fitcd which nicely shows it in total seconds. Is it possible to have the program accept this single value rather than have to make several entries.

Sansgrip...I'm lazier and drink more beer by the way. 8) . Could you leave the "seconds" box without an upper limit so I only have to make a single entry (thought I'd take you mind of the other issue :lol: ). I'm sure the existing underlying algorithm can be left unaltered.....Cheers mate.

kwag	11-20-2002 07:39 AM

Quote:

Originally Posted by SansGrip

Hi SansGrip,

It should be the other way around. When you increase the factor, the file size goes up ;)

-kkag

SansGrip

11-20-2002 09:45 AM

Quote:

Originally Posted by kwag

It should be the other way around. When you increase the factor, the file size goes up ;)

Ok, then we need to rearrange this:

final_size = (secs /100) * (sample_size * scale_factor)

so that MPEG sample file size is isolated on one side. Here's my attempt:

final_size / (secs / 100) = sample_size * scale_factor

final_size / (secs / 100) / scale_factor = sample_size

And this is the formula used in KVCD Predictor. If you feed some numbers into it (movie is 5728 seconds, audio is 87mb, media is 797mb):

sample_size = 710 / (5728 / 100) / 0.95 = 13.04

sample_size = 710 / (5728 / 100) / 0.90 = 13.77

sample_size = 710 / (5728 / 100) / 1.05 = 11.80

As you can see, the sample size rises when the scale factor falls, and vice versa. Either this is the correct behaviour when obtaining the sample size from the final size, or my rearrangement of the formula is wrong, or the formula itself is wrong.

:)

SansGrip

11-20-2002 09:47 AM

Quote:

Originally Posted by ozjeff99

Could you leave the "seconds" box without an upper limit so I only have to make a single entry (thought I'd take you mind of the other issue :lol: ). I'm sure the existing underlying algorithm can be left unaltered.....Cheers mate.

Ok. I think I'm going to change them to textboxes anyway since I don't really like the way that numeric spinners (i.e. what they are right now) behave. That way you can enter seconds and minutes > 59 and it'll auto-normalize the values for you.

kwag	11-20-2002 10:26 AM

Quote:

Originally Posted by SansGrip

As you can see, the sample size rises when the scale factor falls, and vice versa. Either this is the correct behaviour when obtaining the sample size from the final size, or my rearrangement of the formula is wrong, or the formula itself is wrong.

:)

Maybe it's better to describe the factor field as % offset :idea:
So the meaning is clearer, and then 5 would be +5% and make the file 5% larger, and -5 would mean -5%, and make the file 5% smaller :idea:
Instead of 0.95 or 1.05 etc.

-kwag

rendalunit

11-20-2002 11:44 AM

Quote:

Originally Posted by kwag

Maybe it's better to describe the factor field as % offset icon_idea.gif
So the meaning is clearer, and then 5 would be +5% and make the file 5% larger, and -5 would mean -5%, and make the file 5% smaller icon_idea.gif
Instead of 0.95 or 1.05 etc.

@kwag, I thought it would be easier if the scale factor was described like that too (that's why I put it like that in my code :wink:)

Code:

// This is a very simple program to calculate what the final file size of the

// 100 second sample should be using the file size prediction method (thnks Kwag

// and Ozzi!). The estimated audio file size by Headac3he is subtracted from the total space

// available to determine how much video space is available.  This program assumes

// you're using 80 min cdrs (if not change the const cd_size value)

// This program will crash if the wrong data type is entered:-(



#include <stdio.h>

#include <stdlib.h>



int main(void)

{

  const long long cd_size = 800000000;

  const long convert_to_mb = 1000000;

  float framerate, error_margin;

  double sample_video_size;

  int number_of_cds;                           

  long total_movie_frames, est_audio_size;

  long long video_space;





    

  printf("enter estimated audio size in mb >");

  scanf("%d", &est_audio_size);

  printf("%dmb audio \n\n",est_audio_size);

  est_audio_size = est_audio_size * convert_to_mb;

  

  printf("enter the number of 80 min cds to use >");

  scanf("%d", &number_of_cds);

  printf("%d cd(s) to burn \n\n",number_of_cds);

  

  video_space = (cd_size * number_of_cds)-est_audio_size;

  printf("there's %lldmb space for video \n\n",video_space / convert_to_mb);

  

  printf("enter total frames of video >");

  scanf("%d", &total_movie_frames);

  printf("%d total frames \n\n",total_movie_frames);

  

  printf("enter the framerate >");

  scanf("%f", &framerate);

  printf("%.2f frames per second \n\n",framerate);

  

  printf("enter percent error margin (5 is suggested, enter 0 for none) >");

  scanf("%f",&error_margin);

  printf("error margin >%.1f percent \n\n",error_margin);

  error_margin = (100 - error_margin) / 100;

  

  sample_video_size = (video_space / ((total_movie_frames/framerate) /100)) * error_margin;

    

  printf("the target sample size should be about %.1fmb \n", sample_video_size / convert_to_mb);

  printf("\npress <enter> to end..");

  getchar();

  getchar();        

  return 0;

}

joecross98

11-20-2002 12:33 PM

Silly Question

I don't understand what the "sample size" is meant to indicate. I was expecting to see a number reflecting the final file size if I were to use template X, Y or Z, given the AVI information. AM I meant to derive this final number from the sample size number that is indicated?

SansGrip

11-20-2002 09:23 PM

Update...

Just so you know progress is being made...

http://www.digitalfaq.com/archives/error.gif

:)

kwag	11-20-2002 09:29 PM

Oh boy, this is getting better by the hour :lol:

Thanks SansGrip :wink: :mrgreen:

-kwag

muaddib

11-21-2002 12:28 PM

Quote:

Originally Posted by kwag

Quote:

Originally Posted by SansGrip

Hi SansGrip,

It should be the other way around. When you increase the factor, the file size goes up ;)

-kkag

Hey, Hey!
Wait there... I think you 2 are talking about different things.
(It should be, or I'm not drunk... I'm crazy! :lol: )
SansGrip is talking about the sample size, and kwag is talking about the mpeg final size (like I was).

With that same formula...
If you want to find the final size from an encoded sample, then if you increase the factor, the final size goes up.
But, if you want to find the sample size from a fixed final size, then if you increase the factor, the sample size goes down (like you said).

Man... is that right? :roll: :lol:

muaddib

11-21-2002 12:32 PM

Quote:

Originally Posted by SansGrip

sample_size = 710 / (5728 / 100) / 0.90 = 13.77

sample_size = 710 / (5728 / 100) / 1.05 = 11.80

As you can see, the sample size rises when the scale factor falls, and vice versa. Either this is the correct behaviour when obtaining the sample size from the final size, or my rearrangement of the formula is wrong, or the formula itself is wrong.

:)

Yes :!: :lol:
That is the correct behaviour when obtaining the sample size.

muaddib

11-21-2002 12:58 PM

Re: Update...

Quote:

Originally Posted by SansGrip

Just so you know progress is being made...

Man, that’s GREAT!

There is just one thing I would like to suggest...
In Step 3, when you show the size of the encoded sample and the target size; would be good if you also show the predicted final size with this encoded sample. I think that this information will make the decision between stop or try another CQ_VBR easier.

You could add just a complement to the message you are using now. Something like this:
“(…) The sample strip is 0.62 megabytes smaller than the target size, and will produce a mpeg file of an approximate 999.99 megabytes.”

What you think?

johneboy

11-21-2002 02:31 PM

Re: First release

Quote:

Originally Posted by SansGrip

In this version the "Get movie length from source file..." option doesn't work, because it's going to be a lot more complicated than I imagined (for programmers amongst you, I have to learn how to write a managed C++ class to access the Video for Windows framework. Any hints appreciated :)). I'm going to work at it though, and hopefully it'll be ready for 0.2.

Hey SansGrip.. I dont know C++.. but i do know VB..
and i made a program that will let you select a media file (MPEG or AVI)
and it will tell you the length in HH:MM:SS MMM and SSSS
as well as how many frames, and the (estimated) frame rate
(Total Frames / SSSS). It also does other things too, but if you want the code to get the frame and playing time, i would be more then happy to give it to you.. I just dont know if it will help you out, since its in VB
(its all API tho)

let me know

SansGrip

11-21-2002 03:02 PM

Quote:

Originally Posted by muaddib

Yes :!: :lol:
That is the correct behaviour when obtaining the sample size.

Phew! :D

SansGrip

11-21-2002 03:05 PM

Re: Update...

Quote:

Originally Posted by muaddib

In Step 3, when you show the size of the encoded sample and the target size; would be good if you also show the predicted final size with this encoded sample.

Yes, very good idea :).

At the moment that dialog doesn't actually do the calculations -- it's just a mock-up I made after completing the design of the user interface. I'm going to make it work now, and will incorporate your suggestion.

SansGrip

11-21-2002 03:12 PM

Re: First release

Quote:

Originally Posted by johneboy

I dont know C++.. but i do know VB..
and i made a program that will let you select a media file (MPEG or AVI)
and it will tell you the length in HH:MM:SS MMM and SSSS
as well as how many frames, and the (estimated) frame rate
(Total Frames / SSSS).

The main problem is that I'm using .NET, and it doesn't incorporate any video classes yet. While I could read the metadata directly from the AVI header this wouldn't work with AVS files, so I need to use the Video for Windows API instead. Normally this would be easy but because .NET is what's known as "managed" code (it has garbage collection, and so on) I'm going to have to write a "managed C++" class to access VfW, which I can then call from C#.

Luckily I just this morning bought a book on C++ .NET, and it looks like this will be easier than I imagined.

In related news, I also got a book on assembly language, so I'm hoping to start optimizing my filters pretty soon :).

Quote:

Originally Posted by johneboy

but if you want the code to get the frame and playing time, i would be more then happy to give it to you..

I've used VfW before from C++ so that's not a big deal -- what's new for me is accessing a C++ DLL from C#. Thanks for the offer though, I appreciate it :).

BTW, how do you get the MPEG metadata? Do you read the header directly or use DirectShow?

kwag	11-21-2002 05:45 PM

Re: First release

Quote:

Originally Posted by SansGrip

I'm going to have to write a "managed C++" class to access VfW, which I can then call from C#.

"Managed C++" or "UnManaged C++" 8O 8)

-kwag

SansGrip

11-21-2002 08:18 PM

Re: First release

Quote:

Originally Posted by kwag

"Managed C++" or "UnManaged C++" 8O 8)

Managed C++ is MS-speak for C++ with garbage collection. It's the only kind of C++ you're allowed to call from C#. The good news is I've "managed" to get the C# GUI calling a C++ method. Now I just need to make that method return the correct values... ;)

SansGrip

11-22-2002 06:53 PM

Quick update

I thought I'd let you all know the current status of development...

I just finished the "Get movie length from source file..." functionality, so from version 0.2 you'll be able to select a .avs, .avi or .d2v file and have KVCD Predictor automatically determine the running time of the movie. I'll implement MPEG support in a later version if there's any demand for it.

(Side note: In the course of writing the .d2v-parsing code I discovered that FitCD does not use the correct algorithm for determining how many frames are in the movie when reading the .d2v file. This means it can be out by several seconds, at least with NTSC forced film material, so it would be best to let KVCDP read the file instead of simply typing in the values you see in FitCD.)

Before 0.2 can be released I need to finish the sample encoding helper. I've changed my mind slightly about how exactly it's going to work, but hopefully the new way will be more intuitive.

I'm hoping it'll be ready in the next couple of days. I'll keep you posted :).

kwag	11-22-2002 07:30 PM

Re: Quick update

Quote:

Originally Posted by SansGrip

(Side note: In the course of writing the .d2v-parsing code I discovered that FitCD does not use the correct algorithm for determining how many frames are in the movie when reading the .d2v file. This means it can be out by several seconds, at least with NTSC forced film material, so it would be best to let KVCDP read the file instead of simply typing in the values you see in FitCD.)

Maybe that's the reason the prediction fluctuates between 2% to 5% :roll:
Thanks again Sansgrip :D

-kwag

black prince

11-22-2002 07:45 PM

Hi Kwag,

Kwag wrote:

Quote:

Maybe that's the reason the prediction fluctuates between 2% to 5%
Thanks again Sansgrip

If that's the case how reliable is "Framecount" in avs script for
file size prediction. I used VirtualDub to check framecount against
Tmpgenc's source range and they differ. :) Is there a way to view
what value is being used in the script to calculate file size :?: Between
the 3 of them, which one is correct?

-black prince

SansGrip

11-22-2002 08:20 PM

Quote:

Originally Posted by black prince

If that's the case how reliable is "Framecount" in avs script for file size prediction.

I would say if two programs disagree on the frame count and one of them is Avisynth, the other program is wrong ;).

Quote:

Originally Posted by black prince

I used VirtualDub to check framecount against Tmpgenc's source range and they differ.

That's strange. By how much did they differ?

Quote:

Originally Posted by black prince

Between the 3 of them, which one is correct?

All programs accepting input from Avisynth should report the same number of frames as Avisynth shows with Framecount(), unless that program is doing something funky like IVTC.

black prince

11-22-2002 08:22 PM

Hi Kwag and SansGrip,

Just used "ShowFrameNumber()" in avs script:

Tmpgenc ....................--> 146,654 total frames
ShowFrameNumber .....--> 146,654 "
VirtualDub ..................--> 146,655 "
FitCD .........................--> 146,661 "

Seems that framecount is only off if you hardcoded FitCD's value.
The file size formula should be correct. 8)

-black prince

kwag	11-22-2002 08:26 PM

Quote:

Originally Posted by black prince

Hi Kwag,

Kwag wrote:

Quote:

Maybe that's the reason the prediction fluctuates between 2% to 5%
Thanks again Sansgrip

You can use IfoEdit to open your .IFO file, and then read the exact number of frames. But it will give you the total frames AT 30 FPS , so if you're encoding with "Force FILM" on DVD2AVI, your actual number of frames fed to TMPEG are ~Total frames = ( FramesReadByIfoEdit / 30 ) * 24 )

-kwag

SansGrip

11-22-2002 08:27 PM

Quote:

Originally Posted by kwag

...or you could just use KVCDP :P

kwag	11-22-2002 08:32 PM

Quote:

Originally Posted by SansGrip

Quote:

Originally Posted by kwag

...or you could just use KVCDP :P

Of course!, how could I forget :lol:

-kwag

SansGrip

11-23-2002 12:12 AM

New version 0.2

Here's version 0.2 (and source code).

Changes:

Quote:

Changed numeric spinners to textboxes (I prefer their behaviour for this application). Added "Sample points" textbox. Renamed "Scale factor" to "Error margin" and changed algorithm accordingly. Implemented "Get movie length from source file..." functionality; can read running time from .avs, .avi and .d2v files; also works with drag 'n' drop. Added "Sample encoding helper" to ease finding the correct CQ_VBR setting for the movie. Added version to title bar.

See the docs for more information.

I've tested this one a bit and it seems to give the correct results. The helper gives pretty good results -- normally very close to target size within three iterations.

@kwag - I altered the binary search algorithm slightly. Now, if the resulting sample is larger than the target it'll drop down a little more than according to your formula. This seems to speed things up when the CQ is too high, since it's obviously desirable to be slightly below the target size rather than above. I think the algorithm can still be improved, though, perhaps by remembering previous iterations and somehow applying that information when calculating the new CQ... Any thoughts?

Well, have fun and let me know what you think. Hopefully this one won't need two bugfix releases ;).

Page 2 of 5

Show 40 post(s) from this thread on one page