Faster prediction method
 

Hi all,

I have been working on a method to decrease the amount of time required to do file size prediction.

• Here it is:
  1. Figure out the desired sample size.
  2. Take a full size sample at CQx.
  3. Take a small size sample also at CQx.
  4. Figure out the ratio of full sample to small sample.
  5. Continue prediction with the small size sample, but multiply the resulting file size by the ratio and compare to the desired sample size.
  6. After your small samples are complete you have the correct CQ, no need to do another full size sample.

• Example:
  1. Desired Sample Size = 12,012,701
  2. Full Sample at CQ75 = 15,155,365
  3. Small Sample at CQ75 = 1,384,020
  4. Ratio Full:Small = 10.950249996...
  5. Here I continued prediction until a CQ of 63.29 was reached.
  Small Sample at CQ63.29 = 1,096,972
  Small * ratio = 12,012,118
  6. Not necessary, but for example I ran a full sample at CQ63.29
  Full Sample at CQ63.29 = 12,019,680 #almost perfect

The greatest benefit of this method is that it is very accurate while only having to encode a full size sample once.

Regards, Tenra

That's brilliant tenra :D. Maybe instead of using 10% on subsequent runs, it can even be dropped to 1%, because the ratio is always constant :idea:
So just one full sample, and then multiple (Fast!) 1% runs afterwards :!:
Trying it out in a few minutes 8)

-kwag

I considered that too, but haven't tried it yet.

Good luck :!:

Tenra

Just tried it with one 100% run and three 10% runs to get CQ. Wanted file size: 12,059.95KB. Full encoded sample after finding CQ: 12,046KB :mrgreen:
Have a beer tenra :lol: :drink:

-kwag

Note: Keep the second (or more) runs at 10%. I tried 1%, 2%, etc, and it's not very accurate. So the first run of 100% and subsequent runs of 10% seems to hit a <1% accuracy every time :D
Another beer for Tenra :drink: :!:

-kwag

That's a great and clever idea!
As always, the simpler the idea the more brilliant it is!
Here... take another beer :lol: :drink:

Now we just need hedix to integrate this into ToK, and we'll all be drinking more beer :mrgreen:.

Hey hedix, are you listening!?
Hurry... I'm thirsty!! 8O :wink: :mrgreen:

@ friend "net_ten":

8O 8O 8O :) :D :D :D ....


then i read your post again:
:book: 8O 8O :) :) :D :D :encore: :fahr: :drunkard: :drunkard:
i'm in the party now.....fantastic!!!

:wink:


Kwag,
we need some emoticons with fireworks!!

Yep ! one day out, and .... ole! :ole:

So the ratio is constant... You're great, Tenra.

I'm a little busy now, but wait a few days and ToK will do the thing.

Best regards,

Hedix

Great, can't wait :)

Stupid question, but what are the correct parameters to Sampler() for a 10% clip? In the past, Sampler(length=24) has been the default used for a full size clip... is it as simple as Sampler(length-2.4)? Laf... nevermind, can't use 2.4, it isn't an int.

Update: Okay, after finally realizing SansGrip had instructions included with Sampler(), it appears you just take 10% of the running time of the movie in minutes, changing things to Sampler(samples=<movie time in minutes*.1>, length=24).

Hi Daagar,

You can even use something like this:

This will take one second sample for every minute of the movie. Comment the first line and uncomment the second line for 10% of the full sampler :wink:

-kwag

Ah, even better! Less chance of bumbling simple calculations ;) Thanks!

G'day - I'm in and will get the next round of drinks 8) .

:?
a simple and stup question: :oops:

can someone post a sample script,
or explain me the way to use this,please?
:?
thank you.

Hi Jorel

This works for me:
1.Load Avs (without sampler()) and Audio into Tok and start file prediction and peek in log tab to get target video full sample size (TFS). Then exit without finishing. This is a cheats way of not having to calculate it.

2.Use Kwags scripts in this thread above at the end of your avs to give you two samples at say Q=60. One is Full Sample (FS) the other is Small Sample (SS).

3.Calculate SS/FS ratio.

4.Assuming the above ratio is independent of CQ:
SS/FS = TSS/TSF where TSS is the target small sample size
ie. TSS=(SS/FS)*TFS

5.So the problem is one now of playing with the CQ values to give you the correct SS. The great thing is that the SS samples usually take less than a minute. I use a simple linear interpolation formula which is easy to set up in Excell to automatically give me the next CQ to try. There is no need to scale up to full sample size at any time in this process unless you want to re-check at the end that the FS=TFS for the final CQ.

Ecample: The Shining.
1.Tok sais that the
TFS = 10,609 kB
2.Ran Kwag’s full sample script at CQ=60 to get
FS = 9,622 kB
3.As above for small sample to get
SS = 854 kB
4.From above
TSS = 941.6 kB
5.Repeating step 3. for different CQs. At CQ = 64 (3 tries) gave me
SS = 942 kB

For posterity, I ran a FS at CQ = 64 and got
FS = 10,628 kB a 0.01% error !! :lol:

Hope this helps. No doubt we will see all this included in next Tok soon.

Cheers
Ozjeff99

Sooner than an eye blink :wink:

-kwag

I'm glued to this channel. 8O :P

8O

seems... simple and fantastic.

thanks ozjeff99 :D

your explanation is cool!