Faster Prediction Method - digitalFAQ.com Forums [Archives]
 digitalFAQ.com Forums [Archives] Faster Prediction Method

#1
04-04-2003, 12:49 PM
 ARnet_tenRA Free Member Join Date: Jan 2003 Location: Illinois, USA Posts: 73 Thanks: 0 Thanked 0 Times in 0 Posts
Hi all,

I have been working on a method to decrease the amount of time required to do file size prediction.
• Here it is:
1. Figure out the desired sample size.
2. Take a full size sample at CQx.
3. Take a small size sample also at CQx.
4. Figure out the ratio of full sample to small sample.
5. Continue prediction with the small size sample, but multiply the resulting file size by the ratio and compare to the desired sample size.
6. After your small samples are complete you have the correct CQ, no need to do another full size sample.
• Example:
1. Desired Sample Size = 12,012,701
2. Full Sample at CQ75 = 15,155,365
3. Small Sample at CQ75 = 1,384,020
4. Ratio Full:Small = 10.950249996...
5. Here I continued prediction until a CQ of 63.29 was reached.
Small Sample at CQ63.29 = 1,096,972
Small * ratio = 12,012,118
6. Not necessary, but for example I ran a full sample at CQ63.29
Full Sample at CQ63.29 = 12,019,680 #almost perfect
The greatest benefit of this method is that it is very accurate while only having to encode a full size sample once.

Regards, Tenra
Someday, 12:01 PM
 Site Staff / Ad Manager Join Date: Dec 2002 Posts: 42 Thanks: ∞ Thanked 42 Times in 42 Posts
#2
04-04-2003, 02:20 PM
 kwag Free Member Join Date: Apr 2002 Location: Puerto Rico, USA Posts: 13,537 Thanks: 0 Thanked 0 Times in 0 Posts
That's brilliant tenra . Maybe instead of using 10% on subsequent runs, it can even be dropped to 1%, because the ratio is always constant
So just one full sample, and then multiple (Fast!) 1% runs afterwards
Trying it out in a few minutes

-kwag
#3
04-04-2003, 02:25 PM
 ARnet_tenRA Free Member Join Date: Jan 2003 Location: Illinois, USA Posts: 73 Thanks: 0 Thanked 0 Times in 0 Posts
Quote:
 Originally Posted by kwag Maybe instead of using 10% on subsequent runs, it can even be dropped to 1%, because the ratio is always constant So just one full sample, and then multiple (Fast!) 1% runs afterwards
I considered that too, but haven't tried it yet.

Good luck

Tenra
#4
04-04-2003, 02:48 PM
 kwag Free Member Join Date: Apr 2002 Location: Puerto Rico, USA Posts: 13,537 Thanks: 0 Thanked 0 Times in 0 Posts
Just tried it with one 100% run and three 10% runs to get CQ. Wanted file size: 12,059.95KB. Full encoded sample after finding CQ: 12,046KB
Have a beer tenra

-kwag
#5
04-04-2003, 07:02 PM
 kwag Free Member Join Date: Apr 2002 Location: Puerto Rico, USA Posts: 13,537 Thanks: 0 Thanked 0 Times in 0 Posts
Note: Keep the second (or more) runs at 10%. I tried 1%, 2%, etc, and it's not very accurate. So the first run of 100% and subsequent runs of 10% seems to hit a <1% accuracy every time
Another beer for Tenra

-kwag
#6
04-04-2003, 08:41 PM
 muaddib Free Member Join Date: Jun 2002 Location: São Paulo - Brasil Posts: 879 Thanks: 0 Thanked 0 Times in 0 Posts
That's a great and clever idea!
As always, the simpler the idea the more brilliant it is!
Here... take another beer
#7
04-04-2003, 10:42 PM
 kwag Free Member Join Date: Apr 2002 Location: Puerto Rico, USA Posts: 13,537 Thanks: 0 Thanked 0 Times in 0 Posts
Now we just need hedix to integrate this into ToK, and we'll all be drinking more beer .
#8
04-04-2003, 10:55 PM
 muaddib Free Member Join Date: Jun 2002 Location: São Paulo - Brasil Posts: 879 Thanks: 0 Thanked 0 Times in 0 Posts
Hey hedix, are you listening!?
Hurry... I'm thirsty!!
#9
04-04-2003, 11:37 PM
 jorel Invalid Email / Banned / Spammer Join Date: Aug 2002 Location: Brasil - MG - third stone from the sun Posts: 5,570 Thanks: 0 Thanked 0 Times in 0 Posts
@ friend "net_ten":

....

i'm in the party now.....fantastic!!!

Kwag,
we need some emoticons with fireworks!!
#10
04-05-2003, 07:26 AM
 hedix Free Member Join Date: Jan 2003 Posts: 155 Thanks: 0 Thanked 0 Times in 0 Posts
Yep ! one day out, and .... ole!

So the ratio is constant... You're great, Tenra.

I'm a little busy now, but wait a few days and ToK will do the thing.

Best regards,

Hedix
#11
04-05-2003, 11:39 AM
 Holomatrix Free Member Join Date: Aug 2002 Posts: 130 Thanks: 0 Thanked 0 Times in 0 Posts
Great, can't wait
#12
04-05-2003, 09:24 PM
 Daagar Free Member Join Date: May 2002 Posts: 158 Thanks: 0 Thanked 0 Times in 0 Posts
Stupid question, but what are the correct parameters to Sampler() for a 10% clip? In the past, Sampler(length=24) has been the default used for a full size clip... is it as simple as Sampler(length-2.4)? Laf... nevermind, can't use 2.4, it isn't an int.

Update: Okay, after finally realizing SansGrip had instructions included with Sampler(), it appears you just take 10% of the running time of the movie in minutes, changing things to Sampler(samples=<movie time in minutes*.1>, length=24).
#13
04-05-2003, 10:31 PM
 kwag Free Member Join Date: Apr 2002 Location: Puerto Rico, USA Posts: 13,537 Thanks: 0 Thanked 0 Times in 0 Posts
Hi Daagar,

You can even use something like this:

Code:
```interval = round((FrameCount/24)/60)  # Interval spacing ( Full Sample )
#interval = round((FrameCount/24)/60)  / 10 # Interval spacing ( 10% of full sample )
nFrames = round(24) # Frames per sample
SelectRangeEvery( (round(framecount/interval)),nFrames)```
This will take one second sample for every minute of the movie. Comment the first line and uncomment the second line for 10% of the full sampler

-kwag
#14
04-10-2003, 09:10 AM
 Daagar Free Member Join Date: May 2002 Posts: 158 Thanks: 0 Thanked 0 Times in 0 Posts
Ah, even better! Less chance of bumbling simple calculations Thanks!
#15
04-10-2003, 10:17 PM
 ozjeff99 Free Member Join Date: May 2002 Location: Sydney, Australia Posts: 159 Thanks: 0 Thanked 0 Times in 0 Posts
G'day - I'm in and will get the next round of drinks .
#16
04-11-2003, 12:37 AM
 jorel Invalid Email / Banned / Spammer Join Date: Aug 2002 Location: Brasil - MG - third stone from the sun Posts: 5,570 Thanks: 0 Thanked 0 Times in 0 Posts

a simple and stup question:

can someone post a sample script,
or explain me the way to use this,please?

thank you.
#17
04-12-2003, 02:10 AM
 ozjeff99 Free Member Join Date: May 2002 Location: Sydney, Australia Posts: 159 Thanks: 0 Thanked 0 Times in 0 Posts
Hi Jorel

This works for me:
1.Load Avs (without sampler()) and Audio into Tok and start file prediction and peek in log tab to get target video full sample size (TFS). Then exit without finishing. This is a cheats way of not having to calculate it.

2.Use Kwags scripts in this thread above at the end of your avs to give you two samples at say Q=60. One is Full Sample (FS) the other is Small Sample (SS).

3.Calculate SS/FS ratio.

4.Assuming the above ratio is independent of CQ:
SS/FS = TSS/TSF where TSS is the target small sample size
ie. TSS=(SS/FS)*TFS

5.So the problem is one now of playing with the CQ values to give you the correct SS. The great thing is that the SS samples usually take less than a minute. I use a simple linear interpolation formula which is easy to set up in Excell to automatically give me the next CQ to try. There is no need to scale up to full sample size at any time in this process unless you want to re-check at the end that the FS=TFS for the final CQ.

Ecample: The Shining.
1.Tok sais that the
TFS = 10,609 kB
2.Ran Kwag’s full sample script at CQ=60 to get
FS = 9,622 kB
3.As above for small sample to get
SS = 854 kB
4.From above
TSS = 941.6 kB
5.Repeating step 3. for different CQs. At CQ = 64 (3 tries) gave me
SS = 942 kB

For posterity, I ran a FS at CQ = 64 and got
FS = 10,628 kB a 0.01% error !!

Hope this helps. No doubt we will see all this included in next Tok soon.

Cheers
Ozjeff99
#18
04-12-2003, 02:33 AM
 kwag Free Member Join Date: Apr 2002 Location: Puerto Rico, USA Posts: 13,537 Thanks: 0 Thanked 0 Times in 0 Posts
Quote:
 Originally Posted by ozjeff99 No doubt we will see all this included in next Tok soon.

-kwag
#19
04-12-2003, 02:43 AM
 ozjeff99 Free Member Join Date: May 2002 Location: Sydney, Australia Posts: 159 Thanks: 0 Thanked 0 Times in 0 Posts
I'm glued to this channel. :P
#20
04-12-2003, 09:11 AM
 jorel Invalid Email / Banned / Spammer Join Date: Aug 2002 Location: Brasil - MG - third stone from the sun Posts: 5,570 Thanks: 0 Thanked 0 Times in 0 Posts

seems... simple and fantastic.

thanks ozjeff99