Quantcast Faster Prediction Method - digitalFAQ.com Forums [Archives]
  #1  
04-04-2003, 12:49 PM
ARnet_tenRA ARnet_tenRA is offline
Free Member
 
Join Date: Jan 2003
Location: Illinois, USA
Posts: 73
Thanks: 0
Thanked 0 Times in 0 Posts
Hi all,

I have been working on a method to decrease the amount of time required to do file size prediction.
  • Here it is:
    1. Figure out the desired sample size.
    2. Take a full size sample at CQx.
    3. Take a small size sample also at CQx.
    4. Figure out the ratio of full sample to small sample.
    5. Continue prediction with the small size sample, but multiply the resulting file size by the ratio and compare to the desired sample size.
    6. After your small samples are complete you have the correct CQ, no need to do another full size sample.
  • Example:
    1. Desired Sample Size = 12,012,701
    2. Full Sample at CQ75 = 15,155,365
    3. Small Sample at CQ75 = 1,384,020
    4. Ratio Full:Small = 10.950249996...
    5. Here I continued prediction until a CQ of 63.29 was reached.
    Small Sample at CQ63.29 = 1,096,972
    Small * ratio = 12,012,118
    6. Not necessary, but for example I ran a full sample at CQ63.29
    Full Sample at CQ63.29 = 12,019,680 #almost perfect
The greatest benefit of this method is that it is very accurate while only having to encode a full size sample once.

Regards, Tenra
Reply With Quote
Someday, 12:01 PM
admin's Avatar
Site Staff / Ad Manager
 
Join Date: Dec 2002
Posts: 42
Thanks: ∞
Thanked 42 Times in 42 Posts
  #2  
04-04-2003, 02:20 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
That's brilliant tenra . Maybe instead of using 10% on subsequent runs, it can even be dropped to 1%, because the ratio is always constant
So just one full sample, and then multiple (Fast!) 1% runs afterwards
Trying it out in a few minutes

-kwag
Reply With Quote
  #3  
04-04-2003, 02:25 PM
ARnet_tenRA ARnet_tenRA is offline
Free Member
 
Join Date: Jan 2003
Location: Illinois, USA
Posts: 73
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by kwag
Maybe instead of using 10% on subsequent runs, it can even be dropped to 1%, because the ratio is always constant
So just one full sample, and then multiple (Fast!) 1% runs afterwards
I considered that too, but haven't tried it yet.

Good luck

Tenra
Reply With Quote
  #4  
04-04-2003, 02:48 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Just tried it with one 100% run and three 10% runs to get CQ. Wanted file size: 12,059.95KB. Full encoded sample after finding CQ: 12,046KB
Have a beer tenra

-kwag
Reply With Quote
  #5  
04-04-2003, 07:02 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Note: Keep the second (or more) runs at 10%. I tried 1%, 2%, etc, and it's not very accurate. So the first run of 100% and subsequent runs of 10% seems to hit a <1% accuracy every time
Another beer for Tenra

-kwag
Reply With Quote
  #6  
04-04-2003, 08:41 PM
muaddib muaddib is offline
Free Member
 
Join Date: Jun 2002
Location: São Paulo - Brasil
Posts: 879
Thanks: 0
Thanked 0 Times in 0 Posts
That's a great and clever idea!
As always, the simpler the idea the more brilliant it is!
Here... take another beer
Reply With Quote
  #7  
04-04-2003, 10:42 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Now we just need hedix to integrate this into ToK, and we'll all be drinking more beer .
Reply With Quote
  #8  
04-04-2003, 10:55 PM
muaddib muaddib is offline
Free Member
 
Join Date: Jun 2002
Location: São Paulo - Brasil
Posts: 879
Thanks: 0
Thanked 0 Times in 0 Posts
Hey hedix, are you listening!?
Hurry... I'm thirsty!!
Reply With Quote
  #9  
04-04-2003, 11:37 PM
jorel jorel is offline
Invalid Email / Banned / Spammer
 
Join Date: Aug 2002
Location: Brasil - MG - third stone from the sun
Posts: 5,570
Thanks: 0
Thanked 0 Times in 0 Posts
@ friend "net_ten":

....


then i read your post again:

i'm in the party now.....fantastic!!!




Kwag,
we need some emoticons with fireworks!!
Reply With Quote
  #10  
04-05-2003, 07:26 AM
hedix hedix is offline
Free Member
 
Join Date: Jan 2003
Posts: 155
Thanks: 0
Thanked 0 Times in 0 Posts
Yep ! one day out, and .... ole!

So the ratio is constant... You're great, Tenra.

I'm a little busy now, but wait a few days and ToK will do the thing.

Best regards,

Hedix
Reply With Quote
  #11  
04-05-2003, 11:39 AM
Holomatrix Holomatrix is offline
Free Member
 
Join Date: Aug 2002
Posts: 130
Thanks: 0
Thanked 0 Times in 0 Posts
Great, can't wait
Reply With Quote
  #12  
04-05-2003, 09:24 PM
Daagar Daagar is offline
Free Member
 
Join Date: May 2002
Posts: 158
Thanks: 0
Thanked 0 Times in 0 Posts
Stupid question, but what are the correct parameters to Sampler() for a 10% clip? In the past, Sampler(length=24) has been the default used for a full size clip... is it as simple as Sampler(length-2.4)? Laf... nevermind, can't use 2.4, it isn't an int.

Update: Okay, after finally realizing SansGrip had instructions included with Sampler(), it appears you just take 10% of the running time of the movie in minutes, changing things to Sampler(samples=<movie time in minutes*.1>, length=24).
Reply With Quote
  #13  
04-05-2003, 10:31 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Hi Daagar,

You can even use something like this:

Code:
interval = round((FrameCount/24)/60)  # Interval spacing ( Full Sample )
#interval = round((FrameCount/24)/60)  / 10 # Interval spacing ( 10% of full sample )
nFrames = round(24) # Frames per sample
SelectRangeEvery( (round(framecount/interval)),nFrames)
This will take one second sample for every minute of the movie. Comment the first line and uncomment the second line for 10% of the full sampler

-kwag
Reply With Quote
  #14  
04-10-2003, 09:10 AM
Daagar Daagar is offline
Free Member
 
Join Date: May 2002
Posts: 158
Thanks: 0
Thanked 0 Times in 0 Posts
Ah, even better! Less chance of bumbling simple calculations Thanks!
Reply With Quote
  #15  
04-10-2003, 10:17 PM
ozjeff99 ozjeff99 is offline
Free Member
 
Join Date: May 2002
Location: Sydney, Australia
Posts: 159
Thanks: 0
Thanked 0 Times in 0 Posts
G'day - I'm in and will get the next round of drinks .
Reply With Quote
  #16  
04-11-2003, 12:37 AM
jorel jorel is offline
Invalid Email / Banned / Spammer
 
Join Date: Aug 2002
Location: Brasil - MG - third stone from the sun
Posts: 5,570
Thanks: 0
Thanked 0 Times in 0 Posts

a simple and stup question:

can someone post a sample script,
or explain me the way to use this,please?

thank you.
Reply With Quote
  #17  
04-12-2003, 02:10 AM
ozjeff99 ozjeff99 is offline
Free Member
 
Join Date: May 2002
Location: Sydney, Australia
Posts: 159
Thanks: 0
Thanked 0 Times in 0 Posts
Hi Jorel

This works for me:
1.Load Avs (without sampler()) and Audio into Tok and start file prediction and peek in log tab to get target video full sample size (TFS). Then exit without finishing. This is a cheats way of not having to calculate it.

2.Use Kwags scripts in this thread above at the end of your avs to give you two samples at say Q=60. One is Full Sample (FS) the other is Small Sample (SS).

3.Calculate SS/FS ratio.

4.Assuming the above ratio is independent of CQ:
SS/FS = TSS/TSF where TSS is the target small sample size
ie. TSS=(SS/FS)*TFS

5.So the problem is one now of playing with the CQ values to give you the correct SS. The great thing is that the SS samples usually take less than a minute. I use a simple linear interpolation formula which is easy to set up in Excell to automatically give me the next CQ to try. There is no need to scale up to full sample size at any time in this process unless you want to re-check at the end that the FS=TFS for the final CQ.

Ecample: The Shining.
1.Tok sais that the
TFS = 10,609 kB
2.Ran Kwag’s full sample script at CQ=60 to get
FS = 9,622 kB
3.As above for small sample to get
SS = 854 kB
4.From above
TSS = 941.6 kB
5.Repeating step 3. for different CQs. At CQ = 64 (3 tries) gave me
SS = 942 kB

For posterity, I ran a FS at CQ = 64 and got
FS = 10,628 kB a 0.01% error !!

Hope this helps. No doubt we will see all this included in next Tok soon.

Cheers
Ozjeff99
Reply With Quote
  #18  
04-12-2003, 02:33 AM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by ozjeff99
No doubt we will see all this included in next Tok soon.
Sooner than an eye blink

-kwag
Reply With Quote
  #19  
04-12-2003, 02:43 AM
ozjeff99 ozjeff99 is offline
Free Member
 
Join Date: May 2002
Location: Sydney, Australia
Posts: 159
Thanks: 0
Thanked 0 Times in 0 Posts
I'm glued to this channel. :P
Reply With Quote
  #20  
04-12-2003, 09:11 AM
jorel jorel is offline
Invalid Email / Banned / Spammer
 
Join Date: Aug 2002
Location: Brasil - MG - third stone from the sun
Posts: 5,570
Thanks: 0
Thanked 0 Times in 0 Posts


seems... simple and fantastic.

thanks ozjeff99

your explanation is cool!
Reply With Quote
Reply




Similar Threads
Thread Thread Starter Forum Replies Last Post
CQ prediction method WOWIEGURL Avisynth Scripting 13 03-12-2005 04:56 PM
TMPEG 2-pass engine. A key to faster CQ prediction. kwag Avisynth Scripting 100 12-18-2003 09:34 AM
New 10 % prediction method ? hedix Avisynth Scripting 9 03-04-2003 07:27 AM
What is the current prediction method for a 29.97fps movie? Paul0889 Avisynth Scripting 0 01-10-2003 07:25 PM
KVCD: file prediction method? edmund Video Encoding and Conversion 0 10-31-2002 12:39 PM

Thread Tools



 
All times are GMT -5. The time now is 01:39 AM  —  vBulletin © Jelsoft Enterprises Ltd