Quantcast Multi Pass Size Prediction - digitalFAQ.com Forums [Archives]
  #1  
07-03-2003, 10:08 AM
girv girv is offline
Free Member
 
Join Date: Sep 2002
Posts: 108
Thanks: 0
Thanked 0 Times in 0 Posts
Send a message via ICQ to girv
A new twist on file size prediction maybe? I've been encoding using this method for a while now and haven't seen mention of it on the forums, though its so obvious I'm sure others are using it too!

Its only for people with fast machines and/or plenty of time because it requires two full encodes of the video, but the results are great - average 0.18% error on the 50+ encodes I've done this way.

Its so easy - you determine the correct prediction ratio by encoding the full source and seeing what the ratio of predicted size to encoded size actually is, then predict/encode again using that ratio. I guess you could do more passes to improve the accuracy futher, but 0.2% is a 1.6Mb error on an 800Mb target so I figure theres not much point!

Anyway, I'm thinking this could help with the current problems predicting motion adaptive encodes - I haven't tried it as Im not doing any MA stuff myself (yet?).

I'd really like to see this included in ToK :)

/girv
Reply With Quote
Someday, 12:01 PM
admin's Avatar
Site Staff / Ad Manager
 
Join Date: Dec 2002
Posts: 42
Thanks: ∞
Thanked 42 Times in 42 Posts
  #2  
07-04-2003, 06:35 PM
whitemark whitemark is offline
Free Member
 
Join Date: Sep 2002
Posts: 11
Thanks: 0
Thanked 0 Times in 0 Posts
Hi! girv

I guess i'm doing something like this too. Here is what I do, full encode with no filters and then a sample encode with the same CQ and no filters too(it's faster with no filters). Then I get a factor between then wich i use to multiply by the new sample size (with filters) till i get the final desired full size(something like the Tenra method). The problem is to get this new good sample size, i sometimes do 6 or 7 samples to get there, i guess it's easier to let tok do the job but with PAL doesn't seen to work very well at least for me.
Reply With Quote
  #3  
07-04-2003, 07:10 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Have you tried the new method that jorel tested
That is, you select ( 3*frame rate ) the length of sample
So for 23.976fps, you select 72, for PAL 25fps you select 75 and for NTSC 29.97fps encodes you select 90. And keep one sample per minute.

-kwag
Reply With Quote
  #4  
07-05-2003, 03:18 PM
whitemark whitemark is offline
Free Member
 
Join Date: Sep 2002
Posts: 11
Thanks: 0
Thanked 0 Times in 0 Posts
Thanks Kwag but i think i'll stay with this method for a while, i'll keep reading feedback of pal users using tok. By the way, im only doing mpeg2 because my player only suports mpeg1 at 352x288 (bluesky DS-8315), and i've done all kinds of testing, including svcd header tricks. So i use 480x576 and vbv must be set to 112. When using your video.en1 i get an error something about pal source doesn't need to 4:3 pulldown, can't remember exacly, so i've got to replace:
Job.MPEG.Video.VideoEncodeMode = MPEGVideoEncoder_VideoEncodeMode_Progressive_32Pul ldown

by the original:
Job.MPEG.Video.VideoEncodeMode=MPEGVideoEncoder_Vi deoEncodeMode_Interlace

and then everything was ok, except prediction...

ps: reading english is so mutch easier then writhing...
Reply With Quote
  #5  
07-05-2003, 03:25 PM
whitemark whitemark is offline
Free Member
 
Join Date: Sep 2002
Posts: 11
Thanks: 0
Thanked 0 Times in 0 Posts
Has you ntsc guys notice is not 4:3 pulldown but 3:2 pulldown i guess we pal users don't realy see the diference...
Reply With Quote
  #6  
07-07-2003, 06:21 AM
girv girv is offline
Free Member
 
Join Date: Sep 2002
Posts: 108
Thanks: 0
Thanked 0 Times in 0 Posts
Send a message via ICQ to girv
@whitemark: how accurate is your method ?
Reply With Quote
  #7  
07-07-2003, 03:42 PM
whitemark whitemark is offline
Free Member
 
Join Date: Sep 2002
Posts: 11
Thanks: 0
Thanked 0 Times in 0 Posts
I guess it could be very accurate but i'm still having problems with the formulas, i've only done 4 complete encodes until now, max diference in video was 6mg but two of them where above size. What i can tell you for shure is that the first pass can not be done whithout filters, and that's bad news because in my p4 2.4 at work it was faster than real time. I'll post if i get somewhere or i'll send you my excel file if you want, just give me your mail.
Reply With Quote
  #8  
07-07-2003, 03:57 PM
jorel jorel is offline
Invalid Email / Banned / Spammer
 
Join Date: Aug 2002
Location: Brasil - MG - third stone from the sun
Posts: 5,570
Thanks: 0
Thanked 0 Times in 0 Posts
@ girv and whitemark

post your ways to find predictions, will be good for us all test it too.
the target is perfect predictions.
changing ideas and testing we can get the best.

thanks
Reply With Quote
  #9  
07-08-2003, 12:01 PM
girv girv is offline
Free Member
 
Join Date: Sep 2002
Posts: 108
Thanks: 0
Thanked 0 Times in 0 Posts
Send a message via ICQ to girv
@whitemark: after many, many, many hours of tests I've found that the only way to be accurate <1% is to have the first and second passes being as identical as possible. That means the same filters, the same number of frames and so on for each pass. Any difference you introduce means you are approximating something and at the tolerances we are dealing with here everything makes a difference.

@all: FWIW I've completed two more 2-CD encodes recently using my two pass method - all four encodes of about 1 hour of video each finished with <0.2% error in predicted vs. final m1v size.

Has anyone tried this with the MA script to see how accurate it is under those conditions? I'm using my own non-adaptive script optimsed for divx/xvid sources so I haven't tested Kwag's DVD optimal scripts yet.
Reply With Quote
  #10  
07-08-2003, 03:40 PM
whitemark whitemark is offline
Free Member
 
Join Date: Sep 2002
Posts: 11
Thanks: 0
Thanked 0 Times in 0 Posts
Ok, I think I've got something here!!

So you're one of this guys who use Tok and got a file 50mg smaller (or bigger)? Well you can still use it for something. Write down the full video size your file have, then do another pass whith the same settings in tmpeg (including CQ) but adding the sampler function to your script, then divide (full video / sample video = factor).
Then calculate how much video(Kb) your cd can take, something like (video size=(full cd size/1.015)-audio size), I'm using 1.015 for muxing proposes.
Then calculate the real sample video you should have in Tok by doing
(sample video=video size/factor). Now get back to tok and play with "factor for prediction" until you see the correct sample size in the log tab (you must start tok to see this). Don't forget tok is using bytes.

Here is my last encode:

Fast & the furious 1:42:27 pal, using mpeg2 480x576, MA script date 6/7/2003, all values are in kb

full video 1st pass = 723968 (CQ 60)
sample video 1st pass = 13312 (CQ 60)
factor = 54.38
total cd size = 813000
audio size = 84055
video size = 716772
sample video size = 13180 (13496004 bytes for tok)

Tok settings:
New faster prediction = cheked
Precision% = 0.50
Speed up by = 10 times
Factor for prediction = 1.10 (after a few tries)
Length of sample = auto
CQ predicted by tok = 57.3
Final video size = 710650
Final size after muxing = 807028 (total cd size = 813000)

I hope my next encode come out as good as this one. I guess this will only be usefull for pal users, I think tok is doing a god job for ntsc. If you're not doing the first pass with tok like me, keep in mind that tmpeg settings for 1st pass must be the same tok is using (GOP, detect scene change, soften block noise...).

I guess that's all, waiting for feedback...

ps: It took me more time to write this than my fist pass...
Reply With Quote
  #11  
07-09-2003, 09:21 AM
girv girv is offline
Free Member
 
Join Date: Sep 2002
Posts: 108
Thanks: 0
Thanked 0 Times in 0 Posts
Send a message via ICQ to girv
Quote:
Originally Posted by whitemark
full video 1st pass = 723968 (CQ 60)
sample video 1st pass = 13312 (CQ 60)
factor = 54.38
What factor did ToK report in its log? ie: what is the ratio of movie framecount to sample framecount? I wonder if you could calculate your second pass prediction factor (1.10 in your case) from tok-framecount-ratio / firstpass-video-size-ratio (or maybe the other way around :) ?

Taking a guess at what your tok-framecount-ratio would be (60.4 ?) I get: second-pass-prediction-factor = 1.00 * (60.4 / 54.38) = 1.112 so its pretty close at least :)
Reply With Quote
  #12  
07-09-2003, 02:58 PM
whitemark whitemark is offline
Free Member
 
Join Date: Sep 2002
Posts: 11
Thanks: 0
Thanked 0 Times in 0 Posts
@girv

I did the 1st pass manualy so i used sampler() that means the framecount factor will always be 60 for pal (I think) but i still have got the numbers

Total frames (1st pass)= 153675
Sample frames (1st pass)=2561

So 153675/2561=60.00585708

That means 60/54.38=1.1033 so, I guess... IT WORKS!!

I really, really hope you're right girv because the start and stop of tok was the worse part of the process. I'm going to start a new encode to see if it this thing is really true.

Thanks man
Reply With Quote
  #13  
07-10-2003, 03:08 PM
whitemark whitemark is offline
Free Member
 
Join Date: Sep 2002
Posts: 11
Thanks: 0
Thanked 0 Times in 0 Posts
Ok just posting the confirmation...

The four white fethers - 352x288 pal mpeg1 2:05:53

Wanted final video = 696820
Final video after encoding = 695355

I guess there's nothing else to say...
Reply With Quote
  #14  
07-11-2003, 08:39 AM
GFR GFR is offline
Free Member
 
Join Date: May 2002
Posts: 438
Thanks: 0
Thanked 0 Times in 0 Posts
I know that's a lot of work, but,...

If you gradually reduce the size of the 1st pass to see how much you can speed it up and still get accurate results, this would save time for the next encodes...

Let's say, for the first pass you try 1/2 of the movie, then 1/3 of the movie, etc. When the factor you get diverges too much from the factor you got for the full movie, that's the limit. Then when you do another movie, use can use just the necessary fraction of the movie, and save time over a full movie 1st pass.
Reply With Quote
  #15  
07-14-2003, 04:51 PM
whitemark whitemark is offline
Free Member
 
Join Date: Sep 2002
Posts: 11
Thanks: 0
Thanked 0 Times in 0 Posts
@GFR

I'll try to test it, still learning how to choose 1/2 of the movie with sampler()...
Reply With Quote
Reply




Similar Threads
Thread Thread Starter Forum Replies Last Post
Bitrates: How does it do size prediction? Shibblet Video Encoding and Conversion 2 09-09-2004 05:05 PM
Size/CQ curve: a one pass CQ estimation method fabrice Avisynth Scripting 53 01-27-2004 10:07 PM
TMPEG 2-pass engine. A key to faster CQ prediction. kwag Avisynth Scripting 100 12-18-2003 09:34 AM
Encoding: A cheap multi-pass VBR CCE has arrived! digitalize Video Encoding and Conversion 18 07-15-2003 07:44 PM
File Size Prediction For PAL... Jellygoose Avisynth Scripting 8 01-01-2003 09:18 PM

Thread Tools



 
All times are GMT -5. The time now is 11:17 AM  —  vBulletin Jelsoft Enterprises Ltd