Automated File Prediction?

J-Wo · #1 02-14-2003, 02:20 AM

Does anyone have a good method for automated file prediction using KVCDx3 and CQ? I know there is the acp program in development, but the latest version has a buggy formula for CQ calculations, but is fine with CQ_VBR file prediction. I was just getting tired of my manual method of zeroing in on the CQ and wondered if there was an easier method.

kwag · #2 02-14-2003, 02:29 AM

Hi J-Wo,

The manual method is the best. Once you get used to it, it only takes two or three runs to hit your target. What I do is I first start by encoding only up to 10%, as viewed in TMPEG. If the file size multiplied by 10 is not within range of my target, I make CQ adjustments. This is very fast because you're only encoding 10%, so it's a matter of seconds between tries. When my 10% is close ( just slightly below target ), then I run the full prediction. 99% of the times I hit my target in a couple of tries

-kwag

J-Wo · #3 02-14-2003, 02:37 AM

Wow that sounds neat! So do you just press stop when the Sampler() encoding is at 10%, or is there a more accurate way to tell TMPGenc to stop? Also do decimal points matter with CQ values?

kwag · #4 02-14-2003, 02:47 AM

Quote:

Originally Posted by J-Wo

Wow that sounds neat! So do you just press stop when the Sampler() encoding is at 10%, or is there a more accurate way to tell TMPGenc to stop?

That's right

. Just press stop.

Quote:

Also do decimal points matter with CQ values?

Yes they do matter

-kwag

rendalunit · #5 02-14-2003, 03:09 PM

hi kwag,

That's a great idea to press stop after 10%!

That's one thing that's bugged me about this process is the length of time to encode each cycle. If you put this line trim(0,Round(Framecount/10)) before the sampler() line then it will run the prediction on the first 10% of the movie. I tried it once so far and it was ~10% of the actual prediction.

ren

kwag · #6 02-14-2003, 04:01 PM

Quote:

Originally Posted by rendalunit

hi kwag,

That's a great idea to press stop after 10%!

That's one thing that's bugged me about this process is the length of time to encode each cycle. If you put this line trim(0,Round(Framecount/10)) before the sampler() line then it will run the prediction on the first 10% of the movie. I tried it once so far and it was ~10% of the actual prediction.

ren

Thanks ren,

I just added that line to my script

.
It's easier than chasing the 10% mark and hitting stop

-kwag

J-Wo · #7 02-14-2003, 04:30 PM

Wowwww. I'm in amazement of all you smart people!!!

Kwag in another thread you joined the Prediction() and Sampler() function in one line by separating them by a period. That way we only had one line to hash out when our file prediction was done. Can we similarly combine three functions together with periods?

kwag · #8 02-14-2003, 04:44 PM

Quote:

Originally Posted by J-Wo

Wowwww. I'm in amazement of all you smart people!!!

Kwag in another thread you joined the Prediction() and Sampler() function in one line by separating them by a period. That way we only had one line to hash out when our file prediction was done. Can we similarly combine three functions together with periods?

Prediction("S1M0NE", 1, .98 ).trim(0,Round(Framecount/10)).Sampler(length=24)

urban tec · #9 02-14-2003, 08:00 PM

Cant wait to try this, should cut down some of the time doing prdictions especially when using dust and C3d.

Thanks Ren, Kwag, J-Wo

jorel · #10 02-14-2003, 09:45 PM

J-Wo wrote:

"Wowwww. I'm in amazement of all you smart people!!!"

...yes,everyday new ideas that work!!

2 days without came here and lots of good surprises!

rendalunit · #11 02-15-2003, 10:27 PM

hi kwag,

i changed this line ->trim(0,Round(Framecount/10))

to this ->trim(Round((Framecount/2)-((Framecount/10)/2)),Round((Framecount/2)+((Framecount/10)/2)))

so that it will take the 10% from the middle of the movie instead of the beginning because some movies have long intro credits like "Signs" and "Spiderman" that can mess up the prediction. Then when the cq is in the ballpark then comment out the line and do the "real" file size prediction

ren

kwag · #12 02-15-2003, 11:36 PM

Cool ren

black prince · #13 02-16-2003, 08:09 AM

@Kwag,

Give me a sample avs and some steps, so I can include this technique
in the Guides converting Divx (avi) to KVCD. We can always use a
simplier method.

Thanks

-black prince

kwag · #14 02-16-2003, 10:18 AM

Hi black prince,

Use ren's line: trim(0,Round(Framecount/10)) after Sampler and any .avs script.
Then when you run it, try to get your 10% sample file size a little lower in size than the actual wanted (around 10% too ). I've tried a couple of predictions this way, and when I run the complete sampler, it is very close to my wanted size. If I run the 10% and adjust CQ until I get exactly ~10% file size, when I run the complete sampler, it usually goes over the wanted sample file size.

Give it a try and see if this method works for you too. This is a much faster method to zero in on CQ than having to run 3-5 sampler sessions to get the final CQ

-kwag

Jellygoose · #15 02-17-2003, 09:45 AM

Hi ren and kwag!

I tried your method, using ren's line to take 10% of the sample from the middle, and I didn't get the desired results sorry...

I used it on MIB2 and I calculated 10% of the wanted sample size. After I got the desired size, I encoded the full sample, but the filesize was 1.6MB smaller than it was supposed to be

.
I was able to raise the CQ by 4 until I reached the desired filesize...

any comments on that? did I misunderstand this technique?

J-Wo · #16 02-17-2003, 10:31 AM

Quote:

I used it on MIB2 and I calculated 10% of the wanted sample size. After I got the desired size, I encoded the full sample, but the filesize was 1.6MB smaller than it was supposed to be

I had that same problem too until I realized that you should be using a factor of 0.98 in your prediction formula. For some reason it doesn't work if you use 0.99. Give that a try and see if it works!

rendalunit · #17 02-17-2003, 12:07 PM

At the time it seemed like a good idea to take the 10% out of the middle but now I'm not so sure

I'm switching back to that first one trim(0,round(Framecount/10)) which should be just like aborting the encode at 10% of the way through (dumb question- are long intro credits generally higher bitrate or lower than the rest of the movie? I seem to recall seeing large spikes at the beginning of a movie in bitrate viewer

)

-ren

Jellygoose · #18 02-17-2003, 12:13 PM

@J-Wo : I am using a factor of .98 although I don't think that using .99 would do this difference...

@ren : Well they should use lower bitrates than the rest of the movie, but I recall some weird spikes too, when viewed in bitrate viewer. You can see clearly that at some parts of the intro the bitrate goes up, almost all the way for no obvious reason...

strange! however I'll apply this technique again using your first one...
trim(0,round(Framecount/10)) ...

Fiz · #19 02-17-2003, 01:03 PM

Guys,

I've been working on the idea of getting 10% samples with a better mix of the overall film content. Here is something I am trying. It seems to work fine to as close to 10% of the total frames that sampler generates.

Code:

Function Sampler10percent(clip c, int "length")
{

# This formula should maintain the 10% of sample size regardless of the length parameter.
# The length parameter should be reduced to get a better snapshot of samples 
# (e.g. 12 would give 12 frame samples from each area of the movie and cover more scene 
# detection).

length=default(length, 24)

movielength=(framecount(c) / framerate(c)) / 60

samples = movielength * length / 10

length2= round(samples / 10)

oldfps=c.framerate

return c.AssumeFPS((oldfps/24) * (length2 * 10)).
\Sampler(length=length2).
\AssumeFPS(oldfps)
}

Add this function to the bottom of your AVS and instead of doing
Sampler(length=24)
do
Sampler10percent(length=24)

What I have found is that with my dodgy calculations

, you get an arbritary number of frames for each part of the movie (but close to 24). I do this by fooling sampler with the total fps of the movie!! Once sampler is run the fps are returned to normal. (Confused

... so was I for ages!)

The formula is not finalised as I am sure there is a simpler way of doing what I am trying. I hope you guys understand what I am attempting and perhaps can add to it (if you understand it).

I have also found that reducing the length parameter will still give 10% of the sample but with an even better mix of content.

Please test this and you should see as close to 10% of the frames that sampler produces and you will get a better mix of samples from the movie. If it works then

otherwise lets keep trying!!!

rendalunit · #20 02-17-2003, 06:33 PM

fiz that's very clever how you did that! I don't why though it's further from 10% of filesize of sampler(length=24) than trim the first 10%

because theoretically this should be more accurate because it's distributed evenly through the whole thing. Also I don't think the sample length changed from 24 because I put the line ShowFrameNumber and the frame numbers were like this (0..23), (15465..1548

, (30930..30953)...

I'm running a full encode of K19 to see if it's closer to sampler(length=24) or sampler(length=24) called from sampler10percent(length=24)

good job!
ren

edit: oops I just read that you said once sampler is run than the fps returns to normal

this is too complicated for me