digitalFAQ.com Forums [Archives]

digitalFAQ.com Forums [Archives] (http://www.digitalfaq.com/archives/)
-   Avisynth Scripting (http://www.digitalfaq.com/archives/avisynth/)
-   -   TMPEG 2-pass engine. A key to faster CQ prediction. (http://www.digitalfaq.com/archives/avisynth/6202-tmpeg-pass-engine.html)

kwag 10-19-2003 11:51 PM

TMPEG 2-pass engine. A key to faster CQ prediction.
 
Hi all,

First of all, **** THIS IS EXPERIMENTAL (and not for newbies!) ***
What we are about to do, is to use TMPEG's 2-pass engine as a "helper" for CQ encoding.
Today, I said to myself: If TMPEG can properly calculate the final file size of a movie by doing 2 passes, then why can't I do a small 2-pass (20 seconds or so) encode, which will have the correct average bitrate, and use the encoded file size as a reference footprint to encode the same clip in CQ mode, and tune the parameters until I achieve the same footprint :!:
Then, in theory, the full encode (in CQ mode) should be identical in final file size.
So let the game begin :)

Load your source into CalcuMatic, and get your average bitrate as usual for the movie to be processed.
Now follow this steps:

(1) Encode about 20 seconds or so (selected source range) of a mixed high action/normal action clip, anywhere in your movie where you have some of the highest visible action, as 2-pass VBR with MIN bitrate 300 and MAX=Your Max (2,500 for me), and the average bitrate that CalcuMatic gave you.
(2) With Bitrate Viewer, open the encoded sample and note the PEAK bitrate and the file size.
(3) Change to CQ mode, and set the CQ value to 100.
(4) Encode the same source range, and start lowering the MAX bitrate until the PEAK bitrate is as close as possible to the one recorded on the 2-pass.
(5) Now encode the same clip, and start lowering CQ value until you get as close to the same file size as the one created with the 2-pass encode.
(6) Once you get a close file size, you've found your CQ, and now you set back your MAX bitrate to your usual 2,000Kbps or 2,500Kbps, and encode your movie with the CQ found.

Your final file size should never exceed the maximum calculated by the average bitrate.
When we found the CQ, we return the MAX bitrate to the high value we normally use, thus increasing the distance of MAX bitrate from CQ value. This will now expand the dynamic range of the encoder, and give you lower file size on average scenes and higher bitrate peaks on high action scenes.
But the final file size should never exceed the calculated file size, because we calculated a "worse case" scenario on a high action scenes.

Note: If your movie's highest action doesn't go too much above the calculated MAX during the 2-pass run, you might want to leave the MAX bitrate to the value found on step 4. This will guarantee that the final file size will be right on target, just like if a 2-pass VBR encode had been done. But you will still have better quality with the CQ encode, because of the CQ bitrate distribution.

If this method proves to be correct, after some more tweakings, then the next version of CQMatic will do this automatically. So bye bye to long CQMatic loops trying to find the correct CQ value. Let's use the 2-pass encoding engine that TMPEG provides ( in a way it was never designed :!: ), and we should have a correct CQ value in less than 10 minutes :D
Please let me know your results :cool:

-kwag

Gamecraze 10-19-2003 11:58 PM

Speed is always good, like they say, Time is money.

Krassi 10-20-2003 01:17 AM

Sounds good, Kwag.
I'll do a test this evening (PAL :roll: ).

kwag 10-20-2003 01:23 PM

Fine tunning
 
Finished my first encode ( Red Planet at 704x480 ), with a file size of ~50MB lower than wanted ( I did expect that, but quality is still excelent ).
Now I'm compensating by using a MIN bitrate of 25% lower than average (not 300Kbps). This will push the CQ curve up, and increase the average bitrate on low lighted/low action scenes.
Currently encoding K19 at 352x240 ( so I can see the results faster )
Then I'll encode Red Planet again, with the corrections, to see if the results are consistent.

-kwag

azel 10-20-2003 02:10 PM

Funny :wink:
Yesterday I had some of the same thoughs....
Videoencoding improvments????

kwag 10-20-2003 02:14 PM

Quote:

Originally Posted by azel
Funny :wink:
Yesterday I had some of the same thoughs....
Videoencoding improvments????

But your thoughts were 12 hours after I posted. Look at the time stamp ;)
But I read your post, and it's not the same thing :)

-kwag

kwag 10-20-2003 04:07 PM

Finished encode of K19 at 352x240.
Wanted size: 703,106KB
Final Encoded: 687,563KB
Things are looking very good :)
Currently encoding "The Bourne Identity" at 528x480.
4 hours to go.....

-kwag

incredible 10-20-2003 04:14 PM

I'd like to know if there will be a viewable quality advantage by doing this in comparison to our CQmatic way.
8O :D

kwag 10-20-2003 04:19 PM

Quote:

Originally Posted by incredible
I'd like to know if there will be a viewable quality advantage by doing this in comparison to our CQmatic way.
8O :D

Quality is the same, because we're still encoding in CQ mode :D

-kwag

kwag 10-20-2003 08:37 PM

More final tweaks.
 
"The Bourne Identity" at 528x480 done.
Wanted size: 719,786KB
Encoded: 745,296KB

3.4% over. Now to do some minor adjustments with MIN bitrate.
I used 25% under average for MIN. I'll now test with 30% under average for MIN ( for safety margin ).
Now on to encode "Red Planet" at 704x480......

-kwag

fabrice 10-21-2003 12:03 AM

Hi,

I tried that on "the beauty and the beast", with those values:
2 pass VBR :
- res 704x576
- min: 300
- max: 1800
- Aver.: 1089
And it gave me a 1174 peak (and this is the highest motion scene, as I got macroblock in my first encode just right there).
Filesize: 3909Kb.

Swith to CQ, and I had to lower the max bitrate until 1200 to get this max. bitrate value (with 1800, it give me a peak of 1600).

The CQ I encode this movie before was 56,6, but with a 1200 max. bitrate value, the 60 CQ value give me a 3605 Kb file, which is a way smaller than the reference sample.

I'll try again to see where is the error...

Fabrice

kwag 10-21-2003 12:19 AM

Hi Fabrice,

After you get the same MAX bitrate in CQ as you got in the 2-pass run, then you start to lower the cq value until you get the same file size that you got in the 2-pass. After you get that, you set your MAX again up to your 1,800Kbps and encode your movie.

-kwag

Krassi 10-21-2003 04:23 AM

Re: TMPEG 2-pass engine. A key to faster CQ prediction.
 
Quote:

Originally Posted by kwag
(4) Encode the same source range, and start lowering the MAX bitrate until the PEAK bitrate is as close as possible to the one recorded on the 2-pass.
(5) Now encode the same clip, and start lowering CQ value until you get as close to the same file size as the one created with the 2-pass encode.

Hi Kwag,

i'm having some adjust problems here.
Some infos:
Movie k-pax (Source is KDVD), 20 seconds Source Range, average bitrate from CalcuMatic 794.
2-pass (old style) is giving me a Peak of 1180, Average 782.
Now there's my problem in finding the correct Max-value to achieve a peak of 1180. From 400-850 @max bitrate there's no change in peak, it's always 1192 :roll:
Min bitrate is 350.
2-pass filesize is 2.128KB.
When i'm doing an CQ-encode @maxbitrate=850 with the same source range the CQ doesn't matter, file size remains the same. Only if i change max to 400 there's a siginificant change in filesize.
Any change in CQ is ignored, file size remains the same.
Can you suggest anything, please :?:

EDIT: Maybe the problem is the already compressed KDVD :roll:

kwag 10-21-2003 06:18 AM

Hi Krassi,

Did you pick a high action scene for source range :?:
On K-Pax, maybe the part where everyone is running to see the Bluebird :idea:

-kwag

Krassi 10-21-2003 06:21 AM

Quote:

Originally Posted by kwag
Hi Krassi,

Did you pick a high action scene for source range :?:
On K-Pax, maybe the part where everyone is running to see the Bluebird :idea:

-kwag

Its not easy with that movie :lol: I'll try again with another and longer source range.

Krassi 10-21-2003 06:36 AM

I'll scan the whole source movie with bitrate viewer to find the time where the highest bitrates are used :idea:

kwag 10-21-2003 06:37 AM

I'm thinking something else right now, and I'm about to try it :!:
I'm going to try a 2-pass of the slicing that CQMatic creates. That is, the 2 minute slice of the movie. Then, that will leave a sample of a correct size ( properly calculated file size, based on average bitrate ), and then I'll run prediction with CQ, to target that file size.
So it will be one 2 minute 2-pass cycle, and the next cycles as usual, with CQ.
I think the 2 pass needs more footage than 20 seconds to properly map the material.
So with the 2 minute chunks that CQMatic creates, this should give a better accurate mapping of the source.
And this way, there's no fumbling with peak bitrates, etc., and can easily be automated with minor modifications to CQMatic.

Edit: You can try this yourself by running CQMatic as usual. Load your project file, etc., and after clicking on "Execute", start the process and cancel it. Kill TMPEG. Then manually load the file "CQMatic.tpr" in the directory where you have CQMatic installed.
That file will already have your project in time slices. So just use 2-pass (old) to create your sample, and then change to CQ and try to match the file size.


-kwag

incredible 10-21-2003 07:02 AM

Gentlemen ...
Im right now in the office so I can't actively attend this workout.
But I read that sometimes in here its difficult for some people to "catch" useful scenes in TmpgEnc to get the "peak" parts.

Open your import d2v-.avs in Vdubmod .... cut out everything exept about 20 sec of some "peak" szenes and go to the editor to add the "trim"'s to your existing .avs ... paste the trims there and open this again in tmpg to make your "see the peak" encodings.

I did this before too, but with another intention, ... to find out the maxbitrate for CQ mode and to do a prediction by using the known CQ matic way.

So maybe this gives precise scenes to figure it out? Hmmmm :roll:

Krassi 10-21-2003 08:07 AM

Quote:

Originally Posted by kwag
Edit: You can try this yourself by running CQMatic as usual. Load your project file, etc., and after clicking on "Execute", start the process and cancel it. Kill TMPEG. Then manually load the file "CQMatic.tpr" in the directory where you have CQMatic installed.
That file will already have your project in time slices. So just use 2-pass (old) to create your sample, and then change to CQ and try to match the file size.


-kwag

I've predicted a CQ of ~77 this way (targetting for 1600 MB). I'll post the results of the final encode later.
I'm testing without filters to decrease encoding time.

kwag 10-21-2003 08:08 AM

Hi incredible,

With the "Short 2-pass/Multiple CQ" cycle I just described, there is no need to find peaks :)
I'll implement this new technique into CQMatic soon. I'm just double checking that the manual process gives accurate results.
I theory, a dual pass (if properly implemented by the encoder) will give an accurate final file size.
So that's why I came up with the idea of implementing a dual pass on the first CQMatic cycle, and then resume the following cycles as CQ, matching the file size found by the dual pass.
This way, CQMatic will have a result much more accurate than a single random piece of a movie.
This should work :!:
At least in theory :roll:
If it does, I'll roll out a new CQMatic version in a matter of hours :mrgreen:

-kwag

kwag 10-21-2003 08:11 AM

Quote:

Originally Posted by Krassi
I'm testing without filters to decrease encoding time.

Me too :D
Final file size shouldn't matter with or without filters, because of the dual pass accuracy :cool:
Only CQ value will change.

-kwag

nicksteel 10-21-2003 09:49 AM

Quote:

Originally Posted by kwag
Quote:

Originally Posted by Krassi
I'm testing without filters to decrease encoding time.

Me too :D
Final file size shouldn't matter with or without filters, because of the dual pass accuracy :cool:
Only CQ value will change.

-kwag

:?: Do you think this could improve accuracy for MPEG2 prediction?

I always exceed the desired file size using CalcuMatic data into CQmatic for MPEG2 files. :lol:

kwag 10-21-2003 10:34 AM

Quote:

Originally Posted by nicksteel

:?: Do you think this could improve accuracy for MPEG2 prediction?

It shouldn't matter what is encoded, because the accuracy is now determined by the first dual pass.
We'll see how accurate this works. I'm currently less than an hour away from finishing a 352x240 encode of "Sum of all fears", and I also have a current encode (on hold) of "Red Planet" at 704x480.
The latter is on hold, because it takes too long to encode. So I'm testing the 352x240, and then I'll continue the 704x480. My P4 is just too slow to handle two concurrent TMPEG sessions :lol:

-kwag

kwag 10-21-2003 04:01 PM

A chat on "Skype" with Krassi, and his findings.
 
Information relevant to the new prediction:

[1:42 PM] krassi10: Hi Karl, can you please call me when you have some time?
[3:01 PM] krassi10: Hi Karl,
CQMatic gave me a final CQ of 63.17, before it was 35 ; -)
I'm retesting this with a bigger source range.
[3:15 PM] kvcdnet: Hey Marc, did you use 2-pass(old) or 2-pass(new)? Because all tests I did, were using "old". I got different results when I used "new" 2-pass!
[3:18 PM] krassi10: I used the new one. I'm on my 3rd run with the old one now. Forgot that Plus is having the new one as default. Doing a test with a sc of 1.40 min and 10 min. But now i see that the filesize is still bigger with a CQ of 40 and thats a nogo...
[3:20 PM] kvcdnet: Is the current test with the "old" 2-pass ?
[3:20 PM] krassi10: Yes, with padding enabled.
[3:20 PM] kvcdnet: :(
[3:20 PM] kvcdnet: :( :( :(
[3:24 PM] krassi10: Keep smiling Karl, you have done so much great things. Have you tried it with CQ_VBR?
[3:26 PM] kvcdnet: No. Not yet. I'm still focused on the 2-pass thing. Because 2-pass is predictable, I still believe I can "steal" something out of the 2-pass prediction, and apply it to CQ.
[3:27 PM] krassi10: Gee, CQ_VBR @60 is giving me a filesize of 53,744KB (should be 20,471).
[3:28 PM] kvcdnet: =:O
[3:31 PM] kvcdnet: I think the problem is clear. When we make a 2-pass VBR sample of X size, the average bitrate will ALWAYS be the one we selected. Doesn't matter if it's a 1 minute clip, or 10 minute clip. But that is the average of the total sum of the movie. Not independent chunks. So we can't target the same file size with CQ as with 2-pass on a little (small) sample. It must be much larger.
[3:34 PM] krassi10: Yeah. Maybe a dumb question: And if we would calculate the average bitrate only on the source range?
[3:37 PM] kvcdnet: Now that you mention it, we can instead of trying to match the file size that 2-pass made with CQ, we can try to encode in CQ and "match" the same average bitrate !!!!! So drop the 2-pass sample into bitrate viewer and note the average. Then encode with CQ, and start lowering CQ value until the same average is reported.
[3:37 PM] kvcdnet: DOes that sound logical?
[3:38 PM] krassi10: Still reading and trying to understand... Give me two minutes... good that you have written it down...
[3:38 PM] kvcdnet: :)
[3:39 PM] krassi10: I think its worth a try... I'll do one.
[3:41 PM] kvcdnet: If you get about the same average, but there is a substantial difference in file size, then that's it! But in theory, no matter how you encode, if you have two files with the same average bitrate, the file size should be about the same. It's just the way the "bits" are allocated.
[3:51 PM] kvcdnet: Finished Red Planed at 704x480. Wanted: 729,726KB. Encoded: *** 725,003KB ***
[3:51 PM] kvcdnet: I think I know what the problem is!
[3:51 PM] kvcdnet: What was your average, MIN and MAX bitrates????
[3:52 PM] krassi10: Hmm. Optimal CQ (from CQMatic would be 63, with a CQ of 60 bviewer is giving me an average of 2318 and VB had 1619.
Min: 1000 Max: 5000 (KDVD), thats what i thougt before.
[3:53 PM] kvcdnet: Ok. What was PEAK (max) bitrate you got in your 2-pass sample?
[3:53 PM] krassi10: Great Result! CQ of 40 is giving an average of 1807
[3:54 PM] krassi10: Peak is 5033 from 2-pass
[3:54 PM] kvcdnet: Is CQ of 40 about the correct value you need?
[3:55 PM] krassi10: No, it would be 63.
[3:55 PM] kvcdnet: So then you have to increase CQ to match the needed average :)
[3:55 PM] kvcdnet: Let me do a quick test here too....
[3:57 PM] krassi10: Sorry, i meant 63 would be the optimal CQ, but then average bitrate is way off >2300
[3:58 PM] kvcdnet: What average do you get on that sample if you encode with CQ=63?
[3:58 PM] krassi10: Currently encoding...
[3:59 PM] krassi10: Maybe we can find a constant multiplier here, but i don't expect this...
[3:59 PM] kvcdnet: Not with this crazy CQ (un)linearity :)
[4:01 PM] krassi10: CQ=63: Average 2357, Peak is 4772.
[4:04 PM] kvcdnet: Marc, check this out. I just checked the average of my 2-pass sample and the average of the CQ sample I created, and the averages were almost the same. This is the one I got almost perfect result (Red Planet at 704x480). Could you check the average of your 2-pass against the average of your CQ sample ( with the CQ that you got ~980MB instead of 1.4GB ) and see if the averages of both samples don't match?
[4:07 PM] krassi10: I've overwritten it but i'll redo a sample with CQ=35 (what resulted in that file). However i've changed source range but that shouldn't matter.
[4:09 PM] krassi10: Results with CQ=35: Average bitrate:1708, Peak:3609
[4:10 PM] kvcdnet: Are you using "source range", or are you encoding your 2-pass using the file "CQMatic.tpr" ??
[4:10 PM] krassi10: This time i used source range, before it was using the time slices from CQMatic.
[4:11 PM] kvcdnet: Ok. I'll do more tests here. Just trying to find a constant!
[4:13 PM] krassi10: I'm doing a prediction on my second workstation with min=1300 and max=2000 (VB and CQ of course). I think its coming from TMPGEnc's variable bitrate:
Normally its nearly sticked to the average, CQ isn't. Thats why there's such a difference in my tests.
[4:15 PM] kvcdnet: Right! There are not that many fluctuations between low and high bitrate. That's why CQ is better that 2-pass, at least with TMPEG :)
[4:17 PM] krassi10: BINGO. With the latest settings on my second pc theres only a file difference of less than 1000KB with a CQ of 60, before the size was nearly doubled.
Maybe you can make a test with wide-spreaded min-and-high-bitrates.
[4:18 PM] kvcdnet: I think that's the main problem!. The further apart are MIN and MAX to average, the worse the results.
[4:19 PM] kvcdnet: So maybe we should raise MIN if we raise MAX, and keep MIN bitrate at a constant percent below average bitrate (idea!)
[4:20 PM] krassi10: Yes, but personally i want to have 3 movies one on KDVD and the final quality suffers if i'm adjusting min too high, but you're right!
BTW: Nice chat!
[4:22 PM] kvcdnet: :)
[4:22 PM] kvcdnet: Doing a wide MIN,MAX test....
[4:24 PM] kvcdnet: SO, in theory, the higher the resolution, the wider can be the MIN and MAX range. That's why my 704x480 was exact, but my 352x240 was lower in file size!.
[4:25 PM] krassi10: Why can't players do 1600x1280 ;-)
[4:25 PM] kvcdnet: So we must "scale", or find the MIN and MAX bitrate "borders" for different resolutions.
[4:26 PM] kvcdnet: Could you repeat the test you did with the .tpr, but at 704x576 and MIN=300, MAX=2,500?
[4:26 PM] kvcdnet: 1600x1280, I wish :)
[4:27 PM] krassi10: I'll redo with these settings
[4:28 PM] kvcdnet: If you can hit final file size target with that resolution and with those MIN/MAX values, then we know that for 704x or 720x, MIN of 300 and MAX of 2,500 will always be correct!
[4:30 PM] kvcdnet: It's going to be a circus, trying to find the optimal MIN and MAX for 352x240, 352x288, 352x480, etc, etc :( LOL
[4:31 PM] krassi10: Yes, and i'm the clown searching for the KDVD-settings : -)
[4:31 PM] kvcdnet: No, we both are LOL :) :) :D
[4:32 PM] kvcdnet: I hope this is worth it!, and not another "CQ down the drain" attempt :(
[4:37 PM] krassi10: Mee too. I have some time while encoding, so here something funny. Me and my friends were on vacation near Calpe (Spain). Some spanish girls talked to us. One of my friends is indonesian. They asked where we come from, we all said "from Stuttgart", except the indonesian. He said "me too". The girls understood Mi-Tu and asked, where this is located ;-) LOL
So i decided to learn spanish but wasn't very succesfull till now.
[4:37 PM] kvcdnet: LOL :D
[4:38 PM] krassi10: BINGO, filesize is right on target with 704x576 @300-2500. I'll try it with 720576 now.
[4:38 PM] kvcdnet: YES :D:D:D:D:D
[4:39 PM] kvcdnet: It should be good at 720x too. We might be able to increase to at least ~2,800 MAX for 720x (idea)
[4:40 PM] kvcdnet: 'm trying a 352x240 now, and setting MIN to 300 and MAX to 1,500. My previous encode had 1,800 as MAX, and final file size was a little low. So probably with 1,500MAX will hit target.
[4:41 PM] krassi10: But i'm still the clown concerning KDVD. I've tried with MIN=2000, Max=5000, but filesize is 13MB bigger. I'll try with 350-5000.
[4:44 PM] kvcdnet: It might be a problem with such a "wide" range! Maybe a true 2-pass full encode is the only way to accomodate such large bitrate fluctuations :\
[4:47 PM] krassi10: Yep, it's just a try. 720x576 is hitting also!!!
[4:47 PM] kvcdnet: I'll have 352x240 results in a couple of minutes.
[4:50 PM] kvcdnet: Have you noticed that Bitrate viewer gives lower MPEG-1 average bitrate than if viewed with VirtualDub?
[4:51 PM] krassi10: Yes, i've already noticed that. Thats really curious, i think it's the same with mpeg2. But the question is: which one is right ;-)
[4:52 PM] kvcdnet: VirtualDub. I manually calculated an average bitrate, and Vdub is the correct valu.
[4:54 PM] kvcdnet: I'm going to do a scale, by increasing MAX bitrate, until I find the "Roll off" point of every resolution. I think we don't have to worry too much with MIN bitrate, because the problem is mainly theMAX bitrate. So I'll figure a MAX bitrate to be used with every resolution.
[4:54 PM] krassi10: I'm a bit tired now,Kwag. I'm sorry. Do you have something important to test? If not, i'll be offline in a couple of minutes. But i think we thats means you have found a good way for alternate prediction. If you want to, you can post this chat into the thread.
[4:55 PM] kvcdnet: Hey, go to sleep Marc :) It's 11:PM your time!. Hopefully, I'll have a table of MAX bitrates for tomorrow, and your name will be in lights in the forum, for the findings you did :)
[4:56 PM] kvcdnet: Good idea! I'm Copy&Pasting this into the forum now! See you tomorrow :)
[4:57 PM] krassi10: Thanks Kwag. The spotlight should be on you. Good night. I think i'll login tomorrow evening.

fabrice 10-22-2003 12:18 AM

Hi,

It explains why I was so 'out' of target: I'm using a 1800 max bitrate...
I'll try with 2500...

CU
Fabrice

Krassi 10-23-2003 03:11 AM

After some more tests i've figured out that a long source range seems to take away the problem of high bitrates.
Currently testing my 3rd movie with a long sr. Manually calculated CQ was always nearly the same as predicted CQ-value with CQMatic.
I've used a bitrange range of 350-4000 8)

EDIT: @fabrice: Can you try it also with a longer sample size (source range), please :?:

Krassi 10-23-2003 07:00 AM

With the source i used in the chat with kwag i did a test with a source range of 4:49 minutes.
The correct CQ with this sc is about 66. And thats what the optimal CQ should be.
CQMatic gave 63, i'm doing a full encode to test which one is right :)
(Bitrate 500-4000)

EDIT: Here the file sizes:
VB: 141.418KB
CQ@66.2: 141.965KB

Krassi 10-23-2003 08:27 AM

Final File size with CQ=66: 1.35 GB, wanted was 1.4 GB :)

kwag 10-23-2003 09:40 AM

Hi Krassi,

Apparently, a source of 3 to 4 minutes does the trick :)
Actually, the longer the better. But I think ~4 minutes should be fairly accurate. It's just a matter of selecting the right footage ( mixed action segment ).
I'm trying a source range of 3 minutes right now. Then I'll do an encode to see if it matches the wanted final size. All at 352x240 ( for speed reasons )

-kwag

Krassi 10-23-2003 10:32 AM

Hi Kwag,
my prediction results with a sr of 4 minutes:
VB: 48.558, CQ=66: 53.926. So thats not enough for a high Bitrate variance of 500-5000.
Testing with a sr of 5 min. now.
Maybe i should find a better sr, but maybe if we'd use a longer sr CQMatic would be able to set sr by itself (or you could enter the minutes manually :idea:).
For sure a 4 minutes length should be good enough for users targeting for a KVCD :D

EDIT: Results of the 5 min. didn't show any improvement, think that its related to the high Max bitrate value. I'll make a test with 4000 as max.

EDIT 2 :D: difference was less with 4000 as max but still 3,5 MB over VB with a CQ of 66.

Krassi 11-04-2003 04:49 AM

I've just had the following idea :idea: :

I'm testing this prediction method with CCE. My experience with CCE VBR is, that it is behaving better than the one TMPGEnc uses, it's not so "sticked" to the average bitrate.
So, in theory :roll: , this prediction method should be producing better results with CCE. Remember, CQ in CCE is the other way round, lower CQ will give higher filesize...
I've used 2-pass VBR to get filesize and then changed to CQ and matched filesize.

Current state:
I've done a prediction with a sample length (source range) of 60 seconds, 120 seconds and 5 minutes. Average bitrate is 1586, target video size is 1317910KB, min bitrate is 300 and max is 5000.

Results: CQ is different (also because CQ in CCE has a bigger CQ range):
1 min: optimal CQ is 60
2 min: optimal CQ is 55
5 min: optimal CQ is 41.
I'm doing a full encode with 41 now and let you know if it is hitting target.

Maybe i'll do a Mainconcept also, or can you setup one :?:

Krassi 11-04-2003 06:05 AM

Results of the CQ-encode @41:
VS: 1267375 KB
Wanted VS: 1317910 KB.

That's 4% under target.
I'll redo a test with another source range (maybe a longer one).

kwag 11-04-2003 08:36 PM

Hey Krassi, that's great :)
At least the theory of the quick "2-pass/long CQ" does make sense :D
I still think there must be a way to use it with TMPEG. Hopefully I'll have some time in my hands next week, and play with this more.
This week has been a little busy for me, and the little time I've had, has been playing with "MythTV". It's finally working :!: :!: :!:
I hope to move the test computer to the living room ( maybe tomorrow), and then I'll take some pictures with my digital camera and post them here :cool:

-kwag

Krassi 11-05-2003 05:55 AM

Hi Kwag,

i've redone the same prediction with CCE with a source range of 10 minutes 8O
Final predicted CQ was 44.
Results:
Wanted VS: 1317910 KB
Final VS: 1220134 KB
This makes 7,4% :evil:
I've tested this prediction with k-pax. It's a very dark movie and i think it's hard to find a good source range. I'll repeat this test with another source.
I haven't found a possibilty to make time slices with CCE (except with avisynth for sure).

incredible 11-05-2003 06:20 AM

@ Krassi

What type of samples u are using??
A .AVS including Sampler(length=xx, Samples=x) ??
maybe you mentioned already. :? :?:

Krassi 11-05-2003 06:43 AM

Hi incredible,

in CCE you have the possibilty to set the source range from a start frame to an end frame (Input Files->Settings->right-click->edit).
Thats how i've selected my source range.
Maybe i'll play around with some avisynth scripts to create time slices later on.

incredible 11-05-2003 07:40 AM

Quote:

Originally Posted by Krassi
In CCE you have the possibilty to set the source range from a start frame to an end frame (Input Files->Settings->right-click->edit).
Thats how i've selected my source range.
Maybe i'll play around with some avisynth scripts to create time slices later on.

I think you don't need that AVS Time slice way ...
Im at work so maybe lets try something "online" ...
Open the mpeg source in Bitrateviewer (this now depends if it accepts d2v's???? or Avs's including d2v imports?? otherwise forget my thinking) and whatch the peaks and lows, write down the time stamps of theese parts including some average bitrate parts. Determine these parts as timeslices im CCE and do your workaround again .... would be interesting. :roll: :)

Or another method:

Do the AVS time slice way ... but:
Write a sampler(xxx,xxx) based time slice a-script and another one b-script which includes a trim line which gives you an offset of about 2 or 5 mins (depending upon the used numbers of samples)... (you remember the thread in the german forum?)

Maybe this double a/b script based 2-pass/prediction will give you two values where you can see an average of both which should give you just another type of value ... it could also be more precise ???? :roll: 8)

Krassi 11-05-2003 08:06 AM

Quote:

Originally Posted by incredible
Open the mpeg source in Bitrateviewer (this now depends if it accepts d2v's???? or Avs's including d2v imports?? otherwise forget my thinking) and whatch the peaks and lows, write down the time stamps of theese parts including some average bitrate parts. Determine these parts as timeslices im CCE and do your workaround again .... would be interesting. :roll: :)

I've already done this before with TMPGenc, the prediction was better when i've selected peaks and mins manually and "imported" them to TMPGEnc. But thats a lot of work...
We could do this automatically, but before is has to be approved.
In CCE, you can only define from-to frames, no slices...

Quote:

Or another method:

Do the AVS time slice way ... but:
Write a sampler(xxx,xxx) based time slice a-script and another one b-script which includes a trim line which gives you an offset of about 2 or 5 mins (depending upon the used numbers of samples)... (you remember the thread in the german forum?)
Sure i do. It's already on my todo-list. Have you already setup a test?
Quote:

Maybe this double a/b script based prediction will give you two values where you can see an average of both which should give you just another type of value ... it could also be more precisely ???? :roll: 8)
Yes, in theory it could be. My previous tests were always selected with a different source range, so in theory, the average value of the 3 predictions would give the best value.

I also had the idea of making a longer VBR encode (e.g. 10 or 15 min. with slices) and then doing only a prediction with CQ divided by 2 or 4 or...
I mean you would have the size of the VBR (10 min.), e.g. 20 MB. Then you would divide it by 5. So the CQ prediction should match 4 MB when encoding 2 minutes. That's next on my list :wink:
The only problem here i can see is that you can't use the same source range, but in theory, compared to the whole video and the length of the VBR sample that should'nt play a role.
I hope you understand what i mean.

Krassi 11-05-2003 10:04 AM

Quote:

Originally Posted by Krassi
I also had the idea of making a longer VBR encode (e.g. 10 or 15 min. with slices) and then doing only a prediction with CQ divided by 2 or 4 or...
I mean you would have the size of the VBR (10 min.), e.g. 20 MB. Then you would divide it by 5. So the CQ prediction should match 4 MB when encoding 2 minutes. That's next on my list :wink:
The only problem here i can see is that you can't use the same source range, but in theory, compared to the whole video and the length of the VBR sample that should'nt play a role.
I hope you understand what i mean.

Update:
VBR encode: 30 mins. Final sample size: 348.163 KB
So, divided by 5:
6 minutes sample for CQ and a sample size of 69.632,6 KB (348.163/5).
Prediction gave a CQ of 31 (filesize is 70.026 KB).
Doing a full encode now with 31.

incredible 11-05-2003 10:35 AM

Quote:

Originally Posted by Krassi
Have you already setup a test?

Tonight will be the night (I mean this evening *lol*) :)
Do you will stay tuned too?? If yes gimme your ICQ via PN, so we can figure out togehter

BTW: Did you recognised that Kwag changed us to moderators? I was really surprised :o


All times are GMT -5. The time now is 01:01 PM  —  vBulletin © Jelsoft Enterprises Ltd

Site design, images and content © 2002-2024 The Digital FAQ, www.digitalFAQ.com
Forum Software by vBulletin · Copyright © 2024 Jelsoft Enterprises Ltd.