Bitrates: CQmatic 1.3 (Experimental) Results

[kwag]

I'm running three individual "X1 Mode" "prediction-only" phases, and one "X3 Mode", on the movie "Red Planet".
( http://www.kvcd.net/downloads/CQMatic-1.3.00.exe )

Here are the results, using "X1 Mode" Prediction:

Run #1 - CQ = 63.51
Run #2 - CQ = 64.23
Run #3 - CQ = 63.62

Average: 63.786

Here is the result running a single "X3 Mode" prediction:

CQ = 56.69
This one doesn't seem correct
So I'm encoding the complete movie now, with the average of 63.786, and we'll see if the final file size is close to the wanted of 729,726KB
Will post result in about 3h:30min.

-kwag

[Dialhot]

How 3 predictions can lead to 3 different CQ values ?
Whant did you change for each one ?

[kwag]

Quote:

Originally Posted by Dialhot

How 3 predictions can lead to 3 different CQ values ?
Whant did you change for each one ?

Because each time you reload your project file, the sequence of samples will always be different. That's the good thing about this version
CQMatic is now sampling "out-of-order" shots, and this way, you get a better "feel" of the material.
For example, in the above test, on the movie Red Planet, you can see that the bitrate distribution is well normalized throughout the movie, because even at three different runs, the CQ value is almost the same.
On other movies, it will probably be different
I'll take that into consideration for a future version, where it will do two initial runs. If the difference from each run differs by more that 2%, then CQMatic will automatically run three or four runs, and average the CQ values, to get a more accurate final CQ value.

-kwag

[muaddib]

That sounds very interesting!
Do you already have any results?

[Prodater64]

Could CQMatiq calculates the three test average?

[kwag]

Quote:

Originally Posted by muaddib

Do you already have any results?

Not yet
I'm 56% into the encode. About 1 hour 30 minutes to go for te result

-kwag

[muaddib]

Quote:

Originally Posted by Prodater64

Could CQMatiq calculates the three test average?

Quote:

Originally Posted by kwag

I'll take that into consideration for a future version, where it will do two initial runs. If the difference from each run differs by more that 2%, then CQMatic will automatically run three or four runs, and average the CQ values, to get a more accurate final CQ value.

[kwag]

Quote:

Originally Posted by Prodater64

Could CQMatiq calculates the three test average?

Yes, of course
That's why I posted that for a future version, CQMatic will decide if it makes three or four total runs, depending on the first difference of the first two runs.
Because of the way CQMatic now selects the prediction footage (very randomly), if I get a large deviation difference from the first run compared to the second run, it means that the footage varies a lot in activity and that activity is not well distributed.
So depending on how much that deviation is, I can take into consideration additional prediction runs to average the final CQ value

-kwag

[kwag]

Well well, results are not as I expected.
Wanted file size: 729,726KB
Encoded file size: 772,093KB
Diff of +5.48%
Now I'll encode at CQ=56.69, the value X3 mode gave me.
Results tomorrow, because I'm going to bed now

Edit: Maybe these threads should be called: "Taming the CQ beast"

-kwag

[kwag]

Final size with X3 is: 643,571KB, which is way low.

Clue: After looking at this very closely, now I'll attempt to fine tune the number of samples and the sample size.
Why
Because if I got almost the same CQ value in X1 mode with three completely different prediction runs, then it seems logical that the random sampling is producing a constant result.
So I'll work on finding an optimal number of samples/sample size, and when I hit correct CQ for this movie, I'll test it on several diferent movies.
I'll also drop X3 mode, because it won't be needed any longer.
I'll post version 1.3.01 later today.

-kwag

[rds_correia]

Can't wait to know the results.
The only problems is...I don't use tmpgenc anymore
Maybe we'll have to think about working on a CQM 4 MEncoder
Good work Kwag
Cheers

PS-Get some rest. You deserve it.

[kwag]

Quote:

Originally Posted by rds_correia

Can't wait to know the results.
The only problems is...I don't use tmpgenc anymore
Maybe we'll have to think about working on a CQM 4 MEncoder

Not for MPEG-1. Nothing beats TMPEG on MPEG-1 at very low bitrates (yet!)
But yes, I'm already thinking about supporting MEncoder/ffmpeg in CQMatic

Quote:

Good work Kwag
Cheers

PS-Get some rest. You deserve it.

Thanks

-kwag

[kwag]

Well, I'm really fed up with TMPEG
The problem is not CQMatic. It's the damn (again!) unreliable/unpredictable/unlinear CQ curve, which just behaves completely different, even at same CQ values, when different resolutions/parameters are exercised.

Look very closely:

Movie encoded: Red Planet
File size at CQ=62.43: 689,989KB
File size at CQ=63.78: 771,853KB

And my WANTED file size is 729,726KB

How much time do you think that it would take to predict a correct CQ value, in the range from 62.43 to 63.78 for that wanted file size
Because it definitely has to be done with two decimal digits of precision.

At a CQ difference of only 1.35, there is a file size difference of

81,864KB using a controlled environment (same script, same parameters, etc.)
This is hilarious
So that's it. I wish Pegasys, Inc. fixes the CQ curve (if that ever happens, which I really doubt!)

In the mean time, I'm studying an alternate method to zero in on this CQ valley

-kwag

[rds_correia]

Damn
But isn't there a release that's a bit more smoother than the one's we've been using lately.
I thought I read somewhere that it changed a lot between releases but that there was a version
known to be more "predictable".
Or am I wrong
In the worst case scenario you could try to aid Nic with QuEnc and make CQM compatible with it...
Or maybe MEncoder, although I think MEncoder is a beast right now.
Cheers

[kwag]

Quote:

Originally Posted by rds_correia

But isn't there a release that's a bit more smoother than the one's we've been using lately.

Sadly, no.

Quote:

I thought I read somewhere that it changed a lot between releases but that there was a version
known to be more "predictable".

Every version introduces differences, making everything worse

Quote:

In the worst case scenario you could try to aid Nic with QuEnc and make CQM compatible with it...

Maybe I won't have to do this, if vmesquita integrates MEncoder into DIKO

Quote:

Or maybe MEncoder, although I think MEncoder is a beast right now.
Cheers

Yes it is, but it's also very promising
Anyway, I'm doing other tests right now with CQMatic, testing some new algorithms.
So stick around

-kwag

[kwag]

I'm officially announcing the end of support for TMPEG in CQMatic.
Whether I do something with CQMatic in the future to support something else, has yet to be seen.

However, CalcuMatic is always usefull, because it's a good calculator which is not attached to CQMatic.

Final results of my last encode with CQMatic:

Movie "Payback"
Wanted file size: 734,192KB
Encoded file size: 794,272KB

As of "right now", TMPEG support with CQMatic is permanenty discontinued.

Please focus on MEncoder, and keep an eye on DIKO, with possible future support built-in
TMPGEnc WAS a great product, but it has been superceded by MEncoder(ffmpeg).
And I hope Pegasys, Inc. reads this, as they have never responded to me, to any of the E-Mails I've sent reporting bugs and other issues (like the CQ non-linearity issue).

To TMPGEnc: R.I.P.

Lets all move forward to higher quality, multi platform and faster encodes with MEncoder

- The End -

-kwag

[rds_correia]

Gee pal,
Don't speak like that.
You almost give me the creeps
But I think I know what you meant to say.
We'll have to keep using tmpgenc while we don't find a better option.
That could have already been found: FFmpeg/MEncoder/FFdshow&FFvfw/QuEnc.
You name it!
But it's going to take a while for us to start seeing light at the end of the tunnel.
These and other encoders are still beasts.
They need to be packed with multiple arguments that still don't work so good.
We'll get there, I bet you, but don't expect that in a short time.
So let's all enjoy tmpgenc for a while more though we know it is getting old and it needs to step down it's throne.
C ya

[kwag]

Quote:

Originally Posted by rds_correia

Gee pal,
Don't speak like that.
You almost give me the creeps

But I do mean what I said

Quote:

But I think I know what you meant to say.
We'll have to keep using tmpgenc while we don't find a better option.

I don't think so
I'm waiting on some things from vmesquita for confirmation.
Apparently, TMPEG's quality has been matched (or superceded!) with ffmpeg, so we don't need TMPEG anymore.
All my efforts will now be geared towards MEncoder/ffmpeg.

Quote:

That could have already been found: FFmpeg/MEncoder/FFdshow&FFvfw/QuEnc.
You name it!
But it's going to take a while for us to start seeing light at the end of the tunnel.

I thought so too, but as far as the tests I've seen on Linux running MEncoder, if the same results are achievable on Windows, TMPEG is a dead product (at least for me)

Quote:

These and other encoders are still beasts.

Don't worry. We'll wrap them up in easy GUIs, to make life easier

Quote:

They need to be packed with multiple arguments that still don't work so good.

That should be easy to hide behind a constrained GUI, with selectable options.

Quote:

We'll get there, I bet you, but don't expect that in a short time.

You'll be surprised. We're already right around the corner

Quote:

So let's all enjoy tmpgenc for a while more though we know it is getting old and it needs to step down it's throne.
C ya

It's really Pegasys's fault, not to listen to the industry. Not to mention that TMPEG is only a Windows encoder. It's also written in Delphi Pascal, so there's little chance of porting it to other platforms ( except maybe a Linux version, if compiled with Kylix).
But MEncoder/ffmpeg, can run on Windows, Linux, FreeBSD, NeBSD, BeOS, MAC, etc.
Not so for TMPEG.

-kwag

[rds_correia]

Well, I gotta tell you this.
I'll go the way you go .
You're the boss, boss .
Cheers

[muaddib]

Quote:

Originally Posted by kwag

Apparently, TMPEG's quality has been matched (or superceded!) with ffmpeg, so we don't need TMPEG anymore.
All my efforts will now be geared towards MEncoder/ffmpeg.

How about MEncoder MPEG2 quality? Any test already made?

Quote:

TMPEG is a dead product (at least for me)

Oh man... hearing you saying that almost bring tears to my eyes.