I would like to be able to do something like:
(quantizer to the minimum -> q=2.2 unless bitrate > maxrate)
or
(quantizer to the minimum -> q=2.3 unless bitrate > maxrate)

So I can vary the final size, but keep quality constant, this way I could fit in some media, for instance.

Why not decreasing average/maximum bitrate? :?

Seems to me that it would achieve your purpose.
Don't forget that although a quantizer is a measure of compression, it doesn't garantee that it will be able to compress the same in all frames.

The size difference in percentage between a simple and a complex image when compressed with quantizer=2 or quantizer=3 is not necessarily the same IMHO.

A bitrate difference is a much better garantee.

EDIT: This is probably a very old formula for a prediction-man like you... ;)

(CD size (MB) * 1024 (KB) * 8 (Kb) ) / movie time (sec) = avg bitrate


Bilu

If I changed the max bitrate, I would make low detail/motion scenes look good, while high motion/detail scenes would look bad. This is not what I want. I want to lower the quality a bit in all scenes in order to fit some media. Yes, I know rising the quantisizer won't change the bitrate needed in each scene in the same way, but in theory, they'll have the same "degree of distortion" (if there's such a word).
About the final size, I can use my CCE prediction method, with a accuracy of 3%, and go much faster than 2pass. :D

One thing I know now: a rate-control equation that ignores texture complexity and avg bitrate = max bitrate seems like a good quality combo.

How about lowering the vrc_eq value, but still using a number instead of a function? It's fractional (between 0 and 1) and doesn't care about texture complexity, making the quality decrease constant IMHO.

Can you test it?

Bilu

Guys,
You're not posting anymore :) : you're chating in public :D
I'm sorry but we, mere mortals, can't keep up with your level of conversation.
Anyway, something you could spare us to play with?
Maybe you favorite command line until now and some tips of what it's supposed to do?
Thanks guys.
Oh and, by all means, keep up the conversation :D
Cheers

@rds_correa
I am at work now, when I come home I'll post my latest command-line. :D

@bilu
Yes, of course. You're telling the encoder: you can't go over max_bitrate, otherwise, use at your own will! :D But note, this don't lead to optimal use of bitrate, this is near CBR for a very "bitrate-consuming" video material... :(
I tried that. If you use 0, filesize really get smaller (about the half). But any other value seems to make little difference in final filesize. Tried 0.1, 0.001, 0.05, about the same thing.
I am starting to believe there's something wrong with our approach. I just checked a 1-pass VBR CCE encode with bitrate viewer, and the quantization curve seems to vary a lot and stay beyond 5 all the time. Maybe this pursue of the straight Q line is not the best quality option. :?:

Can you try 0, 0.25, 0.50, 0.75 and 1, and tells us about the bitrate distribuition differences ? Please... ;)

Bilu

I'll try. But I think I did try 0.5 and made no difference... I did a test with 2 pass VBR, and I saw the same pattern I've seen before with CCE: the bitrate curve seems to follow the quantisizer curve... Interesting.

That's what 2-pass VBR is all about, I think :)

Simple scenes get higher quantizer to lower their bitrates.
The bits spared will be spent on lower quantizers / higher bitrates on more complex scenes.

Of course this is not 100% true: you can have high quantizers and high bitrates together on a complex scene. But on most movies the complexity on spatial detail doesn't vary that much. Most variations are temporal: stalled, slow motion, high motion.

And it's pretty normal that increasing temporal complexity increases the bitrate: after all, there's much more motion vectors and differences between frames to encode. And temporal complexity is harder to compress, hence the lower quantizers :)


Conclusion: temporal complexity (not spatial) makes the bitrate curve follow the Q curve. And temporal complexity is what video (not image) encoding is all about :)

EDIT: Did you made that test with vqcomp=0 ?

Bilu

How about the quality? And the quantizers?


Check this out:

vqblur=<0.0-1.0>
quantizer blur (pass1) Larger values will average the quantizer more over time (slower change).
0.0 qblur disabled
0.5 (default)
1.0 average the quantizer over all previous frames

Now here is a nice reason for vqcomp=1 not working on that fast action scene of VM's test ! :twisted:
A true VBR killer! :twisted:

Please test again with vqcomp=1 (or vrc_eq=tex) and vqblur=0.
And vqcomp=0 (or vrc_eq=1) with vqblur=0 would also be an interesting test... :)

(I think vqcomp=0 will provide better results - because of perceived frame complexity, dark scenes,etc. . Please use vbitrate=vrc_maxrate in both tests)

Bilu

Bilu,

In that case, quality got really bad.

Sorry but I really feel that there's no reason to keep testing this if we can't vary the minimmum quantisizer in a better way. Maybe only a source Mod can fix this... :(

HiBitrate VBR: that's how far as 1-pass encoding goes.

vrc_maxrate=vbitrate
vrc_eq=tex
vqblur=0

This is similar to a XVID method used by some guys for capturing: let bitrate flow freely and avoid any quantizer delays (vqblur-like). And unlike vqscale=2, you know this method respects the max bitrate boundaries.

Quality decreasing would have to be made by:
- processing the input through resizing/blurring/denoising;
- raise min quantizers;

Quality increasing can still be made by:
-adding noise;
- trell,cbp,mbd=2,mv0;


But for most cases we'll want to go 2-pass. Bits are distribuited and when you need to fit in a smaller media the quantizer raising gets distribuited also.

So let's focus now on 2-pass behaviour. SATD is much slower/better than SAD motion estimation but a SAD 2-pass would still be better (and maybe even faster :) ) than a 1-pass SATD.

And of course, avoid vqscale in 2-pass encodings too.


Bilu

1) We should avoid breaking compatibility. That would mean forking Mencoder and having trouble updating our version everytime a bug in the official one gets fixed. We may not have that much man power around :roll:

2) Same result can be achieve through denoising/blurring/resizing.

3) Is quantizer 3 so much different from quantizer 2 in MPEG-2 that you would need fractional quantizers???

4) Ever wondered if fractional quantizers would break MPEG-2 compatilibity?


Bilu

The HiBitrate VBR method should be really useful for converting MPEG-2 streams from Sony MicroMV cameras (MPEG-2 CBR 12Mbps) to DVD format (9.8 Mbps) :!:

(don't know if those cameras still have the audio desync problem)

Probably this method also works as good in both FFMPEG and FFVFW.

Bilu

I wonder how would requantizing (when difference to desired filesize is less than 10%) work with a HiBitrate VBR stream.

Would it worth instead of 2-pass? :roll:


Bilu

Yes.. specially now that my mencoder compiles are giving "core dumps"... :(
Yes, but that would slow down and filter the picure...
Yes...
No, because the generated file has fractional quantisizers in many points (check with BitRate Viewer)
I was thinking about that... No fractional min quantisizer, no fun with 1-pass. :D

To use 2-pass, it's better to feed mencoder AVI/VOB straight, with no avs, because it goes much faster. So I am playing with the internal filters to find a good filter combination for DivX stuff. I also would have to find the optimal combination of settings for mencoder (I'll re-read all your previous posts... :D )
Also about 2-pass, russianexpat suggested in other thread that it's better to use vqscale=1 in the first pass for more accurate results. I'll try that. :D

Just for curiosity, could you try HiBit VBR 1-pass on that action scene you mentioned before? Just to see the effect of vqblur=0 :)

And I think HiBit VBR 1-pass + Noise would also be great in dark scenes.


About vqscale=1:

I think it would be better to do the first pass with vqscale=2.

Bilu

Ok, I'll try vqblur. Actually I meant vqscale=2, but got confused when typing... :D

More and more we seem to benefit from the MPEG-4 experience :)

This was the only difference I found so far, as exposed by Incredible:

http://www.kvcd.net/forum/viewtopic.php?p=61337#61337

MPEG-4 uses much more keyframes (I frames) than MPEG-2, so MPEG-2 has to preserve better the inter frames (P and B frames), hence using lower quantizers on them.

Well, in practice we find two difference: no longer have a need to deinterlace ;)

The HiBit VBR encoding and using qscale=2 on the first pass of a 2-pass encoding is exactly what is used with Xvid nowadays. And works here too :)

With libavcodec-based encoders there is still another advantage: just change the codec parameter, no need to mess with the rest in most cases :D


Bilu

Sorry you misunderstand.

mpeg4 does have much more Frames within the I frame interval!
Means much more non intra frames! And therefore LESS I frames within a whole stream. Do look at somem Divx or Xvids as they come with about 300 Frames within the Intra Frame steps. But as mpeg4 uses a diff. algor. way, you can cut the non intra frames more agressive.

Understood now: MPEG-4 uses less I-frames and still can compress P and B frames more aggressively. Evolution indeed :)


Bilu

Do you know if Bitrate Viewer is just reporting an average mabroblock quantizer instead? Maybe fractional quantizers don't really exist, and the frame quantizer is an average of macroblock quantizers. :?

Bilu

Maybe you're right... I see no other reason for mencoder to don't accept fractional min quantisizer and round vqscale values.. :? :cry:

Maybe this is non exactly related, but in ffvfw (also libavcodec) it is impossible to even type fractial quantizer value. It don't accept dots, and when you force it (copy x.y value into box) it gets red. So maybe this is libavcodec behavior :roll:

Has anyone here ever seen a MPEG-1/2/4 codec that supported fractional quantizers?

I never did :roll:


Bilu

Hi bilu, hi all:
well, I followed this difficult-to-follow thread (difficult for me, no idea in mpeg encoding, even less mencoder parameters), and tested every suggestion you did. To my eyes, the best ones where the last ones:
vqsquish=0:vqcomp=0:vrc_eq=1
and HiBitrate VBR --> vqsquish=0:vqblur=0:vqcomp=1:vrc_eq=tex
...but
I tried to improve image and raise bitrate in HiBitrate VBR test (already using naq: trell: cbp: scplx_mask=0.5: mv0) and changed vqmax=4:mbqmax=4 from default values I was using. The file size grew a little.
...but
when I tried to employ higher bitrates (thinking in KDVD) and changed vrc_maxrate=8000:vbitrate=8000, I got NO change in file size, neither in quality. I don't have BitrateViewer installed (already enough things installed in my poor PC), but it seems bitrate doesn't raise, and for what I understood in this thread, it had to raise after raising vbitrate and vrc_maxrate.
Am I doing anything wrong?, how can I make bitrate raise?. I'm not even sure if with my previous settings (vrc_maxrate=5000:vbitrate=5000) I'm really getting to these bitrates in bit-demanding scenes. Help please.

Hi digitall.doc,

------------------------------------
Settings for 1-pass HiBitrate VBR are:

vrc_maxrate=vbitrate
vrc_eq=tex
vqblur=0

vqsquish should be left as 1 (default).
------------------------------------
vqcomp setting is not needed when using vrc_eq=tex or vrc_eq=1, it's just a part of the original vrc_eq formula (tex^qComp).
------------------------------------
trell
cbp
mbd=2
mv0

all of this settings improve quality.
------------------------------------
-scplx_mask=0.5
-naq

scplx_mask works like a deblocker and increases compressibility, lowering bitrate and quality. Not what you're looking for ;)

naq is used with *_mask settings only.
------------------------------------
Don't mess with vqmax and mbqmax, only with vqmin and mbqmin.

By messing with vqmax and mbqmax you could be limiting the encoders capability to decrease quality if needed to avoid max bitrate overflows.

Leave vqmin=2 and mbqmin=2 for maximum quality.
------------------------------------
vrc_maxrate=8000:vbitrate=8000 seems fine to me :)
------------------------------------

When aiming for high bitrates, the HiBitrate VBR method is perfectly fine: is goes for the less compression possible unless it goes over the maximum bitrate specified.

However, if you ever want to decrease size to fit in a DVD/CD/whatever, you're better with a two pass process.

The main difference in the settings (plus the 2-pass specific ones of course) is that vqblur will not be used and vbitrate should be calculated like this (vrc_maxrate will still be the same):

(CD size (MB) * 1024 (KB) * 8 (Kb) ) / movie time (sec) = avg bitrate

CD size is not the whole media size, just what's left after the audio/structures/whatever.

EDIT: This is for the 2nd pass, 1st pass must be done without any vbitrate setting and using vqscale=2.

As you can read through my posts I've never used mencoder myself... yet ;)

Bilu

hi bilu,

I'm now on the mencoder encoding side of things here these days :lol:

I've also used ffvfw too, and I do like it, though I'm still a little lost/puzzled
about other's findings (spiky bitrates etc) but since I'm now working w/
mendocoder, (creating a GUI) I am having issues with this bitrates. I posted
this question (similar) in another thread here. Anyways..

I'm trying to find out which mencoder command-line param(s) are the ones
responsible for raising/lowering the final encodes filesize.

I have adjusted the following params (latest is what you see now) and they
are staying the same, in filesize. I can't actually see any difference though,
when I view the video:

One more question, I can't seem to find the param that is responsible of
changing the VBV value. ie, bviewer is showing 208 as my value. I want
to get it to show 224 (as in my DVD encodes are, in TMPG) Actually, I want
to be able to change it to whatever value I want, but I can't figure this one
out :roll:

Oh, and listen, thanks for taking the time to demonstrate and discuss your
findings/research (through hard work/trial 'n error) and sharing it here on
kvcd :!: :!:

-vhelp

Increase/decrease vbitrate. To raise filesize you can also decrease vqmax and mbqmax quantizers. There's also the option to add/remove noise.
Other guys will have to answer this question... :roll:
I never did an MPEG-2 encode. Ever. I want to do it on a old Pentium 200 w/ 64 MB RAM running Linux or FreeBSD, but all I've did before was MPEG-4 and have no MPEG-2 experience. I've been searching docs based on my MPEG-4 knowledge that is similar to MPEG-2, like bitrate control, quantizers, or rate distortion. YOU guys have been testing, I've only got the docs and some theory for you :)

I'm still trying to find what VBV is. This is what I found so far about VBV for MP@ML (Main Profile @ Main Level) used for DVDs:

http://www.mpeg.org/MPEG/DVD/Book_B/Video.html
http://www.ee.columbia.edu/~eleft/e6...s/is138182.pdf

Still checking for more. If someone explained or pointed a good resource it would be nice :)

Thank YOU guys for sharing your testing effort with me. I never used Bitrate Viewer and have no previous MPEG-2 experience that could provide me with comparison references :)


Bilu

http://zebra.fh-weingarten.de/~maxi/.../msg01147.html

vrc_buf_size=<value> buffer size in kbit

VCD: 40 KB *(1024*8/1000) = 40 * 8,192 = 327,68
SVCD: 112 KB * 8,192 = 917,504
DVD: 224 KB * 8,192 = 1835,008

So here lies the answer about VBV. Now I'm trying to find out where did Tsunami and Mplex got those values... :roll:

EDIT1: 224 KB for DVD makes sense to me now: 1835008 bits is the maximum VBV buffer in the MP@ML specification :!:

Conclusion: for VBV parameters use the vrc_buf_size setting with this formula:

Bilu

Hi Vhelp,
I have already posted it several times in answer to your question :wink:
"vrc_buf_size=vbv"
According to equation mentioned by Bilu: vbv=vbv_you_want*1024*8/1000
So for VCD case vbv should be 40*1024*8/1000. Use vrc_buf_size=328
For SVCD case vbv should be 112*1024*8/1000. Use vrc_buf_size=918
For (K)DVD case vbv should be 224*1024*8/1000. Use vrc_buf_size=1835
Here is my last trial using Incs or Amenophis command:
@echo off
mencoder -of mpeg -ovc lavc -lavcopts vcodec=mpeg2video:intra_matrix=8,9,12,22,26,27,29, 34,9,10,14,
26,27,29,34,37,12,14,18,27,29,34,37,38,22,26,27,31 ,36,37,38,40,26,27,29,36,39,38,40,48,27,29,
34,37,38,40,48,58,29,34,37,38,40,48,58,69,34,37,38 ,40,48,58,69,79:inter_matrix=16,18,20,22,
24,26,28,30,18,20,22,24,26,28,30,32,20,22,24,26,28 ,30,32,34,22,24,26,30,32,32,34,36,24,26,28,
32,34,34,36,38,26,28,30,32,34,36,38,40,28,30,32,34 ,36,38,42,42,30,32,34,36,38,40,42,44:mbd=0:
vbitrate=3000:vrc_maxrate=8000:vrc_minrate=300:vrc_buf_size=1835:keyint=15:vqmin=2:vqmax=24:
vmax_b_frames=2:vb_qfactor=1.2:vi_qfactor=0.8:vb_q offset=0.75:vqblur=0.3:vlelim=-4:vcelim=2:
vfdct=6:aspect=4/3 -nosound c:\video\mplayer\bttf_i.avi -o c:\video\mplayer\encoded.mpg
pause
bbdmux encoded.mpg 0xe0 decoded.m2v
pause
In fact bitrate viewer will always claim you're using a vbv buffer half the size you chose. But it's all working perfectly because it does it with all encoders out there.
As for rising or lowering the bitrate I'm also a bit puzzled about it so better wait for a more clear explanation.
Don't get me wrong Bilu. but I didn't understand it with your effort to explain it.
Hope I helped.
Cheers

About that command-line: try mbd=2 and vqblur=0. You won't regret.

About rising/lowering bitrate:

If you add noise, you increase bitrate.
If you remove noise, you decrease bitrate.

If you increase vbitrate (speaking of a 2-pass process, not that HiBitrate VBR 1-pass stuff) you'll increase the average bitrate, so it will use lower quantizers, increasing the filesize.

If you decrease vqmax and mbqmax quantizers you limit the capability of the encoder to compress the stream. This will also increase filesize, but may limit your filesize prediction when doing 2-pass. Good only for normal 1-pass (not high-bitrate oriented)

If this is not enough explanation please post a specific doubt.

Bilu

VCD: 40 KB * 8,192 = 327,68
SVCD: 112 KB * 8,192 = 917,504
DVD: 224 KB * 8,192 = 1835,008

Since this is a maximum buffer size it's better to round down IMHO.

VCD = 327
SVCD = 917
DVD=1835


Bilu

From http://www.doom9.org/mpg/dvd2svcd_advref.html#cce :
From http://www.mpeg.org/MPEG/DVD/Book_B/Video.html :
IMHO there is no greater gain in exploring VBV formulas and bitrates over 9,8 Mbps.

Bilu

Hi bilu:
thanx for all your answers. As you can see, we keep coming back to the thread to share our results. And your help is much appreciated (I still don't know how you manage to avoid temptation to make a encode at a friends' computer, or even at a cybercoffee :lol: )
Well, in my post I asked how could I raise bitrate, because using vbitrate=5000 and vrc_maxrate=5000 gave the same file size than vbitrate=8000 and vrc_maxrate=8000, and the same "visual" aspect (in a scene a aircraft explodes... in lots of macroblocks :wink: ). To get rid of those macroblocks I thought I could raise bitrate from 5000 to 8000, but same filesize and same visual aspect. I didn't test it in bitrateviewer to see if bitrate does increase, but it looks as it didn't change between 5000 and 8000. Any hints?
Another question: what are vb_qfactor, vi_qfactor, vb_qoffset, suppossed to be useful for?. I don't understand manpage explanation, I understand you better. I see that rds, inc and amenophis are using them... so they must be useful. But, what for?, and how tweak them?.

Posting again,... sorry:
I forgot to say:
- I made two tests with the same parameters, but in one I resized via avisynth script (no GripFit, but Inc :wink: method with MovieStacker) and in the other I resized internally in mencoder with crop/scale/expand.
The visual result the same. But MUCH slower with -vf internal filters, to my surprise. It dropped from 8 fps with avisynth, to 4 fps with internal filters
- I'm thinking about testing 2 pass mode... could someone post a example command line (with vqscale in first pass, vqblur in second pass, vbitrate settings for first and second,... and a possible way to make first pass faster to shorter encoding time). Am I asking too much?.
Thanx

Did you use this?

vrc_eq=tex
vqblur=0

trell
cbp
mbd=2
mv0

Also try this, decreases bitrate but works as a deblocker:

-scplx_mask=0.5
-naq

About noise, you should know better than me from the FFVFW noise threads ;)

All of these values establish a relation between P-frame quantizers and the rest.

vi_qfactor is a quantizer factor between I and P frames. default 0.8 means that I = 0.8 * P -> lower quantizers on keyframes.

b_qfactor is a quantizer factor between B and I/P frames. B-frames are generated with info from both the previous and the next frame and generally can be more compressed than I/P frames.

v{b|i}_qoffset is an offset to formulas like:
Bquant=Pquant * Bfactor +Boffset

I must agree with you that this explanation ***SUCKS***. 8O
I'm glad I learned a lot of those concepts through XVid, they're much better at explanations. :?

According to this explanation

if {b|i}factor > 0 then {b|i}quant = Pquant * {b|i}factor + {b|i}offset;
if {b|i}factor < 0 then {b|i}quant = -Pquant * {b|i}factor + {b|i}offset;
if {b|i}factor = 0 then {b|i}quant = {b|i}offset; (not recommended)

These negative quantizer values seem to exist just to avoid locking on the next P-frame quantizer and use normal rate control. Probably not recommended, but who knows??? :roll:

This is XVid-like. B-frames will have an increased compression factor over I/P frames, usually around 1.25, and that's the default.

But the default is vmax_b_frames=0, which means no B-frame encoding.

Bilu

According to http://www.ee.columbia.edu/~eleft/e6...s/is138182.pdf :
So Bitrate Viewer does

VBV = B / (1024*16)

(B is minimum VBV buffer size needed, in bits)

while we do

VBV = (x*1024*8/1000)

(x is the VBV buffer size from TMPGEnc, in KB)

I think Bitrate Viewer checks how many bits it needs to decode the sequence ( the B value ) and calculates VBV according to the specs.

Unless the minimum VBV size is half the max VBV size, there is something strange here. Also as you can see from posts above I was able to prove that the DVD values from TMPGEnc make sense according to the spec :!: :?:


Bilu

Well, I already used all this settings, with lots of macroblocks during the explosion, that didn't improve raising bitrate from 5000 to 8000. I didn't try adding noise, but I still don't understand why the encoder doesn't make use of higher bitrate avaliable to improve the image and avoid this macroblocks...
I didn't try SATD instead of default (?) SAD to see if this improves quality (avoids these macroblocks), but I didn't want to make encoding even slower...
From a theoretical point of view... what values, thinking in quality terms, would you advise for b_qfactor, vi_qfactor, vb_qoffset, and vi_qoffset

And what about a command line for 2pass encoding
Thank you again for support :roll: