CQ vs. CQ_VBR ... VERY INTERESTING...
 

Hi kwag and others...

Last night I ran a couple of tests with bitrate viewer...
I used the file size prediction clip from Frequency, which I want to put on one 80 min. CD-R. I used the new GOP structure and Blockbuster 0.6 together w/ FluxSmooth. I resized it with Avisynth Bilinear resize to the resolution 704x576 (PAL). I also masked the borders in TMPGEnc.

Sample #1 :
The first sample I did with the old KVCDx2 704x576 Template with the Default Matrix using the new GOP of 1,12,2,1,24 and I set the CQ Level to 57...

File Came Out At : 11,391 KB

Sample#2 :
The second sample I did with the KVCDx2 PLUS Template for PAL using KVCD Matrix together with the new GOP, and CQ_VBR. I had to lower the CQ_VBR level to 5,98 to get the same size...

File Came Out At : 11,374 KB

Now here comes the thing I don't understand... Viewed w/ WMP I already saw that the KVCDx2PLUS Template was of less quality than the first sample. Viewed with bitrate viewer the Average Bitrate was 891 for both samples. But... Sample #1 had an average Q-Level of 5.28 , Sample#2 had one of 6.55 ...

Now that's a hell of a lot difference... 8O
Did I do something wrong??

Hi jellygoose,

No you didn't do anything wrong. Maybe it's time to look again at CQ :idea:
Last time I did this was at TMPEG version 2.50 or so, so maybe there is a difference now with CQ. Originally CQ_VBR was better quality than CQ for the same file size. Maybe it's time to review this. Also, before we didn't have prediction, so it was hard to target a file size. I'll do some tests to verify your findings. Mind you, I recall some people had problems playing KVCD encoded as CQ, but not as CQ_VBR 8O . So the data on the stream has a different pattern from CQ to CQ_VBR. This you can see clearly in Bitrate Viewer.

-kwag

Running my first test now...

Indeed I see that in Bitrate Viewer... Well as for me CQ plays fine. I'll run some test later tonight, to see how the KVCD Matrix works with the "Ol'" CQ...

Happy Testing...

Resident Evil, 1h35m, 352x480

CQ_VBR 15.63: avg. Q = 2.12, peak Q = 5.92, size = 11,890,602

P spoilage 0, B spoilage 10

Closest Q level:

CQ 73, avg. Q = 2.36, peak Q = 3.80, size = 11,429,682

Closest size:

CQ 74, avg. Q = 1.84, peak Q = 3.63, size = 11,669,852

P spoilage 5, B spoilage 20

Closest Q level:

CQ 77, avg. Q = 2.34, peak Q = 3.75, size = 11,650,255

Closest size:

CQ 78, avg. Q = 1.84, peak Q = 3.59, size = 12,038,198

P spoilage 0, B spoilage 0

Closest Q level:

CQ 70, avg. Q = 1.85, peak Q = 3.50, size = 10,423,978

Closest size:

CQ 71, avg. Q = 1.86, peak Q = 3.63, size = 11,547,443

Observations

- CQ mode seems to be non-linear between integers, i.e. 73.9 is closer to 73 than to 74.

- P- and B-frame spoilage of 0 provides significant reduction in both Q level and file size, at least at this resolution.

I'm now going to run some "subjective quality" tests.

You see, this is the kind of thing that was very hard to accomplish in the past without prediction :D
Now maybe we can even tune B and P spoilage with CQ IF CQ does provide better quality on the same space as CQ_VBR :idea:

-kwag

It doesn't. When viewed side-by-side the reason for the increased compression is obvious: it looks like I didn't use Blockbuster at all, with DCT blocks in all the usual places.

Seems that CQ mode is more aggressive than CQ_VBR. I'm now testing with more noise.

When you increase the noise to try to compensate for the blockiness, things start to look very bad very quickly.

Based on the tests I've done so far I would say that CQ_VBR provides significantly better subjective quality than CQ for a given bitrate, despite the lower Q level as reported by BitRate Viewer. So it seems the Q level, while generally a good indicator of relative quality, should not be relied upon without a side-by-side inspection of the two encodes.

Here are my test results:

Test #1 CQ_VBR: file size= 11,717KB
http://www.digitalfaq.com/archives/i.../2002/12/3.jpg

Test #1 CQ: file size= 11,517KB
http://www.digitalfaq.com/archives/i.../2002/12/4.jpg

I can see almost no artifacts on the CQ sample. They are clearly more visible on the CQ_VBR sample.

Here are the samples. See for yourselves:
http://www.kvcd.net/test-cq-vbr.m1v
http://www.kvcd.net/test-cq.m1v <-------- Winner or not :?: 8O

Script used:

LoadPlugin("C:\encoding\MPEG2DEC.dll")
LoadPlugin("C:\encoding\fluxsmooth.dll")
LoadPlugin("C:\encoding\sampler.dll")
LoadPlugin("C:\encoding\blockbuster.dll")
LoadPlugin("C:\encoding\legalclip.dll")
mpeg2source("K:\K19\VIDEO_TS\k19.d2v")
LegalClip()
BilinearResize(672,336,0,0,720,480)
FluxSmooth()
Blockbuster( method="noise", detail_min=1, detail_max=10, variance=.5, seed=1 )
AddBorders(16,72,16,72)
LegalClip()
#sampler()


So, what's next :?: :?: :?:

-kwag

That's exactly what I see kwag...

SansGrip: You're right, DCT Blocks don't disappear completely w/ blockbuster and CQ but... With CQ_VBR there are a lot more visible artifacts, especially around movements in the background... as far as my bad eyes can see...

Take a look at kwags samples, they show the exact same results that I have...

:o I just took another look at kwags script again. How come you're using a variance of 5 for the blockbuster filter? is that the default setting?

Another question: How come bitrate viewer tells me my VBV Buffer was 20, when I encoded with a value of 40 ?

Here's another view. Please pay special attention to the water and fire scenes:

These are shorter samples:
http://www.kvcd.net/k19-cq-vbr.m1v
http://www.kvcd.net/k19-cq.m1v

-kwag

Maybe with CQ the noise level needed to eliminate DCT blocks is much lower that the needed level for CQ_VBR :idea:

-kwag

I think so too... I used variance=1 with CQ mode and it gave me excellent results...

Hi Kwag,

Kwag wrote:
I downloaded both tests for CQ vs CQ_VBR and CQ is the winner :)
It's been some time since I encoded a 704x480, but this could be
another discovery. The fire scene was much better in the CQ clip
and other scenes were the same as CQ_VBR to me. This is worth
looking into. :) Someone correct me if I'm wrong but the file prediction
formula for CQ is more of a linear formula than CQ_VBR :?:

-black prince

Hopefully something good will come out of this 8) Specially the fire and the water scene. The CQ_VBR sample shows more visible blocks.Not sure about that yet. More tests have to be made :idea:

The samples I posted above, that's the way the complete movie would fit on one CD at 704x480 8O, so at 528x480 it should be even better. And I like that :mrgreen:

-kwag

kwag's clips definitely show the opposite result to my test. kwag, I think if you try it at 352x480 you'll notice much worse DCT blocks in the CQ version. If anyone wants I can do another couple of strips and post them for people to see.

Looking at the first clips you posted from K19, the CQ version is much better. What I do to compare clips is use VirtualDub to "fast recompress" them using Huffy (this takes a lot of disk space, but is lossless). Then I use a script like this:

to see them one on top of the other. If you do that, load it into VirtualDub and look in particular at frame 186:

http://www.digitalfaq.com/archives/error.gif

Look at the chrome on the top/side of the car, and at the window in the bottom-left corner of the frame, and the wall in the top-right corner. The CQ version is obviously a lot better at this resolution...

Note: I just realized that the samples I encoded above were all done with "High quality" and not "Fast" on motion estimation in TMPEG.
And that takes a LOOOOOOONG time to encode. But if it makes a hell of a difference in quality, I WILL wait for longer encodes if I can get the best quality. I have to try again with "Fast" motion estimation and see if there's any visual difference between CQ and CQ_VBR. On "High quality" estimation, obviously there is a difference.

-kwag

As long as both CQ_VBR and CQ versions were done the same, I don't care -- I'll wait twice as long for a better encode if that's what it takes 8).

Excelent script SansGrip :D
I have to dig more into AviSynth scripting 8)

-kwag

They were both done identical. :wink:

-kwag

Ok, now I'm starting to get excited by this :D. Time for me to try Resident Evil at 704x480... Let's see if it really is the resolution that makes the difference.

We're all hyperventilating at this momemt :mrgreen:

Wouldn't this be a great Christmas present? ;)

Right now I'm figuring out a good CQ_VBR for R.E. at 704x480, 1 disc. Then I'll try to get as close to that size as I can with CQ...

Anyone else noticed that sometimes several CQ values (in my case 52-54) produce almost identical file sizes?

Yes! It's a weird unlinear scale. Probably with "null" points throughout the complete curve :?. Maybe caused by the KVCD Q Matrix :?:

-kwag

So how will using CQ instead of CQ_VBR affect the prediction? Will KVCDP be able to be used the same way or do we have to do manual prediction?

:lol: I LOVE this forum!!! :lol:

Might make prediction harder. But I like a challenge 8).

I just found correct sample sizes for both CQ_VBR and CQ, but am having mixed results. While one particular frame is much improved with CQ (with CQ_VBR it is almost entirely made up of macroblocks), generally the CQ version is slightly more blocky. But this is with motion estimation -- I'm currently reencoding at high quality to see if it makes a difference.

I have a feeling that CQ mode has a better bit rate allocation than CQ_VBR. That's why artefacts are less noticeable than with CQ_VBR. That's just my thought :o

-kwag

@Kwag and SansGrip,

Used 704x480, GOP=1-2-12-1-24, commented Blockbuster noise,
2 CD's with 112kb audio and CQ=85. The movie, Vanilla Sky,
2hr 19 min and the verdict is WOW great 8O Looking at BitRate
Viewer, the average Q = 2.87. I wish there was a corrulation between
picture quality and Q. :? Fast action scenes has the lowest Gibbs noise
I've seen, even better than KVCDx3 (528x480). This template seemed more
compressable the the PLUS templates. I going to try this same
resolution for 1 CD. :D I too would like to know what file prediction
formula should be using for this :?:

-black prince

As a side note:

CQ_VBR 6.5 (motion estimation) = 11,910,158
CQ_VBR 6.5 (high quality) = 10,610,730

Edit: Also:

CQ 56 (motion estimation) = 11,796,690
CQ 56 (high quality) = 11,303,978

Haven't done a visual compare of them yet...

http://www.digitalfaq.com/archives/error.gif

http://www.digitalfaq.com/archives/error.gif

I think I'll let the pictures speak for themselves :).

So how will using CQ instead of CQ_VBR affect the prediction? Will KVCDP be able to be used the same way or do we have to do manual prediction?

Theoretically you should still be able to use KVCDP. When it says "CQ_VBR", pretend it says "CQ" ;). You might want to set the start value to 50 instead of 25, though, or it could take a while to get accurate.

I currently started to encode "K-19" at 352x240 LBR just to test the formula on CQ. It should be the same. Let's see how far off is the final target to the predicted size. I'll post result here when it's done. In about 2 hours.

-kwag

Oh boy, they don't speak, they SCREAM :!: CQ CQ CQ ( Just like a ham radio operator ( me included :wink: ) :mrgreen:

Ah, kwag, there you are -- I thought you'd gone missing ;). What... spending time with the family on Christmas Eve instead of TESTING? :mrgreen:

heheh yes it's pretty persuasive. I'm currently testing 528x480, and then will try with different (more sedate) source material.

Of course :D , I get out of my office 2 minutes, go kiss my wife and check things around the house, then run back to my office for another 2 hours ( She's having a good time with the kids anyway :wink: ) :mrgreen: I want to see those screenshots too :wink:

-kwag

Mine're watching Lilo & Stitch right now. They were allowed to open one present tonight and that's what they chose -- it's a Disney movie so it's giving the new receiver a good workout :D.

I decided I would test 352x480 again as well, just to make sure I did it correctly last time...