Bitrates: Testing CQmatic Versions - Page 5

vhelp · #81 08-06-2003, 06:42 PM

Evening "brickwalls :banghead: hitters" (you guys)

I've ben after this fact for the longest of times (eons) and in many
areas of concirn..

..to develop a consistant "testing platform" for new developing ideas
like CQM !!

Too many peoles have too many variables / differences in their setups
and processes and so on and so forth. Somthing "generic" has to be
implemented, or else things will get more involved, issue'wise

I'll spare you all my regimin, and assume someone already has a working
flow plan ? Anyways, I agree that we need something like this for
these kinds of testings, AND to minimize the countless headaches that
could plague us all (mostly the developer) in achieving a goal.
Are you Matic, cause I'm Matic (jellen) anyways..

-vhelp

jorel · #82 08-06-2003, 07:23 PM

vhelp,one my opinion:

first we need one reference,
after everybody got perfect results with the same progs,scripts,etc,...
we can try some variables or this work will be so hard!
then after this perfect "reference",
the "generic" will be easy with some little changes!
see that if the same dvd movie is ntsc or pal, give differents results.

vhelp · #83 08-06-2003, 07:26 PM

@ Jorel..

Quote:

Originally Posted by jorel

vhelp,one my opinion:

first we need one reference,
after everybody got perfect results with the same progs,scripts,etc,...
we can try some variables or this work will be so hard!
then after this perfect "reference",
the "generic" will be easy with some little changes!
see that if the same dvd movie is ntsc or pal, give differents results.

Yea Jorel.. I agree

Preferebly, a movie that everyone has.. then we can MAP out the issues
as well as the success (pros and cons)

EDIT: - - this way, you build a "template" or "gauge" to use in future
testings..

-vhelp

kwag · #84 08-06-2003, 08:17 PM

Well, I finished my test with the same movie. This is going to be a pain

Look at the results:

I encoded "Red Planet" with this script and CQM 1.1.02:

Code:

## DLL Section ## 
# 
#LoadPlugin("C:\Filters25\MPEG2Dec3.dll") 
  
# 
#
# 
Mpeg2Source("K:\RED_PLANET\VIDEO_TS\red.d2v")
# 

GripCrop(352, 480)
GripSize(resizer="BicubicResize") 


GripBorders() 
LetterBox(16, 16, 16, 16)
Limiter()

Plain and simple RAW .avs script. without filters.

First test:
352x480 MIN=523.83 MAX=2,000
CQ found: 80.25
Encoded file size: 834,313KB

Second test:
352x480 MIN=300 MAX =2,500
CQ found: 74
Encoded file size: 643,633KB

Both CQ values are incorrect

The wanted file size by Moviestacker is 726,338KB

Now that is what I call a problem

Look at the difference in file sizes, just from CQ 74 to ~80

Here's my CQMatic log, with MIN=300, MAX=2,500:

Code:

http://www.kvcd.net
CQMatic Version 1.1.03
Copyright Softronex Corporation, 2003.
All rights reserved.
Time: 13:56:22 Date: 08/06/2003
Ready!
Project: K:\red.tpr

Creating: CQMatic.tpr

f:\temp\red.m1v
Execute.
Movie Time: 106
Average Bitrate: 919
Full Encode mode
Executing Prediction Phase...
Process started at 13:56:53
On 08/06/2003
Setting up initial sampling.
Using CQ of 79.80
Prediction cycle #1
Encoder started...
Process canceled
Project: K:\red.tpr

Creating: CQMatic.tpr

f:\temp\red.m1v
Execute.
Movie Time: 106
Average Bitrate: 919
Full Encode mode
Executing Prediction Phase...
Process started at 14:02:47
On 08/06/2003
Setting up initial sampling.
Using CQ of 79.80
Prediction cycle #1
Encoder started...
Process time: 4.75 minutes.
Encoder end.
File size difference  = 0.915482
Low fence: 2.000000
High fence: 79.800003
Last CQ = 79.80
Current CQ = 73.06
CQ difference = 6.744522
Using CQ of 73.06
Prediction cycle #2
Encoder started...
Process time: 4.73 minutes.
Encoder end.
File size difference  = 1.024159
Low fence: 73.055481
High fence: 79.800003
Last CQ = 73.06
Current CQ = 76.43
CQ difference = 3.372261
Using CQ of 76.43
Prediction cycle #3
Encoder started...
Process time: 4.87 minutes.
Encoder end.
File size difference  = 0.994805
Low fence: 73.055481
High fence: 76.427742
Last CQ = 76.43
Current CQ = 74.74
CQ difference = 1.686134
Using CQ of 74.74
Prediction cycle #4
Encoder started...
Process time: 4.75 minutes.
Encoder end.
File size difference  = 0.996434
Low fence: 73.055481
High fence: 74.741608
Last CQ = 74.74
Current CQ = 74.48
CQ difference = 0.266525
Using CQ of 74.48
Prediction cycle #5
Encoder started...
Process time: 4.75 minutes.
Encoder end.
File size difference  = 0.996678
Low fence: 73.055481
High fence: 74.475082
Last CQ = 74.48
Current CQ = 74.23
CQ difference = 0.247437
Using CQ of 74.23
Prediction cycle #6
Encoder started...
Process time: 5.33 minutes.
Encoder end.
File size difference  = 0.996914
Low fence: 73.055481
High fence: 74.227646
Last CQ = 74.23
Current CQ = 74.00
CQ difference = 0.229034
Using CQ of 74.00
Prediction cycle #7
Encoder started...
Process time: 4.80 minutes.
Encoder end.
File size difference  = 0.998487
Low fence: 73.055481
High fence: 73.998611
Last CQ = 74.00
Current CQ = 73.89
CQ difference = 0.111969
Encoding set to Full encode.
Full encode start...
CQMatic complete!
Total minutes of process: 34.10
Process ended at 14:36:53
On 08/06/2003

Here's the same .avs processed with ToK using MIN=523.83 and MAX=2,000:

Code:

=============================================================
ToK Log: K:\red-raw.avs
=============================================================
 
Resolution (fps):352x480 (23.976 fps)
Total Frames: 153232
Total Time  : 01:46:31
-------------------------------------------------------------
 
Audio Size: 89,474,000
Required Video Size: 729,726,000
 
Factor: 59.940
Desired Sample Size: 12,174,275
 
-------------------------------------------------------------
New Faster Prediction
-------------------------------------------------------------
 
Full Sample
Next CQ: 80.000. Sample Size: 13,777,673
Small Sample
Next CQ: 80.000. Sample Size: 13,777,673
Predicting...
Next CQ: 70.690. Sample Size: 9,261,035
Next CQ: 76.020. Sample Size: 11,339,836
Next CQ: 77.696. Sample Size: 11,353,185
Next CQ: 78.662. Sample Size: 11,360,750
Next CQ: 79.222. Sample Size: 12,395,118
Next CQ: 79.022. Sample Size: 11,966,978
Next CQ: 79.120. Sample Size: 12,138,553

Exit Condition: 0.500 % reached ! yahoo !
Tries   : 8
 

Final CQ: 79.120
Total Time For Predicition: 00:21:59

Total Time (all operations): 00:21:59
 
Finished

And again with ToK, now MIN=300, MAX=2,500:

Code:

-------------------------------------------------------------
New Faster Prediction
-------------------------------------------------------------
 
Full Sample
Next CQ: 80.000. Sample Size: 13,938,086
Small Sample
Next CQ: 80.000. Sample Size: 13,938,086
Predicting...
Next CQ: 69.876. Sample Size: 8,863,847
Next CQ: 75.710. Sample Size: 11,352,100
Next CQ: 77.464. Sample Size: 11,391,108
Next CQ: 78.488. Sample Size: 11,417,857
Next CQ: 79.093. Sample Size: 12,145,163

Exit Condition: 0.500 % reached ! yahoo !
Tries   : 6
 

Final CQ: 79.093
Total Time For Predicition: 00:17:11

Total Time (all operations): 00:17:11
 
Finished

In both cases, ToK predicted almost the same CQ, but by looking at the CQMatic log, a CQ of ~80 with MIN=523.83 and MAX=2,000 we know that the file will be over 100MB in size

So at 79.093 it will still be way over. Then, probably with the same CQ, but MIN=300, MAX=2,500 it will be smaller.

So here we have it. Complete different results, by just a change in MIN and MAX, throws off file size in the range on hundreds of megabytes

Now, I really believe that an alternate prediction method should be investigated, because the sampling method used in ToK, acp and CQMatic obviously doesn't work every time with TMPEG, and we've been lucky every time we hit a correct target. That's what I think.

-kwag

vhelp · #85 08-06-2003, 08:26 PM

Hi kwag..

If I can jump in here for a moment..

member how you favored so well, AVS v2.08 ??
Does that STILL apply today, or R U saying that this version was wrong
all along too ??

Cause, if you're not saing so, then perhaps it's just an AVS v2.52 issue as
I've theorised all along.

Ok, so you've testd AVS v2.52 w/ CQM.. HOW ABOUT TESTING AVS v2.08
w/ CQM (at least w/ prediction only) to get an idea of what's what ??

At least try the above out, and see, so that you don't go pulling your hair
out over an NEW prediction method.

bye for now..
-vhelp

vhelp · #86 08-06-2003, 08:30 PM

@ Kwag..

Hay, I can't believe I'm actually quoating myself, for the first time.. anyways..

Quote:

Originally Posted by vhelp

Hi kwag..
Ok, so you've testd AVS v2.52 w/ CQM.. HOW ABOUT TESTING AVS v2.08
w/ CQM (at least w/ prediction only) to get an idea of what's what ??

At least try the above out, and see, so that you don't go pulling your hair
out over an NEW prediction method.
-vhelp

Since you're not using MA in your script, definately, most defenatly try again
w/ EXACT same script you used in above test w/ same params and things !!

-vhelp

kwag · #87 08-06-2003, 09:06 PM

Quote:

Originally Posted by vhelp

Hi kwag..

If I can jump in here for a moment..

member how you favored so well, AVS v2.08 ??
Does that STILL apply today, or R U saying that this version was wrong
all along too ??

Cause, if you're not saing so, then perhaps it's just an AVS v2.52 issue as
I've theorised all along.

You have a point, which I missed

I'm going to do an encode of "Red Planet", but directly from the .d2v project, bypassing AviSynth completely.
Thanks for bringing this up

-kwag

vhelp · #88 08-06-2003, 09:13 PM

@ Kwag..

Quote:

Originally Posted by kwag

Quote:

Originally Posted by vhelp

Hi kwag..

If I can jump in here for a moment..

member how you favored so well, AVS v2.08 ??
Does that STILL apply today, or R U saying that this version was wrong
all along too ??

Cause, if you're not saing so, then perhaps it's just an AVS v2.52 issue as
I've theorised all along.

You have a point, which I missed

I'm going to do an encode of "Red Planet", but directly from the .d2v project, bypassing AviSynth completely.
Thanks for bringing this up

-kwag

yes, directly via a .d2v file. That was what I ment in my previous post

yes.. excellent !! great movie too, and I have on DVD

..note, I'm only trying to help, if I can - no matter how small

And, you're welcomed

-vhelp

Dano · #89 08-06-2003, 11:08 PM

Don't know if this is relevant or not but I had some problems with the July 17 version of AviSynth 2.52. I went back to the previous version and all is well again.

vhelp · #90 08-07-2003, 08:04 PM

Hi everyone.

@ Kwag..

I think I may have a "possible" cause of the issues w/ prediction havoc

The answer may lie in viewing what happens to shorter-length clips ie,
1 VOB encodes or ~25 min. ( ie, movie, "Matrix" ) predicitions.

I won't say what happens, because I think you may jump the gun, and claim
it's something else or my debug in flawed

But, I must say, I did notice something, that I think might shed some light,
on the matter - maybe.

But, I'll have to run some more tests. I'm heading on to "Get Karter"
cause I also did that one in 1 VOB format (21 minutes)

Below are my "process" params.. follow it to the letter, to obtain the same
results as mine. But, first, here is the LOG file, followed by the process:

Only you will/might know what the issue(s) are after viewing/trying out the test for your self, since you are the developer of CQM

Please let me know what you think about this (after you've performed the
test(s) yourself) - K ??

Thanks for your attention,
-vhelp
.
.

Quote:

http://www.kvcd.net
CQMatic Version 1.1.02
Copyright Softronex Corporation, 2003.
All rights reserved.
Time: 20:18:35 Date: 08/07/2003
Ready!
Project: H:\en\v2.520-matrix.01.tpr

Creating: CQMatic.tpr

H:\matrix.mpg
Project: H:\en\v2.520-matrix.691x394.01.tpr

Creating: CQMatic.tpr

H:\matrix.mpg
Project: H:\en\v2.520-matrix.136x691x394.01.tpr

Creating: CQMatic.tpr

H:\matrix.mpg
Execute.
Movie Time: 136
Average Bitrate: 691
Prediction Only mode
Executing Prediction Phase...
Process started at 20:28:28
On 08/07/2003
CQ set for prediction
Setting up initial sampling.
Using CQ of 60.00
Prediction cycle #1
Encoder started...
Process time: 4.27 minutes.
Encoder end.
File size difference = 3.179569
Low fence: 60.000000
High fence: 90.000000
Last CQ = 60.00
Current CQ = 90.00
CQ difference = 30.000000
Using CQ of 90.00
Prediction cycle #2
Encoder started...
Process time: 4.30 minutes.
Encoder end.
File size difference = 2.867288
Low fence: 90.000000
High fence: 90.000000
CQ ABOVE watermark. CQ set to 90.0
Using CQ of 90.00
CQMatic complete!
Total minutes of process: 8.57
Process ended at 20:37:02
On 08/07/2003

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --

Quote:

* DVD TITLE: "Matrix"
- - - NTSC - 16:9 - 136 minutes

* RIP: smartripper v2.27
- - - ripped in File Mode

* CLIENT: dvd2avi v1.76
- - - forcedFilm, rgb, tv

* VFAPI: reader codec v1.04b
- - - default params, to psuedo .avi file
- - - "for frameserving in vdub to TMPG"

* FRAMESERVER: vdub v1.49["01.00.vdr"]
- - - Filters: nulltransform (for cropping/letterbox)

* Encoder: TMPG v2.520
- - - MPEG-2 ES (VidOnly) / 704x480 / IN(16:9) OUT(16:9)
- - - vid: 23.976 / 3:2pldn / 8bits /
- - - adv: progessive / field b / 16:9 / FS[KARatio]
- - - GOP: 1,5823,2,1,24 / [x] Output bitstream...
- - - Qua: "notch" / [x] Use floating point...

kwag · #91 08-07-2003, 08:12 PM

Hi vhelp,

You entered 136 minutes for a ~25 minute VOB

That's why CQMatic gave you a CQ of 90.
You must enter the correct time in CQMatic, as it is used to calculate the size of the sample needed for the real target.

-kwag

vhelp · #92 08-07-2003, 08:19 PM

Hi Kwag..

Quote:

You entered 136 minutes for a ~25 minute VOB
That's why CQMatic gave you a CQ of 90.
You must enter the correct time in CQMatic, as it is used to calculate the size of the sample needed for the real target.

Ok, thanks.. but what about what I said above ??

In any case, I'm off to try "Get Karter".. just to see the results, to see if
I get the same experience I got w/ "Matrix"

In any event, I'll also try YOUR suggest w/ BOTH these movies afterwards

Question:
* given the above, and your response to my movie lenth (136min) what
or how should I count-balance w/ respect to BITRATE.. assuming I want to
fill a CDR ie, 800mb w/ a 25 minute (Like I'm gonna do that, but you never
know) ... for BITRATE distribution, how should I "balance" it ??
* also, to minimize the amount of time spending on testing your CQM and
predition debuggin, how should I set the bitrate (if it needs to be) and the
time required, if I have ie, 25 minute VOB I want to test predictions with,
rater than waiting 55 minutes for prediction tests, maybe cutting it down to
something like, 5 to 10 minutes ??

Pretty lame questins, but can you give anything for each for them ??

Thanks for the assist and all,
-vhelp

kwag · #93 08-07-2003, 08:26 PM

Quote:

Originally Posted by vhelp

Ok, thanks.. but what about what I said above ??

In any case, I'm off to try "Get Karter".. just to see the results, to see if
I get the same experience I got w/ "Matrix"

In any event, I'll also try YOUR suggest w/ BOTH these movies afterwards

Question:
* given the above, and your response to my movie lenth (136min) what
or how should I count-balance w/ respect to BITRATE.. assuming I want to
fill a CDR ie, 800mb w/ a 25 minute

You can't

Because 25 minutes even at CQ=90, won't fill a CD-R ( with MIN=300, MAX=2,500 )

CQMatic will always calculate a CQ of 90. 25 minutes is just too short, so even at CQ=100 and a resolution of 704x480, you won't fill a CD-R. Maybe, just maybe at 704x480 you get slightly under 90, but I don't think so, unless you set your audio to 224Kbps and MAX=4,000

-kwag

vhelp · #94 08-07-2003, 08:33 PM

@ Kwag..

w/ respect to below, ok.. thanks.. I understand now.. and, it makes LOTS
of sense

dah..

One more thing.. about CQM, could you add in an option somewhere's
(even during copy to clipboard dialog box) a feature to CLEAR the contents
of the LOG, so that I can restart another predict test, w/out having to shut
it down, and start it up again w/ an empty window (minues your (c) .. ) ??

Thanks again, and testing "Get Karter" w/ CQMatic-1.1.02.exe anyways..

-vhelp
.
.

Quote:

Originally Posted by kwag

Quote:

Originally Posted by vhelp

Ok, thanks.. but what about what I said above ??

In any case, I'm off to try "Get Karter".. just to see the results, to see if
I get the same experience I got w/ "Matrix"

In any event, I'll also try YOUR suggest w/ BOTH these movies afterwards

Question:
* given the above, and your response to my movie lenth (136min) what
or how should I count-balance w/ respect to BITRATE.. assuming I want to
fill a CDR ie, 800mb w/ a 25 minute

You can't

Because 25 minutes even at CQ=90, won't fill a CD-R ( with MIN=300, MAX=2,500 )

CQMatic will always calculate a CQ of 90. 25 minutes is just too short, so even at CQ=100 and a resolution of 704x480, you won't fill a CD-R. Maybe, just maybe at 704x480 you get slightly under 90, but I don't think so, unless you set your audio to 224Kbps and MAX=4,000

-kwag

kwag · #95 08-07-2003, 08:47 PM

Quote:

Originally Posted by vhelp

Thanks again, and testing "Get Karter" w/ CQMatic-1.1.02.exe anyways..

Check your PM

-kwag

vhelp · #96 08-07-2003, 08:55 PM

Hi Kwag..

Quote:

Originally Posted by kwag

Quote:

Originally Posted by vhelp

Thanks again, and testing "Get Karter" w/ CQMatic-1.1.02.exe anyways..

Check your PM

-kwag

I'm off to testing CQMatic-1.1.03.exe now..

I'm going to use the single VOB's time length.. ie, 21 minutes for the
movie "Get Karter" in this next (doing now)

I'm trying to cut down on the length of time it takes to test, in hopes that the
test will still hold up to "prediction" debuggin. If it does, great!! But, if it
does not, ..oh well, we'll have to live w/ the longer debuggin time

and
use the full-length of movie, for testing/debuggin for ya

Sorry about the premature hype earlier. But, it did get up hehe

-vhelp

vhelp · #97 08-07-2003, 09:17 PM

Kwag..

Just wanted to let you know, that CQMatic-1.1.03.exe did a better
job at finding a CQ value (for the 21 min test clip) - - gave me 50.17
in 22 minutes time vs. 55 minutes in previous testing of full-length movies.

I hope that it does't matter what length we use (shorter in my case) for the
prediction testings because of this. Please le me (us all) know if this does
taint the prediction process.

The way I read it, CQ 50.17 will fit my whole movie of "Get Karter" w/ 103 min.
onto one 800mb CDR !! (if I read/understood that correctly) ..yes ??

Listen, if using shorter length movie test clips is proved to be no valid for
CQM testing/debuggin, then WHY NOT USE IT ?? Will take less time to work
out the issues, not to mention, less headache

There's a bonus to useing shorter length test clips though

I'm off to give "Matrix" another go, but this time w/ 25 minutes..

Below, is my "Get Karter" LOG and Process, using only a 21 minute test VOB (via psuedo .avi for source) w/ latest CQMatic-1.1.03.exe:

-vhelp
.
.

-----------------------------------------

Quote:

http://www.kvcd.net
CQMatic Version 1.1.03
Copyright Softronex Corporation, 2003.
All rights reserved.
Time: 22:07:24 Date: 08/07/2003
Ready!
Project: H:\en\v2.520-getkarter.103x948x541.cqm(v1.1.03).tpr

Creating: CQMatic.tpr

H:\getkarter.mpg
Execute.
Movie Time: 21
Average Bitrate: 948
Prediction Only mode
Executing Prediction Phase...
Process started at 22:07:49
On 08/07/2003
CQ set for prediction
Setting up initial sampling.
Using CQ of 60.00
Prediction cycle #1
Encoder started...
Process time: 1.18 minutes.
Encoder end.
File size difference = 0.862505
Low fence: 2.000000
High fence: 60.000000
Last CQ = 60.00
Current CQ = 51.75
CQ difference = 8.249680
Using CQ of 51.75
Prediction cycle #2
Encoder started...
Process time: 1.20 minutes.
Encoder end.
File size difference = 0.938964
Low fence: 2.000000
High fence: 51.750320
Last CQ = 51.75
Current CQ = 48.59
CQ difference = 3.158630
Using CQ of 48.59
Prediction cycle #3
Encoder started...
Process time: 1.20 minutes.
Encoder end.
File size difference = 1.010314
Low fence: 48.591690
High fence: 51.750320
Last CQ = 48.59
Current CQ = 50.17
CQ difference = 1.579315
Using CQ of 50.17
Prediction cycle #4
Encoder started...
Process time: 1.52 minutes.
Encoder end.
CQMatic complete!
Total minutes of process: 5.12
Process ended at 22:12:56
On 08/07/2003

--------------------------------------------

Quote:

* DVD TITLE: "Get Karter"
- - - NTSC - forced film - 16:9 - 103 minutes

* RIP: smartripper v2.27
- - - ripped in File Mode

* CLIENT: dvd2avi v1.76
- - - forcedFilm, rgb, pc

* VFAPI: reader codec v1.04b
- - - default params, to psuedo .avi file
- - - "for frameserving in vdub to TMPG"

* FRAMESERVER: vdub v1.49["01.00.vdr"]
- - - Filters: nulltransform (for cropping/letterbox)

* Encoder: TMPG v2.520
- - - MPEG-2 ES (VidOnly) / 704x480 / IN(16:9) OUT(16:9)
- - - vid: 23.976 / 3:2pldn / 8bits /
- - - adv: progessive / field b / 16:9 / FS[KARatio]
- - - GOP: 1,5823,2,1,24 / [x] Output bitstream...
- - - Qua: "notch" / [x] Use floating point...

kwag · #98 08-07-2003, 10:09 PM

That looks good

Now everyone come in, and join the fun ( or the disaster

)
http://www.kvcd.net/CQMatic-1.1.03.exe

-kwag

vhelp · #99 08-07-2003, 11:31 PM

hi Kwag..

I got a stupid question to ask..

Is there any way to get TMPG's old encoding back, so that I can encode
JUST THE AUDIO portion of that last test sample I did ??

I haven't done anything to TMPG yet. I just want to change it to Audio Only
and then I'd like to MUX them (aud/vid) to one file. I want to fix the glitch
that's causing it to studdre in Windvd. Powerdvd plays it fine, but for those
that play it in windvd would not benefit from it otherwise. Is there anything
that I can do (regarding above) after finishing a CQM test, just to get the
audio portion of the SAME and last encode from CQM ??

Hope you understood that. Thanks,
-vhelp

kwag · #**100** 08-07-2003, 11:53 PM

Select "Prediction Only" in CQMatic. When the process is finished, run TMPEG. You'll have the last CQ already set. Just clear your range, select "System (Audio+Video") and add your audio in the "Audio Source" of TMPEG.

Now you can encode full Video + Audio.

-kwag