Quantcast Bitrates: CQmatic 1.4.00 Release Candidate #2 - digitalFAQ.com Forums [Archives]
  #1  
12-23-2004, 01:07 AM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts


What's new in this version

1) Completely re-written prediction engine. Sampling now depends on movie length, so it's now dynamic.
You should be able to throw just about anything at CQMatic, and it will know "how" to encode "whatever" material (PAL/NTSC, etc)

2) Prediction is now done in "phases". Then, using a formula and some tricks (don't ask ), it determines the correct target CQ.

3) Prediction is MUCH faster
On my Pentium 4 @2.8Ghz, I am averaging a full prediction cycle in about 20 to 25 minutes minutes, on a average movie of 1 hour, 45 minutes (on linear range, CQ=~50 to CQ=~85)
Below CQ=~50, you all know it takes longer, and I can't do a thing about it
The first phase, in theory, should be the longest.
All subsequent phases uses the previous found CQ, which is closer to final value, instead of starting every phase at CQ=60, as in previous versions.
Still, you should count on about 1/5 of your total movie time, for prediction time.
Still, it beats the hell out of any multipass encode

4) Removed "Override CQ" option. It's now done automagically

5) Added progress bar, to give you a visual idea where the process is.

Here you have it: http://up1.fastuploads.info/CQMatic-1.4.00-RC-2.exe.zip

Please report any problems on this same thread.
If everything goes well, and no bugs are found, I will cut the software as 1.4.00 Release
If there are inconsistencies, then there might be more release candidates before final release.

Thanks to everyone, and all who BETA tested
Enjoy
-kwag
Reply With Quote
Someday, 12:01 PM
admin's Avatar
Site Staff / Ad Manager
 
Join Date: Dec 2002
Posts: 42
Thanks: ∞
Thanked 42 Times in 42 Posts
  #2  
12-23-2004, 10:53 AM
Racer99 Racer99 is offline
Free Member
 
Join Date: Sep 2002
Location: Massachusetts
Posts: 119
Thanks: 0
Thanked 0 Times in 0 Posts
Great Tool Karl!

I know you got your hands into many things and this would be just one of many in a long line of requests, but what about taking some of this functionality to predict with NuEnc and it's CQ function.

Right now I am doing it manually. It's super simple and easy to determine as it seems mostly linear with Peter Cheat's code modificatins. Right now I am running commandlines via batch files.

Greg a.k.a. Racer99
Reply With Quote
  #3  
12-23-2004, 11:56 AM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by Racer99
but what about taking some of this functionality to predict with NuEnc and it's CQ function.
I will, once libavcodec based encodes prove to be more stable. Their current state is still not useable for production environments.
Quote:

Right now I am doing it manually. It's super simple and easy to determine as it seems mostly linear with Peter Cheat's code modificatins. Right now I am running commandlines via batch files.
I will support it, once I define a very simple API file, where you will put your basic parameters, similar to a .tpr file in TMPEG, and then your batch file. Just to make it transparent and similar to the current process.
Quote:

Greg a.k.a. Racer99
Thanks
-kwag
Reply With Quote
  #4  
12-23-2004, 12:36 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
RC-3 is up.
http://up1.fastuploads.info/CQMatic-1.4.00-RC-3.exe.zip

You might want to give this a try.
RC-2 does 5 test phases.
RC-3 does 7 phases.

I've had extremely good results with RC-2, but on ONE case, I got a file size over the wanted.
This was due to the damn CQ (un)linear range in TMPEG, when target is in the range of ~69 to ~70, where just one click of CQ makes a huge diffference in final file size.
With two aditional phases, this should zero in more on the final precision for these particular cases.
I'm taking a constant prediction time of around 37 minutes with RC-3, for a full movie, which takes me about 2.5 hours to encode.
That's about a 25% (worst case) overhead for prediction, which I consider ok, because what I want is final size to be within a calculated target of no more that -2% to +2%
As I've said before, if your final CQ will fall in a range of ~50 or less, you can count on a VERY long time to predict due to TMPEG's behaviour, and I can't do a thing about that

-kwag
Reply With Quote
  #5  
12-23-2004, 02:22 PM
Dialhot Dialhot is offline
Free Member
 
Join Date: May 2003
Posts: 10,463
Thanks: 0
Thanked 0 Times in 0 Posts
YES !

Movie length : 146 min - Target half DVD
Wanted Size : 2109062
Encoded Size: 2078620

Note: RC-2 found CQ= 81.50 and hit the target.
Beta-4 found CQ=89.5 and was a little over (above 2200000)
Beta-2 found CQ=80.6 and was a lot under (19000000)
Beta-1 found CQ=79.8 and was a lot under (17800000)
Reply With Quote
  #6  
12-23-2004, 03:28 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by Dialhot
Note: RC-2 found CQ= 81.50 and hit the target.


Now predict with RC-3, please
It should be very close, on every run.

-kwag
Reply With Quote
  #7  
12-23-2004, 04:10 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Mi results, with RC3.

Test bed movie: Red Planet.
Time 102 minutes, 39 seconds.

Previously encoded with a CQ of 65.81 (from older version of CQMatic) , which rendered a file size of 709,325KB.
My wanted file size, based on CalcuMatic, is 722,734KB.

Test runs
Code:
CQ found            Time to predict (Min:Sec)

1) 67.997             36:82
2) 69.54              37:80
3) 65.531             37:23
4) 67.6989            32:05
5) 66.5176            27:45
6) 66.098             25:50
7) 66.1033            36:35
Edit: The above were seven individual runs Not a single 7 phase prediction.

I assume that If I encode with all of those CQ values, the final target will be well within the +-2%, as in the calculations in CQMatic
Remember that the maximum deviation of the final target should be no more than 2% below, or 2% above target. It's really a 4% "window", trying to keep the final size within that bracket.

Seems CQMatic 1.4.00 (release) will finally hit the road later today
I just compiled it, and fixed the progress bar to reflect the new 7 phases, and everything else is identical to RC-3.
Just waiting for more feedback, to see if we post it, or if there's a need for a RC-4 (hopefully not )

-kwag
Reply With Quote
  #8  
12-23-2004, 04:42 PM
digitall.doc digitall.doc is offline
Free Member
 
Join Date: Jul 2003
Location: Valencia (España)
Posts: 741
Thanks: 0
Thanked 0 Times in 0 Posts
The link is not working .
Are thousands of us trying to download it?.
I'll try again in several minutes... if there's no problem with the link
Reply With Quote
  #9  
12-23-2004, 04:52 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by digitall.doc
The link is not working .
Are thousands of us trying to download it?.
I'll try again in several minutes... if there's no problem with the link
I guess that site is down
Well, here you have the release
http://www.kvcd.net/downloads/CQMatic-1.4.00.exe

-kwag
Reply With Quote
  #10  
12-23-2004, 05:13 PM
ginoboy ginoboy is offline
Free Member
 
Join Date: Apr 2004
Posts: 922
Thanks: 0
Thanked 0 Times in 0 Posts
nice!
Reply With Quote
  #11  
12-23-2004, 09:16 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Thanks ginoboy

Quote:
Originally Posted by Dialhot
Movie length : 146 min - Target half DVD
Wanted Size : 2109062
Encoded Size: 2078620
And that's exactly 1.4645 % difference

I just re-did "Rambo III", with another script, and look:
Target is 2 CDs, full screen movie (KVCD)

Wanted: 1,520,640KB
Encoded: 1,541,067KB

For a 1.3433 % accuracy

-kwag
Reply With Quote
  #12  
12-23-2004, 09:32 PM
danpos danpos is offline
Free Member
 
Join Date: May 2004
Location: Rio de Janeiro - Brasil
Posts: 538
Thanks: 0
Thanked 0 Times in 0 Posts
@Kwag

It´s very cool, Mr. Karl Wagner! More one "goal" for you! I´ll download it and I´ll let it a shot!

Keep up on work, mate!

My Best Wishes,
Reply With Quote
  #13  
12-23-2004, 09:36 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Thanks danpos
Please test it, and let me know if it fails somewhere

-kwag
Reply With Quote
  #14  
12-24-2004, 01:39 AM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Another test
Just for the hell of it, I encoded (my daughter's DVD) Harry Potter Prisoner Of Azkaban, at 352x340. The DVD is the full screen version.
I just wanted to see how accurate the encode was going to be, because I really don't do any 352x240 encodes anymore
But her TV is 27", so 352x240 look ok, if done properly.

Anyway, here are the results:

Movie time: 141 minutes, 39 seconds.
Target: One CD
Wanted file size: 689,974KB
Encoded file size: 672,345KB

And I'll say NOT BAD, for a 2 hour, 21 minute Full Screen movie, on one CD
I only used this script:
Code:
Undot()
Limiter()

BicubicResize(352, 238, 0, 0.6, 5, 0, 710, 480)

MergeChroma(blur(1.5))
MergeLuma(blur(0.1))

AddBorders(0, 1, 0, 1)
LetterBox(8, 8, 8, 8)
Limiter()
Yes, it could be WAY better with proper filtering, but I just wanted to test CQMatic
Little sample here: www.kvcd.net/downloads/hp.Sample.m1v

-kwag
Reply With Quote
  #15  
12-24-2004, 07:48 AM
Dialhot Dialhot is offline
Free Member
 
Join Date: May 2003
Posts: 10,463
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by kwag
Code:
CQ found            Time to predict (Min:Sec)

1) 67.997             36:82
2) 69.54              37:80
3) 65.531             37:23
4) 67.6989            32:05
5) 66.5176            27:45
6) 66.098             25:50
7) 66.1033            36:35
I assume that If I encode with all of those CQ values, the final target will be well within the +-2%, as in the calculations in CQMatic
I'm not very confident with value #2. Can you encode with this one to be sure ?
Reply With Quote
  #16  
12-24-2004, 10:32 AM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by Dialhot
I'm not very confident with value #2. Can you encode with this one to be sure ?
I sure will
Starting now.

-kwag
Reply With Quote
  #17  
12-24-2004, 10:40 AM
Dialhot Dialhot is offline
Free Member
 
Join Date: May 2003
Posts: 10,463
Thanks: 0
Thanked 0 Times in 0 Posts
BTW what is important is to tell in the stricky post (I guess you updated it with new release) is that new engine works on a random sample and 2 laucnches won't give the same value. So if an encoded is over to far from the target, just restart a CQ guess and the new value will have great chances to be the good one.
Reply With Quote
  #18  
12-24-2004, 10:58 AM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by Dialhot
BTW what is important is to tell in the stricky post (I guess you updated it with new release) is that new engine works on a random sample and 2 laucnches won't give the same value. So if an encoded is over to far from the target, just restart a CQ guess and the new value will have great chances to be the good one.
Yes, you are correct. Every run will give different, but close values.
And it's very possible that CQMatic could fail, if it gets in that non linear range of TMPEG, and will produce a file size much more that the 2% wanted
But the chances are now much lower than in previous versions.
It's really a nightmare to predict in that area

Edit: Sticky updated with note.

-kwag
Reply With Quote
  #19  
12-24-2004, 04:50 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
@Dialhot

Result of encode at CQ=69.54
File size: 714,913KB

Which clearly explains that the area of 65.x to 69.x is almost flat

Look what hapened on Harry Potter, on two separate runs:

CQ = 69.384. File size: 629,105KB

CQ = 70.440. File size:
672,345KB

The CQ difference is only 1.056, but the file size difference is 43,240KB

And that my friends, is what I call a VERY sloppy CQ curve in TMPEG
The only way I think that I could correct this in CQMatic, is charting a complete CQ curve, in steps of .1, and then depending on the final calculated CQ value, compare the value to the chart, and adjust it so it doesn't overshoot the file size.
But the problem is that every version of TMPEG has a different CQ curve, and this would be a nightmare to maintain.
So my advice is: If your file size is way off, run CQMatic one more time. Chances are you'll get the correct file size on the second run.

-kwag
Reply With Quote
  #20  
12-24-2004, 05:55 PM
Dialhot Dialhot is offline
Free Member
 
Join Date: May 2003
Posts: 10,463
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by kwag
Look what hapened on Harry Potter
You won't believe me but... the 146 min long movie I use for testing is... Harry Potter
Reply With Quote
Reply




Similar Threads
Thread Thread Starter Forum Replies Last Post
Bitrates: CQmatic nighthawk Video Encoding and Conversion 36 01-14-2004 11:20 AM
Bitrates: CQMatic & TOK serrabastien Video Encoding and Conversion 9 09-08-2003 08:09 PM
Bitrates: How to use CQMatic? mistermickster Video Encoding and Conversion 1 08-29-2003 06:47 AM
Bitrates: Help when CQMatic is over/under J-Wo Video Encoding and Conversion 1 08-23-2003 12:19 PM
Bitrates: KDVD bitrates with CQMatic nicksteel Video Encoding and Conversion 10 08-06-2003 08:44 AM

Thread Tools



 
All times are GMT -5. The time now is 06:23 PM  —  vBulletin © Jelsoft Enterprises Ltd