CQMatic/CalcuMatic Continued...
 

Thanks Rui :)
Thanks Phil :)

-kwag

Karl, do you envisage any problems using these programs uder WINE?

I managed to get TMPGEnc working (as per your advice of 4/1/05 in this thread http://www.kvcd.net/forum/viewtopic....378&highlight= ). It doesn't seem to be encoding the audio tho - I'll try besweet, although I'm probably lacking mp2 codecs in MPlayer I think...

Its just when I tried Calcumatic using WINE, the program terminates when I point it to a file (in this case a non-warez avi file). Also, the top 3 buttons did not appear until I clicked where they ought to be.

Sorry if this is wandering off topic.... :?

I haven't tried it :!:
But it should work. As you already have WINE installed, why not try it :D Well, you could just encode the audio with ffmpeg :idea: CalcuMatic uses DirectX to open media files. Maybe that's why it doesn't work correctly on Linux under WINE?It's fine :)

-kwag

Hi guys :),
Karl, from what I could read on mplayer's mailing lists, it is not very good to encode audio with it.
And both Mplayer and FFmpeg are based on libavcodec, right?
I wonder if toolame would work under wine?
This is getting pretty interesting, although a bit off-topic :lol:.
What other audio encoders are there that can be used under *nix, Karl?
Cheers

Wasn't tooLAME included some time ago? By the way, the author's apparently working on a new version which should fix a lot of bugs in tooLAME.

Sorry Kwag.
It's not fair to post my result, since it's my first test with CQmatic 1.4.04. But I want to post it to see if I did something wrong.
The DVD is "Los Lunnis", PAL, AR 4:3, 81 min 08 sec.
TMPGenc Plus 2.521.58.169 (just in case...)
My script is MA with triming, RemoveGrain instead UnDot, and FieldDeinterlace.
In first prediction (fast phase with 10 cycles, and 7 prediction cycles) CQ was 60,25. Final file size 698952 Kb (desired 731072 Kb, -4,4% :( ).
I run a second prediction, now CQ 63,82 ( 8O ). After encoding, final file size is 777914 Kb (+6% :cry: ).
Is this just bad luck, and in my first test I met with the worst non-linear face of TMPGenc CQ mode?, or did I anything wrong?.
I don't know what to do for next encoding... the bigger encoded would need a 80 kbps audio... too bad. I'll try something around CQ=62,5 and see.

As far as I remember CQMatic and Trim are very bad friends.

Welcome to TMPGEnc's CQ roller coaster :!:
You had bad luck, and as I've said before, this CAN happen and I (or anybody else!) can't do a thing about it (for the time being).
The ONLY solution would be to add a final phase of prediction, with a very narrow % of accuracy on wanted size. The problem with this, as you have just experienced, is that with only a CQ difference of 3.57 you have a file size deviation of 78962 KB :!:
And if I implement this in CQMatic to try and zero in closer on the target, which can easily be done, then the time to predict that final CQ will probably be 4 to 5 times the time it normally takes. Probably even longer than a regular 1 pass time of a 2 pass encode.
What really pisses me off, is that I have sent on various ocassions E-Mails to Pegasys, Inc. related to this problem, and they have NEVER answered :(
And I am a registered user of TMPGEnc Plus. So it seems they don't really care about the issue.
I'll just keep trying my best, and I'll try other possible algorithms to see if I can overcome this CQ inconsistency.
I have an idea, but it will be very hard to implement and to maintain, and that is to create a CQ table with 0.1 CQ step increments, and then I will know how much to correct a final CQ value.
The only problem is that if I implement this, every CQMatic version will be tied to a specific TMPGEnc Plus version,
So every time Pegasys, Inc. releases a new version, I would have to rebuild the table with their new encoder, and then release a new version of CQMatic for that particular version.
And it seems that this will be the only way to combat their behaviour :twisted:

Edit:
1) I'll probably start on this right now ;)
2) Instead of a specific version of CQMatic for a specific version of TMPEG, I'll probably release a "footprint" configuration file for each version of TMPGEnc :idea: :idea:
This way, I'll write a small "Generator" program that will encode a specified material in steps of 0.1 CQ values, and then I will have a footprint of how the particular version of TMPEG behaves at different CQ values.

Time to crank up the development system again :!:

-kwag

Karl,

I have an other small problem : on win2000 the tmpgenc window isn't iconified by CQMatic and stay in the front screen :-(

8O 8O 8O 8O 8O
Win2k again :x
I'll take a look.
This just never ends, right? :mrgreen:
But I shouldn't feel that bad, after I see these kind of things (again!) :rotf:
http://p2pnet.net/story/3473

-kwag

This is the second time it happens to Bill while a (quite) important presentation. This can be wished...

(how many media will speak about this and would have never written the word M$ if this "accident" didn't occur ?)

Pain in the A**
Here's what the new CQ curve looks like on TMPGEnc version 2.524.63.181 (Latest)

http://www.digitalfaq.com/archives/i.../2005/01/2.jpg

I didn't put any labels. Left shows file size, bototm shows CQ 40 to 92.
The graph shows a 5 second individual encodes, from CQ=40 to CQ=92.
This is worse than previous versions of TMPGEnc!, as I can see an actual decrease of file size at some increasing CQ value points 8O

-kwag

:drunkard: :dark: :drink: :grenade:

This was the case with some older versions, don't remember which. Needless to say, I ran into it a couple of times :x

:banghead: :banghead: :banghead: :banghead: :banghead:

Kwag, due to that bullshit my little remark:
Theres a executable binary on doom9 of that requantizer, the core also used by Rejig. So at least on DVD-R mpeg2 Targets you can offer an option to slightly increase the float of the CQ value where a requant process afterwards could be 100% fixing processor. The procedure we used in case of overheaded mpeg2 encodings.
For mpeg1, there is one option:
We know that there do exist DVD Folder generating commandline apps, like DVDauthor, so the mpeg1 and sound can be processed for outputting a Video_TS architecture which will be shrinked afterwards (does Rejig accept mpeg1 via authored Image inputs like ShrinkDVD does? If yes than this could be an option. Finally the requantized vob will be demuxed to mpeg1 and mp2 again.

The downside: More time will be needed for that DVDauthoring/filesize/requantize postprocessing and maybe the fine requantizing does destroy the gain of the CQ based encoding.
So thats why the lines above are just an already existing idea from this forum which came into my mind.

Hi guys :),
Let us all flood Pegasys, Inc. email box with this issue.
Like a petition until they, at least, send us an email with an explanation of what we can expect from them in future releases of the encoder.
Who cares to join me?
Cheers

AFAIK Karl already tried to point Pegasys on this non-linear CQ issue in the past. :(

But that was only Karl, that is, a single person.
I mean massive email flooding ;-)
I guess they can't turn their back on us this way.
And if they fix it they'll sell a whole bunch more than they do now.

Good idea Rui :)
Please go ahead, and anyone else who feels can help.
Just imagine if Pegasys would fix the CQ issue and make it linear, CQMatic would predict in 2 or 3 runs :D

-kwag

OK, karl, thanx. I was already afraid of that...

Another question: when I run (under XP pro) CQmatic with TMPGenc minimized, if I open later TMPGenc it opens in a vetically stretched window: less than quarter the original height.

Now that's weird. I also run XP Pro (SP2), and I don't see that behaviour.
The only commands I send to the window are to "hide" and "minimize" :roll:

-kwag

I'll try again this option.
Maybe I did something wrong. :roll:

Option ? This is an option ?**LOL**

I didn't see there was a check box :-). That is surely why this does not work on my W2000 system :lol:

CQMatic updated today to 1.4.05

-kwag

Link ??

Hello ? Please ?

http://www.kvcd.net/forum/viewtopic.php?t=5145

Thanks ?

Thanks !! :)

As posted on the main page:

Updated CQMatic to version 1.4.08
Updated CalcuMatic to version 1.1.15

Changes:
Some code optimizations due to new compiler update.
Changed all tooltips to display as "Ballon" help, instead of old style square tooltips.
Minor internal code cleanups.

-kwag

Feature request(s) (still getting size miscalcs)
 

Kwag, you, as always, kick ass. CQMatic has made encoding a lot easier, if for no reason other than giving me a start point in CQ values from which to work with. Which brings me to my first suggestion:

Add final filesize checking to "Deep Prediction" mode. What I mean is have CQMatic check the final filesize to make sure it's within a certain percentage of the target size, and most importantly make sure the filesize isn't larger than the target size. If it is, the program would then adjust the CQ value once more and re-encode with the new settings. Even with Deep Prediction checked a full CQmatic run for me takes only 3 hours; I wouldn't mind if the time doubled, since it runs when I sleep.

The other issue I'm having is with rendering AVI files. CQMatic won't work with them. I don't know why, but the prediction cycle always takes only a few seconds and invariably produces a CQ value of 89.9+. This is a problem for me, as I like to pre-render my video with XviD to convert the credits to greyscale. It seems like it can't accurately gauge the source range, or something. If you can fnd out why it's doing it and fix it, that would rock.


Once again, thanks for the tools, they really do help with encoding.

By the by, I noticed that Wikipedia doesn't have an entry for KVCD. Perhaps someone should add one....

Re: Feature request(s) (still getting size miscalcs)
 

Thanks :D That's a problem :!:
The reason is I don't have any way of looking at TMPEG's time code.
That is, there's no way for me to tell if for example TMPEG is 50% done, so I can measure the current file size.
If I could, I could definitely take a file size snapshot at say 25% of the full encode, and if the file size is larger than what it's supposed to be at that point in time, I could kill the TMPEG process, and restart it with a lower CQ value. Indeed, and it won't work :!:
CQMatic was designed to work with TMPEG "project" files, which can only be saved and produce MPEG-1 or MPEG-2, because that's the only output options available on the main screen of TMPEG.
There's no other option supported on TMPEG's project file, so I can only support what is defined in the project file, for the time being.I just made a submission here ;)
http://en.wiktionary.org/wiki/KVCD

Thanks,
-kwag

Hi,

OT: and why is it in the category "Requests for deletion"?
It seems that Hippietrail delete it...
/OT

salu2
Fabrice

Hi Fabrice,

Where did you see that :?:

-kwag

The 50% or 25% idea had crossed my mind, but there have been movies where the first half took up much less space than the second half, and vice versa. A snapshot like that would only be useful on a "second time around" encode, after plotting a sort of "filesize growth path" with a full encode process. Of course, this is all academic, since you said you can't access TMPGEnc's timecode. But here's a thought: couldn't you just access the MPEG file being written and count how many frames are in it? That would give you a way to tell how much of the file is done.

Anyway, like I said, I was thinking more of having the filesize checked after a full encode is completed. I realize it would take time if the files kept coming in larger than expected, but it would just be doing what the users would have to do manually anyway. Perhaps make it another option, like "Overshoot Protection" or something, with a prompt that lets people know that it could make the total encoding time take a LONG time if the filesizes keep coming in too large.

Also, when I said "rendering AVI files", I meant converting an AVI file to an MPEG-1/2 file, things like fansubbed anime and such. CQMatic just plain freaks out when the video source is an AVI file.

Oh yes, had a bad render just a minute ago:

Source: Baptists at Our Barbecue
Source Length: 85 minutes 18 seconds
Calculated CQ Value: 81.286....
Audio Bitrate: 192 kbps
Target Video Size: 686128 KB
Actual Video Size: 806912 KB

I have no idea what caused it at all, but I thought you might want to know.

Keep up the good work.

CalcuMatic 1.1.16 released!
 

Fixed bug on resulting average bitrate and video size. (Wrong math used :oops: :cluebat: )
Result is now accurate, and using 1024 for Kbps, KB, MB, GB.
Tested against an encoded audio (CBR) and a 2-pass video, to validate final size. Result was perfect!

http://www.kvcd.net/downloads/CalcuMatic.exe

Enjoy!,
-kwag

@Kwag and all: Could you or somebody say me please, how Calcumatic obtains movie time from d2vs, avis, ifos, etc.
Or in a more general way, how can obtain it from anywhere?
I don't mean how can find out it from a program but how can I obtain it from my programmed tool.

Hi Prodater,

For reading .d2v files, take a look at the source code of DVD2AVI.
You can read the sources, and then implement your own routine to decode the data on the project file.
For AVIs, you can use Directshow calls to get the information.
For reading IFOs, that's another story :lol:
Read here: http://dvd.sourceforge.net/

-kwag

@Pro64
And bear in mind that DGIndex is not fully DVD2AVI compliant.
That means that by looking at DVD2AVI sources you might get your tool to properly parse a DVD2AVI-generated-D2V file but it won't work properly for a DGIndex-generated-D2V file.
In such case, probably best to look at DGIndex source code too...
Cheers

Thanks all.

@Kwag: Why you don't put things a little bit more easy for me, you already read that sources, isn't it?
Also I have not idea where to find out those Directshow calls.

Thanks.

Yes, but I don't think the way you do :lol:
What I mean is, the way I code. My algorithms. They will be different to other programmers. Also depends on what language you program.
For example, I program in C for a living (for way too long!) and also in Python (for the last 7 years), and I also program in Purebasic, just for fun.
Purebasic is not "Basic", so my code will not work in your application, if you are using VB. Are you using VB :?:
So you really need to do some pseudo-code, after reading the sources, and apply them to your programming language.Then you need to learn "Windows Programming", because Directshow (DirectX) is build into every Windows OS.
Start by reading here: http://msdn.microsoft.com/library/de...ch_directx.asp

-kwag