KDVD and M2VRequantizer
 

I saw a topic about using DVD Shrink before to get more space out of a KDVD. I as wondering if anyone wanted to take a challenge at an idea. I don't know if this is possible but I think it would be cool if someome used the M2VRequantizer and made KDVD optimizations to it. Read about M2VRequantizer at http://forum.doom9.org/showthread.php?threadid=62849. And if someone caould also make a gui like rejig, then we would have a quick and easy way of making KDVD or KVCD files.

Beautiful :mrgreen:

So just do: Final_Size = Encoded_.m2v_size / Wanted_Target_Size

So if you encoded your file, and the size was over your wanted size , just use the formula above and transcode :!:
I wish it worked with .m1v files :(
But for MPEG-2 files, this is the end of file size prediction :mrgreen:
Just use CQMatic, and use a higher ( ~10% or so average bitrate ), and after you encode, run your .m2v through "Requant" with the formula above, to resize your file to a perfect file size :ole:

-kwag

Oh man, this is GOOD :mrgreen:
One CQ prediction with CQMatic, followed by one "Requant" cycle for a perfect fit :rotf:
:drink:

Thanks Too Tall :D
-kwag

Thanks TooTall,
that's really a great tool 8)
I'm currently looking at the source code. Maybe we can tweak it with the Notch Matrix 8O

God this is awesome!! How come there's no such tool for MPEG-1 yet?

I think it could be modified for MPEG-1, but it will be quite a task. :D
I just made a test on one of my KDVD encodes, and I reduced the size by half, and it still looks very good 8O
From ~1.6GB to 800MB.
However, I notice that on scene changes and high action, there are some visible macroblocks ( which I did expect to see )
Anyway, this program will definitely kill the over sized problems of CQ with all KVCD and KDVD encodes for MPEG-2.
It's a piece of cake to simply use CQMatic, and if it's over the size, do the simple math and "Requant" :cool:
This way, if I'm over, say 50MB, it will be a very small requantization needed, and the transcoding speed will be very very fast.
Maybe I'll add it to CQMatic, so that if encoded file is over the size by some pre-determined %, then automatically execute "Requant" after TMPEG's encode, and tailor the final mpeg size to exactly the wanted size :mrgreen:

-kwag

Well let's get to work than! My programming skills will not be sufficient to modify the program to work with MPEG-1 files...
Who can? :roll:

:?: Since I do MPEG2 only and would like to try this, could someone point me to an executable version of M2VRequantizer (I'm not into compiling)?

I use WindowsXP.

The link at the top of the thread includes the windows binary:
http://home.t-online.de/home/340044300675/requant.zip

So just do: Final_Size = Encoded_.m2v_size / Wanted_Target_Size

I've downloaded requant.zip. I assume it's a dos line command tool. The only instruction I can find is: "requant 2.0 input.m2v output.m2v".

Exactly how do I use this?

NickSteel

@nicksteel:

follow the doom9 thread for this. Nic from the doom9 has written a gui called rejig for this and can take vobs, m2v as input, and you can set the output ratio (in percent)

cheers,

Thanks, rhino
 

Found it, downloaded it and have started testing it. The doom9 thread is interesting, as I use KDVD, but have confined my efforts to the main feature itself. This looks like a practical way to keep the menus, etc, after KDVD encoding for m2v's.

Well, just finished doing a quick 50% recompression test from VOBs of the movie "Red Planet" ( which I have already done KDVDs with TMPEG many times, and looks just awesome even at 704x480 on one CD ), and the result or Requant is not very good :x
On stills and medium action, Requant works ok. But on action scenes, it sucks like a tornado :!:
At least when transcoding VOBs on a 1:1 ( 720x480 size )

This is what an action scene from the DVD of Red Planet looks like, transcoded to 50%. Transcoded .m2v size is 1,664,763KB:
http://www.digitalfaq.com/archives/i.../2003/10/1.png

And here's the original VOB:
http://www.digitalfaq.com/archives/i.../2003/01/3.png

Quite a long way from DVD2ONE, DVD Shrink, etc :roll:

Edit: Here's the same screenshot from my KDVD encode (16:9 non-anamorphic), which has a total .m2v size of 1,018,787KB ( video stream only ) WITHOUT any filters. Just KVCD's "Notch" matrix.
http://www.digitalfaq.com/archives/i.../2003/10/2.png


-kwag

Doom9's forum is where I found out about it. I just thought that if it could be optimized with the KDVD specs that it may be useful to backup some dvd9 people own. I backup everything I have onto DVD's (data/music/video) and use the backup to prevent the original getting scratched. If M2VRequantizer could be optimized with the Notch Matrix, then the video file could be remuxed in ifoedit to make a perfect quality backup. Basically transcoding with the Notch Matrix. This is what I am hoping for.

Hi TooTall,

Right now, Requant is very usefull, if it's to be used with smaller resolutions/compression ratios.
I find that there's nothing better that a full re-encode, instead of a transcode.
I don't think a transcode can be done with another matrix which was not in the original encode. I talked to DVD Shrink ( the developer) a long time about it, and he told me he was going to do some tests, but I never heard from him again. After reading some MPEG protocol stuff, I came to the conclusion that it's not possible ( I hope I'm wrong :!: )
The same goes for transcoding to a lower resolution, which I believe there are some talks on that doom9 thread.
The best thing to do is to do a full encode at the target resolution you want, and then tailor (tweak) the the final file size with Requant. As long as the % is not too great, then this tool is very usefull. But I wouldn't use it to transcode a DVD down to not even 40% lower compression.

-kwag

Kwag,
 

My interest is confined to under 5% compression of KDVD files. Often I miss the final file size by 2 or 3% and have to reencode. This is especially a problem with miniDVD's encoded to 2 CD's at 704x480.

:?: Do you see this as a viable tool under these circumstances?

:?: Could you try the same Red Planet scene at 5% compression for comparison?

If so, it would be a valuable addition to CQMatic as a final routine for MPEG2 encodes.

I agree completely on the quality issue with large compression factors. In fact, I find KDVD with the MA script so superior to DVD2ONE or DVDShrink that I don't use them. It does seem strange at times that my KDVD will not come even close to filling a DVD at 80+ CQ, while being much cleaner than a DVD2ONE or DVDShrink encode filling an entire DVD.

NickSteel

I think what needs to be tested is the following:
What gives better quality: a 5% bigger file transcoded to perfectly fit or a 3% undersized file? I think some serious testing needs to be done on this matter, who cares if the size is perfect if the quality is worse? :?

[]'s
VMesquita

Re: Kwag,
 

Me too :) Exactly :!: Of course :D
I would say that even a ~10% to 15% decrease file size with this tool is still undistinguisheable from the original :cool: Will do it right now :!:
I'll post results in some minutes. Already in the "Todo" list ;) Ditto ;) Same here :D

-kwag

Here's the result, compressed 5% ( 95% from ReJig's menu, the GUI to Requant )

http://www.digitalfaq.com/archives/i.../2003/10/3.png

So I assume that I'm correct, that we could go to 10% to 15%, without visual degradation :cool:

-kwag

I'm going to try a complete dvd with KDVD and the menu methods on the doom9 site. With KDVD compression and this new tweaking program, it should be easy to size the different parts and end up with something superior to DVD2ONE and DVDShrink for a dvd with features.

@ Kwag
Sounds like sweet program and it makes me to wonder if is it possible to make MultyPass transcoding with it :wink: :?
I'll try to explane !
Once in not so far past U ( KWAG ) had idea to get bitrates of all movie frames and according them to make new encoding ( or something like this !)
So I'm wondering if we can make first transcode with let's say just 5% factor and simultaniously create LOG file with quantization factors of all frames( on each pass ) .
Then on second Pass during transcoding , program will check values of quant. factors for each frame from LOG file and if quant. factor is still far from optimal he'll transcode with 5% factor .
If value is close to optimal quant. factor ( that we have to find but according VMesquita it have to be close to 30 ) he will skip transcoding of the frame and go to next one - Just to get all quant. factors near optimal factor -To Get best quality especially for high action scenes.
At the and of cycle check file size and if file is still too big than make one more pass and so on.
With little transcoding steps precission will be better .
So meybe transcoding will take 5-6 passes to get wanted file size with accurate quality controll and it will take 10-15 min for each pass - something like 1-1.5 hour to get final transcoded movie .
As we spend about 10-20 minutes for prediction and 4-6 Hr for encoding
1-1.5Hr must be not so bad compromise ( If It's possible to be done ) .
If I'm wrong please correct me :wink: :wink: :wink:
bman

Hi bman,

I think our best quality will be obtained with CQ, as usual.
With TMPEG, CQ is far superior than 2-pass. So it will still be better to make multiple CQMatic runs, and only if file size is above the wanted target, then run Requant to close the gap. CQMatic usually hits 2% to 3% of target most of the times. So even if we run multi passes with CQMatic to find CQ, it's still faster than any X-Pass, with the addition of one very fast last pass with Requant.
The process of [X-CQMatic-Passes->One_Full_Encode->One_Requant_Pass], is still faster than a single 2-pass TMPEG encode, and will give superior quality ( as long as the % of Requant is low, which should be ).
I have my hands tied up this week. Probably next week I'll integrate Requant into CQMatic, and all of this will be automatic ;)

-kwag

After much experimentation, the rule of thumb I've developed is that if it will fit with less than 20% compression I'll do it with a transcoder otherwise I will reencode it. IMHO 20% compression is slightly lower quality than the original, 15% and above are, as you say, virtually indistinguishable.

FWIW DVDShrink 3.0b5 in "deep" mode is the best quality transcoder I've found. That might change when Nic invents 2-pass mode for requant...

So, you say DVDShrink 3b5 is BETTER than Nic's ReJig?

Does DVDShrink actually work with MPEG-1 Streams too ? 8O

In "deep" mode yeah, I think so, but its much slower. In one pass mode I think ReJig is better quality so Im eagerly waiting for 2-pass ReJig to be created as I think that will fix the quality problems.

No.

DVDShrink only accepts VIDEO_TS folder structures as input. If you can ever even figure how to get mpeg1 into one of those, it probably still wont work ;)

@girv
Actually, if you mux MPEG-1 video as a "DVD non-standard" (or standard, if it's 352x240/288), it should work... You can try muxing using IFOEdit or DVDAuthor (the unix one) and then run throught shrink

@all
I guess nobody payed attention to my post... And I can't find time right now to do the tests. But I still stand on my point: maybe a 5% oversized file transcoded to perfect fit will have the same or worse quality than a 3% undersized file, and this definatelly needs testing. Slightly more DCT data doesn't necessarily means more quality, if the allocation is not optimal.

[]'s
VMesquita

Requant Motion Test - Kwag.............
 

There is a scene in a Futurama that I use for testing that is a takeoff of twilight zone and has a digital clock moving on a black space background. It is very sensitive and, if not near perfect, has small motion jerks.

It is KSVCD at 352x480, interlaced 29.97 (with Incredible's script) and GOP 15. CQ 63.7, minbitrate 766.

I tested Requant at 95, 90, 85, 82.5 and 80%.

Everything stayed smooth thru 82.5%, then started degrading.

I intend to make further tests with Reloaded, but so far, I feel it would be a good addition to CQMatic.

What is the most accurate way to produce a 10% over sized MPEG2 with CQMatic?

Try something like the following:

Encode with CQ=70, test1.m2v
Encode again with CQ=65, test2.m2v
(You may use different CQs, that's just an example).

Now shrink test1.m2v to fit the some size in between test2.m2v and test1.m2v. test2(shrunk).m2v will be smaller than test1.m2v bit bigger than test2.m2v.

Then you compare test1(shrunk).m2v with test2.m2v to determine if the smaller, but CQ only file looks better than the bigger but shrunk after CQ file.

Ok, I did the tests using CCE:
Created a 1 minute clip from "the lawnmover man"
Encoded with CCE at Q40: size: 19.077kb
Encoded with CCE at Q30: size: 21.884kb -> transcoded with ReJig to 92%, got a result 10% small: 19.813 kb.

Please remember that CCE has a different scale: more Q, less quality.

So if 19.813 were the ideal size, my Q30transcoded sample would be perfect fit and my Q40 sample would be 3% smaller than ideal size.

The results: slow motion scenes, the look about the same. In high motion scenes, the non-trascoded version wins, no doubt about it. There are much more macroblocks in the transcoded version. So a 3% undersized file is better than one transcode to 90% to perfectly fit.

If you want to see for yourself, I posted 4 samples in BMP, to avoid losses in JPEG that would prejudice the test. Remember to zoom them, or they will look about the same to you. Get the samples here:

http://www.jltoca.uaivip.com.br/files/snap_samples.zip

[]'s
VMesquita

"Suspeitei desde o princípio!" :)

:cid: :cid: :cid: :cid:
:lol: :lol: :lol: :lol: :lol: :lol:

(This quote he wrote is from a mexican TV program "Chapolin", that is shown here in Brazil)

[]'s
VMesquita

:rotf:

Yes, you can do that using for example TmpgEnc DVD Author including pre- and post-patching the header to a legal DVD size using DVD patcher.
If you know what to do its fast cause there have to be muxed as VOB only ca. 800MBs!

And by using the latest ShrinkDVD Version its even more precise cause of 1percent shrinking steps in the "shrink" bar.

But I think you know how to do that as it was explained also in ther german forum.

"No contaban con mi astuzia" :mrgreen:
Yep, "El Chapulin Colorado", played here in P.R. too :lol:

-kwag

That's another quote
"Não contavam com a minha astúcia."

My quote was
¡Lo sospeche desde un principio!

I also like some other gems like:
"Não priem cânico!" (Calma, calma que no panda el cúnico)
and
"Foi sem querer querendo." (Fue sin querer queriendo)


In Spanish:

Chapulin:
¡No contaban con mi astucia!
¡Síganme los buenos!
¡Todos mis movimientos están fríamente calculados!
Yo opino...
Ya lo dice el viejo y conocido refrán...
¡Es exactamente lo que iba yo a decir!
Calma, calma que no panda el cúnico
¡Lo sospeche desde un principio!
¡Se aprovechan de mi nobleza!
Mis antenitas de vinil están detectando la presencia del enemigo

Chavo:
¡Bueno pero no se enoje!
¡Eso, eso, eso, eso...!
¡Se me chispoteo!
"Ahora si te toco el ocho"
¡Es que no me tienen paciencia!
Fue sin querer queriendo
¡Zas, zas, que yo jugaba...!

http://www.digitalfaq.com/archives/error.gif

http://www.chavodel8.com/

:hihi:

BTW, the rejig utility has been updated with a more accurate file size output. Can also read ifos, etc. Updates are coming quite regularly and is shaping up to be a nice utility.

Cheers,