What's next for kvcd as a format?
 

lets see u managed 180 mins on 1 80 min cd and 360mins on 1 cd whats next for kvcd?? are there any plans in the pipe work??

I don't think that the further development of KVCD will aim for more compressibility.With 360 Minutes on 1 CD,a point is reached where another direction should be taken.The Improvement of the current Quality of the KVCDx3 Template should be the aim for the Future.The KVCDx3 Template has great Potential to reach DVD.And also the KDVD Project deserves some attention,because DVD is the Future Media.

It now really depends more on future encoder generations, taking advantage of our Q. matrix, and all the wonderfull AviSynth filters.
The quality on (SK)VCDs I think is now maxed out. All we've done can be put to great use on other media/transports, as we are currently able to fit ~6 hours full DVD quality on DVD media, and any broadcast station using DVB MPEG-2 technology can take to their advantage the matrix on their hardware encoders to use less bandwidth and maintain the same quality with lower bit rates than the current parameters being used. So the future looks bright :wink:

-kwag

so basically kvcd is at its limits and couldn't get any sweeter :D??

I don't think it can get any better now, so let's have fun encoding :mrgreen:

Could the matrix be better somehow or is that it, how did you came up with that?

Yes, but it would probably take thousands of encodes to find an optimal value, and if probably wouldn't be worth the quality difference. It's already extremely optimized. A lot of work :wink:.

-kwag

Hi Kwag and others..

I think that, although the "size" issue has ben addressed to the max (or until
a new technique could be found through pure luck) there is still hope for
the "quality" side.

Improving "quality" through use of Filter Techniques..
If we consentrate on the Filter(s) (and their respective values) we could
futher enhance the (take advantage of) kvcd.xx.x templates for Each format,
KVCD, KSVCD, KDVD etc. Each one could have it's own Profile
version, tweaked according to Source condition and type.

* A set of "user" Profiles for certain Sources ie, VHS; DVD; DVB; or Captures
..etc. based on "tried-n-true" methods could be associated.

* A new naming convention for each tempates..

* or, build a user-base of "filter-chains" and include best one suited for a given
..template.

* create more "specialized" filters for various source, DVD; VHS etc.

* develope a tool or tools to gauge better "filter" techniques

* work more w/ perfecting CQ ratio settings

* create "OUR OWN" frameserver client !! and the possibilities are endless..

And, lets not forget the encoder project.

I think that we should all starting working together and consentrate on the
improvements of "quality" through the use of filter techniques
(aka, filter chaining)

So you see, there is still lots more fun and exciting work to be done.. lots!!

Have a good day everyone.
-vhelp

Hi Kwag and others..

I missed this..

CheronAph wrote:
>> Could the matrix be better somehow or is that it, how did you came up with that?

>> >> Kwag wrote:
>> >> >> Yes, but it would probably take thousands of encodes to find an optimal value,
>> >> >> and if probably wouldn't be worth the quality difference. It's already extremely
>> >> >> optimized.

Actually, what you'll need is a TOOL to aid in the process, which CAN be done but
that will require a little enginuity.

I too, think that the Matrix COULD be further worked on. But don't forget, that
a revised Matrix table would more than likely require a revised CQ ratio setting..
at least in my experience, it did, but it will depend on the Matrix table, AND, as
a result, required LOTS more work he he..
But, if you do run such tweaks, best to keep a table of results ie, use Excel, as
I did !! :wink: !!

Kwag..
just curious, but in your Matrix tweaking, did you take into account for other
source, ie, Capture AVI (not DVD) ..as I found that in my experience w/ Matrix, that
when I ran test encodes (based on the revised Matrix table) when run w/ an DVD
encode and an AVI captured encode, that the quality WAS in fact different. Some
time mid last year, I ran a battery of Matrix tweaks, and concluded with what I
felt that were TWO of my best Matrix tweaks, and when I encode an Captured AVI
into each, and a DVD in each, I found that one was either blockier than the
other, or one was smoother and grainier (the good grain) than the other.
Some of these LONG and tiring Matrix tweaks were actually memorable. I was up
till 4 A.M encoding various sources. Hmmm, I was thinking about doing some more
work on THAT Matrix I last left off on.. I managed to create even smaller encode
than your KVCD (at least at the time) but I stopped that project due to
other issues going on at that time. I still have it, but it needs work. So in
answering to anydbno1 and other's.., yes, I guess size could be worked on, but
who knows. It would be interesting to do some test comparison though. I miss
working on THAT Matrix. Anyways.. If anything Kwag, Caputred AVI's should
(IMO) have it's own tweaked Matrix.

Have you thougt about that ??

As for CQ..
CQ is (IMO) the best encode mode out their.. fast and accurate (if prop. tuned)
when used under TMPG. Don't mistake it to be best w/ any other encoder ie, MainConcept etc.

-vhelp

I think What we need is and faster filters and also faster Encoders, TMPGEnc is good but its engine feels pretty slow for me, I get desperate having to wait 4-8 hrs for a good encode. :x............ :D

yes friend, i'm with you! :)

:ideasmiley:

i have to buy another pc..one to encode,another to forum....

only one problem:
i don't have money!
:bawl:

How about talking to "the powers that be" and make kvcd the industry standard :)

but certainly should talk to a company and develop a line of standalone player that's compatible with all the kvcd/kdvd template :D , especially now that you think development of kvcd is at final stage.

I would defintely buy that if it's not too expansive and maybe has the added bonus of being multi regional? I'd love that feature.

Just throwing some idea around :D

Talking about "size being important" has always made me feel nervous for some unknown reason. I definitely would like to see more optimised ways of doing good quality VHS captures.

Cheers
Jeff

Also, recommended filters for the types of movies or source would be useful. For instance, a mostly bright daytime action movie (Lone Ranger), an animation with video noise from the capture (Simpsons), and a darky shot drama (Road to Perdition) can all be optimized with different filters, but the best choice is usually only hit upon after either experience or LOTS of forum reading. An online database (maybe even a wizard) of types of sources (or actual movie/show titles) along with the type of signal source and quality, listing what people have found to work well, might be a fun way to share experiences, or solve problesm by searching for others results...

4-8 hours?? 8O 8O
My encodes take 20 hours at least!!! :( :cry:

8O mine take up to 12 hours.

Try defragging your hardrive before encoding :lol: maybe that will help......... :P

i think the main reason is my slow computer (667) and the use of filters.

That's definitely it 8O

-kwag

yeah. i bought it about 3-4 years ago. i haven't got the money to really upgrade yet. :cry:

I uderstand, I my self would like to move up from my 1-1/2 Gig Athlon xp to a danddy 3 Gigabyte Pentium 4, I know TMPGEnc will benefit from it and if a kvcdx3 2Hrs. encode takes me 7-8 hrs. , with the Pentium it should be half that long to encode the same...... :idea: :wink:

I'd really like to see a renewed focus on compatability. The single biggest complaint I hear about this format is usually "the video looks awesome, but the sound doesn't match up right!" On my player, the KVCD standard works great, but on my friends' players it's a crapshoot.

Maybe a little more research on the multiplexing side would knock that out...

Personally, I think that a variable CQ/bitrate builder would move to "the next level".

Creating a tool to annalize the video sequence, break it up into different sections to encode a different max bit rates with different CQ levels and them re-join them at the end. Perhaps I am talking about an better encoder that does a much better job at CQ than the existing encoders.

For example, the "new" encoder could respond to Avisynth filter settings. Filters could be created to change the encoder settings dynamically (by scenes or video segments) or to base dynamic CQ on the presence of mosquitoes or macroblocks. This would give us amazing control over the encoding process with the ability to put our bits where we want them.

Hi MrTibs,

What you are describing, would be to use the force picture type setting in TMPEG. A couple of months ago, I had an idea of analyzing the source to be encoded, similar to what bit rate viewer does, and then applying a bitrate ratio (compressibility of source) to the picture type settings in TMPEG. Basically I thought of making a program that would actually create a text file in TMPEG's frame number list (*.txt) that would be loaded in TMPEG. But because there's no constant relationship between the source compression and the compression that TMPEG would use on certain film, I discarded the idea. It was a good dream :roll:

-kwag

@Kwag

Thanks for the reply but I have some questions.

What if I modified the Blockbuster filter to spit out the txt file you suggested? I'm sure that you understand the issues better but if BlockBuster deteciton is able to determine areas that are hard to encode, (macroblocks) perhaps I could use that engine to control the GOP structure to be produced. This method may be rougher than your idea but perhaps would go a long way to controling compressability. As I posted in another area, I have the greatest challenges with dimly lit scenes. If blockbuster can be used to add noise, perhaps it could be used to detect problem frames.

What do you think? (I'll hack away at Blockbuster as a test it you give me an idea on how the GOP should change for better quality in dark scenes.)

Hi MrTibs,

The idea is great, but still, the problem is that there's no relationship between the source compression and the compression that TMPEG will be able to compress that source :!: I hope you understand what I mean. You could hack Blockbuster's algorithm (or use it) to determine what frames need more/less bit rate, but it wouldn't be a 1:1 relation to what TMPEG is going to do. However, there is something that we could "possibly" try! If we know in advance the compression of the source, or better yet, we dynamically create a table of high/low bit rates of the source, we could then map this table to a corresponding bit rate table for I, B and P frames, and try to average the "wanted" average bit rate as given by moviestacker :?: This would basically be a similar thing as a 2-pass VBR, but we would be making a ultra high first pass. Like bit rate viewer, that can read the complete source in about a minute or so. So after we parse the source, and create a bit rate map, then we know how to export that data into TMPEG's force picture type .txt format! How does that sound? I'm all for it. Maybe just a little standalone program that reads the source file, a la bit rate viewer, and creates the .txt file. Then it's just a matter of loading the .avs into TMPEG as usual, and then going into the GOP settings screen in TMPEG and loading the exported .txt file created with this utility. No more file prediction :), because the .txt file will already have the proper average sum of the complete processed file, so we know the target file size will be bullseye ;)
I'm all for it. Just need to figure out the fastest way to read a source material on a frame by frame basis, and create a bit rate map "on-the-fly" as the files is processed (just like bitrate viewer). I'll give you an example: Say we are going to encode with a MIN of 300, MAX of 2,500. After we parse the complete material, say a DVD (via an .avs), we normalize the bit rates. If the MIN bit rate of the material is 4,000Kbps and the MAX is 8,000Kbps, we'll "map" that range to 300 and 2,500. So because we know the bit rate for every frame, we can interpolate the 4,000-8,000 to 300-2,500 and maintain the average bit rate wanted by MovieStacker (or any VBR calculator) How does that sound?? Any ideas??

-kwag

I think you've got something like that in some 2-pass DiVX encoding tools, after you run the first pass you end with a "log" that you can use to manually fine tune the bit rate allocation.... I think it's the "advanced" binary of xVid... I'll try to find out exactly what I'm talking about :) We could use the 1st pass divx log as a guide to the bit allocation in our kvcd encodes.

Hi GFR,

If the data in that log is detailed for every frame in the source, then it's a piece of cake to write a small utility program to import it, analyze/normalize it, and export it in TMPEG's format :mrgreen:

-kwag

@ KWAG !
What about LOG file that TMPGenc produces during encoding , it have all needed info : bitrates,compressibility,avg. bitrate !
Maybe we can first run at low res as 352x240 (or even lower ) just to get LOG file fast as possible and after that run real encoding with higher res and manually adjust all wanted values ??? :?:
It's possible :?: :?: :?:
bman

Yes it does, but we can't use that, because the information it contains is the output of TMPEG. We need the input information from the source :)

-kwag

It appears that we do know where KVCD will go next...

OK this is where I embarrass myself. In earlier discussions, Kwag suggested we didn't know the compressability of the source frames. In order to find the compressability of the frames will we need to run each frame through a DCT-Q-IDCT filter, then make a bitrate log for each frame?

Clearly, I don't understand the relationship between the Q value and the GOP structure in order to maintain a certain CQ. Perhaps one of you out there could sketch out the process on how the Q values and GOP structures are adjusted when the encoders are compressing the source.

Please forgive me for my ignorance, I understand the theory of MPEG1 but the CQ/VBR process is still mystery.

Hi MrTibs,

Actually we can tell the compressibility of the source. The problem is that we can't tell the compressibility of the mpeg that TMPEG will create :!:
That is why I suggested a "scan" of bitrate per frame on the source, and then scale that to MIN/MAX/AVG and feed TMPEG the data directly as picture type. This way, TMPEG will encode each frame at the specified bit rate, which it does provide for that on the advanced/force picture type settings. I'm currently hacking into DVD2AVI source, as I believe it's the ideal program to genete the raw frame/bit rate data. This way, after we finish saving DVD2AVI's project file, we'll have the .d2v, the AC3 or MP2, and a "someName.txt" file that we can process with a "soon-to-be-made" program that will analyze the data, normalize it, guarantee that the average bit rate will be the one we tell it to be, and generate a TMPEG .txt file ready to be processed. I already have the frame numbers identified in DVD2AVI and I'm looking at the source to find out where/if the bit rate information is available on a per frame basis. If anyone ( sh0dan, canman, sansgrip ) or any developer here that has worked a lot with mpeg2dec sources ( dvd2avi sources ), I'd appreciate if they help me identify the module, or the proper call to identify bit rate information on current processed frames :!:

-kwag

What I was talking about was manual "curve compression" in XVID, maybe we can use a similar strategy in MPEG encoding???

http://www.animemusicvideos.org/guides/avtech/xvid.html

One option would be reading the bitrate/Q/etc. from original .vob files. The only problem is that filters change the compressibility. But maybe it could be predicted what the filters do for compressibility. Or are you already meaning this?

Yes, that's exactly what I mean :wink:
Then scaling the VOB bitrate to the 300-2,300 range we use on KVCDs, and normalizing the data so the total average will be the wanted average.
When this is feeded to TMPEG, it will encode with strict parameters per frame, using the data just like if it had done a 2-pass.

-kwag

Hey Kwag, how's your project going?

I realized after the other posts that you and I were talking about different things. I am mostly working with uncompresses Huffy sources so some kind of first pass scan is required.

To test the idea (very roughly) I did an encode with CQ@100 then re-encoded in TMPGEnc - 2 pass using the values I got out of Bitrate viewer. The result was a smaller file size and better quality than encodes at lower CQ levels but still not as good at the CQ@100. Or course, brightly lit frames looked excelent while dimly lit frames still posed a problem. In fact, I've noticed that even with CQ@100, dimly lit frames don't look all that great. I'm still doing research but I suspect that my resolution of 352x240 is the biggest factor.

This brings me to my question: Has anyone found that different Q Matrix(s) produce better results (not considering compressability) for different kinds of scenes? (i.e. action, dimly lit, bright, static, skin tones) I'm wondering about using multiple Q matrix in a single movie.

Hi MrTibs,

The possibility of changing the matrix at GOP levels is possible with TMPEG, also selectable on the force picture type screen. I have not had more time to play with the DVD2AVI source code, and I didn't find where to pick out the bit rate at the frame level. I already have hooks to write the information to a "stats" file, but I can't find the functions to report the bit rate of each frame. I need help from someone who has worked more with these sources, because I haven't! And It's been 3 days now that I had to put that away, because of work :cry:
Hopefullt this weekend I can run Visual Studio and give it whirl again :)

-kwag

First off can I say the homepage has improved greatly since I was last here!!

Next up I know kwags going to get wary wary angry when I start to mention CCE as he's actually got a wee personal vendandatta(how close was that spelling :lol: ) going against it.

But really with all these filters and all. I have managed to mees about with the CCE matrix and the gop structure a bit. I don't understand mpeg matrices so I can't play about with it until I find a good'un. However I can increase the gop structure which has been intresting....

Also if your on a slow computer don't complian, until recently I was on a 450mhz!!

Baker

20 hours?? 8O what CPU do you have? mine take about 8 hours maximum...

And a good morning to you all :lol:

.
.
Who's still doing 20 hours ??

..less you got some NR filtering going, I can't understand this 8O
.
.
on another note.. whats next for kvcd?? ..

for me, I'm working w/ my Canon ZR-10 dv cam to see how I can get
as much quality from an encode, with all it's noise that DV has when you
shoot footage in low light. I did a few, while de-Interlacing (working some
more on de-Interlacing DV) Also, I'm using the kdvd template on
my DV source. Now, encoding "low light" source materials is proving to
be a pain. You can get by w/ 352x480 but I think that either the Q
will have to be raised, or the bitrate. I'm working on this now. Everyone
knows that the ZR-10 is not the best under low light condtions, but when
this is all you got, it's better than nothing 8)

Filtering a DV source is my last resort, and so far, I haven't had to, but I
might try pixelDut() out today.

Kwag, curious.. do you have a DV cam yet ??
Can't say I remember you talking on this. But, its fun working with it. Now,
depending on my footage and light source, it can sometimes remind me of DVD
quality (course, I'm awake)

Anyone here using kvcd on their DV footage ?? I'm using kdvd for
obvious reasons.

I'll post whatever clips I can of my progress when I get the chance though.
SAMPLE Encodes.. KDVD; KVCD; CVD; SVCD; VCD and "x"
..case anybodys' interested.

So, I can see whats next for kvcd?? perhaps being adjusted or something
with DV sources to make

Everyone have a good day :lol: :lol:
-vhelp