I'm the one still doing 20 hours... :(

Don't see what it could be....encoding to 7200rpm Hard Drive, with 512MBs SDRAM and a 1GHz Athlon Thunderbird. I normally encode to 480x480...using this kind of script:Defragmented a few days ago and it didn't seem to make any difference...what could it be? :(

hi PyRo..

I wish I could help ya, but I'm still using AVIsynth v1.0 beta 5, and most
of those filters mergechroma(blur( 1.58 )) and mergeluma(blur( 0.3 )) for ie,
don't work, not to mention the others that require v2.x and higher.
I'd give that script a test, but I can't :cry:

Well, I don't use that many filters anyways, and I guess that one of my
delimas in making the switch. Besides, I had so many issues w/ color in
my privious encodes of yesterdays, (the past) that I'm just fighting the switch.

Defraging your hd won't do much for your processing time, but it can't hurt.

I have an idea though.. :idea:

How about running a few tests.. for instance, just take ONE item out, and
run an encode. But, don't "go all the way" he he.. actually, just wait to see
TMPG give you an estimation. Then, do the same with each, and also do
a few combinations of filters and see what TMPG's estimation is. Once you
have found a short time estimate, you're on your way to finding the cause
of your filter that is causing your "slow up". If I had only had the exper.
w/ those filters, I'd of probably guesed what was causing the slowness :cry:

Good luck anyways.
-vhelp

Nope. My camera is a ~4 year old Sony Hi-8 :?

-kwag

hi Kwag,

ok.. just thought that you had one.. being that you love so much being
on top of latest in just aobut everything :wink:

Well, I'm waiting for my Osprey-210 analog capture card - - ta add to my
list of video devices. I heard a few GOOD things about it, and want to
give it a try. Don't get me wrong. I LOVE my ADVC-100 but lets face it,
and don't forget.. that its video source is "compressed" hence DV. Anyways..
And, there IS a difference between DV vs. Analog (uncompressed) but not
by much. But I believe it is depend (quantity'wise) per Capture card and
Maker - we'll see in my next testings of my Osprey-210. I hope it proves
positive :wink:

I'm always looking for the best Analog Capture card out there, and I think
the Osprey-210 is it (so far) after shopping around on-line, I was able to
locate a fair price @ $165 (retail $199) - - should arrive around Monday
or Tuesday - can't wait :D :D

-vhelp

.
And, here is a comment snipped on it's notary from:
ducas NEWS..

.
.
http://www.digitalfaq.com/archives/error.gif
just realized that it may have ben modified, being that there's TWO pics.
FROM: RealNetworks - read the specs there too.
.
or
.
http://www.digitalfaq.com/archives/error.gif
And, the on-line store, where I ordered it from: DV Direct

and some specs from www.ht.com

-vhelp

What chipset is in that card :?: That's the most important thing :!:
Is the ADC 8, 9 or 10 bit :?:

-kwag

Hay Kwag..

I just edited the previous page. Apparently, there may have ben a revision made
to the 210 model. The first pic looks like my ATIW cards' breakout connection
(the one w/ the circle at top) so do have a look - at least out of curiosity.

Kwag Wrote:
I'll let you know when I get my hands on the card itself !! But, w/ my luck,
it's probably 8 bits 8O

* Any chance you might know what our WinTV-GO is rate at ??
* And, how about the ATI-TV Wonder pci card ??
* Oh, and the DC10+ ??

Thanks for bringing this up though. I didn't realise how important this is to an
Analog Capture device. I'm sure the higher the bits, the better!

I just did a search on vcdhelp, but they don't have a review. Guess no one has it
there yet or they just didn't post a review for it (probably bad) Maybe I'll post
first he he..

In any event, I'll give you a heads up :wink:
-vhelp

Not sure about the DC10+, but the WinTV and the ATI are 8 bits for sure :x You bet!. There was a review of a card, not so long ago at vcdhelp, about a cheap card that is 10 bits. I can't recall the name, but 10 bits is a huge improvement over any 8 bit capture card. You could also go and grab this: http://www.avland.co.uk/panasonic/dm...d-recorder.htm and then I guarantee you will never look back at any capture card :mrgreen:

-kwag

hay Kwag..

..back again, after a tiring search.

ok, here is what I found on this card, though it's only hear-say from another
forum.. basically, it's basd on a BT878 chipset, but again, this is not
conclusive, rather someone's assumption. But, I'm tempted to believe the above
anyways though. It's looking more so, an 8bit card.

There is a cheap 10 bit card out there, that I'm researching on.
I think PixelView is using the chipset in one.

I started a thread on the Osprey-210 card, for discussions, so not to taint
THIS thread's original topic. You can find the link below:
--> Capture card: Osprey-210.. my experience..

Kwag.. Thanks for the heads-up on the 10 bit.
-vhelp

@Kwag,

Kwag wrote:
Interesting program. I used divx4log, which did something similar for
Divx 2-pass. You run VirtualDub using 2-pass and after the first pass
run divx4log to normalize the bitrate allocated to each frame. Then
substitute the log created by div4log to do the second pass. It forced
Divx to follow the bitrate normalized by divx4log. It didn't work well,
but only due to the developer's algo. His algo was suppose to improve
allocating bitrates to fast scenes and complicated (e.g. water, smoke)
scenes. :)

-black prince

Not at all! CCE is fine for high bit rates and MPEG-2, specially for DVD encodings >5Mbps. But for low bitrates, TMPEG just wins hands down every time :)


-kwag

But with divx4log you had to wait for one full pass right? Which is what I'm trying to substitute for a one pass "Fast Scan", because it's just reading the stream, and not encoding to a temporary "one pass" file for later analysis and encoding of the second pass!.
I've chosen DVD2AVI for this purpose, as we all use DVD2AVI to process our files. So I've integrated this functions transparently in DVD2AVI, so that there is no user interaction. This is how it works: You run DVD2AVI as usual. When you save your project file, say you saved to a name like "movie.d2v". This is an example of what you'll get:

The file movie.d2v.stats, right now just contains the frame number and the type of frame. Like this:

Etc, etc, etc.. :D
If I get this to work, and I find out how to get the bit rate for each B, P, and I frame, then the actual information will be something like this:

Which will contain the exact information for each frame, and this is the information that will be "massaged" with a small external program that will create the ascii file ready to be imported into TMPEG.

Here's the executable so you can try it. It's DVD2AVI 1.76 modified, now named DVD2AVI+Stats. Also I compiled it with full speed optimizations with VC++ Enterprice Edition, so it should be a little bit faster than the regular DVD2AVI 1.76 :wink:. http://www.kvcd.net/dvd2avis.exe
Try it :D

-kwag

@Kwag,

Kwag wrote:
This sounds like an interesting problem for SansGrip :wink: And BTW,
where is SansGrip these days. :?: If this could work, WOW!!! What
control you would have over Tmpgenc for bitrate and motion search.
Keep working on this Kwag, it has great possibilities :D

-black prince

hi Kwag..

wrote..
Some quesitons..
* But, how or where would you plug in those values in TMPG ??

* or, how would TMPG use those values, if it uses ANY form of numbers
.. for any special purpose ??

* I take it that the 1st digit (before the P/B's) are the actual FRAME no's ??

* And, would this (the numbers) be fed in like "frameserved" or something,
.. or would it just be you tipicle .TXT file of some sort ??

Yes, VERY http://www.digitalfaq.com/archives/error.gif interesting http://www.digitalfaq.com/archives/error.gif in deed

Thanks,
-vhelp

@vhelp,

vhelp wrote:
Tempgenc>Settings>GOP Structure>Force Picture Type>Settings>
Load>movie.txt (this contains frame number, p or b or I frame type and
hopefully in the future bitrate). An external program will need to be
developed to change the bitrate for each frame to force Tmpgenc
to do a better job with normailzation and motion search. 8) The
end result will be a smaller encode and an improvement in picture
quality :D Some avs filters won't be needed and that increases
the encoding speed. This could be a BIG step forward to improving
KVCD even more.

-black prince

Hi vhelp,

That code sniplet, 0,I,5203121, etc., would be the intermediate data spitted out by the modified version of DVD2AVI. This text file is then processed by another program that I still have to make, and that program would process that data and create a file in TMPEG's "Force Picture Type" format which is something like this:

Etc.

Go into TMPEG and under GOP tab, click on "Force Picture Type", and there you'll see what we're talking about :)

-kwag

hi Kwag.. black_prince.. and guys,

Thanks for response. Hay, something's wrong w/ your FORUM Kwag..
cause I didin't get any e-mail noticifcation on any responds after my last
one !! 8O

What's up w/ that 8O 8O 8O

Anyhows.. I think I know what you're talking about (got too envolved w/
WWF.. Trish got her ahzz whipped) but I'll have another look into TMPG's
GOP tab.

@Kwag.. I had time earlier to check out your revised DVD2AVI.
Worked like a char !! :lol: :lol: ..what calculating statistics from these
numbers. Are their any uses for them in addition to ??

* Does TMPG generate a silimar table of "values" during certain encodes ??
.. and/or that can be.. say.. "switched" w/ your NEW table of values he he..
.. "obviously, you're working on this, I realize, but.."

Here's what DVD2AVI+ outputed for an encodes (based on a captured source)
Thanks to good old Excel..
* I guess you'll have CBR and VBR values in this project ?

* also, are you taking into consideration that not all TMPG version are or
.. may not act alike in features. You know how THAT song goes.. w/ each
.. version tsumi (<-- spelling) makes.

-vhelp

I'm still not sure if I can set bit rate independently for each I, B and P frame. I might just write a dummy ascii file, and then load it with TMPEG's "Load" button under "Force Picture Type", and see if it follows the forced pattern I set in the ascii file. Have a lot to test :roll:

-kwag

Hey Kwag,

if you can tell me processing needs to be done with your stats file to create the appropriate tmpgenc file, i'll write a perl or php script to do this until an app is developed (i'm a *nix developer, don't have any coding packages for windows). People could either use it locally on a web server on their PC or it could be run off a few websites and people can load their stats log file into the webpage and it would spew out the appropriate tmpgenc file.

Would do in the meantime,

Cheers,

@Kwag,

Kwag wrote:
I tried this and Tmpgenc hung up my computer. My syntax for each
frame must be the problem. Next, I want to use auto-setting for a
.d2v input file and save this as text. If this works, I'm going to play
with bitrate for each frame and try to create a small .mvp file to
look at my results. I can control the frame range with source range.
If I'm lucky, I can pick scenes that could be tested (e.g. water, fire,
smoke, etc.) adjusted for bitrate. The magic is using a program to
do this auomatically :)

-black prince

Ok, now I have a hint on how to process and find the bit rate information ( Thanks Arno :wink: ), so I'm going to play with this later tonight :D

@rhino,

The file analysis would be very slow on anything but a compiled language ( I'm a U*IX programmer too ;) ), because it has to take the complete file that is processed with DVD2AVI+Stats and normalize the data before writing the new TMPEG compatible file. This is time consuming, and Perl, Python, etc., are not good candidates for that! ( I think, maybe not?). But hey, if this gets done, the beauty of it is that you or anyone can write different algos for that data, and we can all optimize the bit rate allocation, etc :)
Stay tunned!

-kwag

@kwag

Its a fair point - compiled is faster than scripting. I generated a 5Mb file of the format you are hoping to output from dvd2avi and to read this in and break it up into an array takes about 16 seconds on a 296Mhz sparc box. So good to prototype with then maybe someone could code the routines into a dll so ToK or MovieStacker could become a front end to it?

Cheers,

@kwag

can you make available your src of DVD2AVI? I hope to have VC++ in the next couple of days and plan to have a poke about dvd2avi.

cheers,

That's not bad at all!Right!, we could use any scripting language ( I personally would use Python or Ruby ) to play with algos and optimizations, but then for the actual program, I would just write it in C :)
So a standalone .exe and/or a .dll can be created. And great idea on the integration of that with ToK and/or MovieStacker ;)

Here's something you could try, and see how fast it works on a scripting language: Create an array of ~170,000 integers (about the size of an average film in frames). No need to be unsigned, because the MAX bit rate on a DVD is ~9Mbps. So fill the array with random values from 2000 to 9000. Now normalize the array with a scale, simulating our bit rates that we will use with TMPEG. That is MIN=300, MAX=2,500 so 2,000(DVD) would be 300(KVCD) low watermark, and 9,000(DVD) would be 2,500(KVCD) high watermark. See how long it takes to process that :!:
Then of course there's one more step, and that is to arrange the bit rate and trim and adjust, to create the average bit rate that MovieStacker suggests for the movie being processsed. That is the step that will take the longest time and the trickiest of all ;)

-kwag

I will, when it actually works and does something usefull! Right now it's just a dummy write of the frames and frame type into the x.stats file.

-kwag

To read in 170,000 entries of format "Frame No,Frame Type,Value", then process the array converting from scale 2000-9000 to 300-2500 takes about 27 seconds using perl.

Now, this is with hard coding the scale ranges and mappings. If we don't know the MIN MAX values (ie. 2000 to 9000) at the beginning then we need to process the array and find the MIN and MAX values. Which is fine, it will not add much time onto the 27 seconds and we can do this whilst reading in the file and it will not take any user time to calulate the scale to scale mappings.

As for the last step, I'm not sure what needs to be done to align to the MovieStacker average, but if you can expain the theory I'll code it up. Then all we need is your modified DVD2AVI to do a few tests.

Cheers :)

That's the best part of all :lol:
First, after normalizing the DVD->KVCD bit rates, we have to scan the array and find the average bit rate. Then using MovieStacker (or other VBR calculator), the array has to be balanced to the wanted average bit rate. This is the part that will take the longest time to process, because if the average bit rate is above wanted, you have to lower bitrates!. So the array will probably have to be sorted by bit rate, and then clip down from the high bit rate part of the array in order to bring down the average. This will require many many passes to fine tune and trim, until the average bit rate is obtained again by reading the complete array again and again after adjustments for re-computing the average bit rate. Fun isn't it :mrgreen:
Well, that would be my algorythm, just off the top of my head. Of course, in reality, it will probably be slightly different (but not much!) :wink:

-kwag

@Kwag,

Can a weighted average be used??? The number of frames divided into
the sum of the bitrates for each frame, or is this too simplistic??

-bp

Yes, I believe so. There are probably many ways to skin this cat :lol:
This might just be another "Fine tunning" adventure, just like the file prediction method :D

-kwag

@Kwag

that actually does not sound too bad. So basically we get an average from our normalised KVCD data and then either scale up to the movie stacker average or down.

Do we then cap at 2500 for any value that goes over it and anything that drops below 300 we fix at 300 (or whatever MIN MAX values are set - this will obviously be configurable:)

I'll crank something out later with the fudged data I am using and we'll see what it comes out with

Cheers,

Right!Exactly :D That sounds great!. We can really generate dummy data, even if it looks like hell!, but then we can try and feed TMPEG with the data just to see if it does work before actually proceeding with more serious stuff. Just in case it doesn't work!

-kwag

@Kwag,

Can't wait to hear about the results (good or bad) from this dummy data.
A crude start like file prediction, but over time the refinements would
prove very interesting. :D

-bp

Just like the "CQ Crap thread". Remember that "?"
And look where we are today :mrgreen:

-kwag

Was talking to a friend last night (far better at maths than i am - he remember things from school!), and he was making a few suggestions on how to take our normalised KVCD data and scale it against MovieStackers average Bit Rate.

So for the first draft, I'll calculate the KVCD average and rescale against MovieStacker, but ideally we says we need to plot the bell curve and factor out redundant points. He suggested that once we get the I, P and B frame information we generate a few graphs from various films to see what kind of curves we get.

Long term it would be good to be able to have a graph plotted in our app so that we can manually tweak upper/lower values to optimise our KVCD average.

Anyway I digress, I'll have a script ready for later. Its written in perl and only about 70 lines long, so rewrite, fix, etc in the language of choice.

Cheers,

Here is a sample of the output I have generated. The Frames are based on the output from Kwags version of Dvd2Avi. I changed the output from "0 -> I Frame" to "0,I,3456" where the bitrate value was created randomly between 2000 and 9000 (see the SRC_MIN and SRC_MAX parameters).

The data is rescaled to the KVCD scale, then re-averaged against the average value in MovieStacker. So based on The Matrix you can see the sample output below.

The re-average of KVCD data is slightly out as I rounded values instead of keeping them as floats till the output section but for now its close enough.

Will upload script later with a pointer so you can mess over the weekend,

Cheers,

Hi rhino,

I just tested your data, but we have a problem with TMPEG :evil:
When I import your data under "Force Picture Type", the frame order is correct just like you specified here:

But TMPEG is not following the bit rate specified for each frame, but instead it's setting an (sort of) average bit rate on every frame by averaging the bitrates on your table 8O
When I load your data, I get this on TMPEG's frame timeline:

Frame 0 (I Picture ) 551Kbps, New Group
Frame 1 (B Picture ) 551Kbps
Frame 2 (B Picture ) 551Kbps
Frame 3 (P Picture ) 551Kbps
Frame 4 (B Picture ) 551Kbps
Frame 5 (B Picture ) 551Kbps

Etc. :x
Something is not correct in TMPEG, or it just works that way :!:
If this can be fixed or made to work the way it should, were fine. If not, we're screwed :cry:
I'm going to play with TMPEG and see exactly why it behaves like this.

-kwag

@Kwag,

This could be important. It's from vcdhelp:

-bp

Thanks bp,

Yes that's the way I have it set (MVBR), but it behaves the same either with MPEG-1 and MPEG-2.
I'm going to try older versions of TMPEG, and see if it does the same :roll:

-kwag

I have been messing with Tmpgenc today (as i'm working from home:) and I noticed that I couldnot set the bitrate for P and B frames. And man does it take a long time to load in 192,422 frames. This could be a big downside to the process.

I'm currently fighting with my webhost - they took away my ssh access because some users screw things up when using shell access.

That I hadn't tried! How long did it take :?: Don't you just hate that :twisted: :lol:

-kwag