Quantcast KVCD Predictor - Page 2 - digitalFAQ.com Forums [Archives]
Go Back    digitalFAQ.com Forums [Archives] > Video Production Forums > Avisynth Scripting

Reply
 
LinkBack Thread Tools
  #21  
11-18-2002, 09:11 PM
SansGrip SansGrip is offline
Free Member
 
Join Date: Nov 2002
Location: Ontario, Canada
Posts: 1,135
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by kwag
BINGO *** Jack Pot ***, you hit it right on the nose
Sheesh... Three versions later . Anyway, now it's producing the right answers, let's see how it holds up to some real vigorous testing...

Edit: You forgot to update the source code link in the announcement . It should be 0.1b_src.
Reply With Quote
Someday, 12:01 PM
admin's Avatar
Site Staff / Ad Manager
 
Join Date: Dec 2002
Posts: 42
Thanks: ∞
Thanked 42 Times in 42 Posts
  #22  
11-18-2002, 09:26 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by SansGrip
...which leads me to the question, should there be a facility for specifying a custom CD size, for those who like to overburn?
That's a good idea!

Quote:
Also, wouldn't it be more logical and intuitive for a "scale" factor to be multiplicative? That is, instead of dividing by 0.95, multiplying by 1.05?
The reason for the .95 came to be, because the sampling taken on a one second basis, always produces ~2% to ~3% lower file size than predicted. The reason is the KVCD GOP, always gives smaller file size than a standard GOP, where there is less compression. So on tighter GOP's, like the standard IBBP ( Not IBBBP as in KVCD ), the prediction is closest to 1.0, and maybe above 1.0, for very active movies with lots of I frames inserted at the point of scene changes. So on KVCD's it will always be a factor below 1.0, and 1.0 or above for standard mpeg GOP structures.
Quote:
Also, @kwag, what do you think of the idea in the readme of some kind of formula to give a guess as to a good CQ/bitrate for producing a sample of the desired size? Too difficult?
I don't think it would be too difficult
Maybe taking instead of 100 initial one second snapshots, take 10 one second shots. Then increase to 20. That should be close enough, but on each step, adjust the CQ_VBR value. Then jump to the 100 samples and zero in, just like on a binary search. Ex: if CQ_VBR at 50 is too high, divide by two, so now it's 25. Else, multiply by two, so it's 75. That way it should zero in on a CQ_VBR target pretty fast Maybe one pass of 10 one pass of 20 and one or two passes of 100 snapshots would hit the target pretty close.
Hope I explained myself correctly

-kwag
Reply With Quote
  #23  
11-18-2002, 09:29 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by SansGrip
Edit: You forgot to update the source code link in the announcement . It should be 0.1b_src.
Sorry.
Done.

-kwag
Reply With Quote
  #24  
11-18-2002, 09:43 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
@SansGrip,

From your readme file:

"0.1b Note to self: Learn difference between multiplying and dividing. "

I'm still laughing at this one
Reply With Quote
  #25  
11-18-2002, 09:58 PM
SansGrip SansGrip is offline
Free Member
 
Join Date: Nov 2002
Location: Ontario, Canada
Posts: 1,135
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by kwag
Quote:
Originally Posted by SansGrip
...which leads me to the question, should there be a facility for specifying a custom CD size, for those who like to overburn?
That's a good idea!
Ok, I hope to get that in for 0.2 along with "Get movie length from source file...".

Quote:
Originally Posted by kwag
Then jump to the 100 samples and zero in, just like on a binary search.
This is what I've been doing, but I didn't really mean what you thought I meant. I think .

What I meant was, it would be nice if KVCD Predictor could display, along with the maximum sample size, a guess as to what CQ and bitrate you'd want to try in order to hit that target. I'm guessing to do this it would need to know the template you're using and maybe whether the movie is "low action", "high action" etc.

That's why I asked if it would be too difficult .

Edit: Though, thinking about it, it shouldn't be too hard to implement that kind of binary search via some kind of wizard interface, with it saying "Encode with CQ 25, min bitrate 500" then you tell it the size of the file, then it says "In that case, try CQ 22, min bitrate 400" and so on... Stupid idea?
Reply With Quote
  #26  
11-18-2002, 10:08 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by SansGrip
I'm guessing to do this it would need to know the template you're using and maybe whether the movie is "low action", "high action" etc.

That's why I asked if it would be too difficult .
I think that would involve something like doing a quick scan on the actual material to be encoded. Then determining, based on the bit rate activity (high/low), what kind of movie it is. Sort of a "Quick Pass" similar to the prediction samples, but actually analyzing the material. Then it would have to be compared to some pre established constant references. Like which material is low action, and which is high action. Which of course could be done by running the program against many movies and creating a "fingerprint" of a DVD or movie for the encoding bit rate to be used.

-kwag
Reply With Quote
  #27  
11-18-2002, 10:23 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by SansGrip
Edit: Though, thinking about it, it shouldn't be too hard to implement that kind of binary search via some kind of wizard interface, with it saying "Encode with CQ 25, min bitrate 500" then you tell it the size of the file, then it says "In that case, try CQ 22, min bitrate 400" and so on... Stupid idea?
No, not a stupid idea at all . But it might be better to keep the bit rate boundaries fixed (MIN/MAX) and only adjust/slide the CQ_VBR value. This way it's only one factor involved. Not three ( CQ_VBR, MIN, MAX ) wich then also change the linearity curve of the CQ_VBR. Like MIN=300, MAX=1,150, CQ_VBR=30 gives a larger file size than MIN=300, MAX=2000, CQ_VBR=30 on a low action scene . Because we changed the MAX from 1,150 to 2,000, and kept the CQ_VBR value constant, the scale in the encoder changed drastically, and produces a worse result. This example only on low action. On high action, then of course the file size is larger with the MAX=2000. Don't know about other encoders, but this is the behaviour of TMPEG.

-kwag
Reply With Quote
  #28  
11-18-2002, 10:31 PM
rendalunit rendalunit is offline
Free Member
 
Join Date: Apr 2002
Location: san jose, Ca
Posts: 1,148
Thanks: 0
Thanked 0 Times in 0 Posts
@SansGrip,

When you enter the time length of the movie does it default to 23.976 fps? If so then if it's 25 or 29.97 then you have to enter # of frames and fps so maybe the fps should be a "must enter" value- or you could have a choice of FILM, PAL, or NTSC on the main screen.

Here's a crazy idea- what if FitCD could be modified so that it will write a line into the .avs with the #of frames and framerate (or you could do it manually), something like this:

# 150000, 23.976

then KVCD Predictor could read this info from the .avs so you won't have to enter anything!

here's crazier part:

then when you've adjusted the CQ to get the target sample size then KVCD Predictor would go into the avisynth script and remove or comment out the file prediction lines so that you're ready to go!

just and idea
-ren
Reply With Quote
  #29  
11-18-2002, 11:23 PM
SansGrip SansGrip is offline
Free Member
 
Join Date: Nov 2002
Location: Ontario, Canada
Posts: 1,135
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by rendalunit
When you enter the time length of the movie does it default to 23.976 fps?
The "Enter movie length in frames and frames per second..." dialog box defaults to 23.976, but the length is actually stored internally as hours, minutes, seconds. Before calculating the target sample size this is converted into seconds, since that's what matters in the formula. The (frames / fps) part of the formula just gives you the length of the movie in seconds anyway.

So there's no need to enter an fps unless you want to specify the movie length in frames, and thus no need for a film/PAL/NTSC choice.

Quote:
Originally Posted by rendalunit
then KVCD Predictor could read this info from the .avs so you won't have to enter anything!
Once "Get movie length from source file..." is implemented, you'll be able simply to browse to either an AVI, AVS or D2V file (perhaps even MPEG at some point, if there's any demand for it) and KVCD Predictor will read the frames/fps information directly from the file. You'll also be able just to drag 'n' drop the video file onto the window to do the same thing.

Quote:
Originally Posted by rendalunit
then when you've adjusted the CQ to get the target sample size then KVCD Predictor would go into the avisynth script and remove or comment out the file prediction lines so that you're ready to go!
I was actually trying to think of a way to get rid of the "add some lines to script" step of the whole thing, but the only "solution" I could come up with was automatic generation of a new, temporary script to use just for encoding the sample. But it saves such little effort for the user that I'm not sure it's worth my effort to implement and debug it .
Reply With Quote
  #30  
11-18-2002, 11:25 PM
SansGrip SansGrip is offline
Free Member
 
Join Date: Nov 2002
Location: Ontario, Canada
Posts: 1,135
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by kwag
I think that would involve something like doing a quick scan on the actual material to be encoded. Then determining, based on the bit rate activity (high/low), what kind of movie it is.
Sounds like motion estimation to me. Ouch!
Reply With Quote
  #31  
11-18-2002, 11:48 PM
SansGrip SansGrip is offline
Free Member
 
Join Date: Nov 2002
Location: Ontario, Canada
Posts: 1,135
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by kwag
No, not a stupid idea at all . But it might be better to keep the bit rate boundaries fixed (MIN/MAX) and only adjust/slide the CQ_VBR value.
I like to keep variables to a minimum. Fewer opportunities for me to mix up my mathematical symbols .

As for the wizard-assisted binary search for the best CQ, that sounds like an interesting idea to me. I imagine something like this:

1. Specify path of sample file. Let x=initial CQ value, say 25
2. Tell user to encode with CQ of x
3. Wait for "done" button to be pressed
4. Read file size
5. If within threshold of target, stop
6. If low, let x=new higher value
7. If high, let x=new lower value
8. Goto 2

The fun part will be figuring out the new higher/lower value. It might not be quicker for an expert (read: kwag) who can get it right in a couple of tries, but it might help us normal folk (read: me) who take five or six attempts to get close :P.

It could even suggest a couple of fractional values towards the end, to really squeeze the most out of it.

How does that sound? Feasible? Useful? Worth the effort?
Reply With Quote
  #32  
11-19-2002, 12:03 AM
black prince black prince is offline
Free Member
 
Join Date: Jul 2002
Posts: 1,224
Thanks: 0
Thanked 0 Times in 0 Posts
@SansGrip and Kwag,

Getting all kinds of errors trying to execute KVCD file prediction after
insatlling Microsoft.NET:

Quote:
Common Language Runtime Debugging Services
Application has generated an exception that could be
handled Process Id=0xffA4sed(-375315),
Thread Id=0xffA46A5(-375131)
Quote:
Registered JIT debugger is not available. Attempt to launch a
JIT debugger is not available. Following command resulted in
an error code of 0x2(2) ....
Then I tried to reinstall microsoft.net and could not remove it
or re-install it.

I don't suppose a standalone version could be created
I can't seem to get this to work with .NET.

-black prince
Reply With Quote
  #33  
11-19-2002, 12:16 AM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by SansGrip
Quote:
Originally Posted by kwag
No, not a stupid idea at all . But it might be better to keep the bit rate boundaries fixed (MIN/MAX) and only adjust/slide the CQ_VBR value.
I like to keep variables to a minimum. Fewer opportunities for me to mix up my mathematical symbols .

As for the wizard-assisted binary search for the best CQ, that sounds like an interesting idea to me. I imagine something like this:

1. Specify path of sample file. Let x=initial CQ value, say 25
2. Tell user to encode with CQ of x
3. Wait for "done" button to be pressed
4. Read file size
5. If within threshold of target, stop
6. If low, let x=new higher value
7. If high, let x=new lower value
8. Goto 2

The fun part will be figuring out the new higher/lower value. It might not be quicker for an expert (read: kwag) who can get it right in a couple of tries, but it might help us normal folk (read: me) who take five or six attempts to get close :P.

It could even suggest a couple of fractional values towards the end, to really squeeze the most out of it.

How does that sound? Feasible? Useful? Worth the effort?
You bet Sounds Feasible AND Useful AND Worth the effort
Here's a simple formula I've been using to zero in faster on the prediction. Say I need 10MB for target and I'm encoding at a starting CQ_VBR of 25, but on the first try I get 6MB. So just do 10/6 =1.6666 and then 25 * 1.66666 = ~41. Run again with CQ_VBR=41. It usually takes me two or three runs (sometimes four) to hit the target. Never fails.
And that's WAY better than waiting for two or three X-pass VBR

-kwag
Reply With Quote
  #34  
11-19-2002, 12:47 AM
SansGrip SansGrip is offline
Free Member
 
Join Date: Nov 2002
Location: Ontario, Canada
Posts: 1,135
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by black prince
Getting all kinds of errors trying to execute KVCD file prediction after insatlling Microsoft.NET:
Very weird. Theoretically if you have the runtime installed you should just be able to double-click it and go.

I asked Dr Google and he came up with a bunch of other people with this kind of problem. One suggested solution I read was uninstalling the .NET framework (through Add/Remove Programs), then installing MDAC 2.7 from here, then reinstalling .NET.

Sorry you're having problems with this . Unfortunately I don't have a copy of Windows ME to test it on, but I can try it on my Windows 98 laptop tomorrow. Hopefully I'll have the same problem, and will figure out how to fix it, and can then tell you .

Quote:
Originally Posted by black prince
I don't suppose a standalone version could be created
Well I could rewrite it in plain C++ and MFC (just like in the bad old days before running water), but that would probably take three-to-four times as long as it did to write it in C# (and the same goes for any future enhancements, of which I hope there will be many). I'd like to try to get .NET working on your machine before I consider the horrors of Windows programming in C++ .
Reply With Quote
  #35  
11-19-2002, 12:50 AM
SansGrip SansGrip is offline
Free Member
 
Join Date: Nov 2002
Location: Ontario, Canada
Posts: 1,135
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by kwag
Sounds Feasible AND Useful AND Worth the effort
Then let's do it!

Quote:
Originally Posted by kwag
Here's a simple formula I've been using to zero in faster on the prediction.
Sounds perfect for automation. Not even I could mis-code that algorithm . It's late, but... I think I might start coding it right now .
Reply With Quote
  #36  
11-19-2002, 06:12 AM
GFR GFR is offline
Free Member
 
Join Date: May 2002
Posts: 438
Thanks: 0
Thanked 0 Times in 0 Posts
The problem with file size predition is that file size and Q don't have a linear relationship, this relationship is not well-known and it varies from movie to movie... This looks like a typical problem where you could give a neural network a try.

I'd imagine a network with 2 inputs: the size of the prediction at a default Q, and a given Q at which you want the estimate. The output of the network would be the approximate file size for the new Q.

To train the network one would need a large database with the filesize prediction for several Q's, for lots of movies (this could be a collaborative effort). The network may take some time to train, but once trained (and it would be trained only once at design time), it runs imediately. So you'd run one file size prediction encode at a default Q, give this number to the network and tweak a slider until the chosen Q gives you the desired file size.

If this estimative turns to be accurate enough, that's the fastest prediction you can get; but, we can collect the data, train the network and find out that it's no good and just forget about it.
Reply With Quote
  #37  
11-19-2002, 06:48 AM
GFR GFR is offline
Free Member
 
Join Date: May 2002
Posts: 438
Thanks: 0
Thanked 0 Times in 0 Posts
This may be worth looking at:

Quote:
Automatic Recogition of Film Genres
Abstract: Film genres in digital video can be detected automatically. In a three-step approach we analyze first the syntactic properties of digital films: color statistics, cut detection, camera motion, object motion and audio. In a second step we use these statistics to derive at a more abstract level film style attributes such as camera panning and zooming, speech and music. These are distinguishing properties for film genres, e.g. newscasts vs. sports vs. commercials. In the third and final step we map the detected style attributes to film genres. Algorithms for the three steps are presented in detail, and we report on initial experience with real videos. It is our goal to automatically classify the large body of existing video for easier access in digital video-on-demand databases.
http://www.informatik.uni-mannheim.d...ephan1995a.pdf
http://www.informatik.uni-mannheim.d...ephan1995b.pdf
Reply With Quote
  #38  
11-19-2002, 07:02 AM
ozjeff99 ozjeff99 is offline
Free Member
 
Join Date: May 2002
Location: Sydney, Australia
Posts: 159
Thanks: 0
Thanked 0 Times in 0 Posts
Predictor is a great tool. Thanks Sansgrip. I get my movie length from Fitcd which nicely shows it in total seconds. Is it possible to have the program accept this single value rather than have to make several entries. This probably sounds trivial
Reply With Quote
  #39  
11-19-2002, 08:41 AM
black prince black prince is offline
Free Member
 
Join Date: Jul 2002
Posts: 1,224
Thanks: 0
Thanked 0 Times in 0 Posts
Hi SansGrip,

SansGrip wrote:
Quote:
Very weird. Theoretically if you have the runtime installed you should just be able to double-click it and go.

I asked Dr Google and he came up with a bunch of other people with this kind of problem. One suggested solution I read was uninstalling the .NET framework (through Add/Remove Programs), then installing MDAC 2.7 from here, then reinstalling .NET.

Sorry you're having problems with this . Unfortunately I don't have a copy of Windows ME to test it on, but I can try it on my Windows 98 laptop tomorrow. Hopefully I'll have the same problem, and will figure out how to fix it, and can then tell you .
Installing .NET says delete product code: {3BD60997-A7E4-4A55.....} before it
will install. Add/Remove reports it can't find file "netfx.mis" and terminates
removing .NET. Tried to use regedit to remove product code from above.

Found it in HKEY_Local_machine\Software\NET Framework Setup\
Full\v 1.0.....\. Deleted this and rebooted. install .NET still says product
code must deleted. So, I can't get rid of old .NET and new .NET won't
install. This is frustrating I'll just wait to a standalone is available.

-black prince
Reply With Quote
  #40  
11-19-2002, 09:43 AM
black prince black prince is offline
Free Member
 
Join Date: Jul 2002
Posts: 1,224
Thanks: 0
Thanked 0 Times in 0 Posts
Hi SansGrip and Kwag,

Don't let my problems with .NET, Flux, and NoMo worry you for now. Keep
working towards your goals and I will solve this eventually.
I am not trying to discourage you in any way. I want to see more
improvements just as bad as you. I have other pressing problems
unrelated to KVCD, but I still will keep watching your progress and
when I can will add layman's comments and suggestions.

-black prince
Reply With Quote
Reply




Similar Threads
Thread Thread Starter Forum Replies Last Post
KVCD Predictor-0.1b download? hansolh960 Video Encoding and Conversion 1 01-03-2012 01:58 PM
KVCD Predictor or Manual Prediction vhelp Avisynth Scripting 12 08-08-2004 02:59 AM
KVCD Predictor? FlavioMetal Conversão e Codificação de Vídeo (Português) 0 12-16-2003 08:52 AM
Kvcd Predictor DKruskie Avisynth Scripting 4 05-14-2003 06:44 PM
KVCD Predictor is out! kwag Video Encoding and Conversion 0 11-18-2002 06:25 PM




 
All times are GMT -5. The time now is 04:14 PM  —  vBulletin © Jelsoft Enterprises Ltd