digitalFAQ.com Forums [Archives]

digitalFAQ.com Forums [Archives] (http://www.digitalfaq.com/archives/)
-   Avisynth Scripting (http://www.digitalfaq.com/archives/avisynth/)
-   -   Ping-pong prediction vs Long-short prediction (http://www.digitalfaq.com/archives/avisynth/6832-ping-pong-prediction.html)

rds_correia 06-10-2005 08:14 PM

Yep, I get it :).
I had noticed that slicer had a very small tendecy to oversize.
Maybe that's got something to do with the way I was using.
Thanks for pointing out my mistake.
Cheers

Prodater64 06-10-2005 08:53 PM

I want to illustrate what I'm saying.

http://www.digitalfaq.com/archives/error.gif

Thanks to ImageShack for Free Image Hosting

Your target is 52045 (is avg between next ping-pong values, just hipotetic to illustrate this)

To obtain it you needed:

One 2% ping = 52371 KB
One 2% pong = 51719 KB
Total encoding 4%

I needed:
One 1% Ping + 1% Pong = 52410 KB
Total encoding 2%
Difference with usual ping-pong 0.77%
Not more than first avs edition needed.
Half time to obtain it.

The script as you point me previously:
Quote:

Video=Mpeg2source("E:\DVD Authoring Working Folder\PB.d2v")
ping = Video.slicer(1,15,1,"ping")
pong = Video.slicer(1,15,1,"pong")
last = ping++pong

#crop, resize and filtering

function slicer(clip input,int percent,int Goplenght,int Gopmulti,string "ping")
{
PercCount=(Framecount(input)/100)*percent
period=int(Framecount(input)/PercCount)*(Goplenght*Gopmulti)
input=(ping=="pong")?input.trim((period/2),Framecount(input)):input
selectrangeevery(input,period,(Goplenght*Gopmulti) )
}
Most important, if you have a command line encoder you can do:

Quote:

set v_target_size=52045
for /L %%G in (1,1,100) do set v_quant=%%G & call :routine
goto :eof
:routine
start /wait Encoder -i "my.avs" -o "my%v_quant%.m2v" -cq %v_quant%
for %%H in ("my%v_quant%.m2v") do set v_sample_size=%%~zH
if v_sample_size geq v_target_size goto :eof
goto :eof

Encoder -i "myfull.avs" -o "myfull.m2v" -cq %v_quant%
:eof
and go to take some coffee or to sleep.
When you come back you will have full elementary stream.

The command line maybe need some adjustements, it is only to give an idea.

Don't see you some advantages in this method?
I see it.

Lesser sample with this method is 2% as slicer function don't supports floating values.

Edited.

rds_correia 06-11-2005 08:43 AM

Pro...
I don't know where you're going here...
I see you have found a break through with a method you are trying but I can't see what is it that you're trying.
Instead of many calculations could you try to explain me what you're trying to achieve?
And how you'd want to achieve it?
And please don't get mad at me for this post but I am really bad at maths :lol:.
Cheers

Prodater64 06-11-2005 08:55 AM

Quote:

Originally Posted by rds_correia
Pro...
I don't know where you're going here...
I see you have found a break through with a method you are trying but I can't see what is it that you're trying.
Instead of many calculations could you try to explain me what you're trying to achieve?
And how you'd want to achieve it?
And please don't get mad at me for this post but I am really bad at maths :lol:.
Cheers

Is a little change in ping-pong method.

Let's say that you want to do a prediction. Do math to obtain 2% of your final target size.
You don't need to do any ping pong as original Incredible method.
You don't need to calculate proportional difference.
You only need to do one pass ping-pongs (as the previous script show) and see in your screen if size match with your 2% target size.
It is an easier mod ping-pong method.

Prodater64 06-26-2005 08:55 PM

I was talking witn Amnon82 from d9 about HC-Qmatic, AutoCQ, and AutoQ2 (by Amnon82). All these tools do predict and encodes with differents encoders CCE > AutoQ2, HC Encoder > AutoQ2 and HC-Qmatic.
Both of us have problems with HC Encoder doubt to that program itself, as we all know.
But with AutoQ2 I talked with Amnon about prediction method and I proposed him my new prediction way with Incredible slicer function (he was using the function but with a ping pong ping pong ping pong ping pong approach):

Quote:

Video=Mpeg2source("E:\DVD Authoring Working Folder\PB.d2v")
ping = Video.slicer(1,15,1,"ping")
pong = Video.slicer(1,15,1,"pong")
last = ping++pong

#crop, resize and filtering

function slicer(clip input,int percent,int Goplenght,int Gopmulti,string "ping")
{
PercCount=(Framecount(input)/100)*percent
period=int(Framecount(input)/PercCount)*(Goplenght*Gopmulti)
input=(ping=="pong")?input.trim((period/2),Framecount(input)):input
selectrangeevery(input,period,(Goplenght*Gopmulti) )
}
Now his tool is predicting matching 99.999% the target size.

Does somebody test it here?

rds_correia 06-27-2005 01:55 AM

Wait, there no reason why his tool wouldn't be accurate doing ping-pong ping-pong ping-pong.
It just meant that it would take more time that's all.
You really should think if it's wise to share information with Avalon, he is a dangerous guy.
Even because he knew the old slicer but he might not know the latest slicer.
Cheers

Prodater64 06-27-2005 03:45 AM

Quote:

Originally Posted by rds_correia
Wait, there no reason why his tool wouldn't be accurate doing ping-pong ping-pong ping-pong.
It just meant that it would take more time that's all.
Cheers

I don't say the usual method is inaccurate.
My "one pass ping pong method" could be more accurated than the original ping pong.

rds_correia 06-27-2005 05:15 AM

Quote:

Originally Posted by Prodater64
I don't say the usual method is inaccurate.
My "one pass ping pong method" could be more accurated than the original ping pong.

The usual method is accurate.
It is just innacurate when used with an encoder that doesn't fully respect the given commands.
That is what is happening with HC.
You give it the right Q figure but it doesn't use it.
It tries to use it but when set for Max bitrate checking it will override the given Q.
And that's why it doesn't output an accurate OPV.
Yes, your method could be more accurate.
But when used with HC (in it's current release status) it shouldn't be accurate.
I'm saying *shouldn't*.
But that's something that I'll try tonight on a couple of movies.
Cheers

Prodater64 06-27-2005 05:54 AM

Quote:

Originally Posted by rds_correia
Quote:

Originally Posted by Prodater64
I don't say the usual method is inaccurate.
My "one pass ping pong method" could be more accurated than the original ping pong.

The usual method is accurate.
It is just innacurate when used with an encoder that doesn't fully respect the given commands.
That is what is happening with HC.
You give it the right Q figure but it doesn't use it.
It tries to use it but when set for Max bitrate checking it will override the given Q.
And that's why it doesn't output an accurate OPV.
Yes, your method could be more accurate.
But when used with HC (in it's current release status) it shouldn't be accurate.
I'm saying *shouldn't*.
But that's something that I'll try tonight on a couple of movies.
Cheers

Don't spend your time in that. As you say, it is not a problem with prediction. It is a problem with HC.
But if you want, you can do 2 predictions, with, maybe TMPGEnc, one with old method, one with my one, and post here results, please.

rds_correia 06-27-2005 06:26 AM

Quote:

Originally Posted by Prodater64
But if you want, you can do 2 predictions, with, maybe TMPGEnc, one with old method, one with my one, and post here results, please.

Oops didn't make myself clear.
That's actually what I had in mind but didn't explain correctly.
Cheers

Dialhot 06-27-2005 07:24 AM

Quote:

Originally Posted by Prodater64
Now his tool is predicting matching 99.999% the target size.

And of course he added a line into the changelog to thanks Incredible and you for that, isn't it ? :roll:

Prodater64 06-27-2005 07:39 AM

Quote:

Originally Posted by Dialhot
Quote:

Originally Posted by Prodater64
Now his tool is predicting matching 99.999% the target size.

And of course he added a line into the changelog to thanks Incredible and you for that, isn't it ? :roll:

The line for Incredible was before I talked with him.
My line was added in last version.

incredible 06-27-2005 06:52 PM

Amnon (AKA Avalon) is very well known!

I would be careful when sharing ANY developements or even thoughts with him. So many times he did not respect other people by their ideas and their developing spirits, then afterwards he excused and said "ok, that was a bad behaviour and I have changed", but if you turn your back to him ... his behaviour continues and thats a fact!

He always tries to catch personality and thats not only a talking of mine, you can see clearly that in his deeds and behaviours everywhere. Look at his type of talking and acting at doom9. At Delphi Praxis community for instance in a thread he "begged" for a way to show him how to parse avs files, after some imho useful replies there he did find out it by himself (orig. quote in the next post of that thread) but .... shurely he didnt post the code there :arrow: nice isn't it. He TAKES but DOESN'T SHARE ... thats for me the conclusion.

My two cents for you Luis.

kwag 06-27-2005 07:13 PM

Quote:

Originally Posted by Prodater64
I was talking witn Amnon82 from d9 about HC-Qmatic, AutoCQ, and AutoQ2 (by Amnon82)..............

Why are you doing this Luis :?:
Aren't you aware that Amnon (alias Avalon) is "Persona Non Grata" in this forum :?:
Aren't you aware of what he did :?:
Are you aware that he can't program at all, and he uses other's code and says it's his :?:
If doom9 wants to permit this, that is, letting someone who is known to steal other's ideas (Avalon), well so be it (that tells me A LOT about doom.9.org, btw).
But here, Avalon was kicked out of this site, exactly for those reasons. For stealing ideas, and saying they were his, and for not giving credit to others.
:arrow: Just wanted to remind you of that :!:

-kwag

Prodater64 06-27-2005 07:45 PM

I thanks you very much your advises.
I will not condemn a person by his past actions.
There is no any present action of Amnon82 againts kvcd.net.
There is no any present action of Amnon82 againts me.
He gives me inmediately credit by my "one pass ping pong prediction" (OPPPP) method. He add a line for me in his thread and in his program.
I saw also an Incredible THX line.
All I am seeing is opposite to that you say me here.
Then I have not any reason to leave talking with him, interchanging knowledge.

And I would like to remark, that here, nobody, none of the people that read my last posts about "OPPPP" gives credit to it neither tested it.
Amnon82 test it inmediately, while we have a chating by messenger.

And at last, my code really is totally free as I release it without any kind of license, in a plain text file as a batch file is, and, of course, anybody can take it with any intention, and I can not do anything against that, neither I want to do.

Time will say me if you have reason or not. Thanks again.

Edited: I don't wanted to do this off topic, public. As I answered first rds advise by PM. But it seems that nobody remember moderators sub-forum.

rds_correia 06-28-2005 05:29 AM

Quote:

Originally Posted by Prodater64
Edited:... But it seems that nobody remember moderators sub-forum.

Hi Luis :),
Please don't take it worng, but perhaps this is an issue also to be discussed in the public part of the forum ;-).
You see, Avalon's past is full of wrong things he did to anybody that was passing by near him.
Some were even very far from him but he tried to get closer just to rip their work.
I'm not talking about open code ;-).
I'm talking about people - such as Hank315 - that have closed source but that he desperatly wants to help.
Why would he be so desperate to help Hank?
We all try to help Hank and many others everyday...
But we don't bug him every half an hour with something like hey, remember me? if you need help I'm here.
So it is obvious that some of us here want to make some points very much clear so that newbies - in the sense of newbies at kvcd.net but somewhat experienced in code development - don't fall in the same trap that Avalon has used so many times.
If you make a search at the HC sub-forum you will find one of my posts alerting Hank315 for such issues ;-).
So, I see this as an issue to be taken care right here at the forum, and perhaps in the Mods room too if you want to go into more details.
That is my most honest oppinion.
As for your latest findings with the so called OPPPP, I just need some info from you to be able to test it.
I always use a 1% ping-pong-ping-ping-ping-...-final pong approach.
What settings do I need to use in OPPPP to compare with such an above approach?
Cheers

Prodater64 06-28-2005 05:45 AM

Quote:

Originally Posted by rds_correia
As for your latest findings with the so called OPPPP, I just need some info from you to be able to test it.
I always use a 1% ping-pong-ping-ping-ping-...-final pong approach.
What settings do I need to use in OPPPP to compare with such an above approach?
Cheers

About Amnon82: If he is gulty. How many time is the sentence. Does him be banned from here for ever, is a "perpetua" or for life sentence. I thanks all your advices, but I repeat, I don't know any bad action in the near present performed by Amnon82. Can you understand this.
If a person did a crime, hi was sentenced and go ahead with that. But if here we are applyng that sentence for ever, we punishing his faults with a harder sentence than murderer.

About the OPPPP, you need to compare it with a 2% usual pinp pong (I said this before) as minimal value you can use in slicer command is 1%.

Then your usual ping-pong will be:

slicer(2,15,1,"ping")

And your OPPPP

slicer(1,15,1,"ping")
slicer(1,15,1,"pong")

Your gopsize and goplenght can be any that you want, but always same in both Incredible and my methods.

Edited: In you case that usually do use 1% slicers, maybe don't give any advantage to use OPPPP, as it will take same time as before, and I don't think that difference be so big. But better is test it and see results.

incredible 06-28-2005 06:35 AM

I do put in in codetext, so monospaced Letters will keep the proportions ..

Code:

Slicer(2,15,1,"ping") results in a prediction pass of ...

_________________________________________________________________________________________________

x.........x.........x.........x..........x..........x..........x..........x..........x..........x
_________________________________________________________________________________________________



Slicer(1,15,1,"ping") =

_________________________________________________________________________________________________

x...................x....................x.....................x.....................x...........
_________________________________________________________________________________________________



Slicer(1,15,1,"pong") =

_________________________________________________________________________________________________

..........x...................x.....................x.....................x.....................x
_________________________________________________________________________________________________



So if you do ..

a=Slicer(1,15,1,"ping")
b=Slicer(1,15,1,"pong")
return a+b

This means both streams will be appended:

                                                                                        AppendingPoint of
                                            Moviecontend ping'ed                                              stream a and b                                    Same! Moviecontend but pong'ed
________________________________________________________________________________________________|_________________________________________________________________________________________________

x...................x....................x.....................x.....................x.....................x...................x.....................x.....................x.....................x
__________________________________________________________________________________________________________________________________________________________________________________________________



... but the real sampled movieinformation will be the same like ..

_________________________________________________________________________________________________

x.........x.........x.........x..........x..........x..........x..........x..........x..........x
_________________________________________________________________________________________________


... and thats Slicer(2,15,1, "ping").


rds_correia 06-28-2005 09:39 AM

@Andrej
So is it mathematically prooven that 1% OPPPP is exactly the same as 2% usual methods?
Because I'm getting ready to try this tonight but if it's scientifically prooven that results are exactly the same there's no reason to do it.
@Luis
I believe this is not the 1st time that Andrej points us this explanation.
Have you checked it already?
Cheers

Dialhot 06-28-2005 01:20 PM

Be carrefull, the two options are not really equivalent : you don't have the same number of I pictures, and they are not generated at the same place.

In other words, you can't tell that Slicer(1,15,1,"ping")+Slicer(1,15,1,"pong") == Slicer(2,15,1, "ping")

Now, I can't tell you which one is more accurate than the other...

Prodater64 06-28-2005 01:32 PM

Quote:

Originally Posted by rds_correia
@Andrej
So is it mathematically prooven that 1% OPPPP is exactly the same as 2% usual methods?
Because I'm getting ready to try this tonight but if it's scientifically prooven that results are exactly the same there's no reason to do it.
@Luis
I believe this is not the 1st time that Andrej points us this explanation.
Have you checked it already?
Cheers

So that Incredible points means:

2%pingSize = 1%pingSize + 1%pongSize = 2%OPPPPSize

That supports my hipotesis.
You don't need to do any calculation neither obtain differences or proportions. Only looking at filesize in the screen, you will have an idea of how much near of your target are you.
A 2% OPPPP is the same than an 1% usual ping pong, but you don't need to edit your avs, neither launch again your encoder, etc, etc. More, if you want to obtain an exact match or so close as you want, you can't do it straight obtaining only ping pongs (as you always have to average them and do calculations), but with an OPPPP you can reach exact match looking at the screen.

Quote:

So is it mathematically prooven that 1% OPPPP is exactly the same as 2% usual methods?
Because I'm getting ready to try this tonight but if it's scientifically prooven that results are exactly the same there's no reason to do it.
Always you should demonstrate hipotesis with facts. I posted my results:

http://www.digitalfaq.com/archives/error.gif

Could you go ahead and do your owns tests. Also same I asking to Incredible. Please.

Edited: Also I would like to know, what method seems you easier, after your test end.

rds_correia 06-28-2005 02:30 PM

Quote:

Originally Posted by Prodater64
I posted my results:

http://www.digitalfaq.com/archives/error.gif

Could you go ahead and do your owns tests. Also same I asking to Incredible. Please.

Edited: Also I would like to know, what method seems you easier, after your test end.

Hi Luis :),
I see your results look indeed interesting.
But I'm a little confused.
The 1% Ping+Pong gave you a 52.410KB result.
Then the 2% Ping / 2% Pong gave you a 52.045KB result.
I would not start giving a party right now, with these results.
You see, in fact you just sampled only 2% of your movie and it went off by 365KB.
I am still not talking about which will be closest in terms of prediction.
I am only stating that it is a big difference.
You see, if you were encoding the whole movie you would have ended up with a difference of 18.250KB, which is a lot.
As to which will be easier or faster or more clean to use, for me it is obvious I would peek OPPPP.
The only downside is that it will take longer in each prediction fase.
Predicting Ping+Pong+Ping+Pong+Ping+Pong+Ping+Pong takes longer than Ping+Pong+Ping+Ping+Ping+Pong and I'm counting the last Pong that from my point of view is almost useless.
Tonight I'll run 2 predictions and let the movies encoding overnight.
Tomorrow I'll tell you the results.
I'm only doing it with 2 movies due to harddrive-available-space restrictions :roll: ...
Cheers

Prodater64 06-28-2005 03:26 PM

Quote:

Originally Posted by rds_correia
Hi Luis :),
I see your results look indeed interesting.
But I'm a little confused.
The 1% Ping+Pong gave you a 52.410KB result.
Then the 2% Ping / 2% Pong gave you a 52.045KB result.

As I mentioned previously, 52045KB is just the avg, as you point, between ping and pong (it means that I obtained that value after doing ping pong passes, averaging it, so you see the usual ping pong matching perfect). For that it seems that traditional ping pong method is more exact. But suppose the target would be 52400?
In other word, do your prediction searching an exact match of your sample size. If you reach an exact match with 2 methods, it should give you same Q value. In this case, only left as important what method was easier and quicker.
If Q value is different, you need to do a whole encoding of same film with 2 differents Q found.
Then evaluate:
Final difference is significant whatever method used. If yes, you must to select final size closer method as winner.
If not, put in the balance what method is quicker and easier.

Quote:

I would not start giving a party right now, with these results.
You see, in fact you just sampled only 2% of your movie and it went off by 365KB.
I am still not talking about which will be closest in terms of prediction.
I am only stating that it is a big difference.
You see, if you were encoding the whole movie you would have ended up with a difference of 18.250KB, which is a lot.
Despite 52045 was averaged from results:

Let's take 2602250KB as total video size (52045*50)

(18250/2602250)*100=0.7%

Does it seems to big for you?
What precission did you obtain usually with traditional ping pong prediction (TPPP)?

Edited: I think for precission comparations you must to do traditional ping pong 1% samples, against 2%(1% ping - 1% pong) OPPPP, as if you do use traditional ping pong 2% samples you will sample double with Incredible's method.
There is a posibility (I not tested already) that 2% OPPPP, sampling half than 2% TPPP, be more exact, but I don't think really it could be so.

kwag 06-28-2005 03:49 PM

Quote:

Originally Posted by Prodater64
Now his tool is predicting matching 99.999% the target size.

I would like to add that NO 1% or 2% sampling of a total footage, no matter how good it is, will ever achieve 99.999% target.
It's :arrow: I M P O S S I B L E, unless you sample the complete footage, because you can NEVER tell what the activity of the other 98% is :!: Period.
Anyone who claims that is a joker :!:

-kwag

Prodater64 06-28-2005 04:05 PM

Quote:

Originally Posted by kwag
Quote:

Originally Posted by Prodater64
Now his tool is predicting matching 99.999% the target size.

I would like to add that NO 1% or 2% sampling of a total footage, no matter how good it is, will ever achieve 99.999% target.
It's :arrow: I M P O S S I B L E, unless you sample the complete footage, because you can NEVER tell what the activity of the other 98% is :!: Period.
Anyone who claims that is a joker :!:

-kwag

Sorry, was my fault.
D9 thread title is:

Quote:

AutoQ2 for CCE - Version 0.5.3 (20050628) - now fits 99.99% the wanted Samplesize

kwag 06-28-2005 04:20 PM

Quote:

Originally Posted by Prodater64
Sorry, was my fault.
D9 thread title is:
Quote:

AutoQ2 for CCE - Version 0.5.3 (20050628) - now fits 99.99% the wanted Samplesize

That sounds more realistic!
But still, 99.99% accuracy on sample can be either -10% to +10% (or worse!) off target size on the full encode, specially if only 2% of the footage is sampled.
And that's why I drop prediction phases on CQMatic once I get to +-1% of wanted sample, because I still can't tell what the final size will be, even with 1% accuracy of the sample.
Still, I've found that with the latest algorithm in CQMatic, the worse case scenario is about 5% undersize to 2% oversize, and that's not that bad.

EDIT: Quoting what I just read:
"With the help of Prodater64 I find now a way to predict more exactly ever wanted. Now AutoQ2 matches 99.999% the wanted filesize."

Second post here: http://forum.doom9.org/showthread.php?t=96420

-kwag

Prodater64 06-28-2005 04:53 PM

Quote:

Originally Posted by kwag
EDIT: Quoting what I just read:
"With the help of Prodater64 I find now a way to predict more exactly ever wanted. Now AutoQ2 matches 99.999% the wanted filesize."

Second post here: http://forum.doom9.org/showthread.php?t=96420

-kwag

Well, regarding the thread title, it must be a typo.
Can I ask him?

Edited:

I didn't test AutoQ2 as I am working on my owns projects.
When I ends it, I'll give a try.
For this, I don't say here that tool is wonderful.
I only talking about about my prediction mod.
AutoQ2 is a proove that it works, at least, matching the target size.

A little bit OT:
You all did feel yourself bad when I talked with Amnon82.
I did feel bad myself, when here, in "my forum", nobody even tested that wath I am talking about, just a "superficial" interest, and disscarted inmediately. Im not claiming you a special interest in all I am saying here (maybe mostly stupid things), but my work related to this site (really a lot of hard work) is for all my friends and partners here, and I would like, some of you just give me a little, little bit of your time and feedback.
As another example, I had to insist hard and very, very much in Calcumatic thread, and I did need @rds help, to reach that somebody hear me about "finding a proportional space". At least, if such tool born any time, I will feel happy knowing that I was first 2 cents contributer.
End OT.

rds_correia 06-28-2005 06:26 PM

Well I'll run the test to see how close I can get in encoded filesize.
I must warn you guys, even if it was simple luck, I've reached less than 1% undersize several time with HC in (C)Q mode.
BTW, I was thinking.
I'm used to do 1% Ping Pong predictions so 1st I'll predict the OPPPP with 2% and then I'll try to do it with 1% also.
For that I'll change the slicer function in order to allow me the usage of decimal figures.
Like:
Code:

function slicer(clip input, float percent, int Goplenght, int Gopmulti, string "ping")
Just for testing purposes.
Hope Andrej doesn't mind :).
Cheers

incredible 06-28-2005 06:55 PM

First I do see that Amnon did put me OUT of the credits, that was shure!
Ok, its a mod on slicer where parts of predictions are used now in a diff. way but the root is very clear .... but who cares. Its the p**** off shit, that people do use other Ideas, techniques etc. ---- using that 99% core and use it just a different way. The name OPPPPP eben contains that "P" based on whatever Ping or Pong.

I dont understand ..... when the first release of slicer came out .... modding did start. Then one year silence. After one year an update came out .... and then modding started again ;)
Well "that" I dont! see proplematic for shure ;) But its nice to see that deeds do start inspirations (not said ironic)

I DO apreciate cooperations and I DO assist, if I have time and if Im not on other projects. But one thing is shure, these approaches do base on slicer.

Do imagine u use the MA script AS it is, just changing unfilter() and Temporalsoften() to other filter.dlls - and btw putting Karl out of the list. :lol:

These are two cents .... so the modder or developer should know by himself.




Quote:

I did feel bad myself, when here, in "my forum", nobody even tested that wath I am talking about, just a "superficial" interest
One moment Luis .... its a diff. kind of thing if someone doesnt have time in the exact same moment like where you want assistance. So "superficial" interest is NOT given. If there wouldn't be interest, some people like mine wouldnt take time in the job to write explanations understandable down in a thread like above.

Quote:

As another example, I had to insist hard and very, very much in Calcumatic thread, and I did need @rds help, to reach that somebody hear me about "finding a proportional space". At least, if such tool born any time, I will feel happy knowing that I was first 2 cents contributer.
The idea on a proportional space was already given at CQ mode a long time before .... Im doing CQ encodes (and Phil also AFAIK) in a way that movies on one DVD do keep the same quality.
BUT! The case of that idea on a 2pass approach DOES base on YOUR cents contribution.

There was NEVER a time in here when someone did start a real new Idea that his name wasnt kept in mind after that. ;) Like Arnet_tenrA.

Quote:

Now AutoQ2 matches 99.999% the wanted filesize
That man is a real marketing king ;)
Good ! Approaches do get even bigger, but 99.999% of wanted FINAL filesize on CQ ARENT possible like Karl pointed to. The reason is that you dont predict ALL peaks and lows, even when using 20% on every prediction turn. Look at my graphs at the beginning of this thread.
But ... you can get closer and closer.

Quote:

Originally Posted by phil
Be carrefull, the two options are not really equivalent : you don't have the same number of I pictures, and they are not generated at the same place.

In other words, you can't tell that Slicer(1,15,1,"ping")+Slicer(1,15,1,"pong") == Slicer(2,15,1, "ping")

Its like "unfold fields horizontal" and "fold fields horizontal back".

The I frames will be the same as thats the purpose of the 15th gop length size. If you set "auto I frame insertion" that that will be linear, but if no auto I frame will be inserted, even then it will stand in a same "propoprtional" relation to the full encoding where also the Iframes will be at diff. positions.

And THATs why I never use Auto-I-Frames when doing OPV predictions.

@ Generally

Tomorrow Ill start tests as I was in Job business till tonight.
But the math and the "real" framecount AND contend on BOTH (1%+1% and 2% scripting) for me is clear. If the "real world" behaves different, then it bases on things like Auto-I-Frame.
Shure u dont have to change that much the encoder source but you need to take twice the time for one turn as you did simply (in math thinking) append.
The pong was just an idea to get more samples by almost the same time for predictionning seen in a whole.


But beside all testings and Credit deserving to people which do develop.
My point to that credit thing of avalon is clear! As he NEVER used own Ideas. If slicer wouldnt have been born he would still use these one slice sampler(xxx,yyy) maths. I knew it that this would end up like this
:lol: :lol: :lol: :lol:


PS:
Quote:

You all did feel yourself bad when I talked with Amnon82.
As you mentioned the "friendship" in here: WE didnt feel bad, as its not on us what youre doing and what you decide ;).
BUT these where 2 cents for YOU! As WE had clear experiences.
And thats a little part of "friendship".
Solamente evitar qué te vas a caer un tiempo en esta situación con avalon. ?Entiendes? ;)

kwag 06-28-2005 07:31 PM

Quote:

Originally Posted by incredible
First I do see that Amnon did put me OUT of the credits, that was shure!

@Prodater,

How about fixing that Luis :?:
As you have contact with Avalon, and we don't, maybe you can ask him why he removed Incredible from the credits, as it's HIS idea :!:

-kwag

Prodater64 06-28-2005 07:36 PM

Quote:

Originally Posted by rds_correia
Well I'll run the test to see how close I can get in encoded filesize.
I must warn you guys, even if it was simple luck, I've reached less than 1% undersize several time with HC in (C)Q mode.
BTW, I was thinking.
I'm used to do 1% Ping Pong predictions so 1st I'll predict the OPPPP with 2% and then I'll try to do it with 1% also.
For that I'll change the slicer function in order to allow me the usage of decimal figures.
Like:
Code:

function slicer(clip input, float percent, int Goplenght, int Gopmulti, string "ping")
Just for testing purposes.
Hope Andrej doesn't mind :).
Cheers

I did try that also, but an error was raised!

Prodater64 06-28-2005 07:42 PM

Quote:

Originally Posted by incredible
Quote:

Now AutoQ2 matches 99.999% the wanted filesize
That man is a real marketing king ;)

Did you read what is the thread name, as I quoted before?
If Amnon82 withdraws your credit line, it is very bad really. I'll talk about that with him.

Edited: I just now PM Amnon82 claiming him that put back Incredible's credits and offered in exchange that withdrown my ones if he think that I don have anyone.
For sure he missinterpreted me and did think that the function is mine. Remember that we, both, don't have english as native language.

incredible 06-28-2005 07:59 PM

LATELY I still do see that the 2pass approach , means the one approch you tried to explain in here isnt the fact of the purpose of that tool there. He is still using CQ mode and a diff. kind of using slicer, based on inspirations of you. He did code, but the ideas do belong to others. --- you see?

And he is still talking of one movie predictions, right? So even that approach is not given keeping same Q on an amount of movies one one target, which was done by using CQ already before.

Now here comes the importand part for you: :arrow: IF its shure that your 1%+1% usage of slicer will give the better results, ... then WHY in the hell did you give it to other coders where you by your own do have enough skills in VB usage for coding that????? You could enhance your own tool without that marketing taste.

Quote:

Edited: I just now PM Amnon82 claiming him that put back Incredible's credits and offered in exchange that withdrown my ones if he think that I don have anyone.
YOU got the idea of 1%+1% ... if it will be proofed that it works much better THEN for shure you got the right to be in there .... much more then him! I dont care if being put in the last line, but eliminating is what teases me.

Quote:

For sure he missinterpreted me and did think that the function is mine. Remember that we, both, don't have english as native language.
He doesnt misinterprete you at all, he gots that good english knowledge, which was enough to force a 20 page long anti MVCD campaign in here!

Prodater64 06-28-2005 08:59 PM

Quote:

Originally Posted by incredible
LATELY I still do see that the 2pass approach , means the one approch you tried to explain in here isnt the fact of the purpose of that tool there. He is still using CQ mode and a diff. kind of using slicer, with a good help of you. He did code, but the ideas do belong to others. --- you see?

You still don't understand me.

1 - OPPPP is not only for 2PassVBR encoding.
2 - With OPPPP he still needs 2 pass, finding 1 sample at given CQ and other at another given CQ. After that he needs only to do maths to obtain final CQ.
3 - With TPPP he needs, at least one ping, one pong, do maths, and another ping, aplly "proportional difference" at this, and do maths again to obtain final CQ
4 - I give him my idea voluntary, he don't forced me in any way.

Quote:

And he is still talking of one movie predictions, right? So even that approach is not given keeping same Q on an amount of movies one one target.
This is another affair.

1 - First I talked with him about my way prediction method that resulted he changed his code.

2 - After that last program version was released, I asked him if he would like his program had a 2 or 3 movies "proportinal space prediction" or if he could be interested in developpe such prediction tool isolated.
Two ideas did seems him good. He is thinking about now.
It liked also that his program would be only (ATM) with that tool.

I would like to remark here that my idea is to 2 Q samples of 2 movies, with let's say CQ=100, they can be with fixed frames number or with a percentage, as doing maths you always can to know ho many KB needs a fixed frames number.
As in those samples, samplesizes are determined by the bitrate assigned by the CQ encoding, if we do aKB/(aKB+bKB) and bKB/(aKB+bKB) we will obtain a proportion based upon bits repart. If you apply then (but you can apply it at same time, I explain it in this way for better understanding) differents movies times, you will obtain final proportions (FP1 and FP2).
Now you only needs to do:

Final Video Size 1 = Mediasize * FP1
Final Video Size 2 = Mediasize * FP2

With this values and your desired audio rate and overhead, you can obtain
an avgBTR.

Finally with all this, a tool as CQMatic could do its known work, but ALSO wichever xPass encoder could encode the streams with the given avgBTR.

This work can be do perfectly with the Boulder's spreadsheet.
Values obtained permits CQ encodings (CQmatic needs only playing time and avgBTR, all other data is given by TMPGEnc project), or xPassVBR encodings (encoders only needs avgBTR, max and min, and also a "Final Video Size"

Quote:

IF its shure that your 1%+1% usage of slicer will give the better results, ... then WHY in the hell did you give it to other coders where you by your own do have enough skills in VB usage for that????? You could enhance your own tool without that marketing taste.
I'm working in my own programs.
This do not avoid I can share knowledge. I liked Amnon82 program and I though I could help him. I also wanted interchange ideas about HC Encoder and its CQmaxbitrate issues.
My programs are mostly as control panels to call useful encoding and another frequent tasks softwares, and also adds batch encoding to some encoding softwares. Now also adds prediction for CQ encoding.
His programs are complete frontends where you can set all variables as in the original one. You can see, programs "are like" but not equals.
Finally, if I can help to develope an useful video encoding tool, I will do it, as you can see in Maurus last tool thread, as I worked with Mencoder and lately with HC Encoder and NuEnc.

Dialhot 06-29-2005 02:06 AM

Quote:

Originally Posted by incredible
The I frames will be the same as thats the purpose of the 15th gop length size. If you set "auto I frame insertion" that that will be linear, but if no auto I frame will be inserted, even then it will stand in a same "propoprtional" relation to the full encoding where also the Iframes will be at diff. positions.

And THATs why I never use Auto-I-Frames when doing OPV predictions.

I was sure someone will tell that. But I hoped he would think a little further. There is not only I frames in a GOP.
Let's talk about P frames : how big is a P frame in a middle of a scene compared to one at a change scene ? What is the proportion of change scene in a 1%+1% compared to 2% ?
Is is the same ? No.
So are the two methods equivalent ? No.

The two methods are equivalent IF no auto-I are introduced AND no P or B frames are used. A not really interresting hypothesis ;)

incredible 06-29-2005 06:54 AM

Verrrrry nice I just typed in here maths and calculations which did take 15 mins - and firefox *bombed* when entering beside kvcd page was open at several other pages on the www :evil:

@ All
The accuratest approch in predictions is if you dont use AutoIframes.
Why?

Cause it gots its sense as slicer and also the sampler() way we know use a chunk length which matches the framecount within a GOP of the final encoding.

Lets say we do allocade a 15th GOP "IPBBPBBPBBPBBPB".

Now every slicelength is set to 15. Based on the used % of the full movie those routines like slicer do take chunks of á 15frames and merge them as ONE bitstream. And as we us a FIXED encoding GOP (non autoIframe) of 15, every I frame will match each chunk at its beginning. So the GopStructures and Lengths will be constant while doing that pass. The I,P,B frames will always be on the same Positions when doing the prediction passes. And as every I,P,B.

Even if I do set to AutoIframe, ... as the content of the slices wont differ when appending a 1%+1% these GOPs also will be constant in their structure


(x) = Slice No.(x)

2%ping = (1)IPBBPBBPBBPBBPB_(2)IPBBPBBPBBPBBPB_(3)IPBBPBBPB BPBBPB_(4)IPBBPBBPBBPBBPB_(5)IPBBPBBPBBPBBPB_(6)IP BBPBBPBBPBBPB_(7)IPBBPBBPBBPBBPB_(8)IPBBPBBPBBPBBP B_(9)IPBBPBBPBBPBBPB_(10)IPBBPBBPBBPBBPB

1%ping = (1)IPBBPBBPBBPBBPB_(3)IPBBPBBPBBPBBPB_(5)IPBBPBBPB BPBBPB_(7)IPBBPBBPBBPBBPB_(9)IPBBPBBPBBPBBPB

1%pong = (2)IPBBPBBPBBPBBPB_(4)IPBBPBBPBBPBBPB_(6)IPBBPBBPB BPBBPB_(8)IPBBPBBPBBPBBPB_(10)IPBBPBBPBBPBBPB

1%ping+1%pong = (1)IPBBPBBPBBPBBPB_(3)IPBBPBBPBBPBBPB_(5)IPBBPBBPB BPBBPB_(7)IPBBPBBPBBPBBPB_(9)IPBBPBBPBBPBBPB_(2)IP BBPBBPBBPBBPB_(4)IPBBPBBPBBPBBPB_(6)IPBBPBBPBBPBBP B_(8)IPBBPBBPBBPBBPB_(10)IPBBPBBPBBPBBPB

means the by appending, the slice alignment has changed but NOT the SlicePixelDataContent (which also is responsable for auto I frames. And even if I would use Auto I frame, THAT slice where in these 15frames of that slice 2 or 3 I frames did result. If appending 1%+1% exactly THAT slice will be shown to the encoder again where again the same 2 or 3 I frames will be allocaded.

See a prediction not in sampling frames, but sampling GOP contents/bits.

Now related to Auto-Gop-Insertion:

If I do encode a whole movie using NO Auto I frames, then I got a continous Gopflow by a content of each GOP of "IPBBPBBPBBPBBPB".

Means in that Movie "I" Frames would cause taking 50% of Bits, "P" Frames 30% and "B" Frames 20% (lets assume these values).

If I also do apply a prediction where I also do use a fixed GOP then also here we do have a relation of 50/30/20 %.

If I do use AutoIframes the proportion of I frames in a range of 15 frames could/will be "out of control" and therefore IF the slicing would touch exactly those scenes where much Schanges do occur, it could happen that the result in kbit of that prediction will be messed up as the relation of added I frames from the prediction to the final full encode could be different! As by this these bitrate peak scenes and their SCs will affect even more the risk of beeing out of target finally.

I made tests on the first version of slicer and already there I did figure out that the possibility to match the final wanted target size is MUCH bigger if AutoIframes are of. Why we are doing predictions? Cause we cant deal with constans in resulted bitcontents and if we now do apply a routine which makes it even more inconstant, then its even harder to predict.

Quote:

What is the proportion of change scene in a 1%+1% compared to 2% ?
Is is the same ? No.
Its not just 1%ping+1%ping as these returned bitstreams are defined by 1%ping+1%pong! Where the pong is exactly in the middle of two pings.
So its the same pixeldatacontent! when comparing to 2%ping ... see my graphs some posts above.

1% ping and 1% pong IS the same as 2% ping.

It doesnt matter "where" the slice including its pixeldata of xKbits does exist in the alignment of slices in the prection stream, then important thing is that it wont differ in its content to the next sliced turn of the same slicing (ping or pong).

Do that on a mpeg2source, open it in Vdub and compare.
The slices are just alignet different, but lentgh of slices is the same and ALSO the pixeldata contents!

Thats the IDEA of slicer that ist NOT needed to predict in 2% passes which need more time, BUT in several 1% passes which just do contain 1 or two pongs for getting a comparison to frames beeing offsetted!

Dialhot 06-29-2005 07:45 AM

Quote:

Originally Posted by incredible
Means in that Movie "I" Frames would cause taking 50% of Bits, "P" Frames 30% and "B" Frames 20% (lets assume these values).

Inc, a P frame can be as big as a I frame at a change scene, and even bigger ! The most scene change you introduce in the middle of a GOP, the more will be the weight of the P frames in the bit consumed.

Now I don't use either of all these methods or any ping pong or pang method, and I don't give a shit about them. Reading carefully your post above and the former one, I think I'd probably badly understood you. Just forget my remark.

Quote:

I made tests on the first version of slicer and already there I did figure out that the possibility to match the final wanted target size is MUCH bigger if AutoIframes are of.
OF COURSE !!! I'm saying that since the very begining of CQMatic ! And is is not a matter of bigger possibility or not : any "sliced" prediction method is a nonsense if I frames are introduced automatically at scene changes.

Note: just to tell why I do not use any "p" method. You all know that even with a 100% accurate sampler method, you can't tell nothing about the size of the final encode. So why do you bother to find such a method ? :lol:

incredible 06-29-2005 08:46 AM

Quote:

Originally Posted by Dialhot
Inc, a P frame can be as big as a I frame at a change scene, and even bigger ! The most scene change you introduce in the middle of a GOP, the more will be the weight of the P frames in the bit consumed.

Well thats cause I told "assume these values" just for explanation - as also here we do have a non-constant.

Quote:

Now I don't use either of all these methods or any ping pong or pang method, and I don't give a sh*t about them. Reading carefully your post above and the former one, I think I'd probably badly understood you. Just forget my remark.
Ahhhh :) I knew that you understood it as 1%ping+1%ping which wasnt the case - shure that makes it confusing.

Quote:

Note: just to tell why I do not use any "p" method. You all know that even with a 100% accurate sampler method, you can't tell nothing about the size of the final encode. So why do you bother to find such a method ? :lol:
@ Generally

Slicer wasnt build to be acurater at the first sight (as that always does base on the % of the movie you do use for prediction IF no constant CQ output encoding is possible). It was build to take in the same number of prediction turns MORE samples due offsetting which MAKES it acurater AND as predictions are NEVER constant (due the encoders lack of constancy while Q encoding) there will be NEVERNEVERNEVRENVER a prediction method which lets the movie result in 99,99 final wanted targetsize, even if you would use nonsenseful 100% when doing a prediction ... the inconstant CQing will not end up 99,99 bit 99,xx in its accuracy.


Quote:

Originally Posted by Luis
2 - With OPPPP he still needs 2 pass, finding 1 sample at given CQ and other at another given CQ. After that he needs only to do maths to obtain final CQ.

Means ...

1%_ping + 1%_pong at CQ 10 = ResultLow
1%_ping + 1%_pong at CQ 90 = ResultHigh

The needed CQ = the average out of ResultLow & ResultHigh and putted afterwards in a rule of three calculation.

The only Problem is ....... :arrow: you do use a math formula wich of course IS CONSTANT ---- but the enemy of that "beeing Constant" is "CQ itself" as its NOT constant. Thats why Karl never did only one Prediction Turn and applied a rule of three.

Such an approach (as I understood) I used to get very close to the near cq WHERE I do begin my real ping-pong-ping-ping-ping-pong prediction.
MEANS: This way is ok to obtain an INITIAL CQ value where the turns do begin but NOT to obtain a final CQ. The "initial" way only makes the prediction taking less time ... but NOT more acurate ;)

But I guess again I didn't understand you in your explanations?

I remember veray good wen we and Karl did approaches on new predictionning thoughts, maybe the first encodings did result verrry acurate and then *boooom* the affection of CQ's nature did its part by beeing non constant and suddelny the big applause changed into silence when a target encoding finally went out of wanted filesize.
So ... IF I would do obtain this evening good results, I would need mooore testsings ... as TIME is the one which does proof things in this case ;)
And on the coming Wednesday Im on holidays at the Lake Garda in Italy.

HC maybe when using these steps gives GIVES good results as it uses an almost constant Q curve in Bitrateviewer which TmpgEnc or CCE dont provide! But that average Q value out of he Q curve will also be never that acurate.
To be most acurate, you would need an encoder using a fixed Quantizer Scale which we know from Libavcodec, but as we do know that the ratecontrol is freaky ... that will also make it not possible doing a rule of three prediction.

incredible 06-29-2005 10:31 AM

Quote:

Originally Posted by Prodater64
Quote:

Originally Posted by incredible
And he is still talking of one movie predictions, right? So even that approach is not given keeping same Q on an amount of movies one one target.

This is another affair.

Seems we're talking about lot affairs in here .... so whats now the subject?

- A CQ based proportional sized avrg bitrate 2pass encoding for keeping a "visuable" constant Quality over several movies one ONE Media or targetspace.

- An approach for getting a CQ based encoding (not 2pass) using slicer in an even more acurate way

- somethign else?

Only that I get a bit updated and for avoiding getting total out of control according to explanations etc.

Quote:

I would like to remark here that my idea is to 2 Q samples of 2 movies, with let's say CQ=100, they can be with fixed frames number or with a percentage, as doing maths you always can to know ho many KB needs a fixed frames number.
As in those samples, samplesizes are determined by the bitrate assigned by the CQ encoding, ...........
With this values and your desired audio rate and overhead, you can obtain
an avgBTR.
So in easy words ....

- Use slicing on i.e. 2 diff. MovieSources of 2% each (no matter what slicing way).

- do encode the first using i.e. CQ 60
- do encode the second using also CQ 60

Do calculate the proportional difference of both outputs.
Lets say the first one needs 20% more of space at the same CQ.

So we do assign out of the final caluclated avg bitrate needed for both for matching the target media 20% more to the first Movie? and in proportional less XX% to the second one. This keeps the "total needed avg bitrate of BOTH" but each gots its bits assignet individually.

Quote:

This work can be do perfectly with the Boulder's spreadsheet.
Values obtained permits CQ encodings (CQmatic needs only playing time and avgBTR, all other data is given by TMPGEnc project), or xPassVBR encodings (encoders only needs avgBTR, max and min, and also a "Final Video Size"
Link to that spreadsheet please? I know it has been posted somewhere in here but ... ough

Prodater64 06-29-2005 02:28 PM

Here is the speadsheet:

http://www.saunalahti.fi/sam08/kbps_eng.xls

http://www.kvcd.net/forum/viewtopic.php?p=117425#117425



Addendum: @Inc: Once I explained Ammon82, that OPPPP is my idea but the function, and the script also (as you corrected my one) are yours, your credits are again in the "about page" of AutoQ2. Also my ones.


All times are GMT -5. The time now is 03:33 AM  —  vBulletin © Jelsoft Enterprises Ltd

Site design, images and content © 2002-2024 The Digital FAQ, www.digitalFAQ.com
Forum Software by vBulletin · Copyright © 2024 Jelsoft Enterprises Ltd.