Time to revise file prediction!
I'm getting inconsistent results lately. There are too many new factors involved. AviSynth 2.52, new motion adaptive script, etc. I think it's time for a revision :!:
-kwag |
I didn't try it since avs252 but the method used by DVD2SVCD always gave me good result even in PAL (where Tok is very bad :-().
The idea of DVD2SVCD is to always make a sample of 100 seconds, taking 1 second every 1/100th of the movie length (in minute) (different from sampler.dll that for instance take only 80 samples when the film is 1h20min). |
I tried even manual prediction, and the final size is off :!:
I've done it twice with "K-Pax", using the optimal 2.52 script. The basic rule on KVCD prediction is to take 1 second for every minute of the movie. It has been like this, even before DVD2SVCD did it that way :!: This is now failing, and is not as accurate as it was when static filters were used, and AviSynth 2.0x. Precious accuracy was ~2% ALWAYS. Not anymore :cry: I'm now off by +5% in this case :!: So I'm revising this stuff now :twisted: -kwag |
cool :D
|
Kwag wrote:
"even before DVD2SVCD did it that way" yes, i remember and you post this in doom9! http://forum.doom9.org/showthread.ph...highlight=kwag :wink: |
Geee! thanks jorel :)
I forgot about that post 8) -kwag |
kwag, I totally agree. I'm getting completely incosistent results too. I'm currently stuck on this one movie that is consistently ending up at 830mb (m1v) with ToK and manual prediction too8O
|
Hi, I mentioned in a different thread that the following steps work well for me.
Start TOK and set factor to 1.00 Let TOK encode the Full and Small predict size. Cancel prediction. Find out the ratio of Full to Small size and divide this by 10.5 Then set the factor to the result. Restart TOK and let it run. Regards, Tenra |
hi Tenra,
can I do this manually like this (psuedocode): clip(a) = SampleClip() 'full sample clip(b) = SampleClip(10) '10% of full sample factor = (size clip(a) / size clip(b) ) / 10.5 Then use this factor in the Predict function? Thanks, ren |
Hi Ren,
Quote:
Quote:
Regards, Tenra |
Script updated. Hopefully, the root of the problem was nailed :!:
Read the change log here: http://www.kvcd.net/forum/viewtopic.php?t=3483 -kwag |
And take the hopefully with a grain of salt :!:
Although I definitely saw the hard shift with -2 on some scenes, which I don't see at all with -1, I just did a small 2 minute clip with -1 and another with -2, and the file size was the same :x I guess I'll have to encode a longer clip to see if the effect does reduce the file size. If not, back to the drawing board :!: Anyway, the -1 does fix the shift on the frame after scene change. -kwag |
@ Kwag..
W/ regards to K-PAX... You said that in your past "predictions" it was on target ?? But, today, that same Movie is NO longer on target ?? * using the same method you used when it did work ?? -vhelp |
Hi vhelp,
The prediction is again chaotic. The new AviSynth 2.52, the MA script, too many factors. I'm trying to zero in on the problems :!: I did "K-Pax", and the file size was ~5% over the wanted size :x Not that it would make visual difference after lowering the CQ a tad, but the problem is we're going "above" final wanted file size. Even with manual prediction, it's not working correctly either :!: I assume this will be the case with any program currently doing prediction and using the MA script. I bet even DVD2SVCD will be thrown off calculations with the MA script :!: So if anyone want's correct prediction, go back to AviSynth 2.08, and use the old script. I won't go back to 2.08 :!:, NO WAY :!: I'll just keep working on this until I (or someone!) find a consistent method to correct the chaos :!: -kwag |
Hi Kwag..
I can save you all the trouble. I've ben toying as well as pulling my hair out on (my own set of..) issues too. And, I believe I have the answer :!: Hint, hint... And, don't get mad, but does "color washout" mean anything to you ?? ?? Yes, the color IS off. And, THAT'S why the "predicton" is OFF !! Ok, FIRST, because there are so many Filters that we use in AVIsynth for the MA script, (and things change fast w/ filter used or not) the issue is they are ALL NOT of equal color space. There are too many. Ok, let met explain.. Some filters require YUV, others YUV2 and others YV12 and so on.. .. .. That alone is enough to throw off our (YOUR) previous DVD movie that USED to be on target in past testings. So, the only way we can have a successful "prediction", is to have those that created ALL the necessary AVIsynth Filters "separately" to: * YV12 * YUV * YUY2 * RGB * etc. Then, we have no trouble w/ any, becuase say, out source is from a capture card, then obviously, we could use ALL the filters that ONLY pertain to YUY2 color space. If our source is DVD rip, then obviously, we could use ALL the filters that ONLY pertain to YV12 color space. THEN, and only THEN, will we have accurate "predictions" for the same source, every time.. .. mix your source's color space (ie, YUY2 w/ YV12) and your "prediction" is off) It's a matter of fine-tuning otherwise, but I doubt that fine-tuning will help w/ mixed color space :!: ..too un-"predictable" What are you comments on this :?: Now.. I've ben toying w/ the idea of using an OLDER mpeg2dec.dll w/ the help of this script: Code:
## v2.52 DLL Filters go here ## I get two "color space" quality differece when I REM ## it out vs. not REM ## it out. ..try it out for yourself. . . OR, . . if you your Code:
#asharp(2, 4) pink, and other side w/ normal video. So, it looks like asharp is causing this issue. Must be another color space 8O More to say, that I've noticed in the past week on issues w/ latest AVIsynth v2.52 but i don't want to get into any debates.. but it looks like too many cooks has spoiled this broath for now. But, for the time being, we can try and figure out the bugs and work around them :idea: With the help of ALL. :) Be good :) -vhelp |
I dont think they even had to go back to Asynth 2.08 to me they can keep using 2.52 n just check off the Adaptive script line. This is what trowing
off prediction because tok has now way to now how long is it going to blur on high motion scenes i keep being of below the target 20~40 Mb even with the old prediction method; BTW not only prediction is off but the time too encode is less than it says, im getting 5 hrs ecodes in 4-1/2, so there you go i guess Tmpeg encode faster in high motion scenes cause of less detail that doen't have to encode, does that make any sence :?: it kind of does to me. Anyway, im not giving up on the adaptive script, n i dont think anybody should, i think is to early for that because every problem has an anwer. :) |
Is 15MB short on a 2CD encode considered accurate? Or is this off? I'm thinking about trying again, but I don't know if 15MB will give me that much more quality to make the re-encode worth it.
|
Oh Kwag..
"prediction" vs. "color space"... Something lese came to mind on "prediction" vs. "color space" In your prevous v2.08 "predictons" w/ Movie "K-PAX", you got a certain rate.. well, have a look below, in hopes to better explain what is truley going on. Have a look at this "analigy" below: A) --> YUY2 <-- * K-Pax * prediction: 7.8 B) --> YV12 <-- * K-Pax * prediction: 9.2 C) --> MIXED YUY2 and YV12 etc Filters <-- * K-Pax * prediction: 11.6 Note 1.. If you ran "prediction" separately on A) and B) you would get two different predicitons. Note 2.. If you were to do same w/ C) don't expect to get the same predict matching A) or even B) Note 3.. Now, if you mix different Filters that have different "color space" requirements ie, C) and say, you got asharp() YUY2 last month, and then a new one (revised) comes along but this time in YV12, you will get an even different predict rate because you stired the pot some.. ie, C) hehe.. @ ovg64.. Note 4 w/ respect to ovg64.. Even though that may be true, you'll still have prediction problems BECAUSE OF THE mixing of color space filters. You have to have consistant color space filters in a project ! Either you have YV12 throughout your project, or YUY2. It would probably be easier to have those associated w/ the creation of their filters to make separate ones.. At very least, have: * YUY2, and * YV12 ..filters . . then to figure out how to work around it, but on the other hand, it's harder to talk to the majority of those peoples, to get them to revise their Filters. Member prev. how it was hard to get people to change for AVIsynth 2.5x ?? Anyways.. . . Then, later, during the process, if user needs to have a final color space conversion, then can use the convertToYV12 etc, at the end of their scripts, and proceed to encode w/ happyness :) I firmly believe that for serious quality freaks (like myself) that want the best in their process, should only be using one color space. Mixing is just an excuse ta get the job done. That should only be if "quality" is not your goal. I hate this band-aid method of mixing if it taints the final quality. It's VERY crude, and causes us geeks countless waist of debuggin even the simplist of encodes when all it really is is just the color space - - that simple. It's always the outsider (me) who has to wake others up to the reality of chaos, and when enough is awake, and stirs up the pot, THATs when things get done! AAARRRR !! Last note.. I'm starved !! Oh, right.. as I was saying.. the above predict numbers I made up. Don't take them literally. -vhelp |
@Vhelp,
If your theory is true than all you would need to do is make a script w say YV12 filters only run prediction and Encode The final file size should now be on target like in 2.08, that would pretty much prove your theory, i think. |
@ ovg64..
Answer.. NO. and, I'll explain in a moment. -vhelp |
@vhelp
I just did a search of the avisynth.org website and every filter in kwag's script supports the YV12 colorspace. Some support YU and YV, but I don't know if you can force it to one or the other. I confirmed Limiter() supports YV12 also. |
@vhelp,
The problem is not related at all to colorspace. If it was a color space problem, then it would also show up in the samples when doing prediction, because prediction is done on the same color space as the final encode. I think the solution, for the time being, is going to be to sample at least 36 or 48 frames per sample instead of 24, and then take ~100 samples per movie ( just like in the original file prediction, a long time ago ). This will increase the prediction time a little, but it will broaden the window of the sampler. If this doesn't work, hell 8O, then we're back to square one :!: -kwag |
Quote:
That's an excelent prediction, even for one CD 8) -kwag |
Quote:
There's no difference at all :!: -kwag |
@ ovg64..
Quote:
Hers's why: In the "prediction" of things, somewhere in the process, VIDEO is being encoded.. right ?? So, when it's encoding the video.. * Ain't it encoding it w/ all those MIXed filters ?? * And, are they the same today, as they were yesterday ?? * Did any ONE of them change ?? Ok, given the above, let say that a user encoded the Movie, K-PAX and it had certain Filters in the script. If EVERY one of those filters were of the SAME color space used in a prevous predict ie, Code:
asharp - YUY2 . . and the predict encoded those samples using the above filters, w/ the YUY2 color space, and user got a 5.0 value, THEN later on, you upgrated a few things, and now you proceed to do the same predict on the same Movie, K-PAX.. Code:
asharp - YV12 * However, you can't go blaing only the filters. There may be other things at play here. Example, v2.52 now has YV12 support. Some things may have changed, that may not be known until things like this come up and they are discovered. Somehow, the conversion of color space is takine place, and as such, something has changed, resulint most likly, "filezie prediciton" Don't forget, that COLOR (no matter how small) plays a part in the encoding. And, that means: * filesize * CQ values * quality * you name it, . ..will be effected down the chain. Mind you all, I'm basing this on the assumption that the predict is somewhere in the process, doing the actual encoding of video, if I remember correctly, that's what it's doing, and that is how it's getting the accurate "filesize predicton". If I'm wrong on this assumption, then I may be off (or wrong) on my analigies. @ Kwag.. * Are you sure that color space is not effected (touched) in those samples ?? * can you run an AVIsynth v2.08 YUY2 predict on the same source, w/ the ...same scenes and frames, and then do anther one w/ AVIsynth v2.52 YV12 and then compare the results ?? I ask, becaues something IS effecting predicts, and after all, video is being encoded, and THAT encode is what is used in the final values you get in your predicts !! I would test this out for myself, but I don't know how to use predict 8O how ironic!! dahh!! -vhelp |
Here's what I'm currently trying, at the end of the script:
Code:
interval = round( ((FrameCount/24)/60) / 2 ) |
I still think the problem is that tok can't read how many times
or how much blur there is going to be on a hole movie, and if asharp(-2,0) was padding file size than that didn't help. So now taking more samples like kwag said may help tok be more accurate but may still not solve the deal here. :? |
Quote:
There are more ways to skin a cat, and we will skin the cat :mrgreen: -kwag |
@ Kwag..
:roll: Let me ask ya a stupid question .. Manual Prediction.. NOThing gets encoded.. and you are just jugling a few numbers around ?? Or, is there SOMEthing getting encoded, and based off the filesize of those encoded snips, you tally those numbers and predict the final CQ ?? :lol: -vhelp |
Quote:
Quote:
And to all :!: I think I (probably!) hit the nail this time :!: It seems that with the MA script, what's throwing us off is the GOP :!: :!: :!: I'm currently testing prediction with the regular formula: Code:
interval = round( (FrameCount/24)/60 ) Here's a sample done with the above formula, and now with a final CQ of 63.66, which is about 3 points below in CQ value from my original encode that was ~5% over in final file size: http://www.kvcd.net/10-sec.mpg We'll have to wait ~4 1/2 hours for the result :!: I'll post here the final file size when the complete movie finishes. If this does indeed correct the problem, then we have to talk to hedix to make the GOP changes in ToK. -kwag |
@ Kwag..
Quote:
By the way, I'm currently trying out ToK (about time i'd say) I guess this discussion motivated me enough to give it a go :) But, having my own problems. It encodes first pass, next pass gives me the error: Code:
Evaluate: division by zero -vhelp |
Quote:
If you have the "Sampler()", remove it :!: -kwag |
@ Kwag..
Quote:
Does this help any ? Thanks, -vhelp |
kwag does the solution you propose and are testing lower the quality since the CQ went down and the GOP is smaller? Sorry I'm just trying to understand all this. You guys come up with some great stuff here that is above my head, but I like to at least try and understand how things work.
Happy weekend :D |
Quote:
-kwag |
looks pretty dam near perfect to me 8O
|
Yeah, the only thing is to wait and see if the final size is ~718,815KB, which is the size calculated by MovieStacker. Wanted file size for prediction is 11,980.25, and with the CQ of 63.66, my sampler was 11,958KB.
So we have to wait :!: -kwag |
Quote:
You mean tok n if so how do you put the line into it :?: Do you edit the ini file or something :?: Quote:
|
Hi ovg64,
I'm predicting the old manual way. After I open the .d2v with MovieStacker, if you look on the "Audio Stream & Authoring" tab, you'll see the video size in KB and in MB. So you take the KB size and divide it by 60, and that's the sampler wanted size :) Then I add the lines: Code:
interval = round( (FrameCount/24)/60 ) -kwag |
Isn't it easy using the old kvcd predictor :lol: i still use it w manual prediction.
|
Site design, images and content © 2002-2024 The Digital FAQ, www.digitalFAQ.com
Forum Software by vBulletin · Copyright © 2024 Jelsoft Enterprises Ltd.