digitalFAQ.com Forums [Archives]

digitalFAQ.com Forums [Archives] (http://www.digitalfaq.com/archives/)
-   Avisynth Scripting (http://www.digitalfaq.com/archives/avisynth/)
-   -   Experimental Revised prediction! (http://www.digitalfaq.com/archives/avisynth/4139-experimental-revised-prediction.html)

audioslave 06-25-2003 11:11 AM

Okay, I'll try it out. Thanks for you help, Boulder. BTW, should I use 24 for sample lenght also - on all source material (NTSCFilm, PAL & NTSC)?

Boulder 06-25-2003 11:19 AM

Yes, it would make sense..until someone comes up with a sample length that gives a better prediction.

audioslave 06-25-2003 11:22 AM

Thank you again, Boulder! You've been a great helper indeed...
:wink:

J-Wo 06-25-2003 01:21 PM

okay, with all the latest talk on motion adaptive scripts and detecting scene changes, blah blah blah... got me thinking, should we be making any more changes to our tmpgenc settings than those in the old kvcdx3 template? I've read to change the motion search precision to fast (motion estimate search). But what about Detect Scene change (in the GOP tab) and the Special Settings (Quantize Matrix tab)? I just noticd there is an unchecked box for "No motion search for still picture part by half pixel"... Thanks guys!

kwag 06-25-2003 01:35 PM

Quote:

Originally Posted by J-Wo
But what about Detect Scene change (in the GOP tab) and the Special Settings (Quantize Matrix tab)?

That was already changed and SET in the http://www.kvcd.net/video.en1 file, ready for ToK.
Quote:

I just noticd there is an unchecked box for "No motion search for still picture part by half pixel"
That actually makes the picture worse :wink:

-kwag

jorel 06-25-2003 04:02 PM

Quote:

Originally Posted by Boulder
Yep, that's what I'd recommend.

right Boulder :!:

for the first time from days this is the first time
that i got high CQ using the Boulder recomendations :!:
maybe somebody remember a weeks ago when i post about low CQ.

i did the test twice and ToK give me
CQFinal CQ: 70,349 :D :D :D
and before was CQ: 56,467 :arrow: huge difference,don't?

i only uncheck the "Auto GOP" and wrote:
1,24,2,1,24!
uncheck the "new faster prediction" and 2nd group of passes
than i wrote this values:
precision% 1,00 and speed up by 10 !

new video.en1 (thanks Kwag) only changing this line to:
Job.MPEG.Video.VideoEncodeMode=MPEGVideoEncoder_Vi deoEncodeMode_interlace
cos my source is 29,970,ntsc.

the CQ: 70,349 is perfect for my desired size now!
:D
i'm happy and encoding now!

thank you all friends, in special Kwag and Boulder for this result!
:D :D :D

audi2honda 06-25-2003 04:21 PM

should NTSC film users be unchecking auto GOP too? and manually entering a value?

Boulder 06-25-2003 04:28 PM

Quote:

Originally Posted by audi2honda
should NTSC film users be unchecking auto GOP too? and manually entering a value?

For film material the auto GOP length should be 24 already as its 23.976fps.

vico1 06-26-2003 02:31 AM

Quote:

Originally Posted by audioslave
@Boulder
Okay. So do you recommend using a GOP of 24 regardless of the source - even for PAL? I'm confused here...

Quote:

Originally Posted by boulder
Yep, that's what I'd recommend.

@Boulder,
You wouldn`t use that GOP (24) with Anime/Cartoon...would you???


*******************************
The Devil`s always.....in the Details!

Krassi 06-26-2003 03:10 AM

Quote:

Originally Posted by Krassi
The Prediction with Sampler length=15 was good :!:
Can any PAL user try this:
Code:

1. Use ToK 0.0.5.2
2. Overwrite file video.en1 in ToK's opt folder with Kwags newest one
3. Set a factor of 1.0 in ToK and Length of Sample to 15. Samples/Minute to 4 or above.
4. Precision to 0.5 or 1%

My results:
Code:

Required Video Size: 330.369.920
Final Encoded Size: 324.024.428

I'll test some other lengths of Samplers tomorrow...
Good night,

Krassi

After some more tests a sampler length of 15 isn't accurate for me :evil: .
I had to reinstall my Windows (even manual repair failed) so i've only done 2 more tests, but i had an idea:
Why not taking a longer sample size to make a better prediction with the adaptive script :?: I had good results with a Sampler length of 50, maybe 48 would be better. I thought that a bigger sample size than GOP would screw up Prediction or Sampler() but it didn't.

Kwag, do you know why TMPGEnc encodes with a GOP of 24 instead of 25 with PAL-Sources :?:

EDIT: Here is a comparison on a short movie (PAL):
Code:

Sampler Length=48:
Required Video Size: 330,369,920
Final Encoded Size: 324,282,527
______________________________
Sampler Length=50:
Required Video Size: 330,369,920
Final Encoded Size: 327,576,154


audioslave 06-26-2003 03:18 AM

I just finished encoding "Enough" (PAL) with GOP and sample size set to 24. And this is what I got:

Wanted size: 726 XXX XXX
Encoded size: 721 XXX XXX
(Don't have the exact numbers. I'm at work now)

AMAZING! A difference less than ~0.7% :D !!! Final muxed movie was 795MB. I changed ToK's setting to what kwag recommended in an earlier post and used kwag's modified video.en1 file. Try it out all you PAL encoders out there. Thank you very much guys! Maybe I'll be able to sleep at night again now :wink: !

Abond 06-26-2003 04:57 AM

Hi, all
My way for PAL prediction is not fully tested (only two movies so far - an AVI and a DVD) but gave me for first time correct file sizes.
Sample=1% of movie with the line in the script SelectRangeEvery(2500,25)
run a sample at CQ100 without black borders. Mark the file size. Run the sample at CQ100 borders added. Mark the file size. Find the difference of the file sizes in %.
Calculation of desired sample size
samplesize=((fullvidoesize/totalframes)*sampleframes) + (% of the difference you found before). Hmm, sorry for the bad english.
An example:
A movie has 137145 frames. Final video size is 740912KB. Sample has 1375 frames. Sample size = (740912/137145)*1375 = 7428.
CQ100 without borders = 15275 KB
CQ100 with borders = 14569 KB
Difference = 4.6%
4.6% from 7428 = 341
7428+341=7769 -> this is your real size of the sample.
The results. AVI - required 730xxx KB, final encode 723xxxKB.
DVD (the example above) encoded with CQ57 with sample size 7785 (a bit higher than the desired 7769). Required size 740912, final encode 742xxx (a bit higher).
Well, it need to be more carefully tested.

Boulder 06-26-2003 05:41 AM

Quote:

Originally Posted by vico1
Quote:

Originally Posted by audioslave
@Boulder
Okay. So do you recommend using a GOP of 24 regardless of the source - even for PAL? I'm confused here...

Quote:

Originally Posted by boulder
Yep, that's what I'd recommend.

@Boulder,
You wouldn`t use that GOP (24) with Anime/Cartoon...would you???

I can't say because I don't encode any anime or cartoons. Surely Kwag knows this stuff better than me so you're better off with his recommendations.

kwag 06-26-2003 09:17 AM

Quote:

Originally Posted by Krassi

Kwag, do you know why TMPGEnc encodes with a GOP of 24 instead of 25 with PAL-Sources :?:

No, becauese I don't encode PAL :)
But I'll take a look :!: Maybe it's a bug in TMPEG :?:
BTW, have you tried encoding with ToK with the screenshot I posted earlier :?:
Here it is again:

http://www.digitalfaq.com/archives/i.../2003/06/7.jpg

-kwag

kwag 06-26-2003 08:20 PM

Well I just have to post this one :mrgreen:

Here's my encode of K19, which just finished 5 minutes ago. Look at the log, and see for yourself :D

Code:

=============================================================
ToK Log: F:\k19.avs
=============================================================
 
Resolution (fps):528x480 (23.976 fps)
Total Frames: 198673
Total Time  : 02:18:06
-------------------------------------------------------------

Audio Size: 116,004,000
Required Video Size: 703,196,000
 
Factor: 59.940
Desired Sample Size: 11,731,665
 
-------------------------------------------------------------
Prediction: 1st Group of Passes
-------------------------------------------------------------
 
Next CQ: 60.000. Sample Size: 11,224,219
Next CQ: 62.713. Sample Size: 11,510,670
Next CQ: 69.538. Sample Size: 12,969,194
Next CQ: 64.936. Sample Size: 12,690,139
Next CQ: 63.477. Sample Size: 12,087,731
Next CQ: 63.050. Sample Size: 11,783,381

Exit Condition: 0.500 % reached ! yahoo !
Tries  : 6
 

Final CQ: 63.050
Total Time For Predicition: 00:39:03

 
-------------------------------------------------------------
Encoding F:\k19.avs
-------------------------------------------------------------
 
Encoding... CQ : 63.050
Final Encoded Size: 700,419,197
Total Time (all operations): 05:44:29
 
Finished

A difference of 0.39% for a movie that is 138 minutes long :mrgreen: :mrgreen: :mrgreen:
Thanks again to Jellygoose for suggesting to change asharp to unfilter on the adaptive part of the script.
I'm updating the current script now. BTW, the settings I used for this movie are exactly as the screenshot above this post :!:

I'm uptating the

bigggt 06-26-2003 08:35 PM

Thats great Kwag :D ,just a quick question with the new change in the script do you think we will be able to go back to the faster prediction in tok or still use the longer one.

Thanx

ovg64 06-26-2003 09:16 PM

Hey Bigg, I just tried Kwags settings vs tenras Fast prediction

Quote:

kwags = 70.70% CQ
tenras = 70.61% CQ

pretty much close dont you think :idea:

kwag 06-26-2003 09:17 PM

Quote:

Originally Posted by bigggt
Thats great Kwag :D ,just a quick question with the new change in the script do you think we will be able to go back to the faster prediction in tok or still use the longer one.

Thanx

I'm doing a fast "Tenra's" CQ prediction right now to see what CQ it finds, and compare that to the CQ I got with the full prediction :)
I'll post the resulting CQ here in a while...

-kwag

bigggt 06-26-2003 09:23 PM

Yes ovg64,i think we all could live with that :lol:

Looks like it might be a good night :D

kwag 06-26-2003 09:44 PM

Quote:

Originally Posted by bigggt
Yes ovg64,i think we all could live with that :lol:

Looks like it might be a good night :D

YES IT IS :!: I'M GOING TO BUY SOME BEER :!: :mrgreen:

Code:

=============================================================
ToK Log: F:\k19.avs
=============================================================
 
Resolution (fps):528x480 (23.976 fps)
Total Frames: 198673
Total Time  : 02:18:06
-------------------------------------------------------------
 
Audio Size: 116,004,000
Required Video Size: 703,196,000
 
Factor: 59.940
Desired Sample Size: 11,731,665
 
-------------------------------------------------------------
New Faster Prediction
-------------------------------------------------------------
 
Full Sample
Next CQ: 60.000. Sample Size: 11,224,219
Small Sample
Next CQ: 60.000. Sample Size: 1,038,416
Predicting...
Next CQ: 62.713. Sample Size: 11,465,075
Next CQ: 69.590. Sample Size: 12,583,578
Next CQ: 65.251. Sample Size: 12,401,976
Next CQ: 63.709. Sample Size: 12,001,579
Next CQ: 63.209. Sample Size: 11,747,168

Exit Condition: 0.500 % reached ! yahoo !
Tries  : 6
 

Final CQ: 63.209
Total Time For Predicition: 00:11:21

The script is now "FREEZED", and file prediction is again considered STABLE :!: :mrgreen:

-kwag

ovg64 06-26-2003 09:47 PM

I guess we skin the pig ehhh :mrgreen:

Edit: you all did :!: :mrgreen:

audi2honda 06-26-2003 10:13 PM

[quote="kwag"]
Quote:

Originally Posted by bigggt
Yes ovg64,i think we all could live with that :lol:

YES IT IS :!: I'M GOING TO BUY SOME BEER :!: :mrgreen:

The script is now "FREEZED", and file prediction is again considered STABLE :!: :mrgreen:

-kwag

It's Miller Time :!: woot. Starting to run my tests and I can't wait to see the results. :D :D :D

kwag 06-26-2003 10:27 PM

Here you go guys :)
Here's the small sampler of K19 (~1MB), so that's how the whole 138 minutes looks like on one CD-R :mrgreen:

http://www.kvcd.net/k19-fast-predict...-cq-63.209.mpg

Enjoy :)
-kwag

DKruskie 06-26-2003 11:12 PM

So do we continue using tok 0.0.5.2 or can we go back to 0.0.5.3 and what settings should we use?



David

kwag 06-27-2003 12:16 AM

Quote:

Originally Posted by DKruskie
So do we continue using tok 0.0.5.2 or can we go back to 0.0.5.3 and what settings should we use?



David

Hi David,

I havent tried that yet :!:
I'll give that a shot too, and see if the CQ matches a very close CQ as 0.0.5.2.
If it does, then we can use 0.0.5.3 :)
I'll run both predictions. The full, and Tenra's "Fast" prediction.
Only if both match, I would then use 0.0.5.3.

-kwag

kwag 06-27-2003 07:11 AM

Here's the result with ToK 0.0.5.3, which uses "SelectRangeEvery()" instead of "Sampler()" for prediction.
This was on my "K-Pax" movie.

CQ found using "Tenra's" Fast prediction: 70.138
CQ found using normal (long) prediction: 69.940

The CQ found by ToK 0.0.5.2 was 65.51 using Tenra's "Fast" mode, and the full prediction was almost the same number. And that is the correct CQ, because I left the movie encoding last night, and the result this morning was +1.73% accurate :)

This proves that the "SelectRangeEvery()" is a problem for prediction, but SansGrip's method of selecting frame range is excelent :)
So I would stick to 0.0.5.2. Other than the file prediction method used, there's no difference from 0.0.5.2 and 0.0.5.3

-kwag

ovg64 06-27-2003 07:55 AM

Hi Guys, last night i encodes (A guy thing) 1:41 min 70.61% CQ on one CD output video was 710MB, audio was 83MB for a total file size of 793MB.
This was using tok 5.2 n tenras fast prediction, so i guess this turkey is really cooked now. :D

kwag 06-27-2003 08:42 AM

Hi ovg64,

What was your % accuracy on that encode :?:

-kwag

jorel 06-27-2003 08:58 AM

and please Osvaldo,
write all details,..if is mpge1 or 2,the "resize",time of the source,
everything cos i loose the high prediction again,
then i need of all informations.
my CQ back to 56 "house" again.

ovg64 06-27-2003 09:36 AM

Hey Guys, here you go; :D

Mpeg 1
712 mb video wanted 710 obtain
83 mb audio at 112 kbps
1, 24, 2, 1, 24 gop

Fast Prediction *
Factor 1.00
Sample Auto *
# sample/min. 1

1 Group Pass
presicion % 0.50
speed up by 10
10:00 % presicion

80min. CD -1


Quote:

ToK Log: C:\Documents and Settings\Osvaldo\Desktop\Movie\Script.avs
================================================== ===========

Resolution (fps):528x480 (23.976 fps)
Total Frames: 145985
Total Time : 01:41:29
-------------------------------------------------------------

Audio Size: 85,246,000
Required Video Size: 733,954,000

Factor: 59.940
Desired Sample Size: 12,244,811

-------------------------------------------------------------
New Faster Prediction
-------------------------------------------------------------

Full Sample
Next CQ: 70.000. Sample Size: 11,254,886
Small Sample
Next CQ: 70.000. Sample Size: 939,425
Predicting...
Next CQ: 76.157. Sample Size: 14,983,732
Next CQ: 72.356. Sample Size: 14,156,339
Next CQ: 70.991. Sample Size: 12,825,089
Next CQ: 70.560. Sample Size: 12,167,126
Next CQ: 70.693. Sample Size: 12,403,408
Next CQ: 70.615. Sample Size: 12,255,363

Exit Condition: 0.500 % reached ! yahoo !
Tries : 7


Final CQ: 70.615
Total Time For Predicition: 00:10:34

Total Time (all operations): 00:10:34


J-Wo 06-27-2003 09:42 AM

ovg64: That's great that you got such an accurate prediction! I've been wondering lately at what point should we predict using 2 samples/min instead of the default 1? I've pretty much been raising it to 2 whenever the movie is shorter than 2:00:00 long, but I see this one is 1:40:00. Any opinions?

ovg64 06-27-2003 09:54 AM

Quote:

Originally Posted by J-Wo
ovg64: That's great that you got such an accurate prediction! I've been wondering lately at what point should we predict using 2 samples/min instead of the default 1? I've pretty much been raising it to 2 whenever the movie is shorter than 2:00:00 long, but I see this one is 1:40:00. Any opinions?

I dont know but it sims like sample() is more accurate than selectevery()
use in tok 5.3 at least in Avs 2.52, cause yesterday i use 3 setting mine, kwags n Tenras and they all came to around the same 70% CQ for this same movie. :)

jorel 06-27-2003 10:43 AM

really strange Osvaldo.
my source:

Resolution (fps):480x480 (29,970 fps)
Total Frames: 159173
Total Time : 01:28:31

Audio Size: 84.967.000
Required Video Size: 734.224.000

:arrow: and my best CQ56,255!

why i with: less resize, less time movie got less CQ :?:
:? :? :? :?

i need a BIG help :!:
:(

ovg64 06-27-2003 10:56 AM

Quote:

Originally Posted by jorel
really strange Osvaldo.
my source:

Resolution (fps):480x480 (29,970 fps)
Total Frames: 159173
Total Time : 01:28:31

Audio Size: 84.967.000
Required Video Size: 734.224.000

:arrow: and my best CQ56,255!

why i with: less resize, less time movie got less CQ :?:
:? :? :? :?

i need a BIG help :!:
:(

What are you ecoding :?: National Security :?: :D 1:28:31 time
post your script i think maybe one of your filter might be doing
this are u using DGbob :?: you are using 29fps that might be another
reason. :!:

jorel 06-27-2003 11:18 AM

no Osvaldo,no DGBob no more i cahnge to fielddeinterlace!

see my script:

MaxTreshold=1.50
nf=0 # Current frame.

Mpeg2Source("D:\La Luna\Segment2\DVD2AVI_PROJECT_FILE.d2v")

FieldDeinterlace()
undot()
Limiter()
asharp(2,4)
GripCrop(480,480,overscan=1,source_anamorphic=fals e)
GripSize(resizer="BicubicResize")
STMedianFilter(8,32,0,0)
MergeChroma(blur(MaxTreshold))
MergeLuma(blur(0.2))
ScriptClip("nf=YDifferenceToNext()"+chr(13)+"nf>2. 5?\
unfilter(-(fmin(round((nf/0.5)),100)),-(fmin(round((nf/0.5)),100))):\
TemporalSoften(2,7,7,3,2)")

GripBorders()
Limiter()

function fmin(float f1,float f2){return(f1<f2)?f1:f2}

thanks in adbvance!
:wink:

ps:
Jell is helping me too in the "best scripts" thread...
of course,something is wrong in my system and
i don't know what is ?!?!?
:?

Jellygoose 06-27-2003 11:37 AM

@jorel

Look at the Optimal Scripts forum... :wink:

ovg64 06-27-2003 02:55 PM

Hey Jo your script looks ok to me, i dont know maybe tok doesn't pred. well with 29 fps. Maybe you want to experiment a bit i would maybe put undot after the resize n take out the deinterlance filter just to see what kind of CQ you get or maybe try another deinterlance like tomsmocomp
for 2.52 and experiment a bit you know im not a brain like you but i some how solve my problems :wink:

Krassi 06-27-2003 03:05 PM

Quote:

Originally Posted by kwag
This proves that the "SelectRangeEvery()" is a problem for prediction, but SansGrip's method of selecting frame range is excelent :)
So I would stick to 0.0.5.2. Other than the file prediction method used, there's no difference from 0.0.5.2 and 0.0.5.3
-kwag

I can confirm this for PAL-Sources. Prediction is very accurate for PAL using ToK 0.0.5.2, a Sampler Length of 24 and the newest adaptive script, even with Tenras new prediction. The prediction was 0,3 % accurate on a middle action movie.
Thanks to Kwag, Jellygoose and all others for helping figure this out.
Maybe we should contact Pegasys-Inc. to determine if theres a bug in PAL-GOP encoding :roll: .

vico1 06-27-2003 03:44 PM

Quote:

Originally Posted by kwag
This proves that the "SelectRangeEvery()" is a problem for prediction, but SansGrip's method of selecting frame range is excelent :)
So I would stick to 0.0.5.2. Other than the file prediction method used, there's no difference from 0.0.5.2 and 0.0.5.3
-kwag

Does this mean that we can continue using v0.0.5.3 and just substitute
SansGrip`s "Sampler"??? (sorry if this has been answered already)

Or do we need v0.0.5.2,<---(???) If so...where can we find that version??


*******************************
The Devil`s always.....in the Details!

jorel 06-27-2003 04:06 PM

@ Jell
thanks, i read all there and waiting your helper answers. :)

@ vico1
good question,i want to know the same.
:wink:

@ Osvaldo :)
tomsmocomp is worse,decrease the CQ.
i change deinterlance to just before and just after resize and change just a little...insignificant.
fielddeinterlace is in the better place in the script,
(thanks to Boulder and Phil for big explanations about deinterlace and ivtc)

you wrote:
"...i'm not a brain like you..." 8O
you are in better perspective,you have a good brain working,
mine is lost. :roll:


All times are GMT -5. The time now is 11:01 AM  —  vBulletin © Jelsoft Enterprises Ltd

Site design, images and content © 2002-2026 The Digital FAQ, www.digitalFAQ.com
Forum Software by vBulletin · Copyright © 2026 Jelsoft Enterprises Ltd.