Quantcast Time to Revise File Prediction! - Page 4 - digitalFAQ.com Forums [Archives]
  #61  
06-22-2003, 06:39 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
@All,

Well here I go again.
Hopefully we start zeroing in on this inconsistency.
I just finished encoding "The Boondock Saints" with the following changes:

(1) Turned ON Scene change detect on TMPEG. Yes , it seems to be working, and for some "I_DON'T_KNOW_WHY_REASON" the file size is smaller with the option turned on. Don't ask me why! Seems TMPEG is playing tricks on us
Apparently, it optimizes the compression much better with variable GOP, which is the case with this option turned on. Yep, you heard it right: The file size is smaller with scene change detection on. Maybe one reason for the erratic prediction BTW, I'm using TMPGEnc 2.513.

(2) Changed the GOP back to 24, because 12 was just as unpredictable as 24. So I'll leave it at 24, for a smoother picture, plus now it's variable anyway with the change described above.

(3) Enabled the scene change detection in ToK's "Video.en1" file. Here it is, so you can copy it to your OPT directory: http://www.kvcd.net/video.en1

(4) As posted on the current optimal script page, I removed the line:
conditionalfilter( last, asharp(-1,0), last, "nf", ">", "scd_trigger" )
Read the change log for the explanation.

So here are my "The Boondock Saints" results this time:

Wanted video size: 714,303.19
Final encoded video size: 690.997
For a difrerence of 3.3%, which I can tolerate as long as is negative
The CQ was 63.02, and my manual CQ calculation was 63 So ToK is doing the job correctly. This was using first and second pass as full sampler. Not the fast 10% sampler. That will come later. I'm just trying to get this back on a consistent state.
Now back to encode "K-Pax", and see what comes out, as that was the other extreme yesterday.

-kwag
Reply With Quote
Someday, 12:01 PM
admin's Avatar
Site Staff / Ad Manager
 
Join Date: Dec 2002
Posts: 42
Thanks: ∞
Thanked 42 Times in 42 Posts
  #62  
06-22-2003, 06:45 PM
vhelp vhelp is offline
Free Member
 
Join Date: Jan 2003
Posts: 1,009
Thanks: 0
Thanked 0 Times in 0 Posts
@ vico1..

>> To bad we can`t use some kinda` "transcode" technology with it...

Well, if we could only just get the "bitrate" accross the time-line, much
like what BV.EXE does, we're in for go, and so much for MA, as we know it.

Yes, I believe that Kwag made mention of these in previous topics
elsewhere's. I've ben wondering about this myself. It's a matter of time
before someone (here or doom9) finds the technique and shares it (we hope) !!

Then, finally we'll be ride of those items that plauged us in past.. some ie's:
* poor quality
* poor predictions
* better CDr/rw - DVDr/rw handling (on a disk per bases)
* anything else to ad here

-vhelp
Reply With Quote
  #63  
06-22-2003, 06:49 PM
vhelp vhelp is offline
Free Member
 
Join Date: Jan 2003
Posts: 1,009
Thanks: 0
Thanked 0 Times in 0 Posts
.
.

And, as for the MA script, I've ben messing around w/ some of the code,
and I've ben playing w/ the:
* scd_trigger = 60, and
* ScriptClip().. w/ asharp( -(fmin((nf/60), 1)) uses

Note the 60 instead of 30.
I think I've had better results, but I'm still debating the above changes I've
ben toying around with.

One thing I noticed still, is the shifting up/dn of the video on the first Bf.
If you remember my Bf/Df/Af analigy in my previous post.

To better analigies it:
CL....FRAME...Fltr.....STRENGTH...Notes
----------------------------------------------------------
Bf - 69379 - blur - 2 - noChange - person sitting
Df - 69380 - blur - 1 - noChange - person sitting
Af - 69381 - clear - 1 - sceneChange - person is walking
...
Bf - 69806 - blur - 2 - noChange - person sitting
Df - 69807 - clear - 1 - sceneChange - person sitting
Af - 69808 - clear - 1 - noChange - person is walking

Note 1,
The blur/blur/clear, and blur/clear/clear (for Bf/Df/Af)

Note 2,
The above is an example of trying to describe the MA process
a little better, I think. A way to visualize in a descriptive manor.
I guess I would catagorize it as a "tool"

Note 3,
The 2 and 1 are the Strength of the filter being applied. ie, I
noticed that on first frame BF, it would be strong hence the 2,
and on the Df frame, it would sometimes be less strength, or 1.

If the above help some, when working out this MA script
I hope this "tool" can be useful.

-vhelp
Reply With Quote
  #64  
06-22-2003, 08:49 PM
vhelp vhelp is offline
Free Member
 
Join Date: Jan 2003
Posts: 1,009
Thanks: 0
Thanked 0 Times in 0 Posts
@ all..

Can we try a different Filter in place of TemporalSoften(2,7,7,3,2)
I'm not liking all that much in my detais when I run it through vdub on
some items. it's smudging the letters too distortedly like.

And, I'm having difficulty adentifying those filters that will work under
YV12 color space. I'm still getting that noticable color washout look when
I compare two vdub's together, I can see one has richer colors. Anyways..

Can someone throw up a few Filters in place of the above ??
(but that will work w/out error showing up in vdub's screen.. YV12 arrr!!)

Thanks.
-vhelp
Reply With Quote
  #65  
06-22-2003, 09:21 PM
ovg64 ovg64 is offline
Free Member
 
Join Date: Jan 2003
Location: Puerto Rico
Posts: 423
Thanks: 0
Thanked 0 Times in 0 Posts
Send a message via MSN to ovg64
Hi Vhelp im currently using the act filter which is a temporal Smoother
n to me it does a pretty good job at smoothing and blending detail, kind of take away the fliks, i use atc(3,6,9,0.5,false) in place of the TemporalSoften(2,7,7,3,2) .

BTW: act is a YV12 filter too

Quote:
MaxTreshold = 1.50
nf = 0 # Current frame.
Mpeg2Source("C:\Documents and Settings\Osvaldo\Desktop\DVD\NS.d2v")
Limiter()
asharp(2,4)
GripCrop(528, 480)
GripSize(resizer="BicubicResize")
undot()
STMedianFilter(8, 32, 0, 0 )
Convolution3D(preset="movieLQ")
atc(3,6,9,0.5,false)
MergeChroma(blur(1.50))
MergeLuma(blur(0.2))
ScriptClip("nf = YDifferenceToNext()"+chr(13)+ "nf > 2.5 ? asharp( -(fmin((nf/30), 1)), 0 ) : \
TemporalSoften(2,7,7,3,2) ")
GripBorders()
LetterBox(0, 0, 16, 16)
Limiter()
function fmin(float f1, float f2) {
return (f1<f2) ? f1 : f2
}
Reply With Quote
  #66  
06-22-2003, 09:30 PM
vhelp vhelp is offline
Free Member
 
Join Date: Jan 2003
Posts: 1,009
Thanks: 0
Thanked 0 Times in 0 Posts
@ ovg64..

Great. I'll look up that filter in my repository hehe..
Ohoh.. looks like I GOT IT. yippy. I'm gonna try. Just finished one of
my test encodes w/ MA script. Looks good too. Hay, now I'll do a test
compar w/ Temporal(2,7,7,3,2) vs. your act(3,6,9,0.5,false)

Thanks.

Any more suggests ?? i'm gain to give them a shot. I'm having fun now
-vhelp
Reply With Quote
  #67  
06-22-2003, 09:40 PM
ovg64 ovg64 is offline
Free Member
 
Join Date: Jan 2003
Location: Puerto Rico
Posts: 423
Thanks: 0
Thanked 0 Times in 0 Posts
Send a message via MSN to ovg64
Yes Vhelp Convolution works like the old conv. and the new version
of STMedianFilter is 0.1.0.1 they r also YV12.

BTW: who's osv64
Reply With Quote
  #68  
06-23-2003, 12:11 AM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Here are the results and analisys on my K-Pax encode:

Wanted video file size: 714,303KB
Encoded video file size: 741,735KB

A difference of ~+3.69%

And this is GOOD
Here's the pattern I see. On action movies, file size will be ~3% smaller MAX.
On low action movies (like K-Pax), file size will be ~3% larger MAX.
So normal "average" films should target ~100% accuracy
There's something I did notice, and that is that just before the end credits, the file size was just at about 714MB, which was the wanted size. So the end credits is REALLY what's throwing off the calculations
I was watching TMPEG as it was encoding the end credits, and the file size was soaring high on the vertical scroll of the credits. "THAT'S OUR MAIN PROBLEM" The function "YDifferenceToNext()" reports a VERY LOW value on the end credits (verified with Vdub adding "nf" variable from script as subtitle), even though the scroll is a very high difference to next frame But because it's mainly a black screen with text scrolling vertically, the returned value is very small, and we are loosing A LOT of compression there
So my suggestions are as follow:

(1) Cut off end credits with AviSynth's function "Trim", and encode with a factor of 1.0. OR take into consideration the type of movie you're about to encode. If it's a low action film, subtract 3% (multiply by 0.97) the wanted sampler size.
If it's an action film, add 3% (multiply by 1.03) the wanted sampler size.
If it's an average film, don't add or subtract anything
(2) If using ToK, the same applies, BUT:
(a) Use a factor of 0.97 for low action movies.
(b) Use a factor of 1.03 for action movies.
(c) Use a factor of 1.0 for regular (average movies)

*** PAL people, could you please test this please ***
Please use the "Video.en1" file I uploaded earlier ( www.kvcd.net/video.en1 ) and copy it to your ToK OPT directory.
*** PLEASE, let us know your results ***


-kwag
Reply With Quote
  #69  
06-23-2003, 12:56 AM
audi2honda audi2honda is offline
Free Member
 
Join Date: Jun 2003
Location: Orange County, CA
Posts: 291
Thanks: 0
Thanked 0 Times in 0 Posts
kwag it's funny you mention the end credits messing up the movie. I've been having fairly accurate results using the old style prediction with a high sample length of 48 and prediction factor of 1.00.

Precition was erratic but kind of on target. Well I allways cut the end credits except for this one movie where I wanted to keep them. That movie has given me so much trouble over the last few days. I've encoded it 4 times lowering the CQ tons and it allways over shoots the mark by several MB. Based on what I've seen in my last few encodes it is pretty much consistant with your results.
Reply With Quote
  #70  
06-23-2003, 01:41 AM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Well, now we just need some feedback from PAL land, and see how the results match

-kwag
Reply With Quote
  #71  
06-23-2003, 04:21 AM
Krassi Krassi is offline
Free Member
 
Join Date: Mar 2003
Location: Germany
Posts: 390
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by kwag
Well, now we just need some feedback from PAL land, and see how the results match

-kwag
I'm currently @work .. I'll start the encoding this evening. I think i will encode "The Fast and the Furious". What factor should i use, what do you think?
Mabye another PAL-User can give any feedback? Jorel .

EDIT: Encoding "The Score" with factor 1.0.
Reply With Quote
  #72  
06-23-2003, 08:23 AM
ovg64 ovg64 is offline
Free Member
 
Join Date: Jan 2003
Location: Puerto Rico
Posts: 423
Thanks: 0
Thanked 0 Times in 0 Posts
Send a message via MSN to ovg64
Quote:
Originally Posted by kwag
Here are the results and analisys on my K-Pax encode:

Wanted video file size: 714,303KB
Encoded video file size: 741,735KB

A difference of ~+3.69%

And this is GOOD
Here's the pattern I see. On action movies, file size will be ~3% smaller MAX.
On low action movies (like K-Pax), file size will be ~3% larger MAX.
So normal "average" films should target ~100% accuracy
There's something I did notice, and that is that just before the end credits, the file size was just at about 714MB, which was the wanted size. So the end credits is REALLY what's throwing off the calculations
I was watching TMPEG as it was encoding the end credits, and the file size was soaring high on the vertical scroll of the credits. "THAT'S OUR MAIN PROBLEM" The function "YDifferenceToNext()" reports a VERY LOW value on the end credits (verified with Vdub adding "nf" variable from script as subtitle), even though the scroll is a very high difference to next frame But because it's mainly a black screen with text scrolling vertically, the returned value is very small
So my suggestions are as follow:

(1) Cut off end credits with AviSynth's function "Trim", and encode with a factor of 1.0. OR take into consideration the type of movie you're about to encode. If it's a low action film, add 6% (multiply by 1.06) the wanted sampler size.
If it's an action film, subtract 6% (multiply by 0.94) the wanted sampler size.
If it's an average film, don't add or subtract anything
(2) If using ToK, the same applies, BUT:
(a) Use a factor of 1.06 for low action movies.
(b) Use a factor of 0.94 for action movies.
(c) Use a factor of 1.0 for regular (average movies)

*** PAL people, could you please test this please ***
Please use the "Video.en1" file I uploaded earlier ( www.kvcd.net/video.en1 ) and copy it to your ToK OPT directory.
*** PLEASE, let us know your results ***

-kwag
Well the thing i see here is that you already had rich file size
before the end credits, so again that shows the inability of tok
predicting when the AScript is turning on n off (blur), as for the
for the end credits shooting up the bitrate i seen that even before
the adaptive script while using 2.08 so i not sure kwag but to me that just
the way Tmpeg encodes. Maybe by lowing the max bitrate from 2500 to
2300 stabilize prediction just a tough.
Reply With Quote
  #73  
06-23-2003, 09:06 AM
jorel jorel is offline
Invalid Email / Banned / Spammer
 
Join Date: Aug 2002
Location: Brasil - MG - third stone from the sun
Posts: 5,570
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by Krassi
Quote:
Originally Posted by kwag
Well, now we just need some feedback from PAL land, and see how the results match

-kwag
I'm currently @work .. I'll start the encoding this evening. I think i will encode "The Fast and the Furious". What factor should i use, what do you think?
Mabye another PAL-User can give any feedback? Jorel .

EDIT: Encoding "The Score" with factor 1.0.
hi Krassi!

in Brasil is pal-m, different from europe pal-g.
then here we encode ntsc-m, the tvs, vk7 and dvds have "dual systems".
ntsc-m is the normal ntsc (like in USA)
the letters "g" and "m" in the final of the system
have a big difference between them,
not only the sizes but all the transmition and reception too.
see, a pal-g tv in Brasil don't work,don't show the image or sound.
the ntsc tv in Brasil stay "black&white" and we need to
make some "transcodifications" as we call,in the color system.
pal and ntsc are the names of the color systems
and "g" and "m" are the "standard" for the remainder signals.
is a big difference.
Reply With Quote
  #74  
06-23-2003, 09:27 AM
Jellygoose Jellygoose is offline
Free Member
 
Join Date: Jun 2002
Location: Germany
Posts: 1,288
Thanks: 0
Thanked 0 Times in 0 Posts

And I always thought PAL=PAL...
Another contribution to my knowledge! Thanks Jorge!
__________________
j3llyG0053
Reply With Quote
  #75  
06-23-2003, 09:29 AM
GFR GFR is offline
Free Member
 
Join Date: May 2002
Posts: 438
Thanks: 0
Thanked 0 Times in 0 Posts
As jorel said, pal and ntsc tell how color info is multiplexed into the analog signal. NTSC is one of the first color muxing standards. PAL is more sophisticated and has more stable colors.

the G and M refer to the vertical scanning frequencies, and set the number of lines. As a rule of thumb this is 1/2 the AC power frequency. There are many standards, like M, N, G.

NTSC-M (Japan, USA) is ~30fps. In these countries AC power is 60Hz. You have 480 lines.

In countries where the AC power is 50Hz (most of Europe?) the video standard is 25fps, 576 lines.

In Brazil we have PAL color encoding, but AC power is 60Hz, so we have PAL-M, ~30fps, 480 lines.

So, when you associate PAL=576 lines, NTSC=480 lines, that's not exact.

PS.: It is a hell to find capture cards that support this (since Brazil is the only country that has this standard). The only one I know is ATI AIW, but it works OK with PAL-M broadcast signals but not with prerecorded PAL-M tapes.

PS2: jorel, eu conheci o cara que ajudou a implantar a TV colorida no Brasil, e que definiu o sistema PAL-M. Era o professor Alcione, do IME. MUIIITO fera, mas doidinho
Reply With Quote
  #76  
06-23-2003, 09:38 AM
Krassi Krassi is offline
Free Member
 
Join Date: Mar 2003
Location: Germany
Posts: 390
Thanks: 0
Thanked 0 Times in 0 Posts
Hi Jorel,
sorry, didn't know about that. Seems to be very complicated. Just saw in a list that Brazil has PAL-Standard .
However, about ~ 3 hours to go...
Reply With Quote
  #77  
06-23-2003, 10:19 AM
jorel jorel is offline
Invalid Email / Banned / Spammer
 
Join Date: Aug 2002
Location: Brasil - MG - third stone from the sun
Posts: 5,570
Thanks: 0
Thanked 0 Times in 0 Posts
yes Krassi,
big complicated.

jell and GFR:
the differences are more than vertical and number of the lines.
i try to explain..if my english help me:

frequences of color transmitions
ntsc-m: 3.579.545
pal-n:3.579.545
pal-m:3.575.561.149
pal-g:4.5...?(don't remember the rest)

the sizes of the "channels bands" and the audio band
is different between "m" and "g".

ntsc have in vector:
zero>the burst signal
+90>the red signal
+180>the blue signal

pal have in vector:
zero>the burst signal
+90>the red signal(one phase)
-90>the red signal(another phase)
180>the blue signal

pal-m is the same like pal-g but
in Brasil we have a circuit called
"pal key" that find what is +90 red and what is -90 red
and turn all red in +90 like in ntsc,without invertion of colors
like sometimes the ntsc turn the red to green and the blue to purple.
this circuit find and turn the lines to the original poinr in vector.

the reference for the colors in vector is the "zero" burst.
in ntsc if the +90 turn to -90,the burst see the red
in 90 degrees and "think" that he is correct in the vector but
the red is 180 degress from the original point and show as green.
than we use the "tint" or "hue" to correct this.
the pal system is to correct the original
-90 red automatically to +90 and turn the system stable.
this is the reason of the name "pal":
Phase Alternating Line!

well this is just only for the color...the tuner band,the audio band,
and lots more things are differents between "m" and "g".

i will that my poor english don't turn it all more confused,
i can't explain better than this.


GFR:
"Era o professor Alcione, do IME."
já ouvir falar muito bem dele...

me pareço bastante com ele na parte do "doidin".
Reply With Quote
  #78  
06-23-2003, 10:51 AM
GFR GFR is offline
Free Member
 
Join Date: May 2002
Posts: 438
Thanks: 0
Thanked 0 Times in 0 Posts
>>this is the reason of the name "pal":
Phase Alternating Line!

Or as people say "Peace At Last"! against "Never Twice the Same Color".

I forgot the funny acronym for SECAM.
Reply With Quote
  #79  
06-23-2003, 12:47 PM
Krassi Krassi is offline
Free Member
 
Join Date: Mar 2003
Location: Germany
Posts: 390
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by GFR
Or as people say "Peace At Last"!
Yepp, thats right ... Here are the results :
Code:
=============================================================
ToK Log: THE_SCORE.avs
=============================================================
 
Resolution (fps):704x576 (25,000 fps)
Total Frames: 170405
Total Time  : 01:53:36
-------------------------------------------------------------
 
Audio Size: 95.428.389
Required Video Size: 723.771.611
 
Factor: 60,000
Desired Sample Size: 12.062.860
 
-------------------------------------------------------------
New Faster Prediction
-------------------------------------------------------------
 
Full Sample
Next CQ: 60,000. Sample Size: 18.108.725
Small Sample
Next CQ: 60,000. Sample Size: 1.528.880
Predicting...
Next CQ: 39,968. Sample Size: 12.650.243
Next CQ: 29,321. Sample Size: 11.162.712
Next CQ: 35,204. Sample Size: 11.853.128
Next CQ: 37,022. Sample Size: 12.429.518
Next CQ: 35,989. Sample Size: 12.411.949
Next CQ: 35,548. Sample Size: 12.126.407
Next CQ: 35,422. Sample Size: 12.031.024

Exit Condition: 0,500 % reached ! yahoo !
Tries   : 8
 

Final CQ: 35,422
Total Time For Predicition: 00:29:21

 
-------------------------------------------------------------
Encoding F:\Recorder\THE_SCORE.avs
-------------------------------------------------------------
 
Encoding... CQ : 35,422
Final Encoded Size: 691.823.124
Factor was 1.0, New Prediction Method in ToK and 3 Samples/Minute. Credits were cut off.
Reply With Quote
  #80  
06-23-2003, 01:34 PM
kwag kwag is offline
Free Member
 
Join Date: Apr 2002
Location: Puerto Rico, USA
Posts: 13,537
Thanks: 0
Thanked 0 Times in 0 Posts
Quote:
Originally Posted by Krassi

Factor was 1.0, New Prediction Method in ToK and 3 Samples/Minute. Credits were cut off.
3 samples a minute, of how many frames per sample
I'm using one sample per minute of 24 frames each

-kwag
Reply With Quote
Reply




Similar Threads
Thread Thread Starter Forum Replies Last Post
Bitrates: Prediction time for CQMATIC epuleda Video Encoding and Conversion 4 05-26-2006 02:11 PM
simple way to do time stretching of a mp2 file? zagor Audio Conversion 12 08-27-2004 05:51 AM
Tmpgenc cannot open source file for file prediction Kane Avisynth Scripting 3 02-03-2003 02:44 PM
KVCD: File Size Fluctuates Big Time! Jellygoose Video Encoding and Conversion 4 01-28-2003 01:55 PM
DVD player read time wrong if file size is different? COMMANDO Video Encoding and Conversion 1 07-22-2002 06:02 PM

Thread Tools



 
All times are GMT -5. The time now is 12:23 AM  —  vBulletin © Jelsoft Enterprises Ltd