Crazy Idea for Better Quality #2...

Jellygoose · #1 12-31-2002, 11:39 AM

hi all!

since this part of the forum has developed to a place to exchange crazy ideas about how to better picture quality for MPEG-files, I wanna share with you guys what came to my mind this morning...

I just finished encoding "The Mothman Prophecies", which turned out really good all in all. It's kind of a drama, so almost no high action scenes... except...
the very last 5 minutes are high action underwater scenes, with fire and everything (you all know that's a pain for your encoder...). That's why these particular scenes don't look very clean, and I don't see a way of getting rid of the blocks that appear in the underwater scenes.

Now my problem was: Since there are a lot of movies, which only have one or two high action scenes, it's often a hard decision, whether to concentrate on the overall-quality of the movie or especially on these high action scenes. that's at least what I experienced. Sometimes increasing the CQ value just gives bigger file-sizes, and doesn't give you any better quality at all...

so I went on and asked myself : Is there a way to tell TMPGEnc to pay more attention to a certain part of the movie? In my case the last 5 minutes?

An example: the average bitrate for the whole movie, as displayed in bitrate viewer would be 900kb... Now when you take a look at the last 5 minutes (which are the only high action scenes in the movie) there the average bitrate is at let's say 1800kb (and still this is not enough to get rid of the blocks). Now how about telling TMPGEnc to use let's say an average bitrate of 860 for the whole movie (which still would be enough for the drama part), and use 2100kb for the last 5 minutes as an average bitrate.

I hope somebody at least did understand what I mean...

SansGrip · #2 12-31-2002, 12:03 PM

Quote:

Originally Posted by Jellygoose

so I went on and asked myself : Is there a way to tell TMPGEnc to pay more attention to a certain part of the movie? In my case the last 5 minutes?

Yes, that and much more is possible using the "force picture type" dialog, accessible through the "GOP structure" settings tab. On a frame-by-frame basis you can specify whether it should be an I-, P- or B- frame, what bit rate (or CQ quality) to use, a custom GOP, a custom Q matrix, and more. This is on a frame-by-frame basis...

There's a great deal we could do with this, but it's pretty complex and time-consuming. I've never played with it much, but theoretically one could achieve significantly higher compression by using, for example, different CQ qualities for different parts of the movie.

Jellygoose · #3 12-31-2002, 12:35 PM

Quote:

Originally Posted by SansGrip

Quote:

Originally Posted by Jellygoose

so I went on and asked myself : Is there a way to tell TMPGEnc to pay more attention to a certain part of the movie? In my case the last 5 minutes?

There's a great deal we could do with this, but it's pretty complex and time-consuming. I've never played with it much, but theoretically one could achieve significantly higher compression by using, for example, different CQ qualities for different parts of the movie.

Yeah that's what I figured...

I guess there's some point where the afford for this gets too high, regarding the advantages you get...

Daagar · #4 12-31-2002, 01:02 PM

I believe something similar to this is already done by the folks over at doom9.org that use DVD2SVCD and/or CCE. The terminology used to be called the 'robsh0t method' I believe (from the poster who first described the process). They had tricks to minimize the amount of work necessary, but it was still fairly time-consuming. Sounded like a cool idea, but not automatable.

dr_nicotine · #5 01-06-2003, 03:36 PM

I can understand that this idea is difficult to implement on different parts of the movie, but maybe it's possible and doable on the credits, xvid can do this if I'm not mistaken.

I don't like to cut credits, but I don't mind if their quality is just good enough to read.

Just a thought, keep working on the more important stuff first :P

Boulder · #6 01-06-2003, 03:59 PM

I've done something in that direction with the Trim function. I usually filter the credits very heavily so that they would compress better, provided that there's nothing really special about them (like something happening while the credits appear etc.)

I don't know if it's of any real use but there's always some extra bitrate to be spent elsewhere in the clip

dr_nicotine · #7 01-07-2003, 07:27 AM

Quote:

Originally Posted by Boulder

I've done something in that direction with the Trim function. I usually filter the credits very heavily so that they would compress better, provided that there's nothing really special about them (like something happening while the credits appear etc.)

I don't know if it's of any real use but there's always some extra bitrate to be spent elsewhere in the clip

Could you explain a bit more? Trim in avisynth? Could you give an example line?

SansGrip · #8 01-07-2003, 07:39 AM

Quote:

Originally Posted by dr_nicotine

Could you explain a bit more? Trim in avisynth? Could you give an example line?

Let's say you've determined (via our friend VirtualDub) that the opening credits start at frame 0 and end at frame 500, and that the closing credits start at frame 200,000 and end at frame 202,000.

You could then do something like this in Avisynth:

Code:

BlahSource("...")

opening_credits = Trim(0, 500)
movie = Trim(501, 199999)
closing_credits = Trim(200000, 0) # 0 means "to the end"

opening_credits = opening_credits.ExtremelyStrongSmoother(...)
movie = movie.NormalSmoother(...)
movie = movie.Blockbuster(...)
closing_credits = closing_credits.ExtremelyStrongSmoother(...)

opening_credits + movie + closing_credits # Concatenate

As you can see, this allows different filters to be applied to different parts of the clip.

If you have any background in object-oriented programming it might help to think of clips as objects, and filters as methods one can call on them that return a clip.

black prince · #9 01-07-2003, 07:49 AM

Hi dr_nicotine,

dr_nicotine wrote:

Quote:

Could you explain a bit more? Trim in avisynth? Could you give an example line?

Here's a sample avs script. Note the Trim and TemporalSoften statements:

LoadPlugin("E:\DVD Backup\2 - DVD2SVCD\MPEG2DEC\MPEG2DEC2.dll")
LoadPlugin("E:\DVD Backup\2 - DVD2SVCD\Blockbuster.dll")
mpeg2source("D:\Temp\movie.d2v")
#
LegalClip()
LanczosResize(336,224)
#
opening=Trim(0,550)
opening=opening.TemporalSoften(3,8,30) # <--- compresses intro
#
movie=Trim(551,158936)
movie=movie.Blockbuster(method="noise",variance=.5 ,seed=1)
#
credits=Trim(158937,0)
credits=credits.Greyscale().TemporalSoften(3,8,30) # <--- compress credits
#
last=opening+movie+credits
#
AddBorders(8,8,8,8 )
#
LegalClip()
#
# ---------- End of Script ----------

This allows you to make smaller movie by compressing the opening and/or
credits

Trim(0,550) says use frames 1 to 550. Trim(158937) says use
frame 158937 to last frame of movie

-black prince

Daagar · #10 01-07-2003, 03:16 PM

Very cool, but I could imagine this would alter the file predicition results if not enough (or too many) samples were pulled from these alternate-filter regions.

black prince · #11 01-07-2003, 04:09 PM

Hi Daagar,

I use script above for movies that just quite don't fit my target file size.
If file prediction tells me I need say 5meg to 10meg more room to get
a movie on 1 CD or 2 CD, then I can compress credits or cut them out
completely. I only found file prediction difficult when I cut out credits
or intro. It changes movie length and time which throws off the formula

-black prince

Boulder · #12 01-07-2003, 04:19 PM

Quote:

Originally Posted by black prince

It changes movie length and time which throws off the formula

But you can compensate that by entering the length of the movie manually in Sampler and also when calculating the sample file size with Kwag's formula

black prince · #13 01-07-2003, 08:11 PM

@Boulder,

Boulder wrote:

Quote:

But you can compensate that by entering the length of the movie manually in Sampler and also when calculating the sample file size with Kwag's formula

Yes!!. I actually use Kwag's small program called "fileinfo.exe" in a
DOS batch file and pipe the output to a text file. I use the avs script
with just the credits or movie to get frames and length. This info is
used in my spreadsheet to calculate file size:

kvcdenc 0.0.30 ALPHA
d:\Temp\movie.avs file details:
Width = 496, Height = 336, Fps = 23.98, Bits per pixel = 16
Video frames = 2568, Length = 107.1 sec.
Video total frames = 2568

Above is a sample of just the credits. You'll notice the frames are 2568 and
the time is 107.1 seconds.

Another way to decrease file size is to encode the credits separate
at a CQ_VBR ~ 15 to 20. Join them together as one mpg file
using "hjsplit" utility then burn with VCDEasy. This is similar to
what can be done with avi in GordonKnot. You can then increase the
CQ_VBR for intro+movie. It's more trouble, but for those looking to
get every bit of compression it's the way to go.

-black prince

dr_nicotine · #14 01-08-2003, 11:03 AM

Thank you Boulder and Sansgrip!

I will try this out with my next encoding

Gaudi · #15 01-08-2003, 11:06 PM

I have come across a good idea, despite the fact I am not proficient in programming.

Perhaps someone out there can do something with it.

First of all. I know very little about compressing and the like. But I think that the bitrate of a scene, provided CQ_VBR is a good index of its action or complexity. Perhaps there are some other methods.

Ok, here is then the idea:
Why don´t encode small clips of the movie, lets say 10 seconds every 1 to 3 minutes (perhaps we could impement some scene change detection here to include complete scenes in each clip avoiding splitting). We can easily do this with a small soft that automates a creation of an avisynth script, sends the commandline to TMPEG, waits for the encoding to complete, and then checks the file size.
We can do this for every sample in the movie, and can the build a "complexity" vs time graph. If we account for the sample to be representative of every clip, then we will have some idea on how to set CQ (or bitrate) for different parts of the movie.

We can then repeat the encoding but this time for each clip (not just the sample) setting different CQ and/or bitrate settings for each one. We will the come to a series of clips or "parts" that will have to be joined together.

No problem with that. And the audio will be ok because it uses CBR.

Perhaps someone can do a small test to check results. I myself will try to do something in excel this weekend to check what comes out.

Another aproach could be to use bitrate viewer and export the data for each clip after encodign the whole movie.

Hope this idea is a good one to trigger some development in this area.

Gaudi