I've been doing a bit of testing today with mergechroma(blur(1.58 )), mergeluma(blur(.6)) and DCTFilter combinations on a 1minute 45second, 4:3 letterboxed movie trailer. Here's my $.02 cents for what it's worth:

• trailer w/o mergechroma/luma or dctfilter=16.96MB(17,783,324bytes)
  
  trailer w/ dctfilter(1,1,1,1,1,1,.5, 0) before gripsize()=16.94MB(17,758,656)
  
  trailer w/ dctfilter(1,1,1,1,1,.5,.5,0) before gripsize()=16.79MB(17,603,768)
  
  trailer w/ mergechroma/luma + dctfilter(1,1,1,1,1,.5,.5,0) before gripsize()=15.85MB(16,621,011)
  
  trailer w/ mergechroma/luma + dctfilter(1,1,1,1,1,.5,.5,0) before gripsize(resizer="lanczosresize")=16.44MB(17,233,4 59)

I found that using dctfilter(1,1,1,1,1,.5,.5,0) gave better compression than dctfilter(1,1,1,1,1,1,.5,0) with no visual difference(at least not to my eyes, but I'm no pro and don't have an hdtv). I also found that using mergechroma/luma softened the picture a bit too much for me(on my monitor...haven't tried on tv yet), but using lanczosresize sharpened it enough to make it acceptable again (again, to my eyes). Blockbuster "sharpen" increased filesize too much, so that's why I used lanczosresize. Also, I found that switching from TemporalCleaner(my usual preference) to Convolution3D(preset="movieLQ") gave an additional 145,060 bytes of compression, for a total of 16.3MB(17,088,399bytes) versus the original 16.96MB(17,783,324). That's a space-savings of 639,421 bytes on this 105-second trailer. Unless my math is wrong, that would be ~ 42MB of additional saved space per 120-minute movie that could be used towards raising the CQ level a bit, upping the audio bitrate, or adding another 2.5 minutes of movie-time at the same CQ. Might not seem like much, but every bit counts, especially with 1-cd encodes.

I'm wondering how much the mergechroma/luma blur values would have to be lowered before it made sharpening via lanczos or other methods totally unnecessary :?: But would any space-savings then totally disappear at that lower blur value?

Kwag, have you given DCTFilter(1,1,1,1,1,.5,.5,0) a try? This is the recommended setting in the docs. It gives a little more compression and I can't really see a quality difference (but maybe you can! :)).
-d&c

script used:

I think the DCTFilter works great when used in other situations, but with all the filters above in the script, I really can't tell a difference. I think that using Lanczos without blockbuster sharpen, and lowering the blur on mergeluma, will be the key point to get a sharper picture while retaining a very good compression. What I like about this is that artifacts (Gibbs) are almost gone :lol:

-kwag

You're right GFR. It should make things faster :wink:

-kwag

With these techniques, I think we can now easily put a 45 to 50 minute episode with KVCD 352x240 PLUS on one of those little CD-Rs :lol:

-kwag

Kwag,

As I understand, MergeLuma and MergeChroma allow you to process Luminance and Chrominance separately.

So you could do HEAVY filtering on the Chrominance, and moderate filtering on Luminance to get compression gains without making the picture look too much blurry.

If you blur each channel before resizing, not only it's slower but it has less effect. It's more like a pre-resizing anti-aliasing filtering than really smoothing.

I guess if you move the MergeChrominance to after resizing you can get good compression without the need of MergeLuma (or with just really subtle MergeLuma).

BTW you can use more complicated things with the Merge filters like:

MergeLuma(Convolution3D(preset="movieHQ"))
MergeChroma(FaeryDust())

:)

I've tried moving MergeChroma and MergeLuma to after the resize, and filesize dropt a lot, faster encoding, but it got TOO blurier.

Since I didn't want to tweak the blur parameters, I just moved MergeLuma back to before resize, and kept MergeChroma after the resize. The video looks OK but filesize dropt only 40k (I could up CQ 0.1) compared to both Merge's before resize. Still faster.

I guess the best would be both Merge's after resizing for speed, but the Luma blur has to be much gentler then.

Have we reached a final consensus yet on whether to use these filters and the best settings so us newbies can use it? Also, should have Maud-dib add it to his proggie

I'm pretty sure using MergeChroma and MergeLuma AFTER Resize is the way to go... I tried it with these values:

Mergechroma((blur(1.3)
MergeLuma((blur(0.3)

For me this is the way to go, because it doesn't influence the sharpness that much that it's necessary to use Lanczos, and therefor the compression of Bilinear Resize is provided too... Try it out and tell me what you think. :wink:

This is for some people trying to cram too much onto one cd. I am backing up my friends dvds. There is about 6 episodes on each dvd, about 135-143 minutes of movie on each. Well it was my goal to put these on one cd. It wasnt really possible to do this until mergechroma and luma came into the light. I ran these filters after resize, cause it took way too long the other way, and i see no visible difference between the two. But after adding these two filters and dctfilter i was able to raise the cq by 4 from 70.5 to 74.5. I used less chroma luma then kwag and jellygoose but got great results. I was too concerned about sharpness at this low resolution. Here is my script.

Branden

Hi guys,

The only change I've done so far is this:

mergechroma(blur(1.58 ))
mergeluma(blur(0.2))

And of course changing the order of these filters AFTER the resizing.
I'm still trying with Lanczos, because I see that using bilinear resize is to soft.

@heyitsme,

Your script is very similar to the one I'm using :) . Only that I have also been using NoMoSmooth and FluxSmooth. Here it is:


And this is still not a final :lol:

-kwag

@ Kwag what kind of filters are fluxsmooth and NomoSmooth, and would it affect sharpness? The problem is that the Friends DVD lacks sharpness. So when i bring it down to vcd res it looks a little soft. So i am trying to come up with a good script that would compress well, but wont destroy the sharpness.

From SansGrip's documents:

NoMoSmooth:
When it comes to smoothers (as opposed to smoothies) there are two main varieties: temporal smoothers and spatial smoothers. Temporal smoothers work by looking at frames surrounding the current one and averaging corresponding pixels together if they're deemed similar enough (and thus likely to be noise). Spatial smoothers work in a similar way, except they look at nearby pixels within the same frame. A few filters are a hybrid of the two, including this one and the excellent Convolution3D.

Each approach has its advantages and disadvantages. The great thing about temporal smoothers is that they do a fantastic job getting rid of noise when you set them up just right, but they can also generate some very ugly artifacts, particularly when dealing with motion. Spatial smoothers, on the other hand, can be quite destructive to the details in your image, but are great with high-motion areas because those have few details and are moving too quickly to notice artifacts anyway.

Ideally, then, one would like to apply a temporal smoother to relatively static areas and a spatial smoother to moving ones. That's what this filter tries to do, and the aim is for a higher quality result (i.e. less noticible smoothing) than Convolution3D but with correspondingly less of an impact on compressibility.

In addition to this motion-based approach, NoMoSmooth employs another technique to try to retain as much existing detail as possible: only pixels that are "fluctuating" are smoothed. In other words, given three frames in a row with the current one in the centre, the pixel being examined will only be considered for either type of smoothing if it is either greater than both the corresponding pixels in the previous and next frames, or less than both. This could be considered a very simplistic "noise detector" and seems to help reduce temporal smoothing on important details such as those in skin tones.

FluxSmooth:
One of the fundamental properties of noise is that it's random. One of the fundamental properties of motion is that it's not. This is the premise behind FluxSmooth, which examines each pixel and compares it to the corresponding pixel in the previous and last frame. Smoothing occurs if both the previous frame's value and the next frame's value are greater, or if both are less, than the value in the current frame.

I like to call this a "fluctuating" pixel, then I like to wipe that pixel from existence by averaging it with its neighbours. This is (by default) done in a spatio-temporal manner, in that for each fluctuating pixel its 8 immediate spatial neighbours as well as its 2 temporal neighbours (the abovementioned corresponding pixel from the previous and next frames) are considered for inclusion in the average. If the value of each pixel is within the specified threshold, it is included. If not, it isn't.

This filter seems to remove almost all noise from low-noise sources (such as DVD) and a lot of noise from high-noise sources (such as cable TV captures), while maintaining a good amount of detail.

The speed at which it operates depends upon the amount of noise in the clip.

----------------- End ------------------------

-kwag

Here's the S1M0NE sample, encoded again with the last .avs I posted above: http://www.kvcd.net/s1m0ne-result.mpg
The complete video size was 688,350MB 8)
Muxed with audio, 795,650MB.
Used CQ=70.5 KVCDx3
That's what the complete movie looks like on one CD :wink:

-kwag

I've got a feeling that a somewhat generous dose of MergeLuma and MergeChroma may help a lot when encoding anime/cartoons.

Any volunteers :?: :D , because I don't do animes 8)

-kwag

This may have been answered further in the thread already, but I can't be bothered to look and then come back ;).

DCTFilter does a discrete cosine transform of the data, then filters in the frequency domain, then converts the data back to pixels. Basically it's quantizing specific frequencies in order to gain compressibility. If you crank up the strength parameters you'll see what happens -- it'll get blocky because you're taking out most of the high-frequency information.

Tom Barry, the author, hasn't done much with it because it turned out to be not so useful with XViD (though this might have changed in my absence). That's not to say it won't work good with TMPGEnc, though. It needs to be tested.

As someone pointed out earlier, MergeChroma and MergeLuma can be used in combination with any other filters. It's often possible to apply extremely heavy filtering to the chroma components without losing noticible detail, because the eyes are much less sensitive to detail in the chroma components.

For example, while kwag finds PixieDust too strong for DVD sources, it probably isn't if it's only applied to chroma:

Obviously I'm exaggerating somewhat, but you get the idea.

Remember also that

is the same as

There's nothing magical about "MergeLuma(Blur(...))" as opposed to a simple "Blur(...)". This is why things might start looking, er, blurry if you use too high a value ;).

Question here! Isn't chroma being assigned 4 times :?: , so then when you call MergeChroma(chroma), it's only applying the last assignment value which is chroma.DctFilter(...) :?: :roll:

-kwag

Think of the "chroma = chroma.XXX" lines are variable assignments. This code:

is simply a short way of saying:

where "last" is a special variable representing the result of the last filter. It's used by default if no clip is specified either via member selection ("last.SomeFilter(...)") or passing the clip as a parameter ("SomeFilter(last, ...)").

So the four lines dealing with chroma above are simply running those filters as usual except instead of using the "last" clip, it uses the clip stored in the "chroma" variable. This way "last" is left untouched, and in this example will contain the output from Mpeg2Source (or the last filter preceeding the first "chroma = " line).

Because DctFilter is applied to the clip stored in the "chroma" variable (which is in turn the result of the Blur filter, which is the result of the TemporalSoften filter, etc.), the "chroma" variable ends up containing the same result as if you'd done this:

and when you do the final MergeChroma, the chroma part of the "last" variable is replaced with the chroma from the "chroma" variable.

I hope that makes sense. The short answer to your question is "no" :mrgreen:.

Ok, so each assignment is actually an "append" :wink:

Thanks,
-kwag

Too bad we can't use two instances of dust in the same script :(

We can't?

I'm afraid we can't :(

On the sample avs that comes with dust5 it says:

##Known Issues:
##slow
##only one instance

Besides, I've tried to run two instances, and I got a black screen at the output.

Both instances were the same kind of dust (clip1.spacedust(), clip2.spacedust()) applied to different clips, maybe two different dust types (like spacedust() and faerysdust()) can live together, I haven't tested it.

Kwag,

I just tried your above script and am kind of confused. How did you get the complete 131min movie to fit on to 1 disk? Was it a dvd-r or cd-r? If it was a cd-r how is it possible because my 89min movie requests a sample size of ~9.528MB @128kbps using the prediction script. I used a CQ of 60 even and got a sample size of 10.4MB. :(

Is there some sort of settings in TMPGEnc that you use that reduces size also (i.e. my GOP is 1-24-2-1-24)?

Oops, just re-read the last paragraph and you mentioned CD :D. So how is that possible :?: :?:

@TCC

The reason Kwag and most of us are able to stick that much info is because our framerate is 23.976 and 25. Seems your framerate is at 29.976 which would result in a bigger file size for the same amount of quality. If you could use force film on your movie, or reduce the resolution to up the CQ.

Branden

OOoohhhhhh, now I get it. :lol:
So even tho my d2v file was 29fps (in the stats window of dvd2avi) and I was making the m1v file at 23fps in TMPGEnc, it was actually doing it at 29fps right?

So the correct way to do all of this is to remake my d2v file and use "force film" to have 23fps. And then use the above script with the KVCDx3 template in TMPGEnc at 23fps correct??

Yes, but only if your DVD was shot FILM. Then you can use "Force FILM" :wink:

-kwag

Sorry to ask you Kwag again about this: What to do if the DVD2AVI preview is:

Aspect Rate: 4:3
Frame Rate: 29.970
Video Type : Film 55% (flicking) NTSC 45% :x :!:
Frame Type : Progressive (flicking) Interlaced :x :!:

Should I use Inverse Telecine (Telecide() and Decimate()) Or Field DeInterlace :?: (I'm trying with MovieStacker :) 8) )

Also, how should be the encode frame rate on TMPgEnc: 23.97 or 29.97 :?:

Again :oops: sorry!
LaBomba

So if my movie is NTSC format then what's the best way to get my 89min movie with good quality/high CQ on to one CD-R?

Can I not just use "force film" even tho it is NTSC format?? Would there be a negative outcome to doing this??

Most NTSC DVDs are shot FILM. :wink:
Sorry, I should have been more specific :oops:

-kwag

Ok, that's good to know. But how can you determine this?

Open your VOBs with DVD2AVI, and press "F5".
Let it run a couple of minutes, and look at the stats. If it says >95% FILM, you select "Force FILM"

-kwag

Just tried some test encodes with mergechroma(blur(1.2)) and mergeluma(blur(0.6)) and found the output unacceptably blurry on my bigscreen TV. Anyone played around with the values and found a more acceptable quality with file size reduction?

Yes, drop mergeluma(blur(0.6)) to mergeluma(blur(0.2))

-kwag

Hey Kwag, I did some of the following tests with the two new filters and this is what I came up with, hopefully I did not make any mistake.

Mpeg2Source("C:\Ripping\We Were Soldiers\Soldier's.d2v")
LegalClip()
mergechroma(blur(1.58))
mergeluma(blur(0.2))
GripCrop( width=528, height=480, overscan=2 )
GripSize()
SpaceDust()
FluxSmooth(7,7)
NoMoSmooth(40,1,6,1,3,false)
Convolution3d(preset="movieHQ")
GripBorders()
LegalClip()
Sampler(length=24)

This was one of the firsts you posted in this forum to do simone 131 min
at a cq of 70. I took out blockbuster cause it was said
not needed at this resolution

My Test - We Were Soldiers 138 Minutes 13.9 MB

My next test with some of your changes

Mpeg2Source("C:\Ripping\We Were Soldiers\Soldier's.d2v")
LegalClip()
GripCrop( width=528, height=480, overscan=2 )
GripSize(resizer="lanczosresize")
mergechroma(blur(1.58))
mergeluma(blur(0.2))
SpaceDust()
FluxSmooth(7,7)
NoMoSmooth(40,1,6,1,3,false)
Convolution3d(preset="movieHQ")
GripBorders()
LegalClip()
Sampler(length=24)

Then this was the last one you had posted and said at cq 70.5
131 minute movie on one cd Changed the order of Merg and Grip
and added Lanczos resize. The change increased file size on my test

My Test - We Were Soldiers 138 Minutes 14.8 MB


Now I understand that your movie was 131 and mine was 138 min
but my tests were done at the default setting of the kvcdx3 mpeg 1.

So in "theory" if you fit 131 min at cq of 70, i should be able to fit this movie at cq of 60 on one cd. ????


Predictor shows that I need 11.38 (test points 138 error 2)

This is what I got when I took out Lanczos Resize, since it was not in the first script and changed the order of Merg and Clip.

Mpeg2Source("C:\Ripping\We Were Soldiers\Soldier's.d2v")
LegalClip()
GripCrop( width=528, height=480, overscan=2 )
GripSize()
mergechroma(blur(1.58))
mergeluma(blur(0.2))
SpaceDust()
FluxSmooth(7,7)
NoMoSmooth(40,1,6,1,3,false)
Convolution3d(preset="movieHQ")
GripBorders()
LegalClip()
Sampler(length=24)

My Test - We Were Soldiers 138 Minutes 13.6 MB

This file was put through dvd2avi 1.76 by LOLI.J and forced film
was used cause it was Film99% . Avisynth 2.07 and TmpgEnc 2.58 and the newest kvcdx3 mpeg 1 template. I guess I am stumpped as to how you got it on one cd at 70.5 and I am not even close at cq 60. I also understand that we were soldiers is a little more action but am I missing something. So can someone please tell me what I am doing wrong.

Hi Tutankhamun,

You can't assume the same CQ values I used, even if your movie is the same length. It all depends on the content of the movie. That's why we have to use prediction for every movie.

-kwag

Hi people.

I red through this thread and immediately applied those Merge Luma/Chroma things.

I used this script:

But i had to find out that there was no reduction of the filesize.I encoded the Lord of the Rings part 1 Sample.The output was 24,3MB (CQ=71,2 CD's).

With my previous script I had a output of 24,3MB as well.

This is my previous script.

What am I doing wrong???

Nothing :D
It probably means that this type of movie you're encoding can't compress more with the chroma and luma settings.

-kwag

OK.

But there's still one question.

I posted 2 scripts,
The samples of both scripts have the same Filesize.But which one provides better quality???

-Bchteam :P

Only you can tell 8O :wink:

-kwag