Incorrect levels when compressing to x264

[bilditup1]

When converting to x264, the levels appear to be significantly boosted in comparison to the source. I have no idea what is happening to cause this, and am not even sure if it's a playback issue or what.
The AviSynth script is simply:

AVISource("E:\Capture\lm3_excerpt.avi", audio=false).AssumeFPS(30000,1001)
Load_Stdcall_Plugin("C:\Video\MeGUI\tools\avisynth_plugin\yadif.dll")
Yadif(order=-1)
ConvertToYV12()

Both are using the same decoders when being played back using MPC-HC. By default I was under the impression that Huffyuv is capturing in regular Y range (16-235), and that ConvertToYV12() isn't supposed to do anything to change this (again, by default). So I'm baffled.
Attached is the compressed mp4. The original source Huffyuv file may be found here.

NB: Yadif() was chosen by MeGUI's analyse function in its AVS Script Creator, which only had 20 seconds or so to work with. This is not meant to be a blanket endorsement of using Yadif(). Additionally, this level problem occurs with other deinterlacers, too, so I doubt that's the root cause.

[vivan]

Sounds like playback issue because I don't see any difference here (frame 277):
http://i.imgur.com/SQG9zPL.png
http://i.imgur.com/MnoUVgD.png

If you're using EVR renderer in MPC-HC - disable all video postprocessing options in the GPU control panel. This stuff works only when video is decoded using GPU, which is why this happens only with H.264 encode.
Or just install madVR and switch to it, it bypasses all this nonsense

[sanlyn]

Quote:

Originally Posted by bilditup1

When converting to x264,

Technically one doesn't "convert" from one codec to another. Rather, the huffyuv AVI was encoded to h264. "x264" isn't a codec, it's an encoder. "h264" is the codec.

Quote:

Originally Posted by bilditup1

the levels appear to be significantly boosted in comparison to the source.

Levels in the avi and the source are the same, in both YUV and when displayed as RGB. Use histograms, vectorscopes, waveforms, pixel samplers, etc., to check levels and other elements.

From the same frame in both samples:

AVI frame 451, YUV:


mp4 frame 451, YUV


AVI frame 451, RGB display:


MP4 frame 451, RGB display:


One major difference between the two samples is that the AVI is YUY2, the mp4 is YV12, and the mp4 is softer with some encoding noise visible as a minutely thicker layer of luma "speckles" in the RGB Waveform (you have to look pretty close at the RGB luma graph to see it).

The script has some oversights or could be written differently. For example:

AVISource("E:\Capture\lm3_excerpt.avi", audio=false).AssumeFPS(30000,1001).

Unless your frame rate is off, AssumeFPS isn't needed. The video is already 29.97fps, as MediaInfo and Virtualdub both report. If you really need AssumeFPS it could also be written this way:

AVISource("E:\Capture\lm3_excerpt.avi", audio=false).AssumeFPS("NTSC_video").

The Load_Stdcall_Plugin statement uses an obsolete plugin loader designed for earlier versions of Avisynth. most people nowadays would replace "Load_Stdcall_Plugin" with the newer "LoadCPlugin". Both will work.

The statement below would present problems in many circumstances, but doesn't matter here for several reasons:

Yadif(order=-1)

order=1 assumes the field priority to be the same as Avisynth's default, which is BFF. The video is more properly TFF, which is the way most VCRs play VHS. Field priority is best determined by inspection, regardless of how the tape is played. One usual exception is DV-AVI (BFF). In this case the VHS is progressive/field-blended with periodic duplicate fields, captured as interlaced. The tape's incorrect oddball frame structure was discussed in the earlier thread. Yadif's default processing mode is mode=0, which keeps the first-priority field, interpolates new data from the other field and discards that other field, then outputs the same number of frames and frame rate as the original. In that respect both samples aren't exactly alike because yadif has discarded alternate fields after interpolation. If you eliminate duplicate fields or frames, the video's original frame rate was 23.976. Unfortunately a lot of the blending can't be eliminated after the earlier guys butchered the original.

Full frame, double frame rate (all fields retained) true deinterlacing is modes 1 or 3. http://avisynth.org.ru/yadif/yadif.html

There are two ways to tell yadif to use the correct field polarity:

AssumeTFF().Yadif()

or:

Yadif(order=1)

Adjust accordingly for BFF where needed.

ConvertToYV12()

The video was played and captured as interlaced. ConvertToYV12 assumes progressive. In a true interlaced\telecined video, this statement would screw up colors. For interlaced/telecine, use:

ConvertToYV12(interlaced=true)

Quote:

Originally Posted by bilditup1

Both are using the same decoders when being played back using MPC-HC

Not likely. The same decoder can't decompress two different codecs. As vivan remarks earlier, I think you mean "renderer". Anyway, both samples look the same to me, but the mp4 looks filtered. They match each other in my MPC-BE, VLC, and MPC. What MPC-HC does with two dissimilar videos of the same content, I wouldn't know. I don't use it.

Quote:

Originally Posted by bilditup1

I was under the impression that Huffyuv is capturing in regular Y range (16-235)

Nope. huffyuv doesn't capture anything. It's a compressor. It sets no Y range limit. That's done by your capture device and/or software, or previous processing.

[vivan]

Quote:

Originally Posted by sanlyn

ConvertToYV12()

The video was played and captured as interlaced. ConvertToYV12 assumes progressive. In a true interlaced\telecined video, this statement would screw up colors. For interlaced/telecine, use:

ConvertToYV12(interlaced=true)

He is using it after deinterlacing, not before, this usage is correct.

[sanlyn]

True. Just a caution to be careful at which point color conversions are made. I tend to slip up myself often.

[bilditup1]

Quote:

Originally Posted by vivan

Sounds like playback issue because I don't see any difference here (frame 277):
http://i.imgur.com/SQG9zPL.png
http://i.imgur.com/MnoUVgD.png

OK, I see. Thanks for pointing this out.

Quote:

If you're using EVR renderer in MPC-HC - disable all video postprocessing options in the GPU control panel. This stuff works only when video is decoded using GPU, which is why this happens only with H.264 encode.
Or just install madVR and switch to it, it bypasses all this nonsense

I'm using VMR-9, not EVR, but - this turned out to be the issue! The (windowed) mode of VMR-9 was allowing the GPU to muck with the levels. I'm not sure why huffyuv isn't messed with by the video card but x264 is, but there ya go. (Clarification: I did not think that on a card this old, avc decoding was possible.) I switched to (renderless) mode and all was well.

Aside - unfortunately it looks like madVR won't work on the Radeon 9000 or 9800. I tried as soon as I set up this computer, but it seems like the GPUs are just too old.

Quote:

Originally Posted by sanlyn

Technically one doesn't "convert" from one codec to another. Rather, the huffyuv AVI was encoded to h264. "x264" isn't a codec, it's an encoder. "h264" is the codec.

They were careless grammatical errors - I understand that x264 is not a codec, and that videos are encoded rather than generically 'converted'.

Quote:

Levels in the avi and the source are the same, in both YUV and when displayed as RGB. Use histograms, vectorscopes, waveforms, pixel samplers, etc., to check levels and other elements.

From the same frame in both samples:

AVI frame 451, YUV:


mp4 frame 451, YUV


AVI frame 451, RGB display:


MP4 frame 451, RGB display:

Thanks for confirming this for me, and for the advice. Which program did you use to make these graphs?

Quote:

One major difference between the two samples is that the AVI is YUY2, the mp4 is YV12, and the mp4 is softer with some encoding noise visible as a minutely thicker layer of luma "speckles" in the RGB Waveform (you have to look pretty close at the RGB luma graph to see it).

I assumed there'd be minor differences like that, but thanks for explaining exactly what those would be.

Quote:

The script has some oversights or could be written differently. For example:

AVISource("E:\Capture\lm3_excerpt.avi", audio=false).AssumeFPS(30000,1001).

Unless your frame rate is off, AssumeFPS isn't needed. The video is already 29.97fps, as MediaInfo and Virtualdub both report. If you really need AssumeFPS it could also be written this way:

AVISource("E:\Capture\lm3_excerpt.avi", audio=false).AssumeFPS("NTSC_video").

The Load_Stdcall_Plugin statement uses an obsolete plugin loader designed for earlier versions of Avisynth. most people nowadays would replace "Load_Stdcall_Plugin" with the newer "LoadCPlugin". Both will work.

Yes, again, as I said in the NB, the script was generated using MeGUI's AVS Script Creator, and I know that it uses obsolete terminology (and obsolete plugins, frankly) when it creates said scripts. I just didn't want to take the time to write one myself, and also wanted to make use of its analysis function instead of figuring out how best to deinterlace/ivtc this clip, especially since you mentioned how messed up it was.

Quote:

The statement below would present problems in many circumstances, but doesn't matter here for several reasons:

Yadif(order=-1)

order=1 assumes the field priority to be the same as Avisynth's default, which is BFF. The video is more properly TFF, which is the way most VCRs play VHS. Field priority is best determined by inspection, regardless of how the tape is played. One usual exception is DV-AVI (BFF). In this case the VHS is progressive/field-blended with periodic duplicate fields, captured as interlaced. The tape's incorrect oddball frame structure was discussed in the earlier thread. Yadif's default processing mode is mode=0, which keeps the first-priority field, interpolates new data from the other field and discards that other field, then outputs the same number of frames and frame rate as the original. In that respect both samples aren't exactly alike because yadif has discarded alternate fields after interpolation. If you eliminate duplicate fields or frames, the video's original frame rate was 23.976. Unfortunately a lot of the blending can't be eliminated after the earlier guys butchered the original.

Full frame, double frame rate (all fields retained) true deinterlacing is modes 1 or 3. http://avisynth.org.ru/yadif/yadif.html

There are two ways to tell yadif to use the correct field polarity:

AssumeTFF().Yadif()

or:

Yadif(order=1)

Adjust accordingly for BFF where needed.

Thanks for explaining how to use Yadif properly, what the defaults are, and how VHS works. I have used its bob functions before, but not in quite a while, and didn't bother to refresh my memory of check any of this. In the future, I'm gonna stop being lazy, and write these myself

Quote:

ConvertToYV12()

The video was played and captured as interlaced. ConvertToYV12 assumes progressive. In a true interlaced\telecined video, this statement would screw up colors. For interlaced/telecine, use:

ConvertToYV12(interlaced=true)

But the statement is at the end of the script, by which point the video isn't interlaced anymore...?

Quote:

Not likely. The same decoder can't decompress two different codecs. As vivan remarks earlier, I think you mean "renderer". Anyway, both samples look the same to me, but the mp4 looks filtered. They match each other in my MPC-BE, VLC, and MPC. What MPC-HC does with two dissimilar videos of the same content, I wouldn't know. I don't use it.

I wasn't talking about the renderer, although that was also kept unchanged (VMR-9 windowed). I meant the same decoder suite (like ffdshow or the 'successor' lavfilters suite). In this case, I was using external ffdshow and turned off the internal lavfilters that MPC-HC uses. IOW I was using ffdshow's Huffyuv decoder and ffdshow's avc decoder.

Quote:

Nope. huffyuv doesn't capture anything. It's a compressor. It sets no Y range limit. That's done by your capture device and/or software, or previous processing.

Aha. Thanks for explaining this. I'm assuming that by default Vdub sets the limit to 16-235? Otherwise I'm not sure what 'extend luma white/black point' do.

Aside - I'm not sure what I need to do to prove to you that I'm not a complete newbie here, even if I sometimes use imprecise terminology or am not familiar with the finer points of interlaced video encoding. I do appreciate your help a lot sanlyn, but it would be nice to be spoken to as if I'm not a complete ignoramus

[sanlyn]

Quote:

Originally Posted by bilditup1

Which program did you use to make these graphs?

The YUV histogram is Avisynth's built-in Histogram function, which has several options. I used Histogram(mode="levels"). The RGB waveform graph was made in VirtualDub with ColorTools1.4, which also has several modes. With any of these graphs, be aware that black borders will almost always show as zero black. Often I crop the borders temporarily just to keep them from confusing matters. Also note that the ColorTools plugin doesn't work in Win7. Avisynth has other graphing plugins, some of which are a hassle to use.

Quote:

Originally Posted by bilditup1

as I said in the NB, the script was generated using MeGUI's AVS Script Creator, and I know that it uses obsolete terminology (and obsolete plugins, frankly) when it creates said scripts. I just didn't want to take the time to write one myself, and also wanted to make use of its analysis function instead of figuring out how best to deinterlace/ivtc this clip, especially since you mentioned how messed up it was.

The VHS is a mix of field blend and pulldown which became hard-coded in previous processing. Deinterlacing in itself won't fix telecine or remove blends. Many of the blends will always be in that particular video. The closest I could get to the original progressive 23.976 speed was:

AssumeTFF().QTGMC(whatever presets)
sRestore(frate=23.976)

The posted encode added 3:2 pulldown flags to get 29.97fps playback. The encoder I used was either TMPGenc Plus 2.5 or TMPGEnc Video Mastering Works, I forget which.

Quote:

Originally Posted by bilditup1

Thanks for explaining how to use Yadif properly, what the defaults are, and how VHS works. I have used its bob functions before, but not in quite a while, and didn't bother to refresh my memory of check any of this. In the future, I'm gonna stop being lazy, and write these myself

I've never been a fan of MEGUI's scripting. On the few occasions I've used it I write a complete script in AVisynth and save the work as AVI, then just use MEGUI's oddball auto script to do nothing more than open the AVI.

Quote:

Originally Posted by bilditup1

But the statement is at the end of the script, by which point the video isn't interlaced anymore...?

True. As I noted earlier, this was FYI on color conversions. Some people aren't aware of the difference. Or some people just forget to type it correctly ...like me, for instance.

Quote:

Originally Posted by bilditup1

Thanks for explaining this. I'm assuming that by default Vdub sets the limit to 16-235? Otherwise I'm not sure what 'extend luma white/black point' do.

VDub capture doesn't set 16-235 limits. The "extend" feature acts something like AGC for the full range. As the name implies, it doesn't limit -- it expands. Don't use it. If you have to adjust, use the "Levels" feature instead, which usually hooks into the capture card's proc amp controls if they exist.

Quote:

Originally Posted by bilditup1

Aside - I'm not sure what I need to do to prove to you that I'm not a complete newbie here, even if I sometimes use imprecise terminology or am not familiar with the finer points of interlaced video encoding. I do appreciate your help a lot sanlyn, but it would be nice to be spoken to as if I'm not a complete ignoramus

Sorry if it came off as such. You've obviously been around this work for a while.

[bilditup1]

Quote:

Originally Posted by sanlyn

The YUV histogram is Avisynth's built-in Histogram function, which has several options. I used Histogram(mode="levels"). The RGB waveform graph was made in VirtualDub with ColorTools1.4, which also has several modes. With any of these graphs, be aware that black borders will almost always show as zero black. Often I crop the borders temporarily just to keep them from confusing matters. Also note that the ColorTools plugin doesn't work in Win7. Avisynth has other graphing plugins, some of which are a hassle to use.

Thanks for the tips there, I'll definitely be trying to get into them.

Quote:

The VHS is a mix of field blend and pulldown which became hard-coded in previous processing. Deinterlacing in itself won't fix telecine or remove blends. Many of the blends will always be in that particular video. The closest I could get to the original progressive 23.976 speed was:

AssumeTFF().QTGMC(whatever presets)
sRestore(frate=23.976)

The posted encode added 3:2 pulldown flags to get 29.97fps playback. The encoder I used was either TMPGenc Plus 2.5 or TMPGEnc Video Mastering Works, I forget which.

Did you mean this as a general principle or only for this extremely messed up Les Miz tape?

Quote:

I've never been a fan of MEGUI's scripting. On the few occasions I've used it I write a complete script in AVisynth and save the work as AVI, then just use MEGUI's oddball auto script to do nothing more than open the AVI.

Yeah, it's failed me several times now. Really it's just for job control, as I don't use any of the automation.

Quote:

True. As I noted earlier, this was FYI on color conversions. Some people aren't aware of the difference. Or some people just forget to type it correctly ...like me, for instance.

Yeah, just saw the follow-up there. Gotcha.

Quote:

VDub capture doesn't set 16-235 limits. The "extend" feature acts something like AGC for the full range. As the name implies, it doesn't limit -- it expands. Don't use it. If you have to adjust, use the "Levels" feature instead, which usually hooks into the capture card's proc amp controls if they exist.

Right, I didn't intend to. This makes sense.
I suppose the card captures at 16-235 by default then though? Nobody has mentioned a setting that would possibly change this to full range.

Quote:

Sorry if it came off as such. You've obviously been around this work for a while.

Thanks - sorry if I came off as haughty there, just was slightly annoyed, heh. I still have much to learn, and look forward to more of your sage advice in the future