Clarification for masking overscan/noise?

Schlockmaster · #1 07-23-2023, 03:21 PM

I've read the forum guide about masking overscan several times. The resize+crop filter in VDub is simple to use, no problems there. I can't quite wrap my head around the proper proportions and number of pixels that should be masked though.

The guide states "Ideally you hide 8 pixels removed from each side" then at the end it says the proper way is to remove twice as many pixels from the x axis as you remove from the y axis. It's pretty confusing and seems contradictory.

On the other hand, Hushpower suggests masking in proportion to the 4:3 ratio is ideal and this seems like the logical approach to me as well. I also remember a post where I think msgohan said masking an odd number at the bottom is okay but never mask an odd number on top because it will screw up the interlacing, and only even numbers on the sides.

To add further confusion, I've seen a few posts that seem to imply (but don't outright say) masking even numbers is bad if it equals an odd number when divided by 2 (ex. masking a total of 10 on the x axis, you'll have 5 pixels masked on each side when the active area is re-centered, not good).

So what is the "correct" way (if there even is one)?

This relates back to the capture I'm currently working on. It has strange noise at the bottom unlike any I've seen before. It can be mostly hidden if I mask off 8 pixels, but there is still a tiny bit protruding upwards and jumping around towards the right side. Masking 9 pixels will completely hide it, but I'm unsure if this will mess something up since most people say to only mask in multiples of 2. This noise also seems to be missing from the very bottom lines of the picture. I'll attach clips. Curious if anyone has seen noise like this before.

clip 1.avi

This may be asking too much for one post, but I'd also like pointers on what filters to experiment with to improve this video (VDub and/or Hybrid). In clip 2 the aliasing(?) around her hair is particularly annoying.

clip 2.avi

Hushpower · #2 07-23-2023, 07:53 PM

I detect a mixing together of two concepts, @Schlock. My understanding of masking is that it is plonking/overlaying of black masking bars on top of the video to cover up the edge blemishes. The original frame size eg 720x576 remains, it's just that there are black bars around all sides. It probably doesn't matter how much is masked, although you'd want to make each opposite side that same so at least you have some symmetry. Particularly on YT, masks, particularly asymmetric ones, look plain ugly, IMO.

As for masking in multiples of 2, I can't see why that would be necessary. the masking is going to be burned into the original video frame so I don't see it being a problem.

Cropping, on the other hand, is cutting the rubbish away. When you do that, you end up with a image going right out to the edge of the screen and you have to do it in proportion so you don't get distortion. You could leave the reduced-size frame (say you took 40 off the sides and 30 off the top and bottom, leaving a 680x546 frame) or you could upscale it back to 720x576. Either way, you have to crop in the 4:3 proportion, and the final frame size dimensions have to be divisible by 2 for the H264 encoder to work.

I hope I have that right!

Hushpower · #3 07-23-2023, 08:03 PM

Clip 1: the noise running along the bottom is "head-switching" noise. All VHS-type captures have it to a certain extent. I think I have read it can be adjusted, to a certain extent, by the mechanicals in a VCR, but yours doesn't look bad to me.

Clip 2: if you're talking about the jaggies of the hair, it just needs to be deinterlaced. Try the Deinterlace filter on Yadif, Double Frame Rate, TFF.

Or Hybrid, you can use the mighty QTGMC, the be-all-and-end-all of deinterlacers.

Nice car; I want one!

Schlockmaster · #4 07-23-2023, 09:01 PM

I understand the difference between masking and cropping. When you add the resize filter in VDub it serves as a canvas that will always be the same resolution no matter how much of the base video you crop. Then it masks over the parts that are cropped and the resolution of the base canvas remains unchanged. I understand that a regular ol' crop removes pixels from the whole canvas and will mess up the aspect ratio if not done correctly. I suck at explaining things, but I assure you we're on the same page here.

According to the guide and people giving masking advice, it does indeed matter how much is masked and has been stated several times to only do multiples of 2. Just like you, I also assume it doesn't matter. But the pros say otherwise. All this confusion is exactly why I made this thread.

I'm also aware of head-switching noise and was asking about this particular noise because it's different from what I've seen in my other captures. It seems to be absent from the very bottom of the video and there is a small bit that extends higher than the rest. I'd have to mask 9 pixels to completely hide it and that's what led me to ask for clarification about this stuff.

I actually did deinterlace that clip with QTGMC before posting this thread, thinking that would smooth it out. But nope, still looked jaggy.

Hushpower · #5 07-23-2023, 11:16 PM

Can you post the raw clip 2? Odd that QTGMC didn't deint it properly. Be interesting to see how it comes out with AVISynth + QTGMC (assuming you did it with Hybrid).

Where's this guide to cropping? I'm lazy, it'll save me looking!

Hushpower · #6 07-24-2023, 12:02 AM

I ran AVISynth+QTGMC on the already-deinterlaced clip and it cleared up the jaggies, but not as well as YADIFx2 TFF.

Feedbucket · #7 07-24-2023, 08:42 AM

Multiples of two only matters for cropping horizontally off the top and bottom. It's to retain field order and parity for deinterlacing.

Schlockmaster · #8 07-24-2023, 03:46 PM

Quote:

Originally Posted by Hushpower

Can you post the raw clip 2? Odd that QTGMC didn't deint it properly. Be interesting to see how it comes out with AVISynth + QTGMC (assuming you did it with Hybrid).

Where's this guide to cropping? I'm lazy, it'll save me looking!

The clips I posted are both raw and unaltered. I tried it with QTGMC Vapoursynth instead of Avisynth, not sure what the difference is between them. Tinkering with the settings would probably make a big difference if I knew what I was doing

Used the "slower" preset and enabled "bob".

Here's the guide

Quote:

Originally Posted by Feedbucket

Multiples of two only matters for cropping horizontally off the top and bottom. It's to retain field order and parity for deinterlacing.

Don't you mean vertically off the top and bottom?

Hushpower · #9 07-24-2023, 08:20 PM

I recall reading something about QTGMC having problems with vertical (or was it horizontal??) lines. Jagabo over at Videohelp suggested code something like "turn right.... QTGMC...turn Left". That might help with the hair.

Thanks for the guide; apart from the technical aspects of cropping, as alluded-to by Feedbucket (looking forward to their answer there), I don't agree with that masking at all. I don't like black borders on my videos, full stop. They just look tacky, in my view. These days, screens display (or can, when set) the whole picture, and I think we should use them that way. What's the point of having a whopper TV if you've got the viewable video in a black box? You can't see behind the black borders, so it may as well not be there.

Hushpower · #10 07-24-2023, 08:22 PM

Quote:

Originally Posted by Feedbucket

Multiples of two only matters for cropping horizontally off the top and bottom. It's to retain field order and parity for deinterlacing.

I understand one of the cardinal rules is that you must deinterlace before cropping (so you don't mess up the fields, as you say).

lordsmurf · #11 07-24-2023, 08:40 PM

2x2 is also for encoding DCT.

Hushpower · #12 07-24-2023, 09:17 PM

Quote:

Originally Posted by LS

2x2 is also for encoding DCT.

Pardon my ignorance, LS, what's DCT?

Schlockmaster · #13 07-24-2023, 09:33 PM

Quote:

Originally Posted by Hushpower

I recall reading something about QTGMC having problems with vertical (or was it horizontal??) lines. Jagabo over at Videohelp suggested code something like "turn right.... QTGMC...turn Left". That might help with the hair.

Thanks for the guide; apart from the technical aspects of cropping, as alluded-to by Feedbucket (looking forward to their answer there), I don't agree with that masking at all. I don't like black borders on my videos, full stop. They just look tacky, in my view. These days, screens display (or can, when set) the whole picture, and I think we should use them that way. What's the point of having a whopper TV if you've got the viewable video in a black box? You can't see behind the black borders, so it may as well not be there.

Thanks, I'll look into that. I did another deinterlace to compare with the original and did notice a bit of an improvement in the hair.

I'm just masking to eliminate the head-switching noise and to center the picture as much as possible. There are already black bars in the original captures, so masking isn't really changing much except eliminating the obvious distractions. It's 4:3 and the sides of the screen are going to have black bars regardless. The small masked borders on the top and bottom are miniscule and not noticeable to me on my 50" 4K TV. It's much better than having a strip of noise there. I hide the bare minimum, which has so far only been 6-8 pixels on the bottom.