x264 blurring along outside edge of frame

[simple_simon]

I try to maintain at least mod4 dimensions when cropping while attempting to minimize aspect ratio errors. Sometimes that means not cropping out the black bars completely. However I've noticed that x264 blurs the edges of the frame when there is any amount of black border present (see attached screenshot). I get the same results no matter what settings I use. I don't remember this being an issue before. Is this a new problem with newer builds and is there an older build I can roll back to that didn't have this problem?

[rwill]

It has always been like that.

[mp3dom]

Yep, especially noticeable on video with grain.

[simple_simon]

Hmmm. Strange I never noticed it before. Is there any workaround? Avisynth filter? Is it better to just crop all the borders then and encode mod2? The tricky part with that would be when the aspect ratio isn't consistent, I'd either need to crop out part of the image on wider AR's to make it uniform or only crop to the widest AR and have blurring on the edges when the AR got narrower.

[mp3dom]

The workaround is to remove all the black on the top/bottom bars or have a mod16 black bars. Since you have 16x16 macroblocks, if the macroblock is full black, there's no issue. The problem arise when in a 16x16 macroblock you have some rows with constant black pixels. In those cases, it seems that x264 gets "fooled" and assigns to the macroblock a way higher quantizer, not enough to preserve quality.

[Boulder]

This phenomenon is clearly visible in quite a few commercial encodes as well.

[LoRd_MuldeR]

I think this is a general "problem", if you have a very sharp edge that is not aligned with a 16×16 macro-block boundary.

Accurately preserving a very sharp edge somewhere in the middle of a macro-block would be incredibly expensive, in terms of bit cost.

And since we only have a limited "budget" of bits, these bits probably are spent better somewhere else in the frame

[VoodooFX]

Originally Posted by simple_simon

x264 blurs the edges of the frame when there is any amount of black border present (see attached screenshot)

I don't see black border in your screenshot.

[mp3dom]

Originally Posted by LoRd_MuldeR

I think this is a general "problem", if you have a very sharp edge that is not aligned with a 16×16 macro-block boundary.
[...]
And since we only have a limited "budget" of bits, these bits probably are spent better somewhere else in the frame

Yes, but this problem is not so extremely visible with other encoders, in an extent that with just this kind of screenshot I can clearly say "ok, this was encoded with x264". At bluray bitrates this loss of details is more visible than some bits spent elsewhere because it's persistent on the whole movie.

This and the other issue (bitrate and quality drop after a very complex scene when vbv is in use) are the real two issues of x264. Unfortunately, I guess they will never be (at least) mitigated or (even better) fixed.

[poisondeathray]

It's also one of the side effects of AQ - bits are taken not only from object edges ("halo" of smoothness around objects), but also frame edges ("halo" of smoothness around frame borders). Does anyone recall the AQ tests on the grainy Island trailer ~15 years ago?

You can reduce the effect at a given bitrate by reducing the AQ strength, and/or using higher bitrates

[mp3dom]

Originally Posted by poisondeathray

You can reduce the effect at a given bitrate by reducing the AQ strength, and/or using higher bitrates

Well, I'm not sure about using higher bitrates. This problem is exactly the same (same flattest macroblock problem and detail loss) even at bluray bitrates (I mean, a 35 Mbps average). In a grainy source (not extremely grainy, just a standard film-grain) this is still clearly visible. I can't recall another encoder that at these bitrates exhibits the same problem or with this same amount.

[benwaggoner]

Note that 1080p is not mod16. Internally it is 1088 tall with 8 "dummy" lines at the bottom. So to get mod16 when framing into 1080, you want mod16 from the top, and mod16 -8 from the bottom.

Better to just crop to active image area if targeting a format that allows that.