Favorable sources for setting Trellis=2 using x264

[Avenger007]

I know it's generally better to set trellis=1 (On Macroblock) instead of trellis=2 (Always On).
What sources would give a significant or perceivable difference between trellis=2 and trellis=1 using x264?

Note: By sources I mean characteristics of video sources, for example: blocky, fast motion, dark, low bitrate, low resolution. etc...

[Dark Shikari]

Trellis 2 tends to favor removal of fine details in dark/flat blocks. So it basically opposes the effect of adaptive quantization.

[Avenger007]

So wouldn't that result in dark/flat areas looking blocky, I thought trellis=2 would usually improve quality even though the improvement may be imperceivable for some sources.

[Dark Shikari]

So wouldn't that result in dark/flat areas looking blocky, I thought trellis=2 would usually improve quality even though the improvement may be imperceivable for some sources.That's the reason Trellis-2 isn't recommended; it tends to remove fine details unfairly in some cases.

[Avenger007]

Thanks, I didn't know that. I guess I was thinking that higher values for x264 settings usually means better, that's because I'm just a :helpful:
I think I'll stick with trellis=1 for all sources, unless there are sources for which I shouldn't (but I don't know of any :)).

[Dark Shikari]

Thanks, I didn't know that. I guess I was thinking that higher values for x264 settings usually means better, that's because I'm just a :helpful:
I think I'll stick with trellis=1 for all sources, unless there are sources for which I shouldn't (but I don't know of any :)).Also, to explain the reason why:

The purpose of trellis is to improve the quality-per-bitrate during quantization. It does this pretty effectively without much loss. However, when deciding what block types to use, x264 normally doesn't use trellis, not just because its slow, but also because this creates a sort of feedback effect--it chooses the blocktype, sometimes, which trellis decimated the most. While this is technically better for PSNR, moving too many bits away from the current macroblock is often bad for visual quality.

[Avenger007]

Also, to explain the reason why:

The purpose of trellis is to improve the quality-per-bitrate during quantization. It does this pretty effectively without much loss. However, when deciding what block types to use, x264 normally doesn't use trellis, not just because its slow, but also because this creates a sort of feedback effect--it chooses the blocktype, sometimes, which trellis decimated the most. While this is technically better for PSNR, moving too many bits away from the current macroblock is often bad for visual quality.

Since we're on the subject of decimating, I have DCT decimation disabled (i.e. --no-dct-decimate) when trellis=1. Is that recommended or not?
Thank a lot for your advice. :thanks:

[Dark Shikari]

Since we're on the subject of decimating, I have DCT decimation disabled (i.e. --no-dct-decimate) when trellis=1. Is that recommended or not?
Thank a lot for your advice. :thanks:DCT decimation "decimates" P-macroblocks in the case that the residual coded is extremely small (that is, probably not noticeable). This will often save bits with minimal quality decrease, since it basically stabilizes areas of the frame that are changing in extremely minor ways (grain is usually complex enough that if your settings are high enough to retain the grain, DCT decimation won't affect it). Its generally best to leave on, IMO.

[mitsubishi]

Its generally best to leave on, IMO.
Is it a "safe"ish option then? I tend to avoid it (I use --no-dct-decimate) since I thought it was one of those options which might on occasion make a bad decision and wreck the encode.

[Dark Shikari]

Is it a "safe"ish option then? I tend to avoid it (I use --no-dct-decimate) since I thought it was one of those options which might on occasion make a bad decision and wreck the encode.No, it will never "wreck" an encode. Its generally best to leave it on.

[mitsubishi]

Thanks for the input.

I was basing that caution on this:

http://trac.handbrake.fr/wiki/x264Options
No DCT Decimate (no-dct-decimate)

To save space, x264 will "zero out" blocks when it thinks they won't be perceptible by the viewer. This negligibly reduces quality, but in rare cases it can mess up and produce visible artifacts. This situation can be alleviated by telling x264 not to decimate DCT blocks. It increases quality but also bitrate/file size, so if you use it when you've specified a target bitrate you will end up with a worse picture than without it. However, when used with constant quality encoding, or if you boost the average bitrate to compensate, you might get a better result.