Disabled mbtree needs higher qcomp?

[Forteen88]

Hi, I wonder, when disabling mbtree, should I increase qcomp value (to like 0.8)? I read some people recommends that.

This is for HD-video.

Thanks

[LoRd_MuldeR]

Generally, you shouldn't mess with the "qcomp" option, unless you understand exactly what it does and have a very good reason to change the default.

Rumors you have heard somewhere are not a good reason to make your decision ;)

[Forteen88]

@LoRd_MuldeR
That's why I asked here, with hopes of some x264-guru (ie who understand exactly what qcomp does) answering :)

[LoRd_MuldeR]

The basic idea behind "qcomp" (quantizer curve compression) is to lower the quality in "high complexity" scenes and raise the quality in "low complexity" scenes.

Quantizer curve compression, or “qcomp”, is a very old and simple idea. Lower the quality in areas of the video with high complexity and raise the quality in areas with low complexity. The original implementation dates at least back to the original libavcodec ratecontrol. There are three justifications for this:

High-complexity scenes generally have high motion, and one is less likely to notice quality loss in a high motion scene, since fine detail is impossible to see in motion anyways.
High-complexity scenes are extraordinarily costly bit-wise, so even if it makes quality somewhat worse, it might be worth it to save those bits to use elsewhere.
Each frame in a high-complexity scene is not referenced very far into the future since there is a great deal of change between each frame, so even from a PSNR perspective, one should allocate fewer bits to those frames as opposed to frames which are nearly static.
x264 uses qcomp pretty much unchanged except that it performs a gaussian blur over the frame complexities to avoid too much local fluctuation between quantizers.float f_qcompress; /* 0.0 => cbr, 1.0 => constant qp */

If you raise the "--qcomp" value towards 1.0, you actually lower the effect of the qcomp algorithm! This means that "high complexity" scenes will get better quality - which may look better at a first glance (especially if you step through those scenes frame-by-frame). However, it also means that "high complexity" scenes will cost a lot more bits! Those bits are will then be missing in other places, of course! Consequently, the overall quality may easily become worse. Bummer!

As with most parameters (especially Psy optimizations), neither very high nor very low values will give optimal results. It's all about finding the sweet spot! And a "--qcomp" value of 0.6 obviously is what the developers consider to be a good default. If a higher (or lower) value would generally be beneficial, they would have made that value the default! This doesn't mean that a somewhat higher (or lower) value couldn't be beneficial for your particular source and/or your personal preferences. But if you decide to mess with "--qcomp", your decision should be based on intensive testing rather than rumors. And if you are too lazy for running an in-depth test, just trust the developer's decision and stick with the default value ;)

(BTW: The basic principle behind the classic "qcomp" algorithm and the "MB-Tree" algorithm is the same. The difference is that qcomp measures "complexity" for complete frames while MB-Tree tracks the "complexity" of individual macro-blocks)

[mandarinka]

I read some people recommends that.

I actually would say the other way around - increase if you are encoding with mbtree. I only encode anime though. Your take might vary.

[Caroliano]

I'm with mandarinka in this one, it is the other way around. I also encode mostly anime and find that the default qcomp value makes mbtree and AQ too agressive. I aways use --qcomp between 0.7 and 0.8 for high bitrate anime. There were threads discussing here before if the default was too low, but no clear consensus was reached and thus the default remained unchanged. I'm on the opinion it is too low. And if you are working in a situation where your low-complexity scenes are already near perfection, and you want even higher quality, raising qcomp is probably a good idea.