x264: aq-mode 2 recommendations?

[Forteen88]

I wonder, in what case(s) is --aq-mode 2 recommended to use than the default (---aq-mode 1)? I get higher SSIM with --aq-mode 2 than default (--aq-mode 1).
I guess that one of the case is when not using mbtree:
https://forum.doom9.org/showthread.php?t=152904

Have anyone noticed any artifacts with aq-mode 2 with disabled mbtree?

Thanks

[RainyDog]

I've used aq-mode 2 (what was aq-mode 4 before it was committed) with no-mbtree for the past 6 months and prefer the results to aq-mode 1 which I used to use at a strength of 0.7 to 0.9 depending on source. I mostly encode film/live action Blu-Ray's 1080p>720p, at CRF 17-19 again depending on the source.

I'm just experimenting with mb-tree again and I'm still not confident with it. I can live with poorer fades, but there appears to be too much of a trade off between the parts it improves and those that suffer as (seemingly) a consequence. It seems to reallocate too many bits into making the already transparent enough frames more transparent at the cost of dark scenes and low luminace areas.

[LoRd_MuldeR]

I wonder, in what case(s) is --aq-mode 2 recommended to use than the default (---aq-mode 1)? I get higher SSIM with --aq-mode 2 than default (--aq-mode 1).

As the original commit message says:

New AQ algorithm option
"Auto-variance" uses log(var)^2 instead of log(var) and attempts to adapt strength per-frame.
Generates significantly better SSIM; on by default with --tune ssim.
Whether it generates visually better quality is still up for debate.
Available as --aq-mode 2.

So you'll have to make your own experiments and decide whether AQ mode 2 generates "visually better quality" for your eyes or not...


I guess that one of the case is when not using mbtree:
https://forum.doom9.org/showthread.php?t=152904

AQ mode 2 has been adapted for MB-Tree RC, so it should work alright with MB-Tree enabled:
http://git.videolan.org/gitweb.cgi?p=x264.git;a=commit;h=74f2fd65af6840aced812e295e1a551ba295459f

[Forteen88]

I'm just experimenting with mb-tree again and I'm still not confident with it. I can live with poorer fades, but there appears to be too much of a trade off between the parts it improves and those that suffer as (seemingly) a consequence. It seems to reallocate too many bits into making the already transparent enough frames more transparent at the cost of dark scenes and low luminace areas.Even with --qcomp 0.75? Thanks for the --aq-mode 2 info, I'll try doing some tests with it as L.M. recommended (higher SSIM maybe isn't as good as I think it is). I mostly encode HD-video (especially 720p) also.

[RainyDog]

Even with --qcomp 0.75? Thanks for the --aq-mode 2 info, I'll try doing some tests with it as L.M. recommended (higher SSIM maybe isn't as good as I think it is). I mostly encode HD-video (especially 720p) also.

Would a higher q-comp allocate more bits to low complexity, low luminance scenes? I thought a lower q-comp value would do that. I know lower q-comp increases mb-tree strength and visa versa, but I still thought that q-comp behaved like it always did despite mb-tree. And that higher values favoured high motion, high complexity frames?

Good luck with aq-mode 2. I prefer it over aq-mode 1 now, with or without mb-tree.

[Forteen88]

Would a higher q-comp allocate more bits to low complexity, low luminance scenes? I thought a lower q-comp value would do that. I know lower q-comp increases mb-tree strength and visa versa, but I still thought that q-comp behaved like it always did despite mb-tree. And that higher values favoured high motion, high complexity frames?

Good luck with aq-mode 2. I prefer it over aq-mode 1 now, with or without mb-tree.Sry, yeah, you're probably right about that (concerning mbtree).
Thanks, I've already almost done my first --aq-mode 2 720p of a full video :)

EDIT: I heard that --aq-mode 2 sometimes have trouble with showing dark scenes correct (blocking).