x264 questions about aq mode

[daimroc]

I would like to know how it really works, because I try to find information and I find nothing.

If I don't wrong, AQ gives more bits to low detail areas if for some reason need more bits. This bits are from other macroblocks of the frame.

If this is true, is it really useful with CRF mode? Because CRF try to ensure a constant picture quality, but if AQ in some ocasions get bits from some macroblock to use this bit in the low detail areas, then the quality in this macroblock wouldn't be the expected quality. Then, In CRF, is a good idea to use AQ?

Other question. I read in a document that aq-mode 1 get bits from other macroblocks of the same frame, but it exists aq-mode 2, that not only it gets bits from the same frame, but also from other frames. Really exits aq-mode 2? Because If I read the longhelp of x264, this option, mode 2, doesn't exist.

This are my questions.



Thanks.
Daimroc.

[LoRd_MuldeR]

AQ moves more bits to "flat" blocks, which greatly reduces blocking and banding in flat areas (like sky and walls). This applies to CRF mode as well!

Compare this:
* http://mplayer.somestuff.org/misc/CQP-vs-CRF.3.htm (CQP mode -> AQ disabled) (http://mplayer.somestuff.org/misc/CQP-vs-CRF.3.htm)
* http://mplayer.somestuff.org/misc/CQP-vs-CRF.4.htm (CRF mode -> AQ enabled) (http://mplayer.somestuff.org/misc/CQP-vs-CRF.4.htm)


And there's no AQ mode 2 anymore since r968. Read the changelog:
commit 173b40c276ee187de31ef630fd207a62de1b2cc7 r968
Author: Jason Garrett-Glaser <darkshikari@gmail.com>
Date: Sat Sep 13 14:03:12 2008 -0700

Move adaptive quantization to before ratecontrol, eliminate qcomp bias
This change improves VBV accuracy and improves bit distribution in CRF and 2pass.
Instead of being applied after ratecontrol, AQ becomes part of the complexity measure that ratecontrol uses.
This allows for modularity for changes to AQ; a new AQ algorithm can be introduced simply by introducing a new aq_mode and a corresponding if in adaptive_quant_frame.
This also allows quantizer field smoothing, since quantizers are calculated beofrehand rather during encoding.
Since there is no more reason for it, aq_mode 1 is removed. The new mode 1 is in a sense a merger of the old modes 1 and 2.
WARNING: This change redefines CRF when using AQ, so output bitrate for a given CRF may be significantly different from before this change!

[Sagekilla]

Yes, it is a good idea to use AQ with CRF. It's good to use it in any mode that you can (Except cq, because constant quantizer is unable to do adaptive quantization.) And you are correct that AQ in it's current state takes bits from areas of high detail in a frame and redistributes it to the low detail areas.

The rationale behind this is that if you take a few bits (Read: It's not taking a HUGE amount of bits, just enough) from a high complexity area, the human eye probably won't notice a big difference. But, by taking those bits and giving them to areas that would likely be given low quants because of their low complexity, you can ensure they don't look blocky to us. It might be PSNR optimal to let those flat areas get high quants, and look like a soupy mess, but it looks ugly so AQ helps in that regard.

I don't know of this other AQ mode you're speaking of though. The original AQ (Haali's version) simply boosted the bitrate allocation for flat areas, resulting in increased bitrate. The new one (Variance AQ) just adjusts a single frame on a macroblock basis, resulting in no extra bits to run it. There's at least one other version I can think of that's floating around (Named "ore" I think, not sure) but I don't know how it works.

[LoRd_MuldeR]

And you are correct that AQ in it's current state takes bits from areas of high detail in a frame and redistributes it to the low detail areas.

I don't think it can be simplified like that. Read the entry from the changelog I quoted above...

I don't know of this other AQ mode you're speaking of though.

He probably referred to the old AQ modes 1 and 2, which have now been merged into mode 1.

There were also several other AQ algorithms in development, but I think none of them ever made it into GIT.

[Dark Shikari]

Because CRF try to ensure a constant picture qualityNo it does not!

CRF, given a set of settings, tries to keep a constant frame-level quality througout a video. It has basically nothing to do with AQ, which redistributes bits within frames.

CRF is not a magical algorithm to "ensure constant quality." It's just an extremely simple ratecontrol algorithm that doesn't have any target bitrate.

[daimroc]

How Lord_Mulder says, in AQ I am refering to the mode. I don't know that mode 1 or 2 are merged in a unique mode.

Then, it's a good idea to use AQ? Because in some proves I notice that I lose detail in some areas, and for me a aq-strength of 0.5 is better than 1.0 and perhaps 0.1 better than 0.5 (this is diffcult to say). So this makes me thought that AQ with CRF like 16 is not a lot of useful. ()

If I use aq-mode 1 and aq-strength is enabled AQ or disabled?

Perhaps this proves are with a particular video, but it is my impression.





Thanks.
Daimroc.

[LoRd_MuldeR]

Then, it's a good idea to use AQ?

YES !!!

How Lord_Mulder says, in AQ I am refering to the mode. I don't know that mode 1 or 2 are merged in a unique mode.

As explained in my first post, there is no AQ mode 1 and 2 anymore since r968. Read the cited changelog entry for details...

If I use aq-mode 1 and aq-strength is enabled AQ or disabled?

AQ mode specified whether AQ is used or not. There are only two "modes" right now: 0 = AQ disabled, 1 = AQ enabled (VAQ).

AQ strength can be used to fine-tune AQ. If you don't specify AQ strength, then the default of 1.0 is used. Of course AQ strength has no effect when AQ is disabled...

[OvejaNegra]

i am encoding hellsing ultimate (anime). The output of the AVS script is ok but in the encoded video,
the edges of some objects in dark scenes look bad (looks like ringing maybe).
I'm using these settings:
program --pass 2 --bitrate 884 --stats ".stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 2 --weightb --partitions all --8x8dct --qpmin 12 --threads auto --thread-input --aq-strength 1.5 --progress --no-psnr --no-ssim --output "output" "input"

After reading this, i think it's AQ taking bits from the objects to the dar parts (like the sky and other stuff)
Should i lower the aq strenght to 0.5? or disable it?
if someone knows hellsing ultimate, well, it has a lot of dark scenes.
Sugestions?
If someone needs screen shots i will post them.
Thanks and sorry for my english.