How to use aq-mod 3&4?

[MichaelRow]

I use taro's x264 build(Known as tmod (http://astrataro.wordpress.com/category/encode/x264/)), and aq-mod 3&4 are included in it. I've read some threads (http://forum.doom9.org/showthread.php?p=1172141#post1172141) about the OreAQ and MixAQ(--aq-mod 3 --aq3-mod 1&2 in this case), afaik, OreAQ and MixAQ can improve the quality in dark scene and flat areas, especially great to encode animes. But, as for aq-mod 4, I have no idea what it exactly does since I didn't find any information while googling.
Besides I don't know what exactly aq3-boundary,i/p/bfator <up:down> means, and how to set the proper parameters for anime encoding.

--aq-mode <integer> AQ method [1]
- 0: Disabled
- 1: Variance AQ (complexity mask)
- 2: Auto-variance AQ (experimental)
- 3: Auto-variance AQ mod1
- 4: Auto-variance AQ mod2
--aq-strength <float> Reduces blocking and blurring in flat and
textured areas. [1.0]
--aq-sensitivity <float> "Center" of AQ curve. [10.0]
- 5: most QPs are raised
- 10: good general-use sensitivity
- 15: most QPs are lowered
--aq-ifactor <float> AQ strength factor of I-frames [1.00]
--aq-pfactor <float> AQ strength factor of P-frames [1.00]
--aq-bfactor <float> AQ strength factor of B-frames [1.00]
--aq2-strength <float> Use 2nd AQ (Haali's AQ) algorithm for support. [0.0]
0.0: no 2nd AQ
1.1: strong 2nd AQ
--aq2-sensitivity <float> "Flatness" threshold to trigger 2nd AQ [15.0]
5: applies to almost all blocks
22: only flat blocks
--aq2-ifactor <float> 2nd AQ strength factor of I-frames [1.00]
--aq2-pfactor <float> 2nd AQ strength factor of P-frames [1.00]
--aq2-bfactor <float> 2nd AQ strength factor of B-frames [1.00]
--aq3-mode <integer> 3rd AQ (OreAQ) method [0]
- 0: Disabled
- 1: OreAQ
- 2: MixOre (experimental)
--aq3-strength <float> Reduces blocking and blurring in bump and
clear-cut areas. [0.5]
<Up:Down> or <Up1:Down1:Up2:Down2:Up3:Down3:Up4:OtherStuff>
Set QP up/down strength.
--aq3-sensitivity <float> "Center" of 3rd AQ curve. [10.0]
- 5: most QPs are raised
- 10: good general-use sensitivity
- 15: most QPs are lowered
--aq3-ifactor <Up:Down> 3rd AQ strength factor of I-frames [1.0:1.0]
--aq3-pfactor <Up:Down> 3rd AQ strength factor of P-frames [1.0:1.0]
--aq3-bfactor <Up:Down> 3rd AQ strength factor of B-frames [1.0:1.0]
--aq3-boundary <int:int:int> OreAQ boundary.
fullrange=off: [192:64:24]
fullrange=on : [205:56:9]
#1: Bright-Middle
#2: Middle-Dark
#3: Dark-M.Dark

[jpsdr]

Here what the author tell me in PM when i've asked him the purpose of these new 3&4 modes :

Hi. The idea was to improve --aq-mode 2 in dark areas by allowing lower (for negatives numbers it mean higher absolute values) QP offsets there. Approximation of QP-offset/Variance curves of different AQ modes can be looked here: 1..100000 range but most of the difference between AQ 2/3/4 are in 1..5000 range.

P.S. As users didn't really take active part in my testing I decided to give up on mode 4 [(0.25f - 3.3125f / (qp_adj * qp_adj - 0.75f) one] as too weak (it changes curve only at extremely low variance).


So, basicaly, ignore mode 4.

[benwaggoner]

Overall, do people find that --aq-mode 3 is a "safe" alternative to using --aq-mode 2? In that it doesn't make anything significantly worse?

[colours]

aq2 and aq3 are in addition to the normal aq present in unpatched x264 builds.

Normal aq has three modes in vanilla x264, and two additional auto-variance modes (--aq-mode 3/4) in some patched builds. These extra modes do not affect and are not affected by Haali AQ (aq2) or OreAQ (aq3).

The command line parsing automatically treats --aq-mod as being short for --aq-mode; consequently, --aq-mod 3 does not set any aq2 or aq3 parameters.

[MichaelRow]

So, basicaly, ignore mode 4.
Thanks for the info.

Overall, do people find that --aq-mode 3 is a "safe" alternative to using --aq-mode 2? In that it doesn't make anything significantly worse?
Some of my tests show that aq3 would make some light areas a little worse than aq1 at low bitrate.

[sneaker_ger]

Since aq redistributes bits it's expected that some areas get worse.

[Asmodian]

Some of my tests show that aq3 would make some light areas a little worse than aq1 at low bitrate.
Since aq redistributes bits it's expected that some areas get worse.

Right, light areas got worse while presumably dark areas got better.

[MichaelRow]

Right, light areas got worse while presumably dark areas got better.
It may looks better on dark areas, since what aq does is just moving bits around blocks and frames.

Normal aq3(aq-mod=3,aq3-mod=0) would use different strength to different frames while aq1 use the same strength to all the frames, and banding caused by low bitrate in light areas are not noticeable, to some extent, aq1 is safe but a little waste of bits. I found these opinions when googling, and I confused.
Maybe my tests about aq3 with bitrate at 1200kbps for 1080P are cherry-picked examples.
I'm just wondering if aq3 can help to save bits without much visual difference when using crf=17,aq-mod=3,aq3-mod=0,aq-strength=0.8 or are there a better settings for anime encode?:thanks:

[benwaggoner]

Some of my tests show that aq3 would make some light areas a little worse than aq1 at low bitrate.
Is that in the same frame that also has some dark areas that are getting improved? Or are you seeing a negative impact on frames without dark areas as well?

[burfadel]

For the 'extra' options I'm using:
--aq-mode 3 --fgo 5 --fade-compensate 0.2 --aq3-mode 2 --aq3-strength 0.3

This is in conjunction with using CRF, and maxing out the standard quality settings.

I'm not saying they're the ideal options, but I feel it's balanced without upsetting the bitrate too much. Setting fgo to 5 is a very light film grain optimisation which gives a nice result. Same with fade compensate being 0.2, it's a small amount, beneficial, without costing too much bitrate. The aq3-mode 2 use is a little experimental, I think it does look good! I use a strength of 0.3 for this, which might sound low, but it seems to be effective. Having this much higher seems to make the bitrate use increasingly higher.

[~ VEGETA ~]

For the 'extra' options I'm using:
--aq-mode 3 --fgo 5 --fade-compensate 0.2 --aq3-mode 2 --aq3-strength 0.3

This is in conjunction with using CRF, and maxing out the standard quality settings.

I'm not saying they're the ideal options, but I feel it's balanced without upsetting the bitrate too much. Setting fgo to 5 is a very light film grain optimisation which gives a nice result. Same with fade compensate being 0.2, it's a small amount, beneficial, without costing too much bitrate. The aq3-mode 2 use is a little experimental, I think it does look good! I use a strength of 0.3 for this, which might sound low, but it seems to be effective. Having this much higher seems to make the bitrate use increasingly higher.

So we can use aq-mode=3 with aq2-strength (haali) AND OreAQ (aq3-mode) at the same time? How will the encoder distribute the bitrate?

Plus, how to use this:

<Up:Down> or <Up1:Down1:Up2:Down2:Up3:Down3:Up4:OtherStuff>

Is it rather than using one value? if you know, please put an example.


I am interested in OreAQ as it is really good as a concept. I like the fact that we can use aq-mode=3, aq-strength=-0.8 which is great for anime.

So basically:

Normal AQ (aq-mode=3 for example): generally good with all frames, use negative values for anime.
aq2-strength: enhance flat areas.
OreAQ (aq3-mode=1,...): enhance dark areas and so on (maybe control bright areas too?).

^ is this correct?

Also, can we really disable the normal aq-mode and use only aq2 and aq3?

thanks!!

[~ VEGETA ~]

I read that there is a way to visually view the difference using avsutil. anyone tried this?