AQ and bitrate

[Chengbin]

I'm encoding some anime using the latest x264 build, and with this setting

cabac=1 / ref=8 / deblock=1:0:0 / analyse=0x1:0x131 / me=umh / subme=9 / psy_rd=0.0:0.0 / mixed_ref=1 / me_range=32 / chroma_me=1 / trellis=1 / 8x8dct=0 / cqm=0 / deadzone=21,11 / chroma_qp_offset=0 / threads=6 / nr=0 / decimate=1 / mbaff=0 / bframes=5 / b_pyramid=1 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / keyint=1000 / keyint_min=25 / scenecut=40 / rc=crf / crf=24.0 / qcomp=0.80 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.40 / pb_ratio=1.30 / aq=1:0.50

I found when I use AQ of 1, the bitrate is 290Kbps, but when I use AQ of 0.5, bitrate is only 206Kbps. That's a 35% (!!!) bitrate reduction. Is AQ strength 1 too strong for anime, and 0.5 being for suitable?

[LoRd_MuldeR]

Even if you still ignore that fact, CRF doesn't deliver perfect constant quality and only delivers similar quality if you don't change any other options.

35% smaller file at same CRF with different AQ doesn't tell you anything, because it's almost impossible to say whether that 35% bitrate saving is worth the loss in quality or not.

Maybe it would have been better to keep AQ at 1.0 and instead raise the CRF value to save 35% bitrate. Maybe not. We can't conclude that from this kind of test.

So to test AQ 1.0 against AQ 0.5, you must visually compare 2-Pass encodes of the same file size (average bitrate).

And yes, AQ strength 1.0 generally is too high for animated material. That's why "--tune animation" will lower the AQ strength to a value of 0.6 ;)

[Chengbin]

I know you can't compare compression with the same CRF value, but a 35% bitrate reduction is HUGE, even more than CABAC and 8x8dct.

I'm just gonna leave it at 0.5, since 1 is too strong for anime anyway.

[LoRd_MuldeR]

I know you can't compare compression with the same CRF value, but a 35% bitrate reduction is HUGE, even more than CABAC and 8x8dct.

If you changed AQ at same CRF then it's not really surprising to see a significant difference in file size.
I think that's because AQ is part of the complexity measure that ratecontrol (and thus CRF) uses.
Also CRF is never guaranteed to deliver same file size for the same CRF value, if other options change.

Therefore you will need to re-balance your CRF value, after you changed AQ. There's no way around that.

[Chengbin]

I learned something today: AQ can drastically change file size

Thanks.

CRF 24 looks good to me.

[10L23r]

I learned something today: AQ can drastically change file size

Thanks.

CRF 24 looks good to me.

i think you may want to decrease crf to 22-23 to compensate for a lower aq.

[Chengbin]

i think you may want to decrease crf to 22-23 to compensate for a lower aq.

I'm watching it on a 5'' (albelt a very high quality) screen. CRF 24 is about 95% of the visible quality on that screen for anime. But you're right, if it is normal video, then I should lower CRF to 18 as CRF 20 is about 95% of the visible quality on the screen.

BTW, so the AutoVAQ patch changes CRF because the value of AQ drastically change bitrate and the variation of AutoVAQ causes a difference of bitrate?