AQ-Mode

[Braciata]

Hi,

I recently saw a lot of h264 encodes with mbtree=0 and something like aq=3:0.80 / aq-sensitivity=10.00 / aq-factor=1.00:1.00:1.00 / aq2=0 / aq3=0

I wondered if no-mbtree is better tha mbtree and if that kind of AQ-Mode is better than a simple aq-mode=3.

Last question.. How can add those params? On handbrake I am not able.
Thank you in advance.

[microchip8]

You shouldn't mess around with AQ strengths. The default of 1.0 is almost always optimal and should be only decreased for very flat content, like anime. mbtree is a bit tricky, it can help with noisy/grainy content

[Braciata]

You shouldn't mess around with AQ strengths. The default of 1.0 is almost always optimal and should be only decreased for very flat content, like anime. mbtree is a bit tricky, it can help with noisy/grainy content

Thank you for your answer.

[DTL]

I recently saw a lot of h264 encodes with mbtree=0 and something like aq=3:0.80 / aq-sensitivity=10.00 / aq-factor=1.00:1.00:1.00 / aq2=0 / aq3=0

I wondered if no-mbtree is better tha mbtree and if that kind of AQ-Mode is better than a simple aq-mode=3.



Oh - as I see the AQ-stuff is the main current development point of x264. In the old docs (builds ?) we have only aq-mode 0,1,2 and aq-strength. Now it looks in newer builds there are lots more params added for tweaking AQ-robot inside.

When I started to experiment with 'very clean' mixed footage after strong temporal noise reduction I found aq-mode=0 may give significantly lower bit rate (with still good PSNR) at the scenes like static background and small talking head. But with footage with mixed scenes content it is not optimal.
Next was aq-mode=1 and tweaking aq-strength to < 1 (like to 0.25). It works close to aq-mode=0 at the static scenes and also not very great at others (like slow camera pan - sort of global frame motion).

The most important I found when trying to add IIR-type temporal filtering (remember samples values and update only if difference above some threshold): The % of B-frames skipped may reach already very high values (like 85..88+%) but x264 start to increase PSNR (to 48+ dB) and not lower output bitrate (at some crf-ratecontrol, aq-mode=1 and aq-strength < 1).

May new AQ-tweaking params be used to tweak AQ algorithm to get lower output bitrate (soft of constant-PSNR mode ? and lower output bitrate as % of skipped blocks become higher) ?

Are there some constant-PSNR AQ modes (or some params set) ? Maybe I need to go to simple constant-qp ratecontrol if I try to feed a 'very clean' source and % of skipped blocks reach about 85+ ?

At some builds/mods of x264 I see even AQ-modes of 3 and 4 added. So it is greatly experimental filed for x264 MPEG encoder ? Some mods in 201x have aq2, aq3 lots of tweaking params - https://forum.doom9.org/showthread.php?t=170599 .

I understand simple PSNR is not great quality metric of imaging but if I finally get situation when some types of noise reduction before MPEG encoding cause constant increasing of % skipped blocks but output bitrate stop decreasing as expected and x264 encoder start to increase PSNR - it may show something not as expected happen with quantization algorithm inside ? Like it skips more blocks and it is expected to lower bitrate, but it also decreases quantizer at the residual blocks so it makes PSNR higher (not needed and not visible) but causes larger output bitrate (not expected).

It is all about compressing FullHD interlaced 25i with crf=18 rateconrol and reaching per-scene bitrate about 3000..6000 kbit/s. So any deviation of bitrate about +-500..1000 kbit/s cause significant change of resulting file and it is worth to find how to configure x264 encoder to make lowest possible output per-scene bitrate without non-needed increasing PSNR (may be it is issue in crf-algorithm for some type of 'very clean' sources ?).

[jpsdr]

Some people make test wrongly, and got the false assumption that not using mb-tree gives better results.
The only case i would suggest to disable mbtree is when targetting Blu-Ray, where you have the gop size limited to 24 (well, the fps value to be precise).
With usual long default gop size (250 if i remember), mbtree is benefit.
Maybe there is others very specific cases where mbtree is not benefit, but i don't know them. The only case i know is with little gop size.
I got this information from mp3dom, who encodes for Blu-Ray at professionnal.

[DTL]

Maybe I misunderstand crf-ratecontrol ? If I feed more and more denoised footage to crf=18 (lets with default aq-mode and all other settings) - will the encoder produce lower and lower output bitrate/filesize ? Or it depends on many internal logic of crf-mode (and all aq-stuff internal) ? If I start to get a still not very low bitrate and the PSNR becomes 'too high' like 48 and more - can I get output bitrate lower with lower 'quality' and lower PSNR by tweaking some aq-type settings while keeping crf=18 ? Or I simply need to tweak the crf also (to 19..20 or more) and look at what happens with bitrate (and PSNR) ?

Currently, as I see, as % of skipped blocks become higher after better denoise - it looks crf-logic starts to make quantizer too low to keep 'quality better' so not allow to make bitrate at the mostly static and clean scenes as low as expected. Can I tweak aq-type settings (or other ?) to allow x264 to keep 'quality/PSNR' lower and so make bitrate (and filesize) lower at such type of scenes without setting 'global' crf-value above 18 ?
Short tests around aq-mode 0 and 1 shows aq-mode=0 allow to have much lower bitrate at static camera scenes but cause more bitrate at camera pan scenes. So total average bitrate and filesize not become lower. Can I set some of the aq-settings around aq-mode > 0 so it will keep bitrate at static and very low noise scenes (% of B frame skipped blocks > 85%) close to the aq-mode=0 ?
Maybe most of the x264 settings are adjusted on average medium noise/grain film footages so at the very clean footage it causes not very nice working of crf-ratecontrol ?

[Braciata]

Some people make test wrongly, and got the false assumption that not using mb-tree gives better results.
The only case i would suggest to disable mbtree is when targetting Blu-Ray, where you have the gop size limited to 24 (well, the fps value to be precise).
With usual long default gop size (250 if i remember), mbtree is benefit.
Maybe there is others very specific cases where mbtree is not benefit, but i don't know them. The only case i know is with little gop size.
I got this information from mp3dom, who encodes for Blu-Ray at professionnal.

Hi. I asked because I see a lot fo x264 ecnodes with no-mbtree. I personally don't like the results.

[DTL]

Original paper about mb-tree - https://huyunf.github.io/blogs/2017/12/06/x264_slice_type_decision/MBtree%20paper.pdf

[mp3dom]

Hi. I asked because I see a lot fo x264 ecnodes with no-mbtree. I personally don't like the results.

It depends by the amount of bitrate you're targetting, and activation of additional restrictions such as vbv-bufsize or closed-gops.

[Braciata]

It depends by the amount of bitrate you're targetting, and activation of additional restrictions such as vbv-bufsize or closed-gops.

I'd like to end with something between 12000 and 15000

[mp3dom]

If you don't have the needs to use the vbv restrictions (maxrate and bufsize), then mbtree is fine.

[Braciata]

If you don't have the needs to use the vbv restrictions (maxrate and bufsize), then mbtree is fine.

Thank you. I'll go with it.

[takla]

Maybe I misunderstand crf-ratecontrol ?

Currently, as I see, as % of skipped blocks become higher after better denoise - it looks crf-logic starts to make quantizer too low to keep 'quality better' so not allow to make bitrate at the mostly static and clean scenes as low as expected.

1) Yes. All x264 settings are supposed to make compression better (quoting a Dev, sorry no source, I've read it a while ago on here) At some point you run out of settings to tweak and so the only way left is to increase bitrate.

2) You shouldn't denoise too much. The encoder expects noise, so leave some. Actually this is a known issue and has been for decades. In some cases people even added grain to make the encoder behave to their needs.

[microchip8]

Too much denoising almost always introduces banding. On very clean sources, when I use x264 8-bit, I even add some noise to the encode to reduce banding and halo's

[Z2697]

Adding noise is a sin! Don't add noise! Use AV1 with grain synthesis if you must!
(just kidding)


Encoders do not expect noise to work.

Noise does not reduce banding and halo - it just make human eyes confused and hard to distinguish those things. In "video processing means" banding is caused by not enough precision, dithering can help that, although dithering may look similar to adding noise but they are not exactly the same. Haloing is typically caused by sharpening (the term is used for many different things), of course noise can't help with that.

Too much denoising almost always introduces banding. On very clean sources, when I use x264 8-bit, I even add some noise to the encode to reduce banding and halo's

Or maybe the noise is to trick encoder to allocate more bits to prevent "banding and haloing in compression artifacts means", then try to adjust aq, psy or other parameters first, think adding noise as the last resort.

[takla]

Noise does not reduce banding

Incorrect. More noise means more bits spend by the encoder and more bits means less banding.

[Z2697]

Incorrect. More noise means more bits spend by the encoder and more bits means less banding.

Incorrect. I did add some explanation about that. Do more reading will ya?

[microchip8]

Adding noise is a sin! Don't add noise! Use AV1 with grain synthesis if you must!
(just kidding)


Encoders do not expect noise to work.

Noise does not reduce banding and halo - it just make human eyes confused and hard to distinguish those things. In "video processing means" banding is caused by not enough precision, dithering can help that, although dithering may look similar to adding noise but they are not exactly the same. Haloing is typically caused by sharpening (the term is used for many different things), of course noise can't help with that.



Or maybe the noise is to trick encoder to allocate more bits to prevent "banding and haloing in compression artifacts means", then try to adjust aq, psy or other parameters first, think adding noise as the last resort.

No amount of tweaking AQ, Psy or other options can match the result of adding a bit of noise to the encode. Trust me, I've tried tweaking AQ and Psy. Couldn't get the desired result!

PS: adding noise to the encode *is called* dither. I don't know if you have your own meaning of dithering.

[Z2697]

No amount of tweaking AQ, Psy or other options can match the result of adding a bit of noise to the encode. Trust me, I've tried tweaking AQ and Psy. Couldn't get the desired result!

PS: adding noise to the encode *is called* dither. I don't know if you have your own meaning of dithering.

Dither is a process to reduce banding when doing quantization(not the same quantization in lossy compression) or bit-depth conversion.
Noise or pattern is the result of dither.
Adding noise can be considered a type of dither if it's performed during quantization or bit-depth conversion.
Adding noise after quantization during encoding is not dither.

Hence, they are not exactly the same.

For a operation of adding noise to serve the purpose of both dithering and "trick the encoder", you need to have a high precision source, apply noise during quantization or bit-depth conversion with enough amplitude so it significant enough to trick the encoder.

This is a special case, calling all type of "adding noise to the encode" operations "dither" is inaccurate.

Some other types of dithering: https://en.wikipedia.org/wiki/Dither#Algorithms

Here's a comparison of same noise applied during or after bit-depth conversion I just made: https://imgsli.com/MzA1NDQ3

[microchip8]

I'm not sure from where you get that adding noise after quantization is not dither? Care to cite a source? Even wikipedia states that adding noise is used to randomize quantization errors and prevent banding and is called dither

[Z2697]

Explain the comparison I made then.

Adding noise after quantization is definitely not called dither.
to randomize quantization errors
How can you randomize it after it's done?

But OK, calling those dither methods "adding noise" is not technically wrong because noise takes all sorts of forms, even the quantization error itself can be considered a noise.
https://en.wikipedia.org/wiki/Noise_(signal_processing)#Types_of_noise

Now, if you say all noise is dither, then everything is dithered because quantization error is noise.

[benwaggoner]

Adding noise is a sin! Don't add noise! Use AV1 with grain synthesis if you must!

Alas, a lot of early generation AV1 decoders shipped with deficient FGS implementations, so I don't think anyone is using it in the wild.

AV2 is fixing the spec and will be providing conformance tests, so hopefully we can finally use it then. There's also an effort to support the MPEG equivalent to VVC.

[benwaggoner]

I'm not sure from where you get that adding noise after quantization is not dither? Care to cite a source? Even wikipedia states that adding noise is used to randomize quantization errors and prevent banding and is called dither
Dithering is generally meant to apply to source pixels before the frequency transform. I've not heard of any encoders adding noise post-quantization. Like some sort of inverse adaptive deadzone?

(the real solution to avoid banding is to use 10-bit).

[hello_hello]

I'm thinking that adding noise when reducing the bitdepth is dithering, while adding noise after the bitdepth is reduced is just adding noise, because the effect would definitely be different.
An exaggerated example, taken from a section of the image Z2697 posted. According to Irfanview, the first pic below has 2656 unique colors.

https://i.ibb.co/7V4tx09/2656-colors.png

Reduced to 6 unique colors while applying Floyd-Steinberg dithering.

https://i.ibb.co/8MvjqmN/6-colors-dithered.png

Reduced to 6 unique colors without dithering.

https://i.ibb.co/KmdPd7V/6-colors.png

Adding noise to the third image won't restore the gradients. You'd have to apply a debanding filter at a higher bitdepth and dither while reducing the bitdepth again.

I'm trying to get my head around this one though. When I compress the images as a jpeg at 80% quality the file size of the dithered image is greater than the non-dithered one (11.56 KB vs 9.76 KB). But when I save them at 50% quality, the file size of the dithered version is smaller (3.10 KB vs 3.92 KB).

Dithered jpeg 50% quality 3.10 KB

https://i.ibb.co/NYp8W3P/6-colors-dithered-50-percent.jpg

No dithering jpeg 50% quality 3.92 KB

https://i.ibb.co/fG2mwKP/6-colors-50-percent.jpg

[takla]

Incorrect. I did add some explanation about that. Do more reading will ya?

Not incorrect. I did (unfortunately) read your mental diarrhea.


then try to adjust aq, psy or other parameters first, think adding noise as the last resort.

I explicitly said, in post #13
At some point you run out of settings to tweak and so the only way left is to increase bitrate.

Which is done, indirectly, by adding noise.

Now YOU read what I wrote, and stop projecting and gaslighting.

[Z2697]

Not incorrect. I did (unfortunately) read your mental diarrhea.



I explicitly said, in post #13

Which is done, indirectly, by adding noise.

Now YOU read what I wrote, and stop projecting and gaslighting.

The difference is that adding noise to trick the encoder to allocate more bits is PREVENTING the banding caused by compression. It will not REDUCE the banding that's already there.

Which is done, indirectly, by adding noise.

Why not do it directly?

And I did read that part. I didn't know you mean by adding noise. Are crf, bitrate and qp param not enough?

Encoders can handle (sorry, more like cope) noise don't mean they expect noise.

[microchip8]

I only add a bit of noise, whether it's called dither or not, when encoding to 8 bit and only on (very) clean sources just to avoid banding from the encoding itself - can't avoid if banding is already there in the source. For 10 bit and above, I don't add noise. Unfortunately, I can't use AV1 as my streamers don't support it (NV ShieldTV from 2018) so I currently only use H.264 and HEVC, the latter always in 10 bit. While I can go 10 bit for H.264 too, for the sake of compatibility I keep it at 8 bit.

[Z2697]

I only add a bit of noise, whether it's called dither or not, when encoding to 8 bit and only on (very) clean sources just to avoid banding from the encoding itself - can't avoid if banding is already there in the source. For 10 bit and above, I don't add noise. Unfortunately, I can't use AV1 as my streamers don't support it (NV ShieldTV from 2018) so I currently only use H.264 and HEVC, the latter always in 10 bit. While I can go 10 bit for H.264 too, for the sake of compatibility I keep it at 8 bit.

Have you tried add ordered dithering with increased amplitude, e.g. fmtc.bitdepth(dmode=0,ampo=[some value > 1])? The result may surprise you. Alternatively mode 8 and mode 9 can be used. Mode 3-7 are error diffusions, I think they don't "temporally stable" enough.

[microchip8]

No, I haven't tried that but will play around when I get some time. So far, I'm happy with the way I do it for 8 bit encodings. Thanks