hevc-aq "production-ready"?

[burban]

Hi,
my current goal is to encode my bluray collection with x265 for easier inhouse streaming. Upon research for good baseline settings I stumbled across the "hevc-aq" option of x265. I could find a few sources stating that it is experimental and not suitable for "important" encodes yet. But it's been a while and it doesn't seem to be experimental anymore, but I also couldn't find any real world tests with it.
So I decided to take a look myself.
I extracted a 5 minute segment from an action movie bluray that contains fast paced action, slow camera movements, dark and bright scenes, landscape shots and close ups. I encoded it with different settings and the results where impressive:


Source: 27.5 Mbit/s (1006 MB)
x265: 6520 Kbit/s (238 MB)
x265 deblock=-1,-1: 6515 Kbit/s (238 MB)
x265 hevc-aq: 2565 Kbit/s (92 MB)


Common settings for all encodes:

Version: x265 3.4+2-g02d2f496c
Preset: slower
Profile: main10
Level: 5.0
CRF: 20
no-sao


After wondering about the huge different in size I did a visual comparison and for me as an average movie watcher the hevc-aq encode looked just like the others. But I wanted to have numbers so I ran a VMAF (vmaf_float_v0.6.1) comparison:


x265: 97.520868
x265 deblock=-1,-1: 97.541947
x265 hevc-aq: 96.264475
x265 hevc-aq, CRF 18: 97.296966

(I did an additonal encode with CRF 18 for hevc-aq. All other encodes are still CRF 20)

As far as I understand VMAF, these are all really good results. Too good be the true. I mean it's less than 40% of the size/bitrate of the other encodes and still maintains a really high quality.
Is there something I'm missing here? Known caveats I didn't notice with my test encode? ...
I understand that there is no magic setting that makes all movies look great. Even though I'm currently wondering if I should use hevc-aq to encode my library or not?

Out of curiousity I encoded the whole movie with the different settings:

x265: 6311 Kbit/s (5054 MB)
x265 deblock=-1,-1: 6188 Kbit/s (5055 MB)
x265 hevc-aq: 1846 Kbit/s (1508 MB)

[benwaggoner]

VMAF has been demonstrated to be poor at discriminating between subjective quality of adaptive quantization algorithms. You'd need to look at the samples to really know if the predictions have a good subjective correlation.

[Boulder]

If I remember correctly, hevc-aq removes quite a lot of details at least at the same CRF level. The reduction in bitrate is most likely a consequence.

[burban]

VMAF has been demonstrated to be poor at discriminating between subjective quality of adaptive quantization algorithms. You'd need to look at the samples to really know if the predictions have a good subjective correlation.

Mhh, ok. I guess PSNR or SSIM are not useful, too? I'm really bad a subjective comparsion.

If I remember correctly, hevc-aq removes quite a lot of details at least at the same CRF level. The reduction in bitrate is most likely a consequence.

Well there is definetly a loss of detail on the plants on the right and on the brown shirt, but nothing that looks like a 60% decrease in bitrate. While playback I didn't really notice a difference.

CRF 20, AQ-Mode 2:
https://i.postimg.cc/xC3w5Vkc/10bit-slower-mkv-snapshot-03-57-320.png (https://postimg.cc/MMX3Z4mS)

CRF 20, HEVC-AQ:
https://i.postimg.cc/gkn7XZKw/10bit-hevc-aq-slower-mkv-snapshot-03-57-320.png (https://postimg.cc/crGhP62Z)

[RanmaCanada]

If you're encoding your own backup copies to make it easier, then the goal should be acceptable quality loss for size. If you can not see the difference, then that is literally all that matters, as you still have the originals on disc. That's my personal opinion on making a backup that is easier to stream throughout the house/cellphones/tablets. If you're going to backup everything and put it on a Plex/Emby/Jellyfin server, you can remux it and take up lots of space and have the server re-encode it every single time, or you can shrink them down with "acceptable quality loss" in an optimized format to make viewing easier.

In the end, all that matters is what is acceptable to you, as you will be the one watching them.

[FranceBB]

You can see the difference in the clouds and also in sharp edges.
If you take a close look at the top left portion of the screen, for instance, you can see a little bit of ringing when AQ is turned on, but most importantly, as soon as you get away from the edge and closer to the sky, the algorithm assumes that it's uniform and removes the original grain pattern:

https://i.imgur.com/htdMFhK.png

If we use the subtract function, we can see what AQ got rid off and we can easily see the grain pattern and also some sharp edges around the two main characters:

https://i.imgur.com/IzP0TQR.png

Now, the reason why I'm telling you this is only 'cause you asked if AQ is production ready and hence my reply would be "no, I wouldn't use it the way it is for our customers", but let's be realistic here, we're not talking about high bitrate masterfiles being encoded for millions of VOD customers, we're talking about a user ripping UHD-BDs and re-encoding them for his personal archive to save space, so... you know what, if they look ok and worth it to you, it's fine as you're the only one who's gonna watch them + family and friends eventually, so it's fine. Objectively we can tell you what it does, but ultimately it's up to you. ;)

If you're encoding your own backup copies to make it easier, then the goal should be acceptable quality loss for size. If you can not see the difference, then that is literally all that matters

Precisely. It's his own personal backup, so if they're good for him, it's fine. ;)

[benwaggoner]

Yeah, optimal settings for a mezzanine are very different than for distribution.

Comparing AQ modes at different bitrates is also pretty fraught. I recommend comparing using 2-pass VBR, so you compare the same ABR and source with different AQ parameters. Maybe hevc-aq can give you the same quality at 20% lower bitrate, even if there are some regressions at less than half the bitrate. Could be that --hevc-aq means you can raise CRF by 2 or something.

[burban]

Now, the reason why I'm telling you this is only 'cause you asked if AQ is production ready and hence my reply would be "no, I wouldn't use it the way it is for our customers", but let's be realistic here, we're not talking about high bitrate masterfiles being encoded for millions of VOD customers, we're talking about a user ripping UHD-BDs and re-encoding them for his personal archive to save space, so... you know what, if they look ok and worth it to you, it's fine as you're the only one who's gonna watch them + family and friends eventually, so it's fine. Objectively we can tell you what it does, but ultimately it's up to you. ;)

Yep, you're totally right. I guess I will just encode a few movies and watch them completly in order to check for any anomalies. It was just hard for me to find information apart from "it's experimental. don't use it". I understand that it heavily depends on the use case, but I'll give it a try.
As benwaggoner suggested I'll experiment with CRF values a bit further. CRF18 with AQ on still reduces the file size by about 33% compared to CRF20 w/o AQ on my sample. If I really encode my whole 700+ disc collection that will be a significant storage saver.


Comparing AQ modes at different bitrates is also pretty fraught. I recommend comparing using 2-pass VBR, so you compare the same ABR and source with different AQ parameters.
Alright, thanks for the clarification. I'll try 2-pass VBR encodes as soon as I got a bit of free time :-)

[tormento]

Any update about this topic?

[benwaggoner]

Yes, what was being developed as --hevc-aq was completed as --aq-mode 4. Don't ever use the old version!

[tormento]

Yes, what was being developed as --hevc-aq was completed as --aq-mode 4. Don't ever use the old version!
Translated: forget about hevc-aq? :)

I am curious anyway about the differences and case use of the 3,4,5 mode. I am currently using aq-mode 5 because it seems the "most complete" but do you have any suggestion about anime vs live action, grainy vs clean, fast motion vs slow motion to apply one of them?

[benwaggoner]

Translated: forget about hevc-aq? :)
Correct.

I am curious anyway about the differences and case use of the 3,4,5 mode. I am currently using aq-mode 5 because it seems the "most complete" but do you have any suggestion about anime vs live action, grainy vs clean, fast motion vs slow motion to apply one of them?

3 is just 2 with a bias for lower QPs in darker portions of the video. This can help shadow quality in 8-bit SDR, but also can increase ABR with CRF quite a bit. It's much less needed for 10-bit, and should not be used for HDR, as PQ fixes the issue by having a lot more code values near black.

4 is probably better overall than the others, and benefits from a slightly lower --aq-strength in general. I've found grainy HDR can be better with 2.

5 isn't in MCW x265, so that'll depend on which build you're using and what it does there.

[tormento]

5 isn't in MCW x265, so that'll depend on which build you're using and what it does there.
I always use Patman’s ones as they are so stable and platform optimized. Do you mind to have a look at its aq-mode?

What mode would you use with anime?

[jpsdr]

The thing several peoples said, it's to not use edge-biased (5 & 3 if not mistaken) with HDR. If not already, maybe you can try/test my build with auto-aq.

[benwaggoner]

The thing several peoples said, it's to not use edge-biased (5 & 3 if not mistaken) with HDR. If not already, maybe you can try/test my build with auto-aq.
in MCW x265, 3 isn't edge based in any way that 2 isn't. The difference is using lower QPs in darker regions.

[benwaggoner]

The thing several peoples said, it's to not use edge-biased (5 & 3 if not mistaken) with HDR. If not already, maybe you can try/test my build with auto-aq.
I should. Do you have a Mac Silicon build?

[jpsdr]

I don't have a Mac and absolutely no knowledge on it...
But the code is on my github to build for those who know how to build for Mac.

Sorry, mistake/mix on specific term.
What i wanted to say it's indeed that several peoples said to not use on HDR a mode "with bias to dark scene", so 3 and 5.

[benwaggoner]

I don't have a Mac and absolutely no knowledge on it...
But the code is on my github to build for those who know how to build for Mac.
No worried. I'm off to HPA tomorrow with just a Mac, but I can check it out on a Windows system a week later.

[jpsdr]

My advise would be to try auto-aq with hysteresis (i add the hysteresis to avoid a possible risk), and if for HDR, enable the HDR mode, so "--auto-aq 6" (bits 1 & 2 set to 1).

[Boulder]

The thing several peoples said, it's to not use edge-biased (5 & 3 if not mistaken) with HDR. If not already, maybe you can try/test my build with auto-aq.

Edge biased are good to use, but luma biased ones are not (because they are pretty much counterproductive to --hdr10-opt). So modes 3 and 5 should not be used and as you mentioned, the auto mode 6 is the one for HDR.

[tormento]

Still have to see aq-mode 6… my count stops at 5… help! [emoji3526]

[Boulder]

Still have to see aq-mode 6… my count stops at 5… help! [emoji3526]

It's bitmapping, so --aq-auto 6 enables the recommended stuff for HDR and --aq-auto 10 does the same for SDR (allows all modes for automatic switching). You can also set different strengths for different modes. I for example use --aq-strength 1.0 --aq-strength-edge 0.75 --aq-bias-strength 0.7 --aq-bias-strength-edge 0.8.

[tormento]

I for example use --aq-strength 1.0 --aq-strength-edge 0.75 --aq-bias-strength 0.7 --aq-bias-strength-edge 0.8.
For SDR or HDR? Same for live action and anime?

[Boulder]

For SDR or HDR? Same for live action and anime?

I use the same for SDR and HDR (even though auto mode 6 skips the bias ones). I don't encode anime so the setting may need tuning for that. Note that the bias strengths are multipliers used with the non-bias one, for example strength for AQ-mode 3 is 1.0 * 0.7 = 0.7 in my case.

[tormento]

I use the same for SDR and HDR (even though auto mode 6 skips the bias ones). I don't encode anime so the setting may need tuning for that. Note that the bias strengths are multipliers used with the non-bias one, for example strength for AQ-mode 3 is 1.0 * 0.7 = 0.7 in my case.
Thank you.

I have tried to set those parameters but I can't understand for what aq-mode. I am using a GUI (StaxRip) and perhaps it's the reason I can't choose freely the number, i.e. I have 0 to 5 only. Choosing one of those, it allows you to modify the aq parameters (of course not with 0).

[Boulder]

Thank you.

I have tried to set those parameters but I can't understand for what aq-mode. I am using a GUI (StaxRip) and perhaps it's the reason I can't choose freely the number, i.e. I have 0 to 5 only. Choosing one of those, it allows you to modify the aq parameters (of course not with 0).

The aq-auto parameter is not the same as aq-mode. Aq-auto does automatic AQ-mode switching in the encoder no matter what mode you set as a parameter. So what you would need is to use jpsdr's x265 fork which includes the parameter and set the parameter --aq-auto 6 or 10 as a custom parameter if StaxRip has such functionality.

[tormento]

So what you would need is to use jpsdr's x265 fork which includes the parameter and set the parameter --aq-auto 6 or 10 as a custom parameter if StaxRip has such functionality.
StaxRip is double bonded with Patman builds, that, AFAIK, has aq-mode up to 5 but no aq-auto.

What multipliers could I use if limited to those modes?

P.S: Isn't --auto-aq instead of --aq-auto?

[Boulder]

StaxRip is double bonded with Patman builds, that, AFAIK, has aq-mode up to 5 but no aq-auto.

What multipliers could I use if limited to those modes?

P.S: Isn't --auto-aq instead of --aq-auto?

Well, the values for aq-strength based on the multipliers in my example would be 1.0 for modes 1 and 2, 0.7 for mode 3, 0.75 for mode 4 and 0.6 for mode 5.

You probably could replace the exe with jpsdr's one to use auto-aq (yes, the param is that way).

[tormento]

set the parameter --aq-auto 6 or 10
Can you please explain me the boolean values that make up the decimal number? I can't find anywhere.

[jpsdr]

@tormento
From the help file:

--aq-auto Configure auto-AQ mode. Disabled 0 (default), <>0 enabled
- Bit 1: If set to 1, enable hysteresis.
- Bit 2: If set to 1, enable HDR mode => don't use biased mode on auto-AQ.
- Bit 3: If set to 1, replace AQ-MODE 1 by AQ-MODE 5. Overrided by bit 2.

Value of a bit = 2^(Number/position of the bit)
8 bit data for exemple, bit 0 (LSB) to bit 7 (MSB).
So, Bit 0 -> 2^0 = 1, Bit 1 -> 2^1 = 2, etc...
So, setting bit 1 and bit 2 result in 2+4=6 value.
Setting only bit 0, so 1, will just enable aq-auto, without any other option activated (or any other bit than [1,2,3]).

[tormento]

@tormento
From the help file
Thank you! Do you maintain the help file on GitHub?

[jpsdr]

The help file is included in the .7z pack releases, but it's generated with the following command : "x265_x64 --fullhelp > x265_Help.txt" (for 64bits version).

[tormento]

Did some tests with the material I encode most, i.e. anime.

Look here (https://github.com/Patman86/x265-Mod-by-Patman/issues/19#issuecomment-2003931391).

Really peculiar how aq-mode 3 is almost always better than any other mode.

--aq-auto was left with no parameter.

[tormento]

It's bitmapping, so --aq-auto 6 enables the recommended stuff for HDR and --aq-auto 10 does the same for SDR (allows all modes for automatic switching). You can also set different strengths for different modes. I for example use --aq-strength 1.0 --aq-strength-edge 0.75 --aq-bias-strength 0.7 --aq-bias-strength-edge 0.8.

x265.exe: unrecognized option `--aq-strength-edge'
x265.exe: unrecognized option `--aq-strength-edge'
x265.exe: unrecognized option `--aq-bias-strength-edge'

What build do you use?

[Boulder]

x265.exe: unrecognized option `--aq-strength-edge'
x265.exe: unrecognized option `--aq-strength-edge'
x265.exe: unrecognized option `--aq-bias-strength-edge'

What build do you use?

I've used a patched build created with Media Autobuild Suite, but I'm sure https://github.com/jpsdr/x265/releases contains releases with all the goodies.

[jpsdr]

And making new build (should be finished in 1 or 2 days) updated with the last Patman patches, especialy a patch called "fix rate control bug", wich worries me... Are all my encodes "screwed up"... as i didn't have this patch on them.

[Boulder]

And making new build (should be finished in 1 or 2 days) updated with the last Patman patches, especialy a patch called "fix rate control bug", wich worries me... Are all my encodes "screwed up"... as i didn't have this patch on them.

Maybe it's for that age-old issue of sudden blocking in certain rather corner cases. I think it was shown to affect also x264 where the code was then ported when x265 was developed.

[tormento]

And making new build (should be finished in 1 or 2 days) updated with the last Patman patches, especialy a patch called "fix rate control bug", wich worries me... Are all my encodes "screwed up"... as i didn't have this patch on them.
Please, if possible, provide some build for avx in llvm for my venerable Sandy Bridge.

The winthread one is so slow compared to msvc one.

[tormento]

So strange. Different builds can make a difference.

https://i.ibb.co/TPGF21r/FFMetrics-x265.png

Seems like the suggested autoaq 10 is lower in quality than the implementation with no parameters at all (Patman builds).

Autoaq + tweaks are the suggested edge parameters.

As I stated, it's a very peculiar content, i.e. anime.

[jpsdr]

It takes allready a lot of time make the builds, i don't want to add more.

[Asmodian]

Seems like the suggested autoaq 10 is lower in quality than the implementation with no parameters at all (Patman builds).

Do not use such small differences in objective metrics to define what is better quality. You have to watch it. These options are about tradeoffs, and what tradeoff looks better does not perfectly match with what optimizes VMAF, etc. Often the better optimization for apparent quality as judged by humans has a slightly lower VMAF, etc.

Codecs would be much easier to optimize if we had an objective metric that perfectly matched human perception, but sadly we do not. :(

[benwaggoner]

Codecs would be much easier to optimize if we had an objective metric that perfectly matched human perception, but sadly we do not. :(
And whenever we get a new metric with high correlation, people start tuning their encoders for that metric and its correlation with subjective quality actually drops over time.

[tormento]

Codecs would be much easier to optimize if we had an objective metric that perfectly matched human perception, but sadly we do not. :(
Benwaggoner suggested P.1203 and P.1204 (https://streaminglearningcenter.com/blogs/itu-t-p1203-p1204.html).

I have found a github repositori for P.1203 (https://github.com/itu-p1203/itu-p1203) and P.1204.3 (https://github.com/Telecommunication-Telemedia-Assessment/bitstream_mode3_p1204_3).

It would be nice to have it in ffmpeg or in standard Windows build but I am not too optimistic about that.

[tormento]

I have just found this paragraph in VMAF paper:

We think that there is value in disregarding enhancement gain that is not part of a codec. We also believe that there is value in preserving enhancement gain in many cases to reflect the fact that enhancement can improve the visual quality perceived by the end viewers. Our solution to this dilemma is to introduce a new mode called VMAF NEG (“neg” stands for “no enhancement gain”). And we propose the following:

Use the NEG mode for codec evaluation purposes to assess the pure effect coming from compression.
Use the “default” mode to assess compression and enhancement combined.

So, I think, it can be used in a bit more flexible way.

[rwill]

Use the NEG mode for codec evaluation purposes to assess the pure effect coming from compression.
Use the “default” mode to assess compression and enhancement combined.


A codec can cheat VMAF - NEG or not.

[Guest]

@tormento
From the help file:

--aq-auto Configure auto-AQ mode. Disabled 0 (default), <>0 enabled
- Bit 1: If set to 1, enable hysteresis.
- Bit 2: If set to 1, enable HDR mode => don't use biased mode on auto-AQ.
- Bit 3: If set to 1, replace AQ-MODE 1 by AQ-MODE 5. Overrided by bit 2.

Value of a bit = 2^(Number/position of the bit)
8 bit data for exemple, bit 0 (LSB) to bit 7 (MSB).
So, Bit 0 -> 2^0 = 1, Bit 1 -> 2^1 = 2, etc...
So, setting bit 1 and bit 2 result in 2+4=6 value.
Setting only bit 0, so 1, will just enable aq-auto, without any other option activated (or any other bit than [1,2,3]).

This might as well be hieroglyphics from an ancient Egyptian tomb, can someone just provide an example of an x265 command.

I'm using either --aq-auto 6 or --aq-auto 10 (according to Boulder)

:thanks:

[Boulder]

This might as well be hieroglyphics from an ancient Egyptian tomb, can someone just provide an example of an x265 command.

I'm using either --aq-auto 6 or --aq-auto 10 (according to Boulder)

:thanks:

Computers like binary things, so this is binary math with 0s and 1s. It just first confused me that the actual first bit is not used, but the three after that :o It's been 25 years since I had to deal with these things in this depth..which is why I didn't remember that the bit index starts from 0 so the bit index numbers are 0-7.

So by default, the 8-bit data would be 00000000 which is zero in decimal. If you flip bit number 1 to 1 to enable hysteresis, the binary data would be 00000010.

Now as jpsdr wrote, the decimal value can be calculated based on the position. Starting from left, the decimal value of each enabled bit (=1) is 128, 64, 32, 16, 8, 4, 2, 1. Here you get the terms MSB (=Most Significant Bit) and LSB (=Least Significant Bit) because the weight in the total value is very different.

So the maths for enabling just the hysteresis (i.e. SDR mode with aq-mode 1 instead of mode 5 available in the autoswitching) would be 0+0+0+0+0+0+2+0 = 2.

If you want to enable hysteresis and do an HDR encode (disabling the counter-productive luma biased modes), the binary data would be 00000110 and conversion to decimal 0+0+0+0+0+4+2+0 = 6. This is where the --aq-auto 6 comes from.

Hysteresis and aq-mode 5 replacing mode 1 in autoswitching would be done by enabling bits 1 and 3, binary data being 00001010, so the conversion is 0+0+0+0+8+0+2+0 = 10 and that is where you get --aq-auto 10.

[benwaggoner]

Computers like binary things, so this is binary math with 0s and 1s. It just first confused me that the actual first bit is not used, but the three after that :o It's been 25 years since I had to deal with these things in this depth..which is why I didn't remember that the bit index starts from 0 so the bit index numbers are 0-7.

So by default, the 8-bit data would be 00000000 which is zero in decimal. If you flip bit number 1 to 1 to enable hysteresis, the binary data would be 00000010.

Now as jpsdr wrote, the decimal value can be calculated based on the position. Starting from left, the decimal value of each enabled bit (=1) is 128, 64, 32, 16, 8, 4, 2, 1. Here you get the terms MSB (=Most Significant Bit) and LSB (=Least Significant Bit) because the weight in the total value is very different.

So the maths for enabling just the hysteresis (i.e. SDR mode with aq-mode 1 instead of mode 5 available in the autoswitching) would be 0+0+0+0+0+0+2+0 = 2.

If you want to enable hysteresis and do an HDR encode (disabling the counter-productive luma biased modes), the binary data would be 00000110 and conversion to decimal 0+0+0+0+0+4+2+0 = 6. This is where the --aq-auto 6 comes from.

Hysteresis and aq-mode 5 replacing mode 1 in autoswitching would be done by enabling bits 1 and 3, binary data being 00001010, so the conversion is 0+0+0+0+8+0+2+0 = 10 and that is where you get --aq-auto 10.
A very cogent explanation for a very confusing topic for those without a computer science background.

And an excellent case for just having the different bits be their own parameters! It's always a bad sign if customers have to look at a 2D table to control multiple independent variables.

[nevcairiel]

Flags are fine, but if you want to be user-friendly could offer a scheme like ffmpeg uses, and many other apps do - symbolic names.

For example:
--aq-auto hysteresis+aq5 (not a real working command)

Still relates all to the same parameter, still easily expandable, but you know what it does!

[jpsdr]

String command analysis stuff is absolutely not my thing... So i took the easy way "duplicating" an easy existing command with just a simple number.