x265: a good compromise between "slow" and "slower" presets?

[Boulder]

I think those are quite typical "artifacts" of x265 encoding. I was just thinking that AQ can affect such things because it is shifting bits inside the frame. Lowering qg-size to 16 could also change things.
By the way, hdr-opt nullifies your settings regarding the chroma QP offsets so it won't help.

[redbtn]

I think those are quite typical "artifacts" of x265 encoding. I was just thinking that AQ can affect such things because it is shifting bits inside the frame. Lowering qg-size to 16 could also change things.
By the way, hdr-opt nullifies your settings regarding the chroma QP offsets so it won't help.If I raise aq-strength, what happens? More complex blocks get more bitrate or opposite?
I read the docs and it seems to me that raising aq-strenght give more bits to flat areas.

Adaptive Quantization operating mode. Raise or lower per-block quantization based on complexity analysis of the source image. The more complex the block, the more quantization is used. This offsets the tendency of the encoder to spend too many bits on complex areas and not enough in flat areas.


Adjust the strength of the adaptive quantization offsets. Setting --aq-strength to 0 disables AQ. At aq-modes 2 and 3, high aq-strengths will lead to high QP offsets resulting in a large difference in achieved bitrates.

[Boulder]

A bigger value should shift bits to less complex blocks. Of course, it would be useful to test drastic changes to both directions and see if there is any effect. Then you know if that is the answer to your issue or if it's something else like the psy options.

[benwaggoner]

There was a presentation on optimization of x265 at SMPTE yesterday, as part of the Compression track I chaired.

A couple of specific optimizations available in 3.2 were called out.

First, --selective-sao 3 only applies SAO for I and P frames, but delivers almost all the potential quality benefit of SAO with a ~4% speedup. This follows general literature that SAO isn't helpful with bi-prediction.

I wouldn't be surprised to see that added to --preset slower down the road.

Second, using --rd 4 and --dynamic-rd 3 instead of --rd 6 for CRF with VBV gives full rd 6 quality at peaks, at the cost of slightly higher bitrates (but similar quality) when CRF instead of VBV is the limiting factor. Speedup is estimated at ~9%, but obviously will vary based on how often the VBV cap is the constraint.

[redbtn]

Thank you benwaggoner for interesting information!

But, there is something wrong with HDR encodes. Because, I made a comparison. 8bit SDR (tonemapped with script) versus 10bit HDR (tonemapped after encode with the same script) versus 8bit SDR x264 (same script).

x264 SDR looks very good.
x265 SDR looks even better! No problem at all.
x265 HDR looks bad. Not transparent encode with color issues that I described above.

All encodes at same bitrate.
What I'm doing wrong? I'd like to know how to do good HDR encode.

[Boulder]

First, --selective-sao 3 only applies SAO for I and P frames, but delivers almost all the potential quality benefit of SAO with a ~4% speedup. This follows general literature that SAO isn't helpful with bi-prediction.

In layman's terms, does that mean that SAO has actually become usable to us who call it "Smooth All Objects"? B-frames in x265 encodes often seem to be much smoother than in x264 ones, even without SAO enabled.

EDIT: I think you mean --selective-sao 2? Option 3 enables SAO for all reference frames.

[excellentswordfight]

Thank you benwaggoner for interesting information!

But, there is something wrong with HDR encodes. Because, I made a comparison. 8bit SDR (tonemapped with script) versus 10bit HDR (tonemapped after encode with the same script) versus 8bit SDR x264 (same script).

x264 SDR looks very good.
x265 SDR looks even better! No problem at all.
x265 HDR looks bad. Not transparent encode with color issues that I described above.

All encodes at same bitrate.
What I'm doing wrong? I'd like to know how to do good HDR encode.
Please provide samples, and full information of the pipeline and paramenters.

[redbtn]

Please provide samples, and full information of the pipeline and paramenters.
Sample https://gofile.io/?c=1b5LFV
VapourSynth R48 RC1 (pipeline vspipe.exe)
x265 3.2+7 by Barough https://forum.doom9.org/showthread.php?p=1887694#post1887694
Simple x264/265 Launcher 2.94 https://github.com/lordmulder/Simple-x264-Launcher/releases

Frame № 37
Screens https://gofile.io/?c=FDrVMt (SDR screen is tonemapped source encoded with the same setting exclude HDR setting)
Tonemap.dll https://gofile.io/?c=Pl4Pmj

The best I could do is --aq-mode 1 --aq-strength 0.8 --cbqpoffs -3 --crqpoffs -3

--level-idc 51 --sar 1:1 --colorprim 9 --colormatrix 9 --transfer 16 --range limited --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50)" --max-cll "3241,902" --hdr --hdr-opt --hrd --aud --chromaloc 2 --repeat-headers --min-luma 0 --max-luma 1023 --no-cutree --no-sao --no-open-gop --no-strong-intra-smoothing --vbv-bufsize 160000 --vbv-maxrate 160000 --min-keyint 1 --keyint 240 --ipratio 1.4 --pbratio 1.2 --deblock -3:-3 --qcomp 0.65 --aq-mode 1 --aq-strength 0.8 --ctu 64 --merange 57 --me star --limit-refs 1 --no-early-skip --weightb --b-intra --rd 4 --rdoq-level 2 --rc-lookahead 72 --psy-rd 2 --psy-rdoq 1.2 --max-merge 4 --lookahead-slices 0 --bframes 8 --ref 4 --subme 4 --tu-intra-depth 4 --tu-inter-depth 4 --limit-tu 4 --no-rect --no-amp --limit-modes --cbqpoffs -3 --crqpoffs -3



import vapoursynth as vs
core = vs.get_core()
core.max_cache_size = 32768
clip = core.lsmas.LWLibavSource(source="E:\sample.mkv", format="YUV420P10", cache=1)
clip = core.std.AssumeFPS(clip, fpsnum=24000, fpsden=1001)
clip = core.std.CropRel(clip=clip, left=0, right=0, top=280, bottom=280)

# color adjustment using ToneMap
clip = core.resize.Bilinear(clip=clip, format=vs.RGBS, range_in_s="limited", matrix_in_s="2020ncl", primaries_in_s="2020", primaries_s="2020", transfer_in_s="st2084", transfer_s="linear",dither_type="none", nominal_luminance=125)
clip = core.tonemap.Hable(clip=clip, exposure=2.000)
clip = core.resize.Bilinear(clip=clip, format=vs.YUV420P8, matrix_s="709", primaries_s="709",range_s="limited", transfer_s="709", transfer_in_s="linear", range_in_s="limited", dither_type="ordered")

clip.set_output()

[benwaggoner]

In layman's terms, does that mean that SAO has actually become usable to us who call it "Smooth All Objects"? B-frames in x265 encodes often seem to be much smoother than in x264 ones, even without SAO enabled.
I don't know that this changes that. But the SAO detail loss issues have become way less in more recent versions of x265.

EDIT: I think you mean --selective-sao 2? Option 3 enables SAO for all reference frames.
Oh, yes. Thanks for the correction!

[benwaggoner]

Sample https://gofile.io/?c=1b5LFV
The best I could do is --aq-mode 1 --aq-strength 0.8 --cbqpoffs -3 --crqpoffs -3
That's a quite complex command line! I'd start without all the tweaks, and just do something more like:
--preset slower --hdr --hdr-opt

I don't know what value you'd get from a min-keyint of 1 or setting min/max luma.

Level 5.1 has maxrate/bufsize limits of 40000, not 160000.

It appears you've not set CRF OR bitrate, which means it's being encoded at the default CRF, which is quite high. If you're aiming for excellent quality, I'd start with 18 and iterate from there.

--hdr-opt will tweak the chroma QPs better than just setting the offsets. And if you didn't set HDR, you didn't get HDR metadata, and the video is likely playing back as if it had SDR values, which would look terrible!

And don't set --chromaloc unless you've actually changed the chroma location. If you DID change that, let me know what tool you used to do it!

Also, where did you get your MaxCLL and MaxFALL values? A MaxFALL of 900 suggest at least one blindingly bright frame, and no consumer TV available can actually display a MaxCLL of 3200 nits. Not that content doesn't get made that way sometimes, but I'd definitely investigate the content to make sure something didn't go horribly wrong.

[redbtn]

After all, I set aq-mode 0 and it almost solves the problem. But i don't know how it affects to encode quality in general.


Level 5.1 has maxrate/bufsize limits of 40000, not 160000.


Level 5.1 High has 160000, --high-tier is a default value


It appears you've not set CRF OR bitrate, which means it's being encoded at the default CRF, which is quite high. If you're aiming for excellent quality, I'd start with 18 and iterate from there.


I use CRF 20 (21-23mbps) for UHD and CRF 12-14 (17-18mbps) for FHD.


--hdr-opt will tweak the chroma QPs better than just setting the offsets. And if you didn't set HDR, you didn't get HDR metadata, and the video is likely playing back as if it had SDR values, which would look terrible!

--hdr-opt is definitely necessary, but --cbqpoffs -3 --crqpoffs -3 fixes color issues a little. I don't know why it needed with HDR.


And don't set --chromaloc unless you've actually changed the chroma location. If you DID change that, let me know what tool you used to do it!

I didn't change chromaloc, but UHD requires Chroma subsampling : 4:2:0 (Type 2), so i add flag --chromaloc 2


Also, where did you get your MaxCLL and MaxFALL values? A MaxFALL of 900 suggest at least one blindingly bright frame, and no consumer TV available can actually display a MaxCLL of 3200 nits. Not that content doesn't get made that way sometimes, but I'd definitely investigate the content to make sure something didn't go horribly wrong.

There is mediainfo of my sample below (it is a general movie, nothing special):

Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : Display P3
Mastering display luminance : min: 0.0050 cd/m2, max: 4000 cd/m2
Maximum Content Light Level : 3241 cd/m2
Maximum Frame-Average Light Level : 902 cd/m2

[Blue_MiSfit]

Turning off AQ is fine sometimes, and in fact is done in the grain preset in order to "[restrict] algorithms that vary the quantization parameter within and across frames":

https://x265.readthedocs.io/en/default/presets.html#film-grain

[Boulder]

Turning off AQ is fine sometimes, and in fact is done in the grain preset in order to "[restrict] algorithms that vary the quantization parameter within and across frames":

https://x265.readthedocs.io/en/default/presets.html#film-grain

It could well be that HDR sources are often better re-encoded without AQ. It is very flat in the eyes of the encoder anyway so the decisions it makes, are most likely very different compared to SDR sources. Needs careful testing, I'd say.

[Blue_MiSfit]

I leave AQ on for HDR, to be honest. Most of the time that works well. Sometimes it doesn't but I don't do any per-encode tuning (for my work, anyway).

[Nico8583]

I leave AQ on for HDR, to be honest. Most of the time that works well. Sometimes it doesn't but I don't do any per-encode tuning (for my work, anyway).
If possible, could you give a command line example for FHD and a command line example for UHD (HDR10 and/or HDR10+) ? Thank you !

[aymanalz]

First, --selective-sao 3 only applies SAO for I and P frames, but delivers almost all the potential quality benefit of SAO with a ~4% speedup. This follows general literature that SAO isn't helpful with bi-prediction.


Wow, thanks for the info. I am getting a huge speed increase by using selective-sao 2 (you might want to edit your post and make it 2, instead of 3). In fact, a lot more than 4%.

[benwaggoner]

I've been meaning to try --aq-mode 4. Since so much of the value from HDR at high resolution comes from strong local contrast, something that preserves edges better could be quite helpful.

[nghiabeo20]

I use bframe=6. According to the docs, this could shed you 25% of the workload (Am I right?). Is it really worth to keep bframes at 8, if the movie I'm encoding doesn't have so many motion?

[Tadanobu]

Best thing is to run a test encode with maximum possible bframes then check the log file so as to determine how many bframes are actually used.