How to limit the maximum bitrate of x265?

[zhouting]

I want to encode UHD Blu-ray M2TS files with x265, my code is like this: enc_ctx = avcodec_alloc_context3(codec);
enc_ctx->codec_id = AV_CODEC_ID_H265;
enc_ctx->bit_rate = 40000000;

When a video has a simple scene, such as 10 seconds, its encoding bitrate will be very small, only 1M or 2M bitrate. In order to achieve an average bitrate of 40M/s, the subsequent bitrate will be very large, sometimes reaching 200M/s, at this time UHD Blu-ray Player will freeze due to high bitrate.
I set
enc_ctx->rc_max_rate = 80000000;
but it seems to have no effect. How can I limit the maximum bitrate? Thanks

[RanmaCanada]

would it not be --vbv-maxrate <integer> So something like --input 1080pSource.y4m --ctu 32 --bitrate 5800 --vbv-maxrate 8700

--vbv-maxrate <integer>
Maximum local bitrate (kbits/sec). Will be used only if vbv-bufsize is also non-zero. Both vbv-bufsize and vbv-maxrate are required to enable VBV in CRF mode. Default 0 (disabled)

Note that when VBV is enabled (with a valid --vbv-bufsize), VBV emergency denoising is turned on. This will turn on aggressive denoising at the frame level when frame QP > QP_MAX_SPEC (51), drastically reducing bitrate and allowing ratecontrol to assign lower QPs for the following frames. The visual effect is blurring, but removes significant blocking/displacement artifacts.

https://x265.readthedocs.io/en/master/cli.html

Sorry just took it straight from the docs. I hope this is correct.

[benwaggoner]

You're correct. With really complex content x265 can overshoot the VBV by a trivial amount sometimes, so lowering it 5% is a good idea if you REALLY need it to be spot on. Almost no uses cases actually do.

[zhouting]

How should vbv-bufsize be set?

[benwaggoner]

How should vbv-bufsize be set?
set --vbv-bufsize and --vbv-maxrate to the desired values. You need to set both for either to do anything.

Setting --level-idc will also default to the Profile @ Level @ Tier maximums. And x265 will default to picking the lowest level that can handle your frame size and fps, so generally sane values will be applied. But if you want something different, as in your case, seeing them manually does the trick nicely.