Problem about rate control of x265

[hanyi]

I use x265 to encode a 720p video with crf+vbv.
The commond as below:
./x265 --input wz_720p.yuv --preset medium --input-res 1280x720 --vbv-maxrate 1400 --vbv-bufsize 2000 --aq-mode 1 --rc-lookahead 16 --keyint 90 --bframes 3 --crf 24 --frame-threads 8 --fps 30 --psnr -o wz_720p.265

I chang the crf from 24 to 30,and I found something strange,when the bitrate achives the maxrate,the lower crf will case lower psnr.
24 1459.5768 40.8106
25 1456.7784 40.7936
26 1456.7792 40.7553
27 1457.0216 40.9836
28 1390.7492 40.9035
29 1233.576 40.2643
30 1093.8648 39.6247
From the result above,we can found the psnr of crf27 is bigger than crf26 while they have similar bitrate.
I have test other seqences,all have this phenomenon. The R-PSNR curve is not monotonic increasing,it will decrease when the bitrate exceeds the maxrate.
I want to know how to solve this problem,when the bitrate achives the maxrate how to make the psnr and bitrate not change with the crf decrease.

[benwaggoner]

This is a somewhat surprising result, but note that x265 by default doesn't even try to optimize for PSNR, as PSNR has pretty weak correlation with subjective quality. If you want x265 to actually optimize for psnr, use --tune psnr. I imagine it would be monotonic in that mode.

As the report is only the mean PSNR of the PSNR of each frame, it may be the different CRF levels are doing things that improve quality differently on different frames. For example, a lower PSNR average could result from I and P frames getting higher PSNR than B-frames, and thus resulting in better subjective quality.

Since mean PSNR just isn't a thing x265 normally cares about, it is free to do all kinds of subjective quality adaptations that don't yield monotonic PSNR.

[Asmodian]

What about when using two pass?

Using crf at settings which cause it to constantly hit the VBV limits is not an optimal rate control situation. VBV is supposed to be a rare protection to prevent buffer underflow/overflow issues, not something that limits the size of every frame. I think this is an issue of inappropriate settings.

[hanyi]

Originally Posted by Asmodian

What about when using two pass?

Using crf at settings which cause it to constantly hit the VBV limits is not an optimal rate control situation. VBV is supposed to be a rare protection to prevent buffer underflow/overflow issues, not something that limits the size of every frame. I think this is an issue of inappropriate settings.

So if given a bitrate,how to choose a proper crf? For example,give a 720p video and assign 1.4Mbps, which crf should set?

[hanyi]

Originally Posted by benwaggoner

This is a somewhat surprising result, but note that x265 by default doesn't even try to optimize for PSNR, as PSNR has pretty weak correlation with subjective quality. If you want x265 to actually optimize for psnr, use --tune psnr. I imagine it would be monotonic in that mode.

As the report is only the mean PSNR of the PSNR of each frame, it may be the different CRF levels are doing things that improve quality differently on different frames. For example, a lower PSNR average could result from I and P frames getting higher PSNR than B-frames, and thus resulting in better subjective quality.

Since mean PSNR just isn't a thing x265 normally cares about, it is free to do all kinds of subjective quality adaptations that don't yield monotonic PSNR.

I use --tune psnr,it's better,thank you.

[benwaggoner]

Originally Posted by hanyi

So if given a bitrate,how to choose a proper crf? For example,give a 720p video and assign 1.4Mbps, which crf should set?

If you want a specific bitrate, use --bitrate instead of CRF and do a 2-pass encode. Internally, x265 figures out and applies the CRF that would give you that ABR. If segments of the video are VBV constrained (and thus encoded with a higher rate factor), CRF will be lowered for the remaining part so the non-VBV constrained segments have as low a CRF as possible within the target ABR.

If you're focusing on PSNR, you don't want to use CRF anyway. CRF is essentially a psychovisual offset to QP, and optimizing PSNR means not using psychovisual optimizations.

Can you share what you're trying to do here?