x265 HEVC Encoder

[Blue_MiSfit]

Great visualizations, LigH! Thanks for sharing

[Balling]

Sorry, Balling, but your explanation is not correct. All the channels (luma Y as well as chroma differences Cb and Cr) have the same bitdepth (say, precision of values). But due to Chroma Subsampling they don't have the same resolution.

The have the same bitdepth when converted to 444. In hardware 420 takes 10 bit for Y plane and 5 bit for both Cb, Cr planes. 15 bits max, and 1 bit as padding.

[Z2697]

The have the same bitdepth when converted to 444. In hardware 420 takes 10 bit for Y plane and 5 bit for both Cb, Cr planes. 15 bits max, and 1 bit as padding.

They have same bitdepth no matter what.
There's no such padding. The data in any case doesn't exist that way. YUV is either packed or planar.

[LigH]

As I said, there is a difference between technical per-sample values (a close-up look into the RAM) and statistical averages over a large number of pixels (seen from a distance).

[Z2697]

Scenecut not working when not using b frames.
https://bitbucket.org/multicoreware/x265_git/issues/1012/scenecut-not-working-when-bframes-0

Now the fix (or to bring it "on par" with x264) will also affect the "normal scenecut" (with b frames being used) slightly, presumably make it more accurate.
Does anyone wants to test? (The main reason I post this;))
https://pixeldrain.com/u/LRqwuw8L (Windows 10 x64 executable, 10bits only)

x264 default this to 1 (C doesn't have bool type, it's int).
It should only set to false when the scenecut is extremely unlikely, not default to false.

diff --git a/source/encoder/encoder.cpp b/source/encoder/encoder.cpp
index 44a09ba..19c7e0d 100644
--- a/source/encoder/encoder.cpp
+++ b/source/encoder/encoder.cpp
@@ -1605,7 +1605,7 @@ int Encoder::encode(const x265_picture* pic_in, x265_picture* pic_out)
inFrame[layer]->m_encodeStartTime = x265_mdate();
/* Set lowres scencut and satdCost here to aovid overwriting ANALYSIS_READ
decision by lowres init*/
- inFrame[layer]->m_lowres.bScenecut = false;
+ inFrame[layer]->m_lowres.bScenecut = true;
inFrame[layer]->m_lowres.satdCost = (int64_t)-1;
inFrame[layer]->m_lowresInit = false;
inFrame[layer]->m_isInsideWindow = 0;



in "bool Lookahead::scenecut" at encoder/slicetype.cpp:

if (bRealScenecut && m_param->bframes)
{
... do flash detection stuff ...
}

if (m_param->csvLogLevel >= 2)
{
... csv logging stuff ...
}

if (!frames[p1]->bScenecut)
return false;

return scenecutInternal(frames, p0, p1, bRealScenecut);


So without bframes - hence without going through that flash detection branch, and I guess that makes sense because only bi-directional prediction have a chance to code flashes more effieicntly - the function will always return false when bScenecut is default to false.

The bScenecut was changed from true to false in

commit f29f2e0071332b708b1d00e61332d12a237b4427
Author: Aarthi Thirumalai <none@none>
Date: Wed Jul 22 00:29:26 2015 +0530

slicetype: Modify Scenecut algorithm to detect scene transition points
to improve Rate Control (refs #160).

identify scene trasitions, fade-ins, fadeouts, sceneCuts and signal the flag bSceneCut
in Lowres structure. This flag will be used by RateControl to adjust the wps during scene cuts.

--HG--
branch : stable

Maybe it's not a good idea to change it back?

[hellgauss]

About a year ago I found a weird behaviour in x264 scenecut with high b-frame number and b-adapt=2. In x264 I use b-adapt=1 for this issue. The problem seems not to be present in x265, I hope that it will not be re-introduced in a future version.

https://forum.doom9.org/showthread.php?t=185544

The last graph I posted explains what to me seems to be a bug (not monotonically incresing curve with high b-frames and b-adapt=2). The graph are y=number of K/I frames, x=scenecut parameter.

[Z2697]

x265 seems to not care scenecut value too much, especially when you increase it, scenecut-bias has a more direct effect.
Whatever. I'm not planning to modify scenecut parameters.

[rwill]

About a year ago I found a weird behaviour in x264 scenecut with high b-frame number and b-adapt=2. In x264 I use b-adapt=1 for this issue. The problem seems not to be present in x265, I hope that it will not be re-introduced in a future version.

https://forum.doom9.org/showthread.php?t=185544

The last graph I posted explains what to me seems to be a bug (not monotonically incresing curve with high b-frames and b-adapt=2). The graph are y=number of K/I frames, x=scenecut parameter.

And you got told that this is no problem with x264 but its flash protection. Still yet here we are, again, with you stating its a problem.

[hellgauss]

And you got told that this is no problem with x264 but its flash protection.

It can be reasonable that, ceteris paribus, flash protection decrease IDR as bf increase. What I claim as "weird" is that, as long as bf is FIXED and very high and b-adapt=fixed=2, the IDR number as a function of scenecut parameter, in the "standard" range 25-70, is decreasing, which is in contrast with what I expect and with MEGUI wiki:

"Higher values of scenecut increase the number of scenecuts detected"

Which I understand as "If you want more IDR frames, increase scenecut".

If it is flash protection it is definitely a not wanted behaviour.

Now, this is x264 stuff and perhaps not in topic here... however Z2697 pointed out that

- New x265 code will be somewhat inspired by x264 behaviour
- Flash protection is related to this issue.

so I just wanted to point out my old thread.

[Z2697]

x265 is based on x264. Not will be.
Quite some drastic devalopments on the HEVC tools, but the fundamental design, especially the psychovisual optimizations, are very much like x264.

To my way of thinking, flash protection is "unrelated" to this issue. Not going thorough flash protection + the default value of bScenecut + the return false branch is the problem.

[DTL]

A simple and not very strict test shows no measureable difference.


In theory merange in MPEG encoders are only limitation for poor performance hosts and some trade between performance and quality. For best quality it always must search with full frame. And result significantly depends on the motion type/complexity in the tested footage. To get benefit from full frame search the test dataset must have lots of motion up to significant size of frame between each frame. And moved blocks must have not too many other types of transforms so a motion search engine can verify it is found. Yes - many natural scenes have much smaller motion vector lengths and typical merange values are simply statistically adjusted by developers to get some average performance/quality balance at some average footage. If we will have good progress in computing hosts performance limitation on merange expected to be removed (or relaxed to most long motion vector allowed in given MPEG standard).

Also currently used motion search methods may be still far for being perfect even with 'exhaustive' because at natural footages both 1 and 2 frames/blocks for motion search are damaged with random noise and simple block-tesselation of the frames can not give best results. Also the tesselation to blocks also creates some additional digital noise/distortion. Somehow better is 'area' next level of abstraction over simple block-based search when we run several shifted block searches in some area around current origin of the block (with radius up to blocksize/2) and make some statistical averaging of the motion vectors found in each search run. This adds an additional level of computing complexity many times.

Most (many) blocks/areas in many footage are not moving at all or moved by large areas with the same speed. So many computing resources for motion search of each block are wasted and completely useless. It may be better to design much more advanced 'adaptive' methods of all motion search part of MPEG encoder parameters control for each cutscene or at least footage types (fast sports or slow motion nature etc). Simply setting best/exhaustive/placebo may be too slow.

[Barough]

x265 v4.1+200-ffba52b
Built on October 25, 2025, GCC 15.2.0
Win32/64 / 8bit+10bit+12bit

DL :
https://www.mediafire.com/file/6jsdrsany41o6jk

[benwaggoner]

Also currently used motion search methods may be still far for being perfect even with 'exhaustive' because at natural footages both 1 and 2 frames/blocks for motion search are damaged with random noise and simple block-tesselation of the frames can not give best results. Also the tesselation to blocks also creates some additional digital noise/distortion. Somehow better is 'area' next level of abstraction over simple block-based search when we run several shifted block searches in some area around current origin of the block (with radius up to blocksize/2) and make some statistical averaging of the motion vectors found in each search run. This adds an additional level of computing complexity many times.
Noise is an interesting case. One (for the most part) benefit of performing coarse motion search at lower resolutions is that the low-pass filtering of the downscale averages out a lot of random noise. Of course, this also means that fine grain may only be analyzed at the final full resolution, throwing off rate control and other estimations a ton.

[benwaggoner]

x265 is based on x264. Not will be.
Quite some drastic devalopments on the HEVC tools, but the fundamental design, especially the psychovisual optimizations, are very much like x264.
I'd say that x265 includes pretty much all of x264's psychovisual optimizations and conceptual foundation. But it also includes features not in x264, most of which aren't applicable.

Thus CRP and --psy are foundational, but x265 also has --psy-rdoq which doesn't have a x264 equivalent. HEVC has SAO with its own set of psychovisual tradeoffs. Etcetera.

To my way of thinking, flash protection is "unrelated" to this issue. Not going thorough flash protection + the default value of bScenecut + the return false branch is the problem.[/QUOTE]

[rwill]

Thus CRP and --psy are foundational, but x265 also has --psy-rdoq which doesn't have a x264 equivalent.

--psy-rdoq is psy-trellis in x264. Nothing new there.

[DTL]

Noise is an interesting case. One (for the most part) benefit of performing coarse motion search at lower resolutions is that the low-pass filtering of the downscale averages out a lot of random noise. Of course, this also means that fine grain may only be analyzed at the final full resolution, throwing off rate control and other estimations a ton.

mvtools also uses hierarchical search of many 2x downsized levels. But this not helps a lot. The downsized blocks also lost significant original image details and probability of bad intermediate decision at some level of search increases and it can not be corrected at finer levels of search. Real best search is simple exhaustive search with *very large* radius better over total frame. And it is very demanding for compute resources.

If we have some low contrast patch damaged by comparable level of noise with details of the patch - most of motion estimation methods return lots of random motion vectors (even in the case of no motion at all at the large areas).

If we expect from MPEG encoder work as moving image encoder with sort of noise-removing tool (encode only moving image and not noise) - we need very advanced motion estimation engine in it. Increasing of motion search radius is not complete solution and only small step to the better future.

[benwaggoner]

If we expect from MPEG encoder work as moving image encoder with sort of noise-removing tool (encode only moving image and not noise) - we need very advanced motion estimation engine in it. Increasing of motion search radius is not complete solution and only small step to the better future.
I think the future is to de-noise and parameterize the source, encode as clean, and then use some kind of FGS to reconstruct the noise/grain.

Only AV1/AV2 are making FGS mandatory for decoders, though.

[Z2697]

Ever heard of bugs that only occur by chance when there's exactly 8 threads (logical processors) available to a process?
I've seen run to run inconsistency with x265 in such setups, unfortunately I don't have an actual 4C8T CPU that's readily able to run the test.
I have test with "8 core" VM (running Windows 11) and the CPU count limit in Windows 10 boot setting (it's a 8C16T laptop but I see no way to turn off SMT).
Encode same Y4M with just the default settings, and occasionally there will be a result file that's not the same with others.

I tried "8 core" VPS from Vultr on both AMD and Intel CPUs, and Arch Linux, same story...

[hellgauss]

My old laptop is 4 Cores - 8 threads. It is a 4700 MQ Haswell.

I'll do some tests, just a few questions:

- What do you mean with "occasionally"? Once each ... encodings ? How long is the y4m?

- If you can share a free source-sample and link a build for which you can reproduce the issue I can reproduct the same test and I can send CRC of my output(s). Also please provide the exact command line which cause the issue.

Note: I have Windows 10 64 bit on that laptop, and I cannot install W11. Also I have only 8Gb ram installed.

[Z2697]

The chance is pretty random, I have my test script run 64 encodings each batch, sometimes there's multiple "hits" per batch, sometimes 0.
So maybe it's 1 in 100 odds overall, but I don't keep track of all the runs...

The executable and OS doesn't matter (much), I have tested with Arch Linux and x265 from its packages.
But here's the executable I use in my VM:
https://pixeldrain.com/u/qqFXajdp

The test clips are some anime openings decoded to Y4M.
I have tested multiple clips that can reproduce the bug, but I assume the ones that are too "tranquil" are less likely to show up.
The type of the content shouldn't matter, I just found that anime openings conveniently have the length I like, and quite some motion in it.

The command line is just "x265 a.y4m {i}a.265", the absolute simplest.

[hellgauss]

The link does not work for me.

I have performed this test, with full gyan ffmpeg release 8.0.1 (20 Nov 2025)

for /L %%i in (0,1,30) do (
ffmpeg -y -bitexact -i "file.mp4" -c:v libx265 -f hevc -bitexact "output%%i.hevc"
)

for %%i in (*.hevc) do (
certutil -hashfile "%%i" MD5 >> testhash.txt
)


In 31 files, 29 are equals. Two are different, and also different between them. Surprisingly, the different ones are about 0.1% smaller.

Source is a 23 min full hd x264.

Here the .zip reports of my hevcqp tool (Macroblocks quant quant analyzer in txt, detailed stats in .dat) and mediainfo of source and outputs. The "standard" output is output0.

The .dat files contains
Line1=filename
Line2=codec
Line3= [MacrBlocks (MB) per frame] [Frame Number]
Then for each frame:
[Frametype] [Frame size] [Frame byte-start] [AVG QP] [STDEV QP of MB] [MIN QP of MB] [MAX QP of MB]

https://limewire.com/d/wGMIX#KqmgBy69cp

PS: If i had to bet, I bet it is wpp fault. Do not take this statement seriously, just a bet :)

[jpsdr]

@Z2697
BTW, you didn't describe what was the bug, or did i miss something ?

[Z2697]

The bug appears as run to run inconsistency, but I can't find the root cause.
Yes there're multiple other things that make encoding non-deterministic, and this is one of them.
I just feel it's very weird that it only happens at 8 threads and with "no good reason".

Turning WPP off seems to make the bug disappear, I need to run the thing more times.

[hellgauss]

Eheh I always encode single thread because I enjoy determinism. Multithreading is also a source of bugs, since it is very difficult to sync them (I'm not a programmer, I only know that there are a lot of complicated stuff related). Since I experienced no issues while tuning threads in x264, I guessed that the source should have been another one, and wpp is the other parameter related to MT.

A fast question on a somewhat related issue: since I use 1 threads, I usually perform parallel encodings. I recently started to try the -mcstf option. On clean source it seems that it has no effect on the output, except a tiny saving on space (it is OK, I know that it is for noisy source).

However I experienced a *HUGE* amount of ram used, which is a problem with parallel encoding. On 720p 4:2:0 10bit it takes 6.3GB of ram, which stabilyze at about 3GB after a few minutes. Without mcstf it is about 1.4GB. Is there any info on ram usage with mcstf?

Note: I'll do some tests changing rc-lookahead and bf. At this moment I use rc=99 and bf=16, with badapt 2.

[benwaggoner]

Eheh I always encode single thread because I enjoy determinism. Multithreading is also a source of bugs, since it is very difficult to sync them (I'm not a programmer, I only know that there are a lot of complicated stuff related). Since I experienced no issues while tuning threads in x264, I guessed that the source should have been another one, and wpp is the other parameter related to MT.
Do you mean --frame-threads 1 or full on --pools 1,0 or something. I use single frame threads by default as well, but leave on wpp and other forms of multithreading. At high resolutions you can still get a lot of CPU threads working at once.

However I experienced a *HUGE* amount of ram used, which is a problem with parallel encoding. On 720p 4:2:0 10bit it takes 6.3GB of ram, which stabilyze at about 3GB after a few minutes. Without mcstf it is about 1.4GB. Is there any info on ram usage with mcstf?
This is somewhat non-intuitive, and could reflect a bug. Perhaps it's caching full source and filtered frames in RAM for all of lookahead.

Note: I'll do some tests changing rc-lookahead and bf. At this moment I use rc=99 and bf=16, with badapt 2.
What's your --keyint? --rc-lookahead can go up to that, but you get diminishing returns. 16 b-frames is an extremely placebo option except for really low bitrate or really clean motion graphics stuff. And even then it's a <5% improvement. Look at the actual b-frame percentages from your encodes; it's not really worth keeping a b-frame count higher than you're seeing your 1% longest usage.

[hellgauss]

Those are parameters I'm using in one of my ongoing encoding via ffmpeg animmouse build. My target is anime, (quite) low bitrate and I'm not interested in encoding time, so I usually go to placebo. I also target low or adjustable CPU usage (I encode either on a fanless PC or on a PC which I also use for work). The memory issue for mcstf seems to be also in official builds. I know I can rise rc but 99 is quite ok for me.


-profile:v main10 -vf "removegrain=1:2:2,format=yuv420p10le,scale=1280:720:flags=bicublin:param0=0.25:param1=0.375" -sws_flags accurate_rnd -c:v libx265 -preset veryslow -crf 18.9 -x265-params "psy-rd=0.5:psy-rdoq=0.7:aq-mode=2:aq-strength=0.6:deblock=1,1:nr-intra=9:nr-inter=19:tu-intra-depth=4:tu-inter-depth=4:merange=58:ref=6:rc-lookahead=99:bframes=16:rd=6:rd-refine=1:subme=7:rskip=0:cbqpoffs=1:crqpoffs=1:scenecut=42:scenecut-bias=5.2:qcomp=0.58:tskip=1:tskip-fast=0:limit-sao=1:selective-sao=2:vbv-maxrate=7500:vbv-bufsize=10615:frame-threads=1:pools=none:no-wpp=1:mcstf=1:level-idc=40:high-tier=1:colorprim=1:colormatrix=1:transfer=1:range=limited"

[Z2697]

LOL mcstf
You'll regret

I'm not so sure about the little adjustments you made to some parameters, but judging from the fact that you think mcstf is ok, you are just doing that for the false feeling of "safety".

[hellgauss]

@Z2697
Could you please elaborate more? Which side effect can have mcstf, other than memory usage? I read that it is very useful on noisy source, but I only performed a few tests on clean one. Frames type choose is equal, size is 0.02% less, quantizer is (almost) identical. Visually it seems ok, but I did not check on big screen.

I also read that it is applied on I/P frames so perhaps it is not so useful with 16bf. However at least 5 bf are required to activate the option.

[microchip8]

6 to 8 bframes are an optimal value. Using 16 is worthless and placebo

Also, consider using aq-mode 1 as 2 has issues with smearing due to the fact x265 devs decided to change its values from x264 and this is what you get.

Not sure why you have increased deblock? Do you get blocking at default values of 0? If not, set it to default. The stronger deblocking is, the more softer image it'll produce

Also, increasing the qp offsets actually makes the encoder compress the blue & red chroma channels more strongly, resulting in less vivid colors.

[GeoffreyA]

Those are parameters I'm using in one of my ongoing encoding via ffmpeg animmouse build. My target is anime, (quite) low bitrate and I'm not interested in encoding time, so I usually go to placebo. I also target low or adjustable CPU usage (I encode either on a fanless PC or on a PC which I also use for work). The memory issue for mcstf seems to be also in official builds. I know I can rise rc but 99 is quite ok for me.

How is the removegrain filter, which I see in your command line? I've been having trouble with the original Blade Runner, but can't use VapourSynth because of libplacebo issues, so options are limited. --nr-inter works but I can see the difference at low settings for this film.

[hellgauss]

@GeoffreyA
It is very nice. It is very useful to reduce bitrate without hurting too much, especially (but not only) with grain. I use it quite as a default. Before the resize is less effective and more soft. I suggest only mode 1 (very soft) or 2 (soft), eventually mode 2 can be applied twice. That command apply mode 1 to luma and 2 to chromas. It actually helps to keep the grain because allows you to reduce crf.

Remarks:
- when used with ffmpeg, it automatically convert the source to 8 bit. So use it before colorspace conversion. If you need it after the resize, I suggest avisynth (I never used vapoursynth, I do not know about it), since resize is better at 10bit.
- It can damage the dithering pattern in HQ source if present.
- Please note that the above CLI is for quite clean and not HQ anime source (WEB), and for low bitrate target. I used noise reduction mainly to lower the bitrate.

[GeoffreyA]

@GeoffreyA
It is very nice. It is very useful to reduce bitrate without hurting too much, especially (but not only) with grain. I use it quite as a default. Before the resize is less effective and more soft. I suggest only mode 1 (very soft) or 2 (soft), eventually mode 2 can be applied twice. That command apply mode 1 to luma and 2 to chromas. It actually helps to keep the grain because allows you to reduce crf.

Remarks:
- when used with ffmpeg, it automatically convert the source to 8 bit. So use it before colorspace conversion. If you need it after the resize, I suggest avisynth (I never used vapoursynth, I do not know about it), since resize is better at 10bit.
- It can damage the dithering pattern in HQ source if present.
- Please note that the above CLI is for quite clean and not HQ anime source (WEB), and for low bitrate target. I used noise reduction mainly to lower the bitrate.

Thanks for the detailed description. I'll give it a go tomorrow. I've found that KNLMeansCL with toned-down parameters gives near-transparent results, while cutting down bitrate a fair bit. BM3D is seemingly on par, too, but impractically slow without CUDA.

[Z2697]

I was gonna write "mcstf causes ghosting" and then I thought "what would it be in single thread".
Well, turns out mcstf does not like pool features wpp and lookahead-slices. frame-threads are fine (but it only gets supported recently).

[LigH]

I created an issue in their bug tracker (https://bitbucket.org/multicoreware/x265_git/issues/1019/wpp-with-many-threads-and-mcstf-together).

It would be useful to explain better, because I still don't understand: Is there any visible artifact in the decoded video, or is it just invisible differences in the encoded video bitstream, causing only a different file but no obvious video artifacts?

[hellgauss]

No evident artifact for my 4C8T test.

Also I noticed an interesting thing. For both files the divergence seems to start at the same GOP near the end of the video (you can check in my .dat files with WinMerge file compare).

Differences persists in some of next GOPs. Of course the byte-start will be no longer recovered, but other fields sometimes are equal and sometimes not.

PS: your issue link does not work

[LigH]

For me it does. One might need to log in to access it? Or it requires a quite recent web browser generation with support for a modern JavaScript version (e.g. it does not work on SeaMonkey).

[Z2697]

The issue needs to be "reviewed" by mcw, before that the issue is not visible to the public...
I don't know the exact reason behind this, but there was a time that some spam message flooded the issue tracker.
Who would have expect, I mean really, spamming message in such place???

[Z2697]

For me it does. One might need to log in to access it? Or it requires a quite recent web browser generation with support for a modern JavaScript version (e.g. it does not work on SeaMonkey).

The issue is open now, but the two are not related.
I've created a issue earlier and also opened now, that only contains the "8 thread" bug.
https://bitbucket.org/multicoreware/x265_git/issues/1017/run-to-run-inconsistency-with-exactly-8

Not exactly duplicate so maybe you can edit your issue to only mention the MCSTF bug?
And it literally creates severe ghosting.

[LigH]

I created this issue because I misunderstood you, I believed you were not able to create one yourself...

You probably want post #9883 (https://forum.doom9.org/showthread.php?p=2025825#post2025825) reported.

[benwaggoner]

Also, increasing the qp offsets actually makes the encoder compress the blue & red chroma channels more strongly, resulting in less vivid colors.
Yeah, I've only ever seen positive chroma qp offsets helpful in Dolby Vision Profile 5 to compensate for Y'CtCp instead of Y'CbCr. Other than that, I've only seen negative values be helpful.

[hellgauss]

I used quite low crf for a "low" bitrate encoding i.e. 18.9. It reduce artifact and banding. See qpoffset as an option to increase luma quality than to lower chroma. I also degraded chroma more than luma in other ways (bicublin resize, removegrain 2 on chroma). Each option has a low/moderate effect on filesize, but "E' la somma che fa il totale (*)"

(*) "The sum make the total" cit. of italian actor Totò

[tormento]

On the very same CPU, what is the speed difference between x265 AVX only encoding and AVX2 one?

I have a server with a Haswell Xeon 1275v3 that, even if it has lower speed than my ancient i7-2600k, it supports AVX2 instruction set.

I think I could have a nice speed bump using it but before installing windows etc I'd like to see some numbers.

Thank you ;)

[Z2697]

On the very same CPU, what is the speed difference between x265 AVX only encoding and AVX2 one?

I have a server with a Haswell Xeon 1275v3 that, even if it has lower speed than my ancient i7-2600k, it supports AVX2 instruction set.

I think I could have a nice speed bump using it but before installing windows etc I'd like to see some numbers.

Thank you ;)

I'd say about 20%

[jpsdr]

For integer calcul, SSE2/AVX is the same, still only XMM registers. It's only on AVX2 there is YMM registers for integer calcul, so AVX vs AVX2 is probably the same than SSE2/AVX2. Meaning i also think the same than Z2697.

[GeoffreyA]

What is the usual way, in x265, of mitigating "microbanding" introduced by the encoder?

[Z2697]

What kind of "microbanding"?

[GeoffreyA]

Thin, watercolour-like streaks, appearing rarely, sometimes for a frame or two, on skin gradients. A lower CRF or CQP does eliminate it, so it is happening when less bits are available. 10-bit encoding seems to tone down but not eliminate it. I have been testing on one frame: not a good approach, I know, but this makes it easier to narrow down what is causing it. I noticed that SVT-AV1 (Blue's latest fork) is not afflicted with this artefact; but switching to that would be more problematic, owing to its rate-control issues.

Happy New Year, by the way. May everyone's encodes be artefact free and saturate all cores.

[Z2697]

I was thinking maybe you can upload an image.
But anyways, assuming you mean something like this:
https://files.catbox.moe/g8wbtc.webp

It's caused by intra prediction (especially on larger blocks that's more noticeable) and not enough bits to signal residual that's able to complete it.
AV1 does have the same problem (and even AV2 will) as long as they use the same/similar intra prediction method.
In fact, the example image is from an AV1 intra frame. ;)

Inter frames can have intra blocks as well.
The solution is just give it more bits, or I assume AV1 is doing some filtering to cover it. (or they are in different places you don't notice)

[GeoffreyA]

Thanks, Z2697. Yes, giving more bits does help. I was going through each common setting, trying to see if one was the culprit.

https://slow.pics/s/FqMnaMTo

-c:v libx265 -preset veryslow -crf xy -x265-params no-sao=1:deblock=-1,-1:aq-mode=1

See the arm. psy-rd and psy-rdoq help to restore the detail but the "banding" is still present, albeit not so noticeably.

[Z2697]

It's just a "natural" result from the coding tools available in HEVC and how x265 devs implement them. (or how poorly, say SAO)
Whatever the parameters you tweak that may make it look better is just tossing the bits around. Unless you want to give SAO a try.

Intra prediction angle modes are like painting the neighboring pixel straightly in a direction.
AVC has smaller blocks and fewer intra angles, so it tends to look blurry instead of painted.
AV1 has more filters that, I assume [will cover up some of it and SVT-AV1 devs implemeted well].