x265 HEVC Encoder

[GeoffreyA]

Makes sense. Its somewhat different tools and ranges yield different-looking artefacts as a side effect, and x265's implementation of these tools may not be the gold standard. qcomp was helping, but as you said, just shifting around or pumping up the bits.

SVT-AV1 can give a better picture at the frame level, but going in the temporal direction, particularly with grain, we're walking on shaky ground. The HDR fork does better, but changes the appearance of the grain, making it coarser.

[benwaggoner]

Originally Posted by GeoffreyA

Makes sense. Its somewhat different tools and ranges yield different-looking artefacts as a side effect, and x265's implementation of these tools may not be the gold standard. qcomp was helping, but as you said, just shifting around or pumping up the bits.

SVT-AV1 can give a better picture at the frame level, but going in the temporal direction, particularly with grain, we're walking on shaky ground. The HDR fork does better, but changes the appearance of the grain, making it coarser.

And, as always, with a video codec, how it looks temporally is really the only thing that matters. Making still images look good in isolation is a much simpler problem than getting a coherent, consistent experience over time. And good psychovisually compression efficiency can involve doing things that can make individual frames paused look not great, but which are invisible during actual playback. Temporal masking is a powerful tool.

[Z2697]

I found that x265's SAO performs better or less bad in 12bit mode some time ago... maybe worth take a look.
But I cannot really understand that part of the code, and since AV1 has better filter(s) and better open source implementation(s), I'm not very motivated

[GeoffreyA]

Originally Posted by benwaggoner

And, as always, with a video codec, how it looks temporally is really the only thing that matters. Making still images look good in isolation is a much simpler problem than getting a coherent, consistent experience over time. And good psychovisually compression efficiency can involve doing things that can make individual frames paused look not great, but which are invisible during actual playback. Temporal masking is a powerful tool.

Yes, we tend to fixate on comparing single frames, but at the end of the day, it's the video. x264/5 seem mostly fine in this regard, the lesser-quality frames feeling consistent.

Originally Posted by Z2697

I found that x265's SAO performs better or less bad in 12bit mode some time ago... maybe worth take a look.
But I cannot really understand that part of the code, and since AV1 has better filter(s) and better open source implementation(s), I'm not very motivated

Well, 12-bit might be challenging to play on some hardware, and I've made peace parting with SAO for life!
Jokes aside, I've found some success with aq-mode 4 these past few days. So far, it's working well.

[Z2697]

Originally Posted by GeoffreyA

Well, 12-bit might be challenging to play on some hardware, and I've made peace parting with SAO for life!
Jokes aside, I've found some success with aq-mode 4 these past few days. So far, it's working well.

The problem is it shouldn't perform distinctively worse in 8 and 10 bits mode, hence there may be some bug in the code.
I mean the SAO mode decision, not the fps perform, you know.

[GeoffreyA]

Originally Posted by Z2697

The problem is it shouldn't perform distinctively worse in 8 and 10 bits mode, hence there may be some bug in the code.
I mean the SAO mode decision, not the fps perform, you know.

So, a precision sort of bug? Perhaps an overflow somewhere in the calculation?

[rwill]

Well you could give my gradient_with_noise sequence a try, with 10 and 12 bit, and check if its no longer blocking things up.

https://forum.doom9.net/showthread.p...76#post1949376

But I guess nothing did change. Last time I checked x265 had no real "mode decision" for SAO but only went by what it understands as "distortion". I remember debugging some large offenders and the numbers checked out, no bug there it seems.

[GeoffreyA]

Does x265's SAO use the same implementation as HM, the reference encoder?

[rwill]

Originally Posted by GeoffreyA

Does x265's SAO use the same implementation as HM, the reference encoder?

Well I guess they did not come up with something novel on their own....

[Z2697]

Originally Posted by rwill

Well you could give my gradient_with_noise sequence a try, with 10 and 12 bit, and check if its no longer blocking things up.

https://forum.doom9.net/showthread.p...76#post1949376

But I guess nothing did change. Last time I checked x265 had no real "mode decision" for SAO but only went by what it understands as "distortion". I remember debugging some large offenders and the numbers checked out, no bug there it seems.

Least "distortion" (and seems calculated in it's own way) is still some kind of mode decision... right?

I tested this same clip before, 12bit does not create same "blocky columns" as 10bit in the test, that's actually what I mean less bad.

But this kind of content should use "no operation" mode I think? 12bit still doesn't do that.

[rwill]

Originally Posted by Z2697

Least "distortion" (and seems calculated in it's own way) is still some kind of mode decision... right?

I tested this same clip before, 12bit does not create same "blocky columns" as 10bit in the test, that's actually what I mean less bad.

But this kind of content should use "no operation" mode I think? 12bit still doesn't do that.

Yes, picking a mode based on least distortion is a mode decision. I meant something like a full RD one with normal SSD and PSY distortion, like the CU mode decision.

That the clip does not exhibit the blocking at 12bit is interesting and probably worth investigating. Its strange that 8 and 10 bit pick the 'offset' mode and 12 bit does not. As for the optimal mode, it would be better if it were selected by real RD costs, that is, recon against source distortion with the (PSY-)RD distortion metric used everywere else.

[benwaggoner]

Originally Posted by GeoffreyA

Does x265's SAO use the same implementation as HM, the reference encoder?

That was my understanding back in the day, and I do not recall that ever changing.

[hellgauss]

Perhaps related to RD, I found an unwanted behavior in rd-refine option. The following screen is from Vinland Saga Season 1 Episode 16, source is blu-ray @34MBps

https://imgur.com/iwfhscB

NOTE: the screen is taken via VirtualDub2 and the artifacts are present in software decoder, so probably it is not the same MV issue I described a few months ago in another thread.

The flickering fire leads to fast moving gradient in a dark background, which is somewhat a worst case scenario for video encoding. There are a lot of similar random macroblocks in the whole scene, each last for 1 frames.

Bat file used (Windows):

set bf=6
set subme=4

for %%i in (0,1) do (
for %%j in (10,12) do (

rem i=rd-refine j=bit

start /low ffmpeg -threads 1 -y -hide_banner -nostdin -stats_period 33 -bitexact -i "Vinland-Saga_S1E16.mkv" -map 0:v -ss 206.580 -t 17.977 -profile:v main%%j -vf "removegrain=1:2:2,removegrain=0:2:2,format=yuv420p%%jle" -sws_flags accurate_rnd -c:v libx265 -preset veryslow -crf 17.2 -x265-params "deblock=-1,-1:no-sao=1:no-strong-intra-smoothing=1:tu-intra-depth=4:tu-inter-depth=4:merange=58:ref=6:rc-lookahead=99:bframes=6:rd=6:rd-refine=%%i:subme=4:vbv-maxrate=20000:vbv-bufsize=24924:frame-threads=1ools=none:no-wpp=1:level-idc=51:high-tier=1:colorprim=1:colormatrix=1:transfer=1:range=limited" -bitexact "Vinland-Saga_S1E16_r%%i_b%%j.mp4"
))

Writing library : x265 4.1+211-9e551a994:[Windows][GCC 15.2.0][64 bit]

If rd-refine=1, artifacts appears both in 10 and 12 bit. With rd-refine=0 there are no wrong macroblocks. However rd-refine seems to give nice results in other scenarios.

[Z2697]

That's a known issue, among some people.
https://github.com/Mr-Z-2697/x265-ex...2a4df2c730ad7a

However my test result didn't favor rd-refine even if it's supposed to be fixed (if I'm lucky).
However yet again I didn't test the ultimate single thread encoding like you do.

I'll port that branch into a latest x265 and compile a test build for anyone that might be interested.

https://pixeldrain.com/u/z9TjFxsh
or
https://workupload.com/file/kcWyHpU2rX3

[hellgauss]

An update: it seems that the rd-refine problem is also related to the limit-tu option.

Test script used (all encodes with rd-refine=1, loop through limit-tu=%%j)

set rdr=1
set bf=5
set subme=3

for %%j in (0,1,2,3,4) do (

rem j=limit-tu

start /low ffmpeg -threads 1 -y -hide_banner -nostdin -stats_period 33 -bitexact -i "Vinland-Saga_S1E16-rdr-scene.mkv" -map 0:v -profile:v main10 -vf "removegrain=1:2:2,removegrain=0:2:2,format=yuv420p10le" -sws_flags accurate_rnd -c:v libx265 -preset slower -crf 17.2 -x265-params "limit-tu=%%j:deblock=-1,-1:no-sao=1:no-strong-intra-smoothing=1:rc-lookahead=99:bframes=%bf%:rd=6:rd-refine=%rdr%:subme=%subme%:vbv-maxrate=20000:vbv-bufsize=24924:frame-threads=1ools=none:no-wpp=1:level-idc=51:high-tier=1:colorprim=1:colormatrix=1:transfer=1:range=limited" -bitexact "Vinland-Saga_S1E16_[rdr_%rdr%][ltu_%%j].mp4"
)

Results:
Macroblocks are evident with limit-tu =0 and 1. Less evident with 2, absent with limit-tu=3 or 4. Replacing with rd-refine=0, all tests present no "macroblocks" artifacts.

EDIT: thanks Z2697 for your build, I'll give it a try as soon as I setup a proper framework (I'm used to ffmpeg and not familiar with x265.exe, but I guess it is not difficult)

[jpsdr]

@Z2697
I took a look on your experimental repo, and see in your experiments the remove of distorstion chroma only for 444. Did test/check and finaly found that it produces better results ?

[Z2697]

Originally Posted by hellgauss

An update: it seems that the rd-refine problem is also related to the limit-tu option.

This bug is quite random, it may disappear here but can happen at other places.

[Z2697]

Originally Posted by jpsdr

@Z2697
I took a look on your experimental repo, and see in your experiments the remove of distorstion chroma only for 444. Did test/check and finaly found that it produces better results ?

See more detail in https://forum.doom9.org/showthread.php?t=186790
In theory, and in the way I assess chroma quality, it's better.

[GeoffreyA]

It seems to be more in line with x264, undoing that commit.

[benwaggoner]

Gosh, I hadn't thought about --rd-refine in years. I never found any examples where it provided a material difference. Have people found it helpful in some situations?