x265 HEVC Encoder

[jpsdr]

@LeXXuz
No, mode 5 is like the others, always flagged at 5, it's the aq-auto which changes the mode flags according scenery. When using aq-auto, the mode set in aq-mode became irrelevant.

Otherwise, about building x265. Just tested with standard msys2 gcc, it seems that in multilib, despite the -DCMAKE_CXX_FLAGS_RELEASE, the library are not linked... :(
I'll try to add -DCMAKE_EXE_LINKER_FLAGS at the last step, if it doesn't work, don't know what to try...

Edit:
Failed... :(
Even when adding -DCMAKE_EXE_LINKER_FLAGS at the last step, the multilib build is not static.
I take any advice...

[Boulder]

Thanks for that explanation. :) I was just wondering because sometimes it shows aq-mode=5 which would be correct and sometimes all the others.

Guess I had the wrong understanding of mode 5. I thought it is a mode of its own which behaves like one of the other modes depending on scenery, therefore constantly flagged as mode 5 throughout the entire stream.

So it is more like a 'fictitious' mode which switches through the other modes depending on scenery and changing the mode flags accordingly? That still confuses me a little. Because in that case Mediainfo should never display aq-mode=5 for some files, no?
If you want to use only aq-mode 5, do not set --aq-auto at all. If you want to use --aq-auto, with SDR the optimal setting is --aq-auto 10, and no need to specify --aq-mode 5 because that mode is then already enabled. The encoder will decide frame-by-frame which mode out of 2-5 it will use.

[jpsdr]

So, the results of my strugles are :
The gcc version provided on www.msystem.waw.pl/x265 is a static build, thanks StvG tips using external cmake, i've been able to make a multilib build, and no needs to add static flags.
This is the only gcc version i've been able to make a static multilib with.
With the others gcc, i'm able to make a static build only on a single lib build. I took any tips to be able to make multilib with them two (the only i'm realy interested in is the mcf).

[LeXXuz]

If you want to use only aq-mode 5, do not set --aq-auto at all. If you want to use --aq-auto, with SDR the optimal setting is --aq-auto 10, and no need to specify --aq-mode 5 because that mode is then already enabled. The encoder will decide frame-by-frame which mode out of 2-5 it will use.

Ah. Now I get it. I thought the --aq-auto 10 mode just replaces the old "--sbrc --sbrc-aq5 --sbrc-hyst" parameters. I didn't know it also switches aq-modes depending on scenery. :)

[Boulder]

Ah. Now I get it. I thought the --aq-auto 10 mode just replaces the old "--sbrc --sbrc-aq5 --sbrc-hyst" parameters. I didn't know it also switches aq-modes depending on scenery. :)

--sbrc used to be like --aq-auto is now. The x265 devs just accidentally pushed incorrect code as SBRC, which is a very different feature in the fixed code now available.

[tormento]

It's a bit I've been here so tl;dr ;)

Is there a "table" with all the new switches? Where can I find an updated version, compatible with Staxrip reformatting?

[jpsdr]

Yeah... The clang build is twice faster than my gcc builds ! :(
It seems that even if the default setting is "Release", i suspect the compiler options being... poor. It's not possible that this speed difference comes only because of the compiler...

[Boulder]

Yeah... The clang build is twice faster than my gcc builds ! :(
It seems that even if the default setting is "Release", i suspect the compiler options being... poor. It's not possible that this speed difference comes only because of the compiler...

Assembly not in use?

[jpsdr]

No, it's not that...
I spend a lot of time reading the gcc optimize options, and i'll try rebuild adding a bunch of option... ;)
If it doesn't help, i'll stop wasting time and just provide clang_AVX2 build, period.

[StvG]

Thanks.

Any idea why this works:
...
and this doesn't:mkdir -p 8bit_x64 10bit_x64 12bit_x64

cd 12bit_x64
cmake -G "MSYS Makefiles" ../../../source -DHIGH_BIT_DEPTH=ON -DEXPORT_C_API=OFF -DENABLE_SHARED=OFF -DENABLE_CLI=OFF -DMAIN12=ON -DCMAKE_CXX_FLAGS_RELEASE="-static-libgcc -static-libstdc++ -static"
make ${MAKEFLAGS}
cp libx265.a ../8bit_x64/libx265_main12.a

cd ../10bit_x64
cmake -G "MSYS Makefiles" ../../../source -DHIGH_BIT_DEPTH=ON -DEXPORT_C_API=OFF -DENABLE_SHARED=OFF -DENABLE_CLI=OFF -DCMAKE_CXX_FLAGS_RELEASE="-static-libgcc -static-libstdc++ -static"
make ${MAKEFLAGS}
cp libx265.a ../8bit_x64/libx265_main10.a

cd ../8bit_x64
cmake -G "MSYS Makefiles" ../../../source -DEXTRA_LIB="x265_main10.a;x265_main12.a" -DEXTRA_LINK_FLAGS=-L. -DENABLE_CLI=ON -DENABLE_SHARED=OFF -DLINKED_10BIT=ON -DLINKED_12BIT=ON -DSTATIC_LINK_CRT=ON -DCMAKE_CXX_FLAGS_RELEASE="-static-libgcc -static-libstdc++ -static"
make ${MAKEFLAGS}

# rename the 8bit library, then combine all three into libx265.a using GNU ar
mv libx265.a libx265_main.a

ar -M <<EOF
CREATE libx265.a
ADDLIB libx265_main.a
ADDLIB libx265_main10.a
ADDLIB libx265_main12.a
SAVE
END
EOF


For both default gcc from msys and mcf gcc after you run the script go to 8bit_x64/CMakeFiles/cli.dir and open build.make. Go to the line starting with msys64/mingw64/bin/c++.exe and delete -Wl,-Bdynamic. Go to 8bit_x64 and run make ${MAKEFLAGS} to rebuild the exe file.

[jpsdr]

@StvG
Again, you save the day !
:thanks:

Otherwise, as i've said, after searching/reading docs on optimize command for gcc, i've added twice a dozen of options, rebuild with them, and finaly got a .exe that's as fast (almost, at less than 1%... :D) than the clang build.

Edit:
Finaly, it didn't work, there is a loooot of lines begining with msys64/mingw64/bin/c++.exe, there is no -Bdynamic in it, and just one line with several -Wl.
I've removed them, but when running make ${MAKEFLAGS} it fails with errors.
The win32 thread version is working fine, i've been able to figure out the commands line to add to build a not slow exe, i'll stop here with gcc, as i have a working process.
I'm now trying to find a process allowing me to make multilib with Visual Studio + llvm.

Edit2:
Been able to do multilib with VS.

[StvG]

Edit:
Finaly, it didn't work, there is a loooot of lines begining with msys64/mingw64/bin/c++.exe, there is no -Bdynamic in it, and just one line with several -Wl.
I've removed them, but when running make ${MAKEFLAGS} it fails with errors.
The win32 thread version is working fine, i've been able to figure out the commands line to add to build a not slow exe, i'll stop here with gcc, as i have a working process.
I'm now trying to find a process allowing me to make multilib with Visual Studio + llvm.

Just tested with default msys2 gcc and it's fine here.

https://i.slow.pics/WoKl4fIq.png
https://i.slow.pics/sUg1ju0r.png

[jpsdr]

My file wasn't like that.
Also, i wanted to say "i'll stop here with others gcc"

[jpsdr]

New release, check my github.
And now i'm able to make multilib, yeah !! :D

[Barough]

x265 v3.5+93
Built on February 18, 2023, GCC 12.2.0

DL :
https://www.mediafire.com/file/2nwbzkhcmwiplv3

[jpsdr]

Seriously...! A new version just right now after all my strugles... :(
They made it on purpose, just after i put my build i spend all this time on...
Have to wait a little time unfortunately for a new build.

[Boulder]

Seriously...! A new version just right now after all my strugles... :(
They made it on purpose, just after i put my build i spend all this time on...
Have to wait a little time unfortunately for a new build.

The latest commits don't concern the regular Windows binary at all I think.

[jpsdr]

Except maybe just one on the asm file, not being sure, i've made new builds, you can get them on my github.

[jpsdr]

Noooooooooooooooooooooooooooooooooooooooooooooooooooooo !!!!!!!!!!!!!!!!!!!!!!!!

[john33]

x265 v3.5+94
Built on February 20, 2023, VS2019 (AVX2 compiler optimisation)

DL: https://www.rarewares.org/files/x265-r3.50.94-20230220.zip

[Morku]

Noooooooooooooooooooooooooooooooooooooooooooooooooooooo !!!!!!!!!!!!!!!!!!!!!!!!

xD Love your build.

[filler56789]

x265 v3.5+94
Built on February 20, 2023, VS2019 (AVX2 compiler optimisation)

DL: https://www.rarewares.org/files/x265-r3.50.94-20230220.zip

Does your build also encode audio, like some ancient mods of x264? :)

[LeXXuz]

Noooooooooooooooooooooooooooooooooooooooooooooooooooooo !!!!!!!!!!!!!!!!!!!!!!!!
LOL. Someone over there must know when you just finished your work. :D:D:D
SCNR!:devil:

Really appreciate your builds. Still using +87 and I'm very happy with it. See no reason to update any time soon. ;)

[john33]

Does your build also encode audio, like some ancient mods of x264? :)

Just a current git build. :)

[jpsdr]

Made a new build, check my github.

[john33]

Since (and including) r3.50.0.87.1, the " // solve "fatal error C1061: compiler limit : blocks nested too deeply"" error has returned with attempted MSVC compiles of the "Patman" version. (source\common\param.cpp) GCC doesn't exhibit this issue. The standard git version does not have this problem so it is obviously to do with the "Patman" mods. And, @jpsdr, you'll be happy to know there is another update! ;)

[filler56789]

And, @jpsdr, you'll be happy to know there is another update! ;)

:devil: :D

That's why I'll keep using version 3.3+10 forever😇

[john33]

x265 v3.5+95
Built on February 22, 2023, VS2019 (AVX2 compiler optimisation)


DL:https://www.rarewares.org/files/x265-r3.50.0.95-20230222.zip

[Barough]

x265 v3.5+96
Built on February 22, 2023, GCC 12.2.0

DL :
https://www.mediafire.com/file/lvwa18uv48f4aq9

[jpsdr]

It's not even funny anymore...

Edit:
After looking the commits, not offect on PC, so no build necessary... :D
Yes !!!

[jpsdr]

The standard git version does not have this problem so it is obviously to do with the "Patman" mods.
It's not the Patman mods, it's the commands i've added.
This make me adds parameters description, and with the standard version it's very close to the limits.
If the standard version adds 3 or 4 others commands, it will hit also the limit.

[john33]

It's not the Patman mods, it's the commands i've added.
This make me adds parameters description, and with the standard version it's very close to the limits.
If the standard version adds 3 or 4 others commands, it will hit also the limit.

So I guess we're already there. ;)

[~ VEGETA ~]

hello,

in my setup i encode lossless 1080p first, then use it to encode final 1080p, 720p, etc.. this is usually in x264, qp 0 ultrafast for lossless which gives me fast enough results.

can I use that output as input to x265?

my lossless is .mkv but i can easily make it .yuv or .y4m if needed. I tried inputting that to x265 but it didn't take all frames even when specified, then outputted garbage video (full of colors and no actual content).

I prefer a direct usage of x265 rather than piping from vspipe or so.

for your kind input please.

[LigH]

New upload: x265 3.5+94-74a312564 (https://www.mediafire.com/file/grue0bgcpsjbuh4/x265_3.5+94-74a312564.7z/file)

[Windows][GCC 12.2.0][32/32XP/64 bit] 8bit+10bit+12bit

[jlpsvk]

it's better to use --aq-mode 2 or --aq-auto 6 for 4K HDR? both giving me almost same bitrate at CRF16 with preset slower (with some manual settings)

[benwaggoner]

it's better to use --aq-mode 2 or --aq-auto 6 for 4K HDR? both giving me almost same bitrate at CRF16 with preset slower (with some manual settings)
--aq-mode 2 is pretty optimal for 4K HDR encoding. The --aq-auto might be just picking that mode repeatedly, hence not much of a bitrate difference. Have you checked visually?

[redbtn]

--aq-mode 2 is pretty optimal for 4K HDR encoding. The --aq-auto might be just picking that mode repeatedly, hence not much of a bitrate difference. Have you checked visually?

Do you still recommend using --aq-mode 2 over --aq-mode 4 for 4K HDR?
I use --aq-mode 4 for SDR, but not sure about HDR.

[benwaggoner]

Do you still recommend using --aq-mode 2 over --aq-mode 4 for 4K HDR?
I use --aq-mode 4 for SDR, but not sure about HDR.
I use --aq-mode 4 for SDR and --aq-mode 2 for HDR.

[ShortKatz]

I recently started to use --aq-mode=4 for everything. It has been said --aq-mode=4 is for "videos with high edge contents (Ex: Ducks take off, Crowd run, Raindrops etc)". (1) And because this can happen in every video, I use 4 now with every kind of video. I've noticed that the encoding time is reduced for --aq-mode=4 compared to --aq-mode=2. I also visually like the result better (HDR and SDR).
(1) https://www.mail-archive.com/x265-devel@videolan.org/msg11661.html

[benwaggoner]

I recently started to use --aq-mode=4 for everything. It has been said --aq-mode=4 is for "videos with high edge contents (Ex: Ducks take off, Crowd run, Raindrops etc)". (1) And because this can happen in every video, I use 4 now with every kind of video. I've noticed that the encoding time is reduced for --aq-mode=4 compared to --aq-mode=2. I also visually like the result better (HDR and SDR).
(1) https://www.mail-archive.com/x265-devel@videolan.org/msg11661.html
In extensive testing, I found --aq-mode 2 to be slightly better overall compared to 4 for CVBR encoding of premium 24p HDRcontent, given a number of other parameters that were also used. But certainly 4 is probably better for some content and/or parameter combinations. The real comparison should probably be between the modes both with their own optimal --aq-strength, as generally --aq-mode 4 tends to be optimal with somewhat lower strengths than the other modes.

It's also possible that the --auto-aq mode isn't optimally tuned for HDR content; I've not tested with the feature myself.

[ShortKatz]

Yes, I still need to find the optimal --aq-strength. This will probably take some time.

[benwaggoner]

Yes, I still need to find the optimal --aq-strength. This will probably take some time.
And aq-strength is pretty content dependent, lower for cel animation and high grain, higher with cleaner natural images and CGI.

[ksks]

Hello,
I noticed that a trouble took place when someone was trying to use --rd-refine in order to encode videos with better quality and smaller size. I wonder if that could been solved as they thought.

https://bitbucket.org/multicoreware/x265_git/issues/630/rd-refine-will-cause-some-glitch-in-some
https://github.com/Mr-Z-2697/x265-Yuuki-Asuna/commit/8d78b0fa48d09f0f2b5a58372d573a4711a432fa#commitcomment-103498859

[benwaggoner]

I just noticed today is the 2nd anniversary of x265 3.5.

We've never had even a one year gap from releases before now. I hope a 3.6 is coming before long. There have been quite a few new features and fixes it would be nice to have in a official build.

[Barough]

I just noticed today is the 2nd anniversary of x265 3.5.

We've never had even a one year gap from releases before now. I hope a 3.6 is coming before long. There have been quite a few new features and fixes it would be nice to have in a official build.

It's been very slow going with x265 for sometime now. I sure do hope for v3.6 in a near future.

[ShortKatz]

Yes, the development of x265 has become really slow. My guess is that MCW is focusing more on x266 at the moment and therefore neglecting x265.

[~ VEGETA ~]

I have ryzen 7900x cpu with nvidia rtx 3060ti, which version should i use? what is the best website to get these up to date binaries from?

also, any specific cli commands regarding my cpu which can increase performance?

[LigH]

The GPU does not matter at all for x265.

The CPU supports AVX2 which any x265 build for an intel/AMD compatible CPU will detect and use automatically. It also supports AVX512 what you would have to enable by a command line parameter, but it will heat up the CPU a lot more and may get its clock rate throttled, so it may be faster or not, but will be a risk for reliability.

The most important core functions of x265 have CPU optimizations in several variants. There are also builds out there which are further optimized in rarely used functions, so their gain will probably be negligible. It is probably much more important to find a sane balance between multithreading of prefiltering (in AviSynth or VapourSynth or whatever conversion system you prefer) and multithreading in x265, to utilize all the cores without severe bottlenecks.

[Lan4]

Tell me the best x265 build for a processor with AVX only. Or similarly, build doesn't matter, they all work the same way?

[LeXXuz]

I have an argument with someone.
If I increase aq-strength, do high motion areas/scenes get more or less bits?

[benwaggoner]

I have an argument with someone.
If I increase aq-strength do high motion areas/scenes get more or less bits?
What aq-strength does is baed on the --aq-mode being used. Generally aq-modes are about preventing blurring in smoother areas caused by more naive PSNR or SAD type metrics. In general x265 will spend fewer bits on areas of high motion that aren't good references for frames later in the GOP; --cutree is the main tool to identify those. This results in adaptive quantization, absolutely, but not directly controlled by --aq-mode or --aq-strength. I'm sure there are some interactions in practice, as all kinds of things can change how x265 allocates bits.

[benwaggoner]

Tell me the best x265 build for a processor with AVX only. Or similarly, build doesn't matter, they all work the same way?
That is going to be some old hardware. It wouldn't take that many compressed titles before it'll be cheaper on electricity to buy a newer, faster, and much more efficient CPU. The fastest pre-AVX CPU probably will have <10% the throughput of a modern enthusiast chip, which will have many more cores, AVX2, and overall better instructions per clock.

[Lan4]

That is going to be some old hardware. It wouldn't take that many compressed titles before it'll be cheaper on electricity to buy a newer, faster, and much more efficient CPU. The fastest pre-AVX CPU probably will have <10% the throughput of a modern enthusiast chip, which will have many more cores, AVX2, and overall better instructions per clock.
Buying a new processor is too easy. :) And for the purchase of a processor, you will need a new motherboard, new memory, a new power supply. Still, my question was about the x265 codec.

[Boulder]

Tell me the best x265 build for a processor with AVX only. Or similarly, build doesn't matter, they all work the same way?

They work the same way, the encoder uses the instruction sets that are detected.

You might get some performance benefits from compiling the executable with specific tuning for your CPU. Media Autobuild Suite can do this, there is a possibility to tweak the compiler parameters accordingly (-march and -mtune in specific).
https://github.com/m-ab-s/media-autobuild_suite#optional-user-files

[LigH]

Just committed:
[PATCH] Fix bug in mcstf

Building...

[Barough]

x265 v3.5+97
https://www.mediafire.com/file/bu7j9tn2c98sw5i

[LigH]

New upload: x265 3.5+97-c666bc3d3 (https://www.mediafire.com/file/9jn1cirkuyz8f0i/x265_3.5+97-c666bc3d3.7z/file)

[Windows][GCC 12.2.0][32/32XP/64 bit] 8bit+10bit+12bit

[jpsdr]

Hello.

Made a new build of my custom mod, check my Github.

Also, i've made out of curiosity some speed tests on my Core i7 6950X, encoding time results :
3.50.0.97 AVX2 GCC 12.2.1 : 614.55s
3.50.0.97 Broadwell GCC 12.2.1 : 608.67s
3.50.0.94 Broadwell LLVM 15.0.7 : 600.10s
3.50.0.97 Broadwell LLVM 16.0.0 : 590.88s

Up to 4% speed difference, well, anything is good to take :D

[FranceBB]

Buying a new processor is too easy. :)

Well, if your CPU is AVX capable only, get the AVX build: http://msystem.waw.pl/x265/

it really is as simple as that...

[john33]

Also did some speed tests with results that kind of surprised me:
Ryzen 9 3900XT - Win10 Pro x64
61,070 frames - 1920x816 --crf 24.0 --aq-mode 4 --no-cutree --no-open-gop --no-sao

Winlibs GCC 13.0.1 1,946.89secs 31.37fps
Winlibs GCC 12.2.0 1,928.15secs 31.67fps
Msys2 GCC 12.2.0 1,903.41secs 32.08fps
http://www.msystem.waw.pl/x265/ GCC 12.2.0 1,898.15secs 32.17fps
LLVM Clang 16 1,718.27secs 35.54fps
VS2019 V16.11.25 1,717.45secs 35.56fps
Intel 19.2 1,600.23secs 38.16fps These were all AVX2 optimised with similar settings.

[FranceBB]

New upload: x265 3.5+97-c666bc3d3 (https://www.mediafire.com/file/9jn1cirkuyz8f0i/x265_3.5+97-c666bc3d3.7z/file)

[Windows][GCC 12.2.0][32/32XP/64 bit] 8bit+10bit+12bit

Not that it matters, but just so you know and the XP community doesn't find out the hard way, the XP build you provided is attempting to call GetTickCount64 which doesn't exist in XP x86, while it should really be calling GetTickCount.

https://i.imgur.com/O0eP498.png
https://i.imgur.com/hPUwAGe.png

Sending GetTickCount64 to GetTickCount fixes the issue of course:

https://i.imgur.com/bcWTp4i.png
https://i.imgur.com/ojeSHqO.png

I don't know if something changed under the hood in the way you compile (or if indeed is an autobuild issue?), but... well... now you know. ;)

[LigH]

Well, thank you for your discovery ... but don't tell me. I only compile. I do not develop. Multicoreware needs to know (via mailing list) and find a workaround if they still officially support XP compatibility at all. Instead they may probably just call it obsolete.

[benwaggoner]

Well, thank you for your discovery ... but don't tell me. I only compile. I do not develop. Multicoreware needs to know (via mailing list) and find a workaround if they still officially support XP compatibility at all. Instead they may probably just call it obsolete.
Given that there is no support or security updates for XP, I think it makes sense to not offer builds for it anymore as that's encouraging people to stick with insecure systems.

For embedded applications things are different, of course.

XP doesn't have explicit NUMA support and is missing a lot of other things helpful for maximizing performance.

[filler56789]

XP doesn't have explicit NUMA support and is missing a lot of other things helpful for maximizing performance.
That's a very-good reason for an HEVC encoder to drop Windowx XP support. :goodpost:

OTOH,
Given that there is no support or security updates for XP, I think it makes sense to not offer builds for it anymore as that's encouraging people to stick with insecure systems.

is just a lame excuse for trollish devilopment. :p

Seriously: 🆖

[LigH]

I don't mind unmounting a dead horse. There are plenty old but functional versions.

[FranceBB]

Given that there is no support or security updates for XP

Although Microsoft pulled support in mid 2019 with the last security update, there's still 0Patch that picked up the task. That + a good antivirus like Avast and it's good enough for a home user, probably.
Anyway, I found out 'cause I do regular testing on all environments in a VM and one of those is indeed XP.


I think it makes sense to not offer builds for it anymore

Nah, as I showed it's easy enough to build it in an XP compatible way, so much so that with 1 minor adjustment the build works. My post was more of a "just so you know", that's it.


Well, thank you for your discovery ... but don't tell me. I only compile. I do not develop.

Yeah, that's the thing, it's not the source code per se, rather I'm trying to figure out why the compiler is calling a non existing kernel function while it really shouldn't given that you're targeting XP in a specific build...

TL;DR Multicoreware has nothing to do this with, rather the compiler or whoever made the autobuild compilation script :P


XP doesn't have explicit NUMA support and is missing a lot of other things helpful for maximizing performance.

True. In a multicore and multisocket environment, it would probably underperform by a margin: no NUMA, no AVX, no AVX2, no AVX512, also 32bit version of bit depth higher than 8bit don't have any intrinsics at all, so they would run in plain C. Anyway, I don't really think there are any businesses out there running XP/Server 2003 to encode stuff; I myself am on Server 2019 x64 with my farm at work, but I was thinking more about some home users, that's it. ;)

[ShortKatz]

Finally someone made a pull request to fix the compilation on macOS arm64.
https://mailman.videolan.org/pipermail/x265-devel/2023-April/013600.html
Hopefully this patch gets applied soon to fix the broken arm64 compilation on macOS.

[LigH]

Multicoreware has nothing to do this with, rather the compiler or whoever made the autobuild compilation script :P

In that case ... it might be possible that newer GCC versions (12) should get added a command line parameter which was not yet required in older versions (9/10)?

[benwaggoner]

Finally someone made a pull request to fix the compilation on macOS arm64.
https://mailman.videolan.org/pipermail/x265-devel/2023-April/013600.html
Hopefully this patch gets applied soon to fix the broken arm64 compilation on macOS.
It was checked in five days ago! Yay!

https://bitbucket.org/multicoreware/x265_git/commits/753305affb093ae15d5e4b333125267b16258c21

[ShortKatz]

It was checked in five days ago! Yay!

https://bitbucket.org/multicoreware/x265_git/commits/753305affb093ae15d5e4b333125267b16258c21

Yes, this are great news. :) Now I don't need the fix from HandBrake anymore to build on Mac.

[DMD]

Good morning
I have always wondered how to do to predict, the final file size based on preset Medium, Slower.... and CFR quality.
Even reading the specific article at https://slhck.info/video/2017/02/24/crf-guide.html

I have verified that in practice the data obtained is different from that predicted in the article.
I assume that the same value of CRF gives different results according to the speed preset set.
I ask if this assumption of mine can be an attempt at estimated prediction.
One could start the procedure and when we are about the 1% value , check in the temp folder the size of the produced file and make a proportion with the percentage of the remaining process.
Could this be correct?
Thank you

[rwill]

Could this be correct?


No. But you could randomly encode 50% of a sequence and then you are halfway there.

[katzenjoghurt]

@DMD

When quality reencoding a movie for smaller size I split the source into 7minute chunks using MKVToolnix. I then sort them by filesize and pick one in the middle where I think it's representative for the rest of the movie (day, night, lighting, grain).

I then encode that chunk with different crf values and compare the quality with StaxRip's built-in video comparison tool.

When changing only the crf value my rule of thumb for estimating the resulting file size is:
File size increases by 1.25^(crf_increase).
So if I rise the crf value by 3 it's estimated_new_size = old_crf_filesize*1.25*1.25*1.25.


May not totally answer your question but that's my approach to get close to the sweet spot and it works 85% of the time for me.

[DMD]

@katzenjoghurt
Thank you for the suggestion, I am also performing tests by processing only 20 % of the file and make an estimate of the total size.




A question I have always wondered why in video files with aspect ratio higher than 16:9, Crop is not used so that the pixel ratio (DAR) is higher than 1.78.
In theory, only the active part of the screen should be processed, also saving time in the compression procedure.
Analyzing with MediaInfo, commercial UHD blurays all have the 16:9 aspetc ratio.
Maybe my reasoning is not correct?
Thanks

[LigH]

The pixel aspect ratio = SAR (sample aspect ratio), means the anamorphic skew factor, is 1:1 in modern commercial video formats. The DAR (display aspect ratio) means the ratio between width and height of the whole image.

The efficiency of compressing temporally stable solid black borders covering complete macroblocks (or partitions in HEVC/H.265) is already quite high. Explicit crop areas with panning offsets may be unsupported by consumer players. Cropping 1088 to 1080 lines in AVC/H.264 video is common, though.

[DMD]

The pixel aspect ratio = SAR (sample aspect ratio), means the anamorphic skew factor, is 1:1 in modern commercial video formats. The DAR (display aspect ratio) means the ratio between width and height of the whole image.

The efficiency of compressing temporally stable solid black borders covering complete macroblocks (or partitions in HEVC/H.265) is already quite high. Explicit crop areas with panning offsets may be unsupported by consumer players. Cropping 1088 to 1080 lines in AVC/H.264 video is common, though.
So to have more player compatibility, it is better to leave the DAR and SAR settings = 1.78 by default.

[LigH]

No, I just explained the difference: DAR = 16:9, SAR = 1:1

And some additional specifications beyond MPEG-4 AVC / HEVC (like Blu-ray, Blu-ray UHD) demand such specific aspect ratios and resolutions.

[megapro17]

Hello. I need help with fully utilizing i7-13700k cpu with x265. Even with -x265-params aq-mode=3:pmode=1:pme=1 it constantly drops usage around a 50% for 1080p60 video with medium preset. What else I can try? I guess I have to increase slices?

x265 [info]: HEVC encoder version 3.5+95-38cf1c379
x265 [info]: build info [Windows][GCC 12.2.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [info]: Main profile, Level-4.1 (Main tier)
x265 [info]: Thread pool created using 24 threads
x265 [info]: Slices : 1
x265 [info]: frame threads / pool features : 4 / wpp(17 rows)+pmode+pme
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : hex / 57 / 2 / 3
x265 [info]: Keyframe min / max / scenecut / bias : 25 / 250 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 3 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 3 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : CRF-28.0 / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip mode=1 signhide tmvp
x265 [info]: tools: b-intra strong-intra-smoothing lslices=6 deblock sao

[benwaggoner]

Hello. I need help with fully utilizing i7-13700k cpu with x265. Even with -x265-params aq-mode=3:pmode=1:pme=1 it constantly drops usage around a 50% for 1080p60 video with medium preset. What else I can try? I guess I have to increase slices?

Are you using HyperThreading? If so, you're probably getting performance values assuming each logical core is a physical core. Your 50% may indicate that you're fully utilizing all your physical cores.

Or perhaps the Efficiency cores aren't getting much work as the Performance cores are so much more useful with x265.

8 physical threads for 1080p should saturate just fine without --pme and --pmode with 4 frame threads.

[guest]

Hello. I need help with fully utilizing i7-13700k cpu with x265.

Have a bit of a read thru this...

https://x265.readthedocs.io/en/master/cli.html

Won't hurt, might help ;)

[megapro17]

Are you using HyperThreading? If so, you're probably getting performance values assuming each logical core is a physical core. Your 50% may indicate that you're fully utilizing all your physical cores.


That isn't a case surely, because fans aren't blowing much, pc is almost silent. In other workloads I can feel a heat. And there's some periods when x265 is loading a cpu fully. I got video encoded on i5-12400 faster than on my cpu, that's ridiculous. Maybe there's something with effeciency cores, is there any information on this?

Have a bit of a read thru this...)

your sarcasm is inappropriate, I read it and enabled pmode and pme that helped, but not too much. I can't see any options besides --slices and --ctu that should help

[Boulder]

Those settings are quite light for modern CPUs so it's no wonder it doesn't work more.
Lowering CTU to 32 will increase multithreading, and it really won't hurt quality or compression. --rskip 2 will increase quality and also give more work to the CPU. Maybe try those, or just choose --preset slow for more things.

[benwaggoner]

That isn't a case surely, because fans aren't blowing much, pc is almost silent. In other workloads I can feel a heat. And there's some periods when x265 is loading a cpu fully. I got video encoded on i5-12400 faster than on my cpu, that's ridiculous. Maybe there's something with effeciency cores, is there any information on this?
Take a look at what cores are doing what in Resource Manager. That should be helpful in tracking down what's up.

Trying --preset slower and seeing if there is a difference would also be useful.

[jpsdr]

Hi.
If i understood properly some previous posts, there is code for AVX512, but it's not compiled by default, you have to use a CMake option for the code being compiled.
I searched in the CMakeLists but didn't find anything.
What is the CMake option to have the AVX512 code branch compiled in the build ?

[guest]

Hi.
If i understood properly some previous posts, there is code for AVX512, but it's not compiled by default, you have to use a CMake option for the code being compiled.
I searched in the CMakeLists but didn't find anything.
What is the CMake option to have the AVX512 code branch compiled in the build ?

You can't use AVX512 on 12th & 13th Gen Intel CPU's.

[guest]

your sarcasm is inappropriate, I read it and enabled pmode and pme that helped, but not too much. I can't see any options besides --slices and --ctu that should help

WHAT sarcasm ???

You read, it helped :confused:

[excellentswordfight]

That isn't a case surely, because fans aren't blowing much, pc is almost silent. In other workloads I can feel a heat. And there's some periods when x265 is loading a cpu fully. I got video encoded on i5-12400 faster than on my cpu, that's ridiculous. Maybe there's something with effeciency cores, is there any information on this?
With default settings, the utilization you are getting are normal, @ 1080p x265 doesnt saturate 24 threads. From my experience it scales to about 16 threads at the lower presets at least. You will probably see close to full utilization with --ctu 32.

If 12400 is faster, it does indeed sound like it could be a issue with the scheduler, what windows version are you using? I havnt seen an issue with the effeciency cores on my 12700k on W11, but I also always encode at close to 100% load. You can confirm if they are the issue or not by just setting cpu affinity for x265 in task manager and removing the efficiency cores and see if that gives you a speed increase for cases were x265 cannot fully saturate the cpu.

[LigH]

New upload: x265 3.5+100-3dae0c3cc (https://www.mediafire.com/file/1u7soqag7gjr1yz/x265_3.5+100-3dae0c3cc.7z/file)

[Windows][GCC 12.2.0][32/32XP/64 bit] 8bit+10bit+12bit

Reflect Zonefile Parameters inside Lookahead.

[Barough]

x265 v3.5+101
[Windows][GCC 12.2.0][32/64 bit] 8bit+10bit+12bit
https://bitbucket.org/multicoreware/x265_git/commits/branch/master

DL :
https://www.mediafire.com/file/rxax09nfkrfk8pu/

[john33]

x265-r3.50.0.101.02ef6de with Patman mods - AVX2 optimised
[Windows][64 bit] 8bit+10bit+12bit

GCC.13.1.0: https://www.rarewares.org/files/x265-r3.50.0.101.02ef6de.GCC13.1.0.AVX2.zip

Intel.19.2: https://www.rarewares.org/files/x265-r3.50.0.101.02ef6de.Intel.19.2.AVX2.zip

[jpsdr]

Hello.

Forgot to tell, made a new build of my custom mod, check my Github.

[LeXXuz]

Over 100 commits and still 3.5. Are they afraid to push it to 3.6 or what is the reason? :confused:

[Barough]

x265 v3.5+101
Built on May 05, 2023, GCC 13.1.0

DL :
https://www.mediafire.com/file/mj2zqkwdpnv8esd/

[jpsdr]

New +103 build, check my github.

[madey83]

New +103 build, check my github.


Thanks for that, but every new release is publish I struggle to find release notes.

Could please attach into GitHub release or here?

[jpsdr]

I personnaly will not bother to copy release notes when they can be checked on the github, just see the text of the commits.

[megapro17]

If 12400 is faster, it does indeed sound like it could be a issue with the scheduler, what windows version are you using?
I'm using Windows 11 with all updates 22621. You was right, apparently, windows restricts background application to only efficency cores! When I pin window at top, my CPU is loaded at 100% all times. This is incredibly stupid, I couldn't even assume that! Problem is known for more than a year and still not fixed. all we had to do is buy the damn 7900x.
Thanks for poiniting for the most dumb problem ever. And none of the solutions are not working, only keeping window at top. Intel thread director, more like intel garbage director

https://www.reddit.com/r/XMG_gg/comments/vlqn6d/psa_rendering_tasks_are_moved_to_ecores_when/
https://www.reddit.com/r/intel/comments/tgg9hz/handbrake_windows_11_efficiency_cores_only/

[Atak_Snajpera]

Have you tried changing power plan from balanced to high performance?

[benwaggoner]

Thanks for that, but every new release is publish I struggle to find release notes.

Could please attach into GitHub release or here?
MCW only provides release notes for official releases, the most recent being 3.5.

I just read the commits for the current branch.

[megapro17]

Have you tried changing power plan from balanced to high performance?

yeah i even tried Ultimate Performance plan, it doesn't change anything

[benwaggoner]

yeah i even tried Ultimate Performance plan, it doesn't change anything
Are the different core types flagged as different NUMA nodes? If so, --pools could be used to pin specifically to the performance cores. I think that may work on AMD, as the big processors have a lot of nodes. Not Intel AFAIK.

[Atak_Snajpera]

Are the different core types flagged as different NUMA nodes? If so, --pools could be used to pin specifically to the performance cores. I think that may work on AMD, as the big processors have a lot of nodes. Not Intel AFAIK.

Only on Windows 7 P-Cores and E-Cores are separated by different NUMA nodes. Basically win7 treats that cpu as "dual-socket". On Win10/11 all cores sit on the same NUMA node.

[guest]

I am trying to get x265 to process a DV clip, and retain the appropriate metadata, or whatever it is.

If I encode a DV clip with "standard" commands I get this output, I've just found out that this is a "Hybrid" and it plays as HDR10:-

This is MediaInfo info.

Dolby Vision, Version 1.0, dvhe.08.06, BL+RPU, HDR10 compatible / SMPTE ST 2086, HDR10 compatible


But I downloaded an LG DV test clip, and it's x265 info is, and it plays as DV :-

Dolby Vision, Version 1.0, dvhe.04.06, BL+EL+RPU

How do I remove the HDR10 compatible / SMPTE ST 2086, HDR10 compatible ??

[rwill]

This is not a Hybrid, this is a Profile 8 Dolby Vision which is HDR10 backwards compatible.

If you want to drop HDR10 compatibility you can try to not insert the mastering display SEI and content light level SEI, then just set the the VUI video information of primaries, transfer and matrix to unknown. This way a playback device will not know how to display the Baselayer correctly at all.

I have my doubts that the resulting DV file will be correct and be conforming to any valid DV Profile though.

[guest]

This is not a Hybrid, this is a Profile 8 Dolby Vision which is HDR10 backwards compatible.

If you want to drop HDR10 compatibility you can try to not insert the mastering display SEI and content light level SEI, then just set the the VUI video information of primaries, transfer and matrix to unknown. This way a playback device will not know how to display the Baselayer correctly at all.

I have my doubts that the resulting DV file will be correct and be conforming to any valid DV Profile though.

Thanks for the info, but that's about as clear as thick mud, to me :(

[rwill]

Thanks for the info, but that's about as clear as thick mud, to me :(

Well then, what is your x265 command line for encoding?

[guest]

Well then, what is your x265 command line for encoding?

I think it would be easier for me to just send you over to have a look at what I've been posting on the RipBot thread with quite a few of the command lines that I've been trying...

https://forum.doom9.org/showthread.php?p=1987445#post1987445

[rwill]

Hmm, what about "--no-hdr10 --videoformat unknown --range limited --colorprim unknown --transfer unknown --colormatrix unknown" and removing "--master-display" and "--max-cll" ?

[guest]

Hmm, what about "--no-hdr10 --videoformat unknown --range limited --colorprim unknown --transfer unknown --colormatrix unknown" and removing "--master-display" and "--max-cll" ?

So from what I understand, what you're saying here is I need to add that to the command line and remove those others ?

OK, I have just checked what these commands to @ x265 read the docs, so I will very interested in seeing what it does, but it won't be until tomorrow now (my time).

:thanks: I will report back later :)

UPDATE:-

Hi, I tried all those commands, and it still doesn't do what I think it should.

How do I remove the "--master-display" and "--max-cll" (this is info from the clip I'm testing with)

--master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)" --max-cll "738,287"

[rwill]

So from what I understand, what you're saying here is I need to add that to the command line and remove those others ?

Yes.

You also might want to pass off your DV RPU as "--dolby-vision-profile 5" to x265 instead of 8.1, as Profile 8.1 is by definition HDR10 backwards compatible. Profile 5 is not.

[tuanden0]

I have 8 cores CPU (Ryzen 7 5800X) but when I encode, it never used full 100% CPU. So I tried to run 2 x265 instances at same time, but I am confuse how to use this options: --pools <string>, --numa-pools <string>

Can someone enlighten me.

Should I use --pools "+" or something.

Thanks

[Selur]

https://x265.readthedocs.io/en/master/threading.html?thread-pools
https://x265.readthedocs.io/en/master/cli.html?highlight=--pools#cmdoption-pools

Instead of messing with pools, try using "--pmode --pme" at least with HD and higher resolution content that helps here,...

[rwill]

I have 8 cores CPU (Ryzen 7 5800X) but when I encode, it never used full 100% CPU. So I tried to run 2 x265 instances at same time, but I am confuse how to use this options: --pools <string>, --numa-pools <string>

Can someone enlighten me.

Should I use --pools "+" or something.

Thanks

Well just don't use --pools or --numa-pools. Just start your two instances and let the Operating System scheduler figure out how to distribute the threads.

And don't use --pmode or --pme, they will increase CPU usage more than they will speed up things.

[tuanden0]

https://x265.readthedocs.io/en/master/threading.html?thread-pools
https://x265.readthedocs.io/en/master/cli.html?highlight=--pools#cmdoption-pools

Instead of messing with pools, try using "--pmode --pme" at least with HD and higher resolution content that helps here,...

Well just don't use --pools or --numa-pools. Just start your two instances and let the Operating System scheduler figure out how to distribute the threads.

And don't use --pmode or --pme, they will increase CPU usage more than they will speed up things.

Thank you for help :goodpost:

I will not use --pools and let the OS do it.

[benwaggoner]

https://x265.readthedocs.io/en/master/threading.html?thread-pools
https://x265.readthedocs.io/en/master/cli.html?highlight=--pools#cmdoption-pools

Instead of messing with pools, try using "--pmode --pme" at least with HD and higher resolution content that helps here,...
--pmode could well help, but unless it <480p, --pme would most likely slow things down.

[Atak_Snajpera]

Solution is Simple. --ctu 16 for SD , --ctu 32 for HD/FHD and --ctu 64 for UHD.

[Boulder]

I tried encoding one track from a Dolby UHD test disc to see if I can make Dolby Vision work on my TV and to test if it falls back to HDR10 when played in Kodi. I extracted the RPU using the demuxing script here in the forum and fed it to the encoder.

With
c:\x265\x265.exe --input inputfile.avs --input-depth 16 --dither --sar 1:1 --profile main10 --rc-lookahead 40 --min-keyint 5 --keyint 480 --splitrd-skip --colorprim 9 --transfer 16 --colormatrix 9 --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50)" --max-cll "1000,355" --frames 3576 --chromaloc 2 --hdr10 --hdr10-opt --repeat-headers --dolby-vision-profile 8.1 --dolby-vision-rpu "rpu.bin" --preset slower --crf 16 --output "dolby_test.hevc"I got these errors:
Dolby Vision requires VBV settings to enable HRD.
Dolby Vision RPU count is greater than frame count in x265.

Well, I checked and the RPU is ok.

Adding --vbv-bufsize 999999999 --vbv-maxrate 999999999, I got the encode going so the complaint about the RPU file is a bug. The encode ran at Level 8.5 :scared:

My question is: is VBV really necessary? Trying to add --no-hrd didn't do any good. I was thinking of patching the thing to skip that check if it's not needed. I know that I don't need it for my use and have never set it.

I could encode normally and just inject the RPU afterwards, but it would be easier and less diskspace consuming to do everything in one go.

[Rousseau]

What happened to the "Consecutive B-Frames" stats in the encode log? I just upgraded from Patman's 37 build to JPSDR's 103 build and it's gone. I tried john33's and djatom's binaries with the same result.

[guest]

I tried encoding one track from a Dolby UHD test disc to see if I can make Dolby Vision work on my TV and to test if it falls back to HDR10 when played in Kodi. I extracted the RPU using the demuxing script here in the forum and fed it to the encoder.

With
c:\x265\x265.exe --input inputfile.avs --input-depth 16 --dither --sar 1:1 --profile main10 --rc-lookahead 40 --min-keyint 5 --keyint 480 --splitrd-skip --colorprim 9 --transfer 16 --colormatrix 9
--master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50)" --max-cll "1000,355" --frames 3576 --chromaloc 2 --hdr10 --hdr10-opt --repeat-headers --dolby-vision-profile 8.1
--dolby-vision-rpu "rpu.bin" --preset slower --crf 16 --output "dolby_test.hevc"I got these errors:
Dolby Vision requires VBV settings to enable HRD.
Dolby Vision RPU count is greater than frame count in x265.

Well, I checked and the RPU is ok.

Adding --vbv-bufsize 999999999 --vbv-maxrate 999999999, I got the encode going so the complaint about the RPU file is a bug. The encode ran at Level 8.5 :scared:

My question is: is VBV really necessary? Trying to add --no-hrd didn't do any good. I was thinking of patching the thing to skip that check if it's not needed. I know that I don't need it for my use and have never set it.

I could encode normally and just inject the RPU afterwards, but it would be easier and less diskspace consuming to do everything in one go.

Don't you have to provide the full path to the rpu.bin ???

[Boulder]

What happened to the "Consecutive B-Frames" stats in the encode log? I just upgraded from Patman's 37 build to JPSDR's 103 build and it's gone. I tried john33's and djatom's binaries with the same result.

It was lost at a recent commit for some reason.. IIRC quietvoid created a patch to get it back though.

Don't you have to provide the full path to the rpu.binNot needed if the file is in the folder where I launch the command line in. The same for "inputfile.avs".

[quietvoid]

What happened to the "Consecutive B-Frames" stats in the encode log? I just upgraded from Patman's 37 build to JPSDR's 103 build and it's gone. I tried john33's and djatom's binaries with the same result.

As said above, it was removed.
The code can be added back with this commit: https://github.com/quietvoid/x265/commit/43e460330af001365c8508d430a35be4377e390e

[benwaggoner]

My question is: is VBV really necessary? Trying to add --no-hrd didn't do any good. I was thinking of patching the thing to skip that check if it's not needed. I know that I don't need it for my use and have never set it.
It seems like VBV is important in your case ;)!

I think it's always a good idea to set VBV to ensure compatibility with a given decoder. Just use --level-idc 5.1 and it will auto set reasonable values broadly compatible with UHD BD, Smart TVs, HW GPU decode, etc. 5.1 has a 40 Mbps peak, which is fine for 2160p24 content.

[jpsdr]

I checked the commit to put back "Consecutive B-Frames" in my moded version, unfortunately, there is big changes, and i don't realy know where and how put the histogram increment count...

[madey83]

As said above, it was removed.
The code can be added back with this commit: https://github.com/quietvoid/x265/commit/43e460330af001365c8508d430a35be4377e390e

Have you checked new updated Patman mode?

https://github.com/Patman86/x265-Mod-by-Patman/releases

[LigH]

New uploads: x265 3.5+103-8f18e3ad3 (https://www.mediafire.com/file/r7g2b57iyqd6l0c/x265_3.5+103-8f18e3ad3.7z/file) / x265 3.5+104-47c18bf12 (https://www.mediafire.com/file/l4lmmx6rwqxskpj/x265_3.5+104-47c18bf12.7z/file) (might be identical, patch counting mistake or different branches?)

[Windows][GCC 13.1.0][32/32XP/64 bit] 8bit+10bit+12bit

[Boulder]

I have a simple question regarding frametypes and chapter marks when using a qpfile. Does a chapter mark need an I-frame or is an i-frame enough? I'm using open GOP so setting 'K' in the qpfile would place i-frames in the encode.

[LigH]

An IDR frame (Intra frame with Decoder Reset) is preferable. When you jump to an intermediate I frame which is not IDR (e.g. to encode a single frame very different from the surrounding video), you will get decoding artifacts from predictions skipping this I frame missing a reference.

[LeXXuz]

Oh darn. Thanks for mentioning that. Now I know why my players always jump a little before or after the specified chapter point. MeGUI sets 'K' as default for keyframes when creating qpfiles. I guess I'll better change those manually to 'I' from now on. :mad:

[benwaggoner]

Oh darn. Thanks for mentioning that. Now I know why my players always jump a little before or after the specified chapter point. MeGUI sets 'K' as default for keyframes when creating qpfiles. I guess I'll better change those manually to 'I' from now on. :mad:
Yeah, only IDR frames are guaranteed to be independently decodable.

[jpsdr]

Hello.
I've made a new build, and also include some new Patman updates.
As usual, check my github.

[sainome]

Solution is Simple. --ctu 16 for SD , --ctu 32 for HD/FHD and --ctu 64 for UHD.

Question: If I specify --ctu 16, what should I specify for merange?

[benwaggoner]

Question: If I specify --ctu 16, what should I specify for merange?
They are orthogonal values, without much connection between optimum values. I'd stick with the one your --preset gives you unless you want to do a bunch of experimentation.

With --ctu 16 I suppose you could try --merange 41, which is 16 below the typical default of 57. It'd be a little faster, and depending on content might not raise bitrate or harm quality materially.

[sainome]

Are you saying that there is no ideal optimal value?

--merange <integer>
Motion search range. Default 57

The default is derived from the default CTU size (64) minus the luma interpolation half-length (4) minus maximum subpel distance (2) minus one extra pixel just in case the hex search method is used. If the search range were any larger than this, another CTU row of latency would be required for reference frames.

x265 docment says it like this, so if you specify "--ctu32", specify 25

[N'Cha]

Hello,
What is the impact (quality of the output, compression efficiency, easiness to decode the encoded video or the 3 maybe?) of using --limit-refs 3 vs --limit-refs 1 and --subme 5 vs --subme 7 ?
I've noticed that using limit-refs 1 makes my encoding take 20-25 minutes longer for a 24 minute anime episode (1h20 vs roughly 56 minutes with limi-tefs 3). Same with --subme 7, it has a lesser impact but still like 3-4 minutes on the encoding and not sure it's worth it (if it gives like 10% more quality/details vs --subme 5 it is worth it)

[LeXXuz]

Are you saying that there is no ideal optimal value?



x265 docment says it like this, so if you specify "--ctu32", specify 25

Interesting. Thought 'default' merange gets adjusted automatically by the encoder to a lower value when ctu32 or ctu16 is specified. :confused:

[benwaggoner]

Are you saying that there is no ideal optimal value?

x265 docment says it like this, so if you specify "--ctu32", specify 25
Limiting it to 25 would improve parallelization, but absolutely could reduce compression efficiency. Lowering CTU makes more WPP parallelism possible, but doesn't mitigate the potential quality hit from a too-small search range.

[sainome]

Sorry. I'm sorry for the afterthought of the information, but the source is SD size (720x480).
It seems that --cti 16 gives better results than --ctu 64/32 for SD size.

I have tried --merange values of 57/25/16, but I can't decide between them, so I wanted to ask if there is a mathematically/theoretically correct setting.

[microchip8]

Sorry. I'm sorry for the afterthought of the information, but the source is SD size (720x480).
It seems that --cti 16 gives better results than --ctu 64/32 for SD size.

I have tried --merange values of 57/25/16, but I can't decide between them, so I wanted to ask if there is a mathematically/theoretically correct setting.

If it's SD, then lower CTU is definitely better and recommended. As for me-range, I doubt you'll see the difference.

[benwaggoner]

Are you saying that there is no ideal optimal value?
Parameters for which there is an ideal optimal value just have those values hard-coded ;). The parameters you can set are ones that you may need different values for sometimes.

[benwaggoner]

Sorry. I'm sorry for the afterthought of the information, but the source is SD size (720x480).
It seems that --cti 16 gives better results than --ctu 64/32 for SD size.
Better in what sense?

I have tried --merange values of 57/25/16, but I can't decide between them, so I wanted to ask if there is a mathematically/theoretically correct setting.
Not really. If it is too small, it'll miss some motion matches that would be more efficient. If it's too big, it'll waste encoding time without making quality or bitrate significantly better.

57 is really a fine default unless there's a big need for more parallelism.

[LeXXuz]

I could use some pro advice about setting aq-strength. Again. :o
After reading some posts and comments, I can't decide if I should use 1.0 or 1.5 as a 'default' setting for my encodes.

I use jp's patman mod with a custom very-slow preset @ CRF17 for 1080p sources. My sources are mostly quite clean as I denoise them with a complex mv-based denoiser script before encoding.

my settings look like these:

--preset veryslow --crf 17.0 --qpfile "D:\WORK\test\chp.qpf" --repeat-headers --input-depth 16 --output-depth 10 --dither
--profile main10 --high-tier --level-idc 4.1 --vbv-bufsize 50000 --vbv-maxrate 50000 --ctu 32 --limit-refs 1 --rd 6 --aq-mode 5 --aq-auto 10
--aq-strength 1.5 --psy-rd 2 --psy-rdoq 4 --rskip 2 --rskip-edge-threshold 2 --no-sao --no-strong-intra-smoothing --deblock -1:-1
--colorprim bt709 --transfer bt709 --colormatrix bt709
(I know some settings are obsolete as they are already part of the preset. But I always like to set them manually as mental reminder what is used with said preset.)

With aq-strength 1.5 file sizes increase, depending on source. So more bits are hopefully spend where it matters.

But is there a downside to a higher aq-strength I should keep in mind (besides bigger file sizes)?

[benwaggoner]

I could use some pro advice about setting aq-strength. Again. :o
After reading some posts and comments, I can't decide if I should use 1.0 or 1.5 as a 'default' setting for my encodes.

I use jp's patman mod with a custom very-slow preset @ CRF17 for 1080p sources. My sources are mostly quite clean as I denoise them with a complex mv-based denoiser script before encoding.

my settings look like these:

--preset veryslow --crf 17.0 --qpfile "D:\WORK\test\chp.qpf" --repeat-headers --input-depth 16 --output-depth 10 --dither
--profile main10 --high-tier --level-idc 4.1 --vbv-bufsize 50000 --vbv-maxrate 50000 --ctu 32 --limit-refs 1 --rd 6 --aq-mode 5 --aq-auto 10
--aq-strength 1.5 --psy-rd 2 --psy-rdoq 4 --rskip 2 --rskip-edge-threshold 2 --no-sao --no-strong-intra-smoothing --deblock -1:-1
--colorprim bt709 --transfer bt709 --colormatrix bt709
(I know some settings are obsolete as they are already part of the preset. But I always like to set them manually as mental reminder what is used with said preset.)

With aq-strength 1.5 file sizes increase, depending on source. So more bits are hopefully spend where it matters.

But is there a downside to a higher aq-strength I should keep in mind (besides bigger file sizes)?
I've not used --aq-mode 5 myself, as it isn't in x265 mainline. The optimal --aq-strength can vary a lot depending on algorithm. For example --aq-mode 4 often has an optimal strength .1-.2 lower than for 1 or 2. That may be true for this mode 5 as well.

In a general sense, 1.0 is a rational default. I'd use a bit lower if I know the content is cel animation, a little bit higher if I knew it was professionally produced film or video. 1.5 isn't crazy high or anything. With high tier crf=17, I would expect the impact of AQ could be more in file size than perceptual quality. There aren't any good public metrics for AQ modes; PSNR and SSIM often show lower scores for better subjective quality, and VMAF is pretty blind to even pretty visible AQ impacts.

Finding your optimal value is really down to trying a few variants and picking what you like best with your eyeballs. Since you're using 1.5 already, trying 1.0 and 2.0 could give you a sense of the impact of different values in subjective quality and file size.

[LeXXuz]

Thanks Ben for your valued input. :)

[benwaggoner]

So, now that Apple is resuscitating stereoscopic 3D video, has anyone done any work getting MV-HEVC working with x265? It's basically left eye as a normal 2D stream and a right eye encoded using the left eye as reference, so not that theoretically hard to extend.

https://developer.apple.com/videos/play/wwdc2023/10071/?time=320

[Rainforrest]

Hi,
Could smbd please share an example of using ZONES (e.g. to assign different CRF) for x265 encoding with FFMPEG? (as far as I know it may be via x265-params)

If no FFMPEG example, same example for direct command line x265 encoder would be helpful also, hopefully syntax isn't much different.
Thank you.

[benwaggoner]

Hi,
Could smbd please share an example of using ZONES (e.g. to assign different CRF) for x265 encoding with FFMPEG? (as far as I know it may be via x265-params)

If no FFMPEG example, same example for direct command line x265 encoder would be helpful also, hopefully syntax isn't much different.
I've not used --zones myself that I can recall. I've used --qpfile for similar purposes.
The documentation lists --zonefile as a CLI-only feature; I don't know that it is implemented in ffmpeg.

Also, CRF isn't one of the parameters that can be specified. Only QP or a bitrate multiplier are listed.

From: https://x265.readthedocs.io/en/3.4/cli.html#quality-rate-control-and-rate-distortion-options
--zones <zone0>/<zone1>/...
Tweak the bitrate of regions of the video. Each zone takes the form:

<start frame>,<end frame>,<option> where <option> is either q=<integer> (force QP) or b=<float> (bitrate multiplier).

If zones overlap, whichever comes later in the list takes precedence. Default none

--zonefile <filename>
Specify a text file which contains the boundaries of the zones where each of zones are configurable. The format of each line is:

<frame number> <options to be configured>

The frame number indicates the beginning of a zone. The options following this is applied until another zone begins. The reconfigurable options can be spcified as –<feature name> <feature value>

CLI ONLY

[maroders2]

Hello,
Please tell me the optimal settings for encoding movies (only 1080p BDRemux to h.265 10bit). I want to get good quality with a normal encoding speed. Avg QP should not be higher than 22!
I currently use the following settings:
--crf 20 --preset slower --output-depth 10 --profile main10 --level-idc 4.1 --rd 4 --dynamic-rd 2 --limit-refs 3 --ctu 32 --limit-tu 1 --refine-mv 1 --no-b-intra --aq-mode 2 --vbv-bufsize 50000 --vbv-maxrate 50000 --no-cutree --subme 5 --bframes 16 --colorprim bt709 --colormatrix bt709 --transfer bt709 --range limited --sar 1:1 --deblock -3:-3 --no-sao --no-strong-intra-smoothing
Are these good settings or change something?

What settings can boost speed encoding with minimal quality loss?

Thanks!

[RanmaCanada]

Hello,
Please tell me the optimal settings for encoding movies (only 1080p BDRemux to h.265 10bit). I want to get good quality with a normal encoding speed. Avg QP should not be higher than 22!
I currently use the following settings:

Are these good settings or change something?

What settings can boost speed encoding with minimal quality loss?

Thanks!

At preset slower, the only way to increase speed is to buy new hardware. Settings would maybe get you a fraction of a frame increase in speed. Real speed increases can only be done by sheer horsepower.

What is your current processor?

[benwaggoner]

Hello,
Please tell me the optimal settings for encoding movies (only 1080p BDRemux to h.265 10bit). I want to get good quality with a normal encoding speed. Avg QP should not be higher than 22!
I currently use the following settings:

Are these good settings or change something?

What settings can boost speed encoding with minimal quality loss?
You're setting a lot of things you probably don't want to be setting. --bframes 16 will slow things down a lot for little gain. --deblock -3:-3 is going to raise bitrate and thus lower speed for little or no benefit. If the source is 8-bit, encoding at 10-bit gives a speed hit without much quality gain (unless you're doing some >8-bit preprocessing on the source). 1080p24 isn't going to hit 50 Mbps.

I'd start with just --preset slower --level-idc 4.1 --ctu 32 --crf 20 --selective-sao 2 and see how it goes quality and speed wise. If it's way too slow, try --preset slow. If it doesn't look good enough, try --crf 18. It looks like you're way overrunning for detail, which you shouldn't start with. That's something to do only if there's visible detail loss during playback. Don't sweat how a zoomed in screen shot looks unless you're going to be watching the video as a zoomed-in screen shots.

I don't know what a "normal" encoding speed is to you, but --preset slower with the settings I give above approaches real-time on a recent desktop processor with 12+ cores. If you are doing preprocessing along with encoding, check that preprocessing isn't using up lots of your CPU, or is serialized in a way that gates x265 performance. A good share of "why is my encoding so slow" problems have little to do with the encoder.

[benwaggoner]

At preset slower, the only way to increase speed is to buy new hardware. Settings would maybe get you a fraction of a frame increase in speed. Real speed increases can only be done by sheer horsepower.
Oh, going from --preset slower to --preset fast will speed up encoding by a pretty good multiple, if encoding speed is the limiting factor. I wouldn't do it myself other than for scratch quality or when I didn't care about bitrate, but there are absolutely ways to get huge speed increases through settings. Just not free ways in terms of quality or bitrate.

Edit: --selective-sao 2 is effectively free. And --ctu 32 for 1080p and below is a nice boost if you have the cores to take advantage of the extra WPP row of parallelism.

[maroders2]

At preset slower, the only way to increase speed is to buy new hardware. Settings would maybe get you a fraction of a frame increase in speed. Real speed increases can only be done by sheer horsepower.

What is your current processor?

I have a cheap r5 3600 and i will change it in the future.
I want to have a good quality of the picture, which would not lose much x264. The speed is in second place, but I would like to have at least 2 fps so as not to encode 20 hours for one movie


--bframes 16

I read in the documentation that in the case of -bframes 16, the encoder itself chooses the one it needs. I also found other settings here on the forum, although I admit that I don't quite understand how to use them.
I heard the advice to start with a slow preset many times, I started with this, but then I started randomly adding other settings:)

[maroders2]

What do you think about the QxR team's BDRips?
I know that BDRips are not the best of quality, but they are ready for use. My PC doesn't meet my needs right now, so I might start building a home movie collection of their BDRips.

[RanmaCanada]

What do you think about the QxR team's BDRips?
I know that BDRips are not the best of quality, but they are ready for use. My PC doesn't meet my needs right now, so I might start building a home movie collection of their BDRips.

Discussion of piracy is not allowed here.

[benwaggoner]

I have a cheap r5 3600 and i will change it in the future.
I want to have a good quality of the picture, which would not lose much x264. The speed is in second place, but I would like to have at least 2 fps so as not to encode 20 hours for one movie
For 1080p, pretty much any modern enthusiast chip can do >>2 fps. My Ryzen 7900X3D was doing 4.6 fps with high quality no frame-threading 4K encoding last night. It probably could do >12 with stock --preset slower at 1080p.

I read in the documentation that in the case of -bframes 16, the encoder itself chooses the one it needs.
Correct, but the time taken to determine the right B-frame pattern increases exponentially with the maximum number. 8 is really all you need with typical film/video content, and there's many better speed/quality options to do before raising b-frames makes. sense.

I also found other settings here on the forum, although I admit that I don't quite understand how to use them.
I heard the advice to start with a slow preset many times, I started with this, but then I started randomly adding other settings:)
Yeah, lots of people have lots of opinions about the right settings, for. different kinds of content with different encoding goals. The defaults are a pretty reasonable middle ground to start with unless there are some specifics you are tuning for.

Admittedly, my default command line for 4K HDR encoding is... 995 characters without file paths. But that's the result of eight years of evolution with the backing of a team that leverages lots of data analysis over hundreds of different variants to narrow down the right ones for a specific class of content for a specific use case.

[vadlerg]

Setting --level-idc is sometimes a useful option. For example, it sets the VBV parameters according to the idc level selected, so it is not necessary to specify the VBV parameters in the command line separately.
If I set a specific idc level and use the --dynamic-rd command line option, dynamic-rd is complaining that VBV is not set so it will disable the use of the option.
Dynamic-rd not recognizing that VBV is set already was fixed some times ago (https://forum.doom9.org/showthread.php?p=1982450#post1982450) if I remember correctly.

Would it be possible to upstream the fix again?

An another question: --aq-auto is not implemented in the latest builds anymore only in @jpsdr (https://forum.doom9.org/member.php?u=19714) builds available as far as I see. Is there any particular reason for the -aq-auto vanishing?

[Boulder]

An another question: --aq-auto is not implemented in the latest builds anymore only in @jpsdr (https://forum.doom9.org/member.php?u=19714) builds available as far as I see. Is there any particular reason for the -aq-auto vanishing?

--aq-auto has never been in the mainline, it only exists in mods.

[benwaggoner]

We've made a number of substantial improvements to x265 in the past few weeks. x265 documentation can be found here... http://x265.readthedocs.org/en/default/

Tom
MulticoreWare
Wow, I just realized we've hit the 10th anniversary of the HEVC forum!

[vadlerg]

Wow, I just realized we've hit the 10th anniversary of the HEVC forum!

Happy 10th anniversary :)
I think default link is to master https://x265.readthedocs.io/en/master/.

[badshah]

Hi friends,
I'm trying to encode a home video which is a few hours long. while it takes only 1 hour to encode to x264, it's showing 24 hours+ as ETA for encoding to x265.
I'm using MeGUI. both x264 and x265 settings are ""very Slow" High/Main @ 4.1. Resolution 1080p. rest all is default.
I tried staxrip and handbrake as well and both are showing similar ETA.
kindly suggest how do I speed up x265 encoding.

Thank you.

[FranceBB]

kindly suggest how do I speed up x265 encoding.

That's a bit too vague as there's no magical setting to speed things up.
Please note that x265 is much more demanding than x264 as they're two very different encoders that output two very different codecs in the end.
The first thing we need to know, though, is if your CPU is already at 100% usage on all cores and what CPU you have.
If it's not, then we might have some settings to speed things up, but if it is, then I'm afraid you're gonna have to sacrifice compression performances in favor of speed.

[Boulder]

In x265, preset veryslow is much more veryslower than in x264. Preset slower is just as good, and even then there are tweaks like --no-amp --limit-refs 3 to speed things up.

[filler56789]

Hi friends,
I'm trying to encode a home video which is a few hours long. while it takes only 1 hour to encode to x264, it's showing 24 hours+ as ETA for encoding to x265.
I'm using MeGUI. both x264 and x265 settings are ""very Slow" High/Main @ 4.1. Resolution 1080p. rest all is default.
I tried staxrip and handbrake as well and both are showing similar ETA.
kindly suggest how do I speed up x265 encoding.

Thank you.

IF you have two or more desktops or laptops, you could give a try to the distributed-encoding feature of RipBot264.😐

[badshah]

Thank you Boulder & filler.
I dont have multiple pcs. someone told me about tdarr, but didnt tell me much, only said it takes 5-15 mins for him to encode a 3hr duration, 50gb file. is that a cloud based encoder?
are there any cloud based encoders which can work faster like this?

[benwaggoner]

Thank you Boulder & filler.
I dont have multiple pcs. someone told me about tdarr, but didnt tell me much, only said it takes 5-15 mins for him to encode a 3hr duration, 50gb file. is that a cloud based encoder?
are there any cloud based encoders which can work faster like this?
Any segmenting encoder can do something like that given access to enough cores and instances. And that's easiest and most affordable to do in the cloud. 15 min on 20 instances has about the same cost as 5 hours on one instance for a service handling multiple jobs like that simultaneously.

[excellentswordfight]

Thank you Boulder & filler.
I dont have multiple pcs. someone told me about tdarr, but didnt tell me much, only said it takes 5-15 mins for him to encode a 3hr duration, 50gb file. is that a cloud based encoder?
are there any cloud based encoders which can work faster like this?
Im not familiar with tdarr, but from the looks of it is that it can be hosted in the cloud, although it seems to be more targeted for "private cloud", i.e you run it on your own hardware.

From the looks of it, it also doesnt do chunk/distributed encoding, so its more doing multiple files/steps across multiple servers/computers. The speed most likely comes from it supporting GPU-encoding, that can be extremely fast compared to x265.

As for your speed question, start by using faster presets, you wont sacrifice that much of going up to 'slow'. For preset 'medium' and above you will start to see some noticeable drop from the slower presets so if thats still to slow you might wanna look at GPU-encoders instead.

In general, if you are encoding UHD with HEVC, and want the best possible quality (i.e. x265 with at least preset 'slow') it will take hours to encode titles, you wont get much more than about 5fps with current consumer hardware, there is no way around that without chunk-encoding in the cloud (or I guess you can also do it across multiple high-end-pcs at home with e.g. rippbot). If you want at least real-time encoding speeds, GPU-encoders (Intel QuickSync or Nvidia nvenc) is the way to go.

[RanmaCanada]

Hi friends,
I'm trying to encode a home video which is a few hours long. while it takes only 1 hour to encode to x264, it's showing 24 hours+ as ETA for encoding to x265.
I'm using MeGUI. both x264 and x265 settings are ""very Slow" High/Main @ 4.1. Resolution 1080p. rest all is default.
I tried staxrip and handbrake as well and both are showing similar ETA.
kindly suggest how do I speed up x265 encoding.

Thank you.

What is your system specs please? x265 very slow as Boulder said is extremely slow, even on my 5800x I average 2fps so a 2 hour movie would take a day (24 hours).

[FranceBB]

Any segmenting encoder can do something like that given access to enough cores and instances. And that's easiest and most affordable to do in the cloud. 15 min on 20 instances has about the same cost as 5 hours on one instance for a service handling multiple jobs like that simultaneously.

Yes, but it becomes really tricky when you have to output a precise, fixed closed GOP every time.
I've been doing some experiments with Avisynth and x26x in the cloud recently, I think it's time to post some results and I'll open a thread about it, but I always worked on a single instance which led to either some high costs and short processing time or some very reduced costs but very long processing times.

(more on that next week)

[LigH]

Current proposed patches shall fix a quality issue related to SBRC (when committed)...

[Selur]

Is interlaced encoding broken?
using:

x265 [info]: HEVC encoder version 3.5+111-c40c3d799
x265 [info]: build info [Windows][GCC 13.2.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2

and
ffmpeg -y -loglevel fatal -noautorotate -nostdin -threads 8 -i "G:\TestClips&Co\files\interlaceAndTelecineSamples\interlaced\bff.m2v" -map 0:0 -an -sn -vf zscale=rangein=tv:range=tv -pix_fmt yuv420p10le -strict -1 -vsync 0 -top 1 -flags +ildct+ilme -f yuv4mpegpipe - | x265 --input - --output-depth 10 --y4m --profile main10 --limit-modes --no-early-skip --no-open-gop --opt-ref-list-length-pps --crf 18.00 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --limit-refs 0 --ssim-rd --psy-rd 2.50 --rdoq-level 2 --psy-rdoq 10.00 --aq-mode 0 --deblock=-1:-1 --limit-sao --no-repeat-headers --interlace bff --range limited --colormatrix bt470bg --sar 8:9 --output "G:\Output\2023-11-23@15_58_44_6310_01.265"
Note the '--interlace bff', MediaInfoLib - v23.10 reports:
MediaInfo.exe g:\Output\2023-11-23@15_58_44_6310_01.265
General
Complete name : g:\Output\2023-11-23@15_58_44_6310_01.265
Format : HEVC
Format/Info : High Efficiency Video Coding
File size : 2.78 MiB
Frame rate : 29.970 FPS
Writing library : x265 3.5+101-d2711b887:[Windows][GCC 12.2.0][64 bit] 10bit
Encoding settings : cpuid=1111039 / frame-threads=5 / numa-pools=32 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=720x480 / interlace=2 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=3 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-eob / no-eos / no-hrd / info / hash=0 / temporal-layers=0 / no-open-gop / min-keyint=25 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=20 / lookahead-slices=0 / scenecut=40 / no-hist-scenecut / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=0 / limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / no-frame-dup / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=-1:-1 / sao / no-sao-non-deblock / rd=3 / selective-sao=4 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.50 / psy-rdoq=10.00 / no-rd-refine / no-lossless / cbqpoffs=-2 / crqpoffs=-2 / rc=crf / crf=18.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30 / aq-mode=1 / aq-strength=0.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=255 / sar-width / : / sar-height=8:9 / overscan=0 / videoformat=5 / range=0 / colorprim=2 / transfer=2 / colormatrix=5 / chromaloc=0 / display-window=0 / cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / opt-qp-pps / opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / no-hdr10 / no-hdr10-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=0 / analysis-save-reuse-level=0 / analysis-load-reuse-level=0 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00 / scenecut-aware-qp=0conformance-window-offsets / right=0 / bottom=0 / decoder-max-rate=0 / no-vbv-live-multi-pass / no-mcstf / no-sbrc

Video
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L3@Main
Width : 720 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Frame rate : 29.970 (30000/1001) FPS
Standard : NTSC
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 10 bits
Writing library : x265 3.5+101-d2711b887:[Windows][GCC 12.2.0][64 bit] 10bit
Encoding settings : cpuid=1111039 / frame-threads=5 / numa-pools=32 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=720x480 / interlace=2 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=3 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-eob / no-eos / no-hrd / info / hash=0 / temporal-layers=0 / no-open-gop / min-keyint=25 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=20 / lookahead-slices=0 / scenecut=40 / no-hist-scenecut / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=0 / limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / no-frame-dup / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=-1:-1 / sao / no-sao-non-deblock / rd=3 / selective-sao=4 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.50 / psy-rdoq=10.00 / no-rd-refine / no-lossless / cbqpoffs=-2 / crqpoffs=-2 / rc=crf / crf=18.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30 / aq-mode=1 / aq-strength=0.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=255 / sar-width / : / sar-height=8:9 / overscan=0 / videoformat=5 / range=0 / colorprim=2 / transfer=2 / colormatrix=5 / chromaloc=0 / display-window=0 / cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / opt-qp-pps / opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / no-hdr10 / no-hdr10-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=0 / analysis-save-reuse-level=0 / analysis-load-reuse-level=0 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00 / scenecut-aware-qp=0conformance-window-offsets / right=0 / bottom=0 / decoder-max-rate=0 / no-vbv-live-multi-pass / no-mcstf / no-sbrc
Color range : Limited
Matrix coefficients : BT.470 System B/G
so no interlacing, and ffmpeg reports:
ffmpeg version N-112813-g1c3b13b9f7-gc961ac4b0c+1 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 13.2.0 (Rev2, Built by MSYS2 project)
configuration: --pkg-config=pkgconf --cc='ccache gcc' --cxx='ccache g++' --ld='ccache g++' --extra-cxxflags=-fpermissive --extra-cflags=-Wno-int-conversion --disable-autodetect --enable-amf --enable-bzlib --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-iconv --enable-lzma --enable-nvenc --enable-zlib --enable-sdl2 --enable-ffnvcodec --enable-nvdec --enable-cuda-llvm --disable-doc --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libdav1d --enable-libaom --disable-debug --enable-fontconfig --enable-libass --enable-libbluray --enable-libfreetype --enable-libmfx --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libwebp --enable-libxml2 --enable-libzimg --enable-libshine --enable-gpl --enable-avisynth --enable-libxvid --enable-libopenmpt --enable-version3 --enable-librav1e --enable-libsrt --enable-libgsm --enable-libvmaf --enable-libsvtav1 --enable-mbedtls --extra-cflags=-DLIBTWOLAME_STATIC --extra-libs=-lstdc++ --extra-cflags=-DLIBXML_STATIC --extra-libs=-liconv --disable-w32threads
libavutil 58. 32.100 / 58. 32.100
libavcodec 60. 34.100 / 60. 34.100
libavformat 60. 17.100 / 60. 17.100
libavdevice 60. 4.100 / 60. 4.100
libavfilter 9. 13.100 / 9. 13.100
libswscale 7. 6.100 / 7. 6.100
libswresample 4. 13.100 / 4. 13.100
libpostproc 57. 4.100 / 57. 4.100
Input #0, hevc, from 'g:\Output\2023-11-23@15_58_44_6310_01.265':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: hevc (Main 10), yuv420p10le(tv, bt470bg/unknown/unknown, top first), 720x480 [SAR 8:9 DAR 4:3], 25 fps, 29.97 tbr, 1200k tbn
so interlaced, but tff not bff.

Cu Selur

Ps.: to be sure my Vaporusynth script doesn't cause the problem, I also used:
ffmpeg -y -loglevel fatal -noautorotate -nostdin -threads 8 -i "G:\TestClips&Co\files\interlaceAndTelecineSamples\interlaced\bff.m2v" -map 0:0 -an -sn -vf zscale=rangein=tv:range=tv -pix_fmt yuv420p10le -strict -1 -vsync 0 -top 1 -flags +ildct+ilme -f yuv4mpegpipe - | x265 --input - --output-depth 10 --y4m --profile main10 --limit-modes --no-early-skip --no-open-gop --opt-ref-list-length-pps --crf 18.00 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --limit-refs 0 --ssim-rd --psy-rd 2.50 --rdoq-level 2 --psy-rdoq 10.00 --aq-mode 0 --deblock=-1:-1 --limit-sao --no-repeat-headers --interlace bff --range limited --colormatrix bt470bg --sar 8:9 --output "J:\tmp\2023-11-23@17_26_19_0110_01.265" and got the same result.

[LigH]

Does x265 report any warning in its console output while encoding?

Also there is a little version difference between your x265 info report and the "Writing library" metadata.

[Selur]

ffmpeg -y -loglevel fatal -noautorotate -nostdin -threads 8 -i "G:\TestClips&Co\files\interlaceAndTelecineSamples\interlaced\bff.m2v" -map 0:0 -an -sn -vf zscale=rangein=tv:range=tv -pix_fmt yuv420p10le -strict -1 -vsync 0 -top 1 -flags +ildct+ilme -f yuv4mpegpipe - | x265 --input - --output-depth 10 --y4m --profile main10 --limit-modes --no-early-skip --no-open-gop --opt-ref-list-length-pps --crf 18.00 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --limit-refs 0 --ssim-rd --psy-rd 2.50 --rdoq-level 2 --psy-rdoq 10.00 --aq-mode 0 --deblock=-1:-1 --limit-sao --no-repeat-headers --interlace bff --range limited --colormatrix bt470bg --sar 8:9 --output "G:\Output\2023-11-23@15_58_44_6310_01.265"
y4m [info]: 720x480 fps 30000/1001 i420p10 sar 8:9 unknown frame count
raw [info]: output file: G:\Output\2023-11-23@15_58_44_6310_01.265
x265 [info]: HEVC encoder version 3.5+111-c40c3d799
x265 [info]: build info [Windows][GCC 13.2.0][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [warning]: Support for interlaced video is experimental
x265 [info]: Main 10 profile, Level-3 (Main tier)
x265 [info]: Thread pool created using 32 threads
x265 [info]: Slices : 1
x265 [info]: frame threads / pool features : 5 / wpp(8 rows)
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [info]: Interlaced field inputs : bff
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : hex / 57 / 2 / 3
x265 [info]: Keyframe min / max / scenecut / bias : 25 / 250 / 40 / 5.00
x265 [info]: Cb/Cr QP Offset : -2 / -2
x265 [info]: Lookahead / bframes / badapt : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 3 / off / off
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 0.0 / 32 / 1
x265 [info]: Rate Control / qCompress : CRF-18.0 / 0.60
x265 [info]: tools: limit-modes rd=3 ssim-rd psy-rd=2.50 rdoq=2 psy-rdoq=10.00
x265 [info]: tools: rskip mode=1 signhide tmvp b-intra strong-intra-smoothing
x265 [info]: tools: deblock(tC=-1:B=-1) sao
x265 [info]: frame I: 1, Avg QP:19.12 kb/s: 11329.87
x265 [info]: frame P: 37, Avg QP:20.97 kb/s: 9996.83
x265 [info]: frame B: 112, Avg QP:25.53 kb/s: 2837.19
x265 [info]: Weighted P-Frames: Y:27.0% UV:21.6%

encoded 150 frames in 2.05s (73.17 fps), 4659.85 kb/s, Avg QP:24.37
=> "x265 [warning]: Support for interlaced video is experimental"

F:\Hybrid\64bit>x265 --version
x265 [info]: HEVC encoder version 3.5+111-c40c3d799
x265 [info]: build info [Windows][GCC 13.2.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2

F:\Hybrid\64bit>MediaInfo.exe G:\Output\2023-11-23@15_58_44_6310_01.265
General
Complete name : G:\Output\2023-11-23@15_58_44_6310_01.265
Format : HEVC
Format/Info : High Efficiency Video Coding
File size : 2.78 MiB
Frame rate : 29.970 FPS
Writing library : x265 3.5+111-c40c3d799:[Windows][GCC 13.2.0][64 bit] 10bit
Encoding settings : cpuid=1111039 / frame-threads=5 / numa-pools=32 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=720x480 / interlace=2 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=3 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-eob / no-eos / no-hrd / info / hash=0 / temporal-layers=0 / no-open-gop / min-keyint=25 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=20 / lookahead-slices=0 / scenecut=40 / no-hist-scenecut / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=0 / limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / no-frame-dup / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=-1:-1 / sao / no-sao-non-deblock / rd=3 / selective-sao=4 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.50 / psy-rdoq=10.00 / no-rd-refine / no-lossless / cbqpoffs=-2 / crqpoffs=-2 / rc=crf / crf=18.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30 / aq-mode=1 / aq-strength=0.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=255 / sar-width / : / sar-height=8:9 / overscan=0 / videoformat=5 / range=0 / colorprim=2 / transfer=2 / colormatrix=5 / chromaloc=0 / display-window=0 / cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / opt-qp-pps / opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / no-hdr10 / no-hdr10-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=0 / analysis-save-reuse-level=0 / analysis-load-reuse-level=0 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00 / scenecut-aware-qp=0conformance-window-offsets / right=0 / bottom=0 / decoder-max-rate=0 / no-vbv-live-multi-pass / no-mcstf / no-sbrc

Video
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L3@Main
Width : 720 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Frame rate : 29.970 (30000/1001) FPS
Standard : NTSC
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 10 bits
Writing library : x265 3.5+111-c40c3d799:[Windows][GCC 13.2.0][64 bit] 10bit
Encoding settings : cpuid=1111039 / frame-threads=5 / numa-pools=32 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=720x480 / interlace=2 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=3 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-eob / no-eos / no-hrd / info / hash=0 / temporal-layers=0 / no-open-gop / min-keyint=25 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=20 / lookahead-slices=0 / scenecut=40 / no-hist-scenecut / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=0 / limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / no-frame-dup / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=-1:-1 / sao / no-sao-non-deblock / rd=3 / selective-sao=4 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.50 / psy-rdoq=10.00 / no-rd-refine / no-lossless / cbqpoffs=-2 / crqpoffs=-2 / rc=crf / crf=18.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30 / aq-mode=1 / aq-strength=0.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=255 / sar-width / : / sar-height=8:9 / overscan=0 / videoformat=5 / range=0 / colorprim=2 / transfer=2 / colormatrix=5 / chromaloc=0 / display-window=0 / cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / opt-qp-pps / opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / no-hdr10 / no-hdr10-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=0 / analysis-save-reuse-level=0 / analysis-load-reuse-level=0 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00 / scenecut-aware-qp=0conformance-window-offsets / right=0 / bottom=0 / decoder-max-rate=0 / no-vbv-live-multi-pass / no-mcstf / no-sbrc
Color range : Limited
Matrix coefficients : BT.470 System B/G

F:\Hybrid\64bit>ffmpeg -i G:\Output\2023-11-23@15_58_44_6310_01.265
ffmpeg version N-112813-g1c3b13b9f7-gc961ac4b0c+1 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 13.2.0 (Rev2, Built by MSYS2 project)
configuration: --pkg-config=pkgconf --cc='ccache gcc' --cxx='ccache g++' --ld='ccache g++' --extra-cxxflags=-fpermissive --extra-cflags=-Wno-int-conversion --disable-autodetect --enable-amf --enable-bzlib --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-iconv --enable-lzma --enable-nvenc --enable-zlib --enable-sdl2 --enable-ffnvcodec --enable-nvdec --enable-cuda-llvm --disable-doc --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libdav1d --enable-libaom --disable-debug --enable-fontconfig --enable-libass --enable-libbluray --enable-libfreetype --enable-libmfx --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libwebp --enable-libxml2 --enable-libzimg --enable-libshine --enable-gpl --enable-avisynth --enable-libxvid --enable-libopenmpt --enable-version3 --enable-librav1e --enable-libsrt --enable-libgsm --enable-libvmaf --enable-libsvtav1 --enable-mbedtls --extra-cflags=-DLIBTWOLAME_STATIC --extra-libs=-lstdc++ --extra-cflags=-DLIBXML_STATIC --extra-libs=-liconv --disable-w32threads
libavutil 58. 32.100 / 58. 32.100
libavcodec 60. 34.100 / 60. 34.100
libavformat 60. 17.100 / 60. 17.100
libavdevice 60. 4.100 / 60. 4.100
libavfilter 9. 13.100 / 9. 13.100
libswscale 7. 6.100 / 7. 6.100
libswresample 4. 13.100 / 4. 13.100
libpostproc 57. 4.100 / 57. 4.100
Input #0, hevc, from 'G:\Output\2023-11-23@15_58_44_6310_01.265':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: hevc (Main 10), yuv420p10le(tv, bt470bg/unknown/unknown, top first), 720x480 [SAR 8:9 DAR 4:3], 25 fps, 29.97 tbr, 1200k tbn

Cu Selur

[Br4twurscht]

Hi together,

normally I'm using StaxRip for my encodings and was always happy. HDR10 was added from the source automatically and I could focus on filters, audio and some x265 settings. But now I wanted to try encodes with dolby vision and doesn't get it working.

Firstly, I wanted to ask a question for general understanding. Will the dv metadata be used in the encoding process of x265 in that way, that it has influence on the encoding result itself? Or is the dv metadata "only" added additionally to the stream afterwards and will then be interpreted by the decoder? Or in other words, assuming a dv source and doing one encoding with the rpu file and one without, would the target stream be the same?

Secondly, I wanted to ask if there are guides how to handle the dv stuff for the encoding. I tried this guide https://github.com/staxrip/staxrip/wiki/Encoding-Dolby-Vision-with-StaxRip-using-x265 and experimented by myself, but run always in an error like
x265 [INFO]: HEVC encoder version 3.5+147+17-e8947f740 [Mod by Patman]
x265 [INFO]: build info [Windows][MSVC 1937][64 bit] 10bit
x265 [INFO]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [FLAW]: Dolby Vision requires VBV settings to enable HRD.
x265 [FLAW]: x265_encoder_open() failed for Enc,
x265 [WARN]: Dolby Vision RPU count is greater than frame count in x265
x265 [INFO]: VES muxing with Dolby Vision RPU file successful in x265
aborted at input frame 1, output frame 0 in x265

I read a little bit in the DDVT Thread, but this gave me the impression, that the dv stuff can be muxed and demuxed into/from the container. So, this led me to qustion #1 and I'm still more confused than at the beginning...

Greetings
Br4twurscht

[RanmaCanada]

Hi together,

normally I'm using StaxRip for my encodings and was always happy. HDR10 was added from the source automatically and I could focus on filters, audio and some x265 settings. But now I wanted to try encodes with dolby vision and doesn't get it working.

Firstly, I wanted to ask a question for general understanding. Will the dv metadata be used in the encoding process of x265 in that way, that it has influence on the encoding result itself? Or is the dv metadata "only" added additionally to the stream afterwards and will then be interpreted by the decoder? Or in other words, assuming a dv source and doing one encoding with the rpu file and one without, would the target stream be the same?

Secondly, I wanted to ask if there are guides how to handle the dv stuff for the encoding. I tried this guide https://github.com/staxrip/staxrip/wiki/Encoding-Dolby-Vision-with-StaxRip-using-x265 and experimented by myself, but run always in an error like
x265 [INFO]: HEVC encoder version 3.5+147+17-e8947f740 [Mod by Patman]
x265 [INFO]: build info [Windows][MSVC 1937][64 bit] 10bit
x265 [INFO]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [FLAW]: Dolby Vision requires VBV settings to enable HRD.
x265 [FLAW]: x265_encoder_open() failed for Enc,
x265 [WARN]: Dolby Vision RPU count is greater than frame count in x265
x265 [INFO]: VES muxing with Dolby Vision RPU file successful in x265
aborted at input frame 1, output frame 0 in x265

I read a little bit in the DDVT Thread, but this gave me the impression, that the dv stuff can be muxed and demuxed into/from the container. So, this led me to qustion #1 and I'm still more confused than at the beginning...

Greetings
Br4twurscht

DV is proprietary, and basically what you need to do is follow the information in the DDVT Thread. You do not encode in DV.

https://forum.doom9.org/showthread.php?t=182311

It's a mess.

[Br4twurscht]

Ok, thx. Then I try my luck with the DDVT.

[benwaggoner]

What's the best source for a multi depth Mac OS ARM x265 binary with all the latest and greatest ARM optimizations?

Last I checked the Xcode version was all messed up and required manual patching. I was never able to get that to work for >8-bit.

It appears the brew version is from 2022!

==> x265: stable 3.5 (bottled), HEAD
% brew info x265
H.265/HEVC encoder
https://bitbucket.org/multicoreware/x265_git
/opt/homebrew/Cellar/x265/HEAD-0b75c44 (11 files, 11.8MB) *
Built from source on 2022-10-20 at 12:33:05
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/x/x265.rb
License: GPL-2.0-only
==> Dependencies
Build: cmake ✔
==> Options
--HEAD
Install HEAD version
==> Analytics
install: 20,013 (30 days), 59,842 (90 days), 198,356 (365 days)
install-on-request: 229 (30 days), 537 (90 days), 1,652 (365 days)
build-error: 16 (30 days)

[qyot27]

MacPorts (https://ports.macports.org/port/x265/details/)? There is a highdepth variant*, and from what I can tell when looking at the portfile, it doesn't look like it's pinned to a release tarball, so it probably is pulling the latest git HEAD as a snapshot tarball and going from there.

*which you'd enable by tacking '+highdepth' onto the install command.

[benwaggoner]

MacPorts (https://ports.macports.org/port/x265/details/)? There is a highdepth variant*, and from what I can tell when looking at the portfile, it doesn't look like it's pinned to a release tarball, so it probably is pulling the latest git HEAD as a snapshot tarball and going from there.

*which you'd enable by tacking '+highdepth' onto the install command.
That worked!

% x265 --version
x265 [info]: HEVC encoder version 3.5+40-0b75c44c1
x265 [info]: build info [Mac OS X][clang 14.0.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: NEON

That said 3.5+40 quite a while ago, and I think lacks a lot of the NEON optimizations. We've already hit at least 3.5+110. Any suggestions on how to get a more recent branch?

[qyot27]

Ah, yeah it looks like it is set to a particular release after all, although I'm not seeing exactly where they're grabbing 3.5+40-0b75c44c1 from, because the Portfile is set to 3.4, and none of the videolan x265 branches are fresh enough to even approach that.

But there might be a solution to edit the Portfile to use a newer version locally and use that. I haven't done so myself (that I can remember, anyway), but the page on the MacPorts Trac is here:
https://trac.macports.org/wiki/howto/Upgrade

There's also this part of the documentation that describes the Port Groups function that interfaces with Github:
https://guide.macports.org/chunked/reference.portgroup.html

So my guess is that the necessary change would be to select a different Github repository that mirrors x265*, and set the version to the most recent commit. Adjust the checksums/size fields to match what those should be for something like that (like this one did when bumping to 3.4 (https://github.com/macports/macports-ports/commit/fd29c3a9a0e32afb2b7ffe7fcc688021f9a5c337)), and then attempt to build it again.

*most of the videolan mirror network repos aren't synced with bitbucket either. I have one I used a few months ago to refresh the Yuuki patchset (https://github.com/qyot27/x265/commits/main), for example; it's only missing the 4 latest commits.

[benwaggoner]

x265 v3.5+115
https://www.mediafire.com/file/ftoufvn6cjvz90n
Apropos of the above, do you have the ability to compile a Mac OS ARM build ;)?

[Barough]

Sorry, it's not possible to do that with m-ab-s.

Sent from my SM-S908B via Tapatalk

[qyot27]

Apropos of the above, do you have the ability to compile a Mac OS ARM build ;)?

https://www.mediafire.com/file/ees2cmznwfig4m8/x265_macos_arm_build.tar.xz/file

It seems to compile okay using AppleClang 14 on Monterey.

[benwaggoner]

https://www.mediafire.com/file/ees2cmznwfig4m8/x265_macos_arm_build.tar.xz/file

It seems to compile okay using AppleClang 14 on Monterey.
Awesome thanks! And it sure did the trick. 4K slower Encoding speeds on a M1 Pro are at least a third faster than my brew 3.5+20 build.

MacPorts made a 32-bit no-asm version for some horrible reason; I didn't even bother to benchmark that.

[Ritsuka]

x265 arch check was broken, it printed 32-bit even on 64-bit ARM versions, actually there is no way to run a macOS 32-bit executable on macOS these days.
Unfortunately there are a lot of repositories stuck on the last "official" x265 version. And some websites like phoronix.com still uses 3.4 in their x86_64 vs ARM benchmarks.

[Hellboy.]

I see some people show this with their encode:

x265 [info]: frame I: 54, Avg QP:13.82 kb/s: 18717.13
x265 [info]: frame P: 1128, Avg QP:15.83 kb/s: 13458.52
x265 [info]: frame B: 6031, Avg QP:20.85 kb/s: 5528.32
x265 [info]: Weighted P-Frames: Y:2.0% UV:1.5%
encoded 7213 frames in 396.22s (18.20 fps), 6867.22 kb/s, Avg QP:20.01

What is "Avg QP"? That mean something for the quality of the encode? Lower or higher is better?
Thanks

[LigH]

QP is the "Quantizer Parameter". Imagine it as a divider to reduce the variety of color component values. The higher the quantizer, the less accurate the storage of the video, the more obvious the loss gets, visible in video artifacts like blocking, ringing around edges, banding in smooth color ramps. Lower values help keeping better quality, but that requires more bitrate because the stored values are more variable, so less of them can be reduced to an "almost the same as somewhere else" (visual redundancy).

The x264 and x265 encoders do not apply a constant quantizer to all frames, except you use a parameter which enforces that; in general (in 1 pass CRF mode or in 2-pass mode with a target size), they use a metric called "Rate Factor" which tries to keep the visual loss caused by the encoding below a specific threshold, which allows the quantizer to vary a bit in relation to the complexity of the video content.

[Hellboy.]

There is some "Avg QP" number that encoders try to achieve? Example: Like below 20.

[LigH]

Only if you enforce that by an additional parameter (which may lead to the resulting bitrate exceeding your expectations).

--qpfile <string> Force frametypes and QPs for some or all frames
Format of each line: framenumber frametype QP
QP is optional (none lets x265 choose). Frametypes: I,i,K,P,B,b.
QPs are restricted by qpmin/qpmax.

-q/--qp <integer> QP for P slices in CQP mode (implied). --ipratio and --pbration determine other slice QPs

--qpmax <integer> sets a hard upper limit on QP allowed to ratecontrol. Default 69

In general, though, the rate control algorithm is allowed to vary quantization values as required by the video content.

[Hellboy.]

Using higher number on these settings always translate in better quality? (For Blu-ray content with 8,000Bitrate or more and knowing that the encode is going to take more time.) (Slow crf17)

bframes
rc-lookahead
subme

I ask because higher preset use higher numbers and a lot of encoders use slow preset with higher numbers.

[LigH]

Not certainly.

B frames are useful in 2-pass encoding to achieve a lower average QP of I and P frames because they often use less bitrate, but use a higher QP themselves by default (I/P/B ratios) because their loss is only visible for a brief time and does not propagate much through time (in contrast to the loss of a P frame sequence). In a CRF encoding they don't improve quality, only help reducing the output size. Also there are hardware decoders which are limited in features. A range of 1-3 consecutive B frames is rather safe. In case of more than 2, enable the B frame pyramid so their weighting is distributed better.

The rate control lookahead also does not help much in the CRF case because there is no rate control on top to alter the target rate factor, you set it constant. It is more important in 2-pass with a target size and in 1-pass with a rather constant average bitrate.

A finer motion estimation is helpful in general. But there is no guarantee for exceptional cases where less than the maximum would already be enough.

[benwaggoner]

B frames are useful in 2-pass encoding to achieve a lower average QP of I and P frames because they often use less bitrate, but use a higher QP themselves by default (I/P/B ratios) because their loss is only visible for a brief time and does not propagate much through time (in contrast to the loss of a P frame sequence). In a CRF encoding they don't improve quality, only help reducing the output size. Also there are hardware decoders which are limited in features. A range of 1-3 consecutive B frames is rather safe. In case of more than 2, enable the B frame pyramid so their weighting is distributed better.
Are there decoders outside of optical disc that are limited to three? I've been using up to 8 on a very wide variety of streaming players for a decade, without any compatibility issues.

The rate control lookahead also does not help much in the CRF case because there is no rate control on top to alter the target rate factor, you set it constant. It is more important in 2-pass with a target size and in 1-pass with a rather constant average bitrate.
Lookahead absolutely can help CRF if the VBV limit it hit (--vbv-bufsize and --vbv-maxrate set, which is required to set a Profile and Level for device compatibility). When the VBV limit would be hit, more lookahead lets the encoder smooth out the bitrate fluctuation, and thus moderate the worst-case QP spike/quality hit.

RAM permitting, rc-lookhead=keyint provides optimal quality, although there are diminishing returns past 50 or so.

A finer motion estimation is helpful in general. But there is no guarantee for exceptional cases where less than the maximum would already be enough.
Yeah, like a lot of quality/speed tradeoff parameters, you're doing stuff that makes it slower for all content that will only help some content. The higher the bitrate, the less likely there will be any visible changes.

[LigH]

Are there decoders outside of optical disc that are limited to three? I've been using up to 8 on a very wide variety of streaming players for a decade, without any compatibility issues.

Maybe early Apple mobile and Quicktime decoders ... surely improving over time.

[benwaggoner]

Maybe early Apple mobile and Quicktime decoders ... surely improving over time.
I'm quite confident that all devices in active use in the last decade (no one is still using the original iPhone!) support up to five b-frames.

[Hellboy.]

(For Blu-ray 1080p content with 8,000Bitrate or more, Slow crf17)

I change ctu=64 to ctu=32.
Is there any other setting that is recommended to change? Maybe "merange" from 57 to 25, i see sometimes that setting is lowered.
Thanks.

[Boulder]

(For Blu-ray 1080p content with 8,000Bitrate or more, Slow crf17)

I change ctu=64 to ctu=32.
Is there any other setting that is recommended to change? Maybe "merange" from 57 to 25, i see sometimes that setting is lowered.
Thanks.

--no-sao and --rskip 2 --rskip-edge-threshold 2 will retain more details.

[LigH]

New upload: x265 3.5+114-74abf80c7 (https://www.mediafire.com/file/d9pxofte208sq4m/x265_3.5+114-74abf80c7.7z/file)

[Windows][GCC 13.2.0][32/32XP/64 bit] 8bit+10bit+12bit

[FranceBB]

Thanks for the updated builds and for still targeting Windows XP, you're the only one left who's still doing it! :)

[tormento]

I am testing PSNR, SSIM and VMAF metrics. Many switches, that were suggested to have higher quality encodes, give worse PSNR, SSIM and even VMAF scores.

How much can I trust synthetic and perceptive metrics?

Do you have any better metric to suggest me?

P.S: I am using FFMetrics (https://github.com/fifonik/FFMetrics).

[LigH]

No objective metric (calculated from a difference of clips) is equal to a subjective opinion gathered in a survey among a bunch of people.

[tormento]

among a bunch of people.
As I can't summon one thousand people in my room every time I need a reference, is VMAF good enough or there is something better?

[LigH]

AFAIR: PSNR is rather simple and easy to confuse; SSIM is a bit better but compares only single frames, like PSNR; VMAF is even more advanced, even takes some temporal development into account. I do not yet know any better objective metric than VMAF.

As a more subjective modification, I don't know how to rate psycho-visual enhancements in comparison, how encoders apply them. But I am pretty sure that they exist for a good reason. And they are best rated while watching the movie at usual speed, not frame by frame, not with a magnifier, not with a difference calculation.

[benwaggoner]

As I can't summon one thousand people in my room every time I need a reference, is VMAF good enough or there is something better?
If there's a VMAF difference of at least three, that generally indicates a visible quality difference. Differences smaller than that can go either way.

VMAF really can't tell the difference between different --aq-mode and --aq-strength values, so it's not helpful to tune those parameters.

VMAF is also luma only, so chroma issues will go entirely ignored.

[tormento]

VMAF is also luma only, so chroma issues will go entirely ignored.

What do you suggest me, but my eyes?

[benwaggoner]

What do you suggest me, but my eyes?
Someone else's eyes ;)?

The best quality metric available to the public I know of is p1204.3.

Some info: https://streaminglearningcenter.com/blogs/itu-t-p1203-p1204.html

[tormento]

The best quality metric available to the public I know of is p1204.3.
Do you know some working Windows build?

[Blue_MiSfit]

There's a Dockerfile in the reference implementation here:
https://github.com/Telecommunication-Telemedia-Assessment/bitstream_mode3_p1204_3

I'll cut a Docker build for you :)

[Blue_MiSfit]

https://hub.docker.com/repository/docker/dprestegard/bitstream_mode3_p1204_3/general


$ docker run dprestegard/bitstream_mode3_p1204_3:latest
usage: __main__.py [-h] [--result_folder RESULT_FOLDER] [--model MODEL]
[--cpu_count CPU_COUNT]
[--device_type {pc,tv,tablet,mobile}]
[--device_resolution {3840x2160,2560x1440}]
[--viewing_distance {1.5xH,4xH,6xH}]
[--display_size {10,32,37,5.1,5.5,5.8,55,65,75}]
[--tmp TMP] [-d] [-nocached_features] [-q]
video [video ...]
__main__.py: error: the following arguments are required: video


If you're not familiar with Docker, you can just install Docker Desktop on windows and then run

docker run dprestegard/bitstream_mode3_p1204_3:latest

You can pass in files on your local system via a bind mount. It's pretty simple :) https://docs.docker.com/storage/bind-mounts/

[Ritsuka]

Does anyone get random crashes when running on a CPU with heterogeneous cores? I've got some reports of on both Apple Silicon and Intel, but couldn't reproduce anything yet.

[LigH]

You mean something like ARM big.LITTLE (https://en.wikipedia.org/wiki/ARM_big.LITTLE) as used in some mobile devices?

[Ritsuka]

Yes, but that's not only in mobile devices, even an Intel desktop CPU like i9-14900K has got 8 performance cores and 16 efficient cores for example.

[tormento]

I'll cut a Docker build for you :)
I'd prefer to use a "cleaner" solution, i.e. direct exe.

[rwill]

Someone else's eyes ;)?

The best quality metric available to the public I know of is p1204.3.

Some info: https://streaminglearningcenter.com/blogs/itu-t-p1203-p1204.html

Its a "no-reference" model? And this is supposed to be the best? Press X for doubt.

[benwaggoner]

Its a "no-reference" model? And this is supposed to be the best? Press X for doubt.
It has the best subjective correlation for perceptual video quality that I've seen.

Of course, VMAF isn't exactly a video quality metric - it is a video distortion metric. It doesn't try to say if video looks good or not, but how perceptually degraded from the source it will be.

VMAF will rate a very accurate reencode of a very terrible quality source very high, and a pretty accurate reencode of a great looking source lower, even though viewers would say the latter looks much better.

p1204.3 predicts how a viewer would relate the final quality delivered. As viewers don't have a source to compare to, that experience is fundamentally no-reference, so a no-reference metric can make sense.

Quality and distortion metrics each have their place, and which is appropriate is based on what you're trying to accomplish.

That said, I find p1204.3 to be more useful in encoder tuning than VMAF, even though a distortion metric would seem more appropriate.

There has been some interesting early work done combining reference, bitstream, and baseband analysis for further subjective correlation improvements. The nice thing about Machine Learning is that you can throw in whatever base metrics you want to for training as long as you've got a good ground truth data set of subjective ratings.

[benwaggoner]

Do you know some working Windows build?
I don't. It's not something I run locally. It gets done in the Cloud on Linux.

[cubicibo]

Is there any reference post or site providing reference x265 encode parameters for UHD BD?

[LigH]

New upload: x265 3.5+119-68298344c (https://www.mediafire.com/file/cdsh3bjxrem3ilk/x265_3.5+119-68298344c.7z/file)

[Windows][GCC 13.2.0][32/32XP/64 bit] 8bit+10bit+12bit

Recent fixes: high bit depth in y4m; lookahead in qp file; min VBV fullness

[LigH]

New upload: x265 3.5+120-7c8a449ff (https://www.mediafire.com/file/0pcbu7pe7n0u87d/x265_3.5+120-7c8a449ff.7z/file)

[Windows][GCC 13.2.0][32/32XP/64 bit] 8bit+10bit+12bit

Recent fixes: Qp tuning in SBRC

[FranceBB]

New upload: x265 3.5+120-7c8a449ff (https://www.mediafire.com/file/0pcbu7pe7n0u87d/x265_3.5+120-7c8a449ff.7z/file)

[Windows][GCC 13.2.0][32/32XP/64 bit] 8bit+10bit+12bit

Recent fixes: Qp tuning in SBRC

Thanks for the new build.


p1204.3 predicts how a viewer would relate the final quality delivered. As viewers don't have a source to compare to, that experience is fundamentally no-reference, so a no-reference metric can make sense.

Yes absolutely.
I think Tektronix uses something similar in Cerify/Aurora called "PVQ" (Perceptual Video Quality) which analyses the video without any reference and spits out a score.
Too bad Telestream bought them and turned them into a cloud pay-as-you-go thingie with the name Qualify... :(
Recently, I used it to analyze the quality effect on different bitrate controls of two different distribution encodings:

https://i.imgur.com/Qcrpc1t.png
https://i.imgur.com/S7HGKp6.png

the first focuses on keeping the bitrate very still and not making it oscillate, while the second is able to preserve the quality on a more homogenous way across the encode (hence less "steps"). I don't deal much with distribution encodes, but rather content acquisition and I can say that those are generally very useful when it comes to mezzanine files 'cause if a supplier delivers a perfectly valid mezzanine file according to our specs (a rarity, but it happens), I don't have to re-encode it, so I just pass it on to the next step as it gets Loudness Corrected and after that it goes into AutoQC to detect freeze frames, black scenes etc. In this AutoQC step, we perform this kind of PVQ control and if the value is lower than the threshold, it gets flagged.
These days there aren't people eyeballing every piece of content coming through (unfortunately :(), so having a no-reference perceptual video quality control that can spot bad encodes automatically is really important before passing to the playout the actual TX Ready file.


It gets done in the Cloud on Linux.

Yep, same here.


Do you know some working Windows build?

I really hope someone makes a free open source cross platform implementation running either in Avisynth or FFMpeg, but I guess we're gonna have to wait. Unfortunately most PVQ implementations are proprietary and what's worse is that these days everything seems to be running on the cloud with a pay-as-you-go model instead of just buying the license and forgetting about it. I have stuff running on prem for which we bought the license 15+ years ago and never ever broke. Those were one-time transactions and those companies never saw a penny after that. Now that the cloud is "a thing" they see the milking opportunity of getting a constant revenue / influx of cash by charging the customers all the time.


This is also why - despite also being in the cloud - I try to run stuff on my own EC2 on AWS and use open source software, the likes of Avisynth, x26x, the BBC BMX mxf muxer etc. I really hope open source will kick everybody's butt in the long run, but after seeing greedy companies profit from this and plenty of people not donating/contributing to open source projects, it starts to feel like we're losing the battle sometimes... :(

[benwaggoner]

I really hope someone makes a free open source cross platform implementation running either in Avisynth or FFMpeg, but I guess we're gonna have to wait. Unfortunately most PVQ implementations are proprietary and what's worse is that these days everything seems to be running on the cloud with a pay-as-you-go model instead of just buying the license and forgetting about it. I have stuff running on prem for which we bought the license 15+ years ago and never ever broke. Those were one-time transactions and those companies never saw a penny after that. Now that the cloud is "a thing" they see the milking opportunity of getting a constant revenue / influx of cash by charging the customers all the time.

This is also why - despite also being in the cloud - I try to run stuff on my own EC2 on AWS and use open source software, the likes of Avisynth, x26x, the BBC BMX mxf muxer etc. I really hope open source will kick everybody's butt in the long run, but after seeing greedy companies profit from this and plenty of people not donating/contributing to open source projects, it starts to feel like we're losing the battle sometimes... :(
Yeah. Were I actually a developer (I stopped writing code for money in, sheesh, 1995?) I could port the p1204.3 to other platforms. It's open source on GitHub and everything: https://github.com/Telecommunication-Telemedia-Assessment/bitstream_mode3_p1204_3.

I failed to mention that .3 brought "Hybrid Mode 0" which combines both bitstream analysis and decoded pixel analysis for better subjective correlation (and much slower processing).

[Boulder]

Umm.. I don't see why that program wouldn't run on Windows. After all, it's a Python project and exports a JSON file.

[benwaggoner]

Umm.. I don't see why that program wouldn't run on Windows. After all, it's a Python project and exports a JSON file.
See, that's exactly the sort of thing I would realize if I was still a developer :sly:.

[benwaggoner]

x265 v3.6+2

Wow, after three years, a new x265 version is coming!

Some good stuff in there!

https://bitbucket.org/multicoreware/x265_git/commits/aa7f602f7592eddb9d87749be7466da005b556ee

Release Notes
*************

Version 3.6
===========

Release date - 4th April, 2024.

New feature
-----------
1. Segment based Ratecontrol (SBRC) feature
2. Motion-Compensated Spatio-Temporal Filtering
3. Scene-cut aware qp - BBAQ (Bidirectional Boundary Aware Quantization)
4. Histogram-Based Scene Change Detection
5. Film-Grain characteristics as a SEI message to support Film Grain Synthesis(FGS)
6. Add temporal layer implementation(Hierarchical B-frame implementation)

Enhancements to existing features
---------------------------------
1. Added Dolby Vision 8.4 Profile Support

API changes
-----------
1. Add Segment based Ratecontrol(SBRC) feature: "--[no-]sbrc".
2. Add command line parameter for mcstf feature: "--[no-]mctf".
3. Add command line parameters for the scene cut aware qp feature: "--scenecut-aware-qp" and "--masking-strength".
4. Add command line parameters for Histogram-Based Scene Change Detection: "--hist-scenecut".
5. Add film grain characteristics as a SEI message to the bitstream: "--film-grain <filename>"
6. cli: add new option --cra-nal (Force nal type to CRA to all frames expect for the first frame, works only with keyint 1)

Optimizations
---------------------
ARM64 NEON optimizations:- Several time-consuming C functions have been optimized for the targeted platform - aarch64. The overall performance increased by around 20%.
SVE/SVE2 optimizations

Bug fixes
---------
1. Linux bug to utilize all the cores
2. Crash with hist-scenecut build when source resolution is not multiple of minCuSize
3. 32bit and 64bit builds generation for ARM
4. bugs in zonefile feature (Reflect Zonefile Parameters inside Lookahead, extra IDR issue, Avg I Slice QP value issue etc..)
5. Add x86 ASM implementation for subsampling luma
6. Fix for abrladder segfault with load reuse level 1
7. Reorder miniGOP based on temporal layer hierarchy and add support for more B frame
8. Add MacOS aarch64 build support
9. Fix boundary condition issue for Gaussian filter

[tormento]

Wow, after three years, a new x265 version is coming!
Did you find some more extensive descriptions of the new features and where/when to use them?

[Boulder]

Those "new" ones are mostly quite old things already I'm afraid, and nothing really special for normal use cases. They didn't even bother fixing the histogram scene change bug I reported a long time ago.

[jpsdr]

What is the bug ?
Is it "just" a bug description, or is there already a fix but not commited in the release branch ?

[Boulder]

What is the bug ?
Is it "just" a bug description, or is there already a fix but not commited in the release branch ?
No fix whatsoever, not even a single comment in the issue I posted a long time ago.

The feature doesn't work for SDR encoding using the Main10 profile.
https://bitbucket.org/multicoreware/x265_git/issues/626/hist-scenecut-is-broken-for-main10-encodes

And this one is interesting, I'd expect them to be interested in clear quality issues.
https://bitbucket.org/multicoreware/x265_git/issues/561/rskip-2-coupled-with-limit-tu-0-and-ctu-64

[benwaggoner]

Those "new" ones are mostly quite old things already I'm afraid, and nothing really special for normal use cases. They didn't even bother fixing the histogram scene change bug I reported a long time ago.
Perhaps you should try resubmitting your bug before they actually release 3.6? Maybe a reminder could help. There's been quite a lot of turnover at MCW since 3.5.

As for as "new" things, yeah, for people who have been using post 3.5 builds we've had a lot for a while.

There is some general purpose stuff in there that should be broadly helpful:
6. Add temporal layer implementation(Hierarchical B-frame implementation)

Reorder miniGOP based on temporal layer hierarchy and add support for more B frame

And there are always welcome performance improvements, especially for ARM. Apple Silicon was over 2x faster than 3.5 last I benchmarked.

I've not extensively tested all the new stuff around scene cut detection and scene cut aware QP optimization, but if they work decently, those should help quality and compression efficiency a bit.

[benwaggoner]

Did you find some more extensive descriptions of the new features and where/when to use them?
This is it other than reading the comments in the individual commits.

Hopefully we'll get the new stuff added to x265.readthedocs.io before too long.

[Selur]

Hopefully we'll get the new stuff added to x265.readthedocs.io before too long.
100% agree, seems like they get sloppy, before they always added those changed when a new option appeared.

[Barough]

They haven't updated it since 2021.

Sent from my SM-S908B via Tapatalk

[LigH]

New upload: x265 3.6+1-dd1ef69b2 (https://www.mediafire.com/file/n0n1p02or3ncz2u/x265_3.6+1-dd1ef69b2.7z/file)

[Windows][GCC 13.2.0][32/32XP/64 bit] 8bit+10bit+12bit

I guess the version numbering might depend on details like the selection of a branch.

[guest]

New upload: x265 3.6+1-dd1ef69b2 (https://www.mediafire.com/file/n0n1p02or3ncz2u/x265_3.6+1-dd1ef69b2.7z/file)

[Windows][GCC 13.2.0][32/32XP/64 bit] 8bit+10bit+12bit

I guess the version numbering might depend on details like the selection of a branch.

Well, this is a little confusing, which is the newer ??

x265 v3.6+2
Built on April 04, 2024, GCC 13.2.0

[Barough]

Mine is built from the master branch.

Sent from my SM-S908B via Tapatalk

[FranceBB]

New upload: x265 3.6+1-dd1ef69b2 (https://www.mediafire.com/file/n0n1p02or3ncz2u/x265_3.6+1-dd1ef69b2.7z/file)

[Windows][GCC 13.2.0][32/32XP/64 bit] 8bit+10bit+12bit


Thanks, still going strong on XP. :D

https://i.imgur.com/UyYOfgr.png

[vpupkind]

The SBRC feature is still borked, hope they'll fix it eventually.
It tends to increase both bitrate and quality and it is not clear whether there is any benefit from it vs plain giving more bits.

[vpupkind]

No fix whatsoever, not even a single comment in the issue I posted a long time ago.

And this one is interesting, I'd expect them to be interested in clear quality issues.
https://bitbucket.org/multicoreware/x265_git/issues/561/rskip-2-coupled-with-limit-tu-0-and-ctu-64
If I remember correctly, rskip 2 uses edges at the smallest CU's to see if they can be painlessly combined. It is meaningful when you do allow small CUs, so my bet is you won't get much out of the feature if you don't allow at least 16x16 CUs.

[Boulder]

If I remember correctly, rskip 2 uses edges at the smallest CU's to see if they can be painlessly combined. It is meaningful when you do allow small CUs, so my bet is you won't get much out of the feature if you don't allow at least 16x16 CUs.

I've always used at least depth 3 so the encoder has been allowed to use small CUs. Also the issue won't exist with CTU 32 with otherwise the exact same settings, or if you use limit-tu.

[LigH]

Well, this is a little confusing, which is the newer ??

As I wrote: I cannot guarantee for patch numbers being counted correctly, it depends on the selected branch and maybe even the method how the update of the local work copy happens. But the very last commits are usually just a minor detail, like adding a new tag to each branch, which changes nothing in the code.

See https://bitbucket.org/multicoreware/x265_git/commits/

[tormento]

In my local anime community there is a discussion about modern anime material, i.e. not pure line art like Dragon Ball used to be (I named a very well known one to give you an idea).

More and more complex animation techniques are nowadays used, from motion capture to advanced CGI and After Effects sfx. One of the most annoying (at least for me) is the use of fake lower resolution and fake film grain, such as in Mushoku tensei anime.

Ben and others encoding guru: can you please address me to a correct x265 usage to take care of the modern anime standards?

We need to maintain clear lines definition, good background clarity with no banding and, sometimes, deal with resize artifacts (halo and aliasing above all) where multiple resolution on the very same frames doesn’t allow us to have a proper downscale+upscale trick.

There is a lot of old beliefs about disabling SAO, using tune animation values or deblock and so on.

Could someone clarify how x265 has evolved towards those aspects and give me some hints for that material type?

[jpsdr]

Made a build of new version of my custom mod. Also add an AVX LLVM version, don't remember who asked for it...

[tormento]

Made a build of new version of my custom mod. Also add an AVX LLVM version, don't remember who asked for it...
Perhaps I am the only one to still use an AVX only CPU.

Thanks. ;)

P.S: Do you have any idea why the metrics (I look at VMAF, mostly) about your auto-aq and the patman one are so different? I have tried almost any value but the patman one almost always gives better results.

[Boulder]

Various AQ modes do not necessarily increase metric scores. I don't understand why people don't trust their eyes more.

What I sometimes run is SSIMU2 to check if the std dev or worst 5% value changes when adjusting parameters. I don't care much about the actual score.

[FranceBB]

More and more complex animation techniques are nowadays used, from motion capture to advanced CGI and After Effects sfx. One of the most annoying (at least for me) is the use of fake lower resolution and fake film grain

This has unfortunately been coming for a long time and I personally really hate it. I think most of it is driven by a cost factor as they can't afford to spend quite as much as they used to and they therefore try to rely on CGI. I still remember when I fansubbed and encoded Shingeki no Bahamut, the first series, in Italian. It was incredibly well drawn, very sharp edges too when I inverted the kernel of the upscaled BD (DeBilinearResizeMT) and encoded at 720p 4:4:4, but there was one thing that annoyed the hell out of me: the dragon. I mean, everything was manually drawn, all the characters and everything and then there was this huge blatantly computer graphics generated dragon (i.e the Bahamut) that quite astoninglishly was popping out and to hide its computer generated nature what have they done? Well, the oldest trick in the house: put tons of grain on it. I hate it. I understand when movie studios do this, for instance all the very recent Star Wars movie have a lot of grain to cover up the fact that some elements of the spaceships and other things are CGI and that's totally and absolutely fine 'cause you have real characters, real sets, enhanced virtualized sets (i.e they shoot with different cameras with the monitors in the background synced at a multiplier of the framerate of the cameras so that it generates a fake background on which it keeps the field of view, thus extending the set) and then fake computer generated items added in post. In other words, it makes total sense 'cause otherwise it would look plasticky. But for anime? Where nothing is real anyway? Nah. Either do it like Disney where everything is computer drawn with Maya created models which are rigged and then animated or just stick to the good old animation techniques with manually drawn characters, the way animes used to be made. Having something in between as they're doing now is utter nonsense.

[tormento]

Various AQ modes do not necessarily increase metric scores.
I don't have a calibrated monitor, I can't trust my eyes.

[Selur]

btw. how does one create film grain files for x265s '--film-grain' option for a given source?

[modus-ms325c]

This has unfortunately been coming for a long time and I personally really hate it. I think most of it is driven by a cost factor as they can't afford to spend quite as much as they used to and they therefore try to rely on CGI. I still remember when I fansubbed and encoded Shingeki no Bahamut, the first series, in Italian. It was incredibly well drawn, very sharp edges too when I inverted the kernel of the upscaled BD (DeBilinearResizeMT) and encoded at 720p 4:4:4, but there was one thing that annoyed the hell out of me: the dragon. I mean, everything was manually drawn, all the characters and everything and then there was this huge blatantly computer graphics generated dragon (i.e the Bahamut) that quite astoninglishly was popping out and to hide its computer generated nature what have they done? Well, the oldest trick in the house: put tons of grain on it. I hate it. I understand when movie studios do this, for instance all the very recent Star Wars movie have a lot of grain to cover up the fact that some elements of the spaceships and other things are CGI and that's totally and absolutely fine 'cause you have real characters, real sets, enhanced virtualized sets (i.e they shoot with different cameras with the monitors in the background synced at a multiplier of the framerate of the cameras so that it generates a fake background on which it keeps the field of view, thus extending the set) and then fake computer generated items added in post. In other words, it makes total sense 'cause otherwise it would look plasticky. But for anime? Where nothing is real anyway? Nah. Either do it like Disney where everything is computer drawn with Maya created models which are rigged and then animated or just stick to the good old animation techniques with manually drawn characters, the way animes used to be made. Having something in between as they're doing now is utter nonsense.
if well-made hand-drawn dragons are your thing, let me remind you that Dungeon Meshi (the anime adaptation) exists and the way dragons are drawn there (no grain filters used!) easily kicks the ever-living crap out of every anime that tries to do dragons at all
Mushoku Tensei S01E12 has hand-drawn dragon animation (zero grain too) but it looks very stilted and super-poor on top of as well, wouldn't wish that on my worst enemy

[benwaggoner]

I don't have a calibrated monitor, I can't trust my eyes.
As long as your monitor is reasonably close, you can still trust your eyes better than you can trust metrics.

[benwaggoner]

btw. how does one create film grain files for x265s '--film-grain' option for a given source?
Don't look at the man behind the curtain there!

This feature is in preemptive support for players that can decode HEVC with AVFG1 film grain synthesis metadata. It's essentially the FGS component of AV1 made general, with a fix so that grain size is based on grain size in the content, not display resolution.

I don't think that there is anything out yet that can play such a file. The HEVC will decode, but the FGS metadata will be ignored.

[quietvoid]

This feature is in preemptive support for players that can decode HEVC with AVFG1 film grain synthesis metadata. It's essentially the FGS component of AV1 made general, with a fix so that grain size is based on grain size in the content, not display resolution.

No it's not. It's for the H.265 Film grain characteristics SEI message. Or now it's better to reference it as H.274 FGS.

MCW used to have a libfgm repo on Bitbucket but they recently hid everything except x265.
I had forked it a while back: https://github.com/quietvoid/libfgm
It can be used to generate the expected binary file.

[Boulder]

No it's not. It's for the H.265 Film grain characteristics SEI message. Or now it's better to reference it as H.274 FGS.

MCW used to have a libfgm repo on Bitbucket but they recently hid everything except x265.
I had forked it a while back: https://github.com/quietvoid/libfgm
It can be used to generate the expected binary file.

Does the feature work with decoders out of the box, or is it some extension that is not mandatory for compliance?

[quietvoid]

Does the feature work with decoders out of the box, or is it some extension that is not mandatory for compliance?

It's not mandatory as it's SEI.
I don't know of hardware that supports it.

FFmpeg does support it in software, and libplacebo in shaders.

[vadlerg]

Made a build of new version of my custom mod. Also add an AVX LLVM version, don't remember who asked for it...

Thanks :)!

[benwaggoner]

No it's not. It's for the H.265 Film grain characteristics SEI message. Or now it's better to reference it as H.274 FGS.
It can be used to generate the expected binary file.
Oh, I haven't heard much about that yet. What's the intended use case?

[quietvoid]

Oh, I haven't heard much about that yet. What's the intended use case?

Film grain synthesis in MPEG codecs. It's been specified for decades and is still barely seeing any use.

[benwaggoner]

Film grain synthesis in MPEG codecs. It's been specified for decades and is still barely seeing any use.
Oh, the old optional one from HD-DVD? I haven't looked at it in AGES, but I remember it not being very capable.

AVFG1 is a much better FGS implementation.

[LigH]

Sounds like the next more complex model beyond "noise reduction" (--nr-[inter|intra]) in x265, which was more a modelling than a simple reduction too.

[FranceBB]

Hey guys, I know most of you saw the screenshots of the 56c/112th and other Intel Xeon monsters CPU I use at work, but you should know that at home I'm still using an old i7 5930K CPU 6c/12th at 3.50GHz which can go up to 4.00GHz.
It served me fairly well over the years, but it's getting older and older and I was thinking about buying a new one.
Unfortunately, nowadays Intel seems to be focused on releasing CPUs with mixed cores only for consumers so much so only Intel Xeons have the good old normal cores, but I'm too broke to afford one.
This means that I'll probably end up buying another consumer CPU (like the Core Ultra or whatever they named them now that they killed the i7-i9 nomenclature) which has Efficiency cores and Performance cores.
I know that the efficiency ones are low powered, low clock and also miss some instruction set support like they don't have full AVX512 support etc while the Performance cores have a higher clock and full instruction set support (SSE, SSE2, SSSE3, SSE4.1, SSE4.2, AVX, AVX2, AVX512). The idea being that the Efficiency cores are supposed to be used for low intensity background activities like checking your email while the Performance cores are supposed to be used for high intensity activities like... encoding I guess.

So, the question is:

assuming I'm on Windows 11 on a new PC and I fire up my good old x265 BAT to encode from an AVS Script.avs as I've always done, what's gonna happen?
Is it only gonna use the Performance cores?
Is it gonna use both the performance and the efficiency cores in a very bad way so that once a frame is split in the various threads I'll end up with the performance cores waiting for the efficiency cores to finish their piece (i.e threads synchronization) and thus they'll impact performances negatively?
Is everything gonna work smartly automagically as it's handled by the OS (Win11) and I don't have to care about it?


Sorry for the various questions, but this is almost definitely gonna impact my choice of what to buy 'cause depending on the answer and given the prices of the Xeon nowadays, I'm flirting with the idea of moving to AMD.

[guest]

I'm flirting with the idea of moving to AMD.

I have been using AMD's for a lot of years, I still have a 3950X, 5900X, 5950X, and when the 7950X were released I knew I had to get one.

I exclusively use RipBot264, and the 7950X is an encoding beast, BUT, 16C CPU's do have issues with x265 encoding, but due to the "behind the scenes settings" in RipBot, this can be easily overcome.

But then the 13th Gen Intel's came out, so took a big chance and got a 13900KF, I wasn't sure how the different architecture would work, but after some time fiddling around, I'm more than happy with it.

In a head to head encode, the 7950X is just slightly faster...I would think that the 14th Gen Intel's maybe a little faster.

And to just sum up, you could build an Intel system quite a bit cheaper that an AMD, DDR4 vs DDR5, but one thing that IS very important, you WILL need a really good cooling system, as both run really hot, but they are stable as.

PS:- No AVX512 support on the Intel's :( Hadn't heard of the Core Ultra 'til your post.

https://www.intel.com/content/www/us/en/products/docs/processors/core-vs-ultra.html

[benwaggoner]

Sounds like the next more complex model beyond "noise reduction" (--nr-[inter|intra]) in x265, which was more a modelling than a simple reduction too.
it's about grain reconstruction based on metadata, ala AV1's Film Grain Synthesis. But a very old basic version originally designed for H.264. The only platform that supported it IIRC was HD-DVD, but AFAIK it wasn't ever used on actual discs. People thought it was limited at best back then.

Perhaps coupled with modern de-grain and parameterization technology, maybe?

I think AVFG1 is more likely to be important.

[benwaggoner]

PS:- No AVX512 support on the Intel's :( Hadn't heard of the Core Ultra 'til your post.

https://www.intel.com/content/www/us/en/products/docs/processors/core-vs-ultra.html
AVX512 hasn't been all that helpful for x265 outside of 4K resolutions at slower presets. It's probably not a big loss.

[qyot27]

assuming I'm on Windows 11 on a new PC and I fire up my good old x265 BAT to encode from an AVS Script.avs as I've always done, what's gonna happen?
Is it only gonna use the Performance cores?
Is it gonna use both the performance and the efficiency cores in a very bad way so that once a frame is split in the various threads I'll end up with the performance cores waiting for the efficiency cores to finish their piece (i.e threads synchronization) and thus they'll impact performances negatively?
Is everything gonna work smartly automagically as it's handled by the OS (Win11) and I don't have to care about it?
If it's anything like macOS, the efficiency cores are executed second.

Not using x265, but an example of the core topology is included in this post from 2022 (https://forum.doom9.org/showpost.php?p=1974382&postcount=2111). Short answer: if Intel's heterogenous architectured CPUs act the same way in conjunction with the thread scheduler in Linux and/or Windows, just use Prefetch() to target the Performance cores if you don't want them mixing.

[excellentswordfight]

assuming I'm on Windows 11 on a new PC and I fire up my good old x265 BAT to encode from an AVS Script.avs as I've always done, what's gonna happen?
Is it only gonna use the Performance cores?
Is it gonna use both the performance and the efficiency cores in a very bad way so that once a frame is split in the various threads I'll end up with the performance cores waiting for the efficiency cores to finish their piece (i.e threads synchronization) and thus they'll impact performances negatively?
Is everything gonna work smartly automagically as it's handled by the OS (Win11) and I don't have to care about it?
I've been encoding on Intels CPUs pretty since they introduced the hyrid architecture. And yes, that pretty much the case. From a performance perspective everything is pretty much just working, I have played with turning of e-cores etc, but I gain nothing from it, there is a net profit in using the e-cores. And as the e-cores have better performance per watt, in cases were you are power restricted (which is always now days) and can saturate the extra threads you will get more performance of having say 4 e-cores, compared to having 1-2 extra p-cores, or giving existing p-cores more of the power-budget.

With that said, its not all smooth sailing, windows does weird shit with its scheduler. For example, if you minimize the application you are encoding with (this is at least the case when running applications through CMD) that application will no longer use p-cores AT ALL, as windows now treats it has a "background application". This can be enjoying, and even a dealbreaker as it can also be triggered just by running something in fullscreen infront of it. But I actually use it as a feature, cause if I wanna game while encoding i just minize it will free up the p-cores completely.

With that said, for just a straight up encoding machine, if we are talking about consumer CPUs I would just get a 7000-series Ryzen instead given the huge performance/w advantage in this performance class (Intel is actually not that bad in this regards if you limit your CPU at say 125W and below) and there you really dont have to care about any hybrid core stuff. Ignoring downclocking and powerlimits etc, out of the box, 7950X gets 90% of 14900k encoding performance at 1/2 of the power concumption.

[DMD]

AVX512 hasn't been all that helpful for x265 outside of 4K resolutions at slower presets. It's probably not a big loss.
I apologize but maybe that is why in the latest release of Patman86 the avx512 version does not appear?
https://github.com/Patman86/x265-Mod-by-Patman/releases

Thank you

[tormento]

AVX512 hasn't been all that helpful for x265 outside of 4K resolutions at slower presets. It's probably not a big loss.
I'd be curious to see AVX2 vs AVX512 performances of a very noisy source in very slow/placebo preset on modern CPUs.

Unfortunately the choice for Intel is now limited to 11 series and Xeons.

I need to decide if to go with Arrow Lake or Zen 5.

Not having AVX512 could be a big no no, as a lot of plugins and software that I use support that instruction set.

AVX10 for Arrow Lake is a big enigma, as it's not completely confirmed if it will be 10.1 (no AVX512) or 10.2.

[tormento]

A Highly Parallel and Scalable Motion Estimation Algorithm with GPU for HEVC (https://www.hindawi.com/journals/sp/2017/1431574/)

[rwill]

"Wer schreibt der bleibt"

[guest]

"Wer schreibt der bleibt"

Those who write remain

[rwill]

Yeah, but in the context of the linked paper the correct translation is "Publish or Perish".

Like I skipped through it and it seems to be missing BD-Rate graphs. To quote: "The decrease of the average PSNR is less than 1.5%.". This makes the whole approach questionable and up to worthless.

I mean its nice, I did GPU based Motion Fields as a hobby exercise with CUDA in 2007 or 2008. That some guys seriously present some HEVC ME stuff 10 years later with a complete disregard for video coding, well I better don't comment further.

[excellentswordfight]

I'd be curious to see AVX2 vs AVX512 performances of a very noisy source in very slow/placebo preset on modern CPUs.

Unfortunately the choice for Intel is now limited to 11 series and Xeons.

I need to decide if to go with Arrow Lake or Zen 5.

Not having AVX512 could be a big no no, as a lot of plugins and software that I use support that instruction set.

AVX10 for Arrow Lake is a big enigma, as it's not completely confirmed if it will be 10.1 (no AVX512) or 10.2.
I have access to some 4th gen Xeon SP/Sapphire rapids systems (golden cove cores, same as 12th gen Core/Alder lake), when I get some time I can redo some AVX512 tests.

But in the past, I have never gotten better performance with avx512 with x265, the downclocking it triggers have eaten all potential performance gain. But AFAIK the downclocking of AVX512 load have been improved since its introduction, and was less of an issue on consumer desktop models were frequency control is possible. But given what i've seen, i'm pretty sure that running it with AVX512 produces worse performance/watt.

[vpupkind]

Don't look at the man behind the curtain there!

This feature is in preemptive support for players that can decode HEVC with AVFG1 film grain synthesis metadata. It's essentially the FGS component of AV1 made general, with a fix so that grain size is based on grain size in the content, not display resolution.

I don't think that there is anything out yet that can play such a file. The HEVC will decode, but the FGS metadata will be ignored.

I believe VLC already supports it.
In any case, MCW has an open source library, libfgm, that should be able to both filter and estimate parameters.

[tormento]

But AFAIK the downclocking of AVX512 load have been improved since its introduction, and was less of an issue on consumer desktop models were frequency control is possible. But given what i've seen, i'm pretty sure that running it with AVX512 produces worse performance/watt.
It’s proven that AVX512 has more performance per watt. About downclocking: if you have a powerful heatsink AFAIK you can disable downclocking in bios.

[excellentswordfight]

It’s proven that AVX512 has more performance per watt
Proven? For all systems with all software? Its pretty much a fact that has been worse for my cases; were enterprise-server hardware has been used. These systems are pretty strict in staying within it's power-specification, so when I get worse performance with AVX512 enabled, but it consumes the same amount of power (I have not messured the powerdraw this mind you, but im pretty sure thats the case given tempeture and freqency readings), then it means that it has a negative impact on performance/w. There are software were AVX512 instructions can do wonders, were it boosts performance by a great ammount, and there im sure that it has performance per watts benefits. But for x265, the performance gains of AVX512 are not that big, and at least with older generation hardware performance per watt has been lower, so unless you also increase power it have given me no gains.
About downclocking: if you have a powerful heatsink AFAIK you can disable downclocking in bios.
Yes, and thats also why i wrote:

"...less of an issue on consumer desktop models were frequency control is possible"

I have only tested AVX512 on Xeon/server hardware, and my old ice lake laptop. But as I said, I will try it on Sapphire Rapids, it will be interesting to see how it has been improved, cause first gen, both xeon and consumer (ice-lake) the downclocking rather big.

[nevcairiel]

AVX512 benefits start if you use it well and thoroughly, using it sparsely or inefficiently can cause more overhead then gain.
Blanket statements never make sense with complex optimizations and complex software. Test your own use-case on your own hardware.

[Atak_Snajpera]

Avx512 will be soon abandoned by Intel. AVX10 is a new toy.

[MeteorRain]

I apologize but maybe that is why in the latest release of Patman86 the avx512 version does not appear?
https://github.com/Patman86/x265-Mod-by-Patman/releases

Thank you

That's just a targeted build, which affects everything other than the core part. The core will still run at different instruction set (meaning you should be able to use AVX512 on a sandy bridge build if you wish). Only the C++ code (which is not part of the core parts) will target at a different instruction set.

I'd be curious to see AVX2 vs AVX512 performances of a very noisy source in very slow/placebo preset on modern CPUs.
Unfortunately the choice for Intel is now limited to 11 series and Xeons.
I need to decide if to go with Arrow Lake or Zen 5.
Not having AVX512 could be a big no no, as a lot of plugins and software that I use support that instruction set.
AVX10 for Arrow Lake is a big enigma, as it's not completely confirmed if it will be 10.1 (no AVX512) or 10.2.

AVX512 on different CPUs with different implementations will behave differently. On Zen4 you should generally see slight improvement with AVX512, while on an Intel CPU you'll see some throttling and slow down if you mix AVX512 workload with AVX2 ones.

Zen4 basically runs AVX512 on AVX2 platform, but since some AVX512 instructions are more efficient than their plain AVX2 equivalent, speed sees some improvement.
Intel CPUs runs AVX512 natively, so they do get great improvement. However Intel CPU has that infamous clock throttling so it clocks lower, impacting normal workloads.

It'll end up being a personal preference. AVX10 is a new thing and will take time to adopt. And AVX10 is, well, to just put "AVX512" back to existing product line.

Just my 2 cents.

[tormento]

It'll end up being a personal preference. AVX10 is a new thing and will take time to adopt. And AVX10 is, well, to just put "AVX512" back to existing product line.
AVX 10.1 will be a subset of AVX512. With 10.2 they will embrace and improve AVX512.

[Boulder]

I really wouldn't hold my breath waiting for x265 to get big optimizations regarding any new instruction sets.

[benwaggoner]

AVX512 on different CPUs with different implementations will behave differently. On Zen4 you should generally see slight improvement with AVX512, while on an Intel CPU you'll see some throttling and slow down if you mix AVX512 workload with AVX2 ones.
And the throttling and performance impact varies between different major Intel design versions. The current design performs quite a bit better with AVX512 than the first round of CPUs did.

Zen4 basically runs AVX512 on AVX2 platform, but since some AVX512 instructions are more efficient than their plain AVX2 equivalent, speed sees some improvement.
Intel CPUs runs AVX512 natively, so they do get great improvement. However Intel CPU has that infamous clock throttling so it clocks lower, impacting normal workloads.
Yeah, even if it is the same fused instruction under the hood, AVX512 has smaller instruction size per bit, and so can stay in L3 cache better.

It'll end up being a personal preference. AVX10 is a new thing and will take time to adopt. And AVX10 is, well, to just put "AVX512" back to existing product line.
Has anyone checked the x265 source code to see if many non-AVX10 instructions are being used? I wouldn't be surprised if the AVX10 subset includes most or all of what x265 uses. Video encoding is made of pretty well understood DSP algorithms, and ones that have broad applicability to things like gaming. So I'd expect them to have better odds of making it to a more consumer-focused AVX512 subset.

[excellentswordfight]

Ok, so here is a fresh AVX512 test with the current generation from AMD, unfortunately I only had STEM2 to test with right now, going to see if I can find a more complex/grainy title to retest with later. I will also add test with Sapphire Rapids Xeon next week.

STEM2 2160p24 re-encode @ crf16

AMD Threadripper PRO 7955WX (Storm Peak) 16C/32T @ 350W.

Medium:
Non-AVX512: 15,59fps
AVX512: 15,42fps

slow:
Non-AVX512: 5,85fps
AVX512: 5,83fps

slower:
Non-AVX512: 1,66fps
AVX512: 1,67fps

Frequency for both non and AVX512 load was between 4,65 & 4,75Ghz. So even though I didnt detect any significant downclock speed was pretty much within margin of error (as the load and frequncy is so dynamic and not pinned at 100% run2run diviations will occour). This is actually worse than I expected, given that the downclocking while running avx512 load didnt look like it could have been more than 100MHz, at the same frequency I would expect 5-10% increase or so for preset 'slower', will be interesting to see how grainy/complex content differ at slower.

[Atak_Snajpera]

Ok, so here is a fresh AVX512 test with the current generation from AMD, unfortunately I only had STEM2 to test with right now, going to see if I can find a more complex/grainy title to retest with later. I will also add test with Sapphire Rapids Xeon next week.

STEM2 2160p24 re-encode @ crf16

AMD Threadripper PRO 7955WX (Storm Peak) 16C/32T @ 350W.

Medium:
Non-AVX512: 15,59fps
AVX512: 15,42fps

slow:
Non-AVX512: 5,85fps
AVX512: 5,83fps

slower:
Non-AVX512: 1,66fps
AVX512: 1,67fps

Frequency for both non and AVX512 load was between 4,65 & 4,75Ghz. So even though I didnt detect any significant downclock speed was pretty much within margin of error (as the load and frequncy is so dynamic and not pinned at 100% run2run diviations will occour). This is actually worse than I expected, given that the downclocking while running avx512 load didnt look like it could have been more than 100MHz, at the same frequency I would expect 5-10% increase or so for preset 'slower', will be interesting to see how grainy/complex content differ at slower.

AVX512 in Zen 4 is performed on 2x256bit instead of 2x512bit like in Intel cpus. Zen5 will finally have 2x512bit.

[benwaggoner]

Ok, so here is a fresh AVX512 test with the current generation from AMD, unfortunately I only had STEM2 to test with right now, going to see if I can find a more complex/grainy title to retest with later.
I doubt graininess will have much of an impact on speed. I'm curious if you'll find out otherwise!

I will also add test with Sapphire Rapids Xeon next week.
That should show improvement from older Intel and current AMD. Curious to see if it does, and by how much[/QUOTE]

STEM2 2160p24 re-encode @ crf16

AMD Threadripper PRO 7955WX (Storm Peak) 16C/32T @ 350W.

Medium:
Non-AVX512: 15,59fps
AVX512: 15,42fps

slow:
Non-AVX512: 5,85fps
AVX512: 5,83fps

slower:
Non-AVX512: 1,66fps
AVX512: 1,67fps[/QUOTE]
FWIW, when AVX512 support first came out, MultiCoreWare said it was most likely to show a performance benefit with 4K veryslow 10-bit. On the first Intel AVX2 implementations, with more common encoding parameters it was common to see encoding fps drop with AVX512 turned on.

AMD at least has no regression, so it's presumably safe to have it on by default instead of only for specific use cases.

[benwaggoner]

I really wouldn't hold my breath waiting for x265 to get big optimizations regarding any new instruction sets.
Intel has historically funded new instruction implementation and optimization for both x264 and x265. I imagine we'll see a new round for new instructions if they think it will give them a higher "% improvement from last generation/competition" number.

Video compression has often been the "up to" in "up to X% faster"

[benwaggoner]

AVX 10.1 will be a subset of AVX512. With 10.2 they will embrace and improve AVX512.
I'm not personally deep on this, but that disagrees with what Wikipedia says: https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#AVX10

The first and "early" version of AVX10, notated AVX10.1, will not introduce any instructions or encoding features beyond what is already in AVX-512 (specifically, in Intel Sapphire Rapids: AVX-512F, CD, VL, DQ, BW, IFMA, VBMI, VBMI2, BITALG, VNNI, GFNI, VPOPCNTDQ, VPCLMULQDQ, VAES, BF16, FP16).
Is that missing anything? And is anything missing used in x265?

[Boulder]

Intel has historically funded new instruction implementation and optimization for both x264 and x265. I imagine we'll see a new round for new instructions if they think it will give them a higher "% improvement from last generation/competition" number.

Video compression has often been the "up to" in "up to X% faster"

If Intel is involved, I'd expect them to put their effort on SVT-AV1 which is being developed constantly. x265 is sadly mostly on life support and we have not seen anything really new in a long time.

[tormento]

I'm not personally deep on this, but that disagrees with what Wikipedia says: https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#AVX10


Is that missing anything? And is anything missing used in x265?


I read Intel documentation. Usually it’s more precise than wiki ;)

[tormento]

I doubt graininess will have much of an impact on speed. I'm curious if you'll find out otherwise!
On my poor i7-2600k, grainy material vs denoised one shows sometimes huge impact on bitrate (ofc) and speed.

[Boulder]

The higher the bitrate gets, the more computationally complex things tend to get.

[excellentswordfight]

AVX512 in Zen 4 is performed on 2x256bit instead of 2x512bit like in Intel cpus. Zen5 will finally have 2x512bit.
Im fully aware of that, but there has been quite a bit of tests demonstrating that AVX512 performance on zen4 is actually quite good regardless.

"On average for the tested AVX-512 workloads, making use of the AVX-512 instructions led to around 59% higher performance compared to when artificially limiting the Ryzen 9 7950X to AVX2 / no-AVX512.

From these results I am rather impressed by the AVX-512 performance out of the AMD Ryzen 9 7950X. While initially being disappointed when hearing of their "double pumping" approach rather than going for a 512-bit data path, these benchmark results speak for themselves. For software that can effectively make use of AVX-512 (and compiled so), there is significant performance uplift to enjoy while no negative impact in terms of reduced CPU clock speeds / higher power consumption (with oneDNN being one of the only exceptions seen so far in terms of higher power draw)."

https://www.phoronix.com/review/amd-zen4-avx512/6


I doubt graininess will have much of an impact on speed. I'm curious if you'll find out otherwise!
The higher the bitrate gets, the more computationally complex things tend to get.
Yes, and that was a bit of my reasoning as well; as complex content can pretty much half the encoding speed compared to easier content that it might also shift the bottleneck a bit and maybe favor AVX512 more. But as im not a programmer and have no idea what calculations avx512 instructions are (supposed) to speed up in x265, so this was just a wild theory.

Anyway, tried it with the good ol SVT sequence, straight up half the speed, but the result compared to not using avx512 was identical to the other test. As mentioned Im going to test with Intel as well, but Im pretty sure that avx512 isnt going to do anything there either, and that its simply doesnt do pretty much anything for x265, if the instructions just isnt suited for x265/videoencoding or if it's a poor implementiotion Im gonna leave up for any programmer to decide, but as a user, I probably continue not to bother with it. Cause even though it was minor, you are right benwaggoner, for preset medium and slow I always saw AVX512 to be slower, and this is without any downclocking! Its was minor/negligible mind you, but still. If that is the case for veryslow? Dunno, dont care as I never use it, and see very little rational for its use cases, I barley go down to slower, even at work.

[excellentswordfight]

As mentioned, Intel test as well.

Intel Xeon Gold 6426Y (Sapphire Rapids) 16C/32T

Medium:
Non-AVX512: 11,72fps
AVX512: 11,87fps

slow:
Non-AVX512: 4,17fps
AVX512: 4,49fps

slower:
Non-AVX512: 1,15fps
AVX512: 1,22fps

So, yes, there has been some major improvements on avx512 load from Intel, I actually didnt see any downclocking at all! So its looks like that part is finally solved on Intel cpus as well. And without downclocking I saw the 5-10% improvement that I was expecting on the AMD side without downclocking. So it does indeed look like AMD is hurt by the "double pumping" of the avx512 load used in this scenario.

So, it looks like for sapphire rapids (and newer), you might wanna run with avx512, with that said, pretty sure apples to apples, AMD will outperform Intel regardless of avx512.

[kurkosdr]

it's about grain reconstruction based on metadata, ala AV1's Film Grain Synthesis. But a very old basic version originally designed for H.264. The only platform that supported it IIRC was HD-DVD, but AFAIK it wasn't ever used on actual discs. People thought it was limited at best back then.

Perhaps coupled with modern de-grain and parameterization technology, maybe?

I think AVFG1 is more likely to be important.
I am wondering, is it possible to get AVFG1 implemented on not just H.265 encoders and decoders but also H.264 encoders and decoders? (I don't see why not) I am one of those people who think that film grain should always be removed at post-production in the same way that interlaced scenes that are added to a progressive stream should always be de-interlaced. Film grain was typically removed when authoring DVDs, the whole idea of leaving film grain in was a flex of HD-DVD and Blu-Ray to advertise their large capacity (aka average bitrates) compared to AVCHD/BD9/HD-VMD and the then-nascent streaming services.

If film grain can be made into an effect instead of making the encoder scream, I'd be much happier. Yes, even if the effect isn't present on existing H.264 and H.265 decoders. Nobody else than film buffs cares.

[benwaggoner]

I am wondering, is it possible to get AVFG1 implemented on not just H.265 encoders and decoders but also H.264 encoders and decoders? (I don't see why not) I am one of those people who think that film grain should always be removed at post-production in the same way that interlaced scenes that are added to a progressive stream should always be de-interlaced. Film grain was typically removed when authoring DVDs, the whole idea of leaving film grain in was a flex of HD-DVD and Blu-Ray to advertise their large capacity (aka average bitrates) compared to AVCHD/BD9/HD-VMD and the then-nascent streaming services.

If film grain can be made into an effect instead of making the encoder scream, I'd be much happier. Yes, even if the effect isn't present on existing H.264 and H.265 decoders. Nobody else than film buffs cares.
AVFG1 is entirely codec agnostic, and absolutely could be applied to H.264 or any other codec that supports SEI. And the grain removal and parameterization process is also codec agnostic, and takes place on uncompressed frames pre-encoder. The input to the encoder is the detrained frames and the SEI messages to embed. If one is encoding to multiple AVFG1 streams, the grain removal and parameterization would be shared across codecs, all of them getting the same frames and SEI stream.

FWIW, HD-DVD did have support for the old H.264 FGS technology mandatory. No mainstream titles used it however, due to technical limitations at the time. Degraining and parameterization were vastly more challenging in 2006 than in 2024. We can spend a couple orders of magnitude more FLOPS/pixel now, and have AI/ML, for which this is a well-suited task.

I never got to play with it enough to determine if the synthesis part was flexibile and high quality enough, though.

[benwaggoner]

Intel Xeon Gold 6426Y (Sapphire Rapids) 16C/32T

Medium:
Non-AVX512: 11,72fps
AVX512: 11,87fps

slow:
Non-AVX512: 4,17fps
AVX512: 4,49fps

slower:
Non-AVX512: 1,15fps
AVX512: 1,22fps

So, yes, there has been some major improvements on avx512 load from Intel, I actually didnt see any downclocking at all! So its looks like that part is finally solved on Intel cpus as well. And without downclocking I saw the 5-10% improvement that I was expecting on the AMD side without downclocking. So it does indeed look like AMD is hurt by the "double pumping" of the avx512 load used in this scenario.
IIRC, the AVX-512 down clocking was internal, and didn't result in the actual CPU frequency dropping. But some internal components wound up running at half clock speed. It took MCW quite some time to figure out what was going on with poor AVX-512 performance, and AVX2 before that.

So, it looks like for sapphire rapids (and newer), you might wanna run with avx512, with that said, pretty sure apples to apples, AMD will outperform Intel regardless of avx512.
I expect that there will be bigger gains with higher resolution, and less or perhaps none with lower resolutions.

[excellentswordfight]

IIRC, the AVX-512 down clocking was internal, and didn't result in the actual CPU frequency dropping. But some internal components wound up running at half clock speed. It took MCW quite some time to figure out what was going on with poor AVX-512 performance, and AVX2 before that.

No, it sure did, and it was pretty aggressive as well, the first skylake-sp models i tried it on dropped from like 2,5Ghz to 1,9Ghz, tanking performance. But it was already vastly improved in ice lake, and pretty much "fixed" now I guess in sapphire rapids.

[benwaggoner]

No, it sure did, and it was pretty aggressive as well, the first skylake-sp models i tried it on dropped from like 2,5Ghz to 1,9Ghz, tanking performance.
Yeah, but the internal SIMD clock speed dropped more in a way that was very challenging to measure. If it was only a 25% drop like above, AVX-512 still would have been a lot faster for anything >256 bits.

[excellentswordfight]

Yeah, but the internal SIMD clock speed dropped more in a way that was very challenging to measure.
Well you stated:

"and didn't result in the actual CPU frequency dropping"

But as I said, it sure did.
https://images.anandtech.com/doci/11616/8180_turbo.png

If it was only a 25% drop like above, AVX-512 still would have been a lot faster for anything >256 bits.
Regardless of if there are any internal SIMD frequency things going on, even the first generation of CPUs with AVX512 support could easily double performance with the right workload (although one could argue that you might as well run that on GPUs), and still x265, even now when you dont have to make up for a lower frequency deficit only see minor performance gains, and to me that simply implies that its simply not a great fit for this workload.

[benwaggoner]

Well you stated:

"and didn't result in the actual CPU frequency dropping"

But as I said, it sure did.
Yes, absolutely. I should have said "reduced in frequency dropping unrelated to the reported CPU frequency"

Regardless of if there are any internal SIMD frequency things going on, even the first generation of CPUs with AVX512 support could easily double performance with the right workload (although one could argue that you might as well run that on GPUs), and still x265, even now only see minor performance gains, and to me that simply implies that its simply not a great fit for this workload.
Encoding is a mix of multiple performance-stressing features. CABAC is all about single-threaded arithmetic performance, for example.

x265 turned out to be more CPU intensive than Intel's then internal "worst case" CPU stress test software, which was pretty surprising to all involved.

[Hellboy.]

I did some tests (slow, crf19, 4K-HDR) with this 3 settings "repeat-headers, aud, hrd", and the quality was exactly the same, the speed was almost the same and the size was almost the same.
I compared some frames using a lot of zoom and there is no different.

So i can leave this 3 settings always enables when encode 1080p, 4K-SDR and 4K-HDR?

Not 100% sure what this settings do but i think they create a more compatible file.
Thanks.

[Emulgator]

They do create a more compatible file satisfying BD restrictions in that regard,
so BD capable devices/software should have no problems if you satisfy their other restrictions as well.

[benwaggoner]

They do create a more compatible file satisfying BD restrictions in that regard,
so BD capable devices/software should have no problems if you satisfy their other restrictions as well.
You need to use --repeat-headers for adaptive streaming as well, especially for HDR to make sure the 2084 metadata goes into each GOP.

[Hellboy.]

So there are no point in use this 3 settings if i don't create a full BD compliant file?

Edit:
Except for --repeat-headers that is needed for HDR.

[benwaggoner]

So there are no point in use this 3 settings if i don't create a full BD compliant file?

Edit:
Except for --repeat-headers that is needed for HDR.
Or if you're making streams for something else that wants them for compatibility. They don't affect the encode itself one way or another.

[LigH]

New upload: x265 3.6+13-0ed28893d (https://www.mediafire.com/file/r2da9fqj12w0fjj/x265_3.6+13-0ed28893d.7z/file)

[Windows][GCC 14.1.0][32/32XP/64 bit] 8bit+10bit+12bit

[Barough]

x265 v3.6+28 (GCC 14.1.0)
https://www.mediafire.com/file/p3zsun9oj3gcutu

[LigH]

Please note that most of the latest patches are related to AArch64 / ARM64 architecture and instruction set. There won't be any relevant improvement for intel x86-64 CPUs.

[benwaggoner]

Is anyone providing builds of x265 for Silicon (ARM) Mac OS yet?

I'm finally around to trying out 3.6 on my laptop, and getting the normal "I am not a developer" issues with Xcode, brew, and lack of downloads.

[jfcarbel]

Ben or others that test x265 with CPUs, looks like AMD soon be released Zen 5 has full 512bit like Intel and TestSpot article is showing some crazy handbrake gains and wonder if attributed Zen 5 with better optimizations and prediction for encoding AVX512 implementation support.

https://www.techspot.com/news/103823-amd-teases-ryzen-7-9700x-performance-ahead-july.html

Would love to see if the Zen 5 is really that much impact on x265 encodes with AVX512 vs prior Zen 4.

[FranceBB]

Would love to see if the Zen 5 is really that much impact on x265 encodes with AVX512 vs prior Zen 4.

Yeah I look forward to seeing the new benchmarks as well, but on paper it should almost definitely make the difference.
Real AVX512 has quite literally been the only reason companies stuck with Intel Xeon and dual socket CPUs despite AMD's success in the consumer sector. Now that AMD has implemented true hardware AVX512 instructions sets, I'm really worried about Intel. Their new AMX instructions didn't really find any real world use cases as servers were better off using NVIDIA GPUs directly to make neural network calculations anyway. Intel really is in trouble, they really need to step up their game, but one thing is sure, this is all gonna benefit us as users as it's gonna drive innovation.

[jpsdr]

Stupid question because i think i've read a post about it : Is x265 like x264 and using automaticaly (if CPU support) AVX512 because asm optimized code is automaticaly included, or is there some build option to activate to have AVX512 code included in x265 ?

[Emulgator]

Weighing in the necessary effort I don't think that this had been done in ASM, at least recently.
Compiler option ? Patman has AVX512-enabled builds, so it may well be worth to study his repo.
Just checked: 3.6+13-avx512-msvc1939.
https://github.com/Patman86/x265-Mod-by-Patman/releases/tag/3.6%2B13%2B1

[FranceBB]

I don't know if things changed, but up until very recently AVX512 were included by default but on runtime in the BAT you had to specify

--asm AVX512

That's what I've been doing up until now on the Xeon I have at work.

(For those wondering I'm broke so at home I only have AVX2 'cause Intel didn't love consumers enough).

[LigH]

As long as I remember, AVX512 is not included in the SIMD extensions which get activated automatically when detected because the ratio between additional encoding performance and additional power consumption (often causing a throttling due to heating) is questionable. In other words: Up to AVX2 is enabled when available, but AVX512 needs to be enabled manually, at own risk.

[benwaggoner]

Is anyone providing builds of x265 for Silicon (ARM) Mac OS yet?

I'm finally around to trying out 3.6 on my laptop, and getting the normal "I am not a developer" issues with Xcode, brew, and lack of downloads.
Ach, anyone have a x265 Mac binary, or could make one?

My haven't-written-code-for-money-since-1995 self has spent another three fruitless hours trying to get a high bit depth binary to compile or install on Mac. Homebrew and MacPorts aren't giving me high depth. Trying to build myself yields weird cmake errors I can't make head nor tail of.

[qyot27]

Copied straight from the instructions I drew up for the build I posted before:

Install CMake and Ninja from either Homebrew or MacPorts or however you prefer.

Clone the sources:
git clone https://bitbucket.com/multicoreware/x265_git && \
cd x265_git && \
mkdir -p source/x265-build/{12bit,10bit,8bit}

Build 12-bit:
cd source/x265-build/12bit && \
cmake ../../ -G "Ninja" \
-DHIGH_BIT_DEPTH:bool=on -DMAIN12:bool=on -DENABLE_SHARED:bool=off \
-DENABLE_HDR10_PLUS:bool=on -DEXPORT_C_API:bool=off -DENABLE_CLI:bool=off && \
ninja && \
cp libx265.a $HOME/x265_macos_arm_build/x265_git/lib/libx265_main12.a

Build 10-bit:
cd ../10bit && \
cmake ../../ -G "Ninja" \
-DHIGH_BIT_DEPTH:bool=on -DENABLE_SHARED:bool=off \
-DENABLE_HDR10_PLUS:bool=on -DEXPORT_C_API:bool=off -DENABLE_CLI:bool=off && \
ninja && \
cp libx265.a $HOME/x265_macos_arm_build/x265_git/lib/libx265_main10.a

Build 8-bit and the full CLI:
cd ../8bit && \
cmake ../../ -G "Ninja" -DCMAKE_INSTALL_PREFIX=$HOME/x265_macos_arm_build \
-DENABLE_SHARED:bool=off -DEXTRA_LINK_FLAGS=-L. -DLINKED_10BIT:bool=on -DLINKED_12BIT:bool=on \
-DEXTRA_LIB="$HOME/x265_macos_arm_build/x265_git/lib/libx265_main10.a;$HOME/x265_macos_arm_build/lib/libx265_main12.a" \
-DENABLE_HDR10_PLUS:bool=on && \
ninja && \
ninja install

[FranceBB]

AVX512 is not included in the SIMD extensions which get activated automatically when detected because the ratio between additional encoding performance and additional power consumption (often causing a throttling due to heating) is questionable. In other words: Up to AVX2 is enabled when available, but AVX512 needs to be enabled manually, at own risk.

Correct, hence the --asm avx512 switch needs to be forced in the BAT. The rationale was that in the initial tests, CPUs were getting so hot that thermal throttling came into play and were performing worse than they would have done on AVX2 only. On the other hand, not only new CPUs are much better (it's been a long way since 2019), but in a server room you have very controlled temperatures and even with air cooling you can prevent thermal throttling entirely and get the speed improvement. In other words, it's 2024, I think that rule should be changed and x265 should follow x264 in turning them on by default automatically if they're detected in the CPU.
Speaking of x264, unfortunately AVX512 are available only for the 8bit version but not for the 10bit one which is still limited to AVX2. :(

[jpsdr]

I wasn't clear enough in my question but i have my answer. What i wanted to know is that if a specific switch was needed during the build to include AVX512 code in x265.
If i understand properly, the code is included in the build automaticaly, but you have to activate/select it when running x265, is it correct ?

[Boulder]

I don't recall seeing any such option in CMake, just one for enabling assembly. So I think it's always there but just needs to be enabled by the user.

[benwaggoner]

Copied straight from the instructions I drew up for the build I posted before:

Install CMake and Ninja from either Homebrew or MacPorts or however you prefer.

...
Awesome, thank you! I put that all in a .sh and ran it.

It does the big hard parts by downloading the git and does a lot of cmake, with a lot of version warnings, but still fails, as the /lib/ folder isn't getting made so there's nothing to link from it.

These are the first instances of potentially relevant warnings/errors:

CMake version complaints:
CMake Warning (dev) at CMakeLists.txt:19 (project):
cmake_minimum_required() should be called prior to this top-level project()
call. Please see the cmake-commands(7) manual for usage documentation of
both commands.
This warning is for project developers. Use -Wno-dev to suppress it.

Does this need NUMA for some reason?
CMake Warning (dev) at /opt/local/share/cmake-3.29/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
The package name passed to `find_package_handle_standard_args` (NUMA) does
not match the name of the calling package (Numa). This can lead to
problems in calling code that expects `find_package` result variables
(e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
cmake/FindNuma.cmake:43 (find_package_handle_standard_args)
CMakeLists.txt:115 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.

And a Nasm error. I confirmed it was installed with home-brew:
"CMake Warning (dev) at /opt/local/share/cmake-3.29/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
The package name passed to `find_package_handle_standard_args` (nasm) does
not match the name of the calling package (Nasm). This can lead to
problems in calling code that expects `find_package` result variables
(e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
cmake/FindNasm.cmake:23 (find_package_handle_standard_args)
CMakeLists.txt:415 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it."

And where it finally fails. I confirmed that the /lib/ folder has not been created. I tried running with sudo which didn't change anything.
-- Performing Test CC_HAS_STACK_REALIGN
-- Performing Test CC_HAS_STACK_REALIGN - Success
-- GIT_EXECUTABLE /usr/bin/git
-- GIT LIVE REPO VERSION RETRIEVED
-- X265 RELEASE VERSION 3.6+34-0dfbe6dee
-- Detected CXX compiler using -O3 optimization level
-- The ASM compiler identification is AppleClang
-- Found assembler: /Library/Developer/CommandLineTools/usr/bin/cc
-- Looking for strtok_r
-- Looking for strtok_r - found
-- Looking for include file getopt.h
-- Looking for include file getopt.h - found
-- Configuring done (2.2s)
-- Generating done (0.0s)
-- Build files have been written to: /Users/benwagg/x265_macos_arm_build/x265_git/source/x265-build/8bit
ninja: error: '/Users/benwagg/x265_macos_arm_build/x265_git/lib/libx265_main10.a', needed by 'x265', missing and no known rule to make it

I'm also attaching the full console output as a .rtf.zip

Any suggestions?

[qyot27]

Oh yeah, that's because the instructions I copied from also included building obuparse, L-SMASH, and AviSynth+ for x265-Yuuki, and those would have created that set of install directories before running the cp operation on the 12bit and 10bit runs. I simply forgot about that when I opted not to copy those parts over.

git clone https://bitbucket.com/multicoreware/x265_git && \
cd x265_git && \
mkdir -p source/x265-build/{12bit,10bit,8bit}
should be
git clone https://bitbucket.com/multicoreware/x265_git && \
cd x265_git && \
mkdir -p source/x265-build/{12bit,10bit,8bit} && \
mkdir -p $HOME/x265_macos_arm_build/x265_git/lib
(technically, you could combine both mkdir commands into one, but this way it's clearer that it's creating a second directory in a different location)


Considering nasm is explicitly for x86(-64) assembly, it's not relevant on ARM. It probably shouldn't even be checking for it once it knows it's building for AArch64.

[FranceBB]

If i understand properly, the code is included in the build automaticaly, but you have to activate/select it when running x265, is it correct ?

Yep. Correct. ;)

[Boulder]

Does anyone know if the -12 limit for chroma offsets is just "a number" or would it hurt to extend the range? As we know, x265 sometimes tends to mangle chroma quite a lot even with the largest possible negative offset so it would be interesting to test a bigger value. With svt-av1, I noticed that it requires offsets like -32 to allow the chroma enough bits. Of course, different encoders and probably a different approach, but goes to show that sometimes insanely high (or low :p) values might be needed.

[rwill]

Does anyone know if the -12 limit for chroma offsets is just "a number" or would it hurt to extend the range? As we know, x265 sometimes tends to mangle chroma quite a lot even with the largest possible negative offset so it would be interesting to test a bigger value. With svt-av1, I noticed that it requires offsets like -32 to allow the chroma enough bits. Of course, different encoders and probably a different approach, but goes to show that sometimes insanely high (or low :p) values might be needed.

A cb/cr offset outside of the range +/- 12 is invalid.

I would also like to note that -12 already means around 2 more bits precision in comparison to luma quantization, which is quite a lot.

[Emulgator]

Ah, that coincides well with my findings: wanting to set x265 UHD-BD qcr, qcb more to the negative than with x264 BD, at least to -4.
TMPG's x264 BD presets came out well at -2, and I concur for x264.
Did not think about going as negative for x265 as to -12, but this confirms my guessed-upon x265 approach:
To depart from x264 and offer considerable bitrate savings on paper:
Let's starve bits where the majority does not complain (chroma, grain),
right from the outset, and preprocess/encode that way to cut bandwidth.

So the quality-driven user might have to counteract to some of these defaults.

[LigH]

New upload: x265 3.6+35-83a7df0be (https://www.mediafire.com/file/ny423r44jsm4zpz/x265_3.6+35-83a7df0be.7z/file)

[Windows][GCC 14.1.0][32/32XP/64 bit] 8bit+10bit+12bit

[benwaggoner]

A cb/cr offset outside of the range +/- 12 is invalid.

I would also like to note that -12 already means around 2 more bits precision in comparison to luma quantization, which is quite a lot.
Yeah, I've never seen a value more than -6 be used for anything, and even -1 is sufficient in many cases.

[FranceBB]

New upload: x265 3.6+35-83a7df0be (https://www.mediafire.com/file/ny423r44jsm4zpz/x265_3.6+35-83a7df0be.7z/file)

[Windows][GCC 14.1.0][32/32XP/64 bit] 8bit+10bit+12bit

My XP says "thank you". :P

[tormento]

Ah, that coincides well with my findings: wanting to set x265 UHD-BD qcr, qcb more to the negative than with x264 BD, at least to -4.
TMPG's x264 BD presets came out well at -2, and I concur for x264.
Did not think about going as negative for x265 as to -12, but this confirms my guessed-upon x265 approach:
To depart from x264 and offer considerable bitrate savings on paper:
Let's starve bits where the majority does not complain (chroma, grain),
right from the outset, and preprocess/encode that way to cut bandwidth.

So the quality-driven user might have to counteract to some of these defaults.


Much better you do some VMAF measurement and report us [emoji56]

[benwaggoner]

Much better you do some VMAF measurement and report us [emoji56]
VMAF is a luma-only metric, so doesn't pick up any impact of chroma offsets.

Optimizing for VAMF would tend to push the offsets to high positive values in order to save bits to reduce luma QP.

[Boulder]

I ran into some interesting experimental x265 modifications, especially the cutree-strength part is something that could be used to avoid the detail/grain loss which cutree may cause.

Based on my initial tests, running a normal encode using aq-auto 6 (jpsdr's mod part for HDR) results in a much smoother image than in the vanilla jpsdr mod. This can be reverted nicely with --limit-aq1. Furthermore, --cutree-strength 1.5 would make cutree have the same weight it has when qcomp is raised to 0.7 from the default 0.6. (Cutree's weight is 5.0 * (1.0 - qcomp) by default in x265). I think some users have battled the too strong cutree by raising qcomp and also adjusting CRF to compensate the increase in bitrate it inevitably causes. With this mod, cutree can be tamed without doing anything else.

I'm not at all sure if any of the other stuff is working or not, but those mentioned seem fine.

This is the commit:
https://github.com/AmusementClub/x265/commit/87155154d789d3e2faf323dcb46db7fa36837672

This file can be applied on top of the current x265 repo to get the changes in. Hope I didn't make any mistakes :o
Patch file for jpsdr's and AmusementClub's modifications to use with MABS (https://drive.google.com/file/d/1T25ZdsryaE-aqVNgWTznigwNLDQy1qvJ/view?usp=drive_link)
Building with Visual Studio won't work, the dreaded "nested too deeply" problem strikes again and I'm not confident on how to fix it.

[BuccoBruce]

You need to use --repeat-headers for adaptive streaming as well, especially for HDR to make sure the 2084 metadata goes into each GOP.

Huh. I've only briefly played with DASH, all SDR, and I always just used --idr-recovery-sei...

Tangent warning: most web/JS DASH/HLS players out there, or I guess the way Chromium based browsers implement ffmpeg, really do NOT like open GOP content. Bizarrely enough if you DASH some crazy long GOP open GOP stuff and try and play it back in mpv or VLC...it works fine. Not that you should, I understand why open GOP shouldn't be used for streaming at least on a technical level.
Closed GOP + RADL instead and still managed to get away with using scene-cut/GOP of differing lengths as long as the keyframe interval from the highest encode on the ladder got used as is for all the others. Then I realized most of the clients were all accessing the highest quality stream and rarely if ever accessed the streams lower on the ladder unless their browser started "lower" and then ramped right up...a handful of hits showing up on the server for "1000k_blah_blah_init.mp4" and maybe the first half dozen segments and then...nothing but traffic on the 6-12 Mbps stuff :p so now any of my "adaptive streaming" stuff is just the one encode :D

Have I been doing it wrong, or would I need "--repeat-headers" there too? Am I even gaining anything by using --idr-recovery-sei...

Tangent 2: Chrome itself adding HEVC support has been delightful for DASH!

[benwaggoner]

Huh. I've only briefly played with DASH, all SDR, and I always just used --idr-recovery-sei...

Tangent warning: most web/JS DASH/HLS players out there, or I guess the way Chromium based browsers implement ffmpeg, really do NOT like open GOP content. Bizarrely enough if you DASH some crazy long GOP open GOP stuff and try and play it back in mpv or VLC...it works fine. Not that you should, I understand why open GOP shouldn't be used for streaming at least on a technical level.
specifically, Open GOP doesn't work with adaptive streaming. If there aren't other streams that need to be seamlessly switched, open GOP is fine. And was commonly used in the pre-http adaptive streaming era.

Closed GOP + RADL instead and still managed to get away with using scene-cut/GOP of differing lengths as long as the keyframe interval from the highest encode on the ladder got used as is for all the others. Then I realized most of the clients were all accessing the highest quality stream and rarely if ever accessed the streams lower on the ladder unless their browser started "lower" and then ramped right up...a handful of hits showing up on the server for "1000k_blah_blah_init.mp4" and maybe the first half dozen segments and then...nothing but traffic on the 6-12 Mbps stuff :p so now any of my "adaptive streaming" stuff is just the one encode :D
RADL is pretty much there to give Open GOP like encoding efficiency boost and interfragment consistency while still preserving seamless switching.

And absolutely, if occasionally buffering is okay when using mobile data and the top bitrate isn't lots of Mbps, a single bitrate can work just fine, and certainly simplifies a lot. It's a rare household in developed countries that can't sustain 4 Mbps almost all the time.

And among other things, only providing a single, good bitrate will force suboptimal player heuristics to start at high quality instead of slowly ramping up.

Have I been doing it wrong, or would I need "--repeat-headers" there too? Am I even gaining anything by using --idr-recovery-sei...

--repeat-headers is needed for HDR so that the metadata gets repeated every GOP, but generally isn't needed for SDR. I don't think there's a downside to including it, though. Bitrate impact is minimal and may make it easier to start playback mid-stream. I don't think --idr-recovery-sei does anything with DASH-like http delivery; it would be more for UDP streaming.

Tangent 2: Chrome itself adding HEVC support has been delightful for DASH!
Agreed! The number of H.264-only client devices is shrinking enormously. Essentially all of mobile supports HEVC now, and it's only very old PC/Mac systems that don't support at least software decoding of HEVC, and the vast majority support full HW and DRM decoding.

[FranceBB]

--repeat-headers is needed for HDR so that the metadata gets repeated every GOP, but generally isn't needed for SDR. I don't think there's a downside to including it, though. Bitrate impact is minimal and may make it easier to start playback mid-stream.

Yep, I always include it anyway in the rare cases in which I have to make end-users encodes (for both SDR and HDR contents), so I don't think there's any harm to it.


The number of H.264-only client devices is shrinking enormously. Essentially all of mobile supports HEVC now, and it's only very old PC/Mac systems that don't support at least software decoding of HEVC, and the vast majority support full HW and DRM decoding.

Can someone please tell this to the BBC, itv and Channel 4? Please? xD

[benwaggoner]

Yep, I always include it anyway in the rare cases in which I have to make end-users encodes (for both SDR and HDR contents), so I don't think there's any harm to it.

Can someone please tell this to the BBC, itv and Channel 4? Please? xD
Ah, but most is not all, and if you only want to use one codec on any player, H.264 is really the only choice. It may still be the only choice a decade from now.

We can make some statements, like HEVC is available on all players that support X level of hardware DRM, and HEVC would be safe to have as the only codec in those cases.

[LigH]

Just building: Latest commit adds support for alpha channel, requires compile option enabled, requires YUVA420 input when alpha processing is enabled.

This might have broken building libavcodec and ffmpeg, though... to be confirmed.
__

New upload: x265 3.6+46-7e38fcc9c (https://www.mediafire.com/file/qjlztku3jp45frv/x265_3.6+46-7e38fcc9c.7z/file)

[Windows][GCC 14.2.0][32/32XP/64 bit] 8bit+10bit+12bit

Alpha Encode Options
===================

.. option:: --alpha

Enable alpha layer encoding support in x265.This option can be enabled
only when ENABLE_ALPHA is set during the make of x265.When enabled
--alpha always expects an input file in YUVA420 format.

**CLI_ONLY**

[LigH]

Just building: Latest commit adds support for MultiView (a.k.a. stereo) video streams (separate MVC or H/V SBS), requires compile option enabled, configuration via text file is recommended.
__

New upload: x265 3.6+56-3a802d9d1 (https://www.mediafire.com/file/ppn674zgtjfywaq/x265_3.6+56-3a802d9d1.7z/file)

[Windows][GCC 14.2.0][32/32XP/64 bit] 8bit+10bit+12bit

Multiview Encode Options
===================

Enable multiview encoding support in x265.This option can be enabled only when ENABLE_MULTIVIEW is set during the make of x265.

.. option:: --num-views <integer>
Specify the number of views in the multiview input video.

.. option:: --format <integer>
Specify the format of the input video
0 : Two separate input videos
1 : One input video with both views in left and right format
2 : One input video with both views in top and bottom format

.. option:: --multiview-config <filename>
File containing the configurations to enable multiview encoding.

Sample config file::

--num-views 2
--format 0
--input multiview-input-01.yuv
--input multiview-input-02.yuv

Other input parameters such as input-csp/input-depth/input-res/fps must be configured through normal CLI and is expected to be same for all views

**CLI_ONLY**

[ShortKatz]

Just building: Latest commit adds support for MultiView (a.k.a. stereo) video streams (separate MVC or H/V SBS), requires compile option enabled, configuration via text file is recommended.
__

New upload: x265 3.6+56-3a802d9d1 (https://www.mediafire.com/file/ppn674zgtjfywaq/x265_3.6+56-3a802d9d1.7z/file)

[Windows][GCC 14.2.0][32/32XP/64 bit] 8bit+10bit+12bit

Cool, that x265 has now also support for MV-HEVC, ffmpeg is also working on it.
Did you also enable alpha layer encoding support in your build with ENABLE_ALPHA?

[LigH]

Obviously, yes. Both sets of new CLI options are reported in the help output.

[Barough]

x265 v3.6+46
Built on August 08, 2024, GCC 14.1.0

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

DL :
https://www.mediafire.com/file/b3pdvab5olo4nbw

[LigH]

Just building: Latest commit adds support for Screen Content Coding (https://hevc.hhi.fraunhofer.de/scc), requires compile option enabled

The Screen Content Coding (SCC) extensions will improve compression capability for video containing a significant portion of rendered (moving or static) graphics, text, or animation rather than (or in addition to) camera-captured video scenes. Example applications include wireless displays, remote computer desktop access, and real-time screen sharing for videoconferencing.
__

New upload: x265 3.6+65-3c3d03746 (https://www.mediafire.com/file/5zoll6p8s1hvdnn/x265_3.6+65-3c3d03746.7z/file)

[Windows][GCC 14.2.0][32/32XP/64 bit] 8bit+10bit+12bit
-DENABLE_HDR10_PLUS=ON -DENABLE_LIBVMAF=ON -DENABLE_ALPHA=ON -DENABLE_MULTIVIEW=ON -DENABLE_SCC_EXT=ON

Screen Content Coding Options
===================
.. option:: --scc <integer>

Enable screen content coding support in x265. Default disabled.
This option can be enabled only when ENABLE_SCC_EXT is set during the make of x265.
SCC enables intrablockcopy in CTU analysis which can be enabled in two modes [1-2].
Mode 1- Does limited search,faster than mode 2
Mode 2- Does full and exhaustive search

Note : Enabling SCC will disable weight prediction.
Enabling SCC expects rd-level of 6.

**CLI_ONLY**

[ShortKatz]

What's going on all of a sudden? First it looked for years as if x265 had fallen asleep and now one new feature after another is being added in no time at all. Astonishing.

[ZORAX]

Hi. I've been doing some encoding tests with x265 for a very grainy movie (Saving Private Ryan) and found some odd behavior of nr-inter parameter:

It seems the nr-inter filter has zero effect on I frames, a very small effect on P frames and a HUGE effect on B frames. This makes for a very weird frame strobing effect where I frames look sharp and grainy, B frames look smooth (even if I set ipratio and pbratio to 1.0) and grain seems to be "flickering" on the image.

Has anyone else experienced this? Is this an expected behavior of the encoder?

[jpsdr]

@LIGHT
What -DENABLE_LIBVMAF=ON is for ?

Didn't see the SCC... I have to redo all the builds... :(

[LigH]

There is no T in LigH ...

VMAF (Video Multi-Method Assessment Fusion (https://en.wikipedia.org/wiki/Video_Multimethod_Assessment_Fusion)) is a video-encoding quality metric which tries to be more similar to a human's subjective impression than PSNR and SSIM.

If enabled, x265 adds VMAF scores to the output of the -r / --recon debugging feature which compares the input video with a reconstructed video (decoding a just encoded video stream).

An objectively measurable "Quality" term is always defined as any kind of difference between an original and a reconstructed video. "Best quality" only means "no measurable difference". This may not mean the same as a casual subjective "quality" term for humans.

[rwill]

Hi. I've been doing some encoding tests with x265 for a very grainy movie (Saving Private Ryan) and found some odd behavior of nr-inter parameter:

It seems the nr-inter filter has zero effect on I frames, a very small effect on P frames and a HUGE effect on B frames. This makes for a very weird frame strobing effect where I frames look sharp and grainy, B frames look smooth (even if I set ipratio and pbratio to 1.0) and grain seems to be "flickering" on the image.

Has anyone else experienced this? Is this an expected behavior of the encoder?

This is expected. nr-inter has zero effect on I frames because I frames do not have inter. Try --nr-intra instead to neuter I frames as well.

[jpsdr]

Sorry for the mispelling, i was a little to quick, and thanks for the answer.

[Jamaika]

lowpassdct.cpp: In function 'void lowPassDct8_c(const int16_t*, int16_t*, intptr_t)':
lowpassdct.cpp:61:23: warning: right shift count is negative [-Wshift-count-negative]
61 | dst[0] = totalSum >> (-1 + (X265_DEPTH - 8));
| ~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~

[Jamaika]

Just building: Latest commit adds support for alpha channel, requires compile option enabled, requires YUVA420 input when alpha processing is enabled.
This project is a bit unfinished.
Why is X265_CSP_BGRA not used?
Error: Alpha encode supported only with CQP mode???
Error: chroma subsampling must be i400 (4:0:0 monochrome), i420 (4:2:0 default), i422 (4:2:0), i444 (4:4:4)???
#if X265_BUILD >= 210
if (avctx->pix_fmt == AV_PIX_FMT_BGRA) {
ctx->params->bEnableAlpha = 1;
ctx->params->internalCsp = X265_CSP_BGRA;
} else if (avctx->pix_fmt == AV_PIX_FMT_YUVA444P ||
avctx->pix_fmt == AV_PIX_FMT_YUVA444P10LE ||
avctx->pix_fmt == AV_PIX_FMT_YUVA444P12LE) {
ctx->params->bEnableAlpha = 1;
ctx->params->internalCsp = X265_CSP_I444;
} else {
ctx->params->bEnableAlpha = 0;
ctx->params->internalCsp = X265_CSP_I444;
}
#else
ctx->params->internalCsp = X265_CSP_I444;
#endif
break;
// 4:2:0, 4:2:2
case 1:
#if X265_BUILD >= 210
if (avctx->pix_fmt == AV_PIX_FMT_YUVA420P ||
avctx->pix_fmt == AV_PIX_FMT_YUVA420P10LE ||
//avctx->pix_fmt == AV_PIX_FMT_YUVA420P12LE ||
avctx->pix_fmt == AV_PIX_FMT_YUVA422P ||
avctx->pix_fmt == AV_PIX_FMT_YUVA422P10LE ||
avctx->pix_fmt == AV_PIX_FMT_YUVA422P12LE) {
ctx->params->bEnableAlpha = 1;
} else {
ctx->params->bEnableAlpha = 0;
}
#endif
ctx->params->internalCsp = desc->log2_chroma_h == 1 ?
X265_CSP_I420 : X265_CSP_I422;
break;

[LigH]

This project is a bit unfinished.

Do not blame me. The developer is Anusuya Kumarasamy.

[Jamaika]

Do not blame me. The developer is Anusuya Kumarasamy.
I don't blame you. You posted info. Previously x265 creators looked here.

[Boulder]

The mailing list is probably the best place to report any issues. Too bad they don't seem to care about the issue tracker.

[LigH]

The mailing list is probably the best place to report any issues.

I can only agree here.

[benwaggoner]

This is expected. nr-inter has zero effect on I frames because I frames do not have inter. Try --nr-intra instead to neuter I frames as well.
Exactly.

-nr-inter does an adaptive deadzone like lowpass filtering of low amplitude high frequency elements in predicted blocks, and helps a lot with grain as it is entirely temporally random. --nr-intra does the same for intra-coded blocks.

It's not really a "noise reduction" filter in the classic sense, as it filters out the noise that's the hardest to compress, not that's the most visible.

As a rule, I find that --nr-intra should be around 25%-50% --nr-inter if you're trying to actually reduce visible noise, not just save some bits or reduce QPs.

[benwaggoner]

This project is a bit unfinished.
Why is X265_CSP_BGRA not used?
Error: Alpha encode supported only with CQP mode???
Error: chroma subsampling must be i400 (4:0:0 monochrome), i420 (4:2:0 default), i422 (4:2:0), i444 (4:4:4)???
These are brand new tools, and I imagine they're far from complete practical implementations yet.

[jpsdr]

Each time i made a build of my custom version, i do a quick test on a very small file, using this:

@echo off

SET E_SRC=%8%1.avs
SET E_DST=%5%1.hevc
SET CHAPTERS=%8%7
SET STAT_FILE=%8%1.stats
SET LOG_FILE_1=%8%1_log_1.txt
SET LOG_FILE_2=%8%1_log_2.txt
SET LOG_FILE_3=%8%1_log_3.txt
SET BITRATE=%2
SET TUNING=%6
SET MCLL=%3
SET MDISPLAY=%4

if %6==film goto :FILM
if %6==FILM goto :FILM
if %6==Film goto :FILM
if %6==none goto :NOTUNE
if %6==NONE goto :NOTUNE
if %6==None goto :NOTUNE
if %6==aucun goto :NOTUNE
if %6==AUCUN goto :NOTUNE
if %6==Aucun goto :NOTUNE

x265_x64 --preset slower --tune %TUNING% --vbv-maxrate 90000 --vbv-bufsize 70000 --bitrate %BITRATE% --stats %STAT_FILE% --level 5.1 --profile main10 --high-tier --level-idc 51 --hist-scenecut --fades --aq-mode 4 --aq-auto 6 --weightb --rc-lookahead 72 --tskip --tskip-fast --no-rect --me hex --subme 2 --b-intra --no-sao --multi-pass-opt-analysis --multi-pass-opt-distortion --video-signal-type-preset BT2100_PQ_YCC -D 10 --max-cll %MCLL% --master-display %MDISPLAY% --hdr10-opt --qpfile %CHAPTERS% --input %E_SRC% --pass 1 -o NUL 2> %LOG_FILE_1%
x265_x64 --preset slower --tune %TUNING% --vbv-maxrate 90000 --vbv-bufsize 70000 --bitrate %BITRATE% --stats %STAT_FILE% --level 5.1 --profile main10 --high-tier --level-idc 51 --hist-scenecut --fades --aq-mode 4 --aq-auto 6 --weightb --rc-lookahead 72 --tskip --tskip-fast --rect --no-amp --me umh --subme 3 --b-intra --no-sao --multi-pass-opt-analysis --multi-pass-opt-distortion --video-signal-type-preset BT2100_PQ_YCC -D 10 --max-cll %MCLL% --master-display %MDISPLAY% --hdr10-opt --qpfile %CHAPTERS% --input %E_SRC% --pass 3 -o NUL 2> %LOG_FILE_2%
x265_x64 --preset slower --tune %TUNING% --vbv-maxrate 90000 --vbv-bufsize 70000 --bitrate %BITRATE% --stats %STAT_FILE% --level 5.1 --profile main10 --high-tier --level-idc 51 --hist-scenecut --fades --aq-mode 4 --aq-auto 6 --weightb --rc-lookahead 72 --tskip --tskip-fast --rect --amp --b-intra --no-sao --scenecut-aware-qp 3 --multi-pass-opt-analysis --multi-pass-opt-distortion --video-signal-type-preset BT2100_PQ_YCC -D 10 --max-cll %MCLL% --master-display %MDISPLAY% --hdr10-opt --qpfile %CHAPTERS% --input %E_SRC% --pass 2 -o %E_DST% 2> %LOG_FILE_3%
goto :FIN

:FILM
x265_x64 --preset slower --vbv-maxrate 90000 --vbv-bufsize 70000 --bitrate %BITRATE% --stats %STAT_FILE% --level 5.1 --profile main10 --high-tier --level-idc 51 --hist-scenecut --fades --aq-mode 4 --aq-auto 6 --weightb --rc-lookahead 72 --tskip --tskip-fast --no-rect --me hex --subme 2 --b-intra --no-sao --deblock -1,-1 --psy-rd 2.5 --psy-rdoq 4 --multi-pass-opt-analysis --multi-pass-opt-distortion --video-signal-type-preset BT2100_PQ_YCC -D 10 --max-cll %MCLL% --master-display %MDISPLAY% --hdr10-opt --qpfile %CHAPTERS% --input %E_SRC% --pass 1 -o NUL 2> %LOG_FILE_1%
x265_x64 --preset slower --vbv-maxrate 90000 --vbv-bufsize 70000 --bitrate %BITRATE% --stats %STAT_FILE% --level 5.1 --profile main10 --high-tier --level-idc 51 --hist-scenecut --fades --aq-mode 4 --aq-auto 6 --weightb --rc-lookahead 72 --tskip --tskip-fast --rect --no-amp --me umh --subme 3 --b-intra --no-sao --deblock -1,-1 --psy-rd 2.5 --psy-rdoq 4 --multi-pass-opt-analysis --multi-pass-opt-distortion --video-signal-type-preset BT2100_PQ_YCC -D 10 --max-cll %MCLL% --master-display %MDISPLAY% --hdr10-opt --qpfile %CHAPTERS% --input %E_SRC% --pass 3 -o NUL 2> %LOG_FILE_2%
x265_x64 --preset slower --vbv-maxrate 90000 --vbv-bufsize 70000 --bitrate %BITRATE% --stats %STAT_FILE% --level 5.1 --profile main10 --high-tier --level-idc 51 --hist-scenecut --fades --aq-mode 4 --aq-auto 6 --weightb --rc-lookahead 72 --tskip --tskip-fast --rect --amp --b-intra --no-sao --deblock -1,-1 --psy-rd 2.5 --psy-rdoq 4 --scenecut-aware-qp 3 --multi-pass-opt-analysis --multi-pass-opt-distortion --video-signal-type-preset BT2100_PQ_YCC -D 10 --max-cll %MCLL% --master-display %MDISPLAY% --hdr10-opt --qpfile %CHAPTERS% --input %E_SRC% --pass 2 -o %E_DST% 2> %LOG_FILE_3%
goto :FIN

:NOTUNE
x265_x64 --preset slower --vbv-maxrate 90000 --vbv-bufsize 70000 --bitrate %BITRATE% --stats %STAT_FILE% --level 5.1 --profile main10 --high-tier --level-idc 51 --hist-scenecut --fades --aq-mode 4 --aq-auto 6 --weightb --rc-lookahead 72 --tskip --tskip-fast --no-rect --me hex --subme 2 --b-intra --no-sao --multi-pass-opt-analysis --multi-pass-opt-distortion --video-signal-type-preset BT2100_PQ_YCC -D 10 --max-cll %MCLL% --master-display %MDISPLAY% --hdr10-opt --qpfile %CHAPTERS% --input %E_SRC% --pass 1 -o NUL 2> %LOG_FILE_1%
x265_x64 --preset slower --vbv-maxrate 90000 --vbv-bufsize 70000 --bitrate %BITRATE% --stats %STAT_FILE% --level 5.1 --profile main10 --high-tier --level-idc 51 --hist-scenecut --fades --aq-mode 4 --aq-auto 6 --weightb --rc-lookahead 72 --tskip --tskip-fast --rect --no-amp --me umh --subme 3 --b-intra --no-sao --multi-pass-opt-analysis --multi-pass-opt-distortion --video-signal-type-preset BT2100_PQ_YCC -D 10 --max-cll %MCLL% --master-display %MDISPLAY% --hdr10-opt --qpfile %CHAPTERS% --input %E_SRC% --pass 3 -o NUL 2> %LOG_FILE_2%
x265_x64 --preset slower --vbv-maxrate 90000 --vbv-bufsize 70000 --bitrate %BITRATE% --stats %STAT_FILE% --level 5.1 --profile main10 --high-tier --level-idc 51 --hist-scenecut --fades --aq-mode 4 --aq-auto 6 --weightb --rc-lookahead 72 --tskip --tskip-fast --rect --amp --b-intra --no-sao --scenecut-aware-qp 3 --multi-pass-opt-analysis --multi-pass-opt-distortion --video-signal-type-preset BT2100_PQ_YCC -D 10 --max-cll %MCLL% --master-display %MDISPLAY% --hdr10-opt --qpfile %CHAPTERS% --input %E_SRC% --pass 2 -o %E_DST% 2> %LOG_FILE_3%

:FIN

If i build with SCC enable, even without using SCC, x265 crash on its initialisation phase of the last pass, before even encode start...:(

I'm not able yet to figure out if it's an issue specific to my custom mods, or if it's an issue with the SCC code.
I don't know how to test a normal build. In theory, i should just remove "--aq-auto 6" in the options, as it's the only one specific to my build, but i have no idea how to feed my file to x265, as actualy it's an avi file, opened in an avs file.
If someone can test my exact same options (just without "--aq-auto 6") on a standard build, feeding a HDR 10bit 4k video (100 frames should be enough).

As i'm not using SCC, for now my solution will just be to build without it.

[Boulder]

You can use avs2yuv64 to pipe into the encoder without Avisynth support. avs2yuv64.exe "script.avs" - | x265.exe ... -o output.hevc -
I think this is the correct build: https://github.com/MasterNobody/avs2yuv/releases/tag/v0.24bm6

[LigH]

I had a little trouble with the last x265 versions complaining about parameters; explicitly using "--input" helped.

[jpsdr]

@Boulder
In that case, what should i put for "--input" ?

[LigH]

Just expand accordingly, I hope (untested yet).

avs2yuv64.exe "script.avs" - | x265.exe ... -o output.hevc --input -

Not sure why you would need that, though, x265 should be able to work with AviSynth scripts.

x265.exe ... -o output.hevc --input script.avs

Ah, you reported a crash. Okay, using a different way of frameserving is an attempt, but from my experience, piping is less reliable.

If x265 is built with libav support (there are patched versions out there), x265 should be able to read a lot of media files directly, AVI files too. So just try if you find one of those:

x265.exe ... -o output.hevc --input video.avi

Or am I outdated? The Yuuki-Asuna patch project seems to be for 5 years.

[Boulder]

@Boulder
In that case, what should i put for "--input" ?

"--input -" should work, - meaning that the input is piped data.

[Boulder]

Not sure why you would need that, though, x265 should be able to work with AviSynth scripts.

Unfortunately no one has submitted those Avisynth or Vapoursynth support patches to mainline so piping is what you need to do if any preprocessing is required.

[LigH]

I probably confused that with x264.

So I created a thread about Modifications of x265 (https://forum.doom9.org/showthread.php?t=185687) to not forget them.

[jpsdr]

@LigH
Your build is a not a custom build, it's directly the x265 code without any patch ?

[LigH]

More or less directly the official git sources. With all officially supported compiler flags enabled. No modifications from 3rd party repos.

[Kurt.noise]

Hi,

I did not look up precisely but what would be the recommended settings nowadays for film & animation contents like we have in the x264 tuning parameters ?

[jpsdr]

Yes !!!! Wonderfull !! It crashed !!! :D
I'm happy, it means the issue is not because of the patches in my version. The version build by LigH also crashed.

First, the Encode_Test.bat file i run:

CALL x265_HDR.bat Test 40000 "9633,2273" "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50)" L:\ film Test_Chapters_x265.txt


The x265_HDR.bat file:

@echo off

SET E_SRC=%8%1.avs
SET E_DST=%5%1.hevc
SET CHAPTERS=%8%7
SET STAT_FILE=%8%1.stats
SET LOG_FILE_1=%8%1_log_1.txt
SET LOG_FILE_2=%8%1_log_2.txt
SET LOG_FILE_3=%8%1_log_3.txt
SET BITRATE=%2
SET TUNING=%6
SET MCLL=%3
SET MDISPLAY=%4

if %6==film goto :FILM
if %6==FILM goto :FILM
if %6==Film goto :FILM
if %6==none goto :NOTUNE
if %6==NONE goto :NOTUNE
if %6==None goto :NOTUNE
if %6==aucun goto :NOTUNE
if %6==AUCUN goto :NOTUNE
if %6==Aucun goto :NOTUNE

avs2yuv64.exe "%E_SRC%" - | x265.exe --input-res 3840x1606 --fps 23.976 --input-depth 10 --input-csp 1 --frames 128 --preset slower --tune %TUNING% --vbv-maxrate 90000 --vbv-bufsize 70000 --bitrate %BITRATE% --stats %STAT_FILE% --level 5.1 --profile main10 --high-tier --level-idc 51 --hist-scenecut --fades --aq-mode 4 --weightb --rc-lookahead 72 --tskip --tskip-fast --no-rect --me hex --subme 2 --b-intra --no-sao --multi-pass-opt-analysis --multi-pass-opt-distortion --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --range limited --chromaloc 2 -D 10 --max-cll %MCLL% --master-display %MDISPLAY% --hdr10-opt --qpfile %CHAPTERS% --input - --pass 1 -o NUL 2> %LOG_FILE_1%
avs2yuv64.exe "%E_SRC%" - | x265.exe --input-res 3840x1606 --fps 23.976 --input-depth 10 --input-csp 1 --frames 128 --preset slower --tune %TUNING% --vbv-maxrate 90000 --vbv-bufsize 70000 --bitrate %BITRATE% --stats %STAT_FILE% --level 5.1 --profile main10 --high-tier --level-idc 51 --hist-scenecut --fades --aq-mode 4 --weightb --rc-lookahead 72 --tskip --tskip-fast --rect --no-amp --me umh --subme 3 --b-intra --no-sao --multi-pass-opt-analysis --multi-pass-opt-distortion --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --range limited --chromaloc 2 -D 10 --max-cll %MCLL% --master-display %MDISPLAY% --hdr10-opt --qpfile %CHAPTERS% --input - --pass 3 -o NUL 2> %LOG_FILE_2%
avs2yuv64.exe "%E_SRC%" - | x265.exe --input-res 3840x1606 --fps 23.976 --input-depth 10 --input-csp 1 --frames 128 --preset slower --tune %TUNING% --vbv-maxrate 90000 --vbv-bufsize 70000 --bitrate %BITRATE% --stats %STAT_FILE% --level 5.1 --profile main10 --high-tier --level-idc 51 --hist-scenecut --fades --aq-mode 4 --weightb --rc-lookahead 72 --tskip --tskip-fast --rect --amp --b-intra --no-sao --scenecut-aware-qp 3 --multi-pass-opt-analysis --multi-pass-opt-distortion --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --range limited --chromaloc 2 -D 10 --max-cll %MCLL% --master-display %MDISPLAY% --hdr10-opt --qpfile %CHAPTERS% --input - --pass 2 -o %E_DST% 2> %LOG_FILE_3%
goto :FIN

:FILM
avs2yuv64.exe "%E_SRC%" - | x265.exe --input-res 3840x1606 --fps 23.976 --input-depth 10 --input-csp 1 --frames 128 --preset slower --vbv-maxrate 90000 --vbv-bufsize 70000 --bitrate %BITRATE% --stats %STAT_FILE% --level 5.1 --profile main10 --high-tier --level-idc 51 --hist-scenecut --fades --aq-mode 4 --weightb --rc-lookahead 72 --tskip --tskip-fast --no-rect --me hex --subme 2 --b-intra --no-sao --deblock -1,-1 --psy-rd 2.5 --psy-rdoq 4 --multi-pass-opt-analysis --multi-pass-opt-distortion --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --range limited --chromaloc 2 -D 10 --max-cll %MCLL% --master-display %MDISPLAY% --hdr10-opt --qpfile %CHAPTERS% --input - --pass 1 -o NUL 2> %LOG_FILE_1%
avs2yuv64.exe "%E_SRC%" - | x265.exe --input-res 3840x1606 --fps 23.976 --input-depth 10 --input-csp 1 --frames 128 --preset slower --vbv-maxrate 90000 --vbv-bufsize 70000 --bitrate %BITRATE% --stats %STAT_FILE% --level 5.1 --profile main10 --high-tier --level-idc 51 --hist-scenecut --fades --aq-mode 4 --weightb --rc-lookahead 72 --tskip --tskip-fast --rect --no-amp --me umh --subme 3 --b-intra --no-sao --deblock -1,-1 --psy-rd 2.5 --psy-rdoq 4 --multi-pass-opt-analysis --multi-pass-opt-distortion --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --range limited --chromaloc 2 -D 10 --max-cll %MCLL% --master-display %MDISPLAY% --hdr10-opt --qpfile %CHAPTERS% --input - --pass 3 -o NUL 2> %LOG_FILE_2%
avs2yuv64.exe "%E_SRC%" - | x265.exe --input-res 3840x1606 --fps 23.976 --input-depth 10 --input-csp 1 --frames 128 --preset slower --vbv-maxrate 90000 --vbv-bufsize 70000 --bitrate %BITRATE% --stats %STAT_FILE% --level 5.1 --profile main10 --high-tier --level-idc 51 --hist-scenecut --fades --aq-mode 4 --weightb --rc-lookahead 72 --tskip --tskip-fast --rect --amp --b-intra --no-sao --deblock -1,-1 --psy-rd 2.5 --psy-rdoq 4 --scenecut-aware-qp 3 --multi-pass-opt-analysis --multi-pass-opt-distortion --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --range limited --chromaloc 2 -D 10 --max-cll %MCLL% --master-display %MDISPLAY% --hdr10-opt --qpfile %CHAPTERS% --input - --pass 2 -o %E_DST% 2> %LOG_FILE_3%
goto :FIN

:NOTUNE
avs2yuv64.exe "%E_SRC%" - | x265.exe --input-res 3840x1606 --fps 23.976 --input-depth 10 --input-csp 1 --frames 128 --preset slower --vbv-maxrate 90000 --vbv-bufsize 70000 --bitrate %BITRATE% --stats %STAT_FILE% --level 5.1 --profile main10 --high-tier --level-idc 51 --hist-scenecut --fades --aq-mode 4 --weightb --rc-lookahead 72 --tskip --tskip-fast --no-rect --me hex --subme 2 --b-intra --no-sao --multi-pass-opt-analysis --multi-pass-opt-distortion --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --range limited --chromaloc 2 -D 10 --max-cll %MCLL% --master-display %MDISPLAY% --hdr10-opt --qpfile %CHAPTERS% --input - --pass 1 -o NUL 2> %LOG_FILE_1%
avs2yuv64.exe "%E_SRC%" - | x265.exe --input-res 3840x1606 --fps 23.976 --input-depth 10 --input-csp 1 --frames 128 --preset slower --vbv-maxrate 90000 --vbv-bufsize 70000 --bitrate %BITRATE% --stats %STAT_FILE% --level 5.1 --profile main10 --high-tier --level-idc 51 --hist-scenecut --fades --aq-mode 4 --weightb --rc-lookahead 72 --tskip --tskip-fast --rect --no-amp --me umh --subme 3 --b-intra --no-sao --multi-pass-opt-analysis --multi-pass-opt-distortion --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --range limited --chromaloc 2 -D 10 --max-cll %MCLL% --master-display %MDISPLAY% --hdr10-opt --qpfile %CHAPTERS% --input - --pass 3 -o NUL 2> %LOG_FILE_2%
avs2yuv64.exe "%E_SRC%" - | x265.exe --input-res 3840x1606 --fps 23.976 --input-depth 10 --input-csp 1 --frames 128 --preset slower --vbv-maxrate 90000 --vbv-bufsize 70000 --bitrate %BITRATE% --stats %STAT_FILE% --level 5.1 --profile main10 --high-tier --level-idc 51 --hist-scenecut --fades --aq-mode 4 --weightb --rc-lookahead 72 --tskip --tskip-fast --rect --amp --b-intra --no-sao --scenecut-aware-qp 3 --multi-pass-opt-analysis --multi-pass-opt-distortion --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc --range limited --chromaloc 2 -D 10 --max-cll %MCLL% --master-display %MDISPLAY% --hdr10-opt --qpfile %CHAPTERS% --input - --pass 2 -o %E_DST% 2> %LOG_FILE_3%

:FIN


I think everyone can reproduce the issue with the last version build with all the options activated.

[LigH]

This file can be applied on top of the current x265 repo to get the changes in. Hope I didn't make any mistakes :o
Patch file for jpsdr's and AmusementClub's modifications to use with MABS (https://drive.google.com/file/d/1T25ZdsryaE-aqVNgWTznigwNLDQy1qvJ/view?usp=drive_link)
Building with Visual Studio won't work, the dreaded "nested too deeply" problem strikes again and I'm not confident on how to fix it.

It did not work.

I guess the problem is that the paths don't match. There are neither "x265-official" nor "x265-jpsdr-v2" directories. I guess I shall only use "a" and "b" as placeholders for original and patched?

Ah, no, I am wrong. There must be a different reason, the patched state gets reversed to the original because something was still active in the background.

[LigH]

I tried to streamline the patch via x265_git_extra.sh but it failed. So I tried to run it manually and it reported several fails and rejects.

[Boulder]

It did not work.

I guess the problem is that the paths don't match. There are neither "x265-official" nor "x265-jpsdr-v2" directories. I guess I shall only use "a" and "b" as placeholders for original and patched?

Ah, no, I am wrong. There must be a different reason, the patched state gets reversed to the original because something was still active in the background.
The command line for the diff doesn't matter as the next two lines tell the source and destination paths. For anyone else wondering how to use it in MABS, this needs to be in x265_git_extra.sh: patch -p 1 -i "$LOCALBUILDDIR/x265-jpsdr_and_kyouko.patch".

I tried to streamline the patch via x265_git_extra.sh but it failed. So I tried to run it manually and it reported several fails and rejects.
Most likely due to the larger changes made recently. I'll have to take a look when I get the proper time for that. I'm doing things in a laboured way instead of forking etc. :D

[LigH]

A patch for ffmpeg has been accepted that adds a version switch to build with newer x265 versions by reverting the yet unsupported parallel encoding of multiple video streams.

[Barough]

x265 v3.6+76
Built on August 16, 2024, GCC 14.1.0

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

DL :
https://www.mediafire.com/file/1u616cvvbh8ufwx

[Guest]

@ jpsdr,

I have been using your 3.60.35 build in RipBot264 for a couple of months now, and I have been using your --aq-auto 1, or is it --auto-aq 1, either way (sorry), and today I updated to your latest 3.60.65 build, and I am getting a warning that there is an extra command (can't remember the exact wording), but it doesn't continue, and I can't find any info on the commands this build uses.

Has anything changed since the previous build ??

Regards

[LigH]

Yes, we discussed before that x265 now seems to be picky about determining what the filename of the input is, so it is advisable to preceed it with "--input".

The reason is probably that MVC encoding accepts multiple input video sources.

[Guest]

Yes, we discussed before that x265 now seems to be picky about determining what the filename of the input is, so it is advisable to preceed it with "--input".

The reason is probably that MVC encoding accepts multiple input video sources.

Hi LigH,

I don't think we're on the same wave length here, it's not a filename, it's an x265 command

--aq-mode 4, 3, 2, 1, or --hevc-aq

[LigH]

So please show us the whole command line with all parameters.

[Guest]

So please show us the whole command line with all parameters.

OK, so as I explained, I have been using jpsdr's x265 build #3.6.0.35, and using this x265 command :-

--level 6.2 --profile main10 --hdr10 --output-depth 10 --ctu 64 --high-tier --repeat-headers --vbv-bufsize 800000 --vbv-maxrate 800000 --asm avx512 --aq-auto 1

But after updating to his latest build #3.6.0.65, using the same command, I get this :-

x265 [WARN]: extra unused command arguments given <->

But now I'm really confused between Patman's & jpsdr's builds...

[LigH]

This is not the complete command line because you omitted the output and the input.

I guess you are using a pipe for the input, hence the "-" as name of the (pseudo) input file. Try to substitute "-" with "--input -", and it will probably work.

[Guest]

This is not the complete command line because you omitted the output and the input.

I guess you are using a pipe for the input, hence the "-" as name of the (pseudo) input file. Try to substitute "-" with "--input -", and it will probably work.

That command line is all RipBot264 needs as far as x265 is concerned, it's doesn't need input & output.

That is done elsewhere...

[LigH]

In this case, RipBot264 needs to become aware that it may have to build the whole command line from this set of additional parameters in a different way, specifying input filenames explicitly from now on. That worked in earlier versions of x265 already, just was not yet mandatory.

[Guest]

In this case, RipBot264 needs to become aware that it may have to build the whole command line from this set of additional parameters in a different way, specifying input filenames explicitly from now on. That worked in earlier versions of x265 already, just was not yet mandatory.

Could you please provide a example of what the new commands "look like", using the command line I posted below ???

Does this apply to all x265 from now on ??

[LigH]

I do not use RipBot264, so I cannot test how to build the set of additional parameters to the command line in a way that a current version of RipBot264 completes it in a way to work with every x265 containing commits after 2024-08-07 when parsing the command line for multiple input sources was introduced. I would instead rather suggest to update RipBot264 to complete its x265 command lines with explicit "--input" parameter instead.

[Guest]

I do not use RipBot264, so I cannot test how to build the set of additional parameters to the command line in a way that a current version of RipBot264 completes it in a way to work with every x265 containing commits after 2024-08-07 when parsing the command line for multiple input sources was introduced. I would instead rather suggest to update RipBot264 to complete its x265 command lines with explicit "--input" parameter instead.

So it's not just adding --input to the commandline, then ??

I don't understand :(

[LigH]

I don't know where in the whole x265 command line which is actually executed the piece you can edit in RipBBot264 is included. Adding the "--input" to the end of this string would only help if RipBot264 adds the "-" for the pipe input immediately after that string. But if it first adds the output file name then it won't help.

After all, it is a RipBot264 issue. Discussing the handling of RipBot264 in this thread about the x265 encoder does not really help anyone. Well, other GUIs using x265 may need to get updated in the same way. But that has to be addressed for those other GUIs. The x265 encoder needed to change its parsing of the command line parameters to support new features, so many applications using x265 need to adapt to these changes.

[Guest]

I don't know where in the whole x265 command line which is actually executed the piece you can edit in RipBBot264 is included. Adding the "--input" to the end of this string would only help if RipBot264 adds the "-" for the pipe input immediately after that string. But if it first adds the output file name then it won't help.

After all, it is a RipBot264 issue. Discussing the handling of RipBot264 in this thread about the x265 encoder does not really help anyone. Well, other GUIs using x265 may need to get updated in the same way. But that has to be addressed for those other GUIs. The x265 encoder needed to change its parsing of the command line parameters to support new features, so many applications using x265 need to adapt to these changes.

Is this the info you're talking about ??

""\\RYZEN-9-7950X\Ripbot264temp\Tools\ffmpeg\bin\ffmpeg.exe" -loglevel panic -i "\\RYZEN-9-7950X\RipBot264temp\job3\Chunks\1.avs" -strict -1 -f yuv4mpegpipe - | "\\RYZEN-9-7950X\Ripbot264temp\tools\x265\x265_x64.exe" --seek 0 --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc
--master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50)" --crf 16 --fps 24000/1001 --min-keyint 24 --keyint 240 --frames 3610 --sar 1:1 --level 5.2 --profile main10
--output-depth 10 --ctu 64 --high-tier --vbv-bufsize 240000 --vbv-maxrate 240000 --y4m --pools "32" --output "\\RYZEN-9-7950X\RipBot264temp\job3\Chunks\1.265" -"

x265 [WARN]: extra unused command arguments given <->

[LigH]

Yes. And as you can see, there is this lonely "-" at the end of the whole command line, but RipBot264 added a few parameters you cannot control manually. RipBot264 needs to insert the "--input" before the "-", you can't do that.

[jfcarbel]

Still very curious if anyone here with Zen 5 9XXX line of AMD CPUs have seen massive encoding speed improvements in x265 with avx512 enabled. Most reviews I bet use handbrake and do not set the x265 flag to turn on avx512 so hard to get good info.

When the 7000 line came out, I compared my 5950X against it and found minimal improvements in encoding speed for price/perfomance ratio.

But if the gains some AMD benchmark ads are showing are true the 9950X might be a speed demon for video encoding and x265.

[Guest]

Still very curious if anyone here with Zen 5 9XXX line of AMD CPUs have seen massive encoding speed improvements in x265 with avx512 enabled. Most reviews I bet use handbrake and do not set the x265 flag to turn on avx512 so hard to get good info.

When the 7000 line came out, I compared my 5950X against it and found minimal improvements in encoding speed for price/perfomance ratio.

But if the gains some AMD benchmark ads are showing are true the 9950X might be a speed demon for video encoding and x265.

I am planning to get a 9950X soon, but I'm not too sure how much better it will be over my 7950X....

I found the 7950X was a LOT faster than my 5950X, and IF the 9950X is the same, then that WILL be great.

I just watched a review on the 9950X, and it was mentioned that it's only a 5% improvement over 7950X, but the 7950X is about 30% better than the 5950X.

I also use the AVX512 command in x265, but not sure if it makes too much difference.

Time will tell.

[RanmaCanada]

I am planning to get a 9950X soon, but I'm not too sure how much better it will be over my 7950X....

I found the 7950X was a LOT faster than my 5950X, and IF the 9950X is the same, then that WILL be great.

I just watched a review on the 9950X, and it was mentioned that it's only a 5% improvement over 7950X, but the 7950X is about 30% better than the 5950X.

I also use the AVX512 command in x265, but not sure if it makes too much difference.

Time will tell.

If you're using AVX512 you will see significant increase, especially so if you use Linux. AMD has stated that the windows scheduler can not handle things properly and we need to wait for the next windows update for things to be "fixed", among other statements...

https://www.phoronix.com/review/amd-ryzen-9950x-9900x/11 From what I understand these tests were not done using avx-512.

[jfcarbel]

I am planning to get a 9950X soon, but I'm not too sure how much better it will be over my 7950X....

I found the 7950X was a LOT faster than my 5950X, and IF the 9950X is the same, then that WILL be great.

I just watched a review on the 9950X, and it was mentioned that it's only a 5% improvement over 7950X, but the 7950X is about 30% better than the 5950X.

I also use the AVX512 command in x265, but not sure if it makes too much difference.

Time will tell.

Thanks for contributing, I am debating upgrading to the 7950X if prices drops some more.

Be interesting to see you run an encode with and without avx to see difference.

[Guest]

Thanks for contributing, I am debating upgrading to the 7950X if prices drops some more.

Be interesting to see you run an encode with and without avx to see difference.

You won't be disappointed if you "only" go for a 7950X (not the 3D), but it's a whole new motherboard, RAM (DDR5), really good cooling IS necessary.

But if you're coming up from a 5950X, you might as well go for the 9950X, it's only a few hundred $ more...it's only money.

You could save a little overall if you get a lower chipset MB.

[excellentswordfight]

Thanks for contributing, I am debating upgrading to the 7950X if prices drops some more.

Be interesting to see you run an encode with and without avx to see difference.
My guess is that it gives about a 10% increase. For zen 4 it gave about 5% improvement with the gimped AVX512 implementation, and latest generation Xeon is at ~10%, so I dont see why the change would give more than 2x uplift from the zen 4 implementation, as well as be much faster than intels implementation.

[jpsdr]

I'm trying to build under msys2 with clang. It "works", meaning it links an exe, but when i run it it's complaining abour missing dll.
Here the script i'm using:

mkdir -p 8bit_Win32 10bit_Win32 12bit_Win32

cd 12bit_Win32
/C/Logiciels/CMake/bin/cmake -G "MSYS Makefiles" ../../../source -DENABLE_LIBVMAF=ON -DENABLE_SCC_EXT=ON -DENABLE_MULTIVIEW=ON -DENABLE_ALPHA=ON -DENABLE_VAPOURSYNTH=ON -DENABLE_AVISYNTH=ON -DHIGH_BIT_DEPTH=ON -DENABLE_HDR10_PLUS=ON -DEXPORT_C_API=OFF -DENABLE_SHARED=OFF -DENABLE_CLI=OFF -DMAIN12=ON -DSTATIC_LINK_CRT=ON -DENABLE_LTO=ON -DCMAKE_CXX_FLAGS_RELEASE="-ffinite-math-only -ftree-vectorize -Ofast -ffast-math -fomit-frame-pointer" -DCMAKE_EXE_LINKER_FLAGS="-static-libgcc -static-libstdc++ -static"
make ${MAKEFLAGS}
cp libx265.a ../8bit_Win32/libx265_main12.a

cd ../10bit_Win32
/C/Logiciels/CMake/bin/cmake -G "MSYS Makefiles" ../../../source -DENABLE_LIBVMAF=ON -DENABLE_SCC_EXT=ON -DENABLE_MULTIVIEW=ON -DENABLE_ALPHA=ON -DENABLE_VAPOURSYNTH=ON -DENABLE_AVISYNTH=ON -DHIGH_BIT_DEPTH=ON -DENABLE_HDR10_PLUS=ON -DEXPORT_C_API=OFF -DENABLE_SHARED=OFF -DENABLE_CLI=OFF -DMAIN12=OFF -DSTATIC_LINK_CRT=ON -DENABLE_LTO=ON -DCMAKE_CXX_FLAGS_RELEASE="-ffinite-math-only -ftree-vectorize -Ofast -ffast-math -fomit-frame-pointer" -DCMAKE_EXE_LINKER_FLAGS="-static-libgcc -static-libstdc++ -static"
make ${MAKEFLAGS}
cp libx265.a ../8bit_Win32/libx265_main10.a

cd ../8bit_Win32
/C/Logiciels/CMake/bin/cmake -G "MSYS Makefiles" ../../../source -DENABLE_LIBVMAF=ON -DENABLE_SCC_EXT=ON -DENABLE_MULTIVIEW=ON -DENABLE_ALPHA=ON -DENABLE_VAPOURSYNTH=ON -DENABLE_AVISYNTH=ON -DHIGH_BIT_DEPTH=OFF -DENABLE_HDR10_PLUS=OFF -DEXPORT_C_API=ON -DENABLE_SHARED=OFF -DENABLE_CLI=ON -DMAIN12=OFF -DSTATIC_LINK_CRT=ON -DENABLE_LTO=ON -DLINKED_10BIT=ON -DLINKED_12BIT=ON -DEXTRA_LIB="x265_main10.a;x265_main12.a" -DEXTRA_LINK_FLAGS=-L. -DCMAKE_CXX_FLAGS_RELEASE="-ffinite-math-only -ftree-vectorize -Ofast -ffast-math -fomit-frame-pointer"
make ${MAKEFLAGS}

# rename the 8bit library, then combine all three into libx265.a using GNU ar
mv libx265.a libx265_main.a

ar -M <<EOF
CREATE libx265.a
ADDLIB libx265_main.a
ADDLIB libx265_main10.a
ADDLIB libx265_main12.a
SAVE
END
EOF

I think it's missing to the linker to links some libraries, but i have no idea what command i should add, and where (only in the last or on all 3).

[LigH]

In my GCC script, for 8 bit, there is -DENABLE_SHARED=ON

[jpsdr]

The following was allready included, not working (or not enough):

-DCMAKE_EXE_LINKER_FLAGS="-static-libgcc -static-libstdc++ -static"


It's to OFF on my scripts, i'll test ON.

EDIT:
Shared to ON not working, still message telling the libc++.dll is missing.

[LigH]

I found a suggestion (https://stackoverflow.com/questions/71991097/clang-static-libstdc-equivalent-for-libc) which may or may not help...

[jpsdr]

Thanks, i'll try.

[jpsdr]

Tested with and without shared, unfortunately, still the message libc++.dll is missing.

[Guest]

Yes. And as you can see, there is this lonely "-" at the end of the whole command line, but RipBot264 added a few parameters you cannot control manually. RipBot264 needs to insert the "--input" before the "-", you can't do that.

I just tested x265 with StaxRip...same thing :(

https://forum.doom9.org/showthread.php?p=2006052#post2006052

[Boulder]

If you don't have an ARM based system, the new versions don't offer anything special compared to the older ones.

[Guest]

If you don't have an ARM based system, the new versions don't offer anything special compared to the older ones.

Despite that, it's still a major problem....if you want to update !!!

[Selur]

using:
x265 [info]: HEVC encoder version 3.6+83-6fa7254
x265 [info]: build info [Windows][GCC 14.2.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2

ffmpeg -y -loglevel fatal -noautorotate -nostdin -threads 8 -i "G:\TestClips&Co\files\test.avi" -map 0:0 -an -sn -color_primaries bt470bg -color_trc smpte170m -colorspace bt470bg -color_range tv -pix_fmt yuv420p10le -strict -1 -vsync 0 -f yuv4mpegpipe - | x265 --input - --fps 25/1 --output-depth 10 --y4m --crf 18.00 --range limited --colormatrix bt470bg --sar 1:1 --output "J:\tmp\test.265"
(same happens when vspipe is used)
never stops,... (ffmpeg does report that it is finished, when removing the '-loglevel fatal' part)
Seems to me like x265 ignores the end signal from the pipe or am I missing something new?
(didn't spot anything in the full help output: https://pastebin.com/LRt1s0Cj)

When aborting x265, I see something like:
aborted at input frame 429, output frame 2765 in x265
So how to limit the output frames automatically to the number of input frames?

Cu Selur

[Kurt.noise]

So how to limit the output frames automatically to the number of input frames?
--frames xxx where xxx are number frames to encode.

[Selur]

Not really what I was hoping for, but I'll make it 'automatic' by using some scripting,.. (worked fine before the latest changes :()

[Kurt.noise]

FrameCount from MediaInfo (https://github.com/MediaArea/MediaInfoLib/blob/abdbb218b07f6cc0d4504c863ac5b42ecfab6fc6/Source/Resource/Text/Stream/Video.csv#L209) might help you to find out this requirement...

[Selur]

For file input sure, but for scripts it's more of a hassle.
In Hybrid I either would have to run the generated scripts through vspipe or avsInfo to get the frame count during job creation or add another job which would analyze the script and adjust later subjobs. :(
Thought about it, but atm. I'm going back to an older version and see whether they fix this for the next release or not.
Seems to be too much trouble atm.

Cu Selur

Ps.: created an issue entry https://bitbucket.org/multicoreware/x265_git/issues/947/cli-y4m-pipe-input-broken-doesnt-stop

[LigH]

In contrast to vspipe and a special version of avs2yuv, ffmpeg doesn't seem to support the Y4M tag "XLENGTH" (https://forum.doom9.org/showthread.php?p=2005576#post2005576) (probably because ffmpeg prefers working as a one-pass filter, avoiding seeks). On the other hand, x265 supports it only with a modification as well.

[Selur]

Haven't tried avs2yuv with y4m output, but when using vspipe (using Vapoursynth R68):
vspipe "J:\tmp\2024-08-26@17_02_59_5110\encodingTempSynthSkript_2024-08-26@17_02_59_5110_0.vpy" - -c y4m | x265 --input - --fps 25/1 --output-depth 10 --y4m --profile main10 --limit-modes --no-early-skip --no-open-gop --opt-ref-list-length-pps --crf 18.00 --opt-qp-pps --cbqpoffs -2 --crqpoffs -2 --limit-refs 0 --ssim-rd --psy-rd 2.50 --rdoq-level 2 --psy-rdoq 10.00 --aq-mode 0 --deblock=-1:-1 --limit-sao --no-repeat-headers --no-mcstf --range limited --colormatrix bt470bg --sar 1:1 --output "J:\tmp\2024-08-26@17_02_59_5110\2024-08-26@17_02_59_5110_02.265"
it runs without end too,...

[LigH]

Only modded x265 recognise the number of frames in a Y4M header's XLENGTH tag without the use of a --frames CLI parameter.

[Selur]

Hmm,.. strange I did build x265 with MBAS as usual, didn't know they changed the way they build x265.

[LigH]

Selur, please read the thread Modifications of x265 (https://forum.doom9.org/showthread.php?p=2005576) where it is explained more verbosely.

[Selur]

I get the patch, but ..
Was this included in the mbas builds before?
I did have to clear my build folder, but I wasn't aware I ever added a patch to x265.
I mean, these calls worked before, so I'm a bit confused.
Did x265 change, did MABS change or have I been using somehow patched x265 builds without knowing it for years on Windows and Linux?

Cu Selur

[LigH]

No, neither of these patches was ever part of MABS. There is only one to split library paths in cmake.

If there are issues now with the recent x265 builds, then they may be caused by the additional features related to multiple inputs. So there should be no important reason to use x265 newer than v3.6+3x unless you want to support specifically these new features (Alpha, MVC).

But it would probably be important to mention that there are issues handling piped input since the last additions. Reproducable tests would help.

[Selur]

Okay, I'll stick to my old builds from end of May (version 3.6+28-4b272f7154) that one worked fine.

[LigH]

I just created a quick test sequence, comparing x265 3.6+[13|35|46|56|65]. Yes, [56|65] encode eternally.
FrameCountTest.avsColorBarsHD()
Info()
Trim(0,99)
I'll write a mail to the x265-devel mailing list.

[Selur]

Thanks for looking into it.

[Guest]

I just created a quick test sequence, comparing x265 3.6+[13|35|46|56|65]. Yes, [56|65] encode eternally.
FrameCountTest.avsColorBarsHD()
Info()
Trim(0,99)
I'll write a mail to the x265-devel mailing list.

I'm not too sure if this is what I commented on about x265 a week or so ago, that it had some different instructions that don't work anymore with RipBot264 and StaxRip...and now even Hybrid.

Something needs to be done, or we'll have to keep using old builds.

[LigH]

For now ... yes, keep using old builds. New features are either not urgently needed by most casual users or don't belong to the most commonly used high-performance platform for the casual user, the intel PC.

[jpsdr]

When i'm building with clang, even trying several options supposed to fix the issue, i'm having the error message of libc++.dll missing.
I've tried to use another gcc than the Win32 version, and same thing, even trying several options supposed to fix the issue, i'm having the error message of libstdc++6.dll missing.

I'm begining to think that maybe i'm not using the proper option.
I'm adding the following in the cmake line (for gcc), but is it correct ?

-DCMAKE_EXE_LINKER_FLAGS="-static-libgcc -static-libstdc++ -static"

Or is it another command i have to put in the cmake line to add the options "-static-libgcc -static-libstdc++ -static" in the compiler ?

[Jamaika]

For gcc options 'static-libgcc', 'static-libstdc++', 'static' doesn't affect object files.
The dllexport function decides whether the object file function is static or not. The final shared function decides about creation of the .dll file.
For static the default is static-libgcc static-libstdc++.

Object files in mingw are affected by ar.exe. This is removal of dead functions. Unfortunately there is no information about what program cuts out if we use it. For example we did not add define XXX so ar.exe deleted unnecessary functions.
Without static function the -L. libraries aren't static.

[jpsdr]

So, for those building x265 without using gcc provided at http://msystem.waw.pl/x265/ what is your script ? Your cmake command line ?

[Jamaika]

The page http://msystem.waw.pl/x265/ has an old toolchain for Windows I don't use cmake, but for me it should be only static. You can do tests yourself to see if the final exe files differ.
I use https://github.com/brechtsanders/winlibs_mingw/releases/tag/11.5.0posix-12.0.0-msvcrt-r1 https://github.com/brechtsanders/winlibs_mingw/releases/download/11.5.0posix-12.0.0-msvcrt-r1/winlibs-x86_64-posix-seh-gcc-11.5.0-mingw-w64msvcrt-12.0.0-r1.zip Is there a problem with compiling the x265 library?
https://bitbucket.org/multicoreware/x265_git/commits/branch/master They depend on whether we expose all warming messages. For clang there may be slightly different functions.
I added Wall Wextra warming and got it for gcc.
threading.h:285:41: warning: comparison of integer expressions of different signedness: 'int32_t' {aka 'int'} and 'DWOR ' {aka 'long unsigned int'} [-Wsign-compare]
285 | return rt != WAIT_TIMEOUT && rt != WAIT_FAILED;
| ^

Lots of bugs in VMAF.

cudata.cpp:2131:7: warning: extra tokens at end of #endif directive [-Wendif-labels]
2131 | #endif;
| ^
cudata.h: In member function 'bool x265::CUData::setQPSubCUs(int8_t, uint32_t, uint32_t)':
cudata.h:272:112: warning: array subscript 5 is above array bounds of 'void (* [5])(uint8_t*, uint8_t)' {aka 'void (* [5])(unsigned char*, unsigned char)'} [-Warray-bounds]
272 | void setQPSubParts(int8_t qp, uint32_t absPartIdx, uint32_t depth) { s_partSet[depth ((uint8_t*)m_qp + absPartIdx, (uint8_t)qp); }
| ~~~~~~~~~~~~~~~
In file included from framedata.h:29,
from cudata.cpp:27:
cudata.h:177:15: note: while referencing 'x265::CUData::s_partSet'
177 | cubcast_t s_partSet[NUM_FULL_DEPTH]; // pointer to broadcast set functions per absolute depth
| ^~~~~~~~~
dpb.cpp: In member function 'void x265::DPB::computeRPS(int, int, bool, x265::RPS*, unsigned int, int)':
dpb.cpp:414:75: warning: comparison of integer expressions of different signedness: 'unsigned int' and 'int' [-Wsign-compare]
414 | if (iterPic->m_param->numViews > 1 && layer && numNeg == iterPic->m_param->maxNumReferences - 1 && (iterPic->m_poc - curPoc) < 0)
| ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
encoder.cpp: In member function 'int x265::Encoder::encode(const x265_picture*, x265_picture**)':
encoder.cpp:2452:29: warning: this 'if' clause does not guard... [-Wmisleading-indentation]
2452 | case 3: if (!IS_REFERENCED(frameEnc[layer]))
| ^~
encoder.cpp:2454:25: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the 'if'
2454 | break;
| ^~~~~
lowpassdct.cpp: In function 'void lowPassDct8_c(const int16_t*, int16_t*, intptr_t)':
lowpassdct.cpp:61:23: warning: right shift count is negative [-Wshift-count-negative]
61 | dst[0] = totalSum >> (-1 + (X265_DEPTH - 8));
| ~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~
----------------------------------------------------------------------------------------------------------------
bool take(const uint32_t time_out = INFINITE)
{
DWORD rt = WaitForSingleObject(m_sem, time_out);
return rt != WAIT_TIMEOUT && rt != WAIT_FAILED;
}
----------------------------------------------------------------------------------------------------------------
#if ENABLE_MULTIVIEW
if (iterPic->m_param->numViews > 1 && layer && numNeg == (unsigned int)iterPic->m_param->maxNumReferences - 1 && (iterPic->m_poc - curPoc) < 0)
{
iterPic = iterPic->m_next;
continue;
}
#endif
----------------------------------------------------------------------------------------------------------------
// replace first coef with total block average
dst[0] = totalSum >> (1 + (X265_DEPTH - 8));

frameencoder.cpp: In member function 'void x265::FrameEncoder::vmafFrameLevelScore()':
frameencoder.cpp:2311:29: error: request for member 'm_fencPic' in '*(x265::Frame**)(&((x265::FrameEncoder*)this)->x265::FrameEncoder::m_frame)', which is of pointer type 'x265::Frame*' (maybe you meant to use '->' ?)
2311 | PicYuv *fenc = m_frame->m_fencPic;
| ^~~~~~~~~
frameencoder.cpp:2312:30: error: request for member 'm_reconPic' in '*(x265::Frame**)(&((x265::FrameEncoder*)this)->x265::FrameEncoder::m_frame)', which is of pointer type 'x265::Frame*' (maybe you meant to use '->' ?)
2312 | PicYuv *recon = m_frame->m_reconPic[0];
| ^~~~~~~~~~
----------------------------------------------------------------------------------------------------------------
#if ENABLE_LIBVMAF
if (m_param->bEnableVmaf == 1)
vmafFrameLevelScore();
#endif
----------------------------------------------------------------------------------------------------------------
#if ENABLE_LIBVMAF
void FrameEncoder::vmafFrameLevelScore()
{
for (int layer = 0; layer < MAX_LAYERS; layer++) {
PicYuv *fenc = m_frame[layer]->m_fencPic;
PicYuv *recon = m_frame[layer]->m_reconPic[0];

x265_vmaf_framedata *vmafframedata = (x265_vmaf_framedata*)x265_malloc(sizeof(x265_vmaf_framedata));
if (!vmafframedata)
{
x265_log(NULL, X265_LOG_ERROR, "vmaf frame data alloc failed\n");
}

vmafframedata->height = fenc->m_picHeight;
vmafframedata->width = fenc->m_picWidth;
vmafframedata->frame_set = 0;
vmafframedata->internalBitDepth = m_param->internalBitDepth;
vmafframedata->reference_frame = fenc;
vmafframedata->distorted_frame = recon;

fenc->m_vmafScore = x265_calculate_vmaf_framelevelscore(vmafframedata);

if (vmafframedata)
x265_free(vmafframedata);
}
}
#endif

[Guest]

So how come x265 has gone to sh*t in the last couple of weeks ??

"Someone" has stuffed it up trying something new, and it's not working.

I've been using either Patman's or jpsdr's build's with RipBot264 for a couple of months, with no issue.

Jpsdr released a newer build 3.60.65, and it errored, but the previous 3.60.35 build is fine.

Same with Patman's, build 3.6+13+1 was/is fine, but 3.6+84+2 error's, and as Patman is used exclusively with StaxRip, I thought I'd try with that, and updating to 3.6+84+2, it failed with basically the same error that RipBot shows.

As for Barough's builds, which I don't use much anymore, have the same issue in his last 2 releases.

Even Selur is complaining, and now that I see that Jpsdr is asking for help, is' obvious that for the time being, it's all gone to sh*t, and "we" have to stay with older builds, if we need to keep encoding with whatever app we choose.

And http://msystem.waw.pl/x265/ is still on 3.6+35-dd594f5, I'm waiting for their next release, to see if it's got the same problems.

I also prefer the aq auto command that both Patman & Jpsdr builds have, none of the others have it, yet.

[jpsdr]

I'm asking for help not because i have issue with x265, but just because i tried to build with another gcc version than the one provided at http://msystem.waw.pl/x265/ and i have issue with static lib link, the fact i try to build with another gcc is not related to the recent changes.

[Kurt.noise]

So how come x265 has gone to sh*t in the last couple of weeks ??

"Someone" has stuffed it up trying something new, and it's not working.

I've been using either Patman's or jpsdr's build's with RipBot264 for a couple of months, with no issue.

Jpsdr released a newer build 3.60.65, and it errored, but the previous 3.60.35 build is fine.

Same with Patman's, build 3.6+13+1 was/is fine, but 3.6+84+2 error's, and as Patman is used exclusively with StaxRip, I thought I'd try with that, and updating to 3.6+84+2, it failed with basically the same error that RipBot shows.

As for Barough's builds, which I don't use much anymore, have the same issue in his last 2 releases.

Even Selur is complaining, and now that I see that Jpsdr is asking for help, is' obvious that for the time being, it's all gone to sh*t, and "we" have to stay with older builds, if we need to keep encoding with whatever app we choose.

And http://msystem.waw.pl/x265/ is still on 3.6+35-dd594f5, I'm waiting for their next release, to see if it's got the same problems.

I also prefer the aq auto command that both Patman & Jpsdr builds have, none of the others have it, yet.
what did you call "release" here, it's not in fact. There are just some compilations from the latest commits of the main branch repository. So, its quite normal to have some problems because not all have been tested yet.

If you stay with 3.6 official release, then you wont have any issues.

[Guest]

what did you call "release" here, it's not in fact. There are just some compilations from the latest commits of the main branch repository. So, its quite normal to have some problems because not all have been tested yet.

If you stay with 3.6 official release, then you wont have any issues.

Fair enough, they are not "proper" releases, but whatever has changed, it's caught some very regular & reliable x265 modder's, off guard.

Fortunately, I do have the last working Patman & Jpsdr builds.

I guess we'll just have to wait 'til the issue is solved.

[LigH]

I'm asking for help not because i have issue with x265, but just because i tried to build with another gcc version ...

You wrote that you try to compile with clang which is not gcc.

[Guest]

You wrote that you try to compile with clang which is not gcc.

I just noticed this build (from your sig link) x265_3.6+65-3c3d03746, which is dated 2024-08-09, is this a "modded" build or basically "vanilla" ?

I am getting the same error in RipBot264, that all the others have :(

I'll have to check some of the older builds.

[LigH]

If it were a build including a modification patch, it would reveal it in its name.

I tried to build a modded version with Boulder's patch, but it failed, so I did not release that. So from me, no current build including AviSynth / Vapoursynth demuxers.

All I added since the alpha channel commits was the additional compiler flags to enable them. But they are official flags by MultiCoreWare.

[jpsdr]

After failing with clang, i tryed with another gcc, as i said in post #9421. My thought was that it would be easier to try with "just" another gcc, and then after try again with clang once i would have been able to build with another gcc.

[jfcarbel]

You won't be disappointed if you "only" go for a 7950X (not the 3D), but it's a whole new motherboard, RAM (DDR5), really good cooling IS necessary.

But if you're coming up from a 5950X, you might as well go for the 9950X, it's only a few hundred $ more...it's only money.

You could save a little overall if you get a lower chipset MB.

Yeah I debated going for the 9950x but yeah motherboards are much higher and found a good Aorus Elite for $200 and also saw some benchmarks showing the 7950x beating the 9950x in some benchmarks.

Cooling wise I use the Dark Rock Pro monster air cooler and it hits within 1-2 degrees from liquid so I prefer this.

[Guest]

Patman has just recently released another build, 3.6+85+1, and it's still got an issue....

[LigH]

Which issue did you expect it to fix? No MCW developer even replied to my bug report about the eternal pipe on the mailing list yet.

[Guest]

Which issue did you expect it to fix? No MCW developer even replied to my bug report about the eternal pipe on the mailing list yet.

I thought he might have known something that nobody else did, so I tested it...

[rwill]

I do not know why you are all so .. 'active'. x265 is open source. So instead of waiting for help, help yourself.

For example there was code like this prior to the partial MV-HEVC support in abrEncApp.cpp around line ~680 (the file which contains the cli app main encode loop):


if (m_cliopt.framesToBeEncoded && inFrameCount >= m_cliopt.framesToBeEncoded)
pic_in = NULL;
else if (readPicture(pic_in))
inFrameCount++;
else
pic_in = NULL;


This was changed to the following to support multiple views in pic_in:

if (m_cliopt.framesToBeEncoded && inFrameCount >= m_cliopt.framesToBeEncoded)
pic_in[view] = NULL;
else if (readPicture(pic_in[view], view) && view == m_param->numViews - 1)
inFrameCount++;
else if (!pic_in[view])
pic_in[view] = NULL;


The main encode loop exits if pic_in[ 0 ] is NULL.

I do not think the following in the new code is going to work:

else if (!pic_in[view])
pic_in[view] = NULL;


There are also other problems I have noticed, but I think that waiting for MCW is not an option as they might take a while to figure stuff out.

[LigH]

So instead of waiting for help, help yourself.

You seem to assume that we all are experienced C/C++ developers only because we are able to run an automated build suite... :sly:

No, I may be a studied computer scientist, but my experience is better in different languages. I don't know the project well enough.

Instead, I do expect of a professional developer team to reply at least briefly to a bug report, the minimum would be a "checked, confirmed".

So, regarding your efforts, which I may forward to the mailing list:

:thanks:

[jpsdr]

I rewrite and put on my github the part exposed by rwill, cleaner from my point of view.
The else if (!pic_in[view])
pic_in[view] = NULL; will partialy work, it's the case there is a picture, and you're not on the last view, but from my point of view it's not clean.

It seems i have to give up trying another gcc or clang, no one seems to know how to make a build in these cases using CMake with the script i've put on #9395.

[LigH]

All I could do is share the scripts (https://www.ligh.de/software/make_x265.zip) I am using right now, so you can compare. For me they work when called in either mingw32 or mingw64 interactive consoles in MABS, depending on the bitness. I don't know how to run all of them in one by cross-compile toolsets.

[qyot27]

Just delete the .dll.a files (or non-destructively rename them to something like .dll.a.bak). That should force it to use the static ones in lieu of flags not working.

[Guest]

For some unknown reason Patman is still releasing builds that don't work...

x265-3.6+85+1-b5cacb584-[Mod-by-Patman]

How to contact him ???

[Kurt.noise]

It seems i have to give up trying another gcc or clang, no one seems to know how to make a build in these cases using CMake with the script i've put on #9395.
I'm using the following script on my machine within msvc & clang tools :
cmake --fresh -B my_own_build -T ClangCL -DENABLE_SHARED=OFF -DHIGH_BIT_DEPTH=ON -DCMAKE_CXX_FLAGS_RELEASE="-flto /O2 /Ob2 /DNDEBUG -march=native" -DCMAKE_C_FLAGS_RELEASE="-flto /O2 /Ob2 /DNDEBUG -march=native" && cmake --build my_own_build --config Release --parallel
only on x64 plateform tough. That works correctly.

For some unknown reason Patman is still releasing builds that don't work...

x265-3.6+85+1-b5cacb584-[Mod-by-Patman]

How to contact him ???
what did you dont understand what we said previously ? Stay with the last one which works for you. You wont notice any difference using the last builds. *sigh*

[Guest]

what did you dont understand what we said previously ? Stay with the last one which works for you. You wont notice any difference using the last builds. *sigh*

I understood perfectly, "Kurt.noise" !!! https://www.cosgan.de/images/smilie/boese/n045.gif

I was just simply stating that Patman is wasting his time, releasing new builds, until this is fixed.

And FYI, I think you will find I was the one that bought the x265 issues to ppl's attention.

[rwill]

I'm trying to build under msys2 with clang. It "works", meaning it links an exe, but when i run it it's complaining abour missing dll.
Here the script i'm using:

...

I think it's missing to the linker to links some libraries, but i have no idea what command i should add, and where (only in the last or on all 3).

Well just FYI, if it links an .exe then all Symbols have been accounted for by the linker. Most likely by means of an .lib import library. If the .exe complains that a .dll cannot be found then the .dll has to be copied to somewhere in the PATH, or next to the .exe, so Windows can find it for the .exe.

Maybe one of the dependencies you link into x265 is missing its .dll dependency. Have you tried building with -DENABLE_LIBVMAF=OFF -DENABLE_VAPOURSYNTH=OFF -DENABLE_AVISYNTH=OFF ? Which libraries do these dependencies pull in? Have you checked in the shell with "ldd x265.exe" and then checked or "ldd whatever_libvmaf_is.dll" (if libvmaf is a .dll) ? There used to be this nifty tool "Dependency Walker" ( "Depends" ) to Debug '.dll Hell' on Windows but I do not know if it is still supported by the modern Windowses.

[Jamaika]

export CPPFLAGS="-D_FORTIFY_SOURCE=2 -D__USE_MINGW_ANSI_STDIO=1"
export CFLAGS="-fstack-protector-strong -mtune=generic -O2 -pipe -mthreads"
export CXXFLAGS="-fstack-protector-strong -mthreads -mtune=generic -O2 -pipe"
export LDFLAGS="-pipe -static-libgcc -fstack-protector-strong -static-libstdc++"

cd /build/x265_git-git/build/msys64_hdr10_ml/
mkdir -p 8bit 10bit 12bit

cd 12bit
make --no-print-directory clean-generated
cmake -G "MSYS Makefiles" ../../../source -DHG_EXECUTABLE=/usr/bin/hg.bat -DCMAKE_CXX_COMPILER=/local64/bin/g++.bat -DCMAKE_TOOLCHAIN_FILE=/local64/etc/toolchain.cmake -DEXPORT_C_API=OFF -DENABLE_SHARED=OFF -DENABLE_CLI=OFF -DHIGH_BIT_DEPTH=ON -DMAIN12=ON -DENABLE_HDR10_PLUS=ON -DENABLE_LIBVMAF=ON -DENABLE_ALPHA=ON -DENABLE_MULTIVIEW=ON -DENABLE_SCC_EXT=ON
make --no-print-directory ${MAKEFLAGS}
cp libx265.a ../8bit/libx265_main12.a

cd ../10bit
make --no-print-directory clean-generated
cmake -G "MSYS Makefiles" ../../../source -DHG_EXECUTABLE=/usr/bin/hg.bat -DCMAKE_CXX_COMPILER=/local64/bin/g++.bat -DCMAKE_TOOLCHAIN_FILE=/local64/etc/toolchain.cmake -DEXPORT_C_API=OFF -DENABLE_SHARED=OFF -DENABLE_CLI=OFF -DHIGH_BIT_DEPTH=ON -DMAIN12=OFF -DENABLE_HDR10_PLUS=ON -DENABLE_LIBVMAF=ON -DENABLE_ALPHA=ON -DENABLE_MULTIVIEW=ON -DENABLE_SCC_EXT=ON
make --no-print-directory ${MAKEFLAGS}
cp libx265.a ../8bit/libx265_main10.a

cd ../8bit
make --no-print-directory clean-generated
cmake -G "MSYS Makefiles" ../../../source -DHG_EXECUTABLE=/usr/bin/hg.bat -DCMAKE_CXX_COMPILER=/local64/bin/g++.bat -DCMAKE_TOOLCHAIN_FILE=/local64/etc/toolchain.cmake -DEXPORT_C_API=ON -DENABLE_SHARED=ON -DENABLE_CLI=ON -DHIGH_BIT_DEPTH=OFF -DENABLE_HDR10_PLUS=ON -DENABLE_LIBVMAF=ON -DENABLE_ALPHA=ON -DENABLE_MULTIVIEW=ON -DENABLE_SCC_EXT=ON -DEXTRA_LIB="x265_main10.a;x265_main12.a" -DEXTRA_LINK_FLAGS=-L. -DLINKED_10BIT=ON -DLINKED_12BIT=ON
make --no-print-directory ${MAKEFLAGS}

Why is ENABLE_SHARED ON for 8bit?

pipe?
The pipe function is declared in the header file unistd.h.
#if NO_ATOMICS

#include <sys/time.h>
#include <unistd.h>
I don't use NO_ATOMICS in x265.

# Work around longjmp interception on glibc >= 2.11, to improve binary
# compatibility. See http://code.google.com/p/webm/issues/detail?id=166
enabled linux && check_add_cflags -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=0

I don't use -D_FORTIFY_SOURCE=2 -fstack-protector-strong for ffmpeg in static.

[Selur]

About the not stopping pipe input I reported (https://bitbucket.org/multicoreware/x265_git/issues/947/cli-y4m-pipe-input-broken-doesnt-stop), I got a reply:
We are able to reproduce it.
We are working on it, will update you once it’s fixed.
=> so there is hope, this will be fixed in a not too distant future. :D

Cu Selur

[john33]

Just delete the .dll.a files (or non-destructively rename them to something like .dll.a.bak). That should force it to use the static ones in lieu of flags not working.
This exactly what m_ab-s does, it renames them to .dll.a.dyn. I've done the same in MSYS2 and it works as expected.

[jpsdr]

You means, i should rename libstdc++.dll.a to libstdc++.dll.a.bak for exemple ? Just this one or more ?

Otherwise, it seems export LDFLAGS="" is doing the same thing than -DCMAKE_EXE_LINKER_FLAGS="".
They both have an effect, because if i change "-static-libgcc -static-libstdc++ -static" with "-static-libgcc -static-libstdc++" i have an error message about another dll file missing using both methods, meaning :
- "static" is necessary
- -DCMAKE_EXE_LINKER_FLAGS="" seems to work, but the link is still missing something.
I'll try the rename tips.

@Selur
When i've finished my builds, maybe you can test if the little modification i've made fix something (very unlikely, but who knows...).

[qyot27]

libc++.dll.a and libunwind.dll.a need to be changed to libc++.dll.a.bak, libunwind.dll.a.bak (or .dyn, if as mentioned above concerning what m-ab-s does; the actual suffix doesn't matter). Then you don't need to pass any flags like -static-libstdc++. And yes, LDFLAGS and CMAKE_EXE_LINKER_FLAGS should be mostly comparable, although I hit the same issue with libc++ and libunwind when building AviSynth+ with llvm-mingw for testing on ARM, and it didn't matter whether I tried using LDFLAGS or CMAKE_SHARED_LIBRARY_FLAGS (since AviSynth+ is a library, not an application); CMake still added them back in and would not use the static versions.

I never resorted to hiding/deleting the .dll.a import libraries, and just copied the relevant .dlls into the test directory alongside AviSynth.dll, but the concept is the same: you need -static if you have both .a and .dll.a versions of a library, because the linker will just prefer the shared version. But if there's ever a problem with how the linker decides whether or not to use the shared or static version of a library, just hide the shared version if you want to link against the static one.

[LigH]

About the not stopping pipe input I reported (https://bitbucket.org/multicoreware/x265_git/issues/947/cli-y4m-pipe-input-broken-doesnt-stop), ...

Oh, bugs should be reported to an issue tracker now? The mailing list is not the primary contact anymore...

[jpsdr]

First, i made a new build.
Those who have specific issue, test, just in case, even if unlikely.

Now, with my tests to build with the gcc version provided with msys2, i'm loosing my head...:eek:
If i do this:

mkdir -p 8bit_Win32_x64_noasm

cd 8bit_Win32_x64_noasm
/G/CMakex64/bin/cmake -G "MSYS Makefiles" ../../../source -DENABLE_LIBVMAF=ON -DENABLE_SCC_EXT=ON -DENABLE_MULTIVIEW=ON -DENABLE_ALPHA=ON -DENABLE_ASSEMBLY=OFF -DENABLE_VAPOURSYNTH=ON -DENABLE_AVISYNTH=ON -DHIGH_BIT_DEPTH=OFF -DENABLE_HDR10_PLUS=OFF -DEXPORT_C_API=ON -DENABLE_SHARED=OFF -DENABLE_CLI=ON -DMAIN12=OFF -DSTATIC_LINK_CRT=ON -DENABLE_LTO=ON -DCMAKE_CXX_FLAGS_RELEASE="-fversion-loops-for-strides -funswitch-loops -fsplit-loops -ffinite-math-only -ftree-vectorize -fivopts -ftree-loop-ivcanon -ftree-loop-if-convert -floop-parallelize-all -floop-nest-optimize -fgcse-las -fgcse-sm -fmodulo-sched-allow-regmoves -fmodulo-sched -fipa-pta -Ofast -ffast-math -fomit-frame-pointer" -DCMAKE_EXE_LINKER_FLAGS="-static-libgcc -static-libstdc++ -static"
make ${MAKEFLAGS}

or this:

mkdir -p 8bit_Win32_x64

cd 8bit_Win32_x64
/G/CMakex64/bin/cmake -G "MSYS Makefiles" ../../../source -DENABLE_LIBVMAF=ON -DENABLE_SCC_EXT=ON -DENABLE_MULTIVIEW=ON -DENABLE_ALPHA=ON -DENABLE_ASSEMBLY=ON -DENABLE_VAPOURSYNTH=ON -DENABLE_AVISYNTH=ON -DHIGH_BIT_DEPTH=OFF -DENABLE_HDR10_PLUS=OFF -DEXPORT_C_API=ON -DENABLE_SHARED=OFF -DENABLE_CLI=ON -DMAIN12=OFF -DSTATIC_LINK_CRT=ON -DENABLE_LTO=ON -DCMAKE_CXX_FLAGS_RELEASE="-fversion-loops-for-strides -funswitch-loops -fsplit-loops -ffinite-math-only -ftree-vectorize -fivopts -ftree-loop-ivcanon -ftree-loop-if-convert -floop-parallelize-all -floop-nest-optimize -fgcse-las -fgcse-sm -fmodulo-sched-allow-regmoves -fmodulo-sched -fipa-pta -Ofast -ffast-math -fomit-frame-pointer" -DCMAKE_EXE_LINKER_FLAGS="-static-libgcc -static-libstdc++ -static"
make ${MAKEFLAGS}

the x265.exe can be run.
But this one:

mkdir -p 8bit_Win32 10bit_Win32 12bit_Win32

cd 12bit_Win32
/G/CMakex64/bin/cmake -G "MSYS Makefiles" ../../../source -DENABLE_LIBVMAF=ON -DENABLE_SCC_EXT=ON -DENABLE_MULTIVIEW=ON -DENABLE_ALPHA=ON -DENABLE_VAPOURSYNTH=ON -DENABLE_AVISYNTH=ON -DHIGH_BIT_DEPTH=ON -DENABLE_HDR10_PLUS=ON -DEXPORT_C_API=OFF -DENABLE_SHARED=OFF -DENABLE_CLI=OFF -DMAIN12=ON -DSTATIC_LINK_CRT=ON -DENABLE_LTO=ON -DCMAKE_CXX_FLAGS_RELEASE="-fversion-loops-for-strides -funswitch-loops -fsplit-loops -ffinite-math-only -ftree-vectorize -fivopts -ftree-loop-ivcanon -ftree-loop-if-convert -floop-parallelize-all -floop-nest-optimize -fgcse-las -fgcse-sm -fmodulo-sched-allow-regmoves -fmodulo-sched -fipa-pta -Ofast -ffast-math -fomit-frame-pointer"
make ${MAKEFLAGS}
cp libx265.a ../8bit_Win32/libx265_main12.a

cd ../10bit_Win32
/G/CMakex64/bin/cmake -G "MSYS Makefiles" ../../../source -DENABLE_LIBVMAF=ON -DENABLE_SCC_EXT=ON -DENABLE_MULTIVIEW=ON -DENABLE_ALPHA=ON -DENABLE_VAPOURSYNTH=ON -DENABLE_AVISYNTH=ON -DHIGH_BIT_DEPTH=ON -DENABLE_HDR10_PLUS=ON -DEXPORT_C_API=OFF -DENABLE_SHARED=OFF -DENABLE_CLI=OFF -DMAIN12=OFF -DSTATIC_LINK_CRT=ON -DENABLE_LTO=ON -DCMAKE_CXX_FLAGS_RELEASE="-fversion-loops-for-strides -funswitch-loops -fsplit-loops -ffinite-math-only -ftree-vectorize -fivopts -ftree-loop-ivcanon -ftree-loop-if-convert -floop-parallelize-all -floop-nest-optimize -fgcse-las -fgcse-sm -fmodulo-sched-allow-regmoves -fmodulo-sched -fipa-pta -Ofast -ffast-math -fomit-frame-pointer"
make ${MAKEFLAGS}
cp libx265.a ../8bit_Win32/libx265_main10.a

cd ../8bit_Win32
/G/CMakex64/bin/cmake -G "MSYS Makefiles" ../../../source -DENABLE_LIBVMAF=ON -DENABLE_SCC_EXT=ON -DENABLE_MULTIVIEW=ON -DENABLE_ALPHA=ON -DENABLE_VAPOURSYNTH=ON -DENABLE_AVISYNTH=ON -DHIGH_BIT_DEPTH=OFF -DENABLE_HDR10_PLUS=OFF -DEXPORT_C_API=ON -DENABLE_SHARED=OFF -DENABLE_CLI=ON -DMAIN12=OFF -DSTATIC_LINK_CRT=ON -DENABLE_LTO=ON -DLINKED_10BIT=ON -DLINKED_12BIT=ON -DEXTRA_LIB="x265_main10.a;x265_main12.a" -DEXTRA_LINK_FLAGS=-L. -DCMAKE_CXX_FLAGS_RELEASE="-fversion-loops-for-strides -funswitch-loops -fsplit-loops -ffinite-math-only -ftree-vectorize -fivopts -ftree-loop-ivcanon -ftree-loop-if-convert -floop-parallelize-all -floop-nest-optimize -fgcse-las -fgcse-sm -fmodulo-sched-allow-regmoves -fmodulo-sched -fipa-pta -Ofast -ffast-math -fomit-frame-pointer" -DCMAKE_EXE_LINKER_FLAGS="-static-libgcc -static-libstdc++ -static"
make ${MAKEFLAGS}

# rename the 8bit library, then combine all three into libx265.a using GNU ar
mv libx265.a libx265_main.a

ar -M <<EOF
CREATE libx265.a
ADDLIB libx265_main.a
ADDLIB libx265_main10.a
ADDLIB libx265_main12.a
SAVE
END
EOF

i have the error message "can't find libstc++-6.dll".
So i renamed libstdc++.dll.a to libstdc++.dll.a.bak.
The error message changed to "can't find libwinpthread-1.dll".
So i renamed libwinpthread.dll.a to libwinpthread.dll.a.bak.
The error message was still the same. So either the gcc provided with msys2 which is a posix version can't build x265, either it's another dll i have to rename. It's late, i stop here for now.
There is no libunwind files in the gcc provided with msys2, but there is such files in the mcf gcc and in the clang with msys2.
I'll try with mcf gcc version tomorrow.

[Guest]

First, i made a new build.
Those who have specific issue, test, just in case, even if unlikely.

I'll try with mcf gcc version tomorrow.

I thought I'd try your latest attempt (3.60.85)...

I'm still getting this:-

x265 [WARN]: extra unused command arguments given <->

[jpsdr]

This is now the new way, you need --input. I was thinking of the pipe issue.

[Guest]

This is now the new way, you need --input. I was thinking of the pipe issue.

I've been told by several others that this --input is now the "new norm", but what I can't understand is if this can be added to the x265 command line, or does the app developer (for RipBot264, StaxRip, Hybrid and others) will have to modify the "behind the scenes" command's.

What does it need to "input" ??

Is there an example of what needs to be done, if it is a command line entry ??

Just did a Google, and it found this:-

https://github.com/staxrip/staxrip/issues/1457

So Dendraspis is onto it for StaxRip :)

[Boulder]

The external app developers have to modify their applications. All in all, it's always much safer to include the parameter names in command lines as things may change but parameter names very rarely (or at least they are kept for backward compatibility).

[LigH]

A patch for the Y4M pipe has just been mentioned in the mailinglist; waiting for commit...

[Selur]

or does the app developer (for RipBot264, StaxRip, Hybrid and others) will have to modify the "behind the scenes" command's.
Hybrid always used 'x265 --input - ' for pipe input, so nothing to adjust there. :)

A patch for the Y4M pipe has just been mentioned in the mailinglist; waiting for commit...
Nice!

Cu Selur

[jpsdr]

I've continued my build tests. Someone even suggest my to use "-Bstatic -l<lib> -l<lib> .. -Bdynamic", to force static link, but it didn't work.
Finaly, what worked was indeed to rename...
For build with mcf version, i renamed libstdc++ and libmcfgthread.
For build with clang just libc++.

[LigH]

New upload: x265 3.6+103-85f1e34+pipefix (https://www.mediafire.com/file/oei7rjjxiilvaur/x265_3.6+103-85f1e34+pipefix.7z/file)

[Windows][GCC 14.2.0][32/32XP/64 bit] 8bit+10bit+12bit

No serious changes in intel x86[-64] code. Added the patch to fix Y4M pipe handling prior to its commit.

[rwill]

What I do not understand is that there is code like this for the 'input' arg:

OPT("input") inputfn[0] = optarg;

So when one specifies --input something then inputfn[0] gets set to that.

Then there is new code like this:

#if !ENABLE_MULTIVIEW
if (optind < argc && !inputfn[0])
inputfn[0] = argv[optind++];
#endif

So having the input filename as one of the last trailing args on the command line is now only supported for non ENABLE_MULTIVIEW builds. I find this kind of strange as from a functionality point of view '--input' and having it 'trailing' is of no difference?

[LigH]

If you have multi-view enabled, and you actually want to encode MVC with separate input video sources for more than one camera angle, you must be able to tell apart one input from the other. That's why the implicit "last parameter is input" doesn't work, there can be only one last parameter.

[rwill]

Yes I know, but this is what their Multiview Config File is for. There can also only be one --input. Instead of breaking compatibility with a couple of GUIs and Scripts they could have used


if( !MultiViewConfigFile )
...


instead of


#if !ENABLE_MULTIVIEW

[LigH]

Telling us doesn't help. Tell MultiCoreWare.

[Z2697]

The pipe problem...
I did my fix weeks ago.
But now the patch in mailing list looks better to me.
(The "better" in the commit msg refers to me previously just remove " if (!pic_in[view]) " )
https://github.com/Mr-Z-2697/x265-Yuuki-Asuna/commit/3d72ad6a97be8d106c71e3e6e3a4121dbd49f793

[Selur]

I did my fix weeks ago.
And you did not post it to the dev list. why?

[Z2697]

And you did not post it to the dev list. why?

Hmm...
It's like, I don't know how. Sorry.

Besides, my code sucks.

[modus-ms325c]

Telling us doesn't help. Tell MultiCoreWare.

And you did not post it to the dev list. why?

so, how come have we, as a society, as the kind of people who just so happen to be grouped in some sort of space or bar or whatever, decided that giving this kind of response to literally anyone who has even the slightest of trust issues to code maintainers, is in any way normal?

or maybe dealing with code maintainers is just more trouble than it's worth being begged to acceptance, idk.

[Selur]

Sorry, but this thread is no where just some sort of space. + don't know what society you live in, since I have no clue about you.
Where LigH and I live (Germany), asking this isn't frowned upon like it seems in the society you live in.
Also posting a fix to an issue tracker or a mailing list is worth it, even if the maintainers do not accept it, others might.
A more philosophical discussion about societal development probably should be help in another environment.
Not in a thread about the development of the x265 encoder.

Cu Selur

[jpsdr]

Euh... It looks like a lot in the same place i put a fix on my github, as i said in post #9449, according rwill post.

Does my version also fix the issue ?

My commit is:
https://github.com/jpsdr/x265/commit/a70b9c0dd269154a92a86c3d6d3ded7556b96562
But no one told me if it fxes the issue or not, as i personnaly not using x265 in a way it produces the issue.

[LigH]

Added the patch to fix Y4M pipe handling prior to its commit.

This patch has been pushed to the master branch.

So now everyone can build (generic or modified) binaries without manually adding that specific patch, it is now included in official sources.

[jpsdr]

LOL ! It's exactly what i've done...! But still, thanks to the report of rwill.

[Selur]

Nice! Thanks for the update!

[rwill]

Now because people are so hesitant to contact MCW, they are a business trying to make money. They also still seem have to a development team so who are we to tell em how to do their job. I just write comments here so maybe people prep up their little forks locally.

Now regarding sending actual patches to x265-devel ... I think they require a signed contributor agreement to give them full copyright. Like, they are a business trying to make money. I personally don't have time for that and it is also a legal problem for me because I have my own AVC, HEVC, VVC, etc. encoders to take care of.

x264 was different, just hop on IRC (which had no account restrictions back then) and tell one of the devs of something you found. These were the good times..

[jpsdr]

Benchmark

Out of curiosity, i've made some benchmark of my builds.

I have a dedicated PC for video encoding/processing, it has no network cable connected, ethernet chipset is disabled in BIOS, all internet related services, printer, an not necessary stuff are disabled, only the mandatory services needed by the "core" are kept.
There is of course no firewall, antivirus or any kind of these stuff installed, so there is "nothing" else running on this PC.

mcf, win32 and clang version under msys2.
LLVM version integrated with Visual Studio.

I've made a multi-pass encode on a small 128 frames 4k HDR video.

mcf v14.2.1 (20240821)
Pass 1: 143.08s
Pass 2: 258.47s
Pass 3: 201.14s

win32 v13.1.0 (20230426)
Pass 1: 141.34s
Pass 2: 255.88s
Pass 3: 200.82s

clang v1.8.1
Pass 1: 139.64s
Pass 2: 254.95s
Pass 3: 198.47s

LLVM v1.8.1
Pass 1: 136.19s
Pass 2: 247.68s
Pass 3: 196.74s

Honnestly, it's not the results i expected... :(
With x264, i had mcf faster than win32 faster than posix.
I expected found at least here mcf faster than win32.

Also, with clang, i can add "-march=broadwell -mtune=broadwell" not supported with LLVM. So, i expected clang be at least as fast as LLVM.
But, i also for clang need to remove 90% of the options i put in gcc are they were either unknow (giving [Error]) or not supported (giving [Warning]). I still have to check with the VisualStudio command line for LLVM and also the site of clang to see if i can add some options on the clang command line, to at leat achieve the same speed than LLVM.

[LigH]

I don't understand why you expected any noticeable difference between binaries executing more than 90% of pre-optimized SIMD assembly code, so that the impact of remaining C/C++ code and compiler optimizations of not at all performance relevant parts remains negligible.

[jpsdr]

I wasn't clear enough on my "disapointment".
It's not the values, it's the order.
I expected clang faster than LLVM faster than mcf faster than win32.

[LigH]

The differences are so tiny, random delays caused by OS tasks and disk access hitting or missing the cache are way more relevant.

[jpsdr]

% between each version (mcf->win32, win32->clang, clang->llvm/mcf->llvm)
mcf v14.2.1 (20240821)
Pass 1: 143.08s
Pass 2: 258.47s
Pass 3: 201.14s

win32 v13.1.0 (20230426)
Pass 1: 141.34s [-1,27%]
Pass 2: 255.88s [-1,00%]
Pass 3: 200.82s [-0,16%]

clang v1.8.1
Pass 1: 139.64s [-1,21%]
Pass 2: 254.95s [-0,36%]
Pass 3: 198.47s [-1,17%]

LLVM v1.8.1
Pass 1: 136.19s [-2,47%]/[-4,82%]
Pass 2: 247.68s [-2,85%]/[-4,17%]
Pass 3: 196.74s [-1,01%]/[-2,19%]

Different are tiny but consistant.
I hoped i could switch from LLVM to clang, it would be faster for me to make build, but i'll stay with LLVM.

And about OS task, i've described the PC used, and how they are reduced to minimum. Disk is SSD. Even 4%, when you have 3 days for one pass, it's almost 3h, it's still interesting.

[Z2697]

I know this is very much a "source: trust me bro" situation but clang or the underlying LLVM will cause the built x265 to output different result compared to GCC built one, and in some situations this "difference" is significant decrease in quality.
So generally speaking, GCC is the way to go. For x265.
To be honest, encoding result being different happens very often. For example even FLAC will produce different result (but still perfectly lossless) when comparing Clang / GCC, static / shared. But for this specific software, x265, it is just worse.

By the way, I think the naming scheme is unclear.
Win32? An operating system API?
Clang? A compiler front-end?
LLVM? The "man in the chair" of Clang?
What is MCF?

[jpsdr]

win32 is the "win32" thread build of gcc by msystem.waw.
mcf is the "mcf" thread build of gcc by lhmouse.
clang is the clang version compiler for msys2 (mingw-w64-clang-x86_64-toolchain).
LLVM is the version you can install and use with Visual Studio (https://llvm.org/ or https://github.com/llvm/llvm-project/releases).

Otherwise, i'm using my LLVM builds for years, without issue noticed. And during these build tests, i've watched the .h265 raw streams created between each version, and didn't seen any difference. I didn't make a bit to bit compare.

Otherwise, to check speed stability, i run 3 more times the encode with LLVM version, results are :

Pass 1: 136.19s/135.94s/135.75s/135.98s
Pass 2: 247.68s/246.62s/247.58s/246.61s
Pass 3: 196.74s/193.37s/192.91s/193.71s

Edit:
I will tomorrow encode my test file with a gcc build and an llvm build and comparte the binary result file.

[Selur]

I've made a multi-pass encode on a small 128 frames 4k HDR video.
+
mcf v14.2.1 (20240821)
Pass 2: 258.47s

win32 v13.1.0 (20230426)
Pass 2: 255.88s [-1,00%]

clang v1.8.1
Pass 2: 254.95s [-0,36%]

LLVM v1.8.1
Pass 2: 247.68s [-2,85%]/[-4,17%]

that's like 2 seconds per frame (0.5 fps) :scared: during the 2nd pass.
Out of curiosity: Is your system that old, or are you using totally insane/placebo settings?

[Jamaika]

What do you think about the added corrections? There is no description of what this results from.
https://bitbucket.org/multicoreware/x265_git/diff/source/common/lowpassdct.cpp?diff2=7c190672efa6&at=master

https://bitbucket.org/multicoreware/x265_git/diff/source/common/lowpassdct.cpp?diff2=dd594f59d951&at=master

[Boulder]

What do you think about the added corrections? There is no description of what this results from.
https://bitbucket.org/multicoreware/x265_git/diff/source/common/lowpassdct.cpp?diff2=7c190672efa6&at=master

https://bitbucket.org/multicoreware/x265_git/diff/source/common/lowpassdct.cpp?diff2=dd594f59d951&at=master

They are related to this experimental feature: https://x265.readthedocs.io/en/master/cli.html#cmdoption-lowpass-dct. I don't think I've ever seen anyone use it.

[Jamaika]

They are related to this experimental feature: https://x265.readthedocs.io/en/master/cli.html#cmdoption-lowpass-dct. I don't think I've ever seen anyone use it.
So why is someone correcting it? Delete it.

How did it happen that first the value of 1 was changed to -1 and then only in one place 1 was differentiated to the value of -1.

[Boulder]

So why is someone correcting it? Delete it.
So you think the encoder should only have features that 99% of the average Joe user base uses? Just take a look at how many real-time encoding related things there are, why not ditch them as well then :p

[Jamaika]

I'm more interested in the train of thought. Does anyone do this on a whim?

Second thing. The https://bitbucket.org website seems to be a mirror without description.

[Boulder]

MCW is a commercial entity and I bet most of the work we see in the repo are results from paid or sponsored work. You can submit patches as a normal user but then you are allowing them to use those in commercial stuff.

[jpsdr]

@Selur
My system was the top when i made it... a few years ago. It's a 10 cores Broadwell overclocked to 3,8GHz.

You can see my settings in post #9368 (https://forum.doom9.org/showpost.php?p=2005481&postcount=9368).

[Z2697]

win32 is the "win32" thread build of gcc by msystem.waw.
mcf is the "mcf" thread build of gcc by lhmouse.
clang is the clang version compiler for msys2 (mingw-w64-clang-x86_64-toolchain).
LLVM is the version you can install and use with Visual Studio (https://llvm.org/ or https://github.com/llvm/llvm-project/releases).

Otherwise, i'm using my LLVM builds for years, without issue noticed. And during these build tests, i've watched the .h265 raw streams created between each version, and didn't seen any difference. I didn't make a bit to bit compare.

Otherwise, to check speed stability, i run 3 more times the encode with LLVM version, results are :

Pass 1: 136.19s/135.94s/135.75s/135.98s
Pass 2: 247.68s/246.62s/247.58s/246.61s
Pass 3: 196.74s/193.37s/192.91s/193.71s

Edit:
I will tomorrow encode my test file with a gcc build and an llvm build and comparte the binary result file.

Results from the different version of the same compiler and same flags is probably gonna be identical. That's not what I'm talking about. I was talking about different compilers.
These kind of difference is kinda common, at least in the encoders I have tested or seen someone tested. It's not necessarily mean something bad happened.

Now talk about the statement that the clang sometimes makes "that difference" of x265 to go bad. I must apology I didn't do a very comprehensive test. A friend told me and what I did was only confirming it.
After I did some more testing, I found that statement is probably not true. The only situation I can confirm is with very unusual encoding parameters. I can basically narrow it down to very low crf (e.g. 4), with negative cb/crqpoffset and rdoq enabled (rdoq-level 1 or 2).
The original example I got is with crf=4 and cb/crqpoffset=-2. Clang-built x265 will have severe blocking in chroma but GCC-built one is fine.
But if I give them more "extreme" values, like crf=-12 (10bit) and offsets=-12, GCC-built x265 will have severe problem too.
So Clang probably just shows this problem "earlier". But they have the same flaws under the hood.

As for the speed comparison, will you interested in doing some profile-guided optimization testing?

[Z2697]

@Selur
My system was the top when i made it... a few years ago. It's a 10 cores Broadwell overclocked to 3,8GHz.

You can see my settings in post #9368 (https://forum.doom9.org/showpost.php?p=2005481&postcount=9368).

Broadwell is almost a decade ago now😂

[tormento]

Broadwell is almost a decade ago now
I7-2600k here, daily overclocked to 4.6 GHz.

[jpsdr]

I was talking about different compilers.
So do I.

I've made an encode with a version build with llvm.
Then I've made an encode with a version build with gcc.
File were of different sizes.
But... to be sure, i redo the exact same encode with the gcc build version. Third file, third different size...
So, even with the exact same exe, running twice the exact same encode didn't even produce bit identical files, as they even are not of the same size. Of course, difference is small (around 2kB~4kB for a 20MB file).

For speed, thanks for the offer, but i will not be doing more testing.

[Z2697]

So, even with the exact same exe, running twice the exact same encode didn't even produce bit identical files, as they even are not of the same size. Of course, difference is small (around 2kB~4kB for a 20MB file).


That's not normal I'd say. Most times the result should be exactly the same, with same exact settings and executable file. (Even the 3.5 and 3.6 versions I compiled prodeces same results (except for header) with my daily settings.)
Something I can remember now is the HME feature, it will produce different result when combined with lookahead-slices.
I've also mainly tested 1-pass CRF only, not sure if it's the 3-pass ABR encoding that's responsible.

[Lucky38]

Just building: Latest commit adds support for Screen Content Coding (https://hevc.hhi.fraunhofer.de/scc), requires compile option enabled


__

New upload: x265 3.6+65-3c3d03746 (https://www.mediafire.com/file/5zoll6p8s1hvdnn/x265_3.6+65-3c3d03746.7z/file)

[Windows][GCC 14.2.0][32/32XP/64 bit] 8bit+10bit+12bit
-DENABLE_HDR10_PLUS=ON -DENABLE_LIBVMAF=ON -DENABLE_ALPHA=ON -DENABLE_MULTIVIEW=ON -DENABLE_SCC_EXT=ON


what is real benefit when SCC is ON?

[LigH]

Did you even read the linked explanation by Fraunhofer HHI?

The Screen Content Coding (SCC) extensions will improve compression capability for video containing a significant portion of rendered (moving or static) graphics, text, or animation rather than (or in addition to) camera-captured video scenes. Example applications include wireless displays, remote computer desktop access, and real-time screen sharing for videoconferencing.

[Lucky38]

Did you even read the linked explanation by Fraunhofer HHI?

i read it... so it is not something that can be used for movies, tv shows then...

[LigH]

Depends. If it is a movie with a lot of artificial screen overlays, maybe. Then test it with and without, compare, and then decide.

Please note, this is an extension to the basic HEVC standard. Not all decoders may support it.


PS: Another batch of AArch64 patches was just pushed. So x86* builds are probably not affected.

[tormento]

I have tried Patman’s build and SCC seems to be encoding but the result is an error about premature end of file and a black unreadable video. He confirmed the issue.

Someone had better luck?

[LigH]

Yes, a freshly built x265 v3.6+130 with SCC enabled encodes but MPC-HC with its internal LAV filters can't decode it properly. That's probably due the libavcodec HEVC decoder to not supporting HEVC SCC Profile bitcode. You would need to find another decoder which is able.

See also: Wikipedia: High Efficiency Video Coding | Version 3 and higher profiles (https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding#Version_3_and_higher_profiles)

[Boulder]

SCC is one those things related to streaming, like webinars with PowerPoint presentations etc. For example SVT-AV1 has had it for a long time already, but of course it's really not meant for your regular content.

[tormento]

Yes, a freshly built x265 v3.6+130 with SCC enabled encodes but MPC-HC with its internal LAV filters can't decode it properly.
Do you know a player that can actually reproduce it?

Why does StaxRip give a frame error at the end? I didn't try with other GUIs so I don't know if it's a x265 or StaxRip issue.

Moreover, when muxing the hevc, MKVToolnix-gui outputs a very short video.

[LigH]

Quite recent MP4Box and FFmpeg versions can multiplex it to MP4. But mkvmerge seems to recognise only HEVC packets known in basic profiles.

And the native HEVC decoder in libavcodec, core of ffmpeg, does not yet support decoding it.

[LigH]

New upload: x265 3.6+130-184bed4 (https://www.mediafire.com/file/pvybf780x601u03/x265_3.6+130-184bed4.7z/file)

[Windows][GCC 14.2.0][32/32XP/64 bit] 8bit+10bit+12bit

No serious changes in intel x86[-64] code.

[LigH]

MKVToolnix-gui outputs a very short video.

Reported to Moritz Bunkus (https://gitlab.com/mbunkus/mkvtoolnix/-/issues/3738) - low priority, there is no need to hurry without a decoder yet.

[LigH]

x265 Version 4.0 has been released:

The v4.0 release of x265 is out now. Below is the list of new features, optimization, and bug fixes in this version.


Version 4.0
===========

New feature
-----------
1. Alpha Channel feature.
2. Screen Content Coding (SCC).
3. MV-HEVC feature.

Enhancements to existing features
---------------------------------
1. Added support for the VMAF v3.x.

API changes
-----------
1. Add command line parameter for Alpha Channel feature :option:`--alpha`.
2. Add command line parameter for SCC feature :option:`--scc 1`.
3. Add command line parameters for the MV-HEVC feature :option:`--multiview-config "multiview_config.txt"`.

Optimizations
---------------------
1. Arm SIMD optimizations: Several time-consuming scalar C functions now have SIMD implementations on Arm platforms. Existing Arm SIMD implementations have also been optimized. These optimizations result in up to 57% faster encoding compared to release 3.6.
2. Arm SIMD optimizations include use of Armv8.4 DotProd, Armv8.6 I8MM, and Armv9 SVE2 instruction set extensions. The following algorithms now have optimized SIMD implementations: SAD, SSE, DCT, SAO, convolution, quantization, intra_planar, intraFilter, intrapred DC and IDCT16x16.

Bug fixes
---------
1. Fix for y4m pipe input broken.
2. Fix SCC crash on multipass encode.
3. Fix mcstf when :option:`--bframes` value was less than 5.
4. Fix lowpass DCT for high bit depth.
5. Added build support for Visual Studio 17.
6. Fix issue in default code flow and memory leak.
7. Framethreads tuning for Windows ARM devices.
8. Fix scc crash on multipass encode.

Thanks & Regrds,

Karam Singh

[ksm]

We have released a new version (v4.0) of x265. You can download the x265 tarball from our downloads page ( https://bitbucket.org/multicoreware/x265_git/downloads/ ).

[LigH]

:thanks:

People will surely start building binaries right away...
Especially the media-autobuild suite makes it pretty easy for the Windows platform.

[madmax3308]

Thanks!

[LigH]

New upload: x265 4.0+4-29d59cb (https://www.mediafire.com/file/klopqt7ecgsgsbw/x265_4.0+4-29d59cb.7z/file)

[Windows][GCC 14.2.0][32b/32b~XP/64b/64b+MV] 8bit+10bit+12bit

Please note: From version 4.0 on I will build all variants without MultiView support so that they may still work without explicit --input parameter, except for a 64 bit build which has it explicitly enabled.

[Guest]

New upload: x265 4.0+4-29d59cb (https://www.mediafire.com/file/x7djosnm3qlf29t/x265_4.0+4-29d59cb.7z/file)

[Windows][GCC 14.2.0][32/32XP/64 bit] 8bit+10bit+12bit

I just wanted to let you know that I've figured out a workaround to get these new builds working in RipBot264.

God only knows how long it'll take the dev to implement the change :(

[Z2697]

I just wanted to let you know that I've figured out a workaround to get these new builds working in RipBot264.

God only knows how long it'll take the dev to implement the change :)

What is problem, and the change?

[LigH]

Probably like in this post (https://forum.doom9.org/showthread.php?p=2006886#post2006886).

[Guest]

Probably like in this post (https://forum.doom9.org/showthread.php?p=2006886#post2006886).

Almost...Wishbringer was VERY close, but didn't he say that it still didn't work ??

[LigH]

It is a manual edit, so it works easily for one job but would be very annoying for a distributed batch.

Why are you hiding your solution from the rest of the world?

[LigH]

I changed my x265 v4.0+4 archive (https://forum.doom9.org/showthread.php?p=2006909#post2006909).

Please note: From version 4.0 on I will build all variants without MultiView support so that they may still work without explicit --input parameter, except for a 64 bit build which has it explicitly enabled.

[Wishbringer]

Problem solved, Atak_Snajpera updated RB and also x265 in his updates. But thanks anyway for your workaround @LigH.

[LigH]

There may be others with similar issues...

[LigH]

x265cli.cpp, lines 808 ff.:
#if !ENABLE_MULTIVIEW
if (optind < argc && !inputfn[0])
inputfn[0] = argv[optind++];
#endif
I assumed that if MV-HEVC is disabled in compilation options, interpreting the last parameter as input filename (if there was no --input parameter yet) is enabled, as it used to work before introducing MV-HEVC.

But it doesn't seem to work in my recently compiled v4.0+4.

[Z2697]

So do I.

I've made an encode with a version build with llvm.
Then I've made an encode with a version build with gcc.
File were of different sizes.
But... to be sure, i redo the exact same encode with the gcc build version. Third file, third different size...
So, even with the exact same exe, running twice the exact same encode didn't even produce bit identical files, as they even are not of the same size. Of course, difference is small (around 2kB~4kB for a 20MB file).

For speed, thanks for the offer, but i will not be doing more testing.

I tested your parameters, In this particular case it's vbv that caused this phenomenon. But I didn't go further (since the vbv option is the first one in your command line after those basic stuff)

[LigH]

Wondering if the cause might be wrong parameters in a cmake cache, I rebuilt my binaries. But they turned out to be the same, that was not the reason.

Now I reported it to x265_git issue 954 (https://bitbucket.org/multicoreware/x265_git/issues/954/x265cli-simple-input-parameter-does-not).

If any competent programmer here could have a look, that would be very appreciated.

[ksm]

Thank you for bringing up these issues to our attention. We are working on fixing all of these issues and once done we will release a minor version. Thanks

[LigH]

While you are at it:

Help messages are not formatted well in new features, sometimes missing manual line breaks which happen randomly in long lines, I remember I spotted typos and strange grammar...

[tormento]

Is possible to write / overwrite encoding library or other fields in a hevc file?

I mean, after it's been created.

[LigH]

Regarding the specifications (in theory): Encoder library and options are metadata, pure text. They can be omitted by an encoder or removed later. Header fields describing the content (e.g. colorimetry flags, aspect ratio) can be edited with special tools (which may take care of checksums, if any exist). They just need to exist and be known ... I do remember that a patcher for raw H.264 (AVC) video streams exists. And I read that ffmpeg can be used to alter some metadata in video streams.

In other words: It is absolutely possible that software can exist which is able to patch H.265 metadata. It may even exist already. I just can't tell you a specific tool, besides ffmpeg.

[tormento]

Regarding the specifications
Recent versions of ffmpeg can write/delete AVC metadata and delete HEVC metadata only.

Any help is welcome.

[LigH]

In case of despair, use a "hex editor" to overwrite the human-readable text meta info about the encoder and its options.

[Z2697]

Recent versions of ffmpeg can write/delete AVC metadata and delete HEVC metadata only.

Any help is welcome.

What exactly are you trying to modify?

[tormento]

What exactly are you trying to modify?
I need to put some "watermark" directly in hevc file metadata, so that muxing won't change or tamper it.

[LigH]

Well, you can certainly use a hex editor to patch that text byte by byte. It is for (tech-literate) humans to casually read, not for software to rely on.

[Z2697]

HEX editor is definitely gonna work but same for anyone that's tech-literate enough if they want to remove it.

If you don't want to do that hex editing you can mess with the version.cmake or param.cpp to change how the version string is generated or how the SEI is "printed".

Or.......... you can use the x265 parameter --nalu-file to insert a custom SEI message but I never used it, I can't guarantee that it'll work, plus good luck for any of your viewers can find this easter egg (if it's only important to yourself then this is no problem)
Alright, I tested it and it works, it looks like you can insert SEI message at any frame from the help msg but I can only get it working at the frame 0.
An example of nalu-file:
0 PREFIX 39/5 VGhlIHF1aWNrIGJyb3duIGZveCBqdW1wcyBvdmVyIHRoZSBsYXp5IGRvZy4=

[tormento]

Patman released ICC builds on his repo (https://github.com/Patman86/x265-Mod-by-Patman/releases)!

:thanks:

[LigH]

@Z2697: Is that Base64 encoding or similar? ... Ah, yes, obviously. Woof!

[rwill]

Or.......... you can use the x265 parameter --nalu-file to insert a custom SEI message but I never used it, I can't guarantee that it'll work, plus good luck for any of your viewers can find this easter egg (if it's only important to yourself then this is no problem)
Alright, I tested it and it works, it looks like you can insert SEI message at any frame from the help msg but I can only get it working at the frame 0.
An example of nalu-file:

Your example seems to be missing the 16 bytes uuid_iso_iec_11578 at the start of the unregistered SEI payload. I suggest creating a hopefully unique 16 byte UUID using some generator and use that for each of your so inserted SEIs.

[Z2697]

Your example seems to be missing the 16 bytes uuid_iso_iec_11578 at the start of the unregistered SEI payload. I suggest creating a hopefully unique 16 byte UUID using some generator and use that for each of your so inserted SEIs.

Judging by the resulting bitstream, x265 handles that.

And LigH, yes it's base64.

[birdie]

Thank you for bringing up these issues to our attention. We are working on fixing all of these issues and once done we will release a minor version. Thanks

Any news on x266?

[LigH]

Not in this thread. See here (https://forum.doom9.org/showthread.php?t=181816) instead.

[birdie]

Not in this thread. See here (https://forum.doom9.org/showthread.php?t=181816) instead.

For visibility and transparency I'm asking a new MulticoreWare account that has no messages in other threads.

I hope you can forgive me.

[LigH]

Of course, understood :)

I was just mentioning the existence of a specific thread.

[tormento]

Seems that there is a strange issue (https://github.com/Patman86/x265-Mod-by-Patman/issues/21) with x265.

I'd like to hear your thoughts too.

[benwaggoner]

Any news on x266?
I didn't hear anything about x266 specifically at IBC. There were a number of companies with live VVC encoders they were demonstrating, up to 8Kp60.

[LigH]

A few more patches are being proposed. Next to several crash and bug fixes (e.g. the left-over filename as input if multiview is disabled), also this:
Add Aom film grain characteristics as SEI message to the bitstream

Available: probably soon™

[Boulder]

A few more patches are being proposed. Next to several crash and bug fixes (e.g. the left-over filename as input if multiview is disabled), also this:


Available: probably soon™

I wonder if the FGS table which aomenc or svt-av1 use can be applied. And if any hardware decoder can use it, it would be a killer functionality if the common Amlogic chips used in Android boxes supported this.

[Jamaika]

The warmings for C++23 are interesting.
sao.cpp: In member function 'void x265::SAO::rdoSaoUnitCu(x265::SAOParam*, int, int, int)':
sao.cpp:1242:66: warning: arithmetic between different enumeration types 'x265::SAO::<unnamed enum>' and 'x265::SAOType' is deprecated [-Wdeprecated-enum-enum-conversion]
1242 | X265_CHECK(sizeof(PerPlane) == (sizeof(int32_t) * (NUM_PLANE * MAX_NUM_SAO_TYPE * MAX_NUM_SAO_CLASS)), "Found Padding space in struct PerPlane");
| ~~~~~~~~~~^~~~~~~~~~~~~~~~~~
common.h:113:37: note: in definition of macro 'X265_CHECK'
113 | #define X265_CHECK(expr, ...) if (!(expr)) { \
| ^~~~
frameencoder.cpp: In member function 'virtual void x265_10bit::FrameEncoder::processRowEncoder(int, x265_10bit::ThreadLocalData&, int)':
frameencoder.cpp:1748:16: warning: '++' expression of 'volatile'-qualified type is deprecated [-Wvolatile]
1748 | curRow.completed++;
| ~~~~~~~^~~~~~~~~

[LigH]

I wonder if the FGS table which aomenc or svt-av1 use can be applied.

Just a very brief look: it tries to open a "film grain characteristics binary file".

[Boulder]

Just a very brief look: it tries to open a "film grain characteristics binary file".

I'm hoping that their "binary file" just means an input file. The film grain table is pure ASCII, and peeking at the proposed patch, looks like the structure could be the same as there.

[Z2697]

No, it's not the pure ASCII film grain table. (I applied the patch and tried it)

Although I dislike the film grain synthesis feature I'm still curious to try but I just have no idea how to get a film grain charastistic data that x265 will accept, that's sad, a little bit.
Heck, the AV1 league just made it basically one-click and have fun.

[Z2697]

I just saw this issue
https://bitbucket.org/multicoreware/x265_git/issues/612/faulty-commit-value-on-compiled-binaries

I don't really remember my bb account and I don't want to create a new one. Since one of the participants, Barough, is here I think I'll post here (is MrB also Barough or just coincidence)
The x265 version is not wrong, it's just that it uses the latest commit hash, and if you use media-autobuild_suite to build x265, it adds one patch here (https://github.com/m-ab-s/media-autobuild_suite/blob/00ef1d393bf2e3f74c9e6470f4a89bc52db68470/build/media-suite_compile.sh#L1717) and that makes the latest commit hash to change.
If you don't want it to interfere x265 versioning you can actually skip that patch, but you will need to rebuild x265 when gcc updates according to the commit message in that patch.

[LigH]

I asked about the same issue in the x265-devel mailing list. Then I learned it is not a good place to ask anymore. (At least when you are not one of the developers yourself and don't want to propose patches?)

Your explanation about additional patches adding to the checksum sounds credible to me.

[tormento]

I have a question about qpfile. I started using them since DGIndexNV began to output it in the format:

frame I -1

What the last -1 stands for? I have looked at the x265 reference but that value remains unknown to me.

[LigH]

x265 documentation: qpfile (https://x265.readthedocs.io/en/master/cli.html#cmdoption-qpfile)

Not explicitly mentioned, but this:
Specifying QP (integer) is optional, and if specified they are clamped within the encoder to qpmin/qpmax.
IIRC, an omitted or invalid value means that you specify only the GOP frame type, but let the encoder decide the quantization.

[tormento]

Not explicitly mentioned, but this
So, -1 has to be considered invalid and I can simply omit it or it has some effect on actual QP?

Unfortunately, I can’t understand this topic in a clear way.

[LigH]

"Specifying QP (integer) is optional" means you can omit that parameter, only specify frame number and frame type. And that should be more obvious than using a -1 value...

[Barough]

I just saw this issue
https://bitbucket.org/multicoreware/x265_git/issues/612/faulty-commit-value-on-compiled-binaries

I don't really remember my bb account and I don't want to create a new one. Since one of the participants, Barough, is here I think I'll post here (is MrB also Barough or just coincidence)
The x265 version is not wrong, it's just that it uses the latest commit hash, and if you use media-autobuild_suite to build x265, it adds one patch here (https://github.com/m-ab-s/media-autobuild_suite/blob/00ef1d393bf2e3f74c9e6470f4a89bc52db68470/build/media-suite_compile.sh#L1717) and that makes the latest commit hash to change.
If you don't want it to interfere you can actually skip that patch, but you will need to rebuild x265 when gcc updates according to the commit message in that patch.

Nah, im not MrB.

So we have to get that m-ab-s patch removed and the m-ab-s compiled x265 binaries will be up to par with the default x265 versioning that MCW shows or have i missunderstood it.

[LigH]

If it still compiles without that 0001-cmake-split-absolute-library-paths-to-L-and-l.patch, then probably yes.

I'll test if compiling fails without...

[Z2697]

Nah, im not MrB.

So we have to get that m-ab-s patch removed and the m-ab-s compiled x265 binaries will be up to par with the default x265 versioning that MCW shows or have i missunderstood it.

Remove / comment out that line in the compile script will be OK

[Barough]

If it still compiles without that 0001-cmake-split-absolute-library-paths-to-L-and-l.patch, then probably yes.

I'll test if compiling fails without...

Looking forward to your results

Remove / comment out that line in the compile script will be OK

Ok

[LigH]

Compiling without the patch works. And the hash matches. So it got removed from M-AB-S.

New upload: x265 4.0+10-304f02f (https://www.mediafire.com/file/5gswu8rcm4rwmbv/x265_4.0+10-304f02f.7z/file)

[Windows][GCC 14.2.0][32b/32b~XP/64b/64b+MV] 8bit+10bit+12bit

[LigH]

They reverted the API change with the multiple video streams. Now ffmpeg doesn't compile anymore, again.

[Z2697]

Reverting API change almost right after release... Isn't that somewhat reckless?

[Selur]

Only if it was part of a release, otherwise, I would say: not really.

[Z2697]

It's 4.1 time
Wow

[benwaggoner]

It's 4.1 time
Wow
x265 development has certainly picked up in 2024!

3.7 and 4.0. BIG speedups for ARM. MV-HEVC. Plus tons of quality improvements, bug fixes, and performance tweaks.

[Ritsuka]

x265 development has certainly picked up in 2024!

3.7 and 4.0. BIG speedups for ARM. MV-HEVC. Plus tons of quality improvements, bug fixes, and performance tweaks.

And many questionable choices, like adding Neon DotProd and I8MM optimisations without any run time check, so it will crash if you compile with those enabled and run it on a CPU that don't implement them.

It's like compiling AVX512 optimisations and trying to run them on a AVX2 cpu.

Oh well, it's not actually hard to add some run time checks.

[asarian]

x265 v4.0+23-487105d
Built on October 11 2024, GCC 14.2.0
Win32/64 / 8bit+10bit+12bit

Yikes, I am still on HEVC encoder version 3.5+2-g2b25c9ba0+45. Maybe I should upgrade. :)

[tormento]

Arrow Lake officially presented.

AVX-512 missing.

Fuck.

I hope it will be on par with Zen5 with AVX-512 or I will have to change cpu brand after 30+ years of Intel use.

Or wait for some cheap used Xeon.

[asarian]

Arrow Lake officially presented.

HEVC encoder version 3.5+2-g2b25c9ba0+45 missing.

Fuck.

I hope it will be on par with Zen5 with AVX-512 or I will have to change cpu brand after 30+ years of Intel use.

Or wait for some cheap used Xeon.

Eh. AVX-512 has always been a nightmare for me (i9 12900K). Problem with AVX-512 is, that it is so good, that it takes too much power. That means, for one, reduced sustained power availability for the entire CPU, and increased heat production; and the clockspeed needs to drop down significantly (in the BIOS) on AVX-512 even.

All extensive x265 testing I ever did with AVX-512 enabled, always worked out worse than just running without. AVX-512 is great on paper, but is, IMHO, indeed best just forgotten (unless you possess professional server cooling).

[LigH]

Sounds like the ratio between gain of efficiency and waste of power was not very comfortable?

But that is rather a separate topic, doesn't concern the development of x265 too much.

[tormento]

All extensive x265 testing I ever did with AVX-512 enabled, always worked out worse than just running without.
Both Zen5 core and last Xeon one show a few watt differences with AVX-512 on on off. That's not a concern anymore.

https://cdn.mos.cms.futurecdn.net/mfYbGGM3VEA8imRWXpSGHm-970-80.png

[LigH]

But AVX-512 support in x265 already exists. So discussing the support in CPU hardware doesn't matter much in this thread...

[benwaggoner]

Both Zen5 core and last Xeon one show a few watt differences with AVX-512 on on off. That's not a concern anymore.
Did you look at fps/watt? That's the more relevant question.

[benwaggoner]

Eh. AVX-512 has always been a nightmare for me (i9 12900K). Problem with AVX-512 is, that it is so good, that it takes too much power. That means, for one, reduced sustained power availability for the entire CPU, and increased heat production; and the clockspeed needs to drop down significantly (in the BIOS) on AVX-512 even.

All extensive x265 testing I ever did with AVX-512 enabled, always worked out worse than just running without. AVX-512 is great on paper, but is, IMHO, indeed best just forgotten (unless you possess professional server cooling).
AVX-512 has gotten better in Zen 5 and more recent Intel architectures. The first iteration seemed to double throughput and then cut internal clock speed in half, so gains were maybe 10% at best.

[asarian]

Both Zen5 core and last Xeon one show a few watt differences with AVX-512 on on off. That's not a concern anymore.

If that small difference in wattage is caused by the rather severe multiplier-throttling beating AVX-512 normally takes on my i9 12900k, then that is, IMHO, already de facto, an 'admission of guilt.' as it were (in that Intel thereby admits AVX-512 cannot run normally, needs reduced speed -- and even then consumes a lot of power -- and is thus effectively a failure).

My i9 12900k is not the latest CPU, though, so I'm sure a Xeon fare better in that regard.

[asarian]

AVX-512 has gotten better in Zen 5 and more recent Intel architectures. The first iteration seemed to double throughput and then cut internal clock speed in half, so gains were maybe 10% at best.

^^ This. Plus, even at reduced speed, the power draw was insane, affecting the max power draw duration of the CPU (set to max); so, overall, negatively impacting the entire job. Not to mention heat issues (with watercooling even).

[Z2697]

AVX-512 is now a lot better than is was, based on some testing results published online, the only AVX-512 cpu I have owned is 7950X at this point. (still using, waiting to upgrade to 9950X)
But the performance benefit in video encoders is still only a few percent.

[Z2697]

If that small difference in wattage is caused by the rather severe multiplier-throttling beating AVX-512 normally takes on my i9 12900k, then that is, IMHO, already de facto, an 'admission of guilt.' as it were (in that Intel thereby admits AVX-512 cannot run normally, needs reduced speed -- and even then consumes a lot of power -- and is thus effectively a failure).

My i9 12900k is not the latest CPU, though, so I'm sure a Xeon fare better in that regard.

https://www.phoronix.com/review/amd-zen5-avx-512-9950x/7
You need to see the full picture.

[Z2697]

Arrow Lake officially presented.

AVX-512 missing.

Fuck.

I hope it will be on par with Zen5 with AVX-512 or I will have to change cpu brand after 30+ years of Intel use.

Or wait for some cheap used Xeon.

30+ years of exclusively using Intel CPU?

Do you mind to share some of your story?

[LigH]

Again, why here in a thread about the development of x265?

[tormento]

Again, why here in a thread about the development of x265?
As you are the only one complaining, eventually evaluate your "in topic" metrics. ;)

[Z2697]

I agree this is off topic, but sometimes a slight off topic is unavoidable.

This starts with an analogy of x265's runtime instruction set detection on ARM chips.

adding Neon DotProd and I8MM optimisations without any run time check

It's like compiling AVX512 optimisations and trying to run them on a AVX2 cpu.


(To be clear, I'm not blaming Ritsuka for this.)

[asarian]

https://www.phoronix.com/review/amd-zen5-avx-512-9950x/7
You need to see the full picture.

I'll leave it be, from here on in, but just read we can forget about AVX-512 on Intel Core Ultra CPU's. So, there's that.

[kurkosdr]

x265 development has certainly picked up in 2024!

3.7 and 4.0. BIG speedups for ARM. MV-HEVC. Plus tons of quality improvements, bug fixes, and performance tweaks.
Nothing surprising here: where there is demand, there is supply. To this day, there is no way to create stereoscopic videos for the Apple Vision Pro other than using the little black box that Apple provides. So, there are lots of people with well-spec'ed PCs used for content creation who want to create stereoscopic videos for the Apple Vision Pro without having to buy a Mac just for that, so there is the demand side.

MultiCoreWare wants their product (x265) to be part of the supply and are moving fast to make it a reality, hence the supply. What remains to be seen is whether it'll be possible to do this using open-source GUI software, since GUIs that use x265 as the encoder need to add support for the feature.

[Z2697]

Nothing surprising here: where there is demand, there is supply. To this day, there is no way to create stereoscopic videos for the Apple Vision Pro other than using the little black box that Apple provides. So, there are lots of people with well-spec'ed PCs used for content creation who want to create stereoscopic videos for the Apple Vision Pro without having to buy a Mac just for that, so there is the demand side.

MultiCoreWare wants their product (x265) to be part of the supply and are moving fast to make it a reality, hence the supply. What remains to be seen is whether it'll be possible to do this using open-source GUI software, since GUIs that use x265 as the encoder need to add support for the feature.

Now some avs and vpy input support will be very convenient for MV-HEVC, otherwise 2 large temp files are required, or maybe named pipe.

8-bits only is disappointing though.

[kurkosdr]

8-bits only is disappointing though.
x265's MV-HEVC encoder doesn't do 10-bit? Not even 10-bit SDR?

[rwill]

x265's MV-HEVC encoder doesn't do 10-bit? Not even 10-bit SDR?

It is more a question of supported Profiles and not core capabilities. Up until recently there was no Multiview Profile with more than 8 bits defined in the H.265 standard. The most recent 07/24 release of the standard then specified a Multiview Profile that also allows for 10 bit.

[Z2697]

The profiles are sometimes too conservative...

[Z2697]

I don't know how Apple is involved in MV-HEVC and Alpha-HEVC but multiview encoding was in AVC already, alpha support is kinda new thing in MPEG world (libvpx supports alpha channel) and probably has more things to do with Apple than multiview.

[kurkosdr]

I don't know how Apple is involved in MV-HEVC and Alpha-HEVC but multiview encoding was in AVC already, alpha support is kinda new thing in MPEG world (libvpx supports alpha channel) and probably has more things to do with Apple than multiview.
The Apple Vision Pro uses MV-HEVC for stereoscopic videos, and Apple provides a tool for converting stereoscopic videos to MV-HEVC-encoded video suitable for the Apple Vision Pro.

This is the only significant use of MV-HEVC so far, that's why people started caring about MV-HEVC only after the reveal of the Vision Pro (note: Blu-ray 3D uses MVC aka the multiview encoding for AVC as you hinted).

[ShortKatz]

FFmpeg is going to mark x265 as experimental because of some "serious issues".
https://ffmpeg.org//pipermail/ffmpeg-devel/2024-October/335119.html

[Z2697]

If it's actually changed, that's breaking their own API (sort of) while blaming mcw break x265 API...
I think the experimental flag should be indicating the implementation in FFmpeg is "not production ready", not really fits this situation? For example, only libx265.c (not including upstream) or native decoders/encoders.

[FranceBB]

Yeah... marking it as experimental after more than 10 years doesn't make any sense. I'm gonna reply to the email chain stating my case hoping to stop this absurdity. Please add your replies as well. The more the merrier.

https://ffmpeg.org//pipermail/ffmpeg-devel/2024-October/335131.html

No, I strongly disagree.
Marking a vital encoder as libx265 that thousands of people rely on as "experimental" after more than 10 years since its inclusion doesn't make any sense and would make more harm than good.

Besides, the experimental flag is supposed to be used to indicate that the implementation in FFmpeg is "not production ready", which doesn't really fit this situation as it's an external open source encoder.

The right thing to do is to contact multicoreware directly and get the bug fixed once and for all.

[rwill]

The right thing to do is to contact multicoreware directly and get the bug fixed once and for all.


It is not a single bug, its multiple and their count grew since certain people have left x265 development. 'Somehow' marking x265 as not stable enough for production is the right thing to do.

[Boulder]

Any kind of proper patch review process went out the door a long time ago, and now there have been so many new things squeezed in that it was bound to create major issues.

[Z2697]

Oops, I searched through commits and found that experimental flag were used for external libraries but rarely, specifically, 3 times: libopenjpeg, libvpx-vp9 and libaom if I didn't miss something and all with a cetrain version check

Here're my assumptions:
API change breaks compilation but does not affect production after patch, ideally.
Memory leak a few hundreds of bytes will affect production in some way, like an encoding server that encodes millions of video streams in a run, but experimental flag mainly affects the command line tools, which won't "save" the sever use case, unless the server is using command line tools, but then the process is probably terminated after each video stream, and OS can reclaim it's memory.
Minor problems. The difference I think is FFmpeg have series of fixes every once in a while, but x265 is... once in a whiiiiiiiiiiiiiiiiiiilllllllllllllllllllllllle.

[FranceBB]

Kirithika Kalirathnam from Multicoreware replied to the mailing list by the way:


Hi al ,

We understand your concern behind marking x265 as experimental but w wanted to lt you know that our x265 deveopment tem is activey working
on resolving multipl issues that have been brought to our notice so fr
through bitbucket issue tracker(including issue #482) and are plnning to
reese v4.1 with al the fixes shortly. We are aso continuously lveling
up our tests to catch the issues beorehand and keep up code hygiene that
we have maintained so fr. Hence we request this forum to reconsider th decision for the beneit of severa x265 users.

*Thanks,*
*Kirithika*




I still think it shouldn't be marked as experimental.
I'm already compiling FFMpeg with a few unmerged patches applied (DolbyED2 decoding) so compiling with yet one more parameter isn't gonna make a difference in my very specific case, but it would have repercussions on plenty of other people, I'm sure.

I'd say "give them a chance".

[Z2697]

Kirithika Kalirathnam from Multicoreware replied to the mailing list by the way:





I still think it shouldn't be marked as experimental.
I'm already compiling FFMpeg with a few unmerged patches applied (DolbyED2 decoding) so compiling with yet one more parameter isn't gonna make a difference in my very specific case, but it would have repercussions on plenty of other people, I'm sure.

I'd say "give them a chance".

I think this is a user standpoint vs developer standpoint situation.
BTW, what happend to you quote? Words are deformed from the actual mail.

[FranceBB]

I think this is a user standpoint vs developer standpoint situation.

Yeah... and as a user I clearly value the practical implications that it would have more than the theoretical ones.


BTW, what happend to you quote? Words are deformed from the actual mail.

No idea, but it's not the actual Doom9 quote, I've got the email like that in my email client, so I think it got truncated/deformed either when the Multicoreware employee sent that to the mailing list or when the server forwarded it to everyone.

[Emulgator]

<JOKE>That might be...the final effort of data reduction, or ? See my signature</JOKE>

[GeoffreyA]

<JOKE>That might be...the final effort of data reduction, or ? See my signature</JOKE>

Perhaps we're advancing into the era of lossy text compression :)

[Z2697]

Sometimes bitbucket says the repo is updated but when I check commits, issues and pull requests there's no update.
Does anyone know why?
I mean, I can guess it's some private change, but I'm not familiar with bitbucket. Are private commits/branches in public repo possible?
https://files.catbox.moe/pfzt3x.png

[tormento]

Did anybody see 9950x x265 benchmarks with and w/o AVX512 enabled?

Asking for Arrow Lake ones is a bit early but please notice them when you see them too.

I feel the urge of changing my rig (I am still on a i7-2600k) but I will wait until AMD shows their Zen 5 3D flavors.

[excellentswordfight]

Did anybody see 9950x x265 benchmarks with and w/o AVX512 enabled?

Asking for Arrow Lake ones is a bit early but please notice them when you see them too.

I feel the urge of changing my rig (I am still on a i7-2600k) but I will wait until AMD shows their Zen 5 3D flavors.
https://www.hwcooling.net/en/how-much-does-avx-512-help-zen-5-in-x265-and-how-to-turn-it-on/

So its the same as from what i've seen from the latest server chips, 5-10% increase.

From what i've seen its very close between 9950x and Core Ultra 9 285K in x265. According to techpowerup they also have pretty much the same perf/w in Cinebench (so we can guestimate that its pretty similar in x265 as well), so from that point on view I guess it doesnt matter which team you choose either.

If its strictly for encoding, do note that the X3D models might be slower, as previous models has been that as there is little gain from the L3 Cache in this workload and the models has been lower clocked.

[Ritsuka]

By the way, it seems they are committing new changes only to a "Release_4.1" branch, and forgetting about the master branch.

[Z2697]

By the way, it seems they are committing new changes only to a "Release_4.1" branch, and forgetting about the master branch.

Now they realized that and pushed a set of commits to master branch just an hour ago.

[benwaggoner]

https://www.hwcooling.net/en/how-much-does-avx-512-help-zen-5-in-x265-and-how-to-turn-it-on/

So its the same as from what i've seen from the latest server chips, 5-10% increase.

From what i've seen its very close between 9950x and Core Ultra 9 285K in x265. According to techpowerup they also have pretty much the same perf/w in Cinebench (so we can guestimate that its pretty similar in x265 as well), so from that point on view I guess it doesnt matter which team you choose either.

If its strictly for encoding, do note that the X3D models might be slower, as previous models has been that as there is little gain from the L3 Cache in this workload and the models has been lower clocked.
At this point, I wouldn't be surprised if Apple's new M4 Max turns out to be the fastest x265 encoding platform on the planet. It is winning in single-core performance, and there's been a LOT of SIMD and other optimizations added in the last couple of years.

It'd be weird and funny if a tricked out Mac Mini is the fastest x265 platform. The mini would certainly win in throughput per rack unit or per watt.

[Z2697]

At this point, I wouldn't be surprised if Apple's new M4 Max turns out to be the fastest x265 encoding platform on the planet. It is winning in single-core performance, and there's been a LOT of SIMD and other optimizations added in the last couple of years.

It'd be weird and funny if a tricked out Mac Mini is the fastest x265 platform. The mini would certainly win in throughput per rack unit or per watt.

But at what cost ($$$:sly:)

[benwaggoner]

But at what cost ($$$:sly:)
US$1599 for the 14 core M4 Pro with 24 GB RAM (should be enough for 4K; the M-series is really memory efficient).

For a cluster, one could upgrade to 10 Gbps ethernet for another $100. Extra RAM and internal storage are pricey, but likely not needed for many applications.

[FranceBB]

At this point, I wouldn't be surprised if Apple's new M4 Max turns out to be the fastest x265 encoding platform on the planet.

If that ever happens, I'm gonna go in the server room and yell to each one of my 56c/112th AVX512 x86_64 Intel Xeon "you're pathetic!".

https://y.yarn.co/e058b4b3-41d8-4850-8e34-89f1b791526a_text.gif

[benwaggoner]

If that ever happens, I'm gonna go in the server room and yell to each one of my 56c/112th AVX512 x86_64 Intel Xeon "you're pathetic!".

https://y.yarn.co/e058b4b3-41d8-4850-8e34-89f1b791526a_text.gif
Yeah, past a certain resolution and frame threads, more cores will start winning over better cores. M4 Max likely wins for 1080p or lower, and possibly 4K if using only 1 frame thread.

Unfortunately the M4 Max is only in MacBook Pro, which is a whole lot more expensive and bigger for headless work. Mac Mini tops out with the M4 Pro currently.

[LigH]

New upload: x265 4.0+42-5203eb5 (https://www.mediafire.com/file/yoww4r7t1qovqet/x265_4.0+42-5203eb5.7z/file)

[Windows][GCC 14.2.0][32b/32b~XP/64b/64b+MV] 8bit+10bit+12bit

[Z2697]

They probably opened release 4.1 branch too early and now they have to push to two branches simultaneously

[LigH]

New upload: x265 4.0+60-e012793 (https://www.mediafire.com/file/xors33eth0xriax/x265_4.0+60-e012793.7z/file)

[Windows][GCC 14.2.0][32b/32b~XP/64b/64b+MV] 8bit+10bit+12bit

contains a lot of important fixes (e.g. of memory leaks) and improvements

[Barough]

x265 v4.0+61-7cc403076
Built on November 13 2024, GCC 14.2.0
Win32/64 / 8bit+10bit+12bit

https://www.mediafire.com/file/i1iuybxy5w3w6g6

[asarian]

x265 v4.0+61-7cc403076
Built on November 13 2024, GCC 14.2.0
Win32/64 / 8bit+10bit+12bit

https://www.mediafire.com/file/i1iuybxy5w3w6g6


A bit confusing. Is your x265 version in competition with that of LigH? And yours is .61 I see.

[LigH]

The difference between v4.0+60 and v4.0+61 is probably:

A few tabs converted to spaces to equalize indentation.

Nothing that would affect the generated code.

[asarian]

The difference between v4.0+60 and v4.0+61 is probably:

A few tabs converted to spaces to equalize indentation.

Nothing that would affect the generated code.


Thanks for the clarification.

[LigH]

More quality related patches (addressing e.g. memory leaks / double frees, crashes) are being submitted. Also pmode and pme will deprecate. Expect commits soonish...

[LigH]

Version 4.1 has been released.

The v4.1 release of x265 is out now. Below is the list of new features, optimization, and bug fixes in this version.

Version 4.1
===========

Release date - 22nd November, 2024.

New feature
-----------
1. Aom Film-Grain characteristics as a SEI message to support Film Grain Synthesis.

API changes
-----------
1. API Support to enable VBV end feature.
2. Reverted the 4.0 api changes.
3. Added command line parameters for Aom film grain characteristics as a SEI message to the bitstream (--aom-film-grain).
4. Added support to configure Bitrate, CRF, and QP at frame level, along with CLI support for frame-level RC configuration (--frame-rc)
5. Declared few params(numapools, csvfn, scalingLists etc) with fixed size to fix memory leaks

Optimizations
-------------
1. Eliminated redundant pointer copies in lowresMC and lowresQPelCost, improving encoding performance by up to 0.9% on modern CPUs.
2. Mcstf optimization - Improved mcstf performance by moving mcstf computation to lookahead and search range optimization.

Bug fixes
---------
1. mcstf crash when using multiple frame threads.
2. CLI parsing for MV-HEVC Encode.
3. segfault and decoder crash in scc.
4. compilation issue in clang.
5. potential stack buffer overflow issues.
6. documentation for b-intra, max-merge, limit-refs and qg-size
7. memory leak fixes for sei, zones, sao, hme, vbv, sbrc, mcstf, alpha, multiview, scc, two pass, rskip, film grain, aq-mode, scaling_list, fades, analysis save and load etc

Known issues
------------
1. Inconsistent output with recon option
2. Output mismatch between analysis save & load with cutree for reuse-levels < 10

Thanks,
Mahesh

[jpsdr]

A lot of commits, but the version is 4.0.0.077, so still a 4.0.0, not 4.1.

[LigH]

You say? ... My MinGW+GCC workflow calculates version 4.1+54-fa27709.

[jpsdr]

Ah... As i have a custom version, maybe something went wrong, even with pushing the tags.
The fact is the exe i built is tagged 4.0.0.77.
Well... i don't realy mind, the more important is to have the commits.

[Z2697]

You say? ... My MinGW+GCC workflow calculates version 4.1+54-fa27709.

4.0+77 and 4.1+54 are both "wrong" (but technically correct ;))
4.0+77 is calculated before the merge of stable to the master (https://bitbucket.org/multicoreware/x265_git/commits/fa2770934b8f3d88aa866c77f27cb63f69a9ed39), tag is not 4.1 because the tagged commit hasn't been merged to "current branch" where he builds the binary (which probably is master).
4.1+54 is calculated after the merge, tag is correct but the distance is greater than is should be, because release branch and master branch are diverged due to the previously mentioned mistake.
By the way, it seems they are committing new changes only to a "Release_4.1" branch, and forgetting about the master branch.
They probably opened release 4.1 branch too early and now they have to push to two branches simultaneously

[LigH]

Their recent code development is remarkable; but their repo mess seriously flaws it.

[MaheshPittala]

The x265 release 4.1 branch was started slightly earlier, requiring patches to be pushed to both the master and release_4.1 branches. As a result, the commit IDs differed between the two branches. When the release/stable branch was merged into master, the commits were treated as new. Using cherry-pick would have been the better solution.

[Z2697]

If you cherry-pick the tagged commit, the new commit hash will also be different, the versioniong cmake code (using git describe) probably won't pick the 4.1 tag.
I think that's just how it goes, no better way to do it.
Just... don't create release branch too early next time, do it right before the real release.

[Joey BoomBats]

Aom Film-Grain characteristics as a SEI message to support Film Grain Synthesis.

This looks like a very useful feature for bit hungry grainy videos.


Has anyone already tested the efficiency improvements of this feature and the visual implications?


Or does anyone know what the command line parameters are?


I see that Handbrake has now included x265 version 4.1 but unfortunately no specific changes have been made to the UI (yet).

[excellentswordfight]

This looks like a very useful feature for bit hungry grainy videos.


Has anyone already tested the efficiency improvements of this feature and the visual implications?


Or does anyone know what the command line parameters are?


I see that Handbrake has now included x265 version 4.1 but unfortunately no specific changes have been made to the UI (yet).
As far as I can see, this is not a complete built-in feature. The parameter added is "--aom-film-grain", so you need to denoise and create a grain parameter file externally, you then point that parameter to that file so it can be injected in to the SEI.

[tormento]

Has anyone tried --frame-dup with Patman (https://github.com/Patman86/x265-Mod-by-Patman/releases) or jpsdr (https://github.com/jpsdr/x265/releases) builds?

It doesn't work on my AVX only CPU.

I get:

Video encoding returned exit code: -1073741819 (0xC0000005)

It's unclear what this exit code means, in case it's a Windows system error then it possibly means:

# for decimal -1073741819 / hex 0xc0000005
STATUS_ACCESS_VIOLATION ntstatus.h
# The instruction at 0x%p referenced memory at 0x%p. The
# memory could not be %s.
# as an HRESULT: Severity: FAILURE (1), FACILITY_NULL (0x0), Code 0x5
# for decimal 5 / hex 0x5
ERROR_ACCESS_DENIED winerror.h
# Access is denied.

[jpsdr]

Does the issue also happen with a not modified build ?

[Z2697]

Who uses frame-dup anyways?

It's better to give more details such as the whole command line and the "variant" of the build used to produce that error, and the input spec, etc.

[rwill]

The interesting thing is that if x265 can frame dupe, the way it is implemented anyway, one could also encode the picture at 1 bit or below per CU and not make the stream variable framerate, cause problems with certain muxers and rely on a correct decoder implementation of picture timing SEIs.

So a very useful and bit saving feature they have there, if only it worked.

[Z2697]

The interesting thing is that if x265 can frame dupe, the way it is implemented anyway, one could also encode the picture at 1 bit or below per CU and not make the stream variable framerate, cause problems with certain muxers and rely on a correct decoder implementation of picture timing SEIs.

So a very useful and bit saving feature they have there, if only it worked.

Using container output (ffmpeg libx265, or supported in some mods) can have "slightly more usable" result, but yeah, currently the feature is working, I guess, but no software can recognize it. Even the "slightly more usable" container output is not utilizing timing SEIs.
The container output is using timestamps support of the container (VFR), while the picture timing SEIs don't contain actual timestamp, but only a flag indicates the corresponding frame should be doubled or tripled, if the decoder is able to support it the decoded video should still be CFR.

Even if it's working properly, both encode and decode side, the use case is still extremely limited.

[tormento]

Who uses frame-dup anyways?
Anime. Mostly old ones.
Does the issue also happen with a not modified build ?
Tried this (http://msystem.waw.pl/x265/x265-4.1+54-fa27709_vs2022.7z) and it works but I don't know if it honors --frame-dup flag.

[Z2697]

Anime. Mostly old ones.

You shouldn't.

To have a better understanding of what this feature means: this feature removes frames based on PSNR thresholding, and signal picture timing SEIs to keep the correct... picture timing... yeah... which no commonly available decoder can recognize.

It also requires VBV and HRD. Since no everyday decoder can recognize the timing SEIs the picture timing will be incorrect if any frame(s) are removed.
If you don't see any incorrect timing,
1) You didn't notice it.
2) You didn't enable the required VBV and HRD.
3) Your video didn't have any frames above the threshold.

Even if it works as it should, the PSNR thresholding is not ideal to begin with, and the bits saved with removing near identical frames are, well, did you know picture timing SEIs cost bits?

[jpsdr]

Does it work on CPU with more than AVX only ?

[Patman]

The reason for the error is the direct support of AVS and VPY scripts (MOD). However, if you use the pipe methods, the encode works. It seems that x265 requires the Y4M or YUV formatfor this case.

[jpsdr]

That's odd, i'm almost sure i used frame-dup on avs scripts with a moded version, during my first attempts of trying to do x265 encodes, before giving up using it because of all the issues i encountered, reasons Z2697 described.
My guess was it could be because of the multiview add, but it was wrong it seems.

[Leo 69]

Does anyone know how to use the new --aom-film-grain feature in the latest x265 encoders (I'm using 4.1+54)? I tried it and it fails each time with either .tbl or even .txt for film grain files. Here's the command:

--crf 18.5 --preset veryslow --output-depth 10 --aom-film-grain grain.tbl

And I always get this error:

x265 [error]: Failed to open Aom film grain characteristics binary file grain.tbl


For grain.txt file the error is the same. Does anyone know how to fix this? Or rather, did anyone manage to make this new feature work at all?

[Z2697]

Does anyone know how to use the new --aom-film-grain feature in the latest x265 encoders (I'm using 4.1+54)? I tried it and it fails each time with either .tbl or even .txt for film grain files. Here's the command:

--crf 18.5 --preset veryslow --output-depth 10 --aom-film-grain grain.tbl

And I always get this error:

x265 [error]: Failed to open Aom film grain characteristics binary file grain.tbl


For grain.txt file the error is the same. Does anyone know how to fix this? Or rather, did anyone manage to make this new feature work at all?

I think it expects "raw" SEI payload, just guessing, haven't tried to understand it, and probably never will because I don't think it's (I'm referring to all the current implementations of FGS features) a useful feature, until it's much more improved.

[benwaggoner]

The interesting thing is that if x265 can frame dupe, the way it is implemented anyway, one could also encode the picture at 1 bit or below per CU and not make the stream variable framerate, cause problems with certain muxers and rely on a correct decoder implementation of picture timing SEIs.

So a very useful and bit saving feature they have there, if only it worked.
Yeah, if only it worked! I've never been able to get a single frame to duplicate as documented, even with some pretty extreme settings. I've not tried in a 4.x build, but don't recall any patches for that part.

Maybe it only works in 2-pass or something?

[benwaggoner]

I think it expects "raw" SEI payload, just guessing, haven't tried to understand it, and probably never will because I don't think it's (I'm referring to all the current implementations of FGS features) a useful feature, until it's much more improved.
The tool is reasonably useful with the proper tools; there just aren't any good open source ones currently. The hardest part is the degrain-and-parameterize before encoding.

The rendering has some challenges with repeated patterns, due to the "drunken walk" nature of selecting a random 32x32 block out of a 64x64 block: pixels near the middle are much more heavily sampled than those near the edge. I so wish computer science degrees required some basic statistics! I work with a lot of engineers who are whizzes at linear algebra, but go blank when I ask "is that statistically significant?"

[Z2697]

Yeah, if only it worked! I've never been able to get a single frame to duplicate as documented, even with some pretty extreme settings. I've not tried in a 4.x build, but don't recall any patches for that part.

Maybe it only works in 2-pass or something?

I'm not sure "which" duplication you are referring to, the first case is the encoding side:

What it does is actually removing frames and using SEI to tell decoder to duplicate existing frames, which actually should be called de-duplication I think.

Maybe you are not looking at the right place (i.e. you think it's duplicating frames while it actually deletes frames), but assuming you are not able to get frames to trigger the de-duplication during encoding:

There's PSNR thresholding, can be configured via --dup-threshold parameter.
If you use low enough threshold, eventually some frames will be de-duped, but of course this should only be used in experiments, low threshold will just destroy the video.

The second case: the decoding side...

You need a decoder that's able to recognize and utilize the picture timing SEI (which only tells decoder to double or triple the frame, no actual timestamp is stored) and I guess things will... "just work"... yeah, who knows, the most common decoder (avcodec) doesn't support it so I can't test.


The tool is reasonably useful with the proper tools; there just aren't any good open source ones currently. The hardest part is the degrain-and-parameterize before encoding.

The rendering has some challenges with repeated patterns, due to the "drunken walk" nature of selecting a random 32x32 block out of a 64x64 block: pixels near the middle are much more heavily sampled than those near the edge. I so wish computer science degrees required some basic statistics! I work with a lot of engineers who are whizzes at linear algebra, but go blank when I ask "is that statistically significant?"

Maybe I just hate noises in general :o
But I agree, it (FGS) has potential... just it still has a long way to go.

[LunaRabbit]

Hello Doom9! It's been nearly 20 years since I last posted here although I've been lurking on and off all these years. I recently got back into this old hobby and have greatly appreciated the information ITT. Since I'd been away since the early x264 days and it took several weeks to get up to speed with x265. If it wasn't for the information ITT I would have spent many more weeks aimlessly testing stuff I'm sure. I have a bunch of questions. I hope you don't mind if I ask them. :)

1) --frame-rc / https://bitbucket.org/multicoreware/x265_git/commits/66352d83ae10330f9528b08238396de62dc618c1

Can someone please explain how to work with this recent commit? Can I just define this stuff in my qpfile? If so, what syntax should I use? There is no documentation about this that I can find and it isn't really explained on the git repo from what I've seen. I've been slowly trying to work my way through ratecontrol.cpp to understand it but I haven't had a chance to throw random stuff in a qpfile and check yet (I will soon when I'm back at home). Has anyone used this feature yet?

I am very interested in this since I already manually create a qpfile anyway to set I-frames at particular frames for various reasons (seeking speed and ensuring frames with overlays get an I-frame when needed). I'm already creating qpfiles by hand for each source I work with. So the ability to manually set CRF/QP/Bitrate would be a much welcomed addition. Up until now I've been having to manually make zones and multiply the bitrate up/down which is less than ideal.

2) Misc. cutree stuff and mods

I did a lot of testing with cutree on/off last year and visually I frankly couldn't see much difference but turning off cutree would sometimes greatly increase bitrates for little gain. The extra bits didn't seem worth what I was getting. I currently "reign in" cutree by tweaking --qcomp typically at something like 0.7 or 0.8 at most. Boulder posted a diff file ITT several pages ago and the repo linked in this post (https://forum.doom9.org/showpost.php?p=2005461&postcount=2) is supposed to allow tweaking cutree directly instead of in the round about way with qcomp. But I've also heard from many people over the last few years that I shouldn't use cutree at all because it's a

Bad implementation of mbtree from x264

I do not believe this because in my experience cutree hasn't degraded visual quality at the CRFs I typically use (15-18 or so). I suspect this is one of those hold over old wives tales from the early days of mbtree in x264 when it wasn't fleshed out yet. I'll be the first to admit I don't fully understand how cutree works in x265. I've been reading the source code in an attempt to better understand it. I've also spent many hours reading various posts here. But there really isn't a lot of good technical discussion about it that I can find. Perhaps it's happening on the mailing list somewhere that I haven't seen yet?

I would greatly appreciate hearing your opinions on cutree and if using the above modifications (along with the hours of testing involved) would be worth my time. I've been building x265 from source for the last year or so. But I only use a handful of modifications and mainly for the AQ modes that aren't in mainline. Which brings me to my next and last question for now;

3) Auto-aq

Earlier this year I built a modded version of x265 from I believe version 3.6 or thereabouts. It included the auto-aq mode which I quickly found to be a huge benefit for the sources I work with. In the version I had it was invoked as so;

--aq-auto 10

Today I got a chance to catch up on things and noticed x265 had developed a lot since I was away. I also quickly found out that this mode no longer worked as it did just a few months ago. Through trail and error and some lurking on the git where it's hosted I found out I can now invoke it like this

--auto-aq

But I didn't see any discussion for why it changed and if you can still supply it with an option (e.g. 6 vs. 10 for HDR or SDR content). Does it automatically determine what kind of source you're working with now? Please advise. I spent a few hours this morning trying to find the exact commit where it was changed and why along with trying to grok posts from here. But there were over 20 pages of new discussion ITT and my searches of the forum's database proved futile.

I do have more questions about how x265 works and some of the settings I've been using for the last 2 years or so. I've gotten really good results at acceptable bitrates so I'm happy with them. But improvements are always welcomed of course. My workstation is getting a bit long in the tooth these days and I'm due for an upgrade. So I'm not able to do as much testing as I'd like. My usual config is already lucky to obtain 1fps. Which means a typical project for me takes several days and sometimes weeks to finish. Although the whole manually creating qpfiles and the type of editing/filtering I do takes up far more time. As I'm usually stepping through things frame-by-frame and splicing together sometimes 4 different sources to restore the content I work with.

This is getting verbose so I'll hold off on asking more questions even though I'm full of them. It's good to be back. I'm happy to see this community is still very active. I prefer using forums to these modern locked down social media networks or God forbid Discord.

Also, thanks again to the kind admin that allowed me to post this today instead of a week from now.

[Z2697]

Some (paranoid) people have been disabling mbtree since x264, no wonder they'll disable cutree as well.
The real problem is not even whether cutree is a bad mbtree mimic or not, they don't even use mbtree to begin with.

[LunaRabbit]

Some (paranoid) people have been disabling mbtree since x264, no wonder they'll disable cutree as well.
The real problem is not even whether cutree is a bad mbtree mimic or not, they don't even use mbtree to begin with.

Yeah I understand that part of it. In their defense early mbtree wasn't that great from what I remember. But I'm sure it improved some after I stopped paying attention and got busy with life.

My limited understanding of cutree is that it isn't the same as mbtree at all. From what I understand cutree is more about looking ahead and attempting to either insert more I-frames or shift data in a GOP towards I-frames. With the hope that the following P/B frames will benefit and thus be smaller. I might be wrong but I've always understood it as being more like lookahead than what mbtree was. But I don't pretend to understand the inner workings of mbtree either.

It's just frustrating that the only discussion I see about cutree is either "turn it off it'll destroy quality" or "leave it on but I don't really know what it's doing. It just saves bits". I want to have a better understanding of how it works.

I do know for my uses/config it does save bitrate without affecting quality much that I can see. I suppose if I pixel peep I could spot very mild drop in quality here and there. But over the course of an entire episode or movie it seems to save a lot of bitrate and produce something of equal quality if nothing else is changed.

But perhaps that has more to do with the fact that I used closed GOPs and usually VBV. I do understand GOPs in x265 fine. I think I understand VBV as well and I don't think it really hurts quality that much the way I use it. I use it to impose an upper limit on the buffer so my encodes play nicer with some set-top devices and over the network. As I want to stream my stuff over my LAN and sometimes over WAN when I'm away from home. I was going to ask about VBV another time. But I only bring it up because maybe that's why I'm not seeing any improvement with cutree off.

The bitrate savings for me with cutree are very good. In a clip of say 100 frames with cutree on and closed_GOP set to 24 frames I'll produce a video stream that's about 1.5-2.0MB. The --no-cutree version would be 3.5-4MB with no gain in quality that I can see. Although I do "reel it in" with qcomp. But perhaps that's placebo. I'm mainly doing that in an attempt to prevent much variation between keyframes and non-keyframes.

Some guidance concerning cutree, how it relates to qcomp and if these modifications to tweak cu-tree are worth exploring would be very helpful and appreciated. Since I was surprised to find so little discussion about it outside of; "turn it off" or "if you're leaving it on make sure to raise qcomp a bit".

I feel like there are a lot of things like this in x265 that is just advice being repeated for the last 10+ years from people assuming it works just like early x264. Some time way back when everyone just decided to do things a certain way and never bothered to do tests as the underlying code and the codec itself changed.

I do know that people claim cutree can hurt backgrounds and scenes with a lot of movement. But in my experience it doesn't. But again. Perhaps I'm not seeing these issues because 1) I use a low CRF for 720p-1080p content anyway and 2) I regularly boost bitrate through the use of zones.

Do you use cutree?

[GeoffreyA]

As you point out, cutree saves a lot of bitrate, sometimes close to -50% in anime, and the same goes for mbtree. Those savings can go to smaller CRFs for example. So, I do not understand what people gain by disabling these tools. There is a small increase in quality on certain parts of the frame, but you've got to look for it, and I think one would better spend bits by lowering the CRF.

Recently, encoding a set of anime at reference quality, I settled on leaving cutree on and using a low CRF (14). I also set the rc-lookahead to 240 because a longer lookahead is said to be more beneficial for cutree.

I think if one is encoding anime in particular, disabling mbtree or cutree is a massive waste of bits, and a 2-pass encode would likely suffer more for it. As for how it works, my limited understanding is that it increases the quantiser for parts of a frame that are less referenced temporally. Dark Shikari wrote a paper about mbtree back in the day but it was a bit technical.

[Boulder]

I recently requested jpsdr to include the cutree strength mod (in addition to those couple of other interesting AQ related tweaks) and it might happen if he just has the time for that. Then it would be easy to control cutree if it seems too strong, I also don't believe that it should be disabled completely. Raising qcomp by 0.1 makes cutree much less effective so that's probably why it has been mentioned that qcomp should be raised a bit.

[Z2697]

I recently requested jpsdr to include the cutree strength mod (in addition to those couple of other interesting AQ related tweaks) and it might happen if he just has the time for that. Then it would be easy to control cutree if it seems too strong, I also don't believe that it should be disabled completely. Raising qcomp by 0.1 makes cutree much less effective so that's probably why it has been mentioned that qcomp should be raised a bit.

I've tested the "cutree-strength" mod and found that it has basically the same effect as qcomp. Not really useful IMO.
Note that I used 2-pass bitrate control in the testing because the "cutree-strength" strongly impacted the CRF rate control (iirc).

[Boulder]

I've tested the "cutree-strength" mod and found that it has basically the same effect as qcomp. Not really useful IMO.

Do you mean increasing qcomp and increasing CRF as well to match bitrate?

[Z2697]

Do you mean increasing qcomp and increasing CRF as well to match bitrate?

I use 2-pass, as I edited in the previous post.
Because without mod, the "cutree-strength" is internally calculated from qcomp, changing the "cutree-strength" without touching qcomp seems to just confuses the CRF rate control so much that iirc, makes them off-chart.
Edit: I just did a quick encode with that mod, obviously I didn't remember correctly... not off-chart.

[LunaRabbit]

I recently requested jpsdr to include the cutree strength mod (in addition to those couple of other interesting AQ related tweaks)

Has anything interesting been added to those mods for AQ in the last 4 months or so? Why did the flag to enable it change? Do you still need to set it manually for SDR (e.g. --aq-auto 10 in the older version).

I was getting very good results with --aq-auto 10 for SDR content over the last several months. But obviously now my copy/pasted general settings no longer work. I did some testing with --auto-aq but I'm not sure how I'm supposed to be using it. Is it just the same thing with a different flag?

Edit: I just did a quick encode with that mod, obviously I didn't remember correctly... not off-chart.

What kind of differences are you seeing as opposed to just raising qcomp a bit? I mainly use CRF/1-pass mode. I haven't used 2-pass in a long time (since the xvid days) but if there is an appreciable gain in quality for the same file size I'd be willing to go back. I'll do some testing when I get some time after the holidays are over.

Thanks for the explanation everyone.

Edit: My hope was I could leave qcomp near the default of 0.6 and tweak cutree directly with the goal of maybe shaving off some file size. My CRF is already set so low that I don't want to reduce it any further. Since I try to keep half hour content under 1GB for HD content if possible (lower is obviously better and it varies from episode to episode).

[Leo 69]

I think it expects "raw" SEI payload, just guessing, haven't tried to understand it, and probably never will because I don't think it's (I'm referring to all the current implementations of FGS features) a useful feature, until it's much more improved.

You know, no matter what I tried, I still haven't been able to get it working. This feature is a nice bonus to x265 and I'd say very much needed for my particular use case.

I'm not a developer but I can partly understand and navigate the source code. So I headed to:

https://bitbucket.org/multicoreware/x265_git/src/master/

I found all portions of code related to --aom-film-grain and found that the encoder actually expects .bin files as input for grain parameters. I also found specifications as to what parameters that SEI payload should include. So I grabbed all of that code and fed it to o1-preview and Claude Sonnet 3.5 and asked them to provide me with compatible .bin files.

Both of them, based on those source code snippets, were able to give me a python script to produce a (supposedly) compatible grain.bin file and that's what I did, eventually. I got those .bin files. However, no matter what I did for about 3 hours, I still couldn't get this feature to work. x265 always throws error that it cannot read characteristics of the grain file.

I'm baffled. I tried to register on bitbucket, but it requires a corporate account to do so, so mere mortals cannot register so they're effectively blocked out from asking questions to developers on that platform. And nobody provided any examples of those .bin files and in what way they can actually be acquired.

The 4.1 release notes are vague, the new commandline parameter is vague. There are no examples anywhere and there's no one that I can ask. I have no idea how to solve this and make it work. Apparently, a simple function of a modern video encoder requires brains of Einstein to even get started with it. Very frustrating to be honest. If anyone can provide any insights, that would be awesome.

[Boulder]

You know, no matter what I tried, I still haven't been able to get it working. This feature is a nice bonus to x265 and I'd say very much needed for my particular use case.

I'm not a developer but I can partly understand and navigate the source code. So I headed to:

https://bitbucket.org/multicoreware/x265_git/src/master/

I found all portions of code related to --aom-film-grain and found that the encoder actually expects .bin files as input for grain parameters. I also found specifications as to what parameters that SEI payload should include. So I grabbed all of that code and fed it to o1-preview and Claude Sonnet 3.5 and asked them to provide me with compatible .bin files.

Both of them, based on those source code snippets, were able to give me a python script to produce a (supposedly) compatible grain.bin file and that's what I did, eventually. I got those .bin files. However, no matter what I did for about 3 hours, I still couldn't get this feature to work. x265 always throws error that it cannot read characteristics of the grain file.

I'm baffled. I tried to register on bitbucket, but it requires a corporate account to do so, so mere mortals cannot register so they're effectively blocked out from asking questions to developers on that platform. And nobody provided any examples of those .bin files and in what way they can actually be acquired.

The 4.1 release notes are vague, the new commandline parameter is vague. There are no examples anywhere and there's no one that I can ask. I have no idea how to solve this and make it work. Apparently, a simple function of a modern video encoder requires brains of Einstein to even get started with it. Very frustrating to be honest. If anyone can provide any insights, that would be awesome.

I think the mailing list is your best bet: x265-devel@videolan.org

It's plain silly that they didn't choose the simple grain table file method which is already used by at least aomenc and SVT-AV1.

[Boulder]

Has anything interesting been added to those mods for AQ in the last 4 months or so? Why did the flag to enable it change? Do you still need to set it manually for SDR (e.g. --aq-auto 10 in the older version).

I was getting very good results with --aq-auto 10 for SDR content over the last several months. But obviously now my copy/pasted general settings no longer work. I did some testing with --auto-aq but I'm not sure how I'm supposed to be using it. Is it just the same thing with a different flag?

The parameter should still be --aq-auto x. Nothing has changed there, so the same parameters will apply (10 for SDR, 6 for HDR if you want to enable all the tweaks it has).

[Z2697]

So, auto-aq is a resurrected form of "SBRC" feature that was "deprecated" from official x265, but then in release note 3.6 the first one is "SBRC", huh... I guess they "repurposed" that name?

[Boulder]

So, auto-aq is a resurrected form of "SBRC" feature that was "deprecated" from official x265, but then in release note 3.6 the first one is "SBRC", huh... I guess they "repurposed" that name?

I think someone mentioned that they committed a wrong set of patches under SBRC at first :D

[LunaRabbit]

The parameter should still be --aq-auto x. Nothing has changed there, so the same parameters will apply (10 for SDR, 6 for HDR if you want to enable all the tweaks it has).

I must have cloned the wrong repo or something. As I couldn't get --aq-auto to work no matter what I did. I'll try again later today thanks.

Do you happen to know anything about the --frame-rc stuff I asked about and if it can be controlled from a qpfile? I'm having the same issue as the person above. The release notes are very sparse about how these new features work. I'll see if I can find an archive of the mailing list in the mean time.

[Boulder]

I must have cloned the wrong repo or something. As I couldn't get --aq-auto to work no matter what I did. I'll try again later today thanks.

Do you happen to know anything about the --frame-rc stuff I asked about and if it can be controlled from a qpfile? I'm having the same issue as the person above. The release notes are very sparse about how these new features work. I'll see if I can find an archive of the mailing list in the mean time.

jpsdr's repo default is 'master' but the mods are in the other branch 'x265_mod'.

Unfortunately I don't know anything about the --frame-rc parameter. Nothing in the online CLI docs either, which is not unexpected as documentation seems to be irrelevant :(

[LunaRabbit]

jpsdr's repo default is 'master' but the mods are in the other branch 'x265_mod'.

Unfortunately I don't know anything about the --frame-rc parameter. Nothing in the online CLI docs either, which is not unexpected as documentation seems to be irrelevant :(

Hey thanks for your help again. I was probably just being dumb and probably built the wrong source directory on my local system. I'm sure I'll figure it out when I have more time. I checked again and my last build doesn't have --aq-auto at all. So it's either an unmodded version (I think it's patman's) or I messed up and used the official repo.

If anyone is curious I tried patching the .diff you posting a few pages back against v4.1 from master. As expected, so much changed it can't apply automatically anymore. But I plan to go through the source code later today to see if I can get it working. Here is the output from patch;

patching file README.md
patching file build/MSYS_jpsdr/Build_Win32.sh
patching file build/MSYS_jpsdr/Build_Win32_AVX.sh
patching file build/MSYS_jpsdr/Build_Win32_AVX2.sh
patching file build/MSYS_jpsdr/Build_Win32_Broadwell.sh
patching file build/MSYS_jpsdr/Build_Win32_x86.sh
patching file build/MSYS_jpsdr/Build_mcf_10_Broadwell.sh
patching file build/VS2019_LLVM/Command_Line.txt
patching file build/VS2019_LLVM/Create_Solutions_x64.bat
patching file build/VS2019_LLVM/Create_Solutions_x86.bat
patching file doc/reST/cli.rst
patching file source/CMakeLists.txt
patching file source/abrEncApp.cpp
Hunk #2 NOT MERGED at 1173-1178.
patching file source/avisynth/avisynth.h
patching file source/avisynth/avisynth_c.h
patching file source/avisynth/avs/alignment.h
patching file source/avisynth/avs/arch.h
patching file source/avisynth/avs/capi.h
patching file source/avisynth/avs/config.h
patching file source/avisynth/avs/cpuid.h
patching file source/avisynth/avs/filesystem.h
patching file source/avisynth/avs/minmax.h
patching file source/avisynth/avs/posix.h
patching file source/avisynth/avs/types.h
patching file source/avisynth/avs/version.h
patching file source/avisynth/avs/win.h
patching file source/cmake/Version.cmake
patching file source/common/CMakeLists.txt
patching file source/common/common.cpp
patching file source/common/common.h
patching file source/common/event.cpp
patching file source/common/event.h
patching file source/common/frame.cpp
patching file source/common/frame.h
patching file source/common/lowres.cpp
patching file source/common/lowres.h
patching file source/common/param.cpp
Hunk #1 merged at 154-155.
Hunk #18 NOT MERGED at 1378-1392.
Hunk #27 NOT MERGED at 2449-2455.
Hunk #28 NOT MERGED at 2478-2488.
Hunk #29 NOT MERGED at 2530-2544,2546-2553.
Hunk #30 merged at 2607.
Hunk #31 NOT MERGED at 2638-2646.
Hunk #35 merged at 2907-2908.
Hunk #37 merged at 3003-3005.
patching file source/common/param.h
patching file source/common/threading.h
patching file source/common/x86/asm-primitives.cpp
patching file source/common/x86/loopfilter.asm
Hunk #1 already applied at 3870.
patching file source/encoder/api.cpp
Hunk #1 merged at 120.
patching file source/encoder/encoder.cpp
Hunk #1 merged at 292, NOT MERGED at 298-302.
Hunk #4 NOT MERGED at 2169-2184,2186-2197, merged at 2199-2202, merged at 2205, merged at 2210.
Hunk #5 NOT MERGED at 2536-2542.
Hunk #7 merged at 3748,3750-3751.
Hunk #11 NOT MERGED at 4618-4625, merged at 4627.
patching file source/encoder/frameencoder.cpp
Hunk #1 merged at 467.
patching file source/encoder/ratecontrol.cpp
patching file source/encoder/ratecontrol.h
patching file source/encoder/rdcost.h
patching file source/encoder/search.cpp
patching file source/encoder/slicetype.cpp
patching file source/encoder/slicetype.h
patching file source/input/avs.cpp
patching file source/input/avs.h
patching file source/input/input.cpp
Hunk #2 NOT MERGED at 41-53.
patching file source/input/input.h
patching file source/input/vpy.cpp
patching file source/input/vpy.h
patching file source/input/y4m.cpp
Hunk #1 NOT MERGED at 43-48.
patching file source/input/y4m.h
Hunk #1 NOT MERGED at 58-62.
patching file source/input/yuv.h
patching file source/output/gop.h
patching file source/output/gop_engine.hpp
patching file source/output/output.cpp
patching file source/vapoursynth/VSConstants4.h
patching file source/vapoursynth/VSHelper4.h
patching file source/vapoursynth/VSScript4.h
patching file source/vapoursynth/VapourSynth4.h
patching file source/x265.h
patching file source/x265cli.cpp
Hunk #4 NOT MERGED at 106-122.
Hunk #6 merged at 163-165.
Hunk #7 NOT MERGED at 306-328, merged at 330, merged at 332-336.
Hunk #9 merged at 565-572, NOT MERGED at 574-601, NOT MERGED at 604-611.
Hunk #11 NOT MERGED at 978-989.
Hunk #12 merged at 1112.
Hunk #13 NOT MERGED at 1201-1207.
patching file source/x265cli.h
Hunk #5 merged at 450.
Hunk #6 NOT MERGED at 505-511.
patching file source/x265res.manifest.in


So there aren't many lines that changed. I'm sure I can manually patch in what patch couldn't do automatically.

[Boulder]

Hey thanks for your help again. I was probably just being dumb and probably built the wrong source directory on my local system. I'm sure I'll figure it out when I have more time. I checked again and my last build doesn't have --aq-auto at all. So it's either an unmodded version (I think it's patman's) or I messed up and used the official repo.

If you are on Windows, jpsdr does offer pre-compiled binaries in GitHub.

Fixing the patch file can be a tedious job but just takes time to sit down and work through the errors one by one. The original changes are from so long time ago that the codebase has changed quite a lot since then.

[Z2697]

Here're 2 test results of "cutree-strength" I just ran.
I leave them as links because they are long images.
Please ignore the speed, I ran them in VM.

https://files.catbox.moe/4qugbj.png
https://files.catbox.moe/1tqlou.png

The curves from "corresponding" qcomp and "cutree-strength" values are almost completely aligned with each other, maybe not easy to see in CRF results since the final bitrate differs quite a lot, so I did a 2-pass test.

(qcomp 0.65 corresponds to cutree-strength 1.75 and qocmp 0.7 corresponds to cutree-strength 1.5)

Also keep in mind this is not a valid quality comparison across different "qcomp and cutree-strength pairs", the matrics usually "performs" poorly when it comes to "bits re-distribution". (my layman's understanding is: aq is bits re-distribution within frame, cutree/mbtree is bits re-distribution across frames)

Update:
I ran a CRF test with more data points but only 2 sets of data. Make the interpolated curve more accurate and the "alignment" easier to see.
And switched to a test build I did back in the day.
The points are from CRF 12 to 32.
https://files.catbox.moe/e5jpea.png

[LunaRabbit]

If you are on Windows, jpsdr does offer pre-compiled binaries in GitHub.

Yes I'm aware. I just prefer to build from source even for the Windows machine I still have kicking around. I'm used to is since I normally live in Emacs. :)

Whenever I get it working I'll update the .diff and maybe build some bins for anyone that wants to try it. Just trying to leave things nicer for the next person. I do appreciate having a diff to work from it's really helpful.


Here're 2 test results of "cutree-strength" I just ran.
I leave them as links because they are long images.
Please ignore the speed, I ran them in VM.

https://files.catbox.moe/4qugbj.png
https://files.catbox.moe/1tqlou.png

The curves from "corresponding" qcomp and "cutree-strength" values are almost completely aligned with each other, maybe not easy to see in CRF results since the final bitrate differs quite a lot, so I did a 2-pass test.

(qcomp 0.65 corresponds to cutree-strength 1.75 and qocmp 0.7 corresponds to cutree-strength 1.5)

Also keep in mind this is not a valid quality comparison across different "qcomp and cutree-strength pairs", the matrics usually "performs" poorly when it comes to "bits re-distribution". (my layman's understanding is: aq is bits re-distribution within frame, cutree/mbtree is bits re-distribution across frames)

Interesting thanks for testing it. Whenever I get it working I'll run some tests and see how it compares to my older settings with qcomp 0.7 and 0.8. Metrics are nice but all I really care about is if there is any gain that I can see in the material. I have a source I'm very familiar with that should serve as a good test for seeing if cutree-strength makes any difference. As I need to re-do the whole thing due to some improvements in the filters it requires as of late.

[Z2697]

frame-rc enables rate control mode (CRF, ABR or CQP) to be reconfigured per-frame, I think.

It enables the control, but how to actually use control... zones? qpfile? api calls? IDK.

[Z2697]

Yes I'm aware. I just prefer to build from source even for the Windows machine I still have kicking around. I'm used to is since I normally live in Emacs. :)

Whenever I get it working I'll update the .diff and maybe build some bins for anyone that wants to try it. Just trying to leave things nicer for the next person. I do appreciate having a diff to work from it's really helpful.



Interesting thanks for testing it. Whenever I get it working I'll run some tests and see how it compares to my older settings with qcomp 0.7 and 0.8. Metrics are nice but all I really care about is if there is any gain that I can see in the material. I have a source I'm very familiar with that should serve as a good test for seeing if cutree-strength makes any difference. As I need to re-do the whole thing due to some improvements in the filters it requires as of late.

But why not build it directly from the mod branch? Why "extract" the patch and apply to master branch then build? Seems unnecessary.

As for the metric I think it works OK to evaluate the similarity of the "corresponding qcomp and cutree-strength pairs", it's just not valid when comparing different pairs.
I used my eyes as well, of course. Just the summary report is easier to post.

[LunaRabbit]

frame-rc enables rate control mode (CRF, ABR or CQP) to be reconfigured per-frame, I think.

It enables the control, but how to actually use control... zones? qpfile? api calls? IDK.

That's what I'm wondering as well. If it can be enabled from the qpfile and just work. The docs do not mention anything about what kind of syntax is expected.

But why not build it directly from the mod branch? Why "extract" the patch and apply to master branch then build? Seems unnecessary.

As far as I'm aware there are no mod branches that have the cu-tree modifications working on v4.1 of x265. If you know of one please share.

I do appreciate the testing and providing the helpful chart. Didn't mean to imply otherwise. :)

[Z2697]

That's what I'm wondering as well. If it can be enabled from the qpfile and just work. The docs do not mention anything about what kind of syntax is expected.



As far as I'm aware there are no mod branches that have the cu-tree modifications working on v4.1 of x265. If you know of one please share.

I do appreciate the testing and providing the helpful chart. Didn't mean to imply otherwise. :)

I think it would be easier to cherry-pick the related commits of cutree-strength mod into jpsdr's x265_mod branch, the code behind it is way less.

[jpsdr]

About mbtree and x264.
Disabling mbtree is recommended when you target Blu-Ray, according mp3dom results and tests. But i think it's the only case (or of course, cases using very very similar encode parameters). The issue seems to be the small (1s) keyint value, not working well with mbtree. With standard [250] keyint value, mbtree is doing a proper job (still according mp3dom).

[GeoffreyA]

About mbtree and x264.
Disabling mbtree is recommended when you targer Blu-Ray, according mp3dom results and tests. But i think it's the only case (or of course, cases using very very similar encode parameters). The issue seems to be the small (1s) keyint value, not working well with mbtree. With standard [250] keyint value, mbtree is doing a proper job (still according mp3dom).

And DS's paper: https://archive.org/download/x264_mbtree/x264_mbtree.pdf

[Z2697]

Hmm, something doesn't feel right.

source/encoder/slicetype.cpp

void Lookahead::cuTree(Lowres **frames, int numframes, bool bIntra)
{
...

double totalDuration = 0.0;
for (int j = 0; j <= numframes; j++)
totalDuration += (double)m_param->fpsDenom / m_param->fpsNum;

double averageDuration = totalDuration / (numframes + 1);


I'm "investigating" (f* around and find out) the "inconsistency in bitstreams encoded by different compiler target flags produced binaries" which reportedly is associated with cutree.
Me, being a incompetent wannabe programmer, also kind of narrowed things down (or, rather, "confirmed") to cutree after, what, like 4 days? And is now moderately sure the quoted codes (in combination with some compiler / ISA optimization) caused the inconsistency.
(only averageDuration is used in the following codes)
As x265 itself does not have variable framerate awareness, this block code is unnecessary, I think, all these code will just result us back to "(double)m_param->fpsDenom / m_param->fpsNum", theoretically, if not the weirdness of FP math is happening, you know, the .1+.2 != .3 and non-associative stuff.


double averageDuration = (double) m_param->fpsDenom / m_param->fpsNum;


After replacing the loop with just this line, things seemed to be consistent. (But not consistent with "pre-this-modification" version)
Alternatively you can put "#pragma GCC novec" or other compiler's equivalent right before the for loop. (This retains the consistency with "pre-this-modification" "nocona" (default -march of GCC) / "SSE3" version) But this is compiler specific.

I presonally think the former "solution" is more elegant.

The "inconsistency between bitstreams encoded by different compilers produced binaries" still exists after this modification. (e.g. GCC vs Clang)

Some suspicious executables (10bit only)
https://files.catbox.moe/nhla9t.7z

Am I missing something? Please help me. plzzzzzzzz

[rwill]

One way to proceed would be to make a small repro case test application and then objdump -S that, checking what the differences in the generated code are.

[GeoffreyA]

Hmm, something doesn't feel right.

source/encoder/slicetype.cpp

void Lookahead::cuTree(Lowres **frames, int numframes, bool bIntra)
{
...

double totalDuration = 0.0;
for (int j = 0; j <= numframes; j++)
totalDuration += (double)m_param->fpsDenom / m_param->fpsNum;

double averageDuration = totalDuration / (numframes + 1);


I'm "investigating" (f* around and find out) the "inconsistency in bitstreams encoded by different compiler target flags produced binaries" which reportedly is associated with cutree.
Me, being a incompetent wannabe programmer, also kind of narrowed things down (or, rather, "confirmed") to cutree after, what, like 4 days? And is now moderately sure the quoted codes (in combination with some compiler / ISA optimization) caused the inconsistency.
(only averageDuration is used in the following codes)
As x265 itself does not have variable framerate awareness, this block code is unnecessary, I think, all these code will just result us back to "(double)m_param->fpsDenom / m_param->fpsNum", theoretically, if not the weirdness of FP math is happening, you know, the .1+.2 != .3 and non-associative stuff.


double averageDuration = (double) m_param->fpsDenom / m_param->fpsNum;


After replacing the loop with just this line, things seemed to be consistent. (But not consistent with "pre-this-modification" version)
The "inconsistency between bitstreams encoded by different compilers produced binaries" still exists after this modification. (e.g. GCC vs Clang)

Am I missing something? Please help me. plzzzzzzzz

You're right because all that loop does is multiply (Denom / Numer) by (numframes + 1), using repeated addition, and those values do not change within the loop. I wonder if there is some reason for such a superfluous piece of code. Perhaps some compiler issue back in the day, or the person was half asleep? :)

[Z2697]

You're right because all that loop does is multiply (Denom / Numer) by (numframes + 1), using repeated addition, and those values do not change within the loop. I wonder if there is some reason for such a superfluous piece of code. Perhaps some compiler issue back in the day, or the person was half asleep? :)

It looks like "they were planning on VFR support" to me, but eventually that didn't happen.

[GeoffreyA]

It looks like "they were planning on VFR support" to me, but eventually that didn't happen.

Looks like it.

[higher]

Yeah, past a certain resolution and frame threads, more cores will start winning over better cores. M4 Max likely wins for 1080p or lower, and possibly 4K if using only 1 frame thread.

Unfortunately the M4 Max is only in MacBook Pro, which is a whole lot more expensive and bigger for headless work. Mac Mini tops out with the M4 Pro currently.

The Mac version of Handbrake now includes x265 4.1 so I thought I make a little comparison between my desktop 5900X and 16" MBP with M4 Pro (on battery).

The 1 minute sample was cut from an UHD Blu-Ray (36th Precinct) and was encoded in 4K using the same version of Handbrake with identical settings (preset slow) on both platforms.

M4 Pro: 4m 20s
5900X: 5m 25s

Quite impresssive.

[Sagittaire]

The Mac version of Handbrake now includes x265 4.1 so I thought I make a little comparison between my desktop 5900X and 16" MBP with M4 Pro (on battery).

The 1 minute sample was cut from an UHD Blu-Ray (36th Precinct) and was encoded in 4K using the same version of Handbrake with identical settings (preset slow) on both platforms.

M4 Pro: 4m 20s
5900X: 5m 25s

Quite impresssive.

Well not really simply because 9950X is more 2X (at least) powerfull than 5900X. 9950X at stock will certainely produce something like ~2 m 30 s to encode this source with x265.

[Ritsuka]

Of course, but the 9950X has a 170W TDP, and the M4 Pro is what, 40 W at max, with 6 less performance cores than the 9950X.

And it seems all the latest arm64 optimizations are again stuck on the x265-devel mailing list.

[higher]

Well not really simply because 9950X is more 2X (at least) powerfull than 5900X. 9950X at stock will certainely produce something like ~2 m 30 s to encode this source with x265.

The 9950X is only 50% faster than 5900X at 4K resolution. I guess an M4 Max could almost match an 9950X while consuming a lot less power.

18790

[Sagittaire]

The 9950X is only 50% faster than 5900X at 4K resolution. I guess an M4 Max could almost match an 9950X while consuming a lot less power.

18790

Techpowerup benchmark like many other no codec specialist are not able to test correctly codec: If you want seriousely make codec benchmark, you don't use gui like handbrake and you use codec profil able to saturate correctly 16C/32T CPU.

I create codec benchmark for make that and 9950X at stock has 74% more performance than 5950X for x265. I don't test 5900X but 5950X have theoricaly 20% more perfomance than 5900X. In correct H265 benchmark (all CPU thread saturation) the 9950X will produce 110% more performance than 5900X.

When Techpowerup use correct CPU saturation benchmark like cinebench, blender or stockfish, you evaluate the correct CPU power

Stockfisk:
5900X: 14.52 Mips
9950X: 30.78 Mips (+111%)

Blender:
5900X: 114.9 s
9950X: 56 s (+105%)

V-Ray:
5900X: 21538
9950X: 48899 (+127%)

https://tpucdn.com/review/amd-ryzen-9-9950x/images/vray.png

[ShortKatz]

The Mac version of Handbrake now includes x265 4.1 so I thought I make a little comparison between my desktop 5900X and 16" MBP with M4 Pro (on battery).

The 1 minute sample was cut from an UHD Blu-Ray (36th Precinct) and was encoded in 4K using the same version of Handbrake with identical settings (preset slow) on both platforms.

M4 Pro: 4m 20s
5900X: 5m 25s

Quite impresssive.

I would be quite interested how much faster my M4 Max would be compared to the M4 Pro.

[Barough]

x265 v4.1+62-441e1e4
Built on December 31 2024, GCC 14.2.0
Win32/64 / 8bit+10bit+12bit

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

DL :
https://www.mediafire.com/file/6nh9e7dfb72b3pi

[excellentswordfight]

Techpowerup benchmark like many other no codec specialist are not able to test correctly codec: If you want seriousely make codec benchmark, you don't use gui like handbrake and you use codec profil able to saturate correctly 16C/32T CPU.

I create codec benchmark for make that and 9950X at stock has 74% more performance than 5950X for x265. I don't test 5900X but 5950X have theoricaly 20% more perfomance than 5900X. In correct H265 benchmark (all CPU thread saturation) the 9950X will produce 110% more performance than 5900X.

When Techpowerup use correct CPU saturation benchmark like cinebench, blender or stockfish, you evaluate the correct CPU power

Stockfisk:
5900X: 14.52 Mips
9950X: 30.78 Mips (+111%)

Blender:
5900X: 114.9 s
9950X: 56 s (+105%)

V-Ray:
5900X: 21538
9950X: 48899 (+127%)

These are two different methodologies, this is not a case of "correct" and "not correct" way of doing it. Single instance encoding is still a thing, and actually the most common case for most users, so benchmarking single instance is still very much relevant.

Most software does not have perfect parallelization scaling, and in most cases were that is the case, i.e. 3d-rendering and simulations etc, those loads usually gain more to be calculated on GPUs anyway. And although I think it makes perfect sense to test both cases here, cause you can just run two encodes at the same time even though you dont wanna start doing chunk-encoding to get "more" out of your CPU. Its not like we have a history of starting to run multiple parallel benchmark of a software cause we dont find the thread-scaling good enough when that the results dont see the full "potential" of the CPU, cause this argument can be made for most of them (audio encoding, compression, compiling etc).

[Sagittaire]

These are two different methodologies, this is not a case of "correct" and "not correct" way of doing it. Single instance encoding is still a thing, and actually the most common case for most users, so benchmarking single instance is still very much relevant.

Most software does not have perfect parallelization scaling, and in most cases were that is the case, i.e. 3d-rendering and simulations etc, those loads usually gain more to be calculated on GPUs anyway. And although I think it makes perfect sense to test both cases here, cause you can just run two encodes at the same time even though you dont wanna start doing chunk-encoding to get "more" out of your CPU. Its not like we have a history of starting to run multiple parallel benchmark of a software cause we dont find the thread-scaling good enough when that the results dont see the full "potential" of the CPU, cause this argument can be made for most of them (audio encoding, compression, compiling etc).


Yes but time to encode wav to mp3 with lame is not really a problem.

Encoding video source can take several hours. And multipart or ABR Ladder to saturate CPU are simply well-known techniques in the professional world.

For exemple ABR Ladder is full option include directly in x265 codec.

Make multiscession encoding is option too and directly in handbrake.

Why buy a $600 CPU to do the fastest possible encoding in AOM AV1, when you can do it 4 times faster with a $200 CPU using the right encoding technique.

[Z2697]

Yes but time to encode wav to mp3 with lame is not really a problem.

Encoding video source can take several hours. And multipart or ABR Ladder to saturate CPU are simply well-known techniques in the professional world.

For exemple ABR Ladder is full option include directly in x265 codec.

Make multiscession encoding is option too and directly in handbrake.

Why buy a $600 CPU to do the fastest possible encoding in AOM AV1, when you can do it 4 times faster with a $200 CPU using the right encoding technique.

In fact you can buy a whole "lowest end" mac mini m4 version with the money of a "just CPU" 9950X. (of course the performance is far away)
Since m4 pro only comes with severely overpriced memory, if you are planning on only use the CPU to do "work" (but why) that's just not worth it.
Or maybe it's the other way around, the basic model of mac mini m4 is underpriced? You know, like the razor and blades model?
I don't know, I don't own a mac.
(wait a minute, m4 pro has 2 variants? and they are very different errrr
it's a great cpu but apple is just confusing)

It's a great chip, but it doesn't come as just a chip, I just don't want to buy it this way. (and the "large scale" customers are likely don't want as well)

It seems like even a m4 in basic model mac mini has more transistors than 9950X (although with integrated memory and GPU), and with the best process node at the time, I'm not surprised that it's performant and efficient, and the best model (m4 max 12p+4e) can come close to 9950X with less power draw.
Physics works, how surprising.

I have to say this is very out of topic now.

[Z2697]

Hell yeah let's just error out if input resolution exceeds 8192x4320

[Barough]

x265 v4.1+78-5223ea7
Built on January 03 2025, GCC 14.2.0
Win32/64 / 8bit+10bit+12bit

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

DL :
https://www.mediafire.com/file/86pd5zd03csrk67

[tormento]

Finally I have a working build with --frame-dup working and I'd like to play with it a bit, as I mostly encode animes.

What value of --dup-threshold should be ok? The default is 70 but it doesn't tell too much to me.

Is there a way to calculate the "difference" between two frames in a way similar to what x265 does?

[FranceBB]

Is there a way to calculate the "difference" between two frames in a way similar to what x265 does?

Well, in Avisynth I'd say:

YDifferenceFromPrevious()
UDifferenceFromPrevious()
VDifferenceFromPrevious()

and the corresponding

YDifferenceToNext()
UDifferenceToNext()
VDifferenceToNext()

[tormento]

Well, in Avisynth I'd say:

YDifferenceFromPrevious()
UDifferenceFromPrevious()
VDifferenceFromPrevious()

and the corresponding

YDifferenceToNext()
UDifferenceToNext()
VDifferenceToNext()


As far as I’ve read, it uses some sort of PSNR.

[jpsdr]

Finally I have a working build with --frame-dup working
You said that both mine and Patman's build didn't work.
Have you done some changes in the code to make it work ?

[tormento]

Have you done some changes in the code to make it work ?
I am talking about the latest Patman's build. I don't know what changes have been done.

[benwaggoner]

Finally I have a working build with --frame-dup working and I'd like to play with it a bit, as I mostly encode animes.

What value of --dup-threshold should be ok? The default is 70 but it doesn't tell too much to me.

Is there a way to calculate the "difference" between two frames in a way similar to what x265 does?
How did you get a working --frame-dup? Do you know what the fix was?

Since the setting hadn't been working until this, we've not had much experience with it. Experimentally, I'd play around with different values and look at how frames get classified in a bitstream analyzer (or just comparing the log file).

What you want to see is frames that are duplicated in the source are mostly set as duplicated frames in the bitstream, and that frames that aren't duplicates in the source are distinct frames in the output.

Anime tends to have more frames duplicates than not, and are pretty distinct between no dup and dup, so it's really the best case for this feature, and you can probably use a lot more aggressive settings than for other classes of content.

[jpsdr]

I take a look at the last Patman's commits, and didn't notice anything specific to a frame-dup fix. If it's fixed, it seems to be a side effect of something else (unless i missed something).

[Z2697]

How did you get a working --frame-dup? Do you know what the fix was?

Since the setting hadn't been working until this, we've not had much experience with it. Experimentally, I'd play around with different values and look at how frames get classified in a bitstream analyzer (or just comparing the log file).

What you want to see is frames that are duplicated in the source are mostly set as duplicated frames in the bitstream, and that frames that aren't duplicates in the source are distinct frames in the output.

Anime tends to have more frames duplicates than not, and are pretty distinct between no dup and dup, so it's really the best case for this feature, and you can probably use a lot more aggressive settings than for other classes of content.

"frame-dup" has been working all along, it's that it's not working as you'd expect from the name.
There're detailed explanations a few pages back in this thread.

I think the claims about how it's "not working" or "now working" by tormento is untrustworthy.
And what he actually means is that somehow on his computer the x265 crashed when enabling frame-dup, and is now not crashing, without any change to the code regadring frame-dup feature. Not how the feature itself is broken.

No offense. I mean the fact that the observation changed without any related interaction is not trustworthy. Not the guy (hopefully).

BTW, was there a x264 feature (or x264 mod) that does the similar thing? I think there was but I don't trust my memory.

[tormento]

Counter-order, guys.

It happens that I am not having --frame-dup working anymore. It's now crashing as the previous build did.

I am trying to reproduce the conditions under which it worked without errors, i.e. resolution and parameters.

Stay tuned but I fear it will be a long path, as it was a random test and I need to recall the conditions when it worked.

[cubicibo]

To have a better understanding of what this feature means: this feature removes frames based on PSNR thresholding, and signal picture timing SEIs to keep the correct... picture timing... yeah... which no commonly available decoder can recognize.

I don't question the looseness of the logic to detect dupe frames, but a decoder that does not implement Pic Timing SEI then relies solely on the decodied frame PTS, and the stream becomes VFR, with strictly identical output to the end user.

If a decoder does not support VFR, they should (shall?) support Pic Timing SEI. The decoded frame PTS timeline should appropriately reflect this. Gaps in the timeline should be filled by using the appropriate pic_struct entry to deliver CFR.

Even if it works as it should, the PSNR thresholding is not ideal to begin with, and the bits saved with removing near identical frames are, well, did you know picture timing SEIs cost bits?

It's not necessarily about bits, but better utilization of B and P frames. Also I am pretty sure 2K or 4K frame dupes are more costly than a few bytes.

EDIT: I dived in x265 codebase, and I think it does not set the frame PTS appropriately :mad:

[higher]

Techpowerup benchmark like many other no codec specialist are not able to test correctly codec: If you want seriousely make codec benchmark, you don't use gui like handbrake and you use codec profil able to saturate correctly 16C/32T CPU.

I create codec benchmark for make that and 9950X at stock has 74% more performance than 5950X for x265. I don't test 5900X but 5950X have theoricaly 20% more perfomance than 5900X. In correct H265 benchmark (all CPU thread saturation) the 9950X will produce 110% more performance than 5900X.

When Techpowerup use correct CPU saturation benchmark like cinebench, blender or stockfish, you evaluate the correct CPU power



It's likely that Zen 5 is not much of an advance in x265 encoding compared to other workloads.

TPU uses x265 with preset slow at 4K resolution. It fully saturates my 5900X and I'm guessing it fully saturates a 9900X as well. Yet, the 9900X is only 25% faster than 5900x while the 9950x is 27% faster than 5950x in he TPU benchmark and it sounds about right.

Power consumption is on a different level though. The 9900X consumes around 170W fully loaded while an M4 Pro consumes less than 50W.
Unfortunately X86 is years behind in this regards and also in single core performance.

[Z2697]

I don't question the looseness of the logic to detect dupe frames, but a decoder that does not implement Pic Timing SEI then relies solely on the decodied frame PTS, and the stream becomes VFR, with strictly identical output to the end user.

If a decoder does not support VFR, they should (shall?) support Pic Timing SEI. The decoded frame PTS timeline should appropriately reflect this. Gaps in the timeline should be filled by using the appropriate pic_struct entry to deliver CFR.



It's not necessarily about bits, but better utilization of B and P frames. Also I am pretty sure 2K or 4K frame dupes are more costly than a few bytes.

EDIT: I dived in x265 codebase, and I think it does not set the frame PTS appropriately :mad:

Does HEVC raw bitstream support VFR at all?
And as I already mentioned, x265 with container output mod or FFmpeg libx265 can get around with that VFR "hack".

As for the bits, x265 now signals that timing SEI for every frame, no matter it's duped (removed) or not.
It results in I think around 2.5kbps for 24fps video. How many duped frames should be removed to compensate that (average out), and how to decide the thresholding... well I just don't even bother.
But yeah, that will outweight the signaling overhead very soon.

Worst case it's just a few kbps, not really a big deal at all.

But since he is not seeing incorrect timing, the probability of that worst case happening is high.

[cubicibo]

VFR can be signaled both on the entire stream or at a CWS level. But specifying the actual picture output-presentation delay is overly complicated here.

Anyway, it does not matter for the current problem. pic_struct should be used with CFR. But frame entry time in decoder must be adapted with respect to the last pic struct instruction. I can't find any such code in x265, so VBV conformance must be way off.

[Z2697]

VFR can be signaled both on the entire stream or at a CWS level. But specifying the actual picture output-presentation delay is overly complicated here.

Anyway, it does not matter for the current problem. pic_struct should be used with CFR. But frame entry time in decoder must be adapted with respect to the last pic struct instruction. I can't find any such code in x265, so VBV conformance must be way off.

The timeing SEI we were talking contains and utilizes pic_struct, unless you are referring a different thing.
It shouldn't be necessary for "normal" CFR, only for "de-duped" CFR.

[tormento]

Well, it happens that I've found something about the --frame-dup crashing on my PC.

The very same video can be encoded when

1920×1080
1600×900
1280×720
960×540
but crashes miserably with

1440×810
1500×844
with error:

Video encoding returned exit code: -1073741819 (0xC0000005)

Any idea?

[cubicibo]

The timeing SEI we were talking contains and utilizes pic_struct, unless you are referring a different thing.
It shouldn't be necessary for "normal" CFR, only for "de-duped" CFR.

We're talking about the same thing, and I am telling you that x265 does not seem to make use of that field in the ratecontrol code. Frame following duplicated ones aren't shifted in time appropriately, buffer has less time to refill and hence fewer bits are allocated to the frames. More problematic, computed HRD fields are probably wrong.

Since there's a copy paste error on the CLI documentation for --pic-struct (claims to be needed for HLG), I will assume they never tested the feature or verified it was working correctly.

[benwaggoner]

It's likely that Zen 5 is not much of an advance in x265 encoding compared to other workloads.
It has full native AVX512 support, which could make using that flag improve performance more, and in more scenarios.

[benwaggoner]

VFR can be signaled both on the entire stream or at a CWS level. But specifying the actual picture output-presentation delay is overly complicated here.

Anyway, it does not matter for the current problem. pic_struct should be used with CFR. But frame entry time in decoder must be adapted with respect to the last pic struct instruction. I can't find any such code in x265, so VBV conformance must be way off.
Back in the VC-1 days we handled this by having the frame in the bitstream, containing just the frame_repeat tag. Didn't require any VFR, time shifts, etcetera.

[Z2697]

It has full native AVX512 support, which could make using that flag improve performance more, and in more scenarios.

If my test is anything to be believed, it's around 5% to 8% "free performance" (more or less), depending on the target quality somehow.
(5% with CRF-14, 6.3% with CRF-18, 6.6% with CRF-22 and 8% with CRF-26, the 4 quality targets I often use to draw RD curve (there's no difference in the RD curve in this case, of course))

[tormento]

Benchmarking with different builds, I've noticed that Patman86's x265-4.1+79+12-81640d428 ICC standard/AVX/AVX2 builds are identical but 1-2 bytes.

It seems a bit strange to me and I've opened a issue on GitHub.

Please be aware of that in the mean time.

[benwaggoner]

If my test is anything to be believed, it's around 5% to 8% "free performance" (more or less), depending on the target quality somehow.
(5% with CRF-14, 6.3% with CRF-18, 6.6% with CRF-22 and 8% with CRF-26, the 4 quality targets I often use to draw RD curve (there's no difference in the RD curve in this case, of course))
Hmm. Perhaps higher QP makes for fewer early exists so doing more in parallel helps? Can you share the command line? I imagine stuff like TU size could have different impact.

[Z2697]

Benchmarking with different builds, I've noticed that Patman86's x265-4.1+79+12-81640d428 ICC standard/AVX/AVX2 builds are identical but 1-2 bytes.

It seems a bit strange to me and I've opened a issue on GitHub.

Please be aware of that in the mean time.

You should compare with both setting no-info, and do a proper bytes compare instead of just telling from the size

Never mind, so you were talking about the executable, I thought you were talking about the encoded hevc stream.

[Z2697]

Hmm. Perhaps higher QP makes for fewer early exists so doing more in parallel helps? Can you share the command line? I imagine stuff like TU size could have different impact.


--preset slow
--rd 6
--ctu 32
--no-rect
--no-sao
--no-strong-intra-smoothing
--no-open-gop
--b-intra
--weightb
--aq-mode 1
--aq-strength 0.8
--qcomp 0.7
--pbratio 1.2
--bframes 3
--cbqpoffs -2
--crqpoffs -2
--deblock -3,-3
--rc-lookahead 80

[benwaggoner]

--preset slow
--rd 6
--ctu 32
--no-rect
--no-sao
--no-strong-intra-smoothing
--no-open-gop
--b-intra
--weightb
--aq-mode 1
--aq-strength 0.8
--qcomp 0.7
--pbratio 1.2
--bframes 3
--cbqpoffs -2
--crqpoffs -2
--deblock -3,-3
--rc-lookahead 80

Huh. What are you trying to encode/optimize for with those settings? And this is with 1080p --crf, right?

Nothing pops out as impacting performance in particular. However, as CABAC is single threaded per frame, just reducing the bitrate itself will improve performance on many systems.

[Z2697]

Huh. What are you trying to encode/optimize for with those settings? And this is with 1080p --crf, right?

Nothing pops out as impacting performance in particular. However, as CABAC is single threaded per frame, just reducing the bitrate itself will improve performance on many systems.

I don't usually optimize x265 parameters for every different sources based on their "style" or characteristic, unless the image quality of the source is very characterized.
These are just... "generic" settings.
I choose my encoding parameters based on the performance (FPS & RD), and the "dark biased AQ" based on... I see fit. I frankensteined AQ 1 and drak-bias as new mode in my mod build, yes :o
(AQ is not really about the RD performance, of course. The FPS performance also don't seem to affected much by AQ except edge based ones.)

However, the major portion of the things I encode is anime. When I do non-test encoding I use slower parameters. (e.g. hme)

And yes, the test is 1080p CRF.

[Sagittaire]

It's likely that Zen 5 is not much of an advance in x265 encoding compared to other workloads.

TPU uses x265 with preset slow at 4K resolution. It fully saturates my 5900X and I'm guessing it fully saturates a 9900X as well. Yet, the 9900X is only 25% faster than 5900x while the 9950x is 27% faster than 5950x in he TPU benchmark and it sounds about right.

Power consumption is on a different level though. The 9900X consumes around 170W fully loaded while an M4 Pro consumes less than 50W.
Unfortunately X86 is years behind in this regards and also in single core performance.

No, x265 at 4K with x265 at slow preset don't saturate 32 threads and by far. And saturate all thread mean 100% CPU charge at power limit during all encoding time.

If TPU codec benchmark saturate all CPU, the difference between 5900X, 5950X, 9900X and 9950X could be equivalent to blender benchmark, for exemple. And it's not the case.

[OvejaNegra]

hi to all!
it's been a long time since i used x265, i had to change the megui version and of course i updated x265 to latest version.

i jumped from version 3.x to 4.1

Since i still have some old habits, i want to know if things have changed so i can update my presets. I have some
presets with some options that i tweaked for encoding speed VS qualiy using advices from other users.

As i said, it's been some time since i used x265 and maybe some old habits must change too.

Sorry in advance for my english, is not my native language.

Here are my doubts:

1 - I'm using rd 4 because rd 6 gave me some artifacts on sharp edges, i remember someone saying that 4 was safe for real life
content and 6 was better for animation (no artifacts with animation)

2 - I still use --aq-mode 1 --aq-strength xx because the other modes were a little inconsistent for me, any changes? (for real life
and animation)

3 - I'm using --max-merge 2, other modes gave me problem with edges on dark scenes (worst on animation).

4 - --rskip 2 --rskip-edge-threshold X is still recommended?

5 - --limit-refs 3 with more refs (5) is still adviced VS --limit-refs 0 and less refs (taking in account encoding time and
benefits of the reference frames)

6 - i'm using --ctu 32 because a bug with default value (on 4k content if i remember correctly) is it fixed? Should i use
64 for everything or just for 4k content (and 32 for the rest).

7 - I'm using --bframes 4 for everything, should i use 2 for live action and 4 for animation?

8 - I'm using --tu-intra-depth 3 --tu-inter-depth 3 always (live action and animation). It's ok? Should i use it only if i'm using

rect and amp?

9 - Is --limit-tu X advised with --tu-intra-depth 3 --tu-inter-depth 3 ?

10 - If i have time, i use --rect for real life content and --rect --amp for animation, is that ok?

11 - --limit-modes is still advised to use with --rect --amp if speed is required (vs not using rect amp at all)

12 - I'm using --rc-lookahead 250 or as big as i can (for abr and crf), is still advised?

13 - I always use --weightp --weightb --b-intra

14 - I use --psy-rd 2.0 --psy-rdoq 0.0 --rdoq-level 0 if it looks ok and --psy-rd 2.0 --psy-rdoq 2.0 --rdoq-level 2
if i need to retain more detail, anything new on that?

15 - Has anyone tested --tskip with animated content? Does it really helps? Should i use --tskip-fast?

16 - For real life content, i don't like / use sao, but i remember it being usefull for animation with low bitrate, any
change on that?

17 - What's the utility of --limit-sao and --sao-non-deblock? (more speed? some kind of intelligent mode? less strong effect?)

18 - --me 4 (star) is adviced? (for real life and animation)

19 - I use --subme 3 for everything ¿Should i use another mode? Should i use a different mode for animation?

20 - Any other advice? Something new that i should use / not use?

Thanks!!

[Z2697]

The problems you mentioned in 1, 3 and 6 feels unrealistic, can you elaborate?
While the "core encoding functions" haven't been changed for years, and you are pretty much will be getting exact same results (I have compared 3.5 to 4.1), I have some advise... or just my 2 cents.

5. limit-refs makes very little difference (~1% speed and ~0.05% bd-rate). ref isn't that big of a deal for regular contents neither.

6. ctu (64 or 32) isn't very impactful to the quality. many other things are actually 32x32 max in HEVC.

7. just don't use too many b frames, they will slow down the encoding with b-adapt and has no real benefit.

8. tu-depth is not "tied" to rect and amp.

9. limit-tu is a good trade-off, but maybe lower the depths to 2 is a better trade-off.

10. they are ok and will improve a little bit of quality. with tu-depths already increased it'll be tiny little bit.

15. tskip only works on 4x4 intra TUs. it's not gonna magically make anything look better. the target situation of its design is esoteric and not even work very well.

18. me 4 is not star.

[OvejaNegra]

The problems you mentioned in 1, 3 and 6 feels unrealistic, can you elaborate?
While the "core encoding functions" haven't been changed for years, and you are pretty much will be getting exact same results (I have compared 3.5 to 4.1), I have some advise... or just my 2 cents.

5. limit-refs makes very little difference (~1% speed and ~0.05% bd-rate). ref isn't that big of a deal for regular contents neither.

6. ctu (64 or 32) isn't very impactful to the quality. many other things are actually 32x32 max in HEVC.

7. just don't use too many b frames, they will slow down the encoding with b-adapt and has no real benefit.

8. tu-depth is not "tied" to rect and amp.

9. limit-tu is a good trade-off, but maybe lower the depths to 2 is a better trade-off.

10. they are ok and will improve a little bit of quality. with tu-depths already increased it'll be tiny little bit.

15. tskip only works on 4x4 intra TUs. it's not gonna magically make anything look better. the target situation of its design is esoteric and not even work very well.

18. me 4 is not star.


I'ts a little hard for me to find the information, as i said, i saw the artifacts, did some research, changed the values and
it was ok (for me):


https://forum.doom9.org/showthread.php?p=1892787#post1892787


"I don't go higher than max-merge 3. I've seen 4 and 5 produce worse results than 3 during fast motion in animated content. I don't know why."


Increasing max merge, gave something like edge ghosting (remember old LCDs?) on dark edges on dark background if i remember correctly


https://forum.doom9.org/showthread.php?p=1963393#post1963393

"CTU 64 will make a mess out of noisy flat backgrounds compared to CTU 32 (qg-size 32 in both cases since x265 doesn't use 64 by default for CTU 64)"


https://forum.doom9.org/showthread.php?p=1963328#post1963328


https://forum.doom9.org/showthread.php?p=1963352#post1963352



5 - Should i stay with limit-refs 3? (faster)

7 - 4 Looked like a balanced default to me, maybe 2 is better for real life content, but that's just speculation

9 - like: --tu-intra-depth 2 --tu-inter-depth 2 --limit-tu 0 instead of --tu-intra-depth 3 --tu-inter-depth 3 --limit-tu 4

10 - So, for general content and animation, is a better investiment --tu-intra-depth 2 --tu-inter-depth 2 and forget about rect amp?

15 - I remember members commenting about the possible bennefits of tskip for animation, maybe someone has some actual experience with it.

18 - Sorry: --me 3

thanks!

[Z2697]

I'ts a little hard for me to find the information, as i said, i saw the artifacts, did some research, changed the values and
it was ok (for me):


https://forum.doom9.org/showthread.php?p=1892787#post1892787





Increasing max merge, gave something like edge ghosting (remember old LCDs?) on dark edges on dark background if i remember correctly


https://forum.doom9.org/showthread.php?p=1963393#post1963393




https://forum.doom9.org/showthread.php?p=1963328#post1963328


https://forum.doom9.org/showthread.php?p=1963352#post1963352



5 - Should i stay with limit-refs 3? (faster)

7 - 4 Looked like a balanced default to me, maybe 2 is better for real life content, but that's just speculation

9 - like: --tu-intra-depth 2 --tu-inter-depth 2 --limit-tu 0 instead of --tu-intra-depth 3 --tu-inter-depth 3 --limit-tu 4

10 - So, for general content and animation, is a better investiment --tu-intra-depth 2 --tu-inter-depth 2 and forget about rect amp?

15 - I remember members commenting about the possible bennefits of tskip for animation, maybe someone has some actual experience with it.

18 - Sorry: --me 3

thanks!

Personally speaking, I think they are experiencing a version of "pigeonhole principle" where some parts look better and some parts look worse and they are biased to their initial hypothesis.

5. it's only 1% faster. but more performance is always good when the downside is negligible right? I wouldn't say you should or should not because it's about trade-off, I think there's no definitive answer.

7. I don't know, I honestly can't tell the difference.

9. sometimes the limit-tu will decrease quality more than "less depth", but we are talking about maybe 1% difference in bd-rate here, and I have biases like everyone.

P.S. in my tests I saw: with tu-depths = 1, the inter frames will not use any 4x4 TUs (I don't know whether or not this is a bug in x265), intra frames still use 4x4 TUs.
starting from tu-depths = 2, inter frames will start to use 4x4 TUs, this is where the big (relatively) differences come from. larger values makes little difference.

10. if you think about it, rect and amp are like "cheap version" of Quadtree split right? so when you allow more Quadtree split, they become much less effective in improving compression while having roughly same computational cost. (less effective, but still effective, yes. it's just harder to justify the computational cost, you may get beffter result spending those computational resource on some other parameter, for example, HME)

P.S. this is an oversimplification, the TUs are not equal to PUs. (TU = Transform Unit, PU = Prediction Unit)

15. 4x4 intra TUs are rare. the odds of them being "chosen to skip" is beyond rare. I haven't seen it imporve anything in my test and I hypothesize it will not have any significant effect on regular contents.
(Most video sources you can get are already "transformed", what "transform skip" will save you from? for truly lossless cases, just go check how many 4x4 intra TUs are there and how many of them are transform skipped, with tools like YUView (https://github.com/IENT/YUView))

I mean, don't get me wrong, tskip works well in its designed use case: sharp, detailed, high-contrast and mostly static contents, like screen recording with mostly texts on simple / pure color background. I just don't think it's worth to enable for most regualr contents.

P.S. by "rare" I mean on average of all frames, if you specifically look Intra frames only, of course they are not that rare.

18. UMH and STAR are close in performance, both speed and quality. A cheap alternative is HEX with some large merange (like, hundreds, for HEX that won't slow down much). If you want something better than STAR 57 in many presets, go with HME.
(--hme --hme-search star,umh,star)
(star in second level will make encoding very slow for some mystic reason.)
(If you want something fast, use --hme-search hex,hex,hex which is still better than traditional ME)
(you can adjust each level's range as well, but default is good enough)

(HME, in combination with lookahead-slices will produce non-deterministic result. If you want your encode with exact same settings output identical bitstream, use --lookahead-slices 0 with HME)
(there are other things that will make encode non-deterministic, most common example is VBV)

[OvejaNegra]

Thanks!

I'll do some test and i'll what can i improve.

:thanks:

[Z2697]

When using more than 1 slices, x265 creates intra blocks in prediction frames out of nowhere.
Let's do a simple test:
ffmpeg -lavfi color=gray:s=hd1080:d=10,noise=allf=u:alls=50 -x265-params slices=2 2sli.265
With the given command line, x265 is allowed to encode a completely static video, static noise is applied to add variance to the image, it does not has to be noise, nor does it has to be completely static, you just need some "adequately" high frequency information, a flat color image does not trigger the bug.
The result file is absurdly larger than single slice result.
Viewing the result file with YUView reveals that the encode has a lot of intra blocks near slice borders.
https://images2.imgbox.com/d2/ab/taiMdVlu_o.jpg

Setting frame-threads=1 seems to mitigate this issue.

The false intra mode in otherwise inter-predictable blocks will make compression ratio a lot worse, if you are planning or forced to use more slices (e.g. UHD Bluray compatible encoding), make sure to use frame-threads=1, slices will provide parallelism silimar to frame-threads.

[LigH]

You should certainly report this in the x265 bugtacker. Can't guarantee that devs read here.

[Z2697]

That's very right... Email sent.

[cubicibo]

IIRC slices (or tiles) are independent and cannot reference each other's coding units. If you encode high frequency content, your file size will double with two slices, triple with three, and so on. No surprise here. But you are right that going from one to two threads should not make the situation significantly worse. I am more concerned by the presence of intras at the bottom of each slice... are the additional threads in each slice set up to the farthest point from the others and made independent too (cannot reference CU from the same slice made by the other thread)?

[rwill]

The false intra mode in otherwise inter-predictable blocks will make compression ratio a lot worse, if you are planning or forced to use more slices (e.g. UHD Bluray compatible encoding), make sure to use frame-threads=1, slices will provide parallelism silimar to frame-threads.

They seem to limit vertical MV on current picture slice boundaries. If someone can point out the need in the H.265 standard please do lol...

[Z2697]

They seem to limit vertical MV on current picture slice boundaries. If someone can point out the need in the H.265 standard please do lol...

The problem is not the "natural" limitations that come with slices, the fact that this issue even happens in completely static videos means it has nothing to do with those limitations. The blocks don't even need to be motion compensated. They are not moved, they don't need to have MV.

[rwill]

The problem is not the "natural" limitations that come with slices, the fact that this issue even happens in completely static videos means it has nothing to do with those limitations. The blocks don't even need to be motion compensated. They are not moved, they don't need to have MV.

Of course they need a MV, the 0,0 one for no movement. In x265 there is code like this in frameencoder.cpp:


// Initialize restrict on MV range in slices
tld.analysis.m_sliceMinY = -(int32_t)(rowInSlice * m_param->maxCUSize * 4) + 3 * 4;
tld.analysis.m_sliceMaxY = (int32_t)((endRowInSlicePlus1 - 1 - row) * (m_param->maxCUSize * 4) - 4 * 4);


and subsequent in search.cpp:

mvmin.y = X265_MAX(mvmin.y, m_sliceMinY);
mvmax.y = X265_MIN(mvmax.y, m_sliceMaxY);


So they are actively blocking MV.Y to be 0 for LCUs at slice borders. This means that a CU cannot rest with no movement but has to predict from within the slice. But within the slice is not the same noise so its most likely encoded Intra. Seems to come from their design decision to run Slices in Parallel when they should just run Frame parallel.. but I might be wrong on that one.

*edit*
And to be honest I do not know of any "natural" limitations coming with slices related to motion vectors.

[Z2697]

Well... the "single row slice" does not have unnecessary intra blocks

(How to encode "single row slice" high resolution video when the default max nal units per frame is 16? (16*64=1024))
(Who asked? (https://github.com/Mr-Z-2697/x265-experimental/commit/d5546ab87fe376dc00d733564db7a1df85ee740d))

[rwill]

Well... the "single row slice" does not have unnecessary intra blocks

Yes, because there is also code like this:


// Handle single row slice
if (tld.analysis.m_sliceMaxY < tld.analysis.m_sliceMinY)
tld.analysis.m_sliceMaxY = tld.analysis.m_sliceMinY = 0;


Does not change the multi row slice mechanics though. So, just use single row slices then. Problem solved.

[Z2697]

This might work

tld.analysis.m_sliceMinY = x265_min(0, -(int32_t)(rowInSlice * m_param->maxCUSize * 4) + 3 * 4);
tld.analysis.m_sliceMaxY = x265_max(0, (int32_t)((endRowInSlicePlus1 - 1 - row) * (m_param->maxCUSize * 4) - 4 * 4));


Second thought, let actually follow why there's 3 * 4 and 4 * 4 in the expressions.

Introduced in a8d7d3d (https://bitbucket.org/multicoreware/x265_git/commits/a8d7d3d37455c2bb944bca0311a2c269d5cee703) as 2*4 and 3*4, later changed to 3*4 and 4*4 in eaeeab5 (https://bitbucket.org/multicoreware/x265_git/commits/eaeeab570b66d943e2b1723776565bcee479b3ae)... but I don't understand why.
I tried to remove these 2 constants and the encoding seems pretty normal, maybe other structures changed in a way that these don't matter anymore?

[rwill]

Second thought, let actually follow why there's 3 * 4 and 4 * 4 in the expressions.


Looks like interpolation filter extends to me.

[jpsdr]

Hi.
Is, for the GCC compile options, the 7975WX a znver4 or znver5 ?
According the technical informations i've found i would say znver4, but i want to be sure.

[RARY]

https://forum.doom9.org/showthread.php?p=2013895#post2013895

13 - I always use --weightp --weightb --b-intra

it may not be efficient for all the videos.
Weightp/weightb is beneficial for sequences with lighting changes (e.g., fade effects), as it reduces bitrate while maintaining quality.

For static lighting conditions, the impact is minimal and not content dependent. Also there may be slight FPS reduction.


6 - i'm using --ctu 32 because a bug with default value (on 4k content if i remember correctly) is it fixed? Should i use
64 for everything or just for 4k content (and 32 for the rest).

I couldn't find any issues like that and may be it would be helpful to look on it, if you mention the complete cli combination


1. I'm using rd 4 because rd 6 gave me some artifacts on sharp edges, i remember someone saying that 4 was safe for real life
content and 6 was better for animation (no artifacts with animation)


Command Bitrate Global PSNR SSIM (dB) Elapsed Time FPS VMAF
-------------------------------------------------------------------------------------------------------
anime
Big_Buck_Bunny_1080P.yuv rd 4 28 1343.25 42.162 15.135 9.19 27.19 89.480158
Big_Buck_Bunny_1080P.yuv rd 6 28 1421.08 42.288 15.254 18.39 13.60 90.415959
sita_1920x1080_30.yuv rd 4 28 3153.66 41.489 18.519 35.10 28.49 97.693194
sita_1920x1080_30.yuv rd 6 28 3016.33 42.224 19.724 73.57 13.59 98.23908

real
crowd_run_1080p50.yuv rd 4 28 15702.50 32.190 9.313 24.44 20.46 83.335517
crowd_run_1080p50.yuv rd 6 28 16542.31 32.521 9.585 126.74 3.95 85.747593
Island_1920x1080_420p_8bit_24fps.y4m rd 4 28 3808.73 39.609 11.053 111.47 21.72 7.735233
Island_1920x1080_420p_8bit_24fps.y4m rd 6 28 3997.55 39.677 11.051 236.79 10.22 7.68822


In my tests, VMAF increased slight linearly from rd4 -> rd 6 for both real-life and anime content, and I did not notice any artifacts. If you can specify the conditions or settings where you observed artifacts, I can check further.




11 - --limit-modes is still advised to use with --rect --amp if speed is required (vs not using rect amp at all)


Using --limit-modes with --rect --amp achieves similar quality (PSNR, SSIM) and bitrate compared to using --rect --amp alone, but with a marginal FPS improvement

If speed is concerned, --no-limit-modes --no-rect --no-amp may be the choice, as It almost doubles the encoding speed compared to using --limit-modes --rect –amp at some cost.






Video | Bitrate (kb/s) | FPS | PSNR (dB) | SSIM (dB) | Time (s)
------------------------------------------------------------------------------------------------------

Netflix_Tango (4K, 60fps) - rect, amp | 6867.63 | 4.78 | 43.288 | 15.935 | 61.51
Netflix_Tango (4K, 60fps) - limit-modes | 6856.90 | 4.70 | 43.284 | 15.934 | 62.61
Netflix_Tango (4K, 60fps) - no-limit-modes | 6925.84 | 7.18 | 43.231 | 15.886 | 40.95



3 - I'm using --max-merge 2, other modes gave me problem with edges on dark scenes (worst on animation).

This may not be the case for any content type (anime, real, dark scenes, etc.). Based on my experiments with x265, the VMAF score increases linearly as the `--max-merge` value increases from 1 to 5.


Command Bitrate Global PSNR SSIM (dB) Elapsed Time FPS VMAF
-------------------------------------------------------------------------------------------------------
dark_high-contrast/firebelly-torches-dv.yuv
--max-merge 2 28 465.87 40.058 11.767 15.31 65.33 86.66916
--max-merge 3 28 463.38 40.073 11.779 15.45 64.74 86.698442
--max-merge 4 28 461.17 40.094 11.792 15.46 64.67 86.721314
--max-merge 5 28 462.03 40.101 11.799 15.92 62.80 86.712682

dark-scene_high-contrast/gothism-dv.yuv
--max-merge 2 28 79.11 42.959 13.031 6.34 127.87 87.432972
--max-merge 3 28 78.72 42.977 13.044 6.30 128.70 87.478732
--max-merge 4 28 78.97 42.990 13.051 6.47 125.42 87.392775
--max-merge 5 28 79.18 42.986 13.054 6.30 128.80 87.386751

dark-scene_high-contrast/forest_jester-dv.yuv
--max-merge 2 28 1419.55 35.167 10.827 5.42 54.25 92.135595
--max-merge 3 28 1415.42 35.205 10.892 5.34 55.09 92.286436
--max-merge 4 28 1410.23 35.229 10.932 5.43 54.14 92.392379
--max-merge 5 28 1409.85 35.236 10.940 5.56 52.86 92.430227

[Boulder]

PSNR and SSIM are not very reliable metrics when it comes to perceptual quality. I'd rather use SSIMU2 which is a little closer, it is far from perfect like all metrics basically, but it works quite well when comparing parameters.

[RARY]

PSNR and SSIM are not very reliable metrics when it comes to perceptual quality. I'd rather use SSIMU2 which is a little closer, it is far from perfect like all metrics basically, but it works quite well when comparing parameters.

But the VMAF scores will generally reflect these degradations( visible compression artifacts, banding, or blurring,) better than PSNR or SSIM and but no artifacts were observed with the given settings.

[Boulder]

But the VMAF scores will generally reflect these degradations( visible compression artifacts, banding, or blurring,) better than PSNR or SSIM and but no artifacts were observed with the given settings.

VMAF has plenty of flaws itself.

[excellentswordfight]

Hi.
Is, for the GCC compile options, the 7975WX a znver4 or znver5 ?
According the technical informations i've found i would say znver4, but i want to be sure.
znver4, as 7975WX has zen 4 cores.

[Z2697]

VMAF has plenty of flaws itself.

That's absolutely true, we should be very careful with VMAF scores, just as with any other metrics.
But with a carefully controlled test, the VMAF scores have a pretty high reference value.

[jpsdr]

@excellentswordfight
Ok, thanks.

[benwaggoner]

That's absolutely true, we should be very careful with VMAF scores, just as with any other metrics.
But with a carefully controlled test, the VMAF scores have a pretty high reference value.
I find p1204.3, which combines full reference and bitstream analysis, to have the highest subjective correlation of currently available metrics. It includes chroma, even. VMAF is luma only, which causes a big blind spot, particularly with HDR content.

Of course, like VMAF and any other machine learning based metric, the quality of the results is highly dependent on training on the right ground truth data, and scores using different ML models will be different.

[Stereodude]

Is anyone making AVIsynth aware/capable builds of x265 anymore, or is there a trick to encode an .avs file without piping with the latest x265 builds Barough is posting?

[jpsdr]

@Stereodude
You can find what you want on my github.

[Stereodude]

@Stereodude
You can find what you want on my github.
Thanks, I see I have lots of choices. :sly:

[Barough]

x265 v4.1+110-0e0eee5
Built on March 07, 2025, GCC 14.2.0
Win32/64 / 8bit+10bit+12bit

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

DL :
https://www.mediafire.com/file/aiuh2h9m2cafyhu

[jpsdr]

Thanks, I see I have lots of choices. :sly:
Things can change between users, but for me, LLVM is the fastest, and as i have a Broadwell, the Broadwell build. Don't have my big AMD yet, so can't tell about Zen4, but within 2 months...
But speed difference is not bigger than 2~3%.
After, one day someone said that gcc build produces better result than LLVM build...
I can't understand how it could be even possible :confused:
In my case i'm using LLVM and never noticed anything.

[Boulder]

With LLVM and Zen, at least with SVT-AV1 you need to be careful with the "march" setting. Znver3 seems to be broken and it bloats the executable and is slower than znver2 for a Zen3 CPU. I've also noticed that using march=native is not the best option for some reason.

[Z2697]

No matter what compiler and flags are used, the performance of x265 will not change a lot, unless you are using no-asm.
x265 has tons of assembly optimizations that's not gonna be affected by the compiler.
(fun fact: however, if you disable too many x86 flags in compiler options, the result x265 binary is broken, it will produce giant bitstreams regardless of the rate control settings)

But Clang does have somewhat broken Ryzen support and generic AVX-512 optimization in one or more recent version(s), I don't know if it's fixed in the latest release or not.
I shall repeat that it's related to the optimization of the "plain C/C++" code, not gonna affect x265's performance in assembly optimized routines.
(take https://github.com/L4cache/hmp3/releases/tag/5.2.4 as an example, it's asm optimization code is outdated and those builds were relying entirely on the compiler optimization, and the clang znver4 performance is strangely poor)

[jpsdr]

I'll have my gig in around 2 months, i'll make tests at this time. Thanks for theses informations.

[chenm001]

This might work

tld.analysis.m_sliceMinY = x265_min(0, -(int32_t)(rowInSlice * m_param->maxCUSize * 4) + 3 * 4);
tld.analysis.m_sliceMaxY = x265_max(0, (int32_t)((endRowInSlicePlus1 - 1 - row) * (m_param->maxCUSize * 4) - 4 * 4));


Second thought, let actually follow why there's 3 * 4 and 4 * 4 in the expressions.

Introduced in a8d7d3d (https://bitbucket.org/multicoreware/x265_git/commits/a8d7d3d37455c2bb944bca0311a2c269d5cee703) as 2*4 and 3*4, later changed to 3*4 and 4*4 in eaeeab5 (https://bitbucket.org/multicoreware/x265_git/commits/eaeeab570b66d943e2b1723776565bcee479b3ae)... but I don't understand why.
I tried to remove these 2 constants and the encoding seems pretty normal, maybe other structures changed in a way that these don't matter anymore?

Please allow me to explain
It is QPEL, so need *4, the interpolate is 8-taps, so (-3, 4), this is reason 3*4 and 4*4

[Z2697]

Please allow me to explain
It is QPEL, so need *4, the interpolate is 8-taps, so (-3, 4), this is reason 3*4 and 4*4

How important is this? Will it break some encoding if not applied, or is it just for the deterministic output?

If it's the latter, I'm afraid that it's not working (maybe due to some later changes), in tests I ran, the encoding options that was deterministic became non-deterministic after I added the slices option.
The option that can make it deterministic is frame-threads=1, and the MV limit isn't even applied when frame-threads==1.

Edit: https://files.catbox.moe/f3ompg.7z here's a build of that exact commit (eaeeab5), let's test how it was back then ;)

Update:
I just ran some quick test of the abovementioned old version against latest git version, and the old version was indeed "more consistent".
Some inconsistencies still occur in some sources, not listed in the hash result below. So it was never "actually works".
The same "unnecessary intra blocks" problem was present in the old version.
So this deterministic fix does lost it's effect along the way, maybe we can remove it for better compression, and for deterministic output, default frame-threads to 1 when slices > 1?
Like this: https://github.com/Mr-Z-2697/x265-experimental/tree/E-2025-02-19
SHA-1
98a2601eb9d1a5539c8af941497bf3ae80fd02ef *new_sli1_enc1.265
98a2601eb9d1a5539c8af941497bf3ae80fd02ef *new_sli1_enc2.265
7cebdd053631750cfe4999708da60aac1612f00b *new_sli2_enc1.265
40757f3f60097d1dc54513a59061bbe5d7022549 *new_sli2_enc2.265
bbc466300d7e310d716ff8ac2ca9b93b07fcca17 *old_sli1_enc1.265
bbc466300d7e310d716ff8ac2ca9b93b07fcca17 *old_sli1_enc2.265
e5ed62a2c42e804954242bdf870081990942bff5 *old_sli2_enc1.265
e5ed62a2c42e804954242bdf870081990942bff5 *old_sli2_enc2.265

[rwill]

How important is this? Will it break some encoding if not applied, or is it just for the deterministic output?

If it's the latter, I'm afraid that it's not working (maybe due to some later changes), in tests I ran, the encoding options that was deterministic became non-deterministic after I added the slices option.
The option that can make it deterministic is frame-threads=1, and the MV limit isn't even applied when frame-threads==1.

Well I don't know anything about the x265 threading model and its implications but you can check for encoder/decoder mismatches with x265's --recon or --hash 1. You need a decoder that writes out unmodified raw .yuv or supports picture hashing then.

[benwaggoner]

Well I don't know anything about the x265 threading model...
For the curious: https://x265.readthedocs.io/en/master/threading.html

[benwaggoner]

No matter what compiler and flags are used, the performance of x265 will not change a lot, unless you are using no-asm.
There were some tests here some years back showing profile-driven optimizations also helped a bit for the workload profiled. But as you say, the more asm, the less compiler optimizations matter.

[rwill]

For the curious: https://x265.readthedocs.io/en/master/threading.html

Sadly this page does not mention slices at all...

[Hellboy.]

For the curious: https://x265.readthedocs.io/en/master/threading.html

I have no idea how x265 works, but reading this it seems that
--frame-threads=1 is better ?

[Z2697]

I have no idea how x265 works, but reading this it seems that
--frame-threads=1 is better ?

Yes, you can also disable thread pools for even better quality!
(and watch the fps go down but let's not talk about that for now:cool:)

disable frame-threads you get 2% lower bd-rate.
disable thread pools you get 0.5% or less improvement (on top of the previous 2%).

[benwaggoner]

I have no idea how x265 works, but reading this it seems that
--frame-threads=1 is better ?
Yes. It used to make a big difference. The gain became smaller over time; I've not tested how much for a couple of years now.

If you're encoding multiple streams on the same hardware simultaneously, you get better net throughput by turning off frame threading and, if the encodes/cores ratio is high enough, reducing threading in general.

[RARY]

PSNR and SSIM are not very reliable metrics when it comes to perceptual quality. I'd rather use SSIMU2 which is a little closer, it is far from perfect like all metrics basically, but it works quite well when comparing parameters.

Any reference implementations for SSIMU2 ?

[Boulder]

Any reference implementations for SSIMU2 ?

I don't know if there's any for Avisynth, but https://github.com/Line-fr/Vship is a good and fast one for Vapoursynth.

[StvG]

I don't know if there's any for Avisynth, but https://github.com/Line-fr/Vship is a good and fast one for Vapoursynth.

https://github.com/Asd-g/AviSynthPlus-ssimulacra (not GPU).
https://github.com/Asd-g/AviSynthPlus-Butteraugli (not GPU).

[hellgauss]

Yes, you can also disable thread pools for even better quality!
(and watch the fps go down but let's not talk about that for now:cool:)

disable frame-threads you get 2% lower bd-rate.
disable thread pools you get 0.5% or less improvement (on top of the previous 2%).

If you have enough RAM, you can tune your scripts to run several encoding in parallel...
I find it useful for series, and also to tune % of %cpu usage if you need your PC for other tasks, especially if you run the encode in low priority (Windows).

[KarthikTdk]

Hi @all

Can i get reference code for H. 265 Encoder which should have 4:2:2 , 10 bit support

Thanks in advance
Karthik.

[RARY]

I find p1204.3, which combines full reference and bitstream analysis, to have the highest subjective correlation of currently available metrics. It includes chroma, even. VMAF is luma only, which causes a big blind spot, particularly with HDR content.

Of course, like VMAF and any other machine learning based metric, the quality of the results is highly dependent on training on the right ground truth data, and scores using different ML models will be different.


But, ITU-T P.1204 allows us to estimate video quality on a per-second MOS score basis. However, for frame-level quality assessment, metrics like VMAF or any non-references(blocking, blurring), with Tools like MSU VQMT would be a choice for detailed per-frame analysis.

is it or any better one than this?

[Z2697]

Hi @all

Can i get reference code for H. 265 Encoder which should have 4:2:2 , 10 bit support

Thanks in advance
Karthik.

Maybe this
https://vcgit.hhi.fraunhofer.de/jvet/HM

[benwaggoner]

But, ITU-T P.1204 allows us to estimate video quality on a per-second MOS score basis. However, for frame-level quality assessment, metrics like VMAF or any non-references(blocking, blurring), with Tools like MSU VQMT would be a choice for detailed per-frame analysis.

is it or any better one than this?
P.1204, done correctly, offers better correlation to subjective ratings for moving images. I don't think per-frame metrics are all that valuable in overall quality assessment or quality tuning. There's lots of stuff we can do that may give one frame poor metrics but isn't visible at normal speed playback, and lots of stuff we can do where each individual frame scores well, but discontinuities between frames cause a visible quality issue.

Per frame is helpful for diagnostics, sure. If I frame QP is lower than P, that's likely to cause issues. And if P, B, and b frames have consistent if small differences on some metrics, that can cause frame strobing and other artifacts.

But if I want to know how good someone subjectively would think a video looked, P.1204 is definitely the best available metric I've used. With the right ground truth data, it can get better than 90% correlation while VMAF can't really hit 80%.

The biggest downside is it is both full reference and bitstream level, so P1204.3 training is codec specific, and supporting a new codec requires a whole new round of test content and subjective ratings to train on. And if there are codec tools that weren't used in the test corpus, the training won't account for that tool either. VAMF doesn't really pick up on when --tskip is valuable, or adaptive quantization. the public VMAF model was originally just trained on x264 encodes using the same preset with varying --crf values. And VMAF's subjective correlation went down there more tools and tuning outside of that are implemented. VMAF was pretty bad at HDR until a bunch of HDR test content was made and subjectively rated, and an HDR-specific model was created.

The upside of VMAF is it is at least somewhat codec agnostic (and can be made much more so the bigger and more varied the content it is tested and trained on), and certainly better than PSNR or SSIM was.

Modern machine learning based metrics aren't really "a" metric. They're really just foundations that define the inputs a machine learning model can be run against. The actual model itself is the most important part, and that's really orthogonal to the metric itself. And they get a lot more accurate than the public model if an organization creates its own ground truth subjective ratings tuned around their content and scenarios.

I get a little frustrated when people talk about "VMAF scores" in the abstract without specifying the model that was used, as the same content can give quite different scores with varying accuracy based on the specific model used.

And also when the mean of scores for an hour long clip is presented as an overall quality, masking whether quality is consistently good or variably excellent or terrible.

How we measure and describe quality over time isn't something I've got a mechanism I'm confident in yet.

[RARY]

Is the VQMT tool metric's calculation accurate ?

For any reference metric, when I gave original sequences for both original and processed one and expecting high value i.e 100 ( based on scale ),
But there are some fluctuations in the score for few frames.

[benwaggoner]

Is the VQMT tool metric's calculation accurate ?

For any reference metric, when I gave original sequences for both original and processed one and expecting high value i.e 100 ( based on scale ),
But there are some fluctuations in the score for few frames.
“Accuracy” is more a continuum than a yes/no. I rate it by correlation with subjective MOS rating. It is also important to know what definition of “quality” you want to measure, and how that relates to the kind of “quality” that the metric is designed to measure.

Per frame quality numbers absolutely will vary, and also don’t always mean a whole lot, as we see video in motion. Great looking video can have frames with obvious artifacts if paused that are invisible (and thus don’t count) when being watched at normal speed.

No metric is or ever will be perfect. I like p1204.3, combining full reference and bitstream analysis, with a ML model trained on the breadth of content and compression being used.

[KarthikTdk]

Hi @all ,

What is syntax element , how we put our data in it and send it to binarization ,
Can anyone explain this , i am unable to understand the working of this , how quantization output is sending input to binarization

Thanks in advance

BR
Karthik

[RARY]

Any widely used implementation available for this p1204.3 standard along with videoParser?

I have been using https://github.com/Telecommunication-Telemedia-Assessment/bitstream_mode3_p1204_3 repo for a while.

[benwaggoner]

Any widely used implementation available for this p1204.3 standard along with videoParser?

I have been using https://github.com/Telecommunication-Telemedia-Assessment/bitstream_mode3_p1204_3 repo for a while.
That's what I know is available.

The software is fine (although some more optimization is always welcome). It's getting or building the right ML model with the right ground truth data that's the hard, expensive part. It was the hard part with VMAF as well; probably 90% of the expense was getting all the subjective rating tests generated and rated.

[asarian]

I am about to buy an AMD 99503D, which supports AVX-512 very well. I wanted to prepare a command line with --asm avx512, but fail to see avx512 as a valid string in the documentation any more. Are there still versions of x265 that support this?

[LigH]

Support for AVX512 certainly still exists, I just read it in the sources. But it will not be auto-detected. So add the parameter explicitly; if it is not supported either by hardware or by your OS, or being removed from the CLI without my knowledge, x265 will tell you when you execute it.

[asarian]

Thank you.

[TR-9970X]

Thank you.

I also have a 9950X3D, and I have this:- --asm avx512 in my x265 command line.

The documentation needs to be updated.

[asarian]

I also have a 9950X3D, and I have this:- --asm avx512 in my x265 command line.

Good to hear. :)

Can you tell me something about the performance boost using AVX512 gives? There are many charts on the 99050X3D, and even on them using x265, but never with AVX512 being used. I feel it will make quite a difference.

[LigH]

In general: In the past, the difference between AVX2 and AVX512 used to be marginal, and AVX512 has a higher risk of thermal throttling.

I can't tell you current results though, sorry. Try to search this thread for AVX512, I believe there have been reports a few months ago.

Like here (https://forum.doom9.org/showthread.php?p=2008991#post2008991)

[TR-9970X]

Good to hear. :)

Can you tell me something about the performance boost using AVX512 gives? There are many charts on the 99050X3D, and even on them using x265, but never with AVX512 being used. I feel it will make quite a difference.

I can't give you any info on performance, as I haven't been doing much recently, and I guess I just use it because I can.

I never noticed any thermal throttling with the 7950X, and I would suggest that the 9950X3D & the 7970X won't either, but I do have pretty good cooling.

But one thing I have noticed is 2 major typos in your post's, when referencing the 9950X3D !!!! :(

[Emulgator]

On a i9-11900K I found a few percent speed gain using -- asm avx512, limited by some of Intels safety features (limiting to base clock)
and the available cooling of a 125W CPU+115W GPU notebook system, although with plenty air intake, exhaust and 2x 5000rpm fan).

On a real desktop AMD 7950X/9950X I would expect considerable speed gains in the range of >25%, so well worth it.

[Z2697]

Unfortunatelly they are nowhere near 25% (compared to avx2)

[asarian]

In general: In the past, the difference between AVX2 and AVX512 used to be marginal, and AVX512 has a higher risk of thermal throttling.

That's the beauty if the 9950X3D! Allegedly it includes an extremely efficient implementation of AVX512, where the CPU doesn't reduce clockspeed for them, and only consumes a few Watt extra on AVX512. So, this should go endlessly better on the 9950X3D.

[rwill]

Unfortunatelly they are nowhere near 25% (compared to avx2)

Yeah, there are not that many situations in a HEVC encoder where one needs 512bit wide registers.

[Emulgator]

Yeah, there are not that many situations in a HEVC encoder where one needs 512bit wide registers.
True. I should have gone down to 15% in my estimate.
How many % speed advantage AVX2 vs AVX512 in the described scenario do 7950X/9950X owners note ?

[Z2697]

1%-1.5% and 5%-8% respectively, for at least the set of settings and hardwares that I tested with.
(why there's a range, because lower quality / bitrate have more speedup somehow)

[excellentswordfight]

True. I should have gone down to 15% in my estimate.
How many % speed advantage AVX2 vs AVX512 in the described scenario do 7950X/9950X owners note ?
1%-1.5% and 5%-8% respectively, for at least the set of settings and hardwares that I tested with.
(why there's a range, because lower quality / bitrate have more speedup somehow)
Sounds about right, I've seen about 5-8% with Xeon (Sapphire Rapids), and only 1-3% with 7000-series (Threadripper Pro). With the avx512 change in Zen5 i assume it should be about the same performance gain as the xeon.

[_DLS_]

I am about to buy an AMD 99503D, which supports AVX-512 very well. I wanted to prepare a command line with --asm avx512, but fail to see avx512 as a valid string in the documentation any more. Are there still versions of x265 that support this?

In my tests with the 9950X3D, the speed bump with --asm avx512 can be between 16-36% depending on the other settings too.

on 2160p:

CTU 32, ref 4, subme 4, rd 4, rect, no-amp, aq-mode 2, tu-intra-depth 3, tu-inter-depth 3, max-merge 5, crf 20 => +15.5%
CTU 64, ref 5, subme 5, rd 4, rect, no-amp, aq-mode 2, tu-intra-depth 4, tu-inter-depth 4, max-merge 5, crf 18 => +36%

Temps are manageable on air cooling.

[Z2697]

In my tests with the 9950X3D, the speed bump with --asm avx512 can be between 16-36% depending on the other settings too.

on 2160p:

CTU 32, ref 4, subme 4, rd 4, rect, no-amp, aq-mode 2, tu-intra-depth 3, tu-inter-depth 3, max-merge 5, crf 20 => +15.5%
CTU 64, ref 5, subme 5, rd 4, rect, no-amp, aq-mode 2, tu-intra-depth 4, tu-inter-depth 4, max-merge 5, crf 18 => +36%

Temps are manageable on air cooling.

Kinda too good to be true...

[Z2697]

x265 will produce nonconformity bitstream when a very sudden change in chroma channel happens and triggered weighted prediction.

https://bitbucket.org/multicoreware/x265_git/issues/996/weighted-prediction-delta_chroma_offset_lx
(Finally remembered my bitbucket account)

Related issue (4 years ago, when I didn't know the root cause): https://bitbucket.org/multicoreware/x265_git/issues/582

Many hardware decoders will fail to decode such frame, and result in corrupted output.

HM will abort due to assertion.


int pred = (128 - ((128 * wp[plane].inputWeight) >> (wp[plane].log2WeightDenom)));
int deltaChroma = (wp[plane].inputOffset - pred);
WRITE_SVLC(deltaChroma, "delta_chroma_offset_lX");


TL;DR: the deltaChroma in the code I referenced above should be in the range of [-512, 511], but there’s no check on the value of pred. For example if the value of pred is a fairly “large” negative number, the deltaChroma can exceed the range.

[benwaggoner]

In general: In the past, the difference between AVX2 and AVX512 used to be marginal, and AVX512 has a higher risk of thermal throttling.
More to the point "the difference between AVX2 and AVX512 used to be marginal, BECAUSE of AVX512 thermal throttling."

The per clock throughput improvements were solid, but the throttling reductions in instructions per second nearly cancelled those out. An implementation that maintains IPS with the IPC gains would be quite impressive.

[benwaggoner]

TL;DR: the deltaChroma in the code I referenced above should be in the range of [-512, 511], but there’s no check on the value of pred. For example if the value of pred is a fairly “large” negative number, the deltaChroma can exceed the range.
Good catch!

[asarian]

In my tests with the 9950X3D, the speed bump with --asm avx512 can be between 16-36% depending on the other settings too.

on 2160p:

CTU 32, ref 4, subme 4, rd 4, rect, no-amp, aq-mode 2, tu-intra-depth 3, tu-inter-depth 3, max-merge 5, crf 20 => +15.5%
CTU 64, ref 5, subme 5, rd 4, rect, no-amp, aq-mode 2, tu-intra-depth 4, tu-inter-depth 4, max-merge 5, crf 18 => +36%

Temps are manageable on air cooling.

Now we're talking! :thanks:

I heard +37% elsewhere (but didn't mention that yet, as I was looking for objective feedback). So, it now seems that number was quite accurate!

[Z2697]

x265 will produce non-conformative bitstream when a very sudden change in chroma channel happens and triggered weighted prediction.

Some "synthetic" inupt to trigger that bug :)

import vapoursynth as vs
core=vs.core
i=core.imwri.Read('a.png').resize.Bicubic(format=vs.YUV420P8,matrix=1)
f1=core.std.BlankClip(i,color=[16,16,16])

f1=core.std.Merge(i,f1,0.9)
f2=i
f3=f2.std.FlipHorizontal()

clip=f1*30+f2*9+f3*30 #69
clip.set_output()

(This does not mean only synthetic input can trigger the bug. There are "legit" buggy videos in the wild)
Example encoding commandline: vspipe aa.vpy -c y4m - | x265 --input - --y4m -b0 -o aa.265
I have to set bframes to 0, otherwise even with --weightb enabled, the weightd prediction won't catch up.
(Again this does not mean the bug will only be triggered when bframes=0)

I guess most images will work, but you can use this lovely nostalgic wallpaper:
https://files.catbox.moe/4c2bsn.png

A patch was submitted:
https://mailman.videolan.org/pipermail/x265-devel/2025-May/014363.html
and you can see the process of figuring out the fix at https://github.com/Mr-Z-2697/x265-experimental/tree/E-2025-05-12

[excellentswordfight]

More to the point "the difference between AVX2 and AVX512 used to be marginal, BECAUSE of AVX512 thermal throttling."

The per clock throughput improvements were solid, but the throttling reductions in instructions per second nearly cancelled those out. An implementation that maintains IPS with the IPC gains would be quite impressive.
I dont think this is totally true, i've encoded on pretty muc every generation of xeon and epyc with avx512 support, still have not seen over 10% improvement. And isnt that also expected, I dont see why an encoder like x265 would benefit that much from 512bit wide registers. And in the case with downclocking, it wasnt a marginal difference for the early implementations, it was more outright an performance regression. I think the only systems were this didnt happen was were you could allow for the huge increase in powerdraw.

In my tests with the 9950X3D, the speed bump with --asm avx512 can be between 16-36% depending on the other settings too.

on 2160p:

CTU 32, ref 4, subme 4, rd 4, rect, no-amp, aq-mode 2, tu-intra-depth 3, tu-inter-depth 3, max-merge 5, crf 20 => +15.5%
CTU 64, ref 5, subme 5, rd 4, rect, no-amp, aq-mode 2, tu-intra-depth 4, tu-inter-depth 4, max-merge 5, crf 18 => +36%

Temps are manageable on air cooling.
Can you share source and complete command line and build versions so this can be validated? I dont have a 9000-series, but I have plenty of systems with avx512 support and I have never seen anything close to this. Would be intresseting to see what the specific criteria is to get this increase.

[benwaggoner]

I dont think this is totally true, i've encoded on pretty muc every generation of xeon and epyc with avx512 support, still have not seen over 10% improvement. And isnt that also expected, I dont see why an encoder like x265 would benefit that much from 512bit wide registers. And in the case with downclocking, it wasnt a marginal difference for the early implementations, it was more outright an performance regression. I think the only systems were this didnt happen was were you could allow for the huge increase in powerdraw.
Yeah. While there are some simplifications possible with the bigger registers (fewer instructions per bit, easier byte alignment), if you're touching 2x the bits you'll need some more power and also more cooling because of that extra power.

It's somewhat more complex as AVX-512 also came with some better permute instructions and such that could allow for more efficient algorithm design, as did ARM's SIMD for bigger registers. How much improvement could be gained via better instructions versus just more width is highly architecture and algorithm dependent.

[tormento]

Can someone explain me the criteria that --crf-min and --crf-max use to increase/decrease CRF?

[TR-9970X]

Can someone explain me the criteria that --crf-min and --crf-max use to increase/decrease CRF?

--crf-max <0..51.0>
Specify an upper limit to the rate factor which may be assigned to any given frame (ensuring a max QP). This is dangerous when CRF is used in combination with VBV as it may result in buffer underruns. Default disabled

--crf-min <0..51.0>
Specify a lower limit to the rate factor which may be assigned to any given frame (ensuring a min compression factor).

[Z2697]

While qpmax is "global", crfmax is only effective when used together with VBV (and of course CRF itself, hence CRF + VBV aka "capped CRF").

BTW, Read The Manual.

BTW (A huge one)

VBV is non-deterministic (when it's engaged), VBV + const-vbv is deterministic (without significant quality drawback, if any, it's hard to do comparison because of the non-deterministic), but const-vbv is not enabled by default.

VBV + const-vbv + qpmax/crfmax (when "hit" by) + rskip=1 is non-deterministic again, use rskip=0 or 2 with this combination makes it deterministic again.
I guess that's related to how rskip=1 is doing skip decision using the RD cost, with 0 you don't skip and 2 is using edge density, which is independent from the RD stuff.

[tormento]

--crf-max <0..51.0>
Your answer is not an answer. It's the manual, that I just read and it arose doubts in me.

Thank you.

[tormento]

BTW
I just want to know what are the criteria used to increase, decrease QP.

SSIM? What?

[LigH]

The "rate factor" is a codec-specific internal metric of the HEVC algorithm, a kind of magnitude of difference between the original video and the reconstructed video, just calculated during the lossy compression, without actually reconstructing and subtracting. You may imagine the codec adding details to a coarse encoding result until a threshold of quality is passed. But the RF may vary for different GOP frame types (e.g. B frames may have a slightly bigger RF than P frames than I frames due to a dependency hierarchy) or due to an encoding mode trying to achieve a less variable bitrate for streaming instead of archival use or VBV restrictions. The two mentioned parameters force the RF in an explicit range and overrule the optimal RF calculated by quality or bitrate based decisions.

I bet Ben "Video Kenobi" Waggoner knows a lot more than me about that...

[tormento]

The two mentioned parameters force the RF in an explicit range and overrule the optimal RF calculated by quality or bitrate based decisions.
So, imposing them is a limitation and not giving more degrees of freedom?

I mean: I know that even setting a CRF, the bitrate can fluctuate a lot but I can't understand if there are no restraints but the vbv values.

In a few words: imposing qp-min and max gives less or more "freedom" to x265 encoder?

[LigH]

I believe to remember that some Presets / Tunings may limit the RF range. A maximum of 51 may be counterproductive to some uses. Don't count on me here, I would have to search the sources to be sure.

[Z2697]

While qpmax is "global", crfmax is only effective when used together with VBV (and of course CRF itself, hence CRF + VBV aka "capped CRF").

What else criteria do you need

(IDK how crf-min will work because... does VBV also increase bitrate?(apart from the "decrease over limit bitrate" we all know it does) otherwise why should we worry about minimum RF anyway?(I'm reluctant to call it CRF(constant rate factor) because it's certainly not constant anymore unless the limit is so high that it's never touched))

[benwaggoner]

In a few words: imposing qp-min and max gives less or more "freedom" to x265 encoder?
They give the encoder less flexibility. A higher min keeps the encode from getting deep into visually lossless even if there are bits available. Although having a meaningful --crf would set a higher minimum than qp-min. qp-min is something I've only used with CBR encodes.

qp-max will cap the maximum compression. I've never used it myself for anything. I'd be wary of it, because the encoder could get into states where it couldn't maintain VBV if qp-max is too low.

[Z2697]

VBV in x265 and x264 is a mess, and can well blow up quite easily (in x264 it's a little better, but still awful), I'd suggest stay away from it, especially as a way of reducing file size, unless you are limited by some target device and/or transmitting media (e.g. bluray)

("blow up" in image quality, the buffer itself is not that easy to break)

[benwaggoner]

VBV in x265 and x264 is a mess, and can well blow up quite easily (in x264 it's a little better, but still awful), I'd suggest stay away from it, especially as a way of reducing file size, unless you are limited by some target device and/or transmitting media (e.g. bluray)

("blow up" in image quality, the buffer itself is not that easy to break)
"unless you are limited by some target device and/or transmitting media" is basically were most of us live, of course. Streams need to work within a huge variety of HW decoders Profile and Level requirements.

Hence 2-pass, lookahead rate control, and lots of other mechanisms to maintain optimal quality with VBV on.

[benwaggoner]

So, imposing them is a limitation and not giving more degrees of freedom?

I mean: I know that even setting a CRF, the bitrate can fluctuate a lot but I can't understand if there are no restraints but the vbv values.
VBV limits the maximum peak bitrate, which provides a maximum average bitrate, but the ABR can also be way less. In a lot of common encoding scenarios the VBV isn't a factor 99% of the time.

In a few words: imposing qp-min and max gives less or more "freedom" to x265 encoder?
Less. When they aren't specified x265 can use anything in the possible range.

[Barough]

x265 v4.1+168-1e993ee
https://www.mediafire.com/file/aru29lv3c52vhmr

[LigH]

VBV does not only limit a maximum bitrate. In case of optical media it also limits a minimum bitrate because reading too slowly may cause loss of tracking, and filling the decoding buffer with more than one GOP causes the "buffer underrun"...

[benwaggoner]

VBV does not only limit a maximum bitrate. In case of optical media it also limits a minimum bitrate because reading too slowly may cause loss of tracking, and filling the decoding buffer with more than one GOP causes the "buffer underrun"...
Optical media is always full of exciting exceptions!

As is streaming to a lesser degree. People who just encode files for local playback don't know how easy they have it ;).

[cubicibo]

VBV does not only limit a maximum bitrate. In case of optical media it also limits a minimum bitrate because reading too slowly may cause loss of tracking, and filling the decoding buffer with more than one GOP causes the "buffer underrun"...

? There are no minimum required bitrate for physical media. Some decoders do choke with abysmal bitrates but that's an implementation problem.

Tracking is guaranteed at all time with the usual PMT, PAT and PSI repetitions in the transport stream.

[LigH]

? There are no minimum required bitrate for physical media. Some decoders do choke with abysmal bitrates but that's an implementation problem.

Once upon a time, when I worked in a DVD Video authoring studio, some of the several dozen DVD Players certainly did not enjoy longer black screens, so developers of MPEG2 encoders visited us to tune their products... my very rusty 2 cents of practical experience. I will be glad to hear that Blu-ray players today are more robust in this regard.

[Z2697]

just don't use too many b frames, they will slow down the encoding with b-adapt and has no real benefit.

Some thought (theory) on this:
x265 only uses one middle frame in a mini-GOP (like a sequence of b frames) as reference frame (or not at all if b-pyramid is disabled),
which means in a long mini-GOP, "far" b frames will have "far" references, and have more residual to code... unless the scene is still.

[tormento]

I am asking myself what values should I use with --chromaloc.

I can obtain the original video one with

ffprobe -v quiet -loglevel panic -print_format json -show_format -show_streams <file>

and I know that

0 left (1080p)
1 center
2 topleft (2160p)
3 top
4 bottomleft
5 bottom

where usually I find 0 for 1080p avc and 2 for 2160p hevc.

What happens if I change the codec and/or the size of the video?

I.e. if I downsize a hevc video from 2160p to 1080p, should I:

1) convert topleft to left
2) apply wanted filters
3) set --chromaloc to 0 (left)

or

1) leave chroma location where it is and don't set --chromaloc at all?

[microchip8]

I always use chromaloc 2 when downsizing 2160p to 1080p. No issues here

[Z2697]

How is this related to x265 specifically?

[FranceBB]

What happens if I change the codec and/or the size of the video?

I.e. if I downsize a hevc video from 2160p to 1080p

Resizers in Avisynth are chroma location aware, which means that if you start with a 3840x2160 with top_left chroma location (i.e 4:2:0 Type 2) and you downscale it to 1920x1080, the chroma location of that will stay the same so you should still be flagging it as top_left. This is what Microchip8 seems to be doing:

I always use chromaloc 2 when downsizing 2160p to 1080p. No issues here

and he's right... BUT (yes, there's a "but") although a player should understand and honor the chroma placement metadata when upscaling the chroma and converting to RGB, there's no guarantee that every single player is gonna do it. You see, taking aside special use cases like MPEG-1 with the center chroma placement etc, for consumers the chroma location really actually became important when H.265 was introduced as it moved the default from left (normal 4:2:0) to top_left (4:2:0 type 2), which means that an astonishing amount of old players just automatically assumed left chroma location all the time before this. If you're downscaling to normal FULL HD 4:2:0 and you're feeding an old hardware player then you *might* be in trouble as it *might* assume left, regardless of what you're flagging the file as.

For your use-case, Tormento, I would personally do:

z_ConvertFormat(chromaloc_op="top_left=>left")

to go from 4:2:0 Type 2 to the normal 4:2:0 and then encode setting the left chroma placement. ;)

[microchip8]

Yea, I don't have anything that old here that doesn't understand chroma location. All my players/streamers are 4k-aware and I mostly use ffmpeg for the resizing (Lanczos3 with accurate_rnd) and encoding, which too has chroma-aware scaling. I sometimes use a zimg/zlib scaler and don't have any issues either.

[rwill]

, for consumers the chroma location really actually became important when H.265 was introduced as it moved the default from left (normal 4:2:0) to top_left (4:2:0 type 2),

Well its not that simple.

The standard says:


When the chroma_sample_loc_type_top_field and chroma_sample_loc_type_bottom_field are not present,
the values of chroma_sample_loc_type_top_field and chroma_sample_loc_type_bottom_field are inferred to be equal to 0.


With the following additional constraint:


When chroma_format_idc is equal to 1 (4:2:0 chroma format) and the decoded video content is intended for interpretation
according to Rec. ITU-R BT.2020-2 or Rec. ITU-R BT.2100-2, chroma_loc_info_present_flag should be equal to 1, and
chroma_sample_loc_type_top_field and chroma_sample_loc_type_bottom_field should both be equal to 2.


So 4:2:0 BT.2020-2 and BT.2100-2 need to always signal chroma loc 2 but others default to 0 if it is not signaled otherwise.

[hellgauss]

Thank you for interesting reading!

What is exactly chroma location? AFAIK in "standard" 4:2:0 video I have one luma plane at full resolution and two chroma planes at quarter (2x2) resolution. Just overlap the three planes to get the final image, perhaps using some "smart" upscale for chroma. How does chroma location enter in this process?

In my mind I have two blocky images for chroma and one fine image for luma in which 4 pixels fit exactly into one chroma pixel. Is this interpretation correct?

[rwill]

Thank you for interesting reading!

What is exactly chroma location? AFAIK in "standard" 4:2:0 video I have one luma plane at full resolution and two chroma planes at quarter (2x2) resolution. Just overlap the three planes to get the final image, perhaps using some "smart" upscale for chroma. How does chroma location enter in this process?

In my mind I have two blocky images for chroma and one fine image for luma in which 4 pixels fit exactly into one chroma pixel. Is this interpretation correct?

Well you can download https://www.itu.int/rec/T-REC-H.265-202407-I and take a look.

On .pdf page 456 (or page no. 438 as stated in the document) are two figures, E.1 and E.2.

E.1 shows the chroma sample location relative to the luma top and bottom fields. E.2 might be a bit easier to understand, it shows which area a chroma sample covers relative to the 4 luma samples it is associated with.

As can be seen in E.2, the easiest one would be chroma loc 1, which is the chroma sample being centered in the middle of the 4 luma samples. I guess this is how Mpeg-1 worked. Now when upsampling one needs to go 1 to 2x2 samples. If one does not want to apply some fancy smooth filter there one can can just assign the chroma sample value to all 4 luma samples the chroma sample covers.

Other locations require other upsampling filters, depending on the position and use case these can be somewhat larger than '1 sample' filters.

Simple chroma loc 0 upsample can look like so:

Horizontal Upsample:
for even luma0 sample: chromasample0
for odd luma0 sample: ( chromasample0 + chromsample1 + 1 ) / 2

( chromasample1 is the one from the next 2x2 luma block )

Vertical Upsample:
for top luma0 sample: ( 3 * chromsample0 + 1 * 'chromsample-1' + 2 ) / 4
for bottom luma0 sample: ( 3 * chromsample0 + 1 * chromsample1 + 2 ) / 4

( chromasample-1 is the sample from the row above and chromsample1 is the sample from the row below )

I hope this makes sense.

[Z2697]

It's amazing how they have come up with more complex "solutions" instead of a obvious and simple one.

[hellgauss]

Thanks for precise reference!

Indeed standards are sometimes misteriosly overcomplicated. I had chosen only Chromaloc=1, perhaps with an optional flag for the default chroma upscale (e.g. point, bilinear, bicubic soft, bicubic). Perhaps there are some cases or hidden reasons to shift chroma half a pixel.

[benwaggoner]

Generally I only use chromaloc 2 for actual Blu-ray authoring. In practice a lot of early UHD mezzanines and HEVC decoders did everything as chromaloc 0 irrespective of metadata.

With UHD Blu-ray at 2160p getting it wrong made only subtle, easy to miss differences. But the lower the resolution, the more visible an offset error can be.

[RARY]

I'm planning to buy a 4K TV and would appreciate your recommendations.
Looking for models that support HEVC, VVC, and MV-HEVC decoding, and also have Dolby Vision support.
If you've come across any good options recently, please do share!

[tormento]

In practice a lot of early UHD mezzanines and HEVC decoders did everything as chromaloc 0
Is there any way to find real position, when metadata is wrong?

[benwaggoner]

I'm planning to buy a 4K TV and would appreciate your recommendations.
Looking for models that support HEVC, VVC, and MV-HEVC decoding, and also have Dolby Vision support.
If you've come across any good options recently, please do share!
No one is making stereoscopic 3D TVs anymore, so any MV-HEVC playback would just be single view anyway.

The three questions in picking a TV are

The ambient light in your viewing environment.
How big you want it to be. I recommend 10" diagonal for every foot your eyes will be from the screen.
How much you want to pay.


OLED is best if you have good light control, but can't go as bright if you don't. Price/Size tradeoffs are personal. Sometimes the best solution is to push your couch forward :sly:.

I'm very happy with both the Sony A95L and LG G5 for OLED Dolby Vision supporting TVs. I've heard great things about the Bravia 8 Mark II, but haven't evaluated it with my own eyeballs yet. It only goes up to 65" though. A95L goes up to 77" and the G5 to 98" (at an astronomical price).

We are truly living in the golden age of display innovation, with mid tier TVs today outperforming the high-end of just five years ago. Get one of the above and put it in Filmmaker Mode, and you can get very close to what the creatives approved on their $30K studio color grading reference monitors. Today's $400 TV is much better than a $4000 TV of ten years ago.

Samsung is also making great TVs, but they don't support Dolby Vision.

[Blue_MiSfit]

Somewhat OT, do you see any significant benefit of Dolby Vision (or any other system using dynamic metadata for color volume transformation) in content that's graded to a 1000 nit peak on a display that can meet or exceed 1000 nits consistently, Ben?

I know with DoVi P5 / P10 there's the proprietary IPTPQc2 space that allows using full range and dynamic shaping from 12+ bits down to 10 bits while the RPUs can reconstruct closer to 12 bit but... in practice this requires a lot of encoder optimization and is maybe kinda not that big of a deal at the end of the day if you're not doing any mapping?

@Rary, you should probably start a new thread :)

[RARY]

Has anyone explored efficient multi-threading strategies for the MCSTF module?

[benwaggoner]

Somewhat OT, do you see any significant benefit of Dolby Vision (or any other system using dynamic metadata for color volume transformation) in content that's graded to a 1000 nit peak on a display that can meet or exceed 1000 nits consistently, Ben?
Not in an appropriately dim ambient lighting environment, no. Dynamic metadata helps map content to the display when the display can only show a subset of the content itself. But if you're watching 400 nit P3 content on a higher end OLED in a 5 nit surround, the display can just show each pixel's spec value perfectly, and the dynamic tone mapping isn't needed. Where tone mapping and dynamic metadata are useful is when content is brighter or more colorful than the display can reproduce, inclusive of ambient light adaptation.

Content can vary a lot, of course. A lot more displays can perfectly reproduce something like Rings of Power than can Inside Out, for example.

I know with DoVi P5 / P10 there's the proprietary IPTPQc2 space that allows using full range and dynamic shaping from 12+ bits down to 10 bits while the RPUs can reconstruct closer to 12 bit but... in practice this requires a lot of encoder optimization and is maybe kinda not that big of a deal at the end of the day if you're not doing any mapping?
Yeah, in theory P5 can help reducing banding as you can have the full 0-1023 range used for any given frame. In practice I've not really seen big net differences from DoVi Profile 8.1 if the HDR-10 base layer is encoded well. Avoiding banding with P5 is easier and can be done at a lower bitrate than 8.1 sometimes. Avoiding banding with HDR-10 can requires some finicky tweaking.

Encoding Profile 5 isn't super hard. Weighted prediction is super important, of course, and you want to use an asymmetric chroma offset because Ct and Cb are more psychovisually distinct than Cb and Cr. It certainly can be done well with stock x265 if one didn't want to use Dolby's tools for some reason. No --hdr10-opt of course!

[Z2697]

Has anyone explored efficient multi-threading strategies for the MCSTF module?

MCSTF is completely useless. Just don't think about it.
It actually reduces quality, unless I'm not testing it right.
There will be ghosting in frames, which are actually it's intent, if i understand it correctly. (imagine a MDegrain filter without strength control, and limiting)
Since there's no "hidden ref frames" in Main HEVC, I guess it's unavoidable. (like how VP-series/AV1 can have temporal filtered hidden alt-ref)
(I use "Main HEVC" because there're extensions, like multi view, I don't know, maybe there will be this kind of extension. Or maybe someone can exploit the views to create hidden reference frames.)

[Balling]

Thank you for interesting reading!

What is exactly chroma location? AFAIK in "standard" 4:2:0 video I have one luma plane at full resolution and two chroma planes at quarter (2x2) resolution. Just overlap the three planes to get the final image, perhaps using some "smart" upscale for chroma. How does chroma location enter in this process?

In my mind I have two blocky images for chroma and one fine image for luma in which 4 pixels fit exactly into one chroma pixel. Is this interpretation correct?

No. You have one plane at 8 bit resolution and then another plane at 4 bit resolution that has both Cb and Cr (that is Cb is not 2 bit and Cr is not 2 bit, they both occupy the same 4 bits) and so basically the Chroma sample since it is less resolution needs to be decoded correctly in a single fashion. And it so happens that since both chroma channels are two times less size than Luma it can be when decoded moved in different directions, in fact ffmpeg supports arbitrary direction of the chroma placement.

[Asmodian]

How does chroma location enter in this process?

In my mind I have two blocky images for chroma and one fine image for luma in which 4 pixels fit exactly into one chroma pixel. Is this interpretation correct?

This wikipedia article has good diagrams of the different locations that are used for 4:2:0.
https://en.wikipedia.org/wiki/Chroma_subsampling

[hellgauss]

@Balling
Sorry, I did not understand your answer. I think bit depth is unrelated to the topic.

@Asmodian
Thanks for reference, those are basically the same pictures as pointed by rwill in the standard. My [possibly wrong] understanding now is that, since a 4:4:4 image must be somewhat reconstructed at some point, the chromaloc define how the half resolution chroma planes must be upscaled and interpolated at full res: either centered or shifted up/down-left/right a little bit during interpolation.

Perhaps there can be specific algorithm to directly upscale/downscale both luma and chroma at screen resolution in one step.

[Asmodian]

Perhaps there can be specific algorithm to directly upscale/downscale both luma and chroma at screen resolution in one step.

You need to scale the three planes separately, but you can scale the chroma planes and center them with luma in one step.

[FranceBB]

since a 4:4:4 image must be somewhat reconstructed at some point, the chromaloc define how the half resolution chroma planes must be upscaled and interpolated at full res: either centered or shifted up/down-left/right a little bit during interpolation.

Yes, pretty much.
4:2:0 -> 4:4:4 -> RGB -> Monitor representation.

You start with the chroma half the resolution of the luma, then it gets upscaled to 4:4:4 so that it has the same resolution as the luma and finally it gets converted to RGB so that it can be represented on the display. Unrelated, but in this step the levels are also converted.

Given that you're Italian, if you're interested in the topic, I actually strongly suggest you to read the first few pages of my 2024 academic publication that you can find here: Link (https://view.publitas.com/p222-8308/francesco-bucciantini-utilizzo-dei-gan-nellupscale-delle-immagini-fisse-ed-in-movimento-2024-edition). I'd say up until page 16 as I go through luma and chroma, 4:2:0, 4:2:2, 4:4:4 and the upscale of the chroma for the RGB conversion so that it can be displayed by the TV subpixels which are, of course, red, green and blue.

For instance, you can think about the process as:


#4:2:0 with 3840x2160 luma and 1920x1080 chroma
ColorBars(3840, 2160, pixel_type="YV12")

#Extracting 3840x2160 luma
Y=ConvertToY8()

#Upscaling the 1920x1080 chroma
U=UToY8().Spline64Resize(3840, 2160)
V=VToY8().Spline64Resize(3840, 2160)

#4:4:4 with 3840x2160 luma and chroma
YToUV(U, V, Y)

#RGB24 to display on the TV
ConvertToRGB24()


https://i.imgur.com/i9Ir3Fw.png

If we were to try to display the chroma without upscaling it, we would end up with a bit of a mess like this:


ColorBars(3840, 2160, pixel_type="YV12")

Y=ConvertToY8()

U=UToY8().AddBorders(0, 0, 1920, 1080)
V=VToY8().AddBorders(0, 0, 1920, 1080)

YToUV(U, V, Y)


https://i.imgur.com/8Y6z24j.png

If we got the wrong sample location, we would end up in a different mess in which the chroma wouldn't be properly aligned with the luma.
Imagine like the colorbars above but with like the chroma of, let's say, one bar shifted on one side etc.
So yeah, the TV is upscaling the chroma to make it match the luma resolution before converting to RGB and the chroma location is telling it where the chroma samples are and where to shift them to make them match the luma so that when they're overlaid one on top of each other they match perfectly. ;)

[Barough]

x265 v4.1+189-c8ceb6b
GCC 15.1.0 / Win32/64 / 8bit+10bit+12bit

https://www.mediafire.com/file/6xt6fi7fryn40tr

[jpsdr]

@Barough
Hi.
What repository are you using ?
I've just checked the one i'm using (the master branch), i'm still only at "+136".

[Z2697]

@Barough
Hi.
What repository are you using ?
I've just checked the one i'm using (the master branch), i'm still only at "+136".

https://forum.doom9.org/showthread.php?p=2010341#post2010341

[jpsdr]

I forgot this one...

[Barough]

@Barough
Hi.
What repository are you using ?
I've just checked the one i'm using (the master branch), i'm still only at "+136".

https://bitbucket.org/multicoreware/x265_git.git

[jpsdr]

https://bitbucket.org/multicoreware/x265_git.git

So do I.
So it must be what Z2697 linked about.

[Barough]

So do I.
So it must be what Z2697 linked about.

Beats me....

[LigH]

I stopped compiling x265 since an issue with the version numbering in M-AB-S was discovered, which should be solved as soon as a new version gets tagged.

[Balling]

@Balling
Sorry, I did not understand your answer. I think bit depth is unrelated to the topic.


It is unrelated, that was just an example. That is for 10 bit it would be 5 bit used for Cb and Cr together. What I am saying is that Cb and Cr together is less data, so it needs to be restored to full 10 bit, that 5 bit needs to be somehow upscled using some FIR algorithm to 20 bits. 5 —> 20 bit. Yep

[LigH]

Sorry, Balling, but your explanation is not correct. All the channels (luma Y as well as chroma differences Cb and Cr) have the same bitdepth (say, precision of values). But due to Chroma Subsampling they don't have the same resolution. In case of YUV 4:2:0, which is most common, chroma planes store average chrominance difference values for each square of 2×2 luminance samples (more or less equal to pixels).

For example, if you have a video with 1280×720 nominal pixels, the luma plane Y has 1280×720 sample values, but the chroma planes Cb and Cr only store 640×360 sample values. All of them have the same precision, though, e.g. 8 or 10 bit per sample value. Splitting these chroma bits per pixel is not valid.

[Z2697]

I guess it's just a very unintuitive way of saying "chroma planes are lower resolution and equivalently less bits". (storage bits per sample)
Which makes little sense. But they are somewhat "frequently" used in the general topic.
Like the "12" in "NV12" (I assume). But then you have "NV21" which makes no sense again. This whole thing is just a hot mess.
You think when the planes are all full resolution, the situation should be better? LOL! Meet RGB16(5-6-5) and Y416.

Anyway, just like anything in the life, you learn the rules, and then you learn the f..kton lot of exceptions to the rules.
"Here's how we name things."
"Here're how we name things differently when they are xxx." (repeated 1000 times)

I don't think Balling is a newbie that mixes up this nonsensical equivalent bits per sample with the precision, but this way of representing things is ambiguous and should be avoided.
Interpolation to more bits makes no sense.

[LigH]

Alright.

It's just sad that the more things are named wrong out of convenience, the fewer precise terms are left which are not yet "burned".

So many terms which originally had a sensible meaning but now are misunderstood by the average people because they never heard of the original meaning and only ever heard the perverted one.

Sorry, off-topic rant.

[Z2697]

I don't think they are even convenient at all, or maybe they were convenient at some point, but caused more trouble down the road.

[benwaggoner]

I don't think Balling is a newbie that mixes up this nonsensical equivalent bits per sample with the precision, but this way of representing things is ambiguous and should be avoided.
Interpolation to more bits makes no sense.
Bits per pixel is really important when figuring out GPU memory requirements, memory bus speeds, etcetera. The biggest value of 4:2:0 is probably in only having to half as much as 4:4:4 in the uncompressed domain in and between hardware. Big cost and heat savings for devices pre/post compression.

Hence HDMI that can do 4Kp60 in 4:2:2 but only 4Kp30 in 4:4:4.

[Z2697]

Bits per pixel is really important when figuring out GPU memory requirements, memory bus speeds, etcetera. The biggest value of 4:2:0 is probably in only having to half as much as 4:4:4 in the uncompressed domain in and between hardware. Big cost and heat savings for devices pre/post compression.

Hence HDMI that can do 4Kp60 in 4:2:2 but only 4Kp30 in 4:4:4.

Which can be easily calculated, at the same time as the resolution calculation.
We still need to calculate resolution with bits per pixel, maybe we should use 24,883,200 bits per frame or 5,97,196,800 bits per second instead, for ultra convenience!

While bitdepth is a defined value.

I get the point that we have formulas, theorems, constants, etc. for complex things, but this one is too simple to worth creating a confusion.

[LigH]

Í don't mind an "average bits per pixel" as long as it is clearly a statistical and not a nominal value.

[rwill]

You are all missing the point. Balling is a troll that should be banned.

[hellgauss]

Changing subject a little bit: what is the "optimal" value for merange?

I use the value 58 with me=star. It seems that something happens if merange >58 , or >57 with me=hex and with other default settings.

According to x265 official reference:

"If the search range were any larger than this, another CTU row of latency would be required for reference frames."

What is the meaning of this? How this threshold [negatively?] affect encoding or decoding? Perhaps it is related to levels and profiles? For placebo preset the threshold is ignored and merange is set to 92. Is this another threshold?

Thank you

[Z2697]

The latency being referenced in that part is related to multi-threading performance, it's not significant to the overall latency, and latency is usually not a concern in transcoding tasks, plus in this specific case we know you use single thread setting so nothing to worry about.

[benwaggoner]

The latency being referenced in that part is related to multi-threading performance, it's not significant to the overall latency, and latency is usually not a concern in transcoding tasks, plus in this specific case we know you use single thread setting so nothing to worry about.
Although adding another CU row to the motion search could increase memory bandwidth and other operations, so it might slow things down quite a bit more going from 57 to 59 than from 55 to 57.

Easily testable, of course; no need to speculate!

[Z2697]

merange 49
q bitrate vmaf speed bd-rate args
14 11201.07 kbps 98.83799352846535 29.42 fps vmaf: 0.00% x265 --y4m -D10 --no-sao --preset slow --no-rect --merange 49 --crf {q} --no-info -o "{o}" -
18 6834.15 kbps 98.64150103225717 32.91 fps
22 4382.48 kbps 98.27342784339902 37.18 fps
26 2889.94 kbps 97.4778062077497 41.89 fps
merange 55
q bitrate vmaf speed bd-rate args
14 11173.45 kbps 98.8377024425385 26.88 fps vmaf: -0.55% x265 --y4m -D10 --no-sao --preset slow --no-rect --merange 55 --crf {q} --no-info -o "{o}" -
18 6810.03 kbps 98.64165278414207 30.58 fps
22 4355.67 kbps 98.27355641995891 34.42 fps
26 2875.14 kbps 97.47837579299564 38.65 fps
merange 57
q bitrate vmaf speed bd-rate args
14 11151.94 kbps 98.83664988596232 26.2 fps vmaf: -0.65% x265 --y4m -D10 --no-sao --preset slow --no-rect --merange 57 --crf {q} --no-info -o "{o}" -
18 6803.86 kbps 98.64255416609721 30.35 fps
22 4349.29 kbps 98.2727458272399 33.81 fps
26 2869.8 kbps 97.47667330704333 37.68 fps
merange 59
q bitrate vmaf speed bd-rate args
14 11151.64 kbps 98.83703011576269 25.05 fps vmaf: -0.65% x265 --y4m -D10 --no-sao --preset slow --no-rect --merange 59 --crf {q} --no-info -o "{o}" -
18 6793.14 kbps 98.6400538109956 29.46 fps
22 4347.95 kbps 98.27270762871724 33.28 fps
26 2869.68 kbps 97.4788605657337 37.43 fps
merange 65
q bitrate vmaf speed bd-rate args
14 11115.13 kbps 98.83714610372621 23.86 fps vmaf: -1.17% x265 --y4m -D10 --no-sao --preset slow --no-rect --merange 65 --crf {q} --no-info -o "{o}" -
18 6761.22 kbps 98.6421163335067 27.38 fps
22 4325.48 kbps 98.27250615125067 30.52 fps
26 2851.55 kbps 97.47258618532004 34.09 fps

A simple and not very strict test shows no measureable difference.

The "threshold" and the CTU size is tested, the average fps's of the test groups are:
("R" for range)
R49 - 35.35
R55 - 32.63
R57 - 32.01
R59 - 31.31
R65 - 28.96

R55/R57 = 101.94%
R57/R59 = 102.25%
Range ± 2 the difference is less than 0.31%

R49/R57 = 110.43%
R57/R65 = 110.52%
Range ± 8 the difference is less than 0.1%

However there's the tendency that "increased range slows more" shown in both groups, but that's probably because the order the tests were ran, later runs have more saturated thermal, I'd say that zero point something percent is within that error range.

I did use HWiNFO to monitor the memory bandwidth but I don't have the data saved. But the bandwidth difference is also insignificant, if you'd trust me.

[Z2697]

BTW if you are looking to improve on the ME step, I recommend using HME, even the very basic 3 level HEX search have better/close to vmaf bd-rate than STAR 92 and is faster.

[LigH]

A long time ago I tried to visualize the different motion search algorithms of x265 (https://forum.doom9.org/showthread.php?p=1789660#post1789660) with up to 3 steps in recursive methods. AFAIR I made mistakes understanding them...

[hellgauss]

Thank for replies. It seems that the only drawback is a little bit of cpu/ram consumption, I assume only in encoding.

Also thanks for suggesting hme, I will try it, although it seems a little bit complex to configure parameters.

[LigH]

a little bit of cpu/ram consumption, I assume only in encoding.

Of course. The decoding will not know how elaborate the motion vectors had been searched in the encoding. It only decodes the final bitstream.

[Blue_MiSfit]

Great visualizations, LigH! Thanks for sharing

[Balling]

Sorry, Balling, but your explanation is not correct. All the channels (luma Y as well as chroma differences Cb and Cr) have the same bitdepth (say, precision of values). But due to Chroma Subsampling they don't have the same resolution.

The have the same bitdepth when converted to 444. In hardware 420 takes 10 bit for Y plane and 5 bit for both Cb, Cr planes. 15 bits max, and 1 bit as padding.

[Z2697]

The have the same bitdepth when converted to 444. In hardware 420 takes 10 bit for Y plane and 5 bit for both Cb, Cr planes. 15 bits max, and 1 bit as padding.

They have same bitdepth no matter what.
There's no such padding. The data in any case doesn't exist that way. YUV is either packed or planar.

[LigH]

As I said, there is a difference between technical per-sample values (a close-up look into the RAM) and statistical averages over a large number of pixels (seen from a distance).

[Z2697]

Scenecut not working when not using b frames.
https://bitbucket.org/multicoreware/x265_git/issues/1012/scenecut-not-working-when-bframes-0

Now the fix (or to bring it "on par" with x264) will also affect the "normal scenecut" (with b frames being used) slightly, presumably make it more accurate.
Does anyone wants to test? (The main reason I post this;))
https://pixeldrain.com/u/LRqwuw8L (Windows 10 x64 executable, 10bits only)

x264 default this to 1 (C doesn't have bool type, it's int).
It should only set to false when the scenecut is extremely unlikely, not default to false.

diff --git a/source/encoder/encoder.cpp b/source/encoder/encoder.cpp
index 44a09ba..19c7e0d 100644
--- a/source/encoder/encoder.cpp
+++ b/source/encoder/encoder.cpp
@@ -1605,7 +1605,7 @@ int Encoder::encode(const x265_picture* pic_in, x265_picture* pic_out)
inFrame[layer]->m_encodeStartTime = x265_mdate();
/* Set lowres scencut and satdCost here to aovid overwriting ANALYSIS_READ
decision by lowres init*/
- inFrame[layer]->m_lowres.bScenecut = false;
+ inFrame[layer]->m_lowres.bScenecut = true;
inFrame[layer]->m_lowres.satdCost = (int64_t)-1;
inFrame[layer]->m_lowresInit = false;
inFrame[layer]->m_isInsideWindow = 0;



in "bool Lookahead::scenecut" at encoder/slicetype.cpp:

if (bRealScenecut && m_param->bframes)
{
... do flash detection stuff ...
}

if (m_param->csvLogLevel >= 2)
{
... csv logging stuff ...
}

if (!frames[p1]->bScenecut)
return false;

return scenecutInternal(frames, p0, p1, bRealScenecut);


So without bframes - hence without going through that flash detection branch, and I guess that makes sense because only bi-directional prediction have a chance to code flashes more effieicntly - the function will always return false when bScenecut is default to false.

The bScenecut was changed from true to false in

commit f29f2e0071332b708b1d00e61332d12a237b4427
Author: Aarthi Thirumalai <none@none>
Date: Wed Jul 22 00:29:26 2015 +0530

slicetype: Modify Scenecut algorithm to detect scene transition points
to improve Rate Control (refs #160).

identify scene trasitions, fade-ins, fadeouts, sceneCuts and signal the flag bSceneCut
in Lowres structure. This flag will be used by RateControl to adjust the wps during scene cuts.

--HG--
branch : stable

Maybe it's not a good idea to change it back?

[hellgauss]

About a year ago I found a weird behaviour in x264 scenecut with high b-frame number and b-adapt=2. In x264 I use b-adapt=1 for this issue. The problem seems not to be present in x265, I hope that it will not be re-introduced in a future version.

https://forum.doom9.org/showthread.php?t=185544

The last graph I posted explains what to me seems to be a bug (not monotonically incresing curve with high b-frames and b-adapt=2). The graph are y=number of K/I frames, x=scenecut parameter.

[Z2697]

x265 seems to not care scenecut value too much, especially when you increase it, scenecut-bias has a more direct effect.
Whatever. I'm not planning to modify scenecut parameters.

[rwill]

About a year ago I found a weird behaviour in x264 scenecut with high b-frame number and b-adapt=2. In x264 I use b-adapt=1 for this issue. The problem seems not to be present in x265, I hope that it will not be re-introduced in a future version.

https://forum.doom9.org/showthread.php?t=185544

The last graph I posted explains what to me seems to be a bug (not monotonically incresing curve with high b-frames and b-adapt=2). The graph are y=number of K/I frames, x=scenecut parameter.

And you got told that this is no problem with x264 but its flash protection. Still yet here we are, again, with you stating its a problem.

[hellgauss]

And you got told that this is no problem with x264 but its flash protection.

It can be reasonable that, ceteris paribus, flash protection decrease IDR as bf increase. What I claim as "weird" is that, as long as bf is FIXED and very high and b-adapt=fixed=2, the IDR number as a function of scenecut parameter, in the "standard" range 25-70, is decreasing, which is in contrast with what I expect and with MEGUI wiki:

"Higher values of scenecut increase the number of scenecuts detected"

Which I understand as "If you want more IDR frames, increase scenecut".

If it is flash protection it is definitely a not wanted behaviour.

Now, this is x264 stuff and perhaps not in topic here... however Z2697 pointed out that

- New x265 code will be somewhat inspired by x264 behaviour
- Flash protection is related to this issue.

so I just wanted to point out my old thread.

[Z2697]

x265 is based on x264. Not will be.
Quite some drastic devalopments on the HEVC tools, but the fundamental design, especially the psychovisual optimizations, are very much like x264.

To my way of thinking, flash protection is "unrelated" to this issue. Not going thorough flash protection + the default value of bScenecut + the return false branch is the problem.

[DTL]

A simple and not very strict test shows no measureable difference.


In theory merange in MPEG encoders are only limitation for poor performance hosts and some trade between performance and quality. For best quality it always must search with full frame. And result significantly depends on the motion type/complexity in the tested footage. To get benefit from full frame search the test dataset must have lots of motion up to significant size of frame between each frame. And moved blocks must have not too many other types of transforms so a motion search engine can verify it is found. Yes - many natural scenes have much smaller motion vector lengths and typical merange values are simply statistically adjusted by developers to get some average performance/quality balance at some average footage. If we will have good progress in computing hosts performance limitation on merange expected to be removed (or relaxed to most long motion vector allowed in given MPEG standard).

Also currently used motion search methods may be still far for being perfect even with 'exhaustive' because at natural footages both 1 and 2 frames/blocks for motion search are damaged with random noise and simple block-tesselation of the frames can not give best results. Also the tesselation to blocks also creates some additional digital noise/distortion. Somehow better is 'area' next level of abstraction over simple block-based search when we run several shifted block searches in some area around current origin of the block (with radius up to blocksize/2) and make some statistical averaging of the motion vectors found in each search run. This adds an additional level of computing complexity many times.

Most (many) blocks/areas in many footage are not moving at all or moved by large areas with the same speed. So many computing resources for motion search of each block are wasted and completely useless. It may be better to design much more advanced 'adaptive' methods of all motion search part of MPEG encoder parameters control for each cutscene or at least footage types (fast sports or slow motion nature etc). Simply setting best/exhaustive/placebo may be too slow.

[Barough]

x265 v4.1+200-ffba52b
Built on October 25, 2025, GCC 15.2.0
Win32/64 / 8bit+10bit+12bit

DL :
https://www.mediafire.com/file/6jsdrsany41o6jk

[benwaggoner]

Also currently used motion search methods may be still far for being perfect even with 'exhaustive' because at natural footages both 1 and 2 frames/blocks for motion search are damaged with random noise and simple block-tesselation of the frames can not give best results. Also the tesselation to blocks also creates some additional digital noise/distortion. Somehow better is 'area' next level of abstraction over simple block-based search when we run several shifted block searches in some area around current origin of the block (with radius up to blocksize/2) and make some statistical averaging of the motion vectors found in each search run. This adds an additional level of computing complexity many times.
Noise is an interesting case. One (for the most part) benefit of performing coarse motion search at lower resolutions is that the low-pass filtering of the downscale averages out a lot of random noise. Of course, this also means that fine grain may only be analyzed at the final full resolution, throwing off rate control and other estimations a ton.

[benwaggoner]

x265 is based on x264. Not will be.
Quite some drastic devalopments on the HEVC tools, but the fundamental design, especially the psychovisual optimizations, are very much like x264.
I'd say that x265 includes pretty much all of x264's psychovisual optimizations and conceptual foundation. But it also includes features not in x264, most of which aren't applicable.

Thus CRP and --psy are foundational, but x265 also has --psy-rdoq which doesn't have a x264 equivalent. HEVC has SAO with its own set of psychovisual tradeoffs. Etcetera.

To my way of thinking, flash protection is "unrelated" to this issue. Not going thorough flash protection + the default value of bScenecut + the return false branch is the problem.[/QUOTE]

[rwill]

Thus CRP and --psy are foundational, but x265 also has --psy-rdoq which doesn't have a x264 equivalent.

--psy-rdoq is psy-trellis in x264. Nothing new there.

[DTL]

Noise is an interesting case. One (for the most part) benefit of performing coarse motion search at lower resolutions is that the low-pass filtering of the downscale averages out a lot of random noise. Of course, this also means that fine grain may only be analyzed at the final full resolution, throwing off rate control and other estimations a ton.

mvtools also uses hierarchical search of many 2x downsized levels. But this not helps a lot. The downsized blocks also lost significant original image details and probability of bad intermediate decision at some level of search increases and it can not be corrected at finer levels of search. Real best search is simple exhaustive search with *very large* radius better over total frame. And it is very demanding for compute resources.

If we have some low contrast patch damaged by comparable level of noise with details of the patch - most of motion estimation methods return lots of random motion vectors (even in the case of no motion at all at the large areas).

If we expect from MPEG encoder work as moving image encoder with sort of noise-removing tool (encode only moving image and not noise) - we need very advanced motion estimation engine in it. Increasing of motion search radius is not complete solution and only small step to the better future.

[benwaggoner]

If we expect from MPEG encoder work as moving image encoder with sort of noise-removing tool (encode only moving image and not noise) - we need very advanced motion estimation engine in it. Increasing of motion search radius is not complete solution and only small step to the better future.
I think the future is to de-noise and parameterize the source, encode as clean, and then use some kind of FGS to reconstruct the noise/grain.

Only AV1/AV2 are making FGS mandatory for decoders, though.

[Z2697]

Ever heard of bugs that only occur by chance when there's exactly 8 threads (logical processors) available to a process?
I've seen run to run inconsistency with x265 in such setups, unfortunately I don't have an actual 4C8T CPU that's readily able to run the test.
I have test with "8 core" VM (running Windows 11) and the CPU count limit in Windows 10 boot setting (it's a 8C16T laptop but I see no way to turn off SMT).
Encode same Y4M with just the default settings, and occasionally there will be a result file that's not the same with others.

I tried "8 core" VPS from Vultr on both AMD and Intel CPUs, and Arch Linux, same story...

[hellgauss]

My old laptop is 4 Cores - 8 threads. It is a 4700 MQ Haswell.

I'll do some tests, just a few questions:

- What do you mean with "occasionally"? Once each ... encodings ? How long is the y4m?

- If you can share a free source-sample and link a build for which you can reproduce the issue I can reproduct the same test and I can send CRC of my output(s). Also please provide the exact command line which cause the issue.

Note: I have Windows 10 64 bit on that laptop, and I cannot install W11. Also I have only 8Gb ram installed.

[Z2697]

The chance is pretty random, I have my test script run 64 encodings each batch, sometimes there's multiple "hits" per batch, sometimes 0.
So maybe it's 1 in 100 odds overall, but I don't keep track of all the runs...

The executable and OS doesn't matter (much), I have tested with Arch Linux and x265 from its packages.
But here's the executable I use in my VM:
https://pixeldrain.com/u/qqFXajdp

The test clips are some anime openings decoded to Y4M.
I have tested multiple clips that can reproduce the bug, but I assume the ones that are too "tranquil" are less likely to show up.
The type of the content shouldn't matter, I just found that anime openings conveniently have the length I like, and quite some motion in it.

The command line is just "x265 a.y4m {i}a.265", the absolute simplest.

[hellgauss]

The link does not work for me.

I have performed this test, with full gyan ffmpeg release 8.0.1 (20 Nov 2025)

for /L %%i in (0,1,30) do (
ffmpeg -y -bitexact -i "file.mp4" -c:v libx265 -f hevc -bitexact "output%%i.hevc"
)

for %%i in (*.hevc) do (
certutil -hashfile "%%i" MD5 >> testhash.txt
)


In 31 files, 29 are equals. Two are different, and also different between them. Surprisingly, the different ones are about 0.1% smaller.

Source is a 23 min full hd x264.

Here the .zip reports of my hevcqp tool (Macroblocks quant quant analyzer in txt, detailed stats in .dat) and mediainfo of source and outputs. The "standard" output is output0.

The .dat files contains
Line1=filename
Line2=codec
Line3= [MacrBlocks (MB) per frame] [Frame Number]
Then for each frame:
[Frametype] [Frame size] [Frame byte-start] [AVG QP] [STDEV QP of MB] [MIN QP of MB] [MAX QP of MB]

https://limewire.com/d/wGMIX#KqmgBy69cp

PS: If i had to bet, I bet it is wpp fault. Do not take this statement seriously, just a bet :)

[jpsdr]

@Z2697
BTW, you didn't describe what was the bug, or did i miss something ?

[Z2697]

The bug appears as run to run inconsistency, but I can't find the root cause.
Yes there're multiple other things that make encoding non-deterministic, and this is one of them.
I just feel it's very weird that it only happens at 8 threads and with "no good reason".

Turning WPP off seems to make the bug disappear, I need to run the thing more times.

[hellgauss]

Eheh I always encode single thread because I enjoy determinism. Multithreading is also a source of bugs, since it is very difficult to sync them (I'm not a programmer, I only know that there are a lot of complicated stuff related). Since I experienced no issues while tuning threads in x264, I guessed that the source should have been another one, and wpp is the other parameter related to MT.

A fast question on a somewhat related issue: since I use 1 threads, I usually perform parallel encodings. I recently started to try the -mcstf option. On clean source it seems that it has no effect on the output, except a tiny saving on space (it is OK, I know that it is for noisy source).

However I experienced a *HUGE* amount of ram used, which is a problem with parallel encoding. On 720p 4:2:0 10bit it takes 6.3GB of ram, which stabilyze at about 3GB after a few minutes. Without mcstf it is about 1.4GB. Is there any info on ram usage with mcstf?

Note: I'll do some tests changing rc-lookahead and bf. At this moment I use rc=99 and bf=16, with badapt 2.

[benwaggoner]

Eheh I always encode single thread because I enjoy determinism. Multithreading is also a source of bugs, since it is very difficult to sync them (I'm not a programmer, I only know that there are a lot of complicated stuff related). Since I experienced no issues while tuning threads in x264, I guessed that the source should have been another one, and wpp is the other parameter related to MT.
Do you mean --frame-threads 1 or full on --pools 1,0 or something. I use single frame threads by default as well, but leave on wpp and other forms of multithreading. At high resolutions you can still get a lot of CPU threads working at once.

However I experienced a *HUGE* amount of ram used, which is a problem with parallel encoding. On 720p 4:2:0 10bit it takes 6.3GB of ram, which stabilyze at about 3GB after a few minutes. Without mcstf it is about 1.4GB. Is there any info on ram usage with mcstf?
This is somewhat non-intuitive, and could reflect a bug. Perhaps it's caching full source and filtered frames in RAM for all of lookahead.

Note: I'll do some tests changing rc-lookahead and bf. At this moment I use rc=99 and bf=16, with badapt 2.
What's your --keyint? --rc-lookahead can go up to that, but you get diminishing returns. 16 b-frames is an extremely placebo option except for really low bitrate or really clean motion graphics stuff. And even then it's a <5% improvement. Look at the actual b-frame percentages from your encodes; it's not really worth keeping a b-frame count higher than you're seeing your 1% longest usage.

[hellgauss]

Those are parameters I'm using in one of my ongoing encoding via ffmpeg animmouse build. My target is anime, (quite) low bitrate and I'm not interested in encoding time, so I usually go to placebo. I also target low or adjustable CPU usage (I encode either on a fanless PC or on a PC which I also use for work). The memory issue for mcstf seems to be also in official builds. I know I can rise rc but 99 is quite ok for me.


-profile:v main10 -vf "removegrain=1:2:2,format=yuv420p10le,scale=1280:720:flags=bicublin:param0=0.25:param1=0.375" -sws_flags accurate_rnd -c:v libx265 -preset veryslow -crf 18.9 -x265-params "psy-rd=0.5:psy-rdoq=0.7:aq-mode=2:aq-strength=0.6:deblock=1,1:nr-intra=9:nr-inter=19:tu-intra-depth=4:tu-inter-depth=4:merange=58:ref=6:rc-lookahead=99:bframes=16:rd=6:rd-refine=1:subme=7:rskip=0:cbqpoffs=1:crqpoffs=1:scenecut=42:scenecut-bias=5.2:qcomp=0.58:tskip=1:tskip-fast=0:limit-sao=1:selective-sao=2:vbv-maxrate=7500:vbv-bufsize=10615:frame-threads=1:pools=none:no-wpp=1:mcstf=1:level-idc=40:high-tier=1:colorprim=1:colormatrix=1:transfer=1:range=limited"

[Z2697]

LOL mcstf
You'll regret

I'm not so sure about the little adjustments you made to some parameters, but judging from the fact that you think mcstf is ok, you are just doing that for the false feeling of "safety".

[hellgauss]

@Z2697
Could you please elaborate more? Which side effect can have mcstf, other than memory usage? I read that it is very useful on noisy source, but I only performed a few tests on clean one. Frames type choose is equal, size is 0.02% less, quantizer is (almost) identical. Visually it seems ok, but I did not check on big screen.

I also read that it is applied on I/P frames so perhaps it is not so useful with 16bf. However at least 5 bf are required to activate the option.

[microchip8]

6 to 8 bframes are an optimal value. Using 16 is worthless and placebo

Also, consider using aq-mode 1 as 2 has issues with smearing due to the fact x265 devs decided to change its values from x264 and this is what you get.

Not sure why you have increased deblock? Do you get blocking at default values of 0? If not, set it to default. The stronger deblocking is, the more softer image it'll produce

Also, increasing the qp offsets actually makes the encoder compress the blue & red chroma channels more strongly, resulting in less vivid colors.

[GeoffreyA]

Those are parameters I'm using in one of my ongoing encoding via ffmpeg animmouse build. My target is anime, (quite) low bitrate and I'm not interested in encoding time, so I usually go to placebo. I also target low or adjustable CPU usage (I encode either on a fanless PC or on a PC which I also use for work). The memory issue for mcstf seems to be also in official builds. I know I can rise rc but 99 is quite ok for me.

How is the removegrain filter, which I see in your command line? I've been having trouble with the original Blade Runner, but can't use VapourSynth because of libplacebo issues, so options are limited. --nr-inter works but I can see the difference at low settings for this film.

[hellgauss]

@GeoffreyA
It is very nice. It is very useful to reduce bitrate without hurting too much, especially (but not only) with grain. I use it quite as a default. Before the resize is less effective and more soft. I suggest only mode 1 (very soft) or 2 (soft), eventually mode 2 can be applied twice. That command apply mode 1 to luma and 2 to chromas. It actually helps to keep the grain because allows you to reduce crf.

Remarks:
- when used with ffmpeg, it automatically convert the source to 8 bit. So use it before colorspace conversion. If you need it after the resize, I suggest avisynth (I never used vapoursynth, I do not know about it), since resize is better at 10bit.
- It can damage the dithering pattern in HQ source if present.
- Please note that the above CLI is for quite clean and not HQ anime source (WEB), and for low bitrate target. I used noise reduction mainly to lower the bitrate.

[GeoffreyA]

@GeoffreyA
It is very nice. It is very useful to reduce bitrate without hurting too much, especially (but not only) with grain. I use it quite as a default. Before the resize is less effective and more soft. I suggest only mode 1 (very soft) or 2 (soft), eventually mode 2 can be applied twice. That command apply mode 1 to luma and 2 to chromas. It actually helps to keep the grain because allows you to reduce crf.

Remarks:
- when used with ffmpeg, it automatically convert the source to 8 bit. So use it before colorspace conversion. If you need it after the resize, I suggest avisynth (I never used vapoursynth, I do not know about it), since resize is better at 10bit.
- It can damage the dithering pattern in HQ source if present.
- Please note that the above CLI is for quite clean and not HQ anime source (WEB), and for low bitrate target. I used noise reduction mainly to lower the bitrate.

Thanks for the detailed description. I'll give it a go tomorrow. I've found that KNLMeansCL with toned-down parameters gives near-transparent results, while cutting down bitrate a fair bit. BM3D is seemingly on par, too, but impractically slow without CUDA.

[Z2697]

I was gonna write "mcstf causes ghosting" and then I thought "what would it be in single thread".
Well, turns out mcstf does not like pool features wpp and lookahead-slices. frame-threads are fine (but it only gets supported recently).

[LigH]

I created an issue in their bug tracker (https://bitbucket.org/multicoreware/x265_git/issues/1019/wpp-with-many-threads-and-mcstf-together).

It would be useful to explain better, because I still don't understand: Is there any visible artifact in the decoded video, or is it just invisible differences in the encoded video bitstream, causing only a different file but no obvious video artifacts?

[hellgauss]

No evident artifact for my 4C8T test.

Also I noticed an interesting thing. For both files the divergence seems to start at the same GOP near the end of the video (you can check in my .dat files with WinMerge file compare).

Differences persists in some of next GOPs. Of course the byte-start will be no longer recovered, but other fields sometimes are equal and sometimes not.

PS: your issue link does not work

[LigH]

For me it does. One might need to log in to access it? Or it requires a quite recent web browser generation with support for a modern JavaScript version (e.g. it does not work on SeaMonkey).

[Z2697]

The issue needs to be "reviewed" by mcw, before that the issue is not visible to the public...
I don't know the exact reason behind this, but there was a time that some spam message flooded the issue tracker.
Who would have expect, I mean really, spamming message in such place???

[Z2697]

For me it does. One might need to log in to access it? Or it requires a quite recent web browser generation with support for a modern JavaScript version (e.g. it does not work on SeaMonkey).

The issue is open now, but the two are not related.
I've created a issue earlier and also opened now, that only contains the "8 thread" bug.
https://bitbucket.org/multicoreware/x265_git/issues/1017/run-to-run-inconsistency-with-exactly-8

Not exactly duplicate so maybe you can edit your issue to only mention the MCSTF bug?
And it literally creates severe ghosting.

[LigH]

I created this issue because I misunderstood you, I believed you were not able to create one yourself...

You probably want post #9883 (https://forum.doom9.org/showthread.php?p=2025825#post2025825) reported.

[benwaggoner]

Also, increasing the qp offsets actually makes the encoder compress the blue & red chroma channels more strongly, resulting in less vivid colors.
Yeah, I've only ever seen positive chroma qp offsets helpful in Dolby Vision Profile 5 to compensate for Y'CtCp instead of Y'CbCr. Other than that, I've only seen negative values be helpful.

[hellgauss]

I used quite low crf for a "low" bitrate encoding i.e. 18.9. It reduce artifact and banding. See qpoffset as an option to increase luma quality than to lower chroma. I also degraded chroma more than luma in other ways (bicublin resize, removegrain 2 on chroma). Each option has a low/moderate effect on filesize, but "E' la somma che fa il totale (*)"

(*) "The sum make the total" cit. of italian actor Totò

[tormento]

On the very same CPU, what is the speed difference between x265 AVX only encoding and AVX2 one?

I have a server with a Haswell Xeon 1275v3 that, even if it has lower speed than my ancient i7-2600k, it supports AVX2 instruction set.

I think I could have a nice speed bump using it but before installing windows etc I'd like to see some numbers.

Thank you ;)

[Z2697]

On the very same CPU, what is the speed difference between x265 AVX only encoding and AVX2 one?

I have a server with a Haswell Xeon 1275v3 that, even if it has lower speed than my ancient i7-2600k, it supports AVX2 instruction set.

I think I could have a nice speed bump using it but before installing windows etc I'd like to see some numbers.

Thank you ;)

I'd say about 20%

[jpsdr]

For integer calcul, SSE2/AVX is the same, still only XMM registers. It's only on AVX2 there is YMM registers for integer calcul, so AVX vs AVX2 is probably the same than SSE2/AVX2. Meaning i also think the same than Z2697.

[GeoffreyA]

What is the usual way, in x265, of mitigating "microbanding" introduced by the encoder?

[Z2697]

What kind of "microbanding"?

[GeoffreyA]

Thin, watercolour-like streaks, appearing rarely, sometimes for a frame or two, on skin gradients. A lower CRF or CQP does eliminate it, so it is happening when less bits are available. 10-bit encoding seems to tone down but not eliminate it. I have been testing on one frame: not a good approach, I know, but this makes it easier to narrow down what is causing it. I noticed that SVT-AV1 (Blue's latest fork) is not afflicted with this artefact; but switching to that would be more problematic, owing to its rate-control issues.

Happy New Year, by the way. May everyone's encodes be artefact free and saturate all cores.

[Z2697]

I was thinking maybe you can upload an image.
But anyways, assuming you mean something like this:
https://files.catbox.moe/g8wbtc.webp

It's caused by intra prediction (especially on larger blocks that's more noticeable) and not enough bits to signal residual that's able to complete it.
AV1 does have the same problem (and even AV2 will) as long as they use the same/similar intra prediction method.
In fact, the example image is from an AV1 intra frame. ;)

Inter frames can have intra blocks as well.
The solution is just give it more bits, or I assume AV1 is doing some filtering to cover it. (or they are in different places you don't notice)

[GeoffreyA]

Thanks, Z2697. Yes, giving more bits does help. I was going through each common setting, trying to see if one was the culprit.

https://slow.pics/s/FqMnaMTo

-c:v libx265 -preset veryslow -crf xy -x265-params no-sao=1:deblock=-1,-1:aq-mode=1

See the arm. psy-rd and psy-rdoq help to restore the detail but the "banding" is still present, albeit not so noticeably.

[Z2697]

It's just a "natural" result from the coding tools available in HEVC and how x265 devs implement them. (or how poorly, say SAO)
Whatever the parameters you tweak that may make it look better is just tossing the bits around. Unless you want to give SAO a try.

Intra prediction angle modes are like painting the neighboring pixel straightly in a direction.
AVC has smaller blocks and fewer intra angles, so it tends to look blurry instead of painted.
AV1 has more filters that, I assume [will cover up some of it and SVT-AV1 devs implemeted well].

[GeoffreyA]

Makes sense. Its somewhat different tools and ranges yield different-looking artefacts as a side effect, and x265's implementation of these tools may not be the gold standard. qcomp was helping, but as you said, just shifting around or pumping up the bits.

SVT-AV1 can give a better picture at the frame level, but going in the temporal direction, particularly with grain, we're walking on shaky ground. The HDR fork does better, but changes the appearance of the grain, making it coarser.

[benwaggoner]

Makes sense. Its somewhat different tools and ranges yield different-looking artefacts as a side effect, and x265's implementation of these tools may not be the gold standard. qcomp was helping, but as you said, just shifting around or pumping up the bits.

SVT-AV1 can give a better picture at the frame level, but going in the temporal direction, particularly with grain, we're walking on shaky ground. The HDR fork does better, but changes the appearance of the grain, making it coarser.
And, as always, with a video codec, how it looks temporally is really the only thing that matters. Making still images look good in isolation is a much simpler problem than getting a coherent, consistent experience over time. And good psychovisually compression efficiency can involve doing things that can make individual frames paused look not great, but which are invisible during actual playback. Temporal masking is a powerful tool.

[Z2697]

I found that x265's SAO performs better or less bad in 12bit mode some time ago... maybe worth take a look.
But I cannot really understand that part of the code, and since AV1 has better filter(s) and better open source implementation(s), I'm not very motivated :o

[GeoffreyA]

And, as always, with a video codec, how it looks temporally is really the only thing that matters. Making still images look good in isolation is a much simpler problem than getting a coherent, consistent experience over time. And good psychovisually compression efficiency can involve doing things that can make individual frames paused look not great, but which are invisible during actual playback. Temporal masking is a powerful tool.

Yes, we tend to fixate on comparing single frames, but at the end of the day, it's the video. x264/5 seem mostly fine in this regard, the lesser-quality frames feeling consistent.

I found that x265's SAO performs better or less bad in 12bit mode some time ago... maybe worth take a look.
But I cannot really understand that part of the code, and since AV1 has better filter(s) and better open source implementation(s), I'm not very motivated :o

Well, 12-bit might be challenging to play on some hardware, and I've made peace parting with SAO for life! :)
Jokes aside, I've found some success with aq-mode 4 these past few days. So far, it's working well.

[Z2697]

Well, 12-bit might be challenging to play on some hardware, and I've made peace parting with SAO for life! :)
Jokes aside, I've found some success with aq-mode 4 these past few days. So far, it's working well.

The problem is it shouldn't perform distinctively worse in 8 and 10 bits mode, hence there may be some bug in the code.
I mean the SAO mode decision, not the fps perform, you know.

[GeoffreyA]

The problem is it shouldn't perform distinctively worse in 8 and 10 bits mode, hence there may be some bug in the code.
I mean the SAO mode decision, not the fps perform, you know.

So, a precision sort of bug? Perhaps an overflow somewhere in the calculation?

[rwill]

Well you could give my gradient_with_noise sequence a try, with 10 and 12 bit, and check if its no longer blocking things up.

https://forum.doom9.net/showthread.php?p=1949376#post1949376

But I guess nothing did change. Last time I checked x265 had no real "mode decision" for SAO but only went by what it understands as "distortion". I remember debugging some large offenders and the numbers checked out, no bug there it seems.

[GeoffreyA]

Does x265's SAO use the same implementation as HM, the reference encoder?

[rwill]

Does x265's SAO use the same implementation as HM, the reference encoder?

Well I guess they did not come up with something novel on their own....

[Z2697]

Well you could give my gradient_with_noise sequence a try, with 10 and 12 bit, and check if its no longer blocking things up.

https://forum.doom9.net/showthread.php?p=1949376#post1949376

But I guess nothing did change. Last time I checked x265 had no real "mode decision" for SAO but only went by what it understands as "distortion". I remember debugging some large offenders and the numbers checked out, no bug there it seems.

Least "distortion" (and seems calculated in it's own way) is still some kind of mode decision... right?

I tested this same clip before, 12bit does not create same "blocky columns" as 10bit in the test, that's actually what I mean less bad.

But this kind of content should use "no operation" mode I think? 12bit still doesn't do that.

[rwill]

Least "distortion" (and seems calculated in it's own way) is still some kind of mode decision... right?

I tested this same clip before, 12bit does not create same "blocky columns" as 10bit in the test, that's actually what I mean less bad.

But this kind of content should use "no operation" mode I think? 12bit still doesn't do that.

Yes, picking a mode based on least distortion is a mode decision. I meant something like a full RD one with normal SSD and PSY distortion, like the CU mode decision.

That the clip does not exhibit the blocking at 12bit is interesting and probably worth investigating. Its strange that 8 and 10 bit pick the 'offset' mode and 12 bit does not. As for the optimal mode, it would be better if it were selected by real RD costs, that is, recon against source distortion with the (PSY-)RD distortion metric used everywere else.

[benwaggoner]

Does x265's SAO use the same implementation as HM, the reference encoder?
That was my understanding back in the day, and I do not recall that ever changing.

[hellgauss]

Perhaps related to RD, I found an unwanted behavior in rd-refine option. The following screen is from Vinland Saga Season 1 Episode 16, source is blu-ray @34MBps

https://imgur.com/iwfhscB

NOTE: the screen is taken via VirtualDub2 and the artifacts are present in software decoder, so probably it is not the same MV issue I described a few months ago in another thread.

The flickering fire leads to fast moving gradient in a dark background, which is somewhat a worst case scenario for video encoding. There are a lot of similar random macroblocks in the whole scene, each last for 1 frames.

Bat file used (Windows):

set bf=6
set subme=4

for %%i in (0,1) do (
for %%j in (10,12) do (

rem i=rd-refine j=bit

start /low ffmpeg -threads 1 -y -hide_banner -nostdin -stats_period 33 -bitexact -i "Vinland-Saga_S1E16.mkv" -map 0:v -ss 206.580 -t 17.977 -profile:v main%%j -vf "removegrain=1:2:2,removegrain=0:2:2,format=yuv420p%%jle" -sws_flags accurate_rnd -c:v libx265 -preset veryslow -crf 17.2 -x265-params "deblock=-1,-1:no-sao=1:no-strong-intra-smoothing=1:tu-intra-depth=4:tu-inter-depth=4:merange=58:ref=6:rc-lookahead=99:bframes=6:rd=6:rd-refine=%%i:subme=4:vbv-maxrate=20000:vbv-bufsize=24924:frame-threads=1:pools=none:no-wpp=1:level-idc=51:high-tier=1:colorprim=1:colormatrix=1:transfer=1:range=limited" -bitexact "Vinland-Saga_S1E16_r%%i_b%%j.mp4"
))


Writing library : x265 4.1+211-9e551a994:[Windows][GCC 15.2.0][64 bit]

If rd-refine=1, artifacts appears both in 10 and 12 bit. With rd-refine=0 there are no wrong macroblocks. However rd-refine seems to give nice results in other scenarios.

[Z2697]

That's a known issue, among some people.
https://github.com/Mr-Z-2697/x265-experimental/commit/69ea7c75e8c245e3323fda336f2a4df2c730ad7a

However my test result didn't favor rd-refine even if it's supposed to be fixed (if I'm lucky).
However yet again I didn't test the ultimate single thread encoding like you do.

I'll port that branch into a latest x265 and compile a test build for anyone that might be interested.

https://pixeldrain.com/u/z9TjFxsh
or
https://workupload.com/file/kcWyHpU2rX3

[hellgauss]

An update: it seems that the rd-refine problem is also related to the limit-tu option.

Test script used (all encodes with rd-refine=1, loop through limit-tu=%%j)

set rdr=1
set bf=5
set subme=3

for %%j in (0,1,2,3,4) do (

rem j=limit-tu

start /low ffmpeg -threads 1 -y -hide_banner -nostdin -stats_period 33 -bitexact -i "Vinland-Saga_S1E16-rdr-scene.mkv" -map 0:v -profile:v main10 -vf "removegrain=1:2:2,removegrain=0:2:2,format=yuv420p10le" -sws_flags accurate_rnd -c:v libx265 -preset slower -crf 17.2 -x265-params "limit-tu=%%j:deblock=-1,-1:no-sao=1:no-strong-intra-smoothing=1:rc-lookahead=99:bframes=%bf%:rd=6:rd-refine=%rdr%:subme=%subme%:vbv-maxrate=20000:vbv-bufsize=24924:frame-threads=1:pools=none:no-wpp=1:level-idc=51:high-tier=1:colorprim=1:colormatrix=1:transfer=1:range=limited" -bitexact "Vinland-Saga_S1E16_[rdr_%rdr%][ltu_%%j].mp4"
)

Results:
Macroblocks are evident with limit-tu =0 and 1. Less evident with 2, absent with limit-tu=3 or 4. Replacing with rd-refine=0, all tests present no "macroblocks" artifacts.

EDIT: thanks Z2697 for your build, I'll give it a try as soon as I setup a proper framework (I'm used to ffmpeg and not familiar with x265.exe, but I guess it is not difficult)

[jpsdr]

@Z2697
I took a look on your experimental repo, and see in your experiments the remove of distorstion chroma only for 444. Did test/check and finaly found that it produces better results ?

[Z2697]

An update: it seems that the rd-refine problem is also related to the limit-tu option.

This bug is quite random, it may disappear here but can happen at other places.

[Z2697]

@Z2697
I took a look on your experimental repo, and see in your experiments the remove of distorstion chroma only for 444. Did test/check and finaly found that it produces better results ?

See more detail in https://forum.doom9.org/showthread.php?t=186790
In theory, and in the way I assess chroma quality, it's better.

[GeoffreyA]

It seems to be more in line with x264, undoing that commit.

[benwaggoner]

Gosh, I hadn't thought about --rd-refine in years. I never found any examples where it provided a material difference. Have people found it helpful in some situations?

[rwill]

Gosh, I hadn't thought about --rd-refine in years. I never found any examples where it provided a material difference. Have people found it helpful in some situations?

Its a small BD-Rate optimization at a somewhat moderate speed cost when tuning for PSNR. It catches the case were a CU is sometimes better coded with a different QP in quantization than the originaly chosen RD QP (lambda).

So mainly helpful if you want to really push the objective metric the encoder is optimizing for in RD, not really helpful otherwise.

That is, if the lambdas for QPs are tuned so one would get just a narrow normal distribution of delta QPs around the originaly chosen QP when using that refine, which hopefully should be the case.

[Z2697]

We have seen such description in x264's subme option. (somehow)
subme is one of the more puzzling options in x264 that does a mixture of things...
I didn't find out exactly what's under the hood of the "RD refinement" here and the connection with x265's rd-refine.
If we search the word "rd_refine" in x264 code we can only find "intra_rd_refine" function.

-m, --subme <integer> Subpixel motion estimation and mode decision [7]
- 0: fullpel only (not recommended)
- 1: SAD mode decision, one qpel iteration
- 2: SATD mode decision
- 3-5: Progressively more qpel
- 6: RD mode decision for I/P-frames
- 7: RD mode decision for all frames
- 8: RD refinement for I/P-frames
- 9: RD refinement for all frames
- 10: QP-RD - requires trellis=2, aq-mode>0
- 11: Full RD: disable all early terminations

[hellgauss]

@Z2697

I tested your build in the same scene, with same results. Square artifacts with rd-refine=1 and limit-tu=0, 1. With 2 they are less evident. With 3 and 4 no square artifacts.

Perhaps they are in general less evident with respect to the official build.

With rd-refine=0 no artifacts whatever limit-tu.

[Z2697]

Well because I have changed the option.
1 is indeed the same.

H0(" --rd-refine <0|1|2|3> Enable QP-based RD refinement for rd levels 5 and 6.\n"
" - 0: disabled.\n"
" - 1: original behavior.\n"
" - 2: proposed fix.\n"
" - 3: try not to code skip CU.\n"
" Default %d\n", param->rdRefine);

[hellgauss]

Same results again with rd-refine 2 or 3 - blocks evident with limit-tu=0, 1 , less evident with limit-tu=2, no blocks with limit-tu=3, 4. It really seems that it is related to limit-tu also.

[Z2697]

Then it's beyond fix. (more like skill issues, dumb dumb Z-guy)
Not that it really matters though, it was a nothing burger (or maybe a positron burger) to begin with.

[Z2697]

We have seen such description in x264's subme option. (somehow)
subme is one of the more puzzling options in x264 that does a mixture of things...
I didn't find out exactly what's under the hood of the "RD refinement" here and the connection with x265's rd-refine.
If we search the word "rd_refine" in x264 code we can only find "intra_rd_refine" function.

-m, --subme <integer> Subpixel motion estimation and mode decision [7]
- 0: fullpel only (not recommended)
- 1: SAD mode decision, one qpel iteration
- 2: SATD mode decision
- 3-5: Progressively more qpel
- 6: RD mode decision for I/P-frames
- 7: RD mode decision for all frames
- 8: RD refinement for I/P-frames
- 9: RD refinement for all frames
- 10: QP-RD - requires trellis=2, aq-mode>0
- 11: Full RD: disable all early terminations


It looks like the "RD refinement" in x264 refers to refining the motion estimation, where x265 rd-refine... is just "FAFO" with different QP for each block...
(I might be wrong)

[Barough]

x265 v4.1+222-afa0028
https://www.mediafire.com/file/wqtdn8iwhgby53m
Win x86/x64 binaries, GCC

[benwaggoner]

Its a small BD-Rate optimization at a somewhat moderate speed cost when tuning for PSNR. It catches the case were a CU is sometimes better coded with a different QP in quantization than the originaly chosen RD QP (lambda).

So mainly helpful if you want to really push the objective metric the encoder is optimizing for in RD, not really helpful otherwise.
Yeah, and I don't care about objective metric improvements at all, only subjective ones. Objective metrics are a very useful tool in defining subjectively-tuned encodes, but are never the goal in and of themselves.

[benwaggoner]

We have seen such description in x264's subme option. (somehow)
subme is one of the more puzzling options in x264 that does a mixture of things...
One of my favorite felicities of x265 was unpacking that mess of options and just making things that should be independent parameters into those.

[nhw_pulsar]

Yeah, and I don't care about objective metric improvements at all, only subjective ones. Objective metrics are a very useful tool in defining subjectively-tuned encodes, but are never the goal in and of themselves.

Yes, I always thought that visual subjective improvement was a lot more important than objective metrics improvement, as in my codec very very often PSNR was worse and visual quality was better, and reversely PSNR was better but visual quality was worse. But on the opposite, MPEG develops for PSNR and surprisingly (for me) it seems to work, as I have tested for image compression, and visual quality follows PSNR: JPEG XR<x264 intra<x265 intra<VVC intra<ECM intra, so always for me, MPEG seems to improve PSNR and in the same time subjective visual quality....

[benwaggoner]

Yes, I always thought that visual subjective improvement was a lot more important than objective metrics improvement, as in my codec very very often PSNR was worse and visual quality was better, and reversely PSNR was better but visual quality was worse. But on the opposite, MPEG develops for PSNR and surprisingly (for me) it seems to work, as I have tested for image compression, and visual quality follows PSNR: JPEG XR<x264 intra<x265 intra<VVC intra<ECM intra, so always for me, MPEG seems to improve PSNR and in the same time subjective visual quality....
This is a very interesting philosophical discussion.

PSNR is a simple metric, so easy to calculate in a consistent way, and being able to make pixels mathematically similar to how they started is a decent first order approximation for quality. And a whole lot of bitstream features are about how to get that kind of mathematical match with as few bits as possible. I think of PSNR as like the individual shard of glass that make up a mosaic.

Now, past that low level of "is this number like it was before" comes subjective optimization. And a codec/bitstream standard needs to have tools that allow for an encoder to do psychovisual optimizations. The psychovisually preferable in-loop deblocking might not be the one that gives the best PSNR. But that's all outside of the normative codec spec. Actual encoding video that looks good and is deliverable within particular constraints is like the artist making the mosaic. Even starting with the same pieces of glass, a great artist can make a much better looking mosaic than a bad one, or some algorithm that scans a picture and assigned a piece of glass to each square. A less mathematically accurate mosaic can still look like the thing it is of a lot more accurately and esthetically.

[nhw_pulsar]

This is a very interesting philosophical discussion.

PSNR is a simple metric, so easy to calculate in a consistent way, and being able to make pixels mathematically similar to how they started is a decent first order approximation for quality. And a whole lot of bitstream features are about how to get that kind of mathematical match with as few bits as possible. I think of PSNR as like the individual shard of glass that make up a mosaic.

Now, past that low level of "is this number like it was before" comes subjective optimization. And a codec/bitstream standard needs to have tools that allow for an encoder to do psychovisual optimizations. The psychovisually preferable in-loop deblocking might not be the one that gives the best PSNR. But that's all outside of the normative codec spec. Actual encoding video that looks good and is deliverable within particular constraints is like the artist making the mosaic. Even starting with the same pieces of glass, a great artist can make a much better looking mosaic than a bad one, or some algorithm that scans a picture and assigned a piece of glass to each square. A less mathematically accurate mosaic can still look like the thing it is of a lot more accurately and esthetically.

Yes, I am not an expert like you, but as you say, it seems that standard bodies like MPEG release a new standard test model with a generational PSNR leap, and then you have to psychovisually tune this base model, independently of PSNR.

This is a unique way of development, proper to MPEG, in my little experience, my codec (NHW) since 2007 has not improved in PSNR, but visual quality has a lot improved...

Cheers,
Raphael

[benwaggoner]

Yes, I am not an expert like you, but as you say, it seems that standard bodies like MPEG release a new standard test model with a generational PSNR leap, and then you have to psychovisually tune this base model, independently of PSNR.
I concur it seems there should be a way to do a psychovisual-forward model of development. And subjective video tests are integrated earlier and earlier in each generation of codec development, and a lot more thought goes into making good syntax for those kinds of optimizations. Both MPEG-4 ASP and VC-1 suffered from inefficient signaling in adaptive quantization, for example, which was a big challenge in psychovisual optimization. Even though the baseline numbers published focus on PSNR, other testing gets done.

One potential downside of this is that non-optiional bitstream features that would improve psychovisual quality while not improving or even reducing PSNR will tend to get ignored. But features that have a bigger psychovisual improvement than PSNR improvement can get more attention than the PSNR improvement alone would merit.

This is a unique way of development, proper to MPEG, in my little experience, my codec (NHW) since 2007 has not improved in PSNR, but visual quality has a lot improved..
Yep, that is psychovisual optimization.

And it's not just MPEG. AOM has done AV1/2 development with a similar model, JXL. Really, all codec development for decades has had PSNR at the core of it.

[nhw_pulsar]

I concur it seems there should be a way to do a psychovisual-forward model of development. And subjective video tests are integrated earlier and earlier in each generation of codec development, and a lot more thought goes into making good syntax for those kinds of optimizations. Both MPEG-4 ASP and VC-1 suffered from inefficient signaling in adaptive quantization, for example, which was a big challenge in psychovisual optimization. Even though the baseline numbers published focus on PSNR, other testing gets done.

One potential downside of this is that non-optiional bitstream features that would improve psychovisual quality while not improving or even reducing PSNR will tend to get ignored. But features that have a bigger psychovisual improvement than PSNR improvement can get more attention than the PSNR improvement alone would merit.


Yep, that is psychovisual optimization.

And it's not just MPEG. AOM has done AV1/2 development with a similar model, JXL. Really, all codec development for decades has had PSNR at the core of it.

Ok, just a very quick off-topic, with my non-expert criterias and eyes, I always found that MPEG and AOM common technology (DCT+inter/intra prediction) lacked of neatness, at least on still-image...

Very curious to see if the next AV2 and ECM will correct this, despite a colossal computation time, and especially at decoding...

[Z2697]

Yes, I always thought that visual subjective improvement was a lot more important than objective metrics improvement, as in my codec very very often PSNR was worse and visual quality was better, and reversely PSNR was better but visual quality was worse. But on the opposite, MPEG develops for PSNR and surprisingly (for me) it seems to work, as I have tested for image compression, and visual quality follows PSNR: JPEG XR<x264 intra<x265 intra<VVC intra<ECM intra, so always for me, MPEG seems to improve PSNR and in the same time subjective visual quality....

They are different generations of codec after all.
How about you compare options within one codec?

[nhw_pulsar]

They are different generations of codec after all.
How about you compare options within one codec?


Actually I did not change/vary options within one codec for still image, I just used the default options (but always YUV420 if I remember correctly)...

That is right for JPEG XR, for x264 intra I used UCI codec default options, for x265 intra I used BPG default options (but often -m 1 fastest speed that has also very good quality), for VVC intra I used Jamaika binaries (VTM 12.3) provided on this forum with default command (slowest preset).

That's right I must admit I did not vary the options to have the best psychovisual results... but on the custom binaries for still image I have, I also don't think there are all the options as in the video codec...

[Z2697]

Actually I did not change/vary options within one codec for still image, I just used the default options (but always YUV420 if I remember correctly)...

That is right for JPEG XR, for x264 intra I used UCI codec default options, for x265 intra I used BPG default options (but often -m 1 fastest speed that has also very good quality), for VVC intra I used Jamaika binaries (VTM 12.3) provided on this forum with default command (slowest preset).

That's right I must admit I did not vary the options to have the best psychovisual results... but on the custom binaries for still image I have, I also don't think there are all the options as in the video codec...

The problem is the more tools available in later codecs will improve quality regardless of how the encoder is tuned (hopefully). Especially for Intra frame.
It's not PSNR works for them, it's the tools that works.
Maybe they are chosen based on PSNR merit, but doesn't stop them from acting as a visual improvement.

[benwaggoner]

Ok, just a very quick off-topic, with my non-expert criterias and eyes, I always found that MPEG and AOM common technology (DCT+inter/intra prediction) lacked of neatness, at least on still-image...

Very curious to see if the next AV2 and ECM will correct this, despite a colossal computation time, and especially at decoding...
Bear in mind that still images aren't the main goal of a video codec. Moving image and thus interframe compression is the essential features. All sorts of interesting transforms work well for still image, like wavelets. But no one has come up with a better way to handle interprediction than block based transforms. And block-based interprediction works very well with discreet sinusoidal transformations. And if you're using those for motion, doing your intra coding that way is lower complexity and a lot easier to integrate.

No one has come up with any inter frame encoding scheme better than using power of 2 sized rectangles with a DCT-ish basic transform.

[nhw_pulsar]

Bear in mind that still images aren't the main goal of a video codec. Moving image and thus interframe compression is the essential features. All sorts of interesting transforms work well for still image, like wavelets. But no one has come up with a better way to handle interprediction than block based transforms. And block-based interprediction works very well with discreet sinusoidal transformations. And if you're using those for motion, doing your intra coding that way is lower complexity and a lot easier to integrate.

No one has come up with any inter frame encoding scheme better than using power of 2 sized rectangles with a DCT-ish basic transform.

Yes, it is however possible to use power of 2 sized rectangles with wavelet transform but I think the main problem will be blocking artifacts.I must admit that in my codec, my wavelet 5/3 and 9/7 transform design introduces some border blocking artifacts that I think, are more pronounced than MPEG and AOM DCT transform ones...

My non-expert thinking makes me suppose that we should apply then the wavelet transform/compression on the whole residual image of the inter coding, but experts seem nearly sure that it won't be as good as x265, AV1, VVC...

[Patman]

I found the discussion about the --rd-refine parameter of x265 and its effects here quite interesting and took a look at the code myself. I have provided three test bins here (https://github.com/Patman86/x265-Mod-by-Patman/issues/49#issuecomment-3848765443) that you are welcome to try out. Maybe there is a version that works quite well with your parameters.

[Z2697]

I found the discussion about the --rd-refine parameter of x265 and its effects here quite interesting and took a look at the code myself. I have provided three test bins here (https://github.com/Patman86/x265-Mod-by-Patman/issues/49#issuecomment-3848765443) that you are welcome to try out. Maybe there is a version that works quite well with your parameters.

The test1 and test2 produce same result (still with glitches), and test3 produces result as if the rd-refine were not enabled.

[Patman]

The test1 and test2 produce same result (still with glitches), and test3 produces result as if the rd-refine were not enabled.
Thank you for your feedback. I have uploaded new files, perhaps one of them will be (partially) successful. Same link as before...

[Z2697]

Thank you for your feedback. I have uploaded new files, perhaps one of them will be (partially) successful. Same link as before...

What did you change with the overhaul?

[Patman]

What did you change with the overhaul?

void Analysis::qprdRefine(const CUData& parentCTU, const CUGeom& cuGeom, int32_t qp, int32_t lqp)
{
- uint32_t depth = cuGeom.depth;
- ModeDepth& md = m_modeDepth[depth];
- md.bestMode = NULL;
+ uint32_t depth = cuGeom.depth;
+ ModeDepth &md = m_modeDepth[depth];
+ md.bestMode = NULL;

bool bDecidedDepth = parentCTU.m_cuDepth[cuGeom.absPartIdx] == depth;

- int bestCUQP = qp;
- int lambdaQP = lqp;
+ int bestCUQP = qp;
+ int lambdaQP = lqp;
+
bool doQPRefine = (bDecidedDepth && depth <= m_slice->m_pps->maxCuDQPDepth) || (!bDecidedDepth && depth == m_slice->m_pps->maxCuDQPDepth);
+
if (m_param->analysisLoadReuseLevel >= 7)
doQPRefine = false;
- if (doQPRefine)
+
+ if (!doQPRefine)
+ {
+ recodeCU(parentCTU, cuGeom, bestCUQP, lambdaQP);
+ md.bestMode->cu.copyToPic(depth);
+ md.bestMode->reconYuv.copyToPicYuv(*m_frame->m_reconPic[0], parentCTU.m_cuAddr, cuGeom.absPartIdx);
+ return;
+ }
+
+ if (!md.bestMode || md.bestMode->cu.isSkipped(0) || cuGeom.log2CUSize <= 2)
{
- uint64_t bestCUCost, origCUCost, cuCost, cuPrevCost;
+ recodeCU(parentCTU, cuGeom, bestCUQP, lambdaQP);
+ md.bestMode->cu.copyToPic(depth);
+ md.bestMode->reconYuv.copyToPicYuv(*m_frame->m_reconPic[0], parentCTU.m_cuAddr, cuGeom.absPartIdx);
+ return;
+ }

- int cuIdx = (cuGeom.childOffset - 1) / 3;
- bestCUCost = origCUCost = cacheCost[cuIdx];
+ uint64_t bestCUCost, origCUCost, cuCost, cuPrevCost;
+
+ int cuIdx = (cuGeom.childOffset - 1) / 3;
+ bestCUCost = origCUCost = cacheCost[cuIdx];

- int direction = m_param->bOptCUDeltaQP ? 1 : 2;
+ int direction = m_param->bOptCUDeltaQP ? 1 : 1;

- for (int dir = direction; dir >= -direction; dir -= (direction * 2))
+ int maxDeltaQP = 2;
+
+ if ((m_limitTU & X265_TU_LIMIT_NEIGH) && cuGeom.log2CUSize >= 4)
+ maxDeltaQP = 1;
+
+ bool strongPsy = (m_param->psyRd > 0.5);
+
+ if (strongPsy && maxDeltaQP > 1)
+ maxDeltaQP = 1;
+
+ int minQP = X265_MAX(m_param->rc.qpMin, qp - maxDeltaQP);
+ int maxQP = X265_MIN(QP_MAX_SPEC, qp + maxDeltaQP);
+
+ int prevQP = (int)parentCTU.m_meanQP;
+ int dirToPrev = (prevQP > qp) ? +1 : (prevQP < qp ? -1 : 0);
+
+ for (int dirSign = 0; dirSign < 2; dirSign++)
+ {
+ int dir = (dirSign == 0) ? direction : -direction;
+
+ if (m_param->bOptCUDeltaQP && dir != 1)
+ continue;
+
+ if (m_param->bOptCUDeltaQP && dir == 1 && (qp + 3) >= prevQP)
+ break;
+
+ int baseThresh = (maxDeltaQP <= 1) ? 0 : 1;
+ int threshold = baseThresh;
+ if (dirToPrev != 0 && dir == dirToPrev && maxDeltaQP > 1)
+ threshold++;
+
+ int failure = 0;
+ cuPrevCost = origCUCost;
+
+ int modCUQP = qp + dir;
+ while (modCUQP >= minQP && modCUQP <= maxQP)
{
- if (m_param->bOptCUDeltaQP && ((dir != 1) || ((qp + 3) >= (int32_t)parentCTU.m_meanQP)))
+ if (m_param->bOptCUDeltaQP && modCUQP > prevQP)
break;

- int threshold = 1;
- int failure = 0;
- cuPrevCost = origCUCost;
+ recodeCU(parentCTU, cuGeom, modCUQP, qp);
+ cuCost = md.bestMode->rdCost;

- int modCUQP = qp + dir;
- while (modCUQP >= m_param->rc.qpMin && modCUQP <= QP_MAX_SPEC)
- {
- if (m_param->bOptCUDeltaQP && modCUQP > (int32_t)parentCTU.m_meanQP)
- break;
-
- recodeCU(parentCTU, cuGeom, modCUQP, qp);
- cuCost = md.bestMode->rdCost;
+ COPY2_IF_LT(bestCUCost, cuCost, bestCUQP, modCUQP);

- COPY2_IF_LT(bestCUCost, cuCost, bestCUQP, modCUQP);
- if (cuCost < cuPrevCost)
- failure = 0;
- else if (cuCost > cuPrevCost)
- failure++;
+ if (cuCost < cuPrevCost)
+ failure = 0;
+ else
+ failure++;

- if (failure > threshold)
- break;
+ if (failure > threshold)
+ break;

- cuPrevCost = cuCost;
- modCUQP += dir;
- }
+ cuPrevCost = cuCost;
+ modCUQP += dir;
}
- lambdaQP = bestCUQP;
}

- recodeCU(parentCTU, cuGeom, bestCUQP, lambdaQP);
+ lambdaQP = bestCUQP;
+ setLambdaFromQP(parentCTU, lambdaQP);

- /* Copy best data to encData CTU and recon */
+ recodeCU(parentCTU, cuGeom, bestCUQP, lambdaQP);
md.bestMode->cu.copyToPic(depth);
md.bestMode->reconYuv.copyToPicYuv(*m_frame->m_reconPic[0], parentCTU.m_cuAddr, cuGeom.absPartIdx);
}

[Z2697]

The most significant change being the QP adjustment range is limited to plus or minus 1 or 2, right?
Technically that would work, but then there's one less reason to use rd-refine.
Too narrow of a range, then the effect is reduced to virtually nothing, too wide of a range, then the potential of glitch is still present.

None of the rd-refine, original or with patch, improves BD rate in my test though. (except the one with less skip, but it's even slower)
Modifying QP on a per block basis without knowing basically anything else is not a good thing, it seems.

Limit-tu is not the problem, nor the fix.

[hellgauss]

Test results for second set of @Patman builds.

Script used. Test A (30 encodes)

set dir=patman
set bf=6
set subme=3

for %%b in (1,2,3) do (
for %%r in (rd-refine,no-rd-refine) do (
for %%j in (0,1,2,3,4) do (

rem b,r,j = build, rd-refine, limit-tu
rem build 1 = m_limit_tu
rem build 2 = overhaul
rem build 3 = overhaul + intrardrefine

ffmpeg.exe -loglevel level+error -i "Vinland-Saga_S1E16-rdr-scene.mkv" -map 0:v -vf "removegrain=1:2:2,removegrain=0:2:2,format=yuv420p10le" -sws_flags accurate_rnd -strict -1 -f yuv4mpegpipe - | %dir%\x265_%%b.exe --y4m -D 10 --input-res 1920x1080 --input-depth 10 --fps 24000/1001 --preset slower --crf 17.2 --%%r --limit-tu %%j --deblock=-1:-1 --no-sao --no-strong-intra-smoothing --rc-lookahead 99 --bframes %bf% --rd 6 --subme %subme% --vbv-maxrate 20000 --vbv-bufsize 24924 --frame-threads 1 --pools none --no-wpp --level-idc 51 --high-tier --colorprim 1 --colormatrix 1 --transfer 1 --range limited --input - --output output.hevc

mkvmerge.exe --deterministic 1 -o "Vinland-Saga_S1E16_ [build_%%b][%%r][ltu_%%j].mkv" -B -T -M output.hevc

)))

Test B (6 encodes, same as A but with preset medium and rd=6)
set dir=patman
set bf=6
set subme=3

for %%b in (1,2,3) do (
for %%r in (rd-refine,no-rd-refine) do (

rem b,r = build, rd-refine
rem build 1 = m_limit_tu
rem build 2 = overhaul
rem build 3 = overhaul + intrardrefine

ffmpeg.exe -loglevel level+error -i "Vinland-Saga_S1E16-rdr-scene.mkv" -map 0:v -vf "removegrain=1:2:2,removegrain=0:2:2,format=yuv420p10le" -sws_flags accurate_rnd -strict -1 -f yuv4mpegpipe - | %dir%\x265_%%b.exe --y4m -D 10 --input-res 1920x1080 --input-depth 10 --fps 24000/1001 --crf 17.2 --%%r --rd 6 --deblock=-1:-1 --no-sao --no-strong-intra-smoothing --rc-lookahead 99 --bframes %bf% --rd 6 --subme %subme% --vbv-maxrate 20000 --vbv-bufsize 24924 --frame-threads 1 --pools none --no-wpp --level-idc 51 --high-tier --colorprim 1 --colormatrix 1 --transfer 1 --range limited --input - --output output.hevc

mkvmerge.exe --deterministic 1 -o "Vinland-Saga_S1E16_ [build_%%b][%%r].mkv" -B -T -M output.hevc

))

In both tests, build 2 and 3 are the same, they give bit-equals output. If rd-refine=0 outputs do not depend on build.

--------------

Artifacts:
evident in test A, build 1, rd-refine and limit-tu=0 or 1, less evident with limit-tu=2. No artifact (or I did not notice them) in other tests.

[Z2697]

Is x265 SAO same as HM? This question was asked some where that I can't recall now.

I'd say it's based on HM but worse.
The structure does have the resemblance, but the core logic is different.
HM uses source pixels to calculate the real cost.
x265 use reconstructed pixels to estimate the cost: SAO NOP = 0 distortion + lambda * signal bits, and other SAO modes actually have "negative" distortion + lambda * some more bits.
So basically x265's SAO loses the "in-loop" attribute, and becomes some awkward game of balancing between 2 "random" things.
This could be an optmization choice... but at the cost of obliterating quality, is it really worth it? Is it effective? Because it still loops over all modes.
Perhaps I'll try bring real cost back and see how things will be.

[benwaggoner]

Is x265 SAO same as HM? This question was asked some where that I can't recall now.

I'd say it's based on HM but worse.
The structure does have the resemblance, but the core logic is different.
HM uses source pixels to calculate the real cost.
x265 use reconstructed pixels to estimate the cost: SAO NOP = 0 distortion + lambda * signal bits, and other SAO modes actually have "negative" distortion + lambda * some more bits.
So basically x265's SAO loses the "in-loop" attribute, and becomes some awkward game of balancing between 2 "random" things.
This could be an optmization choice... but at the cost of obliterating quality, is it really worth it? Is it effective? Because it still loops over all modes.
Perhaps I'll try bring real cost back and see how things will be.
Yeah, give the source and reconstructed frames should both be in L3 cache at that point, I wouldn't think the overhead would be that much. It might make sense as a quality/perf flag. Like:

--sao-mode-full (for proposed behavior)
--sao-mode-fast (for existing behavior)

[Z2697]

Uh, I was wrong, the recon (pre-SAO) to source distortion is stored in a variable during initilization.
So I guess it's on par with HM, which I haven't really used.
This precalculated dist won't tell how the post-SAO distortion is compared to the pre-SAO distortion, but HM and x265 SAO look similar if I squint my eyes, I count that as they are doing the same.
HM passes the source YUV to the init function in the "main structure", while x265 accesses source YUV directly in the init function, that's where I stumbled.
Which means both are not good?

But this SAO-specific distortion calculation does result in negative value A LOT, even like it's the majority.
I'm still under the impression that this is what causes it to be so bad.

I don't know what I'm talking about at this point, really.
Quit.

[benwaggoner]

Uh, I was wrong, the recon (pre-SAO) to source distortion is stored in a variable during initilization.
So I guess it's on par with HM, which I haven't really used.
This precalculated dist won't tell how the post-SAO distortion is compared to the pre-SAO distortion, but HM and x265 SAO look similar if I squint my eyes, I count that as they are doing the same.
HM passes the source YUV to the init function in the "main structure", while x265 accesses source YUV directly in the init function, that's where I stumbled.
Which means both are not good?
My recollection from, sheesh, 7-8 years ago is that x265's SAO is exactly HM's algorithmically. And that MCW hadn't been able to reproduce the SAO errors people were describing at the time, so didn't know how to address it.

But this SAO-specific distortion calculation does result in negative value A LOT, even like it's the majority.
I'm still under the impression that this is what causes it to be so bad.
That I can't speak about. As long as it's all using signed values and not clamping at zero, I don't know that negative would be a problem. Negatives are used all over in encoding, frequency transform on up.

[Z2697]

Because the "baseline" is always positive.
I could be wrong (again) though.

Determin offset:
int64_t bestCost = calcSaoRdoCost(0, 1, lambda);

Determin SAO type:

// RDO SAO_NA
m_entropyCoder.load(m_rdContexts.temp);
m_entropyCoder.resetBits();
m_entropyCoder.codeSaoType(0);
int64_t costPartBest = calcSaoRdoCost(0, m_entropyCoder.getNumberOfWrittenBits(), lambda[0]);

[_Shorty-dammit]

I've got a script that does a binary search encode job that attempts to reach a given VMAF score target. My tests with my couch distance and TV size indicates to me that VMAF 97 tends to be transparent for me in that situation. So I rip all my movies and re-encode with ffmpeg x265 -preset slow with CRF values changing according to the binary search, and a VMAF 97 target to save some space on the file server. The savings of which can be rather substantial in a lot of cases. Anyway, I just ran across a weird one that doesn't make any sense to me. Star Wars Episode II Attack Of The Clones is a 28.4 GB file off the Blu-Ray disc for just the video stream. For some reason, when I re-encode it with my script it ends up as a 30.0 GB file from the CRF 4.78 value that the binary search lands on to meet VMAF 97. I don't understand what the nature of the issue might be.

A Google AI search suggested it was because of the digital noise inherent to the Sony digital cameras used behaving like grain, only worse, but Once Upon A Time In Mexico used the same camera model and it behaves typically. I tried with x264 slow and it gave a file 8.42 GB in size from CRF 18.52 for ep2, which I'd call behaving typically. For some reason, x265 seems to be behaving unusually when it comes to ep2. I've tried x265 slow tune grain after the Google AI suspicion, and it lands on CRF 11.16 for a 23.7 GB video stream. That's a bit of an improvement in filesize, but still atypical. (Not to mention, isn't tune grain usually larger?) I also tried no-sao, and turning strong intra smoothing off, but neither seemed to have any drastic effect.

Checking in FFMetrics to also see PSNR and SSIM scores makes it seem like something's just weird with x265 and this title in general. I'm at a loss to explain it. Any guesses or any clues come to mind for anyone? I figured being digital end to end would make it more easily compressible than typical films due to the relative lack of noise, but nope. I don't know what the nature of it might be, but it seems like perhaps it has more noise than you might think. Whatever it is, there's something about that it puts it into outlier territory for x265. VMAF and SSIM are similar, but PSNR is ugly. I haven't tried the entire movie with tune psnr but testing with a 2-minute clip showed it would probably be a waste of time. The 2-minute clip is even worse than the entire movie, as no settings seem to be able to get it to stop the binary search from reaching CRF 0.0, heh.

VMAF
x264 slow: 97.0156
x265 slow: 97.0158
x265 slow grain: 97.0160

PSNR
x264 slow: 49.4331
x265 slow: 41.7823
x265 slow grain: 41.7037

SSIM
x264 slow: 0.9934
x265 slow: 0.9952
x265 slow grain: 0.9942

[rwill]

Try --tune psnr.

[GeoffreyA]

If the CRF falls too low, perhaps clamp the value to 14 or 17.

[_Shorty-dammit]

Try --tune psnr.

As I mentioned, I tried this already with the 2-minute clip and it still reaches CRF 0.0 like every other set of settings I've tried. I am now trying it for the sake of it with the whole movie, but that'll take the better part of the day to run.

If the CRF falls too low, perhaps clamp the value to 14 or 17.

Well, my goal isn't to simply have a smaller file, but one that also still meets my quality goal. In this case it seems I may as well just go with the x264 file, since it is less than 1/3 the original size and still meets that goal. But I'm still interested in learning what it is exactly about this particular file that kind of makes x265 freak out because it seems quite strange to me.

[hellgauss]

@_Shorty-dammit

Looks like a systematic error. It is probably something very simple, perhaps a wrong/unspecified colorspace on the disc or something went wrong e.g. with crop. What is the exact workflow you follow for the encoding? Does the x265 encoding looks good at crf 16/17?

Also check mediainfo of source and rip.

[GeoffreyA]

Well, my goal isn't to simply have a smaller file, but one that also still meets my quality goal. In this case it seems I may as well just go with the x264 file, since it is less than 1/3 the original size and still meets that goal. But I'm still interested in learning what it is exactly about this particular file that kind of makes x265 freak out because it seems quite strange to me.

VMAF might be misleading in this example.

[_Shorty-dammit]

@_Shorty-dammit

Looks like a systematic error. It is probably something very simple, perhaps a wrong/unspecified colorspace on the disc or something went wrong e.g. with crop. What is the exact workflow you follow for the encoding? Does the x265 encoding looks good at crf 16/17?

Also check mediainfo of source and rip.

Hmm...

T:\VMAF testing-single-file>ffprobe -v error -select_streams v:0 -show_entries stream=color_space,color_transfer,color_primaries,pix_fmt,width,height -of default=nw=1 starwarsclones-video-only.mkv
width=1920
height=1080
pix_fmt=yuv420p
color_space=unknown
color_transfer=unknown
color_primaries=unknown

Unknown. And my script treats it as bt.709, so you may be on the right track. While still waiting for the tune psnr run of the whole movie to complete I decided to try the 2-minute clip with it set to default to DVD colourspace/transfer/primaries instead, and it landed on 16.94 CRF, which would be a typical result in other cases. Interesting. So it would seem someone made a small error in mastering this movie. Good catch! Thanks! I think I'll let that tune psnr job finish just for the sake of it, and then I'll try it again with the DVD values on the whole movie. I suspect it'll return typical results that time.

VMAF might be misleading in this example.

I had considered this to be a VMAF issue at first, which led me to try x264 and av1 to see what happened there. They both seemed to behave in typical fashion for other movies, so that led me to believe it was something with x265 itself. But now that I've tried the small clip with DVD colourspace/transfer/primaries and it seems to have behaved in better fashion I'm wondering if someone just picked the wrong setting during the making of that Blu-Ray, as I just mentioned.

edit: Or perhaps it should be treated as bt.709 but because the original isn't tagged properly some part of ffmpeg's pipeline is treating it improperly?

[hellgauss]

Or perhaps it should be treated at bt.709 but because the original isn't tagged properly some part of ffmpeg's pipeline is treating it improperly?

The standard for bluray 1080/720p is bt.709. If it is not specified it should be assumed bt.709, so the tagging is correct. However some softwares or decoders (e.g. virtualdub2) do not honour this convention and if it is unspecified they convert to rgb using DVD standard.

Check the workflow for comparison too, I'm not expert on psnr and vmaf, but how they are calculated in your script? In my opinion you should compare the exact codec input with codec output, without colorspace conversion(s). If you apply pre-processing filters in the encode or post-processing filters in the comparison you should be very careful.

PS: I would also check chromaloc of source and rip.

[_Shorty-dammit]

The standard for bluray 1080/720p is bt.709. If it is not specified it should be assumed bt.709, so the tagging is correct. However some softwares or decoders (e.g. virtualdub2) do not honour this convention and if it is unspecified they convert to rgb using DVD standard.

Check the workflow for comparison too, I'm not expert on psnr and vmaf, but how they are calculated in your script? In my opinion you should compare the exact codec input with codec output, without colorspace conversion(s). If you apply pre-processing filters in the encode or post-processing filters in the comparison you should be very careful.

PS: I would also check chromaloc of source and rip.

Well, as I said, the tagging of the rip says unknown for colourspace/transfer/primaries. So it seems ffmpeg is making some assumptions that are detrimental in this case. When presented with a 1080p Blu-Ray rip it will usually use a command like this for encoding:

ffmpeg -i input.mkv -c:v libx265 -x265-params scenecut=40:vps-id=0:sps-id=0:pps-id=0:repeat-headers=1:colorprim=bt709:transfer=bt709:colormatrix=bt709
-preset slow -crf 25.5 -an -sn -y split/input.x265.CRF25.5.mkv

and a command like this for getting the VMAF score:

ffmpeg -fflags +genpts -i split/input.x265.CRF25.5.mkv -fflags +genpts -i input.mkv -map 0:v:0 -map 1:v:0 -an -sn -dn
-filter_complex [0:v]setpts=N/FRAME_RATE/TB,fps=<fps>,format=yuv420p[d];[1:v]setpts=N/FRAME_RATE/TB,fps=
<fps>,format=yuv420p[r];[d][r]libvmaf=model=version=vmaf_v0.6.1:n_threads=
<cpu_count>:log_fmt=xml:log_path=split/input.x265.CRF25.5.txt -f null NUL

I only checked PSNR and SSIM in this case with FFMetrics after the fact to try to figure out what was going on. I normally only look at VMAF. Now that I've forced the bt.709 tags onto the original file it seems to be processing the 2-minute clip correctly. So whatever ffmpeg is defaulting to with those bt.709 tags missing seems to be the cause. Manually adding the tags seems to have made it behave. Thanks again for the help, people!

[Z2697]

Have a understanding of what VMAF is and what it's not.

[Boulder]

I suggest using CVVDP instead of VMAF.

[benwaggoner]

I've got a script that does a binary search encode job that attempts to reach a given VMAF score target. My tests with my couch distance and TV size indicates to me that VMAF 97 tends to be transparent for me in that situation. So I rip all my movies and re-encode with ffmpeg x265 -preset slow with CRF values changing according to the binary search, and a VMAF 97 target to save some space on the file server. The savings of which can be rather substantial in a lot of cases. Anyway, I just ran across a weird one that doesn't make any sense to me. Star Wars Episode II Attack Of The Clones is a 28.4 GB file off the Blu-Ray disc for just the video stream. For some reason, when I re-encode it with my script it ends up as a 30.0 GB file from the CRF 4.78 value that the binary search lands on to meet VMAF 97.
Camera choice is just one factor in a long chain of choices and processes that change noise. Just how much light there was when it was shot reduces noise. And lots of post processing stuff can enhance, reduce, or even replace noise. How it gets color graded, especially with HDR, can make a big difference as well.

[benwaggoner]

As I mentioned, I tried this already with the 2-minute clip and it still reaches CRF 0.0 like every other set of settings I've tried. I am now trying it for the sake of it with the whole movie, but that'll take the better part of the day to run.
Of course, --tune psnr is a serious debuff for psychovisual tuning. Even if the VMAF scores get better, it will actually look worse at the same bitrate than just leaving --tune at default.



Well, my goal isn't to simply have a smaller file, but one that also still meets my quality goal. In this case it seems I may as well just go with the x264 file, since it is less than 1/3 the original size and still meets that goal. But I'm still interested in learning what it is exactly about this particular file that kind of makes x265 freak out because it seems quite strange to me.
You can't really say as long as as you're using VMAF instead of your eyeballs. VMAF is calibrated to within 3 points. A score of 96 can look worse than a 94 or might look identical to source. And the mean VMAF over an entire title can be misleading. A title where 95% of frames look perfect and 5% look terrible is a worse overall experience than consistent mediocrity. I find it is more useful to look at points where the VMAF drops to its lowest and see how those look.

[_Shorty-dammit]

Have a understanding of what VMAF is and what it's not.

You'll have to share more than that for me to understand what you're trying to say, so if you'd care to expand that would be great, thank you.

I suggest using CVVDP instead of VMAF.

I hadn't heard of this one before. I'll have to do some reading. Thanks!

You can't really say as long as as you're using VMAF instead of your eyeballs. VMAF is calibrated to within 3 points. A score of 96 can look worse than a 94 or might look identical to source. And the mean VMAF over an entire title can be misleading. A title where 95% of frames look perfect and 5% look terrible is a worse overall experience than consistent mediocrity. I find it is more useful to look at points where the VMAF drops to its lowest and see how those look.

Well, there's definitely something to be said for looking at it in that light. So far I haven't seen anything to complain about with just the mean of 97 for an entire movie. Previously, I had been splitting things up into GOP chunks and performing the same encode job per GOP, but this presented other problems I hadn't even thought about. For example, if you encode the first couple minutes of something like Toy Story in this fashion and look at the result, the GOP for the Pixar jumping lamp gets incredibly mangled. For some reason, perhaps due to the amount of pure black screen surrounding it, it turns into a big macroblock mess. The pure black sections before and after skew the VMAF scoring something awful, and the middle section that actually contains imagery is severely bit-starved. I think with my target of 97 that the CRF mechanism itself is likely doing a good enough job for any challenging sections, at least with all the titles I've processed so far, as I haven't seen anything to complain about.

[benwaggoner]

You'll have to share more than that for me to understand what you're trying to say, so if you'd care to expand that would be great, thank you.
VMAF is the least-bad broadly available metric, but it still has well less than .8 correlation to actual subjective quality ratings. It has a variety of blind spots where it doesn't detect quality issues, and it is sensitive to using the correct ML model for your scenario (SD/HD/UHD, SDR/HDR, whether you allow contrast enhancement to improve scores (Which you should not)) and the pipeline you use to compare pixels. It's a shortcut for when you need to get a ballpark estimate for more content than you can actually look at, but eyeballs are a lot more accurate. Especially your eyeballs if you're encoding stuff for personal use.

A couple of big VMAF blind spots are detection of blocking in gradients, adaptive quantization optimization in general, and anything to do with chroma; it's a luma only metric. The last can be a particularly big issue with HDR which can be quite finicky with chroma issues.

This is not to dump on VMAF. It was the first popular ML based metric and was a big improvement on SSIM, and a massive on on PSNR. But it is showing its age and all metric have blind spots.

I'm partial to full reference with bitstream analysis p1204.4, if you've got a well trained ML model for the codec and kind of content you are testing with.

[_Shorty-dammit]

VMAF is the least-bad broadly available metric, but it still has well less than .8 correlation to actual subjective quality ratings. It has a variety of blind spots where it doesn't detect quality issues, and it is sensitive to using the correct ML model for your scenario (SD/HD/UHD, SDR/HDR, whether you allow contrast enhancement to improve scores (Which you should not)) and the pipeline you use to compare pixels. It's a shortcut for when you need to get a ballpark estimate for more content than you can actually look at, but eyeballs are a lot more accurate. Especially your eyeballs if you're encoding stuff for personal use.

A couple of big VMAF blind spots are detection of blocking in gradients, adaptive quantization optimization in general, and anything to do with chroma; it's a luma only metric. The last can be a particularly big issue with HDR which can be quite finicky with chroma issues.

This is not to dump on VMAF. It was the first popular ML based metric and was a big improvement on SSIM, and a massive on on PSNR. But it is showing its age and all metric have blind spots.

I'm partial to full reference with bitstream analysis p1204.4, if you've got a well trained ML model for the codec and kind of content you are testing with.

Yeah, this is just for my home file server and my own viewing, so perhaps it is sufficient with the higher than usual score I'm targeting. VMAF 97 and CRF encoding may be enough to deal with a lot of troublesome examples, perhaps. Like I say, so far I've found it quite satisfactory. For someone else, or for professional settings, perhaps I'd think differently and take a different approach. But it's been basically transparent for me in my viewing scenario and has saved quite a bit of server space.

[_Shorty-dammit]

A little update. So, I manually added BT.709 tagging to the original AVC file with this:

ffmpeg -i starwarsclones-video-only.mkv -map 0 -c copy -color_primaries bt709 -color_trc bt709 -colorspace bt709 starwarsclones-video-only-bt709.mkv

and ran it through my script with again. This definitely smartened things up, and whatever portion of the pipeline was defaulting to something unhelpful in the absence of those tags would obviously then be doing what it should've been doing. It's now a 2080 Kbps video stream that's 2.07 GB in size, down from the original's 28595 Kbps and 28.4 GB. And the PSNR no longer looks goofy.

PSNR
x264 49.4331
x265 49.5123

SSIM
x264 0.9934
x265 0.9929

VMAF
x264 97.0156
x265 96.9946

That did the trick!

[tormento]

I'd like to encode video chunks in parallel to be joined later.

Don't want to have multiple "HEVC header" in the final stream, is possibile to strip the initial part that x265 writes at the beginning of every hevc elementary stream?

Any help is welcome.

[hellgauss]

Do you mean the info header with codec settings? I've written an utility for that:
https://forum.doom9.org/showthread.php?t=186066

---------------------------------------------------

I'd like to encode video chunks in parallel to be joined later.

This is a very interesting issue which I always wanted to accomplish. However it is not trivial... Perhaps I will open it in a separate thread.

[Z2697]

NAL units are conveniently separated, it's pretty easy.

But I suggest keep them for safety. Except the parameters UDU SEI.

[GeoffreyA]

Z2697,

I saw that you submitted a fix to x265 for aq-motion. What was going on there?

[Z2697]

Z2697,

I saw that you submitted a fix to x265 for aq-motion. What was going on there?

If you mean the one merged in the master branch (cfee9638 (https://bitbucket.org/multicoreware/x265_git/commits/cfee9638c82b655c5887cedbdf1aa856f81b906a)),
https://bitbucket.org/multicoreware/x265_git/issues/1001/poor-2-pass-performance-in-recent-41

In short, there's a bug that occurs when "--aq-motion --aq-mode 0 --no-cutree", and a commit meant to fix that bug messed up 2-pass (or more) CUTree, so I submitted that patch to fix the bug without messing up 2-pass.

* the bad commit is not in the release 4.1

[GeoffreyA]

I see. Thanks for the link and explanation.

[tormento]

Do you mean the info header with codec settings? I've written an utility for that
Thank you, will try it.
This is a very interesting issue which I always wanted to accomplish. However it is not trivial... Perhaps I will open it in a separate thread.
Actually, I use plain Trim() — when I have time I set cuts on scene changes — and launch multiple encodes.

[Hostile_18]

Just ordered this bad boy for $300. Looks great for x265 encoding.

https://i.ibb.co/ksLnhzyT/encode-h265.png (https://ibb.co/jPCKLq5Y)

[tormento]

Just ordered this bad boy for $300. Looks great for x265 encoding.
Problem, now, is memory price.

I will patiently wait for Nova Lake, hoping that the bubble will burst.

[Z2697]

Multiple headers isn't a problem at all. (when they are legit)
You should not worry about it.
But if you insist, you can use filter_units in FFmpeg.
filter_units=pass_types=0-31
And I don't know, awe at the few kilo bytes you saved?
The params SEI is easy enough to remove, I don't put them into count here.

You should be more worried about the drop in quality in starting I frame(s) ovserved in both x264 and x265.

[Hostile_18]

Problem, now, is memory price.

I will patiently wait for Nova Lake, hoping that the bubble will burst.

Yeah I feel that. My hard drives have gone from £320 to £580.

I was lucky with RAM as I bought DDR5 for my NAS a couple of months before everything went up. Luckily there are signs the AI demand will subside somewhat recently, so we shall see.

[OvejaNegra]

hi, how many cores / threads can i give to x265 before it starts loosing quality or efficiency? thanks

[LigH]

This is a gradual issue. The more threads, the smaller the scope. That means each thread might miss e.g. the furthest motion vectors in predicted frames ... but it is not very likely to find the perfect match in long distances anyway, and a good-enough match and a little difference is still smaller than an an intra block.

I, personally, would not recommend much more than 8 threads up to FullHD. More may still be suitable for UHD. But it's just my guess. Other people with more experience may have more specific values for you, so don't count on me alone. Furthermore, there are different kinds of threading strategies, e.g. WPP or Slices, with a different behaviour, so you may be able to tweak the risk of loss.

[OvejaNegra]

thanks, 8 th is what i'm using right now.

[Z2697]

When it comes to pools / frame-threads, the difference in quality is basically only "ON and OFF". (a few percent)

There are multiple "threading options" in x265.
1) pools.
WPP and lookahead-slices depend on this. WPP is on and off, and will not affected by the number of threads in pool. lookahead-slices is a difference but virtually indiscrenible.
Correction: There are 3 stages I think, 0, 1-3, and 4-, apart from that, no difference. But you don't usually wanna use 1-3 because it will often slower than 0.
Correction again: it's because frame-thread is being set to 1 automatically. Or so I think. It's complicated. :(
2) frame-threads.
As long as the number is greater than 1, the number doesn't matter.
3) slices.
Please AVOID. It's buggy when slices and frame-threads are both greater than 1.

If by "efficiency" you mean the multi-therad utilization, it's basically (frame height / ctu).

[LigH]

How long did we wait for this? Months, years? :eek:

Hello everyone,
We are pleased to announce the release of x265 v4.2. This version includes a new feature, optimizations, and critical bug fixes.
Please note that, starting with this release, we will be adopting a pure Trunk-Based Development (TBD) branching model. All development and release activities will be carried out on the master branch, which will serve as the single source of truth.

Version 4.2
===========

Release date - 19th April, 2026.

New feature
-----------
1. Threaded Motion Estimation (Experimental feature)-It uses a dedicated thread pool to precompute Motion Estimation in parallel. Improves encoding speed up to 1.5x for 1080p & lower resolution on multi core machines with low frequency setting. On high frequency systems or on machines with low number of cores, the overhead of additional Motion estimation work may outweigh parallelism

Enhancements to existing features
-------------

1.Add new Levels 6.3 to 7.2 specified in ITU-T H.265 (V9) (09/2023) and above
2.Improve Slices feature with check zeroMv
3.Enable frame parallelism with MCSTF feature
4.Updated support to signal AOM FGM params
5.Improve quality with SBRC feature
6.Updated DolbyVision P5 VUI defaults

API changes
-----------
1. API Support to enable Threaded Motion Estimation(--threaded-me)

Optimizations
-------------
1. RISC V optimizations including SAD, SATD, DCT, IDCT, block copy, pixel utilities, SAO, loopfilter, transpose kernels resulting in 2x encoding speed.
2. ARM SIMD optimizations including the use of NEON and SVE instruction set extensions. The following algorithms now have optimized SIMD implementations: DST, IDCT, SSE, SSD ,intra_pred_planar, pelFilterLumaStrong, interpolation, planecopy, dequant_normal, blockcopy, pixel variance resulting in 8% faster encoding speed compared to v4.1

Bug fixes
---------
1. Fix memory leaks (no command line option, SEI buffer, analysis save/load)
2. Fix chroma qp offset for non yuv444 inputs
3. Fix max supported input resolution
4. Fix bugs with ARM SIMD optimizations
5. Fix Alpha and Multiview feature flag support in x265_config
6. Fix test harness issues, CMake errors
7. Fix inconsistent output with aq-motion
8. Fix crash with hist-scenecut on high bit-depth builds
9. Fix lookahead concurrency bug
10. Fix shared link issue (R_X86_64_PC32), yuv recon output issue, rd-refine and dynamic-refine issue, inputs for Windows named pipe, weighted prediction delta_chroma_offset, crf and vbv issue in abr-ladder, psnr and ssim reported with MCSTF feature, internally overflowed VBV variables

Known issues
------------
1. Output mismatch between analysis save & load with cutree with reuse level < 10
2. Inconsistent hash mismatch with abr-ladder feature
3. Performance regression observed with threaded-me feature on high frequency systems and for higher resolutions (4k)

Thanks,
Kirithika

[rwill]

How long did we wait for this? Months, years? :eek:

Wait for what? There is almost nothing of interest in this 4.2 release.

*edit
And still no news about x266.

[Hostile_18]

Has anyone noticed x265 changing the colour of grain? I'm encoding Black Swan that has orange grain elements and x265 changes a fair amount of it to black/grey. x264 doesn't have this problem. I had a look at other peoples x265 encodes for it and all have the same problem as well.

[LigH]

Wait for what?

For a minor version bump. This should reset some issues regarding a buggy counting of the git commit hash related to release+patches versions in the MinGW building workflow in M-AB-S I had because I kept updating for a long time and the CMake script started confusing some git branches. The result was that several people building the exact same commit got different hashes in the version strings of their binaries. So I decided to stop releasing my builds until I get in sync again.

[Z2697]

Has anyone noticed x265 changing the colour of grain? I'm encoding Black Swan that has orange grain elements and x265 changes a fair amount of it to black/grey. x264 doesn't have this problem. I had a look at other peoples x265 encodes for it and all have the same problem as well.

Maybe it'll be fixed by the correct chroma offset, but who knows
But really, what kind of bitrate are you talking about? Because without it there's no point.

[tormento]

How long did we wait for this? Months, years? :eek:
Can you link the source of that news?

[Hostile_18]

Maybe it'll be fixed by the correct chroma offset, but who knows
But really, what kind of bitrate are you talking about? Because without it there's no point.

It was CRF 18 with 14000kpbs cap.

I am having better results with CRF 19 with 20000 cap. So I may need to adjust!

[rwill]

It was CRF 18 with 14000kpbs cap.

I am having better results with CRF 19 with 20000 cap. So I may need to adjust!

Try not to turn your CRF encodes into CBR only encodes with too low caps then.

[Hostile_18]

Try not to turn your CRF encodes into CBR only encodes with too low caps then.

Thing is with x264 ive seen encodes that are only 5000kbps and the grain keeps it color, with x265 its taking 17000kbps+ which is a crazy difference unfortunately.

[Z2697]

Thing is with x264 ive seen encodes that are only 5000kbps and the grain keeps it color, with x265 its taking 17000kbps+ which is a crazy difference unfortunately.

Seen what, same title or not? Same process or not?

[Hostile_18]

Seen what, same title or not? Same process or not?

Black Swan (same movie). Hard to say the process but the bit rate is 5000kbps in x264 and its beating out x265 at <17000kbps strictly in terms of grain color accuracy.

Black Swan is a rather unique and hard film to encode due to its filming process and also it's one of the only films I've seen with elements of yellow/orange (and chunky) grain. Something x265 codec is doing at lower bit rates is changing the color of the grain. Only at 17000kbps+ in x265 does it match x264.

[Z2697]

https://slow.pics/c/FkM3JcWZ
Can't say I notice any "less colourful" grain with both 5 Mbps 2-pass ABR encoding. (both less than source, though)
Notice I'm not even using chroma offsets in x265.
8bit vs 10bit might be a bit less than ideal, but that's the "standard" right? (well it's a shame that 10bit AVC didn't have it's best time)


ffmpeg -i mkv -vf crop=1920:800 -b:v 5M -pass 2 p2.264
ffmpeg -i mkv -vf crop=1920:800,zscale,format=yuv420p10le -x265-params sao=0:aq-mode=1:ctu=32:slow-firstpass=0 -b:v 5M -pass 2 p2.265


Hate to say, but you are doing it wrong.

[x264N00b]

Those screencaps mean nothing when it's not clear which frame typ they are. 10010-1png looks a way to good for 5000kbps. I-Frame?

[Z2697]

Those screencaps mean nothing when it's not clear which frame typ they are. 10010-1png looks a way to good for 5000kbps. I-Frame?

Both bidir frame.

(BTW I'm wanted to implement "unidir B frames" in x265 for no reason for some time but haven't figure it out LOL)

[Hostile_18]

https://slow.pics/c/FkM3JcWZ
Can't say I notice any "less colourful" grain with both 5 Mbps 2-pass ABR encoding. (both less than source, though)
Notice I'm not even using chroma offsets in x265.
8bit vs 10bit might be a bit less than ideal, but that's the "standard" right? (well it's a shame that 10bit AVC didn't have it's best time)


ffmpeg -i mkv -vf crop=1920:800 -b:v 5M -pass 2 p2.264
ffmpeg -i mkv -vf crop=1920:800,zscale,format=yuv420p10le -x265-params sao=0:aq-mode=1:ctu=32:slow-firstpass=0 -b:v 5M -pass 2 p2.265


Hate to say, but you are doing it wrong.

00:48:00-00:49:00 is my test video. Specifically 00:48:14 when the camera changes from Natalie Portman is a good example. All the yellow grain at the top half of the frame is muted/dark (against the grey wall).

I did think it may be something I was doing even though I was changing alot of variables. The only thing that worked was 17000> bit rate, and even then its not perfect. I checked against popular x265 groups and theirs exhibited the same problem. I didn't try two pass though as unfortunately Fileflows, the program I am using dosn't support two pass ABR (with advanced commands).

[GeoffreyA]

How long did we wait for this? Months, years? :eek:

Well, at least the chroma-offset fix has gone in.