x265 HEVC Encoder

[LigH]

What Boulder said.

IIRC, a specific commit gets the tag "version 3.5" assigned, which may happen "belated" (does not need to be the tip of the branch while assigning).

[ShortKatz]

Bugs not yet fixed:

NASM 2.15.05 multi-line macro warnings (reported August 2020)


Yes, this is a bit annoying. I submitted a patch for this. But they didn't like, even though it was exactly the same which was done for x264.

[Barough]

x265 v3.5+9-bf91444e0 (https://www.mediafire.com/file/dlimoxr9z9lc6n1/x265-3.5%252B9-bf91444e0_Win_GCC102.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.2.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

[tormento]

Is it even measurable when the core routines are all assembler optimized already?
In the spare time, I did some tests.

HEVC encoder version 3.5+20-4c4aee0bc [DJATOM's Mod] - SandyBridge build: 4.15 fps

HEVC encoder version 3.5+20-4c4aee0bc [DJATOM's Mod] - generic build: 3.51 fps

HEVC encoder version 3.5+9+14-6c69ed37d [Mod by Patman] - generic build: 3.76 fps

As you can see, there is some gain.

------------------------- System Environment -------------------------

StaxRip : 2.3.0
Windows : Windows 10 Enterprise 2009
Language : Italian (Italy)
CPU : Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
GPU : NVIDIA GeForce GTX 1060 3GB, Intel(R) HD Graphics 3000

------------------------- Source Script Info -------------------------

Width : 1440
Height : 1080
Frames : 6984
Time : 04:51.291
Framerate : 23.976023 (24000/1001)
Format : YUV420P8

--------------------------- Video encoding ---------------------------

D:\Eseguibili\Media\StaxRip\Apps\Encoders\x265\x265.exe --crf 22 --preset slow --output-depth 10 --colorprim bt709 --colormatrix bt709 --transfer bt709 --range limited --input-depth 16 --dither --output "F:\In\4_02 Zack Snyder's Justice League\Justice_0_temp\Justice_0_out.hevc" "F:\In\4_02 Zack Snyder's Justice League\Justice_0_temp\Justice_0.avs"

[LeXXuz]

In the spare time, I did some tests.

HEVC encoder version 3.5+20-4c4aee0bc [DJATOM's Mod] - SandyBridge build: 4.15 fps

HEVC encoder version 3.5+20-4c4aee0bc [DJATOM's Mod] - generic build: 3.51 fps

HEVC encoder version 3.5+9+14-6c69ed37d [Mod by Patman] - generic build: 3.76 fps



Btw. are there any Zen+/Zen2 optimized builds, if any at all?

[DJATOM]

Yes, I'm also using zen2 build.

[benwaggoner]

Yes, I'm also using zen2 build.
Someone who is good at such things could do us all a big favor by having a page of x265 binaries optimized for common cpu architectures.

That big a perf gain as reported above is surprising. And suggests that profile guided optimization may also yield some further perf benefits.

[LeXXuz]

Someone who is good at such things could do us all a big favor by having a page of x265 binaries optimized for common cpu architectures.

That big a perf gain as reported above is surprising. And suggests that profile guided optimization may also yield some further perf benefits.

Second that. Even a small percentage in performance increase would be great and really pay off for bigger projects.

I'm a complete noob when it comes to compiling and would love to have an optimized build for my 39x0 Ryzen machines. :o

[DJATOM]

You can grab those optimized builds from my repo: https://github.com/DJATOM/x265-aMod/releases/tag/3.5%2B20
They are all in the 7z archive, since it's easier to upload stuff to github with single file.
Zen3 flags aren't present in gcc10 (but that available with gcc11), so no such build for now. I'll consider doing it when will buy 5950x.

PS: cpu-specific builds are 10-bits only, while generic is multilib.

[tormento]

You can grab those optimized builds from my repo
Thanks for your work, that I appreciate with x264 too.

I am always wondering why, on modern computers, you and others builders keep on crunching x265.exe with upx :p

[benwaggoner]

You can grab those optimized builds from my repo: https://github.com/DJATOM/x265-aMod/releases/tag/3.5%2B20
They are all in the 7z archive, since it's easier to upload stuff to github with single file.
Zen3 flags isn't present in gcc10 (but that available with gcc11), so no such build for now. I'll consider doing it when will buy 5950x.

PS: cpu-specific builds are 10-bits only, while generic is multilib.
Excellent!

For a Cascade Lake system, I would use the Skylake version?

[DJATOM]

Yes, from 6xxx to 10xxx Intel CPUs they are using the same microarchitecture.
11xxx however uses Cypress Cove, which near 19% better than Skylake on the same clocks.

[Asmodian]

11xxx however uses Cypress Cove, which near 19% better than Skylake on the same clocks.

At what?

None of the standard benchmarks hold that up, so I am curious in what context that +19% claim is true. The i9-11900K seems to be slower than the i7-10700K for x265.

https://www.techpowerup.com/review/intel-core-i9-11900k/15.html

Edit: I cannot read graphs, apparently. :o
+14% for the i9-11900K v.s. the i7-10700K at only 4% higher clocks, for x265 encoding. It really is the loss of two cores that hurts the i9 11 series this generation. At 10nm and with a 12 core option Cypress Cove would probably be amazing.

[LeXXuz]

You can grab those optimized builds from my repo: https://github.com/DJATOM/x265-aMod/releases/tag/3.5%2B20
They are all in the 7z archive, since it's easier to upload stuff to github with single file.
Zen3 flags aren't present in gcc10 (but that available with gcc11), so no such build for now. I'll consider doing it when will buy 5950x.

PS: cpu-specific builds are 10-bits only, while generic is multilib.

Thank you very much for those builds. :thanks:

[DJATOM]

At what?

None of the standard benchmarks hold that up, so I am curious in what context that +19% claim is true. The i9-11900K seems to be slower than the i7-10700K for x265.

https://www.techpowerup.com/review/intel-core-i9-11900k/15.html

Edit: I cannot read graphs, apparently. :o
+14% for the i9-11900K v.s. the i7-10700K at only 4% higher clocks, for x265 encoding. It really is the loss of two cores that hurts the i9 11 series this generation. At 10nm and with a 12 core option Cypress Cove would probably be amazing.

I just meant that if you lock both CPUs at the same clock speed, you will have 19% uplift due to IPC improvements. That's all. Overall improvement might differ due to termals and other factors.

[excellentswordfight]

At what?

None of the standard benchmarks hold that up, so I am curious in what context that +19% claim is true. The i9-11900K seems to be slower than the i7-10700K for x265.

https://www.techpowerup.com/review/intel-core-i9-11900k/15.html

Edit: I cannot read graphs, apparently. :o
+14% for the i9-11900K v.s. the i7-10700K at only 4% higher clocks, for x265 encoding. It really is the loss of two cores that hurts the i9 11 series this generation. At 10nm and with a 12 core option Cypress Cove would probably be amazing.
I just meant that if you lock both CPUs at the same clock speed, you will have 19% uplift due to IPC improvements. That's all. Overall improvement might differ due to termals and other factors.
Swedish site sweclockers does benchmarks at locked frequency (2.8Ghz)

Cinebench R20 (floating point) 19%
https://cdn.sweclockers.com/artikel/diagram/23153?key=5f49827dc3c9a52beaf553bfe7a4aac8

Geekbench 5 (floating point) 16%
https://cdn.sweclockers.com/artikel/diagram/23189?key=d6ab89c5b375c73a183b5a04d052042e

Geekbench 5 (interger) 14%, matches zen3
https://cdn.sweclockers.com/artikel/diagram/23190?key=ca599e8494a02de1ecf6b86cf624109c

Also note that performance claims could be at stock RAM speed (2933 & 2666 for comet lake vs 3200) and most sites does tests at the same speed across platforms. And that IPC is kind of a bad metric to use in generic terms as it will very a lot between architectures depending at what kind of load we are talking about (Floating Point? AVX? Interger? Does it benefit from large cache? RAM bandwidth? etc.).

And yeah, especially for the money the i9 is disappointing at just 8c, but the i7 compared to the more expensive 5800X seems to be a rather good product.

[DJATOM]

I'd wait for adequate pricing on 5950x, no point for me to buy now for overprice.

[tormento]

I'd wait for adequate pricing on 5950x, no point for me to buy now for overprice.
Waiting for waiting, I will jump to DDR5, much better.

[DJATOM]

Waiting for waiting, I will jump to DDR5, much better.

I already have AM4 platform with 3900x and decent cooler, it will be a pretty cheap upgrade for me.

[tormento]

I already have AM4 platform with 3900x and decent cooler, it will be a pretty cheap upgrade for me.
I have a Sandy Bridge @ 4.6 stable. Still the best for the bucks. :p

[Boulder]

Memory speed has very little effect on the encoding speed. It's the pure calculation power that rocks.

[benwaggoner]

I have a Sandy Bridge @ 4.6 stable. Still the best for the bucks. :p
The lack of AVX2 is pretty painful for x265. A Skylake offers quite a bit better fps per clock.

My Sandy Bridge to Skylake upgrade gave me a >>2x fps improvement, although that included an increase in clock speed and core count.

[tormento]

The lack of AVX2 is pretty painful for x265. A Skylake offers quite a bit better fps per clock.
I know. I have a Haswell Xeon in my server but can't overclock it.

[benwaggoner]

I just meant that if you lock both CPUs at the same clock speed, you will have 19% uplift due to IPC improvements. That's all. Overall improvement might differ due to termals and other factors.
Thermals can be a big problem here. Often the first version of a processor supporting a new AVX version has some severe thermal throttling when heavily using those instructions which get addressed in a later revision. I recall AVX2 got much better for that reason at some point. Similar thermal issues limited the utility of AVX-512. Does anyone know if the latest Intel CPUs address that? Not like we have any with enough cores for optimal x265 encoding yet.

[DJATOM]

Yet 4670K was hot enough for AVX workloads. IIRC 60 degrees at 4.3 GHz with SSE2 and 90 with AVX. I had it delidded btw.

[Boulder]

If you use the system mainly for x265 encodes, a new Zen build is simply the best bang for buck. 3900X is already quite fast and Zen3 CPUs even faster. With new Zens the temps still should be quite low, my Zen2 3900X runs ~65C with full load (fixed clocks at 3.8GHz, no PBO, undervolted). Overclocking only causes a lot more heat and use of electricity will smallish benefits.

[benwaggoner]

If you use the system mainly for x265 encodes, a new Zen build is simply the best bang for buck. 3900X is already quite fast and Zen3 CPUs even faster. With new Zens the temps still should be quite low, my Zen2 3900X runs ~65C with full load (fixed clocks at 3.8GHz, no PBO, undervolted). Overclocking only causes a lot more heat and use of electricity will smallish benefits.
Yeah, I'm going to look at that. Got some compute-heavy stuff coming up in the next few months.

My current workstation is configured to match an AWS EC c5.18xlarge instance, so I can extrapolate performance for those. But there are now the c5a AMD instances. Up to, wow, 96 vCPU.

I wonder if I'll be looking at a Graviton2 equivalent workstation in a couple of years. There's not a lot of info about it for video encoding outside of this oddly specific post comparing H.264 encoding speeds between Graviton2 and Xeon 8-core instances (https://community.arm.com/developer/tools-software/tools/b/tools-software-ides-blog/posts/thirty-six-percent-better-video-encoding-with-aws-graviton2_2d00_based-c6g). A 36% improvement in min/dollar isn't that impressive yet, and a comparison with an AMD instance might have gone the other direction.

Curious to see what x265 can do on ARM with full SIMD optimization. The ARM SIMD world seems to becoming a lot more heterogenous than with x86-64, though.

[LeXXuz]

Overclocking only causes a lot more heat and use of electricity will smallish benefits.

Completely agree. But that was always the case. Both my Zen2 CPUs are watercooled, but still, I leave them running with default PBO settings. The 3950x gets up to 180W TDP with maximum stable OC. That's about 45% more from its default limit. At the same time x265 speed only increases by less than 10%. Absolutely not worth it.

[DJATOM]

I'm also fixing my cpu multiplier to 38 while voltage is 1.086. The temps are near range 58-62C, I'm with air cooling (Dark rock pro 4).

[abban270]

hi every one
am asking for good preset to encode small videos size of home videos without losing most of quality
am using Staxrip
and this this an sample
https://www.mediafire.com/file/vp1x7sh0qe0zmze/test.mp4/file
thanks in advance

[LigH]

If you need StaxRip specific help, better ask in the StaxRip thread...

In general: There is no magic. Preserving enough quality needs enough bitrate (1-pass CRF encoding will help here). The rest is avoiding more loss with wrong options... But this clip seems to be rather generic and without interlacing.

[martinpickett]

I wonder if I'll be looking at a Graviton2 equivalent workstation in a couple of years. There's not a lot of info about it for video encoding outside of this oddly specific post comparing H.264 encoding speeds between Graviton2 and Xeon 8-core instances (https://community.arm.com/developer/tools-software/tools/b/tools-software-ides-blog/posts/thirty-six-percent-better-video-encoding-with-aws-graviton2_2d00_based-c6g). A 36% improvement in min/dollar isn't that impressive yet, and a comparison with an AMD instance might have gone the other direction.

I found this article (https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd) from AnandTech illuminating in general. For specific video encoding data you have to look at the 525.x264_r benchmark in SPECint2017 covered on pages seven (https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd/7) and eight (https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd/8).

[imhh11]

You can grab those optimized builds from my repo: https://github.com/DJATOM/x265-aMod/releases/tag/3.5%2B20
They are all in the 7z archive, since it's easier to upload stuff to github with single file.
Zen3 flags aren't present in gcc10 (but that available with gcc11), so no such build for now. I'll consider doing it when will buy 5950x.

PS: cpu-specific builds are 10-bits only, while generic is multilib.

Hi, thank you so much for doing this.
Which one should I get for Kaby Lake cpu ?

[microchip8]

Hi, thank you so much for doing this.
Which one should I get for Kaby Lake cpu ?

Skylake build

[abban270]

If you need StaxRip specific help, better ask in the StaxRip thread...

In general: There is no magic. Preserving enough quality needs enough bitrate (1-pass CRF encoding will help here). The rest is avoiding more loss with wrong options... But this clip seems to be rather generic and without interlacing.

ok i will post the same thread in staxRip section
and if you have try to re-encode this clip to x265 with good bitrate and small size dont forget to post the result
maybe i will take it
thanks

[benwaggoner]

I found this article (https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd) from AnandTech illuminating in general. For specific video encoding data you have to look at the 525.x264_r benchmark in SPECint2017 covered on pages seven (https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd/7) and eight (https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd/8).
Thanks. And wow, Graviton2 is the winner for 16-core H.264 reference encoder perf! And also for x264 at 64vcpu. Of course, a Graviton2 vCPU is a full core and a Xeon/AMD is half of a SMT core. Still, very impressive!

IIRC, x264 has more even amounts of ARM and x86-64 SIMD optimization than x265, so x265 might not be as competitive yet. I'll try to get some tests run when I've got some time.

[LigH]

Fun with Yuuki-Asuna again:

Because version 3.5 is official, and I saw that the Yuuki branch had a lot of accepted commits 3 days ago, I decided to try to build a fresh clone in MSYS2/MinGW64 (MABS). But the first surprise I saw was the version number being calculated, so I stopped here...

-- x265 Release Version 2.3M+850-g2b25c9ba0+47

Please give some advice if/when/how there will be another chance to get it running again, with your help, and where to discuss that (probably better not here; possibly in msg7086's Issue tracker?)...

[aegisofrime]

Hey,

Could anyone point me to a "for dummies" guide to compiling x265 for Windows with gcc? I'm aware there's instructions on the x265 wiki, but... It looks like it's written for someone who is already very experienced with cross compilation, which unfortunately I'm not. I do have experience with general Linux use, so hopefully I would not need my hand held too much.

As for why I'm planning to cross compile... My objective is just to try out the new znver3 march option on my 5950X.

Thanks!

[LigH]

My binaries were built on an MSYS2/MinGW basis of the "media-autobuild suite" under Windows, but with a small set of manually executed shell scripts in an interactive shell, derived from the "build/msys" templates provided by MultiCoreWare, which I may share, so you could adapt them with your specific compiler optimizations as env-vars.

But if you do so, don't be afraid of a plethora of NASM warnings not yet fixed...

Adapting these scripts for MinGW cross-compilation under a real Linux is also possible, I sometimes tried that in an Ubuntu MATE VM...

[DJATOM]

I just built znver3 optimized binary using gcc10.3, you can get it here (https://github.com/DJATOM/x265-aMod/releases/download/3.5%2B20/x265-x64-v3.5+20-aMod-gcc10.3.0-opt-znver3.7z).

[aegisofrime]

My binaries were built on an MSYS2/MinGW basis of the "media-autobuild suite" under Windows, but with a small set of manually executed shell scripts in an interactive shell, derived from the "build/msys" templates provided by MultiCoreWare, which I may share, so you could adapt them with your specific compiler optimizations as env-vars.

But if you do so, don't be afraid of a plethora of NASM warnings not yet fixed...

Adapting these scripts for MinGW cross-compilation under a real Linux is also possible, I sometimes tried that in an Ubuntu MATE VM...

If you could share them that would be great! :thanks:

I just built znver3 optimized binary using gcc10.3, you can get it here (https://github.com/DJATOM/x265-aMod/releases/download/3.5%2B20/x265-x64-v3.5+20-aMod-gcc10.3.0-opt-znver3.7z).

If it's not too much trouble, could I request for a 12-bit build as well? :)

[DJATOM]

Sure, link (https://github.com/DJATOM/x265-aMod/releases/download/3.5%2B20/x265-x64-v3.5+20-aMod-gcc10.3.0-opt-znver3-12b.7z).

[aegisofrime]

Much obliged, cheers!

[LigH]

x265 multilib build script for x86-64 Windows EXE in media-autobuild suite's MSYS2/MinGW64 mintty shell (https://gist.github.com/LigH-de/30dcb3d8434b29752041107482c77f55)

Configured for media-autobuild suite with ccache enabled. Naming is a bit historical, I have a few similar scripts with different content and architectures (also W32 and W32XP).

Results will remain in /build/x265_git-git/build/msys64_hdr10_ml/8bit in the MABS directory structure. This is for pure git sources from MultiCoreWare, without modifications.

[eclipse98]

Hi All,

Please help me understand what's going on here, because it does not quite compute.

Using a higher level (and slower) preset should, theoretically, result in:

1. Lower QP for the same bitrate.
2. Lower bitrate for the same QP.

Yet, results below on the same 2 min clip seem to contradict this assumption:

slow preset: encoded 7194 frames in 0:09:57.54 (12.04 fps), 5112.86 kb/s, Avg QP:33.05
slower preset: encoded 7194 frames in 0:37:57.12 (3.16 fps), 5515.60 kb/s, Avg QP:33.24
very slow preset: encoded 7194 frames in 1:20:43.82 (1.49 fps), 5440.53 kb/s, Avg QP:33.33

Compared to Slow preset, Slower and 'Very Slow' presets have both produced higher bitrate and higher QP.
Using default preset settings with '--crf 25 --no-sao', x265 3.4+70-aMod-gcc10.2.1 DJATOM Mod.

The only conclusion I can up with is that 'Avg QP' value is not to be trusted.

Cheers and thanks for your help!

[LeXXuz]

Hi All,

Please help me understand what's going on here, because it does not quite compute.

Using a higher level (and slower) preset should, theoretically, result in:

1. Lower QP for the same bitrate.
2. Lower bitrate for the same QP.

Yet, results below on the same 2 min clip seem to contradict this assumption:

slow preset: encoded 7194 frames in 0:09:57.54 (12.04 fps), 5112.86 kb/s, Avg QP:33.05
slower preset: encoded 7194 frames in 0:37:57.12 (3.16 fps), 5515.60 kb/s, Avg QP:33.24
very slow preset: encoded 7194 frames in 1:20:43.82 (1.49 fps), 5440.53 kb/s, Avg QP:33.33

Compared to Slow preset, Slower and 'Very Slow' presets have both produced higher bitrate and higher QP.
Using default preset settings with '--crf 25 --no-sao', x265 3.4+70-aMod-gcc10.2.1 DJATOM Mod.

The only conclusion I can up with is that 'Avg QP' value is not to be trusted.

Cheers and thanks for your help!

Without going into too much detail: from my experience WITH CRF; the higher the preset, the more detail will be preserved (at the same CRF level.)
=> Longer encoding time, bigger filesizes and higher quality of the output. :)

[LigH]

Higher presets preserving more quality is only "true with constraints", more among the faster presets than among the slower ones ... a slower preset enables more elaborate features to search for "redundancies" (similarities useful to save space). If more can be found, chances are that the difference left is smaller, so the precision of difference encoding in P and B frames can be better. But doesn't need to, certainly; it is also possible that saving some additional data for these more elaborate features requires more space than without. At least it's quite probable that there need to be fewer intra encoded blocks, so the encoding can be more efficient, in general. Yet, in practice, most users won't need any slower preset than "slower". And "placebo" is certainly a waste of time and energy.

In addition, the target for the CRF mode is the "rate factor", a metric for the loss of quality. It is possible that the features enabled in a faster preset may return more efficient results for a specific video material than the enabled features of a slower preset. There are no guarantees.

[eclipse98]

Higher presets preserving more quality is only "true with constraints", more among the faster presets than among the slower ones ... a slower preset enables more elaborate features to search for "redundancies" (similarities useful to save space). If more can be found, chances are that the difference left is smaller, so the precision of difference encoding in P and B frames can be better. But doesn't need to, certainly; it is also possible that saving some additional data for these more elaborate features requires more space than without. At least it's quite probable that there need to be fewer intra encoded blocks, so the encoding can be more efficient, in general. Yet, in practice, most users won't need any slower preset than "slower". And "placebo" is certainly a waste of time and energy.

In addition, the target for the CRF mode is the "rate factor", a metric for the loss of quality. It is possible that the features enabled in a faster preset may return more efficient results for a specific video material than the enabled features of a slower preset. There are no guarantees.

Thank you for your help, Everybody! - it seems that my (quite logical) assumption that higher-level preset will automatically result in better QP or bitrate is questionable at best - I'll do more tests on a different video material to confirm.

Cheers! :thanks:

[LigH]

New upload: x265 (Yuuki-Asuna mod) 3.5+2-g2b25c9ba0+45 (Yuuki branch) (https://www.mediafire.com/file/qh6tfc5ji32trs2/x265_3.5+2-g2b25c9ba0+45_Yuuki.7z/file)

MSYS2/MinGW (Win32, Win32XP, Win64)

x265 [info]: HEVC encoder version 3.5+2-g2b25c9ba0+45
x265 [info]: build info [Windows][GCC 10.2.0][64 bit] Yuuki 8bit+10bit+12bit
x265 [info]: (lsmash 2.16.1)
x265 [info]: (libavformat 58.78.100)
x265 [info]: (libavcodec 58.136.101)
x265 [info]: (libavutil 56.72.100)

[MeteorRain]

Fun with Yuuki-Asuna again:

-- x265 Release Version 2.3M+850-g2b25c9ba0+47


I always forget to push new tag (3.5) from upstream to my repo, so it slips all the way to 2.3. Simply let me know next time if I forget again.

Also I've rewritten ruby code with native cmake code. It should now work without ruby package.