x265 HEVC Encoder

[benwaggoner]

I have an argument with someone.
If I increase aq-strength do high motion areas/scenes get more or less bits?
What aq-strength does is baed on the --aq-mode being used. Generally aq-modes are about preventing blurring in smoother areas caused by more naive PSNR or SAD type metrics. In general x265 will spend fewer bits on areas of high motion that aren't good references for frames later in the GOP; --cutree is the main tool to identify those. This results in adaptive quantization, absolutely, but not directly controlled by --aq-mode or --aq-strength. I'm sure there are some interactions in practice, as all kinds of things can change how x265 allocates bits.

[benwaggoner]

Tell me the best x265 build for a processor with AVX only. Or similarly, build doesn't matter, they all work the same way?
That is going to be some old hardware. It wouldn't take that many compressed titles before it'll be cheaper on electricity to buy a newer, faster, and much more efficient CPU. The fastest pre-AVX CPU probably will have <10% the throughput of a modern enthusiast chip, which will have many more cores, AVX2, and overall better instructions per clock.

[Lan4]

That is going to be some old hardware. It wouldn't take that many compressed titles before it'll be cheaper on electricity to buy a newer, faster, and much more efficient CPU. The fastest pre-AVX CPU probably will have <10% the throughput of a modern enthusiast chip, which will have many more cores, AVX2, and overall better instructions per clock.
Buying a new processor is too easy. :) And for the purchase of a processor, you will need a new motherboard, new memory, a new power supply. Still, my question was about the x265 codec.

[Boulder]

Tell me the best x265 build for a processor with AVX only. Or similarly, build doesn't matter, they all work the same way?

They work the same way, the encoder uses the instruction sets that are detected.

You might get some performance benefits from compiling the executable with specific tuning for your CPU. Media Autobuild Suite can do this, there is a possibility to tweak the compiler parameters accordingly (-march and -mtune in specific).
https://github.com/m-ab-s/media-autobuild_suite#optional-user-files

[LigH]

Just committed:
[PATCH] Fix bug in mcstf

Building...

[Barough]

x265 v3.5+97
https://www.mediafire.com/file/bu7j9tn2c98sw5i

[LigH]

New upload: x265 3.5+97-c666bc3d3 (https://www.mediafire.com/file/9jn1cirkuyz8f0i/x265_3.5+97-c666bc3d3.7z/file)

[Windows][GCC 12.2.0][32/32XP/64 bit] 8bit+10bit+12bit

[jpsdr]

Hello.

Made a new build of my custom mod, check my Github.

Also, i've made out of curiosity some speed tests on my Core i7 6950X, encoding time results :
3.50.0.97 AVX2 GCC 12.2.1 : 614.55s
3.50.0.97 Broadwell GCC 12.2.1 : 608.67s
3.50.0.94 Broadwell LLVM 15.0.7 : 600.10s
3.50.0.97 Broadwell LLVM 16.0.0 : 590.88s

Up to 4% speed difference, well, anything is good to take :D

[FranceBB]

Buying a new processor is too easy. :)

Well, if your CPU is AVX capable only, get the AVX build: http://msystem.waw.pl/x265/

it really is as simple as that...

[john33]

Also did some speed tests with results that kind of surprised me:
Ryzen 9 3900XT - Win10 Pro x64
61,070 frames - 1920x816 --crf 24.0 --aq-mode 4 --no-cutree --no-open-gop --no-sao

Winlibs GCC 13.0.1 1,946.89secs 31.37fps
Winlibs GCC 12.2.0 1,928.15secs 31.67fps
Msys2 GCC 12.2.0 1,903.41secs 32.08fps
http://www.msystem.waw.pl/x265/ GCC 12.2.0 1,898.15secs 32.17fps
LLVM Clang 16 1,718.27secs 35.54fps
VS2019 V16.11.25 1,717.45secs 35.56fps
Intel 19.2 1,600.23secs 38.16fps These were all AVX2 optimised with similar settings.

[FranceBB]

New upload: x265 3.5+97-c666bc3d3 (https://www.mediafire.com/file/9jn1cirkuyz8f0i/x265_3.5+97-c666bc3d3.7z/file)

[Windows][GCC 12.2.0][32/32XP/64 bit] 8bit+10bit+12bit

Not that it matters, but just so you know and the XP community doesn't find out the hard way, the XP build you provided is attempting to call GetTickCount64 which doesn't exist in XP x86, while it should really be calling GetTickCount.

https://i.imgur.com/O0eP498.png
https://i.imgur.com/hPUwAGe.png

Sending GetTickCount64 to GetTickCount fixes the issue of course:

https://i.imgur.com/bcWTp4i.png
https://i.imgur.com/ojeSHqO.png

I don't know if something changed under the hood in the way you compile (or if indeed is an autobuild issue?), but... well... now you know. ;)

[LigH]

Well, thank you for your discovery ... but don't tell me. I only compile. I do not develop. Multicoreware needs to know (via mailing list) and find a workaround if they still officially support XP compatibility at all. Instead they may probably just call it obsolete.

[benwaggoner]

Well, thank you for your discovery ... but don't tell me. I only compile. I do not develop. Multicoreware needs to know (via mailing list) and find a workaround if they still officially support XP compatibility at all. Instead they may probably just call it obsolete.
Given that there is no support or security updates for XP, I think it makes sense to not offer builds for it anymore as that's encouraging people to stick with insecure systems.

For embedded applications things are different, of course.

XP doesn't have explicit NUMA support and is missing a lot of other things helpful for maximizing performance.

[filler56789]

XP doesn't have explicit NUMA support and is missing a lot of other things helpful for maximizing performance.
That's a very-good reason for an HEVC encoder to drop Windowx XP support. :goodpost:

OTOH,
Given that there is no support or security updates for XP, I think it makes sense to not offer builds for it anymore as that's encouraging people to stick with insecure systems.

is just a lame excuse for trollish devilopment. :p

Seriously: 🆖

[LigH]

I don't mind unmounting a dead horse. There are plenty old but functional versions.

[FranceBB]

Given that there is no support or security updates for XP

Although Microsoft pulled support in mid 2019 with the last security update, there's still 0Patch that picked up the task. That + a good antivirus like Avast and it's good enough for a home user, probably.
Anyway, I found out 'cause I do regular testing on all environments in a VM and one of those is indeed XP.


I think it makes sense to not offer builds for it anymore

Nah, as I showed it's easy enough to build it in an XP compatible way, so much so that with 1 minor adjustment the build works. My post was more of a "just so you know", that's it.


Well, thank you for your discovery ... but don't tell me. I only compile. I do not develop.

Yeah, that's the thing, it's not the source code per se, rather I'm trying to figure out why the compiler is calling a non existing kernel function while it really shouldn't given that you're targeting XP in a specific build...

TL;DR Multicoreware has nothing to do this with, rather the compiler or whoever made the autobuild compilation script :P


XP doesn't have explicit NUMA support and is missing a lot of other things helpful for maximizing performance.

True. In a multicore and multisocket environment, it would probably underperform by a margin: no NUMA, no AVX, no AVX2, no AVX512, also 32bit version of bit depth higher than 8bit don't have any intrinsics at all, so they would run in plain C. Anyway, I don't really think there are any businesses out there running XP/Server 2003 to encode stuff; I myself am on Server 2019 x64 with my farm at work, but I was thinking more about some home users, that's it. ;)

[ShortKatz]

Finally someone made a pull request to fix the compilation on macOS arm64.
https://mailman.videolan.org/pipermail/x265-devel/2023-April/013600.html
Hopefully this patch gets applied soon to fix the broken arm64 compilation on macOS.

[LigH]

Multicoreware has nothing to do this with, rather the compiler or whoever made the autobuild compilation script :P

In that case ... it might be possible that newer GCC versions (12) should get added a command line parameter which was not yet required in older versions (9/10)?

[benwaggoner]

Finally someone made a pull request to fix the compilation on macOS arm64.
https://mailman.videolan.org/pipermail/x265-devel/2023-April/013600.html
Hopefully this patch gets applied soon to fix the broken arm64 compilation on macOS.
It was checked in five days ago! Yay!

https://bitbucket.org/multicoreware/x265_git/commits/753305affb093ae15d5e4b333125267b16258c21

[ShortKatz]

It was checked in five days ago! Yay!

https://bitbucket.org/multicoreware/x265_git/commits/753305affb093ae15d5e4b333125267b16258c21

Yes, this are great news. :) Now I don't need the fix from HandBrake anymore to build on Mac.

[DMD]

Good morning
I have always wondered how to do to predict, the final file size based on preset Medium, Slower.... and CFR quality.
Even reading the specific article at https://slhck.info/video/2017/02/24/crf-guide.html

I have verified that in practice the data obtained is different from that predicted in the article.
I assume that the same value of CRF gives different results according to the speed preset set.
I ask if this assumption of mine can be an attempt at estimated prediction.
One could start the procedure and when we are about the 1% value , check in the temp folder the size of the produced file and make a proportion with the percentage of the remaining process.
Could this be correct?
Thank you

[rwill]

Could this be correct?


No. But you could randomly encode 50% of a sequence and then you are halfway there.

[katzenjoghurt]

@DMD

When quality reencoding a movie for smaller size I split the source into 7minute chunks using MKVToolnix. I then sort them by filesize and pick one in the middle where I think it's representative for the rest of the movie (day, night, lighting, grain).

I then encode that chunk with different crf values and compare the quality with StaxRip's built-in video comparison tool.

When changing only the crf value my rule of thumb for estimating the resulting file size is:
File size increases by 1.25^(crf_increase).
So if I rise the crf value by 3 it's estimated_new_size = old_crf_filesize*1.25*1.25*1.25.


May not totally answer your question but that's my approach to get close to the sweet spot and it works 85% of the time for me.

[DMD]

@katzenjoghurt
Thank you for the suggestion, I am also performing tests by processing only 20 % of the file and make an estimate of the total size.




A question I have always wondered why in video files with aspect ratio higher than 16:9, Crop is not used so that the pixel ratio (DAR) is higher than 1.78.
In theory, only the active part of the screen should be processed, also saving time in the compression procedure.
Analyzing with MediaInfo, commercial UHD blurays all have the 16:9 aspetc ratio.
Maybe my reasoning is not correct?
Thanks

[LigH]

The pixel aspect ratio = SAR (sample aspect ratio), means the anamorphic skew factor, is 1:1 in modern commercial video formats. The DAR (display aspect ratio) means the ratio between width and height of the whole image.

The efficiency of compressing temporally stable solid black borders covering complete macroblocks (or partitions in HEVC/H.265) is already quite high. Explicit crop areas with panning offsets may be unsupported by consumer players. Cropping 1088 to 1080 lines in AVC/H.264 video is common, though.

[DMD]

The pixel aspect ratio = SAR (sample aspect ratio), means the anamorphic skew factor, is 1:1 in modern commercial video formats. The DAR (display aspect ratio) means the ratio between width and height of the whole image.

The efficiency of compressing temporally stable solid black borders covering complete macroblocks (or partitions in HEVC/H.265) is already quite high. Explicit crop areas with panning offsets may be unsupported by consumer players. Cropping 1088 to 1080 lines in AVC/H.264 video is common, though.
So to have more player compatibility, it is better to leave the DAR and SAR settings = 1.78 by default.

[LigH]

No, I just explained the difference: DAR = 16:9, SAR = 1:1

And some additional specifications beyond MPEG-4 AVC / HEVC (like Blu-ray, Blu-ray UHD) demand such specific aspect ratios and resolutions.

[megapro17]

Hello. I need help with fully utilizing i7-13700k cpu with x265. Even with -x265-params aq-mode=3:pmode=1:pme=1 it constantly drops usage around a 50% for 1080p60 video with medium preset. What else I can try? I guess I have to increase slices?

x265 [info]: HEVC encoder version 3.5+95-38cf1c379
x265 [info]: build info [Windows][GCC 12.2.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [info]: Main profile, Level-4.1 (Main tier)
x265 [info]: Thread pool created using 24 threads
x265 [info]: Slices : 1
x265 [info]: frame threads / pool features : 4 / wpp(17 rows)+pmode+pme
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : hex / 57 / 2 / 3
x265 [info]: Keyframe min / max / scenecut / bias : 25 / 250 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 3 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 3 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : CRF-28.0 / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip mode=1 signhide tmvp
x265 [info]: tools: b-intra strong-intra-smoothing lslices=6 deblock sao

[benwaggoner]

Hello. I need help with fully utilizing i7-13700k cpu with x265. Even with -x265-params aq-mode=3:pmode=1:pme=1 it constantly drops usage around a 50% for 1080p60 video with medium preset. What else I can try? I guess I have to increase slices?

Are you using HyperThreading? If so, you're probably getting performance values assuming each logical core is a physical core. Your 50% may indicate that you're fully utilizing all your physical cores.

Or perhaps the Efficiency cores aren't getting much work as the Performance cores are so much more useful with x265.

8 physical threads for 1080p should saturate just fine without --pme and --pmode with 4 frame threads.

[guest]

Hello. I need help with fully utilizing i7-13700k cpu with x265.

Have a bit of a read thru this...

https://x265.readthedocs.io/en/master/cli.html

Won't hurt, might help ;)

[megapro17]

Are you using HyperThreading? If so, you're probably getting performance values assuming each logical core is a physical core. Your 50% may indicate that you're fully utilizing all your physical cores.


That isn't a case surely, because fans aren't blowing much, pc is almost silent. In other workloads I can feel a heat. And there's some periods when x265 is loading a cpu fully. I got video encoded on i5-12400 faster than on my cpu, that's ridiculous. Maybe there's something with effeciency cores, is there any information on this?

Have a bit of a read thru this...)

your sarcasm is inappropriate, I read it and enabled pmode and pme that helped, but not too much. I can't see any options besides --slices and --ctu that should help

[Boulder]

Those settings are quite light for modern CPUs so it's no wonder it doesn't work more.
Lowering CTU to 32 will increase multithreading, and it really won't hurt quality or compression. --rskip 2 will increase quality and also give more work to the CPU. Maybe try those, or just choose --preset slow for more things.

[benwaggoner]

That isn't a case surely, because fans aren't blowing much, pc is almost silent. In other workloads I can feel a heat. And there's some periods when x265 is loading a cpu fully. I got video encoded on i5-12400 faster than on my cpu, that's ridiculous. Maybe there's something with effeciency cores, is there any information on this?
Take a look at what cores are doing what in Resource Manager. That should be helpful in tracking down what's up.

Trying --preset slower and seeing if there is a difference would also be useful.

[jpsdr]

Hi.
If i understood properly some previous posts, there is code for AVX512, but it's not compiled by default, you have to use a CMake option for the code being compiled.
I searched in the CMakeLists but didn't find anything.
What is the CMake option to have the AVX512 code branch compiled in the build ?

[guest]

Hi.
If i understood properly some previous posts, there is code for AVX512, but it's not compiled by default, you have to use a CMake option for the code being compiled.
I searched in the CMakeLists but didn't find anything.
What is the CMake option to have the AVX512 code branch compiled in the build ?

You can't use AVX512 on 12th & 13th Gen Intel CPU's.

[guest]

your sarcasm is inappropriate, I read it and enabled pmode and pme that helped, but not too much. I can't see any options besides --slices and --ctu that should help

WHAT sarcasm ???

You read, it helped :confused:

[excellentswordfight]

That isn't a case surely, because fans aren't blowing much, pc is almost silent. In other workloads I can feel a heat. And there's some periods when x265 is loading a cpu fully. I got video encoded on i5-12400 faster than on my cpu, that's ridiculous. Maybe there's something with effeciency cores, is there any information on this?
With default settings, the utilization you are getting are normal, @ 1080p x265 doesnt saturate 24 threads. From my experience it scales to about 16 threads at the lower presets at least. You will probably see close to full utilization with --ctu 32.

If 12400 is faster, it does indeed sound like it could be a issue with the scheduler, what windows version are you using? I havnt seen an issue with the effeciency cores on my 12700k on W11, but I also always encode at close to 100% load. You can confirm if they are the issue or not by just setting cpu affinity for x265 in task manager and removing the efficiency cores and see if that gives you a speed increase for cases were x265 cannot fully saturate the cpu.

[LigH]

New upload: x265 3.5+100-3dae0c3cc (https://www.mediafire.com/file/1u7soqag7gjr1yz/x265_3.5+100-3dae0c3cc.7z/file)

[Windows][GCC 12.2.0][32/32XP/64 bit] 8bit+10bit+12bit

Reflect Zonefile Parameters inside Lookahead.

[Barough]

x265 v3.5+101
[Windows][GCC 12.2.0][32/64 bit] 8bit+10bit+12bit
https://bitbucket.org/multicoreware/x265_git/commits/branch/master

DL :
https://www.mediafire.com/file/rxax09nfkrfk8pu/

[john33]

x265-r3.50.0.101.02ef6de with Patman mods - AVX2 optimised
[Windows][64 bit] 8bit+10bit+12bit

GCC.13.1.0: https://www.rarewares.org/files/x265-r3.50.0.101.02ef6de.GCC13.1.0.AVX2.zip

Intel.19.2: https://www.rarewares.org/files/x265-r3.50.0.101.02ef6de.Intel.19.2.AVX2.zip

[jpsdr]

Hello.

Forgot to tell, made a new build of my custom mod, check my Github.

[LeXXuz]

Over 100 commits and still 3.5. Are they afraid to push it to 3.6 or what is the reason? :confused:

[Barough]

x265 v3.5+101
Built on May 05, 2023, GCC 13.1.0

DL :
https://www.mediafire.com/file/mj2zqkwdpnv8esd/

[jpsdr]

New +103 build, check my github.

[madey83]

New +103 build, check my github.


Thanks for that, but every new release is publish I struggle to find release notes.

Could please attach into GitHub release or here?

[jpsdr]

I personnaly will not bother to copy release notes when they can be checked on the github, just see the text of the commits.

[megapro17]

If 12400 is faster, it does indeed sound like it could be a issue with the scheduler, what windows version are you using?
I'm using Windows 11 with all updates 22621. You was right, apparently, windows restricts background application to only efficency cores! When I pin window at top, my CPU is loaded at 100% all times. This is incredibly stupid, I couldn't even assume that! Problem is known for more than a year and still not fixed. all we had to do is buy the damn 7900x.
Thanks for poiniting for the most dumb problem ever. And none of the solutions are not working, only keeping window at top. Intel thread director, more like intel garbage director

https://www.reddit.com/r/XMG_gg/comments/vlqn6d/psa_rendering_tasks_are_moved_to_ecores_when/
https://www.reddit.com/r/intel/comments/tgg9hz/handbrake_windows_11_efficiency_cores_only/

[Atak_Snajpera]

Have you tried changing power plan from balanced to high performance?

[benwaggoner]

Thanks for that, but every new release is publish I struggle to find release notes.

Could please attach into GitHub release or here?
MCW only provides release notes for official releases, the most recent being 3.5.

I just read the commits for the current branch.

[megapro17]

Have you tried changing power plan from balanced to high performance?

yeah i even tried Ultimate Performance plan, it doesn't change anything