x265 HEVC Encoder

[Barough]

x265 v3.5_RC1 (https://www.mediafire.com/file/q7kfnuum7vz2acs/x265-3.5_RC1_Win_GCC102.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.2.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/Release_3.5

[MeteorRain]

Also you can ask Pat to make StaxRip edition, merging and unifying from both worlds :)

Second this, and I think it may be better to create an org to organize these projects. We are individuals and we may not always be available to fix issues or implement new features. If we can manage to get a small group of people collaborating on the stax project it may be easier.

[stax76]

The staxrip github org could be used too, no problem. The needed things are a project name, for instance community mod, people who think can work together, and there is the question if binaries should be released on github.

staxrip change I made yesterday is reading --version output to detect if it's a mod and what type of mod and then use different code paths depending on if it's aMod, Asuna or Vanilla, should have done this before.

[nakTT]

x265 v3.5_RC1 (https://www.mediafire.com/file/q7kfnuum7vz2acs/x265-3.5_RC1_Win_GCC102.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.2.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/Release_3.5

Thanks for the latest build.:thanks:

[LigH]

Note: Multicoreware asks to refresh x265 clone

We had to recently strip a commit that got pushed into Release_3.5 branch due to some unavoidable reasons. Hence, we request all those who auto-update x265 to refresh your git repos so that we'll be on the same page.

E.g. in media-autobuild suite, you would remove the build/x265_git-git directory before running the suite again.

PS: Well, m-ab-s does not aim for the Release_3.5 branch. Would have to be the HEAD if.

[Barough]

x265 v3.5_RC1+2-f3f27198 (http://www.mediafire.com/file/94n1ovortwx1vez/x265-3.5_RC1%252B2-f3f27198_Win_GCC102.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.2.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/Release_3.5

[PoeBear]

Chroma bleeding is actually really weird, it shouldn't really happen and I think I know why it's happening to you!
Your input is an AVS Script from what I see, which handles 4:2:0 Type 1 but not 4:2:0 Type2!
So, the chroma location is wrong, hence the "bleeding".

Can you try with:

ffmpeg.exe -i "AVS Script.avs" -vf scale=out_color_matrix=bt2020nc:out_h_chr_pos=0:out_v_chr_pos=0 -pix_fmt yuv420p16le -strict -1 -an -f yuv4mpegpipe - | x265.exe --y4m - --dither

followed by the rest of your x265 command line? This should solve the chroma bleeding problem. ;)

Finally got around to trying this, after tracking down some 32-bit ffmpeg builds (the 64-bit builds try to load 64-bit AviSynth+ and 64-bit plugins, threw me for a loop momentarily)

But it's drastically altering the colors when running the same exact x265 commands

Here's a comparison between the normal avs2yuv pipe I was using, and then your ffmpeg pipe from above:
https://thumbs2.imagebam.com/d2/a8/80/6b51f91369858504.jpg (http://www.imagebam.com/image/6b51f91369858504) https://thumbs2.imagebam.com/ee/19/74/8322ec1369858473.jpg (http://www.imagebam.com/image/8322ec1369858473) https://thumbs2.imagebam.com/c1/c5/0d/e8b8791369858463.jpg (http://www.imagebam.com/image/e8b8791369858463)

Here they are tonemapped for better comparison of the changes:
https://thumbs2.imagebam.com/11/6c/d3/a31a8f1369859213.jpg (http://www.imagebam.com/image/a31a8f1369859213) https://thumbs2.imagebam.com/47/3c/bf/c346b01369859138.jpg (http://www.imagebam.com/image/c346b01369859138) https://thumbs2.imagebam.com/b5/42/8b/e430db1369859174.jpg (http://www.imagebam.com/image/e430db1369859174)

Here were the commands ran, I just used one of my recent tests so I could do a comparison

avs2yuv.exe "Script.avs" - | "x265.exe" -D 10 --crf 10.5 --profile main10 --level-idc 5.1 --high-tier --preset placebo --input-depth 16 --cu-lossless
--pmode --bframes 16 --qg-size 32 --frame-threads 4 --ref 6 --limit-refs 1 --merange 57 --no-amp --tskip --tskip-fast --limit-modes --no-open-gop --hrd
--cbqpoffs -0 --crqpoffs -1 --no-cutree --deblock -2:-2 --psy-rd 2.50 --psy-rdoq 1.00 --qcomp .6 --aq-mode 2 --aq-strength 1.0 --ipratio 1.3 --pbratio 1.2
--vbv-bufsize 160000 --vbv-maxrate 160000 --aud --hdr10 --hdr10-opt --range limited --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc
--master-display G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50) --max-cll 602,187 --chromaloc 2 --repeat-headers
--output "pmode-sao-tskip-r1-test.hevc" --frames 3000 --y4m -

ffmpeg.exe -i "Script.avs" -vf scale=out_color_matrix=bt2020nc:out_h_chr_pos=0:out_v_chr_pos=0 -pix_fmt yuv420p16le -strict -1 -an -f yuv4mpegpipe - | x265.exe
--y4m - --dither --output-depth 10 --crf 10.5 --profile main10 --level-idc 5.1 --high-tier --preset placebo --input-depth 16 --cu-lossless --pmode --bframes 16
--qg-size 32 --frame-threads 4 --ref 6 --limit-refs 1 --merange 57 --no-amp --tskip --tskip-fast --limit-modes --no-open-gop --hrd --cbqpoffs -0 --crqpoffs -1
--no-cutree --deblock -2:-2 --psy-rd 2.50 --psy-rdoq 1.00 --qcomp .6 --aq-mode 2 --aq-strength 1.0 --ipratio 1.3 --pbratio 1.2 --vbv-bufsize 160000
--vbv-maxrate 160000 --aud --hdr10 --hdr10-opt --range limited --colorprim bt2020 --transfer smpte2084 --colormatrix bt2020nc
--master-display G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50) --max-cll 602,187 --chromaloc 2 --repeat-headers
--output "pmode-sao-tskip-r1-testFF.hevc" --frames 3000

True, but the thing is that he was using chromaloc 2 in x265 but his input (the AVS Script output) wasn't, so it was just wrong. Either he specifies the normal 4:2:0 or he converts it to type2 and specifies chromaloc 2. That's what I meant ;)

My input is just the raw video track from the UHD (which is chromaloc 2 per the UHD Blu-ray spec) indexed with DGDecNV. Is there a limitation with the way AviSynth+ outputs?

Script.avs:
DGSource("UHD.dgi")
Crop(0,276,-0,-276)
z_Spline36Resize(1920,804)

[benwaggoner]

I suspect you need to specify PQ as well as 2020 in ffmpeg somehow, if it supports it yet.

You want the whole SMPTE Rec.2100 package.

[_kermit]

currently I use this to encode:

ffmpeg.exe -i abc.mkv -pix_fmt yuv420p -f yuv4mpegpipe -| x265.exe -o output

But this doesn't use all my new 12 cores :) to a 100%, maybe half.
How can I improve this?

[microchip8]

currently I use this to encode:

ffmpeg.exe -i abc.mkv -pix_fmt yuv420p -f yuv4mpegpipe -| x265.exe -o output

But this doesn't use all my new 12 cores :) to a 100%, maybe half.
How can I improve this?

Have you tried passing --pmode to x265?

[stax76]

But this doesn't use all my new 12 cores to a 100%, maybe half.
How can I improve this?

Split into chunks maybe.

https://github.com/staxrip/staxrip/wiki/Usage#chunk-encoding

[_kermit]

Have you tried passing --pmode to x265?

no, haven't tried anything.
The question came up when using fastflix, which utilized almost 100% with UHD. I don't think it's related to UHD, just that it does it differently.

like this (1080p):
"C:/Users/admin/AppData/Roaming/FFmpeg/bin/ffmpeg.exe" -y -to 6492.7 -i "abc.mkv" -metadata title="abc" -max_muxing_queue_size 1024 -filter_complex "[0:0]crop=1920:800:0:140[v]" -map "[v]" -c:v libx265 -pix_fmt yuv420p10le -x265-params "aq-mode=2:repeat-headers=0:strong-intra-smoothing=1:bframes=4:b-adapt=2:frame-threads=0:hdr10_opt=0:hdr10=0" -crf 20 -preset medium -map_metadata -1 -map_chapters 0 "abc.mkv"

it's not piping but using libx265 (?), but how do I use that?

[benwaggoner]

no, haven't tried anything.
The question came up when using fastflix, which utilized almost 100% with UHD. I don't think it's related to UHD, just that it does it differently.
Being UHD is actually pretty substantial. There's a lot of frame-size proportional parallelism in x265, so bigger frames use more cores. Wavefront Parallel Processing is the most obvious one, where you extra parallelism out of each 64 pixels of frame height.

[_kermit]

Being UHD is actually pretty substantial. There's a lot of frame-size proportional parallelism in x265, so bigger frames use more cores. Wavefront Parallel Processing is the most obvious one, where you extra parallelism out of each 64 pixels of frame height.

ok, I guess I have to check with 1080p and fastflix first, or vice versa.

EDIT:

Indeed, 1080p is also not using all CPU power as UHD does.
Adding --pmode to fastflix doesn't change that.
I'm actually not that much concerned. If it's easy to do fine, otherwise never mind :)

But, any suggestions for 1080p CPU usage?

thanks.

[charliebaby]

@_kermit ? video test 2pass =
https://i.goopics.net/bem88.png

[MeteorRain]

But, any suggestions for 1080p CPU usage?

If you don't mind slightly lower efficiency you can try --ctu 32. That essentially half the row size, doubling the number of rows.

[_kermit]

@_kermit ? video test 2pass =
https://i.goopics.net/bem88.png

I always use CRF. Is 2pass doing something better in that regard?

[_kermit]

If you don't mind slightly lower efficiency you can try --ctu 32. That essentially half the row size, doubling the number of rows.

makes no difference

[LigH]

I always use CRF. Is 2pass doing something better in that regard?

2-pass encoding is only required if you aim for a specified target size. In fact, it will calculate the optimal CRF value for its 2nd pass from statistics gathered during a 1st pass.

[tormento]

x265 v3.5_RC1+2-f3f27198
Thanks!

Would you please release processor builds? At least SandyBridge I still own :)

[Barough]

Thanks!



Would you please release processor builds? At least SandyBridge I still own :)Sry, cant help ya with that. The suite im using to build with doesn't offer any easy way to do that.

Skickat från min SM-G988B via Tapatalk

[LigH]

Is it even measurable when the core routines are all assembler optimized already?

[tormento]

Is it even measurable when the core routines are all assembler optimized already?
Yes, it is. Only a few percentage points but there is a difference and, on a slow Sandy Bridge, it means some minutes less of encoding.

[_kermit]

I always use CRF. Is 2pass doing something better in that regard?

right, thanks.
looks as if I can't get all cores utilized 100% then
I instead just process two files at once :)

[charliebaby]

For Used max cpu add this --pmode --pme --frame-threads 16 --ctu 16 --qg-size 16 --min-cu-size 16 --max-tu-size 16

:-)

[benwaggoner]

For Used max cpu add this --pmode --pme --frame-threads 16 --ctu 16 --qg-size 16 --min-cu-size 16 --max-tu-size 16

:-)
And if it wasn't clear, this is actually not good advice :sly:!

And if we're really trying to encode 480p on 128 cores to heat the house in the winter, don't forget --slices!

[_kermit]

For Used max cpu add this --pmode --pme --frame-threads 16 --ctu 16 --qg-size 16 --min-cu-size 16 --max-tu-size 16

:-)

also made no difference, around 50% util, except that the file got around 20% larger

[_kermit]

And if it wasn't clear, this is actually not good advice :sly:!

And if we're really trying to encode 480p on 128 cores to heat the house in the winter, don't forget --slices!

with 50% util. it's fast enough anyway.
UHD does all, so I don't lose that much time there either.

[benwaggoner]

also made no difference, around 50% util, except that the file got around 20% larger
Those are all parameters that reduce compression efficiency to increase parallelism. And there's a reason they're not used in --preset veryfast, as lots of those tradoffs are Not Good Deals.

[LigH]

OT: We are on page 403 here with the 20 posts/page default ... I start to worry for a moment when I catch a glimpse of the tab title. :o

[FranceBB]

OT: We are on page 403 here with the 20 posts/page default ... I start to worry for a moment when I catch a glimpse of the tab title. :o

OT2: 2719 days passed ever since the topic was created on the 24th September 2013, with 403 pages and your average of 20 posts per page, not taking into account any outliers, it gives us a total of 8060 replies, which, divided by 2719 gives us an average of 3 posts per day about x265. That's quite alright, I guess. :D


And if we're really trying to encode 480p on 128 cores to heat the house in the winter, don't forget --slices!

ROTFL

Ok, back on topic:

Those are all parameters that reduce compression efficiency to increase parallelism.
[...] Lots of those tradoffs are Not Good Deals.

This is absolutely true, no wonder they're not used by default.

@charliebaby... x265 does a pretty good job in using up multicore/multithread CPUs and even multi-socket configurations thanks to the whole Pool/NUMA implementation, especially with UHD contents (a bit less for lower resolutions), so that's generally quite alright. Of course, there is always a trade-off between parallelism and quality and sometimes it's just not worth it, which is why, whenever I have some spare room in my 40c/80th Xeon Platinum, I run more encodes at once rather than being obsessed in getting CPU usage up in one single encode (also 'cause sometimes you see the CPU go up and up without a real improvement in speed or sometimes you do see an improvement in speed at the expense of quality or compression - i.e bigger file).

(for the records, that's one of the server I have a work, I wish I had such a monster at home hehehe... although I would be kinda happy with something like an old Z840 with a 28c/40th Xeon, to be fair, but that's not gonna happen in the near future... :( )

[LigH]

@FranceBB: I meant the association with HTTP errors 403 (denied) and 404 (not found), so I get worried about forum software errors...

[benwaggoner]

I generally run slower, higher quality encodes (--preset slower and up), and on my Xeon 2x18/36 workstation I rarely get much of a boost from using both sockets below 8K. I generally just use --pools "+,-" and "-,+" to run one encode per socket. That automatically reduces the available threads for calculating defaults that change based on available threads, like --frame-threads.

Latency versus throughput are very different tuning targets.

The newer --abr-ladder presumably improves throughput when encoding the same content multiple times, and thus latency when running them all in parallel.

[ShortKatz]

x265 v3.5_RC1+2-f3f27198 (http://www.mediafire.com/file/94n1ovortwx1vez/x265-3.5_RC1%252B2-f3f27198_Win_GCC102.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.2.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/Release_3.5


Is there some need of macOS binaries? I could provide some. Or are there only Windows users here.

[benwaggoner]

Is there some need of macOS binaries? I could provide some. Or are there only Windows users here.
x265 is certainly integrated into Mac products like Handbreak, so it is of interest.

I'd be quite interested in seeing a native M1 ARM version for Mac as well. My vague understanding today is that x265 has better perf running in x64 emulation than native ARM on M1 due to how good Apple's emulator is, even with AVX2 code. But it'd be great to be able to test head-to-head, and see how perf evolves as the ARM (including M1 and AWS's Graviton) optimizations get added.

A good justification to expense a new MacBook Pro :sly:.

[Ritsuka]

If you build a macOS arm64 binary please include Apple's Neon patch, you can find it on HandBrake's git repository.

[benwaggoner]

If you build a macOS arm64 binary please include Apple's Neon patch, you can find it on HandBrake's git repository.
Any data on the perf delta from the patch? Does it include M1-specific optimizations?

[Ritsuka]

~2x than no-asm. There isn't anything specific to M1.

[ShortKatz]

x265 3.5_RC2 for macOS, Universal binary 2 (x86_64 / arm64)
https://polysom.verilite.de/tmp/x265_3.5_RC2_macOS.zip
sha256 checksum: 60c93733f5d3703f8d7d9e1a9152fc5a84f7d35f4ccb9a1ca55c855c3b9f32a7

x265 3.5_RC2 for macOS, Universal binary 2 (x86_64 / arm64e)
https://polysom.verilite.de/tmp/x265_3.5_RC2_macOS2.zip
sha256 checksum: d46ac744a683db0453f77b856c286a789530ef47caf1f3470fa08279c4d2d956

To install, extract x265_3.5_RC2_macOS.tar.gz. With Terminal cd into the folder "x265_3.5_RC2_macOS" and drag&drop install.command into the Terminal window. Hit Enter.
To check if x265 is properly installed, open Terminal and type:
x265 --version

It should respond "x265 [info]: HEVC encoder version 3.5_RC2".

I had to include Apple's Neon patch, otherwise arm64(e) did not build properly. arm64e adds Pointer authentication, Nested virtualization and Advanced SIMD complex number support. I don't have a M1, so I can't test if the arm64e binary will work.

[ghostshadow]

Hello, I have a problem with my settings since version 3.4 + 26 of x265 (I am currently in 3.5_RC2, windows 10 64bit, PC Ryzen 3900x).
My concern is in increasing my Avg QP, I am almost always above 24, whereas before with version 3.4 + 20 I managed to have my Avg QP below 20.
here are the parameters used for the3.5_RC2 : --scenecut-aware-qp 3 --masking-strength 500,2,0,200,-1,-1 **** I also tested: --scenecut-aware-qp 1 --masking-strength 300,2,2 *** --scenecut-aware-qp 3 --masking-strength 500,5,6.5,100,-1,-1
for the rest of the parameters : --rd 6 --rskip 2 --deblock -3:-3 --cutree --ctu 64 --min-cu-size 8 --bframes 4 --b-adapt 2 --rc-lookahead 60 --ref 4 --limit-refs 2 --merange 57 --subme 7 --aq-mode 4 --aq-strength 0.70 --me 3 --psy-rd 2.10 --psy-rdoq 1.35 --gop-lookahead 54
I tested by increasing my --bframes and reducing my --gop-lookahead, it is a little better but nothing more.
In 3.4 + 20 I had this in parameters: --scenecut-aware-qp --scenecut-window 500 --qp-delta-ref 6 --qp-delta-nonref 7 and my QP was impeccable
Thank you for your help

[Barough]

x265 v3.5+8-57e817329 (https://www.mediafire.com/file/1co8t8zsvfvkbe9/x265-3.5%252B8-57e817329_Win_GCC102.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 10.2.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

[benwaggoner]

And the 3.5 release notes (https://bitbucket.org/multicoreware/x265_git/commits/f0c1022b6be121a753ff02853fbe33da71988656) are in github. With some minor formatting fixes:


Version 3.5
===========

Release date - 16th March, 2021.

New feature
-----------
1. Real-time VBV for ABR (Average BitRate) encodes in --pass 2 using option '--vbv-live-multi-pass': Improves VBV compliance with no significant impact on coding efficiency.

Enhancements to existing features
---------------------------------
1. Improved hist-based scene cut algorithm: Reduces false positives by leveraging motion and scene transition info.
2. Support for RADL pictures at IDR scene cuts: Improves coding efficiency with no significant impact on performance.
3. Bidirectional scene cut aware Frame Quantizer Selection: Saves bits than forward masking with no noticeable perceptual quality difference.

API changes
-----------
1. Additions to x265_param structure to support the newly added features and encoder enhancements.
2. New x265_param options option '--min-vbv-fullness' and option '--max-vbv-fullness' to control min and max VBV fullness.

Bug fixes
---------
1. Incorrect VBV lookahead in option '--analysis-load' option '--scale-factor'.
2. Encoder hang when VBV is used with slices.
3. QP spikes in the row-level VBV rate-control when WPP enabled.
4. Encoder crash in option '--abr-ladder'.

[benwaggoner]

I'm curious about "2. Support for RADL pictures at IDR scene cuts: Improves coding efficiency with no significant impact on performance."

Has anyone tested with this? I found some value in --radl with fixed-GOP CBR encoding some years back, but haven't played with it much since. The description doesn't sound any different than what it was originally released in 2.7.

Although I see in ReadtheDocs (https://x265.readthedocs.io/en/master/cli.html#slice-decision-options)that its use in variable GOP encoding has been added.

--radl <integer>
Number of RADL pictures allowed infront of IDR. Requires closed gop interval. If enabled for fixed keyframe interval, inserts RADL at every IDR. If enabled for closed gop interval, in case of --hist-scenecut inserts RADL at every hard scenecut whereas for the --scenecut, inserts RADL at every scenecut. Recommended value is 2-3. Default 0 (disabled).

**Range of values: Between 0 and –bframes

I had not theorized that RADL would be of general benefit at hard scene cuts before. Anyone have any experience/data on this front?


P.S. MultiCoreWare, remember to update the Release Notes on ReadTheDocs! It looks like all the new parameters have already been added.

[vpupkind]

Hello, I have a problem with my settings since version 3.4 + 26 of x265 (I am currently in 3.5_RC2, windows 10 64bit, PC Ryzen 3900x).
My concern is in increasing my Avg QP, I am almost always above 24, whereas before with version 3.4 + 20 I managed to have my Avg QP below 20.
here are the parameters used for the3.5_RC2 : --scenecut-aware-qp 3 --masking-strength 500,2,0,200,-1,-1 **** I also tested: --scenecut-aware-qp 1 --masking-strength 300,2,2 *** --scenecut-aware-qp 3 --masking-strength 500,5,6.5,100,-1,-1
for the rest of the parameters : --rd 6 --rskip 2 --deblock -3:-3 --cutree --ctu 64 --min-cu-size 8 --bframes 4 --b-adapt 2 --rc-lookahead 60 --ref 4 --limit-refs 2 --merange 57 --subme 7 --aq-mode 4 --aq-strength 0.70 --me 3 --psy-rd 2.10 --psy-rdoq 1.35 --gop-lookahead 54
I tested by increasing my --bframes and reducing my --gop-lookahead, it is a little better but nothing more.
In 3.4 + 20 I had this in parameters: --scenecut-aware-qp --scenecut-window 500 --qp-delta-ref 6 --qp-delta-nonref 7 and my QP was impeccable
Thank you for your help

Have you tried using just --scenecut-aware-qp 3 --masking-strength 500,-1,-1 ?

Also, what happens to quality?

[ghostshadow]

Have you tried using just --scenecut-aware-qp 3 --masking-strength 500,-1,-1 ?

Also, what happens to quality?

The quality is very good on my zidoo. But also before.
I don't think I tested the --scenecut-aware-qp 3 with only fwdWindow, fwdRefQPDelta and fwdNonRefQPDelta.
Because in this case the much to be in --scenecut-aware-qp 1? No ?

[LigH]

New upload: x265 3.5+8-57e817329 (https://www.mediafire.com/file/fd739fjr6fnufxb/x265_3.5+8-57e817329.7z/file)

Bugs not yet fixed:

NASM 2.15.05 multi-line macro warnings (reported August 2020)

[filler56789]

New upload: x265 3.5+8-57e817329 (https://www.mediafire.com/file/fd739fjr6fnufxb/x265_3.5+8-57e817329.7z/file)

:thanks:

Bugs not yet fixed:

NASM 2.15.05 multi-line macro warnings (reported August 2020)


Can't you somehow force MABS to use an older version of nasm? :confused:

[LigH]

I see no reason to downgrade. Despite 9 MB of warnings scrolling through, it still works correctly. MABS wants to keep MSYS2 up-to-date, that's sensible. A fix similar to the one in x264 has been proposed to the x265 team but got denied. As a workaround, I wonder if there is an easy way to disable legacy warnings for NASM by adding the CLI parameter mentioned in the warning.

[tormento]

x265 3.5+8-57e817329
The number after "+", i.e. +8, is the number of applied patches?

[Boulder]

The number after "+", i.e. +8, is the number of applied patches?

I think it's the number of commits in that particular release branch since the initial release.