x265 HEVC Encoder

[benwaggoner]

yeah i even tried Ultimate Performance plan, it doesn't change anything
Are the different core types flagged as different NUMA nodes? If so, --pools could be used to pin specifically to the performance cores. I think that may work on AMD, as the big processors have a lot of nodes. Not Intel AFAIK.

[Atak_Snajpera]

Are the different core types flagged as different NUMA nodes? If so, --pools could be used to pin specifically to the performance cores. I think that may work on AMD, as the big processors have a lot of nodes. Not Intel AFAIK.

Only on Windows 7 P-Cores and E-Cores are separated by different NUMA nodes. Basically win7 treats that cpu as "dual-socket". On Win10/11 all cores sit on the same NUMA node.

[guest]

I am trying to get x265 to process a DV clip, and retain the appropriate metadata, or whatever it is.

If I encode a DV clip with "standard" commands I get this output, I've just found out that this is a "Hybrid" and it plays as HDR10:-

This is MediaInfo info.

Dolby Vision, Version 1.0, dvhe.08.06, BL+RPU, HDR10 compatible / SMPTE ST 2086, HDR10 compatible


But I downloaded an LG DV test clip, and it's x265 info is, and it plays as DV :-

Dolby Vision, Version 1.0, dvhe.04.06, BL+EL+RPU

How do I remove the HDR10 compatible / SMPTE ST 2086, HDR10 compatible ??

[rwill]

This is not a Hybrid, this is a Profile 8 Dolby Vision which is HDR10 backwards compatible.

If you want to drop HDR10 compatibility you can try to not insert the mastering display SEI and content light level SEI, then just set the the VUI video information of primaries, transfer and matrix to unknown. This way a playback device will not know how to display the Baselayer correctly at all.

I have my doubts that the resulting DV file will be correct and be conforming to any valid DV Profile though.

[guest]

This is not a Hybrid, this is a Profile 8 Dolby Vision which is HDR10 backwards compatible.

If you want to drop HDR10 compatibility you can try to not insert the mastering display SEI and content light level SEI, then just set the the VUI video information of primaries, transfer and matrix to unknown. This way a playback device will not know how to display the Baselayer correctly at all.

I have my doubts that the resulting DV file will be correct and be conforming to any valid DV Profile though.

Thanks for the info, but that's about as clear as thick mud, to me :(

[rwill]

Thanks for the info, but that's about as clear as thick mud, to me :(

Well then, what is your x265 command line for encoding?

[guest]

Well then, what is your x265 command line for encoding?

I think it would be easier for me to just send you over to have a look at what I've been posting on the RipBot thread with quite a few of the command lines that I've been trying...

https://forum.doom9.org/showthread.php?p=1987445#post1987445

[rwill]

Hmm, what about "--no-hdr10 --videoformat unknown --range limited --colorprim unknown --transfer unknown --colormatrix unknown" and removing "--master-display" and "--max-cll" ?

[guest]

Hmm, what about "--no-hdr10 --videoformat unknown --range limited --colorprim unknown --transfer unknown --colormatrix unknown" and removing "--master-display" and "--max-cll" ?

So from what I understand, what you're saying here is I need to add that to the command line and remove those others ?

OK, I have just checked what these commands to @ x265 read the docs, so I will very interested in seeing what it does, but it won't be until tomorrow now (my time).

:thanks: I will report back later :)

UPDATE:-

Hi, I tried all those commands, and it still doesn't do what I think it should.

How do I remove the "--master-display" and "--max-cll" (this is info from the clip I'm testing with)

--master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)" --max-cll "738,287"

[rwill]

So from what I understand, what you're saying here is I need to add that to the command line and remove those others ?

Yes.

You also might want to pass off your DV RPU as "--dolby-vision-profile 5" to x265 instead of 8.1, as Profile 8.1 is by definition HDR10 backwards compatible. Profile 5 is not.

[tuanden0]

I have 8 cores CPU (Ryzen 7 5800X) but when I encode, it never used full 100% CPU. So I tried to run 2 x265 instances at same time, but I am confuse how to use this options: --pools <string>, --numa-pools <string>

Can someone enlighten me.

Should I use --pools "+" or something.

Thanks

[Selur]

https://x265.readthedocs.io/en/master/threading.html?thread-pools
https://x265.readthedocs.io/en/master/cli.html?highlight=--pools#cmdoption-pools

Instead of messing with pools, try using "--pmode --pme" at least with HD and higher resolution content that helps here,...

[rwill]

I have 8 cores CPU (Ryzen 7 5800X) but when I encode, it never used full 100% CPU. So I tried to run 2 x265 instances at same time, but I am confuse how to use this options: --pools <string>, --numa-pools <string>

Can someone enlighten me.

Should I use --pools "+" or something.

Thanks

Well just don't use --pools or --numa-pools. Just start your two instances and let the Operating System scheduler figure out how to distribute the threads.

And don't use --pmode or --pme, they will increase CPU usage more than they will speed up things.

[tuanden0]

https://x265.readthedocs.io/en/master/threading.html?thread-pools
https://x265.readthedocs.io/en/master/cli.html?highlight=--pools#cmdoption-pools

Instead of messing with pools, try using "--pmode --pme" at least with HD and higher resolution content that helps here,...

Well just don't use --pools or --numa-pools. Just start your two instances and let the Operating System scheduler figure out how to distribute the threads.

And don't use --pmode or --pme, they will increase CPU usage more than they will speed up things.

Thank you for help :goodpost:

I will not use --pools and let the OS do it.

[benwaggoner]

https://x265.readthedocs.io/en/master/threading.html?thread-pools
https://x265.readthedocs.io/en/master/cli.html?highlight=--pools#cmdoption-pools

Instead of messing with pools, try using "--pmode --pme" at least with HD and higher resolution content that helps here,...
--pmode could well help, but unless it <480p, --pme would most likely slow things down.

[Atak_Snajpera]

Solution is Simple. --ctu 16 for SD , --ctu 32 for HD/FHD and --ctu 64 for UHD.

[Boulder]

I tried encoding one track from a Dolby UHD test disc to see if I can make Dolby Vision work on my TV and to test if it falls back to HDR10 when played in Kodi. I extracted the RPU using the demuxing script here in the forum and fed it to the encoder.

With
c:\x265\x265.exe --input inputfile.avs --input-depth 16 --dither --sar 1:1 --profile main10 --rc-lookahead 40 --min-keyint 5 --keyint 480 --splitrd-skip --colorprim 9 --transfer 16 --colormatrix 9 --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50)" --max-cll "1000,355" --frames 3576 --chromaloc 2 --hdr10 --hdr10-opt --repeat-headers --dolby-vision-profile 8.1 --dolby-vision-rpu "rpu.bin" --preset slower --crf 16 --output "dolby_test.hevc"I got these errors:
Dolby Vision requires VBV settings to enable HRD.
Dolby Vision RPU count is greater than frame count in x265.

Well, I checked and the RPU is ok.

Adding --vbv-bufsize 999999999 --vbv-maxrate 999999999, I got the encode going so the complaint about the RPU file is a bug. The encode ran at Level 8.5 :scared:

My question is: is VBV really necessary? Trying to add --no-hrd didn't do any good. I was thinking of patching the thing to skip that check if it's not needed. I know that I don't need it for my use and have never set it.

I could encode normally and just inject the RPU afterwards, but it would be easier and less diskspace consuming to do everything in one go.

[Rousseau]

What happened to the "Consecutive B-Frames" stats in the encode log? I just upgraded from Patman's 37 build to JPSDR's 103 build and it's gone. I tried john33's and djatom's binaries with the same result.

[guest]

I tried encoding one track from a Dolby UHD test disc to see if I can make Dolby Vision work on my TV and to test if it falls back to HDR10 when played in Kodi. I extracted the RPU using the demuxing script here in the forum and fed it to the encoder.

With
c:\x265\x265.exe --input inputfile.avs --input-depth 16 --dither --sar 1:1 --profile main10 --rc-lookahead 40 --min-keyint 5 --keyint 480 --splitrd-skip --colorprim 9 --transfer 16 --colormatrix 9
--master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(40000000,50)" --max-cll "1000,355" --frames 3576 --chromaloc 2 --hdr10 --hdr10-opt --repeat-headers --dolby-vision-profile 8.1
--dolby-vision-rpu "rpu.bin" --preset slower --crf 16 --output "dolby_test.hevc"I got these errors:
Dolby Vision requires VBV settings to enable HRD.
Dolby Vision RPU count is greater than frame count in x265.

Well, I checked and the RPU is ok.

Adding --vbv-bufsize 999999999 --vbv-maxrate 999999999, I got the encode going so the complaint about the RPU file is a bug. The encode ran at Level 8.5 :scared:

My question is: is VBV really necessary? Trying to add --no-hrd didn't do any good. I was thinking of patching the thing to skip that check if it's not needed. I know that I don't need it for my use and have never set it.

I could encode normally and just inject the RPU afterwards, but it would be easier and less diskspace consuming to do everything in one go.

Don't you have to provide the full path to the rpu.bin ???

[Boulder]

What happened to the "Consecutive B-Frames" stats in the encode log? I just upgraded from Patman's 37 build to JPSDR's 103 build and it's gone. I tried john33's and djatom's binaries with the same result.

It was lost at a recent commit for some reason.. IIRC quietvoid created a patch to get it back though.

Don't you have to provide the full path to the rpu.binNot needed if the file is in the folder where I launch the command line in. The same for "inputfile.avs".

[quietvoid]

What happened to the "Consecutive B-Frames" stats in the encode log? I just upgraded from Patman's 37 build to JPSDR's 103 build and it's gone. I tried john33's and djatom's binaries with the same result.

As said above, it was removed.
The code can be added back with this commit: https://github.com/quietvoid/x265/commit/43e460330af001365c8508d430a35be4377e390e

[benwaggoner]

My question is: is VBV really necessary? Trying to add --no-hrd didn't do any good. I was thinking of patching the thing to skip that check if it's not needed. I know that I don't need it for my use and have never set it.
It seems like VBV is important in your case ;)!

I think it's always a good idea to set VBV to ensure compatibility with a given decoder. Just use --level-idc 5.1 and it will auto set reasonable values broadly compatible with UHD BD, Smart TVs, HW GPU decode, etc. 5.1 has a 40 Mbps peak, which is fine for 2160p24 content.

[jpsdr]

I checked the commit to put back "Consecutive B-Frames" in my moded version, unfortunately, there is big changes, and i don't realy know where and how put the histogram increment count...

[madey83]

As said above, it was removed.
The code can be added back with this commit: https://github.com/quietvoid/x265/commit/43e460330af001365c8508d430a35be4377e390e

Have you checked new updated Patman mode?

https://github.com/Patman86/x265-Mod-by-Patman/releases

[LigH]

New uploads: x265 3.5+103-8f18e3ad3 (https://www.mediafire.com/file/r7g2b57iyqd6l0c/x265_3.5+103-8f18e3ad3.7z/file) / x265 3.5+104-47c18bf12 (https://www.mediafire.com/file/l4lmmx6rwqxskpj/x265_3.5+104-47c18bf12.7z/file) (might be identical, patch counting mistake or different branches?)

[Windows][GCC 13.1.0][32/32XP/64 bit] 8bit+10bit+12bit

[Boulder]

I have a simple question regarding frametypes and chapter marks when using a qpfile. Does a chapter mark need an I-frame or is an i-frame enough? I'm using open GOP so setting 'K' in the qpfile would place i-frames in the encode.

[LigH]

An IDR frame (Intra frame with Decoder Reset) is preferable. When you jump to an intermediate I frame which is not IDR (e.g. to encode a single frame very different from the surrounding video), you will get decoding artifacts from predictions skipping this I frame missing a reference.

[LeXXuz]

Oh darn. Thanks for mentioning that. Now I know why my players always jump a little before or after the specified chapter point. MeGUI sets 'K' as default for keyframes when creating qpfiles. I guess I'll better change those manually to 'I' from now on. :mad:

[benwaggoner]

Oh darn. Thanks for mentioning that. Now I know why my players always jump a little before or after the specified chapter point. MeGUI sets 'K' as default for keyframes when creating qpfiles. I guess I'll better change those manually to 'I' from now on. :mad:
Yeah, only IDR frames are guaranteed to be independently decodable.

[jpsdr]

Hello.
I've made a new build, and also include some new Patman updates.
As usual, check my github.

[sainome]

Solution is Simple. --ctu 16 for SD , --ctu 32 for HD/FHD and --ctu 64 for UHD.

Question: If I specify --ctu 16, what should I specify for merange?

[benwaggoner]

Question: If I specify --ctu 16, what should I specify for merange?
They are orthogonal values, without much connection between optimum values. I'd stick with the one your --preset gives you unless you want to do a bunch of experimentation.

With --ctu 16 I suppose you could try --merange 41, which is 16 below the typical default of 57. It'd be a little faster, and depending on content might not raise bitrate or harm quality materially.

[sainome]

Are you saying that there is no ideal optimal value?

--merange <integer>
Motion search range. Default 57

The default is derived from the default CTU size (64) minus the luma interpolation half-length (4) minus maximum subpel distance (2) minus one extra pixel just in case the hex search method is used. If the search range were any larger than this, another CTU row of latency would be required for reference frames.

x265 docment says it like this, so if you specify "--ctu32", specify 25

[N'Cha]

Hello,
What is the impact (quality of the output, compression efficiency, easiness to decode the encoded video or the 3 maybe?) of using --limit-refs 3 vs --limit-refs 1 and --subme 5 vs --subme 7 ?
I've noticed that using limit-refs 1 makes my encoding take 20-25 minutes longer for a 24 minute anime episode (1h20 vs roughly 56 minutes with limi-tefs 3). Same with --subme 7, it has a lesser impact but still like 3-4 minutes on the encoding and not sure it's worth it (if it gives like 10% more quality/details vs --subme 5 it is worth it)

[LeXXuz]

Are you saying that there is no ideal optimal value?



x265 docment says it like this, so if you specify "--ctu32", specify 25

Interesting. Thought 'default' merange gets adjusted automatically by the encoder to a lower value when ctu32 or ctu16 is specified. :confused:

[benwaggoner]

Are you saying that there is no ideal optimal value?

x265 docment says it like this, so if you specify "--ctu32", specify 25
Limiting it to 25 would improve parallelization, but absolutely could reduce compression efficiency. Lowering CTU makes more WPP parallelism possible, but doesn't mitigate the potential quality hit from a too-small search range.

[sainome]

Sorry. I'm sorry for the afterthought of the information, but the source is SD size (720x480).
It seems that --cti 16 gives better results than --ctu 64/32 for SD size.

I have tried --merange values of 57/25/16, but I can't decide between them, so I wanted to ask if there is a mathematically/theoretically correct setting.

[microchip8]

Sorry. I'm sorry for the afterthought of the information, but the source is SD size (720x480).
It seems that --cti 16 gives better results than --ctu 64/32 for SD size.

I have tried --merange values of 57/25/16, but I can't decide between them, so I wanted to ask if there is a mathematically/theoretically correct setting.

If it's SD, then lower CTU is definitely better and recommended. As for me-range, I doubt you'll see the difference.

[benwaggoner]

Are you saying that there is no ideal optimal value?
Parameters for which there is an ideal optimal value just have those values hard-coded ;). The parameters you can set are ones that you may need different values for sometimes.

[benwaggoner]

Sorry. I'm sorry for the afterthought of the information, but the source is SD size (720x480).
It seems that --cti 16 gives better results than --ctu 64/32 for SD size.
Better in what sense?

I have tried --merange values of 57/25/16, but I can't decide between them, so I wanted to ask if there is a mathematically/theoretically correct setting.
Not really. If it is too small, it'll miss some motion matches that would be more efficient. If it's too big, it'll waste encoding time without making quality or bitrate significantly better.

57 is really a fine default unless there's a big need for more parallelism.

[LeXXuz]

I could use some pro advice about setting aq-strength. Again. :o
After reading some posts and comments, I can't decide if I should use 1.0 or 1.5 as a 'default' setting for my encodes.

I use jp's patman mod with a custom very-slow preset @ CRF17 for 1080p sources. My sources are mostly quite clean as I denoise them with a complex mv-based denoiser script before encoding.

my settings look like these:

--preset veryslow --crf 17.0 --qpfile "D:\WORK\test\chp.qpf" --repeat-headers --input-depth 16 --output-depth 10 --dither
--profile main10 --high-tier --level-idc 4.1 --vbv-bufsize 50000 --vbv-maxrate 50000 --ctu 32 --limit-refs 1 --rd 6 --aq-mode 5 --aq-auto 10
--aq-strength 1.5 --psy-rd 2 --psy-rdoq 4 --rskip 2 --rskip-edge-threshold 2 --no-sao --no-strong-intra-smoothing --deblock -1:-1
--colorprim bt709 --transfer bt709 --colormatrix bt709
(I know some settings are obsolete as they are already part of the preset. But I always like to set them manually as mental reminder what is used with said preset.)

With aq-strength 1.5 file sizes increase, depending on source. So more bits are hopefully spend where it matters.

But is there a downside to a higher aq-strength I should keep in mind (besides bigger file sizes)?

[benwaggoner]

I could use some pro advice about setting aq-strength. Again. :o
After reading some posts and comments, I can't decide if I should use 1.0 or 1.5 as a 'default' setting for my encodes.

I use jp's patman mod with a custom very-slow preset @ CRF17 for 1080p sources. My sources are mostly quite clean as I denoise them with a complex mv-based denoiser script before encoding.

my settings look like these:

--preset veryslow --crf 17.0 --qpfile "D:\WORK\test\chp.qpf" --repeat-headers --input-depth 16 --output-depth 10 --dither
--profile main10 --high-tier --level-idc 4.1 --vbv-bufsize 50000 --vbv-maxrate 50000 --ctu 32 --limit-refs 1 --rd 6 --aq-mode 5 --aq-auto 10
--aq-strength 1.5 --psy-rd 2 --psy-rdoq 4 --rskip 2 --rskip-edge-threshold 2 --no-sao --no-strong-intra-smoothing --deblock -1:-1
--colorprim bt709 --transfer bt709 --colormatrix bt709
(I know some settings are obsolete as they are already part of the preset. But I always like to set them manually as mental reminder what is used with said preset.)

With aq-strength 1.5 file sizes increase, depending on source. So more bits are hopefully spend where it matters.

But is there a downside to a higher aq-strength I should keep in mind (besides bigger file sizes)?
I've not used --aq-mode 5 myself, as it isn't in x265 mainline. The optimal --aq-strength can vary a lot depending on algorithm. For example --aq-mode 4 often has an optimal strength .1-.2 lower than for 1 or 2. That may be true for this mode 5 as well.

In a general sense, 1.0 is a rational default. I'd use a bit lower if I know the content is cel animation, a little bit higher if I knew it was professionally produced film or video. 1.5 isn't crazy high or anything. With high tier crf=17, I would expect the impact of AQ could be more in file size than perceptual quality. There aren't any good public metrics for AQ modes; PSNR and SSIM often show lower scores for better subjective quality, and VMAF is pretty blind to even pretty visible AQ impacts.

Finding your optimal value is really down to trying a few variants and picking what you like best with your eyeballs. Since you're using 1.5 already, trying 1.0 and 2.0 could give you a sense of the impact of different values in subjective quality and file size.

[LeXXuz]

Thanks Ben for your valued input. :)

[benwaggoner]

So, now that Apple is resuscitating stereoscopic 3D video, has anyone done any work getting MV-HEVC working with x265? It's basically left eye as a normal 2D stream and a right eye encoded using the left eye as reference, so not that theoretically hard to extend.

https://developer.apple.com/videos/play/wwdc2023/10071/?time=320

[Rainforrest]

Hi,
Could smbd please share an example of using ZONES (e.g. to assign different CRF) for x265 encoding with FFMPEG? (as far as I know it may be via x265-params)

If no FFMPEG example, same example for direct command line x265 encoder would be helpful also, hopefully syntax isn't much different.
Thank you.

[benwaggoner]

Hi,
Could smbd please share an example of using ZONES (e.g. to assign different CRF) for x265 encoding with FFMPEG? (as far as I know it may be via x265-params)

If no FFMPEG example, same example for direct command line x265 encoder would be helpful also, hopefully syntax isn't much different.
I've not used --zones myself that I can recall. I've used --qpfile for similar purposes.
The documentation lists --zonefile as a CLI-only feature; I don't know that it is implemented in ffmpeg.

Also, CRF isn't one of the parameters that can be specified. Only QP or a bitrate multiplier are listed.

From: https://x265.readthedocs.io/en/3.4/cli.html#quality-rate-control-and-rate-distortion-options
--zones <zone0>/<zone1>/...
Tweak the bitrate of regions of the video. Each zone takes the form:

<start frame>,<end frame>,<option> where <option> is either q=<integer> (force QP) or b=<float> (bitrate multiplier).

If zones overlap, whichever comes later in the list takes precedence. Default none

--zonefile <filename>
Specify a text file which contains the boundaries of the zones where each of zones are configurable. The format of each line is:

<frame number> <options to be configured>

The frame number indicates the beginning of a zone. The options following this is applied until another zone begins. The reconfigurable options can be spcified as –<feature name> <feature value>

CLI ONLY

[maroders2]

Hello,
Please tell me the optimal settings for encoding movies (only 1080p BDRemux to h.265 10bit). I want to get good quality with a normal encoding speed. Avg QP should not be higher than 22!
I currently use the following settings:
--crf 20 --preset slower --output-depth 10 --profile main10 --level-idc 4.1 --rd 4 --dynamic-rd 2 --limit-refs 3 --ctu 32 --limit-tu 1 --refine-mv 1 --no-b-intra --aq-mode 2 --vbv-bufsize 50000 --vbv-maxrate 50000 --no-cutree --subme 5 --bframes 16 --colorprim bt709 --colormatrix bt709 --transfer bt709 --range limited --sar 1:1 --deblock -3:-3 --no-sao --no-strong-intra-smoothing
Are these good settings or change something?

What settings can boost speed encoding with minimal quality loss?

Thanks!

[RanmaCanada]

Hello,
Please tell me the optimal settings for encoding movies (only 1080p BDRemux to h.265 10bit). I want to get good quality with a normal encoding speed. Avg QP should not be higher than 22!
I currently use the following settings:

Are these good settings or change something?

What settings can boost speed encoding with minimal quality loss?

Thanks!

At preset slower, the only way to increase speed is to buy new hardware. Settings would maybe get you a fraction of a frame increase in speed. Real speed increases can only be done by sheer horsepower.

What is your current processor?

[benwaggoner]

Hello,
Please tell me the optimal settings for encoding movies (only 1080p BDRemux to h.265 10bit). I want to get good quality with a normal encoding speed. Avg QP should not be higher than 22!
I currently use the following settings:

Are these good settings or change something?

What settings can boost speed encoding with minimal quality loss?
You're setting a lot of things you probably don't want to be setting. --bframes 16 will slow things down a lot for little gain. --deblock -3:-3 is going to raise bitrate and thus lower speed for little or no benefit. If the source is 8-bit, encoding at 10-bit gives a speed hit without much quality gain (unless you're doing some >8-bit preprocessing on the source). 1080p24 isn't going to hit 50 Mbps.

I'd start with just --preset slower --level-idc 4.1 --ctu 32 --crf 20 --selective-sao 2 and see how it goes quality and speed wise. If it's way too slow, try --preset slow. If it doesn't look good enough, try --crf 18. It looks like you're way overrunning for detail, which you shouldn't start with. That's something to do only if there's visible detail loss during playback. Don't sweat how a zoomed in screen shot looks unless you're going to be watching the video as a zoomed-in screen shots.

I don't know what a "normal" encoding speed is to you, but --preset slower with the settings I give above approaches real-time on a recent desktop processor with 12+ cores. If you are doing preprocessing along with encoding, check that preprocessing isn't using up lots of your CPU, or is serialized in a way that gates x265 performance. A good share of "why is my encoding so slow" problems have little to do with the encoder.

[benwaggoner]

At preset slower, the only way to increase speed is to buy new hardware. Settings would maybe get you a fraction of a frame increase in speed. Real speed increases can only be done by sheer horsepower.
Oh, going from --preset slower to --preset fast will speed up encoding by a pretty good multiple, if encoding speed is the limiting factor. I wouldn't do it myself other than for scratch quality or when I didn't care about bitrate, but there are absolutely ways to get huge speed increases through settings. Just not free ways in terms of quality or bitrate.

Edit: --selective-sao 2 is effectively free. And --ctu 32 for 1080p and below is a nice boost if you have the cores to take advantage of the extra WPP row of parallelism.