x265 HEVC Encoder

[excellentswordfight]

x265 is not optimized for speed/multi-threading and AMD CPUs.

What are the basis for that statement? The lower clocked 3700X is on par or faster then 9900k at the same amount of threads. First and second gen wasnt that strong per thread, but that is much cause of avx speed and other cpu-design causes. And I have recieved several Epyc 7402p servers, no issue there either, blazing fast.

[redbtn]

I've also noticed AQ2 seems to have problems in HDR encodes (3.1 & 3.2 builds I've tested), leaving a lot of blotchy grain/puddling in flat areas of some scenes. AQ1 does much better. It's not that visible when you look at it without tone mapping, but in MPV it's very obvious. I tried AQ3 & 4 with no improvement.
I always use AQ2 for HDR encodes, and I never thought about it, cuz somewhere here I read that AQ2 is optimal for HDR. But now, it seems like I should make some tests.
How much bigger bitrate you get with AQ1 vs AQ2?

[mandarinka]

What are the basis for that statement? The lower clocked 3700X is on par or faster then 9900k at the same amount of threads. First and second gen wasnt that strong per thread, but that is much cause of avx speed and other cpu-design causes. And I have recieved several Epyc 7402p servers, no issue there either, blazing fast.

I think what he meant is that results with Beamr can't be taken as something indicating how fast will x265 run.

[mini-moose]

How much bigger bitrate you get with AQ1 vs AQ2?

I think that's because in AQ1 the aq-strength for all frames is the same, while AQ2 uses auto-variance, and x265 applies separate aq-strength for each frame (based on complexity).

At least on paper AQ2 sounds better to me and it is now the default.

One would hope the devs switched to that for a good reason (i.e it's not just slightly faster but also gives a better result).

[rco133]

There seems to be a lot of discussion about AQ mode 1 vs 2.

When it comes to 4k HDR encodes what I found is that with mode 2 the Average QP result is in average 1 higher in mode 2.

For example mode 1 is 21.33 and mode 2 is 22.15.

I have measured the vmaf values of quite a few 4k HDR encode cuts, and not in one single instance has the vmaf score been worse on the aq mode 2 encode. Every single time the vmaf value is higher on the aq mode 2 encode.

Yes, the file size is smaller, and the average QP a bit higher.

The switch from aq mode 1 to aq mode 2 must have been made for a reason. If mode 2 gives worse quality than mode 1, why would the devs change the default.

Either the devs knows nothing about qhat they are doing, or the actually know what they are doing.

All I can say is that I am yet to find a mode 1 encode with a higher vmaf score than the identical mode 2 encode.

rco133

[Boulder]

But have you made any visual comparisons? In my old tests, I found out that mode 2 causes problems in flat backgrounds.

Many of the changes in the default values have been made without any explanation, so we don't really know what the use case for each of them has been. I sometimes feel that many settings are based on squeezing everything out of the material with the least possible amount of bits, even if it means oversmoothing etc. which can then be irritating to others (like me).

[redbtn]

But have you made any visual comparisons? In my old tests, I found out that mode 2 causes problems in flat backgrounds.

Many of the changes in the default values have been made without any explanation, so we don't really know what the use case for each of them has been. I sometimes feel that many settings are based on squeezing everything out of the material with the least possible amount of bits, even if it means oversmoothing etc. which can then be irritating to others (like me).Can I ask what settings do you use for 4k HDR encodes? I mean like --aq-strength --qcomp and CFR. I would like to know for example aq-strength is depends of bitrate or not. Cuz with low bitrate video may be very blocky and requires high aq-strength, but what about for bitrate ~ 20-22mb?
I usually use aq-strength 0.8 and I'm thinking should I raise it to 0.9 or even 1.0?

[Boulder]

Can I ask what settings do you use for 4k HDR encodes? I mean like --aq-strength --qcomp and CFR. I would like to know for example aq-strength is depends of bitrate or not. Cuz with low bitrate video may be very blocky and requires high aq-strength, but what about for bitrate ~ 20-22mb?
I usually use aq-strength 0.8 and I'm thinking should I raise it to 0.9 or even 1.0?

It's been quite a long time since I did any HDR encodes, but I would start with aq-mode 1 with the default strength 1.0. I use qcomp 0.7 and HDR requires a much lower CRF. My base for SDR HD is CRF 18 so I would use CRF 13 for HDR.

The encoder has probably gone through many changes so I would need to test things before doing any HDR encodes. But those would be a good starting point.

[redbtn]

It's been quite a long time since I did any HDR encodes, but I would start with aq-mode 1 with the default strength 1.0. I use qcomp 0.7 and HDR requires a much lower CRF. My base for SDR HD is CRF 18 so I would use CRF 13 for HDR.

The encoder has probably gone through many changes so I would need to test things before doing any HDR encodes. But those would be a good starting point.Thx for your reply! For 1080p HDR I use crf 10-13, but for 2160p HDR crf 13 is too low. With crf 20 I usually get 17-18mb and for grainy films I get 22-24mb.
I'll try to use strength 1.0, but I did some tests and it raise bitrate by 10-11%.

[Boulder]

Yes, CRF 13 is probably quite low for UHD because any problems arising from too few bits will not be as apparent on a 4K TV as with 1080p and lower where they are enlarged during playback.
With aq-strength, you could probably go down to 0.8 or so with grainy sources as grain tends to attract bits anyway, so the flat parts of the image will not look bad.

[benwaggoner]

From different point of view, you can say HEVC is the present, you can say it's not. For example, online streaming still uses AVC (some like Youtube may be using VP9 but still), we still buy Blu-rays instead of UHDs, majority of the content distributors are still using AVC. The use of HEVC would still be quite limited so far, even if assuming HEVC can be encoded at a faster speed. I admit there are other factors like royalty, but still.
There are tons of devices where HEVC is the only supported 10-bit codec. I'd expect pretty much all streaming of 4K or HDR content to those devices to be in HEVC. I don't know of anyone doing HDR or >1080p in H.264.

It'd probably take something like Wireshark to figure out what is being used in other cases. In the adaptive streaming world, there can be dozens of encoded variants of a given title, in different frame sizes, bitrates, DRM, and codecs.

As a wild personal guess, I'd expect a lot more premium content is delivered in HEVC than in VP9+AV1. The On2 stuff has always been most focused on browser-based user generated content without DRM, and there aren't any AV1 HW DRM + decode products in the market, or even announced.

As for quality versus speed, At the same encoding speed, x265 beats x264 in the cases I've tested in the last couple of years. Sure, it doesn't take full advantage of what HEVC can do, but it's still better for most content (there are likely edge cases with a lot of grain). The fastest possible x264 will be faster than the fastest possible x265, of course. x265 and HEVC in general takes better advantage of AVX, multithreading, and 64-bit than x264, so the newer the processor, the better quality @ perf HEVC has.

[sonnati]

There are tons of devices where HEVC is the only supported 10-bit codec. I'd expect pretty much all streaming of 4K or HDR content to those devices to be in HEVC. I don't know of anyone doing HDR or >1080p in H.264.

It'd probably take something like Wireshark to figure out what is being used in other cases. In the adaptive streaming world, there can be dozens of encoded variants of a given title, in different frame sizes, bitrates, DRM, and codecs.

As a wild personal guess, I'd expect a lot more premium content is delivered in HEVC than in VP9+AV1. The On2 stuff has always been most focused on browser-based user generated content without DRM, and there aren't any AV1 HW DRM + decode products in the market, or even announced.

As for quality versus speed, At the same encoding speed, x265 beats x264 in the cases I've tested in the last couple of years. Sure, it doesn't take full advantage of what HEVC can do, but it's still better for most content (there are likely edge cases with a lot of grain). The fastest possible x264 will be faster than the fastest possible x265, of course. x265 and HEVC in general takes better advantage of AVX, multithreading, and 64-bit than x264, so the newer the processor, the better quality @ perf HEVC has.

I would be no surprised if Netflix delivered H.265 to connectedTV even when content is below 4K. This would increase the market share of H.265 considerably because vast majority of Netflix's traffic is on TV and a wide % of those TV has H.265 decoding capabilities. Add to this the 100s millions of iOS devices supporting H.265 (with possible use by Netflix) and the real share should be higher than what can be derived classically from the share seen by cloud encoders vendors or similar.

[Barough]

x265 v3.2+7-37648fca915b (https://www.mediafire.com/file/l9y4j41a6uin5dk/x265-3.2+7-37648fca915b_Win_GCC920.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 9.2.0)
https://bitbucket.org/multicoreware/x265/commits/branch/default

[stax76]

x265 v3.2+7-37648fca915b (https://www.mediafire.com/file/l9y4j41a6uin5dk/x265-3.2+7-37648fca915b_Win_GCC920.7z/file) (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 9.2.0)
https://bitbucket.org/multicoreware/x265/commits/branch/default

Thanks for the build. Why is it that large? 20 MB is too much for staxrip as it consumes 550 MB disk space and 200 MB download space, build by Patman is 2 MB.

[Barough]

Thanks for the build. Why is it that large? 20 MB is too much for staxrip as it consumes 550 MB disk space and 200 MB download space, build by Patman is 2 MB.I guess that Patman use some kind of compression on his EXE.

Skickat från min SM-G975F via Tapatalk

[Atak_Snajpera]

Thanks for the build. Why is it that large? 20 MB is too much for staxrip as it consumes 550 MB disk space and 200 MB download space, build by Patman is 2 MB.

UPX is your friend.
https://upx.github.io/

[stax76]

UPX is your friend.
https://upx.github.io/

I'll use that if it's easy enough, thanks.

[Atak_Snajpera]

I'll use that if it's easy enough, thanks.

Basically drag and drop ;)

[stax76]

That's easy. I use context menu.

https://github.com/stax76/OpenWithPlusPlus

[LigH]

^ Nice extension.

[brumsky]

I was wondering where does HME currently stand? Use it, don't use?

Is it good for anime or film?

I experimented with it a bit a few months ago when it came out. It seems to take longer and I can't really tell a differences in quality. At best I was able to get a file 2.2% smaller but it was around a 50% increase in encoding time.

[stax76]

My newest GUI is for MediaInfo which supports now a presentation for encoder parameters, maybe it's useful for somebody here.

https://forum.doom9.org/showthread.php?t=176886

https://i.postimg.cc/0Q3RNTMD/image.png

[microchip8]

I was wondering where does HME currently stand? Use it, don't use?

Is it good for anime or film?

I experimented with it a bit a few months ago when it came out. It seems to take longer and I can't really tell a differences in quality. At best I was able to get a file 2.2% smaller but it was around a 50% increase in encoding time.

HME shows good benefits at high resolutions. For FHD and lower, the benefit is much smaller and as you say, it has a performance hit. I only use it for anything higher that FHD resolution

[brumsky]

HME shows good benefits at high resolutions. For FHD and lower, the benefit is much smaller and as you say, it has a performance hit. I only use it for anything higher that FHD resolution

Well that's good to know! haha I've been mostly doing UHD encodes. My tests are normally done with 1080p content though... Hmmm I'll have to find a good 4K test clip.

What settings do you typically use for HME and UHD? What has your experience been with HME?

[microchip8]

Well that's good to know! haha I've been mostly doing UHD encodes. My tests are normally done with 1080p content though... Hmmm I'll have to find a good 4K test clip.

What settings do you typically use for HME and UHD? What has your experience been with HME?

For 4k I use hme=1 and hme-search=umh,umh,star. This gives pretty good quality without sacrificing performance too much. It still has a performance hit but is faster (at least here on my Core i7 7700K) than any other combinations I did excluding hex (so all possible combinations of umh and star)

[brumsky]

For 4k I use hme=1 and hme-search=umh,umh,star. This gives pretty good quality without sacrificing performance too much. It still has a performance hit but is faster (at least here on my Core i7 7700K) than any other combinations I did excluding hex (so all possible combinations of umh and star)

Do you see an improvement in quality or a reduction in file size? How much of a performance hit have you seen?

[_kermit]

My newest GUI is for MediaInfo which supports now a presentation for encoder parameters, maybe it's useful for somebody here.

https://forum.doom9.org/showthread.php?t=176886

https://i.postimg.cc/0Q3RNTMD/image.png

Thank you!

[microchip8]

Do you see an improvement in quality or a reduction in file size? How much of a performance hit have you seen?

Quality improvement is difficult to spot in moving frames. You have to look at stills to notice it and you won't always notice it. I was able to spot it in some stills but not in moving frames.

File size reduction here is about a few %. Performance hit is the same as for FHD here. The best compromise between speed and quality seems to be hme-search=umh,umh,star. Excluding hex/dia, other combinations don't bring noticable difference in quality while have slightly bigger performance hit.

[Blue_MiSfit]

My newest GUI is for MediaInfo which supports now a presentation for encoder parameters, maybe it's useful for somebody here.

https://forum.doom9.org/showthread.php?t=176886

https://i.postimg.cc/0Q3RNTMD/image.png

VERY nice!!

[stax76]

Next release will have it sorted alphabetically.


Rate Control
aq-mode=1 / no-aq-motion / aq-strength=1.00
bitrate=4200 / cbqpoffs=0 / no-const-vbv
crqpoffs=0 / cutree / ipratio=1.40 / no-lossless
nr-inter=0 / nr-intra=0 / pbratio=1.30
qcomp=0.60 / qg-size=32 / qpmax=69 / qpmin=0
qpstep=4 / rc=abr / no-rc-grain / no-strict-cbr
zone-count=0

Analysis
amp / analysis-reuse-level=5 / b-intra
ctu=64 / no-cu-lossless / dynamic-rd=0.00
no-dynamic-refine / no-early-skip / no-fast-intra
no-hevc-aq / limit-modes / limit-refs=2
limit-tu=4 / max-tu-size=32 / min-cu-size=8
psy-rdoq=0.00 / qp-adaptation-range=1.00
rd=3 / rdoq-level=2 / rdpenalty=0 / no-rd-refine
rect / refine-ctu-distortion=0 / refine-inter=0
refine-intra=0 / refine-mv=0 / rskip / scale-factor=0
no-splitrd-skip / no-ssim-rd / no-tskip
no-tskip-fast / tu-inter-depth=2 / tu-intra-depth=2

Slice Decision
b-adapt=2 / bframe-bias=0 / bframes=4 / b-pyramid
ctu-info=0 / gop-lookahead=0 / no-intra-refresh
keyint=100 / lookahead-slices=4 / min-keyint=100
no-open-gop / radl=0 / rc-lookahead=30
ref=4 / refine-analysis-type=0 / scenecut=0
scenecut-bias=0.05

Motion Search
no-analyze-src-pics / max-merge=3 / me=3
merange=57 / subme=3 / temporal-mvp / no-weightb
weightp

Loop Filter
deblock=0:0 / no-limit-sao / sao / no-sao-non-deblock

VUI
chromaloc=0 / colormatrix=9 / colorprim=9
no-dhdr10-opt / display-window=0 / no-hdr
no-hdr-opt / max-cll=0,0 / max-luma=1023
min-luma=0 / overscan=0 / range=0 / sar=1
transfer=18 / videoformat=5

Bitstream
annexb / no-aud / hash=0 / no-hrd / no-idr-recovery-sei
info / log2-max-poc-lsb=8 / no-multi-pass-opt-rps
no-opt-cu-delta-qp / no-opt-qp-pps / no-opt-ref-list-length-pps
no-repeat-headers / no-single-sei / no-temporal-layers
vui-hrd-info / vui-timing-info

Input/Output
input-csp=1 / input-res=3840x2160 / interlace=0
total-frames=0

Performance
copy-pic=1 / frame-threads=1 / no-pme / no-pmode
slices=1 / wpp

Statistic
log-level=2 / no-psnr / no-ssim / stats-read=0
stats-write=0

Other
cpuid=1111039 / high-tier=1 / level-idc=0
max-ausize-factor=1.0 / no-allow-non-conformance
no-constrained-intra / no-lowpass-dct / no-splice
psy-rd=2.00 / signhide / strong-intra-smoothing
uhd-bd=0


probably also easy to built would be columns like so:


Rate Control
aq-mode=1 no-aq-motion aq-strength=1.00
bitrate=4200 cbqpoffs=0 no-const-vbv
crqpoffs=0 cutree ipratio=1.40
no-lossless nr-inter=0 nr-intra=0
pbratio=1.30 qcomp=0.60 qg-size=32
qpmax=69 qpmin=0 qpstep=4
rc=abr no-rc-grain no-strict-cbr
zone-count=0

[aymanalz]

Is enabling --rect worth the increased encoding time? Does it give enough quality improvement? It slows down my encode by 41%. :(

My settings are approximately the same as the "slow" preset, except that I use CTU 32 instead of 64, and merange of 26. (2 pass encode, 1080p.)

Should I change parameters other than rect, to gain more quality per bit? Or does rect bring enough quality improvement to justify the 41% increase in encode time?

[Boulder]

I find it usable at least in frame-by-frame checks. Make sure you have --limit-refs 3 and --limit-modes set to gain some performance back, but I think 'slow' already uses them.

[filler56789]

.EXE 3.2+9-971180b is ready.

http://msystem.waw.pl/x265/

[aymanalz]

I find it usable at least in frame-by-frame checks. Make sure you have --limit-refs 3 and --limit-modes set to gain some performance back, but I think 'slow' already uses them.

I have set limits-refs 3 and limit modes, and yet I get a 41% increase in encode time.

Of all the settings in the "slow" preset, this is the one that causes the biggest speed hit for me. That's why I'm wondering if it is really worth it.

[redbtn]

I'd like to know lowering --cbqpoffs -3 and --crqpoffs -3 makes x265 spend more bits to chroma taking them from luma, or it just lowers QP for chroma but don't touch luma?




Of all the settings in the "slow" preset, this is the one that causes the biggest speed hit for me. That's why I'm wondering if it is really worth it.
I decided that it's not worth it, at least for me. My target bitrate 17-18mb for FHD. Maybe at lower bitrate it gives more quality, I don't know.

[microchip8]

I have set limits-refs 3 and limit modes, and yet I get a 41% increase in encode time.

Of all the settings in the "slow" preset, this is the one that causes the biggest speed hit for me. That's why I'm wondering if it is really worth it.

Not worth it. You won't notice a difference in moving frames, only still ones and not always

if you want to squeeze out a bit more quality and lower file size by a % or few, better use --hme=1 --hme-search=umh,umh,star

[Blue_MiSfit]

I decided that it's not worth it, at least for me. My target bitrate 17-18mb for FHD. Maybe at lower bitrate it gives more quality, I don't know.

At those rates you may very well get better results with AVC, to be honest.

[aymanalz]

Not worth it. You won't notice a difference in moving frames, only still ones and not always

if you want to squeeze out a bit more quality and lower file size by a % or few, better use --hme=1 --hme-search=umh,umh,star

Thanks. Ironically, I started off by using and tweaking your own settings posted in this thread, in which you had enabled rect. :D

Is --hme=1 beneficial for 1080p? I am targeting around 5-6 mbps.

[Boulder]

At those rates you may very well get better results with AVC, to be honest.

That's true. For a regular, non-grainy source, over 10 Mbps at 1080p is already quite a lot. My 720p encodes with rather sharp downsizing and very light denoising, I usually get bitrates around 3-7 Mbps at CRF 18. Some episodes of Dark Matter have been below 2 Mbps :D

[aymanalz]

Not worth it. You won't notice a difference in moving frames, only still ones and not always

if you want to squeeze out a bit more quality and lower file size by a % or few, better use --hme=1 --hme-search=umh,umh,star

I just did that, and found that the resulting speed decrease is almost exactly the same as having --rect enabled. ie, using rect or using --hme umh,umh,star causes exactly the same amount of fps loss.

So if --hme gives better quality improvement than --rect, the takeaway should be to use hme and eschew rect, for squeezing out more quality for bitrate. I question whether the "slow" preset should have --rect enabled by default.

I notice that with --hme enabled, the second pass is much faster than the first. I wonder why that is.

[microchip8]

Thanks. Ironically, I started off by using and tweaking your own settings posted in this thread, in which you had enabled rect. :D

Is --hme=1 beneficial for 1080p? I am targeting around 5-6 mbps.

I had it enabled as I was testing a few things :sly:

In my encoding, I no longer use it as it's slow here too. --hme seems to run faster here, at least for me on a Core i7 7700K on Linux

HME is a bit beneficial for 1080p. But it really shows its strength for 2160p and higher

[Boulder]

In this case, VMAF could be useful in determining if it's worth it or not. Just need to make sure that the compared streams are decoded frame accurately so I would probably encode a sample clip, mux to mkv and index and decode with DGDecNV tools.

[benwaggoner]

I'd like to know lowering --cbqpoffs -3 and --crqpoffs -3 makes x265 spend more bits to chroma taking them from luma, or it just lowers QP for chroma but don't touch luma?
If limited by --bitrate or VBV, luma QP will go up to compensate for the bits spent by lower QP chroma.

If doing CRF, it'll just make the file larger.

In general, those values will reduce compression efficiency of most content in most scenarios.

[nghiabeo20]

Is there any option to limit CPU usage of x265 (used via ffmpeg)? I have time & patience, and I just want to limit x265 to 2 cores 4 threads, because I can't do anything when it consumes all 4 cores I have.

[Boulder]

Is there any option to limit CPU usage of x265 (used via ffmpeg)? I have time & patience, and I just want to limit x265 to 2 cores 4 threads, because I can't do anything when it consumes all 4 cores I have.

You could try --frame-threads 1 to limit encoding to one frame at a time.

If your only problem is that your computer becomes unusable, run the process at a lower priority. You can do that in command prompt by using "start /belownormal ...your encoder stuff here..."

[nghiabeo20]

You could try --frame-threads 1 to limit encoding to one frame at a time.

If your only problem is that your computer becomes unusable, run the process at a lower priority. You can do that in command prompt by using "start /belownormal ...your encoder stuff here..."

I use pools=4, and it does use only 4 threads of my 4C8T. Does it affect anything? I thought --frame-threads would severvely affect the performance.

Furthermore, I use htop to see CPU utilization. This is my third run, and the exact same 4 threads are used. Does it imply that ffmpeg only runs on those same 2 cores (or 4 cores) over and over? I know next to nothing about CPU architecture & organization, but if it implies so, does it have a long term effect on those cores? I thought that the OS should shuffle the workload among cores (between run), still I want to confirm.

[Atak_Snajpera]

I use pools=4, and it does use only 4 threads of my 4C8T. Does it affect anything? I thought --frame-threads would severvely affect the performance.

Why don't you try runing x265 in idle priority? What you do is not optimal.

[Boulder]

I thought that the OS should shuffle the workload among cores (between run), still I want to confirm.

The OS scheduler does do that, unless you set the process to use only specific cores (=CPU affinity). But as I and Atak_Snajpera said, set the process priority lower (idle or just below normal) and let the encoder use whatever resources are available to it after other processes.

[birdie]

Is there any option to limit CPU usage of x265 (used via ffmpeg)? I have time & patience, and I just want to limit x265 to 2 cores 4 threads, because I can't do anything when it consumes all 4 cores I have.

Windows: https://blogs.msdn.microsoft.com/santhoshonline/2011/11/24/how-to-launch-a-process-with-cpu-affinity-set/

Linux: man taskset

OS X: You're f*cked.

[redbtn]

If I want to get the same bitrate, which option is better for quality?
1) --no-cutree with higher CRF
2) --cutree with lower CRF