x265 HEVC Encoder

[benwaggoner]

yes my feeling was that it might be slower with pmode, but as i said i never actually did speed tests, just looked at cpu usage lol. my bad...
maybe it's a good idea for me to just remove it.
but going to slower is not an option (for me) - slow -> slower almost increases encoding time 100%

Yeah, it is quite likely that pmode is slowing you down a bunch and turning it off could get you that 100% speed back.

[vidschlub]

I'm having a discussion with someone online and they are citing an early 2016 discussion about 264 vs 265.
Does anyone know if there's a much more recent comparison of 264 to 265?

My assumption is, by now, with the correct settings used in the encoder, 265 should basically provide a superior image at the same bitrate, almost always (until the returns diminish at very high bitrates)
Surely, that is now the case?

[alex1399]

h.264 is still the grain king if you don't care the blocking it have.

[Blue_MiSfit]

h.264 is still the grain king if you don't care the blocking it have.

Unless we're talking 4K, particularly for HDR.

If you can't afford archival level bitrates, HEVC is dramatically better in almost every case, especially at high resolution.

[Forteen88]

I'm having a discussion with someone online and they are citing an early 2016 discussion about 264 vs 265.
Does anyone know if there's a much more recent comparison of 264 to 265?Video Codecs Comparison 2017,
http://www.compression.ru/video/codec_comparison/hevc_2017/ (http://www.compression.ru/video/codec_comparison/hevc_2017/)

Although they probably didn't compare 10-bit x265 vs 8bit x264 (x264 10-bit isn't supported by hardware-decoders!).
EDIT: OK, Nvidia GeForce 950/960 or better PC GPU supports full H265/HEVC 10-bit hardware-decoding, it doesn't support H264 10-bit hardware-decoding.

[NikosD]

Although they probably didn't compare 10-bit x265 vs 8bit x264 (x264 10-bit isn't supported by hardware-decoders!).

I think that mobile SOCs include HW decoding of H.264 10bit.

Probably Smart TVs, too.

[microchip8]

I think that mobile SOCs include HW decoding of H.264 10bit.

Probably Smart TVs, too.

I have 2 recent Smart TVs (Samsung and Panasonic) and 3 blu-ray players (2 from Samsung and 1 from LG). The Samsung BD players are UHD models

None of the devices above support 10-bit H.264 decoding

[excellentswordfight]

I'm having a discussion with someone online and they are citing an early 2016 discussion about 264 vs 265.
Does anyone know if there's a much more recent comparison of 264 to 265?

My assumption is, by now, with the correct settings used in the encoder, 265 should basically provide a superior image at the same bitrate, almost always (until the returns diminish at very high bitrates)
Surely, that is now the case?
I would say yes, if speed is not considered. But when tuning x265 to be as fast as x264 it falls behind imo.

Most of the test I've done has been in the "rip" catagory, I found that x265 --slow --no-sao --crf 18 has very similar fidelity to x264 --slower --tune film --crf 18 for 1080p bluray re-encoding with a 20-30% bitrate reduction. Most test I've done has been on tears of steel, which is a pretty good source for "general" film content imo, but it could ofc be sources were these numbers dont apply at all (but it has been the case on a few other random blurays I've tested on as well).

[Forteen88]

I hope that they finish the Video Codecs Comparison 2018 on that website soon. They've released an Express Report 2018, but it doesn't include "Ultra Ripping: Comparison on extremely slow presets" yet,
http://www.compression.ru/video/codec_comparison/hevc_2018/

[Przemek_Sperling]

I have 2 recent Smart TVs (Samsung and Panasonic) and 3 blu-ray players (2 from Samsung and 1 from LG). The Samsung BD players are UHD models

None of the devices above support 10-bit H.264 decoding

Weird, I own a cheap settop box (Opticum Sloth Combo Plus) and it decodes H.264 10-bit as well as H.265 12-bit. Maybe because of its chipset (Sunplus 1507). Most such devices have Ali chipsets and maybe they cannot decode such material.

[microchip8]

Weird, I own a cheap settop box (Opticum Sloth Combo Plus) and it decodes H.264 10-bit as well as H.265 12-bit. Maybe because of its chipset (Sunplus 1507). Most such devices have Ali chipsets and maybe they cannot decode such material.

I don't know the chipsets of my devices, but you are most likely correct. That said, there are quite a few devices (TVs, BD players & co) that don't support 10 bit H.264. My Samsung TV, however, supports decoding of 10 bits HEVC but not 10 bits H.264. I haven't tested 12 bits HEVC on it yet

[singhkays]

Using --no-sao for a tune film is imo valid. In my experience no-sao does improve fine detail alot with almost no negative effects for general "film" content with lower crf values. Preset slow together with no-sao is imo enough for detail retention now days. Not sure what setting does it, but I find preset Medium to be way softer then preset slow (imo there should only be a bitrate difference between them when doing a CRF encode, but it doesnt work like that I guess).

I have found sao to be usefull for both animation and low bitrate content though (as expected).


To add to this, I see around 70-80% utilization on dual Xeon E5-2680 v3 (48t) systems for 2160p content using preset slow. Imo that is a very reasonable ammount of multithread performance. For 1080p I wouldnt bother with anything more then 8-12C. Start using chunk-encoding if better multithread utilization is needed.

But I still think Atak question is valid, does 2990wx need any NUMA tweaking to perform correctly?

Not just 64 thread CPU, at work I have two Intel Xeon E5-2660V4 14c/28th for a total of 28c/56th and I can't still saturate both CPUs with a 2160p 10bit HDR10 content encoded with preset --medium and bluray compatible specs.


Some consumers are moving to AMD, but the majority of businesses are using Intel Xeon CPUs (my company included), so that's what they ask for optimizations.
They are simply following the market needs, nothing more.

I recently did some investigations around x265 scaling with 128 cores. You might be interested in the results https://www.singhkays.com/blog/x265-128-core-scaling-4k-hevc-hdr-azure-vm/

https://www.singhkays.com/img/128-core-x265-scaling/fps-vs-cores.png

No it is not too low. Dual socket (2 NUMA) Intel Xeon E5-4660 v3 (56 threads total) still scales much better than single socket (4 NUMA) 2990WX.
It would probably scale even better if I set numa pools manually.

According to x265 documentation ( https://x265.readthedocs.io/en/default/threading.html )

Can somebody verify than I'm setting numa pools correctly in my previous post?

See my investigation above. The CPU details are in the blog post. Not sure if I can help you verify something.

[Atak_Snajpera]

4k veryslow is the best case scenario for core utilization. 1080p with default medium preset would require at least 8 concurrent encodes to saturate all those 128 cores.

Ps. I'm not surprised that 2160p scales up to 32 cores. If We divide 2160 by default CU of 64 then we get value of 33.75.

[K.i.N.G]

I have 2 recent Smart TVs (Samsung and Panasonic) and 3 blu-ray players (2 from Samsung and 1 from LG). The Samsung BD players are UHD models

None of the devices above support 10-bit H.264 decoding

my nvidia shields, sony led tv (4k non-hdr) and lg oled tv (4k hdr) all decode it just fine, even my phone (samsung s5) plays them.

[microchip8]

my nvidia shields, sony led tv (4k non-hdr) and lg oled tv (4k hdr) all decode it just fine, even my phone (samsung s5) plays them.

With the exception of the two Samsung BD players, all my other devices are Full HD only. I must have bad luck because none can decode 10 bits H.264, including the Samsung BD players that say "file unsupported" when trying to feed them 10 bit H.264

That said, it's not important for me since I moved over to 10 bits HEVC which is decodable on the Samsung BD players *and* my Full HD Samsung TV. My Panasonic TV (also FHD) doesn't support it so I use one of the BD players to decode and feed it. I also prefer using the BD players to stream to my TVs as I can use Bitstream passthrough for the audio, which I can't when using the TVs directly (they internally convert it to AC3 - I use Toslink to feed audio from TVs to Yamaha receiver. Neither Toslink nor ARC supports lossless audio passthrough)

[benwaggoner]

4k veryslow is the best case scenario for core utilization. 1080p with default medium preset would require at least 8 concurrent encodes to saturate all those 128 cores.

Ps. I'm not surprised that 2160p scales up to 32 cores. If We divide 2160 by default CU of 64 then we get value of 33.75.
There is a 3 CTU lag in frame parallelism in x265, however. Multithreading performance tuning in x265 is a pretty complex matter. I find this invaluable:

https://x265.readthedocs.io/en/default/threading.html

(x265 has the best documentation of any codec, ever!)

[benwaggoner]

I recently did some investigations around x265 scaling with 128 cores. You might be interested in the results https://www.singhkays.com/blog/x265-128-core-scaling-4k-hevc-hdr-azure-vm/

https://www.singhkays.com/img/128-core-x265-scaling/fps-vs-cores.png



See my investigation above. The CPU details are in the blog post. Not sure if I can help you verify something.
Interesting data.

I would expect that, running multiple instances on multiple sockets like that, your 4x performance would be better if you used --pools to lock each instance to one socket to improve cache coherency and reduce NUMA utilization. Ala --pools "+,-,-,-" to lock to just the first socket of four.

[singhkays]

Interesting data.

I would expect that, running multiple instances on multiple sockets like that, your 4x performance would be better if you used --pools to lock each instance to one socket to improve cache coherency and reduce NUMA utilization. Ala --pools "+,-,-,-" to lock to just the first socket of four.

Thanks! I'm doing a follow up based on the comment below on the blog, so I'll include the above optimization as well. Are there other optimizations you'd like to see?

I'm confused about the using HEVC as the input file codec -- this will force decode delay, and unless it is full-intra, it will be multi-frame decode which will churn memory and processing just to get a decoded frame into the encode pipe.

A commercial application of this would be to encode a feature-length mezzanine, say an uncompressed MXF with a bit-depth and raster-size matching the output. Such a file would test raw encoding capability in high-cpu environments.

[benwaggoner]

Thanks! I'm doing a follow up based on the comment below on the blog, so I'll include the above optimization as well. Are there other optimizations you'd like to see?
That's the only one that popped out. Changes --pools will reduce the number of logical cores available and thus will also reduce the default --frame-threads and anything else that is based on core count, but that should happen automatically.

If you really want to stress single-instance encoding across all those sockets, try --pmode, or maybe even --pme if that doesn't saturate things. Those both increase CPU utilization more than they increase speed, but I bet you'd get more net speed out of a single instance with four cores with --pmode.

If you are looking to add more work for the encoder to do, both --cu-lossless and --tskip will help.

[RainyDog]

Unfortunately my PC's just crashed during the 2nd pass of a 2-pass encode.

If I run a 1-pass ABR encode using/reading the stats file that was generated during the 1st pass of the encode that crashed, am I right in assuming the result will be the same as if the 2nd pass had completed?

Thanks.

[LigH]

Why not running the 2nd pass of the 2-pass encode again?

[RainyDog]

Why not running the 2nd pass of the 2-pass encode again?

Thanks LigH. Well, that's kind of what I'm asking... Will a 1-pass ABR encode using the existing stats file produce the same result as if it were a 2nd pass?

Basically I'm using staxrip as the GUI and the encode options are 2-pass, Bitrate (1-pass ABR) and Quality (CRF). If I select 2-pass again then it will just start again and run another 1st pass creating a new stats file.

But if I select Bitrate ABR mode and add --pass 2 and --stats as custom encode options, when I set it going it takes a few seconds to read the stats file and then states --stats read once it starts. So its definitely using the stats file but my concern is is it using the 'proper' 2nd pass algorithm and not the poorer 1-pass ABR algorithm which I believe is internally different to the other modes...

Thanks.

[LigH]

I don't know which GUI you are using (x265 is a CLI encoder). If you can't select "2-pass, 1st pass" and "2-pass, 2nd pass" separately, but only "2-pass" as a bundle, then it may not be very helpful for your case.

The workaround you describe may work, but I didn't think much about it. After you tell us which GUI you are using, we might know a better solution, with or without this GUI.

By the way, the 1st pass may have been so quick, it could have been ran in the meantime...

[RainyDog]

I don't know which GUI you are using (x265 is a CLI encoder). If you can't select "2-pass, 1st pass" and "2-pass, 2nd pass" separately, but only "2-pass" as a bundle, then it may not be very helpful for your case.

The workaround you describe may work, but I didn't think much about it. After you tell us which GUI you are using, we might know a better solution, with or without this GUI.

By the way, the 1st pass may have been so quick, it could have been ran in the meantime...

I'm using Staxrip.

Well, the reason I didn't necessarily want to re-run the 1st pass is that I use custom 1st pass settings which are almost the same as the 2nd pass settings with a few changes to speed it up a bit. So it this instance the 1st pass took aalmost 6 hours... so if I can re-use it's stats file then it would save me 6 hours at least anyway!

Would also like to know for future reference really, in case it happens again.

Thanks.

[nevcairiel]

So just run the 2nd pass again using the stats file? Thats part of what makes the separate passes so nice, you can just run the 2nd pass again. Ordinarily one might use it to tune settings or whatnot, but if it failed you can also just start over.

[RainyDog]

So just run the 2nd pass again using the stats file? Thats part of what makes the separate passes so nice, you can just run the 2nd pass again. Ordinarily one might use it to tune settings or whatnot, but if it failed you can also just start over.

Hi nevcairiel, yes that's what I think I'm doing.

But I was just unsure whether I'd be unwittingly running an encode using ABR rate control mode as that's the option I'm having to select in Staxrip even though it's using a stats file already created from a successful 1st pass.

I believe ABR rate control mode is not recommended under any circumstances unless you must hit a certain bitrate/size and simply dont have the time for a 1st pass as it uses a different interal algorithm to CRF or 2pass encodes...

[nevcairiel]

You should probably ask the authors of that GUI how to run only a 2nd pass again. With the x265 CLI interface its pretty straight forward.

[RainyDog]

You should probably ask the authors of that GUI how to run only a 2nd pass again. With the x265 CLI interface its pretty straight forward.

Do you just add --pass 2 and --stats to the commnd line using the CLI interface?

[nevcairiel]

Basically yes, together with all other encoding options.

[RainyDog]

Basically yes, together with all other encoding options.

Thanks. Yeah, that's what I've done in the Staxrip custom command line options section.

I set it going before I headed off to work this morning and it definitely read/loaded the stats file before starting the encode and stated "stats-read" on the GUI output display anyway. Which it always does on a 2nd pass and states "stats-write" on a 1st pass.

[alex1399]

Hey guys, you might overlook the three pass encode for a fine result. The 1-pass ABR encode have two meaning, the --pass 1 activated first pass ABR encode and the initial ABR encode without --pass 1 activated. Once the --pass 1 is not included in the first pass ABR encode, add the --pass 1 and the --pass 2 for the following second and third pass encode. Else you could add --pass 3 and --pass 2 for the following three pass encode.

[benwaggoner]

Hey guys, you might overlook the three pass encode for a fine result. The 1-pass ABR encode have two meaning, the --pass 1 activated first pass ABR encode and the initial ABR encode without --pass 1 activated. Once the --pass 1 is not included in the first pass ABR encode, add the --pass 1 and the --pass 2 for the following second and third pass encode. Else you could add --pass 3 and --pass 2 for the following three pass encode.
Has anyone seen meaningful improvements from a third pass in x265? Since the 1st pass is a slow firstpass by default, I'd guess the gains would be less than in x264.

[WhatZit]

Has anyone seen meaningful improvements from a third pass in x265?

I toyed with this for several months, and was hard-pressed to identify any tangible benefit, even using naughty flick-screen frame grabs as comparison.

There was only ever the most microscopic detail improvements in low-contrast/high motion over a 2-pass, but these were easily homogenised out by the video processors in my TV's.

[benwaggoner]

I toyed with this for several months, and was hard-pressed to identify any tangible benefit, even using naughty flick-screen frame grabs as comparison.

There was only ever the most microscopic detail improvements in low-contrast/high motion over a 2-pass, but these were easily homogenised out by the video processors in my TV's.
I’ve only seen tangible improvements in 2-pass at low —bitrate —vbv-maxrate and -vbv-maxsize , OR when —vbv-maxrate is significantly higher than —bitrate

[RainyDog]

On the subject of multi-pass encodes, has anyone done much testing of the multi-pass-opt options --multi-pass-opt-analysis and --multi-pass-opt-distortion?

--multi-pass-opt-analysis, --no-multi-pass-opt-analysis

Enable/Disable multipass analysis refinement along with multipass ratecontrol. Based on the information stored in pass 1, in subsequent passes analysis data is refined and also redundant steps are skipped. In pass 1 analysis information like motion vector, depth, reference and prediction modes of the final best CTU partition is stored for each CTU. Multipass analysis refinement cannot be enabled when ‘analysis-save/analysis-load’ option is enabled and both will be disabled when enabled together. This feature requires ‘pmode/pme’ to be disabled and hence pmode/pme will be disabled when enabled at the same time.

--multi-pass-opt-distortion, --no-multi-pass-opt-distortion

Enable/Disable multipass refinement of qp based on distortion data along with multipass ratecontrol. In pass 1 distortion of best CTU partition is stored. CTUs with high distortion get lower(negative)qp offsets and vice-versa for low distortion CTUs in pass 2. This helps to improve the subjective quality. Multipass refinement of qp cannot be enabled when ‘analysis-save/analysis-load’ option is enabled and both will be disabled when enabled together. ‘multi-pass-opt-distortion’ requires ‘pmode/pme’ to be disabled and hence pmode/pme will be disabled when enabled along with it.

Seems that opt-analysis is more for speeding up subsequent passes by storing more data during the 1st pass for re-use. So... I think this might be more safely suited to multi-pass encodes where all passes use the exact (or almost) same settings?

And opt-distortion refines and re-balances QP distribution during subsequent passes based on info gathered in the 1st pass. Which sounds like a no brainer to use as long as it performs as it should... And perhaps even provides a genuine quality advantage to multi-pass encodes over CRF.

[alex1399]

The three pass encode is a refine of two pass encode where the original second pass encode is put at the end and an additional pass of slice-type / bit distribution encode inserted after the first pass. It is often to see that x265 places b-frame on some arbitrary location (not sub-optimal at all) even when the Quantization Parameter is not fluctuating everywhere. For the first pass encode, it get worse when x265 has to guess the target Rate Factor and the slice-type decision would be interfered by those Quantization Parameter fluctuation. The second pass in the two pass encode simply read the slice-type distribution from the first pass without any enhancement. The b-frame distribution is much consistent in the three pass encode. The three pass encode would place those b-frame like a specific case of two pass encode that utilize Constant Rate Factor encode for the first pass and Average Bit Rate encode for the second pass. That's why I prefer three pass encode.

For the --multi-pass-opt-analysis and the --multi-pass-opt-distortion, x265 prefers the Psycho-visual Rate Distortion measure to bias the Quantization Parameter in advance. I'm not a fan of those stuff. The compression efficiency based on non Psycho-visual metrics, for example the PSNR per bit, should still be a fundamental parts of encoder development. I'm aware that some encoder could trick on PSNR, but it doesn't sway my decision making.

[benwaggoner]

On the subject of multi-pass encodes, has anyone done much testing of the multi-pass-opt options --multi-pass-opt-analysis and --multi-pass-opt-distortion?





Seems that opt-analysis is more for speeding up subsequent passes by storing more data during the 1st pass for re-use. So... I think this might be more safely suited to multi-pass encodes where all passes use the exact (or almost) same settings?

And opt-distortion refines and re-balances QP distribution during subsequent passes based on info gathered in the 1st pass. Which sounds like a no brainer to use as long as it performs as it should... And perhaps even provides a genuine quality advantage to multi-pass encodes over CRF.
Yes, that's the one that should improve quality, and I imagine it should always be used in any second pass, particularly when doing CBR or having a constrained VBV. In the cases where I found 2-pass helpful, I was using --opt-distortion.

[RainyDog]

Ma & LigH,

Found the error in 2pass encoding:

Multipass analysis refinement along with multipass rate control
Multipass refinement of qp based on distortion data

If these two are deactivated everything is ok

Maybe the syntax has changed?

Found yesterday that this bug is still present in x265 2.8+66.

1st pass runs ok and creates the usual stats file plus a huge (11.5gb!) analysis file. Then crashes when it tries to run the 2nd pass if --multi-pass-opt-distortion has been set.

Are the developers aware of this? Thanks.

[RainyDog]

Here's my Staxrip log for the error when trying to start the 2nd pass when --multi-pass-opt-distortion is set.

"C:\Portable Programs\StaxRip\Apps\avs2pipemod\avs2pipemod64.exe" -y4mp "C:\Encoding\Source\God Told Me To (1976)_temp\God Told Me To (1976).avs" | "C:\Portable Programs\StaxRip\Apps\x265\x265.exe" --pass 1 --preset slow --profile main10 --output-depth 10 --rd 2 --psy-rd 2.0 --psy-rdoq 1.0 --subme 2 --aq-mode 3 --aq-strength 0.8 --qcomp 0.7 --bframes 6 --ipratio 1.2 --pbratio 1.1 --ctu 32 --no-cutree --merange 32 --qg-size 8 --no-rect --rc-lookahead 40 --lookahead-slices 0 --no-open-gop --early-skip --fast-intra --me dia --cbqpoffs -2 --crqpoffs -2 --deblock -2:-2 --no-sao --no-strong-intra-smoothing --no-constrained-intra --rdpenalty 1 --limit-tu 3 --tu-intra-depth 3 --tu-inter-depth 3 --multi-pass-opt-distortion --analysis-reuse-file "C:\Encoding\Source\God Told Me To (1976)_temp\God Told Me To (1976)_out.analysis" --bitrate 9000 --no-rect --no-cutree --weightb --no-open-gop --no-sao --no-strong-intra-smoothing --no-constrained-intra --frames 128857 --y4m --stats "C:\Encoding\Source\God Told Me To (1976)_temp\God Told Me To (1976).stats" --output NUL -

avs2pipemod[info]: writing 128857 frames of 24000/1001 fps, 1920x1040,
sar 0:0, YUV-420-planar-8bit progressive video.
y4m [info]: 1920x1040 fps 24000/1001 i420p8 unknown frame count
raw [info]: output file: NUL
x265 [info]: HEVC encoder version 2.8+66-88ee12651e30
x265 [info]: build info [Windows][MSVC 1915][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main 10 profile, Level-4 (Main tier)
x265 [info]: Thread pool created using 8 threads
x265 [info]: Slices : 1
x265 [info]: frame threads / pool features : 3 / wpp(33 rows)
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 3 inter / 3 intra
x265 [info]: ME / range / subpel / merge : dia / 32 / 2 / 3
x265 [info]: Keyframe min / max / scenecut / bias: 23 / 250 / 40 / 5.00
x265 [info]: Cb/Cr QP Offset : -2 / -2
x265 [info]: Intra 32x32 TU penalty type : 1
x265 [info]: Lookahead / bframes / badapt : 40 / 6 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 1
x265 [info]: References / ref-limit cu / depth : 4 / on / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 3 / 0.8 / 8 / 0
x265 [info]: Rate Control / qCompress : ABR-9000 kbps / 0.70
x265 [info]: tools: limit-modes rd=2 psy-rd=2.00 rdoq=2 psy-rdoq=1.00
x265 [info]: tools: early-skip rskip limit-tu=3 signhide tmvp fast-intra
x265 [info]: tools: deblock(tC=-2:B=-2) stats-write
avs2pipemod[info]: finished, wrote 128857 frames [100%].
avs2pipemod[info]: total elapsed time is 20045.940 sec.
x265 [info]: frame I: 1114, Avg QP:25.00 kb/s: 16601.32
x265 [info]: frame P: 23173, Avg QP:25.86 kb/s: 10044.17
x265 [info]: frame B: 104570, Avg QP:26.85 kb/s: 8597.98
x265 [info]: Weighted P-Frames: Y:8.1% UV:7.0%
x265 [info]: Weighted B-Frames: Y:8.3% UV:6.6%
x265 [info]: consecutive B-frames: 7.7% 1.6% 4.1% 6.5% 13.2% 53.3% 13.7%
encoded 128857 frames in 20047.53s (6.43 fps), 8927.25 kb/s, Avg QP:26.65

Start: 21:13:51
End: 02:47:59
Duration: 05:34:07

---------- Error Video encoding second pass using x265 2.6+31 ----------

Video encoding second pass using x265 2.6+31 failed with exit code: -1073741819 (0xC0000005)

The exit code might be a system error code: The instruction at 0xp referenced memory at 0xp. The memory could not be s.


------------- Video encoding second pass using x265 2.6+31 -------------

"C:\Portable Programs\StaxRip\Apps\avs2pipemod\avs2pipemod64.exe" -y4mp "C:\Encoding\Source\God Told Me To (1976)_temp\God Told Me To (1976).avs" | "C:\Portable Programs\StaxRip\Apps\x265\x265.exe" --pass 2 --preset slow --profile main10 --output-depth 10 --rd 4 --psy-rd 2.0 --psy-rdoq 1.0 --subme 4 --aq-mode 3 --aq-strength 0.8 --qcomp 0.7 --bframes 6 --ipratio 1.2 --pbratio 1.1 --ctu 32 --weightb --no-cutree --merange 32 --qg-size 8 --no-rect --rc-lookahead 40 --lookahead-slices 0 --no-open-gop --cbqpoffs -2 --crqpoffs -2 --deblock -2:-2 --no-sao --no-strong-intra-smoothing --no-constrained-intra --rdpenalty 1 --limit-tu 3 --tu-intra-depth 3 --tu-inter-depth 3 --multi-pass-opt-distortion --analysis-reuse-file "C:\Encoding\Source\God Told Me To (1976)_temp\God Told Me To (1976)_out.analysis" --bitrate 9000 --no-rect --no-cutree --no-open-gop --no-sao --no-strong-intra-smoothing --no-constrained-intra --frames 128857 --y4m --stats "C:\Encoding\Source\God Told Me To (1976)_temp\God Told Me To (1976).stats" --output "C:\Encoding\Source\God Told Me To (1976)_temp\God Told Me To (1976)_out.hevc" -

avs2pipemod[info]: writing 128857 frames of 24000/1001 fps, 1920x1040,
sar 0:0, YUV-420-planar-8bit progressive video.
y4m [info]: 1920x1040 fps 24000/1001 i420p8 unknown frame count
raw [info]: output file: C:\Encoding\Source\God Told Me To (1976)_temp\God Told Me To (1976)_out.hevc
x265 [info]: HEVC encoder version 2.8+66-88ee12651e30
x265 [info]: build info [Windows][MSVC 1915][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main 10 profile, Level-4 (Main tier)
x265 [info]: Thread pool created using 8 threads
x265 [info]: Slices : 1
x265 [info]: frame threads / pool features : 3 / wpp(33 rows)
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 3 inter / 3 intra
x265 [info]: ME / range / subpel / merge : star / 32 / 4 / 3
x265 [info]: Keyframe min / max / scenecut / bias: 23 / 250 / 40 / 5.00
x265 [info]: Cb/Cr QP Offset : -2 / -2
x265 [info]: Intra 32x32 TU penalty type : 1
x265 [info]: Lookahead / bframes / badapt : 40 / 6 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 1
x265 [info]: References / ref-limit cu / depth : 4 / on / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 3 / 0.8 / 8 / 0
x265 [info]: Rate Control / qCompress : ABR-9000 kbps / 0.70
x265 [info]: tools: limit-modes rd=4 psy-rd=2.00 rdoq=2 psy-rdoq=1.00 rskip
x265 [info]: tools: limit-tu=3 signhide tmvp deblock(tC=-2:B=-2) stats-read
avs2pipemod[info]: finished, wrote 55 frames [0%].
avs2pipemod[info]: total elapsed time is 7.488 sec.
avs2pipemod[error]: only wrote 55 of 128857 frames.

Ignore the parts about using x265 2.6+31, this is just the version of x265 that was bundled with the last official release of Staxrip. I'm using 2.8+66.

[Ma]

Thanks, now I'm able to reproduce the problem.
My first observation is:
8bit+10bit+12bit versions of x265 hangs/not working in second pass
10bit or 10bit+8bit versions of x265 works OK

Under investigation...

[Ma]

The bug is in commit 2.8+48 https://bitbucket.org/multicoreware/x265/commits/b0d31e2bc0dfd79abfc87a7bf366b1fbeb421b2c

Now it is only one function to allocate analysis_data in multilib x265 8bit+10bit+12bit -- it is from 8bit part and sse_t type is uint32_t in this function. For 10bit and 12bit sse_t type is uint64_t, so this function allocate only half required memory for distortionData if you use multilib x265 for encoding 10bit or 12bit.

Patch proposition:
diff -r 88ee12651e30 source/encoder/api.cpp
--- a/source/encoder/api.cpp Thu Aug 16 18:27:01 2018 +0530
+++ b/source/encoder/api.cpp Sat Sep 08 06:45:23 2018 +0200
@@ -411,13 +411,18 @@
bool isVbv = param->rc.vbvMaxBitrate > 0 && param->rc.vbvBufferSize > 0;
int numDir = 2; //irrespective of P or B slices set direction as 2
uint32_t numPlanes = param->internalCsp == X265_CSP_I400 ? 1 : 3;
+#if X265_DEPTH < 10
+ int numCUs_sse_t = param->internalBitDepth > 8 ? analysis->numCUsInFrame * 2 : analysis->numCUsInFrame;
+#else
+ int numCUs_sse_t = param->internalBitDepth > 8 ? analysis->numCUsInFrame : (analysis->numCUsInFrame + 1) / 2;
+#endif

//Allocate memory for distortionData pointer
CHECKED_MALLOC_ZERO(distortionData, x265_analysis_distortion_data, 1);
- CHECKED_MALLOC_ZERO(distortionData->distortion, sse_t, analysis->numPartitions * analysis->numCUsInFrame);
+ CHECKED_MALLOC_ZERO(distortionData->distortion, sse_t, analysis->numPartitions * numCUs_sse_t);
if (param->rc.bStatRead)
{
- CHECKED_MALLOC_ZERO(distortionData->ctuDistortion, sse_t, analysis->numCUsInFrame);
+ CHECKED_MALLOC_ZERO(distortionData->ctuDistortion, sse_t, numCUs_sse_t);
CHECKED_MALLOC_ZERO(distortionData->scaledDistortion, double, analysis->numCUsInFrame);
CHECKED_MALLOC_ZERO(distortionData->offset, double, analysis->numCUsInFrame);
CHECKED_MALLOC_ZERO(distortionData->threshold, double, analysis->numCUsInFrame);

Test x265 Windows binary (multilib exe + patch) www.msystem.waw.pl/x265/x265-2.8+66-patched.7z

[K.i.N.G]

I'm encoding an 4:2:2 1920x1080p 10bit video with HDR and i'm getting this error/warning:

x265 [error]: Recommended Settings for HDR: colour primaries should be BT.2020,
transfer characteristics should be SMPTE ST.2084,
matrix coeffs should be BT.2020,
the input video should be 10 bit 4:2:0
Disabling offset tuning for HDR videos

But the encoding just continues... is it safe to ignore this or should i convert the video to 4:2:0 (i'd rather not but if i really have to...)?

[SeeMoreDigital]

I'm encoding an 4:2:2 1920x1080p 10bit video with HDR and i'm getting this error/warning....

But the encoding just continues... is it safe to ignore this or should i convert the video to 4:2:0 (i'd rather not but if i really have to...)?4:2:0 is still the chroma sub-sampling standard for all commercially released video. No domestic hardware players supports 4:2:2.

[K.i.N.G]

4:2:0 is still the chroma sub-sampling standard for all commercially released video. No domestic hardware players supports 4:2:2.

I'm pretty sure my nvidia shield tv plays it just fine. And I'd be surprised if even the internal player of my tv couldn't handle 4:2:2 to be honest, but I cant test it right now (not at home for the whole weekend).

My concern is that I'm not sure what "disabling offset tuning for HDR videos" exactly means... Will the resulting encode still be HDR?

[SeeMoreDigital]

...And I'd be surprised if even the internal player of my tv couldn't handle 4:2:2 to be honest...None of my smart TV's can. And my OPPO can't recognise such video streams placed within any supporting container!

[RieGo]

i agree. you won't be happy with a 4:2:2 video file. i encoded one file in 422 by accident. none of my devices could play it. plex also couldn't read it iirc. only pc could handle the file.
IMHO best thing to do would be convert your file to 4:2:0 before encoding. (or maybe it's just a problem with your source filter) you won't see a big difference and x265 won't have any problems with hdr metadata

[mini-moose]

Odd question maybe...
Do switches like --max-cll or --master-display have any effect on the actual video encode or are they more like flags for your HDR device?

[sneaker_ger]

flags

[katzenjoghurt]

Hi guys,

I'm currently trying to convert Star Wars Episode III to x265... and hell... it's a beast.
This movie gives me super ugly artifacts in dark scenes.

Original: https://i.imgur.com/PA9cWuR.png
Encoding: https://i.imgur.com/HpVbyf4.png (with meagre 2,300kb/s in that scene, 4,500kb/s on avg for the whole movie)

My settings were: --crf 23 --tune grain --profile main12 --output-depth 12 --rskip --qcomp 0.8 --no-open-gop --no-deblock --no-strong-intra-smoothing

I already tried aq-mode 3 but to retain an okayish quality I had to set the aq-strength to 1.5 and ended up with 12 mbit/s for this scene.
I then tried aq-strength 3 and ended up with with perfect quality... and wopping 43 mbit /s.

Why are dark scenes so much more costly than brighter scenes?
This does not feel right...
What can I do?

[sneaker_ger]

I already tried aq-mode 3 but to retain an okayish quality I had to set the aq-strength to 1.5 and ended up with 12 mbit/s for this scene.
Why did you not lower crf?