x265 HEVC Encoder

[Mzvasturbo]

Hi i changed some settings in x265 and for 4k HDR content i came very close to the slow preset quality with 2-3 times faster speed and around 30% smaller file.
here you can check the pictures. https://www.dropbox.com/sh/yabmid1md...PUI5otsna?dl=0
CRF 17 hdr10-opt was used in both rips.

[filler56789]

x265.exe 3.3+12-00b686782ad0
(GCC 9.3.0, x64, multilib)

Latest changes:

1) Fix: clis in regression txt file.

1. corrects file naming convention.
2. updates deprecated rskip clis options.

2) zone: Enable strict VBV conformance for zone encode as per requirement

3) - Fixes mismatch in BufferRate with dynamic zone reconfiguration when bufferrate varies for each zone
- Logs UnclippedBufferFillFinal into csv when csvloglevel greater than 1

4) Add option to get global maxrate.

This global maxrate can be used for HRD signaling.

5) zone: Remove unnessary conditions on zone reconfig

This commit
- Removes unnessary conditions on zone reconfig
- Fixes crash with dynamic zone reconfig

http://www.mediafire.com/file/nm3997...82ad0.rar/file

[jlpsvk]

Originally Posted by Mzvasturbo

Hi i changed some settings in x265 and for 4k HDR content i came very close to the slow preset quality with 2-3 times faster speed and around 30% smaller file.
here you can check the pictures. https://www.dropbox.com/sh/yabmid1md...PUI5otsna?dl=0
CRF 17 hdr10-opt was used in both rips.

whats your command line???

[Mzvasturbo]

Other settings are from default medium preset this are the changed ones
--crf 17 --level-idc 5.1 --output-depth 10 --rdoq-level 2 --cu-lossless --aq-mode 4 --max-merge 3 --rc-lookahead 25 --lookahead-slices 4 --ref 4 --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)" --colorprim bt2020 --colormatrix bt2020nc --transfer smpte2084 --range limited --max-cll "xxx,xxx" --hdr --hdr-opt --repeat-headers --hrd --aud --deblock -1:-1 --no-strong-intra-smoothing

[filler56789]

x265.exe 3.3+19-1d2f556

http://msystem.waw.pl/x265/

Changes after commit 00b6867:

https://bitbucket.org/multicoreware/x265/commits/all

[Barough]

x265 v3.3+17-gdf2ac512d (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 9.3.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

[LazyNcoder]

Originally Posted by filler56789

x265.exe 3.3+19-1d2f556

http://msystem.waw.pl/x265/

Changes after commit 00b6867:

https://bitbucket.org/multicoreware/x265/commits/all

Tried the VS 2015/2019 AVX build and it gives me error:

x265 [info]: HEVC encoder version 3.3+19-1d2f556ffb12
x265 [info]: build info [Windows][MSVC 1900][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [error]: internalBitDepth must match compiled bit depth
x265 [error]: x265_encoder_open() failed for Enc,
x265 [error]: Failure generating stream headers 0
Error: fwrite() call failed when writing frame: 1, plane: 0, errno: 32
Output 34 frames in 24.73 seconds (1.37 fps)

Originally Posted by Barough

x265 v3.3+17-gdf2ac512d (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 9.3.0)

https://bitbucket.org/multicoreware/x265_git/commits/branch/master

It's same with this one.
But it's OK with 3.3+10-g08d8

[Forteen88]

LazyNcoder, yeah, I had error in the program "Simple x264 Launcher v2" (set to 10-bit encode) with x265.exe from x265-3.3+19-1d2f556_gcc100-AVX2.7z, but when I instead used x265-10b.exe it worked fine!

With old x265-3.3+10 x265.exe worked in "Simple x264 Launcher v2".

[Patman]

Originally Posted by LazyNcoder

Tried the VS 2015/2019 AVX build and it gives me error:

x265 [info]: HEVC encoder version 3.3+19-1d2f556ffb12
x265 [info]: build info [Windows][MSVC 1900][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [error]: internalBitDepth must match compiled bit depth
x265 [error]: x265_encoder_open() failed for Enc,
x265 [error]: Failure generating stream headers 0
Error: fwrite() call failed when writing frame: 1, plane: 0, errno: 32
Output 34 frames in 24.73 seconds (1.37 fps)

There is an error in the source code. I also compiled x265-3.3+17 and got the same error. I'll report it to the devs.

EDIT:

I compiled a x265-3.3+15 build and that version works. I would like to compile a x265-3.3+16 build and it's not possible. I've opened an issue. I use the x265_git as source.

[benwaggoner]

Is there any documentation anywhere on this exciting new feature?

ABR-ladder settings
--abr-ladder <file> File containing config settings required for the generation of ABR-ladder

With scaling also added as a new feature, I presume the file would be a list of resolutions and bitrates at a minimum. But I don't see the syntax documented anywhere yet.

[Blue_MiSfit]

Interesting! They've had this kind of functionality in UHDKit for awhile (commercially productized x265 with lots of neat pro features) so maybe they're bringing some of the features into the open source world

Analysis re-use is a huge deal when implemented correctly (especially for UHD). I wonder if that's part of this!

[benwaggoner]

Originally Posted by Blue_MiSfit

Interesting! They've had this kind of functionality in UHDKit for awhile (commercially productized x265 with lots of neat pro features) so maybe they're bringing some of the features into the open source world

Analysis re-use is a huge deal when implemented correctly (especially for UHD). I wonder if that's part of this!

Yeah, 2-3x speedups for a bitrate ladder are quite achievable via shared motion search and other analysis. Definite improvement in quality @ perf (although time to encode the single most complex output stream will be increased, typically).

[benwaggoner]

Silly me, there is help text in the code:

+.. option:: --abr-ladder <filename>
+ File containing the encoder configurations to generate ABR ladder.
+ The format of each line is:
+
+ **<encID:reuse-level:refID> <CLI>**
+
+ where, encID indicates the unique name given to the encode, refID indicates
+ the name of the encode from which analysis info has to be re-used ( set to 'nil'
+ if analysis reuse isn't preferred ), and reuse-level indicates the level ( ption:`--analysis-load-reuse-level`)
+ at which analysis info has to be reused.
+
+ A sample config file is available in `the downloads page <https://bitbucket.org/multicoreware/x265/downloads/Sample_ABR_ladder_config>`_
+
+ Default: Disabled ( Conventional single encode generation ). Experimental feature.
+
+ **CLI ONLY**

[benwaggoner]

And the sample config file.

Seems pretty straightforward. The "base" stream uses nil for the analysis file, and they chain from the previous one on up the ladder. It presumes a ladder of scaled sources.

A lot of cruft comes from using a .yuv file and needing to specify all the color space stuff for each line. Plus calculating PSNR and SSIM and saving those to a log file.

I'm not sure how the --scale-factor parameter is derived, but the rule seems to be 0 if the analysis file is the same resolution, otherwise 2.

Now that there is a scaling algorithm, an obvious extension would be to allow the scaling to take place within x265 so a single source can be used.

Also, it isn't clear at all if these would be encoded in parallel versus serially. I'm guessing serially as it doesn't say otherwise (in contrast to UHDKit, which can already do its own scaling and parallel encoding).

[360p:0:nil] --input 640x360_5994fps_8bit_420p.yuv --input-res 640x360 --fps 59.94 --input-depth 8 --input-csp i420 --min-keyint 60 --keyint 60 --no-open-gop --bitrate 145 --ssim --psnr --csv 0.csv --csv-log-level 2 -o 0.hevc --cutree --scale-factor 0
[432p:1:360p] --input 768x432_5994fps_8bit_420p.yuv --input-res 768x432 --fps 59.94 --input-depth 8 --input-csp i420 --min-keyint 60 --keyint 60 --no-open-gop --bitrate 300 --ssim --psnr --csv 1.csv --csv-log-level 2 -o 1.hevc --cutree --scale-factor 0
[540p1:1:432p] --input 960x540_5994fps_8bit_420p.yuv --input-res 960x540 --fps 59.94 --input-depth 8 --input-csp i420 --min-keyint 60 --keyint 60 --no-open-gop --bitrate 600 --ssim --psnr --csv 2.csv --csv-log-level 2 -o 2.hevc --cutree --scale-factor 0
[540p2:10:540p1] --input 960x540_5994fps_8bit_420p.yuv --input-res 960x540 --fps 59.94 --input-depth 8 --input-csp i420 --min-keyint 60 --keyint 60 --no-open-gop --bitrate 900 --ssim --psnr --csv 3.csv --csv-log-level 2 -o 3.hevc --cutree --scale-factor 0
[540p3:10:540p2] --input 960x540_5994fps_8bit_420p.yuv --input-res 960x540 --fps 59.94 --input-depth 8 --input-csp i420 --min-keyint 60 --keyint 60 --no-open-gop --bitrate 1600 --ssim --psnr --csv 4.csv --csv-log-level 2 -o 4.hevc --cutree --scale-factor 0
[720p1:10:360p] --input 1280x720_5994fps_8bit_420p.yuv --input-res 1280x720 --fps 59.94 --input-depth 8 --input-csp i420 --min-keyint 60 --keyint 60 --no-open-gop --bitrate 2400 --ssim --psnr --csv 5.csv --csv-log-level 2 -o 5.hevc --cutree --scale-factor 2
[720p2:10:720p1] --input 1280x720_5994fps_8bit_420p.yuv --input-res 1280x720 --fps 59.94 --input-depth 8 --input-csp i420 --min-keyint 60 --keyint 60 --no-open-gop --bitrate 3400 --ssim --psnr --csv 6.csv --csv-log-level 2 -o 6.hevc --cutree --scale-factor 0
[1080p1:10:540p3] --input 1920x1080_5994fps_8bit_420p.yuv --input-res 1920x1080 --fps 59.94 --input-depth 8 --input-csp i420 --min-keyint 60 --keyint 60 --no-open-gop --bitrate 4500 --ssim --psnr --csv 7.csv --csv-log-level 2 -o 7.hevc --cutree --scale-factor 2
[1080p2:10:1080p1] --input 1920x1080_5994fps_8bit_420p.yuv --input-res 1920x1080 --fps 59.94 --input-depth 8 --input-csp i420 --min-keyint 60 --keyint 60 --no-open-gop --bitrate 5800 --ssim --psnr --csv 8.csv --csv-log-level 2 -o 8.hevc --cutree --scale-factor 0
[1440p:10:720p2] --input 2560x1440_5994fps_8bit_420p.yuv --input-res 2560x1440 --fps 59.94 --input-depth 8 --input-csp i420 --min-keyint 60 --keyint 60 --no-open-gop --bitrate 8100 --ssim --psnr --csv 9.csv --csv-log-level 2 -o 9.hevc --cutree --scale-factor 2
[2160p1:10:1080p2] --input 3840x2160_5994fps_8bit_420p.yuv --input-res 3840x2160 --fps 59.94 --input-depth 8 --input-csp i420 --min-keyint 60 --keyint 60 --no-open-gop --bitrate 11600 --ssim --psnr --csv 10.csv --csv-log-level 2 -o 10.hevc --cutree --scale-factor 2
[2160p2:10:2160p1] --input 3840x2160_5994fps_8bit_420p.yuv --input-res 3840x2160 --fps 59.94 --input-depth 8 --input-csp i420 --min-keyint 60 --keyint 60 --no-open-gop --bitrate 16800 --ssim --psnr --csv 11.csv --csv-log-level 2 -o 11.hevc --cutree --scale-factor 0

[vpupkind]

Originally Posted by benwaggoner

And the sample config file.

Seems pretty straightforward. The "base" stream uses nil for the analysis file, and they chain from the previous one on up the ladder. It presumes a ladder of scaled sources.

A lot of cruft comes from using a .yuv file and needing to specify all the color space stuff for each line. Plus calculating PSNR and SSIM and saving those to a log file.

I'm not sure how the --scale-factor parameter is derived, but the rule seems to be 0 if the analysis file is the same resolution, otherwise 2.

Now that there is a scaling algorithm, an obvious extension would be to allow the scaling to take place within x265 so a single source can be used.

Also, it isn't clear at all if these would be encoded in parallel versus serially. I'm guessing serially as it doesn't say otherwise (in contrast to UHDKit, which can already do its own scaling and parallel encoding).
...

The analysis reuse done there depends on whether the analysis-save resolution is a power-of-two multiple of the analysis-load resolution. If the resolutions are same or power-of-two, things like motion vectors and RQT are used as predictors. If not, only frame types are reused in order to maintain IDR alignment in variable duration GOPs.

The older x265 implementation back in 2018 used to be serial (i.e., analysis-save encode had to complete before analysis-load encode starts). This version is closer to parallel -- you only need to finish a number of lower-resolution frames to start encoding at a higher resolution.

The older serial implementation gave us about 2.4x speedup on 4K predicted from 1080p vs "standalone" 4K. I don't have results on the new one yet.

[Blue_MiSfit]

Fascinating stuff! Thanks for sharing

[_kermit]

color primaries

I just re-encoded using this:
--master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)"

Now I noticed that the original file and the encoded one have different values in the primaries:

Original:
Mastering display color primaries : R: x=0.708000 y=0.292000, G: x=0.170000 y=0.797000, B: x=0.131000 y=0.046000, White point: x=0.312700 y=0.329000
Mastering display luminance : min: 0.0001 cd/m2, max: 1000.0000 cd/m2

New:
Mastering display color primaries : R: x=0.680000 y=0.320000, G: x=0.265000 y=0.690000, B: x=0.150000 y=0.060000, White point: x=0.312700 y=0.329000
Mastering display luminance : min: 0.0001 cd/m2, max: 1000.0000 cd/m2

Does that mean anything or is this bad?

-roland

[filler56789]

Originally Posted by Patman

I've opened an issue. I use the x265_git as source.

And the problem hasn't been fixed yet

Just wondering how and why they (MCW) approve the commits without testing the commits...

[Patman]

Originally Posted by filler56789

And the problem hasn't been fixed yet

Just wondering how and why they (MCW) approve the commits without testing the commits...

I can't understand the logic ... Compiling till 3.3 + 15 works without problems and the exe-file works, but after adding abr options and encoder this error is present. So far no response to my issue report.

[jlpsvk]

Originally Posted by _kermit

color primaries

I just re-encoded using this:
--master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)"

Now I noticed that the original file and the encoded one have different values in the primaries:

Original:
Mastering display color primaries : R: x=0.708000 y=0.292000, G: x=0.170000 y=0.797000, B: x=0.131000 y=0.046000, White point: x=0.312700 y=0.329000
Mastering display luminance : min: 0.0001 cd/m2, max: 1000.0000 cd/m2

New:
Mastering display color primaries : R: x=0.680000 y=0.320000, G: x=0.265000 y=0.690000, B: x=0.150000 y=0.060000, White point: x=0.312700 y=0.329000
Mastering display luminance : min: 0.0001 cd/m2, max: 1000.0000 cd/m2

Does that mean anything or is this bad?

-roland

yes... it is bad... you need to enter correct values to to source... read the x265 documentation...

--master-display "G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(10000000,1)" are correct values for your source