x265 HEVC Encoder

[kabelbrand]

I guess you'd have to be a HDR10+ adopter to get access to the Samsung tools (JsonFromVideo, Hdr10PlusGenerator) or instead use a software like Colorfront Transkoder.

[filler56789]

Maybe off-topic but probably a necessary notice...

«After much consideration, we’ve decided to remove Mercurial support from Bitbucket Cloud and its API. Mercurial features and repositories will be officially removed from Bitbucket and its API on June 1, 2020.»

https://bitbucket.org/blog/sunsettin...t-in-bitbucket

[quietvoid]

Originally Posted by LazyNcoder

Thank you.

1- What if the source we already have is HDR10+? Can we somehow extract or bypass the metadata into x265?

2- What tool from Samsung exactly? or any alternative

This might be useful for you: https://github.com/quietvoid/hdr10plus_parser

[LigH]

@filler56789: Good to get this notice way ahead ... enough time for the x265 team to possibly move the repository to either a different hoster or a different versioning system... (I would assume rather the first, since they love funny version tags which annoy some compiler suites not expecting their syntax).

[kabelbrand]

Originally Posted by quietvoid

This might be useful for you: https://github.com/quietvoid/hdr10plus_parser

Thanks, good work! Even though the Samsung Hdr10PlusInjector tool does not like this JSON format I guess as long as x265 is happy that's good enough.

For reference I attached the raw JSON output from my test. There are some additional indexes and ids in the Samsung output but these might be optional.

[quietvoid]

Originally Posted by kabelbrand

Thanks, good work! Even though the Samsung Hdr10PlusInjector tool does not like this JSON format I guess as long as x265 is happy that's good enough.

For reference I attached the raw JSON output from my test. There are some additional indexes and ids in the Samsung output but these might be optional.

Yes, if I remember correctly x265 just adds the metadata for each frame encoded, scene related info is ignored.
At the time of development (possibly now too), HDR10+ titles also have metadata inserted at every frame, and not like 1 SEI message per scene.

After all, some of this is trial and error using x265 as "correct" implementation.
There is still some desync between source JSON and extracted where there is a metadata change (scene change?), but it's usually just 1-2 frames that are different.

[benwaggoner]

Originally Posted by quietvoid

Yes, if I remember correctly x265 just adds the metadata for each frame encoded, scene related info is ignored.
At the time of development (possibly now too), HDR10+ titles also have metadata inserted at every frame, and not like 1 SEI message per scene.

After all, some of this is trial and error using x265 as "correct" implementation.
There is still some desync between source JSON and extracted where there is a metadata change (scene change?), but it's usually just 1-2 frames that are different.

You want to use --dhdr10-opt. That'll insert the SEI only on IDR frames and frames where the metadata changes.

Note that dynamic metadata isn't necessarily static across a shot; if there are dramatic changes in a shot, than there likely will be mid-shot metadata.

[quietvoid]

Originally Posted by benwaggoner

You want to use --dhdr10-opt. That'll insert the SEI only on IDR frames and frames where the metadata changes.

Note that dynamic metadata isn't necessarily static across a shot; if there are dramatic changes in a shot, than there likely will be mid-shot metadata.

I'm aware of x265's --dhdr10-opt, but I prefer to keep it the same as the source (which seems to always be for every frame).
Not sure if it would affect compatibility as well.

[benwaggoner]

Originally Posted by quietvoid

I'm aware of x265's --dhdr10-opt, but I prefer to keep it the same as the source (which seems to always be for every frame).
Not sure if it would affect compatibility as well.

That's just the way the source gets generated. No need to put it on every frame for playback.

[stax76]

Somebody reported an error, encoding finishes but returns exit code 1 at the end, has that a special meaning? What could be the reason?

CPU : AMD Ryzen 9 3900X 12-Core Processor

---------- Video encoding using x265 3.1+11-de920e0 Wolfberry ----------

"C:\Program Files (x86)\VapourSynth\core\vspipe.exe" K:\Downloads\SampleVideo_1280x720_1mb_temp\SampleVideo_1280x720_1mb.vpy - --y4m | C:\Users\Renor\Desktop\StaxRip\Apps\Encoders\x265\x265.exe --crf 20 --output-depth 12 --aq-mode 1 --frames 132 --y4m --output K:\Downloads\SampleVideo_1280x720_1mb_temp\SampleVideo_1280x720_1mb_out.hevc -

The system cannot find the path specified.
y4m [info]: 1280x720 fps 25/1 i420p8 frames 0 - 131 of 132
raw [info]: output file: K:\Downloads\SampleVideo_1280x720_1mb_temp\SampleVideo_1280x720_1mb_out.hevc
x265 [info]: HEVC encoder version 3.1+11-de920e0a3183
x265 [info]: build info [Windows][GCC 9.1.1][64 bit] 12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [info]: Main 12 profile, Level-3.1 (Main tier)
x265 [info]: Thread pool created using 24 threads
x265 [info]: Slices : 1
x265 [info]: frame threads / pool features : 4 / wpp(12 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : hex / 57 / 2 / 3
x265 [info]: Keyframe min / max / scenecut / bias: 25 / 250 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 3 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : CRF-20.0 / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip signhide tmvp b-intra
x265 [info]: tools: strong-intra-smoothing lslices=4 deblock sao
Output 132 frames in 1.92 seconds (68.61 fps)
x265 [info]: frame I: 1, Avg QP:20.28 kb/s: 21838.80
x265 [info]: frame P: 31, Avg QP:20.75 kb/s: 4661.11
x265 [info]: frame B: 100, Avg QP:27.32 kb/s: 311.44
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: consecutive B-frames: 3.1% 9.4% 3.1% 40.6% 43.8%
encoded 132 frames in 2.28s (57.92 fps), 1496.04 kb/s, Avg QP:25.73

------- Error Video encoding using x265 3.1+11-de920e0 Wolfberry -------

Video encoding using x265 3.1+11-de920e0 Wolfberry failed with exit code: 1 (0x1)

The exit code might be a system error code: STATUS_WAIT_1

The exit code might be a system error code: Incorrect function.

[stax76]

The issue was solved:

https://forum.doom9.org/showthread.p...20#post1883020

[markiemarcus]

Regarding RDOQ, am I the only one who really doesn't like the look of it? Though it's effective at maintaining grain motion, it seems to make it awfully boxy. I realise that this is subjective, but I find the results from disabling RDOQ entirely (--rdoq-level 0 --psy-rdoq 0) and increasing Psy-rd (to anywhere between 2.5 to 3.5) to be consistently more pleasing, assuming the same bitrate. Especially in grainy animation. The only explanation I can come up with for this comes from the documentation: that underlying flat colours reveal the shortcomings of RDOQ "biasing towards energy in general" as opposed to "energy in the source". On live action it seems okay, but I've found a number of instances in animation where anything other than tiny amounts can be quite ugly and make a right mess of fine grain or dither, regardless of the bitrate.

My apologies if that sounds overly negative; I realise that there are folks here who know a lot more than I do. Perhaps I'm missing something? Outside of a few difficulties with deep reds, this is really the only head-scratcher I've run into with x265. The only thing I long for is a way to control the strength of SAO. I like it, but sometimes I think half as much or even less would do.

[Barough]

x265 v3.1+14-f08461fdae33 (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 9.2.0)

https://bitbucket.org/multicoreware/x265/commits/branch/default

[filler56789]

Regarding v3.1+14-f08461fdae33...

sadly the compiler warnings are back — for example,

D:/KOMPILES/MCW/x265/source/encoder/analysis.cpp:2498:54: warning: this statement may fall through [-Wimplicit-fallthrough=]
 2498 |                                 case 3: mvpSelect[2] = mode.amvpCand[list][ref][!(mode.cu.m_mvpIdx[list][pu.puAbsPartIdx])];
      |                                         ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
D:/KOMPILES/MCW/x265/source/encoder/analysis.cpp:2499:33: note: here
 2499 |                                 case 2: mvpSelect[1] = mvp;
      |                                 ^~~~
[ 77%] Building CXX object encoder/CMakeFiles/encoder.dir/search.cpp.obj
D:/KOMPILES/MCW/x265/source/encoder/search.cpp: In member function 'void x265_12bit::Search::predInterSearch(x265_12bit::Mode&, const x265_12bit::CUGeom&, bool, uint32_t*)':
D:/KOMPILES/MCW/x265/source/encoder/search.cpp:2276:42: warning: this statement may fall through [-Wimplicit-fallthrough=]
 2276 |                             mvpIdxSel[2] = !mvpIdx;
      |                             ~~~~~~~~~~~~~^~~~~~~~~
D:/KOMPILES/MCW/x265/source/encoder/search.cpp:2277:21: note: here
 2277 |                     case 2: mvpSel[1] = interMode.amvpCand[list][ref][mvpIdx];
      |                     ^~~~
D:/KOMPILES/MCW/x265/source/encoder/search.cpp:2278:42: warning: this statement may fall through [-Wimplicit-fallthrough=]
 2278 |                             mvpIdxSel[1] = mvpIdx;
      |                             ~~~~~~~~~~~~~^~~~~~~~
D:/KOMPILES/MCW/x265/source/encoder/search.cpp:2279:21: note: here
 2279 |                     case 1: mvpSel[0] = interDataCTU->mv[list][cuIdx + puIdx].word;
      |                     ^~~~

And last but not least (since I-don't-know-when, really):

-- cmake version 3.15.2
CMake Deprecation Warning at CMakeLists.txt:10 (cmake_policy):
  The OLD behavior for policy CMP0025 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.


CMake Deprecation Warning at CMakeLists.txt:16 (cmake_policy):
  The OLD behavior for policy CMP0054 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.

[Boulder]

Originally Posted by benwaggoner

I'd expect there's be a lot of convergence at high bitrates. I'm most interested in how the different modes do when they don't have enough bits to do it right.

I'm also replicating the procedure from my encoding challenge, for apples-to-apples

https://forum.doom9.org/showthread.php?t=175776

Do you have any results from this aq-mode comparison? It would be interesting to know if mode 4 has any useful real-world applications. Of course, HDR encodes are a different story then.

[LigH]

@filler56789 - I do not remember anymore when I mentioned "deprecated CMake policies" to the developer mailing list ... could be more than a year ago already. They are still supported, and I can only hope that abandoning them won't break it for long.

[benwaggoner]

Originally Posted by Boulder

Do you have any results from this aq-mode comparison? It would be interesting to know if mode 4 has any useful real-world applications. Of course, HDR encodes are a different story then.

I encoded them and then went on vacation and forgot to look at them !

I'll try to mux and upload before I head to IBC.

[LigH]

Just a side note:

Recently I found archives of images stored in BPG format (Better Portable Graphics), a variant of HEIF. Trying to view them, I found that some famous Windows image viewers don't support it well, partially due to license doubts (IrfanView). For XnView I found a plugin which substitutes the partial support (bpgdec) by basic support via an encoder using x265, but it is not easily configurable (only via configuration file, globally). Web browsers can display BPG images via a JavaScript decoder for the HTML5 Canvas, written by the format author Fabrice Bellard.

The decoders I found so far are not very optimized, especially not multithreaded. Decoding a 150 MPx image takes several seconds. (You will probably guess well if you ask yourself who owns one of such 50K$ cameras...)

[benwaggoner]

Originally Posted by LigH

Just a side note:

Recently I found archives of images stored in BPG format (Better Portable Graphics), a variant of HEIF. Trying to view them, I found that some famous Windows image viewers don't support it well, partially due to license doubts (IrfanView). For XnView I found a plugin which substitutes the partial support (bpgdec) by basic support via an encoder using x265, but it is not easily configurable (only via configuration file, globally). Web browsers can display BPG images via a JavaScript decoder for the HTML5 Canvas, written by the format author Fabrice Bellard.

The decoders I found so far are not very optimized, especially not multithreaded. Decoding a 150 MPx image takes several seconds. (You will probably guess well if you ask yourself who owns one of such 50K$ cameras...)

It should be straightforward to write a version that would hand off the frame to system-level decoders from Edge and Safari. That'd be HW in most cases, but a well optimized assembly decoder is going to be faster than JS by a lot too.

Chrome and Firefox have been hard-coded to explicitly not pass through HEVC streams to the system decoders, although they do so for H.264. It'd be a trivial tweak to Chromium to re-enable that.

[LigH]

I also mean the decoding speed in separate image viewers. Especially sad for IrfanView, which used to be among the fastest for several formats, but is afraid of license trouble when implementing a HEIF decoder. (Or is it the encoder part, to implement the support only wholesome or not at all?)

Has libavcodec a multi-threaded HEVC decoder? I believe so...