Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 20th August 2019, 13:05   #7001  |  Link
kabelbrand
Compression mode: Lousy
 
kabelbrand's Avatar
 
Join Date: Mar 2009
Location: Hamburg, Germany
Posts: 73
I guess you'd have to be a HDR10+ adopter to get access to the Samsung tools (JsonFromVideo, Hdr10PlusGenerator) or instead use a software like Colorfront Transkoder.

Last edited by kabelbrand; 20th August 2019 at 13:07.
kabelbrand is offline   Reply With Quote
Old 20th August 2019, 16:55   #7002  |  Link
filler56789
SuperVirus
 
filler56789's Avatar
 
Join Date: Jun 2012
Location: Antarctic Japan
Posts: 923
Maybe off-topic but probably a necessary notice...

«After much consideration, we’ve decided to remove Mercurial support from Bitbucket Cloud and its API. Mercurial features and repositories will be officially removed from Bitbucket and its API on June 1, 2020.»

https://bitbucket.org/blog/sunsettin...t-in-bitbucket
filler56789 is offline   Reply With Quote
Old 21st August 2019, 04:46   #7003  |  Link
quietvoid
Registered User
 
Join Date: Jan 2019
Posts: 29
Quote:
Originally Posted by LazyNcoder View Post
Thank you.

1- What if the source we already have is HDR10+? Can we somehow extract or bypass the metadata into x265?

2- What tool from Samsung exactly? or any alternative
This might be useful for you: https://github.com/quietvoid/hdr10plus_parser
quietvoid is offline   Reply With Quote
Old 21st August 2019, 08:19   #7004  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,936
@filler56789: Good to get this notice way ahead ... enough time for the x265 team to possibly move the repository to either a different hoster or a different versioning system... (I would assume rather the first, since they love funny version tags which annoy some compiler suites not expecting their syntax).
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 22nd August 2019, 10:16   #7005  |  Link
kabelbrand
Compression mode: Lousy
 
kabelbrand's Avatar
 
Join Date: Mar 2009
Location: Hamburg, Germany
Posts: 73
hdr10plus_parser

Quote:
Originally Posted by quietvoid View Post
This might be useful for you: https://github.com/quietvoid/hdr10plus_parser
Thanks, good work! Even though the Samsung Hdr10PlusInjector tool does not like this JSON format I guess as long as x265 is happy that's good enough.

For reference I attached the raw JSON output from my test. There are some additional indexes and ids in the Samsung output but these might be optional.
Attached Files
File Type: 7z test_hdr10plus_parser.7z (69.4 KB, 13 views)
kabelbrand is offline   Reply With Quote
Old 22nd August 2019, 14:24   #7006  |  Link
quietvoid
Registered User
 
Join Date: Jan 2019
Posts: 29
Quote:
Originally Posted by kabelbrand View Post
Thanks, good work! Even though the Samsung Hdr10PlusInjector tool does not like this JSON format I guess as long as x265 is happy that's good enough.

For reference I attached the raw JSON output from my test. There are some additional indexes and ids in the Samsung output but these might be optional.
Yes, if I remember correctly x265 just adds the metadata for each frame encoded, scene related info is ignored.
At the time of development (possibly now too), HDR10+ titles also have metadata inserted at every frame, and not like 1 SEI message per scene.

After all, some of this is trial and error using x265 as "correct" implementation.
There is still some desync between source JSON and extracted where there is a metadata change (scene change?), but it's usually just 1-2 frames that are different.
quietvoid is offline   Reply With Quote
Old 22nd August 2019, 18:39   #7007  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,952
Quote:
Originally Posted by quietvoid View Post
Yes, if I remember correctly x265 just adds the metadata for each frame encoded, scene related info is ignored.
At the time of development (possibly now too), HDR10+ titles also have metadata inserted at every frame, and not like 1 SEI message per scene.

After all, some of this is trial and error using x265 as "correct" implementation.
There is still some desync between source JSON and extracted where there is a metadata change (scene change?), but it's usually just 1-2 frames that are different.
You want to use --dhdr10-opt. That'll insert the SEI only on IDR frames and frames where the metadata changes.

Note that dynamic metadata isn't necessarily static across a shot; if there are dramatic changes in a shot, than there likely will be mid-shot metadata.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 22nd August 2019, 23:16   #7008  |  Link
quietvoid
Registered User
 
Join Date: Jan 2019
Posts: 29
Quote:
Originally Posted by benwaggoner View Post
You want to use --dhdr10-opt. That'll insert the SEI only on IDR frames and frames where the metadata changes.

Note that dynamic metadata isn't necessarily static across a shot; if there are dramatic changes in a shot, than there likely will be mid-shot metadata.
I'm aware of x265's --dhdr10-opt, but I prefer to keep it the same as the source (which seems to always be for every frame).
Not sure if it would affect compatibility as well.
quietvoid is offline   Reply With Quote
Old 23rd August 2019, 18:41   #7009  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,952
Quote:
Originally Posted by quietvoid View Post
I'm aware of x265's --dhdr10-opt, but I prefer to keep it the same as the source (which seems to always be for every frame).
Not sure if it would affect compatibility as well.
That's just the way the source gets generated. No need to put it on every frame for playback.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 24th August 2019, 11:48   #7010  |  Link
stax76
Registered User
 
stax76's Avatar
 
Join Date: Jun 2002
Location: Germany
Posts: 5,792
Somebody reported an error, encoding finishes but returns exit code 1 at the end, has that a special meaning? What could be the reason?

CPU : AMD Ryzen 9 3900X 12-Core Processor

---------- Video encoding using x265 3.1+11-de920e0 Wolfberry ----------

"C:\Program Files (x86)\VapourSynth\core\vspipe.exe" K:\Downloads\SampleVideo_1280x720_1mb_temp\SampleVideo_1280x720_1mb.vpy - --y4m | C:\Users\Renor\Desktop\StaxRip\Apps\Encoders\x265\x265.exe --crf 20 --output-depth 12 --aq-mode 1 --frames 132 --y4m --output K:\Downloads\SampleVideo_1280x720_1mb_temp\SampleVideo_1280x720_1mb_out.hevc -

The system cannot find the path specified.
y4m [info]: 1280x720 fps 25/1 i420p8 frames 0 - 131 of 132
raw [info]: output file: K:\Downloads\SampleVideo_1280x720_1mb_temp\SampleVideo_1280x720_1mb_out.hevc
x265 [info]: HEVC encoder version 3.1+11-de920e0a3183
x265 [info]: build info [Windows][GCC 9.1.1][64 bit] 12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
x265 [info]: Main 12 profile, Level-3.1 (Main tier)
x265 [info]: Thread pool created using 24 threads
x265 [info]: Slices : 1
x265 [info]: frame threads / pool features : 4 / wpp(12 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : hex / 57 / 2 / 3
x265 [info]: Keyframe min / max / scenecut / bias: 25 / 250 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 3 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : CRF-20.0 / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip signhide tmvp b-intra
x265 [info]: tools: strong-intra-smoothing lslices=4 deblock sao
Output 132 frames in 1.92 seconds (68.61 fps)
x265 [info]: frame I: 1, Avg QP:20.28 kb/s: 21838.80
x265 [info]: frame P: 31, Avg QP:20.75 kb/s: 4661.11
x265 [info]: frame B: 100, Avg QP:27.32 kb/s: 311.44
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: consecutive B-frames: 3.1% 9.4% 3.1% 40.6% 43.8%
encoded 132 frames in 2.28s (57.92 fps), 1496.04 kb/s, Avg QP:25.73

------- Error Video encoding using x265 3.1+11-de920e0 Wolfberry -------

Video encoding using x265 3.1+11-de920e0 Wolfberry failed with exit code: 1 (0x1)

The exit code might be a system error code: STATUS_WAIT_1

The exit code might be a system error code: Incorrect function.
stax76 is online now   Reply With Quote
Old 24th August 2019, 20:08   #7011  |  Link
stax76
Registered User
 
stax76's Avatar
 
Join Date: Jun 2002
Location: Germany
Posts: 5,792
The issue was solved:

https://forum.doom9.org/showthread.p...20#post1883020
stax76 is online now   Reply With Quote
Old 31st August 2019, 00:42   #7012  |  Link
markiemarcus
Registered User
 
Join Date: May 2018
Posts: 4
Regarding RDOQ, am I the only one who really doesn't like the look of it? Though it's effective at maintaining grain motion, it seems to make it awfully boxy. I realise that this is subjective, but I find the results from disabling RDOQ entirely (--rdoq-level 0 --psy-rdoq 0) and increasing Psy-rd (to anywhere between 2.5 to 3.5) to be consistently more pleasing, assuming the same bitrate. Especially in grainy animation. The only explanation I can come up with for this comes from the documentation: that underlying flat colours reveal the shortcomings of RDOQ "biasing towards energy in general" as opposed to "energy in the source". On live action it seems okay, but I've found a number of instances in animation where anything other than tiny amounts can be quite ugly and make a right mess of fine grain or dither, regardless of the bitrate.

My apologies if that sounds overly negative; I realise that there are folks here who know a lot more than I do. Perhaps I'm missing something? Outside of a few difficulties with deep reds, this is really the only head-scratcher I've run into with x265. The only thing I long for is a way to control the strength of SAO. I like it, but sometimes I think half as much or even less would do.
markiemarcus is offline   Reply With Quote
Old 4th September 2019, 16:39   #7013  |  Link
Barough
Registered User
 
Barough's Avatar
 
Join Date: Feb 2007
Location: Sweden
Posts: 333
x265 v3.1+14-f08461fdae33 (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (GCC 9.2.0)
Code:
https://bitbucket.org/multicoreware/x265/commits/branch/default
Barough is offline   Reply With Quote
Old 5th September 2019, 06:49   #7014  |  Link
filler56789
SuperVirus
 
filler56789's Avatar
 
Join Date: Jun 2012
Location: Antarctic Japan
Posts: 923
Regarding v3.1+14-f08461fdae33...

sadly the compiler warnings are back — for example,

Code:
D:/KOMPILES/MCW/x265/source/encoder/analysis.cpp:2498:54: warning: this statement may fall through [-Wimplicit-fallthrough=]
 2498 |                                 case 3: mvpSelect[2] = mode.amvpCand[list][ref][!(mode.cu.m_mvpIdx[list][pu.puAbsPartIdx])];
      |                                         ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
D:/KOMPILES/MCW/x265/source/encoder/analysis.cpp:2499:33: note: here
 2499 |                                 case 2: mvpSelect[1] = mvp;
      |                                 ^~~~
[ 77%] Building CXX object encoder/CMakeFiles/encoder.dir/search.cpp.obj
D:/KOMPILES/MCW/x265/source/encoder/search.cpp: In member function 'void x265_12bit::Search::predInterSearch(x265_12bit::Mode&, const x265_12bit::CUGeom&, bool, uint32_t*)':
D:/KOMPILES/MCW/x265/source/encoder/search.cpp:2276:42: warning: this statement may fall through [-Wimplicit-fallthrough=]
 2276 |                             mvpIdxSel[2] = !mvpIdx;
      |                             ~~~~~~~~~~~~~^~~~~~~~~
D:/KOMPILES/MCW/x265/source/encoder/search.cpp:2277:21: note: here
 2277 |                     case 2: mvpSel[1] = interMode.amvpCand[list][ref][mvpIdx];
      |                     ^~~~
D:/KOMPILES/MCW/x265/source/encoder/search.cpp:2278:42: warning: this statement may fall through [-Wimplicit-fallthrough=]
 2278 |                             mvpIdxSel[1] = mvpIdx;
      |                             ~~~~~~~~~~~~~^~~~~~~~
D:/KOMPILES/MCW/x265/source/encoder/search.cpp:2279:21: note: here
 2279 |                     case 1: mvpSel[0] = interDataCTU->mv[list][cuIdx + puIdx].word;
      |                     ^~~~
And last but not least (since I-don't-know-when, really):

Code:
-- cmake version 3.15.2
CMake Deprecation Warning at CMakeLists.txt:10 (cmake_policy):
  The OLD behavior for policy CMP0025 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.


CMake Deprecation Warning at CMakeLists.txt:16 (cmake_policy):
  The OLD behavior for policy CMP0054 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.

Last edited by filler56789; 5th September 2019 at 06:55. Reason: emphasis
filler56789 is offline   Reply With Quote
Old 5th September 2019, 07:06   #7015  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,598
Quote:
Originally Posted by benwaggoner View Post
I'd expect there's be a lot of convergence at high bitrates. I'm most interested in how the different modes do when they don't have enough bits to do it right.

I'm also replicating the procedure from my encoding challenge, for apples-to-apples

https://forum.doom9.org/showthread.php?t=175776
Do you have any results from this aq-mode comparison? It would be interesting to know if mode 4 has any useful real-world applications. Of course, HDR encodes are a different story then.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 5th September 2019, 07:28   #7016  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,936
@filler56789 - I do not remember anymore when I mentioned "deprecated CMake policies" to the developer mailing list ... could be more than a year ago already. They are still supported, and I can only hope that abandoning them won't break it for long.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 10th September 2019, 17:00   #7017  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,952
Quote:
Originally Posted by Boulder View Post
Do you have any results from this aq-mode comparison? It would be interesting to know if mode 4 has any useful real-world applications. Of course, HDR encodes are a different story then.
I encoded them and then went on vacation and forgot to look at them !

I'll try to mux and upload before I head to IBC.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 11th September 2019, 07:44   #7018  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,936
Just a side note:

Recently I found archives of images stored in BPG format (Better Portable Graphics), a variant of HEIF. Trying to view them, I found that some famous Windows image viewers don't support it well, partially due to license doubts (IrfanView). For XnView I found a plugin which substitutes the partial support (bpgdec) by basic support via an encoder using x265, but it is not easily configurable (only via configuration file, globally). Web browsers can display BPG images via a JavaScript decoder for the HTML5 Canvas, written by the format author Fabrice Bellard.

The decoders I found so far are not very optimized, especially not multithreaded. Decoding a 150 MPx image takes several seconds. (You will probably guess well if you ask yourself who owns one of such 50K$ cameras...)
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 11th September 2019, 19:01   #7019  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,952
Quote:
Originally Posted by LigH View Post
Just a side note:

Recently I found archives of images stored in BPG format (Better Portable Graphics), a variant of HEIF. Trying to view them, I found that some famous Windows image viewers don't support it well, partially due to license doubts (IrfanView). For XnView I found a plugin which substitutes the partial support (bpgdec) by basic support via an encoder using x265, but it is not easily configurable (only via configuration file, globally). Web browsers can display BPG images via a JavaScript decoder for the HTML5 Canvas, written by the format author Fabrice Bellard.

The decoders I found so far are not very optimized, especially not multithreaded. Decoding a 150 MPx image takes several seconds. (You will probably guess well if you ask yourself who owns one of such 50K$ cameras...)
It should be straightforward to write a version that would hand off the frame to system-level decoders from Edge and Safari. That'd be HW in most cases, but a well optimized assembly decoder is going to be faster than JS by a lot too.

Chrome and Firefox have been hard-coded to explicitly not pass through HEVC streams to the system decoders, although they do so for H.264. It'd be a trivial tweak to Chromium to re-enable that.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 12th September 2019, 13:12   #7020  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,936
I also mean the decoding speed in separate image viewers. Especially sad for IrfanView, which used to be among the fastest for several formats, but is afraid of license trouble when implementing a HEIF decoder. (Or is it the encoder part, to implement the support only wholesome or not at all?)

Has libavcodec a multi-threaded HEVC decoder? I believe so...
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 14:07.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.