Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 8th July 2017, 12:48   #5441  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,413
x264 and x265 coding are really problematic with new CPU with multiple core like "threadripper" or "skylake-X". x264 and x265 are unable to make encoding at 100% for CPU charge for 2K or even for 4K with 16C/32T or more.

@ x265 team

Not possible to create a new encoding mode in x265 with multiple instance for better threading compatibility?

Use for exemple high lookahead buffer for make frame type decision and open new instance coding at each new Iframe (will be IDR).

This mode imply certainely high frame buffer but if you have "threadripper" or "skylake-X" CPU, you must have at least 16 GB for RAM or more.

This mode imply just Closed GOP and perhaps short GOP (60 frames maximum) to minimize frame buffer.
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9

Last edited by Sagittaire; 8th July 2017 at 15:05.
Sagittaire is offline   Reply With Quote
Old 8th July 2017, 13:30   #5442  |  Link
pingfr
Registered User
 
Join Date: May 2015
Posts: 182
Quote:
Originally Posted by Sagittaire View Post
Use for exemple high lookahead buffer for make frame type decision and open new instance coding at each new Iframe (will be IDR).

This mode imply certainement high frame buffer but if you have "threadripper" or "skylake-X" CPU, you must have at least 16 GB for RAM or more.

This mode imply just Closed GOP and perhaps short GOP (60 frames maximum) to minimize frame buffer.
Got 512GB of RAM here, should be fine.
pingfr is offline   Reply With Quote
Old 8th July 2017, 21:39   #5443  |  Link
x265_Project
Registered User
 
x265_Project's Avatar
 
Join Date: Jul 2013
Posts: 591
Quote:
Originally Posted by Sagittaire View Post
x264 and x265 coding are really problematic with new CPU with multiple core like "threadripper" or "skylake-X". x264 and x265 are unable to make encoding at 100% for CPU charge for 2K or even for 4K with 16C/32T or more.

@ x265 team

Not possible to create a new encoding mode in x265 with multiple instance for better threading compatibility?

Use for exemple high lookahead buffer for make frame type decision and open new instance coding at each new Iframe (will be IDR).

This mode imply certainely high frame buffer but if you have "threadripper" or "skylake-X" CPU, you must have at least 16 GB for RAM or more.

This mode imply just Closed GOP and perhaps short GOP (60 frames maximum) to minimize frame buffer.
For a single instance of x265, we are limited in the # of threads we can use at any one time due to the serial nature of video encoding. For 4K encoding you should be able to utilize at least 20 threads efficiently. To increase parallelism, you can increase the number of frame threads, but this can affect rate control (although if you are using CRF rate control and you aren't using VBV, you shouldn't have any problem). You can also use --pme and/or --pmode. These functions will definitely increase CPU utilization, but they are not work-efficient (you won't see an increase in FPS that correlates with the increase in CPU utilization).

We have a commercial product called UHDkit that can run multiple x264 or x265 instances, in order to get more performance under different circumstances. As you know, x265 is free for anyone who can comply with the terms of the GPL v2 license. This includes large web video services, who don't distribute x265 (they distribute video). UHDkit is our value-added solution designed as an incentive for commercial customers to come to us to get a commercial license, which means they go from being free-riders on the x265 open source project to being sponsors of the project, which means we can hire more developers to improve x265 for everyone. The features in UHDkit are features that only commercial users would need, like the ability to produce multiple bit/quality rate tiers from a single video file twice as efficiently as doing unique encodes, or the ability to do live 4K 60P 10 bit encoding on a many-core (dual socket) server. Of course, anyone can run multiple instances of x265 on a single machine by themselves, but it wouldn't be easy to match the performance optimization features, or the other value-added features in UHDkit.
__________________
x265 HEVC (H.265) Video Encoder ____________ Follow x265 on Facebook.
x265_Project is offline   Reply With Quote
Old 8th July 2017, 22:08   #5444  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 6,242
High core count on AMD/Intel cpus is not a problem if you are RipBot264 user. All you need to do is activate distributed encoding mode with some servers.

This way you can easily saturate even dual AMD EPYC 7601 cpus (64C/128T) with just 1080p footage.
Atak_Snajpera is online now   Reply With Quote
Old 9th July 2017, 02:42   #5445  |  Link
x265_Project
Registered User
 
x265_Project's Avatar
 
Join Date: Jul 2013
Posts: 591
Quote:
Originally Posted by Atak_Snajpera View Post
High core count on AMD/Intel cpus is not a problem if you are RipBot264 user. All you need to do is activate distributed encoding mode with some servers.
This way you can easily saturate even dual AMD EPYC 7601 cpus (64C/128T) with just 1080p footage.
Is it open source? If you call x264 or x265, you need to comply with the GPL v2, and make your source code available under the GPL v2. This is a good thing, as others can contribute to help make it even better.
__________________
x265 HEVC (H.265) Video Encoder ____________ Follow x265 on Facebook.
x265_Project is offline   Reply With Quote
Old 9th July 2017, 14:13   #5446  |  Link
jd17
Registered User
 
Join Date: Jun 2017
Posts: 60
May I ask a question to the x265 devs here?

Why is --sao enabled by default?
We recently discussed this in another thread...

As the other thread is about high quality encodes, I thought --sao might be beneficial at low bitrates...
So I encoded my sample at 400 kbit/s, default settings, no tune. Once default, once with --no-sao.

Even at this extremely low bitrate I see no real benefit of --sao.
At high bitrates however, nobody seems to like --sao, because it adds a lot of blur.

The savings with a CRF also seem minor, considering this is such a big quality impact...


So, what is the reason for --sao by default?
Am I just missing something or would you consider --no-sao as default?
jd17 is offline   Reply With Quote
Old 9th July 2017, 15:42   #5447  |  Link
littlepox
Registered User
 
Join Date: Nov 2012
Posts: 213
Quote:
Originally Posted by jd17 View Post
May I ask a question to the x265 devs here?

Why is --sao enabled by default?
We recently discussed this in another thread...

As the other thread is about high quality encodes, I thought --sao might be beneficial at low bitrates...
So I encoded my sample at 400 kbit/s, default settings, no tune. Once default, once with --no-sao.

Even at this extremely low bitrate I see no real benefit of --sao.
At high bitrates however, nobody seems to like --sao, because it adds a lot of blur.

The savings with a CRF also seem minor, considering this is such a big quality impact...


So, what is the reason for --sao by default?
Am I just missing something or would you consider --no-sao as default?
I think they will remove it once they are pressured by AV1 or something so that the boss demand them to increase the compression efficiency by 20% in the next release.
littlepox is offline   Reply With Quote
Old 9th July 2017, 15:56   #5448  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 4,959
There are surely cases where SAO makes sense to reduce complexity. Mostly in UHD resolutions where looking at tiny details from a larger distance is less important than looking at small video dimensions close-up, where a pixel still makes an important detail, in relation.
__________________

German doom9 / Gleitz board
MediaFire shares: x265 | VPx | AOM | Xvid

Rémoulade is spoiled
LigH is online now   Reply With Quote
Old 10th July 2017, 11:58   #5449  |  Link
jd17
Registered User
 
Join Date: Jun 2017
Posts: 60
Could someone help me out here?:
https://forum.doom9.org/showthread.p...83#post1811483

I would mainly like to know what exactly --hdr-opt actually does.
It does come with a bitrate saving, so far so good, but is there a (negative) impact on quality? If so, what kind of impact?

I don't see an immediate degradation in that demo video, but it might not be well suited to make out the difference...
jd17 is offline   Reply With Quote
Old 11th July 2017, 00:06   #5450  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 4,959
Most of all, it tells the encoder to expect high dynamic range, and to adapt internal value ranges to characteristics of specific displays.

It is pretty useless for conversion of "usual video" without a high bit depth already in the original video source. If you don't know what "color primaries" mean, don't attempt to use it.
__________________

German doom9 / Gleitz board
MediaFire shares: x265 | VPx | AOM | Xvid

Rémoulade is spoiled
LigH is online now   Reply With Quote
Old 11th July 2017, 04:39   #5451  |  Link
x265_Project
Registered User
 
x265_Project's Avatar
 
Join Date: Jul 2013
Posts: 591
Quote:
Originally Posted by jd17 View Post
Could someone help me out here?:
https://forum.doom9.org/showthread.p...83#post1811483

I would mainly like to know what exactly --hdr-opt actually does.
It does come with a bitrate saving, so far so good, but is there a (negative) impact on quality? If so, what kind of impact?

I don't see an immediate degradation in that demo video, but it might not be well suited to make out the difference...
An MPEG study group looked at the question of whether the HEVC standard needed to be revised in order to optimize encoding of HDR content. People had noticed quality issues in certain places, and so this group looked at what was needed to avoid those issues. They found that no change was needed to the HEVC standard, but they issued some recommendations. --hdr-opt implements these recommendations. It's like a special type of adaptive quantization, which looks at the brightness level of each block of video. It improves encoding quality and efficiency for HDR content. Don't try to use it if you aren't encoding High Dynamic Range content.
__________________
x265 HEVC (H.265) Video Encoder ____________ Follow x265 on Facebook.
x265_Project is offline   Reply With Quote
Old 11th July 2017, 07:37   #5452  |  Link
jd17
Registered User
 
Join Date: Jun 2017
Posts: 60
Thanks for your answers!
Quote:
Originally Posted by LigH View Post
It is pretty useless for conversion of "usual video" without a high bit depth already in the original video source. If you don't know what "color primaries" mean, don't attempt to use it.
Quote:
Originally Posted by x265_Project View Post
Don't try to use it if you aren't encoding High Dynamic Range content.
One click on my link guys...

I did encode a 10bit HDR video:
Quote:
Originally Posted by jd17 View Post
I encoded the "Samsung HDR Wonderland" demo, once with --hdr-opt and once without.

Source MediaInfo:
Code:
Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5.1@High
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 2 min 41 s
Bit rate                                 : 45.7 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.230
Stream size                              : 878 MiB (100%)
Writing library                          : ATEME Titan File 3.7.3 (4.7.3.1002)
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : SMPTE ST 2084
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : R: x=0.680000 y=0.320000, G: x=0.265000 y=0.690000, B: x=0.150000 y=0.060000, White point: x=0.312700 y=0.329000
Mastering display luminance              : min: 0.0500 cd/m2, max: 1000.0000 cd/m2
My encode (HandBrake 1.0.7, x265 2.4):
CRF17, medium, no tune
CL custom: --no-sao --uhd-bd --hdr-opt --hrd --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,500)"
(I would normally add --max-cll too, but the source does not include that information...)

MediaInfo:
Code:
Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5.1@High
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 2 min 41 s
Bit rate                                 : 22.2 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0 (Type 2)
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.112
Stream size                              : 427 MiB (98%)
Writing library                          : x265 2.4+13-26963e98fa64:[Windows][MSVC 1910][64 bit] 10bit
Encoding settings                        : cpuid=1173503 / frame-threads=2 / numa-pools=4 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / 
input-res=3840x2160 / interlace=0 / total-frames=0 / level-idc=51 / high-tier=1 / uhd-bd=1 / ref=3 / no-allow-non-conformance / repeat-headers / annexb / aud / hrd / 
info / hash=0 / no-temporal-layers / no-open-gop / min-keyint=1 / keyint=24 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=20 / lookahead-slices=8 /
scenecut=40 / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=0 / 
dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=2 / limit-refs=3 / 
no-limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / no-sao / no-sao-non-deblock / rd=3 / 
no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=0.00 / no-rd-refine / analysis-mode=0 / 
no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=crf / crf=17.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / vbv-maxrate=160000 / vbv-bufsize=160000 / 
vbv-init=0.9 / crf-max=0.0 / crf-min=0.0 / ipratio=1.40 / pbratio=1.30 / aq-mode=1 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / 
no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=9 / transfer=16 / colormatrix=9 / chromaloc=1 / 
chromaloc-top=2 / chromaloc-bottom=2 / display-window=0 / master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,500) / max-cll=0,0 / 
min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / opt-qp-pps / opt-ref-list-length-pps / no-multi-pass-opt-rps / 
scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / hdr / hdr-opt / no-dhdr10-opt / refine-level=5 / no-limit-sao
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : SMPTE ST 2084
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : R: x=0.680000 y=0.320000, G: x=0.265000 y=0.690000, B: x=0.150000 y=0.060000, White point: x=0.312700 y=0.329000
Mastering display luminance              : min: 0.0500 cd/m2, max: 1000.0000 cd/m2
The encode without --hdr-opt essentially just resulted in a higher bitrate, everything else is identical:

Code:
Bit rate                                 : 25.7 Mb/s

Also, would you be so kind as to "sanity check" those parameters?
Would you change anything for UHD HDR encoding? Is anything essential missing?

I think you also suggested before:
--chromaloc 2
--keyint 48
?

I try to read up on the parameters as much as I can, but everything UHD HDR is not really well documented... So I welcome any help and explanations!

Quote:
Originally Posted by x265_Project View Post
--hdr-opt implements these recommendations. It's like a special type of adaptive quantization, which looks at the brightness level of each block of video. It improves encoding quality and efficiency for HDR content.
Does that mean there will be no visible degradation when I use --hdr-opt?
jd17 is offline   Reply With Quote
Old 12th July 2017, 15:41   #5453  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 4,959
x265 2.4+99-3160e1a0cc5f (merge stable+default)

"Allocate frame threads based on available pool threads" ... + a Mac fix and preps for v2.5 milestone.
__________________

German doom9 / Gleitz board
MediaFire shares: x265 | VPx | AOM | Xvid

Rémoulade is spoiled
LigH is online now   Reply With Quote
Old 12th July 2017, 15:53   #5454  |  Link
pingfr
Registered User
 
Join Date: May 2015
Posts: 182
Quote:
Originally Posted by LigH View Post
x265 2.4+99-3160e1a0cc5f (merge stable+default)

"Allocate frame threads based on available pool threads" ... + a Mac fix and preps for v2.5 milestone.
The big pool thread patch merged-in a few days ago alone is 2.5 milestone worthy IMHO.
pingfr is offline   Reply With Quote
Old 12th July 2017, 17:12   #5455  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 4,959
Another little patch about to be committed, and we will probably have it until the weekend. Optimistic guess.
__________________

German doom9 / Gleitz board
MediaFire shares: x265 | VPx | AOM | Xvid

Rémoulade is spoiled
LigH is online now   Reply With Quote
Old 13th July 2017, 18:03   #5456  |  Link
pradeeprama
Registered User
 
Join Date: Sep 2015
Posts: 41
x265 version 2.5, which includes improvements to grain handling, and improved CSV logging feature which is now built into the library.

Version 2.5 can be downloaded from here (md5: 192e54fa3068b594aa44ab2b703f071d).Full documentation is available at http://x265.readthedocs.io/en/stable/

Release Notes for Version 2.5
=======================

Encoder enhancements
--------------------------------
1. Improved grain handling with --tune grain option by throttling VBV operations to limit QP jumps.
2. Frame threads are now decided based on number of threads specified in the --pools, as opposed to the number of hardware threads available. The mapping was also adjusted to improve quality of the encodes with minimal impact to performance.
3. CSV logging feature (enabled by --csv) is now part of the library; it was previously part of the x265 application. Applications that integrate libx265 can now extract frame level statistics for their encodes by exercising this option in the library.
4. Globals that track min and max CU sizes, number of slices, and other parameters have now been moved into instance-specific variables. Consequently, applications that invoke multiple instances of x265 library are no longer restricted to use the same settings for these parameter options across the multiple instances.
x265 can now generate a seprate library that exports the HDR10+ parsing API. Other libraries that wish to use this API may do so by linking against this library. Enable ENABLE_HDR10_PLUS in CMake options and build to generate this library.
5. SEA motion search receives a 10% performance boost from AVX2 optimization of its kernels.
6. The CSV log is now more elaborate with additional fields such as PU statistics, average-min-max luma and chroma values, etc. Refer to documentation of --csv for details of all fields.
7. x86inc.asm cleaned-up for improved instruction handling.

API changes
-----------------
1. New API x265_encoder_ctu_info() introduced to specify suggested partition sizes for various CTUs in a frame. To be used in conjunction with --ctu-info to react to the specified partitions appropriately.
2. Rate-control statistics passed through the x265_picture object for an incoming frame are now used by the encoder.
3. Options to scale, reuse, and refine analysis for incoming analysis shared through the x265_analysis_data field in x265_picture for runs that use --analysis-reuse-mode load; use options --scale, --refine-mv, --refine-inter, and --refine-intra to explore.
4. VBV now has a deterministic mode. Use --const-vbv to exercise.

Bug fixes
-------------
1. Several fixes for HDR10+ parsing code including incompatibility with user-specific SEI, removal of warnings, linking issues in linux, etc.
2. SEI messages for HDR10 repeated every keyint when HDR options (--hdr-opt, --master-display) specified.

Happy compressing!
pradeeprama is offline   Reply With Quote
Old 13th July 2017, 21:39   #5457  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 4,959
Realistic guess, then!

x265 2.5+2-18fa144d453e (merge with stable; new v2.5 milestone)

SEI payload message writing fixed
__________________

German doom9 / Gleitz board
MediaFire shares: x265 | VPx | AOM | Xvid

Rémoulade is spoiled
LigH is online now   Reply With Quote
Old 13th July 2017, 23:11   #5458  |  Link
pingfr
Registered User
 
Join Date: May 2015
Posts: 182
Hah, I knew it!
pingfr is offline   Reply With Quote
Old 15th July 2017, 17:58   #5459  |  Link
Barough
Registered User
 
Barough's Avatar
 
Join Date: Feb 2007
Location: Sweden
Posts: 234
x265 v2.5+3-3f6841d271e3 (GCC 7.1.0, 32 & 64-bit 8/10/12bit Multilib Windows Binaries)

x265 [info]: HEVC encoder version 2.5+3-3f6841d271e3
x265 [info]: build info [Windows][GCC 7.1.0][32/64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2


Code:
https://bitbucket.org/multicoreware/x265/commits/branch/default
Barough is offline   Reply With Quote
Old 17th July 2017, 05:24   #5460  |  Link
bxyhxyh
Registered User
 
Join Date: Dec 2011
Posts: 333
Would developers update todo list? Last update of that is 2016.01.04.
bxyhxyh is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 19:37.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2017, vBulletin Solutions Inc.