x265 HEVC Encoder

[x265_Project]

What's the usefulness of these new VBV params when you're doing segment encoding?

Yes, I think our documentation could be a bit better for this feature. If you want the whole video (after assembling your segments) to comply to specific VBV settings, set --vbv-end to the same value as you use for --vbv-init. In other words, the fullness of the decoder's buffer at the end of segment X needs to match the initial buffer fullness for segment X+1.

For example, if we assume the buffer is 90% full when we start a new segment, we need to make sure it wasn't more than 90% full when we finished encoding the last segment.

[Blue_MiSfit]

I see.

[LigH]

x265.h: change references into pointers for C compatibility (https://bitbucket.org/multicoreware/x265/commits/533c167774e879f00c0e4e6ee247c628fb81315b)

Building ffmpeg with libx265 should work again.

[MonoS]

Is there any reason why HDR-opt can be enabled only on 4:2:0 stream?

[SeeMoreDigital]

Is there any reason why HDR-opt can be enabled only on 4:2:0 stream?Currently all video encoded for use with DVD, Blu-ray, Ultra HD, internet streaming and television broadcasts is 4:2:0...

[MonoS]

Currently all video encoded for use with DVD, Blu-ray, Ultra HD, internet streaming and television broadcasts is 4:2:0...

So not something like "quantizer for != 4:2:0 encode should be different than the one coded in the codec", just there's no content out there?

[pradeeprama]

Is there any reason why HDR-opt can be enabled only on 4:2:0 stream?

The MPEG study group that recommended the changes x265 has implemented under the --hdr-opt option focused their study only on consumer grade video (HDR10 4:2:0). That is why it is recommended to use with 4:2:0 for now.

If there are other such recommendations for other color spaces, please do share it here.

[LigH]

x265 2.5+55-dd9772385d15 (https://www.mediafire.com/file/unaun0p506mhce9/x265_2.5%2B55-dd9772385d15.7z)

Add CLI option to enable or disable picture copy to internal frame buffer

--[no-]copy-pic Copy buffers of input picture in frame. Default enabled

Allow encoder to copy input x265 pictures to internal frame buffers. When disabled, x265 will not make an internal copy of the input picture and will work with the application's buffers. While this allows for deeper integration, it is the responsbility of the application to (a) ensure that the allocated picture has extra space for padding that will be done by the library, and (b) the buffers aren't recycled until the library has completed encoding this frame (which can be figured out by tracking NALs output by x265)

I can imagine that when x265 is linked inside an application or as DLL; but how is that supposed to work for x265 as a separate CLI application? I doubt it would have access to memory pointers of a calling application.

[pradeeprama]

x265 2.5+55-dd9772385d15 (https://www.mediafire.com/file/unaun0p506mhce9/x265_2.5%2B55-dd9772385d15.7z)

Add CLI option to enable or disable picture copy to internal frame buffer

--[no-]copy-pic Copy buffers of input picture in frame. Default enabled



I can imagine that when x265 is linked inside an application or as DLL; but how is that supposed to work for x265 as a separate CLI application? I doubt it would have access to memory pointers of a calling application.

I understand the confusion this might cause. Let me try to clarify.

If you invoke the x265 application, you won't be able to access the invoking application's memory buffers as you are in separate processes and therefore this option isn't useful. This option is applicable only when you do an integration of the x265 library with another application. In that case, the option is useful because we advocate that you use the x265_param_parse() API call to populate the right fields of the param structure instead of directly writing the struct yourself; doing it through the API ensures that all checks are done by the library. For every param, the string that you pass to populate the param is the cli-option string and therefore, exposing this as a CLI option enables folks who do deep integration to use this option in a clean fashion

[Selur]

Exposing an option to the CLI which con't be used through the CLI seems strange,.. (or did I misunderstand the above?)

[LigH]

As far as I hope to understand, it may not be helpful to the pure x265 CLI encoder, but possibly to other applications derived from this structure, configuring itself internally with a CLI parameter string as well.

[Selur]

So cli parameter that's not for the CLI,... <- this still seems like it shouldn't be exposed through the cli,.. :D

[x265_Project]

So cli parameter that's not for the CLI,... <- this still seems like it shouldn't be exposed through the cli,.. :D
I asked Pradeep the same question today. As the GM and the primary product manager, I agree that we don't want to confuse people who use our command line interface (CLI) to run x265 by exposing options that are invalid when run from the CLI. I understand that the x265_param_parse() function is a part of the CLI, and that it is useful to validate and organize/prioritize parameters when you run x265 via the API, but this looks like something that we could restructure to eliminate confusion for CLI users. It could be as simple as labeling some options as API only, such that if you attempt to run them from the CLI you'll get an error, or we'll recognize them but they will just be ignored with a warning.

Our documentation has a lot to do with x265's usability. You'll notice that most of x265's parameters are documented as CLI options. There are a few things, for example reconfiguring x265 on the fly, that are only possible through the API. Some things are documented in our online docs (http://x265.readthedocs.io/en/default/api.html), and other things are found in header files. As with most software, there is always room for improving our documentation. Suggestions and contributions are welcomed!

[MonoS]

The MPEG study group that recommended the changes x265 has implemented under the --hdr-opt option focused their study only on consumer grade video (HDR10 4:2:0). That is why it is recommended to use with 4:2:0 for now.

If there are other such recommendations for other color spaces, please do share it here.

Not that i know, i just want to encode a 1080p 4:4:4 HDR stream :)

[Midzuki]

..............
As with most software, there is always room for improving our documentation. Suggestions and contributions are welcomed!

How about finally correcting/updating this piece of (mis)information?

--bframes <integer> Maximum number of consecutive b-frames (now it only enables B GOP structure) Default 4

[Barough]

x265 v2.5+58-06979c042350 (http://www90.zippyshare.com/v/DuTCLjnQ/file.html) (GCC 7.2.0, 32 & 64-bit 8/10/12bit Multilib Windows Binaries)

https://bitbucket.org/multicoreware/x265/commits/branch/default

[Midzuki]

..............
As with most software, there is always room for improving our documentation. Suggestions and contributions are welcomed!
How about finally correcting/updating this piece of (mis)information?

--bframes <integer> Maximum number of consecutive b-frames (now it only enables B GOP structure) Default 4

Apparently I will have to return to the x265 mailing list.
Or pester Steve Borho.
Probably both things...

[x265_Project]

Apparently I will have to return to the x265 mailing list.
Or pester Steve Borho.
Probably both things...
Or contribute a patch

[Midzuki]

Or contribute a patch

Yeah, something so difficult as erasing some bytes from a lazily-written help screen really requires an official patch created by someone who stopped programming decades ago :rolleyes:

P.S.: welcome to my Ignore List (again)

[LigH]

Ah, come on ... I am not experienced in creating correct patches either, but when I knew how to improve things, suggesting the kernel of a diff in the mailing list was usually sufficient that another developer included that in a following full patch. They may be a bit picky about the exact style when it comes to official patches, but a professional project with commercial customers requires that amount of pedantism, and amateur suggestions are still welcome.

In your case with the consecutive B frames, I would not be sure which new content would be optimal, but I would know how I would submit a "snippet of a patch" to the mailing list.

[Ma]

Or contribute a patch

It's not always enough -- the patch https://patches.videolan.org/patch/15688/ is forgotten, x265cli.h has changed and now this patch doesn't apply cleanly.

[x265_Project]

I'm just trying to encourage participation. Sure, ideally we'd love patches to be in the right format, able to be applied cleanly to the latest development tip. But for our documentation, we can certainly handle an email to the developer mailing list with your suggested update to the documentation for a particular feature. Our team can figure out how to turn this into the official patch. Developing x265 is a big job... but this is open source, so we want to encourage participation.

Ma - I'm sorry if something was missed. We'll check into it.

[hajj_3]

are there going to be any nice quality improvements in the near future in x265 or are you getting towards x265's limit?

[LigH]

Not speaking for the developers, just my personal guess: I would not expect a revolution anymore, at least in the home consumer area (8-10 bit depth), rather evolution - at most finetuning. "Miracle time" is over, the HEVC specs indicate the range of algorithms which can be used (how to search for redundancies to spare and how to encode the video stream efficiently), and "revolutionary" improvements may not be supported by HEVC specs anymore (but there is e.g. AOMedia AV1). The biggest change I hope for would be integration of libav and AviSynth input modules.

But I could imagine room for more features for professional and specific "niche" usage cases. And of course, steady improvements of speed-up technologies, from more assembly to smarter algorithms in depth.

[pradeeprama]

It's not always enough -- the patch https://patches.videolan.org/patch/15688/ is forgotten, x265cli.h has changed and now this patch doesn't apply cleanly.

My bad - sorry. This patch is now pushed in and the outdated commit message has finally been updated!

[LigH]

x265 2.5+65-a7c2f80c18af (https://www.mediafire.com/file/jv56696q032e6og/x265_2.5%2B65-a7c2f80c18af.7z)

Disable opt-qp-pps and opt-ref-list-length-pps by default; use AVC CU analysis data in anlysis-reuselevel 7 and 8; update an outdated help message for --bframes

[LigH]

Should have waited one more day...

x265 2.5+66-dae558b40d99 (https://www.mediafire.com/file/8842c8ddjaax4ye/x265_2.5%2B66-dae558b40d99.7z)

merge with stable + fixed compiler warning (parentheses)

Milestone v2.6 is in sight!
_

P.S.:

The compilation of x265 will switch from YASM to NASM (min. v2.13); I wonder which were the main reasons: Instruction set support, compilation efficiency, bugs?

[Ma]

the compilation of x265 will switch from yasm to nasm (min. V2.13); i wonder which were the main reasons: Instruction set support, compilation efficiency, bugs?

Only nasm supports AVX-512.

[nevcairiel]

For the record, x264 did the same switch some weeks ago for the same reason.

[Aurelio]

Yasm is more or less EOL. No real development anymore. I also read once that YASM would need a rewrite to support AVX-512. So not going to happen anytime soon.

Some quotes from the x264 mailing list:

"Yes, but I don't see anyone willing to add support for AVX-512. the entire instruction syntax is different so it's a lot of work.
I have 0 hope of yasm ever getting avx-512 support since the instruction encoding is completely different from the old stuff, so it'd be tons of work to add it.
Yes. Yasm is essentially dead. Nasm is actively maintained.
Intel adds support to all new instruction sets to nasm themselves."

[nevcairiel]

Its unfortunate that nasm still suffers from a bunch of shortcomings that yasm didn't have. But maybe there is hope they'll fix it .... after all the years? :p

[jlpsvk]

any benefit of avx-512 in x265? i got i7-7820X, co can't wait.

[LigH]

Again, and again, and again ... there is no AVX-512 source code yet ... so there is no AVX-512 supporting binary yet ... so nobody could have measured an advantage yet.

[burfadel]

I wouldn't be surprised if the most useful instructions for encoding in x265 aren't in the supported feature sets of the mainstream processors. AVX-512 is broken down into several feature sets, only required one is foundation. Heavy AVX-512 use may also be am issue thermal wise but that remains to be seen.

[x265_Project]

You'll need to be patient.

[nevcairiel]

I wouldn't be surprised if the most useful instructions for encoding in x265 aren't in the supported feature sets of the mainstream processors. AVX-512 is broken down into several feature sets, only required one is foundation.

While in theory that is true, in practice there is currently only two "sets" of AVX512 features, one which is used in Purley Xeons and in Skylake-X, and one in Knights Landing - which is entirely uninteresting outside of HPC. (and the one in Skylake-X/Purley is the one we want for video)
Who knows why they split it up like this, maybe in the future there will be CPUs with smaller subsets, but right now the one "mainstream" CPU core that supports it supports all of the relevant things. But its also the first SIMD instruction set that was built "extensible" like this, previously we got AVX2 to extend AVX1 with integer instructions, now we get AVX512-BW/DQ

On that note, its already confirmed that Cannonlake and Icelake will only extend the current AVX512 instructions, and not take away features again. Of course no-one knows what happens if AMD ever implements AVX512.

The bigger problem with AVX512 is the thermal behavior and as such the downclocking it entails. So for AVX512 to be worthwhile to be used, you need to spend a significant amount of time doing AVX512 things, or the downclocking will slow down other things more then the AVX512 helps.

I believe the x265 developers have had experience with this with AVX2 in the past, which also causes a downclock to some degree, although with more optimized nodes that effect has gone down in newer CPUs.

[x265_Project]

The bigger problem with AVX512 is the thermal behavior and as such the downclocking it entails. So for AVX512 to be worthwhile to be used, you need to spend a significant amount of time doing AVX512 things, or the downclocking will slow down other things more then the AVX512 helps.
That's right.
I believe the x265 developers have had experience with this with AVX2 in the past, which also causes a downclock to some degree, although with more optimized nodes that effect has gone down in newer CPUs.
Yes, we do.

[benwaggoner]

Not speaking for the developers, just my personal guess: I would not expect a revolution anymore, at least in the home consumer area (8-10 bit depth), rather evolution - at most finetuning. "Miracle time" is over, the HEVC specs indicate the range of algorithms which can be used (how to search for redundancies to spare and how to encode the video stream efficiently), and "revolutionary" improvements may not be supported by HEVC specs anymore (but there is e.g. AOMedia AV1). The biggest change I hope for would be integration of libav and AviSynth input modules.
A revolution gets built out of lots of little evolutions. 20% improvement a year is still reasonable for HEVC.

But I could imagine room for more features for professional and specific "niche" usage cases. And of course, steady improvements of speed-up technologies, from more assembly to smarter algorithms in depth.
And performance improvements turn into quality improvements, because instead of encoding faster you can choose to encode better.

[LigH]

And performance improvements turn into quality improvements, because instead of encoding faster you can choose to encode better.

Partially ... more efforts don't guarantee better quality retention in every case (objective vs. subjective metrics). I would prefer increasing the bitrate over passing a "threshold of sanity" (which would be somewhere near preset "slower" for me, personally; provided I had potent hardware). :sly:

[pradeeprama]

x265 version 2.6 is now out! The key new improvements include support for segmented encoding, and introducing the ability of x265 to reuse and refine analysis from previously done HEVC and AVC encodes. The tarball of this release can be downloaded from the downloads page.

Release date - 29th November, 2017.
============================

New features
------------------
1) x265 can now refine analysis from a previous HEVC encode (using options --refine-inter, and --refine-intra), or a previous AVC encode (using option --refine-mv-type). The previous encode’s information can be packaged using the x265_analysis_data_t data field available in the x265_picture object.
2) Basic support for segmented (or chunked) encoding added with --vbv-end that can specify the status of CPB at the end of a segment. String this together with --vbv-init to encode a title as chunks while maintaining VBV compliance!
3) --force-flush can be used to trigger a premature flush of the encoder. This option is beneficial when input is known to be bursty, and may be at a rate slower than the encoder.
4) Experimental feature --lowpass-dct that uses truncated DCT for transformation.

Encoder enhancements
--------------------------------
1) Slice-parallel mode gets a significant boost in performance, particularly in low-latency mode.
2) x265 now officially supported on VS2017.
3) x265 now supports all depths from mono0 to mono16 for Y4M format.

API changes
-----------------
1) Options that modified PPS dynamically (--opt-qp-pps and --opt-ref-list-length-pps) are now disabled by default to enable users to save bits by not sending headers. If these options are enabled, headers have to be repeated for every GOP.
2) Rate-control and analysis parameters can dynamically be reconfigured simultaneously via the x265_encoder_reconfig API.
3) New API functions to extract intermediate information such as slice-type, scenecut information, reference frames, etc. are now available. This information may be beneficial to integrating applications that are attempting to perform content-adaptive encoding. Refer to documentation on x265_get_slicetype_poc_and_scenecut, and x265_get_ref_frame_list for more details and suggested usage.
4) A new API to pass supplemental CTU information to x265 to influence analysis decisions has been added. Refer to documentation on x265_encoder_ctu_info for more details.

Bug fixes
-------------
1) Bug fixes when --slices is used with VBV settings.
2) Minor memory leak fixed for HDR10+ builds, and default x265 when pools option is specified.
3) HDR10+ bug fix to remove dependence on poc counter to select meta-data information.

Happy compressing!

[LigH]

x265 2.6+2-32e6f04b8713 (https://www.mediafire.com/file/bbbvs61dxhlu76x/x265_2.6%2B2-32e6f04b8713.7z)

[pradeeprama]

The default branch of x265 has now moved to use nasm. The stable and release tips will continue to use yasm until the next release of x265.

If you are source-compiling x265, please download and install the nasm release 2.13 or newer from here and make sure that the binary is in your path before compiling.

Let us know if you run into any trouble with getting nasm installed on your machine

[LigH]

For people compiling in pre-installed MSYS environments: Both MABS and XhmikosR (+nevcairiel) already provide nasm 2.13.01, no problem to be expected with them; x265 2.6+5 just passed in both. Visual Studio integration may be more interesting.

[Magik Mark]

Hi!

Can you recommend a professional video editor & Encoder that uses this software?

[Midzuki]

The default branch of x265 has now moved to use nasm. The stable and release tips will continue to use yasm until the next release of x265.

If you are source-compiling x265, please download and install the nasm release 2.13 or newer from here and make sure that the binary is in your path before compiling.

Let us know if you run into any trouble with getting nasm installed on your machine

I have tried to compile the latest source-code on my ancient quadcore, but I had to cancel the process because it became incredibly slow :confused:

[ 25%] Building ASM_NASM object common/CMakeFiles/common.dir/x86/ipfilter16.asm.obj

is taking more than 3 minutes :eek: It even seemed I was using my deceased Pentium 4. What happened? The ASM code itself was changed, OR nasm is too slow by default and should be configured to be faster like the previous release of yasm?

OR "none of the above"? :confused:

[Ma]

Yep, nasm is very slow.

[Midzuki]

Then what about switching to FASM?
At least you could check whether it is actually fast =)

http://flatassembler.net/download.php

[qyot27]

I have tried to compile the latest source-code on my ancient quadcore, but I had to cancel the process because it became incredibly slow :confused:

[ 25%] Building ASM_NASM object common/CMakeFiles/common.dir/x86/ipfilter16.asm.obj

is taking more than 3 minutes :eek: It even seemed I was using my deceased Pentium 4. What happened? The ASM code itself was changed, OR nasm is too slow by default and should be configured to be faster like the previous release of yasm?

OR "none of the above"? :confused:
make -j$(nproc)? Or better, just use the Ninja generator instead of make.

Not that it'll speed up nasm on single files, but at least the build process will use all the cores.

[nevcairiel]

Then what about switching to FASM?
At least you could check whether it is actually fast =)

http://flatassembler.net/download.php

The code relies quite a lot on the nasm macro language, which is supported by nasm/yasm, so any other assemblers are not feasible.

[Midzuki]

The code relies quite a lot on the nasm macro language, which is supported by nasm/yasm, so any other assemblers are not feasible.

Thanks for the explanation *THUMBS UP*

P.S.: for the notes, latest nasm version is 2.13.02 since November 29.

https://forum.videohelp.com/attachments/43876-1512111306/nasm-version.png