x265 HEVC Encoder

[Barough]

x265 v2.8+21-a8a5ccf5aaf7 (http://www.mediafire.com/file/lqhocffrmbgvyve/x265-2.8+21-a8a5ccf5aaf7_Win_GCC730.7z) (GCC 7.3.0, 32 & 64-bit 8/10/12bit Multilib Windows Binaries)

https://bitbucket.org/multicoreware/x265/commits/branch/default

[Stereodude]

This may possibly be solved easier in user interfaces; but I will forward this to the mailing list for you to lead the developers' attention here.

P.S.: I wonder if you could use the ShutdownBlockReasonCreate (https://msdn.microsoft.com/en-us/library/windows/desktop/aa376877%28v=vs.85%29.aspx) function to postpone it.
Well, I call it directly from the command line with no GUI.

What's the x265 equivalent to --stitchable with x264 or is nothing needed to piece separate encodes together (that used the same command line)?

[sneaker_ger]

AFAIK there is no equivalent as x265 doesn't do the kind of "header optimizations" x264 did. So you should be able to stitch x265 encodes done with the same settings without problems.

[Selur]

I would have assumed that --repeat-headers (https://x265.readthedocs.io/en/latest/cli.html#cmdoption-repeat-headers) should be used for this scenario. :)

[sneaker_ger]

Repeating headers can mitigate problems. It's the same with x264 if you encode to raw .264 ES, i.e. the SPS/PPS will be at every keyframe. But SPS/PPS can still be different for every encode without --stitchable even with identical settings. But not with x265. Well, at least that was what I concluded when I tested it a long time ago...

[Barough]

x265 v2.8+23-656b5b442f0b (http://www.mediafire.com/file/2r5pq9i9lqa3xqa/) (GCC 7.3.0, 32 & 64-bit 8/10/12bit Multilib Windows Binaries)

https://bitbucket.org/multicoreware/x265/commits/branch/default

[Selur]

CEA 608/708 Support Parse the SEI messages from text file and insert it into the userSEI with cli option. .
Okay, so now we can add closed caption subtitle into the bitstream. Anybody got a script or small tool to convert srt to CEA 608/708 ?

[Ma]

Currently Windows 10 will happily reboot mid x264 or x265 encode even if the CPU is pegged at 100%.

My solution is to switch off Windows Update (before I want encoding):
https://www.easeus.com/todo-backup-resource/how-to-stop-windows-10-from-automatically-update.html (Solution 1. Disable Windows Update Service)

[LigH]

New build postponed, a patch renaming the new option is being proposed...

[LigH]

x265 2.8+24-289b8a3730ae (https://www.mediafire.com/file/940dk2d107fgnfe/x265_2.8%2B24-289b8a3730ae.7z)

--nalu-file <filename> Text file containing SEI messages in the following format : <POC><space><PREFIX><space><NAL UNIT TYPE>/<SEI TYPE><space><SEI Payload>

was --usersei-file a "moment" ago...

[MeteorRain]

My solution is to switch off Windows Update (before I want encoding):
https://www.easeus.com/todo-backup-resource/how-to-stop-windows-10-from-automatically-update.html (Solution 1. Disable Windows Update Service)

AFAIK Windows will try to change it back by itself.
My solution is to add both WUAU and delivery optimization service into Windows Firewall rules, and block all the outgoing traffic.

[RieGo]

adding more offtopic:
best way to prevent windows 10 updates is by disabling automatic installation in group policies (only on win 10 pro i think).
this is an easy and permanent solution.

[LigH]

But some people may not accept the risk of disabling updates and leaving Windows unpatched against new threats...

[Bhavnahari]

Greetings open source enthusiasts!

I am writing on behalf of the x265 developers. We notice that the open source community across the world has been using and experimenting with the x265 encoder and some of these works have been extremely intriguing. We would like to bring such ideas/experiments to the limelight and as an effort, we are reaching out to you to send in non-copyrighted articles that can be posted on the official x265 blog. We urge everyone who has published research papers based on x265 to share short write-ups on your analysis and your recommendations to improve x265. We believe that such contributions have the potential to become the gateway to new and better dimensions of video technology.

Your contributions will be of immense value to the open source fraternity. Looking forward to seeing the fascinating ideas that you all have. Please write to us at <pradeep@multicorewareinc.com> <bhavna@multicorewareinc.com> <vignesh@multicorewareinc.com>

[LigH]

x265 2.8+40-0106f9f2f867 (https://www.mediafire.com/file/st4y5zp5zpsq2rh/x265_2.8%2B40-0106f9f2f867.7z)

several new AVX(2) assembler routines

[registoni]

sorry if offtop but I am currious what settings amazon is using for encoding TV series in HEVC using x265 encoder.
In the MI there is rc=crf / crf=21.5. Are they using some encoding preset (medium, slow, slower, etc) or some fine tuned own setting? (probably stupid question)

[sneaker_ger]

Do they even use x265? x265 is one of many different H.265/HEVC encoders available on the market.

[LigH]

If you can find CRF in the MI, then it's quite probable that they used x265. Other encoders may not know this mode, and possibly not even store encoder options in MI.

But x265 only stores the internal API level (encoder core relevant) options. They are not equal to CLI options in every case. Selur has a tool which can calculate back which preset and tuning you could use to minimize the options. But you can't know whether a preset and tuning was indeed used. Or if a CLI encoder was used at all (maybe they have their own application using the encoder library via API).

[registoni]

Do they even use x265? x265 is one of many different H.265/HEVC encoders available on the market.

some info from MI:
General
Format : Matroska
Format version : Version 4 / Version 2
File size : 3.62 GiB
Duration : 59 min 30 s
Overall bit rate : 8 696 kb/s
Encoded date : UTC 2018-07-02 19:29:19
Writing application : mkvmerge v24.0.0 ('Beyond The Pale') 64-bit
Writing library : libebml v1.3.6 + libmatroska v1.4.9

Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main@L4@Main
Codec ID : V_MPEGH/ISO/HEVC
Duration : 59 min 30 s
Bit rate : 8 054 kb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Standard : NTSC
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Bits/(Pixel*Frame) : 0.162
Stream size : 3.35 GiB (93%)
Writing library : x265 0.0:[Linux][GCC 4.8.2][64 bit] 8bit+10bit+12bit
Encoding settings : cpuid=1173503 / frame-threads=1 / wpp / no-pmode / no-pme / psnr / ssim / log-level=2 / csvfn=/apollo/env/YoshiEncodingWorkflowActivitiesLinuxEncoding/var/tmp/62271779/5e5e8977-f266-439e-9262-f5c3dcf1ac15/771fd41e-a6d5-4188-90ed-6d02fb27bab5_video_1080p_9000kbps.csv / csv-log-level=2 / input-csp=1 / input-res=1920x1080 / interlace=0 / total-frames=0 / level-idc=40 / high-tier=1 / uhd-bd=0 / ref=5 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / no-open-gop / min-keyint=61 / keyint=120 / bframes=5 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=120 / lookahead-slices=0 / scenecut=0 / no-intra-refresh / ctu=64 / min-cu-size=8 / rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=200 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=3 / limit-modes / me=3 / subme=3 / merange=57 / temporal-mvp / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=4 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=1.00 / no-rd-refine / analysis-reuse-mode=0 / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=crf / crf=21.5 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / vbv-maxrate=9000 / vbv-bufsize=12000 / vbv-init=0.9 / crf-max=0.0 / crf-min=0.0 / ipratio=1.40 / pbratio=1.30 / aq-mode=3 / aq-strength=2.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=2 / range=0 / colorprim=1 / transfer=1 / colormatrix=1 / chromaloc=0 / display-window=0 / max-cll=0,0 / min-luma=0 / max-luma=255 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / opt-qp-pps / opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / no-hdr / no-hdr-opt / no-dhdr10-opt / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=0 / no-limit-sao / ctu-info=0
Default : Yes
Forced : No
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

[LigH]

Yes, quite certainly x265. With some additional information, possibly controlled by RipBot264 or similar tools for network distributed encoding.

[sneaker_ger]

Well, it certainly isn't one of the presets. (e.g. aq-mode 3 is in none of the presets)

https://x265.readthedocs.io/en/default/presets.html

[benwaggoner]

I mistakenly used x264 syntax in a x265 encode, and used --nr 350. But the x265 docs don't even document plain --nr as a command!

Anyone know what it does? --nr-inter 350 --nr intra 350? Just --nr-inter 350? That would match x264's behavior; it doesn't have that kind of intra-block filtering. If it's undefined, I'd recommend that --nr just alias--nr-inter.

[RainyDog]

Is --no-slow-firstpass safe to use for 2-pass encodes? I seem to recall reading a post some time ago which said that a fast/turbo 1st pass can be too unreliable for a slow 2nd pass with much higher settings to use as a basis.

Thanks.

[LigH]

x265 CLI option: slow-firstpass (https://x265.readthedocs.io/en/default/cli.html?highlight=slow-first#cmdoption-slow-firstpass)

Enable first pass encode with the exact settings specified. The quality in subsequent multi-pass encodes is better (compared to first pass) when the settings match across each pass. Default enabled.

The "turbo" options are indeed quite different from the original options in slow presets. But I don't remember any test showing verbosely how obvious the loss of quality is... the following pass(es) will still use the intended efforts, just the bitrate calculation may be based on a not really optimal reference distribution.

[benwaggoner]

x265 CLI option: slow-firstpass (https://x265.readthedocs.io/en/default/cli.html?highlight=slow-first#cmdoption-slow-firstpass)

The "turbo" options are indeed quite different from the original options in slow presets. But I don't remember any test showing verbosely how obvious the loss of quality is... the following pass(es) will still use the intended efforts, just the bitrate calculation may be based on a not really optimal reference distribution.
I did some noodling around using the slow-firstpass but reducing subme and a few other low-level things for the first pass. Testing wasn't that complete, but it looked like for a net total encoding time, I could get slightly better results by allowing the second pass to be slower by making the first pass faster. It went to more a 25/75 than 50/50 ratio.

The big problem I see with --no-slow-firstpass is ref=1, which can really impact things with some kinds of content. I would think it is critical to use the same number of reference and b-frames and same badapt mode between passes.


So, using --no-slow-firstpass and then setting --ref to the same as the second pass is likely a good starting point to experiment from

[benwaggoner]

sorry if offtop but I am currious what settings amazon is using for encoding TV series in HEVC using x265 encoder.
In the MI there is rc=crf / crf=21.5. Are they using some encoding preset (medium, slow, slower, etc) or some fine tuned own setting? (probably stupid question)
That is not a question I will be answering in any form :). Everything I write here is my personal opinion and experience only.

[foxyshadis]

That is not a question I will be answering in any form :). Everything I write here is my personal opinion and experience only.

I can just see the headlines in six months. "Famed compressionist Ben Waggoner caught in dirty format lies!" "World scoffs as he claims they were true at the time"

As for --nr... I thought that would be a quick find. No, you actually found a bug in getopt, the venerable GNU freaken getopt. It lets you abbreviate options, normally, and has a sanity check in case it matches a second option... but that sanity check only triggers if the shortname (eg, -V) or whether it needs an argument differs. Just in case the developer brain-farted an inserted the same option twice, it wouldn't blow up. But they never bothered to compare the full names!

So yes, entirely by accident, --nr means --nr-intra. I wonder if my getopt.c patch will break anyone's workflow.

[LigH]

I have just noticed a proposed patch in the mailing list that addresses this issue. Was that yours? ... They prefer patch sources in the mail body, not as attachment only.

[RainyDog]

I did some noodling around using the slow-firstpass but reducing subme and a few other low-level things for the first pass. Testing wasn't that complete, but it looked like for a net total encoding time, I could get slightly better results by allowing the second pass to be slower by making the first pass faster. It went to more a 25/75 than 50/50 ratio.

The big problem I see with --no-slow-firstpass is ref=1, which can really impact things with some kinds of content. I would think it is critical to use the same number of reference and b-frames and same badapt mode between passes.


So, using --no-slow-firstpass and then setting --ref to the same as the second pass is likely a good starting point to experiment from

Thanks benwaggoner.

Yeah, ref=1 and rd=2 were the two settings in --no-slow-firstpass that I saw as cause for concern too. Especially since I sometimes use rd=5 for the 2nd pass too.

How can I specify --ref 4 and --rd 3 with --no-slow-firstpass?

[LigH]

Just in the same command line.

x265 --preset ... --no-slow-firstpass --ref 4 --rd 3 ...

First the "meta options" which change several parameters at once, then atomic options which override these changes.

[RainyDog]

Just in the same command line.

x265 --preset ... --no-slow-firstpass --ref 4 --rd 3 ...

First the "meta options" which change several parameters at once, then atomic options which override these changes.

Thanks LigH.

Yeah I do that anyway as I always make quite a few tweaks to the standard presets. But what if I want to specify --rd 3 only for the fast 1st pass but use --rd 5 for the slow 2nd pass?

My thoughts are that --rd 3 will provide a much more suitable analysis than --rd 2 for the 1st pass and use psy-rdoq.

[LigH]

Just do it; compare --rd 2 / 3 / 5 in 1st pass (and always 5 in last) among each other, tell us if you see any difference. I doubt. You will be sure afterwards.

[RainyDog]

Just do it; compare --rd 2 / 3 / 5 in 1st pass (and always 5 in last) among each other, tell us if you see any difference. I doubt. You will be sure afterwards.

Well, I would... If I knew how to specify the --rd level and ref's in the 1st pass. That's what I was hoping you could tell me :)

Would it be :-

x265 --preset slow --no-slow-firstpass --pass 1 --ref 4 --rd 3(or 5) --pass 2... followed by the remainder of my manually set options.

[LigH]

No. You have to complete the 1st pass before you run the 2nd.

So first, run
x265 --preset slow --pass 1 --no-slow-firstpass --ref 4 --rd 3 ...
until it finished, to produce mainly a statistics file (if it writes also a video file, then this is surely not yet optimally encoded). Then run
x265 --preset slow --pass 2 --ref 4 --rd 5 ...
and now you have your resulting video file.

[IgorC]

An interesting article about compression/performance ratio of x265's presets

http://x265.org/performance-energy-consumption-analysis-x265/

[benwaggoner]

No. You have to complete the 1st pass before you run the 2nd.

So first, run
x265 --preset slow --pass 1 --no-slow-firstpass --ref 4 --rd 3 ...
until it finished, to produce mainly a statistics file (if it writes also a video file, then this is surely not yet optimally encoded). Then run
x265 --preset slow --pass 2 --ref 4 --rd 5 ...
and now you have your resulting video file.
--dynamic-rd could also be an interesting feature to use in tuning quality/speed. In theory it would allow for faster encoding for easy parts of the video, only switching to higher subme etcetera for the harder bits.

[jd17]

An interesting article about compression/performance ratio of x265's presets

http://x265.org/performance-energy-consumption-analysis-x265/

Thank you for posting this.

I feel quite confirmed in my personal findings, i.e. --preset slow being the sweetspot for great quality at a still reasonable encoding speed, since the jump to slower is already immense, while the benefit to quality is practically non-existent in my eyes. :)

[Boulder]

There's also at least the value of --max-merge being raised in the slower presets which (at least in my opinion) affect by degrading quality. It will smooth things more --> lower bitrate.

[benwaggoner]

There's also at least the value of --max-merge being raised in the slower presets which (at least in my opinion) affect by degrading quality. It will smooth things more --> lower bitrate.
The key thing about a first pass is that it has to maintain roughly proportional rate control and similar frame/slice type decisions. If it is just 15% less efficient, no biggie. But if it’s efficiency gap varies a lot throughout the clip, than it just isn’t an accurate initial effort to refine.

So a no-slow-firstpass should focus on features that don’t have highly variable impact based on content. Things looking different is probably a good proxy for that, so if you see max-merge changing the character of the encore significantly, you’d probably want to have it st the same value in both passes.

[imhh11]

Hi, thanks to all the developers.

UHD-BD are encoded at level 5.1 but when I select that level, I get this warning.
Should I ignore the warning or just use 5.0 which isn't uhdbd compliant ?

https://extraimage.net/images/2018/07/13/deb6472663fb6bfbb4810a81a638bd3a.png

https://extraimage.net/images/2018/07/13/a578fe21813d0c7a5cabe7edd6c81208.png

Also, is there anything in my setting that I should modify in order to retain grain better?

Thank you and sorry for my English

[Boulder]

Better grain retention = add --no-sao and --no-strong-intra-smoothing.

[imhh11]

Better grain retention = add --no-sao and --no-strong-intra-smoothing.

thanks, I'll try that.


EDIT. Wow its working well . still not perfect but way better than before.

http://screenshotcomparison.com/comparison/116849

http://screenshotcomparison.com/comparison/116846

http://screenshotcomparison.com/comparison/116847



and never mind about my first question, added --vbv-bufsize 160000 --vbv-maxrate 160000 and no more warning.

[Capella Systems]

--dynamic-rd could also be an interesting feature to use in tuning quality/speed. In theory it would allow for faster encoding for easy parts of the video, only switching to higher subme etcetera for the harder bits.

We have implemented this in our transcoding software (Cambria FTC) on top of x265 and it works quite well - a lot of videos are mostly easy to encode and the 80/20 rule is in our favor here.

[RainyDog]

--dynamic-rd could also be an interesting feature to use in tuning quality/speed. In theory it would allow for faster encoding for easy parts of the video, only switching to higher subme etcetera for the harder bits.

But doesn't --dynamic-rd only affect the RD level and only become applicable when VBV rate control is enabled?

Increases the RD level at points where quality drops due to VBV rate control enforcement. The number of CUs for which the RD is reconfigured is determined based on the strength. Strength 1 gives the best FPS, strength 4 gives the best SSIM. Strength 0 switches this feature off. Default: 0.

Effective for RD levels 4 and below.

[Boulder]

and never mind about my first question, added --vbv-bufsize 160000 --vbv-maxrate 160000 and no more warning.

If you are going to watch the result on a PC or a modern HTPC/media player, I think you can safely skip specifying the decoder level and leave the VBV buffer things out. They can only lower the quality of the result.

For better quality including grain retention, I also recommend testing raising qcomp. Try values between 0.6 and 0.8. The bitrate will rise as well though.

[imhh11]

If you are going to watch the result on a PC or a modern HTPC/media player, I think you can safely skip specifying the decoder level and leave the VBV buffer things out. They can only lower the quality of the result.

For better quality including grain retention, I also recommend testing raising qcomp. Try values between 0.6 and 0.8. The bitrate will rise as well though.

yes, I'm watching it on an HTPC /gtx1070.

Thank you so much for your recommendations, I'm already blown away by the difference. :D

[uneedme]

Hello Dudes,

I have tried to encode a video...

From 5 mins and on, the quality is getting worse and worse steadily(it is like auto crf downgrade). and then I split the source video into pieces and encoded them again. Every outputs are good then.

with the same params, I dont know why......

I suspect "--psy-rd 3.2" is the cause or it is a bug?......Or some wrong commands-combinations cause the chaos...

Cheers.


--crf 18.6 -m 2 --me 1 --rd 5 --rc-lookahead 120 --merange 114 --bframes 6 --ref 6 --min-keyint 23 --keyint 240 --max-merge 5 --tu-intra-depth 3 --tu-inter-depth 2 --aq-motion --qg-size 16 --no-sao --no-strong-intra-smoothing --psy-rd 3.2 --scenecut 80 --deblock -1:3 --aq-mode 2 --qcomp 0.8 --aq-strength 0.8 --colorprim bt709 --colormatrix bt709 --transfer bt709 --input-res 1862x768 --fps 24000/1001 --input-depth 10 -D 10 -o "1" --rc-grain --input -

[microchip8]

remove --aq-motion and try again

[LigH]

x265 2.8+47-e2759ae31c36 (https://www.mediafire.com/file/9ra7f7rvi4oszwb/x265_2.8%2B47-e2759ae31c36.7z)

several fixes related to HDR10+ LLC JSON and dependencies between effects of options

[alex1399]

I've got some 60fps video materials with wrongly encoded frame-rate which has lots of duplicated frames.
Encoding those video tends to exceed the bits per second budget.
1. remove duplicated frame
2. increase qcomp
3. set tu-inter-depth=4:tu-intra-depth=4
The 1. and 2. don't perform well, those "still" frames seems to compromised by the poorly unstable sampling rate and ghosting artifacts.
The 3. is interesting.
Instead, how to set x265 parameters to act just like a corrected frame-rate video but farther Intra-coded (I) frames and high level hierarchy Bi-directional predicted (B) frames?