x265 HEVC Encoder

[LigH]

MediaFire download of v1.9+54 appears to be fixed.

[Ma]

Across the board we saw that performance when we don't specify the pools option on the command line (or equivalently specify --pools +,+) gave nearly better or close performance to specifying --pools 16,16. We did, as expected, notice that when a video was run for the first time, the fps is low because you end up incurring too many page faults but when you do repeated runs and access the video from the RAM, fps stabilizes.

Did you notice more threads on NUMA-node 0 than on NUMA-node 1? I think it is a problem that bothers littlepox.

[Blue_MiSfit]

Hey everyone - I'm working on some really demanding stuff that requires encoding 4096x4096p60. We have to decode in software on a Core i5.

Is there anything I can do to make the decode faster aside from using --tune fastdecode and limiting peak bitrate with VBV?

[nevcairiel]

Make sure WPP is on for better multithreading, I suppose.

[Blue_MiSfit]

indeed it is.


Disregard the below, this was operator error - I specified -r after -i



Also related - I encode elementary streams with x265.exe, then mux to mp4 with ffmpeg. The result (according to mediainfo) is VFR. How can I make this always constant frame rate? I want to limit the number of variables I'm looking at :)

[LigH]

MediaInfo reports videos as VFR already if just the container is able to support VFR (by assigning timestamps or durations to every frame). It doesn't scan the whole file to know if all frame durations are equal.

[x265_Project]

Hey everyone - I'm working on some really demanding stuff that requires encoding 4096x4096p60. We have to decode in software on a Core i5.

Is there anything I can do to make the decode faster aside from using --tune fastdecode and limiting peak bitrate with VBV?
Hey Derek,
Well, you can get a faster decoder; UHDcode. :) On a pure CPU software basis, it's slightly faster than FFMPEG/OpenHEVC (according to our internal testing). Plus, it's OpenCL accelerated, which gives another performance boost on capable platforms.

The challenge is primarily related to the bit rate. The more bits, the more entropy decoding, and that will become the bottleneck. On a Core i5-4760K desktop, at 10 Mbps, we can decode 4K24 or 4K30 content at around 60 FPS. This drops to around 35 FPS at 35 Mbps, and 28 FPS at 80 Mbps. 4K60 content should be a bit faster, as we should have more merge/skip blocks.

I'll email you to follow up. Tom

[Ely]

On a Core i5-4760K desktop, at 10 Mbps, we can decode 4K24 or 4K30 content at around 60 FPS.

I'm having trouble following.. Don't 4K24 and 4K30 mean 4K @ 24fps and 4K @ 30fps respectively ?

edit : nvm, I guess you meant you can decode 4K 10mbps at 60fps, but specifically naming 4K24 and 4K30 had me confused for a moment :P.

edit2 : nope, you were right to name everything like you did, since I presume that 4K24 @ 10mbps is different from 4K30 @ 10mbps. nvm what I said :D . I should think more before posting.

[MeteorRain]

edit2 : nope, you were right to name everything like you did, since I presume that 4K24 @ 10mbps is different from 4K30 @ 10mbps. nvm what I said :D . I should think more before posting.

Just for clarification,
4k24@10m -> 417kbit per frame
4k30@10m -> 333kbit per frame
4k60@20m -> 333kbit per frame, with more bitrate used on scene cut and more skip ctu/mb on p/b frames.

[littlepox]

We ran several tests on a dual-socket E5-2670 v3 machine running windows server encoding 1080p anime clips, and some 4K animation videos. Across the board we saw that performance when we don't specify the pools option on the command line (or equivalently specify --pools +,+) gave nearly better or close performance to specifying --pools 16,16. We did, as expected, notice that when a video was run for the first time, the fps is low because you end up incurring too many page faults but when you do repeated runs and access the video from the RAM, fps stabilizes.

Ma/littlepox: I suspect that the issue is something to do with either the warm-up issue as your first run always had no --pools option, or something with your server setup.

Acknowledged, thank you for your effort to investigate this; currently I think we can just set --pools "N,N" for our cases; it's not a big problem.

[pingfr]

I'm playing a bit around with various options of x265 and aiming at quality encodes, just had 2 quick questions:

1) In regards of the --lossless option, does it matter whether using --preset veryslow or --preset placebo in cunjunction with the --lossless parameter and does it make a slight difference if --lossless has "priority" over either veryslow or placebo presets?

2) In regards to --aq-mode, values can be set between 0 (disabled) up to value of 3. I can't fathom which value grants the best picture quality? is it 0 with AQ disabled completly? is it the default value of 1 or is it worth raising the value to 2 or even it's maximum at 3?

Once again: my main concern here is the resulted encoded quality, which needs to be top notch or as clear as the source (Blu-Ray discs in my case), neither the encoding speed nor the targetted file size matter at this point.

Thanks in advance.

Edit: My current favored parameters at the moment simply are "--psy-rd 2 --tune grain --crf 17 --preset placebo".

[x265_Project]

I'm playing a bit around with various options of x265 and aiming at quality encodes, just had 2 quick questions:

1) In regards of the --lossless option, does it matter whether using --preset veryslow or --preset placebo in cunjunction with the --lossless parameter and does it make a slight difference if --lossless has "priority" over either veryslow or placebo presets?

2) In regards to --aq-mode, values can be set between 0 (disabled) up to value of 3. I can't fathom which value grants the best picture quality? is it 0 with AQ disabled completly? is it the default value of 1 or is it worth raising the value to 2 or even it's maximum at 3?

Once again: my main concern here is the resulted encoded quality, which needs to be top notch or as clear as the source (Blu-Ray discs in my case), neither the encoding speed nor the targetted file size matter at this point.

Thanks in advance.

Edit: My current favored parameters at the moment simply are "--psy-rd 2 --tune grain --crf 17 --preset placebo".
It doesn't make any sense to take Blu-ray disc content (high bit rate H.264 or VC-1), and losslessly encode it to HEVC. The resulting file will be much larger, but the quality would be identical to what you started with.

AQ won't matter at all if you're encoding lossless, nor will any other "quality" algorithms like psy-rd, cu-tree, or tune settings.

AQ reduces the quality in areas of fast motion (where it will be hard to perceive details, and the picture may be blurred anyhow due to motion blur from the camera), increasing quality where you are more likely to notice it. The best AQ setting depends on the type of content you're encoding. With a lot of fast motion scenes, you might benefit from a higher AQ setting. Deepthi (Nandaku2) and other experts here may be able to give better guidance.

I wouldn't waste time using placebo mode.

[foxyshadis]

1) In regards of the --lossless option, does it matter whether using --preset veryslow or --preset placebo in cunjunction with the --lossless parameter and does it make a slight difference if --lossless has "priority" over either veryslow or placebo presets?

Presets are mostly speed/size tradeoffs, so in lossless, a slower preset just means a tiny bit smaller file. They mostly work that way in non-lossless, as well, though quality may shift slightly.

There's usually a sweet spot where the file size reduction isn't worth the extra time spent, depending on your application and CPU power, but placebo is probably going to be way over the line for a long time.

[MeteorRain]

There's usually a sweet spot where the file size reduction isn't worth the extra time spent, depending on your application and CPU power, but placebo is probably going to be way over the line for a long time.

Exactly. That being said, you'd spend 5x the time (probably couple hours or days) on encoding just get 1% smaller file size, or 1% quality improvement in return.

[pingfr]

AQ reduces the quality in areas of fast motion (where it will be hard to perceive details, and the picture may be blurred anyhow due to motion blur from the camera), increasing quality where you are more likely to notice it. The best AQ setting depends on the type of content you're encoding. With a lot of fast motion scenes, you might benefit from a higher AQ setting. Deepthi (Nandaku2) and other experts here may be able to give better guidance.

So I might as well encode with AQ disabled, (--aq-mode=0) if what I'm after is top notch quality above anything else?

[Ma]

So I might as well encode with AQ disabled, (--aq-mode=0) if what I'm after is top notch quality above anything else?

I tried to find right options to encode "Mad Max: Fury Road". I end up with '--preset veryslow --aq-mode 3 --deblock -1'.

You can make some tests encoding on short samples and compare results.

[littlepox]

IMHO, using parameters like --preset to trade time for quality is very inefficient. You are wasting visible things like hours of encode and dollars of electric bill for invisible things.
My own choice is to make heavy testing on rate-control parameters like aq, psy, qcomp, ipbratio. When combined properly, they can improve your quality dramatically without increasing your encoding time.

Sadly to say that currently the x265's default tuning is very poor especially in high bit-rate, that's why people saying 'I don't care bit-rate or encoding time, give me quality' shall typically use x264. However, my own test on parameter combinations have shown an comparable visual quality to x264, in anime encoding.

I have written some materials on how to systematically test for tuning options, but in Chinese:
http://vcb-s.nmm-hd.org/Dark%20Shrine/%5BVCB-Studio%5D%5B%E6%95%99%E7%A8%8B11%5D%E7%BC%96%E7%A0%81%E5%99%A8%E5%8F%82%E6%95%B0%E7%A0%94%E5%8F%91%E6%96%B9%E6%B3%95/

[pingfr]

In my case, the challenge in order to find the proper encoding parameters is associated with the fact I'm doing "batch encoding" over night using the server farm at work (thus, I'm not personally paying for the hardware costs nor the electricity bills).

But what is devastating is setting a batch with several encodes over night, only to come work next morning to realize encodes are all blocky, grainy, under optimized and such.

That is the reason why I'm looking for a top notch standard set of parameters I can pass to x265 without worrying too much and always use these "safe" parameters to every single encoded/archived movie.

Back in the x264 world for 1080p Blu-Ray quality content, my parameters always were --preset slow --subme 8 --bitrate 8000 --aq 1 --2 pass --ref 5.

But in the x265 world this seems rather unpractical, as the x264 params can't just be "carried over" from x264 to x265 hence my choice of --psy-rd 2 --tune grain --crf 17 --preset placebo.

[sneaker_ger]

So I might as well encode with AQ disabled, (--aq-mode=0) if what I'm after is top notch quality above anything else?
No, it's about bit distribution. Constant bitrate (on frame or even block level) does not equal top notch quality. Like x265_Project described it's supposed to reduce bitrate where you do not notice it and move the bits somewhere where you would notice the difference. Lossy encoding is all about human perception.

[pingfr]

No, it's about bit distribution. Constant bitrate (on frame or even block level) does not equal top notch quality. Like x265_Project described it's supposed to reduce bitrate where you do not notice it and move the bits somewhere where you would notice the difference. Lossy encoding is all about human perception.

So, should aq mode be raised to 3? or should it be kept at default value but then use --aq-strength 3.0?

Or is it --aq-mode 3 combined with --aq-strength 3.0 altogether?

[sneaker_ger]

Those are not settings where more = better. You have to test and tune them to your source material. If you don't want to spend any time testing you might want to simply stick to the default values.

[pingfr]

Default values being --aq-mode 1 --aq-strength 1.0 then?

[sneaker_ger]

Default being whatever x265 uses for the preset/tune combination you are using. These are subject to change.

[pingfr]

That doesn't leave much room for improvement optimization-wise.

[LigH]

As long as most of the room for optimizations is still in the responsibility of the developers, the testers will have to catch up. I still avoid talking about users here... ;)

[pingfr]

@LigH: What about power-users then? :p

[LigH]

Optimism FTW! :D

[benwaggoner]

So, should aq mode be raised to 3? or should it be kept at default value but then use --aq-strength 3.0?

Or is it --aq-mode 3 combined with --aq-strength 3.0 altogether?
You don't raise aq-mode, just set the one you want to have. The difference between 2 and 3 is that 3 shifts relatively more bits to darker areas of the frame, which results in better reproduction on LCD displays where subtle artifacts in blacks can be very visible. If everyone still used CRT, we wouldn't have an aq-mode 3 :). Perceptual uniformity based on gamma is a first order approximation, but not always accurate in the real world.

For another example, you wouldn't use aq-mode 3 with HDR encoding, since the PQ curve is more perceptually uniform than gamma, and there are relatively a whole lot more code values in black.

[pingfr]

So it is --aq-mode 3 with default --aq-strength 1.0 on 1080p/720p "quality" encodes from retail regular Blu-Ray sources?

The --aq-mode 3 would remove "ugly blocks" and "banding effects" that can be perceived in background darker areas of a scene? or the banding perceived in the sky or in the water/rain ripples or the color uniformity from say, the sea?

Do I get that right? :)

[benwaggoner]

So it is --aq-mode 3 with default --aq-strength 1.0 on 1080p/720p "quality" encodes from retail regular Blu-Ray sources?

The --aq-mode 3 would remove "ugly blocks" and "banding effects" that can be perceived in background darker areas of a scene? or the banding perceived in the sky or in the water/rain ripples or the color uniformity from say, the sea?

Do I get that right? :)
If the Blu-ray already had artifacts in black, it probably won't help much; maybe some if mixed with a 720p downscale. But aq-mode 3 can help avoid introducing artifacts in black due to recompressing to a lower bitrate.

It shouldn't have any impact on color uniformity; just on distribution of luma bits. If you want more bits in chroma, use negative values with cbqpoffs/crqpoffs

[sneaker_ger]

If you want to avoid banding try >= 10 bit encoding.

[benwaggoner]

If you want to avoid banding try >= 10 bit encoding.
Yes, that will help as well, if the presence of a 10-bit encoder can be counted on. Using aq-mode 3 can still be valuable, however, as detail and artifacts are generally more visible in the low luma range than high luma on typical displays using Rec. 709.

Note that HEVC is a lot better than H.264 when it comes to introducing banding, even in 8-bit. I'm more worried about artifacts and loss of shadow detail than classic banding.

[pingfr]

At the moment I'm only encoding Blu-Ray discs from 2007-2011, so from what I understood, a 8-bit encoder is appropriate.

Not sure what/if using a 10-bit (let alone 12-bit) encoder would serve any purpose at all?

[sneaker_ger]

10 bit encoding reduces rounding errors in the encoding algorithms so it is useful for avoiding banding even for 8 bit sources.

[benwaggoner]

At the moment I'm only encoding Blu-Ray discs from 2007-2011, so from what I understood, a 8-bit encoder is appropriate.

Not sure what/if using a 10-bit (let alone 12-bit) encoder would serve any purpose at all?
If you aren't doing any scaling or other image processing, yeah, you aren't likely to see any benefit to encoding in >8-bit with HEVC. And you'd take a compatibility and encode-time hit.

Things were different with H.264, where you could still see up to a 10% bitrate reduction, but HEVC seems to have solved that problem well in other ways.

[pingfr]

Wait, what? Are you saying that using a x265-10bit.exe with --aq-mode 3 as encoding parameters has "high chances" to improve perceptible subjective visual quality by reducing banding even if the source material is either a DVD from 2005 era or a rather old Blu-Ray from circa 2010 over a "regular" aka 8-bit x265.exe build?

Edit: I was replying to sneaker_ger, not to benwaggoner. Added for clarity. :)

[sneaker_ger]

Not reduce banding of the source, introduce less (new) banding than the 8 bit encoding. As you can see by benwaggoner's post some people think it's not useful with HEVC. I disagree but that's subjective. Test it and make your own opinion.

http://forum.doom9.org/showpost.php?p=1751744&postcount=3086

[benwaggoner]

Wait, what? Are you saying that using a x265-10bit.exe with --aq-mode 3 as encoding parameters has "high chances" to improve perceptible subjective visual quality by reducing banding even if the source material is either a DVD from 2005 era or a rather old Blu-Ray from circa 2010 over a "regular" aka 8-bit x265.exe build?
No, I'm saying that 8-bit v. 10-bit HEVC isn't likely to make a difference for an 8-bit source.

I am saying that aq-mode 3 is likely to allow for a lower ABR without visible artifacts. Really, think of it as a perceptual optimization for adaptive quant. In will quantize low luma blocks less than high luma blocks (based on DC coefficient? Not sure how it's implemented). This is more perceptually accurate, since the difference between adjoining blocks of Y'=16 and Y'=17 can be visible, much more so than with adjoining blocks of Y'=216 and Y'=217.

Video encoding still very much assumes the old presumption that the gamma of CRT displays is a perfect inverse match to the human visual system, and so most codecs weigh the difference between code values as identical error regardless of what the values are. And that is a decent first-order estimation. But with modern displays, we can see variation in black more than white, so it is more perceptually uniform to treat errors in low luma as a greater visual distortion than in high luma.

So, aq-mode 3 will use lower QP in darker areas, trading off higher QP in brighter areas. PSNR will go down, but perceptual quality will go up.

[pingfr]

@sneaker_ger: I can see --psyrd 2 and 10 bit as both clear benefits over a regular 8 bit encoder, let alone without --psyrd 2, but what kind of source material was this to begin with? was it an excerpt from a larger video clip?

[sneaker_ger]

The source is included in the post in case you are interested in using it for your own tests. (--psy-rd 2 is the default since x265 1.9, btw)

[pingfr]

@ben_waggoner: I know ben. I really was trying to reply to sneaker_ger on that one.

Looks like I'm definitely set on --preset placebo --crf 17 or 18 with --psy-rd 2 and --aq-mode 3 so far.

I will stay away from 10 bit or 12 bit encoders for now as there seems to be a "compatibility mismatch" issue going on, even though I can clearly see a *highly* perceptible visual improvement in regarding to what I call the "sky banding" or "deep sea banding" effects.

Anything else I should "turn on" or tweak at this point?

Always remember, my sources materials all are retail oldies Blu-Ray pre 2010~2011, neither the encoded .hevc output size nor the computational time spent to encode a movie matters at this point. Quality of the resulting .hevc output is my priority concern at this point.

Thanks to both of you. :)

[pingfr]

@sneaker_ger: So no need to specify --psy-rd 2.0 manually anymore with 1.9+73-6d06de58c316 and onwards?

[Boulder]

No, I'm saying that 8-bit v. 10-bit HEVC isn't likely to make a difference for an 8-bit source.What about the cases in which you have an 8-bit source but you process it as 16-bit and feed that to x265?
So, aq-mode 3 will use lower QP in darker areas, trading off higher QP in brighter areas. PSNR will go down, but perceptual quality will go up.Does this mean that aq-mode 3 is recommended as the general solution for regular movies (SD or HD)? I've been using the default aq-mode 1 so far.

[Boulder]

And you'd take a compatibility and encode-time hit.Wouldn't 10-bit HEVC actually be more compatible?

[pingfr]

@Boulder: Be compatible with...?

[sneaker_ger]

@Boulder:
More compatible than what? 10 bit AVC? Yes. 8 bit HEVC? No.
So far I have not seen any HEVC decoder that can decode 10 bit but not 8 bit.
But a lot of decoders can do 10 bit. ffmpeg/LAV, Windows 10, many TVs (10 bit HEVC is required for UltraHD), UltraHD BluRay players, Amazon Fire TV, recent top-smartphones. It's coming everywhere similar to how first AVC decoders were baseline profile with low resolution and today smartphones can decode untouched BluRay streams without breaking a sweat.

@sneaker_ger: So no need to specify --psy-rd 2.0 manually anymore with 1.9+73-6d06de58c316 and onwards?
Correct.

[pingfr]

@sneaker_ger: Thanks for the tip in regards to --psy-rd 2.0 being the expected default behaviour in 1.9+.

Do you know by any chance if there's any other params I should raise/use/tweak with my --preset placebo --crf 17 & --aq-mode 3 to at least be on par with what x264 offers in terms of "subjective quality"?

[LigH]

People are still addicted to placebos... :confused: — utility's best friend. :rolleyes:

[pingfr]

@LigH: The placebo preset is the *only* acceptable way to go for me until x265 can match x264 in terms of retaining the grain/details, see issue #122:

https://bitbucket.org/multicoreware/x265/issues/122/loss-of-details-compared-to-x264

So any added extra tweaks leaning towards optimization and subjective quality are a major concern. :)

Edit: Needless to say, the issue is still opened since april 2015... that's nearly a year ago.

[sneaker_ger]

Preset placebo does not magically fix the problems you encounter with other presets. If you think x264 is still better then the logical decision would be to stick to x264 for the time being.