x265 HEVC Encoder

[Asmodian]

No one said it was due to the immaturity of the decoders, only that decoder support was the weakest aspect of the HEVC ecosystem (a lot of devices cannot play it).

[LigH]

x265 1.4+515-cd4117a34a19 (https://www.mediafire.com/download/43v73bqnwo6q320/x265_1.4+515-cd4117a34a19.7z)

Now with temporal layering (--[no-]temporal-layers), which allows video streams with up to 2 layers of temporal resolutions.

IIUC, a slower decoder may use the base layer only to decode with a reduced framerate; the sub-layer will consist of B frames which are unreferenced, to be inserted as higher rate frames when a decoder is fast enough to use them.
__

@ Asmodian + benwaggoner:

You rather mean the availability of consumer devices with HEVC decoders?

UHD via DVB is about to become available; the more UHD TV sets are sold, the more HEVC decoders you will find in consumer homes. And establishing a successor of Blu-ray is the next step. Just like introducing HD-ready and FullHD as successors of DVD Video with widescreen TV sets, carefully testing the market acceptance is always a part of the business.

[Asmodian]

@ Asmodian + benwaggoner:

You rather mean the availability of consumer devices with HEVC decoders?

UHD via DVB is about to become available; the more UHD TV sets are sold, the more HEVC decoders you will find in consumer homes. And establishing a successor of Blu-ray is the next step. Just like introducing HD-ready and FullHD as successors of DVD Video with widescreen TV sets, carefully testing the market acceptance is always a part of the business.

Right, decoder support being weak in an ecosystem does not mean the decoders are weak but that "decoder support is weak" or "decoder support is low".

At this moment HEVC support is still somewhat rare. However, as benwaggoner said, this looks to change a LOT in 2015. ;)

[x265_Project]

I recently spent some time with Matt Smith of DigitalTrends, helping him understand the state of HEVC.

http://www.digitaltrends.com/computing/h-265-hevc-encoding-explained/

[x265_Project]

OK x265 fans... we've pushed a number of changes today, making the default psy-rd strength 0.3, but we've also scaled psy-rd down at high QP values (low bit rates), and scaled psy-rd down for larger block sizes. We're anxious to hear your feedback.

[LigH]

So I'll have to schedule another build tomorrow... fine. :)

[xooyoozoo]

and scaled psy-rd down for larger block sizes.

Maybe I missed something, but I don't think that one has been applied yet?

Either way, I've previously tested that patch, and I quite like it. It seems to mitigate the most blatant artifacts.

[x265_Project]

Maybe I missed something, but I don't think that one has been applied yet?

Either way, I've previously tested that patch, and I quite like it. It seems to mitigate the most blatant artifacts.

Good eye! You're right... thanks for catching that. Steve sent the patch to x265-devel mailing list for review, but didn't commit it yet. Thanks for the feedback.

[LigH]

x265 1.4+527-d26268f9dc19 (https://www.mediafire.com/download/dta4y465s9b5oks/x265_1.4+527-d26268f9dc19.7z)

Psy-RD default is now 0.3; adaptive limit based on quantization and block sizes. Disabling with --no-psy-rd[oq] added.

[ckmox]

is it possible for x265 encoding speed to become as fast as x264? im talking about same quality output and not adjusting (trade-off) the preset of x265 for faster encoding to compare with x264 encoding speed, im sorry if this a stupid thing to ask but i do not follow this thread and googling did not answer this question of mine

thanks for the answers in advance

[LigH]

In my opinion, neither will HEVC encoders easily be as fast as AVC encoders because HEVC algorithms are significantly more elaborate (except you skip most algorithms which are more elaborate than AVC), nor will HEVC have "the same quality as" AVC because the reduction of visual complexity works in different ways in several details (except you want to use so much bitrate that the result will be "transparent", without noticable loss, in both cases).

There is still potential to speed up x265, there have been a lot of assembler optimized patches in the recent weeks and months, and there may still more follow. Nevertheless, even if any HEVC encoder used an optimal code, it would still have to calculate a magnitude more equations and decisions than any AVC encoder, if it is not specifically tuned to skip most of them and trade off quality for speed.

[Ajvar]

Hello everyone.
What is faster: 32 bit x265 or avs4x26X + 64 bit x265?

P.S. first pass with --me hex gave 12 fps while 2nd pass on the same settings but --me umh gave 4.5fps. Me range = 57, presets = slow. Is that really how ME mode differs in speed or because of first pass is always faster?
TNX.

[LigH]

Using the 64 bit x265 should be able to be faster because it can use both twice the width and twice the number of CPU internal registers in x86-64 mode, compared to x86 mode of the same CPU.

[Ajvar]

Thank you very much. Will continue using 64bit then.

People, in P.S. above I wrote about drastic performance difference between 1-st pass and 2-nd while I changed umh ot hex. It appears that for some reason even though I wrote to use HEX in 1-st pass it actually used DIA.
Here is CLI code:
--pass 1 --bitrate 2000 --preset slow --me hex --extra:
--pass 2 --bitrate 2000 --preset slow --me umh --extra:

While here is the beginning of log:
options: 1280x720 fps=25/1 bitdepth=8 wpp ctu=64 tu-intra-depth=1 tu-inter-depth=1 me=0 subme=2 merange=57 no-rect no-amp max-merge=1 temporal-mvp early-skip no-fast-cbf rdpenalty=0 no-tskip no-tskip-fast strong-intra-smoothing no-lossless no-cu-lossless no-constrained-intra fast-intra open-gop no-temporal-layers interlace=0 keyint=250 min-keyint=25 scenecut=40 rc-lookahead=25 bframes=4 bframe-bias=0 b-adapt=2 ref=1 weightp no-weightb aq-mode=1 aq-strength=1.00 cbqpoffs=0 crqpoffs=0 rd=2 psy-rd=0.30 psy-rdoq=0.00 signhide lft sao no-sao-non-deblock b-pyramid cutree rc=abr bitrate=2000 qcomp=0.60 qpmin=0 qpmax=51 qpstep=4 ipratio=1.40 pbratio=1.30
Question: is this normal? Is there some rule or something which could force it like that or it is a bug in the chain GUI>Avisynth>avs4x26x>x265-64?
Thank you.

[hector1980]

hi i need an app as pro as MeGUI for X264 to Encode my file into X265 what do yo recommend?

[LigH]

MeGUI. It already supports x265, to some degree.

[hector1980]

i know that but i wonder that is megui the best app for x265 or there is something better available?

[LigH]

Because the set of CLI options in x265 will keep changing during the continuing development, GUIs would have to keep changing just as much to keep supporting it.

But to recommend more alternatives: I believe StaxRip is a little more current and comprehensive at the moment, stax76 announced that recently. Hybrid is also quite up-to-date, usually.

[stax76]

Is there a link to the source code file which defines the default values for --tune grain?

[easyfab]

stax76 -> http://x265.readthedocs.org/en/latest/presets.html#film-grain-retention

[Selur]

@stax76: /source/common/param.cpp holds all defaults&co
else if (!strcmp(tune, "grain"))
{
param->deblockingFilterBetaOffset = -2;
param->deblockingFilterTCOffset = -2;
param->bIntraInBFrames = 0;
param->psyRdoq = 30;
param->psyRd = 0.5;
param->rc.ipFactor = 1.1;
param->rc.pbFactor = 1.1;
param->rc.aqMode = X265_AQ_VARIANCE;
param->rc.aqStrength = 0.3;
param->rc.qCompress = 0.8;
}

[stax76]

Thanks (the values for psyrd/psyrdoq have not changed there)

[Selur]

No, only the defaults changed in the last few weeks.

[foxyshadis]

Thank you very much. Will continue using 64bit then.

People, in P.S. above I wrote about drastic performance difference between 1-st pass and 2-nd while I changed umh ot hex. It appears that for some reason even though I wrote to use HEX in 1-st pass it actually used DIA.
Here is CLI code:
--pass 1 --bitrate 2000 --preset slow --me hex --extra:
--pass 2 --bitrate 2000 --preset slow --me umh --extra:

Question: is this normal? Is there some rule or something which could force it like that or it is a bug in the chain GUI>Avisynth>avs4x26x>x265-64?
Thank you.

Yes, of course it's normal. You have to pass --slow-firstpass for any of your settings to be considered in the first pass, unless you use --preset placebo. Because it's pointless to waste twice as much time encoding when a fast first pass is almost identical to a slow one.

[Ajvar]

Yes, of course it's normal. You have to pass --slow-firstpass for any of your settings to be considered in the first pass, unless you use --preset placebo. Because it's pointless to waste twice as much time encoding when a fast first pass is almost identical to a slow one.

Now I see why it is there! I thought it was some kind of typo and instead of --slow-firstpass there should be --fast-firstpass:p

Don't get why here is no Rep+ or Like buttons.

[Carpo]

Would it be brave or stupid to use x265 at the moment, I am going to change my files from 2 pass to crf19 because after many tests the files are smaller and give the same quality as the 2 pass, so if am going to re-encode them, should I give x265 a go or stay with x264?

I am going to do a test on The Dark Knight Rises to see which looks better, and will go from there, not normally an early adaptor :p

Version of x265 that I have, which I cross compiled on a linux box is 1.4.0.476

git - 1.4+476-16b2c318feb4

I heard that there may be some improvements and changes in the 1.5 release, should I wait for that?

[LoRd_MuldeR]

Version of x265 that I have, which I cross compiled on a linux box is 1.4.0.476

git - 1.4+476-16b2c318feb4

I heard that there may be some improvements and changes in the 1.5 release, should I wait for that?

Version "1.4+476" is the latest "development" version, or at least it's pretty close to the latest commit. I think they are at "+500 something" at the moment ;)

Furthermore, note that version "1.4+X" is simply going to be tagged as "1.5" as soon as they are done with the features/fixes they want to have in the "1.5" release. And then development continues as "1.5+X".

Consequently, if you currently use a recent "1.4+X" development version, then you already have most of the things that will be in "1.5" and thus the changes to the final "1.5" release will probably be rather small for you.

If, on the other hand, you sticked with the previous "1.4" release (and did not use any "1.4+X" development versions), then the upgrade from the "1.4" release to the "1.5" release would be a big step forward.

[Carpo]

cool :) will give it a test after I have done the film to x264 - 6 hours or so :p

[Ajvar]

You can also use 2 pass crf... at least some programs let you do this.

Does it give any benefit? More precise crf matching to the picked one?

[sneaker_ger]

You should be able to do a first pass CRF (i.e. create a stats file while doing crf encode) but not an nth-pass (n>1). Either --crf or --pass/--bitrate will be ignored (too lazy to test right now).

[detmek]

You can also use 2 pass crf... at least some programs let you do this.

Does it give any benefit? More precise crf matching to the picked one?

There is no such thing as 2-pass CRF, unless you mean using CRF for first pass to get stats file. And there is no such thing as "more precise CRF matching to the picked one".

There were some tests in using CRF for first pass for x264 encoding but it provide no benefit comparing to standard 1st pass. Also, using CRF to generate stats file for 1st pass may result in worse quality if bitrate obtained in 1st pass CRF differes too much compared to 2nd pass bitrate.

[RBX]

When using CRF, will slower presets result in better quality or just better compression?

[LigH]

"Maybe".

Slower presets will take more efforts in searching for "redundancies" = compressable parts of the video. That should usually return a better compression. But it is not guaranteed between every pair of presets for every video clip, there may be exceptions.

Whether it appears with higher quality to you, is not certain though. The "constant rate factor" shall limit the quality loss in a metric a computer can measure, but different presets can lose a different kind of details, so the video may appear slightly differently, which may or may not look less annoying to different people watching it.

[x265_Project]

When using CRF, will slower presets result in better quality or just better compression?

Both. Higher quality (slower) presets will definitely compress your video more efficiently, resulting in smaller files at any given bit rate or CRF setting. x265's slowest presets include multiple algorithms designed to improve visual quality, including psycho visual optimizations (psy-rd and psy-rdoq) and greater rate distortion optimization (RDO). So you end up getting both benefits by running slower.

Our tests at various CRF levels using different presets confirm this. We see both a bit rate reduction and a quality improvement at each CRF value when you use slower, higher quality presets.

[LigH]

Well, I do have clips with very slight exceptions (e.g. "medium" bigger than "fast", "placebo" bigger than "veryslow" for foreman_cif). But they are certainly not the norm. In general, you can expect more efforts leading to better efficienty ~ quality preservation per bitrate. But every rule has its exceptions, sometimes...

[Ajvar]

There is no such thing as 2-pass CRF, unless you mean using CRF for first pass to get stats file. And there is no such thing as "more precise CRF matching to the picked one".

There were some tests in using CRF for first pass for x264 encoding but it provide no benefit comparing to standard 1st pass. Also, using CRF to generate stats file for 1st pass may result in worse quality if bitrate obtained in 1st pass CRF differes too much compared to 2nd pass bitrate.
I see. Then I will contact some developers and tell them this and link to this page.
Both. Higher quality (slower) presets will definitely compress your video more efficiently, resulting in smaller files at any given bit rate or CRF setting. x265's slowest presets include multiple algorithms designed to improve visual quality, including psycho visual optimizations (psy-rd and psy-rdoq) and greater rate distortion optimization (RDO). So you end up getting both benefits by running slower.

Our tests at various CRF levels using different presets confirm this. We see both a bit rate reduction and a quality improvement at each CRF value when you use slower, higher quality presets.

But won't bigger count of b-frames for example (and refs) somewhat decrease quality of latest b-frames in a row?
I am talking about bit-rate converting now but is this differ much with CRF?

[xooyoozoo]

But won't bigger count of b-frames for example (and refs) somewhat decrease quality of latest b-frames in a row?

Every encoder decision precludes a different decision that might offer more quality per bit. The whole point of a modern encoder is to do good cost-analysis and make the best choice. In this specific case, `b-adapt` toggles how to analyze consecutive b-frames.

I am talking about bit-rate converting now but is this differ much with CRF?

CRF is heavily context-based, and changing an encoder setting also changes the context of the compressed stream.

There's literally no point in spending time dissecting CRF values.

[foxyshadis]

The only difference between P and B frames is that B frames also get forward references, and there's a separate pbratio that applies to them, making them about 25% smaller on average. (I'm not entirely convinced that this mixing forward references with heavier compression is optimal behavior, and --tune grain lowers it to ~10%, but I haven't ever seen anyone try making a patch that uses different ratios for b-ref and b-non-ref.) Merely having more B frames should only increase quality thanks to having more nearby references, although an encoder's estimation of quality is never perfect, so that isn't always the case.

If you use cu-tree (default from fast on), any referenced cu will automatically get higher quality, so the B-frame quality penalty won't matter nearly as much.

[x265_Project]

I see. Then I will contact some developers and tell them this and link to this page.

CRF rate control is doing something very different than 2 pass rate control. CRF tells x265 to maintain constant quality, regardless of the bit rate. Additional passes aren't needed to do this. It can be done in one pass.

CRF gives you constant quality - but the bit rate will vary as needed throughout the video, and you don't know how high or low the bit rate will go at any point in the video, or how large your resulting file will be.

ABR seeks to maintain a constant bit rate. Normal (single-pass) ABR knows what bits were used so far at any point in the encode, and it uses this historical information to adjust the quality level in order to end up with the overall ABR you were looking for. ABR has only limited visibility as to how many bits will be needed for the frames of video that are coming next (it gets an idea from the lookahead function, but the lookahead works on a limited number of frames in advance of the frame encoders, and it can't predict the # of bits that will be needed for the coming frames with precision). Two pass ABR can look at the statistics saved by the first pass, estimating how many bits will be needed for the coming frames with pretty good precision, factoring that in to settings used for the current frames to give the overall best quality in each section of the video, while maintaining the target average bit rate. Two pass ABR will end up giving you constant quality, at the overall average bit rate you were looking for.


But won't bigger count of b-frames for example (and refs) somewhat decrease quality of latest b-frames in a row?
I am talking about bit-rate converting now but is this differ much with CRF?
Higher quality presets don't force longer b-frame sequences. They just allow the encoder to use more b frames in a GOP if that is optimal. The optimal GOP structure depends on the content.

[benwaggoner]

The only difference between P and B frames is that B frames also get forward references, and there's a separate pbratio that applies to them, making them about 25% smaller on average. (I'm not entirely convinced that this mixing forward references with heavier compression is optimal behavior, and --tune grain lowers it to ~10%, but I haven't ever seen anyone try making a patch that uses different ratios for b-ref and b-non-ref.) Merely having more B frames should only increase quality thanks to having more nearby references, although an encoder's estimation of quality is never perfect, so that isn't always the case.
Intriguing!

Perhaps instead of pb ratio it should be a ref/non-ref ratio, with reference B-frames treated the same as P-frames. The current ratio structure would make more sense for older codecs where B-frames only had a subset of P-frame features. But when a B-frame is a P-frame that can also look in the future...

[srivani]

Hi,

What is different picture inheritance per slice in HEVC. Can anyone explain?

Thanks,
Srivani.

[xooyoozoo]

But when a B-frame is a P-frame that can also look in the future...

Nitpick, but there are no chronological requirements when choosing between P & B frames. The only difference is 1 vs 2 motion vectors allowed per block.

L0 and L1 are actually defined to implicitly include all available reference pics in each (just in different order). We don't see it because x265 signals a cap so that only "traditional" L0 and L1 refs are used.

[Ajvar]

ABR has only limited visibility as to how many bits will be needed for the frames of video that are coming next (it gets an idea from the lookahead function, but the lookahead works on a limited number of frames in advance of the frame encoders, and it can't predict the # of bits that will be needed for the coming frames with precision). Two pass ABR can look at the statistics saved by the first pass, estimating how many bits will be needed for the coming frames with pretty good precision, factoring that in to settings used for the current frames to give the overall best quality in each section of the video, while maintaining the target average bit rate. Two pass ABR will end up giving you constant quality, at the overall average bit rate you were looking for.

Well, does it mean that I can painlessly minimize rc-loockahead to the minimum (1 sec) in the 2-nd pass? If it already has the stats file for the whole video?

P.S. Thanks everybody. I always forget a phrase "you can set any amount of B-frames, encoder won't use them if it doesn't need it". Now I will try to remember this for a longer period of time.

[FranceBB]

@Ajvar... exactly. If you set -ref 16, for instance, it doesn't mean that it will use 'em if there is no need to use 'em.
But my question is addressed to xooyoozoo and srivani:
B frames are frame that are partially predicted from Intra and they can be used in order to predict others B frame - thanks to the motocompensation - and P frame are totally predicted.
So... the point is: a P frame can't look in the future (as benwaggoner said) 'cause when a GOP (Group of picture) meets a P frame, it will end and it will begin another GOP starting from an Intra frame.
But... what are L0 and L1? :confused:


Originally Posted by x265_Project
We're getting close to the v1.5 milestone. Currently, we've turned psy-rd and psy-rdoq on by default (for high quality presets), but I think we'll tone down the default value of psy-rd from 1.0 to 0.3 before we hit v1.5. We want to err on the side of less detail, but fewer artifacts (more accurate motion and color).

Well, if psy it's too high it will introduce artifacts, increasing bitrate, but on the high quality presets - with a proper strength - it's totally worth it. As to psy-rdoq, it's a good point activating it with high presets 'cause on low bitrates it could add junk, but on high bitrate - such as the one of high quality presets - it will make everything look better. My suggestion is to activate 'em only on veryslow and placebo because they use RDO Quantization and it would be great in high frequencies and it will allow me to add static grain on the source without any troubles as, for instance, Dither_add_grain16(var=0.15,soft=100) with dither tool 16 bit on avisynth.

[xooyoozoo]

But my question is addressed to xooyoozoo and srivani:
B frames are frame that are partially predicted from Intra and they can be used in order to predict others B frame - thanks to the motocompensation - and P frame are totally predicted.

I'm not sure what you mean. Prediction is prediction. Both P&B predicts from an arbitrary set of references. It doesn't matter how those references came to be as long as they exist when the decoder tries to decodes the P or B frame.

So... the point is: a P frame can't look in the future (as benwaggoner said) 'cause when a GOP (Group of picture) meets a P frame, it will end and it will begin another GOP starting from an Intra frame.

That's not how P frames are defined, and that's also not how any encoder works. You can have a multitude of P frames in an intra-period.

But... what are L0 and L1? :confused:

L0 and L1 are the reference lists. A block is only allowed 1 motion vector per list, so the reference lists become a key distinction between P and B frames, as the former can only access L0 while the latter can access L0+L1.

[x265_Project]

B frames are frame that are partially predicted from Intra and they can be used in order to predict others B frame - thanks to the motocompensation - and P frame are totally predicted.

To add to what xooyoozoo said, B and P frames aren't totally "predicted" (which sort of implies inter prediction, referencing other frames as opposed to intra prediction, referencing another block (CU) in this frame). I think the confusion comes from 2 places. First, the P in "P frames" stands for "predicted" and the B in "B frames" stands for "bi-directionally predicted". Second, I frames are Intra only. The reverse isn't true. P and B frames aren't "Inter-only". Both B and P frames can use intra-prediction when that is the best method to encode each CU (although b-intra is disabled by default in x265). The more options the encoder has, the more efficiently it can encode, but the longer it may take to evaluate all the options.



Well, if psy it's too high it will introduce artifacts, increasing bitrate, but on the high quality presets - with a proper strength - it's totally worth it. As to psy-rdoq, it's a good point activating it with high presets 'cause on low bitrates it could add junk, but on high bitrate - such as the one of high quality presets - it will make everything look better. My suggestion is to activate 'em only on veryslow and placebo because they use RDO Quantization and it would be great in high frequencies and it will allow me to add static grain on the source without any troubles as, for instance, Dither_add_grain16(var=0.15,soft=100) with dither tool 16 bit on avisynth.
psy-rd is only in effect with rd-level 3 or higher (medium preset or slower), and psy-rd is only in effect with rd-level 4 or higher (slow preset or slower). We have implemented a new algorithm to try to prevent psy-rd artifacts at low bitrates (reducing psy-rd strength automatically when QP > 42).

[LigH]

x265 1.4+544-f52d723d8c46 (http://www.mediafire.com/download/whvu8btarlikc0c/x265_1.4+544-f52d723d8c46.7z): Mostly small bug fixes, improvements and source nits (if I didn't miss any important change).

[FranceBB]

Thanks to both xooyoozoo and x265_Project, now all doubts are cleared ^_^

@LigH... tomorrow I will try it.

[x265_Project]

We're pleased to announce that x265 has reached another milestone! The v1.5 release of x265 has major improvements in Main10 compression efficiency and performance over the 1.4 release, general improvements in Main performance. Psycho-visual optimizations are now enabled by default in the presets which can support it (medium, slow, slower, veryslow and placebo).

Feature additions:

* analysis re-use features have been completed
* rate control zones have been introduced
* --tune grain introduced
* deblocking tC and Beta offsets are now configurable
* denoise is seperately configurable for inter and intra CUs
* frame based CSV logging has been improved
* New support for VTune task profiles


Presets and defaults:

ultrafast no longer disables the deblocking loop filter
psy-rd defaults to 0.3 (was 0, disabled)
psy-rdoq defaults to 1.0 (was 0, disabled)
aq-mode defaults to 1 (was 2, auto-variance)
4:2:2 and 4:4:4 encodes no longer generate compliance warnings


API changes:

param.rc.rateTolerance has been removed and replaced with a simpler param.rc.bStrictCbr flag.

--log-level debug is now --log-level 4 instead of --log-level 3. A new 'frame' log level was inserted at level 3 in order to support frame level CSV logging without also enabling frame level console logging.
Using the string name 'debug' is unambiguous as its behavior has not changed.

The online documentation has all the details:
http://x265.readthedocs.org/en/1.5/

[lotusgg]

What is the CLI for a 4:4:4 encode and what are the benefits of it?