How HEVC/H.265 works, technical details & diagrams

[*.mp4 guy]

Originally Posted by pieter3d

Another thing that will really help is bilinear gradient creation on the intra predictor array sample for large blocks (32x32), which should really help prevent contouring. See http://phenix.int-evry.fr/jct/doc_en...C-K0139-v1.zip

IIRC, vp8 has this prediction mode already, but it doesn't appear to have helped it in this regard at all. To be fair, I've only tested vp8 once, but it looked worse then x264 with all of its psy turned off. Perhaps it will make gradient preservation cheaper, but just like the deblocking filter, I would again be worried about it becoming useless when faced with shallow gradients, which have always been the real problem. Unless the prediction accuracy is better then 8 bits, that is; though of course it wont be.

Of course, the new prediction mode is still a good addition to the spec. In general the spec looks like it is a solid step forward over avc in every category, which is really quite encouraging. However, I still anticipate teething problems out of the gate regarding banding and a few other issues (such as the motion vector prediction inadvertently causing misprediction issues when paired with rd-optimization).

[edit] I just realized that the linked paper actually includes an example of the technique used on a problem sample, which is much more informative then these papers often are. The prediction mode certainly does help, and at the target quality level they have set, it leads to adequate gradient performance. However, I expect it will not help greatly at more common bitrates. Still, its more useful then I expected.

[falocn88]

Hi Peter,
Thanks for information about HEVC. You explanation is very narrative and easy to understand. I would like get some more information about residual decoding, Can u suggest me any documents or JCT-VC proposals?.

Cheers, Keep posting some more topics regarding HEVC.:-)

[pieter3d]

What exactly did you want to know about it? The current scheme has come by interating over literally 100's of jct-vc proposals, so there isn't just one document I can point you to.

[sirt]

pieter3d,

Thanks for your explanations. Could you tell me when an HEVC encoder would be available for public ? I read an HEVC encoder implementation project lead by a Chinese developer has currently been suspended. So it seems such a project is not yet to be as strong as x264. Moreover, regarding as many people don't even have a Blu Ray player nor any H264 decoder engine, which means they simply don't use x264 but XviD instead for their personal encodes, when do you think HEVC will definitely take the lead ? What about HEVC & HEVC Blu ray players ?

[pieter3d]

I know of no 4k or HEVC disc format, likely it will just be online streaming from now on. The only public encoder right now is HM, the reference model. However it is slow an unoptimized, and doesn't plug well into existing tools. I suspect ffmpeg patches with HEVC support aren't that far off now that the standard is finalized.
I expect to see hardware decoders and encoders show up in consumer devices next year sometime.

[hajj_3]

HEVC will probably be used in the possible upcoming 4k bluray format. An encoder isn't really important at the moment, a decoder is. I doubt intel will have hevc decoding in their upcoming haswell chips but you never know.

[Yellow_]

Hi, i started a thread here: http://forum.doom9.org/showthread.php?t=167312 before finding this one, is the source in the link HEVC?

[pieter3d]

Not a chance, there aren't any consumer products with hevc ready yet

[Stephen R. Savage]

What is the status of full resolution chroma in HEVC? The description you gave in the first posts indicates that the standard is currently YV12-only, which is, to say the least, highly disappointing.

[pieter3d]

Originally Posted by Stephen R. Savage

What is the status of full resolution chroma in HEVC? The description you gave in the first posts indicates that the standard is currently YV12-only, which is, to say the least, highly disappointing.

Correct, the spec as of today is just 4:2:0, where chroma is down sampled by a factor of two in both horizontal and vertical direction. However a range extension profile is in the works for 4:2:2 and 4:4:4. But I would challenge anyone to spot the difference between them when watching an HD video. The human eye is not nearly as sensitive to color as it is to texture. So I wouldn't view the omission of the higher fidelity chroma profiles in this first version as disappointing.

Consumer content is pretty much all 4:2:0, the higher chroma modes are more for post-production and archival. I suppose it would also be desirable in a still picture profile, for largely the same reasons.

[Poutnik]

Back to H265 and pleasing 4:4:4 poster at the same time......

Is there any specific difference to H264/AVC,
how H265 addresses encoding video with solid stable colors, like cartoons and computer graphics video?

[pieter3d]

Originally Posted by Poutnik

Back to H265 and pleasing 4:4:4 poster at the same time......

Is there any specific difference to H264/AVC,
how H265 addresses encoding video with solid stable colors, like cartoons and computer graphics video?

Yes, there is a tool called transform_skip that applies optionally to any 4x4 TU (either luma or chroma). You can see the dramatic improvement in quality when images have sharp details like small text here (the powerpoint): http://phenix.int-evry.fr/jct/doc_end_user/documents/9_Geneva/wg11/JCTVC-I0408-v2.zip

At the time this tool was proposed with this document, it was for intra only but in the final spec that distinction isn't made (can be both inter or intra 4x4's).

Note that coding solid colors is easy, it is the sharp transitions that usually create artifacts which this tool can address.

[Guest]

The whole chroma subsampling discussion/debate has been moved to a new thread at the OP's request.

http://forum.doom9.org/showthread.php?t=167428

Please continue that line of discussion in the linked thread and use this thread for technical issues specifically related to HEVC.

[reuven1984]

Hi Peter3d,
I was hoping you can help me with this issue - regarding Transform Unit size restriction:
I'm trying to encode a video with CUs (Coding Units) up to 32x32 pixels, and with TUs (Transform Units) up to 16x16 pixels.
I'm setting MaxCUWidth, MaxCUHeight = 32, MaxPartitionDepth=3, and QuadtreeTULog2MaxSize = 4 (s.t. 2^4 = 16). QuadtreeTUMaxDepthInter,Intra = 3.
The parameter of TULog2MaxSize only controls the Intra blocks. Meaning, in intra blocks I get max. TU = 16x16, but in Inter blocks, for some reason sometimes I get bigger TUs – I have few Inter TU with size = 32x32.
How do I prevent this from happening? How can I restrict both intra AND inter blocks to be with TU <= 16x16?
thanks

[Sulik]

If the inter CU is not coded the not-coded TU is as large as the CU - I don't think you can avoid that (makes no sense to split a non-coded TU into smaller TUs)

[pieter3d]

Originally Posted by Sulik

If the inter CU is not coded the not-coded TU is as large as the CU - I don't think you can avoid that (makes no sense to split a non-coded TU into smaller TUs)

A 32x32 CU can be skipped, in which case the TU doesn't exist. So although there is no split downto 16x16, there are no 32x32 operations to perform

[reuven1984]

pieter3d: I received also in non-split blocks, meaning regular CU (with merge flag = 0), few TUs with size=32x32.
how to avoid that?
(Another thing: These TUs had no coefficients - all of their coefficients were zero, but still it was marked as a 32x32 TU)

Sulik: Did you mean "If the inter TU is not coded..." or "if the inter CU is ..."?

[pieter3d]

So isn't that good enough? If you have a 32x32 TU that is all 0's, you don't have to perform the large transform. You could always treat it as four 0-TUs that are 16x16

[Shevach]

Dear experts

Let me share "HEVC Overview" presentation (113 slides) which I prepared, it's located at:
https://app.box.com/s/rxxxzr1a1lnh7709yvih

Notice that the overview has been reviewed by several experts. Moreover, the overview has been discussed within LinkedIn HEVC/H.265 technical group (see the link
http://www.linkedin.com/groups/Detai...%2Egmp_3724292).

I brought up many aspects of HEVC implementation and I would like to discuss it in Doom9 forum.

[pieter3d]

Originally Posted by Shevach

Dear experts

Let me share "HEVC Overview" presentation (113 slides) which I prepared, it's located at:
https://app.box.com/s/rxxxzr1a1lnh7709yvih

Notice that the overview has been reviewed by several experts. Moreover, the overview has been discussed within LinkedIn HEVC/H.265 technical group (see the link
http://www.linkedin.com/groups/Detai...%2Egmp_3724292).

I brought up many aspects of HEVC implementation and I would like to discuss it in Doom9 forum.

Some comments:

• VPS is optional
• The variable N in CTB size is usually half CTB size, so that the mnemonics 2Nx2N etc make sense.
• May want to mention that intra prediction follows the TU tree
• Perhaps mention spatial neighbor scaling for AMVP
• Transform can actually be implemented with 28-bit precision
• Slide 47, quantization will typically operate on columns because the coeff block is processed first by columns.
• Visual artefacts on large transform blocks - this isn't always case. It is heavily content and bit-rate dependent.
• Should mention deblocking processing order: Vertical first, then horizontal. When done in-loop with CTU decode (single pass), this presents challenges and requires access to neighboring slice parameters (tc/beta offsets).
• Deblocking boundary strength calculation for bs = 1 doesn't mention the rules regarding the MVs point to the same or different pics
• May want to mention that for inter boundaries, chroma is never deblocked.
• May want to mention SAO has susbstantial gains when there is no biprediction
• SAO filter may also cross tile bondaries, not just deblocker.
• MTU matching can still be done with tiles, since the structure may change from pic to pic.
• No mention of fractional CTB rules at right and bottom picture boundaries.
• No mention of the relationship and rules between tiles and slices
• No mention of temporal MV subsampling (16x16 granularity storage)
• No mention of the (near) lossless tools transform_skip, transquant_bypass, PCM
• No mention of custom quant matrices
• No mention of delta QP operation.