How HEVC/H.265 works, technical details & diagrams

[sdancer75]

Originally Posted by pieter3d

Because the smallest CU is 8x8, there are four 4x4 transform units in a 2x2 arrangement when TX size is set to 4x4. There is never a single 4x4 transform by itself in a CU.

TUs and PUs only match size in intra blocks. For example inter blocks can have non-square PUs with various different sized TUs.

Thank you !


I am a little confused with all this stuff.

1) Can you please make a simple text graph of the 4x4 TUs in 2x2 arrangement ?
2) Does CUs are consisting ONLY from TUs and PUs ?
3) Does TUs and PUs are containing with Luma and chroma data ?

[pieter3d]

Originally Posted by sdancer75

Thank you !


I am a little confused with all this stuff.

1) Can you please make a simple text graph of the 4x4 TUs in 2x2 arrangement ?
2) Does CUs are consisting ONLY from TUs and PUs ?
3) Does TUs and PUs are containing with Luma and chroma data ?

1) The forum here doesn't let me get creative with ascii art,
but think of it similar to the way a 16x16 JPEG/MPEG-2 macroblock has four 8x8 DCT blocks (in luma):
http://slideplayer.com/slide/4759570...ompression.jpg


2) A CU always contains one or more PUs and one or more TUs.

3) Luma and chroma are grouped together when talking about TUs and PUs.

[sdancer75]

Originally Posted by pieter3d

1) The forum here doesn't let me get creative with ascii art,
but think of it similar to the way a 16x16 JPEG/MPEG-2 macroblock has four 8x8 DCT blocks (in luma):
http://slideplayer.com/slide/4759570...ompression.jpg


2) A CU always contains one or more PUs and one or more TUs.

3) Luma and chroma are grouped together when talking about TUs and PUs.

Thank you

So for the question (2) the only data units a CU can contain is TUs and PUs, and for question (3) luma & chroma are grouped together in case we are talking about for TBs and PBs (i suppose not in case of blocks correct ?)

Regards,

[pieter3d]

"Block" is pretty generic, kind of depends on context. It's not an official term in the spec.

[foxyshadis]

Originally Posted by sdancer75

@pieter3d Please clarify this to me.

In your very first post you say "HEVC supports four transform sizes: 4x4, 8x8, 16x16 and 32x32.", but inside "JCT-VC High Efficiency Video Coding (HEVC) Test Model 16 (HM 16) Improved Encoder Description" at Paragraph 4.2.5 “Transform unit (TU) and transform tree structure” it says "The transform unit (TU) is a square region of size 8x8, 16x16 or 32x32 luma samples/pixels defined by a quadtree partitioning of a leaf CU.".

From the specification I understand that there is no 4x4 size TU size. Is that correct ?

That's not the spec, although later in that same paragraph the 4x4 transform blocks are mentioned. The spec is very clear:

transform block: A rectangular MxN block of samples on which the same transform is applied.
....
transform unit: A transform block of luma samples of size 8x8, 16x16, or 32x32 or four transform blocks of luma samples of size 4x4, two corresponding transform blocks of chroma samples of a picture in 4:2:0 colour format;

And following that, a bunch of special cases for 4:2:2 and 4:4:4. It's important to note the difference between TUs (which are either one large or 4 4x4 blocks, in luma) and the TBs; additionally, 4:2:2 chroma TUs always consist of two square TBs. (The "rectangular" wording is leftover from when non-square transforms were one of the proposals.)

[sdancer75]

Originally Posted by pieter3d

"Block" is pretty generic, kind of depends on context. It's not an official term in the spec.

Thank you for clarifying this. I just read the text below in the
https://codesequoia.wordpress.com/20...-cb-pb-and-tb/
and when I read "logical unit" it sounded to me a little generic like logical units in Win32 API that takes a physical form somewhere in the implementation way.

We need to understand an important naming convention here. In HEVC standard, if something is called xxxUnit, it indicates a coding logical unit which is in turn encoded into an HEVC bit stream. On the other hand, if something is called xxxBlock, it indicates a portion of video frame buffer where a process is target to.

[sdancer75]

Originally Posted by pieter3d

2) A CU always contains one or more PUs and one or more TUs.

pieter3d

The PUs and TUs are living together inside a CU in parallel or TUs are always inside the PUs like the image below ? I mean what's the hierarchical-block structures inside the CTU ?

https://imgur.com/a/VWMbWha

[pieter3d]

They are parallel. It is possible to have a tu larger than pu with inter CUs

[foxyshadis]

Originally Posted by sdancer75

Looking inside the `Void TEncSlice::encodeSlice` of the official HM Test software, I found that in a loop the encoder is trying to encode every CTU inside a slice segment ie 512 CTUs in a unique slice in my script example.

`for( UInt ctuTsAddr = startCtuTsAddr; ctuTsAddr < boundingCtuTsAddr; ++ctuTsAddr )`

If I would like to encode the frame line by line (ie to implement wavefront encoding) should I modify the slice segment to be equal with the frame width or is a wrong approach ?

WaveFront is already implemented in HM, and you can't encode line by line, only CTU by CTU. HM's is not amazingly efficient, since it saves and reloads the context with every CTU row in a single thread, but actual thread synchronization is very difficult to get right. For actual threading, you would encode a row at a time, with a pool that gets released after the first CTU's context is released each row, but you'd need to rearchitect a lot more than just changing the for loop.

[foxyshadis]

Originally Posted by sdancer75

Hi

Yes I mean CTU lines or rows and not pixel lines sorry..

You said that HM saves and reloads the context with every CTU row. Can you point me the code inside HM that this code exists ?

Sure. Line 744-759 of the same file, TEncSlice.cpp, loads the context:

  744     else if ( ctuXPosInCtus == tileXPosInCtus && m_pcCfg->getWaveFrontsynchro())
  745     {
  746       // reset and then update contexts to the state at the end of the top-right CTU (if within current slice and tile).
  747       m_pppcRDSbacCoder[0][CI_CURR_BEST]->resetEntropy();
  748       // Sync if the Top-Right is available.
  749       TComDataCU *pCtuUp = pCtu->getCtuAbove();
  750       if ( pCtuUp && ((ctuRsAddr%frameWidthInCtus+1) < frameWidthInCtus)  )
  751       {
  752         TComDataCU *pCtuTR = pcPic->getCtu( ctuRsAddr - frameWidthInCtus + 1 );
  753         if ( pCtu->CUIsFromSameSliceAndTile(pCtuTR) )
  754         {
  755           // Top-Right is available, we use it.
  756           m_pppcRDSbacCoder[0][CI_CURR_BEST]->loadContexts( &m_entropyCodingSyncContextState );
  757         }
  758       }
  759     }

and line 861-864 saves it:

  861     if ( ctuXPosInCtus == tileXPosInCtus+1 && m_pcCfg->getWaveFrontsynchro())
  862     {
  863       m_entropyCodingSyncContextState.loadContexts(m_pppcRDSbacCoder[0][CI_CURR_BEST]);
  864     }

Originally Posted by sdancer75

Wavefront synchro implemented or wavefront algorithm itself ? If the algorithm is actually implemented what's the point when it is used as a single thread ?

Wavefront synchro is HM's name for the Wavefront algorithm, they are one and the same. Why? Proof that it can work, and is decodable based on the spec, is all that's necessary for a proof of concept encoder. In theory, correct multithreading of the Wavefront code should produce identical output, but this does it without the complexity of threaded code. Only minimal efforts were ever put into making HM high-performance, even fewer than the speed overhauls that JM eventually had.

You can see early versions of x265 if you want to see HM code with actual multithreaded Wavefront processesing, before the HM code was ripped out and entirely reimplemented.

[LigH]

https://bitbucket.org/multicoreware/x265/wiki/Home

Clone an early revision using Mercurial (hg).

[LigH]

Mercurial doesn't care much about "version" tags. Use revision numbers or commit hashes. Revision 0 has (brief) commit hash 09fe40627f03.

[sdancer75]

hi there,

Is there anyway to create two different sets of TEncSbac classes just before the compressSlice(pcPic) ie one for the 1st half of the pic and the other for the 2nd half of the pic (I have already done this), and finally join them just after the end of the mentioned function?

The TEncSbac is a class and I wonder where the actual encoded data exists ! I need this to create two independent compression processes.

Regards,

[sdancer75]

Hi,

I noticed a change from HM Reference software v10 to the latest v16 in the Compressing/Encoding CUs. Inside the compressSlice, the older versions are calling compressCU/encodeCU while the newer versions are calling compressCtu/encodeCtu.

So, since both of them ie compressCU & CompressCtu are sequentially calling xCompressCU as well as encodeCU & encode Ctu are sequentially calling xEncodeCU respectivelly, is there any real difference in the compress and encoding procedure ?

[sdancer75]

How CUs are encoded at the tiles boundary since there is no information available from the neighboring sample (not encoded yet)?

[pieter3d]

It's the same process as on frame boundaries. there is also a flag that lets you optionally enable use of information from other tiles if those tiles were encoded previously (i.e. left or above tiles).

[sdancer75]

Originally Posted by pieter3d

It's the same process as on frame boundaries. there is also a flag that lets you optionally enable use of information from other tiles if those tiles were encoded previously (i.e. left or above tiles).

Thank you for you answer. Is that possible to point me this procedure inside the HEVC HM Test model ?

Regards,

[pieter3d]

Check these two:
https://hevc.hhi.fraunhofer.de/trac/...icSym.cpp#L482
https://hevc.hhi.fraunhofer.de/trac/...taCU.cpp#L1009

That should give you a starting point to dig in to.

[sdancer75]

Originally Posted by pieter3d

Check these two:
https://hevc.hhi.fraunhofer.de/trac/...icSym.cpp#L482
https://hevc.hhi.fraunhofer.de/trac/...taCU.cpp#L1009

That should give you a starting point to dig in to.

thanks

[sdancer75]

Originally Posted by pieter3d

Check these two:
https://hevc.hhi.fraunhofer.de/trac/...icSym.cpp#L482
https://hevc.hhi.fraunhofer.de/trac/...taCU.cpp#L1009

That should give you a starting point to dig in to.

Piter3d,

I am in a very strange condition.

if I replace the code at

https://hevc.hhi.fraunhofer.de/trac/...ataCU.cpp#L499

with

 if ((m_ctuRsAddr < 30) || (m_ctuRsAddr > 30)) {

	  if (m_ctuRsAddr / frameWidthInCtus)
	  {
		  m_pCtuAbove = pcPic->getCtu(m_ctuRsAddr - frameWidthInCtus);
	  }
}

restricting essentially ONLY the 1st CTU of the second row to use the CTU above I get distortion to all CTUs. How do you explain that ?