Transform skip flag, and 4x4 TU transforms (in HM)

[rudyb]

I everyone,
I have been reading the intra-prediction portion of the HM code (HM-14), and for most part it started making lots of sense, until toward the end that I would need little help from you guys.

For a 32x32 CU level all the way down to 8x8 it all made sense. So, when I was following the code, I realized that the partition size is 2Nx2N, and I can see that HM (HM-14) forms the mode list of size 3 (3 most probable modes), and for each angle in the mode list, HM calls xIntraCodingLumaBlk() function to compute the predicted/residual/reconstructed data, and then it selects the one with the best RD cost.

But, when it gets to NxN partition size (4x4 TU size), I see that the mode list will be of size 8 (eight best angles), and it repeats the same process. However something doesn't make sense to me, and that is for NxN partition size, it calls xIntraCodingLumaBlk() function twice !!
In other words, it computes predicted/residual/reconstructed data twice for each one of those 8 angles.
And, I noticed this happens since checkTransformSkip flag is set to TRUE for NxN partition size!

Could someone please explain what is happening?

Why the HM model goes through calling xIntraCodingLumaBlk() twice for NxN partition size, which in essence, it will repeat the calculation twice for the 8 angles in the mode list? And why setting checkTransformSkip is set to TRUE for NxN partition.

Thanks a lot in advance,
--Rudy

[rudyb]

I actually found out what is happening.
When it gets down to a 4x4 block (NxN partition size), the HM model always goes through function "xIntraCodingLumaBlk()" twice.
The first time "useTransformSkip" is set to false, and the second time "useTransformSkip" is set to true.

Essentially, for a 4x4 TU block of NxN partition size, HM tries to compute both costs for skipping and NOT skipping the transformation process, and picks the smaller one.

--Rudy