Rate control for I-frames in HM 16.7

[damjeux]

Hi,

I'm trying to understand the rate control algorithm in the HM 16.7 reference software. I have the paper "lambda domain rate control algorithm for High Efficiency Video Coding" by B. Li et al which explains how the Lagrange multiplier lambda for the RDO is found based on a model with 2 parameters (alpha and beta).
I can see how this is applied to P and B frames in HM:

estLambda = alpha * pow( bpp, beta );

However, the lambda calculation for I-frames is different. It is calculated in TEncRCPic::calculateLambdaIntra as:

(alpha/256.0) * pow( MADPerPixel/bitsPerPixel, beta )

First question: Where does this formula come from? Can anyone explain or give a reference where I could look it up?

Then, as you can see from the code above, the calculation is based on the "MADPerPixel", where I assume MAD stands for Mean Absolute Difference. However, when calculateLambdaIntra is called in TEncRCPic::estimatePicLambda, MADPerPixel is given as:

pow(m_totalCostIntra/(Double)m_numberOfPixel, 1.2517)

If I look where m_totalCostIntra comes from, I see it is calculated in TEncSlice::calCostSliceI as a sum of Hadamard transforms of 8x8 blocks of the frame.

Second question: Why is a sum of Hadamard calculated here? What is the link to the calculation of lambda?

Any explanation or link to some explaining references would be greatly appreciated!

[foxyshadis]

You should direct this query to the JCT-VC mailing list. I watched the development of what became HM (TMuC) but I don't recall ever seeing that or the rationale; it's a pretty obscure thing.

Note that this isn't actually part of HEVC at all, that's just an encoder-side formula that any encoder can modify, and most do.