Seeking information about chroma quantisation and test programms.

[Ixal]

Hello.
I am a student currently working on my diploma thesis about H.264 and I seek information about the 2x2 chroma DC transformation for encoding. As resource I use "H.264 and MPEG-4 Video Compression" from Ian E. G. Richardson and the JM (http://iphome.hhi.de/suehring/tml/) reference encoder from the Fraunhofer Institute. (I am not allowed to use x264 as reference because the project might become commercial).
Sadly I have trouble to understand the 2x2DC chroma transformation. You can see the relevant part on vcodex. (http://www.vcodex.com/h264.html), the PDF about Transformation.

What puzzles me is, among other things, at which stage the 2x2 DC transformation are done and if those transformed DCs are put back into the 8x8 block for quantization.

According to the diagram on page 8 of the PDF the DC coefficients of the 4x4 sub blocks of the chroma block are transformed before quantization, but inverse-transformed before rescaling which doesn't make much sense to me.

My questions in short:
1) How are chroma blocks transformed? Are they split into 4x4 blocks and then transformed like luma 4x4 blocks or is there a special chroma transformation?
2) When are the DC coefficient read from the 8x8 block and when are they reinserted into the 8x8 block (reverse path)
3) When the DC coefficients are read and stored in the 2x2 array, are they set to 0 in the 8x8 block?

And a different question, is there any program, for example a partial decoder, which I can use to check if my (intra) encoding works and is compliant without having to program the entopy encoding and transport structure first?

I'm thankful for any any answer and hint and hope that this sort of questions are appropriate for this board.
Christian Maier

[Dark Shikari]

I am not 100% sure (Pengvado can give you a completely sure answer) but the x264 code should help [stripped down to the basics]:

void x264_mb_encode_8x8_chroma( x264_t *h, int b_inter, int i_qscale )
{
int i, ch;

for( ch = 0; ch < 2; ch++ )
{
uint8_t *p_src = h->mb.pic.p_fenc[1+ch];
uint8_t *p_dst = h->mb.pic.p_fdec[1+ch];
DECLARE_ALIGNED( int16_t, dct2x2[2][2] , 16 );
DECLARE_ALIGNED( int16_t, dct4x4[4][4][4], 16 );
h->dctf.sub8x8_dct( dct4x4, p_src, p_dst );
/* calculate dct coeffs */
for( i = 0; i < 4; i++ )
{
/* copy dc coeff */
dct2x2[block_idx_y[i]][block_idx_x[i]] = dct4x4[i][0][0];
h->quantf.quant_4x4( dct4x4[i], h->quant4_mf[CQM_4IC+b_inter][i_qscale], h->quant4_bias[CQM_4IC+b_inter][i_qscale] );
h->zigzagf.scan_4x4ac( h->dct.block[16+i+ch*4].residual_ac, dct4x4[i] );
}
h->dctf.dct2x2dc( dct2x2 );
h->quantf.quant_2x2_dc( dct2x2, h->quant4_mf[CQM_4IC+b_inter][i_qscale][0]>>1, h->quant4_bias[CQM_4IC+b_inter][i_qscale][0]<<1 );
zigzag_scan_2x2_dc( h->dct.chroma_dc[ch], dct2x2 );
}
}

[Ixal]

Thank you.
According to this code Richardson is wrong and the 2x2 DC transformations are applied before quantization and after de-quantization of the 8x8 block (and not before de-quantization).

That only leaves the question if the DC in the 8x8 (or rather the sub 4x4 block) are set to 0 or not.

Thanks for the help. Can you also help me with my second question (testing software)?

[Dark Shikari]

Thank you.
According to this code Richardson is wrong and the 2x2 DC transformations are applied before quantization and after de-quantization of the 8x8 block (and not before de-quantization).

That only leaves the question if the DC in the 8x8 (or rather the sub 4x4 block) are set to 0 or not.

Thanks for the help. Can you also help me with my second question (testing software)?You may be able to hackney the JM reference decoder to do such a thing--or you could "borrow" the x264 entropy encoding code, since you know that already works.

The JM ldecod.trace (http://akuvian.org/src/x264/jm_trace.diff) patch by Akupenguin/pengvado may help your debugging.

[Ixal]

You may be able to hackney the JM reference decoder to do such a thing--or you could "borrow" the x264 entropy encoding code, since you know that already works.

The JM ldecod.trace (http://akuvian.org/src/x264/jm_trace.diff) patch by Akupenguin/pengvado may help your debugging.

The JM decoder is not what I call easily readable, but thank you. Also I need a decoder for this and x264 is just an encoder I thought? I could use the reverse path to test it thought.
I also have other H264 decoder here were I work where I can test this, I just wanted to know if there happens to be an already made programm for this kind of testing.

That only leaves the question if the DC components in the 8x8 block are set to 0 or not.

[Dark Shikari]

The JM decoder is not what I call easily readable, but thank you. Also I need a decoder for this and x264 is just an encoder I thought? I could use the reverse path to test it thought.

I also have other H264 decoder here were I work where I can test this, I just wanted to know if there happens to be an already made programm for this kind of testing.No, JM isn't easily readable, to say the least :p

x264 is an encoder; my point was you could use it to encode your partition decisions/similar to CABAC or CAVLC, and then decode them with the decoder of your choice.

I don't know of any program that lets you check encoding compliance for a stream that isn't entropy encoded yet.

[Ixal]

x264 is an encoder; my point was you could use it to encode your partition decisions/similar to CABAC or CAVLC, and then decode them with the decoder of your choice.


Thats a good idea. I haven't thought about that.
And another question. According to the x264 code the 2x2 DC go first through the inverse DC and then through de-quantification. Is that correct? It looks strange to me.

[akupenguin]

What puzzles me is, among other things, at which stage the 2x2 DC transformation are done and if those transformed DCs are put back into the 8x8 block for quantization.
It works because hadamard is normalized, involves no right-shifts, and all the DC coefficients have the same quantizer. So multiplying by the quantizer before or after the hadamard gives the same result.

[Ixal]

Another question, this time about Chroma Prediction.

I am currently trying to insert my transformed data into a decoder (confidential, so I can't post the code). In this code I only found one variable for the chroma prediction mode. Does this mean that both chroma blocks (U and V) are always predicted using the same prediction mode?

[Dark Shikari]

Another question, this time about Chroma Prediction.

I am currently trying to insert my transformed data into a decoder (confidential, so I can't post the code). In this code I only found one variable for the chroma prediction mode. Does this mean that both chroma blocks (U and V) are always predicted using the same prediction mode?Yes.

for( i=0; i<i_max; i++ )
{
int i_satd;
int i_mode = predict_mode[i];

/* we do the prediction */
h->predict_8x8c[i_mode]( p_dstc[0] );
h->predict_8x8c[i_mode]( p_dstc[1] );

/* we calculate the cost */
i_satd = h->pixf.mbcmp[PIXEL_8x8]( p_dstc[0], FDEC_STRIDE,
p_srcc[0], FENC_STRIDE ) +
h->pixf.mbcmp[PIXEL_8x8]( p_dstc[1], FDEC_STRIDE,
p_srcc[1], FENC_STRIDE ) +
a->i_lambda * bs_size_ue( x264_mb_pred_mode8x8c_fix[i_mode] );

a->i_satd_i8x8chroma_dir[i] = i_satd;
COPY2_IF_LT( a->i_satd_i8x8chroma, i_satd, a->i_predict8x8chroma, i_mode );
}

[Ixal]

Yes.


Oh great :(

How it it determined which pred mode is used for both chroma blocks?

[akupenguin]

How it it determined which pred mode is used for both chroma blocks?
That's the code he just pasted.

Why the :(? I don't see a problem. I can't even think of a decision method for a single block that wouldn't work for 2 blocks.

[Sergey A. Sablin]

So multiplying by the quantizer before or after the hadamard gives the same result.

this is only true for qp < 30 (iirc), cause dequant for qp >= 30 involves right shifts, so the only way for bit exact result is: HAD, Quant, iHAD, iQuant, as any other order will definitely lead to other output.

[akupenguin]

Right, except the other way around. Dequant of qp<30 involves rightshift, >=30 doesn't. And the threshold is qp 6 if you use the default (flat16) cqm, because 4 bits of the shift just undo the *16 of the cqm.

[Ixal]

I need you help again.

I used the example transformation on vcodex.com (H.264 Transformation and Quantization tutorial) to write and test my functions and when I use the example on the last page of the book it works (Although the book is not completely clear if he uses QP 10 because the Multiplication Factors fit for QP4)

But when I insert the quantized block into an decoder I don't get

544 0 -32 0
-40 -100 0 -250
96 40 32 80
-80 -50 -200 -50

but

187 0 -11 0
-11 -28 0 -70
33 14 11 28
-22 -14 -55 -14

Has anyone an idea what went wrong?
A colleague mentioned that he has often seen that the MF Matrix gets transposed and when I look at the JM reference software he could be right (although the whole matrix looks rather strange to me).
Sadly I can't find the matrix in x264 to see how it is structured there.

Thanks again for your help.

[akupenguin]

MF is symmetric (unless you use an asymmetric CQM), so transposing doesn't matter.

[Ixal]

Sorry to bother you again but I have another question.

After speaking with the developer of the decoder I use for testing it seems that using it for tests will not be as easy as I thought as for example the dequantization and inverse transformation are partially mixed together for performance reasons, making it hard to check the outputs against my or the vcodex.com results.

I do know that my program can calculate the example transformation/detramsformation from Richardsons book correctly (see vcodex.com Transformation tutorial).
Can someone confirm that this is the correct, H.264 compliant way to dequantize and inverse transform data?

Thanks.