H.264 Intra Prediction

[trican]

Hi,

I'm sure this is a simple question with a simple answer, nevertheless....

Does the 9 intra prediction modes in H.264 give better prediction than using (for example) a DPCM approach on a pixel by pixel basis within a 4x4 block. Because if it does I can't really see why it would?

Obivouslsy there is increased computational expense doing the DPCM on a pixel basis compared to relicating pixel boundaries with H.264. But surely even this is comparable when the 9 SAD block matches have to be carried out to assert which intra prediction mode works best.

I'd appreicate any comments
thanks!

[Kopernikus]

no, the intra prediction does probably not give a better prediction than DPCM, but it requires less bits. The intra prediction mode is entropy coded with CABAC, so it takes less than the 4 worst case bits to encode 9 different modes.

Your DPCM approach would need much more than 4 bits. Plus the Intra Prediction is a predicition, its purpose is to predict a block, not to code it exactly, as the residuals are coded in a different way.

[trican]

Thanks for the response.

The intra prediction mode is entropy coded with CABAC, so it takes less than the 4 worst case bits to encode 9 different modes.


But if only DPCM is used and the decoder knows this, then no modes need to be sent - granted I'm now talking about a non-conforming codec

Plus the Intra Prediction is a predicition, its purpose is to predict a block, not to code it exactly, as the residuals are coded in a different way.

Absolutely, (unless I'm missing something here?) there's nothing to stop you from taking the PDCM residual and pushing it through the rest of the H.264 tools.

Perhaps the question I'm really asking is why was the H.264 intra prediction chosen the way it is? If anyone knows of any paper which discusses its selection I'd be delighted to hear.

thanks again.

[Kopernikus]

If I understand DPCM correctly, then there is no residual left. I don't know what you call "dpcm residual".

Problem with DPCM is, you have to code sign and magnitude for each of the 15 differences. You would end up with many bits eaten by the prediction that you could spend in a better way.

[trican]

If I understand DPCM correctly, then there is no residual left. I don't know what you call "dpcm residual".


ok, by dpcm (and apologies if I'm using the incorrect term) I mean if in the following situation:

P0 P1 P2
P3 Px

A prediction of Px can be made using a linear combination of P0-P3. Px minus the prediction = residual

I would assume this residual has lower energy than the h.264 intra prediction scheme?

[trican]

I agree with you that the residual needs a sign and magnitude which in effect would mean pixel growth to 9 bits. However considering you dont need to send mode bits I would have thought overall it would work out better after VLC.

[akupenguin]

Does the 9 intra prediction modes in H.264 give better prediction than using (for example) a DPCM approach on a pixel by pixel basis within a 4x4 block?
Ever notice how DPCM-based lossy audio codecs suck compared to MDCT-based codecs?

DPCM is not compatible with DCT. To compute the DPCM prediction of one pixel, you need the decoded values of the adjacent pixels. But DCT works on a block at once. When you quantize the DCT coefficients, that changes every pixel in the block by a little bit, which changes the prediction of the other pixels, which changes their desired residual, etc. So you end up with much more distortion than quantization would conventionally produce.
If you crack out the fancy math and lots of cpu-time, you could simultaneously optimize the prediction, residual, and quantization... Then you'd find that it's equivalent to a block-based prediction (though not exactly one of h264's intra modes) plus a transform that looks kinda like DCT but not quite. And the real DCT works better than that approximation.

Note that this is not a problem when doing lossless compression, and yes some lossless video codecs and most lossless audio codecs do use DPCM or similar. (Actually, Paeth is more common for video (huffyuv, ffv1, lagarith, png))


But surely even this is comparable when the 9 SAD block matches have to be carried out to assert which intra prediction mode works best.
Decoding speed is more important. And there the comparison is one block intra prediction (no SADs) vs DPCM prediction of all the pixels in the block.
H.264 lossless w/ CAVLC is faster to decode than huffyuv...

I agree with you that the residual needs a sign and magnitude which in effect would mean pixel growth to 9 bits. However considering you dont need to send mode bits I would have thought overall it would work out better after VLC. DPCM residual actually does not need a sign: just run the pixel math modulo 256 (huffyuv does this). But dynamic range isn't important: DCT coefficients have a dynamic range of 15 bits or so, and yet they let you compress down to 0.1 - 0.2 bits per sample and still look decent. Whereas DPCM would need some large fraction of its 8 bits to be present in the bitstream.

[trican]

Thanks akupenguin for the detailied response.

DPCM is not compatible with DCT. To compute the DPCM prediction of one pixel, you need the decoded values of the adjacent pixels. But DCT works on a block at once. When you quantize the DCT coefficients, that changes every pixel in the block by a little bit, which changes the prediction of the other pixels, which changes their desired residual, etc. So you end up with much more distortion than quantization would conventionally produce.

Aha, ok that makes sense

If you crack out the fancy math and lots of cpu-time, you could simultaneously optimize the prediction, residual, and quantization... Then you'd find that it's equivalent to a block-based prediction (though not exactly one of h264's intra modes) plus a transform that looks kinda like DCT but not quite. And the real DCT works better than that approximation.


Ok. Just so that I'm clear, if you ignore the issues caused by DCT quantisation, and compare just the residual information a DPCM will have lower energy right?

Just for my own benefit, by "fancy maths" you mean....?

Decoding speed is more important. And there the comparison is one block intra prediction (no SADs) vs DPCM prediction of all the pixels in the block.

Sorry, by SAD block match I meant the comparison required to decide which of the 9 modes resulted in lower energy. I suppose this would be termed the intra mode decision, granted I'm sure there are well established fast mode decision which reduces the burden or 9 4x4 "block match" like comparisons.

H.264 lossless w/ CAVLC is faster to decode than huffyuv...
wow thats interesting to know

Thanks again for your comments, they are much appreciated

[akupenguin]

Sorry, by SAD block match I meant the comparison required to decide which of the 9 modes resulted in lower energy. I suppose this would be termed the intra mode decision, granted I'm sure there are well established fast mode decision which reduces the burden or 9 4x4 "block match" like comparisons.

Actually, it's 22 block matches (9 modes of 4x4, 9 modes of 8x8, 4 modes of 16x16), but due to the nature of DCT not all of those need to be full block comparisons, some can be optimized to process only 4x1/8x1/16x1 pixels. And yes there are fast decision algorithms that eliminate some of the modes.
Still, doing all those comparisons is slower when encoding than doing only one DPCM mode. I was not referring to encoding speed, but to decoding.

Ok. Just so that I'm clear, if you ignore the issues caused by DCT quantisation, and compare just the residual information a DPCM will have lower energy right?
Yes, the DPCM residual is lower energy than any one block-intra mode. Although I'm not sure that's still true when you have a choice of 22 block modes.

Just for my own benefit, by "fancy maths" you mean....?
fancy = a solver for a linear system of equations with discrete variables.
conventional = a matrix multiply.

[alizadehms]

hello i have an other question :
in first macroblock (first row first column) I think standard should use IPCM but when i check coded video i see that it use Intra mode 0 how its can possible ? and when encoder use IPCM?
TNX

[Conquerist]

hello i have an other question :
in first macroblock (first row first column) I think standard should use IPCM but when i check coded video i see that it use Intra mode 0 how its can possible ? and when encoder use IPCM?
TNXThe IPCM mode forgoes all other coding tools in H.264/AVC, including prediction, transform, quantization and entropy coding. The input samples in an IPCM macroblock are directly written to the bitstream. This puts an upper limit on the number of bits needed to code a macroblock. It'll only be of practical use at very low QPs on macroblocks that don't benefit from prediction and/or transform coding. If the overhead of other coding tools would result in more coded bits than the input samples themselves, it makes sense to use the IPCM mode.

To your first question: in the very first coded block of an image (in the upper left corner), there is nothing to predict from, but we still want to use transform, quantization and entropy coding. So in most cases, the IPCM mode doesn't make sense here. The first block is usually coded as an I-Macroblock with DC prediction. Because no samples on the top and/or left are available for prediction, a 50% grey is used as a predictor. The difference between this predictor and the input samples is transformed, quantized and entropy coded, just as in any other macroblock.