Any site/thread describing matrices in detail?

[bur]

I never really understood what matrices do, I know they somehow define how the quantizer id chosen, but I have no idea why it's in the form of a matrix. And how exactly it works.

So if there's any site or a thread here where this gets explained, please post a link. I couldn't find anything.

[Audionut]

You didn't try real hard.

http://forum.doom9.org/showthread.php?t=54147&highlight=matrix

[Dark Shikari]

(Simplified)

i = index of the DCT coefficient being handled

Quantized value[i] = (DCT coefficient[i] + Deadzone[i]) / (CQMValue[i] * QuantFactor)
Dequantized value[i] = Quantized value[i] * (CQMValue[i] * QuantFactor)

Where CQMValue is 16 by default and can be changed in the matrix.
Where QuantFactor is a number that depends on the quantizer (so higher quantizer -> lower precision).

[bur]

I couldn't find the thread you linked, audionut. I only browsed through the first few result pages though. Thanks. :)

Dark Shikari, thanks for the explanation but I don't understand that at all. I'll have a look at the "Understanding matrices" thread first.

[LoRd_MuldeR]

I couldn't find the thread you linked, audionut. I only browsed through the first few result pages though. Thanks. :)

Dark Shikari, thanks for the explanation but I don't understand that at all. I'll have a look at the "Understanding matrices" thread first.

Very briefly it works like this:

First the picture is divided into 8x8 blocks. Then each 8x8 block containing 64 pixels will be DCT transformed. Hence you'll get 64 DCT coefficients.

Each coefficient represents a certain "frequency" in the original block. These coefficients are then "quantized" by divining them trough the quantization matrix.

So for the 64 coefficients of each 8x8 block you will need a 8x8 quantization matrix with 64 entries.

The idea is that after quantization most of the coefficients in each block will be zero. This data can then be entropy coded very efficiently in the following step.

See this to get a first idea: http://en.wikipedia.org/wiki/JPEG#Block_splitting (in H.264 it's more complex, of course)

[bur]

Now I understand the whole encoding process much better, but there is one thing I didn't find anywhere explained in detail (or with too much detail ie mathematical):

How can you transform a DCTed block that has lost all spatial information to a normal pixel block where spatial information is crucial? Sounds to me like there still is spatial information in a DCTed block, otherwise how would you decode?

And is the DCT function on its own, without any quantization applied, lossless?

[LoRd_MuldeR]

Yes, IDCT (Inverse DCT) applied to your DCT transformed data will give you back the original input data. So it's lossless, unless quantization is used.

How do you get the pixel data back? Well, each DCT coefficient refers to one of the "reference" blocks as shown here (http://upload.wikimedia.org/wikipedia/commons/thumb/2/23/Dctjpeg.png/225px-Dctjpeg.png). The weighted sum of these blocks gives you back the original block.

For example for a completely "flat" 8x8 block you'd only use the block/coefficient in the upper left corner, all the other coefficients would be zero.

[bur]

Thanks for the explanation, I still don't really grasp how a picture can be recreated from those reference blocks, but it at least makes some sense to me now how it might work.

So along with each coefficient information to which reference it relates gets stored? And the transformed data without those reference blocks really has no spatial data left?

Those reference blocks, are they part of the mpeg standard and could theoretically be changed from that look or do they have to look like that?

[LoRd_MuldeR]

Thanks for the explanation, I still don't really grasp how a picture can be recreated from those reference blocks, but it at least makes some sense to me now how it might work.

For each block you store 64 coefficients. And each of these coefficient refers to one of the reference blocks (as displayed here (http://en.wikipedia.org/wiki/File:Dctjpeg.png)).

When decoding you simply sum up all 64 reference blocks, each one weighted with the corresponding coefficient. You repeat this for each block and you get your frame back.

Very basically. In reality it's more complex. And for P-Frames and B-Frames only the difference between the predicted frame and the original frame is stored.

So along with each coefficient information to which reference it relates gets stored?

Nope. The position of the coefficient in the matrix implicitly defines to which frequency (or "reference block") it refers.

There's a reason why we have exactly 64 coefficients for 64 "reference" blocks ;)

Those reference blocks, are they part of the mpeg standard and could theoretically be changed from that look or do they have to look like that?

These "reference" blocks come from the two-dimensional discrete cosine transform (DCT). So unless you use a different type of transform, you will get exactly these "reference" blocks.

And the transform method is of course defined in the video compression standard. I think all the MPEG formats use DCT so far. But Snow and JPEG2000 use Wavelet transform instead.

[bur]

I know that weighted sum explanation, it's just hard to imagine how you can get any 8x8 image, no matter what it shows simply by using those 64 reference blocks that just show a different black/white square pattern each. You'd probably have to watch it step-by-step with an easy example like a checkerboard pattern. And then see what happens to the coeffecients if you change some pixels. Do you know of any tool that performs a DCT on a 8x8 block?

[Dark Shikari]

I know that weighted sum explanation, it's just hard to imagine how you can get any 8x8 image, no matter what it shows simply by using those 64 reference blocks that just show a different black/white square pattern each.Those 64 images form a basis for the set of 8x8 images. Take some linear algebra ;)