Some technical questions about quantization

[mimungr]

Let me begin by briefly summarizing my understanding of quantization.

There are two 8x8 quantization matrices (QMs), one used for predicted blocks and the other for non-predicted. The DC element of the QM is fixed at 8 for non-predicted blocks and 16 for predicted blocks. All other elements are greater than or equal to the DC element. There is also a scalar quantization parameter (QP), which is either statically configured by the user or dynamically selected by the rate control mechanism of the encoder. Each DCT coefficient is quantized by dividing it by the product of the QP and the corresponding element from the QM and then rounding to the nearest integer. In other words, if Y is the matrix of luma DCT coefficients,

Yout[i][j] = floor((Yin[i][j] / (QM[i][j] * QP)) + 0.5)

For chroma, there is only a 4x4 matrix of DCT coefficients. The same QM is used, but only the even rows and columns:

Uout[i][j] = floor((Uin[i][j] / (QM[i*2][j*2] * QP)) + 0.5)

Is this correct? I’m trying to be precise, so if I’ve made even a minor error, please correct me.

I don’t understand how the minimum quantizer can be 8. It’s widely considered that a QP of 2 yields excellent quality. However, given that YV12 uses 8 bits for each YUV value, if we quantize with 8 * 2, we’re throwing away half the bits. I realize that it’s difficult to conceptualize how changes in the frequency domain translate to the spatial domain, but consider the degenerate case of a uniform block. In this case, the DC element is the same as all the spatial elements. If we throw away the low order 4 bits, we’re left with just 2^4 = 16 grey scales. This just can’t be. What am I missing?

Is QP constant for a frame? The concept of adaptive quantization implies it isn’t, yet the status window of Xvid shows a histogram plotting the number of frames at each QP, which implies it is a property of the frame.

Are there any inexpensive sources for technical information about MPEG-4? My initial inclination to answer the above was to read the spec rather than wasting your time, but ANSI charges $300 just for part 2, and at the current exchange rate, ISO wants even more. I’m not trying to write my own encoder or anything like that, but I do like to understand the technology I’m working with. Clearly, many of the posters to these fora have a detailed understanding of MPEG-4; where did you get that information?

Thank you so much for any help.

[Koepi]

Hm, I'll just throw in a link which might be an interesting read for you, maybe you get the idea where your idea is flawed ;)

http://tan.informatik.tu-chemnitz.de/~jan/MPEG/HTML/mpeg_tech.html
"Every macro block contains 4 luminance blocks and 2 chrominance blocks. Every block has a dimension of 8x8 values."

Cheers
Koepi

[sysKin]

Originally posted by mimungr
Let me begin by briefly summarizing my understanding of quantization.

There are two 8x8 quantization matrices (QMs), one used for predicted blocks and the other for non-predicted. The DC element of the QM is fixed at 8 for non-predicted blocks and 16 for predicted blocks. All other elements are greater than or equal to the DC element. There is also a scalar quantization parameter (QP), which is either statically configured by the user or dynamically selected by the rate control mechanism of the encoder.
Everything correct so far, except that you should probably call the quantization matrices "inter" and "intra", since "intra" are also, technically speaking, predicted (from intra blocks above and on the left). Also, I've never heard of the limitation of 8 - every coefficient is stored as 8-bit unsigned integer in the bitstream so I don't think there is such limit.
Each DCT coefficient is quantized by dividing it by the product of the QP and the corresponding element from the QM and then rounding to the nearest integer. In other words, if Y is the matrix of luma DCT coefficients,

Yout[i][j] = floor((Yin[i][j] / (QM[i][j] * QP)) + 0.5)

Not quite. This method would produce a very inefficient encoder. Quantization itself is not defined in the standard. Every coefficient takes space, so it's beneficial to drop some of them. "Simple" way of quantizing is similar to what you said, but with an extra bias towards zero and with even bigger bias to turn ones into zeroes ("dead zone"). More advanced method of finding R-D optimal set of coefficient is Trellis.
More info in this Skal's document: http://skal.planet-d.net/coding/quantize.html

For chroma, there is only a 4x4 matrix of DCT coefficients.
Completely incorrect this time ;) Chroma is also in 8x8 blocks, so one chroma block (two blocks if you count them as U and V) "covers the area" of four luma blocks, creating a macroblock (six blocks total).


I don’t understand how the minimum quantizer can be 8. It’s widely considered that a QP of 2 yields excellent quality. However, given that YV12 uses 8 bits for each YUV value, if we quantize with 8 * 2, we’re throwing away half the bits. I realize that it’s difficult to conceptualize how changes in the frequency domain translate to the spatial domain, but consider the degenerate case of a uniform block. In this case, the DC element is the same as all the spatial elements. If we throw away the low order 4 bits, we’re left with just 2^4 = 16 grey scales. This just can’t be. What am I missing?
You're missing the fact that coefficients (and even pixels) are 16-bit precision as far as DCT is concerned.
Is QP constant for a frame? The concept of adaptive quantization implies it isn’t, yet the status window of Xvid shows a histogram plotting the number of frames at each QP, which implies it is a property of the frame.It isn't, histogram is just for fun and shows decisions made by ratecontrol rather than anything else. To find out the real average quantizer, use ffdshow's OSD function.

Radek

[708145]

Originally posted by sysKin
Everything correct so far, except that you should probably call the quantization matrices "inter" and "intra", since "intra" are also, technically speaking, predicted (from intra blocks above and on the left). Also, I've never heard of the limitation of 8 - every coefficient is stored as 8-bit unsigned integer in the bitstream so I don't think there is such limit.

Radek

When I asked around about those quants I learned that in most implementations the DC quant which is actually used has to be at least 8!

That means if your value is i.e. 4 then quant1 is forbidden but quant2 and higher is allowed.

Please correct me if I'm wrong.

bis besser,
Tobias

[Shinigami-Sama]

wow I;ve kinda been looking for something like this
it seems less compilcated than I thought
I might actualy understand half of this:eek:
*reads the link*
-edit-
wow I got a bit of that ^_^
I never knew it as that simple
I though ther ewould a 5line formula for it
not a simple s&s equation like that
you made my day mate