Quantization Matrix

[Prim3]

Hi

I've read about QM a bit from Sharktooth's profile as well as a write-up by someone here. I was wondering if someone could elaborate more on this? Like where/when/how to use perhaps?

Thank-you!

[Sagekilla]

Matrices are usually used to provide a rounding bias. x264 usually uses a flat matrix, which divides all DCT coefficients by the same factor. Other matrices might bias against high frequencies, which means most of them end up being zeroed out. This usually can save a lot of bits, at the cost of fine detail. If you choose large coefficients for the quantization matrix, generally the components of your DCT will tend to be zero, and you'd lose those details.

I can elaborate more later but I have to go!

[Dark Shikari]

Matrices are usually used to provide a rounding bias.No, they're used to provide quantization scaling factors.

x264 has nothing that would prevent you from adding custom rounding biases either (that wouldn't even require a CQM in the bitstream itself, so you could do it for baseline/main profile).

[Prim3]

Oh I see. I read that they can be used to preserve fine details and to squeeze more quality from your encodes. Correct?

Now, how do I go on about using them? I'm quite confused on where to start experimenting.

Thank-you!

[manono]

I read that they can be used to preserve fine details and to squeeze more quality from your encodes. Correct?

Yes, a good matrix can do that, but at the cost of compressibility. That is, if you use a high quality matrix but don't allow a large enough size for it, you'll lose the benefits of using it in the first place as the average quant zooms upward and artifacts appear. Ideally you fit the matrix to the material and the file size you're aiming for. If you don't care about the final size or are allowing plenty of room, then a good matrix can do wonders for your encode.

[Dark Shikari]

Yes, a good matrix can do that, but at the cost of compressibility. That is, if you use a high quality matrix but don't allow a large enough size for it, you'll lose the benefits of using it in the first place as the average quant zooms upward and artifacts appear. Ideally you fit the matrix to the material and the file size you're aiming for. If you don't care about the final size or are allowing plenty of room, then a good matrix can do wonders for your encode.Note that this advice really only applies to formats with a linear quantizer scale (MPEG-2, MPEG-4 (A)SP) where there's basically no room to adjust quality between "QP2" and "QP1", while with H.264 there's no point in using CQMs for the purpose of "increasing quality while decreasing compressibility" (they might be useful for other purposes though).

[manono]

Yes, I was thinking specifically of MPEG-2 and to a lesser extent of XviD. I've had much less experience with H.264, but see nothing to indicate that Prim3's original post referred only to H/X.264. As we all know, sharktooth has developed CQMs for XviD as well as for H/X.264.

[Prim3]

Oh I see. So what would be a good type of matrix and what is considered a large file size? I'm trying to stay below 233MB if that helps for an anime episode.

I calculated the bitrate in MeGUI and it was around 1350kb/s.

Would it be possible for me to use a matrix that can satisfy my file size? Thank-you!

[victor1986]

http://forum.doom9.org/showthread.php?t=54147
checkout this..

[Prim3]

Yes I've already read that, twice. Doesn't tell you how to use it though. My question about 233MB episodes still stands.

Thank-you!

[manono]

Since there's next to no correlation between filesize/bitrate and quality, it's a misguided question. It's all about the "Q" - Q as in "quality" and Q as in "quant". Me, I encode mostly for DVD MPEG-2 (have you even given us your output format yet?) where the resolution is fixed, and choose my matrix based on the final average quant I hope to achieve, given the bitrate needed to fill the DVDR.

Any matrix will "satisfy your file size". Satisfying it with decent quality is another matter entirely. I get my matrices mostly from commercial/retail DVDs. For the most part I don't use "home-made" ones. That's just me and that's just for MPEG-2 encoding. Encoding for other formats will require other guidelines.

[Prim3]

Well I'm encoding in x264 and my final output will be in the MKV container if that helps.

I know what you are trying to say but I just need a jump start.

What I'm trying to ask is, is there a matrix, for anime episodes that will satisfy my file size of 233MB with decent quality?

Also, how do I go on about using them? The pre-made ones I mean.

Thanks!

[Dark Shikari]

You generally shouldn't use CQMs with x264.

You really shouldn't use them with anime.

[Prim3]

Oh really? I didn't know that.

I guess if I shouldn't use them with anime then.

One more question. Besdies filtering, is there any other way, perhaps encoding settings, to make an encode of decent quality? I already have a high bitrate if that helps.

Thank-you!