Quantization Matrices and file size

[DDogg]

I recently stumbled on to something that has me bewildered and I am posting here as I think it effects all encoders that use matrices.

I was doing 1% samples of a test source in CCE 2.67.00.11 using one pass VBR (OPV)set at Q=38. All I did after the initial setup of the script and CCE was to rerun the test selecting different quantization matrices and a different output filename. The results on file size really surprised me.

Using the standard CCE matrix the file size was 7500kb. Matrices like Andreas SVCD, Fox and Kika Trick low returned much larger sizes in the 12-13000 kb range. This brings up a couple of different questions and observations to throw out for discussion:

1> Bach strongly argued that different matrices were a waste of time. If the file-sizes using different matrices are this different on a 1% sample this assertion could not be correct as quantization must have been effected.
2> Kika Trick low returned a much larger file-size than Kika trick High.
3> A matrix that returns a larger file-size must cause higher quantization vales and adversely effect the quality of the encode?
4> What the heck is the correct word usage for the singular? Matrix or matrice? :)

While these tests were done on CCE, I would think they would also effect other encoders that allow matrices. It may well be that I missed something obvious, but even so a discussion would clarify that.

[Kika]

Bach strongly argued that different matrices were a waste of time.

Maybe he never tried it by himself?

Kika Trick low returned a much larger file-size than Kika trick High.

Yes, this can happen. I don't know where you got the Low-Version from, i stopped using it a year ago.

A matrix that returns a larger file-size must cause higher quantization vales and adversely effect the quality of the encode?

Um, my Matrices where build for Anime only. They take care of the special Picture-Structure of drawings. The Goal ist to find a Matrix, which makes the RLE-Encoding working much better. And on Animes, the trick is to use a Matrix with Low Elements. It isn't easy to descripe (even in my native Language).
On Animes, even the Bitrate isn't too low, you can get Artifacts only from the DCT. My Matrices are build to prevent these Kind of Artifacts. On the other Hand, you need more Bitrate for Encoding. In the original Guide i wrote about using those Matrices, i suggested to use medium Min.-Bitrates and very high Max.-Bitrates. So, if you are not using 2pass-Encoding, you might get... um, strange Results. You can use the Matrices on single pass too, but then, you have to play around with the Q-Factor in CCE (or TMPGEnc).

[DDogg]

Sorry I spotlighted yours, Kika as it is for a specific and special use. Actually it would have been better to mention the Andreas SVCD which quite a few people use. It returned a much larger file-size than most others that are designed for standard encoding sources. If so, it would not seem to be a good choice for maximum compression? Would you agree?

[Acaila]

@DDogg

Although I'm not speaking from experience with (S)VCD but from XviD, things should pretty work in the same way. So here goes.

1) Filesizes are supposed to be different with different matrices. A matrix with larger values has a stronger quantization (throws more info away) than one with low values. But generally I agree that using different matrices if a waste of time. Only use one if the standard matrix either i) gives you a too low overal quality, in this case use a matrix with higher values for the high frequencies to 'spare' the low frequencies (but whether or not this is beneficial is still open to discussion), or if ii) the bitrate is so high that you could easily benefit from getting higher quality by lowering all values in the matrix thus getting more detail.

2) Can't comment on this.

3) In 2-pass mode this is correct. By using a matrix with lower values (= more detail = higher bitrate) you force the codec to compensate by using higher quantization to maintain the target bitrate you've set. This higher quantization affects all coefficients in the matrix, so it's generally a bad thing. That's why I personally never use a different matrix unless the bitrate I'm allocating is more than that which is needed by the standard matrix.

4) One matrix, two matrices :)

It returned a much larger file-size than most others that are designed for standard encoding sources. If so, it would not seem to be a good choice for maximum compression? Would you agree?Such matrices are only to be used if the source is very compressible, i.e. to turn a high quality encode into a higher quality encode. For anything else it's pretty much detrimental to quality.

For more info check this (http://forum.doom9.org/showthread.php?s=&threadid=54147&highlight=understanding+quantization) discussion we had over at the XviD forum.

[Kika]

You have to think about, what your Goal is.
Andreas SVCD is a Matrix with a high Quantisation on High Frequencies. This lowers the Sharpness of the Picture and prevents Blockyness at high Motion scenes while using low Bitrates. But, if you are using Single Pass Encoding, then there's a good chance to get... nothing!
The Trick with Matrices like this is only working at high Bitrate-Parts - to prevent blockyness, not to save Bitrate. So, you CAN encode on low Bitrates, using such a Matrix, but you have to use different Encoder-Settings.

Working with special Matrices isn't easy. As an Example: Using my Trick HQ on Real Pictures (non Anime) will result in horrible Blockyness isn't the Bitrate at a very High level. Using Andreas SVCD on Anime will result in heavy Artifacts in low Bitrate-Parts.
Um, and Andreas Matrix is a very old one... never tried it with CCE. But how a Matrix works, depants also on the Encoder-Settings, because the Q-Factor in CCE and TMPGEnc do influence the Quantisation.

[DDogg]

@Acaila, thanks, your reply at least made me feel even the experts around here still are not quite sure of the benefits. Also, thanks for the link for two reasons. First, it was incredibly educational and second, as I have been having trouble sleeping lately, I printed it out :-D

BTW, this seems one area where the mpeg4 and 2 crowd have a strong joint interest although the implementation is a little different.

@Kika, thanks also and maybe I can just ask you what your experience has shown as a good potential SVCD low bitrate matrix for progressive source. I have been spending a lot of time lately trying to get a common man's handle on RoBa so any experience you have had relating to low bitrate (equal or less than 1200) CCE OPV would be appreciated. Perhaps I'll just stay with CCE standard like always.