Some New Custom Matrices

[kilg0r3]

Quite a lot of discussin in this forum, has been about custom matrices, during the last weeks. Not that I had the time read all of it, but, especially, nic's comments on the quantizer-range problem of mpeg for here (http://forum.doom9.org/showthread.php?s=&threadid=55024), have lead me to create some test matrices.

The problem mentioned by Nic was that, the low quants especially 2 and 3 differ very strongly in terms of compression and, consequentially in terms of fidelity. So that, when the codec has relatively many bits available, it has the choice between only two or three strongly divergent options. That is, when there is just not enough bitrate to use a mean quant of two it must do the large jump up to 3 and hence kill a lot of detail.

My 'approach' (this sounds odd) is to create a matrices that contain lower values than the standard matrices. As a result a file that is encoded at constant quant 2 using one of these matrices will have a much larger filesize. As a result, when I do a full two-pass encode, heading for one cd, for instance, the codec is forced to use a much higher mean quantizer. This in return minimizes the difference between the quantizer steps. For instance, a value of 8 is only 12.5% larger than 7 while 3 differs from 2 by 33.3.%.

There are, at least, two issues one has to be aware of when creating/sing such matrices:
1. when the values in the cells are too small the codec might hit the roof in a two pass cenario. That is it would need to use a mean quant higher than 31 which is not possible. This becomes even more probable when using b-frames, because they use a higher quant.

2. you must be very carefull with the b-frames factor because of the reasons mentioned in 1.

3. you can be more light hearted in the use of b-frame offset because the difference in fidelity between the mean quantizers is not so great anymore

4. Streams encoded with matrices containing values lower than 8 are incompatible with libavcodec used by ffdshow. As a workaround, you can just check 'use xvid' in ffdshow. This even worked for my 1er matrix.

The matrix I post below is a combination of the standard Mpeg matrix simply divided by two and the Hvs-best-picture-matrix divided by three, which i used for lower-right corner inorder to filter the frequencies a bit stronger than the mpeg matrix. Since I divided the values of the best-picture matrix by three it does not filter the high frequencies as strongly as the source matrix but still a little bit stronger than the Mpeg matrix.
With this one I already did a full two pass encode, which came out fine. It has about two times the first-pass size of the standard Mpeg matrix.
The rather slow movie was about 97min long. Using a res of 640*336 i put it onto one 99min XCD (=970MB). Consequentially, the settings were relatively conservative, vhq1, max If-int 600, chroma motion, bframes 2,124,100. The final stram used quants manly between 5 and 7.

Looking at the result, it seems, I could have used a normal 700Mb cd and the result would still have been o.k.

O (http://www.mynetcologne.de/~nc-allgeife8/hvs-best-picture-by3.matrix)

This matrix is just the hvs-best-picture matrix divided by three containing values below 8. This one is untested.

O (http://www.mynetcologne.de/~nc-allgeife8/MpegBy2+HfHvsBestBy3.matrix)

Just cut and paste the hieroglyphs intor a a text editor of your choice, save and load in them into xvid vfw interface using the load button on the quantization tab.

Any comments appreciated

[Acaila]

Although the idea of using lower values is nice in theory, it doesn't always work out to your advantage in practice. In 2-pass mode the codec will use higher quants to reach the target bitrate like you said, however because of this the lower frequencies will get quantized relatively heavier with these sort of "high-quality" matrices than with the standard matrices (because those have higher values for higher frequencies, thereby saving the lower frequencies longer).
As can be seen in this (http://forum.doom9.org/showthread.php?s=&threadid=54147&highlight=custom+matrix) thread not every matrix with lower values is automatically usable, because in 2-pass mode they can actually lower the PSNR.

Have you tested the PSNR in 2-pass mode with your matrices, and if so how does it compare with the standard matrices?

[Lobuz]

Maybe some function to modify used matrice based at the results from the first pass would help. It could scale matrice a little or more. Or additional fast pass every n-th frame with that matrice with psnr check?
It's just some silly thoughts. :D

Regards
Lobuz

[OUTPinged_]

Actually there are only 2 cases where using a custom matrix over h263 and mpeg is usefull:

1. Undersizing.
2. Oversizing.

Otherwise, any matrix is just a matter of how high are low frequencies scaled relatively to "high".

If you are averaging around quant3-5, you will gain absolutely nothing. "More precise quantization" is a bullshit. Use H263 if you encode low noise content and MPEG otherwise.

2. If you are averaging around quant7-10, then a "low-bitrate" matrix which scales coefficients _very_ quick will be of some use. But it has to scale them faster than MPEG matrix does, otherwise what's the point? But that isnt because you would run out of quantizers - quant31 is already crappy as hell and if your encode has alot of quant31 frames, then there is something seriously wrong in the way you encode.

1. If you are averaging around 2.5, then lower koefficient matrix is just for you. If mosquito noise will kill all the fun for you (clean source), matrixes which scale strong (MPEG) are a no-no there. Best one (very compatible and low koeff's) to use would be a "Animation-CG" matrix. For cases where you really need to pump some bits into your clip, there is "Fox home entertainment" matrix - nicely scaling koeff's for delta frames from 8 to 11. Use that one for cases where you want to have a near zero differences at quant2.

[mf]

Originally posted by OUTPinged_
But that isnt because you would run out of quantizers - quant31 is already crappy as hell and if your encode has alot of quant31 frames, then there is something seriously wrong in the way you encode.
That's the beauty of custom matrices. You can't say "quant31 is crappy as hell" because there might be a matrix where quant 31 actually looks pretty acceptible. Ok, so 31 will probably always be crappy, but afaik you SHOULD be able to make quant 5 look like quant 2 using a custom matrix. And his point is valid, most of the time quants above 10 aren't used. If you could scale the matrix so that quant 31 will look like 10, you have a lot more precision. And sometimes that does matter.

[kilg0r3]

psnr4avi going to be used within an hour or so ... <^°-°^>

[Koepi]

If i'm not mistaken, with higher quantizers the motion vector length gets capped as well (as higher quants are there for saving bits this is just a logical combination).

So a good motion match will be harder in such a scenario.

It doesn't have only simple "advantages". You loose something for it.

For low motion movies this would be acceptable i think, but action movies will most likely not gain much from this method.

Something to think about.

regards
Koepi

[kilg0r3]

1.

hi koepi,

could you please do a little research regarding the exact nature of this relationship? At which quant does the capping start? Does it gradually increase or is there just one big jump?
Sorry to bother you with this, but I really do not have any idea where to look for such info (yeah I know, the sources haha ... :))


2.

Anything I have to be aware of when using psnr4avi?
may I use bframes? may i input an avs script? sorry this is my first time.

hm, i got
-------------------------------
Total frames: 14382
Average PSNR (Y U V): 1.#J, 1
Total Average PSNR: 1.#J
------------------------------
somethings wrong here. right?

Inputfiles were:

inputfile1: avsscript/fake avi(ffvfw)/huffyuv
inputfile2: xvid.avi/xvid->huffyuv.avi

[Lefungus]

if psnr4avi doesn't work, you can also use compare from avisynth

[Nic]

kilg0r3 PM'd me to comment here, but there isn't much I can write that hasn't already been stated. With quant matrices there is not a straightforward answer, as Koepi mentioned, many factors come into play.

Ill have a think and post something to add that's more useful. Just reading through the quant matrix part of http://bmrc.berkeley.edu/research/publications/1997/144/dbrown.html as I think...

-Nic

[sysKin]

I was made to comment, too ;)

I don't have much to say, except that I can explain this motion estimation issue Koepi mentioned.

I wouldn't call it a big deal. Yes, motion estimation tries to limit the bits needed to code vectors, and the "how much does it count" value depends on quantizer. But unless you do something to turn quant 10 to quant 2, it's not a big problem... And even then, the codec would be just sub-optimal, you can live with it.

VHQ, especially VHQ4 still works perfectly, so it can correct small 'mistakes' ME makes.

Radek

[kilg0r3]

sorry guys to force you to write here :o ;)

here are my discouraging psnr results
of course the clips result from two pass encodes

My Custom Matrix
----------------

Minimum Average Maximum
Mean Absolute Deviation: 0.8599 1.3679 2.1043
Mean Deviation: -0.0751 -0.0098 +0.0654
PSNR: 38.5236 42.3014 46.1368

Standard Mpeg Matrix
--------------------
Minimum Average Maximum
Mean Absolute Deviation: 0.7561 1.3252 1.9083
Mean Deviation: -0.0859 -0.0101 +0.0724
PSNR: 39.4337 42.4924 47.1595

However, I am not sure, if everything went the way it should during the test. So I'd appreciate some 'third-party' testing.

[Acaila]

I can confirm your results, on my short test H.263 matrix gave me 42,26 while yours gave 41,66, a drop of 0.6 dB. This is exactly why I wrote my initial response. I haven't tested every single matrix out there, but those that I have tested performed just like you just experienced, which suggests that writing a good matrix is actually quite difficult. Only Didée’s matrix on the thread that I linked to seems to pass all my tests so far.

[Sigmatador]

Be careful with PSNR. i notice that HVS-Good matrice gave me 0.5-0.8db less than H.263, but for my eyes, HVS-Good is definitively nicer.

[JasonFly]

Yes, psnr results can be strange somtimes.
I think it depend on the noise of the source.

h263 sometimes give me higher psnr than hvsbest.
And sometimes, hvsbest psnr is higher than andreas78's.

[kilg0r3]

The problem is that AFAIU it, psnr is an objective or technical way to measure things. It is relatively unrelated to how humans perceive images. This is why some of these matrices are called hvs for (human visual system). They compress highly detailed struktures more because the human image cognition (not talking about the eye here) is supposed to ignore these.

If this is true, I don't know.

[OUTPinged_]

@killgore:

Unless you actually know what you are doing (by changing coefficients) you will not make any usefull matrixes.

Simply understanding "more to the right bottom = lower frequencies" is not enough. I spent a day tweaking that stuff and all i can say is that fine tweaking couldn't be done without scentific approach. You just cant see any difference between, say, koeff. 20 and 25 for some value in the middle of the matrix, if you only judge it by looks.

@mf:

You can't say "quant31 is crappy as hell" because there might be a matrix where quant 31 actually looks pretty acceptible.
Yes, but would there be a point in this matrix? Look at that: you have MPEG and H263 matrixes as default ones. There is no point in ajusting matrix only because you dont get bad enough quality with quant31 on default matrixes. The only thing you may want to do is change the bias towards high freq. with your custom matrix. IE simply doubling coefficients in MPEG matrix is pointless.

Also, since quant1 is broken in xvid we could use some lower koeff. matrix - but then you are somewhat restricted: first, you wouldn't want to make too much low.freq. coefficients higher than 24 - in that case matrix would produce lower quality than H263 = pointless. Second, scaling coeff. up to 24 too fast (ie, upper left=8, all others=24) will produce mosquito noise, which kinda kills the purpose of "hq" matrix.

There is other tricky thing in using "hq" matrix. It is the curve scaling and rate control algorithms. You got to be very carefull around it since it is very common to make codec use too high range of quantizers (which is imo a bad thing).

[MrBunny]

@OUTPinged_

As far as I recall, coefficients moving towards the lower right are higher frequency, not lower. Am I right or just losing it?

As well, I agree that careful consideration needs to be done before creating a custom matrix. It seems to me that (in general) the main differences in current matrices are the speed in which the matrix ramps to the maximum value and the maximum value itself. The matrix used should definately depend on the bitrate and compressibility of the material being encoded.

In some situations (very few situations), the matrix you described could be feasible. It preserves a great deal of detail and requires a very high bitrate. Of course due to the layout of the matrix, higher quants will cause major mosquito or blocking issues as you mentioned, but if it can be kept at lower quants, the matrix could perform acceptably.

Personally I don't have a real problem with having a HQ matrix with an average quant of 5 or more, though I do agree that designing one to hit really high average quants (to the point of worrying about q31) is a bit excessive.

One quick question too if I may. You mentioned that doubling the default MPEG matrix is pointless, which I agree with. What's your opinion on halving the default MPEG matrix, thus having half the difference between quantization levels. It wouldn't really change the output too much since it is the same scaling, but would allow for better curve control and as a result, better quality.

[OUTPinged_]

@mrBunny
As far as I recall, coefficients moving towards the lower right are higher frequency, not lower. Am I right or just losing it?

Nope, you lost it completely. Sharp details = high frequencies = upper left corner.

What's your opinion on halving the default MPEG matrix, thus having half the difference between quantization levels

Let's see... You get Acaila's matrix then! Not exactly, but pretty close. :-)

(I assume you are talking about matrix that is being used as "MPEG" in xvid(cames as "standard" in CCE), not the one that comes as "MPEG standard" in CCE and which is identical for inter-frames to "CG" matrix")

It wouldn't really change the output too much since it is the same scaling
You are taking this wrong again. There wouldnt be any "better scaling" if your average quantizer is higher than, say, 4. Even with av.=4 you will have very little quant2 frames if you are doing things smart way (more or less constant quality for low motion, a tad less quality for high motion). Saying "omg i want quant2.5 frame because it will scale nicer than quant2 and quant3 frame" is stupid. (low coeff. matrixes were intended to be used for av.=2.5 encodes, and not for "nice scaling") No one would notice a difference between quant2-3 frame sequence and quant2.5 frame sequence. And if your codec isnt a "koepi's latest and greatest", you dont have to worry about some overflow fluctuations too.

[edit: Akaila's nickname was misspelled again :/ ]

[Acaila]

@OUTPinged_ originally wrote
Nope, you lost it completely. Sharp details = high frequencies = upper left corner.
No, MrBunny was correct and you are the one who is mistaken. Upper left corner is lowest frequency (i.e. the average brightness of the entire block), bottom right corner is highest frequency (which is usually noise).

Here's some proof:
http://www.ifi.uio.no/~ftp/publications/cand-scient-theses/SHuseby/html/node7.html
http://www.ece.purdue.edu/~ace/jpeg-tut/jpgquan1.html

[sysKin]

Originally posted by OUTPinged_
You are taking this wrong again. There wouldnt be any "better scaling" if your average quantizer is higher than, say, 4. Even with av.=4 you will have very little quant2 frames if you are doing things smart way (more or less constant quality for low motion, a tad less quality for high motion). Saying "omg i want quant2.5 frame because it will scale nicer than quant2 and quant3 frame" is stupid. (low coeff. matrixes were intended to be used for av.=2.5 encodes, and not for "nice scaling") No one would notice a difference between quant2-3 frame sequence and quant2.5 frame sequence. And if your codec isnt a "koepi's latest and greatest", you dont have to worry about some overflow fluctuations too.
Actually I disagree. When Q2 frame follows Q3 frame, the resulting filesize is very big (bigger than in 1st pass) because codec wastes many bits to re-create all details lost in Q3 frame. If Q3 frame follows that, it usually just wastes the details again.
This is bad, and this is because there is a huge difference between quantization 2 and 3.
So, changing the quantizer matrix to actually use quants, say, 4,5,6 instead of 2,3 should have a positive effect, and this is what high-bitrate matices do.

Regards,
Radek

[OUTPinged_]

@Akaila
Upper left corner is lowest frequency (i.e. the average brightness of the entire block), bottom right corner is highest frequency (which is usually noise).
Heh. That's why it all was getting worse when i begun tweaking it by myself. Anyway, all of my "research" was done with matrixes from CCE, so it stays legit.

Some dude that wrote "explaining how the quantization matrix works" messed me up. I got to blame him. :-)

@sysKin

So, changing the quantizer matrix to actually use quants, say, 4,5,6 instead of 2,3 should have a positive effect, and this is what high-bitrate matices do.

I never denied the usefullness of those "hq" matrixes when it came to encoding stuff at average quantizers lower than 3. But "it better scales any encode" is bullshit. That was my statement.

[kilg0r3]

Originally posted by sysKin
So, changing the quantizer matrix to actually use quants, say, 4,5,6 instead of 2,3 should have a positive effect, and this is what high-bitrate matices do.

That is exactly what I did in my tests. I compared the Mpeg standard matrix against one that mainly contained values half the size of that of the mpeg matrix. only the lower right corner contained some higher values about a third of the hvs-best-picture matrix. Additionally, I used a matrix that contained only 'halfed' Mpeg-standard-matrix values. I don't know why, but, the psnr value of the Mpeg-standard matrix always was the highest.

Any idea why it doe not work?

[Lobuz]

@kilg0r3
So how is investigation going? Any hints where is the problem. Cause it would be really great to use whole quantizers graduality to better achieve scene bitrate needs.
Used standard matrix could be for example scaled based on the first pass or | and small additional pass ( for example every 50 frame ).

Regards
Lobuz

[UGAthecat]

I did some testing similar to Kilg0r3's about 10 months ago, and came to the same results (it's a great theory, but for some reason doesn't work in practice). I actually got worse block noise, I think because of the one DC value that is fixed at 8, or was at the time.
The only explaination I've seen for the psnr problems so far that seems valid is what Koepi mentioned about the ME being less accurate at higher quants. That seems to be the best explaination because more noise would get through to the dct & quant operations, which even at low quants will lead to more noise getting removed, making the psnr worse.
I'm at work right now but I would be interested if anyone would try the following:
take a short clip, run it through the DCTFilter in avisynth, completely removing as much of the higher frequencies as possible without destroying the quality of the pic, or introducing artifacts. Then use this filtered clip as the original, re-do the same tests that were done in kilg0r3's post from 20th June 2003 07:54, and see if the custom quant fares any better.
If the custom quant does do better in this situation, it would indicate to me that the reason the custom quant had lower psnr in the original test is because it was leaving out more noise, not that it was taking out detail.

[kilg0r3]

sorry no time atm

[Soc]

Sorry to bring up an old-ish thread, but I have some questions/thoughts...

Dividing all the numbers in a quant matrix by 2 is equivalent to using the original matrix but having quants of [1, 1.5, 2, 2.5, ..., 15.5] instead of [2, 3, 4, 5, ... , 31]. This helps all the frames with quants < 16, but would force all the frames >16 to have quant 15.5, which could hurt their size quite a bit. The best solution would be to somehow help the low quantizer frames but still be able to have quants up to 31.
Originally posted by Nic
Ill have a think and post something to add that's more useful. Just reading through the quant matrix part of http://bmrc.berkeley.edu/research/publications/1997/144/dbrown.html as I think...
1. In the article you linked to, they say that a separate quantizer is used for each block, and is limited to 5 bits to keep the block sizes down, but it seems like with MPEG-4 we just use one quantizer for the whole frame, is this correct?

2. If we only have one quantizer per frame, then I'm confused as to why it is still limited to only 5 bits. Adding say another 2 bits would only increase filesize by about 50K and would allow 4 times the resolution of the quantizers, eg we could use 2, 2.25, 2.5, 2.75, or 3 instead of just 2 or 3. Am I missing something, or has this simply never been done because of MPEG-4 compatability?

3. Assuming increasing number of bits is out of the question, then we should still be able to make better use out of the 5 bits we have, by choosing a much more efficient scale for the quantizers. As has been pointed out, the difference between 2 and 3 is relatively large compared to 4 and 5, quantizers above 16 are hardly used, and for some encodes quantizer 2 is not good enough. The solution to all these, if possible, would be to use an exponential or similar scaling system (based on some analysis), that would look something like [1.65, 1.82, 2.02, 2.23, 2.46, 2.72, 3.00, 3.32, ... , 29.96] (ie exp(.5) to exp(3.4)) instead of [2, 3, 4, 5, 6, 7, 8, 9, ... , 31]. The big question is, is it possible to do this and remain MPEG-4 compliant? What other problems would there be?


Please let me know if I've missed something (I am still very much a n00b to this field, after all). I hope I'm not just being a blithering idiot :p

- Soc

[EDIT] Typo: wrote :P instead of :p :D