Custom Quantization matrix

[ReferenceDivx]

I was wondering if anyone had experimented with using different matrices.

I just tested a HVS (Human Visual System) luminance matrix i read about in an IEEE paper. I ran it through my image quality metrics program that i wrote and noticed that its claim of about .2db psnr was indeed true(which does not necessarily guarntee a better picture).

the matrix is as follows

16 16 16 16 17 18 21 24
16 16 16 16 17 19 22 25
16 16 17 18 20 22 25 29
16 16 18 21 24 27 31 36
17 17 20 24 30 35 41 47
18 19 22 27 35 44 54 65
21 22 25 31 41 54 71 88
24 25 29 36 47 65 88 115

mpeg amse = 4.66611 apsnr = 41.5306
custom HVS matrix amse = 4.47718 apsnr = 41.7052
amse is average mse
apsnr is average Peak Signal Noise Ratio

I just tried the best setting in Divx 5.02
and got amse = 6.46222 apsnr = 40.1583

[SirDavidGuy]

Once I made one for very low bitrate encoding, but it has since been deleted. Beyond that, I didn't do much with it, because I was experimenting with different aspects of XviD the source.

[rui]

I would like to know if someone already tested this new matrix.

I would do it myself (those who know me know that i don't have problems in testing), but since several days back, i am having computer problems (hard drive failure, still waiting for the replacement), and my testing is reduced to almost zero :( (can't stress the hard drive much or it will failure completely)

But i believe that, if this costum matrix is good, it could represent a step foward in encoding. I mean, look at the diference between h.263 and mpeg (some like h.263 more, others, like me, mpeg, but all see the diference).

ReferenceDivx: did you experienced any diferences visually between a encode made using your matrix and the default mpeg?

[sierrafoxtrot]

@rui

i've tested it :D , and it seems to reduce some of the ringing artifacts caused by the default MPEG matrix. still, i'd like to experiment more with tweaking the H263 matrix than tweaking the H263 matrix. does anyone know if the differences between MPEG and H263 quantization are solely due to their different matrices or are the DCTs done differently?

also, reading up several older threads, i think koepi mentioned that inter H263 quantisation matrix was closer to 32 for all values (all frequencies get equal weight). what values are used for the intra matrix? if someone could post defaults, i can have a starting point to experiment ...

thanks again.

ps: i've done several google searches for H263 and MPEG matrices, but not much relevant info ...

[rui]

Hi sierrafoxtrot :)

It's good to see that you still hang around here. I am starting to feel a little "old", seeing all the new people that comes here :D

But it's a good sign, meaning that xvid is getting bigger and bigger.

Thanks for the test, i can't wait to make some of my owne. :)
That "ringing" efect is the only downside of mpeg, IMHO.
That's why i like the new HQ mod. that Koepi introduced in his latest builds. By using h.263 in quants 2 and 3, it keeps the ringing in edges smaller, and the image is almost as good as using mpeg. (This is the efect that you mean, right?)

[Koepi]

Hehe, still it's better to tweak a matrix to behave that way like mod. HQ or new mod. HQ.

I think I'll test drive that quant. matrix in some days :)

Best regards,
Koepi

[Dali Lama]

Hello,

Well I have tried that custom matrix at high quantizations ~8, and I can say it underperforms h.263 in blocking and ringing artifacts. Also, some disformity of shapes are evident. h.263 is surprisingly good at retaining proper object shape at high quants.

That's all for now, maybe more testing is needed...wait a minute of course more testing is needed :sly:

Dali

edit: sierrafoxtrot compared custom vs. mpeg, while I compared custom vs. h.263...just to clarify.

Also, I agree with sierrafoxtrot about trying to base our custom matrices off of h.263.

edit again: Thanks ReferenceDivx for starting a cool thread.

[Rrrough]

Hi, I tested it with quantizers >5 and it looks great, I think.

@koepi

is it possible to change the behaviour of modulated quantizer mode to alter between h263 and MPEG-custom quantizers instead of the build in matrix???
don't know if it's worth it, just an idea...

cheers

[rui]

Ok, after reading Dali Lama's experiments, this is what i think:
maybe we can use this costum matrix in those movies that we know will use low quantizers, like 2 cd movies, or very compressible movies.

In this cases i always used mpeg because i like a lot the "detail" it keeps (the most relevant experience i have are the wrinkles in peoples faces, they are more visible when using mpeg than h.263).
But lately i noticed that mpeg has more "ringing" efect in edges than h.263
So, i started to use Koepi's HQmod to try to avoid that (by using h.263 in very low quantizers). But, if the new costum mpeg is better at that, in low quantizer movies we can start using that.

In those movies we know that probably will have higher quantizers (1 cd or hard movies, like LOTR, we can continue using either h.263 or the old modulated option.

Or better yet, a costum h.263 matrix, like you two said ;)

EDIT: i was typing this before reading Rrrough experience. His idea is not bad IMHO
But the problem is that new costum matrixs, even better than this one could appear, andn Koepi would have to adjust everytime his builds to them. On the other hand, look at the time it took to appear a costum matrix ;)

[Rrrough]

well, actually I was thinking of a new modulated mode wich is using the custom-matrix as entered in the dialog-box, not the hardcoded one. That way, whenever there's a new (better) matrix, you enter/load it and it's being used in modulated mode as well, and hasn't to be hardcoded by Koepi. That way you could also use the default matrix with modulated mode just by entering/loading it, if that's preferred.
I guess my explanation sucks big time, but I hope someone (esp. koepi) gets the point... :p

cheers

P.S.: taking Dali's experiences into account, I guess taking this idea into new modulated HQ would be counter-productive then, so I'm referring to "classic modulated"

P.P.S.: I was using it in combination with Lanczos Resize and the visual result was astonishing, to say the least

EDIT : Oops, sorry, I was using quant 2-4...:o sorry for the confusion from first post : it's quantizers < 5... :rolleyes:
I will try higher quantizers later...

[sierrafoxtrot]

hello all!

@rui, i know what you mean by all these new faces. i feel like an old-timer ... :D

anyway, went and re-read some old threads and it looks like we can only customize MPEG quantization because H263 is done differently. i guess i really like H263 quants, because of fewer ringing artifacts, and if we can find values for MPEG quant tables that gets rid of that, i'm all for it.

one of koepi's old posts suggests that sticking 32s in the inter-frame matrix will give quantisation similar to that of H263, so i might try that. i just got shakespeare in love R2, and it's fairly difficult to fit on 1CD, so i might try the "all-32" matrix to see if it helps.

just a thought, has anyone tried changing the values in the intra-frame matrix? because that is frame from which other subsequent frames are "predicting", would it be worthwhile finding an intra-quant matrix that reduces ringing noise?

another question ... where does ringing noise come from? i know -h said that MPEG quantisation introduces noise but can it be gotten rid of by changing the matrix so that the frequencies that give rise to it are compensated for? (i'll try this tonight when i get home ... so many tests so little time!)

sorry bring more questions than answers (and not being able to express myself well enough), but i want to spark some sort of debate.

here's a couple of useful threads:

http://forum.doom9.org/showthread.php?s=&threadid=18938&highlight=custom+matrix

http://forum.doom9.org/showthread.php?s=&threadid=22805&highlight=custom+matrix

@rrough: that's a really good idea, hope koepi's listening ...

[rui]

Well, first i must confess that i didn't got Rrrough idea correctly. You are correct when you said i was thinking that you wanted Koepi to hardcode the costum matrix.

But now that he explained, i agree with him ans sierra, it seems good.

But the point is to know if the new costum matrix is better at lower quants than h.263 (i mean keeping those details and not introducing noise in edges). If the answer is yes, then maybe it would make sense to trade the h.263 for quants 2 and 3 in the new HQ modulated mode for the costum matrix, but letting the user decide what that costum matrix is, like Rrrough said very well.

Koepi, this is just a thought, i am not asking for new features :) Is by trading this ideas between us that xvid goes ahead.

I for one, like mpeg all the way, the only reason that i sometimes use HQ mod is for the noise part. If the costum matrix eliminates that, i probably would use it all the time.
But others, like Sierra and Dali think otherwise.

[iago]

Originally posted by rui
I for one, like mpeg all the way, the only reason that i sometimes use HQ mod is for the noise part. If the costum matrix eliminates that, i probably would use it all the time. Well, I just wanted to say that I agree with rui on MPEG quantization. In my experiences so far, it results in less blockiness and better color representation than h.263, though producing more edge noise/ringing artifacts (depending on the resize filter as well).

regards,
iago

[Rrrough]

@Dali Lama

I can confirm that it doesn't really perform very well on quant 8 and above... :(

@SirDavidGuy

Once I made one for very low bitrate encoding, but it has since been deleted. Beyond that, I didn't do much with it, because I was experimenting with different aspects of XviD the source.

how was it performing @ low bitrates ??? Any good ?
that would lead me to an even nastier idea, which I will tell as soon as Koepi is responding...

cheers

[SirDavidGuy]

how was it performing @ low bitrates ??? Any good ?

Relatively good, I'm working on ways of optimizing them for low bitrates as I type (Low as in 300 kbits a second).

[Belgabor]

Has anyone ever thought about modulating between more than two matrixes? like

quant 2-5: new matrix
quant 6-10: h263
quant >10: low bitrate matrix

Cheers
Belgabor

[Koepi]

We even thought about a quantizer matrix for every quant from 1-16 (and some even up to 32).

Regards,
Koepi

[Rrrough]

@Belgabor
yep, that would have been my nasty idea...

@Koepi
do you think that's within the reach of reality ???
something like a quantizer-matrix config file...???

cheers

[SirDavidGuy]

Testing goes slowly;

Which ever build I was using at the moment didn't work 100% with the settings I was using; I had to use my own.

[Dali Lama]

Hi guys,

Don't have much time, but wanted to share some testing I did last night.

After performing quant 2-8 tests using 4 matrices, I have come to this conclusion:

h.263 quants 2-3
mpeg quants 4-5
low quants 6-8
ultralow 8+

I have attached the low and ultralow matrices for those who wish to test also.

Note: Those matrices were taken from CCE SP Custom Matrices.

Take Care,

Dali

[Dali Lama]

Hi guys/gals,

Don't have much time, but wanted to share some testing I did last night.

After performing quant 2-8 tests using 4 matrices, I have come to this conclusion:

h.263 quants 2-3
mpeg quants 4-5
low quants 6-8
ultralow 8+

I have attached the low and ultralow matrices for those who wish to test also.

Note: Those matrices were taken from CCE SP Custom Matrices.

Take Care,

Dali

[Dali Lama]

Here is the Ultra Low Bitrate Matrix

[Rrrough]

@Dali Lama

great news :D:D:D can't wait until your attachments appear, thanks for posting them !!!
which build did you use ?

@Dave

what kind of problems did you run into with "regular" builds ???

@ReferenceDivx

could you please post your image quality metrics program or a link for it ? I would like to test Dali's matrices both visually and objectively.

cheers

[ReferenceDivx]

I don't exactly know what i want to do with the program i'm developing. I've been doing research into video quality metrics and the Human Visual System. I've been working on a program for the last few months that include ideas from the most recent papers and my own ideas about HVS. I'm wondering if i should make the program open source, or make it closed source. I've also been debating if i should make the web site into a research site that has papers on the newest ideas and trends in HVS. My goal is to design a new quality metric that is within 85-95% of how human perceive the quality of video. I am only really interested in mid to high bitrate video streams. Once i have my video metrics i will use some Neural networks or some fuzy logic to find the best ratios and weights for the metric's values. My final goal would be to make a new "psnr" that is almost as fast but would judge the "true" quality of dct based video streams. Finally, I would like to come up with a numbering system that would give a quality metic number to the overall video(taking into consideration many factors). My new site lives at http://www.hvsmetrics.com (however there is only an apache page up now)

Oh ya i would like to get a computer programming job someday too.

[Rrrough]

@ReferenceDivx

:o sorry, didn't intend to sound demanding in any way, didn't know it's that complex. I deeply respect your decision if it'd be keeping it closed-source (of course an open-source version would be nice as well:)). I just thought it could be a valuable tool for testing different matrices and thus maybe improving xvid's already perfect visual quality a bit more...
that's definetly an eager project you're working on, I wish you good luck with it (as well as with the job)

cheers

[Dali Lama]

Greetings,

Whoever is interested in this thread needs to see the thread about DCTune in the General Forum. Exciting stuff could happen.

Bye,

Dali

Edit: @ReferenceDivx, DCTune has a Quality metric. I think you might be interested in that.

[SirDavidGuy]

@Dave

what kind of problems did you run into with "regular" builds ???

It crashed at the beginning of the encoding.

For what reason, I am uncertain of.

[iago]

@all

Though keeping a close eye on this thread for a while, I've just found the time to start a test encode with the new matrix that ReferenceDivx posted. The movie is Salvador (1hr-57min), somewhat hard to compress I guess ;), 512*272, SimpleResize, aiming for 666000kb to fit on 1CD (700mb) with ogg audio. Using Koepi's 04092002-1 build, no lumi, max-min I-frame intervals: 300-5, credits 20quant, (and external cc, quants 2-6/2-16, payback proportionally, payback delay:300 for the second pass). BTW, I'll have the chance to test Statsreader 1.9 for the first time as well, Koepi; besides seeing how the matrix behaves with high quantizers in a full 2-pass encode ;).

And I really wonder how the result will be...

best regards,
iago

EDIT: Unfortunately I couldn't linear-scale the curve for external cc, since statsreader 1.9 crashed and refused my stats file; so I'm going for internal cc with altCC defaults. And average quantizer will really be pretty high with the above target size I guess, since the first-pass size is ~2150 mb ;)...

[SirDavidGuy]

My low-bitrate matrix is going well, but I am occasionally getting a purple macroblock jumping out of no where, any ideas?

[-h]

Purple! Goodness I haven't seen anything like that for a while.

Are you using B-frames? Lumi-masking? What are the lowest/highest values in your matrix? Does the lumi component appear correct, and it's just the colour that's wrong?

-h

[SirDavidGuy]

Are you using B-frames? Lumi-masking?

Nope.

What are the lowest/highest values in your matrix?

(Going From Memory) 12, 99.

Does the lumi component appear correct, and it's just the colour that's wrong?

It too appears incorrect. I'm still looking for a reason.

[-h]

It too appears incorrect. I'm still looking for a reason.

How very curious. Do the artifacts still appear when decoding with ffdshow? It seems we have a bug in custom-matrix quantization.

The only other thing I can suggest is disabling MMX optimizations and trying again.

-h

[iago]

hello again,

With the encoding parameters given in the above post, the second-pass results are as follows:

Quantizers Analisis
---------------------

Quantizers Used For Movie :
------------------------------
Quant 2 Used : 131 Times, Percentage Used : 0.08%
Quant 3 Used : 3736 Times, Percentage Used : 2.19%
Quant 4 Used : 44102 Times, Percentage Used : 25.86%
Quant 5 Used : 73548 Times, Percentage Used : 43.12%
Quant 6 Used : 40827 Times, Percentage Used : 23.94%
Quant 7 Used : 7415 Times, Percentage Used : 4.35%
Quant 8 Used : 699 Times, Percentage Used : 0.41%
Quant 9 Used : 72 Times, Percentage Used : 0.04%
Quant 10 Used : 28 Times, Percentage Used : 0.02%
Quant 11 Used : 1 Times, Percentage Used : 0.00%

Average Quantizer Used for Movie : 5.036

Quantizers Used For Credits :
--------------------------------
Quant 20 Used : 5964 Times.

MPEG Quantization Type Used 176523 timed, Percentage Used : 100.00%

Quantizers prevented from rising too steeply 68 times


Intra-Frame (Key-Frame) Quantizers
------------------------------------

Movie
-------
Quant 6 Used : 1304 Times, Percentage Used : 98.56%

Credits
---------
Quant 20 Used : 19 Times, Percentage Used : 1.44%

Inter-Frame (P-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 131 Times, Percentage Used : 0.07%
Quant 3 Used : 3736 Times, Percentage Used : 2.13%
Quant 4 Used : 44102 Times, Percentage Used : 25.17%
Quant 5 Used : 73548 Times, Percentage Used : 41.98%
Quant 6 Used : 39523 Times, Percentage Used : 22.56%
Quant 7 Used : 7415 Times, Percentage Used : 4.23%
Quant 8 Used : 699 Times, Percentage Used : 0.40%
Quant 9 Used : 72 Times, Percentage Used : 0.04%
Quant 10 Used : 28 Times, Percentage Used : 0.02%
Quant 11 Used : 1 Times, Percentage Used : 0.00%

Credits
---------
Quant 20 Used : 5945 Times, Percentage Used : 3.39%

Frame Analisis
----------------

Number Of Intra-Frames (Key-Frames) : 1323
Number Of Inter-Frames (P-Frames) : 175200
Total Number Of Frames : 176523

0.75% of the Movie is Intra-Frames (Key-Frames)
99.25% of the Movie is Inter-Frames (P-Frames)

Size Analysis
----------------

1-Pass Size : 2270629239 Bytes or 2217411 KBytes or 2165 MBytes
Scaled Size : 676026371 Bytes or 660182 KBytes or 644 MBytes
Actual Size : 675989793 Bytes or 660146 KBytes or 644 MBytes

Usefull Statistics
------------------
Compressibility : -33.39%
Relative Quality of XviD avi : 39.71%
Absolute Quality of XviD avi : 90.89%

---------------------------------------------------------------

Doesn't seem very bright, yes? ;)

But surprisingly the encode looked pretty good to my eyes and it is very watchable imho. A very subjective view of course, but I really liked the result I got with that custom-matrix, though I couldn't linear-scale the curve (due to statsreader 1.9 crash), which I guess would improve the result even more...

best regards,
iago

[Koepi]

There's something wrong with cust. quant matrix in XviD, I can confirm that. The min. frame size rises from 107 bytes @880 macroblocks/image to 189bytes(!!!) - 82bytes per frame...

~10.4MB lost for my movie... back to MPEG quant type...

Regards,
Koepi

[iago]

@Koepi

That's really bad news :(. I'd started to like the results it gives... Yeah, too many bytes lost then.

thanks,
iago

[-h]

There's something wrong with cust. quant matrix in XviD, I can confirm that. The min. frame size rises from 107 bytes @880 macroblocks/image to 189bytes(!!!) - 82bytes per frame...

Sounds like it's forcing a VOL header every frame. I don't think that's desired behaviour..

-h

[rui]

I made a test using constant quant 4 with normal mpeg and costum mpeg, and the costum mpeg one was bigger by about 500 Kb.
The test was The Replacements trailer, so 500 Kb is significative here.
In a limited size test, i assume that costum matrix would give me higher quantizers.

[Dali Lama]

Ok Guys/Gals, here it comes...

CG/Animation Matrix (requires very low noise source)
MPEG (used to define the MPEG-2 Standard, sharp @ high bitrates)
Standard (a general purpose matrix designed for natural film sources)

[list=1]
CG/Animation - (requires very low noise source)
MPEG - (used to define the MPEG-2 Standard, sharp @ high bitrates)
Standard - (a general purpose matrix designed for natural film sources)
Low Bitrate - (inter and intra frame smoothing)
Very Low Bitrate - (more inter and intra frame smoothing)
Ultra Low Bitrate - (high inter and intra frame smoothing)
Ultimate - (sorry couldn't help myself :D This is my custom matrix that I like a lot for clean sources)
[/list=1]

Note: All matrices come from CCE SP, except for Low Bitrate (which I modded from Very Low Bitrate) and Ultimate...:sly:
File is attached as zip. All matrix text files are there.

Note again: I am disappointed by the news of larger filesizes with Custom Matrices, but I am sure that will be resolved.

Now lets test!

I'll post my experiences later,

Dali

[Rrrough]

@-h

didn't you look up the size difference between VOL and VOP header and found a value about 35 bytes ???
if so, where could be the difference coming from ???
(edit : the difference between 35 and 82, of course :sly: )

@Dali

I like the low bitrate matrix for quants ~5-7 from visual experience, but still struggling with DCTune to give me reliable results...

cheers

[-h]

didn't you look up the size difference between VOL and VOP header and found a value about 35 bytes ???
if so, where could be the difference coming from ???
(edit : the difference between 35 and 82, of course :sly: )

The quantization matrix requires 64 bytes, one of them is probably being written for some odd reason.

-h

[Koepi]

-h,

as you're much more firm with the core code, can you look into the cust. quant matrix code and see if there's a simple fix for this issue? Or should I send out a bugreport to the devel-list?

Thanks,

best regards,
Koepi

[-h]

as you're much more firm with the core code, can you look into the cust. quant matrix code and see if there's a simple fix for this issue? Or should I send out a bugreport to the devel-list?

Yep, I'll have a look when I wake up tomorrow. 4 AM is a tad late for me :)

I spent tonight working on a faster set of bitstream functions. They should speed up encoding and decoding, especially when low quants are used. I haven't benchmarked it naturally, but there are half as many branches and instructions.

It's fun working on this stuff again :)

-h

[Rrrough]

@-h

I spent tonight working on a faster set of bitstream functions.
Hooray !!! sounds great.
Sorry for asking trivial questions, but that'd mean 35 plus 64 bytes for VOL header for both intra and inter matrix ??? aren't they 64 bytes large each one ??? that'd come close to the once proposed 140 bytes overhead for those headers then for a matrix-switch, right ? Are they stored in combination or seperately then ?
:confused: ??? confused ??? :confused:

sleep well

[iago]

@all

As the problem with custom-matrices is detected, I'm very hopeful that it will be resolved eventually ;). So we can begin to experiment with them with more reliability. Currently I'm waiting for this issue to be resolved.

P.S.: I'm really looking forward to try especially your "Ultimate Matrix" Dali! ;)

regards,
iago

[MfA]

When you benchmark the code could you post the results -h? I found a pretty nice adaptive quantization method which Id like to try out, but given how fast DQUANT bits start dominating any savings Id like to put in an explicit check against that. Of course at the cost of repeated quantization and bitstream coding, just wondering how much the costs would be.

It's a paper called "A perceptual model for JPEG applications based on classification, texture masking, and luminance masking" BTW. It is totally ad-hoc and empirical, or in other words actually usefull :) (Unlike say Watson's stuff, which I am ashamed to admit I was totally unfoundedly fond of in the past.) It will require DCT coefficients from the original frames, but coded right that should only take about 1-2 millisecond extra per frame on a fast machine (DCT takes about 250 cycles from cache, memory bandwith wont be the limitation ... so with prefetch on a 1+ GHz that should be possible).

[Koepi]

Did you write gruel about that already? He is trying to implement something like that, and I don't know if he knows about your paper.

Regards,
Koepi

(yeah, the one who produces coal...)

[MfA]

It was just a jibe Koepi, I thought your answer in the thread was counterproductive. I tried to indicate the impression you make on me in as an unconfrontational way as possible. Yes, sure it was criticism ... but there was no spite there.

I think Gruel is more interested in standard rate-distortion, with MSE or the like, he also wants to requantize/encode to find real bitrate costs ... but that is pretty much tangential to the perceptual metric in this paper.

[-h]

I just patched core in both branches to prevent all the VOL headers being written. Decoding the new (smaller) streams works with both XviD and DivX 5.x, so all appears well.

-h

[iago]

->I just patched core in both branches to prevent all the VOL headers being written. Decoding the new (smaller) streams works with both XviD and DivX 5.x, so all appears well.

That's really great! Thanks a lot -h!
So I'm sure we'll have a new build by Koepi very very soon ;)...

best regards,
iago

[-h]

Sorry for asking trivial questions, but that'd mean 35 plus 64 bytes for VOL header for both intra and inter matrix ??? aren't they 64 bytes large each one ??? that'd come close to the once proposed 140 bytes overhead for those headers then for a matrix-switch, right ? Are they stored in combination or seperately then ?
:confused: ??? confused ??? :confused:

I didn't notice your question before, sorry!

If a custom matrix is used, the VOL header will grow by 64 bytes for each matrix which differs from the default, otherwise a bit is set that says "just use the default matrix for intra or inter".

If you want to change a matrix mid-stream, a new VOL header must be written which will bring about an additional 35 or so bytes for the header and 64 bytes for each matrix you've customised. So if it's a P-frame, you'll have 100 bytes of overhead if you specify only the inter matrix, or 164 bytes of overhead if you specify a custom intra matrix as well.

-h