Variable Matrices - A New Quality Concept

[TheCreamCrackerBoy]

VARIABLE MATRICES
A New Quality Concept For XviD Encoding

Here is a new idea for the future versions of XviD: with 2-pass encoding, what about the codec - at the first pass encoding - designs an automatic matrix, according not only to the given bitrate, but also for the GOP structure?

I've realized that all matrices that I've found has 128 bytes of data. What about these 128 bytes of matrices were inserted each time that there was a new GOP at the video file? With this data, we could achieve the best matrix for a moving scene GOP, another best matrix for a static scene GOP, another best matrix for a dark-scene GOP, another best matrix for high-contrast-luminosity GOP, another one for great colors GOP, etc.

This, of course, should speed down encoding, but also should increase image quality. This should looks better than using only 1 matrix (ex.: Jawor 1-cd).

How should it work? :rolleyes: Let's consider 2-pass encoding:

* First-pass encoding: the codec should start with an I-frame (ex.: Frame 0). When the codec determines that it is time to make another I-frame (ex.: Frame 62), the codec should analyse Frames 0-61 and creates the best matrix, considering luma, contrast, color and motion for this GOP. This matrix should be inserted associated with fhe first I-frame (Frame 0). When the codec determines the next I-frame (ex.: Frame 96), it should analyse Frame 61 (the first "new" I-frame) until Frame 95 (last frame of this GOP) to determine another matrix for this GOP, considering the same informations of this group of frames(luma, contrast, color and motion). This should be done each time that a new I-frame is determined.

* Second-pass encoding: The codec should encode each frame according to its matrix and GOP.

This is only a suggestion, but I am not a programmer, neither some kind of codec-developer. There are a lot of details of video compression that I don't know. I hope I am not writing anything nOObish.

Thanks

The Cream Cracker Boy

[Manao]

A similar idea was called modulated quant some times ago ( only giving the codec the choice between mpeg and h263, for each frames ). It was implemented, but found useless ( in addition to being non mpeg4 compliant, i don't remember why ).

The idea is nice, of course, not too hard to implement ( though perhaps not totally mpeg4 compliant, at least with low profiles ), but how would you find your "best matrix for the current GOP" ? That's easy to say, harder to do. Nobody yet came with criterions to say : "in such case, you should use matrix 'whateveritsname'"

[Blue_MiSfit]

XviD tried this back in dev-api 3 days. It was called modulated quant (I think), and it switched between h263 and mpeg quantization.

However, its not MPEG-4 compliant at all, so the devs dropped it.

There is probably more to it than this, but I'm not really sure as I started understanding XviD when the 1.0 RC's came out.

[TheCreamCrackerBoy]

Maybe it's not MP4 compliant, but not all features of XviD are Mp4 compliant. In fact, neither all DivX options are compliant.

The idea is not to produce a Mp4-compliant file, but a higher-quality file. This "variable-matrix" processing could not be obrigatory, it could be optional, with some description just like "Variable Matrix Processing - Warning: Not Compliant, it could not be compatible with hardware players".

If this was implemented some time ago, do you have any idea why the devs gave it up?:confused:

Thanks,

The Cream Cracker Boy.

[Manao]

Maybe it's not MP4 compliant, but not all features of XviD are Mp4 compliant. In fact, neither all DivX options are compliant.Huh ? care to say which one ? I can't think of any right now.

If you want to stray away from the mpeg4 standard, go for rv10 / vp6 / snow / h264 ( especially the later ).do you have any idea why the devs gave it upIt has already been answered : uncompliant ( edit : or badly supported, uncompliancy in modulated quant was brought by the frame per frame change iirc ), and useless because nobody knows how to find the right matrix, even when the choice is limited to h263 / mpeg.

[ObiKenobi]

When it was part of XviD I remember that it really didn't offer any substantial gain in quality.

[Manao]

I also remember that the first modulated quant algorithm was to use mpeg for quant < 4, h263 for quant >= 4. Then, the new modulated quant algorithm appeared, and it was doing exactly the contrary. I state this just to show how hard it is to even find a simple criterion to know which matrix should be used.

[ObiKenobi]

Originally posted by TheCreamCrackerBoy
Maybe it's not MP4 compliant, but not all features of XviD are Mp4 compliant. In fact, neither all DivX options are compliant.

The only thing in XviD I can think of which may cause a file to not be mpeg4 compliant is full-quality first pass, but other then that it should be fully compliant.

[stephanV]

/me waits for someone to say "packed bitstream"

[TheCreamCrackerBoy]

Maybe I am disinformed about XviD options...

According to your answers, perhaps I am mistaken. I thought that not all options were MP4 compliant, just like some DivX options for hardware players.

About quality, I thought it could raise the final quality... :)

Thanks

The Cream Cracker Boy

[ObiKenobi]

Originally posted by TheCreamCrackerBoy
According to your answers, perhaps I am mistaken. I thought that not all options were MP4 compliant, just like some DivX options for hardware players.

Just cause a hardware player doesn't support a certain option doesn't mean its not mpeg4 compliant. For example, options like GMC and QPEL are mpeg4 compliant but aren't supported by mpeg4 players due to hardware limitations.

[TheCreamCrackerBoy]

Just cause a hardware player doesn't support a certain option doesn't mean its not mpeg4 compliant.


Thank you, ObiKenobi, i thought these were the same. :)

See ya,

The Cream Cracker Boy

[Soulhunter]

Yeah, XviD is MPEG4 compliant, but this damn DivX standalones are not !!!


Bye

[TheCreamCrackerBoy]

Is there any hardware player (DVD sets) that can play XviD files?

Thanx,

The Cream Cracker Boy

[ObiKenobi]

Originally posted by TheCreamCrackerBoy
Is there any hardware player (DVD sets) that can play XviD files?

Yes they all should be able to, but you aren't able to use the advanced features like GMC and QPEL and are usually limited to only 1 consecutive b-frame. I believe you aren't able to use custom matrices as well, but I'm not 100% certain on that.

[Sharktooth]

Custom matrices are not supported just by few standalones (expecially the ones with old ESS chipsets).
In most cases, if the standalone player does not have an (old) ESS chipset, a new firmware will fix the problem.

[Manao]

StephanV : packeed bitstream has nothing to do with the mpeg4 norm : it's only the way the mpeg4 stream is put into an AVI.

[i1981]

But as about that to do mechanism like adaptiv quant, but unlike it he not will more compress (higher quant), part of the picture (with what that determined by parameter whether level to brightness or determined color gamut, with which beside Xvid appear problems at compression), but on the contrary throw on litle bit on it that is to say increase quality for count of the reduction compresseses determined area of the picture

[Koepi]

Originally posted by i1981
But as about that to do mechanism like adaptiv quant, but unlike it he not will more compress (higher quant), part of the picture (with what that determined by parameter whether level to brightness or determined color gamut, with which beside Xvid appear problems at compression), but on the contrary throw on litle bit on it that is to say increase quality for count of the reduction compresseses determined area of the picture

?

Am I the only one not understanding this post?

[stephanV]

Originally posted by Manao
StephanV : packeed bitstream has nothing to do with the mpeg4 norm : it's only the way the mpeg4 stream is put into an AVI.

:)

hey, i am the last one to care if something is MPEG4-"compliant" or not. It's only what I've been told (http://forum.doom9.org/showthread.php?s=&threadid=78375&highlight=packed+bitstream+compliant)... and i believe muxing packed bitstream in MP4 doesnt work very well...

on a side note - from crusty's FAQ:


Finally you have the 'Unrestricted' profile that will give you unrestricted access to all options, resolutions and framerates. So if you want to experiment with a grayscale interlaced Cartoon with RRV at 2048x1024 @120 Fps, there's your chance. (Just don't expect it to work on anything else)
does that mean unrestricted is not "compliant"?

Am I the only one not understanding this post?
perhaps a bad case of babelfish...
;)

[ObiKenobi]

Originally posted by Koepi
?

Am I the only one not understanding this post?

No, I read it a dozen times and couldn't understand it either.

[Koepi]

To get on topic again:

MfA was thinking out loud about adaptive quant matrices (down to per-frame adaptive) a long time ago. I don#t recall every detail, but I think he even had some working code.

But it's definatly not mpeg4 compliant to use varying quant matrices within a stream (though it could be done - but that's a case for a non-compliant tree).

Regards
Koepi

[i1981]

Damn i knew that translater will mess that i want to say. Ìó level of write on English is weak to post. idea was to create mechanism similar to adaptiv quant (compresses very dark or bright areas stronger than medium ones), but unlike it. this mechanism must lower compresion on some frames (dark scene, scenes that hard to compress with Xvid ).

Sory for crapy English

[Didée]

Edit: catched on the line by i1981 ...


But as about that to do mechanism like adaptiv quant, but unlike it he not will more compress (higher quant), part of the picture (with what that determined by parameter whether level to brightness or determined color gamut, with which beside Xvid appear problems at compression), but on the contrary throw on litle bit on it that is to say increase quality for count of the reduction compresseses determined area of the picture
Let's try a wild mix of a) reading-between-the-lines \and/ b) reading minds ...

My interpretation:

"Similar to the adaptive quant concept, only that it won't compress some parts of a frame stronger (parts that are determined parametrically from the brightness level or from the color gamut - BTW by doing so with XviD, there appear problems in the encoding), but in contrary throw more bits onto them, in order to increase quality on those problematic areas of the frame."
(the "problems" in XviD encoding could also be aiming at low-contrast areas or frames, and not at AQ. EDIT: yes it does)

Or, much shorter: HVS plugin(s).

Oh, a propos ... Syskin ?



On the topic:

I don't think that varying matrices per GOP is worth putting effort in. Firstly, it can't be done on-the-fly, you need one pass with analysing, using the analysis on the next pass. And within the current 2-pass scenario, even that doesn't help too much: quantizers of 2nd pass are supposed to be quite different from 1st pass ones. But, different quants, different "optimal" matrix ... no-go ...
Furthermore, there are scenarios where even one GOP may contain e.g. different motion types (still scene slowly changing to fast motion, so no scenechange/keyframe) - you can't find a *really* optimal matrix for *that*. And in case of sequences with manymany ultra-short GOPs, other problems arise obviously ...

Bottomline: Getting this concept working *properly* would require very much effort, but the possible benefits are IMO at least questionable.

I'm much more looking forward to some HVS goodies within the current framework :)

[RadicalEd]

Originally posted by stephanV
does that mean unrestricted is not "compliant"?
It means that unrestricted allows some combinations of settings which are not supported by any individual profile of the MPEG-4 spec. RR, which is probably going to be removed anyway, is the only encoding tool outside the range of ASP, and as such, the only option which could potentially create a noncompliant stream from XviD. It's definitely not a case for the argument that XviD should be extended out of spec.

[Disabled]

Well using per GOP Matrix could be a bit overkill for the beginning. I tried using different I frame matrixes for short scenes some time ago and tried to create a little python script automating this, but stopped as I realised I would not be able to do a 2 pass mode using this (due to my little programming skills).
Wouldn't:
- encode as normal
- on every I-Frame:
- encode frame with different matrizes
- calculate psnr for every tried matrix
- use the one with the best result

Give at least a bit better PSNR results? The only thing would be the insertion of an extra 128 Bytes (?) per keyframe what could make the improvement obsolete. One could store the Matrizes only once at the beginning and refere to them, but that would most probably not be spec compliant - uhm - variable quant matrix is isn't it anyways, so what

[chilledoutuk]

Originally posted by ObiKenobi
The only thing in XviD I can think of which may cause a file to not be mpeg4 compliant is full-quality first pass, but other then that it should be fully compliant.

why would doing the first pass at full quality be non mpeg-4 compliant?

[Mug Funky]

because the tooltip says so :)

what it actually says is that if you keep your first pass, but have "fast first pass" enabled, then your stream might not be compliant.

i'm assuming this is because some of the things turned off during fast first pass might break compliance in some cases (perhaps the hearders are written saying one thing, but the stream is written without them. things like GMC).

also, full quality first pass is constand quant 2, and that could easily cause buffer underflows, which would break compliance (i'm not sure how ASP mpeg-4's vbv model works, actually. what's the max allowed bitrate?)

[akupenguin]

Originally posted by Disabled
- on every I-Frame:
- encode frame with different matrizes
- calculate psnr for every tried matrix
- use the one with the best result
- There are two quant matrices: one for Intra-blocks and one for P/B-blocks. You could only adapt the Intra matrix this way.
- The benefits of different matrices is mostly psychovisual. PSNR doesn't do such a good job of distinguishing them.

One could store the Matrizes only once at the beginning and refere to them, but that would most probably not be spec compliant - uhm - variable quant matrix is isn't it anyways, so what
Writing a new QM in each GOP header is spec compliant. (For that matter, you're also allowed to change some other settings, like Qpel.) The part that was uncompliant was the adapting per-frame.

Originally posted by Mog Funky
also, full quality first pass is constand quant 2, and that could easily cause buffer underflows, which would break compliance (i'm not sure how ASP mpeg-4's vbv model works, actually. what's the max allowed bitrate?)
XviD doesn't enforce the vbv stuff anyway. Both q2 and 2pass will probably conform to AS@L5, but neither is guaranteed.
(You can see the numbers allowed by the spec in one of the XviD config tabs. AS@L5 allows 720x576x30fps, 8mbit/s with a 1.8mbit buffer.)