Need XviD/ffvfw custom matrices

[LigH]

Dear members,

because I'm writing a tool to visualize, edit and parametrically create custom matrices, I'd like to collect lots of matrices to compare, but it is not quite easy to find them all using the forum search only. I would thank you very much for pointing me to sources of matrices or some matrix collection; currently I found a thread with the HVS and Nitrogen's collection, and one with several variants of Didée's SixOfNine, but I guess there are some more I shall know of...
__

Furthermore, I wonder if there is any matrix representation of the H.263 quantization, like I could find a "Standard MPEG" matrix in a list of matrices for patching MPEG-2 encoders. I'm afraid that I'm not good enough in reading C sources to find that out of the XviD sources. :o

[malkion]

It's kinda redundant since you already have the HVS and Nitrogen collection LigH, however, the Ni Hao build of XviD and all RC builds comes with custom matrices bundled right in. Those are prolly the major ones that I've seen on Doom9.

Edit: You can find the standard and MPG matrice in the bundle!

[Teegedeck]

I suppose LigH knows that very well. Here's a matrix Didée once introduced in this thread (http://forum.doom9.org/showthread.php?s=&threadid=65624&perpage=22&&pagenumber=2).

iago called it half-insane, I typed it in and saved it as 'semi-insane'.
Comes without warranty, typos may be included. It produces a bitrate that brings the encoded movie near to DVD-size at full resolution.

[LigH]

Thank you, Teegedeck - yeah, it looks quite horrible... :p

@ malkion: If you think there are not many more really important matrices, I'll be able to survive that; indeed, it seems not necessary to collect useless matrices. ;)

But still, I wonder if H.263 quantization can be displayed as such a 'ubyte' matrix.

And to you XviD developers: IMHO, you shall use a specific filename extensions, instead of "*.txt", for custom quantization matrix files; for example: "*.cqm" (Custom Quantization Matrix), or "*.qmatrix" (the same as in ffvfw). Else someone could guess that it is human-readable text, which they are definitely not!

[Teegedeck]

Err... No, not quite what I meant. :) At quant=2 the bitrate might come close to that of a DVD but it still looks good at quant=5. So it is a real option for veryclean sources at high bitrates.

[LigH]

Probably the encoded video looks fine; just the matrix looks a little "horrible" - which means "I can imagine how it works by looking at it and guess why it was called semi-insane".

[malkion]

Ligh. Well... for some reason all 3 andrea's produce artifacts on my current system, bulletproof's hq matrix is decent I suppose, but I did defer to HVS-best picture over it. Never tried CG matrix nor Ultimate. Currently favorable to 2 cd encodes, so that leaves the rest out in lieu of the standard or Mpeg matrice. 6 of 9 is nice, however my last video was hi-motion and it produced a much too big 1st pass to my liking. I guess my most heavy usage falls to hvs-best picture, that about sums it all up. If you know of more matrice akin to hvs-best, 6of9 or mpeg, attach 'em here. Be more than pleased to try them out.

[P0l1m0rph1c]

Well, i have all those matrixes mentioned above, plus Fox Home Entertainment, Kika's TrickFilme High, SifOfNine-HVS and Acaila's Matrixes here (http://dcoder.members.easyspace.com/Matrix/).

EDIT: added "28 days later" matrix.
EDIT2: You can get all of them zipped here (http://dcoder.members.easyspace.com/XviDQuantMatrices.zip) .

[Didée]

@ malkion

Tip: With high-bitrate matrices like Andreas' or 6of9(-HVS), it is not a bad idea to do the 1st-pass at quant 3. ;)

- Didée

[Teegedeck]

Will this also scale correctly when you expect your 2nd pass to be partially quant=3, anyway?

[Didée]

Originally posted by Teegedeck
Will this also scale correctly when you expect your 2nd pass to be partially quant=3, anyway?
Yes, of course. Why should it not?

With 1st-pass @ q3, the overall scaling should be a little more easy to XviD, since it doesn't have to scale down from the "huge" q2-framesizes to the later used "normal" framesizes.
And if you aim for a quant-range of [q3,qX], there is no problem at all: q3 in 1st-pass, q3 in 2nd-pass ...

Even if you're going to possibly let the codec use some q2-frames, it should work OK: compare this to an encoding with standard-matrix and quant's of q1 allowed, like it is for some time now.
With 6of9-HVS, the "overshoot" of q2 compared to q3 is not as big as the "overshoot" of q1 to q2 with the standard-matrix. Over the thumb, very roughly, the following numbers are valid:

standard: [framesize_q1] == 2.0 * [framesize_q2]
6of9-HVS: [framesize_q2] == 1.6 * [framesize_q3]

Exact numbers depend on many things, but the relation will just stay like that.

Therefore, less problems even in this situation: if the codec will use quantizer overshooting to compensate for underflow, the scaling and compensation for the forced overflow should be easier to handle.
(However, usually I cap my quants at q3, and use q2 only manually through zones, if I feel certain scenes need bits, bits, bits ...)

- Didée

[Teegedeck]

Hm, now if only fast-first-pass would work with quant=3, too... Errr, of course I can just check 'turbo'...

And thanks for your matrices, by the way!

[LigH]

Thanks to all users posting their matrices, collections and opinions.

But one question, please: Why are some users not allowed to post attachments here?! I'm not, EthanoliX is not...

[Teegedeck]

Simply because there are HD space problems on the forum server ATM...

No special treatment (except for mods who still can attach stuff).;)

[LigH]

Originally posted by LigH
And to you XviD developers: IMHO, you shall use a specific filename extensions, instead of "*.txt", for custom quantization matrix files; for example: "*.cqm" (Custom Quantization Matrix), or "*.qmatrix" (the same as in ffvfw). Else someone could guess that it is human-readable text, which they are definitely not!

Furthermore: The file extension ".txt" makes some servers tell a MIME type "text/plain", and this makes clients (e.g. web servers) believe that the file was sent in ASCII mode. This means: If the file contains bytes with a value 13h, a Windows FTP/HTTP client appends a byte with value 10h to match the line break rules - this makes the file larger and destroys the matrix this way! Now imagine Acaila's matrix...

[malkion]

OT, however, thanks for the tip Didee on using quant 3 encodes with high bitrate matrices.

Offhand observations, I personally feel using Q3 helps with codec saturation, and the pictures also appear more life-like. However, if you enjoy that film grain texture look, Q3 might not be the best. Also tried Q4, Q5. On my encodes, I noticed from Q4, you start losing details in haze or scenes with smoke. Q5 detail becomes compromised however, still viewable. However, since with myself size is an issue, especially on this particular video, using Q3 did not noticeably save bits for me. Sized is shaved, but not enough. Also I enjoy the look of 0 b-frames on 2:35-1 movies and can still hit 2 CDs. for 1:85-1 movies, I have to move up to 1 b-frame. The movie looks better visually (personal view) the less b-frames you use. Also I compress to around 70% of first pass so blocks do not appear in parts of the video. Thanks for reading my 2 cents. That's it for now.

[malkion]

@Didee

I finished some 2nd passes using Q3 encodes. It did not hit desired filesize and was undersized. Instead of using all Q3, I used Q3-31. 1st pass was 2,336,586. 2nd pass desired size was 1,680,582, actual was approx 1,280,000. Visual quality was considerably worse than the 1st pass video however, even though it compressed to 54% there were no blocks present. I was hoping for 72% compression. The test is concluded, I guess I have some unspoken questions. lol.

[Didée]

malkion:

I'm not sure I can really follow you here:

> I finished some 2nd passes using Q3 encodes.

I hope that doesn't mean you did the 2nd-pass with the q3-1st-pass *as input* ? - If so, then everything is *totally screwed*, of course!

> It did not hit desired filesize and was undersized. [...] 1st pass was 2,336,586. 2nd pass desired size was 1,680,582, actual was approx 1,280,000.

This sound even more as if you used the 1st-pass as input for the 2nd! (Taking a 1st-pass as input will compress much better than the original source, but give bad quality.) Especially, since your 2nd pass

> compressed to 54%

it is more than unlikely that you'll get undersizing if the compression it in that range!

> Visual quality was considerably worse than the 1st pass video however

That's not surprising, because the (unwanted) compression to 54% of a 1st-pass @ *q3* is indeed too high, wheras

> I was hoping for 72% compression

would have been a very good aim. But I wonder, why you were *hoping* for a certain compression?
You *tell* the codec what compression you want, and then you *get* that compression, *exactly*.

So, once more:

1. 1st-pass: set up one zone, with forced q3 within

2. 2nd pass: to perform on the same very source as the 1st pass! NOT on the 1st pass!!
Set up one zone with weight=1.0, or perhaps several zones to your likings and needs (credits, etc.)


Okay? (Sorry if I misunderstood your post)


- Didée

edit: typo

[malkion]

@Didee.

> Okay? (Sorry if I misunderstood your post)

Thanks for the consideration. Erm, I think perhaps my post was taken a little too literally. Anyways, instead of using zones and weights, on my passes, I limited the Quants to 3-31 in advanced properties. Then I ran a 2nd pass on the same source using the same settings with bad results. File size was skewed. I encoded a new 2nd pass freeing the quants back to 1-31 in advanced properties, using the source .pass stats (since they were all Q3 frames) and XviD hit the file size perfectly. Now the movie is very nice to watch.

I believe we used the same principle, yes? (I understand I may use zones to provide more flexibility, however..)

Edit: How foolish. It seems when I used advanced properties to set up and limited min quants to anything but default, whether 2 thru 10 for minimum and left max at 31, is what I tested, the 1st pass encoded at Quant 2. Oh well. Live and learn, and on to the next encode... with zone options no less.

[LigH]

Are some serious users interested in looking at my "Custom Quant Matrix Editor" (or so to say, the development stage it reached so far)? Orders, including some reputation (why you may be worth to be a serious tester), please to me per PM. It is not quite final, I just would like to know if it is at least on the right way...

[LigH]

Here my current "probably complete" quantization matrix collection:

http://www.ligh.de/software/qmatrix.zip

[P0l1m0rph1c]

Ahh, nice collection! Thanks for gathering all those in just one archive :)

This should go sticky or at least should be well documented, so other users won't start threads asking for matrixes all over again...

[Piaf]

just saying Cheers to all for this collection :D ;)

[TorgoGuy]

Originally posted by LigH
Here my current "probably complete" quantization matrix collection:

http://www.ligh.de/software/qmatrix.zip
This link is gone already. Does anyone have this file?

Thanks.

[P0l1m0rph1c]

http://moolabs.dcc.online.pt/downloads/codecs/qmatrix.zip

[LigH]

Gone? :confused: - Try upper case, too:

http://www.ligh.de/software/QMATRIX.ZIP

(There is some confusion between Win2K, Win98, and the FTP server, sometimes when a filename is short enough to fit 8:3).

Recently added: "Packed DVD by Fl0ppy"

[TorgoGuy]

Originally posted by P0l1m0rph1c
http://moolabs.dcc.online.pt/downloads/codecs/qmatrix.zip
Thanks!

[TorgoGuy]

Originally posted by LigH
Gone? :confused: - Try upper case, too:

http://www.ligh.de/software/QMATRIX.ZIP

(There is some confusion between Win2K, Win98, and the FTP server, sometimes when a filename is short enough to fit 8:3).

Recently added: "Packed DVD by Fl0ppy"
Yes, all uppercase made it work for me. Thanks for the collection.