collection of available AVC custom quant matrices

[bond]

1) jvt
description: the jvt matrix, used as the default custom matrix by the avc reference software (comparable to the "mpeg" matrix of mpeg-4 asp), uses the same values in all intra4x4 matrices and the same values in all inter4x4 matrices:

#jvt

INTRA4X4_LUMA =
6,13,20,28,
13,20,28,32,
20,28,32,37,
28,32,37,42

INTRA4X4_CHROMAU =
6,13,20,28,
13,20,28,32,
20,28,32,37,
28,32,37,42

INTRA4X4_CHROMAV =
6,13,20,28,
13,20,28,32,
20,28,32,37,
28,32,37,42

INTER4X4_LUMA =
10,14,20,24,
14,20,24,27,
20,24,27,30,
24,27,30,34

INTER4X4_CHROMAU =
10,14,20,24,
14,20,24,27,
20,24,27,30,
24,27,30,34

INTER4X4_CHROMAV =
10,14,20,24,
14,20,24,27,
20,24,27,30,
24,27,30,34

INTRA8X8_LUMA =
6,10,13,16,18,23,25,27,
10,11,16,18,23,25,27,29,
13,16,18,23,25,27,29,31,
16,18,23,25,27,29,31,33,
18,23,25,27,29,31,33,36,
23,25,27,29,31,33,36,38,
25,27,29,31,33,36,38,40,
27,29,31,33,36,38,40,42

INTER8X8_LUMA =
9,13,15,17,19,21,22,24,
13,13,17,19,21,22,24,25,
15,17,19,21,22,24,25,27,
17,19,21,22,24,25,27,28,
19,21,22,24,25,27,28,30,
21,22,24,25,27,28,30,32,
22,24,25,27,28,30,32,33,
24,25,27,28,30,32,33,35when you want to use the jvt cqm with x264 you can do that via the following commandline:
--cqm jvt


2) q_matrix
description: the "q_matrix", coming with the avc reference software, uses the same values in all intra4x4 matrices and the same values in all inter4x4 matrices:

#q_matrix

INTRA4X4_LUMA =
6,12,19,26,
12,19,26,31,
19,26,31,35,
26,31,35,39

INTRA4X4_CHROMAU =
6,12,19,26,
12,19,26,31,
19,26,31,35,
26,31,35,39

INTRA4X4_CHROMAV =
6,12,19,26,
12,19,26,31,
19,26,31,35,
26,31,35,39

INTER4X4_LUMA =
9,13,18,21,
13,18,21,24,
18,21,24,27,
21,24,27,30

INTER4X4_CHROMAU =
9,13,18,21,
13,18,21,24,
18,21,24,27,
21,24,27,30

INTER4X4_CHROMAV =
9,13,18,21,
13,18,21,24,
18,21,24,27,
21,24,27,30

INTRA8X8_LUMA =
6,10,13,16,19,24,26,28,
10,12,16,19,24,26,28,31,
13,16,19,24,26,28,31,33,
16,19,24,26,28,31,33,35,
19,24,26,28,31,33,35,37,
24,26,28,31,33,35,37,39,
26,28,31,33,35,37,39,42,
28,31,33,35,37,39,42,44

INTER8X8_LUMA =
9,12,14,16,18,19,21,22,
12,13,16,18,19,21,22,24,
14,16,18,19,21,22,24,25,
16,18,19,21,22,24,25,27,
18,19,21,22,24,25,27,28,
19,21,22,24,25,27,28,30,
21,22,24,25,27,28,30,31,
22,24,25,27,28,30,31,33when you want to use the q_matrix cqm with x264 you can do that via the following commandline:
--cqm4i 6,12,19,26,12,19,26,31,19,26,31,35,26,31,35,39 --cqm4p 9,13,18,21,13,18,21,24,18,21,24,27,21,24,27,30 --cqm8i 6,10,13,16,19,24,26,28,10,12,16,19,24,26,28,31,13,16,19,24,26,28,31,33,16,19,24,26,28,31,33,35,19,24,26,28,31,33,35,37,24,26,28,31,33,35,37,39,26,28,31,33,35,37,39,42,28,31,33,35,37,39,42,44 --cqm8p 9,12,14,16,18,19,21,22,12,13,16,18,19,21,22,24,14,16,18,19,21,22,24,25,16,18,19,21,22,24,25,27,18,19,21,22,24,25,27,28,19,21,22,24,25,27,28,30,21,22,24,25,27,28,30,31,22,24,25,27,28,30,31,33


3) q_matrix2
description: the "q_matrix2", coming with the avc reference software, uses the same values in all intra4x4 matrices and the same values in all inter4x4 matrices:

#q_matrix2

INTRA4X4_LUMA =
7,16,22,24,
16,22,24,28,
18,22,27,33,
22,24,32,47

INTRA4X4_CHROMAU =
7,16,22,24,
16,22,24,28,
18,22,27,33,
22,24,32,47

INTRA4X4_CHROMAV =
7,16,22,24,
16,22,24,28,
18,22,27,33,
22,24,32,47

INTER4X4_LUMA =
13,15,17,18,
15,17,18,20,
17,18,21,22,
18,20,22,25

INTER4X4_CHROMAU =
13,15,17,18,
15,17,18,20,
17,18,21,22,
18,20,22,25

INTER4X4_CHROMAV =
13,15,17,18,
15,17,18,20,
17,18,21,22,
18,20,22,25

INTRA8X8_LUMA =
7,13,16,18,22,22,24,28,
13,13,18,20,22,24,28,31,
16,18,22,22,24,28,28,32,
18,18,22,22,24,28,31,33,
18,22,22,24,27,29,33,40,
22,22,24,27,29,33,40,48,
22,22,24,28,32,38,47,57,
22,24,29,32,38,47,57,69

INTER8X8_LUMA =
13,14,15,16,17,17,18,19,
14,15,16,17,17,18,19,20,
15,16,17,17,18,19,20,21,
16,17,17,18,19,20,21,22,
17,17,18,19,21,22,22,23,
17,18,19,20,22,22,23,25,
18,19,20,22,22,23,25,26,
19,20,21,22,23,25,26,27when you want to use the q_matrix2 cqm with x264 you can do that via the following commandline:
--cqm4i 7,16,22,24,16,22,24,28,18,22,27,33,22,24,32,47 --cqm4p 13,15,17,18,15,17,18,20,17,18,21,22,18,20,22,25 --cqm8i 7,13,16,18,22,22,24,28,13,13,18,20,22,24,28,31,16,18,22,22,24,28,28,32,18,18,22,22,24,28,31,33,18,22,22,24,27,29,33,40,22,22,24,27,29,33,40,48,22,22,24,28,32,38,47,57,22,24,29,32,38,47,57,69 --cqm8p 13,14,15,16,17,17,18,19,14,15,16,17,17,18,19,20,15,16,17,17,18,19,20,21,16,17,17,18,19,20,21,22,17,17,18,19,21,22,22,23,17,18,19,20,22,22,23,25,18,19,20,22,22,23,25,26,19,20,21,22,23,25,26,27


4) flat 16
description: not really a custom matrix, as its the matrix thats used when no custom quantisation matrix is used, but still listed for completeness:

#flat 16

INTRA4X4_LUMA =
16,16,16,16,
16,16,16,16,
16,16,16,16,
16,16,16,16

INTRA4X4_CHROMAU =
16,16,16,16,
16,16,16,16,
16,16,16,16,
16,16,16,16

INTRA4X4_CHROMAV =
16,16,16,16,
16,16,16,16,
16,16,16,16,
16,16,16,16

INTER4X4_LUMA =
16,16,16,16,
16,16,16,16,
16,16,16,16,
16,16,16,16

INTER4X4_CHROMAU =
16,16,16,16,
16,16,16,16,
16,16,16,16,
16,16,16,16

INTER4X4_CHROMAV =
16,16,16,16,
16,16,16,16,
16,16,16,16,
16,16,16,16

INTRA8X8_LUMA =
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16

INTER8X8_LUMA =
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16to sum it up: iso/itu doesnt seem to think its worth the hassle to define different matrices for 4x4 luma and chromas

happy comparing! :)

attached the available cqm matrices coming with the avc reference encoder developed by mpeg plus soulhunters and sharktooths matrices:

[Soulhunter]

5) Soulhunters AVC v1

Description: My "Soulhunters v3 -> AVC" try... :P

#Soulhunters AVC v1

INTRA4X4_LUMA =
8,12,14,16,
12,14,16,18,
14,16,18,24,
16,18,24,32

INTRA4X4_CHROMAU =
8,12,14,16,
12,14,16,18,
14,16,18,24,
16,18,24,32

INTRA4X4_CHROMAV =
8,12,14,16,
12,14,16,18,
14,16,18,24,
16,18,24,32

INTER4X4_LUMA =
8,12,14,16,
12,14,16,18,
14,16,18,20,
16,18,20,24

INTER4X4_CHROMAU =
8,12,14,16,
12,14,16,18,
14,16,18,20,
16,18,20,24

INTER4X4_CHROMAV =
8,12,14,16,
12,14,16,18,
14,16,18,20,
16,18,20,24

INTRA8X8_LUMA =
8, 8,10,12,13,14,15,16,
8,10,12,13,14,15,16,18,
10,12,13,14,15,16,18,20,
12,13,14,15,16,18,20,22,
13,14,15,16,18,20,22,24,
14,15,16,18,20,22,24,26,
15,16,18,20,22,24,26,28,
16,18,20,22,24,26,28,32

INTER8X8_LUMA =
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16
Bye

[bond]

Soulhunter, it would be great if you can write your matrices exactly the same way as i did, so
- no spaces in between the values, only ","
- using "INTRA4X4_LUMA =", "INTER8X8_LUMA =" aso

cause thats the way the cqm files are written for the reference software.
i think it would be great if there could be a standardised way of writing these files and the reference software files are useable for that i think

[yaz]

@soulhunter
you're a big inventor ;) but would u, pls, drop some words about what would we expect from this matrix ? i don't think we have enough experience w/cqm in avc (at least i haven't, sorry for those who has)
thx
y

[Soulhunter]

@soulhunter
...what would we expect from this matrix ?
- It quantizes intra-frames stronger than inter-frames
- It drops high frequencys stronger than low ones
- It treats the luma and the chroma equally
- It wants to be tested by you... ^^


Bye

[Soulhunter]

6) Soulhunters AVC v2

- It quantizes intra-frames and inter-frames equally
- It drops a lot high frequencys in the 4X4 chroma
- It generally drops a lot high frequencys
- It is a HVS style attempt... ^^;

#Soulhunters AVC v2

INTRA4X4_LUMA =
8,12,14,16,
12,14,16,18,
14,16,18,24,
16,18,24,32

INTRA4X4_CHROMAU =
8,14,16,24,
14,16,24,32,
16,24,32,64,
24,32,64,128

INTRA4X4_CHROMAV =
8,14,16,24,
14,16,24,32,
16,24,32,64,
24,32,64,128

INTER4X4_LUMA =
8,12,14,16,
12,14,16,18,
14,16,18,24,
16,18,24,32

INTER4X4_CHROMAU =
8,14,16,24,
14,16,24,32,
16,24,32,64,
24,32,64,128

INTER4X4_CHROMAV =
8,14,16,24,
14,16,24,32,
16,24,32,64,
24,32,64,128

INTRA8X8_LUMA =
8, 9,10,12,13,14,15,16,
9,10,12,13,14,15,16,24,
10,12,13,14,15,16,24,32,
12,13,14,15,16,24,32,48,
13,14,15,16,24,32,48,56,
14,15,16,24,32,48,56,64,
15,16,24,32,48,56,64,96,
16,24,32,48,56,64,96,128

INTER8X8_LUMA =
8, 9,10,12,13,14,15,16,
9,10,12,13,14,15,16,24,
10,12,13,14,15,16,24,32,
12,13,14,15,16,24,32,48,
13,14,15,16,24,32,48,56,
14,15,16,24,32,48,56,64,
15,16,24,32,48,56,64,96,
16,24,32,48,56,64,96,128
Bye

[Sharktooth]

@soulhunter: remember ranges are 1-255

[Soulhunter]

@soulhunter: remember ranges are 1-255

Well, Im used to start with a 8 !!!

Sort of old habit... ^^;


Bye

[bond]

i compared soulhunters avc v1 and v2 with x264 with the following results on the matrix1 clip (1000 frames):

v1:
x264 [info]: slice I:9 Avg QP:18.67 Avg size: 20475 PSNR Mean Y:46.05 U:49.60 V:50.63 Avg:47.03 Global:46.98
x264 [info]: slice P:393 Avg QP:20.97 Avg size: 6324 PSNR Mean Y:44.13 U:48.18 V:49.22 Avg:45.18 Global:45.03
x264 [info]: slice B:598 Avg QP:22.68 Avg size: 1973 PSNR Mean Y:43.30 U:48.01 V:49.04 Avg:44.45 Global:44.36
x264 [info]: slice I Avg I4x4:52.5% I8x8:29.4% I16x16:18.1%
x264 [info]: slice P Avg I4x4:3.8% I8x8:6.8% I16x16:4.4% P:57.4% P8x8:7.9% PSKIP:19.7%
x264 [info]: slice B Avg I4x4:0.5% I8x8:0.9% I16x16:0.3% P:23.7% B:9.7% B8x8:2.1% DIRECT:3.7% BSKIP:59.1%
x264 [info]: 8x8 transform intra:44.7% inter:18.1%
x264 [info]: PSNR Mean Y:43.65 U:48.09 V:49.13 Avg:44.76 Global:44.63 kb/s:769.9

v2:
x264 [info]: slice I:8 Avg QP:18.88 Avg size: 19858 PSNR Mean Y:45.86 U:48.94 V:50.01 Avg:46.76 Global:46.72
x264 [info]: slice P:394 Avg QP:21.29 Avg size: 6348 PSNR Mean Y:44.19 U:47.84 V:48.95 Avg:45.19 Global:45.05
x264 [info]: slice B:598 Avg QP:22.92 Avg size: 1990 PSNR Mean Y:43.33 U:47.71 V:48.80 Avg:44.43 Global:44.36
x264 [info]: slice I Avg I4x4:53.7% I8x8:30.4% I16x16:15.9%
x264 [info]: slice P Avg I4x4:3.8% I8x8:6.3% I16x16:4.3% P:58.3% P8x8:7.4% PSKIP:20.0%
x264 [info]: slice B Avg I4x4:0.5% I8x8:0.8% I16x16:0.3% P:22.2% B:9.4% B8x8:1.9% DIRECT:3.0% BSKIP:61.8%
x264 [info]: 8x8 transform intra:43.5% inter:30.3%
x264 [info]: PSNR Mean Y:43.69 U:47.77 V:48.87 Avg:44.75 Global:44.63 kb/s:770.0
btw x264 now supports parsing jm-style matrices files. i have attached a zip containing the reference software matrices to the first post

[Sharktooth]

It will be hard to beat the "standard" quantization with a custom matrix...
It's like ASP h263 vs CQMs... h263 wins most of the times even if visual quality is inferior...

[Joe Fenton]

This thread should probably be a sticky.

[lexor]

This thread should probably be a sticky.
it shouldn't because the moment it becomes sticky it will gain too much of an "official" status which would lead many people to believe that matrices presented here are better than default. Which hasn't been shown by any reasonably involved visual comparison.

when a bunch of people do that and show visual difference thorough clips or screenshots then we can start a sticky with those "better" matrices.

P.S. After writing this I'm sort of ashemed of not participating in visual comparison they had in XVid forum for CQM's :(

[Soulhunter]

A CQM sticky in the XviD forum would be nice... XD


Bye

[Soulhunter]

@ bond

Btw, is it possible to add separate scaling for 8x8 chroma ???

Would help to make a real HVS style CQM... ^^


Bye

[Sharktooth]

No, there's no 8x8 chroma quantization in h.264.

[Soulhunter]

Just to keep this list up to date...

-> EQM AVC HR -- BETA -- (http://forum.doom9.org/showthread.php?t=96298)


Bye

[*.mp4 guy]

Don't Know how I missed this thread.

-Intra more aggresively compressed then inter
-Chroma roughly twice as aggressively compressed as luma
-High frequencies are dropped aggressively on intra
-High frequencies are dropped semi-aggressively on inter
-[Edit2] Removed spaces between numbers.

Mp4Guy's low Bitrate [edit1] V2
INTRA4X4_LUMA =
8,16,23,30,
16,25,34,40,
23,34,46,60,
30,40,60,84

INTRA4X4_CHROMAU =
16,20,36,46,
20,38,56,67,
36,56,77,102,
46,67,102,166

INTRA4X4_CHROMAV =
16,20,36,46,
20,38,56,67,
36,56,77,102,
46,67,102,166

INTER4X4_LUMA =
12,15,20,24,
15,22,27,32,
20,27,35,42,
24,32,42,55

INTER4X4_CHROMAU =
18,20,32,38,
20,36,44,56,
32,44,66,78,
38,56,78,100

INTER4X4_CHROMAV =
18,20,32,38,
20,36,44,56,
32,44,66,78,
38,56,78,100

INTRA8X8_LUMA =
8,13,16,18,23,28,29,32,
13,14,18,22,26,27,29,33,
16,18,23,27,28,29,32,35,
18,22,27,30,32,34,38,39,
23,26,28,32,37,42,45,47,
28,27,29,34,42,49,55,57,
29,29,32,38,45,55,63,69,
32,33,35,39,47,57,69,84

INTER8X8_LUMA =
8,12,14,15,16,17,18,20,
12,13,16,17,18,19,21,22,
14,16,18,19,20,22,24,25,
15,17,19,21,24,26,27,28,
16,18,20,24,28,29,30,32,
17,19,22,26,29,32,35,38,
18,21,24,27,30,35,42,46,
20,22,25,28,32,38,46,55

[redfordxx]

Well when there will be again increasing number of Qmatrices for AVC, I expect again confusion which one to choose.
Yeah, for low bitrate use low bitrate matrix for high bitrate high bitrate matrix.
But with bitrate comes compressibility and resolution, so it is not so plain.

How about measuring it with Quantizer? I mean to find some rules like
"the matrix performs best at average Q=20",
or
"If you reach 2nd pass with matrix abc_low_bitrate average Q=15, you should go for abc_mid_bitrate." (I am referring to the fact, that some of matrix producers made whole series for XviD)

[*.mp4 guy]

Some small changes from V3, reduced blocking and hopefully better detail preservation, also shortened the name. I have also changed the recommended Deblocking setting, its now -3:-1 for a sharp picture and 0:1 for a smooth picture.

Mp4Guy's LRM V4
INTRA4X4_LUMA =
8,22,32,47,
22,48,55,57,
32,55,63,69,
47,57,69,84

INTRA4X4_CHROMAU =
16,20,34,64,
20,40,62,78,
34,62,98,138,
64,78,138,168

INTRA4X4_CHROMAV =
16,20,34,64,
20,40,62,78,
34,62,98,138,
64,78,138,168

INTER4X4_LUMA =
12,28,28,32,
28,32,35,38,
28,35,42,46,
32,38,46,55

INTER4X4_CHROMAU =
18,24,34,41,
24,40,48,56,
34,48,64,92,
41,56,92,110

INTER4X4_CHROMAV =
18,24,34,41,
24,40,48,56,
34,48,64,92,
41,56,92,110

INTRA8X8_LUMA =
8,11,10,16,24,28,29,32,
12,13,14,19,26,27,29,33,
14,13,16,24,28,29,32,35,
14,17,22,30,32,34,38,39,
18,22,28,32,37,42,45,47,
24,27,29,34,42,49,55,57,
29,29,32,38,45,55,63,69,
32,33,35,39,47,57,69,84

INTER8X8_LUMA =
13,14,16,18,20,21,22,24,
14,15,18,19,20,21,22,25,
16,18,20,21,22,23,25,27,
18,19,21,23,25,26,29,31,
20,20,22,25,28,31,35,38,
21,21,23,26,31,37,42,48,
22,22,25,29,35,42,51,61,
24,25,27,31,38,48,61,76

mp4 guy: i am not sure if it makes a difference for the encoders, but cqm files dont use spaces between the values, plz remove them!

Well it doesn't make a difference, but I got rid of them anyway.

[bond]

mp4 guy: i am not sure if it makes a difference for the encoders, but cqm files dont use spaces between the values, plz remove them!

[Audionut]

Audionut's High Bitrate Matrix

INTRA4X4_LUMA =
5,7,11,17,
7,10,14,19,
10,13,16,23,
13,17,21,30

INTRA4X4_CHROMAU =
5,7,11,17,
7,10,14,19,
10,13,16,23,
13,17,21,30

INTRA4X4_CHROMAV =
5,7,11,17,
7,10,14,19,
10,13,16,23,
13,17,21,30

INTER4X4_LUMA =
5,7,10,11,
7,10,11,12,
10,11,12,13,
11,12,13,15

INTER4X4_CHROMAU =
5,7,10,11,
7,10,11,12,
10,11,12,13,
11,12,13,15

INTER4X4_CHROMAV =
5,7,10,11,
7,10,11,12,
10,11,12,13,
11,12,13,15

INTRA8X8_LUMA =
5,5,7,9,11,13,14,17,
5,6,8,11,13,14,17,18,
7,9,10,11,14,17,17,19,
9,10,13,13,14,17,18,22,
10,13,13,14,16,17,21,23,
13,13,14,16,17,20,22,26,
13,13,14,17,19,23,25,27,
13,14,17,19,21,24,25,30,

INTER8X8_LUMA =
5,5,7,9,10,10,11,11,
5,6,9,10,10,11,11,12,
7,9,10,10,11,11,12,12,
9,10,10,11,11,13,13,13,
10,10,11,11,12,13,13,13,
10,11,11,12,13,13,14,15,
11,11,12,13,13,14,13,15,
11,12,12,13,13,15,15,15,

Depending on your tastes, you'll need atleast 1000kbit/s.
That's approx the average bitrate I got with a clean source, that had a mix of fast and slow motion scenes at --crf 20.
Fast and fairly intense chapter 20 from star wars ep. 3, required 3200 kbit/s at --crf 20.

edit: Almost forgot, the source was resized to (576,240)

[bond]

why would someone want to make a high bitrate encode with 576x240?

[alea35]

can someone explain to me how to make these matrices work? i usually just use a batch file to encode and was wondering how i could incorporate these into my batch file?

[Revgen]

Use --cqmfile "yourdrive:/yourcqmmatrix.cfg"

Make sure to include the quotation marks.

[Audionut]

why would someone want to make a high bitrate encode with 576x240?

Why Indeed?

The numbers quoted, was from a test I was running to compare the bitrate produced against the other AVC matrices.
I was taking a short cut with the resolution, for faster coding/comparing.

[Audionut]

Audionut's general matrix.

INTRA4X4_LUMA =
8,11,13,19,
11,16,22,31,
13,22,39,60,
19,31,60,110

INTRA4X4_CHROMAU =
8,11,13,19,
11,16,22,31,
13,22,39,60,
19,31,60,110

INTRA4X4_CHROMAV =
8,11,13,19,
11,16,22,31,
13,22,39,60,
19,31,60,110

INTER4X4_LUMA =
14,14,17,26,
14,21,30,44,
17,30,46,87,
26,44,87,162

INTER4X4_CHROMAU =
14,14,17,26,
14,21,30,44,
17,30,46,87,
26,44,87,162

INTER4X4_CHROMAV =
14,14,17,26,
14,21,30,44,
17,30,46,87,
26,44,87,162

INTRA8X8_LUMA =
8,11,11,11,12,13,16,19,
11,11,11,11,12,14,17,20,
11,11,12,13,15,17,20,24,
11,11,13,16,19,22,26,31,
12,12,15,19,25,30,36,42,
13,14,17,22,30,39,49,60,
16,17,20,26,36,49,65,83,
19,20,24,31,42,60,83,110

INTER8X8_LUMA =
14,14,14,14,15,17,21,26,
14,14,14,14,15,18,23,27,
14,14,15,17,20,23,27,33,
14,14,17,21,26,30,36,44,
15,15,20,26,35,42,51,60,
17,18,23,30,42,46,58,87,
21,23,27,36,51,58,78,122,
26,27,33,44,60,87,122,162


I would class this a low to medium bitrate matrix. Tested against Sharktooth's excellent EQM_AVC matrix, it produces better SSIM at bitrates around 700 kbit/s.

[Revgen]

I would class this a low to medium bitrate matrix. Tested against Sharktooth's excellent EQM_AVC matrix, it produces better SSIM at bitrates around 700 kbit/s.

IIRC, Sharktooth's matrix was made for high bitrates.

[Sharktooth]

i confirm what revgen said. 700kbps is too low for EQM AVC-HR (unless the source is supercompressible o.r the resolution is lowered)

[leowai]

The usage of the EQM AVC-HR cqm is mentioned in the download page (2005-08-01).
http://forum.doom9.org/showpost.php?p=677092&postcount=1
EQM AVC-HR rev.1
This matrix was made for medium-high bitrate backups with High-Profile AVC encoders that support custom matrices and 8x8 discrete cosine transform. It also reduces blocking.

Usage:
- high quality backups of DVD material at full resolution (anamorphic) and bitrates of at least 1300-1400kbps (for average motion and not noisy/old movies).
- non anamorphic/vertically resized encodes at bitrates of at least 1000kbps.

Standard version (JM, x264 & Encavc beta2-2 compatible)

Probably cqm users need more information when using the cqm. I think it's encouraged that the author could follow what sharktooth did. Just provide more detail information & recommandations about the usage of their cqm on the download page, i.e.:
- Movie resolution (640x480 up to 720x480)
- Interlaced* or Progressive video?
*Sorry, i'm not familiar to cqm, is there any cqm for interlaced video? :confused: (where is the "I'm with stupid" icon? I can't find it!!)
- Recommended bitrates (1000kbps)
- Recommended movie type (Anime, TV & etc)

[Oline 61]

What matrix should I use for high bitrate (3905 kbit/s) backups of live action DVD movies? Should I just use JVT?

[Oline 61]

bump?
Should I try without any matrix?

[kotrtim]

INTRA4X4_LUMA =
10,12,14,16,
12,14,16,18,
14,16,18,20,
16,18,20,20

INTRA4X4_CHROMAU =
13,14,15,16,
14,15,16,18,
15,16,18,20,
16,18,20,20

INTRA4X4_CHROMAV =
13,14,15,16,
14,15,16,18,
15,16,18,20,
16,18,20,20

INTER4X4_LUMA =
10,12,14,16,
12,14,16,18,
14,16,18,20,
16,18,20,20

INTER4X4_CHROMAU =
13,14,15,16,
14,15,16,18,
15,16,18,20,
16,18,20,20

INTER4X4_CHROMAV =
13,14,15,16,
14,15,16,18,
15,16,18,20,
16,18,20,20

INTRA8X8_LUMA =
10,10,11,12,12,13,14,15,
10,11,12,13,13,15,15,16,
11,12,12,14,15,15,16,17,
12,13,14,15,15,16,17,18,
12,13,15,15,16,17,18,19,
13,15,15,16,17,18,19,19,
14,15,16,17,18,19,19,20,
15,16,17,18,19,19,20,20

INTER8X8_LUMA =
10,10,11,12,12,13,14,15,
10,11,12,13,14,14,15,16,
11,12,12,14,14,15,16,17,
12,13,14,14,15,16,17,18,
12,14,14,15,16,17,18,19,
13,14,15,16,17,18,19,20,
14,15,16,17,18,19,20,20,
15,16,17,18,19,20,20,20

try this?
No matter how much you tweak the default flat16, it is always a drop in psnr...
This is just simply cut and paste from xvid's SixOfNine, merging with soulhunter_v1's 4x4s .....decresing all the values so that no. 20 is the
max...

soulhunter's v2 has some problems, it tends to so some ugly red spot on a clean blue sky with crf 26, well, i tried this sixofnine port, it does solve this problems... still need testing

[Soulhunter]

Red spot on a clean blue sky?


Bye

[kotrtim]

http://img111.imageshack.us/img111/3718/ori7vs.png
http://img209.imageshack.us/img209/1498/soulhunter5me.png
http://img350.imageshack.us/img350/680/eqmavchr1qv.png
http://img480.imageshack.us/img480/729/jvt5tx.png
http://img117.imageshack.us/img117/7871/soulsix6dd.png

picture order
1. original
2. soulhunter v2
3. eqm_avc_hr
4. jvt
5. soulhunter/sixofnine merge

look carefully, the top left corner (North East)
I'm using --nf

besides this, soulhunter v2 will produce errors like one big red area on a clean blue sky, tried eqm, no such errors, i will try to reproduce.

EDIT: sorry, checked eqm, same error, even worst 2 spot, one clear, one faint...jvt does not have such errors, which is very annoying

EDIT2 : Tested with more different settings, the above encodes are all
default settings (except crf 26 , cqm, nf).. I tried subme 6 and it help to make the spot less noticible and also shifted the spot to another location, subme 6 + no fast pskip solved all the problems, it's more like a x264 bug, not a cqm problem, but of course if default settings is used with default matrix, no such problems

So for those who use cqm, subme 6 + no fast pskip is a must, even better if trellis 2 is used

[Oline 61]

Thanks for the matrix, kotrtim. I'll try it out tonight.

[Oline 61]

That matrix looks great. Better than JVT at that bitrate. I think no matrix may still look better at that bitrate.

[Sharktooth]

look at how EQM AVC-HR reduces blockiness in the sky/clouds... while soulhunter/6of9 merge has the worts looking sky/clouds
i will check what causes that red/brown dot though...

[kotrtim]

That matrix looks great. Better than JVT at that bitrate. I think no matrix may still look better at that bitrate.

Yes, the effortless, all 16 seems to be the best, shartooth's said that his CQM is for high bitrate, but I think I could say that his matrix can really reduce big nasty blocks that flat16 produce, especially skies, clouds, smokes... consider using his matrix at --crf 26 ...... or if you get a rate factor of < 28 for 1st pass

when you use crf above 30 ~ 40, the best is still flat 16, tried mp4guy, smooth images too much, flat16 is pretty balance..serves as general purpose, good for every bitrate, it's not same as ASP H.263.

[Oline 61]

The problem is my movie, Friday Night Lights has a desert scene near the beginning, and the brownish desert sand has nasty reddish artifacts on it with EQM-AVC-HR. I will try to get some screenshots up in a minute or two.

EDIT: Actually I can't seem to reproduce this problem. Maybe trying eqm_avc_hr.cfg again would be a good idea.

At my current bitrate and resolution (3955kbit/s 640x480) I don't think I have to worry too much about blocking.

[kotrtim]

Description:
1. It should look more or less like default;
2. with slightly reduced blocks but not more;
3. and reduced ringing artefacts.
4. Therefore, it can serve as all purpose matrix.

since leowai said a matrix should come with desciption, oline 61, do you find this description true with the sample you tested, because this conclusion is made from only one test sample, doing visual comparison is very eye straining and takes lots of time... thanks...

How about calling it "soulhunter_6of9"?:)

Audionut, tried your Hi bitrate matrix too, wow at the same --crf, it needs more than 1.5X the space of all other matrices
Your general matrix seems to treat the cloud well too, I will capture a screenshot

[Oline 61]

I agree with that description 100% kotrtim. I will look at a few more clips with it and report back. I'll also try out those Audionut matrixes, they sound nice.

[Oline 61]

BTW, here is an archive with all the matrices listed in this thread. If you don't want your matrix in this archive, tell me so and I will remove it.
http://home.nc.rr.com/oline61/x264_custom_matrices.7z

I am currently planning to generate several clips designed to test for specific behaviors, and then encode with different quant matrices, and test PSNR and SSIM. What settings should I use to do this?

I don't think --qp or --crf will work because they produce different sizes with different matrices which is no good for SSIM and PSNR testing. Should I do 2-pass or 1-pass ABR? For 2-pass do I need to do a new first pass for each matrix? Should I use slow high quality settings, or faster settings? What bitrates would you like to see represented?

[Kostarum Rex Persia]

Sorry for asking that, but I lost my collection of custom matrices. So, I don't know what matrices are used for ultra-low and low bitrates.

Please, tell me.

[Oline 61]

For low bitrates MP4Guy's Low Bitrate v2 and v3.
They are in the archive I posted and in this thread.
Look at the numbers. Bigger numbers = more compressibility. Generally......

[kotrtim]

For 2-pass do I need to do a new first pass for each matrix?
:( Yes, akupenguin said we should, but if it is 3 pass, 1st pass can be anything, the 2nd and 3rd has to be the same.... it's very time consuming

Should I use slow high quality settings, or faster settings?
The red/brown spot problem can be avoided if you use subme 6 + no fast pskip....but I don't think the "spot" phenomena is gonna happen so often...

[Marsu42]

BTW, here is an archive with all the matrices listed in this thread.

Thanks a lot for your work gathering the matrices. Since the archive seems to be intended to save the time to search all relevant posts, it would be great if you could also append the target bitrate to the matrices' names in the future, eg. audionut_general.cfg would be something like audionut_general_700-1000.cfg or ..._all. the same suggestion goes for the original developers as well.

I am asking for this because in many posts the bitrate is often not declared at all or only as eg. 'low', which imho can anything from 100 to 1000 or even above. Furthermore, one wondersDue to this, it's quite difficult to try them out, which is not the case with names like "Jawor's 1CD Matrix".

[Oline 61]

Marsu42, I plan to test these matrices at different bitrates and determine their PSNR and SSIM. By objectively determining the best matrices for each bitrate I plan to determine which matrices perform well at which bitrate.

Here are the results for 4000kbit/s. I only included matrices designed for high bitrates. I may add more matrices at this bitrate later. These are just preliminary results, and the commandline settings may be refined later.

*EDIT: Added clip info.*
The clip is from Friday Night Lights. I am using the script that I set up for a 1 DVD5 backup of the movie.
MPEG2Source("VTS_01_1.d2v")
Telecide(order=1,guide=1,post=0)
Decimate()
ColorMatrix()
LanczosResize(640,480,taps=9)
BlockBuster(method="noise",detail_max=20)
Trim(1030,169045)
Trim(0,1000)
The clip consists of 1000 frames, starting with some text, moving into a shot of clouds rolling across the sky, then various shots of the desert, including low movement stills and pans across the landscape.


#pass 1
x264 --pass 1 --stats stats.log -B 4000 --subme 1 --me dia --analyse none --cqmfile matrix.cfg --progress --fps 23.976 -o output.264 input.yuv 640x480
#pass 2
x264 --pass 2 --stats stats.log -B 4000 --subme 3 --me hex --analyse all --8x8dct --cqmfile matrix.cfg --progress --fps 23.976 -o output.264 input.yuv 640x480


Here are the results:

audionut_hr.cfg
Average SSIM= 80.18
PSNR Mean Y:44.478 U:56.815 V:58.696 Avg:45.883 Global:45.565 kb/s:4021.56

eqm_avc_hr.cfg
Average SSIM= 80.41
PSNR Mean Y:44.415 U:56.814 V:58.672 Avg:45.826 Global:45.480 kb/s:4000.45

flat
Average SSIM= 80.83
PSNR Mean Y:44.941 U:56.329 V:58.114 Avg:46.257 Global:45.925 kb/s:4022.82

jvt
Average SSIM= 80.76
PSNR Mean Y:44.581 U:56.905 V:58.775 Avg:45.985 Global:45.685 kb/s:4007.02

soulhunter_6of9.cfg
Average SSIM= 81.11
PSNR Mean Y:44.875 U:56.324 V:58.119 Avg:46.197 Global:45.877 kb/s:4010.98

[IgorC]

Seems like for the first time cm (Soulhunter_6of9) has higher SSIM result and only slightly inferior OPSNR. Good.
Who already plaid with cm on 1200-1500 kbit/s ( movie source, not anime) for best SSIM result?

[IgorC]

...So for those who use cqm, subme 6 + no fast pskip is a must, even better if trellis 2 is used

If I'm not mistaking x264 doesn't use fast skip when cm is enable.

[Audionut]

@Online 61. IMO, you should be using B-frames.