collection of available AVC custom quant matrices

[bond]

1) jvt
description: the jvt matrix, used as the default custom matrix by the avc reference software (comparable to the "mpeg" matrix of mpeg-4 asp), uses the same values in all intra4x4 matrices and the same values in all inter4x4 matrices:

#jvt

INTRA4X4_LUMA =
6,13,20,28,
13,20,28,32,
20,28,32,37,
28,32,37,42

INTRA4X4_CHROMAU =
6,13,20,28,
13,20,28,32,
20,28,32,37,
28,32,37,42

INTRA4X4_CHROMAV =
6,13,20,28,
13,20,28,32,
20,28,32,37,
28,32,37,42

INTER4X4_LUMA =
10,14,20,24,
14,20,24,27,
20,24,27,30,
24,27,30,34

INTER4X4_CHROMAU =
10,14,20,24,
14,20,24,27,
20,24,27,30,
24,27,30,34

INTER4X4_CHROMAV =
10,14,20,24,
14,20,24,27,
20,24,27,30,
24,27,30,34

INTRA8X8_LUMA =
6,10,13,16,18,23,25,27,
10,11,16,18,23,25,27,29,
13,16,18,23,25,27,29,31,
16,18,23,25,27,29,31,33,
18,23,25,27,29,31,33,36,
23,25,27,29,31,33,36,38,
25,27,29,31,33,36,38,40,
27,29,31,33,36,38,40,42

INTER8X8_LUMA =
9,13,15,17,19,21,22,24,
13,13,17,19,21,22,24,25,
15,17,19,21,22,24,25,27,
17,19,21,22,24,25,27,28,
19,21,22,24,25,27,28,30,
21,22,24,25,27,28,30,32,
22,24,25,27,28,30,32,33,
24,25,27,28,30,32,33,35when you want to use the jvt cqm with x264 you can do that via the following commandline:
--cqm jvt


2) q_matrix
description: the "q_matrix", coming with the avc reference software, uses the same values in all intra4x4 matrices and the same values in all inter4x4 matrices:

#q_matrix

INTRA4X4_LUMA =
6,12,19,26,
12,19,26,31,
19,26,31,35,
26,31,35,39

INTRA4X4_CHROMAU =
6,12,19,26,
12,19,26,31,
19,26,31,35,
26,31,35,39

INTRA4X4_CHROMAV =
6,12,19,26,
12,19,26,31,
19,26,31,35,
26,31,35,39

INTER4X4_LUMA =
9,13,18,21,
13,18,21,24,
18,21,24,27,
21,24,27,30

INTER4X4_CHROMAU =
9,13,18,21,
13,18,21,24,
18,21,24,27,
21,24,27,30

INTER4X4_CHROMAV =
9,13,18,21,
13,18,21,24,
18,21,24,27,
21,24,27,30

INTRA8X8_LUMA =
6,10,13,16,19,24,26,28,
10,12,16,19,24,26,28,31,
13,16,19,24,26,28,31,33,
16,19,24,26,28,31,33,35,
19,24,26,28,31,33,35,37,
24,26,28,31,33,35,37,39,
26,28,31,33,35,37,39,42,
28,31,33,35,37,39,42,44

INTER8X8_LUMA =
9,12,14,16,18,19,21,22,
12,13,16,18,19,21,22,24,
14,16,18,19,21,22,24,25,
16,18,19,21,22,24,25,27,
18,19,21,22,24,25,27,28,
19,21,22,24,25,27,28,30,
21,22,24,25,27,28,30,31,
22,24,25,27,28,30,31,33when you want to use the q_matrix cqm with x264 you can do that via the following commandline:
--cqm4i 6,12,19,26,12,19,26,31,19,26,31,35,26,31,35,39 --cqm4p 9,13,18,21,13,18,21,24,18,21,24,27,21,24,27,30 --cqm8i 6,10,13,16,19,24,26,28,10,12,16,19,24,26,28,31,13,16,19,24,26,28,31,33,16,19,24,26,28,31,33,35,19,24,26,28,31,33,35,37,24,26,28,31,33,35,37,39,26,28,31,33,35,37,39,42,28,31,33,35,37,39,42,44 --cqm8p 9,12,14,16,18,19,21,22,12,13,16,18,19,21,22,24,14,16,18,19,21,22,24,25,16,18,19,21,22,24,25,27,18,19,21,22,24,25,27,28,19,21,22,24,25,27,28,30,21,22,24,25,27,28,30,31,22,24,25,27,28,30,31,33


3) q_matrix2
description: the "q_matrix2", coming with the avc reference software, uses the same values in all intra4x4 matrices and the same values in all inter4x4 matrices:

#q_matrix2

INTRA4X4_LUMA =
7,16,22,24,
16,22,24,28,
18,22,27,33,
22,24,32,47

INTRA4X4_CHROMAU =
7,16,22,24,
16,22,24,28,
18,22,27,33,
22,24,32,47

INTRA4X4_CHROMAV =
7,16,22,24,
16,22,24,28,
18,22,27,33,
22,24,32,47

INTER4X4_LUMA =
13,15,17,18,
15,17,18,20,
17,18,21,22,
18,20,22,25

INTER4X4_CHROMAU =
13,15,17,18,
15,17,18,20,
17,18,21,22,
18,20,22,25

INTER4X4_CHROMAV =
13,15,17,18,
15,17,18,20,
17,18,21,22,
18,20,22,25

INTRA8X8_LUMA =
7,13,16,18,22,22,24,28,
13,13,18,20,22,24,28,31,
16,18,22,22,24,28,28,32,
18,18,22,22,24,28,31,33,
18,22,22,24,27,29,33,40,
22,22,24,27,29,33,40,48,
22,22,24,28,32,38,47,57,
22,24,29,32,38,47,57,69

INTER8X8_LUMA =
13,14,15,16,17,17,18,19,
14,15,16,17,17,18,19,20,
15,16,17,17,18,19,20,21,
16,17,17,18,19,20,21,22,
17,17,18,19,21,22,22,23,
17,18,19,20,22,22,23,25,
18,19,20,22,22,23,25,26,
19,20,21,22,23,25,26,27when you want to use the q_matrix2 cqm with x264 you can do that via the following commandline:
--cqm4i 7,16,22,24,16,22,24,28,18,22,27,33,22,24,32,47 --cqm4p 13,15,17,18,15,17,18,20,17,18,21,22,18,20,22,25 --cqm8i 7,13,16,18,22,22,24,28,13,13,18,20,22,24,28,31,16,18,22,22,24,28,28,32,18,18,22,22,24,28,31,33,18,22,22,24,27,29,33,40,22,22,24,27,29,33,40,48,22,22,24,28,32,38,47,57,22,24,29,32,38,47,57,69 --cqm8p 13,14,15,16,17,17,18,19,14,15,16,17,17,18,19,20,15,16,17,17,18,19,20,21,16,17,17,18,19,20,21,22,17,17,18,19,21,22,22,23,17,18,19,20,22,22,23,25,18,19,20,22,22,23,25,26,19,20,21,22,23,25,26,27


4) flat 16
description: not really a custom matrix, as its the matrix thats used when no custom quantisation matrix is used, but still listed for completeness:

#flat 16

INTRA4X4_LUMA =
16,16,16,16,
16,16,16,16,
16,16,16,16,
16,16,16,16

INTRA4X4_CHROMAU =
16,16,16,16,
16,16,16,16,
16,16,16,16,
16,16,16,16

INTRA4X4_CHROMAV =
16,16,16,16,
16,16,16,16,
16,16,16,16,
16,16,16,16

INTER4X4_LUMA =
16,16,16,16,
16,16,16,16,
16,16,16,16,
16,16,16,16

INTER4X4_CHROMAU =
16,16,16,16,
16,16,16,16,
16,16,16,16,
16,16,16,16

INTER4X4_CHROMAV =
16,16,16,16,
16,16,16,16,
16,16,16,16,
16,16,16,16

INTRA8X8_LUMA =
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16

INTER8X8_LUMA =
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16to sum it up: iso/itu doesnt seem to think its worth the hassle to define different matrices for 4x4 luma and chromas

happy comparing! :)

attached the available cqm matrices coming with the avc reference encoder developed by mpeg plus soulhunters and sharktooths matrices:

[Soulhunter]

5) Soulhunters AVC v1

Description: My "Soulhunters v3 -> AVC" try... :P

#Soulhunters AVC v1

INTRA4X4_LUMA =
8,12,14,16,
12,14,16,18,
14,16,18,24,
16,18,24,32

INTRA4X4_CHROMAU =
8,12,14,16,
12,14,16,18,
14,16,18,24,
16,18,24,32

INTRA4X4_CHROMAV =
8,12,14,16,
12,14,16,18,
14,16,18,24,
16,18,24,32

INTER4X4_LUMA =
8,12,14,16,
12,14,16,18,
14,16,18,20,
16,18,20,24

INTER4X4_CHROMAU =
8,12,14,16,
12,14,16,18,
14,16,18,20,
16,18,20,24

INTER4X4_CHROMAV =
8,12,14,16,
12,14,16,18,
14,16,18,20,
16,18,20,24

INTRA8X8_LUMA =
8, 8,10,12,13,14,15,16,
8,10,12,13,14,15,16,18,
10,12,13,14,15,16,18,20,
12,13,14,15,16,18,20,22,
13,14,15,16,18,20,22,24,
14,15,16,18,20,22,24,26,
15,16,18,20,22,24,26,28,
16,18,20,22,24,26,28,32

INTER8X8_LUMA =
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,16
Bye

[bond]

Soulhunter, it would be great if you can write your matrices exactly the same way as i did, so
- no spaces in between the values, only ","
- using "INTRA4X4_LUMA =", "INTER8X8_LUMA =" aso

cause thats the way the cqm files are written for the reference software.
i think it would be great if there could be a standardised way of writing these files and the reference software files are useable for that i think

[yaz]

@soulhunter
you're a big inventor ;) but would u, pls, drop some words about what would we expect from this matrix ? i don't think we have enough experience w/cqm in avc (at least i haven't, sorry for those who has)
thx
y

[Soulhunter]

@soulhunter
...what would we expect from this matrix ?
- It quantizes intra-frames stronger than inter-frames
- It drops high frequencys stronger than low ones
- It treats the luma and the chroma equally
- It wants to be tested by you... ^^


Bye

[Soulhunter]

6) Soulhunters AVC v2

- It quantizes intra-frames and inter-frames equally
- It drops a lot high frequencys in the 4X4 chroma
- It generally drops a lot high frequencys
- It is a HVS style attempt... ^^;

#Soulhunters AVC v2

INTRA4X4_LUMA =
8,12,14,16,
12,14,16,18,
14,16,18,24,
16,18,24,32

INTRA4X4_CHROMAU =
8,14,16,24,
14,16,24,32,
16,24,32,64,
24,32,64,128

INTRA4X4_CHROMAV =
8,14,16,24,
14,16,24,32,
16,24,32,64,
24,32,64,128

INTER4X4_LUMA =
8,12,14,16,
12,14,16,18,
14,16,18,24,
16,18,24,32

INTER4X4_CHROMAU =
8,14,16,24,
14,16,24,32,
16,24,32,64,
24,32,64,128

INTER4X4_CHROMAV =
8,14,16,24,
14,16,24,32,
16,24,32,64,
24,32,64,128

INTRA8X8_LUMA =
8, 9,10,12,13,14,15,16,
9,10,12,13,14,15,16,24,
10,12,13,14,15,16,24,32,
12,13,14,15,16,24,32,48,
13,14,15,16,24,32,48,56,
14,15,16,24,32,48,56,64,
15,16,24,32,48,56,64,96,
16,24,32,48,56,64,96,128

INTER8X8_LUMA =
8, 9,10,12,13,14,15,16,
9,10,12,13,14,15,16,24,
10,12,13,14,15,16,24,32,
12,13,14,15,16,24,32,48,
13,14,15,16,24,32,48,56,
14,15,16,24,32,48,56,64,
15,16,24,32,48,56,64,96,
16,24,32,48,56,64,96,128
Bye

[Sharktooth]

@soulhunter: remember ranges are 1-255

[Soulhunter]

@soulhunter: remember ranges are 1-255

Well, Im used to start with a 8 !!!

Sort of old habit... ^^;


Bye

[bond]

i compared soulhunters avc v1 and v2 with x264 with the following results on the matrix1 clip (1000 frames):

v1:
x264 [info]: slice I:9 Avg QP:18.67 Avg size: 20475 PSNR Mean Y:46.05 U:49.60 V:50.63 Avg:47.03 Global:46.98
x264 [info]: slice P:393 Avg QP:20.97 Avg size: 6324 PSNR Mean Y:44.13 U:48.18 V:49.22 Avg:45.18 Global:45.03
x264 [info]: slice B:598 Avg QP:22.68 Avg size: 1973 PSNR Mean Y:43.30 U:48.01 V:49.04 Avg:44.45 Global:44.36
x264 [info]: slice I Avg I4x4:52.5% I8x8:29.4% I16x16:18.1%
x264 [info]: slice P Avg I4x4:3.8% I8x8:6.8% I16x16:4.4% P:57.4% P8x8:7.9% PSKIP:19.7%
x264 [info]: slice B Avg I4x4:0.5% I8x8:0.9% I16x16:0.3% P:23.7% B:9.7% B8x8:2.1% DIRECT:3.7% BSKIP:59.1%
x264 [info]: 8x8 transform intra:44.7% inter:18.1%
x264 [info]: PSNR Mean Y:43.65 U:48.09 V:49.13 Avg:44.76 Global:44.63 kb/s:769.9

v2:
x264 [info]: slice I:8 Avg QP:18.88 Avg size: 19858 PSNR Mean Y:45.86 U:48.94 V:50.01 Avg:46.76 Global:46.72
x264 [info]: slice P:394 Avg QP:21.29 Avg size: 6348 PSNR Mean Y:44.19 U:47.84 V:48.95 Avg:45.19 Global:45.05
x264 [info]: slice B:598 Avg QP:22.92 Avg size: 1990 PSNR Mean Y:43.33 U:47.71 V:48.80 Avg:44.43 Global:44.36
x264 [info]: slice I Avg I4x4:53.7% I8x8:30.4% I16x16:15.9%
x264 [info]: slice P Avg I4x4:3.8% I8x8:6.3% I16x16:4.3% P:58.3% P8x8:7.4% PSKIP:20.0%
x264 [info]: slice B Avg I4x4:0.5% I8x8:0.8% I16x16:0.3% P:22.2% B:9.4% B8x8:1.9% DIRECT:3.0% BSKIP:61.8%
x264 [info]: 8x8 transform intra:43.5% inter:30.3%
x264 [info]: PSNR Mean Y:43.69 U:47.77 V:48.87 Avg:44.75 Global:44.63 kb/s:770.0
btw x264 now supports parsing jm-style matrices files. i have attached a zip containing the reference software matrices to the first post

[Sharktooth]

It will be hard to beat the "standard" quantization with a custom matrix...
It's like ASP h263 vs CQMs... h263 wins most of the times even if visual quality is inferior...

[Joe Fenton]

This thread should probably be a sticky.

[lexor]

This thread should probably be a sticky.
it shouldn't because the moment it becomes sticky it will gain too much of an "official" status which would lead many people to believe that matrices presented here are better than default. Which hasn't been shown by any reasonably involved visual comparison.

when a bunch of people do that and show visual difference thorough clips or screenshots then we can start a sticky with those "better" matrices.

P.S. After writing this I'm sort of ashemed of not participating in visual comparison they had in XVid forum for CQM's :(

[Soulhunter]

A CQM sticky in the XviD forum would be nice... XD


Bye

[Soulhunter]

@ bond

Btw, is it possible to add separate scaling for 8x8 chroma ???

Would help to make a real HVS style CQM... ^^


Bye

[Sharktooth]

No, there's no 8x8 chroma quantization in h.264.

[Soulhunter]

Just to keep this list up to date...

-> EQM AVC HR -- BETA -- (http://forum.doom9.org/showthread.php?t=96298)


Bye

[*.mp4 guy]

Don't Know how I missed this thread.

-Intra more aggresively compressed then inter
-Chroma roughly twice as aggressively compressed as luma
-High frequencies are dropped aggressively on intra
-High frequencies are dropped semi-aggressively on inter
-[Edit2] Removed spaces between numbers.

Mp4Guy's low Bitrate [edit1] V2
INTRA4X4_LUMA =
8,16,23,30,
16,25,34,40,
23,34,46,60,
30,40,60,84

INTRA4X4_CHROMAU =
16,20,36,46,
20,38,56,67,
36,56,77,102,
46,67,102,166

INTRA4X4_CHROMAV =
16,20,36,46,
20,38,56,67,
36,56,77,102,
46,67,102,166

INTER4X4_LUMA =
12,15,20,24,
15,22,27,32,
20,27,35,42,
24,32,42,55

INTER4X4_CHROMAU =
18,20,32,38,
20,36,44,56,
32,44,66,78,
38,56,78,100

INTER4X4_CHROMAV =
18,20,32,38,
20,36,44,56,
32,44,66,78,
38,56,78,100

INTRA8X8_LUMA =
8,13,16,18,23,28,29,32,
13,14,18,22,26,27,29,33,
16,18,23,27,28,29,32,35,
18,22,27,30,32,34,38,39,
23,26,28,32,37,42,45,47,
28,27,29,34,42,49,55,57,
29,29,32,38,45,55,63,69,
32,33,35,39,47,57,69,84

INTER8X8_LUMA =
8,12,14,15,16,17,18,20,
12,13,16,17,18,19,21,22,
14,16,18,19,20,22,24,25,
15,17,19,21,24,26,27,28,
16,18,20,24,28,29,30,32,
17,19,22,26,29,32,35,38,
18,21,24,27,30,35,42,46,
20,22,25,28,32,38,46,55

[redfordxx]

Well when there will be again increasing number of Qmatrices for AVC, I expect again confusion which one to choose.
Yeah, for low bitrate use low bitrate matrix for high bitrate high bitrate matrix.
But with bitrate comes compressibility and resolution, so it is not so plain.

How about measuring it with Quantizer? I mean to find some rules like
"the matrix performs best at average Q=20",
or
"If you reach 2nd pass with matrix abc_low_bitrate average Q=15, you should go for abc_mid_bitrate." (I am referring to the fact, that some of matrix producers made whole series for XviD)

[*.mp4 guy]

Some small changes from V3, reduced blocking and hopefully better detail preservation, also shortened the name. I have also changed the recommended Deblocking setting, its now -3:-1 for a sharp picture and 0:1 for a smooth picture.

Mp4Guy's LRM V4
INTRA4X4_LUMA =
8,22,32,47,
22,48,55,57,
32,55,63,69,
47,57,69,84

INTRA4X4_CHROMAU =
16,20,34,64,
20,40,62,78,
34,62,98,138,
64,78,138,168

INTRA4X4_CHROMAV =
16,20,34,64,
20,40,62,78,
34,62,98,138,
64,78,138,168

INTER4X4_LUMA =
12,28,28,32,
28,32,35,38,
28,35,42,46,
32,38,46,55

INTER4X4_CHROMAU =
18,24,34,41,
24,40,48,56,
34,48,64,92,
41,56,92,110

INTER4X4_CHROMAV =
18,24,34,41,
24,40,48,56,
34,48,64,92,
41,56,92,110

INTRA8X8_LUMA =
8,11,10,16,24,28,29,32,
12,13,14,19,26,27,29,33,
14,13,16,24,28,29,32,35,
14,17,22,30,32,34,38,39,
18,22,28,32,37,42,45,47,
24,27,29,34,42,49,55,57,
29,29,32,38,45,55,63,69,
32,33,35,39,47,57,69,84

INTER8X8_LUMA =
13,14,16,18,20,21,22,24,
14,15,18,19,20,21,22,25,
16,18,20,21,22,23,25,27,
18,19,21,23,25,26,29,31,
20,20,22,25,28,31,35,38,
21,21,23,26,31,37,42,48,
22,22,25,29,35,42,51,61,
24,25,27,31,38,48,61,76

mp4 guy: i am not sure if it makes a difference for the encoders, but cqm files dont use spaces between the values, plz remove them!

Well it doesn't make a difference, but I got rid of them anyway.

[bond]

mp4 guy: i am not sure if it makes a difference for the encoders, but cqm files dont use spaces between the values, plz remove them!

[Audionut]

Audionut's High Bitrate Matrix

INTRA4X4_LUMA =
5,7,11,17,
7,10,14,19,
10,13,16,23,
13,17,21,30

INTRA4X4_CHROMAU =
5,7,11,17,
7,10,14,19,
10,13,16,23,
13,17,21,30

INTRA4X4_CHROMAV =
5,7,11,17,
7,10,14,19,
10,13,16,23,
13,17,21,30

INTER4X4_LUMA =
5,7,10,11,
7,10,11,12,
10,11,12,13,
11,12,13,15

INTER4X4_CHROMAU =
5,7,10,11,
7,10,11,12,
10,11,12,13,
11,12,13,15

INTER4X4_CHROMAV =
5,7,10,11,
7,10,11,12,
10,11,12,13,
11,12,13,15

INTRA8X8_LUMA =
5,5,7,9,11,13,14,17,
5,6,8,11,13,14,17,18,
7,9,10,11,14,17,17,19,
9,10,13,13,14,17,18,22,
10,13,13,14,16,17,21,23,
13,13,14,16,17,20,22,26,
13,13,14,17,19,23,25,27,
13,14,17,19,21,24,25,30,

INTER8X8_LUMA =
5,5,7,9,10,10,11,11,
5,6,9,10,10,11,11,12,
7,9,10,10,11,11,12,12,
9,10,10,11,11,13,13,13,
10,10,11,11,12,13,13,13,
10,11,11,12,13,13,14,15,
11,11,12,13,13,14,13,15,
11,12,12,13,13,15,15,15,

Depending on your tastes, you'll need atleast 1000kbit/s.
That's approx the average bitrate I got with a clean source, that had a mix of fast and slow motion scenes at --crf 20.
Fast and fairly intense chapter 20 from star wars ep. 3, required 3200 kbit/s at --crf 20.

edit: Almost forgot, the source was resized to (576,240)

[bond]

why would someone want to make a high bitrate encode with 576x240?

[alea35]

can someone explain to me how to make these matrices work? i usually just use a batch file to encode and was wondering how i could incorporate these into my batch file?

[Revgen]

Use --cqmfile "yourdrive:/yourcqmmatrix.cfg"

Make sure to include the quotation marks.

[Audionut]

why would someone want to make a high bitrate encode with 576x240?

Why Indeed?

The numbers quoted, was from a test I was running to compare the bitrate produced against the other AVC matrices.
I was taking a short cut with the resolution, for faster coding/comparing.

[Audionut]

Audionut's general matrix.

INTRA4X4_LUMA =
8,11,13,19,
11,16,22,31,
13,22,39,60,
19,31,60,110

INTRA4X4_CHROMAU =
8,11,13,19,
11,16,22,31,
13,22,39,60,
19,31,60,110

INTRA4X4_CHROMAV =
8,11,13,19,
11,16,22,31,
13,22,39,60,
19,31,60,110

INTER4X4_LUMA =
14,14,17,26,
14,21,30,44,
17,30,46,87,
26,44,87,162

INTER4X4_CHROMAU =
14,14,17,26,
14,21,30,44,
17,30,46,87,
26,44,87,162

INTER4X4_CHROMAV =
14,14,17,26,
14,21,30,44,
17,30,46,87,
26,44,87,162

INTRA8X8_LUMA =
8,11,11,11,12,13,16,19,
11,11,11,11,12,14,17,20,
11,11,12,13,15,17,20,24,
11,11,13,16,19,22,26,31,
12,12,15,19,25,30,36,42,
13,14,17,22,30,39,49,60,
16,17,20,26,36,49,65,83,
19,20,24,31,42,60,83,110

INTER8X8_LUMA =
14,14,14,14,15,17,21,26,
14,14,14,14,15,18,23,27,
14,14,15,17,20,23,27,33,
14,14,17,21,26,30,36,44,
15,15,20,26,35,42,51,60,
17,18,23,30,42,46,58,87,
21,23,27,36,51,58,78,122,
26,27,33,44,60,87,122,162


I would class this a low to medium bitrate matrix. Tested against Sharktooth's excellent EQM_AVC matrix, it produces better SSIM at bitrates around 700 kbit/s.

[Revgen]

I would class this a low to medium bitrate matrix. Tested against Sharktooth's excellent EQM_AVC matrix, it produces better SSIM at bitrates around 700 kbit/s.

IIRC, Sharktooth's matrix was made for high bitrates.

[Sharktooth]

i confirm what revgen said. 700kbps is too low for EQM AVC-HR (unless the source is supercompressible o.r the resolution is lowered)

[leowai]

The usage of the EQM AVC-HR cqm is mentioned in the download page (2005-08-01).
http://forum.doom9.org/showpost.php?p=677092&postcount=1
EQM AVC-HR rev.1
This matrix was made for medium-high bitrate backups with High-Profile AVC encoders that support custom matrices and 8x8 discrete cosine transform. It also reduces blocking.

Usage:
- high quality backups of DVD material at full resolution (anamorphic) and bitrates of at least 1300-1400kbps (for average motion and not noisy/old movies).
- non anamorphic/vertically resized encodes at bitrates of at least 1000kbps.

Standard version (JM, x264 & Encavc beta2-2 compatible)

Probably cqm users need more information when using the cqm. I think it's encouraged that the author could follow what sharktooth did. Just provide more detail information & recommandations about the usage of their cqm on the download page, i.e.:
- Movie resolution (640x480 up to 720x480)
- Interlaced* or Progressive video?
*Sorry, i'm not familiar to cqm, is there any cqm for interlaced video? :confused: (where is the "I'm with stupid" icon? I can't find it!!)
- Recommended bitrates (1000kbps)
- Recommended movie type (Anime, TV & etc)

[Oline 61]

What matrix should I use for high bitrate (3905 kbit/s) backups of live action DVD movies? Should I just use JVT?

[Oline 61]

bump?
Should I try without any matrix?

[kotrtim]

INTRA4X4_LUMA =
10,12,14,16,
12,14,16,18,
14,16,18,20,
16,18,20,20

INTRA4X4_CHROMAU =
13,14,15,16,
14,15,16,18,
15,16,18,20,
16,18,20,20

INTRA4X4_CHROMAV =
13,14,15,16,
14,15,16,18,
15,16,18,20,
16,18,20,20

INTER4X4_LUMA =
10,12,14,16,
12,14,16,18,
14,16,18,20,
16,18,20,20

INTER4X4_CHROMAU =
13,14,15,16,
14,15,16,18,
15,16,18,20,
16,18,20,20

INTER4X4_CHROMAV =
13,14,15,16,
14,15,16,18,
15,16,18,20,
16,18,20,20

INTRA8X8_LUMA =
10,10,11,12,12,13,14,15,
10,11,12,13,13,15,15,16,
11,12,12,14,15,15,16,17,
12,13,14,15,15,16,17,18,
12,13,15,15,16,17,18,19,
13,15,15,16,17,18,19,19,
14,15,16,17,18,19,19,20,
15,16,17,18,19,19,20,20

INTER8X8_LUMA =
10,10,11,12,12,13,14,15,
10,11,12,13,14,14,15,16,
11,12,12,14,14,15,16,17,
12,13,14,14,15,16,17,18,
12,14,14,15,16,17,18,19,
13,14,15,16,17,18,19,20,
14,15,16,17,18,19,20,20,
15,16,17,18,19,20,20,20

try this?
No matter how much you tweak the default flat16, it is always a drop in psnr...
This is just simply cut and paste from xvid's SixOfNine, merging with soulhunter_v1's 4x4s .....decresing all the values so that no. 20 is the
max...

soulhunter's v2 has some problems, it tends to so some ugly red spot on a clean blue sky with crf 26, well, i tried this sixofnine port, it does solve this problems... still need testing

[Soulhunter]

Red spot on a clean blue sky?


Bye

[kotrtim]

http://img111.imageshack.us/img111/3718/ori7vs.png
http://img209.imageshack.us/img209/1498/soulhunter5me.png
http://img350.imageshack.us/img350/680/eqmavchr1qv.png
http://img480.imageshack.us/img480/729/jvt5tx.png
http://img117.imageshack.us/img117/7871/soulsix6dd.png

picture order
1. original
2. soulhunter v2
3. eqm_avc_hr
4. jvt
5. soulhunter/sixofnine merge

look carefully, the top left corner (North East)
I'm using --nf

besides this, soulhunter v2 will produce errors like one big red area on a clean blue sky, tried eqm, no such errors, i will try to reproduce.

EDIT: sorry, checked eqm, same error, even worst 2 spot, one clear, one faint...jvt does not have such errors, which is very annoying

EDIT2 : Tested with more different settings, the above encodes are all
default settings (except crf 26 , cqm, nf).. I tried subme 6 and it help to make the spot less noticible and also shifted the spot to another location, subme 6 + no fast pskip solved all the problems, it's more like a x264 bug, not a cqm problem, but of course if default settings is used with default matrix, no such problems

So for those who use cqm, subme 6 + no fast pskip is a must, even better if trellis 2 is used

[Oline 61]

Thanks for the matrix, kotrtim. I'll try it out tonight.

[Oline 61]

That matrix looks great. Better than JVT at that bitrate. I think no matrix may still look better at that bitrate.

[Sharktooth]

look at how EQM AVC-HR reduces blockiness in the sky/clouds... while soulhunter/6of9 merge has the worts looking sky/clouds
i will check what causes that red/brown dot though...

[kotrtim]

That matrix looks great. Better than JVT at that bitrate. I think no matrix may still look better at that bitrate.

Yes, the effortless, all 16 seems to be the best, shartooth's said that his CQM is for high bitrate, but I think I could say that his matrix can really reduce big nasty blocks that flat16 produce, especially skies, clouds, smokes... consider using his matrix at --crf 26 ...... or if you get a rate factor of < 28 for 1st pass

when you use crf above 30 ~ 40, the best is still flat 16, tried mp4guy, smooth images too much, flat16 is pretty balance..serves as general purpose, good for every bitrate, it's not same as ASP H.263.

[Oline 61]

The problem is my movie, Friday Night Lights has a desert scene near the beginning, and the brownish desert sand has nasty reddish artifacts on it with EQM-AVC-HR. I will try to get some screenshots up in a minute or two.

EDIT: Actually I can't seem to reproduce this problem. Maybe trying eqm_avc_hr.cfg again would be a good idea.

At my current bitrate and resolution (3955kbit/s 640x480) I don't think I have to worry too much about blocking.

[kotrtim]

Description:
1. It should look more or less like default;
2. with slightly reduced blocks but not more;
3. and reduced ringing artefacts.
4. Therefore, it can serve as all purpose matrix.

since leowai said a matrix should come with desciption, oline 61, do you find this description true with the sample you tested, because this conclusion is made from only one test sample, doing visual comparison is very eye straining and takes lots of time... thanks...

How about calling it "soulhunter_6of9"?:)

Audionut, tried your Hi bitrate matrix too, wow at the same --crf, it needs more than 1.5X the space of all other matrices
Your general matrix seems to treat the cloud well too, I will capture a screenshot

[Oline 61]

I agree with that description 100% kotrtim. I will look at a few more clips with it and report back. I'll also try out those Audionut matrixes, they sound nice.

[Oline 61]

BTW, here is an archive with all the matrices listed in this thread. If you don't want your matrix in this archive, tell me so and I will remove it.
http://home.nc.rr.com/oline61/x264_custom_matrices.7z

I am currently planning to generate several clips designed to test for specific behaviors, and then encode with different quant matrices, and test PSNR and SSIM. What settings should I use to do this?

I don't think --qp or --crf will work because they produce different sizes with different matrices which is no good for SSIM and PSNR testing. Should I do 2-pass or 1-pass ABR? For 2-pass do I need to do a new first pass for each matrix? Should I use slow high quality settings, or faster settings? What bitrates would you like to see represented?

[Kostarum Rex Persia]

Sorry for asking that, but I lost my collection of custom matrices. So, I don't know what matrices are used for ultra-low and low bitrates.

Please, tell me.

[Oline 61]

For low bitrates MP4Guy's Low Bitrate v2 and v3.
They are in the archive I posted and in this thread.
Look at the numbers. Bigger numbers = more compressibility. Generally......

[kotrtim]

For 2-pass do I need to do a new first pass for each matrix?
:( Yes, akupenguin said we should, but if it is 3 pass, 1st pass can be anything, the 2nd and 3rd has to be the same.... it's very time consuming

Should I use slow high quality settings, or faster settings?
The red/brown spot problem can be avoided if you use subme 6 + no fast pskip....but I don't think the "spot" phenomena is gonna happen so often...

[Marsu42]

BTW, here is an archive with all the matrices listed in this thread.

Thanks a lot for your work gathering the matrices. Since the archive seems to be intended to save the time to search all relevant posts, it would be great if you could also append the target bitrate to the matrices' names in the future, eg. audionut_general.cfg would be something like audionut_general_700-1000.cfg or ..._all. the same suggestion goes for the original developers as well.

I am asking for this because in many posts the bitrate is often not declared at all or only as eg. 'low', which imho can anything from 100 to 1000 or even above. Furthermore, one wondersDue to this, it's quite difficult to try them out, which is not the case with names like "Jawor's 1CD Matrix".

[Oline 61]

Marsu42, I plan to test these matrices at different bitrates and determine their PSNR and SSIM. By objectively determining the best matrices for each bitrate I plan to determine which matrices perform well at which bitrate.

Here are the results for 4000kbit/s. I only included matrices designed for high bitrates. I may add more matrices at this bitrate later. These are just preliminary results, and the commandline settings may be refined later.

*EDIT: Added clip info.*
The clip is from Friday Night Lights. I am using the script that I set up for a 1 DVD5 backup of the movie.
MPEG2Source("VTS_01_1.d2v")
Telecide(order=1,guide=1,post=0)
Decimate()
ColorMatrix()
LanczosResize(640,480,taps=9)
BlockBuster(method="noise",detail_max=20)
Trim(1030,169045)
Trim(0,1000)
The clip consists of 1000 frames, starting with some text, moving into a shot of clouds rolling across the sky, then various shots of the desert, including low movement stills and pans across the landscape.


#pass 1
x264 --pass 1 --stats stats.log -B 4000 --subme 1 --me dia --analyse none --cqmfile matrix.cfg --progress --fps 23.976 -o output.264 input.yuv 640x480
#pass 2
x264 --pass 2 --stats stats.log -B 4000 --subme 3 --me hex --analyse all --8x8dct --cqmfile matrix.cfg --progress --fps 23.976 -o output.264 input.yuv 640x480


Here are the results:

audionut_hr.cfg
Average SSIM= 80.18
PSNR Mean Y:44.478 U:56.815 V:58.696 Avg:45.883 Global:45.565 kb/s:4021.56

eqm_avc_hr.cfg
Average SSIM= 80.41
PSNR Mean Y:44.415 U:56.814 V:58.672 Avg:45.826 Global:45.480 kb/s:4000.45

flat
Average SSIM= 80.83
PSNR Mean Y:44.941 U:56.329 V:58.114 Avg:46.257 Global:45.925 kb/s:4022.82

jvt
Average SSIM= 80.76
PSNR Mean Y:44.581 U:56.905 V:58.775 Avg:45.985 Global:45.685 kb/s:4007.02

soulhunter_6of9.cfg
Average SSIM= 81.11
PSNR Mean Y:44.875 U:56.324 V:58.119 Avg:46.197 Global:45.877 kb/s:4010.98

[IgorC]

Seems like for the first time cm (Soulhunter_6of9) has higher SSIM result and only slightly inferior OPSNR. Good.
Who already plaid with cm on 1200-1500 kbit/s ( movie source, not anime) for best SSIM result?

[IgorC]

...So for those who use cqm, subme 6 + no fast pskip is a must, even better if trellis 2 is used

If I'm not mistaking x264 doesn't use fast skip when cm is enable.

[Audionut]

@Online 61. IMO, you should be using B-frames.

[Sharktooth]

lets be serious... no one encodes at 4mbps with h.264...
sagittaire shown even HD do not need such bitrates...

[Soulhunter]

lets be serious... no one encodes at 4mbps with h.264...
sagittaire shown even HD do not need such bitrates...
Hey, I go even higher... ;P

My aim is a PSNR ~50dB (visual lossless)


Bye

[Sharktooth]

then use xvid :p
what you do is like encoding a 640x480 source at 20mpbs with xvid... it's useless, just use MPEG-2...

[IgorC]

lets be serious... no one encodes at 4mbps with h.264...
sagittaire shown even HD do not need such bitrates...

for H.264 1280x720 3.8-4.5 Mbit/s is normal bitrate. Less than 3.8 mbit/s the video starts to smooth.

[Sirber]

WHAT?!?!?!?!?!

4mbps for 640x480 :eek:

[Sharktooth]

for H.264 1280x720 3.8-4.5 Mbit/s is normal bitrate. Less than 3.8 mbit/s the video starts to smooth.
... get the sagittaire encodes :P

however
LanczosResize(640,480,taps=9)
4mbps for that is an overkill...

[IgorC]

... get the sagittaire encodes :P

however

4mbps for that is an overkill...

And? I saw them. Very smooth. Many details are washed. Those "HD" at 1600 kbit/s? I do 1200-1500 kbit/s for transparent DVD backups.
x.264 is excelent codec but can't do magic.

QT H.264 Baseline Profile +1b (Low complexity) for 1280x720 uses 5-6mbit/s.
I would repeat another time for H.264 MP/HP 3,8-4,5 Mbit/s is a normal bitrate for 1280x720.

[Sharktooth]

ok.. but he encoded a 640x480 source at 4mbps....

[IgorC]

yes, here I agree . 640x480 4 mbps is even more than enoughl for MPEG-2

[*.mp4 guy]

I have encoded movies at ~1500kbps with X264, the encodes have a lot of artifacts, and texture is never kept perfectly. ~3000 kbps is better, but most of the time its still not "Perfect". Theres no harm in trying to get transparent results from X264, and mpeg2 doesn't look transparent on most sources at 4mbps.

[Sharktooth]

maybe some particular sources (very noisy stuff, ect)... but the majority of DVD movies doesnt need that bitrate.
I encoded all the Band of Brothers (very noisy and grainy) episodes at 1600 with AVC HR and it was enaugh to keep both noise and grain with no artifacts.

[Soulhunter]

then use xvid :p
what you do is like encoding a 640x480 source at 20mpbs with xvid... it's useless, just use MPEG-2...

Nah, with XviD i can get such a high PSNR with "just" 4-5mbps, so i can fit most flicks on a single-layer DVD without loosing (visible) quality. Humm, you really need 20mbps via h.264 to reach a PSNR close to 50dB? Probably thats the reason i still use XviD for my daily encodes... ^^;


Bye

[Sharktooth]

Humm, you really need 20mbps via h.264 to reach a PSNR close to 50dB? Probably thats the reason i still use XviD for my daily encodes... ^^;
i didnt say that...

[Soulhunter]

i didnt say that...
Right, you said Id need 20mbps for a 640x480 source to reach a PSNR ~50dB via XviD, correct?

Well, just read through my old encode logs...

The day after tomorrow - DVD PAL/R2

1024x576 (Lanczos) / 25fps / 118min.

XviD 1.0.2
Fixed Quantizer 2 (VHQ1)
I-frame interval = 250
Chroma motion
B-VOP's = 1 / 1 / 1
Matrix = Soulhunters v3

Average bitrate: 3668.7 kbit/s

PSNR: 49.69 dB (Y 48.26 U 49.99 V 50.80)


Gladiator - DVD PAL/R2

1024x576 (Lanczos) / 25fps / 148min.

XviD 1.0.1
Fixed quantizer 2 (VHQ1)
I-frame interval = 250
Matrix = MPEG

Average bitrate: 4589.9 kbit/s

PSNR: 50.30 dB (Y 47.94 U 51.34 V 51.62)


The Italian job - DVD PAL/R2

1024x576 (Lanczos) / 25fps / 106min.

XviD 1.0.0
Fixed quantizer 2 (VHQ1)
I-frame interval = 250
Matrix = Soulhunters v3

Average bitrate: 4931.0 kbit/s

PSNR: 49.51 dB (Y 48.93 U 49.62 V 49.97)


Underworld - DVD PAL/R2

1024x576 (Lanczos) / 25fps / 128min.

XviD 1.0.2
2Pass @ 4425 kbps (VHQ1)
I-frame interval = 250
Matrix = Soulhunters v3

Average bitrate: 4420.0 kbit/s

PSNR: 49.89 dB (Y 49.02 U 49.52 V 51.13)

So if i need 4-5mbps for 1024x576, 640x480 certainly wont need 20mbps, huh?


Bye

[Daodan]

I wonder if Sharktooth isn't refering to global PSNR, not average. By the way, what's global PSNR supposed to be?

[Sharktooth]

i was referring to the fact you said "i go even higher" and i thought you did it with h.264...

[Soulhunter]

Ah, okay... my fault then... my reply was sorta vague, heh? ^^;

Anyway, for high bitrate encodes I recommend to use XviD!

AVC only pwns at low-mid bitrates imho... :\


Bye

[Audionut]

Off-topic.

I hope a mod split this thread at post #47.

[Oline 61]

Yeah, this thread has kind of gone off topic, but.....

The previous results are flawed, because I agree no one uses that much bitrate for 640x480. I will try more testing overnight.

What bitrates do you specifically want to see tested at 640x480?

[*.mp4 guy]

750, 1250 and 2000 would be good.

[kotrtim]

496 x 272 23.976 fps

just emulate noise by addgrain (30,0,0)
that is overkilling the codec, even with 3000 kbps, it still fail to preserve the noise.
really bad psnr 38..only...but for clean spurce it generally does not need such nigh bitrates

What bitrates do you specifically want to see tested at 640x480?

This is just my suggestion... Encode with only "--crf 26/25 --bframes X" , see what is the final bitrate you got for CRF, then use this bitrate to do 2-pass

[Soulhunter]

/me votes for 500kbps (very low), 1000kbps (low), 1500kbps (medium) ^^


Bye

[Sharktooth]

/me votes for 500kbps (very low), 1000kbps (low), 1500kbps (medium) ^^
1000 aint that low... it's "medium"... and 1500 is mid/high.

[Soulhunter]

500kbps (very low), 1000kbps (low), 1500kbps (medium), 2000kbps (high), 2500kbps (very high) :D :p


Bye

[IgorC]

I should say watchig at all custom matrices that x264 does good.
There some matrices to keep filmgrain, another for anime etc.

Some advanced users of Xvid like Soulhunter and Teegedeck say that they prefer Xvid+CM because it keeps "details" perfectly.

I prefer H.264 look and of course maybe H.264 with its inloop filter remove some filmgrain and/or noise, but all details are perfectly preserved at certain bitrate.
There is another considetation. What is the noise and what are the details. But it's matter of taste and nothing else.

[Sharktooth]

AVC-HR at 1600kbps...
http://www.webalice.it/f.corriga/temp/grain_example.png
grain is perfectly preserved... :)

[IgorC]

Exactly :)

[emmel]

I have now tested x264 for a month or two, with different settings and Sharktooth's profiles from "fast to insane". My data consists of about 1T of dvb/pal-recordings in mpeg2 ts, and I have re-encoded some of it into h264 with x264. I'm a visual perfectionist, but I'd say that a bitrate somewhere between 1400-1800kbps, depending on the material of course, is enough to preserve the original quality in any case. More is an overkill, and less you notice as artifacts. So, I agree with the Soulhunter classification of the bitrates. Maybe some day I'll learn more, and change my opinion... :)

[Soulhunter]

AVC-HR at 1600kbps...
http://www.webalice.it/f.corriga/temp/grain_example.png
grain is perfectly preserved... :)
And where is the source frame? ;)


Bye

[Sharktooth]

i need to take a SS from the DVD

[Oline 61]

Bleh, on 640x480 sources I usually get far too much blocking below about 1750 kbit/s.

I need to scrap together a commandline for the test. How 'bout this:
x264 --pass 1 --bitrate 0000 --stats x264stats.log --bframes 3 --b-pyramid --filter -2,-2 --subme 1 --analyse none --direct auto --me dia --cqmfile ~/matrices/matrix.cfg --progress --output out.264 in.avs

x264 --pass 2 --bitrate 0000 --stats x264stats.log --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --filter -2,-2 --subme 6 --trellis 1 --analyse all --8x8dct --direct auto --me umh --cqmfile ~/matrices/matrix.cfg --progress --output out.264 in.avs

[Sharktooth]

Bleh, on 640x480 sources I usually get far too much blocking below about 1750 kbit/s.
That's quite impossible unless your sources are really of crap quality or so noisy to blow your eyes...
I need to scrap together a commandline for the test. How 'bout this:
x264 --pass 1 --bitrate 0000 --stats x264stats.log --bframes 3 --b-pyramid --filter -2,-2 --subme 1 --analyse none --direct auto --me dia --cqmfile ~/matrices/matrix.cfg --progress --output out.264 in.avs

x264 --pass 2 --bitrate 0000 --stats x264stats.log --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --filter -2,-2 --subme 6 --trellis 1 --analyse all --8x8dct --direct auto --me umh --cqmfile ~/matrices/matrix.cfg --progress --output out.264 in.avs
why --filter -2,-2 ?

[Oline 61]

Okay, it is pretty noisy as DVD movies go, and there are already blocks in the MPEG2 video.

But why not filter -2,-2?

[Sharktooth]

you should postprocess/deblock your mpeg2 source then.
also the --filter parameters should be set to a higher level if you have blocking...

[Oline 61]

Using the built in deblocked in DGDecode smears and blurs so much that I'd rather just have blocks. The movie is a Pan and Scanned version of Friday Night Lights on a Dual Layer DVD. I am amazed that they managed to have blocking in a large scale movie on a medium of that capacity though. I guess I'll just live with it. It's not extremely apparent blocking.
I'll try a full encode tonight and post source and compressed screenshots tomorrow. The biggest problem is that the artifacts are very apparent once the 640x480 video is blown up to fullscreen on my 1280x1024 monitor. I wan't to encode at a higher resolution, but it takes so long to PixieDust+LimitedSharpen and then encode at a high res.

[Marsu42]

/me votes for 500kbps (very low), 1000kbps (low), 1500kbps (medium) ^^
1000 aint that low... it's "medium"... and 1500 is mid/high.

... good to have that sorted out :-). Nice way to improve one's post count, though.

Personally, I encode DVDs @non-anamorphic original size (usually 704x300 or something like that after cropping and resizing to square) and target 1CD. The result ususally is ~900kbps with ps-he aac sound.

While I have only used XviD matrices so far, I guess that's the region where cqms make sense: @4000kpbs you shouldn't use cqms, but just encode @5000kpbs instead because space doesn't seem to matter. On the other hand, 640x460@500kbps will look crappy anyway (if it's not a documentaton of still photography).

Therefore, my vote clearly goes to 900-1000kpbs, it would be nice it any of you wizards would come up with a test that shows whether cqms do make a visible difference or I should just forget about them (ffdshow doesn't play them anyway, right?).

[Sharktooth]

... good to have that sorted out :-). Nice way to improve one's post count, though.

Personally, I encode DVDs @non-anamorphic original size (usually 704x300 or something like that after cropping and resizing to square) and target 1CD. The result ususally is ~900kbps with ps-he aac sound.

While I have only used XviD matrices so far, I guess that's the region where cqms make sense: @4000kpbs you shouldn't use cqms, but just encode @5000kpbs instead because space doesn't seem to matter. On the other hand, 640x460@500kbps will look crappy anyway (if it's not a documentaton of still photography).

Therefore, my vote clearly goes to 900-1000kpbs, it would be nice it any of you wizards would come up with a test that shows whether cqms do make a visible difference or I should just forget about them (ffdshow doesn't play them anyway, right?).
support for cqms in ffdshow (libavcodec) has been added some months ago. just get a recent build (from x264.nl, celtic druid, etc) and it will have no problems.
also cqms make sense at any bitrate as in xvid. there are times you rise the bitrate but wont notice any quality improvment... but if you use a cqm you can clearly see some difference.
regarding 640*xxx@500kbps, some ppl use even lower bitrates (ask sirber...) for anime backups and they do not look crappy at all :)

[Chainmax]

Oline 61: try DeBlock_QED_MT2() or FunkyDeBlock() for block removal and FFT3DFilter for noise removal, then LimitedSharpenFaster() after the final resize. This will probably make the movie look better and help x264 make a better encode.

[Oline 61]

I'll give it a shot. I think I might leave out FFT3DFilter so that the encode will be <24 hours. I don't think denoise is nessecary for sharpening without upscaling.

[Chainmax]

Well, you could always try DeGrainMedian(), although it does remove a bit more detail than FFT3DFilter's sigma=1. For some very slight denoising that almost doesn't slow down encoding at all, try RemoveGrain(mode=5).

[siddharthagandhi]

what is a matrice?

[Oline 61]

what is a matrice?
Search!

Thanks for the denoising and filtering advice. I have been experimenting a lot. FFT3dFilter(sigma=1) is too slow, FFT3dGPU(sigma=1) is okay though. I may try out RemoveGrain. FFT3d* seems to muddle some of the picture too much. Also, Deblock_QED doesn't seem to be having any effect at all, even at high settings.

[siddharthagandhi]

I did...it doesnt show any substantive results

[Oline 61]

I did...it doesnt show any substantive results
http://forum.doom9.org/showthread.php?t=54147&highlight=quantizer+matrix
Maybe try a little harder next time?

[siddharthagandhi]

Well, that says matrix. My search query was matrice.

[*.mp4 guy]

They are the same thing, matrices is a commonly used plural form of matrix.

[siddharthagandhi]

Yea I know that but one gives the appropriate search results while the other doesnt.

[Oline 61]

Sounds like it would be a great idea to try multiple differing iterations of your search next time.

[foxyshadis]

Yeah, this thread has kind of gone off topic, but.....

The previous results are flawed, because I agree no one uses that much bitrate for 640x480. I will try more testing overnight.

What bitrates do you specifically want to see tested at 640x480?
If you ever get a chance to further work on this, I think --crf 18, 22, 26, 30, 34 should be tested, rather than any specific bitrates. That will test both how bitrate-hungry a matrix is, in addition to how it looks (which is probably secondary to how big it is, unless there are major visual differences).

[Oline 61]

Haven't got around to it. Next week is Spring Break and I'll be at the beach. Teachers heaped on a lot of work the last couple weeks before break. Hopefully I can do it after Break.

The problem with using CRF is quality cannot be compared. The only result we can deduce from a --crf encode is how bitrate hungry the matrix is, which is not something I am interested in. I want quality results.

[Oline 61]

I've finally gotten around to testing out some matrices.
I'm starting with a 1000-frame clip of the lobby scene from The Matrix @ 1400 kbps.

AVS:
MPEG2Source("VTS_02_1.d2v",idct=3)
DoubleWeave().Pulldown(0,3)
Crop(0,62,720,354)
ColorMatrix()
Lanczos4Resize(848,352)
Trim(146423,147422)
x264 (rev.516):
#!/bin/sh
mkfifo raw.y4m

##########################################

wine "C:\avs2yuv.exe" ~/MATRIX/VTS_02_1.avs raw.y4m & \

x264 --pass 1 --bitrate 1400 --bframes 2 --b-pyramid --filter -3:-3 --subme 1 --analyse none --direct auto --me dia --cqmfile matrix/eqm_avc_hr.cfg --progress --no-psnr --output /dev/null raw.y4m

##########################################

wine "C:\avs2yuv.exe" ~/MATRIX/VTS_02_1.avs raw.y4m & \

x264 --pass 2 --bitrate 1400 --ref 5 --mixed-refs --no-fast-pskip --bframes 2 --b-pyramid --b-rdo --bime --weightb --filter -3:-3 --subme 6 --trellis 1 --analyse all --8x8dct --direct auto --me hex --cqmfile matrix/eqm_avc_hr.cfg --progress --output test.264 raw.y4m

##########################################

MP4Box -add test.264 -fps 23.976 test.mp4

rm raw.y4m

rm test.264
Create AVI for AVISynth+SSIM Calculation:
#!/bin/sh

mencoder test.mp4 -mc 0 -noskip -vf format=YV12 -nosound -ovc raw -ofps 24000/1001 -o test.avi


EDIT: DO NOT PAY ATTENTION TO THE NUMBERS BELOW. THEY IN NO WAY REFLECT THE ABILITIES OF THE QUANTIZATION MATRICES TESTED.
Results:
soulhunter_v2
PSNR: 43.458
SSIM: 75.73

eqm_avc_hr
PSNR: 43.311
SSIM: 75.04

soulhunter_v1
PSNR: 43.460
SSIM: 75.61

flat16
PSNR: 43.733
SSIM: 76.96

jvt
PSNR: 43.444
SSIM: 75.70

I WILL ADD RESULTS AS THEY COME IN!!

[Sharktooth]

EQM AVC-HR was not meant for best metrics...
it was meant to help blocking issues at mid-high bitrates and preserve a decent quality (as you can see from this post on: http://forum.doom9.org/showthread.php?p=796300#post796300 )

[Oline 61]

Yeah, as I look at the videos, the highest metrics (flat16) often look like complete crap compared to eqm_avc_hr. I guess metrics are really a very poor way to compare matrices. I may move toward posting some screenshots.

Do you think that 1400kbit/s is a little low for eqm_avc_hr at 848x352?

[Oline 61]

Here we go, screenshots. I've chosen 3 specific frames to compare from each video. Prepare your browser for an assault of PNG's.

Shell Script:
#!/bin/sh
mkfifo raw.y4m
##########################################
wine "C:\avs2yuv.exe" ~/MATRIX/VTS_02_1.avs raw.y4m & \
x264 --pass 1 --bitrate 1400 --bframes 2 --b-pyramid --filter -3:-3 --subme 1 --analyse none --direct auto --me dia --cqm flat --progress --no-psnr --output /dev/null raw.y4m
##########################################
wine "C:\avs2yuv.exe" ~/MATRIX/VTS_02_1.avs raw.y4m & \
x264 --pass 2 --bitrate 1400 --ref 5 --mixed-refs --no-fast-pskip --bframes 2 --b-pyramid --b-rdo --bime --weightb --filter -3:-3 --subme 6 --trellis 1 --analyse all --8x8dct --direct auto --me hex --cqm flat --progress --output test.264 raw.y4m
##########################################
MP4Box -add test.264 -fps 23.976 test.mp4
rm raw.y4m
rm test.264
mencoder test.mp4 -mc 0 -noskip -vf format=YV12 -nosound -ovc raw -ofps 24000/1001 -o test.avi
rm test.mp4
rm x264_2pass.log

Okay, there are a lot of PNG's now. Too many to put in a forum post, so I refer you here. http://home.nc.rr.com/oline61/
Sorry, I don't feel like writing any HTML right now, so just download them and compare them in an image viewer.

[*.mp4 guy]

If your still doing testing could you include this high detail matrix that I've been working on.

INTRA4X4_LUMA =
12,16,22,28,
16,24,32,40,
22,32,48,72,
28,40,72,112

INTRA4X4_CHROMAU =
16,22,32,72,
22,24,41,144,
32,41,96,200,
72,144,200,255

INTRA4X4_CHROMAV =
16,22,32,72,
22,24,41,144,
32,41,96,200,
72,144,200,255

INTER4X4_LUMA =
16,20,21,24,
20,21,24,44,
21,24,56,80,
24,44,80,112

INTER4X4_CHROMAU =
16,22,22,22,
22,22,22,56,
22,22,72,96,
22,56,96,128

INTER4X4_CHROMAV =
16,22,22,22,
22,22,22,56,
22,22,72,96,
22,56,96,128

INTRA8X8_LUMA =
8,12,13,16,20,22,24,26
12,12,16,19,22,24,26,28
13,16,20,23,24,26,28,33
16,18,22,24,24,28,32,38
19,21,23,25,29,35,44,49
22,22,24,26,35,46,56,67
22,22,25,30,41,54,72,88
22,25,28,36,47,65,88,115

INTER8X8_LUMA =
12,16,17,19,20,21,22,24
16,16,18,18,20,21,23,25
17,18,18,18,19,21,24,26
19,18,18,19,19,21,24,28
20,20,19,19,20,21,26,32
21,21,21,21,21,23,26,38
22,23,24,24,26,26,32,46
24,25,26,28,32,38,46,58

[Kurth]

I dont really understand those PSNR values.
Did higher PSNR value mean better quality and lower PSNR mean worst quality?

eqm_avc_hr
PSNR: 43.311
SSIM: 75.04

flat16
PSNR: 43.733
SSIM: 76.96

Them those results mean that flat16 is better and eqm_avc_hr is worst?
Im confused :confused:
Can anyone explain this ?

[Audionut]

PSNR & SSIM are metrics.

Just because the metrics say one is better than the other, doesn't always mean that, that is actually the case.

http://forum.doom9.org/showthread.php?t=110504

[Sagittaire]

If your still doing testing could you include this high detail matrix that I've been working on.

"high detailled matrix" with really agressive quantisation for high frequencies. It's hard for me to understand that (perhaps that I don't understand very well DCT transformation for H264). IMO simply placebo effect ...

All your matrix sound like old HVS quantisation (low coef for low frequencies and high coef for high frequencies) and this HVS repartition is exclusively fight again blocking. In fact this HVS tuning done better gradiant and certainely not better detail preservation (ringing effect is higher). This repartition is good for eyes if you use codec like ASP but be carefull AVC use inloop. HVS tuning for AVC can't be like for ASP. HVS optimisation is really complexe combinaison between inloop - and 8 CM inter/intra/luma/chroma/16*16/4*4 (and certainely other like curve quant compression, quantizer chroma offset AQ, ratio for I,B frame ...)

But perhaps that my opinion is wrong ... Manao, pengvado or other H264 dev will be certainly very better than me for explain this problem

[Sagittaire]

I dont really understand those PSNR values.
Did higher PSNR value mean better quality and lower PSNR mean worst quality?

eqm_avc_hr
PSNR: 43.311
SSIM: 75.04

flat16
PSNR: 43.733
SSIM: 76.96

Them those results mean that flat16 is better and eqm_avc_hr is worst?
Im confused :confused:
Can anyone explain this ?

If HVS matrix were indisputably better visually than the flat then this HVS matrix would be the default matrix in all H264 codec. Simply placebo effect here

Metric can't evaluate CQM quality optimisation simply because :

1) Metric done very similar result between these matrix (less than 0.3 dB for PSNR and less than 2 pt for SSIM). Conclusion here is impossible ...

2) Metric like PSNR are unable to evaluate pure HVS optimisation like adaptative quantisation, CQM ...

3) These comparison are funny. Personnaly I will wait real public blind test for evaluate these matrix. IMO flat matrix will be the best in most situation for this test ... lol

[IgorC]

I remeber when I tested Ateme HP beta different psy (aka AQ) opsnr failed badly but not SSIM. for SSIM (and my perception) psy 0 was almost always worse than psy1/psy2.
IMO it depends on video source. It is not real to probe each time 1000 matrices for different kind of sources to find best metrics.

[Oline 61]

For anyone confused by the PSNR and SSIM results, remember, they compare differences from the source. They don't determine how good the output looks. Please look at the screenshots, and surprise, eqm_avc_hr pwns flat16.

If your still doing testing could you include this high detail matrix that I've been working on.
Sure, the pictures should be uploaded in a few minutes.

And it appears I have run out of my 5MB of hosting space. ImageShack time :).
http://img227.imageshack.us/img227/4294/mp4guyhighdetail00oo.png
http://img89.imageshack.us/img89/3271/mp4guyhighdetail13gv.png
http://img89.imageshack.us/img89/7103/mp4guyhighdetail23sa.png

[Oline 61]

Following foxyshadis reccommendation I will now do a bitrate test. This scene would probably soak up a bunch of bitrate at --crf 40, but we will start at 18.
x264 --crf 18 --ref 5 --mixed-refs --no-fast-pskip --bframes 2 --b-pyramid --b-rdo --bime --weightb --filter -3:-3 --subme 6 --trellis 1 --analyse all --8x8dct --direct auto --me hex --cqmfile matrix/eqm_avc_hr.cfg --progress --output /dev/null raw.y4m


eqm_avc_hr 2673.82 kb/s
flat 2277.00 kb/s
jvt 2370.68 kb/s

[Audionut]

"high detailled matrix" with really agressive quantisation for high frequencies. It's hard for me to understand that (perhaps that I don't understand very well DCT transformation for H264). IMO simply placebo effect ...


http://forum.doom9.org/showthread.php?t=109378

Read from post 12 onwards.

[Pomyk]

Following foxyshadis reccommendation I will now do a bitrate test. This scene would probably soak up a bunch of bitrate at --crf 40, but we will start at 18.
x264 --crf 18 --ref 5 --mixed-refs --no-fast-pskip --bframes 2 --b-pyramid --b-rdo --bime --weightb --filter -3:-3 --subme 6 --trellis 1 --analyse all --8x8dct --direct auto --me hex --cqmfile matrix/eqm_avc_hr.cfg --progress --output /dev/null raw.y4m


eqm_avc_hr 2673.82 kb/s
flat 2277.00 kb/s
jvt 2370.68 kb/s


I think this kind of test is useless. Blind test with a few different bitrates is the way to go IMO.

[Oline 61]

Allright, I think I might attempt to do a blind test, but if I do, I want to do it right. Here are thing to decide on:
- a commandline (lets go with something that you would use for transparent quality for the given bitrate. Speed is no object)
- bitrates
- provide test samples uncompressed
- where to host test samples
- use a poll on the forum, or something else
- source. My "The Matrix" source sucks, the DVD used a low bitrate, there is too much blocking for an accurate test. In contrast, I have a very good quality "Matrix Reloaded" DVD I could use. Should I use denoising like FluxSmoothT(2)? Should I use 720x304 or 848x352?
I can do all the encoding, but I may need help hosting and organizing.

[*.mp4 guy]

Allright, I think I might attempt to do a blind test, but if I do, I want to do it right. Here are thing to decide on:
1- a commandline (lets go with something that you would use for transparent quality for the given bitrate. Speed is no object)
2- bitrates
3- provide test samples uncompressed
4- where to host test samples
5- use a poll on the forum, or something else
6- source. My "The Matrix" source sucks, the DVD used a low bitrate, there is too much blocking for an accurate test. In contrast, I have a very good quality "Matrix Reloaded" DVD I could use. Should I use denoising like FluxSmoothT(2)? Should I use 720x304 or 848x352?
I can do all the encoding, but I may need help hosting and organizing.

1- I would let the people that designed a matrix set the used inloop filter, the other settings are all pretty strait forward and really are only about how long pwoplw are willing to wait.

2- I would use 1600 Kbps for High bitrate, and If you feel like doing a low bitrate comparison 750 Kbps.

3-this really depends upon space constraints.

4- rapidshare would probably work well enough.

5- Definately don't use a poll on the forum, people could vote without having seen the results, have people PM you results and post the averages when the test is over.

6- The matrix reloaded is a good source, the matrix Revolutions is harder though and it is what doom9 uses in his codec comparisons now, so If you have it it would probably be a better source. I wouldn't use filtering, X264 usually doesn't benefit from denoising.

[Oline 61]

Okay, Rapidshare looks good. I'll probably post the files as MP4's because an uncompressed/lossless AVI for a reasonable length test sample would probably be over 100MB.

I don't have Revolutions, but my Reloaded DVD is probably the best DVD source I have. Here is the script I plan to use:
MPEG2Source("VTS_01_1.d2v",idct=3)
DoubleWeave().Pulldown(0,3)
Crop(0,60,720,358)
ColorMatrix()
Lanczos4Resize(848,352)
Are there any specific sequences anyone would like to see?

I'll do a high bitrate 1600kbit/s test first.
I'll need deblocking settings from the matrix creators first though. If you would be so kind as to post them here :).

Does anyone have any more input on the commandline to be used? Here is something preliminary:
x264 --pass 1 --bitrate 1600 --filter -3:-3 \
--bframes 2 --b-pyramid \
--subme 1 --me dia \
--analyse none --direct auto --cqmfile matrixfile.cfg \
--progress --no-psnr --output /dev/null raw.y4m

x264 --pass 2 --bitrate 1600 --filter -3:-3 \
--ref 5 --mixed-refs --bframes 2 --b-pyramid --b-rdo --bime --weightb \
--subme 6 --me umh \
--analyse all --8x8dct --direct auto --no-fast-pskip --trellis 1 --cqmfile matrixfile.cfg \
--progress --output test.264 raw.y4m
The main question: Is this a commandline YOU would fell comfortable using to YOUR DVD? If not why not? I want to use a commandline that most accurately reproduces a high quality DVD backup scenario at 1600kbit/s.

[Sagittaire]

always useless: these CQM matrix use another relative balance between high and low frequencies. High agressivity for high frequencies and low agressivity for low frequencies. It's typicaly an anti-blocking method and with these matrix you can and you must use lower inloop strenght.

In fact compare these matrix is very complexe because if for your eyes is the best at inloop=-3:-3 then certainely that flat 16 will be the best for your eyes at inloop=-1;-1 or something like that. try to find best matrix is really complexe because most of these setting are no independant setting.

Developper like Manao or Pengvado are by far better than simple users likes us for find good HVS balance ... :(

[nimitz]

Probably a bit out of topic, but can anyone point me to the IEEE paper that ReferenceDivX and Sharktooth referred to when they designed the HVS matrices? I searched the forum but seems cannot find the exact paper title.

Here is the quote from the other thread by Sharktooth

"Notes: Unlike V1 and V2, EQM V3 set of matrices were generated using a matrix built starting from a HVS study on luminance by IEEE. The same used by ReferenceDivX to build his HVS matrices.The differences with V2 are minimal (coz V2 is HVS tuned too, but in a different way) but the new coefficient scaling ensures V3 is full IEEE HVS compliant."

[Sharktooth]

the documents are available here: http://ieeexplore.ieee.org but they want $$$...

[nimitz]

Hi Sharktooth,

Thanks for the reply. I have access to the IEEExplorer(well...my school has its subscription.) Do you have the title of the paper? Or it's a bunch of papers from which ReferenceDivX derived his HVS Good matrix? I have read Nill's 1985 paper and some other papers such as Ngan's 1989 paper and K.R.Rao's 1990 paper, but seems none would lead to the following weighting matrix as ReferenceDivX used in the other thread.

double hvsfwm[64]={ 1,1,1,1,1,1,1,1,
1,1,.9599,1,.9571,1,.9599,
.8746,.9283,.8898,.8898,.9283,
.8746,.7684,.8404,.8192,.7617,
.8192,.8404,.7684,.6571,.7371,
.7371,.6669,.6669,.7371,.7371,
.6571,.6306,.6471,.5912,.5419,
.5912,.6471,.6306,.5558,.5196,
.4564,.4564,.5196,.5558,.4495,
.3930,.3598,.3930,.4495,.3393,
.2948,.2948,.3393,.2480,.2278,
.2480,.1828,.1828,.1391};


Thank you.

the documents are available here: http://ieeexplore.ieee.org but they want $$$...

[Sharktooth]

Sorry i dont remember the paper title :(

[NeD tHe OnE]

BTW, here is an archive with all the matrices listed in this thread. If you don't want your matrix in this archive, tell me so and I will remove it.
http://home.nc.rr.com/oline61/x264_custom_matrices.7z

I am currently planning to generate several clips designed to test for specific behaviors, and then encode with different quant matrices, and test PSNR and SSIM. What settings should I use to do this?

I don't think --qp or --crf will work because they produce different sizes with different matrices which is no good for SSIM and PSNR testing. Should I do 2-pass or 1-pass ABR? For 2-pass do I need to do a new first pass for each matrix? Should I use slow high quality settings, or faster settings? What bitrates would you like to see represented?

http://home.nc.rr.com/oline61/x264_custom_matrices.7z

I need this but its not there in the server

[Oline 61]

http://home.nc.rr.com/oline61/x264_custom_matrices.7z

I need this but its not there in the server
Fixed.

[Oline 61]

Does anyone want to recommend deblocking settings for any of the matrices, or should I continue with -3:-3?

I am thinking of doing some clips of the Nebuchadnezzar, for high detail and motion, maybe some of the green text, although the source is blocky there already, and probably a few fighting scenes.

[Sharktooth]

not me... it really depends on the source and the quant/bitrate you use (i know deblocking is adaptive but it also influences compressibility, so...)

[Oline 61]

How about The Matrix Reloaded at 1600kbit/s. No processing, just IVTC crop and Lanczos4Resize(848,352).

[NeD tHe OnE]

Hey Oline! Wich matrix is better 4 movies like The Patriot .. n what bitrate should I go 4?

[Sharktooth]

How about The Matrix Reloaded at 1600kbit/s. No processing, just IVTC crop and Lanczos4Resize(848,352).
i would go anamorphic with no resize at all...

[*.mp4 guy]

-3:0 is a good setting most of the time, thats what I recommend for my cqm.

[Sharktooth]

Hey Oline! Wich matrix is better 4 movies like The Patriot .. n what bitrate should I go 4?
Stop asking what's best and what bitrate you should use.
:readguid: and :search:.
And FFS :readrule: