High bitrate/quality CQMs used on Bluray

[bond]

Thanks to two colleagues i got my hand on the NAL info for two bluray titles using custom quant matrices:

The Prestige (Disney / Buena Vista) has the cqm stored in PPS
X-Men 3 (20th Century Fox) has the cqm stored in SPS
I guess they were produced by different encoders

If someone has some time to spend he might convert them into .cfg standard style cqm files

[*.mp4 guy]

Prestige CQM:
#Prestige CQM

INTRA4X4_LUMA =
6,7,10,16,
7,7,11,17,
10,11,12,20,
12,13,20,16

INTRA4X4_CHROMAU =
16,16,16,16,
16,16,16,16,
16,16,16,16,
16,16,16,16

INTRA4X4_CHROMAV =
16,16,16,16,
16,16,16,16,
16,16,16,16,
16,16,16,16

INTER4X4_LUMA =
10,13,28,41,
13,14,32,84,
28,32,41,111,
41,46,111,16

INTER4X4_CHROMAU =
16,16,16,16,
16,16,16,16,
16,16,16,16,
16,16,16,16

INTER4X4_CHROMAV =
16,16,16,16,
16,16,16,16,
16,16,16,16,
16,16,16,16

INTRA8X8_LUMA =
9,9,10,10,11,16,21,29,
9,9,10,10,11,16,21,29,
10,10,10,10,11,16,22,31,
10,10,10,11,11,17,23,33,
11,11,11,11,11,19,25,36,
12,12,13,13,14,20,27,40,
15,15,16,16,21,27,31,45,
20,20,21,22,36,40,45,16

INTER8X8_LUMA =
12,13,15,18,20,70,163,255,
13,13,16,18,20,72,170,255,
15,16,17,19,21,81,190,255,
18,18,19,21,23,96,228,255,
20,20,21,23,25,120,255,255,
33,34,36,39,45,32,255,255,
64,66,71,80,164,255,255,255,
151,155,169,192,255,255,255,10
Prestige 8X8 matrices look like this:
http://i19.tinypic.com/2h5o7wj
The 4X4 luma matrices look like smaller counterparts, there are no chroma matrices.




Xmen3 CQM:
#Xmen3 CQM

INTRA4X4_LUMA =
5,9,23,32,
9,12,32,32,
23,32,32,32,
32,32,32,32

INTRA4X4_CHROMAU =
5,9,23,32,
9,12,32,32,
23,32,32,32,
32,32,32,32

INTRA4X4_CHROMAV =
5,9,23,32,
9,12,32,32,
23,32,32,32,
32,32,32,32

INTER4X4_LUMA =
9,13,20,24,
13,20,24,29,
20,24,29,29,
24,29,29,29

INTER4X4_CHROMAU =
9,13,20,24,
13,20,24,29,
20,24,29,29,
24,29,29,29

INTER4X4_CHROMAV =
9,13,20,24,
13,20,24,29,
20,24,29,29,
24,29,29,29

INTRA8X8_LUMA =
5,8,11,15,19,26,32,32,
8,9,15,19,26,32,32,32,
11,15,19,26,32,32,32,32,
15,19,26,32,32,32,32,32,
19,26,32,32,32,32,32,32,
26,32,32,32,32,32,32,32,
32,32,32,32,32,32,32,32,
32,32,32,32,32,32,32,32

INTER8X8_LUMA =
9,12,14,16,18,21,23,26,
12,13,16,18,21,23,26,29,
14,16,18,21,23,26,29,29,
16,18,21,23,26,29,29,29,
18,21,23,26,29,29,29,29,
21,23,26,29,29,29,29,29,
23,26,29,29,29,29,29,29,
26,29,29,29,29,29,29,29
Xmen3 8X8 matrices look like this:
http://i18.tinypic.com/2njipl2.png
The 4X4 luma matrices look like smaller counterparts, The chroma matrices are the same as their 4x4 luma counterparts.

[shon3i]

Indeed interesting

[Sagittaire]

and here for casino royale (really grainy movie)

Pic Scaling List[0] Present Flag: 1
delta: -2
scaling list[0][0]: 6
delta: 1
scaling list[0][1]: 7
delta: 0
scaling list[0][2]: 7
delta: 3
scaling list[0][3]: 10
delta: -3
scaling list[0][4]: 7
delta: 3
scaling list[0][5]: 10
delta: 6
scaling list[0][6]: 16
delta: -5
scaling list[0][7]: 11
delta: 0
scaling list[0][8]: 11
delta: 1
scaling list[0][9]: 12
delta: 1
scaling list[0][10]: 13
delta: -1
scaling list[0][11]: 12
delta: 5
scaling list[0][12]: 17
delta: 3
scaling list[0][13]: 20
delta: 0
scaling list[0][14]: 20
delta: -4
scaling list[0][15]: 16
Pic Scaling List[1] Present Flag: 0
Pic Scaling List[2] Present Flag: 0
Pic Scaling List[3] Present Flag: 1
delta: 2
scaling list[3][0]: 10
delta: 3
scaling list[3][1]: 13
delta: 0
scaling list[3][2]: 13
delta: 15
scaling list[3][3]: 28
delta: -14
scaling list[3][4]: 14
delta: 14
scaling list[3][5]: 28
delta: 13
scaling list[3][6]: 41
delta: -9
scaling list[3][7]: 32
delta: 0
scaling list[3][8]: 32
delta: 9
scaling list[3][9]: 41
delta: 5
scaling list[3][10]: 46
delta: -5
scaling list[3][11]: 41
delta: 43
scaling list[3][12]: 84
delta: 27
scaling list[3][13]: 111
delta: 0
scaling list[3][14]: 111
delta: -95
scaling list[3][15]: 16
Pic Scaling List[4] Present Flag: 0
Pic Scaling List[5] Present Flag: 0
Pic Scaling List[6] Present Flag: 1
delta: 1
scaling list[6][0]: 9
delta: 0
scaling list[6][1]: 9
delta: 0
scaling list[6][2]: 9
delta: 1
scaling list[6][3]: 10
delta: -1
scaling list[6][4]: 9
delta: 1
scaling list[6][5]: 10
delta: 0
scaling list[6][6]: 10
delta: 0
scaling list[6][7]: 10
delta: 0
scaling list[6][8]: 10
delta: 0
scaling list[6][9]: 10
delta: 1
scaling list[6][10]: 11
delta: -1
scaling list[6][11]: 10
delta: 0
scaling list[6][12]: 10
delta: 0
scaling list[6][13]: 10
delta: 1
scaling list[6][14]: 11
delta: 5
scaling list[6][15]: 16
delta: -5
scaling list[6][16]: 11
delta: -1
scaling list[6][17]: 10
delta: 0
scaling list[6][18]: 10
delta: 1
scaling list[6][19]: 11
delta: 1
scaling list[6][20]: 12
delta: 3
scaling list[6][21]: 15
delta: -3
scaling list[6][22]: 12
delta: -1
scaling list[6][23]: 11
delta: 0
scaling list[6][24]: 11
delta: 0
scaling list[6][25]: 11
delta: 5
scaling list[6][26]: 16
delta: 5
scaling list[6][27]: 21
delta: 8
scaling list[6][28]: 29
delta: -8
scaling list[6][29]: 21
delta: -5
scaling list[6][30]: 16
delta: -5
scaling list[6][31]: 11
delta: 0
scaling list[6][32]: 11
delta: 2
scaling list[6][33]: 13
delta: 2
scaling list[6][34]: 15
delta: 5
scaling list[6][35]: 20
delta: 0
scaling list[6][36]: 20
delta: -4
scaling list[6][37]: 16
delta: -3
scaling list[6][38]: 13
delta: -2
scaling list[6][39]: 11
delta: 6
scaling list[6][40]: 17
delta: 5
scaling list[6][41]: 22
delta: 7
scaling list[6][42]: 29
delta: 2
scaling list[6][43]: 31
delta: -8
scaling list[6][44]: 23
delta: -4
scaling list[6][45]: 19
delta: -5
scaling list[6][46]: 14
delta: 2
scaling list[6][47]: 16
delta: 5
scaling list[6][48]: 21
delta: 1
scaling list[6][49]: 22
delta: -1
scaling list[6][50]: 21
delta: -1
scaling list[6][51]: 20
delta: 5
scaling list[6][52]: 25
delta: 8
scaling list[6][53]: 33
delta: 3
scaling list[6][54]: 36
delta: -9
scaling list[6][55]: 27
delta: 0
scaling list[6][56]: 27
delta: 9
scaling list[6][57]: 36
delta: 4
scaling list[6][58]: 40
delta: -9
scaling list[6][59]: 31
delta: 9
scaling list[6][60]: 40
delta: 5
scaling list[6][61]: 45
delta: 0
scaling list[6][62]: 45
delta: -29
scaling list[6][63]: 16
Pic Scaling List[7] Present Flag: 1
delta: 4
scaling list[7][0]: 12
delta: 1
scaling list[7][1]: 13
delta: 0
scaling list[7][2]: 13
delta: 2
scaling list[7][3]: 15
delta: -2
scaling list[7][4]: 13
delta: 2
scaling list[7][5]: 15
delta: 3
scaling list[7][6]: 18
delta: -2
scaling list[7][7]: 16
delta: 0
scaling list[7][8]: 16
delta: 2
scaling list[7][9]: 18
delta: 2
scaling list[7][10]: 20
delta: -2
scaling list[7][11]: 18
delta: -1
scaling list[7][12]: 17
delta: 1
scaling list[7][13]: 18
delta: 2
scaling list[7][14]: 20
delta: 50
scaling list[7][15]: 70
delta: -50
scaling list[7][16]: 20
delta: -1
scaling list[7][17]: 19
delta: 0
scaling list[7][18]: 19
delta: 1
scaling list[7][19]: 20
delta: 13
scaling list[7][20]: 33
delta: 31
scaling list[7][21]: 64
delta: -30
scaling list[7][22]: 34
delta: -13
scaling list[7][23]: 21
delta: 0
scaling list[7][24]: 21
delta: 0
scaling list[7][25]: 21
delta: 51
scaling list[7][26]: 72
delta: 91
scaling list[7][27]: 163
delta: 92
scaling list[7][28]: 255
delta: -85
scaling list[7][29]: 170
delta: -89
scaling list[7][30]: 81
delta: -58
scaling list[7][31]: 23
delta: 0
scaling list[7][32]: 23
delta: 13
scaling list[7][33]: 36
delta: 30
scaling list[7][34]: 66
delta: 85
scaling list[7][35]: 151
delta: 4
scaling list[7][36]: 155
delta: -84
scaling list[7][37]: 71
delta: -32
scaling list[7][38]: 39
delta: -14
scaling list[7][39]: 25
delta: 71
scaling list[7][40]: 96
delta: 94
scaling list[7][41]: 190
delta: 65
scaling list[7][42]: 255
delta: 0
scaling list[7][43]: 255
delta: -27
scaling list[7][44]: 228
delta: -108
scaling list[7][45]: 120
delta: -75
scaling list[7][46]: 45
delta: 35
scaling list[7][47]: 80
delta: 89
scaling list[7][48]: 169
delta: 23
scaling list[7][49]: 192
delta: -28
scaling list[7][50]: 164
delta: 124
scaling list[7][51]: 32
delta: -33
scaling list[7][52]: 255
delta: 0
scaling list[7][53]: 255
delta: 0
scaling list[7][54]: 255
delta: 0
scaling list[7][55]: 255
delta: 0
scaling list[7][56]: 255
delta: 0
scaling list[7][57]: 255
delta: 0
scaling list[7][58]: 255
delta: 0
scaling list[7][59]: 255
delta: 0
scaling list[7][60]: 255
delta: 0
scaling list[7][61]: 255
delta: 0
scaling list[7][62]: 255
delta: 11
scaling list[7][63]: 10

[*.mp4 guy]

That matrix is the same as the one for Prestige, I didn't go through all the coeficients, but I recognized the inter 8X8 matrix, which has a very distinctive look to it, so they are definately the same.

[bond]

Casino Royale is from MGM. Using the same matrix as Disney/Buena Vista might tell that the two studios use the same encoder

[Sagittaire]

really strange matrix with the last high frequencies coef at low value ... ???

[*.mp4 guy]

It could be to preserve artificial dithering introduced by the encoder to preserve gradients, thats my best guess, atleast.

[bond]

or, its a bug :D

[akupenguin]

If only that one coefficient is small, then the only grain you can represent is a pattern that looks exactly like that one dct basis function. And basis function #63 is near zero at all the block edges, so regardless of whether it makes good grain, I don't think it can hide blocking artifacts.

[*.mp4 guy]

I thought It wasn't always near 0 around block edges? Guess I was wrong.

[Golgot13]

Hi all,

the new encoder (for this 3 movies) is BAE-VA 7000 from Sony. It can use network process encoding :)
(like PEP but much better for quality because H264....).
The price it is not nice (around 140.000USD.....). The BD were made by DADC US (may be europe).

Last best H264 encoder is come from Thomson, it use the FGT option (Film Grain Technology).
It design to do everything with best quality of encoding.

To my mind after NAB, I can say VC1 will die soon (all video hardware manufactories have H264 solution)....
I surprise to see all Sonic solution on web but not the other best solutions
( Ulead/MemoryTech, Sony BluPrint 4.0, Tiger AVC, PEP,....).
I see last version of Eclipse to test the compliance of HD DVD project but nothing (cracked on web)
about DataRius, from DCA, used by all replicator in Europe (all replicator in US use Eclipse...).

And I surprise sometime to listen strange song from all video actors:
- Some authoring studio do hacking to make BD from HD DVD source.
- Some video studio do BD on authoring company which made it for first time (lot of problems).
- Some replicator said they can do every format, but BD was not easy
(Sony Glass Mastering work and is work since more than one year;
BD replication work sure if it come from Japanese machine;....).


But the first BD without Sony process will come soon I hope it will work on all BD player
(it is a BD25 because there is only Sony which do BD50, replicator will do BD 50 next year....).

But there are good actor on market which believe at H264 since first time
to use on HD format.


I tolk about european company, I can say "Hi all" because there is lot of companies
(authoring studio, replicator, video studio, MS,...) which come to read this forum to find
"the best" informations (AACS cracking, best solution/way to encode,....).




Golgot13

[christopherw]

If the studios come here for advice and info, surely the admins could charge a consultation fee for advice dispensed which could go towards hosting? ;)

[Neo Fagin]

I'll probably try the prestige/MGM matrix out on a reencode of Layer Cake I'm doing, that movie also has the same crazy grain as Casino Royale did. If people want I'd be happy to post screenshot comparisons of the source vs a ~10mbit reencode using that matrix.

[giandrea]

I'll probably try the prestige/MGM matrix out on a reencode of Layer Cake I'm doing, that movie also has the same crazy grain as Casino Royale did. If people want I'd be happy to post screenshot comparisons of the source vs a ~10mbit reencode using that matrix.

It would be very interesting. But in a new topic. :thanks:

[McCauley]

Hi,

i think it would be more interesting to compare a 10Mbps recode with your standard HQ/Hi bitrate matrix with a recode with the MGM matrix (of cause using the same AVS script). Comparing it with the source, without an indicator how good it is in comparison to a standard matrix seems a bit pointless to me. That comparison couldn't show the advantage over a standard matrix.

Regards
McCauley

[G_M_C]

I'll probably try the prestige/MGM matrix out on a reencode of Layer Cake I'm doing, that movie also has the same crazy grain as Casino Royale did. If people want I'd be happy to post screenshot comparisons of the source vs a ~10mbit reencode using that matrix.

Ha, i see you found the thread ;)

I'll be intersted to see if it makes a significant difference keeping the quality high, without negative effects on bitrate and/or quality of high-movement parts etc. Especialy because the BR seems to be "reference quality".

[Thunderbolt8]

I'll probably try the prestige/MGM matrix out on a reencode of Layer Cake I'm doing, that movie also has the same crazy grain as Casino Royale did. If people want I'd be happy to post screenshot comparisons of the source vs a ~10mbit reencode using that matrix.
yes, screenshot comparism please, thanks

[delacroixp]

:thanks: much


:):D:eek:
Pascal

[R3Z]

Just a quick test on a videoclip thats not really grainy, but it has lots of detail;

Original
http://www.users.on.net/~inet_s51/cqms/orig1.jpg

EQM_AVC_HR
http://www.users.on.net/~inet_s51/cqms/eqmavchr1.jpg

PRESTIGE
http://www.users.on.net/~inet_s51/cqms/prestige1.jpg

Original
http://www.users.on.net/~inet_s51/cqms/orig2.jpg

EQM_AVC_HR
http://www.users.on.net/~inet_s51/cqms/eqmavchr2.jpg

PRESTIGE
http://www.users.on.net/~inet_s51/cqms/prestige2.jpg

Original
http://www.users.on.net/~inet_s51/cqms/orig3.jpg

EQM_AVC_HR
http://www.users.on.net/~inet_s51/cqms/eqmavchr3.jpg

PRESTIGE
http://www.users.on.net/~inet_s51/cqms/prestige3.jpg

Settings used;

x264.exe --crf 19 --trellis 2 --ref 3 --mixed-refs --no-fast-pskip --bframes 2 --b-pyramid --bime --weightb --direct none --filter -2,-1 --analyse all --8x8dct --threads 2 --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "pres_hh.mp4" "hh.avs" --aq-strength 0.9 --aq-sensitivity 15 --cqmfile "either prestige or eqm_avc_hr :P "

[lexor]

umm originals? Because the only difference I can detect on the guy in the street shots is slight shift of noise position, I'd like to see which one is closer to where it should be.

BTW, thank you for putting the time into this, great job.

/EDIT: thanks for adding originals.

[Thunderbolt8]

hm, since the prestige matrix has its purpose to keep the grain, maybe only use it on such grainy sources then, not sure if it does make sense on cleaner ones.

[R3Z]

hm, since the prestige matrix has its purpose to keep the grain, maybe only use it on such grainy sources then, not sure if it does make sense on cleaner ones.

The source has plenty of grain. Anyway, i was encoding this for myself and thought why not share the results. Now i know why people just dont bother.

[Thunderbolt8]

The source has plenty of grain. Anyway, i was encoding this for myself and thought why not share the results. Now i know why people just dont bother.
in your above post you said the source is not really grainy :p

[R3Z]

in your above post you said the source is not really grainy :p

Theres grainy and then theres really grainy.

[foxyshadis]

in your above post you said the source is not really grainy :p

If you saw it in motion it'd quickly be obvious how much grain is in it. In the second a lot of the noise just looks like extra detail, but it'd be moving all over in motion.

[elguaxo]

Thanks R3Z! Thunderbolt8 is like that in every forum...

The source has grain and both CQMs handle it a bit different:

EQM
http://img244.imageshack.us/img244/9623/eqmavchr1jd5.png

Prestige
http://img244.imageshack.us/img244/5085/prestige1na3.png

Two requests. Could you upload the screenshots as untouched PNGs?
And the second request, could you upload a segment of the video encoded with the 2 CQMs?


Thanks!

Edit: Last one, if uploading the samples is too much, what was the filesize using each CQM?

[R3Z]

Thanks R3Z! Thunderbolt8 is like that in every forum...

Two requests. Could you upload the screenshots as untouched PNGs?
And the second request, could you upload a segment of the video encoded with the 2 CQMs?


Thanks!

Edit: Last one, if uploading the samples is too much, what was the filesize using each CQM?

No problems, i will post some pngs or bmp's and a sample of each in the morning.

The really cool thing about the prestige cqm is that the file ended up being 113MB compared to 131MB being the EQM_AVC_HR file. The original source weighed in at 165MB.

[elguaxo]

The really cool thing about the prestige cqm is that the file ended up being 113MB compared to 131MB being the EQM_AVC_HR file. The original source weighed in at 165MB.

:eek:

Edit: a surprise the smaller filesize using the Prestige CQM

[lexor]

unexpectedly, the source is crazy blurry (for guy in the street one), but I'm actually inclined to say that the prestige matrix makes it good looking (EQM_AVC_HR and Prestige are virtually indistinguishable though). I guess there is a limit on how much a Quant Matrix can help in this case. (file size gain is very nice though)

[akupenguin]

The really cool thing about the prestige cqm is that the file ended up being 113MB compared to 131MB being the EQM_AVC_HR file.

Then the only conclusion that would mean anything is if the smaller file were also better quality. Anything else is inconclusive.
crf makes no attempt to adjust for cqm contents, any such comparison should use 2pass.

[Sagittaire]

Then the only conclusion that would mean anything is if the smaller file were also better quality. Anything else is inconclusive.
crf makes no attempt to adjust for cqm contents, any such comparison should use 2pass.

Yes like say akupenguin with your test "flat 8 CQM" will be always the best matrix. If you want make test you must have always the same bitrate ...

[Neo Fagin]

I can post some comparisons from Black Hawk Down using the prestige matrix if people want. If there's any movie with ridiculous grain, that's pretty much it. I already made a complete backup of the movie using that matrix and while I have no desire to reencode the entire thing a second time with sharktooth's, I'm willing to reencode say, the first 10 minutes of it for a comparison.

[Neo Fagin]

This is a comparison I made and posted on another forum with assorted CQMs, including the prestige one, so I assume this is on-topic. I reencoded 2000 frames out of the source, with a good mix of scenes - one long, slow bright outdoor horizontal pan, a few still shots of the actors faces, then a switch to a dark chapel wedding scene. If I had hosting I'd host the resulting MKV files from each test, they're approximately 50MB each.

I got more or less the same filesize but the prestige CQM produced the LARGEST file out of these during the tests, though not by very much. When I did Black Hawk Down using it 2-pass the target bitrate was 10671 and I ended up with 10670, obviously as the length of the encode goes up so too does the accuracy without --ratetol being used.

Movie: When Dreams May Come
Source: 7GB MPEG-2 1080i HDTV ts
x264 cef r650 build w/AQ patch
Steps:
- reencoded ts to 1280x544 huffyuv avi, YUY2 colorspace, avisynth postprocessing, resizing, sharpening and deinterlacing was done during that reencode
- created the following avs:

avisource("g:\dreams.avi")
converttoyv12()
levels(0,1,235,0,255)
trim(4000,6000)

and ran a 2-pass encode on it. first pass was turbo, second pass commandline template:

--pass 2 --bitrate 5112 --stats ".stats" --ref 5 --mixed-refs --no-fast-pskip
--bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --deblock -3:-3
--subme 7 --analyse all --8x8dct --me umh --threads auto --thread-input --cqmfile
"F:\xxxx.cfg" --progress --trellis 1 --aq-strength 0.3 --no-psnr --no-ssim --output
"dreams.mkv" "dreams.avs"

source:

http://nwgat.net/woot/files/21/compares/dreams/dreams-source_thumb.png (http://nwgat.net/woot/files/21/compares/dreams/dreams-source.png)

x264 w/mp4guy high detail 3.1 CQM, deblock -3:-1 (recommended setting for this matrix)
FRF 21.62 on first pass

http://nwgat.net/woot/files/21/compares/dreams/dreams-m4ghd_thumb.png (http://nwgat.net/woot/files/21/compares/dreams/dreams-m4ghd.png)

x264 [info]: slice I:18 Avg QP:19.89 size: 5739900:00
x264 [info]: slice P:1397 Avg QP:22.16 size: 32107
x264 [info]: slice B:586 Avg QP:23.76 size: 12809
x264 [info]: mb I I16..4: 36.2% 59.5% 4.3%
x264 [info]: mb P I16..4: 9.5% 14.6% 1.0% P16..4: 47.6% 17.0% 8.4% 0.3% 0.1% skip: 1.4%
x264 [info]: mb B I16..4: 1.5% 2.7% 0.3% B16..8: 37.9% 2.3% 5.5% direct: 4.7% skip:45.1%
x264 [info]: 8x8 transform intra:58.3% inter:40.6%
x264 [info]: direct mvs spatial:81.1% temporal:18.9%
x264 [info]: ref P 53.5% 20.4% 12.4% 7.8% 5.8%
x264 [info]: ref B 68.6% 15.4% 7.5% 5.0% 3.6%
x264 [info]: kb/s:5118.0

x264 w/prestige CQM, deblock -3:-3
FRF 24.51 on first pass

http://nwgat.net/woot/files/21/compares/dreams/dreams-prestige_thumb.png (http://nwgat.net/woot/files/21/compares/dreams/dreams-prestige.png)

x264 [info]: slice I:18 Avg QP:19.56 size: 7364500:00
x264 [info]: slice P:1397 Avg QP:22.17 size: 32309
x264 [info]: slice B:586 Avg QP:23.59 size: 12175
x264 [info]: mb I I16..4: 1.6% 97.5% 0.9%
x264 [info]: mb P I16..4: 0.0% 9.5% 0.2% P16..4: 50.5% 25.5% 12.7% 0.3% 0.1% skip: 1.2%
x264 [info]: mb B I16..4: 0.0% 1.1% 0.0% B16..8: 35.5% 2.6% 5.1% direct: 4.6% skip:51.0%
x264 [info]: 8x8 transform intra:97.7% inter:88.8%
x264 [info]: direct mvs spatial:84.1% temporal:15.9%
x264 [info]: ref P 51.0% 20.6% 13.1% 8.6% 6.6%
x264 [info]: ref B 66.3% 16.0% 8.1% 5.5% 4.1%
x264 [info]: kb/s:5137.5

x264 w/mp4guy HRM V2, deblock -3:-3, --deadzone-inter 4 --deadzone-intra 6, b-rdo turned off
FRF 26.67 on first pass

http://nwgat.net/woot/files/21/compares/dreams/dreams-hrmv2_thumb.png (http://nwgat.net/woot/files/21/compares/dreams/dreams-hrmv2.png)

x264 [info]: slice I:18 Avg QP:21.06 size: 5738200:00
x264 [info]: slice P:1397 Avg QP:23.59 size: 30199
x264 [info]: slice B:586 Avg QP:25.31 size: 17591
x264 [info]: mb I I16..4: 2.2% 97.1% 0.7%
x264 [info]: mb P I16..4: 0.1% 15.1% 0.1% P16..4: 59.0% 15.6% 9.6% 0.1% 0.1% skip: 0.2%
x264 [info]: mb B I16..4: 7.4% 11.8% 1.2% B16..8: 40.8% 1.3% 2.8% direct:33.2% skip: 1.5%
x264 [info]: 8x8 transform intra:84.8% inter:79.4%
x264 [info]: direct mvs spatial:65.0% temporal:35.0%
x264 [info]: ref P 55.4% 19.6% 12.1% 7.3% 5.6%
x264 [info]: ref B 64.9% 16.8% 8.1% 5.9% 4.3%
x264 [info]: kb/s:5131.1

x264 w/Sharktooth EQM_AVC_HR CQM, deblock -3:-3
FRF 24.84 on first pass

http://nwgat.net/woot/files/21/compares/dreams/dreams-avchr_thumb.png (http://nwgat.net/woot/files/21/compares/dreams/dreams-avchr.png)

x264 [info]: slice I:18 Avg QP:20.22 size: 6962600:00
x264 [info]: slice P:1397 Avg QP:22.88 size: 33148
x264 [info]: slice B:586 Avg QP:24.24 size: 10024
x264 [info]: mb I I16..4: 22.9% 72.0% 5.1%
x264 [info]: mb P I16..4: 1.1% 9.9% 0.3% P16..4: 55.8% 19.6% 12.4% 0.2% 0.1% skip: 0.6%
x264 [info]: mb B I16..4: 0.1% 1.2% 0.0% B16..8: 22.5% 2.2% 3.8% direct: 1.7% skip:68.4%
x264 [info]: 8x8 transform intra:86.0% inter:88.8%
x264 [info]: direct mvs spatial:84.0% temporal:16.0%
x264 [info]: ref P 54.0% 19.8% 12.4% 7.8% 6.1%
x264 [info]: ref B 70.2% 14.4% 6.7% 5.0% 3.7%
x264 [info]: kb/s:5122.1

Zoomin compares:

http://nwgat.net/woot/files/21/compares/dreams/dreams1-compares_thumb.png (http://nwgat.net/woot/files/21/compares/dreams/dreams1-compares.png)

Note the colors in the "source" screen compared to the others very slightly differ, this is because it was taken by loading the second-pass .avs in virtualdub internally, the rest were taken with MPC using VMR9.

[Sharktooth]

Im sorry to resurrect this thread but since i had noticed too much people using The Prestige matrix i wish to warn ppl to NOT USE IT.
Looking at the screenshots above, the prestige matrix creates a shitloat of artifacts (in the form of orizontal line patterns - you can see them in the last picture on the zoom compares too) in the low left area. that's not a x264 bug or whatever, it's just due to the coefficients distribution of the 8x8 matrices. So whatever encoder you'll use with that matrix, the result will be more or less the same (artifacts).

[excentro]

Then what matrix do you recommend ?

[elguaxo]

Look here:

http://forum.doom9.org/showthread.php?t=96159
http://forum.doom9.org/showthread.php?t=96298
http://forum.doom9.org/showthread.php?t=117041

[Terranigma]

IMHO from those screenshots above, EQM AVC HR does the better job at retaining the highest amount of detail without the smearing that Prestige exhibits. Why doesn't someone make tests with the flat matrix as well?

[R3Z]

Prestige from my usage hasnt given me any grief, so you cant blanket statement that the matrix screws things up. My tests in this thread show no artifacts (at least i cant see them, even after difference maps in photoshop).

Heres another set (zoomed in);

Orig
http://www.users.on.net/~inet_s51/cqms/test2/orig.jpg

Prestige
http://www.users.on.net/~inet_s51/cqms/test2/prestige.jpg

EQM
http://www.users.on.net/~inet_s51/cqms/test2/eqm.jpg

Orig2
http://www.users.on.net/~inet_s51/cqms/test2/orig2.jpg

Prestige 2
http://www.users.on.net/~inet_s51/cqms/test2/prestige2.jpg

EQM2
http://www.users.on.net/~inet_s51/cqms/test2/eqm2.jpg

Not only does prestige look great to me and retain as much detail as EQM_AVC_HR, but the filesize is pushing 15% + less.

[akupenguin]

Any setting will be transparent if given enough bitrate. It's only meaningful to ask which is better if there's a difference.

And CQM comparisons are only valid if you use the same bitrate for all candidates. You can either encode at the same bitrate and compare quality, or encode at the same quality and compare bitrate, but not both. And a same quality comparison is only possible with objective metrics; just looking at the picture isn't precise enough to attach a numerical value to a bitrate difference.

[CruNcher]

i think the problem is not allways visible only when you use higher me settings the prestige matrix can make x264 go weired and then you loss precission compared with other matrixes see the different subme results in another thread, with low settings prestige is fine. :)

[Sharktooth]

whatever is it doing, some frequencies get quantized very badly and that is causing artifacts. you may be lucky with some sources though...
also im not telling you to use my matrix, AVC-HR has a different purpouse.

[Terranigma]

sharktooth, i've meant to ask you-Did you use trellis-2 for your test? I remember getting a similar result to what you've posted in those screenshots, but with only using trellis-2 with the prestige matrix. With trellis-1, everything was fine and I was using a bitrate of 1200.

[Sharktooth]

the test was not mine

[Terranigma]

the test was not mine
Oh I see. Well, To the person who conducted the test, was trellis-2 used?

[Sharktooth]

--pass 2 --bitrate 5112 --stats ".stats" --ref 5 --mixed-refs --no-fast-pskip
--bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --deblock -3:-3
--subme 7 --analyse all --8x8dct --me umh --threads auto --thread-input --cqmfile
"F:\xxxx.cfg" --progress --trellis 1 --aq-strength 0.3 --no-psnr --no-ssim --output
"dreams.mkv" "dreams.avs"

[Terranigma]

Thanks. Well maybe using dct decimation with trellis caused some oscillation with the prestige matrix? Maybe this's what LoRd_MuldeR meant by no-dct decimate should always be used with trellis or otherwise it might conflict with it; not sure because I always use no-dct decimate with trellis. =P

[Biggiesized]

Here are the default matrices derived from CC-HDe:

#CC-HDe

INTRA4X4_LUMA =
6,15,17,20,
15,16,18,20,
17,18,19,21,
20,20,21,22

INTRA4X4_CHROMAU =
6,15,17,20,
15,16,18,20,
17,18,19,21,
20,20,21,22

INTRA4X4_CHROMAV =
6,15,17,20,
15,16,18,20,
17,18,19,21,
20,20,21,22

INTER4X4_LUMA =
9,15,17,20,
15,16,18,20,
17,18,19,21,
20,20,21,22

INTER4X4_CHROMAU =
9,14,16,17,
14,15,16,17,
16,16,17,18,
17,17,18,19

INTER4X4_CHROMAV =
9,14,16,17,
14,15,16,17,
16,16,17,18,
17,17,18,19

INTRA8X8_LUMA =
6,13,15,16,17,19,20,21,
13,14,15,16,18,19,20,21,
15,15,16,17,18,19,20,21,
16,16,17,18,19,20,21,22,
17,18,18,19,19,20,21,22,
19,19,19,20,20,21,22,23,
20,20,20,21,21,22,22,23,
21,21,21,22,22,23,23,24

INTER8X8_LUMA =
9,13,14,15,16,16,17,18,
13,13,14,15,16,16,17,18,
14,14,15,15,16,17,17,18,
15,15,15,16,16,17,18,18,
16,16,16,16,17,17,18,18,
16,16,17,17,17,18,18,19,
17,17,17,18,18,18,19,19,
18,18,18,18,18,19,19,19

8x8 Y Luma matrices:

http://upload.hattix.co.uk/uploads/cc-hde_8x8_intra_matrix.png

I didn't know if I should add them here or in the collection of available AVC custom quant matrices (http://forum.doom9.org/showthread.php?t=96159) thread. :o

[nibus]

Is there any real benefit from using custom matrices now?