Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 23rd June 2005, 13:09   #1  |  Link
Sharktooth
Mr. Sandman
 
Sharktooth's Avatar
 
Join Date: Sep 2003
Location: Haddonfield, IL
Posts: 11,768
New Custom Quantization Matrices - EQM AVC Series. Last update: EQM AVC-HR 2005-08-01

EQM AVC-HR rev.1
This matrix was made for medium-high bitrate backups with High-Profile AVC encoders that support custom matrices and 8x8 discrete cosine transform. It also reduces blocking.

Usage:
- high quality backups of DVD material at full resolution (anamorphic) and bitrates of at least 1300-1400kbps (for average motion and not noisy/old movies).
- non anamorphic/vertically resized encodes at bitrates of at least 1000kbps.

Standard version (JM, x264 & Encavc beta2-2 compatible):
Code:
INTRA4X4_LUMA =
 6, 9,13,19,
 9,14,20,27,
13,20,28,35,
19,27,35,42

INTRA4X4_CHROMAU =
 6,10,15,20,
10,16,21,27,
15,21,28,33,
20,27,33,42

INTRA4X4_CHROMAV =
 6,10,15,20,
10,16,21,27,
15,21,28,33,
20,27,33,42	

INTER4X4_LUMA =
 8,11,15,20,
11,16,21,27,
15,21,28,35,
20,27,35,42	

INTER4X4_CHROMAU =
 8,11,16,21,
11,17,22,27,
16,22,28,35,
21,27,35,42	

INTER4X4_CHROMAV =
 8,11,16,21,
11,17,22,27,
16,22,28,35,
21,27,35,42	

INTRA8X8_LUMA =
 6, 7, 8,10,12,14,16,18,
 7, 9,11,13,15,16,18,20,
 8,11,14,16,17,19,21,22,
10,13,16,18,20,22,24,26,
12,15,17,20,23,25,28,30,
14,16,19,22,25,29,34,38,
16,18,21,24,28,34,46,52,
18,20,22,26,30,38,52,72

INTER8X8_LUMA =
 8, 9,10,12,14,16,19,21,
 9,11,13,15,17,19,21,23,
10,13,16,18,20,22,23,25,
12,15,18,21,23,24,26,32,
14,17,20,23,25,27,33,40,
16,19,22,24,27,34,41,52,
19,21,23,26,33,41,53,64,
21,23,25,32,40,52,64,80
Download: EQM AVC-HR

EDIT: fixed to comply with JVT standard.

Last edited by Sharktooth; 1st August 2005 at 14:38.
Sharktooth is offline   Reply With Quote
Old 1st August 2005, 14:36   #2  |  Link
Sharktooth
Mr. Sandman
 
Sharktooth's Avatar
 
Join Date: Sep 2003
Location: Haddonfield, IL
Posts: 11,768
*** Just a bump to make you know the HR matrix is out of beta.
More matrices will follow as soon as i finish internal testing.

thanks for your tests
Sharktooth is offline   Reply With Quote
Old 1st August 2005, 17:44   #3  |  Link
skal
Registered User
 
Join Date: Jun 2003
Posts: 121
Hello Sharktooth,
could you elaborate a little on how you deduce your matrices?
("with my eyes" is ok as an answer;)
-Skal
skal is offline   Reply With Quote
Old 2nd August 2005, 01:35   #4  |  Link
trbarry
Registered User
 
trbarry's Avatar
 
Join Date: Oct 2001
Location: Gainesville FL USA
Posts: 2,092
Hi Sharktooth -

When I wanted consistantly high quality and didn't care greatly about a precise bit rate I would often use 1-pass Xvid with a constant quant of 2, but with 2 b-frames, at q=3. Varyingly this would also be with one of your custom matrices. Other options to taste but mostly using more CPU to get better compression efficiency in a single pass. I tend to store things on external USB drives so a precise size is not that important. While I dislike wasting space it's often not worth another pass just to hit a arbitrary size target if quality demands more anyway.

Can you off hand recommend any sort of fairly high bit rate X264 equivalent set of fixed quant options?

Sorry if this has been covered before but I've been away for awhile and a search didn't turn up anything with these priorities. I don't yet have much feel for what I can get out of X264 (or Nero?) with fixed quants but you seem to work with these things a bit.

- Tom

PS - These are mostly HD caps, at 720p or 544p though I don't know if it matters here.
__________________
My video filters (still) at www.trbarry.com.

Last edited by trbarry; 2nd August 2005 at 01:38.
trbarry is offline   Reply With Quote
Old 2nd August 2005, 19:11   #5  |  Link
Sharktooth
Mr. Sandman
 
Sharktooth's Avatar
 
Join Date: Sep 2003
Location: Haddonfield, IL
Posts: 11,768
Quote:
Originally Posted by skal
Hello Sharktooth,
could you elaborate a little on how you deduce your matrices?
("with my eyes" is ok as an answer
-Skal
well with h264 i had to re-elaborate some of my theories on human eye perception but generally i "draw" a matrix around my initial idea, then i do a set of tests with a set (10 or more) of matrices (usually those are scaled from the "master" matrix with different factors and scaling formula) on 5 different sources.
Then i analyze the source sequencies and for every sequence i assign a score.
After the analysis phase i sum the scores and choose the matrix with the higest one.
This method was pretty fast for xvid, but since AVC is more complex it requires a lot more time...

Quote:
Originally Posted by trbarry
Hi Sharktooth -

When I wanted consistantly high quality and didn't care greatly about a precise bit rate I would often use 1-pass Xvid with a constant quant of 2, but with 2 b-frames, at q=3. Varyingly this would also be with one of your custom matrices. Other options to taste but mostly using more CPU to get better compression efficiency in a single pass. I tend to store things on external USB drives so a precise size is not that important. While I dislike wasting space it's often not worth another pass just to hit a arbitrary size target if quality demands more anyway.

Can you off hand recom
Quote:
Originally Posted by trbarry
Hi Sharktooth -

When I wanted consistantly high quality and didn't care greatly about a precise bit rate I would often use 1-pass Xvid with a constant quant of 2, but with 2 b-frames, at q=3. Varyingly this would also be with one of your custom matrices. Other options to taste but mostly using more CPU to get better compression efficiency in a single pass. I tend to store things on external USB drives so a precise size is not that important. While I dislike wasting space it's often not worth another pass just to hit a arbitrary size target if quality demands more anyway.

Can you off hand recommend any sort of fairly high bit rate X264 equivalent set of fixed quant options?

Sorry if this has been covered before but I've been away for awhile and a search didn't turn up anything with these priorities. I don't yet have much feel for what I can get out of X264 (or Nero?) with fixed quants but you seem to work with these things a bit.

- Tom

PS - These are mostly HD caps, at 720p or 544p though I don't know if it matters here.
Well, you can try with quants from 20 to 24, and you should have the "equivalent" of q2 (or q2+Bs) for xvid (depending on the CQM you used).mend any sort of fairly high bit rate X264 equivalent set of fixed quant options?

Sorry if this has been covered before but I've been away for awhile and a search didn't turn up anything with these priorities. I don't yet have much feel for what I can get out of X264 (or Nero?) with fixed quants but you seem to work with these things a bit.

- Tom

PS - These are mostly HD caps, at 720p or 544p though I don't know if it matters here.
Well, you can try with quants from 20 to 24, and you should have the "equivalent" of q2 (or q2+Bs) for xvid (depending on the CQM you used).

Last edited by Sharktooth; 2nd August 2005 at 19:14.
Sharktooth is offline   Reply With Quote
Old 2nd August 2005, 21:32   #6  |  Link
trbarry
Registered User
 
trbarry's Avatar
 
Join Date: Oct 2001
Location: Gainesville FL USA
Posts: 2,092
Thx. That give me a useful place to start.

- Tom
__________________
My video filters (still) at www.trbarry.com.
trbarry is offline   Reply With Quote
Old 3rd August 2005, 22:34   #7  |  Link
jellysandwich
Registered User
 
Join Date: Mar 2004
Posts: 247
Is there a way to use custom quant matrices
with x264 vfw?

Edit: Also, is there one similar to MPEG for Xvid?
I like the sharpness that MPEG gives...

js

Last edited by jellysandwich; 4th August 2005 at 00:15.
jellysandwich is offline   Reply With Quote
Old 4th August 2005, 09:12   #8  |  Link
yaz
n00b ever
 
Join Date: May 2002
Posts: 627
Quote:
Originally Posted by Sharktooth
... you can try with quants from 20 to 24, and you should have the "equivalent" of q2 (or q2+Bs) for xvid (depending on the CQM you used).
according to my (very incomplete) tests x264 cq equivalent to xvid q2 is somewhat lower. it's somewhere q16-18. (xvid matrix: mpeg, x264 matrix: default, bvop(xvid):1/1/0)
Quote:
Originally Posted by jellysandwich
is there one similar to MPEG for Xvid?
I like the sharpness that MPEG gives...
i would love that too. it's good to see that i'm not the very last fan (nerd?) of that matrix

the bests
y

Last edited by yaz; 4th August 2005 at 09:17.
yaz is offline   Reply With Quote
Old 5th August 2005, 15:24   #9  |  Link
Sharktooth
Mr. Sandman
 
Sharktooth's Avatar
 
Join Date: Sep 2003
Location: Haddonfield, IL
Posts: 11,768
MPEG matrix is different from (and more bitrate hungry than) h263. The values i suggested are intender for a h263-like quant2 compression, also i specified those values depends on custom matrices (in both xvid and x264).
Sharktooth is offline   Reply With Quote
Old 17th August 2005, 21:39   #10  |  Link
TheBashar
Registered User
 
TheBashar's Avatar
 
Join Date: Jan 2002
Posts: 112
Bitrate Variance Question

Hi Sharktooth,

I understand that your AVC matrices are intended for bitrates of around 1350kbps for 720x430 "average motion" movies. From following some of your XviD matrices development, I believe I recall that the matrices are "tuned" for the target bitrate/resolution ratio and perform sub-optimally on lower bitrate/resolution ratios.

Getting (slowly) to my question... will the matrices perform sub-optimally in regions of an encode where due to a default curve compression (qcomp = 0.6 in x264) bitrates are reduced because of slow/no motion detection? For instance, I may have an encode which averages 1350kbps but a particular scene may have only 800 kbps allocated due to it being a static scene. Will these matrices perform poorly in such regions solely becuase of the low bitrate or will they still perform as expected since in fact there is little or no motion?

The reason I ask is because I am considering unbridling the qcomp by significantly raising the ratetol from its default of 1.0.

Thanks for your time!
TheBashar is offline   Reply With Quote
Old 18th August 2005, 13:18   #11  |  Link
Sharktooth
Mr. Sandman
 
Sharktooth's Avatar
 
Join Date: Sep 2003
Location: Haddonfield, IL
Posts: 11,768
Matrices are tuned for a certain compressibility. In regions where the codec assigns less bit, the compressibility is usually higher. So, in static scenes, even if the bitrate gets lower, it doesnt mean the codec is using higher quantizers. It's usually the opposite... However when i made AVC-HR i took into account the possibility to set the bitrate variability from "1" to "inf" and the HR matrix should behave correctly (at least in my tests it proved to do it).
It's all a matter of coefficients scaling... for example if the scaling is too "harsh" it will produce artifacts at lower bitrates (too much difference between quants).

Last edited by Sharktooth; 18th August 2005 at 13:42.
Sharktooth is offline   Reply With Quote
Old 9th September 2005, 10:31   #12  |  Link
TheBashar
Registered User
 
TheBashar's Avatar
 
Join Date: Jan 2002
Posts: 112
ColorMatrix

Hi Sharktooth,

I really like your CQM. When I was trying to determing a good bitrate before I used your CQM, I had trouble with blocky dark backgrounds even at what I thought were too high bitrates. When I finally switched to your CQM, I found the blockyness was dramatically reduced. I'm guessing your CQM gives more bitrate in the dark color range than the standard one does.

So, all was well in the world. I have always been frustrated by my encodes being a tad too dark though. I recently stumbled onto this 9-04 thread about ColorMatrix, which corrects that (or the real problem rather). I tried it and in fact it really improved things. So, I started a reencode. Unfortunately, after my first reencode, I found the blockyness was back. Now, there are other differences as well: x264 cli rev 291B instead of 287A, substituted limitedsharpen instead of xsharpen.

I've had a hard time finding an old 287a x264 rrev to test with, so I thought I post here and ask if you think the ColorMatrix change could be responsible. Is there a chance that you tuned your CQM for the incorrect color spectrum that ColorMatrix is meant to correct? Not accusing or anything, just asking so I can narrow down the permutations I need to test.

Thanks!
TheBashar is offline   Reply With Quote
Old 9th September 2005, 13:06   #13  |  Link
Sharktooth
Mr. Sandman
 
Sharktooth's Avatar
 
Join Date: Sep 2003
Location: Haddonfield, IL
Posts: 11,768
Well, limitedsharpen may be the cause. Sharpen filters usually lower the source compressibility in a significative manner. In this situation a lower bitrate matrix is needed but you can try rising the deblocking filter threshold by 1 or 2 to help compensate the compressibility loss.
However i dont use colormatrix coz i never experienced darkened or desaturated encodes so IMHO it's not a "tuning" problem.
Sharktooth is offline   Reply With Quote
Old 9th September 2005, 14:02   #14  |  Link
Chainmax
Huh?
 
Chainmax's Avatar
 
Join Date: Sep 2003
Location: Uruguay
Posts: 3,103
This matrix will come in handy on my next encodes. I'm doing AVC backups of some high motion cartoon captures @1100kbps and 640x480 resolution, and with this matrix I might even be able to disable deblocking. Once I solve some issues with my current filterchain (SSIQ and aWarpSharp are causing green splotches on the sides of the picture) I'll try this and report back.

Sharktooth, how do you expect customs matrices to affect playback compatibility on future AVC capable standalones?
Chainmax is offline   Reply With Quote
Old 9th September 2005, 14:53   #15  |  Link
Sharktooth
Mr. Sandman
 
Sharktooth's Avatar
 
Join Date: Sep 2003
Location: Haddonfield, IL
Posts: 11,768
Disabling deblocking is always a bad idea unless you're near saturation.
However, custom matrices are part of high profile and it will be played back only if the SAP will support it.
Sharktooth is offline   Reply With Quote
Old 31st October 2005, 16:26   #16  |  Link
Daodan
unrecognized user
 
Join Date: Oct 2005
Location: home of Stella Artois
Posts: 303
Nice thing this matrix, exactly fit for my needs. One question though. I started encoding a movie (140 min movie) with 1800 kbps that I just finished encoding without a custom matrix. The 'problem' is, that the encode is 17 % faster now then before (it's only 30% completed now but the speed increase is obvious). I use the same settings as before so I could compare the results. This is something unexpected I would say.
Is it normal or did my computer start taking drugs?
Well, it just finished and it took 12 hours instead of 17.
__________________
zzz

Last edited by Daodan; 2nd November 2005 at 14:59.
Daodan is offline   Reply With Quote
Old 31st October 2005, 19:06   #17  |  Link
Manao
Registered User
 
Join Date: Jan 2002
Location: France
Posts: 2,856
Quote:
Disabling deblocking is always a bad idea unless you're near saturation.
Deblocking gets disable automatically a long time before reaching saturation ( saturation for h264 is around ~20 mbits for D1, it can't be reached ).

Supporting CQM isn't complicated at all. It doesn't slow down the encoding process, so there's no objective reason not to support it. But now, I'm not a chip maker...

Edit : encoding with CQM doesn't slow down the encoding process.
__________________

Last edited by Manao; 31st October 2005 at 19:16.
Manao is offline   Reply With Quote
Old 31st October 2005, 20:22   #18  |  Link
Kopernikus
Registered User
 
Join Date: Sep 2002
Posts: 125
x264 has different optimized routines for quantizing. if a cqm has low entries a slightly slower routine ist used to prevent overflow.
Kopernikus is offline   Reply With Quote
Old 31st October 2005, 20:33   #19  |  Link
Manao
Registered User
 
Join Date: Jan 2002
Location: France
Posts: 2,856
But only if the quantizer gets really low. And low entries means higher bitrates --> high quantizers at same bitrates, so in the overall it compensates one another ( well, almost at least )
__________________
Manao is offline   Reply With Quote
Old 31st October 2005, 20:35   #20  |  Link
Sharktooth
Mr. Sandman
 
Sharktooth's Avatar
 
Join Date: Sep 2003
Location: Haddonfield, IL
Posts: 11,768
Quote:
Originally Posted by Manao
Deblocking gets disable automatically a long time before reaching saturation ( saturation for h264 is around ~20 mbits for D1, it can't be reached ).
sorry, wrong term... i meant "close to transparency" or a sufficient quality/bitrate to have blocks go away...

Last edited by Sharktooth; 31st October 2005 at 20:39.
Sharktooth is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 10:49.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.