New Quantization Matrices - EQM V3 series (last update: 04/02/2005 - V3ULR)

[Sharktooth]

foxyshadis got it right...

i must say ULR is no longer a good matrix... at least on LCD displays coz they have an ultra-high contrast that will make ULR "sharpening-effect" look ugly and artificial.

[G_M_C]

K guys thx for the input. Ill post the 2 matrices I normally choose from, maybe "you as experts" can see if there OK or point me to better ones ;)

[henryho_hk]

Sharktooth, would you recommend any alternative to ULR?

[Sharktooth]

V3LR if you want to keep details or jawors 1cd matrix if you want a smooth output.

[Teegedeck]

Hm, LR doesn't reach quite the compression you get from from ULR, I think. It's almost playing in the same league as HR. Perhaps one shouldn't use XviD at all for compression so strong that ULR or Javor's 1CD-matrix are needed. :)

XviD's output at the low end cannot compare with x264 but I think the illusion of crispness that ULR provides is a neat trick. Do you really like Javor's 1CD better?

[Sharktooth]

Well, LR is not so bad at high quants. However, yes, the ULR crispness IS a trick but since some time, LCD monitors have impossible contrast ratios (600:1 or more) ... so the "trick" is no longer good...

[CrashUK]

Well, LR is not so bad at high quants. However, yes, the ULR crispness IS a trick but since some time, LCD monitors have impossible contrast ratios (600:1 or more) ... so the "trick" is no longer good...

what do you think is best matrix for LCD screens..

[Sharktooth]

V3LR or Jawor's 1CD should be fine at low bitrates. For higher bitrates there should be no problems.

[halsboss]

Description:
This is an EXTREMELY HIGH BITRATE matrix designed for constant quality encodings ONLY. ... It's application is pretty obvious: it's for backing up original material or HD material without using lossless codecs. EQM V3EHR:

8 8 9 11 13 13 14 17
8 8 11 12 13 14 17 18
9 11 13 13 14 17 17 16
11 11 13 13 13 17 18 20
11 13 13 13 16 17 20 24
13 13 13 16 17 20 24 29
13 12 13 17 19 23 28 34
12 13 17 19 23 28 34 41

8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8
8 8 8 8 8 8 8 8

WARNING: It may not work with some standalones (expecially the ones with ESS chipsets). EDIT: Matrix updated (uhm... completely redesigned) on 05/10/2004.

Some time ago I came across this "Fox matrix" and seek your comment in relation to it and EQM V3EHR... likely pro's and cons (xvid->DVD using HC)

08 08 09 09 10 10 11 11
08 09 09 10 10 11 11 12
09 09 10 10 11 11 12 12
09 10 10 11 11 12 13 13
10 10 11 11 12 13 13 14
10 11 11 12 13 13 14 15
11 11 12 13 13 14 15 15
11 12 12 13 14 15 15 16

08 08 09 09 10 10 11 11
08 09 09 10 10 11 11 12
09 09 10 10 11 11 12 12
09 10 10 11 11 12 13 13
10 10 11 11 12 13 13 14
10 11 11 12 13 13 14 15
11 11 12 13 13 14 15 15
11 12 12 13 14 15 15 16


Hmm, http://forum.doom9.org/showthread.php?p=748689#post748689 says the Fox Home Entertainment Matrix is

08 08 09 11 13 13 14 17
08 08 11 12 13 14 17 18
09 11 13 13 14 17 17 19
11 11 13 13 13 17 18 20
11 13 13 13 16 17 20 24
13 13 13 16 17 20 24 29
13 12 13 17 19 23 28 34
12 13 17 19 23 28 34 41

08 08 08 09 09 09 09 10
08 08 09 09 09 09 10 10
08 09 09 09 09 10 10 10
09 09 09 09 10 10 10 10
09 09 09 10 10 10 10 11
09 09 10 10 10 10 11 11
09 10 10 10 10 11 11 11
10 10 10 10 11 11 11 11

... anyone know so which is the latest "real" Fox Matrix ?
Edit 2: Searching finally located it...
http://forum.doom9.org/showthread.php?p=866287#post866287 says of the 1st of these 2 Fox matrices Fox 2 now comes in HC 018 encoder zip which you can get from doom9's download section...
http://forum.doom9.org/showthread.php?p=759321#post759321 was the original source for me and said

Here's what Fox DVDs use these days, in case you're interested:

[Sharro]

Hi Everybody and Sharktooth,

Sorry to bump in here, I've read the whole thread and didn't find anything that could kill my curiosity.

I've been using hvs-best for quite a few years (damn I'm getting old) and I'm willing to test at least one of your cqm but I need some directions.

Basically I encode at animations (pixar, dreamworks, etc) which compress quite well at bitrates > 1300kbps and resolutions >= 640, lanczosresize and ocasionaly Undot, unfilter or lumafilter.

Which of your cqms should I start with ?

EDIT: Hmppfff, read this thread...didn't read the other... I should definetly start with HR and LR. Any known compatibility problems with standalones ?

Any help appreciated.

All the best,

Sharro

[HeadBangeR77]

From my limited experience, I would say you should give v3HR a try - for my needs it smooths a bit too much, but for your content should be just fine (considering the bitrate & resolution you've given). It is still less compresable than HVS-Best Picture, though the difference is rather small, and depends of course on the content you encode.

Why don't you try some sample clips, one pass, constant quantizer (2 or 3)? I've done hundreds of samples like that and it prooved useful for future encodings. :)

Hope I've helped a bit, cheers
HDBR77

[Sharktooth]

HR may have standalone compatibility issues.
To fix them just replace all "15"s with "16"s in the inter-matrix.

[Sharro]

...for my needs it smooths a bit too much...still less compresable than HVS-Best Picture, though the difference is rather small...

Why don't you try some sample clips, one pass, constant quantizer (2 or 3)? I've done hundreds of samples like that and it prooved useful for future encodings. :)
HDBR77

The HVS-Best/SharktoothCQM difference in size is exactly one of my curiosities. :D

I never encode a movie without 2 or 3 tries, I'll run a few first passes...

HR may have standalone compatibility issues.
To fix them just replace all "15"s with "16"s in the inter-matrix.

Thanks for the info.

EDIT:

Just tried a 2 pass encode with didee's 6of9 to test in my standalones together with Heini's (with inter-matrix 15's replaced by 16's), results:

Philips DVP720SA/02
===Heini's Intermatrix Min 16 -> perfect play
===Didee's SixOfNive hvs -> artifacts on video (blocks)

Philips DVP5960/12
===Heini's Intermatrix Min 16 -> perfect play
===Didee's SixOfNive hvs -> perfect play

Video was 704x288 pixels Xvid 2BF, QP, PB, Chroma Optimizer, Trellis with 1593Kbps set for the second pass

So for the time being I'll have to stick with Heini's IM Min 16. I'll also try it on a Yamada 6700 plus and revert.

END EDIT

[HeadBangeR77]

HR may have standalone compatibility issues.
To fix them just replace all "15"s with "16"s in the inter-matrix.
Thank you, good to know that. :)

@ Sharro
Sample clip form "The Curse of the Black Pearl", NTSC 23.976, 672x288, 12951 frames:

1) MSP=5, VHQ=1, VHQ for B-frames, Chroma ME, Turbo, Trellis, Chroma opt., B-frames 2/1.62/0 (goes very quickly, good for testing CQMs ;))
a) MPEG default QM @ Q2 : 101 MB, 1564 kbps
b) HVS Best Picture @ Q2 : 102 MB, 1588 kbps
c) Sharktooth's V3 HR @ Q2 : 118 MB, 1829 kbps
d) Heini's MR (V3 mod) @ Q2 : 120 MB, 1865 kbps

2) MSP=6, VHQ=3, VHQ for B-frames, Chroma ME, no Turbo, QPEL, Trellis, Chroma opt., B-frames 2/1.62/0 (slooow in comparison to the above ;))
a) MPEG default QM @ Q2 : 106 MB, 1651 kbps
b) HVS Best Picture @ Q2 : 108 MB, 1677 kbps
c) Sharktooth's V3 HR @ Q2 : 123 MB, 1915 kbps
d) Heini's MR (V3 mod) @ Q2 : 126 MB, 1955 kbps

To give Sharktooth his credits, the V3 LR one gave at quantizer 2:
93.5 MB (1448 kbps) and 98.3 MB (1522 kbps) for the above sample-encodes, what places it somewhere in between H.263 and standard MPEG.

[Sharro]

Well...after 2 different full 2 pass encodes on animations...where checking against the 1st pass size of HeadBanger77 I also got to 14 to 15% difference against HVS-Best, so slightly worst quant average on the second pass... I've thrown away the numbers and looked at the visual quality by doing a stack vertical...

Sharktooth :thanks:.

Much more detail retained and overall better looking, certainly a great option for 2 cd rips of animation with 2 AC3 tracks.

Now I have just to test in my standalone.

I'll revert.

All the best,

Sharro

[Sharro]

Hi Guys,

No problems with Standalone Philips DVP5960/12 using HR matrix after following Sharktooth inter-matrix advice.

Errr...as I don't like encoding, for fun I'm redoing my encodes with Colormatrix, damn...Xvid at full...that's going Slllloooowwww.

I've noticed a difference of "brightness" expected amongst different CQM's, specially in red color, do my eyes betray me for staring too much at different scenes ?

All the best,

Sharro

[BigDid]

Hi Guys,

No problems with Standalone Philips DVP5960/12 using HR matrix after following Sharktooth inter-matrix advice.

Errr...as I don't like encoding, for fun I'm redoing my encodes with Colormatrix, damn...Xvid at full...that's going Slllloooowwww.

I've noticed a difference of "brightness" expected amongst different CQM's, specially in red color, do my eyes betray me for staring too much at different scenes ?

All the best,

Sharro
Hi,

While you are comparing you may want to try the V3HR modded by Heini a few posts up in this page. I had good results with it and same tip applies for SAP compatibility ( replace all "15" and below "15" with "16" in the inter-matrix): http://forum.doom9.org/showthread.php?p=854722#post854722

Did

[HeadBangeR77]

While you are comparing you may want to try the V3HR modded by Heini a few posts up in this page. I had good results with it and same tip applies for SAP compatibility ( replace all "15" and below "15" with "16" in the inter-matrix): http://forum.doom9.org/showthread.php?p=854722#post854722
That's one of my favourites! Just slightly less compressable than Sharktooth's original one, and gives more crispy, sharp look, at least IMHO ;) I will edit my post above to add it to the list.

@ Sharro
Good to know the trick works for standalones :)
As to the brightness, I haven't marked anything like that, but I encode too rare to spot such a difference. Are you sure you haven't switched rendering mode in your fav player or output color space?

Red is always tricky - that's why I don't watch anything in YV12, unless I have to. Some terrible color-bleeding there, brrrr!

cheers,
HDBR77

[Sharro]

That's one of my favourites! ...@ Sharro
Good to know the trick works for standalones :)
As to the brightness, I haven't marked anything like that, but I encode too rare to spot such a difference. Are you sure you haven't switched rendering mode in your fav player or output color space?

Red is always tricky - that's why I don't watch anything in YV12, unless I have to. Some terrible color-bleeding there, brrrr!

cheers,
HDBR77

My doubts come from a stackvertical of the d2v with either hvs or v3hr but the "brightness" efect was more noticeable on v3hr.

I didn't had time to check this nights encode (HR with Colormatrix(0,2,d2v="***.d2v",threads=2) as it might have helped the color-bleeding.

I'll roll a few encodes within today and tomorrow and I'll certainly give a try to Heini's V3HR mod.

All the best,

Sharro

[HeadBangeR77]

OT:

1) Watch the same by forcing HQ YV12=>RGB32 conversion or just force YUY2 colorspace.
2) I've tried Colormatrix and I'm not happy with its results, with all the respect to the creator. First, it oversaturates red and human faces, imo. Second, it is forcing tv-scale (not full colors' range) by default, what I don't like especially (washed-out colors, dimmed image). Just a few ideas :rolleyes:

[Sharro]

OT:

...
2) I've tried Colormatrix and I'm not happy with its results, with all the respect to the creator. First, it oversaturates red and human faces, imo. Second, it is forcing tv-scale (not full colors' range) by default, what I don't like especially (washed-out colors, dimmed image). Just a few ideas :rolleyes:

If I well understood the principle of Colormatrix and if used properly it shouldn't give you the above problems, it should just feed the encoder with the colormatrix he's expecting (601) against the (mainly) standard colormatrix of mpeg2 (BT709).

I've compared colors of the mpeg2source and avisource with converttorgb and definetly the encodes with colormatrix had better color matching.

Maybe it could be a mistake of my eyes.

All the best,

Sharro

[Beave]

If the source is not bt709 you will get wrong colors with colormatrix of course.
I actually noticed the wrong colors myself a few years back and happiliy found colormatrix what got the solution to this problem. That was before autogk or megui were using them as standard.

[Vindarath]

is there any 'all purpose cqm'? im encoding hitman with the high rate cqm at cq2, 704w400h. but im getting bitrates below 1600.

[Ranguvar]

is there any 'all purpose cqm'? im encoding hitman with the high rate cqm at cq2, 704w400h. but im getting bitrates below 1600.
Try encoding a sample EQM V3LR, MPEG, and possibly EQM V3HR, and see which looks better to you.

[Lenny_Nero]

is there any 'all purpose cqm'?
I have been thinking about this question since it was posted, and because I do a large amount of DVB-TV to xvid encodes (5~10 a day) I have been looking for the best all-rounder to save me re-encoding things over and over.

Well just as there is no 'best' there is not really a cqm that will work right with everything, for everybody, because what I think looks good others will not.

But so far I have found that, again, this is for me (I like a sharper picture and always used mpeg over H.263) I have been very impressed with the "EQM - AutoGK sharp.xcm" cqm and so far (over 100 encodes) its working right every time and has yet to have a problem with any SAP (nothing below 16 in the inter-matrix).

But I would also do as Ranguvar says and at least try "EQM V3LR, MPEG, and possibly EQM V3HR" as well because they are top quality cqm's.

[Ranguvar]

The all-purpose CQM is now Xivd with the VAQ patch, using H.263 (technically not a CQM) with Adaptive Quantization enabled ;)

[shae]

The all-purpose CQM is now Xivd with the VAQ patch, using H.263 (technically not a CQM) with Adaptive Quantization enabled ;)Wouldn't that still result in a soft image?

[Ranguvar]

In most situations, nope. Perhaps in extremes.

[Lenny_Nero]

Wouldn't that still result in a soft image?
Thats how I would have thought it, or are you saying that now with the VAQ patch you dont need to do anything with CQM/QM's ?

[Sharktooth]

consider VAQ as a sort of "adaprive CQM" that even works on standalones.
If i have to choose between encoding with a CQM or encoding with VAQ, ill go for VAQ first. If and only if im not satisfied then i would use a CQM.

[Video Master]

consider VAQ as a sort of "adaprive CQM" that even works on standalones.
If i have to choose between encoding with a CQM or encoding with VAQ, ill go for VAQ first. If and only if im not satisfied then i would use a CQM.

I too prefer H.263 + AQ over CQMs , they work best for me.

Also i am using Xvid 1.1.2 [xvidcore.dll] Koepis Vanilla CVS Build , Nov 1 2006 specifically.

[Sharktooth]

xvid 1.1.2 AQ is not VAQ... and it sux... :)

[Video Master]

xvid 1.1.2 AQ is not VAQ... and it sux... :)

Point noted :)

[Tuik]

Hello,

What Custom Matrix should I use in SD content with a 2pass bitrate of 2250 kbps? The aim is the best sharpeness possible.

Thank you for the help.

[Jawor]

EQM v3 HR preserves detail pretty well (and works OK with VAQ). Qpel can help a lot as well (if our target player supports it). It's the resizing from 1920x1080 to 720xXXX that will cause a significant loss of detail (of course we have no other choice if we want to play our encodes on “DivX Home Theater” certified players).

[elguaxo]

Read the sticky (http://forum.doom9.org/showthread.php?t=119399) ;)

Here (http://forum.doom9.org/showthread.php?t=93539) you have a thread with compressibility tests using different CQMs and Jawor made a useful comment on compatibility here (http://forum.doom9.org/showthread.php?p=1421926#post1421926).

The higher bitrate matrices keep more details, but they also need more bitrate.


edit: lol, the post with the question I was commenting to disappeared.

[Jawor]

Here (http://forum.doom9.org/showthread.php?t=93539) you have a thread with compressibility tests using different CQMs and Jawor made a useful comment on compatibility here (http://forum.doom9.org/showthread.php?p=1421926#post1421926).

About the min=16 requirement for the INTER part... If you change the 15s to 16s in “EQM v3 HR”, the quality difference won't be really noticeable (I tried this ;) ). MediaTek-based players may require such a change. My Philips DVP3142/12 (Zoran Vaddis 966 chipset) plays every custom matrix I can throw at it. Very old players (e.g. based on the ESS Vibratto chipset) probably won't play any of the “MPEG-Custom” matrices.

[Tuik]

EQM v3 HR preserves detail pretty well (and works OK with VAQ). Qpel can help a lot as well (if our target player supports it). It's the resizing from 1920x1080 to 720xXXX that will cause a significant loss of detail (of course we have no other choice if we want to play our encodes on “DivX Home Theater” certified players).


It's ok to use a custom matrix and VAQ at the same time? And how do i use VAQ in xvid_encraw?

[Sharktooth]

not all quant matrices are VAQ friendly.
test them on a short clip and see if they're behaving correctly.

[henryho_hk]

Copy an updated xvidcore.dll to the same folder as xvid_encraw. Then you can use the lumi-masking option to turn on VAQ.

[SilaSurfer]

Hey Sharktooth very nice collection with EQM series. My favorite is Eqm V3HR modded for standalone support. I was just wondering is Trellis bug fixed or do I have to use these two workarounds?

1. Disable Trellis
or
2. Restrict quantizers to 2-31

I'm using latest Jawors Xvid 1.22 VAQ version.

[Tuik]

Hey,

Right now i'm using xvid-encraw from megui development server.

The aim is to backup bluray content to view in players which not support it.

So i'm encoding bluray to xvid HD 720p and bluray to xvid SD 480p.

I load to xvid a small avisynth script with DSS2, spline16 resize and a little bit of sharpen.

I've been in research for the best xvid options to retain the most detail without blocking and without loosing standalone player compatibility. The above taking in consideration that 480p is to be encoding at a max of 2000 avg bitrate and 720p at a max of 3000 avg bitrate.

The settings i'm using now are:

2 pass, 2 max b frames, bframe ratio 162, bframe offset 0, VHQ for bframes and mode 4, MSP 6, no QPEL, no GMC, Chroma Motion, Closed GOP, Trellis Quant, Adapt. Quantization, Packed Bitstream, No Turbo, max kframe in 240, min qt 2 max qt 31

So now what is missing is the custom matrix...

What CQM would you advise with all the above set?

(also if you suggestions on other used settings please don't abstain :))

Thanks for your help!

[henryho_hk]

1. Disable Trellis

Isn't the Trellis bug fixed years ago? And the min. Q should always be set 2 or higher.

[SilaSurfer]

And the min. Q should always be set 2 or higher.

I know that. I misunderstood what Sharktooth meant by his workarounds on the first page of this thread. The bug came when you would use Trellis + EqmV3HR matrix with quantizer ranges Min 1-31 Max but I never use that so. I thought that restricting max quantizers would show those issues. Thanks and I apologize for taking your time.

[Tuik]

Hey,

Right now i'm using xvid-encraw from megui development server.

The aim is to backup bluray content to view in players which not support it.

So i'm encoding bluray to xvid HD 720p and bluray to xvid SD 480p.

I load to xvid a small avisynth script with DSS2, spline16 resize and a little bit of sharpen.

I've been in research for the best xvid options to retain the most detail without blocking and without loosing standalone player compatibility. The above taking in consideration that 480p is to be encoding at a max of 2000 avg bitrate and 720p at a max of 3000 avg bitrate.

The settings i'm using now are:

2 pass, 2 max b frames, bframe ratio 162, bframe offset 0, VHQ for bframes and mode 4, MSP 6, no QPEL, no GMC, Chroma Motion, Closed GOP, Trellis Quant, Adapt. Quantization, Packed Bitstream, No Turbo, max kframe in 240, min qt 2 max qt 31

So now what is missing is the custom matrix...

What CQM would you advise with all the above set?

(also if you suggestions on other used settings please don't abstain :))

Thanks for your help!
Any help? :)

[henryho_hk]

Is it 720p or 480p @60fps or 30fps?

If you use a good denoiser, QPel can help 480p encodes. Avoid the old AQ.... grab a new xvidcore.dll and use the new VAQ if u need to.
I seem to have read somewhere for best standalone player compatibility, we should use max one consecutive B-frame with "Packed Bitstream" on. If it is a newer player supporting 2 max cons. B-frame, turn off "Packed Bitstream". QPel + 2-max-cons-B saves over 10% bitrate (for more movie details).

max key-frame interval is still 10*fps. But for 60fps encodes I think we may reduce it to 5~7*fps.

H263 with new VAQ offers the best standalone player compatibility and it encodes fastest too. If u want more details, u can use EQM-v2 or EQM-HR-v3-SAP-mod. Make sure that u do have sufficient bitrate because VAQ makes them more prone to blocks and rings for frame peepers. (Actually, H263+VAQ tends to give running textures @ insufficient bitrates too.)

[Tuik]

Is it 720p or 480p @60fps or 30fps?

If you use a good denoiser, QPel can help 480p encodes. Avoid the old AQ.... grab a new xvidcore.dll and use the new VAQ if u need to.
I seem to have read somewhere for best standalone player compatibility, we should use max one consecutive B-frame with "Packed Bitstream" on. If it is a newer player supporting 2 max cons. B-frame, turn off "Packed Bitstream". QPel + 2-max-cons-B saves over 10% bitrate (for more movie details).

max key-frame interval is still 10*fps. But for 60fps encodes I think we may reduce it to 5~7*fps.

H263 with new VAQ offers the best standalone player compatibility and it encodes fastest too. If u want more details, u can use EQM-v2 or EQM-HR-v3-SAP-mod. Make sure that u do have sufficient bitrate because VAQ makes them more prone to blocks and rings for frame peepers. (Actually, H263+VAQ tends to give running textures @ insufficient bitrates too.)

Encodes are @24fps

Well, my standalone player don't support QPEL so i cannot use it.

Hum.. could you explain me what really packed bitstream does? Because, until now, i've always used it because virtualdubmod doesn't show image if i turn packed bitstream off.

Well, for now, i've been using sixofnine HVS with VAQ and it seems to have good results, however, i can't find those matrices you said. Would you help me? Thanks a lot

[henryho_hk]

EQM matrices? Take a look at Sharktooth (http://forum.doom9.org/member.php?u=41310)'s signature.

[crasus]

I think he meant the EQM-HR-v3-SAP-mod (which doesn't seem to in Sharktooth's signature) ?

[henryho_hk]

I see. EQM-HR-v3-SAP-mod is to use CQE (http://forum.doom9.org/showthread.php?t=73666) to modify inter-frame matrix of EQM-HR-v3 setting all <16 coefficients to 16.