New Quantization Matrices - EQM V3 series (last update: 04/02/2005 - V3ULR)

[Teegedeck]

Note: Answered to that in your 'undersize-oversize'-thread, trying to keep your special problem separated from the general discussion about EQM matrices.

[PatchWorKs]

From Afronic DVD Guides' FU forum:

I once tried to include some custom matrices in FU, but this resulted in a lot of feedback from people experiencing bad display on their standalone divx players, and I had to remove (well, actually it's only disabled in the code) the custom matrices handling.

Is this true ?

Check the original 3ad (http://www.dvd-guides.com/component/option,com_simpleboard/Itemid,61/func,view/id,8228/catid,26/)

[Sharktooth]

There are definatly some "problematic" matrices.
Not all decoders (hardware or software...) are able to handle, for example, matrices that have inter-matrix coefficients below 16.

P.S.: Links to the custom matrices are working again.

Edit: @patchworks, are you registered in that forum? if so, please inform them the link to the CQMs (both soulhunter's and mine) are changed and the celtic_druid files are now @ http://celticdruid.no-ip.com/xvid/

[Sharktooth]

EQM V3ULR has been updated.

[IgorC]

Looks like new REV3 ULR has less ringing

XVID 1.1 last beta1
REV3
http://img236.exs.cx/img236/5772/rev36oi.th.jpg (http://img236.exs.cx/my.php?loc=img236&image=rev36oi.jpg)

REV2

http://img236.exs.cx/img236/1332/rev20ri.th.jpg (http://img236.exs.cx/my.php?loc=img236&image=rev20ri.jpg)

[CruNcher]

Looks like new REV3 ULR has less ringing


sounds interesting but if it's now more like h263 it could be also bad
let's test it :)

[Sharktooth]

No, it's still v3ulr :)
h263 is completely different.

[unplugged]

Hello, is there any sticky-like thread talking about matrices and its findings?
Shark, I see your signature with some downloads... where I can find some instant info/comment on these matrices? (their names are too cryptic)
I have been far from this msgboard for long time...
Thanks!

[Sharktooth]

Infos on my CMQs are on the first page of this thread.
There is no sticky for CQMs, if you need more info you have to use the forum search function.

[unplugged]

Thank you very much, I thought 1st page was only part and/or first experiments (in fact, right now I see that posts are updated).
Well, I think to give a try with EQM V3ULR...

[Sharktooth]

I always update the original posts so updates can be easily found.
Once you're done with testing ULR please comment it here.
Thanks :)

[unplugged]

720x304 2-hour movie anamorphic ready, for 630 MB target size (Video only)
Ultra LR or LR ?

1st full-quality pass constant quant 3 (final titles q16) with ECQ V3ULR is 600 MB
(bframe sensitivity -12), I will add also TemporalSoften(5,2,2,15,2)

What you mean when say "V3ULR isn't HVS tuned"?

[Sharktooth]

Are you satisfied with the final quality? Q3 should be good.
Originally posted by unplugged
What you mean when say "V3ULR isn't HVS tuned"?
I mean it is not built starting from the IEEE HVS matrix.

[unplugged]

It went really good, very little artifacts, don't realize how... but crisp too.
During play I see much mosquito noise in medium/low light scenes, but under VirtualDub all seem ok.
Good work, V3ULR seems to have the pluses from H.263 and from MPEG scheme (crisp response). :)
Source is only vertically lanczos-resized, quant 3 result is yet awesome.

[Sharktooth]

Originally posted by unplugged
During play I see much mosquito noise in medium/low light scenes, but under VirtualDub all seem ok.
Check the brightness control in the decoder. Maybe it's too high.

[CrashUK]

I did a full dvd res anamorphic.. using the EQM V3ULR matrix..

its 2.2 gig with 5.1 ac3 sound

video bitrate 1700Kbps
sound bitrate 448kbps

Adaptive Quantixation Off
GMC Off
Qpixel Off
Packed bitstream Off
2 BVOPS

VHQ Mode Wide Search
Use chroma motion

9 hours to encodec

[Sharktooth]

And? Does it please you?
The video bitrate seems a bit high for V3ULR...

[niamh]

I finally decided to cheat on Didée :D , and have a try of your matrices. Out of laziness, is there a single zip package available somewhere? would be nice to download/extract the lot in one go :)

[Sharktooth]

http://www.webalice.it/f.corriga/CQMs/CQMs.rar

EDIT: Updated link

[Audionut]

Originally posted by Sharktooth
I mean it is not built starting from the IEEE HVS matrix.

Where can 1 find this matrix?

[Sharktooth]

http://forum.doom9.org/showthread.php?s=&threadid=33499

[CrashUK]

Originally posted by Sharktooth
And? Does it please you?
The video bitrate seems a bit high for V3ULR...

Yes it does.. v3ulr is a good around matrix..

[Poutnik]

I like ULR matrix. Can I use it with MTK profile
with min quants=2 for HW players (No GMC,No QP, 1 BVOP, packed bs)?
( There are 2 intra coef. < 16 )

If yes, should I disable trellis too ?

[Sharktooth]

Yes, it should work perfectly.
Trellis quantization is done during encoding and does not iterfere with decoding.

[Poutnik]

I thought trellis does some local quant changes, similar to AQ, but in different way and for different reason.
So I was not sure if trellis set a quant for a block to 1,
causing player unhappy.....
Maybe it respects quant limitation.

BTW, why some low coef matrices works and other fails ?

[Poutnik]

Originally posted by Sharktooth
There are definatly some "problematic" matrices.
Not all decoders (hardware or software...) are able to handle, for example, matrices that have inter-matrix coefficients below 16. It seems ULR rev3 matrix is HW player friendly, because inter coefficients are NOT below 16, even if intra ones ARE ?
(there are two 14 ) I always thought under16 can be dangerous in intra matrix too...

[Sharktooth]

The big problems usually shows when INTER matrix coefficients < 16.
Intra matrix starts always with a 8...
You may want to search for trellis on the web for more details.

[Poutnik]

I know about the 8. This is constant by principle. I had in mind AC coeffs. As DCT works, there is possible range of DCT coeffs 0-2048 for DC and -2048 - +2048 for AC ones.
By dividing/rounding by matrix coeff 8(DC) or 16(AC) and quant 1 we get for both 0-255 (signed byte or 128 offset for AC ).
If there is sub16 matrix, quant 1 and high DCT coeff, codec is
in a trouble.....

Very high AC coeff are fortunately unusual.

There is possibility such trouble occurs for inter matrix
just because it is used more often (P,B) then intra (I) ?

I will have a look for trellis, of course..

[Sharktooth]

I never experienced problems with intra matrix coefficients below 16 (and obviously > 8). The decoders limitations were always for inter matrix.
BTW, i always try to avoid quant 1, so maybe i didnt observe the behaviour @ that quant with matrices with coefficients < 16.
Once one of my high bitrate matrices triggered an overflow in trellis but that was promptly fixed by devs...

There is possibility such trouble occurs for inter matrix
just because it is used more often (P,B) then intra (I) ?
No, infact if you use a high bitrate matrix (V3HR for example) and force I-frame Quants @ 1 there will be no troubles (except the filesize...).

[Poutnik]

Thank you, Sharktooth, for patiency, all your replies and not least
your matrices ( ULR and LR are my favourite, work good, look good :) )

[Xpa]

Hi, this is an excinting thread and i have some questions regarding with a movie that i have to compress:
The Source: The Incredibles DVD (Very clean source, inexistent noise)

My idea is to compress this movie splitted in 3 cds (mode 2, 795MB for each) with AC3 AUDIO, and Bitrate about 2400Kbits. (the splitting process is not so dificult, using VirtualDubMod and SubRip for divide sub, i have some practice)
Codec that i use: xvid last koepi bins.

I beleve that the proper matrix to use is EQM V3HR, It is correct?
But, can i use this other (EQM V3UHR) instead?...maybe this other matrix will give me all the quality posible, allways watching the oversize that will generate....

Why 3 cds do you ask? Because my idea is to make 2 versions, one, a ultra-high quality compression (3cds) and a 1 cd backup version.

And a final question,what matrix is best suitable for tv capures (noisy and very noisy sources)?

Regards.

[Sharktooth]

2400kbps for UHR is a bit too low, but since the movie you're going to encode is very compressible i suggest you to try both matrices on a small clip (5 minutes maybe) and see which one looks better.
However, in normal (common) situations i would have choosen HR too :)
For what concerns video capture, well i didnt make any specific matrices but my experience says you need a much higher bitrate than normal and a mid-high bitrate matrix to obtain good results.
Or you could filter all the noise out with avisynth filters and use whatever matrix you want depending on the final filesize and resolution you wanna achieve.

[Xpa]

Well thanks Sharktooth!, i have finished the encodig for 3cds :
Xvid AC3
1:55 dur.
Using the matrix EQM-V3HR is quite formidable, with an extremaly
high comprensibility (about 100%) with a resolution of 808x303 (if i down de res the comp goes far beyond 110%... usless).
The Script:

LoadPlugin("C:\ARCHIV~1\ARCALC~1\dgmpgdec\DGDecode.dll")
LoadPlugin("C:\ARCHIV~1\ARCALC~1\AVS_Plugins\FluxSmooth.dll")
Mpeg2Source("E:\DVDS\VOBMOV~1\MOVIE2~1.D2V")
Crop(0,60,-0,-62)
Lumafilter()
FluxsmoothST(5,3)
Lanczos4Resize(808,330)

And the image quality is clear, perfect, totally block free (it is obviusly at this bitrate isn't?). Maybe its perfect to do a DVD from this xvid in the future, i think.
The size of the video was perfect too (restricting the quantizer 1-3, 1-4, 1-5 , I-Frames,P-Frames,B-Frames) and compressing the final titles (quant 24), thats why i loves xvid! :)

Well, thxs again for the guide Sharktooth, and now, 4 cds?.... no! :)
Instead i will buy a DVD.... and compress in 3cds or 1...

[Sharktooth]

If your compressibility test goes over 100% you can safely do 2 things:
1 - Encode anamorphically (select the correct AR in the codec, crop the black borders on top and bottom and do not resize - basically a 1:1 copy in terms of resolution).
2 - Keep the horizonal res to 720 (and scale vertical res to the corresponding value) and use a higher bitrate matrix (for example UHR or 6of9).
Both methods will produce a higer quality encode.

[Xpa]

Sharktooth, but if i keep 720 x something (whithout black bars) res, the codec may undersize the final size, maybe compensating with a more restricted quantz or more I-Frame Boost (more than 75 actually).
But a bigger resoluttion doesn't give me more quality and sharpess? (over 720x???)
Or the very high res in this case is not the top priority for max quality encoding?
Regars.

[Sharktooth]

Going anamorphic will keep the vertical res to the original value.
Let's say you have a 2.35:1 AR movie to compress.
Cropping the black bars (from the original 720x576 res) results in a 720x432 picture with an incorrect aspect ratio.
Generally to get it right the 432 is resized to 308 (nearest mod16 value) so you obtain a 720*308 picture that looks with the correct AR.
Doing this you're loosing pixels in favor of a correct looking picture.
There's another way (anamorphic encode) to get the correct AR without loosing pixels and almost all DVDs movies use this technique to NOT loose definition.
The video stream contains a "field" where the AR is stored and the decoder, reads the stream and resizes the picture during playback.
That means, watching your movie on a PC monitor (with a higher resolution than a standard TV) or on a widescreen or HD TV will result in a better quality coz you're encode has more pixels than the 720x308 encode.
Resizing over the original res is quite useless coz you're not adding "quality" but interpolated pixels only.
Resizing to 1024x432 (for the correct AR) is an overkill, coz you mantain the original horizontal res but you're adding 304 pixels to the vertical res that contains no significative (and maybe incorrect) data.
Sharktooth, but if i keep 720 x something (whithout black bars) res, the codec may undersize the final size, maybe compensating with a more restricted quantz or more I-Frame Boost (more than 75 actually).
That's why, in that case, i suggested to use a higher bitrate matrix.

[Xpa]

The anamorphic encode is not supported in xvid i guess (in the codec config explains that), so the only option is to resize the height only...
But doing this is better than resize from 720x432 to 808x330...
In fact, using xvid , there was an other option (similar to anamorphic encodig):
Encodig to 720x358 (wichout black bars and no resizing).
The AR in this case will be 2,011. Now, dependig of the software to use, the video will be resized acording to the especified ar, for example: in bsplayer, if i set the video to 235:100, the ar is corrected, but the original is 2,011.
Do yo recomend me this other "method" to encode without resizing and loosing pixels but wrong AR (corrected by the software i use to reproduce it)?

[IgorC]

Xpa, read private message ;)

[Sharktooth]

Originally posted by Xpa
The anamorphic encode is not supported in xvid i guess (in the codec config explains that), so the only option is to resize the height only...
What?!? Where does it explain that?
But doing this is better than resize from 720x432 to 808x330...
In fact, using xvid , there was an other option (similar to anamorphic encodig):
Encodig to 720x358 (wichout black bars and no resizing).
The AR in this case will be 2,011. Now, dependig of the software to use, the video will be resized acording to the especified ar, for example: in bsplayer, if i set the video to 235:100, the ar is corrected, but the original is 2,011.
It's not similar... it IS anamorphic encoding...
Do yo recomend me this other "method" to encode without resizing and loosing pixels but wrong AR (corrected by the software i use to reproduce it)?
Yes, unless the software or hardware player doesnt support it.

[Xpa]

What?!? Where does it explain that?
XviD Configuration -> Aspect Ratio:
"...Aspect Ratio is written to mpeg-4 bit stream,
but infortunately is likely to be ignored if video stream is encapsulated in a general-purpose container (lke .avi, .ogm, .mkv)
Therfore, be aware that using different aspect ratio than default might be ignored by some players, especially when decoded on Windows.
USE AT YOUR OWN RISK.

Maybe i expressed wrong with some things or repetitive....
Well, at this point i have no doubt to do with this video.

Thanks for your time (and patience ;) ) responding me with this matter.

[Sharktooth]

Originally posted by Xpa
XviD Configuration -> Aspect Ratio:
"...Aspect Ratio is written to mpeg-4 bit stream,
but infortunately is likely to be ignored if video stream is encapsulated in a general-purpose container (lke .avi, .ogm, .mkv)
Therfore, be aware that using different aspect ratio than default might be ignored by some players, especially when decoded on Windows.
USE AT YOUR OWN RISK.

Maybe i expressed wrong with some things or repetitive....
Well, it does not say "Xvid does not support anamorphic encoding"... coz this is anamorphing encoding:)

Well, at this point i have no doubt to do with this video.

Thanks for your time (and patience ;) ) responding me with this matter.
np

[dungweaver]

Just reporting in excellent results with eqm_v3ulr_rev3 in day to day usage.

Currently using Q=4 single pass with AQ off and QR(2,31,2,31,2,31).

Source = digital TV (SD or PD) mpeg2, resized to 592x336.
Gspot reports a video of 485 kbps. Qf of 0.098. Image quality is excellent.

This was a documentary with a fair bit of action. Nearly 28 minutes of show into only 124 MB.

Sharktooth does this surprise you? Anyone else getting results like this? I am just a wee bit sceptical; seems too good to be true.

[bugmenotwillyou]

Sharktooth I've just tried your EQM V3HR and the results were great for a 1800kbit encode. But I'm just wondering if the Trellis bug still exists (the one with the two workarounds).
If I use one of your workarounds either disabling Trellis Quantization OR limit the MinQ and MaxQ to 2-31 respectively, wouldn't I lose quality in the process.
I always have kept Trellis Quantization checked since I was under the assumption it gave a quality boost.

EDIT: Forgot to mention that I'am using Xvid 1.10 b2 04042005

[Didée]

If I use one of your workarounds either disabling Trellis Quantization OR limit the MinQ and MaxQ to 2-31 respectively, wouldn't I lose quality in the process.
Limiting quantizers to 2/31 is not a workaround. It is the recommended setting.

In fact, the default of 1/31 is a workaround. (For XviD newbies whining about undersized encodes.)

You won't loose quality by not using quant 1 in a 2-pass scenario. However by disabling trellis, you probably would - but only a tiny bit.

[Sharktooth]

@dungweaver: well, ULR stands for a ultra low rates. however nice to see it's good for you too :)

@bugmenotwillyou: i already answered you by PM, however, as you can see, Didée said the same thing.

[bugmenotwillyou]

Thanks for the prompt replies guys.
The V3HR CQM is perfect then for my encodes.

[Teegedeck]

Sharktooth I've just tried your EQM V3HR and the results were great for a 1800kbit encode. But I'm just wondering if the Trellis bug still exists (the one with the two workarounds).
[...]
I always have kept Trellis Quantization checked since I was under the assumption it gave a quality boost.
Nope, no Trellis bug anywhere in sight. :) And with a matrix like v3UHR or SixOfNine at 1800 kbps for full-res-encodes I am quite positive that Trellis actually is a huge quality-advantage in a two-pass scenario.

Not sure about how it summs up with v3HR and whatever resolution you use.

[bugmenotwillyou]

Nope, no Trellis bug anywhere in sight. :) And with a matrix like v3UHR or SixOfNine at 1800 kbps for full-res-encodes I am quite positive that Trellis actually is a huge quality-advantage in a two-pass scenario.

Not sure about how it summs up with v3HR and whatever resolution you use.
Yeah your post in another thread about Trellis bug being "fixed months ago" was the one that prompted me to ask that question of whether the Trellis bug still existed for Sharktooth's CQMs, so kudos to you.

The EQM v3UHR that you suggested would be too much for my 1800kbps encodes for my 704xXXX anamorphic encodes, no? I usually like to fit 2 movies on 1 DVDR so 1800kbps to 2100kbps would be in the range of EQM v3HR, I would have thought?

I had the impression that EQM v3UHR was for ~2500kbps-4000kpbs (full-res or HD encodes) and EQM v3HR was for 1600kpbs-~2500kbps (assuming using full-res or slightly lower) or is my thinking flawed?

[Sharktooth]

Well, it all depends on source resolution, framerate, compressibility etc.
However in general 1600-2300 is a good range for HR, 1300-1900 for LR, 2300+ UHR.
I may join HR and LR in one matrix in the near future... :)

[Beave]

What would be your suggestions for HDTV encodes to 960x528 @ 1500-2000 bitrates? The source is as usually noisier and blockier than DVD sources. I find 6of9=24 not too suitable for this task. Would the ULR matrix fit the profile? I like a sharp picture, but I dislike artifacts from too low bitrates.