Custom Matrices

[Sharktooth]

no... xvid has h.263 (default quantization for MPEG4 ASP) and MPEG (default MPEG matrix used in previous MPEG encoder standards).

[rkalwaitis]

So the codec assumes that the information taken from the film is the same for each dvd regardless of what it is? You have to base and this base/set of numbers/stream whatever you call it can be changed via a matrix. So what are the numbers the codec sees before the matrix changes them (I think that is the a better way to ask), and if so how do you see them or know what they are ???

[Sharktooth]

quantization matrices work on frequencies and the matrix coefficients represents how much those frequencies are quantized.
you better :search: and read some docs coz it may be a veeery long discussion.

[rkalwaitis]

Ranguvar, you gonna fix snow for us :)

[rkalwaitis]

okay didnt mean to drag it out.
Thanks guys

[Ranguvar]

Ranguvar, you gonna fix snow for us :)

Um :confused:

I wish I had even the bare minimum of the skills needed to do such a thing, but sadly, I do not.

Besides, Snow/Dirac, while very interesting, do not show incredible promise IMO. H.264 still has more potential, and from what I've heard, the benefits of using *lets aren't big enough to justify moving development. By the time H.264 is nearing the end of it's life, I expect there will be odder things to ooh and aah about :)

[rkalwaitis]

MP4

"--deadzone-inter 6 --deadzone-intra 6" applicable with your High Detail matrix? AQ Strength ? 1.3?

Does anyone have issues with the flat matrix with 264 having a fuzzy look at times?

Thanks
Robert

[Sharktooth]

why not? i'd suggest to use psy-rdo and trellis though...

[rkalwaitis]

how do i turn on psy-rdo???
Looks like my trellis is on its set to 2.

program --bitrate 700 --ref 5 --mixed-refs --bframes 4 --b-pyramid --b-rdo --bime --weightb --filter -3,0 --subme 6 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --merange 24 --threads auto --thread-input --aq-strength 1.3 --cqmfile "C:\Users\Baba-Nator\profiles\x264\matrix\M4G_High_Detail_V3.1.cfg" --progress --no-psnr --no-ssim --output "output" "input" --deadzone-inter 6 --deadzone-intra 6

Do I add it to the command line in MeGUi?

Teach me Sharktooth :)

[Sharktooth]

it's enabled by default, if you want to rise it's strenght just add --psy-rd with a value higher than 1 in the custom commandline options.
also if you use trellis you cant use deadzones.
There are two different quantization algorithms: deadzone and trellis. Any given dct of a given block can only use one of them. Because, well, if you had two different values for each coefficient, what would you do with them?
--deadzone-inter and --deadzone-intra tweak a parameter of the deadzone algorithm. (The parameter in question is "deadzone", that's where the algorithm got its name.) --trellis determines when to use the trellis algorithm (0=never, 1=sometimes, 2=almost always); any dcts that don't use trellis must use deadzone instead.
So, the fact that --deadzone* still has some minor effect when you enable trellis does not contradict the fact that deadzone and trellis are incompatible.
Don't do that. If you enable trellis, then deadzone is only used as a fast estimate of trellis, so a reduced value of --deadzone does not provide any grain retention, it just reduces the precision of that estimate and thus of the mode decision process. Sure I could make x264 ignore --deadzone* if --trellis is enabled, but that would contradict my policy of "give the luser enough rope to hang himself".

That said, it might be possible to tweak trellis in a way that has similar results to --deadzone*. In which case --deadzone* could apply to trellis too, despite the name.

BTW, just because an option is printed in the userdata sei doesn't mean it had any effect on the encode. I remove some of the extraneous options (e.g. most of ratecontrol when using cqp), but still some options may be printed even if they're overridden by some other option.

[rkalwaitis]

Thanks Sharktooth, You Da Man

[rkalwaitis]

I guess its good to know that we are afforded enough rope to hang ourselves, that is awfully generous of akupenguin.

[Pitou]

Quick question for Mp4Guy,

For the M4G HRM V2 matrix, you suggest to:

B predict mode should be set to none if your source is very grainy(and you wan't it to stay that way), or has many low contrast areas

Are you referring to the "direct=auto|none|temporal|spatial" option of x264?

Thanks.

Pitou!

[*.mp4 guy]

Yes, "direct=auto|none|temporal|spatial" should be set to "none".

[Codex0nz]

Well im looking to get the most quality out of my x264 encodes, mainly anime, i have a lot of confidence in your ability mp4 guy

Who can recommend what matrix to use for dvd sourced anime encoding?

Or is there not much point in loading one of these up for anime encodes

Also if you use one of these matrixes do you turn b frame mode to none?

Sorry, trying to learn -_-

[*.mp4 guy]

There are very few situations where using a cqm on anime would be a good idea. Unless you have something very specific you are trying to achieve, you are safest with the flat matrix.

[Codex0nz]

There are very few situations where using a cqm on anime would be a good idea. Unless you have something very specific you are trying to achieve, you are safest with the flat matrix.

ah well that solves my concerns :) thanks dude

too bad nothing like this can fix the bloody avatar rips (still trying -_- lol)

I thought using these would improve the quality of what x264 would save etc but guess i was wrong (or just not in this situation)

[LoRd_MuldeR]

The most reasons why people used custom matrices in the past are now obsolete, thanks to VAQ and Psy-RDO.

Furthermore VAQ and Psy-RDO don't work as expected with certain custom matrices...

[*.mp4 guy]

No, they work as expected, the results just aren't good. This is an oversimplification, but that is basically it.

[LoRd_MuldeR]

...the results just aren't good.

That's what I meant. Bad results isn't what I expect.

[Audionut]

For x264 with all the tuning done based on the flat matrix, it makes sense to now remove custom matrix support.

[*.mp4 guy]

It never makes sense to remove support for something when it costs you nothing to leave it intact. Besides, almost all of x264, including the psychovisual's work exactly as they should with cqm's. Ugly skipped mv's remain the only real "mistake" a matrix will induce x264 to make, the other unwanted behavior is mearly the result of transients being butchered by matrixes removing their highest energy components more aggressively, VAQ quantizing reqions with transients more aggressively, psy trellis also quantizing transients more aggressively, and psy-rd de-tuning rd decisions in order to make the picture more closely resemble the local complexity of the source by introducing "artifacts" (which is usually bad for transients).

Matrices and x264's psy enhancements aren't incompatible, or even bad to use together, you just have to be carefull to avoid overdoing the transient hatred. I should also probably update this thread and my deblocking FAQ to take into account that with VAQ 0:0 is usually the best deblocking setting, even with matrices.

[Sagittaire]

Matrices and x264's psy enhancements aren't incompatible, or even bad to use together, you just have to be carefull to avoid overdoing the transient hatred. I should also probably update this thread and my deblocking FAQ to take into account that with VAQ 0:0 is usually the best deblocking setting, even with matrices.

No ...

[*.mp4 guy]

No ...

In my experience your disagreement, despite its vagueness would apear to indicate that I am correct.

[Sagittaire]

In my experience your disagreement, despite its vagueness would apear to indicate that I am correct.

Simply because with CQM you can't never make generalisation. In fact IMO the best solution is not use CQM with psy optimisation simply because psy tools are optimised for flat16 matrix.


I should also probably update this thread and my deblocking FAQ to take into account that with VAQ 0:0 is usually the best deblocking setting, even with matrices.

here for example it's completely false. Alpha and beta for inloop effect are highly dependant of Matrix. Absurd example is flat8 vs flat32 matrix. If 0,0 produce best result with flat16 then 0,0 will be certainely not best setting with agressive or soft matrix for global coef.


IMO Matrix are simply useless now with x264 simply because psy tool have same objective with more efficiency result. If you want really use CQM the better way now is really soft modification with all coef (with flat16 ref) like that for example:

INTRA4X4_LUMA =
12,12,14,16,
12,14,16,20,
14,16,20,25,
16,20,25,32

INTRA4X4_CHROMAU =
12,12,14,16,
12,14,16,20,
14,16,20,25,
16,20,25,32

INTRA4X4_CHROMAV =
12,12,14,16,
12,14,16,20,
14,16,20,25,
16,20,25,32

INTER4X4_LUMA =
14,14,15,16,
14,15,16,17,
15,16,17,18,
16,17,18,19

INTER4X4_CHROMAU =
14,14,15,16,
14,15,16,17,
15,16,17,18,
16,17,18,19

INTER4X4_CHROMAV =
14,14,15,16,
14,15,16,17,
15,16,17,18,
16,17,18,19

INTRA8X8_LUMA =
12,12,12,12,13,14,15,16,
12,12,12,13,14,15,16,17,
12,12,13,14,15,16,17,19,
12,13,14,15,16,18,20,22,
13,14,15,16,18,21,23,27,
14,15,16,18,21,24,28,37,
15,16,17,20,23,28,39,48,
16,17,19,22,27,37,48,68

INTER8X8_LUMA =
14,14,14,14,15,15,15,16,
14,14,14,14,15,15,16,17,
14,14,14,15,15,16,17,18,
14,14,15,15,16,17,18,19,
15,15,15,16,17,18,19,21,
15,15,16,17,18,20,22,26,
15,16,17,18,19,22,27,32,
16,17,18,19,21,26,32,44

[Audionut]

If you want really use CQM the better way now is really soft modification with all coef (with flat16 ref) like that for example:

Which is exactly as mp4 guy said.

the other unwanted behavior is mearly the result of transients being butchered by matrixes removing their highest energy components more aggressively,

I've created some matrices that have increased metrics, but visually have never been as good as the flat matrix.

I always use psy optimisation and AQ though, even when testing matrices. I've never done a test against my matrices and DS's psy ops.

Because I'm fairly certain I know who would lose. And it doesn't make sense to create a matrix that requires features of the encoder to be disabled.

[Sagittaire]

Which is exactly as mp4 guy said.


well it's definitively not the case for all the mp4 guy matrix.


I've created some matrices that have increased metrics, but visually have never been as good as the flat matrix.

CQM don't increase metric.


I always use psy optimisation and AQ though, even when testing matrices. I've never done a test against my matrices and DS's psy ops.

Simply because CQM and x264 psy x264 tools don't make the same thing. CQM is just here to have a little better HVS macroblock look with more optimal gradiant/detail tradeoff. CQM can't help like AQ for better overall picture quality or can't help like SSD for better complexity preservation. Anyway you can use CQM with all the actual x264 psy tools but you must simply use really soft CQM matrix close to flat16 matrix.

[*.mp4 guy]

LRM and HRM still work the same as ever. Smooth works to an extent, MRM is mostly pointless, and High Detail causes too much ringing.