Custom Matrices

[*.mp4 guy]

[edit] I was going to leave this post as it was, but it has been recently bumped to the front page. Currently the cases that these cqms were meant to fix are handled very well by psychovisual optimizations within X264, such as Dark Shikari's aq, psy rd, and psy trellis, which usually work better then cqms, in my (and everyone elses) opinion. Cqms can still be usefull, but not most of the time, and certainly not for the majority of people.

This thread is to provide a place for me to keep up to date versions of my Mpeg4-avc cqms, and somewhere to refer people who want copies of them. If you have any questions about these matrices in particular feel free to ask them here; however I would apriciate it if more general questions (stuff about matrices in general, etc.) were kept out of this thread, to keep it succinct. Some matrices used in commercial HD-DVD/Blu-Ray encodes can be found in another thread (http://forum.doom9.org/showthread.php?t=124755).

With the addition of the deadzone parameter there is yet another value that needs to be tweaked to get the best results, I will specify any deadzone settings that i think are neccesary to get the most out of a matrix by posting the command line, IE "--deadzone-inter 6 --deadzone-intra 6". Megui now has gui options for changing the deadzone settings, reffered to as luma intra deadzone, and luma inter deadzone.

M4G LRM V4: LRM is for low to very low bitrates, 1cd or less, and MRM may even be better for some 1cd encodes at this point. It is designed to avoid artefacts more then to preserve detail. 0:0 for smooth encodes, -3;0 for moderate encodes.
INTRA4X4_LUMA =
8,22,32,47,
22,48,55,57,
32,55,63,69,
47,57,69,84

INTRA4X4_CHROMAU =
16,20,34,64,
20,40,62,78,
34,62,98,138,
64,78,138,168

INTRA4X4_CHROMAV =
16,20,34,64,
20,40,62,78,
34,62,98,138,
64,78,138,168

INTER4X4_LUMA =
12,28,28,32,
28,32,35,38,
28,35,42,46,
32,38,46,55

INTER4X4_CHROMAU =
18,24,34,41,
24,40,48,56,
34,48,64,92,
41,56,92,110

INTER4X4_CHROMAV =
18,24,34,41,
24,40,48,56,
34,48,64,92,
41,56,92,110

INTRA8X8_LUMA =
8,11,10,16,24,28,29,32,
12,13,14,19,26,27,29,33,
14,13,16,24,28,29,32,35,
14,17,22,30,32,34,38,39,
18,22,28,32,37,42,45,47,
24,27,29,34,42,49,55,57,
29,29,32,38,45,55,63,69,
32,33,35,39,47,57,69,84

INTER8X8_LUMA =
13,14,16,18,20,21,22,24,
14,15,18,19,20,21,22,25,
16,18,20,21,22,23,25,27,
18,19,21,23,25,26,29,31,
20,20,22,25,28,31,35,38,
21,21,23,26,31,37,42,48,
22,22,25,29,35,42,51,61,
24,25,27,31,38,48,61,76
M4G MRM V2: MRM is supposed to work well at moderate bitrates (and sometimes does well even at low and high bitrates, though it isn't suited for transparency). It should give a good balance of details and artefacts, it looks similar to HVS best. 0:-1 for smooth results (not recommended) -1:-2 for moderate sharpness, -2:-4 for high sharpness. should be used with "--deadzone-inter 6 --deadzone-intra 6" to get a more detailed picture, at the cost of a small amount of ringing.
INTRA4X4_LUMA =
10,10,20,30,
10,10,34,33,
20,34,52,76,
30,33,76,112

INTRA4X4_CHROMAU =
8,8,24,72,
8,8,41,144,
24,41,96,200,
72,144,200,255

INTRA4X4_CHROMAV =
8,8,24,72,
8,8,41,144,
24,41,96,200,
72,144,200,255

INTER4X4_LUMA =
10,18,20,20,
18,18,34,44,
20,34,52,76,
20,44,76,112

INTER4X4_CHROMAU =
16,22,22,22,
22,22,22,56,
22,22,72,96,
22,56,96,128

INTER4X4_CHROMAV =
16,22,22,22,
22,22,22,56,
22,22,72,96,
22,56,96,128

INTRA8X8_LUMA =
12,12,14,20,24,26,30,32,
12,12,15,22,24,27,30,36,
14,15,20,26,27,31,34,40,
20,22,26,30,35,38,40,46,
24,24,27,35,42,48,50,52,
26,27,31,38,48,56,60,64,
30,30,34,40,50,60,72,80,
32,36,40,46,52,64,80,96

INTER8X8_LUMA =
12,16,16,18,20,22,25,28,
16,16,17,20,20,23,26,28,
16,17,20,22,24,26,29,31,
18,20,22,25,28,31,33,38,
20,20,24,28,33,37,43,49,
22,23,26,31,37,46,56,65,
25,26,29,33,42,56,70,88,
28,28,31,38,49,65,88,115
M4G HRM V2: HRM is for high to very high bitrates depending upon your source, however the new version scales quite well so if you really wanted to you could use it at any bitrate without problems. For transparency it should be used with the "--deadzone-inter 4 --deadzone-intra 6" switches added to the commandline, without the quotes. since it is designed for transparency, there shouldn't be any noticible properties it introduces to the picture. B predict mode should be set to none if your source is very grainy(and you wan't it to stay that way), or has many low contrast areas, otherwise b-rdo should be disabled and b prediction should be set to auto. I would apriciate feedback on any sources that cannot reach transparency with this matrix, as I have run into a few, but cannot currently find a solution.
INTRA4X4_LUMA =
15,6,15,22,
6,7,25,30,
15,25,38,48,
22,30,48,96

INTRA4X4_CHROMAU =
16,6,24,72,
6,8,41,144,
24,41,96,200,
72,144,200,255

INTRA4X4_CHROMAV =
16,6,24,72,
6,8,41,144,
24,41,96,200,
72,144,200,255

INTER4X4_LUMA =
4,11,15,22,
11,11,25,30,
15,25,38,48,
22,30,48,96

INTER4X4_CHROMAU =
5,16,16,16,
16,16,16,56,
16,16,72,96,
16,56,96,128

INTER4X4_CHROMAV =
5,16,16,16,
16,16,16,56,
16,16,72,96,
16,56,96,128

INTRA8X8_LUMA =
24,7,7,12,15,20,21,23,
8,9,14,15,18,21,23,21,
10,12,15,17,17,21,26,20,
14,15,17,17,18,33,30,23,
15,17,17,21,26,39,39,29,
18,18,20,24,33,39,41,35,
21,24,29,34,39,45,45,38,
27,35,36,39,38,38,38,38

INTER8X8_LUMA =
6,10,13,13,15,16,18,22,
10,10,12,15,15,16,21,24,
13,12,15,16,19,21,25,27,
13,15,16,21,24,28,30,34,
15,15,19,24,30,36,37,40,
16,16,21,28,36,42,45,52,
18,21,25,30,37,45,55,72,
22,24,27,34,40,52,72,96
M4G Smooth V1: An all purpose smooth matrix that blocks very little, gives a similar look to smooth mpeg2 video. -3:0 works well for inloop. Some sources may need to have b-frame mode set to none to work properly with this matrix
INTRA4X4_LUMA =
4,9,18,45,
9,16,35,180,
18,35,225,255,
45,180,255,255

INTRA4X4_CHROMAU =
14,22,32,72,
22,24,41,144,
32,41,96,200,
72,144,200,255

INTRA4X4_CHROMAV =
14,22,32,72,
22,24,41,144,
32,41,96,200,
72,144,200,255

INTER4X4_LUMA =
4,9,18,45,
9,16,35,180,
18,35,225,255,
45,180,255,255

INTER4X4_CHROMAU =
16,22,22,22,
22,22,22,56,
22,22,72,96,
22,56,96,128

INTER4X4_CHROMAV =
16,22,22,22,
22,22,22,56,
22,22,72,96,
22,56,96,128

INTRA8X8_LUMA =
5,11,10,16,28,68,80,80,
12,12,14,18,32,73,90,110,
14,13,16,30,51,87,138,112,
14,17,28,36,80,174,160,124,
18,28,47,88,136,255,206,154,
30,42,86,128,162,208,226,184,
60,128,156,174,206,241,240,202,
144,184,190,196,224,200,206,198

INTER8X8_LUMA =
5,11,10,16,22,28,38,46,
12,12,14,18,26,38,45,70,
14,13,16,24,34,41,51,125,
14,17,22,24,37,50,100,155,
18,22,28,42,70,135,160,180,
24,26,39,50,135,175,200,205,
30,45,51,100,160,200,225,230,
48,70,125,155,180,205,230,255
M4G High Detail V3.1: A very sharp matrix, works best at mid bitrates. -3:0 is the recommended deblocking setting. This matrix is NOT designed to keep grain well, its supposed to look sharp and crisp, not keep everything in the source, it should look detailed, but not transparent, so little pieces of grain and noise are usually some of the first things to get removed.
INTRA4X4_LUMA =
6,16,22,28,
16,24,32,40,
22,32,48,72,
28,40,72,112

INTRA4X4_CHROMAU =
12,22,32,72,
22,24,41,144,
32,41,96,200,
72,144,200,255

INTRA4X4_CHROMAV =
12,22,32,72,
22,24,41,144,
32,41,96,200,
72,144,200,255

INTER4X4_LUMA =
9,20,21,24,
20,21,24,44,
21,24,56,80,
24,44,80,112

INTER4X4_CHROMAU =
16,22,22,22,
22,22,22,56,
22,22,72,96,
22,56,96,128

INTER4X4_CHROMAV =
16,22,22,22,
22,22,22,56,
22,22,72,96,
22,56,96,128

INTRA8X8_LUMA =
8,9,11,13,16,21,24,27,
9,9,12,15,18,22,24,29,
11,12,15,19,22,25,28,32,
13,15,19,24,27,30,34,37,
16,18,22,27,34,39,41,46,
21,22,25,30,39,48,59,74,
24,24,28,34,41,59,81,109,
27,29,32,37,46,74,109,160

INTER8X8_LUMA =
12,12,12,12,13,16,19,25,
12,12,12,12,13,16,21,27,
12,12,13,15,18,21,26,33,
12,12,15,21,25,30,37,45,
13,13,18,25,34,43,54,64,
16,16,21,30,43,60,76,94,
19,21,26,37,54,76,102,132,
25,27,33,45,64,94,132,177

matrices in a 7zip (http://forum.doom9.org/attachment.php?attachmentid=6910&d=1173213612)

[Thunderbolt8]

it would be nice if you could add a description to each mask, what it is supposed to do and under which circumstances to use (low/mid/high bitrates etc.)

[*.mp4 guy]

Not sure what you mean by the "source". I'll get around to adding descriptions later.

[shon3i]

Can you put all matrices in one zip together. And can you reccomend me some for 1 CD rip, where bitrate usualy 500-1000kbps. Thanks

[bond]

a description for what each matrix purpose is is definitely needed, or did you just set them up by using dices? ;)

[fields_g]

Can you put all matrices in one zip together.

Another thought.... I know MeGUI supports profile distrobution through the update server. I don't recall it being used for CQMs. Anyone know if the current MeGUI support this if fed the right files? Once the explanations are developed, and M4G (and the server admins) belive your matrix tweaking is mature enough, I wouldn't mind seeing them delivered in such a way. It should work pretty slick once the server is working again.

[foxyshadis]

Nice. But while copying them, I noticed that you're missing INTRA4X4_LUMA and INTRA4X4_CHROMAU from MRM. Which so happens to be the one I most want to test. =p

[*.mp4 guy]

and M4G (and the server admins) belive your matrix tweaking is mature enough, I wouldn't mind seeing them delivered in such a way. It should work pretty slick once the server is working again. M4G is a less unwieldy contraction of my D9 nickname, sorry for the confusion.

Nice. But while copying them, I noticed that you're missing INTRA4X4_LUMA and INTRA4X4_CHROMAU from MRM. Which so happens to be the one I most want to test. =p thanks for noticing that, fixed.

[shon3i]

Aswome, thank you so much, now testing.

[DDogg]

*.mp4 guy, thanks for the work and the descriptions. Can you share your thoughts on matching one of these matrices, or others for that matter, with the CQ-ASP_Q2_eq() profile? Size is not an overriding issue. Transparency is my main goal, but speed as well - moving a 200+ DVD collection to x264 takes a while. The CQ-ASP_Q2_eq() settings seem to provide quality, reasonable speed, and reasonable compression. Realize there is no one size fits all answer- just would appreciate your subjective opinion on a good place to start, matrix wise.

[bond]

M4G MRM V2: MRM is supposed to work well at moderate bitrates (and sometimes does well even at low and high bitrates, though it isn't suited for transparency). It should give a good balance of details and artefacts, it looks similar to HVS best. yummie :)

[*.mp4 guy]

@bond it is very important to use the modified deadzone settings if you want it to look like hvs best.

@ddog I don't have the CQ-ASP_Q2_eq() profile, from the name it looks like its an asp profile for megui(?), if its not could you post a link to the xml file here so I can see what its like.

[*.mp4 guy]

@DDogg MRM will work well, though the results won't always be transparent, HRM might work well too, but you would have to deal with larger files, and disable b-prediction, and sometimes there wouldn't be any benefit.

[shon3i]

@*.mp4 guy i get very good result with M4G Smooth V1 with all parameters you gave, with 720x464 (War Of The Worlds very grainy movie) @ 773kbps, i have few questions, is this matrix desinged for mid-low bitrates, and it is better tha LRM or MRM for same bitrate, and you mean about b prediction, did you mean about --weightb switch or ?

Thanks

[*.mp4 guy]

it is made to work at any bitrate (where you aren't trying for transparency). It isn't "better" then any of the other matrices, its just different, it usually does well on blurry/grainy footage, or on material with lots of gradients. Sometimes X264 doesn't handle matrices well, when that happens setting b-frame mode to none will fix the problem, thats what I meant by "disabling b-prediction".

[desta]

Thanks for these mp4guy. :)

Could i just ask, what sort of values do you have in mind (roughly) when you say "low to very low bitrates" .. "moderate/mid bitrates" .. "high/very high bitrates" ?

Is it something like:

low = < 800
mid = 800 to 1500
high = 1500 +

... sorry to generalise. Just asking for my own benefit.

[*.mp4 guy]

Low = 1cd or less (IE 900 - whatever).
Medium = Anything more then low and less then high.
high = Transparent or almost transparent (dvd quality), bitrate tends to vary.

[desta]

ok, thanks for clarifying. :)

[shon3i]

it is made to work at any bitrate (where you aren't trying for transparency). It isn't "better" then any of the other matrices, its just different, it usually does well on blurry/grainy footage, or on material with lots of gradients. Sometimes X264 doesn't handle matrices well, when that happens setting b-frame mode to none will fix the problem, thats what I meant by "disabling b-prediction".
Ok, thanks for info, indeed i get very good and smooth picture, with this matrix.

[foxyshadis]

Smooth is the best I've used for CG video of any matrix. It's really awesome!

[akupenguin]

How did Smooth's INTRA8X8_LUMA end up being neither monotonic nor symmetric?

[*.mp4 guy]

How did Smooth's INTRA8X8_LUMA end up being neither monotonic nor symmetric?
Well it is predominantly monotonic (though I suppose thats an oxymoron) long story short I stole some of the frequency weighting from the JPEG matrix, which after a lot of testing turned out to work the best for what I wanted to do, anyway, the mpeg matrix isn't monotonic or symetrical either and HVS Best isn't monotonic, though it is symetrical.
Smooth is the best I've used for CG video of any matrix. It's really awesome!
Thanks, just out of curiosity what type of cg is your source, is it like pixars stuff or is it pseudo-realistic cg?

[akupenguin]

Yes, I see that the quant matrix recommended in the jpeg standard is also nonmonotonic and asymmetric, and has a dc value much higher than the other low freqs (like your HRM intra).

Nonmonotonic: ok, I can believe that there's a difference between even and odd frequencies regarding their tendency for blocking artifacts.
Asymmetric: the only ways this can happen, are if there's a difference in the amount of vertical vs horizontal texture in an average movie (this is true for motion, and some motion estimation algorithms exploit that), or if it's tuned for anamorphic. Of course, the best solution in either case would be to generate a matrix based on the content of the current movie, rather than depending on averages. If you can quantify the psy decisions that went into the creation of the average matrix.

[*.mp4 guy]

Yes, I see that the quant matrix recommended in the jpeg standard is also nonmonotonic and asymmetric, and has a dc value much higher than the other low freqs (like your HRM intra).

Nonmonotonic: ok, I can believe that there's a difference between even and odd frequencies regarding their tendency for blocking artifacts.
Asymmetric: the only ways this can happen, are if there's a difference in the amount of vertical vs horizontal texture in an average movie (this is true for motion, and some motion estimation algorithms exploit that), or if it's tuned for anamorphic. Of course, the best solution in either case would be to generate a matrix based on the content of the current movie, rather than depending on averages. If you can quantify the psy decisions that went into the creation of the average matrix.

Its true that relying upon pre-generated matrixes isn't optimal in some regards, but there hasn't been much success whith automated generation of matrices either, so it makes sense to use pre-generated averages to create matrices, in most cases.

[Thunderbolt8]

Thanks, just out of curiosity what type of cg is your source, is it like pixars stuff or is it pseudo-realistic cg?
just out of curiosity, what does cg mean at all ? :p

[foxyshadis]

Thanks, just out of curiosity what type of cg is your source, is it like pixars stuff or is it pseudo-realistic cg?

Pixar type stuff. Minimal texturing and lots of smooth edges and gradients. Haven't tried other stuff with it yet.
just out of curiosity, what does cg mean at all ? :p

Computer generated.

[elguaxo]

Great thread. Thanks *.mp4 guy

[elguaxo]

Is M4G Smooth V1 similar to your M4G-V3 matrix?

[*.mp4 guy]

Is M4G Smooth V1 similar to your M4G-V3 matrix?
Yes, actualy its a new version with a new name, to (hopefully) keep things from getting confusing. The old version was a preliminary design I had been working on for a while, which is why it had a generic name. At this point the only thing thats likely to change are the chroma matrices, I think they may be a bit too agressive, but I have to do more testing.

[check]

hi, Have you any suggestions for matrices to try for ~q18-20 for 720p anime? The particular source I'm playing with alternates between zero detail pure toon scenes and painted backgrounds with oodles of details. Overall complexity is low, and the final bitrate will probably end up around 1.2-1.4mbits (a 1.4gb target size), but 2mbits is also a possibility.

[*.mp4 guy]

At that bitrate you might be able to get away with a cqm on anime, but it is probably still not a good idea. Try adding "--deadzone-inter 6 --deadzone-intra 6" (without quotes) to the custom command line parameters section in megui instead of using a cqm, it will cause less detail to be dropped, and will probablt work better then a cqm on anime.

[elguaxo]

Try adding "--deadzone-inter 6 --deadzone-intra 6" (without quotes) to the custom command line parameters section in megui

Would be the same replacing </CustomEncoderOptions> with <CustomEncoderOptions>--deadzone-inter 6 --deadzone-intra 6</CustomEncoderOptions> in one of the profiles?

[*.mp4 guy]

Yeah that should work fine.

[elguaxo]

:thanks:

[check]

hmm, interesting. So are x264 cqms bad ideas in general for anime, or are your matrices simply not tuned for cartoon sources?

[DryFire]

I took it to mean that at those quants you won't get much from it.

[*.mp4 guy]

Cqms aren't usefull for anime until there won't be any visible ringing artefacts, at which point a cqm that can preserve more detail and block less will probably help. However I don't have a good idea of where the trade off occurs since I don't encode anime much. Which is why I recommended a more conservative solution by just lowering the deadzone setting, which might work better for anime at high bitrates then one of my cqms anyway, as I said I have not done much testing on the subject.

In theory it should be possible to make a matrix that looks better on anime then the flat matrix at higher bitrates, I beleive Soulhuter made one for Xvid a while ago, but its not something I've given much thought to.

[Mc Onyx]

First of all thanks for the matrices and your time spent on them, well done. Second, can you suggest any of your matrices for HD Encoding at 720p 4-5Mbit. I read somewhere here that at that high resolution and bitrate, it doesn't really matter which one to use. And i can simply use flat matrix, but please clarify if i'm wrong. Thanks

[check]

Thanks for the feedback mp4, I'm running some encodes now with modified deadzone settings thanks to your idea.

[*.mp4 guy]

The same general rules apply to HD as SD, the biggest difference comes from the viewing conditions, many displays can easily show the flaws in SD content, but may have trouble showing the same problems in HD content becuase of the increased resolution, which can lead people to falsely assume that HD can be overcompressed without problems. If you wan't the best results you have to find settings that look good to you on sd content, then use the same Q values and settings on hd content, this will ensure that you don't end up throwing away quality that you might miss later when higher resolution display are more readily available.

At your bitrate MRM should give you transparent results, though without knowing more about your source its really hard to say.

[Thunderbolt8]

I need some advice on encoding a movie and the (possible) use of custom matrixes. I am planning to make a 1,46 GB rip from texas chainsaw massacre (1974) ultimate edition. dvd is ntsc, movie length 1:23:20, the audio stream is ~260 mb, so ~1,2 GB should remain for the video stream. I can use either 2 pass encode or -crf mode with a ~corresponding size.
my problem is that my pc is quite slow for encoding and also tends to crash once a day, so I cannot really try too much out without spending a whole week on one movie. thats why I'd like to have some advice for parameters like the use of custom matrixes (which) or deadzone settings (if needed) and so on so I can already aim at the best regions without too much having to try out. I took some screenshots via vlc snapshot (im screenshot newbie), if they arent really useful then tell me plz so I can try to make new ones (and tell me how to do it better -.-).
considering the video quality, I post some comments from dvdcompare.net:

"-disc "Ultimate Edition" - Steelbook Packaging. High definition source, restored by Don May of "Synapse Films". Some quotes from him about the new transfer (source: HTF - hometheaterforum):
*The new HD transfer was not created from 35mm materials. It was created from the original 16mm vault materials.
*The older version was so noise-reduced (with DVNR) that most of the grain was removed and smoothed out, along with detail and sharpness.
*The grain was there because of the film stock used. Tobe himself told that there was no conscious decision to make the film look "documentary-style", with the graininess, etc. He was trying to film the best movie he could. The heavy grain structure you've seen on prints, etc. were caused by a combination of the film stock used and the cheap 35mm blow-ups done by the distributor at the time."

http://img153.imageshack.us/img153/7016/vlcsnap77783qf2.th.png (http://img153.imageshack.us/my.php?image=vlcsnap77783qf2.png)
http://img149.imageshack.us/img149/1267/vlcsnap78747zd0.th.png (http://img149.imageshack.us/my.php?image=vlcsnap78747zd0.png)
http://img153.imageshack.us/img153/3345/vlcsnap84156kw6.th.png (http://img153.imageshack.us/my.php?image=vlcsnap84156kw6.png)
http://img175.imageshack.us/img175/5923/vlcsnap80949ml3.th.png (http://img175.imageshack.us/my.php?image=vlcsnap80949ml3.png)
http://img149.imageshack.us/img149/7954/vlcsnap81217pg0.th.png (http://img149.imageshack.us/my.php?image=vlcsnap81217pg0.png)
http://img153.imageshack.us/img153/7459/vlcsnap83369vz2.th.png (http://img153.imageshack.us/my.php?image=vlcsnap83369vz2.png)

[*.mp4 guy]

There isn't much high frequency detail in any of the screenshots you posted, and they are all fairly noisy, so its a fair bet that the smooth matrix would work the best for you (assuming that you aren't close to transparency at 2mbps, I would guess that you wouldn't) since you don't wan't to do more then one encode you should set "bframe mode" to "none" to avoid possible problems, also make sure you have enabled "no fast p-skip" otherwise just use whatever you like for quality vs speed. -2:-1 should work pretty well for deblocking. The defualt deadzone settings should be fine aswell.

[Thunderbolt8]

what is meant with high frequency detail, has it something to do with the bitrate of the dvd ?
I could do lets say 3 encoded to try out, but I really dont want to make more, cause it simply will take too long on my comp. so there is a little room for experimenting, just not much.
should I use the 2 pass mode or -crf mode ? and what are the usual deadzone settings, since they are not implented in the gui yet ? are they usually diasbled (and trellis enabled) or is it this one here "--deadzone-inter 6 --deadzone-intra 6" ?

Thanks so far, Im gonna try that out now.

[*.mp4 guy]

The usual deadzone settings are what is used when you don't specify anything. You want to hit a specific bitrate, so you should obviously use 2pass mode. High frequency detail more or less corelates to how sharp the image is, though its a bit more complicated then that.

[Thunderbolt8]

just to clarify again, since I am using / have to use a certain profile mostly trellis is activated and the deadzone settings havent been implented into sharktooths profiles yet, so it means in that case for smooth v1 matrix deadzone is off (=not in command line) and trellis activated ?

[*.mp4 guy]

That should work fine, though I haven't done much testing with trellis.

[desta]

I was wondering.. is using AQ together with your matrices something you would say is best not to do, or is it still ok?

[*.mp4 guy]

Well they shouldn't need aq, but if you like the way it looks with them there isn't any reason I can think of that you shoudn't use it.

[desta]

Ok, thanks again. :)

[Zeest]

What would you recommend for older movies (80's) and a video bitrate of around 1400-1500. Movies like The Thing, Come and See etc. I'm using SharkTooth's HQ_Slower profile.

[*.mp4 guy]

I don't know what type of content is typical of 80's films, could you post some representative screenshots, or a short clip, also how long is the source.

[JarrettH]

how about a zip file of them? :devil:

[R3Z]

What would you recommend for older movies (80's) and a video bitrate of around 1400-1500. Movies like The Thing, Come and See etc. I'm using SharkTooth's HQ_Slower profile.

Those movies have a lot of grain in them so if you like grain use sharktooths EQM_HR AVC matrix or if you dont like it use a denoiser and then use whatever matrix you prefer.

[*.mp4 guy]

Matrices are attached in a 7zip archive, I'll add a link to the first post.

[Morte66]

M4G Smooth V1: An all purpose smooth matrix that blocks very little, gives a similar look to smooth mpeg2 video. -3:0 works well for inloop. Some sources may need to have b-frame mode set to none to work properly with this matrix

I've used M4G-V3 for pretty much everything since you posted it, so I was very interested in this one. I re-did my last encode with it, using my usual -2-2 deblocking for both, and your new matrix seems to be a useful improvment. Today I'm going to try it again with the -3-0 deblocking you suggested.

Thanks a lot for these matrices.

[*.mp4 guy]

I'm glad that you (and everyone else who uses my matrices) find them helpful, I'm not completely satisfied with them yet, but unfortunately I can't think of anything else to experiment with at the moment, so they won't be changing much for the time being.

Small thing about deblocking, it is in my experience mostly "in the eye of the beholder", so the settings I suggest are more guidelines then rules, and there is definately some room for tweaking them, depending primarily on source sharpness, display sharpness and personal preference. Basically you want higher deblocking settings the blurier your source and the sharper your display are, and lower deblocking settings the less you like denoising and its associated detail loss.

[huang_ch]

mp4, for M4G Smooth V1, didn't see there's a recommended deadzone settings, so do you mean using the default or also use 6/6? (My purpose is to remove blocks.)

[*.mp4 guy]

It is supposed to be used with the default deadzone settings, lowering the deadzone could help fight blocking but would cause a lot of extra ringing because of the way the smooth matrix avoids artifacts.

[freshNewB]

thanks
i used "mp4_guy's_AVC_Low_Bitrate_matrix_V2" for low bitrate (300~400kbps) 640x480 anime. the size/quality is amazing .
Is M4G-LRM-V4 good at this bitrate?
or is there a mp4_guy's_AVC_Low_Bitrate_matrix_V3 :D

[*.mp4 guy]

mp4_guy's_AVC_Low_Bitrate_matrix_V2 has been replaced by M4G LRM V4, I'm not sure wheather it will work as well on anime as the older versions, though, becuase the older versions were smoother.

[*.mp4 guy]

The HRM matrix has been more or less completely reworked and updated to v2 (see first post), and should be much more flexible now, it should give a smoother transition between "not trasnparent" and "trasnparent" results, feedback is welcome, and was invaluable in making this version of the matrix. I'm aware that some sources may exhibit banding with this matrix, even at very high bitrates, but I have no idea what to do about it, atleast for the time being, as other dificult sources with similar characteristics do not exhibit any problems I can see. This problem is why I delayed the release of this version, but as I see no possible causes or solutions to the problem at the moment, I figured it was time to release it anyway since the previous version had the same problems.

[edit] Also a very minor update to the smooth matrix, probably not noticible, but updated for consistency.

[elguaxo]

Thanks!

[check]

now that the xvid vs. x264 thread has sort of died down, can you give any personal experience / feedback on how these matrices preserve gradients and grain? In my particular case gradients are much more interesting, but I'm sure others are interested in the latter.

EDIT: in the smooth description, you say direct none might be needed. Is this a random requirement, or do certain types of video need it?

[*.mp4 guy]

EDIT: in the smooth description, you say direct none might be needed. Is this a random requirement, or do certain types of video need it?

Its pseudo random, mostly determined by how good your source is, with animated content it would be a better idea to turn off b-rdo and leave direct set to auto, as it will give better compression and avoid any artifacts just as well.

now that the xvid vs. x264 thread has sort of died down, can you give any personal experience / feedback on how these matrices preserve gradients and grain? In my particular case gradients are much more interesting, but I'm sure others are interested in the latter.

LRM (especially older versions) is heavily biased twards gradients, later revisions are a bit more balanced and keep more detail, but still favor gradients more then details, LRM differs from Smooth in that it tries to avoid impacting high frequencies to harshly, as ringing and bluring on lines can become an issue at low bitrates.

MRM does not keep gradients particularly well, at its target bitrate gradient artifacts shouldn't be a big issue with X264, since transparency isn't possible at this bitrate it makes sense to use higher compression on low frequencies because inloop deblocking will fix most of the artifacts to some extent. looks a bit like a sharper, more banding prone version of HVS best

HRM tries to be balanced, so that at a given bitrate its not easy to tell that a video has been recompressed, it softens prominent details slightly and has a tendency to tone down noise without completely removing it. In its intended bitrate range it keeps gradients as well as I can get it too, which is not always very well, at lower bitrates it starts to soften the entire picture as opposed to causing ringing, blocking, banding, or areas of obviously removed detail, gradients tend to be preserved about aswell as noise by this matrix.

High detail preserves gradients a bit better then MRM, it takes a stronger aproach to removing "redundant stuff" then MRM which can lead to pronounced ringing on some material, it tends to get rid of low level noise while maintaining strong noise better then any of my other matrices.

Smooth is heavily biased twords gradients, but does worse on gradients at high bitrates the HRM do to its strong removal of high frequencies, it usually preserves some of the noise from the source, but the noise it keeps is always blurier and less prominent then in the source, always softens the picture by a noticible margin, tends to have the least obvious artifacts at medium bitrates.

It all varies a lot, sources have a large impact on what artifacts a given codec will create, this of course carries over to matrices aswell. I find that with pristine sources X264 has an easy time avoiding many of the problems it encounters when recompressing mpeg2s, most notably blocking and banding artifacts are greatly reduced, and it is possible to get truely trasnparent encodes.

[elguaxo]

I never tried your HRM matrix and I want to do some tests with v2. In the previous version of this matrix I think deblocking was not recommended, but now you say this version is more flexible.

If I use some deblocking, what would be your recommended settings for a sharp picture? Thanks!

[*.mp4 guy]

-2:-4

[SealTooGreat]

@*.mp4 guy
Is there any "--deadzone-inter X --deadzone-intra Y" adjusting for M4G LRM V4 matrice.

[elguaxo]

SealTooGreat, I'm not an expert, but if you want more details you could try M4G MRM V2. It works well at low bitrates (not so low as LRM), and it keeps more details.

[SealTooGreat]

@elguaxo
I'm not an expert, too, but at lower bitrates, blocks are pain in the @ss. It seems,at about 800kbps, that M4G MRM V2 better handle this problem than M4G LRM V4. But under 600kbps M4G LRM V4 is unreplaceable.

[*.mp4 guy]

LRM was made before deadzone adjustment was available in X264, so it has only been extensively tested with the default settings, all the same it should work fine at any deadzone you set, so if you want to fiddle with the deadzone using LRM there shouldn't be any problems. At such low bitrates you are probably best off using trellis instead of manually tweaking the deadzone parameters, low bitrates are where all of X264's compresion enhancing features really become invaluable.

elguaxo is correct about MRM vs LRM on detail retention, SealTooGreat you are right about MRM blocking to much at those bitrates. Its a trade off, details, blocks, ringing, aliasing, you can't have everything, more detail leads to more of something else aswell, more ringing, blocking, aliasing or a higher bitrate.

[SealTooGreat]

LRM was made before deadzone adjustment was available in X264, so it has only been extensively tested with the default settings
Do You plan to update LRM?!
At such low bitrates you are probably best off using trellis instead of manually tweaking the deadzone parameters
*.mp4 guy, you are right...I found that LRM+Trellis2 reduce blocks more than LRM+Trellis1, but I couldn't find any proper deadzone parameters tweaking(Do you have any suggestion?!..or not to bother with deadzone ). It seems that block-reducing do not response on deadzone parameters tweaking as well as on trellis tweaking(about 600kbps)
you can't have everything
I must!:D

[*.mp4 guy]

Trellis and deadzone are mutually exclusive, you can't use custom deadzone settings with trellis. The default deadzone settings work very well for low bitrates anyway, so there isn't much (anything) to be gained by tweaking LRM for different deadzone settings; to clarify some sources may look better with higher or lower deadzone settings, but on average the defaults are better then any tweaked settings. The X264 developers may make it possible to use trellis and specify intra and inter deadzones at the same time in the future, but I am not aware of any plans to do so, with that in mind deadzone tweaking is not as usefull at low bitrates as trellis, so I won't be looking into using modified deadzone settings with LRM. Anyone else who would like to is of course welcome to make a modified version of LRM with custom deadzone settings in mind if they would like to.

to summarize:
-LRM will not be updated for daedzone settings in the forseable future
-don't bother with deadzone settings at low bitrates, trellis is better and can't be used with deadzone parameters.

[Manao]

It's not that deadzone can't be used with trellis, it's just that trellis ignore deadzone for doing the quantization. Deadzone defines how the rounding is done, but trellis tries both rounding anyway, so it doesn't matter.

What the devs wanted to do, however, was to map the relative importance of the DCT coefficients during the trellis onto a deadzone matrix, in order to make the trellis less optimal ( psnr-wise ) but better from your visual point of view.

As of yet, that has not been done.

[*.mp4 guy]

So to clarify, you can set deadzone and use trellis, but when you do the deadzone settings are ignored, which means that you can't specify deadzone settings and use trellis at the same time since when you use trellis it ignores the deadzone parameters you set...

Also, I thought what the devs were talking about doing was having the deadzone settings effect the trellis rate distortion optimisation routines via lambda?

[Manao]

Also, I thought what the devs were talking about doing was having the deadzone settings effect the trellis rate distortion optimisation routines via lambda?Indeed, hence the "less optimal PSNR wise / better from a visual point of view".

[juhu]

I get an "invalid attachment specified" when trying to get your "7zip" matrices pack . Could you repost them please ?
(mewiki.project357.com is down too, so I can't find them anywhere)

Also : What do you think is the "best" matrix for let's say Casablanca final (airport) sequence : a harsh combination of artificial fog, serious grain and moving spotlights (!!!)
the purpose is to rip hd with bitrates around 3000.
.I worked with lanczos+ xvid vhq4 (1152*848), result is ok if not transparent, but the most annoying this is all these moving halos going with the light variations (I don't know if that's what you call "gradient", my english is still "in progress"...)
.same script/bitrate with x264 (hq slower profile) gave me...worse results which came as a rather bad surprise. Clearly less visible details, even part of the inscriptions clearly readable on background with xvid cames as partially erased... and the halos were still the same....

thanks for any ideas.

[*.mp4 guy]

Huh, I must have messed something up (obviously).

Here (http://www.mytempdir.com/1244271) is a temporary link, I have also reuploaded them to the forum, but the they will take a while to be aproved.

To address the rest of your post, try HRM with direct=none and deblock -2:-4, rdo6, hex me with me range=24, intra luma quantizer deadzone=6, inter quantizer deadzone=8.

[audioman]

Hi mp4 guy

How to integrate your matrices in STAXRIP ? is it a way to do it ?

Thanks

[*.mp4 guy]

I'm not sure, Stax (I'm not sure If I spelled his name right) the author of STAXRIP should be able to answer that for you, you could try pming him or starting a thread in the guis forum. I wish I could be more helpful, but I don't use STAXRIP myself.

[juhu]

Here (http://www.mytempdir.com/1244271) is a temporary link, I have also reuploaded them to the forum, but the they will take a while to be aproved.

To address the rest of your post, try HRM with direct=none and deblock -2:-4, rdo6, hex me with me range=24, intra luma quantizer deadzone=6, inter quantizer deadzone=8.

thanks for the matrices & the suggestions, greatly appreciated

[audioman]

I'm not sure, Stax (I'm not sure If I spelled his name right) the author of STAXRIP should be able to answer that for you, you could try pming him or starting a thread in the guis forum. I wish I could be more helpful, but I don't use STAXRIP myself.

thanks

what about megui ?
can we use matrices in megui ? :)

[elguaxo]

what about megui ?
can we use matrices in megui ? :)

Yes! Here:
http://img244.imageshack.us/img244/2847/meguipq9.th.png (http://img244.imageshack.us/img244/2847/meguipq9.png)

[audioman]

thanks mp4guy

Just a question
What is the difference between your matrices and those integrated in megui ( HQ SLOW, SLOWEST )
Which matrix would you advise me to choose for a 1000 Kbs movie ?

Do I have to make a compressibility test or can I avoid using it when making a H264 movie ?

Thanks

[*.mp4 guy]

1 - HQ Slow, and HQ slowest aren't matrices, so I'm not sure how to answer your question

2 - its not the bitrate value that matters, the quantization value is a much better indicator of quality (though still not perfect)

3 - Since you already have picked a bitrate a compressability test is not necessary, just use 2pass mode with your desired bitrate.

4 - elguaxo answered your last question, not me

5 - you might want to read a guide on mpeg4 avc, death the sheep has a pretty nice one



I would like to be more helpful, but I'm not entirely sure what your asking, and I want to avoid confusion.

[audioman]

Thanks mp4guy

I just wanna know what kind of settings do you use to have a great mp4 quality file , near DVD quality
I want to back up my own movies

You are talking about quantisation values , how do you see this value in megui ?

Thanks again

[*.mp4 guy]

http://img80.imageshack.us/img80/3532/meguimh0.png

Set mode to constant quality, or constant quantizer, then set the quality level to control the quantizer. In constant quality mode you specify the average quantizer, whereas in constant quantizer mode you specify an absolute constant quantizer, constant quality mode gives better quality at the same bitrate.

You can also see the average I, P, and B quantizers of an encode by looking at the stats window.

[Keepitsimple]

an extremely noob question but what are matrices(ie matrices in this scenario, i know what a matrice is)?

[JarrettH]

If you don't choose a custom one it uses the default (flat) matrix which is very very good!

[SealTooGreat]

@*.mp4 guy
In your first post, You forgot to put the comma at the end of the rows in INTRA8X8_LUMA and INTER8X8_LUMA for M4G MRM V2, M4G HRM V2, M4G Smooth V1 and M4G High Detail V3.1

[*.mp4 guy]

Thanks, I'll fix that. (Just so people know they work fine without them in X264, but they should be there anyway)

[SealTooGreat]

Yes it works fine with x264, but recently I've tried using ffdshow encoder via VirtualDub with your custom matrices and ffdshow encoder refused to load them, just because of comma missing.
BTW what matrice do you use with above mentioned constant quality 20.0? Is there any other tweak?

[*.mp4 guy]

20 is right around where HRM starts to outperform MRM in most cases, so usually with Q20 I would use HRM.

[SealTooGreat]

@*.mp4 guy
When you compare two different h264 encoders or one h264 encoder with different settings, does inloop(deblocking) postprocessing have to be ON or OFF?!

[*.mp4 guy]

Inloop filtering is part of the AVC standard, it can be on or off, aslong as the same settings are used for both encoders.

[SealTooGreat]

I don't get it, when deblocking postprocessing is used during playback, there's less blocks, but on the contrary there's more blocks. Which solution is better for comparison?
Or, for example, when I say one h264 encoder is better then other, do I have to say, also, that I have (not) used deblocking postprocessing during playback or not?
Especially at the low bitrate, there's a HUGE difference between switching ON and OFF in deblocking postprocessing during playback.
I don't get it how you guys compare different h264 setting/encoders without mention this thing.

[check]

h264 inloop deblocking is not post processing, it's part of the normal decoding process. If you encode the video with it enabled, you should decode it with it enabled, and vice versa.

[SealTooGreat]

If you encode the video with it enabled, you should decode it with it enabled, and vice versa.
Check, thanx for very clear clarification.
But I've read somewhere on doom9 that is not recommended encoding without inloop, and also all presets that I came on have inloop enabled so is there any case where it should be disabled during encoding?!
If I've downloaded h264 movie from net how do I know if it was encoded with or without inloop?! Or inloop is something that is "must have".
*edit*
I've opened new thread h264 encoding/decoding inloop filtering questions (http://forum.doom9.org/showthread.php?p=969265#post969265) to keep this one clean. Mods move #93-98 posts

[check]

if inloop was disabled when the video was encoded, it will NEVER be enabled during playback, no matter what the decoder preferences say :). Because of this, you should always leave the inloop deblocker in decoder prefs on, unless CPU limitations prevent you from doing so.
There are probably some cases where not using inloop makes sense, but there aren't many. What often does make sense is reducing the power of deblocking during encoding, the x264 defaults at least are often too strong for videos.

(btw mods, you might want to split this into a new thread and keep this one clean).

[elguaxo]

Is it ok to use 2 different matrices in a same encode?

I just encoded a movie with a very good overall quality using M4G High Detail V3.1. But there is just a particular scene where the main character is in a low contrast/dark backround that shows a lot of blocks in the encode.

I encoded that scene again using this (http://forum.doom9.org/showpost.php?p=920050&postcount=33) script and used M4G Smooth V1 with very good results. And then I pasted everything together.

Playback is fine with my current setup (ffdshow version Feb 14 2007, Haali Media Splitter 1.7.77.5, Zoom Player v5.00), but I wonder if using 2 different matrices is just a 'hack' that is not officially supported.

[akupenguin]

Multiple matrices are supported. Like any other mixed settings, you have to use different values of --sps-id if you want to mux the result into mp4 or mkv.
Most global settings are stored in the sps/pps, and mp4/mkv store only 1 copy of each sps/pps. But if you give them different ids, then the muxer knows to keep both. (If you mux them first and then concatenate, mkvmerge should notice any different sps with the same id, and refuse to concatenate in that case.)

[elguaxo]

But if you give them different ids, then the muxer knows to keep both.

I see all pieces have the same IDs:
SPS id: 0
PPS id: 0

How do I give them different ids? Is --sps-id an x264 option? Thanks.

Edit: found it in x264's --longhelp!
--no-asm Disable all CPU optimizations
--visualize Show MB types overlayed on the encoded video
--sps-id <integer> Set SPS and PPS id numbers [0]

[S¡nTë£]

Hey
Wondering if there are any recommended settings for deblocking with the HRM mtx.

Also, when you say "B predict mode should be set to none ..., otherwise b-rdo should be disabled", do you mean that for grainy sources you better disable b predict or b-rdo, OR do you mean that for grainy sources you better disable b predict and for other sources you better disable b-rdo?

[*.mp4 guy]

-2:-4 works well for HRM, "you mean that for grainy sources you better disable b predict and for other sources you better disable b-rdo" that is right.

[fields_g]

Many people are using AQ instead of deadzones. Do these matrixes work better with deadzones? Are there some AQ setting recommendations for the various CQMs, especially HRM?

[elguaxo]

*.mp4 guy has already recommended deadzones for some of his matrices, but I think he has not tried the AQ patch a lot: http://forum.doom9.org/showthread.php?p=998178#post998178

For the filesizes/rate factors/resolutions I am using, I am very happy with the M4G High Detail V3.1 matrix with --aq-strength 0.6 to 0.9 and a --aq-sensitivity 10 http://forum.doom9.org/showthread.php?p=1001174#post1001174

[fight2win]

if i use custom commandline in megui and enter deadzone options for hrm matrx, i get it at last as follows, is it ok? also, what is this aq and how can i set it for high bitrate detail retaining encodes, sir mp4guy?

--bitrate 1866 --analyse p8x8,b8x8,i4x4 --vbv-maxrate 25000 --threads auto --thread-input --progress --no-psnr --no-ssim --output "" "" --deadzone-inter 4 --deadzone-intra 6

[*.mp4 guy]

You don't need to use the custom commandline options now, luma inter and luma intra deadzones are accesable in megui. elguaxo is right, I haven't tested aq much, so I can't offer any advice on it.

You haven't given my enough information for me to judge your entire commandline.

[fight2win]

You don't need to use the custom commandline options now, luma inter and luma intra deadzones are accesable in megui. elguaxo is right, I haven't tested aq much, so I can't offer any advice on it.

You haven't given my enough information for me to judge your entire commandline.

thanks sir, can u pls tell me if deadzone 4 is inter or intra luma and is 6 inter or intra luma, also sir, i'm trying to encode my casino royale disc to x264 anamorphic mp4 with dual audio lc-aac 5.1 audio streams, and aiming for a filesize of 2235 mb (1/2 dvd) , so i was thinking of using ur hrm matrix as u say it's pretty sharp. sir, can u please advice me about matrix and what other options to keep sharpness and quality , i want to keep sharpness and quality, filesizes or encoding time don't matter much, thanks in advance

[*.mp4 guy]

I apreciate you're being polite, but could you stop calling me sir, thanks. 4 is intra deadzone, 6 is inter deadzone. HRM isn't aimed at being "sharp" its aimed at being low on artifacts, so there will be a bit of blurring in some cases, but very little loss of detail or blocking/banding. High detail v3.1 and MRM v2 are both sharper then HRM, but have more artifacts, MRM having more blocking, and high detail having more ringing, both causing more banding; however at the bitrate your using no matrix should cause noticible detail loss, so what is imprtant is avoiding artifacts.

I would recommend using these options:
-no fast p-skip
-no dct decimation
-no b-rdo
-aq (with whatever settings you like)
-5 reference frames
-mixed reference frames
-max 3 b frames
-multi hex motion estimation with search range 24 (default is 16)
-rdo6
-keyeframe interval 240 (250 for pal)
-min gop size 48 (50 for pal)
-chroma me
-all macroblock options enabled
-all b-frame options enaabled (except b-rdo)
- deblock -1:-2, or -2:-4 (second is sharper, but maybe more ringing)

-all other options at default

[fight2win]

I apreciate you're being polite, but could you stop calling me sir, thanks. 4 is intra deadzone, 6 is inter deadzone. HRM isn't aimed at being "sharp" its aimed at being low on artifacts, so there will be a bit of blurring in some cases, but very little loss of detail or blocking/banding. High detail v3.1 and MRM v2 are both sharper then HRM, but have more artifacts, MRM having more blocking, and high detail having more ringing, both causing more banding; however at the bitrate your using no matrix should cause noticible detail loss, so what is imprtant is avoiding artifacts.

I would recommend using these options:
-no fast p-skip
-no dct decimation
-no b-rdo
-aq (with whatever settings you like)
-5 reference frames
-mixed reference frames
-max 3 b frames
-multi hex motion estimation with search range 24 (default is 16)
-rdo6
-keyeframe interval 240 (250 for pal)
-min gop size 48 (50 for pal)
-chroma me
-all macroblock options enabled
-all b-frame options enaabled (except b-rdo)
- deblock -1:-2, or -2:-4 (second is sharper, but maybe more ringing)

-all other options at default

thanks sir (pls let me call u sir, talented needs to be respected), sir, one more small question, what is aq and what settings to use for it, is it adaptive quantization? how to set value for it?also, -2 is deblocking strength or threshold in -2,-4 deblocking?

[Tack]

thanks sir (pls let me call u sir, talented needs to be respected)
One might argue that it is disrespectful to not honour his request not to refer to him as "sir." But I understand that this is probably a culture thing. I don't like being called sir either.

what is aq and what settings to use for it, is it adaptive quantization? how to set value for it?
AQ is adaptive quantization, yes. It is not officially part of x264 but is available as an add-on patch. Do x264 --help and if you see options aq-strength and aq-sensitivity, your x264 build has AQ support.

You'll want to experiment with the values to see what meets your needs best. I use aq_strength=0.8 aq_sensitivity=17.

also, -2 is deblocking strength or threshold in -2,-4 deblocking?
Here's the documentation from mplayer's man page for deblock:

deblock=<-6-6>,<-6-6>
The first parameter is AlphaC0 (default: 0). This adjusts thresholds for
the H.264 in-loop deblocking filter. First, this parameter adjusts the
maximum amount of change that the filter is allowed to cause on any one
pixel. Secondly, this parameter affects the threshold for difference
across the edge being filtered. A positive value reduces blocking arti-
facts more, but will also smear details.
The second parameter is Beta (default: 0). This affects the detail
threshold. Very detailed blocks are not filtered, since the smoothing
caused by the filter would be more noticeable than the original blocking.
The default behavior of the filter almost always achieves optimal quality,
so it is best to either leave it alone, or make only small adjustments.
However, if your source material already has some blocking or noise which
you would like to remove, it may be a good idea to turn it up a little
bit.

[fight2win]

One might argue that it is disrespectful to not honour his request not to refer to him as "sir." But I understand that this is probably a culture thing. I don't like being called sir either.


AQ is adaptive quantization, yes. It is not officially part of x264 but is available as an add-on patch. Do x264 --help and if you see options aq-strength and aq-sensitivity, your x264 build has AQ support.

You'll want to experiment with the values to see what meets your needs best. I use aq_strength=0.8 aq_sensitivity=17.


Here's the documentation from mplayer's man page for deblock:

deblock=<-6-6>,<-6-6>
The first parameter is AlphaC0 (default: 0). This adjusts thresholds for
the H.264 in-loop deblocking filter. First, this parameter adjusts the
maximum amount of change that the filter is allowed to cause on any one
pixel. Secondly, this parameter affects the threshold for difference
across the edge being filtered. A positive value reduces blocking arti-
facts more, but will also smear details.
The second parameter is Beta (default: 0). This affects the detail
threshold. Very detailed blocks are not filtered, since the smoothing
caused by the filter would be more noticeable than the original blocking.
The default behavior of the filter almost always achieves optimal quality,
so it is best to either leave it alone, or make only small adjustments.
However, if your source material already has some blocking or noise which
you would like to remove, it may be a good idea to turn it up a little
bit.


oh-kay, well, i'm using cerf's x264 builds, dunno if they support aq or not, and dos window not staying in my vista...

[fight2win]

i'm using latest megui, and cerf's latest x264, but deadzone options are phased out/unselectable:

http://img381.imageshack.us/img381/1407/untitledhw1.th.png (http://img381.imageshack.us/my.php?image=untitledhw1.png)

[elguaxo]

You must disable trellis in order to use the deadzones features.

[fight2win]

You must disable trellis in order to use the deadzones features.

ok, thanks

[SpAwN_gUy]

I would recommend using these options:
-no fast p-skip
-no dct decimation
-no b-rdo
-aq (with whatever settings you like)
-5 reference frames
-mixed reference frames
-max 3 b frames
-multi hex motion estimation with search range 24 (default is 16)
-rdo6
-keyeframe interval 240 (250 for pal)
-min gop size 48 (50 for pal)
-chroma me
-all macroblock options enabled
-all b-frame options enaabled (except b-rdo)
- deblock -1:-2, or -2:-4 (second is sharper, but maybe more ringing)

-all other options at default 'kay .. i suppose theese options are for "film" content... and back to our sheeps (i'm (belo)russian)...
will there be any "cartoon/anime" matrix? ;) ... and what will you (or someone else) recommend settings for anime? ...

'cause i'm still using deadzones=6(as you recommended somwhere in this thread) and now i've added aq settings to command-line, but i dunno is this still "correct" or will i get any profit of using them both at the same time.

[*.mp4 guy]

'kay .. i suppose theese options are for "film" content... and back to our sheeps (i'm (belo)russian)...
(1)will there be any "cartoon/anime" matrix? ;) ... and what will you (or someone else) recommend settings for anime? ...

(2)'cause i'm still using deadzones=6(as you recommended somwhere in this thread) and now i've added aq settings to command-line, but i dunno is this still "correct" or will i get any profit of using them both at the same time.

(1) - I don't encode anime, but there are plenty of other people here who could make an anime matrix; as for settings, high bitrate anime can be treated about the same as high bitrate film (you need to use the same tricks to get rid of those last few artifacts), low bitrate anime however is a completely different matter, that I have even less experience with.

(2) - What is correct is what looks the best to you. There is no inherent problem with using custom deadzones and aq at the same time, though I would recommend using trellis on anime instead of deadzones (unless you are using very high bitrates).

[HeadBangeR77]

I've recently dug out an old grrrainy & noisy sample o'mine, and I'm torturing it atm using your HRM v2 custom matrix. I've had some very good results so far, using both dead zones and AQ. I'm struggling hard to reduce the bitrate, that bumped up form about 3400 kbps (1024x432) with the default flat matrix to 6400 kbps using yours. The target is about 4000 kbps, as I would like to keep the film within 1 DVD-5 limit, including the original AC3 sound.

When I've gone through my 1-pass CFR samples and achieved the results, that I will be pleased with, I will post a sample or two ;)

In the meanwhile:
:thanks: for your custom CQMs and all the effort you've put into creating and improving them :)

cheers /HDBR77

[*.mp4 guy]

HRM is not always ideal for bitrate limited encoding, if you are having trouble reaching your needed filesize with it, it might be a good idea to switch to a more aggressive matrix; as with all advice, this is a generalization and may not apply to your specific situation.

[HeadBangeR77]

AQ has a great influence on the subjective quality of the film (how I would like to watch it), but also a huge impact on the bitrate in my particular case. I haven't done any samples with HRM v2 (dead zones as recommended, no trellis, no fast p-skip, no dct-decimate) without AQ. Should have a closer look, when I've finally found more time for samples. I'm at CRF one pass encoding atm, before I decide to encode the whole film in two pass bitrate mode.

If I use motion estimation at 5, or eventually at 6(RDO), yet without RDO on B-frames, is it safe to use weightened biprediction then? (grain retention is important in this case, but not sth I couldn't live without ;) )

cheers /HDBR77

[*.mp4 guy]

You can safely go to rdo7, b-rdo enabled, 2 (4 if you don't mind a little flicker sometimes) consecutive b-frmes, bi-prediction enabled, as long as you set b-predict mode to none (rdo7 is fine no matter what), there will be a small impact on grain retention, but it shouldn't be significant.

[Sagittaire]

Problem is not really bframe but ratio for bframe. If you want higher quality for bframe use smaller ratio like 1.10 for example. Default ratio at 1.30 is certainely optimal for best overall quality but perhaps not for constant "local quality". With high ratio quality for bframe will be always lower than reference pframe simply because bframe in this case will use "very" higher quant than reference pframe. If you want really see if problem is bframe in this case use ratio at 1.00.

[HeadBangeR77]

Problem is not really bframe but ratio for bframe. If you want higher quality for bframe use smaller ratio like 1.10 for example. Default ratio at 1.30 is certainely optimal for best overall quality but perhaps not for constant "local quality". With high ratio quality for bframe will be always lower than reference pframe simply because bframe in this case will use "very" higher quant than reference pframe. If you want really see if problem is bframe in this case use ratio at 1.00.
I've already marked that B-frames are of higher quality than with XviD, even if I use low compression matrices and settings like 1.00 ratio + 1.00 offset in the latter case. Think you're pointing in the right direction.
Atm I'm using PB ratio like 1.25 or 1.20 depending on the encode. Usually it's enough for me to achieve the quality I want, especially with MRMv2 and HRMv2. Since I always use AQ, and the latest 667b (or is it 667c already, as avinaptic indicates?) pumps more bits in low-quality areas than the original rev. 667, I even get higher quality B-frames than P-frames. Not in terms of quantizers of course, but in terms of grain retention and the effect I would like to achieve (very subjective, though).

@ M4G:
I've had some incredibly good results with your CQM MRMv2 and direct mode set to auto. I'm under impression those samples might be even better than HMRv2 with no direct mode. Must compare still frames to decide. Tomorrow I should have a faster connection, so I will upload some samples, if I manage.
I assume the way to success are: your matrices with recommended dead zones, the rest of the settings I use, and the AQ (it really does the trick, yet not with the default flat one ;)).

cheers /HDBR77

[Razorholt]

Hi HeadBangeR77,

I'm currently encoding at 450kbps a bunch of mpeg2 videos and I think I tried pretty much all the matrices... Just like you (I presume) I'm sticking to the ones developed by mp4guy.

What sort of content are you encoding? Anime or film? And at what birates?


Thanks,
- Dan

[HeadBangeR77]

Hi HeadBangeR77,

I'm currently encoding at 450kbps a bunch of mpeg2 videos and I think I tried pretty much all the matrices... Just like you (I presume) I'm sticking to the ones developed by mp4guy.

What sort of content are you encoding? Anime or film? And at what birates?
I think mp4guy could help you much more than me:
- I'm rather new to x264,
- my target bitrates are high above yours,
- I'm a quality freak, who likes the old grainy style of XviD encodes :cool:

Atm I'm concentrating more on testing individual matrices: MRM v2, HRM v2, and High Detail v3.1 from mp4guy, and I would like to compare them (for my high bitrate grainy purposes ;)) with EQM AVC HR and the Prestige one. I'm pretty busy atm, and sometimes I have to compare dozens of still frames to tell the difference, so I will share my experience no sooner than in a week.

cheers /HDBR77

[SealTooGreat]

@*.mp4 guy

pls, check 3rd_raw/7th_column (29) and 7th_raw/3rd_column (28) in MRM V2 under INTER8X8_LUMA. Is this OK?

[Chainmax]

Are all these matrices (plus the detail-preserving ones extracted from some HDDVDs or BDs, can't remember their names) included in MeGUI's autoupdater?

[elguaxo]

Nope.

[Chainmax]

Do you think one of the MeGUI devs would be interested in doing so if asked?

[Sharktooth]

for what purpouse?

[*.mp4 guy]

@*.mp4 guy

pls, check 3rd_raw/7th_column (29) and 7th_raw/3rd_column (28) in MRM V2 under INTER8X8_LUMA. Is this OK?
It is ok, Just a typo, the 28 should be a 29. I'll fix it in the first post, but since it won't make a difference visually I'm not going to consider it an update.

[Chainmax]

for what purpouse?

For the same purpose it has profiles and avisynth filters: having a variety of options available.

[Sharktooth]

if we start to include matrices, ppl will think EVERY matrix is good for every source and will always be better than standard quantization. That's not true and i want to avoid it or ppl will start complaining for the output quality blaming x264.
matrices in AVC are for advanced users and i dont think that kind of ppl will have problems getting them from the forum and use them in the appropriate situations.

[rkalwaitis]

Mp4 is there a matrix for a wavelet codec?

[rkalwaitis]

How do you extract the data required to create your matrices. Is there a way to see each films data so that you can decide how you would like to change it if at all?

This is interesting stuff.

thanks

[Adub]

Just so you know, cqm's are pretty much outdated. What with the use of AQ, Psy-rdo, and psy-trellis, they are pretty much obsolete.

[JarrettH]

Andddd...Flat is the best looking overall anyway:p

[rkalwaitis]

I like flat too.

[Sharktooth]

How do you extract the data required to create your matrices. Is there a way to see each films data so that you can decide how you would like to change it if at all?

This is interesting stuff.

thanks
video stream analyzers.
IIRC avinaptic can do that... otherwise there should be other free softwares.

[*.mp4 guy]

Mp4 is there a matrix for a wavelet codec?
Kind of, It depends upon the implementation, and in any case, there probably will be far fewer coefficients.

[rkalwaitis]

I can not see any matrix like your custom ones when I use avinaptic to look at an avi file. Can you send me a link?

[*.mp4 guy]

I can not see any matrix like your custom ones when I use avinaptic to look at an avi file. Can you send me a link?

I'm afraid I don't understand the question. Avinaptic doesn't use cqm's.

[Yoshiyuki Blade]

I'm afraid I don't understand the question. Avinaptic doesn't use cqm's.

Avinaptic usually allows you to see the cqm used in a given clip. He says he can't see your cqms them via Avinaptic when he checks for it.

[rkalwaitis]

That is what I was going to ask, I thought I was doing something wrong :)

Okay so I have a vob file for instance. I want to analyze the video stream, which technically could help me determine if or what matrix I could apply to it? Isnt that how custom matrices come to be? I mean dont you have to have some numbers to change around to come up with the custom matrix? I guess I really want to know how to get those numbers.

Am I being stupid? MP4?

avinaptic doesnt necessarily show if a custom matrix is applied for example, it would not for a mkv file.

why is that? container specific?

thanks for answering my sill questions.

[Sharktooth]

MPEG-2 matrices are different than AVC matrices and the custom matrix info is in the video stream. if no info is shown then no custom matrix was used but just standard quantization.
i dont know what you want to do, but i can perceive a lot of confusion. search the forum for "custom matrices", "CQM", "custom quantization", etc... and document yourself a bit.
at present day CQMs are quite useless since the new adaptive quantization algos do great jobs.

[rkalwaitis]

Thanks Sharktooth, I was just trying to figure out where the got the starting numbers they used to make custom matrices. I am enjoying using Megui. Which has the x264. I tried one of MP4s custom matrices and it made a noticeable difference on an older film, I have and was just wondering how they made them.

[rkalwaitis]

Sharktooth doesnt 264 have its own custom matrix "JVT or Flat" with it? Everyone says they custom matrix is obsolete because of x264, but it has two matrices, correct???

[Ranguvar]

There's threads on making custom matrices, search. And the Flat matrix is what is now recommended. It's not really a custom matrix, it's different... I can't explain it as well as Sharktooth or *.mp4 guy can though.

[Sharktooth]

flat is not a custom matrix it's the DEFAULT quantization. JVT matrix is included too though and it's quite useless. like the default mpeg quantization matrix in xvid.

[Ranguvar]

like the default mpeg quantization matrix in xvid.
Do you mean H.263, not MPEG?

[Sharktooth]

no... xvid has h.263 (default quantization for MPEG4 ASP) and MPEG (default MPEG matrix used in previous MPEG encoder standards).

[rkalwaitis]

So the codec assumes that the information taken from the film is the same for each dvd regardless of what it is? You have to base and this base/set of numbers/stream whatever you call it can be changed via a matrix. So what are the numbers the codec sees before the matrix changes them (I think that is the a better way to ask), and if so how do you see them or know what they are ???

[Sharktooth]

quantization matrices work on frequencies and the matrix coefficients represents how much those frequencies are quantized.
you better :search: and read some docs coz it may be a veeery long discussion.

[rkalwaitis]

Ranguvar, you gonna fix snow for us :)

[rkalwaitis]

okay didnt mean to drag it out.
Thanks guys

[Ranguvar]

Ranguvar, you gonna fix snow for us :)

Um :confused:

I wish I had even the bare minimum of the skills needed to do such a thing, but sadly, I do not.

Besides, Snow/Dirac, while very interesting, do not show incredible promise IMO. H.264 still has more potential, and from what I've heard, the benefits of using *lets aren't big enough to justify moving development. By the time H.264 is nearing the end of it's life, I expect there will be odder things to ooh and aah about :)

[rkalwaitis]

MP4

"--deadzone-inter 6 --deadzone-intra 6" applicable with your High Detail matrix? AQ Strength ? 1.3?

Does anyone have issues with the flat matrix with 264 having a fuzzy look at times?

Thanks
Robert

[Sharktooth]

why not? i'd suggest to use psy-rdo and trellis though...

[rkalwaitis]

how do i turn on psy-rdo???
Looks like my trellis is on its set to 2.

program --bitrate 700 --ref 5 --mixed-refs --bframes 4 --b-pyramid --b-rdo --bime --weightb --filter -3,0 --subme 6 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --merange 24 --threads auto --thread-input --aq-strength 1.3 --cqmfile "C:\Users\Baba-Nator\profiles\x264\matrix\M4G_High_Detail_V3.1.cfg" --progress --no-psnr --no-ssim --output "output" "input" --deadzone-inter 6 --deadzone-intra 6

Do I add it to the command line in MeGUi?

Teach me Sharktooth :)

[Sharktooth]

it's enabled by default, if you want to rise it's strenght just add --psy-rd with a value higher than 1 in the custom commandline options.
also if you use trellis you cant use deadzones.
There are two different quantization algorithms: deadzone and trellis. Any given dct of a given block can only use one of them. Because, well, if you had two different values for each coefficient, what would you do with them?
--deadzone-inter and --deadzone-intra tweak a parameter of the deadzone algorithm. (The parameter in question is "deadzone", that's where the algorithm got its name.) --trellis determines when to use the trellis algorithm (0=never, 1=sometimes, 2=almost always); any dcts that don't use trellis must use deadzone instead.
So, the fact that --deadzone* still has some minor effect when you enable trellis does not contradict the fact that deadzone and trellis are incompatible.
Don't do that. If you enable trellis, then deadzone is only used as a fast estimate of trellis, so a reduced value of --deadzone does not provide any grain retention, it just reduces the precision of that estimate and thus of the mode decision process. Sure I could make x264 ignore --deadzone* if --trellis is enabled, but that would contradict my policy of "give the luser enough rope to hang himself".

That said, it might be possible to tweak trellis in a way that has similar results to --deadzone*. In which case --deadzone* could apply to trellis too, despite the name.

BTW, just because an option is printed in the userdata sei doesn't mean it had any effect on the encode. I remove some of the extraneous options (e.g. most of ratecontrol when using cqp), but still some options may be printed even if they're overridden by some other option.

[rkalwaitis]

Thanks Sharktooth, You Da Man

[rkalwaitis]

I guess its good to know that we are afforded enough rope to hang ourselves, that is awfully generous of akupenguin.

[Pitou]

Quick question for Mp4Guy,

For the M4G HRM V2 matrix, you suggest to:

B predict mode should be set to none if your source is very grainy(and you wan't it to stay that way), or has many low contrast areas

Are you referring to the "direct=auto|none|temporal|spatial" option of x264?

Thanks.

Pitou!

[*.mp4 guy]

Yes, "direct=auto|none|temporal|spatial" should be set to "none".

[Codex0nz]

Well im looking to get the most quality out of my x264 encodes, mainly anime, i have a lot of confidence in your ability mp4 guy

Who can recommend what matrix to use for dvd sourced anime encoding?

Or is there not much point in loading one of these up for anime encodes

Also if you use one of these matrixes do you turn b frame mode to none?

Sorry, trying to learn -_-

[*.mp4 guy]

There are very few situations where using a cqm on anime would be a good idea. Unless you have something very specific you are trying to achieve, you are safest with the flat matrix.

[Codex0nz]

There are very few situations where using a cqm on anime would be a good idea. Unless you have something very specific you are trying to achieve, you are safest with the flat matrix.

ah well that solves my concerns :) thanks dude

too bad nothing like this can fix the bloody avatar rips (still trying -_- lol)

I thought using these would improve the quality of what x264 would save etc but guess i was wrong (or just not in this situation)

[LoRd_MuldeR]

The most reasons why people used custom matrices in the past are now obsolete, thanks to VAQ and Psy-RDO.

Furthermore VAQ and Psy-RDO don't work as expected with certain custom matrices...

[*.mp4 guy]

No, they work as expected, the results just aren't good. This is an oversimplification, but that is basically it.

[LoRd_MuldeR]

...the results just aren't good.

That's what I meant. Bad results isn't what I expect.

[Audionut]

For x264 with all the tuning done based on the flat matrix, it makes sense to now remove custom matrix support.

[*.mp4 guy]

It never makes sense to remove support for something when it costs you nothing to leave it intact. Besides, almost all of x264, including the psychovisual's work exactly as they should with cqm's. Ugly skipped mv's remain the only real "mistake" a matrix will induce x264 to make, the other unwanted behavior is mearly the result of transients being butchered by matrixes removing their highest energy components more aggressively, VAQ quantizing reqions with transients more aggressively, psy trellis also quantizing transients more aggressively, and psy-rd de-tuning rd decisions in order to make the picture more closely resemble the local complexity of the source by introducing "artifacts" (which is usually bad for transients).

Matrices and x264's psy enhancements aren't incompatible, or even bad to use together, you just have to be carefull to avoid overdoing the transient hatred. I should also probably update this thread and my deblocking FAQ to take into account that with VAQ 0:0 is usually the best deblocking setting, even with matrices.

[Sagittaire]

Matrices and x264's psy enhancements aren't incompatible, or even bad to use together, you just have to be carefull to avoid overdoing the transient hatred. I should also probably update this thread and my deblocking FAQ to take into account that with VAQ 0:0 is usually the best deblocking setting, even with matrices.

No ...

[*.mp4 guy]

No ...

In my experience your disagreement, despite its vagueness would apear to indicate that I am correct.

[Sagittaire]

In my experience your disagreement, despite its vagueness would apear to indicate that I am correct.

Simply because with CQM you can't never make generalisation. In fact IMO the best solution is not use CQM with psy optimisation simply because psy tools are optimised for flat16 matrix.


I should also probably update this thread and my deblocking FAQ to take into account that with VAQ 0:0 is usually the best deblocking setting, even with matrices.

here for example it's completely false. Alpha and beta for inloop effect are highly dependant of Matrix. Absurd example is flat8 vs flat32 matrix. If 0,0 produce best result with flat16 then 0,0 will be certainely not best setting with agressive or soft matrix for global coef.


IMO Matrix are simply useless now with x264 simply because psy tool have same objective with more efficiency result. If you want really use CQM the better way now is really soft modification with all coef (with flat16 ref) like that for example:

INTRA4X4_LUMA =
12,12,14,16,
12,14,16,20,
14,16,20,25,
16,20,25,32

INTRA4X4_CHROMAU =
12,12,14,16,
12,14,16,20,
14,16,20,25,
16,20,25,32

INTRA4X4_CHROMAV =
12,12,14,16,
12,14,16,20,
14,16,20,25,
16,20,25,32

INTER4X4_LUMA =
14,14,15,16,
14,15,16,17,
15,16,17,18,
16,17,18,19

INTER4X4_CHROMAU =
14,14,15,16,
14,15,16,17,
15,16,17,18,
16,17,18,19

INTER4X4_CHROMAV =
14,14,15,16,
14,15,16,17,
15,16,17,18,
16,17,18,19

INTRA8X8_LUMA =
12,12,12,12,13,14,15,16,
12,12,12,13,14,15,16,17,
12,12,13,14,15,16,17,19,
12,13,14,15,16,18,20,22,
13,14,15,16,18,21,23,27,
14,15,16,18,21,24,28,37,
15,16,17,20,23,28,39,48,
16,17,19,22,27,37,48,68

INTER8X8_LUMA =
14,14,14,14,15,15,15,16,
14,14,14,14,15,15,16,17,
14,14,14,15,15,16,17,18,
14,14,15,15,16,17,18,19,
15,15,15,16,17,18,19,21,
15,15,16,17,18,20,22,26,
15,16,17,18,19,22,27,32,
16,17,18,19,21,26,32,44

[Audionut]

If you want really use CQM the better way now is really soft modification with all coef (with flat16 ref) like that for example:

Which is exactly as mp4 guy said.

the other unwanted behavior is mearly the result of transients being butchered by matrixes removing their highest energy components more aggressively,

I've created some matrices that have increased metrics, but visually have never been as good as the flat matrix.

I always use psy optimisation and AQ though, even when testing matrices. I've never done a test against my matrices and DS's psy ops.

Because I'm fairly certain I know who would lose. And it doesn't make sense to create a matrix that requires features of the encoder to be disabled.

[Sagittaire]

Which is exactly as mp4 guy said.


well it's definitively not the case for all the mp4 guy matrix.


I've created some matrices that have increased metrics, but visually have never been as good as the flat matrix.

CQM don't increase metric.


I always use psy optimisation and AQ though, even when testing matrices. I've never done a test against my matrices and DS's psy ops.

Simply because CQM and x264 psy x264 tools don't make the same thing. CQM is just here to have a little better HVS macroblock look with more optimal gradiant/detail tradeoff. CQM can't help like AQ for better overall picture quality or can't help like SSD for better complexity preservation. Anyway you can use CQM with all the actual x264 psy tools but you must simply use really soft CQM matrix close to flat16 matrix.

[*.mp4 guy]

LRM and HRM still work the same as ever. Smooth works to an extent, MRM is mostly pointless, and High Detail causes too much ringing.