Problem with X264's predicted motion vectors

[*.mp4 guy]

For a long time I was baffled by a problem many of my avc cqm's faced when used in X264. They would preserve detail and texture quite well, but there would often be sliding textures and missplaced blocks in the encodes. I beleive this is caused by the higher average quant these cqm's require to hit a desired bitrate, at higher quants X264 uses a higher amount of predicted mv's istead of actual mv's, which results in motion errors. Is there anyway to reduce the amount of predicted mv's X264 is allowed to use?

[akupenguin]

x264_cqmskip.0.diff (http://students.washington.edu/lorenm/src/x264/x264_cqmskip.0.diff)
I tried this when I first implemented cqm, and for the jvt matrix it produced way too few skips, but maybe it's better with current code or current matrices.

[*.mp4 guy]

I really don't have any idea how to compile X264, would it be possible for this to be implemented as a switch in the cvs untill a better alternative is found?

[Zero1]

I have noticed a similar problem to *.mp4 guy, but like him, I am in the same boat when it comes to patching and compiling x264 (and making it work, I can produce a dud compile just fine :p)

I think having it as a switch was a good suggestion, providing it won't break a whole lot of other things ;P I'd certainly be willing to test it out.

Cheers

[*.mp4 guy]

I tested some more, and even when I use the same exact settings and cq mode low bitrate matrices still use a lot less predicted mv's, why is that?

[akupenguin]

You mean Skip blocks, right? Skip = predicted mv and no residual.
All sane cqms, low-bitrate or not, use a smaller quantizer for dc than the flat matrix does. Smaller quantizer means the quantized dc is more likely to be nonzero. Nonzero coefficients imply not a Skip block. This might or might not be counterbalanced by the larger quantizer of some of the other coefficients, but apparently not in your cqm.

[*.mp4 guy]

You mean Skip blocks, right? Skip = predicted mv and no residual.
All sane cqms, low-bitrate or not, use a smaller quantizer for dc than the flat matrix does. Smaller quantizer means the quantized dc is more likely to be nonzero. Nonzero coefficients imply not a Skip block. This might or might not be counterbalanced by the larger quantizer of some of the other coefficients, but apparently not in your cqm.

My problem is that a high bitrate matrix of mine gets more skiped blocks at the same cq then a low bitrate one, even though all of its ac and dc coeficients are lower.

[akupenguin]

While I don't know what caused your extra skips...
Why would you even make a matrix with all of the coefficients less than 16? You're just confusing everything that thinks it knows what qp is (e.g. the skip decision, the deblocking filter, rate-distortion optimization, ratecontrol, any potential post-processing filter someone might write, etc).
All sane cqms should produce approximately the same bitrate at the same qp. If they don't, then increase the cqm coefficients and decrease the qp.
(Of course the converse the converse also applies to "low bitrate" matrices. "low bitrate matrix" should mean "tuned for high qp", not "produce lower bitrate at normal qp".)

There was a legitimate reason in mpeg4asp to mess with the meaning of a qp: Ratecontrol was very coarse if you had to jump between qp=1,2,3, so you might want to halve the cqm and double the qp, and then you're at qp=2,4,6, which have extra steps in between. That excuse does not apply to h264, with its logarithmic qp scale.

[*.mp4 guy]

While I don't know what caused your extra skips...
Why would you even make a matrix with all of the coefficients less than 16? You're just confusing everything that thinks it knows what qp is (e.g. the skip decision, the deblocking filter, rate-distortion optimization, ratecontrol, any potential post-processing filter someone might write, etc).
All sane cqms should produce approximately the same bitrate at the same qp. If they don't, then increase the cqm coefficients and decrease the qp.
(Of course the converse the converse also applies to "low bitrate" matrices. "low bitrate matrix" should mean "tuned for high qp", not "produce lower bitrate at normal qp".)

There was a legitimate reason in mpeg4asp to mess with the meaning of a qp: Ratecontrol was very coarse if you had to jump between qp=1,2,3, so you might want to halve the cqm and double the qp, and then you're at qp=2,4,6, which have extra steps in between. That excuse does not apply to h264, with its logarithmic qp scale.

I was refering to my own low bitrate matrix, not the standard one, so not all of the values were lower then 16. However it was significantly less aggressive. akupenguin you do realise that if you have a matrix that has has coeficients that are exactly half of another matrice's coeficients that at the exact same bitrate they will produce significantly different pictures that are not the result of ratecontrol differences in mpeg4-asp. It apears the same is true of mpeg4-avc, though it is hard to tell with all of the related dificulties.

[Edit]
Here is an example, The settings used in the encodes are exactly the same in both cases, cq mode was used(to avoid ratecontrol issues), the filesizes are close enough to make no difference on such a short clip. I would have made an example using X264, but it has the afformentioned problems that would make comparisons inconclusive.

8,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,33

8,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
32,32,32,32,32,32,32,32
all32's@Q4 example clip (http://rapidshare.de/files/17076364/all32_Q4.avi.html)

8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8

8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
8,8,8,8,8,8,8,8
all8's@Q9.65 example clip (http://rapidshare.de/files/17076507/all8_Q9.65.avi.html)

The reason that the inter DC coeficient in the all 32's matrix is 8 is because intra dc coeficients are locked at 8 in mpeg4-asp, so it makes the comparison more accurate by keeping all the DC coeficients at 8.

[akupenguin]

you do realise that if you have a matrix that has has coeficients that are exactly half of another matrice's coeficients that at the exact same bitrate they will produce significantly different pictures that are not the result of ratecontrol differences in mpeg4-asp. It apears the same is true of mpeg4-avc, though it is hard to tell with all of the related dificulties.
Yes, it's not just ratecontrol, but also due to the skip decision, the deblocking filter, rate-distortion optimization and in mpeg4-asp also your locked dc coef.

That said, all of those issues are avoidable in h264 if I were to tune the encoder with full accounting for cqms. An encode with the default matrix is exactly the same as an encode with half the cqm entries and +6 qp and -3 deblocking strength and modified lambda, deadzone, chroma_qp_offset, skip and decimation thresholds, etc.
My point was that if you modify cqm too far and don't modify all the other parameters to match (and they aren't all tweakable in x264), then you'll get a suboptimal encode.

[*.mp4 guy]

Yes, it's not just ratecontrol, but also due to and in mpeg4-asp also your locked dc coef.

That said, all of those issues are avoidable in h264 if I were to tune the encoder with full accounting for cqms. An encode with the default matrix is exactly the same as an encode with half the cqm entries and +6 qp and -3 deblocking strength and modified lambda, deadzone, chroma_qp_offset, skip and decimation thresholds, etc.
My point was that if you modify cqm too far and don't modify all the other parameters to match (and they aren't all tweakable in x264), then you'll get a suboptimal encode.

Well I suppose it would be nice if some of those options (Deadzone and decimation thresholds, would be particularly usefull) were available, it would make it easier to make X264 behave properly at high bitrates. I realize in theory having overal lower matrix values shouldn't change anything, but in practice its the only way to affect things like deadzone and decimation that have a significant effect on picture quality. As long as those options are locked and X264 goes into fits when you try to use a matrix to affect them it makes things more difficult then they have to be.

[edit] For instance being able to tweak the deadzone parameter would help with blocking and lack of detail at higher bitrates.

[Audionut]

I would like to agree with *.mp4.guy.
I have also encountered the same problem. At first I thought it might have been a problem with the source I was using and/or a decoder issue. I never got round to finding the real cause of the problem.

I was testing a very high datarate matrix.

INTRA4X4_LUMA =
1,2,3,5,
2,3,5,6,
3,5,6,7,
5,6,7,8

INTRA4X4_CHROMAU =
2,5,6,7,
5,6,7,8,
6,7,8,9,
7,8,9,12

INTRA4X4_CHROMAV =
2,5,6,7,
5,6,7,8,
6,7,8,9,
7,8,9,12

INTER4X4_LUMA =
1,2,3,5,
2,3,5,6,
3,5,6,7,
5,6,7,8

INTER4X4_CHROMAU =
2,5,6,7,
5,6,7,8,
6,7,8,9,
7,8,9,12

INTER4X4_CHROMAV =
2,5,6,7,
5,6,7,8,
6,7,8,9,
7,8,9,12

INTRA8X8_LUMA =
1,1,2,3,3,4,4,5,
1,1,3,3,4,4,5,6,
2,3,3,4,4,5,6,7,
3,3,4,4,5,6,7,8,
3,4,4,5,6,7,8,9,
4,4,5,6,7,8,9,10,
4,5,6,7,8,9,10,11,
5,6,7,8,9,10,11,12

INTER8X8_LUMA =
1,1,2,3,3,4,4,5,
1,1,3,3,4,4,5,6,
2,3,3,4,4,5,6,7,
3,3,4,4,5,6,7,8,
3,4,4,5,6,7,8,9,
4,4,5,6,7,8,9,10,
4,5,6,7,8,9,10,11,
5,6,7,8,9,10,11,12

The result was blocking.

edit: here is a sample
http://rapidshare.de/files/17235790/source.rar.html

The result is not blocking, but moving of pixels.
Not sure excatly how to explain.

[akupenguin]

I was testing a very high datarate matrix.
No, you were using a "lets screw the codec over and see if it can still produce a recognizable picture" matrix.

"High bitrate matrix" is not at all related to "matrix with small coefficients"! (in h264, of course. I don't speak for xvid.)

A "high bitrate" matrix is designed to preserve all the film grain, noise, and other crap that some people think makes a movie look authentic once you've encoded all the real content with acceptable quality. A "low bitrate" matrix is designed to filter out all the details while leaving a minimum amount of artifacts. That design is all about selecting a balance between coefficients, and nothing to do with the absolute value of the coefficients. Very large or very small absolute values just confuse the codec, nothing more, nothing less.
No, I don't plan to change that. RDAQ could sorta hide your errors, but it's not the right answer.


P.S. The exact failure mode in your case was: In order to maintain a given bitrate, x264 had to send qp way up, to like 40 or 50 just to get a bitrate that the default matrix would achieve at 20 or so. Then it's encoding a frame at qp=50, and x264 says "hey, I know how to encode at qp=50, that's where I have to throw away all the information I possibly can because this video is way too complex for the target bitrate." So during motion estimation, it very agressively picks large partitions and small mv deltas and so on.

[Audionut]

Crap, I forgot to post the settings and log file.

Starting job job1 at 2:41:46 PM
Job is a video job. encoder commandline:
--crf 18 --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --filter -1,-1 --subme 7 --trellis 2
--analyse all --8x8dct --ipratio 1.2 --qcomp 1.0 --direct auto --me umh --threads 2 --thread-input --cqmfile "F:\AVC_HIGH.cfg"
--progress --output "E:\STAR_WARS_EPISO\entire movie\source.mkv" "E:\STAR_WARS_EPISO\entire movie\source.avs"
successfully started encoding
Processing ended at 2:46:52 PM
----------------------------------------------------------------------------------------------------------

Log for job job1

avis [info]: 704x288 @ 25.00 fps (1001 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
x264 [info]: slice I:9 Avg QP:17.78 size: 56250 PSNR Mean Y:49.44 U:51.34 V:51.22 Avg:49.97 Global:49.89
x264 [info]: slice P:255 Avg QP:18.02 size: 28736 PSNR Mean Y:48.65 U:49.99 V:49.82 Avg:49.03 Global:49.00
x264 [info]: slice B:737 Avg QP:19.69 size: 1738 PSNR Mean Y:46.22 U:49.16 V:48.83 Avg:46.95 Global:46.83
x264 [info]: mb I I16..4: 0.2% 64.4% 35.4%
x264 [info]: mb P I16..4: 0.0% 1.6% 0.1% P16..4: 55.3% 22.6% 19.4% 0.7% 0.3% skip: 0.0%
x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 2.2% 0.7% 1.4% direct: 0.2% skip:95.5%
x264 [info]: 8x8 transform intra:73.3% inter:88.8%
x264 [info]: direct mvs spatial:91.6% temporal:8.4%
x264 [info]: ref P 63.9% 13.3% 11.3% 5.0% 6.5%
x264 [info]: ref B 82.1% 8.9% 4.7% 2.4% 2.0%
x264 [info]: PSNR Mean Y:46.869 U:49.394 V:49.101 Avg:47.510 Global:47.307 kb/s:1821.15

Actual bitrate after encoding without container overhead: 1822.40
desired video bitrate of this job: 18 kbit/s - obtained video bitrate: 1825.22617382617 kbit/s
----------------------------------------------------------------------------------------------------------



I take it you did not look at the sample I posted.

To me, a high bitrate matrix is one that requires more bitrate at a constant quant, then the default matrix.
But you're saying that, a high bitrate matrix is one that preserves noise etc. To me, that would be a "matrix that preserves details".


Very large or very small absolute values just confuse the codec, nothing more, nothing less.

So the real question then is. The acceptable range for coefficients is not 1-255, but instead it is "insert your answer here".

edit: or

P.S. The exact failure mode in your case was: In order to maintain a given bitrate, x264 had to send qp way up, to like 40 or 50 just to get a bitrate that the default matrix would achieve at 20 or so. Then it's encoding a frame at qp=50, and x264 says "hey, I know how to encode at qp=50, that's where I have to throw away all the information I possibly can because this video is way too complex for the target bitrate." So during motion estimation, it very agressively picks large partitions and small mv deltas and so on.

Are you saying that when engaged in CRF mode, x264 still tries to maintain a datarate based on the datarate that the standard matrix produces?

[Audionut]

"High bitrate matrix" is not at all related to "matrix with small coefficients"!

In order to maintain a given bitrate, x264 had to send qp way up, to like 40 or 50 just to get a bitrate that the default matrix would achieve at 20 or so.

Clarification. Why would x264 have to increase the qp to 40 or 50 with my matrix if it isn't the result of small coefficients?

[akupenguin]

Increasing the qp at constant bitrate is a direct result of the small coefficients.
But just because your matrix produces a huge bitrate at constant qp, does not mean that it looks better than the flat matrix when given the same huge bitrate.

Or am I misusing the term, and everyone else says "high bitrate matrix" to mean "looks like crap at low bitrates" without any reference to what it might actually be good for?

So the real question then is. The acceptable range for coefficients is not 1-255, but instead it is "insert your answer here".
Off the top of my head: range should be maybe [8,40], with an average of no less than 16. The high frequencies might tolerate a little bigger, but if you set them too high you might as well downscale instead.

[Audionut]

Others opinion's as to what constitutes a high bitrate matrix would be welcome.

As I stated before, my idea of a high bitrate matrix, is one that requires a higher bitrate than the standard matrix at any given quant.

Increasing the qp at constant bitrate is a direct result of the small coefficients.

Agreed. Or put differently. Using smaller coefficients in a matrix requires more bitrate at the same quantizer than higher coefficients. Hence, why I called my matrix "high bitrate", because it uses small coefficients, hence higher bitrate at a constant quantizer, or put differently, higher quantizer at a constant bitrate.

But just because your matrix produces a huge bitrate at constant qp, does not mean that it looks better than the flat matrix when given the same huge bitrate.

I never stated otherwise. I mean no disrespect to yourself, as I greatly appreciate your contribution to x264.

However, it seems as though you are of the opinion that high bitrate matrix = high quality.
I belive that there is a difference between the two.

High bitrate, means excatly that, More bits, nothing more, nothing less.

High quality, means, to my eyes, the resulting video looks better.

I never gave the impression that my matrix was the, "be all, and end all of matrices", "the shit", "why use anyone else's".
I posted in this thread to inform yourself, that when using my matrix, errors in the video stream were the result.

you were using a "lets screw the codec over and see if it can still produce a recognizable picture" matrix.

One of the reasons for having open source software is to test it's limitations, and what it can and can't do. Correct?

Off the top of my head: range should be maybe [8,40], with an average of no less than 16. The high frequencies might tolerate a little bigger, but if you set them too high you might as well downscale instead.

Thankyou. Now I know. Fell free to bitch slap me if this has already been posted.

[Manao]

High bitrate, means excatly that, More bits, nothing more, nothing lessYou do agree, however, that that concept is completely useless for h264, since quantizer 1 for the default matrix already gives an insane bitrate. With ASP, quantizer 2 was sometimes not able to deliver a wanted quality, hence a high bitrate matrix might have been useful. With AVC, such a concept is useless.

[Audionut]

With ASP, a properly tuned "high bitrate" matrix could still provide good "quality" at high quantizers.
So that while say the h.263 matrix was usefull for quants 2-4 with reasonable quality, a tuned matrix was usefull for quants 2-6.
Didee pointed out for example that his sixofnine matrix provided extra (quality) steps.

What I am trying to do is, create a matrix for AVC that does virtually the same thing.
If you want to encode at quant 1 or even quant 10 which is very high, you have a matrix that is going to keep all the fine details and produce an outstanding encode. But at the same time, will not punish you for using quants up to say 30.

[Manao]

Then reread the thread, especially when aku clearly underlines the fact that the "sixofnine" concept is completely useless in AVC, since the quantizer scale is logarithmic and quite fine.

You seem not to be aware of the following : whatever the quantizer and the matrix, raising the quantizer by six will multiply the coefficient by 2 and the size (roughly) by 2. Dividing the coefficients by 2 won't give you a higher precision. It'll simply act as though you were using a -6 quantizer.

[Sharktooth]

avc works in a completely different way than asp. you cant apply the same ideas and expect the same results. this is also for CQMs.
in avc a high bitrate matrix is: a matrix that helps preserving more details at the same or lower quants than standard quantization.
a low bitrate matrix is: a matrix that helps removing all the "unnecessary" details but keeping a watchable picture.
so the concept of high-bitrate and low-bitrate matrices becomes quite useless.
it's much better to classify matrices by smoothness/sharpness (smoother good for low bitrates, sharper good for high bitrates).
however ppl are used to hear about bitrates so it will take some time until they realize avc is a competely different story than asp...

[Audionut]

Now it's all really starting to make sense.

IIRC this has not been discussed before.

Perhaps a mod could split this thread from my first post, and label it something appropriate.

[NeD tHe OnE]

Which matrices are good for high bitrates around 2500? I mean will it give good results like sharp picture? and what about @ 800-1000 kbps? Which matrix give u the smoothest picture ... I think Sharktooth Avc is overall Best! What do u say?

[DeathTheSheep]

Perhaps a mod could split this thread from my first post, and label it something appropriate.
I second that opinion! I stumbled upon this thread by accident, and wouldn't have thought to look through it at all because of the title...

[Sagittaire]

Which matrices are good for high bitrates around 2500? I mean will it give good results like sharp picture? and what about @ 800-1000 kbps? Which matrix give u the smoothest picture ... I think Sharktooth Avc is overall Best! What do u say?

These CQM are perhaps good but x264 seem optimized for flat16 matrix. IMO flat16 is simply the best visually in most situation ... but it's just my opinion.

[Mug Funky]

@ NeD tHe OnE:

the impression i get from all this is that you're more likely to get better quality by tweaking other settings before you change the matrix.

stuff like deblocking strength (try -3,-3) would help matters more at a high bitrate like that. you might also want to lower the min qp (it's 10 by default i think), but be careful with it - as has been said a few times, the qp is logarithmic, so think of it more as being in decibels (it almost is - 6 qp = multiplier of 2). so min quant = 4 would give twice the "max quality" that min quant 10 would.

or am i completely wrong? i'm still primarily using mpeg-2 right now :)