What's the one thing you would change about the H.264 spec?

[the_corona]

You often hear about people bitching who know the standard quite well (at least that's my impression) about how retarded some of it's aspects are, including it appears most of the proposals for H.265.

So, I was wondering, if you were in charge, whats the one (or two) thing you would change about the H.264 spec that would improve it the most in your opinion, and why.

Let's stay realistic though, so only ideas that could be reasonably decoded by today's cpu's and could reasonably have a hardware decoding chip implemented for.

And let's exclude killing of Interlacing too....that's a given :devil:

Thanks :-)

[J_Darnley]

And let's exclude killing of Interlacing too....that's a given :devil:

But it's such a good idea it's worth repeating

[Dark Shikari]

If we only include things that could possibly get into an MPEG/ITU standard: replace the i16x16 mode with an actual 16x16 transform and make the inter transform work accordingly (16x16, 8x8, or 4x4). Maybe make the decision quadtree instead of per-MB.

If we include things that they would never, ever include, because it would help far too much for them to possibly allow: energy-preserving quantizer a'la CELT.

[Guest]

Require SPS/PPS with each IDR.

[simonhorlick]

About the actual spec itself. Rewrite it into English and explain non-trivial decisions with rationales.

[Guest]

About the actual spec itself. Rewrite it into English and explain non-trivial decisions with rationales. It's already in English and a specification is not the place for justification of design decisions.

Please don't derail the intent of this thread.

[iwod]

What is Constrained-Energy Lapped Transform?

CELT is a lossy low delay Audio Codec......

[*.mp4 guy]

Greater then 8 bit decoding bit depth so that its possible to encode gradients without hitting really low quantizers, and so that the deblocking filter is actually functional for removing +/-1 blocking/banding.

A deblocking filter that removes blocks, not slight edge discontinuities. Its pathetic how useless it is for filtering i16x16 blocks, and even 8x8 blocks when they get zeroed which is when you really want a deblocking filter to work. It only really works on the 4x4 transform, which really shouldn't be surprising.

Qpel that isn't based on bilinear interpolation, they basically made blur-pel, which is admittedly better then asp's gibbs-pell. Still its just another factor that makes avc blurrier and more dull looking then it has to be, along with being more inefficient quantitatively.

Transforms that aren't optimized for the processing limitations of a wristwatch. The transforms are no where near the largest source of decoding complexity, and yet they keep inventing new fancy sub-optimal transforms that will be oh so much faster on a system where multiplications are done by a newt with a binary abacus.

[Dark Shikari]

CELT is a lossy low delay Audio Codec......With an energy-preserving quantizer that's equally applicable to video.A deblocking filter that removes blocks, not slight edge discontinuities. Its pathetic how useless it is for filtering i16x16 blocks, and even 8x8 blocks when they get zeroed which is when you really want a deblocking filter to work. It only really works on the 4x4 transform, which really shouldn't be surprising.For this you really want a lapping filter (i.e. invertible deblocking filter), not a typical deblocking filter.Transforms that aren't optimized for the processing limitations of a wristwatch. The transforms are no where near the largest source of decoding complexity, and yet they keep inventing new fancy sub-optimal transforms that will be oh so much faster on a system where multiplications are done by a newt with a binary abacus.I like transforms that don't require 32-bit intermediate precision. The H.264 transform is a mere ~1% worse than the idealized one anyways.

If you're going to spend more processing power on the transform, do something interesting with it (e.g. KLT) as opposed to just trying to add more internal precision, which isn't going to gain you much.

[*.mp4 guy]

For this you really want a lapping filter (i.e. invertible deblocking filter), not a typical deblocking filter.I disagree, Lapped transforms (you can call it a filter I suppose, since it happens before the transform) typically cause other problems that outweigh the benefits. Deblocking however, definitely works, and I think it is a mistake to assume it is limited in this aspect.
I like transforms that don't require 32-bit intermediate precision. The H.264 transform is a mere ~1% worse than the idealized one anyways.
It was more a general complaint of how they optimize for the wrong platforms, and rule out good options like 8x8 and 16x16 transforms for bad reasons. Since we have the new transforms, we might as well use them, but it was a waste of effort on a triviality and led them down bad design paths. the HCT is less then 1% faster then a well optimized dct on a modern system, so overall speed improvement of the codec is perhaps 0.125%, for 1% worse performance. That's not a clear win.

If you're going to spend more processing power on the transform, do something interesting with it (e.g. KLT) as opposed to just trying to add more internal precision, which isn't going to gain you much. as per my previous response, I don't think that the processing requirements are comparable here. However I otherwise agree, but would add that If you are going to spend a lot of time trying to optimize for something, at least optimize for something interesting, like the KLT.

[iwod]

How much of an improvement will energy-preserving quantizer?
10+%?

[schweinsz]

You often hear about people bitching who know the standard quite well (at least that's my impression) about how retarded some of it's aspects are, including it appears most of the proposals for H.265.

So, I was wondering, if you were in charge, whats the one (or two) thing you would change about the H.264 spec that would improve it the most in your opinion, and why.

Let's stay realistic though, so only ideas that could be reasonably decoded by today's cpu's and could reasonably have a hardware decoding chip implemented for.

And let's exclude killing of Interlacing too....that's a given :devil:

Thanks :-)

Larger block (32x32, 64x64, even 128x128) and larger transform (16x16), coefficient-adaptive and directional-adaptive directional interpolation filter, coefficient-adaptive deblock filter. Replace M-coder in the CABAC with the PA-coder to reduce the complexity greatly.