Why 16 pixel steps?

[SeeMoreDigital]

Given that digital video encoding has moved on quite a bit in recent years (even months), why is it that many codecs (and their applications) still only generate encodes at 16 pixel steps.

I gather Mpeg2 can go down to 8 pixels, and Mpeg4 AVC down to 4... but why not 2 pixels?


Cheers

[Wilbert]

YV12 interlaced is minimal 2x4 pixels.

See http://www.avisynth.org/DataStorageInAviSynth

[Manao]

Furthermore, MPEG1, 2, 4 and H264 have a base unit ( the macroblock ) which is 16x16. If you use a non mod16 resolution, you'll not be optimal in the use of the codec.

And mod16 isn't that much a limitation.

[SeeMoreDigital]

Some time ago bond posted a link to the following chart (provided by Envivio): -

http://www.envivio.com/images/products/h264_features_matrix.jpg

How accurate is it? And do you think it might be useful to create a more detailed chart which includes other codecs?


Cheers

[Manao]

mpeg-4 can encode any valid YV12 resolution, iirc.

I don't know for mpeg-2.

[Teegedeck]

It actually ain't 16 for MPEG-4 ASP but 8. ...in a way!

To cite bond:

Variable Block Sizes:
in contrary to MPEG-4 ASP (where the Block Sizes can varry between 16x16 and 8x8 pixels (only with Inter4V)), AVC/H.264 offers the division of a macroblock down to 4x4 pixels. The Block Size is adaptive/variable, a good encoder will be smart enough to decide which one is the most efficient Block Size in every specific macroblock

And to cite Koepi:
Usually, you have just have a macroblock of 16x16 pixels.
You can store it as INTRA (key-block), INTER (motion-block).
If you choose a search-precision >4 (mode 5 & 6), you enable the inter4v mode.

INTRA stores the whole block as "texture" with zero movement.
INTER stores 1 motion vector for the 16x16 block.
INTER4V breaks down the 16x16 MB into 4 8x8 blocks and stores 1 motionvector for each subblock (4V like 4 vectors).

If you choose motion search precision >4 you won't get a file with all MBs stored as 4 vectors, this would take up a LOT of space.

[SeeMoreDigital]

Hmmm!

While playing about (like you do) I managed to generate the following: -

PAL anamorphically cropped - Test Cards (http://82.2.167.24/Uploaded_Files/Doom9_Forum_files/PAL_Exact_anamorphic_cropped_-_Test_Cards.zip)

One is down to 4 pixels the other is down to 2...


Cheers

[mpucoder]

MPEG-2 can go down to 8x8, but only for 4:4:4 sampling, which is very rare. You won't find it used on any distribution format (DVD, DTV, etc).

[Manao]

I still maintain that, even if block size is 16x16 in MPEG-4, resolution can go down to multiples of 2. In that case, the codec pads the frame in order to obtain a frame whose resolution is multiple of 16.

The same happens in h264 i think.

[RadicalEd]

The reason goes back to the days when JPEG was in development and the DCT was first being tested for image frequency decorrelation as a preprocessor for lossy compression. The 8x8 block size was found to represent the best trade-off between processor load and compression efficiency. IIRC 16x16 was found to be slightly better PSNR wise, but not enough so to warrant the extra cycles. It follows that 16x16 would be the standard block size, then, for YUV 4:2:0 frames, where chroma information is stored at 1/4 resolution and compressed in 8x8 blocks; 16x16 when scaled to the original size.

[akupenguin]

Originally posted by SeeMoreDigital
Some time ago bond posted a link to the following chart -
How accurate is it?
Rate/Distortion Optimization is a feature of the encoder, not of the standard, and thus doesn't belong on the chart. Also, there are ASP and MPEG2 encoders that support RDO, including XviD, ffmpeg, and DivX5.

Originally posted by Teegedeck
It actually ain't 16 for MPEG-4 ASP but 8. ...in a way! That's exactly the same way in which H.264 supports 16x16 through 4x4 block sizes. (H.264 additionally uses a 4x4 DCT, but that's not what the chart means by "variable block size".)

Another feature that might be useful to compare: H.264 specifies an integer approximation of DCT, which allows and requires bit-exact decoders. Whereas ASP and MPEG2 don't have one standard DCT, so each implementation uses a slightly different version. Result: H.264 can use arbitrarily large keyframe intervals and lose only seeking.

And do you think it might be useful to create a more detailed chart which includes other codecs? When you're comparing codecs developed independently, I don't know how useful it is to compare high-level features like that. But certainly some others could be added.

[SeeMoreDigital]

Thanks for the information akupenguin,

Now... is there anybody here who could create a more suitable chart?


Cheers

[akupenguin]

Here's my chart:
http://students.washington.edu/lorenm/src/mpeg_features.png
HTML version (http://students.washington.edu/lorenm/src/mpeg_features.html).

Note: Snow's motion block sizes are not directly comparable to the others, due to OBMC. e.g. 32x32 overlapped blocks have 1 MV per 16x16 pixels.
Note: H.264 FRext profiles also contain a bunch of extra colorspace options that don't fit in the table (4:2:2, 4:4:4, YCoCg, RGB, up to 12bit sample precision)

edit: updated per virus' and manao's suggestions

[virus]

:confused:

4x4 DCT?

AFAIK H.264 uses a couple of 4x4 integer transforms which are just approximations of a proper DCT: one is the proper Hadamard transform (for DC coeffs in 16x16 blocks), and the other a somewhat modified Hadamard transform (for AC coeffs) - plus an optional 8x8 integer transform in the High profiles.

[akupenguin]

Fixed.

The standard never names it anything - just "the transform". And most discussions, including the x264 source, call it "and integer approximation to DCT". But yes, I see that the matrix is the same as the Hadamard transform used in motion estimation.

In P, B, and I4x4 blocks, it's just the 4x4 Hadamard.
In I16x16, each 4x4 subblock gets transformed, and then the 16 luma DC coeffs get an additional 4x4 Hadamard, and the 4 chroma DCs get a 2x2 Hadamard. This composite is similar, but not identical, to a 16x16 Hadamard.

[Manao]

Slices are also present in MPEG-4, though they may differ from slices in H264.

Interlacing in MPEG-4 is also adaptive ( it's roughly the same as MBaff, except that you don't encode mb by pair, you cut a mb in two fields ), but you don't have Paff. You also have field DCT ( there again, it's a mb decision )

[virus]

yet another boring remark :D

IMHO in the H.264 "coding" box, it would be better to replace "Huffman or arithmetic" with "Exp-Golomb or Arithmetic" because most of the syntax elements in CAVLC are effectively Golomb-coded (while AFAIK in MPEG-4 Visual most of the predefined VLCs have been built using a proper Huffman tree).

[akupenguin]

Originally posted by Manao
Slices are also present in MPEG-4, though they may differ from slices in H264.
I can't find anything related to slices in ISO-IEC 14496-2 (2001). It does use the term "slice", but it means simply a packet, nothing to do with H.264 slices.

[Manao]

It's different from video packet iirc. Slices are just a way to group consecutive mbs, making them independant from the mbs in other slices. It's only useful when hardware can't encode in mpeg-4 a full frame ( because internal memory is too low ).

[akupenguin]

Slices are for error resilience. They do not save memory: you still have to keep a copy of the whole frame in memory, for motion compensation to predict the next frame. And they don't save cache either, because you can encode one macroblock at a time even without slices (x264 does).

[RadicalEd]

IIRC, AVC High Profile/FRExt adds custom quant matrices.

[Mug Funky]

come to think of it, mpeg-1 and 2 support custom quant matrices. i think this is another encoder-specific thing like R/D.

[Manao]

Mug funky : no, custom matrices need to be known by the decoding applications ( else they can't build back the right dct coefficients )

akupenguin : slices in mpeg-4 are only a hardware encoding tool, and don't need to be known in the decoder ( <--- they aren't a part of the norm, so I was wrong to speak of them in the first place ).

However, they are necessary when you work on a DSP, on which you can't get a fast access to the whole frame ( because internal memory is only ~128 KB ). So you copy in the internal buffers only the part of the source picture you're working on ( and the collocated part of the reference picture, extended by the mv range )

Edit : FRext adds also 8x8 transforms and noise parameters transmission in the SEI, which i can't wait to see implemented.

[SeeMoreDigital]

I thought this topic might generate some interesting views, not to mention new/revised developements ;)


Cheers

[Mug Funky]

oh? hmm. in that case somebody's going to have to explain to me what the difference between "custom matrices" and the quant matrices available in TMPG, CCE, QuEnc, etc is. i thought they were the same.

[Manao]

None, they work in the same way. In both cases, you need to write down the custom matrix in the bitstream in order the decoder to know how to get back properly the dct coefficient. The only difference in that in MPEG-4, you also have the h263 quantization which doesn't require a matrix.

[mpucoder]

"custom matrices"? MPEG-1 and MPEG-2 convey the quantisation matrices (there can be 2) in the Sequence Header. The decoder always knows what was used for encoding, and it can change frequently. For DVDs there is a new sequence header every GOP.

[Trahald]

Originally posted by mpucoder
For DVDs there is a new sequence header every GOP.

Well , you dont have to have them every GOP.. as long as you have small gops ;)

[Soulhunter]

Originally posted by mpucoder

For DVDs there is a new sequence header every GOP.

Does this mean you could theoretical use a different matix @ every GOP ???


Bye

[fccHandler]

In theory you could use a different matrix for every picture! MPEG-2 has a "quant matrix extension" which can serve that purpose.

[akupenguin]

Wasn't there an option in XviD that switched between h263 and mpeg quants on a per-frame basis?

[BoNz1]

Originally posted by akupenguin
Wasn't there an option in XviD that switched between h263 and mpeg quants on a per-frame basis?

Yes, it was called modulated quantization IIRC. But of course there were problems with it as you might imagine. BTW I think the latest CinemaCraft encoder has the ability to switch between quantization matrices.

[shlezman]

Originally posted by Manao

akupenguin : slices in mpeg-4 are only a hardware encoding tool, and don't need to be known in the decoder ( <--- they aren't a part of the norm, so I was wrong to speak of them in the first place ).

However, they are necessary when you work on a DSP, on which you can't get a fast access to the whole frame ( because internal memory is only ~128 KB ). So you copy in the internal buffers only the part of the source picture you're working on ( and the collocated part of the reference picture, extended by the mv range )


MPEG-4 has resync markers which together with HEC information are used as video packets (Annex E)which can be decoded independently. This feature is used for error resilience. Slices/Packets are also available in H.263 called GOB.

[akupenguin]

Any other formats people want added to the chart? (Open standards only, please.)

I have a draft of VC-1 here, but it's a mess and I'm not going to read 420 pages just to find out all the little ways in which it differs from H.264.