interlaced avc

[bond]

ok with starting to analyse a bunch of avc hdtv samples i wondered about how interlaced is handled in avc:

there are four types of interlacing possible it seems:
1) interlaced (which means that all frames get interlaced)
2) paff: a mix of interlaced and progressive frames
3) mbaff: a mix of interlaced and progressive macroblocks in each frame
4) a mix of interlaced and mbaff frames


how to detect the interlacing used on an existing stream:
- progressive is signalled by setting frame_mbs_only_flag: 1 in the SPS
- interlaced is signalled by setting frame_mbs_only_flag: 0 in the SPS and field_pic_flag: 1 on all frames
- paff is signalled by setting frame_mbs_only_flag: 0 in the SPS and field_pic_flag: 1 on all frames that get interlaced and field_pic_flag: 0 on all frames that get progressive
- mbaff is signalled by setting frame_mbs_only_flag: 0 and mb_adaptive_frame_field_flag: 1 in the SPS and field_pic_flag: 0 on the frames (field_pic_flag: 1 would indicate a normal interlaced frame)

the mode of interlacing can be changed between the different SPSs


now my questions:
1) is this correct
2) regarding paff: can progressive and interlaced frames varry in one SPS sequence or only between different SPS sequences (one SPS seq is fully interlaced, another one is fully progressive)
3) how does the field_pic_flag need to be set on mbaff-only streams (so no mix with paff)?
4) a stream having frame_mbs_only_flag: 0 but still also all frames with field_pic_flag: 0 would be a normal progressive stream?

thx for the answers

[kwtc]

1. Perfect !
2. With PAFF, one can select progressive or interlaced for every frame, hence the same sequence may contain both types.
3. Mbaff frames are coded with field_pic_flag equal 0. Otherwise the frame is coded as two fields (usual interlaced), thus mixing mbaff + interlaced. However, it is
   not possible in the same sequence to mix mbaff frames and normal progressive frames.
4. Yes.

In other words
- frame_mbs_only_flag = 1 means progressive frames only.
- mb_adaptive_frame_field_flag = 1 means that pictures encoded with filed_pic_flag = 0 will use mbaff
- field_pic_flag = 1 means that the picture is encoded as two interlaced fields.

[bond]

Originally Posted by kwtc

Mbaff frames are coded with field_pic_flag equal 0. Otherwise the frame is coded as two fields (usual interlaced), thus mixing mbaff + interlaced. However, it is not possible in the same sequence to mix mbaff frames and normal progressive frames.

ok i see

so a frame with frame_mbs_only_flag: 0 and mb_adaptive_frame_field_flag: 1 and field_pic_flag: 1 will be a normal interlaced frame

why is it not possible to mix mbaff frames and progressive frames?

what about multisliced frames? is each slice treated like an own frame or are all slices which build a frame treated together as one frame?

thx for your answers!!

[Wilbert]

Just curious. What's the idea of interlacing on the macroblock level?

[bond]

Originally Posted by Wilbert

Just curious. What's the idea of interlacing on the macroblock level?

increased coding efficiency (aka lower bitrate needed) compared to normal interlacing or paff

[Gusto]

Originally Posted by bond

increased coding efficiency (aka lower bitrate needed) compared to normal interlacing or paff

I think it's exactly the opposite: MBAFF introduces some more bins/bits per MB compared to interlace, where this information is gathered on a higer level.

Best,

[Manao]

Gusto : indeed, MBaff adds 1 bit per 2MBs in order to signal whether the 2MBs will be field or frame. But firstly, this bit, in cabac, will take less than 1 bit in average. Secondly, encoding the pair of macroblocks in the correct mode doesn't increase the size, it reduces it. And it reduces it by far more than 1 bit.

[Guest]

Maybe this can enlighten you:

http://neuron2.net/guest/H264.pdf

[hworldjj]

Originally Posted by Gusto

I think it's exactly the opposite: MBAFF introduces some more bins/bits per MB compared to interlace, where this information is gathered on a higer level.

Best,

For interlaced movie, mbaff is definitely better than just frame mode because the correlation between the adjacent lines of the pictures is low in high motion regions. If the picture is arranged into two fields, the efficiency of dct and ME will be improved. This is the turth for MPEG-2 and H.264 and you can analyze the commecial DVDs

hworldjj

[Gusto]

Don't compare MBAFF vs frame mode in interlaced contents. What you can do is to compare:
1. MBAFF vs AFF in interlaced contents,
2. MBAFF vs FRAME in progressive ones

In the first case it really depends on the content. As you've mentioned MBAFF (or more generally macroblock adaptive coding) is widely used in a professional domain (e.g. sattelite broadcasters), but first of all it's because of the uncertain nature of TV conent: some parts of the screen may be interlaced, some may not (e.g. anchorman + bottom ticker + some on-screen graphs, text, etc.). In such a case MBAFF will give overally better coding performance compared to just AFF (interlaced or progressive).

In the second case you'll definatelly lose some bins/bits for coding of the MBAFF modes compared to just plain progressive, since making this decision on a mb level is not necessary and is a waste of bins.

[hworldjj]

Originally Posted by Gusto

Don't compare MBAFF vs frame mode in interlaced contents. What you can do is to compare:
1. MBAFF vs AFF in interlaced contents,
2. MBAFF vs FRAME in progressive ones

In the first case it really depends on the content. As you've mentioned MBAFF (or more generally macroblock adaptive coding) is widely used in a professional domain (e.g. sattelite broadcasters), but first of all it's because of the uncertain nature of TV conent: some parts of the screen may be interlaced, some may not (e.g. anchorman + bottom ticker + some on-screen graphs, text, etc.). In such a case MBAFF will give overally better coding performance compared to just AFF (interlaced or progressive).

In the second case you'll definatelly lose some bins/bits for coding of the MBAFF modes compared to just plain progressive, since making this decision on a mb level is not necessary and is a waste of bins.

First, it's meanless to use MBAFF or PAFF for progressive movies. Second the point is that MBAFF is always better than PAFF in most of cases due to the essence of natual videos, where only part of pictures needed to be encoded with interlaced mode.

hworldjj

[Manao]

Actually, if every decision in the encoder was made in RD, MBAFF would be better than PAFF on interlaced content, and than progressive encoding on progressive content.

[bond]

is it still considered to be a valid statement that mbaff is better than pure progressive encoding on pure progressive content?

if yes, why? ^^

[Manao]

It's a valid statement when the video has a lot of high vertical frequencies ( horizontal stripes for example ). A "classic" example is foreman. On that progressive video, roughly 10 % of the macroblocks will be encoded as interlaced. Of course, foreman isn't the average video ( it's highly aliased ), and it's definitively not representative of most of the DVD.

On more common videos, the additionnal 1 bit per 2 macroblock will, with CABAC, be transformed in 1/5 to 1/10th of a bit per 2 macroblock, so the overhead will be quite small, while interlaced encoding might still kick in if high vertical frequencies show up. But it would be a closer call ( and progressive might still win ).

[akupenguin]

My intuition is: no.
There will be a very few macroblocks where interlaced coding is better than progressive coding. i.e. if the content happens to contain textures that look combed.
But this is offset by the costs of mbaff: a fraction of a bit per macroblock to say "this block is progressive", and odd rows are unable to use the top-right neighbor for mv or intra prediction, and the resolution has to be mod32 (rather than the normal mod16), and the characteristic quantization artifacts of field coding tend to add combs (though this might be hidden if field coding is used only in the blocks that already contain such features).

Foreman isn't just aliased, it's deinterlaced. So it doesn't count as even abnormal progressive content.

[bond]

btw would it be possible to losslessly convert from a mbaff stream that contains only progressive macroblocks to a real progressive stream?

[akupenguin]

Yes. But it's not as trivial as cabac<->cavlc, since you have to reorder the macroblocks and reconsider which ones use skip/direct modes vs coded mvs.

[Manao]

Isnt there an issue with intra prediction ?

[akupenguin]

Oops, yes. If you use diagonal-down-left mode on the top-right sub-block of an intra macroblock in an odd row, mbaff changes the prediction algorithm since the top-right neighbor is unavailable.

[bond]

damn, i hoped that it would be possible to convert the hdtv avc mbaff broadcasts to pure progressive (with progressive content)

i kinda start to prefer paff over mbaff, cause with paff its possible to send pure progressive streams as progressive