Determining the slices that make up a frame

[sameersbn]

Hi,
I am writing an application that provides compressed h.264 data to a hardware decoder to perform the decode. The hardware decoder has a requirement that the data given to the decoder must have one complete frame, or else the decoder returns an error during the decode process.

Earlier i considered that all slices between two access unit delimiters nal units make up one complete frame and this worked very well until i can across streams that do not have the access unit delimiter nal units at all.

Since then i have been trying to figure out a way to determine which slices make up one complete frame. At first i considered that slices belonging to the same frame would have the same frame_num, and simply when the frame_num changes it would mean that this slice belongs to the next frame. But, however, as it turns out this did not seem to be the case as i saw slices that belonged to different frames that had the same frame_num. This i have checked in Elecard StreamEye and it too says that the paricular slice is of a different frame while the frame_num field is the same.

Please can you tell me how to determine whether a particular slice contain one complete frame and if not then how to determine the slices that belong to one frame.

Thank and Regards
~Sameer

[Dark Shikari]

POC, not frame num.

[sameersbn]

Hi,
By POC do you mean Picture Order Count?

[Dark Shikari]

Hi,
By POC do you mean Picture Order Count?Yes, that's what POC means.

[sameersbn]

Hi,
This would mean that i would need to decode the POC and when the POC changes that would mean the slice belongs to the next frame?
Regards
~Sameer

[Dark Shikari]

Hi,
This would mean that i would need to decode the POC and when the POC changes that would mean the slice belongs to the next frame?
Regards
~SameerYes. All slices in a frame have the same POC. Slices not in the same frame do not have the same POC.

[Shevach]

Yes. All slices in a frame have the same POC. Slices not in the same frame do not have the same POC.

If pic_order_cnt_type is 2 then POC is not transmitted at all.
How in such case one can distiguish frames/pictures?

[Dark Shikari]

If pic_order_cnt_type is 2 then POC is not transmitted at all.
How in such case one can distiguish frames/pictures?By finding the person who wrote the encoder and hitting them with a large stick.

[Shevach]

By finding the person who wrote the encoder and hitting them with a large stick.

This is the very reason why I don't write encoders.
By the way if profile is Baseline then one can use first_mb_in_slice parameter to detect a new picture. If first_mb_in_slice=0 then the current slice is the very first slice in a picture.
In order to find frame boundary one can check field_pic_flag.

[Guest]

How in such case one can distiguish frames/pictures? The AVC spec has an explicit algorithm for how to detect frame boundaries in all case. See section 7.4.1.2.4.

[sameersbn]

This is the very reason why I don't write encoders.
By the way if profile is Baseline then one can use first_mb_in_slice parameter to detect a new picture. If first_mb_in_slice=0 then the current slice is the very first slice in a picture.
In order to find frame boundary one can check field_pic_flag.
Is this not true for main profile as well?

[sameersbn]

The AVC spec has an explicit algorithm for how to detect frame boundaries in all case. See section 7.4.1.2.4.

Just making sure i am refering to the correct spec (excuse me)
7.4.1.2.4 Detection of the first VCL NAL unit of a primary coded picture
This is correct right.

[Guest]

That is correct. AVC is a bit clunky in this regard. Here's the code I use (adapted from JM reference decoder). If hadpic == 1 at the end, then you have the first NALU of a new picture. You can easily see how this maps to the spec.

hadpic = 0;
if ((img->frame_num != prev_frame_num) ||
(currSlice->pic_parameter_set_id != prev_pic_parameter_set_id) ||
(img->field_pic_flag != prev_field_pic_flag) ||
((prev_nal_ref_idc != img->nal_reference_idc) &&
((prev_nal_ref_idc == 0) || (img->nal_reference_idc == 0)))
)
{
hadpic = 1;
}
if(img->field_pic_flag && prev_field_pic_flag)
{
if (prev_bottom_field_flag != img->bottom_field_flag)
hadpic = 1;
}
if (prev_idr_flag != img->idr_flag)
hadpic = 1;
if (active_sps->pic_order_cnt_type == 0)
{
if (prev_pic_order_cnt_lsb != img->pic_order_cnt_lsb)
hadpic = 1;
if (prev_delta_pic_order_cnt_bottom != img->delta_pic_order_cnt_bottom)
hadpic = 1;
}
if (active_sps->pic_order_cnt_type == 1)
{
if (prev_delta_pic_order_cnt0 != img->delta_pic_order_cnt[0])
hadpic = 1;
if (prev_delta_pic_order_cnt1 != img->delta_pic_order_cnt[1])
hadpic = 1;
}
if (prev_idr_flag == 1 && img->idr_flag == 1 && (img->idr_pic_id != prev_idr_pic_id))
hadpic = 1;

prev_frame_num = img->frame_num;
prev_field_pic_flag = img->field_pic_flag;
prev_bottom_field_flag = img->bottom_field_flag;
prev_pic_parameter_set_id = currSlice->pic_parameter_set_id;
prev_nal_ref_idc = img->nal_reference_idc;
prev_idr_flag = img->idr_flag;
prev_idr_pic_id = img->idr_pic_id;
prev_delta_pic_order_cnt0 = img->delta_pic_order_cnt[0];
prev_delta_pic_order_cnt1 = img->delta_pic_order_cnt[1];
prev_pic_order_cnt_lsb = img->pic_order_cnt_lsb;
prev_delta_pic_order_cnt_bottom = img->delta_pic_order_cnt_bottom;

[sameersbn]

That is correct. AVC is a bit clunky in this regard. Here's the code I use (adapted from JM reference decoder). If hadpic == 1 at the end, then you have the first NALU of a new picture. You can easily see how this maps to the spec.


Well, thank you very much.

Regards
~Sameer

[sameersbn]

I have made the necessary additions as mentioned by neuron2. Its working well now.
Thanks for all the help.

Regards
~Sameer

[Shevach]

Is this not true for main profile as well?

Opps. I have mistaken. I meant - If profile is not Baseline ...