Random access

[roeville]

I'm just starting to get acquainted with AVC/H.264 so apologies if this is a basic question.

What rules do I need to observe to randomly access an AVC stream and get perfect uncorrupted frames emerging from an AVC decoder? It's very easy with MPEG-2 as you just start from an I-frame, so I'm looking for the equivalent in AVC.

Can I simply start from an Access Unit Delimiter where the primary_pic_type is 0, or are there extra rules I need to take into account?

[Dark Shikari]

You can start decoding at an IDR frame or at a seekpoint SEI. In the latter case, the SEI specifies a number of frames after which you can start displaying the video to the user. This is usually zero, i.e. "you can seek to here", but not necessarily, as there are some very special-purpose encoders that use a continual intra-refresh to avoid ever having IDR frames and thus require you to wait a number of frames after the SEI to begin displaying the images.

[CruNcher]

Dark Shikari do you know why the stream gets unsync when streaming from Googles cache while you seeked forward in the file ?
they have a atom it's called gsst in the mp4 if that isn't 0 = beginning of the file, the stream has major sync issues in vlc such streams go hi wire jumping around in other demuxer they get unsync (Haali/Gabest). And How does the Flash Player can resync it then after the seek and all other fail ? is it Mainconcepts Demuxer that can avoid this seek unsync problem ?

[roeville]

Thanks. I'm not writing a decoder (yet) - I'm interested in where within the stream a file-reader/demuxer needs to begin sending data to a decoder.

You can start decoding at an IDR frame or at a seekpoint SEI.

Do you mean a recovery_point SEI?

So if you were writing a decoder then you'd only start outputting frames:

1. from an IDR or
2. from a recovery_point + recovery_frame_cnt frames ?

But I had noticed that sending data into a decoder from an arbitrary stream position causes the decoder to output several green (or partly green) frames before it started outputing perfect frames. I would have expected it to output NO frames at all until it reached an IDR or recovery_point but perhaps the designer preferred it to output something rather than nothing at all.

If that's the case then it seems you can't just rely on the decoder to start outputting frames only when it receives the first IDR or recovery_point.

So if I'm a file-reader/demuxer sending data to a decoder, then I need to start sending data from a byte position which will NOT cause the decoder to output any initial green frames. My guess is that I should start sending bytes from the Access Unit Delimiter which immediately preceeds an IDR or which immediately preceeds a recovery_point SEI (and hopefully the decoder would do the recovery_frame_cnt counting itself if necessary).

Does that sound OK?

[Dark Shikari]

Do you mean a recovery_point SEI?Yes.So if you were writing a decoder then you'd only start outputting frames:

1. from an IDR or
2. from a recovery_point + recovery_frame_cnt frames ?Correct.
But I had noticed that sending data into a decoder from an arbitrary stream position causes the decoder to output several green (or partly green) frames before it started outputing perfect frames. I would have expected it to output NO frames at all until it reached an IDR or recovery_point but perhaps the designer preferred it to output something rather than nothing at all.

If that's the case then it seems you can't just rely on the decoder to start outputting frames only when it receives the first IDR or recovery_point.Many decoders will assume that you are starting it at a valid point, and that if you aren't doing so, you're doing it for a good reason--they won't stop you from doing so. Generally the demuxer handles seeking.So if I'm a file-reader/demuxer sending data to a decoder, then I need to start sending data from a byte position which will NOT cause the decoder to output any initial green frames. My guess is that I should start sending bytes from the Access Unit Delimiter which immediately preceeds an IDR or which immediately preceeds a recovery_point SEI (and hopefully the decoder would do the recovery_frame_cnt counting itself if necessary).

Does that sound OK?The recover_frame_cnt might not be done by the decoder; remember, recover_frame_cnt simply tells you how long you have to wait until the frames outputted by the decoder are valid data. It may still output frames in the meantime.

Many of these decisions can in practice be implementation-specific in terms of how much the decoder wishes to assign responsibility to the demuxer.

[neuron2]

Starting at an IDR or recovery point is not always enough. You also have to have the active SPS and PPS.

[roeville]

You also have to have the active SPS and PPS

OK. So if the Access UD at the IDR or recovery point did not have a SPS or PPS following it (presumably the active one), then I'd have to seach backward for a previous SPS & PPS and send that (the Access UD followed by SPS & PPS) immediately before sending the Access UD of the recovery point.