How to reorder frames by DTS?

[phib]

I'm doing some preliminary research about multiplexing h.264 data in live applications, especially the frame reordering part (getting frames ordered by DTS).
At the moment, I'm not focussed on a specific encoder. I'm rather trying to learn about h264 in general, respectively learning to understand the h264 spec.

Currently, my main interest is: How to determine the minimum number of frames that need to be buffered for correct frame reordering.
Ideally, I would like to determine this number by evaluationg the stream itself (e.g. SPS/PPS), without knowledge about the encoder used.

I understand that the ultimate upper boundary is the distance between two IDR frames, since all reference frames are invalidated when an IDR-frame is detected.
However, it seems that in practice encoders do not reference frames beyond the nearest I-frame(s).

This is one of the things Im really wondering about: What is the difference between I-frames and IDR-frames? I've read statements like "there could be a really
weird encoder referencing frames beyond the last I-frame, which would not be allowed for IDR-frames". So I guess, I virtually know the answer. But this raises
the next question: "When any practical encoder uses I-frames like they were IDR-frames, why aren't IDR-frames solely used in the first place?"

The frame-reordering-problem seems to get even worse when theres no following IDR-frame in sight. From what I've read, the reference picture indexing is done
via reference picture lists, which are populated with decoded frames on the fly. When I have a stream of encoded images in encoding/presentation order, I might
figure out that a certain B-frame references refImages "1" and "3". But how do I know that I have to wait for one of these, as my list of reference images already
contains images with these indices (but with outdated content).
Of course, once I receive the correct reference image, I can tell by the DTS.

I would appreciate any help on this, thanks for reading!

[Guest]

DTS is a transport stream level concept. You can't use it for AVC elementary streams.

You can, however, use the POC for re-ordering.

See the AVC spec for everything you need to know.

[phib]

Thanks for your answer.

I've read a bit more (especially this document avc_picmanagement_draft1.pdf (http://neuron2.net/library/avc/avc_picmanagement_draft1.pdf)),
and now I understand that it's actually the frame_num value which indicates decoding order. Apart from special cases like
gaps_in_frame_num_value_allowed, it should be pretty easy to reorder frames based on frame_num.

But I'm still wondering if there's a definite upper limit for the buffer size needed for frame reordering. The max number of b-frames (plus 1)
seems to be the value I'm looking for, but this seems to be rather a consensus between current encoder implementations than a specification requirement.

[Dark Shikari]

Thanks for your answer.

I've read a bit more (especially this document avc_picmanagement_draft1.pdf (http://neuron2.net/library/avc/avc_picmanagement_draft1.pdf)),
and now I understand that it's actually the frame_num value which indicates decoding order.No, it's the order in the stream which indicates decoding order.