Find key pictures in streams with no IDRs

[Dust Signs]

Hi,

I'm currently dealing with a lot of European broadcasts and I noticed that there are a lot of H.264 streams being broadcasted with not one single IDR in it. There are some I pictures of course, but how can I distinguish an I frame intended to start a new GOP and an I picture which was just inserted for p.e. a scene change where the pictures coming after it refer to some picture before that I picture? POC 0 as an indicator seems to work sometimes, but not always. Is there any additional information I'm currently missing which indicates that an I picture has been used instead of an IDR picture?

Dust Signs

[Dark Shikari]

Seekpoint SEI?

[Dust Signs]

But SEIs are optional, aren't they? As I also get "properly seekable" streams with IDR pictures in them which I want to process in the same way I cannot rely on the existence of SEIs in the stream. As I can only process the stream in real time I cannot distinguish between streams with I pictures only and streams with IDR pictures as I cannot go back as I have to perform real time processing. But thanks for the hint. Any other ideas?

Dust Signs

[Dark Shikari]

But SEIs are optional, aren't they? As I also get "properly seekable" streams with IDR pictures in them which I want to process in the same way I cannot rely on the existence of SEIs in the stream.Yes you can, because the decoder can only start playing at an IDR or seekpoint SEI. If neither of these existed, your television decoder would never be able to start playing the video.

[Dust Signs]

That makes sense. Helps me a lot :). Thanks

Dust Signs

[Manao]

seekpoint SEIIts proper name is "recovery point"
If neither of these existed, your television decoder would never be able to start playing the video. You don't stream a raw h264 file. There's always a container, and it can have additional information. TS for example has RAI/RAP (random access indicator/point).

[akupenguin]

the decoder can only start playing at an IDR or seekpoint SEI. If neither of these existed, your television decoder would never be able to start playing the video.
If your stream uses reference list reordering, or if you assume no longterm refs, then recoverypoint SEIs aren't strictly necessary: they can be inferred by examining a finite number of neighboring frames. :devil:
IOW, just start decoding wherever, and keep track of which pixels are contaminated by unknown values. If there are recovery points at all, the set of contaminated pixels will empty shortly after you cross one, and then you can start displaying.
(The assumptions are necessary because with longterm refs and no reordering you can't tell which frame a given ref idx points to, and so you can't construct the contamination map.)

[Shevach]

Small comment.

Pictures containing memory_management_control_operation equal to 5 can be used as random access points.
Indeed, if a picture has mmco=5 then at the end of decoding of this picture DPB buffer should be emptied (i.e. all pictures in DPB as declared as non-used for reference).
Therefore the next picture can be used as the entrance to safe decoding.

[akupenguin]

The standard essentially considers mmco=5 to be identical to IDR. I don't know why they even have two options for coding the same info.

[Shevach]

In typical applications mmco=5 actually makes the next picture to IDR one.
But there is a subtle diffrence - the syntax element no_output_of_prior_pics_flag which presents only in IDR slice header.
Indeed, when a command mmco=5 arrives, all pictures that were previously in the DPB are marked as "not used for reference". But that doesn't mean that they are removed from DPB. Actually they "live" in DPB until their output times arrive.

When IDR picture with no_output_of_prior_pics_flag=1 arrives then all pictures in DPB are removed regardless theirs output times.

That is very useful feature when picture size is not constant.

The standard essentially considers mmco=5 to be identical to IDR. I don't know why they even have two options for coding the same info.

[akupenguin]

Indeed, when a command mmco=5 arrives, all pictures that were previously in the DPB are marked as "not used for reference". But that doesn't mean that they are removed from DPB. Actually they "live" in DPB until their output times arrive.
Which is exactly what happens on an IDR with no_output_of_prior_pics_flag=0.
And if you change resolutions, then the standard says that a decoder is allowed to infer no_output_of_prior_pics_flag=1 and act as such, regardless of what the stream says.
Either way, how does mmco=5 help?

[Shevach]

Which is exactly what happens on an IDR with no_output_of_prior_pics_flag=0.
And if you change resolutions, then the standard says that a decoder is allowed to infer no_output_of_prior_pics_flag=1 and act as such, regardless of what the stream says.
Either way, how does mmco=5 help?

You are right no_output_of_prior_pics_flag is inferred to one if a change of resolution occurs.
The question is under what circumstances it is desirabe for an encoder to signal no_output_of_prior_pics_flag=1 (i.e. to empty DPB regardless output time of pictures)?

For example in case of non-seamless splicing it might be useful to set no_output_of_prior_pics_flag=1 at the start of a new excerpt.