Extracting the frame referencing information

[Sangwook]

I'm trying to extract the frame (slice) information from the mp4 file that contains the h264 encoded media. Especially, I need the slice size, slice type, DTS, reference frame list.

Through the stss box and stsz box, I can extract the information about the size, type.

However, I cannot find any information about how to extract the reference frame list for the certain . Which field or box should I look into for extracting frame referencing information? (For example, in case of B frame, the B frame refers the other frames frame 3 and 4)

The main reason of extracting these information is that I am trying to construct the frame dependency map. (e,g Frame A which is 15235 bytes size needs Frame B, C for decoding, Frame B's size is 4222 bytes and need frame D ...etc)

Thanks.

[nevcairiel]

I don't think MP4 holds that information, you would need to go deeper and actually parse the H264 bitstream to get it.

[Sangwook]

Thanks for the reply.

I am looking into the ffmpeg codes, but how can i extract those information through the ffmpeg? Right now, I am trying to extract motionvectors and parse it. However, it also doesn't contain the referencing info.

[LoRd_MuldeR]

What about looking at the specs?
https://www.itu.int/rec/dologin_pub.asp?lang=e&id=T-REC-H.264-201602-I!!PDF-E&type=items

8.4 Inter prediction process8.4.1 Derivation process for motion vector components and reference indices


Keep in mind that each macroblock or even each partition can select its own reference frame. So, I think, a single frame may end up referencing all frames that are in the DPB, by the time that frame gets decoded.