Calculating frame count from MPEG-PS headers' PTS/DTS

[konran]

Hello,

I hope this is the right place to post this one: I'm writing analysis software for MPEG ES+PS files and I need to determine the frame count of a movie without reading the whole thing.

With PS I thought it would be easy: Fishing out PTS and/or DTS from first picture header and from last picture after the last GOP header is found - calc the difference and finally done.

The problem is: I get 1 or 2 frames less by PTS/DTS calculation vs. reading all frames of a file. For the moment I'm ignoring the audio stream(s) silently. Do I have to consider PTS or DTS also for audio streams?

Are there other packet types beneath picture headers, pack headers, system headers, sequence headers, slice blocks and GOP headers to be taken into account to get a proper result?

Any suggestions surely will help :-)

Regards,
Konran

[neuron2]

There is not necessarily one PTS sent for every video frame. That complicates getting exact results using this method.

[konran]

However, if it complicates the thing how can it be done to get an exact result?

For the moment I take the first incoming PTS/DTS of the primary headers and assure to keep the minimum. On the final part I read the last sequence header (if available), then the last GOP and finally all video frames after that. Here I keep the maximum PTS/DTS sample.

The best result of this for now is max(last PTS, last DTS) - min(first PTS, first DTS). In many cases this leads to the same duration (=> frame count) as reading all frames of the file. But it's not exact on every file. Anyway, is it possible at all getting exact results for a file's duration? And if yes, how?

Regards,
Konran

[neuron2]

Find the pictures associated with the first PTS and the last PTS. Calculate and count the frames between them as you are doing. Now it may be incorrect because you can have frames before the first PTS and after the last PTS, because as I said, there is not necessarily a PTS for every picture. You'll have to parse the video before the first PTS and after the last PTS to count those frames.

Also, be aware that this whole scheme fails miserably if there is a PTS discontinuity, and that is more common than you might think.

[konran]

Thanks :-)
Also, be aware that this whole scheme fails miserably if there is a PTS discontinuity, and that is more common than you might think.
Ooh, that's not nice ;-(
In fact I'm doing a full frame read after determining the most possible "best" frame count and I only don't want to read the file twice as a whole.
So, as you wrote, I assume I don't have to take care for DTS'es - right?

[neuron2]

The difference between PTS and DTS reflects frame re-ordering. If you have a final GOP that has a PTS associated with the I frame, then if the GOP is open or closed will change how many frames you have to count and add at the end.

It's all very tricky and, as I said, the scheme is totally destroyed by any PTS discontinuities. What is your application that requires such an accurate frame count? Maybe you have to accept that you can't get an accurate frame count without scanning the entire file.

[konran]

If you have a final GOP that has a PTS associated with the I frame, then if the GOP is open or closed will change how many frames you have to count and add at the end.
Hum ... I do not quite understand the influence of the GOP status - I mean if it is open or closed, what does it matter for the frames following the last GOP if it's closed or not? For the moment I disregard the closed GOP flag and I consider all frames following the last GOP header to be counted. But I do this only on ES files when I don't have any PTS'es. When I have a PS I look for the last PTS I get after the last GOP header in every frame that follows - and I'll extend this to check for the amount of frames that don't have a PTS after the last frame that has one.
It's all very tricky and, as I said, the scheme is totally destroyed by any PTS discontinuities. What is your application that requires such an accurate frame count? Maybe you have to accept that you can't get an accurate frame count without scanning the entire file.
Well, maybe that is not too important for that kind of application - it's a bitrate viewer (that will be fully functional in place of that one I've used before which is buggy). There is a preview painted while reading the file and I need a (quite) accurate frame count of the file to scale the view that I have to determine before the view is displayed the first time.

I know the frame count does not have to be so very accurate ... 1% deviation of the real frame count does not really matter very much. But I thought as I'm writing an interface to MPEG header parsing of my own: If any MPEG editor can open my files and show a frame count that is accurate for the file without reading the whole thing (as long the entries mentioned before meet the MPEG spec's requirements), why shouldn't I be able to get the same result?

Of course, if I get a file that can't be calculated well I'm going to count frames after the last GOP and use GOP times which are also not a good base to use (especially for some kind of VRO material) and if that fails too I interpolate some kind of file size and frame rate - that's better than nothing.

Anyway - as long I have a chance of getting it accurately I'll do it. Then I have an interface I can use also for other applications which need it more accurately.

[neuron2]

Consider this final GOP in coding order and assume you have a PTS for it (i.e., for the I frame):

I0 P3 B1 B2 P6 B4 B5 ...

In display order that is:

I0 B1 B2 P3 B4 B5 P6 ...

In this case, frames B1 and B2 and all those following would not be included in a calculation involving the PTS.

Now consider an open GOP in coding order:

I2 B0 B1 P5 B3 B4 ...

and in display order:

B0 B1 I2 B3 B4 P5 ...

In this case, frames B1 and B2 are already included in the calculation involving the PTS.

The PTS refers to the first access unit in coding order, i.e., the I frame, but the positioning of that frame in display order relative to the other frames depends on whether the GOP is open or closed.

For a bitrate viewer, don't you have to scan the entire file to get correct results?

[konran]

Thanks a lot. I had to read it several times to understand it ... and I think I got it. I'll experiment a few things with it and see what happens...
For a bitrate viewer, don't you have to scan the entire file to get correct results?
Yes, of course. But while scanning the whole file I'm displaying a preview that cumulates all samples reached so far. That looks much better instead of waiting for the full view until it's all scanned. So, for a proper preview window scaling I have to know how many frames to be expected at all.

[neuron2]

Just display it based on the number of frames seen so far. You can rescale it if needed.

[blutach]

Hi again konran, glad to see you here and I've been following this thread with interest.

You could scale initially to the max and just rescale at the end. And yes, it is quite common for PTS disconts. Look in some DVDs you might have.

How fully featured is the viewer going to be, if I might ask?

Regards

[konran]

@neuron2: So you see, flavours are different. My flavour disallows dynamic rescaling from some frames up to any big amount. That would look very jumpy.

You could scale initially to the max and just rescale at the end. And yes, it is quite common for PTS disconts. Look in some DVDs you might have.
Fine to see you again here, Les :)
It's actually your kind of scaling that I'm using for now. At the end it makes only some frames difference and it doesn't seem to be jumpy - only for two cases:
1) files are very short - less than the amount of seconds that fit on the view's width
2) ES streams where I don't get any PTS's so that I'm using GOP times :scared: and e.g. these streams are VOB/VRO demuxed ES'es which tell my MPEG encoders that they have for e.g. 12 minutes in length and they are 3 minutes in real :mad:
Actually nobody can deal with 2) - so don't I - I needn't be a magic wizard :)
How fully featured is the viewer going to be, if I might ask?
Well, the main framework which actually reads the whole sampled file of interest is based on ffmpeg/libav. So beneath reading all kinds of MPEG it is capable of reading all codecs that ffmpeg can deal with. That was not my intention - but who cares? All video frames of an input file are raw read and sampled. Three methods of bitrate calculation are available - as I've discussed that part of math with a friend we first concluded to base it on GOPs. My idea was to base it on seconds as bitrate is measured in bits per second. The conclusion of both is a weighted GOP algorithm that takes care of short GOPs and collects them until having a "bigger" GOP that doesn't make overshooting bitrate samples. OK, what more to tell ... here is a screenshot ... you might see that the GUI strongly resembles DVD-lab PRO's bitrate viewer.
I was using it for the last five years and it always struggles with the 2GB file size boundary. As I like this kind of GUI very much I've designed it similar. My first shot will come as freeware and I'm nearly done with it ... Windows Installer has to be done and it will get a HTML help file and it has even a headline. I'm no native speaker and I get stuck with writing long summaries on it :scared:
For the GUI the last thing I want to implement is Y autoscaling for stuff like MPEG1'es that have very small bitrates or MP4's/DivX's/WMV's and the like where the bitrate maximum is even smaller.

Here we go:
http://www.lichterleben.de/ftp/pub/BitrateViewer/BitrateViewer-500s-PS-muxed-MP2.jpg
there are also some menues:
http://www.lichterleben.de/ftp/pub/BitrateViewer/BitrateViewer-SysMenu.jpg -and- http://www.lichterleben.de/ftp/pub/BitrateViewer/BitrateViewer-TrayWnd-ContextMenu.jpg

Regards,
Konran

[konran]

Well ... BTW, I forgot an important question on this open/closed GOP thing:
When I have open GOPs, what is it on the first GOP? When B-frames are reordered I think (having 3I 4P 1B 2B 7P 5B 6B) that the first two B-frames will be ordered before the first I-frame by the decoder:
1B 2B 3I 5B 6B 7P
Is that correct? And if so, when is PTS delivered - spec says it is on the first access unit - from my point of view that is the I-frame (I hope?!). In this case the "real" start time is two frames before the I-frame's PTS starts - correct?

Addendum: My PTS/DTS and GOP time analysis is not part of ffmpeg - that's my own driven interface coupled into the whole thing.

[neuron2]

When I have open GOPs, what is it on the first GOP? If depends on whether they have become "first" because they were the first frames encoded by the encoder, or because they have become first because GOPs before them have been edited away. In the former case, a smart encoder will do this: For the first GOP, the leading B frames in display order have macroblocks encoded to use only predictions from the I frame and to not use predictions from the P frame in the (nonexistent) previous GOP. In the latter case, your leading B frames have orphaned references and are nondecodable.

When B-frames are reordered I think (having 3I 4P 1B 2B 7P 5B 6B) That's not correct MPEG2 ordering. It's conventional to start at 0, not 1, but using your unconventional notation, it should be:

1I 4P 2B 3B 7P 5B 6B # closed GOP

3I 1B 2B 6P 4B 5B # open GOP

Apparently, you don't understand the MPEG reordering rules. Refer to the specification.

that the first two B-frames will be ordered before the first I-frame by the decoder:
1B 2B 3I 5B 6B 7P
Is that correct? See above.

And if so, when is PTS delivered - spec says it is on the first access unit - from my point of view that is the I-frame (I hope?!). Correct.

In this case the "real" start time is two frames before the I-frame's PTS starts - correct? Yes.

[konran]

Thanks, that makes me understand it much more :)
For the first GOP, the leading B frames in display order have macroblocks encoded to use only predictions from the I frame and to not use predictions from the P frame in the (nonexistent) previous GOP. In the latter case, your leading B frames have orphaned references and are nondecodable.
Indeed I have two test files which are the like: they have open GOP's at the start and end of the file. As I understand the first two B frames don't have a reference to nothing and therefore won't be decodable. I'm wondering why an editor should do that anyway. I can play these files with any MPEG player on my PC and I don't see any problems from start.
It's BTW of course interesting to know what a decoder will do in this case. But for my applications this is not very important. I think these two B frames count for existing in the whole file independently if they can be decoded or not.
Apparently, you don't understand the MPEG reordering rules. Refer to the specification.
Apparently you're right :confused:
... but I'm on the way to understand it better. From the start of the project it was never my intention to know all these things in detail - meanwhile I see that it is necessary to know. So I hope I will understand it - at least for that amount of time needed :)
And meanwhile I also see that the whole thing is really more difficult than expected as (regarding the two test files mentioned) a PTS and/or DTS is not delivered with every I frame. Now I'm tracking the delivery of PTS/DTS and saving the frame number in the current GOP and the frame type so that I can evaluate later for the case of open GOP's the number of B frames to be substracted at first and last GOP and the number of frames added after the last PTS time stamp delivered.
Also it might be better using DTS instead of PTS - I think in this case I'm not involved in frame reordering that strong because the DTS tells about the decoders input position.

I'll have to test some things 'till I'm back.

Regards

[gurpreet1989]

hello


can anyone tell me how to find number of gop in one mpeg file. and number of pictures in one gop. also tell me about the number of pack header in mpeg file and number of audio and video streams in one mpeg file.

is there any byte to show these all……

[gurpreet1989]

can you tell me how to calculate number of frames in one mpeg file.....

[konran]

can anyone tell me how to find number of gop in one mpeg file. and number of pictures in one gop. also tell me about the number of pack header in mpeg file and number of audio and video streams in one mpeg file.

is there any byte to show these all……
Answered your last question first: NO, there isn't. Most of the things you want to know are not pre-determined (assuming you want to analyze MPEG-1 or MPEG-2). I've spent some month during my first part of development of BitrateViewer to get all these informations. You will be best served reading the ISO/IEC 13818-1 and ISO/IEC 13818-2 documents unless you fiddle all of it yourself by looking into tons of MPEG streams.

- number of video and audio streams are given in the program stream map header; normally you'll find one somewhere in the first part of a MPEG-2 system stream.
- the number of GOPs overall is not pre-determined, so you have to do a full file scan to know that
- the number of frames in one GOP can vary, it is said that PAL has a maximum of 15 frames per GOP and NTSC a maximum of 18 but there is no information how many frames are stored actually in a GOP. As long as a MPEG decoder can handle more than the pre-defined maximum length it will play it w/out complain. The frame number per GOP is not a fixum during a whole file - think of scene changes when you insert an I-frame explicitly before a GOP has reached its maximum length. I've also found MPEG sample files on FFMpeg server that have only one frame per GOP.
- what do you want with pack headers? they are also not pre-determined, you have to do a full file scan.
- number of frames in MPEG file: when you've followed this whole thread you should have got an idea how to do that ;-)
For BitrateViewer I make a full file scan after I've read some peaks in the front and end of a file and calculating some bits of analysis from those chunks. I mostly can pre-determine the number of frames when I've got the first sequence header, the first GOP, last sequence header, last GOP and optionally a program stream map. Also optional is the fact having PTS/DTS informations. That only works pretty well with a fixed frame rate over all length. The real number of frames that I get after my full file scan is the same in many cases as pre-calculated but sometimes it differs for a handful of frames.

Maybe you can try using BitrateViewer to see what I mean ... it is free, yet.

Konran

[gurpreet1989]

you said that number of audio and video streams present in program stream map header. But in some file it is not present then how to calculate number of video and audio stream. can you show me any program stream map header in details with example.


I got a formula to calculate number of frame that is number of frames= frame rate*length in seconds.
but it does'nt gives the right number of frames. like i have a video of 30sec having frame rate = 30fps from formula number of frames=900 but it contains frames=930......

please tell me how to find correct number of frames in a mpeg file.

and also i want to know about the pack header of mpeg file. which things we get from it. and the meaning of that things.

plz reply

[gurpreet1989]

IS it possible to find length of mpeg file from it's bytes..........
if yes please tell me about that

[gurpreet1989]

Is it possible to find Number of bytes in I,B,P Frame of mpeg 1 or 2 video file...........

[drmpeg]

Is it possible to find Number of bytes in I,B,P Frame of mpeg 1 or 2 video file...........
I have a program that I use to dump the MPEG-2 telecine flags, but it also dumps the frame size (in bits).

http://www.w6rz.net/flags.zip

Usage (in a Command Prompt window) is:

flags <infile>

The file has to be an MPEG-2 elementary stream.

Ron

[gurpreet1989]

Thanks drmpeg

It's ok you have a program ......and thanks again for it........

Will you tell me is there any bytes in mpeg format which tell me about the size of I,B,P Frame or any formula to calculate it........


And is it possible to find number of frames, Number of Gop, Number of slices in one frame, number of macro block in one slice, Number of Bytes in one macro block of mpeg file................and tell me is it possible to find number of audio and video stream in one mpeg file..................Number of pack Header in one mpeg video file..........

I know i ask so many question ...........But it's compulsory in my research work project on mpeg video file format ........please help me...........

[gurpreet1989]

drmpeg

Please reply.........

[gurpreet1989]

can anyone tell me IS mpeg(0xBA, 0xBB, 0xB8..............) Header are in compressed form or not..........Is they are in compressed form then is there any software to decompress the mpeg file?


please reply.....

[neuron2]

@gurpreet1989

This is another thread hijack. Stop posting in this thread unless it is on topic to the thread (refer title and original post).

You need to make your own threads and not pollute existing ones. All your basic questions are answered in the MPEG specification. Go read it!