Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 22nd April 2021, 10:27   #1  |  Link
TEB
Registered User
 
Join Date: Feb 2003
Location: Palmcoast of Norway
Posts: 357
How to extract and calculate the framerate of the video in a mp4 container

hi, afaik, the mp4 container doesnt have a absolute moov atom describing the actual framerate of the video within, but its rather calculated based on the PTS/DTS timing values.
Are there tools, pref opensource, that can show the PTS/DTS values and do this calculation?
I know mediainfo, ffprobe etc can do it, but i want to understand where in the mp4 container this data resides..
TEB is offline   Reply With Quote
Old 23rd April 2021, 09:39   #2  |  Link
butterw2
Registered User
 
Join Date: Jun 2020
Posts: 207
maybe https://github.com/nu774/mp4fpsmod
Tiny mp4 time code editor. You can use this for changing fps, delaying audio tracks, executing DTS compression, extracting time codes of mp4.
butterw2 is offline   Reply With Quote
Old 23rd April 2021, 13:38   #3  |  Link
videoh
Useful n00b
 
Join Date: Jul 2014
Posts: 1,478
The frame rate can be calculated from the number of samples and the duration:

Code:
u_int32_t mp4_numSamples = MP4GetTrackNumberOfSamples(mp4File, trackId);
MP4Duration duration = MP4GetTrackDuration(mp4File, trackId);
double msDuration = (double)(MP4ConvertFromTrackDuration(mp4File, trackId, duration, MP4_MSECS_TIME_SCALE));
double mp4_Frame_Rate = mp4_numSamples * 1000LL / msDuration;
It may also be specified in the video ES.

Last edited by videoh; 23rd April 2021 at 22:07.
videoh is offline   Reply With Quote
Old 24th April 2021, 00:59   #4  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,728
Quote:
Originally Posted by videoh View Post
The frame rate can be calculated from the number of samples and the duration:

Code:
u_int32_t mp4_numSamples = MP4GetTrackNumberOfSamples(mp4File, trackId);
MP4Duration duration = MP4GetTrackDuration(mp4File, trackId);
double msDuration = (double)(MP4ConvertFromTrackDuration(mp4File, trackId, duration, MP4_MSECS_TIME_SCALE));
double mp4_Frame_Rate = mp4_numSamples * 1000LL / msDuration;
It may also be specified in the video ES.
And that's the problem. It can be specified in multiple places, and there's no enforcement mechanism to make sure they even match.

The ES is generally what players go by, so it'd go for that. The MP4 headers can be wrong, and often give the average fps for variable frame rate content, which isn't that helpful.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 24th April 2021, 09:32   #5  |  Link
SeeMoreDigital
Registered User
 
SeeMoreDigital's Avatar
 
Join Date: Jun 2003
Location: Notts, UK
Posts: 11,834
Quote:
Originally Posted by benwaggoner View Post
The ES is generally what players go by, so it'd go for that. The MP4 headers can be wrong, and often give the average fps for variable frame rate content, which isn't that helpful.
Indeed... Even back in the old MPEG-4 Part 2 encoding days, it soon became apparent after .mp4 container support was added to hardware playback devices, that the fps data needed to be added to the elementary video stream, not the container. Same too with the aspect ratio signalling.
__________________
| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 | My Network Layout & A/V Gear |
SeeMoreDigital is offline   Reply With Quote
Old 28th April 2021, 00:36   #6  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,728
Quote:
Originally Posted by SeeMoreDigital View Post
Indeed... Even back in the old MPEG-4 Part 2 encoding days, it soon became apparent after .mp4 container support was added to hardware playback devices, that the fps data needed to be added to the elementary video stream, not the container. Same too with the aspect ratio signalling.
I remember with WMV that there were THREE different places that picture aspect ratio could be stored, without any clear rules as to which would take precedence.

Were I God of Video, I'd say that the elementary stream would always take precedence, and it's the job for each layer above to conform to it.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 28th April 2021, 05:43   #7  |  Link
rwill
Registered User
 
Join Date: Dec 2013
Posts: 53
Quote:
Originally Posted by SeeMoreDigital View Post
Indeed... Even back in the old MPEG-4 Part 2 encoding days, it soon became apparent after .mp4 container support was added to hardware playback devices, that the fps data needed to be added to the elementary video stream, not the container. Same too with the aspect ratio signalling.
Well yes and no. There is the case with fixed frame rate, then ES and container should match. Then when a video is missing frames the duration a picture might be displayed is a multiple of a picture 'tick's, this is also the case when there are transmission errors. Last case is totally variable frame rate, then timing should just be in the container.
rwill is offline   Reply With Quote
Old 29th April 2021, 23:20   #8  |  Link
TEB
Registered User
 
Join Date: Feb 2003
Location: Palmcoast of Norway
Posts: 357
Soo, how can i extract the ES AR and ES Framerate? Or rather, where exact in the ES does this info recide?
TEB is offline   Reply With Quote
Old 30th April 2021, 01:45   #9  |  Link
videoh
Useful n00b
 
Join Date: Jul 2014
Posts: 1,478
It depends on the video format, i.e., AVC, HEVC, MPEG2, or VC1. Did you have one in mind? ES just means what you would get if you demuxed the video. You can consult the appropriate video specification. They are readily available. I'll help if you specify your format.

Be warned: there may not be a specification in the ES for any given file.
videoh is offline   Reply With Quote
Old 30th April 2021, 10:11   #10  |  Link
SeeMoreDigital
Registered User
 
SeeMoreDigital's Avatar
 
Join Date: Jun 2003
Location: Notts, UK
Posts: 11,834
Quote:
Originally Posted by rwill View Post
...Last case is totally variable frame rate, then timing should just be in the container.
Hmmm...

When it comes to generating encodes suitable for playing in a 'hardware playback device', you should never create files with a variable frame rate, regardless of where the frame rate information is stored.
__________________
| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 | My Network Layout & A/V Gear |
SeeMoreDigital is offline   Reply With Quote
Old 1st May 2021, 17:31   #11  |  Link
TEB
Registered User
 
Join Date: Feb 2003
Location: Palmcoast of Norway
Posts: 357
Quote:
Originally Posted by videoh View Post
It depends on the video format, i.e., AVC, HEVC, MPEG2, or VC1. Did you have one in mind? ES just means what you would get if you demuxed the video. You can consult the appropriate video specification. They are readily available. I'll help if you specify your format.

Be warned: there may not be a specification in the ES for any given file.
HEVC and AVC!
TEB is offline   Reply With Quote
Old 5th May 2021, 12:48   #12  |  Link
Ritsuka
Registered User
 
Join Date: Mar 2007
Posts: 66
If a player ignores the container PTS/DTS, then that player is hopelessly broken.

videoh code is not entirely correct. MP4 has a feature called "edit lists", and the tkhd box duration is equal to the sum of the durations of all of the track’s edits. But the edits can use less the actual samples in a track, so you can't divide it by the number of samples of the track.

To get the actual frame rate, you would need to calculate the actual presentation timestamp of each sample included in an edit list, and them make an average.
And edit list can change the rate at which samples at played or be a "dwell" without any sample for extra fun.

Refer to the free ISO/IEC 14496-12 Fifth edition pdf for a description of all this fun little features.
Ritsuka is offline   Reply With Quote
Old 6th May 2021, 00:25   #13  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,728
Yeah, the MPEG-4 file format is derived from the QuickTime file format, which was Turing Complete! Modern implementations don't support all of the late 90's interactive features of QuickTime, but the file format was capable of things akin to what Director or HyperCard could create.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 6th May 2021, 07:00   #14  |  Link
Ritsuka
Registered User
 
Join Date: Mar 2007
Posts: 66
Yes, there was some weird stuff in there. But *edit lists* are a basic feature that should be required, but unfortunately in the real world is not always implemented.

Last edited by Ritsuka; 14th May 2021 at 09:20.
Ritsuka is offline   Reply With Quote
Old 14th May 2021, 01:21   #15  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,728
Quote:
Originally Posted by Ritsuka View Post
Yes, there was some weird stuff in there. But *edit lists* are a basic feature that should be required, but unfortunately in the real world is always implemented.
OMG, I know. I used to get .mov mezzanine files all the time where an operator trimmed out leader or something and did a Save instead of Save As. It was a coin toss whether any given video tool would read the edit list, or just grab the whole elementary stream. <shudder>.

More a technology can do same thing in different ways, the more implementations will wind up diverging. That was one nice thing about AVI as a format - it was so brain-dead simple it was really hard to get into trouble with it.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:46.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.