MP4: how are chapters and text subs stored?

[Mosu]

Hey,

in 2009 I added support for reading chapters from MP4 files to mkvmerge. The first commit implemented reading them from the CHPL atom inside the UDTA atom (see ticket 385 (https://trac.bunkus.org/ticket/385)). Later that year someone else requested I add another method: by reading a full track of sub-type 'text' and using its "frames" as chapter entries (see ticket 454 (https://trac.bunkus.org/ticket/454)).

However, from time to time I receive bug reports (https://trac.bunkus.org/ticket/815) that mkvmerge detects subtitle tracks as chapters. Looks like they're stored as tracks with sub-type 'text' as well.

Hence my question. How can I distinguish between "text track that contains subtitles" and "text track that contains chapters"? Or put more broadly: how is chapter information, and how are text subtitles stored in MP4 files?

[nevcairiel]

There is a "chap" atom (leaf of a "tref" track reference atom), which holds the stream index to the chapter track, which should allow you to identify which "text" track is the chapter track.

[JEEB]

While I completely coincide with nevcairiel on how to find the needed information, this topic reminds me of the fact of how the body governing the ISO Media Container and what we generally call "MP4" is not exactly down-to-earth in its things (I really love to let out some steam on this every now and then).

A good example of that is that there is no specified way of doing chapters in the ISO Media Format / what we colloquially call "MP4". MOV had its own way (the CHAP atom), and later Nero implemented its completely own way of putting chapters in. Nowadays both can be found in "mp4" files around the world (while now the MOV CHAP atom is set not to be used in the specification [it was just "reserved" for quite a long while or so in the older specifications IIRC, so people were actually expecting it to take the MOV chapters as the official way of doing them], and the Nero way has never even been close to the standard bodies), and what the committee tells one to do is to specify and implement something on top of BIFS. Yup, not overcomplicated and "Everyone please implement your own crap by yourselves within an overcomplicated framework, please." at all.

That said, at least we have the MP4 RA (http://www.mp4ra.org/) for everyone to specify their modifications upon the basic specification. At least in theory.

</rant>

[Mosu]

@Jeeb: Hehehe, I can understand you completely :)

@nev: thanks, I'll look into it.

[Mosu]

Worked nicely, thanks for the explanations.

[Chetwood]

ISO lost all credibility when going against their own rules and allowing Microsoft's crappy competing word format despite having accepted ODF first.

Concerning chapters: I've been fiddling around with these too lately and realized that some tools seem to be better than others at generating them. E.g. adding a chapter file to an mp4 with Yamb (MP4Box) would make them show up in Mediainfo but not when muxed with DVBPortal MP4 Muxer. However, none of which show up in iTunes so I guess there's even more to this...

[Mosu]

Alright. What about subtitles? How is the situation there -- anything official? How are unofficial formats (SRT, SSA...) stored?

[Daiz]

Alright. What about subtitles? How is the situation there -- anything official? How are unofficial formats (SRT, SSA...) stored?

Separately :)

I think the only subtitle format MP4 "officially" supports is TTXT, but I might be wrong. Or maybe the official line is yet another "you could do fancy subtitles... WITH BIFS!" which nothing actually supports.

[JEEB]

Yes, MPEG-4 Part 17 (http://en.wikipedia.org/wiki/MPEG-4_Part_17) (related specs being 14496-17:2006 and 3GPP TS 26.244 / ETSI TS 126 245 [I seemingly found v11 of the spec with the latter naming, while I could only find v9 with the 3GPP naming]) is the only "official" way of putting subtitles into "MP4". One of the　things then hacked into "MP4" were vobsubs, which I'm pretty sure are not in any ways official :) , and unfortunately I have no idea of how they are muxed in.

In theory you could mux a /lot/ of stuff into "MP4", but yeah... it's a mess since no-one then registers their extensions and publishes specs :P

[manolito]

Concerning chapters: I've been fiddling around with these too lately and realized that some tools seem to be better than others at generating them. E.g. adding a chapter file to an mp4 with Yamb (MP4Box) would make them show up in Mediainfo but not when muxed with DVBPortal MP4 Muxer. However, none of which show up in iTunes so I guess there's even more to this...

Basically there seem to be two different methods to store chapter information in an MP4 container. There is the "Apple Text" format (Handbrake and MKV2MP4 create this format), and there is the "Menu" format (StaxRip and Yamb use this format).

I started a thread some time ago asking about those two formats, but the answers I got were not really satisfactory. My personal solution to this issue is to always create MP4 files which contain both chapter formats. This is the MediaInfo result for such a file:

Allgemein
Vollständiger Name : F:\Download\test.mp4
Format : MPEG-4
Format-Profil : Base Media
Codec-ID : isom
Dateigröße : 17,1 MiB
Dauer : 6min 2s
Modus der Gesamtbitrate : variabel
Gesamte Bitrate : 396 Kbps
Filmname : test
Kodierungs-Datum : UTC 2012-12-29 00:59:15
Tagging-Datum : UTC 2012-12-30 01:05:58
Kodierendes Programm : Yamb 2.1.0.0 [http://yamb.unite-video.com]
hdvd : 0
stik : 9

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format-Profil : Baseline@L2.1
Format-Einstellungen für CABAC : Nein
Format-Einstellungen für ReFrames : 1 frame
Format_Settings_GOP : M=1, N=50
Codec-ID : avc1
Codec-ID/Info : Advanced Video Coding
Dauer : 6min 2s
Bitrate : 240 Kbps
maximale Bitrate : 399 Kbps
Breite : 432 Pixel
Höhe : 240 Pixel
Bildseitenverhältnis : 16:9
Modus der Bildwiederholungsrate : konstant
Bildwiederholungsrate : 29,000 FPS
ColorSpace : YUV
ChromaSubsampling : 4:2:0
BitDepth/String : 8 bits
Scantyp : progressiv
Bits/(Pixel*Frame) : 0.080
Stream-Größe : 10,4 MiB (61%)
verwendete Encoder-Bibliothek : x264 core 120 r2120 0c7dab9
Kodierungseinstellungen : cabac=0 / ref=1 / deblock=1:0:0 / analyse=0x1:0x111 / me=hex / subme=2 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=0 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=0 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=0 / threads=24 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=0 / weightp=0 / keyint=50 / keyint_min=5 / scenecut=40 / intra_refresh=0 / rc_lookahead=10 / rc=cbr / mbtree=1 / bitrate=240 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / vbv_maxrate=240 / vbv_bufsize=480 / nal_hrd=none / ip_ratio=1.40 / aq=1:1.00
Kodierungs-Datum : UTC 2012-12-29 00:57:05
Tagging-Datum : UTC 2012-12-29 00:59:17

Audio
ID : 2
Format : AAC
Format/Info : Advanced Audio Codec
Format-Profil : LC
Codec-ID : 40
Dauer : 6min 2s
Bitraten-Modus : variabel
Bitrate : 151 Kbps
maximale Bitrate : 189 Kbps
Kanäle : 2 Kanäle
Kanal-Positionen : Front: L R
Samplingrate : 44,1 KHz
Stream-Größe : 6,53 MiB (38%)
Kodierungs-Datum : UTC 2012-12-29 00:59:16
Tagging-Datum : UTC 2012-12-29 00:59:17

Text
ID : 3
Format : Apple text
Codec-ID : text
Dauer : 6min 2s
Duration_LastFrame : -57s 897ms
Bitraten-Modus : konstant
Bitrate : 4 bps
Stream-Größe : 161 Bytes (0%)
Kodierungs-Datum : UTC 2012-12-30 01:05:58
Tagging-Datum : UTC 2012-12-30 01:05:58

Menü
00:00:00.000 : Chapter 1
00:01:00.000 : Chapter 2
00:02:00.000 : Chapter 3
00:03:00.000 : Chapter 4
00:04:00.000 : Chapter 5
00:05:00.000 : Chapter 6
00:06:00.000 : Chapter 7

After I started doing it this way I never got any complaints about missing chapters in my MP4 conversions.



Cheers
manolito

[Chetwood]

Problem in my case is that I only later decided to add chapters to some mp4s (I created with Hybrid) which was easy with YAMB. But what muxing tool adds iTunes readable chapters? Don't want to (re)encode the whole thing with Handbrake.

[Kurtnoise]

MP4 Muxer from MeGUI for example...

[manolito]

Or MKVtoMP4 by oreons...:D

The procedure is explained here:
http://forum.doom9.org/showthread.php?p=1569412#post1569412


Cheers
manolito


A Happy New Year to everybody

[Kurtnoise]

Worked nicely, thanks for the explanations.
I'm wondering how to read chapter names & chapters timestamps in this case ?

[Mosu]

Chapter tracks are full tracks. Each "frame" is a chapter, its timecode the chapter's start time. The "content" of that frame contains the chapter's name (though its preceded by a two-byte length field whose existence I don't understand -- a frame's length is already given in the track headers).

[Kurtnoise]

Thanks,

So, in other words, this is stored in the ctts atom, right ?

[nevcairiel]

Its called a pascal string. Preceding string length is quite typical for them and mp4/mov uses this string construct all over the place.
They are not 0 terminated like a C string instead.

[nevcairiel]

Its called a pascal string. Preceding string length is quite typical for them and mp4/mov uses this string construct all over the place.
They are not 0 terminated like a C string instead.

[Mosu]

Its called a pascal string. Preceding string length is quite typical for them and mp4/mov uses this string construct all over the place.
They are not 0 terminated like a C string instead.

I've done Pascal myself back in the day, so I know what they are. What I don't really understand why there's a need to signal an implicit string length if the string is the sole content of a full box. The box itself is signalling the length already.

But then again there are quite a lot of other things that don't make much sense to me in MP4 either ;)