[bond]

1) What is MP4? I have heard it is a "container format"? What's that?
A container format allows you to combine different multimedia streams (most of the time audio and video) into one single file.
Multimedia containers are for example the well known AVI (.avi), MPEG (.mpg, .mpeg), Matroska (.mkv, .mka), OGM (.ogm), Quicktime (.mov) or Realmedia (.rm, .rmvb)

MP4 is the global file extension for the official container format defined in the MPEG-4 standard (ISO 14496-14)
MP4 is streamable and supports all kinds of multimedia content (multiple audio-, video-, subtitlestreams, pictures, variable-framerates, -bitrates, -samplerates...) and advanced content (officially called "Rich Media" or "BIFS") like 2D and 3D animated graphics, user interactivity, DVD-like menus... - next to nothing handled by the often used AVI

2) What extension is MP4 using?
- .mp4: only official extension, for all audio and video and advanced content files (and combinations)

other related extensions:
- .m4v: .mp4 files with the WRONG extension introduced by apple for video+audio files, m4v can safely be renamed to .mp4
- .m4a: .mp4 files with the WRONG extension introduced by apple for audio-only files, m4a can safely be renamed to .mp4
- .m4p: DRM protected files sold in iTunes, using the DRM sheme developed by apple
- .m4e: renamed .sdp files used by Envivio for streaming
- .m4v, .mp4v, .cmp, .divx, .xvid, .264: normally raw mpeg-4 video streams (not inside MP4)
- .aac: raw aac audio streams/adts (not Inside MP4)
- .3gp, .3g2: used by mobile phones, can also include content not defined for .mp4 (H.263, AMR), see question 20
- .mov: technologically similar container, but not the same as MP4, see question 20

3) How can i create spec compliant MP4 files?
easy answer: have a look at Doom9's MP4 Guide or Nero Recode2 Guide

detailed answer:
1) To create MP4 files you should preferably use audio/video formats covered by the MPEG-4 Standard:
- MP4 video formats: MPEG-4 SP/ASP: XviD, DivX5, 3ivx, ffmpeg/ffdshow, NeroDigital ASP... and MPEG-4 AVC/H.264: x264, NeroDigital AVC, Apple... and MPEG-2, MPEG-1 (not DivX3/MS Mpeg-4, WM9, RV9 or VP6) - doom9's quality comparisons: 1 2
- MP4 audio formats: Advanced Audio Coding (AAC FAQ): Quicktime/iTunes, Nero, Psytel, FAAC (binary)... (rjamorim's quality comparisons: 1 2) and (E)AC3 (free specifications), MP3, MP2, MP1, CELP (speech), TwinVQ (very low bitrates), SAOL (midi), ALS (lossless)
- MP4 subtitle/text format: MPEG-4 Timed Text (see question 7)
- MP4 picture formats: JPEG, PNG

2) with the following tools you can mux MPEG-4 compatible content into the MP4 container:
- GPAC's MP4Box, commandline, supports AVC, packed b-frames, advanced content/BIFS, TTXT and Chapters (Doom9 Guide)
- MPEG4IP's mp4creator, commandline, able to mux AVC, dont use it with AVI and packed b-frames!
- 3ivx mp4 muxer dshow filter, usable in Graphedit, supports packed AVI b-frames but no AVC (3ivx Guide)
- ffmpeg, commandline, supports ac3 muxing and encoding to MP4, buggy for muxing (especially dont use it with AVC, AVI and (packed) b-frames)
- mp4UI, based on MPEG4IP, dont use it with AVC, AVI and (packed) b-frames!
- VideoLAN, able to mux AVC, dont use it with (packed) b-frames!
- IBM's Toolkit for MPEG-4 which includes XMTBatch and/or AVGen (also a simple version) based on Java
- dvd2mp4gui, GUI using mp4tool, supports muxing multiple audio and subtitle streams (as advanced content) into MP4
- mp4tool/mp4edit from ENST, supports BIFS, mp4edit is the "light" GUI version of the commandline mp4tool
3) the following "all-in-one" MP4 encoding tools are existing:
- Nero Recode2, encodes into MP4 using ASP or AVC and AAC (Doom9 Guide), subs are DVD VobSubs, includes a dshow muxer
- Doom9's MeGUI, supports ASP (Xvid, libavcodec), AVC (x264) and AAC (Besweet - Nero) encoding
- HDX4 supports ASP and AAC
- Dicas' mpegable X4 live, encodes into MP4 using Dicas' codecs
- Envivio's 4Coder encodes to MPEG-4 AAC, ASP/SP and AVC
4) the following tools can be used for editing (eg splitting, appending) MP4 files:
- Avidemux, Virtualdub-style cross platfrom MP4 editor
- MP4Box joins and splits MP4 (AAC, ASP, AVC, TTXT, MPEG-1/2, private streams) time- and sizewise (GUIs: Yamb, Cut Assistant)
- Ulead's VideoStudio 8 with the MPEG-4 plugin
- Sony's Vegas 6
- Quicktime/Pro can be used to create (encode, mux) and edit (split, join) MP4 files, supports MPEG-4 Simple Profile, AAC, AVC

4) Is it also possible to place streams not covered by the MPEG-4 standard in MP4?
Yes, altough i dont like the idea of doing so, the MPEG-4 standard defines how to place any stream in MP4. Similar to how streams are marked in AVI with private FourCCs, in MP4 they get a "private track ID". Note that no player will be able to handle such streams in MP4, but will simply ignore them, if not explicitely specified to do so
the following streams, not covered by MPEG-4, can be placed in MP4 already:
- Ogg Vorbis and Ogg Theora, via MP4Box
- Ogg Vorbis, via a patched mp4creator
- Apple's Lossless Audio (ALAC/ALE - not compliant with MPEG-4 ALS), via iTunes
- DVD picture subtitles (Vobsubs), via Nero Recode2
only the future will show if these private streams will be widely accepted and supported...

5) What do i have to especially look at, when i mux MPEG-4 video from AVI to MP4?
there are potential problems when muxing from AVIs (especially with b-frames/B-VOPs/bi-directional encoding) into MP4:
1) "delay frames", caused by the old VFW interface some codecs still use (XviD, DivX5). only when encoding with VirtualDub/Mod these will be automatically dropped when using b-frames
2) "packed bitstream", caused by the old AVI container, which is not really capable of storing b-frames. PB is used in DivX5 (only with 1 consecutive b-frame set) and by default in new XviD builds (make sure the PB option is unticked in XviD) and packs b-frames together with other frames (read a more detailed description about B-frames in AVI here).
3) "userdata", which in the case of XviD and DivX PB streams signals the useage of PB. some decoders take this flag into account while decoding
4) "vol", which is placed on every keyframe in AVI, but has to be seperated from the movie data in MP4
5) "ctts" atom, which has to be written when muxing b-frames into MP4

Currently only the 3ivx mp4 muxer and GPAC's MP4Box handle all these things correctly! use only them for AVI conversions, especially with packed bitstreams or if you are unsure which b-frame option/encoding app was used. otherwise the chance is great that you will not get a 100% spec compliant MP4 file!

6) Does transmuxing to MP4 reencode the content?
No, this is a lossless process, only taking out the media streams from one container and putting them into MP4

7) How can i mux Subtitles into MP4 files?
there are three possibilites to store subs in mp4:
- In contrary to MPEG-1/2 the MPEG-4 Standard defines its own Text Stream/Subtitle Format under ISO 14496-17 (MPEG-4 Timed Text). Its text-based (not picture-based as DVD subs), uses UTF-8/16 (including cyrilic, arabic, chinese characters...), allows all sorts of animations (scrolling, colors, karaoke aso...), allows attaching of fonts and is streamable
MPEG-4 TTXT creation is already supported in MP4Box and Quicktime. Playback works with Gabest's and Haali's MP4 dshow parser, Osmo4, Realplayer and Quicktime (only when placed in .3gp) and VideoLAN.
- Nero's Recode2 makes use of the possibility of storing private streams in MP4 (question 4) and simply takes the DVD subpicture streams (vobsubs) from the DVD and stores them in MP4 without modifications
playback of these streams works with Nero's, Gabest's and Haali's Parser, VideoLAN and some Hardware MP4 DVD Players
- another way covered by the MPEG-4 Standard for storing subs in MP4 is to translate the subtitle format into the XMT/BT format and then to encode it as "advanced content" (BIFS binary format) in MP4. use mp4box or dvd2mp4gui for example to convert SRT subs
for playback of advanced content you need a filter/player that can handle this kind of content (see question 14)

8) How can i place Chapters in MP4 files?
chapters storage is possible by using advanced content (eg by creating a menu), but Nero uses a MP4 feature called "user space" (udta atom) which allows you to attach anything you want in MP4 (iTunes uses the udta atom for storing tags and cd/dvd-covers ("coverart") in MP4 for example).
atm you can place chapters in MP4 automatically during encoding with Nero Recode2 or, on existing files, with MP4Box and the MP4 directshow muxer filter coming with Recode2 (usable in graphedit)
As players, not knowing the info stored in udta, will simply ignore it, you need a player/filter that can handle this special chapters info (eg Gabest, Haali, Nero or 3ivx)

9) How can i tag MP4 files?
The MPEG-4 Standard defines the "meta" atom to be used for storing tagging-like information, but Apple has introduced a different tagging way with iTunes using the "udta" atom with utf-8 tags (see question 8)
There are various tools for adding apple-like tags to MP4 files, like mp4tags from mpeg4ip, tg or AtomicParsley

10) How can i store infomation about different tracks in MP4 files?
The MP4 container allows the storage of the used language and the name for every stream (no matter if the stream is audio, video or animation)
The language can be set for example by mp4box and quicktime. The streamname feature is used by Nero and can be set indivdually with mp4box

11) How can i author MP4 files (ie create DVD-like Menus)? Are there any samples?
Thanks to IMOON there is now the MP4MenuGUI tool available which lets you easily design dvd-like menus in MP4. It creates a file using the textual language format called XMT or bt (based on VRML, like the well known Flash), which than gets encoded/compiled into mp4 (free available encoders are mp4box, mp4tool or XMTBatch) as binary format (called BIFS)

If you want to try MP4 authoring yourself "by hand" or modify the script created by MP4MenuGUI read the great Tutorial by the GPAC guys available in english and french
to make this process much easier i created a sample script, called MP4Menu (which is also the basis for MP4MenuGUI) which already supports all features a DVD Menu offers and you will only have to adopt to your needs (guide)

if you want to know what can also be possible in MP4, check out this small in-video menu sample, but MPEG-4 Systems also offers things like animations in 2 and 3 Dimensions (think of movies like Toy Story or Finding Nemo), check out this small systems 2D animation sample (doesnt include a video stream!)

For playing MP4 files using advanced content make sure to use a Systems capable Player, like the GPAC Osmo4 player