live streaming + hint atom

[rapso]

Hi,

I'm playing a bit with the x264 source to get familiar with it (e.g. ported it to compile with Visual studio etc.). Now I'd like to use it for live encoding and streaming, which is basically working.
But I've the issue, that my iPad isn't playing the stream, until it's finished with "downloading" (the in-browser video-controls are visible, but you can't press them).
I can dump the videostream and use mp4box to add the hint track, if I stream that data to my iPad, it can play the movie while downloading.

I've been using MP4 Explorer to compare both files, the hint track seems to contain all information of the whole movie file.

Now I'm a little bit puzzled, if I don't know all inter-frames etc. as it's life encoded, I cannot create a hint track on the beginning of the file (actually, I don't even know how many frames I'd need), but if I don't create the hint track, I can't see the stream.

I'm sure there are live streaming solutions that can be live viewed, without a custom player. Can someone point me into the right direction? what do I miss? or is there a way to interleave the tracks? (maybe mp4 explorer just visualizes them as one box, but in reality there are several boxes?)

thanks in advance

[benwaggoner]

There's actually two kinds of optimizations at play here.

A "Fast Start" .mp4 or .mov has the "MOOV" atom that serves as an index to the media at the start of the file. That means that playback can start as soon as actual media packets start showing up in the file. But the MOOV atom isn't generated until the whole file is finished encoding, so many (older) tools just append it to the end of the file. It's not a problem if you have random access to the file, like if it's on local media. Or if the player and server support byte-range seek. But all the player/server can do is just old-school linear download, you don't get any playback with a non "Fast Start" movie until all the media is downloded and THEN you get the MOOV atom.

The simplest way to fix that is to rewrite the file so the MOOV is at the front. That's called "flattening" or "Fast Start" or "Optimize for Streaming" in different tools. Just re-saving the file will generally do it. Some tools will also allow gzip compression of the MOOV, which is an excellent idea, as it can often reduce the MOOV size by ~80%, meaning playback can start that much faster.

Hint tracks are another matter entirely. These are special information that tells a RTSP server how to packetize the media for real-time streaming. They're ignored if you're using http progressive download; you ONLY want them in the file if using canonical RTSP. They're not needed for Smooth Streaming/HTTP Live Streaming/Flash HTTP Adaptive Streaming either.

The reason you don't want those tracks other than if doing RTSP is because they take up space in the file which will get transmitted in progressive downloading, slowing the transfer, but then discarded by the player. Standard hint tracks that just reference the normal video/audio samples add around 5-20% bitrate (a higher percentage the lower the bitrate). Hint tracks can optionally encapsulate all the video/audio data, meaning they're about 105%-120% the size of the media tracks themselves! This is sometimes called "optimized for streaming" but they've not been helpful for streaming servers released in the last decade or so.

Big picture; don't encode with hint tracks, and just resave your files after encoding, and your current workflow should be just fine.

All of the above is an excellent demonstration of why fragmented MPEG-4 is the new hotness!