mp4box adds compliant mpeg-4 text subtitles to .mp4!

[bond]

woaw, i really love gpac. its really an ass-kicking bunch of tools!

according to the changelog the latest cvs version of mp4box now supports muxing text subtitles into .mp4 following the mpeg-4 part17 (mpeg's first and only text stream format)

it seems to support .srt subs as input (and ttxt, which should be a better representation of the original mpeg-4 timed text (it has to be tested what text "effects" are currently supported with it)

next to latest osmo4, to my knowledge videolan should also already support the playback of these subs and 3ivx is working on adding support to it to its splitter (great stuff!)

enjoy :)

last CVS update: 29/11/04
- 3GPP/MPEG4 timed text decoder - supports everything except soft wrap & dynamic highlighting (karaoke) - vertical text not fully tested
- new "ttxt" format (for "timed text"): XML representation of 3GPP/MPEG4 timed text streams - importer (from ttxt and from srt) and dumper (to ttxt only) support.
- MP4Box "-ttxt" option: converts an SRT file to a TTXT one.
- Input plugin for ttxt and srt files.find the gpac project of course here (http://sourceforge.net/projects/gpac/)

[Sharktooth]

GPAC 0.2.2 binaries + extras (celtic_druid build): http://ebola.gamersrevolt.it/celticdruid/gpac/

[SeeMoreDigital]

Could this mean that mp4UI might be able to support this feature at some time?

Bond... can you provide a short clip please?


Cheers

[celtic_druid]

mp4UI is based on MPEG4ip not GPAC, but I guess anything is possible.

From the look of that changlog quote you would need the latest dev version and not 0.2.2 to get the subtitle support.

[yaz]

i found these also interesting
last CVS update: 29/11/04
...
- Changed MP4Box "-import" to "-convert" ("-import" is kept for backward compatibility).
- Added MP4Box "-rem" (removes a track), -nosys (removes all MPEG-4 systems streams and writes an empty IOD)
- Added MP4Box "-add": adds any supported format to an mp4 file (same input conventions as -convert). Several input can be specified (ex: -add audio1.aac -add sub1.srt video.mp4 -out full_movie.mp4)
...
- Full multichannel audio mixing and resampling (and better audio speed support). Multichannel->stereo conversion (not configurable atm).
it seems gpac now supports multiple audio a/o subtitle streams in mp4. great !!! (i'm lovin' it :-))

the bests
y

[SeeMoreDigital]

Does mp4box have a GUI or is it command line only?


Cheers

[gotaserena]

Originally posted by celtic_druid
mp4UI is based on MPEG4ip not GPAC, but I guess anything is possible.

The first thing that comes to mind is that mpeg4ip just copies the corresponding code from GPAC. But, I really doubt this would happen since for some untold reason the developers of both projects hate each other's guts.

[bond]

Originally posted by celtic_druid
From the look of that changlog quote you would need the latest dev version and not 0.2.2 to get the subtitle support.yep, is your build supporting this already? :)

Originally posted by yaz
it seems gpac now supports multiple audio a/o subtitle streams in mp4.mp4box supported multiple audio streams for a long time already, otherwise my mp4menu wouldnt have been possible :)

Originally posted by gotaserena
But, I really doubt this would happen since for some untold reason the developers of both projects hate each other's guts.loool, where did you get that from? i am in contact with devs from both projects and i never heard about that. from my point of view they simply have no contact at all

[gotaserena]

Originally posted by bond
yep, is your build supporting this already? :)

It does import .srt subtitles as
Track # 3 Info - TrackID 203 - TimeScale 1000 - Duration 6751785
Media Type "text" - Media Sub Type "tx3g" - 2191 samples
3GPP/MPEG-4 Timed Text

But so far I haven't been able to actually see them using VLC.

loool, where did you get that from? i am in contact with devs from both projects and i never heard about that. from my point of view they simply have no contact at all

Ok, since you don't know about it, I step back and retract whatever I said on this regard. I must have misread some topic in mpeg4ip forum. Apologies to the parties involved.

[SeeMoreDigital]

Originally posted by gotaserena
But so far I haven't been able to actually see them using VLC. How about WMP9/10 with VS filter?


Cheers

[gotaserena]

Until we have a mp4 dshow demuxer that recognize these tracks, we are outta luck wrt wmp/wmc displaying these subs...

[SeeMoreDigital]

I wonder whether Nero's ShowTime player can already display them?


Cheers

[bond]

gotsarena, where did you get that mp4box compile from that is able to create ttxt streams in .mp4? celtic_druid's one is some weeks old and doesnt include this functionality

Originally posted by gotaserena
Until we have a mp4 dshow demuxer that recognize these tracks, we are outta luck wrt wmp/wmc displaying these subs...does it work with osmo4?

[gotaserena]

Well, I could have compiled the sources myself, couldn't I? :D

But celtic_druid did compile a cvs version supporting this, as can be seen here (http://celticdruid.no-ip.com/xvid/gpac/dev/).

I only tried one file, and Osmo4 did not decode the video properly. BTW, Osmo has been less and less reliable these days.

[bond]

okidoki, i now tested it too :)

- basically the creation of the file works fine
- the .ttxt format seems to be xml, looks interesting :)
- playback in latest osmo4 also works great (tried it with a simple profile video stream)
- in latest videolan it lets me choose the subtitle stream (so vlc definitely realises that there are subtitles), but still no text is displayed (maybe something else has to be done in vlc to make subs possible?)
- also the 3ivx guys might be interested in that its perfectly possible to play the subtitles with the alpha 3ivx splitter and gabests vsfilter :)
- mp4ui is able to recognize the stream as "text"

thanks a lot for the link gotaserena, and of course thanks a lot for the compile celtic_druid :)

[yaz]

just a quick late-night-test

- muxing multiple audio a/o subtitles works fine
- playback worx perfectly only w/osmo
- moonlight demuxer passes all(!) audio streams but no subtitle ffdshow chained well (? maybe, i just misunderstood den)
- nero splitter passes only the default audio stream (quite strange as nero supports multiple audio???) and no subs
- vlc sees all streams but no subtitle playback (???)
- mpc plays back everything it gets from the 'actual' splitter, i've found no synch issue.
- he-aac plays back only as lc-aac :-( (checked w/coreaac)
- playback is smooth but file is not seekable. this should be tested further, cus i passed an xvid stream with i/p/b/s frames. maybe, that was the problem at seeking. playback doesn't stop but gets messy up to the next i frame.

celtic_druid, many thx for the up-to-date shots !

the bests
y

[EDIT]just downloaded the latest snapshot from celtic_druid. now we got this
- AAC SBR import now uses backward compatible decoder config. maybe this helps my he-aac misery :-)

[bond]

yaz, plz always write what splitter you used (ie your last point is pretty useless if we dont know the splitter)

[yaz]

Originally posted by bond
yaz, plz always write what splitter you used (ie your last point is pretty useless if we dont know the splitter) yes, u're right but, pls, note it was 'a quick late-night-test' :-) anyway ... i've tried 041201 build and .... yessss ... i got back my he-aac streams !!! great :-)))

some further tests

- he-aac (aac+sbr) muxed with mp4box now plays back perfectly with each splitter i tried (actually, nero and moonlight)
- strange, but mp4box itself reports lc-aac w/-info but the stream is identified 'he' in osmo (???)
- nero still unwilling to produce more than one audio while moonlight produces all.
- playback w/nero is smooth and seekable but w/moonlight it's messy. when seeking w/latter the screen gets tiled up to the next i-frame.
- '-hint' and '-isma' kicked off moonlight completely when audio was present. '-ocr' seems not to cause such problems.

i've managed to produce a full AxC/MP4 too.
1. a fake xvid stream and all other sreams (2 he-aac & 2 subs) muxed w/mp4box
2. then the xvid stream changed to an x264 stream w/mp4creator
i've had no time to test this tricking but a quick start w/mpc showed no problems when moonlight was splitting.

the bests
y

[SeeMoreDigital]

Originally posted by yaz
- nero still unwilling to produce more than one audio while moonlight produces all. This seems to be an on going problem with Nero.

As you probably already know, their "audio" filter, parser etc, only seem to want to work with ShowTime and WMP9/10.

However, when you play files in WMP9/10, you don't get anything like the options you get in ShowTime - ie: no chapters selection, no subtitles selection and no audio stream selection.


Cheers

[yaz]

Originally posted by SeeMoreDigital
... Nero ... their "audio" filter, parser etc, only seem to want to work with ShowTime and WMP9/10 ... it's stranger than that. before the 6.6 pack 'ndparser' was working 'in shadow', i.e. it never showed up in any filter chain but it was there in the back. (i found it in a hard way by switching off dsfilters one by one so as to get back mp4 support working perfectly before nero updating)
now it's a bit different. ndparser pops up in mpc but no way of using other than the default setting. the same is with audiodec. say, it pops up when opening aac streams (plain or wrapped) in mpc but again it works as is (it's practically an info-box).
on the other hand, the video filter allows even postprocessing, and, imho, it's a damn good decoder. i tried it w/xvid and x264 & it worked fine. and now it's possible to chain to the moonlight demuxer (latest hotfix).
imho, just a slight push on any side (nero a/o moonlight a/o ...) and we'd've got a great present for xmas :-) i mean AxC/MP4, of course :-) and if mpc showed sg for ttxt ... :-)

the bests
y

[SeeMoreDigital]

It is odd, isn't it!

I run a boot drive on my PC which only has Nero's direct-show filters on it (together with WMP10, Zoom and MPC media players) and ShowTime. So I can quickly discover how their filter situation changes between Nero releases!

... I've written to bobololo about the filters many times. I guess they want to keep their higher functions tied to their own ShowTime player, which although being understandable for them... is a shame for us :(

I happen to think their "video" filter is very good too... It's improved greatly since the first release. PAR and DAR signalling detection works flawlessly for AVI, MP4, MPG, VOB etc


Cheers

[zwommie]

Hi,

i tried to hint some 3gp files with ttxt subs in and received the following output :

>mp4box -hint ttxt_subs.3gp
Hinting file with Path-MTU 1500 Bytes
Cannot create hinter (Feature Not Supported)
Saving file into out_ttxt_subs.3gp: Flat storage... done

Am i right that ttxt hinting isnt yet supported?

I'm currently using the december 1st build from celtic

tnx,
zwommie

[bond]

importing ttxt into .mp4 is a very new feature in mp4box, i assume it doesnt support hinting of the stream atm.
but ttxt is definitely streamable, as one of the main focuses when creating it was streamability

[zwommie]

According to jean its added in latest CVS version.

Can anyone provide a build of latest CVS? celtic_druid maybe?
I'm trying it myself, but it wont work yet... :(

[gotaserena]

GPAC is a bit difficult to compile, especially osmo. But celtic_druid has updated the CVS compile today, one hour after your post, if I gather correctly.

[Elias]

Originally posted by bond
woaw, i really love gpac. its really an ass-kicking bunch of tools!Indeed :)

Originally posted by bond
next to latest osmo4, to my knowledge videolan should also already support the playback of these subs and 3ivx is working on adding support to it to its splitter (great stuff!)I really look forward to the next 3ivx release (let's just hope it'll be completely awesome!), but I've tried with the latest VideoLan from the Nightly builds, and although I can choose the subtitle track without any error message poping up, it doesn't display the subtitles. Have I forgot to check something in VLC, or are the VideoLan team halfway there with the ttxt support?

[bond]

Originally posted by Elias
I've tried with the latest VideoLan from the Nightly builds, and although I can choose the subtitle track without any error message poping up, it doesn't display the subtitles. Have I forgot to check something in VLC, or are the VideoLan team halfway there with the ttxt support?thats exactly the same behaviour i get! and btw i get the same also with vobsubs-in-mp4, as done by nero, in vlc too

somehow the subs handling of vlc seems to be broken, i already contacted fenrir, who worked a lot on this about it, but you know nero/ateme devs arent really that much allowed to work on other competing products ;)

[Elias]

Originally posted by bond
thats exactly the same behaviour i get! and btw i get the same also with vobsubs-in-mp4, as done by nero, in vlc too

somehow the subs handling of vlc seems to be broken, i already contacted fenrir, who worked a lot on this about it, but you know nero/ateme devs arent really that much allowed to work on other competing products ;) Hehe, one question though bond, is it safe to mux srt with mp4box and just hang on until VLC/3ivx can play them, or is mp4box still not 100% correct with converting srt into MPEG-4 ttxt?

[bond]

Originally posted by Elias
Hehe, one question though bond, is it safe to mux srt with mp4box and just hang on until VLC/3ivx can play them, or is mp4box still not 100% correct with converting srt into MPEG-4 ttxt? well vlc's ttxt handling is definitely broken, thats what also fenrir said

so mp4box files should be stable to use until prooven otherwise ;)

[Elias]

Originally posted by bond
well vlc's ttxt handling is definitely broken, thats what also fenrir said

so mp4box files should be stable to use until prooven otherwise ;) Great! Let's just hope that there's nothing wrong with mp4box and its srt>ttxt conversion :)

[SeeMoreDigital]

Originally posted by bond
thats exactly the same behaviour i get! and btw i get the same also with vobsubs-in-mp4, as done by nero, in vlc too I don't... Nero's subtitles have worked in most of VLC's builds for me. Even switching between 2No subtitle sets works!

http://img115.exs.cx/img115/3997/nerosubs2mv.gif


Cheers

[Elias]

Originally posted by SeeMoreDigital
I don't... Nero's subtitles have worked in most of VLC's builds for me. Even switching between 2No subtitle sets works!

Cheers That's probably because Nero's subtitles are DVD VobSubs, and VLC can decode that :) We're talking about MPEG-4 Timed Text subtitles though, and as of now, they don't work in the latest nightly build of VLC :/

[bond]

Originally posted by SeeMoreDigital
[B]I don't... Nero's subtitles have worked in most of VLC's builds for me. Even switching between 2No subtitle sets works!yep, according to fenrir it only works when you dont crop away the borders, but who leaves the borders in? ;)

[SeeMoreDigital]

Originally posted by bond
yep, according to fenrir it only works when you dont crop away the borders, but who leaves the borders in? ;) Oh dear... that would be "just me" then :D


Cheers

[JoeBG]

Originally posted by Elias
Indeed :)

I really look forward to the next 3ivx release...

Are there any news from the new 3ivx Codec and the new splitter?

[bond]

Originally posted by JoeBG
Are there any news from the new 3ivx Codec and the new splitter? nope, i think they are busy with some commercial project (yeah 3ivx has to earn money :D ) atm and cant work on their free stuff atm

[Elias]

Originally posted by bond
nope, i think they are busy with some commercial project (yeah 3ivx has to earn money :D ) atm and cant work on their free stuff atm Gah! How about we all use/buy 3ivx pro so they can start releasing awesome updates? :)

[Elias]

I got one question: why does the mp4 file become 2 seconds longer when adding ttxt subtitles in it?

[bond]

Originally posted by Elias
I got one question: why does the mp4 file become 2 seconds longer when adding ttxt subtitles in it? what stream gets 2 seconds longer? the ttxt stream?

[Elias]

Originally posted by bond
what stream gets 2 seconds longer? the ttxt stream? No, the entire mp4 file. I don't know why. It's only when used in a player that can handle the tx3g. Like for instance: VLC/QT. When played in MPC with nero splitter, and ffdshow, the mp4 file has the correct length. I guess this is because the tx3g isn't loaded.

[bond]

how do you read out this length?

[Stacey Melissa]

@Elias - Run mp4box -info movie.mp4 on the file. I bet you'll see that your subs track is longer than the video and audio tracks. You can fix this by editing your .srt (or whatever format) file so it isn't as long, before you mux it in.

[bond]

Originally posted by Stacey Melissa
@Elias - Run mp4box -info movie.mp4 on the file. I bet you'll see that your subs track is longer than the video and audio tracks. You can fix this by editing your .srt (or whatever format) file so it isn't as long, before you mux it in. elias sample indeed gives me:

Track # 1 Info - TrackID 1 - TimeScale 25000 - Duration 00:00:27.760
Media Type "vide" - Media Sub Type "mp4v" - 694 samples
MPEG-4 Config: Visual Stream - ObjectTypeIndication 0x20
MPEG-4 Visual Size 640 x 272 - Simple Profile @ Level 3
Self-synchronized

Track # 2 Info - TrackID 2 - TimeScale 48000 - Duration 00:00:27.733
Media Type "soun" - Media Sub Type "mp4a" - 1300 samples
MPEG-4 Config: Audio Stream - ObjectTypeIndication 0x40
MPEG-4 Audio AAC LC - 2 Channel(s) - SampleRate 48000
Synchronized on stream 1

Track # 3 Info - TrackID 3 - TimeScale 48000 - Duration 00:00:27.733
Media Type "soun" - Media Sub Type "mp4a" - 1300 samples
MPEG-4 Config: Audio Stream - ObjectTypeIndication 0x40
MPEG-4 Audio AAC LC - 6 Channel(s) - SampleRate 48000
Synchronized on stream 1

Track # 4 Info - TrackID 4 - TimeScale 1000 - Duration 00:00:29.114
Media Type "text" - Media Sub Type "tx3g" - 13 samples
3GPP/MPEG-4 Timed Text - Size 640 x 272 - Translation X=0 Y=0 - Layer 0so the ttxt stream is 1.5 secs longer than the other streams

[Elias]

Originally posted by bond
so the ttxt stream is 1.5 secs longer than the other streams Yeah. I didn't even write it 1.5 seconds longer when I created the srt file. And trust me, I didn't do it wrong when writing the srt file... at least not to my knowledge. Fact is, I've been getting this issue on every ttxt file I've added.

[bond]

Originally posted by Elias
Yeah. I didn't even write it 1.5 seconds longer when I created the srt file. And trust me, I didn't do it wrong when writing the srt file... at least not to my knowledge. Fact is, I've been getting this issue on every ttxt file I've added. indeed, i made a bug report on the gpac bugtracker

[bond]

and jeanlf meant this is fixed already in latest cvs :)

[Elias]

Originally posted by bond
and jeanlf meant this is fixed already in latest cvs :) Link?

[bond]

there is no compile available till now

[celtic_druid]

@bond, yeah there is (gpac/dev/mp4box.7z).
@everyone else, you will have to wait until it gets mirrored.

[Elias]

Originally posted by celtic_druid
@bond, yeah there is (gpac/dev/mp4box.7z).
@everyone else, you will have to wait until it gets mirrored. Tried that one, the bug is still there... :/

[celtic_druid]

Well that was an still is built from the latest cvs. Anonymous cvs anyway, which I believe is a little behind.

[Sharktooth]

mirrored.

[celtic_druid]

So maybe when Elias tried it hadn't been mirrored yet? Because I checked before I posted and it had been but I guess that was like 2 hours later.

[bond]

that problem is fixed indeed with cd's latest compile :)

[Elias]

Originally posted by bond
that problem is fixed indeed with cd's latest compile :) Yes, just tried it out. Now, finally we can get this issue off our balls :)

[Sharktooth]

Originally posted by celtic_druid
So maybe when Elias tried it hadn't been mirrored yet? Because I checked before I posted and it had been but I guess that was like 2 hours later.
I mirrored the 2005.04.19 build when as soon as i could since i have to do it manually.
So maybe elias didnt get the latest version.

[Elias]

Originally posted by Sharktooth
I mirrored the 2005.04.19 build when as soon as i could since i have to do it manually.
So maybe elias didnt get the latest version.

Originally posted by Elias
Yes, just tried it out. Now, finally we can get this issue off our balls :) That means I just tried it out and it worked. The bug is indeed gone now. I most likely got the outdated cvs build cd mentioned before that. Let's waste time on finding other bugs :)

[Doom9]

I'm wondering.. does anybody know the ratio of srt file and how much the corresponding timed text stream is going to take in the final mp4?

[Stacey Melissa]

You can check to see exactly what the sub track takes by running mp4box -info on the subs track after it's imported. Which of course doesn't do you much good if you want to know that before it's imported. But I suppose you could add the subs track to its own new .mp4 and then run -info on it.

I haven't checked a lot of movies, but the few I've done have had .srt files around 150KB. After importing, mp4box -info shows them to be around 50KB. I don't know whether that includes any track overhead, though.

Edit: Almost forgot to mention that all my .srt files keep positioning information, which increases their size by about one third over .srt files without positioning.

[Doom9]

hmm.. all I get for info is a number of samples for each track. How does that translate into a size? The only track that could work is audio (assuming it's CBR).. but for the rest, how large is a sample? For instance for the video track I have #samples = number of frames.. but since each frame can have a different size...

[bond]

i think one sample is one text line (including empty lines), which of course can vary in their size

than you have a header which tells how the text should be displayed and which should have the same size for every ttxt stream mp4box creates from a .srt input

[planet1]

It might be a bit basic but this would be my approach:

mp4box -ttxt latin.srt

-> this will create latin.ttxt which should be of the same size as the sub-track in the mp4.

In order to verify that I would compare the filesizes of:

roman1.mp4

roman2.mp4 (=roman1.mp4+sub)


btw mp4box doesnt support Unicode .SRT's (although ttxt is UTF-8), but I doubt there are alot of tools which produce them anyway :rolleyes: .

ave :cool:

[Stacey Melissa]

@Doom9 - Here's an example mp4box command and the output, for what I'm talking about:

mp4box -info 3 "The Matrix - Revolutions.mp4"

Track # 3 Info - TrackID 3 - TimeScale 1000 - Duration 02:09:08.452
Media Type "text" - Media Sub Type "tx3g" - 2115 samples
3GPP/MPEG-4 Timed Text - Size 720 x 352 - Translation X=0 Y=0 - Layer 0

Computed info from media:
Total size 41054 bytes - Total samples duration 7748452 ms
Average rate 42 bps - Max Rate 736 bps

You have to specify the track number of the subs track in order to get the detailed readout.

@planet1 - The size of the .ttxt file is quite a bit larger than what the subs end up as in the .mp4. mp4box uses the .ttxt format simply as an intermediary format that is easy to work with in a text editor.