HowTo mux Subs into MP4 via MPEG-4 Systems

[bond]

As i mentioned in my FAQ the MPEG-4 Standard expects any Text (including Subs) to be stored in BIFS format in MP4 atm

Now i know of two tools which are capable of converting SRT subs into BIFS format:
first dvd2mp4gui (http://www.enst.fr/~concolat/mpeg-4/dvd2mp4gui.exe), with a nice newbie proof interface (but the author removed it from his webspace :( )

the second option is the very powerfull mp4box (http://www.rarewares.org/mp4.html) tool from gpac ("Gpac MPEG4 toolbox")
as its commandline and expecting a bt/xmt script as input this small guide may help you a bit with it:

1) download my small mp4sub script from sourceforge (https://sourceforge.net/projects/mp4menu/)
2) place the script, mp4box and all needed media files in the same folder
3) open the script and search for "pixelWidth", "pixelHeight" and TextAppearance.position and adjust the values fitting your input as described in the script
4) adjust the names of your input files in the script
5) run the following commanline:
mp4box.exe -mp4 mp4sub_0.1.bt6) thats enough to be able to play the output MP4 file with any good systems 2D capable player, like GPAC's Osmo4 (http://rarewares.hydrogenaudio.org/mp4.html)
EnvivioTV (a systems decoder usable in directshow) has some drawbacks, which means it needs some special treatment to make subs work, which is described below
if you play the MP4 file with a non-systems capable player (ie 3ivx) the subs will simply be ignored



Special Workaround for EnvivioTV

i saw that EnvivioTV doesnt seem to be able to handle a file with 2 BIFS tracks (Animationstream) correctly, as the ones created with the steps 1-6 (it also doesnt handle MP3 decoding), thats why we have to merge them into 1 to make it understandable for EnvivioTV, which works as follows

7) after running steps 1-6, rename the output .mp4 file you got to output.mp4 and run the following commanline:
mp4box.exe -bt output.mp4this will create a new .bt script, named output.bt which we now have to merge with the old one, mp4sub_0.1.bt (which already has the updated paths, subtitle size/font aso set...)

8) open your first mp4sub_0.1.bt, which you worked on already, and change "DEF SubtitleText" to "DEF N1"

9) delete the following lines there too
DEF MovieSubtitle AnimationStream {
url [ "od:30" ]
}and
ObjectDescriptor {
objectDescriptorID 30
esDescr [
ES_Descriptor {
ES_ID 3
muxInfo MuxInfo {
fileName "your_srt-subs_name.srt" # only srt subs are supported in mp4box!
textNode SubtitleText
fontNode SubtitleFont
}
}
]
}10) open the new output.bt and copy all lines which look similar like this RAP AT 200 IN 3 {
REPLACE N1.string BY ["But I don't understand, " "Coelho Bastos."]
}

RAP AT 2870 IN 3 {
REPLACE N1.string BY []
REPLACE N2.style BY ""
}and place them in the same order as they were in output.bt at the end of all lines in your first mp4sub_0.1.bt

11) change all lines in the following way that RAP AT 2870 IN 3 {
REPLACE N1.string BY ["But I don't understand, " "Coelho Bastos."]
REPLACE N2.style BY ""
}becomes AT 2870 {
REPLACE N1.string BY ["But I don't understand, Coelho Bastos."]
}which means you have to
- delete all ' REPLACE N2.style BY "" ' lines,
- delete all ' RAP ' and
- delete all ' IN 3 '
- delete all ' " " ', which are placed in the middle of the subtitletext
this process can be made much easier and automated if you use the "search and change" option in m$ word

12) if your audio stream is not in AAC (ie MP3) you need to convert it to AAC! EnvivioTV isnt able to decode other audio formats than AAC
if you have to reencode the audio dont forget to adjust the name of your input audiofile in the script

13) after that run the following commandline again on the new created mp4sub_0.1.bt:
mp4box.exe -mp4 mp4sub_0.1.btnow you have merged the two BIFS streams into one and will be able to play the BIFS subtitles also with EnvivioTV in every dshow based player (WMP9, BSPlayer, Media Player Classic) and also in Quicktime and Realplayer, but of course also with every other Systems 2D player, like Osmo4

13) enjoy :)


edit:
changed the title from "the spec compliant way" to "via MPEG-4 Systems", as the way ahead handles subs in mp4 is not not-spec-compliant

edit2: added a special guide, which makes BIFS subs working with EnvivioTV in quicktime, realplayer and directshow

[Phobos]

hey dude, new mp4box at rarewarez wont run with your script, it says parsing done and right after it i get a windows error something like "referenced memory cant be written" or stuff. so it wont do a thing, and im using the very same settings as the old one :(

[bond]

no problem here

[Phobos]

now i tried out another movie but i got like a 1000 errors like this while importing avi:

"warning: invalid frame ts (CTS<DTS) - frame delay may be wrong"

anyway it now plays well on osmo (i still dont get any sound), but i couldnt see the subtitles this time.

And the "top left corner bug" on media player classic remains

[bond]

hm this message has something to do with the new b-frame handling feature in mp4box

what codec, settings did you use to create the file?

[Phobos]

XVid 1.04 i used VHQ 4, chroma, no trellis, no packet bitstream, MPEG, adaptive quantization no qpel or gmc, motion search precision 6, no turbo and the lowest quant at 2

[JasonFly]

I have a question
That seems to me that we can only mux txt subs into mp4 with the tools you adviced.
I assume that we cannot mux Vobsub subs into mp4, Am I right?

That seem to be a new vesion of the 3ivx splitter these days, can he decode mp4 subs or not?

Thanks

[bond]

well there are atm two ways to have subs in mp4
1) the one described here, which means storing the subs in bifs format (which would btw allow using all sorts of funky designs for the subs (ie "star wars subs" (c) shitowax ;) ))

you can create such subs with any bifs encoding tool available (ie mp4box, mp4tool, xmtbatch)
note that only mp4box handles .srt input directly! with the other tools you would have to transform the subtitle text by hand (as no automatic conversion tools exist atm)

2) nero recode2 adds the subs directly in the vobsub format to mp4, also atm only the nero dshow filter support playing these
but note that 3ivx is also considering handling these subs

[Snollygoster]

After doing what you described with mp4box and the script I get the message parsing done and then cpu usage is 100% by mp4box and nothing happens for a long time. Is that normal? Where it should output the multiplexed mp4 file? If it was just multiplexing shouldn't the cpu usage be less than 100% (avimux, vdubmod or any other tool that just multiplexes uses the hard disk and cpu is around 30%)...

[bond]

hm interesting... what cpu do you use?
also mp4box often rewrites the stream, which isnt displayed but needs some time, so its good to wait

also its important that you use the latest cvs as it includes some important fixes

[gotaserena]

Hi Bond,

The subtitles are supposed to show as "BIFS" type of stream in MP4UI, aren't they?

If yes, I will have to figure out why Osmo4 is not showing them (as a matter of fact I'm still having trouble having it decoding AAC audio, but that's another problem.)

By the way, GPAC 0.1.3 is out. I hope someone has a good mind of posting some binaries soon. I can't get to compile them at home...

[SeeMoreDigital]

Let me begin by saying I didn't know whether or not I should put this request in the 'New A/V Formats - Containers' section but here goes!

I'm after a short Mpeg4 video encode with subtitles all of which needs to be placed in a nice MP4 container. The subtitles must be encoded in the "official" way (ie: the xmt/bt text format) please!

Can somebody create and post an example please, so I can add to my 'test file' collection.

Many thanks

[bond]

Originally posted by gotaserena
The subtitles are supposed to show as "BIFS" type of stream in MP4UI, aren't they? exactly

If yes, I will have to figure out why Osmo4 is not showing them (as a matter of fact I'm still having trouble having it decoding AAC audio, but that's another problem.)can you upload a small sample, including the subs maybe?

and what problems do you have exactly with aac?

[gotaserena]

Originally posted by bond
can you upload a small sample, including the subs maybe?

I'll hunt around for a server... Theses things aren't easy to come by...

and what problems do you have exactly with aac?

The usual "Stream setup failure: media codec not found" when I load the file. I use the 3ivx muxer, and did (re)install the 0.1.2 version which is supposed to have libfaad built-in.

[bond]

Originally posted by gotaserena
The usual "Stream setup failure: media codec not found" when I load the file. I use the 3ivx muxer, and did (re)install the 0.1.2 version which is supposed to have libfaad built-in. [/B]hm kilg0r3 reported the same here (http://forum.doom9.org/showthread.php?s=&threadid=74536) some days ago
try what is described there, it worked for kilg0r3

Originally posted by SeeMoreDigital
I apologise, I totally missed this thread and posted the this request (http://forum.doom9.org/showthread.php?s=&postid=477618#post477618) in another forum. Maybe you guys could help me out please?i merged your thread with this one :)

hm if you need the sample to be widely compatible (ie with quicktime) and you only have few (always to be displayed) subs you maybe want to hardcode them onto the video stream anyways

[SeeMoreDigital]

Originally posted by bond
i merged your thread with this one :)

hm if you need the sample to be widely compatible (ie with quicktime) and you only have few (always to be displayed) subs you maybe want to hardcode them onto the video stream anyways Thanks for doing that.

Actually, this time I'm not after a 'widely compatible' QuickTime solution! Nope, this time I'm interested to see if MP4 compliant subs will work with my Xcard and/or with software players.

Personally, I've never generated any type of encode with subtitles so I would not know where to start!

Cheers

[bond]

Originally posted by SeeMoreDigital
Actually, this time I'm not after a 'widely compatible' QuickTime solution! Nope, this time I'm interested to see if MP4 compliant subs will work with my Xcard and/or with software players.they will work with players capable of systems decoding
i doubt that your xcard handles this :(

[SeeMoreDigital]

Originally posted by bond
they will work with players capable of systems decoding
i doubt that your xcard handles this :( It's worth a go though.

You never know.... I had to wait 2 years for an Mpeg4 codec that could provide anamorphic signalling, decent sounding AAC audio creation software and easy to use MP4 muxing tools before I knew the Xcard could handle anything remotely MP4-able :D

Cheers

[gotaserena]

Hi,

I saved the stream from the file with mp4ui and opened it in a hex editor. The subtitle info is indeed there, at least the text is. Is there any other player that will display them?

I'll try and upload a sample later as soon as I figure out how to cut up srt files. :cool:

[bond]

Originally posted by gotaserena
I saved the stream from the file with mp4ui and opened it in a hex editor. The subtitle info is indeed there, at least the text is. Is there any other player that will display them?you mean you extracted the bifs stream with mp4ui?

hm if i remember it correctly mp4box places the sub stream as a seperate bifs stream in mp4 (so basically we have two bifs streams than)
now if gabests vsfilter could read out the bifs stream containing the subs-only it would be great
basically he only needs to read out the time and the text to display, the rest could be ignored

[gotaserena]

Yeah I did that. I uploaded a sample, I sent the address to you by PM. If you could check if the subtitles are showing up I'd be much obliged. :)

[bond]

the subs are displayed very nicely :)

[SeeMoreDigital]

May I have a look too please?

[gotaserena]

Sent!

It's good that it work -- thanks for the script, btw -- but it's a pity that I can't see them.

I'm fearing it will be down to good'ld "format c: /s" next weekend...

[bond]

gotaserena,
i saw that you used -90 for placing the subs, maybe you want to try -120 or so to place the subs a little bit lower

[SeeMoreDigital]

I can't see anything :( .

I'm assuming you're both using the Osmo4 player?

Cheers

[gotaserena]

Is there any other which would display the subtitles?

[SeeMoreDigital]

Not that I know of :D

I also tried opening the file with Nero's ShowTime player but it just crashed. Which is quite unusual for this player.

The Xcard could not display the subtitles. In fact it struggles to play the file due to the inclusion of the Mp3 audio stream.

Something must be well wrong with my install as even bonds Star Wars 'menu' interactive clip does not work anymore with the Osmo player...

Bummer!

[bond]

Originally posted by gotaserena
Is there any other which would display the subtitles? well you could try every systems 2D player out there, i think links to them are available via the GPAC homepage

but i now tried to make them work with EnvivioTV (EnvivioTV is a dshow filter set which offers Systems playback in all directshow player, but also in Quicktime and Realplayer) but it doesnt seem to handle mp4 files with 2 BIFS tracks correctly, which means it cant display the subs which are stored in the second BIFS track

so we have to merge the two tracks into 1, to make EnvivioTV understand it
on how to do it i added some steps to the guide on the frontpage, it might look difficult at first sight but it isnt

gotaserena,
i changed your sample that way and now i am able to display your subs also in every dshow based player (like wmp9, bsplayer, media player classic aso) but also in quicktime and realplayer with enviviotv installed
it works great! i hope the guide is understandable written, if not feel free to shout at me ;)

beaware that envivio doesnt handle mp3 decoding so you will have to encode your audio with aac, which shouldnt be a problem as aac provides better quality anyways

[SeeMoreDigital]

Originally posted by bond
gotaserena,
i changed your sample that way and now i am able to display your subs also in every dshow based player (like wmp9, bsplayer, media player classic aso) but also in quicktime and realplayer with enviviotv installed
it works great! i hope the guide is understandable written, if not feel free to shout at me ;) Well I won't shout at you!

Can you upload it so we can all give it a go?

Cheers

[gotaserena]

Great, Great, Great!!!! With a dshow there is more chance that the Xcard (or JovePlayer) would support it... I can't wait to get back home and try it out.

BTW feel free to modify the file in any way you see fit. This was a rip I made as I was beginning to be interested in MPEG-4, and it is positively horrible at that! :)

Edit: Yeah it did work but it is surely a lot of work. I guess that one could easily write a tool that translates the .srt subtitles into the format that the mp4mux could understand.

And I've still to figure out why the 3ivx splitter keeps thinking that the video is 320x240... :confused:

[bond]

Originally posted by gotaserena
I guess that one could easily write a tool that translates the .srt subtitles into the format that the mp4mux could understand.yes it shouldnt be that hard, basically you simply have to recalculate the time (dunno what time format systems uses)

to make the process of the deleting of the lines and the changing of RAP AT aso.. easier you can also use the automatic "search and change" function in word

And I've still to figure out why the 3ivx splitter keeps thinking that the video is 320x240...thats because the 0.1.2 version of mp4box was using some old version of the specs i think, its already fixed in the cvs, expect a new binary on rarewares soon

Originally posted by SeeMoreDigital
Can you upload it so we can all give it a go? grap it here (http://mp4menu.sourceforge.net/download/mp4sub_sample.zip)
you can play it in osmo4 and if enviviotv is installed also in quicktime, realplayer and directshow (the audio will not work as enviviotv doesnt handle mp3)

[SeeMoreDigital]

Originally posted by gotaserena
...And I've still to figure out why the 3ivx splitter keeps thinking that the video is 320x240...I also noticed that when you run your encode thru' MP4UI, no 'frame size' (ie: width and height) info is displayed. Would this be the cause of the problem?

bond, thanks for the upload

[gotaserena]

Originally posted by bond
yes it shouldnt be that hard, basically you simply have to recalculate the time (dunno what time format systems uses)

From what I could see from the time codes, it's just the time in miliseconds. Then it gives an order at 10 miliseconds before the next subtitle so that the player can erase the last one. I'll look around the mp4box source files and see what I can do.

Edit 1: BTW, if you are thinking about writing a small program to do this, you may consider jumping one line -- adding " " in the .bt script -- for every new line written in the subtitle. The base line skip is too small and sometimes there's overlap between two consecutive lines.

Edit 2: Bond, may I ask you one more favour? Could you please download a slightly better version of the file, already with subtitles and mp4 sound, from the same address I gave you before? I don't want to ruin my reputation having that awful piece of rip floating around :D

[bond]

Originally posted by gotaserena
Edit 1: BTW, if you are thinking about writing a small program to do this, you may consider jumping one line -- adding " " in the .bt script -- for every new line written in the subtitle. The base line skip is too small and sometimes there's overlap between two consecutive lines.yeah i also saw this, but basically its only there with enviviotv :rolleyes:

so when you add "" you will get too much space with every normal systems player, like osmo4, so i wouldnt do this

you can also simply delete the " " in the middle, which will lead to that the subs are not displayed in two lines but only in 1 anymore (good solution for envivio and osmo4)

Edit 2: Bond, may I ask you one more favour? Could you please download a slightly better version of the file, already with subtitles and mp4 sound, from the same address I gave you before? I don't want to ruin my reputation having that awful piece of rip floating aroundi uploaded a new sample which now contains all updates envivio needs to behave nicely :D
- aac audio stream (64kbps)
- subs displayed in 1 line, not 2 anymore

grap it here (http://mp4menu.sourceforge.net/download/mp4sub_sample_v2.zip)

Originally posted by SeeMoreDigital
I also noticed that when you run your encode thru' MP4UI, no 'frame size' (ie: width and height) info is displayed. Would this be the cause of the problem?yep, that has something to do with it i think

[SeeMoreDigital]

Try as I might I can't get any direct show player to display the subtitles. I must have something missing or currupt on this PC.

Never mind. I guess I will have to try again when I finish moving my PC's around.

Cheers

[bond]

are you sure that you have installed enviviotv? ;)

[SeeMoreDigital]

Originally posted by bond
are you sure that you have installed enviviotv? ;) It would seem my installed version (v1.1) of EnvivioTV was not up to the task!

After installing a newer version, I'm happy to report that I'm now able to view the subtitles...

That said, I wonder if one of you guys could generate a new encode with subs as the current encode (with it's non existent image pixel frame size) refuses to work in Nero's ShowTime player!

Cheers


EDIT: I forgot to mention that I had trouble obtaining a copy of the EnvivioTV plugin. I tried the 'download' tag in bonds helpful MP4 FAQ (which directs you to v1.5) but could not get any joy! Is it just me or have things changed?

[SeeMoreDigital]

I read the thread you posted about Mpeg4 subtitles (http://forum.doom9.org/showthread.php?s=&threadid=74972) in the General Section of the forum.

Looks like things are gonna change!

Cheers

[gotaserena]

I guess one can now hope that one of the people involved in programming subtitles filters will begin to pay attention to mp4...

Until bond's post, searching for "mp4" in the subtitles forum yielded 0 (zero) results... :(

[bond]

Originally posted by SeeMoreDigital
Looks like things are gonna change!hm yes, maybe
from what i read till now the mp4s subs are technically similar to subformats widely used already, like srt/ssa aso, meaning they are stored as text and not encoded as binary as with bifs

also it shows that its possible to store all sorts of subs in the mp4 container (as ahead does with the picture vobsubs for example), but still to ensure broad interoperability its good to have 1 subtitleformat defined (like its the case with audio and video in mp4 too already) and thats ISO 14496-17 "mp4s"

Originally posted by gotaserena
[B]I guess one can now hope that one of the people involved in programming subtitles filters will begin to pay attention to mp4...yep, lets wait and see :)


Originally posted by SeeMoreDigital
EDIT: I forgot to mention that I had trouble obtaining a copy of the EnvivioTV plugin. I tried the 'download' tag in bonds helpful MP4 FAQ (which directs you to v1.5) but could not get any joy! Is it just me or have things changed?thanks for the hint, it seems they removed enviviotv from their server

[Drachir]

Hi, i used the MP4SUB.bt to create a mp4 file with SUB (BIFS). Worked well, but is there a way to place a black border around the text ? The sub are badly to read (look at the picture). I looked by myself but couldn't find anything useful.



I changed the bt File a little bit, now the sub are better to read:
Transform2D {
translation IS position
children [
Shape {
appearance Appearance {
material Material2D {
emissiveColor 1 1 0
filled TRUE
}
}
geometry is text
}
Shape {
appearance Appearance {
material Material2D {
emissiveColor 0 0 0
filled FALSE
}
}
geometry is text
}
]
}

and @ fontStyle the black Border ist better to see with style "BOLD"

[bond]

yep it should be possible (there are various ways to play around with the display of text), still i dont know the command of what you want and i dont have the systems specs at hand, so it might be the best if you ask this question in the gpac forum on sourceforge

[aspianatte]

Could someone help me how to mux subtitle into mp4 container? I have actually only mp4 audio track obtained with nero encoder. Then i must encode video track ( there are some raccomandation here?). I have never done subtitle before so you could help me with simple passage!

P.S. I have downloaded mp4box from rarewares

[Drachir]

I wouldn't use subtitle in mp4 for now. Until there are tools/player that support ISO 14496-17 ( http://forum.doom9.org/showthread.php?s=&threadid=74972 ) or if more player decode BIFS ( don't think this will ever happen :( - but I hope I am wrong ).

MP4Box convert srt subs to BIFS but there are not much player that support BIFS :( .

[aspianatte]

This is the content of directories:
Directory di E:\mp4box

03/10/2004 18:46 <DIR> .
03/10/2004 18:46 <DIR> ..
03/10/2004 17:27 2.036.736 audio.mp3
03/10/2004 16:37 3.575.315 audio.mp4
10/06/2004 22:26 720.896 MP4Box.exe
03/10/2004 18:31 511 MP4Box.exe.lnk
03/10/2004 18:58 4.201 mp4sub_0.1.bt
03/10/2004 18:24 1.415 subtitle.srt
02/10/2004 15:23 12.363.776 Video.avi
7 File 18.702.850 byte
2 Directory 1.706.373.120 byte disponibili

This is the output of mp4box:

E:\mp4box>mp4box -mp4 mp4sub_0.1.bt
Parsing done
AVI DIVX video import - 640 x 256 @ 20.00 FPS
Indicated Profile: Simple Profile @ Level 1
Has B-Frames (1 max consecutive B-VOPs - packed bitstream) - Adjusting CTS
CTS Adjustment done
WARNING: indicated profile doesn't include B-VOPs - forcing Advanced Simple Prof
ile @ Level 5
Import results: 2549 VOPs (24 Is - 1465 Ps - 1060 Bs)
Removed Frames: 0 VFW delay frames - 1060 N-VOPs
import mp4 sample 2741 / 2741

Error: Bad Parameter

In the attachment there is your script edited by me. what is this Error: Bad Parameter?? And another thing: my video is 25 fps and not 20 fps...

[Drachir]

Until a moderator enable your attachment you could try if it help to use a newer version of MP4Box : http://celticdruid.no-ip.com/xvid/gpac/

But normaly the bt script from bond should work.
You will find help here http://sourceforge.net/forum/?group_id=84101 too.

[bond]

Originally posted by aspianatte
Error: Bad Parameter

In the attachment there is your script edited by me. what is this Error: Bad Parameter?? And another thing: my video is 25 fps and not 20 fps...hm strange that you get that error

having a quick look at your script i dont see any possible failures, but maybe you want to import the streams without the .bt script into .mp4 with mp4box and check whether you get the error too that way

[pogo stick]

Originally posted by aspianatte
In the attachment there is your script edited by me. what is this Error: Bad Parameter?? And another thing: my video is 25 fps and not 20 fps...
I just tryed it with script you attached and my files. Everything is alright.
Did you try it with new version of mp4box?

[sbeyer20]

Hi,

is there a way to controll the TTXT Streams with an BIFS Menu like I can control the animation stream? I mean is it possible to switch with an menu the used ttxt stream? How?

Thanks