Some tools for Ogg Media files

[Suiryc]

Hi

For those who are interested, I tried to make a command line multiplexer for ogm files (see attachment).

It should work with AVI (video stream only), Ogg/Ogm, AC3 and SRT subtitle files as input.
I also included a tool to extract information from an Ogg/Ogm file.

Be aware that thoses tools aren't perfect (far from it), so USE AT YOUR OWN RISK!

NB :
- you need the ogg.dll and vorbis.dll files (installed by OggDS filter I think) for the tools to work
- these programs are inspired (not based) by ogmtools (developped by Moritz Bunkus)
- these programs don't use DirectShow

[Edit]Attachment removed. See posts below to find the latest versions of those tools [/Edit]

[Koepi]

Great job, I hope the tools work as expected.

Any plans to make them opensource later on?
I'd like to add that to the oggmux sourceforge project, making it a real OGM suite there...

Thanks for your work,

best regards,
Koepi

[Suiryc]

Great job, I hope the tools work as expected.

Me too .
One more thing : compared to OggMux one cannot specify user comments (i.e. language of audio or subtitle, nor chapters), nor make the output file splitted.

Any plans to make them opensource later on?

Yes of course, just need to be sure this works, and clean up a little the source code


Now, new day, new tool : the Demuxer (see attached file)

You should be able able to demux known stream type, i.e. :
- video stream -> AVI file (be aware there may be problems with files larger than 2GB)
- vorbis stream -> Ogg file
- AC3 stream -> AC3 file
- Subtitle stream -> SRT subtitle file

NB :
- again "USE AT YOUR OWN RISK"
- again needing ogg.dll
- again inspired by Moritz Bunkus ogmtools
- again not using DirectShow

[Edit]Attachment removed, see posts below to find the tool [/Edit]

[Dark-Cracker]

hi,

very nice work but if i can suggest u could try to make an oggcut tool because a lot of people (especialy under win2k) have reported a crash with the tobias tool (oggcut) .

bye

[Taranli Maren]

Thank you very much. This is going to be quite useful, though I have had a problem with the ogm demuxer. It said "Video stream Packet contains more than one Frame, skipped." And the avi ends up smaller than its supposed to be. It plays, but only if I skip to the second keyframe, it crashes when I try to play the first keyframe. The two ogg files also ended up smaller than they were before, but they played perfectly. The srt was good. The video was divx5 compressed if that makes a difference. The ogm was muxed originally with oggmux.

Thanks :)

[Koepi]

Heh, I just started coding an OggDeMux based on Tobias' filters, but I delay that again - it's so complicated to figure all these filter properties out in a program :-/

Well, maybe I find some motivation again later, but for now I'm fed up with DShow framework again

Would've been nice to see the difference...

Best regards,
Koepi

[Suiryc]

Originally posted by Taranli Maren
It said "Video stream Packet contains more than one Frame, skipped."

Strange, I though this message should never appear (I just included it in my sources in case ...). Will see if there could be something else causing that ...

And the avi ends up smaller than its supposed to be.

I also ended up with smaller AVI files after demuxing, but VirtualDub said me there were the same number of frames, so it seems OK.
Anyway if there are problems here I cannot do anything since I completely rely on AVILib
This may be the fact I use AVILib which may behave differently than the encoder (VirtualDub, ...) that have been used to make the original video.

It plays, but only if I skip to the second keyframe, it crashes when I try to play the first keyframe.

This is due (I think) to the "Video stream Packet contains more than one Frame, skipped" message.
As I said this should not happen because this would mean a Packet say it contains at least two video frames, which make then impossible to know where begins the second one and the others :/
Oh yeah, could you use "OGMInfo -p -v3 -l C:\Log.txt ..." on your ogm file and send the first lines (headers and first Page containing "D Packet" from the video stream) so that I could see what is the problem with the first frames in your video stream ?

The two ogg files also ended up smaller than they were before, but they played perfectly.

Are you sure ? There should not be any difference (not even a single byte) before muxing and after demuxing ogg vorbis streams.
But it may be due to DirectShow ... I noticed last time that an ogm file I made using OggMux ended with a "redundant" End Of Stream page for the vorbis stream (while the stream had already ended earlier in the file) ...

The video was divx5 compressed if that makes a difference.

No this should not make a difference.


Happy to see it helps some of you

[Taranli Maren]

I may be giving you too much here, but better safe than sorry...

    0,  1V Page       0 (  0) 28+   57 bytes, (INITPAGE, !CONTINUED), pos          0,        0.00->       0.00ms  SYNC,   1 packet(s)
    0,  1V H Video Header : DX50, 640x480, 23.976 fps
    0,  1V H Packet        0 :      57 bytes,           ,  BOS, !EOS, pos          0
    1,  1v Page       0 (  0) 28+   30 bytes, (INITPAGE, !CONTINUED), pos          0,        0.00->       0.00ms  SYNC,   1 packet(s)
    1,  1v H Vorbis Header : 91200 bps, 2 channels, 48000 Hz
    1,  1v H Packet        0 :      30 bytes,           ,  BOS, !EOS, pos          0
    2,  2v Page       0 (  0) 28+   30 bytes, (INITPAGE, !CONTINUED), pos          0,        0.00->       0.00ms  SYNC,   1 packet(s)
    2,  2v H Vorbis Header : 91200 bps, 2 channels, 48000 Hz
    2,  2v H Packet        0 :      30 bytes,           ,  BOS, !EOS, pos          0
    3,  1T Page       0 (  0) 28+   57 bytes, (INITPAGE, !CONTINUED), pos          0,        0.00ms->             SYNC,   1 packet(s)
    3,  1T H Text Header
    3,  1T H Packet        0 :      57 bytes,           ,  BOS, !EOS, pos          0
    4,  2T Page       0 (  0) 28+   57 bytes, (INITPAGE, !CONTINUED), pos          0,        0.00ms->             SYNC,   1 packet(s)
    4,  2T H Text Header
    4,  2T H Packet        0 :      57 bytes,           ,  BOS, !EOS, pos          0
    0,  1V Page       1 (  0) 28+   48 bytes, ( ...... , !CONTINUED), pos          0,        0.00->       0.00ms  SYNC,   1 packet(s)
    0,  1V C Packet        1 :      48 bytes,           , !BOS, !EOS, pos          0, Comment Packet
Vendor     : Xiphophorus libVorbis I 20011217
    1,  1v Page       1 (  0) 28+   68 bytes, ( ...... , !CONTINUED), pos          0,        0.00->       0.00ms  SYNC,   1 packet(s)
    1,  1v v Packet        1 :      68 bytes,           , !BOS, !EOS, pos          0
    2,  2v Page       1 (  0) 28+   69 bytes, ( ...... , !CONTINUED), pos          0,        0.00->       0.00ms  SYNC,   1 packet(s)
    2,  2v v Packet        1 :      69 bytes,           , !BOS, !EOS, pos          0
    3,  1T Page       1 (  0) 28+   68 bytes, ( ...... , !CONTINUED), pos          0,        0.00ms->             SYNC,   1 packet(s)
    3,  1T C Packet        1 :      68 bytes,           , !BOS, !EOS, pos          0, Comment Packet
Vendor     : Xiphophorus libVorbis I 20011217
Comment  1 : LANGUAGE=English
    4,  2T Page       1 (  0) 28+   68 bytes, ( ...... , !CONTINUED), pos          0,        0.00ms->             SYNC,   1 packet(s)
    4,  2T C Packet        1 :      68 bytes,           , !BOS, !EOS, pos          0, Comment Packet
Vendor     : Xiphophorus libVorbis I 20011217
Comment  1 : LANGUAGE=English
    0,  1V Page       2 (  0) 44+ 4335 bytes, ( ...... , !CONTINUED), pos          0,        0.00->       0.00ms  SYNC,   0 packet(s)
    1,  1v Page       2 (  0) 42+ 3606 bytes, ( ...... , !CONTINUED), pos          0,        0.00->       0.00ms  SYNC,   1 packet(s)
    1,  1v v Packet        2 :    3606 bytes,           , !BOS, !EOS, pos          0
    2,  2v Page       2 (  0) 42+ 3606 bytes, ( ...... , !CONTINUED), pos          0,        0.00->       0.00ms  SYNC,   1 packet(s)
    2,  2v v Packet        2 :    3606 bytes,           , !BOS, !EOS, pos          0
    3,  1T Page       2 (  0) 28+    5 bytes, ( ...... , !CONTINUED), pos          0,        0.00ms->             SYNC,   1 packet(s)
    3,  1T D Packet        2 :       5 bytes,  SYNCPOINT, !BOS, !EOS, pos          0,        0.00->    7087.00ms,   7087 samples
 
    4,  2T Page       2 (  0) 28+    5 bytes, ( ...... , !CONTINUED), pos          0,        0.00ms->             SYNC,   1 packet(s)
    4,  2T D Packet        2 :       5 bytes,  SYNCPOINT, !BOS, !EOS, pos          0,        0.00->    7087.00ms,   7087 samples
 
    0,  1V Page       3 (  0) 44+ 4335 bytes, ( ...... ,  CONTINUED), pos          0,        0.00->       0.00ms  SYNC,   0 packet(s)
    0,  1V Page       4 (  0) 44+ 4283 bytes, ( ...... ,  CONTINUED), pos          0,        0.00->       0.00ms  SYNC,   1 packet(s)
    0,  1V D Packet        2 :    9638 bytes,  SYNCPOINT, !BOS, !EOS, pos          0,        0.00->      83.42ms,      2 samples
    0,  1V Page       5 (  0) 44+ 4224 bytes, ( ...... ,  CONTINUED), pos          3,        0.00->     125.12ms  SYNC,   2 packet(s)
    0,  1V D Packet        3 :    5278 bytes, !SYNCPOINT, !BOS, !EOS, pos         -1,        0.00->      41.71ms,      0 samples
    0,  1V D Packet        4 :     731 bytes, !SYNCPOINT, !BOS, !EOS, pos          3,       41.71->      83.42ms,      0 samples
    1,  1v Page       3 (  0) 50+ 4131 bytes, ( ...... , !CONTINUED), pos      22080,        0.00->     460.00ms  SYNC,  23 packet(s)
    1,  1v v Packet        3 :      12 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet        4 :     112 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet        5 :     176 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet        6 :     187 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet        7 :     195 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet        8 :     188 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet        9 :     186 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet       10 :     186 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet       11 :     188 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet       12 :     180 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet       13 :     198 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet       14 :     193 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet       15 :     176 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet       16 :     191 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet       17 :     194 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet       18 :     197 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet       19 :     198 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet       20 :     183 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet       21 :     194 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet       22 :     196 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet       23 :     201 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet       24 :     190 bytes,           , !BOS, !EOS, pos         -1
    1,  1v v Packet       25 :     210 bytes,           , !BOS, !EOS, pos      22080

[Suiryc]

Thanks.

Originally posted by Taranli Maren
I may be giving you too much here, but better safe than sorry...

...
    0,  1V D Packet        2 :    9638 bytes,  ...  0.00->      83.42ms,      2 samples
...

So here is the problem. I wonder how the decompressor can handle this (I mean if what say the ogm file is correct and there are realy 2 frames).
I can't do anything here, except giving this "new" version of the demuxer (it won't skip the data but send it to AVILib as if it where one frame, and warn you that there were n frames in a packet, so don't know what this will give). Please try and tell if this solve the first frame problem.

PS : when the attachement is validated, I will remove the previous one.

[Edit]Attachment removed. See posts below to find the latest version of the tool [/Edit]

[Selur]

thx, for the tool !!
and like "Dark- Cracker" mentioned a oggCut tool would really be appreciated (especially if it can handle 2GB+ ogm files), but no hurry it's just a thing I'd like to have mentioned,..

Cu Selur

do you have a little html page or something i could look for new versions?

[Suiryc]

Sorry, don't have a HTML page.
I only posted those tools here, and I will let you know when I make some changes.

[Taranli Maren]

Sorry it took me so long to get back to this. That new version works. The avi plays properly, though the filesize is still different, thats not a problem. Thanks

[Mosu]

It's always good to see that one's source code is useful to others

[Suiryc]

@Taranli Maren :
I finally found what was the problem with your video . It is due to dropped frames.
When you have dropped frames in your video, say 1 good frame and then 4 dropped ones, then the OGM (obtained with the DirectShow Ogg Multiplexer) will have a Packet that say to contain 5 samples.
Here you are lucky because you have only one dropped frame, and since I didn't know what happens with dropped frames, in fact my Demuxer acts like it throw away them (so the final AVI clip is shortened by the number of dropped frames you have in your video).

@Mosu :
Hi, your tools was a great help at the beginning because I wanted to see if it was possible to mux "manually" streams to OGM .
Before I publish my sources, I can however tell you about a problem I encountered while rewriting the whole thing (well in fact I didn't quite understand all what was doing your source code, so I rewrote it from scratch ) : The granulepos correspond to the end of data and not the beginning.
Why do I tell you that? Simple : as you when I mux, at the end I decide in which order place the Pages in the file, and of course this order is defined by the timestamp of the Page, the timestamp is calculated depending on he granulepos and the samplerate. The problem : as the granulepos correspond to the end of the Page, it's the same with the timestamp, so in fact ordering Pages depending on their timestamp is not that good (I experienced severe stuttering with my first tests ). In my case I introduced a new variable named last_granulepos that contains the last known granulepos of the stream, and I calculate the "beginning" timestamp of a Page with that value, and of course now the video is more "stable" .
Hope this will help you, and again thanks for your tools .


Well I think that today I will post the latest versions of my tools, plus a cutter (well in fact it's just a splitter at the moment ...).
Expect something great (I hope )

[Mosu]

Hi.

The granulepos correspond to the end of data and not the beginning.

Yes, I know, and I've already experimented with it a bit. But as a matter of fact the OggDS drivers don't do that correctly either.

Nevertheless I've just modified the relevant files (the packetizers and ogminfo, which now bases decisions about 'being out of sync' on the start time (the last page's granulepos of the current stream) rather than on it's end time). The reason I haven't done it before is that with the interleaving being slightly wrong did not give me any problems at all - neither with mplayer nor with OggDS (e.g. ZoomPlayer, WMP6).

Now it should (like 'I hope but am not sure' ) be ok. I'll release a new version soon (as soon as I get MP3 support working correctly...)

[Suiryc]

Eh eh my newest version also include MP3 support, and another format (but you will know more when I post the tools).


BTW your idea of using Reader and Packetizer was really great, but I found a thing that helped me a lot compared to what you did :
You have a Reader for each type of input file (me too), and a Packetizer for each type of stream (one for video, one for vorbis, one for PCM), but I succeeded in having only one Packetizer for everything, which is a bit more handy (I think).
The hard thing was to find how to calculate sample_rate and timestamp in each case : in fact there is one formula that apply to everything, but I found it the hard way (first by guessing it - and it was a good guess - and then by "verifying" it).
So compared to you what I do is fill the stream_header in the Reader, and send it to the Packetizer (which then have everything it needs to calculate timestamps).

As I didn't know that much thing about multimedia and everything, it was also hard for me to find how to correctly fill this damn stream_header , and maybe you are experiencing same things with your MP3 support :
I had the really bad idea that stream_header.samples_per_unit was related to the number of samples contained in one Frame (i.e. 1 sample - 1 frame - for video, and for example n samples for 1 Frame of MP3 - n=384 for LayerI and 1152 for LayerII/III) ... but I ended up with really bad timing at the end (I first experienced that with AC3 : there was no problem playing the file, but the indicated time was completely wrong). Finally I found that it is related to the final sample_rate (i.e. if the MP3 or AC3 is 48KHz, then samples_per_unit must be set to 48000, and so on).
Maybe this will help you (or maybe I am the only person on earth who didn't knew that ).

[tiki4]

Mmh, I've got a question, maybe someone of you can help me. As all of you know it has been quite a hassle to get a format like AC3 into an Ogg container. The problem seemed to be in the first place, that the muxing in Graphedit using the direct show filters didn't work out right.
Then someone had the idea to use a muxed AVI with AC3 sound and mux that stuff directly into the Ogg file. That method works but one cannot add additional soundtracks as the AC3 sound is always played.
When I saw this thread with the new command line muxer I had the idea to use the ogmuxer first on the AC3 stream to create a n Ogg file containing only the AC3 soundtrack. Then I used Koepi's program to mux an XviD AVI without sound, the Ogg containing the AC3 stream and an additional 2 channel Ogg Vorbis stream into the final OGM.
Unfortunately the playback has the same problem as before. The AC3 stream is always played and one has to use mmswitch.ax to switch between the AC3 and the Vorbis soundtrack.
When I render the file in Graphedit the output pin of the AC3 stream says something like AVIform.
Does that mean that the problem is in the OggDS implementation and there's no way to cirumvent that problem? I'd be really interested in a solution as MC Vorbis never worked the right way and the channel coupling may take some more month to get implemented.

Any ideas are welcome.

Regards,

tiki4

[Suiryc]

Hi,

Time for a little update .
As the zipped file exceeded the 200K limit (213K ), I first used RAR (with Solid Compression) (see attachement)...
So you will have to unzip and then unrar it (sorry for that guys).

So what are the changes ?
First I added buffering to speed up a little the process (for all the tools).
I then tried to push AVILib to its (in theory) possible limits (i.e. 4GB AVI files), but I failed on that . So I took some extreme measures, and decided to try some routines coming from a well known tool .... VirtualDub (1.4.10).
This push my tools in a real alpha state because as you know VirtualDub is a Windows GUI application, and so behave its tools. But my tools here are commandline ones . So I had to "remove" some functions (those relying on the GUI part of VirtualDub), and thus I cannot guarantee you that all will work like a charm in every situation.
So more than ever : USE AT YOUR OWN RISK!

Now changes for each tools:
- OGMinfo
-> added some more info (granulepos estimation, & timestamp estimation for Vorbis streams)

- OGMuxer
-> removed the use of AVILib
-> added some reading routines coming from VirtualDub 1.4.10
-> added some functions coming from OggMux (the automatisation setup file parsing). In other words you can use your omx file with OGMuxer
-> enabled MP3 files as input
-> added another known input file type : it begins with an 'A', finish with an 'F' and have a '$' in the middle (yes you read well). As for AVI only the video stream is muxed

- OGDemuxer
-> removed the use of AVILib
-> added some writing routines coming from VirtualDub 1.4.10

And the new tool : OGMCutter (well in fact it's only a splitter at the moment).
OGMCutter should be able to cut your ogm file on (video) keyframes, choosing them depending on the split size you gave.
OGMCutter should also be able to "adjust" the comments that are in the original file (Title and Chapters) according to where it cut (see the txt file that come with OGMCutter).


So what is good in all that :
- the use of some routines coming from VirtualDub should break the 2GB limit of the previous version of the tools (I successfully muxed a 2.3GB / 3 hours AVI file into an OGM file)

BUT

- as I said VirtualDub is a GUI and I had to "cut" some parts, so maybe there will be more problems with the latest version of the tools than before
- there is also a bad news : it seems that >2GB OGM files give headaches to Windows (when I select such a file in the Explorer, then the "explorer.exe" process - i.e. the Windows' Desktop - stay forever @ 99% of CPU use and I have to "kill" it , and moreover when I try to read the file with WMP6.4 the video stutter and I cannot seek). Fortunately when I cut the file into 702MB there is no more problem .


Enjoy.

[Edit]Attachment removed. See posts below to find the latest versions of those tools[/Edit]

[Suiryc]

Originally posted by tiki4
The problem seemed to be in the first place, that the muxing in Graphedit using the direct show filters didn't work out right.
...
Does that mean that the problem is in the OggDS implementation and there's no way to cirumvent that problem?

There is the same problem when you mux everything with OGMuxer : the AC3 track is allways played. So I also think it's related to OggDS (but maybe there is no solution for that, who knows).