Some tools for Ogg Media files

[Mosu]

I know what you mean - that's why I also implemented the subtitle support the way I have. The problem with video and granulepos starting with 1 or 0 however does not pose such a great problem - the video will not be off by that much (I know, 40ms at 25fps, but we start noticing delays at approx. 80ms).

[Mosu]

Just an update. After talking with Tobias I'll be sticking to the current granulepos calculation for subtitle streams and start granulepos values for video streams at 0 for the first frame.

[Suiryc]

OK

I updated my tools according to that also now.
I put those here (http://cyrius.bunkus.org).

I made updates on the splitter (as sherpya proposed), so now you can :
- throw away chapters belonging to a previous part
- throw away chapters belonging to a next part
- keep the last chapter of the previous part (unless the first of the current part start at the very beginning)
- keep all chapters or any combination of the 3 above

You can also prevent my tool from changing the title of the clip (i.e. prevent it from adding " - Part #" at the end of this title, like it does for the filename).

I think this should be enough for everybody ;)

[Emp3r0r]

It's good for me ;) Maybe it is time to start making that GUI i've been dreaming about with an autochapter feature using bn.com like the python libarary.

[mrgone]

wow, cool
these come handy
right what i need now

[ookzDVD]

http://membres.lycos.fr/suiryc

hmmm.... I have a problem connection to that site :(
I hope somebody could provide a mirror. thank you.

[Suiryc]

Hi, I changed some little things to see if the problem is related to my pages (I hope I am not that bad at coding html pages ;) ) or to the site itself.

I received propositions for mirroring or even hosting my pages so if problems remain you will have to wait till this is done ;).

Hope other guys are enjoying my little tools :)


PS : oh I forgot to say I released my sources ...

[Suiryc]

Alright now you will find my tools here : http://cyrius.bunkus.org.

This site is up thanks to Mosu (Moritz Bunkus), so we should all thank him at least twice : first for developing ogmtools (on which I based my tools) and then for being kind enough to host my little tools :)

Thanks again Mosu :D

Regards.

[joerg]

I just want to thank you, Suiryc, for your great tools! I was desperately looking for a good ogm splitter, when I found your tools, which worked just great. Before I tried to use OggMux for both muxing and splitting and it silently crashed i.e. just terminated itself) when splitting or created unusable files. Furthermore it doesn't adjust the chapter-info for the two parts. So I couldn't use it for splitting - only for muxing. Then I tried Ogg File Cutter which worked basically OK, but didn't allow me to specify a MB-size for splitting - so to find the right cut point prooved to be quite difficult. AFAIK it also doesn't support chapter info adjustment when splitting.
Then I found your OGMCutter - which is just great! Thanks a lot :)
eclipsedvd: your shell enhancements for the OGMTools won't work for me :( When I select I want to split files a Window opens for a fraction of a second and closes again immediately. I must admit, though, that I replaced the tools from your package with their latest versions before trying the splitting the first time. Could that be a problem? If you have no idea what might be the problem here: can you at least tell me how to remove the non-working extensions from the context menu?

[Suiryc]

Happy to see the tools seem to work :)

Originally posted by joerg
eclipsedvd: your shell enhancements for the OGMTools won't work for me :( When I select I want to split files a Window opens for a fraction of a second and closes again immediately. I must admit, though, that I replaced the tools from your package with their latest versions before trying the splitting the first time. Could that be a problem? If you have no idea what might be the problem here: can you at least tell me how to remove the non-working extensions from the context menu?

:/
Strange I made only internal changes so all should be OK (even if you replaced the orignal version with the latest one).
Just a stupid question here : you tried the splitting on a file larger than 650MB or 700MB (the two splitting options that are in eclipsedvd enhancement) ? Because I know only 3 ways my splitter stop at the beginning :
1. your ogm is corrupt or something like that (I hope it's not the case ;)
2. the splitting size is too small (i.e. my splitter was unable to cut the file because the size of a part would exceed what you wanted - unlikely to happen except if you precised a really realy small split size, or if there are only one key-frame in your clip, or if there is a bug in my tool ;) ) ... well even for that my splitter first analyse the file so the window should have lasted more time
3. the size of the file already exceeds the splitting size you precised
4. oh there is fourth one : my tool is buggy (very unlikely to happen ;) )

If you still want to remove those extensions here is what you can do :
- open regedit
- go to HKEY_CLASSES_ROOT
- find "ogmfile"
- -> Shell
- remove the "Split" (Split@700MB) and "Split 2" (Split@650MB) keys

[TobiasWaldvogel]

[QUOTE]Originally posted by Mosu


My current implementation sets it to 1 for the first video frame, because Xiph's documentation clearly states that a page's granulepos (and therefore the packet's granulepos as well) corresponds to the [B]end position.

The text from the Ogg docu is "The position specified is the total samples encoded after including all packets finished on this page". As the Ogg framing can only take the granulepos from the packets this means that "granulepos" of the packets also must also reflect the total number of sample, i.e. 1 for the video packet.
In case of vorbis the behavior is a little bit different: the first packet has a granulepos of 0 but this still fits in the rules because a single packet can't be decoded. You will need at least two packets.

I will change that in the next release and reflect the new behaviour with a different identifier in the header packet. I thought it in something like "videoV2" to maintain backward compatibility to all streams allready created. Anyway you can easily update them by just demultiplexing and multiplexing them again. (E.g. with my DS-based OggCut example just open them and save them again without any further action).

Concerning the subtitles the unit used for granulepos is in ms. I would suggest to interpret samples as "ms" and therefore to put the end time in ms as granulepos. Currently I'm treating subtitles as "one" sample but now I think it would be more natural to interpret samples as ms.

I.e. a subtitle which starts at 00:00:01.000 and ends at 00:00:02.799 would have granulepos 2s + 799ms + 1 (because we start 0) = 2800 and with length (inside the packet) set to 1800

I will add this in the next version of the filters and give you all the details to adapt you tools accordingly. I'm sorry for any inconvinience this may cause.

Kind regards,
Tobias

[TobiasWaldvogel]

I'm also going to change the architecture of the filters. I currently creating some libraries which will be load dynamically according to the streamtype. The idea behind is to make it easier to adapt other formats like flac and speex and to make it easier to develop new tools (outside DS).

The functions are desgined to be called without knowledge of the stream contents. These librariese will provide the following functions:

For Splitting:
- translate page granulespos to a reference time (100ns units)
- translate packet granulepos to a reference time (100ns units)

For Multiplexing:
- translate reference time to packet granulepos

For Decoding:
- deliver the header packets to the library
- return mediatype
- PacketIn
- BufferOut

For Encoding
- Return the header packets
- return the mediatype
- BuferIn
- PacketOut


I will give some more details in the next days.

Kind regards,
Tobias

[Mosu]

I still agree that your proposition looks better than the current implementation. I'll follow your lead as soon as you've got something working with the new scheme. If you want to change the identifier to 'videoV2' or whatever then by all means do so :)

[Suiryc]

Originally posted by TobiasWaldvogel
I.e. a subtitle which starts at 00:00:01.000 and ends at 00:00:02.799 would have granulepos 2s + 799ms + 1 (because we start 0) = 2800 and with length (inside the packet) set to 1800

(I hope my math skills aren't too bad now :/)
If the subtitle end @ 2s799, there have been 2799 ms (samples) till the beginning (0->1, 1->2, ..., 2798->2799). This mean the granulepos of the last sample is 2798 (ms).
So if we start counting granulepos from 0, then we should set the Packet's granulepos to 2798, and if we start counting granulepos from 1 then it is 2799 (which correspond to the Xiph's documentation : granulepos = total samples encoded and fully decodable).

Maybe you made a shortcut and your "ends at 00:00:02.799" meant "last sample of the subtitle starts @ 00:00:02.799", or maybe I should go back to school and learn some math again (:p).

[TobiasWaldvogel]

Originally posted by Suiryc


(I hope my math skills aren't too bad now :/)
If the subtitle end @ 2s799, there have been 2799 ms (samples) till the beginning (0->1, 1->2, ..., 2798->2799). This mean the granulepos of the last sample is 2798 (ms).

Maybe it was not quite clear. I meant the last time unit (sample) when the subtitle is displayed is 00:00:02.799, which means that 00:00:02.799 is included.

Kind regards,
Tobias

[Suiryc]

Question concerning data included in the stream_header structure.
Here is what I "think" :

- time_unit is the unit time expressed in a 100ns time reference
(100ns is the reference time for Windows' DirectShow)
So the real time unit is time_unit / 10000000
- samples_per_unit is the number of samples per unit of time
So the number of samples per second is :
samples_per_second = samples_per_unit / (time_unit / 10000000)

samples_per_second 10000000
Or : -------------------- = -----------
samples_per_unit time_unit


The applications are :
- for a video stream there is only 1 sample per unit, so samples_per_second = fps = 10000000 / time_unit
- for an audio stream the unit is 1s, so time_unit = 10000000 and samples_per_unit = samples_per_second
- for a text stream, the unit is 1ms, so time_unit = 10000. As one sample lasts 1ms, samples_per_second = 1000 => samples_per_unit = 1

This made me think about this formula (I put it in my sources) :

granulepos = sample number (since the beginning) (if start @ 1)
1000000 : to get microseconds from seconds
10000000 : reference of the time_unit (100 ns)
So samples_per_unit * (10000000 / time_unit) = samples_per_second
And then timestamp (in s) = (granulepos-1) / samples_per_second
Or timestamp (in micros) = 1000000 * (granulepos-1) / samples_per_unit
This give : (sh is the stream_header)

(granulepos-1) * 1000000 * sh.time_unit
timestamp = ----------------------------------------- micros
10000000 * sh.samples_per_unit

which can be simplified by :

(granulepos-1) * sh.time_unit
timestamp = -------------------------------- micros
10 * sh.samples_per_unit


So the question (for 1M bucks ;) ) : is this good (or at least not too bad)?
If yes, does the precedent applications (for video, audio and text) are the unique ones (i.e. we cannot use other values)?

Thanks

[Suiryc]

Originally posted by Suiryc
Maybe you made a shortcut and your "ends at 00:00:02.799" meant "last sample of the subtitle starts @ 00:00:02.799", or maybe I should go back to school and learn some math again (:p).
Originally posted by TobiasWaldvogel
Maybe it was not quite clear. I meant the last time unit (sample) when the subtitle is displayed is 00:00:02.799, which means that 00:00:02.799 is included

:)

[TobiasWaldvogel]

The formula for the current timestamps (from the packets) is just

granulepos * sh.time_unit
timestamp = -------------------------------- micros
10 * sh.samples_per_unit


In future it will be


(granulepos-packet_len) * sh.time_unit
timestamp = -------------------------------- micros
10 * sh.samples_per_unit


where packetlen is in samples


// Returns the length of a packet in media time (not for vorbis packets)
extern ogg_int32_t stream_packet_len(stream_header* sh,
ogg_packet* op)
{
if (op->packet[0] & PACKET_TYPE_HEADER) return E_NOTDATA;

ogg_int16_t lenbytes;

lenbytes = (*op->packet & PACKET_LEN_BITS01)>>6;
lenbytes |= (*op->packet & PACKET_LEN_BITS2) <<1;

if (lenbytes == 0) return sh->default_len;

ogg_int32_t len = 0;
while (lenbytes)
{
(len) <<= 8;
(len) |= op->packet[lenbytes];
lenbytes--;
}
return len;
}


Kind regards,
Tobias

[Koepi]

Uh, nice to see that the development is going on again!

While you're at timestamps/granulepos here, is it possible to get a more keyframe accurate jump/cut behaviour implemented?

My idea was to fetch an avi-parser like from mplayer or from aviplay and allow the oggmuxer DSF to directly open an avi file... is that possible/desireable?

Regards,
Koepi

[Suiryc]

Originally posted by TobiasWaldvogel

(granulepos-packet_len) * sh.time_unit
timestamp = -------------------------------- micros
10 * sh.samples_per_unit


where packetlen is in samples

This is the beginning time of the Packet. (the formula I posted was a general one i.e. I was getting the timestamp of sample at position granulepos, and not directly the beginning timestamp of a Packet).
This is what I presently use - in another way - for sorting Pages before writing them to file.

[Suiryc]

@Koepi & Tobias

Well I saw that some guys are trying to activate the delay feature in OggMux using TimeLine objects.
I have restricted knowledge in DShow, but I think that maybe there is another thing to do : add a delay possibility directly within OggDS (if possible of course).
Why ? Because I once tried to manually insert delay with my muxer (i.e. I "delayed" the granulepos of the data Packets), and the result was not so good :
For small delays this seem to work but I am not sure the real delay in audio is the one I wanted.
For big delays (for example 2s) the clip doesn't start by itself and I have to seek a little in the clip.

So with my restricted knowledge I think that the problem may be (unless I did something wrong of course ;)) :
the audio decoder after OggDS expect data since the very beginning, and if OggDS doesn't feed it (or tell it to wait a certain time) it gets angry.

Maybe Tobias can enlighten us here : do you think that just delaying the granulepos - manually in my muxer or using TimeLine objects in OggMux - is OK ?

[alky]

just a note: you can remove the mux/demux from the context using the filetypes-properties of the windows explorer

[joerg]

Originally posted by Suiryc
Happy to see the tools seem to work :)
Just a stupid question here : you tried the splitting on a file larger than 650MB or 700MB (the two splitting options that are in eclipsedvd enhancement) ?

Yes, it's 1,35 GB in size.


Because I know only 3 ways my splitter stop at the beginning :
1. your ogm is corrupt or something like that (I hope it's not the case ;)

As the splitter worked from the commandline, I'd say that cannot be the reason.

2. the splitting size is too small (i.e. my splitter was unable to cut the file because the size of a part would exceed what you wanted - unlikely to happen except if you precised a really realy small split size, or if there are only one key-frame in your clip, or if there is a bug in my tool ;) ) ... well even for that my splitter first analyse the file so the window should have lasted more time

As I tried to use the "split @ 700MB" - so that's unlikely, isn't it?

3. the size of the file already exceeds the splitting size you precised

The filesize is large enough (see above).

4. oh there is fourth one : my tool is buggy (very unlikely to happen ;) )

Unlikely as it works from the commandline.
So are there more options? btw: I'm using Windows XP SP1 if that matters.

If you still want to remove those extensions here is what you can do :

Thanks for the info. I do not want to remove the extensions, but unless I can get them to work for me, they are of no use for me.
Is there a possibility to keep the Window open so I can read the message, which probably shows up in there?

[joerg]

eclipsedvd and Suiryc: I just found and fixed the problem with the shell extensions. The %1 in the commandline, which gets replaced by the filename was not enclosed in "" - and because my filename had spaces in it, it wouldn't work. Now, after fixing this, everything works fine for me :)

[Bluedan]

Ahem, back to present.
Suiryc, I tested your tool and it worked fluently. Great!
Nevertheless, I have a suggestion: Could you implement a function to just restrict one part to the desired size? Yesterday I had the case that your tool cut my ~1.35GB stream with size preset 700MB into 3 pieces with the last one being 1,3MB. I think in most cases people want to cut for 2CDs, so if one is slightly oversized... most recorders can overburn nowadays. Still it takes some time to process such a stream again.
I admit that it is in addition a personal problem but my DVD-Rom, preferably as "first drive", is unable to read overburned CD, though it's quality fabrique Pioneer DVD-116, while my Lite-on recorder is able to. So in my "personal" case one (normally the first) CD mustfit the size.
I hate to ask for such things like: "hey mastermind, can you put together a personal solution for me, which is totally useless to other people except myself?"
But rethinking, I find that with the very tight and exact file size prediction of todays codecs in use (XVID, DIVX, RM9 what else?) there's little chance that the second part will surpass a plus of 10 MB if cut size preset is set to 700-702MB? What do you think ?

So far there is no tool to join OGM steams, right?

[Mosu]

My Linux tools implement a '-n' switch that tells ogmsplit how many files it should create at most - even if the last one is bigger than the intended size. This is rather trivial to implement so I'd think suiryc will do that. This is a feature that greatly eases the splitting in a lot of cases - simply split at 700 and go for 2 files max. Several people are very happy with this feature, so it won't be useless :)

[Mosu]

Oh btw, Tobias - I hope you don't take too long with your new version. I'm eager to adjust my tools :)

[Suiryc]

Originally posted by Bluedan
Suiryc, I tested your tool and it worked fluently. Great!
:)

Yesterday I had the case that your tool cut my ~1.35GB stream with size preset 700MB into 3 pieces with the last one being 1,3MB.
You could have also tried with 702MB as plit size (using the commandline) ;)
but ...
My Linux tools implement a '-n' switch that tells ogmsplit how many files it should create at most - even if the last one is bigger than the intended size. This is rather trivial to implement so I'd think suiryc will do that. This is a feature that greatly eases the splitting in a lot of cases - simply split at 700 and go for 2 files max. Several people are very happy with this feature, so it won't be useless :)
I will do that too :)

I hate to ask for such things like: "hey mastermind, can you put together a personal solution for me, which is totally useless to other people except myself?"
Mastermind? Who is that guy? ;)
No problem here. You know sometimes you think that what you want is totally useless for other people but it's generally not the case (at least I am sure that other people would like a workaround here).

So far there is no tool to join OGM steams, right?
Nope. (AFAIK)
I could give this a try, but with the options I added in my splitter (concerning chapters) I will have a lot of things to verify before the merged file looks like the original one (or at least is not too far) ;)

[Suiryc]

Originally posted by joerg
eclipsedvd and Suiryc: I just found and fixed the problem with the shell extensions. The %1 in the commandline, which gets replaced by the filename was not enclosed in "" - and because my filename had spaces in it, it wouldn't work. Now, after fixing this, everything works fine for me :)
Allways something we don't think at first :devil:

[MaTTeR]

Seems to work great for splitting and info. Great work!

Any chance of a GUI or did I miss the link somewhere in this thread?

[Suiryc]

No GUI for the moment.
If anyone want to do one, feel free to do so ;)

[Suiryc]

@ those who use my tools
Could you confirm you have absolutely no problem of desynchronized audio/video after muxing or splitting (nor at the beginning of the file nor at the end).
(my tools still needs test before we can say they work fine ;), especially with multichannels audio streams)

[Mosu]

Toabis, just to make sure. Please use 'videoV2', 'audioV2' and 'testV2' as the new streamtypes - or something like that. Don't reuse any existing tag as this will make playing older files that much harder. I'm pretty sure that you'd do that anyway, but if we change it we better change it in a way that won't introduce new problems.

[Suiryc]

The -n feature have been added.
I updated the Cutter and the Muxer regarding a little bug that could have caused problems with multichannels Vorbis streams.

Enjoy.

@Tobias
Does your cutter follow the Vorbis' team advice, i.e. that when cutting a vorbis stream the two last packets of a part should be duplicated as the two first ones in the next part (due to the "overlapping nature of vorbis")?
And do you think that every cutter should strictly follow this path (presently I just cut the stream where I need and do not report the two last packets)?

[alky]

i made a very small gui for the demuxer and cutter using Perl/Tk it should work on every win32 system after installing active perl fom http://www.activestate.com/Products/ActivePerl

you can grab the 2kb zip file here: http://members.chello.at/demmer

[MaTTeR]

@alky

Thanks for the GUI effort. Seems to work just fine with WinXP.

[Bluedan]

Cannot d/l gui from your site ATM !

[alky]

Bluedan: why not?

anyway copied it to a 2nd server...

[Bluedan]

Not sure why. Makes no difference if opera or IE, Tiny Firewall enabled or disabled...
Would you PM me, because it's sunday ??

[Bluedan]

OK. Success. 'Mirror' link does it. First one still don't.

[Bluedan]

Maybe I found a bug so far.
I demand somebody to confirm that:
2nd split ogm (XVid + 2x OggVorbis) with ~700MB (~1.35GB in total) in size has a slight synch problem towards the end. Sound is late for a quarter second, so already noticable. I used builds 0.9a3 for that splitting task.
Will check if it's not decoding related on my Intel 667MHz CPU (probably not...) concerning filter chain.

[Sigmatador]

You can:
-Demux
-Split at 700
-Split at 650
with a right click

http://www.eclipsedvd.firstream.net/FILES/OGMTools_20020912.exe

[Dark-Cracker]

Hi,

just a little comment. i think in the final release of your tool u need merge the muxer and the splitter and add the possibility to mux & split on the fly (exemple : mux the avi and the 2 ogg file and split at the same time after the desired size, not first mux and after split, this will be great for the little harddisk and avoid a lot of tempory file) i hope this feature was not to hard to add.

and a silly question how react your muxer if the audio lenght was not the same than the video lenght ? (with one or 2 audio file) does it stop the video at the same audio lenght ? or does it stretch the audio stream ? or it use the video lenght ?

Thank u, very nice work.
continu comme ca :)

sorry for my bad english.

[Suiryc]

Originally posted by Bluedan
Maybe I found a bug so far.
I demand somebody to confirm that:
2nd split ogm (XVid + 2x OggVorbis) with ~700MB (~1.35GB in total) in size has a slight synch problem towards the end. Sound is late for a quarter second, so already noticable. I used builds 0.9a3 for that splitting task.
Will check if it's not decoding related on my Intel 667MHz CPU (probably not...) concerning filter chain.
Synch problem on one stream or both?
And there are no synch problem at the beginning (in the first minutes)?
Are those (or one of those) streams 5.1? If yes could you also try the 0.9a4 version (maybe this won't change anything but who knows).

[Suiryc]

Originally posted by Dark-Cracker
Hi,

just a little comment. i think in the final release of your tool u need merge the muxer and the splitter and add the possibility to mux & split on the fly (exemple : mux the avi and the 2 ogg file and split at the same time after the desired size, not first mux and after split, this will be great for the little harddisk and avoid a lot of tempory file) i hope this feature was not to hard to add.
Unfortunately this was not so easy to make the splitter.
I have to admit that Ogg Media file structure is great but very hard to use for precise editing (unlikely to AVI there are no kind of index at the beginning of the file telling what are the data in the file and where are those data). Thus for a precise cutting I have to parse a first time the whole file to know what I need about the video stream (framerate, where are the keyframes, an approximation of how many bytes will do the part if I cut on this keyframe, ...).
When I have those information, I can then reparse the file and cut where this is needed (and I am 99.99% sure that the filesizes won't exceed the split size).
This mean that for merging the muxer and the splitter I would need to make something similar, except that I don't have only one file as input but many files, which is of course harder :(
Anyway as you could see I think that the way I made the Splitter is the maximum one can do, and is really too much to implement in the muxer ;)
Another solution would be to pay less attention to the filesize, and just follow the video stream : when I find a keyframe and the filesize is bigger than the split size, then cut, but in this case you could end with filesize bigger (maybe much bigger if there was a keyframe just before and the next one is far away) than the split size.
Then there is the third solution (the one I prefer, but had not enough time for the moment to do so) : mix the 2 precedent solutions, i.e. just parse the video stream and store where are the keyframes, then reparse all input files and cut where I think this is the best (according to where are the keyframes and an estimation of how many bytes would represent to go till the next keyframe).

So there is still work to do ;)

and a silly question how react your muxer if the audio lenght was not the same than the video lenght ? (with one or 2 audio file) does it stop the video at the same audio lenght ? or does it stretch the audio stream ? or it use the video lenght ?
Simple answer : my muxer follow the longest stream.
So if your video is longer, then at the end you have video with no sound, and if the audio is longer it is (should be, I never tested) the contrary.

So I see coming a next request here : "add an option to end other streams when video finish" ... ;)

Thank u, very nice work.
continu comme ca :)
"Merci"

sorry for my bad english.
Not as bad as mine ;)

[Dark-Cracker]

hi,

>Another solution would be to pay less attention to the filesize, and >just follow the video stream : when I find a keyframe and the >filesize is bigger than the split size, then cut, but in this case >you could end with filesize bigger (maybe much bigger if there was a >keyframe just before and the next one is far away) than the split >size.

thank u very much for your quick answer :) for muxing the joiner and the splitter , i was thinking it was hard :) but i think to splipt after the next keyframe once u have past the desired filesize is a good solution i think u could obtain at more 2 mo and if you try to find the desired less 1 or 2 Mo u could find the keyframe just before the desired filesize. but u have a still a lot of work i suppose this feature will be added in the final release :) (i hope)

if u are interested u can perhaps ask to koepi or cyberdemonII some informations because the have already solve this matter with the muxer & joiner.

>So I see coming a next request here : "add an option to end other >streams when video finish" ...

hum u read in my minds :)

good luck for your next release.
Bye.

[DaveEL]

Originally posted by Suiryc

Another solution would be to pay less attention to the filesize, and just follow the video stream : when I find a keyframe and the filesize is bigger than the split size, then cut, but in this case you could end with filesize bigger (maybe much bigger if there was a keyframe just before and the next one is far away) than the split size.


How about

Read frames in to a buffer untill you reach a keyframe, if size of outputfile + buffer size is over split size start a new file and output the buffer to the new file or if its under the split size add frames in the buffer to the current output file. Then start reading frames in to the buffer again.

DaveEL

[Suiryc]

Originally posted by DaveEL
How about

Read frames in to a buffer untill you reach a keyframe, if size of outputfile + buffer size is over split size start a new file and output the buffer to the new file or if its under the split size add frames in the buffer to the current output file. Then start reading frames in to the buffer again.

DaveEL
Easier to say than to do ;)

Here is the whole problem : I want to cut very precisely.
Presently for writing the OGM file here is what I do :
- I read data from each input file (frame per frame)
- I construct a Packet for each frame, specifying what is necessary (position)
- I give the Packet to the Ogg layer, which in return give me (when it has enough data) a Page (that contains 1 or more Packets)

I read data coming from input files as they arrive, and the Ogg Layer returns pages when they are full.
So it is possible that, at one point, I read the (key) frame of the video stream that correspond to time 1'20"000, and then the data coming from a Vorbis stream that correspond to time 1'20"500, ...
The filesize is still good, so I continue. I give the data to the Ogg layer, which returns Pages etc etc...
It is possible that in the buffer I get this (and this is a really simple case here, believe me ;)) :
"
...
Page of Vorbis stream :
- Packet 1 -> time 1'19"980
- ...
- Packet 20 -> time 1'20"500
Page of video stream
- Packet 1 -> time 1'20"000 (keyframe)
Page of Vorbis stream :
- Packet 1 -> time 1'20"520
...
"
Because the Packet coming from Vorbis stream at time 1'20'500 is included in a Page that starts @ 1'19"980 (so before the video frame 1'20"000).

So I continue till the next keyframe. This one is beyond the split size, so it is time to cut. I put the data in the buffer, and the last keyframe was @ 1'20"000. So I should cut just before the Page containing the Packet corresponding to that frame (the one in red).

I thus close the current output file, open the next file, flush the buffer and continue ...
So I will have this in the beginning of the next file :
"
Page of video stream :
- Packet 1 -> time 1'20"000 (reseted to 0'0"000)
Page of Vorbis stream :
- Packet 1 -> time 1'20"520 (reseted to 0'0"520)
"
The problem for me : the first 520ms of the vorbis stream are in the previous file.
Well, but 520ms is not that much, so if it is the cost for having a good cut, why not ... (I am joking here ;))
But I have another problem : in tests I made, if the first data coming from a stream are too far away, then the clip is frozen at the beginning and I have to seek :(

But this was a simple case, because generally what you have is this :
"
Page of video stream
- Packet 1 -> time ... (delta)
- Packet 2 -> time ... (delta)
- Packet 3 -> time ... (delta)
- first part / 2 of Packet
Page of video stream
- second part / 2 of Packet -> time ... (key-frame)
Page of video stream
- Packet 1 -> time ... (delta)
- Packet 2 -> time ... (key)
...
"
And the problems are also here :
1. For knowing all that I have to parse again the Page the Ogg layer gave me (to know what is inside)
2. If I have to cut just before the red frame, how do I do ? (this frame is in the middle of a Page, so I have to "undo" the Page, and take the good packets etc etc ... but hey remember it's a buffer which is already written and I am trying to change a lot of thing here :()
3. There is another thing that I forgot to mention : each Page contains it's position in the stream (page number, and granular position). But what is in my buffer is already set according to the current output file (page number=150, granularpos=... <=> time=1'20"00), and I cannot use it like that for the next output file (I must reset time to 0, and page number too).

And there could be other problems I forgot to mention :(

Pheew. Hey I wasn't kidding when saying that using Ogg Media File for precise editing wasn't that easy ;)

[Bluedan]

@suiryc
ad 1)
Well, frankly I didn't test the second audio, but so far I suppose it's the same because they are both identical in format and length: english and german soundtrack, OggVorbis 2ch stereo.

ad 2)
Yes: synch in the beginning while off-synch at the end. With the audio being late only at the end an artificial 'offset' introduced since the cutting point is unlikely because it's static.
I doubt that the problem is clearly due to your programme. If it inserts some 'space' frames accidently during parsing the whole file (as you described above) then why the first part is synch?
BTW, I tested the stream before cutting and it was totally synch.
There's some chance that I have an issue with my drives/aspi/dma system instead, which is good for you.
-> OT: Did you ever have a DVD-Rom stepping back to PIO-mode suddenly and refusing to be switched back to DMA (MB-BIOS set to DMA of course)??

Sorry that this post will be unsatisfying as I currently don't have the CDs at hand - lend them to a friend, stupid - to copy the parts again to HD.
But I will report soon.
Thanks for your patience.

[Suiryc]

Originally posted by Bluedan
Yes: synch in the beginning while off-synch at the end. With the audio being late only at the end an artificial 'offset' introduced since the cutting point is unlikely because it's static.
I really hope this is not due to my program (because in this case I have no idea what could have caused that, at least for the moment).

I doubt that the problem is clearly due to your programme. If it inserts some 'space' frames accidently during parsing the whole file (as you described above) then why the first part is synch?
Theorically my program won't cause troubles for the first part, but for the next ones it's another thing ;)

BTW, I tested the stream before cutting and it was totally synch.
There's some chance that I have an issue with my drives/aspi/dma system instead, which is good for you.
:)
Also sometimes DirectShow seems to behave strangely.
I have sometimes desynch of about +/-60ms with a clip, but if I stop the clip, and then play it again the desynch returns to +/-1ms :/ (it's my favorite test clip : 1'40", 25fps, set to about 1400kbps if I remember well, 1 AC3 2ch 192kbps track, 1 Ogg 2ch track, 1 subtitle track ; the desynch is on the AC3 track according to DirectShow).
And sometimes when seeking desynch appears and after waiting a little this is resynched :/

-> OT: Did you ever have a DVD-Rom stepping back to PIO-mode suddenly and refusing to be switched back to DMA (MB-BIOS set to DMA of course)??
OT : Nope. But I had (with Win2000) the surprise to see that after installing some drivers Windows though all my devices (Hard-disks, CD-Rewriter, DVD-Rom) were SCSI ones (which made disk access really slower and no possibility to burn CD-Roms :( because of course my devices are not SCSI ).

Sorry that this post will be unsatisfying as I currently don't have the CDs at hand - lend them to a friend, stupid - to copy the parts again to HD.
But I will report soon.
Thanks for your patience.
No problem here.
If the problem remains here is what you can do for me (if possible) :
Use "OGMInfo -v3 -l <log_file> ..." on your clips (the original one and the two parts), and send me (Zipped of course, and if don't exceed 5-6MB) (suiryc AT yahoo DOT com) those parts :
- First and last 1000 lines of log file for each part (the two cut files)
- First and last 1000 lines + 2000 lines around where it was cut of the log file of the orignal clip

To know where it was cut :
- Open the log file of the first cut file
- go to the end
- Search the lasts Page numbers of the video stream (should be like "... 1V Page # ...."
- Open the log file of the original clip
- Go to the same Page numbers of the video stream (should be around the middle of your log file), and keep the 1000 lines before and 1000 lines after where you are (plus the first and last 1000 lines of the file).

This will help me knowing if the desynch is really due to my program or if it doesn't appear to be any desynch in the file (at least in term of structure).

Thanks :)

Originally posted by Dark-Cracker
if u are interested u can perhaps ask to koepi or cyberdemonII some informations because the have already solve this matter with the muxer & joiner.
You are talking of OggMux ?
If yes, then there are few chances this will help me because OggMux rely on DirectShow/OggDS.
So OggMux constructs the graph, say when to start, when to end, retrieve some information on the graph (to know where to cut, ...), etc etc ... , and DirectShow "follow" (it seems sometimes DirectShow is stubborn ;)) its orders.
(Nb : this is more easy to say than to do, you can believe me because I already worked a "little" with DirectShow ;))
On the contrary I have to do the job of DirectShow/OggDS concerning parsing of the input files and muxing the streams, and I surely do it differently (I take data as they arrive from the input files and synchronize streams before writing, while DirectShow synchronize the streams before sending them in the graph).
This make easier for OggMux to have synchronized streams in the output files, while I have to take care of this point myself in my tools.

Anyway thanks for the tip :)