Calculating Ogg Overhead for OGM

[Laz]

Not sure if this is the right forum or not sorry.

Is there any wau to calculate the ogg overhead instead of guessing so one can achieve perfect 700mb rips?

Thanks in advance.

[Koepi]

Nope, there isn't. The overhead depends on i.e. how many ogg pages are needed to store a frame (-> you need to know the exact framesizes),...

Sorry for the bad news.

Regards
Koepi

[Laz]

Ahh ok.

Is there any dependable way of 'guessing'?
like how large would would one expect for a 120min movie?

[Koepi]

for a 120min movie, an ogg vorbis soundtrack (at quality 0) will take up ~1mb extra space.

So if you have 2 soundtracks @50MB and are heading for a 700MB CDR, your movie should be around 599MB after 2nd pass. (That's the approximation i use in OGMCalc)

Regards
Koepi

[Laz]

Ahh I see.
Thanks for that.

[alexnoe]

The percentage of OGG overhead is *at least*

(28/page size + (1/255))*100.

That means, assuming a page size of 4 kb, the overhead will be at least 1%. In reality, it will be slightly higher, because this 1% assumes that each frame or packet has a size of 255*k-1 bytes (k>=1).

Independant from the page size, it will always be more than ((1/255)*100) = 0,39%.

In other words, either OGM does not have anything in common with OGG, or 1 MB is hilarious....

[Koepi]

Wow, the first time i see this formula. I'll experiment with it - maybe the "good old" approximation (which prooved itself to be quite in range) is superflous with it. I'll put that into OGMCalc and will see how the results are.

Regards
Koepi

[Koepi]

Ok, done some testing with that. Good thing is, I could tweak the "average pagesize approximation" based upon some samples i encoded around here, and the results are now nearly the same as before.

But i only tested the standard scenario (1cd 700mb), maybe this gives better results for higher CD count. I'll soon upload a new version of OGMCalc.

Bottom line, the approximation i gave above isn't worse as the guessing of an average pagesize.
Also the 1% overhead is wrong. I took the desired size (700mb), multiplied with 0.996 (0.39% min overhead), et voila, i had back my 697MB which i was assuming before for 2 audio streams. (Looking at OGMCalc source i implemented "1mb overhead per _stream_ for an average 1cd 700mb encode" which is more accurate.)

Regards
Koepi

[HarryM]

I use GKnot for calculating OGM size.

1xAC3 (1 frame), without 'Calculate Frame-Overhead'

for large about 90-100 min.

For files shorter than 80 min I decrease cca 1-2 MB extra for videostream, for files larger than 110 min I increase cca 1-2 MB extra for videostream.

[alexnoe]

@koepi:

Did you take into account that, when muxing ogg/vorbis into ogm, the overhead of the ogg/vorbis input files is of course striped away?

I just took those formula from the OGG framing specification on xiph, which I needed to implement into avi-mux gui to be able to read ogg/vorbis files...

If there were any specifications for OGM, i could easily implement retrieving the real overhead of an OGM file...

[Koepi]

Well, Tobias "documented" the paket format as he uses it here:

http://tobias.everwicked.com/

For the "rest", the OggDS sources are available, as well as Moritz Bunkus (http://www.bunkus.org/videotools/ogmtools/) and Cyrius'es (http://virtualdubmod.sourceforge.net/)implementation.

IMO it's not possible to accurately calculate the real overhead _before_ - you have to mux the files and then you can parse it and count headers.

I agree that the lack of documentation is a little disturbing.

Regards
Koepi

[alexnoe]

OK:

Mux your source files to MKV, using AVI-Mux GUI. It will then display, on byte accuracy, the size of video, audio and overhead. In the case that the source files themselves contained overhead (like AVI, OGG/Vorbis, MKV), it will of course stripe everything away, so that you see the real overhead.

Then, mux the same to OGM, and you can retrieve the real overhead on byte accuracy.

[Koepi]

Ah, ok, now i get your point: yes, of course the source avi and ogg vorbis files include container overhead. So from the bitstream-point of view you're right.

I thought we were still talking about calculating the "desired size" of i.e. an xvid encode, together with some audio streams.

My fault, I was thinking "dumb" by just taking the source files, multiplexing them together and see how much the final file grew (that's what I / we "need" OGMCalc for - to calculate how big the video part [here usually still in avi] should be).

I apologize, of course alexnoe is right, overhead _is_ over 1%.

Regards
Koepi

[alexnoe]

EDIT changed some b0rked values

I thought we were still talking about calculating the "desired size" of i.e. an xvid encode, together with some audio streams.So I guess the XVID encoder aimes, when indicating the desired size, for AVI, i.e. takes the AVI overhead into account, probably for old AVIs? Well, for a video-only AVI, or a video-only MKV, overhead prediction is pretty simple.

For a video only OGM with known pages sizes, a good guess should also be easy: with a framesize between k*255 and (k+1)*255-1 bytes, you need k+1 bytes for framing, additionally to 28 bytes per OGG page.

First lets put aside the OGG page header, but begin at byte 28, where the packet sizes of the current frame are stored.
Lets assume you want to put 120mins into 600 MB. This is a byterate of 83 kB/s, or about 3 kBytes per frame (roughly rounded). Assuming that every video frame is about 3 kb in average, you need one byte per [11/12*255,..,255] bytes. In other words: the guess for the OGM framing overhead is, if you take the middle, usually off by not more than 4%. Larger frames cause less deviation, smaller frames cause more deviation.

According to Cyrius, the page size is hardcoded in libogg (I never looked at it, so I just believe him...), so that you can assume about 28 bytes per 4kB as given. Those 28 bytes per 4kB are more than the half of the produced overhead, meaning that the part which is off by around 4% causes less than half of the overhead. If those are, lets say 1/2, the total overhead for a given bitstream size should be predictable by +/- 2%.
For MKV, +/-0.1% is possible for a video-only file, and for AVI, it should be even less deviation (though it is more complex to calculate this for MKV than for AVI). I just define +/- 0.1% error in the overhead estimation as 'exact'.

If the XVID codec tries to hit the indicated size for an AVI file, you could retrieve the final bitstream size accurately, and with it, the overhead of the final OGM file by +/- 2%.

With an overhead being off by less than 2%, and an overhead of roughly 1% over the entire file, the final size including overhead should be predictable from the bitstream size, with an error of 0.02%, or 140 kB for a 700 MB file. Even if the overhead were 2% (which it isn't), it would be not more than 300 kB
If you don't reframe the OGG/Vorbis input file (i.e. you keep the page size, which is most likely to happen anyway), the audio will not cause additional overhead.

Now someone has to put this into some nice equations and make a calculator from it.

BTW, OGM is the only container I know which causes overhead by size, and not by frame.

And, so far, I only made an estimation for the *maximum* error in the overhead estimation.

One more aspect: A nonlast segment of a packet must be 255 bytes, i.e. if a new page is iniciated only when 4096 bytes are reached at least, the average real page size will be 4224 bytes (between 4096 and 4351, and assuming equal distribution of packet sizes), so that you have 28 bytes per 4224, and not 28 bytes per 4096 bytes page header overhead.