Exact overhead for Ogg streams?

[Foo]

I just read through the other threads discussing overhead for an Ogg container. However, I still haven't seen any exact calculations for this. Can Tobias or anyone else share the math we need to make a proper determination :D ? Xvid is so accurate with file sizes, it's a shame not knowing the *exact* frame overhead of the final container.

foo

[Actron]

i have about 0,33% overhead....anyone agree with that ?

[Triggerle]

Thanks for the information, but I'm a little confused now.

Is that 0,33% of the movie size, video size or audio size?

And how accurate is this calculation?

I'll start making statistics now, but maybe someone has done that already?

[kastro68]

Movie size, NO Video size...I don't know, but shouldn't really matter

Encode the Audio first and worry about the video size afterwards. Overhead is negligible if you asked me. Besides, 80min cds let you store about 704 megs.

Now, does anyone know how I can separate Audio from Video from an ogm file. Vdub can't read ogm files. How can I do it using Graphedit?

Thx

[debris]

This would also be a nice new feature for Koepi's OggMux:
Demuxing of .ogm streams :)

[FH2]

Originally posted by Triggerle
Thanks for the information, but I'm a little confused now.

Is that 0,33% of the movie size, video size or audio size?

And how accurate is this calculation?

I'll start making statistics now, but maybe someone has done that already?

I've no exact information but at least something for your statistics:

First Movie(83min): XviD + 2 Ogg audio -> Overhead: 3.639.617 bytes

Second Movie(91min): XviD + 1 Ogg audio -> Overhead: 3.483.376 bytes

Third Movie(115min): XviD + 1 Ogg audio -> Overhead: 2.683.691 bytes

[A_Pleite]

Well, thats strange.

min overhead overhead/min
83 3639617 43850,8072289156626506024096385542
91 3483376 38278,8571428571428571428571428571
115 2683691 23336,4434782608695652173913043478

the overhead/min-should not be to different, are you sure about the overheadsize, were you using different fps´s.

A_Pleite

[Triggerle]

Here are my promised observations. Unfortunately, I didn't have the time to do more movies.
Film length Film length Film length Audio size Video size Overhead
bytes time frames bytes bytes bytes

1.470.538.639 02:39:04,083 228831 157.942.237 1.304.539.136 8.057.266
737.842.629 01:57:42,040 176551 122.168.788 613.472.256 2.201.585
1.471.446.848 02:27:01,200 211499 150.664.168 1.312.258.048 8.524.632

Any thoughts?

[Sven Bent]

Originally posted by A_Pleite
Well, thats strange.

min overhead overhead/min
83 3639617 43850,8072289156626506024096385542
91 3483376 38278,8571428571428571428571428571
115 2683691 23336,4434782608695652173913043478

the overhead/min-should not be to different, are you sure about the overheadsize, were you using different fps´s.

A_Pleite

actually the overhead per minut wil ALWAY decrease with increasing size, that is the cause if headers in the .ogm file formet.

the headers are one size no matter how long the video is.

[Koepi]

Sven, your answer is a little too short to cover that up.

In OGG, the headers are inserted per frame, and frames have a fixed size.

So if your movie is 80, 90 or 100 minutes, but always 700 MB, the overhead in MB is always the same.

That's why you get a decreasing "overhead per minute" if you're heading for a certain filesize with "more minutes" movies.

Regards,
Koepi

[A_Pleite]

I don´t get you - maybe I´m somehow stupid, but -
"In OGG, the headers are inserted per frame, and frames have a fixed size." - Are these "frames" just a couple of bytes or real video/audio-frames?

[Koepi]

OGG has it's own "frames".

It's like having pages of ~1500bytes. And these frames have a header.

Regards,
Koepi

[A_Pleite]

So for 2cd´s (1.4GB) we have about
8000 Kbytes
and for 1cd (700MB) we have about
2100 KBytes
if Triggerle is right

is somehow strange - no linear function

A_Pleite

PS: Triggerle sounds bavarian ;)

[Sven Bent]

Originally posted by Koepi
Sven, your answer is a little too short to cover that up.


yes i know but lack of time (had to go to job) and lack of english skill i just wanted to throw it in the debate and let someone with better english skill finish it :-)

but i can se you fixed it up nicely
thanx

[Triggerle]

Originally posted by A_Pleite
PS: Triggerle sounds bavarian ;)

Are you trying to insult me? :angry: (<- j/k)

I'm as far away from a Bavarian as can be (Berlin). :D

As to further our topic...they were all 25fps PAL rips. Unfortunately I don't have time to do more rips right now (work is a bitch ;)), so I posted my table away. It has been sitting unaltered here for weeks.

[Foo]

Originally posted by Koepi
Sven, your answer is a little too short to cover that up.

In OGG, the headers are inserted per frame, and frames have a fixed size.

So if your movie is 80, 90 or 100 minutes, but always 700 MB, the overhead in MB is always the same.

That's why you get a decreasing "overhead per minute" if you're heading for a certain filesize with "more minutes" movies.

Regards,
Koepi

Thanks Koepi. As the person who started this thread forever ago, it's nice to be getting somewhere :D . Can you confirm/clarify any exact figures for the size and number of the "frames" so we can deduce precisely the correct overhead?

700MB auido+video + N (some constant number based on filesize Ogg Media Stream "frames") * the size of these frames = ?

Was the ~1500bytes for the number of frames (like one Ogg frame every 1500bytes) or the size (one ogg frame has a 1500byte overhead)?

The caffeine hasn't kicked in yet, so any light you can shed on this would be much appreciated.

foo

[ingoralfblum]

Ogg saves the data in pages and packets. All overhead is saved in the header of a page and the packets are simply the data part of the pages. So for an exact overhead calculation it is sufficient to look at the pages.

The size of a packet is determined by lacing values, which are stored in the page header. If a frame has a size of e.g. 1000 bytes, the following lacing values are used: 255,255,255,235, because 255+255+255+235=1000. If a page has a size of 255, the lacing values are 255,0, where the 0 indicates, that this is the last lacing value for a page (a value < 255 indicates a page end). So in general for 255 bytes of page data, which can be audio or video, there is one lacing byte. Additionally there's a byte, that specifies the number of lacing values in a page. So overall the lacing has an overhead of around 1/250 of the raw data.

The lacing values are only used to indicate the size of the pages. We still have to discuss the rest of the data in the page, which is:

Page marker "OggS" (4 bytes)
Serial number (4 bytes)
Position, also known as time stamp (8 bytes)
CRC (4 bytes)
Version (1 byte)
Flags (1 byte)

This is 22 additional bytes (I remember, that this is 24, so there might be some things missing. Better look at the specifications) per page. Usually a page contains an integral number of audio/video frames, so the sizes might differ. However currently the size is 4 kB or larger.So if you have video frames with a size of 3 kB each, it is likely, that the pages with video data average around 6 kB. However in that case you have the same amount of header data for more content, so the worst case is the assumption of 4 kB per page. This means you have 22 bytes / 4096 bytes page header overhead, which is ~ 0,54 %. With the lacing values this is around 0,94 % overhead relative to the content data.

A similar calculation, by the way, is already in the format comparison on the MCF project page. But apparently it was too difficult to look there, wasn't it.

Regards,

Ingo

[ChristianHJW]

For your convenience : http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/*checkout*/mcf/doc/container_comparison.html?content-type=text/html

[ingoralfblum]

Originally posted by ChristianHJW
For your convenience : http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/*checkout*/mcf/doc/container_comparison.html?content-type=text/html
There is a little problem with the line endings, probably because the document was converted from .doc to .html with Word. Also the calculations for MCF are not up to date. Perhaps someone wants to fix that.

Regards,

Ingo

[A_Pleite]

It´s hard for me to understand this topic, but I really like to know so

Ogg saves the data in pages and packets. All overhead is saved in the header of a page and the packets are simply the data part of the pages.

Page + Overhaed of this page = packet ?

This is 22 additional bytes

The "convience"-text says it was 26 Bytes ;)
next question:
26 Bytes (see above) + lacing Bytes = Overhead ?

Usually a page contains an integral number of audio/video frames, so the sizes might differ. However currently the size is 4 kB or larger.So if you have video frames with a size of 3 kB each, it is likely, that the pages with video data average around 6 kB.

Whats an integral number (mybe it´s just a translation problem if mine)?
If I understood this right, the data of each packet has at least 4KB. But the packet size grows with the the size of the video-frame. If videoframes are smaller than 4KB, the packet gets be filled with more than one frame, until the pagesize has at least 4KB. Have I understood this right?:confused:

last question: how many audio frames are there per one vid frame in an ogg-stream?

A_Pleite

[ingoralfblum]

Page + Overhaed of this page = packet ?
packets + overhead = page
The "convience"-text says it was 26 Bytes ;)
That's correct.
Whats an integral number
1, 2, 3, 4, ...

If I understood this right, the data of each packet
page
has at least 4KB. But the packet
page
size grows with the the size of the video-frame. If videoframes are smaller than 4KB, the packet
page
gets be filled with more than one frame, until the pagesize has at least 4KB. Have I understood this right?:confused:

Yes, this is the current implementation of the Ogg library. Actually one video frame is one ogg packet (that's just naming). Future implementations might have an average page size of 50 kB or 1 kB or have it dependent on the stream or data type. It is also possible, that a packet (= audio/video frame) spans multiple pages.

last question: how many audio frames are there per one vid frame in an ogg-stream?

That varies, but 512 samples per Vorbis packet (= audio frame), shouldn't be the worst assumption. So one second = 86 audio frames, which means around 4 audio frames per one video frame.

Regards,

Ingo

[A_Pleite]

Thanks for your great post, ingo ;)
With this information I think its possible to do an almost exact overhead claculation.
this tool would have to load the vorbis-stream for checking the size of the audio-frames and the samples per frame.
Then the tool has to load the xvid-firstpass-stats-file and check how big the 2nd-pass-frames are going to be.
With these information it should be possible to calculate an exact overhead.
It even should be possible to implement such a tool in xvid, so it just loads the vorbis-stream and does the 1st and 2nd pass only with the given final ogg-media-size.

Just an idea ;)

A_Pleite

[ChristianHJW]

Originally posted by A_Pleite
With these information it should be possible to calculate an exact overhead.It even should be possible to implement such a tool in xvid, so it just loads the vorbis-stream and does the 1st and 2nd pass only with the given final ogg-media-size. Just an idea ... .... and a good one i guess :) .... anybody from the coders here to comment about feasibility ?

[A_Pleite]

It seems that there is no coder noticing this thread

[Naivete]

From my experience, using Gordian Knot to calculate the overhead for 1x AC3 has given correct file sizes for Ogg streams. I've only tried this with two movies, but I have obtained the size I was aiming at each time. I hope this will help someone.

[PeterTheMaster]

as i understand this thread, muxing overhead depends on final file size not on movie length, right?
so calculate frame-overhead (for avi) should be unchecked and no audio should be selected in gordian knot.
and as desired filesize we should find an appropriate value, like 695 instead of 700mb.
the avi overhead gets lost when the video is repacked into an ogm stream, right?
and i think i read somewhere that ogm overhead is smaller than avi overhead.
wouldnt that mean that the ogm filesize is smaller than the sum of ogg and avi filesizes?
but thats not the case, why? is the avi container packed into the ogm container? wouldnt this be dumb? or is ogm overhead larger than avi overhead?

[FH2]

Originally posted by PeterTheMaster
as i understand this thread, muxing overhead depends on final file size not on movie length, right?
so calculate frame-overhead (for avi) should be unchecked and no audio should be selected in gordian knot.
and as desired filesize we should find an appropriate value, like 695 instead of 700mb.
the avi overhead gets lost when the video is repacked into an ogm stream, right?
and i think i read somewhere that ogm overhead is smaller than avi overhead.
wouldnt that mean that the ogm filesize is smaller than the sum of ogg and avi filesizes?
but thats not the case, why? is the avi container packed into the ogm container? wouldnt this be dumb? or is ogm overhead larger than avi overhead?

My experiences were, that I get less absolute overhead the longer the movie is while keeping the file size constant (approx. 700 MB). Here examples when muxing Video-Only-AVIs with Ogg-Files (1x96 kbit/s):

Movie playtime | Overhead (OGM - AVI ?)
87,04 min | 3,9 MB
90,35 min | 3,7 MB
92,26 min | 3,6 MB
93,00 min | 3,6 MB
94,24 min | 3,5 MB
100,11 min | 3,4 MB
102,27 min | 3,1 MB (audio 80 kbit/s)

So i'm nearly sure, that the avi overhead doesn't get into into the OGM Container since you can propably get negative absolute overhead when the lenght of the movie is very long.

In my mind you may be right, that the OGM overhead isn't influences by the movie lenght, only by the filesize. But the AVI overhead could get bigger when increasing the movie lenght. This would explain the results i've gotten.

The absolute overhead ist mostly positive, because in the AVI there is only one Videostream (at least in the AVIs i used for muxing) and the OGM files contained more streams. If you would mux only one videostream into an OGM it should be smaller. Unfortunately I coundn't check this because OggMux doesn't allow creating OGMs without sound.

edit: I've found out, that OggMux can mux soundless OGMs (even if it says that it can't), und I haven't got the result i expected. Even with the same streams inside the OGM was about 1% bigger then the AVI. But this were only sample files (10 MB), so there may be other results with bigger files. I'll update the results when I have the next movie done.