Algorithm for muxing overhead of muxman?

[honai]

From the behavior and documentation of muxman I understand that the process of muxing video/audio/subtitles is deterministic, i.e. given a set of fixed video/audio/subtitle streams the result will always be the same (unless the underlying engine is changed, that is).

So what exactly is the algorithm to determine the resulting filesizes in the VIDEO_TS folder when the abovementioned video/audio/subtitle stream sizes are know (also N/M, frame count, fps, and so on)? (No 'exotic' features such as multi-angles or menus used here.)

Obviously, I'm asking because I want to refine my bitrate calculator which at the moment just assumes an arbitrary muxing overhead of 2 to 3%.

Any help is appreciated.

[Matthew]

A while back I did some experiments and found that muxing overhead varies according to bitrate distribution, e.g. 2 projects that were identical in every respect except one was CBR and the other VBR, produced different-sized vobs. And if I recall correctly, the difference was not insignificant (as a percentage - the test files were small). Was very irritating.

Unless I'm mistaken, you can take audio out of the equation by using 1.6 percent of the file size for AC3, and 7 percent for MP2. For subs the size of the sub/idx can be used.

But that still leaves the video, and you can't predict bitrate distribution. So you might just have to stick with an arbitrary allocation for video only, but perhaps adjust it according to the length of the video. You should be able to get within 50 MB most of the time, I manage that easy using very crude calcs.

[mpucoder]

Video streams are padded for every VOBU. Packetizing uses 23 bytes per 2048 byte pack. But the padding on average is half a payload every half second (average VOBU duration). This means 2025 bytes of padding on average every second.

[Zeul]

The route i have taken in NuMenu (I think Scenaid is also very similar) is:

# of nav pks = #of frames / (n/m * m) , where n/m and m are the GOP structure
# of asset pks = videosize + audiosize + subsize / 2012 , we believed the first 32 bytes were used for packetisizing

total size = #of nav pks + #of asset pks * 2048

This has always been pretty accurate, but perhaps mpucoder can clarify my procedure also :D

[honai]

Thanks for the suggestions and hints, I appreciate it a lot.

@Zeul,

just to be sure here - does that read:

# of asset pks = (videosize + audiosize + subsize) / 2025

or

# of asset pks = videosize + audiosize + (subsize / 2025) ?

The same here:

total size = (#of nav pks + #of asset pks) * 2048

or

total size = #of nav pks + (#of asset pks * 2048) ?

(I've replaced the 32 byte size with mpucoder's 23 byte.)

[Zeul]

# of asset pks = (videosize + audiosize + subsize) / 2025

total size = (#of nav pks + #of asset pks) * 2048

[honai]

Thanks for the clarification.

I'll post the link to the calculator in the next few days (it'll be a browser/javascript application) so if there are still errors in the algorithm I hope you guys could help me hunting them down.

[honai]

@Zeul

I did some testing using the algorithm you provided. However, the result doesn't seem to be correct.

Here's the algorithm again:


# of nav pks = #of frames / (n/m * m) , where n/m and m are the GOP structure
# of asset pks = (videosize + audiosize + subsize) / 2025
total size = (#of nav pks + #of asset pks) * 2048


Let's assume we use the following assets:


MPV: 3.929.842.266 Bytes
Audio 1 (AC3): 334.172.160 Bytes
Audio 2 (AC3): 334.172.160 Bytes
Subtitles: None
Total filesize of assets: 4.598.365.161 Bytes
Frames: 178.575
Framerate: 25fps
M = 3
N/M = 4


Using Muxman 0.14g:


Total filesize of assets: 4.598.365.161 Bytes (see above)
Total filesize of muxed VOBs: 4.699.643.904 Bytes
Muxing overhead: 101.278.743 Bytes


But the algorithm yields the following:


# of nav pks = #of frames / (n/m * m)
= 178.575 / (4 * 3)
= 14.881,25

# of asset pks = (videosize + audiosize + subsize) / 2025
= (3.929.842.266 + (334.172.160 + 334.172.160) + 0) / 2025
= 2.270.709

total size = (#of nav pks + #of asset pks) * 2048
= (14881,25 + 2270709) * 2048
= 4.680.889.703 Bytes


Difference between calculated and actual total filesize:


4.680.889.703 Bytes
- 4.699.643.904 Bytes
= 18.754.201 Bytes


So the algorithm is roughly 18 MB off the actual DVD filesize(s) as muxed by Muxman 0.14g.

Any suggestions what might be wrong? Perhaps some other parameters are missing in the equations, no?

[mpucoder]

One thing not accounted for is the padding stream, which averages 1012 bytes per VOBU (navpak). So there's another 15MB.

[honai]

Thanks for the correction. So that leaves approx. 3.694.376 Bytes being used somewhere else ...

[Trahald]

i havent seen anything predict perfect results ahead of time for a compile. domain compressers (ie dvdshrink) are a total different animal where the resulting video structure wise is the same as the original.

[mpucoder]

There are a lot of assumptions and small discrepancies in the formulae. First the assumption that all GOP will be the same number of frames when computing the number of VOBU (navpak) - it is possible that the encoder detects scene changes and/or was fed a list of required I-frame locations (for chaptering). The only way to know for sure is to scan the video file or interpret a .vif, and know the algorithm for VOBU composition. In any case, round up, there is no such thing as 0.25 VOBU.

A larger factor is probably the payload size, assumed to be 2025 for all streams. 2025 is the maximum payload, the first video pack of every VOBU is only 2012 bytes. Different audio formats use different payload sizes:
AC3 2021
DTS 2012 (768Kbps) or 2013 (1536Kbps)
LPCM ranges from 1984 to 2010
mpeg 2025

Subpictures are not a continuous stream, so each subpicture is padded and requires an interger number of packs. Again, you can figure on half the payload (1012) as the average padding. But you must know the number of subpictures in the stream.

LPCM payload sizes by number of channels:

16-bit payloads (1-8 channels) 2008, 2008, 2004, 2000, 2000, 1992, 1988, 1984
20-bit payloads (1-6 channels) 2010, 2010, 2010, 2000, 2000, 2010
24-bit payloads (1-5 channels) 2010, 2004, 1998, 1992, 2010

[Zeul]

thanks mpucoder, that clarifies it a bit

[honai]

Thanks for the suggestions and figures, mpucoder. I'll use some approximations then.

Where did you get the figures from?

[mpucoder]

Where did you get the figures from?
MuxMan code.

[honai]

Uhm, well, since I don't have the sourcecode, and assuming that you didn't pull the figures out of nowhere when you wrote the code - any links or reference sources you'd recommend? ;)

[mpucoder]

Most of the information was derived from reverse engineering and experimentation. But for starters the mpeg-2 system specifications give the basics of multiplexing and elementary stream packetization. Combined with the DVD pack size of 2048 this gives you all but the audio stream payloads. My site contains what information that can be shared without divulging the secrets of multiplexing.

[honai]

Thanks, I'll take a look at it.

I've just modified the algorithm, ending up with an error margin of 2.3MB. I think that's tolerable.

A minor question:

Again, you can figure on half the payload (1012) as the average padding. But you must know the number of subpictures in the stream.

What do you mean by the latter sentence? How does the number of subpictures affect the stream/total size?