How compute m2ts result files size ? [Archive]

Selur

21st January 2011, 08:45

Video is 147084 frames at 23.976fps encoded at 5923.32kb.
Audio is AC3 at 448kb. Total overhead is 278286018 bytes.
Audio is AC3 at 192kb. Total overhead is 290556610 bytes.
Video is 119098 frames at 23.976fps encoded at 5921.61kb.
Audio is AC3 at 448kb. Total overhead is 225322715 bytes.
Audio is AC3 at 192kb. Total overhaed is 235254491 bytes.

In general when posting numbers it would be helpful to see the following infos:
1. raw video size (byte)
2. video frame count
3. raw audio size (byte)
4. average audio bitrate (bit/s)
5. length of video (assuming audio&video length is the same), video frame rate
6. size of video only muxed inside m2ts container (byte)
7. size of audio only muxed inside m2ts container (byte)
8. size of video&audio muxed together inside m2ts container (byte)
this would allow to use them as control values for later tests. :)

Actualy, if bitrate audio b is in kilobit (1000 bits), for AC3 audio, one part of the calcul of audio overhead is :
(2000*(C-b)/256)*time, time is duration is seconds. C would be 640 will not surprise me !
Actualy, for my specific cases, with my video bitrate : (2000*(640-audiobitrate)/256)*time + 1830*frames gives a good overhead estimation. If some are interesting to search using this as a base search...
Here's why I'm reluctant to use this as a base for a search:
1. since the package size of m2ts is 192 it would kind of make sense to see it appear somewhere in the formula
2. some value that is assumed as average package fill would make sense
3. calculation video overhead by only looking at the frame count is definitly a dead end even if video bitrate is 1/5 of the current value, video overhead stays the same,.. ;)

video only overhead example:
video frame count: 109645 frames
video frame rate: 23.976 frames/second
video length: frame count/frame rate = 4573.115 seconds
raw video size: 8.472.672 Bytes
video inside m2ts: 37.502.976 Bytes
-> video overhead: 29.030.304 Bytes

(2000*(640-audiobitrate)/256)*time + 1830*frames would be way of:
(2000*(640-0)/256)*4573.115 + 1830*109645 = 5000*4573.115 + 1830*109645 = 22865575 + 1830*109645 = 22.865.575 + 200.650.350 = 223.515.925
I know you are probably aware that the video overhead formula wasn't good. I just wanted to show an extreme example,..

The main point is that a formula to calculate the overhead should not contain constants that fall from the heavens. ;)

Sadly I lost nearly all my muxing data collection a year ago in a hdd crash, only thing I found was some data about a single video-only source I reencoded with different bitrates:
m2ts - videoOnly

Framecount: 109645
Length: 109645/23,976 = 4573,1147814481147814481147814481

quant 51
37.502.976 Bytes inside container
8.472.672 Bytes raw
29.030.304 Bytes overhead

200
143.388.672 Bytes inside container
111.974.627 Bytes raw
31.414.045 Bytes overhead

300
200.534.016 Bytes inside container
166.799.049 Bytes raw
33.734.967 Bytes overhead

400
257.857.536 Bytes inside container
221.618.987 Bytes raw
36.238.549 Bytes overhead

500
315.033.600 Bytes inside container
276.435.819 Bytes raw
38.597.781 Bytes overhead

1000
601.055.232 Bytes inside container
550.501.276 Bytes raw
50.553.956 Bytes overhead

2000
1.173.006.336 Bytes inside container
1.098.623.746 Bytes raw
74.382.590 Bytes overhead

A side from the lag of suspected variables, a main problem with the audio overhead formula (2000*(C-b)/256)*time for me is that using C = 640 would result in no or negative overheads if the audio bitrate is 640+kbit/s. ;)

Cu Selur

Ps.: aiming for a formula that only works if the target size is i.e. 10GB, DVD9, DVD5 or something like that is a thing that would probably be easier, but will probably not result in a all around working formula

jpsdr

21st January 2011, 10:36

As i said, it's for one AC3 audio track, so, in AC3 case, bitrate is not more than 640kb i think, and my actual formula is for now only with the video bitrate i've used.
I'll add later the others files size informations.
I'm currently doing a 10000kb video to try to have video track at different bitrate, to identify video parameter.
I'll also do some tests with 640kb AC3 audio track.
I'm not interested in audio or video only, because they may have different way of doing things, and Scenarist will probably not allow me a track without video stream.

jpsdr

21st January 2011, 18:16

Video1 : 147084 frames @23.976fps raw h264 encoded (x264) at 5923.32kb. .264 file size : 4542170174 bytes.
Video2 : 119098 frames @23.976fps raw h264 encoded (x264) at 5921.61kb. .264 file size : 3676859941 bytes.
Audio 1.1 (for video 1) : AC3 encoded at 448kb. AC3 file size : 343538944 bytes.
Audio 1.2 (for video 1) : AC3 encoded at 192kb. AC3 file size : 147230976 bytes.
Audio 2.1 (for video 2) : AC3 encoded at 448kb. AC3 file size : 278170368 bytes.
Audio 2.2 (for video 2) : AC3 encoded at 192kb. AC3 file size : 119215872 bytes.

M2TS files :
(1) : Video1 + audio1.1 => 5163995136 bytes.
(1.1) : Video1 + 2xaudio1.1 => 5532076032 bytes.
(2) : Video2 + audio2.1 => 4180353024 bytes.
(2.1) : Video2 + 2xaudio2.1 => 4478392320 bytes.
(3) : Video1 + audio1.2 => 4979957760 bytes.
(3.1) : Video1 + 2xaudio1.2 => 5163995136 bytes.
(4) : Video2 + audio 2.2 => 4031330304 bytes.
(4.1) : Video2 + 2xaudio2.2 => 4180353024 bytes.

jpsdr

22nd January 2011, 09:31

Video3 : Video1 encoded at 9971,98kb. .264 file size : 7646802893 bytes.
Video4 : Video2 encoded at 9998,64kb. .264 file size : 6208381775 bytes.
Audio 1.3 (for video1) : AC3 encoded at 640kb. AC3 file size : 490769920 bytes.
Audio 2.3 (for video2) : AC3 encoded at 640kb. AC3 file size : 397386240 bytes.

M2TS files :
(5) : Video3 + audio 1.1 => 8403603456 bytes.
(6) : Video4 + audio 2.1 => 6821910528 bytes.
(7) : Video1 + audio 1.3 => 5348038656 bytes.
(8) : Video2 + audio 2.3 => 4329375744 bytes.

Now, i think there is globaly enough relevant data to create a good estimation of video/audio overhead in case of AC3 audio file.

jpsdr

22nd January 2011, 17:42

For now, i think, for video + audio :
Video overhead is rawvideosize/23 (or rawvideosize/23 + constant).
For at least 448kb and 192kb, audio overhead is : "something"-rawaudiosize/16.
Still not found what this "something" is for audio.
For 192kb and 448kb, a working formula is (1000*(2134-bitrate)/128)*time, but it's only considering if the "something" is directly linked to the time.

jpsdr

22nd January 2011, 20:38

According the data, i have for now the following formulas :
Video overhead = rawvideosize/23
Audio :
AC3 192kb & 448kb : K=696*frames (or K=29*fps*frames, i don't know, only tested with 23.976fps video)
Audio overhead = K-rawaudiosize/16. I don't know for which bitrate value formula switch.
AC3 640kb : Audio overhead=800*frames.

Selur

22nd January 2011, 20:58

may be also worth a try:
qint64 CalculationService::m2tsVideoOverhead(const qint64 completeVideo, const int frameCount, \
const bool audioHandling)
{
qint64 videoOverhead = 0;
qint64 videoSize = completeVideo; //videoOverhead+generalOverhead+rawVideo
if (!audioHandling) {
videoOverhead = frameCount*263 + 9412;
}
videoSize -= videoOverhead;
int packageSize = 192;
int packageFill = 185; //of 192
sendMessage(MYDEBUG, "packageFill", packageFill);
qint64 packageCount = videoSize/packageFill;
qint64 packageOverhead = packageCount*(packageSize-packageFill);
videoOverhead += packageOverhead;
return videoOverhead;
}

qint64 CalculationService::m2tsAudioOverhead(const bool videohandling, const int frameCount, const double framerate, const QString format, const qint64 rawsize)
{
int blocksize = 80;
if (format == "ac3" || format.contains("true") || format == "dts"
|| format == "dts-hd" || format == "pcm") {
blocksize = 907;
}
double packageSize = 192.0;
double packageFill = 182.0;
qint64 audioOverhead = qint64((2*blocksize)/packageFill * 8.0 + 17.0 +0.5) \
* ((qint64)(rawsize*1.0/(blocksize*1.0)+0.5));
if (videohandling) {
double length = frameCount/framerate;
double audiobitrate = rawsize/length;
audioOverhead += frameCount*(packagesize+(audiobitrate/packageFill+1)) + 9412;
}
return audioOverhead;
}

jpsdr

23rd January 2011, 08:28

I've allready try your formulas you give in the post #3, error is too big for me... Has i said, your formulas give around 11MB over a stream with subtitles ! Wich indeed result of around 30MB too much for a stream (subtitles was around 20MB), resulting in 360MB error for 12 streams (my need). Way too much out of the league for me, i'm looking for error <1MB (by stream) !

Selur

23rd January 2011, 08:40

You did notice it's a modified version in the last post didn't you?

jpsdr

23rd January 2011, 08:46

My humble mistake !!
I'll try these asap.
I nevertheless still need with my data, to check (unless you've already done), how compute audio overhead with 2 audio tracks.

How much is "generaloverhead" for video, knowing only "rawvideosize" ?

Selur

23rd January 2011, 09:34

How much is "generaloverhead" for video, knowing only "rawvideosize" ?
The assumtion is that the total overhead is a combination of:
rawVideoSize + videoOverhead + rawAudioSize + audioOverhead + generealOverhead
where generalOverhead is some sort of Overhead that occures when audio,video and other stuff are combined in a source.
-> so for videoOnly there would be no generalOverhead ;)
I deliberately only mentioned it in the comments, so ignore it (assume generealOverhead = 0) :)

Here's how to use the formulars in general:
1. decide if you got audio&video or video- or audio-only
2. assuming you have audio&video
3. you start with the m2tsAudioOverhead calculation
the result of this is than taken to calculate targetVideo - audioOverhead - audioRawSize, which is called completeVideo in the m2tsVideoOverhead call. :)
then you calculate your m2tsVideoOverhead
This would later result in an assumed target size of:
rawVideoSize (here we take the result we really got as rawVideo) + calculated videoOverhead + rawAudioSize + calculated AudioOverhead

As a side note:
I doubt it is even possible to produce a 1MB accurate size prediction when reencoding, since most encoders (x264 too) are not that accurate (1MB on DVD5 size (4479MB) isn't really much) and you would have to predict how much the encoder would stray from your target size. So I would be really happy if we end up with a formula that will at maximum deliver a prediction that will cause the result to be 5MB smaller than the size wanted. (and I'm not even sure if 5MB is to high of a goal)

Cu Selur

jpsdr

23rd January 2011, 10:06

Size prediction accuracy is one thing yes, formula accuracy is another.
My need is for several streams on one Blu-Ray, so the little program i've made is using a basic dichotomy algorithm to find the bitrate.

jpsdr

23rd January 2011, 11:01

@Selur
I've compared with my little program.
Stream (1) :
Your formulas : m2ts = 5168064090 / Total overhead = 282349047
Mine : m2ts = 5164100177 / Total overhead = 278385134
Reality : m2ts = 5163995136 / Total overhead = 278286018
Mine is less than 1MB higher, yours is around 4MB higher.
Stream (3) :
Your formulas : m2ts = 4924849338 / Total overhead = 235442604
Mine : m2ts = 4980061137 / Total overhead = 290654403
Reality : m2ts = 4979957760 / Total overhead = 290556610
Mine is less than 1MB higher, yours is around 55MB lower.
Stream (7)
Your formulas : m2ts = 5350475154 / Total overhead = 317528880
Mine : m2ts = 5348099365 / Total overhead = 315153091
Reality : m2ts = 5348038656 / Total overhead = 315098562
Mine is less than 1MB higher, yours is around 2MB higher.
Case of stream (3) is totaly off...

My formulas may be a little less generic, more specific case, but more accurate apparently.
I think, nevertheless, my formulas are probably not usable for other audio than AC3.

jpsdr

23rd January 2011, 11:40

For 2 audio streams, i found audio overhead is 1008*frames-totalsize/16. Works for 2x192 and 2x448.
Will try to produce more data to check if 448/192 is the same than 192/448, and check with 640...

jpsdr

23rd January 2011, 14:10

Note : Files is the same with either audio 192/448 or 448/192.
M2TS files :
(9) : Video1 + audio 1.1 + audio 1.2 => 5348038656 bytes.
(10) : Video2 + audio 2.1 + audio 2.2 => 4329375744 bytes.
(11) : Video1 + audio 1.3 + audio 1.2 => 5532076032 bytes.
(12) : Video2 + audio 2.3 + audio 2.2 => 4478392320 bytes.
(13) : Video1 + audio 1.3 + audio 1.1 => 5716113408 bytes.
(14) : Video2 + audio 2.3 + audio 2.1 => 4627415040 bytes.
(15) : Video1 + 2*audio 1.3 => 5900150784 bytes.
(16) : Video2 + 2*audio 2.3 => 4776431616 bytes.

According these data, for AC3 video files, audio overhead is, using my formulas for video overhead :
1 audio track :
AC3 with bitrate 192/448 : 696*frames-rawaudiosize/16
AC3 with bitrate 640 : 800*frames.
2 audio tracks :
AC3 track with bitrate 192/448 : 504*frames-rawaudiosize/16
AC3 track with bitrate 640 : 608*frames.
I think formula 192/448 works for any ac3 bitrate <=448.
I don't know for 512/576 what formula is to use.

Sharc

23rd January 2011, 15:07

If i have the following :
A raw h264 video file of size V1.
2 audio files of sizes A1 and A2.
Does someone know the formula wich will give me, with an accuracy of around 1MB for files of around 5GB, the size of the resulting m2ts muxed file ? (Blu-Ray authoring). :confused:
If others parameters like bitrate/frame are to be used, notify me.

Thanks.
Maybe you want to check out this (http://jvance.com/pages/BitrateCalc.xhtml).

jpsdr

23rd January 2011, 15:34

It doesn't fit my needs, i can have several m2ts streams in final (at least until 12).

Selur

23rd January 2011, 23:32

Mine is less than 1MB higher, yours is around 55MB lower.
Yup, I see the problem. :) 192is an 'evil' bitrate, since the package fill is probaby way lower than anticipated with my formula.

Groucho2004

23rd January 2011, 23:56

Maybe you want to check out this (http://jvance.com/pages/BitrateCalc.xhtml).

Haven't tried it but a quick look at the source code reveals this function:

private static int GetM2tsAudioOverhead(AudioCodec audioType, int samplingRate, double length)
{
// TODO: ??
return 0;
}

:D :D :D

jpsdr

24th January 2011, 09:39

Yup, I see the problem. :) 192is an 'evil' bitrate, since the package fill is probaby way lower than anticipated with my formula.
I'll try later to check if 96 still match my formula. If it did, it means error with your formula my increase as bitrate decrease.

jpsdr

24th January 2011, 21:26

Audio 1.4 (for video1) : AC3 encoded at 96kb. AC3 file size : 73615488 bytes.
Audio 2.4 (for video2) : AC3 encoded at 96kb. AC3 file size : 59607936 bytes.
Audio 1.5 (for video1) : AC3 encoded at 128kb. AC3 file size : 98153984 bytes.
Audio 2.5 (for video2) : AC3 encoded at 128kb. AC3 file size : 79477248 bytes.

M2TS files :
(17) : video1 + audio 1.4 => 4906340352
(18) : video2 + audio 2.4 => 3971721216
(19) : video1 + audio 1.5 => 4906340352
(20) : video2 + audio 2.5 => 3971721216
(21) : video1 + 2xaudio 1.4 => 5016760320
(22) : video2 + 2xaudio 2.4 => 4061134848
(23) : video1 + 2xaudio 1.5 => 5016760320
(24) : video2 + 2xaudio 2.5 => 4061134848
(25) : video1 + audio 1.4 + audio 1.5 => 5016760320
(26) : video2 + audio 2.4 + audio 2.5 => 4061134848

Apparently, overhead audio size for 96kb and 128kb have another formula...

Selur

25th January 2011, 11:33

did some testing and atm it seem like:
complete overhead = X*audioOverhead + videoOverhead - (frameCount * 2*packageSize/3)
where packageSize is 192
X = number of audiostreams
audioOverhead = overhead caused when muxing a raw audio stream on it's own
videoOverhead = overhead caused when muxing a raw video stream on it's own

jpsdr

25th January 2011, 12:58

You mean :
videoOverhead = frameCount*263 + 9412;
audioOverhead = qint64((2*blocksize)/packageFill * 8.0 + 17.0 +0.5)*((qint64)(rawsize*1.0/(blocksize*1.0)+0.5));
For on their own ?

Selur

25th January 2011, 13:10

video-only overhead is probably something like:
double packageSize = 192.0;
double packageFill = 186.0;
double packageFactor = packageSize+packageFill/4+packageFill/8;
double frameOverhead = frameCount*packageFactor;
qint64 videoOverhead = frameOverhead + (size - frameOverhead)*8/packageFill;

hadn't had any time to test it and audioOverhead neeeds a rework too. ;)

jpsdr

25th January 2011, 13:34

More trouble with 96kb and 128kb... It seems that audio overhead, with one track, for these is something like K-rawaudiosize.
Actualy, i have K of 1133*frames for 1 audio track, and 942*frames for 2 audio tracks.

jpsdr

25th January 2011, 14:01

According my datas, you should have, when there is audio (d=delta/difference) : d(videooverhead)=d(videosize)/23, or at least, a N/M factor very very near to 1/23...

Selur

25th January 2011, 15:18

a N/M factor very very near to 1/23...
yup, at least for large files,..
As far as I can tell you got two part building the videoOverhead:
1st part being the frameOverhead (which only relies on the frame number and the average fill of the packages and explains the huge overhead on low bitrate files with high frame count)
2nd part depends on the rawSize of the video and the average fill of the package and easily overtakes the frameOverhead when for larger rawFileSize

jpsdr

25th January 2011, 18:13

As i said, i've get the 1/23 ratio making differences, so, even on small file, the 1/23 should appear somewhere.
For exemple : (1) and (5) have exact same audio.
h : global header = m2ts-videosize-audiosize
h1=278286018, h5=413261619, h5-h1=134975601, video3-video1=3104632719, 3104632719/134975601=23.0014
Idem for (2) and (6).
h2=225322715, h6=335358385, h6-h1=110035670, video4-video2=2531521834, 2531521834/110035670=23.0064
These results make me think (but maybe wrongly) that, in the formula of the video header, there is something very very near to size/23.
At least, doing this, all offsets or things linked only to number of frames are removed. Stay only bitrate/size related data, as in both exemples for m2ts [1,5] or [2,6] :
Time is the same, frames are the same, audio is exactly the same, only bitrate/videosize change.

Selur

25th January 2011, 18:22

here are some numbers I just collected (encoding elephants dream):

raw Video 45496754 bytes 15910 frames
raw Audio: 35562240 bytes
muxedAV: 89972736 bytes ->totalOverhead: 8913742
muxedAudio: 40642560 bytes ->audioOverhead: 5080320
muxedVideo: 51357696 bytes ->videoOverhead: 5860942

raw Video 131747982 bytes 15910 frames
raw Audio: 35562240 bytes
muxedAV: 179976192 bytes ->totalOverhead:12665970
muxedVideo: 141367296 bytes ->videoOverhead: 9619314
muxedAudio: 40642560 bytes ->audioOverhead: 5080320

raw Video 61194876 bytes 15910 frames
raw Audio: 35564032 bytes
muxedAV: 106352640 bytes ->totalOverhead: 9593732
muxedVideo: 67737600 bytes ->videoOverhead: 6542724
muxedAudio: 40648704 bytes ->audioOverhead: 5084672

raw Video 86650038 bytes 15910 frames
raw Audio: 10161152 bytes
muxedAV: 106248192 bytes ->totalOverhead: 9437002
muxedVideo: 94310400 bytes ->videoOverhead: 7660362
muxedAudio: 13977600 bytes ->audioOverhead: 3816448

raw Video 16232616 bytes 15910 frames
raw Audio: 10161152 bytes
muxedAV: 32741376 bytes ->totalOverhead: 6347608
muxedVideo: 20797440 bytes ->videoOverhead: 4564824
muxedAudio: 13977600 bytes ->audioOverhead: 3816448

muxed2*AV: 44169216 bytes ->totalOverhead: 7614296
muxed2*A: 25405440 bytes ->audioOverhead: 5083136
videoOverhead: 4564824

total overhead = X*audioOverhead + videoOverhead - (frameCount * 2*packageSize/3)
where packageSize is 192
X = number of audiostreams
audioOverhead = overhead caused when muxing a raw audio stream on it's own
videoOverhead = overhead caused when muxing a raw video stream on it's own
seems to bit okay, 'totalOverhead - x*audiOverhead - videoOverhead' is always around: 2033664
and frameCount * 2*packageSize/3 = 15910 *2 * 192/3 = 2036480

jpsdr

25th January 2011, 19:27

Now, just have to figure out audioOverhead and VideoOverhead... :).
I'll soon make test with AC3+THD.

Selur

25th January 2011, 19:30

especially the audioOverhead is a mystery ;)
(having in mind, that m2ts allows at least: ac3, true-hd, dts, dts-hd, pcm and mpeg audio with bitrates from 96kbit/s to multiple MBit/s, when looking over to BDs)

for 'higher' datarates:
audioOverhead = (rawSize/packageFill + length + rawSize/length/blocksize)*packageSize - rawSize
seems to be fairly good,...

jpsdr

25th January 2011, 20:59

Maybe there is realy as i use actualy : Different formulas for different cases. Only for AC3, and only some bitrate, i'm using 3 different formulas...
New datas :
video5 : 63803 frames @23.976fps encoded at 23703kb : 7847751856 bytes.
video6 : 57925 frames @23.976fps encoded at 23703kb : 7148686802 bytes.
audio5.1 (for video5) : AC3 core encoded @640kb : 212894720 bytes.
audio5.2 (for video5) : THD track : 776107718 bytes.
audio6.1 (for video6) : AC3 core encoded @640kb : 193280000 bytes.
audio6.2 (for video6) : THD track : 759722044 bytes.

M2TS files size :
video5 + audio5.1&audio5.2 : 9686513664 bytes.
video6 + audio6.1&audio6.2 : 8838414336 bytes.

Selur

25th January 2011, 21:03

I'm still aiming for the one formula to rule them all ;)

jpsdr

25th January 2011, 21:17

Yes, but if muxing rules differs with different audio parameters, formula unfortunately, can't be only one...
But, unfortunately, we don't know the rules...

Selur

25th January 2011, 22:29

.. there definitely have to be some strange rules, looking at these numbers:

length = 636,4s

640
raw Audio: 50805760
m2ts Audio: 59701248
audioOverhead: 8895488

576
raw Audio: 45725184
m2ts Audio: 52076544
audioOverhead: 6351360

512
raw Audio: 40644608
m2ts Audio: 48267264
audioOverhead: 7622656

384
raw Audio: 30483456
m2ts Audio: 36839424
audioOverhead: 6355968

320
raw Audio: 25402880
m2ts Audio: 33024000
audioOverhead: 7621120

256
raw Audio: 20322304
m2ts Audio: 25405440
audioOverhead: 5083136

224
raw Audio: 17782016
m2ts Audio: 21596160
audioOverhead: 3814144

192
raw Audio: 15241728
m2ts Audio: 21596160
audioOverhead: 6354432

160
raw Audio: 12701440
m2ts Audio: 17786880
audioOverhead: 5085440

128
raw Audio: 10161152
m2ts Audio: 13977600
audioOverhead: 3816448

112
raw Audio: 8891008
m2ts Audio: 13977600
audioOverhead: 5086592

96
raw Audio: 7620864
m2ts Audio: 13977600
audioOverhead: 6356736

80
raw Audio: 6350720
m2ts Audio: 10162176
audioOverhead: 3811456

64
raw Audio: 5080576
m2ts Audio: 10162176
audioOverhead: 5081600

56
raw Audio: 4445504
m2ts Audio: 10162176
audioOverhead: 5716672

48
raw Audio: 3810432
m2ts Audio: 10162176
audioOverhead: 6351744

40
raw Audio: 3175360
m2ts Audio: 6352896
audioOverhead: 3177536

32
raw Audio: 2540288
m2ts Audio: 6352896
audioOverhead: 3812608

Cu Selur

jpsdr

26th January 2011, 09:37

And it's only for AC3...
Edit : What is the number of frames ? Or the framerate of the audio ?
When authoring Blu-Ray, you have to specify on the audio what framerate it has.

Selur

26th January 2011, 19:52

Looking at m2tsVideoSize, rawAudio, length I think I found a decently working formula for the audio overhead. :)
double packageSize = 192.0;
//ac3FrameSize = (averageBitrate in Byte/s)*4
int ac3FrameSize = int(rawSize/length+0.5)*4;
//number of packages needed per ac3Frame
int packagesPerAc3Frame = ac3FrameSize/packageSize+0.5;
//number of ac3 Frames needed for the audio
qint64 ac3FrameCount = rawSize/ac3FrameSize +0.5;
//NO CLUE ;) just fits
int unknownMult = 21;
//m2ts packages needed for the stream
qint64 packageCount = ac3FrameCount*packagesPerAc3Frame + length*unknownMult;
//m2ts audioOverhead
qint64 audioOverhead = packageCount*packageSize - rawSize;

+ audioOverhead -= frameCount * 2.0 * packageSize / 3.0; //if video muxing with video;

Aside from the 'unknownMult' I'm quite happy with the formula, since it seems kind of logic. :)

Cu Selur

Ps.: just wondering: is anyone besides jpsdr and me reading this thread?

PPs.: if someone owns a dts encoder it would be nice if he could encode the elephantsdream audiotrack with multiple bitrate and a. upload these files somewhere or b. mux each of these files using tsMuxer and post raw audio size, muxed audio size and length of the files.

Zathor

26th January 2011, 21:14

Ps.: just wondering: is anyone besides jpsdr and me reading this thread?
Yes, I read it with interest. As discussed in the PMs I am looking forward to replace the calculation in MeGUI with this one. I have already added the first calculation of this thread and replaced the totaly strange old calculation of MeGUI. Once again thank you very much!

jpsdr

27th January 2011, 00:04

More THD datas.

Video5 : 63804 frames @23.976fps raw h264 encoded (x264) at around 23703kb. .264 file size : 7847751856 bytes.
Video6 : 57925 frames @23.976fps raw h264 encoded (x264) at around 23703kb. .264 file size : 7148686802 bytes.
Audio 5.0 (for video 5) : THD. File size : 776107718 bytes.
Audio 5.1 (for video 5) : AC3 encoded at 640kb. AC3 file size : 212894720 bytes.
Audio 5.2 (for video 5) : AC3 encoded at 448kb. AC3 file size : 149026304 bytes.
Audio 5.3 (for video 5) : AC3 encoded at 256kb. AC3 file size : 85157888 bytes.
Audio 6.0 (for video 6) : THD. File size : 759722044 bytes.
Audio 6.1 (for video 6) : AC3 encoded at 640kb. AC3 file size : 193280000 bytes.
Audio 6.2 (for video 6) : AC3 encoded at 448kb. AC3 file size : 135296000 bytes.
Audio 6.3 (for video 6) : AC3 encoded at 256kb. AC3 file size : 77312000 bytes.

M2TS files :
(27) : Video5 + audio5.0&5.1 => 9686513664 bytes.
(28) : Video5 + audio5.0&5.2 => 9606678528 bytes.
(29) : Video5 + audio5.0&5.3 => 9542811648 bytes.
(30) : Video6 + audio6.0&6.1 => 8838414336 bytes.
(31) : Video6 + audio6.0&6.2 => 8765933568 bytes.
(32) : Video6 + audio6.0&6.3 => 8707952640 bytes.
(33) : Video5 + audio5.0&5.1 + audio5.1 => 9926019072 bytes.
(34) : Video5 + audio5.0&5.1 + audio5.2 => 9846183936 bytes.
(35) : Video6 + audio6.0&6.1 + audio6.1 => 9055856640 bytes.
(36) : Video6 + audio6.0&6.1 + audio6.2 => 8983375872 bytes.
(37) : Video5 + audio5.0&5.2 + audio5.2 => 9766348800 bytes.
(38) : Video6 + audio6.0&6.2 + audio6.2 => 8910895104 bytes.

@Selur : I'll try your new formula.
When you said :
+ audioOverhead -= frameCount * 2.0 * packageSize / 3.0
"+ audioOverhead -"... do you mean, that, after your 1rst calcul, you must do :
audioOverhead -= frameCount * 2.0 * packageSize / 3.0

Selur

27th January 2011, 06:18

More THD datas.
Nice ! :)

do you mean, that, after your 1rst calcul, you must do :
This should come after the audio overhead of all the audiostreams is calculated.
So first calculate and sum up all the audio overheads, then lower them by 'frameCount * 2.0 * packageSize / 3.0'.

Cu Selur

drmpeg

27th January 2011, 10:31

The m2ts overhead for AC3 audio can be calculated like so:

audio packet size (in bytes) = bitrate (in bps) / 250.

The PES overhead is always 17 bytes.

So for 640 kbps AC3, the PES packet is 2577 bytes.

Each Transport Stream packet can hold 184 bytes. 2577 / 184 = 14.00543478

But you can't have fractional TS packets, so it takes 15 packets to send the audio frame (with the last packet having 181 stuffing bytes).

192 * 15 = 2880

So the m2ts overhead for 640 kbps AC3 audio is 2880 / 2560.

Ron

jpsdr

27th January 2011, 13:04

@drmpeg
Thanks for this information.
Is what you've explained for m2ts with audio+video ?
Is it always the same, even with more than one audio track (only AC3 tracks for now).

I like it, it explains the identical final result 96/128 kb , as 96 give around 2.2 and 128 give around 2.8, but both finaly result in 3x192, so padding is more important for 96 than 128.

drmpeg

27th January 2011, 13:17

@drmpeg
Thanks for this information.
Is what you've explained for m2ts with audio+video ?
Is it always the same, even with more than one audio track (only AC3 tracks for now).
Yes, the video stream doesn't have any effect on the audio stream(s) and vice versa. Also, the number of audio tracks does not have an effect.

Ron

jpsdr

27th January 2011, 14:00

Wonderfull, we are near something.
Using your formulas to substract total audio from (1), (2), (5) and (6) m2ts files i've created, i've the following exact result :
Total video for (1) = 24978738*192
Total video for (5) = 41851698*192
Total video for (2) = 20220382*192
Total video for (6) = 33978494*192
So, formula for total video may have a similar rule...

Don't know exactly the right english word, but bigger common divider for [(1) and (2)] or [(5) and (6)] : Different videos with bitrate mostly identicall (=> in theory would result in same package size using something similar to audio) is only 2.

nixo

27th January 2011, 16:52

Very cool!

I guess I'll never encode another 192kbps AC3 file for BD since 224kbps files will mux to the exact same size. :-)

The question is how to deal with different muxers; it's my understanding that output from TSMuxer is not identical in size to that of Scenarist or some other professional muxer.

--
Nikolaj

jpsdr

27th January 2011, 17:16

Warning : All my results are from Scenarist.

jpsdr

27th January 2011, 19:17

If video follow the same kind of rule than audio, but with bitrate wich is the "local" bitrate, it will never be possible to get the exact size of the total video...

Considering 2 is bigger common divider. My computations shows video header is around size/23.
Like audio, video is something like M0/N ratio, both intergers. M0=M*192.
So... (192*M)/N=1+1/23 => 23*192*M=24*N => 184*M=N.
184, 192, i dont know you, but i love these numbers.... If we chose M=2 (bigger common divider), we have N=368.=> Video would be 192/184... But, it doesn't match experimental datas.... Too bad...
......... Or.... there is something else...
Total m2ts = Audio according drmpeg data + 192/184videosize + TBD
TBD can be frame related, or length or... If i adjust result from post #55 considering video is 192/184.
It leave for :
(1) : 293030*192
(2) : 237447*192
(5) : 292986*192
(6) : 237288*192
.... Very very near to frames*2 !!!
I think, m2ts is :
Audio : acording drmpeg
Video : trunc((videosize/184)+1)*192
m2ts = audio+video+frames*2*192.
At least, for AC3 audio.
And one thing seems for sure : 192 & 184 rules !

jpsdr

27th January 2011, 20:44

Others results, for checking :
(39) : Video5 + audio5.2 => 8373024768 bytes.
(40) : Video6 + audio6.2 => 7626620928 bytes.

jpsdr

28th January 2011, 01:43

My formula for video is not correct, don't work with high bitrate...

jpsdr

28th January 2011, 09:22

Madshi, Neuron2, any x264 developper !!!!
If you read this, maybe you can help us.
Note : If for video it's tricky, assume video is constant bitrate...
:thanks:

jpsdr

28th January 2011, 10:16

Still some little something. If i take datas from post #14, results (1) and (3), exact same video, different audios.
For (1), i found muxed video is 24978738 packet (of 192), for (3) i found 24978745.
Others results, using data from post #14 also give me 24978745 a second time, and 24978756 another time...
Difference is short, but even an error of 1, make impossible to try to correlate datas trying to find things like bigger common divider...

drmpeg

28th January 2011, 12:53

The m2ts overhead for video cannot be precisely calculated. However, you can calculate a worst case.

PES overhead is 14 bytes for B-Frames and 19 bytes for I and P-frames.

The worst case stuffing for the last packet of a PES packet is 183 bytes.

For an IBBP video GOP, the equation would be:

max bytes total = ((number of frames in video elementary stream * (15.666 + 183)) + (bytes in video elementary stream)) * 192 / 184

A good estimate can be made by assuming that the last packet stuffing averages out to 183/2 bytes.

estimated bytes total = ((number of frames in video elementary stream * (15.666 + (183/2))) + (bytes in video elementary stream)) * 192 / 184

Ron

jpsdr

28th January 2011, 13:11

Thanks. I'm happy to find out that somehow, my 192/184 wasn't wrong in a way. But your formula is for the total size, not only the overhead, no ?
15.666 is a true constant, or related to framerate ?
If you have any others informations about others streams, like others audio format or subtitles (for wich input data would be the .pes file created by scenarist), we gladly take them.
Again,:thanks:

drmpeg

28th January 2011, 13:16

Thanks. I'm happy to find out that somehow, my 192/184 wasn't wrong in a way. But your formula is for the total size, not only the overhead, no ?
The 15.666 is a true constant, or is it related to framerate also ?
If you have any others informations about others streams, like others audio format or subtitles (for wich input data would be the .pes file created by scenarist), we gladly take them.
Again,:thanks:
Yes, the total size. I'll edit the post to show that. 15.666 is the PES overhead for an IBBP GOP. (19 + 14 + 14) / 3.

Ron

jpsdr

28th January 2011, 13:47

Thanks allready, your help have been realy very greatfull.
So, do you have by any chance informations about others streams ????:rolleyes:

jpsdr

28th January 2011, 15:31

Do you have any idea, with these encoding parameters, what will my structure be ?

x264_x64.exe --profile "high" --preset "placebo" --tune %TUNING% --pass 2 --bitrate %2 --stats %STAT_FILE% --level "4.1" --qpmin 0 --vbv-maxrate %MAX_BR% --vbv-bufsize %BUF_BR% --keyint 24 --min-keyint 2 --mvrange 511 --ref 4 --bframe 3 --slices 4 --b-pyramid "strict" --open-gop "bluray" --fade-compensate 0.8 --aq-mode 2 --aud --nal-hrd "vbr" --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --videoformat "ntsc" --sar 1:1 --threads 0 --thread-input --output %E_DST% %E_SRC% 2> %LOG_FILE_2%

As i've put bframes to 3, i though it would be IBBBP...
So more to 15.25 ?

Selur

28th January 2011, 17:18

I'm still confused about the audio overhead calculation.
Here's how I interpreted the post from drmpeg:

> The m2ts overhead for AC3 audio can be calculated like so:
HAR, HAR,.. here we go :)

> audio packet size (in bytes) = bitrate (in bps) / 250.
int averageBitrate = int(rawSize/length+0,5)*8; //got length and rawSize as input i.e. 640000
int audioPackageSizePerSecond = int(averageBitrate/250+0.5); //+0.5 to round up; i.e. 2560

>The PES overhead is always 17 bytes.
> So for 640 kbps AC3, the PES packet is 2577 bytes.
int effectiveAudioPackagePerSecond = audioPackageSizePerSecond+17; //+17 due to PES overhead //i.e. int(640000/250+0.5)+ 17 = 2560 +17 = 2577

> Each Transport Stream packet can hold 184 bytes.
double packageFill = 184.0;

> 2577 / 184 = 14.00543478
> But you can't have fractional TS packets, so it takes 15 packets to send the audio frame (with the last packet having 181 stuffing bytes).
int containerPackagesCountPerAudioPackagePerSecond = effectiveAudioPackagePerSecond/containerPackageFill+0.5; //+0.5 to round up; i.e. 2577/184 +1 = 14,005434782608695652173913043478 -> 15

> 192 * 15 = 2880
int containerPackageSizePerAudioPackagePerSecond = containerPackageSize * containerPackagesCountPerAudioPackagePerSecond; //192*15 = 2880

So the m2ts overhead for 640 kbps AC3 audio is 2880 / 2560.
and here I'm uncertain, what this really means.
I assumed, that '2880 / 2560' is a factor to multiply the raw size to get the effective m2ts output size and when subtracting the raw size I should end up with the audio overhead, which would give me:
qint64 audioOverhead = rawSize * containerPackageSizePerAudioPackagePerSecond / audioPackageSizePerSecond - rawSize;

looking at this source (http://www.multiupload.com/46MDJLGCC0):

length: 636.4s
raw audio: 3175360
m2ts audio: 6352896
audioOverhead: 3177536
(which you get if you take the audio stream of elephants dream, downmix it to stereo with sox, reencode it with acid aiming for 32bit/s ac3 and multiplex it with tsMuxeR)

I get:
averageBitrate = int(3175360/636.4+0.5) * 8 = 4990 *8 = 39920
audioPackageSizePerSecond = int(39920/250+0.5) = 160
effectiveAudioPackagePerSecond = 160 + 17 = 177
containerPackagesCountPerAudioPackagePerSecond = int(177/184+0.5) = 1;
containerPackageSizePerAudioPackagePerSecond = 192*1 = 192;
audioOverhead = 3175360*192/160 - 3175360 = 3810432 - 3175360 = 635072
since this isn't anywhere near the audio overhead the stream actually has, I suspect I made a mistake somewhere and since I can't see the mistake I thought I ask you guys. :)

-----------------------------

As i've put bframes to 3, i though it would be IBBBP...
So more to 15.25 ?
for mkv video overhead I normally assume:
ulong nbIframes = ulong(frameCount/framerate/2+0.5);
ulong nbBframes = 0;
if (bframes) {
nbBframes = ulong(((frameCount - nbIframes)/2)+0.5);
}
ulong nbPframes = frameCount - nbIframes - nbBframes; which at least for mkv works kind of good. :)

Cu Selur

jpsdr

28th January 2011, 18:33

I don't know, audio seems to work fine for me with AC3 stream...
If you check all the AC3 file size i provided with their bitrate, calculate the packet size : bitrate/250, and divide their raw size by this packet, you've always an integer result.

Selur

28th January 2011, 18:35

and you use the same calculations I do?

jpsdr

28th January 2011, 18:42

Yes, but for bitrate i use the value i've encoded with, not computed from size.
For your file : 40000/250 = 160 (encoded at 40kb according post in previous page).
3175360/160=19846
(160+17)/184=0.9 => 1 => Total audio size = 3810432.
Now, maybe m2ts audio only have something else within them...
I've been able to check and verify it using the datas i've provided, calculating differences and all, it's the formula for audio. You have 13242 packets (of 192) of something else.
For B frame, i think "typical" pattern could be tell according the encoding parameters i use.

Selur

28th January 2011, 18:53

Yes, but for bitrate i use the value i've encoded with, not computed from size.
okay, sad thing is that doesn't change the result. ;)

Now, maybe m2ts audio only have something else within them...
seems like it (hoping drmpeg can shed some light on this)

You have 13242 packets (of 192) of something else.
since I use tsMuxeR and tsMuxer adds them I probably have to look for a solution myself :(

jpsdr

28th January 2011, 19:09

I just remember that i'm doing encoding for now....
According x264 logs, i've (with a 2s gop on 23.976fps video) around 2.6% of I frames, 28.2% of P frames and 69.2% of I frames.
Wich give me around 15.54.

Selur

28th January 2011, 19:12

yup, problem is normally you want to calculate the m2ts file size to adjust your bit rate so you hit a specific target or stay below a specific size, so you need to find a good approximation before you have the x264 stats. ;)

seems like adding an additional int(length*(20+(184-17)/192)+0.5)*192 works fine for tsMuxer, so they add two additional packages per second

jpsdr

3rd February 2011, 17:56

Yeah.... Doom9 is back !!!
I've found something aroud 1.435*frames*192 added.

Selur

12th February 2011, 14:55

Did any one find out how to compute the audio overhead for dts(-hd) audio inside m2ts?

the following doesn't seem too far off:
int DTSFrameCount = length*1000/10.67+1; //rounded down and length in seconds
int frameSize = 2013;
int packagesPerFrame = frameSize/packageFill+1; //rounded down
int packageCount = DTSFrameCount*packagesPerFrame;
int averageBitrate = rawSize/length+1;
double mult = 1;
if (averageBitrate > 192000) {
mult = averageBitrate/194000.0;
}
packageCount = packageCount*mult;
int tsMuxerAddition = int(length+1)*(20+167/192);
qint64 audioOverhead = (packageCount + packageCount/184.0/8.0 + tsMuxerAddition)*192 - rawSize; //rawSize in byte

Cu Selur

jpsdr

13th February 2011, 16:53

And PackageFill is ???

Personnaly, still found no formula for THD+AC3...

Selur

13th February 2011, 17:07

oh, forgot, packageFill is 184
Got no THD+ac3 data,... so I never thought about it :)

jpsdr

14th February 2011, 09:40

Euh..... And what about what i've posted here (http://forum.doom9.org/showpost.php?p=1474499&postcount=50) ?

I have some DTS-HD Master audio data, i'll try to see if i can correlate datas with yours, but i don't think it will be before the end of this month.

Selur

14th February 2011, 09:49

You posted only the raw sizes
Audio 5.0 (for video 5) : THD. File size : 776107718 bytes.
Audio 6.0 (for video 6) : THD. File size : 759722044 bytes.
and the size muxed with videos of different length. (Video5 : 63804 frames @23.976fps and Video6 : 57925 frames @23.976fps)

But to work on a formula for the audio overhead I need:
1. raw size (got that)
2. length of the audio (don't have that,.. normally audio equals video length,... but not in the cases you provided)
3. the size of a m2ts file with only the audio in it (don't have that)
4. data with different audio length and different bitrates,...

Cu Selur

jpsdr

14th February 2011, 13:48

You can't on Scenarist mux stream without video, and, of course, you can't also mux THD stream without an AC3 core, both are not Blu-Ray compliant.
Length is defined by the associated video, so, here length of audio is video. Why do you think it's not the case here ?
If there is different muxes with different stream, it's to be able to work by differences.
Doing :
[video 5 + audio5.1 + (audio5.0+audio5.1)] - [video 5 + audio 5.1] resulted in 7672704 packet of 192 byte (so size 7672704*192) for (audio5.0+audio5.1).
[video 6 + audio6.1 + (audio6.0+audio6.1)] - [video 6 + audio 6.1] resulted in 7066432 packet of 192 byte (so size 7066432*192) for (audio6.0+audio6.1).
THD is a lossless compression pattern by Dolby. It means there is no bitrate, at least, in the way like "encode with bitrate X".

Selur

14th February 2011, 13:59

You can't on Scenarist mux stream without video,...
Since I'm not using Scenarist I'm not bound to this restriction. :)

Length is defined by the associated video, so, here length of audio is video. Why do you think it's not the case here ?
I hope Scenarist then prevents you from muxing audio&video is their length doesn't match up. (otherwise predicting the overhead is probably more complicated)

THD is a lossless compression pattern by Dolby. It means there is no bitrate, at least, in the way like "encode with bitrate X".
Yes, but the THD information does account for content that need to be muxed in the container and as such will cause a container overhead.

Cu Selur

jpsdr

14th February 2011, 14:31

Since I'm not using Scenarist I'm not bound to this restriction. :)

As all my data are Blu-Ray compliant, and don't see the point to try to get data from non compliant Blu-Ray things, unless you're not trying to do only Blu-Ray compliant structure.
In that case, it's different.

I hope Scenarist then prevents you from muxing audio&video is their length doesn't match up. (otherwise predicting the overhead is probably more complicated)

If Blu-Ray standard allow you to do that, it will not prevent it. What it prevents is different framerates. You can't mux a 23.976fps audio with a 24fps video. For information, when you import an audiofile, you have to define his framerate.

Yes, but the THD information does account for content that need to be muxed in the container and as such will cause a container overhead.
I don't realy understand what you mean by that. My THD files come from a Blu-Ray i've bought, extracted with eac3to.

I've personnaly try to figure out things, but i've not been able to figure out any formula pattern.
Now, i've provided datas of 2 differents lenght files, and i've unfortunately nothing more to provide. You're absolutely free to try, if you want, to figure out things. The hard part is that i've no idea how the steam is seen. As one big file ? As 2 files ?

Selur

14th February 2011, 17:11

1. I really don't care about the Blu-ray Standard, when trying to find a formula to calculate the overhead caused by muxing something into a m2ts container.
2. Sorry, but the data sets you provided don't help me. :)

For information, when you import an audiofile, you have to define his framerate.
Okay, and here you lost me audio doesn't have a frame rate it has a length, it might be synchronized against a video with a specific frame rate but that doesn't change the audio.

My THD files come from a Blu-Ray i've bought, extracted with eac3to.
good, and if you would multiplex the audio with tsMuxeR and post
a. the raw file size (byte)
b. the muxed file size (byte)
c. the length of the clip (seconds)
I would have some data I could use. :)

Cu Selur

jpsdr

14th February 2011, 18:08

Okay, and here you lost me audio doesn't have a frame rate it has a length, it might be synchronized against a video with a specific frame rate but that doesn't change the audio.

I know... But it works like this.

good, and if you would multiplex the audio with tsMuxeR and post
a. the raw file size (byte)
b. the muxed file size (byte)
c. the length of the clip (seconds)
I would have some data I could use. :)

Can you with tsMuxer said that you want to create a THD with AC3 core audio track, and so give to it the THD and AC3 files ?

Selur

14th February 2011, 18:13

Can you with tsMuxer said that you want to create a THD with AC3 core audio track, and so give to it the THD and AC3 files ?
Sorry, but I can't make head or tails of this sentence. :)
with tsMuxeR you can extract/mux a 'THD with AC3 core'-audio track from/into a m2ts container.

Cu Selur

jpsdr

15th February 2011, 09:17

Ok. But i've just remember that i've deleted the files, so, i must re-rip my Blu-Ray... :(
I'll do this when i'll go time, either this week or next.

jpsdr

16th February 2011, 19:24

with tsMuxeR you can extract/mux a 'THD with AC3 core'-audio track from/into a m2ts container.

How do you do that ?
I've TSmuxerGui 1.10.6, and i can only open AC3
files with the ADD button, and my THD track is not recognized even if y choose "*.*" files to open it...

Selur

16th February 2011, 19:35

if I give it a .ac3 extension it worked last time I tried. :)

jpsdr

17th February 2011, 10:12

I must specify one thing : I've 2 files : the ac3 and the "pure" thd (file1.ac3 and file2.thd) extracted with eac3to. So, you said that if i rename the .thd in .ac3 it will work ? I've some doubts, but i'll try this evening when back home...
Problem is that in TSMuxuerGui, i've not found any "THD with AC3 core" option, so...

Selur

17th February 2011, 10:14

Nope, no clue how to handle files where THD and core are separated. When I use tsMuxer for extraction, THD&ac3 is one file.

Cu Selur

jpsdr

17th February 2011, 11:32

Well, in that case, it's a dead end, and the only thing avaible are the data i've already provided.
You posted only the raw sizes
Audio 5.0 (for video 5) : THD. File size : 776107718 bytes.
Audio 6.0 (for video 6) : THD. File size : 759722044 bytes.
and the size muxed with videos of different length. (Video5 : 63804 frames @23.976fps and Video6 : 57925 frames @23.976fps)

But to work on a formula for the audio overhead I need:
1. raw size (got that)
2. length of the audio (don't have that,.. normally audio equals video length,... but not in the cases you provided)
3. the size of a m2ts file with only the audio in it (don't have that)
4. data with different audio length and different bitrates,...
Cu Selur

1. : You have the size of both the THD and AC3 files used to create the "THD with AC3 core" track (and for the 2 results). A big AC3+THD file name .ac3 is not relevant, as it's somehow something already partialy muxed.
2. : You have, audio length IS video length in what i've provided.
3. : in post #81 (http://forum.doom9.org/showpost.php?p=1478068&postcount=81) you have the result size of the mux. It's the result of the AC3 and THD (THD with AC3 core).
4. The 2 files have different lenght.

From my point of view, there is all the data.

I will nevertheless, out of curiosity, try to extract with tsmuxer to see what size is the .ac3 result file containing both AC3 and THD.
It may not be a great help, because, for exemple for video5, either file in size is (audio5.0+audio5.1), data we already have, either it's different, in that case it means that it's partialy muxed.
Note : Maybe there is a 2 stage mux. First, muxing of THD+AC3, second muxing this result in the m2ts. I don't know.

Selur

17th February 2011, 14:11

merely for amusement:

looking at:
video5 : 63803 frames @23.976fps encoded at 23703kb : 7847751856 bytes.
video6 : 57925 frames @23.976fps encoded at 23703kb : 7148686802 bytes.
Audio 5.0 (for video 5) : THD. File size : 776107718 bytes.
Audio 5.1 (for video 5) : AC3 encoded at 640kb. AC3 file size : 212894720 bytes.
Audio 5.2 (for video 5) : AC3 encoded at 448kb. AC3 file size : 149026304 bytes.
Audio 5.3 (for video 5) : AC3 encoded at 256kb. AC3 file size : 85157888 bytes.
Audio 6.0 (for video 6) : THD. File size : 759722044 bytes.
Audio 6.1 (for video 6) : AC3 encoded at 640kb. AC3 file size : 193280000 bytes.
Audio 6.2 (for video 6) : AC3 encoded at 448kb. AC3 file size : 135296000 bytes.
Audio 6.3 (for video 6) : AC3 encoded at 256kb. AC3 file size : 77312000 bytes.
[video 5 + audio5.1 + (audio5.0+audio5.1)] - [video 5 + audio 5.1] resulted in 7672704 packet of 192 byte (so size 7672704*192) for (audio5.0+audio5.1).
[video 6 + audio6.1 + (audio6.0+audio6.1)] - [video 6 + audio 6.1] resulted in 7066432 packet of 192 byte (so size 7066432*192) for (audio6.0+audio6.1). I end up with:

audio5.0+audio5.1:
length = 63803/23.976 = 2661.12 (s)
raw size = 776107718 + 212894720 = 989002438 (byte)
muxed size = 7672704*192 = 1473159168 (byte)
-> overhead = 1473159168 - 989002438 = 484156730 (O1)

audio6.0+audio6.1:
length = 57925/23.976 = 2415.958 (s)
raw size = 759722044 + 193280000 = 953002044 (byte)
muxed size = 7066432*192 = 1356754944 (byte)
-> overhead = 1356754944 - 953002044 = 403752900; (O2)

side note: woooha! 40%+ overhead,.. disc manufacturer must really love this container!
Don't want to know what would happen to the overhead if the length would increase and the average bitrate would shrink,...
+ hope there's no typo somewhere

1st trying the old formula (ment for .ac3 only) to get a perspective how bad the THD stuff is,...
double length = frameCount/frameRate;
int packageSize = 192;
qint64 audioOverhead = 0;
double packageFill = 184.0;
int averageBitrate = int(rawSize/length+1)*8;
int audioPackageSizePerSecond = averageBitrate/250+1;
int pesOverhead = 17;
int effectiveAudioPackagePerSecond = audioPackageSizePerSecond+pesOverhead; //+17 due to PES overhead
int containerPackagesCountPerAudioPackagePerSecond = effectiveAudioPackagePerSecond/packageFill+1;
int containerPackageSizePerAudioPackagePerSecond = packageSize * containerPackagesCountPerAudioPackagePerSecond;
audioOverhead = rawSize * containerPackageSizePerAudioPackagePerSecond / audioPackageSizePerSecond - rawSize;

looking at audio5.0+audio5.1
int averageBitrate = int(989002438/2661.12+1)*8 = 371649 *8 = 2973192 (bit/s)
int audioPackageSizePerSecond = 2973192/250+1 = 11893 (packages/s)
int pesOverhead = 17;
int effectiveAudioPackagePerSecond = 11893+17 = 11910 (packages/s)
int containerPackagesCountPerAudioPackagePerSecond = 11910/184+1 = 65
int containerPackageSizePerAudioPackagePerSecond = 192 * 65 = 12480
audioOverhead = 989002438 * (12480 / 11893) - 989002438 = 48813961 byte (C1)
compared to O1 (484156730) C1 (48813961) is a bit small (being 1/10 of the actual overhead -> THD really is evil overhead wise)

stupid approach: :D
484156730/48813961 = 9.9184069491922607960456230954091;
averageBitrate / 640 000 = 2973192/640000 = 4.6456125
magicMult = 9.9184069491922607960456230954091/4.6456125 = 2,14
audioOverhead = (989002438 * (12480 / 11893) - 989002438)*2,14*4,6456125 = 485289394 (C1s)
C1s (485289394) compared to O1 (484156730) is now okay (meaning: < 1MB derivation and calculated overhead > actualoverhead)

looking at audio6.0+audio6.1:
int averageBitrate = int(953002044/2415.958+1)*8 = 394462 *8 = 3155696 (bit/s)
int audioPackageSizePerSecond = 3155696/250+1 = 12632 (packages/s)
int pesOverhead = 17;
int effectiveAudioPackagePerSecond = 12632+17 = 12649 (packages/s)
int containerPackagesCountPerAudioPackagePerSecond = 12649/184+1 = 69
int containerPackageSizePerAudioPackagePerSecond = 192 * 69 = 13248
audioOverhead = (953002044 * (13248 / 12632) - 953002044 ) * 2,14 * (3155696 / 640 000) = 490378461(C2s)
-> C2s (490378461) is larger than O2 (403752900)

-> no surprise, stupid idea bad result :)

another stupid approach:
looking at audio5.0+audio5.1
code]double length = frameCount/frameRate;
int packageSize = 192;
qint64 audioOverhead = 0;
double packageFill = 184.0;
int averageBitrate = int(rawSize/length+1)*8;
int additionalBitrate = 0;
int additionalRawSize = 0;
if (averageBitrate > 650 000) {
int newSize = 640000/8*length;
averageBitrate = 640000;
additionalRawSize = rawSize - newSize;
additionalBitrate = int(additionalRawSize/length+1)*8;
rawSize = newSize;
}
int audioPackageSizePerSecond = averageBitrate/250+1;
int pesOverhead = 17;
int effectiveAudioPackagePerSecond = audioPackageSizePerSecond+pesOverhead; //+17 due to PES overhead
int containerPackagesCountPerAudioPackagePerSecond = effectiveAudioPackagePerSecond/packageFill+1;
int containerPackageSizePerAudioPackagePerSecond = packageSize * containerPackagesCountPerAudioPackagePerSecond;
audioOverhead = rawSize * containerPackageSizePerAudioPackagePerSecond / audioPackageSizePerSecond - rawSize;
audioOverhead += additionalRawSize/1.69; //1.69 being a magic number
[/code]
int averageBitrate = int(989002438/2661.12+1)*8 = 371649 *8 = 2973192 (bit/s)
newSize = 640000/8*2661.12 = 212889600
averageBitrate = 640000;
additionalRawSize = 989002438-212889600 = 776112838
additionalBitrate = 776112838/8*2661.12 = 258166174432
int audioPackageSizePerSecond = 640000/250+1 = 2561 (packages/s)
int pesOverhead = 17;
int effectiveAudioPackagePerSecond = 2561+17 = 2578 (packages/s)
int containerPackagesCountPerAudioPackagePerSecond = 2578/184+1 = 15
int containerPackageSizePerAudioPackagePerSecond = 192 * 15 = 2880
audioOverhead = 212889600 * (2880 / 2561) - 212889600 = 26517682 byte
audioOverhead = 776112838/1,69 + 26517682 = 459238365 + 26517682 = 485756047 (C1s2)

looking at audio6.0+audio6.1:
int averageBitrate = int(953002044/2415,958+1)*8 = 394462 *8 = 3155696 (bit/s)
newSize = 640000/8*2415,958 = 193276640
averageBitrate = 640000;
additionalRawSize = 953002044-193276640 = 759725404
additionalBitrate = 759725404/8*2415.958 = 229433083450
int audioPackageSizePerSecond = 640000/250+1 = 2561 (packages/s)
int pesOverhead = 17;
int effectiveAudioPackagePerSecond = 2561+17 = 2578 (packages/s)
int containerPackagesCountPerAudioPackagePerSecond = 2578/184+1 = 15
int containerPackageSizePerAudioPackagePerSecond = 192 * 15 = 2880
audioOverhead = 193276640 * (2880 / 2578) - 193276640 = 22641406 byte (C1)
audiOverhead += 759725404/1,69 + 22641406 = 472183065 (C2s2)

-> as suspected just another stupid approach :)

For a better approaches we need:
a. more data sets
b. additional infos about THD (Header&Co)

Cu Selur

jpsdr

17th February 2011, 14:46

I've tried several approches.
- Considering a whole file (note : When i import in Scenarist AC3+THD, in the Software, it appears like one file of the total size in Dolby Lossless format, but maybe it's just a way of displaying things, and there is nothing more).
- Considering two files, ac3 part encoded like we already know, and the rest in unknow format..., but not being able to find anything relevant...
The only thing i've found for now, it's that the biggest commun dividor within 7672704 and 7066432 is 64. Is it an hazard or is it the begining of something ?
I've not been able to figure out yet.

Selur

17th February 2011, 15:46

The only thing i've found for now, it's that the biggest commun dividor within 7672704 and 7066432 is 64.
-> http://britton.disted.camosun.bc.ca/gcdlcm/jbgcdlcm.htm :D

Is it an hazard or is it the begining of something ?
looks like a fluke, don't think it's a indicator for anything,..

Cu Selur

jpsdr

17th February 2011, 15:51

I'll try to see this WE if i can make others datas. I'll have to use the same method : working by differences to get the audio track size.
If 64 appears again, it will mean something. But with only 2, it may be a fluke. It there was 3, it would have been another story...

Selur

17th February 2011, 15:56

Even if all the files are dividable by 64 it will probably not mean anything that can help with the formula.
More data sets would be nice may be we got lucky and someone knowing a bit about thd stumbles over this thread. :)

jpsdr

17th February 2011, 18:03

I've extracted with TSmuxer the audio6.0/6.1, and the result was a big .ac3 file of 953002044 bytes...

jpsdr

18th February 2011, 10:51

Is it normal in post #76 (http://forum.doom9.org/showpost.php?p=1477606&postcount=76) that test is made with 192000, and mult calculated with 194000 ?

Selur

18th February 2011, 10:56

Nope, they are ment to be switched,...

if (averageBitrate > 194000) { //194000 for tolerance
mult = averageBitrate/192000.0; //192000*8 = 1536k
}

jpsdr

18th February 2011, 11:29

A little search gave me :
- There is a battle Dobly vs DTS for their lossless format (THD vs DTS-HD MA).
DTS-HD is also a lossless codec, so, i don't know if in that case bitrate will have a meaning. Nevertheless, DTS-HD MA have 2 stream. A DTS-Core at 1509kb, and another stream where the difference within the lossy DTS and source is compressed in lossless format. So, i don't think DTS and DTS-HD MA will have the same formula (but, i may be wrong saying this).

Selur

18th February 2011, 11:36

So, i don't think DTS and DTS-HD MA will have the same formula (but, i may be wrong saying this).
Me neither, but the mult think worked for the DTS-HD MA streams I had here, so I kept it. :)

Problem is:
1. There are no cheap (or free and legally available) DTS-HD/THD encoders, so I can't produce a bunch of samples with different length for testing.
2. I have no clue how the HD (DTS-HD/THD) data is stored (header size, etc.) so the whole thing is mainly guessing.

Cu Selur

jpsdr

18th February 2011, 11:43

I'll work later on blu ray, one of 1h30, and another of 2h. Both have DTS-HD MA tracks, so, i'll be able to provide some information when i'll be doing them.
I'll will later also have information on THD tracks, but they'll be 2.0 audio, and not 5.1 like the provided. But lenght will be of around 2h.
I'll check, i may have another Blu-Ray with 5.1 audio THD of around 1h30.
But i'll not be able to do this soon.

jpsdr

24th February 2011, 10:13

Some DTS-HD MA data.

Video 7 : 129961 frames @23.976fps raw h264 encoded (x264) at around 227446kb. .264 file size : 18596172377 bytes.
Audio 7.1 (for video 7) : DTS-HD MA 5.1 File size : 2873665900 bytes.
Audio 7.2 (for video 7) : DTS-HD MA 5.1 File size : 2051916196 bytes.
audio length=video length.

M2TS files :
(39) : Video7 + audio7.1 => 22616653824 bytes.
(40) : Video7 + audio7.2 => 21759645696
(41) : Video7 alone => 19454337024 bytes.

Note : For (41) my original purpose was to do Video7+both audios, but, some technical issues prevented it.
So, result is :
(39)-(41) => muxed size of audio 7.1 is 16470400 packets of 192 bytes.
(40)-(41) => muxed size of audio 7.2 is 12006816 packets of 192 bytes.

But, as soon as i'll be able to mux both audio with video, i'll uptade to new datas, if there is some change. What i don't like, but maybe for nothing, is to have a stream without audio, if mux rules are different with or without audio.

One thing for sure, result is size related, and not purely length related, because size is different for audio tracks of same length, but different size.

Edit : @Selur, i've try your formula for DTS-HD. Result have a good accuracy for audio 7.2, but it's 60MB too big for audio 7.1.

Selur

24th February 2011, 10:52

Result have a good accuracy for audio 7.2, but it's 60MB too big for audio 7.1.
is the calculated overhead too big or is the resulting file to big after muxing?
(we are speaking of http://forum.doom9.org/showthread.php?p=1477606#post1477606 right?)

jpsdr

24th February 2011, 11:32

Yes, i'm using the formula you provided, and, calculated overhead for 7.1 is 60MB biger than the result muxed file.
Nevertheless, without more information, i don't know if it's even possible to do better...

Selur

24th February 2011, 22:50

Okay, as long as the overhead is to large and not to small one does at least not run in problems when aiming to burn the output to a DVD/BD. :)

Cu Selur

jpsdr

25th February 2011, 09:55

Update DTS-HD MA data. Should replace those on post #104.

Video 7 : 129961 frames @23.976fps raw h264 encoded (x264) at around 27441kb. .264 file size : 18592785452 bytes.
Audio 7.1 (for video 7) : DTS-HD MA 5.1 File size : 2873665900 bytes.
Audio 7.2 (for video 7) : DTS-HD MA 5.1 File size : 2051916196 bytes.
Audio 7.1.1 (for video 7) : DTS core (1509kb) extracted from Audio 7.1 File size : 1022434016
(Note : DTS core of Audio 7.2 has, as expected, the exact same size).
audio length=video length.

M2TS files :
(42) : Video7 + audio7.1 => 22613164032 bytes.
(43) : Video7 + audio7.2 => 21756149760 bytes.
(44) : Video7 + audio7.1 + audio7.2 => 24918466560 bytes.
(45) : Video7 + audio7.1 + audio 7.1.1 => 23783983104 bytes.

(44)-(42) => muxed audio 7.2 is 12006784 packets of 192 bytes.
(44)-(43) => muxed audio 7.1 is 16470400 packets of 192 bytes.
(45)-(42) => muxed audio 7.1.1 is 6098016 packets of 192 bytes.

jpsdr

25th February 2011, 10:33

16470400 = 2^7x5^2x5147
12006784=2^7x19x4937
6098016=2^5x3x63521
10372384=2^5x11x79x373
5908768=2^5x184649

1509000=2^3x3x5^3x503
1022434016=2^5x503x63521
6098016=2^5x3x63521
2 times 63521 on both original and muxed...??? Hummm, it may be, but i don't think it's a fluke here...

Ok, for 1509kb DTS, it seems to be a 4024/4608 ratio.
1509000/375=4024 (250*1.5=375... Maybe also a fluke, but a little too nice for that...).
1022434016/4024 = 254084
6098016/254084=24 => 4608 muxed frame.
Now, how going from 4024 to 4608...

@Selur : Do you have DTS data avaible to check, at least, if raw DTS track can be divided by bitrate/375 ? (Encoded used bitrate, not estimated bitrate by size/time).

According here (http://dvd.sourceforge.net/dvdinfo/pes-hdr.html), i'll chose a 8 bytes header.
(4024+8)/168 = 24.

So, for DTS, something to try :
packetsize=(bitrate/375) [Bitrate=encoded bitrate, not estimated].
numberpacket=rawsize/packetsize
newpacketsize=(packetsize+8)/168 (round to sup integer if not integer)
newpacketsize*=192
finalsize=numberpacket*newpacketsize.

Selur

25th February 2011, 17:39

Do you have DTS data avaible to check, at least, if raw DTS track can be divided by bitrate/375 ?

Audio codecs : DTS-HD
Complete name : d:\dts samples\sample_13,36.dts
File size : 2565556
Bit rate mode : VBR
-> doesn't show an average bitrate

Audio codecs : DTS-HD
Complete name : d:\dts samples\sample_35,34.dts
File size : 6784824
Bit rate mode : VBR
-> doesn't show an average bitrate

Audio codecs : DTS
Complete name : d:\dts samples\sample_37,53.dts
File size : 7206540
Bit rate mode : CBR
Bit rate : 1509750
1509750/375 = 4026
7206540/4026 = 1790

Audio codecs : DTS-HD
Complete name : d:\dts samples\sample_39,4.dts
File size : 7564744
Bit rate mode : CBR
Bit rate mode : Constant
Bit rate : 2046750
2046750/375 = 5458
7564744/5458 = 1385,9919384389886405276658116526

Audio codecs : DTS
Complete name : d:\dts samples\sample_6473,77.dts
File size : 1221591040
Bit rate mode : CBR
Bit rate : 1509750
1509750/375 = 4026;
1221591040/4026 = 303425,49428713363139592647789369

=> worked on one DTS source failed on another,..
(values are taken from mediaInfo --Full output)

---------------------------------
double length = frameCount/frameRate;
int packageSize = 192;
qint64 audioOverhead = 0;
double packageFill = 184.0;
int DTSFrameCount = length*1000/10.6614+1;
int frameSize = 2013;
int packagesPerFrame = frameSize/packageFill+1;
int packageCount = DTSFrameCount*packagesPerFrame;
int averageBitrate = rawSize/length+1;
double mult = 1;
if (averageBitrate > 194000) { //for DTS-HD/MA until we find something better
mult = averageBitrate/192000.0;
}
packageCount = packageCount*mult;
audioOverhead = (packageCount + packageCount/184.0/8.0)*192 - rawSize; (notice I use: 10.6614 instead of 10.67)
Is the formula I'm using in my HybridTest version atm.,...
DTS frames are 10.6614ms in length iirc so I would have suspected the formula to be 'okay'. ;)

Cu Selur

Stereodude

27th February 2011, 02:20

I'm very interested in this thread. Hopefully we can get some decent bitrate calculators out of the work being done. I haven't found one yet that deals with lossless audio in a reasonable manner. :cool:

Currently I end up having to doing the x264 encode more than once to get the size I want. :(

jpsdr

28th February 2011, 10:17

What is the purpose of this ?

int averageBitrate = int(rawSize/length+1)*8;
int audioPackageSizePerSecond = averageBitrate/250+1;

It's not used afterward...

Selur

28th February 2011, 20:52

averageBitrate is used in the if-clause
audioPackageSizePerSecond was a copy&past 'bug' (from another calculation)

jpsdr

3rd March 2011, 11:15

New datas

Video 8 : 179185 frames @23.976fps raw h264 encoded (x264) at around 23340kb. .264 file size : 21803938888 bytes.
Audio 8.1 (for video 8) : DTS-HD MA 2.0 File size : 1617821020 bytes.
Audio 8.2 (for video 8) : AC3 192kb File size : 179365632 bytes.
audio length=video length.

M2TS files :
(46) : Video8 + audio8.1+audio8.2 => 24945770496 bytes.
(47) : Video8 + audio8.2 => 23044644864 bytes.

(46)-(47) => muxed audio 8.1 is 9901696 packets of 192 bytes.

Edit : With these datas, this time formula gives a result 80MB lower than real size.
Maybe this WE i'll have time to provide others datas with DTS core...

Stereodude

4th March 2011, 03:41

Are you guys just looking for raw example data where DTS-MA audio is present like this?

Case #1
Frames: 287901 (24000/1001fps)
Video size: 17184008928 bytes (AVC)
Audio size: 6268941140 bytes (DTS-MA 5.1)
Subtitle size: 32059770 bytes

M2TS size: 24949260288 bytes

Case #2
Frames: 200261 (24000/1001fps)
Video size: 19749963123 bytes (AVC)
Audio size: 3909095072 bytes (DTS-MA 5.1)
Subtitle size: 26082124 bytes

M2TS size: 25022521344 bytes

Selur

4th March 2011, 09:18

the aim is to find out what the overhead is for the audio, so data of (dts.ma) audio only inside a m2ts would be the best, but since we lack a muxer to do this me mux:
a. video and audio into a m2ts
b. video into a second m2ts
c. subtract b from a to get the m2ts size of a audio only m2ts

Cu Selur

Ps.: btw. we really should set up a spreadsheet or something like it to get some overview of the data collected, since at least I'm fed up with trying to get a handle on the data,.. ;)

jpsdr

4th March 2011, 10:38

Forget any data with subtitles.
As rules for audio only or video only may differ from audio+video, to be more secure, doing substract should be done only with audio+video streams.

Stereodude

5th March 2011, 14:50

the aim is to find out what the overhead is for the audio, so data of (dts.ma) audio only inside a m2ts would be the best, but since we lack a muxer to do this me mux:
a. video and audio into a m2ts
b. video into a second m2ts
c. subtract b from a to get the m2ts size of a audio only m2tsOkay, I'll generate you guys up some data to work with.

Stereodude

5th March 2011, 18:46

Case #1:
Length: 154928 frames (24000/1001fps)
Video size: 20,017,828,210 bytes (AVC)
Audio size: 3,680,669,412 bytes (DTS-HD MA 5.1)
Vid only M2TS: 20,930,635,776 bytes
Vid + Aud M2TS: 24,966,426,624 bytes
Audio only M2TS: 4,035,790,848 bytes (calculated)

case #2:
Length: 161003 frames (24000/1001fps)
Video size: 20,134,252,243 bytes (AVC)
Audio size: 3,538,696,340 bytes (DTS-HD MA 5.1)
Vid only M2TS: 21,053,755,392 bytes
Vid + Aud M2TS: 24,948,744,192 bytes
Audio only M2TS: 3,894,988,800 bytes (calculated)

case #3:
Length: 192912 frames (24000/1001fps)
Video size: 14,090,014,978 bytes (VC1)
Audio size: 4,389,191,780 bytes (DTS-HD MA 5.1)
Vid only M2TS: 14,755,854,336 bytes
Vid + Aud M2TS: 19,578,083,328 bytes
Audio only M2TS: 4,822,228,992 bytes (calculated)

case #4:
Length: 153864 frames (24000/1001fps)
Video size: 20,009,509,099 bytes (AVC)
Audio size: 3,682,385,432 bytes (DTS-HD MA 5.1)
Vid only M2TS: 20,921,647,104 bytes
Vid + Aud M2TS: 24,957,560,832 bytes
Audio only M2TS: 4,035,913,728 bytes (calculated)

case #5:
Length: 149994 frames (24000/1001fps)
Video size: 17,497,563,483 bytes (AVC)
Audio size: 3,485,120,788 bytes (DTS-HD MA 5.1)
Vid only M2TS: 18,299,111,424 bytes
Vid + Aud M2TS: 22,124,206,080 bytes
Audio only M2TS: 3,825,094,656 bytes (calculated)

Stereodude

5th March 2011, 18:59

Are we sure that lossless audio is going to have an overhead that can be exactly calculated?

It seems like there is going to be some variation depending on what the audio is (how well it compresses) because the effective bitrate of the audio stream is constantly changing which will cause the overhead to change as the audio bitrate shrinks or grows. In the 5 examples I just posted the average bitrate of the overhead (in kbit/sec) as a percentage of the average audio bitrate (in kbit/sec) ranges between 9.6% and 10.07%.

Selur

6th March 2011, 11:01

Are we sure that lossless audio is going to have an overhead that can be exactly calculated?
1. We are not sure about anything. :)
2. At least I would be happy to have a formula that calculates the overhead a bit to high (!not to low!, so that the resulting file will not overshoot the targetsize).

In the 5 examples I just posted the average bitrate of the overhead (in kbit/sec) as a percentage of the average audio bitrate (in kbit/sec) ranges between 9.6% and 10.07%.
Problem is we don't know if 10.07% is the maximum (otherwise 10.07-9.6 = 0.45% wouldn't be a bad approximation, around 18MB to high for 4GB audio)

Small question about the numbers you posted:
Does the 'DTS-HD MA 5.1' audio still contain the dts core or is it just the hd part?
(I suspect that one probably has to differ between the dts core and the hd part for overhead calculation)

jpsdr

6th March 2011, 11:39

Unless we find a way to get the ISO/IEC 13818-1, it will be difficult to do more than what we have now.

Selur

6th March 2011, 12:13

Unless we find a way to get the ISO/IEC 13818-1
Thanks to Google: -> http://mumudvb.braice.net/mumudrupal/sites/default/files/iso13818-1.pdf
+ http://mumudvb.braice.net/mumudrupal/fr/node/39
+ http://www.dvb.org/technology/standards/index.xml
(but I'm not sure how this helps with the dts-hd/true-hd overhead calculation)

Stereodude

6th March 2011, 13:52

Problem is we don't know if 10.07% is the maximum (otherwise 10.07-9.6 = 0.45% wouldn't be a bad approximation, around 18MB to high for 4GB audio)FWIW, it appears that the lower the bitrate of the DTS-HD MA audio is the larger the overhead it. The 10.07% is for Case #2 which had the lowest average bitrate. The 9.60% was for the highest average bitrate of the 5.
Small question about the numbers you posted:
Does the 'DTS-HD MA 5.1' audio still contain the dts core or is it just the hd part?
(I suspect that one probably has to differ between the dts core and the hd part for overhead calculation)Yes it contains the DTS core embedded into into the DTS-HD MA stream. DTS-HD MA must have the DTS core in order to decode. DTS-HD MA is basically extra data that is overlaid on top of the core, so the core is an essential part.

Selur

6th March 2011, 13:56

Yes it contains the DTS core embedded into into the DTS-HD MA stream.
okay. :)

"so the core is an essential part. "
Hmm,.. that sounds strange since afaik, when muxing dts-hd stream into an mkv the dts-core is dropped and only the hd part is kept.
-> the hd part shouldn't be just an overlay (I suspect that the dts-core is mainly to be dts kompatible and to allow devices that support dts, but can't playback the hd part to play the dts-core)

Stereodude

6th March 2011, 17:42

okay. :)

"so the core is an essential part. "
Hmm,.. that sounds strange since afaik, when muxing dts-hd stream into an mkv the dts-core is dropped and only the hd part is kept.
-> the hd part shouldn't be just an overlay (I suspect that the dts-core is mainly to be dts kompatible and to allow devices that support dts, but can't playback the hd part to play the dts-core)Feel free to read DTS's own white paper (http://www.dts.com/DownloadDocument.aspx?q=a7beda1e-cfe6-4ca4-b6b2-cda9554bb6a5) (PDF). DTS-HD MA has a core + a residual that when added to the core makes it lossless. You can not decode DTS-HD (MA or not) without the core.

Dolby TrueHD is not the same. There is an AC3 core embedded in the TrueHD stream (on blu-rays, but not HD-DVD) that is not needed to decode the TrueHD.

Selur

6th March 2011, 23:49

Okay, so I mixed up TrueHD and DTSHD. -> nice, one less think to think about when thinking about the overhead. :)

jpsdr

7th March 2011, 09:33

Yes, thanks to Google, you've found like me the 2000 version, wich doesn't seem to have the HD part. Apparently, the needed is the 2007 version.

Selur

7th March 2011, 10:13

Ehmm,.. does the rest of the dts formula make sense compared to the pdf?
@Stereodude: Which tool did you use for muxing?

kieranrk

7th March 2011, 13:43

Yes, thanks to Google, you've found like me the 2000 version, wich doesn't seem to have the HD part. Apparently, the needed is the 2007 version.

There is no mention of DTS-HD in the 2007 version. Only MPEG codecs are specified in 13818-1.

Stereodude

7th March 2011, 13:53

@Stereodude: Which tool did you use for muxing?TSmuxer

jpsdr

8th March 2011, 09:27

There is no mention of DTS-HD in the 2007 version. Only MPEG codecs are specified in 13818-1.
Ok, so, what do we need to find out how DTS, DTS-HD and TrueHD are muxed in m2ts ?

kieranrk

8th March 2011, 09:58

Ok, so, what do we need to find out how DTS, DTS-HD and TrueHD are muxed in m2ts ?

Blu-ray spec.

jpsdr

8th March 2011, 10:37

This means that developper of eac3to, dgindex and tsmuxer had access to it ?
Madshi told me that he used only 13818-1 to make eac3to, and used mts specs, difference was only that m2ts is the same thing than mts, with just 4 bytes more (188 mts + 4 timecode/timing bytes).
This is why i thought 13818-1 would provide needed informations.
I don't think google will provide it... :)

Selur

8th March 2011, 15:50

playing around with a little spreadsheet I built atm: - deleted -
Problem is calculations there are pretty good for some and pretty bad for other data sets,.. :/

Cu Selur

jpsdr

8th March 2011, 16:55

Interesting. Well, it's always better than nothing.
I must said, actually, if i'm doing a Blu-Ray with only AC3, i'm good the 1rst time, but for others, i'm doing a first encode with estimated bitrate, mux, check difference and adjust bitrate for a second encode.

Selur

8th March 2011, 17:00

playing with another spreadsheet I found that for normal dts (without HD):
(1147 * 192 * lenghtInSeconds) - rawAudioSizeInByte seems to be a good approximation for the the DTS Overhead, but no clue how to calculate the overhead caused by the hd part of a dts-hd/ma stream :(

-> adjusting the old approximation, current one looks like this:
qint64 dtsRawsize = rawSize;
qint64 hdRawsize = 0;
int averageBitrate = rawSize / length + 1;
if (averageBitrate > 194000) {
dtsRawsize = 192000 * length;
hdRawsize = (averageBitrate - 192000) * length;
}
int dtsMult = 1147;
audioOverhead = dtsMult * packageSize * length - dtsRawsize;
int hdAverageBitrate = hdRawsize / length;
qint64 hdOverhead = hdAverageBitrate / 179.3 * packageSize * hdRawsize - hdRawsize;
audioOverhead += hdOverhead;

Still off, but hopefully a bit better than the last one.

Cu Selur

jpsdr

9th March 2011, 09:24

rawsize is the total size of the DTS-HD/MA in case of such file ?
Are you sure of :

qint64 hdOverhead = hdAverageBitrate / 179.3 * packageSize * hdRawsize - hdRawsize;

It's not :

qint64 hdOverhead = hdAverageBitrate / 179.3 * packageSize*lenght - hdRawsize;

Selur

9th March 2011, 10:28

yup, it's the raw size of the audio stream

Stereodude

10th March 2011, 04:09

case #6:
Length: 287901 frames (24000/1001fps)
Video size: 17,243,957,113 bytes (AVC)
Audio size: 6,268,941,140 bytes (DTS-HD MA 5.1)
Vid only M2TS: 18,072,600,576 bytes
Vid + Aud M2TS: 24,976,164,864 bytes
Audio only M2TS: 6,903,564,288 bytes (calculated)

Selur

12th March 2011, 10:34

small tweak:
int packageSize = 192;
qint64 audioOverhead = 0;
double packageFill = 184.0;
qint64 dtsRawsize = rawSize;
qint64 hdRawsize = 0;
int averageBitrate = rawSize / length + 1;
qint64 hdOverhead = 0;
if (averageBitrate > 194000) {
dtsRawsize = 192000 * length;
hdRawsize = (averageBitrate - 192000) * length;
if (hdRawsize > 0) {
hdOverhead = hdRawsize*packageSize/packageFill + length*1024*10 - hdRawsize;
}
}
int dtsMult = 1147;
audioOverhead = dtsMult * packageSize * length - dtsRawsize;
audioOverhead += hdOverhead;
-> works fine for all cases from Stereodude, see: http://spreadsheets.google.com/ccc?key=0AvWxUS1XGCPAdHRGUUh6N3B0M3hwUEtWalJZSlpqLUE

Cu Selur

Stereodude

12th March 2011, 15:07

small tweak:
-> works fine for all cases from Stereodude, see: http://spreadsheets.google.com/ccc?key=0AvWxUS1XGCPAdHRGUUh6N3B0M3hwUEtWalJZSlpqLUE

Cu SelurImpressive :cool:

Selur

12th March 2011, 15:12

Yeah, so far so good, but we need:
a. more data in general to check the formula
b. especially more DTS-HD stereo data (formula is 12.75mb of for the single 2.0 data we have atm. and tweaking with just one dataset seems to be a waste of time ;))

Cu Selur

Stereodude

12th March 2011, 15:36

Selur

12th March 2011, 15:38

No, problem we'll tweak that once some one posts some data. :)

Stereodude

12th March 2011, 16:32

case #7:
Length: 171250 frames (24000/1001fps)
Video size: 18,327,269,132 bytes (VC-1)
Audio size: 3,847,129,416 bytes (DTS-HD MA 5.1)
Vid only M2TS: 19,171,301,376 bytes
Vid + Aud M2TS: 23,400,849,408 bytes
Audio only M2TS: 4,229,548,032 bytes (calculated)

case #8:
Length: 269542 frames (24000/1001fps)
Video size: 17,690,261,921 bytes (AVC)
Audio size: 5,849,654,964 bytes (DTS-HD MA 5.1)
Vid only M2TS: 18,533,296,128 bytes
Vid + Aud M2TS: 24,976,232,448 bytes
Audio only M2TS: 6,442,936,320 bytes (calculated)

case #9:
Length: 225060 frames (30000/1001fps)
Video size: 18,749,663,420 bytes (AVC)
Audio size: 4,923,173,968 bytes (DTS-HD MA 5.1)
Vid only M2TS: 19,619,395,584 bytes
Vid + Aud M2TS: 24,983,076,864 bytes
Audio only M2TS: 5,363,681,280 bytes (calculated)

Selur

12th March 2011, 16:40

Updated the spreadsheet, formula holds. :)

If someone reading this can provide some data sets with 2.0 and 7.1 DTS-HD please do,.. (I suspect the current formula only works for 5.1 audio correctly)

Cu Selur

jpsdr

13th March 2011, 10:22

Nice, we begin to have something.
Unfortunately, i don't have actualy others DTS-HD datas.
Nevertheless, i'll probably be able to produce in the future others THD datas.

Stereodude

27th March 2011, 16:23

Fixed / Updated

case #10:
Length: 144427 frames (24000/1001fps)
Video size: 26,544,343,977 bytes (AVC)
Audio size: 3,797,944,268 bytes (DTS-HD MA 7.1)
Vid only M2TS: 27,737,769,984 bytes
Vid + Aud M2TS: 31,882,395,648 bytes
Audio only M2TS: 4,144,625,664 bytes (calculated)

Fixed / Updated

Selur

27th March 2011, 16:30

Thanks!

Added it to the spreadsheet and like suspected the calculated overhead is off. :/
(will need more 7.1 data before thinking about tweaks for 7.1 and 2.0 makes sense)

Cu Selur

Stereodude

27th March 2011, 17:05

I went looking and found a few movies in my collection that have DTS-HD MA 7.1 audio (there aren't a whole lot). I'll put together some data for them when I get a chance.

Selur

27th March 2011, 17:06

Nice!

Stereodude

27th March 2011, 21:07

Please note I corrected case #10, which had some mistakes.

case #11:
Length: 161592 frames (24000/1001fps)
Video size: 18,458,379,010 bytes (AVC)
Audio size: 4,039,844,668 bytes (DTS-HD MA 7.1)
Vid only M2TS: 19,305,302,016 bytes
Vid + Aud M2TS: 23,723,827,200 bytes
Audio only M2TS: 4,418,525,184 bytes (calculated)

case #12:
Length: 137875 frames (24000/1001fps)
Video size: 15,012,943,371 bytes (AVC)
Audio size: 3,634,629,704 bytes (DTS-HD MA 7.1)
Vid only M2TS: 15,703,142,400 bytes
Vid + Aud M2TS: 19,668,879,360 bytes
Audio only M2TS: 3,965,736,960 bytes (calculated)

case #13:
Length: 160729 frames (24000/1001fps)
Video size: 14,941,181,034 bytes (VC-1)
Audio size: 4,954,870,348 bytes (DTS-HD MA 7.1)
Vid only M2TS: 15,635,349,504 bytes
Vid + Aud M2TS: 21,007,644,672 bytes
Audio only M2TS: 5,372,295,168 bytes (calculated)

case #14:
Length: 172362 frames (24000/1001fps)
Video size: 18,825,172,771 bytes (AVC)
Audio size: 4,748,508,680 bytes (DTS-HD MA 7.1)
Vid only M2TS: 19,690,653,696 bytes
Vid + Aud M2TS: 24,862,298,112 bytes
Audio only M2TS: 5,171,644,416 bytes (calculated)

Selur

27th March 2011, 21:19

Cool! Added the data to the spreadsheet an it seems that the only problematic result is the stereo sample, all the 5.1 and 7.1 calculations seem fine enough. :)

Cu Selur

jpsdr

31st March 2011, 09:00

More THD datas.

Video9 : 263694 frames @23.976fps raw h264 encoded (x264) at around 33092kb. .264 file size : 45494178979 bytes.
Video10 : 214220 frames @23.976fps raw h264 encoded (x264) at around 39480kb. .264 file size : 44093036946 bytes.
Audio 9.0 (for video 9) : THD. File size : 1133934414 bytes.
Audio 9.1 (for video 9) : AC3 encoded at 448kb. AC3 file size : 615901440 bytes.
Audio 10.0 (for video 10) : THD. File size : 848184778 bytes.
Audio 10.1 (for video 10) : AC3 encoded at 448kb. AC3 file size : 500347904 bytes.
Audio is 2.0 for all here.

M2TS files :
(48) : Video9 + audio9.1 => 48232955904 bytes.
(49) : Video9 + audio9.1 + audio9.0&9.1 => 51437924352 bytes.
(50) : Video10 + audio10.1 => 46628130816 bytes.
(51) : Video10 + audio10.1 + audio10.0&10.1 => 49228922880 bytes.

(49)-(48) => 16692544 x 192 for THD+AC3 track.
(51)-(50) => 13545792 x 192 for THD+AC3 track.

jpsdr

31st March 2011, 17:05

From the few THD data avaible, i've the following rough estimation :
When AC3 + THD : Compute AC3 size as already known, and add the following :
THD 5.1 : size=2435.34066*length*192
THD 2.0 : size=1204.41159*length*192
Result in bytes, length is duration is seconds.

Selur

31st March 2011, 17:14

so THD 2.0 would be around half the size of THD 5.1 :)
would be cool if someone could collect all the ac3&THD data together, since I lost the overview so setting up another spreadsheet will have to wait :)

jpsdr

31st March 2011, 17:20

I'll nevertheless try to check if when changing the bitrate of AC3 track with the THD, my formula is still (roughly) correct...

Selur

31st March 2011, 17:32

Sure do that, if you find a decent solution please post.

Up till now we got decent formulas for:
a. ac3 (http://forum.doom9.org/showthread.php?p=1478668#post1478668)
b. dts&dts-hd (http://forum.doom9.org/showthread.php?p=1484281#post1484281)
would be cool if we could tweak the ac3-formula to also handle thd, but we will see :)

Cu Selur

jpsdr

31st March 2011, 17:34

Using data in my post #50 (http://forum.doom9.org/showpost.php?p=1474499&postcount=50), the difference for video5 with 640/256 AC3 track with the m2ts files is 748448 (x192).
Differences of the AC3 tracks, theorical according the formula we know is 748458 (x192).
I think, at least, considering the fact that in THD+AC3 the AC3 track is computed like we know seems to be validated.

Selur

31st March 2011, 17:39

yes, ac3, dts and dts-hd seem to be fine. :)
(problem lies with: a. thd streams on their own and b. (ac3+thd) - combi - streams)

Selur

31st March 2011, 17:51

with input being rawSize, frameCount, frameRate, I wonder how good/bad:
double length = frameCount / frameRate;
int packageSize = 192;
qint64 audioOverhead = 0;
double packageFill = 184.0;
int pesOverhead = 17;
qint64 ac3rawSize = rawSize;
qint64 hdRawSize = 0;
if (averageBitrate > 88000) {
ac3Rawsize = 80000 * length;
hdRawsize = (averageBitrate - 80000) * length;
averageBitrate = 80000;
}
int audioPackageSizePerSecond = averageBitrate / 250 + 1; // = averageBitrate*8/1000/2
int effectiveAudioPackagePerSecond = audioPackageSizePerSecond + pesOverhead;
int containerPackagesCountPerAudioPackagePerSecond = effectiveAudioPackagePerSecond / packageFill + 1;
int containerPackageSizePerAudioPackagePerSecond = packageSize * containerPackagesCountPerAudioPackagePerSecond;
audioOverhead = ac3Rawsize * containerPackageSizePerAudioPackagePerSecond / audioPackageSizePerSecond - ac3Rawsize;
qint64 hdOverhead = 0;
if (hdRawsize > 0) {
hdOverhead = hdRawsize * packageSize / packageFill + length * 1024 * 10 - hdRawsize;
}
audioOverhead += hdOverhead; would be as an approximation,.. (for ac3/thd/ac3+thd streams)
(Warning: not tested with any numbers, it's basically the ac3 formula with the hdOverhead part used in the dts-hd formula.)

Cu Selur

Selur

31st March 2011, 19:46

Could you recheck the numbers you posted?

Video9 : 263694 frames @23.976fps raw h264 encoded (x264) at around 33092kb. .264 file size : 45494178979 bytes.
Audio 9.0 (for video 9) : THD. File size : 1133934414 bytes.
Audio 9.1 (for video 9) : AC3 encoded at 448kb. AC3 file size : 615901440 bytes.

M2TS files :
(48) : Video9 + audio9.1 => 48232955904 bytes.
(49) : Video9 + audio9.1 + audio9.0&9.1 => 51437924352 bytes.
(49)-(48) => 16692544 x 192 for THD+AC3 track. (= 3204968448 bytes) would for ac3&thd result in:
audioRawSize: 1133934414 + 615901440 = 1 749 835 854 byte
subtracting the audioRawSize from the muxed m2ts we get a targetAudioOverhead of (3 204 968 448 - 1 749 835 854 = ) 1 455 132 594 byte
length = 10999 seconds
average bitrate = audioRawSize/length = 159 091 byte
since max ac3 bitrate is 640kbit/s = 640*1000/8 = 80000 byte/s
I assume the ac3 bitrate to be 80000 byte/s
and the hdbitrate to be 159 091 - 80 000 = 79091 byte/s
which leaves us with a hdrawsize of 79091*10999 = 869921909 byte;
with our ac3 overhead formula, we get an ac3 overhead of 172 694 579 byte
subtracting this overhead form the targetAudioOverhead we get a hdTargetOverhead of (1 455 132 594 - 172 694 579 = ) 1 282 438 015 byte
which is rather huge with more than 100% overhead,..
If the numbers are correct and I didn't make a mistake in the calcultion, I really can't recommend anyone to use trueHD audio inside a m2ts container.

Cu Selur

jpsdr

1st April 2011, 08:51

yes, ac3, dts and dts-hd seem to be fine. :)
(problem lies with: a. thd streams on their own and b. (ac3+thd) - combi - streams)

What i wanted to said is, that in case of an audio track wich is made with AC3+THD, computing the AC3 part size with the formula for AC3 only seems correct.
As i'm working only for Blu-Ray target, THD alone is a non-existant case for me.

And yes, values are correct, THD wastes space, and muxed stream seems somehow to follow some kind of constant fixed bitrate constrain.

I think it's not a good idea to try to work considering global size of THD+AC3, because as i've said, apparently AC3 part follow exactly the normal rule.

jpsdr

1st April 2011, 08:58

length = 10999 seconds
average bitrate = audioRawSize/length = 159 091 byte
since max ac3 bitrate is 640kbit/s = 640*1000/8 = 80000 byte/s
I assume the ac3 bitrate to be 80000 byte/s

Wrong. Haven't you read ? I said AC3 bitrate is 448kb.

Selur

1st April 2011, 09:02

No, I read what you wrote, but unless the ac3 bitrate is limited to 448kBit/s when combined with thd, I don't know for sure that the ac3 datarate really is X kBit/s, so I assume the maximum of 640kbit/s. (I suspect that this sloppiness will not much harm)

jpsdr

1st April 2011, 12:29

I think it's an error. Contrary to DTS-HD MA where lossless part is linked to lossy part, you can't decode the lossless part without the lossy part, AC3 and THD are totaly independant, you can decode the THD part without the AC3 part. The only reason there is AC3 with THD, it's because THD decoder is not mandatory for a Blu-Ray player manufacturer, so, there must be a format audio to "back-up" in case player is not able to decode THD track. AC3+THD track should be, for calculating space, view like 2 differents audio tracks. One with THD and one with AC3.
What can confirm this hypothesis it's that in authoring software, you enter the 2 files (the AC3 and THD), but only one for DTS-HD MA.
More, results in my previous post show that AC3 muxing size follow the standard rule.
Now, it's what i strongly believe, but have no concrete proof for it, and so, i still may be wrong...

Selur

1st April 2011, 13:02

So, now someone just needs to come up with a formula how to calculate the overhead (or resulting size) when muxing a thd file with lenght A and size B into a m2ts container.

Here's the data I extracted from previous posts:
Comment fcount fmult fdiv length raw size muxedsize targetOverhead avgBit AC3rawSize avgAC3 ac3Overhead hdRawSize targetHDOverhead
9.0+9.1 (2.0) 263694 24000 1001 10999 1749835854 3204968448 1455132594 159091 879920000 80000 172694579 869915854 1280601641
10.0+10.1 (2.0) 214220 24000 1001 8935 1348532682 2600792064 1252259382 150927 714800000 80000 140287850 633732682 1110479760
6.0+6.1 (5.1) 57925 24000 1001 2416 953002044 1356754944 403752900 394455 193280000 80000 37933457 759722044 365416072
5.0+5.1 (5.1) 63804 24000 1001 2662 989002438 1473159168 484156730 371527 212960000 80000 41795887 776042438 441916400 (ac3Overhead was calculated by: http://forum.doom9.org/showthread.php?p=1478668#post1478668, ac3rawSize was calculated using 80000byte/s*length)

Cu Selur

jpsdr

1st April 2011, 15:24

I dont have the same point of view.
I personnaly think datas are :

Comment fcount fmult fdiv length raw size muxedsize targetOverhead avgBit AC3rawSize avgAC3 ac3Overhead THDRawSize targetTHDOverhead
9.0+9.1 (2.0) 263694 24000 1001 10999 1749835854 3204968448 1455132594 159091 615901440 56000 43992960 1133934414 1411139634
10.0+10.1 (2.0) 214220 24000 1001 8935 1348532682 2600792064 1252259382 150927 500347904 56000 35739136 848184778 1216520246
6.0+6.1 (5.1) 57925 24000 1001 2416 953002044 1356754944 403752900 394455 193280000 80000 24160000 759722044 379592900
5.0+5.1 (5.1) 63804 24000 1001 2662 989002438 1473159168 484156730 371527 212894720 80000 26611840 776107718 457544890

If someone want to try and compare both point of view.

Selur

1st April 2011, 15:35

ah, okay, so only the 2.0 Streams are 448kBit/s,..

How did you calculate the ac3Overhead?
If I use:
# int audioPackageSizePerSecond = averageBitrate / 250 + 1;
= 56000/250+1 = 225
# int effectiveAudioPackagePerSecond = audioPackageSizePerSecond + pesOverhead;
= 225 + 17 = 242
# int containerPackagesCountPerAudioPackagePerSecond = effectiveAudioPackagePerSecond / packageFill + 1;
= 264/184+1 = 2
#int containerPackageSizePerAudioPackagePerSecond = packageSize * containerPackagesCountPerAudioPackagePerSecond;
= 192 * 2 = 384
# audioOverhead = ac3Rawsize * containerPackageSizePerAudioPackagePerSecond / audioPackageSizePerSecond - ac3Rawsize;
= 615901440 * 384 / 242 - 615901440 = 361396712
I end up with 361396712 but you use 43992960, why?

jpsdr

2nd April 2011, 08:56

Lol... averageBitate should be in bit/s, not byte/s. :)
448000/250 = 1792
(1792+17)/184 =10 => 1920
615901440/1792 = 343695
343695*1920=659894400
659894400-615901440=43992960.
Same thing with 640kb. Ratio is 2880/2560.

Selur

2nd April 2011, 18:57

I always use byte/s in my formulas :)

jpsdr

3rd April 2011, 07:18

So you need to adapt coefficients then. It clear it's in b/s and not B/s in drmpeg post #52 (http://forum.doom9.org/showpost.php?p=1474557&postcount=52).

Selur

3rd April 2011, 08:25

seems like there's something off with my current ac3 calculation,.. -> fixed it,..

Problem is we don't have enough data sets an knowledge about THD atm to solve this.
we got
length THDRawSize targetTHDOverhead
10999 1133934414 1411139634
8935 848184778 1216520246
2416 759722044 379592900
2662 776107718 457544890
but there are probably two or more case switches dependent on unknown restrictions that would need to be approximated,...
-> will thing about it again if we have a lot more different data sets

collected data: (ac3/trueHD/ac3&truehd)
http://spreadsheets.google.com/ccc?key=0AvWxUS1XGCPAdEZ3WGE4UE5JSUROSG12RWJPTWxUZnc

jpsdr

4th April 2011, 09:21

It seems that when you are doing (THDRaw+THDOverhead)/length you get roughly the same result, at least for the same kind of audio (2.0 or 5.1).
That's for now the most close thing we have.

Stereodude

11th May 2011, 01:03

I was shocked today when I muxed a DD THD + AC3 soundtrack + AVC video into a M2TS (with tsmuxer) and found it had way more overhead than an equivalently large DTS-MA audio stream. I have a rather crude formula that still gets me really close to my target size when using DTS-MA. However, with DD THD + AC3 audio and the same formula I ended up with BD25 that was 800MB too large. :(

Selur

11th May 2011, 07:01

Only Problem is the THD Overhead for the rest the formulas we got in this thread are relatively exact. ;)
Problem is no one here seems to know anything deeper about the thd audio so coming up with a formula hit a dead end.

jpsdr

11th May 2011, 08:34

I've put a rough estimation of THD in post #156 here (http://forum.doom9.org/showthread.php?p=1488847#post1488847).
Nevertheless, it's still rough. If like me you want to hit a precise result to fill your BD25 as much as possible, you can't avoid at least encoding twice your video...

Stereodude

12th May 2011, 01:18

If like me you want to hit a precise result to fill your BD25 as much as possible, you can't avoid at least encoding twice your video...Or three times if you manage to just barely overshoot the 2nd time.:o

jpsdr

12th May 2011, 08:28

Yes, when your final muxed result is 25kB too big to fit on BR, you cry...

12th May 2011, 08:44

If like me you want to hit a precise result to fill your BD25 as much as possible, you can't avoid at least encoding twice your video...

Can't you mux audio with first pass video to check the overhead amount and then adjust second pass bitrate accordingly?

jpsdr

13th May 2011, 09:16

No, beacause :
- My encodes are running most of the time while i'm not at home, and, i prefer to have optimal result, that target bitrate is exactly the same during 1st and 2nd pass.
- I've noticed, in practical use, "huge" differences between final bitrate of 1st and 2nd pass, most of the time bitrate of 1st pass is lower, wich make size of 1st pass unrealiable to adjust accordingly, when you want to fit at around 10MB for BR25 and 20MB for BD50.

13th May 2011, 10:19

No, beacause :
- My encodes are running most of the time while i'm not at home

Write a script.

and, i prefer to have optimal result, that target bitrate is exactly the same during 1st and 2nd pass.

That's really not necessary when you are only adjusting by some percents!

- I've noticed, in practical use, "huge" differences between final bitrate of 1st and 2nd pass, most of the time bitrate of 1st pass is lower, wich make size of 1st pass unrealiable to adjust accordingly, when you want to fit at around 10MB for BR25 and 20MB for BD50.

I guess you could adjust ratetol (http://mewiki.project357.com/wiki/X264_Settings#ratetol).

But worrying about a 0.1 % undershoot is pretty insane anyway, IMO.

jpsdr

13th May 2011, 13:21

Not at home, means : If i'm not home during 8h, and encoding take : 1h for 1st pass and 6h for 2nd pass, it will not change anything to stop after 1h or do the encode totaly, in both case, it will finish before my return.

But i'm insane, and using insane setting for encoding also. ;)

Audionut

13th May 2011, 17:31

But i'm insane, and using insane setting for encoding also. ;)

Ok, getting a dual-layer dvd movie that only uses 5gb is pretty crap. But seriously, .05%.:confused::eek:

Encode something at 10000kbit/s, then the same material at 9950kbit/s. See is you can spot the difference. And that is .5%. 10x greater than what you are shooting for.

Stereodude

13th May 2011, 21:57

Ok, getting a dual-layer dvd movie that only uses 5gb is pretty crap. But seriously, .05%.:confused::eek:

Encode something at 10000kbit/s, then the same material at 9950kbit/s. See is you can spot the difference. And that is .5%. 10x greater than what you are shooting for.Well, I'm more curious to know what sort of PC he has that he can encode a full movie at "insane" settings two passes in 8 hours. My i7-2600k at 4.2gHz with a full x64 flow takes about 12 hours for 2 passes at very slow.

Audionut

14th May 2011, 03:20

My i7-2600k at 4.2gHz with a full x64 flow takes about 12 hours for 2 passes at very slow.

Yeah, he does seem to be getting some good speeds for 'insane' settings.
Also, I don't encode 2 pass. But it is my understanding that there isn't a 6:1 ratio between passes either.

Stereodude

14th May 2011, 04:32

Yeah, he does seem to be getting some good speeds for 'insane' settings.
Also, I don't encode 2 pass. But it is my understanding that there isn't a 6:1 ratio between passes either.Unless you use --slow-firstpass the first pass is much faster than the second pass especially at the slower presets. On my system it's roughly 10:1. However, --placebo invokes --slow-firstpass which makes the first pass take as long as the second pass.

I guess we'll have to wait to see the command line he's using...

Selur

14th May 2011, 10:47

would be nice if you could keep the whole "how to tune my command line" out of this thread,...

thanks

jpsdr

16th May 2011, 08:47

I have an i7@980.
Don't have here my commande line, but i use placebo, and ajust a little the 1rst pass. Take me several tries to be sure to have the exact same numbers of I,B and P frame between passes and the overall parameters having globaly the same value. Speed ratio is around 4 between passes, and take around 24h for a 2h 1080p video, and 8h for a 720p video.

You can find my 1080p commande line here (http://forum.doom9.org/showthread.php?p=1459705#post1459705).
For 480p upscaled to 720p, it's the same but with level 4.0, 6 ref frames, 48 keyint, no slices and bitrate/buffer max at 15000.
All my workflow is also x64 (lossless codec x64 (UT Video) + Avisynth x64 + x264 x64).

radius

16th June 2011, 22:39

Hi guys, I'm trying to figure out the same thd overhead calculation, did you guys ended with some formula / tool for this :) ?

Selur

16th June 2011, 22:41

nope, not enough statistical data to even think about a decent formula without knowing some basics about thd compression.

Stereodude

17th June 2011, 00:56

Probably due partly to the fact that THD is pretty rare on Blu-Ray discs. DTS-MA is much much more common.

Selur

17th June 2011, 07:16

+ no one knowns any specification
+ no reasonable priced encoder is available

jpsdr

17th June 2011, 08:04

As i've said, i've put a rough estimation in post #156 here (http://forum.doom9.org/showthread.php?p=1488847#post1488847).
The few datas i've got give me the feeling that it may be impossible to have an very accurate estimation, because final size may be related too much to actual data. But it's only a guess.

Stereodude

18th July 2011, 03:02

FWIW, I had a chance to play with Scenarist a little. It's m2ts files are slightly larger than tsMuxeR's. Like 20MB over a full "25GB" Blu-Ray.

Selur

18th July 2011, 05:50

Stereodude

18th July 2011, 12:15

In general? With what content (steam types) ?
at the moment I roughly approximate the tsMuxeR Overhead by int(lengthInSeconds + 1) * 20.9 * 192 (byte) there might be a similar way to approximate the overhead Scenarist uses.Well, I only had a relatively short time to play with it, but on the 3 "movie only" discs I mux'd of AVC (H.264) + 5.1 DTS-MA (no subtitles) they were bigger.

jpsdr

20th July 2011, 08:25

For information, all the datas i've provided were muxed with Scenarist.

Selur

20th July 2011, 08:27

strange, the formulas did hold up when just leaving out the tsmuxerOverhead,..

jpsdr

29th November 2011, 10:05

@Selur : What the status of information/formulas of DTS-HD MA 2.0 ? If i remember properly, you had several sources and "lot" of information with DTS-HD MA 5.1, but few of 2.0. You made a google excel somewhere, no ?

Selur

30th November 2011, 02:54

Status hasn't changed since I gave up on it. I really don't have much samples or data to start with,..

https://docs.google.com/spreadsheet/ccc?key=0AvWxUS1XGCPAdHRGUUh6N3B0M3hwUEtWalJZSlpqLUE&hl=en_US#gid=0
and
https://docs.google.com/spreadsheet/ccc?key=0AvWxUS1XGCPAdEZ3WGE4UE5JSUROSG12RWJPTWxUZnc&hl=en_US#gid=0

were this I did 1/2 a year ago, but the formulas there are far from being usable,...

Cu Selur

jpsdr

30th November 2011, 09:51

If i remember correctly, you have relatively good working formula for DTS-HD MA 5.1 (because you have a lot of datas), but unfortunately, only one data of 2.0, and formula is not as good with it, but with only one data, it's normal. I can provide you at least another data of 2.0 stream, of about 1h30, wich will provide you 2 packs of data with different duration. So :
1 : Do you think this can help you to improve/tune formulas for DTS-HD MA 2.0 ?
2 : Are you still interested in trying to improve it ?

Your formula in post 141 (http://forum.doom9.org/showpost.php?p=1484281&postcount=141) gives good result with 5.1 and 7.1 according the Excel sheet, at this point, without the specific blu-ray spec, i don't think improvement can be made. But, maybe with a little more datas, improvement can be made on 2.0.

QBhd

16th January 2014, 01:15

Sorry for bringing up an old topic, but it's long over due for MeGUI's bitrate calculator to be fixed. I just recently did an encode after a very long time and decided to try the BC in MeGUI one more time since it's been many updates since the last time I used it... After muxing to Blu-Ray structure (ie. m2ts) it came up 102% in Imgburn... argh! Still not fixed... so I pulled up my calculations (I used to do BD5's and BD9's) and attempted it for my first BD25 encode. Result was 99.25%. So I searched for this topic and thought I would share my results in the hopes this can be fixed.

I break things down into 3 overheads (o/h) in kbps: Video, Audio and Blu-Ray structure. I didn't take the approach of an o/h as a size, but instead looked at it as a rate. We know what the capacity of a disc is, and as such the maximum bitrate it can hold for a given movie; and we know the bitrate of the audio... calculating the video bitrate should follow pretty easily. Since we are trying to calculate a bitrate to encode at, it made sense to do it this way.

First step is easy.... Blu-Ray folders always have the same size when using tsMuxer: 655,360 B so that o/h is simple.

To find the Video o/h, I would just mux the encoded video in an m2ts from RAW and compare the RAW vs m2ts file size and when plotted as bitrates this is what we get:

http://img69.imageshack.us/img69/571/nv2g.png

As you can see, Video o/h is a perfect line.

Next up is Audio o/h... I would go back to tsMuxer add the Audio and then mux again. By taking the resulting m2ts and subtracting the audio, video and video o/h, we get the audio o/h. Again, I looked at this as a bitrate and a few things became very clear. DTS is easy:

http://img854.imageshack.us/img854/7499/vybz.png

That's an average value of 220.58 kbps... a fixed o/h for a fixed bitrate.

For DTS-HDMA it's a little different since the bitrate of the audio file is not fixed like DTS and as such the o/h is not fixed either. And here are those results:

http://img844.imageshack.us/img844/8910/ot2t.png

Again we see a very clear equation emerge. Armed with these 3 equations for o/h (Video, DTS and DTS-HDMA) it is not very difficult to come up with a calculated video bitrate.

After I started this posted I realized a few things. One, I was using the "Size on Disk" values for most (if not all) of the size (B) values... and that is not correct.

Secondly, I have a large collection of video files I can pull apart and remux to get an even better set of data points for all kinds of differing audio formats and configurations. I had just been collecting data as I encoded things, but I may do a few daily and re-do my spread sheet with more data and proper sizes recorded.

And finally, I realized that subs too must have an o/h, and even though they take up a small % of the file size, it may be a good idea to investigate this further too.

I hope this helps to get the MeGUI bitrate calculator fixed :)

QB

Emulgator

16th January 2014, 03:08

Before tsMuxeR 2.0.8b rhythm of insertion of PAT/PMT packs was as follows: (PAT/PMT/SIT: every 285ms).

From tsMuxeR 2.0.8b on the rhythm of insertion of PAT/PMT packs happened more often (PAT/PMT/SIT: every 94ms)

I fixed several errors and warnings in version 2.x.x due to BDROM standart (I checked it using bdrom validation software).
One of recomendation - repeat PAT/PMT data every 100ms or faster. So, I increased PAT/PMT rate.
http://forum.doom9.org/showthread.php?t=168539&page=29

From tsMuxeR 2.1.6b on mux overhead has been reduced again.
I think it would be great if you could adjust PAT/PMT/SIT rates to these numbers (94ms for PAT/PMT and 900 ms for SIT).
Advantages:
- more matching with original Blu-Ray transport stream.
- less overhead.
http://forum.doom9.org/showthread.php?t=168539&page=31

So overhead results will differ with muxer versions.

QBhd

16th January 2014, 04:18

Indeed... but MeGUI still uses 1.10.6 and as such it's the calculator that is broken and should be fixed and I thought this was the thread that was going to be used to help fix it... if I am wrong I apologize.

QB

hello_hello

16th January 2014, 05:57

MeGUI is using tsmuxer version 2.6.11 here. I've no idea how that version effects the accuracy of the bitrate calculator (I never use it myself) but maybe try switching to the development update server and updating whatever needs it. I've been running the latest MeGUI version (2460) since it hit the development update server without any problems.

Then again, the bitrate calculator wiki page has this to say:
http://mewiki.project357.com/wiki/MeGUI/Bitrate_Calculator
Note: The M2TS calculation is a WIP. Also, the quality estimate may require further refinement.

Selur

16th January 2014, 06:02

Since the current m2ts calculation code is based on the code I came up with for Hybrid back when this thread was active before, I could share my current m2ts methods (which I use in Hybrid nowadys) with the MeGui devs if they are interested.
I adjusted my old code after the new tsMuxeR version appeared.
Difference between this code and the old code isn't really much, so adjusting the m2ts size calculation in MeGui shouldn't be hard.

Problems of my code:
a. 2.0 DTS-MA overhead estimation could be better (didn't have much data to adjust it)
b. true-hd overhead calculation is a WILD GUESS and as much off as before
c. formulas are mainly data driven and their main goal is to make sure the calculated overhead is always larger than the actual occurring one so that the resulting file size will not exceed the target size -> they don't try be too accurate since accurate calculations would require to know the actual frame sized of the audio&video frames
d. subtitle and structure sizes are simply used as fixed sizes since I never did any testing in how much overhead they really create
e. no clue if the video overhead is still correct for 3D content, since I don't have any data about 3d content
f. formula is ment for 2pass encoding (since only then does file size estimation make any sense; at least for me)
g. code isn't really documented, but since I use decent variable names the code should be easy to read an follow and quite similar to MeGuis current code

Cu Selur