How compute m2ts result files size ?

[Stereodude]

I went looking and found a few movies in my collection that have DTS-HD MA 7.1 audio (there aren't a whole lot). I'll put together some data for them when I get a chance.

[Selur]

Nice!

[Stereodude]

Please note I corrected case #10, which had some mistakes.

case #11:
Length: 161592 frames (24000/1001fps)
Video size: 18,458,379,010 bytes (AVC)
Audio size: 4,039,844,668 bytes (DTS-HD MA 7.1)
Vid only M2TS: 19,305,302,016 bytes
Vid + Aud M2TS: 23,723,827,200 bytes
Audio only M2TS: 4,418,525,184 bytes (calculated)


case #12:
Length: 137875 frames (24000/1001fps)
Video size: 15,012,943,371 bytes (AVC)
Audio size: 3,634,629,704 bytes (DTS-HD MA 7.1)
Vid only M2TS: 15,703,142,400 bytes
Vid + Aud M2TS: 19,668,879,360 bytes
Audio only M2TS: 3,965,736,960 bytes (calculated)


case #13:
Length: 160729 frames (24000/1001fps)
Video size: 14,941,181,034 bytes (VC-1)
Audio size: 4,954,870,348 bytes (DTS-HD MA 7.1)
Vid only M2TS: 15,635,349,504 bytes
Vid + Aud M2TS: 21,007,644,672 bytes
Audio only M2TS: 5,372,295,168 bytes (calculated)


case #14:
Length: 172362 frames (24000/1001fps)
Video size: 18,825,172,771 bytes (AVC)
Audio size: 4,748,508,680 bytes (DTS-HD MA 7.1)
Vid only M2TS: 19,690,653,696 bytes
Vid + Aud M2TS: 24,862,298,112 bytes
Audio only M2TS: 5,171,644,416 bytes (calculated)

[Selur]

Cool! Added the data to the spreadsheet an it seems that the only problematic result is the stereo sample, all the 5.1 and 7.1 calculations seem fine enough. :)

Cu Selur

[jpsdr]

More THD datas.

Video9 : 263694 frames @23.976fps raw h264 encoded (x264) at around 33092kb. .264 file size : 45494178979 bytes.
Video10 : 214220 frames @23.976fps raw h264 encoded (x264) at around 39480kb. .264 file size : 44093036946 bytes.
Audio 9.0 (for video 9) : THD. File size : 1133934414 bytes.
Audio 9.1 (for video 9) : AC3 encoded at 448kb. AC3 file size : 615901440 bytes.
Audio 10.0 (for video 10) : THD. File size : 848184778 bytes.
Audio 10.1 (for video 10) : AC3 encoded at 448kb. AC3 file size : 500347904 bytes.
Audio is 2.0 for all here.

M2TS files :
(48) : Video9 + audio9.1 => 48232955904 bytes.
(49) : Video9 + audio9.1 + audio9.0&9.1 => 51437924352 bytes.
(50) : Video10 + audio10.1 => 46628130816 bytes.
(51) : Video10 + audio10.1 + audio10.0&10.1 => 49228922880 bytes.

(49)-(48) => 16692544 x 192 for THD+AC3 track.
(51)-(50) => 13545792 x 192 for THD+AC3 track.

[jpsdr]

From the few THD data avaible, i've the following rough estimation :
When AC3 + THD : Compute AC3 size as already known, and add the following :
THD 5.1 : size=2435.34066*length*192
THD 2.0 : size=1204.41159*length*192
Result in bytes, length is duration is seconds.

[Selur]

so THD 2.0 would be around half the size of THD 5.1 :)
would be cool if someone could collect all the ac3&THD data together, since I lost the overview so setting up another spreadsheet will have to wait :)

[jpsdr]

I'll nevertheless try to check if when changing the bitrate of AC3 track with the THD, my formula is still (roughly) correct...

[Selur]

Sure do that, if you find a decent solution please post.

Up till now we got decent formulas for:
a. ac3 (http://forum.doom9.org/showthread.php?p=1478668#post1478668)
b. dts&dts-hd (http://forum.doom9.org/showthread.php?p=1484281#post1484281)
would be cool if we could tweak the ac3-formula to also handle thd, but we will see :)

Cu Selur

[jpsdr]

Using data in my post #50 (http://forum.doom9.org/showpost.php?p=1474499&postcount=50), the difference for video5 with 640/256 AC3 track with the m2ts files is 748448 (x192).
Differences of the AC3 tracks, theorical according the formula we know is 748458 (x192).
I think, at least, considering the fact that in THD+AC3 the AC3 track is computed like we know seems to be validated.

[Selur]

yes, ac3, dts and dts-hd seem to be fine. :)
(problem lies with: a. thd streams on their own and b. (ac3+thd) - combi - streams)

[Selur]

with input being rawSize, frameCount, frameRate, I wonder how good/bad:
double length = frameCount / frameRate;
int packageSize = 192;
qint64 audioOverhead = 0;
double packageFill = 184.0;
int pesOverhead = 17;
qint64 ac3rawSize = rawSize;
qint64 hdRawSize = 0;
if (averageBitrate > 88000) {
ac3Rawsize = 80000 * length;
hdRawsize = (averageBitrate - 80000) * length;
averageBitrate = 80000;
}
int audioPackageSizePerSecond = averageBitrate / 250 + 1; // = averageBitrate*8/1000/2
int effectiveAudioPackagePerSecond = audioPackageSizePerSecond + pesOverhead;
int containerPackagesCountPerAudioPackagePerSecond = effectiveAudioPackagePerSecond / packageFill + 1;
int containerPackageSizePerAudioPackagePerSecond = packageSize * containerPackagesCountPerAudioPackagePerSecond;
audioOverhead = ac3Rawsize * containerPackageSizePerAudioPackagePerSecond / audioPackageSizePerSecond - ac3Rawsize;
qint64 hdOverhead = 0;
if (hdRawsize > 0) {
hdOverhead = hdRawsize * packageSize / packageFill + length * 1024 * 10 - hdRawsize;
}
audioOverhead += hdOverhead; would be as an approximation,.. (for ac3/thd/ac3+thd streams)
(Warning: not tested with any numbers, it's basically the ac3 formula with the hdOverhead part used in the dts-hd formula.)

Cu Selur

[Selur]

Could you recheck the numbers you posted?

Video9 : 263694 frames @23.976fps raw h264 encoded (x264) at around 33092kb. .264 file size : 45494178979 bytes.
Audio 9.0 (for video 9) : THD. File size : 1133934414 bytes.
Audio 9.1 (for video 9) : AC3 encoded at 448kb. AC3 file size : 615901440 bytes.

M2TS files :
(48) : Video9 + audio9.1 => 48232955904 bytes.
(49) : Video9 + audio9.1 + audio9.0&9.1 => 51437924352 bytes.
(49)-(48) => 16692544 x 192 for THD+AC3 track. (= 3204968448 bytes) would for ac3&thd result in:
audioRawSize: 1133934414 + 615901440 = 1 749 835 854 byte
subtracting the audioRawSize from the muxed m2ts we get a targetAudioOverhead of (3 204 968 448 - 1 749 835 854 = ) 1 455 132 594 byte
length = 10999 seconds
average bitrate = audioRawSize/length = 159 091 byte
since max ac3 bitrate is 640kbit/s = 640*1000/8 = 80000 byte/s
I assume the ac3 bitrate to be 80000 byte/s
and the hdbitrate to be 159 091 - 80 000 = 79091 byte/s
which leaves us with a hdrawsize of 79091*10999 = 869921909 byte;
with our ac3 overhead formula, we get an ac3 overhead of 172 694 579 byte
subtracting this overhead form the targetAudioOverhead we get a hdTargetOverhead of (1 455 132 594 - 172 694 579 = ) 1 282 438 015 byte
which is rather huge with more than 100% overhead,..
If the numbers are correct and I didn't make a mistake in the calcultion, I really can't recommend anyone to use trueHD audio inside a m2ts container.


Cu Selur

[jpsdr]

yes, ac3, dts and dts-hd seem to be fine. :)
(problem lies with: a. thd streams on their own and b. (ac3+thd) - combi - streams)

What i wanted to said is, that in case of an audio track wich is made with AC3+THD, computing the AC3 part size with the formula for AC3 only seems correct.
As i'm working only for Blu-Ray target, THD alone is a non-existant case for me.

And yes, values are correct, THD wastes space, and muxed stream seems somehow to follow some kind of constant fixed bitrate constrain.

I think it's not a good idea to try to work considering global size of THD+AC3, because as i've said, apparently AC3 part follow exactly the normal rule.

[jpsdr]

length = 10999 seconds
average bitrate = audioRawSize/length = 159 091 byte
since max ac3 bitrate is 640kbit/s = 640*1000/8 = 80000 byte/s
I assume the ac3 bitrate to be 80000 byte/s


Wrong. Haven't you read ? I said AC3 bitrate is 448kb.

[Selur]

No, I read what you wrote, but unless the ac3 bitrate is limited to 448kBit/s when combined with thd, I don't know for sure that the ac3 datarate really is X kBit/s, so I assume the maximum of 640kbit/s. (I suspect that this sloppiness will not much harm)

[jpsdr]

I think it's an error. Contrary to DTS-HD MA where lossless part is linked to lossy part, you can't decode the lossless part without the lossy part, AC3 and THD are totaly independant, you can decode the THD part without the AC3 part. The only reason there is AC3 with THD, it's because THD decoder is not mandatory for a Blu-Ray player manufacturer, so, there must be a format audio to "back-up" in case player is not able to decode THD track. AC3+THD track should be, for calculating space, view like 2 differents audio tracks. One with THD and one with AC3.
What can confirm this hypothesis it's that in authoring software, you enter the 2 files (the AC3 and THD), but only one for DTS-HD MA.
More, results in my previous post show that AC3 muxing size follow the standard rule.
Now, it's what i strongly believe, but have no concrete proof for it, and so, i still may be wrong...

[Selur]

So, now someone just needs to come up with a formula how to calculate the overhead (or resulting size) when muxing a thd file with lenght A and size B into a m2ts container.

Here's the data I extracted from previous posts:
Comment fcount fmult fdiv length raw size muxedsize targetOverhead avgBit AC3rawSize avgAC3 ac3Overhead hdRawSize targetHDOverhead
9.0+9.1 (2.0) 263694 24000 1001 10999 1749835854 3204968448 1455132594 159091 879920000 80000 172694579 869915854 1280601641
10.0+10.1 (2.0) 214220 24000 1001 8935 1348532682 2600792064 1252259382 150927 714800000 80000 140287850 633732682 1110479760
6.0+6.1 (5.1) 57925 24000 1001 2416 953002044 1356754944 403752900 394455 193280000 80000 37933457 759722044 365416072
5.0+5.1 (5.1) 63804 24000 1001 2662 989002438 1473159168 484156730 371527 212960000 80000 41795887 776042438 441916400 (ac3Overhead was calculated by: http://forum.doom9.org/showthread.php?p=1478668#post1478668, ac3rawSize was calculated using 80000byte/s*length)

Cu Selur

[jpsdr]

I dont have the same point of view.
I personnaly think datas are :

Comment fcount fmult fdiv length raw size muxedsize targetOverhead avgBit AC3rawSize avgAC3 ac3Overhead THDRawSize targetTHDOverhead
9.0+9.1 (2.0) 263694 24000 1001 10999 1749835854 3204968448 1455132594 159091 615901440 56000 43992960 1133934414 1411139634
10.0+10.1 (2.0) 214220 24000 1001 8935 1348532682 2600792064 1252259382 150927 500347904 56000 35739136 848184778 1216520246
6.0+6.1 (5.1) 57925 24000 1001 2416 953002044 1356754944 403752900 394455 193280000 80000 24160000 759722044 379592900
5.0+5.1 (5.1) 63804 24000 1001 2662 989002438 1473159168 484156730 371527 212894720 80000 26611840 776107718 457544890

If someone want to try and compare both point of view.

[Selur]

ah, okay, so only the 2.0 Streams are 448kBit/s,..

How did you calculate the ac3Overhead?
If I use:
# int audioPackageSizePerSecond = averageBitrate / 250 + 1;
= 56000/250+1 = 225
# int effectiveAudioPackagePerSecond = audioPackageSizePerSecond + pesOverhead;
= 225 + 17 = 242
# int containerPackagesCountPerAudioPackagePerSecond = effectiveAudioPackagePerSecond / packageFill + 1;
= 264/184+1 = 2
#int containerPackageSizePerAudioPackagePerSecond = packageSize * containerPackagesCountPerAudioPackagePerSecond;
= 192 * 2 = 384
# audioOverhead = ac3Rawsize * containerPackageSizePerAudioPackagePerSecond / audioPackageSizePerSecond - ac3Rawsize;
= 615901440 * 384 / 242 - 615901440 = 361396712
I end up with 361396712 but you use 43992960, why?

[jpsdr]

Lol... averageBitate should be in bit/s, not byte/s. :)
448000/250 = 1792
(1792+17)/184 =10 => 1920
615901440/1792 = 343695
343695*1920=659894400
659894400-615901440=43992960.
Same thing with 640kb. Ratio is 2880/2560.

[Selur]

I always use byte/s in my formulas :)

[jpsdr]

So you need to adapt coefficients then. It clear it's in b/s and not B/s in drmpeg post #52 (http://forum.doom9.org/showpost.php?p=1474557&postcount=52).

[Selur]

seems like there's something off with my current ac3 calculation,.. -> fixed it,..

Problem is we don't have enough data sets an knowledge about THD atm to solve this.
we got
length THDRawSize targetTHDOverhead
10999 1133934414 1411139634
8935 848184778 1216520246
2416 759722044 379592900
2662 776107718 457544890
but there are probably two or more case switches dependent on unknown restrictions that would need to be approximated,...
-> will thing about it again if we have a lot more different data sets

collected data: (ac3/trueHD/ac3&truehd)
http://spreadsheets.google.com/ccc?key=0AvWxUS1XGCPAdEZ3WGE4UE5JSUROSG12RWJPTWxUZnc

[jpsdr]

It seems that when you are doing (THDRaw+THDOverhead)/length you get roughly the same result, at least for the same kind of audio (2.0 or 5.1).
That's for now the most close thing we have.

[Stereodude]

I was shocked today when I muxed a DD THD + AC3 soundtrack + AVC video into a M2TS (with tsmuxer) and found it had way more overhead than an equivalently large DTS-MA audio stream. I have a rather crude formula that still gets me really close to my target size when using DTS-MA. However, with DD THD + AC3 audio and the same formula I ended up with BD25 that was 800MB too large. :(

[Selur]

Only Problem is the THD Overhead for the rest the formulas we got in this thread are relatively exact. ;)
Problem is no one here seems to know anything deeper about the thd audio so coming up with a formula hit a dead end.

[jpsdr]

I've put a rough estimation of THD in post #156 here (http://forum.doom9.org/showthread.php?p=1488847#post1488847).
Nevertheless, it's still rough. If like me you want to hit a precise result to fill your BD25 as much as possible, you can't avoid at least encoding twice your video...

[Stereodude]

If like me you want to hit a precise result to fill your BD25 as much as possible, you can't avoid at least encoding twice your video...Or three times if you manage to just barely overshoot the 2nd time.:o

[jpsdr]

Yes, when your final muxed result is 25kB too big to fit on BR, you cry...

[nm]

If like me you want to hit a precise result to fill your BD25 as much as possible, you can't avoid at least encoding twice your video...

Can't you mux audio with first pass video to check the overhead amount and then adjust second pass bitrate accordingly?

[jpsdr]

No, beacause :
- My encodes are running most of the time while i'm not at home, and, i prefer to have optimal result, that target bitrate is exactly the same during 1st and 2nd pass.
- I've noticed, in practical use, "huge" differences between final bitrate of 1st and 2nd pass, most of the time bitrate of 1st pass is lower, wich make size of 1st pass unrealiable to adjust accordingly, when you want to fit at around 10MB for BR25 and 20MB for BD50.

[nm]

No, beacause :
- My encodes are running most of the time while i'm not at home

Write a script.

and, i prefer to have optimal result, that target bitrate is exactly the same during 1st and 2nd pass.

That's really not necessary when you are only adjusting by some percents!

- I've noticed, in practical use, "huge" differences between final bitrate of 1st and 2nd pass, most of the time bitrate of 1st pass is lower, wich make size of 1st pass unrealiable to adjust accordingly, when you want to fit at around 10MB for BR25 and 20MB for BD50.

I guess you could adjust ratetol (http://mewiki.project357.com/wiki/X264_Settings#ratetol).

But worrying about a 0.1 % undershoot is pretty insane anyway, IMO.

[jpsdr]

Not at home, means : If i'm not home during 8h, and encoding take : 1h for 1st pass and 6h for 2nd pass, it will not change anything to stop after 1h or do the encode totaly, in both case, it will finish before my return.

But i'm insane, and using insane setting for encoding also. ;)

[Audionut]

But i'm insane, and using insane setting for encoding also. ;)

Ok, getting a dual-layer dvd movie that only uses 5gb is pretty crap. But seriously, .05%.:confused::eek:

Encode something at 10000kbit/s, then the same material at 9950kbit/s. See is you can spot the difference. And that is .5%. 10x greater than what you are shooting for.

[Stereodude]

Ok, getting a dual-layer dvd movie that only uses 5gb is pretty crap. But seriously, .05%.:confused::eek:

Encode something at 10000kbit/s, then the same material at 9950kbit/s. See is you can spot the difference. And that is .5%. 10x greater than what you are shooting for.Well, I'm more curious to know what sort of PC he has that he can encode a full movie at "insane" settings two passes in 8 hours. My i7-2600k at 4.2gHz with a full x64 flow takes about 12 hours for 2 passes at very slow.

[Audionut]

My i7-2600k at 4.2gHz with a full x64 flow takes about 12 hours for 2 passes at very slow.

Yeah, he does seem to be getting some good speeds for 'insane' settings.
Also, I don't encode 2 pass. But it is my understanding that there isn't a 6:1 ratio between passes either.

[Stereodude]

Yeah, he does seem to be getting some good speeds for 'insane' settings.
Also, I don't encode 2 pass. But it is my understanding that there isn't a 6:1 ratio between passes either.Unless you use --slow-firstpass the first pass is much faster than the second pass especially at the slower presets. On my system it's roughly 10:1. However, --placebo invokes --slow-firstpass which makes the first pass take as long as the second pass.

I guess we'll have to wait to see the command line he's using...

[Selur]

would be nice if you could keep the whole "how to tune my command line" out of this thread,...

thanks

[jpsdr]

I have an i7@980.
Don't have here my commande line, but i use placebo, and ajust a little the 1rst pass. Take me several tries to be sure to have the exact same numbers of I,B and P frame between passes and the overall parameters having globaly the same value. Speed ratio is around 4 between passes, and take around 24h for a 2h 1080p video, and 8h for a 720p video.

You can find my 1080p commande line here (http://forum.doom9.org/showthread.php?p=1459705#post1459705).
For 480p upscaled to 720p, it's the same but with level 4.0, 6 ref frames, 48 keyint, no slices and bitrate/buffer max at 15000.
All my workflow is also x64 (lossless codec x64 (UT Video) + Avisynth x64 + x264 x64).

[radius]

Hi guys, I'm trying to figure out the same thd overhead calculation, did you guys ended with some formula / tool for this :) ?

[Selur]

nope, not enough statistical data to even think about a decent formula without knowing some basics about thd compression.

[Stereodude]

Probably due partly to the fact that THD is pretty rare on Blu-Ray discs. DTS-MA is much much more common.

[Selur]

+ no one knowns any specification
+ no reasonable priced encoder is available

[jpsdr]

As i've said, i've put a rough estimation in post #156 here (http://forum.doom9.org/showthread.php?p=1488847#post1488847).
The few datas i've got give me the feeling that it may be impossible to have an very accurate estimation, because final size may be related too much to actual data. But it's only a guess.

[Stereodude]

FWIW, I had a chance to play with Scenarist a little. It's m2ts files are slightly larger than tsMuxeR's. Like 20MB over a full "25GB" Blu-Ray.

[Selur]

In general? With what content (steam types) ?
at the moment I roughly approximate the tsMuxeR Overhead by int(lengthInSeconds + 1) * 20.9 * 192 (byte) there might be a similar way to approximate the overhead Scenarist uses.

[Stereodude]

In general? With what content (steam types) ?
at the moment I roughly approximate the tsMuxeR Overhead by int(lengthInSeconds + 1) * 20.9 * 192 (byte) there might be a similar way to approximate the overhead Scenarist uses.Well, I only had a relatively short time to play with it, but on the 3 "movie only" discs I mux'd of AVC (H.264) + 5.1 DTS-MA (no subtitles) they were bigger.

[jpsdr]

For information, all the datas i've provided were muxed with Scenarist.

[Selur]

strange, the formulas did hold up when just leaving out the tsmuxerOverhead,..