eac3to - audio conversion tool

[xkodi]

"mkvextract --raw" and "mkvextract --fullraw" don't make any different for this particular fragment at the beginning. comparing 20MB after the fragment at the beginning shows that:

- the different between raw_1 and raw_2 is missing from time to time the following sequence:

0CFF[FF]800000000109xx00000001

which is something else + AUD.

- the different between raw_2 and raw_3 is missing from time to time the following sequence:

780A

@sehgal.v7

yes, i agree that such behavior is not what i'm expecting from container format.

[sehgal.v7]

@xKodi
Wait, i'm posting a sample in mkvtoolnix author thread, let's see thr reply.

[madshi]

Originally Posted by sehgal.v7

i spent two whole days working on mkv & h.264 & vc-1, few wks back// and i came to conclusion that once raw VC1 or H.264 went to MKV, u can NEVA get to see exact original size. No PERFECT container for timebeing.

I disagree about VC-1.

Originally Posted by xkodi

- the different between raw_1 and raw_2 is missing from time to time the following sequence:

0CFF[FF]800000000109xx00000001

which is something else + AUD.

Are you sure about that? Can you upload those 20MB for testing purposes? The only difference should be the AUD. And eventually a stripped sequence end code.

Originally Posted by xkodi

- the different between raw_2 and raw_3 is missing from time to time the following sequence

Well, that's your own fault, if you use mkvmerge instead of eac3to...

I'm in contact with Mosu. He seems to consider adding AUDs to mkvextract. Maybe he'll also fix the double sequence headers at the beginning of the stream. That should give us bit perfection. Except this "something else" which we need to identify before we can talk about what to do with it.

[xkodi]

Originally Posted by madshi

I disagree about VC-1.

VC-1 doesn't work here too. i'm doing the same test as with h264. also, will try with MPEG2.

Originally Posted by madshi

Are you sure about that? Can you upload those 20MB for testing purposes? The only difference should be the AUD. And eventually a stripped sequence end code.

eac3to file.m2ts raw_1.h264:

http://rapidshare.de/files/40296284/...20mb.h264.html

eac3to file.m2ts file.mkv
mkvextract file.mkv to raw_2.h264:

http://rapidshare.de/files/40296287/...20mb.h264.html

Originally Posted by madshi

Well, that's your own fault, if you use mkvmerge instead of eac3to...

i'm not aware of any other tool that allows me to merge the video, audio and subtitle streams together.

[edit]

eac3to file.vob raw_1.m2v:

eac3to file.vob file.mkv
mkvextract file.mkv to raw_2.m2v:

compare raw_1.m2v and raw_2.m2v:


so, overall result is that currently merge/extract video to/from MKV is not bit-perfect for VC1, H264 and MPEG2.

[madshi]

Originally Posted by xkodi

VC-1 doesn't work here too.

Sample?

Originally Posted by xkodi

i'm not aware of any other tool that allows me to merge the video, audio and subtitle streams together.

You should use eac3to for muxing the video to MKV. Then you can use mkvtoolnix to add audio and subtitle streams. mkvtoolnix will not damage the video track muxed by eac3to. But if you use mkvtoolnix for muxing a raw video stream you'll have to live with results which are not as good as when using eac3to, sometimes.

Originally Posted by xkodi

eac3to file.vob raw_1.m2v

Sample?

P.S: Please use "--raw" for MPEG2 demuxing. I think at least with MPEG2 there's a big difference.

Originally Posted by xkodi

so, overall result is that currently merge/extract video to/from MKV is not bit-perfect for VC1, H264 and MPEG2.

You should not use mkvmerge + mkvextract. You should use eac3to + "mkvextract --raw". That results in bit perfect results for VC-1 and MPEG2, as far as I remember. Eventually the last video frame will be lost, but that's about it.

[sehgal.v7]

@Madshi
Test file hxxp://rapidshare.com/files/139290511/Test.ts - Demuxing them.
Original size - 9,400,000 (50000x188)

6,750,422 Video by Eac3to.h264
6,770,832 Video by TSMuxer.264
6,770,832 Video by TSRemux.h264

2,243,576 Audio by Eac3to.dtsma
2,244,847 Audio by TSMuxer.dtsma
2,244,847 Audio by TSRemux.dtsma

[madshi]

Originally Posted by xkodi

- the different between raw_1 and raw_2 is missing from time to time the following sequence:

0CFF[FF]800000000109xx00000001

That's filler data and AUD. I already explained the AUD. The filler data is just there to achieve CBR or some other stupid aims. So there's no reason to leave it in the stream. It's like zero padded DTS files. No reason really to keep it in the stream. That's why it gets removed by the MKV muxing process.

[madshi]

Originally Posted by sehgal.v7

@Madshi
Test file hxxp://rapidshare.com/files/139290511/Test.ts - Demuxing them.
Original size - 9,400,000 (50000x188)

6,750,422 Video by Eac3to.h264
6,770,832 Video by TSMuxer.264
6,770,832 Video by TSRemux.h264

2,243,576 Audio by Eac3to.dtsma
2,244,847 Audio by TSMuxer.dtsma
2,244,847 Audio by TSRemux.dtsma

eac3to only outputs frames for which it's sure that they are complete. If eac3to cannot be sure that the last video or audio frame is complete it skips them. That's why video and audio tracks demuxed by eac3to may be one frame short compared to other demuxers. An incomplete audio or video frame could result in rainbow frames or even decoder crashes...

[xkodi]

Originally Posted by madshi

Sample?
P.S: Please use "--raw" for MPEG2 demuxing. I think at least with MPEG2 there's a big difference.

"--raw" doesn't make any difference.

MPEG2 sample:
http://rapidshare.de/files/40296979/VTS_02_1.VOB.html

screenshot from the comparison:

Originally Posted by madshi

No reason really to keep it in the stream. That's why it gets removed by the MKV muxing process.

i really can't get why destroying the original stream by removing data even if they are useless. the only reason that i can think of is to save space, but with removing things like AUD you can't save space anyway.

[nautilus7]

Originally Posted by xkodi

i really can't get why destroying the original stream by removing data even if they are useless. the only reason that i can think of is to save space, but with removing things like AUD you can't save space anyway.

I don't believe that eac3to would do something harmful to our data.
And I like removing useless things from my data. I want them as clean as possible, but that's my opinion.

[madshi]

Originally Posted by xkodi

i really can't get why destroying the original stream by removing data even if they are useless.

Filler data is by design and by h264 specification useless for decoders. So why keep it in the stream? Removing it from the stream is in no way "destroying" the stream. It's more like "cleaning up the stream". I don't see you complaining about eac3to removing the zero padding of DTS streams. So why are you complaining about removing useless filler data from h264 streams? The only effect it has is saving space. To my best knowledge there is no other (positive or negative) effect of removing the filler data.

Originally Posted by xkodi

with removing things like AUD you can't save space anyway.

Look, with VOB and m2ts containers the video data is stored as one continuous stream of bytes. The AUDs are in the h264 streams cause they make it easier to separate frames. But MKV works very differently. It stores each frame separately. So storing the AUDs in MKV doesn't make any sense. I wouldn't even know where to store them! Should I put them in front of each frame or behind? IMO the software which demuxes a h264 stream from an MKV should put the AUDs back in. That's really easy to do. Now unfortunately Mosu doesn't seem to like the idea. But that's a different topic.

[madshi]

Originally Posted by xkodi

"--raw" doesn't make any difference.

I've tested your MPEG2 sample and I get 100% bit perfection when I use "--raw". Caution: You need to put the "--raw" parameter at the right place in the command line! Or else it will have no effect. The right command line would be this:

"mkvextract tracks sample.mkv --raw 1:test.m2v"

[xkodi]

Originally Posted by madshi

"mkvextract tracks sample.mkv --raw 1:test.m2v"

my command was "mkvextract tracks sample.mkv 1:test.m2v --raw"

after put the "--raw" at the right place it is bit-perfect here too.

so, indeed with MPEG2 everything is OK. thank you for you help!

Originally Posted by madshi

Filler data is by design and by h264 specification useless for decoders.

thank you for explaining this, then it is save to remove those "filler data".

what about AUDs - are decoders OK with raw h264 stream without AUDs?

[Roscoe62]

Originally Posted by madshi

So that means that one of the other m2ts files made problems. Most probably if you have the original unmodified folder (with which the problem occurred), eac3to tried to join two (or more) m2ts parts which didn't really belong together. So my suggestion would be to retry with the original folder. Please post the eac3to log when doing "eac3to c:\whatever\mrmrssmith 1)". I think you'll find that there are at least 2 m2ts parts listed by eac3to, or maybe even more...

Well, yes and no. I posted the log already - you'll find it on the last page and it makes no mention of which m2ts files it uses.

However, when I ran the usual commands again, the step prior to that recorded in the log clearly shows that it is adding 00001.m2ts and 00002.m2ts together. Looking at the stream list the 00002.m2ts file is only 54kb in size, and is not required at all. The only thing it does is prevent eac3to from doing it's job properly.

Let me know if there's anything further I can do to help.

[madshi]

Originally Posted by xkodi

my command was "mkvextract tracks sample.mkv 1:test.m2v --raw"

after put the "--raw" at the right place it is bit-perfect here too.

so, indeed with MPEG2 everything is OK.

I hope you can confirm the same bit perfectness for VC-1?

Originally Posted by xkodi

thank you for explaining this, then it is save to remove those "filler data".

I really think so.

Originally Posted by xkodi

what about AUDs - are decoders OK with raw h264 stream without AUDs?

Decoders don't mind at all. If they minded, no h264 MKV or MP4 file would play because both containers don't store AUDs. And the default output of the x264 encoder doesn't contain any AUDs, either, as far as I know. The only containers which use AUDs by default are VOB/EVO and TS/M2TS, I believe. And I think even the VOB/EVO and TS/M2TS splitters remove the AUDs before sending the data to the decoder.

However, eac3to depends on h264 files containing AUDs. Without them eac3to doesn't properly handle h264 files. So basically if you mux a raw h264 stream to MKV, then demux it again, eac3to will not handle it properly, anymore. In the long run I need to find a way to solve this problem. Maybe I'll make eac3to not depend on AUDs, anymore. Or maybe I'll find another way. Not sure...

[madshi]

Originally Posted by Roscoe62

Well, yes and no. I posted the log already - you'll find it on the last page and it makes no mention of which m2ts files it uses.

However, when I ran the usual commands again, the step prior to that recorded in the log clearly shows that it is adding 00001.m2ts and 00002.m2ts together. Looking at the stream list the 00002.m2ts file is only 54kb in size, and is not required at all. The only thing it does is prevent eac3to from doing it's job properly.

Let me know if there's anything further I can do to help.

Ah, that's interesting. Can you please upload that 54kb file? Also I need a 50MB sample of 00001.m2ts again (the original upload you provided is not available, anymore, and I deleted the file already). Thanks!

[EPiPH0NE]

Originally Posted by Roscoe62

Well, yes and no. I posted the log already - you'll find it on the last page and it makes no mention of which m2ts files it uses.

However, when I ran the usual commands again, the step prior to that recorded in the log clearly shows that it is adding 00001.m2ts and 00002.m2ts together. Looking at the stream list the 00002.m2ts file is only 54kb in size, and is not required at all. The only thing it does is prevent eac3to from doing it's job properly.

Let me know if there's anything further I can do to help.

Yeah, once I went back in just used the main M2TS file for my source input instead of my BD-rom, Mr and Mrs Smith ran through fine. Thanks for the tip

[Roscoe62]

Madshi,

I've now uploaded the Mr & Mrs Smith files for you.

This zip file has both a 50Mb sample of 00001.m2ts and also the 00002.m2ts file.

[sehgal.v7]

@Madshi
VC-1 Sample for your study, hxxp://rapidshare.com/files/139429429/Video.ts
Original TS File Size - 5,220,384
4,948,722 Demuxed Video by Eac3to.vc1
4,933,798 Extracted Video by mkvextract.vc1
4,927,251 Muxed by Eac3to.mkv
Not Exact VC1 File.

Regards,
Sehgal

[madshi]

Originally Posted by sehgal.v7

@Madshi
VC-1 Sample for your study, hxxp://rapidshare.com/files/139429429/Video.ts
Original TS File Size - 5,220,384
4,948,722 Demuxed Video by Eac3to.vc1
4,933,798 Extracted Video by mkvextract.vc1
4,927,251 Muxed by Eac3to.mkv
Not Exact VC1 File.

As I said before, the file size alone is not a good indicator. eac3to sometimes drop the last frame which can result in a smaller file size. But that doesn't mean that bit perfection isn't reached.