MKVToolNix v24.0.0 released

[Selur]

Thanks to sneaker2 I learned that it works fine if I extract all the streams I want to merge to raw streams beforehand.

"G:\Test\mkvmerge.exe" -o "D:\Test\output\test.mkv" "D:\Test\temp\test-001_reencode.264" + "D:\Test\temp\test-002.mkv" + "D:\Test\temp\test-003_reencode.264" "D:\Test\temp\test_AudioCut.mkv"
mkvmerge is complaining and no output is generated, or sometimes a broken output

"G:\Test\mkvmerge.exe" -o "D:\Test\output\test.mkv" "D:\Test\temp\test-001_reencode.264" + "D:\Test\temp\test-002.264" + "D:\Test\temp\test-003_reencode.264" "D:\Test\temp\test_AudioCut.mkv"
mkvmerge is complaining but the output plays fine

-> is there some magic option to tell mkvmerge to handle the videostream in the mkv container as if they were raw streams? (to avoid having to demux even the stream parts I didn't reencode)

Cu Selur

[Mosu]

-> is there some magic option to tell mkvmerge to handle the videostream in the mkv container as if they were raw streams? (to avoid having to demux even the stream parts I didn't reencode)

Nope, there isn't.

[Selur]

Nope, there isn't.
-> new feature request. (https://trac.bunkus.org/ticket/798) :D

[Mosu]

Won't be done by me, sorry. You're quite welcome to contribute patches that implement such functionality.

[Selur]

I added it to the bugtracker, so I can find if I run into the problem again :)

[sneaker_ger]

What else does mkvmerge change?

Here's a sample:
https://rapidshare.com/files/384744953/iterations.7z

I muxed and demuxed a source a few times and all resulting raws differed from each other. ( source.h264 -> 1.mkv -> 1.h264 -> 2.mkv -> 2.h264 -> .... -> 5.h264 )

Since it just came up in a discussion somewhere else again, I thought I'd ask if there has been any change on this?

Here's what is added after an iteration of mkvmerge -> mkvextract:

http://www.abload.de/img/differenceb0pc0.png

Sample:
https://rapidshare.com/files/1548641564/mkvtoolnix_h264_muxing_demuxing_bug.7z

[Mosu]

No, there hasn't, and as Matroska is not supposed to be used (or thought of) a lossless container like ZIP or RAR I will most likely never spend time working on this.

[sneaker_ger]

I understand, but could you tell me what the marked part actually is? It is a bit unsettling that every iteration adds new bytes when you have no idea why and what.

[Mosu]

During extraction mkvextract copies the content of the CodecPrivate (the SPS, PPS etc stuff) into NALUs. However, that information is not removed during muxing, hence you get that content added nevertheless. It's also perfectly fine to repeat SPS/PPS information in the raw bitstream, so _playback_ should be bit-identical in each case.

[sneaker_ger]

Yes, now I see that it is just being repeated over and over.
And what is getting removed during muxing? All SPS/PPS from all GOPs? Something else? Basically Blu-Ray compatiblity is lost after muxing h.264 ES to mkv and now there was some discussion about what is exactly being removed/what would have to be restored/recalculated in order to get the original file back.

[Mosu]

A bit of historical context. Initially mkvmerge removed all SPS/PPS NALUs and only created them in the CodecPrivate element in the same format they're stored in MP4 in as well: as an AVCC. During extraction mkvextract converted CodecPrivate content (the 'AVCC') back to SPS/PPS NALUs exactly once at the beginning of the stream. This was producing bit-identical content before merging and after extraction if and only if the source stream contained exactly one occurrence of the SPS/PPS NALUs.

Of course this immediately failed with streams whose SPS/PPS parameters changed in the middle of the stream which often happens after intros etc.

So the current scheme is to convert the first SPS/PPS NALUs found to the AVCC but to leave all SPS/PPS (to be more precise: their Matroska bitstream representation; those are not NALUs but called something else which I don't recall -- same as in MP4. Not prefixed with 00 00 00 01 but with a size field etc) intact in the blocks inside the Matroska file. During extraction mkvextract converts the AVCC back to the SPS/PPS NALUs as long as it doesn't encounter SPS/PPS frames that differ from the ones in CodecPrivate (or something similar to that algorithm). Hence not bit identical before and after merging, but way more compatible.

This

[sneaker_ger]

What happens to things like HRD, pic-struct and AUD?

[Mosu]

As they're not part of the AVCC they receive to special treatment. They should be kept where they are in the bitstream.

[sneaker_ger]

So they are not removed? Thx.

[Mosu]

mkvmerge does not throw anything away from the h.264 bitstream.

Edit: Oh, except for filler NALUs. Just like stuffing bytes from MPEG2 bitstreams and other such space-wasting garbage.

[sneaker_ger]

Edit: Oh, except for filler NALUs. Just like stuffing bytes from MPEG2 bitstreams and other such space-wasting garbage.

That would be the stuff x264 uses when specifying --nal-hrd cbr?

[Mosu]

I have no frigging clue. Why don't you ask the x264 developpers? :)

[Helios61]

Hi community!

While trying to mux a mkv movie with ordered chapters i get the following error message in mkvmerge 5.8.0:

mkvmerge v5.8.0 ('No Sleep / Pillow') built on Sep 2 2012 15:37:04

Error: The XML segmentinfo file 'E:\tags.xml' contains an error: The root element must be <Info>.

This is the related tags.xml ->
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE Tags SYSTEM "matroskatags.dtd">
<Tags>
<Tag>
<Targets>
<EditionUID>640704807</EditionUID>
</Targets>
<Simple>
<Name>TITLE</Name>
<String>Kinofassung</String>
</Simple>
</Tag>
<Tag>
<Targets>
<EditionUID>2119351024</EditionUID>
</Targets>
<Simple>
<Name>TITLE</Name>
<String>Directors Cut</String>
</Simple>
</Tag>
</Tags>

Could anybody please help?

Thanks in advance

Helios

[Mosu]

You seem to be severely confused. You're mixing up three things:

1. You're talking about ordered chapters.
2. The file you're using contains tags, not chapters.
3. You're passing that file to the --segmentinfo option which has nothing to do with either of those two things.

[Helios61]

Hi Mosu!
Big Thanks! You are fully right. I've entered the tag file in segment field of mkvmerge GUI! Maybe the last beer last night (Uuups)!

Again Thanks
Helios

[zeropc]

any chance to finally add the option to keep embedded ac3 from dolby true-hd?

btw, is it possible to add pictures into the mkv file like movie posters? i couldn't find a definite answer to that.

[Mosu]

No, sorry. See https://www.bunkus.org/answers/?qa=25/how-does-mkvmerge-handle-truehd-tracks-with-embedded-ac3-data

[nevcairiel]

any chance to finally add the option to keep embedded ac3 from dolby true-hd?

That would simply be wrong (and most likely break playback with a lot of things). You should never mux two individual tracks into one mkv track. Simply mux the two into two tracks, and you have the choice which to use for playback.

[Selur]

1st Thanks again for the advanced --split options. :) (I really like '--split parts'.)
2nd feature request: would be nice if mkvmerge could also extract the time codes or if mkvextract would also get a --split parts option like mkvmerge
background: I'm playing around with a frame accurate cutting application based on mkvmerge&co and atm. the only way to handle time codes atm. would be to a. extract them and then b. cut them.
Video cutting on I-Frames, reencoding parts of the gops that get modified and cutting the audio works excellent with mkvmerge.
Time codes get problematic, since afaik mkvextract doesn't allow to only extract the time codes of specific parts,..

Cu Selur

[Mosu]

Sorry, but that is such a niche request that I won't spend time on it.

[Selur]

Okay,... :( But thanks for the reply :D

[TheRancher]

Mosu, check this topic (http://forum.doom9.org/showthread.php?t=166592) out. It seems the H.264 bitstream is broken and hence MKVExtract cannot extract it. Can you try to fix it? I've tried many GUI applications and non of them seem to work. Thanks.

[Mosu]

sneaker_ger was right in his assumption: I'm not spending time on trying to fix broken h.264 bitstreams. Sorry.

[73ChargerFan]

Feature request:

.thd+ac3 stream adds only the TrueHD stream and discards the AC3 stream. Please add both streams (as separate, e.g. #2 is TrueHD, #3 is AC3) then let me deselect one if I want.

I'm not asking for a combined stream, which you've rejected, but instead to be able to add the AC3 stream from a .thd+ac3 file.

[Mosu]

Will most likely not happen. I've looked into this in the past, and the effort required didn't seem worth the gain. Often legitimate sources of .thd+ac3 files also contain the raw ac3 file in a separate track. And there are other ways to separate .thd+ac3 into something else; eac3to if I'm not mistaken.

[[ReX]]

Hey Mosu, I was testing the ability to add attachments with mkvpropedit and I noticed something weird happens when you chain the command to add attachments with another command - like changing the name of the first audio stream - and open the file in mmg, go to the Attachments tab, the list is empty (even the attachments that were already there disappear), even though the new attachment was added successfully and works as expected.

Something like this:
mkvpropedit movie.mkv --add-attachment Arial.ttf --edit track:a1 --set name="English Audio"

Changing the order of the commands doesn't have any effect.

[Mosu]

Sounds like a bug. I'll look into it.

[Mosu]

Quick heads up: it's a bug in mkvmerge, not in mkvpropedit. The changed file is OK, mkvmerge just doesn't find the attachments it should.

[Mosu]

Fixed pre-build: http://www.bunkus.org/videotools/mkvtoolnix/win32/pre/mkvtoolnix-unicode-5.8.0-build20121202-474-setup.exe

[DragonQ]

Some of my TS files seem to be breaking when running them through MKV Merge. The original TS files play fine but the resulting MKVs aren't deinterlaced properly by MadVR and EVR has issues too (deinterlacing seems to work but the FPS says 46-48 instead of 50, the playback is jerky and the graph is dodgy also).

Original TS - Sample Clip (http://www.mediafire.com/?1ii1q1nhnlhm6zs)
Dodgy MKV - Sample Clip (http://www.mediafire.com/?ho71875bvmy8m5d)

Original TS - EVR Stats (http://www.aotplaza.com/Files/HTPC/Screengrabs/Dodgy%2050i%20MKV%20Merge/TS.png)
Dodgy MKV - EVR Stats (http://www.aotplaza.com/Files/HTPC/Screengrabs/Dodgy%2050i%20MKV%20Merge/MKV.png)

Any ideas?

Six weeks and no reply? :confused:

[Mosu]

It's a playback issue; I chose not to lend a hand with such issues.

[DragonQ]

Last time something similar happened it turned out to be an error in how MKVMerge dealt with AC3. :/

It seems to only happen with streams from that channel (ITV1 HD). Streams from other channels don't have the same issue.

EDIT: However, it seems that all new MKVs fail to get auto-detected as interlaced by MadVR. My old MKVs aren't affected by this so I'm guessing something broke in a fairly recent version of MKVMerge, or maybe in TS-Doctor. Or maybe it's due to my switch to Windows 8 during that time...

[Mosu]

Get Madshi to look into it; if he can tell me why detection fails I might spend time on it.

[DragonQ]

Will do!

[[ReX]]

Thanks Mosu, pre-build works fine now.
But MKVExtractGUI-2 still can't list the attachments, I think it uses mkvinfo and mkvinfo has the same bug as mkvmerge,.

[Mosu]

mkvinfo shows the attachments just fine. But if mkvinfo is used in that capacity then he's simply using the wrong tool for the job. He's trying to drive in a nail with a screwdriver (mkvinfo) while ignoring the perfectly good hammer (mkvmerge) lying next to it in the tool box. Yes, it is possible to get nearly the same job done with that screwdriver, but it takes more patience, has a lot more potential for injury, simply because the tool was not designed for that particular purpose.

GUIs wanting to call mkvmerge / mkvextract /mkvpropedit should always get their information from "mkvmerge --identify" (or "mkvmerge --identify-verbose"). It has always been this way.

[[ReX]]

Yeah I think it's actually a different problem with mkvinfo, it took me a while but I remembered what I did, now I'm able to reproduce it.
For this test movie.mkv will be a Matroska file with a single audio and video stream and Arial muxed as arial.ttf.

Now run this command on it:
mkvpropedit movie.mkv --add-attachment verdana.ttf --attachment-name arialbd.ttf --replace-attachment name:arial.ttf:arialbd.ttf

Now mkvinfo can't see any attachments, I say it's a different problem because even v5.8.0 mkvmerge without the fix can see them.

[Mosu]

mkvinfo can see the attachments just fine. However, mkvpropedit moves to the attachments to the end of the file, and that's why you don't see them in mkvinfo's output with default settings. Why? Easy. mkvinfo parses the file from front to back, so seeking involved whatsoever (that's its purpose). In order not to overwhelm the user with information mkvinfo stops when it finds the first cluster by default. If the attachments are located behind the cluster then mkvinfo will not print them by default.

Adding one or more "-v" (or "--verbose") will cause mkvinfo to parse the whole file. That way you'll see the attachments near the end.

There is no problem with mkvinfo!

[[ReX]]

Interesting...
Sorry for bothering you and thanks for the information!

[Mosu]

No problem at all.

[DragonQ]

Get Madshi to look into it; if he can tell me why detection fails I might spend time on it.
I've asked him to take a look. Some more info that might help in the mean time:

1) All TS files with AAC audio:
- EVR: TS plays fine, MKV plays fine.
- MadVR: TS plays fine, MKV is treated as progressive so not deinterlaced.

2) Older TS files (e.g. from 10/11/2012) with AC3 audio:
- EVR: TS plays fine, MKV plays fine.
- MadVR: TS plays fine, MKV plays fine.

3) Newer TS files (e.g. from 02/12/2012) with AC3 audio:
- EVR: TS plays fine, MKV plays slowly (46-48 fps) with playback speed jumping all over the place and a really dodgy graph.
- MadVR: TS plays fine, MKV is treated as progressive so not deinterlaced and also playback speed is all over the place.


Looks like two separate issues: one with AAC audio not being detected as interlaced at all (possibly MadVR fault), one with newer files with AC3 audio not having correct timestamps. For the latter, maybe something has changed in the broadcast streams that has broken MKVMerge compatibility?

[Keiyakusha]

Does zlib, bz2, lzo are the only possible compresion options for pgs subs or in theory it can use something else?
because out of these three zlib seems to do best compression, yet it is still 15 times worse than 7z. it is even 2 times worse than good old zip. By OCRing subs or by not including them in container I can save gigabytes of space...

[nevcairiel]

zlib is the only compression still valid in Matroska, bz2 and lzo have been removed from the spec, and will most likely not work properly with most players. And no, you cannot use any other compression.
Also note that compression is done on a per-frame basis, so compression always works rather limited because you only get small packets to work with, and not the whole stream in one go.

If your subtitles are a big space concern for you, something seems wrong. They should be a rather minimal factor in a video file.

[Keiyakusha]

zlib is the only compression still valid in Matroska, bz2 and lzo have been removed from the spec, and will most likely not work properly with most players. And no, you cannot use any other compression.

If your subtitles are a big space concern for you, something seems wrong. They should be a rather minimal factor in a video file.

I see, thanks.
As for my concerns. It maybe sounds like it suddenly become an issue for me, no I was just curious. Actually everything is a space concern for me as long as there is chances to compress something better without sacrificing quality. I never use reference flac encoder for example (too slow, too low compression compared to alternatives) ^__^

Edit: well I wouldn't call 50MB per episode (zlib-compressed) a "minimal factor" (2 sub tracks). I can compress the whole episode to less than 100MB in a way it will be watchable.

[Mosu]

Does zlib, bz2, lzo are the only possible compresion options for pgs subs or in theory it can use something else?
because out of these three zlib seems to do best compression, yet it is still 15 times worse than 7z. it is even 2 times worse than good old zip. By OCRing subs or by not including them in container I can save gigabytes of space...

At the moment only zlib, bz2, lzo and header removal are compression schemes that are (or were) official part of the Matroska specs (bz2 and lzo have since been removed/retired). Of those bz2 and lzo are not widely supported, and I would be somewhat surprised if significantly more programs than mkvmerge were able to read (let alone write) such files.

You cannot simply use any compression scheme you want, even if 7z seems to be better (7z is also a program; the compression scheme to use would be lzma which 7z uses internally and which is a stream compressor format -- meaning 7z itself is a combination of a packaging format like tar and a compression scheme like lzma; for Matroska we only need stream compressors, not package formats).