MKVToolNix v24.0.0 released

[Mosu]

I don't know a lot about CCs, and to be honest being a European I don't particularly care. I do know that they're not simple text that could be converted easily, but that's about all, and that's also one of the reasons why I haven't really bothered looking into it.

Like I said, don't hold your breath.

So Apple uses 608 captions intead of the standardized timed text in MP4 (http://en.wikipedia.org/wiki/MPEG-4_Part_17)? This sounds so ridiculous it isn't even funny anymore.

[ndjamena]

I'm not exactly Pro Apple (I am trying to get them OUT of the m4v, after all), but that doesn't change the fact that these are the best quality versions I can find (I've already wasted a lot of money on DVDs that are now redundant because I didn't want to have to go through iTunes) and I have to deal with these files somehow.

(Both tracks 7 and 8 are listed in each of the other tracks' "chap" headers. Finding a way to add a thumbnail to each chapter is NOT something I'm going to condemn them for, I just wish they didn't have to add them as tracks (and it makes reusing the images without duplicating them impossible).)

[stax76]

I've a MakeMKV created file where MediaInfo shows iw as language code for a vobsub subtitle, when I demux it with mkvextract it's en in the idx and not iw or he, on the ifo file MediaInfo shows he, to MPC-BE iw is unknown too.

[Mosu]

I'd need access to that Matroska file if you want me to comment.

[stax76]

it should be on your ftp in a minute or two.

[Mosu]

Thanks.

This is a bug in MakeMKV. Matroska uses ISO 639-2 language codes (http://www.loc.gov/standards/iso639-2/php/code_list.php) in all of its elements that indicate a language. However, MakeMKV has written »iw« into the »track language« element. While »iw« is a valid ISO 639-1 language code it is not a valid ISO 639-2 language code.

During extraction mkvextract validates that language element, finds it to be invalid and discards it. After discarding it the »track language« is treated the same way as if it were absent from the file, which means that its default value (»eng«) is assumed. Then mkvextract writes the IDX for the VobSub and translates the ISO 639-2 code »eng« into the ISO 639-1 code »en« (VobSub indexes use ISO 639-1 language codes), and that's why you see »en« in the resulting .idx file.

As this has happened in the past mkvmerge corrects such entries automatically during a re-mux: it mapps the ISO 639-1 codes to their corresponding ISO 639-2 codes. Therefore the re-muxed file's track language is set to »heb«. If you extract the VobSubs from that re-muxed file then your .idx file will finally contain »language: he«. But like I said: the original bug is in MakeMKV, not mkvextract.

Edit: fixed information about mkvmerge.

[stax76]

Thanks for looking into it, I reported it to mike. In the past I always ripped German and English only, now I'm always ripping everything to test StaxRip. I map absence to undetermined, it seems more logical to me.

[Mosu]

Mapping absence to »und« is simply wrong. See the specs (http://www.matroska.org/technical/specs/index.html). Yes, »und« should have been the default and not »eng«, but the default value is »eng«, and if you treat absence as anything other than »eng« then your application is buggy and doesn't act like all the other Matroska-supporting applications do.

[stax76]

You are right, sorry for not making a clear statement, I use MediaInfo for everything, it was meant for non Matroska standalone audio files for instance like AC3 or DTS.

[Mosu]

Ah right. That makes perfect sense.

[sneaker_ger]

During extraction mkvextract validates that language element, finds it to be invalid and discards it. After discarding it the »track language« is treated the same way as if it were absent from the file, which means that its default value (»eng«) is assumed. Then mkvextract writes the IDX for the VobSub and translates the ISO 639-2 code »eng« into the ISO 639-1 code »en« (VobSub indexes use ISO 639-1 language codes), and that's why you see »en« in the resulting .idx file.
Mkvextract does this even for mkv tracks that are correctly marked as "und". I tried to search for correct "und" equivalent in idx files but it's not easy to find. In guliverkli source it says "--" but BDSup2Sub does not detect it. There doesn't seem to be a broadly compatible way to do this.

[Mosu]

Thanks for pointing it out, sneaker_ger. I'm not sure whether this behavior is intentional or accidental. I might consider changing it either writing »language: --« or not writing a language line at all if there's interest in such a change.

[ndjamena]

The default value would have been chosen for compression purposes (ie "eng" would likely be used more often than any other language code), have we now decided that that was wrong and that its absence should have indicated a literal lack of a known value?

[sneaker_ger]

No. Absence still means English and this will probably never change.

[sneaker_ger]

Thanks for pointing it out, sneaker_ger. I'm not sure whether this behavior is intentional or accidental. I might consider changing it either writing »language: --« or not writing a language line at all if there's interest in such a change.
I'm not even sure I understood the guliverkli source correctly so I think it's better to leave as is until someone more knowledgeable comes around.

[Mosu]

The default value would have been chosen for compression purposes (ie "eng" would likely be used more often than any other language code)

Disregarding the fact that English is only on third place (http://en.wikipedia.org/wiki/List_of_languages_by_number_of_native_speakers) regarding the number of native speakers this reasoning doesn't really make sense. The language element is five bytes long, therefore space considerations aren't really all that important here.

Even more important: video tracks usually don't have a language. However, most people don't flag a video track as »undefined«. Therefore they implicitly flag them as »English«.

So from a semantic point of view a default value of »und = undetermined« would have been better (meaning anything from »hasn't been set« to »doesn't apply because track has no language«). But specs are specs, therefore this cannot be changed anymore.

[ndjamena]

Most Number of Speakers != most used language code in media files.

Whether video tracks have a language or not is open to interpretation, but things filmed in America can generally be considered "English", because all the signage is in English.

I thought MKVs were primarily designed to hold Movies, TV Shows and other commercial media rather than home movies and such, maybe it's just me being Australian, but I would have thought the majority of those kind of things in circulation in the world would be from the U.S. of A... how often do The Chinese wouldn't watch Indian or Spanish movies?

I have no opinion either way, just assumptions on why things are why they are.

-Edit- And Language alone isn't what improves compression, it's all the defaults combined, including Default Duration.

[Mosu]

Default duration has an impact for each and every group. Its savings scale with the number of frames, and they're the bigger the smaller the frames are (let's TrueHD as an example: one TrueHD frame is often under 1ms in duration resulting in over 1000 frames per second – not having to write the duration for each frame is very, very useful). For the same reasons we introduced the SimpleBlock structure as its simplification over a BlockGroup scales linearly with the number of frames. These savings can be measured in promille or sometimes even percent.

Those optimizations make sense.

The track's language on the other hand is a fixed, five-bytes saving per English track. You usually only have one or two tracks per file which share the same language (one audio, one subtitle or a second audio with e.g. director's comments). Contrast that with the fact that the video track would often have to be tagged with »und« erasing those optimizations. Doing this in order to save space would not make any sense at all; it's savings (if there are any) are not even one thousandth of a promille.

We simply didn't really think it through when we made »eng« the default.

[ndjamena]

We simply didn't really think it through...

I have nothing intelligent to say now, I'm too busy laughing. I've lost count of the number of different ways I've had to find to phrase that sentiment while dealing with the Matroska specs...

I don't suppose MP4 is any better...

[CyberTootie]

Has delay handling recently been changed? I'm having an issue where a h.264 video stream is not being delayed as intended. Negating that value in the audio stream delay gets the sync I want, but cuts off the beginning of the audio.

[hello_hello]

Most Number of Speakers != most used language code in media files.

Whether video tracks have a language or not is open to interpretation, but things filmed in America can generally be considered "English", because all the signage is in English.

I thought MKVs were primarily designed to hold Movies, TV Shows and other commercial media rather than home movies and such, maybe it's just me being Australian, but I would have thought the majority of those kind of things in circulation in the world would be from the U.S. of A... how often do The Chinese wouldn't watch Indian or Spanish movies?

I have no opinion either way, just assumptions on why things are why they are.

As a side note.....
I'd have thought a button to click on that'd automatically set English as the default language might make things easier for English speakers when setting track languages before muxing. It'd make it easier for speakers of any language if you could select which language is set when you click on the default button. The same button could default to setting a track as default and/or forced too, according to user preference. I think I've mentioned the idea before.

Recently I upgraded MKVMergeGUI 7.6.0. I'd been putting it off as I'd read about the introduction of the "Toggle All" button. Sure enough, there it was, mocking me. Laughing at me, fully aware of it's ability to frustrate through it's mere existence. It might have been something useful, a button that with a single click might have reset each track to a pre-specified language and default status etc, but instead I click on the language field, scroll, select English, select the next track, click on the language field, scroll, select English, select the next track, click on the language field, scroll, select English...... whoops, I missed that one and selected "all" instead..... click and scroll again. Now to set the default status for each track.......
And all the while that "toggle all" button laughs at me because it knows it'll rarely be used. If ever.......

Anyway, in all seriousness thanks for the continued work on MKVToolNix Mosu! It's appreciated!

[Mosu]

It might have been something useful, a button that with a single click might have reset each track to a pre-specified language and default status etc, but instead I click on the language field, scroll, select English, select the next track…

It does exactly what the person requesting the feature asked for: enable/disable all tracks.

Anyway, you'll be happy to know that the new GUI (https://github.com/mbunkus/mkvtoolnix/wiki/MKVToolNix-GUI-status/) will allow you to select more than one track at the same time (even all tracks at once!) and change the language for all selected tracks with scrolling through that list only once. It's also written using the Qt library instead of wxWidgets, meaning keyboard shortcuts should work much more consistently (e.g. hitting Ctrl+A when the focus is on the track view should cause all tracks to be selected; hitting any key while a combobox is selected should jump to the first entry starting with that key – e.g. quickly hitting »germ« should leave »German« selected).

[Mosu]

Has delay handling recently been changed? I'm having an issue where a h.264 video stream is not being delayed as intended. Negating that value in the audio stream delay gets the sync I want, but cuts off the beginning of the audio.

I don't recall any such change, and neither can I see a problematic behavior in my short tests (output slighly truncated):

[0 mosu@sweet-chili ~/prog/video/data] mkvmerge -o v.mkv --sync 0:100 v.avi
mkvmerge v7.8.0 ('River Man') 64bit built on Apr 15 2015 15:24:23
'v.avi': Using the demultiplexer for the format 'AVI'.
'v.avi' track 0: Using the output module for the format 'AVC/h.264 (unframed)'.
…
[0 mosu@sweet-chili ~/prog/video/data] mkvinfo -s v.mkv|grep 'track 1'|head -n 1
I frame, track 1, timecode 100 (00:00:00.100), size 12973, adler 0xc59f1503
[0 mosu@sweet-chili ~/prog/video/data] mkvmerge -o v2.mkv --sync 0:200 v.mkv
mkvmerge v7.8.0 ('River Man') 64bit built on Apr 15 2015 15:24:23
'v.mkv': Using the demultiplexer for the format 'Matroska'.
'v.mkv' track 0: Using the output module for the format 'AVC/h.264'.
…
[0 mosu@sweet-chili ~/prog/video/data] mkvinfo -s v2.mkv|grep 'track 1'|head -n 1
I frame, track 1, timecode 300 (00:00:00.300), size 12973, adler 0xc59f1503
[0 mosu@sweet-chili ~/prog/video/data] mkvmerge -o v3.mkv --sync 0:400 v.h264
mkvmerge v7.8.0 ('River Man') 64bit built on Apr 15 2015 15:24:23
'v.h264': Using the demultiplexer for the format 'AVC/h.264'.
'v.h264' track 0: Using the output module for the format 'AVC/h.264 (unframed)'.
…
[0 mosu@sweet-chili ~/prog/video/data] mkvinfo -s v3.mkv|grep 'track 1'|head -n1
I frame, track 1, timecode 400 (00:00:00.400), size 13008, adler 0xce7e1eab

This I'd consider »working as intended«. Can you provide a lot more details?

[hello_hello]

I tested setting a video delay with MKVMergeGUI 7.6.0 and 7.8.0 after CyberTootie posted about the problem here (http://forum.videohelp.com/threads/371374-MKVmerge-video-delay-not-working) and it seemed to work as expected each time. I suspect it's a player issue.

[ndjamena]

In the MMG chapter editor, when navigating in the Tree View using the up and down arrows can we NOT have the focus change to the Name Textbox, up/down doesn't work from there so it limits movement to a single chapter. It would take 9 tab presses or intervention by a mouse to get back to the tree view from there.

(Or maybe allow moving between chapters using up/down while in the Name Textbox, that would fix it too [although considering including the multiple names box could complicate things].)

[Mosu]

Issues in mmg will not be addressed anymore; see https://github.com/mbunkus/mkvtoolnix/wiki/MKVToolNix-GUI-status/

[CyberTootie]

I don't recall any such change, and neither can I see a problematic behavior in my short tests...

This I'd consider »working as intended«. Can you provide a lot more details?

Thanks, I'm not sure how you're getting those stats, here's a screenshot of what I'm doing in MKVmerge and the resulting pertinent info in MKVinfo:
http://imgur.com/eL1j3To

14736

[hello_hello]

Has delay handling recently been changed? I'm having an issue where a h.264 video stream is not being delayed as intended. Negating that value in the audio stream delay gets the sync I want, but cuts off the beginning of the audio.

I don't recall any such change, and neither can I see a problematic behavior in my short tests

It seems the issue is changing the frame rate and setting a video delay at the same time. If the frame rate is changed and a positive video delay is specified, the video delay is ignored. Changing the frame rate in combination with a negative video delay seems to work though.

The behaviour is the same for versions 7.6.0 and 7.8.0.

[sneaker_ger]

I cannot reproduce. Can you upload a sample and post the complete command?

[hello_hello]

"C:\Program Files\MKVToolnix\mkvmerge.exe" -o "E:\\test.mkv" "--language" "0:eng" "--sync" "0:2000" "--default-track" "0:yes" "--forced-track" "0:no" "--display-dimensions" "0:720x404" "--default-duration" "0:24fps" "--language" "1:eng" "--default-track" "1:yes" "--forced-track" "1:no" "-a" "1" "-d" "0" "-S" "-T" "--no-global-tags" "--no-chapters" "(" "D:\\test.mkv" ")" "--track-order" "0:0,0:1" "--disable-track-statistics-tags"

Every single sample I've tried behaves the same way. The above was originally 25fps h264 and stereo AAC audio (I haven't tested without the audio as the whole point of a delay is usually to sync the video and audio). I specified 24fps and a video delay of 2000ms. The result would normally be (when specifying a positive video delay) the first frame being displayed for two seconds. That doesn't happen. The video and audio start off perfectly in sync, even though they wander out pretty quickly due to the frame rate change.

The extracted video timecodes look like this:

# timecode format v2
0
42
83
125
167
208
250
292

The following gives me a first frame that displays for two seconds and the audio sync is 2 seconds out all the way through as I didn't change the frame rate, so I can only assume "--sync" "0:2000" is being applied this time but not the first for some reason.

"C:\Program Files\MKVToolnix\mkvmerge.exe" -o "E:\\test2.mkv" "--language" "0:eng" "--sync" "0:2000" "--default-track" "0:yes" "--forced-track" "0:no" "--display-dimensions" "0:720x404" "--language" "1:eng" "--default-track" "1:yes" "--forced-track" "1:no" "-a" "1" "-d" "0" "-S" "-T" "--no-global-tags" "--no-chapters" "(" "D:\\test.mkv" ")" "--track-order" "0:0,0:1" "--disable-track-statistics-tags"

The extracted video timecodes look like this:

# timecode format v2
2000
2040
2080
2120
2160
2200
2240
2280

The only way I seem to be able to apply a positive video delay and change the frame rate is by doing it in two steps.

[Mosu]

Thanks for the analysis. This is most likely a bug, and I'm pretty sure what happens. Forcing the default duration is implemented by the same code that handles timecode files. Actual timecode files are supposed to supersede any other calculation (this is the equivalent of "I know better, so shut up already!") and is therefore applied after all other timecode modifications have been done.

Unfortunately these semantics conflict with what one would expect of using --default-duration.

I'll fix it; should be easy enough to simply do.

[hello_hello]

I think I heard a penny drop........ (Edit: It dropped before I read Mosu's post. Honestly. We were typing at the same time..... ;))

It makes sense now I see the command line in front of me. Specifying a positive video delay effectively makes it variable frame rate. When you specify a frame rate, MKVMerge outputs a constant frame rate. So the above command line varies the frame rate, then makes it constant again. I assume......

If that's all the problem is, the GUI would just need to add the two to the command line in the opposite order. It'd also explain why a frame rate change and a negative video delay seems to work as I assume the required number of frames are deleted and the output is constant from then on, although if the specified negative video delay and the gop size don't match and MKVMergeGUI needs to apply some sort of positive video delay to compensate, then it changes the frame rate...... I guess doing it in the opposite order would still be better.

[Mosu]

The order on the command line is irrelevant. The order in the source code is. Therefore it will have to be fixed in mkvmerge, not the GUI.

[sneaker_ger]

Every single sample I've tried behaves the same way.
I only tested raw source files, not mkv. Unlike mkv and mp4 they seem unaffected.

[hello_hello]

The order on the command line is irrelevant. The order in the source code is. Therefore it will have to be fixed in mkvmerge, not the GUI.

I was on the right track though, so I'll at least award myself a couple of clever points. :)

Thanks.

[Mosu]

The --default-duration/--sync problem has been fixed in pre-builds 694 and higher (https://www.bunkus.org/videotools/mkvtoolnix/windows/pre/).

[Ex0rcist]

Hi,

http://forum.doom9.org/showpost.php?p=1660312&postcount=3833

I'm wondering the same as the guy in the post I liked above. This only happens when resolution is 718x404. Is there any reason why that happens? If I use the same file and mux it into mp4 container, the "original display aspect ratio" flag isn't present.

[Mosu]

How does that linked post have anything to do with MKVToolNix?

[Ex0rcist]

x264 file:

Width : 718 pixels
Height : 404 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 fps
Scan type : Progressive

mkv file:

Width : 718 pixels
Height : 404 pixels
Display aspect ratio : 16:9
Original display aspect ratio : 16:9
Frame rate : 25.000 fps
Scan type : Progressive

mp4 file:

Width : 718 pixels
Height : 404 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 fps
Scan type : Progressive


It only appears when using mkv container, that's why.

[Mosu]

I ask again, how is that relevant to MKVToolNix? Where is MKVToolNix used? This is a thread about MKVToolNix, not about the Matroska container in general.

Also: if you want to know why MediaInfo outputs information XYZ then please ask the MediaInfo guys first. I don't know their source code!

[Ex0rcist]

I'm sorry, maybe I should be more specific that .mkv file was made using mkvtoolnix. Anyway, you probably are right, it has to be mediainfo issue. I got tricked because mp4 container didnt have that flag enabled but mkv did, that's why I thought it's mkvtoolnix issue. At least I know what is it all about now. Sorry for the confusion.

[ndjamena]

(Windows) If I inadvertently manage to add a colon (:) to the output file name (ie by copying the title field from the container) mkvmerge will run through the files as normal but I'll be left with a zero length file at the end (with the file name truncated at the colon).

It doesn't even produce a warning.

(Should I take it MKVToolNix supports Linux files with colons in the name now?)

[Mosu]

Interesting bug. Note that certain file systems forbid certain characters in their name. On Windows this includes colons, but those are indeed allowed on Linux. mkvmerge doesn't try to be clever about this, it simply tries to open the file name the user specified. The idea is that the OS should tell mkvmerge that the name is not allowed by signalling an error; why that doesn't work as intended is something I will have to investigate at some point.

[DarkSpace]

[I've been tired, misread stuff, and edited it away. I can't remove my post, though.]

[Edit again] (assuming NTFS)
I think your data is written to file:after_colon after all: The after:colon part is an NTFS substream of file, so try accessing it by e.g. moving file:after_colon to file_data ("move file:after_colon file_data") [I have no idea if that will work, but it's worth a try]

[Edit: rewording of the explanation's first part]
The colon tells Windows to access not file itself, but the substream named after_colon of file. Because this works without a problem, mkvmerge works, Windows accesses file, file doesn't exist so it's created, accesses file's substream, the substream doesn't exist, so it's created, then it's opened. I hope this sounds at least somewhat coherent...

[ndjamena]

http://www.faqforge.com/windows/use-alternate-datastreams-to-hide-important-files-windows-ntfs/

So, basically, if there was program support for it, I could move all my movies out of their dedicated folders and move the cover images to "Transformers_ Age of Extinction (2014):cover.jpg" and all the metadata to "Transformers_ Age of Extinction (2014):movie.xml"... Then I could add things like "Alien (1979).mkv:Directors Cut.mkv" so I could have two versions of the movie taking up one slot in the file system... oh, wait...

[Brother John]

Apparently mmg is a bit more clever/restrictive. It rejects colonised file names.

@ndjamena
Basically, yes, you could do that. Though it’s a really fragile solution. The moment you copy such a file to a non-NTFS file system or you copy it with a non-substream-aware copy routine you’ll lose the substreams.

At the end of the day pestering player developers about better attachment, tags and ordered chapters support seems like the more promising approach. ;)

[ndjamena]

Apparently mmg is a bit more clever/restrictive. It rejects colonised file names.

@ndjamena
Basically, yes, you could do that. Though it’s a really fragile solution. The moment you copy such a file to a non-NTFS file system or you copy it with a non-substream-aware copy routine you’ll lose the substreams.

At the end of the day pestering player developers about better attachment, tags and ordered chapters support seems like the more promising approach. ;)

Actually, I'd love to be pestering developers to support those things, and tried for a while. Unfortunately it's impossible to support things that don't work and make no sense.

[ndjamena]

MMG still doesn't notice anything is wrong when I try to save chapters into an mkv while it's locked by AVIDemux. That was 10 minutes wasted. (Saving chapters to a file that didn't have any using [Save to Matroska File]).

(I assume MKVToolNixGUI uses the same code...)

While I'm at it, Explorer adds double quotes to file names copied using the "Copy Path" button which MMG rejects as invalid. Can TNX be written to remove the quotes instead?

[Mosu]

Technically? Sure. Will I invest time in it? No.

The new GUI will allow drag & drop directly onto file name inputs anyway.

[ndjamena]

Um, I have batch files to handle most of the day to day muxing, and only really use MMG for special cases... and I copy paste the file names into the OUTPUT field (for various reasons). Drag and drop works fine for input already.

Actually, the file selection popup windows accept copy/pasted double quoted names as well, handling them is just a basic part of the processing Windows applies to all file names already, it's just the MMG Output Filename at the bottom that doesn't accept them.

It's not important, but it does get a tad irritating and I thought I'd bring it up.