MKVToolNix v24.0.0 released

[ndjamena]

If I select the "Check for updates" option in MMG and close the window before the check is complete a few seconds later MMG will shut itself down.

Remuxing a multi segment VC-1 title from a Star Trek Blu Ray results in occasional broken frames. Extracting those same titles with EAC3To results in a perfectly playable file.

Looking at the Tags MKVMerge adds to the files using FFProbe shows this:

TAG:NUMBER_OF_FRAMES=72760

TAG:NUMBER_OF_FRAMES=72738

The VC-1 stream MKVMerge extracted from the blu ray is missing exactly 22 frames, which - by a strange twist of fate - is the exact number of segments EAC3To reports as being in the title.

[1+2+4+5+7+8+10+11+13+14+16+17+19+20+22+26+28+29+31+32+34+35].m2ts

[Mosu]

About the issue in mmg: I'll look into it.

About the video issue: I'd need the source files in question. Yes, I know that they're huge, but I cannot really do anything about it unless I have sample files.

[ndjamena]

I'm trying to figure out what to send...

I ripped another copy using MakeMKV and the resulting file had the same number of frames as the EAC3To version. So I looked at each of the three files with MKVInfo and it turns out the MMG file is missing the very first frame. I looked at the playlist again with EAC3To to figure out which was the first M2TS and remuxed THAT with MMG, but when I looked at the file with MKVInfo it had the first frame, I'd remuxed an ALAC mp4 using MKVMerge and it had repeated the same timecodes in the cues several times, I've been trying to reproduce that to no avail so I figured you'd changed things in 7.8.0 and all my problems were old news. Just to make sure I ripped the mpls again and... the first frame was missing... Batman has VC1, I ripped that from the mpls and all the frames were accounted for... um, Star Trek TOS disc 1 is a 46GB Blu Ray rip...

-edit-

There's a problem here:
Appending track 18 from file no. 6 ('F:\TeAppending track 9 from file no. 6 ('F:\Temp\backup\STAR_TREK_S1D1\BDMV\STREAM\00010.M2TS') to track 9 from file no. 5 ('F:\Temp\backup\STAR_TREK_S1D1\BDMV\STREAM\00008.M2TS').

[Mosu]

If it's really the very first frame only then it may suffice to have the mpls and the corresponding first m2ts file referenced from that mpls. I wouldn't even need the whole m2ts file; the first 100 MB or so should be plenty.

With such a limited set if data it will be impossible to run certain tools on them (e.g. MakeMKV), but it would be enough for mkvmerge itself.

[ndjamena]

Minor Annoyance.

48000/(24000/1001) = 2002

so encoding audio with a frame size of 2002 samples should make their frame durations match those of NTSC film.

FFMPEG FLAC (MakeMKV): -frame_size 2002

And lo and behold:
| + Codec ID: A_FLAC at 4482
| + Default duration: 41.708ms (23.976 frames/fields per second for a video track) at 4647

Last Audio Block after splitting the file in half:
| + SimpleBlock (key, track number 2, 8 frame(s), timecode 1398.355s = 00:23:18.355) at 4447437747

Eigth last Video block in said file:
| + SimpleBlock (key, track number 1, 1 frame(s), timecode 1398.355s = 00:23:18.355) at 4447401377

And yet:

| + TrackUID: 2 at 4447522639
| + Name: DURATION at 4447522683
| + String: 00:23:18.690000000 at 4447522704

| + TrackUID: 1 at 4447522204
| + Name: DURATION at 4447522249
| + String: 00:23:18.689000000 at 4447522270

One Millisecond off...

***I went with 20mb, it should take about 3 minutes to upload.***

Error: The track number 1 from the file 'E:\Make\backup\IRON MAN DISC 2\BDMV\STREAM\00013.M2TS' cannot be appended to the track number 1 from the file 'E:\Make\backup\IRON MAN DISC 2\BDMV\STREAM\00012.M2TS'. The track parameters do not match.

Should I bother mentioning The Iron Man: Armoured Adventures Season 1 Disc 2, where the PCM audio switches between 16 bit and 24 bit between segments... And how it might be a good excuse to get virtual tracks working... This disc is unrippable using MMG from the mpls at the moment (of course it's actually better to rip each episode individually from their respective m2ts files and add the chapters in manually later, but where's the fun in that?)

[Mosu]

The FLAC packetizer calculates durations in samples. That's pretty much always more accurate than using floating point numbers as the video packetizer has to do. I'm not suprised at all that there's a difference. And no, that's definitely not something I'm going to work on.

About that append issue: well it is how it is. Matroska doesn't support changing track parameters, period. Virtual tracks is an interesting concept, but no one (including myself) has bothered to implement that – so it's only a theoretical cure for such ills. Even if I were to implement support for virtual tracks right now, how long do you think it would take a significant number of player developers and vendors to implement playback support? Look at support for ordered chapters. Abysmal. And that part of the spec has been out for ages now.

[Boulder]

Is mkvalidator still "safe" to use to determine problems in Matroska files? It hasn't been updated for quite a while and gives some unnecessary warnings, but I've disabled those and trusted only the error messages.

[Mosu]

I've updated mkvalidator a couple of days ago with a new release (v0.5.0). It fixes two error message that were just plain wrong. See the following two FAQ entries for MKVToolNix:

ERR009 read DocType version higher than DocType version (https://github.com/mbunkus/mkvtoolnix/wiki/mkvalidator-ERR009-read-doc-type-version-higher-than-doc-type-version)
ERR201 Invalid 'CueRelativePosition' for profile 'matroska v2' in CueTrackPositions (https://github.com/mbunkus/mkvtoolnix/wiki/mkvalidator-ERR201-Invalid-CueRelativePosition-for-profile-matroska-v2-in-CueTrackPositions)

The new release is available on the Matroska website (http://www.matroska.org/downloads/mkvalidator.html).

In order to answer your question: I do consider that release to be safe to use.

[Boulder]

Thanks, will update right away :) I've already saved a lot of work by using the tool, I had some issues with USB drives but I was able to determine which files need to be remuxed instead of all of them..

[Boulder]

Ran a quick test with mkvalidator on a file I just created with the latest mkvmerge; muxed H.264 video, Opus audio and one PGS subtitle track and xml chapters.

ERR201: Invalid 'SeekPreRoll' for profile 'matroska v2' in TrackEntry at 4450
ERR201: Invalid 'CodecDelay' for profile 'matroska v2' in TrackEntry at 4450
ERR201: Invalid 'DiscardPadding' for profile 'matroska v2' in BlockGroup at 2278077190
WRN0C2: The timecode of the Cluster at 1745916440 is not incrementing (may be intentional)
WRN0C2: The timecode of the Cluster at 2278069060 is not incrementing (may be intentional)
................................................................................
WRN0D0: There are 5282 bytes of void data....................Are these anything to worry about?

[Mosu]

ERR201 is the same issue that I've talked about in this FAQ entry (https://github.com/mbunkus/mkvtoolnix/wiki/mkvalidator-ERR201-Invalid-CueRelativePosition-for-profile-matroska-v2-in-CueTrackPositions) and that should have been fixed in mkvalidator v0.5.0. Are you sure you're using the latest version (0.5.0) and not an earlier one?

I also have a FAQ entry for WRN0D0 (https://github.com/mbunkus/mkvtoolnix/wiki/mkvalidator-WRN0D0-void-data).

About WRN0C2: well that can happen, depending on the codecs in question. Just like the warning said, it may be intentional (and is definitely not violating the specs).

[Boulder]

ERR201 is the same issue that I've talked about in this FAQ entry (https://github.com/mbunkus/mkvtoolnix/wiki/mkvalidator-ERR201-Invalid-CueRelativePosition-for-profile-matroska-v2-in-CueTrackPositions) and that should have been fixed in mkvalidator v0.5.0. Are you sure you're using the latest version (0.5.0) and not an earlier one?Yes, that was with the latest one. The previous file passed the test and showed that it was validated with v0.5.0.

EDIT: It seems to happen with Opus tracks. If I use a FLAC track for audio, those warnings do not appear.

[Mosu]

I cannot reproduce this kind of problem. I've just muxed a file with h.264, Opus, PGS subtitles and chapters and the only thing that mkvalidator reports for that file is WRN0D0.

[Boulder]

I've got a muxed sample for you here: https://drive.google.com/file/d/0BzeF_1syecQwYlF4LUpreERQZ1U/view?usp=sharing

Muxed with these settings:
"C:\Program Files (x86)\MKVtoolnix\mkvmerge.exe" -o "b:\\test3.mkv" "--priority" "higher" "--forced-track" "0:no" "-d" "0" "-A" "-S" "-T"
"--no-global-tags" "--no-chapters" "(" "B:\\test2-001_track1_und.h264" ")" "--forced-track" "0:no" "-a" "0" "-D" "-S" "-T"
"--no-global-tags" "--no-chapters" "(" "B:\\test2-001_track1_und.opus" ")" "--forced-track" "0:no" "-s" "0" "-D" "-A" "-T"
"--no-global-tags" "--no-chapters" "(" "B:\\test2-001_track3_fin.sup" ")" "--track-order" "0:0,1:0,2:0"
"--chapters" "B:\\test2-001_chapters.xml" "--engage" "no_cue_duration" "--engage" "no_cue_relative_position" "--disable-track-statistics-tags"

[Mosu]

Aaaaaaaah. The culprit is most likely the two --engage you use. mkvmerge wrongfully thinks that Matroska v2 is enough if neither cue duration nor cue relative position elements are used. But that's wrong if Opus is in the picture as Matroska v4 is also needed for seek pre roll, codec delay and discard padding.

I'll fix mkvmerge accordingly.

[Mosu]

The issue has been fixed in mkvmerge. Pre-builds 686 and higher (https://www.bunkus.org/videotools/mkvtoolnix/win32/pre/) contain the fix. Thanks for the heads-up & the test case.

[Boulder]

No problem, happy to help :)

[sneaker_ger]

Mkvmerge add extra one SEI if mux raw HEVC stream, not add extra one if remux from MP4. x265 1.5+444 gives longer SEI (because --rdoq-level and deblock). If longer SEI is doubled, it crash with nVidia decoder.

My software LAV decoder stop working with your example after the SEI was tripled -- in your original MKV it was doubled, I demux video track and mux again to MKV (2 + 1 = 3).

Workaround to this mkvmerge bug: first mux to MP4 and then to MKV not works with large files -- I try with 6 GB HEVC stream and mkvmerge crashes when remuxing from MP4.

If you take v1.hevc file with nonempty SEI and execute:
mkvmerge -o v2.mkv v1.hevc
mkvextract tracks v2.mkv 0:v2.hevc
mkvmerge -o v3.mkv v2.hevc
mkvextract tracks v3.mkv 0:v3.hevc
mkvmerge -o v4.mkv v3.hevc
mkvextract tracks v4.mkv 0:v4.hevc
mkvmerge -o v5.mkv v4.hevc
mkvextract tracks v5.mkv 0:v5.hevc
mkvmerge -o v6.mkv v5.hevc
mkvextract tracks v6.mkv 0:v6.hevc
...

The number of SEI in the files should be equal the number in filename.

Mkvmerge copies the headers into the mkv CodecPrivate element. If you demux in "normal" mode (i.e. without --raw parameter) using mkvextract those headers will be re-added. Since mkvmerge does not remove them the amount of headers will increase with every generation of mux/demux. While this is not wanted it might not qualify as a bug. I think these might be relevant:
https://github.com/mbunkus/mkvtoolnix/issues/1076
https://github.com/mbunkus/mkvtoolnix/commit/4207f953d9061e8c911df928c8ad5b81b81d891c

/edit2:
(Had old mkvtoolnix installed for first test. Ignore my first edit.)

[Mosu]

sneaker_ger's correct. There are several formats for which the data stored in CodecPrivate is duplicated in the frames. These include h.264, h.265 and even older ones like MPEG-1/2. Why? Several reasons.

One problem is that the private information may change over time (e.g. you can concatenate two MPEG-2 files and most players will play resulting file just fine, even if the sequence headers change mid-stream). Coming back to Matroska this means that the change in CodecPrivate content would have to be signalled somehow. For that we introduced the CodecState element (way back when there was no h.264) which would contain the new content for CodecPrivate and be located right before the frame that needed those parameters. Unfortunately no player implemented support for CodecState, and I had to revert to keeping the sequence headers inline.

That's the answer to »why is that data stored in the frames, too«.

Now to mkvextract. At the moment mkvextract extracts the headers (sequence/video/whatever parameter sets) from the CodecPrivate and writes them into the output file. Then mkvextract finds the first frame (which includes those parameter sets as discussed above) and writes that frame into the output file, too – which now includes the headers twice.

Technically speaking mkvextract could be made to scan the first frame for the headers, compare them with what's present in CodecPrivate and only write the parameter sets if they're missing from or different than the ones in the first frame. But I don't really care about that, and my time is limited, so I will definitely not do that myself.

[filler56789]

Nitpick:

"bunkus.org/videotools/mkvtoolnix/win32/pre/" became a misnomer (sort-of), since the amd64 builds can't be executed on Windows XP :)

[Mosu]

I know, but a lot of people know that directory, therefore I haven't renamed it (yet).

[Mosu]

Meh. I decided to rename it to »windows« and promptly made an error during syncing the directory to my server resulting in the whole directory being lost. Good thing I have backups :)

[filler56789]

Me too has backups of MKVtoolnix, I've been downloading and archiving those "pre" builds since ages ago :D

[Qaq]

Just got 1073741819 error code (2-3 attempts). I tried to mux:
- m2v (50i) video from PAL DVD (demuxed by PgcDemux 1205 without any error);
- FLAC from that DVD (LPCM demuxed by PgcDemux and packed to FLAC by eac3to without any error).
MkvMerge is 7.8.0 x64 version. Any tips on whats wrong and how to fix?

[Mosu]

That error code usually means that mkvmerge ran out of memory which is almost always due to either bogus file data or a bug in mkvmerge. I would need the two files in question if you want me to take a look.

[ndjamena]

Just got 1073741819 error code (2-3 attempts). I tried to mux:
- m2v (50i) video from PAL DVD (demuxed by PgcDemux 1205 without any error);
- FLAC from that DVD (LPCM demuxed by PgcDemux and packed to FLAC by eac3to without any error).
MkvMerge is 7.8.0 x64 version. Any tips on whats wrong and how to fix?

Yes, Mosu, please fix that... :)

(try it with just the m2v and see if it still happens, then try... 7.3.0 I think it was...)

[Qaq]

I would need the two files in question if you want me to take a look.PM sent. oops, sorry, didn't see your signature notification cause I had them disabled.
(try it with just the m2v and see if it still happens, then try... 7.3.0 I think it was...)I tried it first with 7.3.0 and just tried with m2v only. Still the same.

EDIT: Mosu, no need to check it. It seems something went wrong with DVD ripping though and data is incomplete. My bad.

[Mosu]

Yeah… I cannot find a problem with the two files here either. I've muxed both three times: on Linux (64bit) and on Windows (both 32 and 64bit builds). Everything's fine, memory usage stays pretty much constant.

[Chetwood]

Mosu, I've tried to append some files that have identical tracks but mmg doens't see it this way. I've uploaded a small zip to your FTP with Mediainfos output. Maybe that is sufficient.

[Mosu]

Do you mean the file append.error-the.formats.do.not.match.rar from April 1st? If so: I've already deleted that file as I couldn't associate it with any request.

What would the actual error message be, BTW? MediaInfo output is most likely not helpful on its own.

[ndjamena]

When remuxing m4vs...

MMG doesn't seem to take into account a tracks "enabled" flag, which behaves much like a "default" flag in an MKV, is this deliberate?

When extracting chapter images, shouldn't MKVMerge save them as attachments rather than as a video stream, after all: it doesn't extract apple text chapters as a subtitle track.

[Mosu]

mkvmerge doesn't evaluate that flag.

Chapters in MP4 files are a steady stream of funny tales… meaning the different ways suck. It's not really unusual that mkvmerge doesn't behave the way you expect it to.

[ndjamena]

It's itunes, and apparently they've been doing it the one way for quite a while:

http://code.google.com/p/subler/issues/detail?id=27

I don't really care, it just took me a while to figure out what the hell that second video track was.

(mkvextract cues, mkvmerge "split at chapters", mkvextract -raw)

[ndjamena]

Just so I'm sure we're on the same page:

Video #2
ID : 8
Format : JPEG
Codec ID : jpeg
Duration : 1h 49mn
Bit rate mode : Variable
Bit rate : 3 891 bps
Width : 640 pixels
Height : 360 pixels
Display aspect ratio : 1.85:1
Frame rate mode : Variable
Frame rate : 0.002 fps
Minimum frame rate : 0.001 fps
Maximum frame rate : 0.004 fps
Color space : YUV
Chroma subsampling : 4:4:4
Bit depth : 8 bits
Compression mode : Lossy
Bits/(Pixel*Frame) : 8.444
Stream size : 2.78 MiB (0%)
Language : English
Encoded date : UTC 2014-02-14 22:47:03
Tagged date : UTC 2014-02-14 23:02:54

Menu
ID : 7
Codec ID : tx3g
Duration : 1h 49mn
Language : English
Encoded date : UTC 2014-02-14 22:47:03
Tagged date : UTC 2014-02-14 23:02:54
Source duration : 6589756
Source frame count : 12
Source stream size : 215
Frame count : 12
Stream size : 215
00:00:00.000 : Opening Credits/A Shooting Star
00:09:21.000 : Electrical Marvel
00:19:16.000 : "Run, Dorothy, Run!"
00:28:40.000 : The Yellow Brick Road
00:40:41.000 : Princess Mombi
00:46:23.000 : Jack Pumpkinhead
00:54:19.000 : Powder Of Life
01:04:42.000 : Crash Landing
01:12:37.000 : Three Chances
01:26:26.000 : Scarecrow
01:34:16.000 : Emerald City
01:45:08.000 : End Credits
Bit rate mode : VBR



The chapter thumbnails are there in addition to the apple text chapters. I should think there's something in the file that signals what it's for and it seems to be rather standard itunes stuff.

Encoded date : UTC 2014-02-14 22:47:11
Tagged date : UTC 2014-02-14 23:02:54


In case someone didn't already know.

[Mosu]

I currently have no plans of looking into that.

[ndjamena]

I though it was interesting.

And here's the disabled flag:

Track # 3 Info - TrackID 3 - TimeScale 48000 - Media Duration 01:49:49.760
Track has 1 edit lists: track duration is 01:49:49.759
Track is disabled
Media Info: Language "English (en-US)" - Type "soun:ac-3" - 205930 samples
AC-3 stream - Sample Rate 48000 - 5.1 channel(s) - bitrate 192000
RFC6381 Codec Parameters: ac-3
Alternate Group ID 1
All samples are sync

MP4Box doesn't show it when it's enabled, but it works just like a default flag (other than that if there's two of them for one type, both tracks are played at the same time). It's odd how many people don't know that exists.

[Boulder]

I've got a large amount of ERR312's in the mkvalidator log with Opus audio, for example:

ERR312: CueEntry Track #3 and timecode 8544636 ms not found
ERR312: CueEntry Track #1 and timecode 8545454 ms not found

When muxing the same video and subtitle track with FLAC audio, there are no such errors.

I've not seen these errors with other files, but this one (the first Hobbit movie) is over 3 hrs long which makes it a bit different from the others.

[Mosu]

I cannot reproduce such errors with mkvalidator 0.5.0 and a random file with Opus inside. So you'll have to provide me with a sample file if you want me to take a look…

[Boulder]

While trying to remux the file (because I couldn't reproduce it by demuxing to elements and muxing them), I found out that the Matroska file contains errors. This is really weird because I just muxed the original file and moved it over the network from my desktop PC to the external HD without any CRC differences (used Teracopy to move the file). It seems that the external HD is going bad :(

[Mosu]

Bad hardware will get you all the time, yeah :( I just had to replace two hard drives in my RAID 6 array. Luckily I'm a big fan of RAID 6 and not just of RAID 5…

[Carpo]

When using mkvalidator on any MKV I have made using 7.70 or 7.8.0 I always get the output

ERR200: Missing element 'Targets' in Tag at 12282419123.
WRN0D0: There are 5938 bytes of void data........................

file created with libebml v1.3.1 + libmatroska v1.4.2 / mkvmerge v7.8.0 ('River Man') 64bit built on Mar 27 2015 16:31:37


The targets error changes depending on the size of the file, but I always get those 2 lines on every MKV file, is this any thing to be concerned about and if so how to fix?

[Mosu]

I don't get that error with mkvalidator with 0.5.0 with a file I've just created with mkvmerge 7.8.0. Are you attaching custom tags? Are you merging files with tags (apart from the statistics tags mkvmerge creates itself)?

[Carpo]

Ah yes I forgot about that, I am adding imdb information from a custom made xml file, I will try without this information

[Mosu]

Please send me the XML file in question ( moritz@bunkus.org ) along with the actual command line used. Thanks.

[Carpo]

Sent information as requested

edit: here is the command line as requested

"C:\Program Files\MKVToolNix\mkvmerge.exe" -o "E:\\Batman Returns.mkv" "--language" "0:eng" "--forced-track" "0:no" "--display-dimensions" "0:1920x1080" "--compression" "0:none" "-d" "0" "-A" "-S" "-T"
"--no-global-tags" "--no-chapters" "(" "D:\\Videos\\Movies\\Batman Returns_track1_eng.264" ")" "--language" "0:eng" "--sync" "0:0" "--forced-track" "0:no" "--compression" "0:none" "-a" "0" "-D" "-S" "-T"
"--no-global-tags" "--no-chapters" "(" "D:\\Videos\\Movies\\Batman Returns_track2_eng_DELAY 0ms.ac3" ")" "--track-order" "0:0,1:0" "--global-tags" "D:\\Batman Returns Custom.xml"

[Mosu]

Even with the file and the command line you've sent me I cannot reproduce the issue. mkvalidator 0.5.0 stays silent (apart from the void warning).

Which version of mkvalidator are you using? If it's anything else than 0.5.0 then upgrade to 0.5.0, please.

BTW: Note that your XML file is invalid. Even though it is encoded as plain ASCII or UTF-8 its header states »utf-16«.

[Carpo]

Yes I am using 0.5.0, I will change the XML, Let me try something, I wonder if the HTPC software is doing something

edit: seems my PC must have a gremlin, because I have remuxed it on the Linux PC and it doesnt show an error, and have remuxed it in PC again and its fine, so I havent got a clue.

Will Try a few more movies

[ndjamena]

Should I assume Matroska will never support 608 captions in any form?

[Mosu]

Closed captioning? Well, I do have that on my TODO list somewhere (would be converted into something like S_TEXT/UTF8 subs automatically), but don't hold your breath.

[ndjamena]

???

I got sick to death of DVDs (frame rate conversions, edge enhancement, HD shows that were never released on Blu Ray) and finally made the switch to iTunes, they've included ALL their subtitles as 608 caption tracks. I've extracted them with MP4Box and am half way through writing a utility to convert them to srt. The program works perfectly with the sample file I'm using but it still needs a lot of work to be fully 608 compliant...

I don't know what you know about the captions but they're not self contained frames, they're a continuous series of commands that are applied to buffers, with some of said buffers overlaid onto the screen. Pop-up mode is easy enough to convert to srt, Paint-on mode would be odd in srt form (but doable), roll-up style would never work properly... I think Text Mode is a single line that gets updated with miscellaneous data, and as far as I can tell XDS are more like tags...

I'd feel better if I could store the original data in the file though, I can't image any conversion process being completely lossless. VLC can't read them from the mp4 yet (it is on the to do list though) but it can already decode 608 captions from DVD (it's not like 608 caption decoders are hard to come by) and they're essentially the same thing.

(I'm have trouble figuring out which of the control characters actually take up space... I'm thinking it's all the ones that change the style of the text. One line in my sample simply can't work without the midrow commands taking space (it looks right in iTunes), but the documentation I've found doesn't mention it (or at least I haven't found it yet).)

Just figuring out my options.