MKVToolNix v99.0 released

[Klaus1189]

v26.0.0 is singed by "LINET Services GmbH"

[Mosu]

That is correct, as explained on the MKVToolNix home page (https://mkvtoolnix.download/authenticity.html#windows). LINET Services is my own company (https://www.linet-services.de/impressum/). It was easier to get a code signing certificate for an organization than for an individual.

[73ChargerFan]

Ouch.. twice in two hours the v26.0.0 GUI crashed but mkvmerge keeps running in the background.

It happened when I was interacting with the Multiplexer tab, but that's all I know. Restarting the GUI works (Which Is Really Cool) - it shows me the job that's running, and the jobs I'd queued before the gui crashed keep starting like nothing happened.

I'll reinstall and restart my system, maybe all the in place upgrades have finally caught up with me.

EDIT:
Turns out now I have two instances of mkvmerge running.

[Mosu]

Well, you'll have to figure out which steps make the GUI crash exactly. I cannot act on "it happens sometimes somehow".

[73ChargerFan]

I'm spending all day doing this, and will try to maybe pay closer attention some of the time. Thanks.

[anddi]

Does that help?
https://forum.videohelp.com/threads/389849-Remuxing-with-MKVToolNix-removes-videos-thumbnails-and-file-properties#post2526077

Yes. Wow. I came back here because I had just noticed that deleting every tag fixes thumbnails on Win10, but after seeing your reply I tried mkvpropedit with just "--delete-track-statistics-tags" option and that alone fixed my thumbnails on Windows 10. I don't use any software for thumbnails whatsoever.

[mkver]

Yes. Wow. I came back here because I had just noticed that deleting every tag fixes thumbnails on Win10, but after seeing your reply I tried mkvpropedit with just "--delete-track-statistics-tags" option and that alone fixed my thumbnails on Windows 10. I don't use any software for thumbnails whatsoever.
Then I believe that the change introduced because of this issue (https://gitlab.com/mbunkus/mkvtoolnix/issues/2202) is behind this. Someone tell Microsoft that TagDefault has a default value.

[Boulder]

Well, you'll have to figure out which steps make the GUI crash exactly. I cannot act on "it happens sometimes somehow".

I've experienced it as well. This is what Win10 Event viewer shows in the log:

Faulting application name: mkvtoolnix-gui.exe, version: 26.0.0.0, time stamp: 0x00000000
Faulting module name: mkvtoolnix-gui.exe, version: 26.0.0.0, time stamp: 0x00000000
Exception code: 0x40000015
Fault offset: 0x000000000141fcab
Faulting process id: 0x4778
Faulting application start time: 0x01d44126ac2dcb52
Faulting application path: C:\Program Files (x86)\MKVtoolnix\mkvtoolnix-gui.exe
Faulting module path: C:\Program Files (x86)\MKVtoolnix\mkvtoolnix-gui.exe
Report Id: 217c1904-3556-4978-afe9-d91c1dcf07af
Faulting package full name:
Faulting package-relative application ID:

I've been transferring my encode archives from DVDs to portable HDDs. Some of the old archives are MKV split into parts, and I use MKVToolnix GUI to rejoin them to mux them as a single file on the HDD. So basically I drag-and-drop stuff, use Append, select file and mux.

What I have noticed that after some muxes, my OS turns quite sluggish and restarting MKVToolnix GUI will fix that. If I don't restart, after several rounds of rejoining and muxing files, the GUI crashes by simply vanishing and then it leaves its mark in the Event viewer log. I've had it happen twice, and both times it happened when I dragged a file in the source panel.

If you can come up with a debug build, I might be able to trace the problem for you. I may still have some split files to join :)

[Mosu]

Thanks. Doesn't help me that much, though.

What you've described sounds like a severe memory leak; the system thrashing would be swapping activity. The crash might then be due to all free memory being exhausted.

I've tried to reproduce this by appending various file types for several minutes now. Memory usage will rise while mux jobs are open, but that's to be expected, and it's not that huge usage either. With ten open settings with ~4 files each, memory usage rises from ~40 MB at start to ~68 MB.

There does seem to be some kind of leak that I can detect as memory usage doesn't drop back to ~40 MB when I close all settings, remove all jobs from the queue & clear the job log.

With those numbers, you'd need hundreds of open mux jobs in order to exhaust even those systems that don't have a lot of memory.

So far this doesn't help me much.

What you could do is have task manager open while you mux and watch the GUI's memory usage. If you notice severe increases due to certain files, I'd highly appreciate it if you uploaded them to my file server (https://gitlab.com/mbunkus/mkvtoolnix/wikis/FTP-server) along with a description of how exactly you set up the job.

A debug build wouldn't help in this case. I'm convinced that the crash isn't the problem; it's just a symptom of the bug that leads to the system becoming sluggish (like I said, likely a memory leak). Debug builds don't help with memory leaks.

[Boulder]

I just experienced the crash again. There were no signs in the task manager view, normal memory consumption all the time even though the sluggishness started almost immediately after the first job was done. It definitely seems memory related because the sluggishness seems to be more apparent if I have an encoding task running, which means that around 50% of total memory is reserved by vspipe and x265. But like I said, there's nothing visible pointing to a memory leak because there is plenty of memory available right up until the crash.

[Mosu]

How much memory is mkvmerge.exe (not mkvtoolnix-gui.exe) consuming in that case? Note that mkvmerge's memory will jump right when it starts, and it will continue to grow slowly due to in-memory indexes getting bigger (those indexes can only be written all at once when the file's done).

Can you watch your I/O load, too? Is mkvtoolnix-gui.exe causing a lot of I/O activity while the system is sluggish? You can use Windows' "Resource Monitor" tool for that.

[Boulder]

The crash occurs directly in the GUI, for me it happens when I drag and drop a source file in the panel. I've not had a crash while a mux operation has been in progress. I watched a 10 GB mux progress and mkvmerge took around 50-70 MB most of the time, peak working set was about 100 MB. I'll try to see how many muxes it will take for the crash to happen as the sluggishness already appears after the first two or three.

Mkvtoolnix-gui.exe has no activity outside normal usage in Resource Monitor, it seems to be behaving well.

[Mosu]

Can you reproduce the crash with the same file? If not, try removing the "cache" folder (https://gitlab.com/mbunkus/mkvtoolnix/wikis/Location-of-settings,-job-queue,-cache-folder) between tries.

[Mosu]

When you drag & drop & the GUI crashes, is a job still running at that point?

[Boulder]

Can you reproduce the crash with the same file? If not, try removing the "cache" folder (https://gitlab.com/mbunkus/mkvtoolnix/wikis/Location-of-settings,-job-queue,-cache-folder) between tries.

I'll try.. I probably need to run a mux operation instead of just goofing around with the GUI if this is memory related.

When you drag & drop & the GUI crashes, is a job still running at that point?No, I always wait for the mux job to finish before I start preparing a new one. This is basically because I usually get one part from a DVD and all the rest from an HDD.

[Mosu]

BTW, here (https://mkvtoolnix.download/windows/debug/32-bit/) is a build with debug symbols. Don't know how much use those might be to you — I compile on Linux with the mingw cross-compiler. I don't know if its type of debugging info is usable on Windows.

[Boulder]

I was able to reproduce it with the same file(s). I already had finished four different muxes and then did the same mux job four times just like each of them was a new one. After those, I added the same source files normally and added the job into the queue. Removed all source files to readd them once again and the GUI vanished. I could tell that I'm close to success (sic) because the HDD started making swapping like noise.

I'll install the debug build, let's see if we can find something there. It seems to be 32-bit but probably no difference in that sense compared to the 64-bit build that I use (the x86 path is due to an ancient installation).

[Mosu]

Well yeah, I figured from your x86 path that you were using a 32-bit build, hence… :)

[Boulder]

Well, it crashed after adding the same files over and over again and adding the job in the queue :) Where can I get the dump file? I couldn't find it in the program directory.

[Mosu]

No idea, I don't program under Windows.

But I do have an idea where this might come from. I'll try to rewrite the corresponding code section later tonight.

[XinHong]

Hello Mosu,

There is something strange with v26 of MKVToolNix GUI:
- Start MKVToolNix GUI
- Drag & drop a file
- Perform a mux (not mandatory to perform something)
- Right click in the "Source Files" and choose "Remove All Files"
- Close MKVToolNix
- The following message appears :
"The file "< No destination file >" has been modified. Do you really want to close ? All changes will be lost"

I don't remember I had this message in v25, and, as all files in the source list have been removed, I think this message should not appear

[Mosu]

The settings consist not only of the stuff on the "input" tab, but also of all the controls on the "output" and "attachments" tab. If their settings deviate from the last saved state (or the "new settings" state if it has never been saved or added to the job queue), then the GUI will ask for confirmation.

[Mosu]

Well, it crashed after adding the same files over and over again and adding the job in the queue :)

I've created a new debug build (https://mkvtoolnix.download/windows/debug/64-bit/), this time not with debugging symbols, but with a lot of validity checks on the internal structures containing job information. If an inconsistency is found, an error message is shown. Please give this build a try, and if such a message pops up, please provide the error it shows (you can select & copy the message shown in that dialog, BTW).

BTW: I highly appreciate your work on this. As I cannot reproduce the crashes (yet), I have to rely on other people's work.

[mkver]

Where can I get the dump file? I couldn't find it in the program directory.
It's in C:\Users\<Account_Name>\AppData\Local\CrashDumps. But can one even analyze them with the GNU tools?

PS: Have you already tried to find out when exactly the problems started to arise by using these builds? (https://mkvtoolnix.download/windows/continuous/64-bit/25.0.0/) I can't reproduce the crashes and therefore can't do it.

[Boulder]

I've created a new debug build (https://mkvtoolnix.download/windows/debug/64-bit/), this time not with debugging symbols, but with a lot of validity checks on the internal structures containing job information. If an inconsistency is found, an error message is shown. Please give this build a try, and if such a message pops up, please provide the error it shows (you can select & copy the message shown in that dialog, BTW).

BTW: I highly appreciate your work on this. As I cannot reproduce the crashes (yet), I have to rely on other people's work.

No problems regarding testing :) This was a random accident that I happened to start working so heavily on video clips, I usually mux a couple of files at a time. In my daily work, I often need to rely on information from others to reproduce problems so I know how frustrating it can sometimes be to try things.

Now I got the crash with the mux job running. No error messages whatsoever, the OS just turned very sluggish while I was browsing the internet and I knew that it would not take long for something to happen.

I uploaded two most recent dumps if they are of any use. The older one is from yesterday's debug build, the other one is the crash I just had.

https://drive.google.com/open?id=17KJ5ZgwUP26_65WVr2YBocphfn5LKlJ3

mkver, thanks, I think I'll start digging a bit deeper with those builds.

[Mosu]

OK, so the problem is completely unrelated to that particular part of the program. It was one of the few areas of the GUI that I actually changed between v25 and v26. So we'll have to look at other changes.

There's one particular change between 25 and 26 that affects job/config handling. I've reverted that change and created a new debug build (https://mkvtoolnix.download/windows/debug/64-bit/) without said change.

[Mosu]

BTW, the crash dumps don't help me — yet. I'm currently learning a lot about different debug info formats, how to do postmortem and/or just-in-time crash debugs for mingw applications etc. I'll look at bit further into those methods before providing new binaries to test with.

In case anyone is interested: "MinGW and Postmortem Debugging" (http://blog.morlad.at/blah/mingw_postmortem), "Dr. MinGW — Postmortem debugging tools for MinGW" (https://github.com/jrfonseca/drmingw)

[Mosu]

Alright, here (https://mkvtoolnix.download/windows/debug/64-bit/mkvtoolnix-64-bit-26.0.0-revision-016-g78d7708b3.7z) is another debug build. This one links against the Dr. MinGW library. When it crashes, it should write a file called "mkvtoolnix-gui.RPT" in the same file the exe is is (hence me only providing the portable archive — writing to "C:\Program Files…" is normally not possible). This file should contain a stack trace. Please send me that file/upload it somewhere. Thanks.

[Boulder]

Alright, here (https://mkvtoolnix.download/windows/debug/64-bit/mkvtoolnix-64-bit-26.0.0-revision-016-g78d7708b3.7z) is another debug build. This one links against the Dr. MinGW library. When it crashes, it should write a file called "mkvtoolnix-gui.RPT" in the same file the exe is is (hence me only providing the portable archive — writing to "C:\Program Files…" is normally not possible). This file should contain a stack trace. Please send me that file/upload it somewhere. Thanks.

OK, will do :) Let's hope I can reproduce the crash rather sooner than later.

[Boulder]

I was able to reproduce the crash by torturing the GUI but no RPT file appeared in the program folder :( I'll install rev 014 and see how it goes..

[Mosu]

014 isn't linked against that library and wouldn't create such a report anyway.

If there is no report, the crash doesn't happen under the control of the program.

I honestly don't know what else to try here.

[Boulder]

But wasn't 014 the one in which you reverted a change that could be associated with this issue? Currently I have tried only adding files as a source and added jobs to queue and it has worked smoothly and doesn't cause any OS sluggishness. The next step will be muxing and then torturing the GUI.

[Mosu]

In 016 the change is still reverted, just with the Dr. MinGW library linked & invoked. So if 016 still crashes, that aforementioned change isn't the culprit.

[Boulder]

Ah, okay. I think I'll try v25 then, and work my way up from there with those intermediate builds towards v26. Of course, I could find this in v25 as well because it's been quite a long time since I did this kind of work.

[dade49]

Hello Mosu, I have a DTS-HD MA 5.1 track (24 bit) with dialnorm on both the master and the embedded core track. I am using the MkvToolNix GUI to build an mkv from the source and using the option to remove dialog normalization from the track. The resulting file has the dialnorm removed from the core track, but it remains on the master track. Is there a way to remove it from both? Is this an issue that you're aware of? I'm happy to provide a sample if you need it. Thank you for all of your hard work.

[Mosu]

Hello Mosu, I have a DTS-HD MA 5.1 track (24 bit) with dialnorm on both the master and the embedded core track.

I'm not aware of such a problem. Please open a new issue (https://gitlab.com/mbunkus/mkvtoolnix/issues/) for it and upload a sample file.

[Mosu]

Ah, okay. I think I'll try v25 then, and work my way up from there with those intermediate builds towards v26. Of course, I could find this in v25 as well because it's been quite a long time since I did this kind of work.

One thing i did change between releases 25 and 26 was that I updated all of the libraries used by MKVToolNix, specially Qt from 5.10.1 to 5.11.1. Here (https://mkvtoolnix.download/windows/debug/64-bit/mkvtoolnix-64-bit-26.0.0-revision-016-g78d7708b3-01788.7z) is a new build of the same code base (016-g78d7708b3) as before, but compiled against the older set of libraries, the one v25 was compiled against. Please give this build a try, too. Thanks.

[Boulder]

One thing i did change between releases 25 and 26 was that I updated all of the libraries used by MKVToolNix, specially Qt from 5.10.1 to 5.11.1. Here (https://mkvtoolnix.download/windows/debug/64-bit/mkvtoolnix-64-bit-26.0.0-revision-016-g78d7708b3-01788.7z) is a new build of the same code base (016-g78d7708b3) as before, but compiled against the older set of libraries, the one v25 was compiled against. Please give this build a try, too. Thanks.
Thank you, I've been slowly resuming the work today. I'll try to make it crash over the weekend :)

[zn]

Is it possible to set >99 chapters using "simple chapter format", or it only possible with XML based chapter format?

[Mosu]

I haven't actually tried it, but the code uses regular expressions for matching the lines, and the number part is a simple "\d+" — meaning "one or more digits". Meaning "CHAPTER01" should work, as should "CHAPTER1", "CHAPTER0001" or "CHAPTER12345".

[hubblec4]

The OGM specs allows 1000 chapters indexed from 000 up to 999.
But Mosu's Chapter Editor and my chapterEditor supports more chapters.

[Mosu]

Thank you, I've been slowly resuming the work today. I'll try to make it crash over the weekend :)

The problem has been fixed. A crucial piece of information was contributed over on GitLab (https://gitlab.com/mbunkus/mkvtoolnix/issues/2372): it doesn't depend on the files; simply switched between the tools suffices. With that I was able to reproduce it, track it to a font resource memory leak introduced after v25, and fix it. A new, fixed build is available here (https://mkvtoolnix.download/windows/pre/64-bit/).

[Boulder]

Congratulations :) This was a tough one, fortunately others were able to contribute as well.

[Megalith]

Maybe this is related to the above issue (which has been fixed), but I have run into a couple of instances in which MKVToolNix GUI will disappear in the middle of a job, as if the program was closed. Task Manager will show that the program is still running, however.

[mkver]

Maybe this is related to the above issue (which has been fixed), but I have run into a couple of instances in which MKVToolNix GUI will disappear in the middle of a job, as if the program was closed. Task Manager will show that the program is still running, however.
1. Just to be sure: You are running the new, patched version, not the 26.0 release version?
2. What exactly does Task Manager show? Is it mkvmerge.exe running or is it mkvtoolnix-gui.exe? If you started a job from the GUI, then you should have a mkvmerge.exe process and a mkvtoolnix-gui.exe process running at the same time. mkvmerge.exe ends automatically after the completion of the job, so if the GUI crashes during a job you should still have a MKVToolNix related process running (namely mkvmerge.exe), but it shouldn't be mkvtoolnix-gui.exe.

If the answer to 1. is yes and if it's really mkvtoolnix-gui.exe that is running after the GUI crashed, you should open a ticket on gitlab.

[Megalith]

I tried the patched version and have not run into the issue so far, so I assume all is good now.

[Selur]

Using:
rav1e: 0.1.0
mkvmerge: v26.0.0
ffmpeg: version N-91949-g6304268e39
I aimed do the following:
a. convert input to av1, by decoding to yuv420p using ffmpeg, piping to and encoding with rav1e
ffmpeg -y -loglevel fatal -threads 8 -i "F:\TestClips&Co\files\test.avi" -map 0:0 -an -sn -vf zscale=rangein=tv:range=tv -pix_fmt yuv420p -vsync 0 -f yuv4mpegpipe - | rav1e --tune Psychovisual - -o "E:\Output\test.ivf"
.ivf output plays fine in latest MPC-HC so I assume the file is okay
b. remuxing to mkv using mkvmerge:
mkvmerge --ui-language en -o "E:\Output\test.mkv" -d 0 --default-track 0:yes --default-duration 0:25/1fps --aspect-ratio-factor 0:1/1 --engage enable_av1 --no-chapters --compression -1:none --forced-track 0:yes --no-audio --no-subtitles "E:\Output\test.ivf"
but that creates 0 byte file and outputs:
mkvmerge v26.0.0 ('In The Game') 64-bit
'E:\Output\test.ivf': Using the demultiplexer for the format 'IVF (AV1, VP8, VP9)'.
'E:\Output\test.ivf' track 0: Using the output module for the format 'AV1'.
The file 'E:\Output\test.mkv' has been opened for writing.
Error: An exception occurred when writing the destination file. The drive may be full. Exception details: end of file error; No error
Still got 1.3 TB free on that drive.

To be sure this isn't a rav1e problem I also used aomenc for encoding, but got the same error.
(Same problem with: mkvtoolnix-64-bit-26.0.0-revision-021-g39f37fa93-01789)

-> Since the av1 support is still experimental, is there any progress there? (or did I mess up somewhere with the muxing call?)

Cu Selur

Ps.: uploaded the test.ivf to my GoogleDrive (https://drive.google.com/open?id=0B_WxUS1XGCPAUTlILW54VThMTFU).

[Mosu]

mkvmerge's AV1 support isn't usable yet. Neither does it conform to the finalized AV1 bitstream spec, nor had the AV1-in-Matroska specs been finalized when I last worked on that code (which was several weeks ago).

[Selur]

Okay, thanks for the info. Hoped there would have been some progress,.. :)

[Mosu]

I've just released MKVToolNix v27.0.0. This release is smallish but important due to two bug fixes. The first is an annoying crash easily triggered on Windows simply by using the program for a while. The second is indubitably more important: mkvmerge was corrupting E-AC-3 frames when the "dialog normalization gain removal" functionality was used.

Here are the usual links: the MKVToolNix home page (https://mkvtoolnix.download/), the Windows installer/portable version & macOS DMG & Linux AppImage (https://www.fosshub.com/MKVToolNix.html) and the source code (https://mkvtoolnix.download/source.html).

The Windows and macOS binaries as well as the Linux AppImage are available already. The other Linux binaries are still being built and will be available of the course of the next couple of hours.

Here are the NEWS (https://mkvtoolnix.download/doc/NEWS.md) since the previous release:

# Version 27.0.0 "Metropolis" 2018-09-26

## New features and enhancements

* mkvmerge: chapters: the timestamps of chapters read from containers or from chapter files can be adjusted (multiplication and addition) with the new `--chapter-sync` option or using the special track ID `-2` for the existing `--sync` option. Part of the implementation of #2358 (https://gitlab.com/mbunkus/mkvtoolnix/issues/2358).
* MKVToolNix GUI: multiplexer: adjusted & added controls for mkvmerge's new feature of being able to adjust chapter timestamps. Part of the implementation of #2358 (https://gitlab.com/mbunkus/mkvtoolnix/issues/2358).
* MKVToolNix GUI: multiplexer: the GUI can now ask for confirmation when the user is about to create a file that won't contain audio tracks. It does this by default if at least one source file contains an audio track. Implements #2380 (https://gitlab.com/mbunkus/mkvtoolnix/issues/2380).

## Bug fixes

* mkvmerge: AC-3: dialog normalization gain removal was corrupting E-AC-3 frames irreversibly by writing checksums in places where they didn't belong. Additionally only the first E-AC-3 frame in a Matroska was processed but not additional dependent frames in the same block. Fixes #2386 (https://gitlab.com/mbunkus/mkvtoolnix/issues/2386).
* MKVToolNix GUI: fixed a leak of Windows font resources leading to a general slowdown and subsequent crash. Fixes #2372 (https://gitlab.com/mbunkus/mkvtoolnix/issues/2372).

Have fun :)