mkvtoolnix 4.1.1 released

[Isochroma]

810000 bytes is ridiculously small for an entire movie. What is that, 0.81 MB? Movies start at 700MB which would in this case be 0.12%?

For a two-CD size movie, that would be 0.06%, for a 4.7 GB video that would be 0.0172%.

So yes, make the NALU 3.

[honai]

Same here. The overhead of MKV is already very small compared to other containers, so I'm sure we can afford those 810K.

[madshi]

810000 bytes is ridiculously small for an entire movie. What is that, 0.81 MB?
I change my vote to "always 3 byte". When I originally read that 810000 number I was mistakingly thinking of 810 MB, which would have been too much. But 0.81 MB is really too small to worry about.

[Eragon4ever]

What is that, 0.81 MB?
Actually it's just 0,77MB. Just to clear that up.
OK, I'm fine with "2." if the option to set it manually stays.

[Mosu]

OK, I'm fine with "2." if the option to set it manually stays.

The option will definitely stay. If I change it to 3 then I'll also add a message at the end of the muxing if 2 would have sufficed -- so that people who really want the smallest possible size know that they can lower the overhead :)

[bob0r]

beyonce.at.the.bbc.1080mbaff.sample.ts
47.6 MB (49,999,916 bytes)
demuxed .h264 and .mp2 muxed into .mkv works

beyonce.at.the.bbc.1080mbaff.ts
4.19 GB (4,504,797,720 bytes)
demuxed .h264 and .mp2 muxing into .mkv fails.
BBC HD uses H.264 MBAFF video coding.
Also other big captures from BBC-HD fail, this error:

'die' called: common.cpp/saferealloc() called from file src/common/common_memory.cpp, line 33: realloc() returned NULL for a size of 124253 bytes.

[Haali]

Btw, nalu size lentgh of 3 is illegal according to std, and qt decoder doesn't accept such length. Valid values are 1,2,4.

[Mosu]

Btw, nalu size lentgh of 3 is illegal according to std, and qt decoder doesn't accept such length. Valid values are 1,2,4.

Ah crap. This reaaaaaally makes sense... So the question is: change to 4 as the default or not?

BTW: The source code of your muxer you gave me quite a while ago used 3 as the default. I guess you've changed that by now. And MP4Box allows and uses 3, too.

[Mosu]

beyonce.at.the.bbc.1080mbaff.sample.ts
47.6 MB (49,999,916 bytes)
demuxed .h264 and .mp2 muxed into .mkv works

beyonce.at.the.bbc.1080mbaff.ts
4.19 GB (4,504,797,720 bytes)
demuxed .h264 and .mp2 muxing into .mkv fails.
BBC HD uses H.264 MBAFF video coding.
Also other big captures from BBC-HD fail, this error:

'die' called: common.cpp/saferealloc() called from file src/common/common_memory.cpp, line 33: realloc() returned NULL for a size of 124253 bytes.

Any chance you could find a smaller piece of that file for which mkvmerge crashes? If not, would you be willing to upload it to my FTP server? I do have the space and the bandwidth, so it'd be no problem for me.

[foxyshadis]

For x264 and Ateme, it's not much of an issue, since more than 1 or 2 slices is rare (currently impossible for x264). Unless chroma planes are separate slices?

Anyway, I'd go for a default of 4. Maximize compatibility and minimize inconvenience. I'd vote that --nalu 2 throw a warning and simply restart with 4 instead of flat erroring out, if the message is already going to be there. It'd be good for those of us die-hards who make streaming video and want to cut every byte of overhead from our 40kbps streams.

Perhaps this could be integrated with the "always use simpleblock" option. Along with no default header values. Sort of an all-inclusive blood-of-turnip option.

Can it fix the nalu size of premuxed stuff coming from mp4/mkv/avi as well?

(It's still a little too happy to read wmvs as AVC ES. But that's my fault for trying to add one.)

Minor bug report: An extra space between --engage no_default_header_values and --engage native_mpeg4 caused mkvmerge to fail.

[bob0r]

I deleted that source file, i am working on the.recruit.ts 16gb from BBC-HD now, i am trying to cut a none working sample from that.

On my own (clean) captures h264tsto works .ts to .mkv, on this weird cut into 4 pieces .ts it does not, so i hope this will do, else you just need to copy and join the sample as many times till mkvmerge fails :)

I am converting all my .ts files to .mkv, some way or another... i cant stand skipping a .ts and wait 30 seconds for it to continue :O)

Ofcourse i will do this 1x, mirror all files, and never wait again!

Edit:
I am uploading test.h264 842 MB (883,053,919 bytes) to your ftp, this one does this.....
Arg while i am typing this, it did manage to create the file.

So it seems if the file is too big, there is a memory leak or something, you as programmer should be able to make a 4+GB file out of my 50mb sample, just copy copy copy and join them all.
Can't do much more here....


You may delete test.h264

[Mosu]

For x264 and Ateme, it's not much of an issue, since more than 1 or 2 slices is rare (currently impossible for x264). Unless chroma planes are separate slices?

It's not a matter of the number of slices, because each slice is prefixed with the NALU size. Maybe I'm using the term "slice" in the wrong way here, I'm not sure. Maybe I mean that there can be several NALUs for each frame, and each NALU has its own size.

Anyway, I'd go for a default of 4. Maximize compatibility and minimize inconvenience. I'd vote that --nalu 2 throw a warning and simply restart with 4 instead of flat erroring out, if the message is already going to be there.

"Simply restarting" is not as simple :/ Quite a lot of work.

Perhaps this could be integrated with the "always use simpleblock" option. Along with no default header values. Sort of an all-inclusive blood-of-turnip option.

Nah. I'll leave all options separate and not merge them into one "do it this way or the other way" option. But the options will get their own dialog soon.

Can it fix the nalu size of premuxed stuff coming from mp4/mkv/avi as well?

Not yet, but that's easy to implement. I hadn't thought of it before...

(It's still a little too happy to read wmvs as AVC ES. But that's my fault for trying to add one.)

No WMV support in the near future. Sorry.

Minor bug report: An extra space between --engage no_default_header_values and --engage native_mpeg4 caused mkvmerge to fail.

Huh? Extra spaces should be ignored by mkvmerge... OK, maybe mmg is interpreting this as an extra option. Will have to look.

[foxyshadis]

It's not a matter of the number of slices, because each slice is prefixed with the NALU size. Maybe I'm using the term "slice" in the wrong way here, I'm not sure. Maybe I mean that there can be several NALUs for each frame, and each NALU has its own size.
I just meant the added overhead for single- or two-nalu frames will be pretty negligible, sorry, compared to your example.

"Simply restarting" is not as simple :/ Quite a lot of work.
I was thinking, mkvmerge itself would error out with a specific error code, and mmg would go back and feed it the new command line. It seems to be simpler than restarting internally in mkvmerge. If that's what you're thinking, apologies.

[Mosu]

I was thinking, mkvmerge itself would error out with a specific error code, and mmg would go back and feed it the new command line. It seems to be simpler than restarting internally in mkvmerge. If that's what you're thinking, apologies.

That's possible, of course, but there are a lot of folks who use mkvmerge without mmg, and for those this would not be a proper solution.

[madshi]

@Mosu, I feel a bit bad to ask you this, as you've done so much already and you're already looking into TS import (which would be lovely), but have you thought about adding EVOB import yet? I'm asking because I've read in another thread that it's very similar to MPG, just with some additions or something like that. If you don't want to do it, just say no, that's ok. But I thought it wouldn't harm to ask... :)

FWIW, I just dropped a little EVOB with a MPEG2 video stream into mmg and mkvmerge successfully imported the MPEG2 stream into a mkv!!! It just didn't find the audio part. So it seems that mkvmerge already understands EVOB quite a bit, just not completely.

Here's a thread about EVOB demuxing:

http://forum.doom9.org/showthread.php?t=120652&page=5

At page 3 drmpeg offers a little EVOB demuxer. Maybe he'd be willing to spill the beans about what (if any) changes EVOB has to MPG/VOB?

P.S: Actually his EVOB demuxer comes with full source code and it looks VERY simple! :)

[Mosu]

EVOB is definitely not on my top-priority-list for at least the next two months :)

[madshi]

Ok, thanks anyway!

You may ignore this:
(FWIW, my current impression is that EVOB is nothing but MPG. It seems to me that mkvmerge already fully supports EVOB - it just doesn't understand some of the stream IDs. I've just tried the TMPGEnc Demuxer Tool and it shows all streams just fine and can extract them successfully. It seems that 0xBF is Dolby Digital Plus and 0xFD is VC-1.)

[Isochroma]

Sorry to report, but mkvmerge doesn't 'fully' understand EVOBs... mine causes mkvmerge to hang with 100% CPU usage.

[LeMoi]

Error: 'H:\file.mkv' track 1: This AVC/h.264 contains frames that are too big for the current maximum NALU size. You have to re-run mkvmerge and set the maximum NALU size to 3 for this track (command line parameter '--nalu-size-length 1:3').
Nalu case is greyed in MMG :s
EDIT : never mind, i was able to add it "extra options" for the track. Anyway it's not normal that this case is greyed :p

[Mosu]

FWIW, my current impression is that EVOB is nothing but MPG.

Following the discussions and patches on the mplayer mailing lists I think that EVOB is basically VOB with a couple of small but important differences.

Sorry to report, but mkvmerge doesn't 'fully' understand EVOBs... mine causes mkvmerge to hang with 100% CPU usage.

I'm not suprised at all :)

Nalu case is greyed in MMG :s

Ah yes. I think it's only active for AVC ES files at the moment, neither for AVC-in-AVI nor for AVC-in-VfW-mode-in-Matroska. I'll fix that soon.

[Isochroma]

Just to let you know, I'm having the same problem as LeMoi - I have an MKV with VFW track fourcc "H264". I cannot be remuxed with 2.0.0 beause the NALU setting box is grayed out.

When I copy the cmdline and paste into cmdbox with the recommended nalu string added, it doesn't work, complaining about a track missing.

[Mosu]

Also, I've encountered two more .mp4 files that after muxing won't play (stuck on frame 0, won't seek).

Upload, please :)

[Yong]

Mosu, i have a good thing for you:p

mkvtoolnix tell me this sample is h264 es...
its actually is png/qdm2 mov.
http://www.shynola.com/movies/j_s/moveyourfeet.mov

more info about the mov read this thread:)
http://forum.doom9.org/showthread.php?p=926535#post926535

[Mosu]

Mosu, i have a good thing for you:p

mkvtoolnix tell me this sample is h264 es...
its actually is png/qdm2 mov.
http://www.shynola.com/movies/j_s/moveyourfeet.mov

Hmm. Ok, I've fixed the file type detection issue, so the file is recognized as QT/MP4 now. But PNG video tracks are not supported, and I won't add support for them now. So you'd only be able to mux the audio track.

[Yong]

Hmm. Ok, I've fixed the file type detection issue, so the file is recognized as QT/MP4 now. But PNG video tracks are not supported, and I won't add support for them now. So you'd only be able to mux the audio track.

Great, i didnt expect you to add support for it, just a bug report;)
and thx for the hard work.

[Isochroma]

I have an AVI file which has 119.880 fps drop-frame H264 inside. Using previous versions of MKVMerge, I could mux into MKV using the VFW override and it would work great.

The current version can't mux the source because the NALU needs to be set larger.

But I'm wondering, how will MKVMerge deal with the 119.880 drop frames? Because the old way it was stored, the drop frames were just preserved in the VFW-mode structure, right?

But the new native format for AVC will require that those drop frames be removed, because there is no drop-frames outside of VFW tracks, right?

Personally, I'd prefer if the next version of MKVMerge restored the ability to mux H264 from VFW mode to VFW-mode tracks. The reason is until the native support is debugged, there is no way to mux certain VFW mode tracks, and it is uncertain what will happen to drop frames. New versions have become less useful by omitting the VFW function, such that they cannot be used on a significant fraction of files of this type.

This kind of gap is bound to happen in such rapid developement, of course. Rest assured I have faith that soon, such cases will be handled properly. Blame for these cases ultimately reverts to those who used VFW mode for H264 tracks. MP4 and native AVC mode in MKV are a kind of salvation for this sin; and as priests, MKVMerge and MP4Box can provide redemption, perhaps even salvation. Thus it is easy to understand why the AVC-in-VFW option was removed: to prevent the passing on into new files of corruption, a corrupt practice or way of doing something.

I also tried the latest MP4Box GUI YAMB; it reported an unspecified error and failed. So I guess right now MKVMerge is the last hope for this type of file...

[Mosu]

But I'm wondering, how will MKVMerge deal with the 119.880 drop frames? Because the old way it was stored, the drop frames were just preserved in the VFW-mode structure, right?

Wrong. mkvmerge has always removed zero-sized entries.

But the new native format for AVC will require that those drop frames be removed, because there is no drop-frames outside of VFW tracks, right?

They should be removed, yes.

Personally, I'd prefer if the next version of MKVMerge restored the ability to mux H264 from VFW mode to VFW-mode tracks.

This will definitely not happen. The vfw-mode storage has always been the wrong thing to do, and I only left it in because mkvmerge couldn't handle such files any other way at that point.

You can try this build: http://www.bunkus.org/videotools/mkvtoolnix/win32/pre/mkvtoolnix-unicode-2.0.0-build20070201-1.rar

It defaults to a NALU size of 4 and mmg allows the selection of the NALU size if the AVC track comes from an AVI or VFW-mode Matroska file.

BTW: If this build does not work right with your file then I'd be very happy if you uploaded it to my FTP server. I don't have a single 119.880 FPS AVI file, especially not with h264 inside.

[Milvus]

But I'm wondering, how will MKVMerge deal with the 119.880 drop frames? Because the old way it was stored, the drop frames were just preserved in the VFW-mode structure, right?


These 119.88fps pseudo-VFR AVI files are quite an abomination, and storing them as is in MKV is definitely a wrong idea. It can lead to jerky playback on some computer.

The right thing to do was to use avi_tc_package (http://web.missouri.edu/~kes25c/#c3) to convert the original AVI file to a CFR AVI without dropframes and a timecode file. MKVmerge can then use them to produce a true MKV VFR with native AVC.

But now that MKVmerge can handle them correctly, I suppose there's no further need to complicate our life. We have true MKV VFR with native AVC directly !

[dragonle87]

Hello, forgive me if this is the wrong thread to post this but I would like to request two small features be implemented:

1) Can you make it so that users can select which "command line options" to always be enabled by default, instead of having to manually select it from the "Muxing" menu? It's because I always use "--engage native_mpeg4" for every file that I muxed to be clean of avi compatibility hacks. I got this idea when I saw you have the checkbox option "Always use simple blocks" under the "Settings" tab.

2) Another thing i regularly do when muxing files to mkv is naming it in the "file/segment title" textbox under the "Global" tab. The only thing missing is an additional separate "creator/author" textbox and I was wondering if you could add that in your next version. Having a "title" and "author" textboxes is really useful for providing very basic file information. I find this simple, fast, and convenient rather than going through the trouble of creating tags, which I think should be reserved for those users who want to provide more detailed file information.

Most of all, keep up the good work, your program just rocks. As an avid mkv user, I really enjoy using it...very easy and straightforward at creating mkv files.

[LeMoi]

2) Another thing i regularly do when muxing files to mkv is naming it in the "file/segment title" textbox under the "Global" tab. The only thing missing is an additional separate "creator/author" textbox and I was wondering if you could add that in your next version. Having a "title" and "author" textboxes is really useful for providing very basic file information. I find this simple, fast, and convenient rather than going through the trouble of creating tags, which I think should be reserved for those users who want to provide more detailed file information.
+1 :)

[Mosu]

1) Can you make it so that users can select which "command line options" to always be enabled by default, instead of having to manually select it from the "Muxing" menu? It's because I always use "--engage native_mpeg4" for every file that I muxed to be clean of avi compatibility hacks. I got this idea when I saw you have the checkbox option "Always use simple blocks" under the "Settings" tab.

I can certainly do this. I've been planning to move the "settings" tab to its own dialog because it's growing with each release. In such a dialog I'll also add an input box in which the user can set default options which will be added to the command line.

2) Another thing i regularly do when muxing files to mkv is naming it in the "file/segment title" textbox under the "Global" tab. The only thing missing is an additional separate "creator/author" textbox and I was wondering if you could add that in your next version. Having a "title" and "author" textboxes is really useful for providing very basic file information. I find this simple, fast, and convenient rather than going through the trouble of creating tags, which I think should be reserved for those users who want to provide more detailed file information.

I can do that as well. This information would still have to be put into the tags because there's no other place in a Matroska file for this kind of information. The segment title is different; there's an element for this in the segment headers. But it would at least be a convenient way to add some basic information. Maybe I'll offer a complete tab "basic tags" or something like this... I'll have to think about it.

Note that neither change will happen soon. I'm also in the process of rewriting mmg using the Qt toolkit, and this takes precedence over huge changes to the current GUI.

Most of all, keep up the good work, your program just rocks. As an avid mkv user, I really enjoy using it...very easy and straightforward at creating mkv files.[/QUOTE]

[Isochroma]

@millvus: I already tried the avc_tc package. It was time-consuming, and no matter what I did the end result was a file that didn't play right - the audio was always going out of sync.

@Mosu: will give it a try!

[Okraml]

Will there be VC1 support annytime soon?
Then mkvtoolnix would be the first application for muxing vc1 (demuxed from evob) to a new container (maybe with another-language audio stream).

:-) Okraml

[LeMoi]

I was muxing a file with mmg and it fails with code 2 errors, without displaying the error message. I had to use command line to see it :mkvmerge v2.0.0 ('After The Rain Has Fallen') built on Jan 17 2007 10:43:41

Error: Error: Simple chapter parser: 'CHAPTER01NAME=Chapitre n
Shouldn't the GUI display it ?

[Mosu]

Will there be VC1 support annytime soon?
Then mkvtoolnix would be the first application for muxing vc1 (demuxed from evob) to a new container (maybe with another-language audio stream).

:-) Okraml

Hmm I don't have any information about VC1, nor sample files, nor is there a CodecId for it... So... Probably not.

[Mosu]

I was muxing a file with mmg and it fails with code 2 errors, without displaying the error message. I had to use command line to see it :
Shouldn't the GUI display it ?

Hmm it should, yes... I'll investigate.

[Okraml]

Hmm I don't have any information about VC1, nor sample files, nor is there a CodecId for it... So... Probably not.

Would it help, if i would upload sample files?

A working ID could be WVC1. At least thats what mkvtoolnix recognized in this case (http://forum.doom9.org/showthread.php?p=946196#post946196).
(I could even upload one of the files mentioned in the post)

:-) Okraml

[Mosu]

Samples files always help. I'll also need a way to play back the files. Meaning a decoder and some demuxer that can handle evobs. I think mplayer cannot really do that yet, even with current SVN...

[Mosu]

Oh, AND I need files with LOWER resolutions. My computer cannot handle HDTV resolutions.

[Okraml]

Oh, AND I need files with LOWER resolutions. My computer cannot handle HDTV resolutions.

You don't need HDTV resolutions to be able to just play the files. I could play 1080i on a 17inch display with a resolution of 1280x1024.

:-) Okraml

[Mosu]

Well, what I meant is that my PC can only decode 1900x1200 resolutions with 3 or 4 frames per second :) Definitely not enough to see if the muxed file is OK or not.

[Okraml]

Well, what I meant is that my PC can only decode 1900x1200 resolutions with 3 or 4 frames per second :) Definitely not enough to see if the muxed file is OK or not.

Unfortunately i got no vc1 file with lower resolution. I could test for you, if you can get mkvtoolnix to mux a whole file (for now it doesn't recognize the file and with haali matroska muxer i can't get a whole file muxed neither).

:-) Okraml

[Isochroma]

The new build works great; one minor item I noticed, however, is that the muxed file shows in its properties (MatroskaProp) 119.880127 fps.

I have done some VFR MP4 to MKV muxing before, and in those cases the fps shows as something like 27.2 or similar, so I was wondering why it still shows 119.880127?

[Eragon4ever]

I just experimented a bit with "--engage native_mpeg4".
Some of my files get out of sync after muxing them with "--engage native_mpeg4" enabled and at least one seams to drop some frames. When remuxing the mkvs both show a warning like this:

pr_generic.cpp/generic_packetizer_c::add_packet(): timecode < last_timecode (00:00:00.083 < 00:00:00.167) for 1 of 'N:\Bearbeiten\Death Note\[subs4u]_DeathNote_01HQ_(Xvid,ger.sub)[F53CD5E2].mkv'. This should not have happened.

No problem with vfw.

I'll upload a sample if needed.

[Mosu]

Unfortunately i got no vc1 file with lower resolution. I could test for you, if you can get mkvtoolnix to mux a whole file (for now it doesn't recognize the file and with haali matroska muxer i can't get a whole file muxed neither).

:-) Okraml

I've downloaded a couple of samples from mplayer's server. Maybe I can start with those.

Note: Don't hold your breath. This will take lots of time.

[Mosu]

The new build works great; one minor item I noticed, however, is that the muxed file shows in its properties (MatroskaProp) 119.880127 fps.

I have done some VFR MP4 to MKV muxing before, and in those cases the fps shows as something like 27.2 or similar, so I was wondering why it still shows 119.880127?

mkvmerge just uses the AVI's FPS field. It doesn't re-calculate or guess the actual number of frames that aren't dropped. For MP4 files it's similar... Only MP4 files don't really have a FPS field; they have something similar to Matroska's default duration.

[Mosu]

I just experimented a bit with "--engage native_mpeg4".
Some of my files get out of sync after muxing them with "--engage native_mpeg4" enabled and at least one seams to drop some frames. When remuxing the mkvs both show a warning like this:

pr_generic.cpp/generic_packetizer_c::add_packet(): timecode < last_timecode (00:00:00.083 < 00:00:00.167) for 1 of 'N:\Bearbeiten\Death Note\[subs4u]_DeathNote_01HQ_(Xvid,ger.sub)[F53CD5E2].mkv'. This should not have happened.

No problem with vfw.

I'll upload a sample if needed.

Yeah, a sample file would be nice. Maybe it's time to fix native MPEG4 mode for good.

[Eragon4ever]

Yeah, a sample file would be nice. Maybe it's time to fix native MPEG4 mode for good.
Sample is up (DeathNote_01HQ).

And an other thing: Once you have muxed an H.264 AVI with NALU 3 you have to demux it first to set the NALU to 4 in case the decoder does not work with 3 (maybe command line works to but not with GUI). It would be nice to be able to set the NALU for mkv (and mp4), too.

[Isochroma]

I've noticed that the same mpeg-4 asp .avi (dx50, xvid, etc.) muxed in mkv with native and vfw modes, uses more cpu on playback in native mode than vfw mode.

[Mosu]

Sample is up (DeathNote_01HQ).

Thanks.

And an other thing: Once you have muxed an H.264 AVI with NALU 3 you have to demux it first to set the NALU to 4 in case the decoder does not work with 3 (maybe command line works to but not with GUI). It would be nice to be able to set the NALU for mkv (and mp4), too.

Hmmmmmmmmmmm I see your point. Might not be that hard to implement.