mkvtoolnix 4.1.1 released

[bob0r]

Its the same .ts source as that sample

This is from BBC-HD also..... only difference: Its 13GB!

So thats the whole problem, same with Haali's dsmuxer... everything under around 4-5GB works fine, all above that goes wrong.

luxe.hd.ateme.ts and bbc.hd.ts <-- both work fine as 20mb files.

Note: NEW added sample was: luxe.hd.ateme.ts this is also H.264 MBAFF, the .ts can not be opened by elecard xmuxer pro, but mplayer was able to extract the raw.264 and raw.ac3

So in that case, you should still have the sample links.... get the raw h.264, and maybe you can figure out why a large file fails.....



.... Guess i got enough bandwidth anyway:
http://x264.nl/h.264.samples/

[madshi]

There will be no support for DD+ for the time being because a) I cannot find any specs, b) I don't have any means of playing back such a file (neither hardware nor software).
The spec is available here, I think:

http://www.atsc.org/standards/a52.html

Also I can send you an example DD+ stream, and I can get you going with playback though DirectShow, if you want. What do you think?

But maybe you want to wait until ffmpeg supports DD+? Don't know if that would save you time...

[Dr.Khron]

MKVMerge GUI is causing my Windows Explorer to crash!

Pretty much anytime I mux an MKV, halfway through the process, my Windows Explorer crashes, closing all of my open windows, and making most of my system tray dissapear. The only fix is to reboot.

Seems like a conflict of some kind... any ideas what might be conflicting with MKVMerge GUI?

[Mosu]

Neither mmg nor mkvmerge is doing anything low level or interacting directly with the explorer. They only read and write files. What I could think of is that mkvmerge allocates too much memory which would indicate a memory hole somewhere. Maybe you could monitor memory usage during muxing with the task manager.

[Mosu]

The spec is available here, I think:

http://www.atsc.org/standards/a52.html

Also I can send you an example DD+ stream, and I can get you going with playback though DirectShow, if you want. What do you think?

But maybe you want to wait until ffmpeg supports DD+? Don't know if that would save you time...

Haali has also sent me the specs. Sample files are always welcome. DirectShow playback is nice to have, but not exactly helping because I usually develop on Linux. Developing on Windows is always a lot slower for me.

[foxyshadis]

MKVMerge GUI is causing my Windows Explorer to crash!

Pretty much anytime I mux an MKV, halfway through the process, my Windows Explorer crashes, closing all of my open windows, and making most of my system tray dissapear. The only fix is to reboot.

Seems like a conflict of some kind... any ideas what might be conflicting with MKVMerge GUI?
I bet you have some kind of shell extension that monitors mkv files, and crashes when it tries to read it while it's still being written. Probably matroskaprop, possibly haali's, but I don't think his is that badly behaved.

Check with this:
http://www.nirsoft.net/utils/shexview.html

[Mosu]

Something for you guys to play with:

2007-03-02 Moritz Bunkus <moritz@bunkus.org>

* mkvmerge: new feature: Added support for EAC3 tracks in MPEG
program streams.

* mkvmerge: new feature: Added support for EAC3/DD+ (Dolby Digital
Plus) files and tracks (raw EAC3 files or inside Matroska with
CodecID A_EAC3).

http://www.bunkus.org/videotools/mkvtoolnix/win32/pre/mkvtoolnix-unicode-2.0.2-build20070302-1.rar

Note that I still don't have a working playback solution here, so this is reeeeally untested. You will need the latest version of Haali's splitter of course.

[madshi]

Thanks very much - Mosu!!! :)

I've just tested to mux an external EAC3 track into an already existing MKV file which was created by Haali's Matroska Muxer. Here are my results:

(1) I'm once again getting a few of the following warnings:

"Warning: pr_generic.cpp/generic_packetizer_c::add_packet(): timecode < last_timecode (00:59:30.600 < 00:59:30.667) for 1 of 'D:\test.mkv'. This should not have happened. Please contact the author with this error/warning message, a description of what you were trying to do, the command line used and which operating system you are using. Thank you."

Interestingly the first warning is at *exactly* the half of the movie (but not at the timecode where the two EVOs were splitted).

Mosu, does this warning have any practical meaning? Does it mean I should trash the new MKV because there's a glitch in it? If yes, is the glitch only in the new MKV or also in the original one? FWIW, I've checked the video at the 3 places the warnings mentioned and didn't notice any glitches. Does that mean I can safely ignore the warnings?

Besides, I'm always getting these warnings, if I remux one of the MKV files which were created by EVO -> Haali's Matroska Muxer. So this is not a problem related to the new EAC3 support. Maybe Haali's Matroska Muxer does something wrong?

(2) Muxing of the EAC3 track generally seems to work, at least I got no crashes, the file size seems to be right and the track is properly listed in MPC. Unfortunately I cannot really test whether the sound plays, because Haali's Media Splitter reports the EAC3 track in such a way that none of my EAC3 capable DirectShow filters can connect. I guess that's a problem in the Haali Media Splitter, cause the same thing happens if I use Haali's Media Splitter directly on EVO files or on MKV files with EAC3 in them created by Haali's Matroska Muxer.

[madshi]

P.S: I'm not sure whether 0xeac3 is the correct way to sign the track. I think rather not.

[Henrikx]

THX Mosu !!!

[Mosu]

(1) I'm once again getting a few of the following warnings:

"Warning: pr_generic.cpp/generic_packetizer_c::add_packet(): timecode < last_timecode (00:59:30.600 < 00:59:30.667) for 1 of 'D:\test.mkv'. This should not have happened. Please contact the author with this error/warning message, a description of what you were trying to do, the command line used and which operating system you are using. Thank you."

Interestingly the first warning is at *exactly* the half of the movie (but not at the timecode where the two EVOs were splitted).

Mosu, does this warning have any practical meaning? Does it mean I should trash the new MKV because there's a glitch in it? If yes, is the glitch only in the new MKV or also in the original one? FWIW, I've checked the video at the 3 places the warnings mentioned and didn't notice any glitches. Does that mean I can safely ignore the warnings?

Yes, if there are no problems watching the file (these warnings only apply to video tracks) then you can ignore them. Actually I've disabled the warning this afternoon (after having uploaded the current build though). It was meant for myself for detecting errors in mkvmerge. However, due to the nature of video track timecodes especially with advanced codecs this warning is more often simply bogus and can be ignored. It usually unsettles users, that's why I'm removing it.

Besides, I'm always getting these warnings, if I remux one of the MKV files which were created by EVO -> Haali's Matroska Muxer. So this is not a problem related to the new EAC3 support. Maybe Haali's Matroska Muxer does something wrong?

Not necessarily :)

(2) Muxing of the EAC3 track generally seems to work, at least I got no crashes, the file size seems to be right and the track is properly listed in MPC. Unfortunately I cannot really test whether the sound plays, because Haali's Media Splitter reports the EAC3 track in such a way that none of my EAC3 capable DirectShow filters can connect. I guess that's a problem in the Haali Media Splitter, cause the same thing happens if I use Haali's Media Splitter directly on EVO files or on MKV files with EAC3 in them created by Haali's Matroska Muxer.

Ok, thanks for the feedback. I can use graphedit and connect Haali's splitter with the Intervideo Audio Decoder and with the Dump filter; this works with "EVO-to-MKV-with-Haali" file as well as with a "EAC3-to-MKA-with-mkvmerge" file. The results are byte identical, so I'm pretty confident that I haven't screwed up completely ;)

P.S: I'm not sure whether 0xeac3 is the correct way to sign the track. I think rather not.

I don't quite understand what you mean.

[madshi]

Yes, if there are no problems watching the file (these warnings only apply to video tracks) then you can ignore them. Actually I've disabled the warning this afternoon (after having uploaded the current build though). It was meant for myself for detecting errors in mkvmerge. However, due to the nature of video track timecodes especially with advanced codecs this warning is more often simply bogus and can be ignored. It usually unsettles users, that's why I'm removing it.
Thank you for the explanation.

Ok, thanks for the feedback. I can use graphedit and connect Haali's splitter with the Intervideo Audio Decoder and with the Dump filter; this works with "EVO-to-MKV-with-Haali" file as well as with a "EAC3-to-MKA-with-mkvmerge" file. The results are byte identical, so I'm pretty confident that I haven't screwed up completely ;)
I believe the Dump filter eats whatever you throw at it... :) But I can not for the life of me connect Haali's splitter with the Intervideo Audio Decoder. Maybe you have a newer version of the Intervideo Audio Decoder than I have? Or do you have a non-public newer build of Haali's splitter?

I don't quite understand what you mean.
If I try to open a MKV file with an EAC3 track in it, MPC sais this:

"Media Type 0:
--------------------------
Audio: 0xeac3 48000Hz 6ch"

I thought "0xeac3" was what Haali reported as the audio format. I have a filter which can connect demuxed EAC3 files to the Intervideo Audio Decoder and I believe to remember that I saw it reporting 0xac3 and not 0xeac3. But I might be totally wrong. I'm not really an expert in this area...

[Mosu]

I believe the Dump filter eats whatever you throw at it... :) But I can not for the life of me connect Haali's splitter with the Intervideo Audio Decoder. Maybe you have a newer version of the Intervideo Audio Decoder than I have? Or do you have a non-public newer build of Haali's splitter?

Yeah, though I think that Haali only changed a bug with the Matroska CodecID in it. It's http://haali.cs.msu.ru/mkv/mkx.I.2.exe The InterVideo AD is the one you've sent me.

[madshi]

Yeah, though I think that Haali only changed a bug with the Matroska CodecID in it. It's http://haali.cs.msu.ru/mkv/mkx.I.2.exe
Great - thanks for the link!

I can now confirm that EAC3 support in your latest mkvtoolnix version is fully working! At least everything works fine in my first test with one movie.

:thanks:

[madshi]

* mkvmerge: new feature: Added support for EAC3 tracks in MPEG program streams.
I'm wondering: Does that mean that EVO is "officially" supported by mkvtoolnix now?

[Mosu]

No :) Neither VC1 nor h264 can be read from MPEG program streams yet.

[madshi]

No :) Neither VC1 nor h264 can be read from MPEG program streams yet.
Ok, but are MPEG2 and AC3 and EAC3 officially supported in EVO files? I'm asking because after reading your comment about adding EAC3 support to MPEG program streams, yesterday I tried to drop a little MPEG2/AC3 evo into mmg and it seemed to work just fine. However, the resulting MKV was too short. It was only 2-3 minutes instead of 6 minutes and the file size was also short.

Anyway, if this is not supposed to work yet, just ignore my post... :)

[Mosu]

I know that there are several issues with the MPEG PS reader in mkvtoolnix. I want to improve it during the next days. The goal is definitely to have at least MPEG-2 and EAC3 working properly, maybe even VC-1.

[madshi]

Thank you. Your continued work on mkvtoolnix is greatly appreciated!

[Mosu]

BTW: Could you please upload that EVO file? I definitely need more test files, especially ones on which mkvmerge chokes.

[madshi]

BTW: Could you please upload that EVO file? I definitely need more test files, especially ones on which mkvmerge chokes.
The Departed Trailer from here stalls mkvmerge:

ftp://mplayerhq.hu/MPlayer/samples/evob/

I'll see if I can find more problematic evo files.

[Mosu]

I know that the Departed trailer chokes mkvmege; I'm working on fixing it. Departed is also a VERY strange file because the packet IDs indicate "MPEG-2 video" but there's no sequence header found. Not even mplayer can play that file.

[Haali]

It has H.264 in stream 0xe2.

[Gusar]

Not even mplayer can play that file.

It can:mplayer -psprobe 100000 Deprated\ Trailer.EVO

[Mosu]

I see. Thanks for the info, that should make fixing mkvmerge easier :)

[DeepBeepMeep]

There is a tremendous opportunity to boost even more Matroska polularity at the moment since it is the only container that can easily contain VC1 and DD+ thanks to recent progress made on mkvtoolnix and Haali filter. With a complete and simple solution to backup HDDVD and BluRay the Divx success is not far way...

Potential bug report:
-------------------

I don't know if this a bug. I have used HDdVD Evo demux on the second part file of the HDDVD Total Recall to extract the DTS HD track. If I try then to mux it inside a mkv file use mkvtoolnix.
First I have got tons of "Warning: dts_packetizer: skipping XXX bytes " which is mkvtoolnix ignoring the HD part of the DTS track. That's fine with me since there are so many of them the warning log window which keeps all the history gets slower and slower. There should be a way to ignore these types of warning. In order to obtain a fast mux I use the command line.

Then 2/3 in the process I have the warning below:

----------------
...
Warning: dts_packetizer: skipping 780 bytes (no valid DTS header found). This mi
ght make audio/video go out of sync, but this stream is damaged.
DTS header information changed! - New format:
DTS Frame Header Information:
Frame Type : normal
CRC available : no
Frame Size : PCM core samples=32*97=3104, 0.000723 milliseconds, 204
8 byte
Audio Channels : -1 ES, arrangement: unknown (user defined)
Core sampling frequency: 4294967295
Transmission_bitrate : 32000
Embedded Down Mix : no
Embedded Dynamic Range : no
Embedded Time Stamp : no
Embedded Auxiliary Data: no
HDCD Master : yes
Extended Coding : yes, but unknown
Audio Sync in sub-subs : yes
Low Frequency Effects : yes, interpolation factor 64
Predictor History used : no
Multirate Interpolator : non perfect
Encoder Software Vers. : 7
Copy History Bits : 2
Source PCM Resolution : 16
Front Encoded as Diff. : yes
Surr. Encoded as Diff. : yes
Dialog Normaliz. Gain : -13
Warning: dts_packetizer: skipping 794 bytes (no valid DTS header found). This mi
ght make audio/video go out of sync, but this stream is damaged.
Warning: 'E:\L1_mainMovie.DTSHD.stream.01.mpa' track 0: The current packet's tim
ecode is smaller than that of the previous packet. This usually means that the s
ource file is a Matroska file that has not been created 100% correctly. The time
codes of all packets will be adjusted by 1542692ms in order not to lose any data
. This may throw A/V sync off, but that can be corrected with mkvmerge's "--sync
" option. If you already use "--sync" and you still get this warning then do NOT
worry -- this is normal. If this error happens more than once and you get this
message more than once for a particular track then either is the source file bad
ly mastered, or mkvmerge contains a bug. In this case you should contact the aut
hor Moritz Bunkus <moritz@bunkus.org>.
DTS header information changed! - New format:
DTS Frame Header Information:
Frame Type : normal
CRC available : no
Frame Size : PCM core samples=32*16=512, 10.666667 milliseconds, 201
2 byte
Audio Channels : 5, arrangement: C, L, R, SL, SR (center, left, right, s
urround-left, surround-right)
Core sampling frequency: 48000
Transmission_bitrate : 1536000
Embedded Down Mix : no
Embedded Dynamic Range : no
Embedded Time Stamp : no
Embedded Auxiliary Data: yes
HDCD Master : no
Extended Coding : no
Audio Sync in sub-subs : yes
Low Frequency Effects : yes, interpolation factor 64
Predictor History used : yes
Multirate Interpolator : non perfect
Encoder Software Vers. : 7
Copy History Bits : 1
Source PCM Resolution : 16
Front Encoded as Diff. : no
Surr. Encoded as Diff. : no
Dialog Normaliz. Gain : 0
Warning: dts_packetizer: skipping 730 bytes (no valid DTS header found). This mi
ght make audio/video go out of sync, but this stream is damaged.

--------------



mkvtoolnix has incorrectly detected a change in the DTS audio track, then it switches back but add a huge delay. The consequence is that playback after two thirds of the file gives no sound and opening the mkv file takes much more time.

It could be a glitch with HD DVD Evo Demux or a bug in the detection algorithm of mkvtoolnix which instead of skipping the HD part of the DTS stream interprets it as an audio track change.

In any case I think an easy solution could be to ignore audio track change with incorrect audio channels (here -1), this would make mkvtoolnix much more resilient to damage sources.

Anyway, thanks for your great work

[Mosu]

2007-03-05 Moritz Bunkus <moritz@bunkus.org>

* mkvmerge: new feature: Added support for handling AVC/h.264
tracks in MPEG program streams.

http://www.bunkus.org/videotools/mkvtoolnix/win32/pre/mkvtoolnix-unicode-2.0.2-build20070305-1.rar

At least it works with 'Departed Trailer.evo'.

[Mosu]

I don't know if this a bug. I have used HDdVD Evo demux on the second part file of the HDDVD Total Recall to extract the DTS HD track. If I try then to mux it inside a mkv file use mkvtoolnix.

mkvmerge does not yet support DTS HD tracks. If it thinks that it is DTS then you're on your own at the moment.

First I have got tons of "Warning: dts_packetizer: skipping XXX bytes " which is mkvtoolnix ignoring the HD part of the DTS track. That's fine with me since there are so many of them the warning log window which keeps all the history gets slower and slower. There should be a way to ignore these types of warning. In order to obtain a fast mux I use the command line.

You should get the best speed by redirecting the output to another file, e.g.

mkvmerge ... -o output.mkv input.evo ... > log.txt

While I agree that a feature to limit such a flood of message would be desirable I won't spend time on it. The little time I do have goes to improving EVO support in general.

Then 2/3 in the process I have the warning below:
...
Warning: 'E:\L1_mainMovie.DTSHD.stream.01.mpa' track 0: The current packet's timecode is smaller than that of the previous packet.
...
mkvtoolnix has incorrectly detected a change in the DTS audio track, then it switches back but add a huge delay. The consequence is that playback after two thirds of the file gives no sound and opening the mkv file takes much more time.

Hmm, interesting. Can you upload that file to my FTP server? The problem with EVO support is that I have virtually no access to such files except for the ones you guys provide.

[Mosu]

* mkvmerge: enhancement: Fixed the MPEG PS reader so that it will just skip blocks whose headers it cannot parse instead of aborting.

http://www.bunkus.org/videotools/mkvtoolnix/win32/pre/mkvtoolnix-unicode-2.0.2-build20070305-2.rar

Madshi: This build works with DELSCENE4.EVO. Could you please test it against the other files you have?

[madshi]

http://www.bunkus.org/videotools/mkvtoolnix/win32/pre/mkvtoolnix-unicode-2.0.2-build20070305-2.rar

Madshi: This build works with DELSCENE4.EVO. Could you please test it against the other files you have?
Works with all 3. Thanks for the speedy fix... :)

[madshi]

http://www.bunkus.org/videotools/mkvtoolnix/win32/pre/mkvtoolnix-unicode-2.0.2-build20070305-1.rar

At least it works with 'Departed Trailer.evo'.
Thank you!! :) Am just trying that with Equilibrium. Here's how it behaves:

(1) Detects video and audio just fine.
(2) When starting to mux, there's a freeze for half a minute or so.
(3) Then muxing starts and looks promising.
(4) There's a long freeze after 500MB are written.
(5) It continues after a while, but the GUI is always not responding and progress is extremely slow. Now at 750MB after several minutes. Normally it just takes a few seconds to get that far.

Basically it seems to work, but it's extremely slow and the GUI is not responding at all.

I'm sorry, but I cannot send you this evo, cause it's a real movie (would be illegal to send you) and besides it's 17GB.

[DeepBeepMeep]

Mosu,

I have uploaded the DtsHD track on your ftp site. It is a huge file named "DtsHDTrack.dts". It seems something went wrong and obviously only 90% of the file is there. Hopefully, the glitch is there and should occur towards the end of the file. I happened when I had muxed it with a mkv file that contained already a VC1 file (generated with Haali Filter).

Thanks for your help

[Mosu]

Thank you!! :) Am just trying that with Equilibrium. Here's how it behaves:

(1) Detects video and audio just fine.
(2) When starting to mux, there's a freeze for half a minute or so.
(3) Then muxing starts and looks promising.
(4) There's a long freeze after 500MB are written.
(5) It continues after a while, but the GUI is always not responding and progress is extremely slow. Now at 750MB after several minutes. Normally it just takes a few seconds to get that far.

Basically it seems to work, but it's extremely slow and the GUI is not responding at all.

I'm sorry, but I cannot send you this evo, cause it's a real movie (would be illegal to send you) and besides it's 17GB.

What kind of a video track is this? MPEG-2? I know that both MPEG-1/2 and AVC can be slow, and I've witnessed them being especially slow for EVO files. I will probably have to rewrite the complete MPEG-1/2 code. While it uses a nice object oriented approach it is dead slow ;)

That the GUI is not responding is indeed bad, but I won't fix this completely (way too much work to figure out how to do this... multithreading etc etc, a lot of topics that I haven't covered with wxWidgets before).

[madshi]

What kind of a video track is this?
It's H.264/AVC.

That the GUI is not responding is indeed bad, but I won't fix this completely (way too much work to figure out how to do this... multithreading etc etc, a lot of topics that I haven't covered with wxWidgets before).
Moving the GUI to another thread would probably be quite complicated. But I think moving just the mkvmerge encapsulation to a secondary thread should be rather easy. With win32 APIs (using Send/PostMessage for thread safe communication between the threads) at least it would.

But anyway, it's not really necessary IMHO. The responsiveness it normally quite acceptable. It's only this specific case with the 17GB H.264 EVO where the responsiveness was not so good, anymore.

[madshi]

Let mkvmerge run through Equilibrium last night. Today morning the progress bar had stopped at 65% and the status window sais:

mkvmerge v2.0.2 ('You're My Flame') built on Mar 5 2007 13:45:56
'D:\Equilibrium.evo': Using the MPEG PS demultiplexer.
'D:\Equilibrium.evo' track 0: Using the MPEG-4 part 10 ES video output module.
The file 'D:\Equilibrium.mkv' has been opened for writing.
'die' called: common.cpp/safemalloc() called from file src/common/mpeg4_common.cpp, line 1046: malloc() returned NULL for a size of 139610 bytes.

[Mosu]

Let mkvmerge run through Equilibrium last night. Today morning the progress bar had stopped at 65% and the status window sais:

'die' called: common.cpp/safemalloc() called from file src/common/mpeg4_common.cpp, line 1046: malloc() returned NULL for a size of 139610 bytes.

Hmm, sounds like a serious memory leak. I'll have to see if I can find some other EVO which triggers this memory leak.

[DeepBeepMeep]

In order to give you idea how the DTSHD track I mentioned before was mixed in the original EVO file I have uploaded a short sample of the EVO file (VC1WithDTSHD.EVO)

It is an interesting file as wells since mkvtoolnix won't see any track in it for the moment while there are in fact one video VC1 track and three DTSHD tracks (one is 5.1 and two 2.0)

[Mosu]

In order to give you idea how the DTSHD track I mentioned before was mixed in the original EVO file I have uploaded a short sample of the EVO file (VC1WithDTSHD.EVO)

Thanks for the upload.

It is an interesting file as wells since mkvtoolnix won't see any track in it for the moment while there are in fact one video VC1 track and three DTSHD tracks (one is 5.1 and two 2.0)

Quite natural considering mkvmerge supports neither VC1 nor DTSHD at the moment :)

[DeepBeepMeep]

Thanks for the upload.



Quite natural considering mkvmerge supports neither VC1 nor DTSHD at the moment :)

My pleasure. If this can give some inspiration on how to support demuxing EVO with VC1 (which is the video codec the most used with HDDVD) this would make turning an EVO into a MKV file only a one click process.

On a side note, when mkvtoolnix parses an EVO file and finds some DD+ tracks for instance it seems to order them randomly. This can be a bit confusing and can cause trouble when trying to merge two evo files using mkvtoolnix, it seems there is a proper order (based on the PID?) that can be detected by EvoDemux.exe (http://pel.hu/down/EVOdemux.exe) or by evob_demux (http://www.w6rz.net/evob_demux.zip ).

[ToS_Maverick]

@mosu

got another issue for you, hope you have some time to look at it ;)

my workflow as described in another thread (http://forum.doom9.org/showthread.php?p=966425#post966425) is:
Pro7 HD Transport Stream => Demux with Graphedit (Haali Media Splitter => Haali Matroska Muxer) => spilt with mkvmerge GUI (the "unwanted" scenes)

mkvmerge commanline looks like this:

"mkvmerge" -o "F:\Video\DVB\Baby\baby test.mkv" --language 1:eng --default-track 1:yes --display-dimensions 1:1920x1080 --language 2:eng --default-track 2:yes -a 2 -d 1 -S "F:\Video\DVB\Baby\baby.mkv" --track-order 0:1,0:2 --split timecodes:73s,1806s,2365s,3996s,4617s,5800s,6399s,7624s,8180s,9772s


now i wanted to further split the splitted file, to send a sample to neuron2. i set mkvmerge to split after 20M and got this result:


mkvmerge v2.0.2 ('You're My Flame') built on Feb 21 2007 23:40:43
'F:\Video\DVB\Baby\baby splitted-002.mkv': Using the Matroska demultiplexer.
'F:\Video\DVB\Baby\baby splitted-002.mkv' track 1: Using the MPEG-4 part 10 (AVC) video output module.
'F:\Video\DVB\Baby\baby splitted-002.mkv' track 2: Using the AC3 output module.
The file 'F:\Video\DVB\Baby\baby test splitted-002-001.mkv' has been opened for writing.
'die' called: bref_packet == NULL. Wanted bref: -40000000. Contents of the queue:
Packet 0, timecode 0, bref -1, fref -1
Packet 1, timecode 65000000, bref -1, fref -1
Packet 2, timecode 97000000, bref -1, fref -1
Packet 3, timecode 120000000, bref -40000000, fref -1
Packet 4, timecode 40000000, bref 120000000, fref -1
Packet 5, timecode 80000000, bref 40000000, fref -1
Packet 6, timecode 129000000, bref -1, fref -1
Packet 7, timecode 160000000, bref 80000000, fref -1
Packet 8, timecode 161000000, bref -1, fref -1
Packet 9, timecode 193000000, bref -1, fref -1
Packet 10, timecode 225000000, bref -1, fref -1
Packet 11, timecode 257000000, bref -1, fref -1
Packet 12, timecode 280000000, bref 160000000, fref -1
Packet 13, timecode 200000000, bref 280000000, fref -1
Packet 14, timecode 240000000, bref 200000000, fref -1
Packet 15, timecode 289000000, bref -1, fref -1
Packet 16, timecode 321000000, bref -1, fref -1
Packet 17, timecode 353000000, bref -1, fref -1
Packet 18, timecode 385000000, bref -1, fref -1
Packet 19, timecode 400000000, bref -1, fref -1
Packet 20, timecode 320000000, bref 400000000, fref -1
Packet 21, timecode 360000000, bref 320000000, fref -1
Packet 22, timecode 417000000, bref -1, fref -1
Packet 23, timecode 449000000, bref -1, fref -1
Packet 24, timecode 481000000, bref -1, fref -1
Packet 25, timecode 513000000, bref -1, fref -1
Packet 26, timecode 520000000, bref 360000000, fref -1
Packet 27, timecode 440000000, bref 520000000, fref -1
Packet 28, timecode 480000000, bref 440000000, fref -1
Packet 29, timecode 545000000, bref -1, fref -1
Packet 30, timecode 577000000, bref -1, fref -1
Packet 31, timecode 609000000, bref -1, fref -1
Packet 32, timecode 640000000, bref -1, fref -1
Packet 33, timecode 560000000, bref 640000000, fref -1
Packet 34, timecode 600000000, bref 560000000, fref -1
Packet 35, timecode 641000000, bref -1, fref -1
Packet 36, timecode 673000000, bref -1, fref -1
Packet 37, timecode 705000000, bref -1, fref -1
Packet 38, timecode 737000000, bref -1, fref -1
Packet 39, timecode 760000000, bref 600000000, fref -1
Packet 40, timecode 680000000, bref 760000000, fref -1
Packet 41, timecode 720000000, bref 680000000, fref -1
Packet 42, timecode 769000000, bref -1, fref -1
Packet 43, timecode 801000000, bref -1, fref -1
Packet 44, timecode 833000000, bref -1, fref -1
Packet 45, timecode 865000000, bref -1, fref -1
Packet 46, timecode 880000000, bref 720000000, fref -1
Packet 47, timecode 800000000, bref 880000000, fref -1
Packet 48, timecode 840000000, bref 800000000, fref -1
Packet 49, timecode 897000000, bref -1, fref -1
Packet 50, timecode 929000000, bref -1, fref -1
...


it happens with all splitted files, except the first one!

on Nemo, it only happens on a file that i demuxed with mkvextract, all other files work.

on tears of the sun, it happens after splitting... really inconsitent phenomenon and hard to reproduce.

hope you can help me on this one!

[foxyshadis]

Mosu, would you care to look at the VFR problem thread (http://forum.doom9.org/showthread.php?t=123097) in this forum, to see whether the problem is in mkvmerge and whether it can be fixed there?

[Mosu]

2007-03-08 Moritz Bunkus <moritz@bunkus.org>
* mkvmerge: The MPEG program stream reader will now sort the tracks it finds first by their type (video > audio > subs) and then by their stream ID.
* mkvmerge: Disabled the support for DTS tracks in MPEG program streams because DTS HD is not supported yet.
* mkvmerge: enhancement: Implemented a major speed-up for reading


http://www.bunkus.org/videotools/mkvtoolnix/win32/pre/mkvtoolnix-unicode-2.0.2-build20070308-1.rar

And sorry Maverick & foxyshadis, I'm not in the mood for bugfixing such problems at the moment.

[madshi]

http://www.bunkus.org/videotools/mkvtoolnix/win32/pre/mkvtoolnix-unicode-2.0.2-build20070308-1.rar
Thanks!

[Eragon4ever]

* mkvmerge: enhancement: Implemented a major speed-up for reading

... MPEG-1/2 and AVC/h.264 tracks from MPEG program streams.

[Mosu]

Indeed. I f'cked up a simple "copy & paste" :)

[Henrikx]

:thanks: Mosu

[DeepBeepMeep]

Great News! Thanks Mosu!

Well in fact, DTS HD was somewhat already supported beside the bug I have reported which didn't occur on all the streams I have tested. What was supported with DTS HD was the conversion from DTS HD to DTS by ignoring the "HD" part. In fact if you plan to add DTS HD support, an option to keep either the whole DTS HD packets or only their DTS packets could be convenient since there aren't many DTS HD decoders for the moment.

[madshi]

* mkvmerge: enhancement: Implemented a major speed-up for reading MPEG-1/2 and AVC/h.264 tracks from MPEG program streams.
I can gladly confirm that the Equilibrium H.264 EVO -> MKV muxing performance was greatly improved! :)

However, it still fails in the middle of the movie. I've done some further checks and attached you'll find a screenshot of the memory consumption during Equilibrium muxing. You'll notice two things:

(1) The memory consumption increases and decreases in time intervals of different length.
(2) The bottom line of the memory consumption grows over time.

So because of (2) I think there is a memory leak somewhere in mkvmerge. I'm not sure, though, whether the memory leak is really the main problem. It's a problem, but not the only one. From what I can see, Equilibrium has some INSANE GOP distances. It seems to me that mkvmerge is reading one complete GOP into RAM and only then writes it to harddisk. Usually that's a nice approach, but with Equilibrium the GOP distances are so big that it's getting dangerous. I've seen memory consumption go to >500MB with a large GOP in Equilibrium - and that's just in the beginning of the movie. There may be even bigger GOPs later in the movie. Another thing I noticed is that the GUI seems to updated only once every time when a GOP was fully read in. That would explain why the GUI is so much less responsive when muxing Equilibrium compared to other movies, because with Equilibrium the GOP distances are so much longer than with any other movie I've ever seen...

[Mosu]

I can gladly confirm that the Equilibrium H.264 EVO -> MKV muxing performance was greatly improved! :)

Good to hear.

However, it still fails in the middle of the movie. I've done some further checks and attached you'll find a screenshot of the memory consumption during Equilibrium muxing. You'll notice two things:

(1) The memory consumption increases and decreases in time intervals of different length.
(2) The bottom line of the memory consumption grows over time.

I cannot look at the picture yet because it's still awaiting authorization, but (2) is necessary or at least normal. A slow but steady increase in memory consumption will always happen because mkvmerge needs to keep data in memory that it can only write at the end of the muxing process (e.g. the index).

From what I can see, Equilibrium has some INSANE GOP distances. It seems to me that mkvmerge is reading one complete GOP into RAM and only then writes it to harddisk.

True, and this cannot be avoided. Especially with codecs with out-of-order timecodes (which all modern video codecs are; MPEG-1/-2, MPEG-4, AVC...) because you cannot assign timecodes correctly until you've found the next key frame.

Usually that's a nice approach, but with Equilibrium the GOP distances are so big that it's getting dangerous. I've seen memory consumption go to >500MB with a large GOP in Equilibrium - and that's just in the beginning of the movie.

Like I've said, it's not only "a nice approach", it's really necessary.

There may be even bigger GOPs later in the movie. Another thing I noticed is that the GUI seems to updated only once every time when a GOP was fully read in.

Yep. mmg only processes user interaction when it gets input from mkvmerge (e.g. a warning, any other message, or the progress report). This can only be solved with proper multithreading.

[foxyshadis]

You can't figure timecodes as soon as you hit a P frame? Because I thought you could figure the timestamps for the previous segment as soon as you hit the next P frame. Graphically:

IPBBBPBBBPI
IPBBBPBBBPI
IPBBBPBBBPI
IPBBBPBBBPI
IPBBBPBBBPI
IPBBBPBBBPI
IPBBBPBBBPI
IPBBBPBBBPI
....
Grey: Unread. Red: Read, unknown timestamp. Blue: Known timestamp and written.

As far as I know there's no mpeg codecs where b-frames can be timed after the next P frame.