Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > New and alternative a/v containers

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 5th March 2007, 18:58   #181  |  Link
madshi
Registered Developer
 
Join Date: Sep 2006
Posts: 9,137
Quote:
Originally Posted by Mosu View Post
http://www.bunkus.org/videotools/mkv...20070305-1.rar

At least it works with 'Departed Trailer.evo'.
Thank you!! Am just trying that with Equilibrium. Here's how it behaves:

(1) Detects video and audio just fine.
(2) When starting to mux, there's a freeze for half a minute or so.
(3) Then muxing starts and looks promising.
(4) There's a long freeze after 500MB are written.
(5) It continues after a while, but the GUI is always not responding and progress is extremely slow. Now at 750MB after several minutes. Normally it just takes a few seconds to get that far.

Basically it seems to work, but it's extremely slow and the GUI is not responding at all.

I'm sorry, but I cannot send you this evo, cause it's a real movie (would be illegal to send you) and besides it's 17GB.
madshi is offline  
Old 5th March 2007, 19:06   #182  |  Link
DeepBeepMeep
Registered User
 
Join Date: Jun 2006
Posts: 133
Mosu,

I have uploaded the DtsHD track on your ftp site. It is a huge file named "DtsHDTrack.dts". It seems something went wrong and obviously only 90% of the file is there. Hopefully, the glitch is there and should occur towards the end of the file. I happened when I had muxed it with a mkv file that contained already a VC1 file (generated with Haali Filter).

Thanks for your help
DeepBeepMeep is offline  
Old 5th March 2007, 19:41   #183  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 4,083
Quote:
Originally Posted by madshi View Post
Thank you!! Am just trying that with Equilibrium. Here's how it behaves:

(1) Detects video and audio just fine.
(2) When starting to mux, there's a freeze for half a minute or so.
(3) Then muxing starts and looks promising.
(4) There's a long freeze after 500MB are written.
(5) It continues after a while, but the GUI is always not responding and progress is extremely slow. Now at 750MB after several minutes. Normally it just takes a few seconds to get that far.

Basically it seems to work, but it's extremely slow and the GUI is not responding at all.

I'm sorry, but I cannot send you this evo, cause it's a real movie (would be illegal to send you) and besides it's 17GB.
What kind of a video track is this? MPEG-2? I know that both MPEG-1/2 and AVC can be slow, and I've witnessed them being especially slow for EVO files. I will probably have to rewrite the complete MPEG-1/2 code. While it uses a nice object oriented approach it is dead slow

That the GUI is not responding is indeed bad, but I won't fix this completely (way too much work to figure out how to do this... multithreading etc etc, a lot of topics that I haven't covered with wxWidgets before).
__________________
Latest MKVToolNix is v63.0.0

If I ever ask you to upload something, please use my file server.
Mosu is offline  
Old 5th March 2007, 22:54   #184  |  Link
madshi
Registered Developer
 
Join Date: Sep 2006
Posts: 9,137
Quote:
Originally Posted by Mosu View Post
What kind of a video track is this?
It's H.264/AVC.

Quote:
Originally Posted by Mosu View Post
That the GUI is not responding is indeed bad, but I won't fix this completely (way too much work to figure out how to do this... multithreading etc etc, a lot of topics that I haven't covered with wxWidgets before).
Moving the GUI to another thread would probably be quite complicated. But I think moving just the mkvmerge encapsulation to a secondary thread should be rather easy. With win32 APIs (using Send/PostMessage for thread safe communication between the threads) at least it would.

But anyway, it's not really necessary IMHO. The responsiveness it normally quite acceptable. It's only this specific case with the 17GB H.264 EVO where the responsiveness was not so good, anymore.
madshi is offline  
Old 6th March 2007, 08:07   #185  |  Link
madshi
Registered Developer
 
Join Date: Sep 2006
Posts: 9,137
Let mkvmerge run through Equilibrium last night. Today morning the progress bar had stopped at 65% and the status window sais:

Code:
mkvmerge v2.0.2 ('You're My Flame') built on Mar  5 2007 13:45:56
'D:\Equilibrium.evo': Using the MPEG PS demultiplexer.
'D:\Equilibrium.evo' track 0: Using the MPEG-4 part 10 ES video output module.
The file 'D:\Equilibrium.mkv' has been opened for writing.
'die' called: common.cpp/safemalloc() called from file src/common/mpeg4_common.cpp, line 1046: malloc() returned NULL for a size of 139610 bytes.
madshi is offline  
Old 6th March 2007, 09:16   #186  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 4,083
Quote:
Originally Posted by madshi View Post
Let mkvmerge run through Equilibrium last night. Today morning the progress bar had stopped at 65% and the status window sais:

Code:
'die' called: common.cpp/safemalloc() called from file src/common/mpeg4_common.cpp, line 1046: malloc() returned NULL for a size of 139610 bytes.
Hmm, sounds like a serious memory leak. I'll have to see if I can find some other EVO which triggers this memory leak.
__________________
Latest MKVToolNix is v63.0.0

If I ever ask you to upload something, please use my file server.
Mosu is offline  
Old 6th March 2007, 09:35   #187  |  Link
DeepBeepMeep
Registered User
 
Join Date: Jun 2006
Posts: 133
In order to give you idea how the DTSHD track I mentioned before was mixed in the original EVO file I have uploaded a short sample of the EVO file (VC1WithDTSHD.EVO)

It is an interesting file as wells since mkvtoolnix won't see any track in it for the moment while there are in fact one video VC1 track and three DTSHD tracks (one is 5.1 and two 2.0)
DeepBeepMeep is offline  
Old 6th March 2007, 15:31   #188  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 4,083
Quote:
Originally Posted by DeepBeepMeep View Post
In order to give you idea how the DTSHD track I mentioned before was mixed in the original EVO file I have uploaded a short sample of the EVO file (VC1WithDTSHD.EVO)
Thanks for the upload.

Quote:
It is an interesting file as wells since mkvtoolnix won't see any track in it for the moment while there are in fact one video VC1 track and three DTSHD tracks (one is 5.1 and two 2.0)
Quite natural considering mkvmerge supports neither VC1 nor DTSHD at the moment
__________________
Latest MKVToolNix is v63.0.0

If I ever ask you to upload something, please use my file server.
Mosu is offline  
Old 6th March 2007, 16:16   #189  |  Link
DeepBeepMeep
Registered User
 
Join Date: Jun 2006
Posts: 133
Quote:
Originally Posted by Mosu View Post
Thanks for the upload.



Quite natural considering mkvmerge supports neither VC1 nor DTSHD at the moment
My pleasure. If this can give some inspiration on how to support demuxing EVO with VC1 (which is the video codec the most used with HDDVD) this would make turning an EVO into a MKV file only a one click process.

On a side note, when mkvtoolnix parses an EVO file and finds some DD+ tracks for instance it seems to order them randomly. This can be a bit confusing and can cause trouble when trying to merge two evo files using mkvtoolnix, it seems there is a proper order (based on the PID?) that can be detected by EvoDemux.exe (http://pel.hu/down/EVOdemux.exe) or by evob_demux (http://www.w6rz.net/evob_demux.zip ).
DeepBeepMeep is offline  
Old 6th March 2007, 21:23   #190  |  Link
ToS_Maverick
x264 Tester
 
Join Date: Dec 2005
Location: Austria, near Vienna
Posts: 223
@mosu

got another issue for you, hope you have some time to look at it

my workflow as described in another thread is:
Pro7 HD Transport Stream => Demux with Graphedit (Haali Media Splitter => Haali Matroska Muxer) => spilt with mkvmerge GUI (the "unwanted" scenes)

mkvmerge commanline looks like this:
Code:
"mkvmerge" -o "F:\Video\DVB\Baby\baby test.mkv"  --language 1:eng --default-track 1:yes --display-dimensions 1:1920x1080 --language 2:eng --default-track 2:yes -a 2 -d 1 -S "F:\Video\DVB\Baby\baby.mkv" --track-order 0:1,0:2 --split timecodes:73s,1806s,2365s,3996s,4617s,5800s,6399s,7624s,8180s,9772s
now i wanted to further split the splitted file, to send a sample to neuron2. i set mkvmerge to split after 20M and got this result:

Code:
mkvmerge v2.0.2 ('You're My Flame') built on Feb 21 2007 23:40:43
'F:\Video\DVB\Baby\baby splitted-002.mkv': Using the Matroska demultiplexer.
'F:\Video\DVB\Baby\baby splitted-002.mkv' track 1: Using the MPEG-4 part 10 (AVC) video output module.
'F:\Video\DVB\Baby\baby splitted-002.mkv' track 2: Using the AC3 output module.
The file 'F:\Video\DVB\Baby\baby test splitted-002-001.mkv' has been opened for writing.
'die' called: bref_packet == NULL. Wanted bref: -40000000. Contents of the queue:
Packet 0, timecode 0, bref -1, fref -1
Packet 1, timecode 65000000, bref -1, fref -1
Packet 2, timecode 97000000, bref -1, fref -1
Packet 3, timecode 120000000, bref -40000000, fref -1
Packet 4, timecode 40000000, bref 120000000, fref -1
Packet 5, timecode 80000000, bref 40000000, fref -1
Packet 6, timecode 129000000, bref -1, fref -1
Packet 7, timecode 160000000, bref 80000000, fref -1
Packet 8, timecode 161000000, bref -1, fref -1
Packet 9, timecode 193000000, bref -1, fref -1
Packet 10, timecode 225000000, bref -1, fref -1
Packet 11, timecode 257000000, bref -1, fref -1
Packet 12, timecode 280000000, bref 160000000, fref -1
Packet 13, timecode 200000000, bref 280000000, fref -1
Packet 14, timecode 240000000, bref 200000000, fref -1
Packet 15, timecode 289000000, bref -1, fref -1
Packet 16, timecode 321000000, bref -1, fref -1
Packet 17, timecode 353000000, bref -1, fref -1
Packet 18, timecode 385000000, bref -1, fref -1
Packet 19, timecode 400000000, bref -1, fref -1
Packet 20, timecode 320000000, bref 400000000, fref -1
Packet 21, timecode 360000000, bref 320000000, fref -1
Packet 22, timecode 417000000, bref -1, fref -1
Packet 23, timecode 449000000, bref -1, fref -1
Packet 24, timecode 481000000, bref -1, fref -1
Packet 25, timecode 513000000, bref -1, fref -1
Packet 26, timecode 520000000, bref 360000000, fref -1
Packet 27, timecode 440000000, bref 520000000, fref -1
Packet 28, timecode 480000000, bref 440000000, fref -1
Packet 29, timecode 545000000, bref -1, fref -1
Packet 30, timecode 577000000, bref -1, fref -1
Packet 31, timecode 609000000, bref -1, fref -1
Packet 32, timecode 640000000, bref -1, fref -1
Packet 33, timecode 560000000, bref 640000000, fref -1
Packet 34, timecode 600000000, bref 560000000, fref -1
Packet 35, timecode 641000000, bref -1, fref -1
Packet 36, timecode 673000000, bref -1, fref -1
Packet 37, timecode 705000000, bref -1, fref -1
Packet 38, timecode 737000000, bref -1, fref -1
Packet 39, timecode 760000000, bref 600000000, fref -1
Packet 40, timecode 680000000, bref 760000000, fref -1
Packet 41, timecode 720000000, bref 680000000, fref -1
Packet 42, timecode 769000000, bref -1, fref -1
Packet 43, timecode 801000000, bref -1, fref -1
Packet 44, timecode 833000000, bref -1, fref -1
Packet 45, timecode 865000000, bref -1, fref -1
Packet 46, timecode 880000000, bref 720000000, fref -1
Packet 47, timecode 800000000, bref 880000000, fref -1
Packet 48, timecode 840000000, bref 800000000, fref -1
Packet 49, timecode 897000000, bref -1, fref -1
Packet 50, timecode 929000000, bref -1, fref -1
...
it happens with all splitted files, except the first one!

on Nemo, it only happens on a file that i demuxed with mkvextract, all other files work.

on tears of the sun, it happens after splitting... really inconsitent phenomenon and hard to reproduce.

hope you can help me on this one!
ToS_Maverick is offline  
Old 7th March 2007, 13:16   #191  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,563
Mosu, would you care to look at the VFR problem thread in this forum, to see whether the problem is in mkvmerge and whether it can be fixed there?
foxyshadis is offline  
Old 8th March 2007, 17:00   #192  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 4,083
Quote:
2007-03-08 Moritz Bunkus <moritz@bunkus.org>
* mkvmerge: The MPEG program stream reader will now sort the tracks it finds first by their type (video > audio > subs) and then by their stream ID.
* mkvmerge: Disabled the support for DTS tracks in MPEG program streams because DTS HD is not supported yet.
* mkvmerge: enhancement: Implemented a major speed-up for reading
http://www.bunkus.org/videotools/mkv...20070308-1.rar

And sorry Maverick & foxyshadis, I'm not in the mood for bugfixing such problems at the moment.
__________________
Latest MKVToolNix is v63.0.0

If I ever ask you to upload something, please use my file server.
Mosu is offline  
Old 8th March 2007, 17:13   #193  |  Link
madshi
Registered Developer
 
Join Date: Sep 2006
Posts: 9,137
Thanks!
madshi is offline  
Old 8th March 2007, 17:18   #194  |  Link
Eragon4ever
lost program in the net
 
Join Date: Jun 2006
Location: Germany
Posts: 106
Quote:
* mkvmerge: enhancement: Implemented a major speed-up for reading
... MPEG-1/2 and AVC/h.264 tracks from MPEG program streams.
Eragon4ever is offline  
Old 8th March 2007, 17:34   #195  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 4,083
Indeed. I f'cked up a simple "copy & paste"
__________________
Latest MKVToolNix is v63.0.0

If I ever ask you to upload something, please use my file server.
Mosu is offline  
Old 8th March 2007, 17:50   #196  |  Link
Henrikx
Registered User
 
Henrikx's Avatar
 
Join Date: Aug 2005
Location: Germany
Posts: 306
Mosu
__________________
Henrikx
Henrikx is offline  
Old 8th March 2007, 19:10   #197  |  Link
DeepBeepMeep
Registered User
 
Join Date: Jun 2006
Posts: 133
Great News! Thanks Mosu!

Well in fact, DTS HD was somewhat already supported beside the bug I have reported which didn't occur on all the streams I have tested. What was supported with DTS HD was the conversion from DTS HD to DTS by ignoring the "HD" part. In fact if you plan to add DTS HD support, an option to keep either the whole DTS HD packets or only their DTS packets could be convenient since there aren't many DTS HD decoders for the moment.
DeepBeepMeep is offline  
Old 10th March 2007, 16:06   #198  |  Link
madshi
Registered Developer
 
Join Date: Sep 2006
Posts: 9,137
Quote:
* mkvmerge: enhancement: Implemented a major speed-up for reading MPEG-1/2 and AVC/h.264 tracks from MPEG program streams.
I can gladly confirm that the Equilibrium H.264 EVO -> MKV muxing performance was greatly improved!

However, it still fails in the middle of the movie. I've done some further checks and attached you'll find a screenshot of the memory consumption during Equilibrium muxing. You'll notice two things:

(1) The memory consumption increases and decreases in time intervals of different length.
(2) The bottom line of the memory consumption grows over time.

So because of (2) I think there is a memory leak somewhere in mkvmerge. I'm not sure, though, whether the memory leak is really the main problem. It's a problem, but not the only one. From what I can see, Equilibrium has some INSANE GOP distances. It seems to me that mkvmerge is reading one complete GOP into RAM and only then writes it to harddisk. Usually that's a nice approach, but with Equilibrium the GOP distances are so big that it's getting dangerous. I've seen memory consumption go to >500MB with a large GOP in Equilibrium - and that's just in the beginning of the movie. There may be even bigger GOPs later in the movie. Another thing I noticed is that the GUI seems to updated only once every time when a GOP was fully read in. That would explain why the GUI is so much less responsive when muxing Equilibrium compared to other movies, because with Equilibrium the GOP distances are so much longer than with any other movie I've ever seen...
Attached Images
 
madshi is offline  
Old 10th March 2007, 16:37   #199  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 4,083
Quote:
Originally Posted by madshi View Post
I can gladly confirm that the Equilibrium H.264 EVO -> MKV muxing performance was greatly improved!
Good to hear.

Quote:
However, it still fails in the middle of the movie. I've done some further checks and attached you'll find a screenshot of the memory consumption during Equilibrium muxing. You'll notice two things:

(1) The memory consumption increases and decreases in time intervals of different length.
(2) The bottom line of the memory consumption grows over time.
I cannot look at the picture yet because it's still awaiting authorization, but (2) is necessary or at least normal. A slow but steady increase in memory consumption will always happen because mkvmerge needs to keep data in memory that it can only write at the end of the muxing process (e.g. the index).

Quote:
From what I can see, Equilibrium has some INSANE GOP distances. It seems to me that mkvmerge is reading one complete GOP into RAM and only then writes it to harddisk.
True, and this cannot be avoided. Especially with codecs with out-of-order timecodes (which all modern video codecs are; MPEG-1/-2, MPEG-4, AVC...) because you cannot assign timecodes correctly until you've found the next key frame.

Quote:
Usually that's a nice approach, but with Equilibrium the GOP distances are so big that it's getting dangerous. I've seen memory consumption go to >500MB with a large GOP in Equilibrium - and that's just in the beginning of the movie.
Like I've said, it's not only "a nice approach", it's really necessary.

Quote:
There may be even bigger GOPs later in the movie. Another thing I noticed is that the GUI seems to updated only once every time when a GOP was fully read in.
Yep. mmg only processes user interaction when it gets input from mkvmerge (e.g. a warning, any other message, or the progress report). This can only be solved with proper multithreading.
__________________
Latest MKVToolNix is v63.0.0

If I ever ask you to upload something, please use my file server.
Mosu is offline  
Old 10th March 2007, 16:49   #200  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,563
You can't figure timecodes as soon as you hit a P frame? Because I thought you could figure the timestamps for the previous segment as soon as you hit the next P frame. Graphically:

IPBBBPBBBPI
IPBBBPBBBPI
IPBBBPBBBPI
IPBBBPBBBPI
IPBBBPBBBPI
IPBBBPBBBPI
IPBBBPBBBPI
IPBBBPBBBPI
....
Grey: Unread. Red: Read, unknown timestamp. Blue: Known timestamp and written.

As far as I know there's no mpeg codecs where b-frames can be timed after the next P frame.
foxyshadis is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:51.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.