Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > New and alternative a/v containers

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 17th January 2007, 09:03   #41  |  Link
yonta
Registered User
 
Join Date: Mar 2005
Posts: 126
Quote:
Originally Posted by Mosu View Post
This build should fix the issues that Isochroma, yonta and Eragon4ever have experienced: http://www.bunkus.org/videotools/mkv...20070116-1.rar
problem solved! muxed avc es with nalu set to 3 and it plays fine!
Thank you.
yonta is offline  
Old 17th January 2007, 09:25   #42  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 3,877
Quote:
Originally Posted by madshi View Post
Are you seriously working on this? That would be lovely, of course! I've lots of h.264 transport streams. But I guess most are just the same, because they're all captured with the same software from the same TV channel.
I'm seriously considering working on it Problem is that I don't have any specs so far, and it's always easier working with specs than just digging into the source code of various media players and muxing applications.
__________________
Latest MKVToolNix is v48.0.0

If I ever ask you to upload something, please use my file server.
Mosu is offline  
Old 17th January 2007, 09:26   #43  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 3,877
Quote:
Originally Posted by bob0r View Post
@Mosu guess i read your mind
Indeed you do Thanks for the samples.
__________________
Latest MKVToolNix is v48.0.0

If I ever ask you to upload something, please use my file server.
Mosu is offline  
Old 17th January 2007, 10:44   #44  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 3,877
Quote:
Originally Posted by Isochroma View Post
but one more thing, the test.264 that I muxed with Yamb, I set the fps to 23.976, and when the MP4 was merged into MKV, the framerate now showed as 24...
I cannot reproduce that. I've done the following:

Code:
0 mosu@jaina:/ftp/.rip/mkv/bugs/225$ MP4Box test.mp4 -fps 23.976 -add test.264
...
0 mosu@jaina:/ftp/.rip/mkv/bugs/225$ mkvmerge -o test.mkv test.mp4
...
0 mosu@jaina:/ftp/.rip/mkv/bugs/225$ mkvinfo test.mkv | grep 'Default duration'
|  + Default duration: 41.708ms (23.976 fps for a video track)
Quote:
With the new MKVMerge, muxing directly the ES and setting its framerate to 23.976, the output MKV has a framerate of 24.975025? Strange...
Oops, my bad. For some special values (23.976 and 29.97) mkvmerge was supposed to use more exact values, and I made a typo there Please try this build: http://www.bunkus.org/videotools/mkv...20070117-1.rar
__________________
Latest MKVToolNix is v48.0.0

If I ever ask you to upload something, please use my file server.
Mosu is offline  
Old 17th January 2007, 10:53   #45  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 3,877
Defaulting the NALU size length to 3?

Hey,

this is a small "poll" of sorts. It concerns mkvmerge's AVC/h.264 ES handling code. As some of you have noticed mkvmerge has a new parameter, "--nalu-size-length". A short explanation of what it does and why it is neccessary:

In MP4 and Matroska files each h.264 slice is prefixed with its size. This "NALU size" is always a fixed number of bytes long. It can be between one and four bytes, but it cannot vary inside a file. Two bytes means that each slice can be at most 65535 bytes long, three bytes allow for 16777215 bytes. mkvmerge defaults to "two bytes".

However, with HDTV content this almost always seems to be too low, and three bytes are needed. Hence mkvmerge's error message. Now I can have mkvmerge default to "three bytes" for the NALU size. But what would that mean?

Advantage: mkvmerge will no longer abort with this error message.

Disadvantage: For each slice mkvmerge will need one more byte of space. As each frame can consist of many slices (I have files with six slices per frame) this may become quite an extra overhead. For example, take a 25 FPS movie with six slices per frame and a duration of 90 minutes. This would mean that the resulting file would be 90 (minutes) * 60 (seconds per minute) * 25 (frames per second) * 6 (slices per frame) = 810000 bytes.

Another possibility would be to have mkvmerge scan the complete input file first in order to find the "optimal" value for the NALU size length. But that would, as I said, require reading the same file twice -- once for scanning, once for muxing.

So I'd like to know which option you prefer. Is it...

1. keep it like it is (default "two bytes", no scanning),
2. change it to "three bytes", no scanning or
3. implement scanning?

Thanks for the answers
__________________
Latest MKVToolNix is v48.0.0

If I ever ask you to upload something, please use my file server.
Mosu is offline  
Old 17th January 2007, 14:07   #46  |  Link
Eragon4ever
lost program in the net
 
Join Date: Jun 2006
Location: Germany
Posts: 106
I have a 4th possibility (I think):
Keep it as it is but if the NALU is to small try 3 bytes.

Besides, I would prefer scanning as it gives me smaller sizes in a reasonable amount of time (and the chance of using just 1 byte).

Edit: I think I found the TS specs: ISO 13818-1

Last edited by Eragon4ever; 17th January 2007 at 14:19.
Eragon4ever is offline  
Old 17th January 2007, 14:39   #47  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 3,877
Quote:
Originally Posted by Eragon4ever View Post
I have a 4th possibility (I think):
Keep it as it is but if the NALU is to small try 3 bytes.
Nah, not really. This would require either rewriting the part of the file that has already been written or to restart mkvmerge. I don't like either case very much; even though the latter is way easier to implement.

Quote:
Besides, I would prefer scanning as it gives me smaller sizes in a reasonable amount of time (and the chance of using just 1 byte).
Don't underestimate the time it takes to scan a complete file. But anyway, I'll count this as a vote for "3."

Quote:
Edit: I think I found the TS specs: ISO 13818-1
Cool, thanks.
__________________
Latest MKVToolNix is v48.0.0

If I ever ask you to upload something, please use my file server.
Mosu is offline  
Old 17th January 2007, 15:29   #48  |  Link
Henrikx
Registered User
 
Henrikx's Avatar
 
Join Date: Aug 2005
Location: Germany
Posts: 306
@Mosu
Thank You ! Great Work !
__________________
Henrikx
Henrikx is offline  
Old 17th January 2007, 16:12   #49  |  Link
madshi
Registered Developer
 
Join Date: Sep 2006
Posts: 9,137
Quote:
Originally Posted by Mosu View Post
So I'd like to know which option you prefer. Is it...

1. keep it like it is (default "two bytes", no scanning),
2. change it to "three bytes", no scanning or
3. implement scanning?
I vote for implementing 3 as default, as I'm running mkvmerge in batch in background, anyway, so time cost is not the most important factor for me. But if you decide to implement 3, you could still offer 1/2 as alternative options, if you think it's worth it.

Alternative: How late in the game could it happen that you find out that NALU 2 is not enough? If you can find out by scanning only the first 200 MB, that would be my most preferred option.
madshi is offline  
Old 17th January 2007, 17:19   #50  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 3,877
Quote:
Originally Posted by madshi View Post
Alternative: How late in the game could it happen that you find out that NALU 2 is not enough? If you can find out by scanning only the first 200 MB, that would be my most preferred option.
Worst case is that I find out with the very last slice that is read. Therefore scanning the first 200 MB should be enough for almost all files, but I don't want to rely on that too much.
__________________
Latest MKVToolNix is v48.0.0

If I ever ask you to upload something, please use my file server.
Mosu is offline  
Old 17th January 2007, 17:55   #51  |  Link
Isochroma
Registered User
 
Join Date: Mar 2005
Posts: 468
810000 bytes is ridiculously small for an entire movie. What is that, 0.81 MB? Movies start at 700MB which would in this case be 0.12%?

For a two-CD size movie, that would be 0.06%, for a 4.7 GB video that would be 0.0172%.

So yes, make the NALU 3.
Isochroma is offline  
Old 17th January 2007, 18:01   #52  |  Link
honai
Guest
 
Posts: n/a
Same here. The overhead of MKV is already very small compared to other containers, so I'm sure we can afford those 810K.
 
Old 17th January 2007, 20:01   #53  |  Link
madshi
Registered Developer
 
Join Date: Sep 2006
Posts: 9,137
Quote:
Originally Posted by Isochroma View Post
810000 bytes is ridiculously small for an entire movie. What is that, 0.81 MB?
I change my vote to "always 3 byte". When I originally read that 810000 number I was mistakingly thinking of 810 MB, which would have been too much. But 0.81 MB is really too small to worry about.
madshi is offline  
Old 17th January 2007, 20:08   #54  |  Link
Eragon4ever
lost program in the net
 
Join Date: Jun 2006
Location: Germany
Posts: 106
Quote:
Originally Posted by Isochroma View Post
What is that, 0.81 MB?
Actually it's just 0,77MB. Just to clear that up.
OK, I'm fine with "2." if the option to set it manually stays.

Last edited by Eragon4ever; 17th January 2007 at 20:12.
Eragon4ever is offline  
Old 17th January 2007, 20:30   #55  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 3,877
Quote:
Originally Posted by Eragon4ever View Post
OK, I'm fine with "2." if the option to set it manually stays.
The option will definitely stay. If I change it to 3 then I'll also add a message at the end of the muxing if 2 would have sufficed -- so that people who really want the smallest possible size know that they can lower the overhead
__________________
Latest MKVToolNix is v48.0.0

If I ever ask you to upload something, please use my file server.
Mosu is offline  
Old 18th January 2007, 01:17   #56  |  Link
bob0r
Pain and suffering
 
bob0r's Avatar
 
Join Date: Jul 2002
Posts: 1,337
beyonce.at.the.bbc.1080mbaff.sample.ts
47.6 MB (49,999,916 bytes)
demuxed .h264 and .mp2 muxed into .mkv works

beyonce.at.the.bbc.1080mbaff.ts
4.19 GB (4,504,797,720 bytes)
demuxed .h264 and .mp2 muxing into .mkv fails.
BBC HD uses H.264 MBAFF video coding.
Also other big captures from BBC-HD fail, this error:

'die' called: common.cpp/saferealloc() called from file src/common/common_memory.cpp, line 33: realloc() returned NULL for a size of 124253 bytes.
bob0r is offline  
Old 18th January 2007, 01:20   #57  |  Link
Haali
Registered User
 
Join Date: Jul 2003
Posts: 282
Btw, nalu size lentgh of 3 is illegal according to std, and qt decoder doesn't accept such length. Valid values are 1,2,4.
Haali is offline  
Old 18th January 2007, 09:05   #58  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 3,877
Quote:
Originally Posted by Haali View Post
Btw, nalu size lentgh of 3 is illegal according to std, and qt decoder doesn't accept such length. Valid values are 1,2,4.
Ah crap. This reaaaaaally makes sense... So the question is: change to 4 as the default or not?

BTW: The source code of your muxer you gave me quite a while ago used 3 as the default. I guess you've changed that by now. And MP4Box allows and uses 3, too.
__________________
Latest MKVToolNix is v48.0.0

If I ever ask you to upload something, please use my file server.
Mosu is offline  
Old 18th January 2007, 10:02   #59  |  Link
Mosu
MKVToolNix author
 
Mosu's Avatar
 
Join Date: Sep 2002
Location: Braunschweig, Germany
Posts: 3,877
Quote:
Originally Posted by bob0r View Post
beyonce.at.the.bbc.1080mbaff.sample.ts
47.6 MB (49,999,916 bytes)
demuxed .h264 and .mp2 muxed into .mkv works

beyonce.at.the.bbc.1080mbaff.ts
4.19 GB (4,504,797,720 bytes)
demuxed .h264 and .mp2 muxing into .mkv fails.
BBC HD uses H.264 MBAFF video coding.
Also other big captures from BBC-HD fail, this error:

'die' called: common.cpp/saferealloc() called from file src/common/common_memory.cpp, line 33: realloc() returned NULL for a size of 124253 bytes.
Any chance you could find a smaller piece of that file for which mkvmerge crashes? If not, would you be willing to upload it to my FTP server? I do have the space and the bandwidth, so it'd be no problem for me.
__________________
Latest MKVToolNix is v48.0.0

If I ever ask you to upload something, please use my file server.
Mosu is offline  
Old 18th January 2007, 10:28   #60  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,496
For x264 and Ateme, it's not much of an issue, since more than 1 or 2 slices is rare (currently impossible for x264). Unless chroma planes are separate slices?

Anyway, I'd go for a default of 4. Maximize compatibility and minimize inconvenience. I'd vote that --nalu 2 throw a warning and simply restart with 4 instead of flat erroring out, if the message is already going to be there. It'd be good for those of us die-hards who make streaming video and want to cut every byte of overhead from our 40kbps streams.

Perhaps this could be integrated with the "always use simpleblock" option. Along with no default header values. Sort of an all-inclusive blood-of-turnip option.

Can it fix the nalu size of premuxed stuff coming from mp4/mkv/avi as well?

(It's still a little too happy to read wmvs as AVC ES. But that's my fault for trying to add one.)

Minor bug report: An extra space between --engage no_default_header_values and --engage native_mpeg4 caused mkvmerge to fail.
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order.

Last edited by foxyshadis; 18th January 2007 at 10:32.
foxyshadis is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:42.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.