Muxman - simple DVD authoring

[mpucoder]

At version 0.6 some io changes were made, and since then 3 reports of hanging during ifo creation.
So here (http://www.mpucoder.com/Muxman/muxman_0_7a.zip) is a special compile of version 0.7 that goes back to using buffered writes for ifo creation.

Trahald, Sir Didymus, Koepi, and anyone else who experienced hanging please try this version and let me know. I'm also curious which OS (I'm assuming XP)

[Koepi]

For now the result looks the same. I'll give muxman some more minutes though (it's at 99% CPU) to give it a chance to create video_ts.ifo|bup.

OS: Win2k SP4 + all security patches (german)

source drive: 7.200 rpm/2mb 80GB maxtor u-dma 133
target drive: 7.200 rpm/8mb 160GB samsung u-dma 133

athlon xp 2.800+; nforce 2 ultra chipset.

Regards
Koepi

P.S.: forgot to paste the log, maybe the underflows and/or remaining bytes in buffer are the problem here?
MuxMan version 0.7
Opened video file E:\Filme\LOTR1-SE\lotr1-se-DVD.m2v, 3033623797 bytes.
Opened audio 1 file E:\Filme\LOTR1-SE\deutsch.ac3.
AC3 audio, frame size = 0x700.
Opened audio 2 file E:\Filme\LOTR1-SE\english.ac3.
AC3 audio, frame size = 0x700.
17:35:39 Begin multiplex.
P-STD buffer underflow by 7536 bytes at 776890892, sector 1376871.
P-STD buffer underflow by 17464 bytes at 776901692, sector 1376944.
P-STD buffer underflow by 20135 bytes at 776912492, sector 1377018.
P-STD buffer underflow by 10292 bytes at 776923292, sector 1377092.
P-STD buffer underflow by 37956 bytes at 776944892, sector 1377242.
P-STD buffer underflow by 7898 bytes at 776955692, sector 1377314.
P-STD buffer underflow by 4305 bytes at 776977292, sector 1377461.
P-STD buffer underflow by 10324 bytes at 777002492, sector 1377633.
P-STD buffer underflow by 2730 bytes at 777009692, sector 1377683.
SeqEnd at B4D16CF1.
Bytes remaining in buffer = 118640.
17:50:53 End multiplex.

EDIT2: weird. now that I forced the program closing, vts_01_0.ifo|bup get the current timestamp instead of the creation time 15 minutes ago. Maybe that helps.

[SeeMoreDigital]

Lord of the Ring's!

Those movies have run times that are measured in "parts of days" rather than in hours and minutes ;)

That said, it should provide MuxMan with "the ultimate test" :D


Cheers

[mpucoder]

Well, since it is not the io routines, the next thing would be for people having this problem to make a profile and send it to muxman@mpucoder.com so I can hopefully reproduce it.

[Koepi]

Obviously this is true, SeeMoreDigital ;)

@mpucoder:
Another thing I noticed - with mplex-based programs the vts_01_0.[ifo|bup] are 113kb in size. With muxman, they are 63kb. On my 2nd test with a lot more time not touching anything muxman still hangs in the same place.

If I can help you in any other way to track down the issue?

Regards
Koepi

[mpucoder]

@Koepi - if you could run it again with "make profile" checked and then send c:\muxman.mxl to muxman@mpucoder.com that would help. Even if Muxman hangs the profile will contain enough information to reconstruct conditions up to the hang.

[Koepi]

ok, mail is on the way (takes a little due to my max. 16kb/s uplink). My zip program refused to pack it, so it's full 1.2MB.

Cheers,
Koepi

[mpucoder]

Got it OK, thanks. This takes awhile to expand and test, so I probably won't know anything until tomorrow.

[mpucoder]

I found the problem, and man, is that one long movie. Although I don't like releasing so often, especially right after the last version made the news, this is important enough.
Version 0.8 is available here (http://www.mpucoder.com/Muxman/)

Fixes hang while building vobu_admap > 64K (movies > 2h16m)
And although it hasn't happened yet, the same thing could have happened while building the time map for movies longer than about 4H30M. So that has been recoded as well.

[Koepi]

Success! I got the "summary" messagebox at the end and muxman closed. The resulting files are playing correctly. Forgot the chapter list in the first run but repeating that now, then burn the stuff and try it on my standalone.

mpucoder,
one other suggestion though, if the input files aren't compliant and muxman won't accept them, can you spit out an according message too? I don't have my 18frames-per-(PAL)-gop file around anymore but that would be sure helping.

Regards
Koepi

[mpucoder]

MuxMan will diagnose the GOP size, and tell you when it encounters a large one. The bug was that if the first GOP is too large MuxMan attempted to calculate the average bitrate before showing the error message, resulting in division by zero.
The ironic thing is it crashed while trying to tell you. The program can easily handle 18 frame GOPs, they are used in NTSC. But a check was put in, mainly to prevent GOPs larger than 18 from crashing MuxMan (as that is the limit of one table).
All files get checked somewhat before the multiplex. Video is checked to see if the required headers are present (also needed to see if it is MPEG-1 or MPEG-2, if Pan/Scan is present, and for closed-captions). Audio files are recognized by content, not file extension, so they get scanned a little. If the files meet the first checks the multiplex can begin.
Any time MuxMan absolutely can not go on it will tell you why, but sometimes that will be very far into the multiplex as there is no pre-scan.

[stickboy]

Originally posted by mrslacker
I could use an additional element of flexibility: No authoring, just muxing. I have two large m2v files that need to be part of the same pgc. Using dvdauthor, I could list the several MuxMan vobs from the different source files in the same pgc, but I'd rather have just two files. I would be happy to test MuxMan more if it could just make a large vob or mpeg with NAV packets.I'd like to see this too. Is there any chance of entertaining this? Right now I use DVDLab to join the VOB files, but it's a real waste of time and disk.

[Trahald]

Reran 0.8 on those files that caused the hang and worked 100%! Thank you mpucoder

[RFontenot]

While muxing one of my RTV recordings of a 8mm video, I got an error message from MuxMan 0.7 about 2/3 of the way through a 2 hour recording saying it had detected a GOP with more than 18 frames. I clicked the "OK" button, and MuxMan terminated gracefully.

I eventually traced the problem to a section of the original 8mm recording that was flaky. I edited that section out, and MuxMan was then able to successfully mux the entire recording.

No problem to report, just wanted you to know that the NTSC GOP > 18 error handling is working as designed, at least in this situation.

RF

[buzzqw]

may i also ask for a command line version ?

Thanks

BHH

[PatchWorKs]

Do you know MPLEX (http://www.petercheat.host.sk/libav/mplex-howto.html) ?

I think it could be a great feature...

[Koepi]

Originally posted by PatchWorKs
Do you know MPLEX (http://www.petercheat.host.sk/libav/mplex-howto.html) ?

I think it could be a great feature...

Did you at all read the initial post? Or read on the site of mpucoder?

Please do so.

Cheers,
Koepi

[RFontenot]

I had a 4 hour recording of the Olympics still, so I ran it through MuxMan. MuxMan 0.8 successfully multiplexed the video and audio streams into 6 VOB files, but the program abended with the error: "Can't open VOB file" and did not create any .IFO or .BUP files. Clicking on the "OK" button appeared to hang the program, and clicking "Close" results in the "Program Not Responding" error to appear.

This is all that's in the log file:

MuxMan version 0.8
Opened video file M:\newdvd.m2v, 4294967295 bytes.
Opened audio 1 file M:\newdvd.mp2.
MP2 audio, frame size = 0x240.
11:58:49 Begin multiplex.
Cell map size 50.

[EpheMeroN]

I have two requests for a future MuxMan version:

1: Make the muxman.log file goto whatever dir the actual MuxMan.exe is located at, instead of defaulting to C:

2: Option to mux .m2v and .mp2 to a .mpg MPEG-2 video file instead of setting it up for authoring.

Keep up the great work :-)

[Malcolm]

hi mpucoder,
i have another 2 requests for muxman:
1. please add an option to relax some DVD compliancy-checks. that means primarily GOP-length. resolution (e.g. 480x480, 480x576) would be nice too.

i record DVB streams regularly and burn them to DVD. nearly all DVB streams have a few GOPs (<0.1%) with lengths > 18 (15). they all play fine on my dvd-player.
IMHO it's a bit far from practical use if a muxer refuses *at all* to mux such streams. (again only MHO :))

2. ability to add more than 1 video file. there are 2 types of use for this:
a) you have some mpeg files (e.g. 2 or 3 SVCDs) and you want them to play as a single film on the dvd. AFAIK playing all videos after another automatically can be done by adding some navigation commands in the .ifo file.
b) you have some mpeg files (2 individual movies) that you want to select + play individually.

greetings,
Malcolm

[SeeMoreDigital]

I've been experimenting with Mpeg2 high-def 1920x1080 (1088) and 1440x1080 (1088) files!

Would it possible for MuxMan to author such streams into VOB too?

I realise std-def stand-alone DVD players will not be able to spin and display such files but PC's and maybe even high-def stand-alone players might!


Just a thought... Cheers

[Sir Didymus]

Hey guys, :)

Respectfully speaking, but it seems here nobody takes care any longer about silly things like specifications...

About muxing more than a single video stream into a VOB file... this is simply explicitely forbidden...

But in general, lets'assume it's possible to go beyond the specification, generating some exotic super format DVD++. What happens when something goes wrong ? Which way people can judge who is right if person X say a given title plays perfectly and person Y states the same is crappy ?

Firt of all NEW specifications are necessary,... THEN it will be possible to provide some implementation compliant with this superset...

Regarding high definition, that's a feature I'd like to see... But not on my PC. I'd like it on my HD standalone, with my HD television, using my HD media... I mean, let's wait a couple of years, in the most optimistic scenario...

What I know (maybe I am wrong...) is that two consortia are actually fighting for the promotion of two candidate standards: Blue-ray and HD-DVD... All the rest is just dreaming.

Are you actually so disappointed about the DVD quality ?

Cheers,
SD

[mrslacker]

Originally posted by Sir Didymus
About muxing more than a single video stream into a VOB file... this is simply explicitely forbidden...

If you are refering to what Malcolm said, I think this would be very useful. It's not a request for creating a VOB with two video streams, just the ability to join clips. That's just my understanding of what he said.

Butterfingers: Edited spelling.

[Xesdeeni]

Isn't it a feature of DVDs to allow multiple viewing angles? Isn't that multiple video streams?

Xesdeeni

[Malcolm]

Originally posted by Sir Didymus
About muxing more than a single video stream into a VOB file... this is simply explicitely forbidden...I think there's a misunderstandig: muxing multiple videostreams *after* another in one VOB file is exactly what you can do with every DVD authoring program around. Try it with DVD Lab. Multiple .MPVs (same size, etc.) IN -> one .VOB OUT.
I think this is 100% compliant to the DVD standard. DVD Lab gives explicit warnings + errors if you try to do somethings that isn't dvd compliant!

to my request regarding GOP lengths > 15 (and video sizes like 480x576): this is no 'super format DVD++' as you say! supporting such things is of big practical use. DVB streams *HAVE* GOP lengths > 15. what prize do i get for having a super-duper 100% standard-compliant multiplexer when i can't use it because it refuses 99% of all my videos?
i'm recording & burning DVB streams for myself. and my dvd player plays them fine. so for whom should it make them standard compliant?
why *REFUSE* a stream which has 0.1% long GOPs inside?? it's o.k. to give a warning but i expect a tool to *HELP* me, not *HINDER* me!
i mean i'm just talking about a *checkbox* to switch some dvd compliancy-checks on and off! this doesn't makes muxman worse! it's beneficial IMHO!
Btw. no other multiplexer complains about GOP lengths > 15. and this is perfectly ok. otherwise i would have had a lot of trouble getting my videos on dvd!
sure i could use these muxers (as i already do). but if muxman is faster, will have more options and is more reliable, why not have it alltogether?

greetings,
Malcolm

[SeeMoreDigital]

Originally posted by Sir Didymus
About muxing more than a single video stream into a VOB file... this is simply explicitely forbidden... As far as I know, there's nothing forbidden with being able to join multiple video streams (with their corresponding audio streams) into a single VOB file (or more VOB files), if required...

Indeed, such an option would be very handy when coupled with a correctly written "import chapter" file!

I'm quite fed up receiving show-reel DVD's that contain only one "short" video file per VOB.

I got a disc before Xmas that had 24 VOB files on it, with a total run-time of approx 30mins. Which certainly does not conform to the DVD specification!


Cheers

[Sir Didymus]

@Malcom.

Regarding "more than 1 video stream", I have to apologise: just totally misunderstood what you meant. Really afraid.

Well, since the declared intentions (written in the download page) of the author are:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This is only the beginning. The multiplexing engine, dubbed "MME" for MPUCoder's Multiplexing Engine, was written from the ground up to work with multiple angles as well as properly handle mpeg-2 pulldown. This means multiple angles and multiple stories (true seamless branching) are possible without altering the engine. Of course to achieve that the engine needs to be driven properly, but the capability is there.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I think you (and myself) may assume that what you are asking is already planned as an evolutionary step of this "MME"...

About the GOP lenght and the requirements of other kind people asking to "exceed" or to "relax" the DVD specs, well, there are very solid reasons in the standards for requiring the max GOP size of 15 (for PAL) and 18 (for NTSC). That's all. If it is not allowed, it is of no relevance the result play perfectly on 99% of the standalones. There are situations [Well, here I am emphasizing, a little bit, it is just an example...] where you simply must be 100% compliant, and can not tolerate that 1% of your products would be rejected (with valid reasons) by your customers...

Cheers,
SD

[stickboy]

Originally posted by Sir Didymus
There are situations [Well, here I am emphasizing, a little bit, it is just an example...] where you simply must be 100% compliant, and can not tolerate that 1% of your products would be rejected (with valid reasons) by your customers...... and there are situations where you can tolerate 1% incompatibility.

The question is about who should be deciding this: the muxer or the user? I think the user should have the final say. The tool should give a warning, yes, (and this is what muxman does for underflows) and ask the user if they really want to continue, but it shouldn't summarily reject them.

Unconditional rejection is fine if you're in a position to encode the video again to be compliant, but it sucks if you're dealing with a video stream you didn't make.

[mpucoder]

Multiple video/audio files is mentioned in the readme file as something that will come later. The development of MuxMan is going through stages, the first of which is a multiplexer which is as standard-compliant as possible. Other programs have rushed to add features, and the result is what we here call a cluster f*** of non-compliant programs. That is not what I want Muxman to become, or to even help continue.

Some changes mentioned here will come in good time. Others are just not in the path of development I envision. DVB needs to be looked at for ways to make it compliant. Possibly by splitting GOPs (needs an I-frame). HD will come in another multiplexer once the format is known to me.

One workaround for short clips is to join them prior to multiplexing. If they are compatible you can join them using something as simple as copy /b

And in case you're wondering what I'm working on currently - one computer is running compliancy tests, which are getting fixed (they are not severe) for the next release. I've finished I-frame encoding and am working on bmp import for subpictures. That will enable sst support, and together will take Muxman one step closer to making menues.

[mpucoder]

About relaxing the GOP restriction - it could be done, but I need to know by how much. The restriction is there primarily to prevent internal tables from overflowing, so I need an absolute upper limit.

[Sir Didymus]

Originally posted by SeeMoreDigital

...
I'm quite fed up receiving show-reel DVD's that contain only one "short" video file per VOB.

I got a disc before Xmas that had 24 VOB files on it, with a total run-time of approx 30mins. Which certainly does not conform to the DVD specification!


Cheers

Hi, :) Afraid if my poor English generated some confusion...
Again apologise about the point of "more than 1 video stream".

On the other hand the second point you are stating is really surprising to me...

Almost sure I am having again some poor understanding of your words...

You mean that since you receive badly authored DVD on your desktop, and they play well on your standalone, then we should ask to the MuxMan author the "freedom" of producing crappy DVD ? Mhhh...

All the best,
SD

[SeeMoreDigital]

Originally posted by Sir Didymus
You mean that since you receive badly authored DVD on your desktop, and they play well on your standalone, then we should ask to the MuxMan author the "freedom" of producing crappy DVD ? Mhhh... I'm sorry to report, I receive badly authored DVD's all the time SD :(

And although these DVD's are not "store bought" movie titles, they have been created by people in the "audio/video" industry, who should know better!

Quite often I receive disc's that contain a simple menu pointing to many individual VOB files. Or disc's with multiple clips that are in one long VOB file but with no chapter points!

I even receive disc's containing both of the above :eek:


Cheers

[mpucoder]

I can see why you'd want a program to point to and ask them to use. But just what would you like the disks to contain? A single vob with each file as a chapter?
I have thought about adding a "dailies" feature. For those of you not familiar, a daily is the raw footage shot in one day. Put on DVD they contain identification and timecode to aid in critique. If I were to do this the id and timecode would be added by forced sub.

[Xesdeeni]

Please forgive me if this is too far asunder, but putting the timecode as a subtitle is something I'd like to do with my home movies when I convert them from DV to DVD. I can't seem to find a reasonable method of doing this. Might your daily proposal be part of the process?

Xesdeeni

[mrslacker]

Originally posted by mpucoder
I can see why you'd want a program to point to and ask them to use. But just what would you like the disks to contain? A single vob with each file as a chapter?
I don't know about SeeMoreDigital, but A single VOB with arbitrary user specified chapter points would be ideal. I'm thinking along the lines of the way DVDAuthor can form a single VOB and single pgc from multiple sources. On the other hand, having an option to create separate VOBs might be useful for a disc with distinctly different footage, like episodes or daily footage. Not sure I would have any use for the subs you mentioned in regards to "dailies" however.

[mpucoder]

Xesdeeni - That is exactly what it would do. There is a program out there that does just this. Unfortunately, I can't remember its name, or if it is free.

[SeeMoreDigital]

Originally posted by mpucoder
But just what would you like the disks to contain? A single vob with each file as a chapter? If you mean "each clip as a chapter" then... yes please :D

Can one of you guys confirm the maximum quantity of VOB's allowable on a spec compliant DVD please? Because I'm sure many of the DVD's I receive contain way more than there should be!

Thankfully none of them will ever require subtitles.


Cheers

[mpucoder]

Per titleset - 9. But you can have up to 99 titlesets (vts)

[SeeMoreDigital]

Originally posted by mpucoder
Per titleset - 9. But you can have up to 99 titlesets (vts) No wonder some of the DVD's I receive don't work correctly in stand-alone players!


Cheers

[makoto916]

Xesdeeni, this may seem like an obvious solution to your timecode issue, but you can always use an AVISynth script to generate it for you.

video = AVISource("dvsource.avi")
ShowSMPTE(video,video.framerate)

These two lines will load a video file and then display the SMPTE timecode at the bottom. You can get fancier and move it around if you want it to be within TV cropping.

[Xesdeeni]

1. I don't want the timecode (which is zero-based in AVISynth), I want the date/time stamp from the camcorder.

2. I don't want it burned in, I want it controlled on playback, just like a subtitle.

The idea is to see what date each clip in my home movies occurred, to see how old my children were when each thing happened, but not have it on screen at all times. I can do this manually, but it is a huge amount of work. I'd like a way to send the edited DV (from Premiere) through a filter that would create the subtitles for me, that I could add when I author the DVD.

Xesdeeni

[RFontenot]

Perhaps this got lost in all the talk about relaxing the specs for non-compliant elementary streams, but I did run into the problem of MuxMan hanging when multiplexing a 4 hour video. MuxMan successfully created six VOB files, but no .IFO or .BUP files, and the program ended up in a non-responsive state. Since up to nine VOB files are allowed, it should have worked, or is there some limit to video length that I'm not aware of?

Video files from the same source that are less than 4 hours mux just fine.

[mpucoder]

That particular error means Muxman cannot open another vob file. There are 3 reasons that I know of that will prevent that.
1) too many open files - this can't happen because Muxman closes the last file before opening another.
2) too many files in one directory - only the root in FAT file systems has a limit. Other directories and NTFS do not have this problem
3) no disk space.
I don't know of any other reasons, but I will try a simulation with a synthetic 4.2G file to see if there is any other problem.
The error should exit more gracefully, though, and that will get fixed.

[RFontenot]

I have two 11GB NTFS partitions on separate physical drives that I use mainly for ripping/muxing/demuxing video. The target filesystem was empty except for the 6 VOB files created by MuxMan.

I ran the stress test twice, with and without chapterpoints, with the same results.

RF

[mpucoder]

I just ran Muxman out to 9 vob files with no problems. It could be specific to the structure of your video, so create a profile and send it to muxman@mpucoder.com

[Stef]

Originally posted by Xesdeeni
1. I don't want the timecode (which is zero-based in AVISynth), I want the date/time stamp from the camcorder.

2. I don't want it burned in, I want it controlled on playback, just like a subtitle.

The idea is to see what date each clip in my home movies occurred, to see how old my children were when each thing happened, but not have it on screen at all times. I can do this manually, but it is a huge amount of work. I'd like a way to send the edited DV (from Premiere) through a filter that would create the subtitles for me, that I could add when I author the DVD.

Xesdeeni

This if offtopic for Muxman, therefore short version...

Maybe this link is useful for you:
DV_DATECODE (http://www.skydiver.de/stef/datecode_en.htm)

Stefan

[Xesdeeni]

Originally posted by Stef
Maybe this link is useful for you:
DV_DATECODE (http://www.skydiver.de/stef/datecode_en.htm)That looks like just the ticket. Thanks!

Now back to our regularly scheduled topic...

Xesdeeni

[katjarella]

@mpucoder
added single-frame still with audio

Many thank you. I am very lucky, it function. The first tests look very good.

[mpucoder]

I'm glad to hear that someone has used this feature.

[nnigam]

This is a great product. It works better then IFOEdit for me, but I have just started creating dvd's and my first dvd will be created using Muxman. But a few questions.

I see messages about P_STD buffer underflow, and talk of re-encoding. I too am getting the same errors. My process is as below

1. Record 1 hr HD content 720p from my MYHD tv card.
2. Demux using dvd2avi2.
3. Convert dvd2avi project to mpeg2 using QuEnc (Takes 10-12 hrs).
4. Process quenc mpeg file and dvd2avi ac3 file with muxman (takes 10 min).

In QuENC use GOP of 12, Max B-Frames of 2, DC Precision of 10. 16:9 aspect ratio and checked scene detection and auto max bit rate with AC3 audio.

Try 1. I used bitrate of around 9048 or something like that, (max bitrate -192 for the audio). Muxman processed about 600k and then died with numerious P_STD errors.

Try 2. I reduced bitrate to 4092-192. Muxman completed, but compalained that it may not play on my dvd player. It played fine on my computer dvd player, but audio was out of synch near the end. I suspect the reason was that in the advanced options page in quenc, I put audio bitrate of 256 instead of 192.

Try 3. I corrected the audio bit rate in QuEnc and am re-encoding. I am still getting P_STD errors, but it looks like the process will complete. This takes a long time (12 hrs per try in QuENC) and will see what happens.

Some questions.

1. DVD should have AUDIO_TS and VIDEO_TS directories. Muxman only creates files in the video_TS directory. What if anything goes into the AUDIO_TS directory.

2. My Audio file has a delay of 360 ms. I cannot set this in Muxman. I read about the audio delay in this chain, but if capturing from live broadcast, I have no control on the audio delay. Is there a tool to correct this delay.

log file looks something like this. (Removed 11 P_STD error lines)

MuxMan version 0.8
Opened video file D:\DH\T.m2v, 2328158682 bytes.
Opened audio 1 file D:\DH\pdvd AC3 T01 2_0ch 192Kbps DELAY -360ms.ac3.
AC3 audio, frame size = 0x300.
18:50:06 Begin multiplex.
P-STD buffer underflow by 16138 bytes at 125412554, sector 375592.
P-STD buffer underflow by 4535 bytes at 125415557, sector 375613.
...
...
P-STD buffer underflow by 4376 bytes at 334619552, sector 964022.
End of video file
Bytes remaining in buffer = 158330.
18:57:22 End multiplex.
Bitrate - avg: 4181065, min: 450109 (lba 1142416), max: 14198921 (lba 375329).
Fields: 284188, VOBU: 11077, Sectors: 1209917.