Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
4th May 2009, 09:42 | #1 | Link |
Registered User
Join Date: Apr 2009
Location: Moscow, Russia
Posts: 61
|
Another AVCHD/Blu-Ray demuxer
I'm proud to present my own M2TS files demuxer taken from AVCHD camera and Blu-ray disks. Utility have a number of advantages and features when you working with AVCHD materials.
There is a major issue: joining multiple scenes located in different M2TS (MTS)/TS files into one file with audio/vidio sync (optional). After brief topic examination it came to me that such thing can be done by eac3to, tsMuxer or simple 'copy /b' command in Windows and 'cat' in *nix-es, just addition one file in the end of another. But in practice this is not good idea. The question is each M2TS file audio and video may be varied by length. This is urgent for the NTSC with fractional FPS, for example 29.97. Audio length (AC3 fps) and video frames (H264 JVT NAL) could not be variable, consequently there is no way to combine they at the end of the TS stream. Easiest way is to adjust video to audio, in this case variable fps is needed. In result we have audio stream 1 msec longer then video. It is not hard to calculate that for the 100 scenes we will get tremendous audio delay for 100 msec at the end. When playing M2TS files one after one player begins from start and there are no visible synchronization issues. But when we join files, extracting elementary streams and muxing to MKV we are dropping information for synchronization completely. Muxer (mkvmerge) have no idea about initial TS stream properties and therefore sets its own time marks, supposing video and audio synchronized (I would remind you h.264 stream doesn't contain any metadata about FPS and so on). Here the program features: * runs either by Windows, Linux and FreeBSD; * completely opensource , GPLv2; * understands M2TS and TS file formats (type detetection by magic sequence); * batch processing; * joining multiple scenes; * demux to ES or PES streams; * extensions for the output files would be set depending of elementary stream; * takes files list and/or directories list as command line argument (it is possible to specify a path to files for example private/avchd/bdmv/stream, program will take all appropriate files by itself); * extracts shooting date and time of the each clip from mpls files in camera (private/avchd/bdmv/playlist/*.mpl); * extracts elementary streams all programs in TS container or selected by number; * makes files with time codes for appropriate elementary streams synchronizationl (mkvmerge timecode format v2); * determining h.264 and AC3 elementary stream FPS; * determining audio delay after video in each file and brings appropriate corrections into time codes (urgent for Blu-ray, you not needed to calculate and specify delay by yourself when muxing into MKV); * trying to align elementary streams endings inside each scene; * produces file with scenes for saving information about initial material dividing into multiple M2TS files. Scene names gives after file names or shooting time/date from the playlist (mkvmerge xml format); * produces SRT file with subtitles for saving information about shooting time/date from the playlist; * shows invoking mkvmerge command for muxing produced material into MKV; * support MPEG2 Video, MPEG2 Audio, H.264, VC-1, AC3, L-PCM elementary stream types; * parses and shows information about TS stream without elementary streams extraction (shows streams types. length, offset, and fps); * dump TS structure (M2TS timecodes, PTS/DTS ...). Unfortunately there is only console version available yet. It was tested against AVCHD cameras by Sony, Canon, Panasonic, some Blu-ray disks and broadcast TS. Competitors disadvantages: * xport - Unable to join files and can't take TS from the standard input. Output file names not reflecting content type; * eac3to - No source code available, only Windows; * tsMuxer - Also no source code, not so useful console version, joining files in GUI version is not comfortable. And no one of them able to form files for proper streams synchronization for mkvmerge, no ability to generate file with scenes list, and no shooting time/date extraction. I suppose my program may be an alternative to xport. GUI Windows setup (win32): http://tsdemuxer.googlecode.com/file...uxer_setup.exe Windows binaries (win32): http://tsdemuxer.googlecode.com/files/tsdemux.zip Linux binaries (gcc4, static): http://tsdemuxer.googlecode.com/files/tsdemux.tar.gz Sources: http://code.google.com/p/tsdemuxer/source/browse Project home page: http://code.google.com/p/tsdemuxer Simple invocation form: 'tsdemux.exe -m -j -d x:\private\avchd\bdmv\stream' or 'tsdemux.exe -j 00001.mts 00002.mts'. Result will be in five files in current directory: video stream , audio stream, tmc-file for audio, tmc-file for video, xml-file with scenes. Then you should pass all of these to mkvmerge. Command line options: tsdemux.exe [-d src] [-l mpls] [-o dst] [-c channel] [-u] [-j] [-m] [-z] [-p] [-e mode] [-v] *.ts|*.m2ts ... -d discard all TS/M2TS/MTS files from specified directory (use it multiple times for different directories, files will be sorted ascending). For the AVCHD/Blu-ray it should be 'private/avchd/bdmv/stream'. -l path to AVCHD/Blu-ray playlist mpl/mpls file (usually it is 'private/avchd/bdmv/playlist/*.mpls'). If this option set then for the names of scenes shooting time/date will be used also. -o redirect output to another directory -c channel number in TS stream (if not specified then all streams from all programs will be extracted, but scene file for mkvmerge will not generated). -u demux unknown streams -j join elementary streams -m shows mkvmerge invoking options for muxing results in MKV. -z demux to PES streams (instead of elementary streams). -p deny elementary streams extraction, only analyze with information output. -e dump TS structure to STDOUT (mode=1: dump M2TS timecodes, mode=2: dump PTS/DTS, mode=3: human readable PTS/DTS dump) -v turn on verbose output Valid files extensions: m2ts, mts, ts. Output files could be with extensions: sup, m2v, 264, vc1, ac3, m2a, pcm. Files with time codes for each elementary stream for mkvmerge (tmc) and mkvmerge scene file (chapter.xml) will be generated as well. Send bugreports to comments, please Last edited by clark15b; 28th August 2009 at 13:12. Reason: version 1.52 |
4th May 2009, 16:27 | #5 | Link |
Registered User
Join Date: Apr 2009
Location: Moscow, Russia
Posts: 61
|
Currently it is of importance for me to test program functions with different signal sources (different AVCHD camcorders, Blu-ray disks with different codecs). Will be very gratefull for help with this. I will work on GUI in the mean time.
|
4th May 2009, 16:36 | #6 | Link |
Programmer (or just 教务长)
Join Date: Oct 2008
Location: Valencia, Spain
Posts: 4,251
|
A lot of people willing to help you in testing will have troubles without a GUI. You can get test results faster if you compile one for them.
I don't have AVCHD camcoder, but I have a lot of samples and will do some tests and will let you know. Dean |
4th May 2009, 18:36 | #8 | Link |
Registered User
Join Date: Jan 2007
Posts: 530
|
I have tried a few AVCHD files and I am having an issue with the resultant MKV. Whether I mux it myself with MkvmergGUI or your command line, the resultant MKV file takes a very long time to open and it is solid disk activity while waiting for it to open.
I'm using Haali Media Splitter and the 2.7.0 MKVToolnix and I've tried opening it with MPC-HC and jRiver MC13, both exhibit the same extended time to open the MKV. Once open, playback is as expected. HTH |
5th May 2009, 08:41 | #9 | Link |
Registered User
Join Date: Apr 2009
Location: Moscow, Russia
Posts: 61
|
Thanks for your interest, I will make GUI soon.
noee I do not think that a problem in muxer and MKV, I have no problem with reproduction of original M2TS files and resulting MKV on the computer. I use Haali Media Splitter, CoreAVC and MediaPlayer Classic. |
5th May 2009, 08:57 | #10 | Link |
Registered User
Join Date: May 2008
Posts: 1,840
|
Firet impressions is it's really fast, only seconds slower then tsmuxer. eac3to and xport are slow in comparison for some reason.
If it had mkv/mp4 input, avchd output, sup/srt input/output, and multiple audio support I'd be real interested in it mainly because of the cli. Do you have any plans for any of these? I tried it on a retail VC-1 BD and found some problems. - detected the 23.976 VC-1 stream as 29.97 fps, tsmuxer and eac3to detect 23.976. - when I used -l with BDMV\PLAYLIST directory on same BD and got 'invalid file format' - -d BDMV\PLAYLIST\00000.mpls (main movie) crashed tsdemux - chapters.xml only contained 3 chapters starting at 3 hours, the movie is only 90 minutes long - After demuxing the main movie m2ts on same BD it gave these errors: Code:
** bad last PTS in stream 0xfd, use first PTS and current frame rate for fix it ** bad last PTS in stream 0xfd, use first PTS and current frame rate for fix it ** bad last PTS in stream 0xbd, use first PTS and current frame rate for fix it ** bad last PTS in stream 0xbd, use first PTS and current frame rate for fix it ** bad last PTS in stream 0xbd, use first PTS and current frame rate for fix it K:\BDMV\STREAM\00000.m2ts: channel 1, stream 0xfd ('vc1'), length 5212595.84ms ( +1244.71ms) ** align stream 0xfd length to 14315708.00ms K:\BDMV\STREAM\00000.m2ts: channel 1, stream 0xfd ('ac3'), length 5212256.00ms ( +904.87ms) ** align stream 0xfd length to 14315708.00ms K:\BDMV\STREAM\00000.m2ts: channel 1, stream 0xfd ('ac3'), length 5212256.00ms ( +904.87ms) ** align stream 0xfd length to 14315708.00ms K:\BDMV\STREAM\00000.m2ts: channel 1, stream 0xfd ('ac3'), length 5212256.00ms ( +904.87ms) ** align stream 0xfd length to 14315708.00ms K:\BDMV\STREAM\00000.m2ts: channel 1, stream 0xfd ('bin'), length 5347985.00ms ( +136633.87ms) ** align stream 0xfd length to 14315708.00ms K:\BDMV\STREAM\00000.m2ts: channel 1, stream 0xfd ('ac3'), length 5212256.00ms ( +904.87ms) ** align stream 0xfd length to 14315708.00ms K:\BDMV\STREAM\00000.m2ts: channel 1, stream 0xbd ('bin'), length 14257757.24ms (+9104356.30ms), delay 57950.19ms ** align stream 0xbd length to 14315708.00ms K:\BDMV\STREAM\00000.m2ts: channel 1, stream 0xbd ('bin'), length 10041529.73ms (+4926292.13ms), delay 96113.53ms ** align stream 0xbd length to 14315708.00ms K:\BDMV\STREAM\00000.m2ts: channel 1, stream 0xbd ('bin'), length 10256699.44ms (+5112182.63ms), delay 66834.32ms ** align stream 0xbd length to 14315708.00ms K:\BDMV\STREAM\00000.m2ts: channel 1, stream 0xbd ('bin'), length 4963921.22ms ( +2.01ms), delay 247431.92ms ** align stream 0xbd length to 14315708.00ms K:\BDMV\STREAM\00000.m2ts: channel 1, stream 0xbd ('bin'), length 4963919.87ms, delay 247431.27ms ** align stream 0xbd length to 14315708.00ms ** bad last PTS in stream 0xbd, use first PTS and current frame rate for fix it K:\BDMV\STREAM\00001.m2ts: channel 1, stream 0xfd ('vc1'), length 11970.28ms (+1 0288.73ms) K:\BDMV\STREAM\00001.m2ts: channel 1, stream 0xfd ('ac3'), length 12000.00ms (+1 0318.46ms) K:\BDMV\STREAM\00001.m2ts: channel 1, stream 0xbd ('bin'), length 745.34ms, dela y 936.20ms ** bad last PTS in stream 0xfd, use first PTS and current frame rate for fix it K:\BDMV\STREAM\00002.m2ts: channel 1, stream 0xfd ('vc1'), length 11887.67ms K:\BDMV\STREAM\00002.m2ts: channel 1, stream 0xfd ('ac3'), length 11904.00ms (+1 6.33ms) K:\BDMV\STREAM\00003.m2ts: channel 1, stream 0xfd ('vc1'), length 5872.53ms ** align stream 0xfd length to 5873.00ms ** bad last PTS in stream 0xfd, use first PTS and current frame rate for fix it K:\BDMV\STREAM\00004.m2ts: channel 1, stream 0xfd ('vc1'), length 23274.80ms ** align stream 0xfd length to 23275.00ms ** bad last PTS in stream 0xbd, use first PTS and current frame rate for fix it K:\BDMV\STREAM\00005.m2ts: channel 1, stream 0xbd ('bin'), length 1363.13ms ** align stream 0xbd length to 1364.00ms ** bad last PTS in stream 0xbd, use first PTS and current frame rate for fix it K:\BDMV\STREAM\00006.m2ts: channel 1, stream 0xbd ('bin'), length 1386.13ms ** align stream 0xbd length to 1387.00ms ** bad last PTS in stream 0xbd, use first PTS and current frame rate for fix it ** bad last PTS in stream 0xbd, use first PTS and current frame rate for fix it ** bad last PTS in stream 0xbd, use first PTS and current frame rate for fix it ** bad last PTS in stream 0xbd, use first PTS and current frame rate for fix it ** bad last PTS in stream 0xbd, use first PTS and current frame rate for fix it K:\BDMV\STREAM\00007.m2ts: channel 1, stream 0xfd ('vc1'), length 10010.00ms (+8 386.79ms) K:\BDMV\STREAM\00007.m2ts: channel 1, stream 0xfd ('ac3'), length 10016.00ms (+8 392.79ms) K:\BDMV\STREAM\00007.m2ts: channel 1, stream 0xfd ('ac3'), length 10016.00ms (+8 392.79ms) K:\BDMV\STREAM\00007.m2ts: channel 1, stream 0xfd ('ac3'), length 10016.00ms (+8 392.79ms) K:\BDMV\STREAM\00007.m2ts: channel 1, stream 0xfd ('ac3'), length 10016.00ms (+8 392.79ms) K:\BDMV\STREAM\00007.m2ts: channel 1, stream 0xfd ('ac3'), length 10016.00ms (+8 392.79ms) K:\BDMV\STREAM\00007.m2ts: channel 1, stream 0xbd ('bin'), length 687.01ms, dela y 936.20ms K:\BDMV\STREAM\00007.m2ts: channel 1, stream 0xbd ('bin'), length 687.01ms, dela y 936.20ms K:\BDMV\STREAM\00007.m2ts: channel 1, stream 0xbd ('bin'), length 687.01ms, dela y 936.20ms K:\BDMV\STREAM\00007.m2ts: channel 1, stream 0xbd ('bin'), length 687.01ms, dela y 936.20ms K:\BDMV\STREAM\00007.m2ts: channel 1, stream 0xbd ('bin'), length 687.01ms, dela y 936.20ms ** bad last PTS in stream 0xbd, use first PTS and current frame rate for fix it K:\BDMV\STREAM\00008.m2ts: channel 1, stream 0xbd ('bin'), length 207.40ms ** align stream 0xbd length to 208.00ms ** bad last PTS in stream 0xbd, use first PTS and current frame rate for fix it K:\BDMV\STREAM\00009.m2ts: channel 1, stream 0xbd ('bin'), length 207.40ms ** align stream 0xbd length to 208.00ms Last edited by turbojet; 5th May 2009 at 09:20. |
5th May 2009, 09:23 | #12 | Link |
Registered User
Join Date: May 2008
Posts: 1,840
|
Oh ok shucks I guess I'll have to keep trying to get through to tsmuxer developer to add some simple cli. Thanks anyways.
After muxing the demuxed files to m2ts/mkv files it chose the last audio track to demux and it needs about +200 ms delay. Last edited by turbojet; 5th May 2009 at 09:36. |
5th May 2009, 09:47 | #13 | Link |
Registered User
Join Date: Apr 2009
Location: Moscow, Russia
Posts: 61
|
turbojet
Thank you! Do you use Windows or Unix? Could you please send me BDMV/PLAYLIST (at least several files)? Please specify command line which you were using. Could you please send a part of main movie M2TS file? |
5th May 2009, 09:56 | #14 | Link |
Registered User
Join Date: May 2008
Posts: 1,840
|
This was windows, I could test linux if you think it would make a difference.
Here's the files you requested. I used tsdemux -d K:\BDMV\STREAM\00000.m2ts to demux |
5th May 2009, 12:41 | #19 | Link |
Programmer (or just 教务长)
Join Date: Oct 2008
Location: Valencia, Spain
Posts: 4,251
|
clark, I'd suggest you this:
Code:
50 if(memcmp(ptr,"MPLS0100",8)) 51 return -1; and Code:
58 unsigned char* pp=ptr+68; 60 if(pp+num*82>ptr+len) ... 72 pp+=82; Then for the next cycle use pp as the base for other chapter calculations. Last edited by deank; 5th May 2009 at 12:47. |
Tags |
avchd, blu-ray, mux |
Thread Tools | Search this Thread |
Display Modes | |
|
|