Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > (HD) DVD, Blu-ray & (S)VCD > (HD) DVD & Blu-ray authoring

Reply
 
Thread Tools Search this Thread Display Modes
Old 22nd September 2008, 17:55   #1  |  Link
kaid
Registered User
 
Join Date: Jul 2008
Posts: 96
.bdmv, .clpi, .mpls and .m2ts - Exact specs of AVCHD/BD structure and files

Okay, since I was simply unable to find anything specific on the structure and binary contents of the files on a BD/AVCHD, i thought it warrants its own thread (sorry if there already is one, I couldn't find anything upon searching!).

On making my AVCHD menu generator I learned a few things about the structure and the files, but the majority of the bytes remain a mystery to me. Since it's near impossible to get roman76 to react to queries we also don't know how he generates his files in tsMuxer when using the BD mux option. The official AVCHD Info site doesn't tell you shit about exact specs, all they give you is some vague info how the video streams should look like..

So here's my findings:
1) Structure:
Root is always the folders BDMV and CERTIFICATE. On FAT32 this needs to be within a folder called AVCHD and on optical media BDMV and CERTIFICATE need to be on the root level.
CERTIFICATE seems to be mandatory (probably for keyfiles) and contains an (empty) BACKUP folder for redundancy, AFAIK it is always empty for self-authored discs.

BDMV is where it's at, and in there we have:
AUXDATA: Auxilliary Data judging by the name. Probably bonus material for your computer like screensavers etc goes here.
BACKUP: Contains backups of the CLIPINFO and PLAYLIST directories along with the two .bdmv files in the BDMV directory. For redundancy
BDJO: Probably has to do with the Bluray Java, but I'm not sure in how far. This NES emulator for PS3 has a .bdjo file in there
CLIPINF: This contains one .clpi file for each .m2ts stream (also split ones) of the same name with exact specs about the stream
JAR: The Java Applets probably go in here.
META: Dunno. Metadata? D'oh, what kind of Metadata?
PLAYLIST: Contains a .mpls file for every track (not stream!) on the disc. So if a disc made with my script has 4 tracks, there's gonna be 4 mpls files in there, one for each, no matter into how many chunks the stream was split up (plus there must be -also with m2ts and clpi- 00000.mpls, which is what the BD/AVCHD starts with, in my case the menu track!)
STREAM: All .m2ts streams go in here. These are the actual content!

This structure must be followed 100% as far as I know, so even though they might be empty all these directories need to be there in this structure, with the exact names in caps!

2) Differences between AVCHD FAT32/UDF:
The only difference between AVCHD stored on a FAT32 volume and optical media lies in the filenames, because 8.3 naming is used on FAT32, and it's all caps:
*.clpi (UDF) becomes *.CLP (FAT32), *.mpls becomes *.MPL, *.m2ts becomes *.MTS, MovieObject.bdmv becomes MOVIEOBJ.BDM and index.bdmv becomes INDEX.BDM
Even though the .mpls files reference the .m2ts files internally, it seems there is some fault tolerance, probably required by the spec, so there's no need to change the string in the .mpls files accordingly from M2TS to MTS and vice versa.

3) File contents:
Unfortunately I was only able to find out something about the structure of .mpls playlist files by looking at it in a binary editor:
First there's a header, "MPLS0200", which is probably a file identifier with version number. Some bytes later (not always the same offset!), the M2TS file that is played first is referenced, (aways with a P infront).. Around 60 bytes later seems to be the language code, preceded with an "a" (so e.g. it's aeng for english). All of this minus the header is repeated in the same mpls file if one track has more chunks to be played sequentially.
The rest of the bytes is mostly zeroes, so reverse-engineering it should be possible...
.clpi files don't give much away, there's a header again (HDMV0200) and further down after lots of zeroes there's that language code with 'a' infront again..
index.bdmv is the same thing: Header (INDX0100) and mostly zeros...
MovieObject.bdmv: Header (MOBJ0100) and mostly zeros...

So. Does anyone know where to get further specs about this? I'd especially like to know how the menu m2ts (00000.m2ts) is structured so I could inject my own graphics in the menu generator! ;-) I would guess it is rather similar to the structure of those VTSx_0.VOBs on a DVD that contain the track's menu gfx!...

Last edited by kaid; 22nd September 2008 at 22:50.
kaid is offline   Reply With Quote
Old 24th September 2008, 14:14   #2  |  Link
kaid
Registered User
 
Join Date: Jul 2008
Posts: 96
Something else that would be of great interest are the contents of .clpi files, as I would like to be able to generate my own, so all that's needed for my script is m2ts files as input...

I just found this, a patent for recordable BDs that tells a lot about the filestructure, very very interesting:

Quote:
An example of the above-mentioned file structure is shown in FIG. 2. The file structure according to the present invention includes one or more BD directories (BDMV) in a single root directory. The BD directory includes an index file "index.bdmv" and an object file "MovieObject.bdmv" acting as a general file capable of ensuring user interactivity. The file structure includes a variety of directories for storing information of actual data recorded in a disc and other information associated with a method for reproducing the data, for example, a playlist directory (PLAYLIST), a clip information directory (CLIPINF), a stream directory (STREAM), and an auxiliary directory (AUXDATA). The above-mentioned directories and a variety of files included in the directory will hereinafter be described.

The AUXDATA directory includes an additional data file for reproducing data of the disc. For example, the AUXDATA directory includes a "Sound.bdmv" file for providing a user with sound data when an interactive graphic function is executed, and an "aaaaa.otf" file for providing the user with font information when data of the disc is reproduced. For the convenience of description, the AUXDATA directory will herein be omitted in figures since FIG. 6A.

The stream directory (STREAM) includes a plurality of AV (Audio and Video) stream files recorded in a disc according to a specific format, and also includes a file associated with interactive graphic streams. Generally, individual streams are recorded using an MPEG-2 based transport packet. Therefore, the stream directory (STREAM) uses extension names of stream files (1111.m2ts, 2222.m2ts, and 33333.m2ts) as a specific extension name "*.m2ts". The above-mentioned stream may also be called a clip file in the BD standard. Particularly, if video/audio/graphic information from among the above-mentioned streams are multiplexed, the multiplexed information is called an AV stream, and a single title is configured as at least one AV stream file.

The clip information (Clip-info) directory (CLIPINF) is composed of a plurality of clip-info files (11111.clpi, 22222.clpi, and 33333.clpi) connected to the above-mentioned stream files on a one-to-one basis. Particularly, the clip-info files (*.clpi) record attribute information and timing information of the stream files (i.e., clip files) therein. The clip-info files (*.clpi) connected to the stream files (*.m2ts) on a one-to-one basis are generically named a "Clip". In other words, this means that a single stream file (*.m2ts) must correspond to a clip-info file (*.clpi).

A clip associated with the main data recorded in the disc is called a main clip. A clip associated with downloaded sub-data stored in a local storage is called a sub-clip.

The playlist directory (PLAYLIST) includes a plurality of playlist files (*.mpls). Each playlist file (*.mpls) includes one or more playitems (PlayItem) for designating a playing interval during which a specific clip is reproduced. The playitem (PlayItem) includes information associated with a specific clip to be reproduced, i.e., information associated with a reproduction start time (IN-Time) and other information associated with a reproduction termination time (OUT-Time) of a clip determined to be a clip name (Clip__Information_File_name) contained in the playitem (PlayItem).

The playlist file (*.mpls) is used as a basic reproduction management file unit contained in an overall file structure, such that it can reproduce a desired clip using a combination of one or more playitems. The playlist file (*.mpls) may include data different from reproduction data formed by the playitem. The playlist file (*.mpls) may include a sub-playitem (SubPlayItem) distinguished from the playitem (PlayItem).

FIG. 3A is a relationship among files used for reproducing a specific title according to the present invention. Upon receiving a title reproduction command from a user in association with a specific title applied to an index table of an index file, a method for reproducing the specific title will hereinafter be described.

The index file (index.bdmv) includes first playback information "First Playback" indicative of information associated with a first reproduction image when a corresponding disc is loaded, top menu information "Top_Menu" for providing a menu image, and at least one title information "Title #1 ~Title #n".

If the user selects a specific menu from among a plurality of menus of a specific title or a menu image, a specific playlist is reproduced according to a command contained in individual objects "mobj" contained in an object file (MovieObject.bdmv) of a file structure, and a specific clip (denoted by "ClipInfo + AV stream") is reproduced by a playitem contained in the playlist.

Last edited by kaid; 24th September 2008 at 14:24.
kaid is offline   Reply With Quote
Old 26th September 2008, 15:00   #3  |  Link
turbojet
Registered User
 
Join Date: May 2008
Posts: 1,840
The way I understand and recently experienced is AVCHD is a digital camera format which always uses the FAT32 structure changes you mention, it doesn't have the FAT32 4GB restriction. It can be burned onto a DVD or BD-R and work on most BD players. Also from what I gather AVCHD has different specs then BD, for example no DTS support and different max video bitrate.
BD is the format that has BDMV\CERTIFICATE in root, what tsmuxer outputs and what your program is intended for.
I know it's easy to get these 2 confused because some programs output BD and call it AVCHD, and some players display AVCHD, when its technically BD.
turbojet is offline   Reply With Quote
Old 27th September 2008, 17:43   #4  |  Link
kaid
Registered User
 
Join Date: Jul 2008
Posts: 96
Quote:
Originally Posted by turbojet View Post
The way I understand and recently experienced is AVCHD is a digital camera format which always uses the FAT32 structure changes you mention, it doesn't have the FAT32 4GB restriction. It can be burned onto a DVD or BD-R and work on most BD players. Also from what I gather AVCHD has different specs then BD, for example no DTS support and different max video bitrate.
Yeah... Though PS3 still does DTS on AVCHD nonetheless! ;-)
From what i gathered AVCHD isn't always on FAT32, when it's a mini-DVD-R based camcorder it isn't, because then it uses UDF and the different naming scheme (atleast that's what the spec says!)..
But if the camcorder is using SD it HAS to use FAT32 (NTFS on Flashmedia isn't supported by anything!) and FAT32 *always* has the 4GB-limit!

Quote:
BD is the format that has BDMV\CERTIFICATE in root, what tsmuxer outputs and what your program is intended for.
Well, the PS3 says it's an AVCHD, not a BD. As long as I don't go overboard with the bitrate (and don't use DTS) it's the same anyway, right? ;-)

Quote:
I know it's easy to get these 2 confused because some programs output BD and call it AVCHD, and some players display AVCHD, when its technically BD.
Well, if the bitrate, use of DTS and the presence of a /CERTIFICATE folder are all that sets the two apart, I can understand the confusion! ;-)
Or is there some other differences I don't know about yet?
kaid is offline   Reply With Quote
Old 28th September 2008, 07:19   #5  |  Link
turbojet
Registered User
 
Join Date: May 2008
Posts: 1,840
The few avchd streams I've seen from avchd cameras have this structure.
turbojet is offline   Reply With Quote
Old 3rd October 2008, 10:58   #6  |  Link
frank
Registered User
 
Join Date: Oct 2001
Location: Germany
Posts: 743
I want to clarify:

AVCHD headers
index.bdmv - INDX0100
MovieObject.bdmv - MOBJ0100
PLAYLIST *.mpls - MPLS0100
CLIPINF *.clpi - HDMV0100

So NERO does it right.

Blu-ray headers on BD-RE
index.bdmv - INDX0200
MovieObject.bdmv - MOBJ0200
PLAYLIST *.mpls - MPLS0200
CLIPINF *.clpi - HDMV0200

ATTENTION!
tsMuxeR mixed up headers, it sets mpls and clpi headers as Blu-ray = WRONG!
You have to patch *.mpls and *.clpi to AVCHD compliant headers.

Last edited by frank; 3rd October 2008 at 11:01.
frank is offline   Reply With Quote
Old 29th January 2009, 04:29   #7  |  Link
smnckl
Registered User
 
Join Date: Jan 2009
Location: By the Yellow Brick Road
Posts: 58
@kaid

I know its been a while since anyone has contributed to this thread... and I am just looking for info about the layout of the *.clpi file(s).

My issue is that when using tsMuxeR to create a BD structure, it has on many occasions messed up the AR info found in the .clpi file. I have muxed source files which had original ARs 2.35 or 2.4 and tsMuxeR has it set to 4:3 in the .clpi file. When these are burned to disc and played on a standalone, I get a "good" picture on the left half of my 16:9 HDTV and a ghost-like similar image on the right, as tho it was 2 tracks running side-by-side.

I have tried using BDedit but can not edit the .clpi file. I am hoping that you, in your research, have found where in the .clpi the AR info is stored and could help me with the off-set so I could try hexing that to see whether I can get the correct AR without having to re-encode the source x264 file with borders.

TIA, James
__________________
Vista SP1 32-bit w/4GB RAM
AMD Athlon 64 X2 4400+ @ 2.23GHz
nVidia GeForce 8400GS w/512MB RAM
smnckl is offline   Reply With Quote
Old 29th January 2009, 09:37   #8  |  Link
deank
Programmer (or just 教务长)
 
deank's Avatar
 
Join Date: Oct 2008
Location: Sofia, Bulgaria
Posts: 4,247
Usually the AR value is at offset 0x0109 if you use tsMuxer's output.
__________________
multiAVCHD - donate | popBD | uncropMKV | mkv2avi | easySUP
deank is offline   Reply With Quote
Old 30th January 2009, 01:41   #9  |  Link
smnckl
Registered User
 
Join Date: Jan 2009
Location: By the Yellow Brick Road
Posts: 58
Quote:
Originally Posted by deank View Post
Usually the AR value is at offset 0x0109 if you use tsMuxer's output.
for the info, deank.
I took a look at the .clpi from a movie-only rip, using HxD and it shows a value of 30h at offset 109.
MediaInfo tells me that the movie has original AR of 2.4:1.

Where can I find which hex value equals 16:9 and which equals 4:3?

Thanks in advance for any pointers.

James
Attached Images
 
__________________
Vista SP1 32-bit w/4GB RAM
AMD Athlon 64 X2 4400+ @ 2.23GHz
nVidia GeForce 8400GS w/512MB RAM
smnckl is offline   Reply With Quote
Old 30th January 2009, 02:06   #10  |  Link
deank
Programmer (or just 教务长)
 
deank's Avatar
 
Join Date: Oct 2008
Location: Sofia, Bulgaria
Posts: 4,247
In AVCHD or BD you can have only 4:3 and 16:9 ARs.

Try changing $30 to $20.
__________________
multiAVCHD - donate | popBD | uncropMKV | mkv2avi | easySUP
deank is offline   Reply With Quote
Old 31st January 2009, 23:16   #11  |  Link
foolproof
Registered User
 
foolproof's Avatar
 
Join Date: Jan 2009
Posts: 1
to deank

I have mailed you before time links to download samples of AVCHD structure made by your (great) program and made by Panasonic HDWriter 2.5e for the same stream.
I hope you can see differences and this will helpful in your researche.
Your structure is on PanasonicTV unfortunately functionless, from HDWriter is OK. Has Panasonic in his structure some one secret byte or what ?
foolproof is offline   Reply With Quote
Old 31st January 2009, 23:26   #12  |  Link
deank
Programmer (or just 教务长)
 
deank's Avatar
 
Join Date: Oct 2008
Location: Sofia, Bulgaria
Posts: 4,247
The difference is in main INDEX/MOVIEOBJ files. I'm still looking at the folders you've sent me. Another user also sent me some files so now I have like 5-6 different configurations that play on Panasonic TVs.

I think pannies use shortened format and no 'real' playlists. I will try to post a link with a sample for you to test these days (before end of next week).

Dean

edit: Is there a way I can get hands on this Panasonic HD writer program? (is it free?)
__________________
multiAVCHD - donate | popBD | uncropMKV | mkv2avi | easySUP

Last edited by deank; 31st January 2009 at 23:29.
deank is offline   Reply With Quote
Old 1st February 2009, 07:33   #13  |  Link
smnckl
Registered User
 
Join Date: Jan 2009
Location: By the Yellow Brick Road
Posts: 58
Quote:
Originally Posted by deank View Post
In AVCHD or BD you can have only 4:3 and 16:9 ARs.

Try changing $30 to $20.
deank. I will try as you suggested.
I understand the limitation on AR but it seems that after I have backed up the movie, the 16:9 info in the stream is missing and gets replaced by an AR calculated from the actual pixels. The non-AVCHD/BD AR appears after I have run the movie through tsMuxeR.

Thanks again. I will get back here and let you know whether the AR change is successful.

James
__________________
Vista SP1 32-bit w/4GB RAM
AMD Athlon 64 X2 4400+ @ 2.23GHz
nVidia GeForce 8400GS w/512MB RAM
smnckl is offline   Reply With Quote
Old 1st February 2009, 22:38   #14  |  Link
smnckl
Registered User
 
Join Date: Jan 2009
Location: By the Yellow Brick Road
Posts: 58
Quote:
Originally Posted by deank View Post
In AVCHD or BD you can have only 4:3 and 16:9 ARs.

Try changing $30 to $20.
deank, your tip about the hex values worked. When value at offset 109h is $30, BDedit shows me that the DAR is 16:9 while $20 at offset 109h is shown as 4:3.

I am running a re-encode now to see whether x264 and/or tsMuxeR will accept the change and re-encode the stream as 16:9.

Thanks again.

James
__________________
Vista SP1 32-bit w/4GB RAM
AMD Athlon 64 X2 4400+ @ 2.23GHz
nVidia GeForce 8400GS w/512MB RAM
smnckl is offline   Reply With Quote
Old 9th February 2009, 08:43   #15  |  Link
mrr19121970
Registered User
 
Join Date: Mar 2008
Posts: 305
I wrote a little utility that can do this for you, incase you're not too comfortable with a HEX editor:

Editing CLIPINF (*.CLPI or *.CPI)
mrr19121970 is offline   Reply With Quote
Old 6th May 2009, 11:10   #16  |  Link
twazerty
AVCHDCoder developer
 
Join Date: Feb 2009
Location: Oss, Netherlands
Posts: 266
Sorry for kicking this topic but I do have a question. tsMuxer can only handle 1 title. I used Nero Vision to create a multi title disc. Lets say I've put 8 items on a disc with a menu and intro. These files do have the right AVCHD headers etc. But I want to burn the files on a blu-ray disc. Are the only differences the headers? Or are there more things that are different?

So is changing the headers enough to make it fully 100% Blu-ray compatible?

*Also renaming those files but I already know that.
twazerty is offline   Reply With Quote
Old 6th May 2009, 11:57   #17  |  Link
deank
Programmer (or just 教务长)
 
deank's Avatar
 
Join Date: Oct 2008
Location: Sofia, Bulgaria
Posts: 4,247
1) You could have selected blu-ray output in Nero
2) No need to change anything in nero's avchd output (just filenames to long format)

What player will you be using with this bd disc?
__________________
multiAVCHD - donate | popBD | uncropMKV | mkv2avi | easySUP
deank is offline   Reply With Quote
Old 6th May 2009, 13:27   #18  |  Link
twazerty
AVCHDCoder developer
 
Join Date: Feb 2009
Location: Oss, Netherlands
Posts: 266
Quote:
Originally Posted by deank View Post
1) You could have selected blu-ray output in Nero
2) No need to change anything in nero's avchd output (just filenames to long format)

What player will you be using with this bd disc?
Don't have a blu-ray burner so no Blu-ray output. Only AVCHD output. I have the Samsung BD-P1500.

I have to tacke this possible problem for AVCHDCoder. If changing the Header to 0200 is enough it would be nice.
twazerty is offline   Reply With Quote
Old 23rd August 2009, 20:53   #19  |  Link
deank
Programmer (or just 教务长)
 
deank's Avatar
 
Join Date: Oct 2008
Location: Sofia, Bulgaria
Posts: 4,247
Just a reviver note, inspired by an email correspondence.

Although I posted about 20 or 30 times in different threads it seems that it is missed again and again.

0100 header value for clpi, mpls and bdmv files is a BLU-RAY VERSION ID. Changing it to 0200 does NOTHING (except in the cases you really know why you change/author it). 0100 is both for AVCHD and Blu-ray (profile 1.0, 1.1). If you by any chance self-author a BD-J BD or use advanced authoring (with an application you created) then may be 0200 may be a proper header. In all other cases you shouldn't play with these values (not that it will make a difference). This overrated discussion has to come to an end, although certain users decided to confuse people over and over again.

tsMuxeR (1.10.+) output for

BLU-RAY:
* is compliant blu-ray output

AVCHD:
* is BLU-RAY compliant output with AVCHD index.bdmv (so it is a mix)

Now, with AVCHD output from tsMuxeR (which was AVCHD-only since earlier versions) came a lot of questions, troubles, player compatibility reports, etc.

People started playing with deleting folders, changing headers, adding and changing bytes here and there.

So - play with them... And you/we will waste another 2 years finding out what really works.

And a final note: AVCHD is much more complicated format, compared to standard BDMV/BDAV and even BD-J. That is the reason a lot of manufacturers dropped AVCHD support in their players in recent firmware updates. In Blu-ray mode, firmware updates change/update the JAVA core errors and some advanced playlist handling, but the AVCHD part remains the same. It is just something set by sony and panasonic some time ago. You can't update millions of camcoders and TVs around the world with 'ethernet-firmware-update'.

Dean
__________________
multiAVCHD - donate | popBD | uncropMKV | mkv2avi | easySUP

Last edited by deank; 23rd August 2009 at 21:00.
deank is offline   Reply With Quote
Old 1st June 2011, 20:56   #20  |  Link
xpro
Registered User
 
Join Date: Nov 2010
Location: Uppsala, Sweden
Posts: 2
.M2TS file structure for demux?

Hello!

Anyone who has info about how the .M2TS files are structured? How to parse them to be able to demux the video/audio/PGS? Do you know any open source code for demuxing a M2TS / MTS file?

Regards, Tobias
xpro is offline   Reply With Quote
Reply

Tags
avchd, bluray, clpi, m2ts, structure

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 14:03.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.