Converting Bluray disc subtitles

[Rectal Prolapse]

I'm wondering if this is possible yet.

I've tried ripping the data from a bluray disc's .m2ts file. I believe that PIDs starting from 1200 are the subtitle streams, but I have no idea what to do with them.

Anyone have any information on how to convert them to SRTs? Or where I can find the format specifications for them?

[Pelican9]

I'm wondering if this is possible yet.

I've tried ripping the data from a bluray disc's .m2ts file. I believe that PIDs starting from 1200 are the subtitle streams, but I have no idea what to do with them.

Anyone have any information on how to convert them to SRTs? Or where I can find the format specifications for them?

I think it's similar to the HD DVD's subtitle stream.
Could you send me a sample?

[Rectal Prolapse]

I'll try to do that this weekend. The router goes down a lot though, so that may not happen. :(

[DeepBeepMeep]

I'll try to do that this weekend. The router goes down a lot though, so that may not happen. :(

Rectale Prolapse, which tool have used to extract the stream? Thanks. I might try to use it as well to upload a sample for Pelican.

[Rectal Prolapse]

I used TSReader Lite. There are also pay versions of those, but I think the Lite version is all we need right now. I guessed that the PIDs for subs begin at PID 1200. I hope I am right!

I would like to try it with VLC too - and see if I get similar results.

[enantiomer]

Rectal Prolapse, I agree that PID 0x1200 and up are probably where the susbtitles streams start.

Pelican9: did you receive a Blu-ray subtitle stream sample? BTW thanks for your efforts on SUPread. Nice little proggie.

[cfsmp3]

Any chance I can get a sample as well? I'm doing lots of subtitles related work these days.

[Pelican9]

Pelican9: did you receive a Blu-ray subtitle stream sample?

Yes, I did. It's not the same as the HD DVD's subtitle stream. I think it contains some headers.

Any chance I can get a sample as well? I'm doing lots of subtitles related work these days.

http://www.sendspace.com/file/ce4m87

If you figure out the file structure, I'll change SUPread.

[Rectal Prolapse]

Here is another rip of subtitles, from Memento blu-ray:

http://www.sendspace.com/file/ghi4gb

This should be a better rip than the first one, where I used TSReader Lite that doesn't rip from the beginning of the stream!

[Pelican9]

Here is another rip of subtitles, from Memento blu-ray:

http://www.sendspace.com/file/ghi4gb

This should be a better rip than the first one, where I used TSReader Lite that doesn't rip from the beginning of the stream!

I haven't got any info from these files. :-(

[Rectal Prolapse]

Yeah I know how you feel - it's a big job figuring these files out!

Fortunately, I think I figured out the Run Length Encoding scheme used for the bitmaps. However, I haven't figured out where the subtitle time and duration information is stored, and I also need to decode the color table somehow.

Some examples from my RLE findings:

00 47 80 00 00 = The next 1920 pixels across are of color index 0. The 00 pair at the end signifies the end of a line.

00 84 0e = The next 4 pixels are of the color index 14 (decimal).

00 9e 14 = The next 30 (0x1e) pixels are of the color 20.

00 c1 4e 33 = the next 334 pixels are of the color 51.

If the sequence doesn't begin with a 00 then the value indicates the color of the pixel at that point.

If the sequence ends with two 00's then that means we reached the end of the line.

[Rectal Prolapse]

More:

00 03 = The next 3 pixels use color index 0.

00 01 = The next pixel is color index 0.

So, basically, 00 is reserved as an RLE command. To get the color 0 you need to do the above.

Given the above, here are some examples:

00 47 80 00 00 00 42 0F 0E 14 00 84 FE 14 0E 00 0F

Breaks down into:

00 47 80 = 1920 pixels of color 0.
00 00 = end of line, go to the beginning of the next line.
00 42 0F = 527 pixels of color 0
0E = pixel at x location 528 is color 14.
14 = pixel at x location 529 is color 20.
00 84 FE = the next 4 pixels are of color 254.
14 = the next pixel is color 20.
0E = the next pixel is color 14.
00 0F = the next 15 pixels are color 15.

This will go on until 1920 pixels for the line have been specified, and shall end in a 00 00 pair.

Now, not all subpictures will be 1920 pixels across - this depends on the header before the bitmap info.

[Pelican9]

rle_coded_line() {
do {
if (nextbits != ‘0000 0000b’) {
pixel_code (8bit)
} else {
8-bit_zero (8bit)
switch_1 (1bit)
switch_2 (1bit)
if (switch_1 == ‘0b’) {
if (switch_2 == ‘0b’) {
if (nextbits != ’00 0000b’)
run_length_zero_1-63 (6bit)
else
end_of_line_signal (6bit)
} else {
run_length_zero_64-16K (14bit)
}
} else {
if (switch_2 == ‘0b’) {
run_length_3-63 (6bit)
pixel_code (8bit)
} else {
run_length_64-16K (14bit)
pixel_code (8bit)
}
}
}
} while (!end_of_line_signal)
}


pixel_code: An 8-bit code, specifying the pixel value as an entry number of a Palette with 256 entries.

8-bit-zero: An 8-bit field filled with ‘0000 0000b’.

switch_1: A 1-bit switch that identifies the meaning of the fields that follow: if set to the value ‘0b’, the
field indicates a run-length for a pixel value of ‘0x00’ or an end_of_line_signal; if set to the value ‘1b’,
the field indicates a run-length for a pixel value that is not ‘0x00’.

switch_2: A 1-bit switch that identifies the meaning of the fields that follow: if set to the value ‘0b’, the field indicates a small run-length or end_of_line_signal; if set to the value ‘1b’, the field indicates a long run-length.

run_length_zero_1-63: The number of pixels that shall be set to a value of ‘0x00’.

end_of_line_signal: A 6-bit field filled with ’00 0000b’. The presence of this field signals the end of the coded line.

run_length_3-63: The number of pixels that shall be set to the pixel value defined next. This field shall not have a value less than 3.

run_length_zero_64-16K: The number of pixels that shall be set to a value of ‘0x00’. This field shall not have a value less than 64.

run_length_64-16K: The number of pixels that shall be set to the pixel value defined next. This field shall not have a value less than 64.

[Rectal Prolapse]

Thanks Pelican9!

Last week I spent a lot of time doing more work reverse engineering m2ts subpicture streams.

I believe I have figure out the essentials. I will try to post more details in the coming days.

I can say with absolute certainty that the timestamps are NOT included in the elementary PG stream. They are embedded in the PTS values of the PES packets before each new subtitle.

These PES packets should be extracted by a tool (like TSReader or Manzanita) that preserves all the packets in the stream. Unfortunately this means we will need a transport stream parser.

I also discovered how subtitles are forced on.

There are separate commands for define the palette, the bitmap, and a command for blanking the subtitle. The latter is preceded by a PES packet that specifies the PTS.

One problem remaining is determining the subtitle timing relative to the presentation of the first video frame. I don't yet know how to get this value.

I hope to provide very specific details soon. Hopefully you and Haali and whoever else can make use of this information.

BTW, I used a clean-room approach to this - I do not have the full Blu-ray specification, only the publically available one. I also used the XVI hex editor and PowerDVD 7.3 for my experiments. Lots of trial and error was involved. Thanks to everyone who helped!

[Pelican9]

I can say with absolute certainty that the timestamps are NOT included in the elementary PG stream. They are embedded in the PTS values of the PES packets before each new subtitle.
Same as HD DVD.

These PES packets should be extracted by a tool (like TSReader or Manzanita) that preserves all the packets in the stream. Unfortunately this means we will need a transport stream parser.
I've just ask dmz01 to make a subtitle extractor in his app. (called TsRemux).
If he can make a .sup file I will change SUPread to handle it.

There are separate commands for define the palette, the bitmap, and a command for blanking the subtitle. The latter is preceded by a PES packet that specifies the PTS.
Same as HD DVD.

One problem remaining is determining the subtitle timing relative to the presentation of the first video frame. I don't yet know how to get this value.

Same as HD DVD. :-) We have to find the first video PTS and then subtract this value from every subtitle PTS. (It's very easy to find on HD DVD's DSI packet.)

[Rectal Prolapse]

Oh, I'm sure HD-DVD is the same. :)

I find it very strange that no one has posted their findings on these formats before though - is there another forum for that?

EDIT: Nevermind. :)

[Rectal Prolapse]

The subpicture stream consists of data inside sections. Each section begins with an identifier byte, followed by a 16 bit integer that contains the length of the data, followed by the data itself.

Identifiers, order of presentation found in typical subtitle stream:

16 00 13 07 80 04 38 40 00 01 00 00 00 01 00 00 00 00 00 00 00 00

16 = identifier
00 13 = size of section
07 80 04 38 = 1920x1080.
01 = do not clear. If set to 00 instead it will clear the subpicture!

40 00 01 00 = sequence number??

00 = if set to 40 instead of 00 the next subtitle will be forced on.

17 00 0A 01 00 00 00 00 00 07 80 04 38

17 = ?
07 80 04 38 = 1920x1080

14 04 FD 00 00 00 10 80 80 00

14 = palette definition
04 FD = size of section following in bytes.
00 10 80 80 00 = color index 0 with YCbCr = 10,80,80, alpha channel = 0.

15 89 92 00 00 01 C0 00 89 8B 07 80 04 38 00 47 80 00 00

15 = bitmap picture section
89 92 = size of section in bytes following.
00 00 01 C0 00 = unknown !
89 8B = appears to be another length indicator. 89 92 minus 5 bytes.
07 80 04 38 = 1920x1080 image dimensions.

80 00 00
80 = end of picture? No section? End of Epoch? Go blank?

[Rectal Prolapse]

We have to find the first video PTS and then subtract this value from every subtitle PTS. (It's very easy to find on HD DVD's DSI packet.)

This didn't seem to work on the test stream I had. It appeared to be off by 300-400ms. But then again it had VC-1 in it so maybe the TS analyzer (without VC-1 support) I had didn't know where the real video frame was.

[Rectal Prolapse]

BTW Pelican9, you implied that you already knew the format - is that true? If dmz01 can write a subtitle extractor then he must know the format already?

Is there a reason why this information isn't more widely known? On the other hand, I don't have any special access to Blu-ray specs - so NDAs and licenses are not a concern for me - just old-fashioned true reverse engineering. :)

[drmpeg]

Here's an update to xport that demuxes the subtitle (Presentation Graphics) stream.

http://www.w6rz.net/xportpgs.zip

An extra parameter has been added to select the subtitle stream.

xport -h movie.m2ts 1 1 1 1

Output filename is bits0001.pgs and the -u option dumps the PTS.

Ron

[jokin]

ok now we can extract is there any way to put them on a video via avs?

[Pelican9]

Here's an update to xport that demuxes the subtitle (Presentation Graphics) stream.

http://www.w6rz.net/xportpgs.zip

An extra parameter has been added to select the subtitle stream.

xport -h movie.m2ts 1 1 1 1

Output filename is bits0001.pgs and the -u option dumps the PTS.

Ron
You are very fast!
Thanks!

The -h worked well and create a huge pgs (43MB).
How does the -u option work?

Transport Sync Error, skipped 4 bytes, at 3218297
Transport Sync Error, skipped 4 bytes, at 3218298
Transport Sync Error, skipped 4 bytes, at 3218299
Transport Sync Error, skipped 4 bytes, at 3218300
Transport Sync Error, skipped 4 bytes, at 3218301
Transport Sync Error, skipped 4 bytes, at 3218302
Transport Sync Error, skipped 4 bytes, at 3218303
Transport Sync Error, skipped 4 bytes, at 3218304
Transport Sync Error, skipped 4 bytes, at 3218305
Transport Sync Error, skipped 4 bytes, at 3218306
Transport Sync Error, skipped 4 bytes, at 3218307


Is it possible to insert the PTS information to the subtitle stream and write a (semi) compatible .sup file?
The .sup file contains 10 extra byte before every subtitle
'SP',PTS(4bytes),0x00000000

Edit:
We can create a new file format for BD subtitles. We can use other two letters instead of 'SP' (for example 'BD' or something), and the 4 bytes long PTS value, and the subtitle segment data.

[Pelican9]

BTW Pelican9, you implied that you already knew the format - is that true? If dmz01 can write a subtitle extractor then he must know the format already?
Yes.
No.

Is there a reason why this information isn't more widely known?
Yes. This information is confidental.

[drmpeg]

You are very fast!
Thanks!

The -h worked well and create a huge pgs (43MB).
How does the -u option work?

Transport Sync Error, skipped 4 bytes, at 3218297
Transport Sync Error, skipped 4 bytes, at 3218298
Transport Sync Error, skipped 4 bytes, at 3218299
Transport Sync Error, skipped 4 bytes, at 3218300
Transport Sync Error, skipped 4 bytes, at 3218301
Transport Sync Error, skipped 4 bytes, at 3218302
Transport Sync Error, skipped 4 bytes, at 3218303
Transport Sync Error, skipped 4 bytes, at 3218304
Transport Sync Error, skipped 4 bytes, at 3218305
Transport Sync Error, skipped 4 bytes, at 3218306
Transport Sync Error, skipped 4 bytes, at 3218307


Is it possible to insert the PTS information to the subtitle stream and write a (semi) compatible .sup file?
The .sup file contains 10 extra byte before every subtitle
'SP',PTS(4bytes),0x00000000
Try:

xport -hus movie.m2ts 1 1 1 1

Ron

[Pelican9]

Try:

xport -hus movie.m2ts 1 1 1 1

Ron

I've tried, but nothing changed in the file.

[drmpeg]

I've tried, but nothing changed in the file.
I meant just to get rid of the error messages and dump the PTS. I'll add the PTS header tomorrow (left coast time).

Ron

[Pelican9]

I meant just to get rid of the error messages and dump the PTS. I'll add the PTS header tomorrow (left coast time).

Ron

Great! Thanks again.

But I've thought about the header, and it would be better to use another two letters instead of 'SP', because the structure is quite different and it would be easier to program.

[dmz01]

If dmz01 can write a subtitle extractor then he must know the format already?
No.



Pelican9 is correct. I don't know the format. You do not need to know a specific format information just to extract it from a well known container.

[Rectal Prolapse]

Thanks drmpeg! :)

[Pelican9]

SUPread 0.4 is ready to read BD subtitles... :)
http://pel.hu/down/SUPread_BD.jpg
Don't try to download, it's not available yet. I'm waiting for drmpeg's new xport version with PTS info.

[drmpeg]

Here's the update:

http://www.w6rz.net/xportpgs.zip

Make sure you clear your browser cache to get the latest. Files are dated 5/6/2007 at 10:55 PM.

I chose to use "PG" (for Presentation Graphics) instead of "SP".

Ron

[Pelican9]

Here's the update:

http://www.w6rz.net/xportpgs.zip

Make sure you clear your browser cache to get the latest. Files are dated 5/6/2007 at 10:55 PM.

I chose to use "PG" (for Presentation Graphics) instead of "SP".

Ron

Thanks!

This is the structure of one subtitle as I thought:
-------------
'PG', PTS, 0,0,0,0, (start time)
0x16, ... presentation composition definition (0x13 bytes)
0x17, ... window definition (0x0A bytes)
0x14, ... palette definition (0x4F8 bytes)
0x15, ... object definition (0x3087 bytes)
0x80, 0, 0 end of displayset
'PG', PTS, 0,0,0,0, (end time)
0x16, ... presentation composition definition 0x0B
0x17, ... window definition 0x0A
0x80, 0, 0 end of displayset
-------------
Next subtitle


This is the structure of the xported .pgs:
-------------
0x16, ... presentation composition definition (0x13 bytes)
0x17, ... window definition (0x0A bytes)
'PG', PTS, 0,0,0,0, (0xFB9B81) 3:03.215
0x14, ... palette definition (0x4F8 bytes)
0x15, ... object definition (0x3087 bytes)
'PG', PTS, 0,0,0,0, (0xFB9C00) 3:03.216
0x80, 0, 0 end of displayset
0x16, ... presentation composition definition 0x0B
0x17, ... window definition 0x0A
'PG', PTS, 0,0,0,0, (0xFE4620) 3:05.156
0x80, 0, 0 end of displayset
-------------
Next subtitle

What is the extra (first or second) PTS?
Could you change something or I have to handle this structure?

[drmpeg]

I know what's wrong. Some segments have a PTS/DTS PES header, and I was only putting a header in the .pgs file on the segments with just a PTS.

Here's the update:

http://www.w6rz.net/xportpgs.zip

Files are dated 5/7/2007 at 2:48 am

For the segments with DTS, I've put the DTS after the PTS.

"PG", PTS (4 bytes), DTS (4 bytes)

If you think the DTS is useless, and you'd rather have 0x0, let me know.

Ron

[Pelican9]

I know what's wrong. Some segments have a PTS/DTS PES header, and I was only putting a header in the .pgs file on the segments with just a PTS.

Here's the update:

http://www.w6rz.net/xportpgs.zip

Files are dated 5/7/2007 at 2:48 am

For the segments with DTS, I've put the DTS after the PTS.

"PG", PTS (4 bytes), DTS (4 bytes)

If you think the DTS is useless, and you'd rather have 0x0, let me know.

Ron

Yesss! Much better! Thank you!
All segment have their own PTS/DTS info.

What do you think about the file extension?
I mean the .sup is well known as subtitle file.
But on the other hand .pgs shows that it is a different structure.

[manusse]

I vote for the ".sup" extension because there are already too many subtitle extensions and this will mean that it's a binary subtitle (a.k.a. image).

Cheers
Manusse

[Pelican9]

I know what's wrong. Some segments have a PTS/DTS PES header, and I was only putting a header in the .pgs file on the segments with just a PTS.

Ron!

Did you decrease the PTS values with the PTS of the first video frame?
I think you didn't, because the subtitles are delaying about 10 sec.
Could you check it?
And I vote the .sup extension too.
Thanks in advance.

Edit:
SUPread v0.4b1 (http://pel.hu/down/SUPread.exe)

[drmpeg]

Ron!

Did you decrease the PTS values with the PTS of the first video frame?
I think you didn't, because the subtitles are delaying about 10 sec.
Could you check it?
And I vote the .sup extension too.
Thanks in advance.

Edit:
SUPread v0.4b1 (http://pel.hu/down/SUPread.exe)
The PTS/DTS is not adjusted. It's directly from the bitstream. Here's an update where the initial video PTS is subtracted from all PGS PTS/DTS values. Also, filename changed to .sup as suggested.

http://www.w6rz.net/xportpgs.zip

File times are 5/8/2007 at 3:16 am.

Ron

[Pelican9]

The PTS/DTS is not adjusted. It's directly from the bitstream. Here's an update where the initial video PTS is subtracted from all PGS PTS/DTS values. Also, filename changed to .sup as suggested.

http://www.w6rz.net/xportsup.zip

File times are 5/8/2007 at 3:16 am.

Ron
THANKS!
(The working link: http://www.w6rz.net/xportpgs.zip)

Edit:
Something wrong with the PTS/DTS adjusting.
Original - Adjusted:
0xFBB289 - 0xFBB28A
0xFBB281 - 0xFBB282
0xFBB249 - 0xFBB24A
...
Like the first video PTS is -1 (or 0xFFFFFFFF)

I've expected something like this:
Original - Adjusted:
0xFBB289 - 0xEDF6E9
0xFBB281 - 0xEDF6E1
0xFBB249 - 0xEDF6A9
...

Edit 2:
It was my fault.
I've used this command:
xport -h 00001.m2ts 1 9 9 1
because I didn't want to demux the video and the audio stream but only the subtitle stream (not enough space on this drive).
Is there a way to skip the video stream demux but get the first video PTS and make a .sup?

[manusse]

Could you post an example of such an extracted SUP file so that I can try working on its import by SubtitleCreator?
(Or better a file from which to extract the sup plus the command line I must use).
A short sample will be enough.

Thanks
Manusse

[jokin]

I used SUPread on the xported sup and it seems after SUPread is done the SRT times are behind by 11 seconds

[Rectal Prolapse]

Pelican9, do the subtitle times look correct? On my tests I wasn't able to get PTS times that were in sync with the video - they were too early by a second (using TSPE 0.3 and stepping through the PTS values in the video and the PG stream).

I'm hoping the xport times are correct. :)

[Pelican9]

It looks like good (in sync with the video) with the adjusted PTS values.

[Pelican9]

I used SUPread on the xported sup and it seems after SUPread is done the SRT times are behind by 11 seconds

Did you demuxed the subtitle with the video stream?
Without the video the PTS adjusting is not work.

[Pelican9]

A short sample will be enough.


PM sent

[jokin]

Did you demuxed the subtitle with the video stream?
Without the video the PTS adjusting is not work.

Thanks for pointing that out. All is good now.

[Pelican9]

Edit 2:
It was my fault.
I've used this command:
xport -h 00001.m2ts 1 9 9 1
because I didn't want to demux the video and the audio stream but only the subtitle stream (not enough space on this drive).
Is there a way to skip the video stream demux but get the first video PTS and make a .sup?

Ron:
Could you check this?
The first PTS of video stream #1 could be good.

[drmpeg]

Ron:
Could you check this?
The first PTS of video stream #1 could be good.
I've added a new option (-j) to just demux the subtitle stream.

http://www.w6rz.net/xportpgs.zip

Files are timestamped 5/10/2007 3:52 AM.

xport -hj movie.m2ts 1 1 1 1

Ron

[Pelican9]

I've added a new option (-j) to just demux the subtitle stream.

http://www.w6rz.net/xportpgs.zip

Files are timestamped 5/10/2007 3:52 AM.

xport -hj movie.m2ts 1 1 1 1

Ron

It works like a charm.

Thanks.

[hhp]

Becaus there is more language subtitle in Blu-ray, is it possible to outport the subtitle in idx/sub format. Any way thanks again to your great program.

[Pelican9]

Becaus there is more language subtitle in Blu-ray, is it possible to outport the subtitle in idx/sub format. Any way thanks again to your great program.

You can demux all subtitle streams one by one...