Converting Bluray disc subtitles

[azad]

Here's another one, most of the subtitle lines end up with 00:00:00,000.

http://rapidshare.com/files/130135131/forced.sup.html


21
00:11:25,851 --> 00:00:00,000
Line 21

22
00:11:26,852 --> 00:00:00,000
Line 22

23
00:11:27,853 --> 00:00:00,000
Line 23

24
00:11:28,854 --> 00:11:29,980
Line 24

25
00:11:30,189 --> 00:00:00,000
Line 25

26
00:11:31,190 --> 00:00:00,000
Line 26

27
00:11:32,191 --> 00:11:33,359
Line 27

[SpaceAgeHero]

Taktaal,

I've got a problem with the german subtitles of Independence Day Blu-ray.
SupRip keeps telling me that:

************** Ausnahmetext **************
System.Exception: Eine Ausnahme vom Typ SupRip.SubtitleImage+EndOfImageReachedException wurde ausgelöst.

bei SupRip.SubtitleImage.ColumnContainsPixels(Byte[,] image, Int32 column, Int32 limit, Int32 y1, Int32 y2)
bei SupRip.SubtitleImage.FindLetters()
bei SupRip.SubtitleImage.CreateSubtitleArray()
bei SupRip.SubtitleFile.GetSubtitleImage(Int32 n)
bei SupRip.MainForm.LoadSubtitleImage(Int32 number)
bei SupRip.MainForm.LoadSubtitleImage(Int32 number)
bei SupRip.MainForm.LoadSubtitleFile(String fileName)
bei SupRip.MainForm.loadButton_Click(Object sender, EventArgs e)
bei System.Windows.Forms.Control.OnClick(EventArgs e)
bei System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
bei System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
bei System.Windows.Forms.Control.WndProc(Message& m)
bei System.Windows.Forms.ButtonBase.WndProc(Message& m)
bei System.Windows.Forms.Button.WndProc(Message& m)
bei System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
bei System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

Sup files have been uploaded to here if you'd like to investigate.

http://rapidshare.de/files/40125119/independence.day.german.rar.html

Furthermore, in which character code does SupRip save SRT files? I'm asking this because in the end umlaut characters aren't displayed correctly.
"Ä" is displayed as "├ä" - "Ü" is displayed as "├£" and so on.

Hoping for a new release soon. :D Thanks. :)

[Taktaal]

I fixed all of the above problems with the new version 1.00

Furthermore, in which character code does SupRip save SRT files? I'm asking this because in the end umlaut characters aren't displayed correctly.
"Ä" is displayed as "├ä" - "Ü" is displayed as "├£" and so on.

Suprip saves umlauts in Unicode. I tested them with both Mplayerc and VLC, and both those applications seem to properly support extended Unicode letters so I thought they'd be ok. With what mediaplayer did you try to play your movie so that you got the erroneous display?

[laserfan]

I fixed all of the above problems with the new version 1.00Thank you Taktaal for your very nice app!!!

:thanks:

[GZZ]

Is there any way to have a function to save the sup file at a lower resolution. Lets say you load a 1920x1080 subtitle file and if its possible then output it as 1280x720. Then I dont have to do bluray subs -> SRT -> Bluray Subs.

[SpaceAgeHero]

@ Taktaal,

first off - thanks for the new version. :)
SupRip recognizes the subtitles now.

Unicode with srt's seems to be problematic for me however. I just used your latest version to convert Independence Day German subtitles and muxed them into the mkv. Umlauts are still not displayed correct with the latest MPC (included in latest K-Lite Codec Pack). I just attached an image of what's supposed to read "NÖRDLICHE WÜSTE - IRAQ" and what it actually looks like in MPC.

Because of this problem I used to resave the srt files into ANSI code. Everything works fine with ANSI.

Many thanks again however for this nice app. :)

Edit: When I open a srt file saved by SupRip with a texteditor it shows me that it is in UTF-8 Format. :confused:

[Adub]

UTF-8 is fast becoming the standard, so that should not be an issue.

One issue is that you should never use codec packs. All you need is ffdshow and the latest build of Media Player Classic Homecinema (run a google search).

Uninstall that codec pack of yours, download and install the latest ffdshow, and download the latest version of Media Player Classic Homecinema. Set the output to VMR9 in the options menu, select the subtitles you want, and everything should play fine. This is of course assuming that your subtitle was ripped correctly.

[azad]

@Taktaal

Version 1.00 works great!
Thanks again for all the hard work you put into this tool.


I just tested the BD Forced Subtitle detection with another Region B disc, this time no Sony BD, and it worked perfectly! ;)

[Kurtnoise]

@Taktaal: here is (http://www.mediafire.com/?bjq4j9jkzuq) a HDSUP file which isn't supported by SupRip...Hoping you'll find the problem. :)

[boombastic]

Hell oto everyone.
First of all let me thank you for your work.
I'm going to backup my Pathfinder bluray disc (region 2,Italy).
There are a lot of subtitle.
Can i just extract the subtitle with "xport -hu movie.m2ts 1 1 1 1" and mux them into the mkv?How can i determine whic subtitle is the italian one and if is forced or not?
Thanks in advance!

[saint-francis]

First off, use eac3to to demux the subtitles. Second, you can just mux them into a .mkv but you will be hard pressed to play them since neither vobsub or MPC can decode them. So you need to convert them to some kind of format that you can play.

[boombastic]

First off, use eac3to to demux the subtitles. Second, you can just mux them into a .mkv but you will be hard pressed to play them since neither vobsub or MPC can decode them. So you need to convert them to some kind of format that you can play.

This is the log of eac3to for the title containing the main movie:

"C:\Programmi\eac3to\eac3to.exe" "D:\Pathfinde
r\PATHFINDER_F1" 1)
M2TS, 1 video track, 6 audio tracks, 1:47:11
1: Chapters, 25 chapters
2: MPEG2, 1080p24 /1.001 (16:9)
3: DTS Master Audio, English, 5.1 channels, 24 bits, 48khz
4: DTS, Italian, 5.1 channels, 24 bits, 768kbit/s, 48khz
5: DTS, German, 5.1 channels, 24 bits, 768kbit/s, 48khz
6: AC3, Polish, 2.0 channels, 224kbit/s, 48khz, dialnorm: -27dB
7: AC3, English, 2.0 channels, 224kbit/s, 48khz, dialnorm: -27dB
8: AC3, German, 2.0 channels, 224kbit/s, 48khz, dialnorm: -27dB


which stream should i extract for the subtitle?To which format should i convert them to play in MPC?

[azad]

@boombastic

There doesn't seem to be a subtitle stream on this BD. But actually I can't believe this. So what eact3o version do you use? Try the latest version, this should recognize all subtitle streams.

All BD subtitles are shown like this in eact3to: 0...99: Subtitle (PGS), language

[boombastic]

@boombastic

There doesn't seem to be a subtitle stream on this BD. But actually I can't believe this. So what eact3o version do you use? Try the latest version, this should recognize all subtitle streams.

All BD subtitles are shown like this in eact3to: 0...99: Subtitle (PGS), language

I re-download and used the last version of eac3to but i obtain the same output....it's very strange.Maybe there's a switch or a particolar option to use?

I succesfully extract a .sup file with the last version of xport

xport -hu movie.m2ts 1 1 1 1

but if i open it with suprip there's only the english subtitle....

[SpaceAgeHero]

I re-download and used the last version of eac3to but i obtain the same output....it's very strange.Maybe there's a switch or a particolar option to use?

I succesfully extract a .sup file with the last version of xport

xport -hu movie.m2ts 1 1 1 1

but if i open it with suprip there's only the english subtitle....

Hhm, you better report that problem to madshi in the official eac3to thread then.

[azad]

@boombastic
Try using tsmuxer to check the appropriate m2ts file for subtitle streams.

I succesfully extract a .sup file with the last version of xport

xport -hu movie.m2ts 1 1 1 1

but if i open it with suprip there's only the english subtitle....
You have to choose the right subtitle stream number.

xport -hu movie.m2ts 1 1 1 1

This command line extracts the 1st subtitle of the m2ts file. To extract the italian subs you have to know the right stream number. Like mentioned before, try using tsmuxer or powerdvd (if eac3to doesn't work) to find out the right stream number.

[boombastic]

Finally i found that tsmuxer can show me the correct lan guage for each subtitle and save it to a .sup file. Then i load it with Suprip to ocr it and merge it to audio and video with mkvmerge.
The only thing that doesn't work is the distinction of fored sutitle:TsMuxer cannot tell me which one is a forced one;ifi load into suprip the smaller sized one (which is truly the forced one) and flag the option "Forced subtitle only" no line is processed.
Anyway finally all went ok!Thanks for your help!!!

[azad]

The only thing that doesn't work is the distinction of fored sutitle:TsMuxer cannot tell me which one is a forced one;ifi load into suprip the smaller sized one (which is truly the forced one) and flag the option "Forced subtitle only" no line is processed.

In most cases on BD "forced subtitles" are only a separate subtitle stream and not real forced subtitles as you know it from dvd. Only real forced subtitles are detected by suprip's "Forced Subtitle" function.

[Adub]

There's a new version of Suprip available. It fixes a bug when displaying Unicode characters.

[BZeeme]

Could someone give me a brief outline of how to use SUPread to convert .sup to .srt? I've been using SupRip but lately had to do a lot of editing of the resultant .srt in Word (it has problems with , and ' and also leaves some unicode in the output .srt).

BZee

[nautilus7]

Use SupRip instead. It's better.

[dEATHfLOWEr]

I read the whole threat and can't seem to find de definitive answer. Is there a way to export a BD/HDDVD .sup to BMP's with a .srt file reffering to the BMP's? Because I find the .sup> OCR > .srt checking in Notepad (because all the L's are I and C's are O's etc) > muxing back to bitmaps with MNToolnix to make a MKV with embedded subs quite a hassle.

I actually want to keep the subtitle resolution because I just want to make a MKV file from a BD or HDDVD disc while maintaining the 1920x1080 video resolution.

[XhmikosR]

You can just export the original .sup file with eac3to or with any tool you like, and then play the mkv with MPC-HC which now supports playback of .sup subtitles. It needs some fixes though, but you can save a lot of time.

[nautilus7]

XhmikosR are you sure about this? Can mpc-hc play matroska with external sup subtitles? I didn't manage to do it.

[XhmikosR]

Not external .sup files. Just mux the subs with the video and then you can play the mkv with the .sup in it.

[nautilus7]

I can't mux .sup with mkvmerge. It won't accept sup format.

[XhmikosR]

Oh, yes my fault. I thought I muxed it first to mkv but no. Only if you play the m2ts allows you to watch the .sup with MPC-HC.

[rica]

I read the whole threat and can't seem to find de definitive answer. Is there a way to export a BD/HDDVD .sup to BMP's with a .srt file reffering to the BMP's? Because I find the .sup> OCR > .srt checking in Notepad (because all the L's are I and C's are O's etc) > muxing back to bitmaps with MNToolnix to make a MKV with embedded subs quite a hassle.

I actually want to keep the subtitle resolution because I just want to make a MKV file from a BD or HDDVD disc while maintaining the 1920x1080 video resolution.

The best way i think:

Extract sup with eac3to.

Convert to srt via SupRip's latest version. (thanks XhmikosR btw) _you have to teach the program at the beginning but don't worry it seems difficult but it is much easier than you think.

Remux to mkv with mkvmergegui or

Remux just video and audio and make an m2ts or ts with TSMuxer and put it into the same folder where srt is located and rename *.ts/m2ts and *.srt with the same name. Like test.ts and test.srt
(You can not mux srt by TSMuxer since it converts srt back to PGS.)

Watch them on MPC-HC with or without DXVA mode... (for Vista EVR Custom Presenter, for XP VMR9 (renderless) has to be selected.)


_ _ _ _ _

[dEATHfLOWEr]

The best way i think:

Extract sup with eac3to.

Convert to srt via SupRip's latest version. (thanks XhmikosR btw) _you have to teach the program at the beginning but don't worry it seems difficult but it is much easier than you think.
_ _ _ _ _

Well that is the problem. I want to leave out OCRing, as it takes to much time correcting all the bad guesses (with the change of leaving some errors in, which are unnoticed till you play the movie).

With DVD's it is possible with Subrip (not to be confused with SupRip) to save all the bitmaps together with an SRT file which pointed to these bitmaps. That way you could skip the OCR part and retain the original subs and quality.

[rica]

I'm afraid that is the only toy we have in our hands for the time being :)

[BZeeme]

Use SupRip instead. It's better.

That's what I have been using. I got it working better now. I had to drop the similarity tolerance from the default 50 to 15. Now I have to do more entry of unrecognized characters but the resultant .srt is much better. I then use find and replace in MS Word to make corrections.

[Taktaal]

The recent versions are much better than the earlier ones at distinguishing between I and l. However, in fonts where the two are completely equal this doesn't help much of course. I've been working on a system that integrates some dictionary work for deciding between those two letters, but that code doesn't work really well yet, so I haven't put it into the release versions. In retrospect, it would probably be easier to just build a time machine and kill the dumbass who had the idea of making two letters look exactly the same.

[azad]

In retrospect, it would probably be easier to just build a time machine and kill the dumbass who had the idea of making two letters look exactly the same.
:D Great! Thanks for this one! And I agree of course! :)

[Zwitterion]

The best method I found was to use ABBYY's FineReader 8.0 program. Tell Supread to output BMPs, then use ABBYY's FineReader for OCR.
I use this method, works like a charm. However, I found one subtitle stream which doesn't work with SUPread. It opens, but displays just black screens instead of subtitles. The stream is from the American version of Sunshine (SUP sample (http://www.mediafire.com/?3qxxeslght1)).

Suprip displays the subtitles, but unfortunately can't save them as bitmaps (yet?). Having that feature would be great.

Could anyone try with an older version of SUPread whether it can read the sample? I can't find them on the net...

[Jaja1]

Could anyone try with an older version of SUPread whether it can read the sample?
None of the versions of supread I have were able to read the sample. I do encounters lots of sup files that can't be read by supread, while suprip is able to read them. So it would be nice if suprip could save subtitles to bitmaps or another image format.

Originally Posted by Rectal Prolapse
The best method I found was to use ABBYY's FineReader 8.0 program. Tell Supread to output BMPs, then use ABBYY's FineReader for OCR.How do you convert then the txt output of Finereader to srt or whatever subtitle format? The output of Finereader is just a flat txtfile.

[B4tm4n]

How do you convert then the txt output of Finereader to srt or whatever subtitle format? The output of Finereader is just a flat txtfile.

AFAIK srt is just a text file with the srt extention.

[Zwitterion]

How do you convert then the txt output of Finereader to srt or whatever subtitle format? The output of Finereader is just a flat txtfile.

I use Rectal Prolapses tool SRTmaker to merge an SRT with Finereaders HTML output.
http://forum.doom9.org/showpost.php?p=1050444&postcount=227
http://forum.doom9.org/showpost.php?p=1050444&postcount=228

I haven't needed the Photoshop step until now. The whole procedure usually works like a charm. I have to open the resulting SRT with a Word processor and then save it again with "insert line breaks" checked, because otherwise I'd have a problem with disappearing line breaks in Notepad.

I really hope Suprip will support saving subtitles as images. :)

[Jaja1]

Awesome!

One not so minor detail though :)
I don't know s*t about Visual Studio. Is SRTMaker a tool that needs to be compiled first, since I don't see an executable?

[Zwitterion]

The executable was here:
http://forum.doom9.org/showpost.php?p=1056641&postcount=254

[asarian]

Well that is the problem. I want to leave out OCRing, as it takes to much time correcting all the bad guesses (with the change of leaving some errors in, which are unnoticed till you play the movie).

With DVD's it is possible with Subrip (not to be confused with SupRip) to save all the bitmaps together with an SRT file which pointed to these bitmaps. That way you could skip the OCR part and retain the original subs and quality.
I wonder if someone ever came to the idea of using the .bmp's from a .sup file and kinda use them as a transparent layer to superimpose over an existing stream? This OCR'ing seems hopelessly the wrong way.

[Jaja1]

Thanks Zwitterion for the help. And lots of thanks to Rectal Prolapse for creating this tool.

[cavediver]

I extracted the 1st English PGS stream from Apocalypto BD AnyDVD HD rip with tsMuxer then loaded the .sup file into Suprip (v. 1.01), and pressed Auto-OCR. Suprip didn't correctly ocr the file. The SRT tab indicates under "statistics" that all 448 subtitles contain errors and 0 were correctly OCR'd. Any suggestions for how I might get this to work?

[azad]

@cavediver
Don't use the Auto-OCR option, because Suprip won't ask for letters it can't recognize correctly, instead it just chooses a letter which it thinks might be the best choice. Thats the cause for all the errors you get.

Use the OCR option + "Automatically continue ...", then Suprip will ask you for every letter which can't be recognized and it will start to "learn" recognizing the letters you manually entered.

[cavediver]

@cavediver
Don't use the Auto-OCR option, because Suprip won't ask for letters it can't recognize correctly, instead it just chooses a letter which it thinks might be the best choice. Thats the cause for all the errors you get.

Use the OCR option + "Automatically continue ...", then Suprip will ask you for every letter which can't be recognized and it will start to "learn" recognizing the letters you manually entered.

Thanks. That worked.

[revy]

I thought 100mb was a safe bet for maximum reasonable file size. I upped it to 1gb now.

Also, that movie you linked up there seems to have actually marked its forced subtitles, so I got it working in the new version.



It's working better now, but still not perfectly. As far as I understand, that movie has in some places two subtitles displayed at the same time (maybe one on top, the other on the bottom?) Is this correct?

I think this should happen from 0:00:09 to 0:00:17 and 0:02:18 to 0:02:22 if I understand the display codes correctly.

I'm currently ripping this and yes, on the top it says the name and job of the person who is talking, on the bottom is the talk.

In the last version SupRip misses every talk when one of those top frames appears. And in some cases the top frames aren't correctly ripped. ( only half of the text appears )

And the program doesn't detect italic phrases it seems.

[GZZ]

The subpicture stream consists of data inside sections. Each section begins with an identifier byte, followed by a 16 bit integer that contains the length of the data, followed by the data itself.

Identifiers, order of presentation found in typical subtitle stream:

16 00 13 07 80 04 38 40 00 01 00 00 00 01 00 00 00 00 00 00 00 00

16 = identifier
00 13 = size of section
07 80 04 38 = 1920x1080.
01 = do not clear. If set to 00 instead it will clear the subpicture!

40 00 01 00 = sequence number??

00 = if set to 40 instead of 00 the next subtitle will be forced on.



Did anyone find out what the last 15 byte (marked with bold) is, I'm trying to stick together som documentation and I cant figure these bytes out ??

[Taktaal]

16 00 13 07 80 04 38 40 00 01 00 00 00 01 00 00 00 00 00 00 00 00

16 = identifier of a metainformation section
00 13 = length of the data in this section
07 80 = width
04 38 = height
40 00 01 00 00 00 = unknown
01 = number of subtitles in this frame
00 00 00 00 00 00 00 00 = metadata about one subtitle

Most sections of type 16 are 19 bytes long, those are the ones where only one subtitle is on the screen at any one time. Some of them are longer though, 27 bytes if there are two subtitles at once. Maybe it's also possible to have three on screen, but I haven't seen this yet in a movie. The "number of subtitles in this frame" value also goes up if there are more subtitles than one.

What bytes 1 to 3 do in the metadata, I don't know
The fourth byte is the most interesting one. If the subtitle is forced, the bit 64 (on Intel processors) is set.
The last four bytes are used only by some subtitles to place the text on the screen. It just defines the x and y coordinate of the top left corner of where the bitmap should be placed. Most Bluray movies however store their subtitles as full 1920x1080 bitmaps and just lay that over the movie, using 100% transparent pixels on most of the image.

[happyprozak]

Hi Taktaal,

I tried using Suprip 1.01 for Pirates of the Caribbean 1 and 2 on blu-ray and Suprip displays the text but it has a problem putting boxes around the letters. I played with the settings for width etc. and I couldn't get suprip to put boxes around the letters properly. Is this a known issue? Or maybe theres something in these subtitles that is throwing off your program?

I'd be glad to send you the subtitles. I don't know where I could upload them but I'd be glad to email them to you.

[Taktaal]

I know that the current letter boxing algorithm isn't very good. I'm currently working on a completely new one.

[GZZ]

Thanks TakTaal for the information. Still a bit confussed on the Metadata part, gonna look into that.

But I have a little gap left in my code, that I dont know what is (I have marked it with red). If you know, please explain it.

PG FOUND
PTS StartTime: 00:00:36,119
DTS StartTime: 00:00:36,055

PCS: (0000000A)
Identifier: 22 (0x16)
EndOfSection: 32
Resolution: 1920 x 1080
Subtitle in this frame: 1
Metadata: 00 00 00 00 00 00 03 8C

PG FOUND
PTS StartTime2?: 00:00:36,118
DTS StartTime2?: 00:00:36,055

WDS0: (0000002A)
Identifier: 23 (0x17)
EndOfSection: 65
Resolution: 1920 x 117
Position: 0 x 908
Unknown ?: 50 47 00 31 83 76 00 00 00 00

PDS: (00000041)
Identifier: 20 (0x14)
Palettes Found: 22
EndOfSection: 190
Unknown ?: 50 47 00 31 84 33 00 31 83 76

ODS0: (000000BE)
Identifier: 21 (0x15)
EndOfSection: 27191
DataArray Length: 696C

END: (00006A37)

PG FOUND
PTS EndTime: 00:00:40,415
DTS EndTime: 00:00:40,414

PCS: (00006A44)
Identifier: 22 (0x16)
EndOfSection: 27228
Resolution: 1920 x 1080

WDS0: (00006A5C)
Identifier: 23 (0x17)
EndOfSection: 27251
Resolution: 1920 x 117

END: (00006A73)


------------------------------
END OF SUBTITLE 1
------------------------------

And another note, anyone of you know if its possible to convert PTS time into DTS time ?