BDSup2Sub - convert and tweak bitmap subtitle streams (VobSub,BD-SUP,BDN XML,HD-SUP)

[Palmnut]

I'm trying to use the tool to convert .sup subtitles written by ProjectX from a UK DVB-T transport stream (25 fps). The program seems to work well for the vast majority of the subs, except where it detects that the end time for a subtitle overruns the start time for the following subtitle.

As a specific example:
Frame 41 starts at 00:03:42.470 and ends at 00:03:46.884.
Frame 42 starts at 00:03:46.880 and ends at 00:03:51.101.
Frame 43 starts at 00:03:51.100 and ends at 00:03:55.503.
Frame 44 starts at 00:03:55.500 and ends at 00:03:57.388.

Now in this case I would expect the tool to simply shift the end-time of 42 to 0.001s earlier (or 43 later). What I actually get is:
Frame 41 ok, with slight tweaks after the decimal points.
The start time of Frame 42 changes to 00:03:51.080 (near-enough the original start time of Frame 43), the end time of 42 changes to 00:03:51.120 (i.e. a duration of one video frame).
The start time of Frame 43 changes to 00:03:55.480 (near-enough the original start time of Frame 44), the end time of 43 changes to 00:03:55.520 (i.e. a duration of one video frame).

WTF? This means that I don't get to see Frame 42 or 43 at all - in fact this effect seems to ripple down the entire sequence until we get a long break in the titles.

Methinks you might have two variable muddled up when you try to solve this case, as it seems that the start time of the subsequent frame is being written in as the start time of the current frame.

Peter

[0xdeadbeef]

The root cause of such a problem is usually not really visible by just looking at the times. This is probably a combination of several issues, e.g. missing end times and missing start times. So as always, a sample is mandatory to have a look.

Anyway, keep in mind that BDSup2Sub's main purpose is not to fix broken subtitles. For a correct conversion it needs to have consistent timing, but this is not necessarily the same as correct timing. Therefore it outputs a warning for any fix that is done and leaves it to you to do a manual check/correction.

[DivxBr]

Hi, I was trying to find a simple and quick tool to convert '.sup' subtitles (DVD extracted with pgcdemux) to '.sub/.idx' to mux into a mkv container. But everytime I try the bdsup2sub to load the sup file, I get an error message like this: 'Offset 1091309618 out of bounds for file 'xxxxxxxx.sup'.
Maybe its a pgcdemux problem, or I'm doing something wrong?

[0xdeadbeef]

If I may quote the integrated help on this:

Notes on SUP/IFO
The SUP/IFO format as defined by IfoEdit/VobEdit is supported since version 3.9.0 due to public demand. Internally (e.g. regarding RLE encoding, palette definitions etc.) it's virtually the same as SUB/IDX. Note that there are some limitations to this format though. Firstly, the palette and the screen size are stored inside an IFO file. BDSup2Sub needs a valid IFO file for each SUP file and both have to share the same file name (except the extension of course). If you e.g. exported a file "english.sup" from a "VTS_02_1.VOB" and the according IFO is "VTS_02_0.IFO", then you need to copy the IFO and rename it to "english.ifo" to make BDSup2Sub accept it as input. Note that BDSup2Sub recognizes the extension "IFO" for SUP/IFO, but not "SUP". The reason is that BDSup2Sub can differ between BD-SUP and HD-DVD-SUP during import only because both format have different packet identifiers. Unfortunately, HD-DVD-SUP and DVD-SUP use the same (outer) package structure so they can only be distinguished between during parsing of the command buffer. At this time, BDSup2Sub already decided to treat a SUP with that package header as "HD-DVD-SUP" and will throw an error. When exporting, you should be aware that there are only PAL and NTSC resolutions defined in the DVD format (and therefore in the IFO).Thus hires or cropped formats will lead to major problems, even when importing these files back in BDSup2Sub. I'd strongly suggest to use VobSub in such cases.


If you don't have an IFO, you can use more or less any IFO as dummy replacement. Note though that the palette is loaded from the IFO.

[makmalimak]

I cannot see Turkish characters. What should I do?

[0xdeadbeef]

I'm undecided if this is a very strange posting by a human or an actually nearly brilliant posting by a spambot.
Anyway, I'm left puzzled.

[DivxBr]

If I may quote the integrated help on this:



If you don't have an IFO, you can use more or less any IFO as dummy replacement. Note though that the palette is loaded from the IFO.

Thanks very much!

One more question: the ifo file has only one palette or there's one for each pgc or one for each subtitle? IF it has more than one, which one bdsup2sub uses?

[0xdeadbeef]

One more question: the ifo file has only one palette or there's one for each pgc or one for each subtitle? IF it has more than one, which one bdsup2sub uses?

Well, this was a quick hack, but as far as I remember, there's one palette per PGC and BDSup2Sub uses the palette of the very first PGC. The dummy IFOs it creates are more or less empty apart from this one palette and some video attributes for PAL/NTSC.
As a quick way to get an IFO, you can load an SUB/IDX, edit the palette as you wish and store it as SUP/IFO. Then you can use this IFO as base for further palette edits.

[MadMonkey57]

Hi,

I am converting idx/sub to BD sup files:

- via GUI:
--> checking "Convert resolution to 1280x720"
--> change framerate from 25fps to 23.976
--> choosing "Mitchell" filter

...and it works fine.


via CLI: /res:720 /fps:25p:24p /filter:mitchell

OPTION: set resolution to 720p (1280x720)
OPTION: convert framerate from 25fps to 23.976fps
OPTION: set scaling filter to: Mitchell

Converting SUP(BD)

Loading FR_FORCED.idx
# 1
# 2
# 3
...
# 11

Detected 0 forced captions.

Writing FR_FORCED_CMD.sup
Decoding frame 1/11 at offset 0x00000000
Decoding frame 2/11 at offset 0x00001000
...
Decoding frame 11/11 at offset 0x00009800


... and fps seem to be NOT OK, still 25fps

Am I missing anything ?

Thanks

[0xdeadbeef]

via CLI: /res:720 /fps:25p:24p /filter:mitchell
... and fps seem to be NOT OK, still 25fps

Am I missing anything ?

I fear you got me on this one. Indeed the source fps value given via command line is obviously overwritten later when the input stream is read. I guess I already found the problem and a solution, yet I want to fix this only after a little more investigation.
So probably no fix before the weekend.

[MadMonkey57]

I fear you got me on this one. Indeed the source fps value given via command line is obviously overwritten later when the input stream is read. I guess I already found the problem and a solution, yet I want to fix this only after a little more investigation.
So probably no fix before the weekend.

Looking forward to testing it.

:thanks:

[0xdeadbeef]

15.11.2009: 3.9.8 -> 3.9.9

Changed: added "auto" option for CLI switch "/fps:<x>,<y>", e.g. "/fps:auto,24p"
Changed: added "/forceall" CLI switch to set/clear all forced flags
Changed: reworked conversion dialog to allow setting/clearing all forced flags via GUI
Changed: added an application icon to get rid of that coffee cup (note: ICO and ICNS files are included in the JAR)
Fixed: source FPS given via CLI was overwritten by FPS read from stream
Fixed: source FPS detection for SUB/IDX and SUP/IFO was buggy and (nearly) always resulted in 24p

[MadMonkey57]

Thanks for this update. Especially...

15.11.2009: 3.9.8 -> 3.9.9

Fixed: source FPS given via CLI was overwritten by FPS read from stream


..is working fine now.

:thanks::thanks::thanks:

[avivahl]

1st run (no ini file in dir), Settings->Conversion Settings->Nothing happens. Do I need to open a subtitle first? Bug?

[0xdeadbeef]

Hm, the dialog should open without a stream loaded. If it doesn't show without an ini file, this might indeed be caused by a missing INI file. Did you try that from the command line to see exceptions thrown? I'll have a look at it this evening.

Then again it's discussible whether this behavior makes sense or not. After all there's nothing to convert yet and after loading a stream, the conversion dialog will be shown anyway. Indeed I already thought about disabling the dialog before a stream is loaded.

So if this is a bug, it's pretty unimportant one and I'll only fix it with the next release but won't create a release just for this.

[EDIT]
OK, definitely a reference to uninitialized object. However since it happens in the (initialization of the) dialog, the exception is lost somewhere in the Swing jungle. As a side note, the existence of the ini file is not the factor that avoids the access to the object, but the setting of the "convertResolution" entry in the ini. If the entry is true, then the object is not accessed and the dialog is shown. If it is false (which is also the default setting when there's no ini or the entry is missing in the ini), the object is accessed, the exception is thrown and the dialog is not shown.
Anyway, it's already fixed locally, so it will be corrected in 4.0.0 but when this version is released is another question.

[avivahl]

Alright then. Thx.
Any planned "big features" for 4.0? :)

[0xdeadbeef]

Not at all. My "not so well thought out" version numbering now forces me to call the next version 4.0.0 for whatever reason I need to update.
However, currently there are only two minor bugs to fix and to call that 4.0.0 is a little too pathetic even by my standards.

[Sharc]

Problem with subtitles positioning (using the GUI):
- Extracting subtitles from DVD
- Converting to 720p SUP (BD)
=> Works ok so far.
However when I continue
- Move all captions => 'move inside bounds' => Move all
- Save/Export
the subsequent subtitles are all moved to the top left corner of the active picture instead of center bottom.
Do I miss something?

[0xdeadbeef]

Hm, can't seem to follow you here. Tried loading a PAL SUB/IDX, scaled it up to 720p, moved inside bounds, saved it -> Everything looks fine. Now it gets a little vague ... what does "subsequent" mean exactly here? I tried to "move all" repeatedly, I loaded the same SUB/IDX or PAL SUP/IFO, then moved it and so on. Whatever I tried, everything looked ok.

So, sorry, can't reproduce this. Either this is an issue with your subtitle (then I need it) or the instructions to reproduce the issue are not detailed enough (then I need more detailed instructions) or I'm just too stupid to understand this problem (then I might need some sleep).

At a 2nd thought, I could imagine a scenario were the extracted subtitles are all full screen either due to bad authoring or a bad extraction tool. Maybe cropping, scaling and moving at the same time is a scenario I haven't really tested. I'd need the subs then.

[0xdeadbeef]

Update: in the meantime I found a SUB/IDX with too large bitmaps that are cropped to the visible part by BDSup2Sub. With this stream I could reproduce the problem. Indeed there are two issues:

1) When applying the "move all" command to a stream with subtitles that are cropped due to oversized images, all subtitles are misplaced that were not (decoded and) shown before.

2) The "move all" setting partly survives loading another subtitle (indeed it should, but not this way). So when loading such a problematic stream with oversized images after "move all" was used, all cropped subtitles but the very first (which is automatically decoded and displayed) are misplaced without even using "move all" again.

Though I haven't actually found the bug (or even started looking for it), I'm pretty sure where to look. Unfortunately this is the most unpleasant part of the whole code, so fixing it will be probably slightly annoying. Then again, this will be fixed in the 4.0.0 release of course. Dunno if I'll manage to do it this weekend though, but I'll try.

Until then I can provide a temporary workaround: load the problematic stream into a freshly started instance of BDSup2Sub, then save it and load the saved stream. Moving should work ok now. Note that cropping itself works fine and when you save the stream before applying the move command, the cropped subtitles are saved with correct positions. Now if you load this fixed stream, the problem doesn't occur any more.

[Sharc]

Thanks for your reply and efforts to reproducing the case. I was probably a bit vague with my description because I assumed the problem would be easy to reproduce - which is apparently not the case.
The DVD subs are extracted with VSrip.
By 'subsequent' I mean:
The current subtitle which is displayed in the GUI is correctly moved to within the bounds. So it looks as as expected. When I however step through the next subtitle(s), they are moved to the top left corner.
When I Save / Export the 720p sups and reload the saved file again in BDSub2Sup, they are all top left aligned.
I am not sure if it happens in all cases, but I had this at least with 2 or 3 discs. I try to find a sample. It's really a minor issue though of a very nice tool.

Added: ... while I typed.... you were faster. Thanks

Added 2: The workaround helps. No rush to fix the issue.

[0xdeadbeef]

14.12.2009: 3.9.9 -> 4.0.0

Changed: BDSup2Sub is now open source (see integrated HTML help for details about repository and license)
Changed: reworked built-in HTML help.
Changed: added "recent files" menu.
Changed: output format is now persistent. Changed order of output formats in listbox.
Changed: added option "apply move settings" to conversion dialog that let's you choose whether to apply the existing "move all" settings or not when a new stream is loaded.
Changed: "move all" decodes/scales each frame before moving. Added a progress bar since this takes a little longer now.
Changed: added tooptip for the links in the HTML help. Also anchors inside the document are now supported.
Changed: added chapter entries in context menu of built-in HTML help.
Changed: built-in HTML help can now be opened in external browser (via context menu). "copy" only enabled if text selected.
Changed: the cancel button in the conversion dialog now cancels loading a stream.
Fixed: moving subtitles that are cropped during import could result in erratic subtitle positions or inappropriate warnings.
Fixed: conversion settings dialog didn't open when no subtitle was loaded and INI file was missing or the item convertResolution in the INI was either missing or set to false.
Fixed: erasing the extension in the file browser of the export dialog could lead to erratic behavior if the path contained dots. Generally, path separators were ignored when stripping the extension from file names.

[Jaja1]

Thanks a lot, Oxdeadbeef. I use BDSup2Sub very often and it works great.

[Sharc]

Fixed: moving subtitles that are cropped during import could result in erratic subtitle positions or inappropriate warnings.
:thanks:

[hubblec4]

You are the best one.

Thanks alot

[avivahl]

Time to thank you again for all the wonderful work. You've created a wonderful tool.

[0xdeadbeef]

Four thanks in a row without a single "but". Feels like Xmas ;)

[Kratus]

:thanks:And I want to thank too.
(without a but!)

[Paddy97]

This is a life saver. Recording with DVB-S subtitles I demux them and then run them through your gem of a tool to create "proper" SUP files that Suprip can read to OCR.

This is the first and only tool I've been able to correctly read and export DVB-S subtitles.

Thanks! Great job!

[radius]

Hey,

just updated from 3.9.6, thanks for adding the set / clear forced flag :)

Keep up the good work :)

[0xdeadbeef]

You're welcome!

Anyway, I'll be away for a few days and will have only limited internet access.

So Merry Xmas to all of you!

[24673583]

can`t open this web page in this time..
please check url !
thanks!

[avivahl]

can`t open this web page in this time..
please check url !
thanks!Both the javaforge project page and the download link are working fine here.

[medp7060]

I tested Current version: 4.0.0 (14.12.2009):

1. When open a SUP file ("test.sup"), it failed. If via "test.sup.IFO", it aksed "test.sup.SUP"). ProjectX created these two files: "test.sup.IFO" and "test.sup", , i.e. I have to rename "test.sup" as "test.sup.SUP". Is this a bug, or I did sth wrong?

2. Is there a way to use a single clour scheme for all the subtitles? In this case 3 subtitles are not readable at all, no matter how I adjusted the thresholds.

Samples included:

http://rapidshare.com/files/330992836/test.sup.rar

[0xdeadbeef]

1. When open a SUP file ("test.sup"), it failed. If via "test.sup.IFO", it aksed "test.sup.SUP"). ProjectX created these two files: "test.sup.IFO" and "test.sup", , i.e. I have to rename "test.sup" as "test.sup.SUP". Is this a bug, or I did sth wrong?
Of course both files have to use the same name. Only the extension must differ (e.g. "test.sup" and "test.ifo").
The reason why you need to load the IFO is described in the help file:
http://bdsup2sub.javaforge.com/help.htm#Notes_on_SUPIFO


2. Is there a way to use a single clour scheme for all the subtitles? In this case 3 subtitles are not readable at all, no matter how I adjusted the thresholds.
You can keep the original colors by using the palette mode "keep existing" as described here:
http://bdsup2sub.javaforge.com/help.htm#Palette_Modes

Also, you can use the same frame palette for all frames as described here:
http://bdsup2sub.javaforge.com/help.htm#Editing_the_Frame_Palette_and_Alpha

[medp7060]

Thanks for your reply.

But how can I make lines 21 and 22 in the example visible? I tried many combinations setting, still failed.

[0xdeadbeef]

While think I already answered (http://bdsup2sub.javaforge.com/help.htm#Editing_the_Frame_Palette_and_Alpha)this:

The quick and dirty solution would be to select "Edit DVD frame palette" for the first line, then click "Set All". While this works here, this is not guaranteed to work for any other sample. Keep in mind that the same color might be represented by a different color index in a different frame.

The other option would be to edit only the DVD frame palette for line 21 and 22. E.g. set all 4 colors to different colors and all alphas to 15. Then you can easily identify which color/alpha index represents which pixels and change it to a more fitting setting.

[REDtoLINE]

Does anyone have the Blu-ray PGS (.sup) english subtitles for Iron man? Either the entire english subtitles or forced only would be fine. If you have it please let me know. Thanks.

[y168]

Hi,

When I used multiAVCHD to convert VIDEO_TS folder to AVCHD folder, the subtitles are often not converted. I have tracked down the problem which is related to BDSup2Sub. I am hoping that you can help me determine what's wrong.

In multiAVCHD, the subtitles of the VIDEO_TS folder are extracted using mencoder. The syntax is something like:

mencoder.exe dvd://1 -dvd-device "Z:\DVD\VIDEO_TS" -vobsubout "dvddump002" -sid 0 -nosound -o "dvddump.tmp" -ovc frameno

The error often occurs when BDSup2Sub loads the extracted subtitles. The error messages I got when I loaded the subtitles manually are:

===================================================
WARNING: Control buffer size inconsistent.
WARNING: RLE buffer size inconsistent.
WARNING: Invalid end sequence offset -> no end time
WARNING: Invisible caption due to zero alpha (not fixed due to user setting).
# 674
ERROR: Invalid control buffer size
===================================================

Do you know how to fix this problem? I have uploaded some subtitles here that you can use to debug.

http://www.mediafire.com/?yggdecmuimj

Thanks for you help.

[0xdeadbeef]

Hard to tell for me if the problems in this stream are caused by an extraction tool or by the original stream. Anyway, the posted stream is defective and the error message given by BDSup2Sub is correct.

There are two captions with completely invalid packet size and wrong control offset. I fixed both manually to see if there are more problems.

Unfortunately, there are. Indeed exactly 10 subtitles are also damaged, but I didn't have the motivation to investigate any further. By commenting out these 10 lines in the IDX, the SUB/IDX becomes fully readable (well, the 10 lines are missing of course):
http://www.sendspace.com/file/eaw8jy

[sneaker_ger]

Are there any plans on supporting split subtitles produced by avs2bdnxml in an attempt to prevent the buffer from overflowing? I tried a conversion and tested the result with the suptitle plugin but it seems that BDSup2Sub only outputs one bitmap per frame.

[0xdeadbeef]

I still think that this so called "buffer overflow" is a misconception based on an implementation bug in the PS3's PGS decoder.
The only real constraint is bandwidth and splitting images reduces the bandwidth only slightly due to the run length encoding used to compress the bitmap. You could probably even construct cases where the overhead due to the 2nd object increases the bandwidth.
So in a nutshell: it is very unlikely that I will add such a workaround for bugs in some players.

[sneaker_ger]

Didn't know the subtitles were compressed, but it doesn't really matter if the PS3 is bugged or not since the authoring softwares already throw error messages because of the excessive bandwidth usage exceeding the bitrate limits of Blu-Ray.
And it's not that you have to create a workaround - this is already implemented in avs2bdnxml - it's just that BDSup2Sub doesn't read the files correctly.
But I have to admit that I don't have any knowledge about how the buffer is calculated so I don't know if ps auxw's attempts to reduce overflows by splitting and cropping subtitles (for example a line shown on the top and a line on the buttom will produce one very large bitmap while with the splitting the 2 bitmaps will be very small) are futile or not.

[y168]

Hard to tell for me if the problems in this stream are caused by an extraction tool or by the original stream. Anyway, the posted stream is defective and the error message given by BDSup2Sub is correct.

There are two captions with completely invalid packet size and wrong control offset. I fixed both manually to see if there are more problems.

Unfortunately, there are. Indeed exactly 10 subtitles are also damaged, but I didn't have the motivation to investigate any further. By commenting out these 10 lines in the IDX, the SUB/IDX becomes fully readable (well, the 10 lines are missing of course):
http://www.sendspace.com/file/eaw8jy

Thanks! I am wondering if it's possible to modify the codes so that those bogus lines are ignored. I do not mind if I miss 10 lines out of 2000, but now the program just stops and no subtitles are converted at all. If you can program to ignore the errors and just process the good ones, that would be wonderful.

[0xdeadbeef]

Thanks! I am wondering if it's possible to modify the codes so that those bogus lines are ignored. I do not mind if I miss 10 lines of out 2000, but now the program just stops and no subtitles are converted at all. If you can program to ignore the errors and just process the good ones, that would be wonderful.

Yep, I guess this should be possible in most cases. Consider it considered.

[avivahl]

Regarding the source code...
ToolBox:
public static String trim(final String s) - why not use String.trim()?
public static int getInt(final String s) - why not use Integer.parseInt()?
public static double getDouble(final String s) - why not use Double.parseDouble()?

[avivahl]

btw, would you consider accepting an SRT import patch?
If I'll make one, I'll make sure to:
1. Allow different fonts to be used.
2. Properly document everything.
3. Allow font aliasing (using Graphics2D's capabilities).
4. Use/implement your interfaces. (not doing ugly hacks)

EDIT: Another question... do you use eclipse?
Also, why call the directory bin_copy instead of simply "bin"?

[0xdeadbeef]

Dunno if discussing coding issues here is such a good idea since this is supposed to be mainly a support thread for users. Maybe we should discuss detail via PM. Anyway, this time I'll answer here:

About trim/parseInt
The ToolBox parse functions mainly catch the NumberFormatException and return a -1 instead. The trim function could most probably be replaced by String.trim() though. Can't really remember or explain why I did it this way ;)

Regarding SRT:
Well, if this doesn't involve a major increase of the JAR size or the integration of code parts with restrictive licensing, I have no problem to include your code. Indeed I though about it once, but then decided that it would be better to go for SSA/ASS instead, saw that this would be major work and dropped the idea again.

About Eclipse:
Yes, I'm using Eclipse. The problem with using "bin" is that Eclipse deletes this folder during a "clean". I tried to put the files in a "bin" folder in Subversion anyway, but this didn't seem like a feasible approach. So I decided for "bin_copy" to make clear that these files have to be copied to "bin" after a "clean".

[rernst]

Hi,

I wrote a Java frontend for mencoder for / mp4 box / mkvtoolnix (I really needed something that worked across platforms and was not Avisynth based). I am in the middle of adding Bluray support and while I am otherwise driving external code I see that you wrote this thing in Java to start with. I want to simply multiplex the subtitles in the end into the Matroska container (i.e. a trivial minor subset of what you do). Would you mind sharing the source code or at least the part that deals with the SUP->SUB/IDX conversion?

My stuff is open source/

[sneaker_ger]

The source is available under Apache License 2.0. Read the help for info on getting it.