BDSup2Sub - convert and tweak bitmap subtitle streams (VobSub,BD-SUP,BDN XML,HD-SUP)

[Chetwood]

Can't, some moron took my nick. Never happened to me before.

[SassBot]

So I have import and export fully done. Now just need to finish all the conversion options.

[Pawpcorn]

Nice to hear! Good luck on the continued coding!

[SassBot]

Just another quick update. I've implemented almost all functionality except CLI, logging and writing out the INI settings. So hopefully it should be ready to be tested early next week.

[SassBot]

So the code is a bit messy right now, some classes will be refactored out and I'm sure there are a few benign memory leaks and a few cosmetic changes like updating the notes in the help, but I've finished the porting enough to post a test executable for Windows in a couple of hours once I get a static version of Qt compiled. I'll then look into some Linux executables but compiling this will require GCC 4.7 since I use a number of C++11 features so supporting older versions of distros without a current GCC will be too much work for me. Also, if anyone with a Mac would want to compile an executable for OS X that would be appreciated.

Anyone who wants to compile for themselves, the github link is here (https://github.com/amichaelt/bdsup2subplusplus/).

[SassBot]

So there were a few hangover bugs that I just caught and fixed. The source changes aren't pushed to github yet but here is a beta version.

Also to note: This will read and import the program and palette INI files from the oringinal program, but will rename the program INI to bdsup2sub++.ini. So to continue using your old BDSup2Sub.ini file just drop it in the directory with the new executable.

In my tests, I'm getting binary identical output for all files I've imported and converted so please report any bugs in the GUI, output differences (with attached files, please), crashes, etc.

[r0lZ]

libgcc_s_sjlj_1.dll is missing.

[SassBot]

Oops. Forgot the compiler flag to statically link that so I'm still learning all the options since it's the first time I've really used MinGW over MSVC++. Will post a new executable later.

[DoctorM]

@SassBot- There is a bug in the original I don't know if anyone else has seen.

Subtitles generated (at least in NTSC DVD) have an issue where ever other subtitle line will jitter up and down 1 line on screen when displayed.
It has something to do with the way BDSup2Sub crops, because even just loading subtitles and then saving them again will cause this to happen. (I tested this to rule out resize algorithms.)

I've tried different stand alone DVD players and different TVs and it's still present.
However, it doesn't seem to be noticeable when viewing a DVD with these subs on a PC.

Maybe you'll trip across something weird in the code that causes this. I've found BDSup2Sub to have limited use as a result.

[SassBot]

I've not seen that. Can you post sample files so I can look into it?

[r0lZ]

With Jeanl, I have tried to understand a similar problem with DVDSubEdit. My DVD player showed many cropped subtitles with dots or lines above or below the subpic. We haven't been able to fix completely that problem, but it was clear that it happened often when the subpic was moved or cropped by an odd number of lines. Since the problem reported by DoctorM seems related to interlaced video material (due to the jitter), I think it may have a similar origin. Perhaps cropping the subpic only at even lines and/or moving it in Y by multiple of 2 only is sufficient to avoid the problem. But I have never experienced that problem here (in PAL land) so, of course, I can't be sure.

[SassBot]

Yeah, that sounds probable. I'll look into that. Still working out the building to get everything statically linked properly. It's a bit more complicated than with MSVC.

[SassBot]

Ok new executable is here (https://github.com/downloads/amichaelt/BDSup2SubPlusPlus/bdsup2sub++.7z). i tested on a machine without MinGW and it opened fine. Source code on Github has some bugs so don't use that till I merge the fixes tonight that are in this build.

Report any issues with this. :-)

Also, r0lz, I'm working on the change you requested now.

[SassBot]

error message after loading dvd subtitles

http://i.imgur.com/Ipbtv.png

subtitles -> http://www.mediafire.com/?xg9oulfvj9eacpf

this only happens if X is larger than 15.

# Language index in use
langidx: X

I've located the error with this and will push out a new version that has a fix for it. The problem was that the original code was trying to find the subtitle id by taking the byte that identifies the subtitle number and was doing a "& 0xf" when what it need to do was subtract 0x20 (since the subtitle id is made from doing 0x20 + index). The "& 0xf" works as long as the id isn't higher than 15 though, obviously, we can have id's up to 31. I'm searching to see if there is a duplication of this bug in the writing code as well.

[SassBot]

New version is here (https://github.com/downloads/amichaelt/BDSup2SubPlusPlus/bdsup2sub++.7z). It has a the fix for Atak_Snajpera's reported error and some additional fixes and bugs worked out.

Once again, please report any issues that you come across.

[r0lZ]

I haven't really tested it yet, but I have already integrated your ++ version in my GUI. It seems to work fine, but it opens briefly a console window each time it is launched (even if it is called with arguments and its GUI is never opened). It's only a cosmetic issue, but is it possible to hide the console completely (or at least open it in minimized state)?

Another, less important cosmetic thing: There is a BDSup2Sub.ico file included in the original jar file. Could you use it for your exe?

Desktop Integration (Icons)
While BDSup2Sub has an application icon, unfortunately, the default coffee cup of the Java Virtual Machine is used if you created yourself a desktop icon to start BDSup2Sub. This is because a JAR is always opened with the installed JVM and so you get the icon of the JVM, not the one of the application inside the JAR. Still it's possible to change the icon manually (at least in windows, but I assume this is the same in Linux and MacOS). For your convenience, not only the PNG picture (icon_32.png) used by the application is included in the JAR, but also an ICO file for Windows and an ICNS file for MacOS X. I don't know too much about Linux icons, but I's guess you should be able to use the PNG. Anyway, to extract these icons, simply rename the JAR to ZIP and extract the files.

[r0lZ]

Command line bug:

When the program is called from the command line with one or two filenames, the program seems to convert the filenames to absolute paths (even if a path has been provided), but it uses the linux-style directory separator / instead of \. As a consequence, BDSup2Sub++ issues a "File not found!" error dialogue.

Example:

cd "C:\dir containing the sup file"
bdsup2sub++.exe subtitle.sub
--> File 'C:/dir containing the sup file/subtitle.sub' does not exist
bdsup2sub++.exe "C:\dir containing the sup file\subtitle.sub"
--> File 'C:/dir containing the sup file/subtitle.sub' does not exist

BTW, I noticed that the original jar version supports well the unix-style directory separator in the command line:

java -jar bdsup2sub.jar "C:/dir containing the sup file/subtitle.sub"
--> works well

Since some GUIs may pass the filenames in unix format, when a file passed as argument cannot be found, perhaps you could try to convert the / to \ and open the file again. This will ensure a good compatibility with the java version.

Also, the error message is displayed in a dialog. As far as I know, it is not printed to stdout or stderr. Can you print it? It's important to check the errors from the GUIs.
It might also be a good idea to remove the dialogue when the program is launched with two file names, and exit with a non-zero return code, as in that case, it doesn't open its GUI, and it should work without any user interaction.

[SassBot]

The console window opening is a side effect of a Windows Qt app that can also be used as a CLI app. Nothing I can do about that. To get console support you have to add a config in the project for it but it means the console window is automatically launched under Windows. The icon part will be fixed too. I just wanted to make sure nothing was broken first. Looking into the CLI bug now.

[SassBot]

Ok, the CLI bug is just a simple parser error in the copied over code. The program supports Unix separators as that's why Qt converts all filepaths given as CLI arguments to anyway. This is also the reason the dialog printed out the code using Unix separators it was just that I missed calling the "toNativeSepartors" method in that spot so it doesn't look like the native Windows path. Also, the dialog shows up because of it assuming that if you are giving it a single file that you want the GUI launched with that file, this is a carryover from the original version, and for some reason that warning dialog is popping up before the main GUI shows up even though I tell it not to do so before the GUI is visible. That seems to be some weirdness with Qt. I'll have the CLI thing fixed shortly.

[r0lZ]

Perfect. Thanks!

[mjuhasz]

Ok, the CLI bug is just a simple parser error in the copied over code. The program supports Unix separators as that's why Qt converts all filepaths given as CLI arguments to anyway. This is also the reason the dialog printed out the code using Unix separators it was just that I missed calling the "toNativeSepartors" method in that spot so it doesn't look like the native Windows path. Also, the dialog shows up because of it assuming that if you are giving it a single file that you want the GUI launched with that file, this is a carryover from the original version, and for some reason that warning dialog is popping up before the main GUI shows up even though I tell it not to do so before the GUI is visible. That seems to be some weirdness with Qt. I'll have the CLI thing fixed shortly.

BDSup2Sub had another CLI bug I fixed in the meantime: it was not possible to define the input and output file path as absolute path under a Unix-like operating system (Linux, MacOSx, etc). Fixed in this commit (https://github.com/mjuhasz/BDSup2Sub/commit/1667f739ee3705ca27ec7e6272f7c953a2fa37cd#src/main/java/bdsup2sub/BDSup2Sub.java).

The check for '/' at the beginning was meant for detecting the beginning of an option definition but under a Unix-like operating system the slash is also the path separator.

I removed that condition and expect the source to come first and the target next (and then the options).

BTW, there's a snapshot in the downloads section (https://github.com/mjuhasz/BDSup2Sub/downloads) in case you want to test the latest codebase.

I also wanted to compile a Linux version of your C++ code but it seems you use some C++11 features with which g++ is a bit unhappy. I'll look into that later. Nevertheless, very nice job!

[SassBot]

If you want to compile it you have to use at least 4.7. I use MingW-builds with 4.7.1 but just 4.7 should work.

Also, I saw you made that commit so I changed the parser code to only check the leading slash if under Windows.

[SassBot]

So there is something weird about the original BDSup2Sub's original parsing code for determining if there is only one commandline argument passed if it's a filename or a full command string. I've noticed that if you do:

java -jar BDSup2Sub "C:\path\filename with spaces"

or

java -jar BDSup2Sub "C:\path with spaces\filename"

or

java -jar BDSup2Sub "C:/path/filename"

that it loads the file right, but

java -jar BDSup2Sub "C:/path/filename with spaces"

or

java -jar BDSup2Sub "C:/path with spaces/filename"

causes the program to split the file path up and then you get an error that says the piece of the string up to the first space does not exist. I've put in a fix that short circuits its whole single command parsing loop if the single argument is a valid file path which should fix the problem you saw, r0lz. I also posted on your repo about it too, mjuhasz, in case you haven't seen it yet. I've also changed it so if the source file path is for a file that does not exist it will error out and write to stderr instead of opening the GUI with the dialog. I also made sure that anytime the CLI exists out it now properly writes to stderr instead of stdout like it did previously. New file is here (https://github.com/downloads/amichaelt/BDSup2SubPlusPlus/bdsup2sub++.7z). Once again, let me know if you see errors.

[SassBot]

Right now I'm converting the homegrown CLI parser to use a library instead which means the CLI syntax will change slightly but it won't be that drastic. I'll post when I finish.

[mjuhasz]

Right now I'm converting the homegrown CLI parser to use a library instead which means the CLI syntax will change slightly but it won't be that drastic. I'll post when I finish.

Very good.
I started doing the same using the apache cli-commons. I'll break the backwards compatibility with the current cli switches (hence the major version number bump) but that's a well tested library I had good experience with in the past and the current switches use a weird syntax anyway (e.g. /res instead of -r and --resolution). We can match our switches once we have the new code in place and maybe agree to have the same (or at least similar) cli interface.

[SassBot]

If anyone wants to test out the version on a Debian-based distro there is an i386 deb file here (https://github.com/downloads/amichaelt/BDSup2SubPlusPlus/home/adam/build/bdsup2sub++_1.0.0-1_i386.deb). It's built against gcc 4.7 so I'm not sure how old of a version it will run on. I had to build it on Debian testing to get the gcc version I needed.

[r0lZ]

Any chance to have the Win version with the CLI bug fixed?

[SassBot]

Already posted here (http://forum.doom9.org/showpost.php?p=1578722&postcount=923).

[SassBot]

Updated testing Windows executable here (https://github.com/downloads/amichaelt/BDSup2SubPlusPlus/bdsup2sub++.7z) that now has a combo box to select the language if the idx/sub has multiple languages rather than being confined to just the lang idx set in the IDX file. It also now has the proper icon for the taskbar. r0lz, the change you want will make it in once I've completed the transition of the CLI parsing code to the new library. You can currently do shifting from the original X or Y position through the move dialog in the GUI so it will be simple to support from the CLI once that transition happens.

Also, I'm working right now to get this packaged into Debian but I've yet to find a mentor yet.

[eTiMaGo]

I hope someone can help, I've been tearing out my hair over this for a while...

I'm trying to put Thai subtitles from DVD (in idx/sub format) into the equivalent Blu-ray title, using MultiAVCHD. But no matter how I demux or extract the files from the DVD, BDSup2 will crash or give an error. I have tried with 4.0.0, 4.0.1 and the Windows exe version posted above, with the same result. Tried on 3 different computers too, same result:

http://dl.dropbox.com/u/1600280/bdsup2subcrash.jpg
(on 4.0.0 and 4.0.1, a dialog would pop up "index out of bounds")

I don't think Thai subtitles have some kind of special format, so could someone more knowledgeable help me look into this? I've uploaded a sample subtitle rip, the same ones used for the screenshot above:

http://dl.dropbox.com/u/1600280/Fringe%20S1E01%20sub.rar

I would be eternally grateful if a workaround could be found, and will consider naming my firstborn after whoever can solve this :D

[r0lZ]

Have you tried to convert them with SubtitleCreator?

[eTiMaGo]

Thanks for that, there is *some* improvement, but...

The "antialias" layer is somehow like the background layer, a solid block of black around the text. If I set this layer to be at 0 opacity, then the text is play grey, so it may be hard to see...

http://dl.dropbox.com/u/1600280/subtitlecreator.jpg

As you can imagine, OCR for Thai is not so easy, there's some software out there but it takes single images as input.

I'll try and episode like this though, let's hope it's not too unreadable!

[r0lZ]

The problem of the colors and transparency is easy to fix. Just edit the IDX file. (There are also ways to fix them in SC itself, but I don't remember how. IIRC, it is possible to load the IFO in SC to use its palette automatically.)

And you don't need to OCR your SUP stream to convert it to SUB/IDX. (BTW, loading a SUP from the command line converts it to SUB without opening the SC GUI. I have associated the SUP files with SC, and I have just to double-click the SUP file to convert it in the same directory, almost instantly. The manual fix of the palette is longer! :()

Note also that SC does less error checking than BDSup2Sub. For example, it doesn't detect wrong time codes. But most f the time, it's not really necessary (except when your DVD is multi-angle or multi-story).

[r0lZ]

I'm trying to put Thai subtitles from DVD (in idx/sub format) ...
I did not pay attention to that. The format of the subtitles in a DVD is SUP, not IDX/SUB. So, I guess you have already converted the subs somehow, perhaps when you have extracted them from the DVD. The problem might be during that step. What program do you use to rip the subtitles?

[eTiMaGo]

I did not pay attention to that. The format of the subtitles in a DVD is SUP, not IDX/SUB. So, I guess you have already converted the subs somehow, perhaps when you have extracted them from the DVD. The problem might be during that step. What program do you use to rip the subtitles?

typo, should be suP, not suB.. But I will try again with another disc...

And for the OCR, I meant it would be nice if I were able to use it at that point, but it's not really an option :(

[SassBot]

So I've looked into this and your IDX file is broken. What program made it? It has a langidx value of 0 but the only index in IDX is 10 so the program is confused and doesn't know what to read. I'll put in a fix for the crash it causes but I'm not sure what to do in cases like this as a workaround. If there's only one id in the IDX it's simple but won't work outside that case.

[eTiMaGo]

Let me try again with another title that I am ripping right now :)

Meanwhile, can someone confirm what is the best method for extracting subtitles from a DVD that can then be used with BDSup2Sub? I've been using avidemux but this gives idx and suB files while according to r0lZ I should be getting suP files?

[SassBot]

I still just use VobSub Configure. I've yet to encounter any problems with it other than it being a little slow. And ripping as either IFO/SUP or IDX/SUB is fine.

Either way this has exposed some error checking I wasn't doing so this will fix a few other potential crashes.

[r0lZ]

according to r0lZ I should be getting suP files?
Well, SUP files are not really useful. It's the format of the DVD and BD discs, but it is not recognised by the other players. Maybe there is a bug in AviDemux. Anyway, I use PGCDemux (and sometimes DVDSubEdit) to extract the SUPs from the DVDs, and eac3to for the BDs. As far as I know, they have no bugs.

[eTiMaGo]

OK, my goal is simply to add them to an existing BD title either through TsMuxer or MultiAVCHD. Testing with PGCDemux now, let's see what happens!

[eTiMaGo]

Meh... Here's the .sup file right out of the DVD through PgcDemux:

http://dl.dropbox.com/u/1600280/Ep01.sup

Crashed when trying to open directly with BDSup2Sub, Unexpected command 0 at index 00000012

SubtitleCreator was able to open it, but all the palettes were set to grey, this could be fixed, but some stuff was still weird, like this, the half-block background is part of the "text" layer:

http://dl.dropbox.com/u/1600280/subtitlecreator_boardwalk.jpg

I saved it as a new SUP:

http://dl.dropbox.com/u/1600280/Ep01_aftersubtitlecreator.sup

Which does not load into BDSup2Sub either, now with an error Unexpected command 75 at index 00000166

On the other hand, if I save from SubtitleCreator in VobSub format (idx/sub), this can open in BDsup2sub, but again with the weird blocky backgrounds.

These subtitles are extracted from a brand new, original DVD (Boardwalk Empire Season 1), which displays these subtitles absolutely fine when played.

[SassBot]

Downloading the SUP file now. Can you try extracting an IDX/SUB with Vobsub Configure and see if that works?

[SassBot]

The problem with the SUP is that in absence of an IFO it thinks it's an HD DVD subtitle stream since there is no way from the file header to tell them apart from a DVD SUP. That's causing the errors.

That is also the cause of the palette problems you see in Subtitle Creator. You need an IFO to go with the DVD SUP otherwise there is no way to retrieve the palette info.

Edit to add:

So in order to fix the issue you see, I will have a prompt come up similar to what DVDSubEdit does which asks you to open the corresponding IFO file when you load a DVD SUP file. If you cancel this dialog it will assume it is an HD DVD SUP file and will precede accordingly. Does this sound like a fair fix? This will go along with the other crash fix for when the langidx doesn't match up to an index number like in your original IDX/SUB pair.

[r0lZ]

The problem with the SUP is that in absence of an IFO it thinks it's an HD DVD subtitle stream since there is no way from the file header to tell them apart from a DVD SUP. That's causing the errors.
Maybe the program could guess that it's coming from a DVD if all subtitles are contained in a 720x576 area.

[SassBot]

Maybe the program could guess that it's coming from a DVD if all subtitles are contained in a 720x576 area.

Yes, but then you will still get poor results like with Subtitle Creator since the IFO file is needed for palette info. Also, the video attributes like width and height are also in the IFO file or is there also a way to get it out of the DVD SUP file, too? I've appended my previous post to ask for feedback on a suggested change for loading solitary SUP files so it does similar to what DVDSubEdit does.

[eTiMaGo]

Downloading the SUP file now. Can you try extracting an IDX/SUB with Vobsub Configure and see if that works?

Can't get VobSub to work, I am on Win7 x64, even after copying the dll file to system32, the configure option just won't run... any alternatives? I tried DVD Decrypter to demux but it doesn't seem to be the right tool.

The problem with the SUP is that in absence of an IFO it thinks it's an HD DVD subtitle stream since there is no way from the file header to tell them apart from a DVD SUP. That's causing the errors.

That is also the cause of the palette problems you see in Subtitle Creator. You need an IFO to go with the DVD SUP otherwise there is no way to retrieve the palette info.

Edit to add:

So in order to fix the issue you see, I will have a prompt come up similar to what DVDSubEdit does which asks you to open the corresponding IFO file when you load a DVD SUP file. If you cancel this dialog it will assume it is an HD DVD SUP file and will precede accordingly. Does this sound like a fair fix? This will go along with the other crash fix for when the langidx doesn't match up to an index number like in your original IDX/SUB pair.

That sounds like a good fool-proofing system ;)

[SassBot]

Can't get VobSub to work, I am on Win7 x64, even after copying the dll file to system32, the configure option just won't run... any alternatives? I tried DVD Decrypter to demux but it doesn't seem to be the right tool.

Copy them to SysWOW64. That's how I got it working.

[eTiMaGo]

Wait so if I copied the IFO file out of the DVD, into the same folder as my SUP files, they *should* load properly?

[SassBot]

Yes, it should work fine.

[r0lZ]

So in order to fix the issue you see, I will have a prompt come up similar to what DVDSubEdit does which asks you to open the corresponding IFO file when you load a DVD SUP file. If you cancel this dialog it will assume it is an HD DVD SUP file and will precede accordingly. Does this sound like a fair fix?
That's nice, but perhaps you could add an option to assume that the SUP file comes from a DVD, even without an IFO. It is possible to fix the colours in the program anyway.

BTW, in the IFO, it will not be easy to retrieve the correct palette. An IFO can have many PGCs, and a different palette in all of them.