BDSup2Sub - convert and tweak bitmap subtitle streams (VobSub,BD-SUP,BDN XML,HD-SUP)

[hubblec4]

one thing again.

in the save-box where i can choose the language: i miss some languages like norwegian etc. ...

can you add this?

hubble

[0xdeadbeef]

Just a dumb error - exactly half of the languages that ought to be in the list didn't get there. Will be fixed in the next version. I won't release a new version just for this though. But I'm pretty confident that you'll find another issue ;)

[~bT~]

06.03.2009 1.4 -> 1.5

Fixed redraw bug and reduced memory needed for zoom modes.
Fixed bug that caused wrong start time in case of multiple start packets or missing end packets.
Extended primary color detection to support "white,light grey" in addition to "light grey, dark grey"
Added possibility to edit/save/load Vobsub palettes
Fixed blatant bug in export of forced subtitles which would export the same frames over and over again
Set default alpha threshold to 80


thanks a lot 0xdeadbeef! any way to donate?

[0xdeadbeef]

thanks a lot 0xdeadbeef!
You're welcome! And there was not even a "but" ;)

any way to donate?
This is a spare time project and mainly driven by my own needs and the fun of it. While I might implement features requested by other people, I might as well decide to ignore such requests. Accepting donations, among a couple of other issues, would put me in a position where I would feel obliged to implement requested features no matter if I want to or have time to. This contradicts the idea of a fun spare time project (at least in my humble opinion).

[Kurtnoise]

great stuff...:)

May I ask you to add command line support ?

something like:
BDSup2Sub -i input.sup -o output.sub -lang English

:thanks:

[0xdeadbeef]

It's on my list. There is one big feature I'd like to add first though.

[IceM]

Nice work!

I did about the same as you did, but I could not manage to fix the vobsubs properly and put the project in the freezer.
If you want I can send you my reader implementation (all written in java). I managed to get quite a bit of info out of it (though, still some options unknown for me). Or we could work together on it? My plan was to make it an open source app... If interested you can drop me a pm or something.

[0xdeadbeef]

If you want I can send you my reader implementation (all written in java). I managed to get quite a bit of info out of it (though, still some options unknown for me).
I guess my SUP(BD) parser is nearly complete right now and I understand most of the fields. I'd take a SUP-Parser for HD-DVD though ;)


Or we could work together on it? My plan was to make it an open source app... If interested you can drop me a pm or something.
For the moment, this is a one man show. Indeed the project is too small and too "experimental" (other name for "dirty hack") at this time to make sense for shared development. If (and this is a big "IF") I will ever finish the planned core features AND I find the time to restructure and comment the sources to a degree that someone else can make use of them, I plan to release the sources for the common good. I don't think that this will become a big Sourceforge project though. At least it's not planned to become a big"all in one" solution like SubtitleCreator or whatever.
As I already mentioned, I plan to implement only two more bigger feature, where one of them is command line support. Chances are that I'll do only bug fixes after this point.

[0xdeadbeef]

08.03.2009 1.5 -> 1.6

Added support to write (BD-)SUPs (kinda experimental though, SupRip doesn't like'em). This included a major rework of the program's structure, so I hope I didn't break anything.
Rewrote upscaling function (bilinear: slower, but much better quality)
Fixed: half of the languages in the language selection box were not displayed
While I was at it, I sorted the lower part of the list alphabetically

[chros]

A long awaited feature was implemented by You!!! Thank You !
I'll try it ...

[hubblec4]

08.03.2009 1.5 -> 1.6

Added support to write SUPs (kinda experimental though, SupRip doesn't like'em). This included a major rework of the program's structure, so I hope I didn't break anything.
Rewrote upscaling function (bilinear: slower, but much better quality)
Fixed: half of the languages in the language selection box were not displayed
While I was at it, I sorted the lower part of the list alphabetically


thanks for your hard work.

i have tested somthing an found 0 errors.

In the Box "Mode" SUP->SUP better you call this SUP->DVD.SUP maybe


hubble

[0xdeadbeef]

thanks for your hard work.
i have tested somthing an found 0 errors.
Ah, great. I feared I might have broken something that worked before.

In the Box "Mode" SUP->SUP better you call this SUP->DVD.SUP maybe
It's a "BD-SUP to BD-SUP" conversion actually. As discussed in the online help, it's meant to e.g. convert a BD-SUP from 1080p to 720p. Since BD-SUPs are the only SUPs supported by BDSUp2Sub, I thought this was pretty obvious. I'll edit the history though.

[hubblec4]

It's a "BD-SUP to BD-SUP" conversion actually. As discussed in the online help, it's meant to e.g. convert a BD-SUP from 1080p to 720p. Since BD-SUPs are the only SUPs supported by BDSUp2Sub, I thought this was pretty obvious. I'll edit the history though.

ok. i understand. thats helpful for the BD-Copy_Community...

I test your new version again and when i found some bugs i report this...

hubble

[avivahl]

Great work! :)

[SquallMX]

The program works great :D.

I just have a little request, the option to resize subs to 1440x1080p, it's a Bluray compilant resolution helpful for movies too hard to compress at FullHD 1920x1080p (if you don't want to use 720p).

:helpful:.

Thanks for your hard work.

[0xdeadbeef]

I just have a little request, the option to resize subs to 1440x1080p, it's a Bluray compilant resolution helpful for movies too hard to compress at FullHD 1920x1080p (if you don't want to use 720p).

I might add this in the next release. But here's the one for today:

10.03.2009 1.6 -> 1.7

BDSup2Sub now also works from the command line. See online help for parameter description.
Some minor tweaks and correction here and there
Optimized RLE encoding for SUPs a little bit (two pixels of the same color don't use RLE code)
Tried to add some better (?) fake time stamps to the ODS/WDS/PDS/END packets.

[0xdeadbeef]

Me again...

11.03.2009 1.7 -> 1.8

Fixed bug in the palette conversion introduced in r1.6 that could lead to artefacts on the left side of the subpic (SUB/IDX export only)
Fixed wrong alpha threshold default in command line version (SUB/IDX export only)
Introduced new resolution 1440x1080

[SquallMX]

Me again...

11.03.2009 1.7 -> 1.8

Fixed bug in the palette conversion introduced in r1.6 that could lead to artefacts on the left side of the subpic (SUB/IDX export only)
Fixed wrong alpha threshold default in command line version (SUB/IDX export only)
Introduced new resolution 1440x1080


:thanks: You're amazing, thanks for make this incredible program!!!

[0xdeadbeef]

13.03.2009 1.8 -> 1.9

Added support for reading HD-DVD SUPs (I own only one HD-DVD, so chances are it won't work with many HD-DVDs)
The same selectable values appear in the ComboBox for source and target FPS (still can be edited)
Reworked online help a little bit (working Hyperlink to Doom9 thread and copy to clipboard functionality)
Renamed Mode/Format ComboBoxes to "Output Format" and "Output Resolution"
Fixed wrong display of target image size (introduced in r1.6)

[rack04]

Does anyone have a sample m2ts with subs they wouldn't mind sharing?

[turbojet]

Thanks for the sup to sup conversion however I haven't gotten them to display while playing when muxing with tsmuxer. I've tested both resizing and not resizing, powerdvd 8 and mpc-hc. The sup I posted earlier also shows this issue. Could you look into this?

SupRip before the rewrite (I think it was 1.10 or earlier) handles these same sup files ok except some characters are skipped over and I gave up tweaking suprip after about 10 minutes. SupRip versions after rewrite crash on these files but I've seen this happen quite often with the later versions.

Also any chance of a wildcard string for input via command line?
e.g. java -jar bdsup2sub.jar C:\sups\* /res:720 would resize all sup files in C:\sups\ to 720 and adding 720 to each filename during output

rack04: there's an example .sup I posted in this thread from a full movie you can mux that with any video as a test if that's what you are looking for.

p.s. as megaupload seems to be gaining popularity is there any specific reason for preferring it with a silly captcha and wait times, which reminds me of old rapidshare without the speed limiting which raipdshare still has, over one without all this extra stuff like sendspace or mediafire?

[0xdeadbeef]

Thanks for the sup to sup conversion however I haven't gotten them to display while playing when muxing with tsmuxer. I've tested both resizing and not resizing, powerdvd 8 and mpc-hc. The sup I posted earlier also shows this issue. Could you look into this?
Indeed I fear that this is an issue of the PTS/DTS time stamps. While I would assume that all the image related data in the exported SUPs should be ok as should the PTS time stamps of the display start and stop commands packets (PCS), I kinda don't know how to recreate the PTS (and DTS?) time stamps of the other packets (WDS, PDS, ODS and so on). I don't really know why a Muxer would need these time stamps and can only assume that it's possible to detect the frame rate of the SUP stream via packet size/time stamp calculations. It's kinda weird though to not implement a fall back if this fails. It's no like HD-DVD-SUPs or SUB/IDX files would have this kind of information.

SupRip before the rewrite (I think it was 1.10 or earlier) handles these same sup files ok except some characters are skipped over and I gave up tweaking suprip after about 10 minutes. SupRip versions after rewrite crash on these files but I've seen this happen quite often with the later versions.
All my trials with SupRip ended in SupRip simply ignoring my SUPs without the slightest hint why it does so. Any kind of output would help. Heck, I even tried to dig myself though these Delphi sources, but I couldn't even find a single line of code that seems to be related to BD-SUP decoding.

Also any chance of a wildcard string for input via command line?
e.g. java -jar bdsup2sub.jar C:\sups\* /res:720 would resize all sup files in C:\sups\ to 720 and adding 720 to each filename during output
I'll put it on my list. I'd prefer a syntax like this though:

java -jar bdsup2sub.jar C:\sups\*.sup c:\sups\*_720.sup /res:720


p.s. as megaupload seems to be gaining popularity is there any specific reason for preferring it with a silly captcha and wait times, which reminds me of old rapidshare without the speed limiting which raipdshare still has, over one without all this extra stuff like sendspace or mediafire?
There's no special reason to use MegaUpload apart from the fact that it seemed to be popular, simple and fast (enough). Any suggestions?

[turbojet]

Maybe you could compare what BDSup2Sub at 720p outputs from the sup I uploaded earlier to what BD-Rebuilder outputs at 720p (http://www.sendspace.com/file/e8ohwr) to help debug it? (beware that BD-Rebuilder has bad alignment while BDSup2Sub has good alignment)

Even a lot of retail sups I've tried crash the latest SupRip versions, have you tried SupRip 1.01 or earlier?

Your wildcard suggestion makes more sense.

Sendspace (http://www.sendspace.com/) I prefer these days, it seems to have very good routing throughout the world and no captcha, wait, speed limit (it says it does but it doesn't), 300 MB max file size, basic online file manager and a standalone app that supposedly can resume and upload multiple files but I've never tried it.
Mediafire (http://www.mediafire.com/) is another good one but it has a popup window, 100 MB file limit (if it ever matters), not the greatest routing, probably < 95% uptime, and it has just a basic online file manager. I've come across some other ones that are 'clean' but I can't remember the names.

[0xdeadbeef]

Maybe you could compare what BDSup2Sub at 720p outputs from the sup I uploaded earlier to what BD-Rebuilder outputs at 720p (http://www.sendspace.com/file/e8ohwr) to help debug it? (beware that BD-Rebuilder has bad alignment while BDSup2Sub has good alignment)
I don't need to take that indirection as I can simply load any SUP, export it and compare both versions. Unfortunately, the matter is a little more complex than it might seem. See also here: http://forum.doom9.org/showthread.php?t=145654

Besides, BD-Rebuilder cheats around this issue by not touching any important data at all. It only patches the screen size and offsets and doesn't even touch the image data. Since it doesn't change any package length with this attempt, it doesn't need to recalculate the PTS/DTS time stamps and simply copies them. Anyway, this of course means that the subtitles appear quite a bit larger in a downscaled 720p, because they are scaled up together with the video picture.

Even a lot of retail sups I've tried crash the latest SupRip versions, have you tried SupRip 1.01 or earlier?
Apart from the fact that I mixed SubRip and SupRip, I think I know what crashes SupRip, but I honestly think that this is an issue with the RLE decoding there (accepts EOL only if the line is complete - which contradicts the idea of RLE compression). So while I think I could change my exports to satisfy SupRip, I'd rather see SupRip fixing this issue.

[turbojet]

Yup BD Rebuilder texts are huge after resizing to 720p which is annoying, hopefully that's fixed soon.

Ya I agree that it's SupRip's responsibility to fix it as the issue goes way beyond BDSup2Sub output. My current fix is to not upgrade, unfortunately this is a fix that is becoming more and more common these days with various software.

[SquallMX]

13.03.2009 1.8 -> 1.9

Added support for reading HD-DVD SUPs (I own only one HD-DVD, so chances are it won't work with many HD-DVDs)
The same selectable values appear in the ComboBox for source and target FPS (still can be edited)
Reworked online help a little bit (working Hyperlink to Doom9 thread and copy to clipboard functionality)
Renamed Mode/Format ComboBoxes to "Output Format" and "Output Resolution"
Fixed wrong display of target image size (introduced in r1.6)


Hi, is this a bug?, MKVMerge fails to load some parts of the subtitles created with BDSup2Sub:

Original Sub:
http://rapidshare.com/files/209412308/BRSup.rar

IDX/VobSub:
http://rapidshare.com/files/209412099/VobSub.rar

MKVMerge Error Code:
Warning: spu_extraction_duration: Encountered broken SPU packet (next_off < start_off) at timecode 00:55:09.556. This packet might be displayed incorrectly or not at all.
Warning: spu_extraction_duration: Encountered broken SPU packet (next_off < start_off) at timecode 01:07:35.676. This packet might be displayed incorrectly or not at all.

:thanks::helpful:

[saint-francis]

Same here.
http://forum.doom9.org/showthread.php?t=145416
I thought it was an issue with mkvmerge.
It doesn't seem to affect playback though.

[SquallMX]

Same here.
http://forum.doom9.org/showthread.php?t=145416
I thought it was an issue with mkvmerge.
It doesn't seem to affect playback though.

In my case, subtitles for that timecode don't appear (if I load the subtitles has external subs in MPC they work fine), everything else works perfectly.

[0xdeadbeef]

Hm, I converted the "Spanish.sup" and loaded it into SubtitleCreator. Can't see any overlaps there. Can you? Does any other SUB/IDX tool display overlaps?

Then again, this problem might be related to the fact that BDSup2Sub currently is a little relaxed when it comes to creating time stamps. Indeed, BDSup2Sub currently doesn't care for the target frame rate and simply uses a millisecond time stamp, no matter what the output frame rate is.

A multiplexer might stumble over this as the timestamps (and display durations) are supposed to be multiples of a frame duration. If they are not, rounding up an end time and rounding down a start time to the next multiple of the frame duration might indeed lead to the same value of start and end. Which however is still no overlap.

Anyway, I already started reworking BDSup2Sub to always work with the 90KHz time ticks and synchronize the time stamps to the target frame rate. I'm not completely convinced though that this will resolve this issue in all cases (especially when there's also a speed up/down involved).

[rack04]

I haven't had any luck converting .sup to .idx using the command line. Here is the command line that I'm using:

java -jar "C:\Program Files\BDSup2Sub\BDSup2Sub.jar" "C:\Personal\Videos\DELSCENE1.sup" "C:\Personal\Videos\DELSCENE1.idx" /lang:en /res:480

Using this command line I get ERROR: File C:\Personal\Videos\DELSCENE1.sup is not a valid BD SUP stream.

However if I use the user interface I'm able to convert the .sup to .idx no problem.

[0xdeadbeef]

Yeah, I kinda forgot to remove the file check for the command line interface. So it still blocks HD-DVDs from the command line. Will be fixed in the next release.

[0xdeadbeef]

16.03.2009 1.9 -> 2.0

SUP: palette is only exported up to the highest entry actually used (previous versions always exported 256 entries)
Increased resolution of all internal time stamps to 90kHz (was 1kHz)
During export, start/end time (or start/delay) are synchronized to the target frame rate (special handling for PAL, NTSC and 24P)
HD-DVD SUPs were refused from the command line: fixed.
New frame rate: 23.975Hz. equal handling of frame rates for command line and export dialog (e.g. aliases)
Frame rate info is written into the PCS headers of created BD-SUPs (and read for src fps detection)
Wildcard support for command line (read online help for details)

[0xdeadbeef]

Regarding the MKVMerge issue, I got a little more insight what causes this problem. Indeed MKVMerge doesn't complain about a time overlap, but about a "wrong" offset. This seems to happen for each subtitle that doesn't fit into one 0x800 bytes packet and thus uses an RLE buffer split over multiple (usually 2) packets.

While I'm not so keen to claim that there is no chance that I messed something in my implementation, chances are that MKVMerge doesn't consider multi-packets at all. At least the fact that VobSync is able to decode the multi-packet subtitles created by BDSup2Sub hints that the command buffer and the offset can't be that wrong. Besides, I'm pretty sure I saw multi-packet subtitles displayed correctly when used to create a DVD.

So this might be an issue of MKVMerge. I'm not yet 100% sure though.

[SquallMX]

Thanks for the update and the research, you're right, only MKVMerge has problems with the subs, others VobSub programs works fine (VobSub, DirectVobSub, MPC Subtitle Module, SubSync, etc).

:thanks:

[turbojet]

Thanks for the update and uniterrupted download of it.

I tried the wildcard cli: java -jar BDSup2Sub.jar "'00001*.sup' '*-bbsup2sub.sup /res:720"

and I got

OPTION: set resolution to 720p (1280x720)
Exception in thread "main" java.lang.NullPointerException
at BDSup2Sub.main(BDSup2Sub.java:313)

I tried with spaces (added ') and without spaces in path and also outputting to *.sub and also without resziing maybe I'm missing something?

Also exported sups (1080 and 720) are displaying correctly now in mpc-hc and powerdvd, great work!

[0xdeadbeef]

I tried the wildcard cli: java -jar BDSup2Sub.jar "'00001*.sup' '*-bbsup2sub.sup /res:720"

You don't need the single quotes here at all since the file names don't contain any space. So the following should do as well:
java -jar BDSup2Sub.jar "00001*.sup *-bbsup2sub.sup /res:720"

If you need the single quotes (due to spaces in file names), they have to be around a filename, not around filename+parameter. I'll improve syntax checking in the next version though to avoid the exception.

[turbojet]

With same files but no space in path I tried: java -jar BDSup2Sub.jar "00001*.sup *-bbsup2sub.sup /res:720"
and the same error popped up

I also tried: java -jar BDSup2Sub.jar "00001*.sup *-bbsup2sub.sup" /res:720
and got: ERROR: No extension given for target /res:720

[0xdeadbeef]

Ok, it has nothing to do with single quotes. It's just that I always tested with the source files in a path. So I overlooked that no path inside the source file name results in a null pointer exception.
Until I fix this in the next release, use the following workaround:
java -jar BDSup2Sub.jar ".\00001*.sup *-bbsup2sub.sup /res:720"

[turbojet]

Thanks that works, that's not a bad way to do it especially for people with linux experience.

[0xdeadbeef]

While we're at it, there are some restrictions to the characters that the wildcard will replace. Indeed more or less only alphanumeric characters (and ".") are supported. I guess I will switch to a more relaxed regular expression for the next release, bur for the moment, special characters like "(){}[]!%&§#~" etc. are not matched by the wildcard.

I also made some progress regarding the MKVMerge issue. Indeed it's really a bug in BDSup2Sub. It happens only for subtitle frames where the RLE data would fit into 0x800 bytes, but due to the control header, a 2nd packet is needed. In this case, the overall packet length is wrong which is exactly what MKVMerge reports (if you know how to take it). VobSub Resync seems to be pretty relaxed about this and SubtitleCreator crashes when it tries to show such a subtitle. I have also at least one report that hints that this could lead to problems with standalone DVD players.
-> Will be fixed in next version.

[hubblec4]

I also made some progress regarding the MKVMerge issue. Indeed it's really a bug in BDSup2Sub. It happens only for subtitle frames where the RLE data would fit into 0x800 bytes, but due to the control header, a 2nd packet is needed. In this case, the overall packet length is wrong which is exactly what MKVMerge reports (if you know how to take it). VobSub Resync seems to be pretty relaxed about this and SubtitleCreator crashes when it tries to show such a subtitle. I have also at least one report that hints that this could lead to problems with standalone DVD players.
-> Will be fixed in next version.

Thats sounds really good. Thanks again for your hard work!

I had converted some BD.sups and found another issue.
in the idx-file i found some entries like this:

timestamp: 01:26:34:439, filepos: 000ce2000
timestamp: 00:00:00:000, filepos: 000ce3800
timestamp: 01:26:39:444, filepos: 000ce6800
timestamp: 01:26:40:486, filepos: 000ceb800

thats the last 4 entries!

in another case:

timestamp: 01:26:37:775, filepos: 000cdc000
timestamp: 01:26:41:613, filepos: 000cdf000
timestamp: 00:00:00:000, filepos: 000ce2000
timestamp: 00:00:00:000, filepos: 000ce4000


i think this entries are to much.

hubble


sample (http://rapidshare.com/files/210564392/subs.rar.html)

[0xdeadbeef]

Approval may take a while, better use Sendspace or whatever. Then again I had enough for today anyway.

17.03.2009 2.0 -> 2.1

Fixed: usage of wild card from the command line would cause exception when source file name didn't contain a path
Fixed: wrong VobSub packet size if only the control header was shifted to the next packet, but the first packet was not completely filled.
Fixed: delay can be entered in milliseconds again. It's adjusted to target frame rate automatically.
Added command line parameter "/dly" to define a delay
Added target frame rate auto detection for command line
Changed command line parameter "/fps" to allow setting of only the target frame rate

[turbojet]

Wildcards are working as documented in the help now thanks again.
Without really bloating the cli help do you think wildcard usage could be added?

[hubblec4]

17.03.2009 2.0 -> 2.1


Fixed: wrong VobSub packet size if only the control header was shifted to the next packet, but the first packet was not completely filled.



this fix is ok. mkvmerge shows no warnings anymore.

but the 00:00:00-timestamp-entries are there.

thats the last entries in the idx-file(BDSup2sub v.2.1)

timestamp: 01:26:34:355, filepos: 000ccb800
timestamp: 00:00:00:000, filepos: 000cce800
timestamp: 00:00:00:000, filepos: 000cd1000
timestamp: 01:26:41:446, filepos: 000cd6000
timestamp: 01:26:41:613, filepos: 000cd8800
timestamp: 00:00:00:000, filepos: 000cdb000
timestamp: 00:00:00:000, filepos: 000cdc800

its possible to delete this entries automaticly?

sample (http://rapidshare.com/files/210567462/00116_-_31_-_Subtitle__PGS___French__1670_captions.rar.html)


hubble

[0xdeadbeef]

Invalid timestamp are usually corrected by BDSup2Sub. There are obviously special circumstances here that I need to examine.
It's also possible that I broke this correction (partly) in 2.0 or 2.1. It basically still works though, as I tested several SUPs with invalid time stamps before I released 2.1.
We'll see when I get back from work and can have a look at your samples.

[Edit]
Ok, let's say it will be fixed in the next release ;)

[0xdeadbeef]

Only one little change this time:
18.03.2009 2.1 -> 2.2

Fixed: missing end times were not handled correctly.

[hubblec4]

Only one little change this time:
18.03.2009 2.1 -> 2.2

Fixed: missing end times were not handled correctly.


thanks. now i think works the converting perfect!!

question: sometimes is the time too short for showing the subtitles.

is it possible manuelly change the start- and endtime in your tool? :-)


hubble

[Mtz]

I don't think that is possible. But maybe to add some check if a subtitle have duration under 1 second (23 frames), to automatically extend the duration to at least 23 frames for any subtitle, but with the caution to not overlap the timmings with the next one.

enjoy,
Mtz

[hubblec4]

ok but i hope 0xdeadbeef can find a way to implement this feature. because SubtitleCreater doesnt support 1080p.sub/idx-files at the moment.

hubble

[0xdeadbeef]

Editing the times/position is not really planned currently as it would involve a redesign of several program parts.

Detecting too short subtitles is something I thought about some time ago but simply forgot.
I'd like to output a warning as default and fix the length only if it's selected via (export) menu.