BDSup2Sub - convert and tweak bitmap subtitle streams (VobSub,BD-SUP,BDN XML,HD-SUP)

[laserfan]

For a real exercise in frustration, try using BDSup2Sub for cleaning-up the English SDH in BD "Slumdog Millionaire". Easily a third of the 1400 subs are junk like:

[Hindi]
[Shouting in Hindi]
[Shouting]
[Screaming]
[Inaudible Dialog] (!!!!!!)
[Etc. Crapola]

Whoever did the subs for this one should be dragged across India behind an angry bull with diarrhea! My mouse hand still hurts after fixing these...

[Sharc]

You seem to have selected the sups for the hearing impaired.
I don't have this disk, but isn't there a regular sup included as well (means without these [annotations])?

[laserfan]

Originally Posted by Sharc

You seem to have selected the sups for the hearing impaired.
I don't have this disk, but isn't there a regular sup included as well (means without these [annotations])?

Of course I looked for a non-SDH track and there is none, duh. This BD release is in fact remarkably lacking in both audio and ST options...

[mrr19121970]

in both the BD and DVD the subs are 'burned' into the film. it was really bad watching it on my parents 21" 4:3 CRT - you couldn't even read them they were so small.

[laserfan]

Originally Posted by mrr19121970

in both the BD and DVD the subs are 'burned' into the film. it was really bad watching it on my parents 21" 4:3 CRT - you couldn't even read them they were so small.

I was not talking about the forced subs "burned into the film" though (and I can imagine these were hard to read on a 21" TV!). My posts are about the separate EnglishSDH subtitle track, which truly has every bloody line/grunt/song/sound-effect in the movie in its many [....] displays.

[0xdeadbeef]

Hi there, although I was kinda lazy lately and wasted more time reading through several books of Schlock Mercenary instead of coding or even doing something sensible, SUP/IFO support is nearly finished and working to some degree, but now I'd need a bunch of (free) reference tools for testing now.
So, apart from VobEdit, SubtitleCreator and DVDSupEdit: which (free) tools import/export SUP/IFO?

[hubblec4]

Originally Posted by 0xdeadbeef

Hi there, although I was kinda lazy lately and wasted more time reading through several books of Schlock Mercenary instead of coding or even doing something sensible, SUP/IFO support is nearly finished and working to some degree, but now I'd need a bunch of (free) reference tools for testing now.
So, apart from VobEdit, SubtitleCreator and DVDSupEdit: which (free) tools import/export SUP/IFO?

SubRip and IfoEdit are free tools.

[0xdeadbeef]

Ok, it's time for a not so small update again. As usual, I haven't tested nearly as much as I should, so chances are that there'll be a 3.9.1 soon. Anyway, I implemented most of the latest requests, so give it a try.

12.06.2009 3.8.2 -> 3.9.0

• Changed: introduced import and export of SUP/IFO.
• Changed: Opening SUB/IDX files via the SUB file (instead of the IDX) file is supported (also via drag'n'drop).
• Changed: added store+prev/next buttons to edit dialog.
• Changed: added mnemonics/hotkeys for most menus/buttons/checkboxes (Windows: press Alt to see the mnemonics).
• Changed: target palette can be exported to PCGEdit text format (via export dialog or CLI parameter "/exppal")
• Changed: JAR contains (only the) sources which are released under the LGPL and Apache license.

[0xdeadbeef]

Hm, no bug reports? Kinda disappointing (after a fashion)...
Don't tell me you're all out in the aching sunlight

13.06.2009: 3.9.0 -> 3.9.1

• Changed: editing of imported DVD palette is now possible
• Changed: exporting PCGEdit palette only possible if VobSub or SUP/IFO is exported.
• Changed: alpha cropping, minimum merge time and PGCEdit palette export settings are stored in the ini now.
• Changed: improved color edit dialog to allow editing colors by double clicking.
• Fixed: updated embedded JavaDoc (forgot about this in 3.9.0)

[hubblec4]

For me works all very well.

thanks a lot.

[deank]

It is me again.

Can you take a look at this idx/sub pair.

1) I used original dvd subtitles (those I sent you the first time).
2) Used mkvmerge to put these in a mkv so I can get S_VOBSUB
3) Extracted them again (with mkvextract)
4) Tried to load them in BDSup2Sub.

I get Invalid buffer error (#17)

I know it doesn't make sense to go back and forth, but it does for me atm.
I also know that somehow mkvmerge/mkvextract change something thus breaking something with bdsup2sub. I just hope that it is something easy to ignore/fix when parsing them.

They work when loaded by ffdshow and show correctly, but I can't convert them to sup.

I included both: those with underscores (__dvddump..._s-vobsub) do not work and the other (dvddump000) work.

Dean

[0xdeadbeef]

Hm, this subtitle stream is indeed more than a little weird.

Firstly, it uses stuffing bytes in the header (which most SUB/IDX tools don't support). This is probably the reason why SC doesn't load this at all. Most probably not wrong, but definitely a bad idea.

Secondly, the packet size for the packet starting at 0x10800 is wrong by one byte which results in a cropped command buffer. The packet size is given as 0x07E4, but it is really only 0x7E3 bytes in size. The missing byte is the last byte from the command buffer which is cropped off.
Now I can faintly remember that this happened before and I added a workaround. Unfortunately though, there is another issue here which renders the workaround useless.

Which brings us to the third problem: while the IDX file suggest that there's a subtitle at 0x10800 and the next one at 0x11800, the packet starting at 0x10800 indeed already ends at 0x10fff and there is a "garbage" fragment starting at 0x11000. This additional though useless packet spoils my workaround for the wrong size and leads to the crash.

So the stream is not only corrupt, it's corrupt in multiple ways. And while I could probably add another workaround, I don't really like where this is leading. It would clearly make more sense to fix the tool that creates this broken SUBs.

It's hard for me to tell which fault is introduced by which tool. I would assume that the wrong size info could really be an authoring fault. The garbage packet at 0x11000 seems to be introduced by one of the tools in your tool chain though. I suspect that this bug is really caused by the wrong size info in the original stream.

[EDIT]
Ok, I think I found an easy to implement (yet a little dirty) workaround. Still, I'd rather see the tools fixed which create this kind of SUBs.

[deank]

Now when you mentioned it... I think this one extra byte (in the original dvd subtitles) makes mkvmerge add another packet and then mkvextract reported (multiple times) that there is something wrong with the packet it is demuxing/writing (like... 'it is too large...').

I believe the problem came from mkvmerge which doesn't trim the packet (as your workaround does) and then it all goes wrong.

Thank you for looking into it!

It probably is an isolated case with my DVD title so it will probably work with other (properly muxed) subtitles

Dean

[0xdeadbeef]

Originally Posted by deank

Now when you mentioned it... I think this one extra byte (in the original dvd subtitles) makes mkvmerge add another packet and then mkvextract reported (multiple times) that there is something wrong with the packet it is demuxing/writing (like... 'it is too large...').

I believe the problem came from mkvmerge which doesn't trim the packet (as your workaround does) and then it all goes wrong.

Yep, this sounds plausible. Still, I decided to add (another) workaround (for the workaround):

14.06.2009: 3.9.1 -> 3.9.2

• Changed: added another workaround for corrupted VobSub streams.

[deank]

Hah Thanks a lot... I don't want this to break something else!

I did another test:

1) Used another pair of DVD subtitles
2) muxed them into a mkv
3) demuxed with mkvextract

[22:07:41] Processing (D:\Downloads\__TEST\SGA_VOBSUB.mkv)...
[22:07:41] Detected frame reference count: 3 (max:9).
[22:07:41] Detected video: ID#1:V_MPEG4/ISO/AVC 1280x720, fps=23.976, ref=3
[22:07:41] Detected subtitles: ID#3:S_HDMV/VOBSUB Language: bul
[22:07:41] Detected subtitles: ID#4:S_TEXT/UTF8 Language: bul
[22:07:41] Detected subtitles: ID#5:S_TEXT/ASS Language: und
[22:07:41] Detected audio: ID#2:A_AC3 (48000Hz 6ch) Language: und
[22:07:41] Added audio: A_AC3, "D:\Downloads\__TEST\SGA_VOBSUB.mkv", track=2, lang=und, timeshift=50ms
[22:07:41] Extracting SSA/VOBSUB subtitle tracks.
[22:07:41] Plase wait - it may take a while!
[22:08:12] Converting to SRT/PGS...
[22:08:12] Converting D:\Downloads\__TEST\SGA_VOBSUB.005.und.ssa to SRT...
[22:08:12] Converting D:\Downloads\__TEST\SGA_VOBSUB.003.bul.idx to SUP (java:BDsup2sub)...
[22:08:16] *** Imported subpictures: 277/277
[22:08:16] M2TS stream split set at 3999MiB...
[22:08:16] Adding external subtitles (SGA_VOBSUB.005.und.srt) (und)
[22:08:16] Adding external subtitles (SGA_VOBSUB.003.bul.sup) (bul)
[22:08:16] Title is trimmed: 00:00:00 - 00:40:00 (default)

It went all okay (with 3.6.0).

I will now download 3.9.2 to test with.

Dean


***

Great again Thank you so much! It extracted and processed the subtitles which previously failed!

[22:17:01] Extracting SSA/VOBSUB subtitle tracks.
[22:17:01] Plase wait - it may take a while!
[22:17:35] Converting to SRT/PGS...
[22:17:35] Converting D:\Downloads\__TEST\dvddump000_s-vobsub.003.bul.idx to SUP (java:BDsup2sub)...
[22:17:48] !!! BDSup2Sub reported: 283 warnings!
[22:17:48] *** Imported subpictures: 1037/1037
[22:17:48] M2TS stream split set at 3999MiB...
[22:17:48] Adding external subtitles (dvddump000_s-vobsub.003.bul.sup) (bul)

[turbojet]

Thanks for the import/export dvd sup support.

The dvd sup export is muxing great with ifoedit/muxman and pgcedit palette is correct during my limited testing.

However I notice importing dvdsup gives an error. I tried 2 different DVD's so far and all 13 sub streams had the same issue. Any chance you can look at this?

[avivahl]

Just tried opening an idx/sub w/ 3.9.2 and it didn't open.
SubResync opened the file without any problem.
Will send you the idx/sub in a pm.

[0xdeadbeef]

My retarded ISP deleted my account last Monday morning for unknown reason and was not willing or able to repair this issue yet. So I was offline for the last week and will be at least for another week. And even then, my 25MBit VDSL connection will be crippled to a 6MBit ADSL connection for an undefined amount of time. Did I mention that I'm using IPTV for year or so? Being cut off from the Internet is much worse of course.
To be at least able to read my mail, I just bought a serial (!) 56k Modem since I threw away my last one more than ten years ago. It was kinda tough to find such a relict from former times and tricky to install since my PC doesn't even have a serial port by default. Better don't ask how this is feeling compared to my VDSL connection

So just a quick update, but please understand that my internet access is pretty limited currently and I can't respond to issues as usual. Apart from that I'm now off for the weekend.


16.06.2009: 3.9.2 -> 3.9.3

• Changed: editing of frame palette/alpha values is now possible if the input is either SUB/IDX or SUP/IFO.
• Changed: editing of imported DVD palette works on the source image now (seems to make more sense).

@turbojet: I didn't download your sample yet, but it looks like a (DVD) SUP only. The online help explains quite explicitly that and why DVD SUPs need an IFO to be loaded. A single DVD-SUP can't be loaded by BDSup2Sub as it's mistook for a HD-DVD-SUP (same packet header) as explained in the help.

[turbojet]

Yup loading in the sup was my problem. With the same dvd sup (898 KB), I updated to include the ifo, when keeping the original palette the blue/yellow issue is back, swapping had no effect in this case.

Also when creating a new colorful palette while the DVD treats it correctly if you input the colorful ifo into bdsup2sub it looks funky. Is there any way to correct this?

[0xdeadbeef]

Well in short, BDSup2Sub uses the first color table in the IFO. So for a real IFO, this might be the wrong one.

Then again, since the imported palette as well as the frame palette/alpha for DVD based subtitles is editable now, it should be possible to fix this manually.

As a side note, by chance the "INI" format for palettes used by BDSup2Sub since quite some time is nearly the same as the text palette file used by PGCEdit. The main difference is the underscore ("color_0" instead of "color 0"). So by editing a PGCEDit text file and renaming it to INI, you can import the correct palette into BDSup2Sub even if it reads the wrong one from the IFO.