BDSup2Sub - convert and tweak bitmap subtitle streams (VobSub,BD-SUP,BDN XML,HD-SUP)

[SquallMX]

Originally Posted by shon3i

@0xdeadbeef, what about DVD SUP (+IFO palete) I/O ? VobSub isn't common format for many DVD muxers?

Most DVD SUP can be converted to VOB/IDX using Subtitle Creator 2.2+, and vice versa.

[turbojet]

Is there much of a difference outside of headers between BD and DVD sup's?

I would think it would be easier to implement then vobsub actually.

[prenz]

@0xdeadbeef: Great program, very handy for BD to DVD conversions. One thing though: On VobSub-Export, could you use a Padding Stream (i.e. 0x000001BE) instead of just stuffing 0xFF's till the pack end?

[0xdeadbeef]

Originally Posted by turbojet

Is there much of a difference outside of headers between BD and DVD sup's?

Both formats have more or less nothing in common. IMHO HD-DVD-SUPs are closer to DVD-SUPs, but still very different.

Originally Posted by turbojet

I would think it would be easier to implement then vobsub actually.

My understanding is that the DVD-SUP stream format should be pretty close to the SUB file of SUB/IDX, but reading the start times from the SUP, determining which the real start and end of the caption and handling the IFO for the palette make quite a difference. Regarding export, writing the palette to an IFO seems very messy.

Originally Posted by prenz

@0xdeadbeef: Great program, very handy for BD to DVD conversions. One thing though: On VobSub-Export, could you use a Padding Stream (i.e. 0x000001BE) instead of just stuffing 0xFF's till the pack end?

I'm not sure if I can follow you here. The packets/segments are aligned to 0x800 so unused part of a package is filled with padding bytes. In all the samples I saw, 0xff is used as padding byte - probably as this is also the end command.

[prenz]

Deadbeef, this is about the calculation of stream lengths. Let me show a subtitle pack (this is from DVDLab, as displayed by VobEdit):

   	[Pack Header]		
[0000] 	  Pack identifier (start code)	442	[000001ba]
[0004] 	  SCR (System clock reference)	68 2 156 173 181 99 	[44 02 9c ad b5 63 ]
   	    SCR 02725302.177		
[000a] 	  Program Mux Rate: 25200 (1260000 BPS) (10080000 bps)	1 137 195 	[01 89 c3 ]
[000d] 	  Pack stuffing length: 0	248	[f8]
   	  		
   	[Private Stream 1]		
[000e] 	  Private Stream 1 start code	445	[000001bd]
[0012] 	  Length	1728	[06c0]

In this particular stream, the private stream doesn't use up all of the 2k of the pack, but just 0x6C0 bytes, putting the end position to 0x6D4 relative to the start of the pack. Unfortunately some parsers expect the next Header Prefix (0x00 0x00 0x01) to follow immediately. If you just put a bunch of 0xFF here, there is no more Header and Stream/Pack-Parsing may crash. The correct way to pad to the pack end is to use a Padding Stream (as shown below). It has the Prefix (0x00 0x00 0x01) and stream ID 0xBE. Add the length (0x126, the remainder to the end of the pack) and then finally pad 0x126 bytes (usually 0xFF as you have seen in most streams).

   	[Padding Stream]		
[06d4] 	  Padding Stream start code	446	[000001be]
[06d8] 	  Length	294	[0126]

You may also encounter streams, where the remainder to pad is less than 6 bytes (what you would actually need to place a padding stream header). If this is the case, fill the Pack Stuffing field of the Pack Header with the byte count of the padding (Byte 13, Bits 0-2)

[turbojet]

Originally Posted by 0xdeadbeef

My understanding is that the DVD-SUP stream format should be pretty close to the SUB file of SUB/IDX, but reading the start times from the SUP, determining which the real start and end of the caption and handling the IFO for the palette make quite a difference. Regarding export, writing the palette to an IFO seems very messy.

Oh I forgot dvd sup relies on ifo for palette. While input from dvd sup would be badly colored you do have the option to change palette within the program or with vobsub. Also the ability to get colors from an ifo would help.

As for as export I agree writing an ifo would be a mess but there's an alternative at least with pgcedit by importing/exporting raw clb or rgb txt files.

[Mtz]

Originally Posted by 0xdeadbeef

[*]Changed: end time of one frame can be start time of the next frame (no forced gap of 1 frame any more)

Thank you! If is not difficult, I hope you can force 100-300 ms for the first subtitle if the starting value is: 00:00:00,000
So, if first line start with: 00:00:00,000, to have in the saved SUP: 00:00:00,300 In my tests with MPC-HC and tsmuxer are some problems and also my TViX refuse to play this file and I suspect the problem is that 000 ms. 300 ms can't harm the reading of a subtitle.

enjoy,
Mtz

[0xdeadbeef]

Originally Posted by prenz

Deadbeef, this is about the calculation of stream lengths. Let me show a subtitle pack (this is from DVDLab, as displayed by VobEdit):
[...]

I still don't get what this has to do with SUB/IDX export. AFAIK DVDLab doesn't export SUB/IDX and VobEdit doesn't import SUB/IDX for sure. So it seems to me that you confuse VobSub (as synonym for SUB/IDX) with VOB. As the SUB file contains snippets from the SUP stream embedded in the VOB, there are some similarities of course but still it should be clear that a SUB file from SUB/IDX is not a transport stream that can be used directly for muxing.

Originally Posted by prenz

In this particular stream, the private stream doesn't use up all of the 2k of the pack, but just 0x6C0 bytes, putting the end position to 0x6D4 relative to the start of the pack. Unfortunately some parsers expect the next Header Prefix (0x00 0x00 0x01) to follow immediately.

I'm still not convinced that you're talking about VobSub (aka SUB/IDX) here. A SUB/IDX parser would be pretty much braindead to expect the header prefix after a complete caption as the offset to the next caption is defined in the IDX file. Again: are you sure you're talking about VobSub = SUB/IDX and not about VOB or DVD-SUP???

Originally Posted by prenz

If you just put a bunch of 0xFF here, there is no more Header and Stream/Pack-Parsing may crash. The correct way to pad to the pack end is to use a Padding Stream (as shown below). It has the Prefix (0x00 0x00 0x01) and stream ID 0xBE. Add the length (0x126, the remainder to the end of the pack) and then finally pad 0x126 bytes (usually 0xFF as you have seen in most streams).

Indeed I'd bet that most tools featuring SUB/IDX import would plain crash if I did so.

[0xdeadbeef]

06.05.2009 3.5.1 -> 3.5.2

• Changed: VobSub: wrong packet length is detected and fixed (-> now SUB/IDX files exported before 2.1 can be imported)
• Changed: VobSub: improved luminance threshold detection when importing SUB/IDX files.

[prenz]

Originally Posted by 0xdeadbeef

it seems to me that you confuse VobSub (as synonym for SUB/IDX) with VOB.

No, I'm pretty much aware of the differences, although after discussing this with you I'm not too sure if my assumptions about VobSub are correct. Until now I have perceived .sub as a multiplexed program stream with a private stream inside - a subtitle stream according to DVD specs - and the .idx for a bunch of additional information as to e.g. timing, palette, file positions and some other things. Given the fact that .sub looks very much like a program stream, coming with pack and stream headers and all that stuff, I was confused why the packs ended early (because, from a parser's view, after the end of a stream there is either another stream or the end of the pack). Obviously VobSub doesn't care about PS-conformity. Some VobSubs are padded in a PS-conform way (e.g. those generated from DVB-Subtitles by ProjectX), others just fill up with 0xFF (as I have seen in VobSubs created by SubtitleCreator)

Originally Posted by 0xdeadbeef

VobEdit doesn't import SUB/IDX for sure

You'd be amazed what VobEdit will import. Contrary to its name it will edit any program stream you feed it (but indeed, it will crash on displaying packs #2-n of a .sub that doesn't use a padding stream).

Originally Posted by 0xdeadbeef

Indeed I'd bet that most tools featuring SUB/IDX import would plain crash if I did so.

You mean if the VobSub used a padding stream? Why do you think so? Anyway, I'd wager against it. I'm convinced that a VobSub-Import would simply ignore anything after the end of the pack's subtitle stream (including a padding stream or plain 0xFF-Padding), and will look for the next stream at the file position given from .idx. At least BDSup2Sub and SubResync will import a VobSub + padding stream without problems.

[0xdeadbeef]

Hm, I would assume that by adding the padding bytes, the SUB would get very close to a (DVD-)SUP. I never really examined the DVD-SUP format closely enough to tell for sure. I kinda fear that if I go in that direction people will jump onto the bandwagon and insist that the SUB is a 100% consistent and muxable DVD SUP stream. And this idea honestly gives me headaches.

BTW: Did you test if tools like MKVMerge and SubtitleCreator accept SUB files with padding? If so, I might consider adding the padding (and it rhymes, too).

[Ulf]

0xdeadbeef,

BDSup2Sub only treats a "PAL" XML file correctly on import if the video format is specified as "576p" in the XML file. If the video format is specified as "576i" (which is the correct way to specify the format), BDSup2Sub seems to assume that the import format is 1080p.

Could you correct this?
I guess it's the same thing with "480i", but I haven't tried it.

[0xdeadbeef]

Admittedly, I kinda didn't care much about interlaced formats up to now. Now regarding the BDN XML format the question arises if a frame (for the time stamp) is an interlaced frame or a progressive frame.

[mrr19121970]

Do you think it would be possible, when executing say:

java.exe -jar BDSup2Sub.JAR '*.sup' '*_FORCED.sup' /res:1080 /forced

to not stop when

#> 1542 (01:38:51.217)
ERROR: No forced subtitles found.
Press any key to continue . . .

is found, but to continue to the next file ?

that would be great.

[0xdeadbeef]

Yep, will be changed in the next version. Indeed the try/catch-block is already inside the loop, but currently each error exits the program.

[mrr19121970]

is this likely to happen soon, otherwise I'll change clown_bd - but I'm sure you're faster than me

[0xdeadbeef]

Just two quick fixes, untested. Ulf, mrr19121970 please have a look.

07.05.2009 3.5.2 -> 3.5.3

• Changed: when converting multiple files from the command line, a fatal error will not quit the program but just skip the file.
• Fixed: BDN XML: 576i and 480i are used for PAL and NTSC instead of 576p and 480p.

[mrr19121970]

great, works 4 me. thanks.

[Ulf]

Originally Posted by 0xdeadbeef

• Fixed: BDN XML: 576i and 480i are used for PAL and NTSC instead of 576p and 480p.

Thanks 0xdeadbeef - works fine!

[HeartWare2]

Thank you for this excellent program. It works fine, but... (there's always a "but" :-))

I have tried - unsuccessfully - to use this program to deliver something that I can process further with SubRip to do OCR scanning of the subtitles, but so far without any luck. I have tried 720p SUB/IDX, which kinda works, but for some pics SubRip seems to not be able to decode the file properly (shows garbage).

The PAL output seems to be processable, but due to the resizing, it is quite often that SubRip can't recognize the same letter as being such, which leads to an awful lot of manual entering of letters.

1080p SUB/IDX doesn't seem to be supported at all by SubRip.

I have also tried the PNG output, which I then converted to BMP using PMView (again 720p), but these files doesn't seem to be properly read by SubRip either.

Would it be possible to have your program output SUB/IDX files in a format compatible with SubRip (f.ex. without RLE encoding or something like that?).

Or if anyone knows about another utility that can du OCR processing on the SUB/IDX files created by BDSup2Sub (preferably the 1080p versions, as these should be clearer and thus easier to do successful OCR processing on).