BDSup2Sub - convert and tweak bitmap subtitle streams (VobSub,BD-SUP,BDN XML,HD-SUP)

[0xdeadbeef]

Ok, let's say I changed my mind about editing
Anyway, this release includes a major internal rework, so chances are again that I broke some things here and there.
I kinda wished I had used a more sensible system of revisions from the beginning, but indeed I never planned to include half of the features that are in now.
One good thing about the world economic crisis you might say as major parts of BDSup2Sub were written during forced vacations and short-time work.

22.03.2009 2.2 -> 2.3

• Split export dialog into conversion dialog and export dialog.
• Completely reworked the time stamp check. It's done at import now, not at export
  Also the source time stamps are not altered any more to fix missing/invalid times.
• Added edit dialog to change offsets and timestamps
• Added check/fix possibility for very short display durations. Also new command line option "/tmin".
• Automatic selection of language for SUB/IDX export if filename contains language name (e.g. "spanish")

[hubblec4]

Originally Posted by 0xdeadbeef

22.03.2009 2.2 -> 2.3

• Split export dialog into conversion dialog and export dialog.
• Completely reworked the time stamp check. It's done at import now, not at export
  Also the source time stamps are not altered any more to fix missing/invalid times.
• Added edit dialog to change offsets and timestamps
• Added check/fix possibility for very short display durations. Also new command line option "/tmin".
• Automatic selection of language for SUB/IDX export if filename contains language name (e.g. "spanish")

wow amazing. it looks so good.
i will test it in the next time.

so i need no more another programm to convert my BD.sups!
GREAT THANKS TO YOU.

hubble

[hubblec4]

Originally Posted by 0xdeadbeef

22.03.2009 2.2 -> 2.3

• Automatic selection of language for SUB/IDX export if filename contains language name (e.g. "spanish")

ok this feature are very bugy. german, english and turkish works correct but all other language not. when i want save the file it shows me ever French(fr).

test it with:
Thai
Portuguese
Spanish
Polish
Icelandic
Hungarian
Hebrew
Modern Greek
Czech

hubble

[hubblec4]

00001 - 12 - Subtitle (PGS), Czech, 627 captions.sup
00001 - 13 - Subtitle (PGS), Modern Greek, 629 captions.sup
00001 - 14 - Subtitle (PGS), Hebrew, 634 captions.sup
00001 - 15 - Subtitle (PGS), Hungarian, 628 captions.sup

demux by eac3to3.14 (switch, -demux)

hubble

[0xdeadbeef]

Works for me!?

[hubblec4]

Originally Posted by 0xdeadbeef

Works for me!?

mmh sorry for me not

[0xdeadbeef]

This leaves me puzzled. I tried with about ten different languages. No matter what I do, the language pre-selection works like it is supposed to.
Can anybody else confirm a problem with this feature???

[alc0re]

Converted .sup from get smart bluray to 720p...muxed with tsmuxer, played on my panasonic bd35 bluray player. No subtitles shown.

Back to using suprip for now I guess.

[0xdeadbeef]

Well, I would mostly blame the person who decided to include all that damn additional PTS and DTS stuff in the BD-SUPs. Compared to HD-DVD-SUPs, BD-SUPs are really a complete mess and obviously most authoring tools are kinda broken too if you look at all the garbage and redundancy that can be found in a typical BD-SUP.

Anyway: as long as nobody answers my according thread how to exactly calculate these various additional PTS and DTS timestamps, there's nothing I can do about this.

Most probably however, nobody (except the TsMuxer guys maybe) can answer this question at all because IMHO it's the muxer that has to fix all the time stamps of the muxed streams. All it should care about regarding the SUPs are the timestamps for start and end - which are correct almost certainly in the SUPs created by BDSup2Sub. If the Muxer can do this for SRT subtitles, I can't see any reason why it shouldn't be able to do this with BD-SUPs as well.

[~bT~]

Originally Posted by 0xdeadbeef

Can anybody else confirm a problem with this feature???

yes i can confirm. tried with hellboy and king kong so far.. ger is being detected always even tho it should be eng.

[0xdeadbeef]

Was there an "english" in the name(s) at all? "tried with Hellboy" doesn't really tell that much.
Again: what this feature does is simply scan the input file name for one of the integrated language names ("Spanish", "English", "German", "Norwegian" etc.). If there is no language given in the file name or if it's misspelled, then there's nothing to detect.

[~bT~]

^ my bad. it works as intended

[Pati]

Many many thanks 0xdeadbeef for this very useful tool!

However, I do have a problem with the subtitles of one HD-DVD I want to convert to Blu-Ray. Or actually two problems:

1) All the subtitles show all black (or rather, they show nothing).
2) All the subtitles have a duration of 28ms. The starting times seem to be correct, though.

Since the log of BDSup2Sub displayed offsets to the SUP file (a very useful feature, thanks for that!), I managed to do some debugging of this problem with a HEX editor. This is what I found:

1) The subtitles seem to show only black because all "Alpha Info" buffers/tables are filled with hex code 0xFF, instead of 0x00 like in a working HD-DVD SUP file I tested. I tested what happens if I just clear the buffer (rewrite 0x00 over all the 0xFFs), and after I did that to the first subtitle and reloaded the SUP file into BDSup2Sub, the first subtitle showed up fine. I tried the same thing to the second subtitle and that showed up fine as well.

It looks to me like 0xFF should be handled similarly to 0x00 in Alpha Info table. No tool I tried (BDSup2Sub, SUPRead and SupRip) seemed to do this, though. I don't understand *why* 0xFF should be the same as 0x00, but at least that way the subtitles seem to be shown correctly. Also ,it looks like I am not the only one that has encountered this problem: http://forum.doom9.org/showthread.ph...86#post1255086

2) I also compared the timestamps of my problem SUP with a working SUP file, and found out that the timestamp hexcodes of my problem SUP are like this:

#1
DCSQ start    ofs: 0x00002af9  (00:00:55:026) 00 00 00 00 2F 09 01
DCSQ stop     ofs: 0x00002f13  (00:00:55:054) 00 02 00 00 30 11 84
#2
DCSQ start    ofs: 0x000066f0  (00:00:57:262) 00 00 00 00 2D 88 01
DCSQ stop     ofs: 0x00006b0a  (00:00:57:290) 00 02 00 00 2E 90 84

However, in a HD-DVD SUP file that seems to give correct durations, the timestamps are as follows:

#1
DCSQ start    ofs: 0x00001879  (00:23:00:718) 00 00 00 00 1C 89 01
DCSQ stop     ofs: 0x00001c93  (00:23:02:157) 00 7E 00 00 1C 89 02
#2
DCSQ start    ofs: 0x00005148  (00:37:22:579) 00 00 00 00 38 BC 01
DCSQ stop     ofs: 0x00005562  (00:37:25:918) 01 25 00 00 38 BC 02

I don't quite understand the timestamp format, but it seems like there is an increment part and a base part, and looks like BDSup2Sub only handles the increment part.

(That working example is from a HD-DVD forced subtitles SUP file which only contains 5 subtitles events, which made it nicely compact for my tests :-)

Would it be possible for you to fix BDSup2Sub to handle this problem HD-DVD SUP file correctly? I suppose I could code a program that will preprocess my problematic SUP file, but I'm sure it would benefit many others as well if you could fix BDSup2Sub instead!

Thanks a lot for reading this far, and again thanks for BDSup2Sub!

Pati

[Freddy68]

Hi I am new to this forum, I've been using BD RB 2003 . I dont know if this is the right thread to post this question, but I have some hddvd movies that I've converted to bluray using HDBR Extractor and then TS Muxer 1.3.4. and I've been succesful as far as video and audio. But when it comes to subs, after I do HDBR extraction I have to use SupRead to export the subs in an acceptable container so TS Muxer is able to recognize it and convert it to bluray. After that is done, when I burn it, the video flashes when the subs appear and this happens back and forth. Let me remark that if I do the conversion without the subs everything plays perfectly. It only happens when I try to convert from Hddvd to bluray with the subtitles. Can someone help me on this?

[0xdeadbeef]

Originally Posted by Pati

1) The subtitles seem to show only black because all "Alpha Info" buffers/tables are filled with hex code 0xFF, instead of 0x00 like in a working HD-DVD SUP file I tested.

That's because HD-DVD uses an inverted logic for alpha values. 0xff is fully transparent and 0x00 ist fully opaque while usually (RGBA, but also BD-SUP) it's vice versa. A typical HD-DVD uses a fully transparent color as background color and fully opaque colors for the text. Values in between are used for anti-aliasing.

Originally Posted by Pati

I tested what happens if I just clear the buffer (rewrite 0x00 over all the 0xFFs), and after I did that to the first subtitle and reloaded the SUP file into BDSup2Sub, the first subtitle showed up fine. I tried the same thing to the second subtitle and that showed up fine as well.

Not surprising, but this just goes to show that the subtitles are completely transparent in the original stream. Maybe a demuxer author thought that this was a good idea or maybe the stream is defective. Hard to tell.

Originally Posted by Pati

It looks to me like 0xFF should be handled similarly to 0x00 in Alpha Info table. No tool I tried (BDSup2Sub, SUPRead and SupRip) seemed to do this, though.

Of course they don't, since this would disable transparency and would be plain wrong.

I don't understand *why* 0xFF should be the same as 0x00, but at least that way the subtitles seem to be shown correctly.

As I said, this is either a demuxer "feature" (most probable) or a corrupt stream (less likely) or maybe (but highly unlikely) there is some unknown bit somewhere that tells to not allow transparency at all. Anyway, treating 0xff just like 0x00 is wrong for sure.

Originally Posted by Pati

2) I also compared the timestamps of my problem SUP with a working SUP file, and found out that the timestamp hexcodes of my problem SUP are like this:

#1
DCSQ start    ofs: 0x00002af9  (00:00:55:026) 00 00 00 00 2F 09 01
DCSQ stop     ofs: 0x00002f13  (00:00:55:054) 00 02 00 00 30 11 84

The first word at the given offset is the time offset to the PTS for the start/stop display commands. The start display offset is 0, so the subtitle is displayed at the PTS (this is typical). The stop display offset is 0x0002, which is 2*1024/90000 seconds or 22.756ms after the start command. This is obviously crap. Could be a demuxer bug or whatever. The displayed duration of 28ms is due to frame rate synchronization btw.

Originally Posted by Pati

I don't quite understand the timestamp format, but it seems like there is an increment part and a base part, and looks like BDSup2Sub only handles the increment part.

As I said, the start/stop commands define a time offset based on the PTS. Though I don't think that's what you implied.
Anyway, most obviously BDSup2Sub handles both correctly, else every frame for every HD-DVD-SUP would have a zero duration or would start at the same (zero) time.

Originally Posted by Pati

Would it be possible for you to fix BDSup2Sub to handle this problem HD-DVD SUP file correctly?

If the stream is broken (and I'm pretty sure it is), there is no "correct" way to handle it. You're asking for a hack, not for a correction.
This being said, it's hard to impossible to automatically fix every possible defective stream. Anyway, to investigate an issue, I need the stream and I need to know how it was created.

E.g. if the inverted alpha was a common problem, I could add a hack for this (e.g. menu option "invert alpha"). As I said, maybe (but not very likely) it's even a feature of the stream but that's impossible to tell without any data to investigate.

About the timestamps: maybe this is an issue of multiple stop command as BD-SUPs are ridden by multiple start packets. Again: if I had the stream, I could have a look. Then again, maybe the stream is just bollocks since it was created by a buggy demuxer.

I would propose to extract the SUP again with a current version of either EAC3TO or TSMuxer and post it here (or better upload it anywhere and send me a PM) if it still gives you problems.

BTW: with BDSup2Sub 2.3 you can edit the display duration and can even force a minimum display duration of e.g. 2000ms.

[Pati]

Originally Posted by 0xdeadbeef

If the stream is broken (and I'm pretty sure it is), there is no "correct" way to handle it. You're asking for a hack, not for a correction.
This being said, it's hard to impossible to automatically fix every possible defective stream. Anyway, to investigate an issue, I need the stream and I need to know how it was created.
....
I would propose to extract the SUP again with a current version of either EAC3TO or TSMuxer and post it here (or better upload it anywhere and send me a PM) if it still gives you problems.

Thanks for a reply!

Sorry if I implied that BDSup2Sub was broken and needed a fix (that was not my intention, although reading my message again it does look like that). I am indeed after a "hack" that would allow me to use this broken SUP file.

I can upload it somewhere tomorrow (it is late here at the moment) so you can take a look at it. I tried all the tools I could find (and latest versions of all) to extract the stream, and all of them gave a similar result (and one tool, could have been EVODemux, complained about the original stream containing authoring errors). So I don't think the problem is the way I extracted the SUP stream, I believe the problem is in the stream itself.

Thanks again, I'll get back to you tomorrow!

Edit: Private message sent, hope you have the time and interest to look into it!

Edit2: Oh, and I forgot to comment on that duration problem...

#1
DCSQ start    ofs: 0x00002af9  (00:00:55:026) 00 00 00 00 2F 09 01
DCSQ stop     ofs: 0x00002f13  (00:00:55:054) 00 02 00 00 30 11 84

What I meant with the "base and incerement part" was that perhaps the 0x0002 in the "DCSQ stop" is the increment, and 0x3011 is the base (while the base for the DCSQ start is 0x2F09). I saw that in the working SUP file those 3rd words are the same in both "DCSQ start" and "DCSQ stop", while in my problem SUP they differ. Of course, since I know nothing about the timestamp format, other than what you explained above, that 3rd word might have nothing to do with anything.

Pati

[0xdeadbeef]

Originally Posted by Pati

What I meant with the "base and incerement part" was that perhaps the 0x0002 in the "DCSQ stop" is the increment, and 0x3011 is the base (while the base for the DCSQ start is 0x2F09).

Nope. That's what I meant with "I don't think that's what you implied".

Originally Posted by Pati

Of course, since I know nothing about the timestamp format, other than what you explained above, that 3rd word might have nothing to do with anything.

As I already said: only the first word does. the following dword is the (byte) offset to the next DSCQ and has nothing to do with timestamps.

Anyway, I had a look at the SUP and I think I know what the problem is there. Yet this isn't easy to fix and I don't even know if I want to invest much time in this.
The problem seems to be that the subtitles in this SUP are faded in (and out?). This (kinda) explains why the first subtitle is completely transparent at first and why the display duration is so short. Indeed after the display end marker, another alpha buffer follows and then a dozen more. I kinda lost track after the 4th or 5th, but there are lots more alpha buffers. There also seem to be several palettes. As the DSCQ times are accumulated, this also explains the short duration: BDSup2Sub only takes the first fading step, which results in a fully transparent, much too short subtitle.

While at first glance it seems to be easy to skip all the additional alpha and palette buffers and simply use the last one(s), this would only work for a fade-in, not for a fade out. There would be some ways to handle this of course, but I'm not quite sure if I want to invest the time for a dead format.

[Pati]

Ah, okay, thanks for taking a look, and for the explanation of what is going on with the subtitle.

That is fine, I'll just code something myself so that I can handle this particular subtitle (a preprocessor of sorts so that I can use BDSup2Sub for the actual work, and fix the durations manually if needed).

So, thanks again for BDSup2Sub itself!

Pati

[0xdeadbeef]

Don't get me wrong: I'll probably add a fix for finding the correct end time and skipping the first palette/alpha buffers the next days. Then again as I said, this won't completely fix the problem as it will only cure fade ins.

[hubblec4]

Originally Posted by 0xdeadbeef

22.03.2009 2.2 -> 2.3

• Automatic selection of language for SUB/IDX export if filename contains language name (e.g. "spanish")

Originally Posted by hubblec4

ok this feature are very bugy. german, english and turkish works correct but all other language not. when i want save the file it shows me ever French(fr).

test it with:
Thai
Portuguese
Spanish
Polish
Icelandic
Hungarian
Hebrew
Modern Greek
Czech

hubble

This small bug is simple. All BD.sup files works correct but the movie-folder named "French Connection". BDSup2sub choose french for the language.

hubble