BDSup2Sub++ - convert and tweak bitmap subtitle streams (VobSub,BD-SUP,BDN XML,HD-SUP

[SassBot]

Moderator Note: As paradoxical has assumed responsibility for continued development of this tool, the discussion is moved to:

http://forum.doom9.org/showthread.php?p=1613303

and this thread is closed. Thank you, SassBot, for your work and thank you, paradoxical, for your willingness to assume responsibility.

----- [resume original SassBot posting]

So as to separate from the other BDSup2Sub thread, I am creating a separate thread for my port BDSup2Sub++. For anyone who hasn't been following this is a port of the original BDSup2Sub to C++/Qt that I did in hopes of reviving what was a stagnating project. It has all the same functionality as the original version with some additional fixes and enhancements (such as proper support for importing SUB/IDX with multiple languages). If you have any issues or enhancement requests please post them in this thread or here (https://github.com/amichaelt/BDSup2SubPlusPlus/issues). The CLI syntax has been modified and is listed below.

Usage:

BDSup2Sub++ 1.0.0 0xdeadbeef, mjuhasz, Adam T.
Syntax:
bdsup2sub++ [options] -o outfile infile

Options:
-h, --help List options
--load-settings Set to load settings stored in INI file.
--resolution x Set resolution to 480, 576, 720 or 1080. Default: 576.
Supported values: keep, ntsc=480, pal=576, 1440x1080.
--fps-source x Synchronize source frame rate to <x>. Default: auto.
Supported values: 24p=23.976, 25p=25, 30p=29.967.
--fps-target x Convert the target frame rate to <x>. Default: keep.
Supported values: 24p=23.976, 25p=25, 30p=29.967.
--delay x Set delay in ms. Default: 0.0.
--filter x Set the filter to use for scaling. Default: bilinear.
Supported values: bilinear, triangle, bicubic, bell,
b-spline, hermite, lanczos3, mitchell.
--palette-mode x Palette mode: keep, create, dither. Default: create.
--minimum-time x Set the minimum display time in ms. Default: 500.
--merge-time x Set max time diff to merge subs in ms. Default: 200.
--move-in-ratio x Move captions from inside screen ratio <x>.
--move-out-ratio x Move captions from outside screen ratio <x>.
--move-y-offset x Set optional +/- offset to move captions by.
--move-x x Move captions horizontally from specified position.
Supported values: left, right, center, origin.
--move-x-offset x Set optional +/- offset to move captions by.
--crop-y x Crop the upper/lower n lines. Default: 0
--alpha-crop x Set the alpha cropping threshold. Default: 10
--scale-x x Scale captions horizontally by factor. Default 1.0.
--scale-y x Scale captions vertically by factor. Default 1.0.
--no-export-palette Do not export palette file.
--export-palette Export target palette in PGCEdit format.
--forced-only Export only forced subtitles.
--force-all x Set or clear the forced flag for all subpictures.
Supported values: set/clear.
--swap Swap Cr/Cb components.
--no-fix-invisible Do not fix zero alpha frame palette.
--fix-invisible Fix zero alpha frame palette.
--no-verbatim Switch off verbatim console output mode.
--verbatim Switch on verbatim console output mode.

Options only for SUB/IDX or SUP/IFO as target:
--alpha-thr x Set alpha threshold 0..255. Default 80.
--med-low-thr x Set luminance low/med threshold 0..255.
--med-hi-thr x Set luminance med/hi threshold 0..255.
--language x Set language to <n>. Default: de (Vobsub Only).
--palette-file x Load palette file <n>. Overrides default palette.

Output:
-o x, --output x Specify output file.

Wildcard support:
Use "*" for any character and "?" for one character in the source name
Use exactly one "*" in the target file name.
Example:
bdsup2sub++ --resolution 720 --fps-target 25p -o dvd_*.sub 'movie* 1?.sup'


Current version is 1.0.0 (17/7/2012):
Download link: here (https://github.com/downloads/amichaelt/BDSup2SubPlusPlus/bdsup2sub++100.7z).

Currently I only am providing Windows builds, but the source can be obtained here (https://github.com/amichaelt/BDSup2SubPlusPlus) if you want to build yourself. If anyone can provide OS X builds that would be great since I do not have access to a Mac, though you will need at least GCC 4.7.0 or some other compiler with support for all the C++11 features I use.

For Linux support, I am currently working to get this uploaded into Debian Testing. For Arch users, check this (https://aur.archlinux.org/packages.php?ID=60757&detail=1) out. I will also look into making packages for other distros as it is feasible due to the GCC version requirement.

[mood]

Thanks SassBot...

You have done a great job with this awesome tool.

[SassBot]

So I refreshed the 1.0.0 with a fix for what r0lz reported. Redownload if you have already have done so. The link is still the same as in the first post.

[r0lZ]

Happy to see the birth of v1.0! :-)

BTW, I've found another little bug. For whatever reason, I have now a BDSup2Sub++.ini file in my Windows\system32 directory. Perhaps when I did some CLI tests the CD was system32. Anyway, the default ini should always be saved in the installation dir (or in %appdata%).

[SassBot]

That's what happens. It's gets the directory of where the exe is running and saves it there.

I'm changing it so it uses %APPDATA% or on *nix platforms so that it saves to $HOME/.config. That'll be in the next version.

[mini-moose]

I think the download link is broken. It leads to 0 found on videohelp: "0 tool hits, Showing 0 to 0 tools"

[sl1pkn07]

https://github.com/downloads/amichaelt/BDSup2SubPlusPlus/bdsup2sub++100.7z

[SassBot]

I think the download link is broken. It leads to 0 found on videohelp: "0 tool hits, Showing 0 to 0 tools"

Yeah, link seems to be flaky. Sometimes works sometimes doesn't.

[mini-moose]

Supported values: 30p=29.967.

you mean 30p=29.970 (30000/1001) ?

Also, I think actual 24fps is missing (24.000 vs 23.976).

[SassBot]

you mean 30p=29.970 (30000/1001) ?

Yeah, just seems to be a typo carried over from the Java version. Thanks for spotting that.


Also, I think actual 24fps is missing (24.000 vs 23.976).

It's not missing. You just specify "24". 0xdeadbeef defined the '24p' as 24000/1001 and if you want 24.0 you just specify '24'. That's just from the code I absorbed. That list isn't meant to be the only values you can specify just a list of predefined shortcut values so you don't have to specify the rational values.

[mini-moose]

Thanks for spotting that.
It's not missing. You just specify "24".

great. yes, I looked at the gui and saw it has all the relevant fps choices. so the 24p/25p etc are just "shortcuts" to specifying the full number.

[mood]

Hi, SassBot...

I found a bug in version 1.0.0.

When I edit idx/sub created BDSup2Sub++ and use "Edit DVD Frame Palette" to set black background in one line...

In my TV not appears black background on line that I have set, but the problem with randomly background come back in many lines of subtitle.

Like this:
http://img99.imageshack.us/img99/9770/p1107121539.jpg


Here is the idx/sub with the problem:
http://www.mediafire.com/?91q982ruzbq2qze

The bug of randomly background in many lines back again. :(

Note: I use the same image, because the problem is the same

[SassBot]

Yeah, that's still on my list of things to nail down. I don't know why it came back. I'm busy with a few other projects at the moment, but I should begin new work by next week.

[qknet]

Dear SassBot,

I appreciate it when you decided to provide a non-java version.

I had my first try with the C++ version (the most up-to-date one), and it failed to load xml file (this one is handled perfectly by the java version). It says "First subpicture has timestamp of -00:00:15:857. Please specify proper delay value to correct this."

A part of the xml file is as follows:


<Description>
<Name Title="Subtitle" Content=""/>
<Language Code="vie"/>
<Format VideoFormat="1080p" FrameRate="23.976" DropFrame="false"/>
<Events LastEventOutTC="00:08:18:03" FirstEventInTC="00:00:31:20"
ContentInTC="00:00:00:00" ContentOutTC="00:08:18:12" NumberofEvents="49" Type="Graphic"/>
</Description>
<Events>
<Event Forced="False" InTC="00:00:31:20" OutTC="00:00:37:09">
<Graphic Width="1920" Height="1080" X="0" Y="0">00000764.png</Graphic>
</Event>
<Event Forced="False" InTC="00:02:56:07" OutTC="00:02:59:03">
<Graphic Width="1920" Height="1080" X="0" Y="0">00004232.png</Graphic>
</Event>
<Event Forced="False" InTC="00:03:41:19" OutTC="00:03:47:22">
<Graphic Width="1920" Height="1080" X="0" Y="0">00005324.png</Graphic>
</Event>
<Event Forced="False" InTC="00:03:56:13" OutTC="00:03:58:07">
<Graphic Width="1920" Height="1080" X="0" Y="0">00005678.png</Graphic>
</Event>
<Event Forced="False" InTC="00:04:02:13" OutTC="00:04:06:00">
<Graphic Width="1920" Height="1080" X="0" Y="0">00005822.png</Graphic>
</Event>

[SassBot]

Post the file and I'll look at it.

[qknet]

Post the file and I'll look at it.
Here you go. https://dl.dropbox.com/u/19135597/temp.rar

And, can you support idx/sub at 1080p?

[SassBot]

What do you mean? You can export Vobsub at 1080p. There is nothing additional to support.

[mini-moose]

the app crashes on a sup I tried to convert to vobsubs. It's a rather weird one. the old bdsup2sub handles it with tons of
fixing.

it can be found here:

http://www.mediafire.com/?zj86cr232ijlvfw

[SassBot]

Thanks. Will look into it. Been a bit busy with some other stuff (mostly getting distracted by watching Burn Notice and gaming) but I promise to start working on this again real soon. :) I have definitely not abandoned it.

[SassBot]

BTW qknet can you repost your file? The link isn't working.

[mini-moose]

The link isn't working.

hmm I just tried it and it's working fine but here's another anyway

http://www.sendspace.com/file/hszshq

[SassBot]

No, I was referring to qknet's dropbox link. I just get 404s from it.

[rakuen.now]

I just posted in the BDSup2Sub thread about how I fixed, at least on my system, the flashing bug for .sup(BD) output to tsMuxer when played back using XBMC. I don't know if anyone else is having this problem or is interested in fixing it. I looked up the code to the C++ version it looks very much the same as the Java version, so it should be afflicted the same way. Anyways, just a heads up in case anyone cares. If not, I'll go back under my rock...

[SassBot]

It's interesting to me. And, yeah, the code is pretty similar since it was a pretty straightforward port. I noticed you forked on github. If you send a pull request, I'll merge your fixes.

Right now, I'm doing some internal reorganization to cleanup some of the messiness of the port and restructuring things so that multithreading the writing out of files can be done (which at this point is basically impossible based on the current architecture carried over during the porting) so a new version will still be a bit off. I will get it out I promise. :) This cleanup will also allow me to write some unit tests to get decent code coverage to help squelch the regressions a bit due to the complexity of many areas of the code that cause weird cascading effects when you make what should be small changes.

[r0lZ]

I've noticed some bugs (or limitations) in BDSup2Sub++ 1.0.0.

In the "Move all captions" dialog, it is possible to move the captions horizontally from the original X position to the right (with a positive offset) or to the left (with a negative offset). That's fine. But the similar function to moye vertically from the original Y position works only with positive offsets. Therefore, it is not possible to move all captions up. Even positive offsets do not work well. Not sure why, but it's a pity. (I have not verified if it is possible to move the subtitles up with the command line.)

In the same window, it is possible to use the "crop bars" to restrict the move. If you use them, then apply the changes, you can see the subtitles at the correct position. But if you save the file and reload it, the subtitles are not where they should be. They are often under the crop bar (not visible any more). To permanently freeze the position of the subtitles, you must use "Reset crop offset" before saving. IMO, that's not very intuitive. Could you, at least, add a word of caution in the "Move all captions" dialog, or offer to reset the crop offset when saving?

Bug: In the window allowing you to move a single subtitle (available by clicking on the preview in the bottom right corner of the main window), the 4 buttons at the top (to save or cancel the edits and go to the previous or next subtitle) do not work.

[SassBot]

Hmm, weird. Okay, putting all those issues on the list of things to get fixed for 1.0.1. I know when I checked the move vertical that it supported positive and negative offsets but it's possible some other change to support other move options borked it since that code is a twisted mess. There's some behind the scenes math being done that can cause issues. As to the crop issue, you can just specify a 0 in the "crop offset Y" field to get the same effect as the "reset crop offset" menu option (unless I'm not fully understanding you) which I agree is a bit unintuitive. To the last issue, it seems that I just plain forgot to implement the functionality for the buttons. :o Don't know how I didn't spot that earlier.

Thanks again for the reports r0lz.

[r0lZ]

The crop bars are supposed to limit the vertical move in Y. For example, if I use a Move from original Y position with an intentionally big offset of 2000 and a crop value of 50, the subpics are moved to the bottom, but there will always be below them a margin of 50 pixels, due to the crop border. That works fine. If you type 0 to reset the crop border, the subpics will jump to the very bottom of the picture, and that's not what I want. So, I have to leave the crop border unchanged, and close the dialog. (Note that when the dialog is closed, the red lines are still visible, indicating that the crop border is "active".) So, now, it should be possible to save the subtitles, and they should stay where they are in the GUI, at 50 pixels of the bottom border. But that doesn't work. If you reload the subs, the crop border is gone, and the subpics are sticking to the bottom border. So, it seems that the crop border does not what it is supposed to do. However, I've found that if, after having closed the GUI window and before saving the file, I use the "Reset crop offset" menu, the subtitles are correctly saved at their visible position. IMO, that's counter-intuitive, as resetting the crop offset manually in the dialog (by entering 0) has the effect of disabling the crop border, but resetting it from the menu has exactly the opposite effect of "freezing" the subtitles. That may be the way it is programmed, but that's very strange and confusing.

Thanks for considering the two other issues.

[SassBot]

Does it do the same behavior in the original BDSup2Sub? That at least narrows it down to being a bug in the port.

[r0lZ]

Not sure. I'll verify...

[mbcd]

I have a short feature request:

I programmed my own "decoder" for PGS some times before, but had not enough time to do an "encoder" also ...

I need a function for "splitting" a PGS. That means:
I wanna do linked mkvs, so every mkv-part needs only a few subtitles.

My request is: Is it possible to implement a feature where you can specify a row of framenumbers as splitpoints, e.g. : 120, 15896, 24458, 225895, ...

Output gives you in this case 5 different pgs-files with subtitles that occour from frames:
0-120
121-15896
15897-24458
24459-225895
225896 - Subtitlefile-End

Every single pgs-File has to get a new calculated timecode that begins with "0", calculated from the offset where the split was to the first subtitleoccurance.

Mostly unused at this time, but I think that it will get more important in future with seamless branched titles.
Would be nice to get with cli too e.g.:
-split 120,15896,24458,225895 <- split the pgs but keeps original timecode
-split-resetTC 120,15896,24458,225895 <- split the pgs but resets original timecode at beginning of every file

[SassBot]

Okay. Add it to the github issues page (https://github.com/amichaelt/BDSup2SubPlusPlus/issues) and I'll look into it. :)

[mini-moose]

has dev stopped? or will resume when time allows it?

[hamedham]

thanks for this useful post
i searched for *.sub resizer in web but not any sites found
thank you "sassbot"

[jhon marvi]

BDSup2Sub is a tool I wrote initially to convert captions demuxed from a Blu-Ray transport stream (M2TS) into the DVD VobSub format (SUB/IDX) used by many DVD authoring tools - hence the name. Many more features were added over time as was support for other formats. So in the meantime the name seems a little inappropriate. In a nutshell, it's a subtitle conversion tool (for image based formats) with scaling capabilities and some other nice features.
:confused:

[TheShadowRunner]

Hi, I got 2 crashes with 2 SUP files (program crashes in ntdll.dll on XP SP3).
The SUPs can be found here (http://videoff7.free.fr/bdsup2sub_test.zip).
Hoping a fix is possible! Thank you.

[TheSkiller]

Hi guys,

I make my own DVD subtitles for my projects using Aegisub + avs2bdnxml + BDSup2Sub.

And so far I'm very pleased, the subtiles look very nice and this method enables me to take almost all the possibilities of ASS/Aegisub to DVD subtitles which is a HUGE relief. :)


Here's an example from the 16:9 subpic stream:
http://s3.imgimg.de/uploads/subpicexamplesmall77c05699png.png (http://s3.imgimg.de/uploads/subpicexample8f8cb7aepng.png)
(click for 720x574 full size)
I consider this almost perfect. Almost.
Something I have always wondered if it's possible to make it better is the outer edge of the outline, you know the transition between subpic and underlaying video (green here).

There tend to be those "rough" pixels, making the border a bit fuzzy. I noticed it seems to happen to the lowest part of any outline on a subpic mostly, in other words if there are two lines of text the upper one looks almost unaffected by it. Here's an illustration with 400% point-zoom:

http://s3.imgimg.de/uploads/subpicexampleproblem241f5a4dpng.png

I know the "Alpha Threshold" controls this border but it cannot fix it entirely, there are always some rough pixels somewhere, mostly at the bottom. And if I find a value which makes a single subpic look good, it makes others worse.
I mean it's not a huge deal but it keeps telling me my subpics aren't 100% perfect yet. :D :o
So, I keep scratching my head, either I'm doing something wrong or maybe it is some sort of bug in BDSup2Sub (happens with BDSup2Sub++, too, by the way)?
Would it be possible to add some mechanism that smooths the outline in terms of removing rough pixels?

If anyone feels like playing around, here's a ZIP containing the shown subtitle in XML + PNG format (output of avs2bdnxml).
http://www56.zippyshare.com/v/7044747/file.html (redundant, see edit)

The settings I used in BDSup2Sub:
Alpha Tresh: 20 | Med/Low Thresh: 96 | Hi/Med Tresh: 200
And important for my authoring, I use no cropping, by launching BDSup2Sub with /acrop:0

Output format is SUB/IDX.


Edit: Oh, well. I just found out the rough pixels at the bottom are caused by setting the Y font scaling in the ASS script to anything but 100%. So the problem was already in the PNG. At 100% it's gone, it's not a bug in BDSup2Sub. :o
But my question regarding a smoothing remains, as the alpha cropping in itself will still cause single rough pixels here and there which just look out of place.

Another Edit: I just realized such a smoothing algorithm to remove single stand-out pixels would be quite simple.
1) Check all the pixels of the final palletized subpic left to right individually and if the current pixel is [outline color] then check if the pixel left and right of it is [transparency color]. If both are, we have found such an unwanted pixel and can remove it by making it [transparency color].
2) Do the same in a vertical fashion, comparing the top and bottom pixel neighbours of all outline pixels.

[TheShadowRunner]

Since the main developer seems to have gone missing. I can look into it. What specifically were you doing to cause the crash? A step-by-step way to reproduce your crash would be great. :)Thanks for your reply, it's very simple:
I launch the program (with all default settings), and open one of the SUPs I linked. I then go to File > Save, chose "English" and it never reaches 100%, the program crashes halfway through. I think it just doesn't like something in those SUPs :/

[Selur]

anyone managed to compile BDSup2Sub++ on Linux?

[sl1pkn07]

https://aur.archlinux.org/packages/bd/bdsup2subplusplus-git/PKGBUILD
https://aur.archlinux.org/packages/bd/bdsup2subplusplus-git/bdsup2subplusplus-git.tar.gz

[Selur]

looked at the package but it only calls qmake and then make.
problem is if I run 'qmake bdsup2sub++.pro' and than 'make' on Lubuntu 64bit, I end up with a lot of complaints regarding ISO-C++ initialization.
Lubuntu 64bit uses gcc 4.6.3, which might be the cause of the problem,...
-> yup tried another VM with Kubuntu 64bit in it which uses gcc 4.7 and there it compiles.

[paradoxical]

Yeah, it uses C++11 features of GCC 4.7.x and higher. Clang 4.x would work too. I think the major issue is the non-static data member initialization. That's what bumps the requirements to 4.7.x.

[Selur]

I think the major issue is the non-static data member initialization.
yup. Thanks for the help.

[Music Fan]

Is this program able to convert DVB-SUB (sup extension) to sub/idx or other sup format (Blu-ray) ?
If not, could it be added ?
I know only one program able to make it, ProjectX, but it fails to do it correctly (colors and time-code problems).

[sneaker_ger]

Maybe:
Supported Formats

Blu-Ray SUP: Import (since 1.0) and Export (since 1.6)
Sony BDN XML: Import and Export (since 3.3.0)
HD-DVD SUP: Import (since 1.9)
VobSub (SUB/IDX): Import (since 3.5.0) and Export (since 1.0)
DVD-SUP (SUP/IFO): import and export (since 3.9.0)


I suggest you simply give it a try.

[TheShadowRunner]

paradoxical, any luck with these problematic SUP (http://videoff7.free.fr/bdsup2sub_test.zip)?
BDSup2Sub++ crashes during the convertion on my system :/

[paradoxical]

I've found where the issue is happening. It's a bit tricky figuring out why the problem is happening. I hadn't had a lot of time to debug it, but I'm working on it now.

[paradoxical]

Ok, I've located and solved the issue. Apparently there was a bug in doing a deep copy of an object that ended up having catastrophic effects down the line. Will post a fixed binary in just a few minutes after I email a patch to the original author.

[TheShadowRunner]

Wow this is awesome paradoxical, thank you VERY much for your time debugging this!

[Selur]

btw. while at it, it would be cool if the progress indication could be send to std::cerr instead of std::cout in command line mode. (std::cout get's delayed and this way monitoring the progress by watching the console output is a pain, since the progress only happens in large chunks,..)

[paradoxical]

Wow this is awesome paradoxical, thank you VERY much for your time debugging this!

Hehe, no prob. I was up for the challenge. :D The biggest challenge was adding enough log lines to isolate the issue. Once that was isolated it was easy to solve.

btw. while at it, it would be cool if the progress indication could be send to std::cerr instead of std::cout in command line mode. (std::cout get's delayed and this way monitoring the progress by watching the console output is a pain, since the progress only happens in large chunks,..)

Sure, it's a simple fix. I'll add that into the patch I make and will add that to the binary I make.

[Selur]

Sure, it's a simple fix. I'll add that into the patch I make and will add that to the binary I make.
Nice!! Thanks!

[r0lZ]

btw. while at it, it would be cool if the progress indication could be send to std::cerr instead of std::cout in command line mode. (std::cout get's delayed and this way monitoring the progress by watching the console output is a pain, since the progress only happens in large chunks,..)
Sorry, but I disagree. stdout is the correct handle for that kind of information. I use BDSup2Sup++ with a GUI that considers that an error has occurred if something is printed to stderr. Furthermore, it gets the whole content of stderr at the end of the process only. Of course, it is impossible to handle a progressbar that way. (stdout should not be delayed if open the handle in line mode.)

So, IMO, if you really want that, it should be an option of the command line, and still use stdout by default, for compatibility reasons.

[paradoxical]

Okay, I'll add a commandline option for it which defaults to the original behavior. Sounds like a good compromise.

[Selur]

If in GUI mode, it should not write anything to std.
Adding an additional option to get the progress on std::cerr to allow monitoring the progress with another tool/gui would be fine too. :)
(It's just bugging me that atm. progress indication like it is done atm. through CLI is not really useable,..)

[paradoxical]

It doesn't write anything out to a standard stream in GUI mode. It writes everything to the log window.

[r0lZ]

Okay, I'll add a commandline option for it which defaults to the original behavior. Sounds like a good compromise.
Thanks.

[paradoxical]

So I added the commandline option "log-to-stderr" otherwise it defaults to the current behavior. New binary can be grabbed from *removed to recompile*.

[Selur]

Thanks! :)
> The program can't start because libwinpthread-1.dll is missing from your computer. Try reinstalling the program to fix this problem.
-> looks like you compiled it dynamically.

[paradoxical]

No, I just didn't add the lflags in correctly. Recompiling now.

[Selur]

"Recompiling now. " -> thanks :)

[paradoxical]

Try *link removed*.

[Selur]

doesn't output anything and just crashed on a .sup file (send you a link via pm)

Cu Selur

[paradoxical]

Ok, found the error and now your commandline works fine for me Selur. I've notified SassBot on github again with this change. Until he notices and pushes this change *linked updated below* is a patch file for anyone compiling their own. *linked updated below* is another compiled version.

Note this error only effected if you used the logging to stderr.

[Selur]

output seems to be working fine now,..
call is still crashing for me though,... outputs '#> 743 (00:58:59.954)' and then no output any more and a crash. :/ (btw. I'm on Win7pro 64bit)
-> Here comes the kicker: I just tried and renamed my files and called:
bdsup2sub++ --resolution pal --log-to-stderr --output "H:\Temp\test.sub" "H:\Output\test.sup"
-> works
bdsup2sub++ --resolution pal --log-to-stderr --output "H:\Temp\test.sub" "H:\Output\Trying This (2013)_id_3_en_default.sup"
-> works
bdsup2sub++ --resolution pal --log-to-stderr --output "H:\Temp\test.sub" "H:\Output\and This (2013)_id_3_en_default.sup"
-> crashes
bdsup2sub++ --resolution pal --log-to-stderr --output "H:\Temp\test.sub" "H:\Output\Now This (2013)_id_3_en_default.sup"
-> crashes
(I can repeat this, seems like as soon as the starting part of the output name is short I get a crash,..)
-> something is fishy, seems like theres at something of with the parsing of the input file name,... (or my system is going crazy)

[paradoxical]

output seems to be working fine now,..
call is still crashing for me though,... outputs '#> 743 (00:58:59.954)' and then no output any more and a crash. :/ (btw. I'm on Win7pro 64bit)
-> Here comes the kicker: I just tried and renamed my files and called:
bdsup2sub++ --resolution pal --log-to-stderr --output "H:\Temp\test.sub" "H:\Output\test.sup"
-> works
bdsup2sub++ --resolution pal --log-to-stderr --output "H:\Temp\test.sub" "H:\Output\Trying This (2013)_id_3_en_default.sup"
-> works
bdsup2sub++ --resolution pal --log-to-stderr --output "H:\Temp\test.sub" "H:\Output\and This (2013)_id_3_en_default.sup"
-> crashes
bdsup2sub++ --resolution pal --log-to-stderr --output "H:\Temp\test.sub" "H:\Output\Now This (2013)_id_3_en_default.sup"
-> crashes
(I can repeat this, seems like as soon as the starting part of the output name is short I get a crash,..)

Can't repeat it even using the file names you said crashed. It always finishes successfully. Sorry. :(

-> something is fishy, seems like theres at something of with the parsing of the input file name,... (or my system is going crazy)

The input file name comes from the QxtCommandOptions parser. And it's coming out correctly for all those file names you specify. The application itself isn't parsing argv.

[Selur]

Okay, will do some more testing here tomorrow.
Thanks for trying to repeat the problem. :)

Cu Selur

[paradoxical]

Actually, I was finally able to do so! And I think I have a fix for it. Will post a new executable when I make sure I got it fixed right.

Somehow I was able to change options enough to get a 0xbaadf00d hex value, according to Microsoft this is due to an uninitialised allocated heap memory when the debug heap is used, in a field that was being called on that caused a crash. No clue why it wasn't getting this value before or what I exactly did to expose it finally.

[Music Fan]

Don't you know if your program supports DVB-SUB ?

[paradoxical]

Did you read sneaker_gear's post? He quoted the list of file formats it supports.

[Music Fan]

Ok, so I forget your program.

[paradoxical]

Last version for today. I might work on other outstanding issues as I see from the github page that SassBot hasn't gotten to. For one thing, I added the functionality so that the arrow buttons work in the edit dialog in this version as an extra fix.

Here (https://www.dropbox.com/s/r4b0yk4hxw8v6rc/file.patch) are the current patches that I'm pushing to Sassbot. Here (https://www.dropbox.com/s/6429efffiwxlj7u/bdsup2sub%2B%2B.7z) is the newly compiled exe.

[paradoxical]

I can look into adding support for DVB Sub, but I don't promise anything.

[Music Fan]

Thanks, if you need a DVB-SUB file, I can send you this ;)

[paradoxical]

Sure, samples would be good.

[r0lZ]

I can look into adding support for DVB Sub, but I don't promise anything.Great idea!

[Music Fan]

Here is a dvb-sub sup file ;
http://www51.zippyshare.com/v/21056401/file.html
I extracted it from TS with TSDoctor's demuxer.
It's supposed to respect the norm ETSI EN 300 743.

[paradoxical]

Link doesn't seem to work.

[sl1pkn07]

Link doesn't seem to work.

working for me

https://dl.dropbox.com/u/6596386/DVB-SUB.rar

[TheShadowRunner]

Thanks again for your hard work paradoxical, I was finally able to convert the 2 problematic SUPs without having the application crash on me.

[Selur]

I agree thanks a lot, last version works fine here too. :)

[paradoxical]

So looking into the DVB subtitle format more. Does anyone know if it's really valid to extract a raw subtitle stream? From my reading of the specification it seems that by doing so you lose the PTS values as they come as data that precedes the subtitle stream PES packet data. And no tools that read DVB Subtitles seem to accept the raw sup stream you gave me, they all require the transport stream which seems to confirm that a file contain just the PES packets isn't really valid in itself. Even projectx converts the DVB subtitles to another format when extracting.

So, yes, I can add the support for DVB subtitles but it will require you to load the original transport stream in order to do the conversion rather than loading the raw stream of PES packets.

[mini-moose]

Here (https://www.dropbox.com/s/r4b0yk4hxw8v6rc/file.patch) are the current patches that I'm pushing to Sassbot. Here (https://www.dropbox.com/s/6429efffiwxlj7u/bdsup2sub%2B%2B.7z) is the newly compiled exe.

I just tried it with a sup sassbot's latest build was crashing on and this one managed to go through it fine :)
It seems to me the ++ versions are slower to decode/create vobsubs though. I measured the time it took to create a resized to 1280x720 vobsub using ++ and java and it's about 25% slower on the ++ ones. I'm unsure if it's the language or the fixes present in the ++ versions require more decoding time.

[Music Fan]

Even projectx converts the DVB subtitles to another format when extracting.
Actually, it lets the choice between a simple extraction (like TSDoctor) and a conversoin to sub/idx, which fails (with my extract, it works maybe in other cases).

So, yes, I can add the support for DVB subtitles but it will require you to load the original transport stream in order to do the conversion rather than loading the raw stream of PES packets.
Great.;)
Anyway, it seems to be the only solution.
Again, if you need a sample (subtitled TS), I can send you it.
Can you now open the zippyshare link I gave for sup 2 days ago ?
If no, I'll have to upload the TS (if you need it) on another server.

[paradoxical]

I just tried it with a sup sassbot's latest build was crashing on and this one managed to go through it fine :)
It seems to me the ++ versions are slower to decode/create vobsubs though. I measured the time it took to create a resized to 1280x720 vobsub using ++ and java and it's about 25% slower on the ++ ones. I'm unsure if it's the language or the fixes present in the ++ versions require more decoding time.

It's probably nothing more than a little optimization needed. Looking back through commit history I can't really see that most of the would really add any decoding time. I'm gonna be running it through Valgrind later to get some perf data.

Also, there are some architectural problems inherent in original the Java version that were carried over in the porting that need to be fixed so that creating/writing out subs can be properly multi-threaded which should give good speed improvements. Right now, both operations do things in serial.

Actually, it lets the choice between a simple extraction (like TSDoctor) and a conversoin to sub/idx, which fails (with my extract, it works maybe in other cases).

All I know is that what you gave me doesn't have any PTS values since TSDoctor seems to only extract the subtitle PES packets. Which makes it impossible to determine start times for the subtitles.


Great.;)
Anyway, it seems to be the only solution.
Again, if you need a sample (subtitled TS), I can send you it.
Can you now open the zippyshare link I gave for sup 2 days ago ?
If no, I'll have to upload the TS (if you need it) on another server.

I've found a few samples I'm working with now so not at the moment. I might need more in the future just to test the code more.

[Selur]

btw.
1. if anyone manages to build BDSup2Sub++ statically on mac os please send me a build. :)
The gcc 4.7 requirement kills me on mac. I get gcc 4.7 compiled and working, but I can't get Qt and libqxt statically compiled with it. :/
(static compilation works fine on linux, on windows I normally use the VS C++ express compiler and since even VC11 doesn't support C++11 'Non-static data member initializers' that's a no-go too :()
2. if some one has the time and motivation, it would be nice if he could fix the whole 'non-static data member initialization' in the header files (seems to be the only C++11 needed)

Cu Selur

[paradoxical]

You don't need GCC if you're on OS X. Just use clang 4.x which comes with recent XCode. If you really must have GCC, just get it from Macports. There's no need to compile it yourself.

2. if some one has the time and motivation, it would be nice if he could fix the whole 'non-static data member initialization' in the header files (seems to be the only C++11 needed)


More C++11 is used than that. Also, it's not really a "fix" since to change it back to old style C++ since involves code bloat and to maintain the C++11 features would require tons of #ifdefs. All up-to-date version of any decent compiler, MSVC obviously not included in this list since it still sucks at implementing features, has all the C++11 support needed to compile this for quite some time now.

In the end, I can compile a binary when I get home since I already have a Qt environment set up.

[Selur]

I will look into clang,...

In the end, I can compile a binary when I get home since I already have a Qt environment set up.
that would be really appreciated :)

[paradoxical]

Yeah, just grab latest Xcode. It's what I use to compile on OS X for my own testing. But, like I said, if you need GCC just grab it from Macports. There's no good reason to compile it yourself.

[Selur]

last time I installed macports it screwed up my normal build environment so I try to avoid that option. ;)

[paradoxical]

*shrug* I got multiple versions of GCC installed through Macports and Xcode, etc. installed and all is good. You must have just gotten unlucky.

[Selur]

clang doesn't like me either :/
In file included from ../src/bdsup2sub.cpp:22:
../src/Subtitles/subtitleprocessor.h:55:20: error: no matching constructor for initialization of 'QStringList'
static QStringList resolutionNamesXml = { "480i", "576i", "720p", "1440x1080", "1080p" };
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Trolltech/Qt-4.8.4/include/QtCore/qstringlist.h:69:12: note: candidate constructor not viable: requires 0 arguments, but 5 were provided
inline QStringList() { }
-> giving up for today, so would be nice if you later find the time to compile a static BDSup2Sub++ and upload it somewhere.

Cu Selur

[paradoxical]

You need Qt compiled with C++11 support to get the initializer list support. This is why I said there is much more C++11 use than just the non-static member initialization. :p

Yes, I still compile the binary for you.

[Selur]

[qutoe]You need Qt compiled with C++11 support to get the initializer list support.[/quote]
I recompiled QT with platform macx-clang. :/

[paradoxical]

Which isn't enough. You need to pass -std=C++11 so that it will compile the C++11 support in Qt.

[Selur]

Will try that tomorrow. :)

[paradoxical]

Also forgot, you need to pass -stdlib=libstdc++ as well for clang. For g++ you only need the -std=C++11.

[paradoxical]

It seems to me the ++ versions are slower to decode/create vobsubs though. I measured the time it took to create a resized to 1280x720 vobsub using ++ and java and it's about 25% slower on the ++ ones. I'm unsure if it's the language or the fixes present in the ++ versions require more decoding time.

To add further to what I said above in response to this:

With just one tiny fix after function profiling, I got 1080p BDSup with 959 subpics which were downsized PAL for vobsub ouput using Bilinear scaling which originally ran at around ~27 seconds to completion on my quad-core Xeon test machine to ~4 seconds. Doing a similar thing was around 3-4 seconds using the Java version. And I know there are plenty more simple optimizations I can do due to lots of little inefficiencies I'm seeing in Callgrind that add up.

Also, I'm plugging the last few memory leaks in the code as well. For example, BD Sup parsing was still pretty leaky and I got them all plugged up. :)

[paradoxical]

And to update again: Same path as in the previous post is now at around ~1.2 seconds. With no esoteric optimization, just optimizing calls to expensive functions and now the entire operation takes less than 5% of the original time. :)

[Selur]

Nice speed boost. :)

[paradoxical]

Yeah. Going to run through more code paths. It's also why I didn't post a binary for you yet. I want to finish the last bit of memory and function profiling today.

[Selur]

I want to finish the last bit of memory and function profiling today.
keeping my fingers crossed :)

[paradoxical]

:) Doing this profiling is rather eyeopening about how expensive so many functions in Qt can be to call repeatedly that you wouldn't necessarily expect. For example, calls on QImages for their height/width/bytesPerLine/etc which one would expect to be pretty quick can add up to being 50% or the instructions executed, talking a cost of 10s of millions, if the value is retrieved over and over through the accessor over just caching it once since they can get called half a million or more times due to all the loops.

[paradoxical]

Getting real close to posting an updated version. Should be ready by tomorrow. Patch set for merging will be pretty extensive, but the improvements to memory usage and speed are substantial. Which is also helping to run more memory profiling sessions in Valgrind due to the increased speed.

[Selur]

Nice! Hopefully Sassbot will apply the patch also in a timely fashion.

[paradoxical]

Yeah, I've been talking to him off and on through email and he should be able to merge them within a day or so of being sent the patches.

[Selur]

Nice! :D

[Chetwood]

BTW, is there a fast way to determine which of the 150+ VobSubs of a certain dir has some items flagged as forced?

[paradoxical]

Newly compiled version with all the performance and memory leak fixes is *link removed*. I've done quite a few test runs to check for bugs and haven't seen any, but please check in case I've missed anything. Patch diff for anyone who wants to compile their own is *link removed*. Should be merged by Sassbot in a day or so.

[paradoxical]

So of course something slipped through. Seems I found an integer overflow happened in the BDN+XML code which I haven't ever seen before with what was supposed to be a "long" integer. So I've now made all "long" into unsigned 64-bit integers to brute force solve that. :) Fixed executable is *link removed* and the new patch diff which contains both the previous fixes and this integer change is *link removed*.

[paradoxical]

Sassbot just emailed me and said he merged everything from the patches.

[Selur]

@paradoxical: Thanks + don't forget the Mac OS X binary. :)

[Music Fan]

@ paradoxical : have you managed to convert DVB-SUB (in TS) to sup ?

[paradoxical]

It's in progress but was sidetracked temporarily by trying to fix up the code in other areas which is what gave the speed/memory leak fixes. I'll let you know when I have something ready for you to test.

[Music Fan]

Thanks, take your time, it's not urgent ;)

[r0lZ]

A little improvement, if it's possible: can you try to suppress the irritating console window that pops up then the program is launched (including when it is launched from another program with command line arguments)?

[paradoxical]

@paradoxical: Thanks + don't forget the Mac OS X binary. :)

Doing so now. I realized I had to recompile my Qt to be statically linkable.

[Selur]

I realized I had to recompile my Qt to be statically linkable.
yup :)

[paradoxical]

A little improvement, if it's possible: can you try to suppress the irritating console window that pops up then the program is launched (including when it is launched from another program with command line arguments)?

There's nothing that can be done. It's the side effect of having a Qt app on Windows that can be both GUI and CLI. The only alternative is separate apps which is a pain in the ass. Other OSes have no such issue. If you want a fix contact the Qt people.

[Selur]

including when it is launched from another program with command line arguments
that is a problem of the other application (I know this can be suppressed, since bdsup2sub++ doesn't 'popup' in Hybrid)

[paradoxical]

yup :)

Well it's that I thought I had compiled it with the static option, until I checked.

[paradoxical]

Ok try *link removed*, Selur.

[Selur]

Ok try this, Selur.
-> only throws a 'Illegal instruction: 4' on Mac OS X 10.7.5 Lion :(

[r0lZ]

There's nothing that can be done. It's the side effect of having a Qt app on Windows that can be both GUI and CLI. The only alternative is separate apps which is a pain in the ass. Other OSes have no such issue. If you want a fix contact the Qt people.
that is a problem of the other application (I know this can be suppressed, since bdsup2sub++ doesn't 'popup' in Hybrid)
OK, thanks. I'll try to redirect stdin, stdout and stderr. I suppose the QT console is there to connect them to something, right?

[Selur]

I suppose the QT console is there to connect them to something, right?
I only connect to stdout and stderr of bdsup2sub++ (in Qt bdsup2sub++ doesn't popup if you simply start a process with it instead of starting a detached process).

[r0lZ]

OK, I will try. Thanks again.

[paradoxical]

-> only throws a 'Illegal instruction: 4' on Mac OS X 10.7.5 Lion :(

Hmmm... Okay, let me check my settings for compile. Make sure things weren't set at minimum 10.8.

Edit: I think I know the prob. I have the latest Mac Mini which means it compiled with support for all SIMD stuff through AVX which is more likely the issue. I guess I'll limit my Qt to only SSE3 which should work for a minimum Core2Duo.

[paradoxical]

Selur, try this *link removed*. Compiled min 10.7 with nothing beyond SSE3.

[digitall.h]

Hi Sassbot, hi @all.
I'm trying to compile BDSup2Sub++ in Linux Mint KDE 13.
I have all build packages installed, and also installed libqxt 0.6.1 (this is the version in repositories).
I downloaded source from https://github.com/amichaelt/BDSup2SubPlusPlus (here)
When I qmake in terminal I get this error:
WARNING: /usr/share/qt4/mkspecs/features/qxt.prf:7: Variable QMAKE_RPATH is deprecated; use QMAKE_LFLAGS_RPATH instead.


When I then make in terminal, it throws a long lost of errors (I may paste if you think it's necessary) and exits with error 1, and compiling nothing.

What am I doing wrong?, wrong tools version?. Do you need further information to help me compiling?.
:confused:

[paradoxical]

That's not really an error. But, sure if you can post the line you're using to compile and the part where the errors start I can try to help you. Also the version of your compiler is needed too. You need to make sure to be using either Clang 4.x or GCC 4.7.x or higher.

Edit to add: Your version of Qt also needs to be compiled with C++11 support since this code uses initializer lists for Qt containers.

[digitall.h]

Thank you paradoxical.
Probably I decided to try a task too big for me.
I don't almost compile anything by myself. I install programs and tools from repositories.
Please tell me how can I check my Clang and GCC versions.
In terminal Clang -h tells me that Clang is not installed (probably that's a problem to begin with). And gcc -h throws an error about missing files (this might not be the way to check gcc version :( )
My compiling code in terminal is just qmake BDSup2Sub++.pro, and after the warning message make.

I also did not compile Qt myself, it is as it was in the repositories, don't know if it was compiled with C++11 support. Also don't know how to check it...

Do you think I will manage to compile it in my system?. I looked for a Linux (Debian) binary with no success...
:thanks:

[paradoxical]

To find your version type "g++ --version". If you didn't compile Qt yourself it probably isn't compiled with C++11 support. I guess I'll change the pro file so it verifies you have C++11 support and then just errors out so compiling never happens.

[digitall.h]

Aaarrrggghhhh,
GCC version 4.6.6

I'm afraid I won't be able to compile BDSup2Sub++ on my system.
And I don't have the time/skill to build in a virtual machine a Linux version just for compiling purposes.

Is it possible anybody compiled it and can provide binaries for Linux 64bit?.
:)

[paradoxical]

You can always ask someone in Mint to put it in their repos.

[paradoxical]

So I now have collaborator access to Sassbot's repo so I can push changes directly. I've fixed the pro file so that it first checks that your Qt libraries are configured with C++11 support and give you an error message and fail out if not.

[digitall.h]

I suppose it's not possible to make compilation compatible with other configuration, specifically with previous GCC versions, no C++11 compatibility and so, isn't it?
:(
I don't know if common 'buntu users will have this specifications here in Linux world.
:)

[paradoxical]

I suppose it's not possible to make compilation compatible with other configuration, specifically with previous GCC versions, no C++11 compatibility and so, isn't it?
:(

Not without a ton of ifdefs and added code that I don't really want to do. C++11 is the future and Qt5 compiles with C++11 support by default. bdsup2sub++ also compiles with Qt5 just fine. It's been ifdefed to account for the header movement.

I don't know if common 'buntu users will have this specifications here in Linux world.
:)

GCC 4.7.2 is the default g++ in Ubuntu 12.10. So it will get more common. There are also ppas for GCC 4.7.x for both Lucid and Precise here: https://launchpad.net/~ubuntu-toolchain-r/+archive/test so the toolchain can be pulled down. I don't really run Linux outside of doing some Valgrind testing on it in a VM, so I'm not really all that interested in doing ppas for Ubuntu myself. Windows and OS X are my main systems. Like I said, I suggest you ask someone who runs your repos if they can make a package for you.

[paradoxical]

So I'm refreshing the builds bumping the version number to 1.0.1 since I believe all my changes warrant so and I've created a tag to match the current builds. Selur *link removed* the OS X binary. Posting the new Windows binary with bumped version number shortly.

Edited to add:

Here (https://www.dropbox.com/s/6429efffiwxlj7u/bdsup2sub%2B%2B.7z) is the new Windows binary.

Edited to add again:

I'm going to fix the OS X binary I posted. The Qt it is statically linked with apparently was using my system's libjpeg instead of the built-in Qt libjpeg. If anyone already downloaded the previous binary and wants to use it, you will need to know you need to install libjpeg.8.dylib into /opt/local/lib/. Otherwise, just wait until I fix it in about 4 hours.

[Guest]

The discussion for this tool is moving to:

http://forum.doom9.org/showthread.php?p=1613303