BDSup2Sub++ - convert and tweak bitmap subtitle streams (VobSub,BD-SUP,BDN XML,HD-SUP

[Selur]

Sure, it's a simple fix. I'll add that into the patch I make and will add that to the binary I make.
Nice!! Thanks!

[r0lZ]

btw. while at it, it would be cool if the progress indication could be send to std::cerr instead of std::cout in command line mode. (std::cout get's delayed and this way monitoring the progress by watching the console output is a pain, since the progress only happens in large chunks,..)
Sorry, but I disagree. stdout is the correct handle for that kind of information. I use BDSup2Sup++ with a GUI that considers that an error has occurred if something is printed to stderr. Furthermore, it gets the whole content of stderr at the end of the process only. Of course, it is impossible to handle a progressbar that way. (stdout should not be delayed if open the handle in line mode.)

So, IMO, if you really want that, it should be an option of the command line, and still use stdout by default, for compatibility reasons.

[paradoxical]

Okay, I'll add a commandline option for it which defaults to the original behavior. Sounds like a good compromise.

[Selur]

If in GUI mode, it should not write anything to std.
Adding an additional option to get the progress on std::cerr to allow monitoring the progress with another tool/gui would be fine too. :)
(It's just bugging me that atm. progress indication like it is done atm. through CLI is not really useable,..)

[paradoxical]

It doesn't write anything out to a standard stream in GUI mode. It writes everything to the log window.

[r0lZ]

Okay, I'll add a commandline option for it which defaults to the original behavior. Sounds like a good compromise.
Thanks.

[paradoxical]

So I added the commandline option "log-to-stderr" otherwise it defaults to the current behavior. New binary can be grabbed from *removed to recompile*.

[Selur]

Thanks! :)
> The program can't start because libwinpthread-1.dll is missing from your computer. Try reinstalling the program to fix this problem.
-> looks like you compiled it dynamically.

[paradoxical]

No, I just didn't add the lflags in correctly. Recompiling now.

[Selur]

"Recompiling now. " -> thanks :)

[paradoxical]

Try *link removed*.

[Selur]

doesn't output anything and just crashed on a .sup file (send you a link via pm)

Cu Selur

[paradoxical]

Ok, found the error and now your commandline works fine for me Selur. I've notified SassBot on github again with this change. Until he notices and pushes this change *linked updated below* is a patch file for anyone compiling their own. *linked updated below* is another compiled version.

Note this error only effected if you used the logging to stderr.

[Selur]

output seems to be working fine now,..
call is still crashing for me though,... outputs '#> 743 (00:58:59.954)' and then no output any more and a crash. :/ (btw. I'm on Win7pro 64bit)
-> Here comes the kicker: I just tried and renamed my files and called:
bdsup2sub++ --resolution pal --log-to-stderr --output "H:\Temp\test.sub" "H:\Output\test.sup"
-> works
bdsup2sub++ --resolution pal --log-to-stderr --output "H:\Temp\test.sub" "H:\Output\Trying This (2013)_id_3_en_default.sup"
-> works
bdsup2sub++ --resolution pal --log-to-stderr --output "H:\Temp\test.sub" "H:\Output\and This (2013)_id_3_en_default.sup"
-> crashes
bdsup2sub++ --resolution pal --log-to-stderr --output "H:\Temp\test.sub" "H:\Output\Now This (2013)_id_3_en_default.sup"
-> crashes
(I can repeat this, seems like as soon as the starting part of the output name is short I get a crash,..)
-> something is fishy, seems like theres at something of with the parsing of the input file name,... (or my system is going crazy)

[paradoxical]

output seems to be working fine now,..
call is still crashing for me though,... outputs '#> 743 (00:58:59.954)' and then no output any more and a crash. :/ (btw. I'm on Win7pro 64bit)
-> Here comes the kicker: I just tried and renamed my files and called:
bdsup2sub++ --resolution pal --log-to-stderr --output "H:\Temp\test.sub" "H:\Output\test.sup"
-> works
bdsup2sub++ --resolution pal --log-to-stderr --output "H:\Temp\test.sub" "H:\Output\Trying This (2013)_id_3_en_default.sup"
-> works
bdsup2sub++ --resolution pal --log-to-stderr --output "H:\Temp\test.sub" "H:\Output\and This (2013)_id_3_en_default.sup"
-> crashes
bdsup2sub++ --resolution pal --log-to-stderr --output "H:\Temp\test.sub" "H:\Output\Now This (2013)_id_3_en_default.sup"
-> crashes
(I can repeat this, seems like as soon as the starting part of the output name is short I get a crash,..)

Can't repeat it even using the file names you said crashed. It always finishes successfully. Sorry. :(

-> something is fishy, seems like theres at something of with the parsing of the input file name,... (or my system is going crazy)

The input file name comes from the QxtCommandOptions parser. And it's coming out correctly for all those file names you specify. The application itself isn't parsing argv.

[Selur]

Okay, will do some more testing here tomorrow.
Thanks for trying to repeat the problem. :)

Cu Selur

[paradoxical]

Actually, I was finally able to do so! And I think I have a fix for it. Will post a new executable when I make sure I got it fixed right.

Somehow I was able to change options enough to get a 0xbaadf00d hex value, according to Microsoft this is due to an uninitialised allocated heap memory when the debug heap is used, in a field that was being called on that caused a crash. No clue why it wasn't getting this value before or what I exactly did to expose it finally.

[Music Fan]

Don't you know if your program supports DVB-SUB ?

[paradoxical]

Did you read sneaker_gear's post? He quoted the list of file formats it supports.

[Music Fan]

Ok, so I forget your program.

[paradoxical]

Last version for today. I might work on other outstanding issues as I see from the github page that SassBot hasn't gotten to. For one thing, I added the functionality so that the arrow buttons work in the edit dialog in this version as an extra fix.

Here (https://www.dropbox.com/s/r4b0yk4hxw8v6rc/file.patch) are the current patches that I'm pushing to Sassbot. Here (https://www.dropbox.com/s/6429efffiwxlj7u/bdsup2sub%2B%2B.7z) is the newly compiled exe.

[paradoxical]

I can look into adding support for DVB Sub, but I don't promise anything.

[Music Fan]

Thanks, if you need a DVB-SUB file, I can send you this ;)

[paradoxical]

Sure, samples would be good.

[r0lZ]

I can look into adding support for DVB Sub, but I don't promise anything.Great idea!

[Music Fan]

Here is a dvb-sub sup file ;
http://www51.zippyshare.com/v/21056401/file.html
I extracted it from TS with TSDoctor's demuxer.
It's supposed to respect the norm ETSI EN 300 743.

[paradoxical]

Link doesn't seem to work.

[sl1pkn07]

Link doesn't seem to work.

working for me

https://dl.dropbox.com/u/6596386/DVB-SUB.rar

[TheShadowRunner]

Thanks again for your hard work paradoxical, I was finally able to convert the 2 problematic SUPs without having the application crash on me.

[Selur]

I agree thanks a lot, last version works fine here too. :)

[paradoxical]

So looking into the DVB subtitle format more. Does anyone know if it's really valid to extract a raw subtitle stream? From my reading of the specification it seems that by doing so you lose the PTS values as they come as data that precedes the subtitle stream PES packet data. And no tools that read DVB Subtitles seem to accept the raw sup stream you gave me, they all require the transport stream which seems to confirm that a file contain just the PES packets isn't really valid in itself. Even projectx converts the DVB subtitles to another format when extracting.

So, yes, I can add the support for DVB subtitles but it will require you to load the original transport stream in order to do the conversion rather than loading the raw stream of PES packets.

[mini-moose]

Here (https://www.dropbox.com/s/r4b0yk4hxw8v6rc/file.patch) are the current patches that I'm pushing to Sassbot. Here (https://www.dropbox.com/s/6429efffiwxlj7u/bdsup2sub%2B%2B.7z) is the newly compiled exe.

I just tried it with a sup sassbot's latest build was crashing on and this one managed to go through it fine :)
It seems to me the ++ versions are slower to decode/create vobsubs though. I measured the time it took to create a resized to 1280x720 vobsub using ++ and java and it's about 25% slower on the ++ ones. I'm unsure if it's the language or the fixes present in the ++ versions require more decoding time.

[Music Fan]

Even projectx converts the DVB subtitles to another format when extracting.
Actually, it lets the choice between a simple extraction (like TSDoctor) and a conversoin to sub/idx, which fails (with my extract, it works maybe in other cases).

So, yes, I can add the support for DVB subtitles but it will require you to load the original transport stream in order to do the conversion rather than loading the raw stream of PES packets.
Great.;)
Anyway, it seems to be the only solution.
Again, if you need a sample (subtitled TS), I can send you it.
Can you now open the zippyshare link I gave for sup 2 days ago ?
If no, I'll have to upload the TS (if you need it) on another server.

[paradoxical]

I just tried it with a sup sassbot's latest build was crashing on and this one managed to go through it fine :)
It seems to me the ++ versions are slower to decode/create vobsubs though. I measured the time it took to create a resized to 1280x720 vobsub using ++ and java and it's about 25% slower on the ++ ones. I'm unsure if it's the language or the fixes present in the ++ versions require more decoding time.

It's probably nothing more than a little optimization needed. Looking back through commit history I can't really see that most of the would really add any decoding time. I'm gonna be running it through Valgrind later to get some perf data.

Also, there are some architectural problems inherent in original the Java version that were carried over in the porting that need to be fixed so that creating/writing out subs can be properly multi-threaded which should give good speed improvements. Right now, both operations do things in serial.

Actually, it lets the choice between a simple extraction (like TSDoctor) and a conversoin to sub/idx, which fails (with my extract, it works maybe in other cases).

All I know is that what you gave me doesn't have any PTS values since TSDoctor seems to only extract the subtitle PES packets. Which makes it impossible to determine start times for the subtitles.


Great.;)
Anyway, it seems to be the only solution.
Again, if you need a sample (subtitled TS), I can send you it.
Can you now open the zippyshare link I gave for sup 2 days ago ?
If no, I'll have to upload the TS (if you need it) on another server.

I've found a few samples I'm working with now so not at the moment. I might need more in the future just to test the code more.

[Selur]

btw.
1. if anyone manages to build BDSup2Sub++ statically on mac os please send me a build. :)
The gcc 4.7 requirement kills me on mac. I get gcc 4.7 compiled and working, but I can't get Qt and libqxt statically compiled with it. :/
(static compilation works fine on linux, on windows I normally use the VS C++ express compiler and since even VC11 doesn't support C++11 'Non-static data member initializers' that's a no-go too :()
2. if some one has the time and motivation, it would be nice if he could fix the whole 'non-static data member initialization' in the header files (seems to be the only C++11 needed)

Cu Selur

[paradoxical]

You don't need GCC if you're on OS X. Just use clang 4.x which comes with recent XCode. If you really must have GCC, just get it from Macports. There's no need to compile it yourself.

2. if some one has the time and motivation, it would be nice if he could fix the whole 'non-static data member initialization' in the header files (seems to be the only C++11 needed)


More C++11 is used than that. Also, it's not really a "fix" since to change it back to old style C++ since involves code bloat and to maintain the C++11 features would require tons of #ifdefs. All up-to-date version of any decent compiler, MSVC obviously not included in this list since it still sucks at implementing features, has all the C++11 support needed to compile this for quite some time now.

In the end, I can compile a binary when I get home since I already have a Qt environment set up.

[Selur]

I will look into clang,...

In the end, I can compile a binary when I get home since I already have a Qt environment set up.
that would be really appreciated :)

[paradoxical]

Yeah, just grab latest Xcode. It's what I use to compile on OS X for my own testing. But, like I said, if you need GCC just grab it from Macports. There's no good reason to compile it yourself.

[Selur]

last time I installed macports it screwed up my normal build environment so I try to avoid that option. ;)

[paradoxical]

*shrug* I got multiple versions of GCC installed through Macports and Xcode, etc. installed and all is good. You must have just gotten unlucky.

[Selur]

clang doesn't like me either :/
In file included from ../src/bdsup2sub.cpp:22:
../src/Subtitles/subtitleprocessor.h:55:20: error: no matching constructor for initialization of 'QStringList'
static QStringList resolutionNamesXml = { "480i", "576i", "720p", "1440x1080", "1080p" };
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Trolltech/Qt-4.8.4/include/QtCore/qstringlist.h:69:12: note: candidate constructor not viable: requires 0 arguments, but 5 were provided
inline QStringList() { }
-> giving up for today, so would be nice if you later find the time to compile a static BDSup2Sub++ and upload it somewhere.

Cu Selur

[paradoxical]

You need Qt compiled with C++11 support to get the initializer list support. This is why I said there is much more C++11 use than just the non-static member initialization. :p

Yes, I still compile the binary for you.

[Selur]

[qutoe]You need Qt compiled with C++11 support to get the initializer list support.[/quote]
I recompiled QT with platform macx-clang. :/

[paradoxical]

Which isn't enough. You need to pass -std=C++11 so that it will compile the C++11 support in Qt.

[Selur]

Will try that tomorrow. :)

[paradoxical]

Also forgot, you need to pass -stdlib=libstdc++ as well for clang. For g++ you only need the -std=C++11.

[paradoxical]

It seems to me the ++ versions are slower to decode/create vobsubs though. I measured the time it took to create a resized to 1280x720 vobsub using ++ and java and it's about 25% slower on the ++ ones. I'm unsure if it's the language or the fixes present in the ++ versions require more decoding time.

To add further to what I said above in response to this:

With just one tiny fix after function profiling, I got 1080p BDSup with 959 subpics which were downsized PAL for vobsub ouput using Bilinear scaling which originally ran at around ~27 seconds to completion on my quad-core Xeon test machine to ~4 seconds. Doing a similar thing was around 3-4 seconds using the Java version. And I know there are plenty more simple optimizations I can do due to lots of little inefficiencies I'm seeing in Callgrind that add up.

Also, I'm plugging the last few memory leaks in the code as well. For example, BD Sup parsing was still pretty leaky and I got them all plugged up. :)

[paradoxical]

And to update again: Same path as in the previous post is now at around ~1.2 seconds. With no esoteric optimization, just optimizing calls to expensive functions and now the entire operation takes less than 5% of the original time. :)

[Selur]

Nice speed boost. :)

[paradoxical]

Yeah. Going to run through more code paths. It's also why I didn't post a binary for you yet. I want to finish the last bit of memory and function profiling today.