HD subtitle ripping (SUPread)

[Bluedeep]

Can you add extended ascii ocr?

tnx

[zgx]

Downloaded the latest version of SUPread and tried it out on the first HD DVD I could find. Did not work that well as you can see below:

00:02:23,942 --> 00:02:24,034
E·c·se·-·oi. C·es· bie· l0
···habi·e Del·a· Be··· ·

3
00:02:26,173 --> 00:02:26,275
ll es· ?as l0

4
00:02:29,306 --> 00:02:29,408
Re···e· ch··ie

5
00:02:36,615 --> 00:02:36,718
C·es· ·· a·i

6
00:02:40,292 --> 00:02:40,383
D·aille··s·
c·es· l·i ··i a ·c·i· l·ad·esse

7
00:02:46,789 --> 00:02:46,892
Vo·s ·eco··aisse· so· ·c·i···e

8
00:02:49,297 --> 00:02:49,399
J·ai e· d· ·al 0 ··o··e·


Used all the default settings. You can download the SUP file here: http://www.rogepost.com/n/4783213726

Keep up the good work!

[Pelican9]

Unfortunatelly, this font type and even the french letters are not supported yet.
You can improve the result if you decrease the level (e.g. 20).

[tjf]

I have already twice encountered problem where SUPread sometimes omits the end time of a subtitle.

This is example from The Good Shepherd:

4
00:00:55,188 --> 00:00:57,054
WOMAN: You are safe here
with me.

5
00:01:43,770 -->
NEWSCASTER ON RADIO:
Has a decision been reached
on how far this country

6
00:01:46,340 -->
would be willing to go

7
00:01:47,507 --> 00:01:51,535
in helping an anti-Castro
uprising or invasion in Cuba?

8
00:01:51,945 -->
PRESIDENT KENNEDY: Well,
first, l want to say that
there will not be,



I have uploaded the evodemuxed sup file here (http://rapidshare.com/files/29755120/FEATURE_1_FEATURE_2.8bitRLC.s00.English_SDH.sup.html).

Is this a bug in SUPread?

Tomas

[Pelican9]

I have already twice encountered problem where SUPread sometimes omits the end time of a subtitle.

...

I have uploaded the evodemuxed sup file here (http://rapidshare.com/files/29755120/FEATURE_1_FEATURE_2.8bitRLC.s00.English_SDH.sup.html).

Is this a bug in SUPread?

Tomas

It was a bug, but I've fixed it. Which version do you use?

[tjf]

It was a bug, but I've fixed it. Which version do you use?

I thought I have the latest (v0.31), but I see although it has the same version number it has different file size then the one in your signature. That one works without any problem.

Thanks.

[Pelican9]

OK.
Sorry for the inconvenient version numbering.

[manusse]

Hi,

You can try the new version of SubtitleCreator (2.2.2) that can import HD-SUP subtitles.

https://sourceforge.net/project/showfiles.php?group_id=120228&package_id=151984&release_id=505208

To test it, go to Tools->Manipulate SUP... and open your SUP file demuxed using EVODemux. The HD-SUP will be resized to SD and the color number lowered to 3 plus transparency. You can then process the subtitle like any SD subtitle, i.e. save it to .sup or VobSub (sub/idx), OCR it to SRT with SubRip, DVDSubEdit, SubtitleCreator,..., add it to an existing DVD with muxman, play it with VSFilter, resynch it with VobSub tools or SubtitleCreator, ...

Could you please test as many HD-SUP as possible and post the bugs you find in this thread or SubtitleCreator's sticky thread.

Cheers
Manusse

[Rectal Prolapse]

Great work! I can't wait to try it! Thanks!

[cwm9]

When you save the .png format files, can you please save them with the transparency preserved? Right now transparency is being thrown away :eek:

Transparency in .png can be easilly set with the tRNS chunk. Thanks!

[Pelican9]

Are you talking to Manusse or to me?

[cwm9]

To you, Pelican. =)

BTW, thanks so much for your work. It's gratefully appreciated.

I'm interested in recovering the original files and not so interested in OCRing them.

When I open the .sup, the background is grey showing where the image should be transparent, but when the images are saved to .png files the transparency is replaced with black.

Since black is used as a drawing color, it becomes impossible to tell the difference between black and transparent.

Could you please save the transparency information with the .png files? You simply choose an index color and set its alpha to 0 using the tRNS chunk type. (Or, if you are using a library, I imagine it has the ability to do this, too.) I imagine they are now using index 255 as transparent? If that's the case, simply set index 255 to transparent and don't convert it to black. You could also save the images as 24bit/16bit+alpha. I don't know how the alpha information is stored in the .sup, but I don't imagine this should be too hard?

Also, I noticed something strange with the dumped scn-sst files last night. I'm not 100% sure of this, but it looks like the time codes you are printing are drop-frame time codes. I say this because the non-drop video's audio track doesn't line up with the subtitles unless I interpret the subtitle times as drop-timecodes. However, if they are in fact drop-frame timecodes, they don't appear to follow the correct drop-frame rules. That is, the 0th and 1st frame should be skipped at the top of every minute, except on even 10 minute intervals. Perhaps, and this is just a far-fetched guess, the SUP encodes time as a timecode + a number of frames and maybe you don't add the frame number correctly if it's on a minute boundry? Just a guess. If this is what's going on, perhaps it would probably be a lot easier just to print out non-drop time codes?[/B]

I was able to make this part work just by interpreting the timestamps as drop-frame timecode and editing any time that fell on a forbidden frame forward by one frame if it was originally a .00 time or by two frames if it was originally a .01.

Thanks!

[Pelican9]

When I open the .sup, the background is grey showing where the image should be transparent, but when the images are saved to .png files the transparency is replaced with black.

Since black is used as a drawing color, it becomes impossible to tell the difference between black and transparent.

Could you please save the transparency information with the .png files? You simply choose an index color and set its alpha to 0 using the tRNS chunk type. (Or, if you are using a library, I imagine it has the ability to do this, too.) I imagine they are now using index 255 as transparent? If that's the case, simply set index 255 to transparent and don't convert it to black. You could also save the images as 24bit/16bit+alpha. I don't know how the alpha information is stored in the .sup, but I don't imagine this should be too hard?
What have I do with this chunk? I've never used PNG before... I've set the transparent color, the transparency mode but still nothing.

Also, I noticed something strange with the dumped scn-sst files last night. I'm not 100% sure of this, but it looks like the time codes you are printing are drop-frame time codes. I say this because the non-drop video's audio track doesn't line up with the subtitles unless I interpret the subtitle times as drop-timecodes. However, if they are in fact drop-frame timecodes, they don't appear to follow the correct drop-frame rules. That is, the 0th and 1st frame should be skipped at the top of every minute, except on even 10 minute intervals. Perhaps, and this is just a far-fetched guess, the SUP encodes time as a timecode + a number of frames and maybe you don't add the frame number correctly if it's on a minute boundry? Just a guess. If this is what's going on, perhaps it would probably be a lot easier just to print out non-drop time codes?[/B]

I was able to make this part work just by interpreting the timestamps as drop-frame timecode and editing any time that fell on a forbidden frame forward by one frame if it was originally a .00 time or by two frames if it was originally a .01.

Thanks!
The time information on the HD DVD and on the BD is a four-byte value (in 1/90000 seconds). So I know the correct time, but not know the right frame. If you describe the algorythm of the conversion from time (in hh:mm:ss.msec) to timestamp (hh:mm:ss:frame) I'll build in to SUPread. Now, I simply convert the milliseconds to frame based on 24 fps (divide by 1000 and multiple 24) so I think it isn't a drop-frame timecode. I don't know what the right format for the Scenarist, but you could help me.

[cwm9]

What have I do with this chunk? I've never used PNG before... I've set the transparent color, the transparency mode but still nothing.

If you're "raw writing" the .png, the chunk is used to apply a different alpha to each index (in index mode transparency.) If you are using a library, then it should do everything for you. If you already set the transparent color and transparency mode in the library, you can open it with Photoshop to see if the image is properly transparent. Can you let me know if you are using a library, and if so, which one? If you're not using a library, let me know that to...

The time information on the HD DVD and on the BD is a four-byte value (in 1/90000 seconds). So I know the correct time, but not know the right frame. If you describe the algorythm of the conversion from time (in hh:mm:ss.msec) to timestamp (hh:mm:ss:frame) I'll build in to SUPread. Now, I simply convert the milliseconds to frame based on 24 fps (divide by 1000 and multiple 24) so I think it isn't a drop-frame timecode. I don't know what the right format for the Scenarist, but you could help me.

It took me a bit of thinking to figure out the frame code thing, but I think I've got it. HDDVD plays back in 60i mode, but 60i is nominative and is actually 59.94i, or more familiarly, 29.97fps. The film is encoded at 23.976 and uses pulldown flags to convert from 23.976->59.94i/29.97fps.

The correct way to convert frames to timecode at 23.976 would be to convert the whole seconds to H:M:S and then multiply the fractional party by 23.976 to get the frame number, etc, etc, but this is a pain in the a$$ because the fractional time of each frame is different from every other, so instead NTSC uses the nominative value of 30fps. Historically, this made editing much easier. Thus in scenarist, if you select non-drop, you don't get 29.97fps non-drop timecodes, you get _30fps_ non-drop timecodes, even though at playback you will really get 29.97fps. That is, the timecodes are running too slowly. (the higher the fps, the more frames it takes to count off a minute -- so for the same nubmer of frames, the timecode is smaller so the timecode is thus running more slowly. Higher fps = slower timecode.)

In the world of NTSC, to convert 60i nominative timestamps to real-world timestamps you have to use the "drop frame" timecodes. The subtitles are written in absolute real-world time, not in the nominative timecode stamps, and that's why the time stamps "appear to be drop-coded" when written out as timecodes. In reality, they're absolute times which are neither drop nor non-drop.

In order to convert from absolute time to NTSC timestamps you must convert either to a) nominative 60i timestamps or b) nominative 60i drop-frame timestamps. But then, only NTSC behaves this way; pal does not.

This is a mess created by the NTSC drop-frame scheme, and I really don't see an easy way out. You should probably give the option to write out in several formats depending on the needs of the user.

let time=the time encoded in the subtitle in seconds.

a) absolute time (not importable, but useful to some people probably). Simply print 'time'. Useful for those who will want to convert to a completely different format at potentially a different framerate.

b) at a user selectable non-drop framerate. Convert floor(time) to H:M:S. The frame number is then floor(fps*frac(time)+.5). (The +.5 is to make sure rounding errors don't cause the wrong frame number to be calculated.) Useful for those who want to convert to DIVX and for PAL material.

c) at NTSC nominative 60i/30 non-drop. Multiply each time by (29.97/30) to get to the nominative time. Convert floor(time*29.97/30) to to H:M:S. The frame number is then floor(30*frac(time*29.97/30)+.5). Useful for those who need non-drop 60i format for use with zoom player, etc. or for re-authoring.

d) at NTSC nominative 60i/30 drop. Convert floor(time) to to H:M:S. The frame number is then floor(30*frac(time)+.5). Add 2 frames to the frame number IF the sec in H:M:S = 0 UNLESS M MOD 10 is 0 (M is a multiple of 10.) In other words, add 2 frames if you are at an even minute, except for every 10 minutes. (You don't drop the first two frames, as 0 minutes is considered a multiple of 10.) Double check that no frame number is over 29, and if it is, set it equal to frame 29 and print a warning "some frame numbers ended up being higher than 29. They were reset to frame 29. The original was probably not encoded as drop-time, but this will result in an error of no more than 2 frames, once per minute." (This will only open if the original absolute times we not derived from drop-frame codes). Probably only useful for re-authoring.

If you are only going to offer two methods, I'd offer method B and C as this should cover 99% of everyones needs.

In order of importance, i'd say C/B, A, then D. B is critical for PAL, C is critical for NTSC.

Thanks so much for looking into this stuff!

[Pelican9]

I'm using the PNG Image library (by Gustavo Daud).

Which method (a,b,c,d) accepted by Scenarist?


You can edit your post (delete the dupe).

[homerpez]

I hope that SubtitleCreator program can allow me to do some hard-coded subs in VirtualDub... I was watching "Fearless" noticed that the English subs over the Mandarin Dialog were real subs, and I wondered if anyone had figured out how to do this yet...

[manusse]

hope that SubtitleCreator program can allow me to do some hard-coded subs in VirtualDub

SubtitleCreator can convert HD-SUP or srt files to sup or VobSub which can be displayed in VirtualDub using one of the appropriate AviSynth plug-in.

AviSynth sup plugin by emmel: http://forum.doom9.org/showthread.php?p=916838#post916838

AviSynth VobSub plugin: VSFilter http://www.videohelp.com/tools/VSFilter_DirectVobSub

Cheers
Manusse

[cwm9]

Sorry for the slow reply, I've been away for a few days.


You can edit your post (delete the dupe).

Huh, not sure how the dupe got in there.


I'm using the PNG Image library (by Gustavo Daud).




If you're using the library at http://pngdelphi.sourceforge.net/ then I would think the bit transparency mode would be the correct mode... (please forgive me, I'm not familiar with Delphi syntax).

To change to bit transparency mode, simply set a transparent color using the TransparentColor property:

property TransparentColor: TColor;
property TransparentColor: ColorRef;


This example would make the color clBlack be the transparent color (which is probably NOT what you want to do!):

png.TransparentColor := clBlack;


To verify you are in the correct transparency mode, check the read-only property TransparencyMode:

type TPNGTransparencyMode = (ptmNone, ptmBit, ptmTranslucid);
property TransparencyMode: TPNGTransparencyMode;

It should change to ptmBit after you have set the TransparentCOlor.


Which method (a,b,c,d) accepted by Scenarist?


method c is accept as the non-drop format for NTSC in scenarist
method b would be accepted as the PAL format in scenarist
(method d is accept as the drop format for NTSC in scenarist)

There is no one format that will work for both NTSC and PAL because NTSC uses a nominal FPS whereas PAL uses a real FPS. Methods c and b are probably the most portable.

In NTSC, the film is played back at 29.97 FPS but all the timestamps are written AS IF they were going to be played back at 30fps. That's because originally a "second" in "ntsc time" was 30 frames, even though 30 frames * 1/29.97 frames/second does not equal one second exactly. To make NTSC times look like real time, they "dropped" two time codes every minute. (They only drop the time codes, not the actual frames, so it's poorly named!) Scenarist wants NTSC time for NTSC video, but real time for PAL video, so there is no way of getting around needing two formats, which is why there is always so much confusion about timecodes.

[The_Keymaker]

I used EVOdemux to create a .sup file and here is a fragment of the resulting file:

1
05:22:50,773 --> 05:22:54,266
How did those soldiers ever manage
to dig out these tunnels?

2
05:22:55,144 --> 05:22:56,544
Let's get a rotation in here.

3
05:22:56,712 --> 05:22:58,612
Hey, I found something!

4
05:23:13,629 --> 05:23:14,721
Hanako...

Any idea why the times are all wrong? I read earlier in this thread about system time, but my clock is set correctly to my time zone in 12hour format.

[Pelican9]

I used EVOdemux to create a .sup file and here is a fragment of the resulting file:

1
05:22:50,773 --> 05:22:54,266
How did those soldiers ever manage
to dig out these tunnels?

2
05:22:55,144 --> 05:22:56,544
Let's get a rotation in here.

3
05:22:56,712 --> 05:22:58,612
Hey, I found something!

4
05:23:13,629 --> 05:23:14,721
Hanako...

Any idea why the times are all wrong? I read earlier in this thread about system time, but my clock is set correctly to my time zone in 12hour format.

If you share the .sup I will check it.
Or sharing the first 300MB of the EVO would be better.

[The_Keymaker]

Hello Pelican9.

The .sup file is about 13 megabytes (too large for an attachement). Is there a way I can post it?

By the way, thanks for all your work with EVOdemux and SUPread!

The_Keymaker

[Pelican9]

The .sup file is about 13 megabytes (too large for an attachement). Is there a way I can post it?

Use sendspace.com

[The_Keymaker]

Pelican9, here is the link to the SUP file:

http://www.sendspace.com/file/dnsuft

[Pelican9]

Pelican9, here is the link to the SUP file:

http://www.sendspace.com/file/dnsuft

This .sup contains very big PTS numbers, SUPread converts them to times correctly.
(First PTS in .sup: 67E9B567 -> 05:22:50,773)
The questions is why those big PTS values?
Copy the status screen of EVOdemux after reading (check detailed info in options).

[The_Keymaker]

Pelican9,

I fixed the .srt file by adjusting the time stamps using SubTitleCreator (Unfortunately, it crashed on me aftrrwards and it will not re-install!).

Anyway, I'll demux the EVOs and send you the status details later this weekend.

The_Keymaker

[PurpleMan]

@Pelican9:

Don't know if it's just me, but I couldn't import SUPread's .SST into Scenarist ACA. It complained about it not matching the project settings. Only when I changed TV_Type HD60 to TV_Type NTSC it accepted and imported.

Also, confirming the problem with the transparcy that was raised. Indeed the transparacy information in the PNG is needed or after authoring each subpictures covers the ENTIRE video image.

and P.S. - Thanks for your hard and excellent work!

[hm10]

what with the new version that we will can change srt to hd-sup and than add the subs to the blu-ray?

[Pelican9]

The new beta can produce transparent PNGs.
The Scenarist export improved also.

[Rectal Prolapse]

Thanks!

[hm10]

The new beta can produce transparent PNGs.
The Scenarist export improved also.

you gonna release the beta at least?

[Pelican9]

You can download the latest one as usually.

[vvetma]

Hello...

Is it possible with SUPread to convert SRT subtitle into HD-DVD SUP file???

When I open my SRT file and choose EXPORT mode and then click Export SUP nothing happens...

In Status Window there is a line

Exporting subtitle stream: subtitle.HDDVD.23.976.sup
Done.

but that is it... no file no nothing...

[hm10]

Hello...

Is it possible with SUPread to convert SRT subtitle into HD-DVD SUP file???

When I open my SRT file and choose EXPORT mode and then click Export SUP nothing happens...

In Status Window there is a line

Exporting subtitle stream: subtitle.HDDVD.23.976.sup
Done.

but that is it... no file no nothing...


same problem =\
in version 0.45b only when chose blu-ray in export this worked and the hd-dvd didn't work, but the sup didn't work in the tsremux

[Pelican9]

Yes, this is a beta, but I'm working on it (actually the BD sup stream export). I've share this beta because it contains important bugfixes.

Unfortunately, I'm running out of time, I'll go to travelling for at least 7 months (start at next week) and I don't know when, where and how can I continue the work with these tools.
But I try my best in this short available time.

[hm10]

Yes, this is a beta, but I'm working on it (actually the BD sup stream export). I've share this beta because it contains important bugfixes.

Unfortunately, I'm running out of time, I'll go to travelling for at least 7 months (start at next week) and I don't know when, where and how can I continue the work with these tools.
But I try my best in this short available time.

Oh S-h-i-T =\
i'm hope you'll finish it in this week, and if not plz release the source code and info about the hd-sup code, maby my brother will continue it and maby another someone will continue it...

[d0ORk]

Would be cool if it could support german "Umlaute"? like ä ö ü ß `?
:)

[Rectal Prolapse]

I wouldn't mind helping with the source code too. :)

Anyways, thanks for Supread and have a good vacation!

[zeropc]

question: i wanna convert a .sub to a .srt file but it seams i have to go through all 1700 + substitles by hand and fix things. is there any way to speed things up, because this would truly a pain.

[Pelican9]

zeropc:
Do you think I can help you if you don't give me any details of your problem?
SUPread supports only one or two font types. If your movie use another type, you cannot convert the .sup (not .sub) file to .srt automatically, but it's still easier than watch the movie and type the subtitle lines and guess the start and end times. :)

[zeropc]

Pelican9, my problem is this that i mostly get either on dots or words that make sense.

example...

is:
Oh, hi.Yeah, l l··ow.
They··lled me dow· here.


should:
Oh, hi. Yeah, i KNow.
They called me down here.

is:
-·······
-······u··· ·O·
······ ····

should:
- [ Applause ]
- [ Man ] I guess that wraps it up.
Good work, everybody.

you see now what i mean? and this goes over 1700 pages.

if you can give any advice to speed things up, i really apprichiate it :)

[Pelican9]

Share the .sup file.

[zeropc]

there you go pelican :)

http://rapidshare.com/files/49443575/suptitles.rar.html

[Pelican9]

This font type is not supported. :(
It has strange characters like 't' which is smaller than 'i', 'g' is look like a typewriter letter, 'J' is huge, etc.

[musky5789]

Hi all,
i demuxed a sup file from a evo file, and i have run supread to get the scr-sst file that you can import into scenarist 4.2.
7524

when i build the project i get this error:
internal software error:.\core\AUs\Advmux_timeGrip.ccp. line 94 -2
Advmuxmux::TimeGrid::addgrippointfill -- time not on field boundary.
7523

when i take out the subtitles and build the project it will work fine, can someone tell me what im doing wrong???

[woah!]

is your project set to NTSC/HD60 in scenrist

[musky5789]

yeah its set to ntsc/hd60

[killa_kid]

i want to say thanks for a fantastic program!

i do have 1 suggestion though. Is it possible to make the OCR similar to SubRip? Meaning it asks us what each letter is (with your initial test as a suggestion) allowing us to fix things, such as movies with I & l (lower case L) being very similar.

The program overall does a great job, but there are corrections needed throughout since the letters are just too similar.

Thanks!

[Taktaal]

I made my own program to OCR text since I dislike having bitmap subtitles, SUPread doesn't do it very well, and SubRip has been getting quite old.

It works pretty well, and allows you to correct letters the same way SubRip does, but is compatible with HD titles.
Also currently it needs you to convert the SUP file to SCN-SST with SUPread first, until I figure out the SUP format itself which seems to be completely different from the one SD-DVDs used...

http://x0r.ch/suprip/

[manusse]

@Taktaal,

until I figure out the SUP format itself which seems to be completely different from the one SD-DVDs used

You can have a look at SubtitleCreator's source code because it includes HD-DVD decoding features.

Cheers
Manusse

[killa_kid]

Taktaal

The program sounds fantastic, the only problem is that the program crashes everytime i open in. It says something about system.io.directorynotfound in the error. Do you have any idea why this would be happening?