Tech specs for ripping Closed Captions

[McPoodle]

I can't find any tools for extracting Closed Captions from DVD's short of playing the movie through GraphEdit, and I lack the programming know-how to write one myself, but I have worked out where the captions are hiding on a DVD, so if anyone is willing to help, you will make a small group of users very happy. Unfortunately, it appears that the method required would be completely different from the current subpicture extraction procedure.

The captions are stored in the Video Packs of the VOB files, in particular the GOP Video Pack. The caption immediately follows the GOP video stream and is signaled by the stream start code (hex 00 00 01), followed by 0xb2 (identified as a "user data" sub-stream in MPUCoder's MPEG Headers Quick Reference (http://members.aol.com/mpucoder/DVD/mpeghdrs.html). I'm not sure if Closed Captions are the only type of user data used in DVD's, but if it helps to identify it further, the two bytes immediately following the 00 00 01 b2 sequence are a pair of 43's.

To extract closed captions from a VOB, it appears that all that is necessary is to find the sequence 00 00 01 b2 43 within a GOP Video Pack, extract the raw data to the next start sequence of 00 00 01, move on to the next GOP Video Pack, and repeat. The data that GraphEdit extracts includes the user data sub-stream start sequences.

--McPoodle

Note # 1: I checked out SmartRipper and vStrip, and it appears that neither of these applications will extract a sub-stream from within a video stream.

Note # 2: In case anyone is interested but lacks an NTSC DVD with captions to work with, I have placed a 10-second DVD I made in Scenarist with captions at http://www.geocities.com/mcpoodle43/cc_dvd.zip. At the root of this zip file you will find cc.bin, the raw caption file I extracted with GraphEdit (there's also cc.scc, the captions in SCC format, and cc.ccd, the captions in a readable format). The output of any caption ripping program run on this DVD should be byte-for-byte identical to cc.bin. Hint: using VobEdit, I can see the first section of caption data resides at bytes 0060 to 00c2 of LBA 000001. The second section is at bytes 0060 to 00c2 of LBA 000088, the third is at bytes 0060 to 00c2 of LBA 00181, the fourth at bytes 0060 to 00c2 of LBA 000274, and the fifth at bytes 0060 to 00c2 of LBA 000367.

[gabest]

Well, these were already known to me, but how to decode the cc data?

[McPoodle]

Each User Data substream has the following structure:

00 00 01 b2: user-data substream header

43 43 01 f8: I'm guessing this identifies the user-data as closed captions for Field 1 (Field 2 is never used by DVD's). In any case, this data is constant.

8a: This byte is different for each substream. If you strip off the three high bits and the one low bit, you get 05--this is how many closed caption byte pair patterns are located in this substream.

ff xx xx fe 00 00: This is the pattern for each closed caption byte pair, with xx xx being the only data out of the stream we're interested in. Occationally, the last pattern in a substream will lack the "fe 00 00". Since this is the only place "ff" occurs in the closed caption substream, you can ignore the counter byte above and simply grab the pair of bytes following each "ff" in the substream.

To convert this data into an SCC file, create a text file with "Scenarist_SCC V1.0" as the first line and a blank second line.
The third and subsequent lines begin with the timecode of the first byte pair. The byte pairs are then transcribed into text, with spaces between pairs. The byte pair "8080" is a spacer, so normally you end the line when you get a sequence of these pairs, count to the next non-"8080" pair, and start a new line. For counting, each pair takes up a frame (at 29.97 non-drop framerate).

On the other hand, if you want to translate this into something readable, check out http://www.geocities.com/mcpoodle43/DVDMaestro/scc_format.html. Basically, you look for the byte pair 9420 (usually appearing twice in a row), followed by a code for the location of the caption, followed by the caption itself, followed by 942c 942c 942f 942f, which will cause the caption to be displayed (the exact start time is the first 942f). After a number of 8080's, the end time is the next time 942c is encountered.

P.S. I found that Lucida Console 16 pt white text on a black background most-closely resembles closed captions, at least on my television set.

P.P.S. VCD 2.0 uses binary closed caption files to include captions. The exact format is much simpler than the DVD format: Start with hexidecimal ff ff ff ff, followed by the byte pairs (including all of the 8080's).

[zuggy]

Originally posted by McPoodle
Field 2 is never used by DVD's

So why have Field 2 it's info bit in IFO?

[McPoodle]

The broadcast CC standard allows use of both fields (to allow for bi-lingual captioning), but outside of PBS, I've never heard of Field 2 being used by any broadcast channel, and as for DVD's, I doubt there are any software players out there that can support it. Nevertheless, the DVD specification was probably built to support Field 2 captions. Field 2 is also where the V-Chip ratings hide, but DVD's parental controls make this information superfluous.

Just to complicate things, all of the closed caption control codes have different values in Field 2: the caption begins with 1c20 1c20 and ends with 1c2c 1c2c 1c2f 1c2f, with the caption displaying at the first 1c2f and disappearing with the next 1c2c. For a complete list of codes for both fields, look here (http://www.geocities.com/mcpoodle43/SCC_TOOLS/DOCS/CC_CODES.HTML), while you can get the complete list of characters (including those with different values in Field 2) here (http://www.geocities.com/mcpoodle43/SCC_TOOLS/DOCS/CC_CHARS.HTML) (ignore all the "CCD" stuff--it's for a program I wrote to translate SCC files into a readable format).

[gabest]

ff xx xx fe 00 00: This is the pattern for each closed caption byte pair, with xx xx being the only data out of the stream we're interested in. Occationally, the last pattern in a substream will lack the "fe 00 00". Since this is the only place "ff" occurs in the closed caption substream, you can ignore the counter byte above and simply grab the pair of bytes following each "ff" in the substream.
Right the first dvd I tried has half of its text in place of the double zeros and it's beginning with 0xff also:
.. .. .. .. .. .. .. .. „_ .. .. .. .. .. 00 00 01
B2 43 43 01 F8 1E FF 80 „_ 80 FF 94 20 FF 80 80 FF
94 20 FF 80 80 FF 94 6E „_ FF 80 80 FF 94 6E FF 80
80 FF 5B 20 FF 80 80 FF „_ D3 EF FF 80 80 FF 6E 61
FF 80 80 FF F2 20 FF 80 „_ 80 FF D0 E9 FF 80 80 FF
6E 67 FF 80 80 FF E9 6E „_ FF 80 80 FF 67 20 FF 80
80 FF 5D 80 FF 80 80 FF „_ 94 2C FF 80 80 FF 94 2Cp.s. Why all the cc text is in uppercase? This is just nonsense. We will need some post processing tool for this if we want to use ripped cc.

[gabest]

McPoodle: I have a little problem with that sample you linked in your first post in this topic. I can extract the following from it:

0:00:02.002 - 0:00:03.003
HERE IS THE FIRST
LINE OF CLOSED CAPTIONS.

0:00:04.504 - 0:00:06.006
-- SOME MORE CAPTIONS --

0:00:07.507 - 0:00:09.009
LEFT CENTER RIGHT.

The problem is the starting time. The cc data is splitted into two packets, the first one is at the right time (1/4/7s), but the second part has the "End Of Caption" command in it, which should instruct the decoder to show the buffered text on the visible one. So, I would rather think that the real showing up time is at 2/4.5/7.5s.

With the dvd I was also testing, the EOC command is perfectly synced with the subpicture starting times, even if it's not right in the first part of the splitted data. However it has another problem, there is no "Erase Non-displayed [buffer] Memory" command. May be this dvd and yours differ in something I'm not aware of.

[McPoodle]

To respond to the last two posts in order:

I only had a couple of DVD's to work this system out. Your's is the first one I've seen with so much empty space in it (the 80 80's). If you take all that out you get a standard sound effect caption ("[ Sonar Pinging ]" for anyone else following this thread). This brings up one of the most annoying things about closed captions: the bandwidth is so low that the text ripped is often different than what was actually said.

Another annoyance is that all-caps thing. The CC standard was written in 1983, and was required to be backward-compatable with existing settop decoder boxes and TV's that lacked a lowercase character set (they also only had white on black text, which is why the dialog isn't color-coded by speaker). Finally, more than one person can be speaking at the same time, so you really have to pay attention to text positioning.

For the other post, I think your analysis is essentially correct. The start times should be 0:00:02:02, 0:00:04:22, and 0:00:07:19 (that's in frames as opposed to your more-logical format). These are the times that the first of the pair of EOC codes is encountered. The "start time" for SCC files is the first non-8080 byte pair in the caption, as much as a full second before the caption is displayed. As for the display end times, that's when the EDM command is next encounted, at 0:00:03:00, 0:00:06:00, and 0:00:09:00.

Finally, for the ENM command, the EOC command swaps displayed and non-displayed memory, so if EDM is issued before the EOC, the non-displayed memory is automatically cleared.

Of course, I created my DVD from scratch, including writing my own SCC file for Scenarist. Looking back at the file now I spot a few mistakes, like forgetting to double the positioning commands in the last caption. So it may not be precisely like DVD captions, but at least the captions display on my TV the way I they're supposed to.

[gabest]

Originally posted by McPoodle
I only had a couple of DVD's to work this system out. Your's is the first one I've seen with so much empty space in it (the 80 80's). If you take all that out you get a standard sound effect caption ("[ Sonar Pinging ]" for anyone else following this thread). This brings up one of the most annoying things about closed captions: the bandwidth is so low that the text ripped is often different than what was actually said.Meanwhile I managed to handle this correctly. I need to extract the even and odd byte pairs and put them after eachother leaving any empty space out. These blocks can even span across more GOP headers. So one odd byte pair string can be the start of the next even, etc. Also the length of the blocks at the beginning isn't shifted up by one, and all seven lsb bits are valid (8th is the parity). It tells the exact length of the two interleaved byte pair strings together.Finally, more than one person can be speaking at the same time, so you really have to pay attention to text positioning.I use a 32x16 text buffer and handle all the cursor positioning commands (is there any logic in those tables?), so the text is always stored as it should look on screen. Only when saving it need I convert the lines to something compatible with the output file format.

[gabest]

http://vobsub.edensrising.com/get2.php/vobsub.zip

0. (run a dvd player to unlock the dvd drive if you are going to be ripping from it directly)
1. rundll32.exe vobsub.dll,Configure
2. open...
3. select *.ifo from the type list
4. open vts_0x_0.ifo
5. select output dir
6. select any program chain you like and make sure "cc - ..." is in the top right list (if initially not, it means that the field 1,2 bits were not set in the ifo)
7. ok, wait .... , ok, open the output folder in explorer and check vts_0x_0.sub.cc.srt for the closed captions.

Warning: It can't parse already preprocessed vob files!!!

[zuggy]

Just for the interested, on vob I test is the format of 1st CC "ff xx xx fe 00 00" and all others are "fe 00 00 ff xx xx" formatted.

[kxy]

Gabest,

I tested out your vobsub.dll on one dvd so far, it works. The results are something like this... So, besides the caps and lot of questions marks and text positioning, it works. Now we just need a post processing tool to correct them if we want to use ripped cc.


0:00:02.002 - 0:00:03.003
HERE IS THE FIRST
LINE OF CLOSED CAPTIONS.

0:00:04.504 - 0:00:06.006
??

0:00:07.507 - 0:00:09.009
?? ?? ???


0:00:09.009 - 0:00:14.003
ALL CAPS

[gabest]

Those question marks are probably special characters, which can't be mapped to the ascii table (some of them may be, but currenty none of them are handled).

[kxy]

Here is another problem, it seems those close caption might have being italics. The movie is called In the Bedroom.

1
00:00:40,172 --> 00:00:43,175
)irds calling faintly
(

2
00:00:45,678 --> 00:00:47,179
)rass rustling
(

3
00:00:47,680 --> 00:00:49,181
)oman laughing
(

4
00:00:52,685 --> 00:00:54,186
)queals
(

5
00:00:54,687 --> 00:00:57,189
)aughing breathlessly
(

6
00:01:10,736 --> 00:01:13,238
)aughing continues
(

7
00:01:30,255 --> 00:01:32,257
)ind blowing
(

8
00:01:38,764 --> 00:01:41,767
I love it here.

9
00:01:41,767 --> 00:01:43,769
o.
I know you d

10
00:01:47,740 --> 00:01:51,744
I can feel my life,
you know?

11
00:02:04,256 --> 00:02:06,258
)urmurs gently
(

12
00:02:16,769 --> 00:02:18,771
)irds calling
(

13
00:02:25,778 --> 00:02:28,280
Pitch to Hatteberg.
SPORTSCASTER (

[gabest]

If you can wait a little with this dvd, I'll try to modify vobsub.dll to output the raw cc data. That would be more helpful to me to find the problem.

[gabest]

http://vobsub.edensrising.com/get2.php/vobsub.zip

Outputs c:\raw.cc after indexing.

[kxy]

Sorry for the delayed testing. I was away for the week.

I downloaded the file from http://vobsub.edensrising.com/get2.php/vobsub.zip.

However it didn't produce a "c:\raw.cc" after indexing, rather it produced a *.cc.srt, see attatchment. Also I have attatched idx, and sub for you.

[kxy]

Err, due to size limitation, I break it down to 2 files. Rar renamed to zip.

[kxy]

part2.

Need to get approved for it to show.

[frank]

Do you know the tool CCparser 0.05b, downloadable at doom9?

[gabest]

kxy: I could already see the beginning of srt output but that's not useful for me to find the problem. I'd need the not-yet-decoded closed caption data to see which codes aren't handled properly and resulted in those ( ) chars. If you still can please test it again with this dll: http://vobsub.edensrising.com/get2.php/cctest.rar . Now I really hope that it will output a *.cc.raw file (in the same dir) for you too :P

[kxy]

okay here.

If you need the reference idx and sub file to check the results, you can always go back to the previous post.

Keep in mind that I have to rip this movie to the hard drive first using smartripper. Becaue if I use vobsub, it gives this error...
"Key NOT found! (possible reasons: unencrypted dvd / locked dvd / vts_0x_x.vob too short)"

[kxy]

Second try

[gabest]

There is no closed caption data in the *.sub file, it is stored elsewhere in the vob and won't get copied. If you rip the dvd, be sure to make 1:1 copy of the vobs, then it should be no problem for this version of vobsub.

[kxy]

Yes. Of course there won't be any closed caption data in the *.sub file. The CC file is on the post that says second try, a mod need to approve it first.

BTW, do you have any plans to work around this problem? "Key NOT found! (possible reasons: unencrypted dvd / locked dvd / vts_0x_x.vob too short)"

[gabest]

Originally posted by kxy
Yes. Of course there won't be any closed caption data in the *.sub file. The CC file is on the post that says second try, a mod need to approve it first.Ok, I'm waiting.BTW, do you have any plans to work around this problem? "Key NOT found! (possible reasons: unencrypted dvd / locked dvd / vts_0x_x.vob too short)" I only reused vobdec's code, which is not a 100% reliable method for decrypting :)

[ppera2]

Good work Gabest.
I ripped cc from R1 DVD 'This Island Earth', fast and without any trouble.

[aleck]

I tried your procedure Gabest, with all three versions of vobsub.dll, with no success.

When I try directly from DVD, I get:
Key NOT found! (possible reasons: unencrypted dvd / locked dvd / vts_0x_x.vob too short)
(I played the DVD in the player first, closed the player, and started vobsub.dll).

When I try from ripped vobs, this error message doesn't show, but I get empty file.

In both cases, after opening .ifo file, i get a list like this:
00 - [Not detected]
01 - [Not detected]
02 - [Not detected]
...
31 - [Not detected]

What am I doing wrong?

[gabest]

Send me your ifo file onto my email address.

[aleck]

Gabest, I sent it to the e-mail I found on edenrising.com (I couldn't figure out how to attach the file using doom9 private messaging).

[ookzDVD]

@Gabest,

I try to rip the CC from Austin Powers Goldmember,
with your latest VobSub 2.21, I've already followed your steps
from your post, but it not generated the raw.cc :(

Btw, I can't find the "cc-" from the top right :(

PS. It's Infinifilm, and here is the .IFO file generated by DVDDecrypter 3.0.1.4 with stream processing for Video+Audio(6ch)+CC.

http://web.newsguy.com/ekoboy/VTS_01_0.IFO

Thank you.

[gabest]

"Btw, I can't find the "cc-" from the top right"

Ok, but did you add it to the right-side box?

I know dvddecrypter the least (it can't rip from the hdd and I don't have any ntsc movies on dvd), so how about trying other methods: vstrip, smartripper, dvd directly? :)

[gabest]

Originally posted by kxy
Here is another problem, it seems those close caption might have being italics. The movie is called In the Bedroom.
...Finally, this problem should be fixed in v2.22 :)

[ookzDVD]

@gabest,

I still can't rip the closed caption for the Austin Powers in Goldmember. :(
It's InfiniFilm :(

If you have time, I'll send all the .IFO file fom that DVD in one .zip or .rar file.

Thank you.

[mookyking]

An easy way to fix the all caps output in the .srt file is to use ms word...
- load the .stl file, press ^A to select all text.
- clicker Format, change case, lower case.
- clicker Format, change case, sentence case.

[kxy]

That won't fix everything, for example, name and place will still be lower case, and if you have a long file more than 250 pages, MsWord will have trouble converting them all at once.

Easier/fastest way will be assembly.

[mookyking]

I've done it with large files, around 200 pages with no problems. As for capitalization of names and such, I just use the spelling and grammar checker (F7). Word will automatically bring up incorrectly capitalized names and words, and when I make a correction, I just click 'change all' (for example, to change all occurrences of 'mary jane' to 'Mary Jane').
I don't see how creating a special program (not necessarily in assembly) would make this any easier, as you would still have to go back and manually capitalize proper nouns and names... also, word does spell checking on your script too...

[divxcollector]

I tried to rip the cc subs with Vobsub 2.23 from Near Dark R1, Quigley Down Under R1, The Mutant Watch featurette of the X-Men R2 and Arrival 2: The Second Arrival R1.

I couldn't succeed in any of them and always had the cc-(not detected) warning, but they have cc subs.

I rip the Vob and ifo files with DVD Decrypter v3.1.4. Do I have to change the ripper or just I wasn't lucky to find a dvd which has the cc subs in the place where Vobsub is looking for? :)

[gabest]

VobSub 2.23 is getting old, try with this one: http://gabest.org/vsrip.rar

cc-not detected only means that the ifo doesn't have the bit set. It happens often.

[divxcollector]

This vsrip works perfectly. Thank you very much Gabest.

[McPoodle]

To bring this thread back from the dead, I've been recently alerted to a type of closed caption formatting that VSRip and SubRip both choke on when converting to SubRip format.

The format I've seen most often (which I covered in my earlier posts and which works very well with VSRip and SubRip) looks like this:

94ae 94ae 9420 9420 [positioning codes, text] 942c 942c 942f 942f 8080 8080 ... 942c 942c

This translates to the following sequence of codes (written out for two captions in sequence):

94ae 94ae: Erase offscreen buffer
9420 9420: Start caption # 1
positioning codes, followed by text: all written to buffer
942c 942c: Clear screen
942f 942f: Swap buffer and screen (start time of caption #1)
several frames of nothing (8080)
942c 942c: Clear screen (end time of caption #1)
more frames of nothing (8080)
94ae 94ae: Erase offscreen buffer
9420 9420: Start caption # 2
positioning codes, followed by text: all written to buffer
942c 942c: Clear screen
942f 942f: Swap buffer and screen (start time of caption #2)
several frames of nothing (8080)
942c 942c: Clear screen (end time of caption #2)
[/list=1]
Under this scheme, the caption starts at the 942f 942f, and it ends at the next appearance of 942c 942c. Note that this pattern allows each caption to appear on the screen and disappear before the next caption is shown.

The new pattern I've run into jams the captions right into each other, as follows:
94ae 94ae 9420 9420 [positioning codes, text] 942f 942f
This is immediately followed by the next 94ae code. Now here's the translation for two captions in a row:
[list=1]
94ae 94ae: Erase offscreen buffer
9420 9420: Start caption # 1
positioning codes, followed by text: all written to buffer
942f 942f: Swap buffer and screen (start time of caption #1)
94ae 94ae: Erase offscreen buffer
9420 9420: Start caption # 2
positioning codes, followed by text: all written to buffer
942f 942f: Swap buffer and screen (start time of caption #2 and end time of caption #1)
[/list=1]
As you see, with this format you have to keep track of two captions at the same time. I was able to get my CCASDI tool to handle both scenarios by following these rules (in order):
[list=1]
Start with a buffer to build a caption and variables for the complete caption, the start time, and the end time, initializing all three to null. The caption, start time and end time variables will all refer to the same caption, while the buffer could be referring to the next caption. You also need a last code variable (reset to null anytime a code is different from the previous code) as a two-part clear flag (942c, followed by anything other than 94ae, followed by 942f), that should start out unset.
Whenever you find the caption, start time and end time variables all populated, output the caption and reset the three variables to null.
Loop: for each code (two bytes) read:
If the same code appears twice in a row, skip the repeat without following any of the following rules.
If you hit the code 94ae, clear the buffer and reset the two-part clear flag.
If you hit the code 942c, set the two-part clear flag.
If you hit the code 942c and you have a start time but no end time yet, set end time.
If you hit the code 942f and you have a start time but no end time yet, set end time.
If you hit the code 942f, build the caption variable from the buffer and set the start time.
If you hit the code 942f and the two-part clear flag is set, clear the buffer and reset the flag.
Any other code will get written to the buffer.
Jump back up to Loop for next cycle.

Be sure to follow rules 8 and 9 in that order.

I know this is confusing--I had to do a major re-write of my CCASDI tool to handle this. Therefore, I've posted some sample files at my site: http://www.geocities.com/mcpoodle43/SCC_TOOLS/jammed.bin, http://www.geocities.com/mcpoodle43/SCC_TOOLS/jammed.cc.raw, http://www.geocities.com/mcpoodle43/SCC_TOOLS/jammed.scc and http://www.geocities.com/mcpoodle43/SCC_TOOLS/jammed.ccd. When I originally tried to convert the SCC file into SubRip format with CCASDI, I dropped every other line, but once I made my changes, I ended up with http://www.geocities.com/mcpoodle43/SCC_TOOLS/jammed.srt. You can poke around my Perl source code for CCASDI at http://www.geocities.com/mcpoodle43/SCC_TOOLS/ccasdi.pl, but the logic is not particularly easy to follow there.

Anyway, good luck to whoever decides to tackle this problem.

--McPoodle