Ripping Close Caption (Line21) ?

[sw3d]

I'm surprised that almost nobody here
seems to have interest in CloseCaption?!
Some of my NTSC DVD's have French or Spanish subtitles but no English ones...
But they all have CC. (in English)
I tried to follow the instruction which come with 'graphedit' but I'm somehow
not able to dump the output as a raw data file...
I someone managed to do it, I really would appreciate some advice...

[McPoodle]

I figured this out a few weeks ago when writing a tool to convert these dumps to Scenarist Closed Caption import format. The GraphEdit picture (should) be at
http://www.geocities.com/mcpoodle43/SCC_TOOLS/DOCS/dvdcapture_cc.gif

Some stuff I figured out along the way:

The VOB file at the beginning is the File Source (Async) filter.
The "cc.txt" file at the end has to be the Dump filter, not the File Writer filter. This filter is installed from the install.bat file that came with GraphEdit. In my case, the file didn't work until I added the line "set path=%path%;." to the top of the file.
Three filters are different for WindowsXP (the picture is from W2K). Change MPEG-2 Splitter to MPEG-2 Demultiplexer, Line 21 Decoder to Line 21 Decoder 2, and Overlay Mixer + Video Renderer to Video Mixing Renderer.


Good luck!

--McPoodle

[sw3d]

Thanx so much...
Now it works beautifully!
(Just didn't expect I have to watch the whole movie in real-time...:o )
But better than nothing, of course.

How did you come up with the "set path=%path%" thing?

Thanx again...

[Chibi Jasmin]

There's a tool called CCParser on Doom9's download page, that can translate these outputfiles into SubRip format...in case you need it.

[d62ks821]

McPoodle, I found your append very helpful. Rather than capturing CC from DVD, I am trying to capture it from TV or VHS tape. Using graphedit, I have been able to capture CC to file. However, ccparser complains, "

CCParser 0.05 Beta by pa_si
Unable to get first header
Parsing Error
"

Graphedit graph:
---------------
Conexant TVTuner --> Conexant Crossbar --> Conexant Capture --> CC Decoder(1st output) --> dump:out.bin

CC Decoder(2nd output) --> Line 21 Decoder --> Color Space Converter --> Video Renderer

Here is head of my out.bin. Does this look correct for ccparser input? The filter pin property calls this Line21_BytePair format, as apposed to Line21_GOPPacket (which is the format I get when capturing CC from DVD).

FF FF FF FF 20 57 4F 52 C4 D3 20 43 C1 CE 20 C2 45 20 D5 D3 45 C4 91 70 91 6E 54 4F 20 C4 45 D3 43 52 49 C2 45 20 4F D5 52 20 43 C8 49 43 CB 45 CE BA 94 2C 94 2F 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 94 AE 94 20 91 D0 91 CE 57 C8 4F 4C 45 D3 4F CD 45 AE 80 80 80 80 94 2C 94 2F 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 80 94 AE 94 20 91 D0 91 CE D5 D3 C4 C1 20 49 CE D3 D0 45 43 54 45 C4 AE 80 80 80 80 80 94 2C 94 2F 80 80 80 80 80 80 80 80 ...

Perhaps ccparser simply does not support Line21_BytePair format, and I am on my own to reverse engineer this and write a little app to decode this. Any help would be greatly appreciated.

[d62ks821]

I latched on to the following. It does process LINE21_BYTE_PAIR format CC data captured from TV/VHS, but it produces CC Line 21 input for Scenarist, not a .sub file like ccparser can. (As a side note, I tried importing this output into Scenarist 2.5.1 and anything over 100 or so lines would crash Scenarist, ...less work just fine.)

http://www.sonic.net/~mcpoodle/SCC_TOOLS/SCC_TOOLS.zip

I fiddled around with ccparser some more, and was still unable to get it to work with CC data captured from TV/VHS. It does work just fine with CC data captured from DVD.

I was able to generate subtitles using the following steps, but requires considerable manual editing.

(1) Capture CC data to out.bin from TV card using graphedit graph referenced in prior append.
(2) Using SCC_TOOLS, raw2scc out.bin out.scc
(3) Using SCC_TOOLS, ccasdi -s out.scc out.sub
(4) Now comes the fun... Manually edit out.sub to look like subviewer format. In doing so replicate the start time to create a stop time for each sub. Leave start and stop time the same for now. For example:

[INFORMATION]
[AUTHOR]
[SOURCE]
[PRG]
[FILEPATH]
[DELAY]
[CD TRACK]
[COMMENT]
[END INFORMATION]
[SUBTITLE]
[COLF]&HFFFFFF,[STYLE]no,[SIZE]18,[FONT]Arial
00:01:00.09,00:01:01.09
Hello, My name is joe.

00:01:03.11,00:01:03.11
I have a wife and a dog and a family.
.
.
.

(5) Now using subadjust, make each subtitle 3 seconds in duration using delay expander feature.
(6) Now using subadjust, covert to .SSA subtitle format.
(7) Now use MaestroSBT to generate the actual subtitle bitmaps. This process will further adjust the 3 second subtitle duration to ellimate any overlaps that exists.

The resulting subtitles were pretty decent.

[McPoodle]

It looks like you could use scc2dvd, a diagnostic tool I wrote when working on SCCTOOLS. I've uploaded it to http://www.geocities.com/mcpoodle43/SCC_TOOLS/scc2dvd.exe. What you would do is use raw2scc to convert your TV-Closed Caption file into SCC format, then use scc2dvd to convert it back into DVD-Closed Caption format for use with CCParser. I've noticed that CCParser gives me a bunch of error messages with the converted files, but the output looks alright.

Feel free to use whichever method works best for you.

--McPoodle

P.S. My web page address changed lately, so change "www.sonic.net/~mcpoodle" in the previous post to "www.geocities.com/mcpoodle43".

[McPoodle]

By the way, here are the differences between Line21_BytePair and Line21_GOPPacket formats:

Line21_BytePair format is extremely straightforward. The first four characters are ff ff ff ff. Following this are the two-byte opcodes, which are described in the SCC_TOOLS documentation at http://www.geocities.com/mcpoodle43/SCC_TOOLS/DOCS/SCC_TOOLS.HTML.

Line21_GOPPacket format, on the other hand, starts with 00 00 01 b2 and adds a lot of extra characters in between the opcodes, I believe to fit the opcodes around the GOP sizes in the DVD. The rather ugly specification can be found here: http://www.scte.org/documents/pdf/ANSISCTE202001DVS157.pdf.

Here is an example of this format in use:

1. The first section starts with 00 00 01 b2 43 43 01 f8 8a (including the file header).
2. This is followed by the pattern ff xx xx fe 00 00 five times, where xx xx is a two-byte opcode.
3. The second section starts with 00 00 01 b2 43 43 01 f8 8f.
4. This is followed by the ff xx xx fe 00 00 pattern seven times, followed by ff xx xx for the eighth opcode.
5. The third section starts with 00 00 01 b2 43 43 01 f8 16 fe 00 00 (note that it includes the three bytes taken off of the last opcode pattern).
6. This is followed by the ff xx xx fe 00 00 pattern ten times, followed by ff xx xx for the eleventh opcode.
7. The fourth and all following sections start with 00 00 01 b2 43 43 01 f8 1e fe 00 00, followed by the ff xx xx fe 00 00 pattern fourteen times, followed by ff xx xx for the fifteenth opcode.

Apparently, the last byte of the header encodes how many opcode patterns are in the section (or maybe the number of bytes). For simplicity sake, I used the exact layout above in my scc2dvd program.

--McPoodle

[d62ks821]

Thanks for the tips. You are right, the line21_byte_pair data was relatively easy to extract. Moments after my last append, I set out to write a little program to extra it to subviewer format and had something workable within a couple hours.

Comparing what I wrote to the tv->scc->dvd process you suggested, I noticed my subtitle durations were longer, and a little better matched the audio track. What I did was mark start time at the {EOC}{EOC} at the end of each line, and mark stop time at the next {EDM}{EDM} or {EOC}{EOC}, which ever is first. Does this seem right?

Also, Have you had problems with importing .scc data into scenarist 2.51? Scenarist asserts for illegal memory access during vob build process. I have tried on windows XP and 2000. It does work OK if I delete all but the first 10 minutes of so from the .scc file, i.e. Scenarist completes, and powerdvd can display the resulting CC.

[StriderGT]

I have opened the first vob file of my movie with graphedit and
connected the dump (DirctShow filter) input with the closed caption output of the Cyberlink Video/DSP Decoder (I have "registered" graphedit). When I press play, the dump file I use starts gathering data and if I press stop during the playback the closed captions are there and I am able to convert them with ccparser. But if I let the playback of the first vob finish (the play button of graphedit remains pressed even though there is no image) then the dump file becomes 0KB as if all cc data where erased...
Has anyone encounterd the same problem, am I doing something wrong, I just want to convert the captions to subtitltes.

-Is there any way to convert all the vob files captions or do I need to append each one?
-Is there any way to extract the closed caption data without the need to playback the movie (through vstrip for example)?

[wilson2k]

Dear McPoodle,

I'd like desperately to try your "magic" tools for converting the CC to subrip format that I can edit. But the links you provided doesn't lead me to where it's supposed to be. I can't find your tools anywhere on the internet either.
I really need your help!!

[McPoodle]

My web page hasn't moved since December of 2003. Just in case there's something wrong with links in Doom9 all of a sudden, here it is in plain text:

http://www.geocities.com/mcpoodle43/SCC_TOOLS/DOCS/SCC_TOOLS.HTML

--McPoodle

[wilson2k]

McPoodle,

Thank you for your reply.

But it's strange that I can't open the link, each time the explorer just gives me a "can find the site" notice. But anyway, I've got your tools thanks to mpucoder.

I find your tools do great jobs for me, thank you!!:D

[stickboy]

Hi,

I'm trying to capture CC data through my WinTV-401 card. (I'm using Windows 2000 and the open-source btwincap drivers, if that matters.) I tried following the SCC Tools instructions for capturing from an analog source, and I made this filter graph: cc-graph.png (http://www.avisynth.org/stickboy/tmp/cc-graph.png).

Running it generates this binary CC file: cc.bin (http://www.avisynth.org/stickboy/tmp/cc.bin)

I examined it in a hex editor, and the data doesn't start off with FF FF FF FF nor with 00 00 01 B2. I can't access pin properties to any of the VBI or CC pins in GraphEdit.

Sure enough, raw2scc rejects this file:
File type of not recognized, stopped at C:\TEMP\raw2scc.exe line 183.

Any idea what I'm doing wrong?

Thanks!

[stickboy]

Answering my own question:
After examining the sample .bin files included with SCC Tools, it's not clear that I'm missing anything other than the initial FF FF FF FF bytes. I prepended those and sent it through raw2scc and ccasdi, and it seems to report the correct text.

[McPoodle]

Yes, I've been seeing this with quite a few cards. It's somewhat annoying for me, because if the file has no header, how do I know if it's in the right format? I wrote up a simple program that will tack a header onto any file, so people less experienced with hex editors can still use my programs. Download http://www.geocities.com/mcpoodle43/SCC_TOOLS/header.exe. Here is an example of its use from the command line:

header -hffffff infile.bin outfile.bin

This will copy infile.bin to outfile.bin, putting the necessary header first.

[stickboy]

Originally posted by McPoodle
Yes, I've been seeing this with quite a few cards. It's somewhat annoying for me, because if the file has no header, how do I know if it's in the right format?Yeah, that sucks.

If there are only two primary formats, I guess the best you can do is try one, ask the user to verify if the output looks correct, and if not try the other.

Or just document it. :)