Closed caption data in VC-1 stream

[RickyLasVegas]

I'm trying to encode VC-1 compliant WMV files with closed captioning in the elementary stream, according to format described in the VC-1 specs (A/53D Annex F).
A page in the WM Format 11 SDK documentation (http://msdn2.microsoft.com/en-gb/library/aa390696.aspx) seems to suggest that this might be done using a frame level sample data extension of type WM_SampleExtensionGUID_UserDataInfo
Has anyone tried this before, or is there another way to do it?

Alternately: does anyone know of any Windows Media sample files (VC-1 profile) that contain valid CC data in the elementary stream so I can see how it's done?

Thanks for your help.

Rick

[veffremov]

I'm trying to encode VC-1 compliant WMV files with closed captioning in the elementary stream, according to format described in the VC-1 specs (A/53D Annex F).
A page in the WM Format 11 SDK documentation (http://msdn2.microsoft.com/en-gb/library/aa390696.aspx) seems to suggest that this might be done using a frame level sample data extension of type WM_SampleExtensionGUID_UserDataInfo
Has anyone tried this before, or is there another way to do it?


I'm prety sure WMP will not interpret the user data as CC and display it on the screen. What are you trying to accomplish by this?

[RickyLasVegas]

I agree that WMP will likely not recognize the CC data, but I'm working with a set top box that is supposed to do so (provided of course that the data is embedded correctly in the VC-1 stream).
So ideally I would like to find a sample or two that I could use, both to test the box and to analyze the file format.

[zambelli]

The WM_SampleExtensionGUID_UserDataInfo extensions are only applicable to ASF file formats.

[RickyLasVegas]

Yes exactly - in fact I've modified the UncompAVItoWMV sample code from the WM Format SDK 11 to add a WM_SampleExtensionGUID_UserDataInfo extension to each sample. This extension contains the CC data for the frame, formatted as specified in the VC-1 spec.
But perhaps this is not the right way to wrap the closed caption data within the ASF format. So I guess what I'm really trying to find out is: Is this the correct way to carry CC data in the VC-1 elementary stream within a WMV file?
Any assistance and/or samples (code, relevant WMV files) would be appreciated, thanks.

[veffremov]

I'm not sure this will work. I'll ask around, but I don't think we have an official way to do this in the format SDK.

Some backgound info: User data is defined as "data to be interpreted by the application". It is not nesecerily CC. Only VC1 Advanced profile has the capability to carry user data packets. When the user data is embedded in the VC1 elementaty sream the ASF container is completely agnostic to it - it carries the VC1 stream as if it has no user data.

The code decoder code will recognize the user data packets and will notify the application of the presense of such packets. I have no clue how this works in the format SDK, but I can ask.

If you just need some VC1 streams with user data in them, PM me I can send you a few samples.

[zambelli]

The code decoder code will recognize the user data packets and will notify the application of the presense of such packets. I have no clue how this works in the format SDK, but I can ask.
The SDK is unlikely to handle this as it's more or less agnostic of VC-1 specifications. MF might be a different story, though I doubt that type of feature work met the Vista shipping bar.

Bottom line is that any type of user data carried in either ASF extensions or VC-1 bitstreams needs to be handled by the player application - and AFAIK, no such functionality exists in WMP/WMFSDK to handle custom Line21 data. The only WMFSDK supported way to do subtitles in ASF/WMV is via SAMI or script commands. That's the only format WMP knows how to handle automatically.

[RickyLasVegas]

That's what I expected, but I just wanted to make sure that there wasn't an easy way to attach the CC data as user data and get it recognized (even eventually) by WMP.
I'll look into one of the alternate methods for now.
Thanks for your help.

[zambelli]

I'll look into one of the alternate methods for now.
Thanks for your help.
Search MSDN for articles related to SAMI. In WMP (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnwmt/html/wmp7_sami.asp) its handling is pretty much automatic. In DirectShow, this might be of some help: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directshow/htm/samiccparserfilter.asp

The alternative is to use TEXT and CAPTION script commands, embed the WMP ActiveX (OCX) control and handle the TEXT/CAPTION script events programatically:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/wmpsdk11/mmp_sdk/playerscriptcommand.asp