@karl_lillevold : any chance to find out more about Real's codec API ?

[ChristianHJW]

Hi Karl,

sorry for the direct approach using this webboard, but as there may be interest from the users of this board to learn more about Real's opinion here, i thought its the way to go.

I dont know if you are aware of our efforts to overcome the limitations of the currently used VfW and ACM codec APIs, being 'given' to the video and audio compression world years ago, when Microsoft was focussed on earning money with selling Operating Systems, and not by offering video and audio compression technologies.

Both APIs are pretty outdated, like VfW not having proper support for b-frames because of the one-frame-in/one-frame-out approach, and of course ACM having the huge limitation of not suporting VBR audio ( i wonder how M$ will overcome this for WMA ? Offering CBR only ? ).

In our attempt to create an open standard container format replacing AVI we had to face that the above mentioned limitations hurt as badly when trying to interface to codecs from encoding applications writing directly into our container, so we had to face that in addition to the work of defining a new container spec we also have to invest a significant amount of time into such a codec API. Honestly speaking, this almost killed the project because of the major time that had to be invested from our core development team into what we were calling 'transor API' ( Draft of the C++ code is here (http://matroska.sourceforge.net/transor/) and here (http://matroska.sourceforge.net/transor/transor-API.html) ).

Fortunately we were contacted from a gentleman named Alex 'Foogod' Stewart at that time, who by chance had fallen over this proposal and was about to make a very similar thing for *nix/Linux ( there is no such thing as od yet, besides gstreamer plugin API of course, requiring Gstreamer as framework ). We convinced him to enlarge his project from *nix to a multiplatform solution for all OSes, including Windows, project was renamed from 'Unix Codec Interface' to 'Universal Codec Interface' and Alex was a great contributor.

Unfortunately our container format was suffering from a project fork end of last year, and for some very strange reason Alex Stewart doesnt reply to any of our emails since then, nor is there any visible progress on http://uci.sf.net, so fear that UCI may have died a sudden death :( ...

Now, i have personal contact with somebody who was personally involved into a professional project based on Helix, where a nice custom made streaming solution had to be developed in close cooperation with your company ( was wokring very successfully in the end ), and this guy tells me that he was quite impressed by the Real API he was working with !!


My main questions to you are :

1. Is there any chance that we could get our hands on the Real codec API, even on a stripped down version of it ?

2. Do you personally have any comments on the UCI (http://uci.sf.net) specs ( i still hope we will be able to continue working on it to avoid Alex' work is lost, maybe by founding UCI.corecodec.org ) or on the Transor API (http://matroska.sourceforge.net/transor/transor-API.html) as done by robux4 ?

3. Are you aware of any other good codec API suporting multiple frame handling for video streams and VBR audio we could look at ?

4. Did Real ever look at making a Gstreamer (http://gstreamer.net) plugin for Real ( no need to make it open source BTW, gstreamer license will allow closed source implementations based on gstreamer API for companies wanting to jump on the train AFAIK ) to have a good real support for Linux ?

Thanks for your time to answer my questions, if possible

[karl_lillevold]

Hi Christian,

Thanks for your questions, that deserve better answers than I can give off the top of my head. Personally I have struggled with the VfW interface and B frames in the past, so I know about that part of your post (our work-around was dummy frames of a certain number of bytes indicating B frames were in use, and the decoder should go into latency mode :) ) However, I have no experience with the projects you mention, UCI or gstreamer, but I will forward your questions to see if anyone has.

With regards to the RealNetworks codec API, much of it was released with Helix, but maybe not well enough documented to make it easy to plug into another container format. (codeclib for the decoder, a filewriter plugin would be needed for producer / see oggwriter as an example). Then there's the problem that the Helix licenses leave it open for interpretation whether or not such use of the codecs is currently possible. So as you can see, we have not thought a lot about alternative container formats for RealVideo or RealAudio until these last few weeks, but these questions are making the rounds, and are being discussed.

[ChristianHJW]

Originally posted by karl_lillevold
Thanks for your questions, that deserve better answers than I can give off the top of my head. Personally I have struggled with the VfW interface and B frames in the past, so I know about that part of your post (our work-around was dummy frames of a certain number of bytes indicating B frames were in use, and the decoder should go into latency mode :) ) ... yes, DivX Networks was using the same workaround for DivX5 ( Avery Lee even adapted Vdub to support this ), so the XviD Team was doing the same. Just a matter of time IMHO until the developers realize AVI/VfW was driven to its outer limits by doing so ....

However, I have no experience with the projects you mention, UCI or gstreamer, but I will forward your questions to see if anyone has.
Thank you very much, thats more than i could ever expect.

With regards to the RealNetworks codec API, much of it was released with Helix, but maybe not well enough documented to make it easy to plug into another container format. (codeclib for the decoder, a filewriter plugin would be needed for producer / see oggwriter as an example).
May i read from this that your API can also handle more than just one stream of data ( like UCI is doing ), but maybe also handling of other containes such as DV ? We needed this badly to be able to make format conversion using plugins, like OGM --> matroska or matroska --> Ogg/MP4 , and UCI doesnt support this as Alex wanted to keep it simple for codec developers. There was talking about UFI as an extension to UCI, handling that, but this may be in a far future, given the actual status of UCI.

Then there's the problem that the Helix licenses leave it open for interpretation whether or not such use of the codecs is currently possible. So as you can see, we have not thought a lot about alternative container formats for RealVideo or RealAudio until these last few weeks, but these questions are making the rounds, and are being discussed.
Patiently waiting for some more comments. It would be funny though if Real would allow us to use the codec API, so we could start to add interfaces to other codecs ( Vorbis, Theora, AAC, AC3, XviD, etc ) .. could those be used in your Helix framework then ?

[ChristianHJW]

karl, any news here ?

[karl_lillevold]

Hi Christian,
I'm afraid I don't have a lot of news. I did not get any responses to my email about either UCI or GStreamer, which probably means that few here know about these projects. I quickly read through the UCI spec myself. It looks good, and from a technical point of view I am sure our codecs could be plugged into it. I also looked over the GStreamer project, and it appears to provide some of the same functionality and framework as the Helix open-source DNA, which also compiles and runs on Unix. I don't immediately see how the two could work together, but I am no expert on either (yes, I am a little too codec-focused, just trying to find time to work on RV9 improvements). You would get a much wider audience of Helix experts in one of the general Helix mailing lists:
https://www.helixcommunity.org/2002/intro/mailinglists/

[karl_lillevold]

Originally posted by ChristianHJW
May i read from this that your API can also handle more than just one stream of data ( like UCI is doing ), but maybe also handling of other containes such as DV ? We needed this badly to be able to make format conversion using plugins, like OGM --> matroska or matroska --> Ogg/MP4 , and UCI doesnt support this as Alex wanted to keep it simple for codec developers. There was talking about UFI as an extension to UCI, handling that, but this may be in a far future, given the actual status of UCI.
Our lower layer codec API is much like UCI in functionality, it just looks different :) The multi-stream support (SureStream) is handled by a higher layer in the system (not open source).

Other formats than RealMedia, like MP3, MPEG-4, H.263, is handled differently, but that's too "far away" from the codec for me to explain in detail. This handling of other formats, however, is available in full open source for anyone who is interested.

Patiently waiting for some more comments. It would be funny though if Real would allow us to use the codec API, so we could start to add interfaces to other codecs ( Vorbis, Theora, AAC, AC3, XviD, etc ) .. could those be used in your Helix framework then ?
First we would have to decide to open the codec API entirely, and for instance allow and/or develop a DirectShow plugin version of RV9. This is being discussed. If someone were to ask for it on a Helix mailing list with some good arguments, it would bring a little more attention to the matter.

[ChristianHJW]

I am subscribed to some of the lists now. I will first read to get a feeling and maybe after some time post some questions i have. Thanks

[iwod]

Originally posted by karl_lillevold

First we would have to decide to open the codec API entirely, and for instance allow and/or develop a DirectShow plugin version of RV9. This is being discussed. If someone were to ask for it on a Helix mailing list with some good arguments, it would bring a little more attention to the matter.

so.... there are talks about openning the codec API entiirly!!!

MY god......