ffdshow tryouts project: Discussion & Development

[albain]

Is there a way to detect if the stream is EAC3 when the input codec is AC3 ?
By analyzing the header, the bitrate ?
I don't know this part at all, but maybe we can implement a hack to detect the stream before sending it to the decoder

[madshi]

Some pieces of information:

(1) Haali told me that his way of reporting E-AC3 separately from AC3 is not really standard and he did it just for fun, or rather so he can directly see whether a stream is AC3 or E-AC3. But the way splitters are *supposed* to work is to report both AC3 and E-AC3 as simple AC3. I understand that it would be nice to have the ability to handle AC3 and E-AC3 separately. But I think ffdshow can not rely on splitters signaling E-AC3 differently. However, isn't it possible in DirectShow for a decoder to request an audio frame before deciding whether it accepts the connection? Finding out whether an audio frame is AC3 or E-AC3 is really easy. So maybe you could add a solution like that to ffdshow. But honestly, IMHO it would be good enough if you didn't add any special handling for E-AC3, but treated E-AC3 just like normal AC3 tracks. Ok, then you can't use different decoders for AC3 and E-AC3. But I don't consider that a deal breaker cause the libav AC3 decoder works really well (at least the older revision I'm currently using in eac3to).

(2) MLP and TrueHD have slightly different headers, but ffmpeg/libav has a decoder which decodes both perfectly fine. Which means that it should be really easy for you ffdshow guys to add support for this decoder. FWIW, here are the GUIDs used by Nero and ArcSoft splitters to signal TrueHD/MLP:

Nero: {1E889BE7-B276-4064-9A39-160A06895B52}
ArcSoft: {4288B843-610B-4E15-A53B-43007FCFF614}

IIRC PowerDVD uses just another GUID for TrueHD/MLP, but I don't know that right now. There doesn't seem to be a standard GUID for TrueHD/MLP, but every splitter seems to use its own GUID.

(3) There's one complication with TrueHD tracks: Some of them are pure TrueHD while others have AC3 frames interweaved. The AC3 frames are not needed/used for TrueHD decoding, though, so they could (or maybe must) be filtered out before sending the data to the libav decoder. Not sure, maybe the libav parser can remove them, maybe not. In eac3to I'm only using the libav TrueHD/MLP decoder and I'm feeding it with my own parser/splitter which removes the AC3 frames.

Let me know if you need any more information.

[nautilus7]

An E-AC-3 decoder is required to handle normal AC-3 as well. An AC-3 decoder cannot decode E-AC-3, but it can recognize it as an AC-3 variant with an unknown bitstream id, so it will skip those frames.

Wrong. It is a HUGE problem because all the sample files that I have that contain E-AC3 will fail with regular AC3 decoders which all attempt to decode the audio. The specification is simply retarded and should be ignored. Splitter can detect E-AC3 and should pass that info along.

AC3 needs to be separate from E-AC3 because libav FAILS TO DECODE AC3 after 1-2 seconds so we need one filter for AC3 and one for E-AC3 (at least for now) ! :eek:
How does what you said agree with what Justin said? The E-AC3 decoder can decode AC3. It works with AC3 here.

[madshi]

Is there a way to detect if the stream is EAC3 when the input codec is AC3 ?
By analyzing the header
Something like this:

eac3 = ((((char*) audioframe)[5] & 0xf8) >> 3) >= 11;

[Mercury_22]

How does what you said agree with what Justin said? The E-AC3 decoder can decode AC3. It works with AC3 here.

Please test this (http://rapidshare.com/files/143875772/Test_LibavAC3.avi.html) with ffdshow's (svn 2111) libav for AC3 for at least 10 -20 sec and let me know how it works for you !
At my side after 6-7 sec I'm geting just noise instead of sound ! :helpful:

Edit: The 64-bit version of FFDShow svn 2111 it's CRASHING any AC3 or E-AC3 file when I'm using libav for AC3/E-AC3
Please help !
:helpful:

[nautilus7]

Well, I tested it. The file indeed doesn't decode correctly, but that's NOT a libav problem. There's something wrong with your file, because I remuxed it to matroska and plays perfectly! I am listening to it right now.

I have tested some more AC3 files with libav. 100% success here! Everything plays as it should be. (windows xp sp2 - 32bit)

[clsid]

AC3 in MKV works here too. But AC3 in AVI does not. And all files are 100% valid.

[nautilus7]

nevermind...

[_xxl]

But AC3 in AVI does not. And all files are 100% valid.
The same here. Only dca libav decoder seems to work properly.

[madshi]

You guys make it sound as if the decoder was at fault. But how can the decoder be at fault when decoding succeeds as soon as the AC3 track is muxed to MKV? I rather think that the decoder is not being fed correctly. IIRC some libav decoders depend on being fed exactly one frame at a time. Not all splitters do that.

Do you feed the libav decoder directly? Or do you run the data from the splitter through the libav parser before sending it to the decoder?

[clsid]

ffdshow is clearly at fault, not libavcodec. Data gets send directly to libavcodec afaik.

[_xxl]

I rather think that the decoder is not being fed correctly
True. Do you feed the libav decoder directly? Or do you run the data from the splitter through the libav parser before sending it to the decoder?
No, directly. That part to fed libavcodec audio decoder has to be implemented.

[albain]

I have built a new version with channel remapping. Please tell me if this is okay (for AC3, EAC3, AAC streams)

http://rapidshare.com/files/143901134/ffdshow_rev2111_20080909_dbt_ch_remap.exe.html

[_xxl]

AC3 and DTS no sound with:
http://x264.nl/h.264.samples/force.php?file=./premiere.hd.ts
http://samples.mplayerhq.hu/A-codecs/AC3/TomorrowNeverDies-2.1-48khz-192kbit.ac3
DTS:
http://samples.mplayerhq.hu/A-codecs/DTS/dtsac3audiosample.avi
AAC:
http://samples.mplayerhq.hu/A-codecs/AAC/file.avi
Doesn't load aac libav decoder.
More samples here:
http://x264.nl/h.264.samples/
http://samples.mplayerhq.hu/A-codecs/AC3/
http://samples.mplayerhq.hu/A-codecs/DTS/
http://samples.mplayerhq.hu/A-codecs/AAC/

[albain]

Something like this:

eac3 = ((((char*) audioframe)[5] & 0xf8) >> 3) >= 11;

It is not working : I get audioFrame[5] = 48 (0x30)
0x30 & 0xf8 = 0x30, 0x30 >> 3 = 6 which is not >= 11

So I get 6 for an EAC3 stream, and 8 for an AC3 stream

Something I missed ?

[clsid]

There is a difference in pin details:
http://i37.tinypic.com/mmst4y.png

[nautilus7]

AC3 and DTS no sound with:
http://samples.mplayerhq.hu/A-codecs/AC3/TomorrowNeverDies-2.1-48khz-192kbit.ac3

This sample starts with an incomplete frame. Run it though delaycut and it plays, BUT it is identified as 6ch (it's 2.1) and plays in faster speed. Though this behavior is completely different than playing as it is. :confused:

[Liisachan]

"audioframe" is a synch frame, maybe you'll need to find a sync word first

BYTE* pSync=NULL;
p=audioframe;
while(p<audioframe+framesize-6)
{
if( *p == 0x0B && *(p+1) == 0x77 )
{
pSync = p;
break;
}
++p;
}
if( ! pSync ) return;

bsid = ( pSync[ 5 ] >> 3 );
if( bsid == 0x08 )
{
//normal ac3
}
else //if( bsid == 0x10 )
{
//eac3
}

[albain]

Ok, the buffer has to be decoded first (which makes sense) and a sync flag detected

I wanted to detect it before sending the buffer to the decoder...

Thing to do : implement into FFDShow the decoding of the first frame before going further (this is what is done info mplayer).

[cc979]

compiling ffdshow-tryout svn:2112 using gcc-4.2.4 i get this error, does anyone else get this or have any ideas ?

cheers

In file included from ffdshow_imgfilters_all.cpp:7:
TffPict.cpp: At global scope:
TffPict.cpp:54: warning: 'AVPaletteControl' is deprecated (declared at ../ffmpeg/libavcodec/avcodec.h:2067)
In file included from ffdshow_imgfilters_all.cpp:19:
TimgFilterOSD.cpp: In member function 'virtual long int TimgFilterOSD::process(std::list<TfilterQueueItem, std::allocator<TfilterQueueItem> >::iter_list, TffPict&, const TfilterSettingsVideo*)':
TimgFilterOSD.cpp:501: error: no matching function for call to 'TsubtitleTextBase<char>::format(TsubtitleFormat)'
../subtitles/TsubtitleText.h:229: note: candidates are: void TsubtitleTextBase<tchar>::format(TsubtitleFormat&) [with tchar = char]
In file included from ffdshow_imgfilters_all.cpp:39:
avisynth/TimgFilterAvisynth.cpp: In member function 'void TimgFilterAvisynth::Tavisynth::process(TimgFilterAvisynth*, std::list<TfilterQueueItem, std::allocator<TfilterQueueItem> >::iter_list&, TffPict&, const TavisynthSettings*)':
avisynth/TimgFilterAvisynth.cpp:765: warning: deprecated conversion from string constant to 'char_t*'
avisynth/TimgFilterAvisynth.cpp:775: warning: deprecated conversion from string constant to 'char_t*'
make[1]: *** [ffdshow_imgfilters_all.o] Error 1
make[1]: Leaving directory `/home/User/svn/ffdshow-tryout/ffdshow-tryout/src/imgFilters'
make: *** [lib] Error 2

[Liisachan]

Ok, the buffer has to be decoded first (which makes sense) and a sync flag detected [...] Thing to do : implement into FFDShow the decoding of the first frame I'm not sure if I understand you correctly but you don't need to decode the AC3 stream to detect bsid. When decoded, the result is PCM or something? sync word and bsid are something that are in undecoded AC3 stream, not in the decoded ones, and if you can access the raw binary data, all you have to do is to check it starting from the beginning one byte by one byte until you find a sync word, and you're sure it is actually a sync word if the things after it make sense.

btw.. checking the 1st frame should help a lot. But theoretically each SyncFrame has a sync word and those header flags, meaning, even if the first frame is the normal ac3, for example 10th frame may have an eac flag

[CiNcH]

Haali Media Splitter exposes two media subtypes for files with E-AC3 audio:
MEDIASUBTYPE_DOLBY_AC3 and {0000EAC3-0000-0010-8000-00AA00389B71}

Currently, ffdhsow only looks at the first mediatpye it gets. If we can make ffdshow take the second mediatype in consideration, then it would be possible to separate EAC3 decoding from AC3 decoding. Allowing to use a different decoder for regular AC3.

Now ffdshow uses the same decoder for both AC3 and E-AC3. Not a big problem since both decoders are currently compatible with each other. But the annoying thing is that the Info&CPU page in ffdshow will always show that the input is AC3.

Things are really getting worse. If an application added a decoder selection for EAC3, this means that it would include all decoders with the AC3 subtype, whether they support EAC3 or not. My opinion is... define a proprietary subtype.

There is a lack of specification for new A/V formats in the DirectShow world. But it is IMHO better to define something proprietary instead, than to adapt something that just does not really make sense and increases confusion big time.

We are running into severe problems with AC3, EAC3, EAC3-BDMV (with compatible AC3 "core"), TrueHD, TrueHD-BDMV (again with compatible AC3 "core") if we can't distinguish decoders from each other.

[madshi]

I'm not sure if I understand you correctly but you don't need to decode the AC3 stream to detect bsid. When decoded, the result is PCM or something? sync word and bsid are something that are in undecoded AC3 stream, not in the decoded ones, and if you can access the raw binary data, all you have to do is to check it starting from the beginning one byte by one byte until you find a sync word, and you're sure it is actually a sync word if the things after it make sense.
Correct.

btw.. checking the 1st frame should help a lot. But theoretically each SyncFrame has a sync word and those header flags, meaning, even if the first frame is the normal ac3, for example 10th frame may have an eac flag
I've never seen a track which changes from AC3 to E-AC3 or vice versa. I'm not even sure if that is allowed. I wouldn't bother with that because it doesn't really occur in real life...

[ACrowley]

I have built a new version with channel remapping. Please tell me if this is okay (for AC3, EAC3, AAC streams)

http://rapidshare.com/files/143901134/ffdshow_rev2111_20080909_dbt_ch_remap.exe.html

No...still wrong mapping with libavcodec on EAC3 and AC3
Center Channel/Voices are coming aout of the back speakers

Also some AC3 Files are not working anymore. Output is corrupted

[albain]

OK, my mistake.

I have tested your code and actually I get the same result (i.e 8 for AC3 and 6 for EAC3 streams, not 0x10).
This is normal that I get the same result because I analyse the first frame and the first frame has always the sync word.

However your piece of code is useful for one reason : unlike ffmplay, directshow streams may not start from the beginning, so we have to wait for the first sync frame to detect its format.

One other thing that may explain why AC3 works for a few seconds with an avi file : clsid posted a difference in the input pin between MKV and AVI. AVI splitter detects wrongly the format.

In ffmplay the header is updated in the initialisation (it does not trust the given input format). In direct show we have no frame in input to do this. So the initialisation part should be completed in the decoding. Don't know if this is a problem.

[albain]

Decoding is always successful from libavcodec but the sound is garbled : I tried to feed the decoder only by sync frames, it is better but far from perfect.

In mplayer the buffers are :
1/ Input stream (not the buffer) is demuxed (ds_get_packet_pts). Don't know why it has to be done, this is the job of the splitter ?
2/ Decoded

In FFDShow the input buffer (from the splitter) is directly fed into the decoder and the output buffer is directly fed into the output pin.

[Liisachan]

@albain
The value 6 kind of makes sense too: a_52b.pdf says bsid=6 for Alternative Bit Stream (Annex D). Although, it's not what is called Enhanced AC-3 in Annex E, where bsid=16 (0x10).

PS.
from PDF
http://www.atsc.org/standards/a52.html

It is not necessary for decoders to be
aware of this alternate syntax in order to properly reconstruct an audio soundfield; however those
decoders that are aware of this syntax will be able to take advantage of the new system features
described in this Annex. This alternate bit stream syntax is identified by setting the bsid to a value
of 6.

...
D3.2 Legacy Decoder Processing
Legacy decoders do not recognize the alternate bit stream syntax, but rather interpret these bit
fields according to their original definitions in A/52B. The extra bit stream information words
(xbsi1e, xbsi2e, dmixmod, etc.) are interpreted as time code words (timecod1e, timecod1, timecod2e, and
timecod2).
As described in A/52B, the time code words do not affect the decoding process in legacy
decoders. As a result, the alternate bit stream syntax can be safely decoded without causing
incorrect decoder processing. However, legacy decoders will not be able to take advantage of new
functionality provided by the alternate syntax.

[nautilus7]

Do you feed the libav decoder directly? Or do you run the data from the splitter through the libav parser before sending it to the decoder?

Data gets send directly to libavcodec afaik.

No, directly. That part to fed libavcodec audio decoder has to be implemented.
What about this? Could this be the problem?

[albain]

From the sample code that I've seen (drdivx) the data is also sent directly (have not tested it, maybe it doesn't work either). But sample code based on directshow+libavcodec is rare.

And code from mplayer is quite complicated and is not the same issue : decoding data from a file is different from decodingdata from a buffer.

Any help would be greatly appreciated...

[MatMaul]

why not ask on the libav-user mailing list ?
https://lists.mplayerhq.hu/mailman/listinfo/libav-user

[Mercury_22]

Maybe this will help In FFmpeg, libavformat separates E-AC3 from AC3. I understand that ffdshow uses libavcodec for the decoding, but does it have a libavformat "splitter" for demuxing? If not, they can look at the code in libavformat/raw.c and libavcodec/ac3_parser.c to see how it's done.

[madshi]

From the sample code that I've seen (drdivx) the data is also sent directly (have not tested it, maybe it doesn't work either). But sample code based on directshow+libavcodec is rare.

And code from mplayer is quite complicated and is not the same issue : decoding data from a file is different from decodingdata from a buffer.

Any help would be greatly appreciated...
Where exactly is the problem? As far as I can see you have 2 options:

(1) Either run the DirectShow data coming from the splitter through the libav *parser* before sending it to the libav *decoder*. Or:
(2) Add a little buffer which collects data coming from the DirectShow splitter and then parse the data yourself and feed the decoder exactly one audio frame at a time.

Both should correct the problem. Probably doing (1) would be a lot easier cause it should work with any codec libav supports. So you wouldn't have to do anything AC3 or E-AC3 specific. You'd just need to figure out the libav parser API - which is not too complicated, IIRC. So instead of this:

DirectShow input pin -> libav decoder -> DirectShow output pin

... you'd do this:

DirectShow input pin -> libav parser -> libav decoder -> DirectShow output pin

The libav parser does the following tasks:
- searches for the first sync frame
- throws away all trash before the first sync
- splits the data into audio frames
- returns always exactly one valid and complete audio frame at a time

Some libav decoders need this careful data preparation, others don't. But it should never harm running any audio data which you want to decode with a libav decoder through a libav parser, first. That includes all those codecs ffdshow is supporting for a long time, e.g. FLAC. The libav parsers are specifically written to work well together with the libav decoders, of course. So you shouldn't lose anything by adding the libav parser in between...

[albain]

Maybe this will help

No, I tried to force the codec ID to EAC3 and this did not solve the problem.

This is really a problem because FFDShow is not a splitter so it receives the demuxed packets from the input filter.
MKV splitter does this well, but not AVI splitter.

From the other hand, internal liba52 decoder decodes AC3 streams well whereas going through libavcodec gives garbled sound for both AC3 & EAC3 streams.

The solution is not easy to implement : the input data that is received should be reformated into correct audio frames before feeding the decoder (or the renderer).

[Inventive Software]

It isn't a splitter, but for the purposes of these kinds of audio formats, maybe adding basic libavformat support before passing it to libavcodec wouldn't be such a bad thing.

[albain]

I agree, I wish it could be avoided : libavformat is not imported into FFDShow code.

I am currently doing this import, this is some kind of work...

[MatMaul]

isn't parser directly implemented in libavcodec (and not in libavformat) ?
if you look inside avcodec.h you can find a function avcodec_parse_frame

[CiNcH]

Hey guys,

something about H.264 and VC-1 frame parsing...

we finally found the problem with ffdshow not working with most splitters/demuxers in case of H.264 and VC-1 (actually it only works with Haali and the MPC MpegSplitter who have adopted the ffdshow approach).

The ffdshow filter uses ffmpeg/libavcodec to decode the two formats. What this library expects is the Elementary Stream (ES) of a complete H.264/VC-1 frame, in case of H.264 also called a NALU (so from startcode to startcode). ffdshow has taken this approach a level higher to DirectShow as it also expects a whole frame within one DirectShow sample on its input which is then sent directly to the library.
So with knowing this, a splitter could adopt that approach of course (like Haali and MPC did), but in this case the splitter would have to parse the H.264/VC-1 bitstream already. Our understanding of a splitter is that it knows nothing about the bitstream or ES data in particular, nothing about frames, nothing about NALU's, nothing about startcodes. A decoder for a specific format has to deal with that.

Casimir666 from the MPC-HC project implemented a parser within the MPC Video Decoder (which is also a ffmpeg wrapper and also adopted the ffdshow approach) that collects whatever ES data it gets from the splitter (whether it is a whole frame or 10K junks or 100K junks of ES data, whatever) and parses it, means, it builds the frames on its own (from startcode to startcode) and finally sends them to the ffmpeg lib. This basically works now.

Most of todays splitters output a certain amount of ES data per DirectShow sample not knowing what it actually is. In your case a splitter has to exactly know what it is to collect the data that makes up a frame which is a major design flaw IMHO and the reason why it does not work with most splitters for H.264 and VC-1.

Hope I was precise enough and you understood the problem. Casimir will hopefully check the parser code into the MPC-HC project reporistory (MpcVideoDec (http://mpc-hc.svn.sourceforge.net/viewvc/mpc-hc/trunk/src/filters/transform/mpcvideodec/?pathrev=784)). MPC MpegSplitter should have such a frame parser already as it sends out exactly what ffdshow currently needs..

[clsid]

Hmmm, doesn't ffdshow already have such code? There is a function called 'decodeH264SPS' in ffglobals.cpp. I don't know what it does exactly, but I suspect it is related to what you said.

[CiNcH]

Right, it synches to the stream there. But where does it get the char-Buffer from (hdr) from? Maybe it is the buffer it gets from the splitter? SPS and PPS may be in there but not the whole data that makes up the frame. But I think I did not get the whole NALU, SPS, PPS, frame thing completely yet.

[Shakey_Jake33]

This is a stupid as hell question, but also one that bugs me. I assume that if I enable Deinterlacing in ffdshow, it's smart enough to skip deinterlacing on files that aren't interlaced?

[albain]

isn't parser directly implemented in libavcodec (and not in libavformat) ?
if you look inside avcodec.h you can find a function avcodec_parse_frame

No there is not, this method is dedicated only to MPEG streams.

I have not given up on importing libavformat...

[LoRd_MuldeR]

This is a stupid as hell question, but also one that bugs me. I assume that if I enable Deinterlacing in ffdshow, it's smart enough to skip deinterlacing on files that aren't interlaced?

No it doesn't! There is no reliable way to find out whether a video is interlaced or not, except for your own eyes. So ffdshow cannot know it!

Even if the stream was encoded as interlaced, this doesn't mean that the content actually is interlaced :rolleyes:

Some kind of automated film/video detection could be implemented into ffdshow, but such algorithms don't work 100% reliable and eat CPU cycles...

[albain]

I have finished importing libavformat.

Before going further I would like your feedback on possible regressions on FFDShow

Here is the link of the patch (based on FFDShow revision 2111) :
http://damienbt.free.fr/FFDShow_libavformat.zip

Normally, there will be no interferences.

If I have the go, I will commit the code to SVN

Damien

[clsid]

Exact what changes did you make to the functionality of ffdshow?

Your patch seems to contain the whole codebase of libavformat. That is a lot of unneeded stuff. Why not include just those few parts that are actually getting used?

Currently the patch contains a lot of junk. Please don't commit this. Make smaller patches for individual changes. Lets take one step at a time, instead of jumping to the moon.

[albain]

Right, except that this is not that easy : most of libavformat files are dependant with each other.
Libavformat is just a library, nothing is used inside FFDShow code for now.

Importing small parts is more work than importing the library itself.

Anyway, I will continue on it until I get decoding working, after I will see if some parts can be removed...

[_xxl]

Don't hesitate to create a new branch ex: ffdshow-libavformat and commit your patches. It is easier then to have a lot of patches for trunk.

I can't apply your patch.

[clsid]

If it isn't used, then we won't need it and it should not be added.

You only want to parse an audio bitstream. Take should take no more than a couple hundred lines of code. Not a 2,3 MB patch.

[_xxl]

Changes from trunk could be reverted and re added when finished? I'm talking about libavcodec audio decoders that don't work.

[madshi]

If it isn't used, then we won't need it and it should not be added.

You only want to parse an audio bitstream. Take should take no more than a couple hundred lines of code. Not a 2,3 MB patch.
I think the point is that adding libav parsers to ffdshow would make sure that all the funny libav audio decoders could be easily made to work, including AC3, E-AC3, TrueHD, AAC etc etc. Without adding the libav parsers some of those decoders might not work properly cause some of them have strict requirements on how they want to get the data fed. Now a parser for just AC3 and E-AC3 would be only a couple of lines long. But it would be nice to make *all* libav audio decoders work.

Thoughts?

[albain]

Changes from trunk could be reverted and re added when finished? I'm talking about libavcodec audio decoders that don't work.

Which ones ? I am not aware of that. The only things that I have changed is adding a new available decoder for AC3/DTS streams.
Not using this decoder and select the former will (should) give the same behaviour as before.

@clsid : the patch is heavy you said, this is your point of view, but isn't it easier to update ffmpeg when you are closer to their code ?
Also, several decoders are not imported yet into FFDShow and they could be. So why not seing this as an "investment" for the future ? Although I agree that all the parts that will never be used should be removed...
Anyway, this is politics, I won't argue with it, this is the reason why I made a new "tryout" branch. The useful part of it will be merged when everyone will report it as ready...