LAV Filters - DirectShow Media Splitter and Decoders

[marsovac]

Hi nevcairiel,

I have an issue which may be occurring because I'm not really aware of the requirements for LAV about it, but here it goes:

When I setup the filtergraph and tell LAV to use video acceleration it simply does not do it.

I can see that the settings have been accepted if I open the tray icon, but the acceleration is not there (avcodec):



The weird things I'm doing that are not really standard are:

- LAV is not installed and regged on the PC but it is loaded from the ax with:

IntPtr proc = GetProcAddress(lavVideoDll, "DllGetClassObject");

I'm setting up the graph (part relevant to LAV) like this:

var filterGraph = m_graph as IFilterGraph2;

if (filterGraph == null)
throw new Exception("Could not QueryInterface for the IFilterGraph2");

var audioRenderer = InsertAudioRenderer(AudioRenderer);
if (audioRenderer != null)
{
if (_audioRenderer != null) Marshal.ReleaseComObject(_audioRenderer);
_audioRenderer = audioRenderer;
}

if ((System.Environment.OSVersion.Platform == PlatformID.Win32NT &&
(System.Environment.OSVersion.Version.Major == 5)))
VideoRenderer = VideoRendererType.VideoMixingRenderer9;

IBaseFilter renderer = InsertVideoRenderer(VideoRenderer, m_graph, 1);
if (renderer != null)
{
if (_renderer != null) Marshal.ReleaseComObject(_renderer);
_renderer = renderer;
}

ILAVAudioSettings lavAudioSettings;
ILAVAudioStatus lavStatus;
IBaseFilter audioDecoder = FilterProvider.GetAudioFilter(out lavAudioSettings, out lavStatus);
if (audioDecoder != null)
{
if (_audio != null) Marshal.ReleaseComObject(_audio);
_audio = audioDecoder;

if (_audioStatus != null) Marshal.ReleaseComObject(_audioStatus);
_audioStatus = lavStatus;

lavAudioSettings.SetRuntimeConfig(true);
hr = m_graph.AddFilter((IBaseFilter)audioDecoder, "LavAudio");
DsError.ThrowExceptionForHR(hr);
}

ILAVSplitterSettings splitterSettings;
IFileSourceFilter splitter = FilterProvider.GetSplitterSource(out splitterSettings);

if (splitter != null)
{
if (_splitter != null) Marshal.ReleaseComObject(_splitter);
_splitter = splitter;
splitterSettings.SetRuntimeConfig(true);
splitter.Load(fileSource, null);
hr = m_graph.AddFilter((IBaseFilter)splitter, "LavSplitter");
DsError.ThrowExceptionForHR(hr);
}

if (_splitterSettings != null) Marshal.ReleaseComObject(_splitterSettings);
_splitterSettings = (ILAVSplitterSettings)splitterSettings;

ILAVVideoSettings lavVideoSettings;
IBaseFilter lavVideo = FilterProvider.GetVideoFilter(out lavVideoSettings);

if (_video != null) Marshal.ReleaseComObject(_video);
_video = lavVideo;
if (_video != null)
{
if (lavVideoSettings != null)
{
lavVideoSettings.SetRuntimeConfig(true);

hr = lavVideoSettings.SetHWAccel(LAVHWAccel.HWAccel_None);

// check for best acceleration available
if (lavVideoSettings.CheckHWAccelSupport(LAVHWAccel.HWAccel_CUDA) != 0)
{
hr = lavVideoSettings.SetHWAccel(LAVHWAccel.HWAccel_CUDA);
}
else if (lavVideoSettings.CheckHWAccelSupport(LAVHWAccel.HWAccel_QuickSync) != 0)
{
hr = lavVideoSettings.SetHWAccel(LAVHWAccel.HWAccel_QuickSync);
}
else if (lavVideoSettings.CheckHWAccelSupport(LAVHWAccel.HWAccel_DXVA2Native) != 0)
{
hr = lavVideoSettings.SetHWAccel(LAVHWAccel.HWAccel_DXVA2Native);
}
else if (lavVideoSettings.CheckHWAccelSupport(LAVHWAccel.HWAccel_DXVA2) != 0)
{
hr = lavVideoSettings.SetHWAccel(LAVHWAccel.HWAccel_DXVA2);
}
else if (lavVideoSettings.CheckHWAccelSupport(LAVHWAccel.HWAccel_DXVA2CopyBack) != 0)
{
hr = lavVideoSettings.SetHWAccel(LAVHWAccel.HWAccel_DXVA2CopyBack);
}

#if DEBUG
hr = lavVideoSettings.SetTrayIcon(true);
#endif
}

hr = m_graph.AddFilter(_video, "LavVideo");
DsError.ThrowExceptionForHR(hr);
}

IBaseFilter vobSub = FilterProvider.GetVobSubFilter();

if (vobSub != null)
{
hr = m_graph.AddFilter(vobSub, "VobSub");
DsError.ThrowExceptionForHR(hr);
IDirectVobSub vss = vobSub as IDirectVobSub;
if (_vobsub != null) Marshal.ReleaseComObject(_vobsub);
_vobsub = vss;
InitSubSettings();
}

hr = m_graph.Connect(DsFindPin.ByName((IBaseFilter)splitter, "Audio"), DsFindPin.ByDirection(_audio, PinDirection.Input, 0));
if (hr == 0)
HasAudio = true;
else
HasAudio = false;


IBaseFilter dcDsp = FilterProvider.GetDCDSPFilter();
if (dcDsp != null)
{
if (_dspFilter != null) Marshal.ReleaseComObject(_dspFilter);
_dspFilter = (IDCDSPFilterInterface)dcDsp;

if (HasAudio)
{
hr = m_graph.AddFilter((IBaseFilter)_dspFilter, "AudioProcessor");
hr = _dspFilter.set_EnableBitrateConversionBeforeDSP(true);
hr = ((IDCDSPFilterVisualInterface)_dspFilter).set_VISafterDSP(true);
hr = m_graph.Connect(DsFindPin.ByDirection((IBaseFilter)_audio, PinDirection.Output, 0), DsFindPin.ByDirection(_dspFilter, PinDirection.Input, 0));
DsError.ThrowExceptionForHR(hr);
hr = m_graph.Connect(DsFindPin.ByDirection((IBaseFilter)_dspFilter, PinDirection.Output, 0), DsFindPin.ByDirection(_audioRenderer, PinDirection.Input, 0));

var cb = new AudioCallback(this);
hr = _dspFilter.set_CallBackPCM(cb);

object intf = null;
hr = _dspFilter.set_AddFilter(0, TDCFilterType.ftEqualizer);
hr = _dspFilter.get_FilterInterface(0, out intf);
_equalizer = (IDCEqualizer)intf;
_equalizer.set_Seperate(false);
}
}
else
{
if (HasAudio)
{
hr = m_graph.Connect(DsFindPin.ByDirection((IBaseFilter)_audio, PinDirection.Output, 0), DsFindPin.ByDirection(_audioRenderer, PinDirection.Input, 0));
}
}

bool subconnected = false;

hr = m_graph.Connect(DsFindPin.ByName((IBaseFilter)_splitter, "Video"), DsFindPin.ByDirection(_video, PinDirection.Input, 0));
if (hr == 0)
HasVideo = true;
else
HasVideo = false;

if (HasVideo)
{
hr = m_graph.Connect(DsFindPin.ByDirection((IBaseFilter)_video, PinDirection.Output, 0), DsFindPin.ByDirection(vobSub, PinDirection.Input, 0));
DsError.ThrowExceptionForHR(hr);
if (hr == 0)
{
int lc;
((IDirectVobSub)vobSub).get_LanguageCount(out lc);
subconnected = (lc != 0);
IPin pn = DsFindPin.ByName((IBaseFilter)splitter, "Subtitle");
if (pn != null)
{
hr = m_graph.Connect(pn, DsFindPin.ByDirection(vobSub, PinDirection.Input, 1));
((IDirectVobSub)vobSub).get_LanguageCount(out lc);
subconnected = (lc != 0);
}
hr = m_graph.Connect(DsFindPin.ByDirection(vobSub, PinDirection.Output, 0),
DsFindPin.ByDirection(_renderer, PinDirection.Input, 0));
}
else
{
hr = m_graph.Connect(DsFindPin.ByDirection(_video, PinDirection.Output, 0),
DsFindPin.ByDirection(_renderer, PinDirection.Input, 0));
}
}

I'm also using RuntimeConfig since the main purpose of doing this player is to make it a portable app with integrated codecs.

Can you plase help in enabling video accel?

If you need to run the app you can download a zip from this page (yellow button): http://msimic.github.io/MediaPoint/

Thanks!

[Stereodude]

Originally Posted by jkauff

In case you missed it, nev took a job with JRiver. He hasn't told us what he'll be working on for them, so we don't know whether LAV work will be affected.

Well congrats to Nevcairiel on the new job!

[NikosD]

Originally Posted by jkauff

In case you missed it, nev took a job with JRiver. He hasn't told us what he'll be working on for them, so we don't know whether LAV work will be affected.

I didn't know.

I don't use Media Center software, I read that JRiver is one of the best, if not the best.

But 50$ is a lot.

Anyway, I think Nevcairiel is committed to open-source development but a job is a job.

I see almost everyday nightly builds, so someone is writing code for LAV filters.

If not Nevcairiel, then who ?

[sheppaul]

Hendrik

[nevcairiel]

Originally Posted by marsovac

When I setup the filtergraph and tell LAV to use video acceleration it simply does not do it.

From a brief look at your code, it seems like you add DirectVobSub to the filter graph, yes?
DXVA2 Native only works if the decoder is directly connected to a compatible renderer (EVR), if there is anything in between, DXVA2 Native no longer functions.

If you need anything in between, you can use DXVA2 Copy-Back or any of the other modes, only DXVA2 Native has this restriction (because it keeps the image on the GPU, while all others copy it back to system memory once, which costs a bit of CPU time)

[annovif]

Hello Nevcairiel , can i know why i can use dxva2 native with madvr with an Ati 7750 ? I'm the only one or it is a normal thing. Thank you

[nevcairiel]

Why wouldn't you be able to use it?

[annovif]

Because you said this:
DXVA2 Native only works if the decoder is directly connected to a compatible renderer (EVR)
Thank you

[nevcairiel]

That doesn't mean its the only compatible renderer.

[annovif]

ok, thank you

[fluffy01]

I know that ffmpeg does not support MVC and as such, you can not support it for software decoding, but have you considered supporting MVC via DXVA?
According to the specs, it should be pretty straightforward to implement. See here for details.
I know that no free renderers or players currently support it, but that is most likely due to no free decoders have been able to decode it, so if you can get this to work in LAVFilters, I am sure, that we will soon see free players/renderers that will support it too

[andybkma]

Speex Audio : Is this something new? All of a sudden I have been seeing many cam vids coded with AVC for video and Speex for audio. Only PotPlayer can play it for me... could this speex be added to LAV Audio if possible please?

[LigH]

It is not exactly "new", another development by the Xiph team, a low frequency range codec for human speech (suitable for digital telephony).

Speex project website — obsolete since it is included in the Opus codec.
Other formats on RareWares

I wonder if your audio is really Speex, or rather a mis-identified Opus stream.

[nevcairiel]

LAV already supports both Speex and Opus.

[marsovac]

Originally Posted by nevcairiel

From a brief look at your code, it seems like you add DirectVobSub to the filter graph, yes?
DXVA2 Native only works if the decoder is directly connected to a compatible renderer (EVR), if there is anything in between, DXVA2 Native no longer functions.

If you need anything in between, you can use DXVA2 Copy-Back or any of the other modes, only DXVA2 Native has this restriction (because it keeps the image on the GPU, while all others copy it back to system memory once, which costs a bit of CPU time)

Yes I use xyVSFilter for subtitles.

I tried with copyback and it works only if I set that in the LAV configuration dialog.

The following line seems to do nothing:

hr = lavVideoSettings.SetHWAccel(LAVHWAccel.HWAccel_DXVA2CopyBack);

Perhaps SetRuntimeConfig doesn't like setting the HW accel?
(EDIT: wrong - I'm doing it already before adding to the graph)Or do I need to set that before adding the filter to the filter graph?

And I'm also wondering how does MPC-HC use DXVA with subtitles then?

[clsid]

And I'm also wondering how does MPC-HC use DXVA with subtitles then?

Because video and the rendered subtitle are merged by the video renderer.

@nevcairiel
The audio switcher in MPC-HC now supports mediatype re-negotiation, so there are no more obstacles for implementing a bitstreaming fallback

[marsovac]

Originally Posted by clsid

Because video and the rendered subtitle are merged by the video renderer.

You mean by the custom EVR presenter? I don't see EVR having two input pins...

[NikosD]

Yes.
You can't use subtitles with EVR.
If you want DXVA and subtitles you need EVR-CP, the default renderer of MPC-HC.

[mindbomb]

anandtech reporting maxwell supports partial h265 decode, which should be possible through cuvid.

[marsovac]

Originally Posted by NikosD

Yes.
You can't use subtitles with EVR.
If you want DXVA and subtitles you need EVR-CP, the default renderer of MPC-HC.

Thanks Nikos.

Is is even worth the hassle of development to attach my custom presenter to VSFilter? Especially since the presenter is not a filter at all...

Is there much difference between DXVA native and copyback?