LAV Filters - DirectShow Media Splitter and Decoders

[nevcairiel]

More interestingly, the latest NVIDIA driver now supports the new HEVC DXVA decode profiles for 444 (and maybe also 422? I don't have blackwell and don't plan on getting one). Now its not only Intel anymore. Maybe I'll get around to supporting them properly sometime.

[TheGameMaster]

Yep. It's in their new build. It's working fine on my system. Their build in version of LAV works just as well as the stand alone / external version. The stand alone LAV doesn't have it yet. I honestly didn't notice any difference between d3d11 cb external LAV and d3d12 cb internal MPC-BE, running MadVR and XySubFilter. I also run ffdshow tryouts for AAC audio decoding - as it does a much nicer job of faking dolby suround/atmos - but I use LAV (or MPC-BE's built in LAV) for the rest of audio formats. I'm not sure what the differences are between MPC-BE's build in LAV and external LAV.

[lvqcl]

I'm not sure what the differences are between MPC-BE's build in LAV and external LAV.

MPC-BE doesn't have built-in LAV filters.

[ryrynz]

I'm not sure what the differences are between MPC-BE's build in LAV and external LAV.

You mean MPC-BE's internal filters, LAV is specifically Nevcairiel's filters that are also built in to MPC-HC but not BE. Source, Video and Audio filters in BE are created by the MPC-BE team and are updated as part of the ongoing player updates. As far as differences go, it comes down to features mostly with LAV offering more options for a typical power user.

[jmone]

More interestingly, the latest NVIDIA driver now supports the new HEVC DXVA decode profiles for 444 (and maybe also 422? I don't have blackwell and don't plan on getting one). Now its not only Intel anymore. Maybe I'll get around to supporting them properly sometime.

Sounds great, given nvidia has joined intel in the 422 club. While it works, I'm not that keen on turning my source AVC/10-Bit/422 footage into 420 just to make it compatible for LAV/HW Accell.

[v0lt]

Can someone please post a screenshot of the first tab of DXVA Checker 4.7.0 (https://bluesky-soft.com/en/DXVAChecker.html) for Nvidia RTX 50x0.

[Sunspark]

Can someone please post a screenshot of the first tab of DXVA Checker 4.7.0 (https://bluesky-soft.com/en/DXVAChecker.html) for Nvidia RTX 50x0.

I don't have it, but they did list:

GPU MPEG2 H.264 HEVC Main HEVC Main 444 HEVC Main10 HEVC Main10 422 HEVC Main10 444 HEVC Main10 Ext HEVC Main12 HEVC Main12 422 HEVC Main12 444 VP9 Profile0 VP9 10bit Profile2 AV1 Profile0 AV1 12bit Profile2 420
GeForce RTX 5080 4K 8K 8K 8K 8K 8K 8K - 8K 8K 8K 8K 8K 8K -

So just HEVC Main10 Ext and AV1 12bit Profile2 420 with no support.

[v0lt]

I don't have it, but they did list:
The presence of NVDEC decoder does not mean the presence of the same D3D11 and DXVA2 decoder.

[Klaus1189]

OK, I made a screenshot of my 4090 and my 5090 for direct comparison:
4090 (https://drive.google.com/file/d/1xal32BNo6bRnWlctefRUwG70sXVKvaWY/view?usp=drive_link)
5090 (https://drive.google.com/file/d/1EkSd7vtQZ9BZ8UxFGDIuRhvKs12gEG1-/view?usp=drive_link)

If you need to know anything else, just let me know :)

[nevcairiel]

Can you hover over the two 422 devices and let us know which output formats they list? I would expect Y210 and Y216 like Intel, but just want to be sure.
I have a 4090 myself for 444, and they use Y410/Y416 there as expected.

Not that I expect many renderers to actually support native DXVA/D3D11 with these formats. Especially Y210/Y216 is often troublesome, and likely will not be supported in JRVR for native, for example (although if you send it through D3D11-VPP, it might be able to transform it to something more useful, but I don't use that for quality reasons), so copy-back it'll be for those.

[Aleksoid1978]

MPC VR already support all this HEVC Rext format for D3D11.

[v0lt]

OK, I made a screenshot of my 4090 and my 5090 for direct comparison:
4090 (https://drive.google.com/file/d/1xal32BNo6bRnWlctefRUwG70sXVKvaWY/view?usp=drive_link)
5090 (https://drive.google.com/file/d/1EkSd7vtQZ9BZ8UxFGDIuRhvKs12gEG1-/view?usp=drive_link)
Thank you. Some formats (for example H.264 10 bit) are still only available in NVDEC.

[Klaus1189]

Can you hover over the two 422 devices and let us know which output formats they list? I would expect Y210 and Y216 like Intel, but just want to be sure.
I have a 4090 myself for 444, and they use Y410/Y416 there as expected.

Not that I expect many renderers to actually support native DXVA/D3D11 with these formats. Especially Y210/Y216 is often troublesome, and likely will not be supported in JRVR for native, for example (although if you send it through D3D11-VPP, it might be able to transform it to something more useful, but I don't use that for quality reasons), so copy-back it'll be for those.


Here you go:

https://drive.google.com/file/d/1pAOG7Y5dyoqfiQd4qbEOnl0uRHas8XuY/view?usp=drive_link

https://drive.google.com/file/d/1ZIj1jx874PHP3RfNDn-2bM-LNQfW8y60/view?usp=drive_link

[nevcairiel]

Thanks, as expected they use the same recommended format as Intel, so I can use that as well.

[LigH]

LAV Filters crash on AVIs with ASV1 (ASUS V1 raw) codec encoded videos. Will provide a small sample in a github issue (https://github.com/Nevcairiel/LAVFilters/issues/641).

PS: I had an old ffdshow installed, with ASV1 decoding support disabled, that caused the crash. When I enabled decoding support, MPC-HC and MPC-BE played the video. But that makes me wonder why they rely on a system-installed decoder at all. Peeking into libavcodec-61 DLLs of LAV Filters included in MPC-HC, as well as the mpc-be64.exe, I found ASV1 FourCCs mentioned. So I would assume they contain a matching decoder?!

[j7n]

What ffmpeg voodoo is happening with this E-AC-3 sample? There is a pop on the first playthrough. If I rewind and play again, there is no pop anymore. I don't have any tools to verify if the stream is corrupted. There is only one decoder and all PC players are affected by this. Liba52 seems abandoned.

https://pixeldrain.com/u/cUvWnMx7

There was another problem with unclean start of E-AC-3 inside MP4, with samples from 1537..1728 repeated at about 0..192 in the overlap of the first block, but I cannot reprouce it with MPC-Homecinema. Maybe it is fixed in recent ffmpeg. The same EC3 elementary stream is played cleanly with a fade-in ~128 samples as it ought to. The modern way of mastering music is to cut the track right before sound starts.

[tebasuna51]

j7n, your file seems ok, without errors, without pops in my players (begins with 0.2 sec. of silence the decoded wav):
File: C:\Users\Pepe\Downloads\whatifwe.eac3
Size: 850944 bytes
----------------------------------------- First Frame Info
StrmTyp .....................: 0 (0=Ind, 1=Dep, 2=AC3)
SubStreamID .................: 0
FrameSize....................: 3072 bytes (768 Kb/s)
SampleRate ..................: 0 (48000 Hz)
NumBlksCod ..................: 3 (6 Blocks)
Audio coding mode (acmod) ...: 7 (3/2 - L, C, R, SL, SR)
Low frequency effects channel: 1 (Present)
Version (bsid) ..............: 16 (Enhanced sintax)
Dialogue normalization ......: -18 dB
Dynamic Range gain ..........: 9.18 dB
Mixing metadata exist (ignoring rest)
------------------------------------------- Rest of Frames
EAC3 Fram SR TY ID BS BL CH LF DN Compre FrSize Bytes Before
--------- -- -- -- -- -- -- -- -- ------ ------ ------------
2 0 0 0 16 3 7 1 18 11.77 3072 0
...
277 0 0 0 16 3 7 1 18 3.34 3072 0
----------------------------------------- Revised EAC3 Info
Dyn. Range min/max : 1.49/12.30 dB
Frames Tot/Ind/AC3.: 277 / 277 / 0
Duration ..........: 8864 ms (0 h. 0 m. 8.864 s.)
------------------------------------------------- End Info
277 independent frames without errors, without dependents frames, only different compre.

[j7n]

I have a discontinuity at sample position 187136 on the first playback and also when decoding with one of the last ffmpeg XP builds. I use the last release of MPC-HC 2.4.2 x64. The discontinuity doesn't appear in Dolby Reference Player.

https://i.imgur.com/kcO5XyG.png

[tebasuna51]

You are right, seems a decoder problem.
I can't listen the click (near 3.9 sec.) but I can see it in decoded wavs.

Cavernize, LWLibavAudioSource (L-SMASH-Works-r1253.0.0.0) decode it fine (first decode).
ffmpeg (n7.1-186), FFAudioSource (ffms2-5.0-msvc) and BestAudioSource (R11) have the worse decode (2 and 3 decode)

I don't remember how use GraphEdit to save wav files to be sure if LAV filters make the same.

[j7n]

The sound is quite obvious between the words "to" and "be" on the right side (left front channel is clean). Adobe Audition decodes fine. I didn't realize that it could decode too.

[tebasuna51]

The sound is quite obvious between the words "to" and "be" on the right side (left front channel is clean).

Sorry, I listen the high frequencies at 3.905 but not the clik.
Try then GraphEdit to verify if LAV filter decoder is also affected:

[v0lt]

so I've had OpenAI's O3 model try to analyze the differences and it came up with 3 important points, 2 of which should be simple to implement, please consider enhancing this functionality.
I think the main problem with AI is not that it is stupid, but that it does not understand its stupidity. That is why AI writes its answers with unperturbed confidence.
This is where another problem arises. Naive users believe the stupid AI and start writing its answers on the Internet as a good solution. This creates even more garbage on the Internet. I hope that in the future, publishing AI "answers" on many forums will be considered flooding.

[j7n]

Sure, with 32-bit from MPC-HC. https://i.imgur.com/JTSgI5T.png

A second sample, pourque.ec3, problem at 41728 right at the start. Only on the first playback. Maybe ffmpeg has processor-specific code, so it only happens sometimes?

https://pixeldrain.com/u/AFU54eoF

[tebasuna51]

I found how write wav file in avi container:
18945
And using LAV Audio decoder 0.79.2 in first play (x0) -> error, second play (x1) -> OK
18946

[nevcairiel]

Hi Nev, I know it's not the most thrilling of issues to fix, but can you please take look at this, I believe this issue can be partially resolved with a few small tweaks:
https://github.com/Nevcairiel/LAVFilters/issues/652

None of the statements that so called "Intelligence" provides are actually correct or event remotely useful. The code it cites does not exist, the options it recommends are wrong or their meaning distorted or entirely inverted, and the correct options that would do the tasks it suggests are set by default.

As a general rule for anyone ever reading this, any AI interaction will instantly get your post dismissed and probably ridiculed. Developers should know better.

[clsid]

I found how write wav file in avi container:
18945
And using LAV Audio decoder 0.79.2 in first play (x0) -> error, second play (x1) -> OK
18946
The whole spectrogram looks a bit different, not just the broken segment.

I suggest putting the sample files on FFmpeg bug tracker (https://trac.ffmpeg.org/query?status=new&status=open&status=reopened&col=id&col=summary&col=status&col=type&col=component&col=version&col=time&col=changetime&report=1&desc=1&order=changetime).

[j7n]

I have done so (https://trac.ffmpeg.org/ticket/11578).

[Blight]

None of the statements that so called "Intelligence" provides are actually correct or event remotely useful. The code it cites does not exist, the options it recommends are wrong or their meaning distorted or entirely inverted, and the correct options that would do the tasks it suggests are set by default.

As a general rule for anyone ever reading this, any AI interaction will instantly get your post dismissed and probably ridiculed. Developers should know better.

I sincerely apologize. I tried many different ways to understand the performance difference compared to libVLC and the answers O3 provided made sense, I should have dug deeper to verify the code exists.

The issue itself still stands. When performing a seek action on a remote Emby server or switching subtitle tracks, with libVLC the action is near-instant while in DirectShow using LAV Source, it can take 1-30 seconds. Is there any way I can help you debug this issue, taking into consideration that I limited understanding of the code itself? Is there a build that outputs a debug log that may help trace the issue?

[clsid]

A track change triggers a flush in the splitter. That is needed to acquire data packets for the new subtitle track, which can be located before current parsing position.

But perhaps an optimization could be done. If the splitter stores the end timestamp of the last processed sub packet, then it could decide if it is possible to skip the flush of audio and video queues, if last packet end time is in the past.

[max79]

@Nevcairiel

I'm noticed that LIVE-streams(HLS or MPEG-TS) that have subtitle track in it, lead a Graph Studio to crash right after trying to render a Video pin. Example(This site works only in Russia. Please use VPN):

https://rt-ural-sur-htlive.cdn.ngenix.net/hls/CH_R02_OTT_TNR7/variant.m3u8

Stream contain this subtitle track:
MEDIATYPE_Subtitle {E487EB08-6B26-4BE9-9DD3-993434D313FD}
MEDIASUBTYPE_UTF8 {87C0B230-03A8-4FDF-8010-B27A5848200D}
FORMAT_SubtitleInfo {A33D2F7D-96BC-4337-B23B-A8B9FBC295E9}

p.s. I have no no crash only if I render a Subtitle pin first and only then render a Video pin.

Any feedback would be appreciated.

[Blight]

Is line 21 (closed captioning) in live streams supported by LAV Filters?

There are no sub/line21 pins exposed. I verified, a CC track exists on this stream using libVLC.

[nevcairiel]

Is line 21 (closed captioning) in live streams supported by LAV Filters?

There are no sub/line21 pins exposed. I verified, a CC track exists on this stream using libVLC.


https://github.com/Nevcairiel/LAVFilters/blob/master/include/LAVVideoSettings.h#L405

[Blight]

https://github.com/Nevcairiel/LAVFilters/blob/master/include/LAVVideoSettings.h#L405

Thank you 🙏

Is there any downside to always enabling the CC output pin creation?

[Blight]

https://github.com/Nevcairiel/LAVFilters/blob/master/include/LAVVideoSettings.h#L405

I enabled the pin, found a stream that is verified to support closed captions, but trying to connect the line21 pin to the Microsoft's "Line 21 Decoder 2" filter results in an error #0x80040207 (VFW_E_NO_ACCEPTABLE_TYPES).

I noticed the output pin on LAV Video decoder exposes "MEDIATYPE_DTVCCData" instead of the expected "MEDIATYPE_AUXLine21Data" which may be the cause.

Is there something I'm not taking into consideration?

[nevcairiel]

Its designed to be CEA 708 DTV CC data, not the older Line 21 data.

[Blight]

Its designed to be CEA 708 DTV CC data, not the older Line 21 data.

I tried DirectVobSub/xySubFilter and neither one supports this format.

I even tried passing it through the SAMI (CC) Parser filter with no success (error 0x80004002)
https://learn.microsoft.com/en-us/windows/win32/directshow/sami--cc--parser-filter

Can it be used with any existing DirectShow components? I searched but it's not very clear.

[Sunspark]

You could try MPC-BE's decoder with EVR-CP to verify whether or not it works with the media you're working with and go from there. "Line 21 Decoder - enabled for Microsoft DTV-DVD Video Decoder and EVR-CP". Not sure if you also need to check on the line 21 checkbox in the DVD settings (present in both HC and BE) for streams, or if the checkbox is only for DVD.

[Blight]

You could try MPC-BE's decoder with EVR-CP to verify whether or not it works with the media you're working with and go from there. "Line 21 Decoder - enabled for Microsoft DTV-DVD Video Decoder and EVR-CP". Not sure if you also need to check on the line 21 checkbox in the DVD settings (present in both HC and BE) for streams, or if the checkbox is only for DVD.

Nev wrote earlier that he's translating the line 21 data to another format, that's why I'm unable to get it to work, it's incompatible with Microsoft's line 21 decoder.

I tried to see if MPC-BE works any better, it doesn't detect the closed captions at all.

For reference, I'm not playing a DVD, this is a live stream.

[nevcairiel]

The data is not being translated, its exposed as-is from the video bitstream if present (typically from SEI NALs). Basically the data FFmpeg exposes as AV_FRAME_DATA_A53_CC (eg. "ATSC A53 Part 4 Closed Captions.", also known as CTA-708 (formerly EIA-708 and CEA-708))
The data format consists of a number of cc_data_pkt (definition of which can be seen in the CTA-708 specification), the number can be obtained from buffer size / 3 (as each cc_data_pkt is 3 bytes, one control byte and two CC data bytes).

I have no idea if any existing filter exists that can handle it - make your own if needed. I know for a fact that the LAV data is being used for subtitles successfully with custom filters.
Microsoft abandoned DirectShow a long time ago. The Microsoft provided filters are stuck in a state from 15 years ago or so.

This format is the digital successor to Line21 data, which is also mandatory in US broadcasts, so should always be available.

[Blight]

The data is not being translated, its exposed as-is from the video bitstream if present (typically from SEI NALs). Basically the data FFmpeg exposes as AV_FRAME_DATA_A53_CC (eg. "ATSC A53 Part 4 Closed Captions.", also known as CTA-708 (formerly EIA-708 and CEA-708))
The data format consists of a number of cc_data_pkt (definition of which can be seen in the CTA-708 specification), the number can be obtained from buffer size / 3 (as each cc_data_pkt is 3 bytes, one control byte and two CC data bytes).

I have no idea if any existing filter exists that can handle it - make your own if needed. I know for a fact that the LAV data is being used for subtitles successfully with custom filters.
Microsoft abandoned DirectShow a long time ago. The Microsoft provided filters are stuck in a state from 15 years ago or so.

This format is the digital successor to Line21 data, which is also mandatory in US broadcasts, so should always be available.

Thank you for the detailed information,

Unfortunately, my expertise lies in user interfaces and less so in filter development, which is why I hoped for something off the shelf I could integrate rather than telling people to switch to the libVLC engine.

Would it be possible to translate this format to a subtitle format on the output pin?

[Blight]

In order to better understand why seeking with LAV is so much slower than with libVLC, I've had the network connections tracked using Wireshark.

The same exact stream was played (a TV episode on a remote Emby media server) using both libVLC and LAV. In both cases, 4 seeks were made.

The wrapper format is this case is MKV.

And here are the results:

Seeking with libVLC
https://docs.google.com/spreadsheets/d/1tmWxsOn-pyWHSkLDrF15Kx_wr-7WUeSU79Q7AHFzzVk/edit?gid=716414888#gid=716414888

Seeking with LAV
https://docs.google.com/spreadsheets/d/1myw2Dk_LujD3hq1ZGh8CahN2AdRtZbQKit3xdKkVgP8/edit?gid=1650206653#gid=1650206653

As you can see, seeking with libVLC performs 1 HTTP/GET command per seek.

Seeking with LAV performs about 45 HTTP/GET commands per seek.

Can this be resolved?

[masodes]

Hi there,

Is there any possibility to make a captured 3D Frame Packed stream compatible wit LAV Video decoder?
The capture card's directshow filter outputs a 1920x2205 stream with MEDIATYPE_Video and MEDIASUBTYPE_YUY2 for example.
How has this to be transformed for LAV Video decoder to accept it as input and recognize it as 3D and pass it on to the renderer?
The renderer, madVR in this case, should then be able to output again as 3D Frame Packed via HDMI.


Edit: So far I've built a directshow filter that is working as expected in graphstudionext.
But in MPC-HC I'm not able to block the Smart Tee Filter, which makes my own filter useless.
With PotPlayer there is always the Built-in Video Codec/Transform after my filter, so it is not working either.
Does anybody have an idea, how to solve this?

Edit2: Found out that MPC-HC is forcing Capture Sources to use Smart Tee Filter. Removed this restriction now in my own build.

[Blight]

Edit: So far I've built a directshow filter that is working as expected in graphstudionext.
But in MPC-HC I'm not able to block the Smart Tee Filter, which makes my own filter useless.
With PotPlayer there is always the Built-in Video Codec/Transform after my filter, so it is not working either.
Does anybody have an idea, how to solve this?

If you'd like, I can try helping you get it working in Zoom Player.

[masodes]

If you'd like, I'll can try helping you get it working in Zoom Player.

Thank you, but I already found kind of a solution 👍

[clsid]

@nevcairiel
Can you build an updated MinGW pack for GCC 14.3 or 13.4?

[Siso]

There is some sort of corruption visual glitches decoding x264 with LAVFilters-0.79.2-41 and LAVFilters-0.79.2-47. Last working version for me is LAVFilters-0.79.2-31. https://thumbs2.imgbox.com/9e/03/mhAgdxvp_t.png (https://imgbox.com/mhAgdxvp) https://thumbs2.imgbox.com/b7/57/Plp6k6Xz_t.png (https://imgbox.com/Plp6k6Xz)

[manolito]

Just to clarify:

You are talking about this:
Nightly testing builds are available here:
https://files.1f0.de/lavf/nightly/

Is this correct?


The latest stable build is this one:
0.79.2 - 2024/04/08


Cheers
manolito

[nevcairiel]

Going to need a sample file. Also software or hardware decoding? I have no observed anything like that.

[Siso]

MPC-BE latest stable build, madvr 112, LAVFilters-0.79.2-47. Happens only with x264 with one movie: D3D11 Native in LAV video decoder. Sample: https://1cloudfile.com/N16v

[nevcairiel]

Thanks for the sample, I can reproduce the issue. Will take a look.

Edit:
Fixed