LAV Filters - DirectShow Media Splitter and Decoders

[Orf]

nevcairiel, got it, thanks again

[Aleksoid1978]

nevcairiel,
In a custom renderer I'm trying to implement support for getting D3D11 texture samples from LAV video decoder directly. I'm utilizing your ID3D11DecoderConfiguration and IMediaSampleD3D11 interfaces for that purpose.

Connection media subtype NV12 is used and LAV Video decoder correctly calls GetD3D11AdapterIndex (I return 0) and
ActivateD3D11Decoding so I have d3d11 interfaces. At the point the DS graph is built LAV video decoder reports 'd3d11 native' is used.

But at point when first sample arrived LAV video decoder get switched to software decoding and first sample is never IMediaSampleD3D11 but IMediaSample instead. At this time it also reports 'avcodec' is used.

I try to investigate why it happens and found that it is not hardware related because:
1) if I return error from ActivateD3D11Decoding LAV video decoder switched to 'd3d11 cb native' and it works fine
2) if I build graph with madVR as a renderer at first and then disconnect LAV Video decoder from madVR and connect it to my renderer 'd3d11 native' starts work with my renderer just fine too

I seems to me that LAV video decoder expects from renderer to do something else, which madVR does and my renderer doesn't.
I try to find what it might be examining LAV source code but didn't find anything that gives me a clue.

So I need your favour with that, can you please shed some light on this?

You can check here how work with D3D11 Native Decoder
https://github.com/Aleksoid1978/VideoRenderer

[nevcairiel]

No latest nightly build?..

I didn't update the build machine when I upgraded my security keys (so the upload failed), it should be working again now.

[Orf]

You can check here how work with D3D11 Native Decoder
https://github.com/Aleksoid1978/VideoRenderer
nevcairiel already solved the problem I have, but I will check your sources too, thanks

[mzso]

Hello!

I've not been paying attention for a good while. Is it possible to decode and render correctly (for headphones preferably) Atmos/DTS:X nowadays with LAV? Or any other free software?

[huhn]

HRTF mixing is not a feature.

[halcom]

Anyone else had trouble downloading lav0.80.zip (the LAV ffmpeg fork) from
https://gitea.1f0.de/LAV/FFmpeg
Been trying for about a week now. Just get the spinning wait icon with nothing downloaded. I let it run all night and still nothing.
Tried firefox & chrome but both behave the same. Is there an alternate location to download it?
Any help greatly appreciated!

[LigH]

This is a source code repo. I cannot find any ZIP archive there.

[halcom]

https://gitea.1f0.de/LAV/FFmpeg

- In the upper right press "tags"
- takes you to a page where you should be able to download zips of the sourcecode for all the versions.
- normally you can simply right click and do a save but the download never starts.

[LigH]

You are right.

The same does not work for the "Download ZIP" link in the menu dropping out of the blue "<> Code" button. Let me try using git clone ... worked.

Mirror: gitea_LAV_FFmpeg.7z (https://www.mediafire.com/file/ypgs6gbwa8ve3o7/gitea_LAV_FFmpeg.7z/file) (182 MB)

[nevcairiel]

That should hopefully work again, for some reason there were thousands of dead requests that were blocking it.

[Orf]

nevcairiel,
few more things I noticed about d3d11 native:

1) when video coded with h264, and 'H.264' checkbox unchecked in 'Codecs for HW decoding' decoder status says it falls back to 'avcodec' but it get stuck then and never outputs any samples

2) the description of ID3D11DecoderConfiguration mentions:
// To facilitate dynamic switching of the adapter used for decoding, the
// renderer should disconnect the decoder and re-connect it. At that
// point the decoder should query GetD3D11AdapterIndex() again and
// create a new decoder on the new device, as appropriate.

but if I disconnect decoder from renderer and try to reconnect it back connect fails with VFW_E_BUFFERS_OUTSTANDING error. Happens only in d3d11 native mode. As far as I see decoder calls GetAllocator and then instead of NotifyAllocator it calls BreakConnect.
This is not implemented yet?

[halcom]

LigH: Thank you, just downloaded from mediafire !

Nev: Just checked it. Link is working now.

Many thanks to you both!

[JarrettH]

Just wondering something.....is the hardware deinterlacing better quality than me choosing one of the software options? I was probably going to go with bwdif. Thanks.

btw, I see there were some updates to bwdif back in 2023, likely nev saw that.

[huhn]

a properly working hardware decoding is massively better then the lavfilter software deinterlacer.

but they don't always work or at all.

the nvidia one sometimes works and the AMD one is pretty much broken forever at this point making it sometimes worse then bilinear bob.

the software deinterlacer on the other hand always work as advertised and are fine for true interlaced content with no mixed stuff or telecine.

[CruNcher]

The Problem on the hardware side is the software stack compatibility and Nvidias way of doing things and keeping secrets in their Layers and API functions.

A lot of Nvidias work also comes from Vanguard on the IP

[Sunspark]

AMD's is broken, don't even try, use software if you have an AMD GPU.

Intel's does work and you can use it, but it's a deinterlacer, not inverse telecine so you still need software for that. If you're receiving e.g. satellite broadcast, then it's perfectly fine to use the deinterlacer. Old film content on DVD, you want software IVTC.

[JarrettH]

a properly working hardware decoding is massively better then the lavfilter software deinterlacer.

but they don't always work or at all.

the nvidia one sometimes works and the AMD one is pretty much broken forever at this point making it sometimes worse then bilinear bob.

the software deinterlacer on the other hand always work as advertised and are fine for true interlaced content with no mixed stuff or telecine.

Ah ok, I'm trying it (NVIDIA GT 1030) and all modes are broken :D Wasn't something broken with D3D11 for a long time? I can't remember if that was resolved. Anyway, bwdif is just fine for the odd time I need to play a DVD.

[CruNcher]

Could you check if some proprietary stack also shows playback problems in case of Nvidia always a good interop test is Cyberlink PowerDVD, if it also fails for your use case it becomes interesting.
you could and should also try other player implementations like zoomplayer, potplayer ect it could be a very specific combination of setup causing detection and proper VPU Hardware decoding problems.
But the first test should be a complete different stack and here Cyberlink PowerDVD is the first choice.

[lvqcl]

Ah ok, I'm trying it (NVIDIA GT 1030) and all modes are broken :D Wasn't something broken with D3D11 for a long time? I can't remember if that was resolved. Anyway, bwdif is just fine for the odd time I need to play a DVD.

Do you use MadVR? It doesn't support deinterlacing in D3D11 mode.

[JarrettH]

Do you use MadVR? It doesn't support deinterlacing in D3D11 mode.

I do! Yes, that must be it!

[littleD]

Hello, I am testing new laptop and decoding capablities. My Apu is ryzen 8645HS

I am quite satisfied with the results, however among my test video samples I found one that is not playable in hardware acceleration. This is 60FPS VP9 video 8K. I presume the problem is in resolution 8 192 x 4 608 that goes a bit over standard. I have no HEVC sample with same resolution to test.

Typical 8k, Vp9, 60 fps file is decoded at 75% usage of codec engine.

Would it be possible to around this maybe in lavfilters or registry in windows? I see headroom for bit higher res decode.

[clsid]

Give link to sample file.

[littleD]

This is just youtube video of nature. It was in webm format but remuxed and cut to matroska. I have investigated some hifi cameras are using this resolution, probably YT in previous times passed resolution as is with no check.

https://limewire.com/d/zMMyg#HdcrHsAwzi

[huhn]

works on RDNA 4 so maybe...

[nevcairiel]

There is no hardcoded resolution check, so if a file doesn't work, the driver probably rejected it.

[RealSnoopyDog]

I have one question related to "IAMStreamSelect" and subtitle streams: how can I identify the subtitle "stream" that disables subtitles? For LAV splitter, IAMStreamSelect::Info returns the name "No subtitles" and the LCID with the value 0xFFFFFFFF . Would it be sufficient to query for LCID == 0xFFFFFFFF for this?

[nevcairiel]

There isn't really any totally positive identification markers, since the interface is limited. But its always going to be the last subtitle stream, it'll always have that LCID value, and always have the same name.

[RealSnoopyDog]

Thank you, this helps :)

[mark0077]

May I ask all of the resident experts here for a small piece of advice. When downmixing content to stereo, in the past I have used values of 0.71 for center and surround mix, as well as 1.58 for the lfe mix (based on some maths from previous posts which I definitely don't understand :) ). But when asking a few of the ais like chatgpt/gemini, they are all adamant that if the stereo system is bass managed, that the correct figure for the lfe is 0.316, but for a stereo system with no bass management only, use 1.58.

I couldn't find mention of this on any posts nor could the AI so I can't back it up with any humans. For my system (KEF LS50 Wireless II which takes the 2 channel input and sends the bass to a SVS PB16 Ultra, the AI is as mentioned adamant that because the KEF in this case has its own bass management, using 1.58 would be classed as boosting the bass by 14db. Does any humans care to give their inputs on this one :)

LFE at 0.316 → –10 dB (reference)
LFE at 1.58 → +4 dB (boosted)
Difference: ~14 dB more bass with 1.58 compared to 0.316

[Sunspark]

AIs hallucinate, and they only know what they are trained on. They won't say "I don't know". They will give you a number.

I use center 1.00, surround 0.71, lfe 1.58.

This is being sent to an old 2.1 system, but as 2.0, the system itself does the crossover to the subwoofer.

I only use 1.58 because I have the sub. If I didn't, it'd be at 0. Center at 1 is OK because ac-3 5.1 downmix to stereo is too quiet. It won't clip. Plus I still need to use compression on top of that.

The mixer is not a gain control as I understand it.

If your system like mine is doing crossover splitting in the system then by leaving out the LFE by setting it to 0 there is nothing for it to cross-over in those frequencies. According to the specs, my system's crossover frequency for the subwoofer is <=150 Hz and >=4 kHz for the satellites (to tweeter afaik).

I am not an audio expert but I believe this is fine based on my reading.

[tebasuna51]

Each user can downmix at your taste, but remember Dolby do not recommend use the LFE channel at all to downmix 5.1 to 2.0.

Remember LowFrequencyEffects is not a channel for all low frequencies, only for some Efects, all channels can have low frequencies and send them to subwoofer if audio equipment have it.

[Sunspark]

I was looking at the ITU recommendations which does talk about LFE, but doesn't provide a recommendation on it just that DVD video will have LFE at -10 and if you keep it, it should be +10.

I started asking AI where is this 1.58 value from? It said it's +4 dB that assumes stereo summing for the rest of what would normally be 10 dB, and that it's a psychoacoustic compromise based on real-world listening. Supposedly you get basically the same value whether you do math in linear space or RMS power so it seemed to work out.

Going further, apparently Dolby does have a lfemixlev so that's interesting too.

Digging further I found something https://datatracker.ietf.org/doc/html/rfc7845#section-5.1.1.5 where they provide coefficient values for downmix to stereo including the LFE (as well as variations without it) so that's interesting but I don't think it can be safely used here because my assumption is that LAV uses a hardcode of 1.0 for FL and FR.

/FL \
/ \ / \ |FC |
|L| | 0.529067 0.374107 0.000000 0.458186 0.264534 0.374107 | |FR |
|R| = | 0.000000 0.374107 0.529067 0.264534 0.458186 0.374107 | |RL |
\ / \ / |RR |
\LFE/

Have I managed to confuse myself further? Yes. ffmpeg does allow for the use of many custom formulas, but we can't use that here.. just those 3 sliders which are constrained to one interpretation.

The above values are probably good, but the downside I think is that everything will be quiet, and you'd have to turn up the volume on the speakers and then back down again because system sounds/games, etc. would be too loud otherwise.. and my speakers don't have a dial or a number indicator to show what volume they're at.. :(

[tebasuna51]

The above values are probably good, but the downside I think is that everything will be quiet,...

In the doc you link say also:

Matrices for 3 and 4 channels are normalized so each coefficient row sums to 1 to avoid clipping. For 5 or more channels, they are normalized to 2 as a compromise between clipping and dynamic range reduction.

The above values must be the half to avoid clipping, for that each user can do other compromise, for me it is not use LFE and limit the surround coeficients.

[Sunspark]

I understand the Windows mixer doesn't use the same values, apparently they are licensed and they didn't want to pay the fee to Dolby, so they came up with their own values. It's not bad, just different.

Maybe one day a Front slider will appear and then it will be possible to try all the different formulas people have come up with.

Update: Currently experimenting with 1.0 C, 0.5 S, 0 LFE. Seems to be pretty decent for dialogue clarity.

[nevcairiel]

Just assume front is 1.0 and then re-calculate the other values based on that, all you do is change the overall volume when you also change front, which shouldn't be required.

Although the matrix listed above can't be input anyway, because it also cross-mixes the rear channels (Rear Left into Front Right, etc), which isn't currently an option. If you want more fancy mixing, I would recommend to use an actual audio mixer.

[tebasuna51]

...Seems to be pretty decent for dialogue clarity.

Just for that, some automatic downmix let the dialog volume to low.

[Siso]

With LAVFilters-0.80.0-13 I can't preview x265 in subtitle workshop 6.3.4. LAVFilters-0.80.0-12 works fine.

[nevcairiel]

With LAVFilters-0.80.0-13 I can't preview x265 in subtitle workshop 6.3.4. LAVFilters-0.80.0-12 works fine.

This should hopefully be resolved in the latest already.

[Siso]

This should hopefully be resolved in the latest already.

It is working now, thank you.:thanks:

[nevcairiel]

LAV Filters 0.81

LAV Splitter
- NEW: Support for APV (Advanced Professional Video) video streams
- Fixed: VVC in MP4 without CTTS did not play properly
- Fixed: AV1 streams with invalid metadata did not get handled correctly

LAV Video
- NEW: Support for decoding APV (Advanced Professional Video)
- Changed: Re-designed pixel format mapping to be based on a heuristic rather then hard-coded, improving support for rarely-used formats
- Fixed: Deinterlacing of 4:2:2 videos was broken in 0.80

LAV Audio
- Fixed: Bitstreaming TrueHD from Blu-rays with seamless branching could result in out-of-sync audio, or a noticeable gap


Download: Installer (both x86/x64) (https://files.1f0.de/lavf/LAVFilters-0.81.exe) -- Zips: 32-bit (https://files.1f0.de/lavf/LAVFilters-0.81.zip) & 64-bit (https://files.1f0.de/lavf/LAVFilters-0.81-x64.zip)

Finally getting another long-standing TrueHD bitstreaming issue resolved. Otherwise the usual, FFmpeg updates and tackling anything that was immediately actionable and reported.

As always, please report issues, especially regressions, in as much detail as possible, and with a sample file.

[StainlessS]

Thankyou kind sir.

[clsid]

Some crash fixes:
https://github.com/clsid2/LAVFilters/commit/434100dda9686005bdc55b9543519a249104d25f
https://github.com/clsid2/LAVFilters/commit/827e1f483955ddd72577edc2d2d38b0c42f82818
https://github.com/clsid2/LAVFilters/commit/675db83543b0b76cf939fcaeb795069d86e9e9ca
These can happen when accessing tray icon during loading.

And in libavutil\tx.c
https://drdump.com/Problem.aspx?ProblemID=1016430
Disabling those blocks under CONFIG_SMALL‎ fixes it. Seems like unnecessary debug/trace code for release builds.

[manolito]

Hi clsid,

interesting bug report, but frankly as a non-dev I do not really know what to do with it...

I am unable to compile my own version of LAV Filters, so I probably need to wait for nevcairiel to publish a fixed version. But anyways, thanks a lot for your findings...


Cheers
manolito

[manolito]

@clsid

Sorry for my ignorance about Github...
On the first page of this thread I found a link to "nightly test version", and the last commit is this one:
LAVFilters-0.81.0-3.exe

Am I correct that this is your commit? Anyways, I installed it, and all my taskbar problems (disappearing icons) were gone. I would never have guessed that, because I thought that as long as LAV Filters is not used by some software that it would not affect the Taskbar. Now I know better, and this commit made my life a lot easier.


A big Thank You :)

Cheers
manolito

[clsid]

No whatever problem you had is unrelated to LAV Filters.

[clsid]

@nevcairiel
DVD hardware decoding is broken.
https://drdump.com/Problem.aspx?ProblemID=1011044

Time to finally remove this option? It isn't enabled by default. Any hardware from the past 15 years can decode DVD resolution video without breaking a sweat.
Maybe also ditch quicksync as well.

[Sunspark]

Quicksync has adaptive hardware decoding of interlaced video.

If the code paths aren't broken why remove it? People say it is old, but it seems to work fine.

[Klaus1189]

What is "adaptive hardware decoding". Do you mean deinterlacer or indeed decoding?

[Sunspark]

Both.. it would be used with stuff like television broadcast mostly I imagine, perhaps DVD as well. But, it's not an inverse telecine so for DVD I haven't been using it. I did test the functionality though with a TV clip, and it does deinterlace using hardware (which is awesome when one considers AMD for example pretty much has completely broken deinterlacing functionality).

I use quicksync to this day because I always felt there was something "different" about how colour or motion looked with dxva2-cb. I also use overlay output since I have it. I was told that it's not very likely there's a difference but I am unable to capture it in a screenshot. I just remember a long time ago there was a pipeline graph that shows quicksync processes colour at a different stage of the pipeline than dxva2-cb, so it's not impossible that there is a subtle difference.

I looked it up, quicksync talks directly to the intel video firmware while dxva2 goes through the windows driver stack. I also observe that Intel has a new SDK (OneVPL) which continues to use quicksync and ffmpeg is supporting both the old and new.. so..

I get simplicity, but the decoder isn't something like an old forked version of ffmpeg 2.0. It's just the interaction with the firmware as I understand it.