LAV Filters - DirectShow Media Splitter and Decoders

[flossy_cake]

So then, what is the proposal? A toggle button in the player? We already know mpc-hc won't add it. mpc-be, maybe, maybe not. So basically, you'd have to do my two-player solution, where one of the players has lav in pre-scan mode and the other doesn't.

I was proposing it to be implemented at the audio decoder, eg. LAV audio. There are already options to "apply DRC on formats that support it", "normalize matrix", "clipping protection", and some other things like conversion to different sound channel layouts, audio delay etc. So I don't think this type of feature is outside the purview of the kinds of things an audio decoder normally does.

It's not a perfect solution though - some audio tracks can have a handful of brief loud sound effects that make very loud peaks that touches near 0db and so there is no headroom to gain volume. In practice it is most effective with TV shows with mostly dialogue and music.

[flossy_cake]

This is not something I plan on doing. Pre-scanning the entire file does not fit into any of the concepts of LAV. If you are dealing with an entire video file, this takes far longer then a few seconds, especially if you are on a slow HDD.

If you wanted to make it easier to implement, how about this: LAV Audio could simply read a gain value from a text file that the user creates. This offloads all the work onto the user to scan their entire library using their own tools. LAV would only have to read the gain value from the text file and apply it.

edit: for reference, a 90 minute movie takes me 10 seconds to scan using Avisynth's Normalize filter on an i5-4570 (4c/4t) @ 3.4Ghz

[huhn]

instead of asking a decoder to do that you should ask a audio renderer to do that.

an example would be to use gain from foobar not sure about the format but it should be open source and i'm pretty sure it can do that with mkv already.

[flossy_cake]

instead of asking a decoder to do that you should ask a audio renderer to do that.

That would be good too, but I don't know of any active authors who are at the cutting edge like nevcairiel is, let alone taking feature requests from users.

LAV Audio has a mixer so applying a gain amount from a text file should be relatively straightforward

https://i.lensdump.com/i/TpbHxx.png

I'm currently using Avisynth's Normalize and am happy with it except it doesn't have an option to save the normalisation amount to a txt file for next time the file is played, so I have to wait around 10 seconds every time I play a video (unlike Normalize2 (avisynth.nl/index.php/Normalize2) plugin which does offer that feature, but is 32-bit only).

I'm trying to figure out if there is an Avisynth debug tool to make Normalize (64-bit) output its calculated gain amount to a txt file, as this would allow me to automate the scanning of my library with ffmpeg command line like ffmpeg -i "c:\OutputNormalizedGainToTextFile.avs" -f null NUL

[Sunspark]

So basically, MPC-BE's audio renderer as the only currently maintained directshow renderer.

LAV has introduced in the past a feature because it wasn't too much trouble, made sense for the time, and is still technically outside of what a pure decoder does. Namely, the mixing option. It didn't originally have that.

Flossy's idea for the end-user to pre-scan is an interesting one. Subtitles after all are textfiles in the .srt format. I suppose the extension for this one could be called .dbe or .dbm. Should be technically possible to have a checkbox "Use pre-scan dB value if present". Then LAV doesn't have to do anything at all, just accept a value it is given and apply it to the mixer. The splitter already ingests external .srt files.

If one wanted to get really fancy with it, instead of a single number being presented, down the road it could have the option of timestamps for each scene, with a dB value for each.

[flossy_cake]

If one wanted to get really fancy with it, instead of a single number being presented, down the road it could have the option of timestamps for each scene, with a dB value for each.

Yeah, for those cases where dialogue is quiet and explosions are very loud. But then we're getting into compression territory which IS a distortion of the audio (albeit a subjectively desirable one).

Avisynth does actually have a programmable compressor ("compander" in SoxFilter (http://avisynth.nl/index.php/SoxFilter)) but it's currently broken, but it looks like pinterf might be working on fixing it - https://forum.doom9.org/showthread.php?p=1983841#post1983841

SoxFilter("compand 0.3,1.0 6:-90,-90,-70,-70,-60,-20,0,0")

# attack time: 0.3 seconds
# decay time: 1.0 seconds
# soft-knee amount for smoothing between curve points: 6db
# curve point 1: set -90db to -90db (no change)
# curve point 2: set -70db to -70db (no change)
# curve point 3: set -60db to -20db (40db gain)
# curve point 4: set 0db to 0db (no change)

Resulting curve looks something like this but with 6db of soft knee smoothing between each linear segment

https://i3.lensdump.com/i/Tpw6Zc.png

[tebasuna51]

If you wanted to make it easier to implement, how about this: LAV Audio could simply read a gain value from a text file that the user creates. This offloads all the work onto the user to scan their entire library using their own tools. LAV would only have to read the gain value from the text file and apply it.
Like each movie need a different gain the best option for you is recode the audio at your taste (than can be different for other users) and let LAV Audio do their job: decode the audio like it is.

Also remember than some audio can be passed passtrought (without change it) to TV or AVR decoders. The recode option is your best solution.

[nevcairiel]

Should be technically possible to have a checkbox "Use pre-scan dB value if present". Then LAV doesn't have to do anything at all, just accept a value it is given and apply it to the mixer.

And where does that single value come from? LAV getting a value and applying it is one thing, but having to find and read the value on disk somewhere is actually not how a decoder operates. It receives audio data on an input pin. It does not really care where its from. It does not read the file its from, nor does it necessarily know which file its from.

The player should really manage that file, and at that point it can also just offset the volume control of the audio renderer appropriately - don't even need a custom audio renderer, just one that has a volume control.


The splitter already ingests external .srt files.


It does not. That is a player feature and managed by them.

[flossy_cake]

And where does that single value come from?

The only way I can think of would be to match the filepath of the currently playing file to an entry in the database file. This appears to be how Kodi keeps track of what files you've played, and how MPC-HC knows the resume point for each media file. Ideally it would use a hash of the file as its unique ID to prevent collisions, but that is too slow to calculate on large media files, unless perhaps it was a hash of only the audio track, and the audio track is suitably small to keep processing time to a minimum (some audio tracks may be large). edit: or maybe just hash a practicably small number of data bytes from salted addresses inside the file.

Anyway, I remembered that Crendking made an Avisynth directshow plugin that could be used to automatically apply Normalize(0.98) globally to every file without any per-file user intervention, so I'll probably just use that instead.

I also made a script that automatically generates .avs files for all the media files in a top level folder (cloning the entire folder structure but with .avs files instead of media files) but it still requires quite a bit of intervention to ensure a compatible Avisynth source filter is used, and some manual scaling to correct for sources with nonsquare pixels, whereas Crendking's solution manages that automatically.

[senseipiledrive2]

I'd like some guidance to figure out if this an issue worth looking into or not. I am using the latest x64 version of lav 77.2 in combination with mpc-be. I use the default install options. All behavior and how the lav splitter, audio and video icons appear like normal when I open a file for playback.

**However, every time when I boot into Windows 11 (latest), I noticed in the task manager, under processes, there always seems to load a blue LAV filter icon with "Windows Host Process (Rundll32)" is written next to the icon. It uses 0 Cpu and 0.9MB of memory every time I look. It's the only rundll32 windows host process listed. Nothing seems to happen when I close it, files still open for playback like normal it appears. Is this normal? Should I disable it or just leave it be? I have ran SFC scannow and DISM restore with no change. Thanks all!

[ryrynz]

Should I disable it or just leave it be?

Sounds to me like it's related to some third party app that's installed on your system. I would think there's nothing wrong and you should leave it be, but if you have things installed on your PC you're not using uninstall them.

[senseipiledrive2]

I'm not sure how to figure out what put it there, but my guess was the LAV installer package. It does show a Blue LAV graphical icon next to the "Windows Host Process (Rundll32)." When I click open file location it just takes me to the system folder pointing me literally to just Rundll32, so maybe I'm not finding what installed it in the correct way. The only media player I have installed is MPC BE 1.6.6 x64 I use the latest nightly builds.

[el Filou]

This could happen if an application that loads with Windows uses videos and it then loads LAV Filter for them.
For example, Logitech Options does that on my HTPC when I open it because it embeds videos to show the different touchpad gestures, that it apprently uses DirectShow to render.
You could try SysInternals Process Explorer, which shows which app is the parent process.

[glorp]

The tray icons and their options are hosted by rundll32. Turn off the tray icon when LAV loads and you should see it disappear.

[mzso]

Hi!

I noticed that LAV icons disappear when I change DPI scaling percentage, and restart explorer. So I guess this is an issue with LAV?
Most tray icons do reappear, with Facebook's Messenger being the other exception I noticed.

I do this because I use my display both by sitting at the desk a from a couch from a greater distance. And without restarting explorer things appear wrong.

[mzso]

**However, every time when I boot into Windows 11 (latest), I noticed in the task manager, under processes, there always seems to load a blue LAV filter icon with "Windows Host Process (Rundll32)" is written next to the icon. It uses 0 Cpu and 0.9MB of memory every time I look. It's the only rundll32 windows host process listed. Nothing seems to happen when I close it, files still open for playback like normal it appears. Is this normal? Should I disable it or just leave it be? I have ran SFC scannow and DISM restore with no change. Thanks all!

Did you defrag your drive as well? As long as we're doing random things. :)

Anyway. Did you look the process in question via Process Explorer (https://learn.microsoft.com/en-us/sysinternals/downloads/process-explorer)?

With that you can look at various properties of a process (most useful would probably be the command line and the running threads) and see the parent processes.

[el Filou]

I noticed that LAV icons disappear when I change DPI scaling percentage, and restart explorer. So I guess this is an issue with LAV?Could be because there is a shell extension that displays thumbnails for video file formats not supported natively by Windows, and that extension uses DirectShow and therefore the tray icons appear when you browse a folder with those types of videos in it in thumbnail view, Explorer load LAV through that shell extension to generate the thumbnails?
I remember this happening a long long time ago under XP with Haali splitter.
Edit: Anyway, I don't think it's LAV's fault or that nev can do anything about it, it's probably the app's fault. My media players that properly stop/close the playback graph don't leave LAV icons in the tray.

[nevcairiel]

Tray Icons are not designed to be permanent. They only show up when a LAV component is loaded. And they should disappear when the component is unloaded properly. Some things kill it a bit aggressively though, especially shell extensions.

[mzso]

Tray Icons are not designed to be permanent. They only show up when a LAV component is loaded. And they should disappear when the component is unloaded properly. Some things kill it a bit aggressively though, especially shell extensions.

LAV is not unloaded though. Playback is not necessarily interrupted at all. But the tray icons do disappear.

@el Filou

It's not that. I mean the icons that appear during playback. They disappear even though they should continue to be present.

[Sunspark]

Hi Nev, I think I have found an issue which is current as of 0.77.2 on Intel.

h264, L3.1 file, hw acceleration does work with intel quicksync, does not work with dxva-cb or dxva-native (displays avcodec instead).
vp9 files, no acceleration.

LAV does support dxva acceleration of L3.0 files, just not the L3.1 one so I'm thinking it might be a bug. The L3.1 file is dxva2 accelerated with the MPC video decoder.

Not sure about vp9 files. In DXVA Checker it does list "VP9_VLD_Intel: DXVA2/D3D11, SD / HD / FHD / 4K". Not sure if the "_Intel" is something special, there are no other rows for VP9 present while other codecs have duplicate rows with and without the "_Intel" part.

[nevcairiel]

"Level" is a largely irrelevant property. There is likely some other property about that file that prevents hardware acceleration.

A blind guess would be that its "baseline" profile, which is not supported in hardware, only "constrained baseline" is supported.

[Sunspark]

Well, I've uploaded the file here so you can download it. Not a big file, 12 megs: h264l31 problematic file (https://www.dropbox.com/s/xzfz2flsf7ljsqr/luffy.mp4?dl=0)

I feel if it's hw-decodable with quicksync and mpc, it should also be w/ the dxva decoders.

[nevcairiel]

Like I suspected, its baseline profile. Baseline is _not_ hardware compatible. Any given file might work, but there is no guarantee that it would, depending on what features it uses. If anything, other decoders sending baseline to a hardware decoder are bugged, not LAV.

I prefer to have some files not use hardware and all be decoded correctly, rather then use hardware and get decoding errors.

[Sunspark]

Hi Nev, I have a question about D3D11 hw decoding.

With madVR, if I set it to use D3D11 (I am aware that it is a wrapper around D3D9), set LAV to use D3D11-Automatic and then observe that the GPU usage % is almost double that of DXVA2-CopyBack in one of the D3D9 presentation modes, is this due to the Intel video driver, decoder or the renderer? It's confusing because when I tried D3D11-CopyBack via selecting the device, the GPU usage % stayed at the same number that Automatic had. I thought it would be another doubling, but that wasn't the case, the number was the same. So it feels like madVR is running in copyback in d3d11 despite displaying "native" on the filter. Should I see a difference between Automatic and CopyBack with this renderer in D3D11?

Thanks.

[el Filou]

With discrete GPUs, native DXVA (= Automatic in LAV D3D11) puts less load on the system (in theory) than copyback as you don't have to pass decoded frames back and forth over the PCIe bus, but as this is an iGPU you're already using system RAM for everything, so the difference can be much less or even none.

The higher load in D3D11 LAV decode and/or D3D11 madVR presentation mode could be a quirk of the Intel driver.

Note that you don't necessarily have to use D3D11 presentation in madVR to use D3D11 decoding in LAV, they are independent.
Did you try D3D11 in LAV with D3D9 presentation in madVR?

[Sunspark]

Good suggestion to try. I tested that just now.. left presentation mode in D3D9 overlay and put the decoder into D3D11. As measured with gpu-z, GPU usage was 14-15%. With DXVA2-CopyBack it was 12-13%. So for me, the d3d9 copyback decoding is slightly more efficient than d3d11 native decoding. With D3D11 presentation on, the GPU usage w/ D3D11 decoding becomes 21-22%. Putting it into dxva2-native and leaving it in d3d9 overlay the same number results, 21-22%. DXVA2-Native with D3d9 Windowed makes it 24-25%. Flipping it around to D3D11-Windowed and DXVA2-Native makes it even worse, 28-29%.

This is a normal file, h.264 high L4.0 1920x1080. No image scaling being performed, just 2-tap chroma shader scaling.

If I try using mpcvr in d3d11 and the mpc decoder in d3d11, I get a peak of 12% gpu usage, same result if I use LAV as the decoder. Based on that, I'm thinking it's probably not that much the driver, and more so the renderer.

[huhn]

these number doesn't mean anythign without the frequency which can vary drastically on old intel iGPU.
25% can mean less GPU usage then 12% sounds odd but makes total sense if you think about it.

[Sunspark]

I have the gpu set for maximum performance, so it says 900 MHz for the GPU clock. It seldom goes to 950, will go a little lower if it starts to overheat, but otherwise stays at that fixed number all the time. These recorded percentages are all at the same fixed clock.

I think it likely comes down to non-optimized paths for the current OS. 0.92.17 is almost 5 years old and the presentation code paths predate Windows 10. The D3D11 wrapper was implemented in 0.88 before Windows 10 was RTM. madshi did say back then more than once that Windows 8.1 was the best version to be using.

[j7n]

Please add a master volume slider to Audio Mixing. To avoid clipping, the level of the downmix usually needs to be dropped only by 3 to 6 dB and not as much as Normalize Matrix would do, which assumes that all channels could fire in sync.

[htnut]

Question: I'm sure my issue is a "fringe case" but hopefully there is a solution.

I'm running JRiver MC and playing BD/UHD content launched from menus, bistreaming audio over HDMI. I presume it is the job of LAV to connect the exculsive WASAPI audio connection to the audio driver.

Without boring you with all the details of why...

If I pause playback, and another process "breaks" the audio connection to the audio driver - when I come back to resume playback I get no audio. I can "stop" and then endure the menus to "play" again (this reestablishes the audio connection). But, I would really love to find some process to cause the audio connection to re-establish without having to "stop" playback.

Is there some "behind the scenes" process I can perform to cause the filter to re-establish this "exclusive" audio stream?

Thank you!

[e-t172]

I presume it is the job of LAV to connect the exculsive WASAPI audio connection to the audio driver.

It is not. LAV is an audio decoder, not an audio renderer.

Presumably it's JRiver that's doing the audio rendering. It would be more appropriate to ask your question in a JRiver forum.

[htnut]

It is not. LAV is an audio decoder, not an audio renderer.

Presumably it's JRiver that's doing the audio rendering. It would be more appropriate to ask your question in a JRiver forum.

Thanks. I did start there, but the admins at JR aren't interested in my dilemma (I do admit it's a fringe case). They basically say to just deal with it, stop and restart through the menus.

I imagine/hope there's some workaround that I could do to cause the (audio renderer, apparently) to reconnect to the audio driver after an interruption.

[Sunspark]

Maybe you should try a different player.. I didn't test with a DVD, but I paused an audio stream in 1 player in exclusive mode and waited awhile and it did not release the audio lock.

[htnut]

Maybe you should try a different player.. I didn't test with a DVD, but I paused an audio stream in 1 player in exclusive mode and waited awhile and it did not release the audio lock.

No, it's not that it releases the lock, it's that another process I invoke breaks it. I'm hoping to find some way to "reconnect" that audio without having to stop and restart the player. It's in a dedicated Home Theater and trying to keep make the transition between what I'm doing smooth and consistent.

[Sunspark]

That's not how exclusive mode works. There is no "another process breaks it". Your app is releasing the process.

[Klaus1189]

I think so, too. MPC-BE has an option to release it when in idle (default is off)
https://drive.google.com/file/d/1Jjccf3Kbz2hVkCO07Y3Wt8PrtpcDs-3i/view?usp=sharing

[Sunspark]

If switching the player is not an option, you can also use shared mode as well, then the focus loss won't be an issue. The sound will not be bad just because it is in shared mode and it has value in that it can enable other functionality in the soundcard options which won't be exposed/processed in exclusive.

Of the two players which I use regularly for different purposes, one I use in exclusive and the other I use in shared.

[e-t172]

If switching the player is not an option, you can also use shared mode as well, then the focus loss won't be an issue. The sound will not be bad just because it is in shared mode and it has value in that it can enable other functionality in the soundcard options which won't be exposed/processed in exclusive.

I second this, but one caveat is you can't play Atmos that way - it has to be bitstreamed, which requires exclusive mode.

[nevcairiel]

You can actually bitstream even through DirectSound, the windows built-in mixer does take care to deliver bitstream formats untouched, or mute them.

[el Filou]

Even if you find a workaround I'd still ask the JRiver developers to fix that, because for example ReClock has no problem restarting the audio stream in exclusive mode if I pause playback and then play something else in another app.
(Edit: or maybe it's an issue with the audio driver?)

[mzso]

Hi!

I've been trying deinterlacing options. And I noticed that if I use LAV software options the video is noticeably less smooth compared to letting madVR deinterlace. But the latter delivers poor image quality. I'm quessing it uses some trashy built-in deinterlacing method of the radeon driver.

Furthermore watching the same channel on TV seems even smoother. Only this seems completely smooth.

I don't get frame drops or such, but even so the difference in smoothness is noticeable. I guess this is normal? Can I get better deinterlacing somehow? That both looks good temporally and image quality wise.

[htnut]

You can actually bitstream even through DirectSound, the windows built-in mixer does take care to deliver bitstream formats untouched, or mute them.

Thank you. Apologies I wasn't more clear in my original post... the process that "breaks" JR's exclusive hold is a script of my own that disables and immediately re-enables the audio driver. Then, my second application can play audio... but then I want to be able to resume JR (without having to "stop" playback and restart).

Shared mode would be excellent but JR's seetings seem to REQUIRE "use exclusive mode" or else Atmos won't bitstream.

Maybe the support I need from JR is to allow bitstreaming without exclusive mode? Their support page also states "If you disable exclusive.. you must use DSP mode to resample..."

https://wiki.jriver.com/index.php/Exclusive_Access

Thank you for your suggestions!

[htnut]

As a follow up here, this is a comment from a JRiver administrator (attached).

nevcairiel, if I understand your comment correctly, LAV should be able to bitstream (atmos) even without "exclusive" mode. Is there something we can tell the JRiver folks about that can cause MC to bitstream without exclusive mode?

Thanks again.

[Sunspark]

Attachments don't get approved here, link the image to a remote site instead.

[jmone]

@htnut, I don't know who that screen shot refers to from JRiver (as we can not see it) but.... if it is from Hendrik then you may not know that this is the same person as nevcairiel.

[nevcairiel]

The only option to use shared-mode bitstreaming is through DirectSound. Otherwise with your essentially forced driver reset, you will be on your own, thats not what a normal system ever does, sorry.

[htnut]

Ok, Thank you. I'll give up on trying to reset the audio exclusive stream without "restarting" playback.

But, in JRiver MC choosing a DirectSound device won't bitstream (no Atmos). So, either I have some configuration wrong or JRiver has something interfering with the ability to run "shared mode" (which is suggested in the adminstrator "awesome donkey" saying that "bistreaming requires exclusive mode").

Thanks again.

[Sunspark]

@htnut You should do more experimentation, for example, if your machine has an integrated GPU, test to see if the bitstreaming works with that instead of the card you're using now.

[htnut]

I did try on a different machine, but experience the same thing. If I set JRiver to bitstream over HDMI - it only works if "exclusive mode" is checked in JR.

And, consistent with this experience... when I asked JRiver I got that response posted above where I'm told "bitstreaming requires exclusive mode".

But here, I'm told bistreaming should work in shared mode.

So, it seems one of three things:

1. JRiver's reply is wrong - and I simply need to learn how to configure my machine correctly (or use some other hardware)

or

2. JRiver is correct - bitsreaming requires exlcusive mode to bitstream for some reason so I just need to deal with it. (but I sure trust nevcairiel stating that shared mode will [should!] bitstream over DirectSound)

or

3. JRiver does require exlcusive mode to bitstream BUT should not have to. If this is the case, I'm not aware how to ask them to correct this.

[nevcairiel]

WASAPI in shared mode is rather cursed. Its not that helpful. For WASAPI you should always want to use Exclusive.
DirectSound is always in shared mode and requires picking a different output device entirely.