LAV Filters - DirectShow Media Splitter and Decoders

[STaRGaZeR]

It would be very nice to be able to render the subtitles on the resized-for-display-by-MadVR video instead of needing to resize them along with the rest of the video. Or just render them at the display resolution instead of the video resolution, pass them to MadVR and let it do the overlay. :D

The key, in my opinion, is rendering the text at the display resolution so the subs never get resized at all. Allowing the user to pick the resize algorithm for picture based subtitles (DVD, bluray) would be nice too.

Of course, I am not sure how the new subtitle interface works so I will wait and see. :)

The problem with all those solutions is that you depend on madVR. And sometimes that's not possible, or it isn't desired. In most cases, to get good quality subs without depending on exotic stuff video has to be upscaled to screen resolution, then subs rendered to that upscaled video, then delivered to the renderer.

BTW good work on LAV, nev. DVD, mixer, nice additions :)
Waiting for full blown mixer and quality text subs :D

[Xaurus]

nev (or anyone else),

Is banding like this normal with 4:2:0 material?
http://www.imagebam.com/image/8e34df221017965

I notice it sometimes on my TV so I decided to check the same material with my computer monitor as well, and it was the same thing.

Or is my setup wrongly configured?

[AndreaMG]

@Nev, I have some questions about deinterlacing (I set Lav Video to hardware deinterlace via LAVCUVID):

1) playing dvds wrongly flagged I guess could trigger the deinterlacing even when not needed?

2) Is there a way to know while watching a dvd if Lav is deinterlacing or not?

3) Is there a way (or colud possibly be implemented) to turn deinterlacing on and off of the fly via shortcuts keys like MadVR in the cases I'm 100% sure what I'm playing is progressive without having to go to Lav panel and tick "progressive"?

4) If in a dvd there is mixed material (interlaced and progressive) the interlacer will adapt to the new content or is it dumb?

5) I find Lav deinterlacer less heavy than MadVR one, do you have any idea why? Aren't they both hardware?

Thanks :)

[ryrynz]

Is banding like this normal with 4:2:0 material?
http://www.imagebam.com/image/8e34df221017965


That's normal, to improve that you'll need a deband filter activated via FFDshow/Avisynth etc.

[Xaurus]

That's normal, to improve that you'll need a deband filter activated via FFDshow/Avisynth etc.
What is a good deband filter? I don't really want to use ffdshow though, but maybe Avisynth could be an option.

[ryrynz]

For real time playback on 8-bit banded content via Avisynth I recommend using Flash3kyuu. (http://forum.doom9.org/showthread.php?t=161411)

[DragonQ]

nev (or anyone else),

Is banding like this normal with 4:2:0 material?
http://www.imagebam.com/image/8e34df221017965

I notice it sometimes on my TV so I decided to check the same material with my computer monitor as well, and it was the same thing.

Or is my setup wrongly configured?
Ha, I see banding much worse than that on my TV. No amount of setting changes helps, and it even happens on my PC with MadVR in some videos, so I'd say at least part of the problem is the source material.

[DragonQ]

I've recently started playing with LAV Filters for DVD/BD playback in MediaPortal (EVR + DVD Navigator), now that DVD-Video is supported. I've managed to get it working but it has two flaws:

1) For some discs, the very first thing that appears is a language selection screen. However, in MediaPortal I just get a black screen or the MediaPortal main menu continues to show. If I press "enter" on my remote, the language selection screen briefly flashes, then the pre-menu titles play as normal. If I instead move the mouse, then the language selection screen appears.

2) For at least one disc I've tried, if I start playing the main feature from the DVD menu, the total length is detected as 0:00, meaning I can't seek. If I press "next chapter", then the total length gets updated and appears properly, enabling seeking.

I don't know if these are MediaPortal problems or LAV problems but I thought I'd bring it up anyway. I'm using DXVA2 Native.

[nevcairiel]

Note that DXVA2 Native is a bad choice for DVDs, it doesn't properly work with menus. There is now an option to disable HW accel for DVDs, i would suggest to simply turn DXVA off for DVDs, the SD MPEG-2 material can be decoded by a 10 year old PC without breaking a sweat.
The duration and stuff like that is not handled by the decoders, so it has nothing to do with LAV.

[DragonQ]

Hmm that's strange because menus and subtitles work fine for me, it's just that initial screen on some discs. I've also tried disabling "DVD" under "Hardware Acceleration" but the problem remains.

[nevcairiel]

If you get Menus and Subtitles, then you're either not using LAV at all, or at least not with DXVA2 Native.
My bet is on the first.

[DragonQ]

http://www.aotplaza.com/Files/HTPC/Screengrabs/LAV%20used%20for%20MP%20DVD.png

I also get menus and subtitles using MPC-HC with LAV Filters. Maybe DXVA2 Native isn't being used but it's definitely set to be used.

[JaylumX]

Nev, any chance of an upscaling option for content less that ones horizontal resolution i.e my monitor in 2560x1440 but i think it would be nice if 1920x1080 or less content was upscaled to my native res by the GPU. Possibility or is it not practical

[nevcairiel]

The renderer will upscale content using the GPU, a decoder does not upscale.

[nevcairiel]

BTW, I was wondering how you'd feel about adding post-processing capabilities?

First and foremost I'm developing a decoder, not a post-processor. I only made an exception for YADIF Deinterlacing and certain pixel format conversions because those are essential for working playback.
The same thing essentially applies to the mixer in LAV Audio, in certain configurations its essential to be able to remix your audio, or you may end up hearing only half the channels.

If i ever get really bored (yeah, right), it may be a possibility to either write a separate post-processing filter, and/or maybe add AviSynth or VapourSynth support. At this time, AviSynth is just a mess to work with and the 8-bit limitation is just so outdated, and VapourSynth is still too much in flux to look into it properly.

In any case, don't hold your breath.

PS:
I may be persuaded to expose libavfilter based filters, if someone wants one of those desperately, it could be done without much pain or effort. However, i know that maybe only 2-3 of those are even real-time optimized, many others are just very slow, and i also know that many people prefer different filters then the ones available there.

[nevcairiel]

1) playing dvds wrongly flagged I guess could trigger the deinterlacing even when not needed?

If its wrongly flagged, how would it know any better? It has to trust these flags. So yes.


2) Is there a way to know while watching a dvd if Lav is deinterlacing or not?

No.
Well technically you can see if the renderer gets 25/30 or 50/60 frames, which might be an indication, but nothing obvious.


3) Is there a way (or colud possibly be implemented) to turn deinterlacing on and off of the fly via shortcuts keys like MadVR in the cases I'm 100% sure what I'm playing is progressive without having to go to Lav panel and tick "progressive"?

There is not. Maybe in the future.


4) If in a dvd there is mixed material (interlaced and progressive) the interlacer will adapt to the new content or is it dumb?

If the content is properly flagged, it will adapt.


5) I find Lav deinterlacer less heavy than MadVR one, do you have any idea why? Aren't they both hardware?

Implementation differences, who knows.

[leeperry]

First and foremost I'm developing a decoder, not a post-processor. I only made an exception for YADIF Deinterlacing and certain pixel format conversions because those are essential for working playback.
The same thing essentially applies to the mixer in LAV Audio, in certain configurations its essential to be able to remix your audio, or you may end up hearing only half the channels.

[..]

AviSynth is just a mess to work with and the 8-bit limitation is just so outdated
:thanks: for the detailed reply!

Well, yeah, downmixing 5.1 to stereo does require custom coeffs(I personally love the Logic7 matrix, that I'm currently using in ffdshow with fantastic results) together with VST plugins processing in order to turn surround stereo into binaural audio for headphones use http://files.head-fi.org/images/smilies/happy_face1.gif

The same way anyone caring at least a bit about 5.1 audio will have to process each individual channel for bass management/delay and Digital Room EQ if he doesn't want to listen to his walls and furniture("can you move a bit to the left, I can't hear the cymbal properly kthx" :D).

So I realize that you don't want to get into the post-processing dark side, but it really is mandatory for whatever 5.1 or stereo/binaural audio(God bless ffdshow in 32fp!) and I believe that Avisynth can actually output more than 8bit these days, it's just that the Avisynth filter of ffdshow is unable to take advantage of it and the coder who worked on it has gone AWOL :(

And if there's a place to do levels conversion, flipping/mirroring, frames interpolation, audio DRC/VST processing, it's more in the decoder than in the VR/audio drivers(a RME board costs +$500, uses sloppy hardware and is useless for lossless HDMI audio) IMHO.

I realize that at this point, you're in the same position as madshi providing the best software for the job *FOR FREE*. madshi has made clear that at some point he will release a shareware version that will provide more advanced features for a fee....so how about you do that too? Make a $35 version that supports all kinds of audio goodies in 32fp/64fp and 10/16bit video processing?

Paying whiners won't look like spoiled brats anymore and you won't be wasting your precious time providing technical support for free either....anyway, it's just an idea but the decoding/splitting parts of your package both seem fully matured, all it'd need is additional audio/video processing IMHO and I'm sure that your most demanding end-users are well aware that well written code doesn't grow on trees.

All this said, I can do all my 32fp binaural stuff in ffdshow audio and the BenQ W1070 pj I'll be buying very soon provides 12bit mirroring in its OSD, so all I really need at this point is >8bit levels conversion and frames interpolation in order to magically turn 29.97@59.94fps...so I guess that all I would personally need is an Avisynth wrapper that would support P016, and I'm pretty sure that this would make a lot of other OCD'ed videophiles happy to pay for your software as well :cool:

[madshi]

And if there's a place to do levels conversion, flipping/mirroring, frames interpolation, [...], it's more in the decoder than in the VR [...]
I strongly disagree.

[leeperry]

Well, doing it all in ffdshow(currently being the only place where it's possible to do levels conversion/mirroring/avisynth scripting) allows the use of automatic rules. I believe this kind of automatic profiles and video-processing has more its place in LAV/ffdshow than in mVR. I'm not talking about one-shot processing, more like automatic profiles based on resolution and frame rate. And done in P016, it won't temper w/ the PQ when feeding mVR.

Anyway you two are the ones in charge, at this point I'm mostly begging for practical solutions to real world issues :thanks:

[AndreaMG]

Thanks for your answers Nev.

If i ever get really bored (yeah, right), it may be a possibility to either write a separate post-processing filter (...)

Hope you will get bored really soon:D

[nevcairiel]

Hope you will get bored really soon:D

Doubtful thats going to happen before my retirement in 40 years.

[leeperry]

Well, all roads lead to Rome so all we end-users really need is madshi and nev looking into solutions for P016 video processing. We have a decoder, a renderer and nothing to put in between..not even levels conversion for sloppy encodes :scared:

FFdshow can process audio in 32fp so this is a non-issue IMHO, and there's always AC3Filter's EQ for 5.1 processing...it sounds pretty bad but technically it works.

[Keiyakusha]

Well, all roads lead to Rome so all we end-users really need is madshi and nev looking into solutions for P016 video processing. We have a decoder, a renderer and nothing to put in between..not even levels conversion for sloppy encodes :scared:
Yeah, so lets bring some more folks and try to convince nev that we need vapoursynth wrapper.
Alternatively someone may try to hack ffdshow and remove from it everything that is not related to raw processing filter and avisynth scripts... (but I doubt we'll have P016 with this solution)

[nevcairiel]

Here is another option:

Actually write something yourself. :P

[leeperry]

oh yah sure, you can learn to code like a god overnight, piece of cake! It's not like it would take years/decades before you'd make something that would even work :devil:

which sends us back to my original proposition of making a shareware version that would go deeper into the P016 rabbit hole.....some end-users crave advanced processing, let them pay for it. I'll be most happy to pay for mVR and LAV because w/o mVR I still would be stuck w/ HR/EVR and atm there simply isn't any way whatsoever to do P016 video processing :(

Anyway I've made my point, you've made clear that you didn't care too much, madshi disagrees with what I said so that's me hoping that he will soon allow custom levels conversion and provide N-1 frame access in the forthcoming HLSL PS script support of mVR so third party coders will be able to write frame interpolation scripts :)

Too bad the horsepower of my Q9450 will be rendered useless and all the load will be put on the GPU, forcing everyone interested to buy a GTX660/660Ti(or whatever AMD substitute) basically.

[nevcairiel]

oh yah sure, you can learn to code like a god overnight, piece of cake! It's not like it would take years/decades before you'd make something that would even work :devil:

Still sooner then me being really interested in it. :)


which sends us back to my original proposition of making a shareware version that would go deeper into the P016 rabbit hole.....some end-users crave advanced processing, let them pay for it. I'll be most happy to pay for mVR and LAV because w/o mVR I still would be stuck w/ HR/EVR and atm there simply isn't any way whatsoever to do P016 video processing :(

I have no interest in selling my software. Selling means people have expectations (even more so then the people with entitlement-issues today), and i don't want to deal with that shit. Not to mention that there is the issue of LAV being build based on GPL libraries, which prohibits this anyway.

Anyway, unless someone manages to pay me enough so i can quit my dayjob, no amount of money will give me more time. :p

[ryrynz]

I would also like to see (pay for) Vapoursynth/Avisynth support with perhaps condition based loading. It would be awesome to see MadVR tapping into these settings as well for a full config based on filename/resolution/bitrate etc.

I think many of us are looking to retire FFDshow and Avisynth from our toolkits now. Clsid asked about a DirectShow filter for Vapoursynth and TheFluff said it would be fairly easy to implement for someone competent with DirectShow, surely someone will do it eventually.

[MarkT]

Well, keep it free and simple. Cause I hate to break out IDA to figure out the protection. :D I don't see how such a product could pay reliably as much as a regular day job. GPL hin oder her, the market for directshow decoders is tiny. People buy a computer with a BD drive, fire up PowerDVD, which plays about anything and that is that. LAV-what?

As for advanced post-processing, I am missing audio DRC like in VLC (http://forum.videolan.org/viewtopic.php?f=12&t=46020#p256565). Call me nuts, but sometimes after a long night of work I feel like watching a movie I have seen a 1000 times like Die Hard at 03:00am. But dozing off without DRC is asking for trouble. And then the first guy gets shot. ;) The mpc-hc DRC code works, but why settle for less when you can have a nice RMS envelope follower DRC.

Of course I could use the mpc-hc DRC or just VLC but I like the clean concept of LAV.

:thanks:

[6233638]

All this talk of post-processing filters/scripts aside, I would really appreciate some sort of crossfeed function for headphone use, even if it's not possible to implement something like Dolby Headphone or some other form of HRTF/binaural processing.

As I understand it, crossfeed would mostly be an extension of the mixing and delay functionality that already exists inside LAV Audio. (downmix to stereo, then apply crossfeed)
The only other option is very expensive amplifiers like the SPL Phonitor (http://www.abload.de/img/spl_phonitor_11vsgj.jpg).


I think the crossfeed level is simply the amount of left-channel audio that leaks into the right, and vice-versa.

Speaker angle would be controlled by the amount of delay applied to the crossfeed? (because left-channel audio hits your left ear before your right)

I'm not sure how you would control the "center" mix though. (how this sort of thing works is beyond me really)

[Joniii]

I wonder if it would be possible to do something about the small BD stutter on codec level when screen refresh rate is set at 60? Switching to 24p mode with madVR is good but not an ideal solution for computers.

[nevcairiel]

I wonder if it would be possible to do something about the small BD stutter on codec level when screen refresh rate is set at 60? Switching to 24p mode with madVR is good but not an ideal solution for computers.

Thats nothing the decoder influences.
Use ReClock, it can solve such issues.

[Joniii]

Thats nothing the decoder influences.
Use ReClock, it can solve such issues.

I was thinking if it would be possible to add similar method than what reclock uses in LAV.

[nevcairiel]

I was thinking if it would be possible to add similar method than what reclock uses in LAV.

Thats not possible. Only the audio renderer has access to the audio reference clock to do what needs to be done.

[leeperry]

I have no interest in selling my software. Selling means people have expectations (even more so then the people with entitlement-issues today), and i don't want to deal with that shit. Not to mention that there is the issue of LAV being build based on GPL libraries, which prohibits this anyway.

Anyway, unless someone manages to pay me enough so i can quit my dayjob, no amount of money will give me more time. :p
Dayjobs are overrated anyway :D

Fine, make the post-processing filter standalone and ask a fee for it. I'm sure you'd soon find out that quite a number of ppl would be willing to pay for the ability to process their video in P016 http://forum.slysoft.com/images/smilies/agreed.gif

Thats not possible. Only the audio renderer has access to the audio reference clock to do what needs to be done.
How about a frames interpolating algorithm processed in P016 that would automatically make the movie fps match an exact multiple of the refresh rate? Killer combo with Reclock :devil:

I'm not sure how you would control the "center" mix though. (how this sort of thing works is beyond me really)
That's really reinventing the wheel IMHO when all it needs is a 32fp/64fp VST plugin wrapper(currently only possible in ffdshow audio using a lot of undocumented dirty tricks). There are many excellent sounding xfeed plugins out there, the most transparent one to my ears(that I use on a daily basis) being free.

I am missing audio DRC like in VLC
many VST plugings can do that, and with most likely much better sounding results too.

[Joniii]

Thats not possible. Only the audio renderer has access to the audio reference clock to do what needs to be done.

How about something like this? Judder free 24p->60p would be amazing feature on LAV Filters.

http://en.wikipedia.org/wiki/SmoothVideo_Project_%E2%80%93_SVP

[nevcairiel]

How about something like this? Judder free 24p->60p would be amazing feature on LAV Filters.

http://en.wikipedia.org/wiki/SmoothVideo_Project_%E2%80%93_SVP

Just use SVP, don't need it inside LAV.

[leeperry]

Just use SVP, don't need it inside LAV.
but no P016 possible then http://forum-images.hardware.fr/images/perso/2/fegafobobos.gif

[nevcairiel]

Its an AviSynth plugin, no 16-bit anyway. :P

[leeperry]

Avisynth 2.6 can do >8bit FWIR, it's only a matter of not using an ancient wrapper that's stuck to YV12(such as the one currently embedded in ffdshow).

[nevcairiel]

As far as i know, the MVTools plugins (the essential parts behind SVP) only support YV12 or YUY2, and not the hacked 16-bit formats in AviSynth.

Aynway, this is off-topic, LAV will not get support for any of this in any foreseeable future.

[Joniii]

Just use SVP, don't need it inside LAV.

I just thought if similar method than what SVP uses could be easily added to calculate intermediate frames and get software motion interpolation.

As far as i know, the MVTools plugins (the essential parts behind SVP) only support YV12 or YUY2, and not the hacked 16-bit formats in AviSynth.

Aynway, this is off-topic, LAV will not get support for any of this in any foreseeable future.

Ok, sorry for ot.

[RealSnoopyDog]

Hi nev, do you think it is a problem to use the ffdshow subtitle filter as a video prostprocessor with LAV? I use it with Intel Quick Sync and nVidia with DXVA2 and there seem to be no problems.

[nevcairiel]

Hi nev, do you think it is a problem to use the ffdshow subtitle filter as a video prostprocessor with LAV? I use it with Intel Quick Sync and nVidia with DXVA2 and there seem to be no problems.

Its not a problem, but there are is a much better subtitle filter if thats all you want - look at xy-vsfilter.

[RealSnoopyDog]

Thank you, i'll try that. :) Yes, i only need subtitles. I have a lot of BDs, where it's essential to have subs.

[nevcairiel]

Apparently i wasn't done with DVDs yet, so here is the final piece:

x86: http://files.1f0.de/lavf/LAVFilters-0.53.2-7-g9c504fd.zip
x64: http://files.1f0.de/lavf/LAVFilters-0.53.2-7-g9c504fd-x64.zip

Whats new, you wonder?

This version adds (experimental) support for dvd subtitles and full dvd menu support in DXVA2 Native mode. I managed to draw the subtitles onto the D3D surface, as well as copy the surface to be able to redraw images for menu interactivity.

In my limited tests, it seemed to work just fine, but as always, please give it a whirl and see if you can break it. ;)
On my NVIDIA it worked just perfectly, however on my Intel it did seem to occasionally cause a small playback glitch, so lets hear your experiences.

[6233638]

The active decoder is being reported as "avcodec" here when trying to play back anything using DXVA (native) whether it's DVDs, Blu-ray or ripped files.

I think this is software decoding? I don't normally use DXVA though, so it's entirely possible that DXVA is broken in Windows 8. (it does say it's "available" in the configuration)

[sneaker_ger]

Are you using a DXVA2 compatible renderer? (madVR is not DXVA2 native compatible)
Do you have any filter between LAV Video and the renderer? (vsfilter does not work with DXVA2 native)

[Superb]

I thought that drawing directly to the surfaces in DXVA native suppose to cause image corruption...?

[nevcairiel]

It does, but its the most obvious with H.264 because of the extremely high number of reference frames, i figured maybe i get lucky with MPEG2 for DVDs. :P

Anyhow, there are still other things that can be done, like simply copying the frame when it needs to be written to, and modifying that copy. A GPU should be plenty fast copying a SD frame in pure GPU memory.

[6233638]

Are you using a DXVA2 compatible renderer? (madVR is not DXVA2 native compatible)
Do you have any filter between LAV Video and the renderer? (vsfilter does not work with DXVA2 native)Sorry, it had been so long that I forgot DXVA does not work with madVR and that DXVA copy-back or CUVID are required for it.

It seems to be working correctly in EVR.