LAV Filters - DirectShow Media Splitter and Decoders

[nevcairiel]

AYUV output uses a "slow" algorithm, because its not a format natively supported by swscale.

Its a two step process, due to the requirement for dithering:
- Convert the 4:4:4 10bit to 4:4:4 8bit using swscale dithering
- Manually reorder the bytes to fit the AYUV layout, in pure C code, no optimizations yet

I haven't actually compared performance yet, but its certainly possible that this two step operation is much slower then using a one-step operation like YV12 or RGB32, which are natively supported by swscale.

I actually benchmarked this now:
Using a H264 4:4:4 10bit 1920x1080 movie, on a Core i7 2600K - in a debug build, release builds will be faster.

These times are *per frame*


Y410 (native) 7.5ms
AYUV 9.8ms
RGB32 20.8ms
YUY2 19.8ms
YV12 20.5ms
NV12 30.4ms


Granted, i just spent some time optimizing the AYUV/Y410 path before doing the measurements.
However, i do not see the higher processing requirements for AYUV.

These numbers show some interesting things, though. For example, it would be impossible to play this movie smoothly if it was 60fps, unless you renderer supports native 4:4:4 input. I am using a bicubic scaler for down-sampling the chroma, i'm sure the numbers for YUY2/YV12/NV12 would be alot lower if i switched to bilinear or even point scaling.

Also, swscale is not multi-threaded right now, but it could be in the future, so we'll see about that.
PS:
Why is there no planar 4:4:4 pixel format? That would be far more efficient then AYUV...

Edit:
Updated Y410 numbers after optimizing the path as well

[clsid]

Performance is better now with AYUV, but not yet good enough for smooth playback with EVR CP. Judging from your benchmark results, it is probably the mixer in MPC that is not liking AYUV very much.

[clsid]

Does it still stutter for you guys when you play AVI Files?
I does for me when i cut an MKV file with Direct Stream into an AVI file, and then play it, it stutter about all the time.
Maybe it´s a bug or unstable with that scenario, so thought it would be nice to tell about it:)
Keep up the Work Near Carrier, and i tried the Nightly Build by Underground aswell;)H.264 inside AVI stutters here as well. Seems to be a frame order problem.

[nevcairiel]

Performance is better now with AYUV, but not yet good enough for smooth playback with EVR CP. Judging from your benchmark results, it is probably the mixer in MPC that is not liking AYUV very much.

EVR-CP looks completely broken here with AYUV input. I don't think EVR-CP works much with AYUV at all.


H.264 inside AVI stutters here as well. Seems to be a frame order problem.
The H264 in AVI i tried works just fine. But in general, its not a recommended way to store H264.

[clsid]

You are right. Tested with a b0rked sample. Deleted now.

What exactly is broken for you with AYUV and EVR? Wrong colors? Here it is just the performance that sucks.

[nevcairiel]

I figured out why it was broken. Using "Half Floating Point Processing" causes AYUV to break, if i disable it it works. I also don't get any dropped frames on my system with the 24fps clip, but i have a high-end gamer GPU and a very fast CPU in this system. :p

[clsid]

I have a i7 2600K and a GTX570 here.
Just a few dropped frames on the BBB444p10.mkv sample. But the stats graph isn't smooth. Green line skyrockets, then drops back to normal, skyrockets again, etc.

[nevcairiel]

AYUV with "Force 10-bit Input" looks extra good.
http://images.gammatester.com/pics/db6c226b80fb614a2fe01fbf11bd1866.png

[clsid]

For those who appreciate modern art ... :P

The 10-bit and float stuff in current MPC is broken. Jan Willem is trying to fix that.

[Mercury_22]

I can't get thumbnail in WMP / WMC or Explorer using LAVSplitter & LAVVideo for OGV, FLV(H.263), WEBM (LAVSplitter with other decoder it's working for this one so it's a VP8 LAVVideo problem) files while using MPC-HC standalone or "official" (http://xiph.org/dshow/downloads/) splitters and decoders I can get thumbnail in WMP / WMC or Explorer

Can you please fix this?

Samples and reg files (http://www.multiupload.com/ATSL6EH0HN)

P.S. For this formats there's no need for any "media foundation hack" to be able to see thumbnails just the necessary registry

[clsid]

Here is just the reg tweak:
Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\.flv\ShellEx\{BB2E617C-0920-11D1-9A0B-00C04FC2D6C1}]
@="{c5a40261-cd64-4ccf-84cb-c394da41d590}"

Many people like to have thumbnails, so it would be awesome if your filters would be compatible.

ffdshow's code might give some hints for the necessary workarounds. I don't think Explorer supports stuff like dynamic format change.

[nevcairiel]

I have no idea how thumbnails are generated in explorer or in WMP, and i really have no motivation to find out.
You cannot really debug explorer, its not like it behaves like any normal player.

If someone else wants to figure out why it isnt working - be my guest.

[clsid]

That sucks :(

Maybe if you get bored sometimes, you could have a look at TffdshowDecVideo.cpp. I suspect the allowOutChange stuff is relevant. There is also a workaround for powerpoint there that might be helpful.

[mastrboy]

If you get bored i rather you consider putting some work into mkv ordered chapters support ;)

[Mercury_22]

I have no idea how thumbnails are generated in explorer or in WMP, and i really have no motivation to find out.
You cannot really debug explorer, its not like it behaves like any normal player.

If someone else wants to figure out why it isnt working - be my guest.

Too bad ! :mad:
But I don't think MPC's or xiph's splitters and decoders are "doing" something special for thumbnails generation
Anyway if you'll find some free time at some point here is the source code for opencodecs_0.85.17777 (http://downloads.xiph.org/releases/oggdsf/opencodecs_0.85.17777_src.7z) too

[nevcairiel]

I did debug some, and it turns out the interface used to query for video frames does in fact not support the VIDEOINFOHEADER2 format - and thats the only one exposed by LAV Video at this time.
I may add support for VIDEOINFOHEADER (without the 2) in the future, maybe that allows grabbing frames already.

Reference: http://msdn.microsoft.com/en-us/library/dd390180(v=vs.85).aspx

[Mercury_22]

I did debug some, and it turns out the interface used to query for video frames does in fact not support the VIDEOINFOHEADER2 format - and thats the only one exposed by LAV Video at this time.
I may add support for VIDEOINFOHEADER (without the 2) in the future, maybe that allows grabbing frames already.

Reference: http://msdn.microsoft.com/en-us/library/dd390180(v=vs.85).aspx

I guess this apply to the VP8 / WEBM case (since it's working with other video decoder) but for OGV and FLV(H.263) it's something wrong with the splitter too cause even with other decoder (MPC's or xiph's ) it's not working :confused:

[nevcairiel]

The FLV works for me with LAV Splitter and ffdshow.
Note that on x64 windows you need to install the x64 filters for this to work in Explorer - but in WMP its still using the 32-bit, afaik.

[Mercury_22]

The FLV works for me with LAV Splitter and ffdshow.
Note that on x64 windows you need to install the x64 filters for this to work in Explorer - but in WMP its still using the 32-bit, afaik.

LAVSplitter with ffdshow works for me too for FLV(H.263)
So mpc's decoder it's working only with mpc's splitter :confused:

Well then only ogv has a problem in the splitter too

EDIT wrong again OGV it's working too with ffdshow

So you're right again :) it's just LAVVideo

[CruNcher]

LAVFilters-x86-git-fc7d418c13c0.7z (http://www.mediafire.com/?nv51696jut4o06v)

Your build fails with a lot of format playback resulting in a black screen @ for the Video playback (Lav Video) (tested VP6,Bink,Divx 3.1).
roytam1 his build is working

[roytam1]

my newer build: http://www.mediafire.com/file/5o0cg5hqy50n5d9/lavfilters-314a389af4a366a4cb00d0bc13695ae02b2130ee-win32.7z
China mirror http://u.115.com/file/dnhkfbi5

[nevcairiel]

LAVSplitter with ffdshow works for me too for FLV(H.263)
So mpc's decoder it's working only with mpc's splitter :confused:

Well then only ogv has a problem in the splitter too

EDIT wrong again OGV it's working too with ffdshow

So you're right again :) it's just LAVVideo

LAV Video now works for me as well, except on some H264 files, it appears to somehow fail on these, not sure why.

[Mercury_22]

LAV Video now works for me as well, except on some H264 files, it appears to somehow fail on these, not sure why.

AFAIK WMP / WMC & Explorer are using Media Foundation for H264 so unless you're disabling that (using CLSID's tool (http://forum.doom9.org/showthread.php?t=146910)) which I don't recommend due to some side effects
it's intriguing that it's working for you for some files :)
Anyway I can't test right now but I'll test this later although I'm sure it's working so once again :thanks: :thanks:

P.S. I guess this will make Lav Filters very popular in the codec packs now :p

[clsid]

Thanks for the VIH1 support!

The preferred filter settings of Win7 apply to all basic DirectShow apps, including GraphStudio, Explorer, and WMP/MC. So by default LAV Video won't get used for H.264. Perhaps you could add an option to replace fourcc AVC1 with CCV1 like haali does. That option was added exactly for the purpose of bypassing the MS decoder without the need of special registry tweaks.

There are only two downsides I can think of (and that is why it should be optional):
- It only works with a few decoders: LAV, ffdshow, MPC, and CoreAVC. But that could be considered an upside as well ;)
- It might interfere with streaming video to XBOX360 (and trigger trancoding instead). Maybe someone can confirm this?

[nevcairiel]

I don't like that custom media type approach, and i won't be doing it.

[Superb]

AFAIK WMP / WMC & Explorer are using Media Foundation for H264 so unless you're disabling that (using CLSID's tool (http://forum.doom9.org/showthread.php?t=146910)) which I don't recommend due to some side effects
it's intriguing that it's working for you for some files :)
Anyway I can't test right now but I'll test this later although I'm sure it's working so once again :thanks: :thanks:

P.S. I guess this will make Lav Filters very popular in the codec packs now :p
WMP uses DirectShow. One can see it when using DirectShowSpy and connecting to the filter graph.

[Mercury_22]

WMP uses DirectShow. One can see it when using DirectShowSpy and connecting to the filter graph.

WMP12 (windows 7) for H.264 (for thumbnail ) ?

Also I've tested and I can confirm it's working, but not for any H264 in mkv, or flv or... (no tweaks to disable Media Foundation here)

[clsid]

Media Foundation and the preferred DirectShow filter list are two separate things.

[clsid]

This sample crashes swscale with RGB24 output:
http://x264.nl/developers/Dark_Shikari/BBB444p10.mkv

[nevcairiel]

The only crash i get there is in the ffdshow raw video filter when trying to feed RGB24 through it, but swscale processes it fine.

[clsid]

Happens here with this graph:
file > LAV video > Color Space Converter > Video renderer
(with only RGB24 enabled in LAV settings)

[nevcairiel]

No problem here with that setup.

[roytam1]

I can reproduce it here in XP SP3.
- Opening a file in GraphStudio
- remove the "Video Renderer" filter
- readd Video Renderer from [Graph]->[Insert Video Renderer]->[Video Renderer]
- reconnect Video Renderer with LAV Video (Its output format are all checked by default and I did't change it)
* Color Space Converter is magically inserted in the middle, In pin is MEDIASUBTYPE_RGB32, XForm Out pin is MEDIASUBTYPE_ARGB32.
- Click Play button, access violation occurs in quartz.dll (i.e. the Color Space Converter)

[roytam1]

And for RV20+SVT sample, I found a serious situation. If an overlay mixer(system's Overlay Mixer, Overlay Mixer2, etc.) is involved in the graph, such sample will fail to display correctly, regardless the aspect ratio.
http://i.imgur.com/gp8nw.png
and 1337player subtitle mixer is the overlay mixer here.
You can download 1337player here: http://www.daansystems.com/1337player/

[nevcairiel]

Its the overlay mixers fault, its incapable of properly changing the input format. The main problem is that the splitter claims the movie is 640x352, and the decoder corrects that to 320x352 - however the overlay mixer does not manage to deal with that, and still wants a 640 pixel wide image.
Overlay mixer is outdated, and i really have no plans to add alot of magics just for it. Use a modern player if you want to use my modern filters.

[roytam1]

Its the overlay mixers fault, its incapable of properly changing the input format. The main problem is that the splitter claims the movie is 640x352, and the decoder corrects that to 320x352 - however the overlay mixer does not manage to deal with that, and still wants a 640 pixel wide image.
Overlay mixer is outdated, and i really have no plans to add alot of magics just for it. Use a modern player if you want to use my modern filters.

THe problem is that DirectShow framework is a system-wide thing. Not only media player will use it, but also games, transcoders, screen recorders, etc.
Unlike ffdshow, which has white/blacklist to control who can use the decoder, and LAV Video's "weird" behavior that it tries to reconnect to the video renderer when the output size is changed, which leads to a stop of graph and returning a DirectShow error code 0x8004022a.
Such behavior can make some games get stuck in the opening movie.
I don't think correcting the output resolution by LAVVideo is wrong, but many DirectShow applications are unexpected in such behavior, simple DirectShow applications such as mplayer2 and amovie.ocx can demonstrate and not to mention the games.

[nevcairiel]

Its not weird behaviour to reconnect when output size changes, every decoder does that. How would it otherwise work? I have to let the renderer know that the image now has a different size, or very weird stuff will happen.

The only difference is that i will not start doing ugly hacks to support applications that should've been gone out of service 10 years ago. You want to use my filters, you better use a modern system to go with it.
I did however manage to fix the silly overlay mixer right now.

PS:
I cannot test "mplayer2" because its so old that its no longer present on my Windows. :p

PPS:
"System-wide" - many players allow using filters individually, and other tools are present to manage which filter is the preferred one on your system. If you absolutely have to use some broken software, set stuff up with that.

[CruNcher]

Nev no chance to get Lav Splitter working with Arcsofts Decoder for .ASF (sync), it's always funny how Cyberlink and Arcsoft stuff are interoperable with each other (Cyberlinks WMV Splitter (ff broken) + Arcsofts Decoder stays sync) (except they blocking each other for reason) but lav splitter sometimes doesn't fit in their dshow frameworks like with *.WMV . Though Cyberlink + Lav Splitter is currently very well working for .ts and H.264 at least, though their are scenarios especially when DXVA comes into play where Lav Cuvid is overall much better (non DXVA compliant streams) and others where it fails again (Deinterlacing).

[nevcairiel]

Wmv is way to unimportant to warrant much work. Stop watching cheap porn. :)

Use the WM ASF Reader.

[CruNcher]

I prefer Arcsofts ASF Reader instead though (but only because it's sync compared to lav Splitter, found no other issues so far) i dunno how the one in Vista/Win7 does, and only because 80% of all WMV content is Pron their is still stuff that isn't and devices that still use ASF and i don't use Pron for testing but WMV-HD Discs :D and i still prefer to directly play files for several reasons then to remux them all to .mkv which though slowly seems to be the only way to keep those for the future and editing in Open Source software.
It would be nice though to keep the dshow framework as sleek as possible and only use 3rd party stuff where it's absolutely (still) needed bugs aren't yet fixed (or other features missing) :)

[nevcairiel]

WM ASF Reader + LAV Video works fine for basically all WMV content.
So does WM ASF Reader + WMVideo Decoder DMO - not one 3rd party component. :) You have to remember, the Microsoft filters are the native decoders, everything else is third party.

[CruNcher]

The Problem in the combinations you mention is that WMV ASF Reader + WMVideo Decoder DMO doesn't support WMVA with DXVA but playbacks it only in Software @ least on XP (Microsoft seems to forgot it).
and LAV Video is Software only and Lav Cuvid doesn't work with WMV3 acceleration (only WVC1 and WMVA) and another Problem with Lav Video in those regards is that it doesn't separate any of the WMV parts but puts them into 1 package and so it takes the major state and flexible combination is almost impossible with MPC-HC.

[nevcairiel]

Hardware decoding for WMV3 is overrated - its such a simple format, any modern CPU can decode it without breaking a sweat. All the advanced features require advanced profile, which is only in VC-1

[CruNcher]

still there are less cycles used then using libav ;) and nope WMVA also uses them though Microsoft wish is they never released that intermediate to VC-1 AP ;) http://forum.doom9.org/showthread.php?t=112634


A: Indeed it did. Microsoft began working on its implementation of WMV AP before VC-1 spec was finalized. That codec (FourCC: WMVA) shipped with WMP10 (Format SDK 9.5), but as final changes to the VC-1 spec were made, WMVA got out of sync with the official spec. Today WMVA is no longer VC-1 compliant and is generally considered deprecated.

And this is the reason why in no decoder it should be put into a package but seperated if you want to have full flexibility of choice


A: Yes, it does. In order to differentiate it from SP/MP codec (FourCC: WMV3) and the non-VC1-compliant AP codec (FourCC: WMVA) and avoid issues with legacy decoders, the VC-1 compliant WMV9 Advanced Profile codec uses a new FourCC code: WVC1.

1 Scenario would be you want WMV3, WMVA and WVC1 Hardware Decoded and WMV1/2 Software via Lav Video that wouldn't be possible as marking the WMV checkbox in Lav Video would take over everything in certain cases if you have the other Decoder over it and prefered it's still fine but if you for a certain reason need a third one it gets problematic.
In this case ffdshow is the rescue as they have separated them accordingly :) WMV1/7, WMV2/8, WMV3/9, VC-1 (WVC1,WMVA), WMVP so here you just disable what you need don't need and want another decoder to be decoded and in the chain search it will easily fallback as it gets no connection and use the next decoder you have set (most importantly this also would work from a Generic dshow application like mplayer2.exe without any merrit prefer system so for example if you want to have a special transcoding chain not forced inside the application and a playback framework forced inside a application for example MPC-HC).

For example i want ffdshow to force WMV1/2 and everything else i want to force Arcsoft and for .ts i want to force Lav Cuvid (as it doesn't support WMV3) ;) i would put Lav Cuvid on the top enable VC-1 and down bellow ffdshow with only WMV1/2 enabled bellow (Software Fallback) than i put Arcsoft (it should switch for WMV3 inside ASF with DXVA, normaly it would force everything but as ffdshow is above it the chain will never reach it for WMV1/2) :)

You can see this example in action here http://forum.doom9.org/showpost.php?p=1518423&postcount=4516 <- it shows the great flexibility of dshow (in a managed enviroment).

So now it works by logic (you also have to put the Splitter in the correct order) and is switching depending on the connections it can do or cant with Lav Video that isn't possible yet as it would force everything to itself bellow Lav Cuvid.

[roytam1]

My nightly builds are now available here: http://roy.orz.hm/lavf-w32-nightlies/

[nevcairiel]

1 Scenario would be you want WMV3, WMVA and WVC1 Hardware Decoded and WMV1/2 Software via Lav Video that wouldn't be possible as marking the WMV checkbox in Lav Video would take over everything in certain cases even if you have the other Decoder over it and prefered. In this case ffdshow is the rescue as they have seperated them :) WMV1/7, WMV2/8, WMV3/9, VC-1 (WVC1,WMVA), WMVP

Just put the DXVA codec on a higher priority, then it'll get used, and if it refuses, fallback to the software decoder.

As a general rule, go from specific to general. That works in so many cases in life. :p

[roytam1]

Just put the DXVA codec on a higher priority, then it'll get used, and if it refuses, fallback to the software decoder.

As a general rule, go from specific to general. That works in so many cases in life. :p

For here, I'd prefer grouping RV30 and RV40 together, but not grouping RV1-3 together.
For RV1-2, binary decoder is still better than libavcodec at the moment.

[nevcairiel]

I changed the grouping to group WMV1/2 together and have a separate option for WMV3.
Also did RV1/2 and RV3/4. RV1/2 is disabled by default, while RV3/4 is active.

[CruNcher]

For here, I'd prefer grouping RV30 and RV40 together, but not grouping RV1-3 together.
For RV1-2, binary decoder is still better than libavcodec at the moment.

Yep exactly such scenarios VC-1 Interlaced would be another one or H.264 4:2:2 ;) not 4:4:4

[nevcairiel]

H264 4:2:2 is virtually non-existant, and ffmpeg is otherwise truely one of the best H264 decoders out there. I've been told ffmpeg will only get support for 4:2:2 if x264 supports encoding it one day.

LAV Video will now try to detect the interlace flag in the VC-1 sequence header and refuse connection if its found. Works fine for the samples i tried it on.