LAV Filters - DirectShow Media Splitter and Decoders

[nevcairiel]

How much are hardware deinterlacers better? (and why?) By the way, if ffdshow has yadif couldn't LAV too?

The GPU is highly optimized for these kind of tasks, and therefor can run a much more expensive algorithm while still keeping up the frame rate.
NVIDIA uses a temporal-spatial adaptive interpolation, which IMHO beats YADIF in quality flat out, no question.

I don't have a set of comparison images at hand, but just visually comparing the two results yourself on an actual movie will most likely convince you. ;)

And yes, LAV could have YADIF, and i do have plans to offer it eventually, but it'll be some time.

[Midzuki]

By the way, if ffdshow has yadif couldn't LAV too?

That would be a job for LAV Video Processor :D But as we all know very well, wishing is easier than doing :p Anyway, I woud not like to meet another filter that sees no difference between uncompressed video and RLE-compressed bitmaps ;)

[CruNcher]

The GPU is highly optimized for these kind of tasks, and therefor can run a much more expensive algorithm while still keeping up the frame rate.
NVIDIA uses a temporal-spatial adaptive interpolation, which IMHO beats YADIF in quality flat out, no question.

I don't have a set of comparison images at hand, but just visually comparing the two results yourself on an actual movie will most likely convince you. ;)

And yes, LAV could have YADIF, and i do have plans to offer it eventually, but it'll be some time.

Pssst don't let Didée hear that :D

[Thunderbolt8]

is there a speed difference when watching videos just based on the cuda version a GPU uses?

[BloodySword]

Bug in Splitter: FFDShow does not decode AAC splitted by LAV Splitter out of a raw .aac file. Any other decoder does not work. Only LAV Audio decoder accepts this stream and decodes it.

[clsid]

A small update on the progress for the improved pixel format conversions:

All "unscaled" YUV conversions are done and optimized.
Unscaled means all formats that don't require changing of the chroma resolution.

Following are implemented using SSE2 enhancements (especially dithering is *alot* faster then before)

YUV 4:2:0 -> YV12/NV12
YUV 4:2:0 10bit -> P010
YUV 4:2:0 10bit -> YV12/NV12 (dithered)

YUV 4:2:2 -> YUY2/UYVY
YUV 4:2:2 10bit -> P210

YUV 4:4:4 -> AYUV
YUV 4:4:4 10bit -> Y410
YUV 4:4:4 10bit -> AYUV (dithered)

Whats missing is YUV 4:2:2 10bit -> YUY2, but getting a sample for that is kinda hard. Once a file surfaces for this, it'll be easy to add.
I'm still a bit stuck on the chroma upsampling for the RGB conversions. I'll hopefully get to it soon. Paid work has been quite taxing the last week.

In theory i would also need chroma subsampling, for 4:4:4/4:2:2 content and your renderer/processor only accepts 4:2:0, but thats a rather rare case and i'll let swscale keep doing that.Is the fallback to swscale also used when a user has an old CPU (without SSE2)?
FYI, there is a patch on the libav mailinglist with optimizations for the current (slow) conversions. It is supposed to give a ~6x speedup.

Can you add support for PNG in MOV? That is one of the last things missing with regard to mov playback. I can provide a sample in case you need one.

[nevcairiel]

Bug in Splitter: FFDShow does not decode AAC splitted by LAV Splitter out of a raw .aac file. Any other decoder does not work. Only LAV Audio decoder accepts this stream and decodes it.

ffdshow also works if you set it to libavcodec for decoding, its all the libfaad2 decoders that fail without the init data.
The more important question, which other splitter makes it work?

Is the fallback to swscale also used when a user has an old CPU (without SSE2)?

Not right now, but its easy enough to do.
However, thats really too old for me to worry about much. :)
I can add the check, but i guarantee nothing for pre-SSE2 CPUs.


FYI, there is a patch on the libav mailinglist with optimizations for the current (slow) conversions. It is supposed to give a ~6x speedup.

That'll still be slower then a very specialized algorithm like i implemented them, however good to know someone is improving things.



Can you add support for PNG in MOV? That is one of the last things missing with regard to mov playback. I can provide a sample in case you need one.

A sample would be good.

[Cudo]

If all you want is a HTPC, i would even go down to a 550, its still way too fast for a HTPC, but will save you money and possibly some energy.
The 560 would be better if you want some gaming performance as well.

Sadly the 500 series doesn't have a real HTPC GPU.I believe Asus is making excellent HTPC ones: ENGT520 SILENT/DI/1GD3(LP) and ENGT520 SL/DI/512MD3(LP). They also support HD bitstreaming through HDMI.

[nevcairiel]

I believe Asus is making excellent HTPC ones: ENGT520 SILENT/DI/1GD3(LP) and ENGT520 SL/DI/512MD3(LP). They also support HD bitstreaming through HDMI.

The 520 is too slow.
Even NVIDIA admitted that it isn't fast enough for full video playback of interlaced material.

[Cudo]

^ +1, I didn't know that.

[Xaurus]

How much are hardware deinterlacers better? (and why?) By the way, if ffdshow has yadif couldn't LAV too?
Just my 2 cents:

There is a world of difference between hardware deinterlacing and YADIF. For example, with LAV CUVID I get no combing or artefacts what so ever and it's smooth as silk except for a few panning issues (which applies to both methods). I would never go back to software deinterlacing.

[Stephen R. Savage]

Just my 2 cents:

There is a world of difference between hardware deinterlacing and YADIF. For example, with LAV CUVID I get no combing or artefacts what so ever and it's smooth as silk except for a few panning issues (which applies to both methods). I would never go back to software deinterlacing.

Imo, there's still nothing that beats (Q)TGMC, which can even be run in realtime at up to 720p with marginally reduced quality. Unlike other deinterlacing methods, motion-compensation processes can actually recover "lost" information from adjacent frames.

@nevcairel: I want to ask r.e. the new colorspace conversions, are they free from issues with hue shift like the ones in ffdshow for 10-bit --> 8-bit conversion? I have some 10-bit H.264 encodes that were created with dithertools, and presumably free from 8->10 errors.

Edit: Never mind, downconversion is bit-exact between ffdshow and LAV, leaving the source of the ever-present hue shift still unknown.

[nevcairiel]

Imo, there's still nothing that beats (Q)TGMC, which can even be run in realtime at up to 720p with marginally reduced quality. Unlike other deinterlacing methods, motion-compensation processes can actually recover "lost" information from adjacent frames

The only interlaced content i watch are Blu-rays, which are 1080i, and the hardware deinterlacer is the only high-quality adaptive algorithm that works in realtime on those.
It does actually do some sort of motion compensation, but the vendors are not too keen on disclosing implementation details.

Edit: Never mind, downconversion is bit-exact between ffdshow and LAV, leaving the source of the ever-present hue shift still unknown.

Right now it is the same as ffdshows, the next version (thats when those new conversions actually make it in) will have mathematically perfect conversion from 10 to 8 (plain down-shift with dithering)

[Thunderbolt8]

can you get this one always to play fine? http://www.mediafire.com/?8y4afgs2yf20td6 (sample of South Pacific BBC 1080i25 VC-1)

seems to be acting up sometimes, not sure whether due to splitter, WMV DMOdecoder or maybe LAV audio

[hoborg]

Offtopic question:
I got new TV with HDMI input, so i connect it with my HTPC (Radeon 6450) by HDMI and set sound over HDMI.
But now i have some problems with multichanels audio - some chanels are nearly mutted. Before i used analog stereo speakers without any problem.
I am using LAV audio decoder -> ReClock on WinXP.
Is it possible somehow downmix mutlichannels to stereo? I didnt found such option anywhere in WinXP/ReClock/LAV audio decoder setting.
I think this should fix the problem.

[CruNcher]

can you get this one always to play fine? http://www.mediafire.com/?8y4afgs2yf20td6

seems to be acting up sometimes, not sure whether due to splitter, WMV DMOdecoder or maybe LAV audio

The issue looks familiar same problem as with the Beyonce sample back then it stops @ the beginning only seeking is making it to play (most probably damaged @ the beginning) :)
Though you should see that VC-1 Timestamp correction is on Auto else you would get a Black screen even after seeking.
But this doesn't seem like a Big problem not as heavy at least as my demonstrated MPC-HC EVR Custom & Sync problem here http://mirror05.x264.nl/CruNcher/mpc-hc/ (please wait for it to be fully loaded i still have to improve the progressive streaming @ the 1:35 mark those aren't performance issues @ recording but VBV problems with Quicksync :( )

[Kotik]

Offtopic question:
I got new TV with HDMI input, so i connect it with my HTPC (Radeon 6450) by HDMI and set sound over HDMI.
But now i have some problems with multichanels audio - some chanels are nearly mutted. Before i used analog stereo speakers without any problem.
I am using LAV audio decoder -> ReClock on WinXP.
Is it possible somehow downmix mutlichannels to stereo? I didnt found such option anywhere in WinXP/ReClock/LAV audio decoder setting.
I think this should fix the problem.

Yes it is possible by adding ffdshow as postprocessing and configuring it to do the downmixing to stereo.

All this till Nev adds downmixing in LavAudio.

[hoborg]

All this till Nev adds downmixing in LavAudio.
Donwmix is planned in LAV audio decoder? Good :)

[nevcairiel]

You can also just not use WASAPI, and let the Windows mixer do downmixing - at least on Win7 that will work pretty good.

[hoborg]

You can also just not use WASAPI, and let the Windows mixer do downmixing - at least on Win7 that will work pretty good.

My HTPC is based on WinXP. If i am not wrong, WASAPI is not present on WinXP.

[kalston]

My HTPC is based on WinXP. If i am not wrong, WASAPI is not present on WinXP.

Correct. For now I use ffdshow to do it.

[CruNcher]

depending on the hardware your htpc consists off you should really take a move to NT 6 Vista/Win7 into consideration it solves a lot off multimedia issues and doesn't make third party stuff needed for everything (also the possibility of less interoperability problems with different Hardware when using acceleration) :)
The Power saving also works much better it doesn't cause extreme spikes as on XP, that lowers the risk of unexpected framedrops also High Resolution timers are a improvement here (Hardware dependent).

[hoborg]

@CruNcher:
Yes, i known.But...
I spend a lot of time to configure my HTPC to work as i would like to be working, identify freezing durning wake up (thanks to TV card) + found a solution - for this i created "plugin" for MCE standby tools which disable/enable TV card durning standby/resume + "fullresume" while HTPC resume for recording. I even created my own "codec pack" - SAF, just to fit my HTPC needs :). Setup remote contorler to work with my HTPC so even my wife can easilly control it (i dont have mouse/keybard connected), silent DVD drive even after wakeup, modify WinXP to not look like windows, media sharing ower network, be able to start/exit games by remote and control them by gamepad (like game console),etc, etc,... Right now i am very happy how it is working, moving to Win7 x86 is planned, but i really don't want to start from begin :)

BTW, i have two small kids home, so i really dont have much time to do such a big task :D

[betaking]

found a bug lavfilters can not use lavvideo to decoder this mpeg file!
Format : MPEG-PS
File size : 63.1 MiB
Duration : 4mn 13s
Overall bit rate : 2 086 Kbps

Video
ID : 224 (0xE0)
Format : MPEG Video
Format version : Version 1
Format settings, BVOP : Yes
Format settings, Matrix : Default
Duration : 4mn 13s
Bit rate : 1 789 Kbps
Maximum bit rate : 1 800 Kbps
Width : 352 pixels
Height : 240 pixels
Display aspect ratio : 4:3
Frame rate : 29.970 fps
Standard : NTSC
Color space : YUV
Bit depth : 8 bits
Scan type : Progressive
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.706
Stream size : 54.1 MiB (86%)

Audio
ID : 192 (0xC0)
Format : MPEG Audio
Format version : Version 1
Format profile : Layer 2
Duration : 4mn 13s
Bit rate mode : Constant
Bit rate : 256 Kbps
Channel(s) : 2 channels
Sampling rate : 48.0 KHz
Compression mode : Lossy
Delay relative to video : -67ms
Stream size : 7.74 MiB (12%)
MPC-HC only use lav splitter and lav audio!

[pirlouy]

Yes it is possible by adding ffdshow as postprocessing and configuring it to do the downmixing to stereo.

All this till Nev adds downmixing in LavAudio.
Except that it does not work in my case. Even if I add ffdshow audio processor (in MPC-HC "prefer" external filter), it never
shows after LAV audio. I don't understand. :/

You can also just not use WASAPI, and let the Windows mixer do downmixing - at least on Win7 that will work pretty good.
Windows does that automatically or there's an option ?!!

[fastplayer]

Windows does that automatically or there's an option ?!!
Just select the correct speaker-setup in the Control Panel settings of your sound-card and let the mixer do the rest.

[nevcairiel]

I mostly finished the implementation of the new RGB converter, roughly modelled after ffdshows HQ implementation, but with two key enhancements:

- Faster and more accurate chroma upscaling (ffdshow assumes MPEG-1 Chroma siting, while most content is using the MPEG2/H264 siting)
--- I do plan to support the MPEG1 version as well at some point in time, but its no pressing issue
- Increased processing bitdepth to natively support 9/10bit content

The code will always use dithering, because i've found that disabling dithering doesn't yield a worthy performance increase.
My implementation is not yet multi-threaded, but i'm not sure if thats actually needed. I'll see about that.

Below some performance numbers from my i7 2600k, decoding and processing a 1080p 4:2:0 H264 movie

- Decoding w/ YV12 output: 253 fps
- Old low-quality RGB conversion: 235 fps
- Old high-quality RGB conversion: 40 fps
- New RGB conversion: 180 fps
- New RGB conversion without dithering: 183 fps

And for fun:
- New RGB conversion in Debug mode: 20 fps.
Don't watch stuff with a debug version, people! :)

Speaking of multi-threading, anyone know a simple threadpool implementation that does not require me to use boost? :p

[Xaurus]

nevcairiel,

Is this anything that is beneficial for us LAV CUVID users? :)

[Thunderbolt8]

please add multitheading if possible, there are other tweaks like sharpening or deinterlacing (ffdshow) or VC-1 decoding which are all only single threaded and already take a good amount of power from 1 core. more of such filters & co. which only work with 1 core will eventually lead to speed problems which could be avoided with multithreading support

[nevcairiel]

I will eventually offer some post-processing, at first only the filters implemented in libavfilter (mostly for yadif), anything else .. we'll see.
I would like to do some plugin type thing, maybe it'll happen. Not soon, though.

Personally, i just don't use post-processing much.

[roytam1]

I mostly finished the implementation of the new RGB converter, roughly modelled after ffdshows HQ implementation, but with two key enhancements:

- Faster and more accurate chroma upscaling (ffdshow assumes MPEG-1 Chroma siting, while most content is using the MPEG2/H264 siting)
--- I do plan to support the MPEG1 version as well at some point in time, but its no pressing issue
- Increased processing bitdepth to natively support 9/10bit content

The code will always use dithering, because i've found that disabling dithering doesn't yield a worthy performance increase.
My implementation is not yet multi-threaded, but i'm not sure if thats actually needed. I'll see about that.

Below some performance numbers from my i7 2600k, decoding and processing a 1080p 4:2:0 H264 movie

- Decoding w/ YV12 output: 253 fps
- Old low-quality RGB conversion: 235 fps
- Old high-quality RGB conversion: 40 fps
- New RGB conversion: 180 fps
- New RGB conversion without dithering: 183 fps

And for fun:
- New RGB conversion in Debug mode: 20 fps.
Don't watch stuff with a debug version, people! :)

Speaking of multi-threading, anyone know a simple threadpool implementation that does not require me to use boost? :p

This one uses pthread. But I wonder if it will work in windows.
http://code.google.com/p/cppthreadpool/

Or maybe this one:
http://www.codeproject.com/KB/threads/threads.aspx

Or this one:
http://code.google.com/p/spserver/downloads/detail?name=threadpool-0.2.1.src.tar.gz&can=2&q=

In short, people like having their own wheels.

[Qaq]

Except that it does not work in my case. Even if I add ffdshow audio processor (in MPC-HC "prefer" external filter), it never
shows after LAV audio. I don't understand. :/
Make sure you have Uncompressed - all supported in ffdshow audio - codecs.

[Superb]

Below some performance numbers from my i7 2600k, decoding and processing a 1080p 4:2:0 H264 movie

- Decoding w/ YV12 output: 253 fps
- Old low-quality RGB conversion: 235 fps
- Old high-quality RGB conversion: 40 fps
- New RGB conversion: 180 fps
- New RGB conversion without dithering: 183 fps

And for fun:
- New RGB conversion in Debug mode: 20 fps.
Don't watch stuff with a debug version, people! :)What about ffdshow's RGB (low/high quality) conversion? How fast (fps) is it on your machine (for this file)?

[Snowknight26]

Just select the correct speaker-setup in the Control Panel settings of your sound-card and let the mixer do the rest.

If the input channel count is greater than the output channel count then the extra channels are dropped. No downmixing occurs.

[nevcairiel]

What about ffdshow's RGB (low/high quality) conversion? How fast (fps) is it on your machine (for this file)?

ffdshows conversion is multi-threaded, the comparison wouldn't work.

The only meaningful numbers:

ffdshow with YV12 output: 216 fps
ffdshow with RGB: 200 fps

For a real comparison, wait until i implemented multi-threading. :p

I actually just increased LAV Videos RGB performance as well.
Its now at 220 fps with RGB output, in comparison to 253 fps with YV12.

If the input channel count is greater than the output channel count then the extra channels are dropped. No downmixing occurs.

The DirectSound mixer on Win7 does downmixing.

On XP, that depends on your sound card drivers.

[mindbomb]

I will eventually offer some post-processing, at first only the filters implemented in libavfilter (mostly for yadif), anything else .. we'll see.
I would like to do some plugin type thing, maybe it'll happen. Not soon, though.

Personally, i just don't use post-processing much.

i dont use postprocessing much either, but if lav video had yadif, i imagine nearly 100% of ppl using lav video with madvr would use it.

[BloodySword]

ffdshow also works if you set it to libavcodec for decoding, its all the libfaad2 decoders that fail without the init data.
The more important question, which other splitter makes it work?

Oh, i see. Is the libavcodec decoder as good as the libfaad2 one? Does it decode SBR-Hints?

The splitter was the MPA-Splitter from GABEST. I can look if I can provide the version of his splitter wich I had. I think I did a backup of the old 100000 splitters before using LAV ;). But I never would go back to these xD.

Sorry, the file MPASplitter.ax does not have any version information.

[nevcairiel]

As far as i know, SBR is supported by avcodecs aac decoder.

I can still try to construct the proper codec setup data, not sure how much work it would be to find the right values.

[pirlouy]

Make sure you have Uncompressed - all supported in ffdshow audio - codecs.
But I don't want the ffdshow audio decoder; I only want the ffdshow processor... :/

[BloodySword]

As far as i know, SBR is supported by avcodecs aac decoder.

I can still try to construct the proper codec setup data, not sure how much work it would be to find the right values.

I would test it, but I don't have SBR-AAC at the moment. :/
I'll just trust and believe you! xD

ASAK the Gabset source is on sourceforge. You can take a look at it and reimplement (not copy and paste) it. ;)

[Superb]

ffdshows conversion is multi-threaded, the comparison wouldn't work.

The only meaningful numbers:

ffdshow with YV12 output: 216 fps
ffdshow with RGB: 200 fps

For a real comparison, wait until i implemented multi-threading. :p

I actually just increased LAV Videos RGB performance as well.
Its now at 220 fps with RGB output, in comparison to 253 fps with YV12.Wow. So you actually surpassed ffdshow already, even w/o multi-threading. :O
Amazing work.
Can't wait for the numbers after you add multi threading.

[Midzuki]

@ pirlouy, this is what works over here:

1) dis-able "uncompressed" in the audio decoder

2) en-able "uncompressed" in the audio processor

3) with Graphstudio, or RadLight Filter Manager, set the merit of the audio processor to 0x00800000 or greater

4) reboot, or logoff and logon

5) enjoy :)

[nevcairiel]

Wow. So you actually surpassed ffdshow already, even w/o multi-threading. :O
Amazing work.
Can't wait for the numbers after you add multi threading.

First, to rule out any misunderstandings:
LAV Video already uses multi-threading for decoding, its only about the RGB converter, which is right now single-threaded.

Continuing in the text..

Well, its not 100% fair.
LAV Video will overtax your CPU, using <#cores * 1.5> number of threads, while ffdshow uses a maximum of <#cores> for decoding. So with that rule, LAV Video is faster, only because it uses more decoding threads (and of course more CPU power)

Comparing with LAV Video forced to the same number of threads as ffdshow:

8 threads:

LAV Video:
- YV12: 216 fps
- RGB output: 193 fps

ffdshow:
- YV12: 216 fps
- RGB: 200 fps

1 thread:

LAV Video:
- YV12: 55 fps
- RGB output: 49 fps

ffdshow:
- YV12: 54 fps
- RGB: 52 fps

Note that these are the number of decoding threads, ffdshows RGB converter is always multi-threaded, you really cannot turn that off for a fair 1:1 comparison. :)
Anyhow, multithreading the RGB converter gives a small advantage, i would say it cuts the required processing time about in half. I'll look into adding it tomorrow, should be fairly easy.

[Snowknight26]

The DirectSound mixer on Win7 does downmixing.

Have any more details on this? As far as I can see that doesn't happen at all. If 6-channel audio is passed to the default audio renderer, the audio renderer reports the audio stream as having nChannels = 6 and consequently (for a 2-channel device) 4 channels are dropped.

[mindbomb]

go to control panel, sound, right click on your sound device, and go to configure speakers.
then select stereo.

[e-t172]

Have any more details on this? As far as I can see that doesn't happen at all. If 6-channel audio is passed to the default audio renderer, the audio renderer reports the audio stream as having nChannels = 6 and consequently (for a 2-channel device) 4 channels are dropped.

No. That doesn't happen if your audio device is correctly configured. If the audio device is set to stereo, then the Windows mixer will downmix. Assuming you're not using WASAPI Exclusive or Kernel Streaming, of course.

[CruNcher]

This mixed 64/32 Bit environment is strange, i successfully registered the 64 Bit components but i have problems with the 32 Bit ones :(

http://img546.imageshack.us/img546/9343/registerfail.th.png (http://img546.imageshack.us/img546/9343/registerfail.png)

PS: Funny the Installer fails registering the 64 Bit components, though the 32 bit components register with it crazy :P

Ahhh ok get it syswow64 and system32 are reversed

[Snowknight26]

No. That doesn't happen if your audio device is correctly configured. If the audio device is set to stereo, then the Windows mixer will downmix. Assuming you're not using WASAPI Exclusive or Kernel Streaming, of course.

Guess I was wrong. Seems that ffdshow's mixer has incorrect matrix coefficients for downmixing as they definitely don't match the Windows mixer's.


On a side note, LAV Splitter doesn't parse the 'test_audacity.wav' file in this zip (http://stfcc.org/misc/lavfilters_wav_samples.zip).

[mindbomb]

edit: not a lav issue.

[mindbomb]

nev, can you put in the option to make lav video handle raw formats in future versions?

that way, we can, for example, use the RGB conversion with lav cuvid or cyberlink ham?