LAV Filters - DirectShow Media Splitter and Decoders

[Superb]

In context of ogm files I had to use full name - e.g. 'Japanese' (case sensitive as well) did the thing, usual 'jpn' was not enough.

Perhaps it would be good to add extra sentence or so, that certain containers require full name instead of just the code.Maybe it would be a better idea to make an internal (hashed?)table of all the languages and their shorter names. Automatically infer the longer name...

[nevcairiel]

Funny, only happens with madVR 0.74. :/ However, FFDshow works ok with madVR 0.74.

Indeed.
Its really madVRs fault, it seems to keep a reference on the video decoder, so it doesn't get properly destructed. That happesn with ffdshow as well, its just that ffdshow frees the memory earlier.

I changed it now to free the memory earlier as well, so it doesn't result in catastrophic failure when this happens. In any case, you should report this to madshi.

Edit: I reported it.

[nevcairiel]

In context of ogm files I had to use full name - e.g. 'Japanese' (case sensitive as well) did the thing, usual 'jpn' was not enough.

Perhaps it would be good to add extra sentence or so, that certain containers require full name instead of just the code.

Can you provide a small sample file with this problem?

[madshi]

Its really madVRs fault, it seems to keep a reference on the video decoder, so it doesn't get properly destructed.
Oooopsi.

[VipZ]

I found the reason it didn't work, and i fixed it (it wasn't the media type). I'll commit the change when i get home.

PS:
AR is also fixed again.

PPS:
I also fixed the problem with WMV1/2 not showing any image with the WMVideo decoder, hooray. :)

Awesome, thanks :)

[nevcairiel]

LAV Filters 0.33

LAV Splitter
- Improved compatibility with the MS WMVideo decoder
- Fixed the mediatype for raw PCM streams

LAV Audio
- Added support for Vorbis streams demuxed by Haali and MPC-HC Splitters

LAV Video
- VC-1 decoding is now disabled by default
- Fixed behaviour of the Stream AR option
- The maximum number of decoding threads is now 16
- Free decoding buffers when the input pin disconnects to avoid big memory leaks


Download: Installer (both x86/x64) (http://files.1f0.de/lavf/LAVFilters-0.33.exe) -- Zips: 32-bit (http://files.1f0.de/lavf/LAVFilters-0.33.zip) & 64-bit (http://files.1f0.de/lavf/LAVFilters-0.33-x64.zip)

Its mostly just $random bug fixes, but enough to warrant a new version, including some crash fixes.
VC-1 is now disabled by default because the decoder doesn't really offer any notable advantages over the default Windows decoder (WMVideo Decoder DMO). You can of course just re-enable it, if you want.

This does not yet include any of the performance enhancements for LAV Video, i branched those off and still working on them.

Take care.

[Thunderbolt8]

thanks, the WMVideo Decoder works now (also for interlaced stuff, though still limping on 1 CPU :p)

[ForceX]

Thanks for the quick fix and the new build. :)

[msoltyspl]

Can you provide a small sample file with this problem?

Sure:

http://soltys.ziu.info/temp/a.ogm
http://soltys.ziu.info/temp/b.ogm

The first file has plain 'Japanese' and 'English' as language tags on audio track.
The second one uses 'Japanese[jpn]' and 'English[eng]' (seems more proper ?)

Of course, ogm is relatively ancient and obsoleted these days, though lots of stuff with it still around. Simple workaround is to just list the exact language tag value in the list, works as well.

[nevcairiel]

Just "English" is better then "English[eng]". I can match full language names, i guess, but matching some construct like "English[eng]" is really alot harder.

[VipZ]

Nev, I bet you gona want to add me to the ignore list with EVO :)

With the latest changes, VC1 within EVO doesn't work with the MS WM Decoder any more, in both Win 7 and XP. The chain all connects but playback doesn't start and is stuck at 0:00. LAV Video works as normal. There are no issues with M2TS in both Win7 or XP.

[nevcairiel]

And i know why, but it'll take a few day to fix it, because it needs ffmpeg changes, not feeling like ugly hacks today. :)

[VipZ]

Cool, we all know you don't like ugly hacks :)

[Midzuki]

<OT>

Please define what would be a "beautiful" hack :)

</OT>

[nevcairiel]

There are ugly hacks, and there are just hacks. Ugly hacks are those that you just put in, knowing that its just wrong and will be replaced in a few days.

Anyway, i actually fixed it in ffmpeg already. Its still a hack, but it does what i want it to. :)
The problem in this case was that apparently the EVO container does not contain key-frame infos, and ffmpeg relys on its parser to extract them from the frames. The parser is however off for EVOs, because it mangles the timestamps, so i had to introduce a new parsing mode in ffmpeg which still parses everything, but doesn't mangle the timestamps anymore -- and the WMV decoder relys on me indicating keyframes, otherwise it fails to work properly...

[VipZ]

Thanks Nev, working perfectly :)

[Plutotype]

VC-1 is now disabled by default because the decoder doesn't really offer any notable advantages over the default Windows decoder (WMVideo Decoder DMO). You can of course just re-enable it, if you want.


Hi Nev,
I would like to vote for enabling your VC-1 decoder as default.
Im not sure if also by the others, but in my setup, the WMVideo Decoder DMO causes A/V sync issues, when seeking/jumping in the timeline ( lipsync issues ). With VC-1 videos I mean.
Pluto

[nevcairiel]

Hi Nev,
I would like to vote for enabling your VC-1 decoder as default.
Im not sure if also by the others, but in my setup, the WMVideo Decoder DMO causes A/V sync issues, when seeking/jumping in the timeline ( lipsync issues ). With VC-1 videos I mean.
Pluto

So enable it again. :)
I had some sync issues before as well, but try again with 0.33, it might as well just have vanished. :)

[Thunderbolt8]

thanks, the WMVideo Decoder works now (also for interlaced stuff, though still limping on 1 CPU :p)
actually for some strange reason it doesnt work for a VC-1 1080p 23.976 BD here -.- (BBC source)

[betaking]

To:nevcairiel,can you make lavaudio change decoder format write to Registry like it to lavvideo and lav splitters?
[HKEY_CURRENT_USER\Software\LAV\Splitter\Formats]
"4xm"=dword:00000000
"aac"=dword:00000000
or
[HKEY_CURRENT_USER\Software\LAV\Video\Formats]
"h264"=dword:00000000
"vc1"=dword:00000000
For example made ​​of this
[HKEY_CURRENT_USER\Software\LAV\Audio\Formats]
"aac"=dword:00000000

[ageback]

To:nevcairiel,can you make lavaudio change decoder format write to Registry like it to lavvideo and lav splitters?
[HKEY_CURRENT_USER\Software\LAV\Splitter\Formats]
"4xm"=dword:00000000
"aac"=dword:00000000
or
[HKEY_CURRENT_USER\Software\LAV\Video\Formats]
"h264"=dword:00000000
"vc1"=dword:00000000
For example made ​​of this
[HKEY_CURRENT_USER\Software\LAV\Audio\Formats]
"aac"=dword:00000000
+1
That would be easier for other programs to call LAV Audio Config dialog.

[lauhangwoo]

It's already planned.
http://code.google.com/p/lavfilters/issues/detail?id=9

[betaking]

It's already planned.
http://code.google.com/p/lavfilters/issues/detail?id=9

but,still not change to last 0.34 git! :(

[roytam1]

New nightly: http://roy.orz.hm/lavf-w32-nightlies/lavf-my110823-885aeb29.7z

local fix:
diff --git a/decoder/LAVVideo/LAVVideo.cpp b/decoder/LAVVideo/LAVVideo.cpp
index 9e9961d..0525738 100644
--- a/decoder/LAVVideo/LAVVideo.cpp
+++ b/decoder/LAVVideo/LAVVideo.cpp
@@ -597,7 +597,7 @@ HRESULT CLAVVideo::ReconnectOutput(int width, int height, AVRational ar)
if (!m_settings.StreamAR || num == 0 || den == 0) {
if (m_bForceInputAR) {
DWORD dwARX, dwARY;
- formatTypeHandler(m_pInput->CurrentMediaType().Format(), m_pInput->CurrentMediaType().FormatType(), NULL, NULL, &dwARX, &dwARY);
+ videoFormatTypeHandler(m_pInput->CurrentMediaType().Format(), m_pInput->CurrentMediaType().FormatType(), NULL, NULL, &dwARX, &dwARY);
num = dwARX;
den = dwARY;
m_bForceInputAR = FALSE;

[nevcairiel]

New nightly: http://roy.orz.hm/lavf-w32-nightlies/lavf-my110823-885aeb29.7z

local fix:
diff --git a/decoder/LAVVideo/LAVVideo.cpp b/decoder/LAVVideo/LAVVideo.cpp
index 9e9961d..0525738 100644
--- a/decoder/LAVVideo/LAVVideo.cpp
+++ b/decoder/LAVVideo/LAVVideo.cpp
@@ -597,7 +597,7 @@ HRESULT CLAVVideo::ReconnectOutput(int width, int height, AVRational ar)
if (!m_settings.StreamAR || num == 0 || den == 0) {
if (m_bForceInputAR) {
DWORD dwARX, dwARY;
- formatTypeHandler(m_pInput->CurrentMediaType().Format(), m_pInput->CurrentMediaType().FormatType(), NULL, NULL, &dwARX, &dwARY);
+ videoFormatTypeHandler(m_pInput->CurrentMediaType().Format(), m_pInput->CurrentMediaType().FormatType(), NULL, NULL, &dwARX, &dwARY);
num = dwARX;
den = dwARY;
m_bForceInputAR = FALSE;


Something must be wrong with your checkout, that change is in the repository... :)

[roytam1]

Something must be wrong with your checkout, that change is in the repository... :)

yeah. I recloned it again, rebuild and replaced the archive above.

[madshi]

Hi Nev,
I would like to vote for enabling your VC-1 decoder as default.
Im not sure if also by the others, but in my setup, the WMVideo Decoder DMO causes A/V sync issues, when seeking/jumping in the timeline ( lipsync issues ). With VC-1 videos I mean.
Pluto
Why not trying to fix the AV sync issues instead of going back to a slower and less capable (interlaced) decoder? Of course the question is whether the AV sync issues are caused by the decoder or the splitter or by a "misunderstanding" between the splitter and decoder. I'd suggest to double check with other splitters to get an idea about whether it's a problem with the MS VC-1 decoder itself.

[nevcairiel]

I believe the sync issues are fixed. I had that problem before as well, but some quick tests last night showed promising results.
The WMVideo decoder strongly relys on the proper signaling of sync points in the samples, but when i wrote that part, i basically copied that from the MPC-HC MPEG Splitter, which was wrong. I now changed it to detect key-frames in the actual content, and signal those as sync points, and playback greatly improved with the WMVideo decoder. I believe it also uses those sync points to sync its timestamps, which caused the desync before.

The only thing missing now is also detecting keyframes for MPEG-2 video...

[fastplayer]

I'm not sure if this is 100% LAV-related but it brings MPC-HC down to its knees whereas its internal filters work fine.
Here are the steps to reproduce:
1. http://download.microsoft.com/download/e/a/d/eadb9b42-728b-42b0-bfdf-b472fa2a2464/Step_into_Liquid_1080.exe
2. Make sure "Rewind when done playing" is disabled in MPC-HC.
3. Start playback, seek to the near end of the video, wait until playback stops.
4. Now start playback by clicking on the video or pressing Space key.
5. Result: With MPC's internal filters/splitters, playback restarts as expected. Using LAVFilters, MPC-HC freezes 2 out of 3 times.

My setup: madVR 0.74 (windowed mode), LAV 0.33 (default settings), MPC-HC 3694.

Can anyone else reproduce this?

Edit: I narrowed it down to LAVVideo. If it's disabled, restarts are OK.

[nevcairiel]

There is definitely something odd going on when using Microsofts WM ASF Reader as a source filter, which causes the hang you're seeing.
For some reason, it sends a data packet to decode, and at the same time, on another thread, its calling the Stop method on the filter, and eventually this results in a hang. Very odd.

I cleaned up some things that should stop the hang in this case, but still, its weird.

[JarrettH]

I believe the sync issues are fixed. I had that problem before as well, but some quick tests last night showed promising results.
The WMVideo decoder strongly relys on the proper signaling of sync points in the samples, but when i wrote that part, i basically copied that from the MPC-HC MPEG Splitter, which was wrong. I now changed it to detect key-frames in the actual content, and signal those as sync points, and playback greatly improved with the WMVideo decoder. I believe it also uses those sync points to sync its timestamps, which caused the desync before.

The only thing missing now is also detecting keyframes for MPEG-2 video...

When I think of MPEG2 video, I think of DVDs :p:cool:

[fastplayer]

There is definitely something odd going on when using Microsofts WM ASF Reader as a source filter, which causes the hang you're seeing.
For some reason, it sends a data packet to decode, and at the same time, on another thread, its calling the Stop method on the filter, and eventually this results in a hang. Very odd.

I cleaned up some things that should stop the hang in this case, but still, its weird.
I'm glad this all makes sense to you. :)
It's not a biggie though. I use the sample above only for testing and thought I'd report it. It seemed odd to me that MPC's internal decoder handles it just fine while LAVVideo stumbles over. Usually, it's the other way around! :D

[Plutotype]

I believe the sync issues are fixed. I had that problem before as well, but some quick tests last night showed promising results.
The WMVideo decoder strongly relys on the proper signaling of sync points in the samples, but when i wrote that part, i basically copied that from the MPC-HC MPEG Splitter, which was wrong. I now changed it to detect key-frames in the actual content, and signal those as sync points, and playback greatly improved with the WMVideo decoder. I believe it also uses those sync points to sync its timestamps, which caused the desync before.

The only thing missing now is also detecting keyframes for MPEG-2 video...

The MS VC-1 decoder I use, is either from WMP12 or comes with the AMD Catalyst package. I have tested the one in ffdshow (wmv9) and the A/V lipsync issues when jumping in the timeline are exactly the same. The wmv9 decoder in ffdshow is not recommended at all - in my setup, when starting a movie or seeking in the timeline, it takes almost 5 seconds to start synchronised A/V playback ( its way too slow ).

Is there any other/updated MS VC-1 decoder which I can install and test?

Thanks

[nevcairiel]

The MS decoder comes with WMP, and on Win7 there is no way to upgrade it, as it already comes with the latest as-is. :)

I mean the actual decoder, not the one in ffdshow. The filter called "WMVideo Decoder DMO"

[Plutotype]

So I will stick to your libav-based VC-1 decoder.

[madshi]

So I will stick to your libav-based VC-1 decoder.
Have you tried the "WMVideo Decoder DMO"? It works best for me.

[Plutotype]

Have you tried the "WMVideo Decoder DMO"? It works best for me.

yes, it gives me occasional lipsync issue when seeking in a VC-1 video. Tested on Batman Begins, Blood Diamond and Eyes Wide Shut BD.

[nevcairiel]

Did you test all that after LAV 0.33?

I tested batman begins earlier, and i couldn't spot any problems.

[Plutotype]

Yes, I have 0.33, madVR 0.74, MPC-HC 1.5.3.3677, Reclock 1.8.7.7
WMP 12.0.7601.17514

The weird thing is, audio comes later than video, when the problem occurs.

According to madVR OSD, WMVideo Decoder DMO outputs in NV12 and LAVvideo in YV12. Is this correct?

[pirlouy]

Have you tried the "WMVideo Decoder DMO"? It works best for me.
Just for my curiosity, and like it's not the first time you say it, why do you prefer this decoder ? Is it only because of CPU usage, or is there another advantage ??

But indeed, in my case, LAV 0.33 seems to have fixed the sync problem I had with some VC1 Blu-Ray (though it was Microsoft decoder's fault). Thanks.

@nevcairiel: can you post it at the end of this thread, when there's an update please ? or RSS or whatever please ? I've missed the 0.33 for example.

[nevcairiel]

Just for my curiosity, and like it's not the first time you say it, why do you prefer this decoder ? Is it only because of CPU usage, or is there another advantage ??


Its present on every system and can decode basically every stream.
It just works.

I use LAV CUVID, but if i wouldn't have a NVIDIA card, i would probably use the MS decoder.

@nevcairiel: can you post it at the end of this thread, when there's an update please ? or RSS or whatever please ? I've missed the 0.33 for example.

I always post a release announcement here in this thread.
http://forum.doom9.org/showthread.php?p=1521184#post1521184

[madshi]

Its present on every system and can decode basically every stream.
It just works.
Agreed. And it's faster than both the libav/ffmpeg and the Intel Media SDK software VC-1 decoders.

[nevcairiel]

Some news from the optimizations of the pixel format converters:

I noticed last night that i do not need to code converters for big-endian formats, hooray. ffmpeg always outputs in native endianness, and on x86 thats always little-endian. That saved alot of work and made me happy. :)
Last night, i wrote a new converter for YUV420 9/10bit -> YV12/NV12. Its again notably faster then the old one, especially for NV12 output.

Next up are converters for YUV420/422 9/10bit -> P010/P210, and YUV422 -> YUY2. When those are done, all common unscaled YUV conversions will have a optimized converter.

On the weekend, i will tackle chroma upscaling, YUV420 -> YUV422 -> YUV444 -> RGB (with entry and exit points at every step in the chain)
All upscaling will be done in 16-bit integer (RGB conversion with 32-bit intermediate results), and in the end dithered back down to the desired output bitdepth (currently always 8-bit, 10-bit output is only supported for untouched chroma)

I've been pondering on something, though. Is there some real (noticeable) quality loss when dithering the YUV to 8bit before the RGB conversion, or should i take the extra effort and build the RGB converter to actually work with 10-bit input pixels?

[pirlouy]

Its present on every system and can decode basically every stream.
It just works.
Yes, but EVR renderer works too. It displays all files. And that does not means it's the best one.

I use LAV CUVID, but if i wouldn't have a NVIDIA card, i would probably use the MS decoder.
People say GPU drivers caused different problems. And you said the advantage of LAV CUVID is deinterlacing, so if you only have VC1 from Blu-Ray, LAV Cuvid is not necessarily the best solution.

And it's faster than both the libav/ffmpeg and the Intel Media SDK software VC-1 decoders.
In my case, there's not a lot of difference (both at 20% for Harry Potter 2 in my case). The Intel one from madVR uses more CPU (30% for HP2).

Like LAV Splitter is based on ffmpeg, and MS decoder does not bring advantages (except the support of interlaced files I don't use), I prefer to stay with ffmpeg decoder (LAV seems to use a bit less CPU than madVR one). :-)

I always post a release announcement here in this thread.
http://forum.doom9.org/showthread.php?p=1521184#post1521184
Sorry. :/

[nevcairiel]

Yes, but EVR renderer works too. It displays all files. And that does not means it's the best one.

How a file is to be decoded is 100% specified, there are no differences in quality between decoders, so the comparison is kinda off.


People say GPU drivers caused different problems. And you said the advantage of LAV CUVID is deinterlacing, so if you only have VC1 from Blu-Ray, LAV Cuvid is not necessarily the best solution.

Blu-rays also contain VC-1 interlaced, especially documentaries and some concert discs. I just don't want to switch decoders when watching one of those.

Besides, i never had any real trouble with NVIDIA drivers. All complaining i hear is really mostly from ATI/AMD users, but i have no first-hand experience either way.


Like LAV Splitter is based on ffmpeg, and MS decoder does not bring advantages (except the support of interlaced files I don't use), I prefer to stay with ffmpeg decoder (LAV seems to use a bit less CPU than madVR one). :-)

Its a format designed by MS, so why would their decoder be bad? :)
Its faster (less CPU), and it supports more types of movies.

Anyway, in the end its your choice which decoder to use, and if you never watch VC-1 interlaced, then the ffmpeg decoder will probably be fine as well - i just don't think it has any advantages over the MS decoder, thats why its off by default now.
If interlaced support is added to ffmpeg (some day), i'll probably enable it again.

[madshi]

@pirlouy, I'm not really sure what you're trying to say. We've given you 2 good reason why we think the MS VC-1 decoder is the better default decoder: (1) Speed. (2) Interlaced decoding capability. So you're personally not interesting in either of these advantages. Ok, but what is your point? You want us to use the ffmpeg VC-1 decoder as default, although it's slower and less capable? That doesn't make any sense.

[madshi]

Some news from the optimizations of the pixel format converters:

I noticed last night that i do not need to code converters for big-endian formats, hooray. ffmpeg always outputs in native endianness, and on x86 thats always little-endian. That saved alot of work and made me happy. :)
Last night, i wrote a new converter for YUV420 9/10bit -> YV12/NV12. Its again notably faster then the old one, especially for NV12 output.

Next up are converters for YUV420/422 9/10bit -> P010/P210, and YUV422 -> YUY2. When those are done, all common unscaled YUV conversions will have a optimized converter.

On the weekend, i will tackle chroma upscaling, YUV420 -> YUV422 -> YUV444 -> RGB (with entry and exit points at every step in the chain)
All upscaling will be done in 16-bit integer (RGB conversion with 32-bit intermediate results), and in the end dithered back down to the desired output bitdepth (currently always 8-bit, 10-bit output is only supported for untouched chroma)
Sounds good to me! Will the chroma upscaling also support 10bit input?

I've been pondering on something, though. Is there some real (noticeable) quality loss when dithering the YUV to 8bit before the RGB conversion, or should i take the extra effort and build the RGB converter to actually work with 10-bit input pixels?
You'd have to implement dithering twice this way. I've no idea what happens to ordered dithering if you apply it twice. With random dithering (as used by madVR) applying dithering twice increases the noise floor.

[nevcairiel]

Sounds good to me! Will the chroma upscaling also support 10bit input?

Thats easy enough, because 10bit is stored in 16bit internally anyway, and i do need to do processing in 16-bit. I need 4 bit free space for the upscaling, so anything up to 12-bit can be natively processed with the algorithm i have in mind.

You'd have to implement dithering twice this way. I've no idea what happens to ordered dithering if you apply it twice. With random dithering (as used by madVR) applying dithering twice increases the noise floor.
Hm, yeah. I guess it won't be that much more work to actually support 10-bit input, for similar reasons as above. Just have to keep track of the number of bits that are actually valid, so i can apply the proper dithering and shifting at the end.

[madshi]

Maybe you can upconvert everything to 12bit first and then write routines only for that bitdepth? Then for the final step you'd need downconversion routines to 8bit, 9bit and 10bit, I guess.

Edit: Wait, this is all for RGB output, right? In that case you only need 8bit output, I guess, because there are no 9bit or 10bit RGB output FOURCCs. At least I don't know any. You could optionally output 16bit RGB, though.

[nevcairiel]

Yeah i thought about that, it adds one shift operation to every pixel, but i guess the performance difference using SSE2 is minimal, and simplifys the code a bit (the alternative would be to use a template and let the compiler generate three versions of the function for 8, 9 and 10bit each, without much special code).

While i have your attention, just to confirm my logic is alright.
Looking at the MPEG-2 Chroma position, for the 4:2:0 -> 4:2:2 conversion i would use a simple 75:25 interpolation. For the second step, the 4:2:2 -> 4:4:4, i would just use a 50:50 interpolation. Its not really a very fancy algorithm, but its similar to what ffdshow uses, and the quality seems alright. Or am i missing something?


Edit: Wait, this is all for RGB output, right? In that case you only need 8bit output, I guess, because there are no 9bit or 10bit RGB output FOURCCs. At least I don't know any. You could optionally output 16bit RGB, though.
Technically, yes. Although some renderers (Haali) only accept YUY2 (or RGB) input, so at least the 4:2:0->4:2:2 upsampling would be used there as well, but still limited to 8-bit.
Any conversions are really only for renderes or post-processing filters that don't know any better - and those will probably never understand 10bit.

Edit:
Technically, one could use D3DFMT_A2R10G10B10 as a FourCC, the MSDN claims its similar in usage, but i guess nothing supports it anyway.