LAV Filters - DirectShow Media Splitter and Decoders

[roytam1]

@nev:
missing revision again ;-)
fatal: reference is not a tree: f43d0060ca04ecbed8cd5a87b9feb33220192064
Unable to checkout 'f43d0060ca04ecbed8cd5a87b9feb33220192064' in submodule path 'libbluray'

[Keiyakusha]

I have problems with 0.52 stable. I even reinstalled windows before I figured out what is the problem...
So after installing LAV 0.52 both x86 and x64 on Win7 Sp1 fully updated and registering its splitter for avi extension - some players play .avi fine, but some able to display only 1st frame of the stream.
Player I'm talking about written in mixture or C and С# .NET but I can't share it.
Then by unchecking avi in LAV splitter I'm losing ability to play avi files at all! If only player doesn't have something to fallback internally to... graphstudio for example returns "can't render file", mentioned above player crashes with exception CLR20r3
Only complete LAV uninstall returns ability to play avi... Is that intended behavior? As I already said, windows is new, nothing media-related but LAV and madvr was ever installed.

Edit: actually 0.51.3 is no different. Maybe this is problem with installer, just tried manually registering filters (as I always did before, only x86 version) and unchecking avi in splitter settings works correctly. I'm still unable to go beyond 1st frame with mentioned player though and so far this happens only with LAV as decoder... but probably I should live with that.

[nevcairiel]

If you use the installer, it will setup LAV to be the source filter for avi, and because these settings are in protected areas of the registry, the LAV Splitter properties page cannot change that (can only have on source filter per extension, and if it fails, windows assumes the file just won't work). If you want to remove one format, you need to re-install it and uncheck avi in the installer.
Didn't i add a note about that on the properties page?

Only getting one frame sounds a lot like it fails at a dynamic format change.

[Keiyakusha]

If you want to remove one format, you need to re-install it and uncheck avi in the installer. Didn't i add a note about that on the properties page?
Nope :P I see note that primary formats should be configured through installer. To me it means "hey, you know that there is an installer, right? it is easier to do some stuff there, that's what usually installers are for" and my 1st thought is "meh, why would i need that?"
Maybe this is partially because I don't know what is primary formats... for years my primary formats are .mkv and .mov ^__^
Also this is not really intuitive to have checkboxes that in some cases can only break stuff if unchecked. If there can be only one source filter, and changes can't be rolled back by property page, maybe it is better do not allow to change them at all if installer used?

Only getting one frame sounds a lot like it fails at a dynamic format change.
This is probably the case. Not that I can accept format changes as decoded frames go directly into real 3d environment that won't be happy about that. I remember you was explaining something about this behavior, so not a problem then, I'll try to use something like ffms2/vapoursynth input anyway.

[Joniii]

This BD doesn't work with LAV filters. It plays fine with LAV on MPC-HC but fails to play on WMP or WMC, it also reports playtime as 1200 minutes or so. It works fine with ffdshow and WMP/WMC. Dunno if it's a LAV problem or LAV+VSfilter, it has PGS subtitles. Out of my 100+ remuxes i've ripped to my HTPC, this is the only one that doesn't work on LAV.

Dark Shadows sample (50Mb) (http://sdrv.ms/Q61lmZ)

[nevcairiel]

This BD doesn't work with LAV filters. It plays fine with LAV on MPC-HC but fails to play on WMP or WMC, it also reports playtime as 1200 minutes or so. It works fine with ffdshow and WMP/WMC. Dunno if it's a LAV problem or LAV+VSfilter, it has PGS subtitles. Out of my 100+ remuxes i've ripped to my HTPC, this is the only one that doesn't work on LAV.

Dark Shadows sample (50Mb) (http://sdrv.ms/Q61lmZ)

Your sample plays just fine in WMP for me. Says 19 seconds runtime, and plays perfectly, but i don't have VSFilter setup.
I'm not even sure how to setup VSFilter to get loaded in WMP, i don't really care for that player at all =p

Overall, i usually tell people that with WMP they are on their own. One would think that Microsoft would implement their own DirectShow properly, but they don't.

Edit:
Managed to make it load VSFilter, and it still plays just fine.

[clsid]

Any tips on how I can checkout a specific old revision of the FFmpeg submodule? For example:
http://code.google.com/p/lavfilters/source/detail?r=32584bb0aa0eb89bea9cdd13bc3cc224a3df2c18
This did not work:
git checkout 6f1788a98096036ed7b458489393f7fb33320a2c
I can of course select a specific commit in the log, but that doesn't give me the desired combo of upstream FFmpeg + your custom patches.

[nevcairiel]

You can only checkout old ffmpeg versions which are tagged, so for this case it means you can only get the ffmpeg versions present in the release versions of LAV, because thats when i tagged ffmpeg as well. All other revisions get removed from the repository when i move to the next.

[cyberbeing]

LAV Video appears to be using "MPEG-1 centered chroma placement" instead of "MPEG-2 left chroma placement" for DVD subtitles.

[nevcairiel]

LAV Video appears to be using "MPEG-1 centered chroma placement" instead of "MPEG-2 left chroma placement" for DVD subtitles.

If it does, then thats a bug in swscale, and nothing i care about. Blending subtitles in YUV 4:2:0 will always have a high chance of color bleeding.

[cyberbeing]

Use of centered chroma placement is a swscale issue when performing RGB->YCbCr conversion.

Left chroma placement does eliminate at least 90% of 4:2:0 color bleeding, when used with madVR. We saw a rather significant improvement when we fixed the chroma placement code in xy-VSFilter.

[nevcairiel]

I sure don't feel like fixing swscale, the code is so terribly messy. I wonder if swscale does the right thing when i convert it to yuv444 first and then downscale chroma to 420, but probably not..

[SamuriHL]

Hey Nev...is this me?


6>c:\dev\projects\lavfilters\ffmpeg\libavutil\x86/timer.h(33): error C2065: '__a
sm__' : undeclared identifier
6>c:\dev\projects\lavfilters\ffmpeg\libavutil\x86/timer.h(33): error C2143: synt
ax error : missing ';' before 'volatile'
6>c:\dev\projects\lavfilters\ffmpeg\libavutil\x86/timer.h(33): error C2143: synt
ax error : missing ';' before 'volatile'
6>c:\dev\projects\lavfilters\ffmpeg\libavutil/internal.h(174): error C2065: '__a
sm__' : undeclared identifier
6>c:\dev\projects\lavfilters\ffmpeg\libavutil/internal.h(174): error C2143: synt
ax error : missing ';' before 'volatile'
6>c:\dev\projects\lavfilters\ffmpeg\libavutil/internal.h(174): error C2143: synt
ax error : missing ';' before 'volatile'
6>c:\dev\projects\lavfilters\ffmpeg\libavutil/internal.h(174): error C2589: ':'
: illegal token on right side of '::'

[Xaurus]

nev,

I am having some issues sorting out the subtitle selection syntax. Right now I am using your example from the readme file.
I am currently trying to load english subtitles from a german bluray. When I check the m2ts with Mediainfo, there is no mentioning of language for audio (nor for the subs). Where can I see what language they are in?
I checked a few american blurays and it's the same thing there, really. :confused:

[nevcairiel]

The m2ts on a Blu-ray don't contain language information, its all contained in the metadata files on the Blu-rays. If you play the discs and look at the stream selector in MPC-HC, it will show the languages if it can figure them out, or well, not if it can't. :p

[Xaurus]

The m2ts on a Blu-ray don't contain language information, its all contained in the metadata files on the Blu-rays. If you play the discs and look at the stream selector in MPC-HC, it will show the languages if it can figure them out, or well, not if it can't. :p
Alright, that explains a lot. Do you know if any program can add a language tag to such a file, or is it simply not possible for that container format?

I thought about storing some blurays "untouched" by just copying the m2ts files to a hdd. But I see that this can cause some language trouble.

[cyberbeing]

I sure don't feel like fixing swscale, the code is so terribly messy. I wonder if swscale does the right thing when i convert it to yuv444 first and then downscale chroma to 420, but probably not..
If swscale yuv444->yuv420 doesn't work, would it possible to use swscale conversion to yuv444 and then make use xy-VSFilter's optimized hleft_vmid 4:2:0 chroma sub-sampling functions (http://repo.or.cz/w/xy_vsfilter.git/blob/refs/heads/pgs_n_bug_fix:/src/subpic/xy_intrinsics.h)?

[nevcairiel]

The optimized versions can most likely not be used because subtitles rarely are mod16, they have all sorts of odd sizes, and for these small bitmaps trying to pad them to the proper size will most likely be more effort then its worth in the end.
I can probably take the subsampling algorithm and use it, if all else fails.

[Joniii]

Your sample plays just fine in WMP for me. Says 19 seconds runtime, and plays perfectly, but i don't have VSFilter setup.
I'm not even sure how to setup VSFilter to get loaded in WMP, i don't really care for that player at all =p

Overall, i usually tell people that with WMP they are on their own. One would think that Microsoft would implement their own DirectShow properly, but they don't.

Edit:
Managed to make it load VSFilter, and it still plays just fine.

Weird. I have clean install of W8, just LAV filters and VSfilter, no codec packs or anything. I checked the grapgedit and all looks good.

I found out that the files I splitted only the first one doesn't play, part 2 and others work fine, dunno if there is some header problem. Also if I remux that sample with mkvtoolnix without the DTS Master Audio track, it plays fine.

I've set LAV audio to only bitsream AC3 and DTS as my receiver doesn't support DTS-HD. For some reason in Windows 8 WMP/WMC, LAV audio can't decode DTS core from that mkv. Most of my remuxes have DTS-MA and core from those work fine on LAV Audio. Dunno if there is something unusual on this audio track, i've also tried to rip the BD again with eac3to but with the same results.

edit:
I tried to extract the core with eac3to and adding that into the mkv instead of DTS-MA and it worked. Dunno why DTS-MA doesn't.

Could it be that LAV audio can't get the core from that DTS-MA track?

[nevcairiel]

I tried bitstreaming the core only and it works just fine, both in WMC and MPC-HC. But i only use Win7, no idea whats different in Windows 8, you're on your own there.

I suggest to use decoding anyway, especially if your receiver cannot do HD formats.

[SamuriHL]

I bitstream on W8 just fine.

[iSunrise]

Guys, can someone enlighten me, why my Conan the Barbarian NTSC DVD looks a lot sharper, crisper and stable when I manually have "Treat as progressive" enabled in LAV Video instead of just keeping everything at their defaults. I get some weird "line flickering" that kinda looks like LAV Video/CUVID has a hard time of figuring out the right deinterlacing method. What is even more strange is that I don´t see any combing artefacts with "Treat as progressive" checked, even with fast movements.

Isn´t that something LAV Video/CUVID should do automatically? Or did I find such a rare case that actually needs me to manually alter the settings?

[Joniii]

I tried bitstreaming the core only and it works just fine, both in WMC and MPC-HC. But i only use Win7, no idea whats different in Windows 8, you're on your own there.

I suggest to use decoding anyway, especially if your receiver cannot do HD formats.

I bitstream on W8 just fine.

Weird. It also fails with just LAV splitter and video and MS audio decoder. Haali, and LAV Video + Audio works.

SamuriHL, did you try to bitstream just the core, disabled DTS-HD from LAV audio?

[cyberbeing]

After the revised pixel format changes, when NV12/P010/P016 are unavailable and YV12 is supported, LAV Video seems to be negotiating YUY2 with xy-VSFilter over YV12 for 4:2:0 video.

[nevcairiel]

How can i reproduce that properly? What doesn't support NV12 but supports YV12?
I grouped pixel formats into groups, and the 420 8-bit group always includes NV12 and YV12, in that order.

Edit:
I managed to reproduce the problem using ffdshow raw after VSFilter, however i see nothing that i can influence.
LAV exposes a long list of subtypes, starting with NV12 and YV12, then P010/P016, and only rather late in the list is YUY2.

When VSFilter connects its output pin, it tries to re-connect its input pin to LAV, and VSFilter influences which type it uses for this reconnection. All i get are calls to GetMediaType (iterating over all the types LAV exposes), and then VSFilter probes with QueryAccept in this order: P010, P016, YUY2. There it stops, and performs a connection with YUY2.

Don't think anything is going wrong on my end.
Are you sure it started with 0.52? All i did really was add more formats to the mapping, so that every output format is mapped for every input format, but YUY2 was always a possible format for 420 content.

[cyberbeing]

0.51.3 output pin preference order for 8bit 4:2:0 video (NV12 disabled):
YV12
YUY2
UYVY
RGB32
RGB24

0.52.0 output pin preference order for 8bit 4:2:0 video (NV12 disabled):
YV12
P010
P016
P216
P210
v210
YUY2
UYVY
RGB32
RGB24
Y416
Y410
v410
YV24
AYUV
YV12 <--Duplicate YV12 listing as the least preferred format

I expect that LAV Video listing YV12 as both most preferred and least preferred format is causing this issue. Could you fix that?

Also shouldn't the proper order for 8bit 4:2:0 be something like the following normally:
NV12
YV12
P010
P016
YUY2
UYVY
P210
P216
v210
YV24
Y410
Y416
v410
RGB32
RGB24
AYUV

[nevcairiel]

I put the high-bitdepth formats before the 8-bit formats because upscaling chroma will result in more then 8 bits, so if whatever comes afterwards supports more then 8, why not give it to them.
Not sure that makes sense, i was thinking about changing the order a bit again. I'll see why the YV12 is duplicated.

[madshi]

I put the high-bitdepth formats before the 8-bit formats because upscaling chroma will result in more then 8 bits, so if whatever comes afterwards supports more then 8, why not give it to them.
Good thinking, IMHO. Not that it will make a visible difference in 99.99% of all situations, but it shouldn't harm, either (except maybe lowering performance ever so slightly).

[nevcairiel]

Its a rather rare case anyway. First of all it has to be something that doesn't support YV12/NV12, but DOES support 10-bit 4:2:2 ... how common is that? :p

[nevcairiel]

I'll see why the YV12 is duplicated.

I found the reason. It happened because i defined a pixel format which i didn't use yet, so the count was wrong, and the array had an extra 0 at the end .. and 0 happens to be YV12. :p

Should be fixed

[SamuriHL]

Weird. It also fails with just LAV splitter and video and MS audio decoder. Haali, and LAV Video + Audio works.

SamuriHL, did you try to bitstream just the core, disabled DTS-HD from LAV audio?

No I only bit stream the full hd audio streams.

Sent from my Xoom using Tapatalk 2

[Joniii]

No I only bit stream the full hd audio streams.

Sent from my Xoom using Tapatalk 2

Could you test if it works when LAV outputs only the core, disable DTS-HD from the settings?

I did a full format today and first thing I did was installed latest catalyst then lav, after that tried to play the file again but had the same problem.

I guess this is one of those mysterious things.

[cyberbeing]

Blending subtitles in YUV 4:2:0 will always have a high chance of color bleeding.

DVD subtitles with AYUV/YV24 4:4:4 video output in LAV Video seem to have massive chroma bleeding as well.

Is direct RGB -> AYUV/YV24 DVD subtitle conversion and alpha-blending not supported by LAV, or is this another swscale issue?

LAV Video 0.52.0 400% Crop (http://imageshack.us/a/img39/6416/lav400crop.png)
(DVDNavigator-> LAV Video AYUV [deinterlacing disabled] -> madVR [deinterlacing disabled])

xy-VSFilter 400% Crop (http://imageshack.us/a/img28/9130/xyvsflt400crop.png)
(MKV DVD Remux -> LAV Video AYUV [deinterlacing disabled] -> xy-VSFilter -> madVR [deinterlacing disabled])

I found the reason. It happened because i defined a pixel format which i didn't use yet, so the count was wrong, and the array had an extra 0 at the end .. and 0 happens to be YV12. :p

Should be fixed
Tested the fix in LAV Video (git 224c816), and the issue preventing YV12 connections to xy-VSFilter was resolved.

[mark0077]

Hi nev, are you still interested in DVD samples that cause issues with LAV Video? Apart from the problems I mentioned before with the Star Wars DVD's, theres some others that cause freezes of the player when highlighting menu items. I can prepare a sample if you'd have time to take a look.

[nevcairiel]

DVD subtitles with AYUV/YV24 4:4:4 video output in LAV Video seem to have massive chroma bleeding as well.

Is direct RGB -> AYUV/YV24 DVD subtitle conversion and alpha-blending not supported by LAV, or is this another swscale issue?

LAV Video 0.52.0 400% Crop (http://imageshack.us/a/img39/6416/lav400crop.png)
(DVDNavigator-> LAV Video AYUV [deinterlacing disabled] -> madVR [deinterlacing disabled])

xy-VSFilter 400% Crop (http://imageshack.us/a/img28/9130/xyvsflt400crop.png)
(MKV DVD Remux -> LAV Video AYUV [deinterlacing disabled] -> xy-VSFilter -> madVR [deinterlacing disabled])

That looks odd.
I'll check in more detail whats going on.

Edit:
Note that subtitles are blended before LAV does its color conversion, except when that conversion is RGB.
So when your video is 4:2:0, and you output AYUV, it will still blend the subtitles in 4:2:0. If you output RGB, it'll blend in RGB.

I did this because LAV supports quite a lot different output formats, but only a short list of internal pixel formats from the decoders, and supporting them is much easier then supporting every output format.
I only made a special case for RGB because there is only one RGB format, and its the same as the internal RGB format, so the code was already there.

So in short, if you want better looking subs, use LAVs RGB conversion. Especially the 4:2:0->4:4:4 upsampling is of low quality anyway, while 4:2:0->RGB is higher quality. (I know, it seems odd, but forcing 4:4:4 output is rather unusual so i did not implement that path, and only let my "fallback" swscale code handle it)

I will still look into the chroma placement issue, possibly eliminating swscale for subs completely unless i need to actually change the size of the subs.

There is also an issue with subs on odd-line positions.
Say you have a 720x576 image, and the subtitle goes from line 305 to 335. Luma can be positioned accurately, but Chroma is another matter in 4:2:0. Do i start blending chroma at chroma-line 152 or 153? Whichever i choose, it'll bleed either into the image, or into the subtitle.
I considered repositioning them to start on even lines, and be of even height, but that screws with the pixel-exactness of the menu overlays, which looks bad.

[nevcairiel]

Apart from the problems I mentioned before with the Star Wars DVD's, theres some others that cause freezes of the player when highlighting menu items. I can prepare a sample if you'd have time to take a look.

I saw some odd freezes myself, but couldn't see LAV do anything at the time.

If you're using madVR, can you double check that the error also occurs with EVR?
If not, maybe you can instead provide a freeze report from madVRs freeze functionality (ideally download the LAV debug symbols from the first post before, put them next to the .ax/.dll files)

Edit:
Managed to reproduce it and get a proper callstack. I think i can fix it.

[cyberbeing]

That looks odd.
There is also an issue with subs on odd-line positions.
Say you have a 720x576 image, and the subtitle goes from line 305 to 335. Luma can be positioned accurately, but Chroma is another matter in 4:2:0. Do i start blending chroma at chroma-line 152 or 153? Whichever i choose, it'll bleed either into the image, or into the subtitle.
I considered repositioning them to start on even lines, and be of even height, but that screws with the pixel-exactness of the menu overlays, which looks bad.

Paint the VOBSUB palette in 4:4:4, subpixel position the painted 4:4:4 bitmap, and finally convert the 4:4:4 bitmap to 4:2:0 for alpha blending into the video?
I believe that's how VSFilter does it, but I'm not too familiar with the VOBSUB code.

LAV Video NV12 (misaligned centered chroma) (http://imageshack.us/a/img843/3315/lav400cropnv12.png)
VSFilter YV12 (centered chroma) (http://imageshack.us/a/img716/1603/vsflt400cropyv12.png)
xy-VSFilter NV12 (left chroma) (http://imageshack.us/a/img832/7103/xyvsflt400cropnv12.png)

[nevcairiel]

Patches are welcome. :)

I used a straight forward approach. Get subtitle in RGBA, if we need to, convert to an appropriate YUV with swscale, and perform blending.
You really cannot compare those VSFilter images, because there is clearly something else going on that tries to mask the vobsub aliasing by softening them, thats not a result from chroma placement. Most VobSubs don't have that level of anti-aliased edges with transparency, in fact i don't think i ever saw one.

I will most likely instead investigate ways to interface directly with the new subtitle interface, leaving the blending, positioning and conversion to madVR in RGB.
In fact, i already used this internally, although i have to update to the latest version and add all the officially required parameters.

[ryrynz]

For anyone using Xhmikosr's builds, I've been having random bluescreen's and audio drop outs with his recent build 0.52-28-g2889377.

[nevcairiel]

Sounds more like your system is screwed up. Software in user-mode does not cause bluescreens.
Updated a driver lately? =p

[ryrynz]

It's not, I reinstalled 0.52 and haven't had an issue since, it could be his building? I did have similar issues with his ffdshow which I've dropped.
Oh and I'm using Quicksync.

[nevcairiel]

Its certainly possible something is screwed up in the build, i'll post a fresh test build when i get back to my dev PC in a few hours, the laptop doesn't have all the toys.

[ryrynz]

Choice, I don't expect it's anything you've changed, but a test build will make that certain.

[noee]

fwiw, you're "worker thread" fix for the deadlock is working here on two discs that were previously locking (jedi and new hope) when returning from the movie to the menu, then selecting a sub-menu.

[cyberbeing]

You really cannot compare those VSFilter images, because there is clearly something else going on that tries to mask the vobsub aliasing by softening them, thats not a result from chroma placement.

It's not a straight chroma placment comparison, but you can still easily compare the chroma bleeding differences.

If I were to take a guess, the softer luma/chroma either a result of VSFilter not snapping the edges of bitmaps to whole pixels, or possibly vectorizing the VOBSUB bitmap outline. I'll try to find out.

I will most likely instead investigate ways to interface directly with the new subtitle interface, leaving the blending, positioning and conversion to madVR in RGB.
That's probably the best solution.

[nevcairiel]

If I were to take a guess, the softer luma/chroma either a result of VSFilter not snapping the edges of bitmaps to whole pixels, or possibly vectorizing the VOBSUB bitmap outline. I'll try to find out.

I briefly scrolled through VSFilters VobSub source, and it looks like it indeed tries to vectorize the VobSub outlines, which allows it to re-draw them using a much better resolution.
Not something i have on my plans =p

[wanezhiling]

http://www.mediafire.com/?w4b63h295st76np
LAV cant decode the TAK file.

[nevcairiel]

http://www.mediafire.com/?w4b63h295st76np
LAV cant decode the TAK file.

The correct expression is "ffmpeg can't decode the TAK file." :P

I suggest to report this as a feature request over at the ffmpeg trac, if you want it to be implemented eventually.

[wanezhiling]

Ok I see.

[nevcairiel]

Here is a new test build, with all recent improvements to DVD menu rendering.

x86: http://files.1f0.de/lavf/LAVFilters-0.52-39-gf0b33ee.zip
x64: http://files.1f0.de/lavf/LAVFilters-0.52-39-gf0b33ee-x64.zip

Important fixes are:
- A deadlock when navigating the menu on some discs, causing the player to freeze
- Menu Overlays not updating on some discs
- Wrongly positioned subs/menu overlays on some discs
- Some subtitles staying on the screen for ever (actually fixed quite a while ago, just mentioning it again)

I may create a release version after the weekend, still have to figure out what i want to finish for the next version.

I appreciate any feedback on DVD behaviour, and if you have some broken menus on some discs, a sample of that would be great.
You can extract the menu from a DVD by packing up all the ifo/bup files, and the video_ts.vob as well as any VTS_xx_0.VOB. The VTS files ending with zero are for the menu, 1 and above are content.