LAV Filters - DirectShow Media Splitter and Decoders

[Siso]

A question about mixing to stereo, is it better to let lav audio decoder do the mixing or mpc-be's sound processing?

Anyone?

[lvqcl]

@nevcairiel
I wonder why git.1f0.de/gitweb?p=lavfsplitter.git is not updated, and there's no new nightlies..?

[ryrynz]

Not much to wonder about. He doesn't want to spend time on it atm. Real life + work and factor in that it's stable and working well for most use cases means dev work is gonna calm down, + It's only been four weeks
What I'm curious about is why nobody else is helping co-develop it. Lav gets used a lot, so wonder why the lack of interest ¯\_(ツ)_/¯

[nevcairiel]

@nevcairiel
I wonder why git.1f0.de/gitweb?p=lavfsplitter.git is not updated, and there's no new nightlies..?

Updated with what? It has the latest version. There are no nightlies if there are no changes.


What I'm curious about is why nobody else is helping co-develop it. Lav gets used a lot, so wonder why the lack of interest ¯\_(ツ)_/¯

DirectShow isn't exactly a field overflowing with developers, nevermind those wanting to work on open-source software for free. :)
But I also don't think that LAV is in desperate need of much work. There is always some open requests for special format support, or some new overhaul of hardware decoding stuff, but for the majority of people it works just fine as-is.

[lvqcl]

Sorry. For some reason I misread "1 Jun" as "1 Jul" in the date of the latest commit...
I have no idea why it happened. :confused:

[filler56789]

What I'm curious about is why nobody else is helping co-develop it.

By the way, I have always wondered who are the "WE" in the quote below :)
Hi folks,

We've been working on this for quite a while already, and i feel its time to release it into the wild for some more comprehensive testing.

[huhn]

just keep reading
At this point i would like to thank Ti-BEN for his contributions. Without his work on this project, it probably would have stayed a design in my mind.

[nevcairiel]

Yeah that was 9 years ago, beyond the initial prototype that we made together, I was basically on my own though.

[QBhd]

I always thought it was a "Royal" "we" :P

QB

[ryrynz]

I have a partially downloaded video that I find MPC-BE's internal filters handle better than LAVs. With the internal filters the file stops playing, detecting the playback length at 20 seconds.
LAV shows the what would be entire length of the film, once it hits the 20 second point it appears to continue looking for playable content which there likely is none. Nothing basically happens as it continues to chug through the file.
Nev, are you interested at improving this at all? Can upload a chunk if interested, not sure how you feel about broken file playback.

[Sunspark]

Just wondering something.. In LAV, on my Windows 7 system selecting D3D11 as the decoder to use states "D3D11 requires Windows 8 or newer, and is not supported on this OS."

I have DirectX 11.0 installed, but I don't have the platform update installed which would give the system a partial DirectX 11.1. Is it actually DirectX 11.1 that is required, or the OS itself that is at fault?

[lvqcl]

there's this reply from the author:
The problem on Windows 7 is not that the decoder doesn't work, because it technically does - the problem is that NV12/P010 are not supported in D3D11. You need Windows 8 for that. But without NV12/P010, the decoder is mostly useless for most use-cases.

[Sunspark]

Ah, thanks for that info. I wasn't after 10-bit colour since I don't have a display that supports it anyway, but I was curious to take a look at Error Diffusion dithering which requires Direct Compute to function. Direct Compute is only available to dx10 and dx11.

[clsid]

D3D11 decoding and D3D11 rendering are two different and independent things! The decoder in LAV should not be confused with settings in MadVR.

[Asmodian]

Even in madVR they are independant. You can use D3D9 presentation and Error Diffusion dithering. You need DX10/11 installed but you can set every option in the playback chain to DXVA2/D3D9 and still use Error Diffusion dithering.

[Sunspark]

Asmodian, I am not sure that is true, unless it requires DX 11.1 as opposed to 11.0 which is what I have. The reason I say that is because I was looking at dithering patterns and both EDs in DX9 mode looked exactly the same as ordered dithering. At least with the low bit-depth on the video clip I was testing with. This is what made me think that the support just wasn't there, and that it simply just fell back to ordered. I just now tested with a still image 16 bit gradient in a window. Set the display to 2 bit, and turned off change dither pattern every frame. As I apply the selection through the ordered, ED 1 and ED 2 options, there is no change in the dither pattern.

[huhn]

maybe it needs dx 11.1 or your driver have a problem with direct compute but it has nothing todo with the decoder lav filter has nothing todo with that too.

d3d9 can use ED it works flawless here just as a general rule madVR does pretty much everything using d3d9 even with d3d11 decode and even with d3d11 presentation only a very small part is done with d3d11 these are interops and presentation in this example.

[kolak]

I can confirm your issues, LAV Filters for me always had such problems when dealing with HD transport streams. It looks like DirectShow is simply not good at seeking within transport streams... :mad:

Just to make sure that your clip was not corrupted I ran it through TSDoctor, and it came out fine without any complaints. My usual workaround for HD transport streams is to repack them into an MKV container. I either use MKVMerge or ffmpeg (using the dmMediaConverter GUI) for this, and the resulting MKVs play with just about any player and have no problems with seeking.


Cheers
manolito


I can also confirm TS seeking issues. I also know that those TS streams are 100% correct.

[Aleksoid1978]

I can also confirm TS seeking issues. I also know that those TS streams are 100% correct.

Can you upload .ts sample with seeking porblem ?

[el Filou]

I just now tested with a still image 16 bit gradient in a window. Set the display to 2 bit, and turned off change dither pattern every frame. As I apply the selection through the ordered, ED 1 and ED 2 options, there is no change in the dither pattern.You can check by creating a file called ShowRenderSteps (no extension, and delete the file after testing) in madVR's folder, and comparing the 'FinalStep' rendering step in the OSD. On my system, ED2 takes ~3x the rendering time as Ordered (first make sure that the other steps are the same time to eliminate GPU frequency variation).

[kolak]

Can you upload .ts sample with seeking porblem ?

I need to make one, as none of my files can be made public.

[manolito]

I cut out a 100MB sample of a standard DVB-T2 broadcast from German public television, get it here:
https://www.sendspace.com/file/51jlvy
//EDIT//
SendSpace has become painfully slow, here is an alternative link:
https://www.zeta-uploader.com/2126706227

It shows corrupted frames with every seek operation using DirectShow based players (which all employ LAV Filters). It plays and seeks nicely using VLC.

Using the preview in an older 32-bit StaxRip it behaves like this with different 32-bit source filters:

1. DSS2Mod with current stable LAVVideosource:
Corrupt frames

2. FFVideoSource version 2.23.1 (latest stable version by Myrsloik):
Major disaster. Preview is totally unusable. Stepping through frames not possible.

3. LWLibavVideoSource (from lsmash-r929-170224-32)
Works without problems. Seeking does not display any corrupt frames.


Frankly I have no idea what I should make out of this. If LSMASH can do it, why can't LAVVideo and FFMS2 ?


Cheers
manolito

[nevcairiel]

As mentioned a few weeks ago as this topic came up, this has nothing to do with the file being TS (although it doesn't help), but because of the codec being HEVC. Noone yet implemented a function to properly suppress corrupt frames after seeking, which are basically always going to be present, in the ffmpeg HEVC decoder like it was done for H.264. The problem is of course that as codecs get more complex, this logic also does.

This is not something I do in LAV itself, so it needs to happen in FFmpeg. Feel free to contribute if this is a topic of importance for you. :)

As a general rule, TS as a container just does not offer any proper seeking functionality. It barely has usable timestamps, and thats it. No keyframe flags, no index, no nothing. As such, if you want "artifact free" seeking in TS, this has to rely on deep knowledge of the codec inside the TS container. And as mentioned above, noone has really bothered with HEVC for that, because its not a very popular codec in the open-source world due to the patent nonsense.

[ryrynz]

Any chance of an update to latest GIT of dav1d? Thanks.

[nevcairiel]

I currently plan to update it to 0.4.0 once thats available, which should be not that much longer.

[kolak]

As mentioned a few weeks ago as this topic came up, this has nothing to do with the file being TS (although it doesn't help), but because of the codec being HEVC. Noone yet implemented a function to properly suppress corrupt frames after seeking, which are basically always going to be present, in the ffmpeg HEVC decoder like it was done for H.264. The problem is of course that as codecs get more complex, this logic also does.

This is not something I do in LAV itself, so it needs to happen in FFmpeg. Feel free to contribute if this is a topic of importance for you. :)

As a general rule, TS as a container just does not offer any proper seeking functionality. It barely has usable timestamps, and thats it. No keyframe flags, no index, no nothing. As such, if you want "artifact free" seeking in TS, this has to rely on deep knowledge of the codec inside the TS container. And as mentioned above, noone has really bothered with HEVC for that, because its not a very popular codec in the open-source world due to the patent nonsense.

Seeking problems (long/frozen seek) happens for H264 as well. Files are coming from Manzanita muxer (which is by many treated as reference).

[nevcairiel]

TS, as a format, is not designed for seeking. Its designed for broadcast and streaming. If you want local file playback with perfect seeking, use a format that is designed for these tasks. I have said that repeatedly over the years. I will not spend excessive amount of time trying to improve TS seeking, because its fundamentally not a concept helped by the container. There are perfect formats/tools for each job, use the right ones. :)

Seeking in a format like MPEG-TS is always just guess work. You "randomly" (with some educated guesses) look around in the file for a timestamp that matches the one you are interested in, because there is no index or any other clues where your target time is. This is a balance between the amount of file IO you perform to find it, obviously I could just read from the start and stop when I find it, but that would be slow, and the accuracy. Sometimes it finds one a bit too early, or a bit too late, which then takes the playback chain a bit to "catch up", or it just missed the previous key frame (and due to container inadequacy, it also has no clue where those are), hence "slow seeking".

There is a reason Blu-ray, for example, which uses MPEG-TS carries an external seeking index in a seperate file, so it can actually do seeking using that information.

LAV uses the FFmpeg MPEG-TS demuxer, so you know where to send improvements. :)

One final note, for example the cited "LWLibavVideoSource" in an earlier post which does "perfect seeking" - you know how it does that? It reads the entire TS file and builds a complete seeking index. A video processing tool can certainly afford to be a bit slow to do this, but something focused on playback like LAV is not in a position to be that slow on file opening or first seek.

[el Filou]

Personally never encountered any seeking issue in my H.264 TS files (muxed by MediaPortal's TSWriter, but even when opening a raw multi-program TS direct from the cable provider it's fine) when I open them in MPC-HC with LAV.
I don't consider glitches when seeking in broadcast stream an issue. You're seeking, the very operation you're doing is breaking normal playback. It's much less disturbing than audio crackles and pops.

[manolito]

My main requirement for TS streams is to reencode them (editing out commercials and get rid of station logos). Finding the correct cut points in the preview does require a lot of seeking, and always getting corrupt frames is very annoying.

So LAV Filters is simply the wrong source filter in this case, I should use a filter like LWLibavVideoSource which indexes the whole stream first. OK, I got that. But then how is it that FFVideoSource which also indexes the stream gives such catastrophic results in StaxRip when trying to seek? Is it its default seek mode (I did use the defaults)? Or is this a bug in v. 2.23.1 which has been fixed in later versions?

[nevcairiel]

But then how is it that FFVideoSource which also indexes the stream gives such catastrophic results in StaxRip when trying to seek? Is it its default seek mode (I did use the defaults)? Or is this a bug in v. 2.23.1 which has been fixed in later versions?

That I can't answer. It might just be broken, if it does indexing.

If I could realistically index a file from start to finish, seeking would be perfect every time, but alas I cannot, since people want their files to open instantly, and not in 10-20 seconds. :D

[el Filou]

My main requirement for TS streams is to reencode them (editing out commercials and get rid of station logos). Finding the correct cut points in the preview does require a lot of seeking, and always getting corrupt frames is very annoying.Sorry for the off-topic but have you tried TS-Doctor? It has configurable settings for automatically choosing the previous/next I-Frame for cutting, or you can also seek to I-Frames in a preview window. It's not freeware but I find the price well worth it.

[manolito]

TSDoctor is one of my standard tools for dealing with TS files. But cutting at I-Frame borders will not do it for editing out commercials (especially not with HEVC streams where the key frames can be spead apart very widely).

Since I reencode anyways I do the cutting in AVISynth, and this is frame accurate, no matter what the source format is.

[manolito]

That I can't answer. It might just be broken, if it does indexing.

Yes, you are right, v. 2.23.1 is broken... :scared:

After replacing it with this version
https://forum.doom9.org/showthread.php?p=1876703#post1876703

(I believe it is v. 2.31.0) seeking works perfectly using the default seek mode 1.

[Asmodian]

If I could realistically index a file from start to finish, seeking would be perfect every time, but alas I cannot, since people want their files to open instantly, and not in 10-20 seconds. :D

A cool feature would be an option to index or use a separate index file, for those who want better seeking in ts or other bad files. I do encounter broken files which are unable to seek more often than I would like (incomplete mp4 is my most common, due to canceling encodes). The way FFMS2 works has always struck me as very convenient and functional. A way to tell LAV that I wouldn't mind indexing the file, or even automatic loading of a previously created index file I have to generate manually would be nice. At least it doesn't seem crazy. :)

I do have to admit that only a small percentage of the user base would benefit, so it may not be worth the effort, but it would still be cool for me. :o

[huhn]

even very advanced user get confused by the decoder names and what that means just imagine what will happen with that setting when user start wondering why it sometimes takes so long to start playback.

[Asmodian]

Yeah, it would probably generate a lot of extra questions. :o

This is a reason simply detecting an index you have to create manually would be good.. but probably not too important anyway. I can always use Vapoursynth. :p

[clsid]

Good time for a FFmpeg update, now that 4.2 has been branched? ;)

[videoh]

The player should start decoding and seeking without an index and kick off indexing in the background. When the index is ready, the player begins using it.

[el Filou]

Wouldn't that bring a lot of I/O trashing on anything other than an SSD?
If such a feature is ever added, it should be as an option and disabled by default.

[nevcairiel]

Indeed, that would not be very well received by the majority of users. I don't currently have any such plans. LAV is designed primarily as a tool for playback, in which cases perfect seeking accuracy for example is not a strict requirement, as long as it comes close. I realize since its practically the only option for many codecs in DirectShow, some users like to use it for other tasks as well, and while I don't want to discourage that, I also don't plan to work on complex features for such tasks.

[LigH]

I am quite satisfied with the performance of LAV Filters (e.g. as used in MPC-HC), with one exception: playing WMV may have severe issues catching up after seeking; but that may be related to attributes of specific files. I would blame LAV Filters last here.

[filler56789]

I am quite satisfied with the performance of LAV Filters (e.g. as used in MPC-HC), with one exception: playing WMV may have severe issues catching up after seeking; but that may be related to attributes of specific files. I would blame LAV Filters last here.

Indeed, the ASF container is highly pesky and "sucky" :mad: :D
I often remux the WMV files to AVIs — direct stream-copy for the video and re-encode for the audio.

[bluesky507]

Hello
I use lav splitter for play H265 TS file and it works well.
http://87.98.232.3:5050/browser/1.jpg
now i want use lav splitter for play TS from USB DVB-T dongle.
but when i want connect BDA filter to lavsplitter filter i got following error
http://87.98.232.3:5050/browser/2.jpg
How i can solve it?

[nevcairiel]

LAV Splitter is not designed as a broadcast demuxer. You'll need to use a filter designed for that purpose, I reckon.

[bluesky507]

do you any filter can do this? my stream is H265 coded video.

[kolak]

Can we find any use for this:

https://www.anandtech.com/show/14682/nvidia-siggraph-2019-nv-to-enable-30bit-opengl-support-on-geforce-cards

[nevcairiel]

Windows video stuff really doesn't use OpenGL if it can avoid it. Direct3D has supported full 10-bit on consumer cards for a long time already.

[littleD]

Can we find any use for this:

https://www.anandtech.com/show/14682/nvidia-siggraph-2019-nv-to-enable-30bit-opengl-support-on-geforce-cardsThis is for applications, not for video output- which is separated thing from system application workflow. However, i am impressed with that move. 30 bit was only enabled in drivers for quadro/professional cards for so long time that everybody forgotten about it. One of possible reason for this move is that nowadays proffesional cards occupy high performance market by most and enabling 30 bit for graphic designers will not influence nvidia income (Quadro cards are much much expensive than GeForce's consumer cards). Similar situation was wat AMD team.

[el Filou]

Their official stance is that now that there's consumer HDR, 10-bit isn't a pro-only feature anymore (which was true before, tbf). It's just like the simple/double precision float performance, we all know the consumer hardware could do it at 1/2x but they segment their portfolio that way.

[kolak]

This is for applications, not for video output- which is separated thing from system application workflow..

Not sure what you mean by this.
OpenGL is/was used in eg. Photoshop (and some other tools) for previewing at 10bit. Some pro apps use OpenGL surface to display video as apparently this allows for better control and separation from OS influence. For example Assimilate Scratch uses OpenGL. It allows for color accurate video preview and screen calibration. My understanding is that Direct3D is more tight to OS and harder to control. OpenGL is also cross platform and apparently more efficient. I may be very wrong. Of course old days only pro card had 10bit support for OpenGL surface, but now it's changing.

https://en.wikipedia.org/wiki/Comparison_of_OpenGL_and_Direct3D comparison tab explains it quite well (at least for my limited understanding).