LAV Filters - DirectShow Media Splitter and Decoders

[egandt]

All that said, QuickSync wouldn't work for you either way. It doesn't support HEVC, and even if that were implemented one day, it also doesn't have the ability to export the HDR metadata required by madVR for proper HDR handling.
QuickSync should generally be considered "on life support", and DXVA2 decoding should generally be favored. If your Intel GPU is properly setup and active, it could however even be used for DXVA2 decoding, and wouldn't require QuickSync.

From what I've read the 7 Series actually includes HDR/10bit and REC.2020 support, basically the only thing Intel did for Kabi lake was add this support, it is said to be about 2.6x faster now http://www.anandtech.com/show/10610/intel-announces-7th-gen-kaby-lake-14nm-plus-six-notebook-skus-desktop-coming-in-january/3[/URL] I fully understand that it is currently a very small proportion of chips currently so it may not be worth the time to support this, but unlike previous generations it should now be possible on Kabi Lake processors.

:thanks:

ERIC

[nevcairiel]

From what I've read the 7 Series actually includes HDR/10bit and REC.2020 suppor

That doesn't mean the QuickSync decoder in LAV supports any of that.

[madshi]

Doesn't sound like something that happened to me. I have a bunch of Blu-ray sourced VC-1 MKV files (made with MakeMKV), and I don't remember ever seeing any troubles with them. Thats basically all the VC-1 content i generally watch, however.
I assume you don't use any HW codecs, and didn't disable the MS DMO checkbox in the LAV Video -> Formats settings? (or in other words active decoder reports "wmv9 mft" when playing such a file?)

How big of a jump does your OSD show? On a quick check, I sometimes see jumps from 0.000x to 0.03x, which seems like a huge difference, but couldn't really spot desync yet.
Hmmmm... After some tests it seems the crackling occurs regardless of decoder. It seems the other issues only occur when playing VC-1 + FLAC. But it's all rather weird. And my PC definitely has DPC latency issues which might be related. The MS DMO does still seem to work better for me, but it might be a happy coincidence and not the sign of a bug in LAV. For now I'd say ignore this one.

[Reino]

Hello Hendrik,

I have a feature request. Have LAV Source accept optional start en stop time, like File Source (URL).
When I open http://content.omroep.nl/avrotros/transcoding/klassiek/79_1485337306/79_704x396_1485337306.m4v?start=295&end=537 (http://content.omroep.nl/avrotros/transcoding/klassiek/79_1485337306/79_704x396_1485337306.m4v?start=295&end=537) with File Source (URL) --> LAV Splitter, the video is trimmed and starts at 04:55 and ends at 08:57.
LAV Source ignores the suffix and opens the entire video.

[nevcairiel]

I do not plan to support that, but you already have a working combination of things, so why not use that?

[NikosD]

@nevcairiel

I have just read an explanation regarding H.264 issue with Polaris cards.

What do you think ?

"The corruption with the clip only occurs on seeking.

When user perform seek, the player supposed to locate the key frame first. I’ve verified that the first frame that decoder received is indeed a key frame and the output is fine without corruptions.

However, the issue starts happens in the next frame player sent out – the reference frame. While the bitstream is correct, the picture parameter sent by the player is not correct. You can reference the following dxva2 specification provided by Microsoft:

https://msdn.microsoft.com/en-us/library/windows/desktop/cc307941(v=vs.85).aspx

The picture parameter provided for the second frame shows 4 frames in reference list:

However, there should only be one because this frame is right after the I-frame (key frame), analyzer shows this as well:

This triggers the driver code that when it detects a frame is outside the reference list, it will mark this frame as an error frame and trying to conceal this error by copying from the latest frame in reference frame buffer. This error propagates and user will see some image flipping and corruptions

Why this issue doesn’t happen with software decoder? In software decode it always keeps all reference frame in memory and will not destroy it, so it will always able to find the reference frame even though it is outside the limit."

[nevcairiel]

The entire explanation sounds a bit fishy to me.


The picture parameter provided for the second frame shows 4 frames in reference list:

However, there should only be one because this frame is right after the I-frame (key frame), analyzer shows this as well:

Just because a frame is right after an I frame doesn't mean there is no further references, those can go beyond I frames in H.264.

It is fully possible that a reference frame might be missing right after a seek. When this is the case, a empty frame is generated in place of the missing frame, so its not actually missing - just an empty dummy frame. This will lead to corruption when its used as a reference, but this happens to all decoders - software or any hardware. The information just isn't available. When this corruption happens, it fixes itself as soon as new key frames arrive, which is typically rather quickly.



Why this issue doesn’t happen with software decoder? In software decode it always keeps all reference frame in memory and will not destroy it, so it will always able to find the reference frame even though it is outside the limit.

This is not true, on a seek all reference frames are destroyed, and all reference frames that are available are also given to the hardware. Software doesn't have any "extra" frames unavailable to the hardware.
The real question should be: Why does this issue not happen with any other hardware, even older AMD cards? It sounds to me like trying to deflect the blame, instead of just fixing it.

[NikosD]

The real question should be: Why does this issue not happen with any other hardware, even older AMD cards? It sounds to me like trying to deflect the blame, instead of just fixing it.

Yes, this is what we all say.

I'll use your post to reply to them.

[Pat357]

The MS DMO does still seem to work better for me, but it might be a happy coincidence and not the sign of a bug in LAV. For now I'd say ignore this one.
What really puzzles me is that the LAV decoder actually *uses* the MS DMO if you have this option checked in LAV-Video. (is default checked IIRC)
How can the MS DMO give better results in this case ?
(= rhetorical question in case you wondered ;-)

An idea is to check LAV-Splitter under format settings : try to disable/enable "time-stamp corrections for VC1".
The fact that this option is available indicates that this is not the first time that there were or still are problems with VC1 and time-stamps.

[nevcairiel]

An idea is to check LAV-Splitter under format settings : try to disable/enable "time-stamp corrections for VC1".
The fact that this option is available indicates that this is not the first time that there were or still are problems with VC1 and time-stamps.

The option can probably be removed these days, I'm not aware of any cases where its realistically still needed to be changed.

[eddman]

Bit of a newbie question. I want to use the latest LAV with MPC-HC as external filters but without registering/installing them on windows.

I know that I can add .dll or .ax files in MPC-HC.

My question is, when adding LAV, do I have to add all the files or just .ax files?

[clsid]

Why external if you don't register them? In that case you can simply overwrite the internal ones.

The latest nightly already contains the latest LAV:
http://nightly.mpc-hc.org

[TheShadowRunner]

I have ask this too for a long time, and I get never an answer.
So yeah, seems like nev is immune to addressing requests, odd xD

[nevcairiel]

I do not plan to support such undocumented and unofficial Matroska features. And while we're at it, I certainly hope that the CELLAR working group cleans up this whole Matroska mess and removes any mention of all those unused mis-features - which is looking pretty good so far.

[eddman]

In that case you can simply overwrite the internal ones.

I didn't know that. I just checked and the internal ones are named differently, for example:

old: avcodec-lav-56.dll
new: avcodec-lav-57.dll

Do I delete the old ones and simply copy the new ones, or should I rename the new files to the old names?

Say for academical purposes I have to add another LAV as an external filter. Do I add just the .ax files or all of them?

[JoeyMonco]

I didn't know that. I just checked and the internal ones are named differently, for example:

old: avcodec-lav-56.dll
new: avcodec-lav-57.dll

Do I delete the old ones and simply copy the new ones, or should I rename the new files to the old names?

Say for academical purposes I have to add another LAV as an external filter. Do I add just the .ax files or all of them?

You delete and replace them. Incrementing that revision number usually will imply some sort of API and/or ABI compatibility break.

[TheShadowRunner]

I do not plan to support such undocumented and unofficial Matroska features. And while we're at it, I certainly hope that the CELLAR working group cleans up this whole Matroska mess and removes any mention of all those unused mis-features - which is looking pretty good so far.

Thanks for your clear reply.
Are there other matroska (official) features that allow to replicate what TrackSet/TrackSetEx do? I'm all for sticking to the official way of doing things, but for this case specifically, there is no equivalent I could find. Please enlighten me (not necessarily nev!)
What I would like to do is just link a subtitle and an audio track and have the splitter menu propose it.
Exemple: consider a matroska file with :
- 1 video track
- 1 jpn audio track
- 1 jpn commentary audio track
- 1 eng subs track
- 1 eng commentary subs track
I would like to link the jpn audio track and eng subs track and call it "Original audio edition". And then obviously link the jpn comm audio track and eng comm subs track and call it "Commentary edition".
TrackSet/TrackSetEx allow to do exactly that, nothing more, nothing less, and it works beautifully without breaking the official editions / segment linking features.
So is there an official way to replicate this, specifically?

If not, why not support it if it's the only way to do things, and it doesn't break anything else?

[hubblec4]

Hi TheShadowRunner

There is no "official way" at the moment to replicate the Haali-TRACKSETEX feature.
The only way will be, that nevcairiel implements the support of ChapterTrack/ChapterTrackNumber, but I know he will never do this.
Maybe a small "workaround" could help. But you have than 3 files. One is the mainmovie.mkv and the other two are "start/loading"-mkv's.

I don't know enough to write a new Matroska Splitter(maybe in a far future).

[TheShadowRunner]

Thanks for your input hubblec4, understood.
It's pretty strange that the matroska specs are so detailed and in depth for some areas (exemple: the tags), but such a somewhat basic feature is nowhere to be found ^^;

[hubblec4]

It's pretty strange that the matroska specs are so detailed and in depth for some areas (exemple: the tags), but such a somewhat basic feature is nowhere to be found ^^;

Yupp, thats a big problem. A good documentation is needed to realize a feature. Matroska still has a lot of catching up to do in this area.
I read every email from CELLAR, but mostly topics are about implementing new things like VP8,VP9, colormatrix, keep meta-infos and many others, but no one talks about "Matroska playing stuff".

Without LAV-Splitter we would not even have a working Chapter-Segment-Linking system (many thanks to you, nevcairiel), cause Haali works not correctly.

[NikosD]

In this recorded video, it seems that Cybelink's H.264 DXVA decoder works fine, without errors using Polaris cards.

For the same clip LAV filters DXVA H.264 decoding gives errors.

https://drive.google.com/file/d/0B2TV8ofJYDdsel9zRm83bmRPYkU/view

There are only two cases for the video above:

1) Cyberlink has managed to use H.264 DXVA in a different way than LAV filters for Polaris cards, without errors.

2) Cyberlink has managed to identify the specific clips that cause trouble to Polaris cards and use CPU as a fallback to DXVA HW acceleration, without warning to the users.
So although it says that it uses DXVA HW acceleration in general, the truth is that for those specific clips it falls back to SW decoding.

I personally believe it's the second case, so in order to be sure, someone can show us CPU usage for the specific clip using Cyberlink H.264 DXVA and LAV filters H.264 DXVA.

UPDATE:

It seems that Cyberlink's H.264 DXVA decoder works with 2% CPU usage, so there is probably an incompatibility with LAV DXVA H.264 decoder and Polaris HW decoder/driver.

[clsid]

That file does not contain H.264 video. It is H.263.

[NikosD]

How do you know ?

And there is no HW acceleration for H.263 files.

In AMD's forum there are a lot of H.264 files including mine, that can play fine with Cyberlink's H.264 DXVA decoder.

The issue could be a proprietary non standard implementation of DXVA H.264 decoder by AMD for Polaris driver/HW that causes problems with free/ open source projects like LAV filters that Cyberlink has managed to resolve.

[huhn]

it is not that clear that the video shows the problem but is not the problem it self.
and there is hardware acceleration for h263.

[nussman]

"Cyberlink's DXVA Decoder" always works "different" with special workarounds for special circumstances.
Especially for LiveTV this is very well known.

Anyway ... LiveTV H.264 is working with LAV (and MS Decoder) for years now with every other hardware. I still think its AMD to blame ...

[huhn]

i guessing doesn't fix this issue and if AMD polaris needs special treatment why isn't AMD providing a patch for FFmpeg this is the really strange part!

[nevcairiel]

There is so many things AMD could do to resolve such issues -

- Fix them in the drivers
- Send patches to open-source projects like FFmpeg to work-around their quirks
- Or at the very least, document their quirks somewhere so developers can work around them themselves.

Of course anything but the first option feels a bit silly, vendor-specific quirks for *new* hardware in 2017 is just crazy, but if they can't do 1, then any other option is better then doing nothing.

There is also the simple option of it being a subtle bug somewhere in FFmpeg or LAV and not an actual quirk, but then again AMD could follow option 2 or 3 and reach out to FFmpeg, which powers the large majority of all media players out there.

[NikosD]

I tried today MS DS and MS MFT H.264 DXVA decoders and have both the same problems as LAV during playback and seeking.

It would be a crazy coincidence if the decoders of the developers of Microsoft and FFMPEG (LAV) had exactly the same bug.

Also, I think now that it's not proprietary DXVA implementation because you couldn't use it at all.

It's probably what you call quirks, but hopefully it's not hardware (since Cybelink seems to work OK)

[clsid]

Hi nev, which toolchain are you currently using? I tried MSYS2 since the MSYS pack from XhmikosR that I used has some old libs (like openssl, which gives issues, see mpc-hc tracker). FFmpeg builds fine. But VS gives "error LNK2026: module unsafe for SAFESEH image" for x86 build. Disabling SAFESEH is of course bad idea. Do you perhaps have any tips?

[nevcairiel]

I use MSYS2 with my own compiler (https://files.1f0.de/mingw/ - which is the same compiler build XhmikosR packages in his MSYS1 zips)

I doubt that MPC-HC uses OpenSSL though, and LAV certainly doesn't (it uses the Windows TLS/SSL support, and no, I'm not interested in silly reports of XP not supporting websites which require new encryption schemes).

[clsid]

It is working properly again with your GCC build. Thanks. Should have tried that earlier. I am curious why their vanilla mingw build didn't work.

[aufkrawall]

Would "native" CUDA decoding require a backend in the video renderer?
I tried it with mpv and it seems to give me by far the lowest overhead on both GPU and CPU, compared to copyback modes (on Linux too, btw.).

[nevcairiel]

Yes, it would. If you want absolutely minimal overhead then DXVA Native is your only choice.

[MisterXDTV]

Does or will LAV Video support DXVA for VP9 Profile 2 (10 bit) videos from youtube? I can't hardware decode it with LAV decoder no matter what I select (native, copy-back, NVIDIA CUVID). With another decoder it works fine

[NikosD]

OK...

So, with the release of the new non-WHQL AMD driver 17.2.1, the decoding problem of corrupted image during playback and seeking with DXVA ffmpeg based decoders, like LAV and the H.264 video files (MBAFF, progressive, interlaced) using Polaris cards, has gone.

The image is fine like all other HW decoders from AMD, Nvidia, Intel.

The only compromise is a black screen for about 2 secs in the beginning of the file, while you can hear the sound and when you seek you get a frozen image for about 1 sec before normal playback starts again (while you can hear the sound).

[NikosD]

According to AMD and based on some Jellybean samples, there is a bug in FFMPEG DXVA H.264 decoder (and LAV of course)


Based on the clip provided:

When the user performs a seek, the player is supposed to locate the key frame first. The first frame that the decoder received is indeed a key frame and the output is displayed without corruption.

However, the issue starts happens with the next frame player sends out – the reference frame. While the bitstream is correct, the picture parameter sent by the player is not correct and not following DXVA2 specs supported by Microsoft.

The picture parameter provided for the second frame shows 4 frames in reference list. However, there should only be 1 because this frame is right after the I-frame

This triggers the driver code that when it detects a frame is outside the reference list, it marks this frame as an error frame and trying to hide the error by copying from the latest frame in reference frame buffer. This propagates the error and results in image flipping and corruption.

In software decode, the reference frame in always kept in memory, so it is always able to find the reference frame even though it is outside the limit.

[nevcairiel]

Thats the same silly explanation you posted weeks ago, and it still has holes. Software decoders don't have access to any extra reference frames, neither does any other hardware.
If references are missing, there will be corruption, there is no doubt about that, but for some reason AMD never manages to recover afterwards, which it should.

[NikosD]

Ok, but is the DXVA2 specification restricted the way it's described and ffmpeg DXVA2 respects that restriction?

[nevcairiel]

The decoder works in accordance to the H264 specification. If a reference frame is missing, a dummy frame is generated (ie. a blank one) to take its place, so its very well possible that the second frame after a seek already lists 4 references - 3 of those may just be empty dummy frames that are not available due to the seek.

That entire explanation is not logical, it talks about having too many reference frames and then talks about a missing reference frame. Thats not even the same thing.
Not to forget that the Microsoft decoder showed the same behavior, you would think Microsoft would implement their own DXVA2 specification correctly, right?

[ibius]

Does or will LAV Video support DXVA for VP9 Profile 2 (10 bit) videos from youtube? I can't hardware decode it with LAV decoder no matter what I select (native, copy-back, NVIDIA CUVID). With another decoder it works fine

Same here, is it by design? The hint says VP9 decoding is experimental, on what hardware exactly?

Win 8.1 x64, GP107 on Nvidia 376.33 drivers.

[CruNcher]

The decoder works in accordance to the H264 specification. If a reference frame is missing, a dummy frame is generated (ie. a blank one) to take its place, so its very well possible that the second frame after a seek already lists 4 references - 3 of those may just be empty dummy frames that are not available due to the seek.

That entire explanation is not logical, it talks about having too many reference frames and then talks about a missing reference frame. Thats not even the same thing.
Not to forget that the Microsoft decoder showed the same behavior, you would think Microsoft would implement their own DXVA2 specification correctly, right?

Ehmm the Decoder Core isn't theirs but it would be funny if they oversaw this issue together with Mainconcept :D

Cyberlink does it right and Mainconcept and FFmpeg wrong ?

According to AMD and based on some Jellybean samples, there is a bug in FFMPEG DXVA H.264 decoder (and LAV of course)

Are you sure you meant jellybean or those jellyfish samples ?

[nevcairiel]

Same here, is it by design? The hint says VP9 decoding is experimental, on what hardware exactly?

Win 8.1 x64, GP107 on Nvidia 376.33 drivers.

VP9 10-bit will be supported in a future version when I find the time to finish it.

[Shii]

Nvidia released new 378.66 driver with VP9/HEVC 10/12-bit decoding support.
Changelog:
New Features - Video SDK 8.0
High-bit-depth (10/12-bit) decoding (VP9/HEVC)
OpenGL input surface support for encoder
Weighted Prediction
H.264 ME-only mode enhancements

[huhn]

the SDK 8.0 isn't released yet.
and this is maybe a comeback for cuvid.
i have mixed feeling about this.

[LigH]

@ huhn:

"Don't be a chicken!" :D (insider for Germans)

[CruNcher]

New Features - Video SDK 8.0
High-bit-depth (10/12-bit) decoding (VP9/HEVC)
OpenGL input surface support for encoder
Weighted Prediction
H.264 ME-only mode enhancements

Another time :) though i wonder who uses it at all instead of CPU ME

Weighted Prediction would be another quality boost vs Intels Encoder.

wonder if they realize it over CUDA again like the VBR 2pass and Lookahead.


@Huhn
not sure what you fearing but all this means longer support possibilities for Windows bellow 10 mostly not sure why you find that so bad having more direct access than that what Microsoft supplies with DXVA.
You also have mixed feelings about Vulkan and a CUVID Vulkan Render backend ?
Nvidia invests a lot to be independent of Microsoft on the Unix/Linux side and CUVID/VDPAU is a important part of that for the whole Tegra Ecosystem not sure why you have mixed feelings about it and the possibility of backwards compatibility without being forced to 1 OS Version.
Unless you want to consume DRM Protected HQ 4K content, which will be only possible with Windows 10 in the near Future.

You don't seem to understand what Windows 10 is but it is the continuation of TCPA in full force mode in a absolute avg user enforcing way we ever saw being tasked out it's extremely critical what happens currently industry wide and we need to talk about it much more and make the future generation understand to what they currently not so silently agree upon it's their future.

What's New in Video Codec SDK 7.1:
---------------------------------
In NVIDIA Video Codec SDK release 7.1, following features have been added:

1) Enhancements to H.264 motion estimation only mode:
a) Ability to select specific motion vector partitions and intra mode enable/disable for motion estimation only mode.
b) Performance enhancement for stereo mode motion-estimation.
2) Streamlined the nomenclature of rate control modes.
3) Quality improvement for H.264 Temporal Adaptive Quantization(TAQ).

[Sarasa]

Again Nvidia as released an Hotfix X days after an WHQL, 378.72HF -_-;
- Fixed hardware encoding in Steam (In-Home Streaming).
- Resolved PhysX being forced to CPU when "Optimize for Compute Performance" setting is OFF.

[huhn]

@Huhn
not sure what you fearing but all this means longer support possibilities for Windows bellow 10 mostly not sure why you find that so bad having more direct access than that what Microsoft supplies with DXVA.
You also have mixed feelings about Vulkan and a CUVID Vulkan Render backend ?
Nvidia invests a lot to be independent of Microsoft on the Unix/Linux side and CUVID/VDPAU is a important part of that for the whole Tegra Ecosystem not sure why you have mixed feelings about it and the possibility of backwards compatibility without being forced to 1 OS Version.
Unless you want to consume DRM Protected HQ 4K content, which will be only possible with Windows 10 in the near Future.

CUVID is currently buggy and was just rotting around.
getting user in trouble for years now with no real patch.
running an openCL program and CUVID at the same time is not funny at all.
users where running in these bugs all the time.

i don't know why you think Vulkan or DX12 will add anything in term of video rendering? lower CPU overhead that is not used or what? video rendering isn't gaming.

while VDPAU is important for linux it doesn't mean CUVID is important.
VDPAU is used to connect to hardware decoder from AMD and NVIDIA. while CUVID is a nvidia only thing.

my concern is only about CUVID and that it will rotten again. they need to fix the bugs or it never stops rotting it is not like windows 10 is released yesterday.

[nevcairiel]

running an openCL program and CUVID at the same time is not funny at all.

For the record this is a CUDA problem, not CUVID as such, and it happens only on 32-bit, 64-bit is fine.

[aufkrawall]

while VDPAU is important for linux it doesn't mean CUVID is important.
VDPAU is used to connect to hardware decoder from AMD and NVIDIA. while CUVID is a nvidia only thing.

Currently, Nvidia allows HEVC 10 bit/VP9 decoding only via Cuvid on Linux.
So it's really useful there, thanks to Nvidia not supporting VDPAU well (but I'd be surpised if AMD was any better with VDPAU).
Just Intel is doing fine, with bleeding edge updates of open VAAPI.