LAV Filters - DirectShow Media Splitter and Decoders

[VipZ]

Cool, thanks

[tormento]

I have tried with a simple search but had no results.

Can LAV Filters play AC-4?

[el Filou]

No because it's not supported yet in ffmpeg's open source decoders.

[tormento]

No because it's not supported yet in ffmpeg's open source decoders.
It seems that some support (https://trac.ffmpeg.org/ticket/8349) is there.

There are Kodi builds working with AC-4 already.

[richardpl]

That is all illegal, beware of getting sued by Dolby.

[huhn]

welcome to doom9 THE in-place of illegal DVD conversation.
we even have a decryption sub forum.

so what do i miss here is none opensource code direcly shared from dolby or something?

[tormento]

so what do i miss here is none opensource code direcly shared from dolby or something?
Dolby has a Github repository with source code for its plugins. :D

[richardpl]

THen just use their plugins an be happy user.

[Sunspark]

For LAV Splitter "Advanced" drop-down, I had to re-write the subtitle selector string because MPC-HC wasn't picking the right ones up with the old one I was using.

Sharing it here in case anyone wants it and uses English subtitles all the time as I do. The order of selection is left-to-right.

You can also set an audio language preference as well, but better to select the audio track manually depending on what you're watching.

eng@SDH eng|h eng@ass eng|n *|d

eng@SDH - All audio languages, use english subtitles for deaf&hearing impaired.
eng|h - Same as above, I don't recall seeing the hearing impaired flag in use, but including it anyway just in case, possibility it won't work in which case change to *|h if I encounter it.
eng@ass - Needed for a Japanese file that contained both ASS and PGS format subtitles in English with ASS being the preferred one.
eng|n - Needed for some files that have a default,forced english subtitle track that surprisingly contains no subs at all, this will select the English one with subs.
*|d - Needed for a Japanese file that has English subtitles, but won't select it with any of the above rules. eng|d did not work, needed to be *|d to work.

In regard to the SDH subtitle track, it's true it is sometimes unnecessary to see descriptions of sounds e.g. [papers rustling] and perhaps especially descriptions of the music, e.g. [despairing music] or whatever, it is needed because sometimes during a sequence where music is playing, this is the track that will contain the lyrics to the song that is being played (if present). The normal track does not contain song lyrics.

[cremor]

Where did you find that "lang@something" syntax? It's not documented in the readme file.

[Sunspark]

The @ character isn't documented, I saw it on a forum, but it does work. You can use it to match a word in the list of subtitle tracks. Very useful functionality.

Nevcariel: I found a performance issue. One file I have is one of those weird HEVC main 10 anime files. I have observed on my system comparing the stats with dxva2 copy-back vs software decoding, the cpu usage is the same for both, however with software decoding for this file the gpu load is identified as 8% vs 24% using dxva2 copy-back. The stats identify dxva2-cb as being active. This is a 3x difference in gpu load. If one uses software chroma/image scaling instead of dxva, then the differential is even greater. Do you know why the hardware decoder is so much worse than the software one in this example?

[lvqcl]

I suspect that the problem is in your system, not LAV.

[el Filou]

I have observed on my system comparing the stats with dxva2 copy-back vs software decoding, the cpu usage is the same for both, however with software decoding for this file the gpu load is identified as 8% vs 24% using dxva2 copy-back. If one uses software chroma/image scaling instead of dxva, then the differential is even greater.What GPU and renderer on what system?
DXVA copyback has inefficiencies: https://forum.doom9.org/showthread.php?t=176642
It's possible the time the GPU has to wait for the decoded frames to be transfered over the bus and back is counted as GPU active usage time, it depends on how the GPU driver decides to report this.
I find it very weird that decoding 10-bit H.265 would give the same CPU usage when using software and GPU decoding.
so what do i miss here is none opensource code direcly shared from dolby or something?richardpl is the author of that AC-4 code in his own ffmpeg fork that has reportedly been used in Kodi builds and other stuff and the licence is GPL. It's dated from 15 months ago but it hasn't been upstreamed into the main ffmpeg code, so maybe there's a legal problem? :confused:

[Sunspark]

I did some more testing.. there is an issue and it's partially CPU-related.

My CPU is a Broadwell, iGPU HD 6000.

"Broadwell will be implementing a hybrid H.265 decoder, allowing Broadwell to decode the next-generation video codec in hardware, but not with the same degree of power efficiency as H.264 today. In this hybrid setup Intel will be utilizing both portions of their fixed function video decoder and executing decoding steps on their shaders in order to offer complete H.265 decoding. The use of the shaders for part of the decoding process is less power efficient than doing everything in fixed function hardware but it’s better than the even less optimal CPU."

As currently implemented on Broadwell, using copy-back or software on HEVC will have the same CPU load, but GPU load is much lower with software decoding. Native does not have a lower GPU load, but does have a little lower CPU load. Using D3D11 does not change anything.

LAV work-around: uncheck HEVC in the list of codecs for HW decoding to ensure that HEVC is always software-decoded.

I tested MPC-BE's video decoder and left HEVC checked, no difference with the load issue. It's not the video renderer, because just now I was testing with mpcvr, and last night it was with madvr.

I wonder why using software decoding is better than hardware decoding? Could it be that Intel has done something where if it's software-decoded it actually does perform some invisible hardware acceleration to reduce the GPU load, but when it's using one of the hardware decoding codecs used by LAV, etc. it goes through a non-optimal path instead?

[huhn]

well you said it is a hybrid decoder which means it's as usual not that useful or plain bad.

and the GPU is higher in hardware decode here because it is not a asic hardware decoder but a hybrid decoder meaning is is using a lot of actual processing power of the GPU and CPU to do the task.
under certain situations a CPU can be plain better at this.
as you can see they have done a good job in optimising the HEVC software decode in ffmpeg.

what so ever there is no issue.

[lvqcl]

Could it be that Intel has done something where if it's software-decoded it actually does perform some invisible hardware acceleration to reduce the GPU load
LAV uses ffmpeg libraries to decode HEVC in software. I very doubt that Intel drivers hack into ffmpeg libs and alter some code in them.

I wonder why using software decoding is better than hardware decoding?
It's not hardware, it's hybrid and it suxx.

[clsid]

The reason software decoding has less GPU load is super simple to understand. It only has to copy the decoded data to the GPU once.

With copyback, the data first geta copied to GPU in compressed form, then copied back in uncompressed form, and then copied to GPU again.

Hybrid decoding of Broadwell is crap.

[Sunspark]

I just found it odd the results, I was under the impression that using DXVA2-Native or D3D11-Automatic would have better results with HEVC due to reduced traffic, but it only had a bit of CPU reduction and no reduction for GPU.

I accept that the situation for this CPU isn't going to change for this media type.

FFMPEG folks have done an impressive job.

[Msarc]

Noob question: with LAV video decoder already using dithering, is it better to disable dithering in video renderers (madVR / MPC VR)? Or do they apply to different things?

[Sunspark]

Noob question: with LAV video decoder already using dithering, is it better to disable dithering in video renderers (madVR / MPC VR)? Or do they apply to different things?

Different things. If the renderer has support for stuff like dithering, levels, etc. those are passed through and it is the renderer on the other end that sets the settings.

Those settings won't change anything with those two renderers. If you need to change dithering, levels, etc. in MadVR or MPCVR you have to do it on that side.

[Msarc]

Different things. If the renderer has support for stuff like dithering, levels, etc. those are passed through and it is the renderer on the other end that sets the settings.

Those settings won't change anything with those two renderers. If you need to change dithering, levels, etc. in MadVR or MPCVR you have to do it on that side.

Got it, thank you.

[tormento]

I have tried to decode a HEVC 4:4:4 file and, according to this (https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new), my card (1660 Super) should support it.

Anyway, trying to decode with Lav Filters, both in MPC-BE, as addin, and in MPC-HC, as integrated filter, makes them use avcodec software decoding.

The media file has this video properties:

Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Format Range@L5@High
Codec ID : V_MPEGH/ISO/HEVC
Duration : 1 h 49 min
Bit rate : 10.3 Mb/s
Width : 2 592 pixels
Height : 1 080 pixels
Display aspect ratio : 2.40:1
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Chroma subsampling : 4:4:4
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.153
Stream size : 7.84 GiB (79%)
Writing library : x265 2.9+8-27d8424c799d:[Windows][MSVC 1900][64 bit] 10bit
Encoding settings : cpuid=1111039 / frame-threads=4 / numa-pools=16 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=3 / input-res=2592x1080 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=4 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=23 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=25 / lookahead-slices=4 / scenecut=40 / radl=0 / no-intra-refresh / ctu=64 / min-cu-size=8 / rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=3 / limit-modes / me=3 / subme=3 / merange=57 / temporal-mvp / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=4 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=1.00 / no-rd-refine / no-lossless / cbqpoffs=6 / crqpoffs=6 / rc=crf / crf=14.5 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / vbv-maxrate=32000 / vbv-bufsize=15360 / vbv-init=0.9 / crf-max=0.0 / crf-min=0.0 / ipratio=1.40 / pbratio=1.30 / aq-mode=1 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=31 / qpmin=0 / no-const-vbv / sar=0 / overscan=0 / videoformat=5 / range=0 / colorprim=1 / transfer=2 / colormatrix=2 / chromaloc=0 / display-window=0 / max-cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / no-hdr / no-hdr-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-mv-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei
Default : Yes
Forced : No
Color range : Limited
Color primaries : BT.709

Am I doing something wrong?

[Aleksoid1978]

I have tried to decode a HEVC 4:4:4 file and, according to this (https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new), my card (1660 Super) should support it.

Anyway, trying to decode with Lav Filters, both in MPC-BE, as addin, and in MPC-HC, as integrated filter, makes them use avcodec software decoding.


It's support only in NVDEC, you can try it in internal MPC-BE video decoder.

[tormento]

It's support only in NVDEC, you can try it in internal MPC-BE video decoder.
Would be hard to add it in LAV?

[el Filou]

nevcairiel said a while back he had plans to update the internal NVIDIA decoder to use the newer version of their API, but no ETA obviously, and it wasn't clear if this would include support for formats that are decoded only through it and not through DXVA.
(Edit: Intel also supports formats that can be decoded only trough QuickSync and not through DXVA)

[huhn]

there is a * to it but the information for that is currently missing.
the information should show that this is for lossless only.
so it is not useful for general usage.
and it should not work with your file.

[tormento]

the information should show that this is for lossless only
Isn't that just for the encoding part only?

[huhn]

the encoding page is crystal clear.
and the encoding page doesn't use a *.

i could be utterly wrong like always.

[nevcairiel]

NVIDIA supports encoding 4:4:4 in normal as well as lossless, but decoding only 4:4:4 lossless.

I don't plan to support this in the current version of CUVID.

[tormento]

NVIDIA supports encoding 4:4:4 in normal as well as lossless, but decoding only 4:4:4 lossless.
https://i1.lensdump.com/i/tPc5Gb.png

Mmmm... that is not lossless.

[huhn]

good. the information was still present back in the days.
they may just forget to remove the *.

nice very niche upgrade.

[nevcairiel]

You can send it all sorts of bitstreams, and sometimes it might even work, but there is no guarantee that every 4:4:4 bitstream will work, hence we call it not supported.
HW decoders don't usually do a lot of checks. It relies on the software using them to do those checks.

Its not the first time this concept comes up. "It works for this one video I have" is not proof that it works for _every_ video using that encoding. Thats the tricky part about decoders, you have to support every single codec feature. An encoder has it easier - it gets to pick which features it implements.

Unless support is clearly documented, and with which hardware supports what as well, its just not worth the hassle for edge cases. Guaranteed correct decoding is more important then maybe faster decoding.

[tormento]

You can send it all sorts of bitstreams, and sometimes it might even work, but there is no guarantee that every 4:4:4 bitstream will work, hence we call it not supported.
Perhaps long time ago. Nowadays it's supported and working.

https://docs.nvidia.com/video-technologies/video-codec-sdk/nvdec-video-decoder-api-prog-guide/

[nevcairiel]

That link doesn't mention 4:4:4 HEVC at all. In fact it only lists main/main10/main12 support for HEVC in the feature matrix, which is limited to 4:2:0. So not sure what you were trying to show there.
If anything the support for HEVC 4:4:4 is very badly documented, and the only documentation I actually know of mention it special cases it to be lossless only. Hence I likely wouldn't care to offer it, even after the CUVID/NVDEC support was re-written. Badly documented niche features is how you get annoying bugs.

Besides all that, high-bandwidth copy-back decoding is also quickly bottlenecking on memory bandwidth, and finding a renderer willing to add support for native delivery of 4:4:4 formats is quite likely going to be rather challenging.

[jmone]

I for one would appreciate HEVC 10-BIT 4:4:4 hw decoding via NVDEC but I'm also happy to admit that it is a niche case that would be a nice to have rather that a must.

FWIW - I capture in AVC 4:2:2 10-Bit UHD Log (Sony FX6) and typically render out (Davinci Resolve) in HEVC UHD HDR 4:2:0 and it normally looks great. I have had some issues with poor chroma upscaling on high contrast edges with this format with some rendering combinations using nvidia HW accelerated rendering (https://yabb.jriver.com/interact/index.php?topic=120102.0).

My edge case is that I'd prefer to just render out a 4:4:4 version that I could use both for playback (but would need HW accelerated decoding) and keep as the "master" (as the 4:2:0 version is throwing away chroma detail from the original footage). Davinci Resolve supports both decoding and encoding of HW Acceleration of HEVC 4:4:4 using NVDEC, and for those interested here is a zip file (400MB) of a short sunrise clip rendered out in both UHD HDR 4:4:4 and 4:2:0. https://behome.dyndns.info/index.php/s/7BgTfkYSmYrrzsS

[Aleksoid1978]

NVDEC normal support HEVC 4:4:4 8/10/12bit.
If need - MPC VR support "needed" 4:4:4 format for "direct" copy from decoder.

[tormento]

That link doesn't mention 4:4:4 HEVC
Have yourself a "4:4:4" CTRL-F search.

[tormento]

NVDEC normal support HEVC 4:4:4 8/10/12bit.
I was planning to use HEVC lossless 4:4:4 as intermediate file for the most complex encodes, where my PC would struggle to do them in a single passage.

A fast preview player would be nice.

Anyway, with a lot of probability DGDecNV is about having 4:4:4 support. I will play the avs file.

[nevcairiel]

Have yourself a "4:4:4" CTRL-F search.

I don't think you really get it at all.

Its not about there being random mentions of 4:4:4 (without context, without further information which profiles are supported, for which codecs, etc), but the absolute lack of clear information what is actually supported. The feature matrix does not mention 4:4:4 at all, in fact it lists a bunch of profiles that are all 4:2:0. Some other feature pages list 4:4:4, some qualify that to lossless, some don't elaborate at all. Its a messy situation, and one I do not want anything to do with.

[YGPMOLE]

Hi!

I got 2 (very stupid?) questions: even when bitstreaming a multichannel track to an AVR receiver connected by HDMI, the OUT LAV Audio Decoder Pin Info shows a S/Pdif stereo connection; it's a normal behavior?

Second question: some of my Matroska files are opened by the internal MPC Matroska Source and Audio Decoder of MPC-BE instead of LAV Splitter Source and Audio Decoder (with what seems to be an audio filter for each language, instead of LAV Splitter Source + LAV Video Decoder + LAV Audio Decoder as usual): the only difference I noted is that this files are marked as AVC High Profile Level 4.1; this is the filter chain:

MPC-BE 1.6.3.0
Filters currently loaded:
- Default DirectSound Device
- madVR Renderer
- Audio Switcher
- LAV Video Decoder
- MPC Audio Decoder
- MPC Audio Decoder
- MPC Matroska Source

[clsid]

Yes, it is normal that you see such generic pin info in case of bitstreaming.

You need to set LAV Splitter as preferred source filter in MPC-BE options. Go to MPC-BE support topic if you need help, as this is not a problem with LAV Filters itself. Or switch to MPC-HC which uses LAV by default.

[Yups]

I have issues with AV1 videos encoded from Intel Arc A380, the AV1 videos stutter, even on a TGL-U device with Iris Xe hardware decoder.


Potplayer with integrated FFmpeg decoder no problem, runs fine. LAV 0.76.1 and 0.75.0 stuttering. I've tried with MPC Home Cinema and VLC player on two PCs with same result. Could someone check this out?


Sample can be downloaded from here (https://youtu.be/2xo0CIdXHFk): https://drive.google.com/file/d/1jRKkTGYVdYHN-KzVH6cM1k6JVITFzQxR/view

[nevcairiel]

Whatever encoded that file seems to have screwed up the timestamps. The timestamps move backwards and forwards as if the video had B-Frames, but AV1 does not have B-Frames.

[Yups]

Whatever encoded that file seems to have screwed up the timestamps. The timestamps move backwards and forwards as if the video had B-Frames, but AV1 does not have B-Frames.


It's from Intels Arc control panel (ingame recording). AV1 does not have b-frames? Interesting.

[YGPMOLE]

Yes, it is normal that you see such generic pin info in case of bitstreaming.

You need to set LAV Splitter as preferred source filter in MPC-BE options. Go to MPC-BE support topic if you need help, as this is not a problem with LAV Filters itself. Or switch to MPC-HC which uses LAV by default.

Thank you for the answers! Everything it's properly done in MPC-BE, and other files are correctly opened/splitted/decoded by LAV, so it just seemed strange to have DTS 5.1 input and S/PDIF 2.0 out when bitstreaming, but if you says it's OK so I'll take it for the normal behavior of the pin info.

Any idea why LAV Splitter is not able to open that file and MPC Matroska Source it's used instead? It appears a normal Matroska .H264/AVC file to me...

[tormento]

DGTools is out with a beta (https://www.rationalqm.us/board/viewtopic.php?f=8&t=1176), reading 444 flawlessly on nVidia cards. I have played those kind of files thru AVS on MPC-BE too.

I hope that it will encourage 444 support for LAV Filters.

[clsid]

There is no nightly (https://files.1f0.de/lavf/nightly/) build for latest LAV changes.

Can anyone confirm if AV1 hardware acceleration got broken after last ffmpeg update? You can use latest MPC-HC dev build to test, as that contains newest LAV build.

[lvqcl]

MPC-HC 1.9.22.34, LAVF 0.76.1.9-git - no HW acceleration for AV1.

[nevcairiel]

Should be fixed, ffmpeg didn't automatically enable a component it needs.

The nightlies should hopefully also build again, I rebuild the server recently that handles it.

[gendalv]

I thought installing this would make thumbnails show up for x265 10b .mkv files :/
In Codec Tweak Tool all things are selected for thumnails.
How to do it? Win11.