LAV Filters - DirectShow Media Splitter and Decoders

[nevcairiel]

Latest NV CUvid with 375.xx drivers support P016 output and HEVC 12 bit decoding(not tested yet).

They would need to release an updated SDK to make that properly available. We'll see if/when they do that.

[huhn]

even a 39 mbit BD doesn't need more than 22 % CPU usages on an old I3 4130 and the i3 isn't even using it highest clock speed.

[Aleksoid1978]

They would need to release an updated SDK to make that properly available. We'll see if/when they do that.

I tested(do modify LAVVideo) - perfect working P016(decoding HEVC 10 bit) output with MadVR using CUVID.

[NikosD]

H.264 codec is going to an end.

It has been replaced by VP9 in Youtube, which means that probably 80% of Internet's video traffic uses that codec now.

Also, FullHD is the minimum nowadays for almost everything.

Bluray UHD is obviously UHD (2160p) and it uses H.265 10bit and YouTube along with other online services use 4K and even has started testing 8K (!)

Generally speaking, we shouldn't talk so much about H.264 but H.265 and VP9 and a little more of UHD than FullHD.

VP9 and HEVC are a lot more demanding than H.264

We are on late 2016 afterall.

All of these cases are handled a lot better by hardware decoders which are necessary (probably a requirement) when we talk about laptops.

[huhn]

the number of UHD screen and the number of UBD players on the market is still super low.

UHD is not important yet!

and about youtube and VP9 80% traffic just no: http://appleinsider.com/articles/16/01/20/netflix-boasts-37-share-of-internet-traffic-in-north-america-compared-with-3-for-apples-itunes

[NikosD]

You are really unlucky because Netflix is moving to 4K and HEVC 10bit of course:

http://arstechnica.com/gadgets/2016/11/netflix-4k-streaming-pc-kaby-lake-cpu-windows-10-edge-browser/

I have to repeat myself and I hate that.

The world is moving to VP9/HEVC and to larger than 1080p resolutions.

[huhn]

no they are going back to mpeg2...

UHD will be important sometime in the future.
just read your own link and they got the specs wrong too...

And then there's the browser: 4K streaming only works in Microsoft Edge, because it's the only browser that supports PlayReady DRM. Basically, streaming 4K Netflix on a PC requires a CPU that's in barely any devices right now (desktop chips aren't expected until sometime in 2017), the latest version of Windows 10, and using an unpopular browser, making it a largely useless feature.

[NikosD]

I'm not trying to convince anyone that he clearly doesn't want to.

I'm just saying that hardware decoding is a lot faster than CPU decoding and it is necessary for codecs like VP9 and HEVC that are more and more used everyday, especially for resolutions above 1080p.

Just facts.

[Aleksoid1978]

nevcairiel
Bug in YUV 12bit 444 ->RGB conversion.

yuv2rgb.cpp -> yuv2rgb_convert_pixels() finction.

// Shift to 12 bit
if (shift > 4) {
xmm1 = _mm_srli_epi16(xmm0, shift-4);
xmm3 = _mm_srli_epi16(xmm2, shift-4);
} else if (shift < 4) {
xmm1 = _mm_slli_epi16(xmm0, 4-shift);
xmm3 = _mm_slli_epi16(xmm2, 4-shift);
}

forgot about shift == 4.
fix:

else {
xmm1 = xmm0;
xmm3 = xmm2;
}


Short sample - https://yadi.sk/i/u8pggKFqzJLcR

[CruNcher]

I'm not trying to convince anyone that he clearly doesn't want to.

I'm just saying that hardware decoding is a lot faster than CPU decoding and it is necessary for codecs like VP9 and HEVC that are more and more used everyday, especially for resolutions above 1080p.

Just facts.

an I5 (2nd Generation) is still very capable of playing back HEVC 10 Bit 4K at lower complexity (WEB VOD) without any Hardware support even at 60 FPS if done right.

And it becomes really interesting to see how close H.265 and VP9 get then at this merging point of complexity :)

The only area where CPU is not enough anymore is very High IPTV requirements and Higher Complexity Broadcast and partly 360 Video, especially when you have all the DRM overhead coming into play additionally.

And VP9 is a little less complex in decoding requirements overall.

From a Content Provider view it would be rather dumb not to catchup this wider reach possibility ;)

Though obviously you more energy efficient on Nvidias or AMDs current IP even on the complete card overhead side to around 11-15W then pushing 4 cores to around 95W/65W, and try to get that stable with magic draining the batteries in no time ;)
And then there is Hybrid which can be usefull depending on the Scenario supporting and lowering the overall drain a little further :)

[NikosD]

As I have said before, my extremely fast, capable and expensive Core i7-4790 CPU can't decode YouTube 4K 60fps VP9 clips in realtime using latest Chrome x64 browser.

Not that I disagree with all of the above though.

[davidsama]

To decode YouTube 4K 60fps VP9 clips in realtime You have to use windows 10 IE edge browser as it should have hardware acceleration. I can play 2160 youtube in IE edge with no frame drops using Intel's Skylake processor.

[NikosD]

Haswell processors have no hardware acceleration of VP9.

Not even hybrid, like Broadwell and better.

[huhn]

good that this is the lavfilter thread because lavfilter is more than fast enough to decode VP9 none 10 bit in real time on an i7.
https://abload.de/img/wellwhatevera1uia.png

[NikosD]

It's obvious that you don't follow the VP9 thread and the YouTube thread.

It is very well known that off-line decoding of VP9 clips downloaded from Youtube are a lot easier to decode by media players than on-line clips using browsers.

[huhn]

this is the lavfilter thread.

and software decoding is fast enough for the usual files people will encounter.

and it is still the same clip.

frames are dropped in the browser even with VP 9 hardware decoding.

[NikosD]

I'm not talking about occasional frame dropping.

I'm talking about huge loss of frames every second.

And you don't have to remind people reading this thread, what thread we are reading.

[ocyl]

It is very well known that off-line decoding of VP9 clips downloaded from Youtube are a lot easier to decode by media players than on-line clips using browsers.
Are you suggesting that different software can deliver different performance levels when playing back the same clips on the same hardware?

[VictorLS]

I tested(do modify LAVVideo) - perfect working P016(decoding HEVC 10 bit) output with MadVR using CUVID.
Without dithering?
Short sample - https://yadi.sk/i/u8pggKFqzJLcR
Such sample was in this tread about month ago with no reaction from nevcairiel
Are you tried 12bit 4:4:4 after your changes? Shows video-x265-12bit-444.mp4 well?

[VictorLS]

NikosD
Please try to insert youtube VP9 link in MPC-BE and choose 4K while playing it. I have to mention that MPC-BE much better than any browser I tried in playing video. This experiment will be answer for ocyl's question.

[Aleksoid1978]

VictorLS
CUVID also as other HW decoder - support decoding only 4:2:0 :)

[huhn]

Are you suggesting that different software can deliver different performance levels when playing back the same clips on the same hardware?

this is a software(browser) problem and nothing else. it is not harder to decode on line streams.

i get about 3-6 frames dropped per sec with hardware decoding in browsers.
it should be the browser my internet is more than fast enough.
but again this has nothing to do with lavfilter.

[VictorLS]

CUVID also as other HW decoder - support decoding only 4:2:0 :)
I know. I asked two different qustions: Without dithering about native 10bit CUVID decoding and Are you tried 12bit 4:4:4 after your changes? Shows video-x265-12bit-444.mp4 well? about None hardware acceleration in LAV Video Decoder. Now video-x265-12bit-444.mp4 plays bad (with color problems).

[Aleksoid1978]

About P016 output - of course without dithering. About 12 bit 444 - I fix in mpc-be and color is right, LAV must fix by author.

[VictorLS]

About P016 output - of course without dithering.
That's very cool. Where and when we can see working solution for that?
About 12 bit 444 - I fix in mpc-be and color is right, LAV must fix by author.
I agree, but even such simple things nevcairiel usually doing very hard.

[nevcairiel]

nevcairiel
Bug in YUV 12bit 444 ->RGB conversion.

yuv2rgb.cpp -> yuv2rgb_convert_pixels() finction.

[snip]

Short sample - https://yadi.sk/i/u8pggKFqzJLcR

Fixed, thanks.
Generally I would recommend to not rely on the RGB conversion though, its really going to be slow on high resolutions.

[VictorLS]

Fixed
Thank you, I'm sorry I'm wrong this case. And what about possibility to disable Use HQ DXVA Processing with CUVID?

[VictorLS]

Guys, can you try with software and hardware acceleration this sample Men.in.Black.1997.2160p.WEB-DL.mkv (2,3GB) https://yadi.sk/i/I8FFU8PdzNRpt I made from 112GB movie with mkvtoolnix. Is there syncronization between audio and video in both cases?

[Aleksoid1978]

Guys, can you try with software and hardware acceleration this sample Men.in.Black.1997.2160p.WEB-DL.mkv (2,3GB) https://yadi.sk/i/I8FFU8PdzNRpt I made from 112GB movie with mkvtoolnix. Is there syncronization between audio and video in both cases?

With DXVA - perfect playback, A/V sync. In software - x86/x64 build is not enough.

[VictorLS]

Appologies, some add: in Win_x64 try with x64 software decoding, x86 IMHO hasn't even small chance with any modern processor.

[sneaker_ger]

I don't know if it's mkvtoolnix or the sample but if you extract the timecodes using mkvextract before and after demux->mux the number of TrueHD timecodes is different. This may indicate something is broken in the sample which may result in a/v desync without LAV being at fault.

[VictorLS]

sneaker_ger
But this is not answer on question: why with hardware acceleration syncronization quite well? Otherwise, whole 112GB movie with Core i7 software decoding has same desync.

[sneaker_ger]

I don't understand why you posted the sample, then. It's complex so you need either a very fast CPU for software decoding or a hardware decoder. If decoding is too slow you may experience this as a/v desync. If you are only interested in performance comparisons we have a whole thread dedicated to that topic (http://forum.doom9.org/showthread.php?t=171219) with lots of different samples, decoders and hardware.

[VictorLS]

I don't understand why you posted the sample.
Because of from http://forum.doom9.org/showthread.php?p=1787000#post1787000 until this page.

[sneaker_ger]

So you wanted to proof software decoding is not always faster than hardware decoding? No one disagrees, we don't need more proof.

[NikosD]

What a sample!

It's a HEVC 10bit sample at 3840 x 2080 resolution with a mythical 211Mbps (!) bitrate.

Using my Core i7-4790 the sample is completely unwatchable...It's pity I don't have a hardware HEVC decoder to watch that sample flawlessly ;)

@sneaker_ger
Software decoding of HEVC is always slower than HW decoding when we compare any desktop Intel CPU (including Skylake Core i7) with HW decoders of Skylake/Kabylake and GM206 Maxwell/ Pascal

[VictorLS]

What a sample!
I did my best ;)
But there is one file with more than 900Mbps but HDD is narrow neck - I plan to use virtual HDD in RAM.
It's a HEVC 10bit sample at 3840 x 2080 resolution with a mythical 211Mbps (!) bitrate.
Using my Core i7-4790 the sample is completely unwatchable...
But my cheap ~100$ GTX750v2 on GM206-150 plays that absolutely smooth even in WinXP http://forum.doom9.org/showthread.php?p=1784954#post1784954 with 47,952Hz and patched decoder http://forum.doom9.org/showthread.php?p=1786045#post1786045. One thing - frequency of GPU get to the maximum and Video Engine load some increase.
Software decoding of HEVC is always slower than HW decoding when we compare any desktop Intel CPU (including Skylake Core i7) with HW decoders of Skylake/Kabylake and GM206 Maxwell/ Pascal
You are quite right but I some agree with
The key point is that even CPUs these days are plenty fast for FullHD H264 playback without any serious CPU usage, without requiring some i7 high-end CPU, so software decoding is a valid and viable option for a lot of people. Only if you go 4k or exorbitantly high bitrates it may no longer work - but thats far from what the majority actually plays.
After buying 3-core CPU instead of 1-core after playing 4:2:2-4:4:4 files I sometimes forget switch to hardware decoding from "None" in LAV Video Decoder and h264 1080 plays smoothly by software - but when I look on 40-60% CPU load I switch to HW and CPU load decrease to 5-10%.

[nevcairiel]

Please move performance discussions to the existing performance threads, like thisone: http://forum.doom9.org/showthread.php?t=171219. It doesn't really fit in here.
There is no question that NVIDIA and Intel hardware decoders are much faster then software. The only argument was that for the available commercial content in FullHD, software decoding works just as well on the majority of systems.

[NikosD]

I only posted here because of a comment regarding HW decoders vs SW decoders.

BTW, the code of HW decoding of recent codecs like H.264/H.265/VP9 inside LAV filters is it explicitly yours ?

I mean you write it on your own and give it to FFMPEG or is it a work of a team of developers ?

[nevcairiel]

BTW, the code of HW decoding of recent codecs like H.264/H.265/VP9 inside LAV filters is it explicitly yours ?

I mean you write it on your own and give it to FFMPEG or is it a work of a team of developers ?

I contributed HEVC and VP9 DXVA2 decoding to FFmpeg, and I'm the maintainer of the DXVA2 code in FFmpeg. H264 and the older codecs are from someone else before my time, however.

It's important to note however that the hw decoders tightly integrate into the software decoders, so without the people writing those the hw implementations wouldn't have been possible.

[NikosD]

Great job with HW decoders.

[VictorLS]

Great job with HW decoders.
I agree, but some add:
1. Without HW acceleration CyberLink Video Decoder (PDVD Generic) much more fast then LAV Video Decoder, but CyberLink not so universal;
2. Without HW acceleration Elecard & Mainconcept HEVC Video Decoders have useful feature: decoding not all frames (i.e. I-frames only), but only Elecard can connect to LAV Splitter.
3. Aleksoid1978's patches very great too, i.e. LAVFilters-0.68.1-35.exe (10МБ) https://yadi.sk/d/M5KOnw9YyoQWo with "hacky workarounds" - possibility to uncheck Use HQ DXVA processing - and with unchecked Enable Adaptive HW Deinterlacing I see same deinterlacing quality (can anyone check?) but RussiaHD.ts and such other files and SAT channels are playing without artifacts and with hardware acceleration in LAV Video Decoder in Win7. I haven't to install LAV CUVID Decoder anymore - one LAV Video Decoder with NVIDIA CUVID hardware acceleration quite enough for me now in all ocсasions.

[NikosD]

Last time I checked Cyberlink's HW decoder it was perfect for my hybrid iGPU.

LAV Video is generally more optimized for Nvidia HW.

Also I remember Cyberlink supporting OpenCL decoding and it had a very smooth fallback to SW when sample is incompatible with HW decoders.

Generally speaking, it is definitely a more polished decoder than LAV Video but...it is commercial and supported by a company and a few developers who are being paid to do their job.

In case of LAV filters we have a lonely cowboy doing all the job for free and for fun.

Like we all do here actually, contributing to the community with one or the other way.

That's a big difference from Cyberlink and the other companies.

[har3inger]

I'm checking back regarding the bug with Dolby Pro logic II matrix encoding: I can reliably reproduce on two systems an audible and visible shift in audio center and misplacement of audio channels following 5.1 to stereo downmixing. Looking at how Dolby pro logic II is supposed to work, it introduces phase shifts, but shouldn't change relative channel volume. I think maybe there's an error with which channels are going where in LAV audio?

[ocyl]

but again this has nothing to do with lavfilter.
I agree. My last question was intended to be rhetorical. Sorry for taking forum space.

[CruNcher]

Last time I checked Cyberlink's HW decoder it was perfect for my hybrid iGPU.

LAV Video is generally more optimized for Nvidia HW.

Also I remember Cyberlink supporting OpenCL decoding and it had a very smooth fallback to SW when sample is incompatible with HW decoders.

Generally speaking, it is definitely a more polished decoder than LAV Video but...it is commercial and supported by a company and a few developers who are being paid to do their job.

In case of LAV filters we have a lonely cowboy doing all the job for free and for fun.

Like we all do here actually, contributing to the community with one or the other way.

That's a big difference from Cyberlink and the other companies.

Nope it can use a very big amount of decoding tricks, to enhance the user experience that is actually what PowerDVD does dynamically if necessary.

The very small advantage it has in it's Hardware Decoding modes are way way smaller then the CPU deficiencies.

In it's core optimization it doesn't really performs that well these days, the Chinese did way better over at Strongene, very impressive low level optimization threading and memory management work from scratch there ;)




I agree, but some add:
1. Without HW acceleration CyberLink Video Decoder (PDVD Generic) much more fast then LAV Video Decoder, but CyberLink not so universal;
2. Without HW acceleration Elecard & Mainconcept HEVC Video Decoders have useful feature: decoding not all frames (i.e. I-frames only), but only Elecard can connect to LAV Splitter.
3. Aleksoid1978's patches very great too, i.e. LAVFilters-0.68.1-35.exe (10МБ) https://yadi.sk/d/M5KOnw9YyoQWo with "hacky workarounds" - possibility to uncheck Use HQ DXVA processing - and with unchecked Enable Adaptive HW Deinterlacing I see same deinterlacing quality (can anyone check?) but RussiaHD.ts and such other files and SAT channels are playing without artifacts and with hardware acceleration in LAV Video Decoder in Win7. I haven't to install LAV CUVID Decoder anymore - one LAV Video Decoder with NVIDIA CUVID hardware acceleration quite enough for me now in all ocсasions.

1. It would be the difference between the 32 and 64 bit path but i would highly disagree anyways, but depends on the overall System status if you measure in very high resolution that's possible on the Windows Kernel Edge, keeping the overall thread fluctuations to a possible minimum.

You can pinpoint the tools and optimizations Cyberlinks Decoder fails on compared to ffmpeg/libav very easily if you stress the decoder enough and bring it to it's limits additionaly enabling a power save mode and you will see it very fast braking down.

2. It would be easy peasy for nevcariel to make them interoperable but decoder is heavily splitter optimized because of the frame accurate nature of the whole decoder core ;)

And yeah both never parted really that far away from each other especially in their UI Structure either and both still share the same Russian HQ location roughly 3km away from each other :)

2005

Elecard and MainConcept AG team up

2006

Elecard Ltd and MainConcept AG separate and split resources


Elecard head office is located in Akademicheskiy district of Tomsk.

3 Razvitiya ave. Tomsk, 634055, Russia


MainConcept - DivX LLC operates as a subsidiary of MainConcept GmbH. Razvitiya avenue 3. Tomsk, 634021. Russia.

[VictorLS]

CruNcher
1. Sorry, I don't understand even with translator. May be this is answer: I measured fps (min/avg/max) with DXVA Checker in equal conditions in Win7x64.
2. I didn't know. I know why Elecard and Mainconcept almost equal now )
Prog easy add Mainconcept's HEVC Video Decoder support in ProgDVB, DVBViewer has support since h265 support.
Elecard's HEVC Video Decoder always connects with LAV Splitter, but when Elecard HEVC Video Decoder connected to DVBViewer's source it can not decode h265 10bit in right way. So before I buy GTX750 I had to use Mainconcept in DVBViewer and Elecard in KMPlayer (because Mainconcept's Demuxer can not mkv) with Skip PB.

[Gravitator]

This file is played with a scattering of squares through DXVA2 (CUDA normally). > hotfuzz.vc1 (https://drive.google.com/open?id=0BzeF_1syecQwakhsX3RuZGhWcjA) (before playing it is desirable to pack the file through mkvtoolnix to .mkv).
Win10x64, Core2duo 3GHz, DDR3 8GB, GTX750v1 2GB d375.95, MPC-HC-x64 v1.7.10, madVR v0.91, LAVx64 v0.68.1.36.

[VictorLS]

Gravitator
http://forum.doom9.org/showthread.php?p=1784805#post1784805 and some posts lower.
I don't use DXVA at all and I'm happy with CUVID now despite of 10->8bit dithering I don't see on my 8bit (for one color) monitor.
PS. I guess you are using Win10?
Because even CUVID is useless to destroy scattering of squares in Win7 - only patched by Aleksoid1978 decoder can or turning off hardware acceleration.

[Gravitator]

VictorLS
My video card based on GM107 :angry:
Can theoretically simultaneously CUDA and DXVA2 push to decode?