LAV Filters - DirectShow Media Splitter and Decoders

[nevcairiel]

The question about the performance drop still remains though :D

I figured out why its slower, finally.
Apparently there was a long-standing bug in YADIF which caused it to overread the image buffer on the edges of the image, and this was fixed recently - however the fix works in such a way that the edges of the image can no longer use the SSE2/SSSE3 optimized functions and are done in pure C, which brings with it a small slowdown (from your and my measurements, around 5-10%), which explains why the latest test build is slower now.

0.56 was still a bit faster because the aggressive multi-threading which caused the stuttering was mitigating the slowdown a bit.
I tried reverting the fixes to YADIF and it restored it to the same speed as 0.55, however that was only for confirming nothing else is slower, and i'm not going to keep a known bug that overreads the image buffer in the code for a few % of speed. ;)

Maybe i can try to multi-thread YADIF itself (before it itself was still single-threaded, just running on an async worker thread), and give it a speed benefit from this, while staying in sync with the main processing thread to avoid the stuttering issue.

[Mercury_22]

Does this version help? I've not setup my ATI card yet, but maybe this already does it.
http://files.1f0.de/lavf/LAVFilters-0.56-9-g4cd6666.zip
No change tested on 00000.mts (http://aleksoid.voserver.net/Sample/H264/Interlace/) sample

[nevcairiel]

No change tested on 00000.mts (http://aleksoid.voserver.net/Sample/H264/Interlace/) sample

You did try my build, and not Git right? Because i didnt push that yet. :p
Maybe i need to install my ATI card somewhere then.

[Mercury_22]

You did try my build, and not Git right? Because i didnt push that yet. :p
Maybe i need to install my ATI card somewhere then.

Both :p

[Snowknight26]

No change tested on 00000.mts (http://aleksoid.voserver.net/Sample/H264/Interlace/) sample

Confirmed.

[DragonQ]

Apparently there was a long-standing bug in YADIF which caused it to overread the image buffer on the edges of the image, and this was fixed recently - however the fix works in such a way that the edges of the image can no longer use the SSE2/SSSE3 optimized functions and are done in pure C, which brings with it a small slowdown...

Were there any negative consequences of this bug? Cos if it was simply faster...

[nevcairiel]

Were there any negative consequences of this bug? Cos if it was simply faster...

Doesn't the word "bug" already imply that? :p
Feel free to lookup what consequences a buffer overread/out-of-bounds read can have.

[e-t172]

As someone once said, "I can make your program infinitely faster if it doesn't have to be correct".

[DragonQ]

Doesn't the word "bug" already imply that? :p
Feel free to lookup what consequences a buffer overread/out-of-bounds read can have.
Yeah but I don't remember hordes of people complaining that it was broken so I'm confused as to why something with such a huge bug worked at all.

[noee]

edge case? C'mon man, you gotta fix bugs, especially buffer overruns.

[Mangix]

my card ati3650 and mobillty ati 3450 lav dxva mode...both same..
are some of the green screen videos(wmv,vc1)
daum player dvxa mode+madvr no problem..works well....mpc,lav problem...


Off-topic: Nice refresh rate </sarcasm>

On-topic: DXVA mode is broken if you are using patched ATi drivers. Also, to really fix this you need to downgrade your drivers. The latest ones for that chip are terrible.

[madshi]

Feel free to lookup what consequences a buffer overread/out-of-bounds read can have.
Do you allocate the YADIF source buffers yourself? If so, you can make sure the buffer is 32byte bigger than necessary. That way you could use SSE2/3 on the whole buffer without worrying about buffer overruns...

[nevcairiel]

Do you allocate the YADIF source buffers yourself? If so, you can make sure the buffer is 32byte bigger than necessary. That way you could use SSE2/3 on the whole buffer without worrying about buffer overruns...

I do not, i feed it whatever comes from the decoder, anything else would require additional memory copies which would probably negate any advantages.

I'm not sure why everyone is so obsessed about this, YADIF is still plenty fast, and in real-world framerates you'll most likely not even notice any difference, only if you benchmark.

correct > fast, especially if fast includes invalid memory reads.

[madshi]

I do not, i feed it whatever comes from the decoder, anything else would require additional memory copies which would probably negate any advantages.

I'm not sure why everyone is so obsessed about this, YADIF is still plenty fast, and in real-world framerates you'll most likely not even notice any difference, only if you benchmark.
Not obsessed about it. But if you can win a few percent without any disadvantage, why not? E.g. if the decoder is LAV, you could allocate 32 bytes "bigger" in LAV Video Decoder and then do YADIF with full size SSE2/3. But probably it's not worth the extra effort.

[nevcairiel]

if the decoder is LAV, you could allocate 32 bytes "bigger" in LAV Video Decoder and then do YADIF with full size SSE2/3. But probably it's not worth the extra effort.

The problem is that the decoder is not LAV, but ffmpeg, and patching it to ensure there is padding before and after the image sounds like messy business, not to mention the future merge conflicts when updating ffmpeg, which would show up in yadif itself as well as all the other parts i have to fix in ffmpeg for this to work.

[ryrynz]

Could always run it by the ffmpeg guys and see what comes of it.

[madshi]

Ok, just forget it IMHO. Just thought if there were a very easy solution then it might be worth it. But it's not worth any headaches, I would say...

[nevcairiel]

Ok, just forget it IMHO. Just thought if there were a very easy solution then it might be worth it. But it's not worth any headaches, I would say...

I agree, too many headaches already.
Like i said, i rather invest time trying to improve yadif in other ways, like multi-threading the filtering itself, instead of trying to hack 5% speed back in there.

[kerimcem]

Off-topic: Nice refresh rate </sarcasm>

On-topic: DXVA mode is broken if you are using patched ATi drivers. Also, to really fix this you need to downgrade your drivers. The latest ones for that chip are terrible.

potplayer (internal codec)+madvr(1.5.2605) working properly works well h264 vc1 wmv(dxva) no problem,problem lav+mpc...

[kerimcem]

http://l1304.hizliresim.com/18/f/m143x.jpghttp://j1304.hizliresim.com/18/f/m1449.jpg

same video,potplayer no problem....

[nevcairiel]

I already told you that i have no such hardware, so unless you want to send me the hardware required, there is nothing i can do.
As an alternative, you're also free to fix it and send me a patch.

[STaRGaZeR]

snip

Ah, I see what you mean now in the code. Well, the difference isn't that big but this will result in bigger power consumption and heat, even if it's marginal. Since this is visually indistinguishable I'd say the ball is on ffmpeg, they could allocate a sighly bigger buffer and keep using SIMD in the whole line. But somebody try convince them on that, I certainly won't, don't want to waste my time :D

[nevcairiel]

LAV Filters 0.56.1

LAV Splitter
- Fixed various crashes with ordered chapter MKVs
- Fixed a crash when playing an MKV with a disabled subtitle track
- Fixed switching between video streams with different codecs (ie. H264 -> MPEG2)

LAV Video
- Fixed a crash on close with DXVA2 Native
- Re-tuned YADIF for smooth playback instead of benchmarking


Download: Installer (both x86/x64) (http://files.1f0.de/lavf/LAVFilters-0.56.1.exe) -- Zips: 32-bit (http://files.1f0.de/lavf/LAVFilters-0.56.1.zip) & 64-bit (http://files.1f0.de/lavf/LAVFilters-0.56.1-x64.zip)

Just a few fixes, mostly related to ordered chapters or the ffmpeg updates.
In case you missed 0.56, release notes are here (http://forum.doom9.org/showthread.php?p=1623515#post1623515).

YADIF Performance
It was noted that the performance of YADIF is slightly slower now than it was in 0.55.3. This is not a bug.
YADIF had a long-standing bug that caused it to overread its memory buffers, which in the worst case could result in a crash, and even if not, illegal memory reads are no way to write good code. This bug was fixed, however it requires that the edges of the image are handled separately, which costs a bit of performance, making YADIF maybe ~5-10% slower in benchmarks (real-world results will differ), depening on the overall system and decoding speed.

This slowdown was masked in 0.56.0 by too aggressive multi-threading, which caused stuttering/jittering playback, which was now tuned for smooth playback, at the cost of a little bit of performance (and the gain of simpler code!)

If YADIF performance is critical, its very well possible to invest some time to optimize it, (safely) multi-thread it, without resorting to illegal memory reads for the sake of performance.

In any case, please report any issues you find.

Have fun!

[wanezhiling]

:thanks:

First!:D

[Mercury_22]

It's there any chance that now you have time to take a look at this one (http://code.google.com/p/lavfilters/issues/detail?id=338) ?

[wanezhiling]

same video,potplayer no problem....

You should post a download link of that clip at least.

[kerimcem]

You should post a download link of that clip at least.
wmv in most,some mp4 pillarboxing ,mkv no problem...
sample(green screen) potplayer no problem..
https://hotfile.com/dl/202862411/bb3f6fd/04_thefinalcountdown_Reintroandchorus.wmv.html

[wanezhiling]

https://hotfile.com/dl/202862411/bb3f6fd/04_thefinalcountdown_Reintroandchorus.wmv.html

I tested the clip on my old laptop machine:
Win7 x64
ATI M HD4650 with 13.1 legacy driver
PotPlayer 1.5.36746
LAVF 0.56.1 (dxva native of course)
madVR v0.86.1

No any green image occurred during playback, see the pic (http://i.imgur.com/WltAvYL.png).

[kerimcem]

I tested the clip on my old laptop machine:
Win7 x64
ATI M HD4650 with 13.1 legacy driver
PotPlayer 1.5.36746
LAVF 0.56.1 (dxva native of course)
madVR v0.86.1

No any green image occurred during playback, see the pic (http://i.imgur.com/WltAvYL.png).
http://j1304.hizliresim.com/18/f/m1cwk.jpg :D
my old laptop same hd 3400 3xxxhd problem(13.1 driver) and desktop 3650 agp same..
potplayer dxva(internal codec) mode no problem...

[wanezhiling]

No any greenscreen here

lav dxva, mpc-hc/be dxva, ffdshow dxva, potplayer dxva... all these dxva decoders work fine.

[kerimcem]

No any greenscreen here

lav dxva, mpc-hc/be dxva, ffdshow dxva, potplayer dxva... all these dxva decoders work fine.
hd3xxx dxva problem....mpc+lav desktop,laptop same..
potplayer dvxa mode no problem....
http://l1304.hizliresim.com/18/f/m1ffp.jpg
and pot+lav(dxva mode) not work....working lav cpu decode...

[nevcairiel]

I already told you, i cannot fix it unless i get the hardware to reproduce it, and its unlikely you can actually still buy them reliably anymore because they are just so very old, and i'm not in the mood to invest money or much effort for hardware 4 generations old.

You do not need to keep repeating that it doesn't work and posting new screenshots, its not going to help.

[kerimcem]

I already told you, i cannot fix it unless i get the hardware to reproduce it, and its unlikely you can actually still buy them reliably anymore because they are just so very old, and i'm not in the mood to invest money or much effort for hardware 4 generations old.

You do not need to keep repeating that it doesn't work and posting new screenshots, its not going to help.
ok..thanks..
PotPlayer(dxva) loves the old hardware:D PotPlayer use my..:p

[mastrboy]

nevcairiel:
Any good reason that LAV filters is way slower than haali when there many files in a ordered chapters series?

I have a folder with 134 video files, avi and mkv, where only 25 of them use ordered chapters. (The MKV files)
Haali loads the file in 1-2 seconds, while LAV used almost 30 seconds... (28s to be exact)

(The files resides on a SMB/Windows share)

[nevcairiel]

Its simply not optimized for such a use-case yet. It doesn't look at the AVI files, but it would open every MKV file in the folder and parse its header information.
It can probably be optimized a bit so that it only reads the header infos it really needs to find (the segment UID) to determine if it needs to parse the whole header, not sure if that would give you such drastic improvements, i probably need to setup some test folders to see this.

[NikosD]

Hi nevcairiel

There is issue in LAV Video(DXVA native) + LAV Splitter + ATI on H.264 interlaced material. After seek - image corrupted, crumbles & twitches. Microsoft or Cyberlink decoder do not have this issue. MPC DXVA Decoder also have this bug.
If instead LAV Splitter select internal MPC MPEG splitter - LAV Video decode ok after seek.

Recreate decoder avoids all these problems after seeking.

There is some short sample - http://aleksoid.voserver.net/Sample/H264/Interlace/



wmv in most,some mp4 pillarboxing ,mkv no problem...
sample(green screen) potplayer no problem..
https://hotfile.com/dl/202862411/bb3f6fd/04_thefinalcountdown_Reintroandchorus.wmv.html

Trying all of the above clips (6) on my signature system, with both LAV filters 0.56.1 and PotPlayer, didn't find any problems besides a little video decoding problem at the beginning of DVBsub.ts clip. (probably file problem)

PotPlayer handles that clip a little better than LAV Video.

I'll test tomorrow my 3650 AGP card.

[mastrboy]

Ok, if you need any help testing new builds related to this "issue" just let me now, I have multiple series which has over 100 files with ordered chapters which I can test with.

I did a quick compare on the network usage between Haali and LAV, and LAV is reading a lot more data apparently: http://s12.postimg.org/sgpolgnql/orderedchap.png

[nevcairiel]

It'll be a bit, but i'll see what i can do.

[cyberbeing]

The difference mastrboy reported seems quite extreme.

In a folder with 30 mkv files:

Haali: 96 File Operations
LAV: 1396 (!) File Operations

Haali apparently only performs a single 32KB read from offset 0 on each file.

File I/O Logs (http://www.mediafire.com/?m12f0euc27eh2ln)

[nevcairiel]

Like i said, LAV currently parses the full MKV headers of all files in the folder, and i do plan to improve it by doing something similar to Haali, only read as much as required to find the segment UID, and then decide if it needs parsing.
But like i also said, it'll be a bit, i try to take small breaks after finishing a bigger version, helps staying focused in the long run.

The way i go about development is at first go with the "naive" algorithm and get the feature working, and then think about how to do it more efficient.

PS:
What do you use for those logs?
May be helpful to check my efforts.

[cyberbeing]

I already understood that from your previous post. I was only trying to be helpful, not upset you by repeating the issue.

The logs are from Sysinternals Process Monitor (http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx).

[nevcairiel]

Thanks, i'll take a look soon, should be relatively easy to fix, parsing the first headers of an MKV is easy.

[mandarinka]

Like i said, LAV currently parses the full MKV headers of all files in the folder, and i do plan to improve it by doing something similar to Haali, only read as much as required to find the segment UID, and then decide if it needs parsing.
But like i also said, it'll be a bit, i try to take small breaks after finishing a bigger version, helps staying focused in the long run.

The way i go about development is at first go with the "naive" algorithm and get the feature working, and then think about how to do it more efficient.

Could it perhaps help if the check was first performed on files whose filenames contain some string that matches string in the parent file's filename?

Although such trick wouldn't do any good if the whole series was in a single subfolder with nothing else - it would only be able to help if there were other unrelated mkv files in the folder.

/I know I am suggesting hacky features, but perhaps also checking for strings like "OP", "ED", "opening", "ending" could in many cases save some time./

[nevcairiel]

Those attempts are way too fragile, and would only work for a very limited set of files, so there always needs to be a fallback to full scanning.
If the scanning is properly optimized, it should be fast enough, or was this ever a real problem with Haali?

[mandarinka]

There can be a delay, when there is a lot of files in the folder, but not painfully big one. IIRC...

I just meant it as a shortcut to try before the full search. I think it would catch a lot of cases though (esp. the search for "opening" etc).

[NikosD]

Hi nevcairiel

There is issue in LAV Video(DXVA native) + LAV Splitter + ATI on H.264 interlaced material. After seek - image corrupted, crumbles & twitches. Microsoft or Cyberlink decoder do not have this issue. MPC DXVA Decoder also have this bug.
If instead LAV Splitter select internal MPC MPEG splitter - LAV Video decode ok after seek.

There is some short sample - http://aleksoid.voserver.net/Sample/H264/Interlace/


Using an older system with Win 7 x64 SP1 - Radeon 3650 AGP - Catalyst 13.1 official with automatic de-interlacing on (Catalyst control panel) I got these results:

All 5 H.264 Interlaced clips play fine with Microsoft's decoder, MPC decoder, PotPlayer.

4 clips have problem in de-interlacing with LAV Video 0.56.1, but no green screen.
Only DVBsub.ts plays fine with LAV Video.

wmv in most,some mp4 pillarboxing ,mkv no problem...
sample(green screen) potplayer no problem..
https://hotfile.com/dl/202862411/bb3f6fd/04_thefinalcountdown_Reintroandchorus.wmv.html

Here we have Green screen with both LAV Video and MPC decoder.
Microsoft's decoder and PotPlayer play them perfect - as always.

Also just FYI, all of my VC-1 interlaced clips have problems(not green screen) with both LAV Video and MPC decoder.

PotPlayer plays VC-1 interlaced on 3650 just perfect.

PotPlayer is definitely one-way for HD2000/3000 Radeon series.

Maybe nevcariel should place a warning at first page that UVD/UVD+ is not supported for interlaced content.
LAV needs UVD2 or higher

[nevcairiel]

How would you test the Microsoft decoder for VC-1? AFAIK it only does DXVA when used in WMP, so its hard to do real comparisons.
Good would be testing a decoder which can actually be used everywhere, like a commercial one from PowerDVD or so, to check if it works, and can be analysed what it does differently.

[sneaker_ger]

Those attempts are way too fragile, and would only work for a very limited set of files, so there always needs to be a fallback to full scanning.
If the scanning is properly optimized, it should be fast enough, or was this ever a real problem with Haali?

In my older post I gave some suggestions, i.e. do a search prioritization of some sort and stop reading more files when all external segments have been found. Or is this not possible? But since the basic idea is simple, you probably have already thought about that, I guess?

So basically it acts like current Haali versions, I see. I was thinking that maybe there was a better way, like at least stop searching once all linked Segments have been found or do some prioritization for the search with file names(similar name) , writing date (closest to current file) or file size (for example in relation to chapter length). But the current way is good enough me. For the "simple" linking I think the Matroska specs even offers file name tags ("PrevFilename", "NextFilename").

[Blight]

nev:
How does ordered chapters work?

I tried two clips from here:
http://www.ps-auxw.de/ordered/

However, it doesn't seem to be working well.
It seems to detect the ordered chapters and even merges multiple files for playback, but it shows the entire content, even section marked as "you should not see this".

I also tried the "mkv_ordered_chapters_example" (mkv/xml) sample.
It seems to work better, but selecting an edition doesn't change the media duration...
How is the player supposed to know the media-duration changed?
How about supporting the EC_LENGTH_CHANGED event?

Is there a specific interface for getting ordered chapters vs standard chapters?
Or do they share the same interface?

P.S.
I've used 0.56.1 to test.

[NikosD]

I would test it using DXVAChecker as player.
Best and unique software of its kind.

No interest testing commercial players/ decoders when we have PotPlayer.