ffdshow tryouts project: Discussion & Development

[Reimar]

Try to test with 24 or 32 bits only


libavcodec won't do 24 bit since that is a pain to work with - alignment issues, can't easily use SIMD instructions on it etc.
Ideally the decoders should set AVCodecContext.bits_per_raw_sample to indicate how many bits actually are relevant.
The AC3 decoder in libavcodec should be "trivial" to change to output either float or 32 bit integer (basically just replace the float_to_int16_interleave at the end of ac3_decode_frame). I'm not sure there's actually a point in doing so though.

[jruggle]

libavcodec won't do 24 bit since that is a pain to work with - alignment issues, can't easily use SIMD instructions on it etc.
Ideally the decoders should set AVCodecContext.bits_per_raw_sample to indicate how many bits actually are relevant.
The AC3 decoder in libavcodec should be "trivial" to change to output either float or 32 bit integer (basically just replace the float_to_int16_interleave at the end of ac3_decode_frame). I'm not sure there's actually a point in doing so though.
It is very trivial, and I have a working patch for it, but it is slower at decoding because the current float-to-int16 conversion combined with channel interleaving is faster than just plain float interleaving.

[albain]

It is very trivial, and I have a working patch for it, but it is slower at decoding because the current float-to-int16 conversion combined with channel interleaving is faster than just plain float interleaving.

Maybe but sound decoding and especially AC3 is not complex to decode for a CPU, even a non recent.

I think that you should propose your patch to ffmpeg team

In that way we will be able to drop liba52.

Same thing for DCA (DTS)

[CruNcher]

Thx tal.aloni for fixing the crashing problem with libavcodec.dll in ffdshow DXVA on close :) (that bug is history now)

[albain]

I have bad news : I have spent a few hours on multithreading the copy of DXVA buffers into system memory to speed up full postprocessing, with no luck

Things are going a little faster, but that's all.

Either we are doing something wrong (but I begin to doubt it), or else the sense GPU=>CPU gives by designed slow transfers

I hope that one will be able to get in touch with the intel's guy who wrote this article (but I guess that he only tried with low res videos)

Otherwise there is the DXVA HD feature that lets do non linear resize but this is too much work (at least for now)

[dann23]

I have bad news : I have spent a few hours on multithreading the copy of DXVA buffers into system memory to speed up full postprocessing, with no luck

Things are going a little faster, but that's all.

Either we are doing something wrong (but I begin to doubt it), or else the sense GPU=>CPU gives by designed slow transfers

I hope that one will be able to get in touch with the intel's guy who wrote this article (but I guess that he only tried with low res videos)

Otherwise there is the DXVA HD feature that lets do non linear resize but this is too much work (at least for now)


I have some questions. Maybe I'm missing something. You need to copy dxva buffers in memory to enable some filters and this is only for dxva 1. As I know with dxva 2 this is not needed. So are trying to do this because you want to implement dxva 1 in ffdshow? If that's the case then why not implement just subtitles for dxva 1 and use dxva 2 for the other case. And I know that windows xp has just dxva 1 but people are starting to use windows vista/7 so my oppinion is that it's enough to support dxva 1 with subtitles for windows xp.

[tal.aloni]

dann23,
AFAIK, copying the frame buffer from the USWC ("GPU memory") is a prerequisite for custom (not part of the API) post-processing in both DXVA 1.0 and 2.0.

[BatKnight]

About libavcodec TrueHD, I think that it is able to output 32 bits.

About FFDShow output logic, I am not very familiar with it but I know that for example if you let 16 bits check, you will get 16 bits only..
Yes, TrueHD does output 32bits.
The thing is, that on lossless codecs is always better to output the same as a the source, for example it is better to output TrueHD 24bits if the source is also 24bits. But if the 32bits is ticked, it now outputs 32bit. It should somehow recognize TrueHD and don't output more than the source format. This should be the case for all other lossless codecs.
One may think it's better to decode a lossless codec in a higher precision, but it's not. It could sometimes even be worse.

On the other hand, lossy codecs like DTS, AC3, MP3, etc should always be decoded in the higher precision possible (32bit float when possible) no matter what the source is. leeperry could help me elaborate here if needed...

When one ticks 16bit, 24bit, 32bit and 32bit float, the goal isn't to always output the highest possible, but to identify the source codec and output the best format for that codec.

My question is, what can be done to ffdshow that could identify the type of codec and then choose the appropriate format considering what I just explained. The goal of ffdshow is to achieve the best decoding quality possible, isn't it?

Bat

PS: Another thing I've noticed is that checking LFE Crossover changes from 32bit integer to 32bit float, on a TrueHD track, for example. Why?

[Jeremy Duncan]

I have bad news : I have spent a few hours on multithreading the copy of DXVA buffers into system memory to speed up full postprocessing, with no luck

Things are going a little faster, but that's all.

Either we are doing something wrong (but I begin to doubt it), or else the sense GPU=>CPU gives by designed slow transfers


I have a opinion on this problem.
- The cpu reads the stuff from the ram in bits.
- The gpu has it's own hw that acts as a cpu and the gpu has it's own ram on the videocard. But the way the gpu reads the ram is also in bits.

With these two facts agreed on, then integrating them, one for postprocessing the other for dxva, you would need to harmonize the way the read the ram and interact with each other.
I think the problem is they don't mesh together and they are trying to make it primarily cpu or gpu.
To fix this problem let the cpu and gpu sense each other so they don't try and make it either cpu or gpu only.

[Jeremy Duncan]

Thx tal.aloni for fixing the crashing problem with libavcodec.dll in ffdshow DXVA on close :) (that bug is history now)

Do you have any other bug?

[CruNcher]

sowt (quicktime audio) .mov decoding problems and artifacting with ffdshow DXVA and a 60 FPS stream on Nvidia VP2 booth seems to be known by now

[Jeremy Duncan]

artifacting with ffdshow DXVA and a 60 FPS stream on Nvidia VP2 booth seems to be known by now

It would be helpful to the devs here if you provided a sample clip where the problem is.
use dgsplit: link (http://neuron2.net/mine.html), a 100MB sample, and upload it to mediafire and provide the link here. The free uploader has problems sometimes, to fix this create a account and upload the file to your account.

Also if you can say which revision the problem appeared too, that would help. :)

[hydra3333]

Hi, I can't readily find a link answering this ...

Can I install ffdshow 32bit successfully on Windows Vista 64bit (and after an upgrade, Windows7 64 bit) so that MediaPlayerClassic (32bit I guess) uses it ? I'd prefer to use MPC as my standard player for everything. Is there anything special I have to do to get it to install/work ?

Any other advice (eg I'd heard there was something like a 64bit ffdshow and a 64bit MPC) ?

[tal.aloni]

Can I install ffdshow 32bit successfully on Windows Vista 64bit so that MediaPlayerClassic 32bit uses it?
yes

I'd heard there was something like a 64bit ffdshow and a 64bit MPC) ?
that's also true.

just remember, if your player is 64 bit, you'll need all your decoders to be 64 bit as well.

[hydra3333]

Great !

I've been reading in some threads how Micro$oft locks down the Media Foundation codecs in Windows7 so that effectively only their own run, or something like that ... so I'd prefer 32bit MPC with 32bit ffdshow since I guess it avoids all of that hassle.

You'd have thought the US' system and/or the EU would have pinged them for anti-competitiveness (anti-trust?) or something like that. :) Maybe that's coming in th future.

Thanks

[albain]

@dann23
I don't understand your point : in DXVA1 the buffers are allocated by the renderer. In DXVA2 the buffers are allocated by FFDShow, but in all cases this is the same result : buffers are pointers to direct3D surfaces wherever they are allocated.
And in all cases they have to be copied from the GPU
You could argue that they could just be read instead of copied but this would be worse : each internal filter needs to read the direct3D surface (or uncompressed buffer if you prefer) and as this reading is very slow, this is better to copy it once for good.

@Jeremy
I don't understand your point either : there is nothing to do with the CPU/GPU, this is a matter of bandwidth and direction (GPU to or from CPU)
The way you read can be tuned and this is what we tried : with SSE optimizations you fill and copy 128 bits registers but the improvements is very thick

No, the only (and last) lead is the intel's employee who claims to get a bandwidth that is 100x the bandwidth that we get.
But I think he made his tests with low res videos with which the bandwidth is not saturated.

Also note that we are talking about reading (GPU=>CPU), writing is very fast though.

[iron2000]

Just tried SPlayer.
Their GPU acceleration works very well.
No artifacts on seek and fast fullscreen switch.

Hope ffdshow will archive that soon too.

[Astrophizz]

Last I checked, SPlayer achieved GPU acceleration (and quick software decoding) by using a cracked version of CoreAVC. I'm not sure if that's still the case but I had an unpleasant experience uninstalling SPlayer the last time so I'm hesitant to try it again...

Edit: Ok, this (http://trac.splayer.org/changeset/1036) would seem to indicate that CoreAVC is no longer in SPlayer. In that case I can only guess that SPlayer now uses DXVA akin to MPC-HC.

[dann23]

@dann23
I don't understand your point : in DXVA1 the buffers are allocated by the renderer. In DXVA2 the buffers are allocated by FFDShow, but in all cases this is the same result : buffers are pointers to direct3D surfaces wherever they are allocated.
And in all cases they have to be copied from the GPU
You could argue that they could just be read instead of copied but this would be worse : each internal filter needs to read the direct3D surface (or uncompressed buffer if you prefer) and as this reading is very slow, this is better to copy it once for good.


I'm not argue about anything :) All I want is just dxva and subtitles. And as I understand from previos posts is that you don't have to read dxva buffers in this case(hope i'm not wrong). That's what I'm trying to say.

[tal.aloni]

All I want is just dxva and subtitles. And as I understand from previos posts is that you don't have to read dxva buffers in this case(hope i'm not wrong). That's what I'm trying to say.

yes, we have two alternatives that doesn't involve reads (only writes):
1. overlay the subs on top of the decoded picture, this seems to work well. (it's going to be faster than the method used in beta 2)

2. write a second buffer that include only the subtitles and opacity map (AYUV 32), and let the GPU do the alpha-blending. this should be possible, but I can't get it to work yet.

[onomatopellan]

Thanks you guys for your work in ffdshow dxva. I want to show my test with an ancient cpu like an Atlhon XP 2800+ and an ASUS AH3450 AGP 8x (ATI Catalyst 9.2) in Windows 7 32bits. Using MPC-HC 1.3.1503 with ffdshow dxva. The video is 720p and subs are external SSA.

Post Processing Disabled: cpu at 17%. It plays the movie without problem.

Overlay:
ffdshow_rev3206_20100118-DXVA-Post-Processing-Beta1: cpu at 70%. Subs are low quality and somewhat green but it's fast. I can watch 720p movies+subs without problem.
http://i47.tinypic.com/2lsekwk.jpg

ffdshow_rev3208_20100119-DXVA-Post-Processing-Beta2 cpu at 90%-100%. Subs are better and without the green colors. It shows garbage/blocks in the image at the beginning. Almost unwatchable.
http://i46.tinypic.com/357m2qt.jpg

Blend: with rev3208 cpu at 100%. Very good subtitles but unwatchable. Lots of garbage on the image at the beginning.
http://i45.tinypic.com/2m64vih.jpg


Full Post Processing: cpu at 100%. VERY slow (like 1fps). Lots of garbage on the image. Great subs but unplayable and unwatchable.

It would be great if you can implement an "overlay fast mode/low quality" like in rev3206 but without the green subs problem. Is this possible?

[albain]

I'm not argue about anything :) All I want is just dxva and subtitles. And as I understand from previos posts is that you don't have to read dxva buffers in this case(hope i'm not wrong). That's what I'm trying to say.

No problem, we consider any proposition, giving the fact that we are pretty out of ideas

[rsd78]

No problem, we consider any proposition, giving the fact that we are pretty out of ideas

I will add my vote to simply getting dxva + subtitles to work with low cpu usage. I can't speak for all users, but I think for those looking to get dxva working with ffdshow, the big plus would be able to get subtitles in any directshow player.

As you know there are plenty of current methods of getting dxva in general in any player. Currently the only option to get dxva + subs is either MPC-HC or CoreaVC (which requires an NVIDIA gpu). Give me that regardless of the player and gpu (assuming dxva is supported) and you extend the usefulness of ffdshow to a lot of users imo (especially as lower power htpcs are becoming more popular).

[hoborg]

+1 to dxva + subtitles

[Perls]

No problem, we consider any proposition, giving the fact that we are pretty out of ideas

Hi Albain,

When using dxva2, have you allocated the decoded frames in gpu-local memory or in non-cachable cpu memory? As it was an Intel guy who wrote the article I guess he only tried with frames decoded to un-cacheable main memory (intel integrated graphics). Accessing the frames over pci-express probably yields low throughput due to latencies.

[tal.aloni]

onomatopellan,
thanks for testing,
expect performance similar to beta 1, or slightly better.
(unless somebody else will optimize it further)

also, I have to confess, the subs in beta 1 were green because I mistaken (0,0,0) in yuv to be black, while it's actually green. :rolleyes:

[roytam1]

Sorry to bother you but when will ffdshow enable COOK audio codec?

[Jeremy Duncan]

Link to where you can watch this thread for replies to your problem. (http://software.intel.com/en-us/forums/showthread.php?t=71468&o=d&s=lr)

"Tom Craver works with 3D graphics IHVs to help them optimize their driver software on Intel's latest processors.
Previously he developed and validated driver and user interface software for cable modems and for Intel's DVI multimedia technology.
Prior to joining Intel, Tom was a member of the technical staff at the David Sarnoff Research Center in Princeton, NJ, and before that, he was with AT&T's Bell Laboratories.
Tom holds B.S. degrees in physics and computer science from the University of Illinois.
He also has a M.S. degree from Purdue University. His e-mail is tom.r.craver(insert at here)intel.com."

http://thumbnails27.imagebam.com/6537/7b8f0e65362601.gif (http://www.imagebam.com/image/7b8f0e65362601)

Tom Craver wrote that article you are talking about. Email him and ask him to look at the thread I started and reply to it. then registaer there and talk to him about your problem with dxva.

[tetsuo55]

@dann23
I don't understand your point : in DXVA1 the buffers are allocated by the renderer. In DXVA2 the buffers are allocated by FFDShow, but in all cases this is the same result : buffers are pointers to direct3D surfaces wherever they are allocated.
And in all cases they have to be copied from the GPU
You could argue that they could just be read instead of copied but this would be worse : each internal filter needs to read the direct3D surface (or uncompressed buffer if you prefer) and as this reading is very slow, this is better to copy it once for good.

@Jeremy
I don't understand your point either : there is nothing to do with the CPU/GPU, this is a matter of bandwidth and direction (GPU to or from CPU)
The way you read can be tuned and this is what we tried : with SSE optimizations you fill and copy 128 bits registers but the improvements is very thick

No, the only (and last) lead is the intel's employee who claims to get a bandwidth that is 100x the bandwidth that we get.
But I think he made his tests with low res videos with which the bandwidth is not saturated.

Also note that we are talking about reading (GPU=>CPU), writing is very fast though.Haali has some ideas about this iirc, please contact him.

[cc979]

Sorry to bother you but when will ffdshow enable COOK audio codec?

i was curious about that also, as vlc is able to play COOKED audio

[tal.aloni]

Great detective work Jeremy,

email sent.

[BatKnight]

Does anyone here as MKV sample with VC-1 that you tested working with DXVA?
All my MKV VC-1 files don't play in DXVA in ffdshow or MPC-HC, but the MKV H264 play in DXVA, and I would like to test with a VC-1 file that one can assure it's DXVA compatible.

Thanks

Bat

[HeadlessCow]

I've got a couple of VC1 samples that I know work with DXVA at home. I'll try muxing one into MKV tonight and see what happens with ffdshow and the old MPC-HC DXVA filter as well.

[hoborg]

Does anyone here as MKV sample with VC-1 that you tested working with DXVA?
All my MKV VC-1 files don't play in DXVA in ffdshow or MPC-HC, but the MKV H264 play in DXVA, and I would like to test with a VC-1 file that one can assure it's DXVA compatible.

Thanks

Bat

nVidia doesn't have VC-1 DXVA support if i am not wrong...

[Px]

nVidia doesn't have VC-1 DXVA support if i am no wrong...
Very limited support

[BatKnight]

Very limited support
Even with DXVAChecker stating: ModeVC1_IDCT: DXVA1/2, NV12, 720x480 / 1280x720 / 1920x1080?

So this means we (Nvidia owners) will not have DXVA for VC-1 files using ffdshow? Is this a Nvidia limitation or a ffdshow limitation?

Bat

[HeadlessCow]

I only have ATI cards (and integrated Intel on my laptop that doesn't support anything useful), do you still want me to check vc1/mkv support?

[HeadlessCow]

DXVAChecker on my ATI card lists ModeVC1_VLD rather than just IDCT. That's probably the difference that matters.

Also, tal.aloni on my card DXVAChecker also lists this under Processor Device:
DXVA2_VideoProcess_AlphaBlend: No
DXVA2_VideoProcess_AlphaBlendExtended: No

If that refers to the alpha blending operation you're trying to use, I bet that's why it's not working :(

[tal.aloni]

you're going to like beta 3:
+ text input pin has been enabled (thanks Albain).
+ I've restructured some methods to get some speed boost in overlay mode.

I want to focus on overlay for now so we'll be able to commit it soon, other methods will be discussed later.
+ please let me know if it's fast / stable enough to be commited.
+ also, I've fixed a small bug that is related to the software decoder, there is a method that supposed to generate empty picture with black background, but instead generates gray background, I fixed that (needed for overlay), please let me know if this have any effect on the software decoder (subs / osd maybe?), (I'll commit this before the main patch just to be sure)

build:
http://iknowu.net/files/public/ffdshow/DXVAPostProc/ffdshow_rev3227_20100126-DXVAPostProcessing-Beta3.exe

patch:
http://iknowu.net/files/public/ffdshow/DXVAPostProc/3227-DXVA-Post-Processing-Beta3.patch

p.s.
HeadlessCow, you're awesome, I've just checked my primary card,
(Radeon 4550), same results as yours - does not support DXVA alpha-blending. at least I know why it didn't work. I also have a geforce 8200, I might test it as well when I have time.

[tal.aloni]

Developers, CruNcher:
there seems to be indeed a problem with libavcodec like CruNcher reported, he told me earlier that it's build related, but did not confirm yet.

I've got this twice during debug, using xhmikosr's 3227 build:

http://iknowu.net/files/public/ffdshow/DXVAPostProc/ffdshow_libav_crash.png

[XhmikosR]

The only thing I changed recently, as I already have mentioned, is that I updated my GCC to 4.4.3.
Here (http://www.mediafire.com/?o2nvuijq5mo) is a build compiled with GCC 4.4.2 and here (http://www.mediafire.com/?mdmdziknm2m) is one compiled with GCC 4.5.0.

[HeadlessCow]

Beta3 report for me.... is mostly sadness.

MPC-HC crashes whenever I try to close it or change videos when using the DXVA decoder. This definitely fails the stability test ;)
Subtitle pin connects nicely, but doesn't actually display any subs. If I select a file specifically, it displays fine (well, doesn't match vsFilter output, but I'm not sure how close it should be matching anyways. Subs have a distinct black border that seems out of place.)
720p with overlay displaying OSD is about 4% CPU... pretty close to what it was before. Unfortunately I don't really have a slower system with DXVA support that I can test on to see if it performs reasonably on a more mainstream machine, but it's fast enough for me :-D
VC1 in WMV or MKV works fine for me. (Both MPC Decoder and ffdshow one)

[_xxl]

Developers, CruNcher:
there seems to be indeed a problem with libavcodec like CruNcher reported, he told me earlier that it's build related, but did not confirm yet.

I've got this twice during debug, using xhmikosr's 3227 build:

http://iknowu.net/files/public/ffdshow/DXVAPostProc/ffdshow_libav_crash.png
If I remember correctly in latest revisions something had changed for thread free.

[tal.aloni]

The only thing I changed recently, as I already have mentioned, is that I updated my GCC to 4.4.3.

Thanks,
both crash.

3216 doesn't crash.
3219 does crash.

it's easy to recreate, I just drag another file to MPC-HC while playing DXVA content.

[XhmikosR]

OK, so I guess the problem is not GCC related. r3217 (http://www.mediafire.com/?5mmmy2inzrx) and r3218 (http://www.mediafire.com/?dmc00yzzynh).

[littleD]

OK. i thought i can try new ffdshow dxva.
I have sempron and hd3450. Win7 beta. cat 9.12 - i consider them bad for video playing.
First of all bad colorspace (bluefaces/green screen) was gone. VC-1 acceleration now works good. But i think i get higher cpu usage when playing from m2ts container.

I have to set post processing - software overlay for Subtitles and osd to make them appear. OSd make cpu go 100% but video somehow play almost full speed. Shadows of osd font are bad quality, while subtitles are hq and sharp. Havent seen any cpu increase when subs are on.

When i want play another file, or want to close MPC HC, player crashes. Always.

[albain]

Haali has some ideas about this iirc, please contact him.

What do you mean ? Do yo have his contact information ?

[tal.aloni]

Thanks for the feedback,
1. the sharp edges are a given (because we're overlaying and not blending). you can minimise this effect by using different settings.
I've tried different methods for overlay, this one seems to give the best ratio between quality and performance.

2. both my nVidia and ATI adapters does not support DXVA 1.0 Alpha Blending. however, both of them support DXVA 2.0 Substream processing, which was designed for subtitles.
I'm not very motivated to develop something that I can't use on my favourite OS, but it can be developed and used (Vista / Windows 7).
[please keep in mind that it's going to consume more CPU resources than overlay]

3. XhmikosR, I'll test within a few hours. :thanks:

Tal

[clsid]

What does the internal subtitle renderer of MPC-HC use? That works with DXVA and has low overhead.

[CiNcH]

What does the internal subtitle renderer of MPC-HC use? That works with DXVA and has low overhead.
Direct3D I would say...