ffdshow tryouts project: Discussion & Development

[Frans]

I remember I have experienced a similar problem. This time, I can't reproduce for now. Could you send us a sample?

Btw, I have filled a bug report through sourceforge (at Nov 2nd) about the SRT position bug (which already been fixed since 2319 build), and SSA text overlapping (which still present at the latest build). It is broken since 2270 build (I think when albain do Improvements of SSA subtitles : collisions, position, move, layers). I don't know if your problem is the same as me though. I have included the sample there. I don't know if you guys can reproduce it, but it is certainly consistent whenever I tried build 2270 and up, the SSA problem that I described happen... try 2269 and everything is ok.

And I have filled another bug report on SSA with overlay box (like you make a transparent horizontal box with text in it). It isn't very important though (since not many use it), but nice to have it behave like it meant to be.

Thanks

[MatMaul]

I think I found a bug with flash videos : the video isn't cropped correctly if it is not mod16.
I don't have this problem with internal MPC-HC decoder.
here is a sample :
http://www.etud.insa-toulouse.fr/~mvelten/bordel/12431456.flv

[tal.aloni]

@clsid, xxl, haruhiko

I've updated the definition names in 71 files of "our ffmpeg" to match the current ones (ffmpeg trunk). this will allow for quicker ffmpeg updates (less differences).
I've spoken about it with clsid,
if you have any objection / suggestion before I commit,
let me know.

Tal

patch:
http://iknowu.net/files/public/ffdshow/2326-ffmpeg-Def.patch

[Thunderbolt8]

are the people who work on ffdshow-mt the same ones as those of normal ffdshow? is there also a ffdshow-mt forum here or somewhere else where I could state requests?

[tal.aloni]

are the people who work on ffdshow-mt the same ones as those of normal ffdshow?
yes, ffdshow-mt is identical to the regular ffdshow except that it's including ffmpeg-mt instead of ffmpeg (libavcodec decoding library).

[Thunderbolt8]

I've compared some movies with ffdshow and ffdshow-mt with my C2D and I have to say it's pretty similar.

-AVC and VC-1 performance is about the same
-interlaced H.264 is multithreaded with ffdshow-mt, but not yet with ffdshow
-at the current version ffdshow-mt still creates rainbow frames with some AVC videos, while its already fixed with current ffdshow rev.

comparing the AVC multithreading performance I can say that there have been improvements since the last year. last year I had 2 AVC titles which I couldnt play at 3.2GHz (delay piling up), which now play fluently with 2.8GHz. the usage of both CPUs looks quite balanced in task manager (at least for 2 cores, might be different on quad core systems; might be able to look at that next weekend), there used to be a broader gap back then.

so basically whats left is still some VC-1 multithreading improvement. theres still some gap between both cores which is not really a problem for my system (400x7, 2 cores), but on my fathers C2Q6600 (275x10, 4 cores) for example this difference is much bigger, there is 1 predominantly core with high usage, while the other 3 cores hover with comparably lower load and waste a bit potential. thus, the video does not play smoothly, delay piling up.

heres a screen of my system with VC-1 load: http://thumbnails9.imagebam.com/1843/d7375e18420762.gif (http://www.imagebam.com/image/d7375e18420762)

so I would really appreciate it if a bit more of VC-1 multithreading optimizing could be done.

thank you all so far for your work!!

[LoRd_MuldeR]

Thunderbolt8, something must be wrong with your test. Maybe the performance was bottlenecked by the by renderer!?
Or your sample video simply didn't require more CPU time to run smoothly on the test machine...

My test clearly shows that all four cores get used by ffdshow-mt at ~100% when decoding H.264 footage:

http://img362.imageshack.us/img362/6452/ffdshowmtsp2.th.png (http://img362.imageshack.us/my.php?image=ffdshowmtsp2.png)http://img362.imageshack.us/images/thpix.gif (http://g.imageshack.us/thpix.php)

Also my test (http://forum.doom9.org/showpost.php?p=1211080&postcount=34) has shown that the throughput has more than doubled in ffdshow-mt (which uses all cores), compared to ffdshow (which roughly uses two cores).

[STaRGaZeR]

My test clearly shows that all four cores get used by ffdshow-mt at ~100% when decoding H.264 footage:

Because you're forcing it with a benchmark. If you play actual footage at normal FPS you will get something like what Thunderbolt has (BTW he's talking about VC-1, not H.264). This is not bad, it's the normal operation of the Windows scheduler, balancing the load between cores. As you increase the workload, you'll end with the four cores loaded progressively. If possible, you want core 0 loaded and others sleeping to improve power consumption. If more power is needed, core 1 comes to the rescue, and so on. However, there are times in which Windows loads cores without any logic. You can also see single threaded applications spreaded all over the four cores, never using more than a 25% combined.

Exactly what ffmpeg decoders are multithreaded in ffmpeg-mt? H.264 for sure, any other? I wasn't aware about the VC-1 decoder being multithreaded.

[haruhiko_yamagata]

Exactly what ffmpeg decoders are multithreaded in ffmpeg-mt? H.264 for sure, any other? I wasn't aware about the VC-1 decoder being multithreaded.MPEG-4 decoder is multithreaded. However it is not working yet, not enabled in ffdshow.
Only H.264 is multithreaded in ffdshow for now.

[haruhiko_yamagata]

@clsid, xxl, haruhiko

I've updated the definition names in 71 files of "our ffmpeg" to match the current ones (ffmpeg trunk). this will allow for quicker ffmpeg updates (less differences).
I've spoken about it with clsid,
if you have any objection / suggestion before I commit,
let me know.

Tal

patch:
http://iknowu.net/files/public/ffdshow/2326-ffmpeg-Def.patchI think the patch is OK.
By the way, the safest way to apply updates from FFmpeg is to manually apply the patch (new revision) one by one. I guess this is how clsid and xxl is doing every time, unless they are sure that it is safe to overwrite the file.

[_xxl]

I would prefer to merge trunk in ffmpeg-mt, but starting with an older revision. There are to many diff's.

[Thunderbolt8]

MPEG-4 decoder is multithreaded. However it is not working yet, not enabled in ffdshow.
Only H.264 is multithreaded in ffdshow for now.
what does multithreaded mean here? using more than 1 core? or more than two? because in the picture I added I was using VC-1 and as you can see both cores are carrying load. not completely balanced, but still both are used to a certain extend, when I have features which are clearly not supported by multithreading then 1 core shots up to the sky, while the other crawls at bottom.

next weekend I can try to add screens of our C2Q using H264 stuff and VC-1 with ffdshow and ffdshow-mt.

[Dark Shikari]

what does multithreaded mean here? using more than 1 core? or more than two? because in the picture I added I was using VC-1 and as you can see both cores are carrying load. not completely balanced, but still both are used to a certain extendThe operating system automatically swaps between the cores very quickly, resulting in a 100% CPU singlethreaded process using 50% of both cores. This doesn't improve performance.

[_xxl]

see both cores are carrying load. not completely balanced
Look here:
http://forum.doom9.org/showthread.php?p=923944#post923944
Try process explorer's single aggregate graph, in process properties. Or just set MPC's affinity to one CPU. You'll find the graphs are mostly stable, and all you're seeing is the kernel's habit of tossing threads around in strange fashions. It doesn't really affect performance to execute half-and-half on two cpus instead of one.

[Thunderbolt8]

thanks for that explaination. I guess I then have the problem that I dont have normal (not funny like interlaced) vc-1 stuff that uses more load than that and cannot test what would happen if load really gets higher than 100% on 1 core :S

however, when I use 1 core im slightly below 50% use, with 2 cores slightly above. how can that be, does the balancing between cores takes up load as well? and is only the load of the core listed with % which uses most load of all cores?

[haruhiko_yamagata]

I would prefer to merge trunk in ffmpeg-mt, but starting with an older revision. There are to many diff's.Do you mean merging the update in the git to ffmpeg-mt branch in ffdshow?

Though it would take very long time, I think it's best to check out the diffs in the last four commit in the git and apply manually one line by one line.
And I will wait :) .

[Reimar]

The operating system automatically swaps between the cores very quickly, resulting in a 100% CPU singlethreaded process using 50% of both cores. This doesn't improve performance.

Hm, I haven't paid attention on Windows, but I've never seen that on Linux.
This seems like a very stupid thing to do, especially on AMD processors, since all the data you need probably wouldn't be in the cache anymore each time you switch the CPU (and if there was a scheduler that does this for multiple-socket Opterons then its programmer must have had 80% of his brain removed shortly before).

[leeperry]

not sure whether it's a bug(yes I've disabled the "keyboard control" options), but if I press the "c" key in the Avisynth script filter while playing a movie, ffdshow shuts down.

If I do it while a movie is not playing, it's cool :confused:

PS : also happens with pressing <RETURN>...and the arrow keys simply don't do anything.

[clsid]

I would prefer to merge trunk in ffmpeg-mt, but starting with an older revision. There are to many diff's.
The ffmpeg-mt changes focus just on the core of libavcodec (avcodec.h, utils.c etc.) and the MPEG based video stuff (h264.c, mpegvideo.c etc.). So you would have to compare those parts manually. Other files, such as audio stuff and other video codecs, you could simply overwrite with the versions from trunk. That should save a lot of time.

[ikarad]

What is the difference between ffdshow, ffdshow MMX, ffdshow SSE and ffdshow-mt?
http://www.free-codecs.com/download/FFDshow.htm

[LoRd_MuldeR]

What is the difference between ffdshow, ffdshow MMX, ffdshow SSE and ffdshow-mt?
http://www.free-codecs.com/download/FFDshow.htm

ffdshow-mt was built using the "mt" branch of ffmpeg. This means it uses multi-threaded decoders, which are not in the "main" branch yet. It's experimental, try it at your own risk!
For exmaple ffdshow-mt is able to use all CPU cores for H.264 playback, while the "normal" ffdshow is limited to roughly two cores in that case.

As for the other suffixes like "SSE" or "MMX" it simply means the compiler was instructed to build the binary for CPUs that support the MMX (or SSE) instruction set at least.
There usually is only a small speed improvement in such "CPU optimized" builds (compared to the "generic" ones), as all the important functions are hand-optimized ASM code anyway...

I'd recommend to download the latest build from the official ffdshow-tryouts sourceforge project site:
http://sourceforge.net/project/showfiles.php?group_id=173941&package_id=199416

(You also find "ICL" builds there. These are builds created with Intel's Compiler, resulting in some speed-up for certain filters)

[LoRd_MuldeR]

Because you're forcing it with a benchmark. If you play actual footage at normal FPS you will get something like what Thunderbolt has (BTW he's talking about VC-1, not H.264)

Sure, in normal playback the decoder will use as many CPU time as needed for smooth playback. At the point where the frames are decoded in playback-speed (and the frame cache is full) it won't use any more CPU time for obvious reasons. Therefore you can't play "some" video and then complain that the decoder doesn't utilize all of your cores. In fact you should be happy to get smooth playback at low CPU load. But if you want to test the multi-threading capabilities of a decoder, you must use a benchmark (e.g. TimeCodec) and push the decoder to its maximum throughput. Looking at the CPU load in Taskman/Procexp isn't very useful either. You must compare the actual throughput (e.g. the "dfps" value in TimeCodec). High CPU load alone doesn't indicate good performance...

[Thunderbolt8]

ffdshow-mt was built
For exmaple ffdshow-mt is able to use all cores or more for H.264 playback, while the "normal" ffdshow is limited to roughly two cores in that case.
is the number of decoding threads under encoding options automatically determined by ffdshow for each cpu? is it exactly 1 core = 1 thread? if not, would it make sense to alter that number to achieve better effects?

[LoRd_MuldeR]

is the number of decoding threads under encoding options automatically determined by ffdshow for each cpu? is it exactly 1 core = 1 thread? if not, would it make sense to alter that number to achieve better effects?

Usually "threads = cores" should be correct. It doesn't necessarily apply to all applications (e.g for x264 it's "threads = cores * 3/2"), but for ffdshow-mt/ffmpeg-mt it does apply.

At least that is what my test has shown. Using "threads > 4" or "threads < 4" resulted in less throughput with ffdshow-mt and H.264 video on my Q6600 ...

[STaRGaZeR]

Sure, in normal playback the decoder will use as many CPU time as needed for smooth playback. At the point where the frames are decoded in playback-speed (and the frame cache is full) it won't use any more CPU time for obvious reasons. Therefore you can't play "some" video and then complain that the decoder doesn't utilize all of your cores. In fact you should be happy to get smooth playback at low CPU load. But if you want to test the multi-threading capabilities of a decoder, you must use a benchmark (e.g. TimeCodec) and push the decoder to its maximum throughput. Looking at the CPU load in Taskman/Procexp isn't very useful either. You must compare the actual throughput (e.g. the "dfps" value in TimeCodec). High CPU load alone doesn't indicate good performance...

Exactly.

[haruhiko_yamagata]

I would prefer to merge trunk in ffmpeg-mt, but starting with an older revision. There are to many diff's.
Thank you for your work. Now it plays all H.264 as far as I have tested.

[clsid]

In case anyone is wondering what needs to be done before ffmpeg-mt gets merged into the FFmpeg trunk. Here is the ToDo list of Alexander Strange:
http://gitorious.org/projects/ffmpeg/repos/ffmpeg-mt/blobs/master/mt-work%2Ftodo.txt

Of course, once he is done, the code must also go through a review process performed by the head FFmpeg developer.

[tal.aloni]

_xxl, great work, my previously mentioned interlaced files are now playing back smoothly after the last commits!

not pushing or anything, but what is the plan for MLP support in the ffdshow-MT branch? does it require many modifications to the ffmpeg library?

Thanks a lot,
Tal

[_xxl]

This weekend I hope...

[Thunderbolt8]

_xxl, great work, my previously mentioned interlaced files are now playing back smoothly after the last commits!

not pushing or anything, but what is the plan for MLP support in the ffdshow-MT branch? does it require many modifications to the ffmpeg library?

Thanks a lot,
Tal
which version do you talk about? I got the latest clsid 2335, but interlaced h264 is still 1 core there.

btw. why does 1 core h264 stuff clearly looks like 1 core usage with 1 core loaded fully and the 2nd almost not at all, while VC-1 throws it back and forth over the cores?

[tal.aloni]

which version do you talk about? I got the latest clsid 2335, but interlaced h264 is still 1 core there.

I was referring to ffdshow-MT (MT for multi-threading) (you can get the latest from xvidvideo.ru).

[Thunderbolt8]

have it already. I just thought, since people here were talking about merging that those features would have been added to regular ffdshow.

[clsid]

It won't be merged anytime soon. The MT stuff is still unfinished.

[Reino]

There are a couple of things I'd like to talk about concerning FFDShow it's functionality.

First of all; the FPS1 (Fraps) support in FFDShow.
I compared the decoded video image with the original Fraps:
http://img101.imageshack.us/img101/959/frapsvsffdshoweg2.gif
I don't know if it's a totaly different colourspace or just a difference in brightness/gamma...maybe you know (how to fix it).

Secondly, I noticed the Volume Normalization feature has been "patched". Why?
I've been using the Volume Normalization feature for a very long time and never have I had any problems with it!
At the moment I'm using rev.2322 and I can honoustly say that I really dislike the new "patch". Not only have I been using this feature for audio material with too low sound volume, but also for audio files with sound that was too loud!
Right now the amplification never goes below 100%. At moments where it should (too loud audio volume) that's really irritating. But I've also had moments where the amplification stayed at 100% when I was expecting much more (too low audio volume). I've also had moments of amplification 400% the whole time for no special reason at all, the audio was already pretty loud of it self.
Why has this feature been "patched"?

I would also like to make a suggestion.
Within the systemtray-menu you can already select different profiles, subtitle sources, but how about the ability to select a different OSD profile? I often use different OSD profiles, but every time I need to open up the entire Video Decoder Configuration window if I want to switch to another profile. It would be much easier/faster if I could select my profile in the systemtray-menu.

Apart from that...I've been happily using FFDShow for many years now, so keep up the good work!

[bebo]

When can we expect to see MLP capbility in ffdshow? I am trying to play TrueHD in VMC. Video is fine only AC3 streams play. Newer BDs are showing up with the english track only in MLP so for Iron Man, Transformers and Kung Fu Panda I only get French or Spanish in 5.1 WAF is extremely low. Am I doing something wrong? I read that MLP is only active in MPC-HC. I also read somewhere that ffdshow MLP decode doesn't work with Haali? Any enlightenment would be greatly appreciated.

Thanks

[MatMaul]

Secondly, I noticed the Volume Normalization feature has been "patched". Why?
just check "regain volume" and you will have the same behavior as before.

[Leak]

When can we expect to see MLP capbility in ffdshow?
How about "now"?

Download clsid's latest ICL10 build (http://sourceforge.net/project/showfiles.php?group_id=173941&package_id=199416&release_id=439938), it includes MLP.

np: Duo505 - Tsip Tsap (Late)

[rack04]

Is libavcodec stable with MPEG2? The reason that I ask is because this is not a default selection. Should "Use Speedup Tricks" and "DVD Decoding" be selected?

[tal.aloni]

As MatMaul said, there are now two modes of normalization.
I must apologize, but I never implemented an upgrade path. (and it would be too complicated to do so).


I've been using the Volume Normalization feature for a very long time and never have I had any problems with it!

you mean, besides the occasional overflows, right?
anyway, you should like the new one much more. (regain volume checked)


Right now the amplification never goes below 100%

We decided not to support non-standard files.
(more detailed explanation here: http://forum.cockos.com/showthread.php?p=252076#post252076)

Tal

[LoRd_MuldeR]

Is libavcodec stable with MPEG2?

I use it all day long for DVB playback in DVBViewer and haven't encountered problems yet. So I'd say it's pretty stable ;)

[jos99]

I use it all day long for DVB playback in DVBViewer and haven't encountered problems yet. So I'd say it's pretty stable ;)

Any advantage with it? Also as op noted, do you have "Use Speedup Tricks" and "DVD Decoding" selected?

[bebo]

How about "now"?

Download clsid's latest ICL10 build (http://sourceforge.net/project/showfiles.php?group_id=173941&package_id=199416&release_id=439938), it includes MLP.

np: Duo505 - Tsip Tsap (Late)

Leak - Much appreciated. Was very excited to see the new options. So far, I've spent an hour trying to get MLP playback in VMC media player (using Media Contol). So far - nada...

Switched over to MPC-HC. Under 'Navigate' I see the MLP stream selected. In 'Codecs' I have AC3-libacodec; OSD-on

I see the MLP track is trying to be decoded but Jitter=31 and FPS=2

total disaster - Can anyone tell me what I am doing wrong. Would prefer to get MLP audio in VMC - only switched to MPC to test.

Thanks

[littleD]

Bebo, probably ffdshow doesnt try to decode with haali spliter. Enabling internal mpeg/ps/ts.. splitter in mpchc should work.

So, download from mpchc sourceforge site newest standalone mpeg splitter, and give it highest merit (if it hasnt already). I didnt tested but may help with other dshow players.

[Ger]

I use it all day long for DVB playback in DVBViewer and haven't encountered problems yet. So I'd say it's pretty stable ;)

I use DVBViewer as well, and there are some channels that needs MPEG2 in FFDshow set to libmpeg2.

I recorded a short sample from a problem channel. Looks fine with MPC-HC internal (libmpeg2 based) decoder and with FFDshow set to libmpeg2, but is more or less unwatchable if I switch to libavcodec.

MPEG2-sample Problem with libavcodec but OK with libmpeg2.ts - 13.95MB (http://www.zshare.net/download/515348831460b2bb/)

I generally only follow the FFDshow and MPC-HC threads here, so if someone wants to forward this to the ffmpeg/libavcodec developers, feel free. :thanks:

[nm]

Any advantage with it?
libavcodec has better error resilience and concealment features than most other MPEG-2 decoders: http://compression.ru/video/codec_comparison/mpeg-2_2006_en.html
This may not be relevant for DVD sources, but it can have a significant effect in DVB-T reception with poor antennas or bad weather.

[nm]

MPEG2-sample Problem with libavcodec but OK with libmpeg2.ts - 13.95MB (http://www.zshare.net/download/515348831460b2bb/)

I generally only follow the FFDshow and MPC-HC threads here, so if someone wants to forward this to the ffmpeg/libavcodec developers, feel free. :thanks:
MPlayer decodes it fine with libavcodec (both with built-in and lavf MPEG-TS demuxers), so the problem seems to be somewhere else.

[fastplayer]

^ libavcodec had some issues with MPEG2 in the past and probably still has. libmpeg2 is a safe bet.
Any advantage with it? Also as op noted, do you have "Use Speedup Tricks" and "DVD Decoding" selected?
- "Use Speedup Tricks" reduces CPU usage at the expense of image quality.
- "DVD Decoding" is needed if you want to watch DVD content.

[Ger]

MPlayer decodes it fine with libavcodec (both with built-in and lavf MPEG-TS demuxers), so the problem seems to be somewhere else.

OK. I Only tested it in a recent build from ffdshow-tryouts trunk (using either MPC-HC or DVBViewer) and also in an older VLC version (0.8.6i because I have problems with the browser plugin in 0.9.x) in Vista. In VLC it failed even worse than in ffdshow/libavcodec (just a black screen in VLC and not the blocking seen with ffdshow).

[bebo]

...download from mpchc sourceforge site newest standalone mpeg splitter, and give it highest merit (if it hasnt already). I didnt tested but may help with other dshow players.[/QUOTE]

Thanks a lot Little D, will try later tonight and report status.

[nm]

^ libavcodec had some issues with MPEG2 in the past and probably still has. libmpeg2 is a safe bet.
Well, MPlayer already switched to libavcodec by default because it is actively maintained and less buggy: http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/2008-February/056430.html

There seems to be a problem with multithreading (http://lists.mplayerhq.hu/pipermail/mplayer-dev-eng/2008-March/056870.html) though, and I'm not sure if it has been fixed yet. ffmpeg-mt might also change things in the future. I tried threads=2 and that does cause image corruption with my MPlayer/libavcodec revision (r27833, I'll need to update to latest SVN). However, this affects all MPEG-2 videos, not just some specific samples, so it is probably unrelated to Ger's problem.