ffdshow tryouts project: Discussion & Development

[saint-francis]

well if it lets my GPU do the job, this will leave my CPU alone for HD post-processing(gamut conversion/LSF etc) :D

The catch is that decoding with CUDA seems to use no fewer CPU cycles than coreavc. So it's a moot point.

[clsid]

Once Mr VacBob fixes that one known decoding error in ffmpeg-mt, then the remaining issues are:
* Frame accurate seeking. Reported by CruNcher. Details (http://forum.doom9.org/showthread.php?p=1216287#post1216287). Although this might not be a problem specific to ffmpeg-mt. If not, it is not a regression, and thus not a show stopper.
* Which decoders currently use the new MT API in ffmpeg-mt? I vaguely remember a report about mpeg-2 decoding issues with multiple threads. But I don't know if that was with ffmpeg-mt or regular builds.

[clsid]

I tend to disagree,
why does another user should have the power to reset my equalizer settings? I think another user should be able to reset his own settings and the "common" settings. this should be able to ensure a "clean slate" with minimum damage to other users.
That other user is the administrator. The admin has the power to install/uninstall ffdshow and should imo also be able to reset the settings for all users. A regular user won't be able to reset your settings. And we could add a checkbox "for all users" to the installer, so that the admin can choose to only reset its own settings.

[clsid]

Since ffdshow is now able to decode RealVideo, it is finally possible to play RealMedia files in 64-bit players. That's pretty useful since there is not even a 64-bit version of RealPlayer. However, a single problem remains. There is no decoder for Cook audio yet. Our version of libavcodec already contains the decoding code and ffdshow also already has some cook related code, but there is no option yet in the GUI to enable it. I don't know why such option is missing, possibly because it did not work properly long time ago. Does anyone here has some time to look at it and add the missing option?

[squid_80]

The catch is that decoding with CUDA seems to use no fewer CPU cycles than coreavc. So it's a moot point.
Oh it does that alright. Just that the top speed is nowhere near CoreAVC unless you're on a low-end cpu. (Atom anyone?)

[Mr VacBob]

Once Mr VacBob fixes that one known decoding error in ffmpeg-mt, then the remaining issues are:
* Frame accurate seeking. Reported by CruNcher. Details (http://forum.doom9.org/showthread.php?p=1216287#post1216287). Although this might not be a problem specific to ffmpeg-mt. If not, it is not a regression, and thus not a show stopper.


H264 can already have large unpredictable frame delay (up to 16 frames), but frame threading makes it worse. Basically you should use reordered_opaque, and then pass in future frames until you get back the one that was being seeked to (and then NULL frames at the end of the file).

If you don't have future frames available, then you're stuck - this is the API issue I have to work out.


* Which decoders currently use the new MT API in ffmpeg-mt? I vaguely remember a report about mpeg-2 decoding issues with multiple threads. But I don't know if that was with ffmpeg-mt or regular builds.

MPEG2 doesn't use frame-threading, since all 480p MPEG2 has at least 30 slices per frame.

[leeperry]

The catch is that decoding with CUDA seems to use no fewer CPU cycles than coreavc. So it's a moot point.
you're saying that CUDA h264 decoding uses as much CPU as CoreAVC/ffdshow-mt ?
so how come x264 encoding from a h264 source is faster w/ neuron2's software then ?
I was under the impression that it was offering the same kind of performance as DXVA :o

[DigitalDeviant]

you're saying that CUDA h264 decoding uses as much CPU as CoreAVC/ffdshow-mt ?
so how come x264 encoding from a h264 source is faster w/ neuron2's software then ?
I was under the impression that it was offering the same kind of performance as DXVA :o

Core's initial benchmarks (look in the CoreAVC thread) only showed a decent fps increase in single core CPUs. I don't know about actual cycles. Dual core's showed a negligible increase less than 10% I believe and quad core's really took a hit with CUDA enabled. It's not full acceleration so far as anyone has hinted from Core. They say they're working on multicore improvements.

[nm]

Core's initial benchmarks (look in the CoreAVC thread) only showed a decent fps increase in single core CPUs. I don't know about actual cycles. Dual core's showed a negligible increase less than 10% I believe and quad core's really took a hit with CUDA enabled. It's not full acceleration so far as anyone has hinted from Core. They say they're working on multicore improvements.
However, hardware decoding on a GPU does leave the CPU almost completely unused, so leeperry has a point in combining CUDA video decoding with software postprocessing. Also custom GPU postprocessing is possible since video decoding is done in a unit separate from the stream processors. Limited decoding throughput is not a problem for realtime playback.

[STaRGaZeR]

Well, that is a regression.

edit: actually, it was because the dequant matrix changes in the middle of the stream

Which sample? I don't know if it's the same you're refering to, but this one also has decoding artifacts that disappear when only 1 thread is used. It comes from Blu-ray:

http://www.megaupload.com/?d=90KHBJTQ (42MB)

(It's the same I sent to you Haruhiko)

[Snowknight26]

Any plans to implement multithreaded decoding for interlaced H.264 streams apart from in ffdshow-mt (if in there at all)?

[CruNcher]

However, hardware decoding on a GPU does leave the CPU almost completely unused, so leeperry has a point in combining CUDA video decoding with software postprocessing. Also custom GPU postprocessing is possible since video decoding is done in a unit separate from the stream processors. Limited decoding throughput is not a problem for realtime playback.

Exactly and that makes it pretty fast (Energy Efficient) and not as CPU utilizing as CUDA itself so the way CoreAVC implements it is suboptimal, Donald is the only one yet that does it right :)
Though no one says that it isn't possible to create a Interface via ffdshow to Nvcuvid API and use it to render on any surface you can think of being fully independent of DXVAs restrictions :)
Jesus Nvidia gave every Developer the opportunity to leverage the VP2 Decoding Power for Encoding/Decoding it's all in the SDK available to everyone, and currently Donald is doing a lot of debugging work with the Doom9 Users and Nvida Engineers to improve it's capabilities and squash bugs, only requirement is Direct3D currently and a supported VP2 Nvidia Card leveraging it on the Linux side is also possible via VDPAU.
All these improvements help the whole Nvidia Video Ecosystem every 3rd party Developer can leverage from that improvements and with Doom9 Users backing it it should be soon the most stable GPU Decoder solution available it already is more advanced and stable then libavcodec (H264,VC-1) :)

And if you already have the capabilities of using this you should because everytime you load a blu-ray on such a VP2 capable machine with CoreAVC or libavcodec you throw out massive amount of Energy into to the AIR.

[Mr VacBob]

Which sample? I don't know if it's the same you're refering to, but this one also has decoding artifacts that disappear when only 1 thread is used. It comes from Blu-ray:

http://www.megaupload.com/?d=90KHBJTQ (42MB)

(It's the same I sent to you Haruhiko)

Right, I meant that one. It's fixed in the git repository now.

Any plans to implement multithreaded decoding for interlaced H.264 streams apart from in ffdshow-mt (if in there at all)?

It works the same whether it's interlaced or not; ffdshow-mt handles it fine, and mainline handles it fine if it was encoded with slices.

[Snowknight26]

What about the current trunk? I'm asking because with with the current trunk a 20Mbps 1080i25 sample I have is being decoded at 15-20fps (1 core is used), while 35Mbps 1080p24 content decodes at the proper fps yet it uses more than 1 core.

[nm]

What about the current trunk? I'm asking because with with the current trunk a 20Mbps 1080i25 sample I have is being decoded at 15-20fps (1 core is used)
Then it's probably not sliced and you need frame-based threading (ffdshow-mt). Interlacing shouldn't matter.

[Snowknight26]

Last I heard, H.264 Blu-rays used slice coding.

[nm]

Could you upload a sample?

[Snowknight26]

http://stfcc.org/misc/00000.cut.m2ts

[nm]

The actual problem is this, as given by the decoder: "Cannot parallelize deblocking type 1, decoding such frames in sequential order"
To disable deblocking across slices and enable multithreading, you'll need to use the FFmpeg parameter "-flags2 fast" (I don't know how to set it in ffdshow).

[Snowknight26]

Heh, another issue with my infamous 1080i25 video (same as the link above). After being about 1:30:00 into it, the current frame # is up in the upper 500000s. It seems that when I check Grab and have it save every frame, the frame number increments by 5 each time.

Also, although this is probably a libavcodec issue, I didn't know where else to bring it up. That 1080i25 stream is PAFF if I remember correctly. Parts of it are interlaced, parts aren't. The parts that are interlaced are deinterlaced fine when deinterlacing is turned on. However, the parts that aren't interlaced are still deinterlaced with deinterlacing turned on, regardless of having 'Process frames flagged as progressive' unchecked (unless it means the opposite of what I'm thinking).

http://stfcc.org/misc/deinterlacing.ffdshow.off.png
http://stfcc.org/misc/deinterlacing.ffdshow.on.png
And just for kicks, http://stfcc.org/misc/deinterlacing.mpc-hc.png using DXVA with hardware deinterlacing.

[haruhiko_yamagata]

H264 can already have large unpredictable frame delay (up to 16 frames), but frame threading makes it worse. Basically you should use reordered_opaque, and then pass in future frames until you get back the one that was being seeked to
In DirectShow, decoders receive compressed frame from the splitter, find the entry point (ffdshow searches IDR or GDR by decoding the SEI by custom code) and start decoding, start presenting where the timestamp given by the splitter is bigger than 0.
It's working correctly except the garbled image I get when I seek in premiere-paff.ts.
And ffdshow use reordered_opaque to reorder timestamps.
I can't reproduce what CruNcher has said.

(and then NULL frames at the end of the file).
ffmpeg-mt branch of ffdshow does this correctly. I know I should have applied it to our trunk, but I've been to lazy.

If you don't have future frames available, then you're stuck - this is the API issue I have to work out.
I have not experienced this issue.

//EDIT
The seeking is not working perfectly in the trunk build. Though it is more noticeable in the mt build, it may not related to multithreading.

[haruhiko_yamagata]

though stability compared to commercial decoder seems still low i sometimes lose up to 4 frames of the original source input (beginning,end) :( in different rendering scenarios (AVC,ASP,Mpeg-2), also frame exact seeking is still bad too. Using the same Splitter Haali's for all of them ffdshow always loses frames or endsup @ the wrong frame when for example requesting a b-frame compared to CoreAVC,DivX,Elecard,Cyberlink,Arcsoft,Nero :( I lost hope that the Libavcodec guys gonna ever improve that but i wonder if it can't be done via a workaround over dshow forceing it somehow.Sorry for late reply.
Please read the post above.
Please separate AVC,ASP and MPEG-2.
What do you mean by "frame exact seeking"?

I know seeking is broken for libavcodec-MPEG2.
Perhaps we have to give up using CODEC_FLAG_TRUNCATED.
Doesn't it work for AVC (except when we get garbled image)?

Frames lost at the beginning? Could you send us some samples?

As for the last frame, ffmeg-mt branch has the code to handle it properly. I'll port it to trunk when I have time.

[haruhiko_yamagata]

Heh, another issue with my infamous 1080i25 video (same as the link above). After being about 1:30:00 into it, the current frame # is up in the upper 500000s. It seems that when I check Grab and have it save every frame, the frame number increments by 5 each time.

Also, although this is probably a libavcodec issue, I didn't know where else to bring it up. That 1080i25 stream is PAFF if I remember correctly. Parts of it are interlaced, parts aren't. The parts that are interlaced are deinterlaced fine when deinterlacing is turned on. However, the parts that aren't interlaced are still deinterlaced with deinterlacing turned on, regardless of having 'Process frames flagged as progressive' unchecked (unless it means the opposite of what I'm thinking).

http://stfcc.org/misc/deinterlacing.ffdshow.off.png
http://stfcc.org/misc/deinterlacing.ffdshow.on.png
And just for kicks, http://stfcc.org/misc/deinterlacing.mpc-hc.png using DXVA with hardware deinterlacing.
The OSD issue is basically same as the 108fps problem. Low priority but should be fixed someday.
As for the interlacing flags, are you sure that the stream has correct flags?

[haruhiko_yamagata]

@clsid
would it be possible to support Elecards Mpeg and Mp4 Demultiplexers :) watching RAW bitstreams (Annex B Progressive AVC for example) with ffdshow with them isn't really nice CoreAVC, Cyberlink and DivX are interoperable with them ffdshow, MPCs Video Decoder and Arcsoft have problems (garbaged output,black screen, no connection) :(If you provide me a set of files, I'm ready to work on it.

[CruNcher]

If you provide me a set of files, I'm ready to work on it.

http://elecard.ru/products/products-pc/sdk/codec-sdk/

emp4demux.ax
empgdmx.ax

you can just use any simple AVC bitstream and for MP4 any .mp4 (mp4muxer,mp4box) including ASP or AVC getting as much formats to work with ffdshow is the goal .mpg(vob,evo) (Mpeg-1,Mpeg-2,AVC) .ts (Mpeg-2,AVC) ect empgdmx virtually supports everything ISO specified + Blu-Ray :)

[fastplayer]

Can somebody with a high-res TFT (1680x1050 and higher) check out the new ffdshow homepage? Is the font size too small?
http://ffdshow-tryout.sourceforge.net/index2.php

[haruhiko_yamagata]

http://elecard.ru/products/products-pc/sdk/codec-sdk/

emp4demux.ax
empgdmx.ax

you can just use any simple AVC bitstream and for MP4 any .mp4 (mp4muxer,mp4box) including ASP or AVC getting as much formats to work with ffdshow is the goal .mpg(vob,evo) (Mpeg-1,Mpeg-2,AVC) .ts (Mpeg-2,AVC) ect empgdmx virtually supports everything ISO specified + Blu-Ray :)
$299...
If anyone is willing to donate a license.

[thuan]

Well I'm on 1680x1050 and the page looks just fine on my monitor I think the font is 10 or around that. Pretty standard. If you feel it's too small just use zooming feature of your browser.

[fastplayer]

$299...
If anyone is willing to donate a license.
Not if you enter the discount code :p
Then it's only $209.30! :D

[XhmikosR]

Can somebody with a high-res TFT (1680x1050 and higher) check out the new ffdshow homepage? Is the font size too small?
http://ffdshow-tryout.sourceforge.net/index2.php

With 1920x1200, it looks normal.:)

[fastplayer]

Well I'm on 1680x1050 and the page looks just fine on my monitor I think the font is 10 or around that. Pretty standard. If you feel it's too small just use zooming feature of your browser.
With 1920x1200, it looks normal.:)
Thanks for confirmation, guys! As long as it's readable and you don't need to zoom, I'm OK with it.

Any chance you can PM me or post here a screenshot? I'm still tinkering with the floats and padding...

[XhmikosR]

http://img210.imageshack.us/img210/3586/19122008134124cb7.th.png (http://img210.imageshack.us/my.php?image=19122008134124cb7.png)

[fastplayer]

http://img210.imageshack.us/img210/3586/19122008134124cb7.th.png (http://img210.imageshack.us/my.php?image=19122008134124cb7.png)
Thanks! Looks better than expected :)
By the way, don't browse the page with IE6... :D

[XhmikosR]

I don't use IE6, only Firefox.:)

[fastplayer]

I don't use IE6, only Firefox.:)
I wasn't implying that you were. It's just that the page renders correctly in all browsers that deserve to be called like that. All - except IE6... :D

[STaRGaZeR]

Right, I meant that one. It's fixed in the git repository now.

And in ffdshow-MT now too. Thanks :)

The only issues I have with the MT branch are:

- The known garbled image when seeking to certain points.
- ffdshow crashes randomly when seeking in consecutive steps, in my case 5s. I can't reproduce it when seeking using MPC's slider, probably because I can't hit problematic points with its precision. Also this only happens with unknown encoders (i.e. Blu-ray), x264 streams seems to be perfectly fine regarding this issue.

[Kurtnoise]

Can somebody with a high-res TFT (1680x1050 and higher) check out the new ffdshow homepage? Is the font size too small?
http://ffdshow-tryout.sourceforge.net/index2.php
looks ok for me too.


btw, you should add also a link to the FFmpeg website (http://ffmpeg.org)...:)



just my 2€ cents...

[fastplayer]

btw, you should add also a link to the FFmpeg website (http://ffmpeg.org)...:)
Thanks!
There's a link to FFmpeg right on the start page in the 2nd paragraph. I can add it to the link section too, of course.

[Kurtnoise]

you're right...did'nt see it. :p

@haruhiko :

http://pix.nofrag.com/e/3/6/7648861117e32a357206d3e0c1d91.png (http://pix.nofrag.com/e/3/6/7648861117e32a357206d3e0c1d91.html)

[nautilus7]

As long as it's readable and you don't need to zoom, I'm OK with it.Well, it's readable, but i would like it a bit bigger. I think it is a click smaller than normal. I am on 1680x1050 20" monitor (maybe on 21"-22" there's no problem).

[STaRGaZeR]

Sample for testing double speed playback with libavcodec MP3. So far this is the only one I have that has this problem. It doesn't matter the container, you can also demux it.

http://www.megaupload.com/?d=OA592VG2 (29MB)

EDIT: Oops I didn't see your last commit Haruhiko, sorry.

[LoRd_MuldeR]

Well, that is a regression.

I had no time to work on it during school, but now I can again, and I think I found the problem with that sample (some problem with Picture.ref_index). I'll try to fix it soon.

Even after I fix everything in the todo, there are some API and philosophical issues to work out before submitting it to mainline again, though.

edit: actually, it was because the dequant matrix changes in the middle of the stream

Thank you for the info :thanks:

And I can confirm that the "freedom EP1" sample now plays flawlessly with ffdshow-MT r2515 :)

[STaRGaZeR]

The only issues I have with the MT branch are:

- The known garbled image when seeking to certain points.
- ffdshow crashes randomly when seeking in consecutive steps, in my case 5s. I can't reproduce it when seeking using MPC's slider, probably because I can't hit problematic points with its precision. Also this only happens with unknown encoders (i.e. Blu-ray), x264 streams seems to be perfectly fine regarding this issue.

OK I've been able to find a sample that has both problems. I can always reproduce them at the same frame:

- With Haali's Splitter:

When seeking to frame 160 --> Garbled frames.
When seeking to frame 163 --> Video stops, and if you do anything else with the player it hangs and you have to kill it with task manager. If only 1 thread is used then it does something strange but doesn't hang.

- With MPC's internal Matroska splitter:

The same but seeking to frames 159 and 167 respectively.

You can use Ctrl+G in MPC to go directly to those frames.

http://www.megaupload.com/?d=E8X3WSST (55MB)

[clsid]

Release candidate for the official beta 6:
download (http://downloads.sourceforge.net/ffdshow-tryout/ffdshow_prebeta6_rev2527_20081219.exe?use_mirror=)
Please test. If no major issues are reported, then this build will become beta6 tomorrow.

This build includes several H.264 related changes that were made in the past few days. Decoding performance has improved with 4-10% (depending on CPU) :)

[clsid]

A few people have reported an issue with the x64 builds of ffdshow where ffdshow uses an unusually high amount of CPU cycles, resulting in bad playback (stuttering, frame drops). This issue is present for a long time now, so not related to any recent changes.

Weird thing is that the CPU usage returns to normal when the OSD is enabled in ffdshow video decoder.

Does anyone have an idea what might cause this problem, and why the OSD makes a difference?

I myself am unable to reproduce the issue on a clean install of Vista x64.

[LoRd_MuldeR]

This build includes several H.264 related changes that were made in the past few days. Decoding performance has improved with 4-10% (depending on CPU) :)

Do these changes only apply to the "normal" branch or do they effect the MT branch to?

In the latter case I'd need to do new benchmarks :cool:

[LoRd_MuldeR]

Weird thing is that the CPU usage returns to normal when the OSD is enabled in ffdshow video decoder.

Got a similar problem with recent builds (normal and MT, but not x64) when playing interlaced MPEG-2 and using Yadif.
With Yadif enabled (internal version -or- Avisynth version) I get "slow motion" playback, which disappears when the OSD is enabled!
Had a lot of private discussion with haruhiko_yamagata about that problem already, but no solution yet.

It came down to the following result:
* The time ffdshow spends in Yadif is always okay.
* When the slow motion happens, then ffdshow spends a very long time (much too long) in the "convert" function!
* The "convert" time is back to normal with OSD enabled.

Note that there was no such problem in rev2347, seems it started around rev2391 ...

[fastplayer]

Do these changes only apply to the "normal" branch or do they effect the MT branch to?
Both.
In the latter case I'd need to do new benchmarks :cool:
Oh, you shouldn't have said that :D
We desperately need someone who is willing to test one of the pre-Beta 6 builds for our FAQ. The current FAQ entry (http://ffdshow-tryout.wiki.sourceforge.net/FAQ#perf_filters) is based on build 802... :eek:

[LoRd_MuldeR]

Some new numbers:

E:\HD\freedom EP1 sample.mkv, 1920x1080, High@L4.1

[ffdshow, rev2033, Beta-5, 4 threads]
User: 3s, kernel: 0s, total: 4s, real: 22s, fps: 153.4, dfps: 27.1
User: 3s, kernel: 0s, total: 4s, real: 22s, fps: 148.8, dfps: 27.0 <-- 100%
User: 3s, kernel: 0s, total: 4s, real: 25s, fps: 143.4, dfps: 24.1

[ffdshow, rev2509, 4 threads]
User: 3s, kernel: 0s, total: 4s, real: 22s, fps: 142.8, dfps: 27.2
User: 3s, kernel: 0s, total: 4s, real: 22s, fps: 142.8, dfps: 27.2 <-- 100%
User: 3s, kernel: 0s, total: 4s, real: 25s, fps: 140.8, dfps: 24.2

[ffdshow, rev2527, Pre-Beta 6, 4 threads]
User: 3s, kernel: 0s, total: 4s, real: 22s, fps: 149.3, dfps: 27.5
User: 3s, kernel: 0s, total: 3s, real: 22s, fps: 155.2, dfps: 27.4 <-- 101%
User: 3s, kernel: 0s, total: 4s, real: 24s, fps: 147.1, dfps: 24.9

[ffdshow-MT, rev2515, 4 threads]
User: 2s, kernel: 0s, total: 2s, real: 8s, fps: 219.0, dfps: 68.7
User: 2s, kernel: 0s, total: 2s, real: 8s, fps: 252.7, dfps: 68.7 <-- 254%
User: 2s, kernel: 0s, total: 2s, real: 9s, fps: 220.2, dfps: 67.2

[CoreAVC, Version 1.8.5]
User: 0s, kernel: 0s, total: 0s, real: 7s, fps: 679.7, dfps: 82.3
User: 0s, kernel: 0s, total: 1s, real: 7s, fps: 616.0, dfps: 82.1 <-- 304%
User: 0s, kernel: 0s, total: 0s, real: 7s, fps: 691.6, dfps: 82.0

[DivX H.264 Decoder, Beta-3]
User: 1s, kernel: 0s, total: 1s, real: 6s, fps: 458.4, dfps: 88.4
User: 1s, kernel: 0s, total: 1s, real: 6s, fps: 512.0, dfps: 88.4 <-- 327%
User: 1s, kernel: 0s, total: 1s, real: 7s, fps: 499.0, dfps: 87.6


Can't say that the "normal" branch of ffdshow became significant faster between Beta-5 (r2033) and Beta-6 (r2527) on my system...

[leeperry]

We desperately need someone who is willing to test one of the pre-Beta 6 builds for our FAQ. The current FAQ entry (http://ffdshow-tryout.wiki.sourceforge.net/FAQ#perf_filters) is based on build 802... :eek:
if you can provide the generic/GCC/ICL 10.1 SSE/ICL 10.1 SSE2 builds of the same rev, I can run compares on h264 HD decoding + sharpening in RGB32HQ on an o/c Q6600(I'm also a Remoulade beta tester)