ffdshow tryouts project: Discussion & Development

[tal.aloni]

ok, I'll check some low-res files when I get home,
Thanks,
Tal

[HeadlessCow]

Core i7 920
Radeon 4670

Format                           : AVC
Format/Info                      : Advanced Video Codec
Format profile                   : High@L4.1
Format settings, CABAC           : Yes
Format settings, ReFrames        : 4 frames
Codec ID                         : avc1
Codec ID/Info                    : Advanced Video Coding
Duration                         : 45mn 29s
Bit rate mode                    : Variable
Bit rate                         : 2 569 Kbps
Maximum bit rate                 : 11.1 Mbps
Width                            : 1 280 pixels
Height                           : 720 pixels
Display aspect ratio             : 16:9
Frame rate mode                  : Constant
Frame rate                       : 23.976 fps
Resolution                       : 24 bits
Colorimetry                      : 4:2:0
Scan type                        : Progressive
Bits/(Pixel*Frame)               : 0.116
Stream size                      : 836 MiB (100%)
Writing library                  : x264 core 68 r1183M f21daff
Encoding settings                : cabac=1 / ref=4 / deblock=1:-1:-1 / analyse=0x3:0x113 / me=umh / subme=9 / psy_rd=1.0:0.0 / mixed_ref=1 / me_range=32 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=12 / nr=0 / decimate=1 / mbaff=0 / bframes=3 / b_pyramid=0 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / keyint=250 / keyint_min=25 / scenecut=40 / rc=2pass / bitrate=2569 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=50000 / vbv_bufsize=50000 / ip_ratio=1.40 / pb_ratio=1.30 / aq=1:1.00

Overlay works fine at 24fps.
Full postprocessing drops it to ~18fps and the color is off.

Skipping around in the file at all crashes MPC-HC.

[HeadlessCow]

Core i7 920
Radeon 4670

Format                           : AVC
Format/Info                      : Advanced Video Codec
Format profile                   : High@L3.1
Format settings, CABAC           : Yes
Format settings, ReFrames        : 4 frames
Codec ID                         : avc1
Codec ID/Info                    : Advanced Video Coding
Duration                         : 24mn 10s
Bit rate mode                    : Variable
Bit rate                         : 1 151 Kbps
Maximum bit rate                 : 3 059 Kbps
Width                            : 640 pixels
Height                           : 480 pixels
Display aspect ratio             : 4:3
Frame rate mode                  : Constant
Frame rate                       : 23.976 fps
Resolution                       : 24 bits
Colorimetry                      : 4:2:0
Scan type                        : Progressive
Bits/(Pixel*Frame)               : 0.156
Stream size                      : 199 MiB (100%)
Writing library                  : x264 core 80 r1376M 3feaec2
Encoding settings                : cabac=1 / ref=4 / deblock=1:-1:-1 / analyse=0x3:0x113 / me=umh / subme=7 / psy=1 / psy_rd=0.4:0.0 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / sliced_threads=0 / nr=0 / decimate=1 / mbaff=0 / constrained_intra=0 / bframes=5 / b_pyramid=0 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / wpredp=2 / keyint=250 / keyint_min=25 / scenecut=40 / rc_lookahead=40 / rc=2pass / mbtree=1 / bitrate=1151 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=17500 / vbv_bufsize=14000 / ip_ratio=1.40 / aq=1:0.60

Overlay is perfect.
Full postprocessing keeps up with the framerate but looks like this:

[tal.aloni]

Thanks!
I gues I should focus on "merge" for the time being,
It would be perfect for subtitles / OSD / Bitmap overlay.

I'm still wondering why we couldn't get more speed out of the core i7... the intel docs have demonstrated sufficient theoretical speed.
maybe albain could figure it out / optimize.
(perhaps it's not related to speed at at.. maybe the last example tied to colorspace issues)

Tal

[albain]

It may be due to the colorspace.

I can't test it right now but in my (non working) patch I added 2 conversions functions in src/imgFilters/ffImgfmt.h.

And I called csp_planar2packed(FF_CSP_NV12) which gives FF_CSP_NV12|FF_CSP_FLAGS_YUV_ADJ|FF_CSP_FLAGS_YUV_ORDER. This is (I think) the value that we should use when initializing TffPict

static __inline int csp_planar2packed(int csp)
{
 switch (csp)
 {
   case FF_CSP_YUY2:return FF_CSP_YUY2;break;
   case FF_CSP_YVYU:return FF_CSP_YVYU;break;
   case FF_CSP_UYVY:return FF_CSP_UYVY;break;
   case FF_CSP_VYUY:return FF_CSP_VYUY;break;
   case FF_CSP_CLJR:return FF_CSP_CLJR;break;
   case FF_CSP_444P:return FF_CSP_444P;break;
   case FF_CSP_422P:return FF_CSP_422P;break;
   case FF_CSP_411P:return FF_CSP_411P;break;
   case FF_CSP_410P:return FF_CSP_410P;break;
   case FF_CSP_Y800:return FF_CSP_Y800;break;
   case FF_CSP_NV12:return FF_CSP_NV12|FF_CSP_FLAGS_YUV_ADJ|FF_CSP_FLAGS_YUV_ORDER;break;
   case FF_CSP_PAL8:return FF_CSP_PAL8|FF_CSP_FLAGS_VFLIP;break;
 }
}

static __inline int csp_packed2planar(int csp)
{
 switch (csp)
 {
   case FF_CSP_YUY2:return FF_CSP_YUY2;break;
   case FF_CSP_YVYU:return FF_CSP_YVYU;break;
   case FF_CSP_UYVY:return FF_CSP_UYVY;break;
   case FF_CSP_VYUY:return FF_CSP_VYUY;break;
   case FF_CSP_CLJR:return FF_CSP_CLJR;break;
   case FF_CSP_444P:return FF_CSP_444P;break;
   case FF_CSP_422P:return FF_CSP_422P;break;
   case FF_CSP_411P:return FF_CSP_411P;break;
   case FF_CSP_410P:return FF_CSP_410P;break;
   case FF_CSP_Y800:return FF_CSP_Y800;break;
   case FF_CSP_NV12|FF_CSP_FLAGS_YUV_ADJ|FF_CSP_FLAGS_YUV_ORDER:return FF_CSP_NV12;break;
   case FF_CSP_PAL8|FF_CSP_FLAGS_VFLIP:return FF_CSP_PAL8;break;
 }
}

EDIT : this code has been inspired from TvideoCodecUncompressed::beginDecompress code where FFDShow receives packed colorspaces as in DXVA.
In the method the packed colorspace is obtained from the input format, including :

case CODEC_ID_NV12:csp=FF_CSP_NV12|FF_CSP_FLAGS_YUV_ADJ|FF_CSP_FLAGS_YUV_ORDER;break;

[tal.aloni]

I found a bug in full processing mode.

pStore += pitch - width;

(should be divided by 16)

I'll keep working on it in a few hours.

[albain]

Hi Tal,

back home. I am building it with the fix on 16 division

I'll post results in a moment

[tal.aloni]

another bug on full post processing (SSE 2 and 4.1):
some random crashes results from addresses not aligned to 16 bytes. _mm_stream_si128 will crash.

[albain]

Okay, the video is very slow with compression artifacts (not green lines) and I have a 50% CPU charge on my Q9450 (2.66Ghz).

In overlay mode I have 22% CPU

I saw that you made some colorspaces conversions : there is a converter (with optimized CPU instructions). I don't know if it can handle the source & dest colorspace that you want

[clsid]

The FFmpeg devs are working on further improving their H.264 code. Their latest patches are not compatible with the slice based multi-threaded decoding patch that we use in ffdshow's libavcodec.

Here is a patch with the latest changes:
http://pastebin.com/m11865032

Should I remove the slice based multi-threading patch? Or is there anyone willing/able to fix it?

Edit: new patch

[albain]

You are already using optimized code. I don't know what else could be done to improve the perfs. The big problem comes from the GPU to system memory copy


@clsid : this is okay, we are not affected by the patch in DXVA mode (if your question was addressed to us)

[tal.aloni]

Originally Posted by albain

Okay, the video is very slow with compression artifacts (not green lines) and I have a 50% CPU charge on my Q9450 (2.66Ghz).

also, there are still a few issues with the full processing methods (add support when input buffer pointer not aligned to 64, support of inputs with width which is not mod 64)

I wonder if it's worth the effort (full processing). (I'm also wondering if executing the copy in a different stage of the decoding would give us more time)

[THX-UltraII]

The ATI HD5670 was released a few days ago and is already in stock here in the Netherlands. Will this card give any problems with Bitstream support?

[rsd78]

Originally Posted by albain

Okay, the video is very slow with compression artifacts (not green lines) and I have a 50% CPU charge on my Q9450 (2.66Ghz).

In overlay mode I have 22% CPU

I saw that you made some colorspaces conversions : there is a converter (with optimized CPU instructions). I don't know if it can handle the source & dest colorspace that you want

My hunch is that it may not be worth while performance wise for the full performance method.

I'm a little surprised that in overlay mode it still took 22% CPU, albain. How does that compare to software decoding mode for the same file?

[albain]

Originally Posted by tal.aloni

also, there are still a few issues with the full processing methods (add support when input buffer pointer not aligned to 64, support of inputs with width which is not mod 64)

I wonder if it's worth the effort (full processing). (I'm also wondering if executing the copy in a different stage of the decoding would give us more time)

This may be multithreaded with boost as it seems that there are several frames in the buffer when we have to display them, but this is a huge amount of work

[albain]

With software mode only (no DXVA) I have 30-40% of CPU charge

[rsd78]

Originally Posted by albain

With software mode only (no DXVA) I have 30-40% of CPU charge

Thanks Albain. What is the rough cpu usage using ffdshow's dxva but without any overlay? Just to get an idea of the comparative differences we are talking about here. Thanks again for all you and the others have done. Between bitstreaming, and dxva + subtitles I'm in heaven

[albain]

Originally Posted by rsd78

Thanks Albain. What is the rough cpu usage using ffdshow's dxva but without any overlay? Just to get an idea of the comparative differences we are talking about here. Thanks again for all you and the others have done. Between bitstreaming, and dxva + subtitles I'm in heaven

7%

But this is normal as there is no one-way and return to the GPU.

Also there are in overlay mode colorspaces conversions

[mark0077]

Can anyone confirm de-interlacing with yadif + double frame rate, no longer doubles frame rate. I have forced de-interlacing with many types of content, even non interlaced. I cannot get ffdshow to double the frame rate anymore. It does de-interlace content fine so it is being enabled, it just leaves the frame rate as is.

[Mr VacBob]

Originally Posted by clsid

The FFmpeg devs are working on further improving their H.264 code. Their latest patches are not compatible with the slice based multi-threaded decoding patch that we use in ffdshow's libavcodec.

Here is a patch with the latest changes:
http://pastebin.com/m11865032

Should I remove the slice based multi-threading patch? Or is there anyone willing/able to fix it?

Edit: new patch

If you mean frame, I'll fix it tomorrow - slice threading was broken but fixed again in ffmpeg. Splitting the decoder files completely confused git, so I'll have to do a quite large merge by hand. Meant to do it over the weekend but people kept asking me to do other stuff…