VC-1 deblocking doesn't work with DXVA on

[Jay Bee]

I've recently been playing around a bit with the vc-1 encoder and was very happy with the results so far. Except now I have hit quite a show stopper which is that if DXVA is in use during playback, deblocking is not performed :(

Using zambelli's PowerToy to force deblocking doesn't make a difference (except that it messes up the field order but that's for another discussion). Switching off DXVA is the only way to get the deblocking to kick in but this isn't an option for me because I need to run in DXVA mode for the advanced deinterlacing.

I have confirmed this problem on two seperate machines with two seperate PCIE GPU's (6200 and 7950 GT). Is this a known issue?

Here's the script, a sample encode (ultra low bitrate to demonstrate the effect) and two screenshots comparing DXVA on and off.:

cscript wmcmd.vbs -input input.avs -output output.wmv -v_codec wvc1 -v_mode 0 -v_preproc 17 -v_type 1 -v_bitrate 300000 -pixelratio 20 11 -v_loopfilter 1

http://rapidshare.com/files/16154814/clip.wmv.html

http://img6.picsplace.to/img6/25/thumbs/dxva_on_000.png (http://img6.picsplace.to/img.php?file=img6/25/dxva_on_000.png)

http://img6.picsplace.to/img6/25/thumbs/dxva_off_000.png (http://img6.picsplace.to/img.php?file=img6/25/dxva_off_000.png)

Cheers

[Jay Bee]

The more things I try the weirder it gets:

After some more trials I have found a way to playback vc-1 encodes with both deinterlacing and deblocking working:
don't use a player that isn't WMP11 and don't use a bitrate higher than 999999 (bps) when encoding. For some reason setting a higher bitrate breaks deblocking and aspect ratio support on playback in WMP. :confused:

If you do use a higher bitrate you can still get playback to work properly by going to the advanced settings of WMP11 and deactivating DXVA (what's the default setting for this?) and switching on "high quality mode". This only works for WMP though.

So basically for bitrates > 999999 bps:
no AR support or deblocking if overlay mixer and dxva are enabled
AR works but no deblocking if overlay mixer enabled and dxva disabled
AR and deblocking work if high quality mode is enabled and dxva is disabled
For other players: no way to get advanced deinterlacing and deblocking at the same time


So I guess the only way to fix the playback issues without using WMP would be for player devs to rewrite their software using the Windows Media SDK instead of DirectShow?

But what about the strange bitrate problem? If that could be fixed, the files would at least play properly on every default version of WMP11, which at the moment isn't the case.

[zambelli]

I have confirmed this problem on two seperate machines with two seperate PCIE GPU's (6200 and 7950 GT). Is this a known issue?
Sort of. The postprocessing step needs to be implemented by the video card maker with the rest of the DXVA support. Once the WMV decoder hands off the video samples to the DXVA driver for processing, it's gone and never comes back to the WMV decoder - so any kind of postprocessing, be it deblocking or range expansion, needs to be done by the GPU. I believe the decoder signals the need for deblocking+deringing through the DXVA API, but it's up to the video driver whether they want to respect that or not.

So basically for bitrates > 999999 bps:
no AR support or deblocking if overlay mixer and dxva are enabled
AR works but no deblocking if overlay mixer enabled and dxva disabled
AR and deblocking work if high quality mode is enabled and dxva is disabled
For other players: no way to get advanced deinterlacing and deblocking at the same time
I had to think a while to remember what's going on there, but then I remembered: when DXVA support was added to WMP/WMFSDK in v10, it was defined to only kick in for content that truly needed it. The line was drawn at 1 Mbps. In other words, content under 1 Mbps never plays back through DXVA in WMP - regardless of the DXVA setting. Another interesting limitation of the WMFSDK pipeline is that no video content with framerate > 30 fps will play through DXVA.

Neither of these restrictions exist in DirectShow or Media Foundation (Vista WMP), so I think what you're seeing is consistent with your previous finding: the video driver isn't performing deblocking when DXVA is used.

In general, I'd avoid overlays and always use either VMR7 Mixing Mode ("High Quality mode") or WMR9 in DShow. This will ensure correct AR signaling (if the decoder supports it) in the renderer and highest quality deinterlacing. In fact, I don't think DXVA deinterlacing and DXVA decoding are dependant on each other. I'm pretty sure you can get DXVA deinterlacing without DXVA decoding. Using "High quality mode" in WMP should enable this in WMP - the hardware bobber should kick in for any interlaced WMV content, and you can verify this by checking the framerate in Statistics - it should be doubled.

As we continue down that road...

Unfortunately, a separate limitation in the XP WMV decoder restricts external deinterlacing to WMP/WMFSDK playback path only. Renderer deinterlacing will not work in DirectShow under XP, due to the decoder not exposing VideoInfoHeader2 (needed for interlaced signaling) for compatibility reasons. The restriction does not exist in Vista at all. In other words, if you want good deinterlacing for WMV content in XP - you have to use WMP. We fought a hard battle to get the VideoInfoHeader2 support working correctly in XP, but it didn't make the final cut. :(

[zambelli]

I realize that was a lot of information in the last post, so let me try to break it down in a more helpful way:

DXVA WMV decoding support:

XP - DirectShow - DXVA 1.0 - unrestricted
XP - WMP (WMFSDK) - DXVA 1.0 - minimum bitrate 1Mbps, maximum framerate 30 fps - "High quality" mode recommended
Vista - DirectShow - DXVA 1.0 - only in VMR7 and VMR9, but not EVR
Vista - WMP (MF) - DXVA 2.0 - unrestricted, uses EVR
Vista - WMP (WMFSDK) - unsupported


WMV AP external deinterlacing support:
("external" implies deinterlacing in the renderer - but could be software or hardware, depending on video driver support)

XP - DirectShow - supported only when DXVA decoding is used, otherwise just passes through as is
XP - WMP (WMFSDK) - supported with and without DXVA decoding - "High quality" mode recommended
Vista - DirectShow - supported with and without DXVA decoding
Vista - WMP (MF) - supported with and without DXVA decoding
Vista - WMP (WMFSDK) - supported


The list assumes latest WMP (v11) is installed in all cases! Prior to v11, the WMV decoder really lacked good support for DXVA decoding and interlaced content, so anybody wanting those two features in XP should really consider upgrading their WMP or at least their WMFSDK runtime.

[Jay Bee]

Many thanks, that's exactly the kind of reply I was hoping for. :goodpost: It all makes sense now.

So basically what it boils down to is that Nvidia are messing up again (surprise, surprise). They say that they can use their hardware to decode and deinterlace vc-1 files but omit the deblocking although it is obviously part of the decoding process and not an optional postprocessing step. If it was optional there would be no point of having a -v_loopfilter option in the first place. I think it is also clear now that Cyberlink aren't the ones to blame for not using deblocking on their AVC decoder. How do Nvidia get away with this stuff all the time?

So what we can also conclude is that there is absolutely nothing wrong on the Microsoft side of things. If Nvidia would decode to codec spec then all would be fine. Microsoft even makes it possible to workaround the dxva problems (by disabling it) albeit with the limitation that deinterlacing will only work with WMP in XP. And in Vista, simply disabling dxva will take care of everything and let even directshow players work as they are supposed to. Maybe Microsoft should default to "dxva off" if the hardware companies can't actually follow the spec?

It's a shame that the bottom line is still that if you wan't to use a newer codec than MPEG-2 for interlaced content on a PC you will face some kind of playback/configuration problems (in 2007!).

Personally I think I will be using vc-1 for my TV encodes in future hoping that these issues will go away sometime in the future.

P.S. does anyone have information about the ATI side of things. Do they dxva decode vc-1? Do they deblock vc-1? Do they deblock AVC?

Cheers

[zambelli]

If it was optional there would be no point of having a -v_loopfilter option in the first place.
Actually, in-loop filtering is a different process than postprocess deblocking. In-loop filtering happens on both the encode and decode side. Anchor frames get deblocked during encoding, preventing certain artifacts from propagating on to derived frames. The process is then repeated during decoding. Classic deblocking & deringing is entirely a postprocess that can be applied on top of in-loop filtering.

If Nvidia would decode to codec spec then all would be fine.
That's been 80% of the struggle to get DXVA decoding working correctly - getting hardware vendors to correctly implement the spec. In their defense, they often have multiple standards to implement (MPEG-2, H.264, VC-1) and a limited number of codec experienced people to do it. DXVA decoding is a good idea, but from an implementation perspective it's risky business because it shifts the responsibility of implementing decoding from one party to two parties.
Microsoft even makes it possible to workaround the dxva problems (by disabling it) albeit with the limitation that deinterlacing will only work with WMP in XP. And in Vista, simply disabling dxva will take care of everything and let even directshow players work as they are supposed to. Maybe Microsoft should default to "dxva off" if the hardware companies can't actually follow the spec?
Well, that's exactly why MS has been slow to turn on DXVA WMV decoding in the player. The functionality was actually shipped in WMP10 (2004), but it took another QFE before it was actually made useful. It was massively improved in WMP11, but I think WMP11 for XP still shipped with DXVA WMV disabled by default. In Vista it's enabled by default for MF, but disabled by default for DirectShow WMV decoding.

It's a shame that the bottom line is still that if you wan't to use a newer codec than MPEG-2 for interlaced content on a PC you will face some kind of playback/configuration problems (in 2007!).
Granted, it took MPEG-2 DXVA decoding a while to get stable. It'll probably be another year at least before H.264 and VC-1 DXVA decoding is on the same level.

[zambelli]

P.S. does anyone have information about the ATI side of things. Do they dxva decode vc-1? Do they deblock vc-1?
They only support WMV9 (Simple/Main) at this time, and I think they might've even disabled it in Vista drivers. I don't know when Advanced Profile support is expected.

[zambelli]

Hi Jay Bee,

I ran a test of deblocking on my Nvidia 6600GT card. XP Pro, WMP11, Forceware 93.81 drivers. I tested on a 640x480 1080 kbps WMV9 (Main) video. I played each video through the following modes and took a screenshot of the same frame every time:

DXVA On, Post Processing = 0 (Off)
DXVA Off, Post Processing = 2 (Strong Deblocking)
DXVA On, Post Processing = 2 (Strong Deblocking)

I found that the first case had the cleanest picture. The 2nd case had noticeable deblocking done, and the 3rd case had even stronger deblocking done. Therefore, it would seem that Nvidia is actually performing deblocking on DXVA decoded WMV9.

I didn't try WVC1 video, but before I do, could you try performing a similar test on your machine? I recommend you pick a frame that has visible macroblocking in the source and framestep to it in WMP every time, then take a screenshot, save the image, and compare the images when you're done.

[Jay Bee]

Hi,
there's a way to framstep in WMP? How? That would make comparing videos a lot easier, I only know how to do this with ZP.

Anyway I did do quite similar tests using a self-made PAL TV clip (720 x 576i 2000 kbps).

I'll have a quick go at your proposed test on my test clip (you can download a short sample here (http://rapidshare.com/files/16859454/vc1_576i_2000kbps.wmv.html)):

DXVA On, Post Processing = 0 (Off):
http://img6.picsplace.to/img6/25/thumbs/1_047.png (http://img6.picsplace.to/img.php?file=img6/25/1_047.png)

DXVA Off, Post Processing = 2 (Strong Deblocking):
http://img6.picsplace.to/img6/25/thumbs/2_034.png (http://img6.picsplace.to/img.php?file=img6/25/2_034.png)

DXVA On, Post Processing = 2 (Strong Deblocking):
http://img6.picsplace.to/img6/25/thumbs/3_030.png (http://img6.picsplace.to/img.php?file=img6/25/3_030.png)

I didn't get the exact same frames but I think it's still clear to see that case 2 is the only one where deblocking is performed.

All tests were done in "high quality mode". And I always set dxva on/off in both your PowerToy and in WMP's settings as dxva decoding only kicks in when both are enabled, switching it on in PowerToy doesn't suffice. Maybe this could be reason for your above results? Or maybe it's because of vc-1 or interlacing?

Hardware is 6200 and 7950 GT on 93.81 drivers.

[Inventive Software]

That the second Hungary GP2 race Jay Bee? :D

DXVA Off is also the only case, purely judging by the screenshots you've taken, where either the AR is set, or it fills your screen.

[zambelli]

there's a way to framstep in WMP? How? That would make comparing videos a lot easier, I only know how to do this with ZP.
Select View --> Enhancements --> Play Speed Settings. You'll find two Frame Prev and Next buttons in that pane.

Anyway I did do quite similar tests using a self-made PAL TV clip (720 x 576i 2000 kbps).
Ah, I used progressive and Main Profile. Before we deal with interlaced, let's make sure Progressive works. Can you try on a plain WMV9 clip?

[Jay Bee]

That the second Hungary GP2 race Jay Bee? :D

Race 1 actually. But you recognized the track! :) They showed 6 minutes overall of onboard from Hamiltons car in that race, it's a great clip.

New tests for zambelli.

1) wmv9 MP progressive:

DXVA On, Post Processing = 0 (Off):
http://img6.picsplace.to/img6/25/thumbs/1_045.png (http://img6.picsplace.to/img.php?file=img6/25/1_045.png)

DXVA Off, Post Processing = 2 (Strong Deblocking):
http://img6.picsplace.to/img6/25/thumbs/2_032.png (http://img6.picsplace.to/img.php?file=img6/25/2_032.png)

DXVA On, Post Processing = 2 (Strong Deblocking):
http://img6.picsplace.to/img6/25/thumbs/3_028.png (http://img6.picsplace.to/img.php?file=img6/25/3_028.png)

2) wvc1 progressive:

DXVA On, Post Processing = 0 (Off):
http://img6.picsplace.to/img6/25/thumbs/1_046.png (http://img6.picsplace.to/img.php?file=img6/25/1_046.png)

DXVA Off, Post Processing = 2 (Strong Deblocking):
http://img6.picsplace.to/img6/25/thumbs/2_033.png (http://img6.picsplace.to/img.php?file=img6/25/2_033.png)

DXVA On, Post Processing = 2 (Strong Deblocking):
http://img6.picsplace.to/img6/25/thumbs/3_029.png (http://img6.picsplace.to/img.php?file=img6/25/3_029.png)

This time the results are the same as yours zambelli. For progressive encodes (wmv9 and wvc1) deblocking is performed when dxva is on and deblocking is on. And another new finding is that for dxva mode "autodetect deblocking" means don't do deblocking while in software mode it means do deblocking,

So there really are a lot of possible cases with lots of different results.

P.S. I updated the screenshots in my earlier post to identical frames.