NAB news: Silverlight 1080p24 playback and taste of new VC-1

[benwaggoner]

I've got my big blog post up about what's happening with Silverlight at NAB:

http://on10.net/blogs/benwagg/NAB-Day-1-Smooth-Streaming-released-1080p-in-Silverlight-new-VC-1-and-more/

A few things of particular notice:

Silverlight 3 can do 1080p24 on a Core 2 Duo 2.4 GHz
We've got lots of tool vendors and CDNs announcing support for Smooth Streaming
We've got a new VC-1 implementation coming tuned for Smooth Streaming


I thought the latter might whet folks appitite. It's a new 2-pass mode tuned for Smooth Streaming's multiple bitrate approach, so it offers the aligned Closed GOPs we need for seamless bitrate switching. And it offers a big improvement for 2-pass CBR in VC-1 in general due to a bunch of rate control and other improvements. We've got a number of partners working on incorporating it into their products.

Here's a taste of it with Big Buck Bunny at 1920x1080 24 fps @ 4 Mbps.

http://silverlight.services.live.com/31260/Big%20Buck%20Bunny%201080p24%204%20Mbps%20Smooth%20Streaming/video.wmv

There will be a lot more info about it in coming months. I can show some other samples here at NAB; they're also demoing at the Inlet booth.

[Blue_MiSfit]

Looks pretty good, Ben!

Almost flawless, except for some pretty bad banding in the fades. Quite good for 1080p24 at 4mbps CBR. I didn't think VC-1 was capable of this.

Still - you have to admit, Big Buck Bunny is pretty darned compressible :)

I wonder how things would look at ~6mbps for a live action movie. x264's CBR surprises me with its visual quality, but its VBV model seems to choke at higher bitrates when you ask for CBR.

-Derek

[benwaggoner]

Looks pretty good, Ben!

Almost flawless, except for some pretty bad banding in the fades. Quite good for 1080p24 at 4mbps CBR. I didn't think VC-1 was capable of this.
Yeah, we could be doing better at fades and black areas in general. This isn't the final version, but a good indication of the progress since VC-1 Encoder SDK.

We get a little hit by the Smooth Streaming requirement, as we're doing Closed GOPs every 2-ish seconds in order to provide easy switching points.

Still - you have to admit, Big Buck Bunny is pretty darned compressible :)
For VBR, yes, but it's got some very high motion high detail parts as well that can really stress CBR. Plus all that grass is a great place to show detail. Noise can help mask higher QPs as well.

I wonder how things would look at ~6mbps for a live action movie. x264's CBR surprises me with its visual quality, but its VBV model seems to choke at higher bitrates when you ask for CBR.
Actually I've got some film source that looks pretty darn good at 4 Mbps too. I'll see what I've got here in Vegas and try to get something uploaded later.

[Blue_MiSfit]

Actually I've got some film source that looks pretty darn good at 4 Mbps too. I'll see what I've got here in Vegas and try to get something uploaded later.

Awesome! I look forward to it!


We get a little hit by the Smooth Streaming requirement, as we're doing Closed GOPs every 2-ish seconds in order to provide easy switching points.

Indeed - so Smooth Streaming is basically shorter GOP CBR, so you can easily adapt to volatile bandwidth conditions? Smart :)

Thanks for posting info in doom9 as always!

-Derek

[turbojet]

benwaggoner: As you work for microsoft could you inquire about their decoder having the ability to decode all interlaced VC-1 streams?

I posted a sample here (http://forum.doom9.org/showthread.php?p=1274753#post1274753)

Also as VC-1 directshow playback is very iffy on even current hardware do you know if they are planning on multithreaded decoding in the near future?

[zambelli]

Speaking of x264... At our Microsoft booth at NAB we were also showing a 720p24 H.264 version of "Big Buck Bunny" (to demonstrate Silverlight 3's support for H.264/AAC/MP4). I encoded it using x264, of course. :)

[Dark Shikari]

Big Buck Bunny is really great for demonstrating how good a crappy encoder is ;)

[benwaggoner]

benwaggoner: As you work for microsoft could you inquire about their decoder having the ability to decode all interlaced VC-1 streams?

I posted a sample here (http://forum.doom9.org/showthread.php?p=1274753#post1274753)
I think the decoder is generally in good shape. It's just getting the end-to-end pipeline do a proper Bob without tweaking registry keys that's a challenge :).

Also as VC-1 directshow playback is very iffy on even current hardware do you know if they are planning on multithreaded decoding in the near future?
There certainly have been mulithreaded VC-1 decoders built in the past. The Silverlight 3 one is currently single-threaded, as single-core machines are the biggest performance challenge, and we typically have plenty to do on a second core with the other parts of Silverlight (which itself is heavily multithreaded).

[benwaggoner]

Big Buck Bunny is really great for demonstrating how good a crappy encoder is ;)
Perhaps you ought to see more than one output from an encoder before determining its quality :).

[turbojet]

I think the decoder is generally in good shape. It's just getting the end-to-end pipeline do a proper Bob without tweaking registry keys that's a challenge :).


There certainly have been mulithreaded VC-1 decoders built in the past. The Silverlight 3 one is currently single-threaded, as single-core machines are the biggest performance challenge, and we typically have plenty to do on a second core with the other parts of Silverlight (which itself is heavily multithreaded).

I should have mentioned that I was referring to WMVideo directshow decoder.

While WMVideo decodes more than the only other free directshow decoder I know of, libavcodec from ffdshow. It still doesn't handle what has to be 30-40 retail BD streams reported so far.

Hopefully you can help out as I don't see anyone else working on an alternative decoder. I don't know how to contact libavcodec devs outside of ffdshow thread/bug tracker where it's already been posted for months with no reply other than libavcodec doesn't decode interlaced streams, which wasn't helpful. I've also tried contacting microsoft a few months ago without any reply.

Where do you see multithreaded VC-1 decoders?

[benwaggoner]

IWhile WMVideo decodes more than the only other free directshow decoder I know of, libavcodec from ffdshow. It still doesn't handle what has to be 30-40 retail BD streams reported so far.
In software? Normally we'd assume DXVA would be used for those kinds of bitrates. Generally any machine otherwise capable of BD playback would have a DXVA capable GPU anyway.

Hopefully you can help out as I don't see anyone else working on an alternative decoder. I don't know how to contact libavcodec devs outside of ffdshow thread/bug tracker where it's already been posted for months with no reply other than libavcodec doesn't decode interlaced streams, which wasn't helpful. I've also tried contacting microsoft a few months ago without any reply.
Who did you try to contact at Microsoft, and what was the request?

Where do you see multithreaded VC-1 decoders?
The HD DVD VC-1 and H.264 decoders were both multithreaded, for obvious reasons. Since VC-1 doesn't have CABAC, it's pretty easy to parallelize. However, as it doesn't have CABAC, there's less need to :).

It's possible that the Win 7 software VC-1 decoder may also be multithreaded - haven't tried it myself on a machine that wouldn't have defaulted to the GPU.

[turbojet]

In software? Normally we'd assume DXVA would be used for those kinds of bitrates. Generally any machine otherwise capable of BD playback would have a DXVA capable GPU anyway.

I understand your point but the microsoft decoder from windows media player 11 isn't capable of DXVA as far as I know. Plus I'm one of numerous people that has a computer capable of playing 1080p seamlessly but it doesn't have a DXVA supported video card.

Who did you try to contact at Microsoft, and what was the request?

It was a few months ago now but I'm pretty sure I used this (https://support.microsoft.com/contactus/emailcontact.aspx?scid=sw;en;1539&ws=corpinfo&ws=support). If you can put me in contact with someone that works with the WMVideo decoder and I can send that clip to I'd appreciate it. Even easier would be for you to mention the clip to them and with any luck at all they'll fix it so it can decode a larger range of interlaced streams.

The HD DVD VC-1 and H.264 decoders were both multithreaded, for obvious reasons. Since VC-1 doesn't have CABAC, it's pretty easy to parallelize. However, as it doesn't have CABAC, there's less need to :).

It's possible that the Win 7 software VC-1 decoder may also be multithreaded - haven't tried it myself on a machine that wouldn't have defaulted to the GPU.

What I said about multithreading of the WMV9 decoder is inaccurate as I seem to get 60-70% cpu usage on a dual core box with 1080p in MPC-HC. More efficient would be the correct term I guess as there still seems to be many frame drops with microsoft's decoder while cyberlink v7 and 8 plays seamlessly on the same computer without DXVA. I haven't tried windows 7 yet.

If you want to see how widspread the problem is a large majority of these 50 bug reports (http://forum.doom9.org/search.php?searchid=4467173) can be narrowed down to lack of a VC-1 directshow decoder that can decode all interlaced streams.

[benwaggoner]

I understand your point but the microsoft decoder from windows media player 11 isn't capable of DXVA as far as I know. Plus I'm one of numerous people that has a computer capable of playing 1080p seamlessly but it doesn't have a DXVA supported video card.
What GPU do you have? Pretty much all the mainsteam cards have had at least some DXVA for years.

And WMP 11 definitely supports DXVA. I use it all the time.

It was a few months ago now but I'm pretty sure I used this (https://support.microsoft.com/contactus/emailcontact.aspx?scid=sw;en;1539&ws=corpinfo&ws=support). If you can put me in contact with someone that works with the WMVideo decoder and I can send that clip to I'd appreciate it. Even easier would be for you to mention the clip to them and with any luck at all they'll fix it so it can decode a larger range of interlaced streams.
Is this is a perf issue or a compliance issue. If you get a bug playing back, it's more likely you'd get a fix. Can you describe what happens when it doesn't work?

Win 7 has gotten a huge amount of media performance tuning as part of the general adoption of MediaFoundation. However, since that has lots of OS dependencies you shouldn't expect much of that to get backported to older versions of Windows.

What I said about multithreading of the WMV9 decoder is inaccurate as I seem to get 60-70% cpu usage on a dual core box with 1080p in MPC-HC. More efficient would be the correct term I guess as there still seems to be many frame drops with microsoft's decoder while cyberlink v7 and 8 plays seamlessly on the same computer without DXVA. I haven't tried windows 7 yet.
Is this XP or Vista? Vista can be quite a bit more performant for media playback; I can get 1080i30 @ 35 Mbps to playback smoothly on my laptop in WMP 11 on Vista 32-bit with DXVA. With quite nice bobbing to 1080p60.

[mariner]

What GPU do you have? Pretty much all the mainsteam cards have had at least some DXVA for years.

And WMP 11 definitely supports DXVA. I use it all the time.



Greetings Ben.

1. WMVideo decoder only offer up to VC1_IDCT DXVA decoding (mode A,B and C). So DXVA does not work with ATI GPU that only support VC1_VLD (mode D).

You can us DXVA Checker to determine the capabilities of GPU and decoders.

http://bluesky23.hp.infoseek.co.jp/index.html#DXVAChecker

2. So far I have not had any success getting WMVideo decoder to work with interlaced contents.

Thanks and best regards.

[benwaggoner]

@mariner,
Right. I'm not familiar with the backstory, but Mode D isn't ever going to be supported for XP/Vista AFAIK. ATI was informed about that ages ago, so it's really up to them to enable A, B, or C.

My machines that do interlaced playback all have NVidia cards...

[mariner]

Thanks of the kind reply, Ben.

I had no success with interlaced contents on NV based system. Did you use software decoding?

[benwaggoner]

DXVA, on Vista. I did turn on the DXVA and Interlaced Decode reg keys with WMV9 PowerToy.

I haven't tried on XP.

[mariner]

Thanks. Will try again.

Would appreciate you could tell me the registry keys location.

[Comatose]

with WMV9 PowerToy
http://lmgtfy.com/?q=WMV9+PowerToy

[turbojet]

What GPU do you have? Pretty much all the mainsteam cards have had at least some DXVA for years.

And WMP 11 definitely supports DXVA. I use it all the time.

Geforce 6600GT.

Is this is a perf issue or a compliance issue. If you get a bug playing back, it's more likely you'd get a fix. Can you describe what happens when it doesn't work?

Decoding issue, it's from retail BD's which I guess could be non-compliant but cuda decodes them fine. Freezes, blocky and green screen are some of the things I've seen. Check out the sample (http://www.mediafire.com/download.php?tzlnhdynnhq) I posted last week in this thread.

Win 7 has gotten a huge amount of media performance tuning as part of the general adoption of MediaFoundation. However, since that has lots of OS dependencies you shouldn't expect much of that to get backported to older versions of Windows.


Is this XP or Vista? Vista can be quite a bit more performant for media playback; I can get 1080i30 @ 35 Mbps to playback smoothly on my laptop in WMP 11 on Vista 32-bit with DXVA. With quite nice bobbing to 1080p60.

Vista 64 bit but still same issue with xp 64 bit on the same box, I dual boot on an X2 3800+. VC-1 software decoding performance is about 60% compared to tuned H.264 decoders. Should this be the case?

[benwaggoner]

Geforce 6600GT.
Hmmm. I think you may need PureVideo 2 on your GPU to do interlaced VC-1 playback. The systems I'm using are 8??? series card with it. The 6??? series was just the original PureVideo.

http://en.wikipedia.org/wiki/Purevideo

Decoding issue, it's from retail BD's which I guess could be non-compliant but cuda decodes them fine. Freezes, blocky and green screen are some of the things I've seen. Check out the sample (http://www.mediafire.com/download.php?tzlnhdynnhq) I posted last week in this thread.
PureVideo 2 or 3 is what's recommended for Blu-ray playback...

Vista 64 bit but still same issue with xp 64 bit on the same box, I dual boot on an X2 3800+. VC-1 software decoding performance is about 60% compared to tuned H.264 decoders. Should this be the case?
Well, the VC-1 software decoder there wasn't as tuned for interlaced as progressive, which could be your issue. The whole i>p conversion came in pretty late in Vista's development IIRC. I don't think anyone imagined software decode of BD-rate VC-1 back then; that was very much a DXVA scenario.

[turbojet]

9800GT doesn't decode that clip either, it freezes.

Again if you can report to microsoft that their WMVideo Decoder included with WMP11 does not decode all BD's and point them to that clip as an example or let me know how to contact someone who will understand the situation.

I get roughly the same performance decoding interlaced and progressive VC-1 through directshow.

[mariush]

I've read the whole thread but I still don't get it...

Is this video actually supposed to be decoded properly with DXVA on ATI cards?

I'm asking because Media Player Classic Home Cinema on my Radeon 4850 with the latest drivers tries to decode it using DXVA VC-1 Bitstream Decoder and all I can see is junk:

click for picture (http://savedonthe.net/image/226/Clipboard01.jpg)

Same with Videolan Client , some pre 1.0.0 built from their nightly builds.

[benwaggoner]

I've read the whole thread but I still don't get it...

Is this video actually supposed to be decoded properly with DXVA on ATI cards?

I'm asking because Media Player Classic Home Cinema on my Radeon 4850 with the latest drivers tries to decode it using DXVA VC-1 Bitstream Decoder and all I can see is junk:

click for picture (http://savedonthe.net/image/226/Clipboard01.jpg)

Same with Videolan Client , some pre 1.0.0 built from their nightly builds.
That suggests it might be a corrupted stream somehow. VLC is its own self-contained thing and doesn't touch the system decoders.

Does 1.0.0 include the new better ffmpeg VC-1 decoder yet?

[mariush]

That suggests it might be a corrupted stream somehow. VLC is its own self-contained thing and doesn't touch the system decoders.

Does 1.0.0 include the new better ffmpeg VC-1 decoder yet?

It's your very own video.wmv from the the start of this thread.

I'm not sure about VLC's ffmpeg, it's a nightly build of 1.0.
Media Player Classic Home Cinema is 1.2.908.0

Here's the log file of VLC: http://savedonthe.net/download/111/videolanlog.txt

It looks like it uses libavcodec but there's no versioning on the libraries.

I've been doing additional testing on MPC HC now, and while initially I thought it doesn't matter maybe it does so I'm reporting it, with DXVA disabled MPC HC's screen starts to look exactly like VLC's screen, meaning it's like this: http://savedonthe.net/image/243/Clipboard02.jpg

I can't test if it works properly with windows codecs because I can't figure out where to download the codecs from. Media Player 11 won't install because I use Windows 2003, Expression Encoder seems to be willing to save in VC-1 format but the codec is not available to MPC HC, when I disable both internal filters, the same with Windows Media Encoder...