Nvidia and Microsoft demonstrate GPU Compute based transcoding in Win7 [Archive]

View Full Version : Nvidia and Microsoft demonstrate GPU Compute based transcoding in Win7

CruNcher

2nd June 2009, 19:58

This isn't the same as Microsoft demonstrated @ WinHEC back then based on 3rd Party Transcoding Chips (especially Quartics Chip), also this could rather have a big impact making a lot of 3rd party transcoder unneeded (for user with certain goals) though Nvidias GPU Encoder Quality is currently not up to the Quality of x264 and also Elemental Technologies still need some work todo but it's imho a nice thing for the avg Consumer to have it implemented inside the OS :) It seems to working full automatic with the information the Device is giving the Opeating system about it's capabilities (Bitrate,FPS,profile and resolution support) :)

Presentation Video GPU Compute Directx 11 Transcoding (Computex):
http://www.youtube.com/watch?v=S1XCBQV-iQc

Hopefully Microsofts Encoder can be replaced via a x264 call ;) (i wouldn't bet on that)

Presentation Slide Dedicated Hardware Transcoding (WinHEC):
http://www.youtube.com/watch?v=2Q9Ifs0Bd6Y

Presentation Video Dedicated Hardware Transcoding (WinHEC):
http://www.youtube.com/watch?v=kmNwK1FWv2M

Now if the Nvidia Ceo talks about "Improving efficiency by using the right Chip for the right task" definitely he's questioning himself is the GPU the right thing todo trancsoding (Encoding) ;)

ChronoCross

2nd June 2009, 20:15

Please read:

http://en.wikipedia.org/wiki/Full_stop

Sadly I wish I were more impressed with that. While it does show potential I doubt it will ever reach power user level for encoding purposes.

CruNcher

2nd June 2009, 22:25

Though it's interesting you don't know what for capabilities you have in terms of using this API and enhance it or change different aspects of it (Encoder used) :) Ben Waggoner could really shed some light on this also i wonder why no one yet reported about the Win7 build in H.264 Encoder :P (guess it's a Media Foundation Encoder and not accessible for most yet).
If Nvidias Encoder would at least deliver quality i would see above the Power user encoding level thing as this is nothing the AVG user this is targeted @ needs in reality and heck even some of the so called Power user don't :)

I wonder if this works for the PSP and if it can detect the max resolution based on the firmware the PSP connected is using also if the Device supports multiple playback formats and deciding into which format to convert too (though that would be more or less Power User options) ;)

Kurtnoise

3rd June 2009, 13:04

i wonder why no one yet reported about the Win7 build in H.264 Encoder :P (guess it's a Media Foundation Encoder and not accessible for most yet).

Only BaseLine & Main Profiles (http://msdn.microsoft.com/en-us/library/dd797816(VS.85).aspx)...I know you like it. :D

benwaggoner

3rd June 2009, 23:25

Only BaseLine & Main Profiles (http://msdn.microsoft.com/en-us/library/dd797816(VS.85).aspx)...I know you like it. :D
It's a pretty good, fast transcoder for devices, but it's not meant to compete with professional-grade implementations like x264 for high-touch scenarios.

Kurtnoise

4th June 2009, 07:50

Do you know which tools are able to use this compressor ? Windows Movie Maker I guess ?

MfA

4th June 2009, 18:54

It's a pretty good, fast transcoder for devices, but it's not meant to compete with professional-grade implementations like x264 for high-touch scenarios.
The question is, can it even compete with x264 in that transcoding in an honest comparison on a halfway decent CPU? (Say a 3 GHz C2D.)

Gabriel_Bouvigne

5th June 2009, 08:46

It's a pretty good, fast transcoder for devices, but it's not meant to compete with professional-grade implementations like x264 for high-touch scenarios.
Isn't it just a matter of implementation cost that made Microsoft not include high profile?
I understand how using CAVLC instead of CABAC speeds up encoding, but I don't really get how using main profile instead of high profile might significantly speed up encoding.

Of course, we could also think that the goal could be to have WMV encoding shining against H.264, but that would only be pure speculation at this point.

audyovydeo

5th June 2009, 12:53

Isn't it just a matter of implementation cost that made Microsoft not include high profile?
I understand how using CAVLC instead of CABAC speeds up encoding, but I don't really get how using main profile instead of high profile might significantly speed up encoding.

Of course, we could also think that the goal could be to have WMV encoding shining against H.264, but that would only be pure speculation at this point.

I suspect it may have more to do with universal playability. Anything made with/by microsoft has gotta be backward compatible, in a way, to Windoze version X.Y. Main profile, no CABAC would be a way to ensure this on older Win machines.

my 2 cents
audyovydeo

5th June 2009, 13:10

I suspect it may have more to do with universal playability. Anything made with/by microsoft has gotta be backward compatible, in a way, to Windoze version X.Y. Main profile, no CABAC would be a way to ensure this on older Win machines.
Well, Ben already said it is for re-encoding video for "devices", which I guess means portable media players and mobile phones. Most of them only support Baseline or Main profile.

Is there some document that says it doesn't support CABAC in Main profile?

Gabriel_Bouvigne

5th June 2009, 18:14

Wondering which kind of device supports main profile but not high profile (especially considering that high profile is not slower to decode than main profile)

5th June 2009, 18:27

For example PSP and Archos 5.

Dark Shikari

5th June 2009, 18:51

It's a pretty good, fast transcoder for devices, but it's not meant to compete with professional-grade implementations like x264 for high-touch scenarios.Speaking of which, can someone upload the binary for this? I'd like to take a look at it ;)

Kurtnoise

6th June 2009, 01:33

you know you can get Windows Seven RC for free ? so, virtualize & play...:)

Dark Shikari

6th June 2009, 01:41

you know you can get Windows Seven RC for free ? so, virtualize & play...:)I don't intend to install that crap on my computer, even virtualized.

Anyways, I go the binary, and as expected, the asm is terrible. Their transforms even transpose twice for no good reason...

CruNcher

6th June 2009, 02:40

Dark Elecard brought back Complexity & Luminance Masking in the new Converter Studio :)
http://s4b.directupload.net/images/090606/nmsp5qov.png
the way it works seems to have changed though the range is now 1-10 no -range anymore

benwaggoner

6th June 2009, 09:11

Isn't it just a matter of implementation cost that made Microsoft not include high profile?
I understand how using CAVLC instead of CABAC speeds up encoding, but I don't really get how using main profile instead of high profile might significantly speed up encoding.
It was speed of development. Most of the target devices were going to be Baseline anyway. The developers were able to get in plenty of Main features, but didn't get to 8x8 for Win 7.

Of course, we could also think that the goal could be to have WMV encoding shining against H.264, but that would only be pure speculation at this point.
I'm flattered you think we're capable of such a degree of internal coordination :). But no, we really don't care about VC-1 versus H.264 as a goal, and are happy to use either or both as appropriate for any given product and project.

CruNcher

6th June 2009, 11:54

benwaggoner

6th June 2009, 17:58

hilarious, unthinkable ,unbelievable Microsoft would never misuse their power this way and let it look like their Developers are that incompetent and try to broaden WMVs marketshare also in terms of available content user/comercial ;)
still the possibility of being undeniable ?
Do we win anything meaningful if someone uses WMV/VC-1 over H.264? We're in both patent pools, and will be licensing both encoders and decoders at the MPEG-LA caps. VC-1 is the evolution of a technology we've got over a decade of active development on (it's been the same code base since WMV 7), so it's more mature than our H.264 in a number of areas, but the gap is certainly closing. Windows 7, like Silverlight, Zune, and Xbox isn't doing anything to try and enforce codec choices. The gaps are entirely due to H.264 being a lot newer to us, and in general.

benwaggoner i asked a question about how the new inbuilt Device Transcoding in Win7 is deciding into which which format it transcode the input and if WMV is prioritized over anything else the Device supports :) would be nice to get a answer about this and if the user can decide to transcode into the Device supported formats (i guess you left that out because you don't want to overload the user with options of course),but of course why should he chose H.264 Mainline Encoder in the first place if it's quality wise no challenge in Win7 to WMVs Advanced Profie Encoder questions over questions (at least he isn't forced by the Device to do so and then says hey i need a WMV Device all my friends stuff looks better) ;)
So, you're asking which format Win 7 picks for transcoding for a device that supports both WMV and H.264 encode? That I don't know. The Windows H.264 encoder was most tuned for baseline, which is what most devices support, so I'd expect quality and performance to be generally competitive. An, of course, lots of systems are going to include hardware transcode, so both quality and performance will be out of our hands.

Have you tried it? You seem to be assuming it's bad.

Now, H.264 Baseline versus VC-1 Main pits adaptive block size and B-frames versus stronger in-loop deblocking filter and multiref. No doubt that H.264 High can outperform VC-1 Advanced Profile at high compression ratios, but Baseline v. Main can go either way.

PS: The First time we gonna see this H.264 vs VC-1 war will be in Microsofts own Nvidia Tegra Device (APX 2500/2600) based Zune HD
Actually, it was the Zune 2 in 2007 added H.264 (and improved VC-1) playback. What's new in the HD is HD :). Beyond that, MediaRoom has been mainly H.264 in deployment. Xbox has supported H.264 playback for several years. Expression Encoder added H.264 encoding last year.

And given product development scheduled, obviously we'd decided to support H.264 in those areas well before they shipped.

Really, once the licensing and patent pools were finalized for both codecs, there clearly wasn't a strong business reason to care either way, so we started supporting both where it made sense. At our volume, we're going to be at the caps for encoders and decoders anyway, so it's basically no extra cost to have decoding for either or both in any particular product.

Dark Shikari

6th June 2009, 18:02

Have you tried it? You seem to be assuming it's bad.Given that it has the worst assembly code of any commercial encoder I've ever seen (double-transpose DCT, seriously?), I'd say it's probably bad.

It also has no SATD, which means it uses SAD mode decision, and is therefore a joke.

benwaggoner

6th June 2009, 20:24

Given that it has the worst assembly code of any commercial encoder I've ever seen (double-transpose DCT, seriously?), I'd say it's probably bad.

It also has no SATD, which means it uses SAD mode decision, and is therefore a joke.
I don't know anything about the underlying implementation, but I'd be happy to pass on any feedback.

Have you looked at the actual output quality yet?

That said, I'm curious to see how many Win 7 machines use the software implementation versus hardware approaches. I imagine it won't be too long before consumer speed-over-quality device transcoding scenarios wind up being mainly hardware, leaving the software implementation as really a fallback.

Really, the goal for device transcode/sync is to be able to deliver as much quality as is possible within the Mbps as the device's USB interface can take; transcoding should be in parallel with the copy. That should be quite achivable with Baseline on 480x wide screens, as long as the transcoding doesn't require the movie header at the start or something.

For most of today's mobile devices, the screen is going to be a much lower bar on max quality than anything a codec does, alas. I hope the Zune HD's OLED wlll finally give us something better than these terrible < 24-bit LCD displays in so many portable devices. I haven't seen a production Zune HD myself yet.

I thought I'd found a serious banding quality issue in the original Zune 30 transcoding model when it was preparing to launch, but it turned out to be because the display processor not having dithering, so the 16-bit screen was really only ~64 levels between black and white.

Anyway, I'm definitely not going to claim the Win 7 H.264 implementation is close to professional grade. It'll get better (and already has in the new branch since Win 7 went RC).

But from my content ecosystem perspectve, I expect most professional content being pubished to Silverlight etcetera will continue to use the existing H.264 implementations in professional products. We've got no skin in the game for anyone using our implementation over another. We'll be using and buildng ours for scenarios where we see a need for our own implementation due to licensing or customization reasons. So we're certainly going to do an implementation of Smooth Streaming H.264 using our encoder, but will also share with everyone else how to make a compliant and tuned Smooth Streaming bitstream.

Speaking of which, I'm happy to give a brain dump on how to add Smooth support for x264 whenever anyone's curious. It's pretty simple:

All streams need to have an IDR/Closed GOP I-frame at the same frame at all bitrates
Those should be every 2-4 seconds
Okay to have non-IDR I-frames wherever you like
Sequence header/display info at each switch point
Up to High Profile (8-bit 4:2:0) is supported
Progressive scan only

We can start a new thread if anyone would like to discuss further.

Anyway, x264 is inarguably awesome, and I'm pleased as punch whenever someone uses it to make great looking content played back in Silverlight, Zune, Xbox, Media Center, Win 7, etcetera.

MfA

6th June 2009, 20:59

Do we win anything meaningful if someone uses WMV/VC-1 over H.264?
Have the ASF patents ever gotten an alternative license which doesn't hold you hostage in 2012? (Sure in theory an increased use of VC-1 doesn't necessarily mean an increased use of ASF, but in practice it does.)

PS. Cruncher, tons of technology was developed for the military, what does it matter?

Dark Shikari

6th June 2009, 21:09

I don't know anything about the underlying implementation, but I'd be happy to pass on any feedback.

Have you looked at the actual output quality yet?I don't have Windows 7, and I don't intend to install it.That said, I'm curious to see how many Win 7 machines use the software implementation versus hardware approaches. I imagine it won't be too long before consumer speed-over-quality device transcoding scenarios wind up being mainly hardware, leaving the software implementation as really a fallback.I don't expect that to happen any time soon, given that x264 still outperforms practically all consumer-level hardware encoders. Nevermind the fact that the vast majority of consumer PCs don't have the high-end graphics cards necessary to even compete with a low-end CPU in encoding speed.

GPU transcoding is stillborn, despite the absurd levels of marketing hype surrounding it.Really, the goal for device transcode/sync is to be able to deliver as much quality as is possible within the Mbps as the device's USB interface can take400 megabit video on a Zune or iPod? Is this a joke? The USB interface is the last thing to possibly be the bottleneck.

benwaggoner

6th June 2009, 23:38

Have the ASF patents ever gotten an alternative license which doesn't hold you hostage in 2012? (Sure in theory an increased use of VC-1 doesn't necessarily mean an increased use of ASF, but in practice it does.)
It's the codec that has the license fee; doesn't matter what it's wrapped in.

Personally, I think the MPEG-LA usage fees are pretty reasonable. You have to get up to pretty high volume before anything is due, if you're doing that kind of volume, the cost would be a small part of your revenue. And everything but downloads/rentals gets an annual cap.

benwaggoner

6th June 2009, 23:54

I don't have Windows 7, and I don't intend to install it.
Then why are you complaining about it :)?

I don't expect that to happen any time soon, given that x264 still outperforms practically all consumer-level hardware encoders. Nevermind the fact that the vast majority of consumer PCs don't have the high-end graphics cards necessary to even compete with a low-end CPU in encoding speed.
I wasn't talking about GPU transcoding in the OpenCL/CUDA sense, but SHED:

http://msdn.microsoft.com/en-us/library/dd568167.aspx

GPU transcoding is stillborn, despite the absurd levels of marketing hype surrounding it.[QUOTE]
Classic GPUs clearly didn't have the architectural flexibility to make a good encoder, but as GPU/CPU architectures continue to converge, we'll definitely hit a crossover point where GPU's advantages are sufficient that its limitations don't hold it back on net.

A big part of GPU's limitations has been how painful the development model was, so it was hard to do continuous tweaking and tuning. That's why we took the hybrid approach with Tarari, where the higher level rate control, frame type decision, etcetera was handled in software, but quantization and motion search was handed off to hardware.

That's the model that the pixel-shader based encoders are taking; make the highly parallel stuff faster, while the complex logic remains in the CPU. You'll note the CPU specs on the Elemental boxes are pretty high too.

SHED's more like a classic DSP model, of course, so is much more about speed than compression efficiency. Which is appropriate for device syncing, since each encode probably only ever gets watched once.

[QUOTE]400 megabit video on a Zune or iPod? Is this a joke? The USB interface is the last thing to possibly be the bottleneck.
The USB spec can do 400 Mbps, but the actual controllers in mobile devices don't do anything near that fast. Even preencoded content syncing doesn't have more than a small fraction of that performance, or else we'd be able to blast MP3 to devices in about 2 seconds per hour-long album.

If you prefer, pretend I said "transcode while sync at full sync speed."

I'm also saying that's the goal, not where things are at the moment :).

MfA

7th June 2009, 00:36

It's the codec that has the license fee; doesn't matter what it's wrapped in.
I know the ASF license is free now ... I also know that after 2012 you can use the patented technology
solely to the limited extent needed to (i) continue to distribute your Solutions containing your Implementations where such Solutions were first commercially released in a general, non-beta form during the Term (“Legacy Solutions”) and (ii) develop bug fixes and provide product support for your Legacy Solutions.

PS. I see you do have alternative licensing programs now though ... a whole damn truckload, my mistake :)

Comatose

7th June 2009, 09:56

Smooth Streaming support with x264 would be nice.

Kurtnoise

7th June 2009, 11:33

Smooth Streaming support with x264 would be nice.
http://h264.code-shop.com/trac/wiki/Smooth-Streaming-H264
http://smoothhd.code-shop.com/

benwaggoner

7th June 2009, 17:29

http://h264.code-shop.com/trac/wiki/Smooth-Streaming-H264
http://smoothhd.code-shop.com/
Well, look at that :).

We'll obviously get B-frames working in the new parser. I'm actually a little surprised H.264 worked at all in the old component; it was only ever tested with and targeted for VC-1.

What kind of quality impact does -no-scenecut have on x264? In the VC-1 Encoder SDK implementation, we allow a fixed Closed GOP cadence, but insertion of additonal Open GOP I-frames within is supported without changing the cadence.

Dark Shikari

7th June 2009, 21:10

Well, look at that :).

We'll obviously get B-frames working in the new parser. I'm actually a little surprised H.264 worked at all in the old component; it was only ever tested with and targeted for VC-1.

What kind of quality impact does -no-scenecut have on x264?The same it would have on any encoder really.

You don't need to turn off scenecut to do smooth streaming--just run a first pass, and then run all your different bitrates as second-passes off the same first pass, to ensure frametype decision is the same.

benwaggoner

7th June 2009, 21:14

You don't need to turn off scenecut to do smooth streaming--just run a first pass, and then run all your different bitrates as second-passes off the same first pass, to ensure frametype decision is the same.
Yes, tha would be better, and that's one of the things we're doing in our new VC-1 Smooth Streaming SDK.

We've found an average of 2 seconds with a max of 4 seconds to get scene change alignment seems to offer a good balance between efficiency and switching latency.

Gabriel_Bouvigne

8th June 2009, 08:47

It was speed of development. Most of the target devices were going to be Baseline anyway. The developers were able to get in plenty of Main features, but didn't get to 8x8 for Win 7.
I understand. Even if from a project planning POV it sounds really bad, in real life this is the kind of things which can happen.

(I initially thought that you just bought the source/object code, not that you did it in-house)

benwaggoner

9th June 2009, 21:00

I understand. Even if from a project planning POV it sounds really bad, in real life this is the kind of things which can happen.
Well, it wasn't that bad; it's a fine encoder for what it's meant for. It's just not meant for what this audience is interested in.

(I initially thought that you just bought the source/object code, not that you did it in-house)
Yeah, there's a REALLY high bar to bundle 3rd party binaries with Windows for a variety of obvious reasons. So it's almost all Microsoft's own code.

There's been further development since Win 7 went into lockdown. I can create some samples if anyone is curious. So far it has

Multiple Reference Frames
B-Frames
Partition size down to 4x4
CABAC
CBR/VBR

But there's no 8x8 blocks, pyramid B, or much adaptive frame type decision.

Kurtnoise

2nd October 2009, 10:47

There's been further development since Win 7 went into lockdown. I can create some samples if anyone is curious. So far it has

Multiple Reference Frames
B-Frames
Partition size down to 4x4
CABAC
CBR/VBR

But there's no 8x8 blocks, pyramid B, or much adaptive frame type decision.
Are they available only through EE3 or somewhere else ? coz I'm playing a bit w/ the h.264 encoder from the MFT library and we can't control these settings (or I miss something ?)...

btw, from the SDK (http://msdn.microsoft.com/en-us/library/dd318776%28VS.85%29.aspx), I can see this:
enum eAVEncH264VProfile {
eAVEncH264VProfile_unknown = 0,
eAVEncH264VProfile_Simple = 66,
eAVEncH264VProfile_Base = 66,
eAVEncH264VProfile_Main = 77,
eAVEncH264VProfile_High = 100,
eAVEncH264VProfile_422 = 122,
eAVEncH264VProfile_High10 = 110,
eAVEncH264VProfile_444 = 144,
eAVEncH264VProfile_Extended = 88
};
However, the compressor seems to be clamped to the Base/Main Profiles ? Why ? Or is it dedicated to the decoder side (if so, what a bad name imho) ? do you have any infos on this ?

:thanks:

benwaggoner

2nd October 2009, 20:15

Are they available only through EE3 or somewhere else ? coz I'm playing a bit w/ the h.264 encoder from the MFT library and we can't control these settings (or I miss something ?)...
Correct, all the twiddly bits are in EEv3 only for the moment.

However, the compressor seems to be clamped to the Base/Main Profiles ? Why ? Or is it dedicated to the decoder side (if so, what a bad name imho) ? do you have any infos on this ?
I think they just didn't get to the 8x8 transform before code complete :).

The decoder in Win7 handles High just fine, including interlaced (with a lovely 30i to 60p bob no less).

For those who can use x264, there's really no reason to consider the Win7 or EEv3 versions instead. The Win7 implementation is going to be screamingly fast since it accelerates source decode and preprocessing, but isn't going to be competitive in compression efficiency. And while DS found the EEv3 output to be roughly Ateme quality wth his animation test, it's quite slow compared to other implementations. It's most notable as the only H.264 enocder at the moment that's coupled with a muxer that does Smooth Streaming compatible fragmented MPEG-4 muxing and manifest generation.

But now all that info has been published under the Open Specification Promise, I'm hoping to see someone wire up x264 to a compatible muxer.