H.264 support coming in Silverlight

[benwaggoner]

There've been some past discussions about this, so I wanted to give everyone a heads up that we announced forthcoming support for H.264 and MPEG-4 in Silverlight.

Details here:
http://on10.net/blogs/benwagg/H264-and-AAC-support-coming-in-Silverlight/

[Dark Shikari]

Edit: I apologize, I apparently misread "MIPS" as "MBPS". Its probably easier to just say "processor power" given that the distinction between the two terms is rather important...

Good news it seems: Flash won't have much over Silverlight for much longer at this rate.

[smok3]

i could write a really nasty post here, but instead i'll just say: good work Ben!

[Atak_Snajpera]

VC-1 only needs about half as many MIPS per pixel for playback.
So what? DXVA solves that problem

[mitsubishi]

Well it's about time, I've seen a lot recently from MS acknowledging the weight of opinion on this, so thought it would be coming. It should go without saying, but this includes the same codec deal for moonlight? Also what is meant by the version after 2.0? 2.1, 2.0SP1 or 3.0? ie is it happening soon.

EDIT: Thanks btw, one of things I was hoping to hear this year.

[benwaggoner]

So what? DXVA solves that problem
Alas, given how Silverlight rendering works, decode is software only for the time being. Plus we need to run on systems without DXVA, and on XP where DXVA support, particularly for H.264 is... unpredictible. Plus we're also on Mac, where there's really no infrastructure for hardware decode acceleration.

However, it's one of those cases that the systems where DXVA isn't a safe option are also the systems where you'd most want it from a perf perspective.

Agreed that the codec perf difference isn't much of a factor when there's some kind of hardware support, but we're some years away from that being a safe assumption for a browser plugin like Silverlight.

[benwaggoner]

Edit: I apologize, I apparently misread "MIPS" as "MBPS". Its probably easier to just say "processor power" given that the distinction between the two terms is rather important...
Yeah, that's a pretty big difference. I imagine I missed reading something very rude while I was off at lunch :).

Processor Power's been confusing for people as well in the past, but I agree MIPS isn't really any better. I'm happy for any suggestions for what clear terminology for this would be.

Good news it seems: Flash won't have much over Silverlight for much longer at this rate.
I sure hope so!

[toytown]

Great news, i just posted on another forum today, saying that the lack of H264 decoding was really one of the sticking points for webmasters who were pushing video content

[Sagekilla]

Twice the performance or speed would work well too if you're running at 1/2 the mips per pixel apparently ;)

[Sharktooth]

thats indeed a really good news and im sure it will help the silverlight adoption a lot...
just out of curiosity, does that also mean expression encoder is going to have h.264/mpeg4 too?

[benwaggoner]

thats indeed a really good news and im sure it will help the silverlight adoption a lot...
just out of curiosity, does that also mean expression encoder is going to have h.264/mpeg4 too?
Yep. From my blog post:

"At IBC 2008 we will be demonstrating a technology preview of H.264 video and Advanced Audio Coding (AAC) audio playback support in Silverlight, and H.264 authoring using Microsoft Expression Encoder and Windows Server 2008 for delivery. "

I'm excited myself, as I haven't actually seen the build with that working yet :).

[Sharktooth]

i missed that part. glad to see some of my suggestions were usefull.

[benwaggoner]

i missed that part. glad to see some of my suggestions were usefull.
We're always listening, even if we can't always do everything everyone wants immediately.

[Schrade]

Ben, thanks for posting this. I'm sure this will be awesome news for our upcoming software at Winnov :) I want to get my hands on that h.264 enabled version of Expression Encoder.

[benwaggoner]

Ben, thanks for posting this. I'm sure this will be awesome news for our upcoming software at Winnov :) I want to get my hands on that h.264 enabled version of Expression Encoder.
Come by IBC and check out what we've got.

Note that we we won't neccessarily have feature parity on H.264 in the initial version of Expression Encoder that supports it. The more important hurdle is feature parity on the decode side, since the key reason to have H.264 in Silverlight is to support existing libraries of H.264 content. When something is being encoded specifically for Silverlight, using VC-1 isn't nearly as big a deal.

[smok3]

shame i'll miss IBC again this year :/

[CruNcher]

Perfect benwaggoner i guess for many this will be the end of Flash if it stays such resource intense as it is currently is, btw why is DXVA via the Silverlight API not possible, i guess you will also License the H.264 core from DivX/Mainconcept right or CoreCodec ? :)

[Quark.Fusion]

I think DXVA via the Silverlight API not possible because video can be overlayed (player controls, etc.) and DXVA requires direct connection to compatible renderer to work. (Same reason why you can't do postprocess with DXVA)

[CruNcher]

Hmm Quark Fusion that's no real reason you can also overlay Player controls over DXVA via the Exlusive VMR9 Rendering Mode tough it's more work you have todo because its Fullscreen and 3D but its possible, so when you switch in Silverlight to Fullscreen it would just need to load this Exclusive mode and change it's interface (tough a interface isn't really needed @ all honestly in Fullscreen it's also up to your Website Design and how you gonna implement Silverlight into it and for which purpose) :)

[Comatose]

[...] and H.264 authoring using Microsoft Expression Encoder [...]
Will you be providing information on how to author externally for Silverlight?
That is, technical requirements, etc.

Also, will you be using a container like MP4 or a proprietary container?

[pandy]

"Comparing VC-1 Advanced Profile to H.264 High Profile with all the bells and whistles turned on, VC-1 only needs about half as many MIPS per pixel for playback."

ooooops - from my observations this claim is untrue - VC-1 is slow, especialy when decoder from MS is used, also seems that present implementations of encoders/decoders are far from h.264...
seems that "superiority" of the VC-1 over the h.264 is like "superiority" Vista over XP.

[Inventive Software]

He's talking with regard to most stand-alone players, which is mostly true. VC-1 is highly asymmetrical in it's performance (very slow encoding, fast decoding), while H.264 can mostly be made symmetrical, but is best asymmetrical.

[pandy]

In stand-alone solutions this is also untrue - hardware accelerated playback is no problem at all - due request of BD or DVB/ATSC h.264 and VC-1 is mandatory for stand-alone players so both codecs are comparable, also from market point of view VC1 is closely related with MS platforms and in long range term much more costly than h.264.

I use WinXP and VC-1 from MS is 1/3 fast as libavcodec implementations (which itself is far from perfect) - so where is "superiority"?

sorry for emotions, i try to be polite :D

[Sharktooth]

please, there are plenty of avc vs vc-1 threads, dont hijack this one into one of them...

[Quark.Fusion]

Hmm Quark Fusion that's no real reason you can also overlay Player controls over DXVA via the Exlusive VMR9 Rendering Mode tough it's more work you have todo because its Fullscreen and 3D but its possible, so when you switch in Silverlight to Fullscreen it would just need to load this Exclusive mode and change it's interface (tough a interface isn't really needed @ all honestly in Fullscreen it's also up to your Website Design and how you gonna implement Silverlight into it and for which purpose) :)

What reason to switch to fullscreen when you can watch streaming in your favorite mediaplayer with your favorite postprocessor? :)
And that doesn't solve non-fullscreen problem. You need DXVA 2.0 here.

[benwaggoner]

I think DXVA via the Silverlight API not possible because video can be overlayed (player controls, etc.) and DXVA requires direct connection to compatible renderer to work. (Same reason why you can't do postprocess with DXVA)
Yep, that's a big part of it.

Also, DXVA is simply not one of those things you can assume is going to work for a typical user. Guys around here, sure, but not on your average corporate desktop with old video card drivers.

[benwaggoner]

I use WinXP and VC-1 from MS is 1/3 fast as libavcodec implementations (which itself is far from perfect) - so where is "superiority"?
I'm assuming equally well optimized implementations of both codecs. The Win XP software decoder is pretty old at this point; there's plenty of optimization we've done since then. Also, that's going to be using postprocessing by default unless you turned it off or you're on a very slow machine, which will add additional CPU utilization.

It's also hard to tease how much of the decode load is from decoder versus the rest of the player when comparing player to player.

This info is from some cases where we've done both decoders on the same platform, like the Xbox 360. CABAC and a loop filter that touch a lot more pixels add quite a bit of overhead for software decode. It's enough that we've had multiple partners contact me to make sure we're NOT dropping VC-1, since they've found H.264 HD decode too expensive on low end machines.

[benwaggoner]

Hmm Quark Fusion that's no real reason you can also overlay Player controls over DXVA via the Exlusive VMR9 Rendering Mode tough it's more work you have todo because its Fullscreen and 3D but its possible, so when you switch in Silverlight to Fullscreen it would just need to load this Exclusive mode and change it's interface (tough a interface isn't really needed @ all honestly in Fullscreen it's also up to your Website Design and how you gonna implement Silverlight into it and for which purpose) :)
But even when we're running Silverlight in Full Screen we still support multiple elements, compositing, etcetera. Also, Silverlight is mostly run in-browser.

[puffpio]

There've been some past discussions about this, so I wanted to give everyone a heads up that we announced forthcoming support for H.264 and MPEG-4 in Silverlight.


...... .. ... ....... YYYEEESSSS!!!!
I remember a talk or webcast you gave when h264 was not even on the radar as a feature.
hopefully your decoder will blow the pants off flash's in terms of cpu usage (altho i hear the next version of flash will get a nice speed boost).

license coreavc maybe? :D

[smok3]

It's enough that we've had multiple partners contact me to make sure we're NOT dropping VC-1, since they've found H.264 HD decode too expensive on low end machines.
how did they benchmark?

[Dark Shikari]

He's talking with regard to most stand-alone players, which is mostly true. VC-1 is highly asymmetrical in it's performance (very slow encoding, fast decoding), while H.264 can mostly be made symmetrical, but is best asymmetrical.VC-1 is only highly asymmetrical because the Microsoft encoding implementation is very inefficient. I suspect it can be made just as symmetric as H.264.

[pandy]

I'm assuming equally well optimized implementations of both codecs. The Win XP software decoder is pretty old at this point; there's plenty of optimization we've done since then. Also, that's going to be using postprocessing by default unless you turned it off or you're on a very slow machine, which will add additional CPU utilization.


Hmmm so You say that for fast VC-1 i need MS Vista which itself need VERY POWERFULL PC due DRM mambos jambos inside system architecture...


It's also hard to tease how much of the decode load is from decoder versus the rest of the player when comparing player to player.


can we assume that simple graphedit can be used as a reference "player"


This info is from some cases where we've done both decoders on the same platform, like the Xbox 360. CABAC and a loop filter that touch a lot more pixels add quite a bit of overhead for software decode. It's enough that we've had multiple partners contact me to make sure we're NOT dropping VC-1, since they've found H.264 HD decode too expensive on low end machines.

Ben... please... on XP is slow so i must install Vista, for Vista i must use quite power full PC due "flaws" in OS architecture... so where is a difference between "slow" h.264 and "fast" VC-1???

LOW END machines definitely will not take Vista... Will You Ben say that MS create alternative to the MS Vista (maybe open source), focused on media market (real time architecture OS, less jittery than present MS OS's?)

PS
This is not another thread about which is better VC-1 or h.264 - I'm a engineer (not so young and definitely I'm not a fanboy) and i expect some facts not mambosjambos about codecs. I appreciate Ben for His hard work and respect Him but i don't understand reason for marketing or advertisement on Doom9 forum (especially when we don't have facts).

[Ranguvar]

*wonders how this turned from Silverlight discussion, to H.264 vs VC-1, to some sort of Windows bashing*

[Inventive Software]

VC-1 is only highly asymmetrical because the Microsoft encoding implementation is very inefficient. I suspect it can be made just as symmetric as H.264.

It can't, not for HD, and for SD you're gonna struggle on anything less than a Core 2. It's just plain slow encoding.

[benwaggoner]

...... .. ... ....... YYYEEESSSS!!!!
I remember a talk or webcast you gave when h264 was not even on the radar as a feature.
hopefully your decoder will blow the pants off flash's in terms of cpu usage (altho i hear the next version of flash will get a nice speed boost).

license coreavc maybe? :D
We're building our own implementation.

Do we know that Flash's H.264 decoder is slow in particular, or just that Flash is slow :). It's hard to tease out where the cycles are going in something like Flash; if someone has, I'd sure love to know what the target we're trying to hit is!

[benwaggoner]

It can't, not for HD, and for SD you're gonna struggle on anything less than a Core 2. It's just plain slow encoding.
For live encoding?

Try Expression Encoder 2's live encoding with "Dynamic Complexity" turned on. It'll scale the encoding complexity to what the local hardware is capable of without dropping frames. Faster machines will offer better quality, of course, and HD requries a lot of horsepower. But my laptop can certainly do live 640x480 reasonably well.

Anyway, encoding time and decoding time are quite orthgonal.

[benwaggoner]

It can't, not for HD, and for SD you're gonna struggle on anything less than a Core 2. It's just plain slow encoding.
For live encoding?

Try Expression Encoder 2's live encoding with "Dynamic Complexity" turned on. It'll scale the encoding complexity to what the local hardware is capable of without dropping frames. Faster machines will offer better quality, of course, and HD requries a lot of horsepower. But my laptop can certainly do live 640x480 reasonably well.

Anyway, encoding time and decoding time are quite orthgonal.

[benwaggoner]

how did they benchmark?
Some tried in Flash, others with other H.264 decoders. I imagine ffmpeg was in there somewhere.

[benwaggoner]

Hmmm so You say that for fast VC-1 i need MS Vista which itself need VERY POWERFULL PC due DRM mambos jambos inside system architecture...
Talking about using WMP/DirectShow/Media Foundation for playback? Yes, Vista will be better if you're using DXVA, but as long as you have WMP 11 installed on XP, it's about the same for software decode. Vista just has a much higher probabilty of using DXVA.

There's no DRM utilization in WMP if the content doesn't have DRM applied. Yes, I know that guy in New Zealand wrote that whitepaper about what he imagined Vista might have been like if we did everything in the worst possible way, but every emperical prediction he made there has been refuted by those actually running Vista :).

can we assume that simple graphedit can be used as a reference "player"Sure. That's DirectShow, and the same decoder (software or DXVA) as you'd get playing back in WMP.

Ben... please... on XP is slow so i must install Vista, for Vista i must use quite power full PC due "flaws" in OS architecture... so where is a difference between "slow" h.264 and "fast" VC-1???
I don't understand your question.

Also Vista runs reasonably well on "older" machines. You won't get Aero Glass on a pre-DX9 GPU, but that's about GPU, not CPU performance.

My point is that some of the more complex features of H.264, particularly a loop filter that touches more pixels with more taps, and CABAC, mean that the worst-case performance for a H.264 High Profile encode is about 2x the worst-case performance for an Advanced Profile VC-1 encode at the same bitrate. It won't matter in lots of cases.

FWIW, the Silverlight 2 VC-1 decoder is currently decoding 720p60 with about 30% CPU load on a Core 2 Duo laptop (not sure what the clock speed was for that test, but nothing unusually high).

[Comatose]

"Comparing VC-1 Advanced Profile to H.264 High Profile with all the bells and whistles turned on, VC-1 only needs about half as many MIPS per pixel for playback."

ooooops - from my observations this claim is untrue - VC-1 is slow, especialy when decoder from MS is used, also seems that present implementations of encoders/decoders are far from h.264...
seems that "superiority" of the VC-1 over the h.264 is like "superiority" Vista over XP.
Especially when decoder from MS is used?

There are some 3D videos available (google stereoscopic 3d player).
Get one that's double width 1080p and try playing it with ffdshow. You drop frames.

The MS decoder is fast enough to decode in realtime.

[CruNcher]

@benwaggoner

There's no DRM utilization in WMP if the content doesn't have DRM applied. Yes, I know that guy in New Zealand wrote that whitepaper about what he imagined Vista might have been like if we did everything in the worst possible way, but every emperical prediction he made there has been refuted by those actually running Vista.

Yes his Predictions where a little heavy tough people believed this and later the bad Performance of Vista proved what he said in those users eyes, they didn't experienced that the Kernel was actually improved and that the Performance problems came mostly from the added complexity of the Layer above the Kernel (all the new services,more indexing, shdaow copying, and more processes i/o access even now faster and more efficient now double the operations of XP) that eat up more of the Performance that have been gained by the improvements themselves (when staying with the current hardware).
Bad Balancing decisions i would say (of course non finished optimized drivers that doesn't taking every of the changes of the new Driver Model into account also ads up to this and bugs like in beginning in the Multimedia Scheduling then don't do good either, especially for a system that calls itself a Multimedia Experience) and so the PR Chaos was Perfect and now Vista is seen as a disaster by a big amount of users and for many experienced like a back step from XP.

[Quark.Fusion]

There's no DRM utilization in WMP if the content doesn't have DRM applied.
WMP always run wmv playback trough protected path. Maybe there is no decryption, but that prohibit using of ffdshow.

[benwaggoner]

WMP always run wmv playback trough protected path. Maybe there is no decryption, but that prohibit using of ffdshow.
I'm not sure what your ffdshow issue is, but WMP doesn't play back non-DRM'ed files via the protected video path. You can take screen shots and everything.

[Golgot13]

There've been some past discussions about this, so I wanted to give everyone a heads up that we announced forthcoming support for H.264 and MPEG-4 in Silverlight.

Details here:
http://on10.net/blogs/benwagg/H264-and-AAC-support-coming-in-Silverlight/


Hi all,

MS understand the market :eek: :eek:
Silverlight is nice, but MP4 format support BIFS (DVD like on MP4 container).
And I saw some MP4 ISMA with BIFS (use by some "cable network").

Ben, there is a specific muxer or SL (SilverLight) decode MP4 and MP4 ISMA.
SL support MP4 container or MPEG2 TS ?


Last Ben, do you think SVG is a good format ? ;)

[benwaggoner]

MS understand the market :eek: :eek:
Silverlight is nice, but MP4 format support BIFS (DVD like on MP4 container).
And I saw some MP4 ISMA with BIFS (use by some "cable network").
Really? I thought BIFS was DOA, once Envivo shifted their business plan to codecs.

I don't think I've seen a single BIFS player or file in the wild for at least five years.

Ben, there is a specific muxer or SL (SilverLight) decode MP4 and MP4 ISMA. SL support MP4 container or MPEG2 TS ?
We'll have a native MPEG-4 parser. We also have the MediaStreamSource feature of Silverlight whcih allows parsers to be built in managed code. So it's something an enterprising devleoper could write for Silverlight without us having to explicitly build it in.

Last Ben, do you think SVG is a good format ? ;)
Good for what? We didn't use it instead of XAML because SVG wasn't designed to be able to build interactive UX. Silverlight can access the browser DOM, so we could control SVG playback in a web page on a browser that supports it.

[pandy]

Talking about using WMP/DirectShow/Media Foundation for playback? Yes, Vista will be better if you're using DXVA, but as long as you have WMP 11 installed on XP, it's about the same for software decode. Vista just has a much higher probabilty of using DXVA.


DXVA means that we use hardware for decoding so due market regulations (BD, HD DVD etc) support for h.264 and vc-1 is mandatory - difference in speed are irrelevant (from hardware point of view).


There's no DRM utilization in WMP if the content doesn't have DRM applied. Yes, I know that guy in New Zealand wrote that whitepaper about what he imagined Vista might have been like if we did everything in the worst possible way, but every emperical prediction he made there has been refuted by those actually running Vista :).


security by obfuscation...For true... Vista is slower than XP


I don't understand your question.

Also Vista runs reasonably well on "older" machines. You won't get Aero Glass on a pre-DX9 GPU, but that's about GPU, not CPU performance.


No, Vista not runs "reasonably well" on "older" hardware.
Maybe it runs with similar performance in office tasks but definitely not in multimedia, especially when we don't use embedded solutions from MS.


My point is that some of the more complex features of H.264, particularly a loop filter that touches more pixels with more taps, and CABAC, mean that the worst-case performance for a H.264 High Profile encode is about 2x the worst-case performance for an Advanced Profile VC-1 encode at the same bitrate. It won't matter in lots of cases.



Hmmm Ben... CABAC can be disabled?, more taps=more quality?


FWIW, the Silverlight 2 VC-1 decoder is currently decoding 720p60 with about 30% CPU load on a Core 2 Duo laptop (not sure what the clock speed was for that test, but nothing unusually high).

From my point of view Core 2 Duo is definitively not a low end machine...

[CruNcher]

@pandy
read what i wrote the slowdown of Vista doesn't come from the Core it comes from the massive amount of stuff that runs above it.
If i refer to Vista or XP i refer to the Core not the gadgetry that comes with it :P
Vista should be a lot more responsive a little more secure and faster if you bring it down to XP complexity (on the same old hardware).
The main Features for me in Vista are only these.

+ More Efficient I/O and Virtual Memory management (dummy pages, more efficient page file use lifted restrictions dynamic decissioning)
+ ASLR + No execute
+ DXVA 2.0
+ EVR
+ GPU Management
+ DirectX 10

And all of this should add up to the Desktop Multimedia Experience even on older Machines (except ASLR + No execute) that it currently doesn't is a Design Flaw imho in the complexity of it for sure Windows 7 will do better here.

[hurry]

Will Silverlight support x264 MP4 video playback with HE-AAC v2 parametric stereo audio with this new addition? Thanks in advance.

[Sharktooth]

hurry: read again what ben said...
forum rule number 1: Read up before asking.

[hurry]

I have read Ben's blog article and all posts in this thread but I can't find explicit mention of x264 H.264 support in Silverlight. Maybe H.264 support means x264 support also since x264 is a H.264 codec, but still I wanted to confirm it more clearly if H.264 Silverlight support includes x264 H.264 support also and if AAC audio support includes Nero HE-AAC and HE-AAC V2 parametric stereo audio support also, since H.264 has many other codecs also like QuickTime, MainConcept, Nero Digital, Elecard etc. Thanks.

[Comatose]

x264 is a H.264 encoder and MP4 is a container. Since Silverlight will support H.264, it will obviously support x264... A few posts ago, ben also mentioned it'll support MP4.
And MP4 (the container) only supports H.264. The "codecs" you listed are H.264 encoders.

But I think the HE-AAC question is unanswered :P

[hurry]

Thanks Comatose. Ben, can you please tell us if SL will support the Nero HE-AAC and HE-AAC V2 audio? HE-AAC is a highly efficient codec as we all know and it would be great if SL supported it.

[Sharktooth]

We'll have a native MPEG-4 parser.
here it is.
And MP4 (the container) only supports H.264.
not properly correct, MP4 supports all MPEG4 video formats (MP4 supports also MPEG4-ASP, like Xvid/DivX/3ivX/etc, for example) and everything else in the form of private streams.

[Comatose]

Ugh, okay, I failed once more.

[hurry]

Ben, I want to request the support of full 5.1 AAC audio for the MP4 Silverlight support both at 44.1 Khz and 48 Khz and if possible at 96 KHz also. Presently, Silverlight does not even support 5.1 VC1 audio and only supports 44.1 Khz VC1 playback, which is a big limitation for providing HD quality. I found 96 Khz VC1s to sound much more crisper than 44.1 Khz VC1s. So please add support for 96 KHz audio for VC1 also. Thanks.

[pandy]

@pandy
read what i wrote the slowdown of Vista doesn't come from the Core it comes from the massive amount of stuff that runs above it.


Well - until Vista is not spearated from this stuff then Vista is slow, maybe next OS from MS will be faster - i don't know. At this moment for playing faster VC-1 i'm must install Vista...
Where is logic?


If i refer to Vista or XP i refer to the Core not the gadgetry that comes with it :P


until Vista is not separated from slowing Vista components then Vista is slow.


Vista should be a lot more responsive a little more secure and faster if you bring it down to XP complexity (on the same old hardware).


Maybe yes, but maybe not - we don't know such thing, we can only speculate (and speculation is very close to this what taken me to write inside thread - Ben speculate that VC-1 is twice faster than h.264 - i can't confirm such speculation from my experience)


The main Features for me in Vista are only these.

+ More Efficient I/O and Virtual Memory management (dummy pages, more efficient page file use lifted restrictions dynamic decissioning)
+ ASLR + No execute
+ DXVA 2.0
+ EVR
+ GPU Management
+ DirectX 10


AFAIK DX10 is runnable on XP so changes in architecture of kernel of Vista is not so big as MS claims.
DXVA2, EVR, and GPU management are part of DX10 (they should be in DX from long time)...
ASLR is hmmm very controversial, current implementations in Vista have some flaws... also im not convinced that this is proper way for more secure personal computing...

You forgot mention about some flaws - lack system support for hardware audio acceleration (so proprietary solutions are created which is no good from market point of view)



And all of this should add up to the Desktop Multimedia Experience even on older Machines (except ASLR + No execute) that it currently doesn't is a Design Flaw imho in the complexity of it for sure Windows 7 will do better here.

As a user, as a customer i can't agree with such optimistic vision of world.

Please, i think that this is not right place to talk about architecture of Vista - i only says about Vista and XP due arguments of Ben about support for VC-1 (which is MS child and as a creator MS should take care about popularization of this solution). From my experience, my observation, my knowledge Vista is slower than XP, VC-1 is not twice faster than h.264.
Also personally i don't think that VC-1 is worse than h.264, i only says that VC-1 with available on market solutions (from my point of view lack of good open source implementations of the VC-1 marginalize this codec) can't be compared to the h.264 at this moment as Ben do.

[benwaggoner]

Ben, I want to request the support of full 5.1 AAC audio for the MP4 Silverlight support both at 44.1 Khz and 48 Khz and if possible at 96 KHz also. Presently, Silverlight does not even support 5.1 VC1 audio and only supports 44.1 Khz VC1 playback, which is a big limitation for providing HD quality. I found 96 Khz VC1s to sound much more crisper than 44.1 Khz VC1s. So please add support for 96 KHz audio for VC1 also. Thanks.
VC-1 is a video codec, so it's entirely orthogonal to the audio issue.

Silverlight 2 is 44.1 stereo audio rendering only. Honestly, as a web browser plugin we haven't had much demand for multichannel audio in Silverlight yet. That seems to be used much more in classic media file playback, where a traditional media player app could be used.

Remember, Silverlight is really only used inside a browser, and those are the scenarios we're most focused on.

That said, if anyone has as scenario where they'd publish in 5.1 in Silverlight, I'd love to hear it. We're very customer driven in our development, so I always love feedback on enabling features we could add.

[smok3]

with he aac it should be possible, example
http://somestuff.org/flashAVC/flvplayer.php?moviename=movies/BBB-1080downsized-x1280y720.mp4
(flash downmixes to stereo i think)

direct link
http://somestuff.org/flashAVC/movies/BBB-1080downsized-x1280y720.mp4

why not? decoding speed issue?
(anyway, not just 6 channels, i want to have 3 channel option, 4 channel option, 8 channel option, ect)

We're very customer driven in our development, so I always love feedback on enabling features we could add.

ok, i'am not and will not be your costumer, so you may ignore me actually.

[Ranguvar]

AFAIK DX10 is runnable on XP so changes in architecture of kernel of Vista is not so big as MS claims.
There was a project to get DX10 working on XP, but it never reached a usable level. If MS wanted at all to put it on XP, I'm sure it wouldn't be difficult at all. But it doesn't work right now. Unless you mean the hacks out there to get Vista-only games (Halo 2 for PC) working on XP?

I do want to at least try Vista, but I can't... damn black screen of death (http://forums.nvidia.com/index.php?showtopic=33671) (not referring to the "piracy"-related one) :rolleyes:

[benwaggoner]

x264 is a H.264 encoder and MP4 is a container. Since Silverlight will support H.264, it will obviously support x264... A few posts ago, ben also mentioned it'll support MP4.
And MP4 (the container) only supports H.264. The "codecs" you listed are H.264 encoders.
And in fact some of the demo clips we're showing here at IBC are x264 encoded. They work fine. Naturally, we're testing against a variety of H.264 encoders using a variety of settings. We've got a big fuzz test library of interestingly malformed MPEG-4 files and H.264 bitstreams as well.

But I think the HE-AAC question is unanswered :P
That is correct :).

We'll have a FAQ up before long going into all that kind of detail.

We're not going to support every possible variant of MPEG-4 and its codecs initially, of course. Beyond the issues of developing and testing everything, we want to keep the Silverlight installer size reasonable.

Feedback is always appreciate on what you want folks want to see supported, and how you'd use it in Silverlight.

[Golgot13]

Good for what? We didn't use it instead of XAML because SVG wasn't designed to be able to build interactive UX. Silverlight can access the browser DOM, so we could control SVG playback in a web page on a browser that supports it.


IE browser don't support well SVG, and with SL. Many persons think that IE8 will not support it.
MS don't like market and standard (you can read some comment from Sir Tim Berners-Lee).
May be next year MS will implement it (like H264) ?


I hope to see your demo at IBC, Ben. Did you make same demo with VC1 too?
Why: to show what new VC1 encoder optimization can do.

[benwaggoner]

MS don't like market and standard (you can read some comment from Sir Tim Berners-Lee).
Well, we do a TON of stuff based on standards. But there are just so many in existance, we have to pick and choose for particular products based on what they enable.

I hope to see your demo at IBC, Ben. Did you make same demo with VC1 too? Why: to show what new VC1 encoder optimization can do.
You know, I didn't actually do any Apples-and-apples VC-1 v. H.264 encodes for the show. We've got plenty of stuff in both codecs, but I'm not sure we did any with the precise same settings. VC-1 v. H.264 isn't part of our story here.

But if you swing by, bring some content on a USB drive and we'll see what we can whip out. You can also try our (admittedly very limited; really just a couple of podcasting presets at this point) H.264 export out of the Expression Encoder build.

[bond]

i have some questions too

1) maybe i have misread it but you wrote that m$ is going to implement their own h.264 decoder.
Will you also create your own h.264 encoder or will you reuse a h.264 encoder from another company in your tools

2) is the goal of microsoft to make your decoder support full h.264 or only a subset?

3) is the goal of microsoft to make your encoder support full h.264 or only a subset?

4) is the goal of microsoft to provide the world fastest h.264 decoder?

5) is the goal of microsoft to provide the h.264 encoder with the best quality per filesize on the market?

6) will silverlight be able to play the same internet h.264-in-mp4 streams that the current flash player is able to play without needing to change anything in the current flash player?

7) will the current flash player be able to play the same internet h.264-in-mp4 streams that silverlight is able to play without needing to change anything in the current flash player?

:thanks:

[Sharktooth]

2 and 3) i have doubts about full set. it depends in what they are asked to add by the bigger commercial partners.
4) i dont think so.
5) i dont think so.
6) probably yes, it depends in what profiles and subsets their decoder will support
7) probably yes since the flash decoder is a full set decoder.
:p

[bond]

i forgot to add that i would be happy if benwaggoner could answer, and not someone who is only guessing :p

[Sharktooth]

you forgot to look at the -> :p

[hurry]

Will you also create your own h.264 encoder or will you reuse a h.264 encoder from another company in your tools

Imo, an intelligent choice would be to select the fully open source, continuously developed, highest quality among all H.264 codecs and the fastest encoding x264 codec as the default H.264 encoding engine in the MS Expression Encoder.

[hurry]

Ben wrote: Silverlight 2 is 44.1 stereo audio rendering only. Honestly, as a web browser plugin we haven't had much demand for multichannel audio in Silverlight yet. That seems to be used much more in classic media file playback, where a traditional media player app could be used. Remember, Silverlight is really only used inside a browser, and those are the scenarios we're most focused on. That said, if anyone has as scenario where they'd publish in 5.1 in Silverlight, I'd love to hear it. We're very customer driven in our development, so I always love feedback on enabling features we could add.

Imo, full HD web video means support for 96 KHz audio and 5.1 sound. With more and more HD video content including Hollywood movies being sold on the net as playable and downloadable files, I think support for 48 and 96 KHz audio and 5.1 sound is a must if Silverlight has to compete with Flash for the mode of delivery for online HD video. When we receive a HD video disc, it contains 96 Khz audio and 5.1 sound. So if internet is to become a viable option for HD video delivery, then 96 Khz 5.1 audio has to be supported for online players in the browser. It is a matter of who will do it first, SL or Flash. I use a web video playlist jukebox player which has an advanced search option and supports playback of video both in SL and Flash in the left pane according to the format of the video in the playlist in the right pane.

[Golgot13]

@bond and Ben

I'm not sure that MS will developp some H264 encoder and decoder.
They will buy some H264 software (not license but globally) or a company which make it.
Why I said this: because I surprise that MS guys don't know all feature of first VC1 core encoder...
But MS is not alone to do this some eg:
- Sonic use Mainconcept's SDK and said that Cinevision don't use it
(because client could not understand difference between 40K$ of Cinevision and 10K$ of Mainconcept Encoder)
- And DiVX company bought directly Mainconcept company :rolleyes:

There is not lot of H264 specialist companies, so I can recommand at MS to choose Elecard
But USA, and many other countries, don't support Russia today...
(Russians are nice guys, only russian politician like all other politicians are evil...).

[Quark.Fusion]

until Vista is not separated from slowing Vista components then Vista is slow.
[…]
AFAIK DX10 is runnable on XP so changes in architecture of kernel of Vista is not so big as MS claims.
DXVA2, EVR, and GPU management are part of DX10 (they should be in DX from long time)...
ASLR is hmmm very controversial, current implementations in Vista have some flaws... also im not convinced that this is proper way for more secure personal computing...

You forgot mention about some flaws - lack system support for hardware audio acceleration (so proprietary solutions are created which is no good from market point of view)

[…]

Please, i think that this is not right place to talk about architecture of Vista - i only says about Vista and XP due arguments of Ben about support for VC-1 (which is MS child and as a creator MS should take care about popularization of this solution). From my experience, my observation, my knowledge Vista is slower than XP, VC-1 is not twice faster than h.264.
Sorry for continuing offtopic, but:
1) you can separate some bloat with vLite
2) AFAIK DX10 requires new driver model and not compatible with XP, also GPU management include virtual video memory manager.
3) True, it's lack audio acceleration, but on other hand it allows to control audio per-application, which is
excellent feature and AFAIK games can use OpenAL.

To bring back to topic — all this means no full (with post-processing) hardware acceleration of decoding before OpenCL, as architecture needs some changes. (two-way access to hardware video decoder)

[benwaggoner]

i have some questions too

1) maybe i have misread it but you wrote that m$ is going to implement their own h.264 decoder.
Will you also create your own h.264 encoder or will you reuse a h.264 encoder from another company in your tools
We're doing our own encoder as well (we're showing an early version at the show). We'll be compatible with any compliant H.264 encoder, of course.

2) is the goal of microsoft to make your decoder support full h.264 or only a subset?
Subset. I don't think the web needs High 4:4:4 interlaced :).

We'll be putting up a FAQ detailing Profile@Level support in the next few weeks.

3) is the goal of microsoft to make your encoder support full h.264 or only a subset?
Subset, of cousre. See above :).

4) is the goal of microsoft to provide the world fastest h.264 decoder?

5) is the goal of microsoft to provide the h.264 encoder with the best quality per filesize on the market?
We're trying to make a generally good one for Silverlight scenarios to start out with. Initial results are promsing, but I'm not going to make any specific predictions about performance or quality until we'll farther along in testing. I'm sure all of you will share MANY thoughts on it once we ship :).

6) will silverlight be able to play the same internet h.264-in-mp4 streams that the current flash player is able to play without needing to change anything in the current flash player?
No, Flash uses a propritary, undocumented streaming protocol, so we won't be able to play back streams off FMS.

One of our demos here at NAB is streaming H.264 and AAC to Silverlight via Windows Media Services.

7) will the current flash player be able to play the same internet h.264-in-mp4 streams that silverlight is able to play without needing to change anything in the current flash player?
The Flash client only supports streaming using their own propritary codec, so it won't be able to play streams compatible with Silverlight.

There should definitely be a large overlap in what each player is capable of via progressive download.

[benwaggoner]

Imo, full HD web video means support for 96 KHz audio and 5.1 sound. With more and more HD video content including Hollywood movies being sold on the net as playable and downloadable files, I think support for 48 and 96 KHz audio and 5.1 sound is a must if Silverlight has to compete with Flash for the mode of delivery for online HD video. When we receive a HD video disc, it contains 96 Khz audio and 5.1 sound. So if internet is to become a viable option for HD video delivery, then 96 Khz 5.1 audio has to be supported for online players in the browser. It is a matter of who will do it first, SL or Flash. I use a web video playlist jukebox player which has an advanced search option and supports playback of video both in SL and Flash in the left pane according to the format of the video in the playlist in the right pane.
The use you describe at the end sounds interesting. Of course, Flash is limited to stereo only.

Not about Silverlight in particular (doing 96 KHz is way easier than going to 5.1 mixing, since the latter requires speaker fill, 2D positioning, and a lot of other new API stuff), but I'm not personally convinced that > 48 KHz makes any sense for content delievery. Even assuming problematic audio mastering, I think 48 Khz 20-bit is about the limit of what even a golden ear with great headphones can here, and 16-bit is essentially transparent assuming good dithering.

[benwaggoner]

@bond and Ben
I'm not sure that MS will developp some H264 encoder and decoder.
We're really building our own. Come by the booth and see.

Bring a USB drive and I'll let you take one of the files to play around with.

[lcx]

Would be great if Microsoft Silverlight can add caption support using
W3C Timed Text: http://www.w3.org/AudioVideo/TT/

and

3GPP timed text: http://www.3gpp.org/ftp/Specs/html-info/26245.htm

Both are supported by Adobe Flash.
Accessibility via caption support is required for most Government projects.

[Comatose]

No, Flash uses a propritary, undocumented streaming protocol, so we won't be able to play back streams off FMS.

One of our demos here at NAB is streaming H.264 and AAC to Silverlight via Windows Media Services.


The Flash client only supports streaming using their own propritary codec, so it won't be able to play streams compatible with Silverlight.
He was referring to using something like JW flash player to play H.264-in-MP4 over HTTP, which is clearly not proprietary...

And what? That's not true.
All you have to do to be compliant with Flash is use standards compliant H.264, AAC and MP4.

[hurry]

Not about Silverlight in particular (doing 96 KHz is way easier than going to 5.1 mixing, since the latter requires speaker fill, 2D positioning, and a lot of other new API stuff), but I'm not personally convinced that > 48 KHz makes any sense for content delievery. Even assuming problematic audio mastering, I think 48 Khz 20-bit is about the limit of what even a golden ear with great headphones can here, and 16-bit is essentially transparent assuming good dithering.

My experience is that directly exporting VC1 from Vegas with 96 KHz 24-bit Stereo audio produces a much more crisper, sharper and clearer sound than 44.1 or 48 Khz 8-bit or 16-bit audio even for only voice what to speak of music. I could make out the big difference even with a normal sound card and headphones. And one can directly import a m2ts file into Vegas and export it as VC1 with 96 Khz 24-bit stereo audio. That is a big advantage for VC1. So I really really hope you can add at least 96 Khz 24-bit audio support in SL if not 5.1 sound.

[benwaggoner]

My experience is that directly exporting VC1 from Vegas with 96 KHz 24-bit Stereo audio produces a much more crisper, sharper and clearer sound than 44.1 or 48 Khz 8-bit or 16-bit audio even for only voice what to speak of music. I could make out the big difference even with a normal sound card and headphones. And one can directly import a m2ts file into Vegas and export it as VC1 with 96 Khz 24-bit stereo audio. That is a big advantage for VC1. So I really really hope you can add at least 96 Khz 24-bit audio support in SL if not 5.1 sound.
96 KHz sounding better than 48 KHz for VOICE? That suggests something wrong with the audio mastering chain upstream from the codec, or radically different bitrates used. And if you're coming from m2ts source, you're almost certianly starting at 48 KHz.

No WMA codec has an 8-bit mode. 8-bit source is of course hideously pathological.

[hurry]

96 KHz sounding better than 48 KHz for VOICE? That suggests something wrong with the audio mastering chain upstream from the codec, or radically different bitrates used.

96 Khz 24-bit stereo WMA VC1 does sound better for voice too. I used the standard Vegas profile for encoding to vc1 so I don't know what may have gone wrong. But I remember I had checked it thoroughly and repeatedly heard the various file outputted.

And if you're coming from m2ts source, you're almost certianly starting at 48 KHz.

The AVCHD m2ts file from my Sony SR1 camera has dolby 5.1 sound since the camera has 5 mics or at least they say so. And I have found that directly importing the m2ts and rendering it to 96 KHz, 24-bit vc1 gives a much crisper sound. Even when I record in the lower quality mpg (non-5.1 sound) with the camera and publish it to 96 KHz 24-bit vc1, it gives more crisper and clearer sound. Aren't the Blu-Ray players and Creative Audio cards supporting 24-bit and at least 96 KHz audio (if not more) because the sound is much crisper and clearer?

This may not be relevant to this thread but I just want to add that there is big problem in trying to encode a 5.1 avchd m2ts or 5.1 mpg file from the camera to 5.1 wmv vc1 file. It always gives an error. Many have reported this on the steve-digicams and other hd forums. It gets encoded fine to stereo vc1 but the moment you select 5.1 in the WMA 10 Pro codec, it gives an error. I hope MS fixes this asap because m2ts avchd files with 5.1 are fast becoming the standard mode of recording in HD camcorders. And if we lose the 5.1 advantage after converting to vc1, then people would want to try to encode to h.264 5.1 which works fine. Thanks.

[Comatose]

Saying you hear a difference with 96 KHz with a normal sound card (onboard) is silly because almost none of them can do 96 KHz.
It just gets downsampled. It's kinda like how Avisynth internally treats all audio as 8bit.

[hurry]

Saying you hear a difference with 96 KHz with a normal sound card (onboard) is silly because almost none of them can do 96 KHz.

Oh sorry, I have the 3 years old Creative Audigy 2NX which supports 96 Khz 24-bit. And I guess all audio cards nowadays (even built-in) are HD audio which means they support 96 Khz 24-bit. And WMP and VLC player play 24-bit 96 KHz perfectly.

[hurry]

Ben, I have another request. We wanted some compatibility overlap in the H.264 files which can be played in Silverlight and also simultaneously compatible on standalone Blu-ray players for TV as we are discussing in the other thread. It will enable us to publish one HD video file for web SL playback, Blu-ray players and even for broadcast and save us a lot of storage space as Hd videos are quite large in size. This would be possible if Silverlight adds support for the m2ts container (H.264) which is fast becoming the standard for Blu-ray players and HD broadcasting. It would make SL one-up on Flash for HD video.

You wrote: We'll have a native MPEG-4 parser. We also have the MediaStreamSource feature of Silverlight whcih allows parsers to be built in managed code. So it's something an enterprising devleoper could write for Silverlight without us having to explicitly build it in.

Would someone be able to add m2ts playback support to Silverlight via this MediaStreamSource feature since m2ts is also basically H.264 if I remember correctly? Though it would actually be the best if MS can add m2ts playback support in SL natively. It will go a big way in bridging the gap between web video and standalone Blu-ray distributable video.

It has been reported that only the Sony Playstation Blu-ray hardware can presently play Silverlight compatible web vc1 videos. Is MS planning to push .wmv container (vc1) support in hardware Blu-ray players? I have heard that many if not most Blu-ray players will not support wmv container but will support only m2ts container.

[benwaggoner]

Ben, I have another request. We wanted some compatibility overlap in the H.264 files which can be played in Silverlight and also simultaneously compatible on standalone Blu-ray players for TV as we are discussing in the other thread. It will enable us to publish one HD video file for web SL playback, Blu-ray players and even for broadcast and save us a lot of storage space as Hd videos are quite large in size. This would be possible if Silverlight adds support for the m2ts container (H.264) which is fast becoming the standard for Blu-ray players and HD broadcasting. It would make SL one-up on Flash for HD video.
Well, Blu-ray encodes are much higher bitrates and frame sizes than are appropriate for web-delivered content like Silverlight uses. There may be technical compatibility, but the scenarios are different enough that it's certainly not something we're focusing on.

Would someone be able to add m2ts playback support to Silverlight via this MediaStreamSource feature since m2ts is also basically H.264 if I remember correctly? Though it would actually be the best if MS can add m2ts playback support in SL natively. It will go a big way in bridging the gap between web video and standalone Blu-ray distributable video.
M2TS is a file format, so orthogonal to codecs (it can contain MPEG-2, H.264, or VC-1). But yes, a demuxer could be written in Silverlight's MSS.

It has been reported that only the Sony Playstation Blu-ray hardware can presently play Silverlight compatible web vc1 videos. Is MS planning to push .wmv container (vc1) support in hardware Blu-ray players? I have heard that many if not most Blu-ray players will not support wmv container but will support only m2ts container.
WMV isn't part of the Blu-ray spec. There certainly have been players that support it for DLNA or other technologies they support, like the PS3, but that's something different than their BD functionality.

Files encoded for Silverlight would be extremely unlikely to be BD-compliant. When we're doing HD, for example, we're typically going to use longer GOPs and sometimes bigger buffers than BD mandates.

We're focusing on interoperabilty with existing H.264 for/on the web.

[hurry]

Thanks Ben for your detailed answers. My further research seems that it boils down to only one thing. -> Dolby Digital (AC-3) audio support in Silverlight. If you can kindly add this much natively into SL, we could use Silverlight's MSS to write a demuxer for .ts (m2ts) files, and use our .ts file both on the web via Silverlight and burn and distribute those same files via playable Blu-ray data discs too. We plan to make HD video files at 2 to 3 MBps which are very playable on the web via Silverlight and still give great quality to be viewed on standalone Blu-ray players. Both VC1 and x264 are great for the video codec at 2 or 3 Mbps even for 1080p. So we can produce .ts (m2ts) files with (vc1 + ac3) or (x264 + ac3). If you will not add DD AC-3 support to Silverlgiht, can I get it added via SL MSS too?

[benwaggoner]

Thanks Ben for your detailed answers. My further research seems that it boils down to only one thing. -> Dolby Digital (AC-3) audio support in Silverlight. If you can kindly add this much natively into SL, we could use Silverlight's MSS to write a demuxer for .ts (m2ts) files, and use our .ts file both on the web via Silverlight and burn and distribute those same files via playable Blu-ray data discs too. We plan not to make HD video files at 2 to 3 MBps which are very playable on the web via Silverlight and still give great quality to be viewed on standalone Blu-ray players. Both VC1 and x264 are great for the video codec at 2 or 3 Mbps even for 1080p. So we can produce .ts (m2ts) files with (vc1 + ac3) or (x264 + ac3). If you will not add DD AC-3 support to Silverlgiht, can I get it added via SL MSS too?
Interesting idea. The Silverlight 2 MSS doesn't have the abllity to decode as raw audio (the output of MSS needs to be a compressed bitstream to one of our natively decoded formats), but that's a very interesting feature request.

That said, if you're encoding for the web, why not reenode audio at the same time. AAC and WMA 10 Pro are both a lot more efficient.

[Inventive Software]

@hurry: Those Audigy's don't support 24-bit 96 kHz. Read up on it, don't read what Creative said. ;)

[hurry]

Interesting idea. The Silverlight 2 MSS doesn't have the abllity to decode as raw audio (the output of MSS needs to be a compressed bitstream to one of our natively decoded formats), but that's a very interesting feature request.

The problem would be fully solved if you can add .ac3 and .ts (m2ts) support natively to Silverlight. I don't think that will increase the size that much. The mandatory audio codec for Blu-ray is ac3 (DD) and additionally some Blu-ray players may support mp3, aac etc. but that support is not going to be universal. I went through the Blu-ray players available in the market and hardly 10% (only the older ones) have support for mp3.

That said, if you're encoding for the web, why not reencode audio at the same time. AAC and WMA 10 Pro are both a lot more efficient.

The whole point is to produce a single HD video file which plays both in the web browser and on the HD hardware (Blu-ray). And I think Silverlight can make that happen since you are listening to us on Doom9 unlike Adobe. You can use that as one of the unique and outstanding features of Silverlight over Flash.

The problem is that it takes a huge amount of time to render a m2ts avchd file from the camera after editing, adding effects etc. in Vegas to Vc1 for the web. Around 8 hours for a 1 hour video with chroma etc. on a Dual Core. I don't want to do this rendering for the second time for the Blu-ray hardware.

For a HD video file of 2 or 3 Mbps, a few 100 kbps does not matter much so I don't mind increasing the bit rate of the .ac3 stream to 200-400 kbps to match the aac or wma 10 pro quality because it saves me a whopping 10 hours of reencoding the video again for Blu-ray hardware players. It will also save me considerable storage space locally as well as on the web server (it will be halved). In the US, many view the videos on the computer, but my viewership audience is also in many countries where 90% people prefer to view videos on their TV so I am forced to make the second Blu-ray version available.

You are right that if we are looking at something like 6-8 Mbps or 36MBps quality for the Blu-ray discs then we will have to produce 2 files for the web & Blu-ray. But if we making low-budget HD videos (there are many producers who do this) and with the cool codecs like vc1 and x264 we get quite nice full-screen quality at 2 to 3 Mbps, we can distribute many more videos on one Blu-ray and maximize our distribution. I hope you understand. Please do consider .ac3 and .ts support for Silverlight so that we can have this option of one file for both. Thanks so much.

[Sagekilla]

@Hurry: Couldn't you render the video once, then do two different versions with different audio tracks? I agree that it would be great to have AC3 support, but would this not be a good alternative for the time being?

[benwaggoner]

The problem would be fully solved if you can add .ac3 and .ts (m2ts) support natively to Silverlight

The whole point is to produce a single HD video file which plays both in the web browser and on the HD hardware (Blu-ray). And I think Silverlight can make that happen since you are listening to us on Doom9 unlike Adobe. You can use that as one of the unique and outstanding features of Silverlight over Flash.
The challenge is that Silverlight is a software renderer, and optimized for web rate content, not short-GOP video with 40 Mbps peaks.

Silverlight 2 should be able to do 1080p24 VC-1 on a fast machine, but probably not at BD rates, and definitely not wth H.264 at BD rates.

Like most web use, Silverlight targeted "HD" content is normally 720p24 with 2-4 second GOPs around 2-6 Mbps.

[hurry]

Like most web use, Silverlight targeted "HD" content is normally 720p24 with 2-4 second GOPs around 2-6 Mbps.

Precisely. I am talking of producing videos in the range of 2 - 6 Mbps video which will play on the web in Silverlight, in the Blu-ray hardware and for broadcast also. I know of many video publishers who produce video in this range and would immensely benefit in time, resources etc. by having to produce only one file for the web and Blu-ray playback. In fact this would greatly promote Silverlight adoption among these low and middle range video producers. And one can produce pretty sharp and vivid videos at 720p with m2ts and ac3 in this range of 2-6 Mbps at lower GOPs so adding native support for these two in Silverlight will certainly be used by many. Maybe it would herald the beginning of m2ts video for the web. I think Adobe Flash may also eventually take this path. Thanks.

[hurry]

@Hurry: Couldn't you render the video once, then do two different versions with different audio tracks? I agree that it would be great to have AC3 support, but would this not be a good alternative for the time being?

That is a nice idea for the time being. But that still will not solve the problem of the trouble of uploading two big HD video files and providing double storage space on the server as well as locally for the 2 HD video files for backup.

What would be the best way (software) to mux the vc1 or x264 video track with the ac3 and aac (or wma) audio tracks into m2ts (.ts) and .mp4 (or .wmv for web) files respectively? Thanks.

[Sagekilla]

Ah I see, you're using x264 for a hardware player and VC-1 for a software player. Why not just use one, like x264 for both? It would mean you wouldn't have to render the video twice, but the audio would need to be transcoded twice (Which takes a fraction of the time) Then you can simply mux x264 + AC3 into a .m2ts and then the x264 + AAC into a .mp4, I believe this would work.

Software wise:
x264 for H.264 (obviously)
mp4box for mp4
tsmuxer (I think? I'm not sure what's generally recommended for m2ts) for your m2ts.

[hurry]

Thanks Sagekilla. I plan on using either x264 or vc1 for both Web and Blu-ray not both depending on editing process of the video. Which GUI will help me to mux x264 + AC3 into .ts and x264 + AAC into .mp4 files? I am not good in command lines.

[Sagekilla]

As I said, MP4box for MP4s and tsmuxer for m2ts. You can easily find both on google (They should be the first or second search result)

[nm]

Well, MP4Box is a command-line tool. Try Yamb (http://yamb.unite-video.com/) for a Windows GUI.

However, I'm a bit intrigued how Hurry is having a video catalogue so huge that duplicate storage is a problem and still wants to use GUI tools for muxing instead of scripts ;)

[Sagekilla]

Sorry, I keep thinking MP4Box and mean YAMB. My apologies.

[Golgot13]

GoP of 2-4 second are not supported (well) by BD specification.
At atble 9-26 and 9-27 of BD specification (BD Rom 2 Part3),
GoP up to 2second is supported but with restriction of bitrate (2-6Mbps is ok).

[puffpio]

Like most web use, Silverlight targeted "HD" content is normally 720p24 with 2-4 second GOPs around 2-6 Mbps.

I think that's great because 720p H264 in an MP4 container played back in Flash loses audio sync if you don't have the horsepower..if your implementation of the decoder can be at least as efficient as ffdshow that I would be happy.


An earlier post you asked for a use case/scenario for multichannel audio in the browser, and all I can think of is for HD movie trailers. people seem to like HD movie trailers and that would be a compelling experience and a good showcase for silverlight

[benwaggoner]

I think that's great because 720p H264 in an MP4 container played back in Flash loses audio sync if you don't have the horsepower..if your implementation of the decoder can be at least as efficient as ffdshow that I would be happy.
Ugh. There's really no excuse to lose sync. Drop frames, sure, but sync ain't that hard.

At IBC we were demoing 960x540p30 @ 6 Mbps Baseline without any trouble, FWIW.

An earlier post you asked for a use case/scenario for multichannel audio in the browser, and all I can think of is for HD movie trailers. people seem to like HD movie trailers and that would be a compelling experience and a good showcase for silverlight
Yeah, that's the one I was thinking of. But what percentage of web-connected machines actually have > stereo?

[puffpio]

Ugh. There's really no excuse to lose sync. Drop frames, sure, but sync ain't that hard.

At IBC we were demoing 960x540p30 @ 6 Mbps Baseline without any trouble, FWIW.

agreed. i remember in the early days of divx how easy it was to lose a/v sync. i would terminate all unnecessary processes out of my task manager just to make absolutely sure to minimize any random cpu usages other than the intended video.
of course those were the days when burning a cd required a vibration isolation mount for your computer desk :P sneeze and you had a coaster



Yeah, that's the one I was thinking of. But what percentage of web-connected machines actually have > stereo?

who knows, but it's a good product demo, and it's good to get entertainment industry types onboard. and could possibly be a good bullet point to get the non technical dreamers involved in things like this

[hurry]

But what percentage of web-connected machines actually have > stereo?

Almost all boards including the low-end ones have HD Audio card nowadays to which they can attach a 5.1 sound speakers or 5.1 headphones. I predict 5.1 headphones will become quite popular in days to come. I think Silverlight should support 5.1 AC3 sound for true HD quality web video.

[benwaggoner]

Almost all boards including the low-end ones have HD Audio card nowadays to which they can attach a 5.1 sound speakers or 5.1 headphones. I predict 5.1 headphones will become quite popular in days to come. I think Silverlight should support 5.1 AC3 sound for true HD quality web video.
Sure. I meant the percentage of people who actually have 5.1 speakers and are configured to use them.

Even for those with the right hardware, how many wind up being stuck in stereo mode after a Soundblaster driver update :)?

[MySchizoBuddy]

hmm Ben said their is no Hardware acceleration infrastucture on macs.
So how is flash 10 hardware acc on macs, or it isn't?

[benwaggoner]

hmm Ben said their is no Hardware acceleration infrastucture on macs.
So how is flash 10 hardware acc on macs, or it isn't?
Flash 9 just used hardware acceleration for scaling up the final composite to full screen in full screen mode.

Flash 10 is adding hardware compositing for bitmaps, IIRC (not vectors, though).

So, it's "hardware accelerated" playback but not in the decoders themselves. This frees up some CPU for decode, but a software decode can easily remain the long pole.