H.264 support coming in Silverlight

[Comatose]

x264 is a H.264 encoder and MP4 is a container. Since Silverlight will support H.264, it will obviously support x264... A few posts ago, ben also mentioned it'll support MP4.
And MP4 (the container) only supports H.264. The "codecs" you listed are H.264 encoders.

But I think the HE-AAC question is unanswered :P

[hurry]

Thanks Comatose. Ben, can you please tell us if SL will support the Nero HE-AAC and HE-AAC V2 audio? HE-AAC is a highly efficient codec as we all know and it would be great if SL supported it.

[Sharktooth]

We'll have a native MPEG-4 parser.
here it is.
And MP4 (the container) only supports H.264.
not properly correct, MP4 supports all MPEG4 video formats (MP4 supports also MPEG4-ASP, like Xvid/DivX/3ivX/etc, for example) and everything else in the form of private streams.

[Comatose]

Ugh, okay, I failed once more.

[hurry]

Ben, I want to request the support of full 5.1 AAC audio for the MP4 Silverlight support both at 44.1 Khz and 48 Khz and if possible at 96 KHz also. Presently, Silverlight does not even support 5.1 VC1 audio and only supports 44.1 Khz VC1 playback, which is a big limitation for providing HD quality. I found 96 Khz VC1s to sound much more crisper than 44.1 Khz VC1s. So please add support for 96 KHz audio for VC1 also. Thanks.

[pandy]

@pandy
read what i wrote the slowdown of Vista doesn't come from the Core it comes from the massive amount of stuff that runs above it.


Well - until Vista is not spearated from this stuff then Vista is slow, maybe next OS from MS will be faster - i don't know. At this moment for playing faster VC-1 i'm must install Vista...
Where is logic?


If i refer to Vista or XP i refer to the Core not the gadgetry that comes with it :P


until Vista is not separated from slowing Vista components then Vista is slow.


Vista should be a lot more responsive a little more secure and faster if you bring it down to XP complexity (on the same old hardware).


Maybe yes, but maybe not - we don't know such thing, we can only speculate (and speculation is very close to this what taken me to write inside thread - Ben speculate that VC-1 is twice faster than h.264 - i can't confirm such speculation from my experience)


The main Features for me in Vista are only these.

+ More Efficient I/O and Virtual Memory management (dummy pages, more efficient page file use lifted restrictions dynamic decissioning)
+ ASLR + No execute
+ DXVA 2.0
+ EVR
+ GPU Management
+ DirectX 10


AFAIK DX10 is runnable on XP so changes in architecture of kernel of Vista is not so big as MS claims.
DXVA2, EVR, and GPU management are part of DX10 (they should be in DX from long time)...
ASLR is hmmm very controversial, current implementations in Vista have some flaws... also im not convinced that this is proper way for more secure personal computing...

You forgot mention about some flaws - lack system support for hardware audio acceleration (so proprietary solutions are created which is no good from market point of view)



And all of this should add up to the Desktop Multimedia Experience even on older Machines (except ASLR + No execute) that it currently doesn't is a Design Flaw imho in the complexity of it for sure Windows 7 will do better here.

As a user, as a customer i can't agree with such optimistic vision of world.

Please, i think that this is not right place to talk about architecture of Vista - i only says about Vista and XP due arguments of Ben about support for VC-1 (which is MS child and as a creator MS should take care about popularization of this solution). From my experience, my observation, my knowledge Vista is slower than XP, VC-1 is not twice faster than h.264.
Also personally i don't think that VC-1 is worse than h.264, i only says that VC-1 with available on market solutions (from my point of view lack of good open source implementations of the VC-1 marginalize this codec) can't be compared to the h.264 at this moment as Ben do.

[benwaggoner]

Ben, I want to request the support of full 5.1 AAC audio for the MP4 Silverlight support both at 44.1 Khz and 48 Khz and if possible at 96 KHz also. Presently, Silverlight does not even support 5.1 VC1 audio and only supports 44.1 Khz VC1 playback, which is a big limitation for providing HD quality. I found 96 Khz VC1s to sound much more crisper than 44.1 Khz VC1s. So please add support for 96 KHz audio for VC1 also. Thanks.
VC-1 is a video codec, so it's entirely orthogonal to the audio issue.

Silverlight 2 is 44.1 stereo audio rendering only. Honestly, as a web browser plugin we haven't had much demand for multichannel audio in Silverlight yet. That seems to be used much more in classic media file playback, where a traditional media player app could be used.

Remember, Silverlight is really only used inside a browser, and those are the scenarios we're most focused on.

That said, if anyone has as scenario where they'd publish in 5.1 in Silverlight, I'd love to hear it. We're very customer driven in our development, so I always love feedback on enabling features we could add.

[smok3]

with he aac it should be possible, example
http://somestuff.org/flashAVC/flvplayer.php?moviename=movies/BBB-1080downsized-x1280y720.mp4
(flash downmixes to stereo i think)

direct link
http://somestuff.org/flashAVC/movies/BBB-1080downsized-x1280y720.mp4

why not? decoding speed issue?
(anyway, not just 6 channels, i want to have 3 channel option, 4 channel option, 8 channel option, ect)

We're very customer driven in our development, so I always love feedback on enabling features we could add.

ok, i'am not and will not be your costumer, so you may ignore me actually.

[Ranguvar]

AFAIK DX10 is runnable on XP so changes in architecture of kernel of Vista is not so big as MS claims.
There was a project to get DX10 working on XP, but it never reached a usable level. If MS wanted at all to put it on XP, I'm sure it wouldn't be difficult at all. But it doesn't work right now. Unless you mean the hacks out there to get Vista-only games (Halo 2 for PC) working on XP?

I do want to at least try Vista, but I can't... damn black screen of death (http://forums.nvidia.com/index.php?showtopic=33671) (not referring to the "piracy"-related one) :rolleyes:

[benwaggoner]

x264 is a H.264 encoder and MP4 is a container. Since Silverlight will support H.264, it will obviously support x264... A few posts ago, ben also mentioned it'll support MP4.
And MP4 (the container) only supports H.264. The "codecs" you listed are H.264 encoders.
And in fact some of the demo clips we're showing here at IBC are x264 encoded. They work fine. Naturally, we're testing against a variety of H.264 encoders using a variety of settings. We've got a big fuzz test library of interestingly malformed MPEG-4 files and H.264 bitstreams as well.

But I think the HE-AAC question is unanswered :P
That is correct :).

We'll have a FAQ up before long going into all that kind of detail.

We're not going to support every possible variant of MPEG-4 and its codecs initially, of course. Beyond the issues of developing and testing everything, we want to keep the Silverlight installer size reasonable.

Feedback is always appreciate on what you want folks want to see supported, and how you'd use it in Silverlight.

[Golgot13]

Good for what? We didn't use it instead of XAML because SVG wasn't designed to be able to build interactive UX. Silverlight can access the browser DOM, so we could control SVG playback in a web page on a browser that supports it.


IE browser don't support well SVG, and with SL. Many persons think that IE8 will not support it.
MS don't like market and standard (you can read some comment from Sir Tim Berners-Lee).
May be next year MS will implement it (like H264) ?


I hope to see your demo at IBC, Ben. Did you make same demo with VC1 too?
Why: to show what new VC1 encoder optimization can do.

[benwaggoner]

MS don't like market and standard (you can read some comment from Sir Tim Berners-Lee).
Well, we do a TON of stuff based on standards. But there are just so many in existance, we have to pick and choose for particular products based on what they enable.

I hope to see your demo at IBC, Ben. Did you make same demo with VC1 too? Why: to show what new VC1 encoder optimization can do.
You know, I didn't actually do any Apples-and-apples VC-1 v. H.264 encodes for the show. We've got plenty of stuff in both codecs, but I'm not sure we did any with the precise same settings. VC-1 v. H.264 isn't part of our story here.

But if you swing by, bring some content on a USB drive and we'll see what we can whip out. You can also try our (admittedly very limited; really just a couple of podcasting presets at this point) H.264 export out of the Expression Encoder build.

[bond]

i have some questions too

1) maybe i have misread it but you wrote that m$ is going to implement their own h.264 decoder.
Will you also create your own h.264 encoder or will you reuse a h.264 encoder from another company in your tools

2) is the goal of microsoft to make your decoder support full h.264 or only a subset?

3) is the goal of microsoft to make your encoder support full h.264 or only a subset?

4) is the goal of microsoft to provide the world fastest h.264 decoder?

5) is the goal of microsoft to provide the h.264 encoder with the best quality per filesize on the market?

6) will silverlight be able to play the same internet h.264-in-mp4 streams that the current flash player is able to play without needing to change anything in the current flash player?

7) will the current flash player be able to play the same internet h.264-in-mp4 streams that silverlight is able to play without needing to change anything in the current flash player?

:thanks:

[Sharktooth]

2 and 3) i have doubts about full set. it depends in what they are asked to add by the bigger commercial partners.
4) i dont think so.
5) i dont think so.
6) probably yes, it depends in what profiles and subsets their decoder will support
7) probably yes since the flash decoder is a full set decoder.
:p

[bond]

i forgot to add that i would be happy if benwaggoner could answer, and not someone who is only guessing :p

[Sharktooth]

you forgot to look at the -> :p

[hurry]

Will you also create your own h.264 encoder or will you reuse a h.264 encoder from another company in your tools

Imo, an intelligent choice would be to select the fully open source, continuously developed, highest quality among all H.264 codecs and the fastest encoding x264 codec as the default H.264 encoding engine in the MS Expression Encoder.

[hurry]

Ben wrote: Silverlight 2 is 44.1 stereo audio rendering only. Honestly, as a web browser plugin we haven't had much demand for multichannel audio in Silverlight yet. That seems to be used much more in classic media file playback, where a traditional media player app could be used. Remember, Silverlight is really only used inside a browser, and those are the scenarios we're most focused on. That said, if anyone has as scenario where they'd publish in 5.1 in Silverlight, I'd love to hear it. We're very customer driven in our development, so I always love feedback on enabling features we could add.

Imo, full HD web video means support for 96 KHz audio and 5.1 sound. With more and more HD video content including Hollywood movies being sold on the net as playable and downloadable files, I think support for 48 and 96 KHz audio and 5.1 sound is a must if Silverlight has to compete with Flash for the mode of delivery for online HD video. When we receive a HD video disc, it contains 96 Khz audio and 5.1 sound. So if internet is to become a viable option for HD video delivery, then 96 Khz 5.1 audio has to be supported for online players in the browser. It is a matter of who will do it first, SL or Flash. I use a web video playlist jukebox player which has an advanced search option and supports playback of video both in SL and Flash in the left pane according to the format of the video in the playlist in the right pane.

[Golgot13]

@bond and Ben

I'm not sure that MS will developp some H264 encoder and decoder.
They will buy some H264 software (not license but globally) or a company which make it.
Why I said this: because I surprise that MS guys don't know all feature of first VC1 core encoder...
But MS is not alone to do this some eg:
- Sonic use Mainconcept's SDK and said that Cinevision don't use it
(because client could not understand difference between 40K$ of Cinevision and 10K$ of Mainconcept Encoder)
- And DiVX company bought directly Mainconcept company :rolleyes:

There is not lot of H264 specialist companies, so I can recommand at MS to choose Elecard
But USA, and many other countries, don't support Russia today...
(Russians are nice guys, only russian politician like all other politicians are evil...).

[Quark.Fusion]

until Vista is not separated from slowing Vista components then Vista is slow.
[…]
AFAIK DX10 is runnable on XP so changes in architecture of kernel of Vista is not so big as MS claims.
DXVA2, EVR, and GPU management are part of DX10 (they should be in DX from long time)...
ASLR is hmmm very controversial, current implementations in Vista have some flaws... also im not convinced that this is proper way for more secure personal computing...

You forgot mention about some flaws - lack system support for hardware audio acceleration (so proprietary solutions are created which is no good from market point of view)

[…]

Please, i think that this is not right place to talk about architecture of Vista - i only says about Vista and XP due arguments of Ben about support for VC-1 (which is MS child and as a creator MS should take care about popularization of this solution). From my experience, my observation, my knowledge Vista is slower than XP, VC-1 is not twice faster than h.264.
Sorry for continuing offtopic, but:
1) you can separate some bloat with vLite
2) AFAIK DX10 requires new driver model and not compatible with XP, also GPU management include virtual video memory manager.
3) True, it's lack audio acceleration, but on other hand it allows to control audio per-application, which is
excellent feature and AFAIK games can use OpenAL.

To bring back to topic — all this means no full (with post-processing) hardware acceleration of decoding before OpenCL, as architecture needs some changes. (two-way access to hardware video decoder)

[benwaggoner]

i have some questions too

1) maybe i have misread it but you wrote that m$ is going to implement their own h.264 decoder.
Will you also create your own h.264 encoder or will you reuse a h.264 encoder from another company in your tools
We're doing our own encoder as well (we're showing an early version at the show). We'll be compatible with any compliant H.264 encoder, of course.

2) is the goal of microsoft to make your decoder support full h.264 or only a subset?
Subset. I don't think the web needs High 4:4:4 interlaced :).

We'll be putting up a FAQ detailing Profile@Level support in the next few weeks.

3) is the goal of microsoft to make your encoder support full h.264 or only a subset?
Subset, of cousre. See above :).

4) is the goal of microsoft to provide the world fastest h.264 decoder?

5) is the goal of microsoft to provide the h.264 encoder with the best quality per filesize on the market?
We're trying to make a generally good one for Silverlight scenarios to start out with. Initial results are promsing, but I'm not going to make any specific predictions about performance or quality until we'll farther along in testing. I'm sure all of you will share MANY thoughts on it once we ship :).

6) will silverlight be able to play the same internet h.264-in-mp4 streams that the current flash player is able to play without needing to change anything in the current flash player?
No, Flash uses a propritary, undocumented streaming protocol, so we won't be able to play back streams off FMS.

One of our demos here at NAB is streaming H.264 and AAC to Silverlight via Windows Media Services.

7) will the current flash player be able to play the same internet h.264-in-mp4 streams that silverlight is able to play without needing to change anything in the current flash player?
The Flash client only supports streaming using their own propritary codec, so it won't be able to play streams compatible with Silverlight.

There should definitely be a large overlap in what each player is capable of via progressive download.

[benwaggoner]

Imo, full HD web video means support for 96 KHz audio and 5.1 sound. With more and more HD video content including Hollywood movies being sold on the net as playable and downloadable files, I think support for 48 and 96 KHz audio and 5.1 sound is a must if Silverlight has to compete with Flash for the mode of delivery for online HD video. When we receive a HD video disc, it contains 96 Khz audio and 5.1 sound. So if internet is to become a viable option for HD video delivery, then 96 Khz 5.1 audio has to be supported for online players in the browser. It is a matter of who will do it first, SL or Flash. I use a web video playlist jukebox player which has an advanced search option and supports playback of video both in SL and Flash in the left pane according to the format of the video in the playlist in the right pane.
The use you describe at the end sounds interesting. Of course, Flash is limited to stereo only.

Not about Silverlight in particular (doing 96 KHz is way easier than going to 5.1 mixing, since the latter requires speaker fill, 2D positioning, and a lot of other new API stuff), but I'm not personally convinced that > 48 KHz makes any sense for content delievery. Even assuming problematic audio mastering, I think 48 Khz 20-bit is about the limit of what even a golden ear with great headphones can here, and 16-bit is essentially transparent assuming good dithering.

[benwaggoner]

@bond and Ben
I'm not sure that MS will developp some H264 encoder and decoder.
We're really building our own. Come by the booth and see.

Bring a USB drive and I'll let you take one of the files to play around with.

[lcx]

Would be great if Microsoft Silverlight can add caption support using
W3C Timed Text: http://www.w3.org/AudioVideo/TT/

and

3GPP timed text: http://www.3gpp.org/ftp/Specs/html-info/26245.htm

Both are supported by Adobe Flash.
Accessibility via caption support is required for most Government projects.

[Comatose]

No, Flash uses a propritary, undocumented streaming protocol, so we won't be able to play back streams off FMS.

One of our demos here at NAB is streaming H.264 and AAC to Silverlight via Windows Media Services.


The Flash client only supports streaming using their own propritary codec, so it won't be able to play streams compatible with Silverlight.
He was referring to using something like JW flash player to play H.264-in-MP4 over HTTP, which is clearly not proprietary...

And what? That's not true.
All you have to do to be compliant with Flash is use standards compliant H.264, AAC and MP4.

[hurry]

Not about Silverlight in particular (doing 96 KHz is way easier than going to 5.1 mixing, since the latter requires speaker fill, 2D positioning, and a lot of other new API stuff), but I'm not personally convinced that > 48 KHz makes any sense for content delievery. Even assuming problematic audio mastering, I think 48 Khz 20-bit is about the limit of what even a golden ear with great headphones can here, and 16-bit is essentially transparent assuming good dithering.

My experience is that directly exporting VC1 from Vegas with 96 KHz 24-bit Stereo audio produces a much more crisper, sharper and clearer sound than 44.1 or 48 Khz 8-bit or 16-bit audio even for only voice what to speak of music. I could make out the big difference even with a normal sound card and headphones. And one can directly import a m2ts file into Vegas and export it as VC1 with 96 Khz 24-bit stereo audio. That is a big advantage for VC1. So I really really hope you can add at least 96 Khz 24-bit audio support in SL if not 5.1 sound.

[benwaggoner]

My experience is that directly exporting VC1 from Vegas with 96 KHz 24-bit Stereo audio produces a much more crisper, sharper and clearer sound than 44.1 or 48 Khz 8-bit or 16-bit audio even for only voice what to speak of music. I could make out the big difference even with a normal sound card and headphones. And one can directly import a m2ts file into Vegas and export it as VC1 with 96 Khz 24-bit stereo audio. That is a big advantage for VC1. So I really really hope you can add at least 96 Khz 24-bit audio support in SL if not 5.1 sound.
96 KHz sounding better than 48 KHz for VOICE? That suggests something wrong with the audio mastering chain upstream from the codec, or radically different bitrates used. And if you're coming from m2ts source, you're almost certianly starting at 48 KHz.

No WMA codec has an 8-bit mode. 8-bit source is of course hideously pathological.

[hurry]

96 KHz sounding better than 48 KHz for VOICE? That suggests something wrong with the audio mastering chain upstream from the codec, or radically different bitrates used.

96 Khz 24-bit stereo WMA VC1 does sound better for voice too. I used the standard Vegas profile for encoding to vc1 so I don't know what may have gone wrong. But I remember I had checked it thoroughly and repeatedly heard the various file outputted.

And if you're coming from m2ts source, you're almost certianly starting at 48 KHz.

The AVCHD m2ts file from my Sony SR1 camera has dolby 5.1 sound since the camera has 5 mics or at least they say so. And I have found that directly importing the m2ts and rendering it to 96 KHz, 24-bit vc1 gives a much crisper sound. Even when I record in the lower quality mpg (non-5.1 sound) with the camera and publish it to 96 KHz 24-bit vc1, it gives more crisper and clearer sound. Aren't the Blu-Ray players and Creative Audio cards supporting 24-bit and at least 96 KHz audio (if not more) because the sound is much crisper and clearer?

This may not be relevant to this thread but I just want to add that there is big problem in trying to encode a 5.1 avchd m2ts or 5.1 mpg file from the camera to 5.1 wmv vc1 file. It always gives an error. Many have reported this on the steve-digicams and other hd forums. It gets encoded fine to stereo vc1 but the moment you select 5.1 in the WMA 10 Pro codec, it gives an error. I hope MS fixes this asap because m2ts avchd files with 5.1 are fast becoming the standard mode of recording in HD camcorders. And if we lose the 5.1 advantage after converting to vc1, then people would want to try to encode to h.264 5.1 which works fine. Thanks.

[Comatose]

Saying you hear a difference with 96 KHz with a normal sound card (onboard) is silly because almost none of them can do 96 KHz.
It just gets downsampled. It's kinda like how Avisynth internally treats all audio as 8bit.

[hurry]

Saying you hear a difference with 96 KHz with a normal sound card (onboard) is silly because almost none of them can do 96 KHz.

Oh sorry, I have the 3 years old Creative Audigy 2NX which supports 96 Khz 24-bit. And I guess all audio cards nowadays (even built-in) are HD audio which means they support 96 Khz 24-bit. And WMP and VLC player play 24-bit 96 KHz perfectly.

[hurry]

Ben, I have another request. We wanted some compatibility overlap in the H.264 files which can be played in Silverlight and also simultaneously compatible on standalone Blu-ray players for TV as we are discussing in the other thread. It will enable us to publish one HD video file for web SL playback, Blu-ray players and even for broadcast and save us a lot of storage space as Hd videos are quite large in size. This would be possible if Silverlight adds support for the m2ts container (H.264) which is fast becoming the standard for Blu-ray players and HD broadcasting. It would make SL one-up on Flash for HD video.

You wrote: We'll have a native MPEG-4 parser. We also have the MediaStreamSource feature of Silverlight whcih allows parsers to be built in managed code. So it's something an enterprising devleoper could write for Silverlight without us having to explicitly build it in.

Would someone be able to add m2ts playback support to Silverlight via this MediaStreamSource feature since m2ts is also basically H.264 if I remember correctly? Though it would actually be the best if MS can add m2ts playback support in SL natively. It will go a big way in bridging the gap between web video and standalone Blu-ray distributable video.

It has been reported that only the Sony Playstation Blu-ray hardware can presently play Silverlight compatible web vc1 videos. Is MS planning to push .wmv container (vc1) support in hardware Blu-ray players? I have heard that many if not most Blu-ray players will not support wmv container but will support only m2ts container.

[benwaggoner]

Ben, I have another request. We wanted some compatibility overlap in the H.264 files which can be played in Silverlight and also simultaneously compatible on standalone Blu-ray players for TV as we are discussing in the other thread. It will enable us to publish one HD video file for web SL playback, Blu-ray players and even for broadcast and save us a lot of storage space as Hd videos are quite large in size. This would be possible if Silverlight adds support for the m2ts container (H.264) which is fast becoming the standard for Blu-ray players and HD broadcasting. It would make SL one-up on Flash for HD video.
Well, Blu-ray encodes are much higher bitrates and frame sizes than are appropriate for web-delivered content like Silverlight uses. There may be technical compatibility, but the scenarios are different enough that it's certainly not something we're focusing on.

Would someone be able to add m2ts playback support to Silverlight via this MediaStreamSource feature since m2ts is also basically H.264 if I remember correctly? Though it would actually be the best if MS can add m2ts playback support in SL natively. It will go a big way in bridging the gap between web video and standalone Blu-ray distributable video.
M2TS is a file format, so orthogonal to codecs (it can contain MPEG-2, H.264, or VC-1). But yes, a demuxer could be written in Silverlight's MSS.

It has been reported that only the Sony Playstation Blu-ray hardware can presently play Silverlight compatible web vc1 videos. Is MS planning to push .wmv container (vc1) support in hardware Blu-ray players? I have heard that many if not most Blu-ray players will not support wmv container but will support only m2ts container.
WMV isn't part of the Blu-ray spec. There certainly have been players that support it for DLNA or other technologies they support, like the PS3, but that's something different than their BD functionality.

Files encoded for Silverlight would be extremely unlikely to be BD-compliant. When we're doing HD, for example, we're typically going to use longer GOPs and sometimes bigger buffers than BD mandates.

We're focusing on interoperabilty with existing H.264 for/on the web.

[hurry]

Thanks Ben for your detailed answers. My further research seems that it boils down to only one thing. -> Dolby Digital (AC-3) audio support in Silverlight. If you can kindly add this much natively into SL, we could use Silverlight's MSS to write a demuxer for .ts (m2ts) files, and use our .ts file both on the web via Silverlight and burn and distribute those same files via playable Blu-ray data discs too. We plan to make HD video files at 2 to 3 MBps which are very playable on the web via Silverlight and still give great quality to be viewed on standalone Blu-ray players. Both VC1 and x264 are great for the video codec at 2 or 3 Mbps even for 1080p. So we can produce .ts (m2ts) files with (vc1 + ac3) or (x264 + ac3). If you will not add DD AC-3 support to Silverlgiht, can I get it added via SL MSS too?

[benwaggoner]

Thanks Ben for your detailed answers. My further research seems that it boils down to only one thing. -> Dolby Digital (AC-3) audio support in Silverlight. If you can kindly add this much natively into SL, we could use Silverlight's MSS to write a demuxer for .ts (m2ts) files, and use our .ts file both on the web via Silverlight and burn and distribute those same files via playable Blu-ray data discs too. We plan not to make HD video files at 2 to 3 MBps which are very playable on the web via Silverlight and still give great quality to be viewed on standalone Blu-ray players. Both VC1 and x264 are great for the video codec at 2 or 3 Mbps even for 1080p. So we can produce .ts (m2ts) files with (vc1 + ac3) or (x264 + ac3). If you will not add DD AC-3 support to Silverlgiht, can I get it added via SL MSS too?
Interesting idea. The Silverlight 2 MSS doesn't have the abllity to decode as raw audio (the output of MSS needs to be a compressed bitstream to one of our natively decoded formats), but that's a very interesting feature request.

That said, if you're encoding for the web, why not reenode audio at the same time. AAC and WMA 10 Pro are both a lot more efficient.

[Inventive Software]

@hurry: Those Audigy's don't support 24-bit 96 kHz. Read up on it, don't read what Creative said. ;)

[hurry]

Interesting idea. The Silverlight 2 MSS doesn't have the abllity to decode as raw audio (the output of MSS needs to be a compressed bitstream to one of our natively decoded formats), but that's a very interesting feature request.

The problem would be fully solved if you can add .ac3 and .ts (m2ts) support natively to Silverlight. I don't think that will increase the size that much. The mandatory audio codec for Blu-ray is ac3 (DD) and additionally some Blu-ray players may support mp3, aac etc. but that support is not going to be universal. I went through the Blu-ray players available in the market and hardly 10% (only the older ones) have support for mp3.

That said, if you're encoding for the web, why not reencode audio at the same time. AAC and WMA 10 Pro are both a lot more efficient.

The whole point is to produce a single HD video file which plays both in the web browser and on the HD hardware (Blu-ray). And I think Silverlight can make that happen since you are listening to us on Doom9 unlike Adobe. You can use that as one of the unique and outstanding features of Silverlight over Flash.

The problem is that it takes a huge amount of time to render a m2ts avchd file from the camera after editing, adding effects etc. in Vegas to Vc1 for the web. Around 8 hours for a 1 hour video with chroma etc. on a Dual Core. I don't want to do this rendering for the second time for the Blu-ray hardware.

For a HD video file of 2 or 3 Mbps, a few 100 kbps does not matter much so I don't mind increasing the bit rate of the .ac3 stream to 200-400 kbps to match the aac or wma 10 pro quality because it saves me a whopping 10 hours of reencoding the video again for Blu-ray hardware players. It will also save me considerable storage space locally as well as on the web server (it will be halved). In the US, many view the videos on the computer, but my viewership audience is also in many countries where 90% people prefer to view videos on their TV so I am forced to make the second Blu-ray version available.

You are right that if we are looking at something like 6-8 Mbps or 36MBps quality for the Blu-ray discs then we will have to produce 2 files for the web & Blu-ray. But if we making low-budget HD videos (there are many producers who do this) and with the cool codecs like vc1 and x264 we get quite nice full-screen quality at 2 to 3 Mbps, we can distribute many more videos on one Blu-ray and maximize our distribution. I hope you understand. Please do consider .ac3 and .ts support for Silverlight so that we can have this option of one file for both. Thanks so much.

[Sagekilla]

@Hurry: Couldn't you render the video once, then do two different versions with different audio tracks? I agree that it would be great to have AC3 support, but would this not be a good alternative for the time being?

[benwaggoner]

The problem would be fully solved if you can add .ac3 and .ts (m2ts) support natively to Silverlight

The whole point is to produce a single HD video file which plays both in the web browser and on the HD hardware (Blu-ray). And I think Silverlight can make that happen since you are listening to us on Doom9 unlike Adobe. You can use that as one of the unique and outstanding features of Silverlight over Flash.
The challenge is that Silverlight is a software renderer, and optimized for web rate content, not short-GOP video with 40 Mbps peaks.

Silverlight 2 should be able to do 1080p24 VC-1 on a fast machine, but probably not at BD rates, and definitely not wth H.264 at BD rates.

Like most web use, Silverlight targeted "HD" content is normally 720p24 with 2-4 second GOPs around 2-6 Mbps.

[hurry]

Like most web use, Silverlight targeted "HD" content is normally 720p24 with 2-4 second GOPs around 2-6 Mbps.

Precisely. I am talking of producing videos in the range of 2 - 6 Mbps video which will play on the web in Silverlight, in the Blu-ray hardware and for broadcast also. I know of many video publishers who produce video in this range and would immensely benefit in time, resources etc. by having to produce only one file for the web and Blu-ray playback. In fact this would greatly promote Silverlight adoption among these low and middle range video producers. And one can produce pretty sharp and vivid videos at 720p with m2ts and ac3 in this range of 2-6 Mbps at lower GOPs so adding native support for these two in Silverlight will certainly be used by many. Maybe it would herald the beginning of m2ts video for the web. I think Adobe Flash may also eventually take this path. Thanks.

[hurry]

@Hurry: Couldn't you render the video once, then do two different versions with different audio tracks? I agree that it would be great to have AC3 support, but would this not be a good alternative for the time being?

That is a nice idea for the time being. But that still will not solve the problem of the trouble of uploading two big HD video files and providing double storage space on the server as well as locally for the 2 HD video files for backup.

What would be the best way (software) to mux the vc1 or x264 video track with the ac3 and aac (or wma) audio tracks into m2ts (.ts) and .mp4 (or .wmv for web) files respectively? Thanks.

[Sagekilla]

Ah I see, you're using x264 for a hardware player and VC-1 for a software player. Why not just use one, like x264 for both? It would mean you wouldn't have to render the video twice, but the audio would need to be transcoded twice (Which takes a fraction of the time) Then you can simply mux x264 + AC3 into a .m2ts and then the x264 + AAC into a .mp4, I believe this would work.

Software wise:
x264 for H.264 (obviously)
mp4box for mp4
tsmuxer (I think? I'm not sure what's generally recommended for m2ts) for your m2ts.

[hurry]

Thanks Sagekilla. I plan on using either x264 or vc1 for both Web and Blu-ray not both depending on editing process of the video. Which GUI will help me to mux x264 + AC3 into .ts and x264 + AAC into .mp4 files? I am not good in command lines.

[Sagekilla]

As I said, MP4box for MP4s and tsmuxer for m2ts. You can easily find both on google (They should be the first or second search result)

[nm]

Well, MP4Box is a command-line tool. Try Yamb (http://yamb.unite-video.com/) for a Windows GUI.

However, I'm a bit intrigued how Hurry is having a video catalogue so huge that duplicate storage is a problem and still wants to use GUI tools for muxing instead of scripts ;)

[Sagekilla]

Sorry, I keep thinking MP4Box and mean YAMB. My apologies.

[Golgot13]

GoP of 2-4 second are not supported (well) by BD specification.
At atble 9-26 and 9-27 of BD specification (BD Rom 2 Part3),
GoP up to 2second is supported but with restriction of bitrate (2-6Mbps is ok).

[puffpio]

Like most web use, Silverlight targeted "HD" content is normally 720p24 with 2-4 second GOPs around 2-6 Mbps.

I think that's great because 720p H264 in an MP4 container played back in Flash loses audio sync if you don't have the horsepower..if your implementation of the decoder can be at least as efficient as ffdshow that I would be happy.


An earlier post you asked for a use case/scenario for multichannel audio in the browser, and all I can think of is for HD movie trailers. people seem to like HD movie trailers and that would be a compelling experience and a good showcase for silverlight

[benwaggoner]

I think that's great because 720p H264 in an MP4 container played back in Flash loses audio sync if you don't have the horsepower..if your implementation of the decoder can be at least as efficient as ffdshow that I would be happy.
Ugh. There's really no excuse to lose sync. Drop frames, sure, but sync ain't that hard.

At IBC we were demoing 960x540p30 @ 6 Mbps Baseline without any trouble, FWIW.

An earlier post you asked for a use case/scenario for multichannel audio in the browser, and all I can think of is for HD movie trailers. people seem to like HD movie trailers and that would be a compelling experience and a good showcase for silverlight
Yeah, that's the one I was thinking of. But what percentage of web-connected machines actually have > stereo?

[puffpio]

Ugh. There's really no excuse to lose sync. Drop frames, sure, but sync ain't that hard.

At IBC we were demoing 960x540p30 @ 6 Mbps Baseline without any trouble, FWIW.

agreed. i remember in the early days of divx how easy it was to lose a/v sync. i would terminate all unnecessary processes out of my task manager just to make absolutely sure to minimize any random cpu usages other than the intended video.
of course those were the days when burning a cd required a vibration isolation mount for your computer desk :P sneeze and you had a coaster



Yeah, that's the one I was thinking of. But what percentage of web-connected machines actually have > stereo?

who knows, but it's a good product demo, and it's good to get entertainment industry types onboard. and could possibly be a good bullet point to get the non technical dreamers involved in things like this

[hurry]

But what percentage of web-connected machines actually have > stereo?

Almost all boards including the low-end ones have HD Audio card nowadays to which they can attach a 5.1 sound speakers or 5.1 headphones. I predict 5.1 headphones will become quite popular in days to come. I think Silverlight should support 5.1 AC3 sound for true HD quality web video.