Microsoft WMP vs. JPEG-2000 comparison

[DmitriyV2]

As your know Microsoft prefers its own formats - its own WMV (not MPEG-4 or AVC), WMA (not MP3 or MPEG-4 AAC), etc.

In May 2006 Microsoft presented a new format for images - WMP - as a replacement for JPEG-2000:

> The software maker detailed the new image format Wednesday at the
> Windows Hardware Engineering Conference here. Windows Media Photo will
> be supported in Windows Vista and also be made available for Windows
> XP, Bill Crow, program manager for Windows Media Photo, said in a
> presentation.
>
> In his presentation, Crow showed an image with 24:1 compression that
> visibly contained more detail in the Windows Media Photo format than
> the JPEG and JPEG 2000 formats compressed at the same level.
>
> Still, the image in the Microsoft format was somewhat distorted
> because of the high compression level. Typically digital cameras today
> use 6:1 compression, Crow said. Windows Media Photo should offer
> better pictures at double that level, he said. "We can do it in half
> the size of a JPEG file."

We tested Microsoft's codec of this new format to compare it with 9 JPEG-2000 image coders using PSNR and SSIM metrics.

Please find our results here: http://www.compression.ru/video/codec_comparison/wmp_codecs_comparison_en.html

Any comments are welcomed!

[foxyshadis]

Thank you! I had done evaluations with the same tool (getting a license for evaluating it with .net 3 beta has been painful), and although it's certainly faster than most jpeg-2000 tools, quality has been very mixed (primarily comparing to lurawave). I'm glad to see that it's not just my perceptions. If it achieves widespread support over jpeg2k due to the speed and raw formats I won't complain, anything's better than jpeg, but it's annoying that it doesn't live up to its marketing.

Thanks for releasing the test!

[feedback]

Just my subjective opinion, but in the picture of the girl with the headdress, the WMPhoto was the worst of the lot. Her right eye appeared to have no pupil, for example.

BTW, Thanks for all the pictures as one can see for themselves which codec does the best or worst job. PSNR is good empirical data but sometimes, as they say, a picture is worth a thousand words.:)

Indeed, I know I won't be using WMP to archive photos anytime soon, until and unless they make some revisions and improvements to WMP.

Regards,

[DmitriyV2]

I'm glad to see that it's not just my perceptions. If it achieves widespread support over jpeg2k due to the speed and raw formats I won't complain, anything's better than jpeg, but it's annoying that it doesn't live up to its marketing.
Looks like MS will achieves widespread support of WMP due to it's support in MS Internet Explorer + MS Office (where they did not support J2K, of course).

Also, as we can see on WMA example (MP3-players) - MS will show very good negotiations skills, so they will offer good suggestions to photo cameras manufacturers. And next generation of photo cameras and cell phones will work with WMP.

So (as usual) this will not be question of format speed (especially for images).

And marketing... MS marketing will be also as usual. :) "Vista will bring digital photo revolution to your desktops!!!".

Thanks for releasing the test!
Welcome! :)

[DmitriyV2]

BTW, Thanks for all the pictures as one can see for themselves which codec does the best or worst job. PSNR is good empirical data but sometimes, as they say, a picture is worth a thousand words.:)
Sure. Internal idea of WMP compression is another, so they have something like blocking but faster speed.

Indeed, I know I won't be using WMP to archive photos anytime soon, until and unless they make some revisions and improvements to WMP.
Maybe if new small photo camera will offer 3 options: WMP, JPEG and few images in raw - WMP will be best choice.

[check]

thanks for sharing your research! Looks like WMP will be interesting, I think as long as they give codecs to other companies (adobe, mozilla, etc) to use the image format, it could become very popular very quick ;).
btw, what will we have to call windows media player now? ;)

[DarkZell666]

Wow I've learnt about a bunch of new formats too :D

I didn't even know ACDSee was also developing compression software (I only knew about the quick image viewer from a couple of years ago xD).

I'm not too surprised that WMPhoto is behind quality-wise anyway, they've always produced suboptimal-quality software, so it suits them well =) (WMV, WMA, Windows ...) There're just so good at marketing ... it's unbelievable how they just push any new technology anywhere, no matter how cheapely designed it is ... I suppose it's one of the advantages of being a VERY BIG SOFTWARE COMPANY >_<"

Thx for the hard work Dimitry :)

[Sirber]

I just hope this don't get in my camera...

[DmitriyV2]

btw, what will we have to call windows media player now? ;)
They are use WMPhoto name for *.wmp files.

[DmitriyV2]

I didn't even know ACDSee was also developing compression software (I only knew about the quick image viewer from a couple of years ago xD).
Maybe they buy codec from some team, I did not know exactly. Climate in south Florida pretty hard for hard research. Anyway for image viewer it's very good idea to use one of the best codecs.

I'm not too surprised that WMPhoto is behind quality-wise anyway, they've always produced suboptimal-quality software, so it suits them well =) (WMV, WMA, Windows ...) There're just so good at marketing ... it's unbelievable how they just push any new technology anywhere, no matter how cheapely designed it is ... I suppose it's one of the advantages of being a VERY BIG SOFTWARE COMPANY >_<"
Thx for the hard work Dimitry :)
Absolutely. And this is problem for international standards and (sometimes) for developers - it's common situation, when VERY BIG SOFTWARE COMPANY changed "the game rules" and began to use monopoly advantages in technology pricing strategy.

[foxyshadis]

Ah, Bill Crow has provided some illuminating insights (http://blogs.msdn.com/billcrow/archive/2006/10/20/msu-evaluates-windows-media-photo-vs-jpeg-2000.aspx) and concerns about the methodology; it's nice to see him posting again.

Given the very flexible licensing compared to other windows media, I don't have the antipathy a lot of folks seem to toward the new format. I mean, free until 2010 and for anything running on windows? (I wonder how this applies to the cross-platform libavcodec, which I couldn't figure out even after rereading the full license agreement.) I doubt it'd ever be like the unisys debacle, since they're being very upfront about the terms. I'm waiting to see what support it garners beyond the operating system and how good it really is, when I'll make my choice.

[Dayvon]

Does anyone know if WMP supports transparency?

All I have to say about which format I prefer and use most. I hands down say PNG. The size is so good for the quality. It's also the best compressed file type I've found that has transparency, which is essential to picture editing in PhotoShop/PaintShop/Publisher/InDesign-type software. PNG doesn't compress well, but its standard quality and file size is very good. As long as broadband increases in speeds and as file-sizes on pictures become less of a hassle (which will happen), I hope to see more PNG pictures as opposed to JPG or WMP. It's about time for picture quality to be high.

[Shinigami-Sama]

I agree; in most of the developed world broadband is relitivly cheap, storage is cheap. if camra makers got off their asses and supported higher density chips the camras wouldn't be a problem. My HP Photosmart gives me mildy grainy 5MP at max settings, about 1-2MB per shot, and my 2GB flash card thats the size of the last two knuckles on my pinky stores about 980 of them. So I dont see the problem here. Even if you got a very clean very sharp image, I doubt it would take up more than 4MB at 5MP.
as for transfering them; its very easy now to just resize and that cleans up the image a little bit from noise reduction from the resample if you do it right, and lowers the file size signifigantly. So I dont see the point of yet another format to the slue we already have...

[foxyshadis]

I agree; in most of the developed world broadband is relitivly cheap, storage is cheap. if camra makers got off their asses and supported higher density chips the camras wouldn't be a problem. My HP Photosmart gives me mildy grainy 5MP at max settings, about 1-2MB per shot, and my 2GB flash card thats the size of the last two knuckles on my pinky stores about 980 of them. So I dont see the problem here. Even if you got a very clean very sharp image, I doubt it would take up more than 4MB at 5MP.
as for transfering them; its very easy now to just resize and that cleans up the image a little bit from noise reduction from the resample if you do it right, and lowers the file size signifigantly. So I dont see the point of yet another format to the slue we already have...

Nothing but tiff supports the vast majority of raw pixel modes, otherwise you're trapped in proprietary raw formats requiring editing within proprietary underfeatured editing packages (or if you're lucky, there's a photoshop import plugin). Tiff is almost impossible to support except in the most basic forms, as well, as most functionalities that didn't exist in the early 90's are a huge mess of often conflicting vendor-specific extensions.

The current generic interchange formats are tiff, png, and jpeg, each of which has significant strengths and weaknesses, and I see WMPhoto as an attempt to combine the strengths of each into one package. It's the opposite attitude most vendors take, the exact reason we have so many proprietary formats. It sounds good on paper - it remains to be seen what happens in practice.

Lastly, you aren't the photography enthusiast that they hope will be the early adopters. You're a causual user who doesn't have the exacting requirements a lot of photographers do, especially when it comes to printing.

And what do you mean "get off their asses"? Camera card capacity has increased by about 2x a year, faster than hard drive and ram capacity. That's complacency?

Does anyone know if WMP supports transparency?
Practically every format except raw jpeg supports transperency. (Better known as alpha channel.) It'd be insane to leave that out, and it's more than a small mystery of why jpeg did. Jpeg-in-tiff supports it via tiff transperency, a bitmap or alpha channel mask.

[zambelli]

There're just so good at marketing
This just nearly made me spill my drink all over the keyboard. :) Microsoft good at marketing?!? Dude, we can barely sell XBox 360s! :eek:

[bond]

buying customers is not as easy as buying manufacturers ;P

[Sirber]

Dude, we can barely sell XBox 360s! :eek:I'm happy with my PS2 ;)

[Doobie]

There're just so good at marketing ... it's unbelievable how they just push any new technology anywhere, no matter how cheapely designed it is ... I suppose it's one of the advantages of being a VERY BIG SOFTWARE COMPANY >_<"

There's hardly a such thing as good vs. bad marketing. MS can through a lot of resources (quantity) at marketing, which is a big help. But, their quality is no more "good" than the next guy's. But, MS's ace is a virtual desktop OS monopoly which can be leveraged to make WMP a widespread standard.

Say you're Kodak and you want better-than-old-jpg compression for your cameras. The MS sales guy tells you "100% of future MS products will support WMP, but JPEG2000 will generally not be supported." Your choice is a no-brainer. He doesn't even have to point out, "With WMP, your cameras can take 3 pictures/second, but with JPEG2000, you'll be chocked to 2 pictures/second." "Either way, you can still put thousands of pictures, at the same quality, on one 1Gig SD card." "No license cost." "No worry about surprise patent disputes." "Also does lossless compression." "Three letters, H-D-R." "Blah, blah, blah..."

I'd like to see another comparison, a larger comparison that does a better job controlling file-size and one that compares compression/decompression time.

[Shinigami-Sama]

@foxy
I ment off their asses as in supporting 4GB modules, and/or increasing the flash speed(I'd think internal RAID like striping would work). But you are right, they are doing pretty good. But for HDDs the quality and speeds of the drives have been stedily increasing with their capacities; and from what I've been reading we're on the verge of a mass storage explosion from the new perpenticual alignment that I beleave Seagate is working on for their HDDs.

[foxyshadis]

Say you're Kodak and you want better-than-old-jpg compression for your cameras. The MS sales guy tells you "100% of future MS products will support WMP, but JPEG2000 will generally not be supported."
Although Kodak still captures video in .mov (mpeg4 asp, now, used to be sorenson 3), so they don't seem to have made any format licenses from MS yet. =p

[feedback]

Ah, Bill Crow has provided some illuminating insights (http://blogs.msdn.com/billcrow/archive/2006/10/20/msu-evaluates-windows-media-photo-vs-jpeg-2000.aspx) and concerns about the methodology
Thanks for the link, he provided some more insightful information. Indeed, If (not holding my breath:))the yet to be implemented adaptive quantization does come to fruition that would be a desirable thing, IMO.

"While not implemented in our current encoder, the Windows Media Photo compressed format is designed to support adaptive quantization. Rather than use a single quantization value for the entire image, a smarter (and more processor intensive) encoder can analyze the image and adjust the quantization dynamically (at macro-block resolution) based on the image content. This will enable a new class of smart encoders in the future that should deliver dramatic improvements in overall compression quality".

[Manao]

This will enable a new class of smart encoders in the future that should deliver dramatic improvements in overall compression qualitySince when adaptive quantization is new ? It has existed since mpeg1...

[Shinigami-Sama]

maybe still picture codec makers are just that far behind the curve?

[foxyshadis]

No, it's been a feature in high-end jpeg encoders for years too, that's why I kind of laughed when I read that part. Adobe and Macromedia call it "Selective Jpeg compression", but it's entirely manually controlled, like I'm sure the first gen of WMPhoto's will be.

A few even make use of jpeg's ability to switch matrices on the fly, but compatibility on decoding that is worse than arithmetic coding.

[Mug Funky]

i can't say i agree with bill crow's statement about PSNR measurement (MSU's Y-PSNR vs RGB-PSNR). for multi-generational behaviour to be consistent, i suppose total RGB will be _mathematically_ more important, but considering that compressed formats are essentially a delivery medium, Y-PSNR is far more important. the monitor may display in RGB, but people see things in something closer to YUV (ie luma sampled more densely than chroma).

even if cameras are shooting in a compressed format, a competent retoucher will not recompress the image several times - they'll process the image, possibly save lossless intermediates (or just one big PSD with layers of course), and save down to wmp or jpg or whatever for distribution to end users. anyone who continually re-saves in a lossy format deserves uneven Y/C noise distribution, and probably much more.

[DmitriyV2]

Ah, Bill Crow has provided some illuminating insights (http://blogs.msdn.com/billcrow/archive/2006/10/20/msu-evaluates-windows-media-photo-vs-jpeg-2000.aspx) and concerns about the methodology; it's nice to see him posting again.
This is pretty good and detailed posting.

It's good probability that they will use not the worst JPEG-2000 implementation for comparison with WMP next time. (For example error distribution, dicissed by Bill Crow, fully depends on rate control implementation in J2K codec).

For future it's resonable:
1) to use another metrics (including RGB-PSNR).
2) to compare lossless comprassion ratios of WMP & J2K (BWT - different J2K codecs has different lossless ratios also)
3) to compare error distribution in codecs
4) to compare formats features (it's clear - what about RIO support in WMP for example)

Maybe we will organize 1-3, and if somebody here will organize 4 - this will be wonderful.

It's interesting to see real situation.

[DmitriyV2]

i can't say i agree with bill crow's statement about PSNR measurement (MSU's Y-PSNR vs RGB-PSNR). for multi-generational behaviour to be consistent, i suppose total RGB will be _mathematically_ more important, but considering that compressed formats are essentially a delivery medium, Y-PSNR is far more important. the monitor may display in RGB, but people see things in something closer to YUV (ie luma sampled more densely than chroma).
Commonly Y/UV ratio is fixed in codecs. There is no reasons to spend more than necessary or less then necessary data for color components.

But as a formal argument - it's ok. And this is a reason to use another metrics in updated comparison (including RGB-PSNR). It's 99% probability that results will be the same for RGB-PSNR and it's interesting to compare another metrics.

BWT - "remember, life is 1-pass only" - is pretty fine notice! :)