Which image format will see wide adoption? (Poll)

[IgorC]

Which image format will see wide adoption? And why?

[bstrobl]

HEVC/HEIF and other JPEG variants are patent encumbered. No go for many.
Normal JPEG is solid and while not perfect especially in compression efficiency, will reign supreme for a good while to come simply because everyone supports it, just like mp3.
WebP is simply not much of an improvement over JPEG.
PIK while retaining good texture has annoying ringing artifacts at edges, which tend to be more noticeable to an average person.
AV1/AVIF has the best chances of succeeding if the implementation does not get screwed up. It needs lossless mode, alpha channels, animation support etc. to replace JPG, PNG, GIF. If it can't be the format to rule them all, its adoption rate will slow by quite a bit, since many will likely ignore it if it's just another support burden(see webp, compression improvements alone does not make a good replacement). It helps if lots of companies stand behind it and implementation and design issues are properly ironed out, however the release should be a solid 1.0 implementation with everything included, not like webp (wtf is version 0.6.1 of a codec with a likely 20 year lifetime supposed to be???).

[burfadel]

JPEG's in reference to pictures is easier to say than AV1's or HEVC's. It needs to be great quality, efficient, fast, practical, and have a good name. Marketing also helps, proprietory formats have the support of the patent owners to spruik the virtues of the format.

[hajj_3]

HEVC/HEIF and other JPEG variants are patent encumbered. No go for many.

JpegXL will be patent free for the baseline profile, just like regular old jpeg.

[Jamaika]

Which image format will see a wide adoption?
https://caniuse.com/#feat=jpeg2000
https://caniuse.com/#feat=webp
https://caniuse.com/#feat=heif
https://caniuse.com/#feat=jpegxr
None. Each browser has its own format and does not intend to use another. These are patents and profits from this.

[IgorC]

My vote goes for AV1 and/or JPEG XL.
Good thing that all open source projects share a lot of compression techniques between them. AV1 has Daala code, JPEG XL will have at least some of them from both and so on.

JpegXR will be patent free for the baseline profile, just like regular old jpeg.
You probably mean JPEG XL (https://jpeg.org/items/20180221_cfp_jpeg_xl.html) wichi will have royalty-free patent licensing

JPEG XR is a Microsoft thing, patented and supported only by them (Internet Explorer and Edge browser).

[dipje]

Just guessing here but isn't jpeg xr royalty free / free to use ? Didn't read the commercial licensing terms in the GitHub repo.
They called it HDPhoto before and it became jpeg xr when it got standardized isn't it ?

The format got my vote but since it existed for years and didn't see any adoption (well, I know of affinity photo, photomatix HDR and Capture 1) I don't think it will be 'the winner '.

[IgorC]

JPEG XR wasn't much better than JPEG at medium/high bitrates. Later WebP and mozjpeg were released. They had higher quality at those rates.

https://xooyoozoo.github.io/yolo-octo-bugfixes/#pont-de-quebec-at-night&jxr=l&jpg=l

[Sparktank]

As much as I love hearing about new formats and the possibilities, I've given up hopes that industries will strive to reach the full potential of even existing formats.

GIF is a complete mess right now.
GIF is still the most compatible (albeit, slowest) format, even though using video formats for HMTL5 is a lot better.

The video section is even worse.
That's not even including a lot of the forks for the open-source browsers that create their own projects.

I think JPEG will be around forever.
But, it would be nice if some sites even give the option for various formats (given that the new format is from their hi-res source file and just not a conversion from JPG to whatever).

[Phanton_13]

But what you mean by wide adoption? because some times can be a lot of hiden adoption, this is clear with vorbis as is supported and used more than people think, this hiden use is mainly as a asset format and not as interchange format.

[iwod]

My vote goes for AV1 and/or JPEG XL.
Good thing that all open source projects share a lot of compression techniques between them. AV1 has Daala code, JPEG XL will have at least some of them from both and so on.


You probably mean JPEG XL (https://jpeg.org/items/20180221_cfp_jpeg_xl.html) wichi will have royalty-free patent licensing

JPEG XR is a Microsoft thing, patented and supported only by them (Internet Explorer and Edge browser).

I just look at the PR and it seems JPEG XL work hasn't even started.

[dapperdan]

I just look at the PR and it seems JPEG XL work hasn't even started.

Also, they don't have a credible plan to actually deliver a royalty-free baseline. H.264 was supposed to have a royalty-free baseline too. Then, much later, after all the work was done they asked the patent holders and they said, "No".

I believe Apple tried to resurrect that royalty-free baseline for H.264 when web video and VP8 started becoming a thing, but again they failed to convince all the relevant patent holders.

If they even boldly stated that royalty-free baseline was non-negotiable for XL then I'd maybe assume they'd come up with some process to get there, but it was (in the doc I read) stated like a wishy-washy dream rather than a concrete goal.

[iwod]

If they even boldly stated that royalty-free baseline was non-negotiable for XL then I'd maybe assume they'd come up with some process to get there, but it was (in the doc I read) stated like a wishy-washy dream rather than a concrete goal.

I wish they do that. If it is not royalty free, then there is no point starting it. Unless it gives another 30% reduction then AV1.

[raffriff42]

Once AI deep learning image compression is released for public use, all other compression schemes will fade away fairly quickly (both lossy and lossless) :devil:

(all compression === predicting the next bit of data so you don't have to store it. In the near future AI will do this prediction much better than Huffmann, DCT, wavelets etc)

https://en.wikipedia.org/wiki/Hutter_Prize
"The organizers believe that text compression and AI are equivalent problems."

Deep Learning Image Compression
https://www.google.com/search?q=deep+learning+image+compression

(Actually, AI will be too slow for many purposes. There's still room for traditional compression schemes. For a while.)

[IgorC]

I agree. Artificial intelligence looks very promising.

Opus is one of the first formats to implement AI https://people.xiph.org/~jm/demo/rnnoise/

Upcoming 1.3 beta 2 will be much better for speech/music recognition and speech coding.
Results are astonishing.

[iwod]

I agree. Artificial intelligence looks very promising.

Opus is one of the first formats to implement AI https://people.xiph.org/~jm/demo/rnnoise/

Upcoming 1.3 beta 2 will be much better for speech/music recognition and speech coding.
Results are astonishing.

Sorry for going off topic, but these seems to be Noise Removal only. What does it do to normal Music encoding like CD Ripping?

[raffriff42]

Noise reduction takes a damaged or partial source and tries to reconstruct the original.
AI has the potential to become very, very good at this.
Having a way to perfectly denoise means you can apply stronger lossy compression and still recover a good signal.
(thanks IgorC for making me see this)

[IgorC]

Sorry for going off topic, but these seems to be Noise Removal only. What does it do to normal Music encoding like CD Ripping?


Opus is a hybrid codec. It based on enhanced SILK speech codec and CELT music/general purpose codec. It can use SILK or CELT or Hybrid(SILK+CELT for different frequency ranges) for each audio frame

We (humans) are smart enough to detect whether it's speech or music.
How can You transmit that to program like codec?
That's where RNN comes with its perfect speech/music classification.

Current version of Opus (1.2) hasn't perfect speech/music detection.
But that will change.

[foxyshadis]

JPEG XL is as dead as dead gets. JPEG XR is on life support, but still being pushed by Microsoft and a few other vendors, with a legally-binding patent-protection clause forever, which is as good as free in my book. I think XR still has a chance. JPEG XT, which is a layer on top of JPEG, will probably disappear into obscurity as everyone realizes it's encumbered and inefficient.

If I had money on a single format, it'd be HEIC, or its more general cousin HEIF. Not because of inherent technical advantages, although it is a particularly well-designed format, but mostly because Apple has adopted it and that heavily tilts in any format's favor. I hope AV1 gets here with a standardized format quickly, but I see the world standardizing on HEIC soon (or HEIF for those who don't want to pay the HEVC royalties) as a JPEG alternative on the path of least resistance.

[Jamaika]

If I had money on a single format, it'd be HEIC, or its more general cousin HEIF. Not because of inherent technical advantages, although it is a particularly well-designed format, but mostly because Apple has adopted it and that heavily tilts in any format's favor. I hope AV1 gets here with a standardized format quickly, but I see the world standardizing on HEIC soon (or HEIF for those who don't want to pay the HEVC royalties) as a JPEG alternative on the path of least resistance.
Oh, and where is the HEIF software running under Windows correctly. What is on github simply doesn't work.
MediaInfo admittedly added images BPG to the software, but HEIF isn't interested. FFmpeg also.

[foxyshadis]

Oh, and where is the HEIF software running under Windows correctly. What is on github simply doesn't work.
MediaInfo admittedly added images BPG to the software, but HEIF isn't interested. FFmpeg also.

I agree, I've been working on better integrating it into ImageMagick myself, and MediaInfo & ffmpeg are on my list to work on next. It's infuriating how most codecs make no effort to get themselves implemented more generally.

It's also infuriating that Microsoft did the hard work to bring support for wdp/hdp/jxr across the whole closed & open source ecosystem, even into Firefox and Chrome, and no one was the slightest bit interested in using it. JPEG XR is actually a FLOSS format with a ton of capabilities, and people are more interested in HEIC just because Apple uses it, so we're stuck with patents.

[Neillithan]

I'm betting my money on AV1 just because Google shoehorned WebP into Chrome, and we can expect them to do the same with AV1. Once that happens, >60% browser market share will have access to it, while you can provide a fallback method for non compliant browsers. Meanwhile, AV1 has support from an endless list of companies, hardware support is coming, everyone seems to be really excited about this free, open source, royalty free format that's getting collaboration from half a dozen projects. AV1 image format is basically going to make a huge impact whenever it arrives.

That doesn't mean I think AV1 is the best image format. It certainly isn't, and has plenty of room for improvements. It has a problem with saturation, and loss of grainy / noisy micro details. However, if you want a tiny image file with an alpha channel, AV1 is going to be absolutely fantastic for that. Jpeg will still always be king of noisy/grainy images... but chances are, you won't care about that if you care about filesize.

The PIK format is interesting, as it appears to be true successor to JPEG and great at preserving micro details and noise/grain.... however, it has some artifacting issues around edges that I think need to be resolved... and even then, it's pretty much a doomed format since adoption rate will be a big whopping 0%, guaranteed. I'm not losing any sleep over this.

As soon as AV1 image format arrives, apples HEIF/HEIC image format is essentially doomed. Apple is a retarded ass company that somehow manages to make money, but I'm fairly confident their decision to push HEIC/HEIF is about the stupidest decision they've made yet.

[foxyshadis]

I'm betting my money on AV1 just because Google shoehorned WebP into Chrome, and we can expect them to do the same with AV1. Once that happens, >60% browser market share will have access to it, while you can provide a fallback method for non compliant browsers. Meanwhile, AV1 has support from an endless list of companies, hardware support is coming, everyone seems to be really excited about this free, open source, royalty free format that's getting collaboration from half a dozen projects. AV1 image format is basically going to make a huge impact whenever it arrives.

That doesn't mean I think AV1 is the best image format. It certainly isn't, and has plenty of room for improvements. It has a problem with saturation, and loss of grainy / noisy micro details. However, if you want a tiny image file with an alpha channel, AV1 is going to be absolutely fantastic for that. Jpeg will still always be king of noisy/grainy images... but chances are, you won't care about that if you care about filesize.

The PIK format is interesting, as it appears to be true successor to JPEG and great at preserving micro details and noise/grain.... however, it has some artifacting issues around edges that I think need to be resolved... and even then, it's pretty much a doomed format since adoption rate will be a big whopping 0%, guaranteed. I'm not losing any sleep over this.

As soon as AV1 image format arrives, apples HEIF/HEIC image format is essentially doomed. Apple is a retarded ass company that somehow manages to make money, but I'm fairly confident their decision to push HEIC/HEIF is about the stupidest decision they've made yet.

Apple has always been a company that would rather design a better format than use someone else's, especially someone else's unfinished format. They had a large hand in creating HEIF, and some input in AVC & HEVC, of course they're going to use and push it. Given that it will likely be well over a year before any image format based on the still-unfinalized video format, possibly several years if they dally and bicker, those who want to change aren't going to hold onto unrealized hope forever. It's as likely that AV1 suffers the same fate as Vorbis vs MP3 & MP4 or VP8 vs AVC; too little too late. VP9 is afloat because of Google's clout, but hardly dominating the field. I hope AV1 will do better, but my prediction is that it'll always be the alternative, not the normal.

BPG and HEIF show that there's a lot more to it than just having an I-frame with extra color types; if they're smart they'd just re-use HEIF with the underlying codec swapped out, because it pretty much has every bit of flexibility and power that anyone would want, except that vector capabilities are pretty simple -- no SVG competitor there. It reminds me a lot of Photoshop's PSD, except open and based on MP4.

[iwod]

Does anyone know why there isn't an image format based on H.264?

[wiak]

apng & png

[utack]

It is really sad how it is 8 years later and this old blogpost analyzing a WebP pre-release version still holds:
https://web.archive.org/web/20120319180449/http://x264dev.multimedia.cx/archives/541
I have done some recent tests and at the same filesize WebP was doing clearly worse than mozjpeg in a bit more than half of my test images.
libvpx and libaom do the same, they just blur anything to get good PSNR metrics, unusable for human perception

[iwod]

It is really sad how it is 8 years later and this old blogpost analyzing a WebP pre-release version still holds:
https://web.archive.org/web/20120319180449/http://x264dev.multimedia.cx/archives/541
I have done some recent tests and at the same filesize WebP was doing clearly worse than mozjpeg in a bit more than half of my test images.
libvpx and libaom do the same, they just blur anything to get good PSNR metrics, unusable for human perception

Because I don't think there are much interest from companies. It seems as Bandwidth increases every year we have settled JPEG as baseline.

I am hoping we may finally see one from VVC.

[benwaggoner]

Oh, and where is the HEIF software running under Windows correctly. What is on github simply doesn't work.
MediaInfo admittedly added images BPG to the software, but HEIF isn't interested. FFmpeg also.
Microsoft now has a beta HEVC/HEIF Media Foundation Transform on the Windows Store for $0.99.

[benwaggoner]

It is really sad how it is 8 years later and this old blogpost analyzing a WebP pre-release version still holds:
https://web.archive.org/web/20120319180449/http://x264dev.multimedia.cx/archives/541
I have done some recent tests and at the same filesize WebP was doing clearly worse than mozjpeg in a bit more than half of my test images.
libvpx and libaom do the same, they just blur anything to get good PSNR metrics, unusable for human perception
There never was a WebP psychovisually good enough to make it work dealing with the support. It takes the promise of a 2x improvement to really get people to start working towards adoption of new media technologies.

One thing that is important to note is that it is increasingly trivial for a big web site to have an image available in multiple formats, and deliver the appropriate one for a given browser/platform/window size. If you think about big social media and shopping sites, there are SO many pictures, the cost savings and particular page load time improvements of a 2x more efficient codec can be easily justified financially.

And 2x is just for natural images; much better results are seen with synthetic and hand-drawn images. I've been able to beat JPEG 20x with HEIF for comic book art, for example. JPEG just doesn't do sharp edges well. A nice thing about HEIF is that a single frame can mix natural and synthetic elements, being better than JPEG in some areas and better than PNG in others. Features like intra-frame prediction, transform skip, highly variable block and motion partition sizes, and lossless CUs are big advantages for text, screen captures, line art, etcetera.

[iwod]

There never was a WebP psychovisually good enough to make it work dealing with the support. It takes the promise of a 2x improvement to really get people to start working towards adoption of new media technologies.

One thing that is important to note is that it is increasingly trivial for a big web site to have an image available in multiple formats, and deliver the appropriate one for a given browser/platform/window size. If you think about big social media and shopping sites, there are SO many pictures, the cost savings and particular page load time improvements of a 2x more efficient codec can be easily justified financially.

And 2x is just for natural images; much better results are seen with synthetic and hand-drawn images. I've been able to beat JPEG 20x with HEIF for comic book art, for example. JPEG just doesn't do sharp edges well. A nice thing about HEIF is that a single frame can mix natural and synthetic elements, being better than JPEG in some areas and better than PNG in others. Features like intra-frame prediction, transform skip, highly variable block and motion partition sizes, and lossless CUs are big advantages for text, screen captures, line art, etcetera.

I have always used BPG representing HEIF [1], are there any reason why HEIF would be better? I think both are based on x265?


[1] https://wyohknott.github.io/image-formats-comparison/#abandonned-factory&webm=s&webm=s

[Jamaika]

I have always used BPG representing HEIF [1], are there any reason why HEIF would be better? I think both are based on x265?
The creator of bpg have a plugin on the jctvc codec. I don't know if it will be possible to connect a newer jvetvcc.
It isn't known what codec is google pik.
The creators of HEIF bind high hopes with the family of av1 codecs.

[benwaggoner]

Does anyone know why there isn't an image format based on H.264?
No good reason. I did quite a bit of testing with H.264 High Profile IDR frames, and they are a lot better than JPEG and almost always better than PNG. Not as good as HEVC (tskip and sub-frame lossless are huge for some image types). But still decently better for natural images and a lot better for synthetic or mixed natural/synthetic content.

HEIF as a file format supports H.264 as well as HEVC, but I don't know if anyone has ever used it for anything. I could see an HEIF JavaScript parser that hands off H.264 decode to the system being not that hard to implement on all web browsers. Could be a great intermediate/alternative solution when a newer codec isn't available.

[foxyshadis]

Does anyone know why there isn't an image format based on H.264?

It just wasn't good enough. Much like VP8-based WebP, AVC was designed with 8-bit in mind, and despite the range extensions, it never really broke out of that niche; by the time anyone started seriously thinking about utilizing video tech for a new uber-format (as opposed to WebP and JPEG-XR, which I'd call solutions in search of a problem, despite my high hopes for hdp/jxr), HEVC and VP9 were on the scene, AV1 was around the corner, and VVC was in the works. HEIF would be another also-ran if it wasn't for Apple deciding that they finally need something between JPEG and RAW (and since they already have the hardware decoder, might as well make use of it).

I have always used BPG representing HEIF [1], are there any reason why HEIF would be better? I think both are based on x265?


[1] https://wyohknott.github.io/image-formats-comparison/#abandonned-factory&webm=s&webm=s

BPG paved the way, but it's a fairly narrow focus. It's a technically excellent image format. HEIF, in its full standard, sits somewhere between an image and document format; its killer feature is that images can be layered, which makes it possible to mix types of compression where each is most appropriate; imagine layering JPEG areas on a PNG for screenshots, or being able to go back and adjust the exposure an hour later without generational losses. Obviously, Apple hasn't cared about that; they just wanted a better image format and got the stamp of MPEG and ITU, much like they did with AAC, but there's a solid chance it'll give us an image format that'll last as long as good old JPEG did. (Or it could flop like all the other JPEG-killers. Who knows.)

[iwod]

It just wasn't good enough. Much like VP8-based WebP, AVC was designed with 8-bit in mind, and despite the range extensions, it never really broke out of that niche; by the time anyone started seriously thinking about utilizing video tech for a new uber-format (as opposed to WebP and JPEG-XR, which I'd call solutions in search of a problem, despite my high hopes for hdp/jxr), HEVC and VP9 were on the scene, AV1 was around the corner, and VVC was in the works. HEIF would be another also-ran if it wasn't for Apple deciding that they finally need something between JPEG and RAW (and since they already have the hardware decoder, might as well make use of it).



Thanks, I nearly forgot about the whole Hi10P problem.

[skal]

WebP is turning 10 tomorrow. Eventually, it got supported by the majority of platforms/browsers, after the recent iOS14 release.

skal/

[benwaggoner]

The poll is conflating wrapper formats and codces. HEIF can use H.264, HEVC, JPEG, etcetera for its image essence. HEIF + AV1 would be a simple and obvious extension. EVC and VVC are also potential candidates.

Patent issues for still image coding are going to be reduced because all the interframe encoding IP doesn't apply. I don't know that anyone's done a full legal evaluation, though, and IANAL.

And JPEG XR was pretty much abandoned a decade ago. I knew people working on it when I was at Microsoft then, and they never got it psychovisually tuned to the point it was competitive.

[benwaggoner]

Oh, and where is the HEIF software running under Windows correctly. What is on github simply doesn't work.
MediaInfo admittedly added images BPG to the software, but HEIF isn't interested. FFmpeg also.
On the Microsoft Store! Free, although it requires an appropriate decoder be installed.

https://www.microsoft.com/store/productId/9PMMSR1CGPWG

[Jamaika]

Thx for info. This is old thread.
Formats read by Mediainfo
JPEG 8bit {read}
JPEGXT 8/12bit {on github 12bit in 8 bit container}/ RGB/YUV /lossless
JPEGLS 8/16bit {charls don't read}
JPEGXL 8/16bit {google don't read}
libHEIF 8/10/12bit {x265 read}
HTJPEG2000 8bit {kakadoo read}
PGX {don't read}
EXR 64bit {read}