FFMpeg next will likely support AVIF and HEIC/HEIF

[birdie]

Support for both formats has been sorely missing for years.

[Jamaika]

Super. Has anyone compiled heif fix?
https://github.com/FFmpeg/FFmpeg/commit/a0821345eb31b727d93c9c3ed7d74d2774c73afa
mov.c: In function 'avif_add_stream':
mov.c:4952:10: error: 'MOVContext' has no member named 'hvcc_offset'
4952 | if (c->hvcc_offset >= 0) {
| ^~
mov.c:4956:35: error: 'MOVContext' has no member named 'hvcc_offset'
4956 | if (avio_seek(c->fc->pb, c->hvcc_offset, SEEK_SET) != c->hvcc_offset) {
| ^~
mov.c:4956:64: error: 'MOVContext' has no member named 'hvcc_offset'
4956 | if (avio_seek(c->fc->pb, c->hvcc_offset, SEEK_SET) != c->hvcc_offset) {
| ^~
mov.c:4960:65: error: 'MOVContext' has no member named 'hvcc_size'
4960 | ret = ff_get_extradata(c->fc, st->codecpar, c->fc->pb, c->hvcc_size);
| ^~
mov.c: In function 'mov_read_meta':
mov.c:5007:6: error: 'MOVContext' has no member named 'hvcC_offset'
5007 | c->hvcC_offset = -1;
| ^~
mov.c:5008:6: error: 'MOVContext' has no member named 'hvcC_size'
5008 | c->hvcC_size = 0;
| ^~
mov.c: In function 'mov_read_iprp':
mov.c:7899:14: error: 'MOVContext' has no member named 'hvcC_offset'
7899 | c->hvcC_offset = avio_tell(pb);
| ^~
mov.c:7900:14: error: 'MOVContext' has no member named 'hvcC_size'
7900 | c->hvcC_size = sub_size;
| ^~

[FranceBB]

Awesome news, especially for HEIF!

[benwaggoner]

Awesome news, especially for HEIF!
Yeah, Apple has really demonstrated HEIC (HEIF with HEVC IDR frames) as a superior alternative to JPEG. Smaller file sizes, much less annoying artifacts, and much more efficient for synthetic images. For thinks like line art and with a well-tuned encode (using tools like transform skip and full flexibility for TU and CU sizes and shapes), I've seen HEIC outperform JPEG at <<10% the bitrate.

[FranceBB]

Yep yep, I've been trying to make it gain traction too and I have no problems displaying HEIF images, however for most people the fact that they have to install a decoder and that they're not displayed by whatever their OS is using by default is annoying enough.
In other words, despite using HEIF myself, most often than not I had to re-encode it to JPEG before sending it to other people.
As far as mobile apps are concerned, instead, they almost always downscale and re-encode to JPEG anyway, which is really a shame (I mean, don't get me wrong, JPEG served its own purpose for years, but I sort of consider it the mp3 of images nowadays).
Hopefully having a free open source decoder openly available in FFMpeg will make the adoption easier and if Android devices are gonna start decoding them automatically then I think we're gonna see an upward shift in its use. :)

[ShortKatz]

In the meantime, not only iPhones, also some mirrorless Sony cameras can save the images as HEIF images instead of JPEG.


Yep yep, I've been trying to make it gain traction too and I have no problems displaying HEIF images, however for most people the fact that they have to install a decoder and that they're not displayed by whatever their OS is using by default is annoying enough.

But this depends on the OS. If you are on macOS, you have no issues opening HEIF images. On my Windows PC at work running Windows 10 I also have no issue, but I have no idea if our system admins have installed some add-on of some sort.

[birdie]

And it works!

I've checked some files from https://filesamples.com/formats/heic and https://aomediacodec.github.io/av1-avif/testFiles/Link-U/

ffplay version N-113314-g564a15b2ee Copyright (c) 2003-2024 the FFmpeg developers
built with gcc 13 (GCC)

[ShortKatz]

HEIF parsing was improved in FFmpeg now.

[Jamaika]

Test Jamaika {compatibility}
heifenc_avx.exe image.jpg --verbose -L -q 100 -e x265 --no-alpha -p chroma=444 -p tune=ssim --matrix_coefficients=0 --full_range_flag=1 -o image1.heif
Stream #0:0[0x1]: Video: hevc (Rext) (hvc1 / 0x31637668), gbrp(pc, gbr/bt709/iec61966-2-1), 280x420, 1 fps, 1 tbr, 1 tbn
[ffplay_buffer @ 0000019f96d3c410] filter context - w: 280 h: 420 fmt: 71 csp: unknown range: unknown, incoming frame - w: 280 h: 420 fmt: 71 csp: gbr range: pc pts_time: 0
[ffplay_buffer @ 0000019f96d3c410] Changing video frame properties on the fly is not supported by all filters.

heifenc_avx.exe image.jpg --verbose -L -q 100 -e kvazaar --no-alpha --matrix_coefficients=0 --full_range_flag=1 -o image2.heif
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000022259c0d9e0] iloc: extent_count > 1 not supported
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000022259c0d9e0] error reading header
image2.heif: Not yet implemented in FFmpeg, patches welcome

heifenc_avx.exe image.jpg --verbose --jpeg2000 -L -q 100 -e openjpeg --no-alpha --matrix_coefficients=0 --full_range_flag=1 -o image3.heif
Failed to open file 'image3.heif' or configure filtergraph

heifenc_avx.exe image.jpg --verbose --htj2k -L -q 100 -e openjph --no-alpha --matrix_coefficients=0 --full_range_flag=1 -o image4.heif
Failed to open file 'image4.heif' or configure filtergraph

MP4Box.exe -add-image output_uvg266.mp4: primary -ab heic -new image.vvic
Failed to open file 'image.vvic' or configure filtergraph

[benwaggoner]

Yep yep, I've been trying to make it gain traction too and I have no problems displaying HEIF images, however for most people the fact that they have to install a decoder and that they're not displayed by whatever their OS is using by default is annoying enough.
In other words, despite using HEIF myself, most often than not I had to re-encode it to JPEG before sending it to other people.
As far as mobile apps are concerned, instead, they almost always downscale and re-encode to JPEG anyway, which is really a shame (I mean, don't get me wrong, JPEG served its own purpose for years, but I sort of consider it the mp3 of images nowadays).
Hopefully having a free open source decoder openly available in FFMpeg will make the adoption easier and if Android devices are gonna start decoding them automatically then I think we're gonna see an upward shift in its use. :)
It seems like a JavaScript HEIF parser that passes the media decode to hardware should be pretty straightforward as things go. Has no one implemented that as a fallback for browsers that don't have native support?

[SeeMoreDigital]

... (I mean, don't get me wrong, JPEG served its own purpose for years, but I sort of consider it the mp3 of images nowadays).I like the analogy ;)

How is HEIF compared to JPEG 2000?

[FranceBB]

How is HEIF compared to JPEG 2000?

Uhmmm I'm not sure, but I don't actually dislike JPEG2000 to be fair. If we think about it in terms of browser, consumer electronics etc, then it might not get quite into the main picture, however there's one simple reason why I can't dislike JPEG2000: it came in to defeat a far worse enemy -> MPEG-2.
Yes, although in images it didn't have much joy and never really got adopted, in videos it's a whole other story. Before the JPEG2000 adoption, every time we went to the cinemas to see a movie, we had an 8bit yv16 50-85 Mbit/s MPEG-2 Long GOP (up to FULL HD) encode which, projected on a very large screen like the ones we see in movie theatres, was awful. JPEG2000 came to the rescue and MJPEG2000 was standardized as an All Intra codec (basically each frame is a .j2k still that then becomes a bitstream muxed in an .mxf container). With that, we went to the new 2K (and later 4K) standards allowing 125 Mbit/s and 250 Mbit/s respectively of glorious MJPEG2000 All Intra 12bit XYZ 4:4:4 which made MPEG-2 8bit look like garbage and quickly became the de-facto standard for DCP. Nowadays, MPEG-2 is deprecated and any movie we've been seeing in a theatre has been MJPEG2000 12bit for a very long time, without any banding or other artifacts. To me, JPEG2000 is the hero of cinemas and the Wavelet Transform yields far better results than the Discrete Cosine Transform implemented in the old MPEG-2 standard. Back then, the fact of encoding a whole image at full resolution and treating it as one single macroblock rather than dividing it in blocks and macroblocks of 4x4 and 8x8 was seen as unfeasible 'cause it meant that there was no way of parallelizing the process and it took a long time to encode a stream, however modern CPUs are able to encode a single 4K image in a jiffy and by it being an All Intra codec, you can parallelize as much as you want, in fact, in my 56c/112th Intel Xeon from 2019, I generally encode 56 frames at the same time when I have to produce a DCP, making the whole process really fast. I have this in my heart 'cause not only I like to create DCP, but aside from my job, I'm also a consumer, so I do go to the cinema to watch movies, so this is something I'm able to experience the benefit of as a direct consumer. :)
As something indirect, however, there's also IMF. The story with IMF would need a whole chapter for itself and I don't wanna get into the nitty gritty details of it, but I'm a huge sustainer of IMF too and I'm proud to be part of the IMF Usergroup. The use of MJPEG2000 there too meant that we now get deliveries in high bit depth RGB Full Range with PQ or HLG BT2020 in it (or even just plain old BT709 SDR) with very high quality at reduced bitrate (i.e less than 1 Gbit/s), moving away from 1.2 TB Apple ProRes exports per movie (that are generally in the range of 1.3 Gbit/s for grainy movies) and most importantly there are lots of free open source encoders and decoders that can be used for JPEG2000 (and this is without mentioning the advantages of using a CPL instead of having to manually trim textless materials etc).

[Jamaika]

Uhmmm I'm not sure, but I don't actually dislike JPEG2000 to be fair. If we think about it in terms of browser, consumer electronics etc, then it might not get quite into the main picture, however there's one simple reason why I can't dislike JPEG2000: it came in to defeat a far worse enemy -> MPEG-2.
Yes, although in images it didn't have much joy and never really got adopted, in videos it's a whole other story. Before the JPEG2000 adoption, every time we went to the cinemas to see a movie, we had an 8bit yv16 50-85 Mbit/s MPEG-2 Long GOP (up to FULL HD) encode which, projected on a very large screen like the ones we see in movie theatres, was awful. JPEG2000 came to the rescue and MJPEG2000 was standardized as an All Intra codec (basically each frame is a .j2k still that then becomes a bitstream muxed in an .mxf container). With that, we went to the new 2K (and later 4K) standards allowing 125 Mbit/s and 250 Mbit/s respectively of glorious MJPEG2000 All Intra 12bit XYZ 4:4:4 which made MPEG-2 8bit look like garbage and quickly became the de-facto standard for DCP. Nowadays, MPEG-2 is deprecated and any movie we've been seeing in a theatre has been MJPEG2000 12bit for a very long time, without any banding or other artifacts. To me, JPEG2000 is the hero of cinemas and the Wavelet Transform yields far better results than the Discrete Cosine Transform implemented in the old MPEG-2 standard. Back then, the fact of encoding a whole image at full resolution and treating it as one single macroblock rather than dividing it in blocks and macroblocks of 4x4 and 8x8 was seen as unfeasible 'cause it meant that there was no way of parallelizing the process and it took a long time to encode a stream, however modern CPUs are able to encode a single 4K image in a jiffy and by it being an All Intra codec, you can parallelize as much as you want, in fact, in my 56c/112th Intel Xeon from 2019, I generally encode 56 frames at the same time when I have to produce a DCP, making the whole process really fast. I have this in my heart 'cause not only I like to create DCP, but aside from my job, I'm also a consumer, so I do go to the cinema to watch movies, so this is something I'm able to experience the benefit of as a direct consumer. :)
As something indirect, however, there's also IMF. The story with IMF would need a whole chapter for itself and I don't wanna get into the nitty gritty details of it, but I'm a huge sustainer of IMF too and I'm proud to be part of the IMF Usergroup. The use of MJPEG2000 there too meant that we now get deliveries in high bit depth RGB Full Range with PQ or HLG BT2020 in it (or even just plain old BT709 SDR) with very high quality at reduced bitrate (i.e less than 1 Gbit/s), moving away from 1.2 TB Apple ProRes exports per movie (that are generally in the range of 1.3 Gbit/s for grainy movies) and most importantly there are lots of free open source encoders and decoders that can be used for JPEG2000 (and this is without mentioning the advantages of using a CPL instead of having to manually trim textless materials etc).
MJPEG2000 reminds me of cheap cameras from the 90s. Interestingly there seems to be no successors of the MHTJ2K or MJPEGXS type.
JPEG2000 is history. What about the successor JPEGXS which is already paid. It has a chance to exist on the market.

Part 14 specifies an XML representation of the JPEG 2000 file format and marker segments, along with methods to for accessing the internal data of a JPEG 2000 image.
Openjpeg isn't integrated with libxml2
Apparently libheif is integrated with librsvg.

Part 15 speeds-up JPEG 2000 by an order of magnitude at the expense of slightly reduced coding efficiency. The resulting HTJ2K system retains JPEG 2000's advanced features, with reduced quality scalability, while being faster and much more efficient than traditional JPEG. This is achieved by replacing the Part 1 block coder with an innovative block coder for today's vectorized computing architectures. This also allows mathematically lossless transcoding to/from legacy JPEG 2000. Part 15 is intended to be royalty-free.

Part 16 specifies the carriage of JPEG 2000 codestreams in ISO/IEC 23008-12, commonly referred to as HEIF. A revision is underway to support more flexible wrapping of all JPEG 2000 codestreams, including HTJ2K.
https://cdn.standards.iteh.ai/samples/83650/ba003f2a71894c1f90b5617c2ba3ce33/ISO-IEC-23008-12-2022.pdf

Part 17, currently under development, introduces alternate “breakpoint-dependent” spatial wavelet transforms that dependent on an auxiliary image component, known as a “breakpoint component", and scalable coding technologies for these breakpoint components. This improves the coding of media with hard discontinuities. An important example of such media is depth imagery, where each image sample is related to the length of the 3D line segment between the corresponding scene point and the camera. Depth imagery includes stereo disparity maps, where sample values are reciprocally related to depth. Another example of media with strong discontinuities is optical flow data, where each sample location is a two-dimensional vector. In these examples, discontinuities arise naturally at the boundaries of scene objects.

Choose whether pictures are recorded in JPEG or HEIF.

Option Description
JPEG Pictures are recorded in the widely-supported JPEG format.
HEIF Pictures are recorded in HEIF, a format with excellent compression but limited options for viewing and sharing.
JPEG is automatically selected in place of HEIF during filter-effect, panorama, multiple-exposure, and HDR photography.

Selecting HEIF disables CLARITY and sets COLOR SPACE to sRGB.

HEIF pictures are stored on the memory card as files with the extension “.HIF”. Before the pictures can be viewed on a computer, the extension must be changed to “.HEIC”. This occurs automatically when HEIF pictures are uploaded from the camera to a computer via USB.

[SeeMoreDigital]

I wonder how eager the cinema industry is about moving over from JPEG2000 to something else, whether it be HEIC or something else?!

[FranceBB]

I wonder how eager the cinema industry is about moving over from JPEG2000 to something else, whether it be HEIC or something else?!

I'm not quite sure, but I think JPEG2000 is well cemented in today's standard and will be for a very long time.
To be fair, 250 Mbit/s for a 4K 12bit XYZ 4:4:4 encode which consumers will see at the theatres is probably more than we will ever need in a very long time. Last time around, when the 4K specs were updated, they even expanded the frame rates a cinema could play to allow 30p 48p 50p and 60p (on top of the old 23.976p and 24p) so I don't really see specs changing anytime soon. Changing codecs would mean changing all the systems out there which would make cinemas incur in added costs, so I really don't see that happening anytime soon for 4K. The real question will be for 8K but I wouldn't be surprised to see official JPEG2000 specs being extended again, if and when 8K comes around. I know that Japan has been a really big sustainer of 8K since late 2018 but here we are 6 years later with only a bunch of documentaries that you can count in your hands (all made in Japan and more as an example than anything) and no cinema standard nor actual contents yet.

[SeeMoreDigital]

Whenever there's a major change in specification that requires new equipment, such as providing support for 4K images, that's the golden opportunity to provide support for newer image formats too...

[kolak]

... The real question will be for 8K but I wouldn't be surprised to see official JPEG2000 specs being extended again, if and when 8K comes around. I know that Japan has been a really big sustainer of 8K since late 2018 but here we are 6 years later with only a bunch of documentaries that you can count in your hands (all made in Japan and more as an example than anything) and no cinema standard nor actual contents yet.

There is new mode for Jpeg2000, which is called HTJ2K. It's a new, simplified mode which looses around 5-10% in efficiency but has 5-10x gain in encoding/decoding speed.
Current Jpeg2000 is overdone and problematic even today.
New mode changes it and makes it more acceptable. Support is already present in some pro tools. It may come to DCP servers I assume, so then same hardware should do 8K.
CineForm is what Jpeg2000 should be from the start (but they overdone it)- fairly efficient and fast.

[benwaggoner]

I like the analogy ;)

How is HEIF compared to JPEG 2000?
HEIC (the HEVC alternate) provides better compression efficiency than J2K. It doesn't support wavelet subbands, which is a tool some people use, but otherwise I've not seen anything that a well-tuned HEIC doesn't beat J2K.

The tskip and lossless cu modes that allow efficient encoding of synthetic regions can be very powerful for content like text and line hard that frequency-based techniques generally fail on.

[benwaggoner]

I'm not quite sure, but I think JPEG2000 is well cemented in today's standard and will be for a very long time.
To be fair, 250 Mbit/s for a 4K 12bit XYZ 4:4:4 encode which consumers will see at the theatres is probably more than we will ever need in a very long time. Last time around, when the 4K specs were updated, they even expanded the frame rates a cinema could play to allow 30p 48p 50p and 60p (on top of the old 23.976p and 24p) so I don't really see specs changing anytime soon. Changing codecs would mean changing all the systems out there which would make cinemas incur in added costs, so I really don't see that happening anytime soon for 4K. The real question will be for 8K but I wouldn't be surprised to see official JPEG2000 specs being extended again, if and when 8K comes around. I know that Japan has been a really big sustainer of 8K since late 2018 but here we are 6 years later with only a bunch of documentaries that you can count in your hands (all made in Japan and more as an example than anything) and no cinema standard nor actual contents yet.
J2K for DCI supports 12-bit, which isn't widely supported in HW HEVC decoders yet (Nvidia GPUs do, but I can't think of anything else).

Which some file size savings would be possible with newer codecs, it's not like DCI package size has become a bigger problem in the last decade plus ;). Studios and theaters already took that hit.

It was a huge technical and financial effort to get existing DCI hardware into all the theaters, and there would have to be a huge reason to go though that again.

Getting rid of the whole film print manufacture, shipping, and storage saved huge amounts of money that funded the DCI conversion. There's really no big compelling technologies that would save that kind of money when switching up formats.

...as much as I like to think better compression changes the world...

[TEB]

Anyone tested HEIC vs AVIF and ran some benchmarks comparing them to WEBp, Jpg** and the lot?
Also im curious on how the performance is on SW decode mode vs HW offload which many chipset supports

[Zebulon84]

I have done some quick tests for one specific application : compress scans of various administrative documents (e.g. invoices), so quite low quality needed compared to what most people here may wish for, and that's not photos. Small color changes are not a problem, the criteria is only no big artifacts.
The best was AVIF (@40%), second Jpeg XL, then HEIF, WebP and JPG. I haven't kept the test files, but IIRC, AVIF was about 20% smaller than JXL, and quicker to encode. I did not do decode tests as I rarely need to open several documents in a row, and opening one is pretty quick (~1s on my i3-12100).
I used Skanlite (from KDE), I don't know precisely which encoder is used.

Of course, the results may be quite different for high quality photo encodings, screenshots...

[Z2697]

As a matter of fact, HEIF is just a container. The quality and speed is highly dependent on the encoder used, whether it's the HEVC variant (HEIC), AV1 variant (AVIF) or else.

Especially for these Video Codec Based variants, there are more choices and bigger differences than you typically get for image encoders (within same codec).

(WebP is also video codec (VP8) based but you don't have encoders to choose though, also the only supported pixel format is YUV(A)420P8, meh.)



Now the general situation is aomenc and svt-av1 struggles to match the quality of a tuned x265 in mid to high bitrate, for Intra only images, there's less encoding tool they can use, and the "advantage" of AV1 side is even less.

But AV1 might have more widely supported (decoding side) "SCC extensions", that could be a potential benefit but I don't think it's significant for anything not screen content.

My personal ranking is HEIC (x265) > JPEG XL (libjxl) > AVIF (aomenc or svt-av1) for lossy encoding. (WebP is excluded because it sucks, or somewhere at the bottom not even able to compete with JPEG because the lack of 4:4:4 support)

(By sucks I mean I always considered the VP series is half a generation off compared to MPEG series like VP9 is between AVC and HEVC, so that means VP8 is... similar to or less than AVC?)



Now if we need to talk about lossless encoding, well... WebP wins (kind of). (Assuming your input is RGB! If your input is YUV things might get more interesting but I didn't test that.)

Now the VCB formats or HEIF variants sucks because of the lack of specifically designed lossless mode.

JPEG XL is basically two parts (VarDCT for lossy and Modular for lossless) merged together and somtimes cooperating perhaps, WebP actually has a secret sauce called VP8L but only supports ARGB (32bpp / 8bpc) even for no-alpha images. (Despite the similarity of the name to VP8, it's actually better than just a quantization bypass or something like in HEVC or AV1 lossless, in terms of compression ratio, I didn't take a deeper look)

My personal ranking is JPEG XL (libjxl) > WebP > PNG > [VCB HEIF variants, you name it] for lossless encoding.

Be careful that some VCB HEIF encoding tools might be in fact not really lossless and give you illusions that they performed well in lossless tasks. (Like some early versions of a certain library that I'm not sure if it's libheif or not, they convert RGB pixels to YUV first.)



One step further, animations.

Video codec based formats wins again. Even in lossless mode. When compressing single image they are not as powerful, they are better at exploiting inter-frame compressions thus makes up for the shortcomming. (But your animations will need to have enough frames to make the benefits significant enough, or even get to outweight the shortcoming)
Other formats previously mentioned are basically "ranked" like in the single frame "ranks".

(DISHORNORABLE mention: WebP, while being VP8, a video codec based, seemingly not taking the advantage of better inter-frame compression. And VP8 is very old.)

[LigH]

L4cache is trying to put it in M-AB-S. So far there is a linker issue with libde265, so heif-dec contains no HEIC decoder yet ... but there is some progress.

[LigH]

Linking libde265 works now.

For the curious: libheif 1.19.5 c057050 (https://www.mediafire.com/file/1z8e65xr3w2op8p/libheif_1.19.5_c057050.7z/file) (MSYS2/MinGW, GCC 14.2; different sets of codecs per bitness)

[Jamaika]

L4cache is trying to put it in M-AB-S. So far there is a linker issue with libde265, so heif-dec contains no HEIC decoder yet ... but there is some progress.
Interesting. I didn't think anyone would be interested in libheif on this forum.

[Jamaika]

Linking libde265 works now.

For the curious: libheif 1.19.5 c057050 (https://www.mediafire.com/file/1z8e65xr3w2op8p/libheif_1.19.5_c057050.7z/file) (MSYS2/MinGW, GCC 14.2; different sets of codecs per bitness)
M-A-B-S reminds me of ffmpeg compatibility. Interesting that regular ffmpeg doesn't have it.
LigH. What plugins did you add and with what options in the latest ffmpeg? There are a lot of them on github.

[Jamaika]

Test libheif
heif-enc.exe image_2.png --verbose --vvc -q 100 -e vvenc --no-alpha --matrix_coefficients=0 --full_range_flag=1 -o image_vvenc.vvif
I don't like the action. Why is there no verbose information?

heif-info.exe imgae_uvg266.heif/imgae_kvazaar.heif
MIME type: unknown
main brand: vvic
compatible brands: mif1, vvic, miaf

image: 4000x3000 (id=1), primary
colorspace: RGB
bit depth: 8
color profile: nclx
alpha channel: no
depth channel: no
metadata:
none
transformations:
none
region annotations:
none
properties:

but in ffmpeg is...
Stream #0:0[0x1]: Video: hevc (Main 10) (hvc1 / 0x31637668), yuv420p10le(pc, gbr/bt709/iec61966-2-1), 4000x3000, 1 fps, 1 tbr, 1 tbn (default)
https://imgur.com/oNLHtw4
Has LigH added uvg266/kvazaar 10bit?

What are the options for openh264?
heif-enc.exe image_2.png --verbose -q 100 -e openh264 --no-alpha --matr
ix_coefficients=0 --full_range_flag=1 -o image_openh264.heif

[LigH]

Interesting. I didn't think anyone would be interested in libheif on this forum.

Actually it was someone in the German doom9/Gleitz board.

What plugins did you add and with what options in the latest ffmpeg? There are a lot of them on github.

I did nothing more than running L4cache's fork (https://github.com/L4cache/media-autobuild_suite/tree/test-libheif) which is supposed to get pulled into the official M-AB-S some day. Details in issue 2516 (https://github.com/m-ab-s/media-autobuild_suite/issues/2516).

So many times I already told you: I am not a very competent developer when it comes to C/C++, Linux style compilers, and Linux style shell scripts. I can only run them and make halfwit guesses about error messages. Definitely I can not add anything elaborate like a whole codec support chapter to the suite.

[benwaggoner]

Interesting. I didn't think anyone would be interested in libheif on this forum.
Why not? Video codecs have been great still image codecs since AVC was invented, and HEVC heic became a really great one as tools like transform skip and lossless coding units finally made it possible to have high quality at high compression rates for both natural and synthetic content in the same frame.

Plus pretty much any computing device has a HW decoder for them, which can make heic more performant than JPEG. Being able to copy some compressed bits to the GPU where they get decoded and composited is so much better than doing a RGB or RGBA in software and pushing the whole bitmap over. Heck, just decoding to YV12 in software and copying that cuts memory bandwidth in half.

Apple has used heic over jpeg for default for years for these sorts of advantages. Having fast HW encoders as well certainly doesn't hurt (although Apple used a SW encoder for images last I checked).

[Z2697]

Test libheif
heif-enc.exe image_2.png --verbose --vvc -q 100 -e vvenc --no-alpha --matrix_coefficients=0 --full_range_flag=1 -o image_vvenc.vvif
I don't like the action. Why is there no verbose information?

heif-info.exe imgae_uvg266.heif/imgae_kvazaar.heif
MIME type: unknown
main brand: vvic
compatible brands: mif1, vvic, miaf

image: 4000x3000 (id=1), primary
colorspace: RGB
bit depth: 8
color profile: nclx
alpha channel: no
depth channel: no
metadata:
none
transformations:
none
region annotations:
none
properties:

but in ffmpeg is...
Stream #0:0[0x1]: Video: hevc (Main 10) (hvc1 / 0x31637668), yuv420p10le(pc, gbr/bt709/iec61966-2-1), 4000x3000, 1 fps, 1 tbr, 1 tbn (default)
https://imgur.com/oNLHtw4
Has LigH added uvg266/kvazaar 10bit?

What are the options for openh264?
heif-enc.exe image_2.png --verbose -q 100 -e openh264 --no-alpha --matr
ix_coefficients=0 --full_range_flag=1 -o image_openh264.heif

Wait, did you mean that the vvenc and .vvic specified in command line result in HEVC?
What command produced the corrupted result in the link?

[Jamaika]

I've been on this forum for 7 years. Before the pandemic, this was a time of a flood of codecs. Many of them are unlikely to see use.
Why not? Video codecs have been great still image codecs since AVC was invented, and HEVC heic became a really great one as tools like transform skip and lossless coding units finally made it possible to have high quality at high compression rates for both natural and synthetic content in the same frame.

To be honest, I haven't kept up with the latest technologies for a long time. Today I'm interested in the naming and purposes of the container. hevc is heic, vvc is vvic, and h264 is ???.

Plus pretty much any computing device has a HW decoder for them, which can make heic more performant than JPEG. Being able to copy some compressed bits to the GPU where they get decoded and composited is so much better than doing a RGB or RGBA in software and pushing the whole bitmap over. Heck, just decoding to YV12 in software and copying that cuts memory bandwidth in half.
hmmm... So lossless jpeg9 in heif container, what is its efficiency?
I understand that heif in hardware has potential but adding XXX generation jpegs to heif I don't understand it.

Apple has used heic over jpeg for default for years for these sorts of advantages. Having fast HW encoders as well certainly doesn't hurt (although Apple used a SW encoder for images last I checked).

HW encoders are a commercial for higher players. For me, an amateur can watch the work. Create a commercial for an encoder, but in truth, the encoder takes shape after adding useful functions for a specific video editor. For a programmer, it's 5 minutes of work.

[Z2697]

Interesting. I didn't think anyone would be interested in libheif on this forum.

Well, not exactly libheif, but I even wrote an abomination:scared: of script to play with HEIF using FFmpeg (libx265 in particular) and mp4box.

[Jamaika]

Wait, did you mean that the vvenc and .vvic specified in command line result in HEVC?
No. Theoretically it starts to get funny because vvc in heif should be called differently. Just like for av1 it is avif.
For vvc you also have the uvg266 encoder.

[Jamaika]

What command produced the corrupted result in the link?
heif-enc.exe image_2.png --verbose -q 100 -e kvazaar --no-alpha --matrix_coefficients=0 --full_range_flag=1 -o image_kvazaar.heif

[Z2697]

heif-enc.exe image_2.png --verbose -q 100 -e kvazaar --no-alpha --matrix_coefficients=0 --full_range_flag=1 -o image_kvazaar.heif

There must be problem(s) in kvazaar or the way m-ab-s builds it, I just checked FFmpeg recently built with m-ab-s, libkvazaar also produces broken output.

Oh, never mind. FFmpeg's problem is m-ab-s build 10bit kvazaar but FFmpeg just assumes default 8bit.
The problem with libheif is probably a different one.

10bit kvazaar.exe itself is OK.

But if we build 8bit kvazaar instead, both FFmpeg and libheif will be able to produce correct result.
Although they should be considered as bugs and need to be fixed, I wonder why m-ab-s chose to build 10bit kavzaar in the first place?

[Jamaika]

Although they should be considered as bugs and need to be fixed, I wonder why m-ab-s chose to build 10bit kavzaar in the first place?
No. LigH built m-a-b-s libheif correctly with kvazaar 8-bit. The problem is that ffplay displays nonsense.
Is it possible to add kvazaar/uvg266 10bit to libheif? I doubt it.
In ffmpeg it is impossible because plugins are only for 8bit.
You can't add two 8/10bit options for kvazaar/uvg266 like for x264/x265 because they have common functions.
Kvazaar and uvg266, vvenc do not have yuv422 and yuv444 options because they are commercials.
And there is another problem gnu ffmpeg does not support decoder with AVX2. I suppose it is during rebuilding.

[Z2697]

No. LigH built m-a-b-s libheif correctly with kvazaar 8-bit. The problem is that ffplay displays nonsense.
Is it possible to add kvazaar/uvg266 10bit to libheif? I doubt it.
In ffmpeg it is impossible because plugins are only for 8bit.
You can't add two 8/10bit options for kvazaar/uvg266 like for x264/x265 because they have common functions.
Kvazaar and uvg266, vvenc do not have yuv422 and yuv444 options because they are commercials.
And there is another problem gnu ffmpeg does not support decoder with AVX2. I suppose it is during rebuilding.

https://github.com/m-ab-s/media-autobuild_suite/blob/7e35fbd5eecc22cb706c6a1e9ee39b806edd67ed/build/media-suite_compile.sh#L1262

The default behaviour of m-ab-s is to build 10bit kvazaar.

[Jamaika]

Indeed you are right. Apparently frame I was saved in the heif container as 8bit.
Jamaika tests no mabs

I know how it is in mabs but can't save 8bit photo for heif with 10bit codec. I created two 8, 10bit libheifs and for each I changed the output_bit_depth accordingly. Creator has put 10bit output under vvenc.
I don't know what the h264 decoder is supposed to be used for.

heifenc_08bit_avx.exe image_48bit.png --verbose -q 100 -e kvazaar --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_kvazaar_08bit.heic
--- construct_pipeline
from: colorspace=RGB chroma=RGB bpp(R)=8 alpha=no
to: colorspace=YCbCr chroma=420 bpp(R)=8 alpha=no matrix-coefficients=6 colour-primaries=1 transfer-characteristics=13 full-range=no

libheif can't be with functions matrix_coefficients=0, full_range_flag=1. He should have information and disabled functions

heifenc_10bit_avx.exe image_48bit.png --verbose -q 100 -e kvazaar --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_kvazaar_10bit.heic
--- construct_pipeline
from: colorspace=RGB chroma=RRGGBBB_LE bpp(R)=10 alpha=no
to: colorspace=YCbCr chroma=420 bpp(R)=10 alpha=no matrix-coefficients=6 colour-primaries=1 transfer-characteristics=13 full-range=no
It's still bad for 10bit
https://imgur.com/0xDDiaN
Libheif errors:
heifenc_10bit_avx.exe image_48bit.png --verbose -q 100 --vvc -e vvenc --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_10bit.vvic
--- construct_pipeline
from: colorspace=RGB chroma=RRGGBBB_LE bpp(R)=10 alpha=no
to: colorspace=YCbCr chroma=420 bpp(R)=10 alpha=no matrix-coefficients=6 colour-primaries=1 transfer-characteristics=13 full-range=no
final pipeline has 3 steps:
> 27Op_RRGGBBaa_swap_endianness
> 25Op_RRGGBBaa_BE_to_RGB_HDR
> 15Op_RGB_to_YCbCrItE
input spec: colorspace=RGB chroma=RRGGBBB_LE bpp(interleaved)=10
input spec: colorspace=RGB chroma=RRGGBB_BE bpp(interleaved)=10
input spec: colorspace=RGB chroma=444 bpp(R)=10
Could not encode HEIF/AVIF file: Encoder plugin generated an error: Unsupported image type: Bit depth not supported by vvenc <-- Why?

heifenc_10bit_avx.exe image_24bit.png --verbose -q 100 --vvc -e vvenc --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_10bit.vvic
--- construct_pipeline
from: colorspace=RGB chroma=RGB bpp(R)=8 alpha=no
to: colorspace=YCbCr chroma=420 bpp(R)=8 alpha=no matrix-coefficients=6 colour-primaries=1 transfer-characteristics=13 full-range=no <--Why?

image4>heifenc_08bit_avx.exe image_24bit.png --verbose -q 100 --vvc -e vvenc -
-no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_8bit.vvic
--- construct_pipeline
from: colorspace=RGB chroma=RGB bpp(R)=8 alpha=no
to: colorspace=YCbCr chroma=420 bpp(R)=8 alpha=no matrix-coefficients=6 colour-primaries=1 transfer-characteristics=13 full-range=no <--Why? VVenc doesn't have 8bit output
https://www.sendspace.com/file/5xv4ho

ffplay_avx2.exe image_kvazaar08.heic
[mov,mp4,m4a,3gp,3g2,mj2 @ 000002129285c000] error reading header
image_kvazaar08.heic: Invalid data found when processing input
https://www.sendspace.com/file/m5vk82

https://github.com/strukturag/libheif/issues/1410

Edit:
https://www.sendspace.com/file/oiyj2m

[Z2697]

@Jamaika Thanks for the testing, but there's one problem: I don't understand some of what you said, I mean the grammar. I assume you are not native English speaker? No offense but I think it might be better to use a translator, or write in you native language and let us try to understand it ourselves.

[LigH]

@Z2697: Not only that. But on top of it, Jamaika's goal seems to be not the general average build, but a very special one, statically linking into ffmpeg even DLLs which were meant to be installed into the system by default, like Avisynth. That must be one of the reasons why not the most recent GCC is used, but one with the least linking issues for that specific goal. I did not understand that for several months, until recently, because it was either never documented clearly or I missed it because it was mentioned too long ago.

[Z2697]

The latest commit: 5816f6ab kvazaar: support for 10 bit input images
Will it fix this particular 10bit kvazaar problem?

[Jamaika]

I haven't accomplished much but you can create images already libheif with kvazaar 8/10bit.

Testing
https://github.com/ultravideo/kvazaar/pull/420 + uvg266
https://github.com/strukturag/libheif/pull/1412 {full fix 8/10bit}

https://github.com/aous72/OpenJPH/commit/811f3a2eb19b4a0d797e682b496489b5e83547e3
https://github.com/uclouvain/openjpeg/commit/2be5f148ebff79e7456c77e57f85af640cf211de
https://github.com/pnggroup/libpng/commit/c1cc0f3f4c3d4abd11ca68c59446a29ff6f95003
https://gitlab.com/libtiff/libtiff/-/commit/e8233c42f2e0a0ea7260c3cc7ebbaec8e5cb5e07
https://github.com/fraunhoferhhi/vvenc/commit/7cf1e5ffc5aeb33b81fa9401df9fd53ef1dae6d1
https://github.com/ultravideo/kvazaar/tree/3882e9d125843c640f806c21b44deb99250e1aff
https://github.com/ultravideo/uvg266/commit/6999e4484143091067b0d631b624aea7554a46db
https://github.com/m-ab-s/aom/commit/36813a38590b5867158db5ab2ecaddffe30c5c8c
https://bitbucket.org/multicoreware/x265_git/commits/fa2770934b8f3d88aa866c77f27cb63f69a9ed39
https://github.com/google/brotli/commit/841f672884d2d67aae9dcf4725860aafadc5da2b

Encoder libheif 8/10bit:
https://www.sendspace.com/file/byfolq

[Jamaika]

Test corrected libheif
new uvg266 8/10bit
corrected plugin aom + new codec aom

https://www.sendspace.com/file/myd59y

heifenc_08bit_avx.exe image_21447_24bit.png --verbose --jpeg2000 -L -q 100 -e openjpeg --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_jpeg2000_08.heif
heifenc_08bit_avx.exe image_21447_24bit.png --verbose --htj2k -L -q 100 -e openjph --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_htj2k_08.heif
heifenc_10bit_avx.exe image_21447_48bit.png --verbose --jpeg2000 -L -q 100 -e openjpeg --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_jpeg2000_10.heif
heifenc_10bit_avx.exe image_21447_48bit.png --verbose --htj2k -L -q 100 -e openjph --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_htj2k_10.heif


heifenc_10bit_avx.exe image_21447_48bit.png --verbose -q 100 -e kvazaar --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_kvazaar10.heic
heifenc_08bit_avx.exe image_21447_24bit.png --verbose -q 100 -e kvazaar --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_kvazaar08.heic
heifenc_10bit_avx.exe image_21447_48bit.png --verbose -q 100 --vvc -e uvg266 --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_uvg26610.vvic
heifenc_08bit_avx.exe image_21447_24bit.png --verbose -q 100 --vvc -e uvg266 --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_uvg26608.vvic
heifenc_10bit_avx.exe image_21447_48bit.png --verbose -q 100 --vvc -e vvenc --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_vvenc10.vvic

Result: uvg266 tragedy

[LigH]

New upload:

libheif 1.19.5 5_5385aa7 (https://www.mediafire.com/file/x520r34uso4aq7b/libheif_1.19.5_5385aa7.7z/file) (MSYS2/MinGW, GCC 14.2; different sets of codecs per bitness)

[Jamaika]

https://www.sendspace.com/file/4fbs2o

heifenc_08bit_avx.exe image_21447_24bit.png --verbose --jpeg2000 -L -q 100 -e openjpeg --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_jpeg2000_08.heif
heifenc_08bit_avx.exe image_21447_24bit.png --verbose --htj2k -L -q 100 -e openjph --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_htj2k_08.heif
heifenc_10bit_avx.exe image_21447_48bit.png --verbose --jpeg2000 -L -q 100 -e openjpeg --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_jpeg2000_10.heif
heifenc_10bit_avx.exe image_21447_48bit.png --verbose --htj2k -L -q 100 -e openjph --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_htj2k_10.heif


heifenc_10bit_avx.exe image_21447_48bit.png --verbose -q 100 -e kvazaar --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_kvazaar10.heic
heifenc_08bit_avx.exe image_21447_24bit.png --verbose -q 100 -e kvazaar --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_kvazaar08.heic
heifenc_10bit_avx.exe image_21447_48bit.png --verbose -q 100 --vvc -e uvg266 --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_uvg26610.vvic
heifenc_08bit_avx.exe image_21447_24bit.png --verbose -q 100 --vvc -e uvg266 --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_uvg26608.vvic
heifenc_10bit_avx.exe image_21447_48bit.png --verbose -q 100 --vvc -e vvenc --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_vvenc10.vvic

Library encoder: libheif{avif} 1.19.5 8_12bit c++ [33ec3d6] {mingw}
kvazaar(8bit) 2.3.1 c [dd30dd2]
brotli 1.1.0 c [91d96d3]
libJPEG-turbo 3.1.1 8_16bit c [e0e18de] {assembler SIMD}
libtiff 4.7.0 8_16bit c [c474fb8] {zstd,libdeflate,libJPEG-turbo}
libdeflate 1.23.0 c [7805198]
libPNG 1.6.45 c [c1cc0f3] {zlib}
zlib 1.3.1.1 c [ef24c4c]
Compiled by Jamaika

[Jamaika]

Happy New Year

Library encoder: libheif{avif} 1.20.2 8_12bit c++ [81b09ba] {mingw}
aom 3.13.1 c [0dfe179]
svtav1 2.1.1 c [474fec3]
rav1e 0.8.1 R [1fe82de]
x264 0.165 c [0480cb0]
x265 4.1 c [9e551a9]
kvazaar 2.3.2 c [6040962]
uvg266 0.8.1 c [db32bf5]
brotli 1.2.0 c [5fa73e2]
libJPEG-turbo 3.1.3 8_16bit c [af9c1c2] {assembler SIMD}
openjpeg 2.5.4 c [8ac5262]
openjph 0.26.0 c [4d4f4e8]
libtiff 4.7.0 8_16bit c [02c6feb] {zstd,libdeflate,libJPEG-turbo}
libdeflate 1.23.0 c [7805198]
libPNG 1.8.0 c [0f07f70] {zlib}
zlib 1.3.1.2 c [570720b]
https://www.sendspace.com/file/t69531

heifenc_08bit_avx2.exe image_21447_24bit.png --verbose --uncompressed --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_uncompressed_08.heif
heifenc_08bit_avx2.exe image_21447_24bit.png --verbose --jpeg -L -q 100 -e jpeg --no-alpha --matrix_coefficients=6 --full_range_flag=0 -o image_jpeg_08.heif