HEIC / HEIF encoding for iOS

[Blue_MiSfit]

Hey gang!

I'd like to mess around with encoding some wide gamut HEVC images for iOS.

I'm an amateur photographer for fun, and have a nice DSLR so I have some images with big fat gamut. Lightroom works natively in ProPhoto and can export TIFF as Display P3 which is the native space of modern iDevices

I can encode these into HEVC with ffmpeg / x265 without too much trouble, but I'm not sure I'm doing the RGB -> YCbCr in such a way that I preserve the full range and color space.

I think I want to take 16 bpc full range RGB using Display P3 (so P3 primaries and matrix and sRGB transfer), and make full range YCbCr again using P3 primaries and matrix and sRGB transfer. I think 4:4:4 10 bit is ideal. Maybe this should be limited range YCbCr too? Not sure. I think JPEGs use full range YCbCr usually so I'd imagine that's fine in this case too.

The problem is, I don't see P3 as one of the options in x265 for primaries and matrix.

Aside from that I have to do the fMP4 muxing with HEIF compliance. I looked into the nokia open source project but it's pretty low level. I'll dig into that when I get there.

Any suggestions on the encoding?

[Blue_MiSfit]

Looks like mp4box can write HEIF, so that's cool

https://gpac.wp.imt.fr/2017/06/09/gpac-support-for-heif/

Still wondering how to tag the image correctly so it will render in Display P3 on the iDevice

[benwaggoner]

I'd like to mess around with encoding some wide gamut HEVC images for iOS.

I'm an amateur photographer for fun, and have a nice DSLR so I have some images with big fat gamut. Lightroom works natively in ProPhoto and can export TIFF as Display P3 which is the native space of modern iDevices

I can encode these into HEVC with ffmpeg / x265 without too much trouble, but I'm not sure I'm doing the RGB -> YCbCr in such a way that I preserve the full range and color space.

I think I want to take 16 bpc full range RGB using Display P3 (so P3 primaries and matrix and sRGB transfer), and make full range YCbCr again using P3 primaries and matrix and sRGB transfer. I think 4:4:4 10 bit is ideal. Maybe this should be limited range YCbCr too? Not sure. I think JPEGs use full range YCbCr usually so I'd imagine that's fine in this case too.

The problem is, I don't see P3 as one of the options in x265 for primaries and matrix.

Aside from that I have to do the fMP4 muxing with HEIF compliance. I looked into the nokia open source project but it's pretty low level. I'll dig into that when I get there.

Any suggestions on the encoding?
Very cool! The most compatible thing would be to make a 10-bit limited range 4:2:0 stream with the SMPTE 2100 color (PQ, 2020 primaries). That's good old HDR-10, which is supported by anything that could display HDR images. If you get that working, you can experiment with other options. I think PQ + P3 primaries is likely to work on at least some devices.

[Blue_MiSfit]

That's kind of what I was thinking too.

Outstanding questions:

- How to convert my images into Rec2020 (since they are natively ProPhoto and Lightroom can export in sRGB or Display P3)
- How to make them PQ (since all the export options from Lightroom use the sRGB gamma curve)

I also had the thought of making Dolby Vision using Resolve + Dolby Encoding Engine, but I quickly got lost scratching my head on grading the images into PQ in Resolve in a way that looked good.

[benwaggoner]

That's kind of what I was thinking too.

Outstanding questions:

- How to convert my images into Rec2020 (since they are natively ProPhoto and Lightroom can export in sRGB or Display P3)
- How to make them PQ (since all the export options from Lightroom use the sRGB gamma curve)
After Effects and Premiere Pro can take in whatever .psd file or whatever and convert it into a frame of 2100. You could even export a 1 sec long 1 fps .mp4 using Adobe Media Encoder.

I also had the thought of making Dolby Vision using Resolve + Dolby Encoding Engine, but I quickly got lost scratching my head on grading the images into PQ in Resolve in a way that looked good.
I've never heard of anyone trying to use DoVi as an image format. It might "just work" on some devices.

HDR10+ metadata should work fine for images, however.