ffmpeg 709 Colors

[jay123210599]

This is the information for a video I have. If I make a png image sequence out of it using ffmpeg and/or Shutter Encoder, will the images have the 709 colors from the video?

Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High@L4.1
Format settings                          : CABAC / 4 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 4 frames
Format settings, Slice count             : 4 slices per frame
Codec ID                                 : V_MPEG4/ISO/AVC
Duration                                 : 23 min 43 s
Bit rate mode                            : Variable
Bit rate                                 : 38.7 Mb/s
Maximum bit rate                         : 40.0 Mb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.779
Time code of first frame                 : 00:59:59:00
Stream size                              : 6.42 GiB (96%)
Default                                  : No
Forced                                   : No

[FranceBB]

Yes it will, however please note that it will convert the range from Limited TV Range to Full PC Range during the yv12 to rgb24 conversion.

Here's a simple yv12 limited tv range output generated to simulate your source:

ColorBars(848, 480, pixel_type="YV12")

And here's the mediainfo of the corresponding PNG file:

General
Complete name                            : New File 000000.png
Format                                   : PNG
Format/Info                              : Portable Network Graphic
File size                                : 3.54 KiB

Image
Format                                   : PNG
Format/Info                              : Portable Network Graphic
Compression                              : Deflate
Format settings                          : Linear
Width                                    : 848 pixels
Height                                   : 480 pixels
Display aspect ratio                     : 16:9
Color space                              : RGB
Bit depth                                : 8 bits
Compression mode                         : Lossless
Stream size                              : 3.54 KiB (100%)

Notice how it has been converted from YV12 to RGB24, so from 4:2:0 8bit to RGB24 8bit.
BT709 is indeed still there and preserved, but given that Studio RGB (i.e RGB Limited TV Range) isn't really a thing outside of broadcast environments, pretty much any software that converts to RGB will make the picture Full PC Range. In the 8bit case we're going from 16-235 to 0-255. As a side note, the reason why Studio RGB is only used in broadcast and isn't really widespread is that SDI cables need some codes to carry stuff like the timecode and other sync codes, so you wouldn't be able to carry the whole 255 values.


Anyway, this can be easily simulated like this:

ColorBars(848, 480, pixel_type="YV12")
Converttoyv24(matrix="PC.709")
TurnRight.Histogram.TurnLeft #luma
Crop(0, 0, -0, -480)
Subtitle("Original Video Limited TV Range")
video=last

ImageSource("D:\New File (22)000000.png")
Converttoyv24(matrix="PC.709")
TurnRight.Histogram.TurnLeft #luma
Crop(0, 0, -0, -480)
Subtitle("Screenshot Full PC Range")
screenshot=last

StackVertical(video, screenshot)

As you can see, the first one, the original video, never exceeds the 235 top and only ever goes down the 16 bottom in one specific section on the right hand side dedicated to the super black. In other words, it's 8bit limited tv range. On the other hand, the second one, the screenshot, happily hits the 255 ceiling (i.e white) and 0 bottom (i.e black) as it's 8bit full pc range. Again, this kind of conversion is pretty safe and there aren't any issues unless you actually have out of range values (i.e things going above 235 or 0.7V and below 16 or 0.0V) 'cause when your 235 gets mapped to 255 anything above 235 is automatically lost as it can't be mapped above 255. In any case, in a properly graded and encoded limited tv range file there shouldn't be anything going out of range in the first place 'cause every TV is RGB Full PC Range anyway and it would perform the exact same conversion anyway, so any out of range highlight would be clipped out. This is exactly why I always tell all my coworkers to be extremely careful during live events and to always perform soft highlights rollback to dim them and make them fall within the 0.0-0.7V range.

[poisondeathray]

Originally Posted by jay123210599

This is the information for a video I have. If I make a png image sequence out of it using ffmpeg and/or Shutter Encoder, will the images have the 709 colors from the video?

1) It depends on your commandline
2) For a source without colorimetry flags, unless otherwise specified, ffmpeg will use bt601 by default, not 709, for the YUV=>RGB conversion

[jay123210599]

Originally Posted by poisondeathray

1) It depends on your commandline
2) For a source without colorimetry flags, unless otherwise specified, ffmpeg will use bt601 by default, not 709, for the YUV=>RGB conversion

This is the my command line:

ffmpeg -i input.mkv output%d.png

So does both ffmpeg and Shutter need colorimetry flag in order to get the 709 colors? What about for making gifs, webps, and apngs?

[GeoffreyA]

Originally Posted by jay123210599

This is the my command line:

ffmpeg -i input.mkv output%d.png

You can try the following and see if it helps:

ffmpeg -colorspace bt709 -color_trc bt709 -color_primaries bt709 -color_range tv -i input.mkv output%d.png

Usually, I set that on all input, file or pipe, because most content is BT.709. If you're using a batch file, use two percentage signs on Windows.

[Z2697]

Originally Posted by GeoffreyA

You can try the following and see if it helps:

ffmpeg -colorspace bt709 -color_trc bt709 -color_primaries bt709 -color_range tv -i input.mkv output%d.png

Usually, I set that on all input, file or pipe, because most content is BT.709. If you're using a batch file, use two percentage signs on Windows.

I'd rather use zscale
Faster and better.
The only benefit of swscale IMO is it can handle more obscure pixel formats.

[Z2697]

You should extract all planes individually and encode them to grayscale PNG. Absolute perfect color retention.

[poisondeathray]

Beware the problem with ffmpeg using -colorspace etc..., or with colorimetry tagged files and using png output - is the output png still has gAMA and cHRM tags. This causes the colors to be displayed inconsistently in different programs . A browser may display certain colors, a picture viewer something else. Different source filters can display different colors. A long standing ffmpeg issue with several open tickets and discussion

For untagged files, I would use zscale too. If using swscale, beware it uses a "fast" less accurate conversion unless you use -sws_flags full_chroma_int+accurate_rnd

[jay123210599]

Originally Posted by poisondeathray

Beware the problem with ffmpeg using -colorspace etc..., or with colorimetry tagged files and using png output - is the output png still has gAMA and cHRM tags. This causes the colors to be displayed inconsistently in different programs . A browser may display certain colors, a picture viewer something else. Different source filters can display different colors. A long standing ffmpeg issue with several open tickets and discussion

For untagged files, I would use zscale too. If using swscale, beware it uses a "fast" less accurate conversion unless you use -sws_flags full_chroma_int+accurate_rnd

So yes, that and Shutter Encoder need colorimetry flags from videos in order to get the 709 colors? The same goes for making gifs, webps, and apngs?

[poisondeathray]

Originally Posted by jay123210599

Shutter Encoder need colorimetry flags from videos in order to get the 709 colors? The same goes for making gifs, webps, and apngs?

Not sure, I don't use shutter encoder . It depends on what commands it's passing to ffmpeg, and/or other operations it's performing. Maybe it has some logic that makes assumptions about a given resolution and colorimetry. You can test it.

[GeoffreyA]

Z2697, poisondeathray, I do use zscale for actual conversion or scaling; no contest, it leaves swscale in the dust. However, I found that using those switches before the input merely sets the values for the internal structures, as far as I can tell from the debug log, and doesn't do any conversion (the auto-inserted swscale filter).

[jay123210599]

So what should I do for ffmpeg and Shutter Encoder to get the 709 colors when making images out of this type of video? Leave it as it is or add the flags to the video beforehand or during the conversion?

[poisondeathray]

Originally Posted by jay123210599

So what should I do for ffmpeg and Shutter Encoder to get the 709 colors when making images out of this type of video? Leave it as it is or add the flags to the video beforehand or during the conversion?

1) If you leave it as-is, the default YUV to RGB conversion uses Rec601

2) For non colorimetry flagged sources - you control the conversion by specifying the input matrix as 709 using zscale or swscale. Use the sws scale switches mentioned above if you care about less error when using swscale.

eg

-vf zscale=min=709,format=gbrp

3) If you add the colorimetry flags to the source, or for sources with colorimetry flags already present , the ffmpeg PNG's will display incorrect colors in many programs, including browsers (!) - because of gAMA and cHRM tag issue. Other formats like GIF are not affected

[jay123210599]

Originally Posted by poisondeathray

3) If you add the colorimetry flags to the source, or for sources with colorimetry flags already present , the ffmpeg PNG's will display incorrect colors in many programs, including browsers (!) - because of gAMA and cHRM tag issue. Other formats like GIF are not affected

But why would there be gAMA and cHRM tag issues? So other formats like AVIF, JXL, WEBP, JPG, GIF, and APNG will automatically get the 709 colors, even if the colorimetry flags are not present in the source?

[poisondeathray]

Originally Posted by jay123210599

But why would there be gAMA and cHRM tag issues?

Ask a ffmpeg developer . There are tickets for this already.

The gAMA, cHRM tag issue is specific to PNG . Even if the underlying YUV to RGB conversion in ffmpeg is correct, the PNG can have problems being displayed in programs

The most consistent PNG display is no gAMA, cHRM tags - it essentially displays the same everywhere

It's easy to verify this, if you strip the gAMA, cHRM tags, the PNG now displays correctly in browsers and other programs.

So other formats like AVIF, JXL, WEBP, JPG, GIF, and APNG will automatically get the 709 colors, even if the colorimetry flags are not present in the source?

No .

see #2 in the post above "For non colorimetry flagged sources..."

Also point #2 in my first reply "For a source without colorimetry flags, unless otherwise specified,..."


Nothing is guaranteed to be automatically correct in ffmpeg-land. You should always verify your workflow, because there are a bunch of "gotchas"


Jpeg is different because there are YUV variants, subsampled variants, RGB variants. Also, it is supposed to use full range 601 (pc 601) - as per the jpeg spec , not Rec709, not limited range. Common conversions should be handled correctly automatically in ffmpeg already (because jpeg is/was so common, all the bugs have been fixed), but you can test and verify . Obscure input formats might need some other switches for the correct conversion

AVIF isalso different because they have pixel format variations as well, possible different conversions. For example a YUV AVIF wouldn't need any changes to matrix, because the input is YUV. You aren't converting to RGB in that case

Webp lossless is RGB only, but lossy web is YUV. So you have to control the YUV to RGB conversion for lossless webp for the correct colors, otherwise bt601 is used

GIF is actually PAL8 , not sure if the conversion is correct, you have to test

[huhn]

Originally Posted by poisondeathray

Ask a ffmpeg developer . There are tickets for this already.

The gAMA, cHRM tag issue is specific to PNG . Even if the underlying YUV to RGB conversion in ffmpeg is correct, the PNG can have problems being displayed in programs

The most consistent PNG display is no gAMA, cHRM tags - it essentially displays the same everywhere

It's easy to verify this, if you strip the gAMA, cHRM tags, the PNG now displays correctly in browsers and other programs.




No .

see #2 in the post above "For non colorimetry flagged sources..."

Also point #2 in my first reply "For a source without colorimetry flags, unless otherwise specified,..."


Nothing is guaranteed to be automatically correct in ffmpeg-land. You should always verify your workflow, because there are a bunch of "gotchas"


Jpeg is different because there are YUV variants, subsampled variants, RGB variants. Also, it is supposed to use full range 601 (pc 601) - as per the jpeg spec , not Rec709, not limited range. Common conversions should be handled correctly automatically in ffmpeg already (because jpeg is/was so common, all the bugs have been fixed), but you can test and verify . Obscure input formats might need some other switches for the correct conversion

AVIF isalso different because they have pixel format variations as well, possible different conversions. For example a YUV AVIF wouldn't need any changes to matrix, because the input is YUV. You aren't converting to RGB in that case

Webp lossless is RGB only, but lossy web is YUV. So you have to control the YUV to RGB conversion for lossless webp for the correct colors, otherwise bt601 is used

GIF is actually PAL8 , not sure if the conversion is correct, you have to test

this is not fix by ffmpeg.
even if the meta data is correct programs will interpret them wrong.
one of these is literally the windows picture "app" it will assume that your display is gamma 2.2 so not the same issue where HDR assumes sRGB for SDR sources but similar.

well ffmpeg could fix interpreting bt 1886 as something around 2.0 by using the bt 709 encoding function for what ever reason. i don't hold my breath on that...

stripping is the way to go!

Originally Posted by SeeMoreDigital

Why can't us regular folk create Jpeg 2000 images as used with DCP (Digital Cinema Package)/DCI?

doesn't this work for like 15 years or so:
https://developer.nvidia.com/nvjpeg2...get_os=Windows
i guess i simply don't know about a DCI limitation?

[poisondeathray]

Originally Posted by huhn

this is not fix by ffmpeg.
even if the meta data is correct programs will interpret them wrong.

Yes, it's a mess.

Just think of the browser case alone, and the millions/billons of incorrectly displayed PNG images on the net (!) . Think of all the software that uses ffmpeg for the backend, and they probably use the "auto" conversion routine. And it's not just a little bit off, sometimes it's +/- 10-12 valuesfor 8bit RGB (!) .

Not only that, they probably use the "fast" swscale conversion, not the accurate swscale conversion algorithm, another few pixel values off (regardless of PNG tags). Even grey scale values can be off for the latter (!). I'd argue the swscale accurate conversion routine should be the default setting when using swscale. It's not 2001 anymore with single core computers.

What's the point of using a "lossless" format if the displayed values are off, sometimes significantly waaay off , or a some avoidable rounding errors are introduced from a fast conversion. ?

The truth is nobody care much about color accuracy. It if looks mostly ok, that's good enough for most people

Originally Posted by huhn

stripping is the way to go!

I'm sure that's what you say to all the ladies

Or avoid ffmpeg for that scenario, or use workarounds .

Originally Posted by hello_hello

 image of my cat:

But of course... a cat

No, seriously live action content can often compresses differently than anime or cartoons, so thanks for that quicky test

But in terms of compatibility, jpeg-xl still has quite low compatibility for usage cases. For compatibility, unfortunately PNG is still up there for lossless. webp is better in terms of lossless RGB compression, decent for compatibility but still a distant second to PNG

[hello_hello]

jay123210599,
SMPlayer can save screenshots of frames as it plays a video as png, webp or jpeg. I haven't messed around with it much but you can increase the playback speed by a fair amount so you shouldn't have to do it in real time. I'd be willing to give SMPlayer the benefit of the doubt in respect to always saving the images with the correct colors. Here's a few jpegs with a section of the rec.709 video they were taken from. Test.zip (6.9 MB)

[GeoffreyA]

I think JPEG XL should be the gold standard for most use cases, lossy and lossless, and especially at the higher end of quality; we've finally got a successor to JPEG, GIF, and PNG, but astonishingly, it is scarcely supported anywhere. Doubtless, there are agendas not to do so: Chrome, for example, did but support was removed or disabled. Apple is supporting it, though.

When I tried cjxl, I was impressed with the results; it beat cjpegli, PNG, and losslessly repacked old JPEGs, gaining a bit of compression. The last feature would be useful in squeezing the size of the many mobile photos we have today, taking up GB after GB.

Gianni Rosato, the SVT-AV1-PSY developer, did some comparisons:

https://giannirosato.com/blog/post/jpegli/
https://giannirosato.com/blog/post/image-comparison/

[Z2697]

JXL is NOT scarcely supported in most areas, especially in free softwares, but it is scarcely supported in browsers just because some "guy" at Google randomly decided to ditch it and Mozilla also randomly decided to put it behind "beta" walls, that just eliminated 80% of the potential user, thus probably eliminated 99% of the potential use case. (no one wants their pictures only visible to 20% of the users right?)
(I'm calm, I'm very calm)

Although it kinda sounds like "big social media companies offloads their bandwidth cost to people who uses their products (computational cost when decoding)", unlike AV1 which has hardware decoding (although YouTube uses software decoding when HW decoding is not available instead of fallback to VP9), but big medias are gonna "save" the bandwidth anyways, if it's not cost you computational power, then it's quality of the content the cost.