Shutter Encoder 709 Colors

[jay123210599]

Here is a sample of my video without any colorimetry flags:
https://www.mediafire.com/file/0r160p19omfcd6a/sample_tagless_video.mkv/file

Now use Shutter Encoder make png files and/or an apng out of it. Do they have the 709 colors or the 601 ones? Did they have the gAMA and cHRM tag issues?

You can get Shutter Encoder here:
https://www.shutterencoder.com/

[Z2697]

True.

[Z2697]

"C:\Users\a\Desktop\Shutter Encoder 18.6 Windows 64bits\Library\ffmpeg.exe" -strict -2 -hide_banner -threads 0 -i "C:\Users\a\Desktop\sample tagless video.mkv" -compression_level 0 -sws_flags bicubic -an -y "C:\Users\a\Desktop\sample tagless video_%06d.png"
go figure

[jay123210599]

Maybe you'll wanna fix thread title typo.
(click EDIT button, then ADVANCED EDIT, on 1st post)

Fixed it

[jay123210599]

"C:\Users\a\Desktop\Shutter Encoder 18.6 Windows 64bits\Library\ffmpeg.exe" -strict -2 -hide_banner -threads 0 -i "C:\Users\a\Desktop\sample tagless video.mkv" -compression_level 0 -sws_flags bicubic -an -y "C:\Users\a\Desktop\sample tagless video_%06d.png"
go figure

What's this? What does the command mean? Did the PNGs and APNGs have the right colors? Do they have tag issues?

[Z2697]

What's this? What does the command mean? Did the PNGs and APNGs have the right colors? Do they have tag issues?

This is the command that Shutter Encoder uses when converting video to PNG image sequence.
Since it calls FFmpeg CLI tool the same "limitation" applies.

[jay123210599]

This is the command that Shutter Encoder uses when converting video to PNG image sequence.
Since it calls FFmpeg CLI tool the same "limitation" applies.

So it would call the wrong colors and have tag issues for PNG and APNG?

[poisondeathray]

So it would call the wrong colors and have tag issues for PNG and APNG?

No

To recap: for unflagged colorimetry input files, it uses Rec601 for the YUV=>RGB conversion. No PNG chunks written in that case.

See this thread (again)
http://forum.doom9.org/showthread.php?t=185921

I tested Shutter 18.6 - behaviour is the same as default ffmpeg . As you would expect for that command line. There is no other "logic" used in Shutter.

[jay123210599]

No

To recap: for unflagged colorimetry input files, it uses Rec601 for the YUV=>RGB conversion. No PNG chunks written in that case.

See this thread (again)
http://forum.doom9.org/showthread.php?t=185921

I tested Shutter 18.6 - behaviour is the same as default ffmpeg . As you would expect for that command line. There is no other "logic" used in Shutter.

I added colorimetry flags using this command:
-map 0 -c copy -bsf:v h264_metadata=colour_primaries=1:transfer_characteristics=1:matrix_coefficients=1

It should produce the right colors, but will it have the tag issues?

https://www.mediafire.com/file/jsydthtzic94apz/sample_tagged_video.mkv/file

[poisondeathray]

I added colorimetry flags using this command:
-map 0 -c copy -bsf:v h264_metadata=colour_primaries=1:transfer_characteristics=1:matrix_coefficients=1

It should produce the right colors, but will it have the tag issues?

https://www.mediafire.com/file/jsydthtzic94apz/sample_tagged_video.mkv/file



If you add colorimetry tags to the input file, the output file with have PNG chunks and colors will appear incorrect in many programs, including browsers

See this thread (again). You questions were already asked, multiple answers and options give
http://forum.doom9.org/showthread.php?t=185921




3) If you add the colorimetry flags to the source, or for sources with colorimetry flags already present , the ffmpeg PNG's will display incorrect colors in many programs, including browsers (!) - because of gAMA and cHRM tag issue. Other formats like GIF are not affected


In case you've forgotten what you asked 2 weeks ago, and the many answers already provided, use search
http://forum.doom9.org/forum-rules.htm


1a) Use the search function before posting. Chances are your question has already been answered.

[jay123210599]

If you add colorimetry tags to the input file, the output file with have PNG chunks and colors will appear incorrect in many programs, including browsers

See this thread (again). You questions were already asked, multiple answers and options give
http://forum.doom9.org/showthread.php?t=185921





In case you've forgotten what you asked 2 weeks ago, and the many answers already provided, use search
http://forum.doom9.org/forum-rules.htm


1a) Use the search function before posting. Chances are your question has already been answered.


I just want to check and see if Shutter Encoder had the same problems for PNG and APNG as ffmpeg.

Also, can I can correctly assume that the PNG chunks apply to APNG as well, and for unflagged sources, ffmpeg and Shutter Encoder will use 601 by default for all formats like JPG and GIF, and even video formats like MP4 and MKV?

[poisondeathray]

Also, can I can correctly assume that the PNG chunks apply to APNG as well,


You don't have to assume. You already asked. Already answered

See this thread
http://forum.doom9.org/showthread.php?t=185921&page=2


But what about APNG?

Yes ffmpeg apng is affected .




and for unflagged sources, ffmpeg and Shutter Encoder will use 601 by default for all formats like JPG and GIF, and even video formats like MP4 and MKV?

See this thread
http://forum.doom9.org/showthread.php?t=185921

Again, 601 by default for YUV to RGB conversions.

But not all your formats are necessarily RGB

To remind you, JPG usually does not use RGB, but YUV and full range 601 as per the spec JPEG spec, and GIF uses PAL8

See this thread, post 15
http://forum.doom9.org/showthread.php?t=185921


Mp4 and MKV are container formats. So it would depend on what you used inside the MP4 and MKV

[Z2697]

The entity "Jay" (SCP-Doom9-123210599) keeps asking questions follwoing one or more of these rules:
1) The question can easily be looked up with a simple web search engine search.
2) The question can easily be answered by performing the actions involved in the question. "F*ck around and find out".
3) The question is ambiguous.
4) The question is subjective.
5) The question is a variation of previously asked question(s).

:sly:

[jay123210599]

In that case, I'm just going to add flags to my videos and convert it to lossless animated WEBP with the format being bgra, because I know by now that without flags it won't get the 709 colors I want.

Then I'll extract the frames from the WEBP using IrfanView as PNGs where it won't have any tag issues and display the right colors.

[poisondeathray]

In that case, I'm just going to add flags to my videos and convert it to lossless animated WEBP with the format being bgra, because I know by now that without flags it won't get the 709 colors I want.

Then I'll extract the frames from the WEBP using IrfanView as PNGs where it won't have any tag issues and display the right colors.


Or you could use the zscale filter as suggested in that thread. It would be much faster, no intermediate files or extra HDD space (hard drives do break from usage eventually). 1 step instead of 3

[poisondeathray]

In that case, I'm just going to add flags to my videos and convert it to lossless animated WEBP with the format being bgra, because I know by now that without flags it won't get the 709 colors I want...

Also, BTW that proposed 2nd step converts incorrectly. If video has 709 colorimetry flags, it uses 601 for the actual conversion for lossless webp. It even gives you a warning


[libwebp_anim @ 000000b84c414100] Using libwebp for YUV-to-RGB conversion. You may want to consider passing in RGB instead for lossless encoding.

[jay123210599]

Also, BTW that proposed 2nd step converts incorrectly. If video has 709 colorimetry flags, it uses 601 for the actual conversion for lossless webp. It even gives you a warning


[libwebp_anim @ 000000b84c414100] Using libwebp for YUV-to-RGB conversion. You may want to consider passing in RGB instead for lossless encoding.


Well, this is the command I used after adding the flags to my video, and I had not gotten a warning.
ffmpeg -i video.mkv -vf format=bgra -lossless 1 output.webp

[poisondeathray]

Well, this is the command I used after adding the flags to my video, and I had not gotten a warning.
ffmpeg -i video.mkv -vf format=bgra -lossless 1 output.webp

Ok, that looks mostly correct for flagged input, but there are avoidable rounding errors from -vf format's use of swscale's default fast low quality conversion mode. If you look at that same thread, it mentions the swscale issue and workarounds

You decide what's best for you, but IMO it's pointless to use "lossless" if you incur avoidable rounding errors

[jay123210599]

Ok, that looks mostly correct for flagged input, but there are avoidable rounding errors from -vf format's use of swscale's default fast low quality conversion mode. If you look at that same thread, it mentions the swscale issue and workarounds

You decide what's best for you, but IMO it's pointless to use "lossless" if you incur avoidable rounding errors

How should I avoid their rounding errors?

[poisondeathray]

How should I avoid their rounding errors?

Either use the sws accurate flags, or zscale. It was already mentioned and instructions given in yes... that same thread

But if you're using ffmpeg anyways to export webp, then irfanview - you might as well export PNG or apng once directly and correctly from ffmpeg - and skip the adding metadata , webp, and irfanview steps. It was already mentioned... yes in that same thread, the commands to use. No use repeating everything here again

[GeoffreyA]

I gave it a go this morning and the simplest way is:

ffmpeg -colorspace bt709 -i "sample tagless video.mkv" -frames:v 24 -sws_flags accurate_rnd "%%d.png"
ffmpeg -colorspace bt709 -i "sample tagless video.mkv" -frames:v 24 -sws_flags accurate_rnd "out.apng"

It converts the colours correctly, as opposed to the default BT.601 conversion, uses accurate rounding (we hope), and contains no gAMA and cHRM chunks in the resulting PNG and APNG files. Setting the transfer characteristics and primaries are what cause the gAMA and cHRM chunks to be set, respectively. Note that "-frames:v 24" merely limits the processing to 24 frames. You can remove that to process the whole file or change the value.

https://i.imgur.com/KVL8qN8.png

https://i.imgur.com/t16TYxH.png


GIF with error-diffusion dithering

ffmpeg -colorspace bt709 -i "sample tagless video.mkv" -frames:v 1 -sws_flags accurate_rnd -sws_dither ed "out.gif"

https://i.imgur.com/Jb0O9wX.gif

[Z2697]

I gave it a go this morning and the simplest way is:

ffmpeg -colorspace bt709 -i "sample tagless video.mkv" -frames:v 24 -sws_flags accurate_rnd "%%d.png"
ffmpeg -colorspace bt709 -i "sample tagless video.mkv" -frames:v 24 -sws_flags accurate_rnd "out.apng"

It converts the colours correctly, as opposed to the default BT.601 conversion, uses accurate rounding (we hope), and contains no gAMA and cHRM chunks in the resulting PNG and APNG files. Setting the transfer characteristics and primaries are what cause the gAMA and cHRM chunks to be set, respectively. Note that "-frames:v 24" merely limits the processing to 24 frames. You can remove that to process the whole file or change the value.

https://i.imgur.com/KVL8qN8.png

https://i.imgur.com/t16TYxH.png


GIF with error-diffusion dithering

ffmpeg -colorspace bt709 -i "sample tagless video.mkv" -frames:v 1 -sws_flags accurate_rnd -sws_dither ed "out.gif"

https://i.imgur.com/Jb0O9wX.gif

The chroma still looks like point resized
Please use full_chroma_int flag as well
And this way the GIF is not using palette
Please use palettegen (https://ffmpeg.org/ffmpeg-filters.html#palettegen-1)and paletteuse (https://ffmpeg.org/ffmpeg-filters.html#paletteuse)

[GeoffreyA]

Thanks, Z2697. Here's the updated script that should solve the problem, Jay:

:: Create PNG and APNG
ffmpeg -colorspace bt709 -i "src.mkv" -frames:v 1 -sws_flags accurate_rnd+full_chroma_int "%%d.png"
ffmpeg -colorspace bt709 -i "src.mkv" -frames:v 1 -sws_flags accurate_rnd+full_chroma_int "out.apng"

:: Create palette and then GIF
ffmpeg -colorspace bt709 -i "src.mkv" -vf palettegen -sws_flags accurate_rnd+full_chroma_int "palette.png"
ffmpeg -colorspace bt709 -i "src.mkv" -i "palette.png" -lavfi paletteuse -frames:v 1 -sws_flags accurate_rnd+full_chroma_int "out.gif"

PNG:
https://i.imgur.com/ABK9S05.png

GIF made with palette and Sierra-24A dithering:
https://i.imgur.com/pjJ33Fx.gif

Here's a reusable way of doing it. Place this in a BAT file. Change the variables src, start, and frames and run the file.

setlocal

set src=sample tagless video.mkv
set start=2
set frames=24

:: Create PNG and APNG
ffmpeg -ss %start% -colorspace bt709 -i "%src%" -frames:v %frames% -sws_flags accurate_rnd+full_chroma_int "%%d.png"
ffmpeg -ss %start% -colorspace bt709 -i "%src%" -frames:v %frames% -sws_flags accurate_rnd+full_chroma_int "out.apng"

:: Create palette and then GIF
ffmpeg -colorspace bt709 -i "%src%" -vf palettegen -sws_flags accurate_rnd+full_chroma_int "palette.png"
ffmpeg -ss %start% -colorspace bt709 -i "%src%" -i "palette.png" -lavfi paletteuse -frames:v %frames% -sws_flags accurate_rnd+full_chroma_int "out.gif"

endlocal
pause

[Z2697]

Did you notice that the new GIF image is now having "jagged chroma" (well, not actual chroma 'cause it's RGB, but you get the point).
And the old GIF image is somehow not having "jagged chroma"... WTF?

swscale and auto-inserted scale filter is a huge mess.

[jay123210599]

Wait, which one should I go with?

This?
-vf zscale=pin=2:p=2:tin=2:t=2:min=709,format=gbrp

Or this?
-colorspace bt709 -sws_flags accurate_rnd+full_chroma_int

[GeoffreyA]

Did you notice that the new GIF image is now having "jagged chroma" (well, not actual chroma 'cause it's RGB, but you get the point).
And the old GIF image is somehow not having "jagged chroma"... WTF?

swscale and auto-inserted scale filter is a huge mess.

Yes, it did vaguely occur to me but I was focusing more on the skies. I'll see if I can crack this one. The auto-inserted scale filter is, indeed, one big mess; but even if we use zscale, auto-scale is still being called; therefore, it's shorter to leave out zscale.

EDIT: The problem is, for some reason in the GIF example, the sws flags aren't being set:

[auto_scale_1 @ 0000021bcef58300] w:1920 h:1080 fmt:yuv420p csp:bt709 range:unknown sar:1/1 -> w:1920 h:1080 fmt:bgra csp:gbr range:pc sar:1/1 flags:0x00000004

But in the usual case, like for the PNGs:

[auto_scale_0 @ 000001ef02b73200] w:1920 h:1080 fmt:yuv420p csp:bt709 range:unknown sar:1/1 -> w:1920 h:1080 fmt:rgb24 csp:gbr range:pc sar:1/1 flags:0x00042000


Wait, which one should I go with?

This?
-vf zscale=pin=2:p=2:tin=2:t=2:min=709,format=gbrp

Or this?
-colorspace bt709 -sws_flags accurate_rnd+full_chroma_int

It's up to you and which command-line is more compact or readable. Even if zscale is used, FFmpeg's auto-scale is still being called. Here are the two (unnamed) and I don't think one can ABX the difference:

https://i.imgur.com/Cl34Mjv.png

https://i.imgur.com/DWzJuSf.png

[poisondeathray]

Wait, which one should I go with?

This?
-vf zscale=pin=2:p=2:tin=2:t=2:min=709,format=gbrp

Or this?
-colorspace bt709 -sws_flags accurate_rnd+full_chroma_int

If you're using -colorspace, make sure it's before the -i as an input option, otherwise results are wrong

The results are still off slightly using -colorspace bt709 as an input option with -sws_flags accurate_rnd only - you can verify this on colorbars or patterns with known and expected values, there is a brightness shift and even greyscale values are off . Adding full_chroma_int as mentioned in the other thread gets it to the expected values -sws_flags full_chroma_int+accurate_rnd . This gives similar results to zscale

The other way to do it with swscale, instead of -colorspace is with scale


-vf scale=in_color_matrix=bt709 -sws_flags full_chroma_int+accurate_rnd



Personally I would use zscale in ffmpeg

[jay123210599]

If you're using -colorspace, make sure it's before the -i as an input option, otherwise results are wrong

The results are still off slightly using -colorspace bt709 as an input option with -sws_flags accurate_rnd only - you can verify this on colorbars or patterns with known and expected values, there is a brightness shift and even greyscale values are off . Adding full_chroma_int as mentioned in the other thread gets it to the expected values -sws_flags full_chroma_int+accurate_rnd . This gives similar results to zscale

The other way to do it with swscale, instead of -colorspace is with scale


-vf scale=in_color_matrix=bt709 -sws_flags full_chroma_int+accurate_rnd



Personally I would use zscale in ffmpeg

But both -colorspace and zscale give the same results, right?

[Z2697]

Yes, it did vaguely occur to me but I was focusing more on the skies. I'll see if I can crack this one. The auto-inserted scale filter is, indeed, one big mess; but even if we use zscale, auto-scale is still being called; therefore, it's shorter to leave out zscale.

EDIT: The problem is, for some reason in the GIF example, the sws flags aren't being set:

[auto_scale_1 @ 0000021bcef58300] w:1920 h:1080 fmt:yuv420p csp:bt709 range:unknown sar:1/1 -> w:1920 h:1080 fmt:bgra csp:gbr range:pc sar:1/1 flags:0x00000004

But in the usual case, like for the PNGs:

[auto_scale_0 @ 000001ef02b73200] w:1920 h:1080 fmt:yuv420p csp:bt709 range:unknown sar:1/1 -> w:1920 h:1080 fmt:rgb24 csp:gbr range:pc sar:1/1 flags:0x00042000


If zscale is used, there's still auto-instered scale filter, true, but the actual conversion is done by zscale (if you set the parameters and format correctly), the swscale is only handling like shuffling planes or packed pixels, or whatever. Which is pretty safe. Much safer than leave it to FFmpeg to do wacky conversions (and chroma scalings).

[GeoffreyA]

If zscale is used, there's still auto-instered scale filter, true, but the actual conversion is done by zscale (if you set the parameters and format correctly), the swscale is only handling like shuffling planes or packed pixels, or whatever. Which is pretty safe. Much safer than leave it to FFmpeg to do wacky conversions (and chroma scalings).

Agreed. It's better to use zscale whenever one can, and in my encoding tasks, I always do. The exception is marking sources before the input, in which case auto-scale is not called. Yesterday morning, with Jay's problem, I tried using zscale but couldn't get it to work with the PNG-conversion pipeline, so I resorted to the built-in system.

For simplicity and most viewers, the auto-insert system is powerful, negotiating silently between disparate video and audio formats; it is part of FFmpeg's "magic." But for folk like us, who are aiming for reference correctness and full precision, it is a minefield, and I find I've got to verify every encoding script with -loglevel debug or trace to make sure the auto-inserted scale or resample filters aren't making an appearance.

I think the FFmpeg team needs to audit the swscale codebase, line by line, and make sure that all conversions are being done according to reference convention and full precision (as zimg does); it's not 2010 any more and is baffling that "accurate" calculations have to be enabled.

[Ritsuka]

swscale rewrite is already being done and it's at a good point.

[GeoffreyA]

swscale rewrite is already being done and it's at a good point.

I noticed there has been a lot of swscale commits of late.

[jay123210599]

Just to insure, Shutter Encoder have the same problems as ffmpeg when making images out of videos with no colorimetry flags, and has tge same tag issues for making png and apng files out of flagged videos, right?

[GeoffreyA]

Just to insure, Shutter Encoder have the same problems as ffmpeg when making images out of videos with no colorimetry flags, and has tge same tag issues for making png and apng files out of flagged videos, right?

Yes. It uses FFmpeg as a backend, so the result should be the same.

[jay123210599]

One more thing, for videos with colorimetry flags, should I remove the min=709,format=gbrp part and keep the zscale=pin=2:p=2:tin=2:t=2 in this when making PNG and APNG files?

https://forum.doom9.org/showthread.php?p=2010515#post2010515

[GeoffreyA]

One more thing, for videos with colorimetry flags, should I remove the min=709,format=gbrp part and keep the zscale=pin=2:p=2:tin=2:t=2 in this when making PNG and APNG files?

https://forum.doom9.org/showthread.php?p=2010515#post2010515

Leave the first line in. The min=709 is the most important part, defining the colour-space matrix.

[jay123210599]

Leave the first line in. The min=709 is the most important part, defining the colour-space matrix.

What should be the command for videos with colorimetry flags then to avoid tag issues for PNG and APNG?

[GeoffreyA]

What should be the command for videos with colorimetry flags then to avoid tag issues for PNG and APNG?

ffmpeg -i "INPUT.mkv" -vf zscale=pin=2:p=2:tin=2:t=2:min=709,format=gbrp -sws_flags accurate_rnd+full_chroma_int "%%d.png"
ffmpeg -i "INPUT.mkv" -vf zscale=pin=2:p=2:tin=2:t=2:min=709,format=gbrp -sws_flags accurate_rnd+full_chroma_int "OUTPUT.apng"

Note that I added the accurate flags for swscale because even if zscale is used, swscale is still being called. Also, use "-frames:v 24" to limit processing during testing, where 24 is the number of frames. Otherwise, you are damaging your hard disk for no reason. Make sure you're using the latest build of FFmpeg: the color-space infrastructure was overhauled a few weeks ago.

Alternatively, to remove swscale out of the equation, you can try libplacebo, which uses the GPU and requires Vulkan support:

ffmpeg -init_hw_device vulkan -i "INPUT.mkv" -vf libplacebo=format=rgb24 "%%d.png"

[jay123210599]

ffmpeg -i "INPUT.mkv" -vf zscale=pin=2:p=2:tin=2:t=2:min=709,format=gbrp -sws_flags accurate_rnd+full_chroma_int "%%d.png"
ffmpeg -i "INPUT.mkv" -vf zscale=pin=2:p=2:tin=2:t=2:min=709,format=gbrp -sws_flags accurate_rnd+full_chroma_int "OUTPUT.apng"

Note that I added the accurate flags for swscale because even if zscale is used, swscale is still being called. Also, use "-frames:v 24" to limit processing during testing, where 24 is the number of frames. Otherwise, you are damaging your hard disk for no reason. Make sure you're using the latest build of FFmpeg: the color-space infrastructure was overhauled a few weeks ago.

Alternatively, to remove swscale out of the equation, you can try libplacebo, which uses the GPU and requires Vulkan support:

ffmpeg -init_hw_device vulkan -i "INPUT.mkv" -vf libplacebo=format=rgb24 "%%d.png"

So basically, I use -vf zscale=pin=2:p=2:tin=2:t=2:min=709,format=gbrp for both videos with and without colorimetry flag to avoid tag issues?

[GeoffreyA]

So basically, I use -vf zscale=pin=2:p=2:tin=2:t=2:min=709,format=gbrp for both videos with and without colorimetry flag to avoid tag issues?

I tried it on both flagged and unflagged videos (your anime one), and it seems to work all right, producing files without the gAMA and cHRM chunks.