Shutter Encoder 709 Colors

[GeoffreyA]

I gave it a go this morning and the simplest way is:

ffmpeg -colorspace bt709 -i "sample tagless video.mkv" -frames:v 24 -sws_flags accurate_rnd "%%d.png"
ffmpeg -colorspace bt709 -i "sample tagless video.mkv" -frames:v 24 -sws_flags accurate_rnd "out.apng"

It converts the colours correctly, as opposed to the default BT.601 conversion, uses accurate rounding (we hope), and contains no gAMA and cHRM chunks in the resulting PNG and APNG files. Setting the transfer characteristics and primaries are what cause the gAMA and cHRM chunks to be set, respectively. Note that "-frames:v 24" merely limits the processing to 24 frames. You can remove that to process the whole file or change the value.






GIF with error-diffusion dithering

ffmpeg -colorspace bt709 -i "sample tagless video.mkv" -frames:v 1 -sws_flags accurate_rnd -sws_dither ed "out.gif"

[Z2697]

Originally Posted by GeoffreyA

I gave it a go this morning and the simplest way is:

ffmpeg -colorspace bt709 -i "sample tagless video.mkv" -frames:v 24 -sws_flags accurate_rnd "%%d.png"
ffmpeg -colorspace bt709 -i "sample tagless video.mkv" -frames:v 24 -sws_flags accurate_rnd "out.apng"

It converts the colours correctly, as opposed to the default BT.601 conversion, uses accurate rounding (we hope), and contains no gAMA and cHRM chunks in the resulting PNG and APNG files. Setting the transfer characteristics and primaries are what cause the gAMA and cHRM chunks to be set, respectively. Note that "-frames:v 24" merely limits the processing to 24 frames. You can remove that to process the whole file or change the value.






GIF with error-diffusion dithering

ffmpeg -colorspace bt709 -i "sample tagless video.mkv" -frames:v 1 -sws_flags accurate_rnd -sws_dither ed "out.gif"

The chroma still looks like point resized
Please use full_chroma_int flag as well
And this way the GIF is not using palette
Please use palettegen and paletteuse

[GeoffreyA]

Thanks, Z2697. Here's the updated script that should solve the problem, Jay:

:: Create PNG and APNG
ffmpeg -colorspace bt709 -i "src.mkv" -frames:v 1 -sws_flags accurate_rnd+full_chroma_int "%%d.png"
ffmpeg -colorspace bt709 -i "src.mkv" -frames:v 1 -sws_flags accurate_rnd+full_chroma_int "out.apng"

:: Create palette and then GIF
ffmpeg -colorspace bt709 -i "src.mkv" -vf palettegen -sws_flags accurate_rnd+full_chroma_int "palette.png"
ffmpeg -colorspace bt709 -i "src.mkv" -i "palette.png" -lavfi paletteuse -frames:v 1 -sws_flags accurate_rnd+full_chroma_int "out.gif"

PNG:


GIF made with palette and Sierra-24A dithering:


Here's a reusable way of doing it. Place this in a BAT file. Change the variables src, start, and frames and run the file.

setlocal

set src=sample tagless video.mkv
set start=2
set frames=24

:: Create PNG and APNG
ffmpeg -ss %start% -colorspace bt709 -i "%src%" -frames:v %frames% -sws_flags accurate_rnd+full_chroma_int "%%d.png"
ffmpeg -ss %start% -colorspace bt709 -i "%src%" -frames:v %frames% -sws_flags accurate_rnd+full_chroma_int "out.apng"

:: Create palette and then GIF
ffmpeg -colorspace bt709 -i "%src%" -vf palettegen -sws_flags accurate_rnd+full_chroma_int "palette.png"
ffmpeg -ss %start% -colorspace bt709 -i "%src%" -i "palette.png" -lavfi paletteuse -frames:v %frames% -sws_flags accurate_rnd+full_chroma_int "out.gif"

endlocal
pause

[Z2697]

Did you notice that the new GIF image is now having "jagged chroma" (well, not actual chroma 'cause it's RGB, but you get the point).
And the old GIF image is somehow not having "jagged chroma"... WTF?

swscale and auto-inserted scale filter is a huge mess.

[jay123210599]

Wait, which one should I go with?

This?

-vf zscale=pin=2:p=2:tin=2:t=2:min=709,format=gbrp

Or this?

-colorspace bt709 -sws_flags accurate_rnd+full_chroma_int

[GeoffreyA]

Originally Posted by Z2697

Did you notice that the new GIF image is now having "jagged chroma" (well, not actual chroma 'cause it's RGB, but you get the point).
And the old GIF image is somehow not having "jagged chroma"... WTF?

swscale and auto-inserted scale filter is a huge mess.

Yes, it did vaguely occur to me but I was focusing more on the skies. I'll see if I can crack this one. The auto-inserted scale filter is, indeed, one big mess; but even if we use zscale, auto-scale is still being called; therefore, it's shorter to leave out zscale.

EDIT: The problem is, for some reason in the GIF example, the sws flags aren't being set:

[auto_scale_1 @ 0000021bcef58300] w:1920 h:1080 fmt:yuv420p csp:bt709 range:unknown sar:1/1 -> w:1920 h:1080 fmt:bgra csp:gbr range:pc sar:1/1 flags:0x00000004

But in the usual case, like for the PNGs:

[auto_scale_0 @ 000001ef02b73200] w:1920 h:1080 fmt:yuv420p csp:bt709 range:unknown sar:1/1 -> w:1920 h:1080 fmt:rgb24 csp:gbr range:pc sar:1/1 flags:0x00042000

Originally Posted by jay123210599

Wait, which one should I go with?

This?

-vf zscale=pin=2:p=2:tin=2:t=2:min=709,format=gbrp

Or this?

-colorspace bt709 -sws_flags accurate_rnd+full_chroma_int

It's up to you and which command-line is more compact or readable. Even if zscale is used, FFmpeg's auto-scale is still being called. Here are the two (unnamed) and I don't think one can ABX the difference:

[poisondeathray]

Originally Posted by jay123210599

Wait, which one should I go with?

This?

-vf zscale=pin=2:p=2:tin=2:t=2:min=709,format=gbrp

Or this?

-colorspace bt709 -sws_flags accurate_rnd+full_chroma_int

If you're using -colorspace, make sure it's before the -i as an input option, otherwise results are wrong

The results are still off slightly using -colorspace bt709 as an input option with -sws_flags accurate_rnd only - you can verify this on colorbars or patterns with known and expected values, there is a brightness shift and even greyscale values are off . Adding full_chroma_int as mentioned in the other thread gets it to the expected values -sws_flags full_chroma_int+accurate_rnd . This gives similar results to zscale

The other way to do it with swscale, instead of -colorspace is with scale

 -vf scale=in_color_matrix=bt709 -sws_flags full_chroma_int+accurate_rnd

Personally I would use zscale in ffmpeg

[jay123210599]

Originally Posted by poisondeathray

If you're using -colorspace, make sure it's before the -i as an input option, otherwise results are wrong

The results are still off slightly using -colorspace bt709 as an input option with -sws_flags accurate_rnd only - you can verify this on colorbars or patterns with known and expected values, there is a brightness shift and even greyscale values are off . Adding full_chroma_int as mentioned in the other thread gets it to the expected values -sws_flags full_chroma_int+accurate_rnd . This gives similar results to zscale

The other way to do it with swscale, instead of -colorspace is with scale

 -vf scale=in_color_matrix=bt709 -sws_flags full_chroma_int+accurate_rnd

Personally I would use zscale in ffmpeg

But both -colorspace and zscale give the same results, right?

[Z2697]

Originally Posted by GeoffreyA

Yes, it did vaguely occur to me but I was focusing more on the skies. I'll see if I can crack this one. The auto-inserted scale filter is, indeed, one big mess; but even if we use zscale, auto-scale is still being called; therefore, it's shorter to leave out zscale.

EDIT: The problem is, for some reason in the GIF example, the sws flags aren't being set:

[auto_scale_1 @ 0000021bcef58300] w:1920 h:1080 fmt:yuv420p csp:bt709 range:unknown sar:1/1 -> w:1920 h:1080 fmt:bgra csp:gbr range:pc sar:1/1 flags:0x00000004

But in the usual case, like for the PNGs:

[auto_scale_0 @ 000001ef02b73200] w:1920 h:1080 fmt:yuv420p csp:bt709 range:unknown sar:1/1 -> w:1920 h:1080 fmt:rgb24 csp:gbr range:pc sar:1/1 flags:0x00042000

If zscale is used, there's still auto-instered scale filter, true, but the actual conversion is done by zscale (if you set the parameters and format correctly), the swscale is only handling like shuffling planes or packed pixels, or whatever. Which is pretty safe. Much safer than leave it to FFmpeg to do wacky conversions (and chroma scalings).

[GeoffreyA]

Originally Posted by Z2697

If zscale is used, there's still auto-instered scale filter, true, but the actual conversion is done by zscale (if you set the parameters and format correctly), the swscale is only handling like shuffling planes or packed pixels, or whatever. Which is pretty safe. Much safer than leave it to FFmpeg to do wacky conversions (and chroma scalings).

Agreed. It's better to use zscale whenever one can, and in my encoding tasks, I always do. The exception is marking sources before the input, in which case auto-scale is not called. Yesterday morning, with Jay's problem, I tried using zscale but couldn't get it to work with the PNG-conversion pipeline, so I resorted to the built-in system.

For simplicity and most viewers, the auto-insert system is powerful, negotiating silently between disparate video and audio formats; it is part of FFmpeg's "magic." But for folk like us, who are aiming for reference correctness and full precision, it is a minefield, and I find I've got to verify every encoding script with -loglevel debug or trace to make sure the auto-inserted scale or resample filters aren't making an appearance.

I think the FFmpeg team needs to audit the swscale codebase, line by line, and make sure that all conversions are being done according to reference convention and full precision (as zimg does); it's not 2010 any more and is baffling that "accurate" calculations have to be enabled.

[Ritsuka]

swscale rewrite is already being done and it's at a good point.

[GeoffreyA]

Originally Posted by Ritsuka

swscale rewrite is already being done and it's at a good point.

I noticed there has been a lot of swscale commits of late.

[jay123210599]

Just to insure, Shutter Encoder have the same problems as ffmpeg when making images out of videos with no colorimetry flags, and has tge same tag issues for making png and apng files out of flagged videos, right?

[GeoffreyA]

Originally Posted by jay123210599

Just to insure, Shutter Encoder have the same problems as ffmpeg when making images out of videos with no colorimetry flags, and has tge same tag issues for making png and apng files out of flagged videos, right?

Yes. It uses FFmpeg as a backend, so the result should be the same.

[jay123210599]

One more thing, for videos with colorimetry flags, should I remove the

min=709,format=gbrp

part and keep the

zscale=pin=2:p=2:tin=2:t=2

in this when making PNG and APNG files?

https://forum.doom9.org/showthread.p...15#post2010515

[GeoffreyA]

Originally Posted by jay123210599

One more thing, for videos with colorimetry flags, should I remove the

min=709,format=gbrp

part and keep the

zscale=pin=2:p=2:tin=2:t=2

in this when making PNG and APNG files?

https://forum.doom9.org/showthread.p...15#post2010515

Leave the first line in. The min=709 is the most important part, defining the colour-space matrix.

[jay123210599]

Originally Posted by GeoffreyA

Leave the first line in. The min=709 is the most important part, defining the colour-space matrix.

What should be the command for videos with colorimetry flags then to avoid tag issues for PNG and APNG?

[GeoffreyA]

Originally Posted by jay123210599

What should be the command for videos with colorimetry flags then to avoid tag issues for PNG and APNG?

ffmpeg -i "INPUT.mkv" -vf zscale=pin=2:p=2:tin=2:t=2:min=709,format=gbrp -sws_flags accurate_rnd+full_chroma_int "%%d.png"
ffmpeg -i "INPUT.mkv" -vf zscale=pin=2:p=2:tin=2:t=2:min=709,format=gbrp -sws_flags accurate_rnd+full_chroma_int "OUTPUT.apng"

Note that I added the accurate flags for swscale because even if zscale is used, swscale is still being called. Also, use "-frames:v 24" to limit processing during testing, where 24 is the number of frames. Otherwise, you are damaging your hard disk for no reason. Make sure you're using the latest build of FFmpeg: the color-space infrastructure was overhauled a few weeks ago.

Alternatively, to remove swscale out of the equation, you can try libplacebo, which uses the GPU and requires Vulkan support:

ffmpeg -init_hw_device vulkan -i "INPUT.mkv" -vf libplacebo=format=rgb24 "%%d.png"

[jay123210599]

Originally Posted by GeoffreyA

ffmpeg -i "INPUT.mkv" -vf zscale=pin=2:p=2:tin=2:t=2:min=709,format=gbrp -sws_flags accurate_rnd+full_chroma_int "%%d.png"
ffmpeg -i "INPUT.mkv" -vf zscale=pin=2:p=2:tin=2:t=2:min=709,format=gbrp -sws_flags accurate_rnd+full_chroma_int "OUTPUT.apng"

Note that I added the accurate flags for swscale because even if zscale is used, swscale is still being called. Also, use "-frames:v 24" to limit processing during testing, where 24 is the number of frames. Otherwise, you are damaging your hard disk for no reason. Make sure you're using the latest build of FFmpeg: the color-space infrastructure was overhauled a few weeks ago.

Alternatively, to remove swscale out of the equation, you can try libplacebo, which uses the GPU and requires Vulkan support:

ffmpeg -init_hw_device vulkan -i "INPUT.mkv" -vf libplacebo=format=rgb24 "%%d.png"

So basically, I use

-vf zscale=pin=2:p=2:tin=2:t=2:min=709,format=gbrp

for both videos with and without colorimetry flag to avoid tag issues?

[GeoffreyA]

Originally Posted by jay123210599

So basically, I use

-vf zscale=pin=2:p=2:tin=2:t=2:min=709,format=gbrp

for both videos with and without colorimetry flag to avoid tag issues?

I tried it on both flagged and unflagged videos (your anime one), and it seems to work all right, producing files without the gAMA and cHRM chunks.