Overlay produces unexpected color

[george84]

SMILrootlayout_backgroundImage = "C:\Dokumente und Einstellungen\Kevin\Eigene Dateien\SMIL\BSG Prototyp\media\testchart1.png"
SMILrootlayout_width = 1920#
SMILrootlayout_height = 1080
# resizing background to width and height
SMILrootlayout = Lanczos4Resize(ImageSource(SMILrootlayout_backgroundImage,1,1,10,pixel_type="RGB32"), SMILrootlayout_width, SMILrootlayout_height).ConvertToYV12(matrix = "REC709").KillAudio()
S2_width = 360#
S2_height = 202#
S2_bgc = color_red#
S2_bgo = 1#
S2_Nframes = 0
# resizing background to width and height
yyy = BlankClip(length = 0, fps = 10, width = S2_width, height = S2_height, color = S2_bgc)
S2 = BlankClip(length = 0, fps = 10, width = S2_width, height = S2_height, color = S2_bgc).ConvertToYV12(matrix = "REC709").KillAudio()
SMILrootlayout = Overlay(SMILrootlayout, S2, x = 4, y = 4, pc_range = true)
res = Overlay(SMILrootlayout, S2.ConvertToRGB32(matrix = "REC709"), x = 4, y = 210, pc_range = true)
res

Color_red is defined as (255,0,0). In above example the first overlay will produce a red of (234,0,3) while the second overlay produces (255,0,0) as expected.

Is there an explanation for this behavior? The documentation is not clear on this.

[Gavino]

The problem is not with Overlay itself, but instead in the way you are viewing the results.

As the script result is a YUV clip, it will be converted to RGB for viewing by whatever application you are using - this conversion will normally be done using the Rec.601 matrix. If the clip is actually encoded with Rec 709 colorimetry, this process will produce wrong results.

The problem is independent of Overlay, as this simpler script shows the same behaviour:
BlankClip(color=color_red)
ConvertToYV12(matrix="REC709")

In your script, the first overlay clip (S2) has Rec 709, so you get the result you describe when viewing.
(Since the clip is YV12, Overlay does no RGB->YUV conversion and the pc_range parameter is ignored.)

In your second overlay, the clip is explicitly converted back to RGB32 using the correct Rec709 matrix. It will then be internally converted back to YUV by Overlay using Rec.601, so will be viewed correctly in your environment.

The solution to this general problem is, when viewing Rec709 clips, set up your application accordingly if it allows the colorimetry to be set. If it does not, you need to add ConvertToRGB32(matrix="REC709") at the end of your script for viewing purposes.

[george84]

Thank you

So at least I can be sure that avisynth processes colors correctly. That's good news.

If it does not, you need to add ConvertToRGB32(matrix="REC709") at the end of your script for viewing purposes.

I display the avs with Vdub. I used stable version 1.9, but found no way to apply correct settings for output. Then I read that Vdub 1.10.2 has better support for REC709. So I tried there too. In Menu "Color Depth" I tested several entries with REC709 but never got the right result. Do you know if Vdub can be configured to solve this problem?

It seems that when encoding with RipBot264 I get the same problem. I get the right red only by adding ConvertToRGB32(matrix="REC709") at end of script.

It will then be internally converted back to YUV by Overlay using Rec.601
Do you suggest that overlay ALWAYS uses Rec.601 and never REC709?

[Rumbah]

To avoid the up and down converting you could use Colormatrix to change the colorimetry directy from rec.709 to rec.601.

[Gavino]

Do you know if Vdub can be configured to solve this problem?
I'm not up-to-date with the latest VDub, but I don't think there is any way to tell it that its input (ie Avisynth's output) is Rec709. So in that case you need to convert to RGB32 in the script.

Do you suggest that overlay ALWAYS uses Rec.601 and never REC709?
Yes.
(Actually, it uses PC.601 if pc_range=true, but never uses a 709 matrix.)