Trying to recreate a composite signal simulation

[RGMOfficial]

I've been thinking very much about doing an simulation of an video signal through Composite.

But i will need some help in order to get that done.
If this idea works well, i will try to rewrite DotCrawl++ to use this new script.

First is to create a luma signal.
The luma signal has an checkerboard look (and that's where dotcrawls come from).

I will be very thankfully for the help.

Images from DoctorDothraki's Domesday Duplicator uploads.

https://i.imgur.com/iltJuUT.png
https://i.imgur.com/uZV6NN9.png

[DTL]

Luma do not have dotcrawl - it is chroma encoded addition. So to create more or less standard chroma subcarrier data it is required to read documentation and implement colour encoder to target colour system (typically PAL or NTSC) and more or less simply mix to luma data. The full composite encoder may (or typically do) have luma partial suppression filter at chroma band to make cross-distortions less visible. So better composite simulation mixer need also filter luma signal before final output mix.

Signal with checkerboard it is already mixed composite signal (luma +encoded chroma). The PAL/NTSC chroma subcarrier have phase flip to 180degrees (inversion) at each line to make residual distortion (after not ideal luma/chroma separation at decoder) less visible (checkerboard instead of vertical lines).

The sad story about digital processing after composite decoder - the input composite signal may be partially damaged after passing colour decoder (its residual colour subcarrier that cause effect of dotcrawl) so it may be not completely equal to simulated standard composite data. It may have both phase and amplitude errors not present in input composite data and these errors may be unique for each colour decoder implementation. So better way is in that doomsday-duplicator project is full composite colour decoder with dotcrawl effect suppression by single developer - not trying to make addition to (unknown) external decoder.

[RGMOfficial]

Luma do not have dotcrawl - it is chroma encoded addition. So to create more or less standard chroma subcarrier data it is required to read documentation and implement colour encoder to target colour system (typically PAL or NTSC) and simply mix to luma data.

Signal with checkerboard it is already mixed composite signal (luma +encoded chroma). The PAL/NTSC chroma subcarrier have phase flip to 180degrees (inversion) at each line to make residual distortion (after not ideal luma/chroma separation at decoder) less visible (checkerboard instead of vertical lines).

Oh. Understand.
Interesting info though.

Let's see. How i can make this chroma addition?
I need to think about it.

[DTL]

Take any open documentation on your target colour composite system encoder and make math program to some execution environment.

PAL and NTSC quadrature modulation is simple enough like multiplication and sum of sin() functions. I think most information can be taken from ITU-BT 470 - https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.470-6-199811-S!!PDF-E.pdf

For example math equation for chroma signal from 2.9 of Table 2. Also read about phase switching of chroma subcarrier - like 2.16 of Table 2 and may be other notes in that document.

[RGMOfficial]

Take any open documentation on your target colour composite system encoder and make math program to some execution environment.

PAL and NTSC quadrature modulation is simple enough like multiplication and sum of sin() functions. I think most information can be taken from ITU-BT 470 - https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.470-6-199811-S!!PDF-E.pdf

For example math equation for chroma signal from 2.9 of Table 2. Also read about phase switching of chroma subcarrier - like 2.16 of Table 2 and may be other notes in that document.

Very interesting and helpful.
I will see if i can use math with images on AVS, and then use these equations.

[kedautinh12]

If had new ver, can you notification at your doom9 post??

[RGMOfficial]

I'm trying out some way...

Tweak(hue=44)

Y = ShowY()
I = ShowU()
Q = ShowV()

C = BlankClip(last).mt_lutspa(mode="absolute", chroma="-128", expr=("x 4 % 2 < 0 1 ? y 4 % 2 < 0 1 ? + 1 == 0 255 ?"))

I_luma = ex_lutxy(I,C,"x y - abs")
Q_luma = ex_lutxy(Q,I_luma,"x y - abs")

Y = ex_lutxy(Y,Q_luma,"x y + abs")

return Y

https://i.imgur.com/qsLnFsX.png

[RGMOfficial]

So what about the chroma signal?

[RGMOfficial]

Now, if someone can help fix the chroma issue i had.

(I'm using an DotCrawl++ function for the mults at the moment, but i will change to an original function later)

function composite_effect(clip clip, float "subcarrier_amplitude"){
subcarrier_amplitude = Default(subcarrier_amplitude,0.5)

clip.Tweak(hue=44)

Y = ShowY()
I = ShowU()
Q = ShowV()

## LUMA SIGNAL (merge information from chroma planes)

U_C = dcpp_makedotcrawl(2)
V_C = ex_invert(U_C)

I_luma = ex_lutxy(I,U_C,"x y - abs "+string(subcarrier_amplitude)+" *")
Q_luma = ex_lutxy(Q,V_C,"x y - abs "+string(subcarrier_amplitude)+" *")

Y = ex_lutxyz(Y,I_luma,Q_luma,"x y + z + abs "+string(subcarrier_amplitude)+" *")

I = BlankClip(Y)
Q = BlankClip(Y)

c = Overlay(BlankClip(Y),Y,x=2)

sum = ex_lutxy(Y,c,"x y -")

sum = sum.Overlay(c,mode="add",mask=c,opacity=1).blur(1,0)

Y = sum.Levels(0,1,128,0,255)
chroma = ex_lutxy(c,Y,"x y 0.5 * -")

chroma = invert(chroma)

chroma = ex_lut(chroma,"x 0.31 * 0.5 /").invert()

chroma_s1 = Overlay(BlankClip(chroma),chroma,x=2)

chroma_s3 = Overlay(chroma,chroma,x=2,mask=U_C)
chroma_s4 = Overlay(chroma,chroma,x=2,mask=V_C)

I_1 = Overlay(chroma_s3,chroma_s3,x=2,mask=U_C)
Q_1 = Overlay(chroma_s4,chroma_s4,x=-2,mask=V_C)

I_2 = Overlay(chroma_s4,chroma_s4,x=-2,mask=U_C)
Q_2 = Overlay(chroma_s3,chroma_s3,x=2,mask=V_C)

GScriptClip(last,"""
#Y = ConditionalSelect(Y, "current_frame % 2",Y,Y_test)
I = ConditionalSelect(I, "current_frame % 2",I_1,I_2)
Q = ConditionalSelect(Q, "current_frame % 2",Q_1,Q_2)
""",local=true)

I = I_1
Q = Q_1

return YToUV(I,Q,Y).Tweak(hue=-44)
StackHorizontal(last,vid)

#StackHorizontal(original,chroma)
}

https://i.imgur.com/K374Fix.png

[DTL]

After you calculate I and Q chroma signals - you need to quadrature modulate chroma subcarrier and mix (addition) it with your luma data. The composite data is Y-only - not visibly colored. To demodulate it back to coloured you need to process it with composite decoder. I not sure if it is easy to do with AVS scripting only - better with C-program.

[RGMOfficial]

After you calculate I and Q chroma signals - you need to quadrature modulate chroma subcarrier and mix (addition) it with your luma data. The composite data is Y-only - not visibly colored. To demodulate it back to coloured youneed to process it with composite decoder. I not sure if it is easy to do with AVS scriptong only - better with C-program.

Ok, so can you show me in C program? Maybe i can translate this code into AviSynth.

[DTL]

For software composite demodulator you may look at vhs-decode project https://github.com/oyvindln/vhs-decode . Looks like this file https://github.com/oyvindln/vhs-decode/blob/vhs_decode/cvbsdecode/process.py

[RGMOfficial]

Ok, so after some comeback, i did made a new version, but still no correct colors. I need help because i can't seem to find what's the issue?

"your video source"

ConvertToYUV444(interlaced=true,chromaresample="spline64")
Spline64Resize(760-(4+4),height).AddBorders(4,0,4,0)
AddBorders(0,3,0,3)

ConvertBits(16)

CombinePlanes(last,planes="YVU",source_planes="YUV")
Tweak(hue=33)

ex_boxblur(2,0,mode="weighted",Y=1,UV=3)

original = last

## Pattern ##
chk = mt_lutspa(last, mode="absolute", chroma="-128", expr=("x 4 % 2 < 0 1 ? y 4 % 2 < 0 1 ? + 1 == 0 range_max ?")).Invert()
chkInverse = chk.Invert()
chk = Interleave(chk,chkInverse)
chkInverse = chk.Invert()

## Mix into luma ##
function AbsChroma(clip "chan"){
pos=chan.ColorYUV(off_y=-128)
neg=Invert(chan).ColorYUV(off_y=-128)

return Overlay(pos,neg,mode="Add")
}
function AbsChromaAdd(clip "chan"){
pos=chan.ColorYUV(off_y=-128)
neg=Invert(chan).ColorYUV(off_y=-128)

return Overlay(pos,neg,mode="Subtract")
}

chr = ex_lutxy(ExtractU(original),chk.ExtractY(),"x y range_max / *")
chr = ex_lutxyz(chr,ExtractV(original),chkInverse.ExtractY(),"x y z range_max / * +")

chr = chr.AbsChroma()

ex_lutxy(ExtractY(original),chr,"x y +")

## Separate chroma from luma ##
Y = last.SeparateFields().F2Quiver(1,1,128,255,0).Weave()

chroma = ex_lutxy(last,Y,"x y -").ColorYUV(off_y=128)

U = Overlay(chroma,chroma,x=2,mask=chk)
V = Overlay(chroma,chroma,x=2,mode="add",opacity=.5,mask=chkInverse)

## Join planes into one colored clip ##
final = CombinePlanes(Y,U,V,planes="YUV",source_planes="YYY",sample_clip=original)
final = final.Tweak(hue=-33).CombinePlanes(planes="YVU",source_planes="YUV")
return final

[Dogway]

For modulation and demodulation you have to rotate the signal 90º, for that you use sin and cos, for QAM specifically read over here (https://en.wikipedia.org/wiki/Quadrature_amplitude_modulation#Demodulation_of_QAM).

[RGMOfficial]

For modulation and demodulation you have to rotate the signal 90º, for that you use sin and cos, for QAM specifically read over here (https://en.wikipedia.org/wiki/Quadrature_amplitude_modulation#Demodulation_of_QAM).

Can you show how i can do that in something like lutspa or lutxy?

[Dogway]

You have an example here (https://github.com/libretro/slang-shaders/blob/master/ntsc/shaders/maister/ntsc-pass1-encode-demodulate.inc) or this one (https://github.com/libretro/slang-shaders/blob/master/pal/shaders/pal-singlepass.slang). Just mind you the YIQ model was only used up to 1979, since then YUV (not YCbCr) was used for all regions.

If you want to know the exact figures check this file (https://github.com/Dogway/emulation-random/blob/master/RetroArch/Shaders/ntsc/ntsc-signal-bandwidth.slang) where I researched most of them.

[RGMOfficial]

You have an example here (https://github.com/libretro/slang-shaders/blob/master/ntsc/shaders/maister/ntsc-pass1-encode-demodulate.inc) or this one (https://github.com/libretro/slang-shaders/blob/master/pal/shaders/pal-singlepass.slang). Just mind you the YIQ model was only used up to 1979, since then YUV (not YCbCr) was used for all regions.

If you want to know the exact figures check this file (https://github.com/Dogway/emulation-random/blob/master/RetroArch/Shaders/ntsc/ntsc-signal-bandwidth.slang) where I researched most of them.

Ok, so... i did try my best to convert into RPN.

I = ex_lutxy(ExtractU(original),ExtractV(original), "y range_half - 0.83867057 * x range_half - 0.54463904 * - range_half +")
Q = ex_lutxy(ExtractU(original),ExtractV(original), "y range_half - 0.54463904 * x range_half - 0.83867057 * + range_half +")

chromaModFreq = (4.0 * Pi() / 15.0)

expr = ("pi x + y 2 % frameno + + "+string(chromaModFreq)+" *")

chkI = ex_lutspa(BlankClip(last),mode="absolute",expr=expr+" sin").extracty()
chkI = ex_lutxy(I,chkI,"x y *")
chkQ = ex_lutspa(BlankClip(last),mode="absolute",expr=expr+" cos").extracty()
chkQ = ex_lutxy(Q,chkQ,"x y *")
return chkQ.Subtitle(mt_infix(expr))

https://files.catbox.moe/wg2qsj.png

[Dogway]

I guess it's right? You can try to demodulate it to check if it passes the roundtrip test. The Crosstalk matrix is defined here (https://github.com/libretro/slang-shaders/blob/master/ntsc/shaders/maister/ntsc-param.inc#L22) depending if you want composite or s-video.

EDIT: I updated my figures (https://github.com/Dogway/emulation-random/blob/24e938ed54ec741ca1668e302e08048ca050d5b8/RetroArch/Shaders/ntsc/ntsc-signal-bandwidth.slang#L26) since I just read in the Wiki for YIQ that YUV required equal bandwidths, so that's a hint to know whether YIQ or YUV was used.

EDIT2: Currently you are using YUV to YIQ coefficients, but if I'm not wrong your source is in YCbCr not YUV. So you either use YCbCr->YUV->YIQ coefficients for very old composite signals, or simply use YCbCr->YUV coefficients.

YCbCr->YUV
Cb->U = 0.435812284313725 / 0.5 = 0.87162456862745
Cr->V = 0.615857694117647 / 0.5 = 1.231715388235294

YCbCr->YUV->YIQ
Cb->I = 0.871624568627450 * sin(33) = 0.474720768297668
Cb->Q = 0.871624568627450 * cos(33) = 0.731005873796788
Cr->I = 1.231715388235294 * cos(33) = 1.033003446729065
Cr->Q = 1.231715388235294 * sin(33) = 0.670840286601698


Here are all the conversion matrices, you can test them with MatrixClip() using switch=true when RGB is included:

toRad = 33*(pi/180)
YUVtoYIQ = [1.0, 0.00000000, 0.00000000,\
0.0, -sin(toRad), cos(toRad),\
0.0, cos(toRad), sin(toRad)]
Chroma range (±0.435812284313725,±0.615857694117647)
RGBtoYUV = [0.298912, -0.147111592156863, 0.615857694117647,\
0.586603, -0.288700692156863, -0.515290478431373,\
0.114485, 0.435812284313725, -0.100567215686275]
Chroma range (±0.59662443399429320,±0.52277213335037230)
RGBtoYIQ = [0.298912, 0.59662443399429320, 0.21204200387001038,\
0.586603, -0.27492129802703860, -0.52277213335037230,\
0.114485, -0.32170313596725464, 0.31073009967803955]
YCCtoYUV = [1.0, 0.000000000000000, 0.000000000000000,\
0.0, 0.871624568627450, 0.000000000000000,\
0.0, 0.000000000000000, 1.231715388235294]
YCCtoYIQ = [1.0, 0.0000000000000000, 0.0000000000000000,\
0.0, -0.4747207760810852, 0.7310058474540710,\
0.0, 1.0330034494400024, 0.6708403229713440]

[RGMOfficial]

I guess it's right? You can try to demodulate it to check if it passes the roundtrip test. The Crosstalk matrix is defined here (https://github.com/libretro/slang-shaders/blob/master/ntsc/shaders/maister/ntsc-param.inc#L22) depending if you want composite or s-video.

EDIT: I updated my figures (https://github.com/Dogway/emulation-random/blob/24e938ed54ec741ca1668e302e08048ca050d5b8/RetroArch/Shaders/ntsc/ntsc-signal-bandwidth.slang#L26) since I just read in the Wiki for YIQ that YUV required equal bandwidths, so that's a hint to know whether YIQ or YUV was used.

EDIT2: Currently you are using YUV to YIQ coefficients, but if I'm not wrong your source is in YCbCr not YUV. So you either use YCbCr->YUV->YIQ coefficients for very old composite signals, or simply use YCbCr->YUV coefficients.

YCbCr->YUV
Cb->U = 0.435812284313725 / 0.5 = 0.87162456862745
Cr->V = 0.615857694117647 / 0.5 = 1.231715388235294

YCbCr->YUV->YIQ
Cb->I = 0.871624568627450 * 0.83867057 = 0.731005873796788
Cb->Q = 0.871624568627450 * 0.54463904 = 0.474720768297668
Cr->I = 1.231715388235294 * 0.83867057 = 1.033003446729065
Cr->Q = 1.231715388235294 * 0.54463904 = 0.670840286601698


Here are all the conversion matrices, you can test them with MatrixClip() using switch=true when RGB is included:

YUVtoYIQ = [1.0, 0.00000, 0.00000,\
0.0, -sin(33), cos(33),\
0.0, cos(33), sin(33)]
RGBtoYUV = [0.298912, -0.147111592156863, 0.615857694117647,\
0.586603, -0.288700692156863,-0.515290478431373,\
0.114485, 0.435812284313725,-0.100567215686275]
RGBtoYIQ = [0.298912, 0.13892203569412231, 0.61775660514831540,\
0.586603, 0.29551663994789124, -0.51141208410263060,\
0.114485, -0.43443867564201355, -0.10634452104568481]
YCCtoYUV = [1.0, 0.000000000000000, 0.000000000000000,\
0.0, 0.871624568627450, 0.000000000000000,\
0.0, 0.000000000000000, 1.231715388235294]
YCCtoYIQ = [1.0, 0.000000000000000000, 0.000000000000000000,\
0.0, -0.871547758579254200, -0.011572339572012424,\
0.0, -0.016353175044059753, 1.231606841087341300]

Ok. YUV to YIQ is okay, but still the encoding is not what i expected (the expected is being an checkerboard-sine-wave-like pattern, following QAM).

YUVtoYIQ = [1.0, 0.00000, 0.00000, 0.0, -sin(33), cos(33), 0.0, cos(33), sin(33)]

I = ExtractU(MatrixClip(original,YUVtoYIQ,switch=false))
Q = ExtractV(MatrixClip(original,YUVtoYIQ,switch=false))

chromaModFreq = (4.0 * Pi() / 15.0)

expr = ("pi x + y 2 % frameno + + "+string(chromaModFreq)+" *")

chkIm = ex_lutspa(BlankClip(last),mode="absolute",expr=expr+" sin").extracty()
chkI = ex_lutxy(I,chkIm,"x y *")
chkQm = ex_lutspa(BlankClip(last),mode="absolute",expr=expr+" cos").extracty()
chkQ = ex_lutxy(Q,chkQm,"x y *")

chr = Overlay(BlankClip(original).extracty(),chkI,mode="add")
chr = Overlay(chr,chkQ,mode="add")

return ex_lutxy(ExtractY(original),chr,"x y +").Subtitle(mt_infix(expr))

https://files.catbox.moe/q9qsbz.png

[Dogway]

First of all, sorry for the wrong figures since the YUVtoYIQ is wrong in many places, including the Wikipedia. I double checked and added the proper derivations. The post above was edited with the correct matrices for XXXtoYIQ. Soon I will add these to TransformsPack - Models

With that said, the way to convert YCbCr to YIQ is as follows:

YCbCr source

ConvertToYUV444(chromaresample="spline16")
ConvertBits(32,fulld=true)

MatrixClip(YCCtoYIQ, switch=false)
ConvertBits(8,dither=1)

# Now decide what you want to do with Luma, full or narrow range?

I = ExtractU()
Q = ExtractV()

[RGMOfficial]

Hey, Dogway, so i've been researching and i found this NTSC adaptive shader.
Maybe, you can identify what this shader is doing? That may be useful for this project.

https://git.libretro.com/libretro-assets/glsl-shaders/-/blob/master/ntsc/shaders/ntsc-adaptive/ntsc-pass1.glsl
https://git.libretro.com/libretro-assets/glsl-shaders/-/blob/master/ntsc/shaders/ntsc-adaptive/ntsc-pass2.glsl

[Dogway]

Hey, Dogway, so i've been researching and i found this NTSC adaptive shader.
Maybe, you can identify what this shader is doing? That may be useful for this project.

https://git.libretro.com/libretro-assets/glsl-shaders/-/blob/master/ntsc/shaders/ntsc-adaptive/ntsc-pass1.glsl
https://git.libretro.com/libretro-assets/glsl-shaders/-/blob/master/ntsc/shaders/ntsc-adaptive/ntsc-pass2.glsl

A bit busy today but will have a look tomorrow. I also made a shader to blur luma/chroma given the signal bandwidth so maybe I can correlate a few things. Mainly to start of I would only change the RGB<->YIQ with YUV given that further than 1979 (and maybe even earlier) YIQ was never used.