Rainbow and Dot Crawl masking

[real.finder]

I was wonder if it's possible to mask these annoying Analog Video Artifacts

since I am not Engineer I just try with some methods like edges mask and some edges mask in chroma with some lut things, but all not do it, then I went to uncle Google ask for what make Dot Crawl, then after some clicks I get to this interesting link https://bitjammer.github.io/glsl/ntsc1.html

but sadly, I don't know how to use these masks in the link in avs, and if it even possible to do it in masktools

so lets see if anyone here that can do it

[tormento]

https://bitjammer.github.io/glsl/ntsc1.html
AFAIK when talking about high frequency noise, FFT is involved. I used it some years ago to remove moire effect on some images, with a Photoshop plugin. Try and search something about FFT and moire effect, perhaps some useful link will be shown.

[Sharc]

For NTSC footage I found this quite effective for DotCrawl removal.
(It would have to be modified for PAL)
http://home.earthlink.net/~tacosalad/video/dotcrawl.html

Or perhaps you can grab some ideas from here:
https://forum.doom9.org/showthread.php?t=173030

[GMJCZP]

Then you have to travel to the past:

GuavaComb (https://avisynth.org.ru/docs/english/externalfilters/guavacomb.htm)

I have used it and it does remove DotCrawl.

[real.finder]

well thanks everyone for comments, I already have many filters to remove Rainbow and Dot Crawl, but for perfect filtering mask is needed :) that why the title said masking

[GMJCZP]

Camembert or TcannyMod will not have value for you?, I do not know if Khameleon (https://forum.doom9.org/showthread.php?p=1813569)is useful.

[real.finder]

Camembert or TcannyMod will not have value for you?, I do not know if Khameleon (https://forum.doom9.org/showthread.php?p=1813569)is useful.

these masks not for Rainbow and Dot Crawl

in the link in OP there are some Formulas for that

https://bitjammer.github.io/_images/slow-dot-crawl-keen.gif
test gif

https://bitjammer.github.io/_images/dyprimedx.png
Rainbow mask using High Frequency Pixel Changes or luma difference (Y’[x, y] = Y’[x, y] - Y’[x - 1, y]) I don't know any method to get same as this in avs

https://bitjammer.github.io/_images/diqdx.png
Dot Crawl mask, I write this lut mt_lutxy(utoy8,vtoy8, "x 128 - 2 * y 128 - 2 * - abs 2 *") it's kinda do it but I'm not sure since I test it in some clip and didn't mask all Dot Crawl and mask some wrong parts too

[GMJCZP]

I am interested in this question, if this comes to a happy conclusion it could be applied in other situations such as deflickering, its classical methods can cause ghosting. Precisely I had thought about masking luts for this but I do not have the knowledge either, Didée was a genius in this.

[tormento]

Rainbow mask using High Frequency Pixel

Here (https://dsp.stackexchange.com/questions/35763/frequency-and-phase-range-of-the-fft-of-an-image/35765), here (https://dsp.stackexchange.com/questions/10996/how-to-process-video-with-dft-fft-getting-the-frequency-of-green-color-changes) and on Doom9 too (https://forum.doom9.org/showthread.php?t=165409). Get a look to FFT, IMHO it's the best (mathematical) way.

[real.finder]

Here (https://dsp.stackexchange.com/questions/35763/frequency-and-phase-range-of-the-fft-of-an-image/35765), here (https://dsp.stackexchange.com/questions/10996/how-to-process-video-with-dft-fft-getting-the-frequency-of-green-color-changes) and on Doom9 too (https://forum.doom9.org/showthread.php?t=165409). Get a look to FFT, IMHO it's the best (mathematical) way.

as I said, I need mask not filtering :)

[real.finder]

I am interested in this question, if this comes to a happy conclusion it could be applied in other situations such as deflickering, its classical methods can cause ghosting. Precisely I had thought about masking luts for this but I do not have the knowledge either, Didée was a genius in this.

maybe mt_luts (2D) can do it with some "sin", "cos" or "tan" operators, but I don't have the knowledge for that, my previous lut is wrong too, since it didn't subtract them as vectors, and maybe even mt_luts not enough for this and it need to made new dll plugins for this

[WolframRhodium]

Try
mt_convolution(horizontal="-1 1 0", vertical="0 1 0")

[poisondeathray]

It's interesting stuff. But I suspect in real world the "perfect mask" will be limited by crazy compression artifacts on top. But keep on testing

[tormento]

as I said, I need mask not filtering :)
It's what I am telling you. AFAIK the only way to get a real mask from "moire" patterns is FFT, directly or subtracting. You probably think I am talking about FFTFiltering plugin but I am not....

[real.finder]

It's interesting stuff. But I suspect in real world the "perfect mask" will be limited by crazy compression artifacts on top. But keep on testing

indeed, but at less it will be the right mask for the problem, and I think it will be between 70%-90% accurate in real world

[real.finder]

1st step is convert UV to IQ

U = utoy8
V = vtoy8

I = mt_lutxy(u,v, "y 0.83867057 * x 0.54463904 * -")
Q = mt_lutxy(u,v, "y 0.54463904 * x 0.83867057 * +")

edit: not realy correct, this one below is correct
I = mt_lutxy(u,v, "y 128 - 0.83867057 * x 128 - 0.54463904 * - 128 +")
Q = mt_lutxy(u,v, "y 128 - 0.54463904 * x 128 - 0.83867057 * + 128 +")

edit: or SwapUV().Tweak(hue=-33) (http://avisynth.nl/index.php/Tweak#YUV_to_YIQ_conversion)

now, the left is vectors subtract of I and Q... and I think it don't need IQ convert for PAL since it's UV

the luma difference is same in both, it can be done in mt_luts I think

[real.finder]

what about this? :)

# scombmask by RF
# v1.11
function scombmask(clip c, float "ythr", float "ythr2", float "cthr", bool "ntsc")
{
sisphbd = AvsPlusVersionNumber > 2294
sisavs26 = !(VersionNumber() < 2.6)
ythr=Default(ythr,2)
ythr2=Default(ythr2,ythr*5)
cthr=Default(cthr,10)
ymaskb=ythr!=0
cmaskb=cthr!=0
c

Assert(IsPlanar() && !IsRGB(), "scombmask: Planar YUV input only")

fullchr = sisphbd ? is444() : sisavs26 ? isyv24() : false
chr420 = sisphbd ? is420() : isyv12()
chr422 = sisphbd ? is422() : sisavs26 ? isyv16() : false
nochr = sisphbd ? isy() : sisavs26 ? isy8() : false

Assert(!nochr, "scombmask: not work with Greyscale video")

ntsc = default (ntsc, !((framerate()==50 || framerate()==25) && Height()!=480))
MinBlur(uv=ymaskb ? 3 : 0,y=cmaskb ? 3 : 0)
iqchr = ymaskb && ntsc ? SwapUV().Tweak(hue=-33) : last
Y = sisphbd ? ConvertToY() : sisavs26 ? ConvertToY8() : last
I = sisphbd ? iqchr.ExtractU() : sisavs26 ? iqchr.utoy8() : iqchr.utoy()
Q = sisphbd ? iqchr.ExtractV() : sisavs26 ? iqchr.vtoy8() : iqchr.vtoy()
Chr = ymaskb ? sisavs26 ? mt_lutxy(I,Q, "x 0.999999084248 * y -0.001353330381 * -", use_expr=1) : mt_lutxy(I,Q, "x 0.999999084248 * y -0.001353330381 * -") : nop()
ym = ymaskb ? Chr.sypixsub(2,2).mt_binarize(ythr) : nop()
ym = ymaskb ? mt_logic(sisavs26 ? chr.mt_lut("x range_half - abs 2 *", use_expr=2) : chr.mt_lut("x 128 - abs 2 *"),ym,"min").mt_binarize(ythr2) : nop()
ym = ymaskb ? ym.Spline36Resize(width(),Height(),chr422 || chr420 ? 0.5 : fullchr ? undefined : 1.5) : nop()
Chrm = cmaskb ? Y.sypixsub(2).mt_binarize(cthr).Spline36Resize(I.width(),Q.Height(),chr422 || chr420 ? -0.25 : fullchr ? undefined : 0.375) : nop()
ytouv(cmaskb ? Chrm : I,cmaskb ? Chrm : Q,ymaskb ? ym : Y)
ymaskb ? mt_expand(128,mode="horizontal",u=cmaskb ? 2 : 1,v=cmaskb ? 2 : 1).mt_expand(128,mode="horizontal",u=cmaskb ? 2 : 1,v=cmaskb ? 2 : 1) : last
}

edit: scombmask now is part of AdvancedDenoising.avsi (https://github.com/realfinder/AVS-Stuff/blob/master/avs%202.5%20and%20up/AdvancedDenoising.avsi), so any update will be there

[real.finder]

here some test code

#after ivtc/deint
a=last
mt_merge(last,Blur(1.5).Sharpen(0.9),scombmask(),u=3,v=3)
mt_merge(last,Blur(1.5).Sharpen(0.9),scombmask(),u=3,v=3)
mt_merge(last,Blur(1.5).Sharpen(0.9),scombmask(),u=3,v=3)
mt_merge(last,Blur(1.5).Sharpen(0.9),scombmask(),u=3,v=3)
mt_merge(last,Blur(1.5).Sharpen(0.9),scombmask(),u=3,v=3)
FineDehalo_contrasharp(last, a,1)

it kinda work even for PAL dots, don't know if I did it right or not!

[real.finder]

I find these
https://www.shadertoy.com/view/ldf3Rf
https://github.com/joncampbell123/composite-video-simulator

seems what I ask in OP not simple as I think! anyway it will be nice if one of those get avs port with option to output only mask

[Nomolu]

Pardon me for reviving an old thread, but I've ran into a problem when I try to use the latest scombmask function (v1.17) in AdvancedDenoising.avsi: When I specify any values > 0 for the prefilter parameter, it says "There is no function named 'DDSharp'".

I've searched the internet and I can't find any AviSynth functions called DDSharp. Can someone please post it or point me to it?

Thanks!

[StainlessS]

Suggest check out Real.Finder stuff [in his sig], try first Zs_RF_Shared.avsi.

[real.finder]

Pardon me for reviving an old thread, but I've ran into a problem when I try to use the latest scombmask function (v1.17) in AdvancedDenoising.avsi: When I specify any values > 0 for the prefilter parameter, it says "There is no function named 'DDSharp'".

I've searched the internet and I can't find any AviSynth functions called DDSharp. Can someone please post it or point me to it?

Thanks!

https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.5%20and%20up/DDSharp.avsi

[Nomolu]

https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.5%20and%20up/DDSharp.avsi

Thanks a million!