LimitedTemporalSoften - mt_lutxy problem

[anton_foy]

Hi, I cannot get this script to do anything:
# >> LimitedTemporalSoften <<
# A function by Didée
#
# Soften bigger noise by TemporalSoften(),
# while still minimize ghosting.
#
#
# Full de-noising is only achieved with weak noise, but it IHMO does a pretty
# good job in "calming down" any sort of noise, before handing the clip to any final denoiser(s).
#
# Stronger pre-blurring. The pre-blurring is only a rough spatial cleaner to get a
# better measurement from the full, unrestricted temporal blur. For strong noise,
# replace that "blur(preblur)" with something like
# "bicubicresize([33%~~66%],.33,.33).bicubicresize([100%],1.0,0.0)".
# Finding the "sweetspot of preblurring" is the main trick.
#
#
# blur() -> removegrain(11) is no problem.
# fade does a funky kind of soft thresholding.
# It's value is not intuitive, and even is inverted (smaller 'fade' -> stronger filtering).
#

function LimitedTemporalSoften(clip clp, int "frames", int "limit",
\ float "preblur", float "fade", bool "color")
{
frames = default(frames, 1)
limit = default(limit, 2)
preblur = default(preblur,1.0)
fade = default(fade, 4.0)
color = default(color, true)
limstr = string(limit)
fadestr = string(fade)
uv = color ? 3 : 2

spat = (preblur==0.0) ? clp : clp.blur(preblur)
temp_spat = spat.TemporalSoften(frames, 255, 255, 255, 2)
temp = clp .TemporalSoften(frames, 48, 48, 32, 2)

Diff_SpatTemp = mt_lutxy(spat, temp_spat, Yexpr="255 "+limstr+" * x y - abs "+fadestr+" * /"
\ ,Uexpr="255 "+limstr+" * x y - abs "+fadestr+" * /"
\ ,Vexpr="255 "+limstr+" * x y - abs "+fadestr+" * /",U=uv,V=uv)

out = mt_merge(clp, temp, Diff_SpatTemp, Y=3, U=uv, V=uv)

return(out )
}

Whenever I tried out these older scripts it seems mt_lutxy does not work in this way in the newer masktools?

Absolutely nothing happens when running this filter no matter how I tweak the settings.

Edit: yv12lutxy was used from the beginning.

[real.finder]

change return(out ) to return(Diff_SpatTemp ) and you will note that mt_lutxy is not has a problem

[anton_foy]

change return(out ) to return(Diff_SpatTemp ) and you will note that mt_lutxy is not has a problem

Yes thank you. I can see no difference between return(out) and return(Diff_SpatTemp) they do nothing visible at all either of the lines.

[real.finder]

Yes thank you. I can see no difference between return(out) and return(Diff_SpatTemp) they do nothing visible at all either of the lines.

maybe it's some asm code bug in masktools, try put SetMaxCPU("none") in the first line of the .avs script (the one you use to encode not the function .avsi one)

[anton_foy]

maybe it's some asm code bug in masktools, try put SetMaxCPU("none") in the first line of the .avs script (the one you use to encode not the function .avsi one)

Ok I put SetMaxCPU("none") in the beginning of the script but still no change except it went really slow.

[real.finder]

anyway, its fine for me
https://i.postimg.cc/xCrVsnY7/Screenshot-2022-09-10-211434.png (https://postimg.cc/wtkrMK1c)

[anton_foy]

Could it be because I use in 16bit?

[Reel.Deel]

Could it be because I use in 16bit?

More than likely. If you don't scale the numbers then 0-255 does not do much on 0-65535. Try it on 8-bit and see if it works as intended.

[anton_foy]

More than likely. If you don't scale the numbers then 0-255 does not do much on 0-65535. Try it on 8-bit and see if it works as intended.

Working in 8bits thanks! But I would like it to work for 16bit though. That must certainly be mt_lutxy params I need to change then?

[Reel.Deel]

Working in 8bits thanks! But I would like it to work for 16bit though. That must certainly be mt_lutxy params I need to change then?

Yes, you either can do it yourself or auto scaling in masktools: http://avisynth.nl/index.php/MaskTools2#Expression_syntax_supporting_bit_depth_independent_expressions

[anton_foy]

Yes, you either can do it yourself or auto scaling in masktools: http://avisynth.nl/index.php/MaskTools2#Expression_syntax_supporting_bit_depth_independent_expressions

Great thanks!

[anton_foy]

Nope. Now I tried with Yexpr ="255 scalef" Yexpr"255 scaleb" expr_luma"255 scalef" and scalef/scaleb also on chroma (Uexpr, Vexpr and expr_chroma) and still does nothing in 16bit.

[Reel.Deel]

Nope. Now I tried with Yexpr ="255 scalef" Yexpr"255 scaleb" expr_luma"255 scalef" and scalef/scaleb also on chroma (Uexpr, Vexpr and expr_chroma) and still does nothing in 16bit.

Diff_SpatTemp = mt_lutxy(spat, temp_spat, Yexpr="255 "+limstr+" * x y - abs "+fadestr+" * /"
\ ,Uexpr="255 "+limstr+" * x y - abs "+fadestr+" * /"
\ ,Vexpr="255 "+limstr+" * x y - abs "+fadestr+" * /",U=uv,V=uv, scale_inputs="allf")

Seems to do the trick but there is a difference in the output. Probably because of 8/16 bit processing (Blur and TemporalSoften).

[anton_foy]

Diff_SpatTemp = mt_lutxy(spat, temp_spat, Yexpr="255 "+limstr+" * x y - abs "+fadestr+" * /"
\ ,Uexpr="255 "+limstr+" * x y - abs "+fadestr+" * /"
\ ,Vexpr="255 "+limstr+" * x y - abs "+fadestr+" * /",U=uv,V=uv, scale_inputs="allf")

Seems to do the trick but there is a difference in the output. Probably because of 8/16 bit processing (Blur and TemporalSoften).

Sorry, still nothing for me it still remains totally untouched in 16bits but working in 8bits without a problem (scale_inputs="allf" removed). Although I need it for 16bit...
Frustrating :/

My script:


#convertbits(8)
LimitedTemporalSoften(frames=10,fade=3,limit=3,preblur=1.2)

function LimitedTemporalSoften(clip clp, int "frames", int "limit",
\ float "preblur", float "fade", bool "color")
{
frames = default(frames, 1)
limit = default(limit, 2)
preblur = default(preblur,1.0)
fade = default(fade, 4.0)
color = default(color, true)
limstr = string(limit)
fadestr = string(fade)
uv = color ? 3 : 2

spat = (preblur==0.0) ? clp : clp.blur(preblur)
temp_spat = spat.TemporalSoften(frames, 255, 255, 255, 2)
temp = clp.TemporalSoften(frames, 48, 48, 32, 2)

Diff_SpatTemp = mt_lutxy(spat, temp_spat, Yexpr="255 "+limstr+" * x y - abs "+fadestr+" * /"
\ , Uexpr="255 "+limstr+" * x y - abs "+fadestr+" * /"
\ , Vexpr="255 "+limstr+" * x y - abs "+fadestr+" * /",U=uv,V=uv, scale_inputs="allf")

out = mt_merge(clp, temp, Diff_SpatTemp, Y=3, U=uv, V=uv)

return(out)
}


Edit: also tried with ex_lutxy from Dogway but no change.

[anton_foy]

Got it to work in 16bits:

Diff_SpatTemp = mt_lutxy(spat, temp_spat, expr="255 "+limstr+" * x y - abs "+fadestr+" * /"
,U=3,V=3, scale_inputs="allf")

For some reason I had to change to just expr and u=3,v=3. The three line setup didnt work.