Motion Compensate Limited Sharpen 16bit

[Motenai Yoda]

Here a sharpen I'm worked on some time ago, it's inspired to Didée's prof of concept (http://forum.doom9.org/showthread.php?p=1380002#post1380002),
dependencies:
- Dither tools by cretindesalpes
- MVTools 2.6.x/2.7.x
- MaskTools2 and RgTools by tp7

function MCLS_16(clip c, float "strength", clip "sharp", int "thsad", int "plane",\
float "ss", bool "repair", int "dmode", bool "masked", bool "blur_gaussian")
{

#Motion Compensate Limited Sharpen for a 16bit stacked clip by Motenai Yoda
#first relase - beta 7
#it works (or should) with 16bit stacked clip input/output

strength = Default(strength, 1.0)
thsad = Default(thsad, 400)
plane = Default(plane, 0)
ss = Default(ss, Defined(sharp) ? 1.0 : 1.25)
repair = Default(repair, Defined(sharp) ? false : true)
dmode = Default(dmode, 1)
masked = Default(masked, true)
blur_gaussian = Default(blur_gaussian, false)


str = String(strength)
HD = (c.width > 1280) ? 2 : (c.width > 720) ? 1 : 0
Pel = Select(HD, 2, 1, 1)
Truemotion = Select(HD, True, False, False)
Blksize = Select(HD, 8, 16, 32)
Overlap = Select(HD, 4, 8, 8)

Yp = (plane == 0 || plane == 4) ? 3 : 1
Up = (plane == 1 || plane == 3 || plane == 4) ? 3 : 1
Vp = (plane == 2 || plane == 3 || plane == 4) ? 3 : 1

Chromap = (plane == 0) ? false : true

width2 = Round(c.width*ss)
height2 = Round(c.height*ss/2.0)

wpad = 16-(width2%16)
hpad = 16-(height2%16)

bRadius = Round(Select(HD, 3, 3, 4) / 1.25 * ss)


resized_source = (ss != 1.0) ? c.Dither_resize16(width2+wpad, height2+hpad, y=3, u=Up,\
v=Vp) : c
resized_sharp = Defined(sharp) ? (ss != 1.0) ? sharp.Dither_resize16(width2+wpad, height2+hpad,\
y=Yp, u=Up, v=Vp) : sharp : nop()

pre = resized_source.Dither_removegrain16(3, (Up == 3) ? 3 : -1, (Vp == 3) ? 3 : -1)

sharp0 = Defined(sharp) ? Dither_sub16(resized_sharp, resized_source, wrap=False, y=Yp, u=Up,\
v=Vp, dif=true) : Dither_sub16(resized_source, blur_gaussian ? Dither_box_filter16(resized_source,\
radius=max(bRadius-2, 1), y=Yp, u=Up, v=Vp).Dither_box_filter16(radius=max(bRadius-1,1), y=Yp, u=Up, v=Vp).\
Dither_box_filter16(radius=max(bRadius-1,1), y=Yp, u=Up, v=Vp) : Dither_box_filter16(resized_source,\
radius=bRadius, y=Yp, u=Up, v=Vp), wrap=False, y=Yp, u=Up, v=Vp, dif=true)

source0 = Ditherpost(pre, mode=-1, y=3, u=Up, v=Vp)

mask = (masked) ? source0.mt_edge("min/max", y=3, u=1, v=1).mt_deflate(y=3, u=1, v=1).\
removegrain(12, -1, -1) : nop()
mask = (masked) ? stackvertical(mask, mask) : nop()

sup0 = source0.MSuper(pel=Pel, sharp=2, chroma=Chromap)
sharp0m = sharp0.Dither_get_msb()
sup1 = sharp0m.MSuper(pel=Pel, sharp=2, chroma=Chromap)
sharp0l = sharp0.Dither_get_lsb()
sup2 = sharp0l.MSuper(pel=Pel, sharp=2, chroma=Chromap)

bv1 = sup0.MAnalyse(isb=true, delta=1, chroma=Chromap, blksize=Blksize, truemotion=Truemotion, overlap=Overlap)
fv1 = sup0.MAnalyse(isb=false, delta=1, chroma=Chromap, blksize=Blksize, truemotion=Truemotion, overlap=Overlap)

nullclip = source0.mt_lut(y= (Yp==3) ? -0 : 1, u= (Up==3) ? -0 : 1, v= (Vp==3) ? -0 : 1)

diff1 = sharp0m.MDegrain1(sup1, bv1, fv1, plane=plane, thsad=thsad, lsb=true)
diff2 = sharp0l.MDegrain1(sup2, bv1, fv1, plane=plane, thsad=thsad, lsb=false)
diff2_16 = stackvertical(nullclip, diff2)

diff3 = Dither_add16(diff1, diff2_16, wrap=False, y=Yp, u=Up, v=Vp, dif=false)

sharped_back = Dither_add16(resized_source, diff3, wrap=False, y=Yp, u=Up, v=Vp, dif=true)

repaired = (repair) ? Dither_repair16(sharped_back, resized_source, (Yp == 3) ? 1 : -1 ,\
(Up == 3) ? 1 : -1, (Vp == 3) ? 1 : -1).Dither_merge16(sharped_back,sharped_back.mt_lut(y=-76,\
u=1,v=1),luma=true, y=Yp, u=Up, v=Vp): sharped_back

bias = (dmode == 1) ? resized_source.Dither_lut16("x 4096. - 56064. / 0 1 clip pi * sin 0.5 ^ 65536. * "+\
str+" *", y=3, u=1, v=1) : nop()
bias = (masked) ? (dmode == 1) ? bias.mt_logic(mask, "min", y=3, u=1, v=1) : mask : bias

repaired = (dmode == 1 || masked) ? Dither_merge16(resized_source, repaired, bias, y=Yp, u=Up, v=Vp, luma=true) :\
repaired

resized_back = (ss != 1.0) ? Dither_resize16(repaired, c.width(), c.height()/2, y=Yp, u=Up, v=Vp) : repaired

return ytouv((Up == 3) ? resized_back.utoy() : c.utoy(), (Vp == 3) ? resized_back.vtoy() : c.vtoy(),\
(Yp == 3) ? resized_back : c)
}

edit: removed "keep" and do the blur/sharp on resized_source instead of pre
edit2: fixed some flaw about chroma processing when Y plane is not processed (a bit slower than before if plane = 1/2/3)


note 22/06/2016 : I used mt_logic "min" to merge edgemask and lumamask, as edgemask is basically 0/255, when it's 0 it's the minimum, when it's 255 isn't the minimum, moreover with a lumamask value X = x/256 , if multiplied togheter X*0 = 0, X*(255/255) = X*1 = X
so using a lut to multiply them should give roughtly same result.

[johnmeyer]

Do you have any before/after results to share?

[Motenai Yoda]

as it's mainly for animation stuff, it still lacks something like "secure" of lsfmod and is prone to loss some 1px size details.

with masked=false
http://4.t.imgbox.com/9XC7XRHI.jpg (http://imgbox.com/9XC7XRHI) http://8.t.imgbox.com/4ppyFGsY.jpg (http://imgbox.com/4ppyFGsY)

[johnmeyer]

Looks pretty good. Nice, subtle sharpening.

[Motenai Yoda]

screen for animation stuff
original - masked=false - masked=true
http://2.t.imgbox.com/n7pSJUnE.jpg (http://imgbox.com/n7pSJUnE) http://7.t.imgbox.com/7kBqv5NF.jpg (http://imgbox.com/7kBqv5NF) http://3.t.imgbox.com/DRQol8BU.jpg (http://imgbox.com/DRQol8BU)

Anyway it can be used even feeding a yet sharped clip, as well it is 16 bit. Notice only with an external "sharp" clip supersampling and repair are booth disabled.

ie with a min/max sharp

function MinMaxSharp(clip c, int "radius"){

radius = Default(radius, 1)

max = (radius == 1) ? c.mt_expand(u=1, v=1) : c.mt_expand(u=1, v=1).mt_expand(u=1, v=1)
min = (radius == 1) ? c.mt_inpand(u=1, v=1) : c.mt_inpand(u=1, v=1).mt_inpand(u=1, v=1)
avg = mt_average(min, max, u=1, v=1)
diff = c.mt_makediff(avg, u=1, v=1)
return c.mt_adddiff(diff, u=2, v=2)
}

function MinMaxSharp_16(clip c, int "radius"){

radius = Default(radius, 1)
quant= int(pow(((2*radius)+1),2) - 1)

min = c.Dither_median16(rx=radius, ry=radius, rt=0,ql=0,qh=0, y=3, u=1, v=1)
max = c.Dither_median16(rx=radius, ry=radius, rt=0,ql=quant,qh=quant, y=3, u=1, v=1)

avg = Yoda_Average16(max, min, y=3, u=1, v=1)
diff = c.Dither_sub16(avg, wrap=False, y=3, u=1, v=1, dif=True)
return c.Dither_add16(diff, false, u=2, v=2, dif=true)
}

MCLS_16(last, sharp=MinMaxSharp_16(radius=2),masked=false,repair=true,ss=1.25)
http://9.t.imgbox.com/ttpUDym5.jpg (http://imgbox.com/ttpUDym5)

or with a limited supersampled one
function FLSF_16(clip c, float "strenght", int "radius", bool "edgemode",\
float "ss", bool "fast", int "dmode")
{
#v7
strenght = Default(strenght, 1.0)
radius = Default(radius, 1)
edgemode = Default(edgemode, true)
ss = Default(ss, 1.25)
fast = Default(fast, true)
dmode = Default(dmode, 1)

strenght = (fast==true && radius == 1) ? strenght * 0.8 : strenght
nr = !fast

width2 = Round(c.width*ss)
height2 = Round(c.height*ss/2)

wpad = 16-(width2%16)
hpad = 16-(height2%16)

c.converttoy8()
(ss != 1.0) ? Dither_resize16nr(last, width2+wpad, height2+hpad, noring=nr) : last
quant= int(pow(((2*radius)+1),2) - 1)

dark = Dither_median16(rx=radius, ry=radius, rt=0,ql=0,qh=0, y=3, u=1, v=1)
bright = Dither_median16(rx=radius, ry=radius, rt=0,ql=quant,qh=quant, y=3, u=1, v=1)

#avg = (fast==true && radius==1) ? Dither_Box_Filter16_Fixed(3,3,1,1) :\
Yoda_Average16(bright, dark, y=3, u=1, v=1)
avg = (fast==true && radius==1) ? Yoda_GaussianRG16(3,false) : Yoda_Average16(bright,\
dark, y=3, u=1, v=1)

diff = Dither_sub16(avg, wrap=False, y=3, u=1, v=1, dif=True)
diff2 = (strenght == 1.0) ? diff : Dither_lut16(diff,yexpr="x 128 256 * - "+\
string(strenght)+" * 128 256 * + ", u=1, v=1)
sharp = Dither_add16(diff2, false, u=1, v=1, dif=true)

darkdiff = Dither_sub16(dark, sharp, wrap=False, y=3, u=1, v=1)
brightdiff = Dither_sub16(sharp, bright, wrap=False, y=3, u=1, v=1)

darkclamp = Dither_add16(sharp, darkdiff, wrap=False, y=3, u=1, v=1)
brightclamp = Dither_sub16(darkclamp, brightdiff, wrap=False, y=3, u=1, v=1)
clamp = (radius==1) ? Dither_Repair16(sharp, last, 1, -1) : brightclamp

bias = (dmode == 1) ? Dither_lut16("x 16 256. * - 219. 256. * / pi * sin 256. 256. * *",\
u=1, v=1) : last
clamp = (dmode == 1) ? Dither_merge16(last, clamp, bias, u=1, v=1) : clamp

resized = (ss != 1.0) ? Dither_resize16nr(clamp, c.width, c.height/2, noring=nr).\
converttoyv12() : clamp.converttoyv12()


mask = c.Dither_Get_Msb().mt_edge("min/max", u=1, v=1)
mask = stackvertical(mask,mask)
return (edgemode) ? mt_merge(c, resized, mask, u=2, v=2) : mergechroma(resized, c)
}

MCLS_16(sharp=FLSF_16(radius=2,dmode=0),masked=false)
(note FLSF_16() is very prone to loss 1-2 (radius) px detail and flat edges)
http://5.t.imgbox.com/hBCn7e2s.jpg (http://imgbox.com/hBCn7e2s)

ps:
Yoda_Average16() can be replaced by Dither_merge16(sharped_back,sharped_back.mt_lut(y=-128,\
u=1,v=1),luma=true, y=Yp, u=Up, v=Vp)
or
Dither_add16(a,b.Dither_sub16(a, diff=true, y=Yp, u=Up, v=Vp).\
Dither_lut16("x 2. /" , y=Yp, u=Up, v=Vp), diff=true, y=Yp, u=Up, v=Vp)

Yoda_GaussianRG16(3,false)
by
Dither_Removegrain16(20,0).Dither_Removegrain16(20,0).Dither_Removegrain16(20,0)

[Motenai Yoda]

new 0.8 beta
- change boolean "masked" -> integer "edgemode", 0 = no mask, 1 = old "masked = true" (mainly to be used with animation), 2 = new mask (for RL content)
- change use a mt_lutxy to merge mask and bias clip instead of mt_logic
- add new mask parameter to use a custom 8bit mask (only luma is used), as is used in mt stuff, only mt supported format and the same as input clip

http://pastebin.com/MxzbPRzz

[FranceBB]

Are you planning to support 16bit interleave as well in the future?

Oh, by the way, you wrote "unction" instead of "Function" in the pastebin ;)

[Motenai Yoda]

beta 9
- fixed how "strength" is applied, now can be used to increase effect if > 1, or at least have sense
- change "edgemode" now can be negative, -1 or -2, to use an inverted relative mask
- add a "secure" boolean (default: False) to use a lsfmod like secure limiting, but allowing a change of +/- 512 ~ 2 point in 8bit, not 1
- add a "stacked" boolean (default: True) to support 16bit interleaved clip (but custom mask still has to be 8bit for now), FTurn mandatory on not + avs

Also I found the 1px size detail loss is in between repair and supersampling, using edgemode -2, repair false and ss 1, it give a more crispy and feisty2less image


http://pastebin.com/Zg4qAqw7


@FranceBB to compensate this time first I paste it with the old title, then without avisynth highlight, and finally posted the wrong one

btw I'd like to change a lot when new avs+ and 16bit aware mvtools2 will be reality and to put a bunch of Assert() too

ps should be better mcstuffin' the mask too?

[FranceBB]

Got it.
Thank you very much indeed anyway.
As to the current version, I'll test it this evening since I'm at work right now. Anyway, when the new mvtools2 will be released, I don't know how it's gonna handle calculations. In case MVTools2 will be aware of 16 bit stack only (input-output), it would be fine to get it working with your script and leave the interleave option for those who want to use an 8bit mask and 16 calculations. ;) Anyway, I honestly don't know how many benefits will bring a 16bit mask, even considering speed vs quality.

[pinterf]

16 bit mvtools2 will work on 10-16 bits natively with avs+, now I have MDegrain1-5 and MDegrainN working on my pc in 10+ bits

[real.finder]

16 bit mvtools2 will work on 10-16 bits natively with avs+, now I have MDegrain1-5 and MDegrainN working on my pc in 10+ bits

cool, and in normal avs? you will add lsb_in like Dfttest?

[pinterf]

I don't think so, I'm not planning to put yet another hack into this poor plugin

[Motenai Yoda]

new experimental try which trade on avs+ 16bit yv12 format (yv16? yv24? yuv444? Not tested)
this version doesn't support stacked/doublewidth formats
http://pastebin.com/AmN6iT5X

[Motenai Yoda]

Another testing one, now works with 8-16bit y/yv12/yv16/yv24 input, mask and sharp clips (but the same colorformat, unless input is y or plane=0)

http://pastebin.com/SZuLFRP5

[tormento]

Another testing one, now works with 8-16bit y/yv12/yv16/yv24 input, mask and sharp clips (but the same colorformat, unless input is y or plane=0)

http://pastebin.com/SZuLFRP5

Did you try to use ConvertBits(x, dither=n [, dither_bits=y]) from latest AVS+ instead of Dither? Results?

[Motenai Yoda]

Did you try to use ConvertBits(x, dither=n [, dither_bits=y]) from latest AVS+ instead of Dither? Results?
I left only Dither_box_filter16() from Dither,

- dither is (-1 none/truncate or round, 0 ordered, 1 floyd-steinberg)

- dither_bits define at what bitdepth it will dither, unlike Ditherpost where indicates dithering bits, ie: convertbits(bits=8,dither=0,dither_bits=2) return you a 2 bit dithered into 8 bit format, also it has to be min input bit -8 if dither=0

[ale_x]

Hey!

I used your script, but is not working at all ><



https://image.ibb.co/giqGSG/Untitled.png

[StainlessS]

Maybe you would like to tell Yoda which script you are talking about.

The 'Mask' arg to said function is a clip, in older version of function 'Masked' was a bool.

EDIT: I did not find an SMg.avs EDIT: Below Google search found nothing


"SMg.avs" site:forum.doom9.org


Same for this

"SZuLFRP5.avsi" site:forum.doom9.org


EDIT: OK, that 2nd one is a pastebin file above in post #14, where Mask is a clip not a bool, so looks like your smg.avs script is in error.
(or should be used with the older version of the script).

[ale_x]

ok I used this ver of script:

https://pastebin.com/SZuLFRP5


this is what I got, an error !

https://forum.doom9.org/attachment.php?attachmentid=16182&d=1515130676

[Motenai Yoda]

Hi Ale_x try replacing "mask=false" with "edgemode=0"

also on a newer masktools2 version you have to replace all #F with @F

[ale_x]

Hi Yoda!


I did as you told me but still the error exist!

LWLibavVideoSource("00008.m2ts")
converttoyv12()
MCLS_16_plus(last,edgemode=0 )

installed:
-Masktools2 2.2.10
-MvTools2 2.7.24

>>

after I installed this filter RgTools 0.96, I got this error now

( by the way, the old ver of script works perfectly "MCLS_16")

https://forum.doom9.org/attachment.php?attachmentid=16183&stc=1&d=1515137887

[StainlessS]

We all wait with baited breath to see what few words your attachment says :)

Post YOUR exact scripts and the error message (as text).

[ale_x]

It works now =)
thank you StainlessS & special thanks to Yoda for this good script

[ale_x]

I hope from Yoda to do a good wiki for this script


Regards.