SpliceResize

[javlak]

What is it?

It's a function that tries to splice two different resizing algorithms together in order to maintain the advantages of both (i.e. sharpness and no ringing artefacts).

This is achieved by resizing twice, once using a "soft" algorithm, such as Hermite, and a second time using a sharper one, such as Spline36. The function compares the two pictures on a per-pixel basis, and if the difference between two pixels exceeds a certain threshold, it assumes that this must be some sort of artefact and it either averages the pixel values or it selects the pixel of the softer picture.

What isn't it?
It's not a deringing function. Its purpose is not to dering (although it has some built-in features to do that, but they are light), but rather to try and avoid ringing altogether as much as possible by manipulating the resizing algorithm.

How do I use it?

Here are all the customization parameters:

Width: Desired Width
Height: Desired Height
Threshold: If the difference between two pixels is lower than the threshold, then the pixel produced by the sharper algorithm will be used, otherwise either the pixel produxed by the softer algorithm will be used, or their average. Even though the final results of the softer and the sharper algorithm are apparent on a large scale, on a per-pixel basis, the difference between them isn't really too big, so bear in mind that a threshold can be as low as 3, depending on your tastes. The lower the threshold, the more the sharper algorithm gets used but artefacts might get missed, the higher the threshold, the less the artefacts, but the softer the result. Default:5
Mode: Mode 1 selects pixels. If the difference of the pixels is within the threshold, it selects the sharper one. Otherwise, it selects the softer one. Mode 2 averages pixels. If the difference of the pixels is within the threshold, it selects the sharper one. Otherwise, it averages them. The first mode has the advantage that, if there are artefacts caused by the sharper algorithm, they will be removed; the disadvantage is that this might soften the picture, especially if the threshold is high. The second mode has the advantage that it will not soften the picture as much as the first mode, but if there are artefacts caused by the resizing, then these will not be completely extinguished, but rather "softened out", due to the averaging out of the pixels. Default: 1 (Select)
Kernel1: This is the kernel to use for the soft resizing: Kernel1=0 is the Bilinear algorithm, which doesn't produce artefacts, but can produce some bluriness. Kernel1=1 is Hermite, which is an algorithm which is well-known to produce as little blurriness and artefacts as possible. Kernel1=2 is not a resizing kernel per se, as it is the average of the bicubic one, which is a mostly artefact-free and bluriness-free algorithm, and the 2-tap Blackman one, which does produce some artefacts, but not too many. Kernel1=3 is the 2-tap Blackman algorithm, which does produce some artefacts and as a matter of a fact is not what you would call a soft algorithm, but it produces less artefacts than all the algorithms that are used by the function as sharp ones. Default: 1 (Hermite)
Kernel2: This is the kernel for the sharp resizing: Kernel2=0 Is the Blackman algorithm with 4 lobes; kernel2=1, Spline16; kernel2=2 Spline36; kernel2=3 Spline64. In order of sharpening Spline64>Spline36>Spline16>Blackman and in order of artefacts Blackman>Spline16>Spline36>Spline64. Default:2 (Spline36)
Preprocess: If there's some grain or light artefacts in your picture already and you want to clean them before resizing to avoid any artefact accentuation or new artefacts, you can use this. To be honest, if your source isn't clean already you're probably better off using a dedicated noise and/or deringer such as MC_Spuds and YAHR before resizing. Default: False
Postprocess: If you feel that your threshold is too low and you might have artefacts in the final picture, you can use this to clean up the final result. To be honest, you're better off using a dedicated noise and/or deringer such as MC_Spuds and YAHR. Default: False
Sharper: This is used to sharpen up the softer picture. If the original picture is artefact free, and depending on your threshold values and kernel1, this can increase the visual quality perception of the resulting image.
Masktype: The type of mask used to detect edges. The default (Roberts) will definitely miss a few, but is faster than the alternative and noise-tolerant; canny is very accurate, but slower and may also detect noise as an edge. Default: 1 (Roberts)
Sensitivity: This is the sensitivity of the canny mask. Its value is irrelevant if you are using Roberts. Its value can be anywhere from 0 (which corresponds to sigma=1.2 and which is set to be the least sensitive setting and thus might miss a few edges, but will not catch noise) to 10 (which corresponds to sigma=0.6 and which is set to be the most sensitive setting and will detect the most edges, but might catch noise as well). Default: 0 (σ=1.2, l=4, h=4).


function canny(clip v, int "sensitivity")
{
sensitivity = 1.2 - (sensitivity/16.67)
v=v.RemoveGrain(mode=1).RemoveGrain(mode=5)
v=v.Grayscale()
v=v.RemoveGrain(mode=11).RemoveGrain(mode=1).RemoveGrain(mode=5)
v=v.tbilateral(7,3,1.3,1,5,5,1,0,true,false,kerns=4,kerni=3,restype=0)
v=v.tcanny(sigma=sensitivity,t_l=3.9,t_h=3.9)
v=v.RemoveGrain(mode=1).RemoveGrain(mode=5)
return v
}

function SpliceResize(clip v, int "w", int "h", int "threshold", int "mode", int "kernel1", int "kernel2", bool "preprocess", bool "postprocess", bool "sharper", int "masktype", int "sensitivity")
{
threshold = default (threshold, 5)
w=default(w, width(v))
h=default(h, height(v))
mode=default(mode, 1)
kernel1=default(kernel1, 1)
kernel2=default(kernel2, 2)
postprocess=default(postprocess, false)
preprocess=default(preprocess, false)
sharper=default(sharper, false)
masktype=default(masktype, 1)
sensitivity=default(sensitivity, 0)
sensitivity = (sensitivity>0 && sensitivity<11) ? sensitivity : 0
threshold = (threshold>0 && threshold<255) ? threshold : 5
v = (preprocess==true) ? v.RemoveGrain(mode=1).RemoveGrain(mode=5) : v
baseline = (kernel1==1) ? BicubicResize(v, w, h, b=0.0, c=0.0) : \
(kernel1==0) ? BilinearResize(v, w, h) : \
(kernel1==3) ? BlackmanResize(v, w, h, taps=2) : \
(kernel1==2) ? mt_average(BicubicResize(v, w, h, b=0.0, c=0.0),BlackmanResize(v, w, h, taps=2)) : BicubicResize(v, w, h, b=0.0, c=0.0)
sharp = (kernel2==2) ? Spline36Resize(v, w, h) : \
(kernel2==3) ? Spline64Resize(v, w, h) : \
(kernel2==0) ? BlackmanResize(v, w, h, taps=4) : \
(kernel2==1) ? Spline16Resize(v, w, h) : Spline36Resize(v, w, h)
func = (mode==1) ? ("x y - abs "+string(threshold)+" > y x ?") : \
(mode==2) ? ("x y - abs "+string(threshold)+" > y x y + 2 / ?") : ("x y - abs "+string(threshold)+" > y x ?")
baseline = (sharper==true) ? baseline.unsharp(strength=0.2) : baseline
v=mt_lutxy(sharp,baseline,func)
v = (postprocess==true) ? v.RemoveGrain(mode=1).RemoveGrain(mode=5) : v
vmask = (masktype==2) ? canny(baseline, sensitivity).mt_expand(mode="both") : baseline.mt_edge(mode = "roberts").mt_expand(mode="both").RemoveGrain(mode=11).RemoveGrain(mode=1).RemoveGrain(mode=5)
v = mt_merge(v, sharp, vmask)
return v
}


Dependencies:
For SpliceResize:
RemoveGrain
VariableBlur (any version that includes unsharp)
Masktools 2

For canny:
tcanny
MPEG2Dec3DG.dll OR DGDecode.dll

[Mini-Me]

This is a really cool idea! :)

I'll definitely need to try this one out when I get the chance...

[TheProfileth]

Looks interesting I will give this a go, wonder how it compares quality and speed wise to nnedi3, most likely it is faster but lower quality but the difference is how much lower.
edit: hmm this is more fairly compared against a resizer like lanczos4 but even then it is (atleast in my testing) not a fair comparison
when upscaling spliceresize tends to blur/split the edges of things, even at the sharpest settings (with default threshold and mode 1)
still a good idea though, I think I will do a more testing with this.

[javlak]

Maybe kernel1=2 can help with the issue, TheProfileth?

[TheProfileth]

No I mean I was using kernel1=3 and kernel2=3 still had the issue on the edges, it only stopped once I switched to mode=2 , it was quiet odd to be honest I had no idea why the image was being so blurred on the edges.

[Sapo84]

Threshold: If the difference between two pixels is lower than the threshold, then the pixel produced by the sharper algorithm will be used, otherwise either the pixel produxed by the softer algorithm will be used, or their average. Even though the final results of the softer and the sharper algorithm are apparent on a large scale, on a per-pixel basis, the difference between them isn't really too big, so bear in mind that a threshold can be as low as 3, depending on your tastes. The lower the threshold, the more the sharper algorithm gets used but artefacts might get missed, the higher the threshold, the less the artefacts, but the softer the result. Default:5

That bugged me last time I looked at the topic, given the explanation, why would a lower threshold be sharper?
Then I carefully read the function, and it is doing the complete opposite, when the difference is higher than the treshold the sharper output is used, and when the difference is lower it's the softer one that is used.

Anyway I'm not too sure if this function will ever work, if you limit the difference between a sharp resize and a soft one you will probably soften out artefacts, but you will also soften the sharpness.
Ideally you would want a "artefacts mask" to apply the filter only where is needed.

[Didée]

Methinks this already has been mentioned earlier.;) - The "naked" pixel-difference tells very little about which method is better suited for that pixel. The crucial point is the neighborhood, i.e. the signal shape at the local position. Whether the difference is bigger-than-four or smaller-than-seven is not really relevant.

Also, there should be some aliasing-problems occuring. There will always be pixelclusters where in-threshold pixels are located directly beneath out-of-threshold pixels.

Really, I'd rather look into edge-masking for such kind of resizer-mixing. It'll be a little slower, but it's also more reasonable.

[javlak]

TheProfileth and Sapo84, I have now fixed an error I had made; it should have been v=mt_lutxy(sharp,baseline,func), and for some stupid reason I had typed v=mt_lutxy(baseline,sharp,func). So it was the complete opposite of what I was going for.
And Didee's idea of an edge-masking solution sounds interesting. Maybe I can fiddle with something like that if/when I find the time.

[javlak]

Well, I added an edge mask, but not as the core of the program, but rather as a final step where the sharper kernel works no matter what. This should make a difference.

[javlak]

Added the choice between the Roberts and the Canny mask. This should greatly improve accuracy.

[TheProfileth]

Will give this a try tomorrow, might even post comparisons :cool:

[javlak]

Thanks, I didn't have much time to post comparisons/screenies!

[SilaSurfer]

Doing some testing!:p

Edit1:

What is dfttest used for in canny mask, noise removal on the edges?

[javlak]

It's just there to remove noise. Canny edge detection is very, very, VERY fussy about noise (or you can of course just change the sigma, l and h values but then you'd lose accuracy). Just do "return vmask" instead of "return v" in function SpliceResize to see the difference between Roberts and Canny. It's HUGE, but you'll see that denoising is a sine qua non in this case!

[TheProfileth]

It's just there to remove noise. Canny edge detection is very, very, VERY fussy about noise (or you can of course just change the sigma, l and h values but then you'd lose accuracy). Just do "return vmask" instead of "return v" in function SpliceResize to see the difference between Roberts and Canny. It's HUGE, but you'll see that denoising is a sine qua non in this case!
So this is what is being called for canny?
v=v.dfttest(sigma=0.5)
v=v.RemoveGrain(mode=1).RemoveGrain(mode=5)
v=v.Grayscale()
v=v.blur(1.58).blur(1.58).blur(1.58).blur(1.58)
v=v.RemoveGrain(mode=11).RemoveGrain(mode=1).RemoveGrain(mode=5)
v=v.dfttest(sigma=0.5)
v=v.tcanny(sigma=sensitivity,t_l=4,t_h=4).RemoveGrain(mode=1).RemoveGrain(mode=5)
return v

Seems sort of redundant and not to mention slow to call blur 3 times and dfftest twice, why not just use minblur
# Nifty Gauss/Median combination
function MinBlur(clip clp, int r, int "uv")
{
uv = default(uv,3)
uv2 = (uv==2) ? 1 : uv
rg4 = (uv==3) ? 4 : -1
rg11 = (uv==3) ? 11 : -1
rg20 = (uv==3) ? 20 : -1
medf = (uv==3) ? 1 : -200

RG11D = (r==1) ? mt_makediff(clp,clp.removegrain(11,rg11),U=uv2,V=uv2)
\ : (r==2) ? mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20),U=uv2,V=uv2)
\ : mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20).removegrain(20,rg20),U=uv2,V=uv2)
RG4D = (r==1) ? mt_makediff(clp,clp.removegrain(4,rg4),U=uv2,V=uv2)
\ : (r==2) ? mt_makediff(clp,clp.medianblur(2,2*medf,2*medf),U=uv2,V=uv2)
\ : mt_makediff(clp,clp.medianblur(3,3*medf,3*medf),U=uv2,V=uv2)
DD = mt_lutxy(RG11D,RG4D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
clp.mt_makediff(DD,U=uv,V=uv)
return(last)
}

[javlak]

Yeah, maybe MinBlur has a similar effect to what I was going for, hadn't occured to me at the moment. I was just toying around with removegrain, dfttest and fft3dfilter on somewhat noisy sources, while trying to keep the Canny's σ, l and h values decent enough to keep as much detail as possible. Rinse, repeat until happy with the result. I'll test it out with it when I find the time (that should be tomorrow or the day after). To be honest, if I could, I'd just MCSharpen before calling tcanny, but I guess that would make a lot of people unhappy!

[javlak]

Replaced some slow operations in the Canny edge detection function (namely 2 calls to dfttest and 2 calls to blur) with a strategically placed BlindPP. Thanks The Profileth for giving me the idea to look around for more solutions to remove noise!
Also, changed the placement of the sharpening to the softer kernel picture in SpliceResize itself, as the sharpening was interfering with the comparison between soft and sharp kernel pictures.

[TheProfileth]

Hmm, I never did like blindpp, specially since it adds a mod16 requirement to your script I suggest you try something like SmoothD (http://www.funknmary.de/bergdichter/projekte/video/SmoothD/) , since it can do something similar to blindpp but without most of the odd blocking artifacts and mod16 requirements. A personal favorite of mine is tbilateral, I have done extensive testing regarding it, so here is what I suggest you try

function canny(clip src, int "sensitivity")
{
sensitivity = 1.2 - (sensitivity/16.67)
v=src.grayscale.tbilateral(5,3,1.3,1,5,5,1,0,true,false,kerns=4,kerni=3,restype=0).tcanny(sigma=sensitivity,t_l=4,t_h=4))

return v
}
that should be both more accurate and a good bit faster, it was atleast in my tests :cool: if you want it to be more accurate just lower the 5 or less accurate raise the 5 (must be in odd numbers 3,5,7,9,11)

[javlak]

Not much time in my hands right now, but I'll test how things go with tbilateral too when I find some.

Just a note, we don't really care about definition or loss of detail (in the Canny edge detection mask at least), but only to keep our edges, so taking a big softening/degraining/denoising hammer and then (maybe) sharpening just a bit is not illogical.

[javlak]

I really like tbilateral, I can't believe I hadn't used it before! I managed to squeeze a bit more detail and keeping even more edges even though noise tolerance is in nice levels (in my sources and tests anyway). Took the calls to blur completely out!
*Changed the tcanny function; Performance and better edge detection in the Canny edge detection function.

[TheProfileth]

I really like tbilateral, I can't believe I hadn't used it before! I managed to squeeze a bit more detail and keeping even more edges even though noise tolerance is in nice levels (in my sources and tests anyway). Took the calls to blur completely out!
*Changed the tcanny function; Performance and better edge detection in the Canny edge detection function.
Your welcome :cool: