Motion Compensated Sharpening

[rkalwaitis]

I was hoping that someone could please help me find a motion compensated script similar to the process that is used in MC_Spuds.

Thanks

[Didée]

Such can be found in several scripts - TGMC has something like that, MC_Spuds has (borrowed from dunno-where), TemporalDegrain has, ...

Some "basic" scripts to demonstrate the general principle ...

Basic script, variant#1: "spatial sharpen, temporally limited".

source = whatever
sup    = source.MSuper(pel=2,sharp=2)
bv1    = sup.MAnalyse(isb=true, delta=1, [more params to liking], ...)
fv1    = sup.MAnalyse(isb=false,delta=1, [more params to liking], ...)
bc1    = source.MCompensate(sup,bv1,thSAD=500)
fc1    = source.MCompensate(sup,fv1,thSAD=500)
max    = mt_logic(bc1,fc1,"max").mt_logic(source,"max")
min    = mt_logic(bc1,fc1,"min").mt_logic(source,"min")
sharp1 = source.sharpen(1).mergechroma(last) # chroma sharpen usually is unadvantageous

sharp1.mt_clamp(max,min,0,0,U=2,V=2)

edit: corrected vector naming: "bv" -> "bv1"



Basic script, variant#1a: "spatial sharpen, temporally limited, using median-sharpen for more headroom on temporal limiting"

source = whatever
sharp0 = source.mt_adddiff(mt_makediff(source,source.removegrain(4)),U=2,V=2) # "median sharpen" (won't create halos on its own, IF the source is halo-free)
sup1   = source.MSuper(pel=2,sharp=2)
sup2   = sharp0.MSuper(pel=2,sharp=2,levels=1)
bv1    = sup1.MAnalyse(isb=true, delta=1, [more params to liking], ...)
fv1    = sup1.MAnalyse(isb=false,delta=1, [more params to liking], ...)
bc1    = source.MCompensate(sup2,bv1,thSAD=500)  # using the super clip from the median-sharpener, to provide
fc1    = source.MCompensate(sup2,fv1,thSAD=500)  # more headroom for the limiting process
max    = mt_logic(bc1,fc1,"max").mt_logic(source,"max")
min    = mt_logic(bc1,fc1,"min").mt_logic(source,"min")
sharp1 = source.sharpen(1).mergechroma(last) # chroma sharpen usually is unadvantageous

sharp1.mt_clamp(max,min,0,0,U=2,V=2)

edit: corrected vector naming: "bv" -> "bv1"




Basic script, variant#2: "spatial sharpen, temporally averaged"

source = whatever
sharp0 = source.sharpen(1).mergechroma(source)
sup1   = source.MSuper(pel=2,sharp=2)
sup2   = sharp0.MSuper(pel=2,sharp=2,levels=1)
bv1    = sup1.MAnalyse(isb=true, delta=1, [more params to liking], ...)
fv1    = sup1.MAnalyse(isb=false,delta=1, [more params to liking], ...)
bv2    = sup1.MAnalyse(isb=true, delta=2, [more params to liking], ...)
fv2    = sup1.MAnalyse(isb=false,delta=2, [more params to liking], ...)

source.MDegrain2(sup2,bv1,fv1,bv2,fv2)

Note that the MDegrain has "source" as 1st input, not "sharp0". The idea is to use sharpening only in areas where the motion estimation found good matches. In places where no good motion-match was found, it's probably better to not use sharpening. If you want to be more aggressive with sharpening, type "sharp0.Mdegrain2(...)" there.

This method still does *denoise* the source on the way, more or less. May be wished, or it may be not. If denoising is not wished, adapt the script to use the sharp-difference instead:


Basic script, variant#2a: "spatial sharpen, temporally averaged, without denoising"

source = whatever
sharp0 = source.sharpen(1)
sharpD = mt_makediff(source,sharp0)
zeroD = sharpD.mt_lut("x",Y=-128)

sup1   = source.MSuper(pel=2,sharp=2)
sup2   = sharpD.MSuper(pel=2,sharp=2,levels=1)
bv1    = sup1.MAnalyse(isb=true, delta=1, [more params to liking], ...)
fv1    = sup1.MAnalyse(isb=false,delta=1, [more params to liking], ...)
bv2    = sup1.MAnalyse(isb=true, delta=2, [more params to liking], ...)
fv2    = sup1.MAnalyse(isb=false,delta=2, [more params to liking], ...)

zeroD.MDegrain2(sup2,bv1,fv1,bv2,fv2)

source.mt_makediff(last,U=2,V=2)

This code does "no sharpen on bad motion-match" in the same way. For the aggressive variant, type "sharpD.MDegrain2(...)".

Hope there are no errors due to the hurry of typing ... if you find some, do speak.

[rkalwaitis]

Thanks Didee,

Please forgive this question, but in example 1, how do I calculate or determine the min and the max.

[Didée]

No problem, you're welcome. But let me point out these scripts really do only show the "barebone principle". For optimal results, things probably should be done a bit more artistic. E.g., none of those basics does care for "oversharpening" (halo creation).

With script#1, halos are a lesser problem. The problem rather is that you'll only get very little sharpening, due to the pretty strong constraint. For getting more effect, it's needed to provide more "headroom" that the sharpening can expand into. (I quite like median-sharpening for that task, but the available possibilities are countless.)

With script#2/2a, the sharpening can create halos quite easily, so in order to avoid that issue, the basic sharpening process should be more sophisticated than just "sharpen(1)".
But just replacing sharpen() with LimitedSharpen() isn't a too brilliant idea either - LS/F probably is too weak for that. Something in the spirit of SeeSaw's sharpening, used on a slightly pre-blurred input, is what often works good.


It's like it always is: the basic principle is not very hard. But tweaking it so that it works as good as possible in practice, and in as many different situations as possible - that's the harder task.
(And that's also why I find myself quite often just making a whole new script from scratch for any given source, rather then trying to come up with can-do-everything multi-purpose scripts, using two hundred different parameters, which most people wouldn't use anyway...)


Edit: just saw your edit - oh, there's a mistake in script#1. The min+max need also the current frame into account, of course! Script corrected.

[rkalwaitis]

Thanks again, I understand that they are the bare bones, and to be honest that is what I was looking for. It helps me to understand the process better. Im gonna play with the toys you gave me. I dont know why I was thinking that a motion compensated sharpening would be more capable of fending of haloing. I must be motion compensated because I usually am the culprit of the artifacts.

Thanks again

[Didée]

I have added script variant "1a", as an example for how to give more sharpening headroom via a median-sharpener.

Also cleared the syntax of variant#1 - the usage of "last" in two places was a bit muddy, "source" is more clear in syntax/logic.

[rkalwaitis]

Thanks alot Didee, Im all over it after I finish taking care of the kids. Cool flickr gallery

[rkalwaitis]

Didee does it make sense to compensate more than one frame forward and one frame backward when attempting the sharpening? Variant 2a worked well.

[Undead Sega]

wow there's such thing as Motion Compensated Sharpening? i dreamt of this and was thinking of mentioning it but was abit afraid of being tampered down for such silliness but this is quite suprising to me. May i ask how does this differ to other conventional sharpeners out there?

[Didée]

What I have showed is traditional spatial sharpen, plus motion-compensated postprocessing.

You shouldn't "dream" of things just because it "sounds so cooool". Do you want to have a motion-compensated "trim()" filter? Or perhaps a neural network that performs "AddBorders()"?

Before getting all too excited about funky terminology, it's always better to understand what it's supposed to mean (or to do).

[Dogway]

This is interesting, but why not make it motion-compensated contrasharpening?

[scharfis_brain]

maybe with motion compensated sharpening he meant a motion compensated temporal sharpener, which effectively will enhance grain and noise?
(it is the opposite of a temporal smoother (denoiser))

[Soulhunter]

So, sorta like UnSmooth back then?

[Didée]

Originally Posted by scharfis_brain

maybe with motion compensated sharpening he meant a motion compensated temporal sharpener

Experience has told that people oftentimes just do not know what they mean when saying "motion compensated sharpen".

... which effectively will enhance grain and noise?
(it is the opposite of a temporal smoother (denoiser))

Exactly that. In theory, you would use a "mocomp temporal sharpen" principle when trying to squeeze additional information out of the "subpixel level". But in practice, you'll just enhance noise that way.

Originally Posted by Dogway

This is interesting, but why not make it motion-compensated contrasharpening?

Contra-sharpening implies that anything has been removed from the source beforehand.
And ... the basic principle is already included, namely in script#1.

[Nephilis]

<< Basic script, variant#2: "spatial sharpen, temporally averaged" >> is the real deal. I use it since Didée posted it in another thread.. With SeeSaaw it works like a charm

Greetz fly out to Didée

[rkalwaitis]

Nephilis, could you please post your seesaw script so I can see how you use variant#2. Thanks. I like the first script with mdegrain2. MP4.Guy also has a MedSharp that he is working on that is interesting. Its one of the newer posts. You should check it out.

http://forum.doom9.org/showthread.php?t=153201

[Vitaliy Gorbatenko]

Nephilis: Post a complete script, please!

[Undead Sega]

sample images by any chance?

[Nightshiver]

source = whatever
sharp0 = source.sharpen(1).mergechroma(source)
sup1   = source.MSuper(pel=2,sharp=2)
sup2   = sharp0.MSuper(pel=2,sharp=2,levels=1)
bv1    = sup1.MAnalyse(isb=true, delta=1, [more params to liking], ...)
fv1    = sup1.MAnalyse(isb=false,delta=1, [more params to liking], ...)
bv2    = sup1.MAnalyse(isb=true, delta=2, [more params to liking], ...)
fv2    = sup1.MAnalyse(isb=false,delta=2, [more params to liking], ...)

b = source.MDegrain2(sup2,bv1,fv1,bv2,fv2)

a = last
SeeSaw(a, b, whatever settings you want)

Or, if going along with Didee's way of repacing Sharpen() with SeeSaw(), I think it would look something like this:

source = whatever
sharp0 = source.SeeSaw(whatever settings you want).mergechroma(source)
sup1   = source.MSuper(pel=2,sharp=2)
sup2   = sharp0.MSuper(pel=2,sharp=2,levels=1)
bv1    = sup1.MAnalyse(isb=true, delta=1, [more params to liking], ...)
fv1    = sup1.MAnalyse(isb=false,delta=1, [more params to liking], ...)
bv2    = sup1.MAnalyse(isb=true, delta=2, [more params to liking], ...)
fv2    = sup1.MAnalyse(isb=false,delta=2, [more params to liking], ...)

source.MDegrain2(sup2,bv1,fv1,bv2,fv2)

[Dogway]

How can I wrap it into a function?