Soothe() - small helper function for sharpening with less temporal jitter

[Didée]

Hello friends,

this is a small postprocessor function for sharpening filters. The goal is temporal stabilization of clips that have been sharpened before. It is reasonably fast (contains 3 different YV12Lutxy operations and one TemporalSoften - that's about the cheapest possibility for what is done. Plugin coders are welcome :) ), and seems to work pretty well.

The introduction of jitter on the temporal axis is a general problem of sharpening operations, since sharpening (usually) considers spatial aspects only. Therefore, Soothe() does a very simple job: get the difference between [source] and [sharpened source], apply a TemporalSoften on this difference BUT allow only changes towards 128 ("neutral"), and then apply this temporally calmed difference back to the original clip.
Effectively, this will reduce the overall effect of sharpening - less in static areas, and more in moving areas.

Advantages:

- more steady appearance (less "nervous")
- less bitrate required
- somewhat positive effect on detail that is, due to the sharpening, prone to aliasing
- smoother motion compaired to plain-sharpening, since motion-blurred edges will be less sharpened
- less artefacts in moving areas
- LimitedSharpen can run faster, since one can get away with less supersampling ;)
- should only help, never harm

Disadvantages:

- overall sharpening effect is reduced, but this can be compensated by a little more initial sharpening.


Syntax:

Soothe( [sharpclip], [sourceclip], keep )

"keep" is an integer, range 0 - 100, that tells how much percent of the original sharpening will be kept at least. Default is 25.


Example:

We use LimitedSharpen() as sharpener, and we'll keep at least 20% of its result:
dull = last
sharp = dull.LimitedSharpen( ss_x=1.25, ss_y=1.25, strength=150, overshoot=1 )

Soothe( sharp, dull, 20 )
Easy, isn't it?


Script function: (needs the MaskTools (http://manao4.free.fr/), as usual)

edit 21 Nov 05: corrected LUT expression! (old one could cause weird blurring in certain places!)
function Soothe(clip sharp, clip orig, int "keep")
{
keep = default(keep, 24)
keep = (keep>100) ? 100 : (keep<0) ? 0 : keep
KP = string(keep)
diff = yv12lutxy(orig,sharp,"x y - 128 +", U=1,V=1)
diff2 = diff.temporalsoften(1,255,255,32,2)
diff3 = yv12lutxy(diff,diff2, "x 128 - y 128 - * 0 < x 128 - 100 / " + KP
\ + " * 128 + x 128 - abs y 128 - abs > x " + KP
\ + " * y 100 " + KP + " - * + 100 / x ? ?", U=1,V=1)
return( yv12lutxy(orig,diff3,"x y 128 - -",U=2,V=2) )
}

Note: Basically, the function is as dumb as it can only be ... but that doesn't seem to hold it back from doing a nice job. :)


Feel free to post your impressions, and/or if you think something should be altered or improved.

[unskinnyboy]

Whoa! Another Didée product! Can't go wrong, I'm sure. Can't wait to see the effects of this with my own eyes. This seems to effectively address some of the concerns I had with LS() usage. Will be testing soon..

Thanks!

[Pookie]

Thank You, Thank You, Thank You !

Just tried it. Works with other, less sophisticated sharpeners as well. Great job.

[shaolin95]

Didee will this help the problem I posted I have with LimitedSharpen with some DVDs. I created a post about it recently. Some DVDs like Two Towers, Die Another Day looks simply awesome almost HD in perceived detail but others show many artifacts like the superbit titles Underworld and Fifth Element. They show artifacts mainly along vertical lines.

[MOmonster]

Thanks for this new function, Didee.
I´ll test it tomorrow. This is the first time I can fully understand how your function works (thanks for the explaination of the reverse polish notation ;) ).
Keep on the good work.

[shaolin95]

I just tried it like you posted here in the example and I like it! It helped me fix those DVDs I was having issues with when using LimitedSharpen (the artifacts in vertical lines).
This is my call:

Import("C:\Program Files\AviSynth 2.5\plugins\LimitedSharpen.avs")

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MaskTools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\WarpSharp.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\colormatrix.dll")

dull = last
sharp = dull.LimitedSharpen( ss_x=1.0, ss_y=1.0,
\ Smode=3, strength=90, radius=2,
\ Lmode=1, wide=false, overshoot=1,
\ soft=false, edgemode=0, special=false,
\ exborder=0 )

ColorMatrix()

Soothe( sharp, dull, 20 )

I am a Avisynth super noob so most of my calls are copy paste and stich them together style. I am still able to resize 2.5x and get smooth playback with this Soothe and LS so to me is a keeper. Gonna try some other settings this weekend as my wife wasnt to happy I had yet another thing to try tonight with my projector...women!!!

[Firesurfer]

@shaolin95:
I hope you are aware, that the call of ColorMatrix() is not affecting the video in this script. Also ColorMatrix() should be used as first filter in the chain. (Im not sure with interlaced material, though.)

[Audionut]

I can't get it to work with limited sharpen.

error:

Invalid arguments to function "width"
test.avs, line 19

Script:

import("g:\x\ltsmc.avs")
import("g:\x\soothe.avs")
import("g:\x\blinddehalo3.avs")
import("g:\x\lsharpen2.avs")
source=mpeg2source("D:\test.d2v")
source=crop(source,8,8,-8,-8)
source=lanczos4resize(source,704,384)
video=mpeg2source("D:\test.d2v")
video=crop(video,8,8,-8,-8)
video=ltsmc(video,apmode=4,blcksze=4,chromame=true,sharpness=100)
video=lanczos4resize(video,704,384)
video=BlindDeHalo3(video, rx=3.0, ry=3.0, strength=50,
\ lodamp=0.0, hidamp=0.0, sharpness=1.0, tweaker=0.0,
\ PPmode=1, PPlimit=0, interlaced=false)
video=LimitedSharpen(video,ss_x=1.5, ss_y=1.5, dest_x=last.width, dest_y=last.height,
\ Smode=3, strength=300, radius=3,
\ Lmode=1, wide=false, overshoot=1,
\ soft=false, edgemode=0, special=false,
\ exborder=0 )
soothe(video,source,10)

[Didée]

Even without Soothe(), *that* script will not load.

Educational question: Why? ;)


edit: Oh, you cleaned up the script. Much better now ... almost good.

Remaining problem: you are assigning all sources directly to clip variables. Hence there's never anything assigned to the "last" variable, hence the "last.width" and "last.height" are causing the error. To make the script semantically correct, you have to write "source.width" and "source.height" in the LimitedSharpen line.

But even then, this is wrong usage. Actually, you're letting Soothe() reduce ALL effects that you have applied to "video", including the de-haloing and the noise reduction. Most probably that is not what you want to do ...

(BTW, opening the same source two times generally is not a smart idea.)

import("g:\x\ltsmc.avs")
import("g:\x\soothe.avs")
import("g:\x\blinddehalo3.avs")
import("g:\x\lsharpen2.avs")

video = mpeg2source("D:\test.d2v")
video = video.crop(8,8,-8,-8)
video = video.ltsmc(apmode=4,blcksze=4,chromame=true,sharpness=100)
video = video.lanczos4resize(704,384)
video = video.BlindDeHalo3(rx=3.0, ry=3.0, strength=50,
\ lodamp=0.0, hidamp=0.0, sharpness=1.0, tweaker=0.0,
\ PPmode=1, PPlimit=0, interlaced=false)
video2= video.LimitedSharpen(ss_x=1.5, ss_y=1.5,
\ Smode=3, strength=300, radius=3,
\ Lmode=1, wide=false, overshoot=1,
\ soft=false, edgemode=0, special=false,
\ exborder=0 )
soothe(video2,video,10)
Now Soothe() is postprocessing only LS's sharpening and nothing else, like intended.

[Audionut]

Thanks for the clarification.

Works like a charm.

[psme]

Will this work for REALTIME playback in FFDShow->avisynth, like LimitedSharpen? TIA.

regards,

Li On

[Didée]

Depends on the horsepower you've available. My elderly Athlon1800 definetly is too weak for that.

Just try it, and report. :)

[mg262]

Didée,

Yay!

a little relating to this on that thread at

http://videoprocessing.11.forumer.com/viewtopic.php?t=16&postdays=0&postorder=asc&start=15

(Right at the end of page 2 and onwards)

[Didée]

Note: Edited the thread title ("... helper function ...") to better reflect the function's purpose.


Clouded - yes, already saw it. Just can't dive into it ATM :(

[psme]

Initial test for realtime playback via FFDShow->avisynth:

Import("C:\Program Files\AviSynth 2.5\plugins\ModerateSharpen.avs")
Import("C:\Program Files\AviSynth 2.5\plugins\Soothe.avs")
dull = last
sharp = dull.ModerateSharpen(0.3)

Soothe( sharp, dull, 20 )

VERY interesting. Need to test more.... :)

regards,

Li On

[mg262]

Clouded - yes, already saw it. Just can't dive into it ATM No hurry! As you may have noticed I'm being very slow at my end as well...

[shaolin95]

Sup Li On (I am Luis Gabriel Gerena from AVSFORUM),
I am already using it together with LimistedSharpen, check my previous post here and it works fine even using resize 2.5 x but I do need to run my CPU at 2660MHZ (stock is 2.4).

One question, why doesnt the ColorMatrix work in my call script? I just copied that from another thread about Limited Sharpen and all the scripts there had Color Matrix at the end. What is it for anyway and how do I use it?
Regards

[communist]

ColorMatrix (...) What is it for anyway and how do I use it?
Regards
http://forum.doom9.org/showthread.php?t=82217

[mg262]

As anyone who peeked at that link may have seen, I'm trying to write some auxiliary filter-lets to speed up LimitedSharpen and Soothe. Unfortunately, all the content I have at hand at present is a)animation and b) very noisy, which means that it's very hard to tell whether things are working correctly or not. So this is a plea for some small source on which the effect of both filters is very clear -- doesn't matter how small -- even five or 10 frames would be great. [I can keep working by using subtract, etc, but it does make life easier to be able to see what's happening!]

I can receive anything at blendarchive@gmail.com -- and anything will be very gratefully received! (But no 18-rated material or the like please!)

[Firesurfer]

First: My Apologies to Shaolin95. I typed my comment in a hurry without clarifying what I meant. I guess there's even a forum rule against that ;)

Second:
To make my point I repeat your script first:

Import("C:\Program Files\AviSynth 2.5\plugins\LimitedSharpen.avs")

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MaskTools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\WarpSharp.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\colormatrix.dll")

dull = last
sharp = dull.LimitedSharpen( ss_x=1.0, ss_y=1.0,
\ Smode=3, strength=90, radius=2,
\ Lmode=1, wide=false, overshoot=1,
\ soft=false, edgemode=0, special=false,
\ exborder=0 )

ColorMatrix()

Soothe( sharp, dull, 20 )

You use the variables dull and sharp for your video, but you don't specify on which of these variables you want to use colormatrix(), so it uses implicit last, which could be sharp or even the unfiltered stream that AviSource gives (I would presume the second). both are not what you want. After that Sothe() is used on sharp and dull, but not on the stream that ColorMatrix() has filtered.
AviSynth documentation on variables should give you further clarification about this.

After all, I guess you read the ColorMatrix()-thread by now and know if you should be using it and why you should use it as the first filter after ...Source().

[mg262]

you don't specify on which of these variables you want to use colormatrix(), so it uses implicit last, which could be sharp or even the unfiltered stream that AviSource gives (I would presume the second). both are not what you want.I think in this case it will actually generate an error -- AVISynth is pretty good about that.

[Firesurfer]

I think in this case it will actually generate an error -- AVISynth is pretty good about that.

You are right! It should give an error. But since shaolin95 posted his script, I assumed he used it that way without errors and after I tried it myself I found, that it does in fact give not an error. So I'm not sure on which of the variables ColorMatrix() is used and I thougth it would be good to hint at that. :)

[mg262]

Really!? I'm amazed that even dull = last gets through. My mistake.

[Piper]

@Didée: Thanks very much for this filter. It's an elegant solution to a common problem. :D

[shaolin95]

Any ideas on how to improve my script to better use Soothe with my limitedsharpen script?

[Didée]

shaolin95 - I see no problem with the script you posted (except for no source is loaded). Just place ColorMatrix() either behind Soothe() instead before it, or put it directly after loading the source.

***

Hmmh ... it just revealed that Soothe seems to be the solver of the flickering issues of the threatened crystality enhancer, too ... :eek: :D

[mg262]

Oooh... sounds intriguing. But what exactly is crystality ;)?

[Didée]

In fact it's "weak detail overdrive sharpening" - but "crystality" sounds much cooler. :)

Uh, where were the links ... three examples were linked in this (http://forum.doom9.org/showthread.php?p=656722&highlight=crystality#post656722) post ... cant find the post where I linked to this (http://home.arcor.de/dhanselmann/_samples/LOST_comparison#2-2.mkv) example.

[mg262]

Didée, you terrify me. If only your interest was in animated material ;)

Incidentally, the last example doesn't load (but three is plenty).

[shaolin95]

Didee, what is that vertical line artifact that I get with LimitedSharpen and some DVDs. Ive been able to minimize it with Soothe but still is visible sometimes and while is not visible enough to bother me I wouldnt mind getting rid of it completely. Is it due to warp sharps or something like that being used by Limited Sharpen? Is there any special settings for Soothe to eliminate it completely?
Regards

[Didée]

shaolin95, I can guess some things, but reading minds still is slightly beyond. Please show an example of the problem, and make a sample of the original unprocessed source available.

Perhaps it's in the settings, but supposely it's already something present in the source ... slightly below usual perception, and gets just boosted to become noticeable.
LS does not actively care about source artefacts, and Soothe tames down the effect of sharpening only, not any discontinuities that were already present beforehand. If the result is not stable because of original instabilities, then you need additional pre- or post-processing. Possibilities are plenty ... - show a sample.

[shaolin95]

Gonna try to capture with my camera cause I am using VRM9 or perhaps let me try to get screenshots under overlay. Anyway, the problem could be describe as some vertical lines (borders of ojbects) having a "wave" effect to them (wavy lines?).
Regards

[Pookie]

Didée: Is this the 'Crystality' example you were linking to ? :
Lost (http://home.arcor.de/dhanselmann/_samples/LOST_comparison%232-2.mkv)

That one in particular is rather impressive. :cool:

kassandro mentioned elsewhere "With this improved algorithm ReduceFlicker v0.5 will catch much more flicker hopefully without a significant increase of artifacts". Perhaps that might be helpful for Crystality?

[Didée]

Pookie -

Let's see. As of now, ReduceFlicker's scope is slightly too narrow to combat the issue. But the methods are there by now, so hope is rising again ;)


@ shaolin95

camera! What's wrong with making screenshots in VirtualDub?
However, in LimitedSharpen there is no warpsharping done whatsoever - I feel rather confident that you just need additional filtering for stabilisation of the source itself.


@ Clouded

How, doesn't load? It does fine here - DL'ed it myself and played back without problems. Perhaps it's the setup on your machine, regarding MKV?

[mg262]

Sorry... I get this:

FEHLER 404

Die gewünschte Internet-Seite ist nicht erreichbar. Überprüfen Sie bitte die Adresse oder versuchen Sie es später noch einmal.

But like I said, don't worry about it!

@shaolin95,
Like Didée said, in this case you should look in the video menu of VirtualDub. But in general, if you press print screen the contents of the screen are dumped into the clipboard (with a couple of caveats).

[shaolin95]

@ shaolin95
camera! What's wrong with making screenshots in VirtualDub?
However, in LimitedSharpen there is no warpsharping done whatsoever - I feel rather confident that you just need additional filtering for stabilisation of the source itself.
Ok gonna try that tonight. What do you recommend for aditional source stabilisation. I makes sense what you say cause otherwise it will be equally visible with all DVDs right? BTW, thanks for helping a noob here.
Regards

Updated post: Just wanted to add that I am using Theater Tek with ffdshow to improve DVD quality to use with my Infocus 4805 DLP projector. My screensize is 9' wide (124" diag) so improving dvd image is a must.

[Chainmax]

Does this script include a good Soothe call with the default keep value:

LRemoveDust_YV12(17,2)
Deblock(quant=30)
dull = last
sharp = LimitedSharpen()
Soothe(sharp,dull)
AddGrain(3,0,0)

?

[E-Male]

i wonder if this could also be used with other combinations than dull&sharp
for example denoised&original
i'll look into that when i find the time

[Didée]

Oh my dear. The function originally posted had an error in the LUT, that caused some weird blurring in certain spots. (Didn't check for a special condition.)

Function in the first post is updated - *now* Soothe is working correctly, as it was supposed to be.

However it seems like no-one is using this little brownie ... else, someone would have noted the defect and have reported it, no? ;)

[yidaki]

hmm, I use it :-/
Perhaps I enjoy the blurring :D

[Pookie]

I thought it was my source and choice of codec.

[iceborne]

i keep on getting washed out color whenever i applied this script:
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\mt_MaskTools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\removegrainsse2.dll")
LoadPlugin("C:\Program Files\DGMPGDec\dgdecode.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\UnDot.dll")

mpeg2source("e:\dvd\divergence.d2v")

Telecide(guide=1).Decimate()

crop(4,60,-4,-60)

trim(5000,10000)


dull = last
sharp = dull.LimitedSharpenfaster( smode=4, dest_x=720, dest_y=368 )

Soothe( sharp, dull, 20 )

undot()


but if i leave out the resizing parameter
"dest_x=720, dest_y=368 ",
everything's good.

can anyone tell me what's wrong with my script?

[foxyshadis]

Heh, I thought it was LSFaster for a few, until I noticed that you never resized dull. Pick a good scene with very sharply defined colors, and you'll notice the chroma will bleed horribly, since it gets copied from the second argument, not the first. Lesson: Always remember to resize both clips. ;)

[iceborne]

good man!

here's my new script
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\mt_MaskTools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\removegrainsse2.dll")
LoadPlugin("C:\Program Files\DGMPGDec\dgdecode.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\UnDot.dll")

mpeg2source("e:\dvd\divergence.d2v")

Telecide(guide=1).Decimate()

crop(4,60,-4,-60)

LanczosResize(720,368)



dull = last
sharp = dull.LimitedSharpenfaster( smode=4, dest_x=720, dest_y=368 )

Soothe( sharp, dull, 24 )

undot()

trim(5000,10000)

btw, does it matter where i put undot()?

[Mug Funky]

try:
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\mt_MaskTools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\removegrainsse2.dll")
LoadPlugin("C:\Program Files\DGMPGDec\dgdecode.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\UnDot.dll")

mpeg2source("e:\dvd\divergence.d2v")

Telecide(guide=1).Decimate()

crop(4,60,-4,-60)



dull = last.LanczosResize(720,368)
sharp = dull.LimitedSharpenfaster( smode=4, dest_x=720, dest_y=368 )

Soothe( sharp, dull, 24 )

undot()

trim(5000,10000)

[iceborne]

thanhs guy. another question do you think sooth(sharp, dull, 24) is too low? or should i raise it to make the image sharper

[foxyshadis]

In case anyone is feeling the burn and needs for speed, masktools 2:

function Soothe(clip sharp, clip orig, int "keep")
{
Assert(sharp.width == orig.width && sharp.height == orig.height,
\ "Soothe: clip dimentions must match!")
keep = default(keep, 24)
keep = (keep>100) ? 100 : (keep<0) ? 0 : keep
KP = string(keep)
diff = mt_makediff(orig,sharp)
diff2 = diff.temporalsoften(1,255,0,32,2)
diff3 = mt_lutxy(diff,diff2, "x 128 - y 128 - * 0 < x 128 - 100 / " + KP
\ + " * 128 + x 128 - abs y 128 - abs > x " + KP
\ + " * y 100 " + KP + " - * + 100 / x ? ?")
return( mt_makediff(orig,diff3,chroma="copy first") )
}

[edit: Added a check for dimentions, since that's the most common misuse. :p Also Didée TS suggestion.]
[edit2: Added both to wiki: http://www.avisynth.org/mediawiki/Soothe ]

[Didée]

Even faster:

>> diff2 = diff.temporalsoften(1,255,0,32,2)

'cause here, chroma isn't interesting anyways, and TS runs faster when not having to also process the UV planes ... ;)

[iceborne]

cool, will try on next encode.

[diazlaz]

Is it possible to get the soothe function to work, when a frame is being resized by LS? Example, currently have a video that is sized at 352,480, and using LimitedSharpenFaster(ss_x=1.2,ss_y=1.2,dest_x=704,dest_y=480,Smode=4,soft=24) to resize it to 704x480, when soothe runs, it errors with:

YV12LUTxy: Clip size must be the same.


function Soothe(clip sharp, clip orig, int "keep")
{
keep = default(keep, 24)
keep = (keep>100) ? 100 : (keep<0) ? 0 : keep
KP = string(keep)
diff = yv12lutxy(orig,sharp,"x y - 128 +", U=1,V=1)
diff2 = diff.temporalsoften(1,255,255,32,2)
diff3 = yv12lutxy(diff,diff2, "x 128 - y 128 - * 0 < x 128 - 100 / " + KP
\ + " * 128 + x 128 - abs y 128 - abs > x " + KP
\ + " * y 100 " + KP + " - * + 100 / x ? ?", U=1,V=1)
return( yv12lutxy(orig,diff3,"x y 128 - -",U=2,V=2) )
}


Thanks,
Laz.