Help using Pixiedust and LimitedSharpen

[nightrhyme]

Source:
http://img29.echo.cx/img29/8449/ews6ay.th.png (http://img29.echo.cx/img29/8449/ews6ay.png)

I'm trying to learn a little and gain some experience using denoiser and sharpener. After reading a billion threads I have come up with the following script. I would like a little input on the parameters since I don't really understand.

Pixiedust: What parameters to use? I want to remove as much grain as possible.

LimitedSharpen: What parameters to use? I want to make it as sharp as possible. Also is my LimitedSharpen function the most recent ?

Any other recomendations would be very welcome :D

//Thanx in advance

AVS Script:

# PLUGINS
LoadPlugin("C:\PROGRA~1\GORDIA~1\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\UnDot.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\LoadPluginEX.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\dustv5.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\warpsharp.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\MaskTools.dll")

Import("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\LimitedSharpen.avs")


# SOURCE
mpeg2source("E:\Eyes Wide Shut DVD\RIP\EWS.d2v")

# CROPPING
crop(0,0,718,574)

# RESIZING
LanczosResize(656,480)

# DENOISING:
Undot()

converttoyuy2()
pixiedust(limit=5)
converttoyv12()

#Sharpen:
LimitedSharpen(ss_x=2,ss_y=2,Smode=3,strength=100)

# DENOISING:
Undot()



LimitedSharpen Function:
# LimitedSharpen()
#
# A multi-purpose sharpener by Didée
#

function LimitedSharpen( clip clp,
\ float "ss_x", float "ss_y",
\ int "dest_x", int "dest_y",
\ int "Smode" , int "strength", int "radius",
\ int "Lmode", bool "wide", int "overshoot",
\ bool "soft", int "edgemode", bool "special",
\ int "exborder" )
{
ox = clp.width
oy = clp.height
ss_x = default( ss_x, 1.5 )
ss_y = default( ss_y, 1.5 )
dest_x = default( dest_x, ox )
dest_y = default( dest_y, oy )
Smode = default( Smode, 3 )
strength = Smode==1
\ ? default( strength, 160 )
\ : default( strength, 100 )
strength = Smode==2&&strength>100 ? 100 : strength
radius = default( radius, 2 )
Lmode = default( Lmode, 1 )
wide = default( wide, false )
overshoot= default( overshoot, 1)
overshoot= overshoot<0 ? 0 : overshoot
soft = default( soft, false )
edgemode = default( edgemode, 0 )
special = default( special, false )
exborder = default( exborder, 0)
#radius = round( radius*(ss_x+ss_y)/2) # If it's you, Mug Funky - feel free to activate it again
xxs=round(ox*ss_x/8)*8
yys=round(oy*ss_y/8)*8
smx=exborder==0?dest_x:round(dest_x/Exborder/4)*4
smy=exborder==0?dest_y:round(dest_y/Exborder/4)*4

clp.isYV12() ? clp : clp.converttoyv12()

ss_x != 1.0 || ss_y != 1.0 ? last.lanczosresize(xxs,yys) : last
tmp = last

edge = logic( tmp.DEdgeMask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5", divisor=2)
\ ,tmp.DEdgeMask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5", divisor=2)
\ ,"max").levels(0,0.86,128,0,255,false)

bright_limit = (soft == true) ? tmp.blur(1.0) : tmp
dark_limit1 = bright_limit.inpand()
bright_limit1 = bright_limit.expand()
dark_limit = (wide==false) ? dark_limit1 : dark_limit1 .inflate.deflate.inpand()
bright_limit = (wide==false) ? bright_limit1 : bright_limit1.deflate.inflate.expand()
minmaxavg = special==false
\ ? yv12lutxy(dark_limit1,bright_limit1,yexpr="x y + 2 /")
\ : maskedmerge(dark_limit,bright_limit,tmp,Y=3,U=-128,V=-128)

Str=string(float(strength)/100.0)
normsharp = Smode==1 ? unsharpmask(strength,radius,0)
\ : Smode==2 ? sharpen(float(strength)/100.0)
\ : yv12lutxy(tmp,minmaxavg,yexpr="x x y - "+Str+" * +")

OS = string(overshoot)
Lmode == 1 ? yv12lutxy( bright_limit, normsharp, yexpr="y x "+OS+" + < y x "+OS+" + ?")
\ : yv12lutxy( bright_limit, normsharp, yexpr="y x "+OS+" + < y x y x - "+OS+" - 1 2 / ^ + "+OS+" + ?")
Lmode == 1 ? yv12lutxy( dark_limit, last, yexpr="y x "+OS+" - > y x "+OS+" - ?")
\ : yv12lutxy( dark_limit, last, yexpr="y x "+OS+" - > y x x y - "+OS+" - 1 2 / ^ - "+OS+" - ?")

edgemode==0 ? NOP
\ : edgemode==1 ? MaskedMerge(tmp,last,edge.inflate.inflate.blur(1.0),Y=3,U=1,V=1)
\ : MaskedMerge(last,tmp,edge.inflate.inflate.blur(1.0),Y=3,U=1,V=1)

(ss_x != 1.0 || ss_y != 1.0)
\ || (dest_x != ox || dest_y != oy) ? lanczosresize(dest_x,dest_y) : last

ex=blankclip(last,width=smx,height=smy,color=$FFFFFF).addborders(2,2,2,2).coloryuv(levels="TV->PC")
\.blur(1.3).inpand().blur(1.3).bicubicresize(dest_x,dest_y,1.0,.0)
tmp=clp.lanczosresize(dest_x,dest_y)

clp.isYV12() ? ( exborder==0 ? tmp.mergeluma(last)
\ : maskedmerge(tmp,last,ex,Y=3,U=1,V=1) )
\ : ( exborder==0 ? tmp.mergeluma(last.converttoyuy2())
\ : tmp.mergeluma( maskedmerge(tmp.converttoyv12(),last,ex,Y=3,U=1,V=1)
\ .converttoyuy2()) )

return last
}
#__________________

[iradic]

hi

loadpluginex and dust should be last filters loaded (any 2.0x filter also should be loaded before dust also)

maybe removegrain instead undot?

[Chainmax]

I'd recommend you to use LRemoveDust instead of Dust. The former has been said to be almost identical to the latter while being much faster and avoiding the latter's problems when setting high limit values. The recommended call line for LRemoveDust is LRemoveDust(17,x), where x is the denoising strength (for which 2 is a good start value). You can find LRemoveDust latest version here (http://forum.doom9.org/showthread.php?p=663516&highlight=lremovedust+yuy2#post663516) and the required RemoveGrain package at the beginning of that thread.


[edit]
A couple more things:
Your LimitedSharpen call line is redundant as Smode=3 and strength=100 are default values. Just using LimitedSharpen(ss_x=2,ss_y=2) will do the exact same thing.
Also, I remember reading that someone (Didée, IIRC) said that higher ss values could result in slightly more softness, so maybe you should just use LimitedSharpen(). The latest script version is at the first post of its thread if I'm not mistaken.

[nightrhyme]

Thanx for reply.

I see what you mean about LimitedSharpen. My testing so far also showed that default LS looks better.

Ok I'll try LRemoveDust. The reason I chose PixieDust was because everybody seems to regard it as the best filter :)

[nightrhyme]

OK I can't make this work. This is where I'm at. Error= Don't know what MODE means ?


# PLUGINS
LoadPlugin("C:\PROGRA~1\GORDIA~1\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\UnDot.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\LoadPluginEX.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\dustv5.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\warpsharp.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\MaskTools.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\RemoveGrain.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\Repair.dll")

Import("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\LimitedSharpen.avs")
Import("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\LRemoveDust_YUY2.avs")


# SOURCE
mpeg2source("E:\Eyes Wide Shut DVD\Test\t.d2v")

# CROPPING
crop(0,0,718,574)

# RESIZING
LanczosResize(656,480)

# DENOISING:
Undot()

ConverttoYV12()
LRemoveDust_YUY2(17,2)
ConverttoYUY2()

#Sharpen:
LimitedSharpen()

# DENOISING:
Undot()

[Didée]

I'd recommend you to use LRemoveDust instead of Dust. The former has been said to be almost identical to the latter while being much faster and avoiding the latter's problems when setting high limit values.
Dangerous. LRemoveDust for sure is not "almost identical". It comes fairly close, especially when taking the huge speed advantage into account. But looking at the final result only, IMHO PixieDust is clearly superior.

PixieDust might have that issue with occasional blocking. But LRemoveDust, OTOH, is not that well suited for sharpening operations done after it. LRD does produce slight pixelation, which is an inherent effect of the median-alike filters it uses internally. When doing nothing but plain denoising, the pixelation effekt is so weak that it's hardly objectable. But when running a sharpener like LimitedSharpen afterwards, there is danger of enhancing those weak artefacts, making them clearly visible.


@ nightrhyme:

Your "source" picture isn't the source, but the processed output, yes? The resolution is suggesting that ...

My guess is that (limit=3) should be high enough here. Also, I'd suggest to *not* resize before PixieDust (comment out that LanczosResize), and instead call

LimitedSharpen(ssx=2.0,ssy=1.67,dest_x=656,dest_y=480,Smode=3,strength=100).

You might want to additionally try "wide=true", and/or higher strength (up to 150, perhaps?).

If the final output is not "calm" enough, perhaps put the following after LimitedSharpen:
(Needs the "RemoveGrain.dll" plugin to be loaded)

yv12lutxy(last,last.clense(),"x 3 + y < x 2 + x 3 - y > x 2 - x 49 * y 51 * + 100 / ? ?",U=2,V=2)

This helps reducing some shimmering that has been caused by the former sharpening. (But then, it ~might~ cause some slight smearing in flat moving areas. Try and see. )

Finally, for the general idea of PixieDust + Sharpening, there is iiP - if you have the patience ;)
(thread (http://forum.doom9.org/showthread.php?t=70916&page=1&pp=20), and last version (http://forum.doom9.org/showthread.php?p=517100#post517100).)

___

edit: Ooops. Alas we've still no own thread for LRemoveDust ( :o ), and the (various) versions of it are somewhat scattered over threads and forums (Gleitz, too) ...

Please post the LRemoveDust script you used.

[Chainmax]

nightrhyme: one thing I can notice is that you didn't load SSETools, which is a required filter and comes in the RemoveGrain package. I don't know if that should cause that error message, but you certainly won't be able to run the function without it.

Didée: he used the script posted by Boulder at the end of the RemoveGrain thread.

[nightrhyme]

Thanx for reply Didée

As I have a lot of faith in you I will not bother with the LRemoveDust. (I tried loading the sseTools same error)
and the script I used was:

function LRemoveDust_YUY2(clip input, int clmode, int "limit")
{
clmode=default(clmode,17)
limit=default(limit,2)
repmode = 2
clensed = Clense(input, grey=true)
rep = Repair(clensed, input, mode=repmode, modeU=-1)
rg = RemoveGrain(rep, mode=_mode, modeU=-1)
return LimitChange(rg, input, limit, limitU=255)
}


A lot of info there. Will give it all a try.

My Source screenshot was made using only this avs (Only way I know to get the exact same frame):

LoadPlugin("c:\PROGRA~1\GORDIA~1\DGMPGDec\DGDecode.dll")
# SOURCE
mpeg2source("E:\Eyes Wide Shut DVD\RIP\EWS.d2v")
# CROPPING
crop(2,0,716,574)
# RESIZING
LanczosResize(656,480)


Here's the result using your recommendation. Somehow a purple line was introduced in the right side and the cropping is screwed. Don't know why:

AVS Used:

# PLUGINS
LoadPlugin("C:\PROGRA~1\GORDIA~1\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\UnDot.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\warpsharp.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\MaskTools.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\LoadPluginEX.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\dustv5.dll")

Import("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\LimitedSharpen.avs")

# SOURCE
mpeg2source("E:\Eyes Wide Shut DVD\Test\t.d2v")


# CROPPING
crop(2,0,716,574)

# RESIZING
#LanczosResize(656,480)

# DENOISING:
Undot()

converttoyuy2()
PixieDust(5)
converttoyv12()

#Sharpen:
LimitedSharpen(ss_x=2.0,ss_y=1.67,dest_x=656,dest_y=480,Smode=3,strength=100)

# DENOISING:
Undot()


SOURCE:

http://img72.echo.cx/img72/934/source8cw.th.png (http://img72.echo.cx/img72/934/source8cw.png)

ENCODE:

http://img106.echo.cx/img106/3288/sharp5cl.th.png (http://img106.echo.cx/img106/3288/sharp5cl.png)

The result looks very good appart from the cropping and the purple line :D
Please let me know if you have anymore suggestions. Also please help me fix the purple line and cropping :)

BTW: Why the odd value ss_y=1.67

Again Thanx

[pelle412]

The purple/pink line comes from PixieDust. Your frame size is only mod-2 which does not work well with Dust filters. Try mod-8.

[Soulhunter]

@ nightrhyme

Replace PixieDust(5) with FFT3DFilter(Sigma=1) or so... ^^

No need for color conversations and it gives fantastic results imo !!!


Bye

[nightrhyme]

I'm not sure I understand. MOD-SIZE is refering to the resizing right ?
Well my resizing is MOD-16 :confused:

Edit: There was an error in my cropping should be crop(2,0,716,574)
Trying re-encode now

[Didée]

PixieDust requires MOD16 horizontally, and MOD8 vertically. But when I noticed that I should have (re-) mentioned that, you just had dropped offline, nightrhyme ;)

Try like this:

- comment out the initial cropping, too, and then

LimitedSharpen(... , dest_x=656+6, dest_y=480+4, ...)
crop(2,2,-4,-2)

The "odd" value of ssy=1.67 is because the initial resizer now is deactivated. Formerly, you did 200% of vertical supersampling on a frame that was already vertically reduced by 17%. Now 167% supersampling is used on the original frame height. 2.0 * (480/576) = ~167, and 1.67 * (576/576) = ~167, see ...

My experience with FFT3D is rather little, so I can't coment on that one.

Lastly, think over your processing strategie as a whole. Judging from a single screenshot might be tricky - but there is pretty little difference between those two. Sharpness or detail seems not really enhanced, and the noise seems not reduced, but only differently distributed. In a word: I don't see any improvement achieved through the filtering, like e.g. here (http://home.arcor.de/dhanselmann/_samples/LOST_comparison%232-2.mkv) ;) ... perhaps it looks different in motion. How's the compressabilty before/after filtering ?

[nightrhyme]

Ok I'll try what you suggested.
I agree with you. The achieved results are not a drastically improvement over the source but it does look cleaner. The compressabilty isn't improved much either. I would love to be able to achieve results like the sample you posted. But I have no idea how to accomplish that :confused:

Perhaps I could provide you a vob sample. How do I cut a vob for you ?

[nightrhyme]

Ok I made a VOB sample.
Download link: http://rapidshare.de/files/2472303/Eyes.Wide.Shut.-.Sample.vob.html 21MB

If your not familiar with rapidshare you have to wait for the countdown :rolleyes:

[Chainmax]

LRD does produce slight pixelation, which is an inherent effect of the median-alike filters it uses internally. When doing nothing but plain denoising, the pixelation effekt is so weak that it's hardly objectable. But when running a sharpener like LimitedSharpen afterwards, there is danger of enhancing those weak artefacts, making them clearly visible.

Then it's just a matter of using a deringer like BlindPP(quant=0,cpu=0,cpu2="ooooxx") after LRD and before sharpening, isn't it?

[Chainmax]

the script I used was:

function LRemoveDust_YUY2(clip input, int clmode, int "limit")
{
clmode=default(clmode,17)
limit=default(limit,2)
repmode = 2
clensed = Clense(input, grey=true)
rep = Repair(clensed, input, mode=repmode, modeU=-1)
rg = RemoveGrain(rep, mode=_mode, modeU=-1)
return LimitChange(rg, input, limit, limitU=255)
}



Yeah, that's the problem. The idea was that you copied Boulder's post as-is (both versions on the same .avs) and call each version according to the stream. In this case, you have to use LRemoveDust_YV12