Haloing, Ghosting, ... How can I get rid of that?

[moviefan]

Hi guys,

I'm currently denoising a TV series and noticed there is (for my eyes) pretty annoying haloing/ghosting around sharp edges. I already tried several filters that I could find here but none of them did the job, not even close... Can anybody help getting rid of this annoying effect?

On the first screenshot, it is very obvious around the text, on the second one, it is obvious at the border of the red area.

http://img396.imageshack.us/img396/4162/97858288dp4.png
http://img396.imageshack.us/img396/5199/13537878ys8.png

For denoising/degraining, I use MC_Spuds which does a great job, but the sharpening increases the haloing even more. The screenshots are taken from the source, no filters have been applied.

Thanks for your help!

Regards

[mikeytown2]

http://avisynth.org/mediawiki/External_filters#Dehaloing

My preference is Dehalo_alpha (http://forum.doom9.org/showthread.php?p=777956#post777956).

[martino]

Dehalo_alpha or BlindDeHalo3 and some aWarpSharp magic could do the trick.

[moviefan]

I tried those... The result is that the haloing is more blurred, but still there, bigger, but softer. Isn't there anything else I could do or what settings maybe should I use?

[Adub]

Why don't you post your script, so we can help you with your settings?

[martino]

Try something like:

BlindDeHalo3(PPmode=-3)
EdgeCleaner(12,fix=false) #since you'll need to crop afterwards anyway

# EdgeCleaner() v1.03 (06/08/2008)
# - a simple edge cleaning and weak dehaloing function
#
# Description:
# Functions have been briefly tested to work with MT on mode 1 and 2 without any problems
#
# Requirements:
# aWarpSharp, mt_masktools, Repair (optional), RemoveGrain (optional) and Deen (optional) plugins required
# YV12 input required and mod16 or even mod32 input is preferred since aWarpSharp borks sometimes
#
# Parameters:
# strength (float) - specifies edge denoising strength (8.0)
# rep (boolean) - actives Repair for the aWarpSharped clip (true; requires Repair)
# rmode (integer) - specifies the Repair mode; 1 is very mild and good for halos,
# 16 and 18 are good for edge structure preserval on strong settings but keep more halos and edge noise,
# 17 is similar to 16 but keeps much less haloing, other modes are not recommended (17; requires Repair)
# smode (integer) - specifies what method will be used for finding small particles, ie stars; 0 is disabled,
# 1 uses RemoveGrain and 2 uses Deen (0; requires RemoveGrain/Repair/Deen)
# hot (boolean) - specifies whether removal of hot pixels should take place (false)
# fix (boolean) - fixes an aWarpSharp bug by overlaying a healthy pixel from the source clip;
# good idea to set to false when over-cropping afterwards (true)

function EdgeCleaner(clip c, float "strength", bool "rep", int "rmode", int "smode", bool "hot", bool "fix") {

strength = default(strength, 8.0)
rep = default(rep, true)
rmode = default(rmode, 17)
smode = default(smode, 0)
hot = default(hot, false)
fix = default(fix, true)

c = (c.isYV12()) ? c : c.ConvertToYV12()
strength = (smode==0) ? strength : strength+4

main = c.aWarpSharp(strength,1)
main = (rep) ? Repair(main,c,rmode) : main

mask = c.mt_edge("prewitt",4,32,4,32).mt_invert().mt_convolution()

final = (!hot) ? mt_merge(c,main,mask) : Repair(mt_merge(c,main,mask),c,2)
final = (fix) ? Overlay(final,c.ConvertToRGB24().Crop(0,1,-c.width+1,-c.height+2),x=0,y=1) : final
final = (smode != 0) ? mt_merge(final,c,c.StarMask(smode)) : final

return final

}

function StarMask(clip c, int "mode") {

mode = default(mode, 1)

clean = (mode==1) ? c.RemoveGrain(17) : Repair(c.Deen("a3d",4,12,0),c,15).RemoveGrain(21)
diff = (mode==1) ? mt_makediff(c,clean) : NOP

final = (mode==1) ? diff.Greyscale().Levels(40,0.350,168,0,255).removegrain(7,-1).mt_edge("prewitt",4,16,4,16) : \
Subtract(mt_merge(clean,c,c.mt_edge("roberts",0,2,0,2).mt_expand(mode=mt_circle(1)).mt_invert()),c).mt_edge("roberts",0,0,0,0).mt_deflate()

return final

}

# Changelog:
# 06/08/2008 v1.03
# - improved mask that leaves less warping and more original line structure, therefore higher strengths are now safe to use
# - improved StarMask()
# - removed super mode
# - removed srep, sshiqloc, some smodes and VD_SmartSmoothHiQ() due to StarMask() changes
# 01/06/2008 v1.02
# - added srep parameter
# - improved particle masking
# 01/06/2008 v1.01
# - added masking for particles with two parameters; smode and sshiqloc
# 12/05/2008 v1.00
# - removed line darkening, mode 2 mask, RemoveGrain
# - assert changed to colorspace conversion to yv12
# - fixed some logic problems
# - "fixed" the aWarpSharp black pixel bug
# - added Repair

^ That's been put together for anime, and never really tested on live action content, but see how it does (maybe just the mask would need adjusting, who knows).


P.S. If you need to sharpen, try doing it before.

[moviefan]

Try something like:

BlindDeHalo3(PPmode=-3)
EdgeCleaner(12,fix=false) #since you'll need to crop afterwards anyway

That looks pretty good! At least on the text... Regarding the red area, the halo is smooth ... Would you recommend to use the denoising (MC_Spuds) before or after halo removal?

Why don't you post your script, so we can help you with your settings?

There is nothing in the script except loading the video.

[martino]

Would you recommend to use the denoising (MC_Spuds) before or after halo removal?
After probably.

[Adub]

I tried those... The result is that the haloing is more blurred, but still there, bigger, but softer. Isn't there anything else I could do or what settings maybe should I use?

I was referring to the script you tried here. But it seems like martino's ideas are working for you.