BlindDeHalo() - removal of Edge Enhancement artefacts

[BlackMetal]

Originally posted by Manao
It's not a filter of the masktools, but of avisynth. What is your avisynth version ?

Yes, you were right. I had avisynth v2.5.2.0.
Now I tried with v2.5.5.0 and it worked.
Thanx! :)

[Socio]

Originally posted by Blue_MiSfit
Awesome script didee! Works extremely well on the film "Human Stain" that displays horrendous edge enhancement haloing.

BlindDeHalo(2,2,100)
Preserves sharpness much better than
UnFilter(-20,-20)

~misfit

Hey that gives me an idea;

Didee, what would happpen if you replace Unfilter in your IIP script with BlindDeHalo2?

[Didée]

Nothing spectacular would happen.

Remember that iiP offers two different methods for deringing: its "old" routine (called with positive "dering" values) - that's the one which is using "unfilter" -, and the "new method" (called with negative "dering" values), which most times works better in iiP's context.

BlindDH2() is effective against Halos, but doesn't do all too much about mosquito noise. The "new" deringing method of iiP is better suited as "general purpose" deringing filter, as it's designed to counteract both types of artefacting.

So ... if at all, then it's iiP's method that should be merged into BlindDeHalo(), not the other way round ;)


Drifting from topic, that's how LimitedSharpenEX() will be alike: It will clobber you over the head with ... possibilities :)

On the de-ringing/de-haloing side, you'll have a variety of choices - 2 or 3 main types of operation, each coming along with 2 or 3 sub-types of slightly different strengths or characteristics.
Then there will be an asymmetric sharpener available, especially for VHS processing, but perhaps also for aspect conversions 1:1 <--> anamorhic, back'n forth.
Not to mention that the LimitedSharpen principle will be extended by a detail crystality enhancer, an area adaptive contrast booster, and optional color enhancement. Also, some easy implementations of gauss/sharp-mixing will be in.
But there was something more, whatwasit, whatwasit ... ah yes, that little "noise-bypass-sharpening" functionality ... :)

As can be seen, a bunch of topics that arose in this forum in the last few weeks, have already been on my mind when I started the project last autumn ... "Didée, master of sluggish development" :(

(If only I'd find more time to work on it, again. The finishing state drags out like chewing gum, but I won't release some half-finished alpha-thingy of such complexity.)

[Macanudo]

Originally posted by Didée

Drifting from topic, that's how LimitedSharpenEX() will be alike: It will clobber you over the head with ... possibilities :)

On the de-ringing/de-haloing side, you'll have a variety of choices - 2 or 3 main types of operation, each coming along with 2 or 3 sub-types of slightly different strengths or characteristics.
Then there will be an asymmetric sharpener available, especially for VHS processing, but perhaps also for aspect conversions 1:1 <--> anamorhic, back'n forth.
Not to mention that the LimitedSharpen principle will be extended by a detail crystality enhancer, an area adaptive contrast booster, and optional color enhancement. Also, some easy implementations of gauss/sharp-mixing will be in.
But there was something more, whatwasit, whatwasit ... ah yes, that little "noise-bypass-sharpening" functionality ... :)
[/B]

Stop...You are making me drool...(insert picture of Homer Simpson staring at a box full of donuts)...LOL.

[Socio]

Originally posted by Macanudo
Stop...You are making me drool...(insert picture of Homer Simpson staring at a box full of donuts)...LOL.

I say lock Didee in a room til it's finished!

[dvwannab]

Originally posted by Didée
Nothing spectacular would happen.

Remember that iiP offers two different methods for deringing: its "old" routine (called with positive "dering" values) - that's the one which is using "unfilter" -, and the "new method" (called with negative "dering" values), which most times works better in iiP's context.

BlindDH2() is effective against Halos, but doesn't do all too much about mosquito noise. The "new" deringing method of iiP is better suited as "general purpose" deringing filter, as it's designed to counteract both types of artefacting.

So ... if at all, then it's iiP's method that should be merged into BlindDeHalo(), not the other way round ;)


Drifting from topic, that's how LimitedSharpenEX() will be alike: It will clobber you over the head with ... possibilities :)

On the de-ringing/de-haloing side, you'll have a variety of choices - 2 or 3 main types of operation, each coming along with 2 or 3 sub-types of slightly different strengths or characteristics.
Then there will be an asymmetric sharpener available, especially for VHS processing, but perhaps also for aspect conversions 1:1 <--> anamorhic, back'n forth.
Not to mention that the LimitedSharpen principle will be extended by a detail crystality enhancer, an area adaptive contrast booster, and optional color enhancement. Also, some easy implementations of gauss/sharp-mixing will be in.
But there was something more, whatwasit, whatwasit ... ah yes, that little "noise-bypass-sharpening" functionality ... :)

As can be seen, a bunch of topics that arose in this forum in the last few weeks, have already been on my mind when I started the project last autumn ... "Didée, master of sluggish development" :(

(If only I'd find more time to work on it, again. The finishing state drags out like chewing gum, but I won't release some half-finished alpha-thingy of such complexity.)

WOOOHOOOO baby!!!! Bring it on..........bring it on :D

[ATM]

Just wanna add that I love BlindDeHalo2 and can't wait for you to bring out LimitedSharpenEX.

These two shots are from Episode 10 of Trigun from the R2 DVDs

Source:

http://img222.exs.cx/img222/3231/source8lw.png

Filtered with BlindDeHalo2(2.5,2.5,160) & LimitedSharpen(ss_x=2.0,ss_y=2.0,Smode=3,strength=100):

http://img222.exs.cx/img222/1920/filtered8km.png

And the second example:

Source:

http://img222.exs.cx/img222/6365/source29ah.png

Filtered with BlindDeHalo2(2.5,2.5,160) & LimitedSharpen(ss_x=2.0,ss_y=2.0,Smode=3,strength=100):

http://img222.exs.cx/img222/1394/filtered21ck.png

[dvwannab]

ATM,

Nothing meant by my comments, but first I didnt see anything wrong with your source and the applied filters gave a washed-out look. Loss in contrast and saturation. Dont see how LS & Blind helped at all. Actually, they took away. May want to tweak your settings or just not do anything at all :D

[KaiserS]

Originally posted by dvwannab
ATM,

Nothing meant by my comments, but first I didnt see anything wrong with your source

You mean other then the strong EE haloes?

Originally posted by dvwannab
and the applied filters gave a washed-out look. Loss in contrast and saturation. Dont see how LS & Blind helped at all. Actually, they took away. May want to tweak your settings or just not do anything at all :D

They took away the EE and nothing else. Neither BlindDeHalo or LS have any effect on contrast or saturation, and those were the only filters applied. So I have no clue what you are referring to. The EE gives a perceived increase in contrast, but that is completely artificial (same way that it gives a perceived higher level of detail when in fact its just the opposite).

[KaiserS]

Here are the two histograms based on the 2nd screenshots.

*Removed pics see below*

There is a bit of difference in the luma channel but only at the outlying parts (due to the EE haloing being removed). In the color channels there is no real change at all. So I'm not sure how you can say there is washed out colors when the color channels haven't been changed to any noticeable extent.

[Backwoods]

I notice it too and argee with dvwannab.

[KaiserS]

And if you want it displayed out numerically here you go:

*Removed pics see below*

As this shows, there is no difference between the color channels at all. The only difference is in the luma channel, but like I said before this is only at the outlying part due to the EE removal.

[KaiserS]

Originally posted by Backwoods
I notice it too and argee with dvwannab.

Except as I've showed in 2 examples there is no change in the color channels, as BlindDeHalo and LS don't work on the chroma only luma, so this notion that the second is "washed-out" is completely untrue. The only difference between the pics is in the luma channel and only in the outlying parts where the EE was removed. I don't deny that he may percieve a difference, but in reality there isn't and the histograms and the output from ColorYUV show this.

I think you two may want to give this page: http://www.videophile.info/Guide_EE/Page_01.htm a look at.

[Backwoods]

After flipping the two images back and forth between each other, I see the differences more now. In the thread the source appears to be sharper and more vibrant of colours, but also it appears you did get rid of the EE at a cost of dimming the black lines.

[KaiserS]

Originally posted by Backwoods
In the thread the source appears to be sharper and more vibrant of colours, but also it appears you did get rid of the EE at a cost of dimming the black lines.

Of course it appears sharper thats what EE is used for but in reality it isn't really sharper, sharpness being defined by level of detail, it just gives the appearance of sharpness (look at the first example on the link I posted and it shows what I'm talking about). And yes, the thinning of the lines is a result of BlindDeHalo, but with something like FastLineDarken they can be darkened and thickened up. For me, I like the look of it with thinner lines then the thick ones, but that's just me. And since the color channels aren't being modified its impossible for one to look more vibrant then the other (if the chroma channels were being modified I could see where you are coming from but they are untouched).

If the source hadn't had any EE applied to it, it would have looked more like the 2nd pictures (minus the thinner lines).

[KaiserS]

Rather then working off the pngs provided by ATM, since I have to do a ConvertToYV12 to do the the analysis, and since I have the R2 DVDs as well so I'll just use my DVDs instead.

Source:

http://img195.exs.cx/img195/766/source9kt.png

Filtered with BlindDeHalo2(2.5,2.5,160) & LimitedSharpen(ss_x=2.0,ss_y=2.0,Smode=3,strength=100)
& FastLineDarken(strength=100):

http://img195.exs.cx/img195/73/source20xw.png

Notice how the color channels aren't being changed at all, as like I said before neither BlindDeHalo or LS work on the chroma channel, and the only difference in the loose min and max between the source and filtered is the upper end of the luma which is due to the bright halos being removed. So while you may think that the colors are different in one from the other they aren't otherwise there would be a difference between the color channels given by ColorYUV. :)

[Backwoods]

Yes I realize that after the first reply you mentioned. I stand corrected. In this case the EE worked, it fooled me into thinking it was sharper.

Your filtering does correct the image and the black lines seem to fit in more with the overall image but still looked bland. I've heard of FastLineDarken and was going to mention it in my post but I forgot the exact name of the filter (or is it script?). It really makes a difference to your final product. Whenever I watch anime and notice EE (didn't know it was EE) I figured it was how cells looked on the painted backgrounds while being photographed to film. Didn't know it was actually EE. With FastLineDarken, great results.

[KaiserS]

Originally posted by Backwoods
Yes I realize that after the first reply you mentioned. I stand corrected. In this case the EE worked, it fooled me into thinking it was sharper.

Yep and if you watch dvds like this on a normal interlaced set you might not even notice the EE. When watching these discs on a Hi-def 60" screen the haloing on these discs becomes quite noticeable and distracting. At some point when I get a dual layer burner I'm going to reauthor the discs using this filter chain which should make it a bit more pleasant to watch on the big screen tv. :)

[dvwannab]

thanks for the detailed explanation KaiserS. I definitely see what you are talking about. Thanks for the educational lesson. Much appreciated. ;)

I assume the EE-effect is also present in broadcast TV mpeg2 captures as well? No?

[Didée]

To my own surprise, I had some free time yesterday evening. So here is BDH's third incarnation:

BlindDeHalo3

Basically, it does the same as the 2nd version did. New are some tweaking possibilities (partly backported from BDH's initial version), an optional "postprocessing" mode, and ... better performance.

Feeding a cropped 720*432 DVD source, my Athlon XP1800 renders like this:

BDH2: ~10 fps
BDH3: ~25 fps (without PP)

I hope there are no objections against this minor performance increase :D

With PP activated, the performance gain is smaller: ~13 fps in above example. Also, I'm not yet really satisfied with the PP mode. But it may come handy in its current state already, and I don't want to keep that speedy version back from you, just because of that.

Let's start with parameters and explanations. The function comes at the end, as usual.

Full function call with default values:
BlindDeHalo3( rx=3.0, ry=3.0, strength=125,
\ lodamp=0.0, hidamp=0.0, sharpness=0.0, tweaker=0.0,
\ PPmode=0, PPlimit=0, interlaced=false)

Parameter description

rx, ry [float]
The radii to use for the [quasi-] gaussian blur, on which the halo removal is based..

strength
The overall strength of the halo removal effect.

lodamp, hidamp [float] (range: 0.0 ~ ??? / try 4.0 as a start)
With these two values, one can reduce the basic effect on areas that would change only little anyway (lodamp), and/or on areas that would change very much (hidamp).
lodamp does a reasonable job in keeping more detail in affected areas.
hidamp is intended to keep rather small areas that are very bright or very dark from getting processed too strong. Works OK on sources that contain only weak haloing - for sources with strong oversharpening, it should not be used, mostly.
(Usage has zero impact on speed.)

sharpness [float] (range: 0.0 ~ 1.58)
By setting this bigger than 0.0, the affected areas will come out with better sharpness. However, strength must be chosen somewhat bigger as well, then, to get the same effect than without.
(This is the same as initial version's "maskblur" option.)

tweaker [float] (range: 0.0 ~ 1.0)
May be used to get a stronger effect, seperately from altering "strenght".
(Also in accordance to initial version's working methodology. I had no better idea for naming this parameter.)

interlaced
As formerly, this is intended for sources that were originally interlaced, but then made progressive by deinterlacing. It aims in particular at clips that made their way through Restore24.

[b]PPmode [int]
When set to "1" or "2", a second cleaning operation after the basic halo removal is done. This deals for
a) removing/reducing those corona lines that sometimes are left over by BlindDeHalo
b) improving on mosquito noise, if some is present.

PPmode=1 uses a simple gaussian blur for post-cleaning. PPmode=2 uses a 3*3 average, with zero weigthing of the center pixel.

Also, PPmode can be "-1" or "-2". In this case, the main dehaloing step is completely discarded, and *only* the PP cleaning is done. This has less effect on halos, but can deal for sources containing more mosquito noise than halos.

PPlimit [int]
Can be used to make the PP routine change no pixel by more than [PPlimit].
I'm not sure if this makes much sense in this context. However the option is there - you never know what it might be good for.


Comments & tips

Well, I just fiddled it together and checked that everything is working ... there's not very much practical experience yet I can share with you, about the new stuff. However:

- All that PP stuff isn't yet optimized. But it's difficult to make that part noticeably smarter without an aching slowdown, I fear. Try it, share your opinions. Make suggestions.

- Regarding "strength" & "sharpness" & "tweaker"
These three behave somewhat [i]similar, but not the same.
A bigger value for "sharpness" is similar to reducing "strength". A bigger value for "tweaker" is similar to increasing "strength" value -- and vice versa for both.
Also the other way round: when increasing "sharpness", you probably have to increase "strength" a little as well, to get the same removal effect.
Usage of "sharpness" works in particular good when bigger radii (say, >2.5) are used for dehaloing. Usage of "tweaker" is more useful when working with sources containing thin, weak halos.
In the end, it comes down again to try&error ;)

- lodamp and hidamp
Technically, these two are the "n" constants in two scaling factors of the form "x/(x+n)", which are used when creating a LUT from the difference between input clip and its gaussian blurred version.

As described above, "lodamp" deals for better protection of areas where this difference is alredy small. Effectively, it preserves some more of weak detail in areas of high contrast. A value of 4.0 seems a good starting point. One can also try to use bigger values like 16.0 or so, along with a bigger "strength" value. By this, one can somewhat change the characteristics of "what-is-processed-how-strongly" (if I can say so).

"hidamp" tries to do the opposite: protect small detail that is very bright or very dark from being toned down too much. But its usability is much more restricted: it seems to work OK when the actual halos of the source are rather weak, and/or when not too small radii are used - in this case, the protection of very prominent detail works out, mostly.
In case the source contains strong haloing, is's better to not use "hidamp" at all: in this case, the halos itself would be considerd to be "prominent detail" and be protected ... no go.


I think this is close to a "final" version of BlindDeHalo (without a number in the name ;) )


The function script is attached to this post, and will show up upon approval. For reference, the script is posted below.

Have fun!

[Didée]

Here's the script:
function BlindDeHalo3( clip clp, float "rx", float "ry", int "strength",
\ float "lodamp", float "hidamp", float "sharpness", float "tweaker",
\ int "PPmode", int "PPlimit", bool "interlaced")
{
rx = default( rx, 3.0 )
ry = default( ry, 3.0 )
strength = default( strength, 125 )
lodamp = default( lodamp, 0.0 )
hidamp = default( hidamp, 0.0 )
sharpness = default( sharpness, 0.0 )
tweaker = default( tweaker, 0.0 )
PPmode = default( PPmode, 0 )
PPlimit = default( PPlimit, 0 )
interlaced = default( interlaced, false )

sharpness = (sharpness>1.58) ? 1.58 : sharpness
tweaker = (tweaker > 1.00) ? 1.00 : tweaker
strength = float(strength)*(1.0+sharpness*0.25)
RR = string((rx+ry)/2.0)
ST = string(float(strength)/100.0)
LD = string(lodamp)
HD = string(pow(hidamp,2))
TWK0 = "x y - 12 "+ST+" / "+RR+" / /"
TWK = "x y - 12 "+ST+" / "+RR+" / / abs"
TWK_HLIGHT = "x y = 128 "+TWK+" 128 "+TWK+" - "+TWK+" 128 / * + "+TWK0+" "+TWK+" "+LD+" + / * "
\ + "128 "+TWK+" - 20 / 2 ^ 128 "+TWK+" - 20 / 2 ^ "+HD+" + / * 128 + ?"

i = (interlaced==false) ? clp : clp.separatefields()
oxi = i.width
oyi = i.height
sm = i.bicubicresize(m4(oxi/rx),m4(oyi/ry))
mm = yv12lutxy(sm.expand,sm.inpand,"x y - 4 *").expand.deflate.blur(1.58).bicubicresize(oxi,oyi,1.0,.0)
sm = sm.bicubicresize(oxi,oyi,1.0,.0)
smd = yv12lutxy(i.sharpen(tweaker),sm,TWK_HLIGHT)
smd = (sharpness==0.0) ? smd : smd.blur(sharpness)
clean = yv12lutxy(i,smd,yexpr="x y 128 - -").mergechroma(i)
clean = maskedmerge(i,clean,mm)

LL = string(PPlimit)
LIM = "x "+LL+" + y < x "+LL+" + x "+LL+" - y > x "+LL+" - y ? ?"

small = (PPmode>0) ? clean.bicubicresize(m4(oxi/2.125),m4(oyi/2.125))
\ : i .bicubicresize(m4(oxi/2.125),m4(oyi/2.125))
ex1 = small.expand.blur(.5)
in1 = small.inpand.blur(.5)
hull = logic( yv12lutxy( ex1.expand.blur(1), ex1, "x y - 1 1 / * 1.0 ^ 1 - 4 *" )
\ ,yv12lutxy( in1, in1.inpand.blur(1), "x y - 1 1 / * 1.0 ^ 1 - 4 *" )
\ ,"max", U=-128, V=-128)
\ .bicubicresize(oxi,oyi,1.0,.0)
postclean = (PPmode== 1) ? maskedmerge(clean,small.bicubicresize(oxi,oyi,1.0,.0),hull)
\ : (PPmode== 2) ? maskedmerge(clean,clean.DEdgeMask(0,255,0,255,"2 3 2 3 0 3 2 3 2"),hull)
\ : (PPmode==-1) ? maskedmerge(i,small.bicubicresize(oxi,oyi,1.0,.0),hull)
\ : (PPmode==-2) ? maskedmerge(i,i.DEdgeMask(0,255,0,255,"2 3 2 3 0 3 2 3 2"),hull)
\ : clean
postclean = (PPlimit==0) ? postclean
\ : yv12lutxy(clean,postclean,LIM,U=2,V=2)

(PPmode==0) ? clean : postclean
interlaced ? weave() : last

return( last )
}
#---------------------------------------------------------
function m4(float x) {return(x<16?16:int(round(x/4.0))*4)}

[COREiP]

I'm getting an error:

Script error: there is no function named "yv12lutxy"
(BlindDeHalo3.avs, line 32)

[Didée]

Could be your PC being ill. Try some medicine (http://manao4.free.fr/MaskTools-v1.5.6.zip).

[ArabianSwan]

Originally posted by COREiP
I'm getting an error:

Script error: there is no function named "yv12lutxy"
(BlindDeHalo3.avs, line 32)


you need the new masktool ..

[COREiP]

I'm using MaskTools 1.4.16 is there a newer version?

[Backwoods]

Originally posted by COREiP
I'm using MaskTools 1.4.16 is there a newer version?

2 posts above your's.

[COREiP]

lol thnx guys. Keep up the good work Didée the script is great :) :)

[Mr. Brown]

could anyone make a masktools 2 version of BDH3 thanks

[Chainmax]

You should try DeHalo_Alpha, which is made by Didée too. It's miles ahead of BDH3, and there's a masktools v2.x version.

[Mr. Brown]

You should try DeHalo_Alpha, which is made by Didée too. It's miles ahead of BDH3, and there's a masktools v2.x version.
i normally use dehalo_alpha (mt2 version)
but in some cases i need bdh3 for postprocessing

example:
DeGrainMedian(limitY=5,limitUV=5,mode=3)

den = last
dh = Dehalo_Alpha(rx=3.0,ry=3.0).BlindDehalo3(PPmode=-1,PPlimit=4)
edges = dh.removegrain(12,-1).prewitt(multiplier=2.5)
\ .mt_expand().mt_inflate().mt_inflate().removegrain(12,-1).mt_expand()

mt_merge(den,dh,edges)

that's why i a faster BDH3 would help me

[Chainmax]

In my opinion, BDH3 oversmooths the picture and kills small details, even when only using its PP mode. If DeHalo_Alpha doesn't solve it, the only semi-viable option I can think of is HQDering, of which a Masktools v2.x version was made a short while ago, but it might cause jaggyness.

[Didée]

could anyone make a masktools 2 version of BDH3 thanks
Here: BlindDehalo3_MT2 (http://home.arcor.de/dhanselmann/_stuff/BlindDeHalo3_MT2.zip)

- changed for MaskTools v2 (mt_masktools.dll required)
- replaced some things with RemoveGrain-modes (RemoveGrain.dll newer than v0.9 required)
- fixed some oddities
- new PPmode (3 and -3): shouldn't blur, but still remove mosquitos and leftover halo borders

Could need some more testing, was done in a hurry. ;)

[Mr. Brown]

@ Didée
thanks for updating bdh3()
i made a test with the dehaloing that you suggested me in german doom9 forum and the new ppmode's looks very good (bye bye halos and come never back even with seesaw :D)

[wOxxOm]

I need an advice on how to remove very grossly offset halos on Nicolodeon's Avatar movie DVD like these pictures 2,3,4 here: http://forum.doom9.org/showthread.php?p=1084550#post1084550 (quite large frame span!)

It seems that I have to tweak some parameter *inside* BHD, which would that be, please?

[gigantibyte]

Thank you very much for BlindDehalo3_MT2! I've heard many great things about the great Didée. I see why now. The function worked wonders on my video source.

[PVDHP]

Nothing not working for me. I have installed masktools, i gettin this message: http://dl.dropbox.com/u/60527527/Err1.jpg

[Didée]

Welcome onboard. It seems you're not deeply experienced with Avisynth? - Because you're using the script totally wrong. ;)

You save the BlindDeHalo script to your HD, as e.g. "BlindDeHalo2.avsi". (Un-modified! The script starts with "function BlindDeHalo2( clip clp, float "rx", float "ry", ...")

Then, to use it in your actual Avisynth script, you would go like

Import("X:\path\to\BlindDeHalo2.avsi")

AviSource("BR.avi")

BlindDeHalo2( 3,3,125 )

And BTW, the latest version is BlindDeHalo3, not ~2. Look a few lines above, post #82.

[PVDHP]

Hi, thank you a lot for fast answer :)

[PVDHP]

Oh, yeah! Success!

Import("E:\VirtualDub\BlindDeHalo3_mt2.avsi")

AviSource("BR7.avi")
ConvertToYV12()

BlindDeHalo3( 3,3,125 )

Just i dunno :o