halobuster - high quality halo remover

[feisty2]

the filter is very high quality and almost harmless to details
cuz, it picks NLMeans, an advanced denoising algorithm instead of stuff like, blurring, minblur, destructive anyways as the dehalo kernel

Function HaloBuster (clip input, bool "lsb", int "a", float "h", float "thr", float "elast")
{
lsb = Default(lsb, False)
a= Default (a, 32)
h = Default(h, 6.4)
thr = Default(thr, 1.0)
elast = Default(elast, 1.5)
gray = lsb ? input.converttoy8 () : input.converttoy8 ().Dither_convert_8_to_16()
gray = stackvertical (dither_get_msb (gray).padding (a, a, a, a),dither_get_lsb (gray).padding (a, a, a, a))
clean = knlmeanscl (gray, d=0, a=a, s=0, h=h, lsb_inout=True)
mask = TCannyMod (converttoyv12 (clean.ditherpost (mode=-1)), sigma=1.5, mode=1, gmmax=50)
mask = mask.converttoy8 ()
mask = mt_lut (mask, "x 255 / 0.24 - 3.2 * 0.0 max 1.0 min 255 *")
mask = mt_expand (mask)
mask = mt_inflate (mask)
merge = dither_merge16_8 (gray, clean, mask)
limit = Dither_limit_dif16 (gray, merge, thr=thr, elast=elast)
crop = Dither_crop16 (limit, a, a, -a, -a)
luma = lsb ? crop : crop.ditherpost (mode=-1)
final = input.IsY8 () ? luma : ytouv (input.utoy8 (), input.vtoy8 (), luma)
return final
}

Function Padding (clip input, int "left", int "top", int "right", int "bottom")
{
w = input.width ()
h = input.height ()

output = input.PointResize (w+left+right, h+top+bottom, -left, -top, w+left+right, h+top+bottom)

return output
}


parameters:
lsb --- clip is stacked 16bits or not (8bits)
h --- filtering strength
thr --- threshold, 0-255 scale, return pixel from the source clip if the difference < thr, otherwise, return pixel from the processed clip
elast --- softness of the threshold

only luma will be processed, chroma will be copied from the source clip

demo:
before
http://i.imgur.com/PhBdNg5.png
http://i.imgur.com/9x0aAKe.png
http://i.imgur.com/J2Yyi0s.png
http://i.imgur.com/Y868WWg.png

after

HaloBuster ()

http://i.imgur.com/HYzOoMm.png
http://i.imgur.com/fprHnJE.png
http://i.imgur.com/7EZvbJU.png
http://i.imgur.com/hwUin1O.png

[Groucho2004]

Had a quick look, nice.

1.2 frames/second on SD material (i5 2500K @4GHz) - yikes.

[feisty2]

Had a quick look, nice.

1.2 frames/second on SD material (i5 2500K @4GHz) - yikes.

I don't really give a shoot about performance
I care about quality, my one and only goal

[Groucho2004]

I don't really give a shoot about performance
Yeah, you're young, lots of time left. We old geezers need things done faster or we'll be pushing up daisies by the time the job is done. :D

[Reel.Deel]

I know this won't increase performance by much but you could use TCannyMod (http://forum.doom9.org/showthread.php?t=168449) over TCanny. Also how about making padding optional? If the source has black borders (just like your sample) it's useless to add padding since you're essentially padding black borders.

I have some DVDs from the early 2000's, they have massive halos from extreme EE. I wonder how HaloBuster will handle such halos?

Edit: I didn't know mastertapes (http://forum.doom9.org/showpost.php?p=1737496&postcount=29) had halos :rolleyes:.

Yeah, you're young, lots of time left. We old geezers need things done faster or we'll be pushing up daisies by the time the job is done. :D

A few years ago I had the same mentality, fortunately I've gotten most of my priorities straight since then, plus I pay the light bill :D.

[creaothceann]

demo:
http://i.imgur.com/Q54CMHY.gif http://i.imgur.com/uJcmR8N.gif http://i.imgur.com/MZTKOnu.gif http://i.imgur.com/XtblNcJ.gif

ftfy

[feisty2]

I know this won't increase performance by much but you could use TCannyMod (http://forum.doom9.org/showthread.php?t=168449) over TCanny. Also how about making padding optional? If the source has black borders (just like your sample) it's useless to add padding since you're essentially padding black borders.

I have some DVDs from the early 2000's, they have massive halos from extreme EE. I wonder how HaloBuster will handle such halos?

Edit: I didn't know mastertapes (http://forum.doom9.org/showpost.php?p=1737496&postcount=29) had halos :rolleyes:.



A few years ago I had the same mentality, fortunately I've gotten most of my priorities straight since then, plus I pay the light bill :D.

1. not sure why, but, tcannymod gives me quite different result from tcanny, guess that's something related to "gmmax" (this parameter from tcanny just, vanished in tcannymod), so please stick to tcanny
and about padding, maybe you should crop the clip first, didn't crop my test clip cuz I'm sure everyone knows I'm lazy like hell
2. maybe it will work, just did some insane tests to see how it performs in extreme cases
so I just sharpened the already full of halo clip, now it's... no comment

sharpen (1)

http://i.imgur.com/6wBDI5S.png

sharpen (1)
HaloBuster (h=32)

http://i.imgur.com/Y9UUYvG.png
guess it works, you'll need a very large "h"
3.I "make" mastertapes, that means I distribute the "final results" without ANY sacrifice to quality, but the material clips might be... not so nice

edit: I think I just typed "materiel"...

[feisty2]

ftfy

thx, man!
u no gurl, ain't ya

[johnmeyer]

This works extraordinarily well with the subtle halos in your original. It will be interesting to see if it can be tweaked to remove some of the really huge halos that happen with oversharpening of old SD material, like VHS. I look forward to playing around with it.

[Dreamland]

This is my first post..
ciao from Italy :)
very good interesting thread :goodpost:
excuse me for my bad english ..i'm trying to import avs halo remover but there are problems:(

win8.1 32bit Avisinth 2.6 final

http://t6.pixhost.org/thumbs/3715/29378694_1.jpg (http://www.pixhost.org/show/3715/29378694_1.jpg)

http://t6.pixhost.org/thumbs/3715/29378697_2.jpg (http://www.pixhost.org/show/3715/29378697_2.jpg)

http://t6.pixhost.org/thumbs/3715/29378701_3.jpg (http://www.pixhost.org/show/3715/29378701_3.jpg)

I've installed Visual C++ Redistributable Package for Visual Studio 2015 (x86) in my system , but doesnt' work!

Help me !!

thx a lot!

[Bloax]

Had a quick look, nice.

1.2 frames/second on SD material (i5 2500K @4GHz) - yikes.
The biggest performance bottleneck is this:
knlmeanscl (gray, d=0, a=32, s=0, h=h, lsb_inout=True)
'a' values over 15 are basically unusable for non-stillimage purposes due to the enormous calculation complexity put on the GPU. (Not the CPU!)
Consequently the performance could be greatly improved by lowering this value, likely at the slight cost of quality.

Function HaloBuster (clip input, bool "lsb", float "h", float "thr", float "elast", int 'a')
{
lsb = Default(lsb, False)
avalue = Default(a, 32)
h = Default(h, 6.4)
thr = Default(thr, 1.0)
elast = Default(elast, 1.5)
gray = lsb ? input.converttoy8 () : input.converttoy8 ().Dither_convert_8_to_16()
gray = stackvertical (dither_get_msb (gray).padding (32, 32, 32, 32),dither_get_lsb (gray).padding (32, 32, 32, 32))
clean = knlmeanscl (gray, d=0, a=avalue, s=0, h=h, lsb_inout=True)
mask = TCanny (converttoyv12 (clean.ditherpost (mode=-1)), sigma=1.5, mode=1)
mask = mask.converttoy8 ()
mask = mt_lut (mask, "x 255 / 0.24 - 3.2 * 0.0 max 1.0 min 255 *")
mask = mt_expand (mask)
mask = mt_inflate (mask)
merge = dither_merge16_8 (gray, clean, mask)
limit = Dither_limit_dif16 (gray, merge, thr=thr, elast=elast)
crop = Dither_crop16 (limit, 32, 32, -32, -32)
luma = lsb ? crop : crop.ditherpost (mode=-1)
final = ytouv (input.utoy8 (), input.vtoy8 (), luma)
return final
}

Function Padding (clip input, int "left", int "top", int "right", int "bottom")
{
w = input.width ()
h = input.height ()

output = input.PointResize (w+left+right, h+top+bottom, -left, -top, w+left+right, h+top+bottom)

return output
}

To translate that to human speak, just run HaloBuster(blah,blah,a=18) to make performance less glacial at some kind of hit to quality. Even lower might work.
Because let's not forget that we're talking about the same person who absolutely insists on stupendous amounts of colorspace precision (int32 because int16 isn't enough!!) for something that is so extremely destructive in its nature that it would not benefit from it at all (MVTools).

[Groucho2004]

I've installed Visual C++ Redistributable Package for Visual Studio 2015 (x86) in my system , but doesnt' work!

Help me !!

thx a lot!
You need Visual C++ Redistributable Package for Visual Studio 2013 for this plugin.

[creaothceann]

It will be interesting to see if it can be tweaked to remove some of the really huge halos that happen with oversharpening of old SD material, like VHS.
Like this (https://www.youtube.com/watch?v=LUTCTh4ysNo)? :]

[Dreamland]

You need Visual C++ Redistributable Package for Visual Studio 2013 for this plugin.

:thanks:
now works!!:) thx a lot!

[*.mp4 guy]

...but why aren't you removing all of the halos? :confused:
http://i.imgur.com/P4Es8RS.png
http://i.imgur.com/4weN8zV.png
http://i.imgur.com/rHTzBcX.png
http://i.imgur.com/sRX51rq.png
http://i.imgur.com/kTVNWYI.png

[feisty2]

I'm all ears, any more specific suggestions?

[Reel.Deel]

1. not sure why, but, tcannymod gives me quite different result from tcanny, guess that's something related to "gmmax" (this parameter from tcanny just, vanished in tcannymod), so please stick to tcanny


Updated TCannyMod (https://github.com/chikuzen/TCannyMod/releases) with "gmmax" parameter. Thanks Chikuzen! :)

[feisty2]

updated

[Reel.Deel]

@feisty2

Change the following to support Y8 input:
final = input.IsY8() ? luma : ytouv (input.utoy8 (), input.vtoy8 (), luma)

[feisty2]

done.

[Dreamland]

I can't to remove Halos and to preserve quality video :(

this is a pic :

http://t6.pixhost.org/thumbs/3868/29548305_2015-09-23_110353.jpg (http://www.pixhost.org/show/3868/29548305_2015-09-23_110353.jpg)

Halobuster don't remove all the halos > Halobuster()

I use Dehalo_alpha also, but is not so good to preserve quality video

this is a simple video: https://mega.nz/#!UFF2TLDY!CFzfxDiBmpRNRZuCj_LlNLFMyQo4dFunLeASzmbzF0A

Feisty2, please help me!! :thanks:

[feisty2]

Those are ringings (something from frequency domain filtering)
You'll need another tool to remove these

[Dreamland]

Those are ringings (something from frequency domain filtering)
You'll need another tool to remove these
can you give me an example? :thanks:

[feisty2]

See placebo in vaporsynth forum

[~ VEGETA ~]

can you show some comparison between this filter and dehalo_alpha and other filters? like, speed and quality.

I like the idea of caring for quality more than performance (most of the times I can wait forever), but I think you must put options to give performance boost over quality just like the speed of other filters.

I'd like to see comparison images to show effectiveness of this filter, to know if we should bother to wait or not.

thanks for this effort! Happy Eid!

[Mounir]

Which version of masktools do i need to run this filter ?
Right now i get en error: " division by zero at 0x0002f1ed" with masktools v2.0a36 MT 26
i tried with an old version v1.5.8, same thing

[SSH4]

can you show some comparison between this filter and dehalo_alpha and other filters? like, speed and quality.

Well. Interesting result.

HaloBuster() better work on frames with large flat areas.
dehalo_alpha_mt mod with yahr mod (common for me combination for remastering dvd anime) produce a little bit better result (may be not fine tuned for this example, but...)

http://thumbnails113.imagebam.com/46571/0a35e2465709962.jpg (http://www.imagebam.com/image/0a35e2465709962) http://thumbnails105.imagebam.com/46571/d93f7b465709953.jpg (http://www.imagebam.com/image/d93f7b465709953) http://thumbnails105.imagebam.com/46571/ed59d1465709958.jpg (http://www.imagebam.com/image/ed59d1465709958)
http://thumbnails113.imagebam.com/46571/b6611b465709977.jpg (http://www.imagebam.com/image/b6611b465709977)http://thumbnails113.imagebam.com/46571/5cffa0465709967.jpg (http://www.imagebam.com/image/5cffa0465709967) http://thumbnails105.imagebam.com/46571/f98f9b465709972.jpg (http://www.imagebam.com/image/f98f9b465709972)
http://thumbnails105.imagebam.com/46571/0a5e4e465709986.jpg (http://www.imagebam.com/image/0a5e4e465709986) http://thumbnails105.imagebam.com/46571/19bdf3465709979.jpg (http://www.imagebam.com/image/19bdf3465709979)http://thumbnails113.imagebam.com/46571/3e8bb1465709983.jpg (http://www.imagebam.com/image/3e8bb1465709983)
http://thumbnails113.imagebam.com/46571/b98761465709994.jpg (http://www.imagebam.com/image/b98761465709994) http://thumbnails113.imagebam.com/46571/7f3d2c465709991.jpg (http://www.imagebam.com/image/7f3d2c465709991) http://thumbnails105.imagebam.com/46571/3eb5dd465709993.jpg (http://www.imagebam.com/image/3eb5dd465709993)

Actually HaloBuster also need mask protection for source or it can wash out not only halo.
But anyway, interesting script!

[bxyhxyh]

I couldn't find thread "placebo" in Vapoursynth sub-forum.
Where is this?

[StainlessS]

Placebo used in this thread:- http://forum.doom9.org/showthread.php?p=1751614#post1751614

EDIT: But the link to Placebo.py is 404 (not found).

[StainlessS]

This post here suggests Placebo is now called "fulltonative" and toolkit containing it linked to here:- http://forum.doom9.org/showthread.php?p=1751022#post1751022

On GitHub as BMToolkit.py, here:- https://github.com/IFeelBloated/BlockMatchingToolkit/blob/master/BMToolkit.py

[feisty2]

This post here suggests Placebo is now called "fulltonative" and toolkit containing it linked to here:- http://forum.doom9.org/showthread.php?p=1751022#post1751022

On GitHub as BMToolkit.py, here:- https://github.com/IFeelBloated/BlockMatchingToolkit/blob/master/BMToolkit.py

don't use it (or use it at your own risk)
it's still, you know, at a very early stage, like, pre-alpha test..

[lisztfr9]

Hi,

If NLmeans refer to Non Local means, how is it working ? The explanation found in wikipedia is confusing, moreover i'm in serious troubles figuring out how an non local algorithm can be used for denoising. Because you have to find a patch somewhere, according to wkp..

Thanks, L

[ChaosKing]

@feisty2 any chance for a HaloBuster vapoursynth port?

[tormento]

Would be possible to have a AVS+ YV24 supported version?

I have changed every ConvertToY8 to ConvertToY, lsb_inout to stacked in KNLMeansCL call but it gives me the error that INT32 is not supported.

My input is YV24 format.

Thank you!

[real.finder]

Would be possible to have a AVS+ YV24 supported version?

I have changed every ConvertToY8 to ConvertToY, lsb_inout to stacked in KNLMeansCL call but it gives me the error that INT32 is not supported.

My input is YV24 format.

Thank you!

https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.6%20and%20up/HaloBuster.avsi

[tormento]

https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.6%20and%20up/HaloBuster.avsi
Thank you.

[Selur]

I there a Vapoursynth port for this?

[takla]

The woman is still ringing, but I guess it indeed does preserve details.

[aburgesser]

I love the effectiveness of halobuster, but the runtime has always been painful. When I tried busting it out on a BD source it stopped being feasible so I started digging.

I found that knlmeanscl defaults to the 0 index device of the best type it finds. As a consequence, if your system has multiple GPUs, it can run on a weaker GPU. On my normal transcoding rig, that's the difference between an old intel iGPU and a 1070! Exposing and controlling the device_id can make for a significant (more than 10x) speedup!

Here is the edit I made. You may want to play with the device_id default value to find the correct one for your system. It does seem to align with task manager IDs. device_id=0 and device_type="auto" restores prior behavior.

Function HaloBuster (clip input, bool "lsb", int "a", float "h", float "thr", float "elast", string "device_type", int "device_id")
{
lsb = Default(lsb, False) #clip is stacked 16bits or not (8bits)
a= Default (a, 32)
h = Default(h, 6.4) #filtering strength
thr = Default(thr, 1.0) #threshold, 0-255 scale, return pixel from the source clip if the difference < thr, otherwise, return pixel from the processed clip
elast = Default(elast, 1.5) #softness of the threshold
device_type = Default(device_type,"gpu")
device_id = Default(device_id,1)
gray = lsb ? input.converttoy8 () : input.converttoy8 ().Dither_convert_8_to_16()
gray = stackvertical (dither_get_msb (gray).padding (a, a, a, a),dither_get_lsb (gray).padding (a, a, a, a))
clean = knlmeanscl (gray, d=0, a=a, s=0, h=h, device_type=device_type , device_id=device_id)
mask = TCannyMod (converttoyv12 (clean.ditherpost (mode=-1)), sigma=1.5, mode=1, gmmax=50)
mask = mask.converttoy8 ()
mask = mt_lut (mask, "x 255 / 0.24 - 3.2 * 0.0 max 1.0 min 255 *")
mask = mt_expand (mask)
mask = mt_inflate (mask)
merge = dither_merge16_8 (gray, clean, mask)
limit = Dither_limit_dif16 (gray, merge, thr=thr, elast=elast)
crop = Dither_crop16 (limit, a, a, -a, -a)
luma = lsb ? crop : crop.ditherpost (mode=-1)
final = input.IsY8 () ? luma : ytouv (input.utoy8 (), input.vtoy8 (), luma)
return final
}

Function Padding (clip input, int "left", int "top", int "right", int "bottom")
{
w = input.width ()
h = input.height ()

output = input.PointResize (w+left+right, h+top+bottom, -left, -top, w+left+right, h+top+bottom)

return output
}


It's still painful at the default radius (a=32). But I can process a BD movie where this is the star filter with a=16 in about 24 hours as long as VirtualDub doesn't panic. Before I gave up when the expected time was over a week and still had yet to start work on more complex frames than intro logos!

[tormento]

lsb = Default(lsb, False) #clip is stacked 16bits or not (8bits)
Is it aware of AVS+ capabilities?

Stacked workaround for HBD is an ancient think.

Extension to full HBD and spaces supported by AVS+ would be nice.