Removing DeHalo_Alpha artifacts

pcroland · 24th November 2017, 14:09

Hi!

I'm trying to restore an old DVD which has a lot of grain, dct artifacts and a really strong halo. This is my current avs script:

Code:

DirectShowSource("D:\raccoons\DVD1\VIDEO_TS\VTS_01_1.VOB")
QTGMC(Preset="Slow")
SelectOdd()
Crop(14,2,-16,-0)
FillMargins(0,0,0,1)
bbmod(0,2,8,8)
knlmeanscl(D=8, A=5, h=3, device_type="GPU")
DeHalo_Alpha(rx=3.0, ry=3.0, darkstr=0.4, brightstr=1, lowsens=50, highsens=50, ss=1.5)
GradFun3(thr=1.5, radius=6, mask=2, mode=0, smode=0, debug=0, lsb=False, lsb_in=False, staticnoise=False, y=3, u=3, v=3)

I'm satisfied with the grain removal (but you can recommend something better instead of knlmeanscl), the GradFun3() is maybe a little bit too strong and needs some tweaking. But the dehalo is just not that great so far. The strong white halo is now replaced with a slighty darker color than it should be. brightstr is 1 as you can see in the code, 0.8 is the recommended value for anime and cartoon but it seems that it does not do anything if I try something lower than 1. Could you guys recommend something to fix this? Or maybe another dehalo filter? I tried HaloBuster() but I couldn't get it to work.

Here's the comparison: https://testfra.me/0s/

SaurusX · 27th November 2017, 15:18

Your KNLMeansCL h strength is set way too high, not to mention your D's and A's are way higher than what this animation really needs. You're killing detail.

And DeHalo_Alpha does work with settings lower than 1, so I'm not sure where you get that it doesn't. I'd recommend turning off the darkstr entirely (darkstr=0) since that will diminish the true cel animation outlines. The rx and ry values also seem greater than what they need to be. Do something like this:

dehalo_alpha(darkstr=0, brightstr=.1,ss=2)
dehalo_alpha(darkstr=0, brightstr=.3,ss=2)
dehalo_alpha(darkstr=0, brightstr=.5,ss=2)
dehalohmod(smooth=true) #cleans remaining edge noise and dark halos

And then see where you end up. You'll have to adjust the brightstr values to suit your source, but follow the increasing strength scheme.

24th November 2017, 14:09	#1 \| Link
pcroland Registered User Join Date: Mar 2014 Location: Hungary Posts: 108	Removing DeHalo_Alpha artifacts Hi! I'm trying to restore an old DVD which has a lot of grain, dct artifacts and a really strong halo. This is my current avs script: Code: DirectShowSource("D:\raccoons\DVD1\VIDEO_TS\VTS_01_1.VOB") QTGMC(Preset="Slow") SelectOdd() Crop(14,2,-16,-0) FillMargins(0,0,0,1) bbmod(0,2,8,8) knlmeanscl(D=8, A=5, h=3, device_type="GPU") DeHalo_Alpha(rx=3.0, ry=3.0, darkstr=0.4, brightstr=1, lowsens=50, highsens=50, ss=1.5) GradFun3(thr=1.5, radius=6, mask=2, mode=0, smode=0, debug=0, lsb=False, lsb_in=False, staticnoise=False, y=3, u=3, v=3) I'm satisfied with the grain removal (but you can recommend something better instead of knlmeanscl), the GradFun3() is maybe a little bit too strong and needs some tweaking. But the dehalo is just not that great so far. The strong white halo is now replaced with a slighty darker color than it should be. brightstr is 1 as you can see in the code, 0.8 is the recommended value for anime and cartoon but it seems that it does not do anything if I try something lower than 1. Could you guys recommend something to fix this? Or maybe another dehalo filter? I tried HaloBuster() but I couldn't get it to work. Here's the comparison: https://testfra.me/0s/

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

27th November 2017, 15:18	#2 \| Link
SaurusX Registered User Join Date: Feb 2017 Posts: 124	Your KNLMeansCL h strength is set way too high, not to mention your D's and A's are way higher than what this animation really needs. You're killing detail. And DeHalo_Alpha does work with settings lower than 1, so I'm not sure where you get that it doesn't. I'd recommend turning off the darkstr entirely (darkstr=0) since that will diminish the true cel animation outlines. The rx and ry values also seem greater than what they need to be. Do something like this: dehalo_alpha(darkstr=0, brightstr=.1,ss=2) dehalo_alpha(darkstr=0, brightstr=.3,ss=2) dehalo_alpha(darkstr=0, brightstr=.5,ss=2) dehalohmod(smooth=true) #cleans remaining edge noise and dark halos And then see where you end up. You'll have to adjust the brightstr values to suit your source, but follow the increasing strength scheme.