New halo removers discussion

Originally Posted by Chainmax

It's pretty nasty, but it is isolated too . It appears mostly on Vash's costume and a few rare times in sky scenes as well. I can upload a vob sample with the intro if you want to check it out.

Okay, I'm just curious to see how bad it really is. Though I think between the blocking and the rainbowing, that the rainbowing would probably more distracting from the screenshots I've seen.

[Chainmax]

Now that I look at the intro frame with the last filterchain, the white letters do look duller and yellowish. I'll try lowering DDeHalo's (the name I invented for abcxyz) radius to see if that makes them look better.

Originally Posted by Chainmax

Now that I look at the intro frame with the last filterchain, the white letters do look duller and yellowish. I'll try lowering DDeHalo's (the name I invented for abcxyz) radius to see if that makes them look better.

Yeah, I've noticed that lowering it to between 1.75 or 2.00 with HQDering picking up the slack looks better. Doesn't make the bright spots as greyish, and still does a good job of getting rid of the halos. Especially noticeable in the Castle of Cagliostro shots if you look at the chandelier.

[Chainmax]

Castle of Cogliostro?

Is that where the screens you posted in the 2nd page come from?

Originally Posted by Chainmax

Castle of Cogliostro?

Is that where the screens you posted in the 2nd page come from?

Yeah. Are you saying you've never heard of Castle of Cagliostro?

[Chainmax]

I am merely an ignorant newbie in the anime world. So far I've only seen (and sometimes not all of) DB, DBZ, Grave of the fireflies, Captain Tsubasa, Trigun, Saikano, Evangelion, Astroboy (80s methinks), Rurouni Kenshin, Spirited Away, Saint Seiya and a few others I can't remember right now. ++

Originally Posted by Chainmax

I am merely an ignorant newbie in the anime world. So far I've only seen (and sometimes not all of) DB, DBZ, Grave of the fireflies, Captain Tsubasa, Trigun, Saikano, Evangelion, Astroboy (80s methinks), Rurouni Kenshin, Spirited Away, Saint Seiya and a few others I can't remember right now. ++

Ahh you should definitely see it. It's one of Hayao Miyazaki's first films as director.

[Chainmax]

Let's discuss this further via PM .

[Chainmax]

How could you set rad to 1.75? Its type is set to int, I had to switch it to float in order to be able to do that. By the way, 1.75 would be a much better default value, maybe toying with ss using HQDering as fallback could be avoided, can you please test that?

Originally Posted by Chainmax

How could you set rad to 1.75? Its type is set to int, I had to switch it to float in order to be able to do that.

Yeah, that's what I did. To quote myself:

I changed the rad so that it takes float values.

Originally Posted by Chainmax

By the way, 1.75 would be a much better default value, maybe toying with ss using HQDering as fallback could be avoided, can you please test that?

Sure, I can do that.

Edit: Messing around with the ss doesn't do anything.

Actually been messing around with the radius again and I'm starting to like it around 2.25, a good balance IMO between halo removal and preserving the bright areas, and no need for HQDering. Heres some comparisons:

Source:



Radius 3.00:



Radius 1.75:



Radius 2.25:



Source:



Radius 3.00:



Radius 1.75:



Radius 2.25:

[Didée]

Guys. It's a trick function, aiming at removing brighties and preserving darkies. Reducing radius to preserve the brighties will also weaken the removal effekt - note that in your r=2.25 examples the halos are only damped, but not removed. While in the last shot, the letters still are noticeably darkened.

There were good reasons why initially I only posted a linear script, not a function ... and later named the function begged for as silly as "abcxyz": While it might have some potential, still it's only a trick function that *must* fail in places.


BTW, there was quite some mentioning of BlindDeHalo2 in this thread. Did anyone notice that BlindDeHalo has reached its 3rd incarnation since quite some time, and has gotten lots of tweaks to play with?

- "sharpness" parameter (and "tweak" counterpart)
- "lodamp" (and "hidamp" (!) ) parameter
- the "PP" modes, which act somewhat similar to HQdering ...
-- ... where "PP" < 0 runs *only* these routines, without the main dehaloing one

(a quickly fiddled before / after example, posted on the German forum some months ago.)

Know your tools.

[Chainmax]

It might be a trick function, but it works wonderfully, and following rad=1.75 with HQDering is still a great option. Just because some people diss your awesome work that doesn't mean you must do it yourself. I really do believe that DDeHalo deserves its own thread and more testing by various persons.

About BDH3, I didn't mention it because I figured Anonymouses preferred 2 to 3. Besides, I just don't like the BDH line, I feel they oversmooth the whole frame and kill tiny details.

Yeah but BlindDeHalo, even the 3rd incarnation, has a real problem with destroying black lines while this preserves them which is one things I like about it. Maybe something could be added to help preserve the black lines without needing to redarken them after using BDH? And true, while using a radius under 3.00 doesn't completely eliminate the halos it does dampen them enough that something like HQDering can be used to eliminate the last bit without hurting more of the bright areas and preserving the integrity of the black lines.

[yaz]

Originally Posted by Didée

Guys. It's a trick function ... it might have some potential, still it's only a trick function that *must* fail in places ...

yep yep ... make some tuning here and there and finally u get ... BDH anyway. if u think of polishing abcxyz consider to move your resizing into the very last step a/o using spline16resize instead of lanczos. it'll fasten the encoding and decrease halo, definitely.

the bests
y

[Didée]

Oh, I think the resizing is already put as late as possible. And are you sure that spline16resize is faster and/or does create less ringing than lanczosresize?

No matter ... if I'd [manage to] put in the stuff exactly as you all would like, the speed would get very very slow anyway. We're probably talking about utilising median filtering with bigger radii, and such stuff.

[yaz]

@didée
i didn't mean u w/my comments but the abcxyz addicts

as regards spline16, yes, it produces much less ringing. speed? dunno. more or less the same. worth to give it a go.

what i meant by 'moving resizing in' is quite simple. chainmax's script contains a separate resizing step which is unnecessary if the downsizing step after the oversampling would target the final size. i would do it this way, but it's just a clue.

the bests
y

Originally Posted by yaz

yep yep ... make some tuning here and there and finally u get ... BDH

Except BDH is much more destructive on details and the black lines in anime then the abcxyz function, so I'll just stick to using the trick function. Plus the speed is pretty good as well.

[Chainmax]

Originally Posted by yaz

yep yep ... make some tuning here and there and finally u get ... BDH

Your are wrong. Like anonymouses said, abcxyz has much less side-effects than the BDH line.
About Spline16Resize, is it as sharp as Lanczos4? Because if it isn't, then its speed and less likelyhood of creating halos is a moot point for me.

Originally Posted by Didée

if I'd [manage to] put in the stuff exactly as you all would like, the speed would get very very slow anyway. We're probably talking about utilising median filtering with bigger radii, and such stuff

Like many others in this forum, speed is of no concern. The way I see it, it's better to spend twice the time if that will yield better results, after all you'll be seeing your encodes for far longer the time it took you to encode them. Again, I highly encourage you to make a thread for abcxyz and receive some extensive feedback from fellow Doom9ers. Animated content buffs will love it for sure.

[Revgen]

There is a lot of good information on dehaloing and ring removal in this thread, unfortunately this a rainbow removal thread.

Perhaps a mod could move all the dehaloing info to a new thread and a new topic.