Bifrost 2000: improved derainbowing filter for VapourSynth and Avisynth

[jackoneill]

Code (and readme.rst): https://github.com/dubhater/vapoursynth-bifrost
DLLs: https://github.com/dubhater/vapoursynth-bifrost/releases

For those who don't know Bifrost: http://check2pic.ru/compare/32560/

The original Bifrost sometimes creates undesirable artefacts:
http://www.abload.de/img/bifrost-off-on1qjzf.png
http://check2pic.ru/compare/32559/
http://imgur.com/a/fUpWO

This version mostly avoids those issues by processing blocks rather than whole frames. Another consequence is that static rainbows can be processed even if there is lots of motion in other parts of the image.

There are some differences:
- the "scenelumathresh" parameter is renamed to "luma_thresh" and its default value is higher
- the "conservativemask" parameter is renamed to "conservative_mask"

Thanks to Myrsloik for sharing the source code.

[mandarinka]

Yay! Bifrost was one of the better and still useful "old/simple" filters - if it was not for those demonstrated issues.

[Myrsloik]

I approve of the name. This way we can later sell an upgrade to Bifrost 3000.

Apparently the filter has more followers than I thought...

[ChiDragon]

Check2pic links won't load for me. Isn't the imgur link just improperly treated interlaced YV12?

Using blocks seems like a great idea. What about fully pixel-adaptive?

[jackoneill]

Check2pic links won't load for me. Isn't the imgur link just improperly treated interlaced YV12?

Using blocks seems like a great idea. What about fully pixel-adaptive?

check2pic.ru seems to be dead right now. Try later.

What do you mean by "fully pixel-adaptive"?

[mandarinka]

Check2pic links won't load for me. Isn't the imgur link just improperly treated interlaced YV12?

Nope, it really did these things in some scenarios.

/BTW, Shouldn't this be posted in the general avisynth subforum, so that normal encoders notice the filter is out?/

[ChiDragon]

What do you mean by "fully pixel-adaptive"?

Sorry, I've been reading too many docs on commercial comb filters. The original Bifrost must have been.

[mandarinka]

Nope, it made decisions whether to process or not on frame-basis only, according to jackoneill (hence those troubles).

[Myrsloik]

Nope, it made decisions whether to process or not on frame-basis only, according to jackoneill (hence those troubles).

It actually decided per pixel what to process. But it was far too aggressive. It only selected which direction to blend with per frame.

Doing it the same decision per block is a much better idea.

[Reel.Deel]

@jackoneill
Thanks for the Bifrost update.

I've updated the Bifrost AviSynth wiki (http://avisynth.nl/index.php/Bifrost) and have a question about the interlaced parameter.
If true, SeparateFields, DoubleWeave and SelectEvery will be invoked....
I assume that only applies to VapourSynth, so for AviSynth should it be?
If true, SeparateFields and Weave will be invoked....

[jackoneill]

@jackoneill
Thanks for the Bifrost update.

I've updated the Bifrost AviSynth wiki (http://avisynth.nl/index.php/Bifrost) and have a question about the interlaced parameter.

I assume that only applies to VapourSynth, so for AviSynth should it be?

That's right.

[burfadel]

Thanks for updating this!

I was playing around with the settings and came across a situation where it crashes (for me) without fail! If I use bifrost(blocky=2) it crashes, but if I use bifrost(blockx=2) it works fine. Of course, if I mix blocky=2 in with any other setting it also crashes, so I believe the blocky code part may have a bug.

[jackoneill]

Thanks for updating this!

I was playing around with the settings and came across a situation where it crashes (for me) without fail! If I use bifrost(blocky=2) it crashes, but if I use bifrost(blockx=2) it works fine. Of course, if I mix blocky=2 in with any other setting it also crashes, so I believe the blocky code part may have a bug.

Uh, yeah. It's too small. I'll have to add a check for that.

[jackoneill]

Thanks for updating this!

I was playing around with the settings and came across a situation where it crashes (for me) without fail! If I use bifrost(blocky=2) it crashes, but if I use bifrost(blockx=2) it works fine. Of course, if I mix blocky=2 in with any other setting it also crashes, so I believe the blocky code part may have a bug.

Version 2.1 fixes this. It's also a bit faster.

[burfadel]

Thanks :) Is there an Avisynth version of v2.1? There's only v2.1 for Vapoursynth on the website

[burfadel]

So I take it there won't be an updated Avisynth version to v2.1?

[jackoneill]

Maybe if other problems surface. I don't feel like touching the Avisynth part and Visual Studio just for this.

[burfadel]

Ah ok! I was thinking of the performance improvements that was mentioned more than the block size. Were they substantial improvements, or just slight?

[jackoneill]

Ah ok! I was thinking of the performance improvements that was mentioned more than the block size. Were they substantial improvements, or just slight?

Not very big.

[burfadel]

Ah ok :) Fair enough then :D. Thanks!

[Overdrive80]

Nice filter, thanks. I am trying remove this rainbow but I be not able to do it.

I post a split of original source. Rainbow shows in white T-shirt. https://db.tt/zvbOXyTE

Thanks.

[jackoneill]

Nice filter, thanks. I am trying remove this rainbow but I be not able to do it.

I post a split of original source. Rainbow shows in white T-shirt. https://db.tt/zvbOXyTE

Thanks.

You are not able to because the rainbow moves. Bifrost can't fix rainbows that move.

[Overdrive80]

Ok, thanks.

[Kurosu]

My solution some years ago was to use a strong blurring (like an adaptive gaussian filter with high thresholds) followed by some warpsharping on chroma-only to reduce chroma bleeding. Maybe the two could be combined with a temporal activity mask.

[real.finder]

Ok, thanks.

try

from sh0dan script (http://forum.doom9.org/showthread.php?p=398106#post398106) but with masktools2


function DeRainbow(clip org, int "thresh")
{
assert(org.isYV12(),"DeRainbow() requires YV12 input!")
thresh = default(thresh, 10)

org_u = utoy(org)
org_v = vtoy(org)

msharpen(org, threshold = thresh, mask=true)
reduceby2()
greyscale()
uv = blur(1.5).levels(0,2.0,255,0,255, coring=false).blur(1.5).blur(1.5).levels(50,2.0,255,0,255, coring=false)

filtered_u = org_u.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,255,0,3,2)
filtered_v = org_v.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,255,0,3,2)

u_final = mt_merge(org_u, filtered_u, uv)
v_final = mt_merge(org_v, filtered_v, uv)

return ytouv(u_final, v_final, org)
}



Usage:

YV12: DeRainbow(int thresh)

Threshold is optional, and defaults to 10 (which is fairly strong).

Required plugins:
MipSmooth.
Masktools2.
mSharpen.

I found it in my avs autoload folder that I originally take it from a friend

it work good on your sample

[mandarinka]

My solution some years ago was to use a strong blurring (like an adaptive gaussian filter with high thresholds) followed by some warpsharping on chroma-only to reduce chroma bleeding. Maybe the two could be combined with a temporal activity mask.

Sounds heavy-handed and suboptimal. I recommend using motion-compensation instead.

Here is what I routinely use:

#you need to have progressive video for this.
prerb = last
derbmask = tedgemask(threshY=9).mt_inflate()
derbsuper = MSuper()
derbbackward_vectors = MAnalyse(derbsuper, truemotion = true, isb = true, chroma=false)
derbforward_vectors = MAnalyse(derbsuper, truemotion = true, isb = false, chroma=false)
derbforward_compensation = MCompensate(derbsuper, derbforward_vectors)
derbbackward_compensation = MCompensate(derbsuper, derbbackward_vectors)
interleave(derbforward_compensation,last,derbbackward_compensation)
dfmderainbow(maskthresh=10) # Divine Buster!
selectevery(3,1)
mt_merge(prerb,last, derbmask,luma=true)

DFMDerainbow is hell of an old function, but sadly I have not found anything better so far. It must be possible to make something better, but alas nobody with the right skills/knowledge has done so :D
I definitely don't recommend it without edge-masking and motion-compensating (which the above does). Fluxsmooth is too dumb and the chroma blurring this thing does is quite destructive, if not contained.

(I tried mdegrain for the denoiser part, but that has severe issue with flashing pictures. You will get pretty horrible chroma bleeding into next/previous frames.)

[real.finder]

Sounds heavy-handed and suboptimal. I recommend using motion-compensation instead.

Here is what I routinely use:

#you need to have progressive video for this.
prerb = last
derbmask = tedgemask(threshY=9).mt_inflate()
derbsuper = MSuper()
derbbackward_vectors = MAnalyse(derbsuper, truemotion = true, isb = true, chroma=false)
derbforward_vectors = MAnalyse(derbsuper, truemotion = true, isb = false, chroma=false)
derbforward_compensation = MCompensate(derbsuper, derbforward_vectors)
derbbackward_compensation = MCompensate(derbsuper, derbbackward_vectors)
interleave(derbforward_compensation,last,derbbackward_compensation)
dfmderainbow(maskthresh=10) # Divine Buster!
selectevery(3,1)
mt_merge(prerb,last, derbmask,luma=true)

DFMDerainbow is hell of an old function, but sadly I have not found anything better so far. It must be possible to make something better, but alas nobody with the right skills/knowledge has done so :D
I definitely don't recommend it without edge-masking and motion-compensating (which the above does). Fluxsmooth is too dumb and the chroma blurring this thing does is quite destructive, if not contained.

(I tried mdegrain for the denoiser part, but that has severe issue with flashing pictures. You will get pretty horrible chroma bleeding into next/previous frames.)

Compared to sh0dan DeRainbow above? I didn't work on any source with rainbow previously, Could you tell me the strengths and weaknesses for each of them?

--------

edit: my try https://pastebin.com/j5JVNwSP using mt_motion for motion and edge, and of course other filters

ASTDR default values may be high, ASTDR(3,15,2,FluxStv=30,edgem=true) should be enough and without noticeable artifacts or use ASTDRmc

[mandarinka]

Compared to sh0dan DeRainbow above? I didn't work on any source with rainbow previously, Could you tell me the strengths and weaknesses for each of them?

DeRainbow (this (http://www.animemusicvideos.org/forum/viewtopic.php?p=1264945#p1264879)? I'd say it is fairly obsolete today...) is IIRC just spatial, so it doesn't make use of temporal filtering. You would be throwing away quality of the source needlessly.
Edit: oh wait, you posted a different version. I didn't try that. It has temporalsoften (instead of fluxsmooth in DFMDerainbow), so you could try to motion compensate it.

DFMDerainbow does something similar to its spatial processing (if not the same, the blur/deen combination looks familiar), but in addition it removes rainbows based on content in next/previous frames - and if you motion compensate it, this works even with motion (to some extent of course). That snippet also has an edge mask to limit the damage.

[Kurosu]

Sounds heavy-handed and suboptimal. I recommend using motion-compensation instead.

Sounds slow and mostly applicable to limited amount of rainbows.

But then again, my solution is 10-years-old.

[mandarinka]

That's what I meant. 10 years later, more sophisticated techniques are available.