New halo removers discussion

[Didée]

The present haloing knocked me off the chair!

... But when hitting the ground, I had a raw idea worth to try:

 rad = 3.0   #  radius for halo removal
 ss  = 1.5   #  radius for supersampling / ss=1.0 -> no supersampling
 o   = last
 ox  = o.width()
 oy  = o.height()
 
 x      = o.bicubicresize(m4(ox/rad),m4(oy/rad)).bicubicresize(ox,oy,1,0)
 y      = yv12lutxy(o,x,"x 8 + y < x 8 + x 24 - y > x 24 - y ? ? x y - abs * x 32 x y - abs - * + 32 /",U=2,V=2)
 z1     = repair(o,y,1)
 maxbig = y.expand().bicubicresize(m4(ox*ss),m4(oy*ss))
 minbig = y.inpand().bicubicresize(m4(ox*ss),m4(oy*ss))
 z2     = o.lanczosresize(m4(ox*ss),m4(oy*ss))
 z2     = z2.logic(maxbig,"min",U=2,V=2).logic(minbig,"max",U=2,V=2).lanczosresize(ox,oy)

(ss == 1.0)  ?  z1  :  z2

return last
#----------
function m4(float x) {return( x<16?16:int(round(x/4.0)*4)) }

No tweaks (could be done), no safety checks about ranges & so (probably needed), just your source image with raw halo press-down:

[Chainmax]

I don't see that much haloing in the source frame (probably because it's so messy, I can provide cleaner frames with halos if you want) but I'm going to try it, thanks .

[edit]Could you arrange it into a function? I don't know how to include it on the script.

[yaz]

vouw ... nother impressive script from didée
what i can't get :
- what is z1 ? ( i guess, it's the 'repaired' clip )
- why is 1.5 is hardcoded in the supersampling part ( i guess, that'd be ss instead )
... or am i on a wrong way ?

thx
y

[Didée]

Originally Posted by yaz

- what is z1 ...
- why is ...

Dunno. Falling from the chair, I landed on my head ... straight thinking impossible.

Hey, it's just a sudden glance, quickly scribbled, and seems to do [at least] something right. I almost assure you there'll be some overflow-errors in the LUT ... can't evaluate that now.


For Chainmax:

function abcxyz(clip clp, int "rad", int "ss")
{
rad = default(rad, 3.0)   #  radius for halo removal
ss  = default(ss,  1.5)   #  radius for supersampling / ss=1.0 -> no supersampling

ox  = clp.width()
oy  = clp.height()
 
x      = clp.bicubicresize(m4(ox/rad),m4(oy/rad)).bicubicresize(ox,oy,1,0)
y      = yv12lutxy(clp,x,"x 8 + y < x 8 + x 24 - y > x 24 - y ? ? x y - abs * x 32 x y - abs - * + 32 /",U=2,V=2)
z1     = repair(clp,y,1)
maxbig = y.expand().bicubicresize(m4(ox*ss),m4(oy*ss))
minbig = y.inpand().bicubicresize(m4(ox*ss),m4(oy*ss))
z2     = clp.lanczosresize(m4(ox*ss),m4(oy*ss))
z2     = z2.logic(maxbig,"min",U=2,V=2).logic(minbig,"max",U=2,V=2).lanczosresize(ox,oy)

return( (ss==1.0) ? z1 : z2 )
}

function m4(float x) {return( x<16?16:int(round(x/4.0)*4)) }

Obviously and as usual, RemoveGrain/Repair and MaskTools are needed.

[FredThompson]

Just for snicks and grins, look at the first post in this thread and see if enlarging, then noise filtering (maybe even a little heavier), then shrinking, then LimitedSharpen will help. http://forum.doom9.org/showthread.php?s=&threadid=68494 The sample images are gone but the results was wonderful. Maybe, just maybe, this would make the aberrations more aberrant...

FWIW, some folks combat that horrible blockiness in very similar areas by playback with ffdshow and adding a little noise. It can help break up the outlines just enough to fool our eyes during playback.

edit: Hah! I knew all this seemed too familiar: http://forum.doom9.org/showthread.php?s=&threadid=64432

[AVIL]

@Chainmax

AVIL: I used FFT3DFilter at sigma=6 and bt=3 (alone and with TemporalSoften) but it didn't do much to the rainbows.

Try with a script like :

avisource("your.avi")
fft3dfilter(sigma=3,sigma2=3,sigma3=12,sigma4=3,bt=3,plane=1)
fft3dfilter(sigma=3,sigma2=3,sigma3=12,sigma4=3,bt=3,plane=2)

In my case the rainbows are middle-sized so I use heavy denoising en sigma3. Real colour are more afected by sigma4, then I use a moderate value for it. Sigma and sigma2 copes whit subtle rainbows. You can raise this values but in the first example I think it isn't necessary.

Anyway, bifrost and smartssiq are very effective filters. In fact you can combine it:

From bifrost's manual.

Bifrost(scenelumathresh=1.5,altclip=SSIQ(11,100,true),interlaced=true)

But i found fft3dfilter more suited for me, in part for YUY2 support, in part for the easy in tunning the filter.

[Chainmax]

Didée, you are THE MAN!!! By replacing the FFT3DFilter line with a default call to abcxyz, the halos are completely gone and the result is almost identical to the last screen I posted!! . I'll post the three screens with this filterchain soon.

[FredThompson]

Didée does make some really good cleaning stuff. There are those, however, who think he's a shill for CPU manufacturers :P

Methinks I'll try this on some DV moire (camcorder shots of moving roller conveyor) and see how it does. Maybe it will also help with laserdisc moire.

[Chainmax]

Here are the three screens with the last filterchain:



[link removed, screenshot appears now on page 3]



The first one looks pretty much the same as the last attempt, but the second and third one look worse. They are, however, rare special cases that will probably be attenuated(sp?) by x264's deblocking (0,0 has proven too weak so I'll go for 2,2) anyway. I will encode a sample in a taxing sequence and upload it for you guys to examine. Again, thanks for all the input .

[FredThompson]

You could play with MDeblock http://www.mdeblock.de.tf/

I like the results of LimitedSharpen but it's dog slow. The basic idea, modifying the extremes of the sharpening, could be put into dll sharpeners and they'd probalby run a lot faster. My Athlon XP 2200s run HC best profile at 5 fps with LimitedSharpen, ~30 fps without it. Something to keep in mind because it dramatically impacts encoding time.

[Chainmax]

Since these are 25min episodes, I can just compress to ffdshow's YV12 huffyuv and then feed that into MeGUI. I'll try MDeblock soon and report back, although it kinda bugs me that it never was developed over v0.3 and that it went pretty much unnoticed here.

[Didée]

@ Fred

Get this and this, try Soothe(LimitedSharpen(ss_x=1.0,ss_y=1.0,Smode=4),last,24) [or 16], and compare again fps and result.

[FredThompson]

I'm now seeing 10 fps instead of 5 and the oversharp whities are reduced without losing the sharpness of the black lines. Thanks. DVD-RB Pro is now an overnight task, not a "sometime tomorrow" task. Thanks!!

What is the last parameter? I used your first suggestion, not the second. This is some sort of influence limiter?

The thread for Soothe mentioned the possibility of support files for LimitedSharpen. I wonder if anyone other than hanc messes with Fortran. Seems like a natural way to get some speed in the math routines.

Tried UnSharpMask instead of LimitedSharpen but the results just weren't that great. LimitedSharpen seems to do a very nice job of sharpening hard edges and leaving relatively smooth areas less modified. Quite helpful for animation, especially with the relatively low amount of oversharp artifacts.

[Chainmax]

I made a test encode of an expecially taxing sequence. You can download it from [link deleted].

I'd just like to throw in that the dehaloing script you wrote Didée is pretty nice. Gets rid of the halos as good as BlindDeHalo but doesn't make the video blurry and doesn't thin out black lines.

Source:



Using abcxyz():



Using BlindDeHalo2(2.5,2.5,160):



And thanks for pointing out BiFrost Chainmax as it is a nice rainbow filter.

With BiFrost and abcxyz():

I don't want to get too off topic here, but I can't get over how nice that dehalo script is. Here's another couple of samples comparing it against BlindDeHalo2 which I used to use. I changed the rad so that it takes float values.

Source:



default abcxyz():



abcxyz(rad=1.75), turned down the radius to preserve the small bright areas, and HQDering(255) to catch the bit of ringing that the lowered radius doesn't get:



BlindDeHalo2(2.5,2.5,160):

[Chainmax]

Very nice indeed. I'm going to test abcxyz (I renamed it to DDeHalo, i.é: Didée's DeHalo ) on my test source that has lost of small details and see how it compares to HQDering, FixVHSOverSharp and BlindDeHalo3.

Anonymouses, I recently made a separate thread wher I ask for feedback on my Trigun encoding attempts. I'm mulling over two alternatives and would appreciate your feedback.

Originally Posted by Chainmax

Very nice indeed. I'm going to test abcxyz (I renamed it to DDeHalo, i.é: Didée's DeHalo ) on my test source that has lost of small details and see how it compares to HQDering, FixVHSOverSharp and BlindDeHalo3.

Well at least in my tests I like to use both the dehaloer, with the radius turned down from default to stop it from messing with small bright areas, and using HQDering to pick up the haloing that is left behind due to the smaller radius. Though even at lower radiuses does get rid of more of the halo then just HQDering alone even when its set to 255.

Chainmax just curious, how bad is the blocking on the R1 discs or is it pretty isolated? Looking at some of those shots with the pretty bad blocking and having seen how bad the rainbows are on the R1 discs, I'm glad that I bought the R3 boxset.

[Chainmax]

It's pretty nasty, but it is isolated too . It appears mostly on Vash's costume and a few rare times in sky scenes as well. I can upload a vob sample with the intro if you want to check it out.