Can these right-edge halos be corrected?

[Avisynth_challenged]

Hi all,

I have an avi of a laserdisc capture. One of the problems I'd like to fix is a right-edge halo (ghosting?) which is found throughout the source. Here are some screengrabs to illustrate the problem:











As can be seen, haloing (ghosting) is very clear on the right-hand side of edges, such as the outline of on-screen characters, and objects like candlesticks.

Has anyone successfully eliminated edge artifacts like these? It looks like they occur at a fixed-pixel horizontal distance throughout the entire source... so it may be possible to make a function that can detect the problem and apply some sort of area-based correction. I did try the Avisynth plugin "FixVHSOversharp", but it wasn't effective on this source.

My sincerest thanks to all for reading, and for any feedback!

[JanBing]

You might want to check out my postings in THIS thread over at OT.com (my username there is Laserschwert).

[thetoof]

Try Dehalo_alpha

function DeHalo_alpha(clip clp, float "rx", float "ry", float "darkstr", float "brightstr", float "lowsens", float "highsens", float "ss")
{
rx        = default( rx,        2.0 )
ry        = default( ry,        2.0 )
darkstr   = default( darkstr,   1.0 )
brightstr = default( brightstr, 1.0 )
lowsens   = default( lowsens,    50 )
highsens  = default( highsens,   50 )
ss        = default( ss,        1.5 )

LOS = string(lowsens)
HIS = string(highsens/100.0)
DRK = string(darkstr)
BRT = string(brightstr)
ox  = clp.width()
oy  = clp.height()
uv  = 1
uv2 = (uv==3) ? 3 : 2

halos  = clp.bicubicresize(m4(ox/rx),m4(oy/ry)).bicubicresize(ox,oy,1,0)
are    = mt_lutxy(clp.mt_expand(U=uv,V=uv),clp.mt_inpand(U=uv,V=uv),"x y -","x y -","x y -",U=uv,V=uv)
ugly   = mt_lutxy(halos.mt_expand(U=uv,V=uv),halos.mt_inpand(U=uv,V=uv),"x y -","x y -","x y -",U=uv,V=uv)
so     = mt_lutxy( ugly, are, "y x - y 0.001 + / 255 * "+LOS+" - y 256 + 512 / "+HIS+" + *" )
lets   = mt_merge(halos,clp,so,U=uv,V=uv)
remove = (ss==1.0) ? clp.repair(lets,1,0) 
          \        : clp.lanczosresize(m4(ox*ss),m4(oy*ss))
          \             .mt_logic(lets.mt_expand(U=uv,V=uv).bicubicresize(m4(ox*ss),m4(oy*ss)),"min",U=uv2,V=uv2)
          \             .mt_logic(lets.mt_inpand(U=uv,V=uv).bicubicresize(m4(ox*ss),m4(oy*ss)),"max",U=uv2,V=uv2)
          \             .lanczosresize(ox,oy)
them   = mt_lutxy(clp,remove,"x y < x x y - "+DRK+" * - x x y - "+BRT+" * - ?",U=2,V=2)

return( them )
}

function m4(float x) {return(x<16?16:int(round(x/4.0)*4))}

[Avisynth_challenged]

Hey, thanks for the responses thus far. I appreciate the feedback

@ JanBing/Laserschwert:
I've been lurking at OT.com and at the X0 Project sites for several years now. It seems that if it weren't for Star Wars laserdiscs, no one would have much interest in discussing the finest minutiae of digitizing analog sources and post-processing of analog caps. What I've concluded from the discussions is that this right-edge ghosting is a common problem, and I think some of the LD gurus at alt.video.laserdisc said that Pioneer players all had this issue to varying degrees. The discussion has really tapered off since the so-called GOUT DVD release of the original trilogy, so maybe it's time to dust off this topic and see if the power of current state of Avisynth - particularly with Masktools - can be used to fix our caps.

@ thetoof:
Thanks for posting that script. I read *the DeHalo Alpha thread* and concluded that it wouldn't be suitable for my needs. For one thing, it seems like this function was written for, and tested primarily on, anime content ripped from DVD. And in that thread, one user, Mug Funky, reported *here* that his analog cap required a very customized filtering step prior to running the Dehalo Alpha function in order to get decent results.

So I've thought about it a bit, and analyzed the nature of the problem. Consider the following image blow-up which shows the ghosting issue:





Notice that the halos begin on the third pixel to the right of high-contrast edges. Additionally, the halos are a fixed width of 2 pixels wide.

It seems pretty clear that if those 2 pixels could be replaced with the next 2 pixels located just outside the halo to the right, the images in the frames would be very much improved.

So perhaps Masktools2 could be used to do something like this:


1. Start with orig=Clip1
2. Let Shiftr_2px=Clip1.crop(0,0,-2,-0).addborders(2,0,0,0)
3. Let Mask1 = edgemask of Shiftr_2px (mt_edge)
4. Let Shiftr_4px=Clip1.crop(0,0,-4,-0).addborders(4,0,0,0)
5. Let Mask2 = edgemask of Shiftr_4px (mt_edge)
6. Let Mask2i = inverse of Mask2 (mt_invert)
7. Let Maskhalo = logical AND of Mask1 and Mask2 (mt_logic AND)
8. Let Halofix=Clip1.crop(2,0,-0,-0).addborders(0,0,2,0)
9. Let Final = mask merge of clip Halofix with Clip1 (mt_merge)
10. Return Final


Please critique this algo. Does it stand to reason? Thanks again for your interest

[Didée]

The usual "generic" halo-removers (dehalo_alpha for example) can't cope with this kind of EE -- they assume some "symmetry" that infact is not there.

The custom convolution way shown by Mug Funky is a reasonable approch ... some weeks ago I did something similar for a user on the German forum (Gleitz), with quite good success. (Edit: here.)

Did some tinkering during lunch break, with a different approach: (beware: ugly script, dont show that to anyone!)

# needs:  mt_masktools.dll , medianblur.dll , removegrain.dll , repair.dll

o=last

hb  = o.mt_convolution("1 2 3 2 1","1",U=2,V=2)
hm  = mt_luts(o,o,mode="median",pixels="-2 0 -1 0 0 0 1 0 2 0",U=2,V=2)
hbD = mt_makediff(o,hb)
hmD = mt_makediff(o,hm)
DD  = mt_lutxy(hbD,hmD,"x 128 - abs y 128 - abs < x y ?")
mbh = o.mt_makediff(DD,U=2,V=2)

m   = mbh.medianblur(2,0,0)
mD  = mt_makediff(mbh,m)
DDd = mt_lutxy(mD,DD,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?")
DDd = DD.repair(DDd,12)
reS = mbh.mt_adddiff(DDd,U=2,V=2)

shift = mbh.addborders(4,0,0,0).crop(0,0,-4,-0)
diff= mt_lutxy(shift,mbh,"x y - abs 8 - 16 *").crop(4,0,-4,-0).addborders(4,0,0,0)
 \   .mt_expand(mode="horizontal").removegrain(11)
reS = o.mt_merge(reS,diff)

interleave( o  .subtitle("original"),
 \          mbh.subtitle("horizontal clamp"),
 \          reS.subtitle("horizontal clamp, re-sharpened, all @ edge-areas only")
 \          )

return(last)

Not perfect, but not too bad either. (Removing halos is easy. NOT killing lots of detail in the process, that's a problem ...)

[Avisynth_challenged]

Thank you for your response and for the script. I went to the filters list at Warpenterprises, and also the external plugins page at the wiki, but cannot find the repair.dll. Is this dll packaged in a plugin kit that's not called "repair"?

Also, Didée (or anyone) do you not believe that replacing the halo pixels with non-halo neighboring pixels is possible? It sounds like a good idea in theory, and thought that this kind of thing would be fairly easy to implement with Masktools2 through a process like I've outlined in my previous post. But maybe I'm wrong on both counts.... thoughts???

Thanks again everybody

[Didée]

Originally Posted by Avisynth_challenged

cannot find the repair.dll. Is this dll packaged in a plugin kit that's not called "repair"?

It's contained in the RemoveGrain package (this' v1.0beta). Is it possible to use Avisynth without these basic plugins?

Also, Didée (or anyone) do you not believe that replacing the halo pixels with non-halo neighboring pixels is possible? It sounds like a good idea in theory, and thought that this kind of thing would be fairly easy to implement with Masktools2 through a process like I've outlined in my previous post.

What I believe is that you could go ahead and put your suggestion into an Avisynth script. hey, you've the script already finished, just the syntax lacks alittle.
(Well - almost finished. For step 9), you forgot to specify which clip should be used as mask to merge Halofix onto Clip1.)

On the way, you might discover that an edgemask not always gives the automagical result that you would like to have.

[Avisynth_challenged]

Originally Posted by Didée

It's contained in the RemoveGrain package (this' v1.0beta). Is it possible to use Avisynth without these basic plugins?

Got it, thanks!

Originally Posted by Didée

What I believe is that you could go ahead and put your suggestion into an Avisynth script. hey, you've the script already finished, just the syntax lacks alittle.
(Well - almost finished. For step 9), you forgot to specify which clip should be used as mask to merge Halofix onto Clip1.)

Yep, my algo did not specify "Maskhalo" as the mask to merge the clips. Thanks for that. I've never written a function using Masktools before, and had wondered if my understanding of its use is correct (how to make masks, combine masks, apply masks for filtering etc.). It sounds like you understood my algo (what it's meant to do), and that means a great deal to me coming from Didée

Originally Posted by Didée

On the way, you might find that an edgemask not always gives the automagical result that you would like to have.

One possible problem with the algo, I just thought:
What if the image contains higher frequency details, like a pinstripe pattern on clothing, or lots of close vertical edges like, say, an aerial view of a cityscape with lots of high-rise buildings and towers?

In those cases, the unconditional replacement of 2 pixels with the right-neighboring 2 pixels approach would muddle those details. My algo should be modified to include the condition that the mask should be made only if the detection of mask edges are in relative isolation. I'll have to think about this some more.

My sincerest gratitude goes to this forum and its members. You all are excellent, and inspire excellence in the field of digital video processing

[Didée]

.. and in step 7), you wanted to specify Mask2i instead of Mask2.

You're correct about the problem with higher-frequency detail. An example is e.g. in your 3rd screenshot, the folds on the woman's dress.

Keep on with this. The more you think about it (and the more you try out), the more problems will appear from different directions, and bite you. It's fun, really!

To get you started, here's your suggestion as a working script:

Clip1      = last
Shiftr_2px = Clip1.crop(0,0,-2,-0).addborders(2,0,0,0)
Mask1      = Shiftr_2px.mt_edge("min/max",4,48,0,255,U=-128,V=-128) # try different "modes" and 
Shiftr_4px = Clip1.crop(0,0,-4,-0).addborders(4,0,0,0)
Mask2      = Shiftr_4px.mt_edge("min/max",4,48,0,255,U=-128,V=-128) # thresholds for tweaking
Mask2i     = Mask2.mt_invert(U=2,V=2)
Maskhalo   = mt_logic(Mask1,Mask2i,"min") # "min" is for greyscale masks what "and" is for B/W masks
Halofix    = Clip1.crop(2,0,-0,-0).addborders(0,0,2,0)
Final      = mt_merge(Clip1,Halofix,Maskhalo,U=2,V=2)

interleave(Clip1,Final)

return(last)

Result: not quite perfect.

[Avisynth_challenged]

Originally Posted by Didée

To get you started, here's your suggestion as a working script:

Clip1      = last
Shiftr_2px = Clip1.crop(0,0,-2,-0).addborders(2,0,0,0)
Mask1      = Shiftr_2px.mt_edge("min/max",4,48,0,255,U=-128,V=-128) # try different "modes" and 
Shiftr_4px = Clip1.crop(0,0,-4,-0).addborders(4,0,0,0)
Mask2      = Shiftr_4px.mt_edge("min/max",4,48,0,255,U=-128,V=-128) # thresholds for tweaking
Mask2i     = Mask2.mt_invert(U=2,V=2)
Maskhalo   = mt_logic(Mask1,Mask2i,"min") # "min" is for greyscale masks what "and" is for B/W masks
Halofix    = Clip1.crop(2,0,-0,-0).addborders(0,0,2,0)
Final      = mt_merge(Clip1,Halofix,Maskhalo,U=2,V=2)

interleave(Clip1,Final)

return(last)

Result: not quite perfect.

Thank you so much, Didée. I've been messing around with this script for hours.

I've taken another approach - using a ghost correction filter to generate a greyscale clip in hopes that it could be used for masking. Here's an example:


ORIGINAL FRAME (with halos):


GREYSCALE CLIP (halo mask?):



Can the greyscale clip be used as a halo correction mask? I didn't use masktools to make the clip, so I wasn't sure. Here's the script that did make the clip, which is called "diff2"


### BEGIN SCRIPT ###
# This line loads the source
source=DirectshowSource("...source.avi")
############
sourcex=source.converttoyv12

start=source.spatialsoften(1,255,127).converttoyv12.greyscale.Tweak(0.0, 1.0, 0, 1.0, true, false)
ghostA=source.spatialsoften(1,255,127).converttoyv12.LGhost(3, 4, -128).greyscale.Tweak(0.0, 1.01, 0, 1.25, true, false)
diffA=Subtract(ghostA, start).Levels(127, 1, 129, 0, 255).invert().undot()

ghostB=source.spatialsoften(1,255,127).converttoyv12.LGhost(3, 0, -128).greyscale.Tweak(0.0, 1.01, 0, 1.25, true, false)
diffB=Subtract(ghostB, start).Levels(127, 1, 129, 0, 255).invert().undot()

diff2=Subtract(ghostA, ghostB).invert()
return diff2
### END SCRIPT ###

Thanks to all for feedback and comments

[Zep]

Originally Posted by Avisynth_challenged

Thanks to all for feedback and comments

any chance you could post a clip from actual source? Does not need to be much. Say few hundred frames.
I want to experiment

[Avisynth_challenged]

Originally Posted by Zep

any chance you could post a clip from actual source? Does not need to be much. Say few hundred frames.
I want to experiment

Go *here*.

Clip is 30 frames, 720x480i, 29.97fps, YUY2, encoded to Huffyuv 2.1.1 upon capture. This is straight from the raw capture, with no processing whatsoever. This clip highlights the rainbow in fine details, but the ghosting is present as well.

Thanks for reading and your interest

[Avisynth_challenged]

Check it out:


ORIGINAL (with halos):


HALO MASK (used to merge clips):


FIXED:



ORIGINAL (with halos):


HALO MASK (used to merge clips):


FIXED:

[*.mp4 guy]

The "correct" way to remove this sort of problem is with a FIR filter.

string1 = string(" 0 -20 28 -32 36 36 128 128 72 -56 12 12 -4 ")
mt_convolution(horizontal=" "+string(string1)+" ", vertical=" 1 ", u=3, v=3)

3rd image corrected with FIR filter

This is however inadequate as far as sharpness and halo removal are concerned, so practically, you need to follow it up with some non-linear filtering, like this.

MT_Merge(last.blur(-1).mergechroma(last, 0), last.blur(1.0).mergechroma(last, 0), last.halomask1(ss=1.75, rad=6).frfun7(1.01, 256, 256))

image corrected with FIR filter+NL filtering

IMO the "correct" way is to avoid this problem in the first place. The halos are caused by a sharpness or edge enhancement adjustment somewhere along the line being set too high. The problem is more common with VHS, (hence the filter FixVHSOversharp) but better VCRs allow you to adjust the amount or turn off the edge enhancement that's added. AFAIK laserdisc players don't usually have this adjustment, but some capture drivers do, so I would suggest to lok into adjusting the capture settings and re-capturing.

[Avisynth_challenged]

Originally Posted by FlimsyFeet

IMO the "correct" way is to avoid this problem in the first place. The halos are caused by a sharpness or edge enhancement adjustment somewhere along the line being set too high. The problem is more common with VHS, (hence the filter FixVHSOversharp) but better VCRs allow you to adjust the amount or turn off the edge enhancement that's added. AFAIK laserdisc players don't usually have this adjustment, but some capture drivers do, so I would suggest to lok into adjusting the capture settings and re-capturing.

I agree absolutely with this post, but am fairly certain that the problem of ghosting (for me) is due to the use of a Pioneer LD player when I did my capture. Here's some discussion about it on the alt.video.laserdisc newsgroup.

When I capped with VirtualVCR, I had the sharpness slider set midway. I did preview the image with different sharpness settings, and felt that midway was best. Maybe decreasing it would have eliminated the ghost, which was not noticed at this time (it was discovered after capping was done). But while previewing, I saw that it would have eliminated valid higher frequency picture detail as well.

I'm learning that in the video processing realm, it's all about compromises and trade-offs. But I am amazed at what can be done with Avisynth and with the advice of this extremely knowledgeable forum.

[Avisynth_challenged]

That mt_convolution line is brilliant. May I ask how you came up with the numbers for my specific edge halos? I don't know a thing about FIR theory, so maybe you could point me to some good reading material. Thanks a bunch for your input, it really helped out

[*.mp4 guy]

I made that filter by hand, using my intuition and about 10 minutes of guess and check refinement. Its not as good as it could be because there isn't any fast way to make a fir filter for this sort of problem.

a quick rundown of the absolute basics you need to understand fir theory for images.

a fir filter is a linear filter that adds together samples (pixels) from multiple points in an image and outputs the cobined value as the new value for the center pixel.

for example:

pixels:
A B C
D E F
H I J

filter: vertical = 86 86 86 horizontal= 255

If this was run on pixel E the result for E would be B/3+E/3+I/3

on an entire image this would produce a strong vertical blur.

here are a few generalizations to work with, negative values produce sharpening, positives produces bluring. the ratio of the center sample to the absolute value of the other samples can give a rough idea of the strength of the filter. The further away from the center sample a filter point is, the more effect it has on lower frequency components, closer to the center the effect is reversed and the filter point will have more effect on high frequency components.

[zipi]

How did you end up using this "Greyscale clip be used as a halo correction mask" ?
I am trying to fix a similar problem, & if possible Id like to try the script you used & tweak it, if you can post it please, thanks!

[Avisynth_challenged]

Originally Posted by zipi

How did you end up using this "Greyscale clip be used as a halo correction mask" ?
I am trying to fix a similar problem, & if possible Id like to try the script you used & tweak it, if you can post it please, thanks!

@zipi:
Got your PM, I sent you a reply. Sorry for the delay!