Line bleeding?

[HyperSpad]

Hey

Take a look at the black lines on the wall in the background and notice how they sort of bleed to the right and left (kind of turning transparent). You can also see it on Lois's right arm and where her hair meets the right side of her face:
http://www.ultronator.com/doom9/family_guy.jpg
Does anyone know what this is and what filter I can try to minimize/eliminate it.

-Thanks
HYPERSPAD

[Leak]

Does anyone know what this is and what filter I can try to minimize/eliminate it.
Don't ask me what this is called, but my AWarpSharpDering (http://leak.no-ip.org/AviSynth/AWarpSharpDering/) script might just work...

[HyperSpad]

Don't ask me what this is called, but my AWarpSharpDering (http://leak.no-ip.org/AviSynth/AWarpSharpDering/) script might just work...
Can you give me some advice as to how to use it? I can get it working, but the documentation is limited, so I'm not quite sure how to tweak it.


Thanks

[Leak]

Can you give me some advice as to how to use it? I can get it working, but the documentation is limited, so I'm not quite sure how to tweak it.
What my script does is use AWarpSharp to sharpen the linework by "pushing" the surrounding areas against the lines (tweakable with "depth"), then limit these changes only to those parts that didn't change too much (tweakable with "diffthresh" and "lumathresh") as to prevent stark deformations. And since those lines are rather faint, it should (mostly) get rid of those while preserving the rest.

Well, for cartoons that are mostly flat shaded areas the default setting of 32 for depth probably works okay, so try tweaking diffthresh from it's default of 8 to higher values until the bleeded lines are gone or the good lines take a hit from it, or maybe try lowering depth to 24 or 20 at that diffthresh first. That's about all that's tweakable in my script... :)

Also, you might want to apply some noise filtering to get rid of the wavy line noise in the background first (FanFilter or FFT3DFilter comes to mind), but I can't really help you on that since I don't capture stuff from tv...

And you'll probably want to use something like GuavaComb to get rid of the rainbowing.

Oh, and there already was some other topic about "Family Guy" where someone had the same problem - do a search on it, maybe his settings work for you as well, even though he was working from the DVDs IIRC...

np: Richard Devine - Scatter Fold 28 (Lipswitch)

[Mug Funky]

Does anyone know what this is..

ringing. or "gibb's phenomenon". i think it's from an attempt to sharpen a band-limited signal. rather like the "resonance" and "LP
dials on a 303 :)

[Chainmax]

Isn't ringing/Gibb's phenomenon/ghosting an asymmetrical issue (i.é: all ghosts appear either to the left or right)? In any case, I recommend HQDering():

######
##
## HQDering v0.1 by mf
##
## Applies derining by using a smart smoother near edges (where ringing occurs) only.
##
## Usage: Import("HQDering-v0.1.avs")
## HQDering()
##
####

function HQDering(clip input, int "strength", int "overall", string "smoother", string "params") {

strength = Default(strength, 255) # strength, 0-255
overall = Default(overall, 0) # overall smoothing, 0-255
smoother = Default(smoother, "Deen") # filter that smooths
#params = default defined below # filter parameters - use Chr(34) for quotes

defaultnull = Default(params, "")
defaultdeen = Default(params, Chr(34)+"a3d"+Chr(34)+", 4, 15, 15, 20")

params = (smoother=="Deen") ? defaultdeen : defaultnull

try {
smoothedwparams = Eval(smoother + "(input, " + params + ")")
}
catch(err_msg) {
smoothedwoparams = Eval(smoother + "(input)")
}

smoothed = (params=="") ? smoothedwoparams : smoothedwparams

input.EdgeMask(3, 255, 255, 255, "sobel", Y=3, V=1, U=1)
normalmask = last

normalmask.Levels(0, 3.3, 90, 0, 255).Blur(1.0)
amplifiedmask = last

normalmask.Inflate().Inflate().Inflate().Levels(0, 3.3, 90, 0, 255).Blur(1.0).Inflate().Inflate().Inflate().Levels(0, 3.3, 255, 0, 255).Inflate().Inflate()
thickmask = last

Overlay(amplifiedmask.Invert(), thickmask, mode="multiply").greyscale

Levels(60, 3.0, 140, overall, strength)

ringingmask = last

MaskedMerge(input, smoothed, ringingmask)

}

[mg262]

The Gibbs Phenomenon per se is not asymmetric, but in a particular electronic implementation you might find an asymmetry due to the fact that the output of an (electronic) filter cannot depend on future values of the input... ie the filter doesn't know whether and edge is about to turn up or not.(This would manifest as a change of phase in the maths, I think). But possibly such filters have a lag associated with them which would simulate the effect of being able to deal with future values... Anyway, you can see pictures of the Gibbs phenomenon on a square wave all over the place, including here:

http://mathworld.wolfram.com/GibbsPhenomenon.html

Having written that, I'm not sure how useful it is... sorry! One concrete consequence is that I wouldn't expect the ringing in MPEG to be directional... but the stuff that comes from tapes might well be.

[HyperSpad]

@Leak

I upped the value for diffthresh and got better results. Although it seems the filter is only affecting "the left sides" of solid black lines (the ones on the back wall). I had to put it pretty high to start affecting the "areas to the right" and at that point it started to affect the solid black lines.


@Chainmax

HQDering() seemed to have more of an overall effect on both sides of the solid lines, but didn't have quite the power that leak's filter had on a single side.


I didn't have much time to experiment today, and I've been away for the past few days before that. I think if I tweak these a bit more, I can get better results. Or maybe, even run both of them. Maybe I'll post again tomorrow with some new results.

I captured this through a vcr as sort of an experiment to try and lose some of the noise in the picture, but I of course got those ringing effects as a byproduct. I've been trying a lot of things (http://forum.doom9.org/showthread.php?t=98708) lately to get a cleaner picture, but it's definetly good to know I've got some good ways to combat the ringing.

[Didée]

Quick & dirty improvement: (very dirty ;) )

orig = last

offset = orig.YV12Convolution("1 0 0 0 0","1",U=2,V=2) # perhaps ("1 1 0 0 0","1")

fullcomp = logic(orig,offset,"min",U=2,V=2)

final = orig.MergeLuma(fullcomp, 0.5) # play with amount value

Requires, as usual, MaskTools (http://manao4.free.fr/MaskTools-v1.5.8.zip) ...

[mg262]

@HyperSpad,

Sounds like you could filter, flip horizontally, filter again, and flip back?

[Mug Funky]

@ hyperspad: don't be too worried about getting a clean picture - most likely the ringing is present on the source being broadcast... either that or something in the chain is doing a digital filter for whatever reason.

[edit]

btw, you can get this effect with the newest builds of avisynth - just use lanczosresize(somewidth,someheight, taps=128). it'll ring like crazy :)

[mg262]

Incidentally, @hyperspad, what do you mean by capturing through the VCR in this case? Are you sticking the antenna into the VCR (as usual) and using the VCR's outputs?

you can get this effect with the newest builds of avisynth - just use lanczosresize(somewidth,someheight, taps=128). it'll ring like crazy I think we need a limited version of lanczosresize (in the sense of LimitedSharpen); we might be able to get the sharpness without the ringing...

[Didée]

I think we need a limited version of lanczosresize (in the sense of LimitedSharpen); we might be able to get the sharpness without the ringing...
For that, load kassandro's repair.dll and try

repair( lanczosresize(somewidth,someheight, taps=much),
\ lanczosresize(somewidth,someheight, taps=few), 1,1,1 )

Haven't a top-recent version of AS running ATM, so can't try myself. Surely there's danger of aliasing. The basic problems of these "simple" resizing methods can't be escaped that easy - you're always trading one kind of artefacting for another.

[Didée]

@ HyperSpad
BTW, the most dedicated filter for this sort of artefacting should be "FixVHSOversharp". Get it from the cornucopia page (http://www.avisynth.org/warpenterprises/) ;)

[Chainmax]

OMFG, I thought that filter was a virtualdub one (don't ask me why, I just did). Oh, this will come in handy for my X-Men VHS capture filtering. Now all I need is to find a filter that does the exact same thing as aWarpSharp(depth=16,cm=1) without the wiggling on horizontal lines and I'll be the happiest encoder on earth :):):):).


P.S: when it says YUV only, does it mean YUY2 or is YV12 useable too? I never really understoos the difference between YUV and YUY2.

[Didée]

OMFG, I thought that filter was a virtualdub one (don't ask me why, I just did).
Probably because the filter was born as as Vdub filter ...

Now all I need is to find a filter that does the exact same thing as aWarpSharp(depth=16,cm=1) without the wiggling on horizontal lines
Chances are poor. This is mostly a general problem of spatial filtering. Without evaluation of temporal relations, the spatial transformation might look OK for each single frame, but when the effectively achieved deviations point in different directions in subsequent frames, temporal jitter is introduced.
Ooops - idea for TameAWarpSharp() ! ;)

P.S: when it says YUV only, does it mean YUY2 or is YV12 useable too? I never really understoos the difference between YUV and YUY2.
No idea, just try.
YUV is a general colorspace format, as is RGB, as is HSV or HSL, or the "Lab" colorspace. YUY2 and YV12 both are YUV format, but use different chroma subsampling. Old story, isn't it ...

[Chainmax]

Probably because the filter was born as as Vdub filter ...


Chances are poor. This is mostly a general problem of spatial filtering. Without evaluation of temporal relations, the spatial transformation might look OK for each single frame, but when the effectively achieved deviations point in different directions in subsequent frames, temporal jitter is introduced.
Ooops - idea for TameAWarpSharp() ! ;)


No idea, just try.
YUV is a general colorspace format, as is RGB, as is HSV or HSL, or the "Lab" colorspace. YUY2 and YV12 both are YUV format, but use different chroma subsampling. Old story, isn't it ...

I see, so I wasn't wrong in assuming VDub version :). As for the colorspace thing, I understand now, thanks. What kind of issues could arise from using a YUY2 filter on a YV12 source (if such usage is possible at all, all YUY2 filter's I tried always display an error mesage on YV12 material)?

By the way, if you ever do a TameaWarpSharp (maybe LimitedaWarpSharp would be better for filter naming consistency's sake? ;)) that doesn't have that line wiggling issue, I will sacrifice my first (and second, and third) born to you :).

[mg262]

Didée,

I haven't thought about this properly. Sounds like I don't know what I'm talking about!

Chainmax

It doesn't really make sense to apply a filter on a colour space that it is not intended for... if you're lucky you get an error message, but if you're unlucky it will just try and read one the data from one spaces as if it were formatted like the data from another... which will cause either random junk or memory violations.

[Didée]

By the way, if you ever do a TameaWarpSharp [...] that doesn't have that line wiggling issue, I will sacrifice my first (and second, and third) born to you :).
Better start google'ing on "bad karma" :D