code/script wizard challenge - comb mask

[Mug Funky]

okay, there's a lot of great avisynth tinkerers around here. here's the problem - comb masking is annoying as hell, so lets try to find a good one.

simply posting script chunks will do... throw in any ideas you might have no matter how weird they might be.

what we need:
- detects combs/interlacing
- no motion no change
- relatively noise resilient (it wont deinterlace anything with grain near it).

there are plenty of good deinterlacers, or combinations of deinterlacers out there, but we all seem to be in want of a good comb mask that does the above at a reasonable speed.

so let's all put our heads together and see what we can throw at this annoying problem.

[Mug Funky]

this one seems to be working well for me... maybe it can be built on.


overlay(last.doubleweave().selectodd(),last,mode="difference").limiter()
yv12convolution(horizontal="1",vertical="1,-2,1,",Y=3,U=-127,V=-127,usemmx=true)

[scharfis_brain]

the properties, a good comb-mask should have:

- chroma test
- noise resitant
- dectection of slow moving borders
- fullframerate-mask (NTSC:59.94 fps, PAL:50 fps)
- mask shouldn't be too much blown up
- if progressive video gets masked with this fullframerate mask, only every even(odd?) frame should contain a motionmaskedimage, the odd(even?) frames shouldn't be masked, due to the progressive nature of the video!
- static fine detail should be completely supressed in the mask

[sh0dan]

Just for reference, if you haven't already done so, have a look at Donalds research in his "Journal" section at neuron2.net (http://neuron2.net/).

But added a "noise level detection" thingie to a comb mask algorithm sounds interesting.

[SoonUDie]

As a suggestion, try posting some samples of your most problematic material.

[mf]

As a base to start off, subtract a vertically sharpened version from the original, and denoise that. You can convert this into a full-range greyscale mask for further refinement. I've used this method in CrappyCombMask, but as it's crappy(tm), the mask gets converted to binary by a simple Binarize(), and is made smart by canceling it out with a motion mask, which is a lazy and crap way to make a combmask of course. I might actually investigate better ways of refining the mask, as it could lead to something interesting. It could be made better by going outside of the boundaries of AVS scripting, for that I've posted an IRC log in the MaskTools thread.

[Mug Funky]

i've got this so far:


function McComb (clip c, int "cut", int "rolloff")
{
cut=default(cut,1)
rolloff=default(rolloff,cut+16)
c=c.converttoyv12(interlaced=true)
overlay(c.doubleweave().selectodd(),c,mode="difference").limiter()
yv12convolution(horizontal="1",vertical="1,-2,1,",Y=3,U=0,V=0,usemmx=true)
gauss(4).levels(cut,1,rolloff,0,255,coring=false)
}

function gauss (clip c,int "radius",bool "conv",int "precision") {

radius=default(radius,4)
conv=default(conv,false)
precision=default(precision,8)
mul=int(pow(2,precision))

Function siney(string s,int stop,int len,int mul)
{
eqn = string(round(mul*pow(sin(pi*stop/len),2)))
#(stop == 0) ? s : eqn + "," + siney(s,stop-1,len)
return (stop == 0) ? s : siney(s,stop-1,len,mul) + "," + eqn
}

matrix=siney("",radius*2,radius*2,mul)
matrix=matrix.midstr(2,(matrix.strlen()-3))

(conv==false)?c.bilinearresize(4*(c.width/(radius*2)),4*(c.height/(radius*2)))
\.bicubicresize(c.width,c.height,1,0):c.yv12convolution(horizontal=matrix
\,vertical=matrix,Y=3,U=3,V=3,usemmx=true)

}


most of that is just the blurrer that's called inside "Mc Comb", but could be replaced by much simpler stuff. i keep all my funktions in one big avsi, so i tend to make it all very self-referential.

i'm currently using this in a semi-smart bobber, and i'm getting very clean results without needing postprocessing (a first for me). there may be very slight shimmer, but i'd not be so wet about a thing like that :)

if this could be used in intellibob or something else with matching abilities i could replace decomb for hybrids. (i don't have much hybrid stuff on my box, and i just keep it for testing)

[edit]
line breaks... must remember the line breaks

[scharfis_brain]

@sh0dan: I've read neuron's journal some times.
But his method of Field differncing

# luma only
bob().verticalreduceby2().blur(1).greyscale()
subtract(last,last.trim(1,0))

would catch combing very well, BUT cannot decide between static detail and combing!

I'll try the other methods posted here, later.

[Didée]

Haven't found a solution up to now, either - but let me throw in some thoughts, perhaps someone might draw something useful out of it.

Speaking of my beloved plaything "EdgeMasks" ;), note the following:

The usual way of creating an EdgeMask

DEdgeMask(0,255,0,255, "6 10 6 10 -64 10 6 10 6", setdivisor=true, divisor=8)

gives an EdgeMask as expected - but it is VERY sensible to noise!

Compare it with the mask returned by the following operation:

Overlay( DEdgeMask(0,255,0,255, "4 10 4 0 0 0 -4 -10 -4", setdivisor=true, divisor=6)
\ ,DEdgeMask(0,255,0,255, "4 0 -4 10 0 -10 4 0 -4", setdivisor=true, divisor=6)
\ ,mode="lighten" )

This is noticeable slower, okay. But the mask is pretty robust against noise (and catches edges much better, BTW).


About lacing <--> static detail:

I tried to create a sensitive CombMask, and then to somehow subtract an EdgeMask of the bobbed clip. Didn't work out for now, although it should, if done properly.

But what about the following:

Take the difference between the actual scanline and its upper neighbor:
DEdgeMask(0,255,0,255, "2 8 2 -2 -8 -2 0 0 0", setdivisor=true, divisor=4)

Take the difference between the actual scanline and its lower neighbor:
DEdgeMask(0,255,0,255, "0 0 0 -2 -8 -2 2 8 2", setdivisor=true, divisor=4)

Then compare both masks:
- In areas where both masks are similar & bright, we are in the middle of combing.
- In areas where both masks differ, there is a high probability that we have detail instead of combing - at least in the direction of the respective field, that returned the darker mask for that area.


Keep in mind I am brain impaired when it comes to interlacing ;)

- Didée

[Didée]

What I meant was something like this:
XYZsource(...)

altern = DEdgeMask(0,255,0,255, "2 10 2 0 0 0 -2 -10 -2", setdivisor=true, divisor=2)

upper = DEdgeMask(0,255,0,255, "2 10 2 -2 -10 -2 0 0 0", setdivisor=true, divisor=2)

lower = DEdgeMask(0,255,0,255, "0 0 0 -2 -10 -2 2 10 2", setdivisor=true, divisor=2)

# both = overlay(upper,lower,mode="lighten") # or "xor", or "andn" (by MTools) - play around!
both = logic(upper,lower,"min")

# combmask =
overlay(both,altern.invert,mode="multiply")

Looks not bad at first glance, to the naked eye -- alas right now I cannot check now if its really masking as desired (am @ work :( )

- Didée


edit: little update for creation of "both" mask.

[tritical]

Well, here is my rather lame attempt at a comb mask. I threw it together in a hurry this morning and didn't get a chance to really test it (so it may contain a few unwanted features). Checklist:

no motion, no change - yep
detects combing - yep
noise resilient - sort of :p

TCombMask v0.9.3 (http://bengal.missouri.edu/~kes25c/TCombMaskv093.zip)

The default thresholds are pretty low and will catch pretty much anything you could see, unless you are single stepping with your eyes six inches from the monitor :D . They could probably be increased thus making the mask more noise resilient. As with other comb detectors it still gets fooled by moving details (lines), since it is pretty much impossible to differentiate between the two unless you start requiring vertical neighbors on the comb map.

[tritical]

Sorta feel like I'm either hijacking a thread or reviving a dead one by replying to my own post, but I didn't have time to go over the details of how the filter I posted before worked and thought it might be a good idea as someone else can probably suggest improvements.

Consider the series of pixels below that are from three consecutive frames in a video, where pixels a, c, and e are from the same field and pixels b and d are from the same field. We want to check if C2 is combed. This process is the default way that TCombMask works...


frame 1 frame 2 frame 3
a1 a2 a3
b1 b2 b3
c1 C2 c3
d1 d2 d3
e1 e2 e3


The first step is the motion check. To determine whether or not C2 should be considered static, we perform these two tests:

(abs(c1-C2)>mthresh && abs(b2-b3)>mthresh && abs(d2-d3)>mthresh)
(abs(c3-C2)>mthresh && abs(b1-b2)>mthresh && abs(d1-d2)>mthresh)

If either of the above two conditions is true then we consider C2 to be moving and go on to the spatial (area) combing check. If neither of the above conditions is true then we consider C2 static (not combed) and go on to the next pixel. I originally had the above and below motion checks (pixels b and d) ||'ed and not &&'ed but && didn't seem to leave any stray combed pixels and actually helped remove some erroneous pixels.

Now if the pixel is considered to be moving the next step is the actual combing check. TCombMask does this in two steps. First, it checks the condition (b2-C2)*(d2-C2) > 0, to make sure that both b2 and d2 are different from C2 and that they are both different in the same direction. Second, TCombMask simulates a linear bob + blur and subtract to get the actual comb value, this process is very similar to the

bob()
blur(1)
subtract(last.selecteven(),last.selectodd())

routine, which is also very similar to Mug Funky's "1,-2,1" vertical convolution but with a little vertical blurring. (Actually metric 0 in TCombMask is the same thing as a "1,-2,1" vertical convolution as it does not use blurring.) The actual calc used by metric 1 in TCombMask is:

abs(a2 + 4*C2 + e2 - 3*(b2 + d2)) = combing value;

Then if the combing value is greater then 6*athresh, the pixel is considered combed. I also tried simulating a cubic bob + blur, but the linear tended to work better. Anyways, that's the end of the process, aside from the denoising /or plane linking. I haven't been able to think of many improvements, assuming no motion tracking/compensation.

Extra note... The original filter I posted used a slightly different metric 1, but I went ahead and made a v0.9.1 that has the above version. I also noticed that the IsCombed() conditional function in Decomb doesn't consider motion so I added an IsCombedT() conditional function into TCombMask. It is the same exact thing and works the same way as IsCombed() but uses TCombMask's comb map. It also makes the # of pixels inside a 4x8 block to consider a frame combed, user adjustable.

[Mug Funky]

very nice.

i can't get it to work flawlessly yet, but i've only just started fiddling with it... defaults work very nicely in terms of noise-resilience and motion checking.

[scharfis_brain]

jep, very good motion detection...

but: try to input progressive Video. (or use separatefields().selecteven() to simulate it)
It never should mask it :(

[mf]

Scharfi, are you going on about the field shifted PAL thing again? A scripted filter really can't consider something like that, it's hard enough to detect combs.

[Didée]

I don't consider that a M:I - and if it should be, we still can ask Mr. Cruise for help ;)

Basically, one has to analyze the bobbed clip either parallel to the plain input clip, or to do all of the analyzing on the bobbed clip only. The reckognition of progressive content, no matter if phase-shifted or not, should be quite easy to detect through a difference clip that is constantly changing bright-dark-bright-dark -- and even changes between phase-shifted and not-shifted can be detected on changes of just that pattern, unless they appear on places with no motion at all.
It's a cumbersome procedure for sure - but I'm just as sure that it can be done.

But, you cannot create a scripted function that one can throw just anything at (progressive, progressive phase-shifted, blend-interlaced, true-interlaced, hybrid-interlaced, telecine), and the function reckognizes the type of content by itself and applies the correct algo's. Possible with a 2-pass system, impossible with one-pass only.

If we consider the potential user mature enough to specify the basic type of the content, then everything will be fine, somewhen. (And since even the "decomb package" cannot distinguish telecine from pure video etc. and relies on the user to specify the according function, this is already usual practice.)

End of blabla

- Didée


edit

Mug Funky:

When I feed some interlaced video into McComb, I get a mask clip out of it that is nothing but constantly pure white ??

[scharfis_brain]

I'll try tcombmask next time on some fieldblended samples.
I think, there it would perform very well.

The reason, why I ask for a progressive resistive-comb mask is, because I've some Videos containing progressive and phase-shifted progressive within the SAME frame

for example, the PAL-MusicVideo "Talk Talk - Such a shame"
the Film itself is normal progressive, but the rotating cubes are phase shifted progressive mixed into the progressive clip.
this means the cubes become combed.
if we apply a phase shift to the whol video, the Film gets combed and the cubes are looking non-combed!
This canot be corrected using telecide or any other fieldmatcher, but my intellibob() handles this, because it sorts the fields right on the per-pixel-level. Without doing interpolation/deinterlacing.

[Mug Funky]

ouch. that kind of content i usually consign to the "encode interlaced" basket...

masked kernelbob works sufficiently with anime. natural video gets a little jumpy though, and sangnom isn't ready yet (corners and frame edges are no-go areas) which is unfortunate

[tritical]

but: try to input progressive Video. (or use separatefields().selecteven() to simulate it)
It never should mask it


NOTE: I may be way off base on the meaning of this statement and some of the other posts in which case please disregard this :p

Hm, I'm not sure, but I guess you mean it should detect progressive frames vs. interlaced frames and then depending on the result output a blank mask or the actual comb mask? This can actually be achieved with a little workaround by doing:

clip1 = last.tcombmask()
# quick hack to get a blank mask, probably a faster way to do this
clip2 = last.tcombmask(mthreshL=400,mthreshC=400)
conditionalfilter(last,clip1,clip2,"IsCombedT()","=","true")

Of course the above decision will never be 100% accurate. Also, this only works on a per frame basis so your clip with normal progressive content and phase shifted progressive content in the same frame will still confuse it. Although, I don't have any videos with such content to experiment with since I live in ntsc land...

I think the main point, however, is that this filter (currently) is only designed to detect which parts of the frames (that it is passed) are combed, not to match fields and take pixels from other fields to see what can be fixed... so I'm not really sure it is a bug or simply a limitation. (I may be completely off base here). The other thing, is that motion detection (like the simple kind used in TCombMask) and spatial combing detection alone will never be able to correctly distinguish between progressive vs. interlaced content on a per pixel basis 100% of the time. This is why passing TCombMask() progressive content still results in pixels being detected as combed. And afaik there isn't a mask that can reliably do this (if there was, this thread would probably never have been started :D ).

[mf]

You could probably do a comb check on a phase corrected version of the clip and cancel out masking (overlay, mode=darken) on those spots, but that's more up to the deinterlacer than the combmask.

[scharfis_brain]

nope such kind of workaround with comb-checking won't work, because the false phased-progressive video might have only a small area compared to the whole image. this means it remains combed, when trying to do some mask-pre-matching :(

the mask-algorithm have to go straight forward without knowing about the type of the Video.
Only the metrics have to decide whether it is combing or high frequency detail. No iscombed() or something else here.
This is, what I want to do after the mask creation process for doing the intelligent deinterlacing.

[mf]

Scharfi, was that directed at me, or at tritical? Cause the first part of your post could apply to my suggestion (or so it looks..), but the second is clearly aimed at tritical.

[tritical]

My posts are usually long, as is this one so be prepared :D

One thing is that I seem to have a slightly different view of what a general purpose combmask should do then what some other people think it should. That is not to say that one way is correct and the other is wrong, but simply different. When I hear combmask I think one thing... it ouputs a mask showing you which pixels in the frames of video that it is passed are combed. It doesn't do field matching, weaving with other fields, video type detection, or anything else... as too me these are deinterlacer or field matcher specific operations.


the mask-algorithm have to go straight forward without knowing about the type of the Video.
Only the metrics have to decide whether it is combing or high frequency detail. No iscombed() or something else here.


That is the goal, however imho it is impossible for any fixed metric, like those used by current AviSynth deinterlacers or combmasks, to ever be able to correctly distinguish between certain types of high frequency detail and combing.

@scharfis_brain

I seem to be too dumb to understand exactly what you mean by

- fullframerate-mask (NTSC:59.94 fps, PAL:50 fps)

Would that be if you doubleweave() the video?

[Begin Long/Boring Rambling W/ Many Errors and Lots Of Misinformation ;)]

Lets start off by assuming we have some filter that detects which pixels in a frame are combed (not whether or not the combing could be corrected by field matching or taking pixels from previous or next fields, but simply which pixels in the current frame are combed) and doesn't have any idea of what type of video it is passed or have human help /or input. Of course, the first very simple thing to do, as is done by any area based deinterlacer, is to test whether any of the pixels in the frame exhibit the natural combing symptoms. By natural combing symptoms I mean the upper pixel and the lower pixel are both darker or both lighter then the current pixel... what this is really testing is how closely the two fields match. There are plenty of variations on this concept. Some of the usual metrics used are:

1.) (upper-current)*(lower-current) > thresh
2.) ((upper-current > thresh && lower-current > thresh) ||
(upper-current < -thresh && lower-current < -thresh))
3.) abs((upper+lower)/2 - current) > thresh
4.) edge masking and subtracting, like Didee's idea
5.) bob() and subtract type operations

and many others. These usually work well, but will always detect certain types of high frequency detail as combing, simply because certain types of high frequency detail and combing are exactly the same thing according to any fixed metric as those above. Now, lets say the current frame we are examining is completely progressive, but it has a lot of high frequency detail, such as single pixel wide/tall lines, window blinds, or whatever you can think of. Metrics that consider more pixels, (3x3),(4x4), etc..., or that require horizontal or vertical neighbors to be detected as combed can avoid being fooled as often. However, they risk missing actual combing and will still get fooled by high frequency detail that is repeated vertically or horizontally (again window blinds or single pixel wide lines that are repeated every other vertical line).

All isn't lost at this point though since we can still do a motion check. If those high frequency details are static (maybe the window blinds don't move from frame to frame) then we can avoid detecting them as combing. Now the quality of the motion check makes a difference here, if the motion check only checks for motion from the previous frame or from the next frame it isn't really detecting what we want it to, it needs to be a bi-directional motion check for our purpose of comb detection. But what if those window blinds are moving continuously from frame to frame? (remember that we have passed the filter progressive material to begin with) Now the motion check will fail, and we default back to the area combing metric, which we already know will fail. Thus detecting progressive high frequency detail as combing.

However, let's say that instead of a simple motion check we will do full scale motion detection/compensation.. surely this can do what we need it to. The problem though is motion detection/compensation has nothing to do with progressive vs. interlaced content. Motion compensated deinterlacers work by tracking motion between fields (i.e. they track motion between consecutive bottom fields and consecutive top fields) which are (or should be) progressive and then knowing that in interlaced material each field is half-way between its previous and next fields deinterlace the video. So to be of any use to us it would require user input as to what is combed or progressive to begin with. This is also why a motion compensated deinterlacer will not work so well if passed telecined material (that has not been field matched) that still has combed frames.

Thus, all this rambling comes down to one thing, if none of these metrics can tell the difference... how do human beings distinguish between high frequency details and combing? The answer I came up with is rather simple, though probably wrong, image/object recognition. If we look at a frame of video we can usually tell very quickly whether it, or some part of it, is interlaced or not, simply because we know how things in the video should look to begin with (i.e. not interlaced). This seems simple, and it is to a human... but coding a program that can do the same thing, if it's even possible, would be more complex then I care to imagine. I guess the point of all this, is that imho w/o object recognition/identification and tracking it is simply impossible to pass a combmask function (that has no idea of what type of video it is passed) progressive material and for it to never detect a single pixel as combed. This doesn't mean that I don't think that current combmask's can't be improved upon or that there aren't better ways of detecting combing then mentioned above. Just that correctly distinguishing between certain types of progressive details and combing may never be possible using fixed metrics.

[/End Rambling/Babbling]

Feel free to correct the many mistakes and false assumptions I made.

[scharfis_brain]

@scharfis_brain

I seem to be too dumb to understand exactly what you mean by

- fullframerate-mask (NTSC:59.94 fps, PAL:50 fps)

Would that be if you doubleweave() the video?

Jep!

Doubleweave().Tcombmask(params...)

does its job fairly well :)

on progressive video, this construct delivers less masking (false positives, hehe) on even frames and a full motionmask at the odd ones.
This is the point, where I can start applying the matching.
On a phase shifted progressive Video, the odds end evens will be swapped.
(Thats why I need a fullframerate mask)

@mf:
my whole post was based upon tricticals post.

[Didée]

Reminder:

It could be a good idea to have a closer look at the sourcecode of the good old SmoothDeinterlace plugin(->AviSynth (http://home.bip.net/gunnart/video/AVSPorts/SmoothDeinterlacer/), -> VirtualDub (http://biphome.spray.se/gunnart/video/). It deals with all the topics and issues that are discussed here. It can be used either as plain deinterlacer or as bob-deinterlacer, and has so interesting parameters like lacethresh, edgethresh and staticthresh+staticavg ;)
It even outputs an informative and colorful mask - although this mask cannot be used directly, since it's blended over the clip with transparency.

Had almost forgotten this one, although I used it plenty in the past.

- Didée

[mf]

I remember contacting Xesdeeni about it, as the static stuff works only one way (in the past), because of VDub limitations. AVISynth however can look at frames both ways, and thus "predict" if a scene is static. This could get rid of the visible raising of the threshold during playback. But Xesdeeni told me he's not into it far enough to add such a feature to the avs version.

[tritical]

Ah... smooth deinterlace, while I haven't looked at the source code for it in quite a long time, I do remember (hopefully) how all those options work.

First, it uses a variation on the (upper-curr)*(lower-current) formula for area combing detection. That variation is the edgethresh parameter. Basically it also computes abs(upper-lower) and if the difference is large, it considers the pixel to be part of an edge and not combing. Of course, it doesn't do exactly this, as that difference and edgethresh are actually included in the comb value calculation, however this is what it ends up accomplishing. I've tried this out before, but it tended to do more harm (leaving residual combing) then good (avoiding high frequency detail). That is if your already checking if the upper and lower are different from current in the same direction. However, I will definately look into it again.

The staticthresh is just the usual motion threshold, how much a pixel can change from one field or one frame to another. As mf said the vdub version as well as the avs port use only a previous motion check. The avs port also only considers luminance on both motion and area combing detection I think? It then deinterlaces the corresponding areas in chroma.

Static average is somewhat special since smooth deinterlace ends up varying the staticthresh based on how long an area has been static. I personally don't think this is a good idea, as it means the filter can't adapt as fast and is prone to residual combing as well as being slow to adjust. Though it might have been needed since vdub filters cannot look both ways.

The last thing is that smooth deinterlace averages combing values over three vertical pixels when deciding whether the current pixel is combed, although the middle pixel (current line) is given more wieght I think. This means it will be less sensitive to single horizontal lines and details but also means it will leave more residual combing.

Anyways, the more ideas that get thrown out the better. Brainstorming is usually better then anything else in finding solutions to tough problems. Too bad there's always so much to test out and never enough time :(