View Single Post
Old 3rd July 2015, 23:09   #2  |  Link
joj
Registered User
 
Join Date: Jan 2013
Posts: 7
Issues

The envisioned benefit of this plugin over ordinary binarize masks is that it eliminates a lot of noise. However, being currently a purely spatial filter it has the drawback of not being able to differentiate background from foreground and this poses a problem in certain cases. Specifically it tends to perform badly when a shape changes form in particular ways, such as a subject connecting different body parts or when the camera captures only parts of objects, which may cause background to shine through "small holes",

What happens in those cases is that "new shapes" / objects are being created, and if they're too small for the specified threshold, then they're excluded from masking consideration.

Example:


The examples were created as follows:
Code:
Overlay(Invert, mask=Binarize(190, upper=false))
Overlay(Invert, mask=ShapeMask(190, minarea=0.02))
Some thoughts I've had to improve the performance of the filter are:
  • Consider temporal information as well. For example, track if new shapes were previously part of other shapes. That is, deemed connected in the previous frame. Then carry that information forward until the shape vanishes.
  • Leverage motion vector information. E.g. if a shape is more or less static, then override the area based constraint and consider the shape for inclusion anyway, provided it meets the luma threshold of course. Or perhaps compare the motion with other shapes, and if it's different "enough" to that of the "others", then flag it as potentially a new shape. Other would need to be defined somehow, such as 1 or 2 degrees removed but it seems this would provide some additional information for decision making regardless of specifics.
There are still issues I can think of that these new tools can't solve. The rotating subject described above, a white shirt on a white background would still be hard to differentiate since the only indication that something has happened is that the surrounding shape has deformed (expanded) in the location where the shirt would be at say a 90 degree turn from facing the background. How would the boundary be identified at this point?

If someone has any other ideas on how to deal with the shape distinction issue, I'd be glad to hear it.

Last edited by joj; 4th July 2015 at 07:38.
joj is offline   Reply With Quote