ShapeMask - Dynamically mask out bright shapes

[joj]

If you're watching a lot of screen casts and recorded talks, and are as sick as I am of the over-use of white (console backgrounds, web pages, projections ...), then you may want to try out a plugin I created to mitigate that health hazard.

A compiled binary is available here (http://snarl.zapto.org/files/ShapeMask-1.0.zip).
sha1: 5af5727b572b5e14bd32d1f8ee71d73522d41286

Source: github (https://github.com/jojje/shapemask) and there you also find the most readable version of the documentation, including example and screenshots.

The plugin is a bit on the large side since it includes parts of OpenCV, a library that is ... challenging to prune.

A note on the implementation. It could probably be optimized a bit more with AVX and some more SSE2, but as it stands it is sufficiently fast for my needs at least (getting about 350 fps with avsmeter).
It does require a CPU that has SSE2 (I believe, don't have one without) as the CV bits were configured for utilizing that. Hopefully that won't be an issue as even the cheapest laptop has that instruction set nowadays..

Let me know if it crashes on you. If it does, then I've probably stripped CV down too much..

[joj]

The envisioned benefit of this plugin over ordinary binarize masks is that it eliminates a lot of noise. However, being currently a purely spatial filter it has the drawback of not being able to differentiate background from foreground and this poses a problem in certain cases. Specifically it tends to perform badly when a shape changes form in particular ways, such as a subject connecting different body parts or when the camera captures only parts of objects, which may cause background to shine through "small holes",

What happens in those cases is that "new shapes" / objects are being created, and if they're too small for the specified threshold, then they're excluded from masking consideration.

Example:
http://griffeltavla.files.wordpress.com/2015/07/sm_problem_sample2.gif

The examples were created as follows:
Overlay(Invert, mask=Binarize(190, upper=false))
Overlay(Invert, mask=ShapeMask(190, minarea=0.02))

Some thoughts I've had to improve the performance of the filter are:

Consider temporal information as well. For example, track if new shapes were previously part of other shapes. That is, deemed connected in the previous frame. Then carry that information forward until the shape vanishes.
Leverage motion vector information. E.g. if a shape is more or less static, then override the area based constraint and consider the shape for inclusion anyway, provided it meets the luma threshold of course. Or perhaps compare the motion with other shapes, and if it's different "enough" to that of the "others", then flag it as potentially a new shape. Other would need to be defined somehow, such as 1 or 2 degrees removed but it seems this would provide some additional information for decision making regardless of specifics.

There are still issues I can think of that these new tools can't solve. The rotating subject described above, a white shirt on a white background would still be hard to differentiate since the only indication that something has happened is that the surrounding shape has deformed (expanded) in the location where the shirt would be at say a 90 degree turn from facing the background. How would the boundary be identified at this point?

If someone has any other ideas on how to deal with the shape distinction issue, I'd be glad to hear it.