avisynth: "Sky replacement" or "Green screen" like overlay?

[stormy1777]

I'm relatively new to scripting using avisyth, and eager to learn. I have a rather simple "task", a short video of the rather WHITE moon over very blueish sky. I need to make the "sky" area "transparent" and overlay that over another video, so it appears as if the moon is moving over that other video.

I got the overlay (mode="blend" and opacity=0.5) working OK, however, the sky from the moon video impact the other video, is there any way to make it "transparent"?

what I ended up using is roughly this:

moon = tweak(moon,sat=-2, bright=10)

which looked best in the final overlay...

I also tried to experiment with things like:

transparent=resetmask(moon).colorkeymask($365266, 50)
layer(clip2, transparent)

however, that also removed parts INSIDE the moon which have a similar color as the sky :) :)

since the area is round (moon), and it's moving, not sure it is possible to define that in terms of "area", it must be done with colors/hue/etc, but I'm not so good with these, any tips welcomed.

I can post a sample moon photo, although it is very basic, blue sky + white moon.. nothing really special there.

Stormy.

[wonkey_monkey]

Try making a blankclip approximately the colour of the sky (or with slightly higher values) and then subtract it from the moon clip.

If the other clip is also just blue sky, you can then do a straight add to put it in the shot. You might need to tweak the other levels a little first.

[ajp_anton]

Since you already got the transparency mask working for outside of the moon, I would start with that and work on removing the parts inside the moon.

I would probably use mt_hysteresis for this. Create a mask with a tiny dot somewhere in a corner that's guaranteed to always be outside of the moon. Then grow that dot into the whole area outside the moon, assuming the areas inside are really contained and have no contact with the outside sky. The end result should be a mask for the outside sky only.

It's a shame there's no function to fill all empty spaces inside a mask. Kind of like treating it like a landscape, and then let heavy rain form lakes inside holes, and the rest of the water will wash down on the sides of the image. In your example, the holes inside the moon would fill up nicely.

[poisondeathray]

I can post a sample moon photo, although it is very basic, blue sky + white moon.. nothing really special there.

however, that also removed parts INSIDE the moon which have a similar color as the sky

since the area is round (moon), and it's moving, not sure it is possible to define that in terms of "area", it must be done with colors/hue/etc, but I'm not so good with these, any tips welcomed.


So it's not really a "white" moon if there are shared colors with the sky... But how does the hue or saturation of the elements inside the moon compare to that of the sky ?

A picture is worth 1,000 words, a video is worth 10,000 .

You might be able to combine a saturation mask (maskhs, using minsat, maxsat, hue settings) to increase the accuracy of your current mask; since presumably the sky really is mostly "blue", and the moon really is mostly "white" (but some shared hues). You can probably assume that the sky has higher saturation than the "blue" parts of the moon. I'm assuming they would be from reflected hues, or "greyish" craters.

If only the moon is moving (no camera movement, no other objects like clouds, birds, trees), then combining with a motion mask might help improve the mask - either with masktools2 or mvtools2



Other non avisynth tools can be useful here too . A short moon clip could probably be masked out (rotoscoped) with free tools in a few minutes. eg. blender, hitfilm express , fusion ,etc..

[stormy1777]

Thanks so much for the many ideas and information, unfortunately, I'm not a video editing professional, just hobby, some of the suggestions went over the top :) :)

Right, Moon is not pure "white", as you can now see in the photo :) it shares some grayish colors with the blueish sky..

I did try the subtract method, although ended up with BLACK sky, isn't that going to hurt the combine to the main clip (which is not blue sky)... I just don't know what/if any constitutes "transparent" in video processing :)

ajp_anton, not sure what/how to do this "grow that dot to include all area outside of moon); as moon is moving constantly in the video.

As for other suggestions & tools; not sure i'll have time for this soon, but for now the simplistic saturation filter did it good enough.

Thanks.
Stormy.

[WorBry]

You might be able to combine a saturation mask (maskhs, using minsat, maxsat, hue settings) to increase the accuracy of your current mask

Don't know if this function I concocted 'many moons' :rolleyes: ago adds any value:

https://forum.doom9.org/showthread.php?p=1405481#post1405481

Basically applies Tweak through a definable luma range mask (with soft roll off). It was the closest I could get to the equivalent of an 'HSL-like' qualifier (key) in YV12. I was going through a 'film emulation' phase at the time. These days for that kind of thing I use DaVinci Resolve which (including free version) has an excellent qualifier tool set.

But if you can't get good isolation of the moon shape without loss of surface detail/tones (owing to color cross-talk) your only recourse is physical masking ('garbage matte'), like PDR said:

A short moon clip could probably be masked out (rotoscoped) with free tools in a few minutes. eg. blender, hitfilm express , fusion ,etc..

Could maybe also go at it in Resolve with a qualifier and/or tracked 'power window'.

Moon is not pure "white", as you can now see in the photo :) it shares some grayish colors with the blueish sky..


Your sample photo is still pending approval btw, so it's difficult to know if it's a 'simple task' or not.

...some of the suggestions went over the top :) :)


Not really, if that's what it takes to get the best outcome.