Use differences two "remove" background from video ?

[djidjo]

Hi folks,

There's something I'm trying to do under Vegas Video ((here's my post) (http://forum.doom9.org/showthread.php?s=&threadid=90721) and I finally found how to do it (see this tutorial) (http://matthew.chaboud.com/vegas5/twocats/)

But there might be an easier way with frame-to-frame differencing in AVISynth to do it. Shortly said :
I have :
- A video with someone moving on a fixed background (forest)
- A picture of exactly the same background without him

I'd like to :
- Apply a "mask" to remove everything from the background except the man moving and apply transparency or solid color to what is the same.

In AVISynth, parameters would be sth like :
KeepOnlyDifferences (Clip1_moving, Clip_or_StillImagefrom_Background, [color_to_put_on_removed_pixels],[Tolerance_on_color])

Or maybe should I call it Pixel_toPixel_ifference. Does it exist ?

Thanks for any help !
Djidjo

[djidjo]

I found the "subtract" function. It's not exactly what I need, but I'll try it.

[E-Male]

with some scripting it should be possible
also a plug-in could be written for it

but remember that both background and camera would have to be very static for this to work

[djidjo]

Of course, both backgrounds have to be identical OR there must be some tolerance. Subtract is not what I need. YV12Subtract approaches it, though the output is B&W and the background is still visible through the man.
Mmmmh...

[E-Male]

Originally posted by djidjo
YV12Subtract approaches it, though the output is B&W and the background is still visible through the man.:script:


with YV12Subtract and some fiddeling you should get a background-mask

[djidjo]

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MaskTools.dll")
a=ConvertToYV12(AVISource("junk tape - Clip 001 - 005.avi"))
t=AVISource("junk tape - Clip 001 - 003.avi")
b=ConvertToYV12(t++t++t++t) (this is the background clip which is shorter)
YV12Subtract(b,a,tol=0.01)
ConvertToYuy2().Levels(0, 1.8, 255,0, 255) (for the output to be brighter)

I tried with the cats from the other tutorial at the moment since I don't have my DV sources at work
:rolleyes:

I don't really know how masks work, so it's hard for me to do a background mask.

[E-Male]

try this:
-------------
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MaskTools.dll")
a=ConvertToYV12(AVISource("junk tape - Clip 001 - 005.avi"))
b=AVISource("junk tape - Clip 001 - 003.avi").loop(4) #this is the background clip which is shorter
c=colorbars.converttorgb24().bilinearresize(a.width,a.height) #example for the new background
a=a.converttorgb24
b=b.converttorgb24
m=subtract(a,b)
m=m.rgblut("x 128 - abs","x 128 - abs","x 128 - abs")
m=m.converttoyv12
m=m.Binarize(20) #20 is default, experiment with this value
m=m.converttoyuy2
overlay(a,c,mask=m)
-------------

[djidjo]

Your script doesn't work (the resulting movie is identical to source movie), but I get the idea of the mask. I'll make some more tries and tell you.

Thanks for your help.

[djidjo]

That's it (with my real video sources)!

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MaskTools.dll")
a=ConvertToYV12(AVISource("..\secret_immortalite\SdI_.05-02-18_14-23.00.avi"))
b=ConvertToYV12(ImageSource("fond_foret.png",end=FrameCount(a),fps=FrameRate(a)))

m=YV12Subtract(a,b,tol=20).Binarize(10).converttorgb24
c=colorbars.converttorgb24().bilinearresize(a.width,a.height) #example for the new background
a=a.converttorgb24
overlay(a,c,mask=m)

Finally I think I had the right solution but upgrading Masktools to v1.56 resolved a bug in yv12subtract. This is not perfect, since the man running has some pixels disappearing when the color is too close from the background. But he's a wizard, so it's ok :D

Thanks a LOT for your overlay and mask help !
Djidjo