Find out which pixels or part are processed by an AVIsynth filter

[sirt]

Hello,

Well, as I'm getting back into the use of AVIsynth filters, I'm posting there in order to get some advise. Please forgive me for the questions I will discuss below in so far as they may be easy to answer. I've just forgotten most stuff.

So let's say I use an AVIsynth filter, whatever it is. In order to illustrate, let's choose a denoiser like mctemporal denoiser (link (http://avisynth.nl/index.php/MCTemporalDenoise)) and apply it to a specific frame with those settings :

mctemporaldenoise(settings="very low",sigma=1,limit=255,limit2=255,chroma=true)

How could I use a mask to precisely watch which part/pixels are processed like that ? How could I only process luma and/or chroma and also select a particular part of chroma or luma ? I'm asking in a general case and not necessarly with mctemporal denoiser itself. You can use another filter or denoiser if you want.

Moreover how could I manually exclude some pixels to be processed ? Let's say I know the coordinates (x,y) of a particular pixel in a frame and I don't want it to be processed by the denoiser.

[colours]

0. Every pixel is processed. If what you want to know is where the filter has a nonzero effect, mt_lutxy(original,filtered,"x y - abs 255 *",u=-128,v=-128) will tell you that for luma.

1. Use MergeLuma/MergeChroma for processing only luma/chroma (assuming that you're using a filter that doesn't itself support disabling luma/chroma processing).

2. mt_lutspa("x "+string(x)+" = y "+string(y)+" = & 255 0 ?") would give you a one-pixel mask for the (x,y) pixel, from which you can use mt_merge to restore the source for that one pixel.

[feisty2]

a=source
b=a."filters"
mt_makediff (b,a).mt_lut (mt_polish ("x = 128 ? 0 : 255"))
the black parts are untouched, the white parts are processed
edit: missed a back half parenthesis

[sirt]

Okay thanks for your answers guys. In order to try what you asked me, I've gotten a video from Vimeo and used a small sample from it.

a) If I try something like :

AVISource("C\rob.avi")
a=last
b=a.GradFun2DBmod(thr=1,thrc=2,mode=2,radius=3,range=3,adapt=255,temp=100, str=0)
mt_makediff (b,a).mt_lut (mt_polish ("x = 128 ? 0 : 255"))


I get folowing results on a specific frame :

Original frame (http://oi58.tinypic.com/2rgdzi8.jpg)
After mt_makediff mt_lut (http://oi62.tinypic.com/10yphnc.jpg)

As you can guess, I intend to correct banding on the right. From what feisty2 says blacks parts are untouched, so there some parts of the face would be touched ! But, for a reason I don't understand, if you have a look at final result (Final result (http://oi61.tinypic.com/20zpw86.jpg)) banding on the left if somehow corrected whereas I loose some grains on girl's forehead. I precisely would like her face not to be processed here and to "select" which pixel I want to process.

b) On the other hand, doing :

AVISource("C\rob.avi")

a=last
b=a.GradFun2DBmod(thr=1,thrc=2,mode=2,radius=3,range=3,adapt=255,temp=100, str=0)
mt_lutxy(a,b,"x y - abs 255 *",u=-128,v=-128)

Gives me a complete black pixelled video.Should I deduce luma is not processed by the filter ?

Howhever, doing this :


AVISource("C\rob.avi")

a=last
b=a.GradFun2DBmod(thr=2,thrc=2,mode=2,radius=3,range=3,adapt=2,temp=1)
mt_lutxy(a,b,"x y - abs 255 *",u=2,v=2)


is relatively clear to me (result (http://oi57.tinypic.com/f1vzb4.jpg)) : black and gray pixels seem to be those processed by my filter. So, indeed, many parts of the face are processed ! Playing with radius parameter modifies the processing zone but I can't get less pixel than these to be processed (that is the reason I wanted to manually "select" those I want to be processed). Also, playing with threshold parameters (thr, thrc, mode...) only affect the way black and grey parts are processed. So this part seems to be fixed by radius parameter.

c) Finally, colours I'm not sure to understand how to use mt_lutspa from your description in 2. above. Doing :

x=938
y=538
mt_lutspa("x "+string(x)+" = y "+string(y)+" = & 255 0 ?")

intented to create a mask for pixel in position (938,538) doesn't work. Then could you show me an example including the use of mt_merge too ? Moreover, would it be a similar way to create a mask for all black pixels at the same time ? And also for a particular segment of the video (which, I guess, means I may include trim somehow) ? I think I really need some examples.

[feisty2]

what I wrote is a binary mask, any processed pixel, even if it was so slightly processed that the change is almost invisible like the value changed from 128 to 129 will be scaled to 255, and mathematically identical pixels will be valued 0

[feisty2]

if you wanna gradient mask, change the expr in mt_polish to 2*(abs(x-128))

[sirt]

feisty2, please check result from part b) : result (http://oi57.tinypic.com/f1vzb4.jpg) ; as you can see pixels processed are in black and white. Is there a way I could know which pixels are exactly processed ? I would like to "unselect" all pixels from girl's face and hair from the mask. It's quite too difficult to find out which are those pixels I guess.

[feisty2]

im on cell now, will check l8r

[Gavino]

x=938
y=538
mt_lutspa("x "+string(x)+" = y "+string(y)+" = & 255 0 ?")

intented to create a mask for pixel in position (938,538) doesn't work.
You need to add mode="absolute" to the mt_lutspa call.

[sirt]

Well thanks Gavino but the only way it works is :

AVISource("C\rob.avi")

a=last

x=938
y=538
mt_lutspa(a ,expr="x "+string(x)+" = y "+string(y)+" = & 255 0 ?")


AVIsynth says mode parameter doesn't exist. Moreover, those lines don't return me pixel (x,y) but the whole video with parts in black and another (from the face) in red. An, I don't understand how to merge the pixel mask with the video.

On the other hand, do you have a suggestion to detect all black pixels from a stream and create a mask with ?

[Gavino]

AVIsynth says mode parameter doesn't exist.
It looks like you have an older version of MaskTools.
Try instead with relative=false (and still using the expr keyword, which I forgot was required).

Edit: You also need u=-128, v=-128.

[sirt]

Thanks Gavino. Indeed, those lines :

x=100
y=100
mt_lutspa(relative=false, expr="x "+string(x)+" = y "+string(y)+" = & 255 0 ?",u=-128, v=-128)

return a black window with ONE pixel in white : the (100,100) pixel. So I can select manually a pixel but my problem remains the same :

- How can use mt_merge to merge it with original source where only that pixel will be processed ?

- How can I select several pixels at the same time ?

- How can I select all black pixels from original stream and use them as a mask passed to the "expr" parameter ?

[colours]

My bad; I forgot the mode="absolute" for lutspa, since I just quickly typed out a response. (Also, holy crap please stop using ancient outdated versions of plugins.)

If you want to select multiple pixels, do consider if this is something you absolutely must do in Avisynth, because it's a lot easier to just draw a mask in MS Paint (or your image editing software of choice) then load it in Avisynth with ImageSource.

To merge two clips with a mask, you just use mt_merge(source,filtered,mask), where a sample value of 255 in the mask would use the filtered clip (almost), a pixel value of 0 in the mask would use the source clip, and everything in between provides a linear interpolation. In your scenario, it's probably more useful in the form mt_merge(filtered,source,mask), where white pixels in the mask would use pixels from the source clip. (The parenthesised "almost" is because the formula divides by 256 instead of 255—check the MaskTools documentation if you care.)

Also, I have no clue what you're trying to accomplish, but generally using binary masks is not a good idea. But if you must, mt_lut("17 x - 255 *") gives a binary mask for the black pixels, assuming TV-range input, or mt_lut("1 x - 255 *") for full-range input.

[sirt]

colours, thanks for this answer. Indeed I'm using old versions because I haven't focused on this stuff since last year. But, whatever, I will solve this.

Okay so what I'm trying to do is the following :

1) Detect black pixels from a video stream (not only a frame). Then what you told me about seems to work very well :

AVISource("C\rob.avi")

mask=mt_lut("17 x - 255 *",u=-128,v=-128)

Okay, so I guess this point is solved for the moment.

2) Moreover, I simply want to modify some masks I don't like. Let's say I do this :

AVISource("C\rob.avi")

a=last
b=a.GradFun2DBmod(thr=1.5,thrc=1,mode=2,radius=3,range=1,adapt=2,temp=1)
mask=mt_lutxy(a,b,"x y - abs 255 *",u=-128,v=-128)

return mask


Then please have a look at this :

One random frame from my video (http://oi61.tinypic.com/281g5zd.jpg)
Corresponding mask I get by applying GradFun2DBmod above (http://oi58.tinypic.com/eriqde.jpg)

You notice the mask (WHITE pixels) also contains pixels from girl's FACE and HAIR. So this is my original problem : I would like to delete pixels from FACE and HAIR from the mask in order this filter not to apply on these parts. Do you think that is possible ?

I have been reflecting about possibilites like :

- creating an artificial mask (like a rectangle) and doing the logical intersection with previous one in order not to keep those pixel on girl's FACE & HAIR I don't want to be processed by GradFun2DBmod

- deleting those problematic pixels manually in some way

Am I clear enough ? In fact, this could be applied with any avisynth filter knowing the mask this filter uses. Then, to sum up, what I intend to do is restrict a mask used by an avisynth filter. Of course, the filter itself features parameters for this (like radius there) but the picture above is the best mask I can get : it deletes enough banding on the left for my taste BUT it also affects gir'ls FACE & HAIR which I don't want.

[colours]

TinyPic is a bad image host and resizes your images or something. Upload your screenshots somewhere sane like Imgur or imgbox or anywhere-other-than-TinyPic-and-ImageShack.

You don't have to create a difference mask in order to mask out the head. Just draw a mask over the head in whatever image editing software you have and use that. Pixels that aren't affected by gradfun2dbmod will stay unaffected anyway.

Also, try gradfun3 (http://forum.doom9.org/showthread.php?p=1386559#post1386559) instead of gradfun2dbmod, and tweak the thr and thr_det values until you get something satisfactory.

[sirt]

Okay, so here are the new uploads on Imgdur :

Same random original frame from my video (http://i.imgur.com/kNDT0V2.png)
Corresponding GradFun mask used with above parameters (http://i.imgur.com/7kk9FYj.png)

Now, please have a look at this :

AVISource("C\rob.avi")

a=last
b=a.GradFun2DBmod(thr=1.5,thrc=1,mode=2,radius=3,range=1,adapt=2,temp=1)

#GradFun mask

mask=mt_lutxy(a,b,"x y - abs 255 *",u=-128,v=-128)

#Alternative black mask

mask2=a.mt_lutspa(relative = false, expr = " x 450 < 0 255 ?").greyscale

#Logicial intersection of these masks

mask3=mt_logic(mask,mask2,"or")

#Filtering with GradFun using new mask3

rightSPECIALb=crop(0,0,0,0)
fixSPECIALb=GradFun2DBmod(rightSPECIALb,thr=20,thrc=2,mode=2,radius=3,range=1,adapt=2,temp=1)
fineSPECIALb=overlay(last,fixSPECIALb)
c=mt_merge(fineSPECIALb, a, mask3, luma = true)
ReplaceFramesSimple(c,mappings="25")

Then the resulting mask is :

New mask (http://i.imgur.com/pp63qEw.png)

Consequently previous filter is ONLY applied on the left as I wanted. But it still not good : I've "cheated" by intersecting GradFun with a rectangular mask in order not to retain GradFun's pixels mask which are on girl's FACE & HAIR. But, please keep in mind I'm working on a VIDEO. Such cheat doesn't work in next frame because girl's face has moved ! So I would need an adapative solution !

Your alternative "Just draw a mask over the head in whatever image editing software you have and use that" looks very interesting but I have no idea how to do this and to use then this mask with avisynth filter. Could you give me an example ? Howhever, I guess my problem will remain the same when I move from a frame to another : the mask won't be the same, will it ? I might need something to "track" pixels of girl's FACE & HAIR.

[colours]

I suspected as much, but you never made it clear.

What I meant earlier was that the "GradFun mask" is entirely unnecessary. See below; changes are in boldface.

AVISource("C\rob.avi")

a=last
b=a.GradFun2DBmod(thr=1.5,thrc=1,mode=2,radius=3,range=1,adapt=2,temp=1)

#GradFun mask (unnecessary)
#mask=mt_lutxy(a,b,"x y - abs 255 *",u=-128,v=-128)

#Alternative black mask

mask2=a.mt_lutspa(relative = false, expr = " x 450 < 0 255 ?").greyscale

#Logicial intersection of these masks (also unnecessary)

#mask3=mt_logic(mask,mask2,"or")

#Filtering with GradFun using new mask3

#rightSPECIALb=crop(0,0,0,0)
#fixSPECIALb=GradFun2DBmod(rightSPECIALb,thr=20,thrc=2,mode=2,radius=3,range=1,adapt=2,temp=1)
#fineSPECIALb=overlay(last,fixSPECIALb)
c=mt_merge(/*fineSPECIALb*/ b, a, /*mask3*/ mask2, luma = true)
ReplaceFramesSimple(c,mappings="25")

There's also something strange going on on your end because the masks are supposed to be binary and the screenshots you're uploading certainly aren't, but anyway.

If I'm understanding you right, what you want is to apply debanding only on the background. Given that the background looks more or less like a constant colour (not really, but close enough), you can try tcolormask (https://github.com/tp7/tcolormask/).

Addendum: Here's some general advice. If you're asking for help with something and you think you've got parts of it figured out by yourself, don't just ask for the parts you haven't gotten figured out, because it can turn out that you were doing it wrong all along when other people try to help you with it. (It's not so bad when you're not requesting for help from others, because that also means you understand enough to realise when you hit a dead end.)

Automatically masking things perfectly is equivalent to perfect object recognition in image processing, which is not something that exists yet, or will exist ever. You can either deal with it by drawing the masks manually (this is an awful amount of effort and I seriously don't recommend this unless you're paid for it) or by accepting less-than-perfect masks that don't work in every situation.

[sirt]

Well, in fact I realise I am simply stupid. Doing this logic operation is totally useless regarding your alternative code. It is similar to apply GradFun (or gradfun3, whatever) on the rectangle that has mask3 size !

Howhever, it is still "unproper" : I haven't isolated girl's FACE but rather cut the problematic part by using a RECTANGULAR mask. As I said, it may approximately work on a frame but regarding girl's face move from a frame to another, it means I would have to adapt the mask for each frame...

I've never heard about tcolormask. If you already know how to use it and can show me an example (in my context), you would be welcomed. Actually, I guess what I may try is to create a mask with the color of the background then pass this new mask to gradfun function, is that correct ?

[colours]

Not pass it to the gradfun2dbmod, but to merge the results of using gradfun2dbmod with the unfiltered clip using that mask.

There's already an example provided in tcolormask's readme; you just need to specify the colour(s) in hexadecimal RGB.

[sirt]

Unfortunately, I don't manage to make it work. I've downlaoded files as .zip and moved tcolormask_wrappers.avs to my avisynth plugin folder. Then, importing this avs like this :

Import("C:\Program Files\AviSynth 2.5\plugins\tcolormask_wrappers.avs")

and trying to create a mask :

mask=tcolormask("$FFFFFF /*pure white hex*/
000000 //also black
$808080", tolerance=10, bt601=false, gray=false, lutthr=9, mt=true)

returns me an error : there is no function named "tcolormask". I guess something is missing.

In fact the later .avs files features a function called 'tcolormask" l.87 but I can't see it anywhere.

[StainlessS]

In fact the later .avs files

Perhaps you have to include that too (I dont use that script).

By the way, nice to see you back here Sirt, dont be a stranger.

[sirt]

Hi StainlessS, thanks for your answer. Perhaps, you can get the .avs file from there (https://github.com/tp7/tcolormask/) and download all as zip. Then copy paste the .avs to avisynth plugin folder. I guess it doesn't work because "tcolormask" doesn't exist anywhere.

Otherwise, I check sometimes this forum but as a guest. That's why I haven't logged for months. But, don't worry, I am still there to ask unusual things !

[Reel.Deel]

returns me an error : there is no function named "tcolormask". I guess something is missing.


Sounds like the you need the tcolormask plugin (tcolormask-x86.zip) (https://github.com/tp7/tcolormask/releases/tag/1.2). The script is just a wrapper that allows easy usage for multiple colors. Also this plugin requires the latest AviSynth Alpah4/5 or AviSynth+, it also works with SEt's latest MT version too.

[sirt]

Thanks Reel.Dell I've tried to load tcolormask.dll (x86) with LoadPlugin but it doesn't work : I get an error like LoadPlugin fvf%fO...

Then perhaps the new version of AVIsynth you're referring to is required but I don't want to reinstal AVIsynth and eventually face some compatibility problems.

[Reel.Deel]

Any of the AviSynth versions I mentioned are required, tcolormask plugin will not work with any older versions. If the plugins you have are the latest versions you should not face any compatibility problems. If you want to update AviSynth and run into any problems I'll gladly help where I can. For example, I know you said you have an older version of MaskTools2, update to this one (masktools2-x86.zip) (https://github.com/tp7/masktools/releases) and it should work.

Note that all plugins compiled for AviSynth 2.5.x will work with the the latest AviSynth 2.6/+. The only plugins that are problematic are the older 2.6 plugins. Fortunately there were not many and all the important ones have been updated.

BTW, what AviSynth version are you currently using?

[colours]

t_colormask (http://pastebin.com/vAa8fyjp) has a similar syntax and doesn't require Avisynth 2.6, but you should just upgrade to Avisynth 2.6a5 or Avisynth+.

[sirt]

I've just added last masktools ddl for avs 2.5.8 to my plugins and it doesn't work. Futhermore, using Version() tells me my current version is 2.5.8

So how should I upgrade properly to 2.6 ? Do I need to install/uninstall ?

[StainlessS]

Official v2.6Alpha5
http://sourceforge.net/projects/avisynth2/files/AviSynth_Alpha_Releases/

Will install over original v2.58, it still uses the old "C:\Program Files\AviSynth 2.5" directory (or similar on W7+).

A compressed help (MS CHM file) version of docs is available in the DATA directory of MediaFire in my sig below this post,
is same as installed docs folder but perhaps a little more convienient and easily searched (can put on a hot-key, create shortcut and
add hot-key via properties).

Note, v2.6 dll's only work on v2.6 and not v2.58, although v2.5+ dll's work on v2.6, dont have both in plugs dir.

New Wiki site is here: http://avisynth.nl/index.php/Main_Page

Good Luck.

[sirt]

Well thanks StainlessS. As you explained, I've download 2.6 and Version() reports I've indeed that version. But I still can't load that plugin with :

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\tcolormask.dll")

Perhaps I should load it in another way ?

[Reel.Deel]

Do you have the Microsoft Visual C++ 2012 Redistributable Package (x86) installed? If not then download it here: vcredist_x86.exe (http://www.microsoft.com/en-us/download/details.aspx?id=30679)

If that's not it then what's the error message (if any)?

[sirt]

Reel.Deel, I installed that but the problem remains the same. Do I need to load anything ? I've just runned the .exe.

The error is somewhat weird :

link (http://i.imgur.com/g4YBEEx.jpg)

which refers to the LoadPlugin line that fails. It's like this dll is simply not recognized.

[StainlessS]

I've seen that gobble-de-gook before but dont remember what was the cause.
Here for those too lazy to click on link:

https://s20.postimg.cc/5be1oitml/gobbledegook_zps81f8274f.jpg (https://postimg.cc/image/gnqn6b2bd/)

By the way, cpp plugins are autoloaded from your plugins directory, so when problem fixed, you can remove the LoadPlugin line (assuming its a cpp plug).

EDIT: And if it's not a cpp plugin then need LoadCPlugin.
EDIT: I used PhotoBucket.com to embed image into post.

[Reel.Deel]

@sirt

Hmm have not seen that error before... Are you sure you have the correct 32-bit (x86) plugins and runtime dependencies?

(assuming its a cpp plug).


Yup, tcolormask is a regular cpp plugin.

[sirt]

Thanks StainlessS. I've tried your alternative and I get the following error :

Unable to load C PLugin : "C\...\tcolormask.dll" error=0xc1

Then I got back to tcolormask page (https://github.com/tp7/tcolormask/releases/tag/1.2) and got both x86 and .avs files, dragged them to my plugins file and now I get :

Not An Avisynth 2 C PLugin : "C\...\tcolormask.dll"

Reel.Deel : are you referring to tcolormask ? As I said above I got both x86 tcolomask version and corresponding .avs. Regarding AVS 2.6 I hope I installed the good version. At least, MY previous filters still work and I've also updated masktools as I said.

Does at least this filter works for both of us : just get tcolomask.dll + the .avs from previous page and try :

tcolormask("$FFFFFF /*pure white hex*/
000000 //also black
$808080", tolerance=10, bt601=false, gray=false, lutthr=9, mt=true)

[Reel.Deel]

tcolormask is not a C plugin so LoadCPlugin will not work.

Does at least this filter works for both of us
Works for me.

[sirt]

Okays my bad mates, it is finally working ! Loading the plugin was just useless.

Then the original purpus was to create a mask that would contain the color of the background which, according to user colours, can be modelized by a single color. So, as a reminder, here is the original frame :

original frame (http://i.imgur.com/kNDT0V2.png)

Do you have an idea how I could find out average colour of the background (on the left) ? And then pass it to this function ? I would need to convert it to hexa I think.

[Reel.Deel]

Do you have an idea how I could find out average colour of the background (on the left) ? And then pass it to this function ? I would need to convert it to hexa I think.

Look into AvsPmod (http://forum.doom9.org/showthread.php?t=153248).

[sirt]

Okay, we are on the good way now ! At least, the mask created is better now. But I don't manage to substitue it to GradFun mask. Check the following :



a=last #a = source there

rightSPECIALb=crop(0,0,0,0)
fixSPECIALb=GradFun2DBmod(rightSPECIALb,thr=200,thrc=2,mode=2,radius=3,range=1,adapt=2,temp=1)
fineSPECIALb=overlay(last,fixSPECIALb)
mask=a.tcolormask("$E0C0AB", tolerance=10, bt601=false, gray=false, lutthr=9, mt=true)
c=mt_merge(fineSPECIALb, a, mask, luma = true)
ReplaceFramesSimple(c,mappings="25")

I know it is weird but it doesn't even seem to work like that.

[Reel.Deel]

Try adding Invert() to the mask.

[sirt]

Yes indeed Reel.Deel. Here are an example with another frame :

rightSPECIALb=crop(0,0,0,0)
fixSPECIALb=GradFun2DBmod(rightSPECIALb,thr=2,thrc=2,mode=2,radius=3,range=1,adapt=255,temp=1)
fineSPECIALb=overlay(last,fixSPECIALb)
mask=a.tcolormask("$E0C0AB", tolerance=100, bt601=false, gray=false, lutthr=9, mt=true).invert()
c=mt_merge(fineSPECIALb, a,mask, luma = true)
ReplaceFramesSimple(c,mappings="10")

original frame (http://i.imgur.com/w6khrs0.png)
frame filtering with GradFun + tcolormask (http://i.imgur.com/0STVSTv.png)

It doesn't look bad at all ! What do you think ?

Of course it will require to adapt parameters regarding scene, colors ect.

[poisondeathray]

It's the weekend and all, but you might want to flag your last links NSFW (!)

Not really paying attention to the background there, my eyes drift to other parts....LOL

[Reel.Deel]

It doesn't look bad at all ! What do you think ?

Looks good (and yes, I'm talking about the background :p).

To merge two clips with a mask, you just use mt_merge(source,filtered,mask), where a sample value of 255 in the mask would use the filtered clip (almost), a pixel value of 0 in the mask would use the source clip, and everything in between provides a linear interpolation. In your scenario, it's probably more useful in the form mt_merge(filtered,source,mask), where white pixels in the mask would use pixels from the source clip. (The parenthesised "almost" is because the formula divides by 256 instead of 255—check the MaskTools documentation if you care.)

In this case is it more accurate to use mt_merge(filtered,source,mask) instead of the traditional way? I read about this over at the MaskTools issues (replace mt_merge blending formula) (https://github.com/tp7/masktools/issues/12) but no workaround was given.

[colours]

In this case is it more accurate to use mt_merge(filtered,source,mask) instead of the traditional way? I read about this over at the MaskTools issues (replace mt_merge blending formula) (https://github.com/tp7/masktools/issues/12) but no workaround was given.

In this case it actually doesn't matter which way it's done (assuming you invert the mask as appropriate) because a debanding filter doesn't change the value of a pixel much anyway. 256/255 times any small integer would still round to the same small integer.

[sirt]

Hi,

I come back there to ask for a few more questions. Okay, so let's say I still use my Vimeo source. Please find out below, a random frame :

Original frame (http://i.imgur.com/uvDE5jx.png)

Afterwards, I use mt_edge to detect borders according to a prewitt kernel. For some reason, imagine I only need to get borders of singer on the left. I did something like this :


a=my_source

mask1=a.mt_edge(mode="prewitt",chroma="128",thy1=50,thy2=100)
mask2=a.mt_lutspa(relative = false, expr = " x 732 > 0 255 ?").greyscale
mask3=mt_logic(mask1,mask2,"or")

return mask3

The result is the following mask (in white) :

Borders for the singer on the left (http://i.imgur.com/bFxwU95.png)

Finally, my question : it is clear I've detected borders of that guy. How could I create a mask with what's "inside" of him including boarders pixels if I want ? That way, I could somehow "track" the object corresponding to pixels forming part of the guy ! As a consequence, I could apply any filter to an object tracked (by adjusting borders if necessary).

It is clear I need to create a mask with pixels inside of those borders.

[sirt]

What I don't understand is why the intersection of those masks :

Borders (http://i.imgur.com/jQR7D2c.png)
Rectangle (http://i.imgur.com/9qxCCdT.png)

is not the singer shape ?

mask1=a.mt_edge(mode="prewitt",chroma="128",thy1=50,thy2=100)
mask2=a.mt_lutspa(relative = false, expr = " x 675 > x 338 < | 0 255 ?")
mask3=mt_logic(mask1,mask2,"and")

[Gavino]

mask3=mt_logic(mask1,mask2,"or")
"or" gives you the union.
You need "and" to get the intersection.

[sirt]

Yes, well thanks Gavino, I included a mistake in my previous post. But look, using "and" give me this for the above masks (mask1 and mask2):

Intersection "and" (http://i.imgur.com/oeumV66.png)

And "or" :

Union "or" (http://i.imgur.com/X1EmtXZ.png)

Both seem not correct to me. Or, I simply don't understand what occurs ! When doing the intersection between white rectangle and borders, I expect to get pixels of the borders and those inside those borders - for the man on the left only. The same should happen with the union : I should only get white rectangle (and not borders of the girls). I think it is probably possible to do what I want but it is not set correctly.


EDIT

Well in fact it is logical : the intersection (for example) is supposed to give me the pixels that either mask (mask1 and mask2) has in common. Then those white pixels "inside" the borders that are clearly featured on my rectangle don't belong to both masks. Actually, those belonging to both are exactly the borders of the man ! Then there is no contradiction. Unforunately I still don't understand how I could do what I want.

Inverting mask1 for example and doing the intersections gives me :

After "and" + inverting mask1 (http://i.imgur.com/iG4F92d.png)

It's exactly the rectangle without the borders. But what I definitely want is what's inside the borders only.

[Gavino]

Your results are what I would expect.
With "and", you get white pixels in the result where the pixel is white in both masks. This gives you the outline of the singer on the left.
With "or", you get white pixels where either mask has a white pixel, giving the rectangle plus the outline of the singer on the right.

With this method, there is no way to select the (black) pixels inside a shape - mt_logic() works with individual pixels and has no knowledge of 'inside' or 'outside'.
I don't know how this could be done - possibly something using mt_expand() might work, but I don't fully understand what it does!

[Shirtfull]

@ sirt

Try using overlay to extract foreground

Your original frame has a solid background, make an image of the background (mosaic from different frames)
Import image, "difference" source and image with overlay and then binarise.

[sirt]

Thanks Gavino, I don't understand mt_expand either. It's terrible because I would know how to do what I want trough MATLAB for example : reconstruct pixels inside the man by a morphologic operation.

Shirtfull thanks for your answer. Unfortunately, I don't really understand overlay and how it could be used to extract the foreground. Would you have an example (if possible with the original frame) ?. Moreover, my purpose would be to track the singer on the left. It appears your alternative works for one single frame and I will have to do that for each frame one by one. Or maybe I misunderstood.