Project for the brave or Untime Starved

[MrTVideo]

There is a need out there in Pinnacle land for a filter/function that will Highlight a face or object and follow that person/object for the duration of the clip.

See how good you Girls/Guys are. "I know which side my bread is buttered"

I am thinking of a modified denoise filter that can recognise a pixel pattern and apply a round, rectangular or other shape of a given size that becomes the highlight shape.

From what I have seen on this forum there exists some very clever brains out there, who cares about your bodies.

So who has some ideas on achieving this, to lighten up the hearts of those Pinnacle users who have requested it. All credit will be made known to the people on that board to the geniuses who come up with a solution. One thing I am the layman so explain in one sylable sentences.

Do I think you can do it "Bloody oaf" " Aussie slang


in anticipation

Terry :)

[Mug Funky]

hmm... intriguing, but to what end?

any code i could write (after several years?) would operate at comparable speeds to simply going through the video and cropping around te face.

that said, i'm trying to think of ways to descriminate between face-like and non face-like elements in an image. i can imagine some amusing false-positives to come out of it though. just think of how many things look like faces.

there's plenty of assumptions you can make about the positions of features centred around the eyes, but these go askew if, for instance the light comes from underneath (i'm thinking the triangle between the eyes and the shadow cast by the nose would be the easiest to spot).

[SoonUDie]

Anyone interested should take a look at neural-network based face recognition. Defintely the way to go, from what I've read.

[mf]

You want anonimity censoring, right? Well, one assurance, as far as I know, those have always been done by hand :D.

[MrTVideo]

More thought, the area first selected is not going to move much from frame to frame and given the first reference. May be the function can search for simularities frame by frame with a small change in position each time. I am not proposing that the function searches for a face or object,I am saying that the object or face would be selected in the first frame and the highlighted shape would track frame by frame.

I am trying to come to grips with the windows API in video stream handling. I do not hope to compete with the new security face identification software.

It can be done. I'm an old bastard but a stubborn one. :)

Thanks for all the input on this so far.

Terry

[Leuf]

I can think of two possibilities off the top of my head, but neither are the sort of automated filter that you want. But as I see it, if this is for blurring out identies then you're going to have to watch the video after to make sure it worked 100% anyway, so I'm not sure if a little more manual way is going to be that much slower. At the very least you'd want the capacity to override the filters choices so it makes sense to me to start as a manual process and then attempt the automation once that is working.

One way would be to specify the starting position and then each frame throw up a dialog box prompting for change in x and y, defaulting at 0 so if no change is needed you just hit enter. You could also have a grid of buttons for the most common values to cut down on the need to input a value. While this would be a pain in the butt for very long clips, for shorter ones it's not too bad. You could speed it up by only prompting every 2nd, 3rd, whatever frame. I think you could get reasonably close to realtime with this method. I'm not sure whether it's possible / a good idea to throw a dialog box from an avisynth filter, but I've done it from vdub filters. For a vdub filter this would have to take two 2 passes, one to get input and record the choices and then another to process, in avisynth I think you could do it in 1 pass.

Another way would be to have a filter that reads a script which would be able to assign a new position for a given frame or tell it to move from a to b from frame n to m. This would lend itself more toward future automation with override capability, much like the decomb filter.

Neither of these would be difficult to write. As far as automating the decisions, I haven't a clue.


-Leuf

[MrTVideo]

Hmmm! Thats a thought I will do some studying. The highlighting is actually for making for instance a player stand out from team action a boy playing soccer. Its for home movies.

[Kurosu]

You maybe could take a markovian approach: initiate the filter on a range of frames with a position where the face is bound to appear. By both region growing and classification, you would end up with a class more or less matching the face's color parameters (using a texture as class basis could be good, but more complex) with a confidence level you can set. And by updating the class every frame while processing, you might would adapt it to small variations. This way, you wouldn't have to do track the moving object.

After filtering a bit orphan pixels, you'd then have an area where the face is likely to appear. Next step is now to apply whatever filtering to the area you extracted.

Given the complexity of this, it's indeed not something for lazy developpers.

[MrTVideo]

Wow Kurosu

Admire your paradine you really think outside of your square. That is the approach that I am studying. I am new to AVIsynth and long in the tooth in C have not programmed in 20 years. could not even remember what pointers are. Back to creating simple functions "H" files and about to create a dll ,I am getting there though. I do not think it is that hard and I have to get up to speed with pixel interogation. Believe it or not it is simular to a program I did in final calibration of water meters by taking images through the glass view panel and measuring the needles movement by time and water flow. Turned out that my test was more accurate than the mechanical method they set them with. So! I hope to post a working function of this highlight function before you guy's do. Actually I am going against my belief in not expecting others to do what you cannot do yourself.

Gee! this stuff is fun :D

"EDIT" spell check never was good at that

[MrTVideo]

I am now looking into a possible nframe by nframe adjustment with the possibility of using the layer function + conditionalread for making the change of the highlight clip position per 1stClip (position of object highlighted), the highlight being the 2nd clip sized proportionally. The approach I shall try is:-

Use Mediator for the front end(no coding required for the GUI) use program calls to script and create display clips for Mediator to use for positioning of the 2ndClip Highlight over the 1stClip video. This information will enable a program call to write the AVS script plus the x y movement file for the ConditionalRead function for movement of the 2nd highlight clip in relationship to the video.

The second highlight clips will be pre-made of different shapes and colour for highlighting the object (this will take a bit of trial and error to achieve the right effect)
I have played around with the layer and conditional read and it seems quite possible condidering the arguments built into the functions.

Oh what fun, am I enjoying this, almost as good as NLE. Actually most of the things discussed in this forum I acomplish quite easily with Studio without having to reinvent the wheel.

Anyway there is always a need for a better mousetrap ;)


sorry spell check again :(

[MrTVideo]

Ok! not Layer but overlay and conditionalRead using greymask seems to be all thats needed, I will code a script writer to do the frame by frame movement of the mask using a text file to feed ol_x_offset & ol_y_offset. Difficult part is to setup a nframe by nframe display with a x y overlay for recording the position of the object for inclusion in the x y offset text file. Going to search for a GNU display program with source, as the information really needs to be set before feeding to Virtualdubmod. Getting closer. :)

[MrTVideo]

Have to tidy it up but this works

a = AVIsource("D:\Tree_Fall.avi").Flipvertical
b = AVIsource("D:\Round_1.avi").BilinearResize(80, 60)


overlay(a,b, y = 100, x = 110, mode="Add",opacity=0.2, pc_range=False)
ConditionalReader("xoffset.txt", "ol_x_offset", false)

Clip b is made from an image of a grey circle in a black square I made a 1 min clip of this and I will use ConditionReader to track the object nframe by nframe, depending on the amount of movement of object.

I have the task now of feeding the pixel x y positions into a file for scripting.

This highlights with a translucent circle that can be sized to suit.

[Mug Funky]

you could use imagereader combined with loop(number) instead of a 1 min long avi...

[MrTVideo]

Thanks I will look into it. I am working on putting together a simple viewer with x y pixel positions that will be saved by mouse action.
I tried the highlight effect with a demo copy of Media Studio and it is not easy and takes some time, so to nail this one would be indeed be satisfying. Keep in mind the SVCD's I output(Haven't a DVD burner yet) are top class with Menu's chapters etc. assembled with Studio 8. but there are a few things I cannot do easily, Reverse was one, but that I now have in my GUI combination, Thanks to this forum I also have still and moving picture in picture, so other than the cleanup of old 8mm captured film files, Avisynth has answered most of my needs. I hope to give my Pinnacle friends a GUI with all the innovations I have learned here. As you Mug Funky have kindly responded to most of my posts I thank you most sincerely. I have a fair bit of work to do putting this together so I will wait until I have something to offer before coming back.
Anyone wanting to contact me can through my site and critique my present GUI. Nothing like constructive criticism. Till then Cheers to all.
terry_prism@geocities.com
http://www.geocities.com/terry_prism/index.html

[MrTVideo]

Actually getting there, Almost coded a frame by frame cursor position program to write the x,y positions for Overlay. The GUI will follow.
I am retired and this is more fun than "Tomb Raider"