How to detect if a frame is a translation of another ?

Le Furet · 13th December 2003, 15:37

Hello

I want to make a filter that can create a frame between two successive frames of a travelling, to make it smoother (not blending). More generally, it should be able to recreate entire travelling just by specifying first frame, last frame, and the number of frames between.

So, I must find a way to determinate what was the translation vector between 2 frames.

I've read the sources of some motion compensated filters, but it seems to me that those were looking at only very small moves. It isn't suitable for my purpose. And I'm afraid allowing those algorithms to look for more important moves would lead to such a long time for the filter to process that it would become unusable.

So, I'm looking for others ideas.

I thought a thing : suppose that I analyze the pictures, and by one mean or another, I distinguish several "objects" in a frame. Suppose that to each object, I associate a sequence of "invariants by translation", for example area and perimeter, maybe others. I could then determine how these objects moved between 2 frames by comparing position of objects that have nearly the same "invariants", no ?

Since I'm a bad coder, and so coding the simlest of my ideas could spend to me hours of work, I would like ton know if such an approach was already used by one of you, and if it worked.

Unless, of course, you have another idea, better (i've also thought of using Motion vectors generated by first pass encoding, but I must say I don't know much about them, how to get them, how to understand them, and so on)

Manao · 13th December 2003, 16:02

Your invariants are also ( alas ) invariant by rotation also. You need to calculate inertia vectors ( which is not complicated ).

But anyway, it's not a good way to go. You're thinking in terms of objects, which means you need a way to know how evolved an object between the two frames. So you have to be able to :

- find the objects
- tell where it has gone

Which are both very complicated.

Motion vectors are a better way to go. But, they are (afaik) limited in length( inside the codec ), so if the deplacement is too big ( as you seem to fear ), you won't catch it. To get them, you have to consider a block, and then find in the previous frame the block that is closest to this one, by translation ( closest by any distance you want, the more common being the mean square error ).

But, of course, that leads to to much calculations, so you have to reduce the search, by searching to only a limited length, and by trying to get only local minimas instead of global minimas.

But once you've got them, their use is straightforward. You just have to allow some vectors not to be aligned to the others ( because you won't catch the same vectors each times, because of noise / moving details / etc... )

clima · 25th December 2003, 10:59

I'm tempted to reply in French

I know it won't really help you, but Dynapel MotionPerfect is worth a try. But I don't expect to be any source available about a commercial software.

t_2 · 25th December 2003, 21:49

I had a simlar problem. I have to film fast motion objects -people flailing their arms about - against a color key screen for a film we are making. In order to get good seperation between the screen and the motion areas, i.e., so that the screen color wouldn't "bleed through" in the motion areas, I filmed with very high shutter speeds. This gave me very good separation - i.e., sharp images, but when I played back the video instead of smooth motion I saw mulitple hands. I was able to make the motion look smoother by putting the moving object on the background clip using Mask and then blurring it with the background in the motion areas. It made the motion look smooth again. Is this the problem you are up against?

If not, then you are probably just trying to increase the motion resolution of a clip, i.e., increase the frame rate, or do a slow motin that is smooth. If this is your problem, then you are either more interested in getting the job done well, or else you may be more interested in getting the job done with a filter in AVISynth. I can't help you if the latter is the case, but if you have an actual problem that you are contending with and want to get it done right, then the best way to do it is with a morph program. The one I use is spectacular, it's called WinImages Morph and comes bundled with a super powerful effects engine called FX7.

www. blackbeltsys.com for more info

There are also free Morph programs on the Internet which will do the same for you. Hope this helps.

morsa · 27th December 2003, 01:06

How about using a border detection code (like Msharpen has)on the first and following frame and then feed this borders as the mesh mask for Libmorph GPL library for morphing.
I guess this approach could be far better than Barry's Framedoubler.
Don't know, may be I'm a bit silly.
Tell me what you think.

Here is the link to Libmorph.The library is inside Xmorph source code tree.

http://sourceforge.net/projects/xmorph/

note: forget it, I underestimated the problem to generate a reliable mesh.

13th December 2003, 15:37	#1 \| Link
Le Furet Registered User Join Date: Apr 2002 Location: France Posts: 51	How to detect if a frame is a translation of another ? Hello I want to make a filter that can create a frame between two successive frames of a travelling, to make it smoother (not blending). More generally, it should be able to recreate entire travelling just by specifying first frame, last frame, and the number of frames between. So, I must find a way to determinate what was the translation vector between 2 frames. I've read the sources of some motion compensated filters, but it seems to me that those were looking at only very small moves. It isn't suitable for my purpose. And I'm afraid allowing those algorithms to look for more important moves would lead to such a long time for the filter to process that it would become unusable. So, I'm looking for others ideas. I thought a thing : suppose that I analyze the pictures, and by one mean or another, I distinguish several "objects" in a frame. Suppose that to each object, I associate a sequence of "invariants by translation", for example area and perimeter, maybe others. I could then determine how these objects moved between 2 frames by comparing position of objects that have nearly the same "invariants", no ? Since I'm a bad coder, and so coding the simlest of my ideas could spend to me hours of work, I would like ton know if such an approach was already used by one of you, and if it worked. Unless, of course, you have another idea, better (i've also thought of using Motion vectors generated by first pass encoding, but I must say I don't know much about them, how to get them, how to understand them, and so on)

27th December 2003, 01:06	#5 \| Link
morsa the dumbest Join Date: Oct 2002 Location: Malvinas Posts: 494	How about using a border detection code (like Msharpen has)on the first and following frame and then feed this borders as the mesh mask for Libmorph GPL library for morphing. I guess this approach could be far better than Barry's Framedoubler. Don't know, may be I'm a bit silly. Tell me what you think. Here is the link to Libmorph.The library is inside Xmorph source code tree. http://sourceforge.net/projects/xmorph/ note: forget it, I underestimated the problem to generate a reliable mesh. Last edited by morsa; 30th December 2003 at 11:53.

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

13th December 2003, 16:02	#2 \| Link
Manao Registered User Join Date: Jan 2002 Location: France Posts: 2,856	Your invariants are also ( alas ) invariant by rotation also. You need to calculate inertia vectors ( which is not complicated ). But anyway, it's not a good way to go. You're thinking in terms of objects, which means you need a way to know how evolved an object between the two frames. So you have to be able to : - find the objects - tell where it has gone Which are both very complicated. Motion vectors are a better way to go. But, they are (afaik) limited in length( inside the codec ), so if the deplacement is too big ( as you seem to fear ), you won't catch it. To get them, you have to consider a block, and then find in the previous frame the block that is closest to this one, by translation ( closest by any distance you want, the more common being the mean square error ). But, of course, that leads to to much calculations, so you have to reduce the search, by searching to only a limited length, and by trying to get only local minimas instead of global minimas. But once you've got them, their use is straightforward. You just have to allow some vectors not to be aligned to the others ( because you won't catch the same vectors each times, because of noise / moving details / etc... )

25th December 2003, 10:59	#3 \| Link
clima Registered User Join Date: Jun 2002 Posts: 33	I'm tempted to reply in French I know it won't really help you, but Dynapel MotionPerfect is worth a try. But I don't expect to be any source available about a commercial software.

25th December 2003, 21:49	#4 \| Link
t_2 Registered User Join Date: Apr 2003 Location: USA Posts: 107	I had a simlar problem. I have to film fast motion objects -people flailing their arms about - against a color key screen for a film we are making. In order to get good seperation between the screen and the motion areas, i.e., so that the screen color wouldn't "bleed through" in the motion areas, I filmed with very high shutter speeds. This gave me very good separation - i.e., sharp images, but when I played back the video instead of smooth motion I saw mulitple hands. I was able to make the motion look smoother by putting the moving object on the background clip using Mask and then blurring it with the background in the motion areas. It made the motion look smooth again. Is this the problem you are up against? If not, then you are probably just trying to increase the motion resolution of a clip, i.e., increase the frame rate, or do a slow motin that is smooth. If this is your problem, then you are either more interested in getting the job done well, or else you may be more interested in getting the job done with a filter in AVISynth. I can't help you if the latter is the case, but if you have an actual problem that you are contending with and want to get it done right, then the best way to do it is with a morph program. The one I use is spectacular, it's called WinImages Morph and comes bundled with a super powerful effects engine called FX7. www. blackbeltsys.com for more info There are also free Morph programs on the Internet which will do the same for you. Hope this helps.