Seam Carving

[newWang]

How long before we see a Avisynth filter that resizes 4:3 to 16:9 by this method?

Seam Carving (http://en.wikipedia.org/wiki/Seam_carving)

[Zarxrax]

That's a very awesome resizing algorithm, but I doubt it could apply to video. The way the algorithm works, I imagine you would see some VERY odd results if you tried using it on video. Maybe I'm wrong though.
Since there are a number of implementations available on that wiki, perhaps someone could render out a sequence of a few hundred still frames and resize them, and see how it turns out?

[scharfis_brain]

I guess it will come out very instable in temporal domain.

One will have to ensure with motion tracking that over a certain amount of time always the same seams are being used in relation to the movement in the image.

Also noise may become an issue, cause it also will make the seams take another way through the image.

[mikeytown2]

ImageMagick has this
http://www.imagemagick.org/Usage/resize/#liquid-rescale

[Selur]

nice, they are even trying this on video now, see: http://www.faculty.idc.ac.il/arik/

[Terka]

nice, they are even trying this on video now, see: http://www.faculty.idc.ac.il/arik/

looks interesting. maybee it could be useful in avisynth world.

[vcmohan]

I have a plugin to do seam carving. Please see seamer (http://forum.doom9.org/showthread.php?t=135735). This has the carving feature but not the erasing or protecting features as it will be impractical to input the spatial coordinates for video usage.
The video you have provided link for only demonstrates in a video how seam carving works and not that it is useful for carving a video.

[mikeytown2]

I have a plugin to do seam carving. Please see seamer (http://forum.doom9.org/showthread.php?t=135735). This has the carving feature but not the erasing or protecting features as it will be impractical to input the spatial coordinates for video usage.
The video you have provided link for only demonstrates in a video how seam carving works and not that it is useful for carving a video.

Sweet stuff!
Here it is for video's
http://www.youtube.com/watch?v=AJtE8afwJEg

[vcmohan]

Thanks. Has seen the video but could not get to understand what is being done. Is there a written paper on the subject that can be freely accessed?
It is possible to do seam carving satisfactorily if the objects do not move about a lot in the scene or the camera is not panned. The retargeting in case of the last shot (basket ball) was worse than simple rescaling. Even after using face detection algorithm, parts of the hair was removed.
It will not be possible to attain uniform reduction for whole video clip without introducing unacceptable geometrical distortions.
I still think that as some parameters need to be input scene by scene and some restrictions to be specified, it is still not a practical proposition for video.

[mikeytown2]

ftp://ftp1.idc.ac.il/Arik_shamir/SCweb/vidret/index.html
ftp://ftp1.idc.ac.il/Arik_shamir/SCweb/vidret/vidret.pdf

[vcmohan]

Thanks. At third try I could get this link stable for full download. I did a quick read and now understand a little better. Hope to get a working plugin some time (soon?)

[vcmohan]

One of the rules set out in that paper is that a seam need to be continuous not only spatially but also temporally. I personally think that this condition of temporal continuous is not valid. If it is valid then the seam position will change with the fps. An object moving across can not be restricted to a speed of one pixel per frame. I am therefore unwilling to use that formulation.
Regarding removing certain parts or protecting certain parts of frame I will introduce those features but use will be cumbersome.

[mikeytown2]

If you could choose between the Forward and Backward Energy methods that would be appreciated.

For "removing certain parts or protecting certain parts" I would use a 2 masks; one for removal, one for protection.

My take on their temporal method is that the seam from one frame should try to be close to the seam in the next frame. Have the difference between the 2 seams be a user set percentage, thus 0% is 1px (like in the paper) 100% is the entire frame (function doesn't consider the time dimension, its dumb). So if the clip is quite static, the user can set a low percentage, it there is a lot of movement then the user sets a higher percentage, thus allowing seams to be placed further apart, once compared temporally.

[vcmohan]

If you could choose between the Forward and Backward Energy methods that would be appreciated.

For "removing certain parts or protecting certain parts" I would use a 2 masks; one for removal, one for protection.

.

I have already done these and will come out after polishing the code.

On second thoughts the seam in the spatial domain also suffers from similar reasoning as of the temporal domain. If one resizes the seam continuity will be lost.

As it is seam carving runs so slow if temporal factor is also included it will be almost freeze. So I am not planning it for just now.

[vcmohan]

My plugin is ready now. It is named ReTarget. A brief description is here (http://forum.doom9.org/showthread.php?t=139583). At my plugin page the full description can be seen.