Joel Cairo
11th January 2006, 12:04
Sorry for being a bit of a broken record on this topic, but it's one that is somewhat near and dear to my heart...
It's my understanding that the current version of MVTools by Fizick and Manao generates pixel-tracking motion vector information over the course of a video file, which is useful in their various motion interpolation processes. However, I'm just wondering if it wouldn't be useful for at least one other application.
Could an application be created that would operate as follows?:
a) User pulls a .bmp still frame from the 1st frame of their video file (or video file segment, as the case may be).
b) User colorizes this still using whatever process they wish, to create a color "keyframe".
c) The application takes the color channel information of the "keyframe", and overlays it on the 1st frame of the video file to be processed. [it should replace the original chroma information of the 1st frame with the chroma information from the "keyframe"].
d) The video file's motion vectors are then tracked.
e) The application uses the motion vector tracking information generated in step d) to output a new image which has the same luma as the next frame in the original video file image sequence, but has new chroma information derived from the color "keyframe" assigned as the colored objects move through the video image. This way, the new colorization is (somewhat) automated.
I don't know if it's possible to assign this "pixel by pixel" as an object moves, or if it's better to let the application define boundaries for an object, and have the color follow that way-- I would leave this to the experts.
And obviously, a new keyframe will have to be prepared fairly often (each time that new objects enter a frame, or when a scene change occurs, etc.), but if this combination of tracking and color-channel substitution is possible, I think this would be a great start toward automating the process.
Is anyone interested in attempting to code this? If it's possible to do this chroma substitution on a pixel-by-pixel basis, it would seem to be a fairly basic use of the motion vectors that MVTools now creates... if I'm talking about something that's a lot harder than it sounds, please let me know.
Thanks for considering the idea, folks!
-Kevin
It's my understanding that the current version of MVTools by Fizick and Manao generates pixel-tracking motion vector information over the course of a video file, which is useful in their various motion interpolation processes. However, I'm just wondering if it wouldn't be useful for at least one other application.
Could an application be created that would operate as follows?:
a) User pulls a .bmp still frame from the 1st frame of their video file (or video file segment, as the case may be).
b) User colorizes this still using whatever process they wish, to create a color "keyframe".
c) The application takes the color channel information of the "keyframe", and overlays it on the 1st frame of the video file to be processed. [it should replace the original chroma information of the 1st frame with the chroma information from the "keyframe"].
d) The video file's motion vectors are then tracked.
e) The application uses the motion vector tracking information generated in step d) to output a new image which has the same luma as the next frame in the original video file image sequence, but has new chroma information derived from the color "keyframe" assigned as the colored objects move through the video image. This way, the new colorization is (somewhat) automated.
I don't know if it's possible to assign this "pixel by pixel" as an object moves, or if it's better to let the application define boundaries for an object, and have the color follow that way-- I would leave this to the experts.
And obviously, a new keyframe will have to be prepared fairly often (each time that new objects enter a frame, or when a scene change occurs, etc.), but if this combination of tracking and color-channel substitution is possible, I think this would be a great start toward automating the process.
Is anyone interested in attempting to code this? If it's possible to do this chroma substitution on a pixel-by-pixel basis, it would seem to be a fairly basic use of the motion vectors that MVTools now creates... if I'm talking about something that's a lot harder than it sounds, please let me know.
Thanks for considering the idea, folks!
-Kevin