Limitations of GMC

[rjc7394]

I borrowed this from XVID FAQs page:
http://ronald.vslcatena.nl/docs/xvidfaq.html#C3f
GMC stands for Global Motion Compensation and that pretty much tells the story of what it does. If used, it will look at the whole frame and see if there is an amount of motion that all the parts of the frame have in common. It will then take this amount of motion and put it in a single value. The parts of the frame are the macroblocks, and the amount of motion is called a 'motion vector' which has both a direction and a value (as a sort of two-dimensional X,Y value) .

All the macroblocks normally have their own motion vectors, but with GMC the one motion vector that they all have in common (that's why it's called 'Global') will be compensated and put into a single motion vector. Some macroblocks' movement will be completely compensated for by the GMC vector, getting completely nullified by the compensation process. These macroblocks' motion vector will then be removed, as it is the same and is only extra information. The possible benefit is that you can remove many or all the motion vectors of the macroblocks (or even the blocks themselves if there is no altered texture information) in a frame by a single value, thereby making it much smaller.
Note however that this is for one-warppoint GMC. With Multiple warppoints the proces is much more complex, but the principle is the same.

Warppoints, hmm...what's a warppoint?

A warppoint is a motion vector that defines a displacement of one *edge* of the video. Take a piece of paper and move it by its edges and you'll see what I mean.
-The first warppoint defines displacement of top-left edge. If it's the only warppoint, the rest of the picture just has the same vector and the whole picture moves. Think panning.
-A second warppoint defines displacement of top-right edge (not *precisely* true but close enough without getting too technical). Together with the first warppoint, this is enough to define panning *and* zoom. Note that it could be used to define panning and rotation instead, but isn't.*
-A third warppoint means displacement of down-left edge and three warpoints are enough to define panning, zoom and rotation.
-A fourth warppoint would create perspective-like movement.
Note that XviD's GMC uses 3 warppoints, whereas DivX's GMC uses only one. Warppoints are stored in the frame's header, and only if they are used.
My question is: What size are these macroblocks that are compensated for with GMC and does anyone feel that XVID GMC works well or not? I have a hard time telling the difference.

[foxyshadis]

It's one of those things that rarely provides enormous efficiency benefits, normally provides low to no benefits, and can cause its own weird artifacts with low bitrate. Usually it's best on cg or heavily filtered video, grain tends to render it useless, as well as anytime motion isn't "global". Xvid's GMC would be nicer if it had a 1-point options, imho, being faster and more compatible; but even then, I doubt I'd want to turn it on often.

[unskinnyboy]

@foxyshadis, This is the first time I am hearing that GMC on low bitrate causes artifacts. And I can't think of a reason why. Is that based upon your own findings or was there already a discussion here about that which I missed?

About grains rendering it useless, do you say it is so because the random graininess of the video throws off motion detection? While I can partly buy that theory, I would imagine this would happen only with *tons* of grain. I find it incredulous that the video have to be heavily filtered.

My understanding of the usage of GMC is that if you don't mind the long encoding times and also don't care about standalone compatibility, by all means use it. If there is a need for GMC to kick in for a scene, it will kick in, thereby converting the P-frames into GMC'd frames. If not, P-frames will still be encoded as usual, no harm done.

[foxyshadis]

At low low bitrate, the artifacts I see seem to come from not having the bits to counteract the global motion where it doesn't match up, so there's weird warping or blocks sliding around in an object that doesn't move with the rest of the frame. But at the quants I'm talking, higher than normal 1-cd backups, bit starvation is going to cause ugly artifacts anyway, it just changes their nature.

I suppose before I start talking about what disrupts it and what it works on, I should go back and do more tests instead of relying on memories a year or two old. (Based on personal tests, I used to use it a lot but stopped after them. I haven't seen many others' results with GMC, I would be interested.)

[Teegedeck]

I don't see why grain should render GMC useless. GMC adds a global offset to all motion vectors to my knowledge; GMC is something added to normal MVs, not replacing them.

Still, the speed penalty makes it hard to recommend GMC. It makes for (imperceptibly?) higher quality and very minimal space savings. Features that only account for such minimal gains should be more useful at very strong compression, where every bit counts. But I'm not quite sure whether there is any quality left to gain by GMC under such circumstances - and space savings around 0.2 - 0.4% are really not so much to write home about.

One-point-GMC is even less useful than full GMC. I think it was only added to DivX so they could say 'DivX supports GMC!".

[celtic_druid]

XviD's GMC code got faster a little while back. That's if it doesn't crash for you. Personally I don't use it. But still the speed impact shouldn't be as bad as it used to be.

[unskinnyboy]

At low low bitrate, the artifacts I see seem to come from not having the bits to counteract the global motion where it doesn't match up, so there's weird warping or blocks sliding around in an object that doesn't move with the rest of the frame. But at the quants I'm talking, higher than normal 1-cd backups, bit starvation is going to cause ugly artifacts anyway, it just changes their nature.
I don't understand what you mean by "not having the bits to counteract the global motion" - why counteract? But I would give this much that GME as a whole is prone to warping errors, depending on how efficiently (or inefficiently) the global MV was computed from the local MVs. GMC as a whole doesn't do a lot of good for your average encode of your movie, but depending on the source, it should save some bits whereever possible. For very low bitrates, you have other problems to worry about rather than GMC, but that would also probably mean you are aiming for a very high compression, the main scenario where GMC makes sense.

GMC adds a global offset to all motion vectors to my knowledge; GMC is something added to normal MVs, not replacing them.
Once two candidate vectors are computed from the local motion vectors, one of them will be used. From this point, the local motion vectors are replaced by the global one till the duration the "GMC effect" lasts. That is my understanding, at least.

[sysKin]

All the stuff about counteracting or adding global and local motion is completely incorrect.

GMC does not add itself to motion vectors in any way; it replaces the concept of local motion vectors entirely. For macroblocks for which it's used of course. Macroblocks for which GMC is not used are not affected/influenced by GMC in any way.

In other words: for any macroblock, either you have global motion or you have a motion vector, you never have any combination of both.

[unskinnyboy]

GMC does not add itself to motion vectors in any way; it replaces the concept of local motion vectors entirely. For macroblocks for which it's used of course. Macroblocks for which GMC is not used are not affected/influenced by GMC in any way.

In other words: for any macroblock, either you have global motion or you have a motion vector, you never have any combination of both.
Thanks for the clear-cut answer, but isn't that the same thing which I said last? That once the global MV is calculated, it would replace the local ones?

[sysKin]

Thanks for the clear-cut answer, but isn't that the same thing which I said last? That once the global MV is calculated, it would replace the local ones?
Well not quite.
Global motion is not calculated, it's sent in frame's header as three warppoints. There's no more vectors to be calculated, but there's a picture to warp.

This might be what you meant, but let's phrase it as "once the reference picture has been warped"

In the second part, you said "it would replace the local ones". Again, close but let's explain it. How about "if warped picture is used, there's no motion vectors [local on any other]". The "if" is a per-macroblock decision.

If you think you might not understand the concept entirely, think of XviD suddenly having two reference frames (like x264's ref=2). One of them is the usual last-coded frame. The second is its copy, warped and zoomed like in gimp/photoshop. Then, for every macroblock, either of them is used (like in x264) with the extra twist that the warped reference, if used, is forbidden from having motion vectors.

[Manao]

GMC, properly implemented, can never degrade the quality, since you can turn it on/off on a per frame basis.

Now, GMC is hard to implement. XviD's implementation is far from perfect ( since users report it sometimes lowering the quality ), but I don't know a good implementation.

GMC is complicated because :
- you have to estimate the global motion parameters. XviD does that with the motion vectors found during a motion estimation, followed by a refining. It's definitely not the quickest way nor the best, but at least it's rather easy to implement.
- once you have them, you have to estimate whether it would be usefull or not in the frame. There again, XviD could be better. Especially when using RD, since it can easily choose optimally when RD is used.

One-point-GMC is even less useful than full GMCIt allows to skip to be coded with a motion vector. That a huge advantage in some cases, but not at the quantizers which you use ( since at those quantizers, you hardly skip ). It's really a low bitrate stuff.

[unskinnyboy]

Well not quite.
Global motion is not calculated, it's sent in frame's header as three warppoints. There's no more vectors to be calculated, but there's a picture to warp.

This might be what you meant, but let's phrase it as "once the reference picture has been warped"

In the second part, you said "it would replace the local ones". Again, close but let's explain it. How about "if warped picture is used, there's no motion vectors [local on any other]". The "if" is a per-macroblock decision.

If you think you might not understand the concept entirely, think of XviD suddenly having two reference frames (like x264's ref=2). One of them is the usual last-coded frame. The second is its copy, warped and zoomed like in gimp/photoshop. Then, for every macroblock, either of them is used (like in x264) with the extra twist that the warped reference, if used, is forbidden from having motion vectors.
I think the confusion came from the fact that I was referring to GME and GMC interchangeably. The computation of the global MV per macroblock from the block's local MVs happens at GME time in the encoder, while what you say happens in the decoder (warp points are sent in the header and the picture warped accordingly) and I agree with that and had the same impression to start with. True? If so, we are (were) on the same page. Else, I may need to ruminate on this a bit to make sure I get this all right. :-/

[SeeMoreDigital]

Would it be possible to create an application that could allow the user to say, input an MPEG-4 stream that contains 3-warp-point GMC and output an MPEG-4 stream, with re-written headers that can effectively be played in a stand-alone player?


Cheers

[Manao]

No

Cheers <-- that's because "No" alone is too short an answer :p

[SeeMoreDigital]

No

Cheers <-- that's because "No" alone is too short an answer :pLol..

[unskinnyboy]

Would it be possible to create an application that could allow the user to say, input an MPEG-4 stream that contains 3-warp-point GMC and output an MPEG-4 stream, with re-written headers that can effectively be played in a stand-alone player?
Even if you mess with the headers, the chipset still needs to decode those 3-warp point S-VOPs, doesn't it? The whole taboo with using GMC is because of the standalones not being able to do this. Most standalones either right away refuses to play anything with more than 1-warp point and some quit playing when it hits such a frame. Some may just skip over those frames, but then that would lead to jerky video or audio sync issues, ranging from minor to major, depending on how many frames were GMC'd in the video.

[SeeMoreDigital]

I was kinda hoping somebody could create a tool that could re-write the header and say make an S-VOP become an N-VOP (or something)..... Effectively fooling the stand-alone player!

Just a thought :eek:

[unskinnyboy]

Those frames are *inside* the stream, not in the header. Only way to remove/change them is re-encoding.

[SeeMoreDigital]

Those frames are *inside* the stream, not in the header. Only way to remove/change them is re-encoding.Indeed... but what would happen if the header reported them as being a different type of frame?

[GodofaGap]

Then you probably can forget about decoding the rest of the GOP correctly.