What is warppoint?

[plazz2000]

Hi,

I'm looking for a definition of what warp-point means regarding GMC?

I know that XviD and NeroDigital uses 3, and that the latest Mediatek chipset can only decode 2 (so far), but I don't know what it actually means, and I'm curious to find out.

Can someone explain, or point me to an explanation?

Thanks.

[sysKin]

A warppoint is a vector that defines a displacement of one *edge* of the video.
Take a piece of paper and move it by its edges. You'll see what I mean.

First warppoint defines displacement of top-left edge. If it's the only warppoint, the rest of the picture just has the same vector and whole picture moves. Think panning.

Second warppoint defines displacement of top-right edge (not *precisely* true but close). Together with first warppoint, this is enough to define panning *and* zoom. Note that it could be used to define panning and rotation instead, but isn't.*

Third warppoint means displacement of down-left edge and three warpoints are enough to define panning, zoom and rotation.

Fourth warppoint would create perspective-like movement.

:)Radek

* <-- I am not 100% sure, it might be the other way around

[crusty]

Sounds like an excellent addition to the faq to me, sysKin.

So DivX just has one warppoint gmc? Looks like Xvid GMC could then be much more usefull than DivX GMC.
(not *precisely* true but close)
Interesting...could you elaborate a bit more? I'm really curious.

Is the 'weight' of one warppoint to a specific macroblock relative to the macroblocks position?

(In other words, more simply put: Is a macroblock more influenced by a warppoint closer to it than further away?

Exactly how are warppoints stored in a frame? Are they just single vectors with a size or is it more complex? (probably..) :D

Now a less theoretical question:
Is GMC currently safe to use( i.e. does it work), and exactly how usefull is it?

Can't wait to try it out on the famous psychedelic interdimensional spacetrip in 2001: A Space Odyssey. If there's one scene where it should make a difference, that's the one.

Cheers,
Crusty

[crusty]

I've made another possible addition to the faq..I'd like any comments on it.


Q: What's GMC and what is it good for?

GMC stands for General Motion Compensation and that pretty much tells the story of what it does.
If used, it will look at the whole frame and see if there is an amount of motion that all the parts of the frame have in common.
It will then take this motion, put it in a single value, and substract it from all the parts of the frame.
The parts of the frame are the macroblocks, and the amount of motion is called a 'vector' which has both a direction (left,right,up,down etc.) and a value (1 pixel p/frame, 75 pixels p/frame, etc).
All the macroblocks have their own vectors, but with GMC the one big value that they all have in common (that's why it's called 'General') will be substracted and put into a single value. That way all the individual vectors will have much smaller values, possibly getting completely nullified by the substraction process. The possible benefit is that you can replace many or all the motion vectors of the macroblocks in a frame by a single value, thereby making it much smaller.<BR>
Note however that this is for one-warppoint GMC. With Multiple warppoints the proces is much more complex, but the principle is the same.

Any thoughts?

[DevilsChild]

XviD's GMC works fine for me.

[TorgoGuy]

So what is the current recommendation regarding GMC? Is the "Newbie Settings" FAQ out of date? It says:

"Don't use VHQ together with GMC (GMC is not recommended anyway)"

implying that GMC shouldn't be used.

[sysKin]

Originally posted by crusty
So DivX just has one warppoint gmc? Looks like Xvid GMC could then be much more usefull than DivX GMC.Heh, just look at PSNR comparison. Or speed, btw :)
about "not precisely true"
Interesting...could you elaborate a bit more? I'm really curious.I am not absolutely sure, I've never programmed GMC. I think the size of the image is rounded up to nearest 2^n value, so if your image is 688 pixels wide, the second vector does not define displacement of 688th pixel but rather 1024th pixel. This pixel doesn't exist but it makes calculations for all other pixels easier.
Is the 'weight' of one warppoint to a specific macroblock relative to the macroblocks position?GMC does not have much to do with macroblocks. It zooms/rotates/pans whole image, like if you loaded it to a PSP or something. Exact calculations are defined in the standard.
Of course in the end, frame is divided into macroblocks. At this stage, every macroblock can be coded as usual (with a vector, or as intra, o with 4 vectors) *or* it can be a direct copy from the GMC image, calculated before. If the image has been zoomed and rotated, this gives a completely new possibility for the macroblock, which could not be achived with normal motion vectors alone.
Exactly how are warppoints stored in a frame? Are they just single vectors with a size or is it more complex? (probably..) :DIn the frame's header. There are some VLCs but I doubt it's horribly compressed - it's only once per frame.
Now a less theoretical question:
Is GMC currently safe to use( i.e. does it work), and exactly how usefull is it?It is very slow, but it works and it improves the picture (not much) if you use it with VHQ (yes, FAQ is outdated). Without VHQ the decisions about using GMC or not (in given macroblock) are too "fuzzy" and GMC does not help.
There is a useful discussion on xvid.org's forum (yes, the first useful discussion there in a year) about GME (the process of finding best GMC warppoints), I hope GMC will be more useful in the future. Much can still be done.
Can't wait to try it out on the famous psychedelic interdimensional spacetrip in 2001: A Space Odyssey. If there's one scene where it should make a difference, that's the one.If rotation will happen too fast, it will not be found by current GME. It's more for slow and easy camera movement.

Radek

[crusty]

OK, here goes:

Is my explanation of one-warppoint GMC correct?
I can see that adding warppoints makes it much more complicated.

It looks like if it is the other way around, the effect could be more easily explained, so here goes:

With a single warppoint the whole frame gets a motion vector, and all macroblocks get that motion substracted from their current motion. Remaining motion would be substantially less and therefore take up less space.

With a second warppoint the process becomes much more complicated.
You basically say that the second warppoint is created at a virtual location somewhere outside the real frame.
I can see no other explanation that at least the influence of the second warppoint would have to be relative to the distance of each macroblock.

Let's take a zoom for instance.
A zoom centered on the frame would mean that the first warppoint indicates a movement to the left (in general). That would have to count for all parts of the frame equally. But if the second warppoint defines zoom, then that would mean that any part closer to the second warppoint would have to be influenced more by the second warppoint than parts further away.

It seems logical to me, that with a two-warppoint gmc both motion vectors would be added up and then divided by two, with a second formula somehow having to cope with the distance one part of the frame has to both warppoints.

I'm totally crap at mathematics, but here's what I'm thinking about:

Influence of GMC on part of picture=

(warppoint 1 motion vector * distance to part of picture) + (warppoint 2 motion vector * distance to part of picture)
----------------------------------------------------------------------------------------------------------------
2 (basically amount of warppoints used)

Am I talking completely bollocks here, or am I somewhere close?
Off course a warppoint is multi-dimensional since it has both a two-dimensional value, the direction, and a size.

BTW, it sounds only logical that the first warppoint is also somewhere outside the real frame, because it would make no sense to above equation. Equally , a third and fourth warppoint would have to be at the same but opposite locations of the first two warppoints.

To see what I mean, here's another nice asci picture:


warppoint 1 warppoint 2
X X
\ /
\ /
\A)######(D)#####(B/
#\################/#
##\##############/##
###\############/###
####\##########/####
#####\########/#####
######\######/######
#######\#(C)/#######
########\##/######## = The picture (you guessed it, right?)
#########\/#########
#########/\#########
########/##\########
#######/####\#######
######/######\######
#####/########\#####
####/##########\####
###/############\###
##/##############\##
#/################\#
/##################\
/ \
/ \
X X
warppoint 3 warppoint 4

(damn making asci pictures is boring...)
:)
So this basically means that every warppoint HAS to be outside the actual picture to make any sense. And they have to be at symmetrical but opposite locations, in an extended line from the center of the picture to each corner.

So even if we're not talking about macroblocks here, there has to be a correlation between the distance of a part of the picture to each warppoint.

Take locations A and B in the picture above. If two warppoints define a zoom, then in the original picture A would be moving left (and a bit up) and B would be moving right (and also a bit up).
You can also see that C would be moving only a little bit to the right and only a really teensy weensy bit up, whereas point D would move up a bit more.
I can see no other way for this to be portrayed mathematically if distance to each warppoints is taking into consideration.
This goes for any number of warppoints.

I can also guess that there is no 5-warppoint GMC because there is no other dimension to place it in, i.e. no other corner of the picture. Am I correct?
GMC does not have much to do with macroblocks. It zooms/rotates/pans whole image, like if you loaded it to a PSP or something.
So it's more like there's this virtual vector field everywhere in the frame, and any macroblock present at a certain location just happens to get the GMC value that's defined at that location?
At this stage, every macroblock can be coded as usual (with a vector, or as intra, o with 4 vectors)
The first two I get:
'With a vector'is for any non-keyframe, like B- and P-frames
'or as intra' is for keyframes
'o with 4 vectors' is this meant for GMC? Is this one vector for the macroblock itself and the otehrs for the 3-point GMC, or is it something else?
There are some VLCs
What's a VLC?
It is very slow, but it works and it improves the picture (not much) if you use it with VHQ (yes, FAQ is outdated).
Got it...I guess you mean for beta and Release Candidate up only right?
Without VHQ the decisions about using GMC or not (in given macroblock) are too "fuzzy" and GMC does not help.
I guess that's because motion estimation without VHQ (which is basically brute-forcing as precise a motion as possible) would be too inaccurate..
That means GMC is a pretty subtle process.If rotation will happen too fast, it will not be found by current GME. It's more for slow and easy camera movement.
So basically you're saying that GMC status now is:
-It works
-It's safe
-It might help, but not very likely at this moment...
-It's not really optimised
-It's slow (knowing unoptimised options, probably very slow)
-If usefull at all, it is only usefull in combination with VHQ
-If motion is too fast, it doesn't see it and goes into a corner twiddling it's thumbs

Well, that's a whole lot more than in older versions. Good job I say.

btw, the scene from 2001 mostly has perspective-like, and zooming movement. Rotating parts are only slowly rotating, it might be slow enough.

Cheers,
Crusty

[sysKin]

The picture is quite correct, but let me have some fun with ascii-art too and let me exaplain the position of warppoints:

########################
########################
########################
########################
########################
########################


The dark-blue is your picture, the light blue is virtual (bigger) picture as seen by GMC, and the real edges (black) are warppoints.
This makes symmetry a little different, but it's easier to do calculations now because the virtual picture has fixed dimensions (1024x512 for example, nearest 2^n)

Originally posted by crusty
I can also guess that there is no 5-warppoint GMC because there is no other dimension to place it in, i.e. no other corner of the picture. Am I correct?Smart people would surely think of something, but yes, there can be no more than 4 in GMC.
So it's more like there's this virtual vector field everywhere in the frame, and any macroblock present at a certain location just happens to get the GMC value that's defined at that location?No - if you have to think about it this way, there is one vector for every *pixel* in the frame, and this pixel moves (with bilinear interpolation and 16-th pixel precision) to the new location.

I would rather see it as normal picture manipulation, without any connection to macroblocks.
The first two I get:
'With a vector'is for any non-keyframe, like B- and P-frames
'or as intra' is for keyframes
'o with 4 vectors' is this meant for GMC? Is this one vector for the macroblock itself and the otehrs for the 3-point GMC, or is it something else?I was refering to S-VOP which is a P-VOP with GMC (S from 'sprite').
This frame type is just like P-frame, which means it can have intra MBs (like keyframes), one-vector inter MBs (the most normal type) and four-vector inter MBs (more precise, one vector for each 8x8 block).

This was all normal P-VOP - S-VOP can *additionaly* have non-vector inter mode, which means that no vector is coded, and motion compensation comes from GMC-warped picture. It's not a vector that comes from that picture, it's the pixels that come from there.
What's a VLC?Variable Length Code, a code, defined in the standard, which are written to the bitstream. If the codes have been created using Huffman's rules, they mean compression.
Got it...I guess you mean for beta and Release Candidate up only right?Or some older dev-api-4 build (but why?)[/quote]
I guess that's because motion estimation without VHQ (which is basically brute-forcing as precise a motion as possible) would be too inaccurate..No that's not right. It's the decision whether to use GMC-ed mode (the last case, as I explained above) or some other motion mode. GMCed mode almost always means less bits used for motion data (there is no vector) but it can be compensated by worse picture (in terms of data used to fix it, and in terms of quality). This is the kind of decision that only VHQ can make right (by the way, almost all gain from VHQ1 comes from one-vector/four-vector decision which is just as difficult to do).
So basically you're saying that GMC status now is:
-It works
-It's safe
-It might help, but not very likely at this moment...
-It's not really optimised
-It's slow (knowing unoptimised options, probably very slow)
-If usefull at all, it is only usefull in combination with VHQ
Yeah :)-If motion is too fast, it doesn't see it and goes into a corner twiddling it's thumbsThe good thing about GMC is that it can not be used - stream can still have P-VOPs intead of S-VOPs and in such case there is no penalty, no overhead of any kind. So if GMC is not useful for given frame/scene, it's all still great. :)

Radek

[crusty]

The dark-blue is your picture, the light blue is virtual (bigger) picture as seen by GMC, and the real edges (black) are warppoints.
I guess to understand GMC we really have to see the bigger picture ... LOL :D :D
Anyway:
This makes symmetry a little different, but it's easier to do calculations now because the virtual picture has fixed dimensions (1024x512 for example, nearest 2^n)
Does that mean this:
--------------------------------
Say if the resolution is XXX,YYY (heigth, width) then:
-First warppoint is at 0,0
-Second at 0, (YYY * 2)
-Third at (XXX * 2), (YYY * 2)
-Fourth at (XXX * 2), 0
And the lower right corner is dead center of all this
--------------------------------
Right???
I would rather see it as normal picture manipulation, without any connection to macroblocks.
I find that difficult to do, mostly because macroblocks are the essential subparts of each frame. I cannot really see it that theoretical, I have to see some kind of visualisation with parts moving about and being influenced by GMC-vectors to understand.
four-vector inter MBs (more precise, one vector for each 8x8 block).
Completely forgot that that was possible...it was lingering in some remote part of my brain.
(Wasn't that part of VHQ3 and 4, ie 3 using submacroblock precision and 4 redoing that decision multiple times? )
This was all normal P-VOP - S-VOP can *additionaly* have non-vector inter mode, which means that no vector is coded, and motion compensation comes from GMC-warped picture.
OK, so basically an Inter MB without any motion information, just texture information (if altered) or otherwise...just a placeholder of some sort (saying 'Hi codec, nothing's changed this frame')?
It's not a vector that comes from that picture, it's the pixels that come from there.
You mean that in such a type only the texture information (if any) come from there, while the motion comes solely from the GMC-value?
No that's not right. It's the decision whether to use GMC-ed mode (the last case, as I explained above) or some other motion mode.
So what you're saying is that VHQ makes all the decisions about these type of things anyway, and turning on GMC tells the VHQ-process that it can now also choose to create 'non-vector inter mode' type macroblocks if it finds it worth it, thereby making S-VOP's (GMC-frames) in the progress?
The good thing about GMC is that it can not be used - stream can still have P-VOPs intead of S-VOPs and in such case there is no penalty, no overhead of any kind. So if GMC is not useful for given frame/scene, it's all still great.
So basically S-VOPs are created ONLY when they're worth it? That's very good news. So I guess I can add to the list:
-never hurts

[RadicalEd]

Originally posted by crusty

Does that mean this:
--------------------------------
Say if the resolution is XXX,YYY (heigth, width) then:
-First warppoint is at 0,0
-Second at 0, (YYY * 2)
-Third at (XXX * 2), (YYY * 2)
-Fourth at (XXX * 2), 0
And the lower right corner is dead center of all this
--------------------------------
Right???


Wrong, it means the second would be at 2^n closest to YYY. If YYY was 352 it would be 2^9 = 512. If YYY was 640 it would be 2^10 = 1024. If YYY was 1920 it would be 2^11 = 2048. Syskin has been talking powers of 2 closest to the image size.

For instance, a 640x480 video would have warp points of (w,h or x,y):
1 = 0,0
2 = 1024,0
3 = 0,512
4 = 1024,512

[crusty]

OK, got it. Told you I'm crap at mathematics. :D

[sh0dan]

Very interesting reading.

So in general - GMC should (mostly) only be used in cases where it can provide a better reference picture than the motion vectors are capable of. In my mind rotation and zooms come to mind, since motion vectors cannot scale or rotate material. I guess for straight pans/tilts there isn't much to be gained, as they can be represented as motion vectors.

I can also imagine the problems detecting them, as motion vectors only gives hints about zoom/rotates.