Confused about extracted motion vectors from ffmpeg

[serhannn]

Hi, I used the example file extract_mvs.c in ffmpeg documentation and extracted the motion vectors from a H.264 encoded video. I obtained a text file with the following format:

framenum, source, blockw, blockh, srcx, srcy, dstx, dsty, flags


I'm a little bit confused about what src and dst actually means. Until now, I thought dst ist the reference block from which the motion is estimated, and src is the current block in the current frame. So, the x and y compontents of a motion vector is calculated as:

MV_x = dstx-srcx
MV_y = dsty-srcy

However, in the text file, I see lines like this:

2,-1,16, 8, 167, 51, 168, 52, 1, 1,0x0
...
...
113,-1, 8, 8, 157, 156, 156, 156,-1, 0,0x0

So, for example in frame 2, the 16x8 source block seems to be located at (167,51), and in frame 113 8x8 source blocks is located at (157,156). This doesn't make sense to me. Shouldn't the center of the current block always be located at multiples of 8 or 16 along both x and y directions? I thought the "destination" blocks (in the reference frame) would be shifted by the magnitude of the respective motion vectors; but the situation seems to be the other way around here. Do I interpret the meaning of src and dst wrong? Or is this kind of motion prediction actually allowed?

I'm a beginner to H.264 standard so I'm sorry if the question is too naive... Thanks!

[foxyshadis]

You have it backward: src refers to the coordinates in the reference frame that the vector points to, dst refers to the top-left coordinates of the actual coded block. All of these are in the upscaled plane (hpel for MPEG1 & MPEG2, qpel for avc & hevc, either way for asp) so divide by 4 to get the actual x,y location in the output video.

You can get a lot more information by editing mpegvideo.c, though it's pretty hard to know what's useful and what's not.

[serhannn]

Thanks for the correction, in the documentation off ffmpeg there is no information regarding that, so it was a source of confusion for me... Can you point me to a source file where I can verify this?
Except for that, there are still two unclear points:

1. Do src and dst really refer to the top-left coordinates of the respective macroblocks?
2. The function av_frame_get_side_data does not actually give coordinates in upscaled plane as you mentioned (with H.264/AVC videos). E.g. when I run extract_mvs.c in the documentation, I get an output like:


framenum,source,blockw,blockh,srcx,srcy,dstx,dsty
2,-1,16,16, 8, 8, 8, 8
2,-1,16,16, 24, 8, 24, 8
2,-1,16,16, 40, 8, 40, 8
2,-1,16,16, 56, 8, 56, 8
2,-1,16,16, 72, 8, 72, 8
...
...


To me it seems that srcx, srcy, dstx, and dsty give the center point for the respective blocks rather than the top-left point.

EDIT: I also went throught the code in vf_codecview.c (https://ffmpeg.org/doxygen/2.5/vf__codecview_8c_source.html#l00193) and the function filter_frame gave me the impression that dst_{x,y] are the coordinates of the reference block since the draw_arrow function draws an arrow from src_{x,y} to dst_{x,y}. I'm still kind of confused.

[foxyshadis]

It looks like you're completely correct about the center being the point, and it not being upscaled. d'oh, it's been a while since I used it, obviously, sorry about that. Choosing to display it the direction they do is an arbitrary choice; it would make sense either way, although I think it's more intuitive to point to where it went than where it came from.

[serhannn]

Yes, intuitively you are right. I would expect that dst would be the current block in the current frame and src would be the reference block in the reference frame since the the reference block is actually the "source" but it seems to be the other way around.

Unfortunately, I still don't get why src_x and src_y are shifted by the magnitude of the x and y components of the motion vector (MV_x and MV_y). This would be correct if the above-mentioned logic was used but in this case, I'm not sure.

EDIT: Or maybe ffmpeg points the motion vectors not from the current block to the reference block, but the other way around. Is that also possible?