It has been an
extremely tedious process, but I have tracked back to a specific chunk of code that gives differant results when compiled with msvc vs gcc.
(When I say tedious I mean it - tracking backwards through the code, identifying where a particular rd mode evaluation for a particular macroblock for a particular frame goes weird)
The chuck of code is
Code:
static __inline uint32_t
d_mv_bits(int x, int y, const VECTOR pred, const uint32_t iFcode, const int qpel)
{
unsigned int bits;
x <<= qpel;
y <<= qpel;
x -= pred.x;
bits = (x != 0 ? iFcode:0);
x = -abs(x);
x >>= (iFcode - 1);
bits += r_mvtab[x+63];
y -= pred.y;
bits += (y != 0 ? iFcode:0);
y = -abs(y);
y >>= (iFcode - 1);
bits += r_mvtab[y+63];
return bits;
}
in motion_inlines.h; r_mvtab is defined there as well.
The arguments being passed in this particular case are
x=-64 y=63 pred={x=63,y=15} iFcode=2 qpel=0
msvc produces the value 4128837
gcc produces the value 14
I manually calc it, and I get 26
The call stack is:
ModeDecision_BVOP_RD ->
SearchInterpolate_RD ->
CheckCandidateRDInt ->
the first instance in the following statement
Code:
rd += BITS_MULT * (d_mv_bits(xf, yf, data->predMV, data->iFcode, data->qpel^data->qpel_precision)
+ d_mv_bits(xb, yb, data->bpredMV, data->iFcode, data->qpel^data->qpel_precision));
WTF!