XVID_ME_xxxx config flags

[plugh]

I *thought* I had a basic understanding of these flags - ie

the XVID_ME_xxxx flags control motion search options
the XVID_ME_xxxx_RD flags control Rate Distortion Optimization ("VHQ" modes)

and that the RDO stuff was a refinement, that looked for the most bit-efficient coding option...

But - I just stumbled across some code in xvidcore, a routine called MakeGoodMotionFlags, that has me scratching my head. In particular the following set of 4 if's
if (Flags & XVID_ME_HALFPELREFINE16_RD)
Flags |= XVID_ME_QUARTERPELREFINE16_RD;

if (Flags & XVID_ME_HALFPELREFINE8_RD) {
Flags |= XVID_ME_QUARTERPELREFINE8_RD;
Flags &= ~XVID_ME_HALFPELREFINE8;
}

if (Flags & XVID_ME_QUARTERPELREFINE8_RD)
Flags &= ~XVID_ME_QUARTERPELREFINE8;

if (Flags & XVID_ME_QUARTERPELREFINE16_RD)
Flags &= ~XVID_ME_QUARTERPELREFINE16;

The last three seem to indicate that XVID_ME_xxxx and XVID_ME_xxxx_RD flags are mutually exclusive. That setting the 'xxxx_RD' variant means not doing the 'xxxx' motion search.

That contradicts my understanding, but hey, I can accept that perhaps I was mistaken.

But if that is the case, shouldn't the first if also be enforcing this 'mutually exclusive' convention? Shouldn't it be like the second if, i.e.
if (Flags & XVID_ME_HALFPELREFINE16_RD) {
Flags |= XVID_ME_QUARTERPELREFINE16_RD;
Flags &= ~XVID_ME_HALFPELREFINE16;
}

Comments? Are these flags//modes really mutually exclusive?

[sysKin]

The last three seem to indicate that XVID_ME_xxxx and XVID_ME_xxxx_RD flags are mutually exclusive. That setting the 'xxxx_RD' variant means not doing the 'xxxx' motion search.

You are correct. In short, I tested and concluded that doing refinement in both SAD and RDO is a complete waste of time.

You *could* remove those and confirm. So you're not mistaken, you just haven't seen the outcome :)

The only exception - which you noticed - is that halfpel16 refinement is still useful in SAD regardless of being repeated later in RDO. Therefore it's not disabled.

[plugh]

Thanks for the explanation.

The reason I was looking was the apparently contradictory recommendations "out there" regarding VHQ setting. On the one hand many say "higher VHQ gives higher quality", but some say "VHQ above 1 looses detail".

The code, and your reply, indicates that with VHQ={0,1} refinement is SAD based, VHQ={2,3,4} is RDO based - with the one exception. The code literally switches to a differant algorithm.

Food for thought...

[plugh]

Totally non-rigorous unscientific test..

Encoded the same 5000 frame clip at const quant 2, at each of the five vhq settings. As vhq increased, the files got smaller. So logically, for a target filesize constrained encode, increasing bit efficiency at higher vhq's should translate to higher quality.

Also encoded a complete movie at const quant 2, once at vhq=1, once at vhq=3. Watched both copies of the movie on my normal living room setup. I can't really say *why*, as nothing particular jumped out at me as being differant, but my purely subjective reaction was that I simply liked the vhq=1 encode better.

I might re-do this test sometime with someone else selecting which copy is being played...

[Manao]

Quality at a given quantizer depends on the encoding settings. So, first off,
As vhq increased, the files got smaller. So logically, for a target filesize constrained encode, increasing bit efficiency at higher vhq's should translate to higher quality.That is incorrect. Filesize could be lower but quality could be too. You always have to take both into account, and never assume that quantizer is the only factor governing the quality.

Also, you're noticing a difference of quality between VHQ 1 and VHQ 3. That's impressive, because the difference is tight. However, I don't think it was a blind test, so beware because with such a tight difference, knowing the encoding settings has more influence on your eyes that you would think.

Furthermore, you shouldn't compare encoding settings at constant quantizer, since both size and quality are modified at the same time. You'd better try to reach the same target size with both encoding settings ( hence using two passes ), and then compare the quality, at constant size this time.

Finally, some people on this forum are already advising not to use anything over VHQ 2. Though I tend to disagree - they blame VHQ 4 for removing noise - they tend to confirm what you noticed.

[plugh]

As I said, I don't know what was differant, it was just a subjective reaction. And yes, it would be better to do it 'blind', which I'll do at some point (as I also said).

The dichotomy in recommendations got me curious, the shift in the code used across the vhq 1/2 transition got me more curious. I actually expected to prefer the higher vhq encode, and was surprised at my reaction. But it could have been other factors, even change in ambient lighting due to differant time of day.

Purely an anecdotal, nonrigourous, unscientific observation.

[sysKin]

I did some tests months ago and VHQ3 does lower SSIM. There's a flag that's probably the culprit, but I haven't done anything about it...

[plugh]

I'm hesitant to post this, but what the hey...

I _think_ the only _interesting_ flag added going from vhq 2 to 3 is XVID_ME_CHECKPREDICTION_RD

findRD_inter
if (MotionFlags&XVID_ME_CHECKPREDICTION_RD) { /* let's check vector equal to prediction */
VECTOR * v = Data->qpel ? Data->currentQMV : Data->currentMV;
if (!MVequal(Data->predMV, *v))
CheckCandidateRD16(Data->predMV.x, Data->predMV.y, Data, 255);
}findRD_inter4v /* checking vector equal to predicion */
if (i != 0 && MotionFlags & XVID_ME_CHECKPREDICTION_RD) {
const VECTOR * v = Data->qpel ? Data8->currentQMV : Data8->currentMV;
if (!MVequal(*v, Data8->predMV))
CheckCandidateRD8(Data8->predMV.x, Data8->predMV.y, Data8, 255);
}

While fully admitting I understand very little of what's going on in there, but noting the first two arguments of the CheckCandidate calls ("x"=predMV.x "y"=predMV.y) the following line in the CheckCandidate routines seems 'odd' rd += BITS_MULT * (d_mv_bits(x, y, data->predMV, data->iFcode, data->qpel^data->qpel_precision) - 2);given what d_mv_bits does, if I assume qpel equals zerod_mv_bits(int x, int y, const VECTOR pred, const uint32_t iFcode, const int qpel)
{
unsigned int bits;

x <<= qpel;
y <<= qpel;

x -= pred.x;
bits = (x != 0 ? iFcode:0);
x = -abs(x);
x >>= (iFcode - 1);
bits += r_mvtab[x+63];

y -= pred.y;
bits += (y != 0 ? iFcode:0);
y = -abs(y);
y >>= (iFcode - 1);
bits += r_mvtab[y+63];

return bits;
}ie "d_mv_bits" _always_ equals 2*r_mvtab[63] ( == 2*1)
and thus checkcandidate _always_ ends up with rd += 0

Perhaps the checkcandidate call args should be v->x and v->y ?

But, as I don't understand what's going on, it's all probably entirely legit and has nothing to do with this topic. :confused:

[plugh]

Well, I ran some test cases comparing that flag on/off and with/without the postulated code change.

It appears the change turns that option into a big no-op. The output file I get with the change and the flag enabled is _exactly_ the same (binary compare) as the output file I get with the flag disabled. Without the change, the output files are differant with/without the flag enabled.

So either the change is bogus, or it's correct but ends up not actually doing anything with my test clip.

Which behaviour is correct? Beats me...

fwiw, the only other flags that get turned on going from vhq 2 to 3 are the halfpel and qpel 'refine8_rd' flags. Specifically: case VHQ_LIMITED_SEARCH :
frame.vop_flags |= XVID_VOP_MODEDECISION_RD;
frame.motion |= XVID_ME_HALFPELREFINE16_RD;
frame.motion |= XVID_ME_QUARTERPELREFINE16_RD;
break;

case VHQ_MEDIUM_SEARCH :
frame.vop_flags |= XVID_VOP_MODEDECISION_RD;
frame.motion |= XVID_ME_HALFPELREFINE16_RD;
frame.motion |= XVID_ME_HALFPELREFINE8_RD;
frame.motion |= XVID_ME_QUARTERPELREFINE16_RD;
frame.motion |= XVID_ME_QUARTERPELREFINE8_RD;
frame.motion |= XVID_ME_CHECKPREDICTION_RD;
break;Which, as pointed out above, turn off the corresponding "SAD based" flags.

[sysKin]

and thus checkcandidate _always_ ends up with rd += 0

Yes! :) This is exactly what "check prediction" is supposed to mean - that we're checking the position for which the vector costs zero (or in fact two, it's been taken into account before).

This flag improves PSNR but seems to lower SSIM. I haven't really tested that though.

[plugh]

Gotcha. My initial reaction, seeing as it was checking against itself, was that it looked fishy. I understand now.

Odd that it has that effect. Guess it goes to show that none of psnr, sad, ssim, etc are perfect measures (but you have to use something to evaluate the alternatives).

FYI - I read the Sullivan/Weigand paper "Rate-Distortion Optimization for Video Compression", so this is making more sense, but still have tough time tracking through that code.

Anyway, :thanks: for taking the time to respond.

[plugh]

Been running some psnr/ssim comparisons, and came upon a surprising (to me at least) result. Seems Trellis Quantization has a negative effect upon psnr and ssim. Is this "known"? Are the small differences in magnitude "significant"?

Ten encodes (9,500 1280x528 frames), VHQ for B-frames=on, 4MV=off, max bvop=1, Turbo off, I&P quant=2, B quant=3, left column is trellis enabled, right column trellis disabled.
vhq=1 kfh2db4v1h.avi, kfh2db4v1h-not.avi
h263 AVG: 48.72155, AVG: 48.75818
AVG: 0.99275, AVG: 0.99278
147,017,728 148,156,416

vhq=2 kfh2db4v2h.avi, kfh2db4v2h-not.avi
h263 AVG: 48.74540, AVG: 48.78216
AVG: 0.99275, AVG: 0.99278
143,566,848 144,699,392

vhq=3 kfh2db4v3h.avi, kfh2db4v3h-not.avi
h263 AVG: 48.74181, AVG: 48.77846
AVG: 0.99274, AVG: 0.99277
142,886,912 144,019,456

vhq=2 kfh2db4v2m.avi, kfh2db4v2m-not.avi
mpeg AVG: 48.56713, AVG: 48.60319
AVG: 0.99211, AVG: 0.99214
165,267,456 166,391,808

vhq=3 kfh2db4v3m.avi, kfh2db4v3m-not.avi
mpeg AVG: 48.56642, AVG: 48.60270
AVG: 0.99211, AVG: 0.99214
164,196,352 165,326,848
(Also note the vhq 2/3 results with respect to the vhq=3 checkprediction_rd flag. EDIT: added another pair and the file sizes)

However there is more to the story than revealed by the above; a randomly selected segment of frame values from the last encode pair above.
49.34422, 49.34422
48.51561, 48.61453
49.47751, 49.47751
48.46959, 48.55963
49.68299, 49.68299
48.02684, 48.12814
49.63039, 49.63039
48.36683, 48.45472
49.54361, 49.54361
48.32008, 48.42432
49.72342, 49.72342
48.16019, 48.29951
49.63051, 49.63051
48.32448, 48.42447
49.59573, 49.59573
48.54253, 48.61386
49.85568, 49.85568
48.26086, 48.33316Notice the alternating pattern? All the encode pairs show this pattern. Seems my B-frames are degraded by Trellis quant. Is ~0.1db psnr difference significant?

[Manao]

Trellis is a rate/distorsion optimization. You must take both PSNR/SSIM and bitrate into account when judging its effect, so a constant quantizer encoding won't do. Do a two passes encoding, or compute several (PSNR,bitrate) couples with and without trellis, at different quantizer ( lets say 2, 3 and 4 ), and checks if the couple with trellis are higher than those without.

[Manao]

Have a look at this post (http://forum.doom9.org/showthread.php?p=546607#post546607), where such a RD curve is drawn.

[plugh]

Have a look at this post (http://forum.doom9.org/showthread.php?p=546607#post546607), where such a RD curve is drawn.
Thanks for the pointer; that's a great thread, exactly to the point. And the quant aspect isn't an axis I explored above - *very* interesting.

One thing I note (perhaps a change made after that 2004 thread?) is this line of code in mbtransquant.c routine MBQuantInter (the only place the flag is tested)if(sum && (pMB->quant > 2) && (frame->vop_flags & XVID_VOP_TRELLISQUANT))
ie trellis only applied to inter blocks with quant > 2
EDIT 04/21/08: Stumbled across this thread/post, which confirms the above
http://forum.doom9.org/showthread.php?p=577933#post577933 END EDIT
Trellis is a rate/distorsion optimizationIt was exactly this realization that led me to experiment and see if there was an interaction between it and vhq (whilst doing the checkprediction_rd tests above - I got side-tracked).

That only leaves one question unanswered, for a 'newbie' to this psnr/ssim stuff - are the magnitude of those differences considered 'significant'? I understand the logarithmic nature of 'db', still 0.1db is quite small; ditto for ssim diffs in the 4th/5th decimal place. Is it considered visibly differant to the untrained eye?

[Manao]

Trellis is roughly a 0.3 dB gain at same size, or a 6% bitrate saving at same PSNR ( the relationship 0.05dB <-> 1% bitrate is mostly valid for XviD & x264 & most codecs out there).

0.3dB means nothing. You will probably have trouble seeing the difference. 6% bitrate is something though. That's why you should always try to compare two encoding settings giving the same quality ( be it PSNR or SSIM or whatever ), in order to know what is the best, and by how much.

[plugh]

Thanks again. I added file sizes (includes audio) to the table above for completeness sake.

Generally speaking I don't care about plus or minus a few percent in file size, as long as I can get a 2hr-ish HD movie under 4GB (seldom a problem so far) and it is playable on my cpu constrained target platform (my primary constraint).

My encodes are usually I/P/B 2-2-3, 1 B-frame, h263; sometimes 1-2-3, sometimes 2 B-frames, sometimes mpeg - varies depending upon the movie; no qpel, gmc, 4mv - all incur extra target cpu load (as does mpeg quant so I use it sparingly). Within that parameter set, I'm trying to (in order) 1- maintain playability, 2-maximize quality, 3-minimize size. Incurring somewhat greater size for higher quality is "ok", subject to the overriding playability constraint(s).

As a single datapoint example - the 100 minute movie HD encodes from which those 9500 frame measurements were taken ranges from 2.36GB (vhq=3, h263) to 2.82GB (vhq=1, mpeg). Adding a few percent in size for increased fidelity won't matter (it's well under 4GB), as long as decoding it doesn't push the target cpu past its limits. But if xvid can make it smaller at (basically) the same quality by working harder during the encoding, cool!

Anyway, I've learned a lot from this thread, so :thanks: