Variance masking V Luminance masking in Xvid!

[datauser]

Anyone out there who can kindly tell me which one has the greater benefit for an Xvid encoding and if they should be turned on or off in the 1st pass and left on for the 2nd pass only? There isnt much info on variance apart from some dated info on luminance. With not using Xvid for a few years due to H264(now getting to grips with H265!)and having to encode a file up for a friend's old standalone my old noodle is working overtime again. Cheers guys for any help!:)

[kalehrl]

Variance is newer and, if I remember correctly, it was backported to Xvid from h264.
Variance should provide a little more compression.
I'm not sure for what pass it should be used.

[datauser]

I think it was an old thread here on Doom a few years back that with older versions of xvid the 1pass for luminance was counter productive, so enabled only for 2nd one, but regarding the newer variance no idea at all. Thanks for your reply.

[hello_hello]

If it means anything, I checked and I'm pretty sure the last version of AutoGK enables VAQ for both 1st and 2nd pass.

Initially Xvid's adaptive quantisation = luminance masking.
Then when VAQ was first introduced it replaced luminance masking, so adaptive quantisation = variance masking.
Given VAQ is supposed to be much better than luminance masking I'm not sure why later versions of Xvid let you choose either type. Anyone know?

[datauser]

I've been testing out the new Xvid build 1.3.4 and getting really excellent quality at higher bitrates and enabling luminance masking in both 1st pass and 2nd one. Though I'm well used to h264 now and prefer it for lower bitrates, from my own tests I notice that at higher bitrates(1800+) Xvid retains much more facial detail(preserves film grain?) while h264 seems to still produce a glossier/smoothed over effect. All subjective of course and others may disagree!

[Katie Boundary]

This has been bugging me for years. Does anyone have any new information on the subject?

[Motenai Yoda]

lumimasking => increased quality in medium macroblocks, at cost of more compression in dark and bright macroblocks, useful in the 1cd xvid rip era, no good for high quality, nor for today quality standard
variance masking or vaq => port from x264 of Dark Shikari's VAQ, useful for RL content, bad for animation as it can't be tuned
both => worst quality than each one/no one
in my experience both are to avoid when using a hvs/custom matrix