Q-pel, GMC, and full quality first-passes

[Katie Boundary]

In an attempt to see if Q-pel, GMC, and VHQ wide-area searches were worth using, I ran a bunch of XviD first-passes with different settings and compared the resulting file sizes. XviD first-passes have a constant quantizer (2), so anything that improved compression should have resulted in smaller file sizes. To my surprise, all of the files had the exact same size down to the byte.

This reminded me of the "full quality first pass" option, and the fact that it resulted in larger file sizes despite using the exact same quantizer. And then I remembered that it also resulted in slower encoding times, and something something might not be fully mpeg-4-compliant.

So my question is: Does an XviD first-pass disable all those doodads, regardless of what your actual settings are? And does "full quality first pass" mean "screw that, use the settings that I tell you to use"?

[Sharc]

In an attempt to see if Q-pel, GMC, and VHQ wide-area searches were worth using, I ran a bunch of XviD first-passes with different settings and compared the resulting file sizes. XviD first-passes have a constant quantizer (2), so anything that improved compression should have resulted in smaller file sizes. To my surprise, all of the files had the exact same size down to the byte.

This reminded me of the "full quality first pass" option, and the fact that it resulted in larger file sizes despite using the exact same quantizer. And then I remembered that it also resulted in slower encoding times, and something something might not be fully mpeg-4-compliant.

So my question is: Does an XviD first-pass disable all those doodads, regardless of what your actual settings are? And does "full quality first pass" mean "screw that, use the settings that I tell you to use"?
Huh, it feels like ages since I have been using XviD, but as far as I remember:
In a standard 2-pass encode, the first pass disables by default many of the encoder settings which become effective and useful in the 2nd pass only. In particular this applies for VHQ. Also to mention that only the small stats file is produced in pass1 rather than the fully encoded video.
When you select Full Quality First Pass however, all encoder parameters settings are enabled in pass 1 already, making pass 1 much slower with little benefit for the final 2-pass quality. Also, the fully encoded video is kept from pass 1 rather than the stats file only.
As for GMC I think to remember that it has little to nil effect on compressability but increases the encoding time substantially.

[hello_hello]

GMC is still not commonly supported by hardware players. I'd definitely avoid it.

[Katie Boundary]

Also to mention that only the small stats file is produced in pass1 rather than the fully encoded video.
When you select Full Quality First Pass however, all encoder parameters settings are enabled in pass 1 already, making pass 1 much slower with little benefit for the final 2-pass quality. Also, the fully encoded video is kept from pass 1 rather than the stats file only.

Actually, "discard first pass" and "full quality first pass" are separate settings.

As for GMC I think to remember that it has little to nil effect on compressability but increases the encoding time substantially.

I ran a second batch of tests with "full quality first pass" enabled and found that GMC did improve compressibility by a few megabytes. Q-pel, however, wasted more bits than it saved. I'm now running a third set of tests: one-pass, target bitrate eleventy gazillion mbits/sec, railroaded quantizer of 4. So far, it looks like the benefits of GMC are lost at these lower file sizes.

[Gser]

GMC is still not commonly supported by hardware players. I'd definitely avoid it.

For maximum compatibility I would disable Qpel, GMC, limit B-VOP's to 1 consecutive and use a special xvid build that forces no N-VOP's (ESS based players don't like them). And some players also had difficulties with Packed bitstream.

But then again, shooting myself in the face sounds better than returning to the dark ages of MPEG-4 Part 2 Advanced Simple Profile.

I believe it was settled back then that GMC and QPel were fancy techniques that in the end did not do much for compression in the end. I remember playing around with some custom matrixes that helped more than they did but of course custom matrixes WERE NOT supported by all players, goddamn you MPEG-4 Part 2 Advanced Simple Profile.

[Katie Boundary]

For maximum compatibility I would disable Qpel, GMC, limit B-VOP's to 1 consecutive

I always disable b-frames in my AVI files, partly because getting AVI to support b-frames requires an ugly hack, and partly due to just hating bidirectional encoding as a matter of principle.

EDIT: the results are in. GMC improved compression by 2/3 of 1% when using Wide Area Search, and by a whole 1% when using Mode Decision. Also, with GMC enabled, Wide Area Search provided only 1/3 of 1% better compression than Mode Decision; without it, this jumped to 2/3 of 1%. Overall, Mode Decision without GMC was 98.9% as efficient as Wide Area Search + GMC. Very useful data.

[hello_hello]

I always disable b-frames in my AVI files, partly because getting AVI to support b-frames requires an ugly hack, and partly due to just hating bidirectional encoding as a matter of principle.

It doesn't "require" a hack. I think that's strictly a Video For Windows limitation, where it's always one frame in and one frame out. The hack is "packed bitstream", which you should be able to disable.
Personally I've never had a problem with hardware player compatibility, with or without a packed bitstream.

Bidirectional encoding improves compression efficiency a great deal, although admittedly XVid's B-frames aren't the highest quality. You'd be crazy to disable them for x264 encoding. Almost as crazy as still using Xvid.

[Sharc]

...Q-pel, however, wasted more bits than it saved...
The idea behind q-pel is not to save bits but to preserve details better. Hence q-pel tends to increase the bitrate.

[Katie Boundary]

The idea behind q-pel is not to save bits but to preserve details better. Hence q-pel tends to increase the bitrate.

But the quantizer was being railroaded. Q-pel was using more bits to preserve the same level of detail.

[SeeMoreDigital]

Why even bother learning about or using MPEG-4 SP/ASP... It's too old to be useful!

[Groucho2004]

Why even bother learning about or using MPEG-4 SP/ASP... It's too old to be useful!


You'll find the answer in her signature.
If I ask "How do I do X?" or "what happens if I do X?", and X is a very bad thing that no one would ever normally do, assume that I already know this, and that I have Katie reasons for asking anyway.

[manolito]

The simple answer is that if you need to distribute your encodes to lots of different folks with very different playback chains then you need to pick the lowest common denominator. And the formats almost everyone can play on the TV are still DVD (Mpeg2) and DivX/XviD. Many folks own standalone DVD players which play DivX HomeTheater profile (which can be easily mimicked by XviD). And when done right MPEG4-ASP can look very nice at one third of the MPEG2 file size.

And if you feel that MPEG4-ASP is too old to be useful then what do you think about DVD? Even older, but with a huge user base.

Cheers
manolito

[Sharc]

But the quantizer was being railroaded. Q-pel was using more bits to preserve the same level of detail.
Quantizer and q-pel are two different biests which may not be played one against the other for quality.
For 1-pass encodes I would leave the quantizer at 2 and allow 1 or 2 B frames if file size reduction matters. You could also play with the matrix and VHQ (1).
As has been mentioned by others to ensure player compatibility you may however align the XviD settings with the DivX standard (I remember the "DivX certified" players).

[hello_hello]

But the quantizer was being railroaded. Q-pel was using more bits to preserve the same level of detail.

How did you determine "the same level of data"?

https://en.wikipedia.org/wiki/Quarter-pixel_motion
Quarter-pixel motion (also known as Q-pel motion or Qpel motion) refers to using a quarter of the distance between pixels (or luma sample positions) as the motion vector precision for motion estimation and motion compensation in video compression schemes. It is used in many modern video coding formats such as MPEG-4 ASP, H.264/AVC, and HEVC. Though higher precision motion vectors take more bits to encode, they can sometimes result in more efficient compression overall, by increasing the quality of the prediction signal.