VAQ: Making Xvid's Adaptive Quantization Not Suck

[Brother John]

This is fantastic news! Thx for that patch, Dark Shikari. Now, there’s no chance you might get super-bored and port CRF to Xvid?

I did some tests with wizboy11’s binary and short clips (10000 frames) from five PAL DVDs.

Originally Posted by Xvid_Encraw

-cq 2 -qmatrix "Heini MR.xcm" -bquant_ratio 100 -bquant_offset 100 -qpel -bvhq -zones 0,w,1,O
-lumimasking added for VAQ encodings.

File sizes (in Matroska container)

           |   size (KByte)   | Diff. | Diff. by frame type (%)
sample     | VAQ off | VAQ on |   %   |   I   |   P   |   B
-----------+---------+--------+-------+-------+-------+--------
1 dialogue | 100958  |  87338 | -13.4 | -11.3 | -23.0 | +7.5
-----------+---------+--------+-------+-------+-------+--------
2 night    | 186572  | 169400 |  -9.2 |  -8.6 | -14.2 | +2.7
-----------+---------+--------+-------+-------+-------+--------
3 CG       | 180319  | 156962 | -13.0 |  -9.5 | -19.9 | +5.1
-----------+---------+--------+-------+-------+-------+--------
4 action   | 246899  | 218558 | -11.5 |  -8.1 | -16.3 | +2.0
-----------+---------+--------+-------+-------+-------+--------
5 noisy    | 460090  | 403804 | -12.2 | -10.9 | -18.6 | +2.9

Not much to say about visual quality. There’s hardly ever any significant difference between VAQ clips non-VAQ ones. And then only when comparing still frames. So ~10% smaller files without quality or speed drop. I think I might like that.

[Dark Shikari]

Originally Posted by Brother John

This is fantastic news! Thx for that patch, Dark Shikari. Now, there’s no chance you might get super-bored and port CRF to Xvid?

CRF wouldn't work in Xvid except at high quantizers; if your video is mostly QP2, the only thing you can do is drop a frame to QP1 (waste of bits) or raise it to QP3; not much leeway. H.264's logarithmic quantizer gives one vastly more room to do such a thing.

Xvid has really really nasty restrictions on quantizers. From what I recall:

1. A "DQuant" block is a block that has a different quantizer from the previous block. In the case of VAQ, this would be an edge block, for example.

2. DQuant blocks cannot have a difference of more than +/- QP from the frame QP (I think).

3. DQuant blocks cannot use 4mv. (WTF? Why?)

4. DQuant blocks cannot use GMC. (WTF? Why?)

5. B-frames have even weirder restrictions on quantizers.

6. The quantizer scale as a whole is crap. It has less quantizer resolution at high qualities, and high quantizer resolution at low qualities (i.e. the difference between 1 and 2 and 3 is much bigger than the difference between 25 and 26 and 27). The only real reason other than simplicity to lower quantizer resolution is to save bits when encoding QP_deltas... but this means you spend more bits on QP_deltas at lower bitrates, where it actually matters, but fewer bits at higher bitrates, where it hardly matters at all. How senseless.

[Inventive Software]

Holy s**t Dark Shikari! This is great news.

And are those weird restrictions you refer to specific to Xvid, or do they exist in MPEG-4 ASP as a spec?

[Dark Shikari]

Originally Posted by Inventive Software

Holy s**t Dark Shikari! This is great news.

And are those weird restrictions you refer to specific to Xvid, or do they exist in MPEG-4 ASP as a spec?

ASP

[Inventive Software]

That's a shame. No wonder you like to program for AVC so much!

[Didée]

Regarding (2) - AFAIR for DQUANT blocks, QP may not differ more than +/-2 from the previously coded block. So theoretically the full quant range from 1 to 31 can be used in a frame, just the difference between "adjacent" blocks is limited. Dunno, perhaps it's a restriction to make implementations of HW decoders easier? Could also be the reason for not allowing 4mv for DQUANTS, but again dunno.
GMC is "WTF" anyway - major effort, minor gain. There must be a reason GMC isn't in the spec's for AVC ...

[Dark Shikari]

Originally Posted by Didée

There must be a reason GMC isn't in the spec's for AVC ...

Because the vast majority of the gain of GMC can be simulated simply by using better MV prediction, basically.

[Ranguvar]

Yeah, even in animes where there's lots of panning+zooming, I see very little benefit with GMC. And it breaks many, many standalones and portables. Same thing with QPel. Not enough benefit to break the chance that I will use the video on one of those later on.

[Prettz]

Originally Posted by Inventive Software

Holy s**t Dark Shikari! This is great news.

And are those weird restrictions you refer to specific to Xvid, or do they exist in MPEG-4 ASP as a spec?

ASP I don't think was ever intended for the kind of high-quality videos Xvid initially allowed to be produced. And because of that, the ASP standard fought against the Xvid devs every step of the way towards increasing its quality. When you take a step back and look at the situation, it's not surprising that Xvid development hit an extremely flat plateau many years ago. It's not the Xvid developers' faults.

In the grand scheme of things, Advanced Simple Profile was a very transitory standard, and the current state of hardware has actually made ASP's initial purpose totally irrelevant, while the standard at the same time severely constrains any further development. Practically since 2004 there hasn't been anywhere to go except H.264.

Originally Posted by Didée

GMC is "WTF" anyway - major effort, minor gain. There must be a reason GMC isn't in the spec's for AVC ...

Basically put, the MPEG-4 group learned from the early experimentation in ASP that GMC was a great idea that, unexpectedly, totally failed to produce results in practice. Didee, I know you were around during the earliest Xvid days to see this proved before your own eyes.

Originally Posted by Ranguvar

Yeah, even in animes where there's lots of panning+zooming, I see very little benefit with GMC. And it breaks many, many standalones and portables. Same thing with QPel. Not enough benefit to break the chance that I will use the video on one of those later on.

I can't believe there are still standalone players that can't handle either GMC or QPel. Especially considering how little potential ASP has without QPel. Who are these people who own such incredibly old DVD players yet haven't thrown them in the trash yet?

[kypec]

Originally Posted by squid_80

If people want, I can add it to my xvid_encraw as a new option. Unfortunately I have no time to do my own tests at the moment.

Yes, I'd really appreciate that xvid_encraw version with separate VAQ option. The current state where it overrides lumimasking option is not a good solution IMHO. One could easily forget about it and be VERY confused with outcoming encodes later

[Brother John]

@Dark Shikari
I see. So VAQ basically is about as close to CRF as Xvid can get. Those bad quant restrictions indeed are one of the reason why I’d like to abandon Xvid. There are more important ones making me stay, though. And VAQ is definitely one more of those.

Originally Posted by Ranguvar

Same thing with QPel. Not enough benefit

QPel usually doesn’t even improve compression but lowers it! However it makes the picture look a bit grittier. It’s a kind of pseudo-detail and if you like the effect, QPel is a good thing.

Originally Posted by Prettz

ASP I don't think was ever intended for the kind of high-quality videos Xvid initially allowed to be produced.

When reading the MPEG-4 Visual specs I get the impression that MPEG was more concerned with the whole non-rectangle video/compositing thing than plain old movie encoding. Maybe the restrictions make more sense from that point of view.

[Ranguvar]

Originally Posted by Prettz

I can't believe there are still standalone players that can't handle either GMC or QPel. Especially considering how little potential ASP has without QPel. Who are these people who own such incredibly old DVD players yet haven't thrown them in the trash yet?

Sweetest little portable on the market
http://www.archos.com/products/gen_5...global&lang=en

Originally Posted by Brother John

QPel usually doesn’t even improve compression but lowers it! However it makes the picture look a bit grittier. It’s a kind of pseudo-detail and if you like the effect, QPel is a good thing.

Yep, to my eyes, not enough benefit to make up for lost compressibility.

[Ranguvar]

tested with VDub 1.7.8. The new AQ shaves 9MB off a formerly 54MB video. CQ 4, little over 3mins. This is Gears of War footage.

I couldn't tell a difference playing at full speed. Frame-by-frame, no AQ was better overall, but only very slightly. Some frames it actually seemed to help.

One thing I did notice was in the beginning, I had some text that was in a special font. It was a deep red, with small black holes in the letters. Just interesting.

No AQ:



AQ:




It's a lot easier to see a difference flipping between them

[Dark Shikari]

Originally Posted by Ranguvar

...

Note that AQ is *designed* to decrease quality on sharp edges, such as text like that, so you can clearly see the difference there.

The idea is that generally such sharp edges and complex text takes such a large amount of bits that the bits are probably better used elsewhere.

As you can tell, AQ lowers quality slightly at the same QP, but of course can improve quality if you try to keep to the same bitrate.

[laserfan]

I'm sorry, I'm having some trouble understanding the benefit of VAQ and wonder if someone can explain it in layman's terms?

My usage of Xvid these days is mostly HDTV profile w/single pass target 2.00 quantizer and I don't care about filesize--will I see a benefit anyway to VAQ? Or is text the only place one should see a difference?

[Dark Shikari]

Originally Posted by laserfan

I'm sorry, I'm having some trouble understanding the benefit of VAQ and wonder if someone can explain it in layman's terms?

My usage of Xvid these days is mostly HDTV profile w/single pass target 2.00 quantizer and I don't care about filesize--will I see a benefit anyway to VAQ? Or is text the only place one should see a difference?

It raises quantizer in the highest detail areas, where you won't notice it, and lowers quantizer in the flatter areas, where blocking/blurring often occur.

Text is an extreme example of the former.

[Ranguvar]

NVM

[salehin]

What a great addition. I'll definitely give it a go. Thanks a lot, DarkShikari

(Apart from our eyes ofc,) similar to x264 encode (SSIM, Avg QP, etc), are there any numbers that should look which would tell us something about the xvid quality?

[Dark Shikari]

Originally Posted by laserfan

I suppose I should have looked before I posted--the "with VAQ" encoding I made looked considerably "washed-out" compared to the non-AQ one. Or maybe I should say the non-AQ had more contrast (and less visible detail). I'll have to look later using my player and big screen--not sure my monitor is the best judge of this. So I don't know yet which I like better...

The attachments should at least give you a sense of it (lowered the res and output as jpg).

Use Tinypic/similar for images so we don't have to wait for attachments.

And save as full-size PNGs.

[laserfan]

Dark, I somehow managed to get the videos OPPOSITE from what they actually were!!!???!! Man, dunno how I did that--the non-AQ video is the washed-out one; the VAQ has better contrast. I will edit my other post, sorry...