Adaptative Quantization ON/OFF comparison

[jon.schaffer]

Continuation of a discussion here:
http://forum.doom9.org/showthread.ph...294#post497241

Originally posted by Teegedeck
Can you do a comparison sometime and post screenshots of some frames from the same video with and without AQ?

...You could/should make it a new comparison-thread and perhaps leave a redirect here.

Here it is. I tested two movie sequences. The first belongs to Minority Report (1600 frames sequence - about 100 frames are hurt), the second to Singin' in the Rain (1000 frames sequence - the whole sequence is hurt) - I choose these ones because they presented the problem in these full-movies encodings. I used simple AVS scripts (AviSynth 2.53 - MPEG2Dec3dg_110.dll plug-in autoloaded):


MPEG2Source("D:\Werxx\Video\Minority Report\Minority.d2v")
ConvertToYV12
BicubicResize(640,256,0,0.75,4,80,716,416)

###

MPEG2Source("D:\Werxx\Video\Singin' in the rain\Singin.d2v")
ConvertToYV12
BicubicResize(640,480,0,0.75)


Then I used VirtualDub 1.5.10 and XviD 1.0 Final to make 2-pass encodings, with the XviD default settings (except that I used Full Quality 1st-pass - this improved quality of the 2nd pass). I then verified these first results with 1-pass encodings at fixed quantizers - Q2, Q3 and Q4 (I got the same results). I didn't post the original frames (from DVD), their quality is OK.

•The results (captured in VDub; below is the same frame using AviSynth's DirectShowSource plug-in, with the FFDShow's OSD - but... you know it...):

Without AQ:


With AQ:


=>As you can see, for this frame, the normal quantizer is 3, but numerous macroblocks are Q4. For this picture and for these zones, it's too high...


• The following pictures are the same frame as above, but the sequence was encoded with personnal settings (MPEG-quantization; only 1 B-frame each time; B-frame quantizer offset = 0.75; key-frame every 250 frames - all the rest is default):

Without AQ:


With AQ:


=> My settings lead to a Q4 frame (with more detail thanks to MPEG-quantization), which then lead up to Q6-macroblocks with AQ...


• Results of the second sequence (once again: XviD default settings are used, with I]Full Quality 1st-pass[/I]):

Without AQ:


With AQ: (look at the lantern)


Without AQ:


With AQ: (the left side of the picture is the most hurt)



=> here, even some details (the lantern on the first picture) are hurted.



Maybe this blocking may appear slight for some... in particular on still images (and especially the second frame from Singin' in the Rain). But the problem is more important when the movie is played back: the blocks are moving and become much more visible.
It seems to me that the problem in these cases is that the AQ process is applied to zones which are not dark enough nor bright enough for the eyes... AQ is here applied to zones with nearly-plain colors or color gradients, and this seems to be too destructive.

By disabling AQ in such scenes, we can keep good quality without losing much bytes for the rest of the movie.
It's the reason why I evoked a possibility to enable/disable AQ for specific scenes ("zones"). For example, for some scenes with dark colored lightings/skies or underwater views.

Anyway AQ remains, for me, a powerful option... "Still, the effect is remarkable" as you said... ;-)



Hope it may be useful,

Jon


PS: I used low-compressed JPEG and compared them with PNG, and the quality is OK for this test (since the thing to see is blocking). And... beware, photobucket.com is cool... but... pictures above 250 KB are downsized...

[Selur]

yup, would be nice to have 'adaptive quantization' additionally in the zone options which would override the general setting.

In general I have to say that it allways is a question on how much brithness your video get's during playback.

Cu Selur

[Teegedeck]

Thanks a lot Jon; this is the kind of test I dig. What I find remarkable though is that you've selected frames that always have a higher mean-quantizer with AQ than without AQ. If both files have the same filesize, there are bound to be some frames where the difference between the AQ-frame's quant and the non-AQ-frame's quant is a little smaller. The average quant of your two-pass results should always be the same if the filesize of the results is the same, so AQ-frames cannot always have higher quants than non-AQ-frames. I mean, if the AQ frame has a quant of 5.41 it is pretty expectable that it'll look worse than the non-AQ frame at quant 4. Also the difference between AQ-frame->non-AQ-frame of 4.62->3.00 is quite big.

To judge AQ in a frame-by-frame comparison, the contrast is a a bit too big, I think.

I think your test could rather exhibit that the redistribution of bits for the 2nd pass is not so good with AQ. The two-pass algo seems to attain too few bits to those specific frames (probably to the benefit of some other frames). What do you think, could there be some truth in that?

[jon.schaffer]

Hi Teegedek,

Yes, I think you found the right point. In fact, this is the problem I was asking for myself...
I agree with you about the higher quant in frames with AQ...

quote
...you've selected frames that always have a higher mean-quantizer with AQ than without AQ

Well...
To jusitfy me (!), the first thing I'll say is: I selected such frames because... they exist! And they are plenty enough to be visible when playing back...

But, I repeat, I totally agree with what you said...
The problem is that there are lot of such frames in those sequences (that are present in the 2-pass encodings of the full movie [target: 2CD per movie]). I obviously guess - as you said - that the saved bits (due to AQ) are redistributed in other frames which mean quantizers are decreased. But, it was not the case in those sequences (to be sure, I first extracted these sequences from the full-length encodings and studied them...). In other words: those difficult-to-encode-sequences are used by AQ to give bits to the rest of the movie, but they do not benefit themselves from these saved bits... since their basis-quantizer is not decreased (and is even increased). Hum... I don't know if I'm clear...

But a problem still remains: the fact that the AQ is applied to regions of the frames that seem (to me) not... good for AQ...
Search in movies for scenes with dark-blue skies, or with a blue-screen (in some movies "making-of"), or with underwater scenes (Abyss or Titanic introduction), and you'll see some blocks. It seems that AQ tend to consider plain-color surface as "OK to compress a lot" (especially the blue color?). But, I think that AQ is more "at ease" in really dark/light regions.
Let's say that I persist in saying that some regions of these sequences should'nt have been detected and used by AQ... ;-P.

Furthermore, IIRC, the bit-gain is higher when going from quant2->quant3 than from quant3->quant4 (and so on...). So, maybe when a frame is Q4, going to Q6 for some macroblocks is less efficient than AQ applied on Q2/Q3 frames? It's just a guess (it's even a question...;-) ).

In the end, a question: I checked the quantizers on a recent encoding (which used AQ) and observed that the black screens (plain black) at the very beginning of the movie (before credits) do not present AQ, i.e. the "normal" quantizer is used. However, such frames could be encoded easily with high quantizers when AQ is enabled. So: why doesn't a totally black screen involve AQ-macroblocks, whereas the effect would be invisible?
Does AQ absolutely need a highly contrasted frame ("psychovisual")? I would tend to think that the absolute luminance value could be used, in some cases, by AQ (0->10 of luminance on the entire frame could lead to use AQ...)?
[in my encoding, the quantizer couldn't be increased by the rate control, because it was a full-quality 1st pass... ]

(if I'm not clear, feel free to claim it!)


Yours,

Jon

[Teegedeck]

I suppose those black frames simply don't weigh much in terms of bits. Such frames are treated favorably because usually small frames suffer most easily from higher quantization and you don't gain much from compressing them stronger. It's up to you to tell XviD (via 'zones') that these black frames actually aren't important and can be compressed stronger. That much for 2-pass redistribution. About AQ: One of the biggest problems with AQ in the old days was that using absolute luminance values resulted in grossly wrong decisions. I guess we can live better without compressing those black frames strongly (as you don't gain much from it, as said) than we would with a higher risk of wrong decisions here. But sysKin would know better about that than me; he corrected that lumi-masking code, after all.

[jon.schaffer]

I wasn't very clear on - at least - a point (in fact, I'm asking who in the world could understand what I meant!)
And I would like to illustrate that what you said was right... (and it may be useful for someone else...)

quote (from myself)
I obviously guess - as you said - that the saved bits (due to AQ) are redistributed in other frames which mean quantizers are decreased. But, it was not the case in those sequences...


What I meant was (mostly concerning the Minority Report test):

my encoded sequence contained some various scenes AND one hard-to-encode scene (the "eye" scene - yes, it's an eye!), on which AQ was painful...
The AQ applied on this hard-to-encode scene helped only to improve the surrounding various stuff, not this "eye"-scene itself... but this improvement is not easily noticeable.

I didn't mean at all that after using AQ, none of the other frames took advantage of this (by having an increased mean quantizer): it is OBVIOUSLY the case. Else, where would the saved bits be???
Hiding somewhere, just to fatten the file-size??? He, he ,he ;-)

<sorry - with english language... I easily get confused>

Concerning the Singing in the Rain test, it's more difficult: on 1000 frames, about 900 show this blue background which hurt me so...
AQ degrades this blue background quality, and the saved bits are used to decrease some frames quantizers.
Here's an example: I found a frame (it's not alone, of course!) which was Q4 in the "without AQ encoding" and which was Q3.35 in the "with AQ encoding" (i.e. "basis"-quantizer decreased to 3 and some MB raised to 4)...

Without AQ:


With AQ:


This is obviously the awaited behaviour of AQ...
But the overall quality - for this scene - is really better without AQ... (I know, I know, I'm repeating again and again the same thing... ;-) )

Maybe, one will say: "use the deblocking PP, it works sooo well on this type of plain-surface...". Well, I don't like PP too much... it hurts details... (this is another debate!)

Thanks for the information about those black-screens (it's all logical!)


Yours,

Jon.

[Teegedeck]

Thanks again for this test, Jon!

Well, so we should be looking forward for the option to choose AQ in zones only, in 1.1.