Implementing a Xvid-like adaptive quantization scheme

[Magno]

Hi, everyone!

Since some months, I am programming my own home-made MPEG2 coder and i would like to implement some adaptive quantization to increase quality.
I know there are lots of video gurus in the forum, so I want to ask them how adaptive quantization is performed in Xvid codec. I think that it implies increasing/decreasing quantizer for each DCT-coded block, but that is to simple and I guess there is a lot more about it that I don't know.

Would you mind making it me clear, please?

Thank you in advance! :D

[Cyberace]

Why not help improve/enhance libmpeg2 (link) (http://libmpeg2.sourceforge.net/) instead? :confused: ...many open source projects uses it so you be helping many :D

PS! All libmpeg2 users don't actively feed their improvements back to the original libmpeg2 project, don't know if these do:
- MPlayer (http://www.mplayerhq.hu)
- VeMP (VIA enhanced MPlayer) (http://sourceforge.net/projects/vemp), (also contain VIA hardware-specific MPEG-2 decoding acceleration)
- XINE (http://www.xinehq.de), (two variations; "xine-lib/src/libmpeg2/" and "xine-lib/src/libmpeg2new/")
- VeXP (VIA enhanced Xine Player) (http://sourceforge.net/projects/viaexp/), (also contain VIA hardware-specific MPEG-2 decoding acceleration)
- FFmpeg (libavcodec/libavformat) (http://sourceforge.net/projects/ffmpeg), (yes FFmpeg do contain MPEG1/MPEG2 code derivitive from libmpeg2)
- MPC (Media Player Classic) (http://sourceforge.net/projects/guliverkli)
- VLC (VideoLan-Client) (http://www.videolan.org)
- TCVP (http://tcvp.sourceforge.net/)
- DScaler Deinterlacer/Scaler(deinterlace) (http://sourceforge.net/projects/deinterlace)

[Magno]

Well, i am really programming my own coding for a matter of personal growth and improvement; i am also learning a lot about small details and issues in video codecs, so that's why I am asking about adaptive quantization as I don't really know how it works. I have some own ideas for improving video quality and maybe someone befor me has done the same and proved that idea unuseful, so i would like to get the advice of those who know the most.

[Cyberace]

Since nobody else replyed yet, does this libmpeg2 patch to modify the quantization matrices whilst decoding (link) (http://sourceforge.net/mailarchive/forum.php?thread_id=5291184&forum_id=729) help?

[Magno]

Thanks, Cyberace, but it doesn`t :(

What I need is somebody to explain me how adaptive quantization should be performed for maximum improvement in overall quality image. If possible, I would like more a theorical explanation rather than some example code, but it would be helpful too.

I think some of forum's members have developed parts of Xvid, so they know how adaptive quantization works....

[Magno]

Is nobody willing to help me, please?

[Koepi]

Think about texture masking:

you have some surfaces which are "plain" (like a blue sky or in anime). Then you have complex textured areas like trees. You can use higher quanitsation on the tree as the error/artefact won't be noticed as fast in that complex area as it would be in the sky.

IHTH

Koepi

[Magno]

I thought "adaptive quantization" meant something like this: if after DCT coding and quantizing the image residual you have some pattern like this one

A B C 0 0 0 0 0
D 0 0 0 0 0 0 0
E 0 0 0 0 0 0 0
0 F 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0

where all the coefficients are greater than 1, you can increase that macroblock's quantizer to make the lower coefficient 1 so you don't lose any mid-frequency component.
I thought the point in adaptive quantization was to choose the proper threshold when any high-frequency coeffcient is not zero so you can achieve a good trade-off between quality and bitrate.

So, making it simpler, Koepi: if the macroblock has a lot of high-frequency components, you quantize it higher, right?

And thanks for you reply :D