About size and overflow treatment

[jon.schaffer]

@Kagura:
The information you gave in the sticky (Koepi edited it for good reasons) caught my attention. So I ran a few tests. You said it bursted my bubbles... and... really... I would like to know what does that mean :D
You said that according to Syskin, an overflow control strentgh of 0 is equal to a "10,10,10" parameter.
Here's the result (I use a 500 frames test - not much time - I know it's few; I just asked for a final target bitrate bigger than the 1st pass' one)[EDIT: all other settings are default]:

• 0,10,10:
http://membres.lycos.fr/norbert/xvid_tests/00,10,10.png

• 10,10,10
http://membres.lycos.fr/norbert/xvid_tests/10,10,10.png

Conclusion: exact same results - 0,10,10 = 10,10,10


And now for something completely different... (I am in the mood for quoting)

• default settings (5,5,5):
http://membres.lycos.fr/norbert/xvid_tests/05,05,05.png

• 0,5,5 (same improvement and degradation but... the 0 parameter)
http://membres.lycos.fr/norbert/xvid_tests/00,05,05.png

• 20,20,20 (regularly advised)
http://membres.lycos.fr/norbert/xvid_tests/20,20,20.png

• 0,20,20 (same improvement and degradation but... the 0 parameter):
http://membres.lycos.fr/norbert/xvid_tests/00,20,20.png

• 0,4,9 (the values I once recommended):
http://membres.lycos.fr/norbert/xvid_tests/00,04,09.png


Conclusion:
1) 0 as overflow control strentgh is not always equivalent to 10,10,10. It can be interesting to use since, as you can see, there's less high quantizer frames when it's used (I assume here that fewer high quantizer frames is better). We can notice that 20,20,20 produced high quantizer P-frames and B-frames while 0,20,20 produced only high quantizer B-frames.
2) If we go on assuming that fewer high quant is better, then the settings I once recommended are here not the best: the default values produce better "quantizer" results. 0,5,5 seem to be even better...

The quality results:

• 4th frame of the clip with default (5,5,5):
http://membres.lycos.fr/norbert/xvid_tests/frame_4_05,05,05.png

• 4th frame of the clip with 0,5,5:
http://membres.lycos.fr/norbert/xvid_tests/frame_4_00,05,05.png

• 4th frame of the clip with 20,20,20:
http://membres.lycos.fr/norbert/xvid_tests/frame_4_20,20,20.png

• 4th frame of the clip with 0,20,20:
http://membres.lycos.fr/norbert/xvid_tests/frame_4_00,20,20.png


As you can see 20,20,20 is very agressive, confirming what the number of high quantizer frames let us imagine...


All this for what?
Well... to point that 0 as overflow control strentgh seem to me more complex than a simple 10,10,10 setting...
So: what this 0 does exactly?

Don't know if it's a worthy post... But who knows... it may lead to something interesting.

edit: typos

[Kagura]

The log:

[20:30] > 2nd pass options
[20:31] > overflow control strength = 0
[20:31] <sysKin> sorry :) but whatever values are best, have to become defaults :)
[20:31] > max overflow improvement = 4
[20:31] > max degredation = 8
[20:31] > strict scale
[20:31] > everything else default
[20:31] <sysKin> ok
[20:31] > err.. no vbv
[20:31] <sysKin> so that's 10/4/8 if I remember correctly
[20:31] > 0/4/8
[20:31] > default = 5/5/5
[20:32] <sysKin> 0 == 10
[20:32] > ah, interesting
[20:32] > it said 0 = xvid choice
[20:32] <sysKin> yes, and that's 10 :)
[20:32] > =P
[20:32] > thought it was adaptive
[20:32] > but that's just for me
[20:32] > your mileage might vary
[20:32] <sysKin> never assume software is smart when it might be dumb :D
[20:32] > bah
[20:32] > i gave you too much credit =P
[20:33] <sysKin> this goes for all software, always
[20:33] > hehe
[20:33] <sysKin> all "automatic" always means "whatever we think might be right"

I meant 0 = 10 only for the control strength.

Oh, thanks for the graphs. They will really help in some analysis I'm doing. However, the 20/20/20 would never look like your graph for me b/c I cap the quants.

EDIT: Hmm... the 0/5/5 and 0/4/9 (or 8 for me) seem to be the best right now both from a low total average quant and quant distribution standpoint (which will very likely lead to better image quality). I was wondering... what are your bframe settings? I always put 2/1/1 since bframes are the first things that look like crap in high compression.

EDIT2: Oh, this is makign me wonder. How does loose/strict scaling affect the quant distribution with capped quants and 0/4/8 overflow settings?

EDIT3: Oh yes, it is a very interesting post. One of the gems of xvid config setting that hides behind another tab next to 2nd pass hehe. Genius! This will affect image quality as much as changing the matrix used, so pay attention ppl!

[jon.schaffer]

Originally posted by Kagura
I was wondering... what are your bframe settings? I always put 2/1/1 since bframes are the first things that look like crap in high compression.

In this test (I will edit the 1st post to notice that), all the settings are default (if you asked for my own encodings, I prefer use only 1 B-frame because moving mosquito noise over fixed backgrounds make me mad...)

the 20/20/20 would never look like your graph for me b/c I cap the quants.
You mean you use min. quant = 1 and cap the max quantizer?
I prefer capping the min. to 2... this way, with 2 CDs encodings, the codec does not really need to increase the quantizers in a so important way (no need to balance any Q1 frame).

[Kagura]

Okay, defaults. Thanks. All good.

I cap the quants 2-3 i-frame, 2-5 p-frame, 2-6 b-frame. Usually, they never go beyond that but it's just to prevent the unsightly quant 20 bframes that pop up hehe.

EDIT: Yea. I've noticed the moving mosquito noise too sometimes. But usually, I get enough of a compression gain from upping to 2 bframes that the end result usually looks better because of lower quants used. I'm still kinda holding out for strict scaling in the next few xvid builds b/c it really helped to alleviate the swimming noise effect hehe. Oh, 2/1.5/1 wouldn't work well and that I think induced a lot of your noise. 2/1/1 won't have the same problem since bframes are just 1 quant higher than the p ones and won't degrade as much. Helps to balance the fact that there are more of them.

[jon.schaffer]

I tested 10,20,20 and 10,5,5 and verified that 0==10.
Thanks for this.

[CruNcher]

that 0==10 doesn't need to be verified just look in the source

[jon.schaffer]

Originally posted by CruNcher
just look in the source
Of course... but I must give up on this :( :D

[SirCanealot]

Originally posted by Kagura

This will affect image quality as much as changing the matrix used, so pay attention ppl! [/B]

I just thought I'd note that setting these has definately improved quality on some anime I shoved on a while ago.
And it's all for freeeeee :D
So, yeah, go do it, people.

[Bruce Willis]

I'm often encoding high action HDTV (1080i) movies. They are up to 14 GB in size, and I 'squeeze' them into a 1 DVD-sized xvid, using vhs-best matrix. Xvid resolution is usually 1024x576.

The default settings (5-5-5) are not probably not perfect for this stuff. I think it is better if the quants are more spread. At this high bitrate, the easy scenes will still do fine with a high quant, and on the other hand, the fast motion scenes will improve noticably with a lower quant.

I tried 20-20-20, it looks like to be an improvement over the default settings.

What schema do you think is good for this kind of encoding? Do you have ideas about the other settings (keyframes adjustments) as well?

Also, can you explain what the 3 overflow settings exactly do? It is not clear to me.

[jon.schaffer]

Also, can you explain what the 3 overflow settings exactly do? It is not clear to me.

See Crusty's XviD FAQ: http://ronald.vslcatena.nl/docs/xvidfaq.html and search for Overflow Treatment...

[Didée]

I think it is better if the quants are more spread. At this high bitrate, the easy scenes will still do fine with a high quant, and on the other hand, the fast motion scenes will improve noticably with a lower quant.
No. You cannot feed high-bitrate scenes with bits from low-bitrate scenes. Loosely speaking, you'd run out of bits from easy scenes *before* the demanding scenes will even start to look better.

Filling jiggers from a barrel is no problem. Filling a barrel by means of a jigger is ... hopeless.

[Bruce Willis]

The link of the 'unofficial xvid faq' is very good, it explains a lot to me.

I understand now that the overflow treatment is only important if you get badly sized files. What I really was looking for, was Curve Compression. I still think that it can be useful to use that, for instance if you have blocky low-bitrate scenes.

Thanks for your replies.