About size and overflow treatment

jon.schaffer · 1st April 2005, 18:09

@Kagura:
The information you gave in the sticky (Koepi edited it for good reasons) caught my attention. So I ran a few tests. You said it bursted my bubbles... and... really... I would like to know what does that mean

You said that according to Syskin, an overflow control strentgh of 0 is equal to a "10,10,10" parameter.
Here's the result (I use a 500 frames test - not much time - I know it's few; I just asked for a final target bitrate bigger than the 1st pass' one)[EDIT: all other settings are default]:

• 0,10,10:

• 10,10,10

Conclusion: exact same results - 0,10,10 = 10,10,10

And now for something completely different... (I am in the mood for quoting)

• default settings (5,5,5):

• 0,5,5 (same improvement and degradation but... the 0 parameter)

• 20,20,20 (regularly advised)

• 0,20,20 (same improvement and degradation but... the 0 parameter):

• 0,4,9 (the values I once recommended):

Conclusion:
1) 0 as overflow control strentgh is not always equivalent to 10,10,10. It can be interesting to use since, as you can see, there's less high quantizer frames when it's used (I assume here that fewer high quantizer frames is better). We can notice that 20,20,20 produced high quantizer P-frames and B-frames while 0,20,20 produced only high quantizer B-frames.
2) If we go on assuming that fewer high quant is better, then the settings I once recommended are here not the best: the default values produce better "quantizer" results. 0,5,5 seem to be even better...

The quality results:

• 4th frame of the clip with default (5,5,5):

• 4th frame of the clip with 0,5,5:

• 4th frame of the clip with 20,20,20:

• 4th frame of the clip with 0,20,20:

As you can see 20,20,20 is very agressive, confirming what the number of high quantizer frames let us imagine...

All this for what?
Well... to point that 0 as overflow control strentgh seem to me more complex than a simple 10,10,10 setting...
So: what this 0 does exactly?

Don't know if it's a worthy post... But who knows... it may lead to something interesting.

edit: typos

Kagura · 1st April 2005, 18:21

The log:

[20:30] > 2nd pass options
[20:31] > overflow control strength = 0
[20:31] <sysKin> sorry

but whatever values are best, have to become defaults

[20:31] > max overflow improvement = 4
[20:31] > max degredation = 8
[20:31] > strict scale
[20:31] > everything else default
[20:31] <sysKin> ok
[20:31] > err.. no vbv
[20:31] <sysKin> so that's 10/4/8 if I remember correctly
[20:31] > 0/4/8
[20:31] > default = 5/5/5
[20:32] <sysKin> 0 == 10
[20:32] > ah, interesting
[20:32] > it said 0 = xvid choice
[20:32] <sysKin> yes, and that's 10

[20:32] > =P
[20:32] > thought it was adaptive
[20:32] > but that's just for me
[20:32] > your mileage might vary
[20:32] <sysKin> never assume software is smart when it might be dumb

[20:32] > bah
[20:32] > i gave you too much credit =P
[20:33] <sysKin> this goes for all software, always
[20:33] > hehe
[20:33] <sysKin> all "automatic" always means "whatever we think might be right"

I meant 0 = 10 only for the control strength.

Oh, thanks for the graphs. They will really help in some analysis I'm doing. However, the 20/20/20 would never look like your graph for me b/c I cap the quants.

EDIT: Hmm... the 0/5/5 and 0/4/9 (or 8 for me) seem to be the best right now both from a low total average quant and quant distribution standpoint (which will very likely lead to better image quality). I was wondering... what are your bframe settings? I always put 2/1/1 since bframes are the first things that look like crap in high compression.

EDIT2: Oh, this is makign me wonder. How does loose/strict scaling affect the quant distribution with capped quants and 0/4/8 overflow settings?

EDIT3: Oh yes, it is a very interesting post. One of the gems of xvid config setting that hides behind another tab next to 2nd pass hehe. Genius! This will affect image quality as much as changing the matrix used, so pay attention ppl!

jon.schaffer · 1st April 2005, 18:36

Quote:

Originally posted by Kagura
I was wondering... what are your bframe settings? I always put 2/1/1 since bframes are the first things that look like crap in high compression.

In this test (I will edit the 1st post to notice that), all the settings are default (if you asked for my own encodings, I prefer use only 1 B-frame because moving mosquito noise over fixed backgrounds make me mad...)

Quote:

the 20/20/20 would never look like your graph for me b/c I cap the quants.

You mean you use min. quant = 1 and cap the max quantizer?
I prefer capping the min. to 2... this way, with 2 CDs encodings, the codec does not really need to increase the quantizers in a so important way (no need to balance any Q1 frame).

Kagura · 1st April 2005, 18:40

Okay, defaults. Thanks. All good.

I cap the quants 2-3 i-frame, 2-5 p-frame, 2-6 b-frame. Usually, they never go beyond that but it's just to prevent the unsightly quant 20 bframes that pop up hehe.

EDIT: Yea. I've noticed the moving mosquito noise too sometimes. But usually, I get enough of a compression gain from upping to 2 bframes that the end result usually looks better because of lower quants used. I'm still kinda holding out for strict scaling in the next few xvid builds b/c it really helped to alleviate the swimming noise effect hehe. Oh, 2/1.5/1 wouldn't work well and that I think induced a lot of your noise. 2/1/1 won't have the same problem since bframes are just 1 quant higher than the p ones and won't degrade as much. Helps to balance the fact that there are more of them.

jon.schaffer · 1st April 2005, 18:45

I tested 10,20,20 and 10,5,5 and verified that 0==10.
Thanks for this.

CruNcher · 1st April 2005, 19:19

that 0==10 doesn't need to be verified just look in the source

jon.schaffer · 1st April 2005, 19:47

Quote:

Originally posted by CruNcher
just look in the source

Of course... but I must give up on this

SirCanealot · 2nd April 2005, 14:23

Quote:

Originally posted by Kagura

This will affect image quality as much as changing the matrix used, so pay attention ppl! [/B]

I just thought I'd note that setting these has definately improved quality on some anime I shoved on a while ago.
And it's all for freeeeee

So, yeah, go do it, people.

Bruce Willis · 31st May 2005, 18:19

I'm often encoding high action HDTV (1080i) movies. They are up to 14 GB in size, and I 'squeeze' them into a 1 DVD-sized xvid, using vhs-best matrix. Xvid resolution is usually 1024x576.

The default settings (5-5-5) are not probably not perfect for this stuff. I think it is better if the quants are more spread. At this high bitrate, the easy scenes will still do fine with a high quant, and on the other hand, the fast motion scenes will improve noticably with a lower quant.

I tried 20-20-20, it looks like to be an improvement over the default settings.

What schema do you think is good for this kind of encoding? Do you have ideas about the other settings (keyframes adjustments) as well?

Also, can you explain what the 3 overflow settings exactly do? It is not clear to me.

jon.schaffer · 1st June 2005, 10:40

Quote:

Originally Posted by Bruce Willis

Also, can you explain what the 3 overflow settings exactly do? It is not clear to me.

See Crusty's XviD FAQ: http://ronald.vslcatena.nl/docs/xvidfaq.html and search for Overflow Treatment...

Didée · 1st June 2005, 11:16

Quote:

Originally Posted by Bruce Willis

I think it is better if the quants are more spread. At this high bitrate, the easy scenes will still do fine with a high quant, and on the other hand, the fast motion scenes will improve noticably with a lower quant.

No. You cannot feed high-bitrate scenes with bits from low-bitrate scenes. Loosely speaking, you'd run out of bits from easy scenes *before* the demanding scenes will even start to look better.

Filling jiggers from a barrel is no problem. Filling a barrel by means of a jigger is ... hopeless.

Bruce Willis · 2nd June 2005, 12:32

The link of the 'unofficial xvid faq' is very good, it explains a lot to me.

I understand now that the overflow treatment is only important if you get badly sized files. What I really was looking for, was Curve Compression. I still think that it can be useful to use that, for instance if you have blocky low-bitrate scenes.

Thanks for your replies.

1st April 2005, 18:09	#1 \| Link
jon.schaffer Eternally Junior Member Join Date: Apr 2004 Location: France Posts: 230	About size and overflow treatment @Kagura: The information you gave in the sticky (Koepi edited it for good reasons) caught my attention. So I ran a few tests. You said it bursted my bubbles... and... really... I would like to know what does that mean You said that according to Syskin, an overflow control strentgh of 0 is equal to a "10,10,10" parameter. Here's the result (I use a 500 frames test - not much time - I know it's few; I just asked for a final target bitrate bigger than the 1st pass' one)[EDIT: all other settings are default]: • 0,10,10: • 10,10,10 Conclusion: exact same results - 0,10,10 = 10,10,10 And now for something completely different... (I am in the mood for quoting) • default settings (5,5,5): • 0,5,5 (same improvement and degradation but... the 0 parameter) • 20,20,20 (regularly advised) • 0,20,20 (same improvement and degradation but... the 0 parameter): • 0,4,9 (the values I once recommended): Conclusion: 1) 0 as overflow control strentgh is not always equivalent to 10,10,10. It can be interesting to use since, as you can see, there's less high quantizer frames when it's used (I assume here that fewer high quantizer frames is better). We can notice that 20,20,20 produced high quantizer P-frames and B-frames while 0,20,20 produced only high quantizer B-frames. 2) If we go on assuming that fewer high quant is better, then the settings I once recommended are here not the best: the default values produce better "quantizer" results. 0,5,5 seem to be even better... The quality results: • 4th frame of the clip with default (5,5,5): • 4th frame of the clip with 0,5,5: • 4th frame of the clip with 20,20,20: • 4th frame of the clip with 0,20,20: As you can see 20,20,20 is very agressive, confirming what the number of high quantizer frames let us imagine... All this for what? Well... to point that 0 as overflow control strentgh seem to me more complex than a simple 10,10,10 setting... So: what this 0 does exactly? Don't know if it's a worthy post... But who knows... it may lead to something interesting. edit: typos __________________ "Methinks it is like a weasel" (Hamlet in Hamlet, by Shakespeare) Last edited by jon.schaffer; 1st April 2005 at 18:37.

1st April 2005, 18:21	#2 \| Link
Kagura Perennially Silent Member Join Date: Oct 2004 Location: Protector of Skuld and Guardian of Her Hammer Posts: 64	The log: [20:30] > 2nd pass options [20:31] > overflow control strength = 0 [20:31] <sysKin> sorry but whatever values are best, have to become defaults [20:31] > max overflow improvement = 4 [20:31] > max degredation = 8 [20:31] > strict scale [20:31] > everything else default [20:31] <sysKin> ok [20:31] > err.. no vbv [20:31] <sysKin> so that's 10/4/8 if I remember correctly [20:31] > 0/4/8 [20:31] > default = 5/5/5 [20:32] <sysKin> 0 == 10 [20:32] > ah, interesting [20:32] > it said 0 = xvid choice [20:32] <sysKin> yes, and that's 10 [20:32] > =P [20:32] > thought it was adaptive [20:32] > but that's just for me [20:32] > your mileage might vary [20:32] <sysKin> never assume software is smart when it might be dumb [20:32] > bah [20:32] > i gave you too much credit =P [20:33] <sysKin> this goes for all software, always [20:33] > hehe [20:33] <sysKin> all "automatic" always means "whatever we think might be right" I meant 0 = 10 only for the control strength. Oh, thanks for the graphs. They will really help in some analysis I'm doing. However, the 20/20/20 would never look like your graph for me b/c I cap the quants. EDIT: Hmm... the 0/5/5 and 0/4/9 (or 8 for me) seem to be the best right now both from a low total average quant and quant distribution standpoint (which will very likely lead to better image quality). I was wondering... what are your bframe settings? I always put 2/1/1 since bframes are the first things that look like crap in high compression. EDIT2: Oh, this is makign me wonder. How does loose/strict scaling affect the quant distribution with capped quants and 0/4/8 overflow settings? EDIT3: Oh yes, it is a very interesting post. One of the gems of xvid config setting that hides behind another tab next to 2nd pass hehe. Genius! This will affect image quality as much as changing the matrix used, so pay attention ppl! __________________ Proud Member of the Scandanavian Contingent Last edited by Kagura; 1st April 2005 at 18:33.

1st April 2005, 18:40	#4 \| Link
Kagura Perennially Silent Member Join Date: Oct 2004 Location: Protector of Skuld and Guardian of Her Hammer Posts: 64	Okay, defaults. Thanks. All good. I cap the quants 2-3 i-frame, 2-5 p-frame, 2-6 b-frame. Usually, they never go beyond that but it's just to prevent the unsightly quant 20 bframes that pop up hehe. EDIT: Yea. I've noticed the moving mosquito noise too sometimes. But usually, I get enough of a compression gain from upping to 2 bframes that the end result usually looks better because of lower quants used. I'm still kinda holding out for strict scaling in the next few xvid builds b/c it really helped to alleviate the swimming noise effect hehe. Oh, 2/1.5/1 wouldn't work well and that I think induced a lot of your noise. 2/1/1 won't have the same problem since bframes are just 1 quant higher than the p ones and won't degrade as much. Helps to balance the fact that there are more of them. __________________ Proud Member of the Scandanavian Contingent Last edited by Kagura; 1st April 2005 at 18:46.

1st April 2005, 18:45	#5 \| Link
jon.schaffer Eternally Junior Member Join Date: Apr 2004 Location: France Posts: 230	I tested 10,20,20 and 10,5,5 and verified that 0==10. Thanks for this. __________________ "Methinks it is like a weasel" (Hamlet in Hamlet, by Shakespeare)

1st April 2005, 19:19	#6 \| Link
CruNcher Registered User Join Date: Apr 2002 Location: Germany Posts: 4,926	that 0==10 doesn't need to be verified just look in the source __________________ all my compares are riddles so please try to decipher them yourselves :) It is about Time Join the Revolution NOW before it is to Late ! http://forum.doom9.org/showthread.php?t=168004

31st May 2005, 18:19	#9 \| Link
Bruce Willis Registered User Join Date: Mar 2004 Posts: 24	I'm often encoding high action HDTV (1080i) movies. They are up to 14 GB in size, and I 'squeeze' them into a 1 DVD-sized xvid, using vhs-best matrix. Xvid resolution is usually 1024x576. The default settings (5-5-5) are not probably not perfect for this stuff. I think it is better if the quants are more spread. At this high bitrate, the easy scenes will still do fine with a high quant, and on the other hand, the fast motion scenes will improve noticably with a lower quant. I tried 20-20-20, it looks like to be an improvement over the default settings. What schema do you think is good for this kind of encoding? Do you have ideas about the other settings (keyframes adjustments) as well? Also, can you explain what the 3 overflow settings exactly do? It is not clear to me. Last edited by Bruce Willis; 31st May 2005 at 19:30.

2nd June 2005, 12:32	#12 \| Link
Bruce Willis Registered User Join Date: Mar 2004 Posts: 24	The link of the 'unofficial xvid faq' is very good, it explains a lot to me. I understand now that the overflow treatment is only important if you get badly sized files. What I really was looking for, was Curve Compression. I still think that it can be useful to use that, for instance if you have blocky low-bitrate scenes. Thanks for your replies.