Why some peple limit quantization 2,12,2,14 ?

[dihelson]

Hello, Friends,

I have seen on some hints for making Xvid, sometimes, people change the default quantization 1,31,2,31 for others, for example, 2,12,2,14.
In what this change can affect quality, or any other parameter, does it affect speed? Since I often use 2 or 3 CDs, what should I use, the default ?

Thanks in advance,
Dihelson

[TheXung]

Some people have complete faith in XviD's 2-pass curve compression to give them the best possible encoding.

Other's don't. They encode on certain theories that for instance, that very rare scenes can look good at quants above 12 or 14 and they don't want any scenes at such quants even though XviD's 2-pass may generate those.


*How on earth can the defaults be 1-31 I-frames if the first pass is done at quant 2

[KpeX]

Originally posted by Koepi
We more or less hardcoded min quant 2 It's quite possible that even in 1pass quant mode it's upping it to 2.

(Since it's consensus amongst the devs that quant1 is of no use and led to a few bug reports which were easy to solve by limiting quantizer range to 2-31...).

That was originally posted by Koepi in the thread on the 03052003 Xvid build, and I believe it is still accurate. (correct me if i'm mistaken) I.E. there is no quantizer 1. And what TheXung said is the only reason I know of for capping quantizers. Remember you can always check average quantizer with ffdshow's OSD option.

[Zarxrax]

I use quant 1 quite often when I need the best quality. I can see a definate difference in quality between quant 1 and 2. Is it bad for me to be doing this?

[TheXung]

Originally posted by Zarxrax
I use quant 1 quite often when I need the best quality. I can see a definate difference in quality between quant 1 and 2. Is it bad for me to be doing this?

My concern about ever using quant 1 is that it takes a rather powerful processor to decode. Often times, you're talking about three times as much data to process over quant 2. I mean if all you ever plan to do is watch it on your current rig, then there isn't much harm, but I have clips 640x416 mostly quant 3 with qpel and bframes that cannot be played back on an 800 thunderbird. I'm not saying all encodes at quant 1 will fail to playback just the high bitrate stuff.

The other concern is if you ever want it to play on a set top box. It's the same issue. The cause of the concern is that there is currently no format standards for how mpeg4 should be done. It isn't set in stone just how much bitrate the processors have to process. The closest thing we have now is DivX Network's validation program but you'll notice that none of their profiles use qpel or multiple bframes.

[Selur]

The only reason I stopped using quant 1 ist, that it's not 'supported' during 2pass encoding like KpeX posted.

I would recommend to witch to a high quality custom matrix, at least for me that helped, encoding quality at higher bitrates a lot.

Cu Selur

[K_R]

Anime, in particular, does not handle high q too well due the large amount of flat colour combined with high contrast. So it's better have to restrict the range, rather than let the odd q=14 fly in.

[mf]

Originally posted by K_R
Anime, in particular, does not handle high q too well due the large amount of flat colour combined with high contrast. So it's better have to restrict the range, rather than let the odd q=14 fly in.
As an anime encoder, I must disagree. Ratecontrol has never let me down with quant decision, even though dev-api-3 has a bug somewhere regarding that. I have yet to encounter it. For now, quant capping is senseless, ESPECIALLY if you have no real knowledge of the codec. Use it only if you know what you're doing, and it seems to me your thin arguments aren't really showing you know what you're doing.

[K_R]

Originally posted by mf
it seems to me your thin arguments aren't really showing you know what you're doing.

Perhaps... But I have yet had people tell me my anime encodes look like they were done by someone with NFI. Have you not noticed excessive noise around subtitles when the q start to go past 6?

[mf]

Originally posted by K_R
Perhaps... But I have yet had people tell me my anime encodes look like they were done by someone with NFI. Have you not noticed excessive noise around subtitles when the q start to go past 6?
Depends. A keyframe with subtitles at a high quant looks crap, yes, but that shouldn't happen with proper ratecontrol (and it never has with me). However, a P-frame with high quant at almost no movement, with the subtitle already coded (that means, subtitle start was a few frames earlier), quant won't matter. You could even do it at quant 31 if there's no motion, and you wouldn't notice a thing. The only flaw here is that on those moments it won't matter if you have quant 1 or 31 bitrate-wise, because there's nothing to code anyway. The only thing that might help is high-quant B-frames, since that could decrease noise. Anyway, I still don't have proof why you don't trust xvid's ratecontrol.

[yingx2]

Hi, mf
I think I have something different to say.
Originally posted by mf
You could even do it at quant 31 if there's no motion, and you wouldn't notice a thing.
Not quite true. Perhaps you could try using quant 31 to encode a 5-sec clip with exactly identical frames in it. You would notice a big thing.

If you encode one of these still sences using quantizer mode at quant 2, and raise your bframe quant ratio high enough to get q=31 bframes, yes, the result would look good. But it has to be an absolutely still scene (low motion scene would still look like crap with this method), and the framesize of those q=31 bframes would also be abnormally large.

am I mistaken here?
Originally posted by mf
Anyway, I still don't have proof why you don't trust xvid's ratecontrol.
I always limit my quant range to 2,3, or 3,4 , or 4,5...., after knowing how compressible the source is, in order to get a nearly constant quality result, and prevent the codec from doing potentially stupid things.

Maybe I can make a tricky clip to prove that the rc of xvid is not always reliable. I'll try.:)

[sungey]

i remember syskin said that in one of his encodes, there was a part where the quant jumped too high ... thus producing very bad result ... he then recommended capping the quantizer.

as far as i remember ... most still scenes are not completely still scene. There are tiny noises. Use ffdshow "show motion vector" to see this phenomenon.

Personally i have been capping the quantizer since the Nandub days (it was called DRF at that time though).

[Acaila]

Maybe I can make a tricky clip to prove that the rc of xvid is not always reliable. I'll try.I once got a clip from someone who proved just that. It was high motion in the beginning, low motion in the middle and pure black at the end. XviD didn't even reach the correct filesize (in 2-pass mode), and the overall quality was crap after encoding. So XviD's RC is far from perfect.

I too usually cap after knowing what the quant ranges are for the uncapped encode. Call me paranoid, but I see it more like playing safe.

[mf]

Originally posted by yingx2
Hi, mf
I think I have something different to say.

Not quite true. Perhaps you could try using quant 31 to encode a 5-sec clip with exactly identical frames in it. You would notice a big thing.

If you encode one of these still sences using quantizer mode at quant 2, and raise your bframe quant ratio high enough to get q=31 bframes, yes, the result would look good. But it has to be an absolutely still scene (low motion scene would still look like crap with this method), and the framesize of those q=31 bframes would also be abnormally large.

am I mistaken here?

I have recently tested ultra-low bitrate encoding in XviD, and I have indeed found that a high quant after a low quant "key"(with that I mean important frame, not necessarily an iframe)-frame does not harm quality. This was on a 50kbps. I also encountered the "ratecontrol bug", but as I tried to work around it I found that it was actually useful in low bitrate encodes, as the overall percieved quality is higher when a P-frame chips in with a low quant when there hasn't been any motion for some time. Trying to reproduce that instantly (at keyframe instead of after some time) by capping quants in weird ways, only resulted in a more constant, but lower percieved quality than what the bug was doing.

I always limit my quant range to 2,3, or 3,4 , or 4,5...., after knowing how compressible the source is, in order to get a nearly constant quality result, and prevent the codec from doing potentially stupid things.

Well, setting a low quant limit isn't really going to give you much more quality. In fact, you could probably better do a constant quality encode using average target quant, than doing a real second pass if you want constant quality. You're crippling ratecontrol that much that it can't do much magic anymore anyway.

Maybe I can make a tricky clip to prove that the rc of xvid is not always reliable. I'll try.:)
Yes, we know of the bug. With some clip you'll find it. That doesn't mean it's a smart thing to do in all cases.

Originally posted by sungey
i remember syskin said that in one of his encodes, there was a part where the quant jumped too high ... thus producing very bad result ... he then recommended capping the quantizer.

Errhm COUGH! Not really. Here's the quote:
Originally posted by sysKin
Since then, I'm capping quantizers (usually 3-5). I can't recommend the same unless you really know what you're doing.
This is the same as I said above. Capping quantizers should really be considered "advanced" and only done if suboptimal results are being made.

as far as i remember ... most still scenes are not completely still scene. There are tiny noises. Use ffdshow "show motion vector" to see this phenomenon.
High quants shouldn't code very much of that and if you have enough b-frames that should actually reduce noise (something sysKin witnessed when we were first testing B-frames).
Originally posted by Acaila
I once got a clip from someone who proved just that. It was high motion in the beginning, low motion in the middle and pure black at the end. XviD didn't even reach the correct filesize (in 2-pass mode), and the overall quality was crap after encoding. So XviD's RC is far from perfect.

Tell me how long ago "once" is. Ratecontrol might have improved since then.

I too usually cap after knowing what the quant ranges are for the uncapped encode. Call me paranoid, but I see it more like playing safe.
Sure, if you don't like re-encoding, but I'd like to squeeze out that extra bit of quality that uncapped quants deliver, and only when that fails, re-encode with capped quants.

[sungey]

Originally posted by mf
Sure, if you don't like re-encoding, but I'd like to squeeze out that extra bit of quality that uncapped quants deliver, and only when that fails, re-encode with capped quants.

hehehe, *cough* re-encode ... reminds me the old divx3/nandub days ... :P

btw, i believe the bad/good of capping depends on how you cap it. If the capping is too restrictive for the given source, then it could be bad. Even if its bad ... i wonder if it will be visible. (video quality-wise).

[K_R]

Originally posted by mf
High quants shouldn't code very much of that and if you have enough b-frames that should actually reduce noise (something sysKin witnessed when we were first testing B-frames).

Unfortunately b-frames, in motion scenes, look noticably bad - blocky.

[Acaila]

Originally posted by mf
Tell me how long ago "once" is. Ratecontrol might have improved since then.I don't remember the exact time, but it was a few months ago, 2 or 3 or so.

Originally posted by mf
Sure, if you don't like re-encoding, but I'd like to squeeze out that extra bit of quality that uncapped quants deliver, and only when that fails, re-encode with capped quants.In fact I don't mind re-encoding. A movie that needs tweaking is much more interesting than one that succeeds in one go :D. As for getting that extra bit of quality, I don't agree with you that that is what you get with uncapped quants. All you're doing is making some scenes worse to improve others, overall quality doesn't necessarily increase with a greater dynamic range compared to a smaller one. In the end I consider this whole "cap-not cap" issue a matter of preference nothing more, because it can't really be proven which is the better way (if you cap right that is).
But the problem with capping is that you can't know how to cap up front, it must be on a 2nd 2nd-pass. That's why for most people it is probably a bad idea.

[TheXung]

Come now, aren't you people that have complete faith in the 2-pass being a little haughty. I mean take a look at the science that it is operating on.

1. Noise in high motion scenes are less noticeable than low motion scenes.
2. Approximate high motion scenes by frame size.

Ok, the first assumption is probably supported by all the years of use of nandub, divx, etc. But what about the second assumption? A bright scene and dark scene of the same amount of motion will have p-frames of drastically different sizes. The first statement can only be correctly carried out with an accurate number for the amount of motion.

If the curve compression is so faultless, then why does it need conditionals in it to prevent it from changing too drastically?

[yingx2]

Originally posted by Acaila
All you're doing is making some scenes worse to improve others, overall quality doesn't necessarily increase with a greater dynamic range compared to a smaller one.

It's a fact so I don't have to agree with you.:D

@mf
Let's simplify things a little bit.

1. Contsant quality is a good thing.

2. The goal of 2-pass mode or any other multilpass system is to reach the target filesize at a constant quality level. (It's even written in xvid's manual). A real constant quality encode has to be done with a fixed quantizer.

3. A multipass encoding process is all about finding a constant quanitzer which, if used on every frame, will reach your desired filesize. In theory, the more pass you do, the less quants will be used. If you find that your movie is encoded at the same quantizer, you will not need another pass.

4. variable quatizer mode = variable quality mode

OK, I have some questions for you:

capping quants in weird ways, only resulted in a more constant, but lower percieved quality than what the bug was doing.
Capping quants in weird ways? How can I do that? Sounds too fancy to me.

Imagine this:
After doing a comp test, I cap quants to 4-5 because it's 48%.
scenario 1 : I get a 700MB file. So, it's a nearly constant quality encode. I cannot ask more.
scenario 2: The comp test didn't work well. I get an undersized (or oversized) file, so I recap the quants to 3-4 (or 5-6), and run my "3rd pass" --> back to senerio 1

Nothing else will happen if I cap quants like that.
Capping quantizers should really be considered "advanced" and only done if suboptimal results are being made.
What is this "suboptimal result" for you?
A constant quality encode is the optimal result that can be produced by a multipass encoding method.
You're crippling ratecontrol that much that it can't do much magic anymore anyway.
What magic? What do you expect the codec to do? Varying quantizers to get "optimal" result? That's not how it works. And I'm crppling rate control by doing a nearly constant quality encode? How? Why?

If you see a q=5 pframe coming after a q=3 pframe, what would you think?
1.)xvid is doing its magic so it's varying quantizers aggressively according to its analysis or, 2.) the rate control is problematic.

which one?

You should always see a q=4 or q=2 pframe (or q=3 pframe of course) coming after a q=3 pframe anyway.

The rate control system of XviD is not perfect, but it works well in most cases. The only problem is that you'll never know when it will mess things up.

[Acaila]

Originally posted by yingx2
Imagine this:
After doing a comp test, I cap quants to 4-5 because it's 48%.
scenario 1 : I get a 700MB file. So, it's a nearly constant quality encode. I cannot ask more.
scenario 2: The comp test didn't work well. I get an undersized (or oversized) file, so I recap the quants to 3-4 (or 5-6), and run my "3rd pass" --> back to senerio 1I very much disagree with that method.

You should never cap quants until you know what quantizer amounts the uncapped encode uses, so never do it right from the compressibility test result. Also if you only judge a bad result on under-/oversize then a lot can go wrong between your scenario 1 and 2 and you wouldn't even notice it. When you cut away too many higher quants the average will go up, making the video worse instead of better, but it needn't necessarily result in oversizing. So you will not even realize it has worse quality if you only look out for under-/oversizing.

The thing you need to do to do it right is keep the average quant the same (+/- 0.1) with the capped encode compared to the uncapped encode, that way you can't go wrong.

[yingx2]

Originally posted by Acaila
I very much disagree with that method.
When you cut away too many higher quants the average will go up, making the video worse instead of better.


I'm cutting BOTH higher quants and lower quants, so that there are only two quants used. The avg quant should be very close to that of the uncapped encode.

What can go wrong with a constant quality encode?
And why shouldn't I remove these higher quants and lower quants when 2-pass mode is all about trying to produce a canstant quality encode?

[Acaila]

The avg quant should be very close to that of the uncapped encode.How can you know the average quant if you start capping right after the 1st pass? Compressibility test result can only give you an estimate of what the average quant is going to be. Or do you have a formula to calculate it exactly?

And why shouldn't I remove these higher quants and lower quants when 2-pass mode is all about trying to produce a canstant quality encode?I meant being careful about not cutting off too many high quant frames.

As an example suppose you have an uncapped distribution of something like:
Quant 2 : 33901
Quant 3 : 82156
Quant 4 : 18141
Quant 5 : 847
Quant 6 : 137
Quant 7 : 5
Quant 8 : 7

Average: 2.899

In your case I guess you would set the I and P range to 2-3 because those are the values that directly surround the average. However in that case you'll cut off almost 20.000 frames that now have to be redistributed to quants 2 and 3. Knowing the codec it will put all of those into quant 3 and subtract some extra ones from quant 2 and also put those into 3. You will now end up with an average of roughly 2.98 or so. That's quite a substantial increase in my opinion (on second thought, maybe the deviation of +/- 0.1 that I mentioned earlier is even too much, +/- 0.05 would be better).

A much better cap range would be 2-4 because then you'll still cut off the highest quants, but the codec only needs to redistribute 1000 frames so the average stays the same.

Maybe this is an extreme example, but I hope it clarifies what I said earlier.

[yingx2]

@Acaila

how about this:
All you're doing is making some scenes worse to improve others, overall quality doesn't necessarily increase with a greater dynamic range compared to a smaller one.
:D

You want to keep those q4 frames in exchange for more "beautiful" q2 frames, and I want more q3 frames and remove all those "ugly" q4 frames.

Let's call it even?

In this case, do you think that reaching a "slightly"(you would call it substantially) lower avg quant at the expense of preserving some(actually lots of) lower quality frames makes more sense?

[yingx2]

Originally posted by Acaila
However in that case you'll cut off almost 20.000 frames that now have to be redistributed to quants 2 and 3. Knowing the codec it will put all of those into quant 3 and subtract some extra ones from quant 2 and also put those into 3. You will now end up with an average of roughly 2.98 or so. That's quite a substantial increase in my opinion.
And how can you be so sure about the 2-3 quant combination elevating the avg quant value? can't happen. You would have to know the eventual quant redistribution to say that.

1. more quant threes and less quant twos will raise the average a little bit.
2. eliminating quant fours and other high quants will make the average go down.

I would say it's 2.55 after capping:D
Let's cap them 2-3.

[Acaila]

And how can you be so sure about the 2-3 quant combination elevating the avg quant value? can't happen. You would have to know the eventual quant redistribution to say that.For the example I gave earlier I happen to have the capped distribution as well (capped to 2-4):
Quant 2 : 34240
Quant 3 : 78655
Quant 4 : 22331

Average: 2.912

Capping to 2-3 will increase the effect on the average even more.

For the more general case I can't prove it with numbers since I don't actually keep a record of every temporary encode just the final result. But I'm telling you from experience, cutting off high quants increases the average. Cutting off a lot of frames increases the average by a lot (=bad), cutting off a few frames increases it by a little (=acceptable), but it will increase no matter what.


For those who are interested, there's a recurring pattern to the redistribution as well. Suppose uncapped the codec uses (no need for numbers):
quant 2
quant 3
quant 4
quant 5
quant 6
quant 7
quant 8

Capping to 2-5 will have the following effect on the remaining quants:
quant 2 -
quant 3 +
quant 4 --
quant 5 ++

[yingx2]

Originally posted by Acaila

For the more general case I can't prove it with numbers since I don't actually keep a record of every temporary encode just the final result. But I'm telling you from experience, cutting off high quants increases the average.

Your experience is very much appreciated, but still, it depends on how these quants were distributed originally which you need to know to have a reasonble guess about the final redistribution. Cutting off many high quants can increase OR decrease the average.
Imagine this:

Quantizer/frames/Total Framesize
Quant 2 : 34240 = 330MB
Quant 3 : 78655 = 340MB
Quant 4 : 22331 = 30MB

It's also an extreme example as you can see that q4s and q3s happened to go to those relatively small frames.
If you cap quants 2-3, it will take very few from quant 2 to compensate the whole redistrbution, like this:

Quant 2: 29003
Quant 3: 106203
avg = 2.78

This example may not do justice, but your "avg = 2.98 after capping" doesn't do much, either. Removing high quants will raise the avg quant significantly only when they were previously distributed to the larger frames of the source.(Did you activate the "payback with bias" ?)

Anyway, I still like the "if you gain some here, you lose some there" theory. You can always boost the quality of 40 small frames by compressing 20 large frames strongly, and end up with a smaller avg quant value as long as it makes sense to you.

Though I'm convinced that the 2-3 enocde is better than the 2-4 one, I doubt if I can tell any difference when comparing them side by side, frame by frame. I bet you can, that's why you said "I very much disagree with that method." ;)

[Acaila]

Removing high quants will raise the avg quant significantly only when they were previously distributed to the larger frames of the source.Which they always are, otherwise the codec wouldn't have given them a higher quantizer in the first place...

it depends on how these quants were distributed originally which you need to know to have a reasonble guess about the final redistribution. Cutting off many high quants can increase OR decrease the average.I always run an uncapped encode followed by a capped encode. And for me capping increases the average every single time. Maybe I just have bad luck and pick the wrong videos to encode? ;)

[TheXung]

Originally posted by Acaila
And for me capping increases the average every single time. Maybe I just have bad luck and pick the wrong videos to encode? ;)

That has been my experience as well.