Which options are disabled on first-pass?

[PiXuS]

If "Full quality first pass" is unchecked?

I ask because I need to use the "Zones" option for the first-pass but I read that in zones, no options are automatically disabled -- i.e., we have to disable them manually.

I DID search the XviD forum, but I didn't find an exact answer.

Is there something I could grep in the code which would tell me?

Thanks !

Edit: I saw this (http://forum.doom9.org/showthread.php?s=&postid=563912#post563912) recent post from Manao.

He mentions

VHQ
4MV (?)
Trellis
Chroma Motion

Is that it? What about AQ, QPel, GMC, Motion Search Precision (can be lowered to 0 or 1 on 1-st pass?)

[lordadmira]

The Full Quality First Pass setting doesn't have anything to do with zones. It only has to do with the First Pass data being saved. U can just ignore it.


LA

[sysKin]

Originally posted by PiXuS
If "Full quality first pass" is unchecked?

I ask because I need to use the "Zones" option for the first-pass but I read that in zones, no options are automatically disabled -- i.e., we have to disable them manually.
Not quite right. In *quant* zones (not weight zones) these options are indeed not disabled. However, there is a good reason for this and therefore you can't disable them manually, or everything will go wrong.

The reason is: quant zone in 1st pass is expected to take exactly the same amount of space (kilobytes) as the same zone in second pass. This is why stuff like VHQ cannot be disabled in quant zone.


Is there something I could grep in the code which would tell me?

Well yes: look here (http://www.xvid.org/cvs/chora/co.php/xvidcore/src/plugins/plugin_2pass1.c?login=2&r=1.3) and look at the flags that are disabled in rc_2pass1_before() function.
I think I can summzrize it as: motion precision 1/2/3 (they're the same heh), no chroma motion, no vhq, no brute-force intra prediciton decision (can't disable it with GUI), no trellis. Also no qpel or gmc. "Turbo" becomes active.

Radek

[loni_blues]

sysKin,

What's "motion precision 1/2/3"? Is it motion search precision?
I thought "Turbo" was related to first pass, but found some posts that state it is second pass related. Does it do a faster motion estimation for b-frames and qpel in first pass?
Finally, can we mimic fast first pass if we disable the options you mentioned (and enable Turbo) for the first pass? Is it safe?

Regards,
loni_blues

[sysKin]

Originally posted by loni_blues
What's "motion precision 1/2/3"? Is it motion search precision?Yes, motion search precision.
I thought "Turbo" was related to first pass, but found some posts that state it is second pass related. Does it do a faster motion estimation for b-frames and qpel in first pass?
Turbo is already active in first pass by default (as part of general speedup), so activating it again won't make 1st pass any faster. It is a second-pass setting like VHQ (only working the oposite way).
Finally, can we mimic fast first pass if we disable the options you mentioned (and enable Turbo) for the first pass? Is it safe?
Why would you "mimic" fast 1st pass if it's there aleady? Do you mean activating "full quality 1st pass" and then disabling qpel, vhq etc?

I see no point.

[Surco]

Hi

I don’t know loni_blues's reasons, but I for sure replicate this fast first pass thing when I encode with high bitrate matrix or at extreme low bitrates. I encode the first pass at the nearest possible average quant of the final video, this gives a more consistent video quality.

Bye

[loni_blues]

sysKin,

Thanks for your reply and sorry for not explaining properly. The point is that I want to make a first pass at quant 3 since I use Dideé's 6of9 matrix. And if I set q3 in zones in the first pass, fast first pass is automatically disabled. Since my CPU is slow and the movie is very long, I would like this first pass at q3 to "mimic" the quality and speed of the default fast first pass.
Kind regards,
loni_blues

[PiXuS]

@loni_blues

This is exactly what I am trying to figure out! Thanks for asking more precisely than I did. ;)

[sysKin]

Originally posted by loni_blues
Thanks for your reply and sorry for not explaining properly. The point is that I want to make a first pass at quant 3 since I use Dideé's 6of9 matrix. And if I set q3 in zones in the first pass, fast first pass is automatically disabled. Since my CPU is slow and the movie is very long, I would like this first pass at q3 to "mimic" the quality and speed of the default fast first pass.
Hah, indeed. So yes, the answer is yes, you can do that.

Does the trick with higher quant 1st pass work? I never tried to decipher 2pass2 to figure out if it handles it or not ;) but after my exams are over, I have plans to make it somehow standard :)

Radek

[Audionut]

For my eye's, the quality of a final encoded movie is higher with quant 2 for first pass, rather than quant 3.

Quantizer distribution is greater with first pass at quant 2.

[Didée]

Originally posted by Audionut
For my eye's, the quality of a final encoded movie is higher with quant 2 for first pass, rather than quant 3.
You are seeing things. If I'd change the names of your test files, most probably you'd come to the same conclusion.


Quantizer distribution is greater with first pass at quant 2.
Ah, really :D

(A wider span of quant distribution is a *bad* sign. The method resulting in a narrower range of quant distribution is the *better* one.)

[Audionut]

Originally posted by Didée
(A wider span of quant distribution is a *bad* sign. The method resulting in a narrower range of quant distribution is the *better* one.)

Really.

If a frame can be compressed at quant 6 for a given quality, why on earth would you want to compress it at quant 4.

Just so you have less quant distribution:confused:

With Koepi's new beta build, you will most likely know that there is a new option to choose different curve scaling.
From my observations of the new curve scaling, it reduces quant distribution.

Koepi likes it.

However,
Originally posted by Koepi
GomGom strongly votes against strict scaling, in his tests it always gives lower PSNR. (Which can be right, since the fluctuation of quality gets minimized we don't have near-black quant2 frames in an average quant5-encode anymore which raises PSNR quite a bit. It's all a matter of taste, and I think it should be possible to check the difference with a wider user base.). And remember, GomGom's mostly always right.

It's all a matter of taste

Originally posted by sysKin
In theory, P/B decision alghorithm is "tuned" to work in quant 2 (ie with high-quality frames).

[lordadmira]

Originally posted by loni_blues
Thanks for your reply and sorry for not explaining properly. The point is that I want to make a first pass at quant 3 since I use Dideé's 6of9 matrix. And if I set q3 in zones in the first pass, fast first pass is automatically disabled. Since my CPU is slow and the movie is very long, I would like this first pass at q3 to "mimic" the quality and speed of the default fast first pass.
Um, I don't know why you would want to do this, since ur doing 2-pass. What possible benefit could there be to the final outcome by doing the first pass at non quant 2?

[PiXuS]

Originally posted by lordadmira
Um, I don't know why you would want to do this, since ur doing 2-pass. What possible benefit could there be to the final outcome by doing the first pass at non quant 2?

Please, go read this thread (http://forum.doom9.org/showthread.php?s=&threadid=83061).

[Prettz]

Originally posted by sysKin

Well yes: look here (http://www.xvid.org/cvs/chora/co.php/xvidcore/src/plugins/plugin_2pass1.c?login=2&r=1.3) and look at the flags that are disabled in rc_2pass1_before() function.
I think I can summzrize it as: motion precision 1/2/3 (they're the same heh), no chroma motion, no vhq, no brute-force intra prediciton decision (can't disable it with GUI), no trellis. Also no qpel or gmc. "Turbo" becomes active.

Radek
Can I ask why you disable chroma motion and QPel? Those seem like they would be quite integral to how every picture in a video compresses. Can you really guarantee that there's no detectable quality loss or wasted/misallocated space from not using those in the first pass?

[sysKin]

Originally posted by Prettz
Can I ask why you disable chroma motion and QPel? Those seem like they would be quite integral to how every picture in a video compresses. Can you really guarantee that there's no detectable quality loss or wasted/misallocated space from not using those in the first pass?
Well, the only thing 1st pass does is: it collects information about frame sizes. As long as frame sizes are kept reasonably well, second pass looks the same, +- some random variation. That variation can mean lower quality just as well as higher quality.

Since qpel pictures are often larger than halfpel, it actually fixes the problem of lower motion precision, which works the oposite way. So in the end, it's possible that final frame sizes are closer to "full quality"...

Whatever happens by disabling them, it's worth it speed-wise.

[PiXuS]

Originally posted by sysKin
Well, the only thing 1st pass does is: it collects information about frame sizes. As long as frame sizes are kept reasonably well, second pass looks the same, +- some random variation. That variation can mean lower quality just as well as higher quality.

Since qpel pictures are often larger than halfpel, it actually fixes the problem of lower motion precision, which works the oposite way. So in the end, it's possible that final frame sizes are closer to "full quality"...

Whatever happens by disabling them, it's worth it speed-wise.

But, I think Prettz question was really pertinent (you didn't say otherwise :) ). In the function you pointed to me, there is this comment:


/* Finnaly VOL flags
*
* NB: Qpel cannot be disable because this option really changes
* too much the texture data compressibility, and thus the
* second pass gets confused by too much impredictability
* of frame sizes, and actually hurts quality */
#ifdef FAST1PASS_QPEL_TOO
/* or maybe we can disable it after all? */
data->vol_flags &= ~XVID_VOL_QUARTERPEL;
#endif


Which leads one to think that disabling QPel on the first pass might not necessarily be "optimal". BTW, is FAST1PASS_QPEL_TOO defined on Koepi's builds? Koepi? ;) I understand that disabling them is an "acceptable" compromise.

BTW, sysKin, thanks for your constant aid in answering our questions. It is really appreciated!

[Didée]

Originally posted by Audionut
Really.

... unrelated stuff ...

... unrelated quotes ...

Yes, really.

You can't escape into burbling about different behaviour of different curve scaling strategies. This has nothing to do with the initial question - you're comparing apples to oranges and bananas.

The talk was about that, using a high-BR matrix set, 1st-pass @ q2 results in quantizers in 2nd-pass spread wider than 1st-pass @ q3 (which is correct). You even seconded this, but then drew the conclusion that the former delivers better quality (which is wrong, IMO).

The basic goal of curve scaling is to achieve an encoding with just that average quantizer that will lead to the specified destination size, while minimizing unnecessary quant deviations. And you yourself gave the answer that was to be expected.


I'll try it once more, like this:

1. Curve scaling tries to minimize quant fluctuations

2. q2 produces a wider spreading of quantizers

3. q3 produces a more narrow spread

4. Conclusion is obvious.


But of course it's no "general rule" that 1st-pass should be done @ q3. This is true only if 2nd-pass is going to achieve an avg. quantization > 3 (depending on bitrate and matrix). For lower quantization, q2 should be preferred.

[lordadmira]

So what ur saying is that optimally, for a 2nd pass average quantizer of n, the 1st pass quantizer should be int(n), or is it n - 1. Or even int(n) - 1.

LA

[Didée]

I vote for int(n), yes.

Some weeks ago, I already explained my opinion in another thread:
XviD is guessing 2nd-pass framesizes from 1st-pass one's. The bigger the actual difference is, the more the guessing may go off, and the more work overflow control will have to do. Guessing the framesizes in a (q3+q4) sequence from a q3 base will be more accurate then guessing from a q2 base. That's all of the story.


Exercise:

1. Take a high-BR matrix (say, eqm-v3ehr or -v3uhr ;) )
2. Encode a sequence (not too short) with q=3.5 or q=4. Note down the resulting filesize.
3. Make two different 2-pass encodings, target size = size of (2.):
3a. 2-pass with 1st-pass @ q2
3b. 2-pass with 1st-pass @ q3
4. compare the resulting quant distributions & target size accuracy.

[Ark]

I did this to see quality differencies using sixofnine, and filesize/bitrate accuracy is much more improved with 1st pass@Q3, only 10-20kb + or - that desired bitrate, against 50-150 with 1st pass@Q2...

[lordadmira]

Ah so the culprit is instability or indeterminism in the good old rate control algorithm (ref rate control anarchy). Maybe we can have an option to manually set the 1st pass quantizer. Why doesn't the 1st pass obey the min quantizer settings? Seems like that could solve it right there.


LA

[PiXuS]

Originally posted by lordadmira
Ah so the culprit is instability or indeterminism in the good old rate control algorithm (ref rate control anarchy). Maybe we can have an option to manually set the 1st pass quantizer. Why doesn't the 1st pass obey the min quantizer settings? Seems like that could solve it right there.


LA

Read 8 posts ago in this very thread. There IS an option (albeit, a semi-hack) to force a quant other than 2 on the 1st pass.

[lordadmira]

If ur talking about the quant zone I already know about that. I'm talking about a *real* "official" option to set the 1st pass quantizer.


LA

[loni_blues]

@sysKin

Wow! I did an encoding with a first pass at q3 and manually disabling the options you listed (with the exception of brute-force intra prediction decision) so as to get a fast first pass:
motion precision 1/2/3 (they're the same heh), no chroma motion, no vhq, no brute-force intra prediciton decision (can't disable it with GUI), no trellis. Also no qpel or gmc. "Turbo" becomes active.
and got a file undersized by 70 MB!
What might be wrong?
Thanks for your help!

[Surco]

If you don't give us some more information nobody can tell you what was going on.

What was your first pass size (at quant 3), what the matrix you used, and the target file size? What's the quant distribution in second pass?

Bye

[loni_blues]

Let's see.
First pass size not available because "discard first pass" was active. I used Didée's 6of9 matrix. Target filesize was 1870 MB. And let me remember: average i-frame quant= 3.77, average p-frame quant= 4.17 and average b-frame quant= 7.02 (more or less).
Regards,
loni_blues