thoughts on 2-pass

[Teegedeck]

As you perhaps have realized, I'm a bit hesitant to embrace Foxer's curve-scaling-algo and the I-Frame restrictions whole-heartedly, although they produce good results. One part of it is the thought that the codec shouldn't need that much manual tweaking. The possbiliby to tweak should IMO only be used to find good settings that get hardcoded lateron. As it is, it works fine, but it's bound to confuse your everyday-user. We should keep in mind that there's an inherent paradoxon in what we're trying to reach when using both features:

1) The idea behind Curve-Compression is that you take away bits from big frames and give them to small frames because it doesn't hurt big frames much but helps small frames a lot.
2) The idea behind restricting I-Frames/having big I-Frames is that it looks better if you calculate P-Frames based upon high-quality I-Frames.

BUT I-Frames are just the kind of big frames that Curve-Compression tries to take bits away from... The Curve-Compression algo doesn't know what the quantizer restriction is going to do and the quantizer restriction is blind to the aims of Curve-Scaling. Consequently, the bit-distribution gets more erratic the higher you set Curve-Compression and the lower you set max quantizers for I-Frames.

This means, IMHO, that restricted quantizers for I-Frames should ATM be regarded only as an insurance against failings of the Curve-Compression algo, not as an integral part of 2nd-pass-bitrate-distribution. Maximum values like 6-8 sound senseful to me.

And we should look for Curve-Compression values that raise medium-small frames by around 1 quantizer if compared to 0%/1% Curve-Compression because much lower quantizers for small frames than for big frames don't make much sense. Is 25% already based on thoughts like this?


Finally, could the following be a good alternative to the current system?
As the main problem right now is that the curve-compression algo and the quantizer-restrictions ignore each other, it sounds good to me to have a second pass that FORCES quantizers and I-Frames in the second pass. This would make it possible to take restricted I-Frames into consideration when calculating the curve. And users could be warned if harsh restrictions on I-Frame quants would trigger dangerously high quantizers on P-Frames (though I still lean toward hardcoded restrictions). Also, I-Frames could be decided upon more accurately this way and consecutive I-Frames prevented. We could borrow that algo form Gordian Knot. And yes, the whole idea stems from Gordian Knot... ;)

[Franko30]

Interesting thought!

I hope some people that know the details of all this (and that's not me) start a lively discussion on this.

One thing to add from my side, though:

I would prefer some basic settings/recommendations in the interface (rather more action, rather more storytelling, war movie etc.) to hardcoded settings. That way your thought of "not confusing the users" could be incorporated, but still leaving room for people to fine tweak, discuss and feel better than the rest :D

It just doesn't sound plausible to me that one hardcoded scheme could be just right for every movie type - or could it? If yes, then: why?

Cheers

Frank

[MaTTeR]

Just from my limited testing, the code does produce very impressive 2ps results.

Would this code have to be hardcoded or put into the core though? My thinking is... what if you just had an extra tab in the codec for the manual tweaks? I know you might be hesitant to clutter the codec window up further but perhaps the tab could be toggled off/on via the INI installer file. Using this method would still ensure that the everyday user wouldn't be confused. I'm not a coder in the least bit so my apologies if I'm way off base :) I'm just think aloud but would this be possible?

[Koepi]

I thought about a checkbox called "[ ] Extreme advanced options". If unchecked, you just get a Maintab without much you can do, just bitrate, quantizer, all depending on which encoding mode you choose, like it is now.
Only when checking that box the advanced, debug and 2pass tab show up or something like this - isn't really hard to code and would save us from some trouble :)
But therefore we need to implement some good default values if the user shouldn't see those (our values now are good though).

Just my oppinion.

Regards,
Koepi

[lar1r]

Hardcoded options are important to minimize the complexity and expand the user base.

Right now, I would continue testing the available options to find good values to hardcode, but still be able to change them via ONE simple advanced tab.

LAME handles this well, with quality presets that access well tuned parameters.

For now - just keep tweaking those parameters and we'll continuing testing. Once there is a good consensus of what works best then maybe add some AI to adjust parameters for dumb users (ie: at certain bitrates do this etc), and advanced options for those willing to dive into the codec.

Just my 2 cents!

[yokem55]

I agree, some sort of "simple" mode ought be implemented. As Xvid is developing, it is attaining an increasingly complex interface that is approaching nandub sbc levels of complexity. Granted this complexity is very useful for those of us that know how to use it, but it isn't going to be for everyone, and new users may feel very put off if they are unable to dive right in with just tweaking simple settings and getting decent quality. Then, once they are comfortable with using the simple settings, they may get ambitious and see that 3hr DVD they've been wanting to get onto a single cd, and then they will want to break out the advanced functionalities.....

[Foxer]

@Teegedeck

lol

I already said why I accidentally used the wrong formula when I first implemented asymmetric curve compression and the problems with the builds with symmetric curve compression were due to -h overlooking a few small details of the code I sent him but hey.. He was in a rush to get out the door and off to work :)

As for the quality with (a)symmetric curve compression. If it doesn't look good using one setting, use another or encode with high and low curve compression disabled (entering zero into the entry boxes is allowed now) :) Curve compression doesn't affect keyframes and of course quantizer locking is applied to both i-frames/p-frames after the frame is scaled/compressed.

Increase "Min keyframe inverval" under the advanced tab. Note. This variable is currently used so that at least the said number of p-frames are put inbetween keyframes.

[Foo]

@ Teegedeck

This is a valid point.

<puts on flame proof gear>

One of the most interesting aspects of DivX4 IMO is the quality of the results, for a myriad of different sources, with absolutely default settings. For one disc settings with Xvid (unfiltered), currently a sensible "default" seems to be h263 quant with some reasonable level of both inter/intra frame lock, and quant smoothing. Reasonable meaning giving the codec enough freedom that it doesn't have to crank up the quants when it doesn't want to in order to maintain file size, but restrictive enough to prevent any funny business (currently I see this mostly on intros with large areas of uniform color).

cheers,

foo

PS Please no lame ass "DivX4 is the buggiest codec, me and my friends get blue screens" replies.

[Koepi]

But why not?

"DivX4 is the buggiest codec, me and my friends get blue screens"


:P

Regards,
Koepi :)

[Teegedeck]

Originally posted by Foxer
@Teegedeck

lol

As for the quality with (a)symmetric curve compression. If it doesn't look good using one setting, use another or encode with high and low curve compression disabled (entering zero into the entry boxes is allowed now) :) Curve compression doesn't affect keyframes and of course quantizer locking is applied to both i-frames/p-frames after the frame is scaled/compressed.

Increase "Min keyframe inverval" under the advanced tab. Note. This variable is currently used so that at least the said number of p-frames are put inbetween keyframes.

rotfl

I know how to use these things. ;) I've been around for quite a while... (since the first available Nandub builds) and BTW was the first heretic who dared to say that using Nandub's DRF-table might not be the ultimate way to use this piece of software. It's only that I decided to swich to other tools after having used Maras'/RH's assymetric CC-feature for a long time.

Congrats on locking I-frames! This renders my main criticism invalid.

As I wrote in my post my point is, having one feature of the codec (min/max I-Frame quantizers) countering another feature (CC-algo) just isn't a perfect solution, so we should not stop here. These things should relate to each other. For the time being, your algo is the way to go. I use it with very low compression percentages and very loose quantizer restrictions.

[Foxer]

At the moment, curve compression only takes into account & operates on p-frames and only those p-frames which are outside of credits and leaves keyframes scaled normally.

I am uncertain of what you mean by having CC and quantizer locks 'relate to each other'

The quantizer locks counter CC and target bitrate just as the DRF ranges in nandub would counter it's CC and target bitrate and one can't really adjust the pre-locked quantizer choice range without changing the bitrate and/or CC of the encode.

[Teegedeck]

Originally posted by Foxer
and one can't really adjust the pre-locked quantizer choice range without changing the bitrate and/or CC of the encode.

That's want I meant with stuff 'not interrelating' with each other. Wrong words, I guess.

Please forgive me if I'm a bit uncomprehendable sometimes; I still have problems not being a native English speaker. What I meant was that the weight of keyframes should be part of curve-scaling, but that they should be treated different from p-frames (I misunderstood what you meant by 'locking keyframes' there). If keyframes are treated more benignly than p-frames, we can stop guessing on absolute limits for keyframe-quants. Only perhaps there should be a hardcoded limit for max-keyframe quant, but I think it could be dropped completely. If the keyframes are incorporated into the calculation also 'gauge' would become less relevant. To do all this, the codec would have to rely on the 2-pass algo for its inter-/intra-decision and the choice of quantizers. The ultimate goal would be to make the treatment of keyframes and p-frames dependend on the bitrate, but we need to find good values first, as we all agree. I don't know if there are logical mistakes in my thought?