I-frame ranges are being ignored when enableling B-frames

[Acaila]

Below are the results of a test I did with a 1500 frames clip encoded with:
1) I-frame and P-frame ranges at 2-31.
2) I-frame range at 2-2 and P-frame range at 2-31 WITHOUT B-frames enabled.
3) I-frame range at 2-2 and P-frame range at 2-31 WITH B-frames enabled.

Number 1
Intra-Frame (Key-Frame) Quantizers
------------------------------------

Movie
-------
Quant 3 Used : 23 Times, Percentage Used : 100.00%

Average I-Frames Quantizer Used for Movie : 3.000

Inter-Frame (P-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 36 Times, Percentage Used : 2.44%
Quant 3 Used : 1113 Times, Percentage Used : 75.36%
Quant 4 Used : 328 Times, Percentage Used : 22.21%

Average P-Frames Quantizer Used for Movie : 3.198

Number 2
Intra-Frame (Key-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 23 Times, Percentage Used : 100.00%

Average I-Frames Quantizer Used for Movie : 2.000

Inter-Frame (P-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 30 Times, Percentage Used : 2.03%
Quant 3 Used : 1054 Times, Percentage Used : 71.36%
Quant 4 Used : 393 Times, Percentage Used : 26.61%

Average P-Frames Quantizer Used for Movie : 3.246

Number 3
Intra-Frame (Key-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 8 Times, Percentage Used : 47.06%
Quant 3 Used : 8 Times, Percentage Used : 47.06%
Quant 4 Used : 1 Times, Percentage Used : 5.88%

Average I-Frames Quantizer Used for Movie : 2.588

Inter-Frame (P-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 374 Times, Percentage Used : 25.22%
Quant 3 Used : 971 Times, Percentage Used : 65.48%
Quant 4 Used : 126 Times, Percentage Used : 8.50%
Quant 5 Used : 11 Times, Percentage Used : 0.74%
Quant 6 Used : 1 Times, Percentage Used : 0.07%

Average P-Frames Quantizer Used for Movie : 2.850

What you see here is that the I-frame range is not used when B-frames are used in the encode. In fact the I-frame range is equal to the P-frame range that you set but that's hard to show on such a short clip, I know that from encoding a full movie several times. All other optional features were off, so this is definately an effect of using B-frames. This effect also remains with less stringent capping.
I haven't encoded anything for a while, so I don't know if any builds other than the newest suffer from this (I used Koepi's 17-02-2003 build). Also I normally don't use B-frames at all, but for the movie I first noticed this on I had to.

I haven't been able to find any other mention of this effect by anyone, so I assume I'm alone in this observation? Is this part of the "in 2-pass mode B-frames don't work correctly yet" issue?

Comments are appreciated.

[sam_b]

I can confirm this happens for me, but thats about it I'm afraid. Never bothered me too much as xvid seems pretty good at working these things out for itself.

[SleepEXE]

I've definitely experienced the same thing with all builds after Dec 9. It's been mentioned in a few places, and while I can't argue with the pristine quality being produced by recent builds, I sure would like this feature to work again. Beggars can't be choosers and all that. Using B-frame settings of 3/130/100 (typically using ~65% of first pass bitrate), I've encountered stretches as high as twelve thousand frames without a keyframe which understandably makes seeking within the file very difficult.

BTW, the 12,000 frames is logical as a worst-case scenario...assuming the default Max I-frame interval of 300 is kept and using maximum of 3 B-frames -> 300(3 bframes + 1 interframe ) = 1200 frames

Best regards,
SleepEXE

[mfluder]

@SleepEXE

Acaila was talking about a different type of problem, you might want to reread his post for a better understanding. But I agree with you about this problem of not respecting max keyframe interval when b-frames are enabled. It's a known issue and Koepi said it's going to be fixed soon. It is obviously a 2pass code problem (specificly second pass because stats file has all keyframes placed correctly) because when using a fixed quantizer max keyframe interval is respected.

It would be really great if someone have time to take a look at this because I think it's the only large problem left. I'm not talking only about seeking problems when there are no keyframes for a long period but a more serious problems caused by this. For example I compile my own builds and I enable simple idct because I also use qpel which have a problem if using default xvid's idct (smearing problem) and when decoded with latest libavcodec (compiled on Linux). When using simple idct it looks great but if there is a long scene and there is no keyframe for a long time there is this smearing problem again, it's much less visible than when using default idct but it's still there.

One more question for someone from XviD team. Does anyone know if simple idct will be default idct in the future. I ask this because it's in the CVS but it's not enabled by default.

Regards,
mfluder

[Koepi]

I wouldn't saay "that's the only problem left", the cause for it is very "deep" within xvid (very deep in the core).

It's simply impossible now to correctly control xvid's 2nd pass, the dynamic I/P/B-mode-decision is always active. Thus a keyframe from the first pass doesn't end up as a kayframe in the 2nd pass necessarily as the images have other differences after higher quantization - resulting in i.e. less intra-coded macroblocks and thus not triggering a intraframe.

I wrote that I _hope_ for a fix soon as this problem really bothers me (hey, I started helping with xid exactly in the 2pass arena and now that's a bit unusable or at least not as precise as I want it to be ;) )

I hope this helps you understanding the problem better. And thanks for the 100th report of this behaviour %)

Regards
Koepi

[Selur]

@Koepi:
Didn't want to open a new thread, but are there any plans to fix "hinted ME" in the near future ?

Cu Selur

[Koepi]

Selur:
I didn't notice anyone interested in hinted ME in current development. This thread's topic is higher priority, if those bugs are fixed, maybe someone feels like fixing it. It's useless to fix it now as there is no proper 2pass control.

Regards
Koepi

[Selur]

"It's useless to fix it now as there is no proper 2pass control."
Yes, I understand that just wanted to make sure it's not forgotten,..

Cu Selur

[sysKin]

Originally posted by Selur
Didn't want to open a new thread, but are there any plans to fix "hinted ME" in the near future ? I looked at hintedME some time ago. First, second pass needs to be fixed - P/B/I decisions need to be exactly the same in second pass as in the first. Secondly, there are three ways for hintedME:
1. Just copy the vectors from first pass. This _will_ produce not only measurable quality loss, but visible quality loss. The gain in speed might be 20 - 30% or something like that.
2. Use hinted vectors for limited search. I tried that. There is still a quality loss, while speed gain is minimal (5% ? maybe a bit more with bframes, 1% if you plan to use VHQ). You can have better quality and similar speed by reducing motion search precision...
3. Just forget this idea, it's stupid anyway.

As far as I'm concerned 3 is the best option.

Radek

[Acaila]

Koepi wrote:
It's simply impossible now to correctly control xvid's 2nd pass, the dynamic I/P/B-mode-decision is always active. Thus a keyframe from the first pass doesn't end up as a keyframe in the 2nd pass necessarily as the images have other differences after higher quantization - resulting in i.e. less intra-coded macroblocks and thus not triggering an intraframe.
Thank you for that explanation, it's very enlightening.

I hope this helps you understanding the problem better. And thanks for the 100th report of this behaviour %)It sure does. I'm sorry for bringing up a known problem but I was't aware it was related to this, I had only read about problems with P/B decisions and resulting filesize problems. I didn't know I-frames were part of this.

[Selur]

"As far as I'm concerned 3 is the best option."
Okay,.. if you say it's not worth it, I beleave you :)
Thx for the info.

Cu Selur

[pandv]

About the b-frames, and because the Dev-Api-4 is in the works.

I think, a setting to grant a number of "normal" frames between B-frames can be useful.

For example, making a sequence (maximum) as PPBPPBPPBPPBPP....

If the B frames are perceived as worse than P frames, this can help to alleviate the problem.

pandv

[echo]

Hi!

I just wanted to say that the problem with the I/P frames decisions that Acaila is talking about happens also in 1-pass encodings as I posted here: http://forum.doom9.org/showthread.php?s=&threadid=44317
So I don't think it is a 2nd pass problem as koepi is saying. From what I've seen it is also not related to B-frames as this problem is also in the latest stable build by koepi.

Cheers

echo

[Koepi]

Did you use lumi masking?

[wing1]

@echo

??? 1-pass encoding I/B/P not respecting set keyframe interval???
It seems to me that this issue has been resolved since Koepi's 26012003-1.exe build. If anything is wrong is the fact that more keyframes are being added on top of what I'd specified ( multiple keframes that is jumbled up is back ). However, that can be resolved by setting mimimum keyframe interval to 6. It does not stop the 'multiple' keyframes issue but it minimizes the occurrences. I can live with that :D

[echo]

@koepi

No, I haven't used lumi masking for what seems like ages now. And as I already said this problem is also present in your stable build so all the new features (b-frames, GMC etc) have nothing to do with it. It may be very deep in the XviD core as you said.


@wing1

What I -and Acaila- was talking about is not the keyframe intervals but that the min and max quantizers of keyframes are not being respected and that the quantizers set for P-frames are used instead.

Regards

echo

[wing1]

@echo

my fault..my appology.

[Shayne]

Originally posted by Koepi
I wrote that I _hope_ for a fix soon as this problem really bothers me (hey, I started helping with xid exactly in the 2pass arena and now that's a bit unusable or at least not as precise as I want it to be ;) )

I hope this helps you understanding the problem better. And thanks for the 100th report of this behaviour %)

Regards
Koepi

Well is there a work around for the time being? lower i frame max help?

Dont use b frames?

Closure would be nice for the time being because it sounds to me your on a journey to the centre of the earth (the core)

Peace