Smarter redistributing of forced keyframes

[HarryM]

I taken with Koepi's experiences with controlled degradation of consecutive keyframes. Very good idea.

I have next idea...

We know, that XviD insert keyframe:

a) at scene change ("detected keyframes")
b) upon 'max. keyframe interval' ("forced keyframes" - mainly in dark scenes(why?!), end credits, etc.)


Question ad b):
It can be redistribute the forced keyframes smarter than stupidly by 'max. keyframe interval'?
Surely, I dont need forced keyframes accurately by (e.g.) 500 frames..., do you?

I think, that forced keyframes are more, much more effective at frames with max. relative bitrate...

Except 'max. keyframe interval' I suggest using of frame-deviation for 'max. keyframe interval', e.g. 'max. keyframe interval'= 500+-200...

[Koepi]

We need those keyframes at those places for one reason:

seeking.

That's why it's added to XviD.

Regards,
koepi

[HarryM]

All right.
But this view (max KF int=500, e.g.)...

nmbr frame, type
----------------------
...
154, detected KF (scene change)
387, detected KF (scene change)
887, forced KF
1387, forced KF
1389, detected KF (scene change)
...

I think, that KF at frame1387 is 'a little' useless...
In addition, in interval 887+-deviation (1387+-deviation) exist surely better adepts for KF instead of frame887 and frame1387...

[primitive]

Originally posted by HarryM
All right.
But this view (max KF int=500, e.g.)...

nmbr frame, type
----------------------
...
154, detected KF (scene change)
387, detected KF (scene change)
887, forced KF
1387, forced KF
1389, detected KF (scene change)
...

I think, that KF at frame1387 is 'a little' useless...
In addition, in interval 887+-deviation (1387+-deviation) exist surely better adepts for KF instead of frame887 and frame1387...

I think you have a point here. If you're doing a 2-pass encode, surely you have the time to do the analysis for smarter keyframe positioning.

In the above example, if you absolutely had to have two keyframes between 387 and 1389, you could place them optimally for the fastest seeking instead of having one at 1387 and at 1389. Of course, you could also save bitrate by removing the second, mostly-redundant keyframe.

I think a fuzziness in keyframe placement would be appropriate. Perhaps +-10%, which with a max keyframe interval of 300 would yield a fuzziness of about 1 second either way.

-p

[duartix]

Introducing tolerance in keyframe placing introduces complexity.
Based on what would you decide wether or not to place the 1387, forced KF?
Why not rather place a forced KF halfway at frame (1389+887)/2 as primitive suggested?

The later seems more logical. I can't remember if GKnot or some other tool like Nandub tried to do that or even acomplished it.

But now you have to face this: Size predictability can be affected!
If you start placing keyframes that weren't treated as keyframes in the first pass, my friend you will have different file sizes. A KF placed diferently from the first pass can have a very diferent size. Chances are these up and down diferences could dilute along the whole movie, but be prepared.
This kind of implementation would also require the first pass to diferenciate forced KF from scene detected KF.

Anyway it should be a nice feature, but it's up to the developers to decide if it is worth the possible trouble.
Cheers.

[HarryM]

Every P-frame has any size (in bytes), all right?

I think, that the best adept for 'smarter' KF (in interval +-deviation) is the biggest P-frame or P-frame with maximum of newly defined macroblock.

Suggestion:
Secondary, this can help indentify hidden scene changes in dark part (e.g. end credits) of movies, where XviD use only 'forced' KF :(, because XviD cannot identify these scene changes as regular KF (detected frames).


IMHO:
Size predictability can'not be affected, that forced KF are used mainly in dark scenes = these accordant frames has relativelly small size.

[-h]

I would go against this.

In my opinion, a KF-interval of 300 frames is excessive. MPEG-4 is only "supposed" to have a maximum of 132 consecutive inter-blocks before the block *must* be coded as intra, to avoid iDCT artifacts from different decoders. XviD doesn't enforce this (yet), but I know many people are going to start playing XviD/DivX5 content on hardware MPEG-4 players, and wonder why their long scenes turn pink.

The XviD devel list has already seen a couple cases of artifacts caused by iDCT inaccuracies.

-h

[TripleA]

Originally posted by -h
I would go against this.

In my opinion, a KF-interval of 300 frames is excessive. MPEG-4 is only "supposed" to have a maximum of 132 consecutive inter-blocks before the block *must* be coded as intra, to avoid iDCT artifacts from different decoders. XviD doesn't enforce this (yet), but I know many people are going to start playing XviD/DivX5 content on hardware MPEG-4 players, and wonder why their long scenes turn pink.

The XviD devel list has already seen a couple cases of artifacts caused by iDCT inaccuracies.

-h

That's quite disturbing.

But, does DivX 5 enforce this limit?

I ask because most hardware player, IMO, will try to support DX5. So if DX5 enforces not the limit, perhaps we're fine...

[R3g]

Originally posted by duartix
The later seems more logical. I can't remember if GKnot or some other tool like Nandub tried to do that or even acomplished it.

Nandub did it, it was the "sart-keying" feature. I think GKnot introduced a procees similar to it, but by doing three pass to avoid predictability problems : the first pass to know where to place keyframes, the second to make a "normal" first pass, and the third to encode.
Of course, all of this was done with the Nandub ECF files, which have no equivalent with XviD

[primitive]

Originally posted by TripleA


That's quite disturbing.

But, does DivX 5 enforce this limit?

I ask because most hardware player, IMO, will try to support DX5. So if DX5 enforces not the limit, perhaps we're fine...

I think the point is to support the standard rather than to support the codec.

If DX5 turns pink in long scenes, it's not the fault of the hardware that's doing exactly what the spec defines, it's the fault of the codec that flaunted the spec.

Those things being said, I'm going to start setting the max interval to 132 to avoid those potential problems. All those keyframes worry me a bit, though.

-p

[TripleA]

I know that strictly speaking the CODECs should stick to the spec. But if the biggest source of MPEG4 video is DX5, then I'd guess the hardware people will support whatever DX5 supports. Even if it's not in-line with the spec.

Just a guess, of course.

[mgerald21]

IMO the max. of 132 consecutive inter-blocks before the next intra-block is not really a mandatory rule of the MPEG4 guide, but a recommendation.

As -h wrote, the quality of the playback will suffer from iDCT inaccuracies, if the next KF is much away from 132.

I can't imagine that a hardware-player will check such things and refuse to play videos, if two KF are more than 132 frames away.

Well, once again it's a matter of quality :confused:


But back to the original topic of this thread
"Smarter redistributing of forced keyframes" :)

Before the 2nd pass, the internal or the external curve compression function already calculates the position of the KFs, right?
What about something like a plausibility check of the distribution of the KFs.

E.g. necessity of some forced KFs, if a frame change KF follows afterwards. Or if a forced KF is more effective on an other position (limited to e.g +-10 frames), since the motion vector is higher at this frame.

[sysKin]

Before the 2nd pass, the internal or the external curve compression function already calculates the position of the KFs, right?

Nope. The second pass just copies inter/intra frame decision from the first pass. It doesn't even know if the first pass put a keyframe because of scene change or because the limit of P frames was reached.

What about the following idea. It's not perfect, but it should be better than current solution:

Dynamic inter/intra threshold. Currently, threshold is fixed at 50% (if 50% of all macroblocks are intra macroblocks, keyframe is created).
What if we made the threshold 50% for the first 9 seconds (assuming that 10 seconds is the limit) and then lineary decrease it to 25% at the 10th second, and continue to 0% (==forced keyframe no matter what) at the 11th second?.

It would simply mean putting forced keyframes in a bit smarter way, still not perfect but - possibly - a bit better.

We might also start the threshold at, say, 70% and decrease it to 50% during the first second after keyframe. It could help dealing with one keyframe after another.

What do you think? Please comment.

[Koepi]

Looking at my x-thousands matrix encode currently that would result in a very strange keyframe distribution.

I still vote for a proper SCD like the alternative one in nandub to make good decisions on KFs, this would help it much more than anything else we did think of here I guess :)

Best regards,
Koepi

[-h]

IMO the max. of 132 consecutive inter-blocks before the next intra-block is not really a mandatory rule of the MPEG4 guide, but a recommendation.

The original m4if post that mentions it is http://lists.m4if.org/pipermail/technotes/2002-January/000490.html

As is mentioned there about the 132-inter-block rule, "some sequences will catastrophically fail if this is not met".

Of course they will be decoded, they'll just look terrible.

-h

[Nic]

I love reading the MPEG guys posts....I love the ones that start "Dear Experts," :)

@h: You know far too much! your head may well explode with information :)

-Nic