H264 random seek

[kolak]

Question:

I have h264 stream with e.g. 60 frames closed GOP and typical 2B frames structure, 4 ref frames.
When I jump to some random frame in the GOP (lets assume worse cases scenario, so some B frame I assume) how many frames need to be decoded to show current one?

Can we also roughy guesstimate how much more CPU power (cycles) is needed (e.g. 2x, 3x more)?

[sneaker_ger]

It's adaptive. It could be all preceding frames, only the I frame and frame you seek to or anything in-between. If you assume a static order without any adaption and without b-pyramid only the P and I frame(s) will be referenced. So if you seek to e.g. frame #30 you have 30/3 + 1 = 11 frames to decode. (1/3 because with 2 b frames you always have PBBPBBPBB i.e. every third frame is a reference + 1 I frame.)

[kolak]

Thank you.

What about CPU power (cycles).
If you need x amount for I frame, do you need the same for P/B frame? (after all referenced ones are decoded)?

This means that relatively short, closed GOP with fixed structure is much more seeking friendly. It also explains why people in some cases have problems with x264 streams which uses by default long GOPs and all advanced h264 features.

[sneaker_ger]

Closed GOP or open GOP makes no difference. Difference is reference structure. With no b-pyramid seeking should be faster, esp. with more bframes (more bframes = less pframes = less references to decode). GOP length is important. Average seeking times will increase greatly with longer GOPs. But at some point any decent encoder will throw new keyframes, if only for scenechanges.

If you need x amount for I frame, do you need the same for P/B frame? (after all referenced ones are decoded)?
With the same amount of bits an I frame should be decoded faster (let's just say 30%). But it's not so important because we have so few I frames compared to P/B anyways. And we usually put much more bits into I frames.

[kolak]

So B frames are quite an evil :)
It there anything else which can improve seeking (some advanced header info)?

[sneaker_ger]

So B frames are quite an evil :)
No.

It there anything else which can improve seeking (some advanced header info)?
Everything that increases decoding performance also increases seeking speed, naturally.

[kolak]

Ok, thank you.
1 B frame would make a good compromise for seeking v quality.
I assume 1 B frame would be more effective for seeking issue (compensated by higher bitrate) than lower bitrate with 2B frames?

[sneaker_ger]

I didn't say b frames were bad for seeking.

[kolak]

Well, they are because in order to access B frame you need to decode many others. They are the worst out of I,P,B.

[sneaker_ger]

No. If you set your encoder to use max. 4 references that's the limit. Most of the decoding will be the chain of reference and that won't differ between P and B. The difference is negligible.

[kolak]

Ok, so best to keep ref at 1 when seeking is important.

[sneaker_ger]

No. It may depend on the decoder but I would simply not touch number of b frames or number of reference frames or b-pyramid to increase seeking speed. Just make the keyint not too long, e.g. 4 seconds, and be done with it. Depending on your preferences of course.

[kolak]

Well, this is not good enough as some NLEs are not that well optimised for h264 seeking. You need do help them and because quality is not the key point here (these files are proxy), so you have few options- reducing ref frames, GOP size etc.

[sneaker_ger]

Feel free to test different settings (and share your results) but I doubt you will achieve the desired speedups by playing with the b and ref frame settings. Keyint is .. key.

[benwaggoner]

Roughly, with a good decoder, worst case seek latency~keyint/bframes.

Using b-pyramid strict is optimal for seeking, so that the B/b frames between any pairs of P frames won't reference frames outside of that (mild handwaving). So basically you have to decode the IDR frame, all the P/I frames that are in the reference chain for your frame.

Not using B-frame means that it's all P-frames, and so to get to frame 90 all or most of frames 1-89 must be decoded first. But if only every 8th frame is a P-frame, worst case is you'd decode frame 1, 9, 17..97 (12 frames total). Then decode the reference B frames between P frames 89 and 97, and then decode just the non-reference b-frame you want to look at (no other non-reference b-frames need to be decoded, since nothing references those). If you want to see a P or ref B, you just stop when you hit the one you want.

Of course if you allow dynamic b-frame structures, it could get more complicated, but you also get more encoding efficiency. H.264-the-standard allows a lot of weird things to happen, but x264 and real-world content is generally a lot more predictable.

If you have a bad decoder that doesn't know to skip frames it can skip, hopefully you can stop using that decoder :). You just can keep keyint as low as possible.