golden frames (On2 VP8)

[lovelove]

VP7 still had the stupid "golden frame" thing though.

golden frames ... As for the usefulness of that, I don't know. It must help in some contrive cases

Why is it stupid, Dark Shikari?
And Manao, here is a list of 4 scenarios, which are hardly "contrived". On the contrary, golden frames seem to be a pretty good idea if you ask me:

http://www.on2.com/index.php?602

[Dark Shikari]

Just because On2 says something doesn't make it true. They listed 4 applications--none of which are even particularly relevant to "golden frames". I mean, I can say that my 500 overclocked hamster wheels help improve my computer's performance, but that doesn't mean they actually do.

The primary reasons they are simply useless:

1. The previous few frames are a better reference frame than the last keyframe, by far.

2. In videoconferencing situations, which inherently have a very strict latency requirement, you can't afford to have keyframes to begin with. Thus, the entire concept of golden frames isn't even applicable.

Furthermore, in H.264, where you have a flexible referencing system, you can implement golden frames via long-term references. This is far better than a system where you're forced to use them no matter what, even if they're totally useless.

I certainly won't say that the idea of storing the last keyframe in the reference is completely useless, having not tried it in x264; maybe by inserting it at the end of the reference frame list you could get 0.1-1% extra compression. But it's probably not worth it unless you're already using at least a half dozen normal refs.

Explicitly:

Foregrounds Matter Most
The first use we found was segmentation of foreground and background video. For example, in most video conferencing applications there is a person talking in front of a static background. The speaker occludes the background but, as the speaker shifts in his seat, he reveals new parts of the background. By updating golden frames with only non-moving high quality blocks, TrueMotion codecs maintain high-quality background images despite fast-moving foregrounds.Regular multi-ref handles this far better.Packet-Loss Recovery
We also use golden frames in situations where packets are lost. In typical video conferencing systems, when the receiver notices packet loss it signals the sender. When the sender receives this signal, it recovers by encoding a frame that has no dependencies on prior frames (in other words, a keyframe). Since this frame must be encoded from scratch, it´s usually a very big frame, which causes choppy video playback. Alternately, it´s a poor-quality frame that looks blurrier than surrounding frames, and causes a distracting visual pulse.

On2 VP8 video conferencing systems have a more palatable option: the sender can send a frame that references only the golden frame. We call this a recovery frame. Since we aren´t encoding a frame from scratch, the result is better quality and a smaller frame than is otherwise possible.Periodic keyframes cannot be used in low-latency video.

Furthermore, I hereby welcome On2 to the mid-90s. This problem was solved years ago: either use intra refresh (in non-interactive systems) or have the encoder respond by referencing an older frame in the reference buffer or selective intra blocks (in interactive systems).Video Conferencing
Multi-party video conferencing also exploits the golden frame. In multi-party conferencing systems, users have disparate connection bandwidth. The typical solution is to throttle bandwidth to a presumed least common denominator: all users receive data at a rate that the slowest connection can receive.

On2 VP8-based systems have an alternative. Through use of the golden frame, normal frames, and droppable frames, VP8 achieves four levels of limited temporal scalability. This means we can produce a single bitstream that degrades as needed for each party. High-def parties pay no penalty for the slower connections in the conference. Best of all, there are no additional CPU costs to this approach.Periodic keyframes cannot be used in low-latency video.

Furthermore, this is an argument for pyramidal B-frame structures, not for golden frames. The topic of "golden frames" seems shoved into here, as it's utterly unrelated.Golden Quality
On2 VP8 also uses the golden frame to improve quality. In very slow moving pans or zooms, a periodic high-quality golden frame preserves image quality by restoring detail lost to repeated application of a loop filter or sub-pixel motion. The results can be quite stark.They really think it's a good idea to make the image "flash" back to full quality every few dozen frames? Are they insane?

Furthermore, perhaps the reason they have to restore detail after repeated applications of the loop filter is because their loop filter is way too strong and blurs the image completely. Ever thought of that? Oh wait, they haven't, because they're still optimizing for PSNR in the year 2010.In other situations the golden frame serves as a good predictor for when something on screen returns to a position held earlierRegular multi-ref handles this far better.