I_Frame rate in same video seens but different copies.

[Rouhi]

I like to discuss about I_Frame rate in two same video. I like to know in below situation, the I_Frame rate can be different or not:

1-if you made a cam cording video from a video presented on a cinema screen.

2-If make a copy of the original video by save as ...

3-if some recording parameter change such as resolution, brightness....for a same seen

[Rouhi]

Please give your idea in this field....

[Selur]

1 -> can differ from original
2 -> is the same as the original
3 -> can differ from the original
New compression can always lead to a different I-frame placement due to different I-frame placement algorithms and changed parameters for these algorithms.

[LoRd_MuldeR]

New compression can always lead to a different I-frame placement due to different I-frame placement algorithms and changed parameters for these algorithms.

...which can also happen in case (2), depending on how the program does "save" the copy of the video.

If "save as" is done in a "direct stream copy" mode, then the I-Frame placement will not change. But if the video is actually re-encoded, then it may change, of course.

Actually the I-Frame placement might change, even if we re-compress the video with the same encoder that was used to compress the original. That's because even if the same algorithm (and parameters) is used, the first time it was applied to the uncompressed original video and now it is applied to the already compressed one. There is at least a chance that this might result in different decisions...

[Rouhi]

I think it was better to change the title to Encoder criteria for decision type of frames(Slices , I,B,P)

As far as I know the I-Frames should contains the key picture of the video stream. Such concept is base of this fact that the I-Frames are referencing picture for other frames or slices such a P or B. even p frame can be referenced by B frames but the main referenced picture which even P frames reference to, are I-Frames. In another word, this fact shows that the location of I-Frame pictures among an image stream (video) should be selected very carefully, every where among the image stream that we encounter with a big difference in the content of two consecutive images is a point that the second picture can be a good candidate for being an I-Frame.
So it seems that every video segment must start with a special I-Frame (IDR picture) and even in each segment, on every scene cut or every where that a big difference among pictures exists (like very dynamic segments), should start with an I-Frame too.
If we accept the above logic, the encoders are not free in selecting any frame as I-Frames (with except of long image stream with low dynamicity or diversity). There exist some similar locations that even different encoders should encode the images as I-Frames respectively.

I would be grateful to receive your idea about this subject

[kypec]

Yes, your idea is perfectly valid in theory. However, the practice is where the encoders work -> one (encoder A) might have different thresholds than the other (encoder B or even the same encoder A configured differently) hence it can decide that certain frame is a good candidate for I-frame whereas B decides otherwise.

[LoRd_MuldeR]

If we accept the above logic, the encoders are not free in selecting any frame as I-Frames (with except of long image stream with low dynamicity or diversity). There exist some similar locations that even different encoders should encode the images as I-Frames respectively.

Well, first of all the encoder is free to select arbitrary frames as I-Frames. The encoder may put the I-Frames every it likes. It may even make every frame an I-Frame or it may not use any I-Frames at all (except for the very first one). Whether this choice is a good one or a bad one that is a completely different question! It of course makes sense (with respect to compression efficiency) to put I-Frames at scene changes, but this isn't a necessity! After all it isn't trivial to define/find scene changes. Some "simple" encoders use a fixed I-Frame interval and thus will put the I-Frames at regular intervals without even looking at the content of the video. More sophisticated encoders will use adaptive I-Frame intervals and they will try to put the I-Frames at the "optimal" position - but the algorithms/thresholds used to find an "optimal" position (and therefore the decisions) may still differ between different encoders. Moreover there may be restrictions like "minimum key-frame distance" and "maximum key-frame distance" that force the encoder to put an I-Frame at a "suboptimal" position. Last but not least there are things like so-called "flash frames" which should not be made I(DR)-Frames, but many encoders will make them I-Frames (because they are very different from the previous frame and thus may easily be interpreted as a scene change). Other encoders, like x264, have a special "flash frame" detection to avoid these situations...

[Rouhi]

Originally from LoRd_MuldeR :
there are things like so-called "flash frames" which should not be made I(DR)-Frames, but many encoders will make them I-Frames (because they are very different from the previous frame and thus may easily be interpreted as a scene change).

Thanks for your comprehensive answer. It was so useful for me. I agree with you idea partially. Actually encoders can put I-Frame every where in the video stream (in worst case, all frames can be encoded as I-Frames), BUT, there are some places (like first frame of each scene cut) that all encoders should consider the first frame as I-Frame. The reason is that because of huge difference in pixel domain values of two consecutive frames, motion compensation vector used in P-Frame of the second image can not handle the huge difference value. In another word I can say the difference in such places should be more than all thresholds used in different encoders.

BTW do you have any statistics regarding fixed and adaptive I-Frame rate used in general public videos? Or which encoders use which techniques? Or is it among encoding parameter of encoders?

[LoRd_MuldeR]

Thanks for your comprehensive answer. It was so useful for me. I agree with you idea partially. Actually encoders can put I-Frame every where in the video stream (in worst case, all frames can be encoded as I-Frames), BUT, there are some places (like first frame of each scene cut) that all encoders should consider the first frame as I-Frame. The reason is that because of huge difference in pixel domain values of two consecutive frames, motion compensation vector used in P-Frame of the second image can not handle the huge diff

Well, you can always encode the first frame of a scene as a P-Frame (or B-Frame), if you really want to. I see no reason why this shouldn't be possible. AFAIK there isn't such a thing as a "maximum possible diff" for P-Frames. If the frame is very different from previous one(s), then it may not be a good idea to code it as P-Frame (with respect to compression efficiency), because the residual will be huge. But it still is possible! So some encoder might do it (sometimes). It simply cannot be excluded. Also if there was a scene change just a few frames ago, the "minimum key-frame distance" might simply forbid the encoder to put an I-Frame at the next scene change - H.264 encoders can put a "normal" I-Frame (rather than an IDR-Frame) in this situation though.

In another word I can say the difference in such places should be more than all thresholds used in different encoders.

What is the maximum I-Frame threshold of all encoders in existence? Do they even measure "difference" in a comparable way? I don't know ;)

Also: Some encoders use adaptive thresholds. The threshold is lowered over time and reset on scene change (I-Frame). So the longer the last I-Frame (scene change) was ago, the more likely it will be to trigger another I-Frame (scene change). This makes sense because it's better to trigger a scene change at a somewhat suitable position instead of running into the "maximum key-frame distance" limit. Once again this can lead to pretty different decisions between different encoders...

[Shevach]

Some "simple" encoders use a fixed I-Frame interval and thus will put the I-Frames at regular intervals without even looking at the content of the video.

Perhaps some "simple" encoders use fixed I-frame intervals in order to provide random access and other trick modes for decoders.
For example in order to provide the fast forward mode it is desirable to insert I-frames at fixed interval.

[LoRd_MuldeR]

For example in order to provide the fast forward mode it is desirable to insert I-frames at fixed interval.

It doesn't necessarily have to be fixed though. A reasonably short "maximum key-frame interval" should be sufficient.

[Rouhi]

Well, you can always encode the first frame of a scene as a P-Frame (or B-Frame), if you really want to.

I cant understand how a P-Frame can be located in start of a closed GOP or even scene change . because P-Frame always reference to an I-Frame prior to them. If the scene change located to an open GOP, i think there should be a very close I-Frame with very similar image to the start frame of scene change. in that case may be starting P-Frame reference to that I-Frame. Am i right?

BTW is there any standard to consider maximum I-Frame interval? Even in very static scenes such as face of a reporter in News TV program which we have very little changes among consecutive images in a long time?

If we accept that I-Frames are key frames among image stream, so we return to our previous discuss regarding scene changes and the role of I-Frames in them. because key frames should be rich enough in content to be able for offering more information for motion compensation and inter prediction of following (or even previous) P & B frames.

[sneaker_ger]

I cant understand how a P-Frame can be located in start of a closed GOP or even scene change . because P-Frame always reference to an I-Frame prior to them. If the scene change located to an open GOP, i think there should be a very close I-Frame with very similar image to the start frame of scene change. in that case may be starting P-Frame reference to that I-Frame. Am i right?

Lord_Mulder didn't say "start of a GOP", he said "start of a scene". Of course you're correct in that it's a stupid thing to do in practice, Lord_Mulder just wanted to make an example.

BTW is there any standard to consider maximum I-Frame interval? Even in very static scenes such as face of a reporter in News TV program which we have very little changes among consecutive images in a long time?

On Blu-Ray it's either 1 or 2 seconds, depending on bitrate. I don't know if there's any standard for DVB, but I'm pretty sure they're similar. Otherwise you couldn't zap through the channels fast.

[LoRd_MuldeR]

A GOP always starts with an I-Frame. P-Frames either refer to the previous I-Frame or to the previous P-Frame. B-Frames refer to the surrounding I- or P-Frames.

See also:
http://upload.wikimedia.org/wikipedia/commons/thumb/e/e2/GOP_2.svg/1000px-GOP_2.svg.png

At least that is the situation with good old MPEG-2. In H.264/AVC things are a bit more complex, because it supports multiple-references.

However all this doesn't say where the GOP boundaries are located in the video! Optimally (with respect to compression efficiency) the GOP boundaries should be located at scene boundaries.

But for the various reasons discussed before, this may not always be possible (min/max key-frame interval restrictions, encoder inability, etc). So you definitely cannot rely on that!

As sneaker_ger said, the BluRay standard enforces I-Frames every 1 or 2 seconds. There aren't that many scene changes in normal movies, so most I-Frames will not be at scene changes.

Moreover a scene change can be in the middle of a GOP. Coding the first frame of a new scene as P- or B-Frame is not very efficient, but it still is possible...

[Rouhi]

regarding GOP and role of I-Frame you have point to a useful picture of closed GOP. But there exist another type of GOP which is called open GOP and even can start with B-Frames.
http://i53.tinypic.com/208w4qq.jpg

[LoRd_MuldeR]

To my understanding, "Open GOP" means that a GOP can end with B-Frame(s). This means that the last B-Frames in a GOP make reference to the I-Frame of the next GOP.

So it is "open" in the sense that in order to decode all pictures from the current GOP, you also need frames from the next GOP. While a "closed" GOP can be decoded entirely on its own.

In a "closed" GOP we must insert a P-Frame before the end of the GOP, like this "I-B-B-P-B-B-P-B-P-I-(...)". Here the last I-Frame in the sequence is the start of the next GOP!

With an "open" GOP we would be allowed to do the following: "I-B-B-P-B-B-P-B-B-I-(...)". Now the red B-Frames need the I-Frame of the next GOP to be decoded.

Also: In the bitstream the frames are not stored in display-order (as visualized here), but in decoding-order. Therefore the red B-Frames would actually be stored after the I-Frame.

(So if you think in decoding-order rather than in display-order, then the next GOP will actually contain frames that belong to the current GOP, if "open GOP" is used).


Still, this all has to do with how the frames are coded, but it does NOT say anything about the content or the "meaning" (such as "scene-cut" -vs- "not a scene-cut") of these frames!

Once again: Optimally GOP boundaries should be placed at scene boundaries, but there are many reasons why this might not be the case (as discussed before).

Any frame, of any type (I, P or B) at any position of the GOP might (or might not) be the first frame of new scene. It's totally up to the encoder's choice and/or restrictions...