Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 29th August 2007, 10:54   #1  |  Link
jasonme
Registered User
 
Join Date: Aug 2005
Posts: 29
A rookie question regarding on B frame in AVC

Hello guys, I have a rookie question on B frame coding in AVC, please help me clarify this, thanks in advance.

In AVC, the parameter NumberOfReferenceFrames defines the maximum number of reference frames an inter frame can use for prediction. But what is the maximum number of reference frames for a B frame? Is it still NumberOfReferenceFrames or 2*NumberOfReferenceFrames?

Since I'm trying to allocate memory pool for reference frames, so this is very important for me. Thank you very much for your kindly answers!
jasonme is offline   Reply With Quote
Old 29th August 2007, 11:14   #2  |  Link
akupenguin
x264 developer
 
akupenguin's Avatar
 
Join Date: Sep 2004
Posts: 2,393
Still NumberOfReferenceFrames.
The only difference between P- and B-frames is that in B-frames 1 block can use 2 of the reference frames at once.
akupenguin is offline   Reply With Quote
Old 29th August 2007, 12:08   #3  |  Link
jasonme
Registered User
 
Join Date: Aug 2005
Posts: 29
Thank you for clarifying me that.

There is another thing that's confusing me now. How to control play flow of a H.264 bit-stream. For example, If I have a bit-stream of format I P B1 B2 B3 P, then I know that after decoding I frame, it can be displayed. Then the P frame is gonna wait there until next P frame decoded for display. But the display order of B frame is not such a clear case. So along with the decoding flow, how can I control the display flow to display the video in the correct order?

Thank you guys a lot for everything. These are all basic questions, hope you guys don't mind me asking these things here.
jasonme is offline   Reply With Quote
Old 29th August 2007, 12:21   #4  |  Link
akupenguin
x264 developer
 
akupenguin's Avatar
 
Join Date: Sep 2004
Posts: 2,393
In each frame header, the encoder writes the frame number in display order.
So the stream goes I0 P4 B1 B2 B3 P8 ..., where the number is not just my annotation but actually in the stream.
akupenguin is offline   Reply With Quote
Old 29th August 2007, 20:03   #5  |  Link
Sergey A. Sablin
Registered User
 
Join Date: Dec 2004
Location: Tomsk, Russia
Posts: 366
Quote:
Originally Posted by jasonme View Post
Thank you for clarifying me that.

There is another thing that's confusing me now. How to control play flow of a H.264 bit-stream. For example, If I have a bit-stream of format I P B1 B2 B3 P, then I know that after decoding I frame, it can be displayed. Then the P frame is gonna wait there until next P frame decoded for display. But the display order of B frame is not such a clear case. So along with the decoding flow, how can I control the display flow to display the video in the correct order?

Thank you guys a lot for everything. These are all basic questions, hope you guys don't mind me asking these things here.
check out picture_order_counter (POC) from slice header and dbp_output_delay from PictureTiming SEI message.
Sergey A. Sablin is offline   Reply With Quote
Old 30th August 2007, 03:41   #6  |  Link
jasonme
Registered User
 
Join Date: Aug 2005
Posts: 29
Got it, thanks. This helps a lot with my player. Thank all you guys very much for the help!
jasonme is offline   Reply With Quote
Old 30th August 2007, 09:34   #7  |  Link
jasonme
Registered User
 
Join Date: Aug 2005
Posts: 29
Wait, I still have one more problem.....

In JM software, there is a parameter:"BReferencePictures", which indicates whether B frames can be used as a reference frame or not. But surprisingly, this info is not written to the bit-stream. Then how can I know on the decoder side that whether I should keep decoded B frames in the decoded picture buffer as a reference frame?

Thanks again!
jasonme is offline   Reply With Quote
Old 30th August 2007, 10:04   #8  |  Link
akupenguin
x264 developer
 
akupenguin's Avatar
 
Join Date: Sep 2004
Posts: 2,393
There is no "whether B frames can be used as a reference frame or not". Each individual frame can be referenced or not, indicated by the NAL priority. "disposable" = not referenced, the other 3 possible settings are all referenced. Also look at "memory control operations" which can manipulate the DPB beyond just choosing which pictures get added.

Last edited by akupenguin; 30th August 2007 at 10:07.
akupenguin is offline   Reply With Quote
Old 30th August 2007, 11:21   #9  |  Link
jasonme
Registered User
 
Join Date: Aug 2005
Posts: 29
Quote:
Originally Posted by akupenguin View Post
There is no "whether B frames can be used as a reference frame or not". Each individual frame can be referenced or not, indicated by the NAL priority. "disposable" = not referenced, the other 3 possible settings are all referenced. Also look at "memory control operations" which can manipulate the DPB beyond just choosing which pictures get added.
So can I put it this way, if a NAL unit has the property of nal_ref_idc=0, then this frame is a reference frame, but if not, this frame is a non-reference frame. Am I right about this?
jasonme is offline   Reply With Quote
Old 30th August 2007, 11:33   #10  |  Link
akupenguin
x264 developer
 
akupenguin's Avatar
 
Join Date: Sep 2004
Posts: 2,393
other way around.
akupenguin is offline   Reply With Quote
Old 30th August 2007, 12:09   #11  |  Link
jasonme
Registered User
 
Join Date: Aug 2005
Posts: 29
Ahh, got it, thank you so much!
jasonme is offline   Reply With Quote
Old 31st August 2007, 09:01   #12  |  Link
jasonme
Registered User
 
Join Date: Aug 2005
Posts: 29
As the AVC player process goes on, more rookie questions are on my way. First I would like to thank your patience to let me keep throwing these stuff to your guys. Thank you very much for this.

This time, my question is that if I have a sequence decoded in the following order: I1 P5 B3 B2 B4, the number indicate the display order of the frame. But for B3, the frame is marked disposable. By my earlier questions, this frame should be decoded and discarded not using as a reference frame. But I can't really discard it right? I need to hold the decoded B3 frame a little bit longer to wait for B2 get decoded and displayed. Then I can display B3 and release the memory it occupies. Thus, by defining a frame disposable, it only means that I'll not use it for reference but I can't give the frame up until until it is displayed for as long as it takes.

Am I right about this? Or there is no such way to generate a sequence I described above. To my best memory, SVC has sequences like this I believe. I don't know what about how this thing is in the main profile AVC.

Thank you very much for the help!
jasonme is offline   Reply With Quote
Old 31st August 2007, 09:14   #13  |  Link
Sergey A. Sablin
Registered User
 
Join Date: Dec 2004
Location: Tomsk, Russia
Posts: 366
Quote:
Originally Posted by jasonme View Post
As the AVC player process goes on, more rookie questions are on my way. First I would like to thank your patience to let me keep throwing these stuff to your guys. Thank you very much for this.

This time, my question is that if I have a sequence decoded in the following order: I1 P5 B3 B2 B4, the number indicate the display order of the frame. But for B3, the frame is marked disposable. By my earlier questions, this frame should be decoded and discarded not using as a reference frame. But I can't really discard it right? I need to hold the decoded B3 frame a little bit longer to wait for B2 get decoded and displayed. Then I can display B3 and release the memory it occupies. Thus, by defining a frame disposable, it only means that I'll not use it for reference but I can't give the frame up until until it is displayed for as long as it takes.

Am I right about this? Or there is no such way to generate a sequence I described above. To my best memory, SVC has sequences like this I believe. I don't know what about how this thing is in the main profile AVC.

Thank you very much for the help!
the sequence you shown is similar to hierarchical B-frame coding (or pyramid coding), when B3 was encoded before B2 and B4 and was used as reference to those two. But in this case it should not be disposable neither in AVC nor in SVC as SVC share same features with AVC for base layer coding.

In case B3 isn't used for reference, but there is a gap between P5 and B3 in pic_order_cnt, you should store this non reference B-frame in DPB until it's time to display will come.

[edit] oops, sorry - gap between I1 and B3, not between P5 and B3
Sergey A. Sablin is offline   Reply With Quote
Old 31st August 2007, 09:46   #14  |  Link
jasonme
Registered User
 
Join Date: Aug 2005
Posts: 29
Quote:
Originally Posted by Sergey A. Sablin View Post
the sequence you shown is similar to hierarchical B-frame coding (or pyramid coding), when B3 was encoded before B2 and B4 and was used as reference to those two. But in this case it should not be disposable neither in AVC nor in SVC as SVC share same features with AVC for base layer coding.

In case B3 isn't used for reference, but there is a gap between P5 and B3 in pic_order_cnt, you should store this non reference B-frame in DPB until it's time to display will come.

[edit] oops, sorry - gap between I1 and B3, not between P5 and B3

So I am right about having to keep disposable frames in some certain circumstances then.

This will lead to other issue. If there is a lot of B frames between I/P frames and they are coded in pyramaid pattern. But the weird thing is that they are all disposable frames. How much memory I have to allocate to deal with this situation? Since we only keep the buffer for display not for reference, so I guess that the more B frames we have in a row, the more buffer we need. This will definitely make the hardware solution harder right? Especially for hardware with limited RAM.

Then how can we avoid this thing form happening? Thanks!
jasonme is offline   Reply With Quote
Old 31st August 2007, 10:35   #15  |  Link
akupenguin
x264 developer
 
akupenguin's Avatar
 
Join Date: Sep 2004
Posts: 2,393
See num_reorder_frames which is an optional field in the SPS header. If it is not present, the standard gives a rule to infer an upper bound. And in any case it will never be more than 16 frames.
It's not directly related to disposability: Even if all frames are referenced, they could still be nontrivially reordered.
akupenguin is offline   Reply With Quote
Old 31st August 2007, 12:58   #16  |  Link
jasonme
Registered User
 
Join Date: Aug 2005
Posts: 29
So basically, to be safe I better keep a buffer of 16 reference frames and 16 display frames buffer to make sure that all sequences are decodable, right? That's a lot of memory for high definition AVC decoding.
jasonme is offline   Reply With Quote
Old 31st August 2007, 13:13   #17  |  Link
akupenguin
x264 developer
 
akupenguin's Avatar
 
Join Date: Sep 2004
Posts: 2,393
Up to 16 frames total, and at any given time some of them will be referenced or in the reorder buffer or both.
The main factor that determines the bound is Level. Each of the Levels puts an upper bound on the amount of memory you have to allocate to decoded frames. So higher resolution means fewer frames at a given Level.

Last edited by akupenguin; 31st August 2007 at 13:16.
akupenguin is offline   Reply With Quote
Old 31st August 2007, 13:39   #18  |  Link
jasonme
Registered User
 
Join Date: Aug 2005
Posts: 29
This is great news for me, thank you very much for your help all the time and also for Sergey A. Sablin. I may have further questions. So I'll paste my other issues here too. Thanks in advance for everything!
jasonme is offline   Reply With Quote
Old 31st August 2007, 18:32   #19  |  Link
Sergey A. Sablin
Registered User
 
Join Date: Dec 2004
Location: Tomsk, Russia
Posts: 366
Quote:
Originally Posted by akupenguin View Post
See num_reorder_frames which is an optional field in the SPS header.
max_dec_frame_buffering.
Sergey A. Sablin is offline   Reply With Quote
Old 31st August 2007, 18:47   #20  |  Link
akupenguin
x264 developer
 
akupenguin's Avatar
 
Join Date: Sep 2004
Posts: 2,393
num_reorder_frames is how many frames might be reordered, which is the question I answered. max_dec_frame_buffering was the next question, involving amounts of memory.
akupenguin is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:01.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.