Demuxing h264 from a transport stream

[tidi]

Hi,

I'm trying to demux a Transport stream with h264. I can get almost everithing but I've got a couple of questions...

1) it seems that while parsing a Video Unit I find an Audio one.
is it standard to put a stream unit inside another one?

the stream is something like this

TS PACKET: PES VIDEO START
TS PACKET: VIDEO DATA
TS PACKET: VIDEO DATA
TS PACKET: VIDEO DATA
TS PACKET: VIDEO DATA
TS PACKET: AUDIO PES + DATA
TS PACKET: VIDEO DATA
TS PACKET: VIDEO DATA
TS PACKET: PES VIDEO STOP

2) the packet len in the PES of the video stream (h264) is zero.
this means that I do not have a PES packet len and I need to build the Unit until I found the end of the Video unit..
Can I get the PES size somewhere else, maybe h264 bitstream? or I need to parse the stream until the next pes to figure out the size?

Thanks Matteo

[kieranrk]

Hi,

I'm trying to demux a Transport stream with h264. I can get almost everithing but I've got a couple of questions...

1) it seems that while parsing a Video Unit I find an Audio one.
is it standard to put a stream unit inside another one?

the stream is something like this

TS PACKET: PES VIDEO START
TS PACKET: VIDEO DATA
TS PACKET: VIDEO DATA
TS PACKET: VIDEO DATA
TS PACKET: VIDEO DATA
TS PACKET: AUDIO PES + DATA
TS PACKET: VIDEO DATA
TS PACKET: VIDEO DATA
TS PACKET: PES VIDEO STOP

2) the packet len in the PES of the video stream (h264) is zero.
this means that I do not have a PES packet len and I need to build the Unit until I found the end of the Video unit..
Can I get the PES size somewhere else, maybe h264 bitstream? or I need to parse the stream until the next pes to figure out the size?

Thanks Matteo

Number 1 shouldn't be the case.

Number 2 is normal. You will have to parse the stream until the next PES.

[tidi]

Number 1 shouldn't be the case.

Number 2 is normal. You will have to parse the stream until the next PES.

Thanks for your reply!

about the number 1
You can confirm that should not happening?

VLC plays it back so I suppose I must handle it too...:-(

number2:
is there a max size for the video unit of a video elementary stream into TS so I can allocate enough memory at the beginning?
If I'm correct the max size for a NAL unit is 65535.

Cheers

[Guest]

Audio and video packets can be interleaved. There is no requirement that all the packets of a video access unit be completely sent before some audio packets can be sent.

[kieranrk]

Audio and video packets can be interleaved. There is no requirement that all the packets of a video access unit be completely sent before some audio packets can be sent.

I assumed he was talking about the Video PID and audio PID being the same but I guess he wasn't.

[popper]

BTW , hows Your TS muxer coming along kieranrk, will we see your working code for x264 soon?

[kieranrk]

BTW , hows Your TS muxer coming along kieranrk, will we see your working code for x264 soon?

Understanding Dolby E is more interesting right now. The muxer will come after that.

[tidi]

Audio and video packets can be interleaved. There is no requirement that all the packets of a video access unit be completely sent before some audio packets can be sent.

perfect, I need to do some wotrk then :-)

does someone know if there's a max size for the video unit of a video elementary stream into TS so I can allocate enough memory at the beginning?
If I'm correct the max size for a NAL unit is 65535.

thanks

[Shevach]

does someone know if there's a max size for the video unit of a video elementary stream

If I understand your question correctly - you ask whether there are constraints on picture size in H.264?
Except for the parameter max_bytes_per_pic_denom (which is signaled in VUI extension) there is no direct restriction on the maximal picture size.
But in H.264 there is clear constraint on MB size:
any MB shall not exceed 128 + RawMbBits bits, where RawMbBits for 4:2:0 equal to 256*8+64*8+64*8=3072.
So, if each MB does not exceed 128+3072 = 3200 bits and the regarded picture contains N macroblocks then maximal picture size is
N*3200 + overhead_for_slice_headers.
The problem is to estimate overhead_for_slice_headers. Generally speaking the maximal size of slice header is above 4Kbits if prediction weights are signaled and number of references is 32.
Let's suppose that stream level is 3 then SliceRate is 22, i.e. up to 22 slices can be present in a single picture.
So, maximal picture size is N*3200 + 4000*22
Anyway for HD stream the maximal picture size can easely exceed 65Kbytes.

[tidi]

If I understand your question correctly - you ask whether there are constraints on picture size in H.264?
Except for the parameter max_bytes_per_pic_denom (which is signaled in VUI extension) there is no direct restriction on the maximal picture size.
But in H.264 there is clear constraint on MB size:
any MB shall not exceed 128 + RawMbBits bits, where RawMbBits for 4:2:0 equal to 256*8+64*8+64*8=3072.
So, if each MB does not exceed 128+3072 = 3200 bits and the regarded picture contains N macroblocks then maximal picture size is
N*3200 + overhead_for_slice_headers.
The problem is to estimate overhead_for_slice_headers. Generally speaking the maximal size of slice header is above 4Kbits if prediction weights are signaled and number of references is 32.
Let's suppose that stream level is 3 then SliceRate is 22, i.e. up to 22 slices can be present in a single picture.
So, maximal picture size is N*3200 + 4000*22
Anyway for HD stream the maximal picture size can easely exceed 65Kbytes.

Actually I'm looking for the max size of a PES Packet if the content of the TS is an h264.

The problem is that with h264 the PES_packet_length is set to 0 (at least the one I've got), so I need to parse the file until the next PES packet to find out the length of the PES packet.

Now If there's a max size for the PES packet/Access unit for an h264 then I can skip the parsing of the next PES.

[Guest]

Length 0 indicates that the packet length is "unbounded". That's all that can be said with certainty.

Even if there was a certain maximum length, you still need to parse for the next one to determine the *actual* length of the current one.

[Shevach]

So, if PES packet size is unbounded then the maximal H.264 picture size actually "bounds" it.
In other words if someone knows the maximal picture size in 264 stream then he knows maximal PES packet length.

[Guest]

It could be stuffed arbitrarily. :)

[Shevach]

ATSC/A-53 puts some restrictions on PES packet.