Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
20th November 2015, 15:29 | #1 | Link |
Registered User
Join Date: Aug 2005
Posts: 18
|
Decoding H.264 in MPEG-2 transport streams
Hi,
I am writing a program to identify IDR frames in an H.264 in MPEG-2 transport stream. The source video is streamed by vlc (v2.2.0) with "RTP/MPEG Transport Stream" destination and H.264+MP3 (MP4) profile. So far I am not able to find a resource which documents the packet format. I can see documentation about packet structure of H.264: NAL Units, but documentation about how they are contained in MPEG-TS container is not to be found. I can see the TS headers starting with 0x47, but what comes next is a mystery. Any pointers is much appreciated. Following is the raw dump of some of the packets I am seeing for reference: Start of dump of packet 386 47004518 246D6246 D9639AB8 76C15270 1F7001F4 EE89F77A D529056A 1D6F6B47 2A9DCDBB 52C06C9A 362D435A FA28E400 6D64E5C5 020F8A25 3EB4D0BB 2C8E8269 B2B67D81 B4E1EBDF 2D0AB171 1EE6B73E C17C6AE8 07DDD566 A0F5748F DA9AA472 8E46ABC9 73C22BE8 0E9DB7BB 1E326861 B3727009 B65700E1 2CB401F9 33A7277E 2774C958 EE19A561 913FBE92 BDA1E1C1 9B7183FC 129CC115 8FD6ACB5 AFDFF8F9 D9C1F74A D1E0BB65 1A67C9D3 B277B486 B3852A9C B3BC3367 BAC141C6 00000000 Start of dump of packet 387 47404415 000001C0 01A98080 052190E7 97E3FFFD 80044433 33322232 22222211 11249249 00000900 00000000 ABAAABB9 AAAAA6AE 1C739138 E4145565 9761959A 5D765761 86DC6DB7 1C71D71B 6DD7227D F420E461 3564DC8D 5CCE2B71 2BE94EEB B7E38DE8 91020292 CDB4DAD5 36CD33A2 77E2DD4A A2E17252 592072C0 A1FA3438 76870ECE CE5836AC 646B74EA 63B5DDA5 6379DAF6 2F2C7152 384DB9AD B9F533CD D5374042 391AD528 ED7354A6 BAA178DA 10E04347 16DFA8ED 7AED2045 00000000 Start of dump of packet 388 47004519 3A98BBB1 56DE0C76 DF359974 3AD6C36E B5C92FE6 91FB9282 1C773F28 EFE4EB90 FD25A18F C28A81C7 01437068 AEAF6D9E 04F7BAC8 D228DB46 839A6A85 F16403F7 FF9B7E7A 0788B29C B72FE010 A79996E6 BE7C3893 517602B0 8D7A5378 4067D8E4 7C0BAC0E 2CD85D5F 05BE42F9 83381659 505E63F7 5739C3C4 BBD138E9 22DB602C 7DC82BA7 463D8171 A0311912 80B6EEC5 7F08DE97 ED158BA4 14D7C036 E9784122 2C297F09 5C5690E5 18694202 F62D5E89 7B99AE71 B2869145 00000000 Start of dump of packet 389 4700451A 2071EA49 3F285273 36CF3308 877BA784 45E5D173 4B2F52F7 697A0C6F A72DFF3E C0A5986A C26B33FD 6C37656F 22F40C96 E369F566 34E284B7 C63376E8 26AA7840 47582682 C26F1A18 6D41C1F5 384AE189 D04C85FB 3FD1F2DC 38DECC2F A025086A 6396A8B3 D63BED58 AB81A922 1C9102CE DD484BC2 86267476 562F7B34 E0A559D7 26A517B5 2F4D1700 4FDA8C55 28992B03 8CD43F93 F06C5E40 498C1931 31410C40 90497216 6FD7D172 93D32599 B01F0D61 CB98B500 997EFE12 00000000 Start of dump of packet 390 4700451B F60EF8B1 20FC5EC4 0C3B2071 C60362E1 DA4CFEFA 87ACB0D5 87A685C3 41FBE318 759B5256 0782FA1C 04EB47FB C21D1BDB 1CBBDD6B 96FFE3EC 73CF7CD4 4E8D42FD 476AEB66 A8BA60EA E01AAFD2 2FC4686F D45E5477 6195ED33 46C0D2E5 2B469519 5AF4AED1 D377F5F6 3C1279BE 5CAA6FB4 CE33AC74 FE50253C 6FF1DCE2 8039B20E E7ED7D5F 138B4F3E 7A506E4C 8E075283 F9BBF801 96EC2F5F 56F560D5 8EE93BDF 492C351B 52F23525 BF753B6A 8DBEFA08 EDCBBC6B CD8E05F4 00000000 Start of dump of packet 391 4700453C 8600FFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFF2E 2D287D58 DC1829A7 A6196089 C59CC23A 29CDD44D EBB4558F 3B070489 E9E65C7A 5C02F6AC 3E2BD107 ED6BAE24 3F924B20 00000000 Start of dump of packet 392 4740451D 000001E0 146A80C0 0A3190E7 D72F1190 E79C9700 00000109 E0000000 01419A33 49E10F26 53027FB0 F6E417BB 6BCB3929 2E359B04 5CB03DB6 4157981D EBAEE340 A72CF089 81C653AD 4A191FC3 CE534FFD 7E3545F4 0512613F 1BE17DE0 DBA9648D B8D18DB6 22E5881F D904B78F D3244E21 DC5D46A0 D2925288 4AF04550 F54AEFA3 E49F9CCA B4C34D1F 460CFFEA D98BB2F0 19EF5D17 2B081543 AB16867E 1C968C7C 3C55156D F2135C80 1CD2B37B 4D5C40D1 4611341F 09C08504 00000000 |
20th November 2015, 19:42 | #2 | Link |
Registered User
Join Date: Mar 2006
Posts: 1,049
|
There is no such thing as MPEG-2 TS - there is MPEG TS and it is covered by http://www.itu.int/rec/T-REC-H.222.0 - you should be able to find all information's there.
Side to this you may analyze code for existing open source TS demuxers:ffmpeg , https://code.google.com/p/tsdemuxer/ , http://www.w6rz.net/ xport etc. |
21st November 2015, 11:29 | #3 | Link | |
Life's clearer in 4K UHD
Join Date: Jun 2003
Location: Notts, UK
Posts: 12,227
|
Quote:
That being said, the 'transport stream' is specified within Part-1 of the MPEG-2 ISO/IEC 13818 standard
__________________
| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 | My Network Layout & A/V Gear |
|
|
21st November 2015, 22:59 | #4 | Link |
Angel of Night
Join Date: Nov 2004
Location: Tangled in the silks
Posts: 9,559
|
Short answer: Search for 00000165, 00000145, or 00000125. You're guaranteed to find all IDR, but you might accidentally find something else if you happen to be in the middle of a non-AVC stream. If you write a TS, PES, and ES parser all of that information will naturally be available to you. You can also piggyback on ffmpeg's demuxer.
Long answer: You posted a limited number of packets that makes it hard to parse, but 386, 388, 389, 390, and 392 are all video stream, 387 is audio, and 391 is something else, based on the PIDs. It's obvious which are which because 387 and 392 have 000001 PES packet start codes (Cx-Dx is audio, Ex-Fx is video); the 474x in the TS header requires them there anyway. Since 392 is the only actual start of a video frame, we'll look at that. The full PES header is 000001E0 146A80C0 0A3190E7 D72F1190 E79C97. That's not too important for now, except it tells us the frame size is 5216, or 185 TS packets. Following that is the ES, with 00 00000109, the (optional) AUD, the start of a H.264 frame. AUDs (and IDRs) almost always have 3 zero bytes instead of 2, just just as a gentleman's agreement, to make it easy to find new frames: Just search for every 00000001. E0 means any slice type. Now you get to 000000 0141. 41 parses out to "referenced slice of non-IDR pic". If it was an IDR, it would be 45 (or 25 or 65, damn you AVC). x264 always uses 65 for IDR and 4x for referenced non-IDR, but some other encoders just use 4 for everything. Anyway, now you have your answer to seeking IDR startcodes. Last edited by foxyshadis; 21st November 2015 at 23:01. |
23rd November 2015, 15:29 | #6 | Link | ||
Registered User
Join Date: Aug 2005
Posts: 18
|
foxyshadis,
I could understand the parts upto PES headers, could not understand what comes after that, before H.264 frame. Quote:
What is AUD? For confirmation, does H.264 frame starts after 00 00000109 E0? Quote:
Thanks again for your help. |
||
24th November 2015, 09:06 | #7 | Link |
Registered User
Join Date: Aug 2005
Posts: 18
|
I was looking for 00000165 to check for IDR frames, and found that it occurs deep inside the packet (marked in red). The dump is:
4740471B 000001E0 5A2980C0 0A3D61FB 0EE91D61 F9D45100 00000109 E0000000 01676400 1FACD940 400E7E7C 04400000 03004000 000C23C6 0C658000 00000168 EBE3CB22 C0000000 01658884 0023FFFC 39A9F2E1 6E7DA87E 0F17B2F6 65276E25 314443AC 423F1DF6 04E9BEC1 E15D492D 63E87C54 8AA37A53 3F8CD9B6 4586A380 62992B50 8C9F1FD0 E84CB72B 7D4A3811 B3AA15AB 296C88F8 1250C5E6 8FE42B97 DC0597A3 E9B6D01C 5C476777 F5C0E773 19C26B92 E3BEE60C DCC0828C 00000000 While I was expecting it at location marked in green. Just wondering if this is where it should be expected? |
24th November 2015, 10:22 | #8 | Link | |
Angel of Night
Join Date: Nov 2004
Location: Tangled in the silks
Posts: 9,559
|
Quote:
Immediately after E0 is the beginning of the first slice of the frame, yes, the byte being the three fields you stated. |
|
24th November 2015, 10:26 | #9 | Link | |
Angel of Night
Join Date: Nov 2004
Location: Tangled in the silks
Posts: 9,559
|
Quote:
|
|
24th November 2015, 11:20 | #10 | Link |
Registered User
Join Date: Aug 2005
Posts: 18
|
Thank you for the detailed answer!
Just trying to map the bytes I am seeing with whats on standards. Is that "Access unit delimiter" nal_unit_type ? I am reading through ISO/IEC 14496-10 standard doc, but could not figure out the values and its meaning of primary_pic_type. Last edited by raj2569; 24th November 2015 at 14:40. |
24th November 2015, 18:31 | #11 | Link | |
Angel of Night
Join Date: Nov 2004
Location: Tangled in the silks
Posts: 9,559
|
Yup.
Quote:
I haven't worked with multi-slice video in a long time, so it's always more useful to me to just read a few more bytes and grab the actual frame type instead of the AUD set. The AUD set is only potentially useful in the case where a frame has multiple slices and each slice has a different type and it's vitally important to know that. A similar marker that's useless to decoders is that if the first slice is 5-9, every slice in the frame must be the same type. I rarely see this, but it's more informative than the AUD set. |
|
Tags |
h.264, mpeg-ts |
|
|