Help on MPEG TS internal header format

[konran]

Hello,

I've got stuck during reading, parsing and analyzing MPEG TS headers after reading a lot of stuff from ISO/IEC 13818-1 ... there I didn't find (or understand it right) how sections, table ID's and PID's work together.

For example when I want to scan a .ts file for the PMT and PAT I think I first have to search for a PID 0x0000 to get a PAT entry because the value of the PMT is not a fixed one and would be found as one of the entries of PAT. Correct? - Now, how do the table ID values fit into this concept ... are they always the first bytes in the payload when the header says that a section starts in a packet?

Is it mandatory that PAT and PMT will repeated within some time interval in the case of switching into a TS stream at any time?

Is there any good guide on this TS stuff out there?

Best regards,
Konran

[neuron2]

Take a look at the source code in pat.cpp of DGIndex. It has everything you need to parse PAT/PMT tables.

[konran]

That sounds very good ...
Take a look at the source code in pat.cpp of DGIndex. It has everything you need to parse PAT/PMT tables.
... but it seems I must be quite blind - the only reference I find is an EXE of GordianKnot that includes it but w'out the sources. sourceforge.net also seem to share EXE only.

Please tell me: where did you hide it?

Regards,
Konran

[neuron2]

You could try my home page:

http://neuron2.net/dgmpgdec/dgmpgdec.html

(GK is violating GPL??? The horror.)

Funny, the first Google linked returned on a search for "DGIndex" is the one above.

http://www.google.com/search?sourceid=navclient&aq=t&ie=UTF-8&rls=GGLD,GGLD:2005-08,GGLD:en&q=DGIndex

BTW, your starting point is AnalyzePAT().

[konran]

Thanks a lot :thanks:
You could try my home page:

http://neuron2.net/dgmpgdec/dgmpgdec.html
...
Funny, the first Google linked returned on a search for "DGIndex" is the one above.
I googled "DGIndex" with "pat.cpp" together - unfortunately that didn't show your link :scared:
BTW, your starting point is AnalyzePAT().
I've checked it shortly and it looks good as it gives many hints for my questions. It commits some things I've already tried and is adding information on things I've not yet done. I'll get deeper into it in the evening back home because I don't have VS in my office.

Cheers,
Konran

[konran]

Hello again,

with your help I've coded PAT and PMT detection, extraction and checking with my own C++ class schematics, of course. As I don't have any TS files of my own I used samples from the ffmpeg test site. But it seems they've put only files there that have encoding failures more or less. Some of that stuff leads me to the question how to deal with section length splitting. The ISO/IEC spec notes 0x3fd as a maximum length for a section. So when TS packet size is 188 bytes (or 192 or 204) such a section that exceeds the TS packet size must be split into several TS packets ... so I understand it, right?

Now I've got a TS test file that has several copies of the same PMT every time it occurs. I find 6 valid PID/stream type pairs. My code and also DGIndex detects the same 6 pairs. BUT the problem is: the PMT's section length is 0x0d3 and I have only 0x0b4 bytes left for one single section space in that packet. The section_syntax_indicator is true, current_next_indicator is true, section_number and last_section_number contains the same value and the PMT version_number is 0x18 within each PMT I find in this file (PID of the PMT and the program_number is also always the same and is in tune with the PAT entries). So from this point of view the PMT must be valid and complete, there couldn't be any follow up TS packets that would complete the PMT. The missing thing is some bytes of the descriptors of the last PID/stream type pair.

What's wrong with this? Should I ignore the missing part of the PMT which is on a stream type of 0x0b?

Are there any links to TS test files that are encoded correctly? Maybe different ones that have PMT's and others with PSIP tables to test this feature?

Regards,
Konran

[neuron2]

The ISO/IEC spec notes 0x3fd as a maximum length for a section. It's 1024 for 13818-1 defined sections and 4096 for private sections.

So when TS packet size is 188 bytes (or 192 or 204) such a section that exceeds the TS packet size must be split into several TS packets ... so I understand it, right? Yes, of course, and my code allows for that.

Now I've got a TS test file that has several copies of the same PMT every time it occurs. In a broadcast environment they must be periodically repeated.

What's wrong with this? Should I ignore the missing part of the PMT which is on a stream type of 0x0b? I don't see why you think there won't be a following packet with the remainder of the section. Nothing you have cited ends the section. The only thing that ends it is when all of it's bytes have been received (or possibly a new section number in the case of missing packets).

I don't know about sample files on the net as I have accumulated my own set of samples over the years.

[konran]

Thanks for your additional information :)
I don't see why you think there won't be a following packet with the remainder of the section. Nothing you have cited ends the section. The only thing that ends it is when all of it's bytes have been received (or possibly a new section number in the case of missing packets).
Because current_next_indicator says that THIS packet's PMT data is applicable and I mixed up the meaning of the fields section_number and last_section_number with the PAT's syntax. But when you say the only criteria for the sections end is after the full section_length is read then I have a problem. This particular stream has one valid video stream and two audio streams and 3 others. I checked repetitions of the PMT and I always get the same section length and contents, i.e. all PIDs and stream types and descriptors.

So how should it look when a section is splitted:
- will these PMTs follow up normally in subsequent packets?
- will the section_length be the same for all PMT packets that belong together within follow-up packets?

Regards,
Konran

[neuron2]

So how should it look when a section is splitted:
- will these PMTs follow up normally in subsequent packets?
- will the section_length be the same for all PMT packets that belong together within follow-up packets?
It won't necessarily be the subsequent packets, because other PIDs could be interleaved. But it will be the next packets on that PMT PID.

Only the packet that starts the section will carry the section_length (and other section header fields). Subsequent packets just carry on with the data of the section (after the transport overhead).

If you'd like to post a piece of your transport stream, I'll have a look and tell you what is going on.

I'd also be happy to give you any test streams that might help you.

[konran]

If you'd like to post a piece of your transport stream, I'll have a look and tell you what is going on.

I'd also be happy to give you any test streams that might help you.
Oh yes, I would appreciate that very much :)
I send you PM with further details.

Regards,
Konran