M2TS muxing - BDAV arrival timestamps & leak rate?

[Phoil]

Hi guys,

I'm trying to track down an issue in a BDAV .m2ts muxer I built, and would like any help you guys can provide.

So... I have everything working fairly well (muxing raw x264 and PCM into an m2ts stream), but in some decoders (e.g. VLC, WinDVD) I get badly artifacted video playback.

From looking at Dr.Mpeg's posts in this archived thread (http://forum.doom9.org/archive/index.php/t-134104-p-2.html), I believe the issue has to do with the video buffer and the 4-byte "arrival timestamp" and the "leak rate". The reason I believe it's related to the arrival_timestamp values is that I can run my .m2ts through tsMuxer, and the playback is perfect. But there only seem to be a couple of things that tsMuxer does to the stream:

Changes the arrival_timestamps to have a constant delta between each pair of PCR packets.
Inserts the PMT/PAT/SIT packets at slightly different places. (Shouldn't cause issues, from my understanding.)
Puts a 0 length in the PES header for some video packets. (That's just odd in my book.)


So, I think it's the arrival_timestamp deltas, but I haven't found much good information on this, and a calculated constant delta seems incorrect to me. Can someone shed some light on the matter?

My mux (https://www.dropbox.com/s/n4rgwjzd1nxbwpw/00001.m2ts) (broken)
Remuxed (https://www.dropbox.com/s/6cd3bbrh4a5yx61/re.m2ts) (works)


Thanks, guys!

[drmpeg]

The key to building a good HDMV (or .m2ts) muxer is to first build a regular fixed bitrate 188 byte (.ts) muxer. Once you have a T-STD compliant regular TS muxer, a T-STD compliant HDMV muxer will fall right out of it.

I realize this is probably quite a bit of extra work for you, but I assure you it's the difference between a hack like tsMuxer and a technically correct muxer that's just as good as the ones included in professional BD authoring packages.

If you're willing to go down this path, then it's very easy for me to help you. I have a professional Transport Stream analyzer (Manzanita MP2TSAE), and I can test your bitstreams with minimal effort on my part. Otherwise, it's too difficult to debug your .m2ts streams since it is not a sane starting point.

Ron

[drmpeg]

BTW, it's super common to use a PES length of 0 for video packets. A fairly up to date MPEG-2 systems document can be found here.

http://www.itu.int/rec/T-REC-H.222.0-200002-S/en

PES packet length is discussed in section 2.4.3.7

The AVC amendment can be found here.

http://www.itu.int/rec/T-REC-H.222.0-200403-S!Amd3/en

Ron

[Phoil]

It won't be difficult to create the 188-byte .ts; adding the 4-byte BDAV timestamps is just a flag in my muxer.

I'll read up on the PES 0-length, apply any needed changes, and get a .ts to you to evaluate. I'd love to get this muxer generating T-STD-compliant streams.

Thank you again, Dr. Mpeg!

[Ghitulescu]

We would be happy to get another muxer beside tsMuxer whose development seems to be arrested since 2009.

[Phoil]

Okay, after remuxing without the arrival timestamp, it appears my .ts still has what appears to be a video buffer (leak rate?) issue:

Test .ts (https://www.dropbox.com/s/s821wd4lu11zy78/wildlife.ts)

I didn't make any changes to apply a PES packet_length = 0, since I'm still unclear when/where this is appropriate. (Is it something I need to do at all?)

[Phoil]

Oh, man... I just realized that PES length should be 0 (unbounded) when the packet length is over 0xffff.

I'm guessing this is the problem causing my artifacted playback. I'll check.

[Edit: Yep, that was it. Sheesh.]

[Phoil]

Okay, now that I've fixed my PES packing issue... are there any other issues in my .ts, Dr. Mpeg?

I appreciate any help.

[Guest]

Your first GOP is closed so the PTS and DTS timestamps should be the same for the first I frame, but the stream has a difference:

V1011 PTS 54000000 [600000 ms]
V1011 DTS 53996246 [599958 ms]

That's based on my experience with transport streams, not any understanding of the T-STD model. :)

The stream plays fine and has good PAT/PMT so it looks promising to me. Nice job!

[drmpeg]

Okay, now that I've fixed my PES packing issue... are there any other issues in my .ts, Dr. Mpeg?

I appreciate any help.

There are no stuffing packets (PID 0x1fff) in the test file "wildlife.ts". This means it's not a fixed rate Transport Stream, and there's no way to determine if the PCR is correct (which is the starting point for T-STD compliance).

The best way to think of HDMV (.m2ts) streams is that they are normal fixed rate Transport Streams with the stuffing packets removed and the functionality of the missing stuffing packets replaced by the arrival_time_stamp.

The arrival_time_stamp is really just the truncated (and normalized) PCR of each TS packet (minus 10 bytes, since the PCR is the timestamp of the PCR itself and the arrival_time_stamp is the PCR of the beginning of the 188 byte packet).

Ron

[Phoil]

Thank you!

I haven't been inserting any stuffing packets in the .ts, but for HDMV/.m2ts I do fill out the last 6Kb "aligned unit" with PID 0x1fff (per a spec I saw somewhere).

Okay, just so I'm clear:

For muxing a T-STD .ts, I need to stuff it to make it a fixed-rate TS (basically a constant time-to-packet ratio, as I understand).
To create a T-STD HDMV/.m2ts, I just insert the proper arrival_timestamp derived from the stuffing packets (which get removed).

...Correct?

Do you have an algorithm for inserting stuffing packets in the right places? Basically, I'm building the stream as fast as the AV packets come in, and inserting the PCRs and PMT/PAT/SIT as needed based on the timestamps.

[drmpeg]

Do you have an algorithm for inserting stuffing packets in the right places? Basically, I'm building the stream as fast as the AV packets come in, and inserting the PCRs and PMT/PAT/SIT as needed based on the timestamps.

There are many ways to skin the cat. But whatever algorithm you end up with is the difference between the freeware crap you see on the internet and professional TS muxers that cost $8000 a copy.

Essentially, you have it backwards. You don't create the PCR based on the timestamps from the elementary streams, but instead you must create a PCR timebase based on the fixed bitrate (basically you add a fixed increment to an internal PCR for each packet, stuffing or otherwise). As the internal PCR ticks away, you schedule the release of elementary stream packets based on the DTS of each PES packet and the current value of the internal PCR.

If you try to read the part of the MPEG-2 Systems specification (ISO/IEC 13818-1) that describes the T-STD buffering, it's horribly obtuse. However, it can be broken down into a couple of simple principles.

1) All the bits of a PES packet must arrive (be placed in the stream) before the PCR becomes equal to the DTS of the PES packet.

2) You can't send the entire PES packet in one burst. You have to space each 188 byte portion of the PES packet based on bitrate. For example, let's say that you have a 10 Mbps Transport Stream and a single 1 Mbps elementary stream. The spacing in the resulting Transport Stream should be very close to 1 elementary stream TS packet followed by 9 stuffing TS packets. This the the "leak rate" that I've mentioned in previous posts. The elementary stream has to "leak out" at a rate that doesn't overflow one of the T-STD buffers. BTW, the leak rate has to account for PES and TS packet overhead, so it's always somewhat higher than the elementary stream bitrate.

Ron

[Phoil]

Ah, you're right, I do have it backwards. I've been inserting the PCRs into the stream based on the AV timestamps, rather than inserting the AV packets into a proper PCR timebase.

It'll take some work to rework it so it starts with a fixed-rate stuffing+PCR base. For the audio packets this should be pretty straightforward, but the video I'm using is variable bitrate (constant ratefactor), so calculating the leak rate may be problematic. Any suggestions on that?

[drmpeg]

Ah, you're right, I do have it backwards. I've been inserting the PCRs into the stream based on the AV timestamps, rather than inserting the AV packets into a proper PCR timebase.

It'll take some work to rework it so it starts with a fixed-rate stuffing+PCR base. For the audio packets this should be pretty straightforward, but the video I'm using is variable bitrate (constant ratefactor), so calculating the leak rate may be problematic. Any suggestions on that?

The leak rate for VBR video streams is equal to the peak bitrate. Of course, defining a peak bitrate can be a little difficult. In the broadcast environment, VBR video streams usually have the peak bitrate capped at some sane value.

The good news is that there's some leeway in the T-STD buffering, so you don't have to have ultra precise leak rates. Also, once you get your packet scheduler running, you'll know when the leak rate is too low because the last PES packet will still be leaking out when it's time to release the next PES packet.

Ron

[Phoil]

Refactoring my code as a packet-scheduler will take some work, but I can see it's definitely the right way to go, especially since I want it to create both .ts and .m2ts streams correctly.

I'll post back when I get this closer to complete.

Thank you again, Ron. :)

[drmpeg]

I'll post back when I get this closer to complete.


Sounds good. I'll keep my eye on this thread. In the meantime, here's some info that will hopefully tie a few things together for you.

The release time for CBR streams (like AC3 and LPCM audio) is easy to calculate. It will be equal to the DTS of the PES packet (for audio, DTS = PTS) minus some fixed offset.

PES packet release time = PES packet DTS - offset

offset = ((((T-STD elementary stream buffer size (in bytes) * 8) / 2) * 90000) / bitrate)

For AC3 streams, the T-STD buffer size is 3584 bytes (unless it's an ATSC TS, in which case it's 2592 bytes). For Blu-Ray LPCM streams, the T-STD buffer size is much larger. 536832 bytes for 48 and 96 kHz LPCM and 1073664 bytes for 192 kHz LPCM.

Here's what an AC3 stream looks like on the TS analyzer. The muxer was using the ATSC buffer size of 2592 bytes and employs a small fullness offset (which is entirely optional). The fullness offset just means that the PES packets are being released a little earlier than the equation shown above.

http://www.w6rz.net/fullness.png

Ron