DVB-S2 - TS file plays fine, jerky when muxed to MKV

[DragonQ]

Hi,

First of all, sorry if this is in the wrong forum...there's so many of them it's a bit confusing! :)

I have a TS file that was recorded from a DVB-S2 stream by MediaPortal. It contains a ~9 Mbps 1440x1080i25 AVC video stream and a 192 kbps AC3 audio stream. This TS file plays fine using a number of splitters and decoders (LAV, ffdshow, MPC-HC, Haali).

I want to convert it to MKV in order to save disk space but so far I've been unsuccessful. I've been using MKVMerge and I've also tried running the TS file through TS Doctor before remuxing - either way, the resulting MKV file keeps playing in slow motion, then swiftly catching up, then playing in slow motion again. Since the problem exists with multiple splitters and decoders, and the slow motion bits are always in the same places, I can only assume there's some damage in the TS file or something?

Is there anything I can do to get this file working in an MKV container? I've attached a sample below in both formats:

TS (http://www.mediafire.com/?g4ablqrna06bfvz)
MKV (http://www.mediafire.com/?bvxon64yb8w5aey)

Thanks!

[CruNcher]

the behavior is indeed very strange does MKVmerge support direct .ts to .mkv now ? if yes tryto demux the bitstream from the .ts (directly without remuxing with mkvmerge) and mux them then together seems the timestamps get completely messed up, try also ffmpegs muxer,hallis,solveig to see if they endup with the same result. You could also try a newer version then mkvmerge v5.1.0 ('And so it goes') built on Nov 28 2011 23:58:28 to see if its a bug

original (Mediaportal .ts)

http://img829.imageshack.us/img829/820/originalu.png

mkvmerge sample (Mediaportal DragonQ .mkv)

http://img233.imageshack.us/img233/8554/remuxed.png


very odd

[DragonQ]

Yeah I've tried using MKVMerge to go straight from the TS to MKV and also demuxing using TSMuxer first, then muxing with MKVMerge. I get the same result either way. 5.1.0 was the latest version of MKVMerge when I was trying this but I just tried 5.2.0 and get the same result. :(

I'm not familiar with those other muxers (at least in the context of creating MKVs from elementary streams). Do you have links?

[CruNcher]

DragonQ does this work for you http://www.mediafire.com/?qao5cljkmn73b9x
interoperability should be good though mpc-hcs native matroska splitter still fails :(

http://img543.imageshack.us/img543/1108/mpchchalli.png

[DragonQ]

CruNcher, yes that file works fine for me. How did you make it?

[CruNcher]

using dshow (same as mediaportal) and as input mpc-hc mpeg splitter and output hallis matroska muxer see graph above though it fails with mpc-hcs own parser/splitter also solveigs splitter fails best results are with halli,lavsplitter,potplayer,mirillis splash and av splitter also im not sure if this will work fine with hardware parsers though i dont see why it shouldnt (most seem anyway based on libmatroska) the interesting question though is why mkvmerge fails @ remux (failing with the timestamps ?) and why is mpc-hc also failing @ playback of the dshow remux (it seems to be a timestamp issue) :(

[DragonQ]

Yay, apparently this has been fixed in MKVMerge build 414. Will download the next version when it comes out and test it out. If it works, I'll definitely remux most of my larger TS recordings into MKV because they take up less space!

[CruNcher]

sure it's the best you can do and if you need editing in applications that doesn't support .mkv you just mux it back that's how i work out my framework until everything supports .mkv ;)

[Mosu]

It's out now.

[CruNcher]

Great will do a very huge .ts to .mkv interop test :)

[DragonQ]

Hmm I can't seem to get a specific recording from ITV1 HD to mux to MKV correctly with v5.4.0. All of my other recordings from BBC HD and BBC One HD work fine. However, Mosu says it works for him. I've double-checked my MKV Merge GUI version and settings, plus I'm using the same filter/decoder/renderer/player chain as him (LAV + MadVR + MPC-HC). Can anyone else test if they have the same issue playing back my MKV, or playing back an MKV muxed from the original TS please? The files are below:

Working TS: http://www.mediafire.com/download.php?w29f1b5s7gih5x1
Broken MKV: http://www.mediafire.com/download.php?0ropww86al65na2

Thanks!

[midnightsun]

Hi DragonQ, I tested your TS file: it fails mmg 5.3.0 (as do other AVC 1080i's that I have recorded; with those gdsmux has worked fine for me), but the latest build produces a correct mux on my end

[Superb]

Hmm I can't seem to get a specific recording from ITV1 HD to mux to MKV correctly with v5.4.0. All of my other recordings from BBC HD and BBC One HD work fine. However, Mosu says it works for him. I've double-checked my MKV Merge GUI version and settings, plus I'm using the same filter/decoder/renderer/player chain as him (LAV + MadVR + MPC-HC). Can anyone else test if they have the same issue playing back my MKV, or playing back an MKV muxed from the original TS please? The files are below:

Working TS: http://www.mediafire.com/download.php?w29f1b5s7gih5x1
Broken MKV: http://www.mediafire.com/download.php?0ropww86al65na2

Thanks!
Your mkv file says:
General
Unique ID : 207125762327212234845063550589170598820 (0x9BD2F885D3183D4A8D266C73C41BC3A4)
Complete name : C:\Users\Superb\Desktop\England 2 - 3 The Netherlands (Friendly, 2012-02-29) [1080i] SAMPLE.mkv
Format : Matroska
Format version : Version 2
File size : 36.3 MiB
Duration : 30s 800ms
Overall bit rate : 9 878 Kbps
Encoded date : UTC 2012-03-10 13:41:56
Writing application : mkvmerge v5.3.0 ('I could have danced') built on Mar 8 2012 09:15:42
Writing library : libebml v1.2.3 + libmatroska v1.3.0
v5.3.0...

[Mosu]

If you want to actually test something useful, like DragonQ asked, then download his original TS file (not his MKV), mux it with the _current_ MKVToolNix, meaning v5.4.0, and "just let mkvmerge do its job" -- meaning do not try to force the default duration/FPS to anything. Just leave it blank.

Then play the resulting MKV.

Thanks.

[sneaker_ger]

If you want to actually test something useful, like DragonQ asked, then download his original TS file (not his MKV), mux it with the _current_ MKVToolNix, meaning v5.4.0, and "just let mkvmerge do its job" -- meaning do not try to force the default duration/FPS to anything. Just leave it blank.

Then play the resulting MKV.

Results in the slow motion (and subsequent frame drops if renderer can't keep up) described in the first post.

[Mosu]

Let me copy what I've written in the corresponding bug report (https://trac.bunkus.org/ticket/688):

I've tried your England file here with 5.4.0 in three versions: once without --default-duration, once with 25p, once with 50i. I've tried playing all those files and the original one on two players: 1) MPC-HC with LAVFilters and MadVR, 2) VLC. I've even tried muxing on both Linux and Windows (this is in addition to what I've written in said bug report).

All of those cases played perfectly fine. Therefore I consider this a problem on your the playback end and will not investigate this case further.

[sneaker_ger]

Did you mux with or without the audio?

[Mosu]

No, I muxed with audio each time and played back with audio each time.

[sneaker_ger]

Very very strange.
No audio = perfect playback
LAV Audio = perfect playback
MPC Audio Decoder = video slow downs :confused:

[Mosu]

I'm indeed using LAV Audio. Playback issue -> not my problem, but MPC's :)

[sneaker_ger]

Demuxing AC3 with eac3to, then use the raw for muxing = also good playback with MPC Audio Decoder.

While there should never be any video playback problems from wrong audio muxing and LAV Audio plays the file fine, I don't want to jump to the conclusion that mkvmerge does everything correctly just yet. Would be interesting what madshi or nev have to say about this.

[Mosu]

Be that as it may, I still don't consider it my problem at this point. I have no means to debug such playback problems -- I've never seen MPC's source code from the inside, I cannot get useful information why it plays what when and why it slows down from it etc etc. If one of you cares enough then you can ask the appropriate people to come up with specific information about what makes the file stutter. Maybe the authors of the programs that make it actually work are not the best people to ask -- if Nev sees that the file plays just fine with his filters then why would he investigate in the first place?

When you (whoever that is, sneaker_ger or dragonq or someone else) have that information then I'm willing to look at mkvmerge's side again.

I'm not trying to be difficult here, I'm not trying to deny that there may be a bug in mkvmerge's AC3 handling. This is about a realistic estimation if I can do anything about it at all and if I could how long it would take me to debug it and if that would be worth the time spent. As playback works in two popular (and completely different) playback chains it is not worth my time at the moment.

[sneaker_ger]

Yes, I understand.
With "interesting" I didn't want to limit it to "what's different about this file that makes these playback errors appear", but a more general "how can the video chain be affected by the audio part of it in the first place".

[nevcairiel]

LAV Audio has some pretty smart code to try to deal with any sort of audio timestamp problems in the source file.
I can try to look at the file and see if something obvious shows up that would cause such problems.

PS:
In DirectShow, the audio renderer defines the reference clock, if the audio timestamps jump all over the place, it will also affect the video playback.

[sneaker_ger]

That would be great.
I have already uploaded the remuxed version:
http://www.mediafire.com/?8h0960iamj3krom

I created it for the MPC-HC thread, but I'll hold back on posting there until you have taken a look at the file.

[sneaker_ger]

PS:
In DirectShow, the audio renderer defines the reference clock, if the audio timestamps jump all over the place, it will also affect the video playback.

Ah, OK.

There are indeed many timecode deltas != 32ms in the remuxed version.
Question is: is the TS file broken or just mkvmerge's remuxing?

[nevcairiel]

I looked over the timstamps, and this is the result:


rtStart(calc): 2580000, rtStart(input): 2580000, sample duration: 320000, diff: 0
rtStart(calc): 2900000, rtStart(input): 2900000, sample duration: 320000, diff: 0
rtStart(calc): 3220000, rtStart(input): 3220000, sample duration: 320000, diff: 0
rtStart(calc): 3540000, rtStart(input): 3540000, sample duration: 320000, diff: 0
rtStart(calc): 3860000, rtStart(input): 3860000, sample duration: 320000, diff: 0
rtStart(calc): 4180000, rtStart(input): 1760000, sample duration: 320000, diff: 2420000
rtStart(calc): 4500000, rtStart(input): 2080000, sample duration: 320000, diff: 2420000
rtStart(calc): 4820000, rtStart(input): 2400000, sample duration: 320000, diff: 2420000
rtStart(calc): 5140000, rtStart(input): 2720000, sample duration: 320000, diff: 2420000
rtStart(calc): 5460000, rtStart(input): 3040000, sample duration: 320000, diff: 2420000
rtStart(calc): 5780000, rtStart(input): 3360000, sample duration: 320000, diff: 2420000
rtStart(calc): 6100000, rtStart(input): 3680000, sample duration: 320000, diff: 2420000
rtStart(calc): 6420000, rtStart(input): 6420000, sample duration: 320000, diff: 0
rtStart(calc): 6740000, rtStart(input): 6740000, sample duration: 320000, diff: 0
rtStart(calc): 7060000, rtStart(input): 7060000, sample duration: 320000, diff: 0
rtStart(calc): 7380000, rtStart(input): 7380000, sample duration: 320000, diff: 0


The first value is the calculated start time by LAV Audio, sequentially increasing by 32ms per audio frame.
The second value is what is in the file. You can see a difference between the calculated value and the actual value of 242ms for 7 audio samples, then it goes back to normal.

After that, its fine for a while, and then the same happens, again exactly 242ms offset.
Because its only for a very short period, LAV Audio ignores the jump and does not adjust its calculated value. If it would go on for a longer time, LAV Audio might think there was a gap or overlap in the audio and adjust its timings.

I looked at the TS file, and it does not show the same problem. Something seems to be going wrong in the remuxing process.

[Mosu]

Thanks for the insight, I'll investigate.

[Mosu]

Found the issue and hopefully implemented a fix. Can you please try playing this file: http://www.bunkus.org/misc/england-fix-try1.mkv

For me (MPC HC, MPC's audio decoder) this plays flawlessly while previous muxed versions didn't. Same with nev.

I'll provide a new MKVToolNix build once I get positive feedback on the file itself.

[sneaker_ger]

Yes, working fine.

[DragonQ]

Wow, lots of useful replies overnight, thanks everyone.

Just to clarify, the MKV I uploaded was with build 420 or something like that (pre-5.4.0), where the original fix was already implemented. I get the same issue when using 5.4.0. For what it's worth, I'm using ffdshow for my audio, not MPC's audio decoder. If I use LAV Audio, the MKV I uploaded in this thread plays fine.

Just tried the new sample posted by Mosu and it now works fine with ffdshow audio. :)

I am curious as to why this issue never appeared in the BBC HD & BBC One HD streams, when both those and the ITV1 HD streams use AC3 stereo audio? I know that TV TS streams can be fickle things though and people have had issues with ITV1 HD going out of sync (whilst other channels are fine) so I guess there's something weird going on!

[Mosu]

The problem with mkvmerge was how it calculates AC3 timecodes that are missing. mkvmerge is trying its best to keep the timecodes that the source container provides. MPEG TS is known to provide timecodes only for some of the packets, though -- this can happen both to video and audio tracks. What mkvmerge was doing wrong was that it calculated from zero and not from the previously available timecode.

Example: MPEG TS provides a timecode for the first AC3 packet, let's say it's 200ms. MPEG TS then does not provide timecodes for the next two packets. Now mkvmerge was assigning them the values 0ms + 1 * 32ms and 0 + 2 * 32ms instead of 200ms + 1 * 32ms and 200ms + 2 * 32ms.

This has been fixed in build 422 and higher at http://www.bunkus.org/videotools/mkvtoolnix/win32/pre/ (still building, upload should be done in a couple of minutes).

[DragonQ]

Thanks for the explanation and thanks for another quick fix. :)

[Mosu]

Thanks for providing the intel, your three.