Fixing sync issues in broadcast recordings?

[CodeOptimist]

I'm currently using a number of tools to convert HD recordings from my MythTV box and have run into an occasional problem. Currently, I use comskip to demux the video (.mpg files, MPEG2-TS format), then Cuttermaran to cut commercials out of the elementary streams, DGIndex and AVISynth to scale and process the video, and finally VirtualDubMod to encode to 720p XviD.

The process works great 99% of the time (and the output looks quite nice), but once in a while the original file has a second or two of 'glitchiness' (for lack of a better term), most likely some random signal strength dip. Sometimes it's fine (the video just skips ahead slightly), but the audio doesn't always line up after that point... usually it ends up being delayed enough to be noticeable in the output file. (I'm guessing the video stream ends up being slightly shorter than the audio stream)

Any ideas on how to fix or get around an issue like that?

[rica]

We have been discussing a similar issue on this topic:

http://forum.doom9.org/showthread.php?t=142026



_ _ _ _ _

[lchiu7]

I looked at that thread but it seems that the sync loss is caused by the editing process. I don't get that - this is my situation which is more similar to the question the OP raised

I have a similar but different problem with DVB-T broadcasts. In my neck of the woods the DVB-T broadcasts are H.264 and often (AAC HE LATM) and AC-3. I am not interested in the AAC streams.

I can cut the commercials out with tspacket editor and create a file with no ads. This plays fine on my Popcorn Hour (in sync) and with PowerDVD8.

But if I want to compress the video (a 42 minute show in HD is 3G) I have to demux the audio, compress the video (using MeGui) and then mux the video back with the original AC-3 stream. I have tried MeGui (x264) to create both a MP4 stream and so mux back to MKV or h.264 stream (AVC) so mux back to ts format.

Therein lies the problem.

While the process work okay some of the time, occasionally because of transmission issues the AC-3 stream will have CRC errors in it. Both MKVMerge and Txremuxer complain about these errors and indicate sync will be lost. And it is.

I have tried the following methods to fix the AC3 streams before muxing them back

1. Delaycut with all options (fix, silence and correct). No joy - the resultant stream goes out of sync where the CRC errors occurred

2. Fix the AC-3 stream with ac3fix - again no joy.

The ts file (or MKV file) starts in sync and then goes out at the glitch point indicating the fix programs are not doing their job.

About the only thing that works is identify where in the stream the CRC errors occur(Delaycut indicates the frame numbers) and using tspacket editor (sure H264cutter would be fine too) actually remove a few seconds of the ts file that has the glitch. It's a bit severe but it works.

Wonder what other solutions are out there?

Thanks

Larry

[poisondeathray]

Ichiu7 - did you try vdub + directshow input driver method? (activate the plugin by using the drop-down menu when opening the file)

Have a look here http://forum.doom9.org/showthread.php?t=140927 , that sample stream there was full of errors. When transcoding, most encoders will drop frames on errors and all other methods seemed to fail as you described; but that method seemed to preserve the broadcasted glitches, so everything ended up in sync.

[lchiu7]

No haven't tried that. But have found something that seems to work (needs more testing). This is demux the AC3 stream, pass it through besweet to produce a wav file with 5.1 audio and re-encode that as AC3 again using Besweet. While the decode/encode cycle might cause some quality loss when I muxed it back with the original H.264 part of the ts file, there were no errors and it stayed in sync.

Next step is to see if I can mux it back with a compressed version of the video file and keep sync. A bit labour intensive but it only needs to be done when I encounter CRC errors and that's not all the time.

Larry

[edit]

Spoke too soon. Problem still occurs and it might also explain by using delaycut didn't fix it.

Read someplace the ts files maintain the PTS (time stamps) so that even if audio drops out for a bit, the timestamps ensure sync remains.

What I had been doing was demuxing the video which is h.264, compressing it to MP4 using x264.exe and then muxing it back with the original audio (and various variations on it). I could easily where sync was lost since there is an obvious glitch in the picture.

However if the fixed audio (as above) is muxed back with the original (h.264) video, it stays in sync across the glitch. This leads me to think that the timestamps in the ts file keep sync but once the video is compressed, the timestamps are lost. I can check this by trying to mux back the various files I passed through delaycut.

So far the only way I know around this is to cut the offending section out which is okay if it's only a second or two.

[rebkell]

I'm running into the same kind of problems with the Hauppauge HDPVR recorder. It has intermittent dropouts and the video gets out of sync. I've been trying to figure out how to fix it, and I've got something that is sort of working for me, but I'm not intelligent enough to figure out the PTS, DTS, PCR stamps enough to get it down pat. Basically all I'm doing is reading in the stream and adding the time of the video frames and audio units and when the difference in the times are farther apart than an audio unit, I add an audio unit, it chirps when you do that at that point, but it seems to keep the sync pretty close.

What we need is an application that will read in the ts stream and add or drop audio frames when needed to keep the sync within reason. When we demux we have lost the glue that the PTS,DTS stamps use to keep our streams in sync.

The above is only my very very limited opinion, but all the great tools seem to be based on nice source material, whereas we both are dealing with streams that go corrupt at random intervals.

Edit: eac3to will actually do the above to an extent, but it still will try to correct the mkv video file to line up with the audio file, it's close but not quite what we need.

[lchiu7]

Agree with you. My problem is the recordings are in sync in their original form, despite frame drops etc. since the timestamps seem to maintain sync. But as you say, once you demux and compress, all bets are off. eac3to didn't work for me since it didn't recognise the ts streams at all, perhaps because they contain h.264 video and two audio tracks viz. AC3 and AAC-HE.

Out of interest what tool are you using calculate the PTS DTS stamps and then adding an audio unit? I am using TS Packet Editor which finds the errors but have no tool to correct them. So these days I am just removing the offending parts which so far have only been a second or two so it doesn't impact the final video too much.

[rebkell]

Agree with you. My problem is the recordings are in sync in their original form, despite frame drops etc. since the timestamps seem to maintain sync. But as you say, once you demux and compress, all bets are off. eac3to didn't work for me since it didn't recognise the ts streams at all, perhaps because they contain h.264 video and two audio tracks viz. AC3 and AAC-HE.

Out of interest what tool are you using calculate the PTS DTS stamps and then adding an audio unit? I am using TS Packet Editor which finds the errors but have no tool to correct them. So these days I am just removing the offending parts which so far have only been a second or two so it doesn't impact the final video too much.

It's just some c# code I cobbled up and modified and made it work for me, I used this open source application found here:
http://www.codeproject.com/KB/cs/transport_stream_analyzer.aspx

I just hacked it up and used it, and all I really do is look for the start of 0x1011 packets(which are the video in all my captured h264 ts files) and 0x1100 the audio, I just keep a running total of the time adding up all the video times and audio times and when they get off by more than 32 ms(the time of an ac3 48 KHz audio frame) I just add an extra audio set of packets. I'm basically just writing the same ts file back out with added audio packets here and there. Then I demux the audio from this rewritten ts file and it will come pretty close to being in sync no with the video after I reencode it. I always encode the entire capture before trying to edit anything. I used DGAVCDec for demuxing and as a frame server for encoding.

[lchiu7]

That is a bit hard for me! While I have been a programmer in previous lifes, not sure I would have the time to learn to do this.

For me it was more of an academic exercise than something that had to be done. The broadcast videos play in sync even if there are frame drops (presumably because of the timestamps). And after cutting out ads with TS Packet Editor they remain in sync. The issue I was trying to manage was, 42 minutes of a show is about 3G and it easy to compress down to about 1G and maintain quality (but lose sync!).

Have decided to leave the files as they are. But what I have managed to do is run TS Packet Editor on the files and it identifies where the files are broken. If the segments are small I just cut them out (1-2s) and then the files process okay.

[rebkell]

That is a bit hard for me! While I have been a programmer in previous lifes, not sure I would have the time to learn to do this.

For me it was more of an academic exercise than something that had to be done. The broadcast videos play in sync even if there are frame drops (presumably because of the timestamps). And after cutting out ads with TS Packet Editor they remain in sync. The issue I was trying to manage was, 42 minutes of a show is about 3G and it easy to compress down to about 1G and maintain quality (but lose sync!).

Have decided to leave the files as they are. But what I have managed to do is run TS Packet Editor on the files and it identifies where the files are broken. If the segments are small I just cut them out (1-2s) and then the files process okay.

I understand, it's just a hobby for me, I used to be a programmer in previous lifes also, I only recently started tinkering with c#, and just wanted to try and understand the ts file structures and how things worked. If no one comes up with a solution, I'll continue to work on this solution.

[lchiu7]

I understand, it's just a hobby for me, I used to be a programmer in previous lifes also, I only recently started tinkering with c#, and just wanted to try and understand the ts file structures and how things worked. If no one comes up with a solution, I'll continue to work on this solution.

Well when you get it finished I would be interested in a copy:thanks:

Event based programming in Windows always seemed to hard to learn. But just using C# and possibly .Net in a command line application might not be so hard to work out.

[Turtleggjp]

This has also been a problem for me lately. Normally I use ProjectX if my capture has glitches in it, but that has become less and less reliable now (it doesn't seem to like it when they switch between hard telecined 29.97 fps and 23.976 fps with pulldown flags). Plus, when I did Amazing Race last week (no pulldown flages there) ProjectX failed to keep it in sync, which I thought was odd.

About 2 years ago, I dusted off my C++ compiler and fixed the .vcf parsing code in AC3 cutter (http://forum.doom9.org/showthread.php?t=92325), and have been using that ever since to do my frame accurate (not just GOP accurate) edited encodes of my captured TV shows. I was thinking of trying to do something like that again, but have it use the original transport stream as a source and pull out all the audio frames needed based on the timestamp codes in the .ts file. By doing this, it should be able to drop or add frames as needed to maintain sync with the video. This is all just based on some ideas I have, I haven't actually started to code anything yet. I'm surprised that there aren't more tools to work with this kind of thing yet, I guess around here people are more interested in their perfect streams from discs...

Matt

[rebkell]

This has also been a problem for me lately. Normally I use ProjectX if my capture has glitches in it, but that has become less and less reliable now (it doesn't seem to like it when they switch between hard telecined 29.97 fps and 23.976 fps with pulldown flags). Plus, when I did Amazing Race last week (no pulldown flages there) ProjectX failed to keep it in sync, which I thought was odd.

About 2 years ago, I dusted off my C++ compiler and fixed the .vcf parsing code in AC3 cutter (http://forum.doom9.org/showthread.php?t=92325), and have been using that ever since to do my frame accurate (not just GOP accurate) edited encodes of my captured TV shows. I was thinking of trying to do something like that again, but have it use the original transport stream as a source and pull out all the audio frames needed based on the timestamp codes in the .ts file. By doing this, it should be able to drop or add frames as needed to maintain sync with the video. This is all just based on some ideas I have, I haven't actually started to code anything yet. I'm surprised that there aren't more tools to work with this kind of thing yet, I guess around here people are more interested in their perfect streams from discs...

Matt

Evidently, I hope you do the project, I've been having more and more trouble with my captures lately and surprisingly, since I'm a total dunce when it comes to the DTS, PTS PCR_PID stuff, I actually just add up the times of the video frames and audio frames and when the video time is more than 24 ms ahead of the audio( I check this when I receive an audio start packet), then I just forecefully shove an extra audio frame in the stream and it has actually worked very well. I'm sure someone that can actually interpret the timestamps, could nail it to the wall.

[Turtleggjp]

Have you tried indexing your file with DGIndex? I did that today with one of my captures, and used the option of logging timestamps. This gave me a bunch of Video PTS, Audio PTS, and PCR values to mess with for now. It doesn't look like DGAVCIndex has this option, but hopefully I can figure out how DGIndex is getting these values and apply it to AVC videos as well someday...

[rebkell]

Have you tried indexing your file with DGIndex? I did that today with one of my captures, and used the option of logging timestamps. This gave me a bunch of Video PTS, Audio PTS, and PCR values to mess with for now. It doesn't look like DGAVCIndex has this option, but hopefully I can figure out how DGIndex is getting these values and apply it to AVC videos as well someday...

xport will dump them for you, drmpeg wrote it, let me dig up a link for you.

http://www.w6rz.net/xport.zip it has the source code included, but I'm just not comfortable with c code enough to get a good handle on it, it has some of the deepest nested if ifelse statements I've ever seen.