A utility to merge .m2ts files keeping audio in sync: SyncMergeM2ts

[Mutantape]

Purpose :
=======

Your camcorder creates one new file each time you shoot a scene. When you want to merge all of them
into one file using tsmuxer, for example, audio drifts relative to video, and the drift increases with the number of clips involved.This is due to the fact that the soundtrack in each scene is slightly longer than the video. When concatenating files, the extra sound bits add up and cumulated drift becomes visible.

This utility trims sound down to the video length by cutting out AC3 frames from the digital stream
there is no recoding, just operation in the compressed domain. I hesitate to label it as lossless,
since obviously some sound frames are removed. But what remains of the sound is not modified.

Sometimes it may be needed to add some silent frames rather than cutting.

Sound is adjusted to +/- 16 ms of the video, since an AC3 frame is 32 ms. The program design avoids
accumulation of errors so it should in principle work for 60 fps devices, with their strange non
integer frame duration, but was not tested in this case.

The delay given in the container is applied to the sound and is set to zero in the final merged file.

A mix of clips with different delays is possible.

Details:
======
This is a DOS utility, with no GUI. I've developed it for my own use, and brought it to the point where it was good enough for me. I do not plan to do much maintenance on it and just put it here in the hope it will be useful to someone as it is.

User manual included in the bundle.

The OS i use is Windows XP.

The camcorder is a device with 50P/60P capability, later expanded to 3D and from a well known japanese brand (no, not them, but the one who bought them...). I am giving no names since some people are currently swamping the web with ads for a payware that handles its files, and i don't want to be caught up in searches with them...

It is a standard-shaped camcorder with 3 variants depending on whether there is a hard disk, or additional internal memory, or just a memory card. Enough said, i guess.

The files it produces use m2ts (AVCHD) container with AVC/H.264 encoded Video (up to 28.8 Mbps), and AC3 sound.
The .m2ts i create with avisynth scripts do not behave exactly like those of the camcorder (mediainfo does not see the same number of frames), this points to different flavors of .m2ts files so i am not sure it will work with other camcorders.

I work with 50P and rarely 25I. I've not tested 60P. The files produced always have negative delays.

There is code to handle positive ones, but i've not tested it.

The code i wrote uses resources from Ac3Cutter (and is embeded in its code). It is a small layer on top of tsmuxer and mediainfo. I add those last two in the bundle for convenience, though it increases the size by
a large relative amount.

The code is designed to work around the bugs i found in those versions, so it may not be a good idea to replace them with newer releases if any is available.

I've scanned the files with a free antivir version.
Feel free to re-scan or recompile from sources if you are the paranoïd type.

This link will have a limited lifetime if nobody uses it...
http://dl.free.fr/nWdDLd12i

It is a french website, but you just have to know that "Télécharger ce fichier" means "Download this file". No password, click on that and it should start (4 MB).

[Groucho2004]

This is a DOS utility

And yet the PE header of the main executable reads "This program cannot be run in DOS mode." :rolleyes:

[Mutantape]

Then perhaps i should have just said a command line utility for the DOS emulation of Windows XP ,as it is the only way i have tested it. Possibly it would fail under real DOS.

I have no idea where this warning comes from. Something the compiler puts in by default, maybe. Anyway, it is not a problem under the conditions of use i have described.

[Midzuki]

Then perhaps i should have just said a command line utility for the DOS emulation of Windows XP ,as it is the only way i have tested it. Possibly it would fail under real DOS.

Blame it on Microsoft themselves ;) Starting with Windows 4 (aka Windows 95), the CLI interface became something far greater than the "bootloading" 16-bit operating system. Notwithstanding, one of the shortcuts in the Start Menu insisted on saying "MS-DOS prompt" :rolleyes:

Anyway, your application has a point — it doesn't require the pesky dot net framework :D

[Groucho2004]

I have no idea where this warning comes from. Something the compiler puts in by default, maybe.

It's not a warning, it's part of the exe-header for Win32 binaries. I was just being a bit sarcastic. :D

[Mutantape]

Anyway, your application has a point — it doesn't require the pesky dot net framework :D

Yes, at least i've got this one right ;)

I've done some editing to put back the purpose description where it belongs.

If anyone has a chance to test that with 60P material, i 'd be interested to know it it works, just out of curiosity.

[Ghitulescu]

How are treated the files where constantly the sound is longer than the video (a delay within the duration of a frame)?
Do you shift the audio of the second file with the remaining delay and so on until a full frame is obtained, then drop the extra audio frame (the last audio frame from that n+1 file)? I assume leaving "empty" spaces in audio will result in a "click".
And for the opposite, do you drop video frames (GOPs)?

[tebasuna51]

eac3to can manage audio gaps/overlaps between m2ts files.

[Mutantape]

ffmpeg.exe -i concat:"%mts1%|%mts2%" -vcodec copy -acodec copy -sn "%join_mkv%"

Will that really be equivalent ? I guess the "copy" statements means there is no recoding. But will that really handle the difference in length between sound and video so that there is no sync drift over a large number of files ? Will that really take all the files in a given directory and add them just once? Also, this will not allow to concatenate with an ordering that is different from the one derived from the file names. In that respect, it would be the same as the most basic usage of my program.

[Mutantape]

How are treated the files where constantly the sound is longer than the video (a delay within the duration of a frame)?
Do you shift the audio of the second file with the remaining delay and so on until a full frame is obtained, then drop the extra audio frame (the last audio frame from that n+1 file)? I assume leaving "empty" spaces in audio will result in a "click".
And for the opposite, do you drop video frames (GOPs)?

I am not sure what you mean by : a delay within the duration of a frame?

I never drop video frames, the final file is the simple concatenation of all the video frames of the individual clips.

I am considering a collection of clips where the sound track is always longer then the video track inside each individual clip. The difference in length will usually be larger than the duration of a frame.

For the first clip, i remove the number of audio frames needed to be as close as possible to the (say negative) delay specified in the container.I remove frames at the end of the clip to be as close as possible to the point where the soundtrack of the second clip should begin, considering its specified delay.

For each following clip one considers the accumulated duration of video, and the accumulated length of audio (number of frames divided by framerate). As the first video frame of that clip will come after the already concatenated ones, one decides what needs to be done for the junction between the two soundtracks to be as close as possible to the position that the (negatively) delayed sound of the second clip should have relative to the first frame of that second clip : remove some audio frames or add silent frames. Around the calculated point, choosing one audio frame or the next will produce an error that is either superior or inferior to half the duration of a frame. Of course, the case that produces the smaller error is selected.

So the code does not accumulate error until it is time to drop a frame, though its behavior may be close to that nonetheless as a side effect. Existing audio frames are dropped or silent frames added at the beginning and end of a clip soundtrack so that it occupies its intended position in the cumulated audio/video sequence with an error of half an audio frame duration at most. the accumulation is not on error but on the previous clips video and audio tracks duration.

Note that the sound of a clip will start to play a the end of the previous one, the overlap being approximately the value of the negative delay.
I could have put silence instead, but this keeps more of the original sound and did not produce annoying problems in my tests.

For clips with the same delay, adding silent frames is usually not needed, it is more likely to happen when the delay changes from one clip to the next.

[Mutantape]

eac3to can manage audio gaps/overlaps between m2ts files.

All the tutorials i came across about eact3to took for granted that one started from a blu-ray containing a variety of info files describing the .m2ts clips, that eac3to would use to trim things correctly.

Not so here for the camcorder files. You have a collection of .m2ts clips, and nothing else.

Can you describe exactly what simple command line you would use to concatenate my camcorder files keeping sound in sync despite the difference in audio/video length ?

Furthermore, i can easily reorder files without scripting (just by moving lines in a text editor). And there are the little extras for avisynth shifts pre-compensation, which are useful in my environment (but of course this was taylored to my own needs).

[Ghitulescu]

I am not sure what you mean by : a delay within the duration of a frame?

I have to draw on paper, but unfortunately my internet PC doesn't have any graph tool.

But it appears that what you do is more or less correct.

[Mutantape]

Finally i found the information to use eac3to in a simple way to do the same basic thing (as was suggested in a previous post).
Probably, it is so simple that no one bothered to explain it (but it is available in the help if you execute it with no argument).

The syntax is just as follows:

eac3to file1.m2ts+file2.m2ts+....+filen.m2ts result.mkv

And you get an mkv file with sound in sync. Still possible to turn it to m2ts with tsmuxer if needed, but i have not tried it so cannot guarantee there is no side effect.

The latest versions of mkvmerge/mkvmerge Gui (5 something) can also take m2ts files as input, but with my 50P files, or even 25i, the result did not play well (not smooth and with variable speed).

Syncmergem2ts will still be useful to me in case i have to handle files with mixed formats, in preparation for avisynth processing.
Perhaps someday i will try to modify it to use eac3to instead of mediainfo + tsmuxer, if only to simplify the process of producing the files list.

I explored this route because there are sometimes visual glitches at the junction between clips (as if one frame was not in the correct place).
I thought this might be a bug in tsmuxer so i tried to use something else. Unfortunately, it is exactly the same with eac3to. There must be a problem with the camcorder files themselves.
I am thinking of cutting a frame or two at the beginning or/and the end of the clips, but i've yet to find a good free tool for "lossless" frame accurate trimming.

It was not a big problem for me initially because i mainly used this as a preparation for avisynth, which tends to remove the problem (perhaps because a frame is missing when importing video files).
But now, most of my electronics have been upgraded to 50p capability , and i am more inclined to work with the original files : the glitch is more annoying.

In the mean time, the eac3to process i use is roughly, under windows command line window ("DOS"):

cd x:\SourceFilePath
dir /B *.m2ts > Y:\DestinationPath\list.txt
reorder,trim,... list.txt with notepad
with an hex editor (or one that shows control codes), replace all '0D 0A' end of line sequences by '+'
at the beginning of the line, add : "C:\PathToEac3to\eac3to.exe"
At the end of the line remove the last '+' and add Y:\DestinationPath\Merged.mkv
Copy the whole line thus built from list.txt (the usual CTRL A CTRL C)
Paste it in the command line windows (still pointing to x:\SourceFilePath) and Enter

None of the video files i use have spaces in their names, it will probably be a problem if yours do.