How to restore the audio sync in a x264/AC3 MKV ?

[guista]

Hi all,

1/ I reencoded an initial MKV 10 Mbps x264 MKV to a 4 Mbps x264 MKV and the video was correctly reencoded.

2/ When trying to extract the AC3 audio (to remux it with reencoded video), eac3to (ran from MeGUI/HD streams extractor) stopped at 21% with a "bitstream parsing error".

3/ I decided to extract the audio AC3 track with mkvextract (this tool is provided with the MeGUI package) and it worked.

Unforunately, the audio is out of sync.
By playing the movie, I noticed that the delay between video and audio increase from the begining to the end.

In order to resync the audio with the video, I plan to find the time delta. Then, I am thinking of doing like this : find the video length, find the audio length, makes the difference and fill this value in the MeGUI MKV merge field (in the audio input section) then merge again.

The problem is : MediaInfo, the only available MKV information tool displays the tracks' length in minutes whereas I need them to be expressed in milliseconds (I think the delta is less than 1 second).

* Do you know a tool which could give me the tracks' length with a millisecond precision ?

Thanks in advance.

[zmaster]

...and the video was correctly reencoded.I think you made a mistake somewhere. ;)
You can show the MudiaInfo source and your AviSynth script?

[yetanotherid]

I find generally using the option for changing the frame rate is easier than trying to work out the time stretch required. Often a little change from (for example) 23.976 to 23.975 will fix the problem.

I often convert MKVs to standard definition AVIs for use with DVD players in the house. I extract the streams with MKVCleaver and use it to convert them to an AVI containing x264 which I then convert to XviD/AVI with AutoGK. Quite often I have to bump the frame rate of the AVI AutoGK produces a little, even though it's the same frame rate as the original. I've no idea why though.

Try using MKVmerge GUI, which comes with mkvtoolnix (http://www.bunkus.org/videotools/mkvtoolnix/). It'll probably make re-muxing easier, or at least give you more precise control (I'm not sure what you can do with the MeGUI muxer exactly).

"* Do you know a tool which could give me the tracks' length with a millisecond precision ?"

I use foobar2000 (http://www.foobar2000.org/) as an audio player. It'll also play the audio inside MKV and MP4 video files (as well as a few other formats). I think MKV support is native these days (no plugin required) and foobar2000 displays track length to a millisecond.
Keep in mind though it reads the information from the file header so if you're wanting to be certain you'd probably have to convert the audio to wave and then check the length of the wave file.
Foobar2000 is a nice program to get to know. If I'm converting audio independently of the video I generally use foobar2000. It'll convert to many formats (AC3, AAC, MP3 etc) whether natively or via a DirectShow plugin. It'll happily decode and encode 5.1 channel audio and there's a plugin for down-mixing 5.1 to stereo. I'd be lost without it.

While it's not popular around here I often use Super Video Converter to convert problem audio. Rather than time stretch it or change the frame rate, Super will convert the audio inside MKV files. I've found that if I select the DirectShow decode option for the audio, the output Super produces (from the original MKV) will then mux into the new MKV you encoded with MeGUI without sync issues. Well.... most of the time....

[guista]

I think you made a mistake somewhere. ;)
You can show the MudiaInfo source and your AviSynth script?

I do not think I made any mistake : I always use the MeGUI HD streams extractor and it always worked perfectly for me except for this movie.

I am pretty sure that the audio sync problem comes from the bitstream parsing error which occurs in eac3to (the original MKV may have a problem).
mkvextract is more permissive so it let me extract the whole track while eac3to stopped at 21%

The drawback is it produce a bad audio sync.

[zmaster]

How about a little test? Need filters dss2 (For dss2 work needed Haali package (http://haali.cs.msu.ru/mkv/). After installing the file avss.dll (.../HaaliMatroskaSplitter/avss.dll) must be copied to .../AviSynth 2.5/plugins.), ffms, software VirtualDub.

Extract the audio track with MKVextract. Then, create a semblance of this script:
LoadPlugin("C:\Program Files\megui\tools\ffms\ffms2.dll")
V = dss2("C:\mybigmkv.mkv", fps=23.976) #mybigmkv.mkv = 10 Mbps x264 MKV
A = FFaudioSource("C:\myac3.ac3")
AudioDub(A,V)
Open it in VirtualDub, mark and save a small piece of video.
Q1: Is there a desync?

Repeat the operation, replacing the ac3 file to mybigmkv.mkv
LoadPlugin("C:\Program Files\megui\tools\ffms\ffms2.dll")
V = dss2("C:\mybigmkv.mkv", fps=23.976)
A = FFaudioSource("C:\mybigmkv.mkv")
AudioDub(A,V)
Q2: Is there a desync?

[guista]

I find generally using the option for changing the frame rate is easier than trying to work out the time stretch required. Often a little change from (for example) 23.976 to 23.975 will fix the problem.

This is a good solution if the frame rate can be changed without reencoding the video each time (it took me 5 hours). If I understand well, you decelerate the video speed instead of accelerating the audio. I ask myself how you find out the right frame rate for resynchronizing the audio.

Try using MKVmerge GUI, which comes with mkvtoolnix. It'll probably make re-muxing easier, or at least give you more precise control (I'm not sure what you can do with the MeGUI muxer exactly).

MKVMerge is invoked by MeGUI so I am already using it.

I use foobar2000 as an audio player. I think MKV support is native these days (no plugin required) and foobar2000 displays track length to a millisecond.

While it's not popular around here I often use Super Video Converter to convert problem audio. Rather than time stretch it or change the frame rate, Super will convert the audio inside MKV files.

Thanks for the tool names. I am going to try Super Video Converter first then my method using foobar2000 and let you know if it worked for me.

[guista]

How about a little test?

Before doing your test, I mention that the original (big) MKV is perfectly synchronized (I play it with MPC) whereas my reencoded MKV is not.

[yetanotherid]

This is a good solution if the frame rate can be changed without reencoding the video each time (it took me 5 hours). If I understand well, you decelerate the video speed instead of accelerating the audio. I ask myself how you find out the right frame rate for resynchronizing the audio.

Yes it can be changed without re-encoding. You simply set a new frame rate and re-save it as a new file.
The only way to find out what frame rate to use... well the easiest.... is simply to guess, save the file, then play a section towards the end.

I don't really know why but at least 90% of the time when bumping into your problem I find a change from 23.976 to 23.975 does the job (or the other way around). Sometimes it's a little more, sometimes you have to go the other way, and sometimes if you're really fussy you'll find a value such as 23.9745 hits the spot.

I've successfully time stretched audio inside an MKV but getting the exact amount seems to be harder given you've got to know the exact length of both video and audio... but the fun part comes when you realise audio and video streams aren't always the same length in the first place, so an assumption that making them the same length will always put them in sync could result in you banging your head on your desk really hard trying to understand why it doesn't. ;)

I don't know why the sync issue occurs, I can only assume the original audio was time stretched inside the original MKV, but sometimes it doesn't make sense. I've literally converted the same audio and video to both MKV and AVI and while one of them will be perfectly in sync, the other loses sync even though it's the same frame rate so I've had to change it a little.
I've given up trying to work out why for fear of going mad.

My other "sync solving" method is to use AnyVideoConverter. For it to work you need to convert both video and audio although it only converts to stereo. It does however have two different sync settings and an option to boost it's sync fussiness level. I've used it to convert a file (just to low resolution and bitrate video) then extracted it's audio track to mux with my video file.
Or.... when every other method has failed, I've used it to convert both video and audio to full resolution, lossless formats, then used my regular conversion program to convert that monster file to AVI or MKV etc.

[yetanotherid]

I am going to try Super Video Converter first then my method using foobar2000 and let you know if it worked for me.

If you get a chance try Super twice, once with DirectShow encoding enabled and once with it disabled. You can force foobar2000 to use DirectShow for some formats but for some reason it doesn't change the result.
Often when there's a sync issue such as yours, Super will produce output audio of different durations depending on whether DirectShow encoding is enabled or not.

MeGUI has a DirectShow decode option I've not tried... but if it can't encode the audio without demuxing it first I guess it won't make a difference anyway.

[guista]

I finally solved the problem by listening to the advices of the ones and the others.
The solution is very simple :o so here is a step by step mini-tutorial (for linear sync problems only).


I reencoded/demuxed a big MKV movie (let's call it "movie-big.mkv") which produced 2 files : a "movie.mkv" (with a single video track) and "movie.ac3" (the audio track). Because of some MKV parsing problem, "movie.mkv" and "movie.ac3" are out of sync :mad:.


I checked that the video to audio delay grew linearly (the audio plays faster than the video) because my solution would not work in some other case.
By playing the movie, I can notice that the audio is synchronized at the beginning then progressively become unsynchronized : it tends to prove that the sync problem is linear.


Run MediaInfo


Turn MediaInfo to "Debug" => "Advanced mode" (to show the durations' millisecond)


Choose the text mode which is easier to read


Drop "movie.mkv" to MediaInfo and remember the video "Duration" expressed in milliseconds. Mine is 7808354 ms


Drop the "movie.ac3" audio file and remember the audio "Duration" which is 7807200 ms => I can notice a 1154 ms video to audio duration difference


Run MKVMerge GUI


Drop "movie.mkv" and "movie.ac3" into MKVMerge GUI


Select the AC3 audio track


Fill the "Stretch by" field with the 7808354/7807200 (i.e. 1,0001478) fraction => the idea is to stretch the audio track so it matches the video duration


Merge the audio with the video


I finally got a perfectly synchronized movie (from the beginning to the end) :o
Thanks to everybody for your help and hope this will help some other people.

Thanks to the MediaInfo's and MKVMerge GUI's authors who greatly helped me thanks to their tools !

[yetanotherid]

I finally solved the problem by listening to the advices of the ones and the others.
The solution is very simple :o so here is a step by step mini-tutorial (for linear sync problems only).

As I mentioned previously though there's no guarantee the original video and audio were the same length. A time stretch based an an assumption they should be of equal length could just have easily made the sync problem worse.

It'd be nice if MKVMerge had an "auto stretch" function which would do what you did without you needing to bother with the math though.
When working with AVIs and VirtualDubMod it can change the frame rate to match the audio length, but it'd only be the correct frame rate about 10% of the time.

Just out of curiosity, did MediaInfo report your old and new MKV files as both having the same frame rate? I sometimes wonder where the frame rate info comes from. For instance if I resave an MKV which was originally 23.976 to an MKV with a frame rate of 23.975, most software including media players looking at the file seems to look at the original encoding frame rate and report 23.976.
I have opened MKVs with one particular program which reports two frame rates.... original and current. I just can't remember which program....

[guista]

As I mentioned previously though there's no guarantee the original video and audio were the same length. A time stretch based an an assumption they should be of equal length could just have easily made the sync problem worse.

Of course, you are right. That's why I played the film (beginning, middle, end) to check how the delay was evoluting in time.
Because it seemed to increase with time, I supposed it was a linearly growing delay.
I suppose there are some cases where this is not linear or there is just a constant offset because the video begin before the audio or somewhat else ...
But trying to stretch took me only 4 mins and worked perfectly.

It'd be nice if MKVMerge had an "auto stretch" function which would do what you did without you needing to bother with the math though.
When working with AVIs and VirtualDubMod it can change the frame rate to match the audio length, but it'd only be the correct frame rate about 10% of the time.

Yes, "auto-stretch" would be a nice feature to add to MKVMerge.
With MKVMerge, note that you can also stretch the video.

Just out of curiosity, did MediaInfo report your old and new MKV files as both having the same frame rate? I sometimes wonder where the frame rate info comes from. For instance if I resave an MKV which was originally 23.976 to an MKV with a frame rate of 23.975, most software including media players looking at the file seems to look at the original encoding frame rate and report 23.976.
I have opened MKVs with one particular program which reports two frame rates.... original and current. I just can't remember which program....

Yes, my original file and reencoded file both have the same rate of 23.976

[yetanotherid]

Of course, you are right. That's why I played the film (beginning, middle, end) to check how the delay was evoluting in time.
Because it seemed to increase with time, I supposed it was a linearly growing delay.
I suppose there are some cases where this is not linear or there is just a constant offset because the video begin before the audio or somewhat else ...
But trying to stretch took me only 4 mins and worked perfectly.

I don't think we're talking about the same thing.
It's quite possible, in fact even probable, the video and audio inside a container may be of different lengths even when perfectly in sync. The audio track may just end milliseconds or seconds before the video.
When you've got an audio track that's delayed by 'X' and is shorter than the video track by 'Y' and it's losing sync by 'Z' per second...

No doubt you had a linearly increasing delay and it was a 4 minutes well spent, but you also got lucky. I've timed stretched streams which were slightly different in length when in sync, but as the duration difference is unknown in order to do it I've changed the frame rate until the audio is in sync, worked out the percentage difference in the video's duration due to the change of frame rate and then applied the same percentage change to the audio in the original video after fracturing in any delay. A few attempts at doing so though and I realised stopping at the point where changing the frame rate fixed the sync was easier.

Yes, "auto-stretch" would be a nice feature to add to MKVMerge.
With MKVMerge, note that you can also stretch the video.

I don't think video stretching is recommended though.
The equivalent in VirtualDubMod is the auto frame rate change but it's pretty hit and miss as to whether it's the right change to fix sync problems. It's a place to start though.

Yes, my original file and reencoded file both have the same rate of 23.976

Hopefully before I die I'll understand why the audio needed to be time stretched to sync with the original video but not the re-encoded one. Unfortunately though, it's not looking too likely....

[guista]

I've timed stretched streams which were slightly different in length when in sync, but as the duration difference is unknown

OK, after reading you, I understand better : if global duration difference is unknown (or unsync with no global duration difference at all), you have to guess the sync difference by trying and trying ...

in order to do it I've changed the frame rate until the audio is in sync

I suppose you played the video in a tool which allows to change the frame rate while playing so you can adjust the sync quite easily.
Please tell me if you really do like this and if so give me your tool name.

I don't think video stretching is recommended though.

This is highly possible that MKVMerge changes the frame rate to stretch the video.

Hopefully before I die I'll understand why the audio needed to be time stretched to sync with the original video but not the re-encoded one. Unfortunately though, it's not looking too likely....

I won't be helpful for this !

[yetanotherid]

I suppose you played the video in a tool which allows to change the frame rate while playing so you can adjust the sync quite easily.
Please tell me if you really do like this and if so give me your tool name.

For AVIs I'd use VirtualDubMod, but for MKV or MP4 I've been using MKVMergeGUI to re-save the file with a new frame rate and playing it to see what happens. Trial and error too, but at least it's fairly easy to do it in steps of .001 fps until it comes out right.
I guess any decent editing software should let you change the video frame rate and then play video, but I've not got any decent editing software installed.

I was looking at some video frame rates using VLC yesterday and something caught my eye so I opened a few more.
Every single 23.976 video I've made myself has a frame rate of 23.975895 if it's an AVI and 23.976215 if it's an MKV.

24 fps video I've encoded seems to be 23.999980 for AVIs or 24.000384 for MKV or MP4.

25 fps is always 25 fps.

I did find a few AVIs and MP4s I know I didn't mux or encode myself, with frame rates of 23.976023 and 24.000023 so I can only assume for some reason most muxers don't get the frame rate 100% correct and each they're constantly incorrect each time.
Not that it's by much, but I found it curious. Out of all the files I opened I only found one which was exactly 23.976 and one which was exactly 24.000
Unless VLC media player is lying to me.

[guista]

For AVIs I'd use VirtualDubMod, but for MKV or MP4 I've been using MKVMergeGUI to re-save the file with a new frame rate and playing it to see what happens.

Thanks for all your advices and for sharing your experience here.

[robertcollier4]

Hi all,
The problem is : MediaInfo, the only available MKV information tool displays the tracks' length in minutes whereas I need them to be expressed in milliseconds (I think the delta is less than 1 second).

* Do you know a tool which could give me the tracks' length with a millisecond precision ?


You can make MediaInfo show durations in milliseconds precision from Toolbar "Debug > Advanced Mode"