Strange sync problems muxing

[j8ee]

Trying to transcode some HD video into smaller files, I'm running into sync problems, and these problems seems to be connected with muxing.

Muxing video (x264) and audio (aac-mp4 or ac3) from separate files always results in files with audio out of sync. Doesn't matter if audio are the transcoded aac-mp4 files or the source ac3 files, and it doesn't matter if I mux with mkvmerge or mp4box.

But trying out avidemux 2.5.8, when I encoded a quick mp4 test file with a small, crummy video and original ac3 audio, I noticed that it stayed in sync. So I remuxed that file with mkvmerge and replaced the existing video with a proper one, and it was in sync too afterwards.(!?) But I don't actually want the source ac3 files in there of course, I was just surprised that it worked, and I guess it showed that my transcoded video files were working ok and that the problems most likely are in the muxing and/or the audio files.

The mp4 audio seems impossible to mux in sync as they are. Necessary timing adjustments vary from file to file. One file needs 200 ms to start with and 400 in the end, another 200 in the beginning and 1200 in the end, all 45 min long. I guess it's the video's framerate that varies, but how come my encodes are in sync with the ac3, and not the aac-mp4?

Video is encoded with megui/x264. Reported as 60fps when indexing and by mediainfo. I tried doing encoding one video with 59.94, but that was completely wrong, and needed sync adjustments also vary from file to file as said. For audio I tried a number of combinations with different kind of decoders and nero and qaac/apple as encoders, but audio sync has been off in exactly the same way regardless.

Are there some kind of framerate flags that can be passed on when remuxing from mp4 to mkv? Are there other things that can make that video and audio work in sync in the above situation? And what should I try to finally be able to make my encodes watchable...?

If anything is unclear or you need more information, just ask along.

[sneaker_ger]

Two possible causes:
- the video has a variable framerate
- the audio has gaps (which is similar to a variable framerate)

For such tracks to stay in sync timecodes for each frame/block are stored in containers such as mkv/mp4. Describing them through single "fps tags" is not sufficient. Also: when demuxing to raw formats (.264, .ac3) these timecodes are lost. Mkvmerge can read timecodes from containers that have them so a direct remux results in synced files while a demux->re-encode->mux will result in desync. Try to read directly from the source file instead of demuxing. For video tracks that are destined to run through AviSynth you need to do one of the following:
- do a VFR->CFR conversion (for example ffvideosource() has fpsnum and fpsden parameters for that) or
- extract timecodes to an external file and re-import them later on (for example using mkvtoolnix)

[j8ee]

Thanks for the info, things start to make sense now. Only that the source files doesn't seem to have any timecodes when opening them in mkvcleaver and mkv extract gui. Still most of the files I'm transcoding are in sync now, by opening the original mkv files and sort of replacing the content. For some I had to make manual delay adjustments with mkvmerge. Maybe some delay got added when I muxed video and audio separately and they were in mp4, but not when they are muxed into an existing container? I think I was wrong in that the timing and offset fluctuated in one video, it was probably a linear error. Well, doesn't matter now.

But - some files I had to edit and cut via avisynth, and they are still off and hard to get right. Mediainfo says source is a fixed 60fps framerate, and it's encoded with x264 core 125 so I guess it should be reliable, still encoded video is ever so slightly off.

Since video is transcoded with ffvideosource and a set framerate (FFVideoSource("<video path>", fpsnum=60, fpsden=1, threads=1)), I guess I have to start tweaking that framerate and trying to get it to match the audio in each segment, and then append them to each other? Adjustments for stretch and delay that I made in mkvmerge for each part weren't passed on when I tried to merge such segments.

Audio isn't off that much in these cases, maybe 200 ms in 30 minutes, but then it's noticeable and different from source.

[sneaker_ger]

Only that the source files doesn't seem to have any timecodes when opening them in mkvcleaver and mkv extract gui.
Every track of every mkv file has timecodes, no exceptions.

Mediainfo says source is a fixed 60fps framerate, and it's encoded with x264 core 125 so I guess it should be reliable, still encoded video is ever so slightly off.
MediaInfo is not reliable.

Since video is transcoded with ffvideosource and a set framerate (FFVideoSource("<video path>", fpsnum=60, fpsden=1, threads=1)), I guess I have to start tweaking that framerate and trying to get it to match the audio in each segment, and then append them to each other?
If you opened your original, in-sync source file that way with ffvideosource the resulting video file should match the original file. The next thing you have to focus on is the audio conversion to see if any gaps might have gotten lost.

Adjustments for stretch and delay that I made in mkvmerge for each part weren't passed on when I tried to merge such segments. Audio isn't off that much in these cases, maybe 200 ms in 30 minutes, but then it's noticeable and different from source.
Don't bother with stretches, they cannot perfectly match gaps/VFR. The audio must be decoded directly from the in-sync source file so that all gaps are passed as silence to the encoder.

[j8ee]

First - many thanks for taking your time with this.

I have gotten all the files watchable by tweaking delay and stretching in mkvmerge, trying to minimize the errors over the whole file in large. Most files managed with just delay adjustments, which I'm happy about.

But I will continue a bit more just for the sake of understanding what's happening and trying to get it to work properly.

Every track of every mkv file has timecodes, no exceptions.

Damn GUI's...

... The next thing you have to focus on is the audio conversion to see if any gaps might have gotten lost.

Don't bother with stretches, they cannot perfectly match gaps/VFR. The audio must be decoded directly from the in-sync source file so that all gaps are passed as silence to the encoder.

Any suggestions on what I can try to find a way to decode this audio (ac3 6-channel) that does that? Dropped tiny gaps in the audio would probably match the sync errors I'm getting.

I have no clue of the inner workings of MeGUI, but I tried NicAudio, FFAudioSource and DirectShowSource from the MeGUI encoding and DirectShowSource in an avs-file - ... hmmm... which were then used via MeGUI also, which could have made it pointless I guess. Also I tried converting the mkv/ac3 via avidemux and foobar, but no difference.

I must admit I didn't do the tests/encodings in a structured way, it was just a single problemfile I decoded and encoded in a number of ways to see if one of the resulting audio files happened to be in sync, but it was during the avidemux session I discovered that my video encodes were in sync to the original audio, and that the problem should be related to audio and muxing.

[sneaker_ger]

I don't know how MeGUI handles audio. NicAudio is very likely a no-go. ffaudiosource and directshowsource might lead to success but - like I said - it's essential they open the in-sync source file directly without any prior demuxing step.

[foxyshadis]

DirectShowSource (and DSS2) can do that, but did you originally process the audio with eac3to? That will insert silence wherever there are audio gaps.