X264 Ongoing Audio Sync Problems

[Cyberpro60]

I am desperate for some help resolving a recent ongoing problem with x264 and out of sync audio/video.

Here's what I note is happening:

Source: 4 hour MPEG2 file 720 x 576 resolution with a DAR of 16:9 [ie Stock Standard PAL widscreen DVD format]. Audio is MP2. Framerate is 25. Interlacing is Progressive.

Output Required: MP4/AAC video file encoded via X264 at the same specifications as the original.

Here's what happens .... regardless of the actual software used (MeGui, HDConvertToX, AviDemux etc) I always end up with a video that gets progressively more out of sync as time progresses. That is, whilst the video and audio are roughly in sync at the beginning of the video by the end an actor can, visually, complete an entire sentence before the words are heard.

I have come the conclusion that the problem must either be with the either the audio or video codec. I say this becase I get the same result regardless of the encoder software used or, indeed, the machine used to do the encoding (I get similar sync problems when encoding on a second machine).

If I use a different codec, such as Main Concept's H264 (via TMPGenc Express) the results are totally perfect except that, in my view anyway, encoding with X264 is, usually, faster with better and smaller output file.

The odd thing is that this audio sync problem is of relatively recent nature as I have been encoding successfully with X264 for many years. Since this problem has cropped up I have read virtually every thread on the subject and tried a dozen different suggestions all to no avail. It is my get feeling that, somewhere along the line the encoder starts using an incorrect framerate somewhere however it is beyond my ability to know how to track this error down and resolve it.

Your assistance would be very gratefully received.

[sneaker_ger]

mp4box had some problems with recent x264 versions. Try an updated mp4box version (http://vfrmaniac.fushizen.eu/MP4Box/MP4Box_0.4.6-DEV-rev.7(2010-10-03).rar).

[Cyberpro60]

Ok Thanks ... Checking the current version in MeGui gives 0.4.6-DEV (internal rev. 7) which is the same as the version in your link.
The only difference I can see is that the currently installed version reports "GPAC_DISABLE_3D" whereas your "newer" version says "GPAC_HAS_JPEG". Never-the-less I will give this version a workout and see if that solves my problem.

[sneaker_ger]

OK, I think the current version of MeGUI should already have a fixed mp4box so it's probably really not the cause. (If you want to look at it anyways, problems were described in these threads: http://forum.doom9.org/showthread.php?t=155705 ; http://forum.doom9.org/showthread.php?t=156110 )

That was the only known sync problem with x264/mp4 I know of so the problem might lie in the source / the AviSynth script.

[AlekseiV]

Can you play your source file and have it be in sync?

Edit: and now I see you mentioned that MainConcept H264 doesn't have the problem.

When you encoded using MeGUI, what was the AVS script you used?

[Trahald]

The resolved sync issue was a change in x264 that took advantage of a change in the h264 spec that mp4box never accounted for. x264's output was legal but it confused the old code in mp4box. mp4box code has been updated but you need very recent beta compiles for an executable with the change. Let us know the results of your test. please make 1000% (yes there are 3 zeros) that you are using the updated mp4box. if you still have problems, you MUST find a place to post sources that can reproduce the problem (audio/video) no matter how big. Try to make them as small as you can of course.

[hydra3333]

anyone have a link to these "recent" betas ? The only link I have is the 30 Sep 2010 dev build here http://kurtnoise.free.fr/mp4tools/

[Audionut]

You obviously missed the second post in this thread!

[hydra3333]

Yup. Although that link's broke :)

[Trahald]

My patch was committed mid September so that Kurtnoise build should be fine. I'll check it later.

[Midzuki]

To whom this may interest :)

by VFR_Maniac

http://vfrmaniac.fushizen.eu/MP4Box/

by Komisar

http://komisar.gin.by/tools/

by X5-452

http://sada5.sakura.ne.jp/files/index.php?folder=TVA0Qm94

by Chikuzen

http://www.mediafire.com/?fcwhwzfgc4fkc

Take your pick :p

[kypec]

Midzuki, any idea whether all those builds were compiled from same sources? Or does any of them include some special patches...
Anyway, thanks for the links, these should definitely be put in some stickies around here!

[audyovydeo]

My patch was committed mid September so that Kurtnoise build should be fine. I'll check it later.

it's been out for a while (30 sep) :

http://kurtnoise.free.fr/mp4tools/MP4Box-0.4.6-rev2080.zip
http://kurtnoise.free.fr/mp4tools/MP4Box-0.4.6_x64-rev2080.zip


cheers
a/v

[Cyberpro60]

Sorry for the delay in responding to the posts in this thread but I have been away on serious family matters.

Regarding Audio Sync problems however:

(1) I downloaded and installed the latest MP4Box.exe version as suggested earlier in this thread. The version I installed is listed as GPAC version 0.4.6 (internal rev. 7) but this did NOT solve the audio sync problem.

Let me reiterate, the original MPEG2 video plays perfectly in sync from start to finish. If I encode using another H264 codec (Main) then the end result is also perfectly in sync. If I use X264, regardless of the actual front end used (ie MeGui, AVIDemux etc) then I end up with a progressively out of sync MP4 video file.

(2) I visited the link suggested by sneaker_ger and found that very interesting (if a little intimidating) but discovered no real suggestions to fix the problem.

(3) The avs script used (generated by MeGui) is as follows:

-----------------------------------------------
LoadPlugin("C:\Program Files\MeGUI\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("E:\TEMP\ROTK.d2v", info=3)
LoadPlugin("C:\Program Files\MeGUI\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true, threads=0)
#deinterlace
#crop
#resize
#denoise
-----------------------------------------------

(4) The original video source is 7GB so it is a little too large to upload.

(5) I have recently updated MeGui and the component versions operating are as follows:

Core 0.3.5.18
dgindex 1.58b
mp4box 2010.09.30
ffmpeg 20900.12.20
yadif 1.7
enc_aacplus 2009.12.30
x264 1745


These errors are so noticable that it renders x264 virtually useless for encoding so I am very anxious to get this problem resolved asap.

[nm]

Let me reiterate, the original MPEG2 video plays perfectly in sync from start to finish. If I encode using another H264 codec (Main) then the end result is also perfectly in sync. If I use X264, regardless of the actual front end used (ie MeGui, AVIDemux etc) then I end up with a progressively out of sync MP4 video file.

Try MKV instead of MP4 output. If the MKV stays in sync, your problem is still in MP4 muxing. Otherwise it could be in decoding (if the stream is broken so that the decoder skips parts of it) or incorrect framerate.


(4) The original video source is 7GB so it is a little too large to upload.

How long is it in minutes, and how much does it get out of sync by the end?

[sneaker_ger]

(1) I downloaded and installed the latest MP4Box.exe version as suggested earlier in this thread. The version I installed is listed as GPAC version 0.4.6 (internal rev. 7) but this did NOT solve the audio sync problem.

Let me reiterate, the original MPEG2 video plays perfectly in sync from start to finish. If I encode using another H264 codec (Main) then the end result is also perfectly in sync. If I use X264, regardless of the actual front end used (ie MeGui, AVIDemux etc) then I end up with a progressively out of sync MP4 video file.

(2) I visited the link suggested by sneaker_ger and found that very interesting (if a little intimidating) but discovered no real suggestions to fix the problem.

Sorry, I think I should've been more clear: your version of mp4box is totally fine.
The problem lies somewhere in the source. (vfr or damaged)

Try the following script (needs ffmpegsource plugin (http://code.google.com/p/ffmpegsource/)):
audio=ffaudiosource("source.ts")
video=ffvideosource("source.ts",fpsnum=25,fpsden=1)
AudioDub(video, audio)

Try to encode a small part at the end of the video to see if it solves the sync problems.
I'm not really experienced in .ts sources, but I remember seeing people having similar problems around here. Try a forum search if you haven't already. I highly doubt that it's a problem of x264.

You could also try to encode directly via x264 (use mkv or mp4 output):
x264 -o output.mkv source.ts

[MasterNobody]

You could also try to encode directly via x264 (use mkv or mp4 output)
Don't use mp4 output until you sure that is was compiled with good gpac version (not always the case) or compiled with L-SMASH instead of gpac.

[Cyberpro60]

@NM: I will try your suggestion of MKV output and see if that resolves the problem and report results here shortly.
@sneaker_ger: The original source was not .ts. It was .mpg. I will however run through ProjectX (Demux util) to see if there are any reported errors and report back. I will also try the .avs script additions you suggested to see what effect that has. Also, regarding the length of the original mpeg file ... it was 3 hours and 8 minutes. By the end of the file sound is about 2 seconds later than the video. That may not sound like much but, in pratice, this means that an actor can speak a whole sentence and have their mouth shut before their voice is heard. Very disconcerting!

Thanks everybody for your input and suggestions and for the time and effort your have put in to framing your responses. It is very much appreciated.

[Trahald]

@CyberPro60
Are you now having this issue with every source you try? (ie not only the 7gb one you mentioned but every source)

Make sure you are using as new of a mkvtoolnix as possible as older versions suffered from a sync issue for the same reasons as mp4box.

[yetanotherid]

What if you tackle it the other way around and try making your out of sync encode in sync?

If you open the file with MKVToolnix you can adjust the frame rate, which is the method I use as my first attempt to fix the problem, then you'd re-save the video and audio to a new file. It can be a bit of trial and error sometimes, and sometimes you have to adjust the audio delay as well, but if a video is supposed to have a frame rate of 23.976, for example, and the audio is gradually losing sync, I'll change it to 23.975 (or 23.977) to see if it's any better then try again if need be. Sometimes you even need to specify a value such as 23.9745 before they lock together.

The other alternative would be to try to use the time stretching feature to get them to match, but I find changing the frame rate is usually easier.

Here's a question I don't know the answer to but your problem problem prompts me to ask it. Given that the audio can be time stretched inside an MKV file, can the same be done with other formats? i.e. vob files?
It's just that from your description... where the audio gradually loses sync as much as it does, it almost sounds like the original video and audio were 24fps or 23.976fps and it's been sped up to 25fps, but the audio has just been time stretched inside the vob file.

Of course if it was a one-off problem it'd be different but it sounds like it's happening all the time. Is the frame rate of the output video 25fps?
MeGUI has an option to encode audio via DirectShow. I wonder if that'd make a difference? What if you use the original audio rather than converting it?

Could it be another rouge codec or splitter on your computer causing problems? Sometimes the quickest option is to start again with a fresh install of Windows. I have a second drive in my PC with a partition containing Windows and nothing more and I've imaged it so I can restore it to keep it fresh and new. Many times when I've hit problems I've installed the software giving me issues on the second copy of Windows as that way it tends to pinpoint the problem as being the software itself, or some sort of conflict with other software installed. Do you have an old hard drive lying around which you could put in the PC to install Windows just for testing purposes? Is there something recent.... in respect to media software or updates etc which your two PCs would have in common? Something which would have changed before the problem started?

Do GUIs such as MeGui, HDConvertToX, AviDemux etc all use "internal" codecs or is it possible for some reason they're all loading a system codec which is causing the problem?

[yetanotherid]

If it is just the one video, and as I'm clueless as to the cause, I generally resort to converting the long way.
I use the free version of AnyVideoConverter to convert the problem video/audio to lossless formats, usually inside an AVI or MKV file. I don't' know why it's so clever, but if you set the A/V sync to default, then go into the program's options and bump up the A/V sync fussiness level, it'll spit out a perfectly synced video every time. I then convert the lossless video using my usual program, but you can use AnyVideoConverter to convert to a variety of formats. I don't think it'll do anamorphic video or multi-channel audio, but I've converted the video using my usual program, and then converted both audio and video using AnyVideoConverter (just to a tiny resolution) then I've extracted it's audio track, muxed it with my video encode, and it's done.

FormatFactory is another free program which is good at maintaining sync, although AnyVideoCoverter is better.

[yetanotherid]

One last thought.... are you sure the loss of sync is gradual, or does it actually happen in steps? I'm just wondering if the audio inside the vob files is a different length to the video, or has gaps.... so if they were joined together there'd be numerous gaps in the audio. Then along comes your encoding program which simply de-muxes the audio as one big file, ignoring the gaps, and converts it.
Could it be that it's not the codec which is the problem. Is TMPGenc Express itself good at fixes sync issues and you just happen to be using a different codec? I've never used TMPGenc Express myself.

I have used ConvertXtoDVD on problem video though and watched it's log file. It hits gaps in the audio and adds extra blank bits of the appropriate length as it converts. Is TMPGenc Express capable of doing the same? I'm pretty sure programs such as MeGUI or HDConvertToX can't.

[Cyberpro60]

OK, Back again. I tried encoding to MKV and the results was the same ... audio still out of sync and lagging behind the video progressively. I also ran the original video file through ProjectX and there were No errors or Warnings which gives me confidence that there is nothing untoward with the source file.

@yetanotherid: All that you say sounds fine except for the fact that the original video file plays perfectly in sync and the video encodes perfectly in sync when I use a different H264 codec like Main's for example. This problem only has arisen with recent updates to X264. Prior to that I have encoded hundreds of video files with no sync errors at all. I will try using the original audio file tho as a test to see what happens.

[Cyberpro60]

I have conducted a few experiments to see if I could track down the cause of this problem and I have had some interesting results.

First I took the standard .MKV output from MeGui and demuxed audio/video and discarded the audio component and then remuxed using the 6 channel .AC3 audio track extracted from the original source MPEG video. The output was perfectly in sync throughout however with an .AC3 audio file I am restricted to .mkv output. At least I now have a useable MPEG4 video file - finally.

Second, I used an external AC3 to AAC conversion utility (OjoSoft Audio Converter - an excellent little audio utility) using the original source AC3 file as input and muxed the resulting .AAC audio file to the H264 video file produced by MeGui. Again, both the resulting MKV and MP4 video files were perfectly in sync throughout.

This strongly suggests to me that perhaps there is a serious issue with the AC3 to AAC conversion routines utilised by MeGui, AVIDemux and HDConvertToX although I am not exactly certain where in the conversion/muxing pipeline this error is occuring or why.

Any thoughts on how to fix this?

[asarian]

I have conducted a few experiments to see if I could track down the cause of this problem and I have had some interesting results.

First I took the standard .MKV output from MeGui and demuxed audio/video and discarded the audio component and then remuxed using the 6 channel .AC3 audio track extracted from the original source MPEG video. The output was perfectly in sync throughout however with an .AC3 audio file I am restricted to .mkv output. At least I now have a useable MPEG4 video file - finally.

Second, I used an external AC3 to AAC conversion utility (OjoSoft Audio Converter - an excellent little audio utility) using the original source AC3 file as input and muxed the resulting .AAC audio file to the H264 video file produced by MeGui. Again, both the resulting MKV and MP4 video files were perfectly in sync throughout.

This strongly suggests to me that perhaps there is a serious issue with the AC3 to AAC conversion routines utilised by MeGui, AVIDemux and HDConvertToX although I am not exactly certain where in the conversion/muxing pipeline this error is occuring or why.

Any thoughts on how to fix this?

Personally, I always encode video only. I use eac3to to create an mkv for processing with just the video HD track, and only mux in the extracted (and/or converted) audio later. Rarely had any problem with it.

x264 will tell you at what fps it's taking in; like 24000/1001, or whichever is applicable. If that looks off to you (don't use a GUI), you can immediately abort the process. So far I only had 1 occassion where I needed to add an AsumeFPS statement to make x264 encode at the right speed (and that, mind you, was not x264's fault either, but caused by FFVideoSource interpreting the fps erroneously).

[Cyberpro60]

@asarian. Thanks for the tip about eac3to. I will investiage this utility some more but I can't really see how it can help with the problem I have, that is, the conversion errors produced by the 3 main mp4 encoders in respect of AC3 to AAC conversion.

[asarian]

@asarian. Thanks for the tip about eac3to. I will investiage this utility some more but I can't really see how it can help with the problem I have, that is, the conversion errors produced by the 3 main mp4 encoders in respect of AC3 to AAC conversion.

Well, eac3to, while it can do video these days too -- and do it well -- it was originally just an audio converter; so I wouldn't be at all surprised if it does your AC3 -> AAC conversion flawlessly too.

[stax76]

Not only for sync issues but for any kind of problem in most cases it's helpful to have info about the source, info such as ripping software, ripping mode, capture hardware, capture software, capture settings, DVB standard, MediaInfo etc.

[Cyberpro60]

@asarian: From my reading so far I don't think eac3to has the capability of converting audio to AAC format.

@stax: The video problems I have mentioned apply to many videos from different sources [but not all]. Most were captures from a digital televison card [DVB]. These videos were captured as .ts files, edited and saved in PAL mpeg format [25 fps 720x576px 16:9] with .AC3 6 channel sound. As indicated above, I ran these videos through MeGui as per usual to reduce file size for storage. I also want to stress that I have been doing this for a number of years now and only recently has this audio problem surfaced. The exact same problem is evident if I encode on another machine ... that is, these errors are not some artifact created by something odd about my specific computer's configuration.

The experimental result posted above we conducted on a DVD movie file ripped from disc by DVD Decrypter. With this test audio ran faster than the video. That is you could hear somebody speak before you actually saw them open their mouths. The degree of error increased as the video progressed. As indicated above when I encoded the AC3 file to AAC with an external encoder the audio/video was perfectly in sync. The output video file had exactly the same number of frames which suggests to me that the error, what ever it happens to be, is not so much a fault of the X264 encoder but of the audio conversion/muxing software that is used.

[stax76]

It seems you mixed different formats and tools in this thread and didn't understand the problems yet.

MPEG-2 TS should be recorded as single file and demuxed with ProjectX, as audio encoder I still suggest BeSweet for mp2 and ac3, if you have a new mp4box built there shouldn't be any sync issues, this method is known to be reliable.

For AVC TS users have reported success fixing sync issues with tsmuxer, ts-doctor and VideoRedo.

Regarding DVD sources I recently wrote a couple of things in two other threads:

http://forum.doom9.org/showthread.php?p=1454375#post1454375

http://forum.doom9.org/showthread.php?p=1448439#post1448439

Most GUIs use eac3to and NicAudio/Nero/Lame and that should work fine with DVD and Blu-ray sources, I'm not sure if it works reliable with DVB sources.

[Audionut]

I started using Jeeb's builds with the Audio patch here, http://x264.fushizen.eu/

Perfect sync everytime.

It can't read AC3 yet. But I just use eac3to to convert to aac and then use --audiofile --acodec copy

I can go back to using x264 for my phone encodes now instead of xvid.

[Cyberpro60]

@stax76: Regarding your suggestions for video file handling. FYI: I generally don't demux my .ts files with ProjectX unless they are damaged and need special processing. For the most part I prefer to use VideoRedo as it combines both editing and demuxing into the one package and does a truly exceptional job. Let me stress again that I have been processing video recordings for several years now without any of these sync problems so I am quite familiar with all the major software available in this genre. Even so, I actually went to the trouble of demuxing some of my problem files with ProjectX and then tried converting them ... but the audio was still out of sync!

Let me also stress again that the all the videos that eventually end up out of sync after conversion to .MP4/MKV all started out as near to perfect .MPG files. The audio and video were perfectly in sync from start to finish .... ergo .... we are not dealing with corrupted source files here and these sync errors must result from the conversion process.

Today I process a batch of about 5 different video files - all TV captures in .TS format. Three had .MP2 sound and two had .AC3 sound files. Interestingly, all the videos with .MP2 audio converted perfectly. All those with .AC3 sound were seriously out of sync. I think it is therefore reasonable to conclude that MeGUI (and other such programs) seem to be having difficulty converting .AC3 to .AAC format and keeping them in sync with video.

The question in my mind is why? I have processed literally hundreds of video files over the years and only with recent updates to MeGui and other X264 encoders has this problem with .AC3 arisen. I am not exactly sure what goes on "under the hood" of MeGui with respect to the .AC3 to .AAC conversion. That is, I am not sure exactly what program it is that is doing the actual conversion for it would seem to me that this is the step that is introducing all these new errors.

Anyone care to comment or make suggestions for fixing this annoying problem?

@Audionut: Thanx for the suggestion. I will give this new encoder a workout.

[stax76]

You write you are experienced but then I don't understand why you can't find the problem, it's can't be hard to find. MeGUI writes a log file giving useful info for troubleshooting, if a new MeGUI version uses the same tools then the problem must be updated components, likely the decoder which should be NicAudio for AC3.

[raeltheimperialaerosolkid]

Cyber,
I don't know if it can help but just yesterday I came over a synch problem that was annoying me a real lot. It was due (i don't know why) to the FFmpg-mt decoder in the FFDhsow (last stable build). All the encode that I made using whatever build of x264 (with meGui or Avidemux, it didn't matter) started to go out of synch after few moment. At the beginning I started to think that It was a VFR problem in the mp4 container but it wasn't. I simply changed from the FFDshow configuration the decoder of the AVC videos to the Libavcodec and all the synch problem were gone.
I hope it helps...:o