Out of Sync mp4 container

[bnshrdr]

Here is the dilemma I have run into. My ultimate goal is to have a video with h.264 video and aac audio inside an mp4 container.

When I am on windows, I use x264 to encode the video (which is excellent by the way) and faac to extract the audio. I don't have a windows version of ffmpeg so I boot into linux and combine the two:
ffmpeg -i input.mp4 -i input.aac -vcodec copy -acodec copy out.mp4.
When I do this, the sync is waaaay off.

Now if I do all the encoding on linux straight through ffmpeg (video + audio simultaneously), the sync is still off, but not as bad (still not acceptable, like 1-2 seconds).

Then I tried ditching the mp4 container and tried matroska, and lo and behold, its perfectly synced. The problem is, I dont want matroska, I want mp4.

Once again, I tried searching multiple forums but all threads led to dead ends. Does anyone know why this is happening?

Here are all the flags I have been using (for ffmpeg, but I used the equivalents for x264):
-acodec libfaac -ab 96k -vcodec libx264 -b 954k -subq 7 -b_strategy 2
-partitions +parti4x4+partp8x8+partb8x8 -flags2 +wpred -me_method esa -flags2 +mixed_refs -flags2 +dct8x8 out

All help appreciated!

[nm]

Try MP4Box (or Yamb) for muxing the streams to MP4.

You haven't described your source formats and how they are handled before feeding to x264 and faac. If possible, upload a sample clip of the source video that exhibits the problem so we can try it ourselves.

[b66pak]

hi,

use mp4box for muxing .(h)264 with .aac (under windows)...

here is a GUI:

http://forum.doom9.org/showthread.php?t=115459
_

[bnshrdr]

Ok. Well I tried mp4box, still way out of sync. But I feel like I didn't do something right, do you just add the video and audio as inputs and let it fly?

@nm
Here is what ffmpeg is interpreting as the source V/A, which is correct:
Input #0, mpeg, from 'out.vob':
Duration: 00:00:37.81, start: 660.500000, bitrate: 4687 kb/s
Stream #0.0[0x1e0]: Video: mpeg2video, yuv420p, 720x480
[PAR 32:27 DAR 16:9], 9800 kb/s, 59.94 tbr, 90k tbn, 59.94 tbc
Stream #0.1[0x80]: Audio: ac3, 48000 Hz, 5.1, s16, 448 kb/s

Output #0, mp4, to 'output.mp4':
Stream #0.0: Video: libx264, yuv420p, 720x480
[PAR 32:27 DAR 16:9], q=2-31, 954 kb/s, 90k tbn, 20 tbc
Stream #0.1: Audio: libfaac, 48000 Hz, 5.1, s16, 96 kb/s
It also doesn't seem that the encoder was getting saturated because it said my average qp was around 20.

Here is the source video:
http://rapidshare.com/files/261491564/out.vob

Here is my attempt at encoding:
http://rapidshare.com/files/261491922/output.mp4

If you guys want a host other than rapidshare let me know.

[nm]

Ok. Well I tried mp4box, still way out of sync.
Try encoding to a raw .264 elementary stream with x264, instead of using x264's GPAC MP4 output or ffmpeg.

It also doesn't seem that the encoder was getting saturated because it said my average qp was around 20.
And even if it did, it wouldn't cause sync problems.

Here is the source video:
http://rapidshare.com/files/261491564/out.vob
I'll try it later.

If you guys want a host other than rapidshare let me know.
MediaFire is nicer (less waiting), but I got the first file.

[b66pak]

mediafire is much better (also megaupload)...
_

[bnshrdr]

I tried extracting both streams, x264 into a raw 264 elementary stream, and the audio into aac stream, and the muxing failed. I'm stumped.

[poisondeathray]

This is the frame rate from mediainfo of your encoded sample, likely the cause of your sync issues.

Frame rate mode : Variable
Frame rate : 19.789 fps
Minimum frame rate : 2.222 fps
Maximum frame rate : 20.000 fps
Original frame rate : 20.000 fps

I tried your sample with dgindex, megui and it came out the proper 23.976 cfr

Did you specify the fps when you tried mp4box?

[bnshrdr]

Good observation. I set the fps manually this time and it does give some different results. What is happening now is that the video plays in sync for about 5 seconds, but then quickly gets out of sync. Is this an issue with my audio track? I extracted it using mencoder and ffmpeg and both results are the same.

[poisondeathray]

Sorry, I'm not too familiar with mencoder, and only a bit with ffmpeg

If you have access to windows, why not do it the most consistent way ? i.e. DGIndex / avisynth / some x264 gui

For example, there was a -326ms delay in your sample, I'm not sure if your current method took that into account

Maybe some others can chime in that are more familiar with mencoder/ffmpeg

[roozhou]

ffmpeg -i input.mp4 -i input.aac -vcodec copy -acodec copy out.mp4.


I don't think this will work with an unmodified ffmpeg build. Muxing from adts aac to something else will result in corrupt files.

You should either add -absf aac_adtstoasc, or try win32 build from here (https://sourceforge.net/projects/direct264/files/Related%20Programs/ffmpeg%20%28demuxer_muxer%20only%29/ffmpeg_svn_r19469.7z/download).

[bnshrdr]

You should either add -absf aac_adtstoasc, or try win32 build from here (https://sourceforge.net/projects/direct264/files/Related%20Programs/ffmpeg%20%28demuxer_muxer%20only%29/ffmpeg_svn_r19469.7z/download).

That's not a valid switch for ffmpeg (well at least on mine). Also, I have actually downloaded that ffmpeg build for windows and tried it as well, same results as my linux ffmpeg.

I think I'm gonna just go with matroska for now just because I have spent hours trying to find the right way to get this to work and the solution just keeps eluding me.

[roozhou]

Are you sure input.mp4 and input.aac are of the same length? I guess matroska will give your the same async. issue.

[nm]

I got the same 300 ms audio delay with ffmpeg's MP4 output while MKV output did work without sync issues. I had to force the correct framerate (-r 24000/1001) though.

I'd suggest HandBrake(CLI (http://trac.handbrake.fr/wiki/CLIGuide)) for multi-platform command line encoding to MP4 and MKV. Unlike ffmpeg, HandBrake has a detelecine filter to handle hard-telecined sources, which you may encounter at some point since you seem to be encoding NTSC DVDs. It also supports variable framerate for hybrid sources with both progressive and interlaced sequences. Here's an example for constant quality (CRF) encoding with x264 and FAAC (LAME MP3 probably has better quality at these bitrates though):
HandBrakeCLI -i input.vob --detelecine -e x264 -q 0.6 \
-x ref=5:mixed-refs:bframes=3:b-adapt=2:b-pyramid=1:weightb=1:8x8dct=1:subme=9:me=umh -E faac -B 96 -o output.mp4

[bnshrdr]

I forgot to mention that the way it worked in an mkv container was to force the frame rate. At least I wasn't the only one with the problems.

I'll give handbrake a try, it's been a while since I've used it. I think I used the GUI version before, which I like to get away from if possible. I didn't know there was a CLI.

Thanks for the recommendation!

[b66pak]

i used a GUI (iPodMe) for ffmpeg on the sample (out.vob) and all is ok!


here is the result:

http://www.mediafire.com/?j5qwd4uj2gg


here is the log:

ffmpeg.exe -y -i "out.vob" -f mp4 -r 23.976 -vcodec libx264 -level 30 -flags +loop -flags2 -wpred-dct8x8 -subq 4 -cmp +chroma -me_method umh -me_range 16 -trellis 2 -mbd 2 -refs 3 -partitions +parti4x4+partp4x4+partp8x8 -crf 19 -deinterlace -maxrate 1500k -bufsize 4M -g 250 -coder 0 -threads 0 -acodec libfaac -ac 2 -ab 128k -s 640x432 "out.mp4"

FFmpeg version SVN-r16596-Sherpya, Copyright (c) 2000-2009 Fabrice Bellard, et al.
libavutil 49.12. 0 / 49.12. 0
libavcodec 52.10. 0 / 52.10. 0
libavformat 52.23. 1 / 52.23. 1
libavdevice 52. 1. 0 / 52. 1. 0
libavfilter 0. 2. 0 / 0. 2. 0
libswscale 0. 6. 1 / 0. 6. 1
libpostproc 51. 2. 0 / 51. 2. 0
built on Jan 14 2009 07:24:48, gcc: 4.2.5 20080919 (prerelease) [Sherpya]

Seems stream 0 codec frame rate differs from container frame rate: 29.97 (30000/1001) -> 59.94 (60000/1001)
Input #0, mpeg, from 'out.vob':
Duration: 00:00:37.81, start: 660.500000, bitrate: 4687 kb/s
Stream #0.0[0x1e0]: Video: mpeg2video, yuv420p, 720x480 [PAR 32:27 DAR 16:9], 9800 kb/s, 59.94 tb(r)
Stream #0.1[0x80]: Audio: ac3, 48000 Hz, 5.1, s16, 448 kb/s
Output #0, mp4, to 'F:\Tools_Video\iPodME_v2.2\out.mp4':
Stream #0.0: Video: libx264 (hq), yuv420p, 640x432 [PAR 6:5 DAR 16:9], q=2-31, 200 kb/s, 23.98 tb(c)
Stream #0.1: Audio: libfaac, 48000 Hz, stereo, s16, 128 kb/s
Stream mapping:
Stream #0.0 -> #0.0
Stream #0.1 -> #0.1
[libx264 @ 01AD5BF0]using SAR=6/5
[libx264 @ 01AD5BF0]using cpu capabilities: MMX2 SSE2Slow
[libx264 @ 01AD5BF0]profile Baseline, level 3.0
Press [q] to stop encoding

video:5738kB audio:586kB global headers:1kB muxing overhead 0.366587%
[libx264 @ 01AD5BF0]slice I:5 Avg QP:19.61 size: 34327
[libx264 @ 01AD5BF0]slice P:1129 Avg QP:21.03 size: 5052
[libx264 @ 01AD5BF0]mb I I16..4: 9.4% 0.0% 90.6%
[libx264 @ 01AD5BF0]mb P I16..4: 1.9% 0.0% 3.5% P16..4: 32.7% 13.5% 6.2% 1.0% 0.4% skip:40.9%
[libx264 @ 01AD5BF0]ref P L0 83.3% 9.5% 7.2%
[libx264 @ 01AD5BF0]SSIM Mean Y:0.9856357
[libx264 @ 01AD5BF0]kb/s:1243.4
Total time elapsed: 00h00m50s


http://nodadev.wordpress.com/pc-projects/ipodme/
_

[bnshrdr]

Huh, well look at that. I'll give that tool a look and see how it works.

Thanks for taking the time with this, I really appreciate it.

[nm]

i used a GUI (iPodMe) for ffmpeg on the sample (out.vob) and all is ok!


here is the result:

http://www.mediafire.com/?j5qwd4uj2gg
Yep, that's ok.

ffmpeg.exe -y -i "out.vob" -f mp4 -r 23.976 -vcodec libx264 -level 30 -flags +loop -flags2 -wpred-dct8x8 -subq 4 -cmp +chroma -me_method umh -me_range 16 -trellis 2 -mbd 2 -refs 3 -partitions +parti4x4+partp4x4+partp8x8 -crf 19 -deinterlace -maxrate 1500k -bufsize 4M -g 250 -coder 0 -threads 0 -acodec libfaac -ac 2 -ab 128k -s 640x432 "out.mp4"
I tried this command with ffmpeg r19289 and there was the same 300 ms delay as previously. So maybe it's a regression somewhere between r16596 and r19289.

[bnshrdr]

Also, if you don't mind me asking, how is it that you figure out the delay between video and audio? Is it just a trial and error? That seems like the only way unless there is some super intelligent tool, but I don't see any other way.

[qyot27]

Well, the sample is Film, and registers as such in DGIndex. As far as DGIndex is concerned, telling it to demux the audio when it saves the index file prints the delay as part of the audio file's filename. In the case of that sample, it was -326ms.

This was just the rough-and-dirty remuxing with the demuxed MPEG-2 and AC3 streams from DGIndex, but mp4box handled it fine if I fed it this (although I could only play the resulting file in VLC; for H.264/AAC, though, that problem wouldn't be there):
mp4box -add "out.demuxed.m2v":fps=23.976 -add "out T80 3_2ch 448Kbps DELAY -326ms.ac3" -delay 2=-326 "output.mp4"

So as long as you can manage to correctly handle the pulldown flagging during the transcode step and plug in the delay value, that'd work. You would need to add the aspect ratio information back in, though, unless that was dealt with automatically by your transcoding tool of choice.