Audio Out of Sync After Encoding

[MysteryX]

Most of the times, encoding is working fine. In some cases, however, audio and video are out of sync after muxing back the original audio into the encoded video with FFMPEG.

Original video (https://mega.nz/#!uUQkVaLA!HfKXpPJqL1xQdCQr7oURn0-L-fV0mG8Zl3ogMaCU77A)

Video encoded with x265, muxed with the original MP2 audio (https://mega.nz/#!bcJShRyC!YD0KzsjKY3rOjw-l3bH-PoPWcavU8uPJlC-9qmqanQs)

Any idea why this is happening and how to avoid this?

[kuchikirukia]

Don't know what you did to the video. Delaying the audio by 500ms in mkvmerge works.

[tebasuna51]

File : EncodeSyncOld.mpg EncodeSyncNew.mkv
---------- ---------------------------- ------------------------
Video
Format : MPEG Video CustomMatrix/BVOP HEVC Main@L4@Main
Frames : 5269 <> 12631
Framerate : 25.000 Progressive <> 59.842
Duration_V : 00:03:30.760 (210760 ms) <> 00:03:31.072 (211072 ms)
Resolution : 352x288 1244x720

Audio
Format : MPEG Audio Layer 2 = MPEG Audio Layer 2
Duration_A : 00:03:30.782 (210782 ms) = 00:03:30.782 (210782 ms)
Delay_vid. : 0 = 0

You can't preserve the sync audio/video if you change the num_frames in a distinct ratio (2,39723) than fps (2,39368).

Seems a insane video conversion (fps = 59.842 ??), more frames than original in a progressive video??

[MysteryX]

You can't preserve the sync audio/video if you change the num_frames in a distinct ratio (2,39723) than fps (2,39368).

Seems a insane video conversion (fps = 59.842 ??), more frames than original in a progressive video??
InterFrame increases the frame rate.

It works fine with most other videos.

I tried both 29.97*2=59.94 (well... 59.842 isn't that far from the target...) and with 60fps. With 60fps, I would get dropped frames and skips, not sure why.

Looking at this, it's clear the problem is with the video encoding. For some reason, it encoded at a slightly slower frame rate than requested.

Looking at the encoding window, x265 detects the input stream as 59.94. Could it be that there's a bug in FFMPEG's muxer than altered the video frame rate?

Looking at other encoded videos, one is at 59.928, 59.935, one is at 59.936 ... they're all approximations

[MysteryX]

I tried encoding with Double Frame Rate. This time, I got 49.929... still the same problem.

I get rid of the ghost effect by removing InterFrame's "Smooth" preset, but 2x Frame Rate gives a quality flickering that I find irritating to the eyes.

[MysteryX]

This time I encoded at 60fps. The video came out as 59.842fps

At 25fps, however, the video came out as 25fps as expected.

[MysteryX]

I tested with muxing into both MKV and MP4 format and both had this problem with x265. Both formats work fine with x264.

I updated my FFMPEG library to the latest. This time, the MP4 output is 59.94 while the MKV output is 59.663.

Definitely looks like a bug in the FFMPEG library with x265.

First I generate a .264 stream. There it is 59.94.

Then I convert video into MP4 container because FFMPEG fails to muxe 264 streams directly into MKV. There it still is 59.94.

Then I convert to MKV and merge back with original audio. There it becomes 59.663.

However, I find it curious to observe that the 264 stream contains PCM audio *and* AAC audio when I'm piping to FFMPEG via avs2yuv... and there I thought it was only passing the video? And looking at the code, I do explicitely pass "-an" to FFMPEG so I have no idea how it still processes the audio!

[MysteryX]

It happens both with x264 and x265 when muxing into MKV. I posted about this bug on the FFMPEG forum. (http://ffmpeg.gusari.org/viewtopic.php?f=11&t=2576)

[sneaker_ger]

It happens both with x264 and x265 when muxing into MKV. I posted about this bug on the FFMPEG forum. (http://ffmpeg.gusari.org/viewtopic.php?f=11&t=2576)
The mp4 is technically speaking using a variable framerate. On top of that the video has a negative delay via an edit box. ffmpeg skews all timings so that the file starts at 0 again, the negative delay is lost. ("negative delay" means the player has to skip a part of the video at the start)

Do not even attempt to just look at MediaInfo values, this is too complicated for that.

[MysteryX]

Why is this so f*cked up? I'm sending standard AVS output to FFMPEG for encoding and my video source shouldn't have any impact.

[geometer]

When I get sync problems, (guess: it seems that some information in an aac stream can disturb the muxer), it often helps when I decode the audio to WAV. Then I compare the duration of the video with that. Sometimes I edit the WAV with cutting or adding silence. Further, when the original was segmented into chapters, but gets linearized here, silence between chapters may come out wrong. I can measure every chapter duration and correct the audio correspondingly.
Then, I do a final mux with AviDemux, and have it re-encode the audio to mp3. This will create a working video almost everytime.
But probably it won't work though, when the video has variable frame rate or complex timing information. Then, mkvtoolnix can recompute timing information, and sometimes improves the situation.

[MysteryX]

Why is it producing a variable frame rate to begin with? If I first encode my source video to AVI, I'm sure to be starting from a fixed bit rate, yet that doesn't help.

[sneaker_ger]

What is "it"?
You have to be way more specific about what you are doing, what you are expecting and what you think is wrong.

[geometer]

Why is it producing a variable frame rate to begin with? If I first encode my source video to AVI, I'm sure to be starting from a fixed bit rate, yet that doesn't help.

but would you like to check the WAV method, just to know whether it's the audio itself messing up?

what processing steps are you doing so far?

[MysteryX]

This is my command to encode
Encoder\avs2yuv.exe "Job1_Script.avs" -o - | Encoder\ffmpeg.exe -y -i - -an -c:v libx265 -preset medium -crf 22 "Job1_Output.264"

Perhaps the audio is messing things up? FFMPEG is ignoring the "-an" parameter and is still passing along the audio. Is there a way to prevent avs2yuv from passing the audio? I didn't even know until today that it was passing the audio.

[sneaker_ger]

Avs2yuv cannot pass audio, let alone interleaved with video. You must be confusing things. Also, result of AviSynth is not variable framerate.

[geometer]

BTW is there an orphaned dash or minus sign?

[MysteryX]

BTW is there an orphaned dash or minus sign?
"-i -" means standard input stream

If avs2yuv isn't supposed to pass the audio, then how the heck does the audio get passed along? I'm using an old version of avs2yuv btw

[geometer]

if the whole process is not meant to do audio, then you need 2-3 passes more with other tools.
given the video part is ready.
1. demux and 2. optionally decode the audio from the source. write down the audio track as its own file!
decoding to WAV is recommended when the timing is flawed, or audio needs editing.
3. remux the audio into the video. a tool like avidemux can encode the audio on the fly, and set a final audio delay.

[MysteryX]

if the whole process is not meant to do audio, then you need 2-3 passes more with other tools.
given the video part is ready.
1. demux and 2. optionally decode the audio from the source. write down the audio track as its own file!
decoding to WAV is recommended when the timing is flawed, or audio needs editing.
3. remux the audio into the video. a tool like avidemux can encode the audio on the fly, and set a final audio delay.
I'm already doing that.

Yet something gets messed up along the way, and I don't understand how the audio gets passed to the encoder.

[geometer]

when you remux after encoding, then it is irrelevant what the encoder does with the audio or not.

[MysteryX]

Yeah it's irrelevant and that's why I hadn't realized that before. But now that I see that the encoding/muxing process is messing up, I have to figure out where it's going wrong. And ideally I'd want to avoid having PCM audio in the output stream to save on disk space requirements and data transfers.

Yet with the command I posted, I know for sure the audio is being written into the file at the same time as the video because when I open the .264 file in MPC-HC while it's still encoding, I do get the audio playing.

[Desbreko]

.264 files are raw H.264 bitstreams. They can't contain audio. I'd guess that MPC-HC is autoloading an external audio file in the same folder when you play it.

[manono]

Since this is no longer an AviSynth-specific problem, if it ever was, I'll close this thread. If you disagree, MysteryX, you can PM me.