Syncing audio from DVD to Bluray video

[simple_simon]

I'm trying to sync an english dub audio track from a dvd edition to the video from a bluray of the same movie. Both are progressive 23.976fps. Usually I would open both in avisynth and go to the same frame of the scene in each version and subtract the time of one from the other to get the difference and then just set the delay for the audio when muxing. But the audio drifts over the course of the film in this one so by the time it gets to the end there's a 19 frame difference. Not terrible, but noticeable. The bluray also has 25 extra black frames at the beginning compared to the dvd. The 19 frame difference is calculated after adding the 25 frames to the beginning of the dvd version to make it start in sync with the bluray version.

This is what I've come up with to slow down the audio. I'm sure there's a more elegant way to do it (maybe someone can offer suggestions) but it seems to work to my eyes.

LoadPlugin("LSMASHSource.dll")
LoadPlugin("ffms2.dll")
audio=LWLibavAudioSource("audio.ac3")
video=FFVideoSource("video.mkv", fpsnum=24000, fpsden=1001, threads=1, colorspace="YUV420P8")
both=audiodubex(video,audio)
last=BlankClip(both,Length=25)++both
n=19
fc=Framecount(last)
m=Float(fc+n)/fc
timestretch(tempo=100/m)
ChangeFPS(m*last.framerate)
assumeFPS(24000,1001)
return last

Now I want to adjust the english subtitles from the dvd in the same way but can't figure out how to do so in avisynth and don't know how to transpose the calculations to Subtitle Edit (for instance). Any ideas?

[kedautinh12]

Remux to .mkv with MKVToolNix and extract subtitles with gMKVExtractGUI

[simple_simon]

Both files are already in mkv format. I ripped the bluray & dvd using makemkv. I don't see how that helps the issue I described though?

[kedautinh12]

Both files are already in mkv format. I ripped the bluray & dvd using makemkv. I don't see how that helps the issue I described though?

After extract sub from .mkv and use vsfilter.dll to add sub in avisynth but it will hard sub. If you want softsub, you only remux sub from DVD to BD

[coolgit]

The bluray also has 25 extra black frames at the beginning compared to the dvd.

Why not trim the 25 frames from bluray version as surely that will trim the audio too.

[tebasuna51]

...Both are progressive 23.976fps.
...
The 19 frame difference is calculated after adding the 25 frames to the beginning of the dvd version to make it start in sync with the bluray version.

This is what I've come up with to slow down the audio.

1) First to slowdown all the audio (progressive async) check if the 19 frames difference (792 ms) are in only one point. Sometimes the video difference is in only one scene. If the time point (VirtualDub BD mkv) where you need add silence is 1:23:45.678 use eac3to to delay (25 frames = 1043 ms) and insert 792 ms silence:

eac3to.exe "audio.ac3" "audio_BD.ac3" +1043ms -edit=1:23:45.678,792ms -silence

Now you have the audio in sync without lossy recode.

2) If the async is really progressive you can try the TimeStretch method (losse quality), but I can't understand your AviSynth script and for what involve video in the process

LoadPlugin("LSMASHSource.dll")
audio=LWLibavAudioSource("audio.ac3").DelayAudio(1.043)
ad = AudioDuration
m = Float(ad + 0.792)/ad
timestretch(tempo=100/m)

and recode it to the new audio in sync.

Now I want to adjust the english subtitles from the dvd in the same way but can't figure out how to do so in avisynth and don't know how to transpose the calculations to Subtitle Edit (for instance). Any ideas?
In Subtitle Edit -> Synchronization ->Adjust all times and apply the delay
In Subtitle Edit -> Synchronization -> Change speed (percent) with 100+m

[simple_simon]

The 19 frame difference isn't just at one point. Once 25 frames are added to the beginning of the dvd video they start out in sync. The sync drift progressively gets worse throughout until, at the end, the dvd video is 19 frames behind the bluray, but the difference is smaller and smaller the closer to the beginning it is.

The reason my script was including video was because I wanted to base all the calculations on frames instead of time. 1) because I wasn't sure how to convert frames to time but I think I figured that out (total time/total number of frames) 2) because I'm not real familiar with editing audio in avisynth (just video) and I know that the audio isn't necessarily always the same length as the video and since I'm basing my calculations visually by the video frames I wanted to edit based on that to reduce the possibilities for errors. If I use audio duration as you suggest, then if the audio ends before the video the percentages will be off because I'm basing it on the video length.

And I had tried using subtitle edit already, pretty much as you suggested (instead I had tried using "100/m" as the percentage instead of "100+m"). The problem with that is it shifts all the subtitles by that percentage. But once the 1043ms delay is added, the subs are in sync at the beginning and only progressively go out of sync as the movie plays (just like the audio) so shifting all the subs by the same percentage just makes them go out of sync at the beginning and progressively get in sync as it plays to the end instead

[kedautinh12]

About subtitles, you can use aegisub and edit sub to match subtitles from DVD to BD

[tebasuna51]

1) because I wasn't sure how to convert frames to time

FramesPerSecond = 24/1.001 = 23.976...
Frame Duration = 1/(24/1.001) = 1.001/24 = 0.041708
19 frames = 0.792... sec
25 frames = 1.0427... sec

2)... audio isn't necessarily always the same length as the video

But the difference can't be noticeable when we calcule m, for a standard movie of 6000 seconds, same output for 10 seconds more or less

m = 6000.792/6000 = 1.000132

Instead AudioDuration you can use video duration from the VirtualDub info

And I had tried using subtitle edit already, pretty much as you suggested

In the Change speed (percent) pass you need use 100.0132 for the 6000 sec example.

[simple_simon]

In the Change speed (percent) pass you need use 100.0132 for the 6000 sec example.

So it's m times 100?

In Subtitle Edit -> Synchronization -> Change speed (percent) with 100+m

You said m+100 before.

Multiplying m by 100 gets it a lot closer but it's still off at the beginning. Like I said, after applying the 1043ms delay to account for the 25 frame difference, the subtitles are synced at the beginning and only progressively get unsynced moving forward. So applying any percentage to ALL subtitles will make the beginning become unsynced. What's needed is something to acts like timestretch does for the audio, have the beginning point remain where it is and stretch/compress the time out from that point.

[Boulder]

Use SubtitleEdit's point sync option to set timecodes for the first and last sub. If you use just two points, it will fix the starting position and then stretch or compress based on the second point.

[kedautinh12]

Time shift of aegisub can do it too :D

[StainlessS]

Boulders point sync might be called ‘visual sync’ or similar. Works well.
Mobile.

[PoeBear]

In instances like this (sources that don't match due to missing frames) I usually do one of two things, depending on if the audio source contains all or more of the video frames contained in the source video

1.) If it does, then I will create two scripts. One of just a simple LWLibavVideoSource of the video source, to have it ready for easy AvsP comparison. The second script is of the audio source with LWLibavVideoSource and LWLibavAudioSource, then a bunch of BlankClips, Trims, and Merges to get it to exact frame match the video source. The stick that into BeHappy (https://github.com/jones1913/BeHappy) and output it as a FLAC that will be frame accurate to the video source. It's so much faster than calculating all the eac3to delays, especially if you have a really troublesome source. You're also not bound by AC3 frame length anymore either, so you don't have to worry about all the cuts adding up to another audio drift. You could always output to something lossy, like AC3 again, if it's something you needed and don't mind transcoding. But FLAC, at the cost of size, will at make your edits lossless, and you won't lose any clarity

2.) If it doesn't then I essentially do the inverse on the scripts, and instead of encoding the audio, I would do a large encode of the video source with the missing frames cut out so that it's frame accurate to the audio source

Subtitles, maybe there's an easy way, but I just use my script to quickly see what time all the shifts are required, and use SubtitleEdit to shift all the subtitles after each Trim point by 41.7ms per 1 frame added/subtracted

[simple_simon]

In instances like this (sources that don't match due to missing frames) I usually do one of two things, depending on if the audio source contains all or more of the video frames contained in the source video

How do you determine the trims? Do you just step through the whole thing and compare frame by frame?

[PoeBear]

How do you determine the trims? Do you just step through the whole thing and compare frame by frame?

Not quite frame by frame. I will load them both in AvsPmod and switch between the two tabs. As long as they're the same resolution and frame length, they will share seeking progress when you switch between the tabs. I just skip through 5-10 minutes at a time until I find a section that's off, and then backtrack to find where the frames actually diverge, so I can edit my WIP script with a Trim and/or BlankClip, until I get all the misalignments. Alternatively, you could load both sources in the same script with a few more calls, and then compare them side by side with something like StackHorizontal. But since you're ultimately only going to touch one of the sources when encoding, I use the tab method

Here's a example of a final script that was to match an audio source. It was for a TV episode, and the commercial breaks were of a different length than the audio source, so it needed to be adjusted each time:
SOURCE=FFVideoSource("Source.mkv").BicubicResize(1280,720,B=-0.5,C=0.25)
OUTPUT=BlankClip(23,1280,720,"YV12",24000,1001,0)+\
SOURCE.Trim(0,11592)+\
Blackness(28,1280,720,"YV12",24000,1001,0)+\
SOURCE.Trim(11596,24422)+\
Blackness(37,1280,720,"YV12",24000,1001,0)+\
SOURCE.Trim(24436,34386)+\
Blackness(34,1280,720,"YV12",24000,1001,0)+\
SOURCE.Trim(34397,46264)+\
Blackness(36,1280,720,"YV12",24000,1001,0)+\
SOURCE.Trim(46274,53703)+\
Blackness(40,1280,720,"YV12",24000,1001,0)+\
SOURCE.Trim(53719,61145)+\
Blackness(28,1280,720,"YV12",24000,1001,0)+\
SOURCE.Trim(61149,62044)+\
Blackness(21,1280,720,"YV12",24000,1001,0)
OUTPUT
And here's how you could compare them in the same tab, obviously you wouldn't encode this:
AUDIOSOURCE=FFVideoSource("AudioSource.mkv").BicubicResize(1280,720,B=-0.5,C=0.25)
VIDEOSOURCE=FFVideoSource("VideoSource.mkv").BicubicResize(1280,720,B=-0.5,C=0.25)
OUTPUT=BlankClip(23,1280,720,"YV12",24000,1001,0)+\
VIDEOSOURCE.Trim(0,11592)+\
Blackness(28,1280,720,"YV12",24000,1001,0)+\
VIDEOSOURCE.Trim(11596,24422)+\
Blackness(37,1280,720,"YV12",24000,1001,0)+\
VIDEOSOURCE.Trim(24436,34386)+\
Blackness(34,1280,720,"YV12",24000,1001,0)+\
VIDEOSOURCE.Trim(34397,46264)+\
Blackness(36,1280,720,"YV12",24000,1001,0)+\
VIDEOSOURCE.Trim(46274,53703)+\
Blackness(40,1280,720,"YV12",24000,1001,0)+\
VIDEOSOURCE.Trim(53719,61145)+\
Blackness(28,1280,720,"YV12",24000,1001,0)+\
VIDEOSOURCE.Trim(61149,62044)+\
Blackness(21,1280,720,"YV12",24000,1001,0)
StackHorizontal(AUDIOSOURCE,OUTPUT)

There might be a more proper way to write them? But they matches my source now, and it seems to encode just as fast as without any trims

[staticmotion]

I've been encountering a similar problem, and to be honest the quickest solution I found doesn't involve Avisynth. There's a piece of software designed to align vocals in music production, called VocAlign. I tried a demo and it worked perfectly, and aligned a 50 minute TV programme in just a few minutes, with no manual intervention. Quickest software purchase I've ever made!