Audio sync lost when playing grid of 4 movies

[wrybread]

I'm trying to play 4 videos in a 2x2 grid. Everything works great, except that I lose audio syncing when I introduce more than 2 videos.

Here's my code for a 2 video grid, which works:


# load the clips
a = DirectShowSource("D:\AVISynth\000119.mov")
b = DirectShowSource("D:\AVISynth\000120.mov")

# convert to rgb32
a = a.ConvertToRGB32()
b = b.ConvertToRGB32()

# this version is a 2x1 stack, and audio sync works
stack = StackHorizontal(a, b)

# blend the audio
audio = MixAudio(a, b, 1, 1)
stack = AudioDub(stack, audio)

return stack

However, when I do a 2x2 grid, I lose audio sync. Here's my code for that:


# load the clips
a = DirectShowSource("D:\AVISynth\000119.mov")
b = DirectShowSource("D:\AVISynth\000120.mov")
c = DirectShowSource("D:\AVISynth\000121.mov")
d = DirectShowSource("D:\AVISynth\000122.mov")

# convert to rgb32
a = a.ConvertToRGB32()
b = b.ConvertToRGB32()
c = c.ConvertToRGB32()
d = d.ConvertToRGB32()

# resizing doesn't seem to help
#a = a.BilinearResize(320,240)
#b = b.BilinearResize(320,240)
#c = c.BilinearResize(320,240)
#d = d.BilinearResize(320,240)

# this version is a 2x2 stack, and audio syncing is lost
stack = StackVertical(StackHorizontal(a, b), StackHorizontal(c, d))

# blend the audio
audio = MixAudio(a, b, 1, 1)
stack = AudioDub(stack, audio)

return stack

If anyone wants to try with the actual videos, here they are:

http://sinkingsensation.com/dropbox/vids.zip

They're each 640x480, 60fps. I tried changing their FPS to 30 using ChangeFPS(30), but it didn't seem to make a difference.

I know shooting at 60fps is excessive, but that's how my camera (Canon T2i DSLR) shoots the video, and for this project I can't re-encode them.

Does anyone have any tips on how to improve the audio syncing? Maybe some more efficient way of stacking the videos? Or some other clever suggestion?

Thanks for any tips.

[IanB]

I see nothing obvious in your script.

If you are trying to play this in real time then, you are asking a lot of your processor. Leaving the data in YV12 (or YUY2) as returned by DSS may save a few cycles, but just decoding 4 streams at once is a big ask.

If you are encoding this for later viewing, then we need to look deeper into why the audio is being skewed.

[wrybread]

Aha! Keeping the videos as YUY2 (I think that's their format) completely fixed the syncing.

I now need to independently change the color of each video, like an Andy Warhol effect. Can I do that while keeping the YUY2 color space, or do I need to convert to RGB to do that?

Also, I was wondering if there a playback performance benefit to converting the fps to 24 when importing the video, as in this statement:

a = DirectShowSource("D:\AVISynth\000119.mov", fps=24, convertfps=true)

Also, any tips on pre-rendering the video with AVISynth so I'm not rendering everything on the fly?