Log in

View Full Version : [OPEN SOURCE] AudioAlign: Audio Synchronization And Analysis Tool


PatchWorKs
11th December 2023, 17:40
After more than 5 years, an updated version (now with prebuild Windows binary) finally out:

AudioAlign is a research tool to investigate automatic synchronization of audio and video recordings that have either been recorded in parallel at the same event or contain the same aural information. It is designed as a GUI for the Aurio (https://github.com/protyposis/Aurio) library.

To see what AudioAlign can do, check the demo videos linked below.
https://raw.githubusercontent.com/protyposis/AudioAlign/main/audioalign1.png

Use cases
AudioAlign has been developed for a research project with the goal to automatically synchronize audio and video recordings, recorded at the same time at the same event, e.g. a speech or a music concert. The idea was to synchronize all those videos taken from the crowd and combine them to


generate multicamera cuts by switching between perspectives (video (https://www.youtube.com/watch?v=QXQVPXjR3Lc)),
create videos with full or at least better event coverage,
replace bad quality audio or video tracks with better ones, or
detect interesting moments (where many recordings have been captured at the same time).
For more, see the readme (https://github.com/protyposis/AudioAlign/blob/main/README.md).

AudioAlign can be used for a lot more uses cases though, including

video mashups (video (https://www.youtube.com/watch?v=cdv4-gOxxZ0))
comparison of live performances (video (https://www.youtube.com/watch?v=4yUSLa4K3GE))
synchronization of different cover interpretations (video (https://www.youtube.com/watch?v=Jo2XPXUmkK0))
voice dubbing (video (https://www.youtube.com/watch?v=f89isFfLgvg))
ground truth creation (website (http://protyposis.github.io/JikuMVD-SynchronizationGroundTruth/))
evaluation of fingerprinting algorithms
...

What's new
1.5.1
- silent startup crash on missing native dependency

1.5.0
- allow skipping of missing project assets on load [features]
- improve scheduling of many CC operations [features]
- display number of matches [features]
- bump Aurio to 4.0.0 (incl. FFmpeg (https://www.videohelp.com/software/ffmpeg) 6.0) [features]

See CHANGELOG (https://github.com/protyposis/AudioAlign/blob/main/CHANGELOG.md)

[B]Support
For questions and issues, please open an issue on the issue tracker. Commercial support, development and consultation is available through Protyposis Multimedia Solutions (https://protyposis.com/).

License
Copyright (C) 2010-2023 Mario Guggenberger mg@protyposis.net. This project is released under the terms of the GNU Affero General Public License. See LICENSE (https://github.com/protyposis/AudioAlign/blob/main/LICENSE) for details.


Official git: https://github.com/protyposis/AudioAlign/
Download link: https://github.com/protyposis/AudioAlign/releases/
.NET SDK 6 (needed): https://dotnet.microsoft.com/en-us/download/visual-studio-sdks

PatchWorKs
29th January 2024, 09:59
New Aurio and AudioAlign versions are out !

Aurio v4.2.0 changelog:
Features

bump FFmpeg (https://www.videohelp.com/software/ffmpeg) to 6.1.1
direct audio proxy file writing
improve FFmpeg seeking
read FFmpeg samples without intermediary buffer


Bug Fixes

exception on FFmpeg proxy file creation
infinite read loop at end of FFmpeg stream


https://github.com/protyposis/Aurio/releases/tag/v4.2.0

AudioAlign v1.6.0 changelog:
Features

bump Aurio to 4.2.0
https://github.com/protyposis/AudioAlign/releases/tag/v1.6.0

Frank62
29th January 2024, 11:22
Will this work maybe also for different dubs in different languages somehow?

PatchWorKs
31st January 2024, 17:28
Will this work maybe also for different dubs in different languages somehow?
Maybe (never tested for this purpose personally), anyway consider FFmatch (https://github.com/fsinapsi/ffmatch?tab=readme-ov-file#ffmatch) (downloads here (http://fsinapsi.altervista.org/code/ffmatch/)) for...

PatchWorKs
31st January 2024, 17:31
More bugfixes, more fun !

AudioAlign v1.7.0 (https://github.com/protyposis/AudioAlign/releases/tag/v1.7.0)
Features

selective match flipping


Bug Fixes

alignment graph plots offsets incorrectly



AudioAlign v1.6.1 (https://github.com/protyposis/AudioAlign/releases/tag/v1.6.1)

Bug Fixes

bump Aurio to 4.2.1
missing sign on alignment graph offset axis


Aurio v4.2.1 (https://github.com/protyposis/Aurio/releases/tag/v4.2.1)
Bug Fixes

FFmpeg end-of-stream handling breaks stream contract
FFmpeg sometimes seeks one frame too far
initial position not zero when first PTS not zero
wrongly accepted first PTS when file not seekable

Frank62
1st February 2024, 12:54
Maybe (never tested for this purpose personally), anyway consider FFmatch (https://github.com/fsinapsi/ffmatch?tab=readme-ov-file#ffmatch) (downloads here (http://fsinapsi.altervista.org/code/ffmatch/)) for...
Thanks for the hint! But FFMatch I tried several years ago, with not much success.