PDA

View Full Version : Realtime audio source, with sync problems


NerdWithNoLife
30th January 2010, 00:56
I almost got a method for using AviSynth to animate live, based on the amplitude of the signal coming from a microphone. Audio was loaded into the script from a filter graph that is simply the sound card's capture source.

It works, with little CPU usage, but there's a pretty big problem of audio sync. After some research, it appears DirectShowSource has a default buffer of at least 500ms. I cannot for the life of me find a way to modify the thing called IAMBufferNegotiation, which would theoretically solve the problem.

Since this is being done live, DelayAudio will not help, barring the possibility of time travel. This seems to be a pretty advanced question regarding GraphEdit/GraphStudio, so if anyone has any idea how to fix this, please do your best to dumb it down for me.

NerdWithNoLife
31st January 2010, 19:58
Aha. Found the solution. Instead of adding it as an Audio Capture Source, I just had to insert a filter for my sound card under WDM Streaming System Device. And that's it. End of filter graph.
http://img683.imageshack.us/img683/7986/dooml.th.png (http://img683.imageshack.us/i/dooml.png/)
Then with the following script, I can animate a character's mouth with live audio (i.e. Conan O'Brien's live via satellite segments)!
#Load Sources
Aud=DirectShowSource("c:\path\graph.grf",video=false,framecount=(17982),fps=30,seek=false)
m00=ImageSource("c:\path\00.png",pixel_type="rgb32",0,0)
m25=ImageSource("c:\path\25.png",pixel_type="rgb32",0,0)
m50=ImageSource("c:\path\50.png",pixel_type="rgb32",0,0)
m75=ImageSource("c:\path\75.png",pixel_type="rgb32",0,0)
m100=ImageSource("c:\path.png",pixel_type="rgb32",0,0)
#Set Parameters
FRateNumerator = 30000
FRateDenominator = 1001
High = -6
Low = -24
Steps = 3
#Set up clip
Gap = (Low-High)/Steps
AudLength=AudioLengthF(Aud)/AudioRate(Aud)
Slug=BlankClip(m00).AssumeFPS(FRateNumerator,FRateDenominator).loop()\
.trim(0,Ceil(AudLength*FRateNumerator/FRateDenominator))
Mouth=AudioDub(slug,aud)
#"Animate" Clip
ScriptClip(Mouth,"AM=AudioMax(1)
AM >= High+Gap*0 ? m100 \
: AM >= High+Gap*1 ? m75 \
: AM >= High+Gap*2 ? m50 \
: AM >= High+Gap*3 ? m25 \
: m00")

KillAudio()

Trim(0,-9999)
This is an adaption of the discussion in this thread (http://forum.doom9.org/showthread.php?t=141993).

Edit: Added seek=false to the call of DirectShowSource, for added stability.