How to replace and/or modify audio at runtime?

[fvisagie]

Further to my recent feature request (http://forum.doom9.org/showthread.php?t=172575), is there currently any way, however brutal, of modifying (including replacing) audio at runtime?

I've written a script that overcomes Avisynth's input filter limit (i.e. running out of memory after n simultaneous clips). The script's been successfully tested on up to 500 simultaneous artificial clips and 350+ real-life ones. However, it doesn't output any audio ;).

I'd be grateful for any help in making this work. Note that this is a proof of concept that has not been cleaned up or optimised yet - the focus is on getting audio output working first.


sourcescript = "_SetSource.txt"

# "output" is the desired name of the output clip, although the way the code is currently structured makes that immaterial
AVSCat("ADDSOURCE", "output", """AudioDub(BlankClip(length=48), Tone(length=2)).Subtitle("file000").ShowFrameNumber()""", sourcescript)
AVSCat("ADDSOURCE", "output", """AudioDub(BlankClip(length=48), Tone(length=2)).Subtitle("file001").ShowFrameNumber()""", sourcescript)
AVSCat("ADDSOURCE", "output", """AudioDub(BlankClip(length=48), Tone(length=2)).Subtitle("file002").ShowFrameNumber()""", sourcescript)

AVSCat("GETSOURCE", "output", "", sourcescript)

function AVSCat (string mode, string "clipname", string "source", string "sourcescript") {
mode == "ADDSOURCE" ? AddSource(clipname, source, sourcescript) : \
mode == "GETSOURCE" ? GetSource(clipname, sourcescript) : \
Assert(false, "AVSCat: invalid 'mode' '" + mode + "'")
}

function AddSource(string clipname, string source, string sourcescript) {
startframe = Eval("""
try {
FrameCount(Eval(clipname ))
}
catch (err_msg) {
Eval("global " + clipname + " = BlankClip(length=0)")
writestr = "function SetSource(int current_frame) {"
WriteFileStart(Eval(clipname), sourcescript, "writestr", append=false)
writestr = " global " + clipname + " = \"
WriteFileStart(Eval(clipname), sourcescript, "writestr", append=true)
0
}
""")
Eval("global " + clipname + "= BlankClip(" + source + ", length=FrameCount(" + clipname + ") + FrameCount(" + source + "))")
endframe = FrameCount(Eval(clipname))-1
writestr = " current_frame >= " + String(startframe) + " && current_frame <= " + String(endframe) + " ? " + source + \
".Trim(current_frame-" + String(startframe) + ", -1) : \"
WriteFileStart(Eval(clipname), sourcescript, "writestr", append=true)
}

function GetSource(string clipname, string sourcescript) {
writestr = " NOP()"
WriteFileStart(Eval(clipname), sourcescript, "writestr", append=true)
writestr = " return output"
WriteFileStart(Eval(clipname), sourcescript, "writestr", append=true)
writestr = "}"
WriteFileStart(Eval(clipname), sourcescript, "writestr", append=true)
Import(sourcescript)
ScriptClip(Eval(clipname), "SetSource(current_frame)", show=False, after_frame=false)
}

[raffriff42]

The easiest way to handle hundreds of clips would be to concatenate them all into one, then Trim() your subclips as needed.
If the sources are AVI and hopefully intra-frame-only, you can do this live with VirtualDub "append AVI segment" + frameserving.
Otherwise you would have to create a physical intermediate file.

[fvisagie]

In my case, input clips are generally in M2TS format. Also, the typical sizes of projects make intermediate file versions very undesirable.

Both of which led me to doing the concatenation in Avisynth in the first post. However, the runtime audio problem will probably see me fitting in the VirtualDub approach for the time being.

Thanks for the suggestions.

[shekh]

If you want to concatenate m2ts in VirtualDub:
I have implemented concatenation with my input driver based on ffmpeg. However I have never tried to concatenate more than 2 files and with 300 it will eat enormous amount of memory for sure. But if you claim it solves some fundamental problem then this issue is fixable :) Link to try: https://sourceforge.net/projects/vdfiltermod/
Btw your input clips must be joinable sample-to-sample, otherwise you get audio sync problem.

[fvisagie]

If you want to concatenate m2ts in VirtualDub:
I have implemented concatenation with my input driver based on ffmpeg.

Very interesting. If I understand you and the Wiki correctly, that means you've also overcome VirtualDub's insistence on only concatenating AVI files.

However I have never tried to concatenate more than 2 files and with 300 it will eat enormous amount of memory for sure. But if you claim it solves some fundamental problem then this issue is fixable :) Link to try: https://sourceforge.net/projects/vdfiltermod/
Btw your input clips must be joinable sample-to-sample, otherwise you get audio sync problem.

Thanks. I'll let you know if this becomes my last resort, but I'm not convinced audio sync will not be a problem. Each file was created independently by the camera, and I haven't found documentation on whether or not provision is made for sample-level joining. In case you know more (or can find out more about this) than me, the camera is a Canon Legria/Vixia HF G30.