Dynamic timestretch (audio)

[ajp_anton]

I'm creating a function that gradually changes the speed of a video. It's starting to look really good visually, but I'd like to include sound as well. Is there a way to gradually also change the speed of the audio?

The way the video part works is to use trim inside scriptclip to pick the frame you want to display at any given time (or blended frames or even MV interpolated frame). Doing this for audio wouldn't be perfect, as it would have constant speed for the duration of each output frame, but it doesn't work anyway because scriptclip just passes through audio untouched. I also tried to animate timestretch, but it doesn't seem to change gradually, rather doing a sudden transition at the end of the animate range.

Is this even possible using scripts, or is an actual proper plugin needed for this?

[StainlessS]

I dont think that there is any way you can do that in script.

[johnmeyer]

You didn't say how your function works, so I don't know how you are doing it. However, Gavino posted a variable speed function in this 2011 post:

https://forum.doom9.org/showthread.php?p=1512077#post1512077

Perhaps his use of Animate could be adapted to audio. Of course, since you didn't tell us how you are doing your speed changes, maybe you are already using Animate.

BTW, I adapted Gavino's code to use MVTools2, but I have no notes as to whether I actually ever used it, so I don't know if the code was ever tested and therefore I hesitate to post it. I often write code to try something out, and it ends up in my "scripts (extra)" folder so I can go back to it without starting over. My main scripts folder contains the scripts that I've actually used on a real project.

[Gavino]

Gavino posted a variable speed function in this 2011 post:
https://forum.doom9.org/showthread.php?p=1512077#post1512077
Perhaps his use of Animate could be adapted to audio.
Thanks for the mention, John, but unfortunately Animate() doesn't do any useful animation of the audio.
(If I remember correctly, it always gives you the audio produced by applying the specified filter with the starting set of arguments.)

I think a new plugin is probably the only solution to the OP's problem.

[johnmeyer]

I think a new plugin is probably the only solution to the OP's problem.I'm sure you're right because you know more about AVISynth than I do.

I've almost never done anything with the audio in AVISynth because I have much better tools for that. Same with speed changes: Vegas Pro lets you apply "speed ramps" to audio and video so you can visually get exactly what you want. The only reason I was interested (back in 2011) in your speed change code is that AVISynth is obviously much better if you have to apply the same thing over and over again (although, since Vegas also has scripting and I've written hundreds of those scripts, that still is usually a better alternative).

I just talked to the second person in two days who just returned from vacation in Spain. They both loved it, as does everyone else who goes there. Enjoy it!

[ajp_anton]

Whoops, forgot to check this thread for a while.

Like I said, I'm using Scriptclip, so not Animate. The way Animate handled speed changes (changing the framecount) was a bit unintuitive, and while I think I finally kind of figured it out, I was already quite invested in the method I started using instead. Also the way Animate handles audio was very weird, like it tries to just pass it through but it gets messed up by the changing framecount.

Basic description of my script (it's huge - should I post it here?):
It inputs a list of pairs of numbers: [[f0,s0], ... ,[fn,sn]], where all f's and s's can be any real numbers. The f's are kind of frame numbers, but are really treated as continuous time measured in units of 1/framerate. s is a speed multiplier. The script outputs a video, where
a) it starts with frame f0 and ends with fn, but can contain frames from outside that range, depending on the parameters.
b) at the exact moment when the input frame/"timestamp" fk is shown, the speed multiplier should be sk.
c) the acceleration is finite (and limited to a maximum value), so the speed always changes smoothly with no sudden jumps.

Internally, for each segment between two points it finds a polynomial function inputframe(outputframe) that satisfies the set conditions. The polynomial can be of degree 3 if needed, for example [[100,1],[50,1]] where it starts at frame 100 going forward, decelerates, goes backwards a bit (past 50), decelerates and goes forward again, picking up speed to finally hit frame 50 at speed 1.

Finally Scriptclip uses this polynomial at each output frame to find out what input frame to display, and uses Trim to display it (with optional interpolation between input frames, works with both blend and MV).


If Scriptclip could process audio, I could probably write something that makes individual Timestretches for each frame and join them seamlessly with no cuts left between the pieces. It wouldn't change perfectly smoothly, but with high enough framerate it would be close.

[StainlessS]

If Scriptclip could process audio ...

JFYI,

Colorbars

SSS="""
Killaudio # kill audio for current frame {ie all frames, one at a time}
"""

Scriptclip(SSS) # No change to audio, still same as from Colorbars, ie No matter what Scriptclip SSS script does, audio result same as input to scriptclip.