TimeStretch tempo on Multi-channel audio

[speedyrazor]

Hi, I am using TimeStretch tempo on a Quicktime file which has multichannel audio (10 channels) but am having some issues.

Source file is a Quicktime Prores, 1920x1080, 23.98, 10 channels of 24 bit PCM. So I am converting from 23.98 HD to 25 SD keeping the audio pitch.
This is the avs script I am using:

QTInput("Prores_10_Channel_Audio.mov", quality=100, audio=2)
ColorMatrix(mode="Rec.709->Rec.601", clamp=0)
Spline64Resize(720, 576)
TimeStretch(tempo = 25.0/(24000.0/1001.0)*100.0)
AssumeFPS(25, 1)

The output file has lots of 'phasing' between the channels, for example, if I just listen, and look, at the Stereo left and right, there is a slight difference where the TimeStretch tempo has 'cut n shut' the audio, this is understandable, but not a nice effect. Is there a way to apply TimeStretch tempo to all channels as a whole, not individually, thus hopefully eliminating the 'phasing' effect?
Or is there a better way to 'stretch' the audio, with a different plugin maybe?

Kind regards.

[Wilbert]

The output file has lots of 'phasing' between the channels, for example, if I just listen, and look, at the Stereo left and right, there is a slight difference where the TimeStretch tempo has 'cut n shut' the audio, this is understandable, but not a nice effect. Is there a way to apply TimeStretch tempo to all channels as a whole, not individually, thus hopefully eliminating the 'phasing' effect?
No there isn't. You will get this problem with multichannel audio (just stereo should work fine). See: http://forum.doom9.org/showthread.php?t=71632
I think the current sources are based on SoundTouch 1.4.0. There are a lot of improvements (http://www.surina.net/soundtouch/README.html) since that time, so it will be worthwhile to update the library. However in the changelog of the library I see nothing about multichannel, so i can't tell whether they fixed it or not.

You can always ask them and report back to us ;)

edit: sounds like that added it a year ago (https://bugzilla.mozilla.org/show_bug.cgi?id=847827#c20). Will try to update the library when I have some time ...

[speedyrazor]

Thanks for your reply, nice to know it has been fixed. Is there anyway I can use the latest version of SoundTouch now with Avisynth? I see they have a SoundTouch DLL library 1.8.1 for Windows, can I put the SoundTouch.dll in the Avisynth plugins folder and use?

Kind regards.

[Wilbert]

No, that won't work. It links statically with soundtouch.lib. So i need to update the library and header file and recompile AviSynth. Perhaps it is better to make a plugin out of it.

[speedyrazor]

Hi Wilbert, thanks for your reply. Sorry to ask, but would you be updating the library and header file and recompiling AviSynth soon? Im itching to try out this with my avisynth scripts?

Kind regards.

[Wilbert]

Please try and report your findings: http://www.wilbertdijkhof.com/avisynth_soundtouch_1.8.0.zip

[hello_hello]

I tried the new Avisynth with a few samples and listened to the TimeStretched version after downmixing to mono on playback, and aside from the speed difference I can't pick the timestretched version from the original. I don't have a very good sound system but so far the obvious phasing seems to be gone. Thank you.

The only downside for me is MeGUI uses Avisynth+ by default and that requires a separate TimeStretch.dll plugin. Is Avisynth+ still being developed or is it pretty much dead?

Now I wonder if someone will update the foobar2000 SoundTouch plugin. The current version only works with stereo audio (for obvious reasons) but it's also only accurate to two decimal places (time stretching specified as a percentage) and even then I don't think it's particularly accurate anyway. I guess I can only dream...... :)

[speedyrazor]

Please try and report your findings: http://www.wilbertdijkhof.com/avisynth_soundtouch_1.8.0.zip

Wilbert, you're a star! Many thanks for taking the time to do this.
I will report back once I've run some tests.

Kind regards.

[sneaker_ger]

Anyone know how SoX handles the multi-channel scenario?

[speedyrazor]

Please try and report your findings: http://www.wilbertdijkhof.com/avisynth_soundtouch_1.8.0.zip

Hi Wilbert, it works a treat! Many thanks for taking the time to re-compile this. I don't suspect it is, but thought I would ask, is this the MT version?

Kind regards.

[Wilbert]

That's great to hear!
I don't suspect it is, but thought I would ask, is this the MT version?
No it isn't. It's based on the regular RC1. I will include the modified sources tomorrow.

[speedyrazor]

Hi Wilbert, will this be in the official RC2?

[speedyrazor]

That's great to hear!

No it isn't. It's based on the regular RC1. I will include the modified sources tomorrow.

Hi Wilbert, any news on this, you did an excellent job, your dll works a treat on multichannel audio. Can this be 'plugged' into the next official release?

Kind regards.

[speedyrazor]

Hi, any news on the SoundTouch Library 1.8 getting into the main 2.6 and MT versions please?

Kind regards.

[Wilbert]

http://forum.doom9.org/showthread.php?p=1719412#post1719412

[speedyrazor]

http://forum.doom9.org/showthread.php?p=1719412#post1719412

Wilbert, that would be amazing to get a plugin to do this, I can't wait.
sorry to ask, but is this something you're planning on doing soon?

Kind regards.

[Wilbert]

I hope to get a version within a few weeks.

[speedyrazor]

I hope to get a version within a few weeks.

Great, I will keep an eye out for it.

[speedyrazor]

I hope to get a version within a few weeks.

Hi Wilbert, hows it going with the plugin?

Kind regards.

[Wilbert]

link: http://www.wilbertdijkhof.com/TimeStretch_v258.zip

license: gpl

documentation: http://avisynth.nl/index.php/TimeStretch

changelog:
* version: 2.5.8.0 (requires at least AviSynth v2.58)
* filter is named to TimeStretchPlugin
* multichannel support
* updated SoundTouch library to 1.8.0

usage:

LoadPlugin("D:\AviSynth\Plugins\TimeStretch\Release\TimeStretch.dll")
v = BlankClip()
a = NicMPG123Source("D:\AviSynth\Plugins\TimeStretch\sample_orig.mp3", false)
c = audiodub(v,a)
c.TimeStretchPlugin(pitch=100*pow(2.0, 3.0/12.0))


Let me know if you find something. I cleaned the code up a bit, but it should behave the same (except that it will return an error if there is no audio in the clip).

[speedyrazor]

link: http://www.wilbertdijkhof.com/TimeStretch_v258.zip

license: gpl

documentation: http://avisynth.nl/index.php/TimeStretch

changelog:
* version: 2.5.8.0 (requires at least AviSynth v2.58)
* filter is named to TimeStretchPlugin
* multichannel support
* updated SoundTouch library to 1.8.0

usage:

LoadPlugin("D:\AviSynth\Plugins\TimeStretch\Release\TimeStretch.dll")
v = BlankClip()
a = NicMPG123Source("D:\AviSynth\Plugins\TimeStretch\sample_orig.mp3", false)
c = audiodub(v,a)
c.TimeStretchPlugin(pitch=100*pow(2.0, 3.0/12.0))


Let me know if you find something. I cleaned the code up a bit, but it should behave the same (except that it will return an error if there is no audio in the clip).

And here it is.... You're a star Wilbert. I will do some testing and report back. Thanks for all your hard work.

Kind regards.

[speedyrazor]

I have taken the file TimeStretch.dll and put it in my Avisynth Plugins folder, If I try to run a script I get 'Script Error: there is no function named “TimeStretchPlugin”'. I am using:

TimeStretchPlugin(tempo = 25.0/24.0*100.0)

[manolito]

Weird, works fine here...
(AviSynth 2.60 RC3, WinXP)

Have you tried to load the TimeStretch manually in your script with "LoadPlugin" ?


Cheers
manolito

[speedyrazor]

If I use:
LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\TimeStretch.dll")
then I get:
LoadPlugin: unable to load "C:\Program Files (x86)\AviSynth\plugins\TimeStretch.dll", Module not found. Install missing library?

[manolito]

Rechecked it again:

No error if I have the plugin in the AviSynth\Plugins folder.
Manually loading the plugin also works for me...


Cheers
manolito

[Wilbert]

I compiled it with VC2010 express. So you need to install Microsoft Visual C++ 2010 Redistributable Package (x86) (https://www.microsoft.com/en-us/download/details.aspx?id=5555). Let me know if that fixed it.

[speedyrazor]

I compiled it with VC2010 express. So you need to install Microsoft Visual C++ 2010 Redistributable Package (x86) (https://www.microsoft.com/en-us/download/details.aspx?id=5555). Let me know if that fixed it.

That would explain why it is working on another system I have (must already have MVC 210 installed)! I would have NEVER guessed. Cheers.

[speedyrazor]

I have found an issue with this plugin, in that whilst time stretching from 23.98 to 25, i small blip / glitch is being introduced.
I have been using this for a while now, and it's been great until now.
Can I send you a before and after sample, to see if this issue can be rectified please?

Kind regards.

[Wilbert]

Yes sure, but if it is a bug in soundtouch itself i won't be able to fix it. Can you say a bit more when it shows up? Only when doing upconversions? both stereo or multichannel? If it shows up with stereo too, does this issue also exist with the older version of soundtouch?

[speedyrazor]

Yes sure, but if it is a bug in soundtouch itself i won't be able to fix it. Can you say a bit more when it shows up? Only when doing upconversions? both stereo or multichannel? If it shows up with stereo too, does this issue also exist with the older version of soundtouch?

Hi Wilbert, where can I send these files, they will be video and multichannel audio?

Kind regards.

[speedyrazor]

Hi, just looking at the command line Windows version of SoundStretch V1.9, there are a few switches, -quick, -naa, -speech, and I was wondering which ones you used for this plugin, and if they may have an effect on the issue I am seeing.

This is the command I am generally using for everything going from 23.976 to 25:

TimeStretchPlugin(tempo = 25.0/(24000.0/1001.0)*100.0)

Could my command be an issue?

Kind regards.

[manolito]

Just curious, have you tried to use avisynth.dll from this post
http://forum.doom9.org/showthread.php?p=1710608#post1710608
instead of the plugin version?

If your issue is also present using this version of avisynth.dll then it should be pretty clear that the issue has nothing to do with Wilbert's plugin but that it lies in SoundTouch 1.8.0 itself.

Your TimeStretchPlugin command looks OK to me...


Cheers
manolito

[Wilbert]

Hi Wilbert, where can I send these files, they will be video and multichannel audio?

Kind regards.

Please check your pm.

Hi, just looking at the command line Windows version of SoundStretch V1.9, there are a few switches, -quick, -naa, -speech, and I was wondering which ones you used for this plugin, and if they may have an effect on the issue I am seeing.
The problem is that a decent description of their api is missing, at least i can't find it on their site.

Dunno about the speech switch.

You can set the switches in the plugin:

c[tempo]f[rate]f[pitch]f[sequence]i[seekwindow]i[overlap]i[quickseek]b[aa]i


// sequence
if (args[0].Defined()) sampler->setSetting(SETTING_SEQUENCE_MS, args[0].AsInt());
// seekwindow
if (args[1].Defined()) sampler->setSetting(SETTING_SEEKWINDOW_MS, args[1].AsInt());
// overlap
if (args[2].Defined()) sampler->setSetting(SETTING_OVERLAP_MS, args[2].AsInt());
// quickseek
if (args[3].Defined()) sampler->setSetting(SETTING_USE_QUICKSEEK, args[3].AsBool() ? 1 : 0);
// aa (i guess this is the same as the naa setting, must be a multiple of 4)
if (args[4].Defined()) {
int i = args[4].AsInt();
if (i<0 || i%4 != 0)
env->ThrowError("TimeStretchPlugin: AntiAliaser filter length must divisible by 4.");

if (i)
sampler->setSetting(SETTING_AA_FILTER_LENGTH, i);
else
sampler->setSetting(SETTING_USE_AA_FILTER, 0);
}


If you don't supply these values, i guess the library sets default values. But i don't know what these default values are or where exactly they are set. Will check later.

[speedyrazor]

Please check your pm.

Hi Wilbert, pm sent, let me know how you get on.

[manolito]

Did you guys make any progress on this?

I did some testing a while ago with applying timestretching on demanding audio sources, and I finally decided to ditch pitch correction altogether because even ProsoniQ TimeFactory introduced too many artifacts for me.

Now I searched the web again for this topic, and most of the studio pros seem to agree that today iZotope Radius offers the best quality. Radius is currently only available for Macs, but iZotope did license the algorithm to CakeWalk (Sonar 6) and Adobe (Audition).

Incidentally you can get Adobe Audition 3 (almost) legally from Adobe itself:
http://homerecording.com/bbs/general-discussions/digital-recording-and-computers/get-adobe-audition-3-0-free-legally-352167/

The included manual is not too informative, but there is a good help page on the web:
https://helpx.adobe.com/audition/using/time-pitch-manipulation-effects.html


Cheers
manolito

[Wilbert]

Hi Wilbert, pm sent, let me know how you get on.
I have been busy with other things. Hopefully i will have some time next weekend.

[speedyrazor]

Hi Wilbert, just to let you know I don't actually think there is a problem with the plugin. After further investigation I think that the last stereo pair, which was the dolby E track, was causing the issue, trying to include this noisy data track in the re-pitching process, this gave very strange results. what would be ideal is to be able to select groups of audio tracks to be re-pitched together. Some Tracks will never be heard with other tracks, so they can be re-pitched independently.

[Motenai Yoda]

Dunno about the speech switch.



if (params->speech)
{
// use settings for speech processing
pSoundTouch->setSetting(SETTING_SEQUENCE_MS, 40);
pSoundTouch->setSetting(SETTING_SEEKWINDOW_MS, 15);
pSoundTouch->setSetting(SETTING_OVERLAP_MS, 8);
fprintf(stderr, "Tune processing parameters for speech processing.\n");
}

/// Default values for sound processing parameters:
/// Notice that the default parameters are tuned for contemporary popular music
/// processing. For speech processing applications these parameters suit better:
/// #define DEFAULT_SEQUENCE_MS 40
/// #define DEFAULT_SEEKWINDOW_MS 15
/// #define DEFAULT_OVERLAP_MS 8
///

But I don't know if those settings will be auto-tuned as the default ones if they are.
/// Calculates processing sequence length according to tempo setting
void TDStretch::calcSeqParameters()
{
// Adjust tempo param according to tempo, so that variating processing sequence length is used
// at varius tempo settings, between the given low...top limits
#define AUTOSEQ_TEMPO_LOW 0.5 // auto setting low tempo range (-50%)
#define AUTOSEQ_TEMPO_TOP 2.0 // auto setting top tempo range (+100%)

// sequence-ms setting values at above low & top tempo
#define AUTOSEQ_AT_MIN 125.0
#define AUTOSEQ_AT_MAX 50.0
#define AUTOSEQ_K ((AUTOSEQ_AT_MAX - AUTOSEQ_AT_MIN) / (AUTOSEQ_TEMPO_TOP - AUTOSEQ_TEMPO_LOW))
#define AUTOSEQ_C (AUTOSEQ_AT_MIN - (AUTOSEQ_K) * (AUTOSEQ_TEMPO_LOW))

// seek-window-ms setting values at above low & top tempoq
#define AUTOSEEK_AT_MIN 25.0
#define AUTOSEEK_AT_MAX 15.0
#define AUTOSEEK_K ((AUTOSEEK_AT_MAX - AUTOSEEK_AT_MIN) / (AUTOSEQ_TEMPO_TOP - AUTOSEQ_TEMPO_LOW))
#define AUTOSEEK_C (AUTOSEEK_AT_MIN - (AUTOSEEK_K) * (AUTOSEQ_TEMPO_LOW))

#define CHECK_LIMITS(x, mi, ma) (((x) < (mi)) ? (mi) : (((x) > (ma)) ? (ma) : (x)))

double seq, seek;

if (bAutoSeqSetting)
{
seq = AUTOSEQ_C + AUTOSEQ_K * tempo;
seq = CHECK_LIMITS(seq, AUTOSEQ_AT_MAX, AUTOSEQ_AT_MIN);
sequenceMs = (int)(seq + 0.5);
}

if (bAutoSeekSetting)
{
seek = AUTOSEEK_C + AUTOSEEK_K * tempo;
seek = CHECK_LIMITS(seek, AUTOSEEK_AT_MAX, AUTOSEEK_AT_MIN);
seekWindowMs = (int)(seek + 0.5);
}

// Update seek window lengths
seekWindowLength = (sampleRate * sequenceMs) / 1000;
if (seekWindowLength < 2 * overlapLength)
{
seekWindowLength = 2 * overlapLength;
}
seekLength = (sampleRate * seekWindowMs) / 1000;
}