Audio Development

[Myrsloik]

Audio support is finally mostly done. See this blog post (http://www.vapoursynth.com/2020/01/audio-support-and-how-it-works/) for a longer explanation and list of what's currently implemented.

Test5 installer (x64) (https://www.dropbox.com/s/l16vtaj4twpq8vx/VapourSynth64-R51-audio5.exe?dl=1)
Test5 portable (x64)
(https://www.dropbox.com/s/kpcu7yqvdtoeuqe/VapourSynth64-Portable-R51-audio5.7z?dl=1)

Test4 installer (x64) (https://www.dropbox.com/s/tcac6i210penql6/VapourSynth64-R51-audio4.exe?dl=1)
Test4 portable (x64)
(https://www.dropbox.com/s/2pwk662cwn1kzi8/VapourSynth64-Portable-R51-audio4.7z?dl=1)

BestAudioSource thread (https://forum.doom9.org/showthread.php?t=177337)

[DJATOM]

Looks like 1 audio frame has a length of 2 seconds and I can't trim less than that. clip2 = clip2.std.AudioTrim(0, 95999) works, while clip2 = clip2.std.AudioTrim(0, 47999) fails to output any data.

[Myrsloik]

Looks like 1 audio frame has a length of 2 seconds and I can't trim less than that. clip2 = clip2.std.AudioTrim(0, 95999) works, while clip2 = clip2.std.AudioTrim(0, 47999) fails to output any data.

Doh, the one case I forgot to test when passing through audio frames. The files in the first post have been discretely updated.

Audio frames are currently 96000 samples so that's well spotted.

[DJATOM]

Now it trims as intended. Great!

[lansing]

I got the "No attribute with the name bas exists" after running the installer

[Myrsloik]

I got the "No attribute with the name bas exists" after running the installer

Did you actually get the plugin?

[lansing]

Did you actually get the plugin?

I don't think I have, I checked "core/plugins", there's no dll with the name "bas".

[AzraelNewtype]

I don't think I have, I checked "core/plugins", there's no dll with the name "bas".

Did you... click the Best Audio Source Thread link in the OP and manually grab the plugin, which the question was phrased specifically to indicate was not actually built in and therefore would not be in core/plugins?

[lansing]

Did you... click the Best Audio Source Thread link in the OP and manually grab the plugin, which the question was phrased specifically to indicate was not actually built in and therefore would not be in core/plugins?

Ok I got it, I thought the plugin was included in the installer

[lansing]

Playing back the vpy script in media player classic is choppy, audio constant skips

[DJATOM]

That problem only occurs with w64 header, raw output is fine.

[Pat357]

How to choose to output W64 or raw ?
What can I use to play raw audio ? I suppose FFplay should do the trick ?
What extra <parameters> do I have to use in example below to play raw-audio ?

Here's what I have :

vspipe -i betteraudio_test.vpy -
[mp3float @ 000001B60C074C00] Could not update timestamps for skipped samples.
Samples: 12856320
Sample Rate: 48000
Format Name: Audio32F
Sample Type: Float
Bits: 32
Channels: 2
Layout:

ffplay -f f32le -sample_rate 48000 -channel_layout 3 -i rawaudio.raw
[pcm_f32le @ 000002ce23786700] Channel layout 'stereo' with 2 channels does not match specified number of channels 1: ignoring specified channel layout
[f32le @ 000002ce23779180] Estimating duration from bitrate, this may be inaccurate
Input #0, f32le, from 'rawaudio.raw':
Duration: 00:08:55.68, bitrate: 1536 kb/s
Stream #0:0: Audio: pcm_f32le, 48000 Hz, 1 channels, flt, 1536 kb/s
[pcm_f32le @ 000002ce237d3280] Channel layout 'stereo' with 2 channels does not match specified number of channels 1: ignoring specified channel layout
24.58 M-A: 0.000 fd= 0 aq= 184KB vq= 0KB sq= 0B f=0/0


Vspipe -i says 2 channels, but ffplay can only find one !
One channel produces only noise, while the other plays the music, but much to slow (tempo and pitch are both to low).
The woman in this song sounds like a man with a deep voice.

[Myrsloik]

How to choose to output W64 or raw ?
What can I use to play raw audio ? I suppose FFplay should do the trick ?
What extra <parameters> do I have to use in example below to play raw-audio ?

ffplay -f rawaudio <parameters> -i rawaudio.raw

<parameters> = ??

You use -y or --y4m wo add w64 heqaders. A bit confusing but I've basically changed it to mean "add headers". Not sure how to play the raw audio in ffmpeg.

[Pat357]

Finally I figured it out and give it here in case other users want to use ffmpeg to process the output :

To playback or process the raw audio output from Vspipe, do the following :

vspipe -i betteraudio_test.vpy -
[mp3float @ 000001B60C074C00] Could not update timestamps for skipped samples.
Samples: 12856320
Sample Rate: 48000
Format Name: Audio32F
Sample Type: Float
Bits: 32
Channels: 2
Layout:

Notice the following parameters :
Sample Rate: 48000 = -ar 48000
Format Name: Audio32F
Sample Type: Float
Bits: 32 3 above together = -f f32le
Channels: 2 = -ac 2

To playback :
vspipe betteraudio_test.vpy - | ffplay -f f32le -ac 2 -ar 48000 -i -

To process further using ffmpeg :
vspipe betteraudio_test.vpy - | ffmpeg -f f32le -ac 2 -ar 48000 -i - <filters> -acodec xxx ... outputfile

PS. In vapoursynth only raw audio works ok in this test version, output in w64 always gives a lot of skips with noise.
With Avisynth I've no problems found so far.

[Myrsloik]

...
PS. In vapoursynth only raw audio works ok in this test version, output in w64 always gives a lot of skips with noise.
With Avisynth I've no problems found so far.

I'll look into the bugs, must've gotten some value wrong in the w64 header. Expect a new version sometime next week since I'm quite busy.

Anyway, here's the general plan:

Try to fix all the reported issues.
Add the filters ShuffleChannels (audio equivalent of ShufflePlanes), MatrixMix, AssumeSampleRate and maybe AudioLoop.

[Boulder]

Would it be possible to also include the SoundTouch library functionalities to allow adjusting pitch, tempo etc.?

[Myrsloik]

Would it be possible to also include the SoundTouch library functionalities to allow adjusting pitch, tempo etc.?

Start writing a plugin!

[Boulder]

Start writing a plugin!

I'm just a little bit too dumb for that.. not much but enough to not push me over the edge :)
I was just thinking if it was possible to port the implementation already included in native Avisynth+.

Nevertheless, thanks for the work. I tested the plugin briefly in Avs+ to adjust audio (5.1ch FLAC file) tempo for an ugly 24.975 -> 25 fps conversion. It worked without a hitch :thanks:

[Myrsloik]

I'm just a little bit too dumb for that.. not much but enough to not push me over the edge :)
I was just thinking if it was possible to port the implementation already included in native Avisynth+.

Nevertheless, thanks for the work. I tested the plugin briefly in Avs+ to adjust audio (5.1ch FLAC file) tempo for an ugly 24.975 -> 25 fps conversion. It worked without a hitch :thanks:

I'm sure someone (maybe even me) will start to create/port the useful stuff seen in Avisynth once I get a little bit further with the official audio support.

[Myrsloik]

That problem only occurs with w64 header, raw output is fine.

You found an 11 year old bug from the original AVFS author. Congratulations!

So much for borrowing the w64 code from there. Will be fixed in the next version.

[Myrsloik]

Another audio update:

Progress is in fact being made but a few problems have become apparent in the design so that's why there hasn't been a new build yet. Expect something functional later this month.

[Myrsloik]

I've released test2. It's now more or less feature complete and fixes the issues of test1 (incorrect wave64 headers and so on). It also adds the functions ShuffleChannels, SplitChannels and AudioMix. See the bundled documentation for usage examples.

YOU WILL NEED TO UPDATE BESTAUDIOSOURCE OR IT WILL CRASH.

Comment on things in general. Especially how to make the audio function syntax suck less, they all make shuffleplanes look trivial in comparison.

TEST IT!

[DJATOM]

Now it seems w64 files are valid, but some programs doesn't support w64. It would be nice to have an option to output with Wave header as well.
Also I'd like to know why audio is upsampled to 32 bit (from 24 reported in mediainfo)
C:\Temp\VapourSynth64-Portable-R50-audio>VSPipe.exe -i test_aud.vpy -
Samples: 68260320
Sample Rate: 48000
Format Name: Audio32
Sample Type: Integer
Bits: 32
Channels: 2
Layout: back center, side left, side right, top center, top front left, top front center, top front right, top back left
Layout is also look weird to me, there are only 2 channels.

Mediainfo from the input file:
General
ID : 0 (0x0)
Complete name : 00004.m2ts
Format : BDAV
Format/Info : Blu-ray Video
File size : 6.01 GiB
Duration : 23 min 42 s
Overall bit rate mode : Variable
Overall bit rate : 36.3 Mb/s
Maximum Overall bit rate : 48.0 Mb/s

Video
ID : 4113 (0x1011)
Menu ID : 1 (0x1)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 4 frames
Codec ID : 27
Duration : 23 min 42 s
Bit rate mode : Variable
Bit rate : 30.2 Mb/s
Maximum bit rate : 40.0 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.608
Stream size : 5.01 GiB (83%)

Audio #1
ID : 4352 (0x1100)
Menu ID : 1 (0x1)
Format : PCM
Format settings : Big / Signed
Muxing mode : Blu-ray
Codec ID : 128
Duration : 23 min 42 s
Bit rate mode : Constant
Bit rate : 2 304 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Bit depth : 24 bits
Stream size : 391 MiB (6%)

Audio #2
ID : 4353 (0x1101)
Menu ID : 1 (0x1)
Format : PCM
Format settings : Big / Signed
Muxing mode : Blu-ray
Codec ID : 128
Duration : 23 min 42 s
Bit rate mode : Constant
Bit rate : 2 304 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Bit depth : 24 bits
Stream size : 391 MiB (6%)

[Myrsloik]

The upsampling, if any, happens inside ffmpeg. Maybe send me a short sample file and I'll take a look at it.

And the channel listing. Maybe I forgot to fix it...

[Myrsloik]

A lot of bugs were found and fixed that prevented proper operation. Added wav header support to vspipe. The audio output bitdepth now gets rounded up to the nearest multiple of 8 bits meaning that 24 bit output works as expected.

YOU WILL NEED TO UPDATE BESTAUDIOSOURCE OR IT WON'T BEHAVE PROPERLY.

Test again and see how soon it explodes.

[DJATOM]

No major problems spotted so far (with 24 bit stereo PCM muxed into m2ts container). I tried to trim audio by calculating samples per video frame and it worked.
The only one problem I faced is
>>> aclip.format._as_dict()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "src\cython\vapoursynth.pyx", line 980, in vapoursynth.AudioFormat._as_dict
AttributeError: 'vapoursynth.AudioFormat' object has no attribute 'samplesPerFrame'
while
>>> vclip.format._as_dict()
{'color_family': <ColorFamily.YUV: 3000000>, 'sample_type': <SampleType.INTEGER: 0>, 'bits_per_sample': 8, 'subsampling_w': 1, 'subsampling_h': 1}
works.
Upd.: proposed a PR for it.

[Myrsloik]

I sneakily updated the test3 build with some more fixes and improvements on what happens with "too long" audio in certain cases.

Try again and see what you find.

[tebasuna51]

Now it seems w64 files are valid...

Don't work for me.

Using VapourSynth64-Portable-R50-audio3.7z in W10

from vapoursynth import core
c = core.bas.Source(r"C:\Test\5_1.ac3")
c.set_output(0)

C:\Test\Vapour>VSPipe.exe zzaud.vpy zzaud.raw
[ac3 @ 000001C20EF50100] Estimating duration from bitrate, this may be inaccurate
Output 24 frames in 0.05 seconds (491.17 fps)

Correct raw pcm data 6 chan, 32 float, 48000 samplerate

C:\Test\Vapour>VSPipe.exe -y zzaud.vpy zzaud.wav
[ac3 @ 000002B824F81900] Estimating duration from bitrate, this may be inaccurate
Output 24 frames in 0.05 seconds (491.92 fps)

That write the same raw audio data but with 24 headers: FRAME/n
each 576000 bytes (0,5 sec)

I need use:

VSPipe.exe zzaud.vpy - | wavfix (https://forum.doom9.org/showthread.php?p=1520399#post1520399) - zzaud.w64 -o 3 -m 0 -f 32 -c 6

or, if we expect less than 4 GB:

VSPipe.exe zzaud.vpy - | wavfix (https://forum.doom9.org/showthread.php?p=1520399#post1520399) - zzaud.wav -m 0 -f 32 -c 6

[tebasuna51]

The audio output bitdepth now gets rounded up to the nearest multiple of 8 bits meaning that 24 bit output works as expected.

The expected output of any lossy audio decoder must be 32 (or 64) float, only lossless audio must have output 8,16,24 int.

Avisynth have ConvertAudio (http://avisynth.nl/index.php/ConvertAudio) functions but never must be used by default. Let 32 float please.

[Myrsloik]

Don't work for me.

Using VapourSynth64-Portable-R50-audio3.7z in W10





Correct raw pcm data 6 chan, 32 float, 48000 samplerate



That write the same raw audio data but with 24 headers: FRAME/n
each 576000 bytes (0,5 sec)

I need use:

VSPipe.exe zzaud.vpy - | wavfix (https://forum.doom9.org/showthread.php?p=1520399#post1520399) - zzaud.w64 -o 3 -m 0 -f 32 -c 6

or, if we expect less than 4 GB:

VSPipe.exe zzaud.vpy - | wavfix (https://forum.doom9.org/showthread.php?p=1520399#post1520399) - zzaud.wav -m 0 -f 32 -c 6

Can you provide a short test file? Maybe it's a specific type of ac3 that triggers it since throwing a wav file at it didn't do anything.

I've fixed the header problem so you can't mismatch it now. The test3 build has been sneakily updated so download it again.

[Myrsloik]

The expected output of any lossy audio decoder must be 32 (or 64) float, only lossless audio must have output 8,16,24 int.

Avisynth have ConvertAudio (http://avisynth.nl/index.php/ConvertAudio) functions but never must be used by default. Let 32 float please.

Nope. Complete and utter BS and a waste of bits. For mp3/aac compressed from 16 bit sources you'd just get a nice wasteful format where OVER HALF THE BITS ARE COMPRESSION NOISE. Take your audio placebo and go somewhere else. Moar bits=BETTERAR is the new brain rot of the doom9 forums.

[tebasuna51]

I used the -y parameter because --wav don't work with VapourSynth64-Portable-R50-audio2

Now --wav and --w64 work with VapourSynth64-Portable-R50-audio3

BTW both headers (wav and w64) must be WAVE_FORMAT_EXTENSIBLE for multichannel, simple headers are only allowed for mono or stereo.

[tebasuna51]

Nope. Complete and utter BS and a waste of bits.

But..., do you want make a proper soft or only a joke?

[feisty2]

lossy audio files are assumed to be fp32 when imported in Audition

[tebasuna51]

Here there are 3 valid AC3 files with 3 channel each one, there are also the the info logs of them and of the wav/w64 decoded by ffmpeg.

The relevant info is:

File ........: 3p_LR_C.ac3
Audio coding mode (acmod) ...: 3 (3/0 - L, C, R)
Low frequency effects channel: 0 (Not present)
File ........: 3p_LR_C.ac3_.wav or .w64
MaskChannels : 7 (FL FR FC)

File ........: 3p_LR_LFE.ac3
Audio coding mode (acmod) ...: 2 (2/0 - L, R)
Low frequency effects channel: 1 (Present)
File ........: 3p_LR_LFE.ac3_.w64 or .wav
MaskChannels : 11 (FL FR LF)

File ........: 3p_LR_S.ac3
Audio coding mode (acmod) ...: 4 (2/1 - L, R, S)
Low frequency effects channel: 0 (Not present)
File ........: 3p_LR_S.ac3_.wav or w64
MaskChannels : 259 (FL FR BC)

The decoder know the Audio coding mode of AC3 and the output must write the correct MaskChannels.
AviSynth can't obtain from the decoder the MaskChannels, maybe VapourSynth can.

[Myrsloik]

It's already in there if you print(clip.format). I just didn't add WAVE_FORMAT_EXTENSIBLE to the output headers yet.

Audio Format Descriptor
Id: 11001000
Name: Audio32F
Sample Type: Float
Bits Per Sample: 32
Bytes Per Sample: 4
Samples Per Frame: 24000
Channels: FRONT_LEFT, FRONT_RIGHT, BACK_CENTER

[tebasuna51]

It's already in there if you print(clip.format). I just didn't add WAVE_FORMAT_EXTENSIBLE to the output headers yet.

Good to know, thanks.

[Myrsloik]

I've sneakily updated the test3 build again with waveformatextensible support. Note that it's always on (unlike avisynth).

[tebasuna51]

Now work fine, thanks.

By the momet I read there are:
Audio filters

BestAudioSource – a new sample accurate but somewhat slow FFmpeg based source filter (usage: core.bas.Source(“rule6.mp4”))
BlankAudio – a classic
AudioSplice and AudioTrim – with the expected Python overloads of course

I check clip.std.AudioTrim(0, 100000)
and I obtained 100001 audio samples.

There are some docs to know how use BlankAudio, AudioSplice and others future functions?

About BestAudioSource seems decode ac3, eac3, aac, flac, mp3 but still not:
opus (vapoursynth.Error: Couldn't open 'C:\Test\MultiCan\Test_.opus')
dts (Filter Source has no audio samples in the output.)
thd (Filter Source has no audio samples in the output.)

[Myrsloik]

Now work fine, thanks.

By the momet I read there are:


I check clip.std.AudioTrim(0, 100000)
and I obtained 100001 audio samples.

There are some docs to know how use BlankAudio, AudioSplice and others future functions?

About BestAudioSource seems decode ac3, eac3, aac, flac, mp3 but still not:
opus (vapoursynth.Error: Couldn't open 'C:\Test\MultiCan\Test_.opus')
dts (Filter Source has no audio samples in the output.)
thd (Filter Source has no audio samples in the output.)

The range is inclusive just like trim in avisynth. Note that python array slicing syntax (clip[0:100000]) does not include the last value and will return 100000 samples.

Check the included local documentation. It's mostly complete.

BestAudioSource is compiled without additional decoding libraries since it's a huge pain in the ass to do. Especially when developing. More compatible builds will probably be created by someone once I declare development mostly finished.

[Lypheo]

Do you plan on adding documentation for the audio filter api?

[Myrsloik]

Do you plan on adding documentation for the audio filter api?

Yes, at some later point. It works 99% like video filters and you can look at audiofilters.cpp to see how they're implemented.

The only quirk you really need to know is that the audio frame size is currently always the same for all formats. And that I'll probably use this branch to clean up the API in general so at some point trivial changes and a quick recompile may be necessary.

[l33tmeatwad]

I see this is on GitHub, is there currently an easy way to compile this for testing on unix based systems?

[Myrsloik]

I see this is on GitHub, is there currently an easy way to compile this for testing on unix based systems?

The build system should work. Or at most only miss audiofilters.cpp in the list of files to compile.

[Myrsloik]

The build system is now updated so all you not-windows-people can test things out.

[l33tmeatwad]

In case anyone was curious, if you have a VapourSynth enabled copy of FFMPEG and want to encode video and audio at the same time it works rather nicely.

from vapoursynth import core
video = core.ffms2.Source('8bit Sample.mp4')
audio = core.bas.Source('8bit Sample.mp4')

video.set_output(0)
audio.set_output(1)

vspipe --wav -o 1 Sample.vpy - | ./ffmpeg -f vapoursynth -i Sample.vpy -i pipe: -map 0:0 -map 1:0 -f AVI -c:v utvideo -pix_fmt yuv420p -colorspace bt709 -c:a pcm_s16le -y sample.avi

[poisondeathray]

Something is buggy with the video for R50 vapoursynth audio3 version (even without audio) when using LWLibavSource (non indexed) for MP4. It previews ok; but it crashes when using vspipe to ffmpeg, or even the vapoursynth editor internal benchmark F7 , when loading a simple youtube video, or other random MP4 video


clip = core.lsmas.LWLibavSource(r'video.mp4')
clip.set_output()



But it works ok when indexed with LibavSMASHSource


clip = core.lsmas.LibavSMASHSource(r'video.mp4')
clip.set_output()



Switching back to R50 "no audio" vapoursynth version fixes LWLibavSource

EDIT: actually I'm not sure that's it. It seems to crash randomly, even with LibavSMASHSource +/- filters. Adding filters causes it to crash faster. But the no audio R50 version works ok

[Myrsloik]

Define crashes. Which os? Does it happen with other source filters?

[poisondeathray]

Define crashes. Which os? Does it happen with other source filters?

In vsedit benchmark (F7), "script editor has stopped working" . "A problem caused the program to stop working correctly. Please close the program"

Win8.1

Yes, ffms2 script crashes too when some filter is added e.g. clip = haf.SmoothLevels(clip, 18,1,235,0,255) , but not with source filter ffms2 alone

Or internal levels filter, in case there was an issue with haf.SmoothLevels

clip = core.ffms2.Source(r'video.mp4')
clip = core.std.Levels(clip, min_in=0, max_in=0, gamma=1, min_out=255, max_out=235, planes=[0,1,2])
clip.set_output()



But blankclip + filter works ok

[Myrsloik]

In vsedit benchmark (F7), "script editor has stopped working" . "A problem caused the program to stop working correctly. Please close the program"

Win8.1

Yes, ffms2 script crashes too when some filter is added e.g. clip = haf.SmoothLevels(clip, 18,1,235,0,255) , but not with source filter ffms2 alone

Or internal levels filter, in case there was an issue with haf.SmoothLevels

clip = core.ffms2.Source(r'video.mp4')
clip = core.std.Levels(clip, min_in=0, max_in=0, gamma=1, min_out=255, max_out=235, planes=[0,1,2])
clip.set_output()



But blankclip + filter works ok

Reproduced but will definitely take a while to figure out why it happens.

[Myrsloik]

Test 4 posted. Adds the functions AudioReverse and AudioLoop. Improves AudioSplice to be faster with many clips and fixes AudioMix and ShuffleChannels so they have sane argument lists. Plus all the things that changed during test3 that's not very well documented.

Note that the crash bug still remains but doesn't happen ever happen when using vspipe.

The todo list:

Fix crash bug
Add a function for simple volume adjustment
Audio format conversion functions (format and/or samplerate)
Do a huge cleanup of the API and internal code before declaring audio support stable.

[tebasuna51]

Thanks for the new version.

I try to make some test, to emulate a simple volume adjust (Amplify in Avs), but seems I have some syntax error here:
from vapoursynth import core
c = core.bas.Source(r"C:\tmp\tri16 - f32.wav")
# a simple mono wav
c = core.std.AudioMix(clips=c, matrix=[0.5], channels_out=[vs.FRONT_CENTER])
c.set_output()
and obtain:
Script evaluation failed:
Python exception: name 'vs' is not defined

Other tips:

std.SplitChannels(anode clip)
(In docs like Trim/AudioTrim function)
How obtain the separate clips? Maybe:
fl,fr,fc,lf,sl,sr = std.SplitChannels(source)

AudioTrimS(anode clip[, float first=0, float last, float length])
Values in seconds with int(value * samplerate)
Accept negative values for first, to replace DelayAudio(value), insert silence at the begining.

Do you want make internal fuctions for change samplerate to replace Avs ResampleAudio, SSRC, TimeStretch?
Or maybe support external plugins like TimeStretch in Avs+ or other plugins like Sox to audio management?.

Thanks for your job.

[DJATOM]

Add 'import vapoursynth as vs' above core import

[tebasuna51]

Add 'import vapoursynth as vs' above core import

Perfect, thank you.

Now, how can know the max float value in order to do a Normalize() after a
AudioMix(... matrix=[1.5]...) for instance?

Loading the float output in Audacity I have values > 1 than can produce clip when convert to int.

[Myrsloik]

Thanks for the new version.

I try to make some test, to emulate a simple volume adjust (Amplify in Avs), but seems I have some syntax error here:
from vapoursynth import core
c = core.bas.Source(r"C:\tmp\tri16 - f32.wav")
# a simple mono wav
c = core.std.AudioMix(clips=c, matrix=[0.5], channels_out=[vs.FRONT_CENTER])
c.set_output()
and obtain:


Other tips:

std.SplitChannels(anode clip)
(In docs like Trim/AudioTrim function)
How obtain the separate clips? Maybe:
fl,fr,fc,lf,sl,sr = std.SplitChannels(source)

AudioTrimS(anode clip[, float first=0, float last, float length])
Values in seconds with int(value * samplerate)
Accept negative values for first, to replace DelayAudio(value), insert silence at the begining.

Do you want make internal fuctions for change samplerate to replace Avs ResampleAudio, SSRC, TimeStretch?
Or maybe support external plugins like TimeStretch in Avs+ or other plugins like Sox to audio management?.

Thanks for your job.

I'm going to add a function for simple volume adjustment in the next version. You can emulate it by passing a scaled identity matrix to AudioMix until that happens.

If you check the documentation you see that AudioMix always needs input channels*output channels arguments as the matrix. So for stereo it'd be [0.8, 0, 0, 0.8] as an example.

If you want to determine the max value you have to scan the whole clip yourself. You should be able to implement it as a pure python script or simply pass the vspipe output to something that can figure out the max values.

[tebasuna51]

...If you want to determine the max value you have to scan the whole clip yourself. You should be able to implement it as a pure python script...

Yes, but how can obtain the sample values in the script?

For a clip, for instance c = core.bas.Source(r"C:\some.wav")
I can know:

NumSamples = len(c)
SampleRate = c.sample_rate
MaskChannels = c.format.channel_layout
NumChannels = c.format.num_channels
BitDepth = c.format.bits_per_sample
SampleFormat = c.format.sample_type
TimeLength = NumSamples/SampleRate

And can do a for loop by samples and channels if I know the sintax to obtain c.VALUE(samples,channels):

max = 0.0
for samples in range (0,NumSamples-1):
for channels in range (0,NumChannels-1):
if c.VALUE(samples,channels) > max:
max = c.VALUE(samples,channels)

[Myrsloik]

Yes, but how can obtain the sample values in the script?

For a clip, for instance c = core.bas.Source(r"C:\some.wav")
I can know:

NumSamples = len(c)
SampleRate = 2 * c.format.samples_per_frame (for what a frame 0.5 sec?)
MaskChannels = c.format.channel_layout
NumChannels = c.format.num_channels
BitDepth = c.format.bits_per_sample
SampleFormat = c.format.sample_type
TimeLength = NumSamples/SampleRate

And can do a for loop by samples and channels if I know the sintax to obtain c.VALUE(samples,channels):

max = 0.0
for samples in range (0,NumSamples-1):
for channels in range (0,NumChannels-1):
if c.VALUE(samples,channels) > max:
max = c.VALUE(samples,channels)

Short answer: Not implemented yet, raw data access only works for video frames at the moment. Will probably be implemented for test5. That's how you know you're in a development thread...

[Myrsloik]

Test4 build sneakily updated. Changes:

Added AudioGain filter
Added the possibility to directly access audio data in the same way as video
Fixed crashes
Fixed broken wave64 headers

[Myrsloik]

How to get the max value of audio as a simple script:

maxval = 0
for frame in c.frames():
for channel in range(c.format.num_channels):
data = frame.get_read_array(channel)
for val in data:
if abs(val) > maxval:
maxval = abs(val)

print(maxval)

[tebasuna51]

Thanks.

[tebasuna51]

The todo list:
...
Audio format conversion functions (format and/or samplerate)
...

- The format conversions in Avisynth are:

ConvertAudioTo8/16/24/32bit/Float

For me we only need ConvertAudioTo16/24/Float

The 8 bit unsigned int, still not supported like input:
Error: Unsupported audio format from decoder (probably 8-bit)
is not necesary at all.
The 32 bit int signed, accepted like input, can be managed (Trim, Splice, Loop, Reverse, Shuffle) but don't have sense upconvert other format to it.

The conversion between 24 int signed and 32 float can be considered lossless (precission preserved with the same bits used for mantissa).

Any arithmetic operation (Gain, Mix, Resample, Filter, ...) are a lossy operation and is recommended make it in float format to avoid clip (overflow if use int format). Maybe each operation must store the max value reached in order to gain or attenuate before reconvert the float to int (Normalize).
Arithmetic operations aren't allowed in 32 bits int format in AviSynth.

Many encoders support float input but the user must selecct the final output format.

- I check some wav input formats and work fine with 16, 24, 32 int formats, 32 float, u-law and a-law.
With 8 bit int and 64 float show the same error:
Error: Unsupported audio format from decoder (probably 8-bit)
64 float is not supported in AviSynth also.
The adpcm_ms and adpcm_ima read a 44100 samplerate 20 seconds like 18.375 seconds samplerate 48000.

I try correct the error with:
c.format.samples_per_frame = 22050
and
AttributeError: attribute 'samples_per_frame' of 'vapoursynth.AudioFormat' objects is not writable

Avisynth have the AssumeSampleRate function and is a method to make some duration changes, for instance:
"SpeedUp 23.976 -> 25">
AssumeSampleRate(last, (AudioRate()*1001+480)/960).SSRC(AudioRate(last))

[tebasuna51]

I was wrong, in this audio:
Audio Format Descriptor
Id: 11001000
Name: Audio16
Sample Type: Integer
Bits Per Sample: 16
Bytes Per Sample: 2
Samples Per Frame: 24000
Channels: FRONT_LEFT, FRONT_RIGHT
the Samplerate is not 2*SamplesPerFrame = 48000

How we can know the Samplerate of that audio?

The same Audio Format for samplerate 8000 to 192000, the Samples Per Frame data is useless.

The concept of Frame in audio is not the same than video.
The equivalent to video frame is the audio Sample

Also the output message:
"Output 160 frames in 0.07 seconds (2240.11 fps)"
don't have many sense in audio, maybe:
"Output 3 840 000 samples in 0.07 seconds (54 857 142 sps)"
or better, in time format like some encoders:
"Output 20 seconds in 0.07 seconds (285x)

[Myrsloik]

I was wrong, in this audio:

the Samplerate is not 2*SamplesPerFrame = 48000

How we can know the Samplerate of that audio?

The same Audio Format for samplerate 8000 to 192000, the Samples Per Frame data is useless.

The concept of Frame in audio is not the same than video.
The equivalent to video frame is the audio Sample

Also the output message:
"Output 160 frames in 0.07 seconds (2240.11 fps)"
don't have many sense in audio, maybe:
"Output 3 840 000 samples in 0.07 seconds (54 857 142 sps)"
or better, in time format like some encoders:
"Output 20 seconds in 0.07 seconds (285x)

Hint:
clip.sample_rate

[tebasuna51]

Thanks. The audio support is now:
Avisynth Audio functions and their VapourSynth equivalents
==========================================================
Lossless functions (must work over ALL format samples, 8 bits int not necesary)
------------------
AssumeSampleRate Pending, make attribute 'sample_rate' writable
AudioTrim std.AudioTrim()
DelayAudio Pending, maybe std.AudioTrim(-delay) insert silence at first
GetChannel std.ShuffleChannels(), std.SplitChannels()
MergeChannels std.ShuffleChannels()
+ + or std.AudioSplice(), std.AudioLoop()
AudioDub/AudioDubEx Pending, maybe never necesary
EnsureVBRMP3Sync Not needed
KillAudio/KillVideo Without sense now

Lossy functions (must work only in FLOAT format, in Avisynth the marked [16] work also with 16 int)
---------------
ConvertAudioTo8bit Not needed, maybe the source plugin can supply 16int (also 32f for 64f)
ConvertAudioTo16bit Pending
ConvertAudioTo24bit Pending
ConvertAudioTo32bit Not needed, it doesn't make sense upconvert other format to 32 int
ConvertAudioToFloat Pending
Amplify/AmplifydB [16] std.AudioGain()
ConvertToMono [16] std.AudioMix()
MixAudio [16] std.AudioMix()
MonoToStereo [16] std.AudioMix()
Normalize [16] Search MaxValue and std.AudioGain()

ResampleAudio [16] Pending, maybe with external plugins
SSRC Pending, maybe with external plugins
TimeStretch Pending, maybe with external plugins
SuperEQ Pending, maybe with external plugins

TimeStrech work like a plugin in Avs+ and can replace to ResampleAudio and SSRC
Sox can work like a plugin and replace SSRC, SuperEQ and add many other audio functions

Can we use avs.LoadPlugin() with Avs+ plugins?
I tried with
core.avs.LoadPlugin(r"C:\plugins\TimeStretch.dll")
c=c.avs.TimeStretch(tempo=100.0*25.0/(24000.0/1001.0))
and:
vapoursynth.Error: TimeStretch: argument c1 was passed an unsupported type (expected clip compatible type but got AudioNode)

[MeteorRain]

My personal opinion regarding to ConvertAudio* family:

I would rather getting the support of 32 bit instead of 24 bit. 24 bit (3 bytes) is not aligned to SIMD boundary and processing of it is difficult. 32 bit processing offers a huge improvement on performance because operations can be done using SSE or AVX.

[DJATOM]

And my personal opinion is that 24 bit sometimes the maximum supported bitness for certain audio codecs (for example, flac). Leaving it behind a boat is bad.
Why not support all possible cases, if possible?

[tebasuna51]

I hope are you talking about 32 bits float, because 32 int is a format than I never see in audio tracks of movies.

All decoders of lossy formats output 32 (even 64) bits float, and many lossy encoders accept 32 float like input then most the time we don't need any conversion.

Lossless formats (TrueHD, DTS-MA) use 16/24 int sample and the decoder must supply that, if we use only lossless functions (Trim, Add, Shuffle) we dont need convert the audio and output the same.
If we need use a lossy function we need convert to float (16/24 bits int -> 32 float) after that recover the previous format is not recommended.

I don't know if a final conversion 32 float -> 24 int can save time, of course save space if we need a phisical file output.

[Myrsloik]

Does anyone actually use double for audio processing? If so, where can I find it. Which applications support it?

[feisty2]

Adobe Audition has support for fp64 audio

[MeteorRain]

@DJATOM @tebasuna51 No I'm talking about 32 bit integer. If I mean float I'd say float.

24 bit audio is stored in 3 bytes. For each m128 register you can store 5.33 samples, and a m256 register holds 10.67 samples.
If 16 bit is insufficient, I'd prefer using 32 bit as internal depth.
Of course, float or double may be a better format. But in any case, no 24 bit for internal processing unless no arithmetic involved.

If 24 bit output is needed, process in float or 32 bit and then convert back to 24 bit.

[tebasuna51]

Does anyone actually use double for audio processing? If so, where can I find it. Which applications support it?

eac3to with the parameter -full and using libav/ffmpeg can obtain 64 float:

eac3to v3.34
command line: eac3to 6a321.ac3 6a321.wav" -full
------------------------------------------------------------------------------
AC3, 5.1 channels, 0:00:20, 448kbps, 48kHz
Decoding with libav/ffmpeg...
Writing WAV...
Creating file "D:\Test\AudioD\Samples\ac3\6a321.ac3_.wav"...
The original audio track has a constant bit depth of 64 bits.
The processed audio track has a constant bit depth of 64 bits.

By default:
Decoding with libav/ffmpeg...
Reducing depth from 64 to 24 bits...

[feisty2]

you guys confused the data structures of intermediate representations (data accessed via vsapi->getReadPtr) and the final output (uncompressed raw produced by vspipe), the two are not necessarily the same. it is definitely possible to pad uint24 to uint32 for vsapi->getReadPtr and retain the uint24 structure when the audio script is materialized thru vspipe.

this is already true for videos since each row of a frame returned by vsapi->getReadPtr is guaranteed to be 32-byte aligned, meaning two consecutive rows might not be consecutive in memory. however when vspipe materializes a frame, consecutive rows are definitely consecutive in the uncompressed raw, the data structures of two cases are already different as you can see.

[feisty2]

simple rule: a 24-bit sample should always be padded to uint32 until it is materialized. all your codecs and stuff only deal with materialized output so that won't be a problem.

[Myrsloik]

You know you could look at the actual API or ask BEFORE committing virtual hate crimes against 24 bit integers.

Storage is obviously always a power of two number of bytes just as for images. Packing only happens for output purposes. Just like video. Was that so hard to guess?

Those of you who know how lazy programmers can be would've also assumed I'm greatly inspired by the FFmpeg API and its solutions.

Btw, the next build will probably take a while longer due to the other code cleanups and improvements I want to do as well.

[richardpl]

FFmpeg solutions is better that anyone could code in several decades.

[tebasuna51]

@feisty2, @Myrsloik, OK

I only want say than lossless operations (Trim, Shuffle, Splice, Split, Loop) are only rearrange data, and can't be a problem for any kind of data.

But lossy operations (Gain, Mix, Resample, Filter,...) implies aritmethic operations and must be done always in float format, 32 float is enough for audio with a precission equivalent to 24 bits int (the human ear can difference only to a precission of 20 bits)

The final output must be a choice of user, if only do lossless operations the output can be the same than input, and be recoded with lossless formats (FLAC, DTS-MA,...)
When use lossy operations doesn't make sense recode to lossless formats, we don't need recover int format except for some cases: we want burn a CD audio, the recoder don't support 32 float (strange), ...

[MeteorRain]

24 bit audio was stored as 3 bytes stream in AviSynth+.

If you would store it as 32 bit, then how's it any better than actually using a full 32 bit sample?
You get less accuracy for the ... exact same computation cost. And you would think that's the "benefit" of using 24 bit instead of 32 bit data?
And you would pack the output by truncating 1 byte from it, which is, well, exactly the same as packing a full 32 bit audio to 24 bit output.

Sorry, maybe I'm too dumb to get the point.

[_Al_]

This is amazing, thanks Myrsloik and all developing, testing.

last VapourSynth64-Portable-R51-audio4.7 and last BestAudioSource.dll,
this works:
VSPipe --wav audio_tests.py - | neroAacEnc -ignorelength -lc -cbr 96000 -if - -of audio.m4a
but, this does not:
VSPipe --wav audio_tests.py - | ffplay.exe -i -
G:\VapourSynth64-Portable-R51-audio4.7>VSPipe --wav __audio_tests.py - | neroAacEnc -ignorelength -lc -cbr 96000 -if - -of audio.m4a
*************************************************************
* *
* Nero AAC Encoder *
* Copyright 2009 Nero AG *
* All Rights Reserved Worldwide *
* *
* Package build date: Feb 18 2010 *
* Package version: 1.5.4.0 *
* *
* See -help for a complete list of available parameters. *
* *
*************************************************************

[mov,mp4,m4a,3gp,3g2,mj2 @ 00000000028F7F40] stream 0, timescale not set
Output 89 frames in 1.15 seconds (77.17 fps)


G:\VapourSynth64-Portable-R51-audio4.7>VSPipe --wav __audio_tests.py - | ffplay.exe -i -
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000000002917F40] stream 0, timescale not set
ffplay version N-95015-gba24b24aab Copyright (c) 2003-2019 the FFmpeg developers
built with gcc 9.2.1 (GCC) 20190918
configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --e
nable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus -
-enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --ena
ble-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enabl
e-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-ffnvcodec --enable-cuvid --enable-d3d1
1va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
libavutil 56. 35.100 / 56. 35.100
libavcodec 58. 58.101 / 58. 58.101
libavformat 58. 33.100 / 58. 33.100
libavdevice 58. 9.100 / 58. 9.100
libavfilter 7. 58.102 / 7. 58.102
libswscale 5. 6.100 / 5. 6.100
libswresample 3. 6.100 / 3. 6.100
libpostproc 55. 6.100 / 55. 6.100
Input #0, wav, from 'pipe:':aq= 0KB vq= 0KB sq= 0B f=0/0
Duration: N/A, bitrate: 3072 kb/s
Stream #0:0: Audio: pcm_f32le ([3][0][0][0] / 0x0003), 48000 Hz, stereo, flt, 3072 kb/s
SDL_OpenAudio (2 channels, 48000 Hz): WASAPI can't initialize audio client: CoInitialize has not been called.

SDL_OpenAudio (1 channels, 48000 Hz): WASAPI can't initialize audio client: CoInitialize has not been called.

SDL_OpenAudio (2 channels, 44100 Hz): WASAPI can't initialize audio client: CoInitialize has not been called.

SDL_OpenAudio (1 channels, 44100 Hz): WASAPI can't initialize audio client: CoInitialize has not been called.

No more combinations to try, audio open failed
Failed to open file 'pipe:' or configure filtergraph
nan : 0.000 fd= 0 aq= 0KB vq= 0KB sq= 0B f=0/0
Error: fwrite() call failed when writing frame: 1, errno: 32
Output 6 frames in 20.29 seconds (0.30 fps)
script is simple:
import vapoursynth as vs
from vapoursynth import core
a = core.bas.Source('video.mp4')
a.set_output(0)

[feisty2]

the same applies to videos you know, apparently the computation cost on 10-bit videos is the same as 16-bit videos, so why hasn't 16-bit become the mainstream video format?
you're still mixing up the intermediate representation and the materialized output.

[MeteorRain]

ConvertAudioTo24bit Pending
ConvertAudioTo32bit Not needed, it doesn't make sense upconvert other format to 32 int

ConvertAudio* family is not only used in materialized output, but is frequently used in intermediate representation.

Assuming you are objecting my reply, you'd be thinking that we don't need 32bit intermediate format, and filters should work with 24bit if required to be using intergers?

[tebasuna51]

You get less accuracy for the ... exact same computation cost. And you would think that's the "benefit" of using 24 bit instead of 32 bit data?
And you would pack the output by truncating 1 byte from it, which is, well, exactly the same as packing a full 32 bit audio to 24 bit output.

Sorry, maybe I'm too dumb to get the point.

Sorry, maybe I don't know how explain you the point.

I will try:

- Don't exist any standard input with better precission than 24 bit int.

- We can't output any standard format with 32 bits of precission.

- The human ear can difference until a precision of 20 bits, then 24 int or 32 float is more than enough

- You can use 32 int to do internal operations (https://forum.doom9.org/showthread.php?p=1916468#post1916468) if you think is better, but like audio format is useless.

- But I recommend use 32 float to do any aritmetic operations if posible, to recover overflows.

[MeteorRain]

My point was 32 bit audio is no where close to Not needed, it doesn't make sense upconvert other format to 32 int.

Now, because I was redirected to here from AVS threads, and AVS never had padding on 24 bit audio, we always get 3 bytes unaligned samples, hence my point to not use 24 bit at all.

The idea of padding a 24 bit audio effectively make it a 32 bit sample (except less accuracy than true 32 bit but the rest are exactly the same).
Because If you pad and strip on LSB instead of MSB you automatically get a 32 bit.

[feisty2]

AVS never had padding on 24 bit audio, we always get 3 bytes unaligned samples, hence my point to not use 24 bit at all.


now I'm actually curious how 24-bit audio is handled in avs, since int24 is not a valid integer type


using uint24_t = std::uint8_t[3];

maybe?

[MeteorRain]

now I'm actually curious how 24-bit audio is handled in avs, since int24 is not a valid integer type


using uint24_t = std::uint8_t[3];

maybe?

No "using".

Just (uint8_t *).

[tebasuna51]

Like you can see there aren't functions lossy (with aritmetic operations) than support 24 bits, the lossless functions can be managed like some array of bytes without problems.

For lossy functions can't use 24 bits, use 32 int if you want, but I thing 32 float is easy to control overflows and preserve the required precission.
Also all decoders of lossy formats supply 32 float by default.
At end we need the option to convert to 24 bits int because there are formats/encoders than need that precission.

[tebasuna51]

last VapourSynth64-Portable-R51-audio4.7 and last BestAudioSource.dll,
this works:
...
but, this does not:
...
script is simple:
import vapoursynth as vs
from vapoursynth import core
a = core.bas.Source('video.mp4')
a.set_output(0)

The input is a mp4 with video and audio, by the moment VapourSynth-Portable-R51-audio4.7 only work with audio, maybe there are some data related to video sended to output.

Try extract the audio data only and repeat with
a = core.bas.Source('audio.m4a')

NeroAacDec/Enc admit like input mp4's with video/audio (ignoring the video) maybe for that work fine.

EDIT: I can't reproduce the problem, work for me:

C:\Test\Vapour>VSPipe.exe --wav zzaud.vpy - | C:\Portable\0\ffplay.exe -i -
ffplay version git-2020-04-03-52523b6 Copyright (c) 2003-2020 the FFmpeg developers
...
Input #0, wav, from 'pipe:':aq= 0KB vq= 0KB sq= 0B f=0/0
Duration: N/A, bitrate: 3072 kb/s
Stream #0:0: Audio: pcm_f32le ([3][0][0][0] / 0x0003), 48000 Hz, stereo, flt, 3072 kb/s
Output 267 frames in 132.73 seconds (2.01 fps)0KB sq= 0B f=0/0
133.39... M-A: 0.000 fd= 0 aq= 0KB vq= 0KB sq= 0B f=0/0

I neeed stop the process with Ctrl/C
The output wav is correct.

[_Al_]

Try extract the audio data only and repeat with
a = core.bas.Source('audio.m4a')

That produces one more line (yellow warning): [aac @ 0000000002A11000] Could not update timestamps for skipped samples.
But again Nero encoder works and ffplay does not. I downloaded and updated to latest ffplay and still it does not work.
But ffmpeg build with vapoursynth enabled works. Two outputs v.set_output(0) and a.set_output(1) in script. I used Pat357 example:
VSPipe --wav -o 1 audio_tests.py - | ffmpeg -f vapoursynth -i audio_tests.py -i pipe: -map 0:0 -map 1:0 -f AVI -c:v utvideo -pix_fmt yuv420p -colorspace bt709 -c:a pcm_s16le -y ffmpeg.avi
So something about that ffplay.

[qyot27]

That produces one more line (yellow warning): [aac @ 0000000002A11000] Could not update timestamps for skipped samples.
But again Nero encoder works and ffplay does not. I downloaded and updated to latest ffplay and still it does not work.
But ffmpeg build with vapoursynth enabled works. Two outputs v.set_output(0) and a.set_output(1) in script. I used Pat357 example:
VSPipe --wav -o 1 audio_tests.py - | ffmpeg -f vapoursynth -i audio_tests.py -i pipe: -map 0:0 -map 1:0 -f AVI -c:v utvideo -pix_fmt yuv420p -colorspace bt709 -c:a pcm_s16le -y ffmpeg.avi
So something about that ffplay.
Judging by the SDL_OpenAudio/WASAPI errors, I'd say it's definitely a problem with the SDL2 lib in the build environment of whoever built ffmpeg/ffplay. Maybe it's actually some sort of problem in Windows itself (if it truly is coming out of WASAPI and not SDL2), but I'd see if it's fixed with an ffplay linked against a different build of SDL2.

It works here (build used (http://www.mediafire.com/file/f5xe8bhex020x2z/ffmpeg_r98029%252B8.7z/file)):
E:\Documents>vspipe --wav test.vpy - | ffplay -i -
ffplay version r98029+8 master-fb17ba86a8 HEAD-e0cb5ba49a
contains: avsrgb datetime new_pkgconfig silent_invoke vapoursynth_alt versioninfo
Copyright (c) 2003-2020 the FFmpeg developers
built on Jun 2 2020 01:27:59 with gcc 10.1.0 (GCC)
libavutil 56. 49.100 / 56. 49.100
libavcodec 58. 90.100 / 58. 90.100
libavformat 58. 44.100 / 58. 44.100
libavdevice 58. 9.103 / 58. 9.103
libavfilter 7. 84.100 / 7. 84.100
libswscale 5. 6.101 / 5. 6.101
libswresample 3. 6.100 / 3. 6.100
libpostproc 55. 6.100 / 55. 6.100
[opus @ 00000239FA49FBC0] Could not update timestamps for skipped samples.
Input #0, wav, from 'pipe:':aq= 0KB vq= 0KB sq= 0B f=0/0
Duration: N/A, bitrate: 3072 kb/s
Stream #0:0: Audio: pcm_f32le ([3][0][0][0] / 0x0003), 48000 Hz, stereo, flt, 3072 kb/s
1.90 M-A: 0.000 fd= 0 aq= 385KB vq= 0KB sq= 0B f=0/0
Error: fwrite() call failed when writing frame: 6, errno: 32
Output 11 frames in 2.39 seconds (4.61 fps)
As does setting -o 1 when I have both video and audio and a.set_output is directed to 1.

[_Al_]

ok, thanks, so focusing on that error, I have Windows 7,

and googling that error (which I should do before) and this (https://stackoverflow.com/questions/46835811/ffplay-wasapi-cant-initialize-audio-client-ffmpeg-3-4-binaries) helped, setting SDL_AUDIODRIVER variable to directsound.

All ffplay builds now work.

[Myrsloik]

A bit of an update in case you're not paying attention to the commits:

Development has continued but mostly on creating the next big API revision with several improvements and cleanups. The current state is a bit too unstable and so far adds nothing in terms of functionality to audio and that's why there have been no new builds.

[DJATOM]

In fact It's even impossible to compile python module at the time: https://pastebin.com/HNjkQYh5

[Myrsloik]

The test builds have now resumed after the big api cleanup. Test5 has fixed all known bugs and should be fairly stable. Adding headers in vspipe is now done with -c wav/w64/y4m. The only known bugs are:

vspipe's "--graph simple" option isn't implemented and the logging management functions in python are not working well. Apart from that report anything weird you find.

BestAudioSource needs to be updated again as usual.

[DJATOM]

I've got some log in vsedit

2020-08-02 16:30:18.013
API MISUSE! Function 'eedi3' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'eedi3' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'Dilate' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'Dilate' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'Erode' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'Erode' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'Open' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'Open' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'Close' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'Close' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'TopHat' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'TopHat' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'BottomHat' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'BottomHat' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'RemoveGrain' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'RemoveGrain' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'Repair' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'Repair' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'Clense' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'Clense' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'ForwardClense' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'ForwardClense' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'BackwardClense' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'BackwardClense' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'VerticalCleaner' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'VerticalCleaner' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'Vinverse' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'Vinverse' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'VFM' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'VFM' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'VDecimate' failed to register with error: Argument 'clip' has invalid type 'clip'.
API MISUSE! Function 'VDecimate' failed to register with error: Argument 'clip' has invalid type 'clip'.
2020-08-02 16:30:18.446
Script was successfully evaluated. Output video info:
Frames: 555 | Time: 0:00:23.148 | Size: 1920x1080 | FPS: 24000/1001 = 23.976 | Format: YUV420P8
Probably due to https://github.com/vapoursynth/vapoursynth/blob/doodle1/src/filters/vivtc/vivtc.c#L1623 must be a vnode type.

[Myrsloik]

Doh, test5 now sneakily updated AGAIN!

[Pat357]

How can I create a list of all functions for the VS_audio_test5 ?

Here is what I get :
d:\programs\VapourSynth64-Portable-R51-audio5\VapourSynth64Portable\VapourSynth64>python
Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 bit (AMD64)] on win32
>>> import vapoursynth as vs
>>> core = vs.core
>>> print(core.list_functions())
<stdin>:1: DeprecationWarning: list_functions() is deprecated. Use "plugins()" instead.
<stdin>:1: DeprecationWarning: get_plugins() is deprecated. Use "plugins()" instead.

After this, I never get the >>> prompt back and it seems like python is hanging. Also no ctrl+Z to exit.
Only way was killing python.exe in taskman.

[Myrsloik]

How can I create a list of all functions for the VS_audio_test5 ?

Here is what I get :
d:\programs\VapourSynth64-Portable-R51-audio5\VapourSynth64Portable\VapourSynth64>python
Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 bit (AMD64)] on win32
>>> import vapoursynth as vs
>>> core = vs.core
>>> print(core.list_functions())
<stdin>:1: DeprecationWarning: list_functions() is deprecated. Use "plugins()" instead.
<stdin>:1: DeprecationWarning: get_plugins() is deprecated. Use "plugins()" instead.

After this, I never get the >>> prompt back and it seems like python is hanging. Also no ctrl+Z to exit.
Only way was killing python.exe in taskman.

Fixed now. Redownload test5. I've also fixed some other bugs so everyone should redownload it.

You can also use the included documentation which should have all new audio functions.

[lansing]

Test 5 is not releasing memory. I ran a script in vseditor, then closed the tab, but the memory didn't get release.

[Myrsloik]

Test5 is updated again. This time it fixes the missing mimalloc dlls which prevented it from working outside vspipe most of the time.

[tebasuna51]

Thanks for your updates.

I have time to test now and I want explain the changes I see.

1) The command line I use now is:

VSPipe.exe -c wav zzaud.vpy zzD1sec.wav

Before vspipe need --wav

The zzaud.vpy I use is:
import vapoursynth as vs
from vapoursynth import core
c = core.bas.Source(r"C:\Test\Vapour\zzaud_.wav")
#
NumSamples = len(c)
SampleRate = c.sample_rate
MaskChannels = c.channel_layout # before was = c.format.channel_layout etc.
NumChannels = c.num_channels
BitDepth = c.bits_per_sample
SampleFormat = c.sample_type
TimeLength = NumSamples/SampleRate
print("SampleRate: %5d, BitDepth: %2d, %s" % (SampleRate, BitDepth, SampleFormat))
print("MaskChannels: %3X (%4d), TimeLength: %9.3f seconds" % (MaskChannels, MaskChannels, TimeLength))
#
delay = core.std.AudioTrim(c,1,48000)
delay = core.std.AudioGain(delay,0)
c = delay + c
NumSamples = len(c)
TimeLength = NumSamples/SampleRate
print("MaskChannels: %3X (%4d), TimeLength: %9.3f seconds" % (MaskChannels, MaskChannels, TimeLength))
#
c.set_output()

2) Now access to properties in a different way

And I obtain a output of:
Warning: Use logging.info instead of print.
Information: SampleRate: 48000, BitDepth: 32, SampleType.FLOAT
Information: SampleRate: 48000, BitDepth: 32, SampleType.FLOAT
Information: MaskChannels: 3F ( 63), TimeLength: 12.000 seconds
Information: SampleRate: 48000, BitDepth: 32, SampleType.FLOAT
Information: MaskChannels: 3F ( 63), TimeLength: 12.000 seconds
Information: MaskChannels: 3F ( 63), TimeLength: 13.000 seconds
Output 624000 samples in 0.04 seconds (14269777.41 sps)

3) How Use logging.info instead of print?
I obtain the messages but some duplicated and with the undesired prefix "Information: "

- Like you can see I try to emulate a delay of 48000 samples, or 1 second, maybe that can be abreviated with:
c = core.std.AudioTrim(-48000)

- In /doc I don't see any change with the precedent version.

[ChaosKing]

With a video clip you can also trim like this clip = clip[:-30] #remove last 30 frames, maybe it works with audio too.

[tebasuna51]

I don't want remove, I want insert silence before.

[TheFluff]

If you want to add a second of silence, you can create silence with BlankAudio() and splice. If you want an actual delay, as in you want to keep the audio samples the same but adjust their timing relative to the video, I think you'd need to manipulate the video timestamps instead (shifting all frame times backwards by one second). Hypothetically you could do that by manipulating the frameprops, but I don't think that's actually a supported thing. Video timestamps don't have to start at 0 in VS and they can be negative (since that happens in the real world, e.g. in MPEG TS where the timestamps just wrap around), so that's not a problem, but the docs say that only source filters are supposed to touch the _AbsoluteTime prop. I don't know if it'd actually work even if you were allowed to change it either - I'm just theorizing here.

e: the new VS API (V4) doesn't seem to have timing info for audio, it's just a bunch of samples AFAICT. BestAudioSource has some comments indicating that exposing an audio track's delay relative to a video track (or relative starting point) is or was a planned feature though.

[tebasuna51]

BlankAudio() is not a function documented, there are BlankClip but not BlankAudio.
With my method we only need know the samplerate to emulate the avisynth DelayAudio().
We can work with samples.

I only want make a 'Avisynth Audio functions and their VapourSynth equivalents (https://forum.doom9.org/showthread.php?p=1917206#post1917206)' table like there are in the /doc for video.

[TheFluff]

For reference BlankAudio is here (https://github.com/vapoursynth/vapoursynth/blob/doodle1/src/core/audiofilters.cpp#L868) in the source code.

My post was just a suggestion, how to accomplish what you were asking for with the currently existing tools. I think your idea of a DelayAudio-like filter is a reasonable one, but I'm not the person to talk to regarding that.

[DJATOM]

def DelayAudio(aclip: vs.AudioNode, delay: float = None):
if not delay:
raise ValueError("DelayAudio: missing delay value.")
sps = aclip.sample_rate
delay = delay * sps
return aclip[delay:]

Close enough?

[TheFluff]

def DelayAudio(aclip: vs.AudioNode, delay: float = None):
if not delay:
raise ValueError("DelayAudio: missing delay value.")
sps = aclip.sample_rate
delay = delay * sps
return aclip[delay:]

Close enough?

I don't think so? that only works in one direction (making the audio start earlier by cutting off the start) and it's also inverted compared to avisynth - setting delay to 1 will make the audio start one second earlier, not one second later like in avisynth. what tebasuna wants is delayaudio(1) to add 1 second of silence to the start of the track and delayaudio(-1) to do what your function does when you say delayaudio(1) - that is, cut the first second off the track

[Myrsloik]

This all reminds me to finish implementing BlankAudio and documenting it.

[DJATOM]

I don't think so? that only works in one direction (making the audio start earlier by cutting off the start) and it's also inverted compared to avisynth - setting delay to 1 will make the audio start one second earlier, not one second later like in avisynth. what tebasuna wants is delayaudio(1) to add 1 second of silence to the start of the track and delayaudio(-1) to do what your function does when you say delayaudio(1) - that is, cut the first second off the track

Sure, so delay must be inverted before slice operation.

[Myrsloik]

Thanks for your updates.

I have time to test now and I want explain the changes I see.

1) The command line I use now is:

VSPipe.exe -c wav zzaud.vpy zzD1sec.wav

Before vspipe need --wav

The zzaud.vpy I use is:


2) Now access to properties in a different way

And I obtain a output of:


3) How Use logging.info instead of print?
I obtain the messages but some duplicated and with the undesired prefix "Information: "

- Like you can see I try to emulate a delay of 48000 samples, or 1 second, maybe that can be abreviated with:
c = core.std.AudioTrim(-48000)

- In /doc I don't see any change with the precedent version.

Logging: core.logMessage(vs.MESSAGE_TYPE_INFORMATION, 'hello there')

I'll probably have the warning removed since I find it more annoying than helpful. Keep using print() if it works for you.

One second audio delay:
clip[0]*clip.sample_rate+clip

[tebasuna51]

Thanks for your answer.

[l33tmeatwad]

Seems like BestAudioSource repository got some updates but VapourSynth4.h is out of date and by just adding in the portion it's complaining about, nfMakeLinear, it will compile but Source will not show as a function (although bas registers).

[Myrsloik]

Seems like BestAudioSource repository got some updates but VapourSynth4.h is out of date and by just adding in the portion it's complaining about, nfMakeLinear, it will compile but Source will not show as a function (although bas registers).

I plan to make a proper RC-ish build soon. Just got distracted by actual paid work as usual. There are also a few annoying/boring issues left to fix.

[l33tmeatwad]

Cool, thanks for the update, I'll keep an eye out for it.

[tbjm]

I'm trying to import the CCD VirtualDub filter, but I'm encountering some errors.

core.avs.LoadPlugin(r"C:\vsplugins\VDubFilter.dll")
core.avs.LoadVirtualdubPlugin(r"C:\vdubplugins\ccd_64bit.vdf", 'CCD', 0)

Works fine using R52, but when using R51-audio5 I get the following error.

API MISUSE! Function 'CCD' failed to register with error: Argument 'c1' has invalid type 'clip'.
API MISUSE! Function 'CCD' failed to register with error: Argument 'c1' has invalid type 'clip'.
API MISUSE! Function 'CCD_2' failed to register with error: Argument 'c1' has invalid type 'clip'.
API MISUSE! Function 'CCD_2' failed to register with error: Argument 'c1' has invalid type 'clip'.

Possibly related to https://forum.doom9.org/showpost.php?p=1919973&postcount=93 ?

[unix]

There's no way to trim/cut the Video and audio in one/same script?!

[DJATOM]

Something like that should work
from vapoursynth import core, VideoNode, AudioNode
from typing import Sequence, Union

def Trim(clip: Union[AudioNode, Sequence], start: int = None, end: int = None, fps: Sequence = None) -> Union[AudioNode, tuple]:
if not start and not end:
raise ValueError('Trim: missing "start" and/or "end" options.')
vclip, aclip = None, None
if isinstance(clip, Sequence) and len(clip) == 2:
vclip, aclip = clip
assert isinstance(
vclip, VideoNode), 'Trim: first clip must be video type.'
assert isinstance(
aclip, AudioNode), 'Trim: second clip must be audio type.'
vclip = core.std.Trim(vclip, start, end)
elif isinstance(clip, AudioNode):
aclip = clip
if not fps:
raise ValueError('Trim: missing "fps" option.')
else:
raise ValueError(
'Trim: clip must be audio type or sequence with video and audio clips.')
if isinstance(vclip, VideoNode) and not (vclip.fps.numerator == 0 and vclip.fps.denominator == 0) and fps is None:
fps = (vclip.fps.numerator, vclip.fps.denominator)
samples_per_video_frame = aclip.sample_rate / fps[0] * fps[1]
audio_start, audio_end = start * \
samples_per_video_frame, end * samples_per_video_frame
aclip = core.std.AudioTrim(aclip, audio_start, audio_end)
return aclip if not vclip else (vclip, aclip)

And how to use
vclip = core.dgdecodenv.DGSource(r'....dgi')
aclip = core.bas.Source(r'....wav', track=0)
vclip, aclip = Trim((vclip, aclip), 18544, 18568)
vclip.set_output(0)
aclip.set_output(1)

[unix]

Thank you, but if I want to trim many parts how it gonna be?!

like:
vclip, aclip = Trim((vclip, aclip), 18544, 18568).Trim(20000,240000)

?

[DJATOM]

No, it probably will not work that way. But you could test Trim((vclip, aclip), 18544, 18568) + Trim((vclip, aclip), 20000, 240000), that might work.

[DJATOM]

I've done some modifications to make possible to add few Trim instances with each other. The code is now looks like that
from vapoursynth import core, VideoNode, AudioNode
from typing import Sequence, Union


class Trim:
def __init__(self, clip: Union[AudioNode, Sequence], start: int = None, end: int = None, fps: Sequence = None) -> Union[AudioNode, tuple]:
""" Convenient wrapper for trimming audio samples by frame numbers """
if not start and not end:
raise ValueError('Trim: missing "start" and/or "end" options.')

self.vclip, self.aclip = None, None
self.mode = 'audio'

if isinstance(clip, Sequence) and len(clip) == 2:
self.mode = 'video_audio'
self.vclip, self.aclip = clip
assert isinstance(
self.vclip, VideoNode), 'Trim: first clip must be video.'
assert isinstance(
self.aclip, AudioNode), 'Trim: second clip must be audio.'
self.vclip = core.std.Trim(self.vclip, start, end)

elif isinstance(clip, AudioNode):
self.aclip = clip
if not fps:
raise ValueError('Trim: missing "fps" option.')
else:
raise ValueError(
'Trim: clip must be audio type or sequence with video and audio clips.')
if isinstance(self.vclip, VideoNode) and not (self.vclip.fps.numerator == 0 and self.vclip.fps.denominator == 0) and fps is None:
fps = (self.vclip.fps.numerator, self.vclip.fps.denominator)
samples_per_video_frame = self.aclip.sample_rate / fps[0] * fps[1]
audio_start, audio_end = start * \
samples_per_video_frame, end * samples_per_video_frame
self.aclip = core.std.AudioTrim(self.aclip, audio_start, audio_end)

def __add__(self, other):
if not isinstance(other, Trim):
raise ValueError(f'Trim: impossible to operate on non-Trim inputs.')
if self.mode == 'video_audio' and other.mode != 'video_audio':
raise ValueError(
'Trim: add failed. Probably you\'re mixing inconsistent instances (audio + video and audio or so).')
if self.mode == 'video_audio':
self.vclip += other.vclip
self.aclip += other.aclip
return self

def data(self):
if self.mode == 'video_audio':
return self.vclip, self.aclip
else:
return self.aclip

But with class rewrite we need to call method data() to return clips
vclip, aclip = (Trim((vclip, aclip), 18544, 18582) + Trim((vclip, aclip), 18687, 18724) + Trim((vclip, aclip), 18792, 18861)).data()
I tried to find a way to implicitly return data on assignment, but it seems that's too tricky.

[unix]

Thank you DJ
I will try it later

[forgetfool]

As far as I understand, BAS is Windows only solution, right?

Is there a Linux audio plugin for VapourSynth? "vs.core.damb.Read()" seems to have no effect, i.e. at does not attach sound to video.

[feisty2]

Video and audio are separate entities in vaporsynth. You may define an av container that associates an audio node with a video node, but you cannot attach an audio node to a video node.

[forgetfool]

As for the first part of the question, someone mentioned that I need to manually compile BAS (that it works for them on Ubuntu).

As for the second part, about DAMB, ok, I understand the point, but how would you actually produce a video with a sound? What is wrong with the code below? I feel I am still missing some basic understanding.

import vapoursynth as vs
src = vs.core.ffms2.Source(video.avi')
src = vs.core.damb.Read(src, "audio.wav")
src.set_output()

I am reading the documentation nere: https://github.com/dubhater/vapoursynth-damb

[feisty2]

class Muxer:
def Render(self):
self.VideoStream.set_output(0)
self.AudioStream.set_output(1)

av = Muxer()
av.VideoStream = core.ffms2.Source('video.mov')
av.AudioStream = core.bas.Source('audio.wav')

av.Render()


edit:
and damb is obsolete (it is based on API v3 which has no native support for audio), don't use it.

[jackoneill]

As for the second part, about DAMB, ok, I understand the point, but how would you actually produce a video with a sound? What is wrong with the code below? I feel I am still missing some basic understanding.

import vapoursynth as vs
src = vs.core.ffms2.Source(video.avi')
src = vs.core.damb.Read(src, "audio.wav")
src.set_output()

I am reading the documentation nere: https://github.com/dubhater/vapoursynth-damb

It's useful if you want to trim/splice the audio at the same time as the video. You use Read right after your video source filter, trim and splice the video as needed, then at the end of the script you use Write to export the audio to a file. vspipe will still output just the video. If you use only Read then the audio will be read from the file and nothing else happens.

[forgetfool]

Ok, so you are saying vspipe will never produce audio with video together, right?

I am now trying to use BestAudioSource plugin instad of DAMB (without much success either) -- see this thread: https://forum.doom9.org/showthread.php?p=1939019&posted=1#post1939019

[jackoneill]

Ok, so you are saying vspipe will never produce audio with video together, right?

I am now trying to use BestAudioSource plugin instad of DAMB (without much success either) -- see this thread: https://forum.doom9.org/showthread.php?p=1939019&posted=1#post1939019

Maybe with BestAudioSource it will. I'm not sure how it's supposed to work.

[DJATOM]

BAS uses AudioNode entity to represent audio data. Literally it's the same as VideoNode, but for audio clips. To draw data from it, user has to assign node to certain output, for example, audio.set_output(1) - assuming that your audio clip named audio.

[poisondeathray]

Can you release an updated audio build when you get the chance ?

Thanks

[xekon]

I've done some modifications to make possible to add few Trim instances with each other. The code is now looks like that
from vapoursynth import core, VideoNode, AudioNode
from typing import Sequence, Union


class Trim:
def __init__(self, clip: Union[AudioNode, Sequence], start: int = None, end: int = None, fps: Sequence = None) -> Union[AudioNode, tuple]:
""" Convenient wrapper for trimming audio samples by frame numbers """
if not start and not end:
raise ValueError('Trim: missing "start" and/or "end" options.')

self.vclip, self.aclip = None, None
self.mode = 'audio'

if isinstance(clip, Sequence) and len(clip) == 2:
self.mode = 'video_audio'
self.vclip, self.aclip = clip
assert isinstance(
self.vclip, VideoNode), 'Trim: first clip must be video.'
assert isinstance(
self.aclip, AudioNode), 'Trim: second clip must be audio.'
self.vclip = core.std.Trim(self.vclip, start, end)

elif isinstance(clip, AudioNode):
self.aclip = clip
if not fps:
raise ValueError('Trim: missing "fps" option.')
else:
raise ValueError(
'Trim: clip must be audio type or sequence with video and audio clips.')
if isinstance(self.vclip, VideoNode) and not (self.vclip.fps.numerator == 0 and self.vclip.fps.denominator == 0) and fps is None:
fps = (self.vclip.fps.numerator, self.vclip.fps.denominator)
samples_per_video_frame = self.aclip.sample_rate / fps[0] * fps[1]
audio_start, audio_end = start * \
samples_per_video_frame, end * samples_per_video_frame
self.aclip = core.std.AudioTrim(self.aclip, audio_start, audio_end)

def __add__(self, other):
if not isinstance(other, Trim):
raise ValueError(f'Trim: impossible to operate on non-Trim inputs.')
if self.mode == 'video_audio' and other.mode != 'video_audio':
raise ValueError(
'Trim: add failed. Probably you\'re mixing inconsistent instances (audio + video and audio or so).')
if self.mode == 'video_audio':
self.vclip += other.vclip
self.aclip += other.aclip
return self

def data(self):
if self.mode == 'video_audio':
return self.vclip, self.aclip
else:
return self.aclip

But with class rewrite we need to call method data() to return clips
vclip, aclip = (Trim((vclip, aclip), 18544, 18582) + Trim((vclip, aclip), 18687, 18724) + Trim((vclip, aclip), 18792, 18861)).data()
I tried to find a way to implicitly return data on assignment, but it seems that's too tricky.

WORKS GREAT! I was ripping my hair out trying to figure out how to encode some video and audio using trim/audiotrim in vapoursynth, the ONLY example I could find for working with the audio in vapoursynth was on this page but it does not include how to trim in the example:

https://www.vapoursynth.com/2020/01/audio-support-and-how-it-works/

This function is excellent and in my opinion needs more exposure, just try searching google for this:

Vapoursynth AudioTrim Example (spoiler you wont find this thread)

I ended up searching for

Vapoursynth "bas.Source"

Which still did not find this thread, but it did find this one https://forum.doom9.org/showthread.php?t=177337 in which somebody linked to this thread, so I finally found it.

What I think would be absolutely killer is if all the functions had a small minimal example script of how to use it, even better is if these examples were right on https://vsdb.top/ so when you search for a function there you can find a practical example of how to use it, this part of the site could also be community driven like a wiki...

Now that I have it working I am very very very pleased with this Audio support and the ability to do all of my processing with VapourSynth, its absolutely wonderful!

[xekon]

trim.vpy:


import vapoursynth as vs
from typing import Sequence, Union
from vapoursynth import core, VideoNode, AudioNode

class Trim:
def __init__(self, clip: Union[AudioNode, Sequence], start: int = None, end: int = None, fps: Sequence = None) -> Union[AudioNode, tuple]:
""" Convenient wrapper for trimming audio samples by frame numbers """
if not start and not end:
raise ValueError('Trim: missing "start" and/or "end" options.')

self.vclip, self.aclip = None, None
self.mode = 'audio'

if isinstance(clip, Sequence) and len(clip) == 2:
self.mode = 'video_audio'
self.vclip, self.aclip = clip
assert isinstance(
self.vclip, VideoNode), 'Trim: first clip must be video.'
assert isinstance(
self.aclip, AudioNode), 'Trim: second clip must be audio.'
self.vclip = core.std.Trim(self.vclip, start, end)

elif isinstance(clip, AudioNode):
self.aclip = clip
if not fps:
raise ValueError('Trim: missing "fps" option.')
else:
raise ValueError(
'Trim: clip must be audio type or sequence with video and audio clips.')
if isinstance(self.vclip, VideoNode) and not (self.vclip.fps.numerator == 0 and self.vclip.fps.denominator == 0) and fps is None:
fps = (self.vclip.fps.numerator, self.vclip.fps.denominator)
samples_per_video_frame = self.aclip.sample_rate / fps[0] * fps[1]
audio_start, audio_end = start * \
samples_per_video_frame, end * samples_per_video_frame
self.aclip = core.std.AudioTrim(self.aclip, audio_start, audio_end)

def __add__(self, other):
if not isinstance(other, Trim):
raise ValueError(f'Trim: impossible to operate on non-Trim inputs.')
if self.mode == 'video_audio' and other.mode != 'video_audio':
raise ValueError(
'Trim: add failed. Probably you\'re mixing inconsistent instances (audio + video and audio or so).')
if self.mode == 'video_audio':
self.vclip += other.vclip
self.aclip += other.aclip
return self

def data(self):
if self.mode == 'video_audio':
return self.vclip, self.aclip
else:
return self.aclip

core = vs.core
video = core.ffms2.Source(source='/media/storage/vpy/over9000.mkv')
audio = core.bas.Source(source='/media/storage/vpy/over9000.flac', track=-1)

video, audio = (Trim((video, audio), 28406, 28491)).data()
#video, audio = (Trim((video, audio), 18544, 18582) + Trim((video, audio), 18687, 18724) + Trim((video, audio), 18792, 18861)).data()

video.set_output(0)
audio.set_output(1)



encode audio:

vspipe -o 1 -c wav /media/storage/vpy/trim.vpy - | ffmpeg -i pipe: -c:a aac -b:a 128k -ac 1 /media/storage/vpy/audio.aac
vspipe -o 1 -c wav /media/storage/vpy/trim.vpy - | ffmpeg -i pipe: -c:a libopus -b:a 96k -ac 1 /media/storage/vpy/audio.opus


encode video with previously encoded audio:

vspipe -o 0 -c y4m /media/storage/vpy/trim.vpy - | ffmpeg -i pipe: -i /media/storage/vpy/audio.aac -c:v libx264 -threads 1 -profile:v High \
-level 4.1 -preset veryslow -crf 18 -x264-params "ref=6:bframes=16:merange=32:fast-pskip=0" -c:a copy "/media/storage/vpy/trim-x264.mkv"
vspipe -o 0 -c y4m /media/storage/vpy/trim.vpy - | ffmpeg -i pipe: -i /media/storage/vpy/audio.aac -c:v librav1e -qp 50 -speed 2 \
-tiles 1 -tile-columns 1 -tile-rows 1 -c:a copy "/media/storage/vpy/trim-av1.mkv"


remove temporary audio files:

rm -rf /media/storage/vpy/audio.aac /media/storage/vpy/audio.opus


vmaf check:

ffmpeg -i /media/storage/vpy/trim-x264.mkv -i /media/storage/vpy/trim-av1.mkv -lavfi libvmaf='feature=name=psnr|name=ciede' -f null -

[Myrsloik]

I guess some easy way to trim audio along with video would be useful. My proposal is this:
Extend Trim like this:
std.Trim(vnode clip[, int first=0, int last, int length, anode[] audio])
If audio clips are supplied then the output in python will be a dict
{clip: <video here>, audio: <array of trimmed audio clips>}

[xekon]

A built in and officially supported method of trimming both the audio and the video with one or more range of frames would be wonderful!

I am sure however you decide to do it will be Awesome!

[PatchWorKs]

Very interesting.

Do you think some Python Audio Restoration Suite (https://github.com/HENDRIX-ZT2/pyaudiorestoration) functions can be "ported" to VS ?

From thair git:
A set of tools to restore audio quality from a variety of old analog sources, such as tape, cassettes, acetates and vinyl.

Features

Wow & Flutter Removal
Speed matching to hum frequency
EQ matching with differential EQ
Spectral Temporal Alignment
Automatic Dropout Restoration
Spectral Expander / Decompressor


Hope that inspires !

[unix]

I guess some easy way to trim audio along with video would be useful. My proposal is this:
Extend Trim like this:
std.Trim(vnode clip[, int first=0, int last, int length, anode[] audio])
If audio clips are supplied then the output in python will be a dict
{clip: <video here>, audio: <array of trimmed audio clips>}

Can you elaborate and give us an example