VapourSynth - AUDIO SUPPORT AND NEW API BETA

[_Al_]

testing about these, so it might help someone
API4 audio_test.py script:

import vapoursynth as vs
from vapoursynth import core
source_path=r"video.mp4" #stereo audio
v = core.lsmas.LibavSMASHSource(source_path)
a = core.bas.Source(source_path)
v.set_output(0)
a.set_output(1)

some line here are taken from somewhere else from this forum:

AUDIO
VSPipe -o 1 -c wav audio_tests.py - | neroAacEnc -ignorelength -lc -cbr 96000 -if - -of nero.m4a
VSPipe -o 1 -c wav audio_tests.py - | ffmpeg -y  -i - ffmpeg_output.wav
VSPipe -o 1        audio_tests.py - | ffmpeg -y -f f32le -ac 2 -ar 48000 -i - ffmpeg_output.wav
VSPipe -o 1 -c wav audio_tests.py  vspipe_to_wav.wav   #but double size (32bit if original is 16bit)
VSPipe -o 1        audio_tests.py  vspipe_to_wav.wav   #but double size (32bit if original is 16bit)
VSPipe -o 1 -c wav audio_tests.py - | ffplay -
VSPipe -o 1        audio_tests.py - | ffplay -f f32le -ac 2 -ar 48000 -i -
VSPipe -o 1 -c wav audio_tests.py - | ffmpeg -i video.mp4 -i - -map 0:0 -map 1:0 -f AVI -c:v utvideo -pix_fmt yuv420p -colorspace bt709 -c:a pcm_s16le -y ffmpeg_utvideo.avi

VIDEO
VSPipe -o 0 -c y4m audio_tests.py - | ffplay -i -
VSPipe -o 0 -c y4m audio_tests.py - | mpv -

btw. can be two nodes (vnode,anode) piped into ffmpeg or ffplay in one line?

[Myrsloik]

No, to do that vspipe needs to wrap the streams into a more complex container like mkv.

[Myrsloik]

Originally Posted by _Al_

All working here so far, this is nice.
Also attempt to drag audio along with video, here in vapoursynth attaching audio to video frames using props.
But the code below does not work because ModifyFrame does not work with anode. vnode only. ModifyFrame (or FrameEval) is planned in future or is it a bad idea?

oh I see, n is not the same for vnode as is for anode, this could not work, only if bunch of anode frames (perhaps list) is attached to vnode and then reconstructed again, no idea how anisynth does it,

also deleting previous code to not confuse anyone

Frame props are nonsensical since audio frames are a very arbitrary thing. That's why I haven't really exposed the manipulation and I want audio support to mostly allow you to pass audio through in a convenient way.

I think what you should do it simply create wrapper functions for the trim, splice and other reorder functions and use those. Most filters don't reorder the clip. That would be my first attempt. I'll write an example some day of it.

Avisynth has a weird symbiotic relationship between one audio track and one video track. Have more than one audio track and things get iffy.

[feisty2]

you can attach audio frame(s), possibly audio frames from multiple audio tracks, to a video frame using the C++ API, it should be no more complicated than

auto [PictureFrame, EngAudioFrame, FrenchAudioFrame] = Node::AcquireFrameGiven(Index, GeneratorContex).From(Video, EngAudio, FrenchAudio);
auto CompositeFrame = Core.CopyFrameFrom(PictureFrame);
CompositeFrame["_Audio"] += EngAudioFrame;
CompositeFrame["_Audio"] += FrenchAudioFrame;

[Myrsloik]

R55-API4-RC4

Changes:

• Fixes the expr issue causing weird values
• Fixes lots of vfw and avfs bugs that snuck into RC3
• No longer bundles any filters at all
• Comes with a working vsrepo to help you with the previous point
• Has a half-reworked documentation with a new theme
• AverageFrames is now a core function

Will release an api3 build too with the expr and vsrepo fixes if it's working here.

[Boulder]

The expr issue seems to be fixed, MDSI produces normal scores.

[Yomiko]

How does VSFilterDependency help? If I invoke another filter to get a vnode, should I also include it in the dependency list for filter creation?

[Myrsloik]

Originally Posted by Yomiko

How does VSFilterDependency help? If I invoke another filter to get a vnode, should I also include it in the dependency list for filter creation?

If your filter requests frames from the node it should be in the list. Simple rule.