Visualizing audio to figure out delay

[Selur]

Cleaning up my desktop, I just found a script I started a few weeks ago, where I'm trying to come up with a way to visualize audio to figure out the audio delay of a file.

Here's what I came up with so far:
import misc
# https://github.com/Selur/VapoursynthScriptsInHybrid/blob/master/misc.py

# load audio
core.std.LoadPlugin(path="%FILTERPATH%/SourceFilter/BestSource/BestSource.dll")
# bs.AudioSource(string source[, int track = -1, int adjustdelay = -1, int threads = 0, bint enable_drefs = False, bint use_absolute_path = False, float drc_scale = 0, int cachemode = 1, string cachepath, int cachesize = 100, bint showprogress = True, maxdecoders = 0])
# adjustdelay: Adjust audio start time relative to a video track number. Pass -2 to disable and -1 to be relative to the first video track if one exists. Specifying a non-video track is equivalent to passing -2. Note that the offset is always relative to the first CPU-decodable frame in the stream meaning that it may not be the correct delay when hwmode and variableformat are used.
# see: https://github.com/vapoursynth/bestsource
audio = core.bs.AudioSource(source="%INPUTFILE%", track = -1, adjustdelay = -2)

delay_ms = 0 # Here you can set custom delay values
audio = misc.DelayAudio(audio, delay_ms=delay_ms)

# rendering of an analog oscilloscope to display audio waveforms
core.std.LoadPlugin(path="%FILTERPATH%/Support/faveworm.dll")
# https://gitlab.com/EleonoreMizo/faveworm
sweep = clip.fps_den / clip.fps_num # 1/fps
scope = core.fw.scope(clip=clip, audio=audio, mode=2, sweep=sweep, beam_size=8, y_gain=1.0, y_ofs=-0.3) # doubles the visual height of the waveform

# draw vertial line every 10ms
scope = misc.AddVerticalLines(scope, interval_ms=10, color=1.0)

# overlay scope on video and display multiple frames instead of just one
compareFrameCount = 3 # number of frame to show
clip = misc.Overlay(clip, scope, opacity=0.5)
clip = misc.ShowFramesAround(clip, count=compareFrameCount)
https://i.ibb.co/5W1Fxqjx/grafik.png (https://ibb.co/svgHJL1J)

Just wanted to share this:
a. in case someone has use for it.
b. to ask : Has someone a better way to do this or suggestions on how to improve this?

Cu Selur

[Adub]

This is pretty darn cool. I have a few projects I might use this with.

Thanks for sharing!

[Columbo]

Related plugins already existing are Waveform and AudioGraph.

[Selur]

There is a Waveform for Vapoursynth? :eek:
(Only thing I know of is faveworm (https://gitlab.com/EleonoreMizo/faveworm) which I used,...)
Can you share links to Vapoursynth versions of Waveform and AudioGraph?

Cu Selur

[StainlessS]

There is also WaveForm_FilmStrip(), [EDIT: Avisynth, Req Wonkey_Donkey Waveform() plug ]
https://forum.doom9.org/showthread.php?p=1876933#post1876933
[and a couple more in that thread too, I think]

Window=3 W=128 [individual thumb width = 128 : Full width (Window*2+1)*128 = 7*128=896 ]
https://i.postimg.cc/26N4fw7f/Wave-Form-Film-Strip-00.jpg (https://postimages.org/)

Window=3 W=-1(total width smaller or equal original width[640], out width=630[each thumb width mod 2, ie (Window*2+1)*90 = 7*90=630])
https://i.postimg.cc/SxX825Zh/Wave-Form-Film-Strip-01.jpg (https://postimages.org/)

EDIT: Also NOTE, it is kinda pointless using massive clips for this purpose (HD/FHD+), little tiddlers are usually quite sufficient.
(Selur images download real slow, I'm assuming are big. I gave up on download/view).

https://i.postimg.cc/6p1FqnD7/Waveform-Filmstrip.gif (https://postimg.cc/xXM6FJjY)

[Columbo]

Sorry, missed the Vapoursynth requirement. :confused:

[Selur]

@StanlessS: That uses Avisynth, not Vapoursynth which I'm looking for,...

[StainlessS]

Yip sorry, I found the post via "Quick Links/Todays posts", did not realize was for Vapoursynth.

EDIT: So long as there is some kind of "WaveForm" like function in VS, then should not be too difficult to script something similar,
there is not much to Waveform_FilmStrip if you dont count the embedded docs.


Function WaveForm_FilmStrip(clip c,int "Window",Float "Height",Bool "Under",Float "Zoom",Float "SubSize",Bool "ShowAudio",Int "W",Int "H") {
/* Waveform_FilmStrip:- by StainlessS @ Doom9 ::: https://forum.doom9.org/showthread.php?p=1965005#post1965005
Where ShowAudio=True, Requires WaveForm v2.0 plugin (c) Wonkey_Monkey, https://forum.doom9.org/showthread.php?t=182866
[Wonkey_Monkey, The Prince formerly known as David.]

WARNING - Marks arg no longer a setting in WaveForm v0.3 x64 or v2.0 - so removed from WaveForm_FilmStrip
Also, Made Under default False as Suggested by Wonkey.

WaveForm_FilmStrip(clip c,int "Window"=0,Float "Height"=0.333,Bool "Under"=FALSE,Float "Zoom"=1.0,Float "SubSize"=0.0,Bool "ShowAudio"=True,Int "W"=-1,Int "H"=W_Based)

Window, Default 1 (0 <= Window <=3), Filmstrip thumbnail Images across = 2 * Window + 1.
Height, Default 0.333=1/3 of FilmStrip Height. See Waveform docs for more.
Under, Default False, If True, then audio graph stacked under video, else overlay over video.
Zoom, Default 1.0. Amplify audio signal for viewing (1.0=No Amp).
SubSize, Default 0.0(0.0 <= SubSize <= 1.0). Suggest 0.0(No Frame Number) to about 0.5. : 1.0=biggest size for current clip Height (probably too big).
ShowAudio, Default True. If False, Switches OFF WaveForm Display, Shows FilmStrip Only.
W, Default -1. Governs width of thumbnail images making up the FilmStrip.
-ve) Entire FilmStrip width (multiple thumbnail images based on Window arg) will be about same [smaller or equal] as input clip width.
0) Each FilmStrip thumbnail image is SAME size as input clip, result could be a quite wide clip.
Else) Each FilmStrip thumbnail image Width is set by W (best Mod 2).
H, Default based off W. (Suggest Dont supply H arg, let it default based on W arg. Can call with H=Undefined where will behave as if defaulting).
IF H is Supplied ie not defaulted THEN H = Supplied H # could be -ve, 0, or +ve(explicit height).
else if W <= 0, then H = W, ELSE H = input clip height Aspect Ratio Scaled to match W.
Here, H could be -ve, 0, or +ve, defaulted or non defaulted.
Where H,
-ve) FilmStrip height scaled down to Aspect Ratio match FilmStrip width of individual thumbnail images.
0) Each FilmStrip thumbnail image Height is SAME size as input clip height.
Else) Each FilmStrip thumbnail image Height is set by H (best Mod 2).
*/
Window=Default(Window,0) Height=Max(Default(Height,0.333),0.0) Under=Default(Under,false) SubSize=Default(SubSize,0.0) ShowAudio=Default(ShowAudio,True)
W=Default(W,-1) H=Default(H,W<=0?W:Int(W.Float/c.Width*c.Height)/2*2)
Assert(0 <= window <= 3,"WaveForm_FilmStrip: 0 <= window <= 3")
Assert(0.0 <= SubSize <= 1.0,"WaveForm_FilmStrip: 0.0 <= SubSize <= 1.0")
Frms = 1 + (2*Window) WW=(W<0)?c.Width/(Frms*2)*2:(W==0)?c.width:W HH=(H<0)?c.Height/(Frms*2)*2:(H==0)?c.Height:H
sc=(WW!=c.Width||HH!=c.Height) ?c.BiCubicResize(WW,HH,b=-0.5,c=0.25) :c scSubsSz=Round(sc.Height*SubSize)
SSS="""Subtitle(String(current_frame,"%.0f")""" + String(scSubsSz,",SIZE=%.0f") + String((sc.Height-scSubsSz)/2.0,",Y=%.0f") + """,align=8)"""
sc=(scSubsSz!=0)?sc.ScriptClip(SSS):sc bc=sc.BlankClip(Length=1,Color=$000000)
p1c=bc+sc p2c=bc+p1c p3c=bc+p2c
n1c=sc.FrameCount>1?sc.Trim(1,0)+bc:bc n2c=n1c.FrameCount>1?n1c.Trim(1,0)+bc:bc n3c=n2c.FrameCount>1?n2c.Trim(1,0)+bc:bc
Select(Window,sc,StackHorizontal(p1c,sc,n1c),StackHorizontal(p2c,p1c,sc,n1c,n2c),StackHorizontal(p3c,p2c,p1c,sc,n1c,n2c,n3c))
\ .AudioDubEx(c).Trim(0,-c.FrameCount).DelayAudio(0.0)
return (ShowAudio && HasAudio) ? WaveForm(window=Window,height=Height,under=Under,zoom=Zoom) : Last
}

[Selur]

Ah, okay. :(

[StainlessS]

NOTE, IIRC, the DelayAudio(0.0) on 2nd from last line,
is a kludge to fix some AVS problem, problem was fixed in later AVS but I left in-situ anyway,
so probably not necessary in VS ver$.

[Selur]

Ok, at least I won't try to port that (horribly formatted) script, but maybe someone else wants to try it. :)

Cu Selur