waveform - audio waveform display

[wonkey_monkey]

This filter is now deprecated in favour of version 2.0

Inspired by audiograph, waveform is a plugin to overlay audio waveforms on video clips - useful for identifying sync issues and audio glitches.

Download: waveform0.3.zip



(these screenshots are now out of date; see post further down)

No prizes for guessing what my favourite TV show is...

Improvements over audiograph are:

• A nicer, variable height overlay
• Separate graphs for each channel, works with an arbitrary number of channels
• Works with YV12 video, as well as YUY2/RGB24/RGB32
• Audio format is passed through unchanged

waveform
========

An Avisynth plugin to display audio waveforms superimposed on video clips.

Usage
=====

waveform(clip, int "window", float "height", bool "under", float "zoom", bool "marks")

Parameters:
-----------

window: the number of frames either side to display a waveform for (default: 1)
height: >=1 : height in pixels of each audio channel displayed
        <1 : total height of waveform display as a fraction of video height (default: 0.333)
under: if true, display the waveform underneath the video instead of superimposed on it (default: false)
zoom: scale up the waveform (default: 1)
marks: display frame boundary marks (default: true)

[Wilbert]

Nice! It would me even more nicer if you make it open source. Is there a reason you keep your plugins closed source?

[wonkey_monkey]

Only embarrassment over my lazy coding

(download updated to include source)

[RedDwarf1]

Thanks, I will give it a try.

You must be looking forward to the new series very soon

[StainlessS]

Lovely, Thank you.

[wonkey_monkey]

Originally Posted by RedDwarf1

You must be looking forward to the new series very soon

New series?!

I've made a couple of tweaks to the filter - a new parameter, zoom, magnifies the waveform vertically without amplifying the clip's audio (this may result in what looks like clipping, but isn't), and past/future frames are now darker than the current frame (this is because I spent half-an-hour wondering why my clip contained a glitch I thought I had trimmed out, when actually my VirtualDub window was cropped and I wasn't looking at the right section of the waveform... )

The filter will also now overlay the clip with "waveform: no audio" if there is, fittingly, no audio.



David

[StainlessS]

Very nice work, love the zoom option, 1st usage v0, thought it was not working because of the
nearly flat line audio, it was just quiet. Much better that AudioGraph.
Was on my list of things to do, clean up AudioGraph, not gonna bother now.
Good stuff, thanks very much.

EDIT: PS, request future version numbers on Zip file.

[pandy]

Nice one, very useful - Thank You!

[StainlessS]

David, finally had a little play with it (second version), dont work I'm afraid.

Waveform(window=1,height=0.333,under=false,zoom=1.0) # Set to stated defaults, nothing happens (YV12/YUY2/RGB32
Waveform(window=0,height=0.0,under=false,zoom=1.0) # Set to actual defaults, nothing happens (YV12/YUY2/RGB32
Waveform(window=2,height=0.333,under=true,zoom=1.0) # Green screen (YV12/YUY2/RGB32
Waveform(window=1,height=0.333,under=true,zoom=1.0) # YV12 green screen
Waveform(window=1,height=0.333,under=true,zoom=1.0) # YUY2 green bottom screen
Waveform(window=1,height=0.333,under=true,zoom=1.0) # RGB32 green bottom screen
Waveform(window=1,height=60.0,under=true,zoom=1.0) # YV12 Multicolored Greens

At no time saw any audio graph.

Old version worked fine.

Also, cpp file is not for current dll, it does not compile due to vsprintf_s, needs additional header
and 1 too few args. EDIT: Suggest change vsprintf_s to vsprint, then debug() should work OK I think.

Also some kind of planar misalignment in YV12, and dont like use of memcpy/memset.
Pitch can be greater than Rowsize, and start of row not necessarily aligned at offset 0 WRT pitch,
if you splat the last raster line with memset at size pitch, it can wrap around and mash up memory that does not
belong to the last scanline, offset 0 + pitch is start of next row but the could be mem beween there and end of last row (eg crop).
Did not look any further into code, looks well compilcated.

Keep at,

PS, I was wanting to test the version preroll that I did, as you did not even acknowledge its existence, I guessed
it was of no interest to you and so did not bother any further with it. It has been tested in sofar as it dont crash,
but I did not try to find any dodgy audio to fix.
By the way, typical request for samples is 6000, not 60,000 as I mis-remembered in preroll thread, maybe
about 1/8 second with 48khz audio.

[wonkey_monkey]

Originally Posted by StainlessS

David, finally had a little play with it (second version), dont work I'm afraid.

At no time saw any audio graph.

Old version worked fine.

Currently working through your suggested fixes (not sure what I did wrong but the uploaded dll is definitely broken), but I've discovered a crash I can't narrow down - something to do with GetAudio, but it only seems to arise when the video is small (like version is), and happens even if I bypass all the drawing code entirely.

David

[wonkey_monkey]

Okay, I give up on the crash for now - for all I know right now, it could be some bizarre bug in tone itself (replace tone with a wavsource of tone and the crash is gone).

Anyway, better that there is at least a somewhat-working version (no idea what I did to that last dll) for people to use...

Download: waveform0.2.zip

I commented out the function which uses vsprintf_s and hopefully fixed those rowsize/pitch issues (but only for YV12/YUY2 so far).

Still not sure what you meant about the YV12 misalignment?

David

[Wilbert]

Currently working through your suggested fixes (not sure what I did wrong but the uploaded dll is definitely broken), but I've discovered a crash I can't narrow down - something to do with GetAudio, but it only seems to arise when the video is small (like version is), and happens even if I bypass all the drawing code entirely.

Could you post the script which crashes?

[StainlessS]

I think this one, caused the suspected planar misalignment,mostly greenish, with a sort of yellow separate image with offset (I think).
Waveform(window=1,height=60.0,under=true,zoom=1.0) # YV12 Multicolored Greens

I think if you re-establish the debug() func as suggested, it should work OK and be of use in debugging, so can use sprintf type syntax.
The _s version is I think an MS "Safe" version which prevents buffer overflow, as buf is 1024, you are unlikely to overflow it unless
deliberate. The _s version, just has an extra arg for the buffer size (which is missing in your source) so it looks like a failed attempt
to use the _s version.
There is a thread/s somewhere, about AudioGraph, the cache bug in 2.58, and if I remember correctly; about it being not quite thread
safe to pull in audio from a video GetFrame. Think IanB and Wilbert participated in those threads.

Oh, and about the memset stuff, could I think use something like "n * pitch - (Pitch-Rowsize)" which should account for the wrap around.

[wonkey_monkey]

Originally Posted by Wilbert

Could you post the script which crashes?

Sure, it was simply this:

version.audiodub(tone).waveform(1).info

It would crash, usually, at 66, and using DebugView it didn't seem to be getting further than my GetAudio.

Inserting a "getchannel(1)" before the the waveform(1) stopped it crashing. Swapping "version" for "stackvertical(version,version)" moved the crash point to around frame 34. Changing to "version.bicubicresize(720,576)" would result in no crashes.

I should point out I'm currently back on AviSynth 2.58, since the only reason I moved to 2.6 was to try to solve my preroll problem, which I have a filter for now (and a new version of NicAudio), so...

There is a thread/s somewhere, about AudioGraph, the cash bug in 2.58, and if I remember correctly; about it being not quite thread
safe to pull in audio from a video GetFrame. Think IanB and Wilbert participated in those threads.

...this may be the reason.

David

[wonkey_monkey]

Originally Posted by StainlessS

Oh, and about the memset stuff, could I think use something like "n * pitch - (Pitch-Rowsize)" which should account for the wrap around.

I'm very tired tonight and I've just realised I managed to screw up the rowsize stuff earlier. I've reuploaded the zip file, again I think it should be right now (and now includes your suggestion, thanks!).

David

[Gavino]

Originally Posted by StainlessS

There is a thread/s somewhere, about AudioGraph, the cache bug in 2.58, and if I remember correctly; about it being not quite thread
safe to pull in audio from a video GetFrame.

You're probably thinking about this post.

As IanB says, the problem should be fixed for Avisynth clients that use the VfW interface, but some use the API directly instead.

[lisztfr9]

Hi,

These plugins are nice but i tried to use it for adjusting subtitle delay, maybe they are not suitable for that....?

L

[Sparktank]

Thank you so much for this!
Not only practical in serving a useful purpose, but looks quite fun as well ^__^

I am going to have so much fun with this.

-----

I don't know how difficult requests can get; but is there any way to add a color spectrum to the wave, based on volume level (dB)?

-----

This is actually quite interesting to use.
I'm using "AVS 2.6.0 Alpha 3 [110525]" with SEt's "Avisynth 2.6 MT 2012.08.28". (Though I haven't implemented any SetMT functions in the script, yet.)

I was having issues at first trying to get a working script.
At first, I got the video to load without problems in VirtualDub,and added in the AudioDub(v,a) at the end.
But it was telling me the waveform was an unsupported colorspace.
Which was extremely odd.
I tinkered with the script and finally looked at your short script and clued in to get it working.

The script finally worked for me and I was able to get VD running without errors.

Then I decided to try with the waveform underneath the video and do some cropping.
This took some math and trial and errors to get the aspect ratio correct, but I finally got it (I think).

For fun, I've taken up a new project.
I plan to take my Bluray of Big Trouble In Little China and convert it to DVD with the black bars cropped, proper aspect ratio kept, waveform added (waveform will calculated to replace black bars) for the isolated score (one of the special features), and use the main audio for playback.
The isolated score won't be playing during the video, I've loaded the main movie audio in VD separately while the isolated score is in the AVS script.
It should really make for a fun watching experience to "see" the music play throughout the movie.



A Question About Zoom:
Is there a maximum you can input?
I've tried anywhere from 0.5 to 90.5 and haven't really seen a difference. Am using it wrong?



I converted the audio to w64 using eac3to and used NicAudio 2.06 to import.

Here's a screencap of current results (MPC overriding AR to 16:9)


Here's the script I used...

Video = DirectShowSource("L:\1080p (Remux)\Big Trouble In Little China\Big Trouble In Little China.mkv", fps=23.976, audio=false)
Video = video.crop(0,134,-2,-134)
Video = video.spline144resize(720, 304)
Audio = RaWavSource("L:\1080p (Remux)\Big Trouble In Little China\Big Trouble In Little China.w64", SampleRate=48000, SampleBits=24, Channels=6)

AudioDub(video,audio).Waveform(1, .366, true, 1).LanczosResize(720,480)
### Override AR to 16:9 in media player for preview and to convert to DVD in AVStoDVD/HCenc

I haven't slept yet. I may have done something wrong in that script.

[Reel.Deel]

@davidhorman

I'm having some issues with waveform. I'm using the following script in AviSynth RC 1 MT (OS: 64-bit Windows 7).

audio = last #16-bit stereo in my case

fps = 30000
den = 1001
length = Round((AudioDuration(audio)*fps)/den)
BlankClip(length, width=1920, height=512, fps=fps, fps_denominator=den, pixel_type="YV12", color_yuv=$008080)
AudioDub(last, audio)
waveform(window=45, height=height/2, under=false, zoom=2.0)

With window=45 I get this error message:

Avisynth: access violation at 0x0000281A in waveform.dll, attempting to write to 0x061AFC60

With window=50 VDub just freezes but with window=240 it works.

Also I was wondering if you can add a few options to waveform.
1) Option to select the background color. I thought waveform would work something like Histogram("AudioLevels") when used with BlankClip but I cannot get the desired color after waveform.
2) Option to only display the the pure waveform without the the window marks (center ones are fine, is just when the window is set high the marks are very close together and become a little intrusive).
3) Can anything be done to make waveform scroll a bit more smoothly?

[wonkey_monkey]

Try this, which I think fixes the crash:

http://horman.net/avisynth/download/waveform0.3.zip

New parameter "marks" can be set to false remove the vertical lines.

1) Option to select the background color. I thought waveform would work something like Histogram("AudioLevels") when used with BlankClip but I cannot get the desired color after waveform.

That histogram example looks nothing like waveform's output, so I'm not sure what you're after, but in any case I think changing background colour is a bit outside waveform's remit, being meant only as a diagnostic filter. You could adjust the input video before adding waveform, or you could take waveform's output and use other filters on it, but for speed reasons all waveform does is shift the underlying pixel values one bit to the right.

2) Option to only display the the pure waveform without the the window marks (center ones are fine, is just when the window is set high the marks are very close together and become a little intrusive).

Done, but only as all-or-nothing.

3) Can anything be done to make waveform scroll a bit more smoothly?

Not really. It has to scroll a certain number of pixels per frame, and that's determined by the window size and the framerate of the video.