Cutting/editing audio - Avisynth mangling it?

[wonkey_monkey]

Hi all,

I've just come across this oddity. I've cut a clip from a DVD rip in two using trim, then spliced the two clips back together with +. Using audiograph I can see the waveform of the audio, and on first stepping through the frames in VirtualDub (so not playing audio) it looks like this:

http://img443.imageshack.us/img443/2818/goodhdh.jpg

But after playing back any of the audio (before, during, or after the splice), it becomes this:

http://img163.imageshack.us/img163/4215/badd.jpg

And the glitch is audible. Reloading the AVS file resets it again.

Can anyone shed some light as to what's going on, and tell me if there's any way to avoid this?

David

[hartford]

Most interesting. I've not seen this before.

What version of Avisynth are you using?

[poisondeathray]

Try alignedsplice ++ instead of unalignedsplice +
http://avisynth.org/mediawiki/AlignedSplice

If you have VBR audio use EnsureVBRMP3Sync
http://avisynth.org/mediawiki/EnsureVBRMP3Sync

[wonkey_monkey]

My AviSynth version is 2.58. I tried ++ and the same thing occured. The audio is AC3, loaded with nicac3source.

David

[wonkey_monkey]

Strangely (at least it seems strange to me) EnsureVBRMP3Sync may have "fixed" it. On the downside, it only works if I have Audiograph in the chain, otherwise it refuses to play (VirtualDub just sticks at "Dub in progress" but doesn't progress). It also seems to make Audiograph even more unstable.

Davd

[wonkey_monkey]

More info: it seems I don't have to play any audio at all. If I step through to the cut in VirtualDub, the first time I go by it shows the clean join, the second time (either forwards or backwards, and as long as I go far enough [to get it cleared from a cache?]) it's glitched.

I wonder if it's a problem with nicac3source - but then how would nicac3source know to return anything other than the correct unglitched audio when it's requested? Would it "know" that the audio is at the start/end of a clip and fade it like this?

David

[poisondeathray]

it only works if I have Audiograph in the chain, otherwise it refuses to play (VirtualDub just sticks at "Dub in progress" but doesn't progress)


was it 6ch audio? I think vdub has problems playing 6ch audio through avs script

but the same video loaded directly works, or playing .avs in media player works, or if you downmix to stereo it works

it doesn't seem to matter how it's loaded (different source filters, nicac3source, ffaudiosource, etc.... or if you use separately with audiodub() )

this seems to occur on a variety of different video samples, and at least a few people have experienced similar results, so I think it's a vdub bug

see this discussion
http://forum.videohelp.com/threads/344362-VirtualDub-doesn-t-play-audio-in-preview

[wonkey_monkey]

It is initially 6ch AC3, but the first thing I do after loading it is a getchannel(3,3) to have just the center channel (while still keeping it stereo for later mixing). In this case it just seems to have been EnsureVBRMP3Sync (without audiograph) that broke it.

David

[IanB]

This looks like the Audio cache stuff up fixed at version 1.33 of cache.cpp, "Bugfix AudioCache snafu (Gavino)".

Try the AviSynth 2.6.0 Alpha3 [May 25th] which includes this fix.

[wonkey_monkey]

That seems to have sorted it out, thanks Ian!

David

[wonkey_monkey]

Just to report I have just spotted the same glitch (using the same clips as my last report, regardless of + or ++) happening even with 2.60 Alpha 3 - on first view there is no visible splice, but after advancing (but not rewinding) more than about 7 frames, then returning, the splice is visible/audible.

(I'm currently trying to reproduce the problem using blankclip and tone so it's easily reproducible, but haven't had any luck yet)

If I was running my script through Avidemux via AVSproxy, for example, would this bug be avoided because none of the audio would have been cached? I'm really hoping this to be true because my project is now nearly 15 minutes long with lots of tricky cutting!


Edit: it seems this is occurring throughout my project wherever I've cut/spliced a clip - which is a lot. Is there any way to disable audio caching entirely - if that's practical - even if I have to compile my own DLL?

David

[IanB]

The cache if it is working correctly should make no difference, apart from performance. If the audio sample number 123456 has a value of 42 then each and every time you fetch that sample the value must be 42.

The problem fixed at version 1.33 of cache.cpp, meant that under some conditions the buggy version of the cache mistakenly returned samples from a different part of the clip, catastrophic, yes.

The other issue with the cache is how it complicates requests to an audio source filter. In a simple linear only access script it is very simple, every audio sample is requested once only and in order. With Trim's, MergeChannel's and other audio processing the situation can become very complex. The source filter will see requests for audio samples in chunks and in an order determined by the topology of the script audio processing graph.

The cache can suppress some of these requests if that data has previously been requested. When the cache buffer is not big enough to satisfy all requests for duplicate data then the source filter will see partial duplicate requests for only the uncached data samples i.e. cache misses. The audio cache is self tuning, it initially starts in the OFF state. When it receives requests for data that have been previously requested it enters the ON state with a modest buffer size. In the ON state it monitors for partial cache miss events, when they occur the cache increases it's size. So initially the source filter will see full duplicate requests, then some partial duplicate requests and eventually only initial requests that reflect overall gross scene trimming and order. If the source filter cannot reliably seek and return the identical samples as it did for the first time request then things can indeed be very dire.

The EnsureVBRMP3Sync() filter does 2 things, one it adds an initial 1MB audio cache, and two it enforce a linear order of access for requests to the source filter. Any backwards order requests result in a seek to the very beginning followed by a series of slow linear requests up to the required access point.

If you want to roll your own avisynth.dll without audio cacheing. Edit src\core\cache.cpp look for Cache::GetAudio. After the range checking (before zero, after end of clip) and protection code add a return child->GetAudio(start, count, env); Good luck.

[wonkey_monkey]

The other issue with the cache is how it complicates requests to an audio source filter. In a simple linear only access script it is very simple, every audio sample is requested once only and in order. With Trim's, MergeChannel's and other audio processing the situation can become very complex. The source filter will see requests for audio samples in chunks and in an order determined by the topology of the script audio processing graph.

Ah, so it could be that the non-linear access is confusing nicac3source, rather then it being a problem with AviSynth - which might also explain why I haven't yet found the issue on sources where I've used ffmpegsource2 (which I can't do on all sources since it's not frame-accurate with VOBs).

If you want to roll your own avisynth.dll without audio cacheing. Edit src\core\cache.cpp look for Cache::GetAudio. After the range checking (before zero, after end of clip) and protection code add a return child->GetAudio(start, count, env); Good luck.

Thanks!

David

[tebasuna51]

Ah, so it could be that the non-linear access is confusing nicac3source,...

And all decoders.

If I have a Test.ac3 and split the ac3 in two, Test1.ac3 (frames 0 to 284, 285 frames at 32 ms -> 9120 ms) and Test2.ac3 (the rest), and decode the full stream to wav and the two parts you can see in the attached image a glitch (5 ms) similar to the image in your first post.

To decode a frame correctly the decoder need a initilization value from previous frame, if isn't present the first 5 ms are incorrect and can produce audible glitches.

If AviSynth request a sample exactly a 9120-9125 ms, and the previous decoded frame isn't the 284, the decoder isn't correctly initialized to decode the frame 285.

I don't know for what you need manage video+audio with many Trims, but I recommend you decode the full ac3 to wav before and use RaWavSource to add the audio.

[mastrboy]

a workaround might be using split_aud: http://mod16.org/hurfdurf/?p=33 and muxing in the output from that later instead of encoding the audio within avisynth script containing trim()

edit: almost forgot, https://github.com/wiiaboo/vfr/ is a python alternative to split_aud

[wonkey_monkey]

And all decoders.

I don't know for what you need manage video+audio with many Trims, but I recommend you decode the full ac3 to wav before and use RaWavSource to add the audio.

Thanks for the idea - but as my sources amount to over 60 hours of video and my script has almost 1000 trims it may not work in this case :)

It seems that my earlier problems with EnsureVBCMP3Sync are no longer occurring (previously it wouldn't play back at all unless I had audiograph in my script), and that seems to be solving the problem - just not sure how much it will slow down my encoding yet!

David

[IanB]

@tebasuna51,

Would a preroll decoding of the previous frame when ever you seek out of order get rid of the 5ms glitch or is the dependency more complicated?

[tebasuna51]

Thanks for the idea - but as my sources amount to over 60 hours of video and my script has almost 1000 trims it may not work in this case
60 hours! Well, if you have enough disk space is not a problem.
RaWavSource can support WAV files greater than 4 GB or W64 files.
You can use eac3to or BeHappy to decode AC3 to W64.

Would a preroll decoding of the previous frame when ever you seek out of order get rid of the 5ms glitch or is the dependency more complicated?
Maybe. I will see.

[wonkey_monkey]

@tebasuna51,

Would a preroll decoding of the previous frame when ever you seek out of order get rid of the 5ms glitch or is the dependency more complicated?

That was going to be my next question - all of my sources run through a function to add a timecode, so if I added to that a custom filter which automatically fetches frame n-1 (or at least, the audio) when frame n is requested, would that likely solve the problem?

Would such a filter be able to keep track over all its instances of which frame from which source was previously decoded? Or can it only do so per instance?

60 hours! Well, if you have enough disk space is not a problem.

I don't :(

David

[wonkey_monkey]

Preliminary results suggest that this filter solves the problem! :D

#include "stdio.h"
#include "windows.h"
#include "avisynth.h"

class preroll : public GenericVideoFilter {
public:
preroll(PClip _child, IScriptEnvironment* env);
void __stdcall GetAudio(void* buf, __int64 start, __int64 count, IScriptEnvironment* env);
};

preroll::preroll(PClip _child, IScriptEnvironment* env) : GenericVideoFilter(_child) {
}

void __stdcall preroll::GetAudio(void* buf, __int64 start, __int64 count, IScriptEnvironment* env) {
if (start!=0) child->GetAudio(buf, start-count, count, env); // if this isn't the first frame, get the audio for the previous one
child->GetAudio(buf, start, count, env); // now get the audio for the current frame
}

AVSValue __cdecl Create_preroll(AVSValue args, void* user_data, IScriptEnvironment* env) {
return new preroll(args[0].AsClip(),env);
}

extern "C" __declspec(dllexport) const char* __stdcall AvisynthPluginInit2(IScriptEnvironment* env) {
env->AddFunction("preroll", "c", Create_preroll, 0);
return "`preroll' preroll plugin";
}

This is very rough - I've never written an audio filter before, and don't quite understand the principles yet. So if anyone more experienced could cast a critical eye over the code or test it out (preroll.zip (http://horman.net/preroll.zip)), it would be greatly appreciated.

To do: skip the pre-fetch if that previous frame was the last "current" frame (may not be possible with multiple instances of the filter running)

David

[tebasuna51]

Maybe is easy to implement.

Now if the desired frame isn't the next the decoder is reset and search the desired frame:

// Seek to the frame, if required
if (FrameIndex != _FrameIndex + 1)
{
// Reset decoder
if (State)
a52_free(State);
State = a52_init(Accel);

_fseeki64(Stream, StreamOffset + __int64(FrameIndex) * __int64(FrameLength), SEEK_SET);
}

// Read the frame
if (!ReadFrame())
env->ThrowError("m2AudioAC3Source: error in file \"%s\"", StreamName);

But we can read the previous frame first:
// Seek to the frame, if required
if (FrameIndex != _FrameIndex + 1)
{
// Reset decoder
if (State)
a52_free(State);
State = a52_init(Accel);

if (FrameIndex == 0)
_fseeki64(Stream, StreamOffset, SEEK_SET);
else
{
_fseeki64(Stream, StreamOffset + __int64(FrameIndex - 1) * __int64(FrameLength), SEEK_SET);

// Read previous frame to initialize the desired frame
if (!ReadFrame())
env->ThrowError("m2AudioAC3Source: error in file \"%s\"", StreamName);
}
}

// Read the frame
if (!ReadFrame())
env->ThrowError("m2AudioAC3Source: error in file \"%s\"", StreamName);

Please check the new NicAudio.dll.
Only ac3 decoder changed, I will see if others decoders need changes.

This is the first release I encode with C++ Express 2010, I hope all is OK.

New release NicAudio.dll v2.0.6 with ac3 and dts decoders changed:

[wonkey_monkey]

Maybe is easy to implement.

Please check the new NicAudio.dll.

That seems to fix the problem too - thanks!

David

[StainlessS]

Looks like tebasuna51 will have sorted you out, but as I had already written below code, I thought
I'de post it anyway, it is roughly what you were wanting ToDo, compiles but untested.


#include <windows.h>
#include "avisynth.h"

class preroll : public GenericVideoFilter {
__int64 A_ms; // Default 20 milli secs worth of audio samples
__int64 prev; // last audio sample previously fetched.
public:
preroll(PClip _child,int _ms, IScriptEnvironment* env);
void __stdcall GetAudio(void* buf, __int64 start, __int64 count, IScriptEnvironment* env);
~preroll(){};
};

preroll::preroll(PClip _child,int _ms, IScriptEnvironment* env) : GenericVideoFilter(_child) {
if(!vi.HasAudio()) env->ThrowError("Preroll: requires Audio");
A_ms = (__int64)(vi.SamplesPerSecond() * (0.001 * _ms) + 0.5); // _ms milli secs worth of audio
prev = -1; // Init to 'never'
}

void __stdcall preroll::GetAudio(void* buf, __int64 start, __int64 count, IScriptEnvironment* env) {
// start and count are in multichannel samples, and 'start' is beginning of requested output samples to store @ buf.
if(count > 0) { // count condition avoids infinite loop and No Store of invalid prev
// If not first frame and audio seek since previous fetch, prefetch audio prior to requested audio
if(start > 0 && prev != start - 1) {
__int64 prefetch = A_ms; // Prefetch (default 20) milli secs worth of audio samples
if(prefetch > start) // Cannot prefetch prior zero
prefetch = start;
__int64 s = start - prefetch;

do { // loop is just in case count < prefetch size, most likely 1 iteration only.
__int64 tcount = prefetch;
if(tcount > count)
tcount = count; // but limit to max count, known buffer size
child->GetAudio(buf, s, tcount, env); // now prefetch audio
s += tcount;
prefetch -= tcount;

} while (prefetch > 0);
}
child->GetAudio(buf, start, count, env); // now get the requested audio
prev = start + count - 1; // store last sample fetched for next time.
}
}

AVSValue __cdecl Create_preroll(AVSValue args, void* user_data, IScriptEnvironment* env) {
int ms = args[1].AsInt(20); // Default 20 ms prefetch
if(ms<1) ms=1; // Minimum 1 milli sec
return new preroll(args[0].AsClip(),ms,env);
}

extern "C" __declspec(dllexport) const char* __stdcall AvisynthPluginInit2(IScriptEnvironment* env) {
env->AddFunction("Preroll", "c[PrefetchMS]i", Create_preroll, 0);
return "`Preroll' Preroll plugin";
}

EDIT: Above Code Edited, removed reliance on Video frame duration, added PrefetchMS arg, default 20(ms).

EDIT:

To do: skip the pre-fetch if that previous frame was the last "current" frame (may not be possible with multiple instances of the filter running)

Data members are local to an individual filter instance (unless defined 'static'), so there are no conflicts with the 'prev' variable.
I would think a single instance immediately after each problematic audio source filter would suffice (on v2.6a3)
EDIT: From memories of when last (and first) I played with audio (Prune), the count number of samples seem to be unrelated to
the approx audio samples for a video frame,
I seem to recall the number 60,000 samples in a typical request but may be wrong on that, perhaps totally down to eg Vdub or Mplayer.
Also, the sample size is dependant on number of channels and type eg int16, int32 float.
Also note, the frames referred to in tebasuna51 code are audio frames not video, audio source filters of course have no idea about video frames.

EDIT: Actually usual 6000 samples request.