returning video plus silence from a plugin?

[davidlallen]

Folks,

I have written a simple plugin which is similar to "fast forward"
on a DVD. The video part works fine but the plugin appears to
mess up the audio. The code is a modification of "Trim":

PVideoFrame FastForward::GetFrame(int n, IScriptEnvironment* env) {
// A 10-frame still from the middle of each 300 frames
return child->GetFrame(((n/10) * 300)+myFirstframe+150, env);
}

The effect is to show an occasional still image while zipping
through at 30x.

What I *want* is to have the audio silent during these frames. I
have tried a few things but I am stuck.

a. I can't figure out a way in the plugin API to return silence
of the right length. I tried copying some lines from KillAudio
into my constructor:

vi.audio_samples_per_second=0;
vi.num_audio_samples=0;

But, I get an error during playback: "Splice: one clip has audio
and the other doesn't (not allowed)".

b. The audio for "Trim" works fine; I tried copying the "Trim"
audio section like this.
In the constructor:
vi.num_frames = ((lastframe - myFirstframe) / 300) * 10;
vi.num_audio_samples = vi.AudioSamplesFromFrames(vi.num_frames);
myAudioOffset = vi.AudioSamplesFromFrames(myFirstframe);
With the audio function:
void __stdcall FastForward::GetAudio(void* buf, __int64 start,
__int64 count, IScriptEnvironment* env) {
child->GetAudio(buf, start+myAudioOffset, count, env);
}
However, the sound gets out of sync with the video. It seems
that sound from the *next* clip is played during the FastForward
frames, and the next clip after the FastForward clip has to
run the video too fast to "catch up" with the audio.

How can I get silence, or at least audio of the right length?

Thanks in advance,
- Dave Allen: dave@jendaveallen.com

[sh0dan]

Either do a). This is the correct way to remove audio. The error is due to your script.

c) insert:

void __stdcall FastForward::GetAudio(void* buf, __int64 start, __int64 count, IScriptEnvironment* env) {
memset(buf, 0, BytesFromAudioSamples(count));
}

This will simply make your audio silent, if any is present.

[davidlallen]

Sh0dan,

> memset(buf, 0, BytesFromAudioSamples(count));

Ah. That produces silence for this clip; thanks. But I am still
having trouble with the audio sync in the following clip. After
setting vi.num_frames in my constructor, I also call:

vi.num_audio_samples = vi.AudioSamplesFromFrames(vi.num_frames);

It seems that sound from the *next* clip is played during the
FastForward frames. The next clip after the FastForward clip
either has out-of-sync sound, or the video runs too fast
so it can "catch up" with the audio.

Is there something else I should be doing to resync the audio?

- Dave Allen: dave@jendaveallen.com

[sh0dan]

Try testing in the simplest environment, while testing, and leave out cuts and crops. Otherwise post your script.

Inspect Vdub/File/File information to see information about what your script returns. (Number of samples - length of sound, etc.)

Your code doesn't seem right for what you're trying to do. It is a bit more complex than that.

[davidlallen]

Sh0dan,

> Your code doesn't seem right for what you're trying to do. It is a bit
> more complex than that.

I'm sure that is the problem. But I can't tell what I have left out.
Here is an example script:

LoadPlugin("c:\whatever.dll")
c = DirectShowSource("c:\whatever.mpg")
Trim(c,0,999) + FastForward(c,1000,2999) ++ Trim(c,3000,4000)


Here is the interesting part of the C source. It is mostly based on "Trim".
I assume the Create and class declaration are "not interesting".

FastForward::FastForward(int firstframe, int lastframe, PClip _child,
IScriptEnvironment* env) : GenericVideoFilter(_child) {
// Adjustments to first/last frame copied from Trim
if (lastframe == 0) lastframe = vi.num_frames-1;
myFirstframe = min(max(firstframe, 0), vi.num_frames-1);
if (lastframe < 0)
lastframe = myFirstframe - lastframe - 1;
lastframe = min(max(lastframe, myFirstframe), vi.num_frames-1);
// A 10-frame still from the middle of each 300 frames
vi.num_frames = ((lastframe - myFirstframe) / 300) * 10;
vi.num_audio_samples = vi.AudioSamplesFromFrames(vi.num_frames);
}
PVideoFrame FastForward::GetFrame(int n, IScriptEnvironment* env) {
// A 10-frame still from the middle of each 300 frames
return child->GetFrame(((n/10) * 300)+myFirstframe+150, env);
}
bool FastForward::GetParity(int n) {
return child->GetParity(((n/10) * 300)+myFirstframe+150);
}
void __stdcall FastForward::GetAudio(void* buf, __int64 start,
__int64 count, IScriptEnvironment* env) {
// No audio
memset(buf, 0, (unsigned int) vi.BytesFromAudioSamples(count));
}

When I run the example script, the audio from the 3000,4000 trim starts
playing in the middle of the FastForward video frames. Then when the
video for 3000,4000 starts playing, it goes really fast in order to
catch up to the audio.

If you can easily see what I have left out, I'd appreciate it.
- DaveA

sh0dan: Inserted code tags.

[davidlallen]

Augh! The forum posting page seems to trim all leading spaces.
I had carefully formatted the C code, honest.
- DaveA

[stickboy]

Use the [ code ] ... [ /code ] tags.

[sh0dan]

Are you sure it isn't just because frames are lagging behind in media player? When you skip 300 frames at the time DirectShow has to seek on every frame. Your player might simply be lagging frames.

Try saving out the avi from vdub (maybe compressing the video as xvid, divx or whatever), and watch the result.

[davidlallen]

Sh0dan,

> Are you sure it isn't just because frames are lagging behind in
> media player?

The audio and video get badly out of sync since the next clip's
audio starts in the middle of the FastForward video. I suppose
that could be media player's "fault", if there is nothing
obviously wrong with my C code. But it seems that media player
"should" wait until the next clip starts to start playing the
corresponding audio.

> Try saving out the avi from vdub (maybe compressing the video as
> xvid, divx or whatever), and watch the result.

I haven't had much luck with vdub (see my thread on the vdub forum
hereabouts) but I will give that a try.

- DaveA

[davidlallen]

> > Try saving out the avi from vdub (maybe compressing the video as
> > xvid, divx or whatever), and watch the result.

> I haven't had much luck with vdub (see my thread on the vdub forum
> hereabouts) but I will give that a try.

Well, great. The output avi is fine. However, I get such a crummy
frame rate from vdub that I can't really use it. It seems that
Windows Media Player is pretty lame if it starts playing the audio
for the third clip in the middle of the second clip.

Is there any other way I could go about this which would not result
in such lame WMP behavior (WMP 9, Win98 SE2)?

- DaveA

[sh0dan]

I personally only use vdubmod for testing. Try enabling "DirectDraw Acceleration". This usually increases fps a lot.