Filter idea: DirectStreamSource()

[timecop]

Would be nice if someone written something like "DirectStreamSource()" avisynth filter, that would pass a mp3|ac3|dts|aac|whatever stream as-is in the audio stream, but trim it according to the video trims.

It would have to include special cases for all those audio formats to cut on the correct block / packet sizes or whatever.

Is this doable?

is it easy?

Is it fun?

discuss.

[sh0dan]

A new application would be the best way to handle this. It is completely incompatible with the existing AviSynth.

AviSynth could however be used as a base for this.

[timecop]

Originally posted by sh0dan
A new application would be the best way to handle this. It is completely incompatible with the existing AviSynth.

AviSynth could however be used as a base for this.

Why would it be incompatible?
Could this (theoretically) be implemented as a plugin?
Is there any rule that says Audio data in avisynth must be uncompressed PCM?

After all since avisynth outputs video + pcm data as its output, there isnt too much preventing it from saying audio data is compressed using codec x and feeding vfw those compressed frames?

Or am I way off on this one?

[timecop]

bump.
What are the technical reasons why this is "impossible"?
Does avisynth "have" to provide uncompressed PCM as audio?
Are you saying this isn't possible to be done as a plugin because plugins cant influence format of outgoing data, or something like that?
If its part of the core would there be a huge number of hacks required to accomplish this?

Offtopic, is there a frame-accurate AC3 cutting solution that could be used?

[Richard Berg]

Look at all the changes necessary in the other thread; those are just the beginning. Basically, it is impossible* because all of the internal data structures assume that a video buffer can be directly interpreted as having discrete pitch, height, and so on. Similarly for audio.

*without drastically changing the plugin API, that is

[mf]

Seems like something for Bidoche to break his head over in AVISynth 3.0 ? :)

[sh0dan]

@mf: No - Compressed material is NOT AviSynth. If it is done, it should be called something else.

[mf]

Originally posted by sh0dan
@mf: No - Compressed material is NOT AviSynth. If it is done, it should be called something else.
At least it still fits the name, cause it's still a synthesized AVI. I don't see why AVISynth must limit itself to uncompressed audio - a lot of people use AVISynth for video only, and having for instance an AVISource and opening in VirtualDub, you'd need to first open your AVI in Vdub, extract the audio to a WAV or MP3 (Vdub has the weird limitation of not being able to select AVI files for audio streams), and then import that along with your AVISynth script. Also, AVI files can contain subtitles, which, if loaded as a raw stream, could be used by filters like VSFilter() to render them onto the video stream. Again, I don't see why AVISynth is limiting itself on purpose. Basically, a RAW stream type with filter behavior dictated by the source filter would probably do the trick. It's then up to the effect filters (Trim, VSFilter) to actually interpret the stream. In fact, you could probably leave AVISynth thus far responsible ONLY that it allows for the possibility, with none of the source filters or effect filters supporting it. It's then just up to the filter writers to do something with it, and it shouldn't at all be hard to maintain.
Edit: The only difficulty then would be the script returner, which needs to find a way to present the streams to the host app.

[Bidoche]

The problem is not that we can't represent compressed data.
But all the new limitations they enforce on filters.
Trim for example assume that it can cut audio and video at whichever point it wants, not true for compressed data.
You can't handle frames individually either, you need all the frames before you...

Maybe it could be done to add the appropriate checks everywhere it needs...
But for what good ?

[bill_baroud]

why not forbid the use of any filter like that when "direct stream copy" is used ?

hmmm, well no, it wouldn't be that usefull (i suppose many people who want this feature, want to cut out commercials and process video without processing audio)

[Prettz]

There would have to be format-specific handling code for each different audio format that's supported, and new code would have to be written to make new formats supported. Also, as I understand it there's no way to communicate to an application like VirtualDub what audio format it is being passed, and VD would also probably need format-specific code to handle it as well.

[sh0dan]

@Prettz: I think vdub should be able to decode them using acm-codecs - vdubmod should also be able to handle AC3 - but I really don't know. VBR streams is of course not editable, AFAICT.

Have a look at Dsynth (http://forum.doom9.org/showthread.php?s=&threadid=71482). As said, incorporating it into the existing API is not an option.

[esby]

I've looked on the problem...

They are two kinds of problems basically:

* "avs behaviour".

* "data structure problem".


I use "avs behaviour" to refers to problems linked to the 'how avs could handle a filterchain with compressed data in input...)

one working solution could be to test the filterchain for the filter in it,
if any non supporting directStreaming filters is found,
avisource should decompress the video (and go by the existing method).
If all the filters used are compliant with directStreaming (audio, video or both),
it should call the methods to retrieve the compressed data,
pass them unaltered via the filterchain according to the spec needed( only audio compr, only video compr, both comp?),
And where we fall to the other problem "data structure problem":
And i would not dare to answer this yet, but i think there might be a possibility...

For the audio part, i think it might be 'easier', supposed we find a way to nasty job first.

On another note:

Basically there are two 'uses cases':
* Filtering video and serving compressed audio frames...
* Not Filtering video, but assembling parts of videos, splicing some,
in directstream video and we probably won't care about the audio state...

Possibles workarounds:
Second case could be solved by the use of something like Dsynth.
First would mean something like a wavSynth working in direct streaming... (or adding audio support to Dsynth, supposed we can first load an audio from another avi/avs in vdub directly, which is not possible (unless you save the audio first))

PS:
@Bidoche
Trim for example assume that it can cut audio and video at whichever point it wants, not true for compressed data.
You can't handle frames individually either, you need all the frames before you...

That's not really true,
in fact you don't have to handle it in avisynth...
you will just create crap avifile if you don't trim correctly (according to avi structure)...
If you preview the result in vdub, it will throw you a decompressing error... but it will be a normal error...
And not the problem here.

[timecop]

avisynth with direct stream support as a fork or as a core functionality would be absolutely awesome.

<insert a bunch of forum smileys here>