Gabriele
13th September 2002, 15:19
Now you would call me crazy, but what if we make a directshow filter of Avisynth?
Because currently there are the following problems:
- Avisynth supports only AVI in both read and write
- AVI is dying because it has too many limitations for the third millennium. Just look at http://mcf.sourceforge.net/ to have an idea
- Many new standards are emerging: among the others Ogg and MCF are very interesting
- There is currently no way to nonlinear-edit OGGs and MCFs
- There is currently no way to nonlinear-edit a directshow audiovideo stream in general
Furthermore a directshow-avisynth could have the following advantages:
- Avisynth could support multichannel audio in that way, quite easily
- Avisynth could support clips with multiple audio and multiple video streams easily (and even text streams like subtitles maybe)
- Avisynth should not bother anymore about (audio) compressed formats because would be handled outside it.
- Avisynth could easily support also those formats which have a decoder only in directshow: In Avisynth currently we have the DirectShowSource(...) to access directshow sources, but with that you can't configure ANY of the directshow filters upstream which do the decoding; so you can't ask for reference IDCT, you can't configure deinterlacing, you can't disable the 3:2 pulldown when present, you can't configure subtitles, you can't ask for a specific audio track of the various which can be avaiable, you can't choose how many audio channels you want in case of various possibilities (sometimes 5.1 is in the stream and some downmixing choices are available in the audio decoder) and so on. All this would be totally solved.
- Avisynth could automatically support new formats when they are released, as soon as somebody makes a directshow splitter and a directshow mux for them (they MUST do that anyway, at least the splitter, or Microsoft Media Player would not be able to play them!)
- There would be much less code because lot of things would be done by other directshow filters (file open, file save, decompression, most code for seeking, multiplexing and demultiplexing, etc.)
-----
Does everybody know about GraphEdit? Search with Google. It's a MS application which allows you to load and connect directshow filters manually. It's VERY powerful for doing transcoding matters, but currently it's quite impossible to do nonlinear things because nobody has ever invented the right filters for doing that. You can download GraphEdit from here to have a look at it:
http://www.doom9.org/software.htm
Some vague info on GraphEdit and links to important directshow filters
must-have-for-doing-processing are in my page here:
http://www.trombettworks.com/directshow.php
-----
This is my idea in the detail for the new Avisynth directshow filter:
You load the filter in Graphedit. The filter starts with a video input pin (videoin1), and an audio input pin (audioin1), and no output pins. If you rightclick and select the properties, an editbox appears (or way to launch Notepad...): you can write the avisynth script in there.
When you connect a video input to videoin1 an additional video input pin is created (videoin2) so that you can add more inputs. Same for audio.
Accepts any YUY2, RGB24 and RGBA32 source for video, any uncompressed PCM WAVEFORMATEX (possibly also uncompressed PCM WAVEFORMATEXTENSIBLE = multichannel) for audio.
(The audio decompression, if needed, is handled upstream by other directshow filters.)
So we allow multiple sources (needed, of course).
From the script, you can access video input 1 from the variable videoin1, audio input 1 from variable audioin1 and so on.
In the script you can define new variables as usual, but the ones ranging from videoout1 to videoout9 and audioout1 to audioout9 are reserved for outputs.
When you use those output variable names in your script and then click OK (in order to close the filter properties) the script gets parsed and new output pins appear on the filter so that you can connect those pins to some downstream filter (e.g. AVI Mux or Ogg Mux, or renderers). The output media type would be defined by the script of course (YUY2, RGB24 or RGBA32).
If there are some errors in the script (e.g. you joined two video streams which don't have the same X-Y size or type [YUY2, RGB24, RGB32] or you joined two audio files which don't have the same samplerate) they are notified at this point, with a messagebox.
Note that joining two video files which don't have the same framerate is acceptable and is not an internal problem anymore, because of the way directshow works. The downstream filters (mainly muxes) can complain about that, though. The MCF Mux will be able to handle variable framerate videos when it will be released (2003 as it seems).
That's all, more or less.
The scripting language would be quite OK like this, the only thing I would add to support this directshow thing are the "group variables", which can be used to perform the same action on many audio and video streams together, useful in case they belong to the same logical clip (which can have many audio and many video streams)
some_group = group(video_angle1,video_angle2,audio_eng,audio_deutsch,audio_ita)
#the group function returns
#a group variable made of those pieces
some_group.Trim(500,24130)
#the trim is performed to all of them together
#the first video stream of the group
#(video_angle1) drives the conversion from
#frames to media times, then an internal
#Trim(mediatime start,mediatime end) is called:
#directshow works better with mediatimes.
... in order to perform the same action on many audio and video streams which you intend to mux together in the downstream filters. But we can bear the absence of group variables, at least at the beginning.
What do you think about it?
Regards
Gabriele Trombetti
Because currently there are the following problems:
- Avisynth supports only AVI in both read and write
- AVI is dying because it has too many limitations for the third millennium. Just look at http://mcf.sourceforge.net/ to have an idea
- Many new standards are emerging: among the others Ogg and MCF are very interesting
- There is currently no way to nonlinear-edit OGGs and MCFs
- There is currently no way to nonlinear-edit a directshow audiovideo stream in general
Furthermore a directshow-avisynth could have the following advantages:
- Avisynth could support multichannel audio in that way, quite easily
- Avisynth could support clips with multiple audio and multiple video streams easily (and even text streams like subtitles maybe)
- Avisynth should not bother anymore about (audio) compressed formats because would be handled outside it.
- Avisynth could easily support also those formats which have a decoder only in directshow: In Avisynth currently we have the DirectShowSource(...) to access directshow sources, but with that you can't configure ANY of the directshow filters upstream which do the decoding; so you can't ask for reference IDCT, you can't configure deinterlacing, you can't disable the 3:2 pulldown when present, you can't configure subtitles, you can't ask for a specific audio track of the various which can be avaiable, you can't choose how many audio channels you want in case of various possibilities (sometimes 5.1 is in the stream and some downmixing choices are available in the audio decoder) and so on. All this would be totally solved.
- Avisynth could automatically support new formats when they are released, as soon as somebody makes a directshow splitter and a directshow mux for them (they MUST do that anyway, at least the splitter, or Microsoft Media Player would not be able to play them!)
- There would be much less code because lot of things would be done by other directshow filters (file open, file save, decompression, most code for seeking, multiplexing and demultiplexing, etc.)
-----
Does everybody know about GraphEdit? Search with Google. It's a MS application which allows you to load and connect directshow filters manually. It's VERY powerful for doing transcoding matters, but currently it's quite impossible to do nonlinear things because nobody has ever invented the right filters for doing that. You can download GraphEdit from here to have a look at it:
http://www.doom9.org/software.htm
Some vague info on GraphEdit and links to important directshow filters
must-have-for-doing-processing are in my page here:
http://www.trombettworks.com/directshow.php
-----
This is my idea in the detail for the new Avisynth directshow filter:
You load the filter in Graphedit. The filter starts with a video input pin (videoin1), and an audio input pin (audioin1), and no output pins. If you rightclick and select the properties, an editbox appears (or way to launch Notepad...): you can write the avisynth script in there.
When you connect a video input to videoin1 an additional video input pin is created (videoin2) so that you can add more inputs. Same for audio.
Accepts any YUY2, RGB24 and RGBA32 source for video, any uncompressed PCM WAVEFORMATEX (possibly also uncompressed PCM WAVEFORMATEXTENSIBLE = multichannel) for audio.
(The audio decompression, if needed, is handled upstream by other directshow filters.)
So we allow multiple sources (needed, of course).
From the script, you can access video input 1 from the variable videoin1, audio input 1 from variable audioin1 and so on.
In the script you can define new variables as usual, but the ones ranging from videoout1 to videoout9 and audioout1 to audioout9 are reserved for outputs.
When you use those output variable names in your script and then click OK (in order to close the filter properties) the script gets parsed and new output pins appear on the filter so that you can connect those pins to some downstream filter (e.g. AVI Mux or Ogg Mux, or renderers). The output media type would be defined by the script of course (YUY2, RGB24 or RGBA32).
If there are some errors in the script (e.g. you joined two video streams which don't have the same X-Y size or type [YUY2, RGB24, RGB32] or you joined two audio files which don't have the same samplerate) they are notified at this point, with a messagebox.
Note that joining two video files which don't have the same framerate is acceptable and is not an internal problem anymore, because of the way directshow works. The downstream filters (mainly muxes) can complain about that, though. The MCF Mux will be able to handle variable framerate videos when it will be released (2003 as it seems).
That's all, more or less.
The scripting language would be quite OK like this, the only thing I would add to support this directshow thing are the "group variables", which can be used to perform the same action on many audio and video streams together, useful in case they belong to the same logical clip (which can have many audio and many video streams)
some_group = group(video_angle1,video_angle2,audio_eng,audio_deutsch,audio_ita)
#the group function returns
#a group variable made of those pieces
some_group.Trim(500,24130)
#the trim is performed to all of them together
#the first video stream of the group
#(video_angle1) drives the conversion from
#frames to media times, then an internal
#Trim(mediatime start,mediatime end) is called:
#directshow works better with mediatimes.
... in order to perform the same action on many audio and video streams which you intend to mux together in the downstream filters. But we can bear the absence of group variables, at least at the beginning.
What do you think about it?
Regards
Gabriele Trombetti