Baked code removal - Proposal for AviSynth 2.6

[IanB]

@Bidoche,

Sorry, I assumed everybody had read the main 2.6 thread goals. The aim is to support version 2.5 pluggins (with current baked code) directly and remove all the baked code from 2.6+ pluggins and have the pluggins source compatible.

If we were starting from scratch then this would be easy and most (all?) of your ideas would be right on the money.

Not sure I get your point here.
If you mean we need a factory method to create an object, I am aware of that.
The solution is just

class AvisynthServer
{
  virtual PVideoInfo CreateVideoInfo() const = 0;
};

The problem is when building separate .dll's designing where the vtable constructor code actually comes from. You have to make sure some other code in the right .dll constructs the object and passes it back. What you have posted is only half the answer, the unspecified half is AvisynthServer objects must always be created inside avisynth.dll(or OtherServer.dll) code.

@MfA,

Patients man, patients. I am still chewing on your ideas. Something about Grandmothers, Duck eggs and sucking upon same.

And to be sure you have not missed a point, If you put non-inline code in avisynth.h then every *.cpp that includes it emits that code. When you try to link you get redefinition errors. The rat hole is packing #if's around the code as a crappy fix, not gonna happen!

[Bidoche]

Originally Posted by Richard Berg

@Bidoche -- how does the plugin interface work in 3.0? I haven't kept up.

What do you want to know exactly ?

So far, I am not too happy with what I came up first and it is bound to change.

[MfA]

Originally Posted by IanB

What you have posted is only half the answer, the unspecified half is AvisynthServer objects must always be created inside avisynth.dll(or OtherServer.dll) code.

Life would be so much easier if this was an lvalue.

[IanB]

Dear All,

With suggestions from MfA, here is another issue. I've gone with the function pointers this time and done away with the need for loading a library or other chunk of code, the plugin author now must call env->InitServerFunction() in their AvisynthPluginInit3 routine. On balance MfA has convinced me that it was arduous messing around with a library, so we only get 99% source compatibility.

I think I still prefer the class full of virtual functions instead of the function pointers but it makes little difference to this, it really only effect the contents of the avisynth.h generator macro, which I have not supplied.

By using a macro to emit the required baked code stubs I sort of half get my wish for no code, what code there is comes from the resolution of a macro. So at least on the surface it looks like there is no code, I can live with that.

More thoughts and comments please.

//------------------------
// USER_PLUGIN.cpp side
//------------------------
...
extern "C" __declspec(dllexport) const char* __stdcall
  AvisynthPluginInit3(IScriptEnvironment* env) {

    // New 2.6 requirment!!!
	// Initialise the server pointers
    env->InitServerFunction();
	
	// Add the name of our function
    env->AddFunction("Plugin", "c", Create_Plugin, 0);
	
    return "Plugin";
}

//------------------------
// avisynth.h
//------------------------
...
// Server Function pointers
struct ServerFunctions {
  ...
  bool (*viHasVideo)() const;
  bool (*viHasAudio)() const;
  ...
  bool (*viIsSameColorspace)(const VideoInfo &vi);
  ...
}
#ifndef __AVISYNTH_CORE__ 
// Global pointer to server function table
ServerFunctions *serverFunctions;

#define code macro plugin mode
...
#else

#define code macro server mode
...
#endif
...
struct VideoInfo {
  int width, height;    // width=0 means no video
  ...
  // useful functions of the above
# In Server case a macro emits this text
=======================================
  bool HasVideo() const;
  bool HasAudio() const;
  ...
  bool IsSameColorspace(const VideoInfo& vi) const;
# In Plugin case a macro emits this text
=======================================
  bool HasVideo() const { return this->(serverFunctions->HasVideo)(); }
  bool HasAudio() const { return this->(serverFunctions->HasAudio)(); }
  ...
  bool IsSameColorspace(const VideoInfo& vi) const {
    return this->(serverFunctions->IsSameColorspace)(const VideoInfo& vi);
  }
...
};
class IScriptEnvironment {
...
  // Note the default argument is the global
  // pointer to the server function table
  virtual void __stdcall InitServerFunction(
       ServerFunctions* &vector=serverFunctions)=0;
...
};
...

//------------------------
// plugins.cpp
  ...
  // 2.5 plugin has baked code
  // doesn't need protection
  env->enable_AddFunction();
  // Init the plugin
  result = *AvisynthPluginInit2(env);
  ...
  // 2.6 plugins have pointers to code
  // Prevent env->AddFunction until
  // server pointer has been loaded.
  env->disable_AddFunction();
  // Init the plugin
  result = *AvisynthPluginInit3(env);
  ...

//------------------------
// avisynth.cpp
void ScriptEnvironment::AddFunction(...) {
  // If they haven't called InitServerFunction()
  // then bark at them
  if (IsAddFunctionsDisabled())
    ThrowError("blah blah 2.6 Plugins must call InitServerFunction()");

  function_table.AddFunction(name, params, apply, user_data);
}
...
ServerFunctions serverFunctionsVectors;
...
  serverFunctions.viHasVideo = &VideoInfo::HasVideo;
  serverFunctions.viHasAudio = &VideoInfo::HasAudio;
  ...
  serverFunctions.viIsSameColorSpace = &VideoInfo::IsSameColorSpace;
...
void ScriptEnvironment::InitServerFunction(ServerFunctions* &vector)
{
  // Give the plugin a pointer
  // to the Server functions
  vector = &serverFunctionsVectors;
  // Pointer has been loaded,
  // allow env->AddFunction
  enable_AddFunction();
}
...

//------------------------
// VideoInfo.cpp

bool VideoInfo::HasVideo() const { return (width!=0); }
bool VideoInfo::HasAudio() const { return (audio_samples_per_second!=0); }
...
bool VideoInfo::IsSameColorspace(const VideoInfo& vi) const {
  if (vi.pixel_type == pixel_type) return TRUE;
  if (IsYV12() && vi.IsYV12()) return TRUE;
  return FALSE;
}

IanB

[IanB]

A small refinement;

If I define the serverFunctionsVectors thus :-

extern "c" __declspec(dllexport) ServerFunctions *serverFunctionsVectors;

I can find and initialize it using GetProcAddress() as I load the plugin. I can then still call AvisynthPluginInit2 and leave it's contents unchanged. I also won't need InitServerFunction() or disable_AddFunction(). However I will need the plugin author to instantiate static storage of serverFunctionsVectors (well I was going to need this anyway) so source will still need a minor tweak.

Any thoughts or comments?

[mg262]

Ian,

I don't think you should worry about creating small amounts of work for plug-in authors. If you are going to recompile anyway, something like this is trivial -- IMO you could get away with requiring bigger changes than this, so long as they were mechanical.

Thank you for all your hard work!

[sh0dan]

Looks great!

[MfA]

Originally Posted by IanB

However I will need the plugin author to instantiate static storage of serverFunctionsVectors (well I was going to need this anyway) so source will still need a minor tweak.

Any thoughts or comments?

Id say put extra code a plugin author has to have in a .icbiinah file which you can simply include (a I Can't Believe It Is Not A Header file).

[foxyshadis]

Call it bakedcode.h (or .cpp)

[IanB]

Possible, but its just 1 more line of boiler plate code a plugin needs.

//------------------------
// USER_PLUGIN.cpp side
//------------------------
#include "avisynth.h"
...
// New 2.6 requirment!!!
// Declare the server pointer
ServerFunctions *serverFunctionsVectors = 0;

extern "C" __declspec(dllexport) const char* __stdcall
  AvisynthPluginInit2(IScriptEnvironment* env) {
  // Add the name of our function
  env->AddFunction("Plugin", "c", Create_Plugin, 0);
  return "Plugin text";
}

[mg262]

IMO putting actual object instantations in header files is... ick. (Sorry,MfA!) Anyway, if it really bothers you you can create an include file with these two lines...

#include "avisynth.h"
ServerFunctions *serverFunctionsVectors = 0;

...yourself.(You can even call it AVISynth.h if you want, by playing with paths.)

[IanB]

Dear All,

It's been a while (a very long while), so to stop our endless spinning with endlessly trying for perfection, I have just plunged in and bashed out a solution. Wherever the concept came unglued I have just hacked in a workable kludge so we can move forward. So there is some ugliness and kludginess.

I have committed code to CVS, see here :- avisynth.h, interface.cpp and plugins.cpp Also see the end of directshow_source.cpp for how to use the new linkage interface.

On the user plugin side the entry-point is now called AvisynthPluginInit3 and it take 2 arguments, an IScriptEnvironment* and an AVS_Linkage*. The user need to declare static storage for AVS_Linkage* AVS_linkage; and in their AvisynthPluginInit3 code save the supplied 2nd value to the AVS_linkage variable. The rest of the code should be the same as for 2.5 plugins, i.e. env->AddFunction, etc.

In avisynth.h the type AVS_Linkage struct is declared and macros emit stub baked code that calls the appropriate code in the server via the AVS_Linkage function pointers. The stub code checks to make sure AVS_linkage is not Null and the offset of the function pointer is within the current size of the structure. If okay the function is called otherwise it evaluates as 0. Within the Avisynth.dll project these macros resolve to a simple ";" so no stub code is emitted just declarations occur, this is triggered by the definition of the reserved symbol AVISYNTH_CORE.

In interface.cpp the real code is defined and the instance of the AVS_Linkage struct is declared and initialised. In plugins.cpp the address of this is passed to the users AvisynthPluginInit3 code along with the ScriptEnvironment address. A minor hurdle was that you may not take the address of constructor or destructor routines in C++ so I to move forward I just nested the appropriate code for these down one method and used the address of the nested method. I unimaginatively called these CONSTRUCTORn() and DESTRUCTOR().

In directshow_source.cpp the changes were trivial. Declare the storage for AVS_linkage, rename the .dll entrypoint from AvisynthPluginInit2 to AvisynthPluginInit3 and add the extra AVS_Linkage* argument.

Thoughts and comments please.

//------------------------
// USER_PLUGIN.cpp side
//------------------------
...
/* New 2.6 requirment!!! */
// Declare and initialise server pointers static storage.
AVS_Linkage *AVS_linkage = 0;

/* New 2.6 requirment!!! */
// DLL entry point called from LoadPlugin() to setup a user plugin.
extern "C" __declspec(dllexport) const char* __stdcall AvisynthPluginInit3(IScriptEnvironment* env, AVS_Linkage* vectors) {

  /* New 2.6 requirment!!! */
  // Save the server pointers.
  AVS_linkage = vectors;

  // Add the name of our function
  env->AddFunction("Plugin", "c", Create_Plugin, 0);

  // Return plugin text identifier.
  return "Plugin";
}

//------------------------
// avisynth.h
//------------------------
...
// Server Function pointers structure
struct AVS_Linkage {
  int Size;

  bool    (VideoInfo::*HasVideo)() const;
  bool    (VideoInfo::*HasAudio)() const;
...
  bool    (VideoInfo::*IsSameColorspace)(const VideoInfo& vi) const;
...
  int      (VideoFrame::*GetPitch)(int plane) const;
...
}
...
// Macros
#ifdef AVISYNTH_CORE
/* Macro resolution for code inside Avisynth.dll */
# define AVS_BakedCode(arg) ;
# define AVS_LinkCall(arg)
#else
/* Macro resolution for code inside user plugin */
extern AVS_Linkage *AVS_linkage;

# define AVS_BakedCode(arg) { arg ; }
# define AVS_LinkCall(arg)  !AVS_linkage || offsetof(AVS_Linkage, arg) >= AVS_linkage->Size ? 0 : (this->*(AVS_linkage->arg))
#endif
...
// Baked stub code
struct VideoInfo {
  int width, height;    // width=0 means no video
  ...
  // useful functions of the above
  bool HasVideo() const AVS_BakedCode( return AVS_LinkCall(HasVideo)() )
  bool HasAudio() const AVS_BakedCode( return AVS_LinkCall(HasAudio)() )
...
  bool IsColorSpace(int c_space) const AVS_BakedCode( return AVS_LinkCall(IsColorSpace)(c_space) )

};
...

//------------------------
// interface.cpp
//------------------------

// Real code
bool VideoInfo::HasVideo() const { return (width!=0); }
bool VideoInfo::HasAudio() const { return (audio_samples_per_second!=0); }
...
bool VideoInfo::IsColorSpace(int c_space) const {
  return IsPlanar() ? ((pixel_type & CS_PLANAR_MASK) == (c_space & CS_PLANAR_FILTER)) : ((pixel_type & c_space) == c_space);
}
...
// Static linkage structure declaration and initialisation
AVS_Linkage AVS_linkage = {                 // struct AVS_Linkage {

  sizeof(AVS_Linkage),                      //   int Size;

  &VideoInfo::HasVideo,                     //   bool    (VideoInfo::*HasVideo)() const;
  &VideoInfo::HasAudio,                     //   bool    (VideoInfo::*HasAudio)() const;
...
  &VideoInfo::IsColorSpace,                 //   bool    (VideoInfo::*IsColorSpace)(int c_space) const;
...
};
...

//------------------------
// plugins.cpp
//------------------------
  ...
      typedef const char* (__stdcall *AvisynthPluginInit3Func)(IScriptEnvironment* env, AVS_Linkage* vectors);
      AvisynthPluginInit3Func AvisynthPluginInit3 = (AvisynthPluginInit3Func)GetProcAddress(plugin, "AvisynthPluginInit3");
      if (!AvisynthPluginInit3) {
  ...   try other entry point names _AvisynthPluginInit3@8, AvisynthPluginInit2 & _AvisynthPluginInit2@4
      } else {
        result = AvisynthPluginInit3(env, &AVS_linkage);
      }
  ...

[tebasuna51]

That mean than all plugins must be changed to be 2.6 compliant?

Maybe this is a good moment to add the necesary audio property mask_channels.

[IanB]

No 2.6 can load and use 2.5 plugins within the 2.5 feature set. That is all the current 2.6 alpha can actually do, i.e. there is no 2.6 API yet. (This thread is it)

To maintain compatibility unfortunately visible structure like VideiInfo cannot be changed.

[LaTo]

Great work! It may be time to put avs 2.6.0 in a beta state

[-Vit-]

I've posted a collection of new interface plugins in the QTGMC thread. No problems in early testing (against SEt's 2.6MT build). Distribution is an issue, now have two versions of each plugin. I haven't renamed the new dlls since that might cause hidden problems if the user puts both versions in the plugins folder (?). Still wondering about the best approach though.

[Fizick]

Thanks, IanB!

[pbristow]

Originally Posted by LaTo

Great work! It may be time to put avs 2.6.0 in a beta state

As a user rather than a developer, I'm inclined to agree. The baked code removal/"2.6 API" is a significant change that is taking a lot of time and work to get right. Certainly worth doing, but it might be worth officially postponing it to version 2.7 (and making it the *only* change made at that version), to allow the excellent new features and performance improvements that have already gone into 2.6 (and which some of us have been using intermittently for a while now!) to become more widely established. (How practical that is, I'm not clear - i.e. is debugging/finalising the existing features dependent in some way on the baked code removal/interface changes? - but I put the suggestion here for your consideration.)

It will also do the image of AVIsynth as an active project good, I think, to have a new non-alpha version (with an offical build) finally out there!

[Robert Martens]

A quick, potentially dumb question, if I may: should one expect to successfully link to Avisynth 2.6 as a library yet? In preparing a version of TurnsTile that works under a variety of Avisynth versions (namely Avxsynth, but that's working quite well), I've been trying to properly set up a test executable that loads Avisynth and runs through some checks. Both implicit and explicit linking work correctly with 2.5.8 (even the 64 bit version, though I won't be driving myself crazy to support it), and I've been able to compile the plugin itself as an actual new interface 2.6 plugin; AVS_linkage, PluginInit3 and all, using it in a script works as intended.

Using revision 1.34 of avisynth.h from the CVS repository, and SEt's May 16th build of 2.6MT, however, I'm getting everyone's favorite error, "unresolved external symbol "struct AVS_Linkage * AVS_linkage"" no matter how I go about linking the thing.

The loading, use and unloading of libraries is still new to me, and I intend to continue my research after posting this message, but it seemed worth checking in to see if what I want is even supposed to be possible yet. Does the API need to be finalized before this works as expected, or is my problem coming from somewhere else (e.g. me, or "post your code, you dolt")?

[StainlessS]

Still in the dreamy stage I suspect. Keep dreamin.