Baked code removal - Proposal for AviSynth 2.6

[mg262]

IMO putting actual object instantations in header files is... ick. (Sorry,MfA!) Anyway, if it really bothers you you can create an include file with these two lines...

#include "avisynth.h"
ServerFunctions *serverFunctionsVectors = 0;

...yourself.(You can even call it AVISynth.h if you want, by playing with paths.)

[IanB]

Dear All,

It's been a while (a very long while), so to stop our endless spinning with endlessly trying for perfection, I have just plunged in and bashed out a solution. Wherever the concept came unglued I have just hacked in a workable kludge so we can move forward. So there is some ugliness and kludginess.

I have committed code to CVS, see here :- avisynth.h (http://avisynth2.cvs.sourceforge.net/viewvc/avisynth2/avisynth/src/core/avisynth.h?revision=1.34&view=markup), interface.cpp (http://avisynth2.cvs.sourceforge.net/viewvc/avisynth2/avisynth/src/core/interface.cpp?revision=1.9&view=markup) and plugins.cpp (http://avisynth2.cvs.sourceforge.net/viewvc/avisynth2/avisynth/src/core/plugins.cpp?revision=1.12&view=markup) Also see the end of directshow_source.cpp (http://avisynth2.cvs.sourceforge.net/viewvc/avisynth2/avisynth/src/plugins/DirectShowSource/directshow_source.cpp?revision=1.34&view=markup) for how to use the new linkage interface.

On the user plugin side the entry-point is now called AvisynthPluginInit3 and it take 2 arguments, an IScriptEnvironment* and an AVS_Linkage*. The user need to declare static storage for AVS_Linkage* AVS_linkage; and in their AvisynthPluginInit3 code save the supplied 2nd value to the AVS_linkage variable. The rest of the code should be the same as for 2.5 plugins, i.e. env->AddFunction, etc.

In avisynth.h the type AVS_Linkage struct is declared and macros emit stub baked code that calls the appropriate code in the server via the AVS_Linkage function pointers. The stub code checks to make sure AVS_linkage is not Null and the offset of the function pointer is within the current size of the structure. If okay the function is called otherwise it evaluates as 0. Within the Avisynth.dll project these macros resolve to a simple ";" so no stub code is emitted just declarations occur, this is triggered by the definition of the reserved symbol AVISYNTH_CORE.

In interface.cpp the real code is defined and the instance of the AVS_Linkage struct is declared and initialised. In plugins.cpp the address of this is passed to the users AvisynthPluginInit3 code along with the ScriptEnvironment address. A minor hurdle was that you may not take the address of constructor or destructor routines in C++ so I to move forward I just nested the appropriate code for these down one method and used the address of the nested method. I unimaginatively called these CONSTRUCTORn() and DESTRUCTOR().

In directshow_source.cpp the changes were trivial. Declare the storage for AVS_linkage, rename the .dll entrypoint from AvisynthPluginInit2 to AvisynthPluginInit3 and add the extra AVS_Linkage* argument.

Thoughts and comments please.
//------------------------
// USER_PLUGIN.cpp side
//------------------------
...
/* New 2.6 requirment!!! */
// Declare and initialise server pointers static storage.
AVS_Linkage *AVS_linkage = 0;

/* New 2.6 requirment!!! */
// DLL entry point called from LoadPlugin() to setup a user plugin.
extern "C" __declspec(dllexport) const char* __stdcall AvisynthPluginInit3(IScriptEnvironment* env, AVS_Linkage* vectors) {

/* New 2.6 requirment!!! */
// Save the server pointers.
AVS_linkage = vectors;

// Add the name of our function
env->AddFunction("Plugin", "c", Create_Plugin, 0);

// Return plugin text identifier.
return "Plugin";
}//------------------------
// avisynth.h
//------------------------
...
// Server Function pointers structure
struct AVS_Linkage {
int Size;

bool (VideoInfo::*HasVideo)() const;
bool (VideoInfo::*HasAudio)() const;
...
bool (VideoInfo::*IsSameColorspace)(const VideoInfo& vi) const;
...
int (VideoFrame::*GetPitch)(int plane) const;
...
}
...
// Macros
#ifdef AVISYNTH_CORE
/* Macro resolution for code inside Avisynth.dll */
# define AVS_BakedCode(arg) ;
# define AVS_LinkCall(arg)
#else
/* Macro resolution for code inside user plugin */
extern AVS_Linkage *AVS_linkage;

# define AVS_BakedCode(arg) { arg ; }
# define AVS_LinkCall(arg) !AVS_linkage || offsetof(AVS_Linkage, arg) >= AVS_linkage->Size ? 0 : (this->*(AVS_linkage->arg))
#endif
...
// Baked stub code
struct VideoInfo {
int width, height; // width=0 means no video
...
// useful functions of the above
bool HasVideo() const AVS_BakedCode( return AVS_LinkCall(HasVideo)() )
bool HasAudio() const AVS_BakedCode( return AVS_LinkCall(HasAudio)() )
...
bool IsColorSpace(int c_space) const AVS_BakedCode( return AVS_LinkCall(IsColorSpace)(c_space) )

};
...//------------------------
// interface.cpp
//------------------------

// Real code
bool VideoInfo::HasVideo() const { return (width!=0); }
bool VideoInfo::HasAudio() const { return (audio_samples_per_second!=0); }
...
bool VideoInfo::IsColorSpace(int c_space) const {
return IsPlanar() ? ((pixel_type & CS_PLANAR_MASK) == (c_space & CS_PLANAR_FILTER)) : ((pixel_type & c_space) == c_space);
}
...
// Static linkage structure declaration and initialisation
AVS_Linkage AVS_linkage = { // struct AVS_Linkage {

sizeof(AVS_Linkage), // int Size;

&VideoInfo::HasVideo, // bool (VideoInfo::*HasVideo)() const;
&VideoInfo::HasAudio, // bool (VideoInfo::*HasAudio)() const;
...
&VideoInfo::IsColorSpace, // bool (VideoInfo::*IsColorSpace)(int c_space) const;
...
};
...//------------------------
// plugins.cpp
//------------------------
...
typedef const char* (__stdcall *AvisynthPluginInit3Func)(IScriptEnvironment* env, AVS_Linkage* vectors);
AvisynthPluginInit3Func AvisynthPluginInit3 = (AvisynthPluginInit3Func)GetProcAddress(plugin, "AvisynthPluginInit3");
if (!AvisynthPluginInit3) {
... try other entry point names _AvisynthPluginInit3@8, AvisynthPluginInit2 & _AvisynthPluginInit2@4
} else {
result = AvisynthPluginInit3(env, &AVS_linkage);
}
...

[tebasuna51]

That mean than all plugins must be changed to be 2.6 compliant?

Maybe this is a good moment to add the necesary audio property mask_channels.

[IanB]

No 2.6 can load and use 2.5 plugins within the 2.5 feature set. That is all the current 2.6 alpha can actually do, i.e. there is no 2.6 API yet. (This thread is it)

To maintain compatibility unfortunately visible structure like VideiInfo cannot be changed.

[LaTo]

Great work! It may be time to put avs 2.6.0 in a beta state :)

[-Vit-]

I've posted a collection of new interface plugins (http://forum.doom9.org/showpost.php?p=1568142&postcount=1126) in the QTGMC thread. No problems in early testing (against SEt's 2.6MT build). Distribution is an issue, now have two versions of each plugin. I haven't renamed the new dlls since that might cause hidden problems if the user puts both versions in the plugins folder (?). Still wondering about the best approach though.

[Fizick]

Thanks, IanB!

[pbristow]

Great work! It may be time to put avs 2.6.0 in a beta state :)

As a user rather than a developer, I'm inclined to agree. The baked code removal/"2.6 API" is a significant change that is taking a lot of time and work to get right. Certainly worth doing, but it might be worth officially postponing it to version 2.7 (and making it the *only* change made at that version), to allow the excellent new features and performance improvements that have already gone into 2.6 (and which some of us have been using intermittently for a while now!) to become more widely established. (How practical that is, I'm not clear - i.e. is debugging/finalising the existing features dependent in some way on the baked code removal/interface changes? - but I put the suggestion here for your consideration.)

It will also do the image of AVIsynth as an active project good, I think, to have a new non-alpha version (with an offical build) finally out there!

[Robert Martens]

A quick, potentially dumb question, if I may: should one expect to successfully link to Avisynth 2.6 as a library yet? In preparing a version of TurnsTile (http://forum.doom9.org/showthread.php?t=158695) that works under a variety of Avisynth versions (namely Avxsynth, but that's working quite well), I've been trying to properly set up a test executable that loads Avisynth and runs through some checks. Both implicit and explicit linking work correctly with 2.5.8 (even the 64 bit version, though I won't be driving myself crazy to support it), and I've been able to compile the plugin itself as an actual new interface 2.6 plugin; AVS_linkage, PluginInit3 and all, using it in a script works as intended.

Using revision 1.34 of avisynth.h from the CVS repository, and SEt's May 16th build of 2.6MT, however, I'm getting everyone's favorite error, "unresolved external symbol "struct AVS_Linkage * AVS_linkage"" no matter how I go about linking the thing.

The loading, use and unloading of libraries is still new to me, and I intend to continue my research after posting this message, but it seemed worth checking in to see if what I want is even supposed to be possible yet. Does the API need to be finalized before this works as expected, or is my problem coming from somewhere else (e.g. me, or "post your code, you dolt")?

[StainlessS]

Still in the dreamy stage I suspect. Keep dreamin.

[IanB]

@Robert Martens,

You need to declare the storage for the AVS_linkage pointer and assign the value of the vectors argument to it in your plugin code.

See the green comments and blue code fragments in the USER_PLUGIN.cpp side section of my post above (http://forum.doom9.org/showthread.php?p=1567792#post1567792)


Thoughts on a better way to do this or to document this are very welcome.

[Robert Martens]

I'm sorry, I should have explained myself more clearly. The plugin compiles and operates correctly; I've set up the code as shown in your user_plugin.cpp example, AVS_linkage declared and initialized at file scope, 'vectors' passed in to AvisynthPluginInit3, and its value assigned immediately to AVS_linkage. I #ifdef between the new 2.6 interface and the existing PluginInit2 style based on a CMake-created preprocessor define, but to eliminate that as a potential complication I've tried simply hardcoding the 2.6 version, and both approaches work. The plugin is loaded without issue, and it processes video the way it's meant to.

My problem is getting the error when linking to avisynth.dll as a video processing library from my own executable. When linking to 2.5.8 implicitly (using the .lib and header) or explicitly (using LoadLibrary and the header, since the return value of GetProcAddress must be cast to a pointer to an IScriptEnvironment, a class which is only declared in avisynth.h), I can successfully create a script environment, invoke AVISource to load a clip, query its VideoInfo to get the width and height, and print that information to stdout. Trying to do the same with 2.6 gives me the unresolved external symbol error.

Do I need to do something with AVS_linkage in the executable that loads Avisynth, or is Stainless correct that I'll just have to wait for further developments?

[IanB]

Sorry, I misunderstood your original issue. So yes you have to wait for the next development.

[Robert Martens]

Excellent, thanks for the confirmation!

[IanB]

Dear All,

I have had a chance to review my first efforts, resolve a few issues and add functionality for host applications to establish the AVS_Linkage.

The good news.

I have tagged the main instance of the AVS_Linkage structure static and const and I have created a Dll exported const pointer to it, AVS_linkage. This provides a static means for applications that use Avisynth.dll functions to establish the AVS_Linkage. Also I have extended IScriptEnvironment:: with GetAVSLinkage() as a dynamic method to establish the AVS_Linkage. To use the static linkage method #define AVS_LINKAGE_DLLIMPORT immediately before your #include "avisynth.h". To use the dynamic method declare AVS_linkage as in a plugin and after creating env with the CreateScriptEnvironment() do AVS_linkage = env->GetAVSLinkage().

The bad news.

I discovered that compiler was emitting dll initialisation code to build the static AVS_Linkage structure at runtime instead of allocating it as a static initialised constant structure at compile time. It was also allocating maximum size 16 byte unknown inheritance structures instead of the expected simple 4 byte units for the function pointers.

To fix the 16 byte function pointers I added __single_inheritance to the forward declarations of the appropriate classes. This also mostly fixed the stupid initialisation code emission. Seems the compiler also emits initialisation code for overloaded definitions so I added some uniquely named methods to the AVSValue class to manually resolve the problem.

This of course breaks binary compatibility so any early API adopters will need to recompile to match this version. Hopefully no other nasties appear and break future binary compatibility.

Thoughts and comments please.
//------------------------
// USER_PLUGIN.cpp side
//------------------------
...
/* New 2.6 requirement!!! */
// Declare and initialise server pointers static storage.
const AVS_Linkage *AVS_linkage = 0;

/* New 2.6 requirement!!! */
// DLL entry point called from LoadPlugin() to setup a user plugin.
extern "C" __declspec(dllexport) const char* __stdcall
AvisynthPluginInit3(IScriptEnvironment* env, const AVS_Linkage* const vectors) {

/* New 2.6 requirment!!! */
// Save the server pointers.
AVS_linkage = vectors;

// Add the name of our function
env->AddFunction("Plugin", "c", Create_Plugin, 0);

// Return plugin text identifier.
return "Plugin";
}//------------------------
// avisynth.h
//------------------------

// Forward references
struct __single_inheritance VideoInfo;
class __single_inheritance VideoFrameBuffer;
class __single_inheritance VideoFrame;
class IClip;
class __single_inheritance PClip;
class __single_inheritance PVideoFrame;
class IScriptEnvironment;
class __single_inheritance AVSValue;
...
// Macros
#ifdef AVISYNTH_CORE
/* Macro resolution for code inside Avisynth.dll */
# define AVS_BakedCode(arg) ;
# define AVS_LinkCall(arg)
#else
/* Macro resolution for code inside user plugin */
# ifdef AVS_LINKAGE_DLLIMPORT
extern __declspec(dllimport) const AVS_Linkage* const AVS_linkage;
# else
extern const AVS_Linkage* AVS_linkage;
# endif
...
// IScriptEnvironment
class IScriptEnvironment {
public:
...
virtual const AVS_Linkage* const __stdcall GetAVSLinkage() = 0;

}; // end class IScriptEnvironment...//------------------------
// interface.cpp
//------------------------

// Static linkage structure declaration and initialisation
static const AVS_Linkage avs_linkage = { // struct AVS_Linkage {

sizeof(AVS_Linkage), // int Size;
...
};
extern __declspec(dllexport) const AVS_Linkage* const AVS_linkage = &avs_linkage;
...

[Robert Martens]

I'm sorry to say I can't offer any terribly substantial comments, but after installing my old copy of VC6, wrestling with some configuration and SDK issues, and building the current CVS tree (I'd been using SEt's builds, but he hasn't released since these changes and I don't want to bug him) I can confirm that the linkage by an external application works. The tests I mentioned earlier now run properly when linked against 2.6, thanks for the update!

[SEt]

You sure don't need the ancient and buggy VC6 to build Avisynth. I'm using VS2010 for my latest builds.