Create DLL with Code::Blocks

[jmac698]

This sounds like a silly problem, but I just can't get it to work. I tried compiled a plugin in Code::Blocks, and it compiled with no errors, however no dll was created. Instead, some libName.dll.a and Name.def were made.
I tried that .a file but it won't load as a plugin.
How do I make the real dll?

[TheFluff]

If you want to compile native Avisynth plugins (as opposed to stdcall plugins, aka. c-plugins) you can't use Code::Blocks (since it uses g++; native Avisynth plugins must be built with MSVC since C++ is retarded).

I have never used Code::Blocks myself, but .a is the conventional Linux file name ending for static libraries.

[jmac698]

Hi,
I am compiling a C plugin. Code Blocks also supports many compilers, including the MSVC compiler. Btw, do you know exactly what is different with the compilers in C++? I mean why they can't be compatible... wouldn't it be theoretically possible to write a binary converter of some sort?
What a stupid standard if C++ can't be compatible.

[Gavino]

do you know exactly what is different with the compilers in C++? I mean why they can't be compatible... wouldn't it be theoretically possible to write a binary converter of some sort?
They are not compatible at the binary level due to name mangling.

From http://en.wikipedia.org/wiki/Name_mangling:
C++ compilers are the most widespread and yet least standard users of name mangling.
...
The C++ language does not define a standard decoration scheme, so each compiler uses its own. C++ also has complex language features, such as classes, templates, namespaces, and operator overloading, that alter the meaning of specific symbols based on context or usage. Meta-data about these features can be disambiguated by mangling (decorating) the name of a symbol. Because the name-mangling systems for such features are not standardized across compilers, few linkers can link object code that was produced by different compilers.

BTW I successfully use Code:Blocks with the (free) Visual C++ 2003 compiler to build native Avisynth plugins.

[Groucho2004]

native Avisynth plugins must be built with MSVC since C++ is retarded

Don't blame the language. C++ can be quite handy, it all depends on how you use it.

Of course, if you write something that largely depends on M$ libraries, conventions, etc., portability goes out the window.

[jmac698]

Looks like it is possible to use g++ for native Avisynth plugins. There's two ways to do this, one specifies a conversion between the name mangling, and another method which is easier.
http://www.codeguru.com/forum/archive/index.php/t-343634.html

Still didn't help me make my dll though :)

Gavino,
Certainly you could try compiling http://avisynth.org/warpenterprises/files/demosaic_20071206.zip with your setup and report the project file settings? I can't write a guide nor make any plugins until I can compile :(

[Groucho2004]

Gavino,
Certainly you could try compiling http://avisynth.org/warpenterprises/files/demosaic_20071206.zip with your setup and report the project file settings? I can't write a guide nor make any plugins until I can compile :(

Why don't you just download VC2003 express and try it yourself?

[jmac698]

That's not the problem, even if I used vc2003 compiler with Code::Blocks I'd have the same problem. I've also avoided vc for two reasons; it's over 1GB of download to write avisynth plugins, and I don't know C++.
Also, learning VC doesn't help me in the long run, I compile a lot of cross-platform code; see my compiling FFMBC guide
http://forum.doom9.org/showthread.php?t=162615
This is already a big jump for me; I started with Bloodshed Dev-C which is only 8mb and worked fine for me, however it's not updated anymore.

[Groucho2004]

I've also avoided vc for two reasons; it's over 1GB of download to write avisynth plugins
VC2003 Express is a 30MB download. See here (M$ have taken it off their site):
http://xona.com/2004/06/29.html

Edit: OK you'll also need the platform SDK but that's also not that big.

and I don't know C++.
No idea what this has to do with compiling a C plugin.

[Gavino]

Gavino,
Certainly you could try compiling http://avisynth.org/warpenterprises/files/demosaic_20071206.zip with your setup and report the project file settings?
Before I go any further, are you using the makefile from that zip?
It shows the linker command to create the dll should look like (start with):
g++ -shared -o demosaic.dll ...
You should see this in the build log when you try to build.

[jmac698]

Yes I believe I was able to use the makefile in the past (though I had quite some difficulty getting it to work), however I want to find a solution that's automated, ie doesn't need to customize a makefile.
If you open the cbp project file, it's xml and fairly clear what it's doing. I have a working and non-working cbp side-by-side and I just don't see why one works.
One difference I spotted is

<Compiler>
<Add option="-DBUILDING_DLL=1" />

They both also have

<Option createStaticLib="1" />

Which corresponds to the GUI checkmark box, in project properties I believe.

The project which works is the imported .dev file from my former setup of Bloodshed Dev-C

[jmac698]

My build log:


-------------- Build: Release in Demosaic ---------------

Linking dynamic library: bin\Release\Demosaic.dll
Creating library file: bin\Release\libDemosaic.dll.a
Process terminated with status 0 (0 minutes, 0 seconds)
0 errors, 0 warnings

My output directory:

05/11/2011 12:13 AM 790 libDemosaic.dll.a
05/11/2011 12:13 AM 0 libDemosaic.dll.def

[Gavino]

When you create your project, just start from template "Dynamic Link Library" and it should go through the wizard to set it up correctly for your compiler. I'm not sure what you want me to do - how are my settings going to help when I am using a different compiler to you?

[jmac698]

I tried that as well. It creates some sample files for DLL creation which I removed. I then added the project files. Same problem :(
I'm not sure the compiler matters, I'm hoping it's just the project settings.
Simply, are you able to build that source with your setup, and post your .cbp file would be fine.
Thanks for your efforts to help :)

[Gavino]

Simply, are you able to build that source with your setup, and post your .cbp file would be fine.
Builds OK with my compiler.
Here is my project file (note I am probably using an older version of Code::Blocks).
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<CodeBlocks_project_file>
<FileVersion major="1" minor="6" />
<Project>
<Option title="Demosaic" />
<Option pch_mode="2" />
<Option compiler="msvctk" />
<Build>
<Target title="Release">
<Option output="..\..\bin\Demosaic" prefix_auto="1" extension_auto="1" />
<Option object_output="..\..\obj\Test\" />
<Option type="3" />
<Option compiler="msvctk" />
<Option createDefFile="1" />
<Option createStaticLib="1" />
<Compiler>
<Add option="/MD" />
<Add option="/Ox" />
<Add option="/Og" />
<Add option="/W3" />
<Add option="/DBUILD_DLL" />
<Add option="/DNDEBUG" />
<Add directory="C:\Program Files\AviSynth 2.5\FilterSDK\include" />
</Compiler>
<Linker>
<Add library="avisynth.lib" />
<Add directory="C:\Program Files\AviSynth 2.5\Extras" />
</Linker>
</Target>
</Build>
<Unit filename="demosaic.cpp" />
<Extensions>
<code_completion />
<envvars />
<debugger />
<lib_finder disable_auto="1" />
</Extensions>
</Project>
</CodeBlocks_project_file>
I thing Option type="3" is what corresponds to creating a DLL.
Note also the correct settings for compiler include directory and linker libraries.

[TheFluff]

Looks like it is possible to use g++ for native Avisynth plugins. There's two ways to do this, one specifies a conversion between the name mangling, and another method which is easier.
http://www.codeguru.com/forum/archive/index.php/t-343634.html

This really isn't a good idea, since Avisynth can and will throw C++ exceptions from its DLL, which can lead to blatant insanity even if you use MSVC (throwing C++ exceptions from a DLL like Avisynth does is just a Bad Idea in general, but it gets even more fun if you used different compilers for the DLL that throws the exception and the DLL that is supposed to catch it). Remember that g++ does not support SEH or its relatives.

I'm also not sure if avisynth.h is even compileable with g++, since it unconditionally does #include <objbase.h> as one of the first things it does. COM stuff isn't exactly known for its portability.

[LoRd_MuldeR]

There is some preliminary support for SEH in MinGW. See the macros provided in "excpt.h".

(No idea how well that works in practice)

[jmac698]

Ya, there was talk of older versions not handling exceptions but I think it's done now.
Thank Gavino, I think I found my problem. First of all, I was wondering where that .lib was officially stored. Next I think there was a problem with spaces in filenames, which is odd, but only relative paths could work.
So I got it to work, yay...
Hmm, I had some environment variables set for another mingw installation, could that be a conflict?

As for MSVC, Avery Lee also goes on about how bad a compiler it is...
Visual Studio debugger occasionally locks up the entire Windows GUI
http://www.virtualdub.org/blog/pivot/entry.php?id=118

Can I have my VC6 project build back, please?
http://www.virtualdub.org/blog/pivot/entry.php?id=111
etc.

[TheFluff]

As for MSVC, Avery Lee also goes on about how bad a compiler it is...
Visual Studio debugger occasionally locks up the entire Windows GUI
http://www.virtualdub.org/blog/pivot/entry.php?id=118

Can I have my VC6 project build back, please?
http://www.virtualdub.org/blog/pivot/entry.php?id=111
etc.

I don't think you quite understand what you've been reading, and I'm not sure if you have understood the difference between a compiler and an IDE either. Both of those entries are from 2006 and are concern Visual Studio 2005 running under Windows XP. Neither of them are posts about the actual compiler; one of them is about the IDE, while the other is about quirks in the Visual Studio build system (which has been completely reworked in VS2010, I might add).

There are certainly things you can complain about regarding MSVC's code generation, but I don't think it's any significantly better or worse than g++ in general. g++ has the annoying property that if you always stick to the latest version you're quite likely to run into random miscompilation problems that can cause all sorts of bizarre and odd behaviours.

Also, using MSVC (the compiler) does not necessarily imply you have to use Visual Studio (the IDE) as well. Usually it's a good idea to do so (VS2010 is probably the least bad C++ IDE; there are no good ones), but you don't have to.

Finally, I should mention that there really is no reason to stick to ancient shit like MSVC 2003 either. Visual Studio 2010 Express (http://www.microsoft.com/visualstudio/en-us/products/2010-editions/visual-cpp-express) is a free download which gives you both the compiler and the IDE; the only possibly relevant thing it lacks is the ATL libraries, but those are pretty easy to find elsewhere, and you don't need them for most Avisynth stuff anyway.

[Gavino]

So I got it to work, yay...
One thing I noticed with that Demosaic source is that, although using the Avisinth C interface, the source file is actually C++.

When I renamed it to .c instead of .cpp, it failed to compile, as it is using some syntax extensions from C++.
Perhaps for your tutorial, you should ensure that your example is pure C.

[jmac698]

Interesting, thanks for the info! Yes I know the difference, as I pointed out to you earlier, Code::Blocks allows you to use it's IDE with the MSVC compiler. But I was responding to someone suggesting that I download 2005 express. (and platform SDK, and Masm, and a bunch of manual configuration...).

I have also been reading many entries where he complains about the intrinsics creating bad code, and a bunch of other performance related issues.

Anyhow, at this point I'm not doing heavy coding, though as I said before I also compile other projects which require mingw. However, if I do use the compiler only, I could do some profiling and at a simple level just choose the faster executable.

I do have a question though; I believe I read somewhere that MSVC doesn't include various codepaths for each architecture, so if you optimize for SSE2 for example it will just crash on a lesser machine, where for example Intel can include multiple paths automatically. Could you explain the situation?

[Groucho2004]

I have also been reading many entries where he complains about the intrinsics creating bad code, and a bunch of other performance related issues.
Sounds interesting. Do you have a link?

[TheFluff]

I do have a question though; I believe I read somewhere that MSVC doesn't include various codepaths for each architecture, so if you optimize for SSE2 for example it will just crash on a lesser machine, where for example Intel can include multiple paths automatically. Could you explain the situation?

I have no idea what Intel's compiler does but yes, if you tell MSVC to use SSE or SSE2 then the resulting code will not run on CPU's without those instruction sets (not that there are very many such CPU's anymore). This really doesn't matter all that much, however, since all known compilers are pretty bad at autovectorization and enabling code generation for those instruction sets generally does not speed up the produced code by all that much. Most SSE/SSE2 code that actually speeds up Avisynth plugins to a significant degree is hand-written with manual runtime CPU detection and an alternative C++ codepath.

Sounds interesting. Do you have a link?

Just go to virtualdub.org and enter "intrinsics" in the search field and you'll get a long list of articles on the subject. Of course, it's all quite academic, because using compiler intrinsics at all is a pretty bad idea. If you want handwritten SIMD code, write handwritten SIMD code, because then you'll actually get what you asked for and not something else.

[LoRd_MuldeR]

I have no idea what Intel's compiler does but yes, if you tell MSVC to use SSE or SSE2 then the resulting code will not run on CPU's without those instruction sets (not that there are very many such CPU's anymore). This really doesn't matter all that much, however, since all known compilers are pretty bad at autovectorization and enabling code generation for those instruction sets generally does not speed up the produced code by all that much. Most SSE/SSE2 code that actually speeds up Avisynth plugins to a significant degree is hand-written with manual runtime CPU detection and an alternative C++ codepath.

With the Intel C Compiler you can use the /arch switch to define the desired instruction set. The default (aka "not set") is /arch:SSE2, so you have to explicitly set /arch:IA32 in order to make sure it will run on all CPU's. Also the Intel C Compiler supports runtime CPU detection via /Qax switch. For example the combination of /arch:IA32 and /QaxSSE2 will generate a SSE/SSE2 optimized code path in addition to the IA32 code path. At runtime it will pick whatever is supported by the CPU - at least that's the theory. There are complaints it doesn't always pick the "best" path on AMD CPU's. Anyway, the /Qax switch is still supposed to work with Non-Intel CPU's. At the same time the /Qx switch, like /QxSSE2, enables even more aggressive optimizations - and adds code that will abort the program with error message on AMD CPU's...

[kemuri-_9]

I have no idea what Intel's compiler does but yes, if you tell MSVC to use SSE or SSE2 then the resulting code will not run on CPU's without those instruction sets (not that there are very many such CPU's anymore).

Intel's compiler does runtime detection of the instruction set supported by the CPU. It uses this for
A) using instructions supported by the CPU that are beyond the minimum instruction set that was specified where applicable in the standard codepaths. It does this even without the specification of the options mentioned in B).
B) uses this for the codepath(s) it made while optimizing for the instruction set(s) it was told to optimize for specifically. Intel's compiler has options to support a minimum set, such as SSE2, but then optimize for other later sets such as SSSE3 & SSE4.2, though this functionality tends to explode when used in combination with profiling...

It should also be noted that the runtime detection by the Intel Compiler is the infamous 'non Intel crippler' as the runtime detection only optimizes for Intel processors, that is unless you override the detection function to be fair like what x264 does (http://git.videolan.org/?p=x264.git;a=blob;f=common/osdep.c;h=75936151c28a4694415cab844d31b634fc4e48e3;hb=HEAD#l93).

[jmac698]

I found these good reading
A short article about VC intrinsics
http://www.virtualdub.org/blog/pivot/entry.php?id=310

Optimizing a filter to run 3x faster
http://www.virtualdub.org/blog/pivot/entry.php?id=307


If you want to get into optimizing, you really need to know each CPU architecture.

The comments also said that GCC and Intel were both better at optimizing than VC2010.

Interesting discussion. So does GCC do this runtime detection as well?

I'll be doing some benchmarks soon on my own to compare compilers. I've done assembly on different CPU's for years, just not x86, and I don't know the architecture. I could get as far as understanding the instructions and puttting them to use, which I'm sure will be a speedup, but I wouldn't be aware of stalls and so on.

Intel compiler slows down AMD accusations:
http://www.agner.org/optimize/blog/read.php?i=49

Awesome resource on hardcore optimizing, also seems to include a utility to convert object files between platforms - just as I imagined, there must be a way to convert calling convertions and mangling.
http://www.agner.org/optimize/#manual_cpp

[Gavino]

also seems to include a utility to convert object files between platforms - just as I imagined, there must be a way to convert calling convertions and mangling.
http://www.agner.org/optimize/#manual_cpp
The problem is not as simple as you imagine.

The object file converter is principally for converting between different object file formats, not for resolving binary incompatibilities between the outputs of different C++ compilers. It does not automatically convert name mangling, you have to explicitly say what names you want converted, and to what. Moreover, it will not deal with incompatiblities of constructors/destructors, exception handling, etc, etc...

[TheRyuu]

Interesting discussion. So does GCC do this runtime detection as well?

No but I believe that it is planned and will use function pointers (unlike intel's which does a check every function call). /arch:SSE2 is fairly good about being cpu neutral with Intel's compiler although I believe there are still a few things which it may penalize non-Intel processors. As already said there is a dead simple workaround to get around this behavior.

That being said I don't see how it really matters at this point considering that gcc isn't exactly known for autovectorization so I haven't a clue how it would be useful. Using SSE/SSE2 is probably most beneficial to floating point math I would guess.

Just be careful with code size as well. Code locality and cache misses which result from O3 can be pretty bad with the compilers (at least with gcc) sometimes. O2 will most likely be faster the majority of the time unless you're using fprofile. LTO and some optimization changes (inlining) with gcc 4.7 might help keep code size down in that regard, at least a little I guess.