DynObj - Dynamic C++ Plugin Objects

The Problem

The linking process of a modern C++ application is a complex process, involving name mangling, symbol visibility, namespaces, template instantiation, to name a few issues. And usually the object modules depend on a configuration process and a set of preprocessor defines, making the build process quite specific to time and system.

Attempting to link together object modules generated at different times, or on different systems, or by slightly different compilers can be quite challenging, even if the source code base seems the same. A full re-build is the usual answer.

Extending such applications at run-time at this C++ link level would activate that whole complexity. It would imply sharing full build scripts and a large part of the source code base. It's not practical and hardly doable. The more practical solution has been to use C level linking:

extern "C" a_plugin_function( /*parameters*/ );

and then manually resolve symbols from a loaded shared library.

The DynObj Solution

DynObj allows for handling classes and objects with virtual functions built by other compilers (a different major/minor version, or even a totally different compiler) almost identically to classes and objects generated by the host compiler:

// %% DYNOBJ class(VObj) bases(SignalI,ParamI)
class OurFilterI : public BasicSignal, public BasicParam {
- virtual const char* GetName( ){ return "Our filter"; }
  // GetLength and GetSample implemented by base class BasicSignal
  virtual bool Apply( const SignalI& signal_in, GrowableSignalI& signal_out,
  - DynStr& err_str ){ /* Do filtering */ }
};

To the host program, objects of type SignalI, DynStr, ParamI are handled just the same way, regardless wheter they originate from the host, or from a plugin.

A cross-compiler QueryInterface style casting function:

template<class T, class U >
T* do_cast<T>( U* pu );

takes the same role as ordinary C++ dynamic_cast<T*>(U* pu). The difference is that do_cast works across compiler boundaries and that runtime objects compiled totally separate from the main application can be accessed in a type safe way.

No static linking!

One key point in this scheme is to understand that no static linking takes place between a host and a plugin. And that eliminates the whole complexity discussed above (no name decoration, no visibility issues, no namespaces, ...).

An object with virtual function table is 'self-contained' from the perspective of linking. It has its pointer to the table with functions, set up by the plugin compiler. Selecting an index into this function table is all the linking that needs be done by the host. The virtual functions, and any public data members effectively sets the boundary for interaction.

Responsibilities of the DynObj library

When using an object and querying for interfaces across plugin boundaries, we must be sure to use only type information generated by the plugin compiler. So DynObj must build and maintain a type database and update it on plugin loading / unloading.

We must also verify that the host and the plugin refer to the same interfaces (and their individual functions) when they share an interface. This is accomplished through type ID:s and, when matching, comparing that they agree on the size of the interface. On loading, a check is performed to verify that the two compilers agree on calling conventions, size of primitive types and how to use VTables.

During the plugin initialization process, the DynObj library is passing some parameters to the library and receiving one in return. In this way, a minimal shared state can be established. A plugin may for example return a reference to one static global object that the application can then use as a starting point.

The DynObj library is also responsible for object instantiation and de-allocation. It must be able to translate C++ type information into data that can be communicated to the plugin. And finally, it must translate raw pointers into typed C++ interfaces when returning objects back to the host.

The cost of not relying on static linking is that there is no shared global namespace (between host and plugin). To provide a point of reference between the two, both host and plugin can be built to access a global singleton doRunTime. This object provides a shared namespace (implementing NamedRefI) among other practical functionality. A plugin may register objects with doRunTime and a host may query it for them at a later point.