DynObj - C++ Cross platform plugin objects

A C++ class

We have this class definition:

class AnExample {
public:
    AnExmaple( int i ) : m_i(i) { }
    virtual int Add( int i );
    int Sub( int i );
    int Get( ){ return m_i; }
    bool operator < (int i){ return m_i     static int StaticFunc( );
protected:
    int m_i;
};

For linking (which is the key issue in dynamic loading), this applies to the members of AnExample:
  • The constructor AnExample(int i) is inline so it does not generate any linking
  • The virtual function int Add(int i) is an entry in the VTable of the class. To the compiler and linker, it is an index in this table.
  • The function Sub(int i) does generates linking. It refers to a function implemented somewhere else.
  • Get() is an inline function and does not generate linking.
  • operator < is also an inline function (no linking).
  • The StaticFunc() member requires linking. This is true for any static member.
  • The member variable m_i is a type and offset into the binary object. It does not generate linking.
So, it seems it's only static members and functions that are both non-virtual and non-inline member that generate any linking!

VTables

To clarify things here a bit we should remember what virtual functions are:
  • Each class (that has one or more virtual functions) has a unique virtual function table (VTable).
  • The VTable is per type, not per instance.
  • A pointer to the VTable (if the object has one) is always stored first in the object. It is often referred to as the VPTR.
  • A virtual member function is at run-time an index into this VTable where the address of the function to use is stored.
  • A derived class has a copy of the base class VTable first in its own VTable. The new functions it introduces are stored at the end of that copy.
The VTable in itself is in itself a symbol located inside a DLL. However, if an object is instantiated from inside the DLL/SO that owns the VTable, this is not part of the link process either.

So it seems we have found some middle ground.

C++ Inheritance

We have a class that uses inheritance:

class SideBaseClass {
public:
    virtual const char *GetPath( );
    virtual bool SetPath( const char *path );
};

class MultiBaseClass : public AnExample, public SideBaseClass {
public:
    virtual int Add( int i );
    virtual bool SetPath( const char *path );
protected:
    int m_j;
};

MultiBaseClass has two base classes and overrides a function from each base class. From a run-time perspective, the inheritance boils down to the following object layout for an instance of MultiBaseClass:

  word offset 0  VPTR for MultiBaseClass
  word offset 1  m_i (base class data)
  word offset 2  m_j (derived class data)
  word offset 3  VPTR for SideBaseClass

The VTable for MultiBaseClass will be:

  slot 0  MultiBaseClass::Add(int i)
  slot 1  SideBaseClass::GetPath( )
  slot 2  SideBaseClass::SetPath( const char *path)

This object data layout can vary with compiler and data type (compilers have settings for padding and alignment). However, this is constant also between compilers:
  1. The VPTR (if any) is always stored first in the object.
  2. The first virtual function in a base class always occupies slot 0 in the VTable.
  3. Subsequently declared virtual functions stored in order of declaration.
For derived classes, the following applies:
  1. A derived class inherits its default VTable from its first base class that has a VTable (call it main base class).
  2. New virtual functions are appended (in order of declaration) at the end of the default VTable.
  3. For other base classes with VTables (side bases), a full copy of the object is stored at an offset into the binary object, including a copy of side base class VTable.
  4. The VTable of side bases can be modified (functions are overridden) but it cannot be extended.
This is the main picture, there are a couple of exceptions to above rules. They are:
  1. The ordering of overloaded functions in the VTable does not correspond with declaration order for some compilers (MSVC among others, for historical reasons).
  2. Virtual destructors are handled in different ways by different compilers. They cannot be directly invoked across a compiler and DLL/SO boundary.
A central part of the DynObj framework is to account for offsets between multiple bases generated by potentially different compilers.

What is important to recognize here is that: Inheritance (neither single nor multiple) does not generate any linking.

Common ground defined

A class definition:
  • with one or more (non-template, non-virtual) base classes
  • with any number of virtual functions
  • with operators that are either virtual or inline
  • and any inline function
  • with any non-static data members
can be reused and implemented by both a host application and a plugin. Furthermore, they can use eachothers implementations of these classes. Code made by one compiler may use such a class compiled from another.

For function members, this works all the time, also in the cross-compiler case. For data members, it works as long as the compilers on each side have used the same sizes and padding for the data members.

When moving (casting) between base classes, the address offset must at all times be calculated based on offsets generated by the source (plugin) compiler.

This common ground is of course in addition to the old extern "C" MyFunction(...) style. That is however an important part that we will make use of below.

The run-time link

We know now that instances of a class fulfilling above spec can be used by both the host application and the plugin, without requiring a linking process. But how do we handle creation and destruction of object instances?

Since a call across the plugin boundary is essentially typeless, we cannot communicate the type directly to the plugin as a C++ type.

Plugin object creation

Say that from the host, we want the plugin to create an instance of AnExample:

AnExample *pe = new AnExample; // This doesn't work

This would just make the host instantiate it.

Since we don't have the compile time type machinery here, (remember, we're communicating with a binary module possibly from another compiler), we have to communicate the type to the plugin in some other way.

To solve this problem, we use a factory function in the plugin that take the type encoded as a string and an integer:

extern "C" void* CreatePluginObject( const char *type_name, int type_id )
    if( !strcmp(type_name,"AnExample") &&
        type_id == ANEXAMPLE_TYPE_ID )
        return new AnExample;
    else return NULL;
}

Then on the host side we can do:

typedef void* (*PluginFactoryFn)(const char *type, int id);

// First locate CreatePluginObject inside a loaded DLL
PluginFactoryFn create_fn = /* Locate it */;

// Now create object
AnExample *pae = (AnExample*)create_fn( "AnExample",
        ANEXAMPLE_TYPE_ID );


The DynObj framework automates this conversion step so that we can use the expression:

AnExample *pe = do_new<AnExample>; // This works

to instantiate objects from inside plugins. The conversion:
  • C++ type => (type string,type ID)
is taken care of by templates classes available in the host application.

Plugin object destruction

There are some points to consider here to keep cross-compiler compatibility:
  • We cannot be sure that the host and the plugin share the same memory allocator (on Windows this is often not the case). So using C++ delete on plugin objects is not a good idea.
  • Virtual destructors are used in different ways by different compilers.
Essentially, the host must make sure a pluigin object is 'recycled' by the same plugin that created it. To handle this, the DynObj framework has used a solution where each object that is created has a virtual member function doDestroy():

DynObj *pdo = /* Create object and use it */;
pdo->doDestroy(); // End of object

We see here that we have used DynObj as a base class for objects that are created by a plugin.

The DynObj framework works with any classes, but objects that can be created and destroyed by plugins must derive from DynObj.

Linking revisited

The solution with factory functions gives the responsability of setting up the VTable to the plugin, and so, all the functions we need from the plugin are contained in these pre-linked VTables. Each instantiated object comes back with a VPTR as its first binary member.

The only run-time linking we have to do is to lookup these factory functions inside the plugin DLL (and possibly some other init/exit functions).

This keeps the host and the plugin in a loosely coupled relationship, defined by the plugin interface.

Next comes the description of the DynObj solution using this approach.