C++ Plugin API

The C++ Plugin API is a pluggable API designed to allow the succinct expression of biometrics algorithms as a series of independent plugins. The API exposes a number of data structures and plugin abstractions to use as simple building blocks for complex algorithms.

Data Structures

The API defines two data structures:

  • Files are typically used to store the path to a file on disk with associated metadata (in the form of key-value pairs). In the example above, we store the rectangles detected by Cascade as metadata which are then used by Draw for visualization.
  • Templates are containers for images and Files. Images in OpenBR are OpenCV Mats and are member variables of Templates. Templates can contain one or more images.

Plugin Abstractions

A plugin in OpenBR is defined as classes which modify, interpret, or assist with the modification of OpenBR compatible images and/or metadata. All OpenBR plugins have a common ancestor, Object, and they all are constructed from string descriptions given by Files. There are eight base abstractions that derive from Object, one of which should be the parent for all production plugins. They are:

Plugin Function
Initializer Initializes shared contexts and variables at the launch of OpenBR. Typically used with third-party plugins.
Transform The most common plugin type in OpenBR. Provides the basis for algorithms by transforming images or metadata.
Distance Used to compute a distance metric between Templates.
Format Used for I/O. Formats handle file types that correspond to single objects (e.g. .jpg, .png, etc.).
Gallery Used for I/O. Galleries handle file types that correspond to many objects (e.g. .csv., .xml, etc.).
Output Used for I/O. Outputs handle the results of Distance comparisons.
Representation Converts images into feature vectors. Slightly different then Transforms in implementation and available API calls.
Classifier Classifies feature vectors as members of specific classes or performs regression on them.

Additionally, there are several child-abstractions for specific use cases. A few examples are:

Plugin Parent Function
UntrainableTransform Transform A Transform that cannot be trained.
MetaTransform Transform A Transform that is not independent.
UntrainableMetaTransform UntrainableTransform A Transform that is not independent and cannot be trained.
MetadataTransform Transform A Transform that operates only on Template metadata.
TimeVaryingTransform Transform A Transform that changes at runtime as a result of the input.
UntrainableDistance Distance A Distance that cannot be trained.
ListDistance Distance A Distance with a list of child distances. The ListDistance is trainable is any of its children are trainable.
MatrixOutput Output A Output that outputs data as a matrix.

As previously mentioned, all plugins in OpenBR are constructed using strings stored as Files. The construction is done at runtime by a Factory class1. The Factory expects strings of the form PluginName(property1=value1,property2=value2...propertyN=valueN) (value lists are also permitted). It looks up the PluginName in a static registry and builds the plugin if found. The registry is populated using a special macro BR_REGISTER; each plugin needs to register itself and its base abstraction in the factory to enable construction. The purpose of this is to allow algorithms to be described completely by strings. For more information on algorithms in OpenBR please see the tutorial