Policy-Based Design in the Real World

Jan Reher is employed at a Danish software house that specializes in the defense and healthcare sectors. He can be reached at [email protected].

Have you ever been in a situation where you'd discovered a nice, new solution but you didn't have a problem to apply it to? That's how I felt after having read Andrei Alexandrescu's book Modern C++ Design, which describes a number of amazing stunts you can do with C++, especially with templates. Alexandrescu pleads his case well, but I was nevertheless in doubt: This is all very cool and spiffy, but is it useful? Are these techniques really applicable to real-world programming, or are they just stunts? I put the book back on the shelf and went back to work. A few weeks later, I encountered a problem in the system I work on that looked as if it would fit one of Alexandrescu's techniques nicely. That design problem and the technique I applied to it is the topic of this article.

The technique, or pattern if you like, is called "policy-based design." It is applicable when you need to implement a set of related classes whose functionality varies along a number of independent dimensions. These dimensions are captured in small mix-in classes called "policies," which then are combined in various ways to implement the set of related classes.

Background and Context

The system containing the policy-based design is a military message-handling system (MHS) developed by my company. Among its capabilities is a message interface that allows data in messages stored in the MHS to be exchanged with practically any external system via a programmable command-line-based interface. For example, an incoming message containing the position of enemy forces may be sent through the message interface and used to update a geographic information system. Or, logistics information in a command-and-control system's database may be retrieved through the message interface and used to generate a new message in the MHS, which can then be sent to other systems; see Figure 1.

Now assume that the MHS has received an intelligence message whose body describes the type, nationality, and status of a number of military units using some well-defined format, and whose attachments contain photos of the units. Further assume that ShowOnGIS.exe is a fictional executable that takes as input parameters an XML file containing data about the type, nationality, and status of a number of military units, and a directory containing digital photos of the units, and uses the data to update icons on a map. ShowOnGis.exe then corresponds to the "Other Interface" in Figure 1. The MHS may then execute an Update operation on the message, executing a command line like this:

ShowOnGIS.exe $InXmlBdy $InAttachDir

Before the MHS executes this command line, it substitutes the $InXmlBdy token with the name of a temporary file and substitutes the $InAttachDir token with the name of a temporary directory, then writes the body of the message in XML format to the file and writes the attachments as files in the directory. When the command line is executed, the fictional ShowOnGIS executable updates the map.

Complementary to Update operations, Generate operations extract data from external systems and write the data to a message in the MHS. Here is an example command line:

ExtractFromDB.exe $OutEnv $OutBdy

The MHS substitutes the two $ parameters with names of temporary files. Then, the MHS executes the command line, and the fictional executable then extracts some data from a database and writes a new message envelope and body to the two temporary files. Finally, the MHS reads the files and creates a new message with the provided envelope and body.

Naming conventions state that $In parameters send data from the MHS into the external system, while $Out parameters do the opposite. (If you think it ought to be the other way around, you are not the only one. But there you have it.) Update operations may only use $In parameters, while Generate operations may use both kinds.

Granted, it's a primitive and general interface mechanism that lets two computer systems exchange data. It is primitive because it is based on reading and writing data in plain files, and not some modern, tight integration mechanism such as SOAP, OLE, or whatnot. It is also very general because almost any computer system can read and write data in files. It's not cool or anything, but it has been a part of the MHS for many years, earning its keep and then some in many contexts.

The Design Problem

Recently, we had to extend the message interface to comply with new requirements. This has previously happened several times, and the original design was all but lost under layers of patches. The code had contracted all the usual diseases: It was difficult to read, control flow was tangled, and performance was poor. It was time to clean it up and redesign some parts from scratch. This was where the opportunity for using a policy-based design arose. A central part of the message interface has to perform these steps:

1. Substitute $ parameters with names of temporary files and directories.
2. Fetch message data from a source and write it to $In parameter files and directories.
3. Execute the command line.
4. Read message data from $Out parameter files and directories, and write the data to a sink.
5. Remove temporary files and directories.

For clarity, I've removed almost all error handling from the code presented here. However, the full production code is available at http://www.cuj.com/code/.

The Design Solution: Using Parameter Objects

First, I need to describe the classes that use the classes that express the policy-based design. The MHS executes a command line by instantiating a CommandLineOperation object and calling its Execute() member function; see Figure 2 and Listing 1.

The CommandLineOperation class is responsible for parsing the command line, substituting parameter names with names of temporary files and directories, and executing the command line, thereby invoking the external program. The derived classes contain information about which $ parameters are applicable to Update and Generate operations, respectively.

The CommandLineOperation class depends on three interfaces:

• IDataProvider provides information about the current message.
• IDataConsumer consumes data produced by Generate operations, and writes the data to the current message.
• The IStatusTextConsumer interface consumes status text telling the MHS how the execution of the command line went.

Listing 2 presents these interfaces.

The $ parameters are handled as three lists of objects implementing the IParameter interface:

• inParameters, those that transport message data from the MHS to the external system.
• outParameters, those that transport message data from the external system to the MHS. This list is empty for Update operations.
• statusParameters, there is only one parameter in this list: It transports status information from the external system to the MHS.

Note how the three lists correspond to the three required interfaces: Each incoming $ parameter must be connected to its corresponding function in IDataProvider, each outgoing $ parameter to its corresponding function in IDataConsumer, and the single status parameter to its corresponding function in IStatusTextConsumer. This connection is provided by a large set of classes implementing IParameter; one for each kind of $ parameter. Objects of these classes are thus responsible for either transporting data from an IDataProvider function to a temporary file or directory, or transporting data from a temporary file or directory to a function in IDataConsumer or IStatusTextConsumer. This makes an IParameter object an occurrence of the Pluggable Adapter design pattern: It adapts a function from IDataProvider, IDataConsumer, or IStatusTextConsumer, and makes it possible for the CommandLineOperation class to invoke that function through IParameter::Execute.

A Generate operation is executed like this (you may want to trace the execution in the code listings):

1. A CommandLineGenerate object is constructed from a command-line string and objects implementing the IDataProvider, IDataConsumer, and IStatusTextConsumer interfaces.
2. CommandLineGenerate's constructor builds the three lists of IParameters containing the possible in-, out-, and status parameters for the operation. (If this were an Update operation, the out-list would be empty, thus abstracting away knowledge of whether this is an update or a generate operation. The SetPossibleXXXParameters member functions make the CommandLineGenerate and CommandLineUpdate constructors occurrences of the Template Method design pattern.)
3. For each IParameter object in each of the three parameter lists, CommandLineOperation::Execute invokes a functor that scans the command-line string for the parameter's $ token, and substitutes the parameter name with the name of a temporary file or directory. The IParameter object stores the name of the file or directory for later use.
4. CommandLineOperation::Execute executes the in parameters. An in parameter's Execute member function fetches message data from the IDataProvider, and writes it to its file or directory.
5. CommandLineOperation::Execute executes a system call to execute the command-line string.
6. CommandLineOperation::Execute executes the status- and out-parameters. Similar (but opposite) to the in parameters, their Execute member functions read data from their files or directories and pass them to functions in IStatusTextConsumer and IDataConsumer.

The Design Solution: Implementing Parameter Objects

And finally, the policy-based design. The $ parameters that can be used on a command line vary in two dimensions—the direction that they carry data and the medium that carries the data. Direction can be data going into the external system ($InEnv, for instance), data going out of the external system ($OutAttachDir), status going out of the external system ($Status, for example), or no direction ($Temp). The medium can be either a file (such as $OutEnv) or a directory (like $InAttachDir); see Table 1.

This variation in two dimensions is captured in a policy-based design, as shown in Figure 3 and in parameter.h (available at http://www.cuj.com/code). The template class TParameter acts as host for a Medium policy class and a Direction policy class provided as template parameters. It inherits both of them, thereby giving member functions in TParameter and its derived classes access to functionality and state contained in the policy classes. Furthermore, it implements the interface IParameter. The concrete classes in the design are then derived from instantiations of TParameter, each modeling one $ parameter.

The concrete parameter classes are trivial to design since they are almost exclusively combinations of Medium and Direction. The only contribution from the classes themselves is the name of the token, and the Execute function, which adapts an appropriate function from IDataProvider, IDataConsumer, or IStatusTextConsumer. NoDirection parameters (there is currently only one of those) have empty Execute functions.

For example, consider the $InXmlBdy parameter—-it is file based, carries data into the external system, and its token is $InXmlBdy. Hence:

class InXmlBdyParameter : public TParameter<DataIn, File>
{
public:
  InXmlBdyParameter(IDataProvider& dp) : 
               TParameter<DataIn,File>("$InXmlBdy", dp) {}
  virtual bool Execute() {return provider.ProvideXmlBody(path);}
};

There is a slight complication: The IDataProvider and IDataConsumer interfaces have an asymmetry, which means that the CommandLineOperation class must remove files and directories that carry data from the external system to the MHS ($Out parameters) after use, and that the CommandLineOperation class may not remove files and directories that carry data the other way ($In parameters). The reason for this asymmetry is outside the scope of this article. You may think of it as one of those strange constraints that always pop up and disfigure an otherwise pure and simple piece of design.

This complication means that only $Out parameter classes may delete their medium in their Medium policy destructors. This is why the constructors in TParameter look as they do: The Direction template parameter has a constant member that tells the Medium template parameter's constructor which way data is flowing.

Conclusion

I've provided two versions of the code in its entirety. One is the actual and (nearly) unexpurgated MHS production code. It is a good bit more complicated than that presented in this article, and you won't be able to compile it, since the code depends on header files not included. The other version is a compilable distillate of the production code. Its purpose is to show the policy-based design plus a little context. The code presented in this article is copied from the distillate. Both versions are available at http://www.cuj.com/code/.

My main conclusion is that the policy-based design idea actually works in real life, and that it is not that difficult to use. The MHS is available in UNIX and Windows versions, and none of the compilers we use have had any problems digesting the code. So it is not as if you need a bleeding-edge research compiler to try this yourself. On a more personal note, I remain amazed at the expressiveness and flexibility of C++.

Note that policy-based design is a specialized tool with a rather limited applicability: Its purpose is to structure a set of related classes that vary along a number of independent dimensions, and such design problems do not occur frequently. This is no fault of policy-based design as such: Any design pattern has a limited field of applicability.

Template-based designs are sometime criticized for being difficult to debug. That was not the case for this design. Not that there weren't any bugs in it, but they had nothing to do with the policy-based design. That just worked.

The only drawback I have discovered is that it is difficult to explain a policy-based solution to people not familiar with the concept. This probably applies to many designs exhibiting a high level of abstraction, including most design patterns.

The resulting design is tight, succinct, and easily extendible in several directions. This has convinced me that the rest of the solutions in Alexandrescu's book could also be useful, and I am eagerly looking for new real-world problems on which to try them out.