An Embeddable Lightweight XML-RPC Server

While the idea was to abstract away the details of the remote call, the problem was not that simple. For instance, since the original RPC was a binary protocol, what happens when the byte order of the host and remote target differ (Little/Big endian)? Or what if floating-point arguments are passed and the machines provide different representations? The solution to this particular problem was External Data Representation (XDR), which provided a standard for the description and encoding of data to be transferred between differing computer architecture. The result, while functional, became complex and difficult to use.

Sun Microsystems developed the first widely used RPC protocol in the early 1980s. Others, such as CORBA and DCOM, followed.

XML-RPC

XML-RPC is a straightforward, text-based protocol that provides a network-based RPC mechanism using an XML encoding. Function calls and return arguments are wrapped within XML messages that have a specific structure (see http://www.xml-rpc.org/ and http://www.xmlrpc.org/spec). The first implementation of XML-RPC was developed by Dave Winer (and others) who devised an earlier protocol called RPC, from which XML-RPC and SOAP were born. As history (or folklore) goes, Winer, tired of Microsoft's apathy over the protocol development, released the XML-RPC specification to get the ball rolling. Looking at the tools and implementations available for XML-RPC, he succeeded.

Currently, the major push for wire-protocols such as XML-RPC is for the development of web services. XML-RPC permits a client written in any language on any platform to communicate with a server, regardless of its implementation language or architecture. In a world filled with Java, C/C++, Perl, and the variety of client and server-side scripting languages and operating systems, this has tremendous value.

The vision of web services is to collect distributed services together from around the Web to dynamically build a useful application on the fly. XML-RPC provides the glue to bind all of these pieces together and let them communicate.

XML-RPC uses the HyperText Transfer Protocol (HTTP) to relay XML messages between hosts; see Figure 1. After the function name and arguments are encoded within an XML message, this message is transported to the peer as an HTTP body.

Anatomy of XML-RPC Dialogs

To examine XML-RPC messages, I look at a session from its beginning as a client request to the peer and back. For illustration purposes, I use the XML-RPC library for C/C++ by Eric Kidd (http://xmlrpc-c .sourceforge.net/). In this example, you communicate with a remote device to gather device status. The remote device can be queried using a value to define the verbosity of the data you're looking for (0 represents the simplest request). You must also provide a user name and password for authentication.

From the client, you call the remote procedure through the xmlrpc_client_call function. This function takes the function name, arguments, and other parameters, and provides the marshaling of data and transport to the remote peer. Listing One presents the sample call.

The first parameter, env, is of type xmlrpc_env and is used to store information following a call to be parsed by later routines. The next argument is the location of the remote device. An "http://" spec is used since the application layer protocol used for transport is HTTP. Next is the function name you want to call at the remote device (getDeviceStats). The next field ("ssi") is the specification of the arguments that are being passed. This field states that you're sending two strings (denoted by "s") and an integer ("i"). The server implementation also supports Boolean arguments ("b") and doubles ("d"). The final three arguments are the parameters to be passed to the remote function.

Listing Two is the XML-RPC message generated from Listing One. The first part of the message is the HTTP header. XML-RPC messages are provided to the remote HTTP server as POST requests (since we're not actually requesting). The Content-Type of the header also defines the message as type "text/xml" so that it can be properly parsed.

The lightweight XML-RPC implementation presented here provides the response in Listing Three. As in the request, the message starts with an HTTP response header for a message type of text/xml. The lightweight implementation returns only structure data (since this composite type covers the simplest and more complex responses). Each return element in the structure is named, which simplifies the job of parsing the response message.

Back at the peer sample function, you expect to get back an integer (status) that defines the status of our request and also two strings, the last command executed at the device and the current state. This is specified in the XML-RPC client in Listing Four.

You pass the original env variable into the parser, along with the return result that was provided by the client call (in Listing One). You then see another argument spec, but notice that this one differs from your original call. The braces surrounding the arguments define the return as a structure, whereby each element of the structure is named. The remaining arguments are variable string name/value pairs that define which elements of the returned structure to provide back to the caller.

The simple function interfaces provided by the XML-RPC C client API is all that's necessary for remote-procedure-call interaction.

Code Discussion

The lightweight XML-RPC server implementation I present here fits comfortably under 800 lines of C code. The server also provides its own parser and HTTP layer to reduce dependence on larger libraries that would inhibit its use on tiny embedded devices. Figure 2 illustrates the basic architecture for the XML-RPC server.

XML-RPC messages arrive as HTTP POST requests and are parsed in the HTTP interface. You verify that the HTTP message includes an HTTP header and message body and that the body is the length that was defined within the HTTP header via the "Content-Length:" element. You then segregate the XML-encoded message from the HTTP message and pass it to the XML message parser.

Since XML encoding has a uniform structure, one of the simplest ways to parse and validate XML messages is using a predictive parser. Given a tag (or symbol in the case of a parser), you know what should come next based upon the XML-RPC grammar. The value of a predictive parser in this design is that it is conceptually simple and very efficient.

Once the XML message has been parsed, the result is contained within an xmlCall_t structure that contains the method to be called and the arguments that will accompany it. The function callUserFunc takes the xmlCall_t structure and then calls the appropriate user-defined method. The xmlCall_t structure keeps state information about the request and response including any error information.

User functions are provided with an API to extract arguments from the XML-RPC call and to build XML-encoded responses. The XML response generator encapsulates the return structure with XML tags as defined by the method caller. The HTTP header is also applied here for a complete XML-RPC response.

User Interfaces

The lightweight XML-RPC server includes an API to support the addition of user-defined methods as well as for extracting arguments from the XML-RPC request and then building an XML-RPC response. To illustrate, I'll build a function called getDeviceStats that was used in the prior XML message examples. All user methods are passed the xmlCall_t structure and return an integer defining the result of the operation (for error handling). Listing Five shows the declaration of the method prototype and its addition into the XML-RPC server. The purpose of the addMethod function in Listing Five is to create a relationship between the ASCII string function name and the actual function that is to be called when a request for this method arrives.

Listing Six shows the sample method getDeviceStats. Lightweight XML-RPC API functions are shown in bold. All user-defined functions can be defined in three parts: argument capture, processing, and XML response generation.

Within the do loop in getDeviceStats, the arguments are extracted based upon the expectations for the user function. For getDeviceStats, you expect a username string, a password string, and finally a status integer. Four functions exist to extract the basic types supported by the server (integer, boolean, double, and string). If the type that you expect does not appear, you break out of the single-pass loop construct to return an error to the client. The server automatically generates a fault response that will be returned to the client when a user-defined function returns a nonzero result.

The single-pass loop construct provides you with forward goto functionality without the nasty goto. Since the do {} while(); construct performs at least one loop, you force it to a single pass by providing a False condition at the end. Using a break statement then lets you break out of the loop where you can perform subsequent cleanup activities such as error checking.

The second part of the method is processing. Since this is an example function to illustrate the API, no processing is performed and you simply generate some dummy data.

Finally, you generate an XML-RPC response. The function xmlBuildResponse builds the response given the arguments passed along with the arguments specification. The string "{iss}" specifies that you're returning a structure with types integer, string, and string. The curly braces define a structure response (the only composite type supported by the lightweight XML-RPC server). Returning a structure requires not only the arguments to be returned, but also the names for the arguments. As in Listing Six, you pass the string name of the argument and then the actual argument. This particular xmlBuildResponse resulted in the XML-RPC message in Listing Three.

Building an XML-RPC C Client

To validate the lightweight XML-RPC server, I used the XML-RPC C client library. The XML-RPC C client library requires the w3c-libwww package, a general-purpose web API for C (see http://www.w3.org/ Library/). Both of these packages are built using standard configure scripts (followed by a make and make install). The w3c-libwww package must be built first, as the C/C++ XML-RPC library depends upon it.

Building an XML-RPC C client can be complicated due to library and cflag specifications. This process is simplified using the xmlrpc-c-config command provided with the XML-RPC library. The script in Listing Seven shows how the C client is built. CLIENT_CFLAGS and CLIENT_LIBS are both expanded based on the results of the xmlrpc-c-config command. This reduces the effort required for creating build scripts.

Writing a Python Client

The XML-RPC client could also have been written in Python (or any number of other languages; see http://www.pythonware .com/products/xmlrpc/). Listing Eight presents it entirely in Python (http://www .python.org/), an interpreted object-oriented scripting language that is both simple and powerful. Compare, for instance, the size of the Python client—seven lines—with the C client—50 lines. Amazingly, they achieve the same result.

Conclusion

XML-RPC is an interesting protocol for the development of distributed systems, which permits dynamically linking applications developed in a variety of languages on a variety of different processor architectures.

While it's true that XML-RPC messages can be large due to their ASCII encoding, many of the problems associated with traditional RPC (such as byte ordering and floating-point representation) disappear with XML-RPC. XML-RPC is also very easy to debug since all of the messages are immediately readable by the developer.

The Internet pioneer Jon Postel, who developed SMTP, among other things, believed in ASCII-based protocols. SMTP, which is itself an ASCII protocol, is still in wide use today and is a testament to this design paradigm.

DDJ

Listing One

result = xmlrpc_client_call( &env, "http://192.168.2.151/device",
           "getDeviceStats", "(ssi)", "username", "password", 0 );

Back to Article

Listing Two

POST /device HTTP/1.1
Accept: */*
TE: trailers
Host: 192.168.2.151
User-Agent: XML-RPC_Device_Data_Gathering_C_Client/1.0 libwww/5.3.2
Connection: TE,Keep-Alive
Date: Tue, 05 Mar 2002 11:24:53 GMT
Content-Length: 287
Content-Type: text/xml
<
?xml version="1.0" encoding="UTF-8"?>
<methodCall>
<methodName>getDeviceStats</methodName>
<params>
<param><value><string>username</string></value></param>
<param><value><string>password</string></value></param>
<param><value><i4>0</i4></value></param>
</params>
</methodCall>

Back to Article

Listing Three

HTTP/1.1 200 OK
Connection: close
Content-length:  418
Content-Type: text/xml
Server: Lightweight XMLRPC
<
?xml version="1.0"?>
<methodResponse>
  <params><param>
  <value><struct>
  <member>
    <name>status</name>
    <value><int>1</int></value>
  </member>
  <member>
    <name>lastCommand</name>
    <value><string>reboot</string></value>
  </member>
  <member>
    <name>currentState</name>
    <value><string>Normal Operation</string></value>
  </member>
  </struct></value>
  </param></params>
</methodResponse>

Back to Article

Listing Four

xmlrpc_parse_value( &env, result, "{s:i,s:s,s:s,*}",
                         "status", &status, "lastCommand", &lastCommand,
                         "currentState", ¤tState );

Back to Article

Listing Five

extern int addMethod( int (*func)(xmlCall_t *), char *name );
extern int getDeviceStats( xmlCall_t *xmlCall );
ret = addMethod( getDeviceStats, "getDeviceStats" );

Back to Article

Listing Six

int getDeviceStats( xmlCall_t *xmlCall )
{
  char username[80], password[80];
  char lastCommand[80], curState[80];
  int request = 0, status, ret;

  /* Grab the arguments */
  do {
    ret = getStringArg( xmlCall, username );
<pre>    if (ret != XML_NO_ERROR) break;

    ret = getStringArg( xmlCall, password );
    if (ret != XML_NO_ERROR) break;

    ret = getIntegerArg( xmlCall, &request );
    if (ret != XML_NO_ERROR) break;

  } while (0);
  if (ret == XML_NO_ERROR) {

    /* Processing */
    /* Just dummy up some data... */
    status = 1;
    strcpy(lastCommand, "reboot");
    strcpy(curState, "Normal Operation");

    /* Generate XML response */
    ret = xmlBuildResponse( xmlCall, "{iss}", "status", status,
                    "lastCommand", lastCommand, "currentState", curState );
  }
  return ret;
}

Back to Article

Listing Seven

#!/bin/sh
PATH=$PATH:/usr/local/bin
CLIENT_CFLAGS=`xmlrpc-c-config libwww-client --cflags`
CLIENT_LIBS=`xmlrpc-c-config libwww-client --libs`
gcc $CLIENT_CFLAGS -o client client.c $CLIENT_LIBS

Back to Article

Listing Eight

#!/usr/bin/python
import xmlrpclib
server_url = 'http://192.168.2.151/device'
server = xmlrpclib.Server(server_url);

result = server.getDeviceStats( "username", "password", 0 );

print "Status : ", result['status']
print "Last Command was : ", result['lastCommand'];
print "Current State is : ", result['currentState'];

Back to Article