Nick Busigin has a chemical engineering degree from the University of Toronto. He works in the Advanced Engineering Department for Standard Products (Canada) Ltd., currently specializing in control system design and implementation. Nick can be contacted through Compuserve: CIS ID 74075, 1501.
This article describes the QNX operating system v2.15C which runs on the IBM 80x86 class of computers. It is sold by Quantum Software Systems in Kanata Ontario, Canada. This article will focus on the structure of the QNX operating system and the inter-task communication mechanisms available to the QNX programmer. The bibliography at the end of this article provides sources for additional information.
Background Information
QNX is a multitasking, multi-user operating system designed with the real-time software developer in mind. It has a message passing, server-oriented architecture that facilitates distributed processing and provides transparent access to all resources across a network. Most of the QNX operating system was written in C.
I am a chemical engineer and have been using QNX for a little over four years. I am most interested in process control. I have used QNX in SCADA (Supervisory Control And Data Acquisition) and networked embedded control applications.
Structure Of The QNX Operating System
QNX consists of a small kernel and five system tasks that are created when the operating system is booted. The QNX kernel is small — about 10Kb in size. Its primary function is to look after inter-task message passing (hence task synchronization) and clock interrupt handling. The rest of the operating system functions are handled by special administrator tasks.
Task Administrator (task)
This operating system task creates and terminates tasks, allocates memory, and registers task names. It runs at priority level 1. (QNX priorities range from 1 to 15, with 1 being the highest and 15 being the lowest.)
Device Administrator (dev)
This operating system task runs at priority level 2. It looks after all the character devices attached to the system, such as the printer and serial ports, serial terminals, modems, and full screen windows (virtual terminals). The device administrator looks after setting hardware parameters, such as serial port baud rate, parity, stop bits, etc. It also provides character echoing, line editing, line erase, and line recall features. This task also handles requests to open, read, and write to devices.
File System Administrator (fsys)
This task runs at priority level 3. It looks after the QNX file system. This task handles all requests to open, read, and write files.
Network Administrator (net)
The net task looks after communication of messages over the QNX (Arcnet) network. The net task runs at priority 3. It is started when the system is booted but will automatically terminate itself if it is running on a system that does not contain a network card.
Idle Administrator (idle)
The idle task runs at priority level 15, which is the lowest priority available under QNX. Its main purpose is to use up any CPU time that is not used by other tasks.
There are a number of other administrator tasks that come bundled with QNX. Running these tasks is optional, as the operating system is fully functional without them. They provide additional operating system services if there is a need for them. This "mix and match" approach to configuring your operating system environment comes naturally with a server-based architecture. The optional QNX administrator tasks are described below.
Timer Administrator (timer)
This task provides a variety of timing services under QNX. The resolution of the timer service can be set to as small a value as 1 millisecond. The timer administrator provides a mechanism for event timing and wake-up services. These services are useful in writing communication routines and time-critical process control applications.
Spooling Device Administrator (spooldev)
This optional administrator task creates a mechanism for creating spooling devices under the QNX operating system. Everyone is familiar with "physical" devices which are associated with the computer's hardware. For example, a serial port is a physical device. A spooling device is a "logical" device that provides a disk buffer between a program and a physical device. When information is written to a spooling device, it is written to a temporary file. This temporary file is processed by a user specified command when the file is closed. The most common use of the spooldev administrator is to implement print spooling devices.
Queue Administrator (queue)
The queue administrator task provides a non-blocking queued method of passing messages between tasks. A task can open a named message queue that is local to the machine that it is running on, or it can open a named queue that is available across a whole network. The queue administrator can treat the queued messages as a byte stream or as discrete messages.
Locker Administrator (locker)
The locker administrator provides QNX with UNIX System V Interface Definition compatible record locking capabilities. This optional system task also provides file caching options not available with the ANSI style stream I/O functions.
Other Administrator Tasks
Programmers can write additional administrator tasks to extend the services available under the QNX operating system. Administrator tasks have a special status in the QNX operating system. They are immortal, ie., they cannot be killed by other tasks. The only way that an administrator will die is by committing suicide. An administrator task can (optionally) ask to receive notification of the death of every task in the system. All administrator tasks run in the background, at a priority higher than that of the client tasks that will be sending messages to it. Many applications that run under QNX as administrator tasks in order to provide a secure network wide resource. Examples include data base file servers, interfaces to process control systems, electronic mail systems, communication servers that provide gateways to other networks and equipment, etc.
Mountable Drivers And Shared Libraries
Device drivers are not part of the operating system. They must be mounted so the particular devices which they manage are available to the operating system. This is typically done in a sys.init shell script file that serves the same purpose as autoexec.bat and config.sys files under DOS. Shared libraries are usually mounted by commands in the sys.init file as well. Quantum provides the source to most of the available drivers (free of charge) on their 24-hour-a-day BBS.
Inter-Task Communication
If you've only programmed under single tasking operating systems, such as MS-DOS or CP/M, this area may be unfamiliar. When working with a multi-tasking operating system, inter-task communication can become one of the lost important aspects of writing programs. QNX provides a rich variety of methods by which tasks can communicate with each other. These include messages, named queues, ports, exceptions, shared memory, and shared files. Some of these are network wide communication methods while others only provide communication that is local to one machine. We will take a fairly detailed look at each of these methods in the following sections.
Messages
Message passing is the fundamental communication mechanism of the QNX operating system. It is the method by which both the operating system tasks and the user tasks communicate with each other. A message can be sent from any task to any other task in a local area network. It is this message passing mechanism that allows QNX to distribute processing over a network in a transparent fashion. In fact, the message passing mechanism allows a network of computers to appear as one virtual machine (with multiple processors, disk drives, modems, printers, etc.) to both the programmer and the user.
A message is a sequence of bytes that does not have any particular structure as far as the operating system is concerned. The tasks that send messages to each other must impose whatever structure is required by themselves on their messages. Messages can be 0 to 64Kb in length. When a message is sent by one task and received by another, the message data is copied from the address space of the sending task to the address space of the receiving task. The sending task is suspended (blocked) until it receives a reply (to confirm the receipt of the message) from the task to which it sent the message.
There are three primary functions that deal with messages: send, receive, and reply. Their descriptions (summarized from the QNX C compiler manual) are shown below.
int send(tid, msg, reply_msg, msg_size)where
unsigned rid, /* Task identifier of the task to send to. */ msg_size; /* The number of bytes to send. */ char *msg, /* A pointer to the message to send. */ *reply_msg; /* A pointer to the buffer for the reply to the send. */
Send
The message pointed to by msg is sent to the task identified by tid. The sending task will BLOCK (go to sleep and not consume any CPU time) until the receiving task has received the message and sent back a reply. The reply will be placed in the buffer pointed to by reply_buffer.
If the receiving task is RECEIVE-BLOCKED (ie., blocked because it called the receive function when there was no message waiting to be received), the transfer of data into its address space will occur immediately, and the receiver will be unblocked and made ready to run. Otherwise, the sender is placed in a queue (perhaps with other tasks that are SEND-BLOCKED on the receiving task), and the transfer does not occur until the receiving task does a receive that satisfies the send.
Sending to a non-existent task or to one which dies before replying will cause the sending task to unblock and send to return 0.
Return Value
The return value of a successful send is the task id of the task to which you sent the message. A 0 means the target task did not exist or died before replying. A -1 indicates a failure due to an exception.
int receive(tid, msg, msg_size)where...
unsigned tid, /* The task identifier of the task to wait on. */ msg_size; /* The maximum size in bytes of the message to receive. */ char *msg; /* A pointer to the buffer in which to receive the message. */
Receive
The receive() function allows a task to receive a message of a msg_size bytes (or less) from the task identified by tid. The receiving task will RECEIVE BLOCK if no acceptable message has been sent to it.
If tid is specified as 0, (indicates a non-specific receive) then the first task sending to the receiving task will satisfy the receive.
If a task is SEND-BLOCKED on the receiver and satisfies the tid specified, then the receive will not block and the transfer of data will occur immediately.
The task id of the task that sent the message is returned for later use in a reply. An attempt to receive from a non-existent task or one that dies before sending will cause the receiver to unblock and the receive will return 0.
Return Value
Task identifier of the task being replied to if successful or 0 if the reply failed. A -1 indicates a failure due to an exception.
int reply(tid, reply_msg, msg_size) unsigned tid, /* The task identifier of the task to reply to. */ msg_size; /* The number of bytes of the reply message. */ char *reply_msg; /* A pointer to the message to reply with. */
Reply
The message pointed to by reply_msg is sent as a reply to a task which has sent a message and is awaiting a reply. The reply must occur after the receive. The data transfer occurs immediately and the replying task does not block.
Replying to a non-existent task or one that is not awaiting a reply is a null operation.
Reply changes the state of the replied-to task from REPLY-BLOCKED to READY. (A sending task is SEND-BLOCKED until its message is received. After the receipt of its message it is considered REPLY-BLOCKED until it receives a reply.)
Message Passing And Task Synchronization
If many tasks send messages to a receiving task, the sending tasks will block and their messages will be queued up. The receiving task will receive the messages in the time order in which they were sent. This message queuing mechanism provides a controlled sequential access to a resource.
Message Passing And Task Location
To send a message to a task, you must know the task's id number (tid). QNX provides several methods that allow you to determine the task id. When a task spawns a son task, it is returned the task id of the son as part of the call that created the son task. QNX also allows a task to register a name using the name_attach function call. Other tasks can call the name_locate function which locates a task by its registered name and determines its associated task id. Names can be registered locally, globally, and with a distributed name server called clrhouse. As a result of this flexibility, it is not necessary to know the location of a task that you want to communicate with ahead of time. The task of interest may be on the same node or on a remote node somewhere else on the network. This allows for transparent access to all resources across a network and facilitates distributed processing.
Message Passing Speed
The speed of message passing under QNX is impressive: on a 20MHz AST Premium 386C it takes approximately 140 microseconds to pass a 1-byte message and receive a 1-byte reply between two tasks on the same machine, and about a millisecond to do the same between tasks running on different nodes on a QNX network. Larger messages don't take much longer. For a 100-byte message and a 100-byte reply the corresponding times were approximately 150 microseconds and 1.8 milliseconds.
An Example
The following example illustrates the degree of network transparency that is provided by the message passing mechanism to both the programmer and the user.
The program holder.c (Listing 2) provides a simple service: it either stores a character string sent to it or replies with a previously stored string. A complementary program sender.c (Listing 3) either sends a string to holder.c for storage or retrieves a previously stored string. These trivial programs illustrate how simple it is to write code that will work whether the programs are run on a single machine or whether they are run on different nodes of a network. The functions name_attach and name_locate, coupled with the network distributed name server (clrhouse) establish a connection between tasks no matter where they are located. Once this connection is established, the operating system takes care of routing the messages. Refer to sidebar entitled "At The Shell Prompt" on page 116 for additional information on how easy it is to run these programs anywhere on the QNX network.
Named Queues
It is not always convenient (or necessary) to have a task block and wait for a reply to a message that it has sent. Quantum provides an administrator task (queue) that allows you to create elastic message buffers between tasks. The number of message queues that can be opened are limited only by available memory. The amount of data that a given message queue can buffer is limited to about 50Kb, which is plenty for most applications. When a task opens a queue it gives the queue a name. Other tasks locate a queue by this name to write messages to it. The queue administrator looks after storing the messages and replies immediately to the task that wrote a message in order that it does not block. You can limit the number of messages that a queue will buffer. If this limit is exceeded, the task writing to the queue will block on a queue write until a message is removed from the queue. As mentioned previously, a queue can be looked at as containing discrete messages or it can be treated as a byte stream. Queues are a resource that are available either locally or across the entire network.
Ports And Signals
In the QNX operating system, there are hardware I/O ports and software ports. Hardware I/O ports refer to device registers and are referenced by hardware I/O addresses. Software ports are unique to QNX. In the rest of this section the term 'port' will always refer to a software port.
Ports can be viewed as software devices that are visible to all tasks on a given node. They are not visible across the network. Functions involving port usage provide the following services:
- non-blocking communication
- interrupt handling communication
- task identification
- non-blocking semaphores
Ports may be used for simple non-blocking inter-task communication. If a task is attached to a port, another task (perhaps an interrupt handler) may send a signal (not the same as a signal under UNIX) to that port. This causes the port to send a special message to the task that is attached to that port. The message sent to a task by a port as a result of a signal is special for the following reasons:
- Messages sent by ports have a higher priority than regular messages. They are placed at the front of a message queue, bypassing the normal time ordering of messages sent by the send function.
- Messages sent by ports are zero bytes in length and are distinguishable from regular messages.
A signal to a port under QNX is not the same as a signal under UNIX. A signal to a port under the QNX operating system results when a task calls the QNX signal function. The task which is signaled must call the receive or await (on a port) function to be notified of the signal. Therefore, it can be considered synchronous with respect to the receiving task. A signal under the UNIX operating system is entirely asynchronous and is another name for an interrupt. Exceptions (discussed in the next section) under QNX are analogous to UNIX signals. There is no analog in UNIX for a QNX signal. Unfortunately, Quantum decided to use the names signal and port for the above functions instead of choosing new names. This is being rectified in the upcoming POSIX version of QNX.
Ports are a limited resource. There are 28 ports per computer available under the real mode version of QNX and 40 ports per computer under the protected mode version. Of these, the first 16 are reserved by the operating system. Interrupt handlers have the special ability to signal multiple ports with only one signal system call. Regular tasks can only signal one port per system call.
Exceptions
Exceptions provide a means of interrupting a task no matter what it is doing and what state it is in (similar to signals under UNIX). They are usually generated as a result of some unusual or abnormal event. For example, typing ^C will generate a keyboard break exception on a task. Other examples of events that will generate an exception on a task are floating point errors, divide by zero errors and a modem hangup. In all, QNX allows 32 possible exceptions that may be set on a task. Of these 32 exceptions, the first 16 are reserved for use by the operating system.
QNX provides considerable flexibility with regard to generating and handling exceptions. Normally, the operating system terminates a task that has an exception set on it. A programmer may, however, allow a task to handle exceptions with a programmer-defined function. You can also defer exception processing to a more convenient time. Multiple exceptions may be set on a task by another task. Because exceptions are indicated by setting the bits of two 16-bit integers (one word for operating system exceptions and one for programmer defined exceptions), this opens up another means of inter-task communications. Exceptions may be set on tasks that are on another node of a QNX network as well as on local tasks.
Shared Memory
This method of inter-task communication refers to a situation in which two or more tasks have access to the same locations in memory. To achieve this under QNX, one task must allocate a segment of memory and then pass the segment address or selector (when in protected mode) to whatever tasks it wants to share this memory with. Once that is done, multiple tasks can access this memory via pointers and communicate without the overhead of message passing. No problem, right? Not really. This scheme of inter-task communication raises the possibility of lost or corrupted data due to the possiblity of more than one task trying to write to the same area of memory at the same time. Another potential problem with this approach is one task reading a section of shared memory while another task is modifying it. These problems can be overcome, but the mechanism by which this is achieved is up to the programmer. A possible solution is to appoint one task as a memory manager that would control access to the shared memory.
Note that shared memory for inter-task communication is local to a node as it doesn't make sense to pass pointers across a network.
Shared Files
This is the slowest form of inter-task communication because it involves disk file accesses, which are several orders of magnitude slower than memory accesses. The same problems that were mentioned under the shared memory section apply here as well. One possible solution for the problems associated with the contention for records in a file would be to use the UNIX SVID style record locking facilities provided by the locker administrator task. Another approach would be to use a custom data base server task such as the one used with FairCom's c-tree ISAM package. A RAM disk could speed things up considerably but limits the size of the files. This type of inter-task communication is most appropriate for data base type of transactions. It is available across a network since the QNX file system is a network wide resource.
Conclusion
QNX's unique server-based, message passing architecture provides a rich array of inter-task communication facilities. The beauty of many of them is that they are available across the network in a transparent fashion. These features, coupled with the small size and speed of the QNX operating system, make it ideal for networked embedded control applications. The richness of the QNX operating system utilities and the growing pool of software (both commercial and public domain) make it an attractive choice for general computing as well.
Additional QNX Information and References
"Message-Passing Operating Systems," Dan Hildebrand, Dr. Dobbs Journal of Software Tools, June 1988.
Dan Hildebrand is a programmer at Quantum. This article provides an overview of the QNX operating system.
"Spreading Out — Distributed Applications on a LAN Are No Longer a Pipedream," Dan Hildebrand, LAN Magazine, October 1988.
In this article, Dan demonstrates how easy it is to develop distributed parallel processing applications under QNX. A fractal generating program that distributes the processing of screen slices over a network is used as an example.
"QNX — A Real-Time Multi-Everything OS That Runs on LANs," David W. White, LAN Magazine, January 1989.
A description of David White's "test drive" of the QNX operating system.
"Inter-task Messaging under QNX," William Boyle, The C Users Journal, November 1988.
W. Boyle is a software engineer at FASTech Integration, Inc. and is involved in systems integration for the industrial automation market. His article describes the implementation of a utility (source code provided) which allows viewing a remote node's display console.
"Real-Time Messaging: A Quantum Approach to Scheduling," Embedded Systems Programming, May 1989.
I haven't read this one, so I can't comment on it.
"The QNX Operating System: A Multi-tasking, Networking Alternative To UNIX," Marta Greenberg, MIPS, January 1990.
Marta Greenberg is a vice president of Software Innovations Inc., a UNIX consulting firm. She has written a short review of QNX, which highlights its design and compares it to UNIX using a number of benchmark programs. The article is targeted at programmers familiar with UNIX.
"Off The Shelf: QNX — A UNIX-like Lan Environment," Jon Johnston, UNIX Review.
Jon Johnston is an independent consultant based in the Twin Cities of Minnesota. He specializes in microcomputer LANs. This article provides a brief overview of the QNX operating system.
"The QNX Operating System — Programming with Messages in a Distributed Environment," Frank Kolnick, Basis Computer Systems Inc., 1989.
This is an excellent book for C programmers who want to take advantage of the unique strengths of the QNX operating system. I wish I had this book when I first started programming in the QNX environment.
"A Technical Overview of the QNX Operating System"
"QNX Reference Guide Version 2.1," Quantum Software Systems Ltd., 1988.
"QNX C-Compiler And Assembler Manuals," Quantum Software Systems Ltd.
QUICS, Quantum's Update and Interactive Conferencing System
This is an on-line update and conferencing system similar to BIX that operates 24 hours per day. You can download the latest version of the software which you have purchased from QNX for free. This service also provides a large treasure trove of public domain software and source code to many of the QNX drivers, utilities and C library functions. It allows you to send electronic mail to Quantum staff. A variety of topics are discussed in the forums. Questions are typically answered within one day and very likely by the author of the software package you need help with.
Quantum Software Systems, Ltd.
175 Terrence Matthews Crescent
Kanata K2M 1W8 Ontario
Canada
(613) 591-0931
Listing 1
Sidebar: Operating Systems
Sidebar: Real Mode vs. Protected Mode Operation
Sidebar: Update: Quantum Releases A Posix Compliant Verion of QNX
