The evolution of database management systems has been driven by the goal of eliminating latency and increasing the prioritization of data management tasks. New physical database structures—ISAM, VSAM, clustered indexes, and column-based databases—are responses to the need for faster data retrieval. Innovations in storage modality have also played a role: Caching was introduced as a means to keep selected data in memory for faster access, while in-memory database systems (IMDS) offered the ability to store entire databases in main memory, eliminating the overhead and unpredictability of disk I/O and caching logic.
More than any other software component, operating system kernels embody the kind of high-priority, zero-latency responsiveness sought by database system developers. Typically viewed as the lowest-level software abstraction layer, the kernel is responsible for resource allocation, scheduling, low-level hardware interfaces, network, security, and other integral tasks. Security applications (access-control systems, firewalls, and the like) and operating-system monitors commonly place their functions in the operating-system kernel and have a need for local, high-performance data sorting, storage, and retrieval. In typical access-control application scenarios, for instance, the data structures and queries are inherently complex, yet the lookups and updates must be nearly instantaneous.
However, the kernel has generally been off-limits for database management systems because DBMS overhead (file and disk I/O, locking, cache management, and related logic) could overwhelm kernel resources and disrupt OS-critical tasks. Kernel module developers have been left to either reinvent the wheel of fundamental database capabilities for use in the kernel (albeit in a limited, lightweight fashion), or deploy a complete DBMS in user-mode space and rely on expensive (in performance terms) context switches whenever kernel-mode processes require data lookup.
The advent of ultra-small footprint and resource-conserving embedded in-memory databases presents an efficient alternative, making it possible to integrate database engines with the operating-system kernel. McObject (the company we work for) has developed the eXtremeDB Kernel Mode Edition that, to the best of our knowledge, is the first commercial off-the-shelf database-management system designed for such deployment. The tool consists of the standard eXtremeDB in-memory embedded database runtime adjusted for kernel usage. The adjustments are relatively minor: Database locking is implemented via kernel-mode spinlocks rather than through synchronization primitives available in the user space, such as semaphores and spinlocks, and the database runtime does not use the C runtime. To facilitate the implementation of user-mode applications accessing the kernel-mode database, eXtremeDB-KM provides a simple interface compiler utility that is conceptually similar to the well-known Remote Procedure Call Interface Definition Language compilers (RPC IDL).
To illustrate its use, the eXtremeDB-KM distribution includes a sample application that creates a kernel-mode database to enforce access rules as part of an access-control system. The reference design includes another kernel module that intercepts filesystem calls and provides a file access authorization mechanism to the system. This "filter module" exports two types of interfaces: the "direct" API available to other kernel modules and drivers, and the "indirect" API that implements the ioctl interface to the database module.
