Open C Case Study: Porting SQLite for S60
To show how to use Open C, we port SQLite (www.sqlite.org), a small C library that implements a self-contained, embeddable, zero-configuration SQL database engine, from the desktop to the S60 platform.
But before beginning the port, we need to touch on platform security that is involved in the port. Platform security protects the integrity of devices, provides confidentiality for sensitive data, and controls access to sensitive operations. In application development, platform security entails acquiring certificates and determining capabilities for applications. The two major aspects of platform security are:
- Data caging. Applications and users have access only to certain areas of the filesystem. In practice, the applications can access their own private directories and directories marked as open. This means that one application cannot access another application's private directory and data, nor access the directory where executables are stored.
- Capabilities. Used to specify what functionality an application is trusted to use; capabilities are allocated to the application at build time and policed at runtime. Once the capabilities are assigned to applications, they cannot be changed. Thus, applications have a set of unalterable capabilities that describe what access the application has to the APIs. Capabilities are specified in the MMP file.
S60 3rd Edition requires mandatory .sis file signing, which means that all .sis files must be signed using the SignSIS or CreateSIS utilities (included in the SDK) before installing them to the device. The installation package can be signed with a self-created key and the certificate can be generated with the SDK. The .sis signing makes it difficult to tamper or otherwise change the original installation package.
Additionally, applications can be certified with Symbian Signed. A Symbian Signed application has passed certain tests and the originator's identity has been verified. The aim behind these measures is to avoid malicious software by providing a tamper-proof digital signature to the application. For more details on platform security and signing, see "The SymbianOS Security Model," by Regan Coleman (www.ddj.com/dept/architect/193100464).
Now we're ready to start the port, which consists of eight steps.
1. Initial Analysis. Again, Open C is a subset of functionality from selected libraries, not the full-blown implementation we find in most of the UNIX systems. Therefore we begin our SQLite port by configuring (./configure) and compiling (make) SQLite on a Linux box. At this point, it's worth noting that we must capture the build log; we'll need it later to identify which source files are for what libraries. Once we have successfully compiled the source code in the Linux box, we use standard tools (such as ldd and nm) to see what libraries and functions are being used by it. In doing so, we notice that SQLite is pretty much only dependant on libc.
2. Moving the Source to S60 Environment. Now let's copy the environment we had in the Linux box to the S60 development environment. This is usually worth doing, because often (./configure) scripts create configuration files (such as config.h) that define the functionality of your environment, and we are better off with some initial values. Because the building environment for Symbian is different from that in the UNIX world, we need to build some new intermediate makefiles. Following S60 coding conventions and directory structures, we make a group folder for our project in the S60 environment: \SQLite\group.
3. Defining the Environment and Makefiles. This group folder in the Symbian/S60 development environment defines the target environments, exportable header files, the Symbian makefiles (called "MMP-files" in Symbian development), and some other project resource files. We begin by creating a project configuration file called bld.inf (see Listing Two).
-- begin file bld.inf -- // Project configuration file for SQLite3 // Project platforms PRJ_PLATFORMS WINSCW ARMV5 GCCE // Project exports PRJ_EXPORTS // MMP files for project components PRJ_MMPFILES libsqlite.mmp sqlite3.mmp -- end file bld.inf --
The fourth line in Listing Two defines the target platforms. In this case, that's the emulator WINSCW (for "Windows CodeWarrior," an IDE for Windows developers) and ARMV5 (the target compiler), and GCCE (for target compiler using gcc). PRJ_EXPORTS states that we are about to export a header file (sqlite3.h, in this case) and identifies the location we export it to. PRJ_MMPFILES defines the Symbian makefiles for both targets—one for the SQLite library, the other for the executable so that we can use the library.
4. Creating Project MMP Makefiles. Next, we need to build up MMP-files listed in the bld.inf. At this time it is good to have the build log handy to see what files are needed for building the SQLite library as well as the SQLite application. Listing Three presents the MMP file.
-- begin file libsqlite.mmp --
// General properties
TARGET libsqlite.dll
TARGETTYPE dll
CAPABILITY NONE
UID 0x1000008d 0x00000001
//EPOCHEAPSIZE 4 4194304 // min. 4KB and max 4MB
// Allow global writeable static data
EPOCALLOWDLLDATA
// Source files
SOURCEPATH ..\src
SOURCE alter.c analyze.c attach.c auth.c btree.c build.c callback.c complete.c
SOURCE date.c delete.c expr.c func.c hash.c insert.c loadext.c main.c os.c
SOURCE os_unix.c os_win.c os_os2.c pager.c pragma.c prepare.c printf.c
SOURCE random.c select.c table.c tokenize.c trigger.c update.c util.c vacuum.c
SOURCE vdbe.c vdbeapi.c vdbeaux.c vdbefifo.c vdbemem.c where.c utf.c
SOURCE legacy.c vtab.c
USERINCLUDE ..\src
USERINCLUDE ..
// System include paths
SYSTEMINCLUDE \Epoc32\include
SYSTEMINCLUDE \Epoc32\include\stdapis
// Library dependencies
LIBRARY libc.lib
-- end file --
In the Symbian MMP makefiles, we define the target, the target type, the capabilities the executable needs, and the sources for our binary. All projects where Open C is used must specify the system include path for Open C headers. Open C headers are located in the \epoc32\include\stdapis folder.
At the end of the file, we define the libraries needed for compiling SQLite; see Listing Four for the specific MMP file.
-- begin file sqlite3.mmp // SQLite3 executable component properties // General properties TARGET sqlite3.exe TARGETTYPE exe CAPABILITY NONE UID 0x100039CE 0xA000029F VENDORID 0 START RESOURCE sqlite_reg.rss #ifdef WINSCW TARGETPATH \private\10003a3f\apps #else TARGETPATH \private\10003a3f\import\apps #endif END //RESOURCE // Allow global writeable static data EPOCALLOWDLLDATA // Source files SOURCEPATH . SOURCEPATH ..\src SOURCE shell.c USERINCLUDE ..\src // System include paths SYSTEMINCLUDE \epoc32\include SYSTEMINCLUDE \epoc32\include\stdapis // Library dependencies STATICLIBRARY libcrt0.lib LIBRARY libc.lib LIBRARY euser.lib LIBRARY libsqlite.lib -- end file sqlite3.mmp --
The executable (sqlite.exe) has a different-looking MMP file. Here we state that the target type is an EXE. There is also an application registration file for sqlite.exe, important so that an icon appears on your S60 phone. We are also linking a static library libcrt0.lib to our binary, so that we can code the main() function to be the entry point for this executable. The libcrt0.lib must always be specified as the first library and contains the E32Main entry point required by Symbian applications. Additional libraries should come as no surprise, so we also need to have the Symbian base library called euser.lib, the libc, as well as our brand new sqlite library. Libc is the only mandatory library that you have to link against when using Open C. Depending on the APIs, you might need to specify other libraries as well.
5. Showing the SQLite Application the S60 Software Grid. The application registration file (called sqlite_reg.rss) looks like this:
#include <appinfo.rh>
UID2 KUidAppRegistrationResourceFile
UID3 0xA000029F
RESOURCE APP_REGISTRATION_INFO
{
app_file="SQLite3";
embeddability=KAppNotEmbeddable;
}
6. Modifications Required to Source Code. Now that we've done the groundwork, we need to start modifying the source code. Recall that we mentioned that even though all the source code is C, we do not export any functions without explicitly annotating so. This is because we are compiling the source code as if it was C++ code. For the library to export functions or for an application to import a function from the library, we need to use the EXPORT_C and IMPORT_C macros, respectively. In a statically loaded DLL, an import library is needed to resolve the reference at link time. Therefore, exporting a particular function from a library, we add EXPORT_C in front of the function as an example. Here are some examples:
/* src/main.c */—
/* The version of the library */
EXPORT_C const char
sqlite3_version[] =
SQLITE_VERSION;
EXPORT_C const char
*sqlite3_libversion(void){
return sqlite3_version; }
EXPORT_C int
sqlite3_libversion_number(void){
return SQLITE_VERSION_NUMBER; }
/* sqlite3.h */—
IMPORT_C int sqlite3_open(
const char *filename,
/* Database filename (UTF-8) */
sqlite3 **ppDb
/* OUT: SQLite db handle */
);
IMPORT_C int sqlite3_open16(
const void *filename,
/* Database filename (UTF-16) */
sqlite3 **ppDb
/* OUT: SQLite db handle */
);
In S60 3rd Edition Feature Pack 2, you don't need to use IMPORT_C/EXPORT_C with the new target types. This makes porting easier.
One modification for the executable sqlite.exe's file shell.c is to comment out the signal.h file, because signals are not supported by Open C. The only signal this executable used was SIGINT, which was used to abort the execution of the process.
Because sqlite3.exe is an executable with main() entry point, these statements must be added to shell.c to overcome an issue with the GCCE toolchain (this applies only to the Open C SDK plug-in):
#ifdef __SYMBIAN32__ /* GCCE specific header file for */ /* GCCE toolchain problem that */ /* must be included when using */ /* main() entry point.*/ #include <staticlibinit_gcce.h> #endif
If everything goes okay, we should now do the following to build and make all the libraries.
7. Building and Finalizing Everything from the Command Line. Once the components have been built, we need to make an installable for the S60 phone. To make an installable, make an installation package text file (called "pkg file") containing instructions on how to build the software installation packages (called "sis files"):
;Header
#{"sqlite3"},(0xA000029F),1,0,0
;Supports S60 3.0
[0x101F7961], 0, 0, 0,
{"S60ProductID"}
;Localized Vendor name
%{"sqlite3"}
;Unique Vendor name
:"Vendor"
;Files to install
;sqlite3.exe
"\epoc32\release\ARMV5\UDEB\sqlite3.
exe"-"!:\sys\bin\sqlite3.exe"
"\epoc32\release\ARMV5\UDEB\libsqlite.
dll"-"!:\sys\bin\libsqlite.dll"
"\epoc32\data\z\private\10003a3f\
import\apps\sqlite_reg.rsc"
-"!:\private\10003a3f\import\apps\
sqlite_reg.rsc"
To build the SIS file, use the Makesis (which builds the SIS package) and Signsis (which signs the package) tools. Because of the advanced platform security features in Symbian OS 9.1 and later, all SIS files need to be signed prior to installation. At this point, you can sign the package:
makesis sqlite.pkg sqlite_unsig.sis
signsis -s sqlite_unsig.sis
sqlite.sis yourcert.
cer yourcert-key.pem
or get developer certificates from Symbian (https://www.symbiansigned.com).
Conclusion
So what do we have? We have a running SQLite on S60 smartphone. A source code line count with find . -name "*.[c,h]" -exec cat {} \; | wc -l shows 84,064 lines of code. And how much time did it take? Roughly three hours. Portability sure pays off.
