A C++ project that is coded for a single platform contains any number of subtle functions, data types, and language quirks that bind the project to its original platform. The porter’s job is to identify, isolate, replace, and rewrite such dependent features of a code base. That’s what I’m doing with the tactical warfare game Laser Squad Nemesis [1]. The application contains 130,000 lines of C++ code written for Windows and DirectX, and our customers want to play it on a modern Linux system. Our long-term plan is to remove all of its hardwired Windows dependencies, but removing all the dependencies in one swoop would be a difficult job that would require changes to large parts of the code base all at once. Furthermore, these changes would have to occur in parallel with ongoing work on the Windows version. Trying to maintain separate branches for different platforms would run the risk of leaving two disparate and unmergeable sets of source code, which is something a small development house cannot afford.
WineLib library [2] offers a solution to this problem. WineLib is an almost-complete implementation of the Win32 C/C++ programming interface for Linux. In other words, WineLib provides all the functions, types, and interfaces you need to compile a Windows program successfully, only not under Windows. WineLib is actually the library side of Wine, a project that grew from the primary goal of creating a binary compatibility layer for Linux. WineLib is free to integrate with most projects, even commercial projects, and it is well supported by its authors, some of whom are paid to work on it.
Wine and WineLib
The Wine project offers two approaches to Windows compatibility:
- Binary compatibility through a tool called wine (often referred to as the wine binary loader). You can simply execute a Windows program in Linux using the command wine myprogram.exe.
- Source-code compatibility through the Win32-equivalent components of the WineLib library.
The wine binary loader does not exact any direct performance penalty over creating the equivalent Win32 rebuild with WineLib. However, the binary loader does not really adapt the program to the benefits of the Linux environment. (The whole point is that your program still thinks it is running in Windows.) If performance matters, you may wish to rewrite part of the code to take advantage of Linux-specific features. For instance, I want the game I’m porting to Linux to use the SDL libraries instead of the DirectDraw and DirectSound implementations called by wine. If you wish to write Linux-specific functionality into the program, you’ll need to port your source code to Linux and recompile. WineLib provides the Win32-equivalent API you’ll need to recompile your program in Linux. WineLib by itself does not optimize your program for Linux; however, WineLib gets your whole application up and running in Linux quickly. You can then follow up by tailoring small, critical bits of the code to the peculiarities of the Linux platform. For instance, I can integrate parts of my application to SDL gradually rather than having to eliminate all the DirectX at once. WineLib allows you to perform much smaller, and therefore more reliable, refactoring steps.
How Does It Work?
WineLib enables you to compile your Windows program in the Linux environment. However, the notion of “compiling against WineLib” to give you a “native” application is naïve at best. Although WineLib allows a relatively straight port from the compiler’s point-of-view (in that most of your Windows code can remain unchanged, and the code runs directly on the Linux host), the wine binary loader still implements a Windows runtime environment to keep track of Windows concepts such as DLL relationships, drive letters, and so on. The binary loader is needed for any application relying on Wine functions. This is to ensure that applications running under the binary loader need no modification to run under Linux. WineLib is not just a simple C library, but an understanding and a rationalization of the facilities that Windows has to offer the application programmer. I’ve found that understanding WineLib is a great step towards learning about Windows as a platform.
Getting Wine
The first step I’d recommend is to fetch and build the Wine distribution right now. For most Linux systems, the following commands will work:
# check we have version 2.95.2 or better gcc -v export CVSROOT=\<br> :pserver:[email protected]:/home/wine cvs login # type "cvs" as the password cvs co wine cd wine ./configure && make
These command downloads about 50 MB, and the build structure will be around 400 MB when you are finished. The Wine distribution has little in the way of external dependencies: only the X11, OpenGL, and curses libraries, which are standard parts of any Linux distribution. Although you can find many pre-built versions of Wine for your system, a distribution built from source that’s easily available and tweakable will aid your debugging, since back traces into Wine source code will be available. When I make reference to the “wine” directory, I am referring to the root of your Wine source-code build.
WineLib
Look inside wine/include, and you’ll find windows.h and friends. You can immediately try compiling code written for Windows like this:
g++ -Wall -c -Iwine/include -o HelloWindows.o HelloWindows.cpp
You may find that your source files compile against the WineLib headers without complaint, as they would under Windows. What you’re left with will be a set of object files for your Linux system. However, Win32 executables have an entry point called WinMain, and simply linking them all together with ld to form a Linux executable will not work. Although the object files reference Win32 symbols, Wine is not implemented as a simple external library that you can link to in the usual manner. It needs the Wine runtime environment and loader. If you want your program to have a WinMain, it must be run through the “wine” executable, which is built with the rest of the code. Your “executable” must in fact be a Linux shared library (.so) supplemented with some fix-up code generated by the winebuild tool, which I’ll describe later.
An article by Eric Pouech from a couple of years ago [3] explains these reasons well. In brief, Wine’s loader must have DLL linkage information from any binary that it loads on the same terms as it would read the information from any native DLL. So Linux-based DLLs or EXEs must have extra information contained within them for Wine to understand whether they require further DLL code to be loaded. They must also contain symbols that point to the same binary data that would be found in a DLL in order to define relationships to other DLLs. Because Wine treats real Windows DLLs and EXEs with the same semantics as “native” DLLs and EXEs, it allows Linux programs to call functions in plain Windows DLLs and vice versa, so your program can use a mixture of natively compiled code and third-party Windows DLLs. Of course the shared object files are completely normal and can have Linux-specific dependencies specified in the normal way that will be respected by the Linux dynamic linker. Because of the perceived equivalence from Wine’s point of view of a Linux “augmented” .so file and a Windows DLL, such native Wine binaries are conventionally given the extension either .exe.so or .dll.so.
winebuild
The Wine distribution includes a tool called winebuild, which generates the fix-up code and binary data necessary to mark a Linux .so file as a DLL or EXE with which the Wine runtime can work. winebuild references a file called a “spec file” that is roughly equivalent to the import and export declarations necessary when building a Windows DLL. winebuild uses the spec file data to generate a .c file, which will be compiled and linked in with your final shared library. There are plenty of examples of spec files in the wine/dlls directory for the built-in Windows DLL implementations (kernel32.dll and friends), but the winebuild tool does not necessarily need a spec file. I’d suggest that maintaining a spec file is unnecessary for a simple .exe that does not need to export symbols to other EXEs or DLLs. You can get your fix-up code for most stand-alone executables with a simple script such as the following:
#!/bin/bash WINE=../wine export LD_LIBRARY_PATH=$WINE/library:$WINE/unicode $WINE/tools/winebuild/winebuild \ -fPIC -DSTRICT -D_REENTRANT -exe MyApp.exe -mgui *.o \ -L$WINE/dlls -luser32 -lgdi32 -ladvapi32 -lkernel32 \ -lddraw -ldsound -lwinmm > MyApp.exe.c
winebuild will go through your application’s object files and match up any symbols imported by your object files with symbols exported by any DLLs you specify. The generated .c file contains some horrendous-looking static data and an inline assembler, which can then be compiled with the rest of your program. You can use ld -shared to generate your final .exe.so file.
Once you have your .exe.so, you can run it through the wine binary loader with a script like this:
WINE=/home/mattbee/Work/wine LD_LIBRARY_PATH=<br> "$WINE/dlls:$WINE/library:$WINE/unicode:$LD_LIBRARY_PATH" WINEDLLPATH="$WINE/dlls:$WINE/programs" WINESERVER="$WINE/server/wineserver" WINELOADER="$WINE/miscemu/wine" WINEDBG="$WINE/programs/winedbg/winedbg" $WINELOADER MyApp.exe.so
Or you can use gdb to start the binary loader and debug your program as you’d expect, with the added advantage of being able to back trace into Win32 functions.
Cleaning Up
While using the binary loader on your Visual C++ compiled .exe will probably work flawlessly, the gcc-compiled .exe.so file will probably fall over. About 70% of my time with Laser Squad Nemesis has involved the obvious and mundane chore of ensuring everything in the program that referenced a filename used upper and lower case consistently and used forward slashes for paths instead of backslashes. The obvious cases will be in #include directives: try to #include "headers\foobar.h" and Linux gcc won’t find it — likewise #include "headers/FooBar.h" when the filename is foobar.h. For filenames that are used within the code, you may get failures a little way into the running of the program when a fopen call returns a NULL or something similar. Laser Squad Nemesis had a couple of points where such error conditions were just ignored and injected erroneous data into the program that only revealed itself sometime later. My solution in that case was to write a wrapper for fopen called xfopen, which replaced any backslashes in filenames with the platform-specific path separator that I defined in a macro. Another solution (mentioned below) is to use the Windows C library MSVCRT supplied with Wine whose fopen call will cope with backslashes automatically. I would advise against this since your program will have problems integrating into a Unix file system when its fopen calls are case-insensitive, and different sets of characters are not permitted in filenames between the two systems. You might as well port your program in every sense rather than just make it run.
A useful first port of call for these kinds of problems is the winemaker tool, which comes with Wine. It’s a script designed to process a source tree straight from Microsoft Visual Studio and produce a “first draft” WineLib source tree, including Makefile. With a backup of your source tree, try something like:
cp -r MySourceDirectory \ MySourceDirectory.new wine/tools/winemaker --windows \ MySourceDirectory.new 2>warnings diff -urN MySourceDirectory \ MySourceDirectory.new | less
which will spit out a new source tree for you to work on with a few of the “obvious” problems described above fixed and allow you to examine the changes it has made. It also (by default) converts DOS to Unix line endings, warns you about Visual C-specific directives, renames your source files to be consistently lower or upper case, and gives you a Makefile that can be processed by GNU autoconf. All of this behavior can be controlled from the command line. You should certainly look at what it produces before forging ahead with manual changes, because it’s a “one-shot” tool intended for saving you a lot of work before you start.
Your next most basic portability problem will be which C library to use: WineLib includes an implementation of the Microsoft Visual C Runtime DLL, which you can use in preference to Linux’s more usual GNU C Library by making sure you use -Iwine/include/msvcrt on the command line. This will give you I/O calls that respect Windows drive-letter references and backslashes and are case-insensitive. On the other hand, you may decide that exorcising all but ANSI functions from your code will be an aid to future portability, as I did. This involved writing trivial inline reimplementations of occasionally used Microsoft-specific C library functions such as _strupr and, in my case, changing functionality to read directory entries. In general, I noticed a lot of liberally licensed code available for Win32 that emulated POSIX standards, whereas Wine is the only public code base that attempts the opposite feat. So it made sense when trying to increase a piece of code’s potential portability to use a POSIX interface where it’s available on Win32 or to write your own abstraction where there isn’t a POSIX standard. In Laser Squad Nemesis, for instance, I rewrote code that scanned files in directories using the POSIX struct dirent and friends and then added some freely available code to the Win32 build [4] that emulated the semantics for Win32.
The hardest portability problem I faced with WineLib was in using COM objects. I noticed it when a piece of code that called a COM method generated an abort that was inside a completely different COM method. For example, if you #include <ddraw.h> (i.e., any other Windows header file that defines COM objects), you will have defined an IDirectDraw7 type that should provide a set of virtual functions for your C++ program to use. This is a point where binary compatibility between WineLib programs and native DLLs is stretched and sometimes broken because WineLib code absolutely must assume that Visual C++’s virtual function tables are used, both when implementing COM objects and when calling them, because programs running under the Wine runtime can make calls between genuine Windows DLLs and Wine DLLs. Visual C++’s virtual function tables put the first function pointer at offset 0 and are thus the binary compatibility standard imposed on compilers wanting to implement COM interfaces. g++ has for a while started its virtual function tables at offset 8, and this has even changed between major gcc versions. So in order to avoid the situation where gcc simply generates calls to the wrong virtual functions, the WineLib authors use a complex (though documented [5]) set of macros to persuade gcc to generate compatible function tables. The upshot for gcc users is that you need to make a difficult choice when compiling your programs. Either you can use the latest and greatest version of gcc (3.2), which recognizes __attribute__((com_interface)) as a prefix to a struct declaration in order to selectively generate Visual C++-compatible function tables, or you can ditch Windows binary compatibility and compile all the Wine COM implementations with gcc’s eight-byte offset. The reason I chose the latter is that the Linux version of Laser Squad Nemesis did not need to call on any Windows DLLs, and my chosen Linux distribution’s C++ compiler version of choice, gcc 2.95.4, does not support the attribute. Finally the binary calling interface for compiled C++ differs incompatibly between gcc 2.95 and the latest gcc 3.2, so I would have had to maintain versions of all the external C++ libraries that the game relies on separate from those that ship with mine and most other people’s Linux distributions: notably STLport. This situation will change in time as Linux distributors standardize on g++ 3.2 as their standard C++ compiler, but for now the choice as to which way to go with regards to COM will depend on the needs of your project.
This has been too brief an overview for my liking; I’ve left out how to use the resource compiler wrc and how to write spec files for DLLs, mainly because my project didn’t need either. But I hope this gives you some confidence that WineLib isn’t difficult, just patchily documented from the conceptual point-of-view. You may also question why you need to “port” to WineLib when wine will run your unmodified binaries much more easily. For me, WineLib is a stopgap allowing me to get a partially ported version tested before getting rid of all the Windows-code from the Linux version for good. So raise a glass to your project’s best escape from Windows.
Notes
[1] <www.lasersquadnemesis.com/>
[return to text]
[2] <www.winehq.com/>
[return to text]
[3] <http://kt.zork.net/wine/
wn20001218_74.html#4>
[return to text]
[4] <www.two-sdg.demon.co.uk/curbralan/
code/dirent/dirent.html>
[return to text]
[5] In wine/include/obj_base.h in the Wine distribution.
[return to text]
About the Author
Matthew Bloch is a graduate of King’s College, Cambridge and one half of Bytemark Computer Consulting Limited, a consulting and Linux hosting firm based in York, UK. You can email him at [email protected].
