A Conversation with Michael Cowlishaw

Since joining IBM over 20 years ago, Michael F. Cowlishaw has worked in virtually all of IBM's major research centers--from the T.J. Watson Research Center in Yorktown Heights, New York, to the Laboratories Systems Technology Group in Great Britain. Over the years, Cowlishaw has received several internal awards. Most significantly he was named an IBM Fellow in 1990, which allows him to work on projects of his own choosing. His current interests include user interfaces, electronic publishing, and Internet protocols. Still, Cowlishaw is best known for developing the Rexx programming language. Frequent DDJ contributor Jack Woehr recently chatted with Cowlishaw about Rexx and other topics.

DDJ: Programming languages make kind of an abstract, aesthetic impression. Did you know Forth before you wrote Rexx?

MFC: I made it my business to know something about every language around at that time. I always considered myself a professional language designer, really.

DDJ: Rexx seems to have a little bit of a lot of languages in it. It kind of looks like Basic, but there's a factoring that's reminiscent of Forth, and a striving toward simplicity and convenience for the end user. Rexx is just an easy language to use at the command line.

MFC: That was the intent.

DDJ: The value of Rexx is that most shell languages are lousy as programming languages, and most programming languages are lousy for performing simple system functions. Rexx is halfway between.

MFC: The general principle is that very few people have to implement interpreters or compilers for a language, whereas millions of people have to use and live with the language. One should therefore optimize for the millions, rather than the few. Compiler writers didn't love me for that, because Rexx got to be a hard language to interpret or compile, but I think it has paid off for people in general, certainly programmers in general.

DDJ: Rexx's parser is weird, at least if you think that the way C does things make sense.

MFC: Right. Earlier languages all had very formal grammars, which we needed to parse and process, but they always had these syntax quirks which languages such as C and Pascal come across. I very deliberately didn't have that kind of formal, or simplified language grammar, in order to make it easier to work with. So rather, it was tuned for what people wanted to do and the way they wanted to write programs, rather than ways which made it easy to write compilers.

DDJ: So the difference between Rexx and other languages really is an aesthetic judgment that you and your associates made.

MFC: I think that's fair, yes. It's, "Try to apply good taste," or something like that, but "goodness," that's quite, of course, hard to define.

DDJ: Do you think that "classic" programming languages, where users type something into an editor telling the computer what to do, have any future?

MFC: I think they're going to be around for a long time. There are many instances where one does require precision of expression which a formal language such as a programming language makes possible.

Though there still may be many things that you will be able to do, say, by verbal instruction and/or drag-and-drop programming and these kinds of things, there will still be--for some applications at least--a need for precise algorithms. And programming is really just a way of expressing an algorithm.

One possible example is banking, where you want your bank account to be worked out very precisely.

DDJ: Isn't that the kind of program that is increasingly being done by drag-and-drop?

MFC: Right. But underlying that program is something that does arithmetic. It's a program, whether it's represented by a macro on a piece of hardware or whether it's a program written by typing text.

DDJ: Someone is still going to have to hew the wood and draw the water.

MFC: I think you have probably made a fair point that banking is one area where the underlying building blocks already exist, and now you can put the building blocks together. But there are still huge areas of commerce and human endeavor where the basic building blocks you're going to put together do not yet exist, and still have to be programmed.

DDJ: How long until they all exist?

MFC: Well, if they did all exist, then you'd never be inventing anything new anymore. I hope that doesn't ever happen.

DDJ: In A Discipline of Programming, (Prentice Hall, 1976) Edsger Dijkstra lavishes much care over a greatest-common-denominator algorithm. Is that sort of thing disappearing from computer science?

MFC: I don't think it has. People still refer to Don Knuth's books, which are very similar in their detail. Indeed, I'm writing an arithmetic package today, and that's the first place I went to decide which algorithm I'm going to use.

When I write something, I try and make sure it's [as] good as possible. From long experience, I know that programs can often last very much longer than you'd expect. I had a piece of electronic mail just the other day that suggested a new feature in a program, which I didn't recognize the name of. Then I looked back at my files. The last time I touched that program was 1979. He's still running it, and it works just fine. I had forgotten about it, and he suddenly had this idea for a new feature! I'm actually quite surprised that an assembly-language program written in 1979 is still running without being broken by changes in the operating system since then. This was under VM ("Virtual Machine Operating System" for mainframes), which was my primary operating system from 1976 until 1987, though I was using other operating systems as well: UNIX, DOS, of course, and now mostly I use OS/2.

All through the '80s, I was writing for PCs as well as mainframes, but it wasn't really a satisfying environment for me because of the 16-bit [limit] and other limits of the operating system. It wasn't until OS/2 came out--OS/2 2.0 in particular, which is 32-bit--that I really switched over totally.

Since the late '80s just about everything I write, except where it involves user interface, the GUI, is intended to be cross-platform portable. I try to keep out things that are specific to other operating systems. I have a character-based program we ported to seven different operating systems in a week.

DDJ: I look at Rexx with some awe in that it's a language which spread very quickly through the mainframe world, and beyond into Amiga, DOS, OS/2, NT, and UNIX.

MFC: That was largely due to IBM's internal network, VNET.

DDJ: Can you briefly discuss the Virtual Machine (VM) operating system?

MFC: VM's main attraction...certainly back in the '70s and some would say it still is...[is that it is] one of the best development environments around. Every user sharing that machine has effectively their own machine. It really was a series of PCs connected by a LAN, except they all sat in one box, and everyone had a complete, simulated virtual machine. It was a similar kind of development environment to what people have today. Each person had their own personal single-user operating system, with security since the boundary of each virtual machine was very well defined. It was a delightful environment to use and to program.

DDJ: Were you a VM enthusiast in its heyday?

MFC: I would say so, yes.

DDJ: Back in 1988, IBM was referring to a 386 running OS/2 as a "programmable terminal." The mainframes were still so powerful that a little box, even with a nifty GUI, didn't impress many IBM technicians.

MFC: It's such a big company that you have different people working in different places which are geographically and culturally widely separated. People often work in isolation and don't have the opportunity to spend time getting to know what's going on in other corners of the corporation.

The AS/400 division was already very successful before many people in IBM were aware of it or what it did or what its computers looked like or how they worked. This is the case of the PC in IBM. People in research were well aware of the potential, but people working on mainframes, because they were working on mainframes, had no need for PCs and therefore knew very little about them.

DDJ: I've heard IBMers say, "VM has outlived many of the executives who tried to kill it." People don't always realize how much VM influenced what we have today.

MFC: There is a parallel situation on personal-computer operating systems, where the operating system is setting up a virtual machine under which you run a copy of either the same or a different operating system. You can set up a DOS box under OS/2, and it's such a complete simulation of a PC you can actually boot DOS from a diskette into this virtual machine. That's essentially what VM did...provide you with a large number of 360/370 virtual machines, all running under the same operating system. This is a pleasant environment, since every user had their own "machine." It was also a good way of testing operating systems, because you could run them under VM and test without bringing down an entire machine, making it unavailable to users, until you had done that testing.

The idea of virtual machines didn't originate with IBM, but [VM's antecedent] CP67 was one of the earliest environments to use virtual machines, and VM first brought them to their potential.

DDJ: The engineers who developed the Intel 80386 and its V86 mode must have seen VM.

MFC: That's certainly true. VM took the concept of virtual machines to considerably greater lengths than the people who originally thought of it had in mind. VM did not stop at virtualizing the process unit, but began simulating [mainframe I/O] channels and channel adapters. Effectively, users had a complete system simulated, done in a way which, thanks to various hardware innovations, exhibited great efficiency. When you've got your time slice, you run as though you're the native machine. It's not all simulation. These [are] concepts we now see in OS/2 and so on.

DDJ: Are the mainframes still going to be there in ten years?

MFC: I think so. Firstly, for some applications, they're particularly well-suited, commercial applications with centralized databases. They're optimized for getting data on and off disks much faster than workstations usually are. They will evolve, and I suspect in ten years you may not be able to easily draw a distinction between what's a mainframe and what's not. I've been wishing for some years that someone would take the mechanics of a PC, which belch out heat and noise, and put them in a room miles away from where people are sitting, so that all you'd have on your desk would be the input and output devices you need.

That's essentially what the mainframes gave you, in that they consolidated the disk drives and the power supplies and the central processing units in one place and people had something very simple on their desks.

I'm not the first to point it out, but the World Wide Web browsers of today are effectively dumb terminals.

DDJ: Do you program in C++?

MFC: I program at the moment in either Rexx or C, including ObjectRexx.

DDJ: I find ObjectRexx the best new language idea of the 1990s.

MFC: IBM has stated its intention to make a version of ObjectRexx available for Linux. This is a free, use-as-you-will source version released for a vendor-neutral platform.

DDJ: What do you think of PC DOS Rexx?

MFC: It's a direct port of OS/2 Rexx. They took very little out of it, the double-byte character support. They have a different version of DOS in Japan anyway.

DDJ: Where are the hotbeds of Rexx usage today?

MFC: In the United States, obviously. Germany is very strong. I have a Rexx bookshelf which is about four feet long: German, French, Japanese, Swedish...

DDJ: Yet many people don't know Rexx is there because it's not common under Windows.

MFC: People often only know one operating system. Perhaps they've used another in the past, but now they only use one. Or if they've only used one computer language, the same thing applies, they often have the view there couldn't be anything better. I went through a phase very early on where I had to use some pretty awful languages. Then I came across Fortran. I was using Fortran for a while, then someone tried to persuade me to use PL/I. I thought, "Nothing could be better than Fortran." Then I was required to learn PL/I for my job, and I found out it's actually much better than Fortran. This opened my eyes [to the fact] that there's very much more to the computer business than what one happens to be using today.

A professional computer person does need the experience of a wide variety of operating systems and languages to be able to have a broad view and to be able to contribute to the field.

DDJ: What about the syntax of C?

MFC: I use C every day, so it's hard to be objective. If one's trying to write a concise language, then it doesn't do too bad a job. There are clearly things about it...I have more trouble with the semantics of argument passing and pointers than I do with the syntax. I think nowadays one could take C and apply the Rexx kind of syntax to it and still keep it efficient.

To some extent, needs have changed over the years. It was pretty important in the '60s and '70s to use notations to save typing because the main output devices in those days were teletypes and 2741s, which were very slow. APL was extremely popular in those days because on the slow output devices you could have big programs typed out in a very short time.

Later, conciseness began to be a burden rather than an advantage. It became more important to have readable programs than concise ones.

Now the need is to build things out of smaller components which some expert has previously written. ObjectRexx and SOM and OpenDoc are good ways of building such components, so I believe they will become very important in the future.

DDJ: Have you programmed these systems much?

MFC: Actually, most of what I do now is pretty low level. I'm writing a Web server now.

I was doing some research on neural networks, in particular a neural-network-based text-retrieval system. I wanted to be able to test that. Unlike most programs, where you can wrap something around them and go away, I had no idea what kind of queries they were putting in and how well the retrieval system was responding to them. I had to run it on my own machine to study how the algorithms were working, yet allow other people access to it.

I wrote a gopher server to do that a few years ago, and what happened is that because when I write these things, I attempt to generalize them, the gopher server had a Rexx interface. It was completely programmable, so someone figured out that you could program the gopher server to be a rudimentary Web server. But that wasn't ideal for various reasons, various assists it would have been really useful to have in the server.

So the gopher server evolved into a Web server. In doing that, I spent a lot of time.... I was unhappy with existing Web servers because they seemed to be very slow responding even when doing trivial things. I went back and with the advantage of hindsight, knowing what Web servers were being used for, I could try and build something optimal for that use. I concentrated on reduced response time and programmability. I used the fact that Rexx can run from memory without being fired up as a separate process to make a very fast, scriptable Web server.

For example, on a 486/50, it will respond to an incoming request in 20 milliseconds, including running a Rexx script. If you do some caching so it doesn't use the Rexx script, then it's about ten milliseconds. Logs are written straight through to disk but they don't affect response time. Quite a number of interesting interactions between the subsystems of a server made it quite an interesting project.

DDJ: Those numbers imply careful testing.

MFC: Careful measurements, not so much serious testing. I've found that when everyone is concerned about performance, the only clearly sensible way to go about it is do measurements. I've often...seen programmers realize that their program is too slow and spend days and weeks optimizing parts and [then] find it doesn't make any difference because they didn't do measurements to find out what's really taking the time.

I've seen this on a very large scale. I won't say what company it was in, not to embarrass anybody. But there's a very large project that was written entirely in Rexx, tens of thousands of lines. It was too slow as an application. The project team decided it was obviously because it was written in Rexx, so they split up and got four programmers and four subcontractors and they all went away and rewrote the components. Then they put them together for system test at the end and found that the response was only 2 percent better, which made absolutely no difference at all to the end user. Of course, the time wasn't being spent in Rexx at all, the time was being spent in database lookups and communications over networks.

I've seen examples of that over and over again in my career. Everyone knows intuitively what is wrong so nobody takes any measurements.

DDJ: How does one produce good software engineers?

MFC: I don't think there's one simple answer. It's self-motivation of the programmer, much more dependent on the individual personality than the training or whatever they have. It helps if they come across a lot of good people and work with a variety of different projects and operating systems.

Many of the best programmers I know were hardware engineers originally. A few years ago, if you were designing hardware, you'd make sure your design was right before you ever put it together. After you'd laid out the cards, and gathered the components, if you hadn't done your design right, you just had a piece of junk.

When I program, I don't find a lot of bugs. If it compiles--that is, after I correct my typos--it usually runs the first time. One question I often ask a programmer is, "How good are you at using the debugger?" If they're an expert at using the debugger, then I know they're not a very good programmer.

There are things where you do have to be expert at using the debugger, such as writing operating systems or device drivers, and those are different cases. But in general, if you are writing application-level code, then you shouldn't have to use a debugger. It implies you weren't being very thorough in writing the program in the first place.

DDJ: Do you feel you have a lot of foresight about where computing is going?

MFC: In questions I ask myself, I usually feel confirmed. One always misses some of the things that happen. One I didn't really see coming was the World Wide Web, like many people, because that year I had my head down and I wasn't on the Internet very much, and all of a sudden it happened.

Sometimes a very unexpected direction can have a great effect. I suppose in some sense Rexx was a bit like that. It was something I decided to do in the middle of one night. Maybe if I'd had something else for dinner the night before and felt ill it would have never happened, but now there are millions of people programming in it.

DDJ: How would you compare Rexx and Perl?

MFC: Larry Wall [creator of Perl] came to our Rexx Symposium a couple of years ago. We had an interesting joint session discussing languages. Perl's designed for a different audience. It's for a C-like programmer and it fits that audience. Rexx probably has a more general appeal.

There's a lot of overlap. Both languages are good, for example, for writing scripts for the Web. They are really quite different philosophies on how to design a language, both valid. Larry's philosophy is to put anything into the language that anybody asks for. Then it's there for everybody. Everybody has their own favorite features, and he makes people happy that way.

My approach is the other way 'round, that is, don't put anything in unless it's really, really necessary, because then you end up with a really small language with few notations, and you make people happy for a different set of reasons.

DDJ: What is your connection with ObjectRexx?

MFC: ObjectRexx was originally designed by Simon Nash, a long-time colleague of mine. It's very much his design. My contribution was more as a consultant, somebody to bounce ideas off, mostly about keeping it in line with the existing philosophy of Rexx.

We used to sit around the table in a pub in the village of Hursley and discuss things, rather than me saying, "You can't do it that way." It was done by consensus.

DDJ: What's Hursley like?

MFC: It's a small village halfway between Winchester and Romsey. It has two pubs next to it in Hursley Park, an old manor house which now forms headquarters of the IBM UK Laboratories, along with the other new buildings that have been built around it. It was previously owned by Vickers; it's where the Spitfire was designed in the Second World War. IBM took over more than 30 years ago and has restored the house pretty much to the way it was. Paneling, and the Wedgwood room still has its china, and these sorts of things.

DDJ: In addition to Rexx, what related work are you proud of?

MFC: One of the things I'm most proud of was some work I did on color perception, answering the question, "How many bits per color do you need in a pixel on the screen not to be able to tell the difference from the human point of view?" It works out that for a standard screen at a standard reading distance, you need about four bits in green, three bits in red, and two bits in blue. If you sample your image for that particular spread, it's indistinguishable from eight bits in each color. It's what formed the basis of the standard color palette in OS/2, which gives you more colors which are related to green than to blue or red, so if you have just eight bits for the display, you can do better than if the bits were equally divided between the colors.

DDJ: You've spent your time on interesting things!

MFC: I'm very interested in...how humans interact with displays, and languages...in some sense they are related. I've always been a fan of using color in displays [and the] use of color for indicating things. At the Oxford University Press project, the intent was that I should write an editor for a black-and-white screen because they felt a monochrome screen had better definition and [was] easier to read. I insisted on making it suitable for color as well. They then ran a formal test at the end of the project to decide whether they should use color screens, which were visibly fuzzier, or monochrome screens. They came out with a conclusive answer that it should be color. People dealing with markup on monochrome screens were kind of snapping at people, and losing their temper, and giving up halfway through the task, while the people using the color screens sailed through this interesting test.

DDJ: Can you tell us about the Oxford University Press project?

MFC: Around 1985, the Oxford University Press needed an editor that could handle highly structured data: the content of the Oxford English Dictionary, which is about a 20-volume, 1000-page-per-volume dictionary. They had chosen to mark it up with SGML, but they didn't have a good editor. In fact, nobody had a good editor at that time for dealing with that complexity of data. So I wrote an editor for them called "LEXX" which ran on IBM mainframes.

The Oxford English Dictionary was originally only in hard copy. It all had to be retyped to be put into electronic form, and then the structure of that data was marking the things that were headwords, things which were content, things that were quotations...that markup is all SGML.

LPEX is a reimplementation of LEXX. It's the same design under the covers, as far as the data and the way that the parsing is done, and so on, but it's reimplemented for OS/2 and AIX platforms. It's now mostly used for program editing, because of its ability to parse data and color keywords, and other features.

DDJ: In other words, your programming career hasn't just brought technical satisfaction. You've also performed a chore of great social import, in that you have aided the computerization of the Oxford English Dictionary.

MFC: That's correct.