Notes from the Construction Zone

We are in the midst of adding a second floor to our house while living in it. The crashing, banging, and overall chaos make it hard to think about much else. Accordingly, we have been observing how builders go about their craft, and comparing that process with how software developers go about theirs.

Analogies can be slippery; arguing by analogy can be misleading. In particular, software is fundamentally different from hardware of all kinds, including the kind that goes into houses — different in ways that should dissuade us from drawing analogies between software and hardware too readily. For example, hardware has physical constraints that software does not share. Not everyone is strong enough to lift a sheet of plywood, let alone carry one up a ladder, and once you've cut a piece of lumber to the right size for part of a wall, you can't just type a few keystrokes and replicate that piece of lumber for the rest of the wall.

Therefore, we will resist the temptation to draw sharp conclusions about software from our construction project. Nevertheless, comparing the software and construction industries may yield useful insights for software practitioners. Our construction project occupies a similar number of people to a small software project over a similar period of time, and shares some of the problems of design, implementation, estimation, testing, components, tools, and debugging with software projects. Accordingly, we offer some observations about the project and invite readers to draw their own conclusions.

Design and Implementation

An early step in our project was to consult an architect. Software people who are familiar with Christopher Alexander's writings about architecture [1] may imagine that an architect's main purpose is to define a space that behaves suitably for its intended purpose in its social context. In our project, that part of the architect's work turned out to be severely constrained by facts outside his control. For example, if we had to move, we wanted to move once rather than twice. Therefore, if we were going to add onto our house, the addition would have to be buildable while we were living there. Also, zoning restrictions prevented us from enlarging the house's footprint, which meant that the only place to build new space was above existing space. Therefore, a surprising amount of the architect's effort and ingenuity went into solving two problems that we had not forseen:

• Whoever originally built our house did not intend to add a second floor. Accordingly, the first floor had to be strengthened before we could build on top of it.

• A key part of this strengthening was to make the ceiling joists strong enough to support the floor that would be built over them. If the roof is off, it is easy to replace ceiling joists with stronger ones. However, without a roof, the house needs a temporary covering to keep the rain out, and such coverings are never completely watertight. Accordingly, to minimize the difficulty of living in the house during construction, it was important to get the new roof up as quickly as possible — which argued against removing and then replacing the joists.

To solve these two problems, our architect specified a whole new set of floor joists to be installed above the old ones, together with beams to support them. In effect, rather than using the existing first floor to support the new second floor, our architect arranged for most of the weight of the second floor to rest directly on the foundation, with the weight carried by beams to be installed during construction. Moreover, by installing the new joists above the old ones, he made it unnecessary to remove the old joists. By doing so, he made the builders' job much easier, and also reduced the time during which our house would be open to the elements.

In short, much of our architect's work was to solve our unexpected problems: He ensured that the structure would perform appropriately (that is, that it would be strong enough), and he made the structure easier to build — and live in — than it would have been otherwise.

Perhaps the reason we did not anticipate these problems is that they are of a kind that software architects often ignore. Surely, you have heard of software projects that, once built, turn out to be unacceptably slow. Have you ever wondered why performance problems didn't surface until the project was nearly complete? Why wasn't performance considered as part of the architecture?

Similarly, you have probably also heard of projects that make sense in isolation, but cause havoc to the systems around them. All too often, software architects forget that their programs have to work with systems that already exist.

We spent a lot of time with the architect going over plans. An architectural plan is interesting both in what it specifies and what it doesn't. For example, it shows room dimensions accurately enough to make it possible to figure out where furniture will fit. With only a little practice, it is possible to imagine quite a bit about how a particular space feels, given only a drawing of it. On the other hand, even though the builders would ultimately use these drawings as detailed construction guides, the drawings left many details unspecified, such as the exact locations of studs, joists, and many other components that went into the final structure.

We can think of no analogy to architectural drawings in the software world. Screenshots come to mind, but they emphasize the details of how users deal with a system instead of what the system actually does. There are certainly specification languages of various kinds, but they tend to be oriented more toward implementers than toward users. We wonder whether it would be possible to put together a rough mock-up of a software system — not even a prototype — that would give users an idea of how the system would behave before the implementation is actually begun.

Estimation and Economics

Builders normally operate in a way that is rare in our experience in the software world: There is a general contractor, who is responsible for the overall job and who typically agrees to do the whole project at a fixed price. The general contractor is then responsible for choosing subcontractors. Some of these subcontractors do their part of the job at a fixed price; others are paid for time and materials. What differs from our experience in software projects is that if there is an error in estimation, it is the general contractor who gains or loses.

One great advantage of fixed-price contracts is the incentive that they provide to finish projects promptly. A disadvantage is pressure to cut corners, especially when the client doesn't know enough to tell when corners have been cut. However, there is a force that balances the pressure to finish too quickly — a force that, as far as we know, has no equivalent in the software world — namely, the building inspectors.

Inspections and Oversight

Houses are built according to detailed rules called "building codes." Before a house is considered complete, building inspectors will have checked every aspect of its construction. We have lost count of how many times we have had to arrange for inspectors to come and verify that the work done so far was "according to code" before construction could proceed.

For example, as we write this paragraph, a mason's crew is digging a trench that will eventually hold concrete blocks that, in turn, support the stone platform in front of the front door. The mason's crew is working here because the door has moved a few feet to one side [2] so the old platform is in the wrong place. The inspector will not approve this work until the trench is deep enough to expose the footing — the concrete base on which the platform rests. The inspector ensures that this footing complies with the building code. The notion of an inspector who is empowered to check compliance with the building code, and who can demand that work be redone if it doesn't meet the code, is unusual in the software field. Perhaps the reason is that the building codes represent an engineering consensus about how buildings should be built, and no similarly strong consensus exists about how to write programs.

Components

Builders have access to a bewildering variety of components. For example, when we were choosing tile for the bathroom, we visited at least four tile stores, each of which had hundreds of varieties of tile available. Why isn't there similar flexibility in the choice of software components?

One reason may be that physical components, such as tiles, have much simpler interfaces than software components. It may seem silly even to talk about the notion that something as simple as a tile should have an interface, but it does. For example, some tiles are intended to be used on floors, others on walls. Tiles come in different sizes and shapes. Some of them have finished edges, others are designed to be placed only next to other tiles. All have the ability to be attached to a flat surface using any of a number of standard adhesives.

Another characteristic of physical components is how often the builders wind up modifying them. For example, our builders had to cut many tiles to make them fit in the available space. More generally, almost every piece of lumber and wallboard that went into this project had to be cut to fit. The accuracy of these myriad cuts is one important factor that determines the quality of the finished product.

In software, components tend to have enormously complicated interfaces, and it is often impossible to change either the interfaces or the components themselves. Moreover, components often depend on other components. A library written in one language will be useless in other languages. Indeed, a library may well depend on a particular compiler or OS.

Tools

Builders use tools. They seem to have truckloads of tools — literally. Our general contractor is on vacation at the moment, and he has left his van parked in our driveway so that his colleagues can have access to his tools.

Unlike programmers, builders expect to have to pay — out of their own pockets — for tools that they need to do their job. How often have you seen programmers willing to spend their own money on specialized tools that will make their work easier?

Specialization

We mentioned earlier that general contractors usually deal with subcontractors for specific jobs. In our case, we had subcontractors for plumbing, electrical work, painting, heating/air conditioning, flooring (two different ones: one for the vinyl floor in the laundry room; another one for the wood floors in the rest of the house), masonry, wallboard, and probably a few other items that we've forgotten. All of these subcontractors were there because their specialized skills made it possible for them to do their part of the job less expensively than if the general contractor's crew did it themselves.

With software, it is not unusual for people to become experts in particular parts of a system. But what is unusual — at least in our experience — is the idea of specializing in solving a particular kind of subproblem.

Debugging

Anyone who has tried to debug a large program knows how easy it is for a problem in one part of a system to affect other parts. Fortunately, the problems that plague house construction are subject to the laws of physics, which make them easier to find — at least in principle.

In practice, debugging houses is still difficult. For example, we had a long-standing problem with the skylights in another part of the house: Whenever the weather was cold enough, water condensing on the inside of the skylights would drip onto the floor. We described the problem to a representative of the skylight manufacturer, who responded with an unequivocal, "That shouldn't happen."

It turns out that the skylights were incorrectly installed. If there is not enough insulation between the skylight and the shaft in which it is installed, warm air can seep out through the gap between the skylight and the shaft. When that air touches the underside of the metal flashing that surrounds the skylight and goes under the roof shingles, the water in the air condenses and leaks back through the shaft. This complex interaction of the skylight, flashing, air, condensation, and insulation reminded us of the kind of interaction that sometimes happens when several software packages conflict over a shared resource.

Sometimes, houses even crash. We heard about a house in which its owner had installed a furnace that vented its combustion gases out through a side wall instead of using a chimney. In principle, there was nothing wrong with this installation — the manufacturer approved it and all of its components. However, in this particular case, the vent was installed too close to the ground. One day, a huge snowstorm and drifting snow covered the vent, and the furnace was unable to discharge its exhaust gases. Rather than release possibly dangerous gases into the house, the furnace shut down, as it was designed to do. Unfortunately, the owners were away on vacation at the time. By the time they returned, the house had gotten cold enough that the water pipes had frozen. It cost many tens of thousands of dollars to repair the resulting damage.

It is partly to forestall just this kind of problem that the building code for furnace vents requires the vent to be installed a significant distance above the likely maximum snowfall. Unfortunately, this particular winter was much worse than usual.

Surely, you have seen software systems fail in similar ways.

Discussion

It is often said that the later in development a software problem is found, the more expensive it is to fix. This notion is even more true in house construction: It is much less expensive to redo an architectural drawing than it is to rip out a wall and build a new one. Perhaps for this reason, we found that much more thought went into design before implementation began. Indeed, the design was complete well before we had to choose who would implement it. Because the design was so complete, we could get bids from several potential implementers. Moreover, because of fixed-price implementation contracts, there was a strong incentive to get the job done quickly, but with building codes and inspectors acting to forestall work from being done too quickly.

Most of the construction uses standardized components, chosen from a vast array of possible suppliers. Most of the components have to be modified in some way, typically by using tools, such as table saws, before installation. The builders select, and pay for, their own tools — so they are willing to buy tools that will make their work more efficient, and to reject tools that don't do what they claim.

Some jobs, such as plumbing, electrical work, wallboard, and painting, are specialized enough that it pays for builders to contract with other people to do that work. In turn, there are companies that do nothing but install wallboard.

Houses, like software systems, have bugs. Sometimes they are straightforward to find; at other times, they are obscure. Once in a while, a cascade of errors can cause an entire house to fail. In part, building codes are intended to make such events unlikely.

We are still pondering some questions: Would the software industry be better off if it adopted some practices from the construction industry, such as building codes, inspectors, and fixed-price contracts? Or do these practices stem from the fact that it's much easier to test software and correct problems after it has been written than it is to rebuild a house that has problems? Is it possible for end users to get a more accurate idea of how their systems will behave before those systems are actually built? What about a more accurate idea of what they will cost? Why don't programmers buy their own tools and build the cost into what they charge? It is still difficult to build software systems out of components. Why? And, might it make sense for programmers, or groups of programmers, to specialize in particular kinds of subproblems?

This project has given us a new appreciation for how much complexity goes into building even something as small as a single-family house, and how much of the effort involved comes from dealing with that complexity. In that sense, houses and software systems are very much alike.

Notes

[1] Alexander, Christopher. A Pattern Language, Oxford University Press, 1977; and The Timeless Way of Building, Oxford University Press, 1979.

[2] More precisely, our builders removed the old door, reworked the framing in that part of the wall, and installed a new door that has much better insulation, but you know what we mean.