Solid research supporting the value of agility is beginning to come in, but the tough task of convincing the Doubting Thomases to "cross the chasm" may take further effort.

It can be difficult to convince senior management that agile approaches are effective: Some people are honestly skeptical because they're afraid that agile software development is yet another fad, whereas others are happy with the status quo or fear change, demanding proof that agile techniques really work. Well, Thomas, doubt no more: Proof is starting to emerge.

The "Technology Adoption Curve" presents critical insight into the proof question. First described by Geoffrey Moore in his groundbreaking book Crossing the Chasm (HarperBusiness, Revised Edition, 2002), the curve distinguishes among five types of technology adopters:

1. Innovators who pursue new concepts aggressively.
2. Early adopters who pursue new concepts very early in their lifecycle.
3. The early majority who wait and see before buying into a new concept.
4. The late majority who are concerned about their ability to handle a new concept, should they adopt it.
5. Laggards who simply don't want anything to do with new approaches.

Facts from the Past

Agility's been around long enough now that a significant amount of proof is emerging. Craig Larman, in his new book Agile and Iterative Development: A Manager's Guide (Addison-Wesley, 2003), summarizes a vast array of writings pertaining to both iterative and incremental (I&I) development, two of agility's most crucial tenets, noting the positive I&I experiences of software thought leaders (including Harlan Mills, Barry Boehm, Tom Gilb, Tom DeMarco, Ed Yourdon, Fred Brooks and James Martin). More importantly, he discusses extensive studies that examine the success factors of software development. For example, he quotes a 2003 study conducted by Allen MacCormack and colleagues, to be published in IEEE Software, that looked at a collection of project teams of a median size of nine developers and 14 months' duration. Seventy-five percent of the project teams took an iterative and incremental approach, and 25 percent used the waterfall method. The study found that releasing an iteration's result earlier in the lifecycle seems to contribute to a lower defect rate and higher productivity, and also revealed a weak relationship between the completeness of a detailed design specification and a lower defect rate. Larman also cites a 2003 Australian

study of agile methods, in which 88 percent of organizations found improved productivity, 84 percent experienced improved quality, 46 percent had no change to the cost of development, and 49 percent lowered costs. He also cites evidence that serial approaches to development, larger projects and longer release cycles lead to a greater incidence of project failure. A 2001 British study of 1,027 projects, for example, revealed that scope management related to waterfall practices, including detailed design up-front, was the single largest factor contributing to failure, cited by 82 percent of project teams.

Barry Boehm and Richard Turner also explore the viability of agile approaches in Balancing Agility and Discipline: A Guide for the Perplexed (Addison-Wesley, 2003). They compare agile approaches with traditional approaches, which they call plan-driven approaches (a term I'm not fond of because planning does occur on agile projects), providing a risk-based approach to determine when to

use each. Boehm and Turner's book reflects a reasonable, open-minded look at agile techniques by the industry's "old guard." "Agile methods," they write, "handle changeability and invisibility by building a shared vision of the project's objectives and strategies into each team member's shared store of tacit knowledge."

Not TMI Yet

Despite these encouraging offerings, the existing research isn't completely convincing. First, rather than specifically addressing agile approaches, much of it compares traditional waterfall approaches with ad hoc development. Second, although the two books mentioned effectively summarize the existing research, much about agile techniques remains to be examined. Agility's foundation is clearly solid, but we still must determine whether the agile methodologists have built something that will stand the test of time.

Fortunately, many researchers are busy exploring agility's effectiveness. At this summer's XP/Agile Conference in New Orleans, a research workshop was organized by Grigori Melnik and Adam Geras of the University of Calgary and Laurie Williams of North Carolina State University to analyze the current state of ongoing empirical research into agile methods.

In the workshop, participants reviewed and discussed three research projects, all of which are described in detail on the workshop website. Bill Krebs of North Carolina State University and IBM described his findings exploring the adoption of Extreme Programming (XP) by a development team. The most interesting part of his research was the suite of simple metrics that he used to judge the effectiveness of agile software development teams, such as unit test classes per user story, pairing frequency and user stories per person-month.

Brian Hanks of University of California, Santa Cruz, discussed his study on the effectiveness of pair programming, in which he compared two classes of students taking the same course in different terms. One class was taught how to pair program and then asked to do so; the other wasn't. Although it's always hard to extrapolate student-based research to the commercial world, the results indicated that the pair-programming students learned the material better, supporting the claim that pair programming spreads knowledge across a team more effectively.

Hakan Erdogmus of the National Research Council Canada described the work that he did with Maurizio Morisio, Marco Torchiano and Andrea Capiluppi of Politecnico di Torino in Italy comparing test-first and "test-last" development. It found that the test-first group, once again students, was 70 percent more productive than the group following a traditional approach. The test-first group also wrote 60 percent more tests than the test-last group, implying that the test-first mindset instilled greater discipline and thus led to greater productivity. Interestingly, on the final exam, the students reverted to a test-last approach, indicating that the culture shift required to adopt agile approaches is longer than a single school term.

The anecdotal evidence certainly supports agile techniques, as does the foundational evidence gathered since the 1950s. W. H. Dana's 1993 study of NASA's X-15 hypersonic fighter project cited incremental delivery as a primary success factor ("The X-15 Lessons Learned," NASA Drysden Research Facility). It will take several years, perhaps even a decade, until we have incontrovertible proof that agile software development techniques work in practice. The research community clearly has its work cut out for it, and I hope that the IT's old guard allows and encourages these new lines of research. A handful of universities are leading the way; more will follow.

—Scott W. Ambler

This article was originally published in the November 2003 issue of Software Development.

>>HOT LINKS

For more information about determining when a model is agile, visit http://www.agilemodeling.com/essays/whenIsAModelAgile.htm

For more information about determining when a document is agile, visit http://www.agilemodeling.com/essays/agileDocumentation.htm#WhenIsADocumentAgile

To learn more about agile documentation, visit http://www.agilemodeling.com/essays/agileDocumentation.htm

For overviews of a wide range of models, including all 13 UML 2 diagrams, visit the Agile Models Distilled page at http://www.agilemodeling.com/artifacts

For tips on software modeling on whiteboards, read the essay posted at http://www.agilemodeling.com/essays/whiteBoardModeling.htm

The Agile Alliance homepage is the best starting point for anyone interested in learning more about agile software development. http://www.agilealliance.org

The principles of Agile Modeling v2 are described at http://www.agilemodeling.com/principles.htm

The practices of Agile Modeling v2 are described at http://www.agilemodeling.com/practices.htm

Check out the Agile Modeling mailing list at http://www.agilemodeling.com/feedback.htm

Get agile modeling training resources at http://www.agilemodeling.com/training.htm

Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.