Last month I described the origins of Lean Production and Total Quality Management, the paradigm shift that revolutionized business in the 1980s, and discussed its application to software development, which I call "Lean Programming" (Project & Process Management, May 2001). Lean Production, which evolved from Taiichi Ohno's efficient system for creating high-quality automobiles (the Toyota Production System), is based on the absolute elimination of waste, in both product and process. However, Ohno's revolutionary contributions can't be separated from W. Edwards Deming's teachings on quality management. While Ohno sought to eliminate waste and improve production in the years following World War II, Deming taught Japanese managers how to empower workers to investigate problems and systematically improve business processes. The movement, Total Quality Management (TQM), emphasizes a culture of continuous improvement of both process and product.
The basic practices of Deming's TQM movement and Ohno's Lean Production can be summed up in these 10 points:
- Eliminate waste.
- Minimize inventory.
- Maximize flow.
- Pull from demand.
- Meet customer requirements.
- Do it right the first time.
- Empower workers.
- Ban local optimization.
- Partner with suppliers.
- Create a culture of continuous improvement
In part one, I discussed points one through five. The final five rules of TQM and Lean Production follow.
Lean Rule #6: Do It Right the First Time (Incorporate Feedback)
Before Lean Production arrived at our videocassette manufacturing plant in the early 1980s, we occasionally had output of marginal quality. To identify and correct this, we tested to find good products and reworked bad products. After implementing the "Do It Right the First Time" rule, we closed down rework stations and stopped trying to test quality into the product after the fact. Instead, we made sure that each component was good at every handoff by employing tests and controls throughout the manufacturing process. In this way, we could detect when a product was drifting away from the specifications and stop production before any bad products were made.
"Do It Right the First Time" does not mean "Freeze the Spec." On the contrary, product (and software project) specifications change constantly. Lean discipline demands instantaneous adaptation to changing market conditions, which is best effected with a flexible product architecture that readily accommodates manufacturing change, techniques for monitoring that detect errors before they occur and tests that are designed before manufacturing begins.
In his "Industrial Software Metrics Top 10 List" published in the September 1987 issue of IEEE Software, Barry Boehm notes that it costs 100 times more to find and fix a problem after software delivery than in the early stages of design. This observation and the "Do It Right the First Time" rule have been widely used to justify the decision to develop a detailed system design before code is written.
The problem with this approach lies in the assumption that customer requirements are static and can be defined by a predetermined system. Because requirements do change—and frequently—throughout the life of most systems, they cannot be adequately fulfilled by a rigid design. "Do It Right" has also been misinterpreted as "don't allow changes." In fact, once we acknowledge that change is a fundamental customer requirement, it becomes clear that "Doing It Right" requires that we provide for change at all stages of a project's life cycle.
If we acknowledge that customers may not know what they want at the beginning of development and that their needs might change midstream, we must incorporate a method of obtaining customer feedback during development. Instead, most software development practices include a complex "change control process" that discourages developers from responding to user feedback. Far from ensuring a quality result, these change-resistant processes actually get in the way of "Doing It Right."
Lean Programming employs two key techniques that make change easy. Just as Lean Production builds tests into the manufacturing process to detect when the process is broken, Lean Programming builds tests into the beginning of the development process. As programming proceeds and changes are made, the unit and regression tests are run. If the tests don't pass, programming is stopped until the problem is found and corrected. A comprehensive testing capability is the best way to accommodate change throughout the development process.
The second technique that facilitates change is refactoring, or improving the design of existing software in a controlled and rapid manner. With refactoring, initial designs can focus on the basic issue at hand rather than speculate about other features that may be needed in the future. Later in the process, refactoring techniques can incorporate these additional features as they are required, making it easy to accommodate the future if and when it becomes the present.
Lean Rule #7: Empower Workers
A basic principle of Lean Production is to drive decisions down to the lowest possible level, delegating decision-making tools and authority to the people "on the floor." As Paul Adler noted in his article, "Time-and-Motion Regained" (Harvard Business Review, Jan.- Feb. 1993), when Toyota took over GM's manufacturing plant in Fremont, California, in 1983, it inherited workers with the worst productivity and absenteeism record in the industry. Within two years, those same workers, organized into teams trained in work measurement and improvement techniques, had doubled their quality and productivity scores.
Often when software development environments under-perform, the instinctive reaction is to impose more rigid processes, specifying in greater detail how people should do their jobs. Lean Production principles suggest exactly the opposite approach. When there are problems in manufacturing, a team of outside experts is not sent in to document in more detail how the process should be run. Instead, people on the manufacturing floor are given tools to evaluate and improve their own areas. They work in collaborative teams with the charter to improve their own processes and the links to nearby processes for which they are suppliers or customers. Their supervisors are trained in methods of forming and encouraging work teams to solve their own problems.
Lean Programming similarly prizes people and collaborating teams over paperwork and processes. It focuses on methods of forming and encouraging teams to address and resolve their own problems, recognizing that the details must be determined by the people doing the work.
Software development involves the handoff of information at least once (from user to programmer) and often more than once (from user to designer to programmer). One school of thought holds that it's best to transfer all such information in writing, but in fact, a great amount of tacit knowledge is lost by handing off information on paper. It's far more effective to have small collaborating teams work across the boundaries of an information handoff, minimizing paperwork and maximizing communication.
Lean Rule #8: Ban Local Optimization
In the 1980s, the worst enemy of Lean Production was often the accounting department. We had big, expensive machines in our plant, and the idea that they should not be run at full capacity was radical, to put it mildly. The accountants didn't want to abandon our daily work-in-process (WIP) inventory reports just because there was virtually no WIP to report.
A generation of accountants had to retire before it became acceptable to run machines below full capacity, and designing machines for rapid changeover rather than highest throughput remains a tough sell even today. After 20 years, Lean Production is still counterintuitive to those who lack a broad business view.
In this context, let's examine the role of managing scope in a software development project. Project managers have been trained to focus on managing scope, just as we in manufacturing concentrated on maximizing machine productivity. However, Lean Programming is fundamentally driven by time and feedback. In the same way that localized productivity optimization weakens the overall manufacturing process, so focus on managing scope impairs project development.
Think about it—holding the scope to exactly what was envisioned at the beginning of a project offers little value to the user whose world is changing. In fact, it imparts anxiety and paralyzes decision-making, ensuring only that the final system will be outdated by the time it's delivered. Managing to a scope that's no longer valid wastes time and space, necessitating inefficient issue lists, extensive trade-off negotiations and multiple system fixes. However, as long as limiting a project to its original scope is a key project management goal, local optimization will flourish—at the expense of the project's overall quality.
Lean thinking dictates that scope will take care of itself if the domain is well understood and there is a well-crafted, high-level agreement on the system's function in the domain. Scope will take care of itself if the project is driven in time buckets that aren't allowed to slip. Scope will take care of itself if both parties focus on rapid development and problem-solving, and adopt waste-free methods of achieving these goals.
Lean Rule #9: Use Evolutionary Procurement (Partner with Suppliers)
Lean Production didn't remain within the confines of the manufacturing plant. Once the idea of partnering with suppliers was combined with an understanding of the value of rapid product flow, Supply Chain Management was born. People began to realize that it took tons of paper to move material between companies. Moreover, the paperwork was more expensive than one might expect, as was the ensuing delay in product flow. Today, by cutting the cost of business-to-business transactions, Web portals have generated billions of dollars in savings.
Supply Chain Management made companies take a close look at their business-to-business contracts. All too often, they focused on keeping the companies from cheating each other. In addition, it was common to pit one vendor against another to ensure supply and to obtain the lowest cost. Again, Lean Production changed this paradigm. Deming taught that trust-based relationships with single suppliers create an environment that best benefits both companies.
Throughout the 1980s, companies revitalized their supply chains by reducing the number of suppliers they used and working as partners with those who remained. The high quality and creativity of these supply chain partnerships far outweighed the putative benefits of competitive bidding and rapid supplier turnover. Partner companies helped each other improve product designs and product flows, linking systems to allow just-in-time movement of goods across several suppliers with little or no paperwork. The advantages of this collaborative supply-chain relationship are lasting and well documented.
Wise organizations realize that traditional software development contract practices generate hidden wastes. As the manufacturing world revealed in the 1980s, trusted relationships with a limited set of suppliers can yield dramatic advantages. Freed from the adversarial relationship created by a concentration on controlling scope and cost, software development vendors can focus on providing the best possible software for their customers, stabilizing requirements as late as possible in the development process to provide the greatest value for the available funds.
Lean Rule #10: Create a Culture of Continuous Improvement
When software development seems to be out of control, organizations often hasten to increase their level of "software maturity" with awards and certifications. This might seem to be in line with good manufacturing practice, in which ISO 9000 certification and Malcolm Baldridge awards are sometimes equated with excellence. However, these process-documentation programs indicate excellence only when the documented process excels in the context of its use.
In many software development projects today, excellence means the ability to adapt to fast-moving, rapidly changing environments. Process-intensive approaches such as the higher levels of Software Engineering Institute's (SEI) Capability Maturity Model (CMM) may lack the flexibility to respond rapidly to change. In a recent e-mail advisory (E-Projects in India, Cutter Consortium's e-Project Management Advisory Service, March 1, 2001), Jim Highsmith highlights the tension between such heavyweight methodologies and the light- weight development models inspired by Lean Production.
One suspects that process-documentation certification programs may stifle, rather than foster, a culture of continuous improvement. Deming would probably turn over in his grave at the tomes of written processes substituting for his simple Plan-Do-Check-Act approach:
- Plan: Choose a problem. Analyze it to find a probable cause.
- Do: Run an experiment to investigate the probable cause.
- Check: Analyze the data from the experiment to validate the cause.
- Act: Refine and standardize based on the results.
Iterative development can effectively employ the Plan-Do-Check-Act method. During the first iteration, the handoff from design to programming or programming to testing may be a bit rough. It's OK if the first iteration provides a learning experience for the project team, because the subsequent iterations will allow the team to improve its process. In a sense, an iterative project environment becomes an operational environment, because processes are repeated and Deming's techniques of process improvement can be applied from one iteration to the next.
Product improvement is also enhanced in the iterative process, particularly if refactoring is used. In fact, refactoring provides a tremendous vehicle to apply the principle of continuous improvement to software development.
However, we need an improvement model that can span more than a single project. We must improve future project performance by learning from existing ones. Here again, Lean Production can point the way. During the 1980s, the set of practices summarized in the 10 rules of Lean Production were adopted widely across most manufacturing plants in the U.S. and Europe. These practices then spread to service organizations, logistics organizations, supply chains and beyond. In these multiple domains, application of Lean Production principles has engendered remarkable and continuing success.
The simple tenets of Lean Production have effected dramatic improvements in a myriad of industries. Applied to software project management as Lean Programming, these practices will lead to the highest quality, lowest cost, shortest lead-time software development possible.