In her comprehensive SD West presentation, “Effective Metrics for Pragmatic Project Managers,” project-management maven Johanna Rothman got to the meat of the matter in the first few minutes: Confronting her audience with the question, “What things are important about your project?” she proceeded to tackle all the enumerated ducks in a row, provoking a lively interchange with her inquisitive audience.
Noting the comment attributed to Hewlett Packard’s Stephan Leschka, that “Without metrics, you’re just another person with a different opinion,” Rothman stated, “The more measurements we take at the beginning, the more likely we are to have a fabulous result at the end.”
The best chance for success, Rothman maintained, depends on a collaborative data-gathering method that quantifies project risks and risk-handling methods. Successful project management requires far more than simple task analysis, Rothman warned. PM courses tend to focus on task lists, she explained, but the critical paths in software projects go beyond tasks to people and sometimes machines.
Focusing on the human element, Rothman stressed the necessity of first determining who’s playing schedule chicken. “The first fellow says he’s going to make Friday, so the next one says Friday, and the next one, too, but what they don’t know is that everyone’s sweating, knowing that Friday’s an impossible dream,” she explained, to the audience’s knowing laughter.
The way to avoid schedule strangulation? Throughout the class, Rothman reiterated one concept: Early intervention. “The farther along you go in the project, the more likely you are to play chicken, and the less likely you are to make helpful changes.”
Rothman then discussed the necessity of developers employing uncertainty to their advantage. “In answer to the dreaded question, 'When will it be done,’ you don’t want to say 'I don’t know.’ But there are times in the project when you really don’t know the entire state of the project.”
Rothman described essential project management awareness as a pair of parallels:
What’s done and what’s left to do; what’s working and what’s
not working. To elucidate this, she stressed the necessity of identifying defects
and unearthing obstacles at the earliest possible time. Breaking bad news early,
she advised, tends to soften the blow.
Major Players
Next enumerating measurements for project completion, Rothman ticked off the
major players in the PM equation:
Earned value, she explained, is especially difficult to determine in software projects, where parts are interdependent. “If you can’t tell when a part is complete,” she explained, “you can’t measure earned value.” Thus, the necessity for breaking up a project into usable chunks at the very beginning, not halfway through, when the entire feature set is “smushed together.” If a customer can use the first chunk, she explained, project managers can calculate an earned value for that deliverable.
“Ask yourself,” she exhorted the audience, “what’s the earliest possible date you can get something done. Your best bet is to make the smallest chunks possible and then measure earned value: ‘How much did it cost me to get to this chunk?’” For Extreme Programming projects, she stated, iterative development can play into the chunking technique, because at the end of each iteration, developers are providing something of value, and need ask only if the estimate met their actuals.
EQF Explained
To evaluate to estimation metrics, Rothman recommended efficiency expert Tom
DeMarco’s Estimation Quality Factor process, which provides a histogram
of the most significant duality in project management: the date of estimate
versus estimated end date.
EQF is useful in three ways, Rothman stated: First, as an early warning sign to see if events outside your project are consuming people when they should be focused on your project; second, as a check against the initial estimations on your next project; and third, as a means of determining if you have a chance of completing the current project.
Conspicuous Consumption and Corporate Overload
When the project starts to slip, Rothman recommended, ask team members to log
their time for a week and let you know—but don’t show—the
results. One team member, she recalled, was a coffee addict, logging in six
to eight time-consuming coffee breaks per day. She had no need to admonish him—“I’m
going to tell another adult he can’t drink coffee?” she laughed—self-monitoring
kick-started him into a caffeine diet.
Rothman then attacked corporate overload dictating multiple parallel projects, noting that most developers just can’t handle context shifting among several projects at different levels in a single day. This problem can best be dealt with by negotiation to arrive at a reasonable portfolio management. “Go to the other project manager and say, ‘We can’t do this; give me Sally for a week,’” she suggested.
Changing Course Midstream
Rothman then recommended measuring the number of requirements early on in the
project, as well as enumerating the number of major and minor changes per week
over the course of the project, warning about the disaster potential of midstream
major requirements changes.
For midstream project trends, defect find and close rates rise to the fore. “I track my own defects on everything,” she explained, “when we review requirements specs and in project plans, all on the same chart, so the developers are much more likely to track defects in the products they create, too.”
Tracking defect trends can help project managers keep control of the process,
enumerating how long it takes to fix defects, as well as their costs. In what
seemed like a conundrum, she used example charts from projects completed to
detail the truism that the more proactive you are, the higher your costs will
be later in the project. “The fewer defects you find later,” she
stated, “the higher the cost of each defect, but the lower the overall
total cost.”
Rothman then explained the Fault Feedback Ratio, examining the total picture
of productivity, asking the Big Questions “What do the creators create?
How much is good stuff, and how much is redo?” FFR, she stated, tells
you the ratio of problem solving to wheel spinning.
To fix a high FFR, Rothman advised, allow nothing to go into the configuration
management system until it’s been reviewed by more than one person. “Pair
programming works great for this,” she recommended.
Better Late than Never
As the project winds down, it’s not too late to save your bacon: Defect
trend analysis can still be useful if you ask the right questions. Determining
the number of defects that remain open and making sure developers aren’t
getting stuck fixing the easiest problems first can go a long way in salvaging
a project that seems headed for the barrel room.
For diagnostic techniques, Rothman advised a mixed bag. “Maybe I need to use different techniques for the substrate or a piece of the project than for the rest of it,” she explained, again recommending pair programming and increased communication with developers.
Gathering the Goods
For all this data collection, Rothman claimed that simple tools are usually
sufficient. “You need a defect tracking system, a configuration management
system and a project scheduling tool you can extract info from,” she
stated. “I set up scripts, the release engineers help me with the defect
tracking system and config management system, and I check how many open and
closed defects I can find.” In the Unix environment where she often
works, she said that ChronJobs does the trick, but stressed that many tools
are equal to the job, depending on the nature of your project.
Finally, Rothman admitted, beyond numbers, project management is as much art and craft as it is science. No matter what metrics you use, you’re best served by navigating the middle course between unbridled optimism and paralyzing pessimism.
