In This Issue
- Calculating Documentation CRUFT
- Hot Links
Calculating Documentation Cruft
In late June I was involved in an interesting discussion on the XP mailing list about measuring the effectiveness of documentation. The conversation began with my observations about the value of detailed requirements specifications and it quickly became apparent that it was an issue with documentation in general. What emerged from the discussion was a tongue-in-cheek formula for calculating the cruftiness of a document. In this newsletter I present this formula, work through a couple of examples, and then more importantly overview some strategies for improving the quality of the documentation produced on software development projects.
According to Wikipedia, cruft is a slang term which refers to areas of something which are badly designed, poorly implemented, or redundant. It is important to note that cruft is based in part on the perception that someone has of an artifact in addition to the actual quality of that artifact. With respect to documentation, cruft occurs when out of date, redundant, out-of-scope, or poorly written information is included.
It's easy to describe documentation cruft, and often easy to identify it once you see it, but it's hard to estimate how "crufty" a document actually is. Furthermore, it's often hard to convince the creators of a document that "their baby" isn't as beatiful as they believe it to be. The following factors, measured as percentages, affect the cruftiness of a document:
- C = The percentage of the document that is currently "correct".
- R = The chance that the document will be read by the intended audience.
- U = The percentage of the document that is actually understood by the intended audience.
- F = The chance that the material contained in document will be followed.
- T = The chance that the document will be trusted.
The CRUFT rating of the document, with 100% being a bad thing, is calculated with the following formula:
100% - C * R * U * F * T
For example, when everything is perfect and all of the factors are 100% then there is 0% cruft in your document. More realistically, let's consider a requirements document written early in the lifecycle following a traditional, reasonably serial lifecycle. Fair values for the CRUFT factors might be C = 65%, R = 90%, U = 80%, F = 100%, and T = 90% in a given situation. Although C appears low, the Standish Group has a good argument that when a big requirements up front (BRUF) approach is taken that on average 45% of the functionality delivered is never used. The implication is that C = 65% represents an above average requirements elicitation effort. Therefore, in this case, it appears that 58% (1 - .65 * .9 * .8 * .9) of the requirements document is cruft.
The following agile documentation strategies should enable you to reduce the cruft in your documentation:
- Write stable, not speculative, concepts. The correctness factor is heavily influenced by changes in your environment. When you document speculative ideas, such as detailed requirements information at the beginning of a project, it is likely to change and hence the correctness of your documentation decreases. Waiting until a concept is stable increases the chance that written documentation will remain correct.
- Prefer executable work products over static ones. Executable specifications, such as customer tests which capture detailed requirements and developer tests which capture detailed design, are far more useful to developers than static documentation. By writing executable specifications you will improve all of the cruft factors. Because the executable specifications offer actual value to developers, they're more likely to work with them and to keep them up to date.
- Document only what people need. A primary reason why documents contain far too much information is because the writers don't know what the readers actually require. The implication is that you must work closely with the target audience of a document to identify how they will use that documentation in practice. Active participation by the audience in the creation of documentation potentially improves all of the cruft factors.
- Take an iterative approach. You can take an iterative approach to writing documentation just like you do for software. This provides opportunities for you to gain and then act on feedback, thereby improving the quality of the documentation.
- Be concise. Brevity increases the chance that the documentation will be read and understood and decreases the overhead of keeping it up-to-date. It may increase the chance of it being followed and even trusted because it is easier to read than verbose documentation.
The term "cruft" has been bandied about within the software development community for a long time. As I've shown here, it appears as if it's fairly easy to estimate the amount of cruft within a document. Worst case at least I've helped to illuminate some of the potential challenges that you face when writing effective documentation. I'd like to acknowledge Steve Gordon, Tim Ottinger, and Joe Rainsberger for the ideas that they posted on the XP list which improved upon my original posting.
Hot Links
Here's a discussion of Agile Documentation strategies.
Documentation is the worst way to communicate, yet effective communication is critical to your success on software development projects. Here's an overview of the issues surrounding communication.
The article Examining the 'Big Requirements Up Front (BRUF) Approach' overviews and discusses evidence as to why comprehensive modeling up front puts your project at risk.
You can explore architectural issues on software development projects without having to write extraneous documentation. Here's a description of initial architectural modeling strategies.
The Agile Alliance homepage is the best starting point for anyone interested in learning more about agile software development.
The Agile Models Distilled page provides links to overviews of a wide variety of models.
A discussion of the principles of Agile Modeling v2.
A discussion of the practices of Agile Modeling v2.
Check out the Agile Modeling mailing list.
