Testing During a Construction Iteration
The majority of testing occurs during construction iterations on agile projects. Your testing effort, just like your system, evolves throughout construction. Figure 2 depicts two construction iterations, indicating that there is confirmatory testing performed by the team, and in parallel, investigative testing efforts ideally performed by a independent test team (I've adopted the terms "confirmatory" and "investigative" testing from Michael Bolton, a thought leader within the testing community). Although it isn't always possible to have an independent test team, particularly for small projects, it is highly desirable. Confirmatory testing focuses on verifying that the system fulfills the intent of the stakeholders as described to the team to date, whereas investigative testing strives to discover problems that the development team didn't consider.
There are two aspects to confirmatory testing: agile acceptance testing and developer testing, both of which are automated to enable continuous regression testing throughout the lifecycle. Confirmatory testing is the agile equivalent of testing to the specification, and in fact, we consider acceptance tests to be the primary part of the requirements specification and our developer tests to be the primary part of the design specification. Both of these concepts are applications of the agile practice of single sourcing information whenever possible.
Agile acceptance testing is a mix of traditional functional testing and traditional acceptance testing because the development team and their stakeholders are doing it collaboratively. Developer testing is a mix of traditional unit testing and traditional class/component/service integration testing. Developer testing strives to verify both the application code and the database schema (for more information, see my article "Ensuring Database Quality"; www.ddj.com/architect). Your goal is to look for coding errors, perform at least coverage if not full path testing, and to ensure that the system meets the current intent of its stakeholders. Developer testing is often done in a test-first manner, where a single test is written and then sufficient production code is written to fulfill that test (see www.agiledata.org/essays/ tdd.html for details). Interestingly, this test-first approach is considered a detailed design activity first and a testing activity second.
Automation is an important aspect of construction testing due to the increased need for regression testing on evolutionary projects. The Fitnesse testing framework (www.fitnesse.org), arguably a requirements documentation tool, is often used to automate agile acceptance tests. It is also possible to generate acceptance test cases from use cases and scenario definitions or from process diagrams such as UML activity diagrams or flow charts, and tools are beginning to emerge to do exactly this. The XUnit framework—JUnit (www.junit.org) for Java and VBUnit (www.vbunit.org) for Visual Basic—is used to automate developer tests. Commercial testing tools such as HP Mercury's TestDirector (www.mercury.com/us/products/quality-center/testdirector) or IBM Rational's TestManager (www-128.ibm.com/developerworks/rational/ products/testmanager) are also good options to consider, as they often prove more sophisticated than their open source alternatives. Static code analysis tools, such as FindBugs (findbugs.sourceforge.net), are often included in automated testing runs to help identify potential quality problems in the source code.
