Rethinking Software Development, Testing and Inspection
Michael Fagan first found software inspections yielded a massive productivity improvement when working for IBM in the early 1970s. Now, 35 years later, we need to ask certain key questions: Are the dynamics of software development the same, are Fagan's findings still relevant in today's world of agile development and "extreme programming"? And, if so, what should your organization do about it?
Wed, February 01, 2012
Extreme Programming, or XP, brought many insights into software development, but at least two are important for purposes of code inspections. First of all, the idea of pair programming implies continuous peer code review, while Test Driven Development, or TDD, prevents the sort of compile-time, off-by-one, index and pointer errors that inspections were designed to address.
The second insight extreme programming brings involves the "cost of finding a defect late."
The historic interpretation is that the cost of fixing a defect grows exponentially with each phase. That is, a defect found and fixed in production might cost hundreds, if not thousands, times more than if it were prevented before the requirements were "signed off." One classic image for this is this one, based on the work of Barry Boehm in his book "Software Engineering Economics" often referenced by Steve McConnell and others *
The graph above illustrates a general principlesthat the costs of fixing a defect go up over time. It is important to note that this graph is an illustration. There is no numbered Y axis, and your actual results may vary. The numbers are based on averages and assume a bell-curve distribution, but that distribution may not be accurate. The cost of fix curve can look very different for different categories of bugs, something that Alan Page, the lead author of "How We Test Software at Microsoft" pointed out in a conference paper in 2011.
The image also adds an interpretation: that price goes up by phase. An alternative interpretation is that cost goes up over time. Remember: When Boehm did his this research, nearly all projects followed a waterfall model, so linear time and phase of the project were the same thing.
Today, an Agile Software team will move from requirements to design to code and test in a matter of days, possibly hours or minutes. That means for two identical projects, one with a two-week iteration and the other organized as a six-month waterfall, the agile project can find the defect in a week or two, before the review would even occur in the waterfall project, keeping the costs low.
At the same time, the internet was rapidly reducing the replacement cost of new software. Suddenly, instead of having to ship CD's and physical floppy disks in boxes to customers, we could deploy to one server or server farm.
All this comes together to mean that as the value of inspections has decreased, the very risks they were designed to mitigate have also decreased, or may be addressed by other means, such as TDD, Pair Programming, or rapid exploratory testing.