How to Adjust to the Changing Face of Software Testing
Beyond testing scripts and automating everything, a new approach to software testing is gaining traction in larger organizations. Proponents including Barclays, the world's fourth largest bank. Should your team listen?
Thu, March 14, 2013
CIO — Keith Klain is the head of the Global Test Center for corporate and investment banking and wealth management at Barclays Bank. He manages hundreds of software testers in United States, Europe and the Asia-Pacific region.
Most large organizations that structure testing as a centralized function use something Klain calls "factory methods" to manage the work. Life in the test factory is an assembly line. A small group plans the work and a larger group executes these "test cases," adhering to detailed step-by-step directions.
What many people see as a strength of the factory is its precise repeatability. Klain and his team see that as a weakness.
Humans following step-by-step scripts tend to ignore everything off-script, creating a sort of inattentional blindness. They lose what is known in chess as the ability to "see the whole board," then adjust to the situation in the moment.
In order to get to "precise" repeatability, some companies insist the scripts be followed exactly the same way each time. This eliminates the testers' ability to react, learn and change approach. It's the kind of thing that happens in chess every time an opponent makes an unexpected move.
One alternative to detailed direction is to have the person doing the work drive it—that is, to design, execute and report test results while learning and adapting. That's something Cem Kaner called "exploratory testing" in the first edition of his book, Testing Computer Software.
Klain and his team hold it up as an example. It's not the only way to test, but, perhaps, it's a place to start.
The Need for Software Testing Change at Barclays
Klain says the factory model that forms the basis of traditional testing is breaking down and cannot meet the needs of competitive companies. "Over the last 15 years or so, software testing has frequently been prioritized to adopt outsourcing and offshoring extensively, and the financial models used to justify that decision are leveling out due to rising wages, cost of living increases and currency fluctuations," he explains.
"Most of the improvement models used to rationalize the commoditized testing approach use strictly quantitative metrics to assess quality or measure improvement, an approach which breaks down rather quickly beyond any first-order metrics," says Klain. "There is an increased focus on business value and testing skills, which means you have to bring more to the table than just the ability to do it cheaper."