Talking testing

In my June column Bringing testing on board I described how Geoff Thompson of the testing consultancy Experimentus and I went back to basics and asked: “What is the essential purpose of (software and systems) testing?”

We concluded that purpose of testing was confidence in outcomes. The board wants to have confidence that, when it invests significant sums of money in new software and systems capabilities, intended outcomes will be delivered, on time and to budget.

So I was delighted at a recent testing conference in Seoul when Jan Jaap Cannegieter of the Dutch quality assurance consultancy SysQA urged his audience to get out of the cave.

He was reporting on the outcomes of an exercise in which he had assessed the positioning and effectiveness of the test processes of 25 companies.

The results were clear — relatively high scores in test design and execution and in setting the test environment, but relatively low scores in setting test policy and strategy and in test planning.

Jan Jaap argued that this demonstrated a bias towards the testing techie’s heartland (doing the technical task as efficiently as possible) and away from addressing the wider issues of operational and strategic context (identifying the correct things to do).

In the words of the late Peter Drucker: “There is nothing so useless as doing efficiently that which should not be done at all.”

These wider issues of context are where dialogue with the board is vital.

So Jan Jaap’s analysis of his fellow testing professionals is we need to get out of the cave by developing constructive approaches to wider stakeholder involvement and management.

He went on to demonstrate his main thesis in a most interesting way.

He asked the audience of over a hundred experienced testing professionals at the TMMi Seoul International Conference to stand and then, starting at the very techie end of the spectrum of activities that the Test Maturity Model integration (TMMi) framework identifies, asked those that could honestly say that they practiced that element of the framework to remain standing.

At the techie end of the framework (such as test design and execution), the vast majority remained on their feet. By the time Jan Jaap reached the strategic end of the framework, hardly anyone was standing.

But once out of the cave, what should be the basis of this board dialogue? This comes back to an issue that I find vexes the upper echelons of IT professionals, and leads to the heart-warming vision of CIO becoming CEO.

The tendency is to believe that the solution lies in boards that are technically literate and can match techie speak with the best, but that is not the purpose of boards.

Boards are about business, about operational strategy and about business governance.

I am reminded of an experience in the late 1970s when I was responsible for procurement of the European fuel requirements for the former global chemical major ICI.

One strategic project I pursued was the conversion of a major ICI mid-Cheshire power station from burning fuel oil to burning coal — a conversion vital to the competitive survival of the business concerned.

The business was not strategic to ICI, so I concocted an off-balance-sheet deal, at the heart of which was a contractual guarantee from the (then nationalised) UK Coal Board that the delivered price of coal would never rise above 70 per cent of the competing fuel oil prices, unless the price of crude oil fell below $15 a barrel.

All looked to be in place for the main board of ICI to sanction the £3.5m capital project. Then came a call to attend the office of one of the main board directors.

“What will happen, young man, if the price of oil does fall below $15?,” he asked. A solution had to be found — and it was, in the form of a £350,000 insurance policy placed on the Lloyds Contingency Markets — “Well done, young man!”

In my June column I declared my interest as chair of the independent TMMi Foundation.

The TMMi framework provides a structured means of measuring maturity levels that scope the effectiveness as well as the efficiency of the testing process.

What Jan Jaap’s presentation and active interaction with his audience demonstrated is the need to build from the TMMi framework the basis for genuine board-level dialogue at the business’s strategic and operational level that will allow testing to deliver its real professional objective: the assurance of agreed outcomes.