Recent experience is showing us that investing in an ICT initiative is one of the highest risk activities the public sector could be involved with. It is surely the case when headlines like ‘millions wasted’, ‘years late’, and ‘minister resigns’ becomes the public’s corporate memory of a complex ICT project. So why does the public sector keep looking for ‘silver bullet’ ICT solutions when the available evidence shows continued under-performance, under-scoping and under-estimation of complexity and risk?
The answer may lie in the way we humans think when we are faced with risk, adversity, ambiguity and complexity. In those circumstances we tend to rely on hope, more than evidence. And if we are senior and powerful, we also tend to believe that somehow, through sheer force of will, we can change the risk environment to suit our wishes.
The former US Secretary of Defense, Donald Rumsfeld, identified this phenomenon in one of his typically cryptic quotes: “…because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns — the ones we don’t know we don’t know.”
My experience from reviews of major ICT projects in Victoria over the last few years indicate that senior officers rely on project managers to tell them what they need to know, while project managers tend to focus on what they actually know. But no one is looking for the unknown unknown (or ‘black swan’) issue that may bring down the whole investment.
Complexity of citizen service need is translating into ICT complexity
A further part of the problem is the inherent complexity of the citizen service issues that ICT projects are trying to address. Many large and complex public sector issues are often being transmuted into a software or system outcome, well before the organisational or policy (that is, people not machine) matters have been effectively understood and resolved. A few classic examples:
E-health (development of a cross-jurisdiction and holistic health care model).
Computers in schools/education (ostensibly to improve literacy and numeracy).
Smartcard transport ticketing (to reduce fare evasion and collect trip data).
E-justice and policing (to improve crime detection resolution).
Smart (electricity) meters (to reduce energy spikes through live price and usage monitoring).
My observation is that it’s not uncommon for complex and expensive initiatives like these to be announced via a ministerial press release.
Typically, the press release defines a project cost and delivery date, well before a detailed business case, specification or schedule has been developed… So is it surprising that many of these ICT projects don’t go so well?
Strategic ICT reviews confirm common trends
A recent better practice guide published by the Victorian Auditor’s-General’s Office, Investing Smarter in Public Sector ICT, confirms several common themes arising from major ICT investments, such as:
The relationships between complexity and risk aren’t well understood (or if they are, they aren’t being explicitly documented for decision makers).
Under-scoping is leaving agencies na?ve and more susceptible to ‘vendor-push’ and ‘vapour-ware’ offerings.
Procurement choices often don’t reflect the real ‘native’ risks that surround ICT projects (the default proposition for the public sector buyer is often a hard-nosed, fixed-cost tender-based contract, rather than risk sharing, relationship-based contracting, such as an Alliance or PPP — which are now being effectively used in the construction industry for high risk projects).
There is a continued and real gap between technology rhetoric and service-delivery reality — Star Trek is still science fiction.
Purported benefits arising from ICT investments aren’t being measured, and if they are, they don’t stack up.
The better practice guide used a meta-analysis approach by taking the key observations and findings from 10 recent ICT performance audits and examining them for incidence of common risk and problem themes. The themes were then matched against industry and government best practice to give useful advice to public sector practitioners.
The way forward
One of the easiest ways to avoid public sector ICT pitfalls is to actively and consciously use evidence and better practice to support improved outcomes in your project. It involves a few practical steps:
Read and reflect on known issues so as not to repeat errors.
Actively watch-out for ‘optimism bias’, as it is heavily over-represented in ICT projects.
During the planning phase, get a third party review of your business case and key financial models or forecasts, particularly sensitive items such as likely delivery of benefits. You should also use similar project peer review to reduce ‘optimism bias’ (refer to Professor Bent Flyvbjerg’s writings, particularly his book Megaprojects and Risk).
Engage a probity reviewer and use them for stage/gate sign offs during the transaction phase.
During the delivery and commissioning phases, conduct ongoing project-level QA/progress reviews and adopt a gateway review approach.
Chasing the ICT Holy Grail
Embedding citizen-focussed benefits and outcomes into techno-centric solutions and methodologies is a key challenge for public sector technologists. Tips:
Stay focussed on the citizen service outcome and experience, not just the coolness of the app, the colour of the box, or the spin from the vendor.
Remember you’re spending other people’s money — would you make the same choices if it was your cash?
Technical methodologies fundamentally engineer out the project team’s receptiveness to ‘black swan’ issues — keep asking the ‘What if?’ questions.
No matter what the contract says, if it doesn’t work out, the Minister will wear the community’s outrage (and will then come looking for you!)
Paul O’Connor is a lecturer and doctoral candidate in the School of Property, Construction, and Project Management at RMIT University. He is on sabbatical leave from his position as sector director (performance audit), at the Victorian Auditor-General’s Office, where he has led the Office’s key ICT and major project reviews. Paul also has wide experience in Commonwealth and Northern Territory agencies.