Novopay and Queensland Health: One-offs or symptomatic of a deeper problem in the IT industry?

Generally speaking the longer a project the more likely it is to run into problems. More so on large-scale programmes with major technology components and requirements that undergo significant change once work begins.

Why? People come and go during implementations and technology changes just as rapidly. In some cases, projects are simply too big and best practice is not always followed.

For the past 20 years, I have worked on the legal aspects of some of these projects – on either the customer or supplier side. Based on that experience, the themes are constant and, if anything, the adverse consequences of failure (such as delay and cost over-runs) have worsened (perhaps reflecting increased scale, complexity and ambition).

Some examples are particularly disturbing. These include US retail chain Kmart, whose problems in the IT area and two other business projects contributed to its decision to file for bankruptcy, as described by Bent Flyvbjerg and Alexander Budzier in an article for the Harvard Business Review in September 2011.

Another is the infamous patient records IT programme in the UK where billions of taxpayer pounds were spent over several years before it was dismantled by the UK government in 2011.

More recent cases hit closer to home – Novopay and Queensland Health (QH) – which were “perfect storms” given the combination of problems experienced and the extreme levels of disruption, delay and wasted expenditure suffered.

Is it time for a radical re-think on the approach to IT procurement, particularly large-scale, complex projects? Michael Bywell, Minter Ellison Rudd Watts

QH had the added feature of inappropriate conduct by both customer and supplier side representatives at the bid stage. (See sidebar next page: A tale of two inquiries)

What is it about IT projects that causes these problems and what is the solution?

Page Break

The short answer is that there is no “silver bullet” and despite several attempts in several countries to capture lessons and publish guidance on best industry practice, the rate of failure remains high.

Indeed an investigation by the Victorian Ombudsmen in Australia in 2011 found that:

“Each of the 10 projects I examined failed to meet expectations; most failed to meet delivery timeframes; and all ran over budget. The original budgets for these projects totalled A$1.3bn. The latest estimated cost is A$2.74 billion– an additional A$1.44 billion”

This data reveals an average cost overrun of 210 percent.

For their part, Flyvbjerg and Budzier in their 2011 research found that one in six projects studied had a cost over-run of 200 percent percent percent and a schedule over-run of 70 percent percent.

But does this high rate of failure mean that the available best practice guidance is inadequate?

The Novopay and QH reports, for instance, contain specific references to several examples of published guidance. These include: Guidelines for Managing and Monitoring Major IT projects, 2002 (New Zealand), Guidance for Monitoring Major Projects and Programmes, 2011 (New Zealand); Getting a Grip: How to Improve Major Project Execution and Control in Government, 2013 (UK) and Managing Successful Projects with PRINCE2 (UK).

Plainly these guidelines were carefully put together by their authors and contain valuable information. So the quality of those documents is not the issue.

The problem area is getting the industry to actually use this material. In Novopay, for example, the authors of the report found that the “The lack of discussion by the Project Board about the [existing published guidance] was a major failing.”

The same may also be true of internal guidelines produced by IT supplier organisations. In reality key individuals involved in major procurements may be more focused on securing new business than published best practice.

The behaviour by IBM staff in QH provides an extreme but informative example of this. In that case, internal guidelines existed but did not prevent IBM staff from over-stepping the mark and breaking the rules at the bid stage.

That is not to say that public inquiries should not be held or guidelines produced – in theory at least they serve a very useful purpose. However, the real challenge lies in getting the industry to adopt the recommendations in practice and make improvements to how they manage and deliver IT projects.

What else can be done? Is it time for a radical re-think on the approach to IT procurement, particularly large-scale, complex projects? Should customer-side organisations be more realistic about what is achievable when formulating their IT strategies? Will organisations enjoy more success by taking on smaller, more manageable implementations?

Similarly, are they better-off maintaining the status quo (with smaller, achievable improvements) than embarking on high risk, landmark IT projects that try to integrate or harmonise different business divisions that may have operated in an imperfect but adequate manner for many years?

What is a manageable size?

Experience suggests that a more standardised solution can mean more change on the customer side (i.e., to how it operates its business processes) but a lower risk of delay, cost-over-runs and legal disputes – provided the customer is in fact prepared to change.

Conversely, a heavily customised or bespoke system that is required to map to a customer’s business, without requiring any significant adjustment to existing processes, is likely to carry higher risk.

On this approach (i.e., buying less) the overall risks should be lower – provided the customer does not change its mind part way through and start pushing for a “Rolls Royce” solution when it contracted for something else.

On the other hand, is it realistic to go for the safer option when the reality is that a complex, large-scale solution with increased automation is an absolute “must have” in order for the business to remain competitive and overcome potentially greater risks associated with an ageing legacy system?

Will organisations enjoy more success by taking on smaller, more manageable implementations?Michael Bywell, Minter Ellison Rudd Watts

The answer of course is that “it depends” and each case will need to be considered on its merits. However, there is an increasing body of evidence which suggests that very large procurements over several years pose a much higher risk and may either fail altogether or result in significant delay and cost over-runs.

The overlooked equation: Neither business or technology

It is my long-held view that greater weight should, in any event, be placed on an intuitive approach to the procurement and delivery of IT projects.

Over-reliance on process, procedure and bureaucracy can cause delay and distract the parties from what really matters to the well-being and success of the programme.

To this end the best people should be recruited to undertake the procurement and delivery of IT systems, particularly more complex systems and those that require significant change to existing practices and/or business processes.

Indeed good people, on all sides, are more likely to deliver what the business needs even where the underlying documentation may be inadequate.

It is therefore vital that the project is fully and properly funded and resourced by the right people.

Based on an assessment of the key findings in Novopay and QH there is significant commonality across the two reports. This provides further support for the proposition that IT projects run into difficulties for the same reasons.

The lessons are clearly expressed in both reports and once again there is significant overlap.

More broadly, evidence from around the world suggests that the rate of failure of IT projects remains high and that this is a truly global phenomenon.

The increasingly adverse consequences of failed projects probably reflects increased scale and the fact that today software runs entire enterprises, not just certain business functions as may have been the case 20 or more years ago.

At a macro level one area to consider may be scaling back customer side expectations and recognising that buying less and/or simplifying business processes and/or adopting a staged approach to large programmes may promote better outcomes.

Otherwise the key to success probably lies in the quality of the people engaged on all sides, together with education and (ideally) adherence to published guidance on best practice and lessons learned from past mistakes.

A tale of two inquiries: Novopay and Queensland Health

Exactly a year ago, reports were published across New Zealand and Australia about these two cases – the timing was purely coincidental, and both cases were unrelated. Yet, both share common themes on why things went wrong.

Dissecting the Novopay case

Novopay is a web-based payroll system for 110,000 teachers and support staff in state and state integrated schools New Zealand.

The project was delivered two years late with significant cost overruns and a highly problematic go-live (under and over-payments and call centre problems).

The inquiry was undertaken by Sir Maarten Wevers (a senior public servant) and Deloitte.

Main findings

The customer’s requirements (i.e. Ministry of Education or MOE) were never fully completed and the relevant contract schedules were left blank.

MOE should have moved away from existing practices and simplified processes.

Schools were not prepared for the changes, according to the report.

The design, development and testing was done in parallel rather than phases with stage gates. Opting for a “big-bang’ go-live was also the wrong approach.

The Project Board misrepresented the system’s readiness for go-live in a paper for Ministers. The post go-live period was chaotic (but the pay error rate has since improved).

There was poor vendor management by MOE (for example, no contemporaneous breach notices were issued) together with poor governance, project management and leadership.

Probing Queensland Health

The QH investigation was led by The Hon Richard Chesterman QC, a retired Court of Appeal Judge.

The payroll system for Queensland Health cost four times the original price and took three times longer.

It was called a “catastrophic failure” but the problems were eventually fixed. Interestingly, there was no breach of contract by IBM according to the report (the contract was changed or varied and they were paid more).

However, IBM should have been excluded due to inappropriate conduct by them at the bid stage. Likewise, a key individual on the QH side should have been removed for showing inappropriate support for IBM.

Main findings

Unrealistic timeframes due to perceived urgency (QH thought the legacy system would not last).

Deficient and unstable scope throughout.

Abbreviated testing and standards lowered.

No proper assessment of go-live risks was undertaken.

Nothing was done to re-set the project – go-live should have been postponed.

Tasks were performed at the same time which is poor practice.

There was too much bureaucracy.

The best people should be recruited to undertake the procurement and delivery of IT systems, particularly more complex systems and those that require significant change to existing practices and/or business processes. Michael Bywell, Minter Ellison Rudd Watts

IBM should have been excluded because:

• They sought to gain unauthorised and unlawful access to a rival’s bid information by accessing QH’s database;

• They received (by illegitimate means) an email (source unknown) containing a rival’s “Not to Exceed” Price” and failed to report it;

• They were prepared to circulate (without objection) “intel” about the evaluation process that had been leaked to them (source unknown).

• The QH Programme Delivery Director showed inappropriate partiality towards IBM during the procurement (for example, he invited IBM – but no one else – to participate in a “dry-run” at the bid stage).

• No probity advisor was formally appointed and the conflicts register was not properly administered.

• QH settled with IBM on a full and final basis against legal advice and without knowing the value of its rights.

Despite the settlement, recent press reports suggest QH has since launched legal action against IBM in Australia seeking compensation.

Lessons in forward planning and probity

These two cases can provide key lessons for other organisations embarking on massive change management programmes involving business technology systems and software.

The first is to ensure forward planning for all legacy systems – to avoid truncated transition periods. Customer side organisations should be prepared to simplify their existing practices and processes to avoid heavy customisation (which can be expensive and time-consuming).

The parties should have a clear plan for capturing requirements as work proceeds – including stage gates or check points.

It is important to consult with users and condition them for change – ensure they are fully prepared.

Avoid short cuts (testing), parallel work and “big bang” implementations.

Ensure that readiness assessments (i.e., to go-live) are frank and honest.

Put in place experienced and empowered project managers to perform an assurance function.

Be prepared to query or stop projects that may be in trouble.

Ensure contingency plans are in place and consider protective correspondence and/or breach notices where appropriate.

Ensure that a probity adviser is formally appointed.

A conflicts register should also be set up and property administered.

And lastly, ensure that legal rights and remedies are fully understood before settling claims.

Silver-lining?

It must also be acknowledged that many IT projects enjoy success and that it tends to be the problematic projects that make the headlines and get everyone’s attention.

Even then it is usually possible to “turn-around” or “save” projects and relationships that have run into the sand and thereby avoid complete failure.

Secondly, there now exists a golden opportunity for the IT industry to make changes so that the rate of success increases further in the future and unhappy stories such as Novopay and QH are confined to history.

A great deal can and should be learned from the mistakes of the past and we should not forget that any new industry (which the IT industry surely is) takes time to mature and get everything right.

Michael Bywell is a consultant at law firm Minter Ellison Rudd Watts (michael.bywell@minterellison.co.nz)

Follow CIO New Zealand on Twitter:@cio_nz

Sign up for CIO newsletters for regular updates on CIO news, views and events.

Join us on Facebook.