Stuck on ROI

What's a good CIO to do when facing a clamour from executives, boards and shareholders to present a compelling business case, while knowing almost no one will believe that business case when presented?

The Central European Bank director was bristling with frustration. He routinely performed return on investment (ROI) calculations that promised spectacular returns, he told Cutter Consortium publisher Karen Coburn - sometimes as much as one thousand percent - yet at the end of the day, IT was not delivering anything useful to the business.

"Ah," said Coburn, "I know what you mean. It's pretty clear the root cause of most of the mistrust with ROI has been IT's history of not always delivering anything of use to anybody."

It has become almost a ritual for business cases for IT projects to be met with high levels of mistrust and suspicion. Executives and senior managers have learned to greet ROI claims with a generous sprinkle of scepticism, doubting claimed benefits can be realized and that identified costs will fall in line. And it is undeniably true that in too many cases it is a mistrust forged in the fires of bitter experience. Executives once bitten by overruns, delays and a failure to fully realize promised goals are likely to have the encounter burned into their brains and souls.

The issue has become so pressing the August 2004 Cutter IT Journal (CITJ) was dedicated to trying to answer the question "Analyzing IT ROI: Can We Prove the Value?"

Business cases - the financial models and supporting documentation used to evaluate IT investments - are among the least understood, least trusted tools that managers encounter in running a technology operation, notes CITJ's guest editor Mark Cotteleer. Almost no one believes the work product that is eventually delivered. Yet business cases are constantly being demanded by executives and boards.

In fact "nobody believes the ROI" was a sentiment expressed by numbers of participants at Cutter Consortium's conference in Boston last May. It is a challenge that business and IT professionals have wrestled with for decades. Prominent IT gurus Erik Brynjolfsson, Nicholas Carr and Paul Strassmann, among others, have questioned whether a business case even exists for most projects, notes Clarity Consulting president and Cutter senior consultant Ian Hayes.

Yet Hayes, who has advised dozens of Fortune 1000 companies on a variety of IT issues, points out that our finance textbooks (not to mention our CFOs) tell us that we have a fiduciary responsibility to shareholders to invest the firm's capital wisely. That means not investing money unless there is a guarantee of a return.

"Experience in the field reinforces the point. A recent Cutter survey, for example, showed that 63 percent of IT executives are required to perform ROI analyses in order to justify IT investments," Hayes says. "Forty-two percent of respondents indicated that these policies are more rigidly enforced than they have been in previous years. Other research, published in CIO Insight, indicates that as many as 87 percent of firms require the development of a business case prior to investing in information technology."

Nevertheless, Aberdeen Group, which last year reviewed users of project portfolio management (PPM) software to look for real examples of ROI results, confirms that while most firms talk a good story regarding ROI, few actually live it. Aberdeen Group's survey results showed that an "eye-popping" 5 percent of firms actually collect ROI data on PPM implementations. While most respondents could identify the kinds of benefits that they expected and could even identify some benefit types that have been realized post-implementation, the researchers found that when it came to documenting key processes, metrics and so forth, pre- and post-implementation, virtually none of the firms could identify their ROI results.

Page Break

Such is the heat on CIOs to deliver ROI that some have even resorted to delivering fictitious business cases, according to Dr Kevin McIsaac, research director with Meta Group and a veteran of many business case assessments. "I look through many vendor business cases and they're wildly optimistic; there's a lot of fudge factor thrown into them. To get their pet projects through, many people start with the ROI they need to have and work backwards. We've got to be a lot more critical about these things," McIsaac told CIO Government last November.

"Lots of times in firms the ROI just becomes an exercise for justifying something that you already want to do," CITJ's Cotteleer says, "and so there is a process whereby we go through and either fudge or make up the numbers to kind of get done what we have other reasons for wanting to get done. That doesn't mean that it can't be a useful exercise, and it doesn't mean there aren't firms out there that do in fact try to make it a meaningful exercise - it just means that you need to have some discipline with the process."

So how do CIOs reconcile the demand on the part of firms, executives and shareholders to present a valid business case with the reality that almost no one - even in some cases the CIO presenting it - believes the work product that is eventually delivered?

At Tempo Services Limited, group IT manager Dermot Musker says the importance of ROI to his company "depends on who you ask - the higher up in the organization the more important it gets".

The need for an ROI at Tempo Services is determined on a case-by-case basis according to the size of the investment. Musker says there is no specific return on investment that must be proven before a project is engaged; however, "the expected gains must be described in the business case".

"The executives don't necessarily have their own rule of thumb," he says, "they would be relying on the experts to provide them with their best judgment. It's more on the lines that an executive would test the decision, based on whether they can understand the business case, the wording and how it's explained, and whether they feel that the recommendation made is commensurate with that business case."

Nevertheless, Musker says recognizing that it can be extremely challenging to present a business case for upgrades and investments in security, for example, executives in some cases will accept an explanation of the risks involved, and a judgment based on those risks.

Likewise Bruce Rice, general manager IS at RACQ General Insurance, says ROI is always important, although sometimes the business case will include a high degree of non-financial gain. For instance, as a mutual organization, one main stream of RACQ's business is member services, so sometimes Rice may be able to present a case that stacks up mostly in terms of a new business process that saves some staff or staff time or telecommunications costs.

"There are instances where a proposed project might not fully meet a 12 percent or 20 percent or 100 percent ROI," he says, "but there might also be perceived a degree of improved member services. So in those cases we might trade off an ROI which is not complete, against some real - although not quantified in dollar terms - benefit to customers."

Rice says RACQ spends "buckets of money" on strong security and an e-mail protection environment. "And I guess those things probably don't pass an ROI, or if they do, it's [a question of] how do you put a price on the likely intrusion or the likely closure of your network?

"[In those cases] we fully understand the cost, and probably leave the benefit as a set of words rather than a set of dollars. If you were going to say: 'We need to spend $45 million and it's because it's a nice thing', probably that's not going to be enough. But if it was: 'We need another $30,000 or $100,000 for a higher level of intrusion detection', and you have a set of words which will wrap around the risks and the protection you bind - that may well be enough for an ROI at that cost."

Page Break

The Real World

However much CIOs and pundits pay lip service to the notion that ROI is important, a recent survey of 464 senior executives by marketing communications agency Doremus concludes that in the real world, ROI is not the be-all and end-all when it comes to giving an investment the thumbs up.

The survey, conducted with publisher The Economist Group, polled C-level executives including CEOs, CFOs and CIOs regarding their business spending habits. Fifty percent said they regularly spend money on projects they believe in whether an ROI case exists or not. CIO (US) reports such reliance on gut instinct is more prevalent among executives both at larger companies and those in high-margin industries such as pharmaceuticals. In such cases, says Doremus's director of research, Hope Picker, executives may go with their gut rather than a firm ROI figure because they are allowed more room for error than executives at low-margin companies.

In terms of major spending, "faith in their own intuition" still figures prominently, with 44 percent of respondents saying they do not require ROI analysis if they strongly believe in the project. Only 5 percent say they never rely on gut instinct, says Picker.

Even so, of executives who claimed to believe that shareholder value is a critical component for making corporate decisions, 75 percent reported requiring ROI analysis before making an investment choice. But as every CIO who has ever tried to put an ROI on a security measure knows, delivering an ROI on many technologies can be an extremely tough ask.

Explaining the Productivity Paradox

Scepticism about the returns from IT investments is a phenomenon almost as old as IT itself. In 1987 it became tagged the "computer paradox", when Nobel Laureate in economics Robert Solow made the famous quip: "We see the computer age everywhere except in the productivity statistics." Much loved by economists, Solow's "productivity paradox" recognizes that productivity growth has slowed every decade since the 1960s while investments in information technology have grown dramatically. Some take this as proof that information technology does not affect productivity.

Yet for CITJ contributor Paul Strassmann, recognized as one of the 12 most influential CIOs of the past decade by CIO (US) in 1997, the answer to Solow is that you cannot prove exactly what contributions computers make because it is impossible to replay history to see what might have happened under different circumstances. Nor can you measure the worth of the future value of IT without the gift of prophecy (which is notoriously absent among technologists).

"All you are left to do then, in the quest for valuation of IT, is to evaluate the best decision you can make at the time when you commit to a credible plan," Strassmann writes. "The logic of such reasoning propels you to the most obvious conclusion: making no changes to IT as it is presently can be the only valid basis from which all other options can be assessed. If your budget inquisitors can accept such reasoning, you may be able to claim (and get away with it) that the value of IT can indeed be calculated using conventional methods of financial analysis."

Better, perhaps, than trying to come up with ROI figures no one can greet with a straight face.

"So far - to my best knowledge - nobody has been able to demonstrate that there is a positive correlation between money spent on IT and sustainable profits. Sure, there are articles about the positive contributions of IT. But the proof that could be applied to justify greater IT spending as a sure cure for poor financial numbers is still missing," writes Strassmann in an article called "Six Rules for Finding Value" in CITJ.

Strassmann says the quest to demonstrate the directly measurable value of IT can be added to the list of fascinating but hitherto unfulfilled ambitions to attract academic fame or consulting contracts. Sadly, academics never reveal their data sources or make available the metrics that would let outsiders independently verify benefits.

"What is always missing is a repeatable technique for performing the calculations that would satisfy a firm's methods for making investment decisions. Even in the rare cases where someone detects a trend favouring IT, one cannot find evidence that the cases picked to support the assertions were not biased," Strassmann says.

"Just about everything that has been published on the subject of IT value can be found in either academic journals or sales brochures. I regret that I have not yet found a single academic paper that could be used to back up my frequent budget presentations . . . With regard to the vendors' projections of huge ROIs from IT investments, a prudent CIO would be well advised to abstain from using such tainted goods."

Page Break

Nevertheless, Strassmann says, there are ways of finding IT value - it is just that they are all indirect. You can plead that IT creates value, provided you come well prepared. Just as in repainting a house, the most important part of the job lies in proper preparation, not in spreading the coating.

1 2 3 Page 1
Page 1 of 3
7 secrets of successful remote IT teams