The Metrics Trap...And How to Avoid It

Joe Drouin caught a lucky break when he became VP and CIO of TRW Automotive in 2002—or so he thought at the time. A consultancy brought in to benchmark all of TRW’s internal functions—everything from IT to legal to sales—found that the company was spending less on IT as a percent of overall revenue than the industry average, which was about 1.5 to 2 percent.

Not one to look a gift horse in the mouth, Drouin played the metric for everything it was worth, highlighting it in every PowerPoint presentation he could during his first year as CIO. "I used it to say, we are managing IT effectively, and here’s the confirmation from this outside firm," recalls Drouin. "It made me one of the good guys in the eyes of the CEO and COO." At one point, the CEO, who believed that inexpensive IT was good IT, joked that he expected to see Drouin and his staff outfitted with T-shirts that had the percentage stamped across their chests in big, block numbers.

But as that first year wore on, Drouin felt less and less like wearing that T-shirt. "I was guilty of using the number not because it demonstrated the value of IT but because it showed a positive trend," he admits.

The shallowness of the benchmark became clear as Drouin prepared his first budget presentation. The CEO asked him to break out IT spending as a percent of revenue for TRW Automotive’s 12 individual business units, each of which had its own legacy infrastructure and independent IT spending patterns. As it turned out, costs ranged significantly across the units. In fact, some units were spending two to three times more than others on a percentage of sales basis.

Suddenly, Drouin didn’t look like one of the good guys anymore.

He looked like a manager whose costs were out of control.

Much to Drouin’s chagrin, the CEO initially tried to use the percentages to reduce the budgets of the higher-spending units. But it quickly became clear that the spending had no correlation to business success. Some of the units spending less on IT as a percentage of revenue were not doing as well as units spending more. Worse, if TRW Automotive’s overall revenue fell (which was a distinct possibility given the auto industry’s struggles), Drouin’s IT spending as a percentage of revenue was going to rise even if he didn’t spend an extra dime on IT.

"It’s disappointing to be measured on one simple metric, only half of which I have any real control over," a sadder but wiser Drouin says now.

Caught in the Metrics Trap

Using percent of revenue as a foundational metric to measure IT tends to cast IT in the role of a cost to be controlled. "When CEOs focus on spending as a percent of revenue, it’s because they’ve already decided to cut IT spending and they use the data as a justification," warns Barbara Gomolski, research vice president for Gartner. "It’s hard to use this metric to show that you are doing the right thing." In this zero sum game, success is defined simply as lowering the percentage over time. "It’s not clear how low it should go," says Drouin. "Joking with the CEO, I said, ’In your mind it should be zero.’ We had a good laugh, but at what point do we decide it’s at the right level and you don’t drive it down further?"

Drouin wanted to burn those old PowerPoint slides. "I used the metric to my advantage, and it turned around and bit me," he laments.

It was too late to put the cat back in the bag. Nor was it an option to complain about how unfair the metric was. Rather, Drouin knew he needed to educate the CEO and COO about what constituted constructive cost-cutting in IT versus that which would rob IT of its ability to provide strategic differentiation. "I still mention the metric," says Drouin, "but I don’t dwell on it." Instead, he emphasizes benchmarks that are more specifically targeted—such as the per-user cost of ERP systems in the different business units—to show that he’s lowering costs in a way that will help the business rather than paralyze it.

But it’s hard to get away from such a deceptively simple measurement—IT spending as a percentage of revenue—which remains "far and away the most popular metric for benchmarking IT spending," says Gartner’s Gomolski, who’s not at all happy about it.

"Top business executives don’t want a dozen different metrics," Drouin explains. "The COO said to me, ’Come up with another metric, and I will leave this one alone.’ I consulted the consultants and the analysts, and they couldn’t come up with one."

The reason Drouin’s consultants couldn’t offer him a magic metric is because one doesn’t exist.

There is no single metric for determining the right amount of money, the right percentage of revenue, to spend on IT. Even those CIOs who believe that measuring IT spending as a percentage of revenue produces a legitimate benchmark, such as Dow Chemical’s David Kepler, consider it a very rough start to a much deeper discussion of IT investment strategy. "My experience is it’s relevant to describe about how much you should spend," Kepler says. But, he adds, "it says nothing about how effective your spending is."

The problem most CIOs have to confront is that, in the view of much of the business, IT spending is a big black box. In fact, it’s often the company’s single largest capital expense and one that promises to grow larger as the world grows ever more digital. "The reason people attack IT over this metric is because they don’t appreciate what you’re spending and why," says Kepler. Unless CIOs can fill this gap in understanding with a delicate combination of relationship building, education and a series of contextual metrics designed to give businesspeople a sense of IT spending effectiveness, efficiency and value over time, CIOs will see their budgets reduced to a cost to be managed against a vague, often completely irrelevant, constantly sinking average.

Where the Wild Variables Live

The idea of describing and analyzing IT spending in relation to overall revenue has been around for a long time—decades, say some analysts. One can attribute its longevity as a business metric to its simplicity and the ease with which one can make the calculation. It’s also easy to get the benchmarking information out of your competitors. No one’s giving away any trade secrets by filling out (often anonymously) a survey with their IT spending information.

But it’s also a legacy from an earlier era. "The metric is from a time when people viewed IT as a utility rather than as something to provide strategic differentiation," says Scott Holland, senior business adviser for The Hackett Group.

It was, in many ways, a simpler time. In the days when mainframes ruled the earth, IT was more centralized than it is today and, consequently, costs were much easier to calculate. IT back then was basically a handful of really big machines, and the software and staff needed to run them. By comparison, today’s Internet and PC-based computing infrastructures are much harder to account for. It’s pretty easy for a rogue marketing department, for example, to buy its own servers and software without IT ever knowing. It’s much harder to sneak a mainframe past building security. And IT today is a challenge to pigeonhole. For example, should a cell phone/

PDA combination count as IT or telecom? And does telecom—traditionally separate from the IT budget—count toward IT spending? Is software depreciated over time or counted as a onetime expense? In all these cases, it depends on the analyst or consultancy doing the counting.

There are so many ways to account for IT that most CIOs dismiss the percent-of-revenue metric unless it is carefully targeted for their specific industry or industry segment. Yet even in an industry context, the averages can mask huge variations in the sample. Companies at the high end can spend as much as 100 times more than those at the low end, according to CSC Consulting. "If I say that the average age of my two children is 10, do I have an 11-year-old and a 9-year-old, or does my family consist of an 18-year-old and a 2-year-old? It is not enough to know the average. One must also understand the spread of the data," wrote Eugene Lukac, a partner with CSC, in a report about a recent CSC spending survey. Nor does the average account for a company’s unique situation; for example, costs will vary wildly between a chemical company with 10 plants spread around the globe and one with three plants, all in New Jersey. (For other variables that figure in overall spending calculations, see "Why You’re Spending More," this page.)

The Battle Over the Numbers

Douglas Novo, former CIO for Venezuelan chemical company Polinter, knows all about these variables. The chemical industry is capital-intensive, dependent on expensive plants and equipment, with low profit margins and unpredictable revenue. That volatile combination means chemical executives are always looking for ways to cut operating costs. To do that, they bring in consultants about every two years to compare their company with its peers and to run the numbers.

In Novo’s case, the consultants told him in 1995 that he was spending 1.5 percent of Polinter’s revenue, or about 1 percent more than his peers. Novo’s response was simple and blunt: "Impossible!" (Indeed, other sources told him that the industry average for the petrochemical sector at that time was about 2 percent.)

What was he to do? The numbers he was being shown didn’t jibe with his notion of reality but neither did they come out of thin air.

"There’s no equivalent of Generally Accepted Accounting Practices for IT spending," says William Mougayar, VP and service director for Aberdeen Group. "Without some kind of standard for how to account for spending, you can’t use percent of revenue to compare across individual companies. It’s too hard to know what’s being included and what isn’t." (For a downloadable PDF of U.S. IT spending benchmarks by industry and by size of company, go to www.cio.com/040106.)

But now Novo was on the defensive. He had to justify $2 million in spending that suddenly seemed to place him on the wrong side of the chemical industry norm. First, he pushed back against the consultants’ findings. He got them to accept the idea that perhaps Polinter only seemed to be spending more than its peers because those companies were failing to take all IT operational and investment costs into consideration. Novo argued that the greater centralization of Polinter’s IT operations, as opposed to its competitors, made Polinter’s accounting more accurate. And why should he be penalized for his accuracy? But basically it was still Novo’s word against the consultants.

So he fought back in a more constructive way: He created his own metrics. He took Polinter’s history of IT spending and broke it down as a percent of revenue, per employee, per employee served by IT, per IT employee and per ton of finished chemical product (a common metric in the industry). "Using historical data is important," Novo says, "because then you can show executives how spending has changed over time and how it responds to changes in the business, such as during bad years when revenues fall."

Novo’s goal was to get off the defensive and shift the discussion from an argument over numbers to an analysis of performance. "It’s important to focus on what you are doing—not someone else," he says. "The CEO wants to understand why you’re doing what you’re doing and what you’re doing to improve."

Yet hanging over his discussions with his boss was the suspicion that by offering his own numbers Novo was simply making excuses to avoid having to cut his budget. To allay those suspicions, he divided his numbers into two pots: IT spending on ongoing operations (essentially the infrastructure and the services needed to keep the lights on) and IT spending on new applications, research and services. By slicing the spending this way, Novo could focus cost-cutting scrutiny on those parts of his budget that he can and should be cutting, while preserving the portion of spending that, if cut, could threaten IT’s ability to provide competitive differentiation.

"The smart IT people are continually tuning the infrastructure to make it lower cost and more efficient so they can free up more funds for new work and innovation," says Laurie Orlov, vice president and research director for Forrester Research.

On Toward Better Metrics

The same thinking as Novo’s guided Drouin to begin giving his COO and CEO data on the cost of providing employees in TRW Automotive’s different business units with ERP software. The idea, says Drouin, is to cut the costs of providing important capabilities to the business without endangering the capabilities themselves.

Related:
1 2 Page 1
Page 1 of 2
Get the best of CIO ... delivered. Sign up for our FREE email newsletters!