Comparing Your IT Shop to Others

Benchmark the right way to gain useful insights.

CIOs should strive to improve organizational effectiveness and value. One popular way to focus IT performance improvement is to compare an IT organization's performance to other companies through benchmarking—typically comparing IT costs. But benchmarking on cost is not sufficient and can be dangerous; cost alone does not take into consideration firm-specific factors such as business expectations or context, nor does it give insight into how a firm achieves its numbers; a low-cost organization could simply do "less with less." To drive performance improvement, CIOs must look at the practices that underlie costs, and they need to ensure they are comparing themselves to firms with similar demands and challenges.

Benchmarking Your IT Shop

CIOs must often answer questions from other execs on how the firm's IT compares with that of similar firms. When these questions arise during budget time, CIOs must defend budgets or show the value a firm is getting from its IT dollar. In these situations, CIOs must show that they are running an appropriately lean operation for the needs of their business, and that they are pursuing ways to run even more efficiently and effectively. Comparing an IT organization to various benchmarks is an oft-used tactic, but a limited one, as benchmarks:
  • Poorly factor in firm-specific context. Benchmarks can be accurate and precise around commodity technology functions, such as servers per server administrator. But they are more misleading than useful when applied outside these commodity functions. Firm-specific attributes, such as the degree of centralization, or the place of technology in the firm's operational model, drive how IT operates and thus drive spending. A CIO who cannot explain why his firm has a different cost profile will be on the defensive in all planning and budgeting activities.

  • May suggest opportunities, but not explain how to pursue them. Regardless of what benchmark numbers show, you need to know how your management practices compare with known best practices—and practices that are feasible for firms like yours. For example, if your application maintenance spending is higher than the benchmark, do you have strong practices to manage this demand and ensure it reflects business priorities, not just business demand slipping in the back door? If you spend less on architecture activities, does this mean the scope of your architecture program is narrower than best practice firms? Understanding this will tell you whether a "bad" number actually is appropriate, or a "good" number shows missed opportunities.

Process Used for Comparison Determines Usefulness of Results

Firms often use a Balanced Scorecard to manage their performance. The genius behind the Balanced Scorecard is the recognition that managing over time by a single dimension, such as financial metrics, is dangerous. Similarly, comparing your IT organization's efficiency using a single dimension is misleading, and taking action based on these conclusions is dangerous. An IT organization is a complex system, and so multiple factors and dimensions should be captured and assessed together. For example, the volatility of a market environment, the portion of IT budget applied toward new capabilities, and the practices used to select, resource and manage business initiatives should all be considered as a picture; you could see whether higher spending is due to business demand or to weak project processes—or both. A prudent CIO will take the following approach for comparing an IT organization to others:

1 2 3 4 5 6 Page 1
Page 1 of 6
Download CIO's Roadmap Report: Data and analytics at scale