The IT Measurement Inversion

Are your IT investment decisions based on the right information?

1 2 Page 2
Page 2 of 2

Truly Important Variables

The single most important unknown is whether the project will be canceled. This is a binary phenomenon-it either happens or it doesn't-and so is represented with a single percentage probability (not a range). If cancellation occurs, no benefits are ever received; but at least some development costs accrue.

The next most important variable is utilization of the system, including how quickly the system rolls out and whether some people will use it at all. Like cancellation, uncertainty about how soon people use the system never shows up on most ROI calculations. Many simply assume that benefits begin upon implementation, with no allowance for ramp-up or difficulties.

Of course you need to estimate development costs when making a decision about IT investments. However, you don't usually need to reduce the uncertainty about those costs to make an informed decision. Reducing the uncertainty about the utilization rate and the likelihood of cancellation of a new system is much more important when deciding how to spend your money.

Beat the Paradox

Why the IT measurement inversion? Perhaps we focus on measuring what we feel comfortable measuring regardless of how important it is. The IT industry also emphasizes development costs; there are a lot of "software metrics" firms that focus on measuring development costs, to the exclusion of other more important variables. Another reason for the paradox may be simpler. Many people simply don't know there is a formula for the economic value of information.

Counteract measurement inversion by measuring the value of information. First, learn how to model uncertainties in ROI calculations. Start doing this as soon as possible, especially on your largest IT investments. Your organization may already have people with the statistical training to do this.

Second, for the investment decision, don't bother hiring software metrics services to reduce initial uncertainty about costs. A range of, say, $2 million to $4 million for an IT investment may seem very uncertain, but it is probably less so than some of the benefits. You may need to count function points later for managing programmers' productivity, but spend your money and efforts to reduce other uncertainties first. And be sure to consider costs for systems maintenance.

Third, take what you would have spent on software metrics and focus it on researching the system's benefits, how quickly people will start using the system, how much it will be used and what the risk of cancellation is.

To make the best investment decisions, IT management must learn how to compute the value of information, and start modeling utilization and cancellation in cost-benefit arguments. Once they do, measurements will be properly prioritized and the IT measurement inversion will disappear.

div-hp-01.gif

Douglas Hubbard recently founded Hubbard Ross LLC in Glen Ellyn, Ill. He can be reached via e-mail at dwhubbard@hubbardresearch.com.

Related:

Copyright © 2007 IDG Communications, Inc.

1 2 Page 2
Page 2 of 2
Download CIO's Roadmap Report: 5G in the Enterprise