ERP Newsletter
 
NEWSLETTERS
 

CIO.com updates, insights and advice on technology, management and your career.

 
 
 
LEADERSHIP
 
CIO Executive Programs
The Leader in Face-to-Face Education for Senior Executives

Offering regional and national programs, CIO (and CSO) events bring together some of the most respected names and thought leaders in information technology and security. Presented by CIOs and other senior level executives, these invitation-only programs offer timely topics and strong networking. Learn More »

 
CIO Executive Council
A Peer-Advisory Service and Professional Association for CIOs

Turn Geeks into Leaders

June 17, 11:30 AM - 12:30 PM U.S./ET (GMT-4)

Larry Bonfante, CIO of the U.S. Tennis Association, will discuss the skills and approaches that your rising IT leaders must learn to be effective in an executive capacity.

How to Handle Your New CEO: Managing Turnover at the Top

June 18, 11:00 AM - 12:00 PM U.S./Eastern (GMT-4)

Turbulent times have increased turnover at the top. Find out what Council CIOs have done to "break in" new CEOs—build relationships, set expectations, educate on the role of IT.

Mid-Market CIO Panel: Tips and Techniques for Improving Vendor Relationships

July 15, 4:00 PM - 5:00 PM U.S./Eastern (GMT-4)

We'll highlight relationship priorities and best practices identified in a Council study, and we'll interact with a CIO panel on the approaches they've used to improve strategic vendor partnerships.

Executive Competencies Assessment Tool

Assess Your Business Leadership Skills with the Council's new benchmarking tool. Rate yourself in change leadership, strategy, customer focus and more.

More / Register »

Learn more about the CIO Executive Council »



 
 
RESOURCE CENTER
 
 
 
SUBSCRIBE TO CIO
 
Are you involved in setting the direction for your company's IT budget or strategy?

Apply today for a FREE subscription to CIO Magazine!

 
 

Avoiding Unreliable Metrics

I confess. I’m a numbers kind of guy. But I’m concerned. I see too many numbers, with too little meaning, ...

 

October 01, 2003CIO

I confess. I’m a numbers kind of guy. But I’m concerned. I see too many numbers, with too little meaning, cavalierly factored into too many IT value discussions. Metrics are often management’s single most important determinant of value. So allowing anything less than top quality numbers—and the correct context for these measurements—leads to wrong decisions about which IT investments to select, which implementation paths to choose or what portfolios to manage.

There are three common ways that people misuse metrics: tolerating ambiguity, discouraging guesstimates and overlooking available sources. Let’s take a look at some solutions for avoiding these missteps.

Tolerating Ambiguity Numbers falsely convey a mystique of truth, by their very existence. But rarely do numbers actually mean what they seem to imply. That is because a metric is not "the result." It is shorthand for a story about the result. If the metric user leaves out the story portion of the metric, we are left to make potentially erroneous assumptions to fill in the gaps. For example, if someone states "we have a 25 percent customer retention rate," he is providing us with an ambiguous metric. Because the story behind the metric is missing, his message has no infallible meaning. In this example, a good metric story might say: "Our internal audit committee confirmed last April that our Southern division has experienced a 25 percent customer retention rate, for those customers who have bought at least $10,000 annually from us, for the past three years in a row. Sue Smith, vice president of finance, has stated that this retention rate is too low to allow us to realize the gross margin goals for this year." Armed with that explanation, we are less likely to misuse the metric’s meaning and implications, and can make an informed decision.

Discouraging Guesstimates Another misuse of numbers occurs when management refuses to accept guesstimates (informed estimates) as legitimate metrics. This problem occurs whenever someone says, "If you can’t prove it, we won’t use it." In this mistaken worldview, only hard metrics from factual situations are valid. The clear message is that guesses don’t count. The reality is that informed guesses are what makes the world work. If business investment strategies were 100 percent fact-based, we wouldn’t need high-price executives to guide the company. Computers could probably do the job via fancy calculations. The whole discipline of risk management is ultimately grounded in probabilities, which themselves are estimates. The medical profession begins and ends with the reliability of doctor diagnoses, none of which are 100 percent correct. But they get enough right that we don’t ban their profession. A similar situation exists with our economy’s reliance on meteorologists. They don’t hit the mark 100 percent of the time. But their track record is good enough for airplanes to fly safely through troubling storms, not to mention properly attiring our families for school and work. The key to accepting informed guesses as valid metrics is to make sure they: A. Come from knowledgeable, experienced and dependable people; and B. Are accompanied by clear explanations of the guesstimate’s premises, assumptions and logic.

 
 
Loading...
 
WHITE PAPERS

Portfolio & Project Management

Learn about opportunities to reduce costs, increase the strategic value of IT and better align resources with organizational priorities.
 

Investing in Business Analytics Technology

Find the answers to your questions about business anyalytics initiatives.
 

Don't Sacrifice Speed

Learn how to ensure custom content is obtained in a fast manner and doesn't drive your customers away.
 

Virtualization: Unparalleled Agility

VMware joined BMC Software to co-author 7 must-read tips for striking the perfect balance in your virtual environment.
 

Effective Service Management Eases the Virtualization Transition

Virtualization adds a whole new level of complexity to the IT infrastructure.
 

Meet Rising Demands on IT and Cut Costs

Strategies for Modernizing IT, Reducing Costs, and Improving Operational Efficiency
 

WEBCASTS

IT Consolidation Made Easy

The Primary IT Initiative for Reducing Costs
 

Webcast with Dan Vesset: Investing in Business Analytics Technology

What exactly is business analytics and why should you care? Dan Vesset of IDC and Gaurav Verma of SAS answer this a...
 

Capitalize on Your SAP Content

After 18 years of partnership and over 3,000 successful customer deployments, Open Text has become SAP's premier pa...
 

Enterprise Cloud Computing: Ready for Primetime?

The progression toward enterprise cloud computing is happening today, as industry leaders deploy technologies that ...
 

BSM in the Field, Practical Insights from Peter Armaly

Have you thought about BSM, but haven't quite gotten the buy-in you need? Get down and dirty with BSM installations...
 

BMC Service Assurance Demo

What if you could predict disruptive IT events and automate their resolution -- all before they disrupt your busine...
 

Resource Alerts

Get instant email notifications by topic when white papers, webcasts, and case studies are added to our library.

 
FEATURED SPONSORS
 
 
 
SPONSORED LINKS
 

Taking the Service Desk to the Next Level

Communications and Collaboration Needs at Business Organizations

Using Open Source to Deploy Web Applications

Secondary Market Primer: Your Network at Half Price

Top-line Performance that's Bottom-line Efficient

Learn how a virtualized enterprise can help your company reduce costs

Why Isn't Server Virtualization Saving Us More?

Learn how to save 30% through project & portfolio management.

5 Steps to Automating Accounts Payable

BPM Survey Results: The Real-World Analysis

Ready to Act: 3 Recommendations for Agile Processes

Achieving Business Agility with Application Grid

Next Generation Enterprise Applications

Achieving Pervasive Performance Management

Smart Decisions: The Role of Key Performance Indicators

The Link Between Effective Online Business Banking and Web 2.0

Get Google Enterprise Search for your business information.

Accenture IT Consulting: Enabling high performance. More...

Top Five CIO Challenges

Insight makes it easy to spend your Microsoft subsidy check.

Five minute business analytics assessment. Immediate results.

Dangerous Collaboration Practices: 5 Ways IT Can Minimize Risk

Accenture: Outsourcing for uncertain times. Click to learn more.

Payback in 9 months with CA Spectrum solutions

The Case for Investing in Business Analytics Technology. Read white paper.

Why Data Loss is Increasing--and What You Can Do About It

Learn how to managing client systems in the enterprise.

Build a High-Performance Open Web Platform

How Interactive Viewer Reduces the Effort to Meet Visualization Requirements

Stop Application Fraud at the Source with Device Reputation

Building the Virtualized Enterprise with VMware Infrastructure

The Global Marketplace Today: Strategies for Tough Times

Top 10 Business and IT Drivers for the Wealth Management Sector

8 Key Ingredients to Building an Internal Cloud

BPM: Leveraging Competencies and Streamlining Processes to Achieve Operational Excellence

White Paper: The Building Blocks for Cloud Computing

Craft a Strategy to Lower Your Total Cost of Ownership

A Natural User Interface for Enterprise Applications

Delivering Secure and Reliable Data through Spreadsheet Automation

Financial Institutions Need Rich Internet Applicatons

"Enterprise-Proven" is the Prerequisite for Enterprise SaaS Portal Solutions

Introducing the new HP ProLiant G6 server family

Accenture: Outsourcing for Competitive Advantage. More...

Better spam protection with Postini for just $1/user/mo

Introducing the new HP ProLiant G6 server family

infoBOOM! - The Mid-Sized Company CIO's Exclusive Community

Accenture IT Consulting: Logical meets technological. More . . .

The Fraudster Economy Model: Operating a Business in the Underground

Get agile IT security with CA Security Management

Trade in your old laser printer and get up to $1000 back!