The most important metrics to drive IT as a business

Several years ago, I got to work with CIOs on a balanced scorecard product. It was great to learn what measures mattered to CIOs versus their IT counterparts. And how CIOs wanted to connect corporate goals to themselves and other IT leaders. It has been nearly 10 years, so I wanted to hear the CIOs perspective on measurement and management.

How important is it that you drive the business of IT by metrics and KPIs?

CIOs suggest that metrics and KPIs are critical. They should be used to validate the success or failure of the change that IT implements. They should be used as a reference point for IT targets regarding performance, efficiency, sustainability, and resiliency.

A key thing to understand, say CIOs, is when a metric as trailing indicator can result in IT or business failure. This is especially the case for team and staffing issues. CIOs say that metrics need to have a wide scope. They should include critical updates related to strategy and innovation. CIOs suggest, however, that many IT leaders still are working at understanding the value of metrics. Measurements, nevertheless, can help IT organizations make sure they are contributing to organizational growth and profitability. Some CIOs worry about assigning responsibility for delivering a metric alone. These CIOs say this can easily lead someone to focus on what is important on paper, but not critical or truly opportunistic at the time.

CIOs believe that most organizations need a balanced scorecard with a mix of innovation, operational excellence, risk management, and cultural KPIs. CIOs say it is important to differentiate between metrics for run (keep the lights on), grow (upgrades, additional functionality), and transform (totally new business or technology capability).

CIOs say cultural KPIs really matter. They say that they should include the following:

• Increased collaboration
• Increased diversity of thinking
• Improved learning opportunities
• Increased mentoring
• Progress against diversity objective
• Reduction in workplace drama
• Increased career planning

The hard part, clearly, is agreeing upon metrics, finding easy ways to measure them, and driving results against them. Clearly today, metrics are needed that measure agility and DevOps. It is important, say CIOs, to have conscious discussions regarding how to select and measure metrics and resulting KPIs. As the old saying goes, what gets measured gets managed. So, you better measure the right things.

Creating business-friendly metrics

CIOs say that they like continuous improvement metrics. This includes things that measure improvement—such as a reduction over time in unplanned downtime or a performance resulting in direct, monetary penalties. CIOs suggest the more you can demonstrate and encourage proactive and value-additive engagement, the less important trailing indicators become. This is even true for staff-oriented measures. CIOs want to measure the delivery of committed projects. CIOs suggest that the business can understand them if the CIO does an excellent job of coaching and are persistent about using them. A CIO many years ago argued for persistence in using metrics—it takes, in their words, several months to understand what a squiggle in any direction means.

At the same time, it is important to measure what matters and demonstrate why further investment in IT is warranted. Too many IT organizations let their tools decide what gets measured versus letting strategy be the decider. My friend, Allan Hackney, former CIO of John Hancock, held up an application development to ensure it created the data to enable the improvement metrics and goals.

One CIO, at this point, volunteered that they had not had much luck with quarter over quarter measurement for the transformation portion of the project portfolio. They suggested that measures here need to be based on milestones that the business cares about throughout the program lifecycle. Another CIO suggested that a nice equilibrium involves looking at customer acquisition cost, customer lifetime revenue, return on advertising spending (ROAS), total cost of ownership (TCO), time to market (TTM), human capital/customer retention rate, ROI/product margin, and internal/external SLA/KPIs. These are all impacted by IT delivery of services and innovation.

Does IT need internal metrics that tell it how well it’s running IT?

CIOs suggest that optimizing only the metrics regarding where their organizations are today is a losing proposition. Many table-stakes metrics are baked into tools and frameworks. In today’s era, internal metrics are helpful, but they can’t be depended upon. Deep business integration combined with daily feedback is seen as the best way to measure. That said, every group should have ways to force continuous improvement. It is important that IT be working against resource gaps, including the following:

• Talent and budget
• Projects completed in alignment with strategy
• Project displacement (bumped by unplanned)

There are many detailed metrics that IT organizations can use that the rest of the business will not care about. These includes things like the percentage of PCs with the latest patches. For some CIOs, customer satisfaction surveys if done too frequently is self-defeating. These CIOs say a once a year survey is as much as the business will stand for. It is different, they say, if you are asking about an individual experience with the helpdesk. I found at one of my startups that the frequency can be increased if the number of questions is reduced to four. You can get what you need by simply rotating the four questions between users.

Meanwhile, CIOs think KPIs should quantify strategic goals. It is important to align IT on what’s important and actively track progress. If IT wants to know how well they are doing, it’s better to have relationships with leaders/stakeholders and ask them. IT will get more meaningful feedback simply by listening.

One CIO at this point said that they use health indicators for run and grow. They track the percentage reduction in change and project defects. They try and correct issues on a quarterly basis with people, training, and better process. They claim it is important to look at resource availability, dependencies and schedules; progress of projects; tickets (open/closed/time to resolution); skills needed based on demand; and funding (actual vs. budgeted vs. needed). CIOs say the accuracy needs to be verified for internal metrics. Without benchmark data, they say, it is challenging to set SMART goals for a team and the business.

Being measured by the same things as the CEO

CIOs believe that their organization’s efficacy should be measured by the same things as their CEO. CIOs say this is essential for IT to be taken seriously. However, it can be very difficult to relate every IT activity to a business outcome.

Everyone needs to work in support and service of organizational strategic goals and direction. Whether it is the board or the CEO or their executive team behind a business initiative, everyone should be in lockstep whenever it is possible.

CIOs suggest that IT leaders must take this step to be treated seriously. However, CIOs suggest it can be difficult to relate IT activities to every business outcome. Oftentimes, IT projects lead to future value and quarter to quarter measurement can lead to short term thinking. For this reason, it makes sense to measure ROI by project and the value generated from cost reductions.

CIOs say that some companies use the run, grow, and transform models to take business strategy and drive it down to all business units with KPI’s for each business unit tied back to each business strategy initiative. Succeeding with this requires good leadership and analysis. Oftentimes IT projects lead to future value and quarter to quarter measurement can lead to short term thinking. For this reason, it is better to measure the total ROI for a project

Should CIOs have a balanced scorecard that relates to the CEO’s scorecard?

CIOs believe in balanced scorecards that are all inclusive. They say if IT is part of the business then it must have a scorecard too. Some CIOs, however, suggest having an overly prescriptive balanced scorecards can show alignment but create some organizational paralysis. For this reason, they like goals, KPIs, and leaving the rest to agile practices.

Clearly, it is up to CIOs to do a better job in communicating. There is no panacea for doing this. You must make sure your scorecard matches the culture and style of the company and its leadership team. Some organizations want details, others want high level reporting. Some want only hard goals while others will want only soft goals. Clearly, it is important for CIOs to make their boss look good and solve the problems that keeps them up at night. Having, a CIOs scorecard is part of demonstrating they are doing this.

CIOs agree with the concept of measuring IT performance with metrics and KPIs. What is left open for many is to decide which measures make sense for their organizations. Clearly, acting will lead to better results and alignment. The question for CIOs is are they ready to start their measurement journey.