CASE STUDY 3: Why You Keep Score

Before Jeff Campbell took over the CIO post at BNSF Railway (BNSF) in September 2002, the IT department at the Fort Worth, Texas-based company was viewed as a mystery. Business executives understood the potential of technology to improve efficiency and increase profitability on an intellectual level, but, says Campbell, “they didn’t have anything concrete they could put their hands on that showed the value IT brought to BNSF.” As a result, Campbell says, executives questioned whether the company would be better off outsourcing IT. (They outsourced some of their infrastructure to IBM in August 2002.) And Matt Rose, the CEO, didn’t understand why demand for computing resources was increasing at a time when business was flat. In short, executives had no idea how IT operated. Campbell had to provide some answers.

So he turned to the Balanced Scorecard to monitor strategic IT metrics and to bring transparency to IT’s financial and operational performance for the business. Craig Hill, BNSF’s vice president of mechanical and value engineering, had introduced Campbell to the Scorecard, which is used in various departments throughout the rail company. Hill uses his Scorecard to track productivity, quality and cost control.

Balanced Scorecards became a popular management tool in the mid-1990s for helping companies improve their performance by translating their strategic visions into actions and by focusing on metrics that drive organizational change. While they’re difficult to develop, require long-term commitments from everyone in the organization, and take months of planning and fine-tuning, Balanced Scorecards are profoundly effective when well-conceived and carefully implemented. By taking the time to develop metrics relevant to both the business and IT, and to make sure that IT staff didn’t view the Scorecard as yet another management tool for hammering them, Campbell got critical buy-in from the IT staff and launched a Scorecard team that tightly couples IT’s goals with the business’s.

PODCAST

BNSF’s Best Practices for Implementing Balanced Scorecard

Because of the way the Scorecard communicates the connection between IT initiatives and the corporate strategy, BNSF’s IT staff now clearly understands the impact of their work on business goals. The Balanced Scorecard has also given the entire company—not just the IT department—a singular view into just how much their business applications cost to run, leading to a more prudent use of IT resources. Since Campbell rolled out the Balanced Scorecard in June 2003, IT has managed to keep costs flat while delivering increased service. For example, BNSF’s cost per million instructions per second (MIPS) is now just $29 per MIPS, compared with $42 five years ago. Campbell says the reduction is partly a result of a Balanced Scorecard goal around cost control and partly a result of the financial provisions in BNSF’s contract with IBM Global Services.

“With the Balanced Scorecard, we have more transparency into IT investments today than we’ve ever had before,” says Campbell. That financial clarity gives him insight into where he needs to shift technical and human resources so that he can stay on top of the business’s needs without incurring extra costs, which is so important to the railway business.

A Tough Slog

But no one working on the Balanced Scorecard thought it would be as time-consuming or tough to develop as it turned out to be. Chérie Coles, BNSF’s manager of planning and cost control, technology services—and now resident Balanced Scorecard guru—says the process took four months, twice as long as the group thought it would take.

BNSF’s Balanced Scorecard covers the four traditional areas: financial, customer, process, and learning and growth. To align IT strategies and projects with corporate strategies, each of those areas maps to BNSF’s five strategic themes: service, growth, ease of doing business, efficiency and people. BNSF had to develop metrics for each of these areas, and for the scorecard to be most effective, those metrics had to support the IT department’s and the company’s strategic goals, and encourage changes in employee behavior.

BNSF developed metrics that helped it monitor performance on a daily basis and take action on a function heading south before it was too late.

As part of this effort, BNSF developed leading indicators—metrics that helped it monitor performance on a daily basis and take action on a function heading south before it was too late. For example, while IT projects had always been tracked to ensure they were completed on time and within budget, the problem with this metric is that it doesn’t identify projects that are over budget until it’s too late to rein them in. BNSF Balanced Scorecard developers wanted to come up with a leading indicator that would help them identify projects getting off track before the projects completely derailed, so developers decided to measure the number of projects in compliance with their project management methodology. The argument is that the more closely a project adheres to the methodology, the more likely that project would come in on time and on budget.

Tracking the number of projects in compliance with project management methodology necessitated establishing new processes for obtaining the data needed to track them. For instance, each quarter, IT directors have to review projects in progress under their management against a checklist that outlines the steps that must take place at each phase of a project, says Coles. The directors enter their information on a shared spreadsheet, which is linked to the Balanced Scorecard and sits on a network drive to which the Scorecard team has access.

The Balanced Scorecard developers also set up targets for good, average and poor performance, with respect to all of their metrics as well as the amount of leeway they’d have before metrics moved from green (good performance) to yellow (needs attention) to red (unacceptable). Establishing targets and tolerances for new metrics required trial and error. When the developers first established a metric for tracking the number of train delays caused by IT, they didn’t give themselves any wiggle room because train delays are so bad for their core business. If one train delay was caused by IT, the metric would immediately turn red. They realized they weren’t being fair to the individuals whose performance was being measured against this metric, so they had to track the number of train delays caused by network outages or system crashes for a while before they could establish reasonable tolerances. Jeffrey McIntyre, Campbell’s assistant vice president of enterprise supports in the IT department, says it can take a couple of years before an organization is comfortable with its metrics.

The Balanced Scorecard began as an Excel spreadsheet. IT staff later made this available to the technology services department through a Web link. Then, a dashboard was developed for displaying daily metrics, and a Web-based application called Showback—currently available to IT management and a pilot group of business users—was developed as a module on that dashboard. It displays IT costs for business users and IT staff, and explains what each department in the company spends on IT on a monthly basis. It offers both high-level and deeply detailed views of IT costs. Departments can see, for example, costs of supporting particular users as well as the total number of CPU hours and the resulting cost of critical business applications based on unit costs that IBM Global Services charges IT.

Larry Murphy, an HR director at BNSF, had only piloted Showback for two months but was already impressed with the level of detail the application provided. He can see all of his software, hardware and support costs down to the specific application, server or user. He anticipated Showback would help him and his department with their annual planning and budgeting cycle. “Getting the total cost of an application is going to help us make better cost-benefit decisions,” he says. He also thinks it will show him what technologies, such as his department’s data warehouse, might be underutilized.

Taking It Slow

Sensitive to political challenges—such as staff resentment and pushback—which can accompany such initiatives, Campbell didn’t immediately roll out the Scorecard to rank-and-file IT workers when he had his first version of it on June 1, 2003. Because he was going to use it as a performance measurement tool, he wanted to make sure that the data and the metrics in the Scorecard were accurate and relevant to the business strategy and IT goals. And because staff would eventually be compensated on the basis of their performance against the Scorecard, Campbell wanted to make sure staff understood that the Balanced Scorecard was a way to get them focused on strategic departmental and business goals, and not another stick with which to beat them.

Six Keys for Balanced Scorecard Success

To get everyone in the department comfortable with the Scorecard, McIntyre and Coles discussed it regularly at management meetings and in town hall meetings, and Coles met with small teams to walk them through the Scorecard. Knowing IT workers wouldn’t trust any metrics that seemed unjustifiable, a project team led by Coles authored a metrics dictionary as a means to prevent a backlash, and to address up front any potential pushback. The dictionary is published on the intranet and explains how each metric is calculated, the data that goes into it, the strategic objective to which it links and the name of the IT executive responsible for it. For example, the entry for the percent of time spent with business customers shows that the metric is calculated by dividing the number of hours spent with business customers by the number of hours spent at work each month, minus hours taken for vacation. The objective associated with this metric is for technology services to become more of a strategic partner with the business, and the individuals responsible for it are Dave Sturgeon and Bonnie Henn-Pritchard. Each metric has a go-to guy or gal when anyone has a question or concern about it.

To give everyone time to grow accustomed to the Scorecard, it was also rolled out in phases starting with assistant vice presidents in IT in the second half of 2003. No one’s performance objectives were tied to the Scorecard until the beginning of 2004. “We wanted the time to communicate what this new thing was and why we are supporting it before people were held accountable to it,” says Campbell.

The Payoff

BNSF’s Tracked Metrics

To this day, the Balanced Scorecard is a work in progress. As the business shifts and as the economy expands and contracts, the Balanced Scorecard is updated. Coles leads a monthly Scorecard review in staff meetings to debate metrics that should be dropped or new ones that ought to be added. An IT governance committee approves all changes to the Scorecard. They added some metrics to the Scorecard pertaining to the performance of their field telecom workforce, such as the number of first-time repairs by technicians and the number of times a technician went to the same site to fix the same problem. The metric now also measures network availability.

The effort has made employees both inside and outside of IT more sensitive to IT costs and has imbued them with a healthy sense of competition. When the IT governance committee saw the level of proposed spending in 2003 for “growth and ease of doing business” (with BNSF) was far exceeded by the proposed spending on “efficiency-related” projects, they enhanced their focus as a group on increasing the “growth and ease” proposed projects in 2004. In response to this new focus, from 2003 to 2004, marketing increased the number of IT projects it proposed that enabled growth and ease of doing business by 9 percent, according to McIntyre. In addition, BNSF’s Balanced Scorecard team worked with a marketing director to reduce computing costs of SAS programs. “I think he was feeling a little peer pressure to make sure his staff was being as prudent as they should in their use of IT,” says McIntyre.

McIntyre also notes that software developers got into a good-natured competition in 2004 to see who could save the company the most money by creating and tuning the most efficient applications. One developer looked into how much one process in BNSF’s pricing system cost and went to the business with that information to see if the business was generating enough revenue from the system to offset its costs. In fact, the business wasn’t bringing in enough money, so the developer took the transaction out of production. A team of developers reconstructed the transaction in a way that increased efficiency and put it back into production.

Because of the resourcefulness that transparency through the Balanced Scorecard engenders, the IT department was $1.5 million under their financial plan in 2003.

“The Balanced Scorecard has established our credibility and our visibility, and has really aligned us with the business,” says Campbell. “They know that we have investment teams and resources focused on their needs.”

You can reach Editor Richard Pastore at rpastore@cio.com and Senior Writer Meridith Levinson at mlevinson@cio.com.

PHOTO OF JEFF CAMPBELL BY TRÉS SMITH