No Comparions: The Outsourcers’ War on Benchmarking

When Darius Jackson became ING’s head of IT infrastructure support and service delivery in January 2005, his job was to clean up a mess. Two years earlier, the financial services company had outsourced its IT infrastructure (hardware, software, help desk and so on) to a major service provider in a seven-year, $600 million deal. But now the business leaders of the company are worried that they aren’t getting the value they want out of the relationship.

Jackson has a crowbar for leveraging more value out of the deal, though he hasn’t used it.

Yet.

It’s called the benchmarking clause, a small paragraph hidden in a vast, complex outsourcing contract that gives him the right to assess the outsourcer’s prices using an independent benchmarking firm. If those prices turn out to be way above the going rate, he could bring his outsourcer back to the negotiating table. The outsourcer probably wouldn’t be too pleased with Jackson should he choose to exercise that right. “They like to look at the relationship as a partnership,” says Jackson. “Their preference is that you come and talk to them about issues and concerns and try to work through them as opposed to looking under the covers.”

RELATED LINKS

Outsourcer Benchmarking: The Sanity Clause

Outsourcing Contracts: Clause Control

Patent (Fight) Pending

Still, Jackson admits, “having [the benchmarking] option is invaluable.”

But what Jackson and other IT leaders may not know is that the power of the benchmarking clause, which is hidden inside most outsourcing agreements today, has diminished and the practice of price benchmarking is in danger of disappearing altogether. “Suppliers have scant enthusiasm for benchmarking, which shaves their margins and tends to be invoked just as their contracts start to become profitable,” says George Kimball, a partner with law firm Baker & McKenzie who represents outsourcing customers. Not surprisingly, benchmarking clauses are among the most contentious and negotiated terms in an outsourcing contract, says Kimball.

In fact, the major IT service providers have launched an all-out assault on benchmarking, fighting to turn it to their advantage. “[They are restricting] what you can benchmark, how often you can benchmark and the amount the benchmark can reduce the price of services,” says Mark Robinson, executive director of advisory services sourcing consultancy EquaTerra.

Other experts say the big outsourcers are looking to avoid the process altogether. “They want to stonewall it,” says Howard Rubin, a senior adviser to research company Gartner.

All of the major providers we spoke to for this story—EDS, Hewlett-Packard and IBM—deny that they are out to kill benchmarking. However, all express frustration with the current state of the practice and all are trying to change it—to their benefit. “Benchmarking isn’t ever going to go away, but I think it will change,” says Jon Stewart, VP of market management for Electronic Data Systems (EDS). “I think we need, first and foremost, more credible [benchmarking firms]. There’s not a lot of players in that space. Second, there needs to be some recognition of what you can realistically expect from a benchmark. The old mind-set is, ‘Ah, I’ll go to a benchmarker and they will have a robust view of the market.’ They don’t.”

If the power of the benchmarking clause diminishes, CIOs will concede one of the only tools for controlling the cost of outsourced services. The risk with any outsourcing contract is that you end up paying through the nose for services that should be getting cheaper, particularly in the infrastructure area, where prices for hardware are dropping constantly. “If you leave it up to the vendor, there won’t be any benchmarking. Your next shot to adjust prices is when the deal’s up,” says Robert Finkel, partner with law firm Milbank, Tweed, Hadley & McCloy.

The Birth of the Benchmarking Clause

The benchmarking clause dates back to the mid-’90s, when the number of mega-outsourcing deals began to explode, along with the lengths of the agreements. Signing a 10-year deal with a multibillion-dollar price tag was a big risk—one that customers wanted to mitigate.

In the beginning, says Kimball, benchmarking clauses were kinder and gentler—designed only to bring outsourcer and customer together to confer diplomatically if prices seemed higher than market averages. But as the decade drew near a close, CIOs and their business colleagues got nervous. They wanted some assurance that increasing competition among the outsourcers and constantly dropping costs of computing power would be reflected in what they were paying. Outsourcing consultants and external counsel were only too happy to oblige. Midwifed by such advisers, new benchmarking clauses emerged. With teeth.

The clauses not only enabled periodic benchmarking, they forced a remedy. Should charges fail to fall within the bottom 10 percent of the market, the supplier had to lower its prices. Some more aggressive clauses even required retro¿active reductions. “They were planned to be punitive to the providers,” says Rubin.

In the vendor feeding frenzy of the time, outsourcers signed these customer-centric provisions left and right. “Back then, it was all green pastures and room enough for everyone. Outsourcers just chased deals,” says Stewart. “There wasn’t a lot of thinking about how the market might change in the future.”

A few years later, however, when some large customers began to flex their benchmarking muscles, outsourcers felt the pain. In one of the most extreme cases in 2002, Britain’s Cable & Wireless sued IBM for more than $200 million for price gouging after a disputed benchmark. IBM counter¿sued and brought Cable and Wireless’s benchmarking company, Compass, into the lawsuit. They ultimately settled out of court. The outsourcers suddenly woke up, as if from a nightmare. “They started to say, What the hell have we agreed to?” says Geraldine Fox, global sourcing practice lead for Compass. “The price standard they were being held to—the lowest decile—was ridiculously strict.”

Why Vendors Hate It

Attribute the sharpness of the bite to the emergence of benchmarking companies around 1998. Benchmarkers had been evaluating IT shops’ internal operations since the late ’80s, but when the outsourcing boom hit they came up with an enticing pitch for CIOs: Let us invoke the benchmarking clause, and we guarantee to cut your prices. With benchmarks costing anywhere from $100,000 to a million dollars, depending upon scope, it was hard for benchmarking firms to seal the deal without pledging to get at least some of that money back. But that promise poisoned their objectivity, some say. “When you say we guarantee to save you 20 percent, you’re not really being objective,” says Adam Strichman, senior partner at research and benchmarking company Nautilus Advisors and former director of outsourcing strategies at Meta Group.

The outsourcers screamed foul. The whole benchmarking process, they contended, was a crock: Service provider cost models are complicated; there are financial and operational dependencies among different services; outsourced services are rarely commodities but rather are sold in varying combinations upon a wide array of terms. For example, the price for mainframe services can vary by plus or minus 40 percent, says Stewart of EDS, depending on what software is running on the machines. “You have to understand the client environment, service levels and the financial assumptions that went into the deal to come up with an accurate benchmark,” he says. “And benchmarkers don’t know the deal specifics.”

Furthermore, say the outsourcers, the benchmarkers’ attempts to account for the variations among companies—what is euphemistically known as “normalization” in the industry—isn’t an adequate substitute for real numbers and knowledge. “The vendors complain benchmarking is more art than science,” says Robinson of EquaTerra. “And a black art at that.”

The Benchmarking Business

Benchmarkers, whose overall market now exceeds $200 million per year, according to one industry analyst who asked not to be named, concede some of these points. “There is an art to it,” admits Strichman. “Benchmarking is far more than statistical analysis and number-crunching.” Benchmarkers say that over the years they have refined their processes to reconcile unlike data and adjust for differences such as the client environment and service levels.

But while benchmarking may require some apples-to-oranges comparisons and fact-finding to adjust for different environments, it’s the only method currently available for CIOs to ensure the competitiveness of the prices they’re paying. “There’s value in the benchmarking clause. It gives a customer security when signing a long-term contract with a single provider,” says Neil Barton, Hewlett-Packard Services’ benchmarking manager for Europe.

And while outsourcers complain that the benchmarkers’ numbers don’t add up, they are loath to admit that their own numbers don’t add up either. Service providers typically grant customers a great price on IT services on day one but backload their costs to recover initial investments later on. Compass reports seeing year-one savings of as much as 18 percent (usually 10 percent to 15 percent) turn into costs in excess of 23 percent above market rate by year three and more than 35 percent in longer deals. Outsourcers aren’t likely to open up their books and show you how they’ve arrived at your charges, so benchmarking is an important tool. “Whatever its limitations,” says Kimball, “benchmarking can be an effective catalyst for renegotiations that raise service levels and reduce charges.”

The Vendors Strike Back

A few years ago, major outsourcers began taking active steps to restrict benchmarking. They created dedicated teams to analyze costs and negotiate benchmarking clauses to their advantage. “What was once just grumbling turned into organized grumbling,” says Strichman. “Now every major service provider has a group that manages the process.” These groups are charged with analyzing internal intelligence; benchmarking to calibrate their pricing models (the outsourcers use the benchmarkers’ services too); and managing the process when a customer decides to invoke its benchmarking right.

Alan Yamamoto set up such a group at IBM five years ago. True to Big Blue form, the company has filed a patent in this area (see “Patent (Fight) Pending,” this page). Meanwhile, IBM’s arch rival, HP, has created a team, as has EDS. “[Vendors] definitely have gotten more vigorous in protecting their interests,” says Stuart Harris, a partner with outsourcing consultancy TPI. “I can’t say they’re systematically obstructive, but some of their clients might.”

One of the tactics these teams use is to make a benchmarking clause so specific in its requirements that it becomes too difficult or expensive to invoke—for example, requiring too many peers in the benchmarking group (five is generally plenty, say benchmarking companies). Other terms the outsourcers may seek include negotiating a detailed limit on how soon or how often the customer can benchmark, the opportunity to review and negotiate draft findings, and caps on mandatory reductions in charges, among others.

Shutting Off the Data

But perhaps the most obstructive effort by outsourcers so far is their attempt to stymie benchmarker data gathering. In order to build their market cost estimates, benchmarkers need to pool data from many different outsourcing customers. Since the beginning of the benchmarking era, outsourcers have allowed benchmarkers to reuse the data they gather during their benchmarking engagements as long as they agree not to reveal customer names.

Yet within the past year and a half, vendors have begun asking benchmarkers to sign a legally binding document promising that they will not reuse the data they gather from the outsourcer’s customers, thereby preventing the benchmarkers from making comparisons across companies—the very essence of what they do and the foundation for the service they provide their customers. Indeed, some providers, such as IBM and EDS, have banned data reuse—with rare exceptions for particularly large and determined customers. This is despite the fact that most service providers use third-party benchmarkers themselves to construct and maintain competitive deals. “If one believes in the reuse of pricing data, you have some obligation to permit that to happen,” says HP’s Barton, noting that HP employs the services of benchmarking companies Compass, Gartner and Germany-based Maturity Consulting. HP is in the process of “reviewing” its data reuse policy, he says.

EDS’s Stewart sees the contradiction: He complains that the benchmarkers lack sufficient data to make good pricing estimates, and now he’s exacerbating the problem. He doesn’t much care. “The reality is I have better data than the benchmarkers do,” Stewart says. “We participated in 9,000 deals last year. We have much more robust information. We don’t need [benchmarkers] to define our view of the market.”

But CIOs do. And the trouble is, many of them buy the outsourcers’ pitch that by preventing reuse of data, they are simply trying to protect customers’ privacy. “The client will say, ‘That sounds scary. Thanks for bringing it up,’” says Nautilus Advisors’ Strichman. “They don’t see the agenda underneath.” If every customer agreed to such restrictions, price benchmarking would cease to exist.

Meanwhile, consolidation has left just a handful of big, full-service outsourcers—all with tremendous power in the marketplace. IBM, for example, has the biggest market share of any outsourcing company, more than double that of its nearest rival, EDS. The two companies’ combined market share is more than their four next nearest rivals combined, according to Gartner. Other major providers may ban the reuse of data now that the two biggest players have started moving in that direction. If CIOs don’t want the benchmarking clause to go away, they will need to take a more active role in negotiations.

Don’t Mess with Texas

Keeping benchmarking alive was top of mind for leaders at the Texas Department of Information Resources (DIR) before they signed a seven-year, $863 million data center services contract with IBM in November 2006. “Benchmarking was a hot issue for us. Our belief was that it would allow us to have the best insight into how our deal compares to market pricing,” explains Kim Weatherford, director of statewide technology operations. “Over time, technology [improvements are] going to drive down rates for various [services], and we want those rate reductions,” he adds.

During the state-mandated bidding process, Weatherford noticed that all the vendors seemed allergic to the concept. Indeed, while most issues were resolved before final negotiations with the winner, IBM, benchmarking was not resolved until the final days of negotiations. “IBM had experienced problems with [benchmarking],” says Weatherford, “but we worked hard to get language in there that allows us to do it regularly.”

Initially, Texas went old school on the clause, seeking automatic rate reductions for charges that did not fall in the lowest 10 percent of market pricing. IBM pushed back and raised it to the lowest quartile. IBM also demanded a cap on annual pricing adjustments: no more than 5 percent in discounts in years two through four and no more than 7 percent in years five through seven. The clause allows the state to benchmark annually and even includes language that compels IBM to waive its ban on data reuse.

Weatherford admits that the dollar value of the state’s contract went a long way toward getting a more balanced benchmarking clause. But any determined customer can and should secure similar benchmarking rights, he insists. “You really have to know what you want and what outcomes you’re willing to live with, and put that on the table,” says Weatherford. “The outsourcer will figure out a way to give it to you; it’s just a matter of money.”

An Annual Affirmation

Campbell Soup Senior Vice President and CIO Doreen Wright brings in benchmarkers once a year to benchmark specific services and technologies that have been outsourced in her 10-year deal with IBM. “In some cases, we find we’re paying too much and in others too little,” she says. The important thing, Wright says, is that Campbell spent a full year on the latest contract renegotiations with IBM (the original deal dates back to 1995) to get initial pricing right, determine which services should remain with IBM rather than come back in-house, and ensure benchmarking rights for the length of the deal.

Ironically, Wright says she insists on benchmarking because she has a good relationship with IBM. “They’re an extension of our own team, and we can lose objectivity,” she explains. “So we need to bring in an objective third party who bases their assessment on facts.”

The issue of data reuse is trickier. There’s certain data IBM will not allow the customer to share with the benchmarker, says Andy Croft, Campbell’s vice president of global services. “We’re not crazy about sharing corporate data either,” says Croft. “But we realize the whole industry needs to benchmark.” And the process itself is arduous. In fact, Wright had to skip benchmarking one year because Campbell was rolling out a new SAP system in Canada. Still, “just having [the benchmarking clause] in there is healthy for the relationship,” says Croft. “It makes a lot of implicit things explicit. And it eliminates the vendor’s entitlement mentality.”

The Forecast: Uncertain

But in the battle to preserve benchmarking, Campbell Soup and the state of Texas are likely exceptions rather than the rule. Benchmarking is on the ropes. “Benchmarking is an arena in which both buyers and vendors have an opportunity to play games,” says Cynthia Beath, professor emeritus at the University of Texas-Austin. “If I were to bet on the outcome of any outsourcing game, I’d put my money on the vendor, who plays more frequently.”

Benchmarking may be imperfect, but pricing in outsourcing contracts is opaque to CIOs, who may negotiate only one or two of these deals in a lifetime. “No one can make heads or tails of outsourcers’ pricing. CIOs have no idea—is this a good price or a bad price?” says Strichman.

Just as benchmarking is becoming less of a sure thing, he adds, there’s “even more need for it.”