The Truth About Customer References: How Sweet Deals Can Be Rotten Business

Like many CIOs, Steve Pickett is happy to serve as a reference for his favorite vendors. If he spends a couple of hours talking to a prospective customer, the vendor might not bill his company the next time he needs a service visit not covered in his contract. He doesn’t benefit personally from the deal; the perks go to his company, which approves of the practice, so why not?

But when the time comes for Pickett to make a decision about buying his own software, he doesn’t place much stake in what his colleagues have to say?and he doesn’t always think that the CIOs who call him should either.

“Some of the companies that have called me have taken [the reference process] much too lightly,” says Pickett, vice president and CIO for Penske, a Detroit-based transportation services company. “If they’re calling me to find out whether a particular supplier is doing a good job, without knowing the

complexities within my company, I’m not sure that they’re learning anything. I may be giving them misleading information because their business is different from mine.”

Pickett says he’s up front about this. Nevertheless, he admits there is a natural tendency to avoid the negatives if things are going well. “If you have a good relationship with the supplier, you’re not going to bad-mouth them,” he says, though he’s careful to point out that if he’s receiving bad service from a vendor, he’ll say so.

All of which is to say, customer references come with baggage?big, unwieldy baggage. Vendors’ long-held practice of offering enticements to customers to speak to potential new customers is widespread and widely accepted by CIOs. Yet despite the fact that reference checks are possibly the most important step in the software selection process, CIOs generally know little about the references they call and even less about the terms of the arrangement between the reference and the vendor.

Companies that serve as references are likely to get things that average customers never get: preferential treatment, influence on the development cycle?even cash rebates. These customers aren’t average in any way and might not reflect the reality of your situation unless you become a reference too. And even if the reference isn’t getting special treatment, most software projects have become so big and complex that no two companies are likely to have the same experience anyway. All this means that customer references?especially in the realm of big enterprise software projects like ERP and CRM?have ceased to have much real meaning.

Worse, serving as a reference yourself can have tragic consequences for your company and your career. Accepting personal gifts or money for being a reference is unethical and can, of course, get you fired. But even if the gifts and special treatment benefit only your enterprise, not you, they can still violate your company’s code of ethics or jeopardize your company’s fortunes on Wall Street. Saying good things about bad or problematic software can send up a red flag with Wall Street analysts, who now track software projects more closely and downgrade company stocks at the slightest whiff of trouble. The reference game is enough of an ethical quagmire that the federal government has banned its IT executives from making endorsements of any kind.

The government may be on to something. The benefits that companies gain from acting as references for vendors may no longer justify the risks?both personal and companywide?that are part of the package. In fact, the only viable reason for agreeing to be a reference for a supplier may turn out to be the simplest one of all: because you think it’s good.

The Care and Feeding of You

According to one survey by the IT Services Marketing Association, 52 percent of IT buyers make purchase decisions based on referrals from colleagues. If you don’t have a buddy you can call for a reference, vendors are ready to help. They enlist and groom armies of happy reference customers to take calls about their experiences. “There’s a science to the care and feeding of these references,” says Analyst Jim Shepherd of Boston-based AMR Research. That’s because references are the cornerstone of the software sales cycle.

Being a reference can be informal and simple?such as agreeing to be listed on the vendor’s website or doing the occasional favor for the vendor’s sales team. Or it can be quite complex, with specific agreements written into the software contract between the reference and the vendor. The contract might stipulate taking a certain number of calls from prospective customers, the press or analysts, for example, or speaking at a user conference, or approving a certain number of press releases and case studies about the implementation.

At large software companies like Pleasanton, Calif.-based PeopleSoft, managing these references is a full-time job. In fact, it’s five full-time jobs, overseen by Wendi Wolfgram, program manager for the user reference program, for which more than 700 companies have signed up. Wolfgram and her staff keep a database of information about those customers, including details about each installation, multiple contacts in different parts of the business, and the customer’s level of commitment to and history of serving as a reference. When a salesperson needs a reference for a prospective customer, the reference team looks for a good match?a company in a similar industry that’s not a direct competitor, that has a similar shop and that hasn’t been overused as a reference. According to Wolfgram, the program is so valuable that it affects more than 78 percent of deals closed each quarter. And it’s so valuable that PeopleSoft puts a great deal of time, effort and money into keeping it running smoothly.

“We do have rewards, and it’s kind of like a frequent flier program,” says Heather Loisel, PeopleSoft’s vice president of global sales productivity. “It’s completely voluntary, and it kicks in after a certain level of volunteer activity. The reference earns points, and those points can be used for things like a pass to one of our member conferences.”

Givers and Takers

Free conference passes are only the beginning. “You usually get something out of it,” says Eastman Kodak CIO Mark Gulling, who has gotten free consulting and, from SAP, a rebate for serving as a reference. He won’t give specifics but says the rebate is minimal and benefits his Rochester, N.Y.-based employer, not him. “[Vendors] understand that it takes time for you to do this, but that’s not why you do it,” he says. “It gives me contacts that I can go back and talk to later.”

The vendors CIO spoke with for this story?i2 Technologies, Manugistics, Oracle and PeopleSoft?all downplayed these incentives. (Other competing vendors declined to be interviewed for this article.) “We don’t want to have a program in place where the reward is so large that the customer says, I’m going to go after that award,” says Katrina Roche, chief marketing officer for Dallas-based i2 Technologies. “We want people to know we appreciate them, but we don’t want that appreciation to sway that decision.”

But it would be naive to assume that shady reference deals aren’t cut in the software industry. Roche remembers working for one small software vendor that offered a customer a $7,000 discount to serve as a “marketing center”?participating in ad campaigns and the like.

Dave Munn, CEO of Lexington, Mass.-based IT Services Marketing Association, says typical reference deals involve credit toward training classes or a future purchase. And a check in the mail? “Clearly that happens,” he says.

Patti Morrison, CIO of Office Depot in Delray Beach, Fla., and former CIO of GE Industrial Systems and Quaker Oats, does not count herself among the takers. But she does feel pressure from vendors that want her to put a reference agreement in the software contract?especially smaller startups trying to solidify their position in the marketplace with references so that they can get financing. “They say, If you do a press release by the end of our first quarter, we’ll give you X percent discount,” Morrison says. “They might use it as a negotiating tactic for price, but that [discount] is not a good decision on which to make the choice of that vendor.” Morrison, whose company has a policy of not putting reference agreements into contracts, tells them that she’ll serve as an informal reference?only after the project is complete and has been successful. But even in this reduced role she still benefits, albeit in a roundabout way.

“It’s an additional incentive for the vendor to support you,” she says, noting that serving as a reference sometimes gets her better customer service, including access to early software releases, and better response from the vendor’s senior managers and software developers. “It’s a great way to build a relationship with the supplier [and get] their skin in the game to make you successful.”

Blinded by the Lighthouse

Among companies that agree to link their name to a vendor, some have more value as references than others. These are the giants, the companies people instantly recognize, the brands they trust. A short jaunt through any vendor’s annual report, conference lineup or website makes it clear which customers are so-called lighthouse accounts: Procter & Gamble and NestlŽ for SAP; Charles Schwab and Deutsche Telekom for Siebel; Dell and Home Depot for i2 Technologies.

These lighthouses can help vendors gain legitimacy, sell products and boost shareholder confidence by proving that the vendor has a steady paycheck from a company with deep, well-constructed pockets. But they can also break their vendors, by talking up a project and then pulling back on it. Last February, for example, TheStreet.com reported that Manugistics’ stock price fell 18.5 percent reportedly after a research analyst said that one of Manugistics’ lighthouse customers, Ford Motor, had put some IT projects on hold. Manugistics’ CFO Raj Rajaji had to assure nervous investors that Lloyd Hansen, Ford’s vice president of revenue management, was speaking at an upcoming user conference?proof that the partnership was still strong.

How Happy Talk Works

A reference may sound good, but it’s rarely an accurate representation of reality. Companies that agree to serve as references are self-selected, happy customers, with a vested interest in the vendor leading a long, productive life. “You want your vendor to be successful and gain resources and clout so that they can improve their software,” says Wendy Close, an analyst for Stamford, Conn.-based Gartner. This is one incentive for a CIO to become a reference?but it’s also an incentive for a CIO to be a positive reference.

References also use their position to influence the direction of the technology. “If I do things for [vendors], I end up having some influence on them in terms of the future direction I’d like them to take,” says the CIO of a retail company who spoke on condition of anonymity. When asked if that makes him less valuable as a reference, there’s a long pause. “An average customer would have to think that customers like me have their best interest in mind,” he says finally, noting that most prospects don’t ask about his relationship with the vendor.

It’s worth taking the time to understand that relationship. In some cases, a company may form such a strong bond with a vendor that the company becomes something more than a customer if something less than a partner. But you have to dig to find that out for yourself?vendors won’t spell it out for you.

Consider, for example, Charles Schwab’s affiliation with Siebel. The brokerage has long been prominent in Siebel’s marketing materials. And a casual visitor to its website would assume that Schwab is nothing more than a good customer. But the CEO of Schwab has been on Siebel’s board of directors since 1994, and back in 1996 the brokerage accounted for fully 23 percent of Siebel’s revenue, according to an SEC filing. In other words, assuming that Schwab’s experiences with Siebel will predict yours would require an imaginative leap.

What Is Your Word Worth?

Of course, there’s nothing inherently wrong with forming a strategic partnership with a trusted vendor. And there’s not necessarily anything wrong with accepting something in return for providing a reference either. But companies had better be willing to talk about it, ethics experts say.

“If a customer is unwilling to be transparent [about the rewards], then the conclusion that a disinterested party might draw is that the person has been bought,” says Daryl Koehn, director of the Center for Business Ethics at the University of St. Thomas in Houston.

So what can CIOs ethically accept in return for serving as a reference? Ideally, nothing, says Koehn. “If the vendor’s products are good, the customer should be willing to make a recommendation for that vendor free of charge, right?” she asks. “The question is, Why does the customer feel entitled to demand some kind of recompense, or why does the vendor feel the need to offer some kind of recompense?”

If the reference process doesn’t take much time, the reference shouldn’t take anything in return, Koehn suggests. But “if you’re talking about the company’s employees being tied up for days, it’s not necessarily inappropriate for the customer to get something in return, like a small discount” or free training that takes up roughly the same time as the site visit, she says.

Regardless, the CIO had better believe in the product without reservation. That’s why some companies set strict parameters around referencing. For instance, The Hartford Financial Services Group has a policy forbidding its IT group from negotiating better prices or accepting free services in return for acting as a reference. “We pay for the technology and the services that we use in dollars, not in references, not in endorsements,” says a spokesperson for the $15 billion insurance company in Hartford, Conn.

“If we’ve had a good experience with a vendor, we will occasionally share what the results were, but we try to make sure that nothing could get misinterpreted,” says Hartford CIO David Annis. That means making sure that comments made during the reference process could not be interpreted as an endorsement.

The federal government has barred its employees from endorsing vendors and accepting any gifts unless they go through an elaborate legal approval process. “A government employee who holds the decision-making authority cannot personally benefit from that authority,” says Mark Forman, associate director for IT and e-government for the U.S. Office of Management and Budget. What’s more, he says, the department as a whole cannot benefit from that kind of arrangement either. “I would consider that inappropriate,” says Forman, the federal government’s de facto CIO.

Getting at the Truth

If the reference process is so troublesome, then how can CIOs make sure they’re not getting hoodwinked? You can protect yourself; first, put the software through its paces as much as possible. Then, look for a trusted peer to talk to you about the software before you go to the vendor. That is why it’s so important to stay in touch with your peers, either by attending conferences or joining a professional organization. “It’s your personal relationship with the CIO that you have to leverage to find out what really happened. You won’t get that out of a formal reference call,” says the anonymous CIO.

When he does talk to vendor references, this CIO asks for contacts at specific companies on the customer list rather than accepting the ones the vendors try to give him. When you call the phone numbers, ask the companies what they did to get on the list. “It’s entirely fair to ask, What have you received from the vendor in exchange for allowing me to sit here in your conference room?” says Andrew McAfee, an assistant professor in technology and operations management at the Harvard Business School, who is doing research on the CIO decision-making process. “I would imagine it’s a question that gets asked less often than it should.”

Merely knowing there are references is not enough. Rob Cohen, vice president and CIO at AstraZeneca, the London-based pharmaceutical company, says that two-thirds of the time, when he agrees to talk to a prospect, the company never calls. This suggests that some companies skimp on the homework?and that’s dangerous. “I can’t tell you the number of times where [vendors have] trotted out some very big name, but the reference is only using part of the product in one department,” says Jim Sutter, senior partner at The Peer Consulting Group in Newport Beach, Calif., and former CIO of Xerox and Rockwell International.

Companies that decide to do a site visit should send in people who will be on the front lines of the implementation, not just the CIO, Sutter recommends. “Particularly to a peer, [the CIO reference] will be reluctant to say, I really messed up when I made this decision,” he says. “If you send some technical people in, they will uncover what worked and what hasn’t worked.”

Then, during conversations, ask lots of questions?carefully, to extract facts, rather than opinions or generalities. For instance, you might ask, “How long does it take for them to get back to you? Not just, How’s their service?” AstraZeneca’s Cohen says.And remember Pickett, the reference who wouldn’t listen to himself if he were making a purchase decision? That’s not to say he finds references useless. He just won’t use them to make a decision. Pickett has found that they’re really only helpful once he’s made up his mind.

“If it’s a highly valued customer, I’m going to find out from them what extra mile [the vendor was] willing to go to get [them],” Pickett says. He’ll probe to find out, for instance, what the reference was able to get over the standard vanilla contract. Then, he’ll ask for tips on how to actually implement a product?not pick it. “It’s free advice if you use it correctly,” he says.

And if you don’t? Well, you probably won’t be worse off for following your own mind. After all, that’s what Pickett does.