How to Measure Vendor ROI

A vendor appears at your doorstep with impressive ROI claims. A golden justification opportunity or a trap for the too-trusting? Embracing vendors’ cost-benefit aid is tempting in today’s “prove it or lose it” environment. Unfortunately, dangers abound. The risks in using vendor input are many?so CIOs must proceed carefully. My experience indicates that for every buyer who successfully uses vendor ROI input, there are four purchasers who risk sabotaging both their project justifications and their reputations. Tolerating suppliers that are (intentionally or not) oblivious, incompetent or misleading in their ROI words and deeds is ill-advised. Here are five tips for vetting vendor ROI claims and leveraging them for your own gain.

Know Thy Enemy

1. Two types of vendors are to be avoided: those with misleading ROI claims and those that say nothing about ROI at all. Warning signs of such value-challenged vendors include feature-laden demonstrations, jargon-invested proposals, unsubstantiated claims and too much focus on the technology, rather than on business value. Be especially alert if the vendor executives themselves appear value-adverse. When they can’t talk convincingly about ROI, most likely neither can their troops. Even if their field folks can manage some occasional ROI-speak, chances are it won’t be reliable. Fact is: Business-value focus reflects the culture of an organization. And a culture of pro, anti or neutral ROI talk begins at the top.

Ban Practices That Attract Bad Guys

2. Too often buyers employ evaluation techniques that unintentionally open the door to ROI-weak vendors. Here are four methods to avoid:

• Feature-focused, rather than value-focused, requests for information and RFPs. If you insist on 105 whizbang features, with minor attention to business benefits, don’t be surprised when ROI-challenged vendors respond with bits- and bytes-laden proposals.
• Excessive conversations about features and functions. Don’t let your evaluation people spend 95 percent of their vendor conversation time discussing those things. Instead, insist they spend quality time learning from the vendor, beginning early in the evaluation cycle, on what true business value is possible, where such value resides, how it can be measured, how to make it happen and who says so and why.
• Avoid using obscure and opaque decision-making criteria and methods. Such methods become a breeding ground for vendors that prefer behind-the-scenes “buddy” talk to publicly scrutinized business-value discussions. In addition, murky decision-making practices are totally out of step with today’s push for more visible and trustworthy IT governance. If you have a well-understood and transparent process for making decisions internally, vendors will also be expected to conform.
• Be intolerant of FUD arguments from ROI-weak vendors. Be clear that purchases made in your shop are based on manifest value, not market dominance.

  Look for Backup Data

  3. Vendors with ambiguous ROI claims undermine business-case integrity since their ROI assertions appear real but aren’t. A common sign of misleading value statements are claims that have no data backing them up. For example, an impressive advertisement or case study may tout that “Client A obtained a 125 percent ROI.” If you see no explanations, rationale or footnotes that lead to a clear explanation, ring up a “no sale” for that vendor. By the way, hold industry analyst firms to this same standard. Just because such advisers may have an impressive client list doesn’t mean any given ROI statement is a fact. If their publications don’t keep this guess-versus-fact distinction crystal clear, call them on it. Always ask for specifics and any hard numbers they can give you. The best way to determine if any claim is valid is to go directly to the source to learn what the claim is based on.

  Hold Vendors to a Higher Standard

  4. Take the attitude that an ROI claim must be held to a four-way test of tangibility, understandability, measurability and relevance. For example, I know of one collaboration management solutions vendor that claims its client has recouped its investment cost within the first two months of implementation just from saving employees time in manually managing Web seminars. Close examination of the vendor’s claim reveals that the savings are tangible, clear, measurable, link directly to the vendor’s solution and are substantiated in writing by the manager directly associated with the project. If the environment upon which such claims are based is analogous to yours, then this type of claim is eligible for confident placement in your business case.

  Leverage the Good Guys

  5. ROI-savvy vendors can boost the cost-effectiveness of your entire evaluation process. I overheard one CFO tell a vendor, “Had you come to us [with your ROI tools] earlier, you would have saved us six months of lost vendor and product evaluation time and effort.” Another client asked a vendor to conduct an ROI discovery workshop that unearthed $16 million in benefits, enough to get the application approved by an ROI-obsessed board of directors. Using vendors that know their ROI can provide you with faster and more comprehensive assessments, greater confidence in your recommendations, and fewer funding disappointments. When it comes to IT projects, an ounce of business-value focus is worth at least a pound of features and functions. Reward such suppliers by giving plenty of evaluation points to those who demonstrate such ROI intensity. Once you have collared an ROI-aware vendor, use it for all it’s worth. Request the company’s help for such activities as researching relevant benefits, getting interview data, providing business-case creation tools and supplying samples from other clients. After all, most vendors will fly multiple sales and demo support people to you at the drop of a hat. Ask them for the same level of support for your all-important business case. If they are hesitant, maybe their priorities are misplaced.

  Good vendors often bear gifts of ROI help. Distinguishing the trustworthy suppliers from the fraudulent or clueless is the first crucial step toward ultimate project success.