Buyer beware: How auditing RFPs can help you make smarter enterprise software purchases
Aggressive salespeople and “over-optimistic” RFPs can doom enterprise software projects. Audit RFPs to verify they meet requirements as claimed before purchasing the software, and avoid software disasters.
When buying enterprise software, organizations usually rely on vendors responding to RFPs (or RFIs or RFQs). However, as real world experience shows, responses to RFPs do not always reflect software reality. In all cases of RFP inaccuracies, the buyer pays the price in more ways than one. Why does this happen and what can you do about it? Two reasons are:
Poorly written RFP requirements. Salespeople always interpret unclear requirements in a way most favorable to their product.
Over Optimistic RFP Responses. Some salespeople are simply too aggressive when responding to RFPs.
The first problem can be remedied to some extent by the creator of the RFP. See this post on writing better requirements. The second problem is more insidious, remaining undetected until implementation or customer acceptance tests, when you discover the software does not truly meet the requirements. By then it is too late to do anything about it because the purchase has been made. While you can build some protections into the purchase contract, it is far better to avoid the problem in the first place.
At this point in the project, you may have done a thorough evaluation of potential products and selected the best-fit software for your organization’s particular needs. However, you are still relying on the vendor accurately responding to your RFP requirements. The way to uncover inaccurate and “over-optimistic” responses is to audit the RFP to verify the software does indeed perform as expected.
When aggressive sales people make “over-optimistic” RFP responses, the audit exposes what appeared to be the best-fit software for what it actually is. The true best-fit software can then be selected. The audit has caught the problem before making the purchase, and the organization avoided a potential software disaster.
Auditing an RFP
When initially entering RFP responses from vendors into the evaluation, mark each response to show the rating method, which documents the source of that rating. For example, since the RFP responses are typically by email, and any subsequent clarifications are by email or phone, we usually mark vendor responses as “Verbal or email.”
Ask the vendor for evidence that their software does indeed meet your requirements. Marketing materials are seldom detailed enough for RFP audits. Rather you need the following:
User and admin documentation that describes features, and how to configure and use them
Vendor demo videos showing how to use features
An online account where you can login and test if features adequately satisfy your requirements
To audit an RFP, extract all “Showstopper” and “Critical” requirements where the vendor claims “Fully meets.” By definition, all these requirements have a fit score of 100 percent. See previous post on RFP scoring. For each requirement in the extract, examine the evidence to verify the software adequately meets it. If a requirement is inadequately met, reduce the rating appraisal from “Fully meets” to something lower, e.g. “Partly meets” or even “Does not meet.”
After auditing a requirement, update the product rating method to reflect this change, for example to “Audited” in the screenshot below. Add supporting evidence in the rating comments, for example, include a quote and a link to the source documentation. If the feature was tested by logging into the software, include a description of the test and the results.
The final step of the audit is to filter out everything except the audited product ratings. Recall that the fit score for the requirements in the extracted sample was 100 percent because all requirements are fully met. If you have audited a large enough sample of requirements and the fit score decreases significantly, you can assume the fit score for all requirements will decrease by a similar amount. We have seen products drop from ranking first place to third place or lower. On the other hand, if the fit score remains substantially the same after the audit, you can be confident that the RFP accurately reflects how well the software will meet your organization’s requirements.
Once a software product has passed the audit you can proceed with the purchase, confident that it matches your organization’s needs. You have substantially reduced project risks, and can take the next step forward in a successful enterprise software implementation.
The next post describes using product ratings to manage software implementation costs.
Chris Doig graduated from the University of Cape Town, South Africa with a bachelor of electrical engineering degree. While at university, he founded Cirrus Technology to supply information technology products to the corporate market. The focus at Cirrus was helping companies buy the best IT products for their particular needs. Cirrus also developed custom software for the South African 7-Eleven franchise holder and other corporate clients.
In the 1990s, Chris immigrated to the United States and worked at several companies in technical and IT management roles: Seagate, Biogen, Netflix, Boeing, Bechtel SAIC, Discovery Communications and several startups. At all of these companies he repeatedly saw software being purchased with an immature selection process. Invariably this software would take longer to implement than planned and cost more than budgeted. To make matters worse, the software seldom met expectations.
Having struggled with software selection himself, Chris founded Wayferry, a consulting company that helps organizations acquire enterprise software. He is also the author of Rethinking Enterprise Software Selection: Stop buying square pegs for round holes. While ERP projects account for much of Wayferry's work, other types of enterprise software acquisitions include CRM, HRIS, help desk, call center software, clinical trials management systems and so on. For Chris, the ultimate satisfaction is when clients report meeting or even exceeding expectations with their new software.
The opinions expressed in this blog are those of Chris Doig and do not necessarily represent those of IDG Communications Inc. or its parent, subsidiary or affiliated companies.