Many methods of quantifying ROI are inherently flawed. These tips can help you calculate your real return on investment. Service-level agreements and key performance indicators are the most common ways to measure a vendor’s performance. While these types of metrics are highly quantitative, they’re fundamentally flawed in that they measure only a limited perspective of the overall value expected from a vendor. To measure real value, an IT department must develop appropriate metrics that quantitatively measure the more intangible aspects of vendor performance, such as commitment, innovation and flexibility—namely, the value of the business relationship. To holistically measure overall vendor performance and value (overall value = tangible metrics + intangible metrics), a “balanced scorecard” is an ideally structured methodology. It looks at a number of weighted metrics both collectively and individually, and gauges how a vendor’s performance-to-metrics is helping achieve the business goals of an organization, such as IT. From overall vendor performance measurement and value-for-money attributes, the balanced scorecard methodology examines four elements of performance: relationship, cost management, quality and delivery. Depending on the organization’s needs and concerns, each of these elements will likely have multiple different measurements. The following table is an example of the four balanced-scorecard elements and underlying metrics that could be applied to an IT environment. Nonitalicized text is an example of more traditional service-level agreements metrics or key performance indicators. Relationship Cost Management Quality Delivery Commitment On-budget delivery Quality/ expertise of staff On-time delivery Flexibility Discounted pricing Staff turnover Lead time/ Order cycle time Innovation Shared price reduction Order accuracy Response/ repair time Customer satisfaction Invoice accuracy No. of defects/ Conformance to specs Orders delivered completely Percentage of vendor-managed inventory Dead on arrivals/ Shipping damage Order costs Warranty returns Transportation/ shipping Mean time between failure/ repairs In attempting to “measure the immeasurable” through value-for-money metrics, each customer must seek out attributes that represent the most important considerations relating to commitment, flexibility and innovation. The following attributes can quantitatively measure value-for-money metrics. Commitment Number of account management visits Special access to new developments within the vendor’s research and development activities Tours of vendor facilities Access to vendor’s sensitive information Access to vendor subject-matter experts Quality of vendor-customer executive relationships Trust ratio = promises made by vendor + promises kept by vendor Flexibility Willingness or ability to respond to unanticipated demand Willingness to modify order entry systems or other vendor systems facilitating customer Flexibility of contract terms and conditions Ease of negotiation Willingness to change products or services to meet changing needs of customer Number of contract disputes Innovation Joint research, design and development Sharing by the vendor of business improvement strategies that it has adopted, that it believes the customer could use Customer ability to participate on vendor’s customer advisory board Progress of vendor in achieving relevant industry certifications (such as the Capability Maturity Model for software vendors) Continuous improvement ratio = ideas implemented by vendor / ideas suggested by vendor As illustrated by the list, converting seemingly subjective elements of intangible vendor performance into quantifiable and quantitative measurements that can be weighted and scored is realistically achievable. Looking at value-for-money metrics and their ongoing evaluation, both the customer and the vendor can see whether there is true “partnership-in-practice” behavior. The metrics also serve other important purposes, such as when categorizing vendors for purposes of vendor management, conducting a vendor rationalization program, determining which vendors to invest more time in, and developing vendor relationship rules of engagement. Formal account reviews with vendors should be conducted on a regular basis to discuss the balanced scorecard. The intent of the account review is to highlight the more strategic aspects of the relationship and not to rehash the vendor’s delivery performance (unless the performance has been deficient). The account review should be concise and brief, not inordinately lengthy in duration. A helpful structure for account reviews is the “6-6-60” format, in which the discussion is focused on the past six (6) months’ value-for-money metrics and other important aspects of the relationship as well as the vendor’s next six (6) months’ plans relating to continuous improvement and other relevant issues—all within sixty (60) minutes or less. Stephen Guth is the executive director of the National Rural Electric Cooperative Association’s Vendor Management Office. He is a Certified Commercial Contract Manager (CCCM), Certified Purchasing Manager (C.P.M.) and a Certified Technology Procurement Executive (CTPE).He is the author of The Contract Negotiation Handbook: An Indispensable Guide for Contract Professionals and The Vendor Management Office: Unleashing the Power of Strategic Sourcing. Related content feature Key IT initiatives reshape the CIO agenda While cloud, cybersecurity, and analytics remain top of mind for IT leaders, a shift toward delivering business value is altering how CIOs approach key priorities, pushing transformative projects to the next phase. By Mary Pratt May 30, 2023 10 mins IT Strategy IT Leadership opinion Managing IT right starts with rightsizing IT for value While there are few universals when it comes to saying unambiguously what ‘managing IT right’ looks like, knowing how to navigate the limitless possibilities of IT is surely one. By Thornton May May 30, 2023 6 mins Digital Transformation IT Strategy IT Leadership brandpost Designing the campus of the future starts with high-quality 10Gbps connectivity By Huawei May 30, 2023 4 mins Network Architect Networking Devices Networking feature Red Hat embraces hybrid cloud for internal IT The maker of OpenShift has leveraged its own open container offering to migrate business-critical apps to AWS as part of a strategy to move beyond facilitating hybrid cloud for others and capitalize on the model for itself. By Paula Rooney May 29, 2023 5 mins CIO 100 Technology Industry Hybrid Cloud Podcasts Videos Resources Events SUBSCRIBE TO OUR NEWSLETTER From our editors straight to your inbox Get started by entering your email address below. Please enter a valid email address Subscribe