Vendor Management: How Do You Measure Value for the Money?
Many methods of quantifying ROI are inherently flawed. These tips can help you calculate your real return on investment.
Tue, February 19, 2008
CIO — Service-level agreements and key performance indicators are the most common ways to measure a vendor's performance. While these types of metrics are highly quantitative, they're fundamentally flawed in that they measure only a limited perspective of the overall value expected from a vendor. To measure real value, an IT department must develop appropriate metrics that quantitatively measure the more intangible aspects of vendor performance, such as commitment, innovation and flexibility—namely, the value of the business relationship.
To holistically measure overall vendor performance and value (overall value = tangible metrics + intangible metrics), a "balanced scorecard" is an ideally structured methodology. It looks at a number of weighted metrics both collectively and individually, and gauges how a vendor's performance-to-metrics is helping achieve the business goals of an organization, such as IT. From overall vendor performance measurement and value-for-money attributes, the balanced scorecard methodology examines four elements of performance: relationship, cost management, quality and delivery. Depending on the organization's needs and concerns, each of these elements will likely have multiple different measurements.
The following table is an example of the four balanced-scorecard elements and underlying metrics that could be applied to an IT environment. Nonitalicized text is an example of more traditional service-level agreements metrics or key performance indicators.
| Relationship | Cost Management | Quality | Delivery |
|---|---|---|---|
| Commitment | On-budget delivery | Quality/ expertise of staff | On-time delivery |
| Flexibility | Discounted pricing | Staff turnover | Lead time/ Order cycle time |
| Innovation | Shared price reduction | Order accuracy | Response/ repair time |
| Customer satisfaction | Invoice accuracy | No. of defects/ Conformance to specs | Orders delivered completely |
| Percentage of vendor-managed inventory | Dead on arrivals/ Shipping damage | ||
| Order costs | Warranty returns | ||
| Transportation/ shipping | Mean time between failure/ repairs |
In attempting to "measure the immeasurable" through value-for-money metrics, each customer must seek out attributes that represent the most important considerations relating to commitment, flexibility and innovation. The following attributes can quantitatively measure value-for-money metrics.
Commitment
- Number of account management visits
- Special access to new developments within the vendor's research and development activities
- Tours of vendor facilities
- Access to vendor's sensitive information
- Access to vendor subject-matter experts
- Quality of vendor-customer executive relationships
- Trust ratio = promises made by vendor + promises kept by vendor
Flexibility
- Willingness or ability to respond to unanticipated demand
- Willingness to modify order entry systems or other vendor systems facilitating customer
- Flexibility of contract terms and conditions
- Ease of negotiation
- Willingness to change products or services to meet changing needs of customer
- Number of contract disputes
Innovation
- Joint research, design and development
- Sharing by the vendor of business improvement strategies that it has adopted, that it believes the customer could use
- Customer ability to participate on vendor's customer advisory board
- Progress of vendor in achieving relevant industry certifications (such as the Capability Maturity Model for software vendors)
- Continuous improvement ratio = ideas implemented by vendor / ideas suggested by vendor
