Service-level agreements and key performance indicators are the most common ways to measure a vendor's performance. While these types of metrics are highly quantitative, they're fundamentally flawed in that they measure only a limited perspective of the overall value expected from a vendor. To measure real value, an IT department must develop appropriate metrics that quantitatively measure the more intangible aspects of vendor performance, such as commitment, innovation and flexibility—namely, the value of the business relationship.

To holistically measure overall vendor performance and value (overall value = tangible metrics + intangible metrics), a "balanced scorecard" is an ideally structured methodology. It looks at a number of weighted metrics both collectively and individually, and gauges how a vendor's performance-to-metrics is helping achieve the business goals of an organization, such as IT. From overall vendor performance measurement and value-for-money attributes, the balanced scorecard methodology examines four elements of performance: relationship, cost management, quality and delivery. Depending on the organization's needs and concerns, each of these elements will likely have multiple different measurements.

The following table is an example of the four balanced-scorecard elements and underlying metrics that could be applied to an IT environment. Nonitalicized text is an example of more traditional service-level agreements metrics or key performance indicators.

In attempting to "measure the immeasurable" through value-for-money metrics, each customer must seek out attributes that represent the most important considerations relating to commitment, flexibility and innovation. The following attributes can quantitatively measure value-for-money metrics.

Commitment

• Number of account management visits
• Special access to new developments within the vendor's research and development activities
• Tours of vendor facilities
• Access to vendor's sensitive information
• Access to vendor subject-matter experts
• Quality of vendor-customer executive relationships
• Trust ratio = promises made by vendor + promises kept by vendor

Flexibility

• Willingness or ability to respond to unanticipated demand
• Willingness to modify order entry systems or other vendor systems facilitating customer
• Flexibility of contract terms and conditions
• Ease of negotiation
• Willingness to change products or services to meet changing needs of customer
• Number of contract disputes

Innovation

• Joint research, design and development
• Sharing by the vendor of business improvement strategies that it has adopted, that it believes the customer could use
• Customer ability to participate on vendor's customer advisory board
• Progress of vendor in achieving relevant industry certifications (such as the Capability Maturity Model for software vendors)
• Continuous improvement ratio = ideas implemented by vendor / ideas suggested by vendor