Computerworld — It's not quite the Consumer Reports of the IT world yet, but VendorRate.com wants to make it easier for IT users to find and hire their next IT vendor.

Since going live in March, the site has given business IT users a place to register comments and reviews for some 12,000 IT providers. The vendors are rated on 10 qualities, including integrity, reliability, expertise, communications, budget and timeline, so that others can determine if the listed vendors are a good match for upcoming IT projects.

"Vendors who are really, really good love us," said Richard Schaefer, CEO of the Los Gatos, Calif.-based site. "There is no other place to get this level of information that can be trusted."

While IT and business analysts can provide similar information, he said, they may not be as objective—especially if they rely on sponsored research paid for by vendors.

Reviewers are allowed anonymity so they can freely share opinions, while vendors are protected through the use of an algorithm, or "rating engine," that analyzes the ratings and corrects them for inconsistent input from users.

IT users register on the site using their business e-mail address, which is verified by VendorRate.com to ensure that the user is qualified to rate IT vendors and services. In addition, other analysis is done to "credential" users, Schaefer said, to ensure that the registered users don't work for rivals looking to badmouth competitors. The company also factors in the review scores left for vendors, how long users took to enter their ratings, and how often those users rate vendors.

Schaefer declined to be more specific about the credentialing process so users wouldn't find a way around it. "We have a whole set of mechanisms" to ensure who users, he said. "There are two approaches to validating the ratings. You look at the [person doing the rating] and their credentials and you look at patterns of the ratings themselves, then you can identify suspicious ratings."

As a result, "one bad rating is a grain of sand on a beach," Schaefer said. "We want to make sure that a guy can't put in 20 bad ratings for the same company" and unfairly throw off the ratings.

The algorithms are adjusted as needed to keep the methodology accurate, he said. The system also relies on people to look at the collected ratings data, so it can be manually validated and screened for problems. "In addition to the automated mechanisms, you have to have eyeballs watching this and continuously fine-tuning it."

Continue Reading