When the favored enterprise software receives good reviews it is tempting to skip the evaluation project, but doing so can set you up for failure. Here's a look at why you can't rely on reviews when selecting enterprise software. Reviews are often used when selecting enterprise software, but not many people consider their limitations. Let’s start by looking at two definitions: Dictionary.com defines a review (noun) as: A general survey of something, especially in words; a report or account of something A critical article or report, as in a periodical, on a book, play, recital, or the like; critique; evaluation An evaluation (noun) is defined as: An act or instance of evaluating or appraising Notice that the definition of a “review” includes the word “evaluation.” So what is the difference between a review and an evaluation, and why does it matter when selecting software? Reviews In the context of buying enterprise software, a review is a critical examination done of one or more software products written to help organizations make a purchasing decision. Reviews are sometimes commissioned by software vendors, and those are usually biased in favor of that vendor. They are also written by analyst companies like Gartner, Forrester, IDC, etc. and sold. Occasionally they are written by knowledgeable users of a product who want to share their experience for the benefit of the community. Reviews may include comparisons with other products, benchmarks, the strong and weak points of each product and often have graphical charts like Gartner’s famous Magic Quadrant. The most important thing about reviews is to realize they must be written for the “average” users of the products being reviewed. While they can be written for a subset audience, they are never written for just one organization. One particularly useful feature of multiproduct reviews is that they can alert you to products you didn’t know you should be considering. A technique of finding relevant reviews on the Web is to search for the names of two or three software products in one search phrase. Reviews that include these products appear near the top of the search results, and these reviews sometimes include the names of products you don’t know about. When searching, remember to filter results by date; for example, a six-year-old review is much less useful than one that is six months old. Evaluations Like a review, an evaluation helps make a purchase decision. The primary purpose of an evaluation is to measure how well various potential software products fit the organization’s particular needs, where those needs are expressed as requirements. For any given type of software, e.g. ERP, CRM, etc. we discovered that all organizations have essentially the same requirements. What makes each organization unique is how important those individual requirements are to them. For example, a financial institution will typically have much stricter security requirements that a retail organization. With an evaluation, requirements may differ significantly from that of the average user of the software, which is why an evaluation is specific to an organization. Also, note that evaluations are far more thorough and detailed than reviews. Conclusion The primary difference between reviews and evaluations is that reviews are aimed outwards at multiple people. Software reviews are a very important source of information, especially in the early stages of a project when information is being gathered. However, because reviews are aimed at the average purchaser of the software, they are no more than a rough guide at best. On the other hand, evaluations are aimed inwards, usually at a single purchase decision. Good evaluations are a detailed analysis that measures how well the evaluated software products meet an organization’s particular requirements. Use reviews in the early stages of gathering information, but don’t make the mistake of thinking that because a product is recommended by one or more reviews you can’t go wrong. The TCO of enterprise software purchases often runs into tens of millions of dollars. Maximizing the return on that investment demands a thorough software evaluation. A bonus is that a thorough evaluation can also reduce software implementation risks. This article is an updated version of an article that originally appeared on the Wayferry blog. Related content opinion How IT can both deliver business value and 'keep the lights on' IT teams spend too much time on u201cdaily churnu201d rather than delivering real value to the organization. Read how Mike Guggemos, CIO at Insight Enterprises tackled the problem. By Chris Doig Feb 16, 2018 5 mins CIO IT Leadership opinion 15 places to use requirements when selecting enterprise software Not understanding how important requirements are and where they are used is the root cause of most problems with implementing software. By Chris Doig Jan 29, 2018 9 mins Enterprise Applications opinion The backward way of gathering enterprise software requirements Organizations ask users for their requirements, only to find that when enterprise software goes live, it doesnu2019t meet user expectations. It turns out that we have been doing this backward for years. By Chris Doig Oct 05, 2017 5 mins IT Governance Frameworks SaaS IT Governance opinion The hidden costs of poor software purchasing exposed! Many companies think they know how to purchase software when in reality they have no idea of how little they know about the process! This article looks at the three places where money is squandered. By Chris Doig Jul 14, 2017 4 mins IT Governance IT Strategy SaaS Podcasts Videos Resources Events SUBSCRIBE TO OUR NEWSLETTER From our editors straight to your inbox Get started by entering your email address below. Please enter a valid email address Subscribe