Knowing when a full software evaluation project is worthwhile

With more than half of all IT projects still failing, this article by Sharon Florentine  suggests the odds are not in your favor when making a major software purchase. Software evaluation and selection projects are an enormous amount of work for any organization because there are so many factors to consider. It can take months from the start of the project to making the selection and purchasing the software. Is it worth doing all this work up front, either internally or by hiring outside consultants?

Key to answering this question is determining the value the new software will provide to the organization. While you can undertake a formal ROI analysis, a simple way to estimate this value is to look at the total cost of the proposed software over the period of interest. For enterprise software, this is usually 5 to 10 years, depending on the type of software being considered. 

When it comes to estimating cloud versus off-the-shelf software costs, the consensus is that cloud is cheaper up front but more expensive in the long run, with a breakeven point of about 5 to 7 years.

Whatever the size of the project, it is appropriate to start with simple calculations. The biggest danger with these estimates is omitting hidden costs. For an idea of those hidden costs see a previous article which examined in detail the total cost of ownership for enterprise software.

Estimating value example

Since the software has not yet been selected, estimates must be approximate. This example uses round numbers that can be easily scaled up or down.

If the selected product will be in the data center: 

• Assume an initial software purchase of $1 million
• Implementation would be around 100 percent to 300 percent of the initial cost. Assume the middle of the range, namely $1.5 million
• 22 percent annual maintenance is typical, which is almost $1 million over five years

If the selected product will be in the cloud:

• Annual subscription fees of $400K, or $2 million over five years
• For the sake of this example, assume a similar implementation fee to the above, namely $1.5 million

Assume both would have $500K hidden costs over five years (see previous article for examples of these costs). In both cases, the total cost of this software is about $4 million over five years.

It stands to reason that if $4 million is being spent on software, that investment should return at least $4 million in value to the business. If it returns any less than this, you may as well have put the money in the bank and avoided the effort and risk.  However, since this is a major software purchase, assume a return in business value of at least 2x is desired, i.e. a total of about $8 million over five years. (2x over five years corresponds to 15 percent compounded annually). This value would come from cost savings and the ability to do new things that could not be done before.

Every major software purchase has risk associated with it. Some projects are complete failures where the total amount is written off. Others end up as partial failures, where the software isn’t bad enough to discard, but doesn’t return anything like the value used to justify the purchase. There are successes, but these are less common than they should be. Reducing this to one number, you could say that the probability of success associated with the typical major software purchase is 80 percent. Factoring this risk into the business value returned by the software means you should be looking to generate a risk-adjusted value of $10 million (i.e. $8 million / 80 percent)

When is it worth doing a software evaluation project?

The key to a quality job when painting a building is adequate preparation. Likewise, the key to a successful software purchase is an adequate evaluation & selection. Having a few IT people work on the project part-time in addition to their regular duties almost guarantees a failure, as does hiring consultants on the cheap. Perhaps this is one of the reasons why so many major software purchases do not deliver the value that was promised when the project was initially justified.

The value of a comprehensive software evaluation & selection project comes from thoroughly understanding your requirements, selecting the best-fit software for those requirements, an adequate implementation, and then from using the software in production.

Using the numbers above we can conclude that for every $1 million spent on purchasing software or $400K / year spent on cloud subscriptions, you should be looking for the software to return $10 million of value to the business. If the return is substantially less than this, then you will have a partial or outright software failure on your hands.

The question then becomes this: Given the planned return on your software investment, what resources and funds will you commit to the evaluation project? Only you can answer that question, but the wrong answer can mean a lot of pain for your organization.