How the Software Selection Maturity Scale can improve enterprise software selection
Ever wondered why selecting enterprise software is so difficult? Use the Software Selection Maturity Scale to evaluate the state of your organization’s selection process, and as a roadmap for improvements.
Buying business-critical enterprise software entails significant risks for an organization. The goal of an evaluation and selection project is to purchase software that maximizes the ROI and minimizes implementation risks, but the sheer number of requirements make this difficult to achieve. For example, a comprehensive ERP selection for a larger company can approach 10,000 requirements. Also, there is the risk described by the Dunning-Kruger effect where IT professionals overestimate their software selection abilities and underestimate the project effort.
Software selection process improvement
This article focuses on a very critical part of software acquisitions, namely the evaluation and selection process, and examines how to improve that process. You could think of it as business process improvement focused on software selection and optimization. Although information from evaluations is passed over to implementation teams, this article does not cover the actual implementation of software.
Based on CMMI publications we have developed the Software Selection Maturity Scale (SSMS) to measure the maturity of the software selection process. While CMMI for Acquisition covers processes for acquiring products and services, SSMS covers just enterprise software evaluation and selection. Note that there is not a one-to-one correspondence between CMMI and SSMS.
SSMS has six levels, numbered 0 to 5 as shown in the diagram below. As you move up the list, processes are more mature and purchasing risks reduced. Each level rests on the one below.
Level 0 – None
There is no process in place for evaluating and selecting software. Selection is based solely on the advice of others, e.g. referrals from colleagues, product reviews or sales people. Software may be chosen for unethical reasons like favors or bribes. If there are demos, the most persuasive pitch wins.
There is a major risk of the software failing to adequately meet organizational needs.
Little leverage over vendors who over promise and under deliver.
Implementation schedules slip, and budgets are exceeded.
Level 1 – Initial
Evaluating and selecting software is ad hoc and chaotic, with success depending on individual efforts. Even if a software purchase is successful, there is no ability to repeat that success with the next project.
Minimal project management. IT usually manages the selection project with little planning for needed resources.
Software is informally evaluated against these requirements. If used, scoring is rudimentary.
Software selection is subjective, and may be chosen by a committee. May also be selected based on previous experience, e.g. “I know what I am doing” (see Dunning-Kruger effect), but with little attempt to examine organizational differences.
Unrepeatable and subjective decision-making.
Implementation schedules slip and budgets are exceeded. Little leverage with vendors who over promise and under deliver.
Significant risk of software not adequately meeting organizational needs.
Level 2 – Basic
A basic software evaluation process is established and followed. Organizations can achieve partial success but usually fail to meet the ROI used to justify the project.
Informal ROI estimate. Some project management. Limited IT resources with no specific software selection training.
Non-functional requirements like support, licensing, etc. tend to be examined only superficially, leading to risks of problems later.
Somewhat subjective software selection process that is not readily repeatable.
Moderate risk of software failing to meet organizational needs adequately.
Implementation schedules often slip, and budgets exceeded.
Level 3 – Defined
A software evaluation and selection process is established, followed and managed.
Formal ROI estimate before a project starts. Reasonable project management in place. Adequate resources are available that may include external consultants. Resources are trained on the principles of evaluating software. Some resources are familiar with the type of software being evaluated.
Software acquisition risks are managed and minimized. The purchase contract is optimized to share risk with the vendor. Testing verifies the software’s business performance. Post purchase process improvement is undertaken.
Adequate project management. Post implementation analysis is actively used to improve future software acquisition projects.
Adequate product research and evaluation of potential products. Vendor claims audited for accuracy. Objective and repeatable software selection process.
Purchase contracts are written to fairly share project risk with the vendor. The purchaser pays more in exchange for reduced risk of project failure. Contracts are written based on outcomes (as opposed to meeting milestones).
Information from the winning product is used to estimate and plan the implementation project. No “new” requirements appear during implementation, which is on budget and on time.
Implementation verified against requirements with formal user acceptance tests. Purchase contract includes terms that require the software and / or implementation vendor to achieve a particular score on those tests. Failure to meet scores triggers penalties.
Post implementation analysis identifies requirement and acceptance test gaps. These are added to the requirements libraries for future software selections. Requirements library is actively managed for quality.
Use the SSMS to measure the maturity of your software selection process. Note that different projects can operate at different places on the scale. For example, an organization operating at Level 3 may hire a new VP who buys a particular software product that he likes, without considering alternatives. That project will have performed on a Level 0.
Use the SSMS as a road map for software selection process improvement. As an organization moves up the scale, the ROI achieved gets closer to the ROI used to justify the software project. Also, the implementation stays within budget and is completed on time.
Ultimately, a mature software selection process helps organizations meet or exceed the ROI that was used to justify the software purchase in the first place.
Acknowledgments & references
This article builds on the CMMI concepts, especially CMMI for Acquisition
Chris Doig graduated from the University of Cape Town, South Africa with a bachelor of electrical engineering degree. While at university, he founded Cirrus Technology to supply information technology products to the corporate market. The focus at Cirrus was helping companies buy the best IT products for their particular needs. Cirrus also developed custom software for the South African 7-Eleven franchise holder and other corporate clients.
In the 1990s, Chris immigrated to the United States and worked at several companies in technical and IT management roles: Seagate, Biogen, Netflix, Boeing, Bechtel SAIC, Discovery Communications and several startups. At all of these companies he repeatedly saw software being purchased with an immature selection process. Invariably this software would take longer to implement than planned and cost more than budgeted. To make matters worse, the software seldom met expectations.
Having struggled with software selection himself, Chris founded Wayferry, a consulting company that helps organizations acquire enterprise software. He is also the author of Rethinking Enterprise Software Selection: Stop buying square pegs for round holes. While ERP projects account for much of Wayferry's work, other types of enterprise software acquisitions include CRM, HRIS, help desk, call center software, clinical trials management systems and so on. For Chris, the ultimate satisfaction is when clients report meeting or even exceeding expectations with their new software.
The opinions expressed in this blog are those of Chris Doig and do not necessarily represent those of IDG Communications Inc. or its parent, subsidiary or affiliated companies.