New software development success metrics

Measuring success of software products has always been a difficult task. But, at least in a waterfall delivery model, there were certain metrics that were fairly easy to track and report. Forrester referred to these as the “iron triangle” of cost, scope and deadline.

As project delivery models evolve to be more iterative and agile, these success metrics lose their relevance. This is particularly true for the most significant and strategic efforts that involve customer-facing, enterprise-evolving or product development needs.

Innovation requires IT to write requirements not in stone, but in sand. Teams must be ready to adapt and respond to changing end user demands. As scopes change, so do deadlines, as enterprises emphasize minimal viable product and delivering value with exceptional speed. This variability means costs can be estimated but not fixed.

In this new environment speed, or velocity, becomes a predominant measure of success. Within that framework, here are three metrics that you should ask your development partner for to gauge project and team success.

Time to stable velocity

Many products my company engages with are behind schedule or in need of drastic improvements fast. Clients do not have months to spend waiting for development teams to get up to speed with project goals or technical stacks.

So, quickly figuring out what a team needs to do and deliver is critical to both immediate and long-term project success. An average team, one that is newly formed or contains one all-star developer and the rest are new hires, may be able to reach stable velocity in five or six sprints.

However, a team that is cohesive, has already worked together on past products and matches the culture of a client can reach this level in half that time, or two to three sprints. This difference means that the business will know sooner what will be delivered in each sprint. It lowers costs and gives greater confidence as to when important product features will be delivered.

Time to peak velocity and steady-state

The idea of predictability for product delivery date carries through from ramp up time to ensuring consistent velocity through the life of the product.

Even though I would venture to say a majority of waterfall projects do not meet their delivery date, the concern with agile and innovation work is that, “You cannot tell me when I will have this thing?”

By measuring the consistency of sprints once a stable velocity is reached, you can report with greater confidence on delivery dates given the current scope. As the scope changes, teams can adapt to the new requirements each sprint and produce a minimal viable product sooner.

If delivery is changing dramatically between sprints, you need to figure out why. Was it due to new requirements added to the project? Did a team member leave and her throughput was what was keeping the project on track? Or, was there a communication breakdown that resulted in developers stopping work because they did not receive proper guidance or a timely response?

By asking for velocity rates from your development partner, and by tracking those in relative terms from sprint to sprint, you can track not only the delivery of your project, but also ensure they are providing you with a valuable and trustworthy team.

Velocity by person

The idea of measuring individual performance on a development team is, to say the least, controversial. Agile encourages the team and suppresses individuality.

But, I have seen time and again where other organizations will put one strong person in front of a weak team. This results in dramatically different contributions across the team, raising the level of risk that the project will fail.

If the one all-star takes time off or leaves the team, the project risks falling behind schedule, or worse. Conversely, a team of consistently strong engineers can be more than the sum of its parts. This benefit cannot be achieved when there is a large disparity of talent across the team.

The metric to watch closely is the standard deviation of velocity per person across the team. The smaller the deviation is, the more consistent the team. You can compare this ratio across teams and sprints to see if or when it changes, and correct course if needed.

These three metrics will help you better measure the success of development efforts. Just do not be afraid to ask for them. A development partner should be able to provide them to show the value of their work. If not, it is a red flag that what they promised might not be what they can deliver.