Story point normalization — what’s the point?

When it comes to agile project delivery, progress is normally measured in the form of story points, or hours burned for a given sprint. From the business owners down, leadership is always giving the mantra, “When will it be done?” and “Where are we now?” Typically, leadership wants that consistency and confidence in knowing what to expect and when in order to build trust in delivery.

Notice that consistency is the key word here. However, the reality is that balancing the workload in a consistent manner in a single team, let alone across multiple teams, is challenging. When using story points, since they are relative, two different same-sized teams could be producing the same amount of total work, but team A could show 30 story points a sprint and team B 150 story points a sprint. This naturally drives up leadership frustration and decreases trust!

Now that doesn’t necessarily mean team B is delivering five times more results or working five times as hard. Since story points are relatively sized, it more likely means that team B’s baseline is five times bigger than team A’s. However, there is a core challenge with having that level of variance between team performance when it comes to working across multiple teams in a scaled delivery model or having a standard delivery practice across multiple projects for budgeting and shared delivery purposes.

One solution to this challenge is to normalize your delivery team pointing scale.  Details of this can be found at the Scaled Agile Framework site under the subheading “Normalizing Story Point Estimation.”

But the normalization process doesn’t stop at the setup. There are further actions needed to ensure and keep story points normalized so they don’t go too far into the “deep end” with variability.

First, look at comparing baselines. Traditionally, teams will take the smallest story and baseline that.  Here are a few more suggestions on ensuring a consistent baseline:

1. Don’t baseline just the smallest story. Instead follow Mike Cohn’s suggestion of baselining two stories, called “triangulation.” My suggestion is to do three stories. That way there is more consistency in the baseline

2. Make sure you have a person from an external team cross-check the baseline for agreement. If that person sees a large delta in the baseline, then that should be adjusted to the “normalized baseline” immediately

 The next part is to manage your delivery closely for any variances and work on eliminating them. Here are some example cases:

1. Velocity inflation: The team slowly “shifts” its baseline up so that over time a “2” story is now a “3” story and so on. This inflation happens when a team’s success is measured purely by velocity. This should also be periodically cross-checked by external team members and business value should be focused more than story points, since velocity inflation provides only the appearance of more value but is a “false positive.”

2. Poor requirements: Consider the backlog the “jet fuel” for flying to any product destination. If the fuel isn’t refined enough, the jet will sputter, slow down and possibly crash. To ensure the highest quality fuel, set standards for story quality when entering a sprint and review during sprint planning. This doesn’t need to go overboard, but “just enough” to get the team to commit and do the work without further delay or creating the wrong result.

3. Not enough quality: The whole agile delivery process is like a machine. If one part fails, the whole machine will fail. If velocity begins to slow over time, look at the amount of QA resources and measure the bug counts and severities. Make sure the code quality is high by employing proper programming quality practices. Many sources exist, including the books Pragmatic Programmer by Andy Hunt and Dave Thomas, Clean Code by Robert “Uncle Bob” Martin and Extreme Programming Explained by Kent Beck.

There are many points of failure and few points of success when it comes to normalizing your story pointing for delivery. In some mature organizations, estimation isn’t even practiced anymore, since interpreted as a productivity loss and work is measured strictly by the value of the software produced.  However, this requires a high degree of consistency, experience, confidence and trust to reach. For most organizations and consulting companies, delivering with a high level of consistency is a holy grail.