Getting your environment right

When it comes to software development, we as executives and managers often focus on delivering value in the form of new features that address our customers’ needs.  Additionally, most executives focus on non-functional requirements and even keeping code “healthy” by constantly addressing technical debt.

However, how much focus do you spend on perfecting your development, QA and staging environments?  Many leaders will focus on building out the environment “on demand.” That is a prudent approach, but the truth is often that without a solid environment strategy for new development, you will severely cripple your software delivery.

We have all heard about leveraging DevOps for our current production systems, but what about new systems? What about addressing full “lift and shift” rebuilds of existing systems? Having your environments set quickly and correctly for new initiatives are critical to delivery as well.

Challenges and improvements

Now consider the level of potential improvement. Here are several measurements cited by HP in Jez Humble’s blog on continuous delivery:

• Overall development costs reduced by ~40%
• Programs under development increased by ~140%
• Development costs per program reduced by 78%
• Resources driving innovation increased by 5x

Now the context around HP was that reaching this point took years to complete, but the scale of effort for rebuilding and consolidating dozens of HP printer operating systems into one is significant. The point here is that the time your teams spend on building, deploying, testing and releasing has a tremendous impact on overall velocity of delivering quality software.

Well, then you may say, “Hey, we aren’t HP and our software is not even in production!” This point still holds true for new software development. For instance, I was working with one major financial institution where they had major deployment problems for a new software product in their Integration environment. It would take the team members days to complete a full build that would crash multiple times during the build process and team members had to manually restart from where it left off, frequently crashing again at the same point. This continued for over six months until one infrastructure expert discovered a single environment variable was incorrectly set and then the builds went smoothly without crashing.

The teams lost hundreds of precious hours nursing the build process when it took 10 minutes for the right person to fix the problem. Not a very cost-effective approach.

Consider this: Per the 2017 State of DevOps report, on average, continuous delivery increased time spent on new work by 44%. Is that not a worthy investment?

A path forward

The goal as Jez Humble stated at the Agile 2017 conference should be to “make deployments boring.” 

Here is the key point of this blog: As leaders, your goal should be constantly keeping your environments ahead of your teams, just like keeping your requirements backlog with any UX designs ahead of your teams. Otherwise, delivery velocity will suffer. In general, the loss of team productivity outweighs the effort for setting up a full deployment pipeline.

Here are examples of improvements you can quickly make to improve your road to continuous delivery:

• Configure the system to email all team members with the code breaks, and all building is halted until corrected.
• Enable the team to complete its work without needing communication and coordination with people outside the team.
• Have fewer than three active branches as a requirement.

The idea is to “make the right things the easy things.” The processes and automated systems must support behaviors to increase focus on delivering quality software (such as delivering in small batches by checking in code frequently), and not restrict teams into struggling with the environment.

These DevOps concepts have gained tremendous adoption. Companies like Spotify, Netflix, Amazon and eBay must have safe, automated build processes to ensure uninterrupted uptimes, since even a minute of downtime during a deployment may cost millions in revenue and more in a negative reputation.

So how do you know the quality of your environment? The killer metric here is to ask your teams how much time they spend checking in code, making deployments, building releases and testing their code. Start with understanding where your teams’ time is going, and then address those pain points through a culture of automation and environment improvements.

Conclusion

• Yes, you may have to spend extra time (expect a lot) on improving your software environments.
• Yes, if you do not have separate development, test, QA, UAT and production environments, you may have to build them out.
• Yes, you have to establish an automated testing framework for faster feedback and higher quality.
• Yes, you may have to hire or train your staff to understand how to make those environments “boring.”

Do you want a boring release? Sounds like heaven to me.