How much freedom should be given to microservice teams?

Pre-microservices: complete centralization

Until recently, IT was inherently a centralized function within an enterprise. Taking an application to production was an enormously expensive and lengthy process, requiring support from dozens of teams, millions in capital expenditures, and many months or even years of time. This led to CIOs standardizing on specific software and hardware products, multi-year vendor contracts, extended negotiations with vendors, centralized architecture review boards, and the involvement of procurement. Until recently this overhead was necessary because cloud didn’t exist. If you wanted an environment, you had to build it and run it yourself.

The cost of all this overhead is speed. Many large-scale commerce platforms are released to production once a month or even once a quarter. If it takes four tries to get a feature right and you’re releasing once a month, it takes four months to start seeing any revenue from that new feature. Amazon releases 50 million times a year to production, often iterating on a feature multiple times within the same day in production until it’s perfect. Retailers and brands are facing an existential threat from Silicon Valley-based tech companies. In just the past decade, tech companies have grown into the five most valuable in the world. The strength of these new companies is using technology to rapidly innovate. Those companies don’t have the same problems that most enterprises are facing. Small teams can deploy a new concept to production in just a few weeks, for a small fraction of what it would cost a traditional enterprise.

With public cloud now fairly mature, any individual or enterprise has access to hundreds of services that were previously the prevue of large tech companies. Rather than having a big team of Cisco network admins, for example, any developer can now provision an entire network with a single command line. No hardware or configuration required!

Microservices: decentralization

Microservices are now the default for large-scale commerce implementations. Amazon long-ago pioneered microservices and Jet, Walmart, and a who’s who of large retailers around the world have followed suit. Now even medium-sized brands and retailers are starting out with microservices for their new commerce platforms.

With cloud, small vertical cross-functional teams now have the resources they need to build, deploy and manage their own microservice applications. Rather than having to ticket the database team to provision a database, for example, anyone on the microservice team can just consume database-as-a-service from the cloud vendor of choice in a matter of seconds and pay a few cents an hour.

Over time, you’ll build a library of a few hundred of these microservices, each fronted with an API. You want to get to the point where you have dozens if not hundreds of these small teams each iterating on their own microservices. These microservices serve as the sole system of record across the entire enterprise, with many different front-ends and other systems consuming data from these microservices.

Microservices work because they decentralize development, pushing ownership and responsibility down to each team. Teams are owners of a small product (inventory, pricing, promotions, etc.) rather than members of a large horizontal team working on a specific technology layer (infrastructure, network, back-end development, storage, etc.). It’s an entirely different mindset. As owners, teams have far more freedom to select the supporting software and infrastructure required by their application, their application’s architecture, their team’s development methodology, and so on. What matters is that the API each team exposes is solid and that the application behind the API stays up and performs well. The “how” of development is largely up to each team.

How much freedom is too much?

The freedom given to microservice teams is a significant departure from how enterprises have historically managed IT, but it’s ultimately much better. Rather than dictating every detail of how applications should be built, deployed and managed, and hoping for the best, you’re holding individual teams accountable for the decisions they make. Accountability can be measured by the availability of the API, the performance of the API, how frequent the application is being deployed, and how much the team spends on cloud services. This quantitative data can be used to evaluate each team, especially if you publish each team’s numbers internally so that they can be evaluated in relation to their peer teams.

The microservice purists would say that each team should have 100% freedom to do whatever they want so long as the API that’s exposed to the world is working. A team could pick version .01 of a cool new JavaScript framework they just read about on Hacker News, but they’re also responsible for fixing problems at 2 AM on a Sunday when something doesn’t work. Freedom comes with responsibility and accountability.

While giving each team complete freedom and holding them accountable sounds great, enterprises have legitimate interests in exerting some control, including:

• Legal: License types, patent infringement, data privacy, etc.
• Financial: Raw cost of 3^rd party software and cloud services, budgeting, etc.
• Support: Support availability from 3^rd party software and cloud services
• Security: Compliance with regulations like GDPR and PCI, mandatory patching, etc.
• Staffing: Portability of people across different teams, ability to recruit for specific skills

An individual within a microservice team, or a team collectively may not care about these interests, but an enterprise certainly does. If someone mistakenly uses a GPL license on a commercial product, violates GDPR or unknowingly infringes on someone else’s intellectual property, the enterprise may be exposed to enormous financial and other penalties. Executives can end up in prison for the mistakes of their subordinates. An employee isn’t going to have that broader awareness or context and probably won’t feel any direct pain from their mistake. Freedom has its limits.

Each team should always have the complete freedom to architect and implement their own microservice as they see fit. If the implementation isn’t very good, you’ll quickly see poor availability and performance of the API they expose. If a team does an exceptionally poor job, the microservice should be small enough that it could be rewritten over the course of a few weeks with a new team. If you dictate every architecture and implementation detail from the top down, you’ll end up with teams that lack ownership. Empowered members of a team that own a product care, have responsibility and can be held accountable. People working on just another project for a few months do not.

For programming language, cloud vendor, datastore, DevOps tooling and so on, you’ll want to offer each team a menu of options. You’ll want to give them the choice of Java, JavaScript or .NET as programming languages, for example. Maybe you’ll want to offer Amazon Web Services and Google Cloud as cloud vendors. Maybe you’ll want to offer Google Cloud Datastore, PostgreSQL on AWS and MongoDB Atlas as datastore options. Allow each team to pick the software and services that work best for their application. You can then establish enterprise-wide contracts with these vendors so that your legal, financial and support needs are met.

It’s tempting but try not to force teams to share infrastructure or software because it introduces dependencies between teams. Yes, all of your microservices could share one enormous instance of PostgreSQL on AWS, but these services are all so inexpensive now it just doesn’t make sense anymore. You can run a fully featured clustered instance of PostgreSQL on AWS for $0.14/hour. When this supporting technology was millions of dollars, it made sense for many applications to share underlying software and infrastructure. Sharing resources across teams requires coordination which by itself is expensive. More important than cost, sharing slows down development velocity which is a much higher cost.

While sharing is often bad, in some cases it is required. You’ll want to have one messaging system, one source control system, and one monitoring system for example. Imagine if every team had their own messaging system and you had to forward and translate messages between systems all the time. You’ll also want to standardize on how you define your APIs. Do you use REST or some binary-level protocol like gRPC? Do you use JSON or XML in the case of REST? Do you use RAML or Swagger? Do you offer GraphQL? The benefits from standardizing on a particular approach for all of the “outer architecture” (the glue that binds together microservices) far outweighs the small amount of freedom given up by each team.

It’s also important to segment the software you’re building and apply different levels of control and freedom to each team. A team that’s upgrading a payroll system from the 1990’s should have very little freedom. That type of legacy back office software doesn’t change much, can’t go down, and doesn’t offer the ability to make your company any money. A team building a new Machine Learning microservice to enhance product recommendations should have near complete freedom. As a microservices goes from innovative experiment to something that’s used in production by real people and starts to make money, freedom should be slowly restricted. Context matters.

Final thoughts

For enterprises used to centralizing IT, adapting to a more microservice-based approach where teams have freedom can be a huge cultural shift. Remember that freedom comes with responsibility. Teams are freer to make their own decisions but they’re responsible for and are measured on the availability of the API, the performance of the API, how frequent the application is being deployed, and how much the team spends on cloud services. Teams ultimately have to deliver and it’s now easier than ever to objectively quantify a team’s performance. Stepping back even farther, the whole reason for implementing microservices is to allow more features to be released to production more quickly and to do that, you’ll need to strike the right balance between freedom and control.