The Cloud Was Made for Automation

This post was originally published on LinkedIn by Omri Gazitt, Chief Product Officer at Puppet. It is re-published here with his permission.

 Without automation, moving to the cloud sacrifices its most important advantage: agility

The move to the cloud represents the biggest sea change for IT in a generation. In the 2000s, virtualization allowed IT organizations to consolidate many workloads on the same physical hardware, thereby increasing utilization and efficiency. It also provided a critical stepping stone. Building on virtualization, cloud platforms provide the ability to run these virtualized workloads on rented infrastructure managed by the cloud provider, freeing IT from having to maintain physical data centers and the infrastructure that runs in them. Avoiding what Werner Vogels coined “undifferentiated heavy lifting” frees both capital resources and human resources to focus on delivering business value.

Automation unlocks agility

As attractive as it is, increased efficiency is not the primary benefit of cloud platforms. The real game-changing aspect of these platforms is that they offer uniform API-driven provisioning of compute, storage, and networking resources, which makes them inherently automatable. This in turn means that IT organizations can now completely automate workflows that were once done manually, vastly increasing their agility as they strive to keep up with the ever-increasing demands of the business.

This combination of agility and efficiency offers a compelling alternative to on-premises data centers, and we are now seeing cloud platforms eclipse traditional data centers as the home for both new and existing workloads alike. And the most important ingredient for unlocking agility in the cloud is automation.

Adopting the cloud vs. becoming cloud-native

Moving to the cloud is not the same as moving to a cloud-native architecture. Thanks to the last decade’s virtualization wave, the standard “unit of compute” for most IT organizations is a virtual machine. But whether a “machine” is physical or virtual, it looks like a “node” to the systems management software that IT organizations employ to administer it. We call this a node-centric architecture.

By contrast, the most distinguishing characteristic of a cloud-native architecture is foregoing the notion of managing individual nodes in favor of scheduling workloads on a distributed system that appears as “one big computer.” These workloads are typically architected as microservices that make very few assumptions about the local resources available to them, or on their proximity to other microservices; and they are almost always packaged as docker containers to increase their portability. The most popular scheduling and orchestration system for these containerized workloads is Kubernetes, which completely abstracts away the concept of a node, and defines the container pod as the “unit of compute.”

The move to the cloud can therefore be thought of as two related but independent steps. First, moving the virtualized nodes that comprise these workloads from on-premises data centers to virtual machines running on cloud platforms, retaining their node-centric architecture. Second, transforming the workloads from monolithic applications running in virtual machines to microservices running in containers, achieving a cloud-native architecture.

Deciding between node-centric or cloud-native

Moving a workload to the cloud typically involves effort, and the decision of whether to move it (and how) is ultimately about return on investment. Many workloads simply don’t meet this test — the effort to move them exceeds the operational savings. But for the ones that do, a key decision is whether they are worth rewriting as cloud-native microservices, or whether it is best to “lift” the workload as a virtual machine and “shift” it to the cloud provider, retaining their node-centric architecture.

There isn’t a right or wrong answer to this question. Just like the decision to move a workload to the cloud, the decision to re-architect it depends entirely on the potential ROI. An application that needs to evolve and grow quickly based on burgeoning usage and scale will often benefit from (and perhaps even require) this re-architecture. An application that grows more slowly in these aspects may be fine the way it is.

The benefits of automating cloud workloads

There is one common attribute for workloads that do meet the ROI test for moving to the cloud, whether they are to remain a collection of nodes, or get transformed into microservices and packaged/deployed as containers. These workloads are always worth automating. Automation allows an organization to consistently and effortlessly create multiple identical environments for the workload (e.g. test, staging, production), making it far safer to test changes before they are rolled into production.

In addition, automation unlocks the ability to easily move the workload to different availability zones or regions. Using platform-neutral automation tools extends this portability benefit across cloud providers, or even back to API-driven hyperconverged infrastructure running on-premises, providing maximum flexibility in the choice of deployment environment. The benefits of reproducibility and portability are critical in reducing the manual, error-prone, time-consuming, expensive work involved in standing up a workload. Portability also mitigates the threat of vendor lock-in, thereby significantly increasing the agility of the organization.

Last but not least, automation allows the organization to “bake in” its standards around security, policy enforcement, compliance, and governance into every workload, thereby increasing the consistency and safety of each of the workloads it runs in the cloud.

Automating node-centric vs. cloud-native workloads

The most important enabler of automation in cloud platforms is the APIs they expose for provisioning and managing any kind of resource they support. For node-centric workloads, automation entails orchestrating the workflow for creating and tearing down the infrastructure resources required to support them, including virtual machines, storage volumes, networks, security groups, firewall rules, and so on. In addition, automating the configuration of resources inside the VMs (packages, services, users, groups, files) is critical for being able to completely reproduce an application’s environment on demand, allowing the organization to capture the agility benefits of automation.

Automating cloud-native workloads involves a mental shift in the kinds of resources that are the building blocks of automation. Instead of VMs and security groups, the relevant resources are container images, pods, services, deployments, configmaps, and secrets. Automating the deployment of cloud-native workloads requires tools that support orchestrating these kinds of resources, but the benefits of this automation are just as important as for node-centric workloads. In fact, given that cloud-native architectures tend to involve a larger number of microservices that in turn have a greater number of interdependencies, automation becomes an even more critical tool for helping manage that inherent complexity.

Automating application delivery

Many IT workloads are custom applications that are built and delivered by the business. A foundational part of automating the delivery of these workloads involves orchestrating the virtual or cloud-native resources they run on, as we already discussed. But the other critical aspect of application delivery is automating the build-test-deploy workflow for the application. This is done via continuous integration/continuous delivery (CI/CD) tools.

Continuous integration (CI) involves the automated building and testing of every code commit. Continuous delivery (CD) automates the creation of deployable artifacts from commits that have passed the continuous integration cycle. And continuous deployment is the practice of automatically deploying these artifacts into a running environment.

Whether an application is node-centric or cloud-native determines what are the deployable artifacts and what infrastructure model to deploy them on. In either case, the practices of continuous integration, continuous delivery, and continuous deployment apply equally, and will make application delivery much safer and more predictable, thereby making it even easier for the organization to reap the agility benefits of the cloud.

Cloud automation enables next-gen DevOps practices

Collectively, automating both infrastructure provisioning as well as application delivery is a critical part of unlocking the agility benefits of the cloud. Those are also the ingredients of DevOps: By automating every part of the infrastructure and application-delivery lifecycle, an organization can increase the reliability, evolvability, safety, velocity, security, compliance, and governance of its application-delivery efforts. And the cloud makes this possible because every resource it provides is exposed over an API, and therefore can inherently be automated.

In summary, moving to the cloud is the perfect opportunity to adopt automation tools. Organizations that use this opportunity to automate both the infrastructure provisioning and application-delivery aspects of their business will reap the most rewards from their cloud transition.

For further reading, we recommend:

• Transitioning to a Hybrid Infrastructure: Key Considerations During Cloud Migration
• Cloud Bound: Preparing for Migration
• A No-Nonsense Guide to Deploying Containerized Applications
• Puppet Joins the Cloud-Native Computing Foundation