From Traditional to Agile Software Development – Changing Culture, Processes, and Architecture

Delivering working, tested, and deployable software on a continuous basis delights customers and gives organizations more visibility and adaptability throughout project lifecycles. But making this transition from traditional to agile development means adjustments for the entire organization – from management and collaboration to software development and testing.

As a recent product group leader within Hewlett Packard Enterprise (HPE)¹, I was part of a team that successfully worked through this transformation. To ensure overall success, my team focused on three key pillars: culture, processes, and architecture.

Promote a new culture

Because HPE is a company originally built on producing hardware, a waterfall mentality drove the culture. Manufacturing hardware, delivering that hardware to customers, deploying hardware, and following roadmaps driven by the evolution of chipsets conspired to drive a sequential set of interdependent steps. In contrast, today’s agile software development culture is a more organic, creative, and flexible process. As HPE began to promote a new culture within the company – one with more collaboration and agility – the team needed a completely new mindset.

There are several key imperatives in creating a new culture:

• Collaborate across all groups: A new mindset involves everyone – not just developers and IT operations, but C-level executives and even customer service. All levels across every discipline must be involved in order to break down silos and improve outcomes.
• Invert top-down command and control: Those closest to the technology must drive innovation. To meet customer needs more effectively, DevOps needs empowerment and influence to determine the direction software applications take.
• Fast fail (fail early, fail fast, fail often, fail better): Innovative companies focus on the concept of fail fast and fail often. As long as employees can learn from their failures, this strategy helps the company grow in an agile way.
• Shift left for a continuous culture of learning: Testing earlier in the lifecycle and finding problems results in saving time and money. Successful organizations need to apply this mindset to all areas, encouraging more collaboration to promote learning.

Embrace new processes

The traditional waterfall approach to building software means that development, test, and product management are typically separate teams. Product management generates requirements and passes them off to development, who produces software and throws it over the wall to test teams. The siloed teams embark on a cycle of iterations, each working separately, which results in big bang releases where the connection to the customer is several steps away from the creative process.

As a new collaborative mindset takes hold within an agile organization, these old approaches no longer work, which means new processes must be implemented. In an agile environment, all groups are integrated, which removes the thick walls separating the teams and provides better collaboration. Product management and full-stack developers constantly work together on one team, generating small increments of value and validating what they build with real data (i.e. A/B testing, canary deploys.)

Teams must implement enough process to develop a consistent rhythm, which takes the form of agile ceremonies (planning, retrospectives), agile work tracking (sprints, Kanban boards), and agile roles (scrum master, product owner). The idea is to have just enough process for teams to function but not so many that the process itself becomes the focus.

These new processes allow organizations to maintain an immediate feedback loop during development. Every team delivers on their own cadence, while promoting continuous integration, development, deployment, testing, and delivery.

Discover a new architecture

An organization transitioning from traditional to agile software development must also change the way it architects its products. Even with a new mindset and new processes, if an organization is building monoliths, they cannot be agile. As an esteemed colleague of mine once said, “Stack is a four letter word.”  This is because monolithic software is highly connected, complex, and difficult to decompose. Monoliths can’t easily be partitioned into smaller services because of the lack of durable APIs and explicit contracts between concerns. Because all of the software is intertwined, teams must coordinate carefully with other teams, making it difficult to deliver autonomously.

Conversely, in a modular, microservice architecture (MSA), developers define portions of the software and constantly work with product management, software testing, customers, and others to iterate small changes in parts of the software on a weekly or even daily basis. Developers quickly try out new features to see if they have merit. If a new feature looks promising, the team can canary deploy to a subset of customers and monitor analytics (i.e. clicks), thereby testing and receiving valuable feedback faster.

Results of developing with a new agile methodology

Several years ago, I initiated a team tasked to develop new multi-cloud management software that simplifies private and public cloud usage. The team designed an as-a-service offering to provide customers improved access to a pool of IT resources spanning public cloud services and their on-premises environments.

To develop a better quality product and deliver it faster, the HPE OneSphere team (named after the application) needed to leave the old culture, processes, and architecture behind. HPE OneSphere is a hybrid-cloud management platform that provides a unified experience across public clouds, on-premises private clouds, and software-defined infrastructure. Through its software-as-a-service (SaaS) portal, HPE OneSphere provides organizations access to a pool of IT resources that spans the public cloud services they subscribe to, as well as their on-premises environments. HPE OneSphere is designed for IT operations, developers, and business executives seeking to build clouds, deploy applications, and gain insights faster.

Embracing a new agile methodology, the team successfully released the project on an aggressive schedule with quality. But more importantly, the team established a culture of continuous innovation, continuous integration, continuous deploy, and continuous release. Transforming the team into a modern, agile-oriented organization allows us to react quickly to changes in the market, improve quality, continuously deliver value, and ultimately delight customers.

Three pillars of agile software development

Delivering working, tested, deployable software on an incremental basis is possible. But it involves more than just getting teams within development and operations to work together. Agile software development involves the entire organization and a commitment to change three important pillars:  culture, processes, and software architecture.

^{(1) At the time this article was written, Wayland Jeong was the Vice President and GM of Hybrid Cloud Business, Software Defined and Cloud Group at Hewlett Packard Enterprise.}