Getting Clueful: 7 Things CIOs Should Know About Agile Development

Agile methodologies for software projects can help organizations create better software faster. Yeah, yeah, you've heard that before. Here, experienced programmers explain the key ingredients to make those goals achievable.

Though it may surprise cynical IT managers to learn this, developers care passionately about the software they write.

Managers want top-notch, bug-free applications delivered on time, which meet or exceed user expectations—and the people who build that software have the very same intention.


ABCs of Agile

Book Chapter: What an Agile Process Looks Like

Agile Development Can Avert Disaster

Sign up for our Development and Architecture newsletter

How managers and developers achieve those goals, however, can be far different. The "old" school of software development followed a path in which software specifications were defined explicitly in the form of an inviolate requirements document. Then managers told the development staff to write that application...and come back when finished.

The result, too often, was a great answer to the wrong question: software that's late, that does interesting things but doesn't solve the user's need, and—well, you're probably too familiar with the results.

Over the last decade or two, software engineers have invested a huge amount of energy in finding better methodology for creating great software that users love. Agile development embraces several processes, such as test-driven development, and extreme programming (XP). There are quite a few more processes, each with its adherents and evangelists. But they all consider themselves agile.

They have quietly been taking over your development department, too. According to recent research from Evans Data (registration required), more than half of North American developers are using agile methods today. But IT managers may not be on board.

Agile has crossed the chasm, and "the majority of organizations are doing agile, and agile works better in practice," says Scott Ambler, author of several books (including Agile Database Techniques: Effective Strategies for the Agile Software Developer) and practice leader for agile development in IBM's Methods Group.

If you aren't a programmer, or if it's been so long since you coded that you get nostalgic for JCL, it behooves you to listen up. I asked online of more than 50 developers, "If you could get the boss to understand one thing, just one thing, related to agile development...what would it be? Why that?" Their summarized answers will help you prioritize the essentials to incorporate these development practices into your software projects.

Steven Smith, CIO of Lake Quincy Media and also a software developer, wraps up most developers' opinions succinctly: "I wish management understood that the quality value provided by agile technologies, and most especially from well-tested code, pays big dividends in terms of reduced QA expense and faster time to market for any significant-sized project. We have some projects in-house that were developed in an agile fashion, and others that were not—we dread touching the ones that were not."

1. Agile Creates Better Software

Agile methodologies aren't something that your development departments adopt because it's a cool way to write code. They see the major benefits in business terms, just as managers do.

In the experience of Kelly Anderson, C# developer and technical team lead at Sonic Innovations, the agile approach is less expensive. "If you count the cost of bugs that make it too far downstream in the process, there is simply no way to financially justify any approach except some kind of agile approach," he says. "Poor quality costs money. Lots of money. Agile leads to higher quality, enough higher that it is cheaper."

Plus, Ambler adds, it's easier to govern agile projects than it is to govern traditional projects.

"Agile development can reduce the cost of late changes, making it easier for IT organizations to respond as stakeholders' needs evolve," says Cem Kaner, author of Lessons Learned in Software Testing and Testing Computer Software.

Kaner, who is professor of software engineering at the Florida Institute of Technology, adds: "The most important thing the executive can do is keep asking the critical question, 'How does this practice (paired programming, test-first programming, whatever the group is proposing this week) make us more able to be more responsive later?'" If those implementing the process can't answer this question convincingly, Kaner says, it's a big red flag.

Agile's business value comes from organizations' ability to focus on which projects to do, what features to include in those projects, and which processes to use, according to Kent McDonald, business systems coach at consulting firm Knowledge Bridge Partners. "Value is provided by only doing the important things that directly contribute to the sustainable competitive advantage of the organization, and value is detracted when activities are performed that do not contribute to that sustainable competitive advantage," he says. "Organizations can adopt a focus on value without necessarily adopting agile development practices, but it is a cornerstone of those approaches."

"Enterprise agility means that the enterprise can respond to business needs faster—and that it can maintain this," adds Alan Shalloway, CEO of Net Objectives. "It means that product managers select functionality that will make the most sense to the business and deliver in stages so resources can be reallocated as necessary to provide the greatest value."

The business benefit is a key message for CIOs and IT managers. But it's not the issue about which developers feel most passionate.

2. The Agile Mind-Set Is More than Processes

Why is the agile approach such a big deal to developers? They see it as a bigger change than how software is developed.

Agile is a way of working and of thinking about work, says Mike Sutton, who runs agile consultancy Wizewerx. "It challenges entrenched thinking. It's painful because it forces organizations and individuals to confront waste, inefficiency and shortcomings (personal, professional and organizationally). It covers every part of the organization: engineering, stakeholder management (regular quality feedback and refactoring), team interactions (geeky developers must now talk to each other and the rest of the world—heaven help us) and customer satisfaction."

You can mess with the methodology to some degree, but if you don't adopt the mind-set wholeheartedly, assert developers, don't bother. "I'd hope my CIO understood that half-agile is very, very dangerous," says developer John Overbaugh. "If the organization wants to test out agile, we need to pick a project or two to be agile. What we don't want to do is make each project 'somewhat' agile." Doing so, he feels, dooms all to failure. Part of what an agile developer rejects is cultural impediments and attitudes, including a management demand for command and control, or customers and business analysts expecting to throw the requirements over the wall and ending up with an "agile" result.

That's why support from the top is critical to making agile development succeed in an organization.

Often, its adoption is from the bottom up—developers seeing agile as a solution, and then selling it to senior management...or trying to. CIOs need to adopt the agile attitude, these folks say, or it won't be accepted successfully throughout the company.

Process-oriented managers who are used to the waterfall model may have a hard time adjusting to the different philosophy, and that can cause friction.

In the waterfall model, software development is seen as flowing steadily downwards (like a waterfall) through different stand-alone phases: requirements analysis, design, implementation, testing, integration and maintenance.

Agile developer Tim Ottinger suggests that IT managers begin by looking at agile values, to learn whether they can really buy into the methodology. "Agile requires a certain release of control and change of values," Ottinger says. "If they've enshrined long hours and centralized control, or if they're suffering through relational problems (labor/management style), or if they want larger commitments done faster instead of more measured, regular success...well, you see. There is a reason that some companies have a lot of trouble transitioning."

So what are those values? Jeff Tucker, software engineer at Milliman Care Guidelines, explains, "Agile is basically the idea that requirements change because people change. You develop software so that you get rapid feedback and can accommodate change rapidly and easily.... As long as you're doing something that realizes that benefit, you're agile. A lot of executives either don't understand that and therefore fail to see the benefit, or they're extremely dogmatic about it. And either of those situations is a disaster that's either happening or waiting to happen."

Sutton adds, "If you look at the U.K.'s only real huge scale agile success, [global service provider] BT, it took the CIO, Al-Noor Ramji, to provide real executive leadership and say, 'You have to do agile.' That is the best support for the real innovators in development teams to roll out process agility." After BT nailed the process (Scrum, XP and a few others thrown in) and they agreed on a language for now (Java/JavaEE), says Sutton, their ongoing task became challenging people and entrenched cultures.

3. Agile Changes More than Development Workflow

Part of an organization's tackling the agile development process is accepting that it will—and should—change company culture as well as process.

The important changes include pervasive use of feedback and increased collaboration—factors that force business and IT colleagues to be truthful with each other and bring visibility into problems an organization may have to address.

Consultant George Dinwiddie from iDIA Computing suggests using a burn-down chart to track project progress. Doing so, he says, shows how progress is converging (or not) on the goal line and lets a manager know when to take action. "That action might be to move the goal line, or to make changes in the way the software is being produced to try to speed it up."

When a developer asks the business customer about a feature, he gets feedback on the desired goal; the customer can see from the incrementally growing software how well it works. Says Dinwiddie, "The feedback should be generated with as little delay as possible. Reducing the time between when a decision is made and when it is validated will pay off in reduced waste."

Another effect is increased long as management recognizes its side effects. "Everyone in the organization should be collaborating, with the organizational end goals in mind, instead of just minding their own business and doing their assigned jobs in isolation," says Steven Gordon, an independent agile software development coach.

Note: That collaboration brings to the surface a lot of issues, problems and obstructions in the organization. Do not kill the messenger, Gordon urges; agile is not creating these issues, only making them more apparent. CIOs have to prioritize and address the problems. "If the organization appears to be ignoring these issues after they are surfaced, people will come to believe that the agile principles make no difference and will go back to just minding their own business and doing their jobs in isolation," Gordon says.

But in the widest sense, a long-term effect of agile development may be truthfulness. "I won't mince words: If you can't be truthful, you can't do agile successfully," says Mishkin Berteig, president of Berteig Consulting. "Truthfulness is the foundation of the practices, it is the foundation of even the principles. If you look at the agile manifesto, you can see that the idea of truthfulness is at the foundation in communication, in human interactions, in construction, and in customer-vendor relationships."

Berteig is often surprised how often this concept is a revelation to executives. "Most of our organizational cultures in the corporate world are based on distrust and a decided lack of truthfulness. To change to a culture of truthfulness is a deep, long and difficult change—just like any other culture change!"

Yet, he asks, without a culture of truthfulness how can one learn from the inevitable mistakes? How can an organization create mutually beneficial relationships with clients? How can you be certain that at the end of each iteration the team is delivering what it really says it's delivering, or be certain that the stakeholders really want what they say they want? Berteig adds, "Traditional bureaucratic processes and procedures are often a result of a lack of truthfulness and an attempt to make up for that deficit."

4. Agile Doesn't Mean "Chaos is Good"

Managers who are comfortable with one-step-at-a-time methodologies may be uncomfortable with the iterative nature of agile development. But agile development doesn't imply disorganization.

Isaac Sacolick, vice president of technology at BusinessWeek and previously founder of TripConnect, says that while agile gives sponsors the ability to react to market conditions and reprioritize requirements, "Agile does not preclude long-term planning, architecture, standards or project management. Enterprises have to think through how and where to apply agile development."

1 2 Page 1
Page 1 of 2
Download CIO's Winter 2021 digital issue: Supercharging IT innovation