The Secret to Software Success

EVERYONE KNOWS the rules for managing software projects. You’re supposed to test the code. Frequently. You’re supposed to take ownership of the process, have a business sponsor and keep an eye on developments. You’re supposed to be alert to problems and potholes, and you’re supposed to be ready to step in to fix the glitches as they arise. Or kill the project if it looks hopeless.

The only problem is the rules don’t help.

In our Jan. 15 “Bugs!” story about companies struggling with poorly written software, we suggested that CIOs frequently test and take ownership of the process. The year before, CIO ran “Another Trip to Hell” (Feb. 15, 2000), and about a year before that we published “To Hell and Back” (Dec. 1, 1998)?both stories about software project failures and the lessons they teach: “Test, test and test again,” “Ownership is essential” and “Don’t let a doomed project run on.”

And lest you think this is a relatively recent development, you should know that in 1967, Ken Kolence, cofounder of Boole & Babbage, a pioneering software testing company, published a paper for the first NATO Software Engineering Conference outlining some best practices for the new field of software engineering. It featured instructions on how to test code, assign a manager to own a project and kill projects that were going nowhere.

That’s right. The accepted wisdom for managing software development hasn’t changed in almost 35 years.

And what has the accepted wisdom achieved?

Not much.

We Begin with a Litany of Failures

A landmark 1994 white paper, “The Chaos Study,” published by The Standish Group, a West Yarmouth, Mass.-based consultancy, reported that just 16 percent of software projects succeed. The rest either failed (31 percent) or were challenged (53 percent)?a term encompassing cost and time overruns and missing features.

Standish turned “Chaos” into a longitudinal study, collecting case studies (30,000 and counting) and each year publishing success, failure and challenge rates. Its 2000 report, “Chaos in the New Millennium,” is about as encouraging as its cover art, which includes the grim reaper rising through clouds, brandishing his scythe.

Outright failures, Standish reported, have declined from 40 percent to 23 percent during the past five years, but challenged projects swelled from 33 percent to 49 percent in the same period. That’s bad because challenged projects often are more painful than projects that simply fail, like peeling off a bandage slowly instead of quickly. And they’re often just failures-in-waiting, dallying dismally until the patience (or the money) for getting them right runs out.

Put another way, almost three-quarters of all software development in the Internet era suffered from one or more of the following: total failure, cost overruns, time overruns, or a rollout with fewer features or functions than promised. CIO could run “Still Yet Another Trip to Hell” next month. The article could include Nike’s glitch with i2’s inventory software, which prompted Nike CEO Phil Knight to wail, “This is what we get for our $400 million?” It could mention Sobey’s, a Canadian grocery chain that in February suddenly and publicly canned an SAP project. (You could practically hear the sonic whoosh of $50 million following the press release into the trash bin.) Or it could tell the tale of a 100-year-old management consultancy that, according to an inside source, spent tens of millions on a failed ERP deployment, then tried and failed again after sinking another $8 million into the black hole.

But what would be the point?

A litany of failures and maddeningly familiar advice?Nike needed to test more; Sobey’s needed a project sponsor; the consultancy should have killed the project?simply restates what everyone already knows. The problem isn’t with the advice; the advice is sound. The problem is bigger. The problem is the development methodology itself. The problem, says Martin Fowler, programmer and cofounder of The Agile Alliance, is that companies construct software much the way people build bridges. (And Fowler should know. His wife, Cindy Chabot, is a structural engineer. Bridges are what she builds.)

Why Building Software Is Not Like Building a Bridge

A bridge begins with a blueprint based on mathematical certainties and predicted levels of tolerance that never change. The blueprint is followed precisely. Changes are costly and therefore anathema. The tools used to build the bridge are standard and don’t change during construction. The materials are familiar and behave predictably.

Because of all that, we’re pretty good at building bridges. They rarely fall down.

But bridges are built to do only one thing: connect two pieces of land so that people can cross between them without falling. Software tries to do many things, and many of them have never been tried. And the tools used to build the software change continuously.

Therefore, as long as software engineers act like bridge builders, they are doomed to fail. And the cost of that failure is beginning to rise.

“If this were the pharmaceutical indus-try, we’d be killing people,” says Joshua Greenbaum, analyst at Enterprise Applications Consulting in Daly City, Calif. Greenbaum tracks ERP, the clay pigeon of software failure stories. “There’s no reason to tolerate this level of failure. It tars the software industry. And frankly, it’s bad for the CIO.”

Check that. It used to be bad for the CIO. Now, in today’s economic climate, it’s disastrous. Nike issues an earnings warning, and Knight shuffles his executive team. Sobey’s ditches a grocery application and its CIO, Bradley Jardine. The consultancy ousts most of its C-level crew.

Desperate to avoid the scapegoat’s horns, some technology executives are finally beginning to take up arms against this sea of failure, redefining how software is built. They call it Agile Development, a disciplined, minimalist approach that’s both elegant and arduous, and maybe IT’s best hope to avoid “Yet Another Trip to Hell.”

We, the Programmers…

The nicest thing Fowler says about traditional software development is “It’s great except that it doesn’t work.” His accent is British Midlands. His outfit is L.L. Bean. His home-brewed coffee is Starbucks, but “better than they make it at the store.”

He spent several years programming at Coopers and Lybrand back when it was one of the Big Eight. Then he worked for eight years as an independent consultant for large software projects. “Programming, requirements analysis, I’ll even do project management if you force me to,” says Fowler.

He is also one of the 17 founding fathers of The Agile Alliance, a group dedicated to revolutionizing software development. The group members met for the first time in February, in ski country on Utah’s Wasatch Range. They knew something was wrong with the way software was being built, and they wanted to do something about it.

Coming in, no one agreed on precisely what would work. Kent Beck, a project manager for hire, touted Extreme Programming, a system that breaks a project into tiny steps. (See “Variations on the Agile Theme”.) You’re not allowed to go on to the next step until the first is proven to work. Jeff Sutherland, a methodologist and consultant, was wont to use a method he invented called Scrum, in which teams of programmers went over each other’s work every day. Jim Highsmith, yet another project management consultant, had published papers on Adaptive Software Development, which emphasized close collaboration between business and IT. And there were several other theories and methodologies.

Surprisingly, the 17 developers in fact were able to weave the common threads of their philosophies into the fabric of The Agile Alliance.

Agile means what it sounds like: fast and efficient. Small and nimble. Less money, fewer features, shorter projects. The Alliance framed a manifesto and posted it on its homepage (www.agilealliance.org).

How to Be Agile

Slash the budget. Small budgets force developers to focus on the essential. Small budgets also make it easier to kill failing projects. For example, imagine a project that has already cost $20 million. Imagine it’s going down the tubes. With that much skin in the game, it’s tempting for the CIO to invest another $10 million in an attempt to rescue it rather than take a huge loss. All too often, he ends up with a bigger one.

Jim Johnson, chairman of The Standish Group, says he forced the CIO of one Fortune 500 company to set a $100,000 ceiling on all software projects. No exceptions without approval from the CIO and CEO. Johnson claims the company’s project success rate went from 0 percent to 50 percent.

If it doesn’t work, kill it. Bring marketing, program and project management, and IT executives together at the beginning of a project and as it progresses to evaluate every piece of code in development. Is it doing what the business wants? Is it working? Any code that isn’t should be mercy killed.

This is called triage, and it’s “the perfect place to kill a software project,” says Pat Morgan, senior program manager at Compaq’s Enterprise Storage Group. He holds monthly triage sessions and says they can be brutal. “At one [meeting], engineering talked about a cool process they were working on to transfer data between GUIs. No one in the room needed it. We killed it right there. In our environment, you can burn a couple of million dollars in a month only to realize what you’re doing isn’t useful.”

Keep requirements to a minimum. Don’t start with everything you want the software to do; start only with what it absolutely must do. And don’t worry about writing all your needs down, because requirements change.

Every software project traditionally starts with a requirements document that will often have hundreds or thousands of elements. But The Standish Group estimates that only 7 percent of the features of any given application are actually needed. And a major reason for software failure is feature overload?when a programmer adds a feature that interferes with an essential process. Fixing that disconnect in turn creates another disconnect, and so it goes.

Tom DePauw, manager of IT at Cater-pillar Financial Services in Nashville, Tenn., is using Agile Development to build Cat FinancExpress, a massive Web-based system that integrates three older software systems used for helping customers finance heavy equipment purchases.

“When the project started, Caterpillar Inc. [the parent company] wanted to see the book,” DePauw says, referring to the requirements document. “I held up this single sheet of paper with a diagram on it and said, ’This is it.’”

Build on success, not hope. As often as once a week, and not less than once a month, complete a piece of software. Then have your business deciders test and approve it.

That doesn’t mean the software is shipped or deployed, but it must work and it must be bug-free. This is the Agile concept’s most radical departure from traditional development. In some software projects, no one shows working code for years.

Ken Moskowitz, CIO of New York City-based Standard & Poor’s, dictates weekly builds of his company’s software. Every week, the pieces being worked on are compiled and tested.

“We’ve had a great deal of success with it,” Moskowitz says of this rigid weekly schedule. “I don’t want things thrown across the transom. I don’t want ’Here, I think I need this. Go and build it, and we’ll see if it was what I really needed in the first place.’”

Keep your development teams small. The fewer developers the better. Developers should team code.

Proponents of Agile Development swear that team coding is more efficient and produces stronger code than solo efforts. But even Fowler admits that this is one of the harder tenets of Agile Development to accept. Who is teamed and why? How do you budget their time? Expect a learning curve.

Caterpillar’s Agile project (in progress for two years with a core team of 15 developers) encompasses 200 distinct financial screens and contains 1.5 million lines of code.

Assign non-IT executives to software projects. Non-IT execs should coordinate with the technical project manager, test iterations to make sure they’re meeting user needs, and act as liaison between executives and IT.

With business involved full time, “It’s hard for us to say after three months of iterations that we didn’t really know how it was going,” says Chris Colleran, CTO of a Salt Lake City market research outfit called SpreeRide. Colleran is using Agile Devel- opment to set up his company’s website and some back-end applications and has business executives full time on the project.

“Emotionally it’s hard to commit businesspeople to the development process, but it’s only counterintuitive because of the way it has always been,” he says. “If you think about it, the perfect developer would be half a businessperson and half a programmer.”

No More Software Development Failures, Please

Fowler has joined ThoughtWorks, a Chicago-based software developer that uses Agile techniques almost exclusively. ThoughtWorks has worked on the Caterpillar project, and it employed iterative development on a retail application for The Gap. For Cisco, it integrated sales software with financial tracking systems.

“The way software is developed is fundamentally flawed,” says ThoughtWorks CEO Roy Singham. “Think of it as climbing Mount Everest. Do you set a plan and say, ’On day seven we’ll be here and the weather will be this and I’ll need my ice pick’? You can’t do that.

“CEOs have irrational expectations for what [software] can do, and CIOs don’t have the gumption to tell them the truth,” Singham adds. “Agile is a much more honest approach. I believe it’s the next wave.”

So does Kolence, who thinks Agile Development is the best hope yet to render his 1967 paper irrelevant.

For Kolence, it’s been a long, frustrating struggle. “At North American Aviation in the early ’60s, I was responsible for programming commercial applications at one of the airports,” he recalls. “We never got anything done on time.”

Just last year, Kolence worked with an aerospace company that couldn’t write a requirements document, he says. It got so bad, he told off the head of the project and warned him that they’d end up trashing the software. Which is exactly what they’re doing now.

“It’s frustrating, especially when it’s all ad hoc stuff,” Kolence says. “There are times I’m just so unhappy with the whole field. You can hear the strain in my voice. This is why I’m semiretired. I’m working at a local hardware store now. It’s fun. It’s fun to deal with people who are fixing stuff. It’s fun to say to them, ’This is how you do plumbing.’”