Waterfall is washing away IT’s potential

The statistics are depressingly familiar – how all too many technology projects end up over budget and over deadline, with an average failure rate of as high as 60 per cent.

Markets move so quickly and customer expectations are now so high that it is impossible for us to predict what customers will need next quarter, let alone in 24 months. Despite this, we ask CIOs to scope out their business system expectations far in advance of deployment.

CIOs are finding agile development attractive. Agile puts the emphasis on proper feedback and communications across the enterprise. For organisations that have dipped their toes in the agile water, the benefits are clear. Most organisations report to analyst firms like Gartner that development time with agile is three to five times faster than with other approaches. Gartner has already predicted agile to become ubiquitous in the next three to five years.

The traditional waterfall approach to software projects creates a delay as the business waits for its IT needs to be translated into a system. A waterfall approach is inefficient and loaded with risk in our experience. Each stage happens sequentially, following an exact specification drawn up and agreed at the outset; beyond that point, users have almost no involvement in the project until it is too late. This places enormous pressure on the organisation to know exactly what is needed in advance, and what it will need in the future – something that is increasingly impossible to predict given commercial volatility.

There can be a much more collaborative and fluid approach to development with agile. When ‘sprints’ of software functionality development are delivered to users at regular intervals, there is time to test and refine with user feedback. This project phasing not only ensures that real needs are being met, but that the finance can be signed off in phases too.

CIO however, need to be realistic: agile requires a big change in the way IT departments operate. Separating roles of business analysts, software developers, code testers and systems operators does not work in agile. Instead, stakeholders become cross-functional project members ensuring all members maintain consistent productivity.

For programmers accustomed to solitary activities or working with specialist groups of analysts and designers, the suggestion of shared learning, reflection workshops, pair programming, and collaborative decision making may be a significant change. A new generation of agile-friendly development platforms provide an abstraction from the detailed code so that stakeholders can focus on analysing user needs and ‘composing’ applications rather than coding them. This is how agile teams deliver business value, not lines of code.

Traditional IT project management styles move from their traditional role of ‘planner and controller’ to facilitators who direct and co-ordinate the collaborative efforts of those involved in development – thus ensuring that the creative ideas of all participants are reflected in the final decision.

The adoption of agile methods has created demand for a new generation of tools and platforms that facilitate an iterative approach. Some even allow you to harness feedback from end users that automatically feeds into an agile project management suite. This way, improvements are added iteratively, and users have something to work with quickly, giving the organisation the quick wins it needs.

Development environments and tools to help facilitate such a different way of working will continue to play a critical role.

One of the biggest barriers to change may an IT team’s dependence on tried and tested waterfall techniques. This can be addressed through training, re-skilling and a shifting of the balance of power to management and away from the development team.

About the author::

Derek Roos is Chief Executive Officer at Mendix; derekroos@mendix.com