According to The 9th Annual State of Agile Report, released last year by VerizonOne (in conjunction with independent survey consultants, Analysis.Net Research) 94 percent of the organizations are practicing agile development. The report collected data from 3,925 respondents to determine the challenges organizations face when implementing agile.
According to our experts, it requires a shift in thinking for all parties involved and the full support of C-suite leadership, the business and developer teams. Here are some things to consider when making the transition from waterfall to agile.
"The waterfall development model is also known as the classic or traditional model for the systems development life cycle for software engineering/development. It is described as a linear and sequential model that has distinct goals for each phase. Once you proceed to the next phase, you cannot turn back, just like a waterfall flowing over steps of rock down a mountainside, "says Joe Mack an Enterprise Technology Transformation Consultant with Transphorm. The waterfall process works best in situations where expectations are clear and stakeholders and clients don't have the ability to change the scope of the project. Unfortunately, there aren't a lot of real-world IT problems that fall into that category.
"One of the major benefits of agile over waterfall is that you see a deliverable on an iterative basis and the Product Owner can decide to make changes to the product backlog, "says Sudhakar Gorti the CIO for Environmental Data Resources.
Getting agile right
Before taking on this transformation, organizations need to know what problems they are working to solve and how agile can solve those problems. Companies must also contemplate the following: What would you consider a successful outcome? How committed is your business to the transformation? Will your culture support this change? Do you need outside help?
Agile can do great things for your organization. It can create engagement between teams and workers that don't regularly collaborate. It can increase the quality of the code your teams create. It can shorten time to market. Increase customer satisfaction. It can reduce development costs too. But only if it's done right. "Done badly, agile development will create a lot more problems than it solves," said Nathan Wilson, research director at Gartner in a recent press release.
Get everyone on board
According to Gartner, the agile transformation is what it describes as a joint business-IT activity and requires everyone from the CEO to developers in the trenches to be moving in the same direction. "Use of agile methods has the capability to transform IT-business relationships and have a major positive impact on IT value delivery. However, the value will be delivered only if the CIO and the entire IT management team are dedicated to the culture change that is necessary for success," Wilson said.
Mack, agrees and points out that this is the heart of agile. "The iterative process for all agile methodologies features business involvement at every planning stage and even some feature involvement in the day-to-day operations."
Successful agile adoption needs to include everyone in the process from the start. The business and the development teams need to work together to define the specs, KPIs, timelines, budgets and more. "You have to engage business stakeholders from day 1. They tell the whole story, which you as an IT leader have to break down into story lines [a logical and testable group of functions] that make sense from an application and technology point of view," says Michael Spano, an executive manager of infrastructure services with IBM Resiliency Services.
Challenges when transitioning to agile
Agile can be tricky to implement, according to Gorti. "Getting everyone aligned with how the new process will work is a challenge. In addition, some people have misconceptions and/or lack of understanding to what agile really means." Educating your team and leadership on what agile is and how it can benefit your company is essential to getting the C-level buy-in necessary to make this transition.
Cultural Change is another big challenge, according to Mack, "Depending how long waterfall has been used, it is sometimes more difficult to change the culture of the development organization."
Derek Plunkett, Assistant Vice President of IS application development with John Hancock Retirement Plan Services, agrees: "One of the biggest challenges is aligning the mindset and behavior of the team towards team results instead of individual performance." One tip he offers IT leaders is aligning goals and monetary rewards for the members of the agile teams to help reinforce this behavior.
ROI can be difficult to nail down. Agile, Mack says, can be a frontloaded endeavor, in terms of cost. Sometimes the corporate leadership gets impatient at the length of time it takes to get a return on their investment, sending workers to class, paying for certification exams and consulting fees.
There can also be bumps along the way. "Stakeholders often want to time box or assume a certain velocity of the swarm (scrums/epics/cycles/sprints) that leads to poorly coded or tested story lines (functions). Additionally, many developers and stakeholders do not realize sometimes a new story line can have impacts on early ones, and regression testing or even rework might be warranted to ensure earlier story lines still operate as deployed. The 'good enough' approach vs. stakeholder expectations, results in endless development & test," according to Spano.
[ Related Story: 9 ways technology will change within the next 10 years ]
3 most common reasons for failure
[ Related Story: Leading digital change means moving beyond the mission statement ]
Errors in planning
"All requirements and estimates can only be done during the appropriate phase, and if the entirety of relevant information is not known, then it is all but certain that the estimates, and therefore the timeline (and budget) will be wrong. Before development starts!" says Mack.
"The second reason agile projects fail is due to scope creep, which can happen even if you engage as many stakeholders as possible early on. New data becomes available and 'simply must be included.' This delays swarms and often causes overall project delay," says Spano.
"It is most common for the scope to expand during the requirements phase as new requirements are exposed unless the team is extremely disciplined. This then lengthens the timeline and leads to missing original budget estimates. You can see projects die or be shelved at this phase because either they have become too costly or business priorities have changed since the initiation of the project," says Plunkett.
The inability to adapt to change. "When more information about the requirements becomes known, it is impossible to make changes to the requirements, other than enacting the change control process, which leads to ever more timeline and cost overruns," Mack says.
Should you bring in third-party agile consultants?
The answer varies and depends on where you are in the process. "If you bring [third-party consultant] in early and train the team and help with the super swarm or discovery portion things go much better than if you bring them once the Swarms are in motion, Spano says.
If you are starting from square 1, then it makes more sense to bring in someone to educate your teams. This real-world experience, says Mack, is paramount to early success with agile. "Unfortunately, sending an internal resource out for training in how to be an agile coach will not provide any of the real-world experience working in, and with, multiple agile development shops that is so crucial to your success. So, bringing in external help is riskier, but almost certainly worth it."
Bringing in the "right" agile coach is just as important, according to Gorti. "Bring in new talent who has had agile experience and be sure that this new talent can act as a role model for all current employees." This is also the time to identify the people on your team who you think will be the most impactful in the transition. "Additionally, identifying change agents within a team and getting them aligned by engaging them in the transition process can significantly accelerate adoption," says Plunkett.
You can avoid the shock of going through this transformation all at once, according to Plunkett. "Start the agile transformation with a small number of pilot teams. Once the proper training and tools are in place and the cultural impact is accounted for, additional teams can be transitioned." This type of approach tends to limits the risk of pushing the whole business to transition at once and allows leaders to thoughtfully assess and address any obstacles that are exposed during this pilot period.
Use what works for your organization
Agile isn't going to solve all your problems. According to Gartner, different classes of development problems will require different approaches. While it may seem that the development world has gone all agile, experts note that waterfall, hybrids and other methodologies will always have a place in the modern world of development. "Waterfall projects have their place when requirements are clear and fixed and time is not a critical factor, and while agile will also work the benefits are more significant when those things are not true. This is the world I live in," says Spano.