Most of the 400 presentations at the Agile 2008 conference, held last month in Toronto, were geared for developers and testers. But the event held more than a few revelations and "Aha!" moments for IT managers, particularly revolving around team workflow, business value, company culture and the new role of the manager. Here are the key messages communicated by and to the Agile community.
1. Trust the Wisdom of Teams
Software is a result of a team effort, so a manager's main concern should be nurturing the team. But how many of us see our roles as enlightening an otherwise substandard and underperforming bunch of geeks who would escape their duties the moment we take our eyes off them? No, we never talk this way, but smart developers can conclude from our actions when a manager does not actually trust the team. The result is lower morale, underutilized brain resources and lost opportunities.
In his conference keynote, James Surowiecki, author of The Wisdom of Crowds, described how groups demonstrate better decision-making results than do individuals. We need to aggregate the intelligence available to organizations, he said, and develop frameworks for making near-optimal decisions.
Among the pitfalls that Surowiecki described was the importance of diversity because homogeneous groups become "dumber." The role of devil's advocate is essential, he said, to avoid group degradation. However, over time, a homogeneous group gets used to its devil's advocates and learns to disregard their reasoning. As a result, you need to mix up who wears the devil's-advocate hat.
2. Even Self-Organized Teams Require Coaching
Command-and-control practices inevitably lead to substandard results in software development. However, it would be naive and irresponsible to imagine that turning teams into self-organized machines means abdication from managerial duties. On the contrary, Agile principles require managers to become leaders, and that goes all the way up to the CIO.
Many presentations revealed several results-based techniques in team dynamics. One was a tutorial on "Coaching Self-Organized Teams" delivered by Joseph Pelrine and Steve Freeman.
Pelrine and Freeman focused most of their attention on models managers can use to leverage team conflict, such as the Abide model (attractors, barriers, identities, diversity, environment). By changing those parameters, the presenters said, you can change the results. Using the Flow model, managers can pay attention to the acceptable level of challenge based on an individual's skills to find a balance between anxiety and boredom; they suggest you can manipulate the flow channel based on the learning patterns team members demonstrate.
3. Adapt or Get Out of the Way! The New Manager's Role
What is Agile leadership? What does it mean to lead in an "agile" manner? This was the theme of the hands-on, dynamic and creative workshop "Agile Leadership" given by Johanna Rothman, the author of Behind Closed Doors and Manage It!, together with her coauthor Polyanna Pixton. The workshop concentrated on collaborative environments, trust, transparency of information, and building productive and sustainable teams.
The group considered that the most important duty of an Agile leader is to build trust on every level; it compared situations where "trust has not been built yet" and "trust has been broken." Trust is a fragile binary state; regaining trust is much more difficult than building it. Sometimes, broken trust cannot be rebuilt; a leader needs to simply accept this fact and move on, said the presenters.
According to Pixton, a key factor in building trust with the team is a leader's consistency in making decisions and what these decisions concern. We managers often forget the second part of that statement. When we do so, it leads to micromanagement, underutilization of the wisdom of teams and eventually to the deterioration of trust.
No matter how many people are on your team, the presenters insisted, you need one-on-ones with every team member at least once every two weeks. This includes the leader's peers. While this maxim hardly seems feasible for larger teams, every inch of retreat from it costs in trust deterioration.
The workshop was full of such "simple revelations" that offer "obvious" but often underutilized bits of wisdom. For example, one participant asked, "How do you make team members trust each other?" Rothman immediately replied, "You trust them first!" Perhaps the answer seems obvious, but do we always "walk the talk"?
Not every bit of advice is as easy to adopt. One presenter said, "Never rescue your team." Teach as little as possible, she said; instead, create as many options as you can. The team should feel the same pressure its manager feels. It's the only way to force your team to think and to offer solutions; and, she said, it's the only way to create a self-organized team. The manager's duty is to discuss with the team the risks associated with each solution. But when a team member comes to you for a solution, said the presenters, return the baton to him by first asking, "How would you do it?"
For example, one presenter said to a subordinate, "I have to apologize that I had to step in and do it for you. You probably think now that I do not trust you anymore. I apologize; this will not happen again." That's a tough lesson for a conventional manager!
Next: The Yearly Employee Appraisal, and other ways to de-motivate teams
4. Motivate, Don't De-Motivate: Appraisals, Bonuses and Compensation
Managing IT teams includes human resources (HR) tasks. According to Mary Poppendieck, one of the icons of the Agile movement, everything you are doing in your HR functions is wrong, because it is supported only by our illusions, not by facts.
In her presentation, "Appraisals and Compensation: The Elephant in the Room," Poppendieck offered her view on appraisals, bonuses and compensation, and on their dramatic negative impact on performance. She discussed the history and literature of reward systems in application to environments that require collaboration—a topic often avoided, as it causes conflict between rewards and teamwork.
Everyone hates annual appraisals. A significant amount of managerial time and energy is wasted on this process, bringing very little value. (The first known piece of literature about the negative influence of the appraisal system comes from sixth-century China, Poppendieck said.) Few people consider appraisal and reward systems fair, particularly in the eyes of the receiver. Our appraisal practices are based on assumptions that are seldom specified or confronted, such as the motivation to improve performance, career and development guidance, a paper trail for corrective action, and a basis for pay and promotion.
Step by step, Poppendieck demonstrated that these goals are rarely (if ever) achieved, but the harm they bring is almost certain. The only way to change behavior is to change the consequence, not the antecedent, she said. People invest their souls into their job only with positive reinforcement. When faced with negative reinforcement, people do just enough to avoid the threat.
Conventional appraisal and reward systems often create competition within the team. The consequences are obvious: If managerial efforts to create a collaborative environment contradict the company's appraisal system, team members will always believe what the appraisal system suggests. Should we be surprised when our incentive systems extinguish collaboration if individuals compete for rewards?
Incentives cannot solve a systemic problem. Nor can incentives increase training or skill. In software development, Poppendieck said, most people think others are motivated by money, but claim they personally are motivated by other factors.
Another assumption built into typical appraisal and reward systems is that an individual's performance can be reliably and unambiguously assessed. However, this is true only when performance can be objectively measured and attributed to individuals and when individual jobs have almost no interdependence. These conditions do not apply to software development.
But then what do we measure, and on what should we base our appraisal systems?
According to Poppendieck, it is difficult to come up with a good and sustainable system in the software development domain. Since most systems tend to demotivate people and teams, it is safer to abandon appraisals altogether. Use other means to motivate people, she said, and to create a high-performance culture.
5. Write Value-Based Vendor Contracts
Another long-standing Agile tenet is "reduce the waste." In this context, "waste" is anything that does not bring clearly defined and immediate value. Today, the Agile movement is knocking on the door of the CFO, offering new approaches to software development vendor contracts.
Jeff Sutherland, one of the Agile movement founders, wants the higher performance demonstrated by Agile teams to be rewarded. In his talk, "Money for Nothing and Your Change for Free: Agile Contracts," he reminded us that clients are tired of vendors who promise low-ball prices and then make their money on change requests. Current practice suggests that vendor and client agree at the start on project overruns. Sutherland offers a way out for both parties. His approach is:
Create a project backlog identifying all the features to be developed. Prioritize the features on business value (for example, expected ROI).
Offer a usual fixed-price contract with time and materials (T&M) for changes.
Add a "change for free" clause that allows a client to introduce a change and to remove features from the bottom of the prioritized list. The overall amount of work (project size) would not change; the vendor assumes the risk of late delivery.
If a client is unwilling to remove the less-important features, the contract reverts to a regular T&M schema for changes.
This approach looks attractive to a client because it permits scope changes. It also avoids the trap of high cost of changes.
Sutherland also suggested that Agile contracts include a "money for nothing" clause. A high percentage of developed features are never used. When a development team releases software in stages, with the highest-priority and best business value features completed first, at some point a client may be satisfied with the partially completed project. In this case, the client would have the option of terminating the contract at any time, for 20 percent of the remaining contract value. This allows the client to get back 80 percent of the remaining money without severing the vendor relationship.
Of course, the "money for nothing" clause cannot be applied if the project backlog is not prioritized or if trust between management, the development team and the client is lacking.
Next: Building Agile sustainability in the organization
6. Avoid Building Plank Roads
A movement's strength can be evaluated based its self-criticism, and several presentations were dedicated to questioning Agile. For example, in her presentation, "Expanding Agile Horizons: The Five Dimensions of Systems," Mary Poppendieck raised issues of Agile's sustainability, using a story about plank roads as an analogy.
About 160 years ago, she explained, the U.S. had a massive boom in plank road construction. The road construction technology required high capital investment but brought immediate positive result. However, plank roads deteriorated quickly; their annual maintenance was 20 to 30 percent of the initial cost. The approach was eventually abandoned. Is Agile a sustainable solution, Poppendieck asked, or a plank road?
It wouldn't be the first such "silver bullet" offered to the software development community. High-level languages initially promised to solve what Edgar Dijkstra called a "software crisis" in his famous Turing lecture 50 years ago, but COBOL, Fortran and PL/1 made programmers' jobs more complex. Every industry veteran remembers the waves of structured programming, 4GLs, rapid application design, "programming without a programmer" and other solutions to all our problems. With time, all these "universal" solutions found a much more humble place.
Waterfall was a plank road, said Poppendieck, aimed at solving the process problem by applying project management methods already used in other industries. But its applicability to software development was dramatically overestimated, and the ramifications of this strategic mistake haunt us to this day. She cited an article from ACM Software Engineering Notes from April 1982: "To contend that any life cycle scheme, even with variations, can be applied to all system development is...to fly in the face of reality."
Said that article, "System requirements cannot be stated in advance, not even in principle, because the user doesn't know them in advance—even in principle." The same paper suggested that our development process changes the user's view of the requirements. To think that this was said in 1982! Poppendieck also suggested additional approaches to provide longevity and sustainability [PDF] to system development.
7. Who Do You Trust?
The presentations given by author Linda Rising were the jewel of the conference. Deeply philosophical, appealing to both the brain and the heart of the audience, they call for revisiting the foundations of one's value system.
Rising's presentation, "Who Do You Trust?" was about team building, but it went way, way deeper. It was about building ourselves, not just building software in teams.
Rising reminded her audience of something we know but try to forget: We are all guilty of stereotyping when we interact with others. We subconsciously label people. We, as managers, sort employees into "winners" and "losers," and we do it within a few days. We forgive our own behavior but not others'. And, by labelling others, we lose all other dimensions of their talents and complexity.
We do this to ourselves, too, she said. Doing so limits our behavior, our talent, the ways we communicate and the results we achieve.
Moreover, stereotyping is so powerful that we even refuse to consider facts that contradict our expectations. We filter incoming information and rationalize away what contradicts our stereotypes. "Stereotypes change our behavior. Our behavior has an effect on others' behavior, and without anyone understanding any of it, you have a self-fulfilling prophecy," said Rising. Once we classify a person, he or she is lost to us: We will not notice anything this person does or says if it goes against our stereotype. "Reality doesn't matter anymore," Rising added. Don't we support this statement by signing a vendor contract where the requirements are set up front, once and for all?
We all believe we are unbiased, pointed out Rising, and we don't want to notice reality. The Museum of Tolerance in Los Angeles has two doors: one for those with prejudice and one for those without. Everyone tries to enter through the latter, Rising said, but it is permanently locked.
Not only are we all biased, but our biases are extremely strong. For example, as the Blue Ghosts and Red Genies experiment demonstrated, group behavior even trumps religion. (For more on this experiment, see Lutfy N. Diab's "Study of Intragroup and Intergroup Relations among Experimentally Produced Small Groups," in Us and Them: Understanding Your Tribal Mind.)
So, there is no hope, then? There is some, Rising assured us. "Yes, we are hardwired to classify 'us versus them,' but we are also hardwired to work in teams," says Rising. We are not hardwired for a particular classification. Therefore, if we know that we subconsciously endorse anything and everything within our group then maybe we can use this to build stronger teams that, for example, include a client. Maybe we can exclude managerial actions that work against team building. Remember those appraisal systems that create competition within the team from Mary Poppendieck's presentation?
Cooperation in work toward shared goals builds strong ties and helps resolve existing conflicts. "This cooperation must be nourished at all levels in the system, building a sense of interdependence that lies at the heart of a culture of peace," said Rising. Isn't this the same as face-to-face communication, which Agile culture promotes through daily stand-up meetings, pairing, short iterations and retrospectives?
We have scientific evidence of cooperation. Rising cited an experiment in which two primates cooperated by pulling a bowl of fruits toward one of them. They cooperated even though one primate knew she would not necessarily benefit. But in fact, almost always, she got part of delicacies from her luckier partner.
Ours is a culture of social interdependence. Team members have common goals. Everyone is linked with others so that one cannot succeed unless others do. Individuals can reach their goals if and only if the others in the group also reach their goals. Thus, individuals seek outcomes that are beneficial to all those with whom they are cooperatively linked. As a result, respect for others' abilities and contribution is produced. Mutual efforts improve both individuals and the group. This results in psychological health and increased self-esteem, decreased anxiety and depression. "Is this why Agile teams are better?" asked Rising.
These were only a few of the high points from the Agile conference. I could continue for many more pages. Yet, the lessons I learned reminded me of an important issue that many CIOs appear to miss. Time after time, corporations introduce Agile without understanding its key philosophical distinction: Agile development is not a set of instructions—it is a mind-set. If you implement Agile techniques as a set of prescriptions, the result will be much worse than any waterfall. You will discredit the Agile idea, and also you will fail the project, break the existing (albeit waterfall-ish) mechanism and ruin team morale. Don't blame the methodology. When I hear a manager saying, "Agile methodology tells us that at this point in our project we need to do this. So, go do this," I can hardly refrain from smacking this "manager" across his empty head. THINK OUTSIDE THE BOX, PEOPLE! YOU ARE IN SOFTWARE DEVELOPMENT!