Hiring Software Developers: The Agile Aptitude Test
Menlo Innovations' Richard Sheridan and Lisamarie Babik explain how "extreme interviewing" can help hiring managers choose Agile developers who are ready to apply its principles.
By Matt Heusser
The Agile development model has spawned hundreds of books and dozens of conferences. There’s a significant track record of successful companies moving to Scrum, Extreme Programming (XP), and other lightweight, highly collaborative development models. Agile can fundamentally change the way software development is done, yet the methods for recruiting and hiring developers hasn’t changed a bit. One company has adopted a new approach, and they share their methods here.
Most senior programmers and managers grew up in a time when software engineering classes emphasized “complete, consistent, correct” specifications and the ideal developer work environment was a quiet, private office. It was a time when the way to be most effective as a developer was to be left alone.
The management team at tech problem troubleshooting consultants Menlo Innovations needed to hire several programmers in 2004. But traditional interviewing methods, with the traditional résumé and interview process, failed to take into account a developer’s aptitude to work in a highly collaborative workspace. Instead, Menlo’s founders Richard Sheridan and James Goebel decided to implement an “Extreme Interviewing” event, led by Lisamarie Babik, Agile advocate and Menlo’s evangelist. CIO caught up with Babik and Sheridan right before a presentation of their ideas at XP West Michigan to discuss practical implications of hiring for Agile aptitude, and how to add these skills to an existing enterprise.
CIO: Where did the initial idea for the extreme interview come from?
Sheridan: After experimenting with extreme programming for six months in 1999 at Interface Systems, where I was the vice president of development, my boss and CEO Bob Nero asked me to double my team—from 14 to 28 developers—as quickly as possible. I knew that traditional interviewing practices wouldn’t work. My concern was that it would be impossible to describe what paired programming in an open and collaborative environment would feel like. My fear was that we would successfully hire 14 more people who would quickly learn to hate the new environment. I had to find a different way.
Babik: We used it for the first time at Menlo in April 2004. We were sitting around talking about how we needed to hire some folks for a new project we were starting. Rich offhandedly said, “You know, what we need is an Extreme Interview.” I had heard Rich talk about it in passing, but didn’t really know the nitty-gritty details. I was about to find out.
CIO: What does the interview entail? How is it different than a traditional interview?
Babik: The interview is more of an audition than a traditional interview. Participants are called together in a large group; each is paired with another participant who is interviewing for the same position and an observer who is a current Menlo team member. During the course of the interview, candidates are paired with three partners and assigned three problem-solving tasks.
What they are told, however, is that we are not particularly interested in the outcome of the problem solving. Rather, what we’re looking for is good “kindergarten skills.” In particular we want participants to “make their partner look good.”
CIO: “Make your partner look good?” Can you clarify what you mean by that?
Sheridan: We are looking for evidence that you’re going to help out your partner. So our instructions include hints like, “If your partner is looking for an answer that you know, give it to them. If they get stuck, help them out of their rut.” Why? Because this is the way we work every day at Menlo Innovations. We want to begin the training in our process at the moment of first contact, the day of the interview.
CIO: What kind of results do you see from this process?
Sheridan: Out of the 40-50 people we bring in for the Extreme Interview, we usually invite back 16-20 people for a second interview based upon their collaboration or kindergarten skills. As a result, the 8-10 who make the final cut have already demonstrated their ability to team with others.
Babik: At first, we only used this process for programmers, but we found the process could be used for other roles, such as project managers and high-tech anthropologists. When we didn’t use this process, we found that new team members struggled to fit in. It’s almost like Extreme Interviewing has become a rite of passage to join the team.
CIO: Now, long after the initial hiring spree: how has your staff retention been? How do these employees differ from those hired using traditional methods?
Sheridan: We have what I call a “negative attrition rate.” We do lose people over time for a number of reasons. However, many end up coming back and we welcome them. We find they are actually even more dedicated the second time around.
Babik: There are also those that make it through the initial process and still wash out in the long term. It’s just a fact that no interview process is perfect. We acknowledge that we might miss some good candidates with this process.
CIO: If you had to do it all over again, what would you do differently?
Sheridan: We have used this process two or three times per year since 2004. We continue to refine it, experiment with new ideas, etc. But the basic process has remained the same since we first initiated it in 1999.
Babik: I’ve asked myself that question many times when using a “traditional” hiring method, after I was burned by someone who wrote an excellent résumé, but lacked the skills—soft or technical—to back it up.
As for extreme interviewing, we’re always trying new things to improve our results. Most recently, our changes have been to the evaluation process based on some things we read in The Experience Economy by James H. Gilmore.
CIO: Put yourself in the shoes of a technology executive for a mid-size company considering an Agile transition. What advice would you give for how to change hiring?
Babik: Absolutely. Don’t focus on what they already know. Focus instead on whether or not they exhibit a life-long love of learning. Are they eager to learn new things? Are they excited? In today’s fast-paced society, the ability to adapt to change is paramount for success.
CIO: Our readers would probably agree that collaboration is key to any software endeavor, regardless of methodology. If you could give one idea to a technology executive—just one tweak to the hiring process to hire for collaboration—what would it be?
Sheridan: The traditional interviewing process is two people pleasantly lying to one another for an hour or two and then making a really big decision that they will live with for years to come. Harkening back to Thomas Edison’s interview process: Try as hard as you can to simulate the work environment in the interview process.
Babik: I’d have to say, “Align your inside perception with your outside reality.” Be upfront and honest about who you are and about the environment for which you’re hiring. If you’re reluctant to share some aspect of the culture or people, it might be a sign that you need to re-examine corporate priorities and the environment you created for your team.
CIO: Where can our readers go to learn more?
Babik: We conduct tours of our workspace almost every day. All people need to do is e-mail or call and we’ll arrange a time.