CIOs, recruiting is sorting wheat from chaff

The phrase that “people are our greatest assets” is something of a cliché, yet in technology there is some truth in it. Studies done by IBM back in the 1970s showed that the top one per cent of programmers were up to 10 times as productive as the average, and yet had lower software defect rates despite the volume of code they were churning out.

Anyone who has run software development teams will acknowledge that there is a gulf between the best programmers and the also-rans. Yet most companies pay little attention to the process of selecting the best candidates. All too often the process consists of wading through a pile of CVs, picking the most promising ones and conducting an interview or two before making an offer.

A few years ago I spent some time researching this. It turns out that considerable academic research has been done in this area, and there are a number of tools and techniques that can be brought to bear. A striking finding was research that measured an employee’s performance five years after they were hired, comparing their performance rankings with their scores at their original interview. The research looked at companies that used a range of techniques and measured the correlation between job success and the selection techniques used. Companies that did ‘normal’ interviewing achieved a correlation of around 0.15 with job success (a correlation of 1 is perfect, 0 is entirely random), and following up references fared no better (0.1 correlation). For all the time and trouble that is put into job interviewing and selection, many companies use exclusively a technique that is barely better than rolling a dice.

Intriguingly, the use of ability tests (essentially an IQ test) fared much better, with a correlation of around 0.5. This is still not fantastic, but is a lot better than regular interviews and shows a similar level of correlation across all job types, from finance to sales, administration to management: smart people just do better on average. A variant on this is a job-related test: if you were hiring a welder you would ask them to do some welding, while you could ask a programmer to write or critique some code.

It has been shown that the more sophisticated personality tests can be effective when matching up personality types to particular job profiles. Carefree creative types might not be an ideal match for accounting or actuarial jobs, while thin-skinned sensitive people are likely to struggle in sales roles, where they will encounter rejection a great deal. Statistically, the use of such tests alone has a success correlation of 0.3. This is still less useful than ability tests, and personality tests are also more complex to interpret, and require more qualified people to administer than ability tests.

You can also improve your success rates by sprucing up the good old-fashioned interview. By having a highly structured, evidence-based interview you can significantly improve upon what passes for interviews at most companies. I remember when I was at Shell doing the graduate milkround interviews. Before being allowed out on campus I was sent on a five-day course to deliver a half-hour interview. The interviews that resulted were carefully structured and repeatable, ensuring that all interviewers were marking to the same standard. Having such well-structured interviews helps the interviewer considerably as well as being fairer on the candidates, and the process raises the success correlation to 0.3.

The best results are achieved by combining techniques. If you carry out structured interviews, ability tests and personality tests together then the success correlation goes up to 0.65. Of course this is still not perfect and some mistakes will be made, but there is a big difference between a correlation of 0.65 and 0.15, especially given the considerable costs of hiring staff who turn out not to be suited to a role, and then do not perform well.

What rather surprised me was the degree to which these assorted tests can be greeted with scepticism and even downright hostility. Although 80 per cent of the Fortune 500 use such tests, they are much less common in smaller firms, and yet the effort involved in training your hiring staff in them is not that great. The costs are dwarfed by the costs of taking on unsuccessful candidates, training them and then having to replace them.

Even with the widest array of techniques you are still going to get such situations, but you will have a lot less to deal with. Still, inertia is a powerful force, and it is hard to get people to change the way they are used to doing things. After insisting on using ability tests to screen candidates prior to interview, one exasperated recruitment adviser said to me: “Hardly any software firm does these kinds of tests, only companies like Microsoft and Google”. Quite.

About the author

Andy Hayler is founder of research company The Information Difference. Previously, he founded data management firm Kalido after commercialising an in-house project at Shell