How to Close the Big Data Skills Gap by Training Your IT Staff

Research firms paint a dire picture of a massive big data skills gap that will get worse over time. But companies like Persado, which uses big data to help marketers optimize their messages, are finding success training their existing staff in the new big data technologies.

By
Wed, October 02, 2013

CIO — It's difficult to talk about big data without also discussing the big data skills gap in nearly the same breath. But is it as bad as it seems?

According to a recent CompTIA survey of 500 U.S. business and IT executives, 50 percent of firms that are ahead of the curve in leveraging data, and 71 percent of firms that are average or lagging in leveraging data, feel that their staff are moderately or significantly deficient in data management and analysis skills.

These firms see real costs associated with a failure to come to grips with their data, from wasted time that could be spent on other areas of their business to internal confusion over priorities, lost sales, lack of agility and more.

big data

Forecasters paint a seemingly dire portrait of a skills shortfall that will only get worse over time. The McKinsey Global Institute estimates that by 2018, there will be a shortage of 1.7 million workers with big data skills in the U.S. alone—140,000 to 190,000 workers with deep technical and analytical expertise and 1.5 million managers and analysts with the skills to work with big data outputs.

[Related: Who's Training the Next Generation of Data Scientists?]

But Tim Herbert, vice president of research and market intelligence at CompTIA and author of its second annual Big Data Insights and Opportunities study, says the situation may not be as drastic as you think.

"Hadoop is a new technology and there really is a skills gap. But you can easily cross-train people. It's not that the technology is incomprehensible. You just need to take existing developers, analysts and admins and cross-train them."
--Sara Sproehnle, vice president of Educational Services at Cloudera

"There will be a situation where, at the highest levels, probably the Fortune 100, there will be a skills shortage," Herbert says. "For most medium and small companies, they probably will be able to satisfy their skills needs by a combination of retraining and additional staff. The tools associated with big data will mature. The capabilities and ease of use will mature over time and that will certainly help. Like a lot of other technologies, there will be individuals that maybe they weren't trained to do it but they will have an aptitude to work with data."

Hadoop Isn't Incomprehensible

Sara Sproehnle, vice president of Educational Services at Cloudera, provider of one of the most popular Hadoop distributions, agrees.

"Training has really been a strategic component of what we do at Cloudera," she says. "Hadoop is a new technology and there really is a skills gap. But you can easily cross-train people. It's not that the technology is incomprehensible. You just need to take existing developers, analysts and admins and cross-train them."

Continue Reading

Our Commenting Policies