Big Data Resources Are Right Under Your Nose

As businesses face a widening big data resource gap, they are looking within to identify the right resources to meet their needs.

gettyimages 763162321 1280x8001
Dell EMC

When it comes to turning big data into business value, organizations are increasingly turning to data science — only to find that data scientists are in short supply.

This is a point that’s been made over and over in industry research. For example, a workforce report by LinkedIn found that U.S. employers are facing a nationwide shortage of more than 150,000 people with data science skills.1 Meanwhile, the data scientist role made Indeed’s list of the best jobs of 2019. The site recently reported that job postings for data scientists grew by 78 percent in the 2015 – 2018 timeframe, and now offer an average base salary of $131,389.2

In light of numbers like these, it wouldn’t be a surprise if you have been wondering why data scientists aren’t responding to your job opening posted on Indeed. And maybe you’ve been asking yourself: Oh where, oh where are all those data scientists? Where are the people who have all the traits I am looking for?

In particular, where are the people who:

  • Have an amazing emotional quotient (EQ) that is joined by equal parts coding skills and statistical knowledge
  • Walk with a confident swagger, because they know everything about how a business runs, the industry it is in, the competition and all the data sets that exist to be analyzed together to find correlations and even causations
  • Know how to grab those different data sets, store them together, then write software programs to identify new insights, with which they confidently stroll into the CEO’s office, kick up their feet on her desk, and lay out the plans for the business to make billions with this newly gleaned information that no one else in the industry has
  • Are Ph.D.s with multiple-years of industry experience, a professional speaking circuit and documented mad skills with data management, data governance, MapReduce, R, Python, Splunk, Hadoop, Kafka, Spark, Hive, Yarn, MongoDB, CouchBase, RapidMiner and other technologies from businesses you normally see only on partner slides at a data analytics event

What’s the problem?

I recently conducted research to understand how IT managers who are responsible for finding the right big data resources are meeting their big data demands. This research provided valuable insight into the challenges businesses are facing:

  • One of the more opinionated participants observed that applicants responding to his open data scientist position “either understand the technology, but don’t know $#!% about our industry, or vice versa.” In fact, nearly all the study’s subjects identified something that was lacking: social, communication or interpersonal skills; advanced math or statistic skills; or knowledge of data management, data mining or some component of the technical piece.
  • Of the several areas participants identified as lacking, soft skills were mentioned more often than any other.
  • While the research did not directly specify “data scientist,” it did refer to “big data resources,” and the “data scientist” job title was one of four types of big data resources businesses said they were seeking.
  • Technologists, leaders and developers also were identified as needed.

The solution

These survey responses led me to understand that IT managers are no longer seeking one mythical, gifted person. Rather, they are looking to build a team of people with different, complementary skills. In addition, people who have specific desirable strengths, versus strengths across all big data areas, are already working for the same businesses that were previously looking fruitlessly to hire an uber-talented person from the outside.

Let’s take a look at key skills and their availability within most organizations. Do most businesses have

  • A group of people who understand their business and industry? Check!
  • Individuals who can code? Check!
  • Employees with strong communication skills? Check!
  • People who can translate findings into business-speak? Check!
  • Team members who know math and statistics? Maybe.
  • Employees who understand the latest data analytics technologies? Likely not.

So, businesses can turn to employees who are literally working right under their noses! They can send employees who already understand their business and who have strong soft skills to technical training, so they can build out their skill sets by learning about the latest technology. This approach is much easier and takes less time than trying to start with technically strong people and teach them how to communicate effectively and coach them in the specific nuances of a business and its industry.

How one company did it

After two years of being unable to find the right person from outside the company to come in and help provide value from big data analytics, the CIO of a large healthcare network decided to send out a company-wide email. He invited anyone who had majored in math or statistics, and who would be interested in applying their knowledge in a new way for the organization, to contact him. When I first heard about this approach, I wondered whether it was an anomaly in big data resource finding practices.

However, since completion of the big data resources study, and taking into account today’s widening gap between demand and availability of qualified big data resources, I have come to understand that looking internally has become the most common and successful approach to finding the right resources to meet businesses’ big data demands.

To learn more

You can find more information on topics referenced in this blog at

To learn more about unlocking the value of data with data analytics solutions, explore Dell EMC Ready Solutions for Data Analytics.


1 LinkedIn, “August LinkedIn Workforce Report: Data Science Skills are in High Demand Across Industries,” August 10, 2018.

2 Indeed, “The Best Jobs in the U.S.: 2019,” March 14, 2019.

Copyright © 2019 IDG Communications, Inc.