Data Science vs IT: It Shouldn’t Be a Battle

office worker tech1
Dell Technologies

Data scientists are increasingly being asked to deliver business value, and they are being held accountable for their results. Given this reality, data science teams can’t wait weeks for IT administrators to come up with the resources they need to develop proofs of concept and train artificial intelligence models. Yet too often, this is exactly what happens, leaving data scientists at odds with their IT departments.

This is a bad situation for all parties. If IT takes a traditional approach to resource provisioning — with a cost-center mindset and limits to system access — data scientists are likely to circumvent standard procurement processes and run their workloads in the cloud. That creates shadow IT that is both risky and costly.

Shadow IT is risky because of the lack of corporate governance and enterprise security that arises when IT users take their applications and data to the cloud without the involvement of IT administrators. And it’s costly in many ways. When multiple groups within an organization buy cloud services on their own, the enterprise loses out on potential volume discounts. The enterprise also misses out on data-sharing opportunities, because datasets are out there in their own disconnected siloes.

This is a huge problem. A McAfee survey found that the average organization thinks it uses 30 cloud services, but in reality it uses 1,935. As a McAfee blogger notes, “This disparity is shadow IT — and it’s expanding your attack surface.”[1]

If you’re in IT, it’s important to recognize this reality and approach the data scientist as a collaborator, rather than a gatekeeper. But too often, that’s not the way things play out. Instead, the interaction between the data scientist and the IT admin goes like this:

data science vs. it

Data science suffers, and business value is lost, when IT acts as a gatekeeper that limits access to computational resources.

Data science is different. It exists in an ecosystem of often open source tools and hybrid cloud implementations. In order to work effectively, data scientists require iterations, permutations and a very different mindset from IT than they have had in the past.

So how do you make this shift to a new way of doing business? Here is some simple advice on how your IT organization can be a more effective advocate for your data scientists:

Assume the role of a consultant.

Stop thinking of yourself as a cost center. Think of yourself as a consultant who enables stronger business results, and know that the data scientist is your partner in this process. Embrace your data scientist’s objectives above the concerns of managing costs and corporate governance.   Listen to what they want. Lean on people who have been down this path before you, such as the high performance computing and artificial intelligence experts at Dell Technologies.


Turn shadow IT into hybrid IT.

Understand why shadow IT is happening. See it as a project’s evolution into your management process, and then be willing to adopt shadow IT that comes your way. Once adopted, you can help alter it and morph it so that it matches the security and financial needs of the company. Realize that your attitude about this shadow IT has much to do with the level of information and impact you can have — rather than actually gate controlling it. To that end, jump in and be willing to take on vendor relationships enthusiastically, even if you weren’t part of the initial procurement process.

Be a service-driven IT organization.

Focus on customer service rather than gatekeeping. Work with your management team to alter the key performance indicators (KPIs) for your organization to help you evolve in the results-oriented IT infrastructure of the future.

Key takeaways

Data scientists have many challenges. IT infrastructure shouldn’t be one of them. With the full support of the IT organization, your data scientists can gain access to the resources they need to accelerate data analytics and the development and training of machine learning models via standard corporate processes. Your IT leaders, in turn, can help your data scientists deliver business value while avoiding shadow IT that can leave data at risk and drive up your overall costs.


[1] McAfee, “Are Your Employees Using Your Data in the Shadows?” May 28, 2019.

Copyright © 2020 IDG Communications, Inc.