Navigating the Rocky Road of Data-Driven Insights


Companies of all shapes and sizes are chasing the holy grail of data-driven transformation, driven by the need to unlock insights to fuel new revenue opportunities, drive efficiencies, and deliver innovative products and services.

In a survey of 2,300 global business and IT leaders by MIT Technology Review Insights, in association with Pure Storage®, 87% of respondents said data is key to new business growth and for delivering better results for customers and clients. Eighty-six percent of those surveyed viewed data as the foundation for making important business decisions.

Yet simply collecting and managing data doesn’t guarantee results. Companies are facing a variety of challenges transforming data into meaningful business value. Sometimes the hardest part is figuring out exactly where to start. “Companies need to determine what is worth the initial time and cost investment for immediate ROI and is that even a realistic expectation in their industry?” says Dominic Halpin (@domhalps ), founder of TechNative.

Data Management 101

From there, it’s all about understanding the company’s data landscape, including what’s available and how to effectively map it to core objectives. “The biggest challenge is knowing where strategic data resides, understanding the data definitions, and developing reusable analytics that answer a variety of business questions,” notes Isaac Sacolick (@nyike), president of StarCIO, author of Driving Digital, and contributing editor at CIO and InfoWorld. “All three challenges are amplified by the growing volume, velocity, and variety of data as well as the need to implement data governance.”

Organizations should start by asking the right questions and figuring out what data is necessary to help develop those insights. “At one end of the process, you ask questions of the data—at the other end, you want results that lead to actionable insights,” says David Geer (@geercom), cybersecurity journalist.

Getting that process right has everything to do with your success. “If you ask the wrong question, you risk a downturn by focusing on something that isn’t necessarily relevant,” explains Jason Wankovsky (, CTO and VP of consulting services at Mindsight.

Framing the data with context is another important part of the mix. “Unfortunately, most databases don't include that context,” says Kevin L. Jackson (@Kevin_Jackson), founder of GovCloud Network. “That makes reconstituting that context and pairing it with current and past data repositories the #1 challenge for today's decision makers,” he says.

One of most important aspects of any kind of data initiative is data quality and data governance. Lack of consistent strategies, having multiple copies of similar data, issues with aging and duplication, and a lack of integration between complementary data types make it difficult to take advantage of advanced analytics, notes Larry Larmeu (@LarryLarmeu), cloud technology leader at Accenture.

“How do you bring it all together in a meaningful and workable way ensuring that your data is in one place and of a useful quality,” says Tim Richardson (@OCSL_UK), Enterprise Architect for OCSL, part of the CANCOM family. “The adage of ‘garbage in--garbage out’ definitely applies here.”

Timing is also critical. If insights aren’t delivered to the right people at the right time, they lose impact. “If insights aren’t quickly directed to the appropriate stakeholders, they might be ignored or become too obsolete to be utilized,” says Phil Siarri (@philsiarri), founder of Nuadox.

The same can be said about building a culture around data. “The insight tools tend to be home grown and operate in siloes--even an ‘out of the box’ solution would be an improvement,” says Hugo Harris (@Hugo_Harris), co-founder of Kraytix. “As such, they need data specialists to draw the insights, and even then, they are not actionable.”

Welcome AI and Machine Learning

Machine learning and artificial intelligence (AI) capabilities, both important to finding the proverbial “needle in a haystack” insights, are being baked into new analytics tools and data management platforms. However, machine learning and AI are new competencies for most IT organizations, which are sorely lacking data science domain expertise, let alone a flush bench of skilled talent.

“Here's the thing: Data is not intelligence--the people doing smart things, finding valid models to identify signals, and then putting in context that transforms context to insights remains the challenge,” says Wayne Anderson (@DigitalSecArch), Enterprise Security Architect at McAfee. 

That talent can be hard to find given the limited universe of skilled experts and the fact that there is booming demand for their services. “Both businesses and public sector agencies acknowledge the unprecedented shortage of data scientists,” says Will Kelly (@willkelly), technical writer and content development manager. “While I expect to see more strategic and technology-savvy enterprises establish in-house training for business users to use low code/no code tools to draw valuable insights from data, many organizations aren’t to that point yet.”

Algorithms, the mathematical models behind calculating and automated reasoning, can also present problems when it comes to accuracy. “For a given application, the first attempt at drawing insights can produce an algorithm that has 60% accuracy, which is considered poor,” explains Brent Kirkpatrick (@BrentKirkpatri3), founder of Intrepid Net Computing. “It can take years of tailoring the algorithms to produce 90% accuracy.”

Data Security Looms Large

Given that data- and analytics-driven insights are a modern-day organization’s crown jewels, it goes without saying that security is a top priority.

The principles around Zero Trust, which assumes all network identity, access points, and traffic can be potentially compromised so they must be regularly verified, should also be applied to data at rest, contends Valentin Bercovici (@valb00), founder and CEO of PencilDATA. “With the epidemic of malware and data breaches, the integrity of my data is now under reasonable doubt,” he says. “My #1 challenge drawing valuable business insights from data is therefore determining the veracity of my data.”

It’s no longer enough to have a Security Information and Even Management (SIEM) system or layer in commercial threat data, deploy a deception system, or prioritize assets--there’s simply no one-size-fits-all security solution. “This is still more art than science,” says Kayne McGladrey (@kaynemcgladrey), a director of security and information technology. “An effective solution needs to incorporate elements of all of those products or solutions to create meaningful and actionable intelligence.”

While organizations are making great strides addressing these and other challenges, the quest for data-driven insights and analytics-led business transformation is still looking like a long and winding road.

For more information on Pure Storage, visit

Copyright © 2019 IDG Communications, Inc.