Big data projects on the rise (but data use could be better)

The good news is that about half of U.S. companies say they now have some form of big data initiative in place, according to a recent study. The bad news:  Few have  managed to reach their data-related goals.

“The amount of data crossing the wires and airwaves is mind-boggling,” says Seth Robinson, senior director, technology analysis, with the nonprofit Computing Technology Industry Association (CompTIA) trade association. “So while individual pieces of a holistic data solution may be improving, these individual pieces are not yet integrated in a way that drives ideal results.”

CompTIA surveyed 402 U.S. business and IT professionals online in September and October 2015 to support its Big Data Insights and Opportunities report. It found that 51 percent of respondents reported having a big data project in place today (up from 42 percent in 2013) and another 36 percent reported they were in the project planning stage.

[ Related: Hiring a data scientist? You’re doing it all wrong ]

While Robinson conceded there may some “big data washing” going on in that number, he says the net result is companies are moving forward with data initiatives and seeing results: A net 72 percent of companies said their big data projects have exceeded expectations.

“Some companies may be calling their data management ‘big data’ regardless of the size of the data,” Robinson says. “It’s becoming the catch-all phrase. But if companies are taking steps with data management, then they’re doing the right thing.”

“A really high number of companies said they were seeing benefit out of it,” he adds. “You can really get a sense that utilizing data correctly in today’s business environment is something that pays dividends.”

Are we there yet?

A significant number of respondents, 31 percent, said they are exactly where they want to be in managing and using data (up from 18 percent in 2013), but the sentiments varied depending on the respondent’s position in the company hierarchy. Only six percent of business unit workers felt their organization was where it needed to be with regard to data usage, compared with 42 percent of executives and 31 percent of IT employees.

[ Related: 6 analytics trends that will shape business in 2016 ]

Robinson’s takeaway is that the business line need for actionable data insights is not being met, even if it’s improving in other areas of data management.

In all, 75 percent of companies agreed their business could be stronger if they could harness all their data — a percentage that has remained consistent across multiple years of this study.

Further, 75 percent of companies said they should be more aware of data privacy issues, and 73 percent felt they need better real-time analysis. While the holistic data solution may be improving, these pieces have not yet been integrated to drive the ideal results, Robinson says.

“While some businesses may have made progress in select areas of data management, many have not fully connected the dots between developing and implementing a data strategy in order to have a positive effect on other business objectives, such as improving staff productivity or developing more effective ways to engage with customers,” the report says. “For IT solution providers or vendors working in the big data space, this should serve as an important reminder to connect data-related solutions to business objectives, emphasizing outputs over the nuts and bolts inputs.”

Respondents cited a number of factors for the increased importance of data:

• 63 percent rely on data for day-to-day operations
• 61 percent cited sensitivity around data privacy
• 60 percent use data to better understand customers
• 59 percent rely on data to measure business objectives
• 56 percent say they store data outside the company

Robinson says that if companies wish to improve their data usage, the first question they must answer is which behaviors they hope to improve through enhanced data management and analysis. Making better/faster decisions and reducing costs/overhead topped the list of strategic objectives for 48 percent of respondents each. They were followed closely by improving workflows/communications (43 percent), improving business operations (42 percent) and reaching new customers (34 percent).

Decisions, decisions, decisions

“It is no surprise to see a desire for better decision making at the top of the list,” the report said. “The big data trend saw a rapid ramp in hype due to two factors. First, it followed the momentum of cloud computing, where companies without much infrastructure started with a cautious approach but quickly jumped on board as they realized that cloud could more easily help them expand. Second, the discussion around big data always centered around decision making, a topic with more universal appeal than infrastructure.”

The report also argues that the focus on cost reduction reflects an oversimplification of the technology’s potential. The ultimate goal, it says, is business transformation, whether that takes the form of improved customer relations, new business offerings or innovative thinking. And all of those changes imply value beyond simple cost savings.

For those organizations aiming to amp up their data usage, Robinson recommends taking measured steps at each of the three stages of data usage: collection and storage, processing and organization and analysis and visualization.

He notes that many organizations have indicated a willingness to work with third-parties for help with their data initiatives. More than one-third of companies currently work with an IT firm for their data needs. However, those engagements then to be somewhat “simplistic” — data storage and data backup, for example. He suggests that as companies become more aggressive with their data initiatives, IT channel firms may find opportunities to offer comprehensive end-to-end services around data.

“The role of IT is changing and becoming one where the CIO might not tell every department, ‘here’s how you should do your data,’ but the CIO is orchestrating the whole thing,” Robinson says. “Data and security initiatives need to be managed from a central location.”

“The focus on data and management of data is going to grow out of the domain of whatever database administrators might have existed within the IT function,” he adds. “It’s going to be its own mini-function within the overall IT department.”