12 Big Data Trends Changing the Face of Business

Big data is slowly but surely changing the way companies do business. “There are different use cases in every industry,” says Ron Bodkin, CEO and founder of big data consulting firm Think Big. “Big data is unlocking an ability to work with data that has been stressed for a long time. There’s been a lot of pent-up demand.”

Here are 12 trends shaping big data and the future of business.

Updated on October 8, 2013

[ 10 Real-World Big Data Deployments That Will Change Our Lives ]

[ 8 Real-World Big Data Deployments ]

An increasing number of firms, large, small and in-between, believe that falling short in the management and use of data costs them in both revenue and efficiency. According to CompTIA’s Big Data Insights and Opportunities report, organizations say the top five costs associated with shortcomings in managing or using data are:

1. Wasted time that could be spent in other areas of the business 2. Internal confusion over priorities 3. Inefficient or slow decision-making; a lack of agility 4. Inability to effectively assess staff performance 5. Lost sales and reduced margins due to operational inefficiencies

The skills gap is an ever-present consideration, but CompTIA’s survey found companies still place the greatest importance on skills related to managing servers, data centers, storage and information. Analytics skills and knowledge of data-specific tools like Hadoop are in demand, but firms rank them lower. CompTIA expects that to change as more organizations begin exploring their big data options.

While sentiment analysis and crunching click-stream data is still an important thread in the big data world, machine data is taking on increasing importance, says Ron Bodkin, CEO and founder of Think Big, a big data consulting firm. From RFID tags and industrial equipment to jet engines and consumer electronics, the world around is generating ever increasing amounts of data. Companies are beginning to use that data to improve products, drive efficiencies, identify defects and enhance security.

The new cornucopia of public and private data is providing a new opportunity to mash up multiple big data sets to gain new insights beyond what a single big data set allows. “The highest value of big data is coming from combining big data sets together,” says Ron Bodkin, CEO and founder of Think Big, a big data consulting firm. For instance, Land O’ Lakes’ WinField business, a provider of agricultural seed and crop protection, combines big data sets—including weather data, soil moisture data, soil type data, seed data and more—to help its growers achieve maximum yields.

From an open source core, companies are building an array of big data platform technologies, tools and components. “The open source core of big data continues to be the center of the action,” says Ron Bodkin, CEO and founder of Think Big. “It’s fundamentally being driven by the open source model, but the innovation is around the organizations that are taking the open source components and pushing it forward.”

More vendors are providing tools to ease the pain of implementing big data solutions, including General Electric, which is providing tools to help manufacturers harness their data, to Microsoft, which is working closely with Hadoop distribution provider Hortonworks to help business users and analysts access big data sets through Excel. 

Many early big data projects were skunkworks projects intended to prove the value of big data, but that’s changing. “We’ve seen a lot of viral spread of success,” says Ron Bodkin, CEO and founder of Think Big, a big data consulting firm. “But we think there’s a better way than relying on skunkworks innovation. It’s about taking a more proactive approach to identifying where big data can really have an impact. We do think it’s important to have a proven test case, but with executive sponsorship you can get results a lot faster.”

Test-bed projects have dominated the big data industry for the past several years, but these days there are a lot more actual production projects, says Ron Bodkin, CEO and founder of Think Big, a big data consulting firm. For now, he says, these projects are largely about achieving data scalability and cost containment, like building a data lake, but the innovators who got started early are beginning to turn their attention to leveraging their new analytics capabilities for business transformation. “They’re spending a lot less time chasing down data and a lot more time actually looking at data and answering questions,” Bodkin says.

Large enterprises took to big data initiatives in a major way in 2012. Fifty-three percent of 1,217 large companies surveyed in a global study conducted by Tata Consultancy Services (TCS) undertook a big data initiative during the year. They also have a great deal of confidence in their initiatives—43 percent predicted a return on investment greater than 25 percent.

Most companies aren’t spending a lot on their big data initiatives, but some are investing heavily. Tata Consultancy Services (TCS) found that large companies undertaking big data initiatives are spending a median of $10 million. Twenty-five percent of companies spent less than $2.5 million on their initiatives in 2012.

On the other hand, 15 percent of the companies surveyed by TCS spent more than $100 million on big data initiatives in 2012; 7 percent spent more than $500 million. TCS found that companies in the telecommunications, travel, high tech and banking sectors spent the most, while companies in life sciences, retail and energy/resources spent the least.

Perhaps unsurprisingly, business functions that generate and maintain revenue get the most investment when it comes to big data initiatives, according to Tata Consultancy Services. In fact, 55 percent of spending went into four business functions: sales (15.2 percent), marketing (15 percent), customer service (13.3 percent) and R&D/new product development (11.3 percent). Nonrevenue-generating functions get less investment: IT (11.1 percent), finance (7.7 percent) and HR (5 percent).

While revenue-generating functions like sales and marketing are garnering the lion’s share of investments (together they represent 30.2 percent of the big data budget), Tata Consultancy Services (TCS) found functions like logistics and finance (which get just 14.4 percent of the big data investment pie) expect a much greater ROI on their investments.

In fact, when TCS asked the companies it surveyed to rate 75 activities in eight business functions on their potential to benefit from big data initiatives, companies around the world listed as many logistics activities in the top 25 as sales activities.

[ Big Data ROI Will Take Time, Clear Goals and Talent ]

While companies still struggle with technological challenges around big data, companies told Tata Consultancy Services that the biggest hurdle to achieving success in their big data initiatives is getting business units to share information across organizational silos. The technological issue of dealing with the volume, velocity and variety of data still ranks highly, but getting at the data in the first place takes precedence. Companies are also struggling with which data to use to make better business decisions.