How big data is disrupting the oil and gas industry

The role of CIOs in the success and long term survival of companies in today’s competitive landscape is growing. Technology is the leading differentiator between rival companies, and the next battleground industry could be the biggest one yet.

 The industry to watch in the coming years is oil and gas. While a small sector in terms of employment, the industry is a major IT user. Oil and gas spent $700 million on high-performance computing in 2014, according to the latest available data from IDC. Defense agencies spent $968 million, by comparison.

 Oil and gas “is one of the most IT-dependent industries,”said Chris Niven, research director for IDC Energy Insights. This IT push in America’s shale fields has advanced more rapidly than in any other segment of the energy industry. Shale 2.0 will ultimately yield break-even costs of $5–$20 per barrel — in the same range as Saudi Arabia’s vaunted low-cost fields and, as the Wall Street Journal recently reported, America can perhaps become the world’s biggest oil exporter in the coming years. The confluence of technological maturity and market opportunity is ideally aligned for the upcoming pivot to data-centric Shale 2.0.

Sure, the big guns are already knee deep in the race, like Splunk, Microsoft, and Teradata driving the supermajors. But tech-savvy startups are becoming increasingly valuable to the mid-major oil companies in large because of their agile infrastructure that captures and reads data more quickly.

The startup influence

Oil and gas companies are seeing an influx of these savvy startups disrupting the industry much like they did in manufacturing, financial services, retail, etc. The spry startups mean extremely quick integration with the ability to capture, understand and actionize big data for superior business operations.

Startups are known for bootstrapping and having a hustle mentality to get things done on a small budget. They’re also known for being huge economic drivers for innovation. In oil and gas, the startup mindset will become increasingly important as these companies seek reliable and consistent ways to maximize cash flow by way of big data analysis.

Shale 2.0 will rely heavily on the startup mentality to run efficient operation. Shiva Rajagopalan, founder and CEO of Seven Lakes Technologies, advises that every member of the team be versatile:

“Startups place a large focus on resilient team members from the fruition of their company, but they also need to focus on utility players who can be thrown to pitch a PR article just as quickly as a financial value brief for enterprise level sale.  These players then continue to grow with the company and are groomed to function in specific roles as the team grows.”

The most influential ways big data startups will influence the oil and gas industry are as follows:

 Quick to innovate

A recent PwC report found a clear correlation between innovation and success in growing revenues across industries. In breakthrough innovation and growth, they report that over the past three years, the most innovative 20% in the study grew at a rate 16% higher than the least innovative.

 And in the span of only five years, the top innovators forecast that their rate of growth will further increase to almost double the global average, and over three times higher than the least innovative.

 “By leveraging innovations like smart data analytics, big oil has the potential to save billions,” says Rajagopalan. “Companies will save by having data that’s accessible throughout an organization, not siloed by departments. They can leverage this information to pump by exception and in return, see a reduction in downtime.”

Cloud computing

Aforecast from the UK-based Oil and Gas Council, argues that the oil and gas industry stands to benefit particularly from cloud computing. Using a hybrid version of the technology that combines software as a service (public cloud) and software infrastructure as a service (private cloud) could not only bring IT costs down, but could also improve the way the business is managed.

The hesitation oil and gas companies have in adopting cloud technologies has to do with data security, access, and control. If something was to happen to their goldmine of data, oil and gas companies would have had no way to measure production and run mission critical parts of their business. That means there is a perceived risk associated with moving a multi-billion dollar business over to the cloud for the first time.  A lot has changed for both the oil and gas industry and cloud computing technologies in the recent years. 

“Cloud computing is far more sophisticated, proven and secure than ever before. The growing standard across nearly every global enterprise – from banking to e-commerce, to the government – cloud computing is the competitive edge oil and gas companies need to optimize production,” says Rajagopalan.

Industry-wide digital transformation of silo’d systems

Companies will save by having data that’s accessible throughout an organization, not separated by departments. They can leverage this information to pump by exception, and in return, see a reduction in downtime.

As global demand for oil continues to rise America’s shale fields generate a vast new supply, have more access to better data analytics has never been more crucial to the industry.

“Startups are built on vision, and it’s imperative to keep it steady and a bit nimble. Markets shift rapidly (in our industry, the price of oil can go drastically south), and moving flexibly to shift execution pace became necessary to make our product successful in the long haul,” says Rajagopalan. “With software that improves quality of analytics, it creates confidence within the operation for those on the front lines to make decisions. Being disciplined with big data means you get accounting, production and budget systems all communicating effectively.”