Federal Government Wrestles With Big Data and Sequestration

Despite the challenges of the budget sequestration that went into effect on March 1, federal agencies are pressing forward with big data initiatives, hoping to squeeze big savings out of more efficient use of their data.

In fact, based on the federal government’s FY12 budget actual expenditures of $3.538 trillion, federal IT managers could potentially recognize nearly $500 billion in savings across the federal government via big data initiatives, according to a new study by MeriTalk. MeriTalk is a community network for government IT developed as a partnership by the Federal Business Council, Federal Employee Defense Services, Federal Managers Association, GovLoop, National Treasury Employees Union, USO and WTOP/WFED radio.

[Related: To Tap Big Data, Federal IT Must Partner With Tech Industry]

MeriTalk surveyed 150 federal IT executives for the report, Smarter Uncle Sam: The Big Data Forecast. Forty-eight percent of the respondents were from the U.S. Department of Defense. The remaining 52 percent were from civilian agencies.

Big Data Casualties of Sequestration

Sequestration has posed challenges. Forty-one percent of respondents to MeriTalk’s survey said their agencies’ big data budget has been cut by 10 percent or more as a result. Federal IT executives identified the following sequestration casualties:

• Training and workforce development: 51 percent.
• Hardware upgrades: 48 percent.
• Software upgrades: 41 percent.
• New application development: 40 percent.

That said, the effect may not be entirely bad, says Rich Campbell, chief technologist of EMC Federal at EMC, which underwrote MeriTalk’s report.

[Related: Federal Government’s Big Data Efforts Lagging]

“As far as the sequestration goes, probably 50 percent of projects have been affected,” he says. “Of that 50 percent, probably 25 percent to 30 percent have experienced significant impact. Sequestration has definitely had an effect, but from my perspective it’s been somewhat beneficial.”

The upside, Campbell explains, is that agencies have been forced to more rigorously define their goals for initiatives and find as much efficiency as possible.

“Big data is transforming government,” he says. “Each agency needs to first identify how big data can support their mission objectives, then assess the infrastructure, the savings opportunity and start with a pilot project. There is enormous opportunity ahead for government to apply big and fast data to manage data growth, gain new insights from data and innovate in ways that weren’t possible before due to technology limitations. It will enable agencies to be more productive, work smarter and be more agile—to keep up with the pace of change.”

Federal IT Execs Push Forward with Big Data Initiatives

MeriTalk found that nearly a quarter of federal IT executives say they have launched at least one big data initiative, with goals ranging from processing scientific data to records management/eDiscovery, to radio signal monitoring data interpretation, fraud detection, utilizing transaction data to determine the best locations and methods for consolidating data centers and joint operations planning. Federal IT executives believe the top federal functions that will win with big data include:

• Executing military/intelligence, surveillance and reconnaissance (ISR) missions: 54 percent.
• Combating fraud/waste/abuse: 48 percent.
• Managing the transportation infrastructure: 27 percent.

[Related: Big Data Problem Plagues Government Agencies]

In most cases, the initiatives are still in the early stages. Thirty-one percent of the federal IT executives MeriTalk surveyed have invested in IT systems or solutions to improve data capture. Thirty percent said they have identified specific challenges they can use big data to solve, and 38 percent have invested in IT systems of solutions to optimize data processing.

In another study conducted by MeriTalk last year, government IT workers expressed serious misgivings about their agencies’ ability to store big data. This year’s study found that 43 percent of federal IT executives have invested in IT infrastructure to optimize data storage and 44 percent have improved the security of stored data.

Most Execs Feel Their Agency Lacks Sufficient Big Data Strategy

Despite this work, MeriTalk found that only 31 percent of respondents believe their agency has a sufficient big data strategy. They believe their agencies need to significantly increase their data management efforts. Whereas today they successfully tag 26 percent of data and analyze 23 percent, these executives believe that ideally agents should be tagging 46 percent of agency data and analyzing 45 percent. They say their agencies need to invest dollars in four key areas:

• Increasing server storage capacity to house and analyze big data.
• Developing a cloud-like infrastructure to support big data tool comparisons.
• Determining bandwidth needs for big data storage and analysis.
• Deploying advanced data mining practices

If agencies can make inroads in these areas, 70 percent of federal IT executives say they believe that in five years big data will be critical to fulfilling federal mission objectives, including the following:

• Improving processes and efficiency: 51 percent.
• Enhancing security: 44 percent.
• Helping their agency predict trends: 31 percent.

Unavoidably, it comes with a price tag. Federal IT executives also predict they’ll spend about 16 percent of their IT budget (nearly $13 billion) on big data in five years.

“Big data’s different from other IT initiatives—because it’s not an IT initiative,” says Steve O’Keefe, founder of MeriTalk. “If assuming the same behavior and expecting a different outcome is the definition of insanity, big data may provide some common-sense therapy we need to make better decisions in government.”

Thor Olavsrud covers IT Security, Big Data, Open Source, Microsoft Tools and Servers for CIO.com. Follow Thor on Twitter @ThorOlavsrud. Follow everything from CIO.com on Twitter @CIOonline, Facebook, Google + and LinkedIn. Email Thor at tolavsrud@cio.com