The open government data movement came into its own with the Obama administration, at a time when governments—and society more broadly—was beginning to truly realize the value of government data, and open-standards were taking root as drivers of innovation. The thrust of this movement was to identify all valuable Government data sets, and to require agencies to make them available to the public, at no cost, and in open-standard formats that ordinary citizens and enterprises could easily access and leverage.
These key principles were enshrined in the Data.gov initiative, established in May 2009, by then-Federal Chief Information Officer (CIO) of the United States, Vivek Kundra. Ten years later, Data.gov still serves to provide public access to high value, machine readable datasets generated by the Executive Branch of the Federal Government, creating the first publicly-available repository for federal, state, local, and tribal government information.
Another milestone for open government data was the Digital Accountability and Transparency Act of 2014 (the DATA Act), which established a legislative mandate that government data be made freely available in standardized, open formats.
Today, there is unanimous agreement about the value of open government data, not only as an enabler of greater transparency and public accountability, but also as a driver of innovative new services, citizen-centered services, and improved decision-making.
Open government data is a driver of innovative new services
However, non-federal government data sets are not covered by the Executive Branch open data policy or by Federal law. They can be included as part of Data.gov only with the permission of the private sector controller of the data. In this way, the rights of private sector providers of proprietary data can be protected, while still allowing the government to use private sector data resources to accomplish its mission.
In a recent white paper, Hybrid Data and the Promise of a Modern Digital Government, Rich Beutel highlights the history of open data policy and the challenges posed by what he calls “hybrid data” or “curated data”—data licensed to or collected by the government that comprises both public and private sources. As Beutel points out, not all data can be open, and access to and use of hybrid data is increasingly critical for government to transform data into actionable information.
The government, citizens and enterprises benefit from these public-private data partnerships. For example, the National Centers for Environmental Information (NCEI), operated by the National Oceanographic and Atmospheric Administration in the U.S. Commerce Department house data and tools that provide timely and relevant climate information to decision makers from various sectors, climate-service partners, and the general public. The NCEIs also integrate and store some non-NOAA climate data that has been made available, together with traditional NOAA data sources, leading to the provision of value-added data products and services. But these arrangements do not result from open data mandates that require companies that partner with the NCEI to make their own data available to the public—this only happens when there is agreement and mutual benefit for private sector entities and NCEI.
Open government data policies must accommodate public-private partnerships where the government collects or licenses private sector data sets and combines them with data produced by the government. In many cases, this private sector data is proprietary and comes with intellectual property or contractual protections, providing for use by the government, but restricting re-use by other parties, including the public.
The government should continue to collaborate with the private sector to have access to the best data sets
As the government takes on the important task of modernizing its own systems and services, they should continue to collaborate with the private sector in order to have access to the best data sets, to spur innovation, to limit duplication of services, and to spur economic growth. Any open data policy which nullifies intellectual property or contractual rights of private sector could prevent these private sector collaborations.
If private companies forgo data-collaboration partnerships with the government because of blanket open government data requirements, the government will be forced to create, develop, and maintain its own datasets, instead of using better, more accurate, and more cost-effective datasets available from companies and private sector entities that have been building and improving these datasets over decades.
The goals of promoting open government data and promoting government utilization of the most accurate and cost-effective data available, are not mutually exclusive. It is not a retreat from open data policies to recognize the value of public-private partnerships that involve public and private data, and to ensure that open data mandates aren’t stated so broadly that they discourage these partnerships.
These lessons apply to efforts to craft new open government data policies, such as the Open Government Data Act (OGDA) that is currently pending in Congress. The OGDA is part of a broader proposal created in 2017 by House Speaker Paul Ryan (R-WI), the Foundations for Evidence-Based Policymaking Act, which seeks to increase transparency and provide information to the public about how government agencies are using their information to enable evidence building and ensure appropriate restrictions for qualified researchers to protect privacy.
The public must continue to reap the benefits of open government data. But to do this effectively, policymakers need to craft sensible policies to ensure the public and private sectors can continue collaborating for the benefit of government and its citizens.