The movement toward open data in the United States began with the Freedom of Information Act (FOIA), first passed 50 years ago to give the public the right to request records from any federal agency. FOIA requires federal agencies to disclose any information requested unless it falls under one of nine exemptions that protect interests such as personal privacy, national security, and law enforcement. But while FOIA ensures a high degree of government transparency in principle, it does not always do so in practice.
The federal government now receives more than 700,000 FOIA requests a year, which are processed by offices throughout federal agencies. FOIA policies and practices have evolved through court decisions and changes in the law and technology over the years. However, there is no real central coordination or standardization for the agencies’ FOIA operations. Thus, some agencies release data in PDF, Word documents, or other formats that are not helpful for mining the data or analyzing patterns in FOIA requests. Moreover, many agency FOIA offices have trouble keeping up with demand and may have delays of half a year or more in responding to FOIA requests, as a 2015 study showed.
The FOIA Improvement Act, which became law in June 2016, is one of the most significant improvements to FOIA since the original law was passed in 1966. Among other changes, the Act requires setting up a single electronic portal for all FOIA requests and establishes a FOIA Officers Council for interagency coordination.
The Office of Management and Budget (OMB) should build on the FOIA Improvement Act with additional provisions that would rapidly make FOIA a more effective vehicle for opening federal data. To begin, while the FOIA Improvement Act requires agencies to release information electronically, it does not specify how that should be done. OMB should define the “electronic format” required by the Act to refer to a machine-readable format. Specifying release in machine-readable formats will make the information much more useful and bring FOIA in sync with the federal Open Data Policy.
Beyond that, OMB, in consultation with the FOIA Officers’ Council, should ensure the development of a searchable, sortable public database of all FOIA requests—not just the portal for inputting requests that the Act requires. While the law requires agencies to keep records of the requests they receive and how they handle them, they have not been required to publish logs of these requests in an easily usable form. FOIA.gov publishes information on the number of FOIA requests received by each agency, but not the actual requests themselves.
A single unified database would make it possible for anyone to find out what information has been requested and what has been released under FOIA on topics of interest. The database can be developed using the request portal required by the FOIA Improvement Act as a starting point, with participation of the FOIA Officers’ Council to agree on format, data fields, and other details. A possible model is FOIAonline.regulations.gov, which is a website for the public to submit FOIA requests, track their progress, and search information previously made available. FOIA Online currently publishes this information for only 12 agencies and offices, but could help provide a basis for a larger effort.
FOIA should operate on the principle of “release to one, release to all”: If an office releases information to one person or organization that requests it, that information should be made available to everyone. The FOIA Improvement Act requires electronic publication of any information that has been requested at least three times. Rather than waiting for multiple requests, agencies should simply use electronic publication proactively for all the information they release under FOIA, making it available to the public without waiting for multiple requests. To provide a framework for agencies to use, OMB should structure the database of FOIA requests so that the requests in the database link to online versions of the actual information released in response. With this structure, anyone who finds a request of interest in the database would be able to access the information that was released with a single click.
Creating a standardized and automated FOIA database will reduce repeated requests, since anyone considering a FOIA request will first be able to see whether the information he or she is looking for has already been released. The database will also make it possible to search by topic, not by agency, reducing the chance that a request will go to the wrong agency to handle it. These improvements will enable agencies to manage FOIA requests more easily. Agencies will also benefit from the ability to analyze interest in FOIA request data by measuring traffic to different parts of the FOIA database, giving them a better understanding of what datasets are high priority to the public.
Overall, these changes will help to democratize FOIA and the information available through it. A disproportionate number of FOIA requests now come from commercial entities and from hedge funds, which are able to afford the time and expense required to submit large numbers of requests for business-related information. A FOIA database and electronic publication will make this information available to all, not just those with extensive resources.