Improving User Experience for Public Records Requests

Ruminations and meditations on public records and open data

Going into this project, I knew absolutely nothing about public records. In fact, I had never even seen a public records request portal. Here’s an image of one, in case you’re anything like me.

As an intern with the City of Seattle this summer, I am pursuing a two-track project, concerned with open data and preemptive disclosure of data. Specifically, I’m performing an analysis of public records request data, mining the raw text for insight using some machine learning techniques.

In parallel, my colleague Leslie Denning and I are working together to try to foreground best practices when sharing data across government institutions. Part of completing that task entails understanding how public records requests work.

What have I learned so far?

1. Public records are adversarial. Requesters treat public records as a last-ditch effort to get the information they want. Sometimes what they seek has not yet been made available, and other times folks were not consulting the right resources to dredge up what they were looking for. Internally, public records officers are held liable legally if they are unable to fulfill a request within a particular time period, or if they do not meet the standards of a “good faith effort” when sharing information with the public. Occasionally people capitalize on this by serially suing institutions that don’t provide the records they want.
2. Public records are largely unstructured and are open to interpretation. People ask for all kinds of things in a public records request — they’re an incredibly varied body of knowledge about consumer preferences and needs. On most platforms, people submit records requests as a huge body of text. As such, there is frequent and detailed dialogue between requesters and records officers to clarify the intent of a request. Each government institution collects different metadata about public records, and there are not clear best practices or standards as to what can (or should) be collected.
3. Government bodies share data formally (not through ad hocmethods) and don’t tend to share public records across jurisdictions. They produce agreements with other institutions to guarantee data will be maintained in a particular form, over a particular span of time. If there is an open data portal for an institution, a different government body typically signs a contract to get access to a more robust picture of the data living on a particular repository.

Why are these three points important? Institution fulfilling information needs should not have to be on guard against abuse. If they are, they have a split personality — expected to both maximize and minimize the information they put out there for external stakeholders. This tension makes it harder to share information, complicates analysis, and makes it near-impossible to aggregate data across jurisdictions. As a result, internal data analysts (like myself) really have their work cut out for them.

How can we address this problem?

I’ve outlined some deep and onerous issues above, ones which I certainly won’t solve in a brief blog post. But I would propose the following as a means of addressing some of the issues:

1. Pester your requesters. Maybe this is a misnomer, or there’s a nicer way to say it. Governments can’t force people to provide demographic information about themselves, but they can ask requesters nicely to submit more information. See the screenshot below for one possible way to nudge requesters.
2. Fill forms with fresh fields. As you saw above, a requester picks the department and then basically gets a text box to fill in. Providing more structure to a request (perhaps by shifting some of the more frequent requests into a controlled vocabulary) could be a way to gather more information about requests, providing more structure.
3. A/B test! Some institutions get a few thousand requests a year (I’m looking at you, SPD…), which means their records request portals are frequently used. It could be a good idea to serve a series of different request pages, testing which page layouts or form combinations are the most viable to records requesters to share their information.

An example dialogue box, showing how goverrnment offices could prod users into giving more information.

I believe using some of these techniques could simultaneously improve the external request process, and the internal data analysis process. Next time, I’ll dig into the data itself, talking about the analysis process and some of the things I discovered. Stay tuned!