We’re backing improvements to Freedom of Information requests
Making Freedom of Information requests easier for journalists is crucial for greater public interest reporting.
That’s why the Data Team at The Times and Sunday Times is pleased to get behind mySociety as they build a journalist toolkit aimed at improving the handling of FOI requests.
The Times and Sunday Times supported this initiative by providing journalist feedback for their research and hosting the launch event here at the News Building.
Below is a post from Myf Nixon about the event and where the alpha tool is headed.
— -
We’re mySociety, the UK’s longest-standing digital democracy organisation.
You may have heard of some of the sites we run here in the UK, and if you’re an investigative journalist, the chances are that you’re aware of our Freedom of Information website WhatDoTheyKnow.com.
WhatDoTheyKnow makes it very easy for anyone to put in an FOI request; it also publishes both the requests and the responses, creating a huge archive of public information. It’s that last point, though, that means that journalists tend not to use the site: for obvious reasons, making a request in public is not really conducive to keeping a scoop under wraps.
Thanks to funding from the Google Digital News Initiative, we now have the chance to create WhatDoTheyKnow Professional, a toolset based on the software that underpins WhatDoTheyKnow, but with features which we believe will better serve the needs of journalists — and a recent event at the Times Building allowed us to test that belief.
When building a tool for journalists, who better to talk to than journalists themselves? It was incredibly useful to be able to check our research with the people who’d be our potential users, to have an open discussion about features and issues, and to recruit people to take part in testing, once we’ve got to the alpha and beta stages.
We also had a couple of expert presentations: Matt Burgess of Buzzfeed talked about some brilliant use of FOI to expose criminal landlords, and listed FOI officers’ biggest complaints about journalists. Josh Boswell of the Sunday Times was equally insightful as he ran through the ways that he uses FOI when developing stories.
Where we are so far
We’ve started the project with a ‘discovery phase’: our plan was to find out as much as we could about journalists, our prospective users — and particularly just how they go about using FOI in their work. Ultimately, though, we just wanted to know whether journalists really would want, or need, the product as we were imagining it.
The first thing we did was to go and talk to several people at both ends of the FOI process: on the one hand, journalists who make requests, and on the other, the information officers who respond to them.
Since we’re planning on making WhatDoTheyKnow Professional available in other countries, it also made sense to conduct similar interviews outside the UK, and thanks to links with a Czech partner organisation running an FOI site on our software, we were able to do so.
These conversations allowed us to produce a simple process model showing how journalists work when they’re pursuing an investigation using FOI.
The life of an investigative journalist is never simple
After conceiving of a story that requires input from one or more FOI request, every journalist will go through three broad phases: research, request and response, and the final data analysis and writing. The more complicated cases can also involve refused requests and the appeals process.
For a busy working journalist, there are challenges at every step. Each of these adds time and complexity to the process of writing a story, which is an anathema to the normal daily news cycle. FOI-based stories can be slow, and timing unpredictable — Editors do not particularly like being told that you’re working on a story, but can’t say when it will be ready, or how much value it will have.
During the research phase diligent journalists will make a time-consuming trawl through resources like authorities’ own disclosure logs and our own site WhatDoTheyKnow, to see if the data they need has already been released.
Where a ‘round robin’ request is planned, asking for information from multiple authorities — sometimes hundreds — for information, further research is needed to ensure that only relevant bodies are included. In our two-tired council system, where different levels of authority deal with different responsibilities, and not always according to a consistent pattern, that can be a real challenge.
Wording a request also takes some expertise: get that wrong and the authorities will be coming back for clarification, which adds even more time to the process.
Once the request has been made it’s hard to keep on top of correspondence, especially for a large round robin request. Imagine sending a request to every council in the country, as might well be done for a UK-wide story, and then dealing with each body’s acknowledgements, requests for clarifications and refusals.
When the responses are in journalists often find that interpretation is a challenge. Different authorities might store data or measure metrics differently from one another; and pulling out a meaningful story means having the insight to, for example, adjust figures to account for the fact that different authorities are different sizes and cater for differently-dispersed populations.
Sadly, it’s often at this stage that journalists realise that they’ve asked the wrong question to start with, or wish that they’d included an additional dimension to the data they’ve requested.
What journalists need
As we talked through all these difficulties with journalists, we gained a pretty good understanding of their needs. Some of these had been on our list from the start, and others were a surprise, showing the value of this kind of exploration before you sit down to write a single line of code.
Here’s what our final list of the most desirable features looks like:
An embargo We already knew, anecdotally, that journalists tend not to use WhatDoTheyKnow to make requests, because of its public nature. It was slightly sobering to have this confirmed via first person accounts from journalists who had had their stories ‘stolen’… and those who admitted to having appropriated stories themselves! Every journalist we spoke to agreed that any FOI tool for their profession would need to include a way of keeping requests hidden until after publication of their story.
However, this adds a slight dilemma. Using WhatDoTheyKnow Professional and going through the embargo-setting process introduces an extra hurdle into the journalist’s process, when our aim is, of course, to make the FOI procedure quicker and smoother. Can we ensure that everything else is so beneficial that this one additional task is worthwhile for the user?
Talking to journalists, we discovered that almost all are keen to share their data once their story has gone live. Not only does it give concrete corroboration of the piece, but it was felt that an active WhatDoTheyKnow profile, bursting with successful investigations, could add to a journalist’s reputation — a very important consideration in the industry.
Request management tools Any service that could put order into the myriad responses that can quickly descend into chaos would be welcome for journalists who typically have several FOI requests on the go at any one time.
WhatDoTheyKnow Professional’s dashboard interface would allow for a snapshot view of request statuses. Related requests could be bundled together, and there would be the ability to quickly tag and classify new correspondence.
Round-robin tools Rather than send a notification every time a body responds (often with no more than an acknowledgement), the system could hold back, alerting you only when a request appears to need attention, or send you status updates for the entire project at predefined intervals.
Refusal advice Many journalists abandon a request once it’s been refused, whether from a lack of time or a lack of knowledge about the appeals process. WhatDoTheyKnow Professional would be able to offer in-context advice on refusals, helping journalists take the next step.
Insight tools Can we save journalists’ time in the research phase, by giving an easy representation of what sort of information is already available on WhatDoTheyKnow, and by breaking down what kind of information each authority holds? That could help with terminology, too: if a request refers to data in the same language that is used internally within the council, then their understanding of the request and their response is likely to be quicker and easier.
Onwards to Alpha
We’re currently working on the next part of the build — the alpha phase.
In this, we’re building quick, minimally-functional prototypes that will clearly show how WhatDoTheyKnow Professional will work, but without investing time into a fully-refined product. After all, what we discover may mean that we change our plans, and it’s better not to have gone too far down the line at that point.
If you are a journalist and you would like to get involved with testing during this stage and the next — beta — then please do get in touch at alaveteli-professional@mysociety.org
