Behind the scenes: Ophan — how the Guardian democratised data
Three years ago the Guardian’s audience editor Chris Moran stood up in front of a room full of the news organisation’s developers, most of whom he had never met. He explained what his role involved: to find the widest and most relevant audience across the web for the Guardian’s award-winning journalism. And then he told them his problem.
“I can’t do my job.”
The reason was simple. The data tools the Guardian used at that time, in common with most publishers, were complicated, featured a significant lag, cost money per login and only really answered the questions people asked it. Without easy access to up-to-the-second data, Moran couldn’t be certain that the changes he was trying to implement across the newsroom actually worked. And with no data to back him up he couldn’t persuade fellow editorial staff to make those changes either.
One of the developers present was Graham Tackley, the Guardian’s head of architecture. He’d been thinking about the challenges of realtime data but had previously had little contact with editorial staff. Within 24 hours he hacked up a dashboard which, from 10% of the server log files, showed Moran everything that had happened on the Guardian site in the previous three minutes.
The impact was instantaneous. Over the course of the next three years, Ophan was slowly born, the dashboard going through major changes such as being developed into a true realtime analytics system and having logins tied into the Guardian’s Gmail accounts, democratising data at a stroke across the organisation. A host of smaller changes, always tied to genuine need in the newsroom, meant that Ophan developed into a newsroom analytics tool at the cutting edge of technology, tied to the Guardian’s api and hosted in the cloud, which offered a vision of how data could be integrated meaningfully into editorial action, supporting journalists and journalism rather than threatening it. All of this was achieved by a team of ultimately only three developers, in tandem with Moran, all working on it as a second job.
Today Ophan has been used by more than 1,000 employees in the last month at the Guardian to inform coverage, approach and promotion of journalism and is a template for good practice acknowledged around the world. By creating an instant feedback loop it allows journalists and editors to see how readers are reacting to their content in realtime. Most importantly it tells us about digital behaviour and gives the journalists the data to inform decisions — it is a critical point that Ophan means the entire organisation has instant data at its fingertips to make better decisions.
No other newsroom can boast the same level of penetration of data or such a positive culture around analytics. It’s a peerless example of disruptive technology emerging from within a legacy newsroom and going on to change the fabric of the entire organisation.
Ophan’s key points:
Ophan processes around a quarter of a billion data events per day for around 400–500 articles — displaying the data onto Ophan’s dashboard with a lag of less than five seconds. This data can be broken down and viewed instantly by different segments, including: time, section, device, browser, country, referrer, loyalty, if the user is logged in, attention time and age of content.
At a glance editors can see the top 20 performing articles, a summary of the top articles by editorial desk (eg: sport, news, culture), which articles have just been published, which articles are also in print, what’s happened over the last 24 hours and a graph that clearly shows the traffic levels. Ophan was designed to be instant in every sense of the word.
The heartbeat of Ophan is the article page graph. This shows the page views, attention time, social referrers and breakdown of readers by geographical region and device. It also shows us how we have promoted it, where the reader has come from and where they have gone to next. We can also see who the biggest twitter social influencers have been in driving the traffic.
We have demoed Ophan to other journalists, at conferences and to other publishers — and the response is always the same: ‘How can we get Ophan?’ We have been asked countless times if we will license and sell Ophan, and it’s no surprise to see many other news orgs starting to develop their own in-house data analytics tool. To date Ophan has featured in tech and trade press including Venture Beat and many more.
For a walk through Ophan — watch Graham and Chris demoing this at Mind the Product event.
Ophan is a undoubtedly a first. Born in an organisation that champions creativity, data and technology, Ophan allows everyone in the building to use the data to make better decisions and inform the creative process.
Ophan has been breaking ground since it was first conceived at the hackday. A product of necessity, it has evolved with input from the consumers it is built to help — the journalists who make the content. It was designed with the journalists and journalism at the heart, meaning it is the easiest and most-user friendly data analytics tool there is.
Each and every person in the building can access the data. The consumers are the journalists. It has helped us to understand and prove that digital content needs to be promoted — no one is going to magically find a URL. We have found that content that does not perform well generally has one thing in common — we have not promoted it enough. With Ophan we can see this in an instant and take action — we can tweak the headlines, put it on Facebook and see the immediate effect. This capability ensures that we give every piece of journalism the chance it deserves.
Three years ago the Guardian was reporting record traffic of 71.8 million monthly unique browsers (digital abc). One year ago we recorded almost 110 million. Traffic today is nearer to 150 million monthly unique browsers. And whilst many factors have gone into this over the years, including a new global domain, opening US and Aus offices and continuous world-class and agenda-setting journalism, there is no doubt that Ophan has played its role in the Guardian’s impressive surge forwards. The last year has seen a turbo boost, with more Ophan features rolled out and more people using it than ever before.
It is a critical tool in the newsroom and beyond. It allows everyone one in the building to be data informed and use data meaningfully to make better decisions. Ophan enables the Guardian to have one of the most bespoke, advanced and democratised approach to data of any news organisation in the world.
