“Journalism is a conservative industry”: a Russian editor about introducing analytics in the newsroom
by Alisa Ivanitskaya
Imagine you own a store on the main street in your city. Thousands of people walk by it every day. How will you decide if your sales team works effectively? Do you put a counter on your front door to estimate the number of people who enter and base your assessment of the quality of your employees’ work on this number? That’s exactly how newsrooms in Russia evaluated their work in the past (and some still do), says Olga Sidorova, a former chief editor of media projects at Mail.Ru Group and co-creator of media analytics service Mediator. But the number of people who have seen an article doesn’t tell you whether the readers liked it, found it interesting or have even read it.
The problem
Where does the metric of counting views come from? Newsrooms took their approach from marketing. It makes sense to count how many people have seen an ad, usually a colorful image and a short message. However, for more complicated articles editors needed a more sophisticated tool.
The problem that Sidorova’s team was solving is how to give the editors answers in real-time to the following questions:
What do the readers do on the article page? (A typical article page consists of a headline, text of the story, photos, videos, infographics, comments, but also can include teasers of other stories, ads, buttons, and other extraneous elements)
Do people read, or scroll and cherry-pick?
Do readers read until the end or close the article because they find it boring?
Do they read the story better if you adjust the lead or parts where people stop reading?
Do the readers read cutlines and watch videos, or do they study teasers?
Services like Google Analytics, Chartbeat, etc., answer some of these questions, so the idea to research readers’ behavior is not new. Four years ago, Sidorova’s team decided to go further and create a tool localized for a Russian audience that will help journalists in newsrooms to understand their readers.
Prototyping
“It is even awkward to recall but for our prototype product, we used a task-trecker, Jira,” Sidorova says. “Jira is widely used in web development, but it also turned out to be handy to just store the data from multiple counters we put everywhere on our pages and made some operations with it”. Seven days after every publication the Sidorova’s team collected and analyzed the data.
It gave some insight into readers’ behavior but also made clear that such analytics can be useful only in real-time. “The problem with post-analytics is that editors receive a lot of pieces of information that are hard to process and can’t be used without testing hypotheses in real-time. You can talk for an hour for what can be improved but then you leave the meeting and forget, ” she says. Jira didn’t allow the team to collect analytics more often, so they started to build a real product.
Beta-testing
The raw product went through many iterations and tests as Mail.Ru Group has several media projects: news, real estate, health, children, pets and women. So, the newsrooms of every media project contributed to testing and adjusting the product. Then the product was proposed to partners — a limited number of Russian publishers, who tested Mediator and gave their feedback.
Developing a product is a never-ending process, says Sidorova, as you can always find out that some features can be added, visual representation of data improved as well as the accuracy of the counters.
“We were always very limited with our resources, so we had to make hard choices,” she said. “We had an idea of what this product is about that guided us: we want to understand readers, so for us it’s essential to measure the way they engage with content: the time they spend on reading, whether they read the material and where they stop. When the idea came that it is cool to look at the layout of the article, for example, we had to reject it, as it was not essential but took away a lot of resources, so it was easier to put a link to the article then develop a new interface.”
The new tool gave the journalists the capability to receive feedback from their audience and not everyone was prepared for this.
“Media is a very conservative industry. Before the analytics existed, every editor could come and say: I think it’s important for us to cover this topic in this particular way,” Sidorova says. “And then the article received millions of views with the attractive headline because it was on the main page of the outlet, but actually nobody read it.” For many journalists, this feedback came as a surprise. “It was painful for the writers, especially because usually the analytics were introduced by their managers to assess their work and productivity.”
