The metrics that matter

‘Working in media organisations is kind of like playing Where’s Waldo’, said Esra Dogramaci, Senior Editor, Digital at DW, during her session ‘How to build digital strategy’ at the International Journalism Festival. According to Dogramaci, the media landscape is so crowded that organisations find it hard to distinguish themselves. They look at their competitors and copy them, meaning everyone suddenly pivots to video, VR, and Facebook live all at once.

Where’s Waldo?

How can media organisations rid themselves of their red-and-white-striped shirts, bobble hats, and glasses and change into something more, well, noticeable?

A big part of the solution is getting to grips with analytics. This means understanding the numbers to understand the audience, leading to more informed decisions about the kind of content that’s produced, and in turn, driving engagement. We caught up with Dogramaci to get some advice on analytics: she told us why she isn’t too worried about Facebook algorithm changes, that vanity metrics work for marketing but not for news, and about the time DW broke records when they teamed up with Twitter.

Insights into analytics from Esra

(In Dogramaci’s words, adapted from an exchange between Dogramaci and the Global Editors Network — edited for clarity and brevity.)

Esra Dogramaci at the International Journalism Festival

Pay attention to dwell time and retention rates

There are a number of great tools available to listen to your audience. There’s Spike by Newswhip, Parse.ly, and Facebook’s Crowdtangle. In addition, all key social platforms — YouTube, Twitter, and Facebook — have their own analytics platforms available (for free).

What I do with these tools is look for patterns: Do we see one month where traffic spiked? If we dig deeper, what was behind that spike? Was it a breaking news story or something else? What format was it — picture, video, text, or an interactive?

I’ll look at long term patterns to see what audiences like and don’t like. The way to tell is by looking at metrics, such as engagement or retention rates.

Things like views, reach, clicks and impressions may look impressive on aggregate but are very superficial. They aren’t actionable metrics — meaning we can’t really use them to feed into editorial or content strategy. Things to pay attention to are dwell time, retention rate and watch time. Look at how your content is consumed and shared.

I remember at the BBC, when YouTube still had annotations on videos (chapters you could overlay), we had a video of Angelina Jolie divided into sections. One was about her latest movie, one was about her then-relationship with Brad Pitt and her family, another was about her double mastectomy, and so on. We were then able to see where in the video viewers were clicking; what they were most interested in. We discovered it was the double mastectomy. The audience wasn’t coming to the video for her being a celebrity, per se, but they were interested in a celebrity having an issue that they could relate to. Running tests like this over and over again reveals patterns of what works and what doesn’t work for your audience.

What we often find is that it is the human interest stories — less about celebrity and more about celebrities handling the same challenges we all face.

Don’t overproduce

  • Look at what’s working and what’s not working;
  • Look at what times your audience is active and inactive;
  • Look at what times of day and what days of the week work best for you.

I was looking at some of the UN Twitter accounts for instance — they generally produce good content, but are publishing so frequently that the audience disengages. If they can decrease quantitatively what they’re doing, they can use that extra time and resource to increase the quality or they can start to invest in other forward looking digital projects. If you’re working in digital, you should always be spending time looking at the next big things. Innovation, as well as the audience, has to be at the heart of everything you do.

Grow your female audience

I started building in gender metrics for the BBC back in 2013 when working on YouTube. I noticed that the female audience across the board was significantly underrepresented, so we made them an unofficial target. By making small changes, we started to see an increase in the female audience.

We started basically by making sure more females were represented in the thumbnails of videos. BBC Azeri did this and in a matter of weeks started to see a shift towards more female viewers. BBC Vietnamese went one step further: Their most successful news product on YouTube was a weekly hangout which broke many records for them. It lifted BBC Vietnamese to number 4 out of 20 BBC language services on YouTube — a big feat considering they didn’t have 24 hour TV to support them with content like Arabic or Persian did. For BBC Vietnamese it wasn’t enough to simply have a female host, but they made sure they had females on their hangout as well, which naturally translated into a shift towards a female audience.

Every broadcaster is interested in the reach and the size of their audience. At the BBC, we went one step further with responsible reach by tracking quarterly what the gender split looked like for each language service, how each service was performing against each other, and then what our gender split looked like overall.

What works in some countries doesn’t work in others

During my work with YouTube at the BBC we discovered that the number one video the Vietnamese audiences were coming for was a weekly Google Hangout on YouTube. This ran for at least 45 minutes and regularly brought in over a million views.

For Turkish audiences, we discovered they enjoyed a weekly cartoon or satire, coverage of big Turkish news events, and also what we could call ‘interesting’ news stories: A video of a whale about to explode on a beach was one of their top videos for months on end!

Spanish audiences enjoyed science and particularly explainers of how things worked, such as what happens to you physiologically when you fall in love. BBC Azeri and BBC Spanish found creative ways to cover events they didn’t have access to: Spanish did a series of how to behave at Copa America (football) when they didn’t have match or footage access. All of this comes about through knowing your audience and that begins with looking at your numbers.

It’s not about copying, but inspiring

I never look at other organisations to copy, but I do look at them for inspiration. I really think print publications are sometimes much further ahead than broadcasters when it comes to digital. I love the digital experience the New York Times gives its audiences on Facebook — particularly its very well produced videos. Look at this video of Simone Biles. The results are phenomenal: it had over 500,000 shares and 57 million views. It broke all the rules of a typical Facebook video — it’s horizontal, works best with sound, and is far longer than the typical bite size 90 second experience.

The Guardian and Financial Times also come to mind — the latter particularly with their interactives, such as the Uber game.

You get a sense that these established papers are working really hard not just to be relevant in the digital space, but coming up with products to bring in a new audience. That’s really smart. The FT also releases its source code on projects upon completion. This means any newsroom could theoretically duplicate it for their local context. It’s not about copying, but inspiring. I’d like to see more of that in the industry.

Don’t fret about algorithm changes

Facebook changes its algorithm every six months or so but the latest one was so specific because it came off the back of News Feed test changes in Cambodia for instance. It was directly affecting News Feed, so there was a lot of concern — even panic. However, I contend there’s really nothing to worry about because the thing for broadcasters and others to do is to focus on their audience. If you are delivering good content and getting good engagement, you won’t be affected by any algorithm change. I’ve written in further detail about that here.

However, I was in Indonesia recently where I met a Cambodian vlogger who had seen a significant drop in traffic because of the Facebook algorithm change. The only explanation she had of this was that Cambodia’s Facebook population was so small that Facebook could afford to ‘test’ there.

This does have consequences. This particular vlogger was outspoken about issues not discussed in mainstream media and had amassed quite a following. In places where you have challenges or restrictions to free speech, independent journalism and voices are particularly important. She also mentioned having yet to obtain an explanation from Facebook either personally or publicly as to why the change happened. Her content fit within Facebook’s rules, yet she’s not been able to take back the success she previously had. Other independents like Nuseir who is behind Nas daily, have seen their numbers go up.

Don’t hate on numbers

‘Less than half of newsrooms consult analytics daily,’ according to an ICFJ Survey, ‘The State of Technology in Global Newsrooms’.

When working in analytics, you can’t have the mindset of a consultant who analyses data, comes up with a prescription, and then leaves. I really think it’s about sitting down with teams, understanding how they work — what are they good at, what efficiencies can be identified, and how can you then slowly bring data into the process. There are a few crucials though:

  • You absolutely must have someone from senior management who believes in this and supports you. Without leadership and projects to change the culture, even just getting into the mindset of paying closer attention to analytics will be a struggle.
  • Second, you’ve got to have buy in. If you’re advocating to teams or people who are absolutely, vehemently resistant, then expect no success. Instead, look for those who are curious and willing to accompany you on the analytics journey. Also seek out people who are still on the fence — they can be convinced by numbers.
  • There are far too many newsrooms who value ‘vanity metrics’ — things like clicks, reach, views, impressions. These numbers are indicative but you can’t do anything with them editorially or for building content strategy. Unfortunately I’ve seen far too many CMS/analytics systems in newsrooms that are either made for marketing or just not useful to news. What then ends up happening is that people are educated incorrectly about the numbers. When someone like me comes along and points this out, there’s a lot of resistance. My numbers are lower, but in the long run more meaningful (interaction, retention rates/dwell times, uniques and so on).

Take for instance the Guardian and other papers — shifting from an advertiser revenue model to subscriber driven model. That is all premised off the back of engagement — of having a relationship with the reader, knowing who they are, what resonates, what doesn’t, and delivering on all of that.

Example of a successful partnership with platform

Twitter’s former News Partner manager, Rob Owers, approached me in 2017, suggesting that we cover the German election night on Twitter. A lot of the credit goes to Rob, because he was an excellent partner manager who made things happen and metaphorically held my hand through the process — especially when it came to technical requirements.

For DW it was an opportunity that couldn’t be missed. I was lucky to have DW’s head of distribution and technical Guido Baumhauer’s immediate support. He saw the possibility of putting DW on the map in a big way.

Partnering with Twitter meant that whoever visited Twitter on German election night, whether logged in or out of Twitter, would see the DW News livestream. Seeing as the feed was with Periscope — a product that was still being developed at the time — I had limited feedback on specific demographics. But we had over 609,000 viewers — not views, but viewers — watching for an average of 10 minutes or more. If you’re familiar with Twitter, you’ll know that never happens.Twitter is a scrolling experience and videos are much shorter — typically no more than two or three minutes. Holding on to an audience for that long on a platform that really isn’t thought about as a video platform, literally broke records. Last year Facebook live was all the rage and YouTube already has its place in livestream and as the leader in video. But the experience with Twitter really challenged that and got me thinking that this was an opportunity that publishers maybe overlook and Twitter is keen to invest more into.

Thanks Esra!


Esra Dogramaci is Senior Editor, Digital at DW. Prior to this she was a digital consultant at the BBC and a New Media Analyst at Al Jazeera