Changing the way we approach editorial analytics: an interview

Like so many useful tools and services, Content Insights grew out of a sense that something was missing. In this interview, company founder, Dejan Nikolic, discusses how he grew to realise that publishers and content marketers were missing the point when it came to analytics — that single metric measurements weren’t strong enough, especially when dealing with content that could have a serious effect on how both the subject and the writer were perceived.

Like so many useful tools and services, Content Insights grew out of a sense that something was missing. In this interview, company founder, Dejan Nikolic, discusses how he grew to realise that publishers and content marketers were missing the point when it came to analytics — that single metric measurements weren’t strong enough, especially when dealing with content that could have a serious effect on how both the subject and the writer were perceived.

Dejan’s aim for Content Insights is to help bring about a fundamental change in the way that we look at editorial analytics, and subsequently alter the way we approach digital publishing. A lofty ambition, to say the least, but such changes have to begin somewhere. In this case, it all began with a fictional drunk man and an equally fictional shark.

Before we delve into Content Insights itself, tell us a bit about your background. I understand you cut your teeth on a satirical news website?

I was actually on sick leave from work and it was three months of lying down so I started writing very, very bad articles about what I didn’t like about politics and I tried to twist it like The Onion would, or Jon Stewart would. I didn’t think it was going to catch on because the style of satire in our parts [Serbia] is quite different: it’s more metaphoric, it’s less direct. The website, Njuz.net, is written in a journalistic style — a very serious style — with absurd content, and that’s very confusing. The further you go East, the more people would consider this kind of thing a lie instead of being satirical or funny, so I didn’t think it was going to succeed. But I was wrong, luckily.

And what about the name? Is it an English pronunciation?

Yes, it’s a totally English pronunciation. The trick is that we gave it a Serbian spelling, so it’s obviously a trick, right? It’s about news, but there’s something wrong with it! It took us a while to find the name, but we wanted something that you needed a bit of a higher IQ to figure out.

And I understand we have you to thank for one of the biggest internet hoaxes of the last decade?

Yes, well our goal is not to hoax people, least of all the media, and back then I didn’t even think it was possible, but then we did this story. It was a story about a drunk Serbian tourist. He’s walking along the coast in Sharm-el-Sheik in Egypt with his friend, and he suddenly says, ‘here, hold my beer and watch this’. He jumps off the cliff, lands on a shark (which has terrorised Sharm-El-Sheik for the last week and a half), kills it on the spot, wakes up in the hospital — not because of the encounter with the shark, but because of alcohol poisoning — and saves the Egyptian tourist season.

‘If a news agency is the source of the story, then it must be true… right?’

Now, it would have been easy for me not to run the story — it’s fun, but what’s satirical about it? But we thought, in small nations you always tend to exaggerate when someone from your country does something abroad, so we thought maybe this could be a critique of that kind of media attention. So we ran it, and initially nothing really happened, but then the Macedonian News Agency translated it into English and changed some details in the story. They gave their own journalist the byline and never mentioned the source, so basically njuz.net ceased being involved from that moment forward.

What force it would take for a Serbian tourist to kill a shark if he weighed 80kg?

“A Russian news agency called in some physicists to calculate what force it would take for a Serbian tourist to kill a shark if he weighed 80kg”

Various blogs picked it up because it got some kind of credibility coming from a news agency. So before long it was on the usual places, like Reddit — places where people find stories, as someone from the New York Post did. She ran the story, again without citing the course, except for the one from the Macedonian News Agency (and if a news agency is the source of the story, it must be true, right?) When the New York Post ran it, the credibility went up one notch more, so then it was all over the world, East to West. I mean all the mainstream news media picked it up. A Russian news agency, one of the most serious news agencies in the world, called in some physicists to calculate what force it would take for a Serbian tourist to kill a shark if he weighed 80kg, and then they made graphics to go with it and it became an even bigger hit. It spread around very quickly. We were very entertained and our readers were watching it, and they were sending us links from every country the story popped up in, so we knew exactly where it went.

They say news travels fast, but that seems particularly quick. How did it unravel?

It took ten days for people to see through the story, and it was my fault because I used a picture of a basking shark, which is the only shark that isn’t a meat eater, and somebody noticed. So the same news outlets ran a whole other cycle of stories about that story being fake. They started writing about us as well because they found that we were the source of the story.

‘Everyone just wanted to hear about a black racist. It was ridiculous, and I wanted to find out why this happened’

At the time we wondered whether you would even bother to check a story like this. Why would anybody invent it? It’s just so absurd. We thought maybe that was it: it was just a fun piece, but then we saw other stories that had a similar effect, but with more direct consequences for the people involved. These weren’t feel-good stories: these were stories that actually ruined lives and careers.

Shirley Sherrod is a prime example: she was an African-American who worked for American government and was accused of being a racist because of something she was alleged to have said during the NAACP Congress. The media picked up the story, only to find that the first blogger who first ran it actually edited her speech in a way that makes it look like she was racist. It wasn’t only the conservative media that picked up the story, it was the Huffington Post, it was CBS, it was CNN. Nobody checked anything — they just ran it. It was clickbait. Everyone just wanted to hear about a black racist. It was ridiculous, and I wanted to find out why this happened.

‘It’s about incentives. You’re paid according to the number of pageviews you get’

After talking to hundreds of journalists, I came to the conclusion that it’s about incentives. You’re motivated to do it because you’re paid according to the number of pageviews you get. Staff writers are overworked: they have to dish out ten to fifteen stories a day, they have 20 or 30 minutes to prepare, so they have to source it, check it, write it, check the grammar and post it online, and you have to ditch at least three of those five steps if you want to make it. They ditched checking the source. Why? Because it already had some credibility online because some other people ran it before they did.

And presumably, the faster you run the story, the more clicks you get?

Exactly! As simple as it sounds, one of the reasons that digital journalism and storytelling is suffering is because the incentives are wrong. When you have a clear incentive, a clear currency that you have to chase (like pageviews or unique visitors, or any other single metric), that’s where your attention will go. If your payment and your status is attached to the number of pageviews, you go from being a writer to a page chaser. So now your job isn’t to inform people, it is to get more pageviews, and you do this in any way that is available to you.

So how did you manage the incentives for your writers?

Well, I didn’t want to pay them a flat fee because they were all at different levels, and I didn’t want to pay them in pageviews because I saw what that does to people and to stories. I started to look around at what other people were doing. Most news organisations tried some combination of those single metrics, but we knew that it didn’t work because they tried something different every year! We knew that compounded metrics (add metrics to each other in a formula, like page views + shares = how awesome is your story) are a no-no, so we began researching what is important in analytics, looking for a way to measure performance more holistically.

What this means is that you want to look at the ratio between metrics. For example, if I told you that a website has 100,000 page views would you think it’s success?

Well, I’d say that it sounds like a good number, although I suppose it depends on the size of the website…

But, yeah, there’s another thing isn’t there? What if they’re publishing a thousand stories every day?

Well, then no!

So that’s the ratio. You need the ratio between the two. So now we’re looking at more than 100 of those ratios to calculate what we call a Content Performance Indicator (CPI), and that’s what powers Content Insights. It didn’t look like that at the beginning. Initially it was a simple formula that didn’t work, but I paid my writers accordingly anyway. It took about two years to get it right and another year to hone it to the level where we’re actually sure it can be an engine. So now it’s an algorithm that can calculate a lot of stuff. It’s very different to when you look at one metric as a currency. We don’t want to miss any signs the reader is doing something with the story: how he came to the story, why he came to the story, what sort of engagement he had with the story.

‘With CPI, writers who are writing about serious issues can actually compete against writers who are writing about Miley Cyrus, or kitten videos on YouTube’

When you can evaluate how one particular story performs, you can attribute it to a particular writer or content topic and you can build an analytics tool around how to make sense of it all. With CPI, writers who are writing about serious issues can actually compete against writers who are writing about Miley Cyrus, or kitten videos on YouTube. So that’s how Content Insights came about: it came out of my necessity to pay my writers in a fair way, and it has turned out to be a tool that allows old-school editing and old school ethics to work in a digital environment where everything is measurable and everything is science and not so much art.

How could this way of measuring success help writers and journalists (particularly freelancers) who might be trying to sell their skills into an organisation? It’s a hard market to break into, isn’t it?

Well, yes. Content Insights is made to be as useful for freelancers like those we had working at njuz.net as those who work in-house. Writers that really know the subject they write about will really benefit a lot from it because it is as a very fair and objective measurement system. Why? Because CPI picks up on signals that show how writers sync with their audience, if they write regularly about the subjects they are good at. In a data-driven marketplace, freelancers will be able to get hired in a much quicker and less subjective way.

Take the riots that are happening in Montenegro. Say you need to cover it for the Guardian but you don’t know anyone there. With Content Insights, an editor can find people who are writing about that event successfully (in terms of people being engaged and the article being cited and referenced on other websites), which will make it score very highly on the CPI scale. You know that they’re doing it in English, whether on their blog or for other news organisations, so you can contact them directly. You can do it in minutes though a search, without the middle men.

So, in effect you’re creating a kind of marketplace too?

Yes — that’s the thrill of the data-driven marketplace. Right now, news organisations are reluctant to open the analytics audience to journalists because they’re scared they’ll become pageview hunters or clickbaiters. Using the CPI for the reasons that we have discussed, with Content Insights, there’s no fear of democratizing analytics for everyone. Furthermore, we designed it in a way that even I can understand! It’s clear enough for a six year old to understand!

‘News organisations are reluctant to open the analytics audience to journalists because they’re scared they’ll become pageview hunters or clickbaiters’

As for how they can help freelancers get hired, well you have proof. Imagine the conversation: ‘Why should I hire you to write about the next Formula One race?’ ‘Well, I wrote about the previous one and look at my numbers. And actually I have better ones here because I use CPI too!’

How do you think journalists will deal with being measured?

I know journalists are afraid of being measured, but that is inevitable. People will measure you and pay you according to how well you’re doing — all the trends suggest that it’s going that way.

There are two ways to deal with this: the first is to use metrics, which is the trend we have now, and it’s going to be pageviews or attention minutes that evaluate you.

‘The only way to game CPI is to write better stories for your audience’

The other way is our way — CPI. Using CPI there can be no editor in the world who will be able to tell you to get more page views or to go and promote your stuff on Twitter, because they don’t know how that influences the CPI — it may be counter productive! The only way to game CPI is to write better stories for your audience. We’re basically bringing old school to the digital environment and the change you have to make as a writer is to change how you’re aware of the audience, or people won’t pay you for that sheet of paper that you charged a flat fee for. That’s dying and will be dead very soon. We’re the bridge between those two islands.

Do you think CPI might be used at the more serious journalistic end of the spectrum, or does it have an equally useful application across the board?

We’re hoping for the latter, because we want to create a marketplace and in order to do that you have to be fairly present everywhere. It can’t be a top shelf product for people who can afford it, and we’re not only looking to help journalists in news organisations but also if you’re writing for Cosmopolitan or if you’re writing for your own blog.

‘Even if you consider yourself a true amateur, your writing will get better if you use analytics, and Content Insights is designed specifically for that’

In order to do that we have to ask how everybody could benefit. Let’s take one end of the writing spectrum: a blogger who has absolutely no commercial agenda or ambitions, so they’re just writing about how good their day was. Using analytics they can see what works and what doesn’t and with our tool you can set what ‘successful’ or ‘unsuccessful’ actually means: is it about how many people are reading it or how they are reading it or how they’re reacting to it? Even if you consider yourself a true amateur, your writing will get better if you use analytics, and Content Insights is designed specifically for that. Don’t forget, at Njuz.net, we started as a small website and we grew into a very big one just because of that approach. We took care over so many other things other than just page views or writing for our own amusement. So that’s the territory we cover.

To find out more about how Content Insights could help you understand your publication, click on: editorial analytics.


Originally published at contentinsights.com on November 6, 2015.

Like what you read? Give Content Insights a round of applause.

From a quick cheer to a standing ovation, clap to show how much you enjoyed this story.