Should publishers use Facebook’s Breaking News Label?
In our most recent study, we conduct an analysis of Facebook’s upcoming feature for publishers, the Breaking News Label. After normalising for the popularity of each publisher’s content we find that shares to Facebook with the Breaking News Label get on average 51% more traffic than comparable shares without the label. However, we also observe that publications using the label typically see no change, or even a decrease, in overall traffic, clicks, reshares or engagement because other shares perform more poorly than they normally would when the Breaking News Label is used.
The Breaking News Label is an upcoming feature on Facebook that enables publishers to apply a visual indicator to stories that are considered breaking news. As part of Facebook’s ongoing effort to bolster the effectiveness of quality journalism on the platform, the Breaking News Label is designed to give publishers a tool to clearly differentiate breaking news posts from less time-sensitive editorial content. In March 2018, the feature was announced to be in closed beta, available only to a handful of select publishers.
Publishers can choose for the label to remain active for a period up to six hours. Apart from the initial visual indication, the Breaking News Label also allows publishers to string together updates to an ongoing news story. To prevent overuse, each publisher can only use the Breaking News Label for one share per day, with an additional budget of five labels per month that can be used at any time.
To assess the value of this feature, our data science team has conducted a study into the effect of the Breaking News Label on publishers’ traffic from social media.
Among of the hundreds of publications relying on Echobox daily are several major publications which took part in the Breaking News Label beta. Since March 2018, they have used the Breaking News Label to tag close to 200 shares on their main Facebook Pages as breaking news. In our study, we compare how much traffic was generated by the labelled shares relative to over 8,000 shares that were not labelled.
When choosing the article to apply the Breaking News Label to, publishers will naturally tend towards articles they consider to be of high interest to their audience. This introduces selection bias, since these articles are not equally drawn from the population of all articles, but rather from a subset of high-potential articles. To avoid our results being distorted by this selection bias, we needed to identify shares for which the Breaking News Label has not been used but which are still representative of the kind of high-interest article a publisher might have used the label for.
To do this, we used our AI’s prediction of how well each article was likely to perform on social media, based on a large set of various factors. Combining this prediction and the actual performance after being shared allowed us to measure by how much each share outperformed our initial prediction. Figure 1 visualises this for shares with the Breaking News Label (orange) and those without it (blue). We observe that on average articles with the Breaking News Label outperform our initial predictions by a far greater degree than articles without the label. On average, shares with the Breaking News Label outperform our AI’s initial prediction of the share’s performance by 51%.
To test whether this increase could have occurred randomly, we drew 3,000 samples of 200 shares each from all shares without the Breaking News Label and measured how often a random sample would exhibit a comparable difference in performance. The observed variance in the data indicates a probability of less than 2% that the Breaking News Label posts outperformed the other posts by chance. This is strong evidence that individual shares with the label do indeed generate more clicks than they would have otherwise.
However, we do not find any impact on overall traffic. If shares labelled as breaking news benefited without any negative impact on other posts, a medium-sized publisher, with 25 shares to Facebook per day, could theoretically expect a ~4% increase in monthly social traffic by using the full monthly quota of Breaking News Labels. However, the same publishers for which we just showed the significant positive impact of the Breaking News Label on individual share performance actually saw an overall decrease of 2.5% for clicks, similarly for reshares, likes and comments after they started using the Breaking News Label (Figure 2). Additionally, when considering other publishers, we have found no evidence supporting the expectation that the Breaking News Label benefits overall page performance.
While the Breaking News Label is clearly effective at boosting the traffic of individual labelled shares, we have found that these performance benefits do not translate to any overall performance increases.
The root cause is most likely that shares for which the Breaking News Label was not used performed relatively worse than they could have. It seems likely that this is at least partly a symptom of an underlying cannibalisation effect. Shares with the Breaking News Label engage viewers’ attention more aggressively and could thereby make other editorial content on the page seem less interesting or relevant in comparison to the flashy breaking shares.
While the Breaking News Label is evidently generating increased viewership for breaking stories, it decreases traffic to other stories and therefore does not raise overall traffic from social media.
For more frequent updates, follow @EchoboxHQ on Twitter. For more about Echobox, go to www.echobox.com.
