Here’s What We Know about the Audience for Election Predictions

The midterm elections in the US are next week. Who’s leading and who’s going to win? This is the season when FiveThirtyEight and The Economist break out their election predictions to try to answer those questions. They build statistical models drawing heavily on polls and other information to simulate the possible outcomes.

The political stakes are high. For a while it looked like FiveThirtyEight had the Democrats ahead in terms of their chances for keeping control of the Senate. But recently the forecast has the Republicans pulling ahead. These are anxious days.

Publishing election predictions in the news media is somewhat controversial. Some, like Zeynep Tufekci, have argued that we would be better off just ignoring these predictions. They are devilishly complex and hard to get right. In 2022 both FiveThirtyEight and The Economist admitted technical errors in their forecasts, publishing lengthy explanations about what happened and why. They probably need to hire some Quality Assurance engineers.

Predictions are hard to communicate too, often with lots of uncertainty. One study found that “presenting forecasted win probabilities decreases the impression that an election is competitive compared to vote-share projections,” and that this could potentially depress voter turnout due to the overconfidence in the predicted outcome. And people can misinterpret or mix-up probabilities, vote shares, and even seats won, including journalists that report on those numbers. For instance, Newsweek had to post a correction in July when it misinterpreted FiveThirtyEight’s forecast as meaning that the Republicans would win 60 seats in the Senate, rather than that they had a 60% chance of winning a majority in the Senate.

What’s the Audience for Election Predictions?

Given their potential to impact behavior and be misinterpreted, it’s important to get a handle on how pervasive these predictions are in people’s media diets. To do that I looked at some data from Comscore. Comscore maintains a panel of more than 1.3M people who consent to having their internet activity measured. Using this panel data, which tracks desktop browsing behavior, we can see just how many people visited FiveThirtyEight’s and The Economist’s forecasts in 2020.

In all of 2020 there were 336k visits recorded for any of FiveThirtyEight’s election forecast pages. These visits came from at least 5,202 people, which represent about 0.38% of the Comscore panelists. If we extrapolate these numbers to the US adult population in 2020, we could estimate that perhaps 1M people directly visited the FiveThirtyEight election forecast pages, and there were maybe around 64M visits to those pages in total. The numbers are much much lower for The Economist (5,407 page impressions, 641 visitors; ~120k total visitors, ~1M total visits) and so I’ll focus the rest of this post on FiveThirtyEight.

Here’s how the visits vary over time, showing a clear peak on election day:

On November 2, 2020, the last full day before the election, there were 16,657 visits to FiveThirtyEight’s forecast, which is only 0.002% of the total traffic generated by the Comscore panel that day. This was from 510 unique visitors, reflecting about 0.15% of the active panel that day.

The vast majority of visits to FiveThirtyEight’s forecast were to the top-level forecast page (https://projects.fivethirtyeight.com/2020-election-forecast). Only about 1% of visits went to the Senate or House forecast pages, or to specific state races. Where there was traffic to sub-pages it went to some of the more competitive races in states like Pennsylvania, Florida, Texas, Georgia, and Arizona.

I also looked at what other sites were referring traffic to FiveThirtyEight’s forecasts. Is it a hit on Facebook or a popular search on Google? It turns out not: 98% of the traffic came from their own homepage (fivethirtyeight.com). There is strikingly little traffic that comes in externally from search or social media, or from other news media either.

This suggests that the audience for these forecasts is going directly to the homepage and then clicking into the forecast from there — a dedicated audience that is seeking out the forecasts rather than bumping into them via other channels. Even go.com, whose parent company ABC also owns FiveThirtyEight, only referred about 0.8% of the traffic. Outside of their core audience, the reach of the FiveThirtyEight forecast seems rather limited.

Of the 5,202 people who visited the FiveThirtyEight forecast at any time in 2020, I was able to link 3,449 of them to panelist demographic info (the rest of the traffic lacked panelist identity information). Based on this I can say that the audience for the FiveThirtyEight forecast is probably what you’d expect: whiter, older, and more male. It was 68% male (compared to 53% for the whole panel), 94% non-Black (compared to 82.6% for the panel), and with an average age of 53 (compared to 44 for the panel).

What about Downstream Audience?

So far, we’ve seen that the audience for election predictions is somewhat limited in terms of direct visits, reaching perhaps a million people. But what about people who might see the predictions as reported in other media? Some critics have suggested that elites (like Jim Comey in 2016) might be moved to action by an air of confidence engendered by prediction-driven coverage.

To look at this, I gathered some data from GDELT, a massive online tracking project that ingests content from thousands of news sites. I collected all English-language news articles that contained the key terms “FiveThirtyEight” and “Forecast” between August 12, 2020 when the forecast launched, until election day on November 3.

In total there were 663 articles matched, or about 8 per day on average. There was a decent spread of sources too, not just national media like the New York Times and the Washington Post but also including a bunch of ABC and Fox News affiliates, and other local and city papers which might reach more of a local audience. Some of the outlets that wrote about the forecast the most included MSN (77 times), Newsweek (35 times), and Yahoo News (12 times). Here are the top 20 sources:

In Sum

Overall we find that the reach of election predictions is fairly limited — perhaps to around a million people — with most referral traffic coming from the homepage of FiveThirtyEight itself. Little traffic comes from search and social platforms, or even from direct links from news media. And so there doesn’t seem to be much of an audience for these predictions outside of the audience FiveThirtyEight has built. That audience is whiter, older, and more male. It’s worth thinking about the political implications of election predictions given this more narrow audience. But there’s also a not-insubstantial audience for the predictions as they get reported on and published by other media outlets. How exactly those media talk about the predictions, and whether they nuance the numbers and are transparent about uncertainty is something I’d like to look at in the future.

Note: Special thanks to Jack Bandy at Northwestern University for facilitating access to the Comscore data I analyzed for this post.