How to Read the Polls Like a Pro.
Breaking down the election coverage you’ll see this fall.
I’m Isaac Saul, and I’m the executive editor at Tangle where I write an independent, nonpartisan, subscriber-supported politics newsletter that summarizes the best arguments from across the political spectrum on the news of the day — then “my take.” For more political analysis like this, subscribe to Tangle here!
Today, I’m sharing a piece written by Tangle’s Managing Editor Ari Weitzman.
How to read the polls like a pro.
As November draws near, election coverage is going to get more and more ubiquitous. You’re going to hear about dozens of “game-changing” narratives, see a news story every time a new poll gets released, be inundated with countless takes about how candidates are performing on the campaign trail, and read an opinion attached to every piece of news saying how it’s going to affect the presidential election.
At Tangle, we do our best to separate the noise from the news and focus on each story in its own right. But campaign events are often news in their own right, and we know that the race between former President Donald Trump and Vice President Kamala Harris is going to consume more and more of the oxygen in the room the closer we get to November.
By nature, election coverage is hyperventilatory, and we’ve all got to focus on our breathing. So today, I want to hand you an oxygen mask.
New polls are going to be released every week — you should know how they work. Headline coverage about election forecasting is going to get louder — you should know how to interpret them. And each campaign is going to work overtime to tell you why their preferred issues should be central, what fundamental details provide the framework for this election, and then spin the news as we approach each milestone. We’re going to tell you what the game is, and how the Democrats and Republicans are playing it.
How polls work.
Yesterday afternoon, Scott Keeter joined Isaac and me for the first part of our Sunday podcast to talk about how polls work. Suffice it to say, Scott knows what he’s talking about; he is a senior survey advisor at Pew Research Center, where he provides methodological guidance to all of Pew’s research areas. Before that, Scott was Chair of Public & International Affairs at George Mason University and Chair of Political Science and Public Administration at Virginia Commonwealth University. He has also published four books about U.S. politics, polling, and how they continue to evolve.
At their most basic level, surveyors like Scott Keeter are statistical journalists: They craft polls to give all of us information about what citizens think and believe. Keeter told us about the methodology that Pew uses.
“We recruit a group of people using a very rigorous and large mail survey to people’s residential addresses with incentives — meaning money — telling people that we would like for them to take surveys with us,” Keeter said, noting that participants usually get somewhere between $5 and $20 per survey. “If they agree, we will impanel them and put them into our American Trends Panel, which currently has over 10,000 people in it. We use this panel two or three times a month to survey the American public — a representative sample of the American public — on various topics, including the election.”
Just this week, Pew released a new poll on the state of the presidential race that showed a dead heat: 49% for Harris, and 49% for Trump. The poll surveyed the American Trends Panel (ATP) that Keeter described, with a response rate of 91% and a margin of error of 1.3%. This poll follows a standard methodology Pew uses for their ATP polls, where they “recruit people by mail, but they take their surveys online,” as Keeter said, adding that they include an option to reach panelists by phone.
In order to make its results as reflective of the voting population as possible, Pew — like all polling outfits — has to adjust its sample to make their results more accurate.
“We recruit what we hope to be a random sample of the public, but inevitably better educated people, older people, and non-Hispanic white people are more likely to take our surveys than other kinds of people,” Keeter told us. “So we use a couple of different techniques to try to fix that. The one that’s most common through the industry is called statistical weighting, which is simply that the people that we get who don’t have a college degree or are in other demographic categories that are underrepresented — like younger adults, people under 30, for example — we simply give them more statistical weight when we tally up the totals. That goes a long way to correcting the biases.”
Lastly, they weight these panel results to account for the political affiliations of the electorate they’re polling, which they derive from a separate survey on the views of the American public. Every year, Pew performs a survey called the NPORS — the National Public Opinion Reference Survey. Pew spends hundreds of thousands of dollars working to get a random sample of American adults and then make it as easy as possible for them to respond to their survey, allowing respondents to answer either online or through the mail and offering cash incentives to do so.
“Other pollsters are using a similar method,” Keeter told us. “In fact, some pollsters are using our NPORS numbers to weight their surveys.”
According to the latest NPORS, 47% of U.S adults lean Republican, 46% lean Democrat, and 7% refused to lean one way or the other.
Polling v. forecasting.
Pollsters are very interested in providing snapshots in time as accurately as possible. As citizens keeping up with the news, it is tempting to look at these snapshots and plot them over time to create a kind of movie — to watch the presidential race like a literal race. Through this “horse race” model, when you put all the polls together over time, you can get a sense of direction and momentum.
But that’s only part of the story, because there is a difference between polling and forecasting. Pollsters are interested in describing the views of the electorate as accurately as possible. Forecasters, on the other hand, want to not only predict what the electorate will say but who the electorate will be — that is, who is actually going to turn up, and in what proportion.
“Forecasting is its own game,” according to Keeter. “And it’s a refuge of pollsters to say, ‘Our poll was good except we didn’t get the turnout right.’ But of course political practitioners know the turnout is kind of everything.”
Polls are data, and data is king in our era of informed decision making. One of the reasons why you don’t see any maps like Reagan’s 1984 win anymore — where the Republican incumbent won by 18 points and carried every state except Minnesota and Washington, D.C. — is that politicians have access to much more data about where voters stand on key issues. Every new poll gives campaigns information on how their message is coming across, telling them how to tweak their message to reach the largest number of people. For this reason, polling is a big business (or, in the case of organizations like Pew, big non-profits).
Surveys from Pew along with polls from places like Rasmussen, Siena College, Quinnipiac, Gallup, Marist College, Morning Consult (and others) allow politicians to craft their positions based on what they think voters already believe rather than trying to sell the populace on their own vision. You can like it or not, but that’s the way things are.
If you want to read the polls like the campaigns do, you have to know where to focus your attention. Which issues will motivate voters to turn up and vote? Will the fundamental aspects of the election — like the economy and foreign affairs — shift voters’ mindsets? And what campaign milestones are in the rearview or on the horizon that could sway what the numbers are saying?
Issues.
We not only have polls gauging how voters feel about the candidates they have to choose from, but how they feel about the issues. Taken together, the polls give us a lot of information on issues voters care about, cross-referenced by demographics on age, race, gender, education level, and state of residence. These data segments are called crosstabs.
Click to see the crosstabs of the latest New York Times/Siena College poll.
Good crosstabs indicate how much, at the macro level, different demographic sections care about the issues. However, this is only true at the national level, where the election isn’t technically decided. Much more actionable data comes from the state level, where high-quality polls are harder to come by. Since national polling is hard enough in a country of over 260 million adults, limiting respondents by state is even harder to do accurately.
That’s part of the reason crosstabs are so important, because cross-referencing solid data from national polls with state-level demographics can serve as a useful proxy for information among states. For example, if I know that 83% of poll respondents who do not have a college degree say the economy is only fair or poor and that 63% of North Carolinians do not have a college degree, then I can make the reasonable inference that the majority of North Carolinians are dour on the economy.
Another piece of the puzzle is down-ballot polling. How respondents in each state show support for Senate, House, or gubernatorial candidates gives us a good indication of where voters stand on positions generally held by Democrats versus Republicans, as well as a sense of candidate popularity. For instance, we know that Democratic Senate candidates in swing states were running well ahead of Biden a few months ago.
When Harris replaced Biden at the top of the Democrat ticket, that gap decreased — but it’s still present.
This tells the campaigns a few things: First, given the consistency of the spread, it looks like Democrats have a slight edge on the issues. Second, it seems that moderates and independents are cool on Harris. Either that means that they won’t be enthused to vote for her, or it means there are a lot of people who are willing to vote Democrat but are waiting to be convinced. Or it just means that these voters will be influenced by a suite of factors known as election fundamentals, which are a different kind of beast.
Fundamentals.
Election fundamentals are historical indicators that show how people make up their minds in elections based on big trends, not necessarily stances on issues. For instance, when the economy is doing well, the incumbent party stands to benefit. When the economy is doing poorly, it hurts the incumbent.
Nate Silver, who used to run the election forecast model at 538 and now runs his model independently (with the same underlying intellectual property), described what fundamentals he weights in his forecast at Silver Bulletin, and how.
“First, we take a current ‘snapshot’ of the race — which is based on polling, although with some fancy adjustments to smooth out the data in states where there isn’t much polling — and then regress it toward a prior based on ‘fundamentals’ (which in our case consist of the economy and incumbency).”
Those fundamentals help forecasters to correct polls and adjust forward-looking expectations. Silver interprets the fundamentals to show an advantage towards Trump. “Our model’s answer, based on the economy being about average and there being no true incumbent in the race, is that ‘normal’ means a tie in the popular vote but Democrats being underdogs in the Electoral College,” he said.
That’s why Silver gives Trump a 60% chance of victory as today, even though Harris maintains a 2-point lead in the national polls.
Silver’s former company, the now ABC-owned 538, takes a different view. 538 sees Harris as representative of the “incumbent party,” and thus credits her for a normal economy in their forecast.
Logan Phillips, who runs the site RaceToTheWH (which was 93% accurate in predicting the results of the 2020 elections and 96% accurate in 2022), takes into account economic data with results from special elections and fundraising. The RaceToTheWH forecast aligns much more closely with 538, conferring to Harris a 56% chance of victory.
Milestones.
Polling gives us a snapshot in time — but how voters answer polls in September doesn’t exactly tell you how they’ll vote in November. Over time, election forecasters have learned that there are a few moments in the campaign that provide predictable bumps in support; and if you correct for those bumps, you’ll be able to get a clearer picture.
This year’s presidential race is far from conventional, so forecasters have been having a hard time getting a read on it — how do you historically weight the polls after the attempt on Trump’s life or Harris replacing Biden as the Democratic candidate? One of the few normal events so far has been the Democratic National Convention, which forecasters expected to bump Harris by a larger amount in the polls than they did. Accordingly, her boosted poll numbers decreased her chances of winning in many forecasting models.
Moving forward, we’ll have a few more predictable bumps. After Tuesday’s debate between Trump and Harris, opinion polls mostly showed Harris as the winner. But will that have any effect on the polls? If Harris’s numbers go up, are they going to increase by enough to indicate independent voters are changing their minds? When polls go down for one of the candidates after the dreaded “October surprise,” are they going down enough to show that voters are swayed? And when we get bad news about a candidate — like Trump’s felony conviction — will the polls remain stubbornly inert and show candidate loyalty?
In sum, when you look at the polls, you can’t just interpret who’s ahead right now in terms of the candidate in the lead. Ask yourself: Are the fundamentals of this election favoring the incumbent or the challenger (and do you believe this election truly has an incumbent)? Are the polls moving in response to the most recent piece of big news more or less than we expected? And how are the candidates polling in key demographics in swing states on the issues that matter? Furthermore, what are those issues that are going to matter in this election?
Watch the issues, the fundamentals, and the milestones.
If you tuned in for the debate on Tuesday, you may have noticed something: Donald Trump fought hard to get the last word on most issues throughout the night, whether the topic of discussion was the 2020 election, immigration, the economy, or abortion. That’s one of the reasons Trump has kept hold of national-level appeal for so long; he not only wrangles the spotlight, but he knows how to keep it. And as the presidential race enters the last two months, getting that last word is going to be important.
But the election isn’t a debate. Neither candidate is going to be the one who gets the final word inked before voters enter the booth on November 5. Instead, the campaigns are vying to set the terms of the last topic for discussion. Throughout October, each campaign is going to try to frame the election to be a referendum on one or two of their preferred issues. Don’t be surprised to watch this unfold. Instead, get ready for how both campaigns are going to try to frame your choice.
One quick disclaimer: This is not to say that the following issues will be the only ones that matter. As we saw over the summer, we’re always one unexpected event or terrible debate performance from the entire race turning on its head. This is just to describe the elements of the game each side is playing: the issues, the fundamentals, and the milestones.
Three issues that will matter.
When it comes to the issues, both Trump and Harris have a delicate line to walk: They want to win over as many of the voters in the middle as possible while retaining the enthusiasm of their respective bases. They want to tip the polls while shaping the electorate to their favor.
To unpack these issues, let’s look at the latest Pew poll and the September New York Times/Siena College poll crosstabs. Remember: These are just two polls, nearly two months out from the election. We can’t draw conclusions about the outcome of the race from analyzing this information, but we can make some valuable observations.
- The Economy
Let’s start with the economy. Trump’s largest advantage is on the economy, which is also the issue most voters rate as most important this year. According to Pew, “a 55% majority of voters say they are very or somewhat confident in Trump to make good decisions about economic policy, compared with 45% who say that about Harris.” According to NYT/Siena, the economy is the #1 issue for 35% of Trump supporters compared to 8% of Harris supporters.
Simply put, Americans are by and large more optimistic about the possibility of living in a Trump economy than a Harris economy. For Trump, the game is simple: Talk about how strong the economy was before Covid and tax cuts, talk about how harmful inflation was during the Biden-Harris term, and ask a lot of questions about why Harris hasn’t done what she said she wants to do in the last three and a half years. This is how Trump put it during the debate: “Everybody knows what I’m going to do. Cut taxes very substantially. And create a great economy like I did before.” This is a simple and resonant message, and the data shows that it is in his best interest to repeat it.
For Harris, she has three options for how to counter Trump’s economic advantage: Chip away at his image as a successful economic leader, improve her image as an economic leader, or downplay this issue and make the election about something else. Her chosen tactic is pretty clear: Attack Trump’s image. During the debate, Harris quoted a study by Penn’s Wharton Business School (Trump’s alma mater) that claimed the former president’s plans would hurt the economy. She also has attacked him for how his handling of Covid led to Biden inheriting a disaster, clearly trying to focus voters on the Trump economy of 2020, not the Trump economy of 2019.
As much as Harris is trying to sell voters on her “opportunity economy” and plans for tax incentives for small businesses, she probably knows that she really doesn’t have enough time to compete with Trump here. Because Americans are still feeling the effects of inflation and worried about economic forecasts, she won’t win over any moderates by deflecting. Instead, Harris clearly is attempting to bring this issue to a stalemate and then pray for a sunny economic forecast. That’s risky, because the way the economy evolves over the next two months is out of her control (more on that later).
2. Abortion
On abortion, it’s the opposite story.“Harris’ lead over Trump on abortion is a near mirror image of Trump’s on the economy: 55% of voters have at least some confidence in Harris, while 44% express confidence in Trump,” according to Pew.
Here, Trump is truly stuck between the rock of losing his pro-life base and the hard place of getting clobbered by suburban women and independents who are still furious about the repeal of Roe v. Wade. However, it’s clear based on his strategy that Trump has chosen to run to the middle. It’s not hard to see why. Abortion is the #1 issue for 3% of Trump supporters, compared to 28% of Harris supporters, according to NYT/Siena. The cold election calculus incentivizes Trump to trust that his pro-life base will not abandon him if he maintains his message of letting the states decide.
Since the fall of Roe and the disappointing 2022 midterms for Republicans, Trump has learned to sing a more electable tune on abortion. Sure, he’ll still hit the high notes warning against late-term abortion, or celebrating his remaking of the Supreme Court, but his song is about states’ rights. Mentions of abortion were all but removed from the Republican Party platform, it is not mentioned at all in his Agenda 47, and he insists he would not make any decisions about abortion if elected president any time he is asked about it. “Each individual state is voting. It’s the vote of the people now. It’s not tied up in the federal government,” Trump said during the debate.
3. Democracy and Immigration
Then there are the #2 issues for Harris and Trump supporters respectively: democracy and immigration. 92% of Harris supporters believe Trump did something wrong following the 2020 election, with 88% saying he broke the law, according to Pew. For Trump supporters, those numbers are 27% and 4% respectively. This is an issue where the strategy is relatively simple: If Harris can make January 6 and election denialism a defining issue of the election, she’ll energize her base while putting serious doubts in 25% of Trump’s. It’s a slam dunk. For Trump, the best he can do is force a stalemate.
The far more complicated issue is immigration. On the surface, the numbers and tactics are similar, though less extreme. 53% of voters trust Trump to make good decisions on immigration policy, compared to 43% for Harris (according to NYT/Siena) — the second-most favorable issue for Trump behind the economy. From her position within the Biden administration, whether you call it the “border czar” or “the person whose job it was to address root causes of immigration in South and Central America,” it’s clear that Harris is at the very least partially answerable for the Biden administration’s figures. During the last four years, Customs and Border Patrol (CBP) has reported over 10 million encounters at the southwest border. Those numbers started to rise at the end of Trump’s term, but to nowhere near the levels sustained under Biden.
However, that’s where things get interesting. CBP reports monthly data at the beginning of the following month — i.e. at the start of September we got the full report for the month of July. Based on recent trends, migrant encounters were expected to tick up in July and to continue to rise through October. However, we saw encounters go down — from 130,000 encounters to 104,000, the lowest number since February 2021. From here, even an increase of 20% or so from where we are wouldn’t get us close to the level we saw in January, when Texas law enforcement faced off against federal officials on the banks of the Rio Grande.
That all sounds like good news for Harris — and it is. However, the issue of immigration is not going away; it might just be getting started. Instead of stories about “a border crisis,” get ready for more stories about immigrant communities impacting small-town America where they might help fill open jobs, but can also overwhelm social services and create tensions on the ground. From now until November, we’ll always be one gruesome story about an unauthorized migrant crashing their car or committing a violent crime away from immigration becoming the defining issue of the 2024 election.
A few fundamental questions.
Remember, fundamentals refer to historical trends that have been found to impact all elections in the modern era. Though there’s definitely way more to it than this, in a nutshell, incumbent party candidates require a few things to be put in a good position to win. One: Don’t be involved in any unpopular foreign wars. Two: maintain a strong economy.
What goes into a strong economy? A number of variables — like the unemployment rate, average amount of savings for households, inflation-adjusted cost of living, inflation itself, the stock market’s performance, and GDP — all comprise a holistic view of economic health. But at the end of the day, the economy is felt by individuals, and that feeling is measured by consumer sentiment surveys (the gold standard for measuring sentiment has been the University of Michigan’s survey of consumers).
Sometimes, there’s a gap between how economists expect people to feel about the economy and how people say they feel about it. Economist Kyla Scanlon coined this phenomenon as “the vibecession.”
When Isaac interviewed Scanlon in January, she gave us some reasons for this discrepancy.
“I think a lot of people are looking around and seeing some elements of the political stagnation (especially as we go into an election year), seeing inflation and not feel like it’s going down, seeing a housing crisis, feeling like their wages have stagnated, and coming out of the other side of the pandemic I think there’s a lot of mental health issues, too,” she said. “So of course they’re going to be feeling down about the economy.”
And to be sure, all of those things have dragged Joe Biden’s presidency down. Politics (especially in Congress) seem gridlocked. Inflation has been sticky. Housing is still expensive and unavailable. Wages have been stagnant (until very recently). Regardless of high GDP or low unemployment, this has all been bad news for Biden.
However, there is good news for Harris. After replacing Biden atop the Democratic ticket, consumer sentiment rose by a higher-than-expected margin. On Tuesday, Census data showed that real wages in 2023 rose 4% from 2022, the first gain measured by the Census bureau in four years. On Wednesday, a new report from the Bureau of Labor Statistics showed real wages increasing by 1.3% year over year, outpacing inflation as measured by the consumer price index (CPI). The latest jobs report showed unemployment falling slightly from 4.3% in July to 4.2% in August. The S&P 500 is up. With the election fast approaching, the odds that The New York Times runs a headline that reads “U.S. Entering Recession” are getting longer and longer. Could the vibes finally be right for Democrats?
It’s certainly possible, though a few alarm lights are blinking on the economic dashboard. Lurking in the jobs report is the fact that the gender wage gap just increased for the first time in four years. That may not be a big deal, or even really statistically significant, but the CPI report contains its own heavy caveat: The latest numbers just missed economist expectations. The best piece of economic news Harris can hope for to stymie attacks about inflation is a hearty 0.5% interest rate cut when the Fed board meets next week, but the CPI missing expectations makes a 0.25% decrease much more likely. That wouldn’t be terrible, but would also not be the win Harris is hoping for (along with most prospective borrowers).
Harris will have to hold her breath through November. If investors have been expecting a heartier rate cut and the market takes a dip, that’s a win for Trump. If the next jobs report shows a tick up in unemployment, that’s a win for Trump. If inflation jumps again, that’s a win for Trump. And if the Fed decides not to cut rates at all (which is highly unlikely), that’ll be a massive win for Trump.
None of these things have to spell an immediate recession; all they have to do is raise the question. Remember: Trump wants to make the last word on the election about the economy. As the erstwhile incumbent, Harris needs bad news about the economy to stay out of the headlines for another two months — or better yet, a slow parade of positive headlines about inflation cooling and the Fed cutting rates and a soaring stock market.
Fundamentally, there really is no “normal” to compare this election to.
A strong jobs report might do absolutely nothing for the vice president. A dip in the economy may not be the gust of wind in the former president’s sails that we expect it to be. We assume both of those things are true — but we just don’t know.
Six upcoming milestones.
As November 5 draws closer, the following dates will mark important moments that will sharpen the focus of the election picture. Brace yourself for the headlines on these dates, and for the accompanying forecast adjustments.
September 16: This will be the first Monday after the second presidential debate. Trump has said that there will not be a third. Therefore, the polls we can expect to see on Monday will clue us in to how voters see the relative strengths of the candidates after a brief side-by-side comparison.
September 17–18: The Federal Reserve Board will meet and decide whether they will cut interest rates for the final time before the election.
September 27: The next release date of the Personal Consumption Expenditures (PCE) index, the Fed’s preferred measure for inflation. The PCE will be released one last time before the election in late October.
Beginning of October: CBP will release its August border encounter numbers, likely its last report before the election.
October 1: The vice presidential debate between Gov. Tim Walz (MN) and Sen. JD Vance (OH). This could be the last time representatives for the Democratic and Republican tickets share a stage together before the election.
October 4: The Bureal of Labor Statistics will release its September jobs report.
October 10: The Bureau of Labor Statistics will release its final CPI report before the election.
November 1: The Bureal of Labor Statistics will release its October jobs report, its final jobsreport before the final.
Unknown: The dreaded “October surprise.” We don’t know what it will be, when it will come out, which candidate it will target, or even how many “surprises” there will be; but you can expect some kind of “bombshell” report to come out in mid-to-late October (and you can expect media outlets to be sensitive to news reports and to claim that every piece of news we get after October 15 is of grave significance).
Take a breath.
Just yesterday, Isaac and I sat down and finished our podcast exercise of ranking every election since 1988 on a scale from least interesting to most interesting. Not to spoil the ending, but we agreed that the most interesting election of that period is the one we’re covering right now. We’ve had two candidates coast to their nominations without really participating in their parties’ primaries, a historically decisive debate, two fascinating picks for running mates, one candidate getting shot in the ear, the other being replaced on the ticket, and we still have two more months to go.
Take a breath. I know that you likely cannot stand one of the candidates in this race; or, alternatively, you firmly dislike both of them. You probably are eager for your side to win and fearful of the other side winning.
Between now and November, there are going to be a lot of headlines designed to capitalize on your fears. Remember the game the campaigns are playing. There are predictable milestones coming up — know them. There are fundamental factors that partisan media outlets will want to exaggerate or downplay — anticipate them. And there are a set of issues that each side will want to boil the entire election down to. Be aware of how they’re being framed, and remember the issues that matter to you.
I can’t promise you that the next two months of our coverage will be smooth and stress-free. When you get overwhelmed by the news, don’t be afraid to take a break. We’ll be here when you come back; we’ll still be here until the last ballot is cast, and we will be here after. We all will; I can promise you that.
If you liked this article, you can subscribe to Tangle here to receive free politics newsletters Monday-Thursday and can sign up for a paid membership to receive special Friday editions like this one.
