Dispelling Three Fictions About the Polls

Ryan Matsumoto
Sep 7, 2016 · 6 min read

As we return from Labor Day weekend, people are starting to freak out about the latest political polls again. The most noteworthy polls from Tuesday included a CNN poll that gave Trump a 2-point lead nationally (45% to 43%) and a Washington Post / SurveyMonkey 50-state poll that gave Clinton a 6-point lead nationally (48% to 42%) but a strong 244–126 edge in the Electoral College. The poll also found Johnson doing surprisingly well in the 4-way race, with >= 15% support in 15 states and >= 10% support in 42 states.

As usual, partisans of all sides have tried to interpret the poll results to fit their own narratives. Today I will dispel with three fictions about the latest poll results.

Fiction #1: The CNN Poll is Skewed

Many political pundits, including MSNBC’s Chuck Todd, have criticized the CNN national poll (Trump +2) as a “skewed poll” that undersamples minorities, college-educated voters, and younger voters. Todd adjusted the CNN poll to reflect the electoral composition from the 2012 exit polls, giving Clinton a 4-point lead. (See below)

The problem with this analysis is that there is evidence that the 2012 exit polls were not representative of the actual electorate. Exit polls’ main purpose is to examine how different groups voted and why, rather than estimate the demographic composition of the electorate. New York Times’ Nate Cohn found that two other measures of the electorate’s composition (The Census’ Current Population Survey (C.P.S.) and the Catalis voter file) were more accurate than exit polls. These two alternative measures found a significantly older, whiter, and less-educated 2012 electorate than the exit polls. (See below for NYT graphic)

Interestingly, all three of these groups that the exit polls underestimated (white, no bachelor’s degree, 45+) are among Trump’s strongest. Adjusting the CNN poll to match the 2012 exit polls would naturally give Hillary a larger lead given Trump’s strength with the exit polls’ underrepresented groups. The differential between exit polls and C.P.S./Catalist suggest that Obama overperformed with white working class Northern voters relative to the exit polls. Trump’s path to the White House runs right through these potential crossover voters in the Rust Belt.

Cohn calculated that adjusting the CNN poll to match C.P.S. data suggests a Trump +1, much closer to the topline result than MSNBC’s “unskewed” poll. Is Trump actually winning the popular vote right now? Probably not. FiveThirtyEight estimates that Clinton is winning by 2.5 points, but a Trump +2 poll is not out of the bounds of reason.

Fiction #2: Millennials are Being Excluded from the CNN Poll

Another rumor floating around the Internet is that the CNN poll excluded voters ages 18–34 in order to suppress the Johnson/Stein numbers to protect the two-party duopoly. They point to the crosstabs of the CNN poll (see below), which show “N/A” for the 18–34 age group but actual numbers for the other age groups.

This is simply not true. CNN’s methodology section on page 21 states that

“All respondents were asked questions concerning basic demographics, and the entire sample was weighted to reflect national Census figures for gender, race, age, education, region of country, and telephone usage.

Crosstabs on the following pages only include results for subgroups with enough unweighted cases to produce a sampling error of +/- 8.5 percentage points or less. Some subgroups represent too small a share of the national population to produce crosstabs with an acceptable sampling error. Interviews were conducted among these subgroups, but results for groups with a sampling error larger than +/-8.5 percentage points are not displayed and instead are denoted with "NA".”

Basically, this means that their unweighted national sample did not have enough young voters to make a statistically significant estimate of presidential support among 18–34 year olds. This may be because young voters are less likely to answer political polls than older voters — in this case CNN would have weighted the young voters who did answer a bit higher than older voters to match the Census demographics.

Fiction #3: Clinton Has an Electoral College Advantage

Another rumor is that although Trump is gaining in the national polls, Clinton will almost certainly still win because she does better in the Electoral College than the popular vote. As we all know from the 2000 presidential election, it is certainly possible for one candidate to win the popular vote and another to win the Electoral College and the presidency.

Support for this theory comes from sources like the RealClearPolitics ‘No Toss-Ups’ electoral map, which estimates that Clinton/Kaine would defeat Trump/Pence by a healthy 340–198 margin in the Electoral College if the election were held today, despite having a slim 3.3 point edge in the popular vote.

However, the data does not support this hypothesis. FiveThirtyEight, the political blog which was extremely accurate in their 2012 presidential election state estimates, estimates in their polls-plus model that Clinton has a 7.0% chance of winning the popular vote and losing the Electoral College vs. only a 1.6% chance of vice-versa. This actually suggests that Trump has an Electoral College advantage vis-a-vis the popular vote.

So why are the RealClearPolitics estimates wrong? One simple explanation is that their estimates include the results of older polls that are no longer as relevant. RealClearPolitics’ state polling averages from states like Michigan, Florida, and Virginia include polls from earlier in August when Hillary was leading Trump nationally by 7–10 points rather than 2–4 points now. Once more state polls come in, we can expect the race to get tighter in the swing states.

Other proponents of the ‘Clinton Electoral College advantage’ theory point to today’s Washington Post / SurveyMonkey 50-state poll, which shows that Clinton has a very strong 244–126 advantage in the Electoral College despite a somewhat close 6-point race nationally. She’s leading him in Texas!

However, what really matters is what happens if the 6-point race nationally closes to a tied race. That is when the Electoral College / popular vote divide would matter, like in 2000. Assuming each state shifts 6 points in Trump’s direction to match a 6-point shift nationally, here is how the Washington Post / SurveyMonkey electoral map would look if the race were tied nationally. Trump would win the Electoral College 320–218, sweeping many of the swing states.

What might explain why Trump has an Electoral College advantage? One theory is that he is underperforming in traditionally red-states in the South (i/e Texas, Arizona, Georgia, Florida, North Carolina) because of losses among college-educated whites and Hispanics while overperforming in Midwestern blue-states due to his strength among non-college educated whites. In a close race, Trump could hold onto all the traditionally red states by smaller margins than Romney while pushing himself narrowly over the top in Midwestern states like Wisconsin, Michigan, and Iowa.

It is certainly possible that as the election progresses, state/national polling will shift to suggest a Clinton Electoral College advantage vis-a-vis the popular vote. But the data suggests otherwise at the present.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade