Releasing the @stefanhilts Election Forecast Model
After a final batch of tweaking, the @stefanhilts electoral forecast model is ready to release, and surprise!, its results look quite a bit like some of the other forecasters out there. So, what’s the point of putting this together? In a similar way that having multiple pollsters helps add confidence around individual findings, having more analytic eyes looking at those polls should also help justify the findings of any individual poll-watcher. I’ve included details on the methodology at the bottom of the article, but I’ll skip to the punchline now for those less interested in details.
As of 9/26/2016 polling data (the day of the first debate), the model is predicting 70% odds of Hillary Clinton winning the presidency, with an expected 295 electoral votes. This was near 75% as recently as a week ago, but negative poll numbers out of North Carolina, Nevada, and Ohio have helped to push this down. These don’t add to 100% because of rounding and a small slice of a tie.
Results are concentrated in a relatively narrow band, with 41% of modeled results within a “Florida band” between 240 and 298 electoral votes. This is the range over which Florida’s 29 electoral votes flipping one way or the other effectively decides the election. There’s almost no results on the fringes, since Trump has a more or less guaranteed 99 electoral votes from solid republican states, with another 87 strongly leaning (see chart in the next section). For her part, we have Hillary effectively locked into 161, with another 89 strongly leaning. The number of true ‘tossups’ is actually very small, which is of course why you see so many resources dedicated in a few battleground states.
‘Path to Victory’
The chart below highlights how few states are really in play. To be fair, these categories are effectively arbitrary, but I’ve defined >10 as solid, and >4 as leaning, with anything <4 as a true toss-up. The idea of this chart is to show the ‘Path to Victory’, meaning the theoretically easiest way for a candidate to win the presidency based on our current odds of winning in each state. As it stands now, if Clinton wins all the Solid and Lean Democrat states, she needs to pick up only Wisconsin, Maine, and Colorado out of the tossup pile to win, taking a lot of pressure off Florida, which has swung wildly over the last few weeks but remains a real coin-flip. Trump, on the other hand, would need to sweep in North Carolina, Ohio, Arizona, Nevada, Florida, and Colorado to win assuming he doesn’t pick off any of the less likely victories.
This difference is what’s really at the heart of the 70%/30% overall election odds right now, though Trump’s gains in Nevada and North Carolina in the last week have significantly improved his chances. There’s a lot of ways to make electoral math work for both, but in short, Trump needs to win in territories where polls show him as an underdog, whereas Hillary simply needs to hold serve.
State-Level Odds vs. Other Poll-Watchers
Finally, this is the state-level probabilities of a Hillary Clinton win, sorted in order of the @stefanhilts model, compared to 4 other major poll-watchers: the NYTimes Upshot, Nate Silver’s 538, the Daily Kos, and Predictwise.com. The number at top summarizes the expected total electoral votes for Clinton.
Overall, the results are very comparable, and the @stefanhilts numbers are basically right in the middle of the pack, with the Upshot at the most Hillary-bullish and The 538 skewed quite a bit more towards a total election tossup. AS of 9/26, 538 is suggesting Hillary’s odds are at 55% of winning the presidency.
In most cases, odds differ by a few points here or there, but the most important discrepancy here is Pennsylvania’s 20 votes, which 538 has at 15 pts lower than the consensus average, and nearly 20 pts lower than @stefanhilts. Personally, I just don’t see the evidence for this as a true swing state at thus point. Recent polls have narrowed somewhat, but only 1 out of 29 polls analyzed since the beginning of the year showed Trump in the lead (by 6 points, Quinnipiac in early July), and otherwise the Clinton lead has bounced around at about 5%, including two subsequent Quinnipiac polls putting Hillary at +5 and +9. I’ll dive deeper into PA in a subsequent post, but the PA projection is a huge one. Assuming that PA goes for Trump, my Hillary numbers drop to 57%. That’s the kind of swing you could say makes it a “must win”, but at today’s numbers, a loss in Pennsylvania would imply some damaging numbers in Michigan, Wisconsin, and Florida. At this point, there are much more important battlegrounds up for grabs.
Conclusions
There’s a lot of the race left to go, not least in just one hour, with the first of three debates which is sure to swing these numbers one way or the other. I’ll be updating this model frequently throughout the rest of the election season, and I encourage you to reach out to me on twitter @stefanhilts with any thoughts, observations, or questions.
Methodology Summary
In any aggregation analysis, there are a lot of decisions both large and small that need to be made, and together, these can have very significant impacts on the final results. Here’s a list of the most important decisions made in putting together this analysis:
1. Which polls to include? Since I’m not an expert on polling methods, I’ve deferred on this decision to electoral-vote.com, who has also aggregated polling data in an easy-to-analyze csv format, and analyzed historic bias of these polls to decide are reliable enough to include. Thanks guys! In general, the polls they include are from respected survey firms and universities across the political spectrum (Fox News, PPP, Quinnipiac, JMC Analytics, Suffolk University, and large list of others), and use a variety of methodologies, but are dominated by phone surveys of likely voters.
2. How to weight different polls? Here’s where a lot of the analysis comes in. In a heavily polled state like Florida, you might get 5 new polls a week during the late election season, but others might only see one a week. Ohio, for example, has 7 polls since September 1st despite being one of the biggest battleground states. With election events (conventions, debates, gaffes) swaying numbers significantly, the trick is to capture as much of the poll shift as possible without relying too heavily on any individual polls, since margin of error on these can be significant. There are a few factors used to determine the weight each poll receives in the final state-level analysis.
a. Age: More recent polls are better. In technical terms, I use a sigmoid function decayed by an exponent function, with the final effect being that all polls under a week old get the same weight, and polls 3 weeks old lose half of their weight. This might seem overly generous to aging polls, but otherwise, there’s a bit too much reliance on individual polls for a number of states.
b. State Trend: The downside of the weighting scheme above is that true shifts due to events are diminished, understating the extent to which the race has turned. The state trend analysis works to walk this back by observing the trend in recent polls. This module can be summarized as deciding whether it is more likely that a poll is an outlier (noise), or that opinion has shifted (signal). As evidence mounts for the shift, the state trend analysis adjusts the age-weighted values to keeps the odds of signal/noise equivalent in the face of new data.
3. National Trend: More important when polling is infrequent, this adjusts for the national movements since the last state polling date. For example, if we haven’t seen a new state poll in a month, but national support for Clinton has trended down 2% in that time period, we need to adjust our numbers derived from the Age and State-Trend analysis to capture recent movements. This is done based on a correlation matrix from state-to-state and state-to-national values.
4. Running the Mock Elections: This part has a ton of smaller decisions that are made, but one of the major choices is how to use random numbers. There’s a handful of ways to do this that are equally valid depending on your assumptions, but we’ve used two different values for the @stefanhilts model.
a. For the state-level voting margins going into the election, we assume uncorrelated errors, meaning that there is not a consistent bias across states in the polls used. If our analysis suggests Hillary has a 2-point lead in Florida, the true value could be higher or lower by a normally distributed random amount which is unrelated to the values in another state. This controls for errors in the polling analysis and errors in the aggregation analysis that we’ve done.
b. Nationally, we assume voting-day randomness to be normally distributed, but correlated on a national level. This controls for late swings, variations in turnout of particular demographics, and a host of other factors which may swing results one way or another from final polls. This would also capture variance from systematic bias in the polls as long as it remains within a reasonable margin.
1%
