The curious case of Wisconsin in 2016
Let’s use Benford’s law to look for vote-tampering!
Why look at Wisconsin?
The 2016 election results from Wisconsin were improbable and weird. Wisconsin was the only state in which the 2015 Presidential election results fell outside the margin of error of every major pre-election poll.
It’s entirely possible the unprecedented difference between the polling numbers and the actual results could be attributed to polling blindspots and methodology holes. More nefariously, the improbable results could be the result of direct tampering by ‘bad actors’.
How to Detect a ‘bad actor’?
So which is it? Short of physical evidence, how can vote-tampering by ‘bad actors’ be detected? How can valid election results be distinguished from tampered results post the election? Can we test the results using only public voting data?
One simple way to look for election tampering is to use the same analytical tool the SEC uses in accounting forensics to spot fraud; a tool called Benford’s law.
How ‘Benford’s Law’ Works
What is Benford’s Law? Essentially, Benford’s law says the digits in results from counting things always follow a predictable pattern.
To test any natural data set for ‘normalness’, we can compare our test set of data, the test distribution, against this expected pattern, the Benford distribution. The more poorly our test distribution ‘fits’ the Benford distribution, the more likely the data has been ‘made-up’ or tampered with, e.g. there is something rotten with the numbers. ‘Fit’ can be measured by calculating a simple correlation between the test and Benford distribution. For example, the test distribution seen below in yellow matches the expected Benford distribution (blue) very closely, so at the least we can be confident there hasn’t been any unsophisticated tampering with the underlying tallying.
However if the test distribution does not ‘fit’ Benford’s, it has a low correlation with Benford’s, this is evidence of tampering.
low correlation = bad fit = evidence of data tampering
Wisconsin Analysis: something rotten?
Wisconsin 2016 presidential voting data, like all voting data, should look ‘normal’ with respect to Benford’s law, especially at a granular level. So if county (ward) level voting data is ‘abnormal’ with respect to Benford’s, it’s suspicious.
Analyzing the Voter Data — Method
Step 1: Compile the data — Gather Ward-by-ward Wisconsin election results
Gathering the data: a saga
Good times were had (thanks Joe!) entering, by hand, ward by ward, voting data for each Wisconsin county. (only to have the commissioner provide a complete data-set one day after we’d finished…) I won’t bore you with details, except to say it was boring.
Step 2: Calculate the test distribution) by summing up all the ward level data and calculating the frequency of each digit (see data + analysis here)
Step 3: Calculate the correlation of the (Benford distribution) vs (the actual observed distribution for each county) for each of 3 data sets: 1) the election winner vote count 2) the loser vote count and 3) overall vote totals.
Again, data for each set should match Benford; so the lower the Benford correlation score, the more abnormal the underlying voting data, and therefore the more ‘suspicious’ the corresponding results.
The Results
Looking at the ‘clump’ of counties in the chart above, you can see that the average correlation with Benford’s is around 85%, pretty high, this is good, as this is matches what we’d expect from untampered voting data. The clumping nature of the results also tells us most counties’ voting data follows the law quite closely as expected. However, as you can see, there are a few outlier counties that display a significantly lower correlation.
Above is a zoomed-in view of the bottom part of the Benford correlation chart: this is the ‘abnormal’ zone, the zone of curiosity, the zone of outliers, the zone of intrigue, the zone of mystery… you get the idea…
As you can see in the chart, Wisconsin counties Ozaukee, Door, and Kewaunee had the most abnormal voting data with respect Benford’s. The above chart show Benford’s vs vote totals, but there was another highly anomalous county that jumped out when looking at just individual candidate totals, Milwaukee.
Milwaukee Seems Curious
Overall, Milwaukee voting data jives almost perfectly with Benford’s law, with a 99% correlation, exactly as we’d expect for a county with the highest population in Wisconsin. At the aggregate level, there’s nothing weird here.
Trump Vote Totals Correlation: 92% —again, Trumps data is in close correlation with the expected Benford distribution, nothing weird here either.
However, in Milwaukee, comparing Clinton’s vote total vs Benford’s showed a correlation of only 21%. These results are far from the average ‘fit’ of 85%, and even more curious in context. Overall votes (set 1) and Trump’s vote totals (set 2), closely match their expected distribution, so why not Clinton’s vote totals (set 3)? I don’t know what’s going on here, if any data scientists have more insight please comment.
Curious Counties vs Voting Machine Type
As we also saw above, there were three smaller counties that showed abnormal Benford distributions with respect to vote totals. I did not attempt to evaluate the stat significance of these results, there may be nothing to see here, however these counties also had something else in common.
Doing a bit more digging, I noticed that the top three ‘suspicious’ counties all use the same voting machine vendor, Dominion. I didn’t do further regression analysis of voting vendor vs each county, but now, more than a year after this study was complete, as evidence clearly points to specific attempts to tamper with the election results in Wisconsin, I sincerely hope officials conducted a thorough forensic analysis on these machines (and all vote counting systems downstream).
Where to go from here?
You can play with the raw ward-level data analysis here
If you want to do your own analysis, just clone the linked sheet and have at it! The above analysis using Benford’s is only one way of analyzing Wisconsin data, and it is not particularly sophisticated, for example, I did not calculate statistical significance, confidence intervals, or do any power analysis, or try any other Benford-like fraud detection methods.
Why publish this now?
This analysis was initially completed in Nov 2016, and was sent to election officials in Wisconsin. I then wrote half this article, shrugged, and moved on to other projects. I thought the results were interesting, but were not a smoking ‘numerical’ gun. Also, at the time, I thought the idea that direct vote tampering was attempted in the 2016 election didn’t rise above conspiracy-level conjecture. However, based on the mountain of evidence that has come to light regarding direct attempts by state actors to tamper with the 2016 election, specifically in Wisconsin, I thought I might as well put it out there.
Q&A
Q: Are you saying this shows Wisconsin election results were tampered with?
A: No, not at all, even assuming my modeling is correct, Benford’s provides faint, indirect, evidence, but it’s worth investigating (there have been plenty of investigation since this was written a year ago…)
Q: Wouldn’t state-sponsored ‘hackers’ be smart enough not to leave fingerprints in the data that can be teased out via numerical analysis, like Benford’s? I mean even crooked accountants have figured this out in 2016…
A: I don’t know, but it’s actually tricky to avoid detection if you don’t have full control over all the election data, and even professional accounts get caught this way all the time.
