Ward level results from the EU referendum

My estimates hold up reasonably well

This morning, Martin Rosenbaum, the BBC’s Freedom of Information Act expert, published some ward-level results from last year’s EU referendum.

Some local authorities had already made ward-level information available. Other local authorities merged ballot papers from different wards before counting even began. For other local authorities still, it was necessary to submit, and aggressively follow up on, requests for further information.

The release includes information from 1305 wards across 84 counting areas across the UK. By matching wards to parliamentary constituencies, it’s possible to calculate known results for 113 seats.

(Pedantry corner: in some local authority areas the postal votes were not allocated to any ward. The aggregated ward-level results will therefore not match the local authority wide totals. This makes almost no difference to the percentages won by Leave and Remain).

This in turn allows me to assess how good my estimates of the referendum outcome by Westminster constituency are.

Let’s begin with a simple scatter-plot. Here, I’ve plotted my estimates on the horizontal axis, and the known results from the FOIA-d data on the vertical axis. The dotted line indicates the line of perfect fit.

As the figure shows, the correlation between my estimates and the known results is very high. A correlation of greater than 0.95 would normally, as my music teacher used to say, be close enough for jazz.

In this instance, however, it’s worth examining more closely some of the disparities between my estimates and the known results. A high correlation doesn’t always imply that the errors involved are negligible. Here’s a histogram of the errors in my estimates:

The good news is that the estimates are unbiased — the error across all constituencies averages out to zero, or very close to zero. That’s to be expected. The way my estimates worked, I started with some demographic relationships, projected these on to small units, and ensured that those projections “added up to” the local authority results. That last step should have made for unbiased estimates.

In terms of the absolute error, we can say that:

• 47 of 113 seats (42%) were within one percentage point of the true result
• 25 seats (22%) were between one and two percentage points off the result
• 21 seats (19%) were between two and three percentage points off the result
• 20 seats (18%) were more than three percentage points off the result

This last category included some fairly serious misses in Birmingham. (There’s clearly a bigger story to be told about voting behaviour in Birmingham, but it’s not for me to tell it).

It’s clearly useful to know something about average errors, but in the particular context of a referendum, we often pay disproportionate attention to errors either side of 50%.

An error of two percentage points may matter very little when a seat is estimated to have voted 70% to Leave. It matters a great deal when the estimate is closer to 50:50.

With that in mind, here’s my list of constituencies which I had estimated voted Remain, which in fact voted Leave:

• Croydon Central: I estimated 49% Leave; results say 50.3% Leave
• Portsmouth South: I estimated 49% Leave; results say 51.8% Leave
• Shipley: I estimated 48.5% Leave; results say 52.2% Leave
• Sutton Coldfield: I estimated 46.7% Leave; results say 51.7% Leave

And those that estimated to have voted Leave which in fact voted Remain:

• Harrow East: I estimated 50.5% Leave; results say 47.5% Leave
• Wallasey: I estimated 53% Leave; results say 49.9%(!) Leave

It’s because of errors like these that I have on occasion felt awkward about the way in which my estimates have been used to criticise named MPs for ignoring the will of their constituents — particularly when these MPs are called traitors or enemies of democracy. I’ve updated my spreadsheet with known results where these are available. You should feel free to continue to use these estimates for the remaining five hundred or so seats for which we do not have complete results — but please do bear in mind the qualifications noted above.