Oops, I did it again…

A tale of two forecasts

In 2015, I was involved in an election forecast. It didn't go well, largely because the polls were (collectively) wrong. The total seat error on the forecast was 110 seats.

I also forecast this election. I assumed that, though they would not be as wrong as in 2015, the pollsters would continue to over-estimate Labour, as they had done in all elections since 1983.

That turned out to be the wrong assumption. The polls, on average, under-estimated Labour, and over-estimated the Conservatives.

As a result, the error on my forecasts was… 120 seats, ten more than last time.

Had I not made any assumptions about (pollsters') error, I would have got closer (total seat error of 98), and would have been less confident about a Conservative majority (90%, compared to 95%), but I would still have had a very high error.

The good news (if there is any) is that even with this assumption I didn't rule out a hung parliament (though I assigned it a very low probability of 5%), and the results were within the 95% forecast interval for the Conservatives, Lib Dems, and SNP (but not for Labour).

I'll post a fuller evaluation of the forecast later on. I'm glad to see some of the ideas I used were used (to better effect) by others (see, for example, Kiko Llaneras' prediction here), but I'll need to test whether the seat-by-seat predictions offered any advantage over uniform swing.

One of last night's great winners — the YouGov MRP team — did provide an exceptionally accurate forecast, which nailed the patterns of swing that we saw. I want to record my admiration for Ben Lauderdale and Jack Blumenau (and for Doug and Delia, who I don't know). Their seat predictions were incredibly courageous, and led to them being pretty roundly panned. They became even more courageous when YouGov's national polling team made a last minute decision on the reallocation of don't knows, causing divergence between the MRP and standard figures. Their total seat error is just 36. That's an astonishingly good performance.

The other great winner of last night was Survation, who are in the running to be GE2017’s most accurate polling company. I was working with Survation on a seat-by-seat model, using their online data rather than their telephone data. The model I built for them together with Zsolt Kiss turned out to be substantially more accurate (at least for the Conservative figure) than my public forecast. I don't like to mention it (because having more than one forecast is the refuge of the scoundrel), but it does suggest that MRP with good online data can be pretty accurate (even if, on this score, Ashcroft let the side down). But everyone at Survation can be very pleased with themselves.

Given that at least some people mentioned the possibility of a hung parliament, it's possible that the UK will be prepared for the political negotiations that will result. The timing is unfortunate, to say the least. This election has already taken away time from the Article 50 negotiation process. So too will a tricky government formation process. Anyone who mentioned the possibility of a hung parliament has done themselves and a favour — and might also have done the country a favour.

Show your support

Clapping shows how much you appreciated Chris Hanretty’s story.