What’s going on with this election, then?

UPDATE: Naturally, as soon as I published the article below it was overtaken by the resumption of campaigning and a new lurch in the polls. As we’re now only hours away from election day, I’m going to put a few final thoughts here, to add to the original article below.
 
The general rule in election campaigns, I confidently said, is that things don’t change very much during them. This one has seemed likely to prove all of that wrong… except I suspect it isn’t. I wouldn’t be at all surprised by a result much closer to the pre-election polls than current polls suggest. The fundamental considerations working in favour of, and against, each party haven’t completely vanished, after all. But as it is, all polls are showing the gap between Labour and the Conservatives narrowing, with some suggesting it is very small indeed. But that said, let’s note how enormously high the Conservatives are still polling — at or above the voter share enjoyed by Labour in 1997!
 
Into this heady mix we can add YouGov’s already somewhat notorious model, which attributes certain characteristics to seats and then plugs national polling data into them, in the expectation that if they’ve mapped the characteristics of the voters in the poll correctly, the resulting seats forecast will be accurate. In its approach it’s not unlike the exit poll model, which takes a survey outside a relatively small number of polling stations and then maps the results across the country using a model of the characteristics of the different seats. But, for all the reasons I went into last time, polling data may not prove as reliable as data from actual voters who have just voted…
 
Although there has been a trend across all polls showing an upward drift for Labour, unlike in 2015 the polls have diverged rather than clustered — some companies are going to come out of it much better than others, but that will at least tell us which methodological adjustments have been best for countering the problems of two years ago. Modelling turnout appears to be key: putting it crudely, young non-voters appear to be reporting themselves more likely to vote than ever, hence polls that rely on self-reported turnout predictions give Labour the biggest scores; if this comes to pass in the election proper, it will be a colossal break from previous behaviour. Probably, therefore, it won’t happen; but voting among the young has slumped in the last 20 years relative to turnout as a whole, so it’s not impossible that it might clamber back towards more traditional levels.
 
Also, I don’t buy the line that the media have fallen into the trap of looking too closely at polls again. Unlike in 2015 the polls have changed substantially, and in a variety of ways — there’s been a good case to think something substantial has been happening. That said, the BBC’s focus on a Labour-SNP deal in particular has been deeply stupid (except under the most optimistic projections for Labour, the numbers weren’t there for it), and will probably have influenced voting to Labour’s disadvantage as we saw in 2015.
 
Another reason to avoid getting excited about Labour’s prospects, even assuming their surge has been real rather than some phenomenon within the polls such as shifting biases within samples, is the likelihood of swing back to the Tories for multiple reasons: they will have gone heavy on their Facebook ads in the last week of the campaign; the timing is such that people’s enthusiasm for Labour, such as it is, could wear off just before polling day, as with Clegg in 2010; and the surge has made the prospect of Corbyn winning with a ‘coalition of chaos’ actually seem not ridiculous — and we know both elements of that are problematic for Labour at the ballot box
 
That said, even if Labour’s support doesn’t reach dizzying heights, it does appear that the Prime Minister has suffered politically for her conduct of the campaign. How much she has suffered may depend on the final size of the majority she secures, but it could even amount to a Gordon Brown-style collapse in personal credibility. I don’t for a moment think her party will let her fight another election as leader after the mess she’s made of this one — and she’ll probably realise as much and hand over to a successor in four years’ time. 
 
For now though, if Labour do indeed lose seats overall, it will put them in territory from which past experience shows it’s impossible to fight back to win a majority in a single election, barring a 1997-scale landslide. But let’s just imagine a scenario: a disastrous cliff-edge Brexit that people take against, with May and her long-in-office Tories taking the blame; plus a credible Labour leader to soak up the disaffected voters… there’s a lot of moving parts there, but if they all turn the right way the next election could be genuinely up for grabs with a big swing. The next choice of Labour leader will matter a lot.
 
But for now, where does all this leave us? In 2015, I looked at the vast array of data that had been gathered and forecasts that had been constructed, all pointing the same way, and concluded that they must surely be right. That was my error. This time, I’m much more interested in the soft intelligence from the ground (still looking grim for Labour, despite the surge in headline polling), indicators beyond voting intention polls, and structural factors that could cause the VI polls to be off (the latter two explored in my original post below, and possibly even underplayed according to that Labour Uncut article). All of those factors still point towards a bad results for Labour in terms of seats, even if they do manage to pile up a decent vote share, as losing vote shares go. To the extent that I’m willing to stick my neck out and make a prediction, it’s for a bad-ish result for Labour (low 30s vote share or below, sub-200 seats, possibly by quite a lot), and another bad night for the opinion pollsters. That said, none of the non-VI-based forecasts I’ve seen have Labour doing quite that badly. 
 
Bu finally, I’ll caution against my own caution. Recently in British — and sometimes global — politics, the trick to predicting the future correctly has been to look for the extreme, unthinkable result that seemed just beyond the horizon of what was truly possible… but turned out to be very possible indeed. At the start of the campaign, a Labour collapse to sub-25% and sub-150 seats seemed to be the unthinkable-but-actually-thinkable result. But what if the unimaginable result we should have been looking for was a decent one for Labour, improving on its 2015 result to a mid-30s vote share and somewhere north of 250 seats? It seems hellish unlikely, but we all know what that counts for these days…

Original article starts here…
The general orthodoxy is that general election campaigns don’t change very much, and parties generally poll much the same at the end of a campaign as at the start (barring, pre-2010, usually a small uptick for the Lib Dems). There are exceptions, of course: the huge spike in Lib Dem support during 2010’s ‘Cleggmania’ meant that the pattern was broken thoroughly in that election. And in 2015, the polls appeared to show the race tightening slightly over the campaign, having been tight to begin with — though we now know there were systematic errors in those polls.
 
Even so, this election has been unusual. After it was called, support for Labour and the Conservatives increased appreciably, while UKIP, the Lib Dems and the Greens all suffered drops in support. In a second phase of polling changes, the gap between Labour and Conservatives appeared to narrow, due to a further increase in the Labour vote. There could well be more twists in the remaining fortnight of the campaign: the dreadful events in Manchester have prompted a halt to campaigning, so it’s uncertain what the state of the parties will be when it resumes. This article is my attempt to piece together what I make of all this. 

A polling miss?
The outlook for the Conservatives is not difficult to divine in outline: they will hoover up many ex-UKIP votes and win comfortably. But for Labour, things are far less clear: their results could be anywhere from a bad defeat to a catastrophic one. Accordingly, I particularly want to look at Labour’s fortunes here.

When I first started planning this piece early in the campaign, I wanted to wonder if we might be heading for another polling miss, and an even bleaker outlook for Labour than there then seemed to be. It was starting to look to me like the polls might be over-stating Labour a bit and, while other factors have since come into play, this still seems very possible. Voting intention (VI) polls showed Labour at 28–31% (a slight uptick), and the Conservatives in the mid-to-high 40s (also an uptick, drawing on UKIP’s collapse). The first thing to say is that there is more reason to doubt the Labour side of this picture than the Tory one: in 2015, as historically, it was over-stating the Labour vote that was a particular problem. 
 
The voting intention polls were wrong two years ago, and the key to forecasting the result accurately was to look at other indicators. Number Cruncher Politics called it right by looking at local election results and favourability ratings for parties on leadership and economic competence.
 
Now, I can’t use that data with Matt Singh’s sophistication, but a glance at those indicators gives me pause. Projected national vote share from this month’s local elections gives Labour 28% and the Conservatives 39%. The pattern of previous general elections that closely followed local elections suggests that the party in the lead will exceed their performance in the locals, and the second place party under-shoot theirs — as Stephen Bush puts it, Labour’s 28% could be their ceiling for the general election, and the Conservatives’ 39% is probably their floor. With Labour’s notional local election score at the lower end of their early-campaign VI poll performance, that was one indication that their eventual vote share could be lower than the polls were suggesting. 
 
The other indicators are equally troubling: the gulf between May and Corbyn in people’s perceptions of competence is huge, and the perceived economic competence gap between the parties is also massive. Leo Barasi modelled the leadership scores along the same lines as Matt Singh did in 2015, and predicted a Conservative lead over Labour of 15–18% — less than the polls then suggested, but still consistent with a monumental defeat for Labour. 
 
My initial conclusion was that Labour losing a third of their seats, reducing them to a touch over 150, and taking a national vote share between 25% and 28%, seemed a credible and conservative prediction. But the lesson from recent politics appears to be that the result can always be more extreme than one expects: we didn’t just have close misses with Brexit and Trump, but they actually happened; the Conservatives weren’t just competitive in the Tees and West Midlands mayoral races, but won them; the SNP didn’t just take a lot of seats in Scotland, but nearly all of them; and of course Corbyn did in fact win the Labour leadership. Labour dropping below 150 seats and/or 25% of the vote therefore seemed a plausible outcome for the general election in June based on the early indicators. A result of, say, Lab 24 and Con 42 would be in line with Barasi’s suggested 18-point gap (and would allow the smaller parties to pick up a few more percentage points between them than the polls currently suggest). 
 
The surge — part 1
However, what started as a slight uptick for Labour appeared to become more consistent and solid, with the party regularly polling at around or above 30%. Albeit an overstatement, this started to get called a ‘surge’. However, a combination of polling error and bad vote distribution could mean this does not signal any great hope for Labour.
 
First, let’s consider polling error. There is plenty of reason to suspect it is happening, over and above those already outlined — which still apply. Firstly, the ‘surge’ seemed to arise from ‘don’t knows’ heading back to Labour. But this Labour vote is potentially soft — will they turn out to vote at all? And if they do, could there be some genuine late swing, with the Conservatives attacking Labour in forensically targeted Facebook ads, as in 2015? You can bet they’ll be trying it.
 
It also appears to be the case that high numbers of Labour-leaners didn’t vote in 2015–16 — now, Corbyn’s entire pitch was to get non-voters voting and win that way… but every time it’s been tried it’s failed. Including by Ed Miliband. So that’s another indicator that Labour’s surge might overstate its eventual score.
 
Secondly, let’s consider vote distribution. It may be that Labour’s core vote is being shored up in safe areas, while it is doing disproportionately badly in the key seats it needs to defend. This could translate into a small drop in vote share from 2015, but a huge drop in seats. Corbyn’s campaigning pattern, well away from the Labour seats with 5–8,000 majorities that are the key battleground, could support this view — is he focusing on strengthening his position on June 9th by shoring up the vote share? This could be the case — but another explanation might be that he’s just being kept away from the really tough seats so he doesn’t screw things up by association for the incumbent.
 
Either way, some people are monitoring intelligence on the doorstep and piecing it together to construct a picture of massive seat losses. This all feels a bit finger-in-the-air, but that doesn’t mean we should dismiss it altogether. On the ground, Labour canvassers knew the 2015 election was lost long before the national media confirmed it — the voters they spoke to simply weren’t behaving as the polls predicted. A lesson from 2015 is to pay attention to this intelligence, soft and fragmented though it is.
 
A further aspect of vote distribution is that UKIP are not standing in numerous key seats. It’s one thing for the Conservatives to appropriate large chunks of the UKIP vote in a genuine contest, but another again for ex-UKIP voters not even to have the option this time. This could make a difference in several dozen seats, which will depress Labour’s final tally significantly if it happens.
 
At this stage of the campaign, up to, say, the Conservatives’ manifesto launch, we therefore seemed to be at risk of a double-whammy polling miss: a VI polling score in the low 30s could be out by a few percentage points due to the ‘surge’ being soft and not sticking on election day; and a few more due to the ‘baked-in’ polling errors, if indeed they are present. In that scenario a calamitous, sub-25% showing could still be on the cards. 
 
The surge — part 2
Since, and perhaps because of, the Conservatives’ manifesto launch, Labour’s support appears to have spiked up to 34–35% across multiple polls, and the Conservatives’ support nudged down. Given that this is a trend across multiple polling companies rather than an isolated poll, it could be genuine — if so, the key question is whether it will last.
 
One widely-touted explanation for this has been the Conservatives’ proposals on social care, which would see some people paying more than under current arrangements — and also informed voters more broadly that social care is not free to use like the NHS, which they hadn’t realised and therefore misinterpret as a new set of charges on the elderly. A complementary explanation for this and the earlier stage of the ‘surge’ may be the further collapse of Lib Dem support, right back to 2015 levels and eliminating all the ground apparently gained during their 2015–17 ‘fightback’ — even in the urban seats they had hopes of, word is they are struggling. 
 
But it’s a superficial explanation. We’re only talking about a small number of polls, and there’s ample time for a sharp bounce to fade away as quickly as it arrived without us ever being able to tell if it would have resulted in a genuine shift of votes. Theresa May’s u-turn on the social care proposals may help this to happen — the drop in Conservative support appeared to come with an increase in ‘don’t know’s, who could now swing back in behind the Tories again.
 
But if Labour’s support remains close to this level, even if there’s a polling miss one would expect them to land in the high-20s, rather than the mid-20s as at the start of the campaign.
 
Scenarios for Labour
We can therefore sketch out three broad scenarios for Labour:
1) Their polling gains fade or were illusory, the polls were overstating them to begin with and they do disproportionately badly in marginal constituencies: this remains possible, and could lead to something like a 24% vote share and a collapse below 150 seats
2) Their polling gains are genuine but modest, and over-stated by polls, but the unexpectedly high vote share is badly distributed, leading to a collapse in seats anyway to around 150–170, on say 27–30% of the vote
3) The polling gains are genuine, and big enough (combined with local campaigning) to hold back some of the tide and limit Labour to a merely bad defeat — maybe a vote share in the low 30s, and 180–200 seats.

As has been observed, the relationship between vote share and seats isn’t straightforward, so the final result could be a pick ’n’ mix of the above — in extremis including a high-ish vote share (high 20s or more) and low-ish seat total (140 or so). It’s possible.
 
One of the few things we can say for certain is that there will be a mighty, possibly final, battle for the Labour Party after this election. The pattern of the results could matter greatly in determining its outcome.
 
But for now, when campaigning gets underway again we will have to see where the polls sit — and there may just be time for one last swing, real or phantom. Beyond that, we’ll need to await the one poll that most people still have confidence in — the exit poll at 10pm on election day. Which, er, under-stated the Conservatives a bit in 2015 — so actually, don’t lean too heavily on that either.

One clap, two clap, three clap, forty?

By clapping more or less, you can signal to us which stories really stand out.