A Methodological Summary of 2019 Election Polls

This is how each polling company estimates vote intentions.

Anthony B. Masters
Dec 5, 2019 · 8 min read

In order to assist public understanding of opinion polling in the 2019 General Election, this article will outline the basic method undertaken by the companies. This article is based on their public methods and data tables.

Survey mode, target population, weightings, turnout model, and vote intention questions are shown for each company.

The Common Method

Polling companies generally use either opt-in internet panels or computer-assisted telephone interviews. For internet panels, the respondent administers the questionnaire themselves, ticking boxes or clicking radio buttons. For telephone interviews, an interviewer asks the survey questions and responses are recorded.

Across both types of polls, demographic population totals — such as ONS estimates of the numbers of men and women — are used to set quota targets. A minimum sample size is specified, usually based on price and time. If a poll of 1,600 adults requires 790 men and 810 women, the survey continues until these quotas are ‘filled’.

Once the sample is gathered, weights are applied. Typical weights are for age, gender, and region.

All members of the sample are asked how likely they are to vote, and which party (or candidate) they intend to vote for. This is typically the first pair of questions asked in each vote intention poll.

If people are not decided, they may be ‘squeezed’: asked a follow-up question about how they might vote. Undecided respondents may be excluded or allocated in the reported figures.

From their sample of adults, companies derive a sample of voters.

Chances of turning out to vote applied to each respondent, usually based on their self-reported likelihood of coming out to vote. Some companies filter the responses. For example, they may only include those people who say they are 10/10 likely to vote in their headline estimate.

Others weigh the responses in a more graduated fashion, giving a score of 1 to people who say 10/10, 0.9 to those who say 9/10, and so on. One exception to this step is Kantar, who use turnout likelihood in their main weightings.

The vote intention shares of this weighted sample of voters is then published as the headline vote intention estimates.

When will this method be accurate?

In order for a pre-election poll to be accurate, three stringent conditions must be met — or produce errors which cancel in the headline estimates:

  1. Representative sampling: the vote intention shares must be the same as the population within each part of the quota and weightings.
  2. Correct turnout model: the likelihood of a respondent turning out to vote should be the same in the sample as the population.
  3. Stated vote intention is equal to the actual vote: Respondents should accurately state how they intend to vote, and there should be no change in their intentions between answering the survey and going to vote. This should be true in aggregate — differences should cancel.
Presciently, turnout models were identified as a potential weakness. (Image: NCRM)

The Companies

This section looks at the different methods undertaken by each company. The companies are: BMG Research, Deltapoll, ICM Unlimited, Ipsos MORI, Kantar, Opinium, Panelbase, Savanta ComRes, Survation, and YouGov. This article is also based on two articles from the British Polling Council.

BMG Research

Survey mode: Internet panel
Target population: Great Britain
Weightings: Age by gender, Government Office Region, Indices of Multiple Deprivation, household income, 2017 General Election vote recall, and 2016 EU referendum vote recall.
Turnout model: Factors have been applied to the Westminster vote intention figures to account for rates of electoral registration among key groups.
Vote intention question:

If a General Election were held today, which of the following parties would you vote for?

Respondents are only able to select from those parties with candidates confirmed as standing in their constituency. If they are undecided, respondents are offered the following ‘squeeze’ question:

And which of the following parties are you leaning towards most?

Substantive responses are included in the headline vote intention, whilst undecided people from that question are excluded.

Deltapoll

Survey mode: Internet panel
Target population: Great Britain
Weightings: Age by gender, social class, household tenure work status, Government Office Region, educational attainment, 2017 General Election vote recall, 2016 EU referendum vote recall, and political attention.
Turnout model: Self-reported non-voters are excluded.
Vote intention question:

In the general election to be held on 12th December, which party will you vote for?

Respondents are only shown parties that are standing in their constituency. Undecided voters are also excluded from the headline vote intention estimate.

ICM Unlimited

Survey mode: Internet panel
Target population: Great Britain
Weightings: age, gender, region, social grade, household tenure, work status, 2017 General Election vote recall, and 2016 EU referendum vote recall.
Turnout model: Turnout factor based on their stated likelihood to vote (0–10). As an example, those who say 7/10 are assigned a score of 0.7.
Vote intention question:

If there were a general election tomorrow, which party would you vote for?

ICM only offer respondents parties that are standing in their constituencies. Only those are say they are eligible to vote and expressed a vote intention are included in the headline estimate, weighted by self-reported turnout.

Ipsos MORI

Survey mode: Telephone (landlines and mobiles)
Target population: Great Britain
Weightings: ‘Weighted to match the profile of the population.’ These weights include: age, gender, region, educational attainment, ethnicity, and newspaper readership.
Turnout model: Only includes self-reported registered voters with turnout likelihood of 9/10 or 10/10, and those who say they always, usually or sometimes vote in General Elections.
Vote intention question:

How would you vote if there were a General Election tomorrow?

If people are undecided, they are then asked:

Which party are you most inclined to support?

People are then asked how they would vote if their preferred party was not standing. In constituencies where a participant’s first preference party were not standing, answers are reallocated to their second party preference. Undecided respondents and refusals are removed.

Kantar

Survey mode: Internet panel
Target population: Great Britain
Weightings: age by gender, gender by working status, 2017 General Election vote recall, 2016 EU referendum vote recall, age by education, region, and likelihood to vote in the next General Election.
Turnout model: Turnout likelihood is based on the respondent’s self-reported likelihood to vote (four options), their age, whether they voted in the last general election and whether they are registered. This turnout probability is used in the weightings.
Vote intention question: This question is asked the vote likelihood question, and was worded in 2015 as:

Still thinking about the next UK general election, which party, if any, will you vote for in your own constituency?

Respondents are asked a ‘squeeze’ question, and then asked:

Imagine that you were allowed to cast more than one vote in the election, which other party (or parties) might you consider voting for?

Respondents are allocated based on their preference standing in their constituency, and imputed where their intentions are still unknown.

Opinium

Survey mode: Internet panel
Target population: Great Britain
Weightings: ‘Nationally representative criteria’. These weights include: age, gender, region, working status, 2017 General Election vote recall, 2016 EU referendum vote recall.
Turnout model: Only those who self-reported as registered and eligible to vote with a turnout likelihood of 10/10 are included.
Vote intention question:

And these are the parties with candidates standing in your constituency. Which party will you vote for on the 12 December?

As the question states, only parties that are standing in the respondent’s constituency are shown. Whilst the survey is of UK adults, headline vote intention is presented for Great Britain.

Panelbase

Survey mode: Internet panel
Target population: Great Britain
Weightings: age, sex, region, 2017 General Election vote recall, and 2016 EU referendum vote recall.
Turnout model: Their turnout likelihood question is used as a filter, containing those who say their chance of voting is 8/10, 9/10 or 10/10.
Vote intention question:

Who do you currently intend to vote for?

Respondents are asked which party they would vote if their preferred party is not standing in their constituency. In that scenario, people are reallocated to that second preference. Undecided voters are excluded.

Savanta ComRes

Survey mode: Internet panel
Target population: Great Britain
Weightings: ‘Demographically representative of all GB adults’. These weightings include: age, social grade, work sector, region, 2017 General Election vote recall, and 2016 EU referendum vote recall.
Turnout model: Respondents are weighted by self-reported turnout likelihood, only including those saying they are 5/10 (or higher).
Vote intention question:

Looking ahead to the (UK) General Election on Thursday 12th December to elect MPs to Westminster, would you vote [a party] or for another party?

Other parties are: Brexit Party, Green, Some other party. Undecided voters are ‘squeezed’, with the following question:

If it were a legal requirement for you to vote at the General Election on Thursday 12th December, do you think you would probably vote [a party] or for some other party?

The BPC article states Savanta ComRes respondents will only be able to select parties with standing candidates in polls (from 2nd December).

Survation

Survey mode: Telephone (landlines and mobiles)
Target population: United Kingdom
Weightings: Age, gender, region, 2017 General Election vote recall, 2016 EU referendum vote recall, and 2019 European Parliament election vote recall.
Turnout model: Turnout factor based on their stated likelihood to vote (0–10). As an example, those who say 7/10 are assigned a score of 0.7.
Vote intention question:

Now thinking about your own constituency and the candidates likely to stand, if the General Election were tomorrow, which party would you vote for?

Respondents are being invited to state a vote intention from only among the named candidates standing in their constituency. Undecided respondents, or those who refused to answer, are removed.

This is only referring to Survation’s latest vote intention polls on behalf of Good Morning Britain. The company has also conducted polls through an internet panel.

YouGov

Survey mode: Internet panel
Target population: Great Britain
Weightings: age by gender by education, political attention, social grade, 2017 General Election vote recall by region, and 2016 EU referendum vote recall.
Turnout model: Turnout factor based on their stated likelihood to vote (0–10), with an applied down-weight for 2017 non-voters.
Vote intention question:

There will be a UK general election on 12 December 2019. The following candidates and parties will stand in your constituency. How do you intend to vote in the upcoming election?

(Prior to candidates being confirmed, a different wording was used.) Respondents are offered the candidate and party names in their constituency. Those who said they would not vote, undecided, or refused to answer are excluded.


These methodological differences, with natural variation from sample to sample, mean companies produce different vote intention estimates. Similar descriptions of weights can disguise choices, such as alternate age groupings and vote recall (earliest available recall versus recall in the latest survey).

    Anthony B. Masters

    Written by

    This blog looks at the use of statistics in British political debates, and is written by RSS Statistical Ambassador @anthonybmasters.

    Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
    Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
    Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade