Is Labour’s lack of trust in research methods making them less effective?
The new politics of the Labour party has the laudable aim of trying to bring together the distributed knowledge of Labour members to create a better political platform. But I’m slightly worried by the first attempts by the new leadership to conduct this democratic process. So far this has been research (rather than a deliberative process where the process itself helps shape minds) and yet it is not using well-worn research methods that would be more effective at doing it.
The Syria consultation
Most recently the party tried to consult its members on the government’s proposal to expand its aerial campaign against the so-called Islamic State into Syria. They did this by sending email with a link to a web form asking for people’s names, emails, and by giving them an open text box to express their views on the subject. After receiving more responses than they expected, the party looked at a number of these responses (how many exactly is subject to some disagreement) sorted them into those who supported, opposed and were unsure on the subject.
The problem was that they hadn't decided what would be done with the responses when they came in or who would read them. This meant that the form wasn't created with considerations of resource constraints in mind. There is no point in collecting so much data (by asking an open question) if no one is going to have the ability to analyse it. If they asked a closed question restricting the options to picking between support, oppose and unsure then they could have realistically analysed all of the results. It would have looked an awful lot like a poll.
People who respond to surveys are weird
The Syria consultation form had no questions to check how representative the people who filled it in were. It didn't ask their age, gender, or what they thought about other issues. This is important because by and large people who respond to surveys are not like everybody else. They will differ in many key ways from the group of people you actually want to find out the opinions of and unless you take the fact that they’re weird into account then your results will be less useful.
For example, what if the people who filled in your survey were more wealthy than the group of interest as a whole? You would want to put less weight on what wealthy people thought in their replies and more on what less wealthy people thought so that the balance of responses more closely reflected the group of interest. If wealthy people have a different response to less wealthy people then this weighting process could completely change your outcome. If you did not know how wealthy the people are then you wouldn't know that they differ and wouldn't know your answer is unrepresentative.
This weighting is what pollsters do that makes their work more valuable than simply asking a similar sized group on twitter or over email. Sometimes they get this weighting wrong (as happened with the recent general election) but they will be more accurate than an un-weighted sample more often than not.
People who talk to politicians are weird
Thinking about issues of representativeness points to a bigger problem with the Labour party’s consultative process. The people who take part in meetings and who respond to surveys are different from everyone else. They are not like the rest of people who support and vote Labour and they are not like the rest of the public. Professional polling on the same Syria question showed that Labour members differed from these other groups quite substantially.
If you want to persuade the whole public to elect a Labour government then you need to talk to people who are representative of the whole public. Politicians cannot do this simply by speaking to more people. No matter how many tours of the country, town-hall meetings or conference workshops you hold will never get a sense of whether the message will be appealing to the general public. Every single person who you meet is weird. The people we encounter are selected by the things we have in common. In politics it is their interest in politics (particularly party politics) that makes them weird and not representative of the ordinary public. Their interest will mean they are both more partisan and more engaged with the facts of the argument than most of the people that need to be persuaded.
This is why it makes sense to speak to people who are representative of the group you want to persuade and test your messages on them: you can see if they are persuaded and you can ask them if not, why not. That would look an awful lot like a focus group.
Both focus groups and polling have been used by politicians of all stripes for a long time for a reason. These methods have a bad name from their use under New Labour where they were seen as part of an inauthentic professionalised political class. However, they are a crucial part of the modern democratic process. Using these tools wouldn't compromise the new leaderships vision if used well it would only make them more effective.