Here we go again, another day, another survey

We’ve been here before, polls on Muslims, headlines, debate. This time, it’s Policy Exchange’s poll, conducted by ICM Unlimited (same polling company involved in Trevor Phillip’s debacle).

I have lots of “skin in the game”, as a Lebanese thinker and statistician Nassim Nicholas Taleb might say. First, My PhD is in the field of religion and social sciences, so I have lot to say about approaches to studying human society. Second, I’m a British Muslim, so this debate is ultimately a debate about me and my own role in Britain.

There’s a few reasons why I’m sceptical not just of this survey, but similar one’s issued regularly.

1) Accuracy

Any survey has a degree of error, a margin by which the results may be wrong. There is an ever present risk of extrapolating from a few hundred/thousand up to a few million. We accept this risk for the benefits of what it can tell us. We accept polls and survey are indicative, but not absolute.

Of course polls can get things wrong. Three big examples. The General Election in Britain 2015 (which failed to predict the confident Conservative majority), the Brexit Referendum in 2016, and of course, the US elections. These are huge upsets, unpredicted by the polling companies.

In all these examples, statisticians and pollsters have had decades to perfect their techniques in a relatively straight forward arena (they are straightforward yes/no or candidate A/candidate B choices) which are then played out in real life so as to assess the accuracy. Pollsters have had years to see what identity markers are significant and which can be ignored. It’s big picture, trying to assess the entire country, and that is where these polls work best — on general trends rather than smaller cases. Basically, there is no field better suited to the survey than elections, but we know, of course there is still a significant degree of error. They still get things wrong.

Imagine now a new context. No longer are survey simplistic and straightforward candidate A/candidate B choices, but hugely complex ones with shades of interpretation and caveats (such as “do you support sharia in Britain?”). Next, we’re no longer trying to gain a picture of the entire country, but a minority within it (Muslims), for whom the significant markers of identity are still unknown (is a Bangladeshi heritage Muslim more likely to vote Labour than a Arab heritage Muslim? Who knows!?). The possible margin of error neccesarily increases.

I had this discussion yesterday on Twitter, which ended with me being accused of a biased Islamist commentator, but my points remain the same. The ICM poll for Trevor Phillips surveyed 11 Arabs, who represent about 6% of the Muslim population in Britain, but they only represented about 1% of the sample. So the they are weighted to balance out the possible results. If these 11 Arabs are representative of the wider Arab population, all good, if not, then the entire survey is skewed by a small margin. Do this again and again for each potential identity marker (ethnicity, gender, socio-economic profile) and you have lots of small margins of error. Polls just aren’t a very good way of knowing much about small and geographically dispersed groups on complex issues.

Do these invalidate the results? Not entirely. We can still glean information from them, but with a heavy dose of scepticism. But ultimately, when speaking about complex issues within minority groups, surveying is rarely accurate. The results to similar questions between ICM’s two polls this year were hugely varied (for example, on whether the respondent would report a potential terrorist to the police, nearly a twenty point difference). Same poll, same pollsters, same year, same respondents, same questions — wildly different results.

2) Interpretation

I made this point in my article about Trevor Phillip’s poll. Dr Maria Sobolewska stresses that British Muslim responses to controversial or complex questions in surveys are “mostly an artefact of what they get asked and that the non-Muslims answer similar questions in a similar fashion”. http://blog.policy.manchester.ac.uk/featured/2014/08/can-we-ever-estimate-how-many-british-muslims-will-become-islamic-extremists/ Dr Stephen Jones, another researcher exasperated by polling companies says the following: -

“The question that really annoys me is about whether or not Muslims support ‘Sharia’. This is routinely used to imply that large numbers of Muslims in Britain support the replacement of British democracy with theocracy. The problem here, which religiously illiterate polling companies don’t recognise, is that for many Muslims Sharia means something akin to ‘God’s path’, and so they don’t want to reject the notion entirely. That doesn’t mean, though, they have any interest in the kind of political system advocated by groups like Hizb ut-Tahrir.”

So the three key points. One, responses to complex questions are largely a result of the way the question is asked. Two, there is usually not a significant difference between responses by Muslims and non-Muslims in polls regarding Islam. Three, questions can often miss the way words mean different things to different respondents (sharia being a key example here).

3) Purpose

So what is the purpose of this newest poll by ICM? Well, let’s not shy away from the fact it was completed by a neoconservative think-tank, Policy Exchange. Surveys about Muslims have consistently been used to essentialise and other Muslims, to section them off as a “nation within a nation” (to quote Trevor Phillips). Surveys depersonalise. It means you do not have to deal with the individual Muslim, and their individual views, but can now treat them as a communal bloc, that can be held communally responsible for the views of other Muslims, and who can be subject to communal discrimination. It leads to views such as the following, made by fellow neocon and think-tanker Douglas Murray: -

Conditions for Muslims in Europe must be made harder across the board: Europe must look like a less attractive proposition. Where a mosque has become a centre of hate it should be closed and pulled down. If that means that some Muslims don’t have a mosque to go to, then they’ll just have to realise that they aren’t owed one. Grievances become ever-more pronounced the more they are flattered and the more they are paid attention to. So don’t flatter them.

This type of communal oppression only works if Muslims are amalgamated into one whole, and this is exactly what surveys keep doing.

For newspapers, politicians, journalists and similar who desire to know what Muslims think — go out and meet some, you’ll experience a world of diversity and difference, and everyday life far removed and much richer than any poll could ever indicate. For those however who desire to keep Muslims at arms length and forced into a manageable “block”, then there are plenty of surveys out there for you.