Australia is about to get less religious.
The Australia Census changed the design of the religion question. It will cause an increase in people answering ‘no religion’. I’m partially responsible.
Most Australians have just heard the 2016 Australian Census is approaching. In 2011 my buddy Tom and I criticised the design of the religion question. The question looked like this:
What’s wrong with this?
We argued they there was 2 ways the question leads to overstate religion.
1 - Bias through framing by listing religions first.
There are many Australian’s I would say are historically religious. For example, I have lots of friend who where baptised, maybe had a handful of religious education clasess at school, but that’s basically where religion in their lives ended. Many of those I suspect would see the faith they are baptised in, mark that box before seeing ‘no religion’. Or if they do see ‘no religion’ option they’ve seen a religion they kinda sorta maybe identify with first and it becomes the default.
2 - Obscured and Hidden.
The question is non-compulsory. So with ‘no religion’ pushed down below under the rest I think it’s possible a good number of non-religious people don’t see the option and move on.
Citizens who don’t answer are not included in the count as it’s (rightly) assumed that whether they are religious or not is unknown. But his implicitly distributes their answer with those that did answer. I bet there is a larger ‘no religion’ in ‘not stated' than those that did answer.
Any evidence?
Well, it’s hard to know for sure if the measurement is off. That’s what the census is there for, actually. But in 2011 ‘no religion’ climbed to the second most selected option. This represents a continued rise over recent years.
But what rose even more is the number of people who did not answer. (I hesitate to say ‘chose’ not to answer given what I have just said above, though it is true for some). In fact the ‘not stated’ or ‘missing’ has been the fastest growing segment over the past several censuses. That’s not hard evidence. But if feels fishy.
What’s better?
We didn’t stop with a complaint. We proposed a number of alternative designs that may reduce accidental bias. This lead to a good online discussion with some experts in survey and form design. Options included:
Look closely and you’ll see us playing with the location of ‘no religion’. We also played around with the difference between ‘no religion’ and ‘not religious’, as I suspected that would have an impact. We even took an (ironically unscientific) poll of our alternatives.
After some consultation we actually landed on a different preferred option. We separated out the question into two parts. This is consistent with other questions on the census.
Did anything happen?
We submitted our views to the Australia Bureau of Statistics (ABS) who administer the Census. They received a lot of correspondence on this question. We where contacted by several people also submitted based off our posts. To be honest, I didn’t really expect a change.
We’ll blow me down and call me Shorty. The ABS not only listened to us. Their response looked like my was referring directly to us.
“During the 2016 Census topic review process, many submissions recommended changes to the Religious affiliation question due to perceived bias in the question format and consequent potential underestimates of the number of people who stated they had no religion.”
Not only did we get noticed, they CHANGED THE QUESTION!!!
“After user consultation and testing, the ABS has decided to move the No religion response category to be the first response category in the question, so it will be more consistent with other questions and the order of their response categories.”
BOOM!!! BOOM!!! BOOM!!!
Here is what the question will look like.
No religion is moved to the top and spaced so it is subtlety separate to the listed religions. It’s not our preferred option, but nearly identical to one of our proposed alternatives; better even because the difference is the subtle spacing. Importantly it addresses both the criticisms from the start of this article.
So what will it mean?
Quite bit, I reckon. So I’m going to go on the record and make a carelessly high prediction.
I bet that the number of people stating no religion will jump by 5% from the 2011 numbers.
Some of this will be the natural trend – which is how the mass media will probably report it – but the majority will be down to the design of the question.
Why am I making a fuss?
The measurement matters. We should get it right. It’s the Census. That is the point. The ABS itself says that it’s ‘our moment to pause and make a difference”. I don’t think the question was correctly measuring what it sought to measure. I’m not an activist. I’m and ACCURATEist. I think this will improve the accuracy.
The obvious reason accuracy is important is that the government makes decisions on funding based on these numbers. Schools, chaplains, charities and other things you’d notice everyday use the census data as an input in dividing tax-dollars.
But I think a more important reason is for how we think of ourselves as a nation. It’s my belief that most Australian’s think the dominant culture to be something like ‘light Christianity.’ This effects all sorts of debates. Perhaps most importantly it effects the prediction of what politicians, journalist and business leaders expect the public to think. This will effect what they say and the decisions they make.
If my prediction is correct, I think this vote will shift the dominant cultural force to something more akin to ‘lightly agnostic’. Or at least closer to that.
This will take awhile to filter through. But ultimately I think it will change they way we think about who, on mass, we are. This, in turn, will change what we do.
The Census is on August 9.
Please be thoughtful and accurate.
FOLLOW UP EDIT
There was actually huge technical problems with the census that may compromise the results. However, the reason the technical problem could break the census is because it was now mostly online.
I did it on an iPhone 6. And the question actually looks different because no religion appears above the fold and half the answers below the fold. The space between no-religion and the other answers is also larger than on the paper version.
Had the original format stood the no-religion would have been even more hidden on mobile. The new design makes no-religion stand out to the point that it feels like a default to me. I’d say that people would need to be much more strongly religious to select the option than on the previous paper form.
