When Science Reporting Goes Wrong –It’s not always the journalist’s or the scientist’s fault.
It would appear that my brain these days only let’s me focus on small tasks such as writing blog posts, commenting on facebook and getting irritated. (I fell off my bike on Friday (4/12/15) and since then I’m trying to regain control over how large a span of time I can focus –it’s getting better as you can see).
Anyway… The story is not about me!
The story is about a good paper, which made really sensible claims, but got misleadingly reported in the press thereby, highlighting one of the ills of scientific writing and reporting of today.
What happened?
Yesterday morning, a colleague posted a recent paper published by the group of neurolinguist David Poeppel. This paper reports really cool evidence showing that, while processing language, our brains also process its hidden governing structure (hierarchy, as it is commonly referred to in the literature).
In other words, when your brain reads, or hears “New plans gave hope”, it knows that “new plans” go together and that “gave hope” go together and that once put one after the other “new plans” do something, namely giving hope.
When the researchers analysed the brain activity of people processing sentences such as these, they found that, besides reacting directly to the presentation of each word (normal: stimuli comes, something happens in the brain), the brain also processed this added layer of structure I couched in words earlier. Very schematically, it looks something like that:
This is a cool finding, and it’s certainly novel. But all it shows is that the brain figures out the hidden structure governing this sentence. For you this might be news or totally normal or maybe you had no idea about it before. But that is all the paper shows. And all that the paper is designed to show and, basically, all that the researchers claim in the original paper.
What came out in the press…
However, shortly after posting the paper on his facebook wall, my colleague posted a press release accompanying this paper. The original NYU press, endorsed by James Devitt, release can be found here.
In short (i.e., from the title), the press release claims that:
Chomsky Was Right […] We Do Have A Grammar in Our Head.
BAAM! Just like that!
What’s even more annoying, is that this claim was repeated in other scientific media outlets (e.g., article in Medical Daily and here). Since Chomsky is such a big name in linguistics this also went “viral” on social media. I think some heated discussion is probably still going on on Reddit (82 comments in 2 days that’s viral for such a topic).
So, in a blog post I wrote yesterday, I summarised (and grossly simplified) the findings of the research in the most accessible way as possible like I did above. I encourage you to have a quick look at it –it’s short and has pictures and some interesting discussion in the comment section.
But I even more strongly encourage you to read the paper along with the very fair and accurate interpretations made by the authors. What I also did in that blog post was to point out very quickly (because I had to react fast) that although the results are cool and the paper is neat, the data does not lend support to Chomsky’s theory of language –at least not more than most other theories of language, or in fact of learning, or in fact of how the brain itself is “wired” (I hate using that word when talking about the brain but that’s a discussion for another day). Here’s a short recap of my points for your convenience –because I’m nice.
Chomsky wasn’t more right or wrong before Ding et al.’s (2015) data.
Chomsky’s theory of language posits that our brains come pre-equipped with rules of grammar so that it helps us, as kids mainly, to figure out how language works. Concretely, it means that when presented with linguistic stimuli, our brain “imposes” those innate rules on the stimuli so that it is parsed and understood correctly. Chomsky also claims that this is a property specific to humans. This is a gross simplification of Chomsky’s theory of course and I’m probably going to attract criticism from those researchers who work within that framework –but so be it.
What’s important, is that the data does not lend support to Chomsky’s idea any more than it does to most theories of language (although some do not, see, Frank, Bod and Christiansen’s [2012] paper in proceedings of the Royal society). Indeed, most theories of language admit hierarchical structure in language, but this structure does not have to be innate.
Further, the data does not show that this ability to extract, or impose structure on language is specific to humans nor does it show that similar patterns of activity in the brain would not be observed when processing non-linguistic stimuli like music, geometrical shapes, mathematical formulae, faces, objects and so on...
What’s left in favour of Chomsky’s theory in this article?
Not much, apart from the fact that he was probably one of the first, or most prominent people to claim that structure existed in language. This does not make his theory the only one compatible with the data presented in Ding et al.’s (2015) paper.
What went wrong?
We often criticise journalists of inaccurately reporting scientific research. Indeed, the style of journalism does not lend itself well to the usual caution and hedging used in scientific journals.
But, we also know, that press releases are often strongly influenced by what the authors “feed” to those journalists. While the authors were really parsimonious and careful in the original paper, this was not the case in the comments made in the press:
“I’d say, on balance (comparative language research, language acquisition research, these kinds of brain data) the empirical research favors the Chomskyan view, as unpopular as it is,” wrote Poeppel.
Prof. Peoppel is an amazing researcher, one that has influenced my own research a lot, and for whom I have utmost respect. However, I find these kinds of comments misleading. Although, it is important to keep in mind that it may be harder to have control over your spoken words to a journalist than over your written ones in your paper, as pointed by a colleague of mine.
In closing: The fundamental issue with science
This is not a personal critique, of course. I think such practices could be pointed out of many scientific press reports. It reveals a big problem behind scientific research: 99% of our results are not sexy! But, in order to be heard, in order to sell, and attract attention –which means more chances to secure funding– researcher have to resort in embellishments of the truth. The question is: How far should we go with our embellishments? And why can’t we just stick to the truth?
What goes wrong are the “laws of the publishing market” by which scientists have to abide. Laws, that often push scientists to embellish their claims, to engage in questionable research practices (cf., recent shortcomings of reproducibility in psychological science), and to publish only sensational results at the expense of null results (i.e., experiments whose results show no effect) or slightly boring replications. That is the real problem!
When science reporting goes wrong, sometimes it’s not always the journalist’s fault nor really the scientist...
About the Author:
You can find out more about me here: bastienboutonnet.com
Acknowledgements
Many thanks to Richard Kunert, Maria del Carmen Parafita-Couto and Leticia Pablos, who provided fast comments and opinions on the piece early this morning :)
