The science ghetto
Aidan Ward and Philip Hellyer
Science can be a way out of the most horrific messes of human thinking. And it can be the way in too. The scientific method as a phenomenon of the social and cultural world is mostly misunderstood and abused. The implications of real science and real work are almost always on the side of irritating the existing power structures and limiting career progression. Much of what we think we know because “science” has “discovered” it is plain wrong.
· That the trials of Prozac in children were identical to the trials of other SSRIs and other antidepressant drugs in this age group — negative. There are more negative Prozac trials for depression in this age group than for any other antidepressant.
· Part of our problem is that MHRA and NICE don’t want to be seen to go back on judgements they made 14 years ago when they licensed Prozac. Better children die than regulators lose face.
· That all of the literature in this area is ghost or company written.
· That there is no access to data from clinical trials — MHRA don’t have access, NICE don’t have access — no-one does. David Healy
Here is David Healy laying it on the line. David is a friend of a friend and a professor and scourge of the medical establishment. So, is what NICE do science? Is what the big pharmaceuticals do science? Evidently it is poles apart, and I am calling this the science ghetto: something that purports to be science and which prevents you doing real work.
Not coincidentally, what David Healy is writing about most immediately is the BBC. The BBC interviewed him for a programme (File on Four) on this subject on which he is a world authority. And yet, they could not find time for his contribution in their programme. The BBC using their national platform to seal the incompetence and mendacity of the health institutions is precisely the sort of move that creates the ghetto.
Last week we spoke of intelligent conversations. Such conversations can only take place in a ghetto if the ghetto is acknowledged to be a ghetto where real science cannot happen. It is precisely because this almost never happens, aided and abetted by the BBC, that the ghetto remains a place of darkness and despair and cannot do any real work.
Tragically, we note that it seems that immigrant children in the US have not only been removed from their parents but given psychotropic drugs without consent of those parents or anyone else. The violence just beneath the surface is ultimately what makes it impossible to do real work.
The ethnography of “science”
I came across this thirty years ago in some work done by an outfit called CRICT at Brunel University. They studied the work of what was then Andersen consultants, codenamed Luxuriant Consultants, working on a new billing system for Thames Water. Lots of white shirts, fresh out of college, failing to listen to people who had worked on billing water for ten or twenty years. I recall some particular issue came into focus and the ethnographic study asked “why does this feature have to work this way?”. They asked the architects, they asked the analysts, they asked the implementation team, they asked everyone. And everyone pointed to someone else and said it had to be that way because of them over there.
So, if we go to a place of ghetto “science”, to an institution of some sort where people claim to apply scientific method, can we tell whether they are doing real work or not? That to me is such a fascinating question, to apply science to itself and ask if we can tell the difference between science and pseudo-science, between real work and ghetto science. What might we be able to observe in ethnographic style? Indeed, what would we choose to look at? What does David Healy observe and then obsessively research in order to understand the difference between what is claimed and what is done?
The plonky answer is that there is a right and a wrong way to apply the scientific method. That if you do a proper review of exactly how a piece of research is conducted, its premises and the evidence for its conclusions, then you can tell if it is real work or not.
There’s an emerging trend in ‘triple blind’ studies, where the data is analysed without knowledge of which subjects received the treatment/intervention. Some go further, and their proposed methodology is scrutinised independently of the research and the researchers. This could go some way to combatting Nassim Taleb’s recurring criticism that most so-called science lacks an understanding of real-world statistics and probability.
The problem here is that the seemingly independent space is thoroughly colonised by official institutions and by journal review processes, and indeed by the plethora of meta-studies that try and combine the results of many experiments. It is probably even harder to insert real work into this space than it is to conduct proper research.
As an example here, consider a chapter out of Rupert Sheldrake’s book, The Science Delusion. He takes the speed of light, being about the “hardest” of hard science constants, and merely notes that estimates of the speed of light have changed fairly systematically over historical time. Estimates cluster for perhaps a decade and then shift and a new cluster forms. I choose this example because it illustrates how even the notion of a basic physical constant needs to be questioned as an assumption if you are doing real work.
Our first clue here to whether a particular cluster of human activity is real work or ghetto science is the range of questions that are being asked and, perhaps even more, how destabilising those questions might be if assumptions needed to be overturned. Even as a project manager, I used to ask whether people really wanted the information that a particular measure might give, to avoid finding out if the answer was going to be unhelpful. For instance, if the estimated end date for a task is not getting any closer over a period of time, real work is obviously called for but may not be welcome.
On the meta-level, you’re studying some phenomenon and you get some positive findings. That doesn’t tell you much until you take some other researchers who are studying a phenomenon you know doesn’t exist — but which they themselves believe in — and see how many of them get positive findings. That number tells you how many studies will discover positive results whether the phenomenon is real or not. Unless studies of the real phenomenon do significantly better than studies of the placebo phenomenon, you haven’t found anything. Scott Alexander.
It is nigh on impossible to do science in such a way that the false positive results are weeded out rigorously. Not least because negative findings are not interesting to journals and such studies may remain in desk drawers rather than being published, even in the relatively frictionless age of internet publishing. And in an environment where management only want plausibly positive results, or even results that can be made to appear plausibly positive enough to sell something, then it is absolutely career limiting even to suggest it. This is the ghetto. This is why a few maverick souls make more progress than the rest of official science put together.
And let me just qualify that last paragraph. Some of those mavericks like David Healy and David Graeber become celebrities by being maverick. I would like to suggest that the scope for becoming a maverick celebrity is as limited as any other sort of celebrity: it doesn’t follow that because they can do it and stay employed that you can!
Portrait of a ghetto: Three Billboards
In the quite wonderful film, set in Ebbing, Missouri, the billboards ask why the police are making no progress with a horrific rape case. That is our spirit here, of course: why don’t big pharma corporations discover/invent new drugs anymore?[1] Why is it impossible for government to implement IT systems? Why do schools act as prisons?[2] Just semi-rhetorical questions that have no answer because they are addressed to systems whose POSIWID lies elsewhere and is not legitimate. Just ghettos where the culture means that real work cannot be done.
In the film, the police and the army protect their own as a first priority. The US army of occupation in Iraq spent 99% of its resources defending itself, in the process committing gross abuses of people and country. In Ebbing, the police beat up black people: it what they have always done and they consider it necessary to maintain “order”. Apparently big pharma are winning the “war” with Trump on drug pricing: that is, their purpose is to gouge and make profits. This is a ghetto.
The other day I was forced to go to a main car dealership to get a new car key reprogrammed. They kindly gave me a smart folder documenting all the other things they needed to do to my car. Talking later to the local garage about the exhaust, the main man there explained that he was a refugee from working in a main car dealer ghetto. His example was that if an oil leak was discovered and the car was under warranty, he was not allowed to inform the customer, so that they could charge for the work at a later date.
Our question in this blog is what happens to “science” in a science ghetto? Does it have any intrinsic strength or ability to adapt to allow it to do real work in an environment where real work is not a thing?
When unproved hypotheses are enthusiastically proclaimed as facts, it is timely to reflect on the possibility that other explanations can be given for the phenomena observed. Pete Ahrens 1957
Pete Ahrens, in this quote from 60 years ago, shows that we have always been here and gives us a clue as to what to observe ethnographically. The premature trumpeting of conclusions is a sure sign that real work is not being done and that science has been ghettoised into a travesty of itself. It is in people’s reactions to developments in the process that we can observe whether people are really inhabiting the scientific method or merely pretending to. This insight applies equally to studies and to meta-studies and reviews in general.
For a really gory example, remember the lovely Dr Shipman, thought to have dispatched perhaps 270 elderly patients to an early grave. Taken individually, the cases had sufficient medical ambiguity, but (h/t to Patrick Hoverstadt) a cumulative plot of excess deaths in that GP practice readily reveals a problem years before it came to light. The question is not detection but whether anyone wants to know. If anyone plotted the rate of fault detection against the age of car for a particular dealership, the sudden blip after the warranty had expired would be very plain.[3]
We can detect ghettoes if we want to, and we know absolutely that real work is not done in ghettoes. We can easily see the behaviours of undue and premature enthusiasm that flag the slippery slope into anti-science, into make-believe and fraudulence. But do we want to know?
[1] Scott Alexander speculates that the two most exciting developments in psychopharmacology in the 21st century so far have been ketamine for depression and MDMA for PTSD, both rave feel-good drugs, despite Big Pharma having had some of the smartest people in the world working for decades on ways to treat psychiatric diseases…
[2] In the UK they also look rather like prisons, surrounded by high government-mandated wire fences
[3] Related: the German equivalent of AA / RAC publishes a reliability index of car makes and models, based on the number of call-outs and the sales figures.
