I was just reading the nth iteration of the science irreproducibility horrorshow that’s been going on for a while, and also PubPeer’s response to the latest ancien regime salvo, and also also the most recent goddamn gay gene fiasco.
(the reader will forgive me for being addicted to Retraction Watch’s “Weekend Reads” posts.)
And the main question that I have, rehashing all this in my head, is why people do research on humans anyways. It’s expensive, generally underpowered, ethically fraught, confounded by more factors than I could reasonably list, and errors have a much larger potential to destroy lives inadvertently. Oh, also, there’s something weirdly Heisenbergian about an object trying to observe itself… it seems like there’s a conflict of interest preventing us from being able to operate with anything like objectivity.
I mean, I KNOW why certain research has to be done on humans. Medicine, for a lot of researchers, is why they got in the game. Also, a lot of disciplines simply wouldn’t exist if we didn’t do research on humans. My question is whether, as in medicine, it would be better simply to do no harm.
When I see all of these replication failures, and the vast proliferation of studies which are apparently positively wrong (like, not that they are just quantitatively imprecise, or a stepping-stone to future work, but just plain garbage), what I notice is that they are nearly all done on humans. And it naturally leads me to wonder why we as a society spend so much effort and money to get results that have nothing to do with reality, when there is so much science that can be done where an answer is knowable. Let me give an example here.
I work in a lab with a person who studies plant morphology. let’s call her Kjersten. Kjersten’s job is to repeatedly measure a certain feature of plant seedlings, over and over, and evaluate how variable that feature is, between different kinds of plants. Note that this is statistically a MUCH harder question than assessing whether one kind of plant has a higher mean value of that feature; Kjersten regularly does experiments with n ~ 100 for isogenic siblings (isogenic as in exactly genetically identical, or close enough that it doesn’t matter). Kjersten recently got worried because, when she started using a new incubator, all of her measurements started consistently being a millimeter shorter. Note that, because she is measuring variability, this difference is pretty much meaningless. But nonetheless, Kjersten went ahead and carried out a series of new experiments to test whether small differences in the humidity between incubators might be affecting her trait.
Now, in my opinion, Kjersten has gone a little off the deep end in terms of endlessly repeating experiments to be absolutely sure of things that aren’t her primary research goal, and also don’t actually have any bearing on her results. But, at the end of the day, her findings will be rock-solid. What is more, they are CHEAP, because the most expensive component of these experiments is her own labor. Her statistics are beautiful: she has huge power to find small effects (of humidity, for instance), even on a trait as wishy-washy as variability, because she can do multiple replicates of big experiments of genetically identical individuals, who were harvested as seeds at the exact same time from the same self-fertilizing parent. Everything is perfectly controlled.
Of course, just because her results are certain doesn’t mean they are interesting. And, I mean, why study plants when what we are interested in is humans? Forgetting for a moment that human existence depends on agriculture? That is fair. I will leave it to Kjersten to argue for her research project, should she choose to.
I will, however, note that often (usually?) the best way to study humans is to study non-humans. Thinking about my field (genetics), I will point out that a large variety of other studies have shown that it is a lot easier to figure out how human cells work in yeast than it is in human cells. I could, for instance, point out that most of what we know about cancer cell biology was figured out in yeast. More recently, a report showed a high-resolution characterization of common mutations of the famous protein BRCA1, whose mutations are responsible for the most aggressive cancers, particularly of the breast (BReast CAncer 1), in a yeast system. A major problem in the clinic is that a physician will find that someone has a mutation in the BRCA1 gene, but it hasn’t been characterized. It thereby becomes a Variant of Unknown Significance: i.e. something that could be either totally harmless or that could kill you. This obviously sucks as a diagnosis. Following this work I point to above, the problem isn’t entirely solved, but there is a clear and defined method for saying whether a variant will kill you. And none of it could have been done in humans, or even in human cells, at our current level of technology.
You see, yeast are small, tough, easily manipulable organisms. You can’t really see them very well, so they aren’t charismatic enough for animal ethics people to get up in arms about them (screw pandas). They’re incredibly simple to culture, and also to express foreign genes in. And you have more or less infinite population sizes of them!
Recently, I saw a very inspiring talk by Sue Lindquist, the Harvard biochemist. She has figured out a series of strategies to study neurodegeneration, that most human of maladies, in yeast. She has pulled out all kinds of potential drugs, and made major strides towards understanding how brains fall apart, and all by studying a single-celled organism. What good are humans, anyways, in research?
I am reminded of an animal rights talking point, that replication success of treatments between animals and humans is <10%. This, as you may be thinking, ignores the possibility that replication success between HUMANS may be no better. The problem, as always, is logistical constraints of the system.
Choose a shitty system, and you will get a meaningless outcome. Humans are the worst system imaginable.
And yes, I know that I am just beating the model organism drum that people have been pounding for decades.
