Backing out of the dead end Prasad and Cifu have reached on ‘reversal’ — Part 1

In Ending Medical Reversal, authors Vinayak K. Prasad and Adam S. Cifu relate a body of significant and important scholarship, using many crisply told stories and making numerous valid points along the way. Time and again, they have us stand beside them at the moment of a ‘reversal’, surveying together the harms wrought by the premature, widespread adoption of yet another health care practice that has just been shown — invariably, by a randomized controlled trial (RCT) — to have been ineffective or outright harmful. They open with the tale of internal mammary artery ligation, which should thoroughly astonish any reader not already familiar with it. In the chapters that follow, we learn about similar reversals of prematurely adopted practices of all kinds: surgical, intravascular, screening, primary prevention, and systems-level interventions.

This book should be read. But I think it immensely important that it be read with far more philosophical awareness than the authors have brought to their subject. The authors’ response to (what they call) ‘medical reversal’ involves a series of linguistic and philosophical misadventures that — ironically, in a book about the uncritical adoption of medical practices — seem to have escaped critical examination by the authors themselves.

The authors appear to start from the same place we all start: with a profoundly disturbing experience of the injustice that pervades medicine. This injustice is felt everywhere, and it gives rise to an amazing variety of accounts. In How We Do Harm, Otis Webb Brawley breaks ranks to expose rampant professional greed; in The Loss of Sadness, Horowitz and Wakefield examine the construct validity of a diagnostic category run amok; in Medicine in Denial, Larry Weed dismantles medicine’s untenable self-mythology. Prasad and Cifu for their part seize upon ‘reversal’ as the phenomenon through which injustice expresses itself most powerfully and distinctly to their own sensibilities. They state unequivocally (p. 18), “We believe that reversal is the most important problem in medicine today.”

It is easy to appreciate the heady effects of that crystal-clear moment when a longstanding medical practice is reversed by the definitive conclusion of a multicenter randomized controlled trial. At that moment, we stand upon the epistemic summit of an RCT, possessed of an almost godlike knowledge of a causal effect. From these heights we can cast our gaze upon the benighted tribes in the valleys below, who wallow in the filth and depravity of their professional and industrial venality. One wonders if the intoxicating effects of such altitude, with its proximity to God and temptations to judgment, have perhaps set the psychological stage for the authors’ subsequent errors. A little hubris would certainly go a long way toward explaining how the authors could have allowed themselves to make the intellectual errors I will examine here. However, I will confine my further comments and advice as much as possible to those aspects of their argument that are objectively accessible to scrutiny: namely, their use of language and logic.

Suppose two Generals at the Pentagon reviewed our military history, documented a pattern of ill-considered military engagements, and announced that ‘retreat’ was the most important problem in defense policy today. Shouldn’t we advise them to select a better term? The misnomer ‘reversal’ is hardly the end of the troubles of Ending Medical Reversal, but it sure does get these troubles off to a strong start. This is no mere ‘quibbling’ of the kind that Prasad and Cifu protest altogether too much in their book. Inattention to the meaning of words is neither cute nor innocuous; its most damaging effect is to dull the critical faculties of the reader. However well ‘reversal’ may have correlated with their initial research methodology, it certainly does not correspond to the problem Prasad and Cifu actually want to solve. Still, it is likely no accident, nor a mere concession to academic ‘branding’, that Prasad and Cifu retain this word as their banner.

By focusing our attention on the moment of reversal, the authors valorize the RCT that plays so prominent and muscular a role at that moment — and forms a central tenet of their EBM dogma. (As they note [p. 74], “The common thread among all medical reversals is that a large, well-done study — typically a randomized controlled trial — finds no [net] benefit … for a common practice.”) Equally important, by deflecting our attention from the initial adoption and dissemination of medical practices, ‘reversal’ helps the authors evade a deep conceptual analysis of those processes. With senses dulled by the drum-beat of “reversal”, the reader may never pause to reflect on how utterly remarkable it is that those processes where the principal error itself occurs receive such superficial treatment in this work. Nor may this reader ever glimpse the irony that, in this book which so belabors surrogate outcomes, the authors focus our attention on a mere ‘marker’ rather than the ‘disease’ itself. Prasad and Cifu might in their further work consider ‘premature ejaculation of therapy’ as one highly marketable alternative with an excellent and fully defensible Latin pedigree. (And who would not celebrate the day when a skeptical patient first asks, “Doctor, are you prematurely ejaculating this therapy?”) However, in service to the actual content of my own argument, in what follows I will call our problem simply uncritical adoption.

About uncritical adoption, the authors tell us little more in this book than that it happens because doctors ‘blindly’ ‘assume’ their ‘perfect mechanism’ makes ‘perfect sense’ on the basis of ‘inadequate data’. Chapter 11, titled “Scientific Progress, Revolution and Medical Reversal,” which could have been a great opportunity for the authors to expand, refine and correct these notions, instead degenerates into a gross philosophical misadventure bordering on farce. But it is a most revealing misadventure; the chapter may very well contain the original source of the irony that has metastasized so widely throughout the authors’ argument. The authors select as the fount of their philosophy of science the work of Kuhn, who within a few years of his famous 1962 publication was already finding himself accused of turning away from his original claims. Indeed, the authors introduce Kuhn admitting [p. 127] that “his work has fallen in and out of fashion,” suggesting they themselves may be vaguely aware of its status as a ‘philosophic reversal’. The choice of Kuhn would seem odder still, given that the amount of print the authors must devote to apologetics leaves little ink left for actually applying Kuhn’s ideas to illuminate anything. But in retrospect, Kuhn’s contribution to Chapter 11 has nothing to do with his ideas. Rather, aided by the confusion generated in their discussion of Kuhn’s philosophy, Prasad and Cifu are able to exploit that infamous word ‘paradigm’ to effect a most remarkable debasement of medical science and its possibilities.

In the standard understanding of science shared alike by professional scientists and the educated lay public, the process of science involves advancing theories, which (if they are to have a scientific status) must yield predictions that can be tested. (We have all recently seen a stunning example of this process at work in the detection of gravitational waves predicted by Einstein’s General Theory of Relativity.) Against this understanding, however, in the section of Chapter 11 headed How Science and Medical Science Differ, Prasad and Cifu wish to teach us this: “When medical science functions properly, medical paradigms suggest hypotheses: for example the hypothesis that tight blood sugar control benefits diabetics. The next step in the scientific method is to test this hypothesis, in this case by a randomized trial testing whether strict blood-sugar control is better than lenient control.”

Let’s review this linguistic snatch-and-grab in slow-mo. The first bit of legerdemain is to replace theory with Kuhn’s inscrutable paradigm, thus depriving theory of an essential characteristic — its amenability to criticism. The word hypothesis, used interchangeably with theory by a philosopher such as Karl Popper, is also disposed of — hidden in plain sight by debasing it as a substitute for prediction. This latter trick has two profitable effects for the authors’ argument. Firstly, it sweeps aside the dangerous idea of prediction, lest it remind us of the power and precision of which theories are capable. (Heaven help the authors should the idea of precision medicine based on predictive models cross the reader’s mind whilst reading this chapter!) Secondly, in its thus-debased usage, hypothesis now all too readily suggests the ‘null hypothesis’ central to the authors’ favored RCT methodology. The reader who now takes the authors’ linguistic Monopoly money for philosophical gold, need hardly think (and is this the point?) in order to accept the author’s RCT recipe for medical science.

What we have been robbed of in this philosophical mugging are richly textured theories/hypotheses capable of producing a wealth of predictions by which they can be tested, criticized and improved. We are bereft of anything that could have provided substance for criticism, and thus for the critical adoption of medical practices. The very problem to which Prasad and Cifu ought to be addressing themselves is thus whisked away. What we have in its place is a caricature of medical science, wherein hypotheses lose all of their theoretical character and are reduced instead to atavistic grunts: “Me think strict glycemic control good; me try it.” This serves well an agenda in which the RCT — as blunt a caricature of scientific method as will ever be found — must be elevated to a privileged epistemic status.

Prasad and Cifu drop all kinds of hints that they don’t take very seriously the task of Chapter 11. Their contempt for philosophy seems equal even to their contempt for theory. It is most characteristic of the anti-philosophical spirit of Chapter 11, that in concluding their defense of their selection of Kuhn’s philosophic framework, Prasad and Cifu draw this lesson: “the only way to know something works is to know that it works.” How could one possibly display a more thoroughgoing contempt for epistemology?

More to the point, however, is the question: How might the authors have proceeded if they had sought to make their case not by mere wordplay, but through logical analysis and argumentation? One demonstration they might have attempted would have been to show that in at least some cases of reversal, an RCT upended a theory that had otherwise withstood vigorous criticism leveled using a range of methodologies short of the RCT. The data set they have assembled (their Appendix catalogues 146 reversals!) would make a highly suitable starting point for searching for such examples. Indeed, the planning of every one of these ‘reversing’ RCTs ought on ethical grounds to have included a thorough review of all available critical perspectives, to ensure that the necessary clinical equipoise was present.

Our tardy adoption of Bayesian adaptive trial designs, however, suggests that very little critical thought has gone into the ethical demands of equipoise. I conjecture that a search through Prasad and Cifu’s Appendix would find few examples of an impressive body of theoretical developments and critical assessments preceding a trial. Roughly speaking, “Me think; me try; also pretend try; look difference; get p-value” may actually be a pretty accurate positive picture of clinical science as practiced, whatever its deficiencies as a normative principle.

Clearly, what is required at this point is not to heap more abuse upon authors who have made a significant and worthwhile scholarly contribution, nor to lecture them on guarding the distinction between the positive and normative. It will not have escaped notice that I have basically conceded the positive question to Prasad and Cifu. To accept this description also as normative, one need only adopt a pessimistic view of human intellectual potential. Thus, what is now required to support my criticism of Ending Medical Reversal is an affirmative demonstration of the potential of theory and theory criticism to move clinical science beyond the Stone Age. In Part 2 of this post, I will advance some ideas in that direction.

Go to Part 2