Backing out of the dead end Prasad and Cifu have reached on ‘reversal’ — Part 2
In Part 1 of this review of Prasad and Cifu’s Ending Medical Reversal, I argued that their formulation (and even their naming) of ‘reversal’ impoverishes our sense of the problem and our search for solutions. But all is not lost. We can start from roughly where they leave off, and make serious progress against the problem of uncritical adoption, provided we are willing to call it that.
Indeed, I hinted already at a direct extension of the authors’ prior research, through a conjecture that suggests a research programme along the following lines: (1) start with one of those celebrated ‘reversal’ RCTs, and explore the arguments supporting the equipoise necessary to make the trial ethical; (2) identify in those arguments one or more theories of the therapy; (3) document to what extent those theories have been subjected to meaningful criticism; (4) describe whatever empirical criticism might have been pursued short of a full-blown RCT.
To illustrate how such research might proceed, I investigated the case of ‘Thomas Galbraith’ from Chapter 3 of the book. Drs. Prasad and Cifu wisely let an indignant Mr. Galbraith speak for himself (“Are you kidding me?”) when he learned that the ACCORD trial had repudiated the HbA1c<7.0 goal driving his burdensome type 2 diabetes regimen.
My investigations have shown that, continuing a pattern I exposed in Part 1, Prasad and Cifu failed to dig for the full story here. It turns out that Thomas Galbraith has a sister. She’s an electrical engineer who, like her brother, has type 2 diabetes. Fortuitously, she’s also in a concierge practice. In a HIPAA-defying feat of ‘investigative reporting’, I have reconstructed the following excerpt from a 2006 conversation between Laura Galbraith and her doctor:
Doc: For every 1 percent rise in HbA1c, your risk of stroke or heart attack goes up nearly 20 percent and your risk of death goes up 12 percent. (Reversal, p. 31)
Ms. G: That’s exactly what my brother’s doctor told him! And now poor Thomas is tearing his hair out over an unbearable diet trying to achieve a near-normal A1c. On principle, don’t you think there must be diminishing returns to all his extra effort? I mean, there has to be a curve here, right? Can you show me my curve? Am I still on the section with the slopes you just quoted, or has all my hard work brought me closer to a normal level where the curve flattens out?
Doc: I really appreciate your challenge to discuss this on a deeper level, Laura. This is exactly why I switched to direct primary care, and I know it’s why you did too. Now in fact, a “J-shaped” relationship has been known about in this context since at least 2003, with the bottom of the ‘J’ located at normal blood sugar levels, just as you’re thinking. Unfortunately, like so many studies, the meta-analysis I was citing didn’t use individual-patient data, and the various studies it summarized themselves used simple linear regression instead of modern methods like restricted cubic splines that admit curvilinear relationships. So I can’t really use my literature to sketch this curve for you from trial data with any amount of confidence. I really feel I’m at a loss here. As you know, I see evidence-based practice as an essential part of my commitment to you as my patient.
Ms. G: I can empathize with you here, Doc. You know, the electronic components I use in circuit designs are sometimes manufactured to very crude tolerances — capacitances and resistances can be up to 30% off the nominal values. But I still have confidence in my circuits because I use design heuristics that don’t depend on high precision in every component. I bet we could approach this question in a similar spirit. Honestly, I would be hard-pressed to quantify the ‘burdensomeness’ of this diet to 30% precision! So even your roughest, back-of-the-envelope guess could serve me pretty well as I try to make a reasoned decision about whether to commit to an even harder regimen than what I’m already doing. Can we find a good ‘design heuristic’ for my ‘decision-problem’ that lets us come to a reasoned decision based on our inevitably imperfect knowledge? We can always rethink my decision when new information comes in, or if a new drug comes out that looks like it would be worth trying.
Doc: Well, if you’re willing to accept something approximate, Laura, then maybe you’re looking for something like this figure? It’s from a paper published in 2000, where they attempted to tease out the mediating role of glycemic exposure (as measured by HbA1c) in the different types of organ damage that diabetes causes. I have to tell you, though, the take-home message from this paper was there’s probably no threshold for HbA1c lowering.
Ms. G: I like this plot! I can’t help noticing the figure is adjusted for men, but otherwise it looks pretty interesting. First of all, the absolute event rate on the vertical axis is helpful orientation — assuming women aren’t drastically different from men. I can see that where I stood at first with an A1c of 9.5 meant something like 4% chance per year of eye or kidney damage — and heart attack too. That’s terrible! But at 7.5 [A1c] now, it looks as if I really got the microvascular stuff under control — down around 1.5%. And just as I thought, there really do seem to be diminishing returns on that blue curve. From how hard this has been so far for me, I have to tell you … that next half-percent of risk reduction just wouldn’t be worth it!
Doc: But on the other hand, Laura, look at the heart attacks. Heart attacks and other ‘big-vessel’ stuff like stroke and peripheral vascular disease would be regarded as your greater risks now, and that’s what I’m advising you to consider as motivation for targeting an even lower A1c.
Ms. G: I’m trying not to look at the red curve, Doc! [Laughs.] But seriously, I think I have a good reason for that. This ‘no lower threshold’ thinking just doesn’t seem realistic or logical to me. It implies you’re mentally drawing a straight line all the way down to a normal A1c, which is what, 5? What happens when you get to ‘normal’? Does the curve keep sloping down? If so, there’s enough room on the left side of ‘normal’ to treat non-diabetics and completely eliminate heart attacks. Which is preposterous, of course! On the other hand, does it flatten out suddenly, or even bend up sharply like a ‘V’? I have a very hard time believing any biological system could ever have a response curve with a sharp bend in it. Sure, maybe nerve cells, like digital circuits, can abruptly respond to a stimulus. But not a whole organism, and definitely not an entire, heterogeneous population like what’s in that plot! There must be other factors mixed in with the red line, making it impossible to interpret straightforwardly as a response curve.
Doc: I do take your point, Laura. Certainly, as we discussed early on, your type 2 diabetes is generally understood as a syndrome involving metabolic issues that go beyond simple high blood sugar. Now if researchers took your style of thinking and ran with it, I bet they could estimate at least an upper bound on the benefits from intensive A1c lowering — using observational data thoughtfully combined with existing experimental data. Upper-bound estimation like that has been done in other contexts, but I’m not aware that anyone up to now has tried it in type 2 diabetes. If someone has, it probably was regarded as ‘too speculative’ for publication. Unfortunately, our clinical literature prefers subtly disguised uncertainty over explicit uncertainty. That’s why we’re so scandalized when subjective Bayesian priors expose our uncertainty to scrutiny. It’s also why the abstruse methodological subterfuge of null hypothesis significance testing — and these godawful p-values — remain so dominant despite the complete unsuitability of frequentist probability notions for medical decision making. Ultimately, our literature is shaped by the mentality and prerogatives of guideline-driven, assembly-line medicine like what your brother Thomas gets, with very little thought given to supporting collaborative, individualized decision-making like what you and I want to do.
Ms. G: So the way it looks to me, then, there just isn’t a fully worked-out story here that would persuade me to push harder on my regimen right now. For now, I think I’m going to feel pretty good about where I’ve been able to get to on that blue curve, and maybe hold off on any intensification until there’s a better worked-out theory behind the red one.
Doc: That’s absolutely fair, Laura. I think you’ve asserted your values here very coherently. And I do very much appreciate that you’re willing to revisit this when more evidence comes in. There are in fact a few trials going on right now that might tell us a little more about this decision, once they’re published. But let’s not hold our breath for a proper ‘response curve’ that would satisfy you as an electrical engineer — or me as someone who treats individual patients. Until conversations like this one become the norm in medicine, I’m afraid we’re going to be stuck with makeshift adaptation of significance-test ‘evidence’ to our decision-making. I really wish I could offer you more; thanks for meeting me where I am with my evidence-base.
(Thank you for reading! Your reflections — and refutations — are welcome.)