Noise in Medical Decisions

Noise and bias affect medical decisions. It’s an unfortunate reality. So it’s important to understand those effects and what steps we can take to reduce them.

This article from the Harvard Business Review discusses what they call “noise” in decision making:

Noise: How to Overcome the High, Hidden Cost of Inconsistent Decision Making

When pathologists made two assessments of the severity of biopsy results, the correlation between their ratings was only .61 (out of a perfect 1.0), indicating that they made inconsistent diagnoses quite frequently.

They first establish examples (and there are a lot) of how high level professionals, like physicians, can come to different conclusions even when given the same input (noise). The article authors diagram this nicely below, showing that different professionals might reach different conclusions given the same starting case (Examples B and D). Whether or not those conclusions are close to “right” isn’t the point. The point is that different people are coming to different conclusions, when they should be coming to similar conclusions.

They go on to investigate how and why these differing conclusions occur:

When asked about what their colleagues would say, professionals expect others’ judgments to be much closer to their own than they actually are.

…

Where there is judgment, there is noise — and usually more of it than you think.

Some users even ask us, “Why do I need to hear an answer from another user, when I might be able to look it up myself?”

We designed MedCurbside to be a crowd sourced venue for a reason: we often assume in medicine that if we read the evidence and we have experience, we’ll come to the same decision as our peers. But, as noted above, this isn’t always the case. And we think that it’s important to have a forum in which we can hear answers from multiple people and use features (voting as one example) to give us a sense of how the the community’s members are thinking.

We need to be open to hearing about the variation among physicians’ conclusions. And we should do so in a venue that lets us evaluate the merit of each perspective.

It’s not a perfect solution: It’s not the the real-time algorithms noted in the HBR article and it relies on participation, but we think its a step in the right direction. And that will hopefully get us closer to more consistent decision-making.