What To Do When You Can’t Trust The Data
Data will always provide answers. But are they the right answers?
You’re struggling with a health problem. You see your doctor who gives you a diagnosis with almost complete certainty. He wants to do a quick, non-risky surgery to treat the issue. He gives you the option of an MRI, but says it is unnecessary — he has seen your symptoms before, and they match his diagnosis. An MRI is expensive and would slow down the process. You trust your doctor, but you also don’t want any surprises.
Should you get the MRI?
A few weeks ago, I was faced with this exact question. My orthopedic surgeon diagnosed the stiff pain in my knee as a cyclops lesion (localized anterior anthrofibrosis), a rare (5%) complication of ACL surgery. I had ACL surgery last August after getting injured on the soccer field.
Because this would be my second knee surgery in seven months, I wanted to be absolutely sure it was necessary. My pain told me something was wrong. My surgeon told me something was wrong. But I needed more facts. Data. An MRI would tell me exactly what was wrong, right?
I went through with the MRI. It showed the cyclops lesion that my surgeon had predicted, but it also showed edema—swelling—around my ACL.
The MRI said I had torn my ACL again.
According to my surgeon, the MRI wasn’t necessarily correct. “In my experience, I’ve seen that MRI’s are not that accurate so soon after surgery. They also aren’t great with ACL’s in general.”
Now, with new knowledge from the MRI, the situation was worse than I had initially thought. Maybe. With more data, I was also more confused.
I had lots of potential facts: There was a cyclops lesion that needed to be removed; an ACL that needed to be revised; Other.
“It’s highly unlikely that you’re ACL is torn again,” my surgeon explained. He had additional data: “Usually when the ACL is torn, the knee is loose. Yours is tight. And there’s always a story about tearing it. For example, you were out dancing or you were playing soccer. Do you have a story like that?”
For the next few days, I tried to resolve the dissonance between my various sources of data. I tried to marry my MRI results (“Failure of anterior cruciate ligament graft at site of proximal graft fixation in the femoral tunnel”) and scientific findings (3% of reconstruction surgeries fail) with my surgeon’s words (“In my experience,” and “highly unlikely”). I had so many answers and yet no answers at all.
Which data could I trust?
I lost a lot of sleep over it.
In the minutes before the surgery, my surgeon came to talk to me. “This is a case where data is a good and bad thing,” he told me. “MRIs tend to overstate matters of the ACL. It’s just so unlikely your ACL is torn again. But, if it is, we’ll fix it.”
In that vulnerable pre-operation moment, I realized something: Regardless of what was to come, my surgeon’s human instincts gave me more confidence than any other piece of data could. At this point, the data didn’t even really matter.
“I’m going to do whatever is best for the knee.”
It turned out that the MRI wasn’t entirely right or wrong, but my surgeon wasn’t entirely right or wrong either. They were both correct about the cyclops lesion, and both slightly incorrect about the ACL. It wasn’t completely torn and it wasn’t completely healthy.
Like most things in life, it was something in between.
As someone who uses qualitative and quantitative data to make business decisions, I am well-practiced in stringing together observations and statistics to tell a cohesive story. But in this case, perhaps because the data was so contradictory or maybe because it carried personal implications, I couldn’t trust it the way I usually do.
Data today is more credible than ever before. But data will always be a double-edged sword. It can both help and hurt and because of that, we need to be discerning about how much we allow it to drive our decisions.
The MRI didn’t teach me anything about my ACL, but it did teach me three things about data:
- In most cases, no one source of data can provide a complete story. No one source of data can give you the “right answer.” The MRI was only part of a puzzle.
- One piece of data can be more or less helpful depending on the context and its source. It’s up to us humans to make sense of (or reject) data based on what we know about a situation, the world, and ourselves. In this case, my surgeon’s experience told him that the MRI data was not completely accurate.
- When data is informing life-altering decisions, it suddenly becomes about much more than probability percentages. It becomes about managing expectations and emotions. About blending optimism and reality. These were driving forces in the way my surgeon presented information to me.
The best data analysts think critically about the data they gather. They respect what they learn, while also relying on their instinct and expertise. They trust the data. But they also trust themselves.
When I woke up from the post-surgery anesthetics and looked down at my knee, I knew that my surgeon had done whatever was best for my knee. And I needed no additional data.