Building Trust and Adoption in Machine Learning in Healthcare: What Clinicians Say Matters — TOC
A table of contents for the published materials from my MPH capstone for UC Berkeley’s School of Public Health
Thank you to Dr. Ziad Obermeyer and Vince Law for being my readers and guiding me through this work.
Visit my Building Trust and Adoption in Machine Learning in Healthcare site (link here) as another way to access all of this information and any new research.
All underlined text are links to the original published material.
The Capstone
The Abridged Version
Insights from interviews with 18 Clinicians and recommendations for Product Managers and ML Engineers / Researchers, all in 9 minutes of reading
The Full Text
Read it all!… all 22 minutes, including citations, endnotes, exhibits, and detailed recommendations
How to Stay Updated on ML in Healthcare
A list of my favorite newsletters to stay updated on ML in healthcare and digital health as well as top journals for medical science that have machine learning content.
👆 I oftentimes share this article with folks who want to learn more about healthcare in general or healthcare ML
How I Conducted My Research
An excerpt of the methods, results, and references sections from the full text
Clinician Interview Summaries
For those who want the specifics by clinical specialty, I published interview summaries. Note they are de-identified for ease of reading and privacy, and they were approved for distribution by the interviewee.
My Clinician Interview Guide
The clinician interview guide that I used across 18 interviews
Academic Hospitalist
“My current thought is that all new clinicians should be at least somewhat aware of the technology at a bare minimum — knowing very vaguely of how it works, which use cases are better vs. worse, how human clinical judgement will be impacted, and how clinical specialties might look in the future. At the moment, I think all of this information could be included in about four short lectures. But in the future, there may need to be a significant curriculum redesign.”
Cardiologist
“I think it is a bad idea for young clinicians to use ML. There are a lot of subtleties that exist, and if you use ML, then you don’t get the knowledge nor art form of medicine.”
Dermatologist
“I worry that ML will be used by people who don’t have the training to confirm the model’s output.”
Emergency Medicine Physician
“I also think about the ethical questions around how these ML tools get deployed in an equitable way that is usable for all patients.”
(Another) Emergency Medicine Physician
“You can win a Kaggle competition on performance, but it doesn’t mean much when I am thinking about using it with patients’ lives.”
Endocrinologist
“I am excited by the amount of information that I will get access to.”
Family Medicine Physician
“I have been working in the field for so long, so it is hard for me to have the time or capacity to understand how things like ML work.”
Gastroenterologist
“…there is a palpable sense that we are closer than we have ever been to computers being able to reason over healthcare data. That is something that is unbelievably philosophically amazing!”
Internal Medicine Physician
“It is tough for me to say. I don’t think I would be inclined to use it [Machine Learning], since I don’t have any personal experience with it. I can’t compare it to anything.”
(Another) Internal Medicine Physician
“In a world with full-blown ML, training clinicians would be totally different. I would have to be both a data scientist and a clinician. Our jobs would be all about communication with patients and communicating with models — being data science and medical science translators.”
Medical RN
“I am concerned that ML will get rid of the human connection.”
Plastic & Reconstructive Surgeon
“I know how to use the outputs of X-ray machines, CT scanners, and MRI machines to help my patients. And as I think of it, I only have a very rudimentary understanding of how those things work. So maybe for an ML tool, I don’t need to know as much.”
Radiologist
“It has to have a very clear value proposition. What is the ROI going to be? Will there be a meaningful return? These companies tell me how we will practice better, but I also need to know how we will save or make money. Sadly, the system doesn’t incentivize us to do better, it incentivizes us to work faster.”
Radiation Oncologist
“Much of my time, up to eight hours per week, is spent on detailed interpretation and labeling images. If I had a reliable ML tool, I could better focus on acute issues of my patients or treat more people.”
Trauma RN
“Sometimes the computers actually do know more than we do, so it’s not the worst thing to have.”
Urologist
“I think if systemwide diagnostic accuracy can increase, then that will be more important to humankind than what other changes happen to the profession.”
What Patients Say Matters
Clinicians are just one part of the equation; patients also must buy in to enable significant improvements of health outcomes. So, I have additional insights from ad hoc patient-centered events.
My Notes from the FDA Patient Advisory Meeting on AI & ML in Medical Devices
The FDA held a day-long, 7-hour public virtual meeting to gather the Patient Engagement Advisory Committee to provide advice on the regulation and use of AI/ML devices.
