AI can save lives, but we can’t let it decide which ones

Fresh off the heels of the #hlth conference: Consultant Ben Alsdurf comments on a recent study that highlights the risks inherent in using historical data to train machine-learning algorithms to make predictions in the health care system.

“For individuals with chronic conditions care management programs offer a glimmer of hope — additional support to address the litany of health issues that often plague this group, while ideally reducing costs to the system by mitigating the impact of co-morbidities. When you talk about #populationhealth these programs are critical as they reach the patients driving cost for the system.

The problem, in this case, was that the tool used future cost-projections of individual patients as a proxy for health needs, when health-seeking behaviors and access to care were not standardized. As a result, African American patients’ relative health was overestimated and thus additional care management services withheld.

What can we learn?

The best practices for auditing algorithmic bias, particularly in health, where lives are literally on the line need to improve. Commercial organizations don’t want to open up the secret sauce of their decision making publicly but bringing in external auditors (or researchers) who can identify problems like in this research is crucial for meaningful cost savings and improved care. As more people are moving from volume to value-based payment schemes, particularly in Medicare, the need to deliver outcomes will become increasingly important.

As the authors note “health is one of the most important and widespread social sectors in which algorithms are already used at scale today, unbeknownst to many.” Bringing the role out in the open and transparency to factors being used to calculate risk could offer a competitive advantage in a value-based care environment”.