Using Artificial Intelligence to Treat Patients with COVID-19 at NYU Langone Health

At NYU Langone Health, our doctors now have a custom artificial intelligence (AI) tool to help treat patients with COVID-19. To my knowledge, this is the first AI risk system for COVID-19 that has been integrated into routine clinical care. Working in partnership with a diverse group of physicians, clinical informaticians, data scientists and others within the institution and New York University (NYU), we went live with our system on May 15, 2020. Since then, it has produced half a million scores for over a thousand patients. A study examining our real-time prediction model for favorable outcomes in hospitalized COVID-19 patients was recently published in npj Digital Medicine.

Our tool embeds a predictive model into the electronic health record (EHR) to estimate a patient’s risk in real-time and updates regularly as new data is collected. Scores are displayed in the EHR and color-coded, e.g. green for low-risk. When the user hovers over the score, a score trendline and the top contributing predictors are shown to help interpret the results.

The Challenge

COVID-19 cases started escalating in New York City in March of 2020, and by the end of the month many of our incoming patients had, or were suspected to have, COVID-19. Early reports from Asia and Europe warned of patients decompensating and rapidly requiring ventilation or critical care. We needed new tools to help triage patients by their risk to ensure each patient received the care they needed. On the other hand, physicians were justifiably apprehensive about discharging patients with COVID-19, especially older patients, those with underlying risk factors, or who required supplemental oxygen.

Our group, the Predictive Analytics Unit at NYU Langone Health, part of NYU Langone’s IT department, led an effort to develop, validate and deploy an AI system to help providers identify patients at low-risk of adverse events related to COVID-19 — those likely to have favorable outcomes. These low-risk patients may be safe for discharge where they could continue to convalesce in the comfort of their own home.

Our Approach

The Predictive Analytics Unit is the central group facilitating translation of AI research into clinical practice at NYU Langone Health. We work closely with clinical leadership and clinical informatics to identify impactful applications and plan a project around a specific objective or intervention. As data scientists and data engineers, we have the capacity to write and refine custom queries to pull large volumes of data from our data warehouses. We set up a pipeline to flexibly extract and cache data each day to enable both reproducible and auto-updating analyses.

Over several weeks we gathered a diverse range of reported prognostic factors (vitals, biomarkers, oxygen support, comorbidities and symptoms from clinical notes) while, in parallel, testing different algorithms and conducting sub-group analyses to converge on an approach. We wanted a score that could apply to any patient with COVID-19, could be readily integrated into our enterprise-wide, integrated EHR and could be described to clinicians in terms of a small number of clinically meaningful predictors.

Death is not the only adverse event that COVID-19 patients may experience. We formed a composite outcome of death, transfer to the intensive care unit (ICU), intubation, readmission after discharge and any oxygen support more significant than nasal cannula at 6 L/min. We designed a model to use recent laboratory results, vital signs and current oxygen support to predict who would have none of these adverse events in the next four days. By updating with new data, the model can learn a patient’s trajectory over time and find patients who may be ready for discharge.

Working closely with colleagues at NYU’s Courant Institute of Mathematical Sciences, we followed a two-step process to develop a nonlinear, ‘blackbox’ model and distilled those results into a simple, linear model that could be easily interpreted and integrated into the EHR. After clinical validation (manual review of patient data) we worked with Epic, our electronic health record, to translate this model into a cloud-based version using Epic’s Cognitive Computing Platform.

The model is configured to score any hospitalized adult with active COVID-19, updating every 30 minutes. Scores are visible in two places: as a column that can be added to patient lists, and a custom COVID-19 summary report that distills other key COVID information into one place. As part of an ongoing randomized controlled trial, half of these scores are hidden from view so we can measure the tool’s impact in terms of length of stay, particularly those estimated as ‘green’ or low-risk patients. We hope to finish enrollment and have preliminary results by the end of 2020.

We are openly sharing the model — the coefficients are described completely in the paper — and are also sharing our Epic integration. If you are interested in validating or using this model at your institution, we can transfer the model and configuration files to you.

Read the full paper for free here.

  • Vincent J. Major, PhD, Data Engineer, Predictive Analytics Unit, Department of Population Health, NYU Langone Health.
  • Connect with me on LinkedIn here.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store