Empowering Team-Based Advance Care Planning with Artificial Intelligence

Anand Avati*, Ron C. Li*, Margaret Smith*, Jonathan Lu, Andrew Ng, and Nigam H. Shah

* Equal contribution

Project team (alphabetical order): Neera Ahuja, MD; Anand Avati, MS; May Carr, MS, RD; Margaret Dougherty, MS OTR/L MSHC; Scott Fleming, MS; Paul Georgantes, MSN RN; Melanie Hanson, BA; Stephanie Harman, MD; Grace Hong, BA; Ken Jung, PhD; Sehj Kashyap, MS; Ron Li, MD; Steven Lin, MD; Jonathan Lu, MS; Pooja Rao; Amelia Sattler, MD ; Briththa Seevaratnam, MHA; Nigam Shah, MBBS PhD; Heather Shaw, GNP-BC; Lisa Shieh, MD PhD; Margaret Smith, MBA; Winifred Teuteberg, MD; Samantha Wang, MD;

“Your condition is worsening. If we don’t put you on a breathing machine in the next 24 hours, you may not survive. However, given your longstanding lung disease, it’s possible that you’ll never safely get off of the machine and leave the hospital. Do you want us to proceed? We need to decide now.”

This decision may seem impossible to make so quickly, but such choices must routinely occur in hospitals for patients with life-threatening conditions. Most Americans near the end of their lives are not able to make choices that reflect what matters most to them.

In a recent survey of Californians about the attitudes towards death and dying, 70% responded that they would prefer to die at home, but in 2009, only 30% of all deaths happened at home while 60% occurred in a hospital or nursing home.

What if the patient had discussed their goals of care with their medical team six months ago, with more time to think about their options? Almost 80% of respondents in the same survey indicated that they would like to talk to their doctor about care at the end of life, but only 7% have ever had a doctor bring up the subject.

Simply put, conversations about end-of-life care decisions, known as advance care planning (ACP), are done too infrequently and too late — even for patients with chronic, serious illnesses.

Understanding the problem and opportunities for change

We tackled this problem at Stanford Health Care, a quaternary care academic medical center, by focusing on ways to improve timely ACP with patients admitted to the general medicine service. As a team of clinicians, improvement consultants, and Artificial Intelligence (AI) engineers, we first sought to understand the problem’s root causes and key drivers of solutions, rather than jumping to a technical product. We interviewed the entire inpatient care team, including physicians, nurses, social workers, occupational therapists, speech and language pathologists, and nutritionists, who each play a vital role in caring for hospitalized patients.

Key reasons for not performing ACP were:

• lack of time during high patient volumes;
• not thinking ACP needed to be addressed if it did not affect the patient’s current hospitalization; and
• lack of a shared mental model among care team members regarding who should receive ACP.

This last point was especially key: non-physician care team members often felt that patients with repeated hospital admissions needed ACP, but were unsure if the physician would agree with their assessment and so did not feel empowered to act. Interestingly, physicians welcomed the idea of including non-physician providers to help with ACP conversations as it would decrease the physicians’ time burden. However, they were unsure how to efficiently arrive at a consensus on which patients to engage.

A team approach to advance care planning

We identified several key drivers for a potential solution:

• a shared understanding across the team of how to identify which patients need ACP
• a shared workflow for conducting and documenting ACP
• empowerment of each team member rather than only the physician to initiate the ACP process.

Therefore, we asked how can a computer generated prediction enable one or more of these drivers?

We identified an opportunity of using AI to enable the first driver i.e. identifying patients who would benefit most from ACP, which could be accomplished accurately and consistently with a model to estimate the patient’s 12-month mortality risk. We also hypothesized that this objective way to identify patients for ACP for a shared workflow would empower non-physician providers to initiate conversations.

Typically, the decision to initiate ACP depends on the physician’s prognosis for the patient, which is subjective, but difficult for a non-physician to challenge. However, an AI-generated prognosis could provide a foundation for a shared discussion regarding the patient’s appropriateness for ACP. We envisioned the AI system predictions as an enabler for a new work system for ACP that empowered the entire care team, including non-physicians.

Building an AI-enabled Work System

Achieving the new ACP workflow required us to treat AI as part of a complex socio-technical work system, where outcomes arise from a set of interconnected units, including people, workflows and technologies, that form distinct structures, processes, and patterns of behavior. For example, timely ACP for a hospitalized patient is a result of complex interactions between people (eg. clinicians and patients), workflows (eg. morning clinical rounds, afternoon multidisciplinary huddles), and technologies (eg. the electronic health record used for documenting ACP, the mobile apps clinicians use to communicate with each other). We recognized that the current work system was hierarchical and siloed (Figure 1-L); the physician decided which patients needed ACP, and other members of the clinical team followed suit. This structure contributed to missed ACP opportunities and disempowerment for non-physician team members.

We sought to use AI to enable a more democratic, collaborative structure for the ACP work system. This required that all team members receive knowledge of which patients needed ACP without it flowing from the physician (Figure 1-R). The AI model played a crucial enabling function for this system. The model’s regularly emailed recommendations to the entire care team empowered team members to approach ACP outside implicit professional hierarchies.

Figure 1. Rather than depend on the physician to choose patients to receive ACP (left), our AI-enabled workflow informs and empowers all members of the care team with a list of suggested patients who may benefit from ACP (right).

Designing the workflow

From a series of participatory design sessions, we designed the following workflow for the AI-enabled ACP system that incorporates new interactions between members of the clinical team and leverages tools within the electronic health record (EHR) (Figure 2).

Figure 2. Parallel workflow for the primary medical team and Social Work, Therapy, Nutrition and Nursing leadership to jointly address the different aspects of the ACP conversation with patients.

Knowing the Standard Hospital Medicine rounding procedures, the team identified that model predictions would best be delivered each morning prior to attending rounds in order to inform rounding strategies and topics. Sharing the model predictions with the entire care team in the morning was a key design decision to enable a collaborative and decentralized approach, which built resilience to individual capacity constraints of any one care team member.

Figure 3. The Serious Illness Conversation Guide, developed by Ariadne Labs, used by the main care team.

Additionally, the team decided to leverage the Serious Illness Conversation Guide developed by Ariadne Labs (Figure 3) as a shared mental model and evidence-based approach to advance care planning. The conversation guide allowed clear ownership assignment for different aspects of advance care planning, and a unified approach to documentation and information sharing.

Integrating AI into the work system

Figure 4. Our daily informatics workflow involves 1) pulling patient data from the Epic EHR and physician schedules/assignments from QGenda, 2) performing 12-month mortality risk predictions on a secure server using a gradient-boosted trees model (NGBoost), and 3) emailing the list of high-risk patients of to members of the care team.

We aligned on the 12-month mortality risk for hospitalized patients as a good proxy for patients with serious illness who needed ACP. We so trained an AI model on historical EHR data to estimate this mortality risk. The model is run every day on data from currently admitted patients (Figure 4). A shortlist of those who exceed a risk threshold is created, and a list of patients is securely emailed to care team members, customized based on their scope of care (Figure 5).

Figure 5. An example of the morning email a care team member would receive providing a list of patients with higher risk of 12-month mortality, for ACP conversations.

Using the PDSA cycle to iterate on the work system

To iterate on our deployment, we leveraged the structured ideation framework: Plan, Do, Study and Adjust, or PDSA (Figure 6). This process starts by planning a pilot with well defined questions, carrying out experiments, monitoring progress, and addressing any unexpected workflow or technical issues with qualitative and quantitative data, at regular meetings.

Figure 6. We performed several Plan, Do, Study and Adjust cycles before aligning on our current workflow.

Some examples of changes that resulted from the PDSA iterations include:

• During the winter of 2020/21, care team members were feeling stressed due to an increase (about 30%) in the number of patients flagged by the model. Based on this observation, we made small wording changes in the email (to “select 1–2 patients each week” from the lists, which was previously unspecified) that made a difference in setting healthy expectations for providers and appealing to their clinical judgement.
• Based on qualitative data collected, we found that email based notification is not always the ideal mechanism to convey the model predictions for some clinical roles. In light of this feedback, we developed additional software to upload model predictions into the EHR system, which is more readily accessible by clinicians.

Results

Our deployment began in early 2020 as we piloted our first workflows, sending daily emails out and iterating on the PDSA process. The pilot was put on hold in March due to COVID-19. Since resuming in July, the number of patients discharged with documented ACP has steadily increased (Figure 7).

Figure 7. We tracked the patients discharged from Stanford inpatient medicine who had had ACP. After most care team members were trained in August 2020, we saw a steady uptick in patients receiving ACP.

Furthermore, we have observed a healthy mix of various care team members participating in the ACP process (Figure 8).

Figure 8. We stayed informed about different care team members’ engagement with the system.

Key Learnings

In this end-to-end process, from identifying a problem to developing an AI model to implementing an AI-enabled solution for clinical use, we learned several key lessons for future clinical AI projects.

First identify the clinical problem that needs solving.

AI is a novel technology, and projects are sometimes driven more by the desire to test out the technology rather than to solve a problem. However, it is far more important to busy healthcare providers that patient care problems are addressed, rather than new technologies tested. We committed to solving the problem of improving ACP as our guiding objective. This early alignment was key to our approach to designing the solution (everything from aspects of the AI model to the workflow) and engaged with clinician end users.

AI is one component of a broader solution.

Healthcare delivery is complex and occurs over many different moving pieces of people, processes, and technologies. When thinking about the any AI product that the end users will experience, the AI model is only a piece of the puzzle; we must consider the other critical components, such as the workflows, digital applications, clinical processes; all of which comprise what we refer to as an “AI-enabled work system.” Simply asking what will a user do with an AI prediction is often not sufficient; rather, the more important question is how will AI enable a better work system?

Engage stakeholders early in the process.

Participatory design is key to creating solutions that users find useful. There is no amount of theorizing that could have replaced the feedback from clinician end users who had a much deeper understanding of the problems around ACP we were trying to solve. This also helped our team be more creative and open minded about the possibilities for how the AI model can improve the work system. Many of these users became our early adopters and clinical champions, and were key to successful engagement and implementation in the broader clinical environment.

Adapt quickly.

Things change quickly in healthcare, and often unpredictably. Shortly after our launch, the COVID-19 pandemic hit our shores. In response, we quickly put our implementation on hold and restarted it when our clinical partners felt like they were ready. There were also many adjustments and decisions we made quickly in response to user feedback and observations over multiple PDSA cycles. The real world is often much more complex than anticipated, so even the best designed solutions will need to adapt and improve once implemented.

We are continuing to iterate on our AI-enabled work system for ACP with the goal of scaling this approach to other clinical units at Stanford Health Care, and a prospective study is under way. In particular, the strategy of using AI to empower team based care is a powerful theme that emerged from our work, and is one our group will explore for future clinical AI projects.