The Development of a Chatbot During a Pandemic: A Conversation with Maguire Herriman and Elana Meer (Podcast #66)

Intro:

From the Penn HealthX COVIDCast series, Ryan O’Keefe and Jonathan Wakim welcomed Maguire Herriman and Elana Meer. Maguire and Elana are medical students at the Perelman School of Medicine who worked on creating an innovative chatbot and patient triage tool during the COVID-19 pandemic. In this podcast, Maguire and Elana discuss their background, what drew them to medicine and the MD/MBA degree, and how they’ve been able to help as project managers for the Chatbot project.

Pathway to Medicine and the MBA program:

Both Maguire and Elana pursued their undergraduate studies at Princeton University, yet they rarely crossed paths. Elana studied neuroscience and economics. Her thesis was centered around using game theory applications to better dissociate psychiatric diagnoses from one another. Elana has always been fascinated by decision making, which carried over into medical school. This interest pushed her towards the MD/MBA route to get exposure to healthcare decision making along with device innovation and improvements in healthcare delivery.

Maguire studied health policy while he was at Princeton and developed an interest in system-level thinking for healthcare. Maguire is interested in tackling organizational problems in healthcare at a system-level possibly through the role of hospital administration, which fueled his desire to pursue the MD/MBA.

Who Was Involved in the Chatbot Project

A week after COVID cutoff their rotations, Maguire reached out to Kevin Volpp, a former research mentor, who was already involved in the initial stages of the Chatbot project. Dr. Volpp connected both Maguire and Elana to the Chatbot project to utilize their skills as project managers. Both Maguire and Elana worked alongside Kevin Volpp, director of CHIBE, and Roy Rosin, chief innovation officer at Penn Medicine. Together these four individuals served as the central team in bringing together the COVID Chatbot. In addition, a team from Google, helped with the technical infrastructure. It was a combined effort between Google, medical students, and the clinical expertise of Penn Medicine staff to help develop thorough answers to convoluted questions.

What is the Chatbot Project

The Chatbot project was designed to solve a central problem of the COVID pandemic for the Penn Medicine Health system. A ton of questions was flooding the phone lines and the myPennMedicine messaging system, such as “What are my risks for COVID as a cancer patient,” “What is going to happen to my medical appointment,” and “How can I stay safe during the pandemic.” The number of questions overwhelmed the staff and the idea of the Chatbot was to standardize generalized responses to repetitive patient questions. The Chatbot would serve to not only satisfy patient inquiries but also free the healthcare staff to divert more attention to in-patient care.

Two major tasks of the project were content generation and content evaluation. By looking through the myPennMedicine web page and the phone lines, a list of salient questions was generated. A group of medical students was recruited to help provide answers to these questions. The drafts of answers generated for each of the questions were then brought to the various health teams at Penn, such as Infectious disease, Oncology, and Occupational Medicine staff, who reviewed the initial set of answers to ensure validity and accuracy. Additionally, as policies or procedures changed, the Chatbot would be updated with the latest information pertinent to individual questions.

How does the Chatbot Work

For this chatbot, a set number of answers were given and the design was to utilize those sets of answers for a myriad of questions. Currently, the bot has a set of 130 answers and the team had to train the bot in all the different ways a user may ask a question that would drive at the same answer. On the backend, Elana, Maguire, and the team from Google inputted training phases into the bot, which are all the different combinations that a person may ask a certain question. One challenge was having the bot distinguish between similar questions such as “Can I go see my family for Easter” compared to “Can I go see my family in the hospital,” which needed to be mapped to different answers. The training phrases, machine learning, along with AI helped the bot to recognize and answer the different types of questions a user may have.

Future of the Chatbot Project

One future avenue of the chatbot project is to incorporate features of a symptom or triage tracker. Currently, the chatbot directs most users who have questions about their symptoms to contact their primary care provider. The next steps would involve nailing down the severity of the symptoms and providing more specific feedback based on the clinical severity of the symptoms reported. In case there is a surge in the Philadelphia area, the bot would be prepared with algorithms to help walk patients through a series of questions and be able to triage a patient into different levels of clinical care.

While COVID has had a detrimental impact on the healthcare system it has exposed several areas for potential innovation. One example is the importance of organizing information from a decentralized system through processes to keep information current and accurate in a centralized model. The other area is the use of automation within the healthcare system to alleviate the burden of clinical teams. The Chatbot is one example of an innovative product that utilizes automation and the concept of centralized information to help improve our delivery of care as a health system.