Q&A with Intellicare, winners of the 2019 Fung Institute Mission Award
Edited by Ashley Villanueva
Each year, the Fung Institute Mission Award is awarded to the Capstone team that best exemplifies the mission of the institute: “transforming scientists into leaders who can take risks and develop technical, social and economic innovations.” Finalists are nominated by Fung Instructors; winners are chosen by Fung Institute staff based on the project brief. This year, the recognition goes to the IntelliCare: Machine Learning to Fight Alarm Fatigue in Hospitals Capstone team, advised by Prof. Gabriel Gomes, Prof. Xiao Hu, and Prof. Michelle Pelter.
IntelliCare: Machine Learning to Fight Alarm Fatigue in Hospitals
Eight out of ten bedside hospital alarms are false alarms. This costs the US healthcare system more than $2.5 billion annually and limits the effectiveness of nurses in real situations of urgency.
Machine learning algorithms could help improve monitors but they require large annotated datasets. However, manual annotation of ECG records is a time-consuming process for cardiologists. In collaboration with UCSF, our team has developed a semi-automatic annotation tool focussing human resources on only the most informative ECG records. This will ultimately lead to the creation of a dataset 100 times larger than any other available resource.
We developed a proof of concept by focusing on Premature Ventricular Contraction, the most frequent cardiac arrhythmia. Our team designed a variety of machine learning pipelines, including feature-based models and deep learning on time series data, to recognize irregular beats. These models were then incorporated into an active learning framework requiring minimal human input to classify unannotated data.
The resulting annotated dataset will help improve existing hospital monitors and allow nurses to focus on patients most in need.
We had a chance to speak with the Capstone team (Thomas Galeon, Xudan Luo, Oskar Radermecker, Clement Ruin, Yixun Tan, Wei Tan, Jidapa Thanabhusest, Sruthi Thomas, and Jessica Wu) about their experience.
How did you define the scope of the Capstone project?
If you’ve ever been to a hospital ICU or Emergency care department you’ll have noticed that heart rate monitor alarms go off all the time. In 80% of cases these are false alarms due to patient movement and scratching. This wastes valuable time and resources for hospitals. False alarms also put patient’s lives at risk since nurses become desensitized to the urgency and no longer respond to alarms. We also came to understand that excessive alarms induce stress and anxiety among patients. These were the motivations behind our project.
Our Capstone project was suggested by University of California San Francisco’s (UCSF) hospital which, like hundreds of other hospitals across the country, has been facing a high number of false alarms. They asked us to develop a method to limit erroneous alarms in their monitors.
Our team focused on the most common type of cardiac arrhythmia, premature ventricular contraction (PVC), which accounts for a third of all alarms. We developed a proof of concept for filtering PVC false alarms. In the future, we hope to extend our work to other types of cardiac arrhythmias.
What was your most significant teaming challenge & how did you tackle it?
The most significant challenge we faced as a team was probably team size. Having nine people on a team can be both an asset and a liability. In our first few meetings, ideas were coming from every direction and we struggled to channel our discussions. To tackle this issue we decided to split into functional teams: machine learning, architecture and product development. We focused meetings on intra-subteam challenges and next steps whilst ensuring regular inter-subteam communication. This considerably increased our productivity. Good ideas no longer passed unnoticed and each team member now had a specific task. Ultimately this increased accountability and ensured all members were involved in the project.
What was your most significant project management challenge & how did you tackle it?
The most exciting and challenging part of our project was working with a variety of stakeholders from the medical and engineering world: doctors and nurses, machine learning research teams from UCSF as well as the Capstone staff at Berkeley. Aligning the objectives from all our stakeholders and ensuring that we met their expectations was not always straightforward.
We soon realized that to make this project a success we needed a clear communication path with each stakeholder and established a plan for communicating with each of our partners. This greatly facilitated the coordination of our efforts and ensured all parties involved had the same vision for the project.
What was your most significant technical challenge & how did you tackle it?
Working with UCSF’s hospital was an incredible opportunity. The hospital gave us access to a billion electrocardiogram records which had triggered an alarm for PVC. This dataset is 2 orders of magnitude larger than any other publicly available dataset. However the data came unannotated: we had to determine whether or not each record was a false alarm.
Manually annotating the records was unfeasible and would have required million of hours of annotation by trained cardiologists. To tackle this issue we used active learning, a machine learning technique which focuses manual annotation efforts on only the most informative records. Instead of randomly annotating samples, our algorithm identifies the most informative records to annotate. The other records are annotated automatically.
Ultimately, once the whole process is complete, we will have an immense dataset of true and false alarms for PVC. That dataset will then be used for studies on PVC by clinicians and will also enable the team to train another machine learning algorithm to identify false alarms in real time.
Have you had a capstone project in your undergrad and, if so, how has this capstone experience been different?
Most of us did have a Capstone project during undergrad. The proximity and easy access to professionals working in the medical space are what set this Capstone apart from our previous projects. Working with nurses and researchers at UCSF provided extra motivation for our work since we knew exactly how and where our product could be used. It also helped us understand the greater impact of our project on the healthcare system.
You probably had a number of course-based projects during your MEng year. How is the capstone different?
As any engineering project at UC Berkeley, the Capstone has a strong technical component.
What sets the Capstone apart is all of its non-technical aspects, especially project management and stakeholder analysis.
Project management does not necessarily scale well and methods which work with groups of two or three completely fail with a group of nine. Discovering this for ourselves was a fantastic learning experience and prepared us for the intricacies of corporate life.
Dealing with multiple stakeholders was also a major learning point of the Capstone. Working at the interface between clinicians and engineers comes with its own sets of challenges since these different worlds do not always share the same vocabulary, let alone objectives. Carefully analyzing the needs of the nurses and doctors through a range of interviews allowed us to develop a solution that could actually be used.
The Capstone therefore has a much larger scope than other course-based projects because it deals with most of the steps of the product development life cycle: need identification, market analysis, technical development, go to market strategies, etc. Course-based projects are at most a semester long and hence have a smaller scope, only focusing on one or two of these components. During the Capstone we explored the main steps of the project life cycle, turning our technical results into impactful product recommendations for our stakeholders.
Is there anything else you’d like to add?
We would like to thank our academic advisor Mechanical Engineering Prof. Gabriel Gomes who has been a great support during the execution phase of our project. His guidance helped us focus on what is important and achieve deliverables using a “divide and conquer” strategy (which we would recommend for teams of more than average size!). We also want to give a special shout-out to our industry mentor Nate Tran who directed us towards the right technical approaches and helped us throughout the project despite his busy schedule.
Connect with the team: Thomas Galeon (BIOE), Xudan Luo (IEOR), Oskar Radermecker (BIOE), Clement Ruin (IEOR), Yixun Tan (BIOE), Wei Tan (IEOR), Jidapa Thanabhusest (IEOR), Sruthi Thomas (IEOR), and Jessica Wu (BIOE)
