How can Data Analytics improve clinicians’ daily practices? Advice from Martin Pusic [Interview]

Hey everyone. I had the great opportunity to talk with Doctor Martin Pusic, who’s responsible for the “Healthcare by the numbers: Populations, Systems, and Clinically Integrated Data”. He’s also the Director of the Division of Learning Analytics for the Institute for Innovations in Medical Education at NYU School of Medicine. We had a long talk on how clinicians’ daily practices can be improved with machine-learning.

Dr Pusic co-leads the “Healthcare by the numbers: Populations, Systems, and Clinically Integrated Data” three-year long program of education for students that is based on the real clinical data of practices.

We felt like he was the right guy to talk with about Healthcare and Data, which limits they are faced with, and how they can partner to increase patients’ final care.

RD: Hello Martin, could you tell us a little more about the “Healthcare by the Numbers” project?

MP: With the “Healthcare by the numbers: Populations, Systems, and Clinically Integrated Data” project we’re leading, we wanted to create a curriculum that would teach physicians to become data literate. We want to teach physicians how to analyze data so that in 20 years’ time they will be able to adapt their practices to what the healthcare system will need at that point. You may know that New York has a large SPARCS dataset containing information on every hospital admission in the state. There are around 30 fields of information ranging from patient zip code, gender, to length of stay and the physician info, etc. There are around 2.4 million admissions a year and therefore 2.4 million rows in this database. It’s a wonderful example of the “Big Data” that is available.

“The ultimate goal is to give doctors the tools and skills necessary to care for not just an individual patient, but for an entire population of patients”

We ask our students to work on this database for two reasons:

• It’s interesting for data literacy: how do they handle spreadsheets, how do they generate analyses and graphics.
• This database shows the granularity of the data currently available. For example, it allows everyone to see every surgery that has been performed in the state and by whom. It lets you benchmark yourself and assess your own performance in relation to other physicians.

The ultimate goal is to give doctors the tools and skills necessary to care for not just an individual patient, but for an entire population of patients by leveraging insights derived from system-wide clinical performance measures & outcomes.

RD: As a doctor, you are trained to be an autonomous decision-maker and highly attuned to your patients on a human level. How do you reconcile this with the ‘blackbox’, impersonally derived insights produced by machine learning?

MP: I think that a core machine-learning principle is that data is ‘king’ and the one who has the best data wins. To me, a physician is an advocate for his or her patients. I am an advocate for the 72-years-old woman in room 7 who is having difficulty with chest pain. But I am also an advocate for the ~1,500 other 72-years-old women who came into the emergency department in the last 4 years with this condition.

It’s an epidemiologic approach that lets us help each and every one of them. I’m indeed deeply convinced we are going to be responsible across patients potentially as much as we are individually. And the best way to do so is to gain insights into how best to deal with that one patient by thinking across patients and that’s where data-mining techniques are going to be helpful for us. Especially as we have more and more digital substrate with ubiquity of electronic healthcare records and databases.

RD: What are some of the barriers you have seen in your experience with doctors in terms of opening their eyes to the analytic side of things?

MP: To me, Data Analytics’ main issue is its “blackbox” algorithms.
I hate it when I try to explain a neural network, and how we train it and nobody really understands what is happening inside that box. However good the output is, the process is totally opposite to the way physicians think of themselves in terms of approaching a problem. Indeed, where we add value as physicians is understanding the mechanism of diseases. Our job is going deeper and deeper and understanding health in greater depth. So the more blackbox the algorithm is, the more difficult the culture mismatch is.
The main barrier is that we present things as closed loops, waiting for the machine to spit out an answer. Over and over again, in medical informatics, those sort of decision-support things that don’t involve physicians in the loop are not going to be trusted. We are meant to be critical of the information that comes in and carefully decide when to integrate that information. That’s why we need ‘white box’ data analytic tools that could easily onboard many clinicians.

What’s inside the box?

RD: You have an academic interest in cognition, is there a cognitive aspect to why doctors seem to have difficulty embracing data analysis and its toolset? Are they not visual enough?

MP: Sure, a high level of creativity can be communicated through graphical visualization. I made it clear that Data Analytics in healthcare is all about generating trust for providers. So, getting a visualization that is trustworthy and that lets us get an understanding of how variable x relates to outcome y is our dream.

“We need tools that are visual but also don’t restrict the connections that can be made.”

That’s why, in our “Healthcare by the numbers” project, we encourage our students to use the SPARCS database to understand how a patient’s background impacts their health outcomes. But we also have them focus on how they communicate that in order to advocate on behalf of their patients. They have to explain why this thing is related to that. We are not prescriptive of how we do that. Of course, being able to show how the algorithms work is a huge plus.

RD: What types of tools are comfortable for doctors to exercise their intuition and creativity rather than taking a prescriptive approach?

MP: Using SAS or writing a program is difficult. You need a degree to know how to do this and we, clinicians, spend our time learning about our healthcare subjects. So most of us use spreadsheets, even if our skills are inconsistent. Some providers won’t touch one (spreadsheet), others are very good with them. We need tools that are visual but also don’t restrict the connections that can be made. We are trying to maximize freedom of insight in the face of health IT systems that are often not interoperable. Today, people use mostly static spreadsheets. But we can see the day where data models will enable people to make connections between various Electronic Health Records. When this day arrives, we’ll need tools that are collaborative enough to span different health IT systems.

RD: Timing is key in Healthcare and doctors must be accustomed to making quick decisions. It often takes a long time for insight to be distilled from raw data. How do you deal with that?

MP: Let’s consider an example. In research, we have a long tradition of creating clinical decision rules. We use many techniques that, in some ways, are used in data-mining to determine which clusters of patients behave which way. Let’s be frank, it takes us years to develop a clinical decision rule. But once the data rule is built, things go much faster. Today, it unfortunately still takes us a long time to make sense out of data.

“We can see the day where data models will enable people to make connections between various Electronic Health Records.”

Researchers, IT guys are going to spend a great deal of time to gather, clean and aggregate datasets. So we clinicians are not going to use SPARCS datasets to make decisions at the point of care, rather we’ll make use of a compilation of tools that together help augment clinical decisions. There is not one tool for one problem. Whatever tools are available to you; they need to work in concert with each other to provide context at the population level as well as inform our view of the individual patient’s needs.

RD: It sounds like data analytics development has a lot to with evidence-based medicine. Do you agree?

MP: Well, yes and no. The evidence-based medicine story has a lot to do with the ‘big data’ story. It starts with researchers experimenting with these techniques. Then it gets mainstream and there is the hope that everyone can become a super-user. And some of them do become better clinicians if they understand evidence-based medicine. But you have to be realistic, we can not train every single physician to become as evidence-based as they could be. So instead,we carefully consider what evidence-based technique is for physicians, and what to instead push down in the IT system. And it’s the same story for Data Analytics.

There will be companies like Dataiku that will push tech to the edge but there will be at the same time different levels of expertise. You will have super users, general users and frontline persons that will have different expectations. We then have to define which part of the algorithm is useful to them and how we can make it available to the understanding of all of them. It’s just like electrocardiogram reading, which is ‘only’ 90% accurate. Being wrong 10% of the time is simply unacceptable from a clinical perspective. We need a synthesis between machine-learning and human understanding. The best way to achieve this by is using an open data analytics system that can increase human skills and understandings.

“Our ultimate goal is to keep patients away from hospitals”

RD: How far away do you think healthcare is from adopting ‘dynamic treatment regimes’ (algorithmically informed dosage adjustment)?

It will be some time, but we’re getting there. Being able to optimize inputs can save resources and help avoid unnecessary treatments. And I think this is the concept we are moving towards in terms of system-based practices. Every trainee we graduate now should be system-aware, which means the ability to increase the sampling rate: instead of seeing the physician every six weeks we can now dynamically sample some clinical data every second. It’s very much where the whole thing is headed. In the US, the big project has been to build EHR and getting people to use it. So we are dumping a lot of information into these boxes. Getting this information out and having a data-driven guide helping us to predict what happens next is on the horizon. We now need to make use of this data.

“Advanced modeling is the next step”

RD: Which other innovations do you see in Healthcare and how could Data Analytics be part of it?

Well, I only have a limited point of view but I’d say telemedicine could have an interesting impact on healthcare. Our ultimate goal is to keep patients away from hospitals, right? Alas, it’s very difficult to build systems to make sure they work in real time. But we managed to create a telemedicine division in our emergency department. Our emergency doctors are now supporting people who don’t have instant access to emergency medical care. It’s a good way to provide help and counselling to people waiting for the ambulance. It improves the response of the system from something that is measured in hours to something that is measured in minutes.

The other huge opportunity I see for healthcare is predictive modelling. A great example is the Veterans Hospital. They developed a Big Data system with 150 variables to let them predict who will need hospital admissions and who is at risk of dying in next 90 days. They collect the information on each patient and put that into a statistical model. It allows them to prioritize their patient outreach by health risk. When a huge hurricane struck New York, the VA hospital had to be shut down, but they still had access to their risk models and were able to reach out to high-risk patients in order to ensure they got the treatment they needed elsewhere. It was a kind of pre-compiled triage system. Advanced modeling is the next step. I hope our command of Data Analytics will soon enable us to do patient forecasting more accurately. Our belief in the reliability and validity of those techniques is going to be very important in the future.

Dr Pusic’s interview is now over. Thank you for reading! We will soon talk more about healthcare and big data. Meanwhile, go and have a look at our “Advanced Analytics for Efficient Healthcare” ebook!