Jul 29, 2020

6 min read

Process Mining with Python tutorial: A healthcare application — Part 1

This article is the first of a tutorial series made up of the following parts:

  • Part 1 (this article): Introduction to process mining, data preprocessing and initial data exploration.
  • Part 2: Primer on process discovery using the PM4Py (Python) library to apply the Alpha Miner algorithm.
  • Part 3: Other process discovery algorithms and model representations.
  • Part 4: More holistic models which integrate control flow, time (e.g. bottlenecks, wait times), resources (e.g. personnel capacity and performance, inter-personnel relationships, department/ward capacity and performance), case attributes (e.g. patient demographics, clinical condition).

We will be working through a series of exercises (originally intended for process mining software) which come from this Process Mining in Healthcare course. You can find the complete source code and data for this tutorial series here.

For a more general introduction to process mining in Python, you might want to check out this easy-to-follow article.

Stethascope next to a laptop.

Process mining in healthcare

In recent years, digitisation of healthcare records and systemetisation of healthcare processes has resulted in the generation of more and more data from complex healthcare processes. There has also been growing interest in using process mining techniques to optimise and debug processes to improve the quality and efficiency of care. Such techniques have been used to:

  • Discover processes and characterise them in process models. Different graphical languages such as petri nets, directly-follows graphs and business process models might be used to represent such models.
  • Discover bottlenecks and identifying opportunities for improving efficiency by analysing throughput and the time spent on each event.
  • Determine to what extent real processes adhere to those in good practice guidelines and treatment pathways.

What is a process?

In process mining, a process is sequence of events (discrete actions) that are executed to reach a particular goal or outcome. For example, we can characterise each patient’s hospital journey as a process, starting from when they are admitted to when they are discharged. Everything that happens to them in the hospital is an event that makes up this process.

Establishing cases, events and resources

For the purposes of process mining, an event log must contain:

  • case identifiers, which track entities through the events of a process, e.g. patient id, doctor id;
  • event names/descriptions/identifiers, which give us a handle on the event types that feature in the processes;
  • time stamps (associated with each event), which allow us to determine the order and duration of events.

Optionally, you will also have resources associated with each event, e.g. the equipment or medical professional involved. Event attributes other than timestamps and resources are also possible, e.g. transactional information, costs. Attributes are also possible for cases, e.g. patient age, diagnosis, outcome, flow time, contextual attributes.

The first thing to establish is what we are trying to find out and which columns we are treating as the cases, events and (optionally) resources. From this you’ll then be able to decide which column you are treating as the case, which you are treating as the event, and which you are treating as the resource (optional).

Here we give a worked through example, which you can follow through yourself by downloading the event log data here.

Here are the first few items:

Extract of the event log of hospital cases

Let’s say our objective is to better undertand the different journeys that patients go through, cases would be associated with the patient column, events with the action column, and resources with the resource column (but if we had a different objective, we might need to choose a different mapping, e.g. we might treat the resource column as the case if we were studying the processes from the perspective of medical practitioners rather than patients).

Preparing and exploring the event log

A first step in preparing the event log is to calculate the relative time of each event, which is time the event occurs with respect to the beginning of the process.

The dotted chart

To get an initial feel for what processes ‘look like’, it is useful to visualise the events associated with each case over (relative) time. This can be done with what is known in the process mining community as a ‘dotted chart’. You can implement this using a scatter plot or strip plot in Python, which plots the event sequences of each case against time.

Dotted chart of event occurrences within each case (patient)

This is quite difficult to read. To make the plot easier to read, we should order the cases by overall process lengths. This also gives you a better feel for the distribution of process durations.

Ordered dotted chart of event occurrences within each case (patient)

The dotted chart can also be used to get an idea of the event flow over absolute time, e.g. if cases are coming in regularly, if there are weekly or daily trends.

Dotted chart of cases over (absolute) time

This shows a steady flow of cases over time.

You can also study events with respect to the resources by plotting events with respect to them.

Dotted chart of events with respect to resources over time.

To investigate weekly trends, you first need to convert the time stamps into days of the week.

This time you should use a jitter plot rather than a strip plot because the x axis (day of week) is categorical.

Dotted chart of cases.
Dotted chart of cases over days of the week

Filtering events

One final thing you might want to look at is which events are shared by all processes and which are not, since in process mining it is the non-shared differentiating events that we are interested in

Heatmap showing which events were shared (and not shared) between cases (patients)

In this example, First consult, Blood test, Second consult, Physical test and Final consult are the events shared by all patients, while Medicine, Surgery and X-ray scan are the differentiating events.

Coming up next

In the next article we will start using the pm4py library to discover process models.