2 years of AI-lab at Ruter — Part 2
How our innovation hub helps make public transport better
This is part 2 of a series of articles about the AI-lab at Ruter. In the first iteration of AI-lab, the students were to figure out a way to help the customer support team, read part 1 here! For the 2021 iteration, a new group of four students were tasked with trying to predict delays across Ruter’s huge transport network.
2021 — Mobility and travel time predictions
Delays are a burden for travelers using public transportation. Travelers experience uncertainties about whether they will arrive on time. An internal analysis from Ruter estimates that delays are 2.5 to 6 times more burdensome for travelers than the actual travel time itself. Models for accurately predicting these delays in real-time are therefore needed, and the team of four students set out to use machine learning models to achieve this.
The current method for calculating estimated time of arrival uses a reactive approach. When a delay is detected on some link, the rest of the links for the journey are adjusted by this delay. This method assumes that a delay on a link does not compound for the downstream links, but this is often the case if there is a lot of traffic or during rush hour. The goal of the 2021 AI-lab was to try a data driven approach for predicting these delays proactively using ML. The solution would have the potential to:
- Improve user experience by providing more accurate information on display screens and apps
- Give users information about expected delays when planning their journey
- Facilitate for dynamic route planning
The arrival and departure times we used came from GPSs located on the buses. Unfortunately, GPS data has a tendency of being unstable at times. Sometimes the signal is either lost or lagging because of coverage issues, and this produces odd positioning data. Also, obstacles such as tall buildings can disrupt the GPS signal. The logic used for registering arrival and departure on a given stop is called bubble-logic: arrival is registered as the first time a bus enters a stop’s bubble, and departure is defined as the first registration outside a stop’s bubble.
This logic can sporadically give faulty data that is difficult to discover unless one dives into the data and look at specific routes or bus stops, which is hard when one is processing millions of rows. A significant amount of time was therefore spent categorizing bus stops and routes with multiple errors, and then deciding if it would be beneficial to simply remove these datapoints from our dataset to improve the performance of our ML models.
The whole process we followed for solving the task, from gathering data to analyzing and determining the best ML model, can be summarized in the following main steps:
Gathering bus data:
Before fetching our desired data, we had to get an overview over the data warehouse and the different tables it contained. Scripts were written in SQL, fetching data from a PostgreSQL database.
Gathering external data:
To increase the predictive capabilities of future machine learning models, we wanted to add relevant external data. Historical weather data and traffic data were therefore fetched from the internet and merged into our dataset. However, due to a coarse resolution of traffic stations, among other things, this data had a minimal impact on our machine learning models.
Data cleaning:
The raw data was not at a usable format, while also containing several erroneous registrations and incomplete trips. Lacking data and duplicates were also problematic. To handle this, a python script heavily relying on the pandas library was developed. This script was the result of a longer process of trial and error using Jupyter notebook and AWS Sagemaker.
Feature Engineering:
We found a significant correlation between the travel time of a bus on a specific route and the travel time of a bus driving the same route earlier. With inspiration from time series analysis we created several features we thought could help the model - including what route was being driven as categorical feature seemed natural. This meant that the team encountered “The Curse of Dimensionality” as even our small selection of bus routes contained hundreds of different stretches between two stops. Instead, it was decided to include this information using different averages for different stretches, while also avoiding a memory leak in our dataset, more specifically making sure that averages provided as features did not contain data that appeared later than the current data point.
Training machine learning models:
There were a lot of freedom associated with the testing of different machine learning models. We were encouraged to try both machine learning models we knew beforehand and learn new models. A linear regression model was used as a baseline, and an extract of the models that were tested against it are: “Ridge regression”, “GAM”, “Neural networks” and “Boosted tree models”. Boosted tree models, such as XGBoost, yielded the best results, though the improvement from the baseline linear regression model was minor. This arose from the large correlation between the travel time and the feature “mean of the last 10 travel times for this link”. Our main tools for this part were Amazon Sagemaker with python.
Prediction:
After training a machine learning model it can be saved and used to predict travel times for new datasets. A continuation of our assignment could include productionizing the model with a pipeline that fetches bus data, performs data transformation and feeds the data into the model. The predictions could then be sent to the Ruter app and display screens.
Analysis of ML-model:
After the model selection, we dug deep into the details of the model predictions and residuals. Among others, we wanted to know how much better this model performed than the current solution, get a visualization of the error for the different links of a given bus route, find where the model performed badly and why. The answers were visualized through different plots, and the analysis became the main event of our presentation at the end of the summer.
Since summer of 2021, the logic for estimating delays have been further optimized, so the comparisons might look different today.
To summarize the summer of 2021, we were 4 students working with the task: “Using machine learning models to predict travel times in real-time”. The summer concluded with a presentation about new discoveries, problems, results, and an analysis of the results. Our work with data gathering, data cleaning, results and insight in the data laid the groundwork for the two of us that continued as interns working part-time. As interns working part-time, some of the things we did was to look at how machine learning models perform when we omit features involving the immediate past, and prediction further into the future. The part-time internship concluded with an analysis of data using new techniques for registering arrival and departure.
In 2022 we had several interesting use-cases for ML in public transportation, stay tuned for part 3 of the AI-lab article series!
Written by Sindre Skaugset Olderkjær and Fredrik Wilhelm Butler Wang
