Understanding Agent-Based Models via Elara
Introducing a new open-source tool for wrangling outputs from MATSim agent-based transport models
This article introduces Elara, a tool for wrangling outputs from agent-based transport models using MATSim. Elara helps analysts translate massive, semi-structured data into useful formats and summaries and aids in model calibration and validation. Elara is now open source and our code is available on GitHub.
At Arup’s City Modelling Lab, we build lots of agent-based simulations. These types of tools allow us to directly observe how synthetic individuals behave and make complex decisions as they interact with their environment and each other. For modelling transport scenarios, we like using MATSim — an open-source ABM framework in which agents seek to optimise their transportation choices (mode choice, time-choice, and route-choice, etc) between various activities over a 24-hour period.
One difficulty with building ABMs in MATSim is they are very data hungry in terms of inputs. But on the output side, they also produce huge amounts of data.
The primary outputs of a MATSim run comprise a detailed manifest of everything that happened across the simulated day. These files inventory individual agents’ choices and second-by-second events on the network in lengthy XML documents, which can grow to many gigabytes in size for complex simulations. These data need to be parsed and transformed to produce the higher-level analytical insights our projects require.
That’s why we built Elara — a tool to help us delve into our simulations, understand the results, and calibrate MATSim’s underlying models. Elara lets us easily produce both traditional summaries (e.g. mode-shares, hourly flows, and link speeds) and microscopic information about how agents in the model have chosen to execute their daily plans. It also features tools to compare how well a MATSim simulation performs against real-world, observed data.
And best of all, Elara’s outputs come in familiar, easy-to-use tabular and geospatial formats, like CSV and GeoJSON, to make further analysis a breeze.
How Elara Works
Elara is designed to efficiently stream through the contents of MATSim XML files element-wise, handing down data to one or more individual handlers, which are responsible for parsing and processing the information into tabular or spatial outputs.
This structure has two key advantages. First, streaming the XML avoids loading large documents into memory, allowing Elara to work even with geographically large and complex simulations. Second, each handler is designed to know exactly how to analyze the data it receives. Delegating responsibility for processing and writing outputs makes it easy add to new features quickly in response to emerging project needs.
The list of available output handlers can be found on the README, and is updated periodically to reflect new and improved features.
Handling Plans and Events
MATSim produces two core outputs: plans and events.
The output plans file reports how each agent executed their daily activity sequence.
It describes when each activity took place, for how long, and what travel modes and routes the agent used to move between them. It also contains the agent’s attributes and the final “score” of the plan — which can be useful to compare across scenarios.
Some of the most widely used plan handlers in the Lab include:
- Modeshares and counts, optionally filtered by specific activities;
- Sequenced lists of individual agents’ activity, trips, and trip-legs; and
- Scored agent plans
While the plans file is focussed on agent-choices, the events files catalogues many of the physical aspects of the simulation. For instance, MATSim records an event each time a vehicle enters or leaves a link on the network, tagging them with an event type (“vehicle enters link”).
Agent interactions with the public transport network are also stored as events, including waiting at a train station, or boarding or alighting a bus. Monetary transactions, such as paying a toll, are also events. Analysed temporally, the events can be used to reconstruct aggregate roadway volumes, or a single agent’s speed along their route.
Some key event handlers outputs include:
- Hourly link volumes and speeds;
- Counting the number of riders between public transport stops;
- Getting passenger boardings, alightings, and wait times at public transport stops; and
- Calculating how much agents pay in tolls for road user charging scenarios.
Elara also includes some detailed “logs” which simply list out events without further aggregation — examples include a record of each vehicle’s entry/exit times on every link in the network and public transport vehicle delays at stops. While these non-summarised output files can be very large, they allow us to produce bespoke project analyses that are difficult to anticipate.
A separate post-processing module re-ingests outputs from other tools to do some useful calculations, such as binning trip distances and durations, calculation of vehicle-kilometers of travel from traffic volumes on links, and summarising agent plans.
Benchmarking
One of Elara’s most powerful features is that it provides the ability to evaluate the quality of a MATSim run by automatically comparing the outputs to (user-provided) real-world data.
We use benchmarking to evaluate many different aspects of a simulation. For example, we use survey data (either census or travel diary) to determine whether the modal splits in our model resemble real people’s choices. Given traffic count data, we evaluate whether there is sufficient traffic on the roadway network at the right times of day. Is the same true for public transport boardings and alightings at public transport stops? At the micro-scale, we use journey-time data obtained from various APIs to validate the trip distances and speeds for individual agents.
We carefully examine benchmarks closely so we can make our models more realistic by tweaking MATSim’s parameters or changing some of the physical aspects of our simulations (such as the free-flow speeds on the network). At the beginning of a project, failing benchmarks can also help quickly diagnose problems in the network or population inputs.
Recently, we’ve added new tools to Elara that evaluate how closely a given simulation configuration can reproduce its own inputs. If agents in the model are synthesized directly from travel diary data, for instance, one measure of how well a model is performing might be if MATSim’s utility function is correctly steering agents into choosing the same mode of travel stated in the survey for a particular trip.
Manual examination of the outputs is important, but still requires time-intensive human intelligence, which may not be possible when you need to compare large numbers of simulations. To make at-a-glance evaluation possible, each benchmark in Elara is also scored using a loss function score (usually, mean absolute percent error). Users can weight each benchmark to reflect its importance to calibrating the simulation, which may vary by project. In the near future, we will be implementing an auto-calibration framework which uses Elara’s benchmark scoring to explore simulation parameter spaces and quickly calibrate models.
Dive deeper into impacts
Many of Elara’s tools support additional options that allow you to drill down into the results of your MATSim simulation, helping craft a more complete and compelling story from the model.
A key feature of Elara is the ability to break down results by arbitrary agent attributes. This is especially important to us at the Lab, where many of our analyses focus on the distributional impacts of various transport schemes and policies.
For example, a change in public transport fares and concessions might impact different income groups differently. Understanding how this policy affects modeshares for different types of agents is as easy as configuring the trip_modes handler with the appropriate options.
Results for most handlers can also be disaggregated by the modes configured in your simulation. This is useful in a variety of cases, for instance counting boardings/alightings at a shared station by regional versus local buses or bringing focus to how many agents a link is carrying by different modes.
Installing and Running Elara
Elara is written in Python 3.7 and can be easily installed from source from our GitHub repository using pip.
Elara ships with a command-line interface. The usual way to invoke an Elara run is via a terminal session via the run command using a relatively straightforward configuration file, written in TOML:
elara run elara_config.tomlYou can learn more about the full range of handlers and configuration options in the project’s README file. The config is broken out into a few logical groups, corresponding to the available tool-types (plan handlers, event handlers, benchmarks, and post-processors), along with some required contextual and path information.
Within the config, you can request Elara run any number of available tools (or “handlers”), which each result in one or more outputs.
The general format for requesting a handler is handler_name = {options}.
If you don’t want to include any additional options, you can also use handler_name = [“all”] .
Available options vary by tool and, in most cases, Elara will cheerfully ignore any invalid options passed. Two of the common options alluded to above are disaggregating results for different types of agents or modes.
For instance, the following config would instruct Elara to produce separate modal counts and shares for different income and age bands as well as CSV and GeoJSON outputs with volumes on the road network separated out by bus and car. Note that the benchmarking tools do require a special path argument to tell Elara where to find the comparator dataset.
[plan_handlers]
trip_modes = {groupby_person_attributes=["income", "age"]}[event_handlers]
link_vehicle_speeds = {modes=["bus", "car"]}
Benchmark tools require a special path option, which tells Elara where to find the benchmark dataset, e.g.:
[benchmarks]
link_counter_comparison = {modes=["car"], benchmark_data_path=/path/to/your_data.json"}A brief caveat: with all this power comes great responsibility! Overloading your configuration with too many handlers or multiple breakdown options can significantly increase runtimes and result in some very unwieldy file sizes. It’s usually best to start with a minimal configuration and expand it as project needs dictate.
Side note about the DAG
A neat feature of Elara is that it will always try to work out what needs to be done, and in what order, using a directed acyclical graph (or DAG).
Each handler in Elara is coded with a specific set of requirements. When a run is initiated, Elara builds a graph from the config, and ensures that the run will proceed in an order that won’t result in a failure because some other tool should have been run first.
Less abstractly, to calculate vehicle-kilometers of travel, the vktpost-processor needs to know the volumes on each link. Rather than perform this task a second time, the DAG ensures that the link_vehicle_counts handler will run with the right options even if the user didn’t request it.
This means that users experience fewer errors due to mis-configuration and that failures mostly happen early in a run. It also reduces redundant calculations, making Elara run more efficiently when there are many overlapping requirements.
Contributing to Elara
We think that more transparency and generosity in the planning disciplines is absolutely key to achieving a more sustainable future. The City Modelling Lab is proud to open Elara to the MATSim community worldwide.
If you’re a MATSim modeller, we hope you will try incorporating Elara into your next project to see how it can improve your workflow and provide feedback on how we can improve the software.
We welcome external contributions to the project as well. If you want to get involved, the best place to interact with us is via our GitHub issues page. You can alert us to bugs, request new features, and ask questions. You can also contribute code directly via a pull request.
