Quantifying the Impact of Covid-19 Restrictions on Mobility Around the World

Authors: Vincent Faber and Łukasz Bolikowski

Governments around the world have responded to the Covid-19 pandemic in widely different ways, but no country is immune from the fatigue effect most populations experience as a result of these responses. From closing down schools to partially shutting off public transportation to restricting national and international travels, policy-makers and health officials have access to a large set of distinct levers to reduce local virus reproduction rates and flatten the curve. The restriction stringency has a significant impact on mobility, consumer spending, and other consumer-activity indicators, but its effectiveness varies depending on how well individuals and organizations abide by them.

In this article, we present a modeling framework to estimate the impact that a given level of containment is expected to have on mobility. The results can be used to inform decision-makers in the public and private spheres and help them quantify the mobility impact of the restriction measures they implement.

1. The Data

Restriction and Containment Measure Data

An Oxford University research team has built and now maintains an impressive database¹ that spans more than 170 countries, allowing to track each national government’s response on a daily level across multiple indicators, including:

• Schools closure
• Workplace closure
• Public events restrictions
• Gathering restriction
• Public transportation shut down
• Stay at home orders
• Internal travel restrictions
• International travel restrictions

We first leverage the Oxford data to compute the Restriction & Containment Index (RCI). This index is defined as the projection of the data onto its first principal component. It constitutes a daily measure of the stringency of Covid-19 related containment measures across the world, as seen below:

Mobility Data

Google’s Covid-19 Community Mobility Report² provides granular daily-level mobility data. It is quoted in terms of percentage compared to a baseline computed by Google for each day of the week during the Jan 3rd to Feb 6th, 2020 period. In particular, this publicly available data set enables users to closely track the evolution of mobility in segments such as retail, grocery, or transit.

2. The Model

Model Definition

Now that we have our hands on some high-quality data, let’s see how well restriction stringency can help us explain variation in mobility around the world since the beginning of the year. The first step is to design a function linking mobility with the restriction stringency level. Additionally, we choose to explicitly model what we call the fatigue effect: In many countries, once restrictions have been in place for a few weeks, the general population grows tired of them and starts following the public health guidelines less strictly, which, in turn, leads to increases in both mobility and spending.

This model’s function might look a bit strange at first, but we will deconstruct it and explore each component separately in the next sections:

In other words, daily mobility (X𝚖) is assumed to follow a Gaussian distribution whose mean depends on the restriction level (X𝚛𝚌𝚒) and the amount of time since these restrictions were put in place (X𝚍).

Deep dive: Modeling the Impact of Restriction Measures on Mobility

For now, let’s ignore the fatigue effect term and focus on the role that the restrictions have on the expected mobility. We have:

This function is a good starting point. When the restriction index (RCI) is at 0, the modeled expected mobility is at 100%. As the restriction stringency increases, the modeled mobility gradually decreases, reaching its minimum when RCI is at 1 (e.g. stay-at-home order). Though we have not fitted the model yet, this is what the modeled expected mobility looks like if we were to let β𝚛𝚌𝚒=7.1 and α=2.9

Deep dive: Modeling the Impact of the fatigue effect on Mobility

As mentioned earlier, after having been in place for a while containment measures tends to become less and less effective and people start venturing out, regardless of infection trends. A striking example of such a phenomenon can be observed in the U.S.:

Though other factors are likely to be at play here, we make the assumption that the fatigue factor is responsible for a large fraction of this effect. In particular, we model its impact as a function of the number of days that have passed since the last time the restriction measures were increased:

This modified sigmoid function is a good candidate. For illustrative purposes, this is what the fatigue impact would look like if we let β𝚏=6, γ=159, and ε=0.046

Model Training

The next step is to estimate the value of the unknown parameters β𝚛𝚌𝚒, α, β𝚏, γ, ε, and S from the given data (X𝚖, X𝚍, X𝚛𝚌𝚒). For this step, we use a Bayesian framework. The priors for each of the parameters are defined below:

The expanded model formulation is:

Notes:

• Thanks to relatively low model complexity (2 variables, 6 parameters only) and the ease of interpreting the results, this modeling approach is quite robust to overfitting.
• We fit a different model for each country and mobility type (e.g. retail, transit, etc.) but the structure remains the same across models. Another approach could be to leverage Bayesian hierarchical modeling.
• We were able to further improve performance by factoring in weekly seasonality. Doing so, however, increased the model complexity and did not provide particularly interesting insights about the relationship between restriction stringency and mobility.
• Using the python Bayesian statistical modeling library PYMC3, we can derive posterior distributions for the model parameters using a Markov chain Monte Carlo algorithm. For instance, the trace and posterior distributions for the U.S. retail mobility model are displayed below:

3. The Results

Though performances will vary across geographies, this modeling framework yields overall good results while remaining simple, fully transparent, and interpretable. The framework is quite versatile and can be applied to different countries and mobility types, as illustrated in the next sections.

Use Case: Transit Mobility in the U.S.

We fit the model to the U.S. transit mobility data and obtain the following results

• In-sample MAPE = 4.7%
• Out-of-sample MAPE = 5.5%

Now that we have a posterior distribution for each of the 6 parameters (β𝚛𝚌𝚒, α, β𝚏, γ, ε, and S) we can use them to derive the estimated transit mobility level given a certain restriction stringency in the U.S.:

Interestingly, the U.S. has a strong fatigue effect that is clearly observable on the left-hand side graph. For instance, an RCI level of 0.8 is at first expected to cause the transit mobility level to drop to 60% of its pre-Covid-19 levels. However, the fatigue effect quickly kicks in and 2 months later the transit mobility is expected to be back at 80% of its baseline. The right-hand side graph highlights the standard deviation of the predictions’ distributions. This helps monitor how confident the model is, based on where the data point is located in the plane.

Use Case: Retail Mobility in France

We now fit the exact same model to French retail mobility data:

• In-sample MAPE = 16.9%
• Out-of-sample MAPE = 9.5%

One again, we can now use our trained model to derive the estimated transit mobility level given a certain restriction stringency in France:

Unlike the U.S., the fatigue effect seems basically absent in France. Looking at the left-hand side graph, it becomes clear that the amount of time that has passed since the restriction stringency increase has no real impact on the predicted mobility level. In fact, deep-diving into the model’s trace, we find that the mean of the posterior distribution for β𝚏 is 0.005, which is one of the lowest values across all modeled countries. We also notice that the prediction’s variance tends to be higher than that of the U.S. transit mobility model.

Use Case: Fatigue Effects Around the World

After training the model for all countries, we can compare the mean of the fatigue factor’s posterior distribution across countries. This constitutes a good quantitative measure of how relatively effective each country’s containment measures are likely to be as time passes.

Closing Remarks

• This modeling approach helps to estimate the impact that a given level of restriction stringency is likely to have on various mobility indicators. The fatigue factor in particular provides an interesting way to assess how quickly the effectiveness of containment policies can wear off in different countries.
• By leveraging the Restriction & Containment Index, the current modeling framework does not differentiate between restriction measures. Moreover, it does not address subnational level responses. Further iterations could solve this shortcoming by considering distinct containment measures such as school closure or national travel restriction, and explicitly model their individual and combined impact on mobility at a finer geographic granularity.

Sources

1. Oxford COVID-19 Government Response Tracker, Blavatnik School of Government (Thomas Hale, Sam Webster, Anna Petherick, Toby Phillips, and Beatriz Kira. 2020). GitHub: https://github.com/OxCGRT/covid-policy-tracker
2. Google LLC “Google COVID-19 Community Mobility Reports”. Accessed: July 2020. https://www.google.com/covid19/mobility