To lock down or not to lock down? An evidence-based approach to anti-covid measures
As you might have noticed, we have been in the middle of a pandemic for about nine months now. There has been much talk, and much controversy, about what does and does not work to counter the spread of the covid-19 (coronavirus disease 2019) inducing agent, known as severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2).
Just the other day I was having a conversation about this with a follower on Twitter, who was rather skeptical of government lock-downs. He presented me with some home generated graphs drawn from public databases that seemed to make his point. I was, however, a bit skeptical of his skepticism. At some point I thought, wait a minute, surely by now there are serious peer reviewed studies on this! Let’s take a look.
Sure enough, a quick Google Scholar search turned out a number of peer reviewed papers. I picked two in particular, on the basis of three criteria: they are very recent (both published this month), they are fairly comprehensive in terms of datasets and anti-covid interventions, and they were published in the two top scientific journals in the world: Nature and Science.
You can download the full articles here (Nature) and here (Science), but of course they are fairly technical, especially in terms of methodology and statistical analyses. So I’ll do my best to summarize the key findings, because they have tremendous consequences for public discourse on covid as well as on the implementation of public policies.
Let’s start with the Science study, which is less wide ranging, as it focused on a small number of NPIs — nonpharmaceutical interventions. The authors, Jan Brauner and collaborators, evaluated the impact of several NPIs on 34 European and 7 non-European countries. They used a single, but very powerful approach, known as a Bayesian hierarchical model. Their results proved to be very robust, and they discovered that some NPIs outperformed others under all tested conditions. Here is the rundown:
NPIs with large effects: closing down schools and universities; limiting gatherings to 10 people or less.
NPIs with medium effects: closing most non-essential businesses;
NPIs with small effects: issuing stay-at-home orders.
Interestingly, as the authors report, “closing most nonessential face-to-face businesses was only somewhat more effective than targeted closures, which only affected businesses with high infection risk, such as bars, restaurants, and nightclubs.” Also not surprisingly, “issuing a stay-at-home order had a small effect when a country had already closed educational institutions, closed nonessential businesses, and banned gatherings.” I guess once you do that, there really isn’t much point in getting outside one’s home anyway…
A nice feature of the study is that the authors set up an “epidemic forecasting calculator” (here). You can use it to play with the individual and combined effects of different NPIs. For instance, if you don’t limit gatherings of people at all, the effectiveness of your overall measures will drop from 73.7% to 54.6%. By limiting gatherings of 1,000 people or less you get back up to 64.9%, which you can improve to 70.2% by getting more restrictive and limiting gatherings to 100 people or less. It’s a very effective educational tool, give it a try.
The Nature study is by Nils Haug and collaborators, who looked at a much larger number of NPIs using three different datasets spanning 79 territories, meaning both countries and different states within the US. The authors used four different kinds of data analyses: case-control analysis, so-called LASSO time-series regression, random forests, and transformers. It would be far too technical to go into what these are, but the essential point is that the paper focuses on results that were robust across techniques, which raises confidence in the robustness of the findings.
Turns out that the most effective NPIs were: curfews, lockdowns, closing or restricting places for people to gather in small or large numbers (shops, restaurants, etc.), closing of educational institutions (both schools and universities), and border restrictions. The issue of school closure has been particularly controversial in the US, and early research showed that this was not a very effective anti-covid measure. But the new research indicates otherwise: “school closures in the United States have been found to reduce covid-19 incidence and mortality by about 60%.” That’s a lot. Moreover, a contact-tracing study carried out in South Korea clearly shows that adolescents aged 10–19 are more likely to spread the virus than either younger children or adults.
The least effective NPIs are environmental measures like cleaning and disinfecting objects and surfaces in both public and semi-public spaces. So much for all that scrubbing down my wife and I have been doing since the beginning of the pandemic! Surprisingly, the authors found no evidence for the effectiveness of social distancing on public transportation, though they suggest that this may simply be because usage of public transport has dropped significantly, so there is a natural social distancing already at play.
Haug et al.’s study also addressed a number of nuanced issues. For instance, while they found national or state-wide lockdowns to be effective, they remark that a lockdown is actually a combination of different NPIs (closure of borders, closure of schools, closure of non-essential shops, prohibition of gatherings, and prohibition of nursing homes visits). This is crucial, because it means that if a government acts early on it can avoid a complete lockdown by implementing more targeted NPIs. A lockdown, then, is advisable only if things have already span out of control. As the authors put it: “a smaller package of such measures can substitute for a full lockdown in terms of effectiveness, while reducing adverse impacts on society, the economy, the humanitarian response system and the environment.”
And there are also a number of educational or support strategies that also work: communicating the importance of social distancing and wearing masks (especially when politicians don’t make it into an ideological issue) and promptly enacting government assistance programs — so that people feel more comfortable self-isolating if they don’t fear losing their job, their home, or not been able to put food on their table.
Moreover, some NPIs had different effect depending not only on when, but also where they were implemented. Here is a perhaps not surprising and yet sobering observation: “[the] gross domestic product [of a country] is overall positively correlated with NPI effectiveness whereas the governance indicator ‘voice and accountability’ is negatively correlated.” That is, richer countries are responding better, while countries that are politically unstable, or where the government is less effective, are doing worse.
So this is the best evidence we have, so far. Of course, we are talking science, not magic. So both studies have limitations, and their findings may will likely be partly revised by future, better studies. Nevertheless, the next time someone tells you that social distancing, school closings, closing of businesses, etc. don’t work, you have the data to counter the claim. Assuming, of course, that the person in question cares about evidence. But that’s a different story.