Evaluating Interventions on the Spread of Corona
An outline on how to estimate causal effects.
In the current coronavirus outbreak, governments face hard decisions on how to best combat the spread of the virus. Policymakers are operating under considerable uncertainty when trying to weigh costs and benefits. Attempts at causal estimation of the effects of different interventions have been few and far between. Randomized trials have been, more or less, out of the question due to the need of acting fast and most of the other classic techniques used to estimate causal effects, not being applicable due to the lack of reliable data.
Specifically, in order to evaluate policy interventions, we need reliable and comparable measures of contagion. Few countries have enough test capacity for confirmed cases to be a reliable measure of actual spread. Using confirmed cases is problematic also because testing policies differ between countries and change over time. Hospitalizations and deaths are more accurate, but even deaths are not counted the same everywhere[1]. Moreover, these measures suffer from a considerable time lag. Italian statistics puts median time from symptom onset to hospitalization at four days and to death at nine days[2]. Incubation adds another five days[3]. This is especially troublesome in times where conditions and policies are rapidly changing. Many non-pharmaceutical interventions (NPI), like lockdowns, carry substantial costs and should to be lifted as soon as one can do so without risking unnecessary damage. Causal estimates of NPI effectiveness are in dire need.
Each direct measure of SARS-CoV-2 spread is riddled with problems. In this PM we suggest an alternative measure, using the spread of other contagious respiratory viral infections as a proxy for SARS-CoV-2. Many of these infections have a much faster incubation time than SARS-CoV-2 and can therefore be used to, faster and thus with more precision, measure effects of NPIs. In this document, we present an outline of the methodology we have in mind and and quick example of an application. We are publishing this outline with the hopes of receiving feedback and to initiate discussions with other researchers thinking about similar questions.
An Alternative Outcome Measure
To increase estimation efficiency, we need an outcome measure that responds faster to policy changes (shorter incubation) but still follows similar transmission mechanics as SARS-CoV-2. Influenza, the common cold and other respiratory viral infections seem to be natural candidates. They spread similarly to the Coronavirus and have an incubation time of one to four days, compared to two to fourteen for Covid-19[4]. Policy impacts on the spread of these diseases should therefore be detectable within days, allowing for quick evaluation of the effectiveness of the policy.
The spread of these viral infections are however not currently reliably tracked in detail. Most countries’ testing capacities are being refocused and regorinaized to deal with SARS-CoV-2 and hence a change in the number of confirmed cases could be due to time-variant measurement error. Furthermore, people who suspect that they have the influenza (or a similar infection) could, during a pandemic, act differently compared to how they would otherwise. Specifically, they might be more hesitant to visit a hospital. This hesitation is likely to increase as the knowledge and severity of the Corona outbreak increases, again creating measurement error.
Therefore, the alternative we propose is to study online search queries for respiratory disease symptoms. A US survey found that 65% of Americans google their symptoms when sick[5]. Hence, a high positive correlation between search intensity and disease spread is to be expected. The graph below shows the weekly confirmed cases of influenza and Google-searches for fever in Sweden 2016–2019. The correlation is 0.89. A reduction in search intensity should therefore indicate a drop in the actual prevalence of said infections.
We are of course not the first to suggest that online searches can be used to predict contagion. Online searches have been found to be able to predict the prevalence of a disease accurately and quickly and are in fact being used for that purpose by e.g. Folkhälsomyndigheten (the Swedish CDC)[6]. Assuming SARS-CoV-2 transmission is similar enough to other respiratory infections, we can thus use the same procedure in conjunction with classic empirical identification strategies to estimate the effect of NPIs, more or less, immediately after implementation.
Proof of Concept — Comparing Norway and Sweden
To fight the spread of SARS-CoV-2, Norway instituted a lockdown on March 12 and a travel ban on March 13. Schools, kindergartens, fitness centers, hair salons etc. were closed and sports events and cultural and gatherings were banned. Its neighboring country Sweden, similar in many ways, has taken a more lenient approach; relying on recommendations and information rather than enforced restrictions. We believe a traditional difference-in-difference approach could be applied in this setting to estimate the effect of NPI stringency.
If there is an effect of the Norwegian intervention, we would expect a divergence in the search patterns for fever within a few days after the intervention was introduced. The graph below shows the daily searches for fever in Sweden and Norway normalized at the day of intervention.[7] The difference in search intensity between the countries is also shown.
Looking at the graph it seems like the difference in searches (relative to the search intensity at the date of the intervention) between the countries before the intervention is small. Within a few days of the intervention however there seem to be an increase in the difference with Swedes becoming relatively more likely to search for the term fever.
There is considerable daily variation in the graph above. To get a clearer picture we average the search intensity over weeks and index the time series to the week before the intervention. These results are shown in the graph below. The pattern now becomes very clear. Before the intervention the trends are basically parallel. However, already in the first week after the intervention a significant gap of about 22 points has developed. The gap persists until the beginning of April. Given the fact that the trends were parallel before the intervention it is not unreasonable to assume that, in the absence of the intervention, they would have continued to move in conjunction. Hence, the observed gap after the intervention can be thought of as an “effect” of the intervention.
This makes policy evaluation possible. For example, let us assume that the relative changes in search intensity in the two countries does, in fact, reflect underlying relative changes in the viral infections causing these symptoms. Let us further assume that the estimate of the correlation between searches for the word “fever “ and actual cases of influenza of 0.89 is valid in this setting. We can then can translate the “effect” on searchers to an “effect” on viral infections causing fever. Under these assumptions, the “effect” of the Norwegian lockdown on the prevalence of viral infections would be about 20 points.
Going forward
The above text is a rough sketch of how to use Google Trends in conjunction with data on the timeing of introduction of NPI:s to study the effect of said policies. In order to produce precise point estimates with a clear interpretation, additional data needs to be collected. Specifically, we hope to:
- Add additional interventions
- Add more countries/areas/regions
- Add mobility data to measure the intensive margin intervention response
- Add searches for additional symptoms
- Add additional search phrases
- Add more and better search engine data (from general search engines or preferably medical ones such as webMD). Specifically the interpretation would benefit from actual numbers of searches instead of indices.
- Do the analysis for coronavirus spread directly by studying the amount of search queries for anosmia (loss of sense of smell), a seemingly unique and early coronavirus symptom.[8]
More effort also needs to put into the link between the search patterns and the prevalence of contagious viral infections. For example, we do observe a spike in searchers for fever in most counties around the time when interventions are implemented. This spike is likely not correlated with actual infections but rather due a general interest in Corona and its symptoms. This should potentially be corrected for as it means that the actual effect on viral infections could be underestimated.
Another relevant question is when, where and for what interventions this setup can be used. The prevalence of contagious viral infections varies geographically and seasonally meaning that the methodology might be invalid (or at least in need of adjustments) in other contexts. Further, potential differences in how SARS-CoV-2 and other contagious viral infections are transmitted might invalidate this approach when it comes to certain interventions. Related to this is also how these causal estimates can be translated to relevant metrics used in epidemiological models.
Finally, we should be clear that all of the authors of this article are economists trained in causal empirical analysis. We do not have any expertise in virology or epidemiology and are humble before to the fact that we might have missed a crucial point that makes our suggested approach fruitless. If you believe so, please get in touch. We decided to publish this text now to hopefully initiate collaborations with anyone that can contribute, time is after all of the greatest essence here. So if you are an epidemiologist that have thoughts about our idea, or work for an institution with access to symptom search data that you are able to share (anonymized of course) please contact us as soon as possible.
Written by: Dany Kessel, Ph.D in Economics (dany.kessel@sh.se), Adam Altmejd, Ph.D in Economics (adam.altmejd@hhs.se), Elisabet Olme, Ph.D in Economics (elisabet.olme@gu.se), Elin Molin, Ph.D in Economics and Nils Lager, BA in Economics (nils.lager@alumni.hhs.se).
[1] Some countries count all cases where Covid is the cause of death, other count each patient who was Covid-positive at time of death. https://www.thelocal.fr/20200311/coronavirus-death-rates-why-do-they-vary-so-much-between-countries
[3] According to WHO’s Q&A on the coronavirus, estimates of the incubation period ranges from 1 to 14 days, but most commonly around 5 days (the Q&A is continuously updated, these numbers were collected on April 9, 2020).
[4] See eg. Centers for Disease Control and Prevention’s information about the seasonal influenza (https://www.cdc.gov/flu/about/disease/spread.htm).
[5] See https://www.phillyvoice.com/new-survey-finds-googling-symptoms-problematic/.
[6] See, for example, https://www.nature.com/articles/nature07634 and https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0004378
[7] Data downloaded from Google Trends the 5th of April.
[8] See e.g. https://sniffoutcovid.org/.
