The Potential Consequences of Testing Data Reporting Criterion

As local governments try to reopen the economy, decision-makers would increasingly rely on testing and contact tracing data to decide whether it is safe to reopen an area. My previous articles have talked about criteria that are useful to gauge whether enough testing and contact tracing has been conducted to curb the spread of COVID-19, with positive test rates and the number of contact tracers hired as key metrics to look for in judging whether local governments are doing enough to safely reopen. While much of testing and contact tracing information is openly available, the detail and rigor of such information reported are different across states. This article will look at the importance of COVID data reporting criteria, especially for testing data, and the potentially dangerous public health consequences that unclear criteria could bring.

How Are Testing Data Reported in the US?

Testing data in the US is generally aggregated at the state level. Many states also provide some county-level testing data. Figure 1 shows how testing information is provided at official sources (generally state health department websites) at the state and county level, in terms of persons tested or specimens tested. Persons tested means testing numbers are reported as unique individuals tested, while specimens tested means testing numbers are reported as unique tests performed. Except states reporting both criteria and those not providing such information, about equal number of places are providing each criterion of measurement at both state level and county level.

Fig. 1 Testing Data Reporting Criterion at State and County Level (From State Health Department Websites.) Map by Ryan Wang.

It is obviously better to have both criteria for testing data reported so that decision-makers and researchers could benefit from both channels of information. However, when looking at testing positivity to judge whether there is enough testing, Johns Hopkins University’s Coronavirus Resource Center suggests to use persons tested to calculate positivity, as people with positive test results could be tested multiple times. As states gradually reopen, testing will also become more of a common occurrence, hence many more individuals might get tested more than once (e.g. getting tested for school reopening etc). Hence, using persons tested to decide if there is enough testing is crucial to prevent an overly-confident result.

Are We Really Safe to Reopen?

How do we see if a place is ready to reopen using testing information? A New York Times article has provided a national outlook on how testing data could help decide if it is safe to reopen in the context of reopening schools. This article concludes that most parts of the country are not ready to reopen schools, as most places are not doing enough testing, calculated using “positivity test rates”. But how big of a difference does using different testing criteria make in categorizing a place as ready/not ready to reopen? We will examine this difference for the two states that provide both specimens and persons tested information at the county level - Nevada and Maryland.

Fig. 2 Difference between Using Specimens Tested and People Tested in Gauging Positive Test Rates — Nevada. Map by Ryan Wang.

Figure 2 and Figure 3 show the counties that would be categorized as having enough testing and not having enough testing, based on a standard of 5% positivity test rates recommended by the World Health Organization. In this case, the positivity is calculated using cumulative positive tests over the cumulative total tests, for both specimens and persons tested (Data current as of August 2020). While the ranking/distribution of county-level positivity rates are the same with both criteria in both states, whether states are doing enough testing are showing different stories using the two different criteria. Using specimens tested would lead to overconfident results, identifying fewer areas not having enough testing. This trend is evident for both states that we are examining here, and the discrepancy is especially significant in Maryland (See Figure 4).

Fig. 3 Difference between Using Specimens Tested and People Tested in Gauging Positive Test Rates — Maryland. Map by Ryan Wang.

Using specimens tested instead of persons tested in calculating positivity rates would lead to a difference of 10 counties (out of 24) that will be falsely classified as having enough testing.

That is 42% of counties potentially opening too early.

Fig 4. Maryland Counties That Would Be Falsely Considered as “Safe” if Specimens Tested is Used instead of Persons Tested. Map by Ryan Wang.

These results are certainly alarming, as reopening too early could lead to new surges of cases, potentially stressing healthcare systems and causing yet more disruption to daily lives; but at least for states with known testing criteria, officials can look at their data with a grain of salt, keeping the impact of testing criterion on testing positivity results in mind. It is the notable amount of other states currently not reporting criterion of their testing numbers that is more concerning. Other issues in test reporting, such as CDC and states not distinguishing between test results from different methods of tests (molecular and antibody), are also worth concerning. As the nation prepares for more widespread testing, the importance of accuracy in test reporting, especially the availability of how numbers are reported (persons/specimens), becomes a more urgent need in painting accurate pictures of this pandemic.

• Ryan Wang is a Spatial Data Science Summer Fellow at the University of Chicago.