The Unbearable Lightness of the COVID-19 Mortality Rate

I’m mostly a reasonable person, and I like to think that I reach hair-on-fire panic less easily than most people. I don’t want to be accused of doomsaying, and I recognize this article might invite such an accusation. So let’s just nip that in the bud.

This article isn’t saying anything. It’s asking. This article could fairly be described as doom-asking, and I’m okay with that.

What I want to ask about is the deadliness of COVID-19.

It seems to me that the medical/media consensus is looking at the wrong numbers to instill the correct level of regard for the disease. (Yes, I’m using “level of regard” since that sounds unbiased. It may turn out that the correct level of regard is hair-on-fire panic, but that’s what we’ll be asking here.)

For the last month-plus, we’ve been hearing about a 1–3% mortality rate for COVID-19, with the lower end of that range getting emphasized by the more sober-sounding experts. Expectations for the USA (based on my subjective sampling) seem to be “we’ll see less than 1%.”

Note: I’ve seen the terms “mortality rate,” “lethality,” and even “death rate” used various places — mostly interchangeably. Apparently “mortality rate” is the correct term for a disease, while lethality is used for intentional-killing things, like “the lethality rate of rat poison.” For a bioweapon, I guess, one could pick their favorite term and be right either way.

Mortality Rate is computed exactly as one would naively expect: It’s the number of dead people (numerator) divided by the number of people who caught the disease (denominator). If 100 people catch Disease X and 32 of them die, epidemiologists would say “Disease X has a 32% mortality rate.”

Sounds simple enough — except that this is the real world, not a middle-school story problem.

Pinning down the numerator and the denominator in the real world is tricky as hell, both the actual data and even the definitions.

Definitions: the Case of the Viral Whodunnit

Saying that such-and-such a dead person died of COVID-19 seems straightforward. If the person is dead (easily checked) and has tested positive for COVID-19 (not as easily checked as we’d like, but theoretically possible), then this should be open-and-shut.

But apparently some countries are distinguishing between people who died with COVID-19 and people who died from COVID-19. If someone dies of organ failure — perhaps because of low blood oxygenation exacerbated by COVID-19 — does that count as a COVID-19 death or an organ failure death? Some bureaucrat will ultimately make this decision . But, annoyingly, different bureaucrats in different jurisdictions will come down on different sides of this fence…so the data’s going to be sloppy.

Related: Washington Post: Americans dying of COVID-19 but left out of official statistics.

Furthermore, there will almost certainly be COVID-19-adjacent deaths from people who can’t receive the care that they’d get in normal circumstances because their local medical facilities are flooded, their standard E.R. doctors overworked, etc. It’s tough to say how these cases should be counted…but they’re certainly relevant. (Pandemic collateral damage?)

Data-Gathering Without a Flux Capacitor

Dead people get noticed. Counting them is easy, and except in the weirdest and rarest of circumstances, deaths get reported without much delay. So our numerator (“number of dead COVID-19 victims”), while not necessarily accurate, is at least timely.

Our denominator, on the other hand — the number of people infected with COVID-19 — is a backward-looking number, likely to be wildly underestimating the true value at any given time. There are several reasons for this:

• We know that some people don’t feel sick from the disease. Unless such people have reason to suspect they’ve been exposed, they have no impetus to seek testing.
• Tests are (infamously) unavailable in many parts of the world, and elsewhere are rationed out to only those who have severe symptoms, infected relatives, NBA contracts, etc.
• There have been questions of the accuracy of some of the tests used. (False positives and false negatives are a normal part of medical testing, but the rapid development for COVID-19 tests has apparently made this a bigger problem than for diseases that haven’t caught us by surprise.)
• There is a time delay between infection with SARS-CoV-2 (the virus causing COVID-19) and the moment when enough viral proliferation has occurred where testing can yield a positive result.

Future medical historians will have mostly-accurate numbers for both the total number of deaths (normalized for whodunnit definitions) and the total number of infections. But, for us here in the present, we don’t.

To make matters worse, we get our numerator and our denominator at different times: rapidly in the case of the numerator, slowly for the denominator. So our calculated mortality rate (a mash-up of these two numbers) is inherently wonky.

As a comparison, take BMI (Body Mass Index). BMI is calculated as a person’s weight divided by their height. Implicit in this metric is the idea that you measure both the person’s height and weight at the same time. Imagine, instead, tracking a kid’s BMI as they go through a growth spurt — but with a significant time delay between gathering the weight and the height.

The resulting BMI wouldn’t be meaningless, but it would be inherently flawed because of the time lag.

Gathering data from a worldwide pandemic in real time, we can’t avoid the time-inconsistency between the two numbers in the mortality rate metric.

Good, Old-Fashioned Lying

Finally, we can’t ignore the fact that many officials will be motivated to make the numbers come out a certain way. Whether seeking to avoid inciting panic, dodging personal culpability, gaining international bragging rights for a “more effective response,” or making a brutal geopolitical calculus (“weaponized nonchalance”), there are many reasons why a jurisdiction’s death count might be willfully underreported.

China is the standout example here; it’s likely that China’s official number of deaths is maybe 5–10% of the reality. Many people were highly skeptical that the world’s second-largest economy would put three-quarters of a billion people under house arrest on the basis of a disease that had infected (according to their numbers at the time) just a few tens of thousands.

Get Yer Damn Story Straight

Considering all this, we have reason to expect that the official “Total Confirmed Cases” is a massive undercount. And there are equally compelling reasons to think Total Deaths may be a massive undercount as well. But there is no reason to think the magnitude of these undercounts will be equal (and thus “cancel each other out”). We might be that lucky, but don’t bet on it.

All of this is bad news for fans of numerical accuracy. But the one piece of good news we have is that as the once-epidemic jumped national boundaries to become a pandemic, we started getting diverse data streams from all over the world.

They’re all crappy data, to some extent, but their crappiness is smoothed out by the sheer volume. Individual nations may be fudging their numbers, but they’re not coordinating the fudging of the numbers. Definitions of “did this person die from COVID-19?” may be imperfect, but they’ll be imperfect in different ways.

The Law of Large Numbers dictates that as we get more data from more sources, the numbers should converge toward something more reliable.

The Number That Worries Me

Going by Johns Hopkins University’s COVID-19 tracker, at the time of this writing the global mortality rate is 5.34% (60,887 Total Deaths divided by 1,140,327 Confirmed Cases). This percentage has been creeping up over the past few weeks as the numbers from Italy, Iran, and Spain have been markedly worse than the numbers out of China.

But I’m not really hung up on the mortality rate. It’s not so much that I distrust it as I disrespect it — because of its reliance on the “Total Confirmed Cases” ingredient. Total Confirmed Cases seems to be the squishiest of the many squishy numbers we have to choose from.

The number I find more trustworthy is “Total Recovered” — because there’s a greater chance of accuracy in this number. For one thing, you can’t be counted as “recovered” if you haven’t first been made a “confirmed case.” The number of recovered people will always be an undercount — because there is a time-lag between someone being recovered in actuality and their doctor being super-duper-certain enough to sign off on their recovery.

These two numbers — “Total Deaths” and “Total Recovered” — constitute a catch-all for people for whom COVID-19 has run its course. On a long enough time frame, everyone who gets the disease will fall into exactly one of these two buckets.

That’s why the ratio of deaths-to-recoveries is the number I’ve been watching since January.

It has been rising dramatically over time.

(Rising is bad; each death is outnumbered by fewer recoveries.)

When the reported cases were almost all Chinese, the ratio hovered at about 1:11. In the past 36 hours, the total global ratio is now worse than 1:4. (As I type this, it’s 1:3.81.)

This should scare the bejeezus out of us.

The implication isn’t that people who get COVID-19 really have a one-in-five chance of dying. (Recoveries are an undercount, and slower to be certified than rather-more-obvious deaths.) But seeing this ratio get worse as China’s numbers get subsumed in the larger data set is compelling evidence that China was* lying.

* And that they still are.

As such, the “1–3% mortality rate” conventional wisdom that policymakers have been relying on is unrealistically low.

What’s your favorite scary number?

Am I right that the deaths-to-recoveries ratio is a more useful metric for COVID-19 than the mortality rate?

This is a matter of opinion, but it sure seems so to me, because there is less “wobble” in its constituent numbers. They’re more hemmed-in by strict medical observation.

An important point: Future epidemiologists will look back on these two metrics and see no difference. By then, they will have converged on a “correct answer.” This can’t be otherwise; eventually, deaths plus recoveries will equal the number of confirmed cases.

But right now, in our present moment, the equation is out of balance by nearly a million people.

For whatever reason, the metric that almost everyone has chosen to look at is, by far, the less scary of the two options.

But I wonder if choosing to focus on this less-scary number won’t lead us to a much more-scary outcome.