So last night I wrote a tweet about how terrible the general thinking was around in-epidemic CFR (Case Fatality Rate).
So here I am to pay the piper and write up my thoughts around it.
The big problem with the way everybody approaches this is they are thinking in terms of absolutes. Now there is some precedence here, during the SARS epidemic WHO calculated the CFR at 4% (link) when ultimately it ended up at 11% (link). This is the way people are currently thinking about CFR.
Let’s say we have 100 people get infected, at 1 person dies immediately (but the other 99 are still infected, and have just gotten the sickness)
In this scenario everyone is quick to claim the CFR as 1%, and then further say “oh and since we’re not doing enough testing it’s probably more like 0.2%”, and 0.2% doesn’t seem scary, so nothing to worry about! Alright… I’m being a bit facetious there but this is more or less the rationale I see constantly and everybody using the 1% number as if it’s a fixed thing.
The problem of course is that, well, 99 of these people are still infected. Until they have resolved either through death or recovery it’s just pending. So in this case the FLOOR of CFR for this cohort is 1%, and we only stay at 1% if 100% of the remaining infected cases make a full recovery. How likely is this? Well let’s look at some current stats from major countries around the world ranking from best (lowest) to worst.
Deaths per Recovery:
South Korea: 0.06, Japan: 0.20, Germany: 0.34, Spain: 0.49, Italy: 0.78, UK: 1.06, US: 5.5, Switzerland: 6.75, France: 12.33
Now anything over 1 I’m going to just treat as anomalous for the moment so let’s focus on some of the more optimistic cases. Korea has done the most testing of any country so we’ll start with that. At the time of writing this they currently have 8,320 confirmed cases, 81 deaths, 1,407 recoveries, 6,832 active cases. This puts their current CFR at just about 1% (0.97%). However they still have 82% of their cases sitting unresolved, and they are one of the countries with the most resolved cases. If rather than declaring this as their current CFR we extrapolate their 0.06 death/recovery rate out over the remaining population it would come to 372 more deaths, 6,460 more recoveries once everything was all resolved and this would put the CFR in Korea ultimately at 5.4%.
Obviously if we run the same numbers in the other countries things get way way worse, again I’ll only focus on those <1.0 for now but their ultimate CFRs would look like this: Japan: 16.76%, Germany: 25.55%, Spain: 33.11%, Italy: 43.97%
So do I actually believe this is what the ultimate CFRs will look like?
Let me just go ahead and get that out of the way. I’ll also address all the common criticisms.
“But Bryan, testing is so poor all of these people with mild infections don’t ever get diagnosed. It’s off by at least an order of magnitude in many places”
“But Bryan, Death/Recovery will surely improve as this goes further along”
Will it? I mean.. it should, and I hope it does, but up until this point this has been the progression of death/recovery for the countries that I’ve tracked regularly (Canada, UK, UAE, Germany, Australia, Thailand, HK, USA, Italy, Iran, Singapore, Japan, S Korea, Spain, France) starting Feb 25th
Now there are a TON of factors that impact this, too many to accurately model at this time. Among those are places like Italy having their health system becoming overwhelmed and likely only logging the most severe of cases, things like the US likely currently under testing by a factor of 10–100, and just so many others.
But the point of this entire article isn’t for me to tell you what the accurate CFR is. I have no idea. The point is for me to tell you to STOP thinking about CFR the way you are currently, and start looking at the ratio of outcomes because until we get more (most) of these cases resolved we won’t have a good picture of what the actual fatality rate is and underestimating it (and thus diminishing the risk) because you’re looking at the earliest deaths/infections when 80–95% of the patients are still sick just seems like such a dangerous thing to do.
Since some people have asked me for a stronger conclusion/summary, my summary is this (edited in): The reason the world (partially) is in this mess is because people have a very hard time dealing with growth and projections, everybody likes to think in terms of absolutes. “Oh it’s killed less people than the flu” (while not looking at an R0 of >2x and a mortality rate trending much higher) and so on. I’d ask that people focus on 2 things, the growth rate of infections and the ratio of those outcomes (death and recovery). Until you see both of those start tending downwards I would continue to operate with an abundance of caution. My family and I have been self quarantined for almost a month now and we are in Canada where the numbers are still relatively low for the moment although given how poor mitigation measures and testing standards have been thus far it seems almost certain those numbers will rise soon. Until I see each of those numbers begin to trend down heavily I likely won’t be out any time soon, not because I’m at significant risk or any of my family, but because we would be needlessly putting many others at risk and I do have friends and extended family who do have significant risk. So please, rather than saying “well we only have 50 cases” or “we only have X deaths” look at the rest of the world and think of where it’s headed. Just 2 weeks ago UK had 50 cases and France, Spain and Germany were just nearing 200 and look at where things are headed for them now. So focus on trajectory and not absolutes, monitor growth rates, and stay safe ❤.
Ok fine, we’ll think about it differently but what are you going to do? You’re just an idiot with a keyboard who happens to like numbers, what are you doing to help? I’ve called and offered my help to almost every agency in both the US and Canada and many of the hospitals, I’ve also offered an entire team of developers with extensive experience developing for healthcare (CDC, Cigna, major hospitals, etc.)… both entirely for free, for as long as they are needed. Locally my wife and I have also offered to go and get groceries or any necessary supplies for any high risk person (old, immunocompromised, etc.) and drop them at their door.
Ultimately I am just an idiot with a keyboard (who happens to like numbers), but I hope this post will help change the absolute-ist way that everyone is looking at CFR and focus more on outcomes and growth rates.
If you want to get in touch with me feel free to message me here (is that a thing on medium?), twitter, or however else you can figure out how to reach me. Questions, comments, reminding me I’m an idiot, all totally welcome!
If you are working on something to help with covid, please get in touch I want to help and will provide all of the resources I can. Ideally direct (non-commercial) help at the government or hospital level is preferred. You need developers, someone to crunch numbers, working on a solution with AI. Fine, all fine, hit me up.