Got a Positive COVID Test? Here’s What It Really Means — part I
As fall approaches and the temperatures drop, I find myself battling COVID for the third time since 2020 (sigh), despite being fully vaccinated and boosted. Fortunately, my symptoms are mild, but I still need to rest and self-isolate. More than four years after the pandemic began, COVID may no longer dominate the headlines, but it’s still a persistent issue in society — and in some cases, a serious one. As of September 2024, COVID-related deaths still account for around 2.5% of the total in the US!
What’s perhaps even more troubling is the lasting impact of the virus. Around 10% of those infected go on to develop some form of “long COVID”, a condition marked by ongoing symptoms that can persist for months after the acute infection. Fatigue, brain fog, and shortness of breath are just a few of the debilitating issues long COVID can bring. And beyond the personal toll, its economic impact is staggering. David M. Cutler, professor of applied economics at Harvard University, estimates that the cost of long COVID in the US could add up to $3.7 trillion in reduced quality of life, lost earnings, and higher spendings on medical care.
With all this in mind, it’s easy to see why taking even mild symptoms seriously is important. At first, I thought my sore throat and headache were just signs of a regular cold. But the next day, when my symptoms worsened, I found myself back in that familiar pandemic mindset: “What if it’s COVID?” Sure enough, a rapid test confirmed it.
It had been a while since I last used a COVID test, so I had to re-read the instruction manual to take the test correctly. That’s when I started thinking about the numbers behind these tests. If you’ve ever looked at a COVID test manual, you’ve probably encountered terms like sensitivity and specificity — key concepts that are thrown around with percentages without much explanation. These terms might seem intimidating, but at their core, they’re really all about simple Bayesian probability!
In today’s post (and the next one), I’ll demystify sensitivity and specificity, explain how they’re calculated, and show why they’re crucial for understanding the reliability of medical tests with several examples. Whether you’re into numbers or just want a clear understanding of how tests work, I’ve got you covered. Let’s jump in!
A binary test
Let’s start from the beginning. At its core, a COVID test — or any diagnostic test that detects a pathogen — produces one of two possible outcomes: a positive result, meaning the pathogen has been detected, or a negative result, meaning it has not (for simplicity, we’ll ignore inconclusive or faulty results). This kind of test is called a binary test because it offers only two distinct outcomes.
The picture we’ve painted so far assumes the test is 100% reliable. Of course, this is an idealization — there is no such thing as a perfect test! If the test correctly identifies when the pathogen is present, that’s great — we call this situation a true positive. At the same time, if the test correctly detects that the pathogen is absent, we call this a true negative. A good test obviously needs to correctly predict both the true positives and true negatives. Failure to do so leads to two key problems.
First, the test may fail to detect the pathogen even when it’s present. This can happen if the pathogen is present in very low amounts, the sample is collected improperly, or the test is conducted too early in the infection when the virus hasn’t reached detectable levels. This means that the person is already infected, but the test fails to detect it. We call this situation a false negative. False negatives can be dangerous because they give a false sense of security, leading individuals to believe they are disease-free when they are actually infected. This can result in delayed treatment and increased risk of spreading the disease to others, especially in the case of contagious illnesses like COVID.
On the flip side, a test may incorrectly indicate the presence of a pathogen when it’s not there. This situation is called a false positive. False positives can occur due to contamination, detecting fragments of dead pathogens, or other technical errors. This results in a person being incorrectly identified as infected when they are not. While often seen as less harmful than false negatives, false positives can lead to unnecessary stress, isolation, and medical interventions, while also diverting resources from those who actually need care. Moreover, they can distort public health data, leading to overestimated infection rates.
From our discussion so far, then, it’s clear that a good test needs to possess these four attributes:
- It maximizes the amount of true positives.
- It maximizes the amount of true negatives.
- It minimizes the amount of false negatives.
- It minimizes the amount of false positives.
As we shall see when revisiting the contingency table, it turns out that points 1 and 4 are really two faces of the same coin. The same applies to points 2 and 3. So, the focus should really be on maximizing the true positives and true negatives. Sensitivity and specificity are nothing but technical terms for these concepts.
Sensitivity, Specificity, Accuracy — In words
Sensitivity measures the test’s ability to correctly identify the pathogen in those who actually have it (i.e. the true positives). More precisely, sensitivity is defined as follows:
Sensitivity = True Positives/(True Positives + False Negatives).
Sensitivity tells you how good the test is at detecting positive cases, and it is typically given as a percentage, such as 95%. This means that out of 100 people with the pathogen, the test correctly identifies 95 of them (and gives 5 false negatives). A highly sensitive test will catch most people who are actually positive, but it may also pick up some false positives.
Specificity measures instead the ability of a test to correctly identify those who do not have the condition (i.e. the true negatives). Specificity is defined as follows:
Specificity = True Negatives/(True Negatives + False Positives).
It tells you how good the test is at ruling out people who are not sick. A highly specific test will accurately identify most people who are negative, but may miss some true positive cases.
Sometimes, you will also see the term accuracy used to describe the test. Accuracy is a measure of the overall correctness of the test and takes into account both true positives and true negatives and shows how often the test gives the correct result, whether positive or negative (as such, it can actually be directly obtained from the sensitivity and specificity if we know the prevalence of the condition):
Accuracy = (True Positives + True Negatives)/Total Population
Note, however, that accuracy alone can be misleading if the condition is rare, as it doesn’t distinguish between the ability to detect positives versus negatives. That’s why sensitivity and specificity are typically the main quantifiers for a test.
Revisiting the contingency table
Now that we’ve defined the necessary concepts, let’s dive into the math behind them to better understand the information in the contingency table from the introduction.
The data compares two tests: the rapid test I used and a more accurate PCR test, which serves as a baseline. This is why the manual refers to relative sensitivity and specificity — it’s impossible to provide absolute values since we can’t determine the presence of the pathogen with absolute certainty. The experiments measured the performance of the two tests against each other, rather than against a perfect standard, as real-world data always contains some errors. For the sake of this discussion, though, we’ll assume that the more accurate PCR test provides a 100% reliable baseline.
I’ve rewritten the table below for clarity. The color coding indicates the true positives and negatives in green, and the false positives and negatives in red.
Now that we have the definition of sensitivity, specificity, and accuracy, it’s an easy task to calculate the corresponding percentages. We just need to sum the table entries corresponding to true positive, false positive etc. according to the definition. This results into the values reported at the bottom of the table. As we can see, the values are all quite high, which demonstrate that the test is fairly reliable (but we will discuss the details on how to gauge these numbers next week).
Sensitivity, Specificity, and Accuracy are Bayesian probabilities!
Now that we know what sensitivity and specificity are, and how to calculate them (or read them out of a contingency table), let’s see how they are related to Bayesian probability. We will assume a frequentist interpretation of the data contained in the tables, meaning that we can convert between data and probabilities by using the naive definition of probability:
probability = favorable outcomes / total outcomes.
For a given random person who takes the test, let’s define the following events:
- Event D: the person has the disease (e.g. they are infected with the pathogen in question). The complement of this event, D^c, describes the person not having the disease.
- Event T: the test taken by the person turns out to be positive. The complement of this event, T^c, describes the test being negative instead.
Given these two events and their complements, we can reconstruct all the cases contained in the contingency table. We simply condition on the actual status of the person (diseased/non-diseased). The ones we’re mostly interested in are the following two:
- P(T|D) is the probability that a person tests positive given that they have the disease. This is nothing but the probability of a true positive, i.e. the sensitivity of the test.
- P(T^c|D^c) is the probability that a person tests negative given that they don’t have the disease. This corresponds to the probability of a true negative, i.e. the specificity of the test.
Sensitivity and specificity are just conditional probabilities! By the same token, we can also obtain information about false negatives and false positives:
- P(T|D^c) is the probability that a person tests positive given that they don’t have the disease, i.e. the probability of false positive.
- P(T^c|D) is the probability that a person tests negative given that they actually do have the disease, i.e. the probability of a false negative.
Note, however, that this information is redundant if we already have know the values of the sensitivity and specificity, since P(T|D^c) = 1-P(T^c|D^c) and P(T^c|D) = 1-P(T|D). This follows directly from the definition of the probability for a complement of an event. In other words, the probability of false positives is one minus the specificity and the probability of false negatives is one minus the sensitivity, so knowing those two quantities already convey the entire information contained in the contingency table. This discussion then justifies our initial statement that sensitivity and specificity are really all that we need to describe the performance of a test.
The table below summarizes all the conditional probabilities we encountered with the same color coding we used for our original contingency table.
Why does that matter?
We’ve proved that sensitivity and specificity are simply conditional probabilities. At first glance, it may seem like we’ve just restated the same concepts in a different language. By reframing everything as conditional probabilities, though, we’ve actually achieved much more. First of all, we’ve translated our wording into rigorous mathematical concepts. This formalism allows us to leverage the full power of Bayesian probability, including Bayes’ rule and the law of total probability, which help us better understand the relationships between sensitivity, specificity, and accuracy.
Most importantly, instead of focusing solely on the probability of testing positive given a person’s disease status, what we are really after is the reverse: What is the probability that I have the disease, given that my test is positive? By expressing everything in terms of conditional probabilities, we can easily calculate this using some basic algebra.
Next week, we will do just that by exploring several concrete examples. In the meantime, stay tuned, stay safe, and get tested!
