What do two negative Lateral Flow tests tell you about your likelihood of being Covid-negative?
Number 105: #USSbriefs105
Andrew Chitty, University of Sussex
Please see two additional notes at end of this Brief.
Until now the standard test for Covid-19 has been the PCR (Polymerase Chain Reaction) test. But in the last six weeks a new type of test, a Lateral Flow test made by Innova which gives a result in 30 minutes, has been rolled out for mass testing in Liverpool and Merthyr Tydfil, and in hospitals and care homes. This test has also been offered to students at over 100 universities before they leave at the end of term. However the new tests have been widely criticised as insufficiently accurate for purpose (BMJ 17 Nov 2020, Guardian 1 Dec 2020, Guardian 3 Dec 2020, BMJ 4 Dec 2020, Telegraph 4 Dec 2020, Financial Times 11 Dec 2020).
This brief addresses the question of just how much two negative Lateral Flow tests tell you about your likelihood of being Covid-negative, specifically in relation to students who have been tested in the last two weeks. The answer will be that it all depends on the results from other students who were tested in the same programme.
We begin by summarising the typical course of a Covid-19 infection, explaining how a PCR test works, and distinguishing the sensitivity and the specificity of a test. Then we summarise the findings of the two published studies of the sensitivity of the Lateral Flow tests, and finally come back to our title question.
1. The typical course of a Covid-19 infection
For what follows, we shall assume that the typical course of a Covid-19 infection for an individual is:
(1) Day 0: the individual is infected.
(2) Day 3: the individual becomes infectious to others (and remains so for about 12 days thereafter).
(3) Also on day 3, the individual also becomes ‘Covid-positive’ (and remains Covid-positive for somewhere over 15 days thereafter).
Here for an individual to be ‘Covid-positive’ on a given day means that they would test positive for the Sars-CoV-2 virus in a well-administered PCR test on that day. Conversely for them to be ‘Covid-negative’ on a given day means that they would would test negative in such a test on that day.
From the last two assumptions we can say that being Covid-positive on a given day corresponds roughly to being infectious on that day.
All this is for an individual who becomes symptomatic. However a recent study (Ra et al, 22 Sep 2020) has found that the viral loads of individuals who contract the virus but remain permanently asymptomatic are similar to those of individuals who become symptomatic, and one expert has commented that ‘many studies have shown the same thing’ (Tang, 22 Sep 2020). We can conclude that all of the above, except for assumption (3), also holds for individuals who remain permanently asymptomatic.
2. How the PCR test works
Next we need to say something about how the PCR test works. A letter to the BMJ explains that viral culture is ‘the gold standard for confirming the presence of viable, infectious virus’ but is both difficult and expensive to perform. A PCR test is the nearest widely available equivalent:
PCR tests work by repeatedly doubling the amount of genetic material in the original sample, until there is a detectable quantity of it. Each doubling is referred to as a “cycle”; and the number of cycles or doublings before there is a detectable quantity of genetic material is called the “cycle threshold” (CT or Ct). The larger the amount of viral RNA there is in the sample, the smaller the number of cycles that are required before it can be detected. And since the number of cycles is the CT value, the lower the CT value, the more virus there was in the original sample, and the more likely it is thought to be that the case was actually infectious, rather than still carrying leftover RNA, which is not clinically significant. (English, 23 Aug 2020)
So the ‘viral load’, meaning the quantity of viral RNA in an original sample taken from a patient, can be expressed as a Ct value, with a low Ct indicating a high viral load and vice versa. Therefore we can label the vertical axis in figure 1 with Ct values that reduce as we go up the axis. The top horizontal line represents a Ct value of about 15. The bottom horizontal line represents a Ct value of about 40, in that normally a PCR test is stopped after 40 cycles (PHE, Oct 2020, p. 6).
This means that, to be accurate, for an individual to be ‘Covid-positive’ on a given day means that their sample would show a detectable quantity of Sars-CoV-2 genetic material after 40 or fewer cycles in a well-administered PCR test on that day, and likewise for ‘Covid-negative’.
3. Sensitivity and specificity
We have to distinguish two kinds of accuracy of a test for Covid-19:
(1) The sensitivity of a test is the percentage of those who have the virus who test positive using this test. By contrast those who have it but test negative are ‘false negatives’. So:
Sensitivity (%) + false negatives (%) = 100%
(2) The specificity of a test is the percentage of those who do not have the virus who test negative using the test. Those do not have it but test positive are ‘false positives’. So:
Specificity (%) + false positives (%) = 100%
With regard to our question, what matters is the sensitivity of the test rather than its specificity. The higher the sensitivity, the lower the percentage of false negatives, and the lower the chance, if you have a negative result, that you have the virus. A specificity of less than 100%, i.e. the existence of false positives, raises a different problem which we will not pursue.
We have defined sensitivity relative to ‘those who have the virus’. By this standard the PCR test is not 100% sensitive, since it does not detect the virus for the first 3 days after infection (see section 1 above). Even after day 3 it is not 100% sensitive. However as we saw in section 1, being Covid-positive on a given date corresponds roughly to being infectious on that date. So it is reasonable to pose our title question in terms of ‘Covid-negativity’, as opposed to ‘not having the virus’.
For the same reason, it is reasonable to measure the sensitivity of other tests relative to PCR results. This is how the sensitivity of the Innova Lateral Flow test has been measured in the two field studies on it done so far: the one by Porton Down and the University of Oxford, and the other by the University of Liverpool.
4. The sensitivity results of the two studies
The Porton Down and University of Oxford study (8 Nov 2020) used 6,967 patients. It had four phases, of which the two field studies were phase 3b and phase 4.
In phase 3b the tests were “either by laboratory scientists at PHE Porton Down or by fully trained research health care workers at the testing site” and the sensitivity was measured for different viral loads. Sensitivity varied dramatically, from 100% for highest viral loads (CT < 18) down to about 33% for lowest detectable viral loads (CT > 35). (See figure 2). Across all individuals with a detectable viral load the sensitivity was 76.8%.
In phase 4 the sensitivity was measured for different kinds of testers. Across all individuals with a detectable viral load the sensitivity was 79.2% when the test was used by “laboratory scientists”, 73.0% when it was used by “trained health care workers” and 57.5% when it was used by “self-trained members of the public given a protocol”.
The Liverpool study (dated 26 Nov 2020, published 11 Dec 2020) used 3,199 patients tested with “military supervised self-administered swabs”. Again it found that the sensitivity varied dramatically, from 85.7% (12 out of 14 cases) for highest viral loads (CT < 20 ) down to 11% (1 out of 9 cases) for lowest detectable viral loads (CT > 30). Across all individuals with a detectable viral load the sensitivity was 48.9%.
Both studies were available to the government before it published its Community testing: a guide for local delivery (30 Nov 2020). Its statements there summarise — in a very broad brush way — the results of the studies
In field evaluations, such as Liverpool, these tests still perform effectively and detect at least 50% of all PCR positive individuals and more than 70% of individuals with higher viral loads in both symptomatic and asymptomatic individuals.
and then a little later:
In the field evaluation in Liverpool, compared to PCR tests, these tests picked up 5 out of 10 of the cases PCR detected and more than 7 out of 10 cases with higher viral loads, who are likely to be the most infectious.
5. Discussion
Let’s apply these results so as to answer our title question.
First, for our purposes we should ignore the differences between the sensitivities found for different viral loads, since for each of these viral loads we are looking at a group who are equally Covid-positive. So the sensitivity figures that matter are just those figures across all individuals with a detectable viral load.
Second, the two studies found that sensitivities across all such individuals varied greatly according to who was administering the test. Summarising the above results we get the following sensitivities:
Military supervised self-administered swabs (Liverpool): 48.9%
Self-trained members of the public given a protocol (Porton Down / Oxford): 57.5%
Trained health care workers (Porton Down / Oxford): 73.0%
Laboratory scientists (Porton Down / Oxford): 79.2%
At universities, where a lot of scientific expertise is typically available, it seems reasonable to assume that the standard of testing will be towards the upper end of this spectrum. So we shall assume that the sensitivity of Lateral Flow tests in this context is about 70%.
Third, the sensitivity of two successive tests will be greater than that of a single one. If there is a 70% chance of getting a false negative from one test, there is only a 30% x 30% = 9% chance of getting two false negatives in a row. So the sensitivity of a double test is 91%. That is, a double test will pick up 91% of Covid-positive individuals.
Fourth, at this point we need to introduce a new concept: the negative predictive value (or predictive value of a negative test). This is the likelihood that someone who gets a negative Lateral Flow test result is Covid-negative. The negative predictive value is calculated in two stages:
(1) Multiply the prevalence of Covid-positivity in the population being tested by the percentage of false negatives in the test. This gives the chance that someone who gets a negative test is nevertheless Covid-positive.
(2) Subtract this percentage from 100%. This gives the chance that someone who gets a negative test is Covid-negative.
So the negative predictive value depends both on the sensitivity of the test and on the prevalence of Covid-positivity in the population being tested. The only exception is the special case where the sensitivity of the test is 100%, in which case the negative predictive value is automatically 100% as well.
The Liverpool study gives a negative predictive value of 99.23%. The method it used to calculate this figure must be as follows:
Total tested (excluding voids): 3026
Total PCR positive: 45
Covid-positive prevalence in the tested population: 45 / 3026 = 1.49%
Total of those who are PCR positive who are also Lateral Flow positive: 22
Sensitivity of Lateral Flow test: 22 / 45 = 48.9%
Percentage of false negatives: 100% - 48.9% = 51.1%
Chance that someone who tests negative is Covid-positive: 51.1% x 1.49% = 0.77%
Negative predictive value: 100% - 0.77% = 99.23%
Let us do this calculation again with the same prevalence but assuming that the sensitivity is 91% (corresponding to two successive tests each with a 70% sensitivity):
Covid-positive prevalence in the tested population: 1.49%
Sensitivity of Lateral Flow test: 91%
Percentage of false negatives: 100% - 91% = 9%
Chance that someone who tests negative is Covid-positive: 9% x 1.49% = 0.13%
Negative predictive value: 100% - 0.13% = 99.87%
So someone who took part in this reimagined version of the Liverpool study and got two negative tests would have had a 99.87% chance of being Covid-negative, or only a 0.13% chance of being Covid-positive. This seems impressive. However we need to remember that even if this person did not know their test result they would still only have a 1.49% chance of being Covid-positive. Knowing that they had a double negative test reduced their chance of being Covid-positive, but from an already low base.
From all this it follows that if I am a student who has had two negative Lateral Flow tests, then to find out how likely it is that I am Covid-negative I need to know not only the sensitivity of a double test (which we have assumed to be 91%) but also the prevalence of Covid-19 among the population tested at my university. For this I need to ask my university:
(1) How many students it tested in the period 30 November to 11th December (excluding void tests)
(2) How many of these students tested positive either on the first or second test.
I can then take the number who tested positive, multiply it by 100/91 (assuming a sensitivity figure for the double test of 91%), and divide the result by the total number tested to work out the prevalence of Covid-positivity in the population tested. From this I can use the above method to work out the chances that I am Covid-positive or Covid-negative, on the day of the second test.
Of course for this calculation to be valid I have to be a typical member of the tested population in the key relevant respects. But this seems plausible if I have been part of a university testing programme, for here the tested population are all students at my own university, they are all like me asymptomatic, and they are all like me sufficiently health-conscious to take the test. If I am untypical then this will change my chances. For example, if I am much more cautious about social distancing than the average member of the tested population then my chance of being Covid-positive will be reduced, and so on.
An immediate conclusion is that universities need to publish the above two pieces of data immediately. Without it students will be unable to form an assessment of how likely it is that they are Covid-negative after getting two negative Lateral Flow test results — whether now or on returning to university in January and February 2021.
___
Additional note 1 (21 December 2020)
The main text has now been clarified by adding the italicised words to this sentence:
‘So someone who took part in this reimagined version of the Liverpool study and got two negative tests would have had a 99.87% chance of being Covid-negative, or only a 0.13% chance of being Covid-positive.’
Also, the main text says that to get the negative predictive value of a Lateral Flow test we have to “multiply the prevalence of Covid-positivity in the population being tested by the percentage of false negatives in the test” then “subtract this percentage from 100%”. However this is not fully accurate. To show this, see the Liverpool results (adapted from the second table in the report):
Sensitivity = true positives / all PCR positives = 22/45 = 48.89%
Specificity = true negatives / all PCR negatives = 2979/2981 = 99.93%
The correct figure for the negative predictive value is true negatives divided by all negatives, which is 2979/3002 = 100% - (23/3002) = 99.2338%.
Instead the figure reached by using the method in the main text is 100% - (prevalence x percentage of false negatives), which is 100% - [(45/3026) x (23/45)] = 100% - (23/3026) = 99.2399%.
As can be seen, the difference is very small. It will always be small as long as the prevalence as measured by the Lateral Flow test is low, i.e. as long as the total of Lateral Flow positive tests (here 24) is small compared to the total number tested (here 3026).
Correspondingly the method given in the last paragraphs of the main text for calculating the negative predictive value of a Lateral Flow test is also not fully accurate. The method given was in effect:
‘Divide the number who tested positive by the total number tested, divide this by the sensitivity, multiply the result by the percentage of false negatives, then subtract the result from 100%.’
This can be summarised in the formula:
NPV = 1 - [(number of positive tests / total tested / sensitivity of test) x (1 -sensitivity)]
A fully accurate calculation of the negative predictive value of a test needs to take into account its specificity, as well as its sensitivity and the prevalence in the population tested. For a further explanation see Boskey, 11 December 2020, and for the exact formula see Positive and negative predictive values, Wikipedia.
Additional note 2 (28 December 2020)
The main text says ‘we can say that being Covid-positive on a given day corresponds roughly to being infectious on that day’, but this is inaccurate. The literature suggests that although the periods of infectiousness and of PCR-positivity start at about the same time (namely about two days before the onset of symptoms) the period of infectiousness is much shorter. This is because the PCR test detects not only ‘live’ infectious virus but also ‘dead’ noninfectious viral genetic material that can remain in the body for some time after the immune system has cleared the infectious virus. For a summary see Deeks et al 22, September 2020, and for a graphical representation the final figure in Jefferson et al, 5 August 2020.
This paper represents the views of the author only. The authors believe all information to be reliable and accurate; if any errors are found please contact us so that we can correct them. We welcome discussion of the points raised and suggest that discussants use Twitter with the hashtag #USSbriefs105; the authors will try to respond as appropriate. This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
