The PCR’s false positive

The latest subject making the rounds on Covid today is that of the PCR test being crap because it can give false positives of 0.8%, most notably this article by Michael Yeadon. When you dig at this being something you should take note of… you just sigh at how a complex subject has been ridden roughshod over by militant sceptics, sowing doubt in the very thing to release us from this madness soonest.

Worse, the source of this 0.8% is this Lancet review, which cites this study, which confirms no assessment has yet been made to measure it for C19 labs, and that the 0.8% was an estimate based on previous uses of PCR assays for other viruses, so a guess.

UPDATE: see below, since this review the situation has changed.

At the heart of this story, like most, is people being happy to let confusion sow doubt. The reality is there are two concerns about PCR tests, and both are being called FPs:

1. random error: process failures, x-contamination, mistakes, cross testing with other pathogens, etc — i.e. things that randomly affect any sample
2. systematic error: conscious policy choice of the level that you set the cycle threshold (CT/CQ) to, leading many to think weak cases are FPs.

What makes it worse, is people like Kevin in this thread on twitter, who are core parts of this narrative, are happy to let this confusion reign, as they disagree with the latter, and want support for their argument, so are happy to let the confusion remain. This conflation has led people to believing the scale of the former is far larger than it actually is.

Digging into these two in more detail.

1) So what process failures can lead to false positives?

The most important rating for a test is its specificity. This means does it only tell you if you had the disease in question. As per this study into this very subject the SARS-CoV-2 PCR assay has 100% specificity. Meaning it will only detect this virus, and cannot be positive in the presence of something else.

This misguided take in the Spectator is right about some of the causes, the real world — e.g. samples being polluted, labs not all being equal, etc. Further, as per this study we need to be careful in dealing with a false positive rate. As per wikipedia’s definition:

FPR = FP / (FP + TN), where FP is the number of false positives, and TN is the number of true negatives.

So there is risk as the true case rate gets small, and similar to this rate, that you’ll find it hard to distinguish between real cases and noise.

The problem with the rumour is we already have all the evidence we need to know this is being exaggerated. For some time we’ve known that cases cannot be taken in isolation. With the cases/tests, or positivity rate, being a good indicator of whether case concentration is truly rising, here is the plot of that rate of Pillar1&2 testing.

We also know that, when cases are low this rate will converge on the FPR, or the lowest possible positive rate you can sample without just getting noise, but importantly, that the lowest nadir we’ve seen has to be the most the FPR can be.

Crucially, though as per this government note, the ONS uses the same Pillar2 labs to do its prevalence testing.

So we are left with two elephants in the room of the FPR theory.

How can the FPR now:

1. be higher than the July nadir?
2. have been higher than 0.05% in July, given this was the ONS prevalence survey managed to show through the same labs, at a time Pillar1&2 symptomatic testing was showing a rate 8x higher?

It can’t, worse case we had 100% FPs in July for the ONS, and 12.5% for Pillar1&2, and since, much lower.

UPDATE: since first publishing this article, the ONS have actually done this study, concluding that the July 0.05% measure had an FPR of 0.005%, so 10% of prevalence +ves were FPs, meaning only 1.25% of the 0.4% symptomatic +ves, were FPs, so what the hell was @juliahb1 going on about?

So the whole accusation of random error is just a militant contrarian wet dream.

The key part of that study is this plot, which also blows the argument of a dead virus pandemic out of the water. All cases were not at a CT of 45. The median is 25, in the 2nd wave with just 0.21% >37.

The data is made freely available by the ONS as part of the infection survey here.

Worse, when you push people like Kevin (in the thread above) on this, and they will agree with this conclusion about the random error, but they do so in such a dismissive way, purposefully allowing the ambiguity to remain as it helps there fight against their true gripe, use of systematic error to find old cases (which we discuss in the next section).

To make the dynamics of random error clear, some observations:

1. The FPR is a proportion of ALL tests, not just +ves. So as the +ve rate goes up, the proportion the FPs are of that +ve rate falls — e.g. 0.005/0.05=10%, but 0.005/0.5=1%.
2. If testing levels stay similar, but the +ve rate goes up, sure something might have changed about the testing process, but it seems more likely that it is just an increase in prevalence, which is what we’ve likely seen heading from the summer into the 2nd wave.
3. Being constant seems unlikely. If cross contamination is the main source, the higher the +ve rate, the more there will be. So this 0.005% will have gone up. Likely more driven by Pillar2 samples in the labs, so if they’ve gone from 0.4% to 10%, this might have increase x-contamination 20 fold, meaning the FPR is now 0.005 x 20 = 0.1%.
4. With this dyanmic nature being entirely why the 0.8–4% estimated from prior studies should not be taken so literally. What were the prevalences for these studies? If the 0.8% had 20% and the 4% had 75%, these immediately become x-contamination rates in very different circumstance, not like the 0.1% prevalence we had in July.
5. Most labs know about this, and when samples are weak, they rerun them to make sure it was not x-contamination. They can do this as they are a smaller proportion, and symtomatic test demand is likely lower too, so lab capacity will be more available. Something they can’t do as easily at 20%, which will increase the FPR, but this is entirely the reason the govt has added capacity without a linear increase in tests.
6. If the FPR was 1%, and we genuinely think there are 0.1% of cases, a jump to 1.6%, means you’ve gone from 0.1% to 1.6%, which is a jump of 1600%, which means we have a much larger R0 than you’d have in a jump from 1% to 1.6%, which is only 60%. So ironically it makes their point that we should be more worried.
7. Being blind in March, testing too little meant we did not know the true case rate. If now we don’t trust that serology tests for just IgG antibodies tells us about all types of immunity, we need a way to triangulate the data differently. We know hospital numbers and deaths, what we need is a more accurate way to sample true cases, asymptomatics included, to plan and manage case loads in a more targeted way.
8. Add to this, to avoid lockdowns we need to know that test&trace is staying on top of outbreaks, keeping the rate of cases from exploding, and below NHS capacity. The ideal would be to run cases just below capacity to true herd immunity. That deaths lag cases means, without being able to see a relationship between cases and deaths, we have no early warning system until it is too late.
9. Crucially, against the biggest complaint that this will subject 1% of false positives to measures that might not be needed, this means at best 99% see fewer measures— sounds like an improvement to me!

2) Why does the CT/CQ lead to policy based false positives?

Essentially it boils down to whether the test tells you if you’re currently infected, or not.

People like Kevin are angry because the test can only tell you the concentration of virus RNA. I agree if you were only designing a testing architecture to tell individuals they actively have something that is not dramatically infectious, you’d tune the CT low enough to make it statistically very likely that you are (lower CT more virus).

However, we’re not just doing that are we. We’re trying to build a test&trace architecture to find where C19 has gone. We need a search tool.

Just think about how it spreads, and the layers in infection this will create. Worse, you know that only 15–20% of cases are choosing to present for testing driven by symptoms. The end result is you know that in telling B they have a CT of 32, you’re probably telling them they are beyond infectivity, but you really want to include them in t&t, as without doing so you won’t trace their contacts and find the other half of the tree.

Crucially, we know from out look at random errors, that they are low, so this very likely not to be a random goose chase. In Germany they call this “backward searching the tree”, and everyone raves about how they have a better approach. So why are we choosing to lampoon our own attempts at it?

This is all the government is trying to do by setting the CQ/CT higher. Think of it this way, it makes the PCR more like a serology test.

Plus, at the other end, of the infectivity window, it will be catching cases early too, before they have chance to become more infectious, and in cases were it is possible 50% of people can be shedding, but with no symptoms, this has huge policy advantages too.

Worse though, in not being sympathetic to the government’s dilemma, people fail to be honest about the question, “what else do we do”?

To explore this let’s consider the question:

“Did you get wet in the rain?”… “Maybe a little bit… not sure.”

The same applies to any infection. Sure there are RNA fragments either:

• wrapped in a protein layer that can infect cells
• or, those without the layer “dead virus” just swimming about unable to penetrate your cells as you’re at the tail end of infection

But, designing a test to distinguish and rolling it out as rapidly, is years away.

Even if you could, there will always be subjective questions about what being infected truly is. Maybe you had a huge viral load, but you knocked it on the head easily, and the symptoms you had were an entirely separate parallel infection. Or, perhaps you were simply genetically incapable of having the virus penetrate your cells, so that you snogged someone with it and got a huge viral load, is irrelevant. This can happen as the corona on the virus wrapper, the part that attacks and penetrates you cells, is believed to be unable to attack some small proportion of people due to genetic variation.

All this subjectivity aside, as per the quantile plot above from the ONS study, the baby that is being thrown out with the bath water here is that this is a very good test. If you were designing one, you’d want it to be analog, as this pretty much is, not digitial— e.g. the result is the number of cycles, not a yes/no. This gives you a policy platform. A strategy to be sure with weak high CT results. It gives you all the tools to more effectively build t&t. Lesser tests would have meant flying more blind.

Sure, they’ve made political choices with the CT/CQ, and one that sees those likely nolonger infectious being asked to isolate, but in my view, even as a libertarian, I can see that the needs of the many, the 99%, in being able to control the spread of the virus and reduce measures on everyone, might in this sad case outweight the needs of the few, the 1% isolating.

More so, as tests becomes something we can do more frequently, say weekly or daily, the power of a test like this is you can change the CT. If the test becomes as convenient as a breatherliser for a driving, designed to tell you your CT at home first thing in the morning, then policy can just say, below a CT of 30 wear a mask, below, 10 work from home. Far more options.

For people in a position to understand this distinction, to conflate the term false positive, and to be happy with mass confusion, and way less faith in test&trace, is reprehensible. Especially when this is the only tool we have to maximise liberty, and in reality, if false negatives are too high, it is less effective, and polices like national lockdowns become more likely, leading to less liberty.