False Positives go mainstream
Statistics are far more interesting when we care about the topic
24 April 2020
Statisticians are always on the look out for ways to engage the general public in their subject. Over the years I've seen many presenters talking about 'False positives' and 'False negatives' and how these can lead to the misdiagnosis of a medical condition. Unfortunately the public (especially younger audiences) often seem rather indifferent to the counter-intuitive maths behind false positives. There's an element of "This doesn't affect me, so why should I care?"
Covid-19 has changed all that. The accuracy of testing is now important to all of us.
Over the coming months, we'll all be obsessed with antibody tests. These tests (usually blood tests) can be used to establish whether you have had Covid-19 in the past and have now recovered. A positive test is good news, because it means that (in theory) you are now immune to Covid, and you are very unlikely to pass it on to anyone else.
Unfortunately, these tests have so far proved to be unreliable. I've been told that at the moment, even the best antibody tests are only 90% reliable, which means 10% of the time they will give you a positive result when you haven't actually had the illness. There will also be false negatives, in which you have had the illness, but your antibodies aren't picked up by the test.
The health authorities and government are anxious to know how many people have had Covid already. Many might have had the illness without symptoms. The more people that turn out to have had it, the better, because that means we have begun to develop 'herd immunity'.
But until a lot of people have recovered from Covid, the stats could be extremely unreliable.
Let's suppose 5% of people have had Covid, and that antibody tests give false positives 10% of the time. And to simplify things, let's suppose there are no false negatives.
Now we test 100 people. On average, we expect 5 of them to have had Covid, all of whom get a positive result to the antibody test. However, of the remaining 95 people, 9 or 10 will also get a positive result. So the stats will suggest that 15% of people have had Covid, about three times the true value.
But what if 50% of the population have had the infection? We will get 50 true positives and 5 false positives, suggesting that 55% of the population have been infected - much closer to the right figure.
This is useful when Public Health are trying to get an estimate, but it doesn't help an individual trying to understand their own risk, and this is the other reason why we want to reduce the number of false positives. A person who has wrongly been told they've had the illness might behave inappropriately, putting themselves and other people at risk.
So how can we reduce the number of false positives?
Double Testing
One interesting suggestion is to do DOUBLE tests. You test somebody for antibodies (10% chance you'll get a false positive) then you test them again. A person is only 'cleared' if they get positive results on both tests. The chance that both tests will give you a false positive is 10% x 10%, which is 1%. Now that is a LOT better - a ten times reduction in false positives with only twice as many tests.
Sadly there is a snag. The maths above only works if the result of one test is independent of the result of another test. It turns out this is not always the case. False positives might be down to the way the test was carried out, or a faulty lab process, or the fact that a particular individual's blood make-up makes them more prone to giving a false positive result. The result of the second test might therefore have the same error as the first.
The maths and statistics of this pandemic are full of traps for the unwary, but one thing is for sure - mathematical modelling and statistics have rarely been as much in the limelight as they are now.