Tests to diagnose whether you have the COVID-19 coronavirus are not always correct.
Just like any medical tests, it may say you have the disease when you don’t – called a false positive – or it may say that you don’t have the disease when you actually do – called a false negative.
The inaccuracy inherent in such testing is the likely reason for last week’s reports of a false positive result in the Free State.
These reports referred to a “local transmission” – someone who had caught the coronavirus without travelling outside South Africa. However, when test were performed to confirm the diagnosis, they came back negative.
To ensure they don’t get a false negative, the National Institute for Communicable Diseases (NICD) performs two tests on patients who test negative after initially testing positive.
Only if both tests are negative do they consider the patient negatively tested for the COVID-19 coronavirus.
The chance of having a disease if you test positive
False positives can happen for many reasons, including human error in collecting or handling the sample taken from the patient.
It could also be a result of the test and not human error. When you are testing thousands of patients, even tests that are 99% accurate are bound to give incorrect results.
Another interesting quirk of statistics is that even if you initially test positive for a relatively rare disease, even if the test is extremely accurate, the chance that you actually have the disease is much lower than you might think.
Bayes’ Theorem provides a formula for calculating this probability. To better understand Bayes’ Theorem, it is useful to visualise it as follows.
Imagine that you test positive for a disease where the test’s accuracy is 99%, but the disease only occurs in a small fraction of the population – five in every 1,000.
To calculate your chances of having the disease you must not look at the test’s accuracy, but at the total number of people who may have tested positive for the disease. These include false positives and true positives.
In this example, a 99% accurate test means that 10 in every 1,000 people will test positive even if they don’t have the disease.
When you add the five true positives, that means your real chances of having this rare disease after you test positive are five out of 15, or 33%.
It should be noted that this is not a complete explanation of the concept, and is only intended to provide an idea of how Bayes’ Theorem works.
Calculating test probability
The real power of Bayes’ Theorem comes from its ability to update prior beliefs.
In the case of a medical test, it lets you see how the certainty that someone has a disease (or does not have a disease) increases as more tests are conducted.
Bayes’ Theorem, in this context, states that the probability of having a disease if you test positive is equal to the accuracy of the test weighted against our prior belief that the patient has caught the disease.
This is then divided by the overall probability of the test returning a positive result — including false positives.
In mathematical terms it is expressed as follows:
- P(Disease|+): Probability that you have the disease given that the test is positive.
- P(+|Disease): Accuracy of the test.
- P(Disease): Our initial belief that you have the virus.
- P(+): Overall probability that a test comes back positive.
Unfortunately, there is currently no data available on the accuracy of the tests that detect the COVID-19 coronavirus.
When asked for details on the accuracy of its tests, the NICD said that it differs for each kit and test.
“Some of these details are not available as these tests are only recently being used to diagnose COVID-19 disease in the clinical setting globally,” the NICD explained.
For the sake of demonstration, let’s assume that the tests are 95% accurate. In other words, they have a 95% chance of being positive when you have the disease and a 95% chance of being negative if you don’t have the disease.
The last piece of information that is needed is “P(Disease)”.
This is generally the most difficult number to determine and often comes down to nothing more than a guess. However, Bayes’ Theorem allows us to update this probability with new information, so it is safe to underestimate this probability.
If I test positive, what are the chances I have COVID-19?
The Department of Health has published useful numbers to help calculate a reasonable initial belief.
As of 17 March 2020, the SA coronavirus portal stated that of the 2,405 people tested, there had been 62 positive cases identified.
We can therefore say our initial belief that you have the disease if you are sick enough to be tested is around 2.6%. Therefore:
- If you test positive, assuming a 95% accurate test, the chance that you actually have COVID-19 is around 33%.
Bayes’ Theorem now allows us to use that 33% result as the new value for P(Disease) if you run a second test.
- If the second test comes back positive, the probability that you have COVID-19 shoots up to nearly 91%.
This is the reason the NICD confirms all test results. And as the tests can take 24 hours to provide a result, this is why it takes up to 48 hours to confirm with reasonable confidence whether someone has the COVID-19 coronavirus.
A note on “accuracy”
The term “accuracy” is a simplification of how the effectiveness of medical tests is measured.
In reality, the accuracy of tests will be given in terms like “sensitivity” and “specificity”.
Sensitivity is the probability of a positive test if the patient has the disease. In our above calculations, this would be P(+|Disease).
Specificity measures the probability of a test returning negative when the patient does not have the disease. This probability would be used to help calculate P(+).
For the sake of simplicity, sensitivity and specificity were assumed to be equal in the example calculation above.
Running simulations with different values for these two accuracy measurements yields interesting observations, though:
- The sensitivity of a test can be low and still yield good results, as long as the specificity is high.
- If the specificity of a test is even as low as 85%, the results become poor. Even after two tests, you can’t say with reasonable confidence whether someone has a disease if they have tested positive for it.
In other words, the ability of a test to avoid false negatives is far more important than its ability to avoid false positives.
How the NICD tests for coronavirus
The NICD told MyBroadband that it tests for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) – the causative pathogen for COVID-19 – through a polymerase chain reaction (PCR) molecular test, on a respiratory tract sample.
“The tract sample is from the nose, throat, or chest. The test detects the presence of the genetic sequence of the virus in samples,” the NICD said.
Avoiding false positives
The flow chart below is from an NICD document titled “Guidelines for case-finding, diagnosis, management and public health response in South Africa”.
It shows that the NICD requires two positive test results before confirming whether someone has COVID-19.
It also shows that if the initial test was positive, the NICD requires two negative results before confirming the person as being negative for the virus.
This process helps guard against confirming false positives as legitimate cases of the coronavirus, and it prevents false negatives from being allowed to leave isolation and infect others.
When asked how it avoids false negatives and false positives, the NICD said it runs a number of laboratory controls.
These include regulating specimen quality with each test, checking the correct functioning of the laboratory reagents, and monitoring for contamination.
“Our SARS-CoV-2 real-time PCR assays are the same as those being used in other international public health laboratories and our laboratories are SANAS accredited,” the NICD said.