A disease affects 1 in 1,000 people. A test for it is 99% accurate. You take the test, and it comes back positive. The chance you actually have the disease is not 99%. It is about 9%.
That is not a trick, and the test is not broken — it really does perform at 99%. The gap between 99% and 9% is one of the most consequential illusions in probability, and famous studies have shown that most doctors fall for it [1][2]. In this paper we resolve the paradox without a single formula, using nothing but counting.
Two numbers hide inside "99% accurate":
- Sensitivity: of people who have the disease, 99% test positive.
- Specificity: of people who don't have it, 99% test negative — meaning 1% of healthy people get a false positive.
Both are statements that begin with "given your true condition...". But you sit in the opposite position: you know your test result and want your condition. Flipping a conditional statement around is exactly what our intuition does badly — the question "how often does a positive person turn out sick?" is a genuinely different question from "how often does a sick person test positive?", and it has a different answer.
Count people, not probabilities
The fix, championed by psychologist Gerd Gigerenzer, is to stop juggling percentages and instead imagine a concrete town of people — so-called natural frequencies [1]. Take 100,000 people and let the numbers fall where the rates say they must.
- Sick: 1 in 1,000 means sick people. The test catches 99% of them: 99 true positives (1 sick person slips through).
- Healthy: the other 99,900 people. The test wrongly flags 1% of them:
Now line up everyone who received a positive result: people. Only 99 of them are actually sick:
A positive result moved your risk from 0.1% to 9% — ninety times higher, and well worth following up — but it is still overwhelmingly likely that you are fine.
The culprit is the mismatch in group sizes. The healthy group (99,900) is a thousand times larger than the sick group (100), so even a tiny 1% error rate applied to the enormous group produces more false alarms (999) than the near-perfect detection of the tiny group produces true ones (99). When a condition is rare, the false positives of the many swamp the true positives of the few.
Prevalence is doing all the work. Run the same count with a disease affecting 1 in 100 people and the same test: 990 true positives against 990 false positives — a positive result now means a 50% chance of illness. The test never changed; the base rate did.
This is precisely the question Casscells and colleagues put to staff and students at Harvard Medical School in 1978 (with a 1-in-1,000 disease and a 5% false positive rate). Almost half answered "95%"; the correct answer was about 2%, and only 18% of them got it [2]. Gigerenzer and Hoffrage later showed the failure largely evaporates when the same problem is phrased in counts of people, exactly as we did above [1].
What this does to mass screening
Now scale the paradox up: screen an entire population for a rare condition and the arithmetic follows you. Our imaginary town of 100,000 generated 1,098 positive results, of which 91% were false — over a thousand people sent for anxiety, follow-up appointments and sometimes invasive confirmatory procedures, to find 99 genuine cases. This is a real dilemma in public health, and it is why mass screening programmes restrict themselves to higher-risk groups (raising the base rate before testing) and never treat a single screening result as a diagnosis.
The good news is that the same maths rescues us: a positive result changes your base rate. Among our 1,098 positives, 9% are sick — so retest them. Counting again: 99 sick people yield about 98 second positives, while 999 healthy people yield about 10. A second independent positive means roughly
Two cheap, imperfect tests in sequence do what one cannot.
Always ask how rare it is
A 99% accurate test for a 1-in-1,000 disease leaves a positive patient with only a 9% chance of being sick — because 999 false alarms from the healthy majority bury 99 genuine detections. The paradox dissolves the moment we trade percentages for a headcount, which is why natural frequencies are now taught in medical statistics. The deeper lesson is one our intuition keeps failing: probabilities live or die by the base rate, just as coincidences multiply with hidden pairs in the birthday paradox and leading digits follow hidden structure in Benford's law. Before you trust any test — medical, security or spam filter — ask how rare the thing it hunts for is.
References:
[1] Gerd Gigerenzer and Ulrich Hoffrage, "How to Improve Bayesian Reasoning Without Instruction: Frequency Formats," Psychological Review 102, no. 4 (1995): 684–704.
[2] Ward Casscells, Arno Schoenberger, and Thomas B. Graboys, "Interpretation by Physicians of Clinical Laboratory Results," New England Journal of Medicine 299 (1978): 999–1001.
Note: Real screening tests rarely have equal sensitivity and specificity, and retests are seldom fully independent of the first result; the 9% figure illustrates the structure of the problem rather than any particular disease.