Problem 3: Medicine

The problem in brief

A disease affects 1 % of the population. A new test detects it in 99 % of sick people and correctly reports "negative" for 98 % of healthy people. Someone chosen at random tests positive: how likely is it that they are really sick? The answer is a probability written with four decimals. This is the textbook illustration of Bayes' theorem, and of the base-rate fallacy: most people, including many doctors in classic surveys, answer "about 99 %". The truth is very different.

How you are scored

A single exact answer with four decimals, worth 10 points. Our answer, obtained exactly with fractions and confirmed by a simulation of two million people:

$$\boxed{0.3333}$$

First look: name the numbers

Write $S$ for "sick" and $+$ for "the test is positive". The statement gives three numbers, each with a standard name you will meet again in classifier evaluation:

quantity	value	name
$P(S)$	$0.01$	prevalence, the prior
$P(+ \mid S)$	$0.99$	sensitivity, or recall / true positive rate
$P(- \mid \bar S)$	$0.98$	specificity, so $P(+ \mid \bar S) = 0.02$ is the false positive rate

The question asks for $P(S \mid +)$, the precision of the test (positive predictive value). The trap is to confuse $P(S \mid +)$ with $P(+ \mid S) = 0.99$: they are conditional probabilities in opposite directions.

The key idea

Bayes' theorem flips the conditioning, and the law of total probability gives the denominator by splitting the positives into those who are sick and those who are not:

$$P(S \mid +) = \frac{P(+ \mid S)\,P(S)}{P(+)},$$ $$P(+) = P(+ \mid S)\,P(S) + P(+ \mid \bar S)\,P(\bar S).$$

Plugging in the numbers:

$$P(+) = 0.99 \times 0.01 + 0.02 \times 0.99 = 0.0099 + 0.0198 = 0.0297,$$ $$P(S \mid +) = \frac{0.0099}{0.0297} = \frac{1}{3} \approx 0.3333 .$$

Notice the neat structure: $0.0198$ is exactly twice $0.0099$, because the false positive rate ($2\%$) times the healthy share ($99\%$) is twice the prevalence ($1\%$) times the sensitivity ($99\%$). For every true positive there are two false positives, hence one chance in three.

The same thing with natural frequencies

Imagine 10 000 people.

• 100 are sick; the test flags $0.99 \times 100 = 99$ of them.
• 9 900 are healthy; the test wrongly flags $0.02 \times 9\,900 = 198$ of them.

Among the $99 + 198 = 297$ positive results, only 99 belong to sick people: $99/297 = 1/3$.

Why is the answer so low although the test is "99 % accurate"? Because the disease is rare. The healthy group is 99 times larger than the sick group, so even a small false positive rate applied to that huge group produces more false alarms than there are sick people to detect. The right panel shows how strongly the answer depends on the prior: the same test would give a posterior above 0.9 if 20 % of the people tested were sick, and a nearly useless 0.05 at prevalence 0.1 %.

What would actually help

• Better specificity matters more than better sensitivity here: with 99.9 % specificity the posterior jumps to $0.0099/(0.0099 + 0.00099) \approx 0.909$.
• A second, independent positive test uses $1/3$ as the new prior: $\frac{0.99 \cdot 1/3}{0.99 \cdot 1/3 + 0.02 \cdot 2/3} = \frac{99}{103} \approx 0.9612$. This is why screening positives are always confirmed by a second test.

The full solution

solution.py computes the answer three independent ways:

from fractions import Fraction as F
prev, sens, spec = F(1, 100), F(99, 100), F(98, 100)
p_pos = sens * prev + (1 - spec) * (1 - prev)   # 297/10000
post = sens * prev / p_pos                      # exactly 1/3

• exact fractions with Bayes' formula: $1/3$;
• natural frequencies on 10 000 people: $99/297 = 1/3$;
• a Monte Carlo simulation of 2 000 000 random people: $0.3318$, within sampling noise of $0.3333$ (only about 59 000 of them test positive, so the standard error is about $0.002$).

Answer to submit, rounded to four decimals: 0.3333.

Results

approach	value	verdict
Bayes' theorem with exact fractions	$1/3 = 0.3333$	correct
natural frequencies (10 000 people)	$99/297 = 0.3333$	correct
Monte Carlo, 2 000 000 people	0.3318	agrees within noise
confusing $P(S \mid +)$ with the sensitivity $P(+ \mid S)$	0.9900	wrong: base-rate fallacy
forgetting the false positives in the denominator	1.0000	wrong
using $P(+ \mid \bar S) = 0.98$ instead of $0.02$	0.0101	wrong: specificity is about negatives

What to remember

• $P(A \mid B) \ne P(B \mid A)$. Bayes' theorem is the tool to turn one into the other, and it needs the prior $P(A)$.
• Rare events make precision low even with excellent recall and specificity: false positives come from the much larger negative group.
• Natural frequencies ("out of 10 000 people...") make Bayes problems almost impossible to get wrong; use them to check your formula.
• In ML terms: sensitivity is recall, $P(S \mid +)$ is precision, and class imbalance (the prior) is what separates them. The same reasoning explains why accuracy is misleading on imbalanced data.
• Posteriors chain: today's posterior is tomorrow's prior, which is exactly how Naive Bayes combines independent pieces of evidence.