Bayes’ theorem: turning probabilities around correctly

In short: Bayes’ theorem links two conditional probabilities and lets you compute, from “how likely is the test result if you are ill?”, “how likely are you ill if the test is positive?”.

In more detail: Formula: P(A | B) = P(B | A) · P(A) / P(B). P(A) is the prior, P(A | B) the updated one (posterior).

In Depth

• Classic example: A disease affects 1 in 1,000 people. A test detects the ill with 99 % and wrongly reports positive for 5 % of the healthy. Of 1,000 people 1 is ill (almost surely positive) and about 50 healthy ones test positive. Only about 2 % of those testing positive are really ill.
• Lesson: For rare events even good tests are often mostly false positives. The base rate (how common is the thing at all?) is decisive and often overlooked.
• Natural frequencies: With absolute numbers (“out of 1,000 people …”) it is easier to compute than with percentages.
• Spam filter: A naive Bayes filter estimates from words how likely a mail is spam.
• Bayesian updating: New evidence shifts the assessment step by step. If the base rate is low you need strong evidence.
• Applications: Medical tests, diagnostic systems, filters, sensor fusion, Bayesian optimisation (hyperparameter search).

See also: probability basics