Table of Contents
What Is Bayes' Theorem?
Bayes' theorem describes how to update the probability of a hypothesis given new evidence. Named after Thomas Bayes (1701-1761), it provides a mathematical framework for reasoning under uncertainty and is the foundation of Bayesian statistics, spam filters, and machine learning.
The theorem relates P(A|B) to P(B|A), the prior P(A), and marginal P(B). It is essential in medical diagnosis, machine learning, and legal reasoning.
Formula
P(A|B) = P(B|A) × P(A) / P(B)
P(B) = P(B|A)P(A) + P(B|~A)P(~A)
Medical Test Example
| Parameter | Value | Meaning |
|---|---|---|
| P(Disease) | 1% | Prevalence |
| P(+|Disease) | 95% | Sensitivity |
| P(+|Healthy) | 5% | False positive |
| P(Disease|+) | 16.1% | Posterior |
Even with an accurate test, a positive for a rare disease has low posterior probability. This is the base rate fallacy.
FAQ
What is the Bayes Factor?
The ratio P(B|A)/P(B|~A) measuring how much evidence supports A over ~A. Greater than 1 supports A.
Why is the base rate important?
Low base rates mean even accurate tests produce many false positives relative to true positives.