Difficulty: Intermediate | Prerequisites: Conditional probability basics, the multiplication rule, event independence (see companion study notes).
Bayes' theorem is the tool you reach for when you observe evidence and need to update a belief. It reverses the direction of conditional probability: given P(evidence | hypothesis), it tells you P(hypothesis | evidence). In ECON 20, this sits right after the foundational rules of probability and independence, and it will reappear whenever the course moves into statistical inference, estimation, and hypothesis testing. You should already be comfortable with conditional probability, the multiplication rule, and the law of total probability before working through this material.
Bayes' theorem lets you flip a conditional probability around: if you know the probability of seeing certain evidence under each hypothesis, and you know the prior probability of each hypothesis, you can compute the probability of each hypothesis given the evidence. The denominator uses the law of total probability to account for every way the evidence could have arisen.
Prior probability
The probability you assign to a hypothesis before observing any new evidence. Written P(H).
Think of it as: your starting belief, before the data comes in.
Likelihood
The probability of observing the evidence assuming a particular hypothesis is true. Written P(E|H).
In simple terms, this means: if this hypothesis were correct, how probable is what I just saw?
Posterior probability
The updated probability of a hypothesis after taking the evidence into account. Written P(H|E).
Think of it as: your revised belief, after the data.
Law of total probability
If H₁, H₂, ..., Hₙ are a partition of the sample space, then P(E) = Σ P(E|Hᵢ) · P(Hᵢ). It gives you the overall probability of the evidence by summing over every way it could have happened.
In simple terms, this means: add up the contributions from each possible scenario.
Bayes' theorem
P(H|E) = P(E|H) · P(H) / P(E). Combines the prior and the likelihood, normalised by the total probability of the evidence, to produce the posterior.
Think of it as: the formula that turns "probability of evidence given hypothesis" into "probability of hypothesis given evidence."
Partition
A set of mutually exclusive, collectively exhaustive events. Every outcome in the sample space belongs to exactly one event in the partition.
In simple terms, this means: a clean, no-overlap division of all possibilities.
Start from the definition of conditional probability: P(H|E) = P(H ∩ E) / P(E).
By the multiplication rule, P(H ∩ E) = P(E|H) · P(H).
Substitute: P(H|E) = P(E|H) · P(H) / P(E).
The denominator P(E) is computed via the law of total probability when there are multiple hypotheses.
This is the kind of problem Bayes' theorem was built for: you observe data and need to figure out which scenario produced it.
Setup:
200 dice in a bag.
1 trick die: sixes on all faces (always rolls 6).
199 normal dice: each face shows a different number (probability of rolling 6 on any single roll = 1/6).
You pick one die at random and roll it three times. It lands on 6 all three times.
Define events:
T = you picked the trick die.
N = you picked a normal die.
E = you rolled three sixes in a row.
Step 1: Priors
P(T) = 1/200 = 0.005
P(N) = 199/200 = 0.995
Step 2: Likelihoods
P(E|T) = 1 (the trick die always rolls 6, so three sixes is certain).
P(E|N) = (1/6)³ = 1/216 ≈ 0.00463 (each roll is independent for a fair die).
Step 3: Total probability of the evidence
P(E) = P(E|T) · P(T) + P(E|N) · P(N)
P(E) = 1 · (1/200) + (1/216) · (199/200)
P(E) = 1/200 + 199/43200
P(E) = 216/43200 + 199/43200 = 415/43200
Step 4: Posteriors via Bayes' theorem
P(T|E) = P(E|T) · P(T) / P(E) = (1 · 1/200) / (415/43200) = (43200/200) / 415 = 216/415 ≈ 0.5205
P(N|E) = 1 − P(T|E) = 199/415 ≈ 0.4795
Interpretation:
Before rolling, there was only a 0.5% chance you had the trick die. After three sixes, that jumps to about 52%. The evidence has dramatically shifted your belief, but it has not made the trick die a certainty, because a normal die can also roll three sixes (just rarely).
P(H|E) = \frac{P(E|H) \cdot P(H)}{P(E)} \quad \text{(Bayes' theorem)}P(E) = \sum_{i=1}^{n} P(E|H_i) \cdot P(H_i) \quad \text{(law of total probability)}P(A \cap B) = P(A|B) \cdot P(B) = P(B|A) \cdot P(A) \quad \text{(multiplication rule)}For the dice problem specifically:
P(T|E) = \frac{P(E|T) \cdot P(T)}{P(E|T) \cdot P(T) + P(E|N) \cdot P(N)} = \frac{1 \cdot \frac{1}{200}}{1 \cdot \frac{1}{200} + \frac{1}{216} \cdot \frac{199}{200}} = \frac{216}{415}Bayes' theorem is the engine behind medical diagnostic reasoning: if a test is positive, Bayes' theorem combines the test's sensitivity (likelihood) with the disease's prevalence (prior) to give the probability of actually having the disease (posterior). It is also the foundation of spam filters, which update the probability that a message is spam based on the words it contains.
Students often confuse P(H|E) with P(E|H). These are not the same. P(rain|clouds) ≠ P(clouds|rain). Bayes' theorem exists precisely to convert one into the other.
A frequent mistake is forgetting the denominator. P(E) is not optional; it normalises the answer so the posteriors sum to 1. Skipping it gives you an unnormalised score, not a probability.
Students sometimes assume that a high likelihood alone makes a hypothesis probable. It does not. A rare hypothesis (low prior) with a high likelihood can still have a low posterior if the evidence is also likely under common alternatives.
In the dice problem, some students multiply P(T) by P(E|T) and stop there. That gives the joint probability P(T ∩ E), not the posterior P(T|E). You must divide by P(E).
⚠️ Bayes' theorem problems are among the most commonly tested in introductory statistics. Expect at least one on every exam.
⚠️ The dice bag problem is a classic template. Variations include drawing balls from urns, selecting defective items from production lines, and medical test accuracy. The structure is always: two (or more) hypotheses, one piece of evidence, find the posterior.
⚠️ You will need the law of total probability for the denominator in nearly every Bayes problem. Practise computing P(E) as a weighted sum.
⚠️ Watch the arithmetic. The fractions in Bayes problems are fiddly. Find a common denominator early and simplify at the end.
True or false: P(H|E) = P(E|H) whenever E and H are defined.
Fill in the blank: The denominator of Bayes' theorem is computed using the ___.
True or false: If the prior probability of a hypothesis is very low, a single piece of supporting evidence is guaranteed to make the posterior high.
Fill in the blank: In the dice bag problem, P(E|trick die) = ___.
True or false: The posteriors P(T|E) and P(N|E) must sum to 1.
Answers: 1. False. 2. Law of total probability. 3. False. 4. 1. 5. True.
Q: You have 200 dice in a bag. One is a trick die (sixes on all faces). The other 199 are fair. You pick one at random and roll three sixes in a row. What is the probability you picked the trick die?
A: Let T = trick die, N = normal die, E = three sixes. P(T) = 1/200, P(N) = 199/200. P(E|T) = 1, P(E|N) = (1/6)³ = 1/216. P(E) = (1)(1/200) + (1/216)(199/200) = 216/43200 + 199/43200 = 415/43200. P(T|E) = (1/200) / (415/43200) = 216/415 ≈ 0.520. The probability it is a normal die is P(N|E) = 199/415 ≈ 0.480.
Q: A disease affects 1% of the population. A test has 95% sensitivity (positive when disease present) and 90% specificity (negative when disease absent). If someone tests positive, what is the probability they have the disease?
A: Let D = disease, + = positive test. P(D) = 0.01, P(+|D) = 0.95, P(+|Dᶜ) = 0.10. P(+) = (0.95)(0.01) + (0.10)(0.99) = 0.0095 + 0.099 = 0.1085. P(D|+) = 0.0095 / 0.1085 ≈ 0.0876, or about 8.8%. Even with a fairly accurate test, the low prevalence means most positives are false positives.
Q: State Bayes' theorem and explain each component in words.
A: P(H|E) = P(E|H) · P(H) / P(E). P(H) is the prior (initial belief about H). P(E|H) is the likelihood (how probable the evidence is if H is true). P(E) is the marginal probability of the evidence (computed by the law of total probability). P(H|E) is the posterior (updated belief about H after seeing the evidence).
Q: Why does the denominator of Bayes' theorem matter? What happens if you leave it out?
A: The denominator P(E) normalises the result so the posteriors across all hypotheses sum to 1. Without it, you get P(E|H) · P(H), which is the joint probability P(H ∩ E), not the conditional P(H|E). The number would be too small and would not be interpretable as a probability of the hypothesis.
This connects directly to event independence: when H and E are independent, P(E|H) = P(E), and the posterior equals the prior. Independence means the evidence is uninformative.
Bayes' theorem also lays the groundwork for Bayesian inference, which you may encounter in more advanced econometrics or data science courses. The idea of updating beliefs with data scales from these two-hypothesis problems to models with continuous parameter spaces.
The law of total probability, essential to the denominator, connects back to the partition and complement material from earlier in the chapter.
Bayes' theorem, Bayes rule, Bayesian updating, conditional probability, posterior probability, prior probability, likelihood, law of total probability, total probability theorem, inverse probability, base rate fallacy, false positive, sensitivity, specificity, dice bag problem, urn problem, ECON 20, analytical statistics, NYU, Chikhany, problem set 1