Difficulty: Intermediate to Advanced | Prerequisites: confidence intervals for proportions, the binomial distribution, basic probability rules including Bayes' Theorem.
Up to this point in the course, you have been doing frequentist statistics: constructing confidence intervals and interpreting them as long-run coverage rates. Bayesian inference takes a fundamentally different approach. Instead of asking "how often would my procedure capture the true parameter?" it asks "given the data I observed, what do I now believe about the parameter?" This problem set question walks you through both approaches on the same data, so you can see exactly where they agree and where they diverge. You need to be comfortable with the binomial distribution and with Bayes' Theorem as a formula.
A frequentist confidence interval says "if I repeated this experiment many times, 95% of my intervals would contain the true value." A Bayesian posterior probability says "given what I observed and what I believed beforehand, here is the probability each hypothesis is true." The same data can lead to different conclusions depending on which framework you use and what prior information you bring.
Prior probability
Your belief about a parameter before seeing the data. In this problem: P(coin is fair) = 0.99, P(coin is weighted) = 0.01.
Think of it as: what you walked into the room believing, before flipping the coin.
Likelihood
The probability of observing the data given a specific parameter value: P(data | hypothesis). Computed using the binomial distribution in this problem.
In simple terms, this answers: "If the coin really were fair (or really were weighted), how likely is it I would see exactly 60 heads out of 100 flips?"
Posterior probability
Your updated belief after seeing the data, computed via Bayes' Theorem: P(hypothesis | data). It combines prior and likelihood.
Think of it as: what you now believe, having seen the evidence.
Bayes' Theorem
P(H | D) = P(D | H) * P(H) / P(D), where P(D) = sum over all hypotheses of P(D | H_i) * P(H_i).
In simple terms, this is the formula that turns your prior belief and the evidence into an updated belief.
Frequentist confidence interval
An interval constructed from the data such that, in repeated sampling, a stated fraction of such intervals would contain the true parameter. It says nothing about the probability that this particular interval contains the truth.
In simple terms, it is a procedure with a known long-run success rate, not a probability statement about the parameter.
Normal approximation to the binomial
For large n, the binomial distribution B(n, p) is approximately N(np, np(1 - p)). This lets you use z-scores and the normal table for binomial problems.
Think of it as: the bell curve is a good stand-in for the binomial when n is large enough.
Setup
You flip a coin 100 times and observe 60 heads.
Sample proportion: p-hat = 60/100 = 0.6.
Constructing a 95% CI for p
SE = sqrt(p-hat * (1 - p-hat) / n) = sqrt(0.6 * 0.4 / 100) = sqrt(0.0024) ≈ 0.049.
95% CI: 0.6 +/- 1.96 * 0.049 = (0.504, 0.696).
Does the interval contain p = 0.5?
The lower bound is 0.504, so p = 0.5 falls just outside the interval.
At the 95% level, the frequentist procedure rejects the hypothesis that the coin is fair.
This means: if the coin really were fair, observing 60 or more heads (or 40 or fewer) would happen less than 5% of the time.
Setting up the prior
Two hypotheses only: H_fair (p = 0.5) with prior 0.99, and H_weighted (p = 0.6) with prior 0.01.
Computing likelihoods
Data: 60 heads in 100 flips.
P(60 heads | p = 0.5) = C(100,60) * 0.5^60 * 0.5^40 = C(100,60) * 0.5^100.
P(60 heads | p = 0.6) = C(100,60) * 0.6^60 * 0.4^40.
The C(100,60) cancels in the ratio. What matters is the likelihood ratio: (0.6^60 * 0.4^40) / (0.5^100).
Computing the likelihood ratio
Take logs: 60ln(0.6) + 40ln(0.4) - 100*ln(0.5).
This equals 60*(-0.5108) + 40*(-0.9163) - 100*(-0.6931) = -30.649 - 36.652 + 69.315 = 2.014.
Likelihood ratio = exp(2.014) ≈ 7.49.
The data are about 7.5 times more likely under p = 0.6 than under p = 0.5.
Applying Bayes' Theorem
Posterior odds = prior odds * likelihood ratio.
Prior odds of fair: 0.99 / 0.01 = 99.
Posterior odds of fair: 99 / 7.49 ≈ 13.2.
P(fair | data) = 13.2 / (13.2 + 1) = 0.930, or about 93%.
P(weighted | data) = 1 - 0.930 = 0.070, or about 7%.
Interpretation
Despite seeing 60 heads, the Bayesian analysis still puts 93% probability on the coin being fair.
The strong prior (99% belief in fairness) overwhelms the moderately surprising data.
The data shifted belief from 99% to 93%, a meaningful update but nowhere near rejecting fairness.
The frequentist result
The 95% CI (0.504, 0.696) excludes p = 0.5.
A frequentist would say: "We reject the null hypothesis that the coin is fair at the 5% significance level."
The Bayesian result
P(coin is fair | data) ≈ 0.93.
A Bayesian would say: "There is a 93% probability the coin is fair."
The disagreement
The frequentist rejects fairness; the Bayesian strongly favours it. Same data, opposite conclusions.
The difference comes from the prior: the Bayesian incorporates the strong prior belief (99%) that the mint produces fair coins. The frequentist procedure ignores prior information entirely.
What is the actual probability the 95% CI contains p?
If the Bayesian posterior is correct, there is a 93% chance p = 0.5 (which the CI excludes) and a 7% chance p = 0.6 (which the CI includes).
So the actual probability the CI contains the true p is: 0.93 * 0 + 0.07 * 1 = 0.07, or about 7%.
This is dramatically less than 95%. The CI's nominal coverage rate does not account for what you already knew about the coin.
The lesson: a 95% confidence interval has 95% coverage in the long run across all possible parameter values. For a specific problem with strong prior information, the actual coverage can be very different.
P(H_i \mid D) = \frac{P(D \mid H_i) \cdot P(H_i)}{\sum_j P(D \mid H_j) \cdot P(H_j)}P(k \text{ heads} \mid p) = \binom{n}{k} p^k (1-p)^{n-k}\text{Likelihood ratio} = \frac{P(D \mid H_{\text{weighted}})}{P(D \mid H_{\text{fair}})} = \frac{0.6^{60} \cdot 0.4^{40}}{0.5^{100}} \approx 7.49\text{Posterior odds} = \text{Prior odds} \times \text{Likelihood ratio}Students often think a 95% confidence interval means "there is a 95% probability the true value is inside." That is the Bayesian interpretation, not the frequentist one. A frequentist CI is a procedure with a 95% long-run hit rate.
A common error is computing the Bayesian posterior without normalising. You must divide by P(D), the total probability of the data across all hypotheses.
Students sometimes think Bayesian and frequentist methods always agree. This problem is designed to show they can reach opposite conclusions when the prior is strong.
When computing likelihood ratios with large exponents, students often try to compute each likelihood separately and overflow their calculator. Work with logs or ratios instead.
⚠️ This question is a classic exam setup: construct a CI, then do a Bayesian analysis, then compare. Be ready to explain in words why the two methods disagree.
⚠️ You may be asked to compute the "actual" probability a CI contains p using the posterior. That requires weighting each hypothesis by its posterior probability and checking whether p falls inside the CI.
⚠️ Know the difference between "95% confidence" (a property of the procedure) and "95% probability" (a Bayesian statement about the parameter).
⚠️ Be comfortable computing likelihoods from the binomial formula and combining them with priors via Bayes' Theorem.
True or false: A 95% confidence interval gives a 95% probability that the parameter is inside. ___
Fill in the blank: In Bayes' Theorem, the posterior is proportional to the ___ times the ___.
True or false: A strong prior can override moderately surprising data in a Bayesian analysis. ___
Fill in the blank: If the likelihood ratio is 7.5 and the prior odds are 99:1, the posterior odds are approximately ___.
True or false: The frequentist and Bayesian approaches must always reach the same conclusion. ___
Answers: 1. False (it is a long-run coverage rate). 2. Likelihood, prior. 3. True. 4. 99/7.5 ≈ 13.2 in favour of the prior hypothesis. 5. False.
Q: You flip a coin 100 times and get 60 heads. Construct a 95% CI for p using the normal approximation.
A: p-hat = 0.6. SE = sqrt(0.6 * 0.4 / 100) = 0.049. CI = 0.6 +/- 1.96 * 0.049 = (0.504, 0.696). The value p = 0.5 is not inside this interval.
Q: With priors P(p = 0.5) = 0.99 and P(p = 0.6) = 0.01, compute the posterior probability that the coin is fair after observing 60 heads in 100 flips.
A: The likelihood ratio L(p = 0.6) / L(p = 0.5) = (0.6/0.5)^60 * (0.4/0.5)^40 ≈ 7.49. Prior odds of fair = 99. Posterior odds of fair = 99/7.49 ≈ 13.2. P(fair | data) = 13.2/14.2 ≈ 0.93.
Q: Given the Bayesian posterior, what is the actual probability the 95% CI from part (a) contains the true p?
A: The CI is (0.504, 0.696). If p = 0.5 (93% probable), the CI does not contain p. If p = 0.6 (7% probable), the CI does contain p. Actual probability = 0.93 * 0 + 0.07 * 1 = 0.07, or 7%.
Q: Explain in plain language why the frequentist and Bayesian conclusions differ here.
A: The frequentist looks only at the data and asks whether 60/100 is "surprising enough" to reject fairness. It is, barely. The Bayesian also considers the strong prior belief that the mint produces fair coins, which overwhelms the moderate evidence from 100 flips. The prior acts as a counterweight to the data.
Q: If the prior had been 50/50 instead of 99/1, what would the Bayesian posterior be?
A: Prior odds = 1. Posterior odds of fair = 1/7.49 ≈ 0.134. P(fair | data) = 0.134/1.134 ≈ 0.118, or about 12%. With a weaker prior, the data dominate, and the Bayesian would also lean toward the coin being weighted.
This material is a bridge to Bayesian statistics as a whole. The discrete-prior setup here is the simplest case; more advanced courses use continuous priors (beta distributions for proportions, normal priors for means).
The comparison with frequentist CIs connects back to the interpretation of confidence intervals covered in the first set of notes. The key distinction, coverage rate versus posterior probability, is one of the most important conceptual divides in statistics.
The likelihood ratio appears again in hypothesis testing (the Neyman-Pearson lemma) and in information theory (log-likelihood ratios).
Bayes' Theorem, Bayesian inference, Bayesian updating, prior probability, posterior probability, likelihood, likelihood ratio, frequentist inference, confidence interval, confidence level, coverage rate, binomial distribution, normal approximation, prior odds, posterior odds, coin flip, hypothesis testing, p-value, significance level, discrete prior, Analytical Statistics, ASTATS, NYU, PS4, Problem Set 4, Timothy Roeper