Difficulty: Intermediate | Prerequisites: Basic probability (random variables, CDFs, PDFs), point estimation, order statistics.
Confidence intervals sit at the heart of statistical inference. Where point estimators give you a single best guess for an unknown parameter, confidence intervals give you a range that captures the true parameter value with a specified probability. This topic builds directly on your understanding of sampling distributions and estimator properties. If you are comfortable with CDFs and can work with the distribution of a sample maximum, you are ready for this material. If not, revisit those topics first.
A confidence interval is a data-dependent range [A(X), B(X)] that contains the true parameter θ₀ with probability 1 − α. You build one by finding the estimator's distribution, solving for bounds that trap the parameter, and plugging in your observed data. The procedure works for any distribution where you can derive the estimator's CDF.
Confidence interval
An interval [A(X₁, …, Xₙ), B(X₁, …, Xₙ)] constructed from the sample such that P(A ≤ θ₀ ≤ B) = 1 − α. Think of it as a net you cast around your estimate, wide enough to catch the true value most of the time.
Confidence level (1 − α)
The probability that the interval contains the true parameter. A 90% confidence level means α = 0.10, so you allow 5% in each tail. In simple terms, if you repeated the experiment many times and built a new interval each time, about 90% of those intervals would contain the true parameter.
Significance level (α)
The total probability split between the tails of the estimator's distribution that falls outside the confidence interval. For a 90% CI, α = 0.10.
Estimator (θ-hat)
A function of the sample data used as the starting point for constructing the interval. In simple terms, this is your best single guess for the parameter before you widen it into a range.
Sample maximum (Xₙ)
The largest observation in the sample, max{X₁, …, Xₙ}. Used as the estimator when the parameter is the upper bound of a uniform distribution.
CDF of the estimator
The cumulative distribution function of θ-hat. You need this to find the constants a and b that define the interval. For the sample maximum of a Uniform(0, θ) distribution, F(t) = (t/θ₀)ⁿ for 0 ≤ t ≤ θ₀.
A 1 − α confidence interval for parameter θ₀ is an interval [A(X₁, …, Xₙ), B(X₁, …, Xₙ)] depending on two data-dependent functions A(·) and B(·) such that:
P_θ₀(A(X₁, …, Xₙ) ≤ θ₀ ≤ B(X₁, …, Xₙ)) = 1 − α
The endpoints A and B are random (they depend on the sample), but the parameter θ₀ is fixed. The interval moves from sample to sample; the parameter does not.
Choose an estimator. Get θ-hat = θ-hat(X₁, …, Xₙ) and determine the distribution of θ-hat.
Find probability bounds. Find a(θ) and b(θ) such that P(a(θ) ≤ θ-hat ≤ b(θ)) = 1 − α.
Invert the inequality. Rewrite the event by solving for θ, so the inequality becomes P(A(X) ≤ θ ≤ B(X)) = 1 − α.
Evaluate at the data. Plug in the observed sample X₁, …, Xₙ to compute A(X) and B(X).
State the interval. The 1 − α confidence interval is CI = [A(X₁, …, Xₙ), B(X₁, …, Xₙ)].
The inversion step (step 3) is the crux. You start with an inequality about the estimator and rearrange it into an inequality about the parameter. Everything else is bookkeeping.
CDF of the sample maximum for Uniform(0, θ₀):
F_θ-hat(t) = 0 if t ≤ 0; (t/θ₀)ⁿ if 0 ≤ t ≤ θ₀; 1 if t > θ₀
Solving for the tail bounds (90% CI):
Set F(a) = 0.05 and F(b) = 0.95, giving:
a = ⁿ√(0.05) · θ₀ and b = ⁿ√(0.95) · θ₀
Final confidence interval for θ₀ (Uniform example):
CI = [ max{X₁, …, Xₙ} / ⁿ√(0.95) , max{X₁, …, Xₙ} / ⁿ√(0.05) ]
The key algebraic move: starting from ⁿ√(0.05) · θ₀ ≤ θ-hat ≤ ⁿ√(0.95) · θ₀, divide through by the constants to isolate θ₀ in the middle.
Setup: X₁, …, Xₙ are i.i.d. Uniform(0, θ₀). Construct a 90% confidence interval for θ₀.
Step 1 – Choose the estimator. Let θ-hat = max{X₁, …, Xₙ} = X₍ₙ₎, the sample maximum. This is a natural choice because the largest observation carries the most information about the upper bound.
Step 2 – Find the CDF of θ-hat. The CDF of the maximum of n i.i.d. Uniform(0, θ₀) variables is F(t) = (t/θ₀)ⁿ for 0 ≤ t ≤ θ₀.
Step 3 – Set up the probability statement. We need constants a and b such that P(a ≤ θ-hat ≤ b) = F(b) − F(a) = 0.95 − 0.05 = 0.90. A symmetric tail split puts 5% in each tail.
Step 4 – Solve for a and b. From F(a) = 0.05: a = ⁿ√(0.05) · θ₀. From F(b) = 0.95: b = ⁿ√(0.95) · θ₀.
Step 5 – Invert to isolate θ₀. The inequality ⁿ√(0.05) · θ₀ ≤ θ-hat ≤ ⁿ√(0.95) · θ₀ rearranges to:
θ-hat / ⁿ√(0.95) ≤ θ₀ ≤ θ-hat / ⁿ√(0.05)
Step 6 – Plug in the data. Replace θ-hat with the observed sample maximum:
CI = [ max{x₁, …, xₙ} / ⁿ√(0.95) , max{x₁, …, xₙ} / ⁿ√(0.05) ]
The lower bound is always slightly larger than the sample maximum (dividing by a number less than 1), which makes sense: the true upper bound of the uniform must be at least as large as the biggest observation you saw, and likely a bit larger.
Confidence intervals are everywhere in applied statistics. When a polling firm reports that a candidate has 52% support "with a margin of error of ±3%," they are giving you a confidence interval. In manufacturing, quality engineers use CIs to estimate the true defect rate from a batch sample. In clinical trials, the efficacy of a drug is reported as a CI for the treatment effect, and regulators use the lower bound to decide whether the drug works well enough to approve.
"There is a 90% probability that θ₀ is in the interval." This is the single most common mistake. The parameter θ₀ is fixed. The interval is random. The correct statement is that 90% of intervals constructed this way will contain θ₀. Once you compute a specific interval from your data, the parameter is either in it or it is not.
Confusing confidence level with the probability of a single interval. The 1 − α refers to the long-run coverage rate of the procedure, not to any one realised interval.
Assuming wider intervals are always worse. A wider interval at the same confidence level comes from more variability in the data or a smaller sample. The width tells you something about precision, not about whether the method is correct.
Forgetting to invert the inequality. Students sometimes stop after finding a and b in terms of θ, and report the interval for θ-hat rather than for θ. The inversion step (step 3) is where the interval flips from being about the estimator to being about the parameter.
⚠️ You will almost certainly be asked to construct a confidence interval from scratch for a given distribution. Memorise the five-step procedure.
⚠️ The interpretation question appears regularly: "What does a 95% confidence interval mean?" The answer is about the procedure's long-run coverage, not about the probability of a single interval.
⚠️ The uniform distribution example is a classic exam problem. Be comfortable with the CDF of the sample maximum and the algebra of inverting the inequality.
⚠️ Know how to split α between the tails. The symmetric split (α/2 in each tail) is standard, but the exam may ask for an asymmetric split or a one-sided interval.
True or false: A 95% confidence interval means there is a 95% chance the true parameter is inside the interval.
Fill in the blank: The CDF of the sample maximum for n i.i.d. Uniform(0, θ) variables is F(t) = ______.
True or false: In the five-step procedure, step 3 (inverting the inequality) converts a statement about the estimator into a statement about the parameter.
Fill in the blank: For a 90% CI with a symmetric tail split, each tail carries ______% of the probability.
True or false: The confidence interval endpoints are fixed numbers once the data is collected.
Answers: 1. False (it is the procedure that succeeds 95% of the time). 2. (t/θ)ⁿ. 3. True. 4. 5%. 5. True (the randomness is gone once you plug in observed data).
Q: Define a 1 − α confidence interval for a parameter θ₀. What are the two key components?
A: A 1 − α confidence interval is an interval [A(X₁, …, Xₙ), B(X₁, …, Xₙ)] depending on two data-dependent functions A(·) and B(·) such that P(A ≤ θ₀ ≤ B) = 1 − α. The two key components are the data-dependent endpoints and the coverage guarantee.
Q: Suppose X₁, …, X₁₀ are i.i.d. Uniform(0, θ). Construct a 90% confidence interval for θ using the sample maximum.
A: The CDF of the sample maximum is F(t) = (t/θ)¹⁰. Setting F(a) = 0.05 and F(b) = 0.95 gives a = ¹⁰√(0.05) · θ and b = ¹⁰√(0.95) · θ. Inverting: CI = [max{Xᵢ} / ¹⁰√(0.95), max{Xᵢ} / ¹⁰√(0.05)].
Q: A student says "There is a 90% probability that θ is between 3.2 and 4.8." What is wrong with this statement?
A: θ is a fixed constant, not a random variable. After computing the interval, the parameter is either inside it or not. The correct interpretation is that the procedure used to construct the interval captures the true θ in 90% of repeated samples.
Q: Why do we use the sample maximum as the estimator for the upper bound of a Uniform(0, θ) distribution?
A: The sample maximum is a sufficient statistic for θ in the Uniform(0, θ) model. It carries all the information the sample has about the parameter, and its distribution is tractable (a clean power function), making the inversion step straightforward.
Q: In step 3 of the construction procedure, what does "inverting the inequality" mean, and why is it necessary?
A: Inverting means algebraically rearranging the inequality a(θ) ≤ θ-hat ≤ b(θ) so that θ is isolated in the middle: A(X) ≤ θ ≤ B(X). This is necessary because we want bounds on the parameter, not bounds on the estimator. The estimator is observed; the parameter is unknown.
Confidence intervals build directly on point estimation. The estimator you choose in step 1 is the same object you studied when learning about bias, consistency, and efficiency. A good estimator leads to a tighter interval.
This material also connects forward to hypothesis testing. A 1 − α confidence interval and a two-sided level-α test are two views of the same thing: θ₀ is rejected by the test if and only if it falls outside the CI. Understanding one makes the other immediate.
The Bayesian counterpart is the credible interval, which places a probability distribution on the parameter itself. The frequentist CI discussed here does not do that, and understanding the distinction is a common exam topic in courses that cover both paradigms.
Confidence interval, CI, interval estimation, coverage probability, confidence level, significance level, alpha, 1 minus alpha, point estimator, sample maximum, order statistics, Uniform distribution, CDF inversion, pivotal quantity, interval estimate, margin of error, two-sided interval, one-sided interval, hypothesis testing duality, credible interval, Bayesian vs frequentist, NYU Analytical Statistics