Source: Principles of Statistics I, Texas A&M / Tamhane & Dunlop Ch. 6–7, 9

Tags: interval estimators, confidence interval, prediction interval, tolerance interval, pivot method, t-distribution, chi-square, F-distribution, pooled variance, Welch-Satterthwaite, paired data, exponential CI

TL;DR

There are three types of interval estimators: confidence intervals (for fixed parameters), prediction intervals (for a future observation), and tolerance intervals (for a proportion of the population). Confidence intervals are most commonly built using the pivot method, where you find a function of the data and the parameter whose sampling distribution is free of unknowns, then invert the probability inequality.

Key Terms

Confidence interval (CI)

A 100(1 − α)% CI for a parameter θ is an interval (θ̂_L, θ̂_U) constructed from a random sample such that P[θ̂_L ≤ θ ≤ θ̂_U] = 1 − α. It reflects the uncertainty in using a point estimator for θ.

Prediction interval (PI)

An interval (Ŷ_n+1,L, Ŷ_n+1,U) that captures the next random observation Y_n+1 with probability 1 − α. It forecasts a single future value, not a parameter.

Tolerance interval (TI)

An interval (L_P,γ, U_P,γ) such that we are 100γ% confident it contains at least 100P% of the population values. Two tuning parameters: P (proportion captured) and γ (confidence that the interval achieves that capture).

Pivot

A function g(Y, θ) of the data and the unknown parameter whose sampling distribution does not depend on θ. The pivot is the engine behind most exact CI constructions.

Level of confidence

The 100(1 − α)% value refers to the long-run process: across many repeated samples, approximately (1 − α) of the constructed intervals will contain θ. Any single realised interval either contains θ or it does not.

Coverage probability

C(θ, n) = P[θ ∈ (θ̂_L, θ̂_U)]. Compares the true capture rate to the nominal level 1 − α.

Expected width

E[W(θ, n)] = E[θ̂_U − θ̂_L]. A measure of precision. Between two CIs with the same coverage, the one with shorter expected width is preferred.

Core Content

The Three Interval Types at a Glance

• CI targets a fixed parameter (μ, σ, p, β, etc.). Example: "We are 99% certain the mean tumour-appearance time is in (40, 60) hours."

• PI targets the next random observation. Example: "We are 98% certain the tensile strength of the next specimen is in (1.23, 2.31)."

• TI targets a region of the population. Example: "We are 95% certain that 99% of bearing diameters fall in (0.99, 1.02) cm."

The Pivot Method (3 Steps)

1. Find a function g(Y, θ) whose sampling distribution does not depend on θ.

1. Use the cdf of g to obtain percentiles C_α/2 and C_1−α/2 satisfying P[C_α/2 ≤ g(Y, θ) ≤ C_1−α/2] = 1 − α.

1. Algebraically invert the inequality to isolate θ, giving L ≤ θ ≤ U.

The cdf of the pivot can be determined by exact derivation, by asymptotic (large-n) approximation, or by bootstrap.

Exact CI for μ in a Normal Population

Setting: Y₁, …, Y_n iid N(μ, σ²), σ unknown.

Pivot: g(Y, μ) = (Ŷ − μ) / (S / √n), which follows a t-distribution with df = n − 1.

100(1 − α)% CI for μ:

Ŷ ± t_{α/2, n−1} · S / √n

The t-percentile t_{α/2, n−1} = qt(1 − α/2, n − 1) in R.

Exact CI for σ in a Normal Population

Setting: Y₁, …, Y_n iid N(μ, σ²).

Pivot: g(Y, σ) = (n − 1)S² / σ², which follows a chi-square distribution with df = n − 1.

100(1 − α)% CI for σ:

( √[(n−1)S² / χ²_{1−α/2, n−1}] , √[(n−1)S² / χ²_{α/2, n−1}] )

Note the inversion: the larger chi-square percentile goes in the denominator of the lower bound.

CI for μ₂ − μ₁: Two Independent Normal Samples

Case 1: Equal variances (σ₁ = σ₂ = σ, unknown)

Pooled variance: S²_p = [(n₁ − 1)S²₁ + (n₂ − 1)S²₂] / (n₁ + n₂ − 2)

Pivot follows t with df = n₁ + n₂ − 2.

CI: (Ŷ − X̄) ± t_{α/2, n₁+n₂−2} · S_p√(1/n₁ + 1/n₂)

Case 2: Unequal variances (σ₁ ≠ σ₂)

Do not pool. Use separate sample variances.

Pivot is approximately t with Welch-Satterthwaite df:

df_WS = (w₁ + w₂)² / [w₁²/(n₁ − 1) + w₂²/(n₂ − 1)], where w_i = S²_i/n_i

CI: (Ŷ − X̄) ± t_{α/2, dfWS} · √(S²₁/n₁ + S²₂/n₂)

This is an approximate CI because the pivot distribution is only approximately t.

CI for μ₂ − μ₁: Paired Data

Setting: (Y₁, X₁), …, (Y_n, X_n) are iid pairs. Define D_i = Y_i − X_i.

If the pairs are positively correlated, pairing reduces Var(Ŷ − X̄) compared to the independent-samples case:

Var(Ŷ − X̄) = Var(Ŷ) + Var(X̄) − 2·Cov(Ŷ, X̄)

The problem reduces to a one-sample CI for μ_D:

CI: D̄ ± t_{α/2, n−1} · S_D / √n

Note: df = n − 1 (number of pairs minus one), not 2(n − 1).

Trade-off of pairing: variance is reduced, but df drops from 2(n − 1) to n − 1, making the t-percentile larger. The net benefit depends on the strength of the positive correlation.

Exact CI for β in an Exponential Distribution

Setting: Y₁, …, Y_n iid Exp(β). The MLE is β̂ = Ŷ.

Pivot: g(Y, β) = 2nŶ / β, which follows a chi-square distribution with df = 2n.

This works because nŶ = ΣY_i ~ Gamma(n, β), and 2W/β ~ χ²(2n) when W ~ Gamma(n, β).

100(1 − α)% CI for β:

( 2nŶ / χ²_{1−α/2, 2n} , 2nŶ / χ²_{α/2, 2n} )

R commands: χ²_{α/2, 2n} = qchisq(α/2, 2n) and χ²_{1−α/2, 2n} = qchisq(1 − α/2, 2n).

Example: For a 95% CI with n = 10, use df = 20. χ²_{0.025, 20} = 9.591 and χ²_{0.975, 20} = 34.170.

Exact CI for β₁/β₂: Comparing Two Exponential Distributions

Setting: Y_i iid Exp(β₂), X_i iid Exp(β₁), independent.

Pivot: (Ŷ/β₂) / (X̄/β₁), which follows an F-distribution with df = (2n₂, 2n₁).

100(1 − α)% CI for β₁/β₂:

( (X̄/Ŷ) · F_{α/2, 2n₂, 2n₁} , (X̄/Ŷ) · F_{1−α/2, 2n₂, 2n₁} )

Useful F-table identity: F_{1−α, n₁, n₂} = 1 / F_{α, n₂, n₁}

Formulas / Key Distributions

Why It Matters / Exam Flags

⚠️ The confidence level refers to the process, not to any single interval. A realised CI either contains θ (probability 1) or does not (probability 0).

⚠️ For paired data, df = n − 1, not 2(n − 1). Using the wrong df is a common exam error.

⚠️ When inverting a chi-square pivot, the larger chi-square value goes in the denominator of the lower CI bound. The direction flips.

⚠️ The Welch-Satterthwaite CI is approximate. The equal-variance (pooled) CI is exact under its assumptions.

⚠️ For the exponential CI, the chi-square df is 2n, not n − 1.

Practice Q&A

Q: What are the three conditions required for the t-based CI for a normal mean to be exact?

A: (1) X̄ has a normal distribution, (2) (n − 1)S²/σ² has a chi-square distribution, and (3) X̄ and S are independent. All three hold when sampling from N(μ, σ²).

Q: A sample of n = 10 from an Exp(β) distribution yields Ŷ = 5.3. Construct a 95% CI for β.

A: The pivot 2nŶ/β ~ χ²(20). CI = (2·10·5.3 / χ²_0.975,20, 2·10·5.3 / χ²_0.025,20) = (106 / 34.170, 106 / 9.591) = (3.10, 11.05).

Q: Why does pairing sometimes produce a narrower CI than two independent samples, and when might it not?

A: Pairing reduces variance when pairs are positively correlated, because Var(D̄) = Var(Ŷ) + Var(X̄) − 2Cov(Ŷ, X̄). However, the df drops to n − 1, raising the t-percentile. If the correlation is weak, the increased t-percentile can outweigh the variance reduction, producing a wider interval.

Q: For the F-distribution pivot comparing two exponential means, why is df = (2n₂, 2n₁) rather than (n₂ − 1, n₁ − 1)?

A: Because each 2n_iŶ_i/β_i is chi-square with df = 2n_i (from the Gamma-to-chi-square transformation), not from a sample-variance calculation which would give n_i − 1.

Related Terms / Search Tags

confidence interval, pivot method, t-distribution CI, chi-square CI, pooled two-sample t-test, Welch-Satterthwaite degrees of freedom, paired t-interval, exponential confidence interval, F-distribution ratio, interval estimation, exact confidence interval, sampling distribution of pivot, level of confidence interpretation