Source: Principles of Statistics I, Texas A&M / Tamhane & Dunlop Ch. 7, 9, 14.6
Tags: central limit theorem, improper CLT use, sample size determination, distribution-free CI, quantile CI, median CI, order statistics, skewed distributions, coverage probability failure
Using the t-based CI when the population is non-normal can badly miss the stated coverage, especially with skewed or heavy-tailed distributions at small n, because Ŷ and S become correlated. Sample size formulas let you choose n before collecting data to guarantee a desired margin of error. Distribution-free CIs for population quantiles use order statistics and the binomial distribution, requiring no assumptions about the population shape.
Asymptotic CI
A CI whose validity relies on large-sample (CLT) approximations rather than exact distributional results. Works well when n is large enough for the pivot's distribution to be approximately normal or t.
Improper use of CLT
Applying the t-based CI (Ŷ ± tα/2 S/√n) to data from a distribution where the required conditions fail, particularly when Ŷ and S are correlated. Common with skewed distributions at moderate n.
Sample size determination
Choosing n so that the estimator θ̂ is within Δ units of the true parameter with confidence 1 − α. Solved by inverting the margin-of-error equation.
Distribution-free CI for Q(u)
A CI for the uth population quantile that uses only order statistics and the binomial distribution, valid for any continuous distribution.
Order statistics
The sorted sample values Y(1) < Y(2) < … < Y(n).
The t-based CI for μ (Ŷ ± tα/2, n−1 S/√n) requires three conditions from normal sampling:
Ŷ has a normal distribution
(n − 1)S²/σ² has a chi-square distribution
Ŷ and S are independent
For non-normal populations, the CLT makes Ŷ approximately normal for large n, and S is consistent for σ. But the independence of Ŷ and S can fail spectacularly with skewed data.
Exponential example (n = 20): When sampling from Exp(β), Ŷ and S are strongly positively correlated (Corr ≈ 0.75) because both estimate the same parameter β (since μ = σ = β in the exponential). Instead of 5 out of 100 intervals failing to contain μ, about 9 out of 100 fail. The stated 95% level is really only about 90%.
Simulation results across distributions (n = 20, nominal 95% CI):
Normal: coverage 0.948, Corr(Ŷ, S) ≈ 0.001 (works perfectly)
Exponential: coverage 0.918, Corr(Ŷ, S) ≈ 0.750 (poor)
Lognormal: coverage 0.868, Corr(Ŷ, S) ≈ 0.853 (very poor)
Pareto: coverage 0.868, Corr(Ŷ, S) ≈ 0.817 (very poor)
Symmetric distributions (double-exponential, parabola): coverage near 0.95 even with moderate Corr, because symmetry mitigates the problem
Key insight: The culprit is a high positive correlation between Ŷ and S, driven by right-skewness or heavy tails. Symmetric distributions, even non-normal ones, tend to maintain adequate coverage.
Increasing n to 30 helps only modestly for highly skewed distributions. Lognormal and Pareto coverages remain around 0.884 even at n = 30.
The question: "How many observations do I need?" Answer it by setting up the margin-of-error equation and solving for n.
We want P[|Ŷ − μ| ≤ Δ] ≈ 1 − α. Using the normal approximation:
n = Z²α/2 · σ² / Δ²
σ is typically unknown. Options for estimating it: previous studies, published literature, a pilot study, or the crude approximation σ̂ ≈ Range/4.
We want P[|p̂ − p| ≤ Δ] ≈ 1 − α.
n = Z²α/2 · p(1 − p) / Δ²
p is unknown. Strategies:
If you can bound p (e.g., 0 ≤ p ≤ 0.10), replace p with the boundary value that maximises p(1 − p), which is the boundary closest to 0.5.
If you cannot bound p, use p = 0.5 (worst case). This gives the largest possible n.
The function p(1 − p) is maximised at p = 0.5 where it equals 0.25.
Example: Find n for 99% confidence that p̂ is within 0.01 of p.
If 0 ≤ p ≤ 0.10: n = (2.576)²(0.1)(0.9) / (0.01)² = 5,973
If 0.8 ≤ p ≤ 1.0: n = (2.576)²(0.8)(0.2) / (0.01)² = 10,618
If no bound on p: n = (2.576)²(0.5)(0.5) / (0.01)² = 16,590
A more accurate formula uses the Wilson CI structure and yields slightly different values (e.g., 5,978 vs 5,973 in the first case).
Setting: Y1, …, Yn iid with continuous cdf F. Want a CI for Q(u) = F−1(u).
CI: (Y(r), Y(s)) where 1 ≤ r < s ≤ n
The coverage probability is:
P[Y(r) ≤ Q(u) ≤ Y(s)] = P[r ≤ B ≤ s − 1]
where B ~ Bin(n, u). In R: pbinom(s − 1, n, u) − pbinom(r − 1, n, u).
The values r and s are chosen to make this probability as close to 1 − α as possible from above (never below).
Why it works: By the probability integral transform, F(Yi) ~ Uniform(0, 1). The event "Q(u) is between Y(r) and Y(s)" is equivalent to "u is between U(r) and U(s)" where U(i) are uniform order statistics, and counting how many uniforms fall below u is a binomial problem.
Set u = 0.5 and require the CI to be symmetric: s = n − r + 1.
CI for Q(0.5): (Y(r), Y(n−r+1))
where r is the largest integer such that P[r ≤ B ≤ n − r] ≥ 1 − α with B ~ Bin(n, 0.5).
Example: For n = 50 and 95% confidence, R code yields r = 18 with coverage probability 0.96716. The CI is (Y(18), Y(33)).
Standard tables (e.g., CRC Handbook) provide r values for various n and confidence levels.
For a general quantile Q(u):
n = 50; L = .95; P = .75
s = ceiling(n*P) - 1; r = floor(n*P) + 1; cov = 0
while(s < n-1 && r > 1 && cov < L) {
s = s + 1
cov = pbinom(s-1, n, P) - pbinom(r-1, n, P)
if(cov >= L) break
r = r - 1
cov = pbinom(s-1, n, P) - pbinom(r-1, n, P)
}
For Q(0.75) with n = 50: r = 32, s = 44, coverage = 0.9519. CI is (Y(32), Y(44)).
Sample size for μ: n = Z²α/2 σ² / Δ²
Sample size for p: n = Z²α/2 p(1 − p) / Δ²
Distribution-free quantile CI coverage: P[r ≤ B ≤ s − 1], B ~ Bin(n, u)
R command for quantile CI coverage: pbinom(s-1, n, u) - pbinom(r-1, n, u)
⚠️ The t-based CI for μ has much lower than nominal coverage when sampling from skewed distributions (exponential, lognormal, Pareto), even at n = 30. Do not blindly invoke the CLT.
⚠️ The correlation between Ŷ and S is the diagnostic to watch. High positive correlation signals that the t-based CI will undercover.
⚠️ In sample size formulas, always round n up to the next integer.
⚠️ The sample-size formula for p requires a guess at p. Using p = 0.5 is safe but often gives a much larger n than necessary if you have a bound on p.
⚠️ Distribution-free quantile CIs have coverage ≥ 1 − α, not exactly 1 − α, because of the discreteness of the binomial.
⚠️ The distribution-free CI for the median uses df and Bin(n, 0.5), not Bin(n, p) for a general p.
Q: Why does the t-based CI badly undercover when sampling from an exponential distribution, even though the CLT makes Ŷ approximately normal?
A: Because Ŷ and S are strongly positively correlated (Corr ≈ 0.75) when sampling from the exponential. The t-pivot requires Ŷ and S to be independent, and violating this assumption degrades coverage more than any non-normality of Ŷ.
Q: A researcher wants to estimate a population proportion to within 0.03 with 95% confidence. She believes p is between 0.05 and 0.15. What sample size does she need?
A: Use p = 0.15 (the value in the range closest to 0.5). n = (1.96)²(0.15)(0.85) / (0.03)² = (3.8416)(0.1275) / 0.0009 ≈ 544.3, so n = 545.
Q: For a distribution-free 95% CI on the upper quartile Q(0.75) with n = 50, what order statistics form the CI?
A: R code gives r = 32, s = 44. The CI is (Y(32), Y(44)) with true coverage 0.952.
Q: What is the crude estimator of σ used when no prior information is available for sample-size calculations?
A: σ̂ ≈ Range / 4, where the range is the difference between the largest and smallest plausible values.
central limit theorem failure, Ŷ and S correlation, skewed distribution CI, sample size formula mean, sample size formula proportion, margin of error, distribution-free confidence interval, nonparametric quantile CI, order statistics CI, median confidence interval, binomial coverage, probability integral transform