Source: Principles of Statistics I, Texas A&M / Tamhane & Dunlop Ch. 5, 6.1
Tags: sample mean, sample median, sampling distribution, central limit theorem, CLT, finite population correction, correlated data, AR(1), equi-correlated, standard error, asymptotic relative efficiency, ARE
The sample mean X-bar is unbiased for μ with variance σ²/n under independence, and the CLT guarantees it is approximately normal for large n regardless of the population shape. The sample median is asymptotically unbiased but has a different variance that depends on the density at the population median. The Asymptotic Relative Efficiency (ARE) compares these two estimators and shows the median can outperform the mean for heavy-tailed distributions.
Sample mean (X-bar)
The arithmetic average of the sample: X-bar = (1/n) Σ Xᵢ. Unbiased for the population mean μ.
Standard error of the mean
The standard deviation of the sampling distribution of X-bar. For iid data: SE(X-bar) = σ/√n. Estimated by S/√n.
Finite population correction factor (fpcf)
The multiplier (N−n)/(N−1) that reduces the variance of X-bar when sampling without replacement from a finite population of size N. Always less than 1, so sampling without replacement gives a smaller variance than sampling with replacement.
Central Limit Theorem (CLT)
For large n, the standardised sample mean (X-bar − μ)/(σ/√n) converges in distribution to N(0,1), provided σ < ∞. This holds regardless of the shape of the population distribution.
Sample median
The middle value of the ordered sample. For odd n, it is the ((n+1)/2)th order statistic. For even n, the average of the two middle order statistics.
Asymptotic Relative Efficiency (ARE)
The ratio of the asymptotic variance of X-bar to the asymptotic variance of the sample median. Values less than 1 indicate the mean is more efficient; values greater than 1 indicate the median is more efficient.
Equi-correlated data
Data where every pair of observations has the same correlation ρ. Under equi-correlation, X-bar is still unbiased but its variance changes to (σ²/n)[1 + (n−1)ρ].
AR(1) process
A time series model where each observation depends linearly on its predecessor: Xₜ = θ + ρXₜ₋₁ + eₜ. Correlation between observations decays as the time gap increases.
With replacement (or iid sampling)
E[X-bar] = μ (unbiased for all n)
Var[X-bar] = σ²/n
Without replacement from a finite population of size N
E[X-bar] = μ (still unbiased)
Var[X-bar] = ((N−n)/(N−1)) × (σ²/n)
The factor (N−n)/(N−1) is the finite population correction factor
It is always less than 1, so variance is smaller than with-replacement sampling
Exact result for normal populations
If the population is N(μ, σ²), then X-bar is exactly N(μ, σ²/n) for any sample size n. No large-sample requirement.
Central Limit Theorem (general populations)
For large n, regardless of the population shape:
(X-bar − μ) / (σ/√n) → N(0,1)
So X-bar is approximately N(μ, σ²/n).
The quality of this approximation depends on two things:
The size of n (larger is better)
The shape of the population (more symmetric with normal-like tails gives better approximation for a given n)
Even when data are correlated, X-bar remains unbiased for μ. The variance, however, changes.
Equi-correlated data (all pairs have correlation ρ)
Var(X-bar) = (σ²/n)[1 + (n−1)ρ]
If ρ < 0: variance is smaller than σ²/n
If ρ > 0: variance is larger than σ²/n
This follows from the general variance formula for a linear combination:
Var(Σ cᵢXᵢ) = Σ cᵢ² Var(Xᵢ) + Σᵢ≠ⱼ cᵢcⱼ Cov(Xᵢ, Xⱼ)
AR(1) correlated data
For a stationary AR(1) process with autocorrelation ρ:
Var(X-bar) ≈ (σ²/n) × (1+ρ)/(1−ρ)
Positive ρ inflates the variance, sometimes dramatically.
The naive estimate SE(X-bar) = S/√n underestimates the true standard error when ρ > 0.
A corrected estimator: SE(X-bar) = (S/√n) × √((1+ρ-hat)/(1−ρ-hat))
Ozone data example (AR(1) impact)
Yonkers: S/√n = 2.31 vs corrected SE = 3.68, a 59.3% underestimate
Stamford: S/√n = 4.47 vs corrected SE = 6.33, a 41.6% underestimate
Ignoring positive autocorrelation leads to confidence intervals that are too narrow, with actual coverage well below the stated level (e.g. 80% actual coverage for a stated 95% interval).
A population of 500 pennies, where X = age of penny (2019 minus date on penny), was used to illustrate the sampling distribution of X-bar.
Population distribution: very right-skewed
Population mean: μ = 12.038, population SD: σ = 9.281
Five-number summary: Min = 1, Q1 = 5, Median = 10, Q3 = 18, Max = 40
Histograms of 10,000 sample means at different sample sizes showed the distribution of X-bar becoming more symmetric (approaching normal) and more concentrated around μ = 12 as n increased from 5 to 10 to 20. This is the CLT in action.
Definition
For odd n: Q-hat(0.5) = X₍₍ₙ₊₁₎/₂₎
For even n: Q-hat(0.5) = 0.5 × [X₍ₙ/₂₎ + X₍₍ₙ₊₂₎/₂₎]
Exact distribution
The moments of the sample median depend on the population CDF F and the joint distribution of order statistics. For n odd:
mₖ = E[Q-hat(0.5)ᵏ] = n × C(n−1, (n−1)/2) × ∫ xᵏ [F(x)(1−F(x))]^((n−1)/2) f(x) dx
The mean is m₁ and the variance is m₂ − m₁².
Asymptotic distribution (large n)
Q-hat(0.5) is approximately N(Q(0.5), (0.5)² / [n × (f(Q(0.5)))²])
So the asymptotic standard deviation is:
σ_A = 0.5 / [f(Q(0.5)) × √n]
This depends on the population density evaluated at the median, f(Q(0.5)). The sample median is asymptotically unbiased, but for small n it is generally biased.
Definition
ARE(Q-hat(0.5), X-bar) = (asymptotic variance of X-bar) / (asymptotic variance of Q-hat(0.5))
= 4σ² × [f(Q(0.5))]²
When ARE < 1, the sample mean is more efficient. When ARE > 1, the sample median is more efficient.
Results for specific distributions
Normal N(θ, σ²): ARE = 2/π ≈ 0.64. The mean is substantially more efficient.
Uniform(θ−a, θ+a): ARE = 1/3. The mean is much more efficient.
Logistic(θ₁, θ₂): ARE = π²/12 ≈ 0.82. The mean is more efficient, but the gap is smaller.
Shifted t-distribution (df = ν):
df | ARE |
|---|---|
3 | 1.62 |
4 | 1.12 |
5 | 0.96 |
8 | 0.80 |
∞ | 0.64 |
For df ≤ 4, the median is more efficient. For df ≥ 5, the mean is more efficient.
Key takeaway: For heavy-tailed symmetric distributions, the sample median is less variable (more efficient) than the sample mean for estimating the location parameter. For light-tailed or normal distributions, the mean wins.
Variance of X-bar (iid): Var(X-bar) = σ²/n
Variance of X-bar (without replacement): Var(X-bar) = ((N−n)/(N−1)) × σ²/n
Variance of X-bar (equi-correlated): Var(X-bar) = (σ²/n)[1 + (n−1)ρ]
Variance of X-bar (AR(1)): Var(X-bar) ≈ (σ²/n)(1+ρ)/(1−ρ)
Asymptotic StDev of median: σ_A = 0.5 / [f(Q(0.5))√n]
ARE formula: ARE = 4σ²[f(Q(0.5))]²
⚠️ The CLT applies to the sample mean from any population with finite variance, not just normal populations. For normal populations, the result is exact.
⚠️ Know the finite population correction factor and when it applies (sampling without replacement).
⚠️ Positive correlation inflates the variance of X-bar. The naive S/√n underestimates the true SE. This is a common exam trap.
⚠️ The ARE depends on the population distribution. Memorise the normal case (2/π ≈ 0.64) and the key insight: for heavy tails (e.g. t with small df), the median beats the mean.
⚠️ The asymptotic variance of the sample median depends on f(Q(0.5)), the density at the population median. If the density is low there, the median has high variance.
Q: State the Central Limit Theorem for the sample mean.
A: If X₁, …, Xₙ are iid with mean μ and finite variance σ², then (X-bar − μ)/(σ/√n) converges in distribution to N(0,1) as n → ∞. Equivalently, X-bar is approximately N(μ, σ²/n) for large n.
Q: If you sample without replacement from a population of N = 1000 with n = 50, what is the finite population correction factor?
A: fpcf = (1000 − 50)/(1000 − 1) = 950/999 ≈ 0.9509.
Q: For equi-correlated data with ρ = 0.3 and n = 10, by what factor is the variance of X-bar inflated compared to the iid case?
A: Factor = 1 + (10−1)(0.3) = 1 + 2.7 = 3.7. The variance is 3.7 times larger than σ²/n.
Q: What is the ARE of the sample median to the sample mean when sampling from a normal distribution?
A: ARE = 2/π ≈ 0.64. Since ARE < 1, the sample mean is the more efficient estimator.
Q: For which family of distributions does the sample median become more efficient than the sample mean?
A: For heavy-tailed symmetric distributions, such as the t-distribution with small degrees of freedom (df ≤ 4).
Q: Why does the asymptotic standard deviation of the sample median depend on f(Q(0.5))?
A: The median is determined by the order statistics near the centre of the distribution. If the density is high at the median, observations cluster there, making the median more precisely determined. If the density is low, observations are sparse near the median, increasing its variability.
sample mean, sample median, sampling distribution, central limit theorem, CLT, standard error, finite population correction factor, fpcf, equi-correlated, AR(1), autocorrelation, asymptotic relative efficiency, ARE, order statistic, asymptotic variance, asymptotic distribution, normal approximation, heavy tails, t-distribution, location parameter, efficiency, pennies example, Tamhane Dunlop, STAT Texas A&M, Handout 10