Source: Walpole Ch. 8, Lecture Weeks 9–10
Tags: sampling distribution, central limit theorem, CLT, sample mean, standard error, law of large numbers, STAT 3021, UMN
Difficulty: Intermediate | Prerequisites: Random Variables (Ch. 3-4), Normal Distribution (Ch. 6)
Sampling distributions are the bridge between probability theory and statistical inference. Up to this point in the course, you have been studying individual random variables. Now you study what happens when you take a sample of size n and compute a statistic (like the sample mean). The central limit theorem (CLT) is the reason that so many statistical procedures work: it guarantees that the distribution of the sample mean is approximately normal for large n, regardless of the population's shape. This is arguably the most important theorem in the course.
The sampling distribution of the sample mean X-bar describes how X-bar varies from sample to sample. Its mean equals the population mean μ, and its standard deviation (called the standard error) is σ/√n. The central limit theorem says X-bar is approximately normal when n is large, even if the population is not normal.
Statistic
A numerical value computed from a sample. The sample mean, sample proportion, and sample variance are all statistics. Think of it as a number that summarises some aspect of your data.
Sampling distribution
The probability distribution of a statistic over all possible samples of size n from the population. In simple terms, if you could take every possible sample and compute the statistic each time, the resulting distribution of those values is the sampling distribution.
Standard error (SE)
The standard deviation of a sampling distribution. For the sample mean: SE = σ / √n. It measures how much the sample mean varies from sample to sample. Larger samples produce smaller standard errors.
Central limit theorem (CLT)
If X₁, X₂, ..., Xₙ are independent random variables from a population with mean μ and variance σ², then for large n, the sample mean X-bar is approximately N(μ, σ²/n). In simple terms, averages of large samples are approximately normal, no matter what the original distribution looks like.
Law of large numbers
As n increases, the sample mean X-bar converges to the population mean μ. In simple terms, the more data you collect, the closer your sample average gets to the true average.
If the population has mean μ and variance σ², then:
E(X-bar) = μ (the sample mean is an unbiased estimator of the population mean)
Var(X-bar) = σ²/n
SD(X-bar) = σ/√n (the standard error)
As n increases, the sampling distribution becomes more tightly concentrated around μ
If the population is normal, X-bar is exactly normal for any n
If the population is not normal, the CLT says X-bar is approximately normal for large n
"Large n" typically means n ≥ 30 as a rule of thumb, though the required n depends on how skewed the population is
For symmetric populations, even n = 10 or 15 may suffice
For heavily skewed populations, you may need n > 50
To compute probabilities about X-bar, standardise: Z = (X-bar - μ) / (σ/√n)
If the population proportion is p, the sample proportion p-hat = X/n where X ~ Bin(n, p)
E(p-hat) = p
Var(p-hat) = p(1-p)/n
For large n, p-hat is approximately N(p, p(1-p)/n)
Rule of thumb: np ≥ 5 and n(1-p) ≥ 5 for the normal approximation to be reasonable
\bar{X} \sim N\left(\mu, \frac{\sigma^2}{n}\right) \quad \text{(exactly if population is normal, approximately by CLT for large } n\text{)}Z = \frac{\bar{X} - \mu}{\sigma / \sqrt{n}}\hat{p} \approx N\left(p, \frac{p(1-p)}{n}\right) \quad \text{for large } nThe CLT is why opinion polls work: a sample of 1,000 voters can estimate the preference of millions with a predictable margin of error, because the sample proportion is approximately normal. Quality control in manufacturing relies on sampling distributions to decide whether a batch meets specifications based on a small number of measured items.
Students sometimes think the CLT says the data itself becomes normal. It does not. The CLT is about the distribution of the sample mean (or sum), not the individual observations.
Confusing σ (population SD) with σ/√n (standard error). The standard error shrinks as n increases; the population SD does not.
Assuming you always need n ≥ 30. This is a rough guideline, not a universal rule. The more symmetric the population, the smaller n can be.
Forgetting to divide by √n when standardising X-bar. If you use Z = (X-bar - μ) / σ (without the √n), you are standardising a single observation, not a sample mean.
⚠️ CLT problems are high-probability exam questions. Practise the full workflow: identify μ, σ, and n, compute the standard error, standardise, and use the z-table.
⚠️ Know the difference between a question about a single observation (use σ) and a question about a sample mean (use σ/√n).
⚠️ Be ready to state the CLT in words. A common exam question asks you to explain what the CLT says and why it matters.
True or False: The CLT requires the population to be normally distributed. (False. The CLT works for any population shape when n is large enough.)
Fill in the blank: The standard error of the sample mean is σ divided by ___. (√n)
True or False: Doubling the sample size halves the standard error. (False. Doubling n multiplies SE by 1/√2 ≈ 0.707. You need to quadruple n to halve SE.)
Q: A population has mean 100 and standard deviation 20. If a random sample of 64 is taken, what is the probability that the sample mean exceeds 105?
A: SE = 20/√64 = 20/8 = 2.5. Z = (105 - 100)/2.5 = 2.0. P(X-bar > 105) = P(Z > 2.0) = 1 - 0.9772 = 0.0228.
Q: In a large population, 40% favour a policy. In a random sample of 200, what is the probability that the sample proportion is between 0.35 and 0.45?
A: SE = √(0.4 × 0.6 / 200) = √(0.0012) ≈ 0.03464. Z₁ = (0.35 - 0.4)/0.03464 = -1.44. Z₂ = (0.45 - 0.4)/0.03464 = 1.44. P(-1.44 < Z < 1.44) = 0.9251 - 0.0749 = 0.8502.
Q: Why does increasing sample size improve the precision of the sample mean as an estimator?
A: Because the standard error σ/√n decreases as n increases, the sampling distribution of X-bar becomes more concentrated around μ. Larger samples produce less variable estimates.
The sampling distribution of X-bar is the direct foundation for confidence intervals (Chapter 9) and hypothesis tests (Chapter 10). The formula for the standard error appears in every confidence interval and test statistic in the rest of the course. The CLT also explains why the t-distribution (used when σ is unknown) approaches the normal distribution as n grows.
sampling distribution, central limit theorem, CLT, sample mean, X-bar, standard error, SE, law of large numbers, sample proportion, p-hat, normal approximation, σ/√n, sampling variability, unbiased estimator, STAT 3021, Walpole Chapter 8