Difficulty: Intermediate to Advanced | Prerequisites: Expected value, variance, confidence intervals, two-sample tests.
Real survey data rarely arrives in tidy, representative packages. When different subgroups respond at different rates, a simple average of the responses is biased: it over-represents the group that replied more often. This topic shows you how to detect that bias mathematically, prove an estimator is biased, and construct a weighted alternative that recovers an unbiased estimate. These ideas are central to any survey-based research and appear whenever you move from textbook sampling to messy real-world data.
If a survey over-samples one group relative to its share of the population, the naive sample mean is biased. You fix this by weighting each group's sample mean by its true population proportion rather than its sample proportion.
Non-response bias
A systematic error that occurs when the people who respond to a survey differ in meaningful ways from those who do not. In the practice exam, freshmen are three times more likely to respond, so respondents are disproportionately freshmen even though freshmen are only 1/4 of the population.
Unbiased estimator
A statistic whose expected value equals the parameter it estimates. Formally, an estimator T is unbiased for theta if E[T] = theta. Think of it as: on average, across all possible samples, the estimator hits the target.
Biased estimator
An estimator whose expected value does not equal the target parameter. The bias is E[T] - theta. In the survey problem, the naive sample mean has E[X-bar] = (1/2) * mu_F + (1/2) * mu_U, which does not equal the population mean mu = (1/4) * mu_F + (3/4) * mu_U unless mu_F = mu_U.
Weighted estimator
An estimator that assigns weights to subgroup means according to their true population proportions rather than their sample proportions. This corrects for differential response rates.
Stratified sampling
A sampling design that divides the population into subgroups (strata) and samples from each. Even when stratification was not planned, you can use the idea after the fact by reweighting stratum means to match population shares.
Population: NYU students, 1/4 freshmen, 3/4 upperclassmen.
Survey sent to 1,000; 162 respond. Freshmen are 3x more likely to respond, so 1/2 of respondents (81) are freshmen even though they are only 1/4 of the population.
Known variance for all students: sigma-squared = 162.
Sample means: freshmen = 90, upperclassmen = 100, full sample = 95.
Under H0, the two population means are equal. Use a two-sample z-test (variance is known).
SE of the difference = sqrt(162/81 + 162/81) = sqrt(2 + 2) = sqrt(4) = 2.
z = (90 - 100) / 2 = -5.
At alpha = 0.05 (two-sided), the critical z-values are plus or minus 1.96. Since |-5| > 1.96, reject H0.
Point estimate: 90 - 100 = -10.
Margin: 1.96 * 2 = 3.92.
Interval: (-13.92, -6.08).
The naive X-bar treats all 162 observations equally: X-bar = (sum of 81 F_i + sum of 81 U_i) / 162.
E[X-bar] = (81 * mu_F + 81 * mu_U) / 162 = (1/2) * mu_F + (1/2) * mu_U.
The population mean is mu = (1/4) * mu_F + (3/4) * mu_U.
These are equal only if mu_F = mu_U. Since the data suggest they differ, X-bar is biased for mu.
Weight each subgroup by its true population share:
X-hat = (1/4) * F-bar + (3/4) * U-bar, where F-bar = sum of F_i / 81 and U-bar = sum of U_i / 81.
E[X-hat] = (1/4) * mu_F + (3/4) * mu_U = mu. Unbiased.
Numerically: X-hat = (1/4)(90) + (3/4)(100) = 22.5 + 75 = 97.5.
Var(X-hat) = (1/4) squared * (162/81) + (3/4) squared * (162/81) = (1/16)(2) + (9/16)(2) = (10/16)(2) = 10/8 = 1.25.
SE = sqrt(1.25) ≈ 1.118.
For 95% coverage: margin = 1.96 * 1.118 ≈ 2.19.
Interval: 97.5 plus or minus 2.19, so approximately (95.31, 99.69).
Naive sample mean (biased when groups are unequally represented): X-bar = (sum of all observations) / n
Weighted (unbiased) estimator: X-hat = w_F * F-bar + w_U * U-bar, where w_F and w_U are the true population proportions (here 1/4 and 3/4)
Proving unbiasedness: E[X-hat] = w_F * E[F-bar] + w_U * E[U-bar] = w_F * mu_F + w_U * mu_U = mu
Variance of the weighted estimator (independent subsamples, known variance): Var(X-hat) = w_F-squared * (sigma-squared / n_F) + w_U-squared * (sigma-squared / n_U)
95% CI margin: Margin = z*(0.025) * sqrt(Var(X-hat)) = 1.96 * SE
Students often think "more data is always better" and assume the full-sample mean of 95 is the best estimate of mu. It is not, because the sample is not representative of the population.
Students confuse the sample proportion of freshmen (1/2) with the population proportion (1/4). The weights in the corrected estimator must use the population proportions.
Students sometimes propose dropping one subgroup's data to fix bias. This throws away information. The weighted estimator uses all 162 observations and is still unbiased.
Students forget to square the weights when computing the variance of the weighted estimator. Var(aX) = a-squared * Var(X), not a * Var(X).
⚠️ Q3 is the highest-value question on the practice midterm (28 points) and tests several skills in sequence: hypothesis testing, CI construction, proving bias, proposing a fix, and computing the CI for the fix.
⚠️ "Explain, using the mathematical definition of unbiasedness" means you must write out E[X-bar], show it does not equal mu, and identify the source of the discrepancy. A verbal answer alone will not earn full marks.
⚠️ The question explicitly asks you to "establish that it is in fact unbiased," so show E[X-hat] = mu step by step.
⚠️ Known variance (sigma-squared = 162) means you use z-values, not t-values, for the CI and hypothesis test.
True or false: An estimator can be biased even when based on a large sample.
Fill in the blank: If freshmen are 1/4 of the population but 1/2 of the sample, the correct weight for the freshman mean in an unbiased estimator is ______.
True or false: When sigma is known, you use the t-distribution for hypothesis tests.
Fill in the blank: The bias of the naive sample mean in the subway problem equals E[X-bar] - mu = ______ * (mu_F - mu_U).
Answers: 1. True (bias is about systematic error, not sample size). 2. 1/4. 3. False (use z). 4. 1/4 (since E[X-bar] = (1/2)mu_F + (1/2)mu_U and mu = (1/4)mu_F + (3/4)mu_U, the difference is (1/4)(mu_F - mu_U)).
Q: You survey 162 NYU students about subway spending. 81 are freshmen (sample mean 90) and 81 are upperclassmen (sample mean 100). The population variance is 162. Test H0: mu_F = mu_U at alpha = 0.05.
A: SE = sqrt(162/81 + 162/81) = 2. z = (90 - 100)/2 = -5. Critical values: plus or minus 1.96. |-5| > 1.96, so reject H0. There is strong evidence that freshmen and upperclassmen spend different amounts.
Q: Construct a 95% CI for mu_F - mu_U.
A: -10 plus or minus 1.96 * 2 = -10 plus or minus 3.92, giving (-13.92, -6.08).
Q: The naive sample mean X-bar = 95. The population is 1/4 freshmen, 3/4 upperclassmen. Show that X-bar is biased for the true population mean mu, and propose an unbiased estimator.
A: E[X-bar] = (81 * mu_F + 81 * mu_U)/162 = (1/2)mu_F + (1/2)mu_U. But mu = (1/4)mu_F + (3/4)mu_U. Since (1/2) does not equal (1/4) and (1/2) does not equal (3/4), E[X-bar] does not equal mu unless mu_F = mu_U. Unbiased alternative: X-hat = (1/4) * F-bar + (3/4) * U-bar. E[X-hat] = (1/4)mu_F + (3/4)mu_U = mu.
Q: Using X-hat, how wide must the 95% CI be?
A: Var(X-hat) = (1/16)(162/81) + (9/16)(162/81) = (10/16)(2) = 1.25. Margin = 1.96 * sqrt(1.25) ≈ 2.19. The interval extends 2.19 above and below X-hat = 97.5.
The concept of weighting to correct for non-representative samples extends to post-stratification and inverse probability weighting in more advanced courses. The proof-of-bias technique (computing E[estimator] and comparing to the target) is the same method you use to show that dividing by n - 1 makes the sample variance unbiased. The z-test and CI mechanics here are identical to those in the single-sample case, just applied to a difference.
non-response bias, survey bias, unbiased estimator, biased estimator, weighted average, stratified sampling, post-stratification, population proportion, sample proportion, expected value, two-sample z-test, confidence interval, known variance, inverse probability weighting, representative sample