Difficulty: Intermediate | Prerequisites: Sample variance, standard error, t-distribution basics, z-distribution
Much of applied statistics involves comparing two groups: a treatment vs a control, men vs women, freshmen vs upperclassmen. This requires pooling variance estimates, computing the variance of a difference, and building confidence intervals around that difference. The second half of this material tackles a subtler problem: what happens when your sample is not representative of the population? Non-response bias in surveys can make the naive sample mean a biased estimator of the population mean, and fixing that requires constructing a weighted estimator that rebalances the over- and under-represented groups. This is one of the most practically important topics in the course.
When comparing two population means, pool the sample variances (assuming equal population variances), compute the variance of the difference, and build a t-based confidence interval. When a survey over-samples one group, the simple sample mean is biased; fix it by weighting each group's observations according to their true population proportions.
Pooled variance (s_p^2)
A weighted average of two sample variances, used when you assume both populations share the same true variance. The weights are the degrees of freedom from each sample. Think of it as: combining the variance information from both samples into a single, more precise estimate.
Variance of the difference in means (Var(d))
The variance of X-bar minus Y-bar, which equals sigma^2 * (1/n_X + 1/n_Y) when both populations share the same variance. In simple terms, this means: the uncertainty in the difference depends on how much data you have from each group.
Two-sample t-interval
A confidence interval for the difference between two population means, built using the pooled variance and the t-distribution with df = n_X + n_Y - 2. Think of it as: the range of plausible values for how far apart the two population means are.
Non-response bias
A systematic error that occurs when certain groups are more likely to respond to a survey than others, making the sample unrepresentative of the population. In simple terms, this means: if freshmen reply three times as often as upperclassmen, the sample over-represents freshmen, and the simple average of all responses will not estimate the true population mean.
Unbiased estimator
An estimator whose expected value equals the true parameter it is estimating: E[estimator] = parameter. Think of it as: if you repeated the sampling and estimation many times, the average of all your estimates would land exactly on the true value.
Weighted estimator
An estimator that assigns different weights to different observations or groups to correct for sampling imbalances. In simple terms, this means: you give less weight to the over-represented group and more weight to the under-represented group so the estimator reflects the true population composition.
Given two samples (X and Y), first compute each sample mean and sample variance separately.
Example: X observations are 4.5 and 6.5. X-bar = 5.5. s_X^2 = [(4.5-5.5)^2 + (6.5-5.5)^2] / (2-1) = 2.
Y observations are 11.5 and 9.5. Y-bar = 10.5. s_Y^2 = [(9.5-10.5)^2 + (11.5-10.5)^2] / (2-1) = 2.
The pooled variance combines them: s_p^2 = [(n_X - 1)*s_X^2 + (n_Y - 1)*s_Y^2] / [(n_X - 1) + (n_Y - 1)]
= [(1)*2 + (1)*2] / [1 + 1] = 2.
Var(d) = Var(X-bar - Y-bar) = sigma^2 * (1/n_X + 1/n_Y). Estimated by s_p^2 * (1/n_X + 1/n_Y).
With s_p^2 = 2, n_X = 2, n_Y = 2: Var(d) = 2 * (1/2 + 1/2) = 2.
The mean difference: d-bar = X-bar - Y-bar = 5.5 - 10.5 = -5.
Degrees of freedom for the two-sample t: df = n_X + n_Y - 2 = 2 + 2 - 2 = 2.
For an 80% CI, look up t_{0.90, df=2} = 1.886 (the 0.90 quantile because 80% CI leaves 10% in each tail).
CI: d-bar +/- t * sqrt(Var(d)) = -5 +/- 1.886 * sqrt(2) = [-7.67, -2.33].
Since this interval does not contain 0, you can reject that the means are equal at the 80% significance level.
Setup: NYU surveys 1,000 students; only 162 respond. Freshmen are 1/4 of the student body but make up 1/2 of respondents (because they are three times more likely to respond).
The naive sample mean (sum of all 162 responses / 162) gives equal weight to each response, but the sample is 50% freshmen while the population is only 25% freshmen.
The population mean is mu = (1/4)*mu_F + (3/4)*mu_U.
The expected value of the naive sample mean with 81 freshmen and 81 non-freshmen: E[X-bar] = (81mu_F + 81mu_U) / 162 = (mu_F + mu_U) / 2.
Since (mu_F + mu_U)/2 does not equal (mu_F + 3*mu_U)/4, the naive estimator is biased.
With mu_F = 90 and mu_U = 100: the true mean is 97.5, but the naive estimator targets 95.
To fix the bias, weight the non-freshmen observations by 3 (to reflect that each non-freshman respondent represents 3x as many students in the population as each freshman respondent).
X-hat = [sum(F_i) + 3sum(U_i)] / [81 + 381] = [sum(F_i) + 3*sum(U_i)] / 324.
E[X-hat] = (81mu_F + 813mu_U) / 324 = (mu_F + 3mu_U) / 4 = mu. Unbiased.
Var(X-hat) = [81Var(F_i) + 819Var(U_i)] / 324^2 = [81162 + 819162] / 324^2 = 1.25.
Standard deviation: sigma_{X-hat} = sqrt(1.25) = 1.1180.
X-hat = (90 + 3*100) / 4 = 97.5.
95% CI: 97.5 +/- 1.96 * 1.1180 = [95.31, 99.69].
s_p^2 = \frac{(n_X - 1)\, s_X^2 + (n_Y - 1)\, s_Y^2}{(n_X - 1) + (n_Y - 1)}\text{Var}(d) = \sigma^2 \left(\frac{1}{n_X} + \frac{1}{n_Y}\right)\text{CI} = \bar{d} \pm t_{\alpha/2,\, df} \cdot \sqrt{\text{Var}(d)}, \quad df = n_X + n_Y - 2\hat{X} = \frac{\sum F_i + 3 \sum U_i}{81 + 3 \times 81} \quad \text{(weighted unbiased estimator)}\text{Var}(\hat{X}) = \frac{\sum \text{Var}(F_i) + 9 \sum \text{Var}(U_i)}{324^2}Pooled variance and two-sample inference appear in every A/B test: comparing conversion rates between a control and treatment group, comparing test scores between schools, or comparing salaries between departments. Non-response bias is one of the most common problems in survey research, from political polling (where certain demographics are harder to reach) to customer satisfaction surveys (where unhappy customers may be more or less likely to respond). Weighting is the standard fix in practice.
Students often forget that the pooled variance formula uses (n - 1) as weights, not n. The degrees of freedom from each sample are the weights.
A frequent error is using z instead of t for the two-sample CI when sigma is estimated. If you are using s_p^2 (estimated from data), the distribution is t, not z.
When computing degrees of freedom for the two-sample case: df = n_X + n_Y - 2, not n_X + n_Y.
Students commonly assume the sample mean is always unbiased. It is unbiased when the sample is representative, but with non-response bias, the sample composition differs from the population composition, making the simple average biased.
When constructing the weighted estimator, the weight of 3 on non-freshmen observations comes from the ratio of population proportion to sample proportion (3/4 divided by 1/4 = 3 relative to the freshmen weight of 1), not from the response rate difference.
⚠️ Showing an estimator is biased requires computing E[estimator] and showing it does not equal the true parameter. This is a proof-style question, and full marks require every algebraic step.
⚠️ Proposing an unbiased estimator and proving it is unbiased is a common exam question. The proof follows the same structure: compute E[new estimator] and show it equals the parameter.
⚠️ For the two-sample CI, know how to look up the correct t-quantile for the given confidence level and degrees of freedom. An 80% CI uses t_{0.90} (not t_{0.80}).
⚠️ The variance of the weighted estimator is not the same as the variance of the simple mean. The factor of 3^2 = 9 on the non-freshmen variance term is frequently missed.
True or False: The pooled variance is the simple average of the two sample variances.
False. It is a weighted average, weighted by degrees of freedom (n_i - 1).
Fill in the blank: The degrees of freedom for a two-sample t-test with n_X = 2 and n_Y = 2 is ___.
2
True or False: If 50% of your survey respondents are freshmen but only 25% of the population are freshmen, the simple sample mean is an unbiased estimator of the population mean.
False. The over-representation of freshmen biases the estimator.
Fill in the blank: To make the weighted estimator unbiased, non-freshmen observations are weighted by ___ relative to freshmen.
3
True or False: When computing the variance of a weighted estimator, you can ignore the weights in the variance formula.
False. The weights enter squared into the variance calculation.
Q: You have two samples, X = {4.5, 6.5} and Y = {11.5, 9.5}. Assuming equal population variances, compute the pooled variance.
A: s_X^2 = 2, s_Y^2 = 2. s_p^2 = [(1)(2) + (1)(2)] / [1 + 1] = 2.
Q: Using the pooled variance above, estimate Var(d) where d = X-bar - Y-bar.
A: Var(d) = s_p^2 * (1/n_X + 1/n_Y) = 2 * (1/2 + 1/2) = 2.
Q: Construct an 80% confidence interval for the difference in means (mu_X - mu_Y).
A: d-bar = 5.5 - 10.5 = -5. df = 2. t_{0.90, 2} = 1.886. CI = -5 +/- 1.886 * sqrt(2) = [-7.67, -2.33].
Q: A survey of 162 students has 81 freshmen (mu_F = 90) and 81 non-freshmen (mu_U = 100). Freshmen are 25% of the population. Show that the simple sample mean is biased and propose an unbiased estimator.
A: E[X-bar] = (mu_F + mu_U)/2 = 95. The true population mean mu = (mu_F + 3mu_U)/4 = 97.5. Since 95 does not equal 97.5, X-bar is biased. The unbiased estimator weights non-freshmen by 3: X-hat = [sum(F_i) + 3sum(U_i)] / 324. E[X-hat] = (mu_F + 3*mu_U)/4 = mu.
Q: Compute the 95% CI for the population mean using the weighted estimator.
A: Var(X-hat) = [81162 + 819162] / 324^2 = 1.25. sigma = 1.118. X-hat = 97.5. CI = 97.5 +/- 1.961.118 = [95.31, 99.69].
The pooled variance and two-sample t-interval are the foundation for the independent-samples t-test, which you will encounter in hypothesis testing. The concept of bias in estimation connects to the broader topic of estimator properties (consistency, efficiency, sufficiency) covered later in the course. Non-response bias also ties into experimental design: randomised experiments avoid this problem, but observational studies and surveys almost always face it.
Pooled variance, pooled standard deviation, two-sample t-test, independent samples, difference in means, confidence interval for difference, Var(d), equal variance assumption, non-response bias, selection bias, sampling bias, representative sample, unbiased estimator, biased estimator, weighted mean, weighted estimator, inverse probability weighting, population mean, survey statistics, response rate