Difficulty: Intermediate | Prerequisites: Sampling distributions, standard error, z-scores, basic confidence intervals.
This set of notes covers the methods you need when comparing two independent groups, whether the outcome is continuous (t-tests) or binary (proportion tests). It draws on Problems 1, 5, and 6 of the NYU Analytical Statistics Exam 2 practice set. If you are comfortable with one-sample z- and t-tests, you have the foundation; these notes extend the same logic to two groups.
When you have two independent groups and want to know whether their means (or proportions) differ, you pick a test based on what you know about the variances and whether the outcome is continuous or binary. Equal variances and continuous data: pooled two-sample t. Unequal variances: Welch's t. Binary outcome (pass/fail, yes/no): the two-proportion z-test. Each method constructs a test statistic that measures how far apart the groups are relative to sampling variability, and each produces a confidence interval for the true difference.
Pooled variance (s_p^2)
A weighted average of the two sample variances, used when you assume both populations share the same variance. The weights are the degrees of freedom from each group. In simple terms: you blend the two groups' variability into one common estimate so you can build a single standard error.
Welch's t-test
A two-sample t-test that does not assume equal population variances. It uses each group's own sample variance in the standard error and adjusts the degrees of freedom downward with the Welch-Satterthwaite approximation. Think of it as the safer default when you have no good reason to believe the variances are equal.
Pooled proportion (p-hat)
The overall success rate computed by combining both samples, used under the null hypothesis that the two population proportions are equal. In simple terms: you lump everyone together and ask, "What fraction succeeded overall?"
Standard error of a difference
The standard deviation of the sampling distribution of the difference between two statistics (two means or two proportions). It measures how much the observed difference would bounce around across repeated samples.
Degrees of freedom (df)
The number of independent pieces of information left after estimating parameters. For the pooled t-test: df = n_1 + n_2 - 2. For Welch's t: df is computed by the Welch-Satterthwaite formula. Think of it as the denominator's "budget" of information, which determines how heavy the tails of the t-distribution are.
Confidence interval for a difference
An interval estimate of the form (point estimate) +/- (critical value)(standard error). For means, the critical value comes from the t-distribution; for proportions, from the standard normal.
The setup: two independent groups (Classical music, n_C = 2, scores 85 and 81; Jazz music, n_J = 2, scores 72 and 68). We assume equal population variances and normality.
Sample means
C-bar = (85 + 81) / 2 = 83
J-bar = (72 + 68) / 2 = 70
d-bar = C-bar - J-bar = 13
The pooled variance formula is:
s_p^2 = \frac{(n_C - 1)s_C^2 + (n_J - 1)s_J^2}{n_C + n_J - 2}Compute each sample variance:
s_C^2 = [(85 - 83)^2 + (81 - 83)^2] / (2 - 1) = (4 + 4) / 1 = 8
s_J^2 = [(72 - 70)^2 + (68 - 70)^2] / (2 - 1) = (4 + 4) / 1 = 8
Pooled variance:
s_p^2 = [(1)(8) + (1)(8)] / (2 + 2 - 2) = 16 / 2 = 8
The variance of the difference in sample means under the pooled assumption is:
\text{Var}(\bar{d}) = s_p^2 \left(\frac{1}{n_C} + \frac{1}{n_J}\right)Var(d-bar) = 8 (1/2 + 1/2) = 8(1) = 8
Standard error: SE = sqrt(8) = 2sqrt(2) ≈ 2.828
Degrees of freedom: df = n_C + n_J - 2 = 2.
For a 90% CI with df = 2, the critical value is t_(0.05, 2) = 2.920.
The interval:
d-bar +/- t * SE = 13 +/- 2.920 * 2.828
Margin of error = 2.920 * 2.828 ≈ 8.258
90% CI: (13 - 8.258, 13 + 8.258) = (4.742, 21.258)
Interpretation: we are 90% confident that the true difference in mean memory scores (Classical minus Jazz) lies between about 4.7 and 21.3 points.
Key formula summary for the pooled t-test:
t = \frac{\bar{X}_1 - \bar{X}_2}{s_p \sqrt{\frac{1}{n_1} + \frac{1}{n_2}}}, \quad df = n_1 + n_2 - 2The setup: Program A (n_1 = 12, x-bar_1 = 84, s_1 = 6) vs Programme B (n_2 = 10, x-bar_2 = 78, s_2 = 11). Do not assume equal variances.
H0: mu_1 = mu_2 (no difference in mean outcomes)
H1: mu_1 ≠ mu_2 (the programmes differ)
Because the sample standard deviations are quite different (6 vs 11), the pooled t-test is inappropriate. Use Welch's t:
t = \frac{\bar{X}_1 - \bar{X}_2}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}}Compute the standard error:
s_1^2 / n_1 = 36 / 12 = 3
s_2^2 / n_2 = 121 / 10 = 12.1
SE = sqrt(3 + 12.1) = sqrt(15.1) ≈ 3.886
Test statistic:
t = (84 - 78) / 3.886 = 6 / 3.886 ≈ 1.544
Welch-Satterthwaite degrees of freedom:
df = \frac{\left(\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}\right)^2}{\frac{(s_1^2/n_1)^2}{n_1 - 1} + \frac{(s_2^2/n_2)^2}{n_2 - 1}}Numerator: (3 + 12.1)^2 = (15.1)^2 = 228.01
Denominator: (3^2)/11 + (12.1^2)/9 = 9/11 + 146.41/9 = 0.818 + 16.268 = 17.086
df = 228.01 / 17.086 ≈ 13.34
Round down to df ≈ 13 for a t-table lookup. The two-tailed critical value at alpha = 0.05 with df = 13 is t_(0.025, 13) ≈ 2.160.
Since |t| = 1.544 < 2.160, we fail to reject H0. There is not sufficient evidence at the 5% level that the programmes differ.
CI = (x-bar_1 - x-bar_2) +/- t_(0.025, df) * SE
CI = 6 +/- 2.160 * 3.886
Margin of error = 8.394
95% CI: (6 - 8.394, 6 + 8.394) = (-2.394, 14.394)
The interval includes zero, consistent with failing to reject H0.
The pooled procedure assumes sigma_1 = sigma_2. Here s_1 = 6 and s_2 = 11, nearly a 2:1 ratio. When the true variances differ and sample sizes are unequal, the pooled t can have an actual Type I error rate that deviates substantially from the nominal alpha. Welch's t adjusts both the standard error and the degrees of freedom, so it controls the Type I error rate properly regardless of whether the variances are equal.
The setup: 180 first-year students, 117 support a policy (p-hat_1 = 117/180 = 0.65). 220 seniors, 121 support (p-hat_2 = 121/220 = 0.55).
H0: p_1 = p_2
H1: p_1 ≠ p_2
Pooled proportion (used for the test, because under H0 the proportions are equal):
p-hat = (117 + 121) / (180 + 220) = 238 / 400 = 0.595
Pooled standard error:
SE_{\text{pooled}} = \sqrt{\hat{p}(1 - \hat{p})\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}SE = sqrt(0.595 * 0.405 * (1/180 + 1/220))
1/180 + 1/220 = 0.005556 + 0.004545 = 0.010101
SE = sqrt(0.595 * 0.405 * 0.010101) = sqrt(0.002433) ≈ 0.04933
Test statistic:
z = (p-hat_1 - p-hat_2) / SE = (0.65 - 0.55) / 0.04933 = 0.10 / 0.04933 ≈ 2.027
Two-tailed critical value at alpha = 0.05: z_(0.025) = 1.96.
Since |z| = 2.027 > 1.96, we reject H0. There is statistically significant evidence that the support rates differ between first-years and seniors.
For the confidence interval, use the unpooled standard error (because we are no longer assuming H0 is true):
SE_{\text{unpooled}} = \sqrt{\frac{\hat{p}_1(1 - \hat{p}_1)}{n_1} + \frac{\hat{p}_2(1 - \hat{p}_2)}{n_2}}= sqrt((0.65 * 0.35)/180 + (0.55 * 0.45)/220)
= sqrt(0.001264 + 0.001125)
= sqrt(0.002389) ≈ 0.04888
95% CI: (0.65 - 0.55) +/- 1.96 * 0.04888 = 0.10 +/- 0.0958 = (0.0042, 0.1958)
The interval is just barely above zero, consistent with the marginal rejection.
The rule of thumb: n * p-hat >= 5 and n * (1 - p-hat) >= 5 for each group.
First-years: 180 * 0.65 = 117 >= 5, and 180 * 0.35 = 63 >= 5. Fine.
Seniors: 220 * 0.55 = 121 >= 5, and 220 * 0.45 = 99 >= 5. Fine.
The normal approximation is reasonable.
The hypothesis test uses the pooled standard error, because under H0 the two population proportions are equal, so we estimate a single common proportion.
The confidence interval uses the unpooled standard error, because we are estimating the actual difference and do not assume the proportions are equal.
Students often use the pooled t-test by default, even when the sample standard deviations are very different. If s_1 and s_2 differ substantially (a rough guide: the ratio exceeds 2:1), Welch's t is the safer choice.
Students sometimes use the unpooled standard error for the hypothesis test of two proportions. The test assumes H0: p_1 = p_2 is true, so you pool. The confidence interval does not assume equality, so you do not pool.
A confidence interval that includes zero does not mean the population difference is zero. It means the data are consistent with zero difference, but also consistent with the other values in the interval.
Students confuse the degrees of freedom for the pooled t (n_1 + n_2 - 2) with the Welch df (the Welch-Satterthwaite formula). The Welch df is always smaller than or equal to the pooled df, which makes the critical value larger and the test more conservative.
⚠️ You will almost certainly be asked to choose between pooled t and Welch's t. Know the decision rule: if the problem says "assume equal variances," use pooled. Otherwise, or if the sample SDs look very different, use Welch's.
⚠️ The two-proportion z-test frequently appears. Be ready to compute the pooled proportion and explain why pooling is appropriate for the test but not for the CI.
⚠️ Know the normal approximation conditions for proportions (np >= 5, n(1-p) >= 5) and be prepared to verify them.
⚠️ Confidence interval construction is heavily tested. The pattern is always: point estimate +/- (critical value)(standard error). What changes across problems is which critical value (z vs t) and which standard error formula.
True or False: The pooled two-sample t-test requires the assumption that both populations have the same variance.
Answer: True.
Fill in the blank: For the two-proportion z-test, the standard error used in the test statistic is based on the ______ proportion.
Answer: pooled.
True or False: Welch's t-test always has more degrees of freedom than the pooled t-test.
Answer: False. Welch's df is always less than or equal to the pooled df.
Fill in the blank: A 95% confidence interval for mu_1 - mu_2 that includes zero is consistent with ______ to reject H0 at alpha = 0.05.
Answer: failing.
Q: Two groups have sample sizes n_1 = 15 and n_2 = 20, with sample variances s_1^2 = 9 and s_2^2 = 9. Compute the pooled variance.
A: s_p^2 = [(14)(9) + (19)(9)] / (15 + 20 - 2) = (126 + 171) / 33 = 297/33 = 9. When the sample variances are equal, the pooled variance equals that common value.
Q: In a two-proportion test, you compute z = 1.80 for a two-tailed test at alpha = 0.05. What is your conclusion?
A: The critical value is 1.96. Since 1.80 < 1.96, you fail to reject H0. There is not sufficient evidence that the proportions differ.
Q: Why does the confidence interval for p_1 - p_2 use an unpooled standard error while the hypothesis test uses a pooled one?
A: The test assumes H0: p_1 = p_2 is true, so a single pooled estimate of the common proportion is appropriate. The confidence interval estimates the actual difference without assuming equality, so each group's own proportion is used.
Q: A Welch's t-test gives df = 17.4. You are using a t-table that only lists integer df. What do you do?
A: Round down to df = 17 to be conservative (the critical value will be slightly larger, making it harder to reject).
This material connects directly to one-sample inference: the two-sample t-test is the natural extension of the one-sample t-test to comparisons between groups, and the two-proportion z-test extends the one-proportion z-test. It also connects to ANOVA, which generalises the two-sample t-test to three or more groups. Understanding the pooled vs unpooled standard error distinction here prepares you for the same distinction in more advanced regression settings.
Related Terms / Search Tags
two-sample t-test, independent samples t-test, pooled variance, pooled standard deviation, Welch's t-test, Welch-Satterthwaite degrees of freedom, unequal variances, two-proportion z-test, pooled proportion, unpooled standard error, confidence interval for difference in means, confidence interval for difference in proportions, normal approximation for proportions, Type I error, significance level, NYU analytical statistics, ASTATS exam 2