Source: Handout #13, Tamhane/Dunlop Ch. 8, Ch. 9
Tags: pooled t-test, separate variance t-test, Welch test, Satterthwaite, Wilcoxon rank sum, Mann-Whitney U, paired t-test, Wilcoxon signed rank, two-sample test, hypothesis testing, multiple populations, sample size, power function, noncentrality parameter
When comparing two population means, the choice of test depends on three conditions: normality of the distributions, equality of variances, and independence of samples. If the data are normal with equal variances, use the pooled t-test. If variances differ, use the separate variance (Welch) t-test. If the distributions are nonnormal, use the Wilcoxon rank sum test. For paired data, use the paired t-test or Wilcoxon signed rank test depending on normality of the differences.
Parameter of interest (θ)
θ = µ₁ − µ₂, estimated by θ̂ = X̄ − Ȳ. This is the quantity every two-sample location test is ultimately about.
Pooled standard deviation (Sp)
Sp = √[(n − 1)S₁² + (m − 1)S₂²] / (n + m − 2). A weighted average of the two sample standard deviations, used when σ₁ = σ₂ is assumed.
Noncentrality parameter (∆)
∆ = (θ − θ₀) / [σ √(1/n + 1/m)]. Governs how far the t-distribution shifts under the alternative hypothesis, and therefore controls the power of the test.
Wilcoxon rank sum statistic (W₁)
The sum of the ranks assigned to one sample after ranking all N = n + m observations together. Equivalent to the Mann-Whitney U statistic via the relationship W₁ = U₁ + n(n + 1)/2.
Paired differences (Dᵢ)
Dᵢ = Xᵢ − Yᵢ. Converting paired data to differences reduces the two-sample problem to a one-sample problem on D̄, eliminating the need to estimate the correlation ρ between the two samples.
Three conditions must hold:
C1: Both populations are normally distributed
C2: σ₁ = σ₂ = σ (unknown)
C3: The X and Y samples are independent
Under these conditions, θ̂ = X̄ − Ȳ follows a normal distribution with mean µ₁ − µ₂ and variance σ²(1/n + 1/m).
The test statistic is:
T = (X̄ − Ȳ − θ₀) / [Sp √(1/n + 1/m)]
Under H₀, T follows a central t-distribution with df = n + m − 2.
Under Hₐ, T follows a noncentral t-distribution with df = n + m − 2 and noncentrality parameter ∆.
Three hypothesis forms:
Upper-tailed: H₀: µ₁ − µ₂ ≤ θ₀ vs H₁: µ₁ − µ₂ > θ₀. Reject if T > t(α, df). p-value = 1 − pt(T, df).
Lower-tailed: H₀: µ₁ − µ₂ ≥ θ₀ vs H₁: µ₁ − µ₂ < θ₀. Reject if T < −t(α, df). p-value = pt(T, df).
Two-sided: H₀: µ₁ − µ₂ = θ₀ vs H₁: µ₁ − µ₂ ≠ θ₀. Reject if |T| > t(α/2, df). p-value = 2(1 − pt(|T|, df)).
R commands:
Critical value: qt(1 - alpha, df)
p-value (upper): 1 - pt(Tobs, df)
Power (upper): 1 - pt(qt(1 - alpha, df), df, delta)
The pooled t-test can break down when its conditions are violated. A simulation study varying σ₁ = kσ₂ across different sample sizes demonstrates this clearly.
Effect of unequal variances (Table 1, nominal α = .05):
When n₁ = n₂ = 10, the actual Type I error rate ranged from .048 to .064 regardless of the variance ratio. Equal sample sizes protect the test.
When n₁ = 10, n₂ = 40, and σ₁ = 4σ₂ (smaller sample paired with larger variance), the error rate reached .295, nearly six times the nominal level.
The mechanism: when the smaller sample comes from the population with larger variance, the pooled Sp² underestimates the true variance of X̄ − Ȳ, inflating the t-statistic and producing too many rejections.
⚠️ The most dangerous scenario is smaller sample + larger variance. Equal sample sizes are a strong safeguard against unequal variance problems.
When σ₁ ≠ σ₂, modify the test statistic to use separate variance estimates:
t* = (X̄ − Ȳ) / √(S₁²/n + S₂²/m)
The degrees of freedom are approximated by the Welch-Satterthwaite formula:
ν = [(C + 1)² (n − 1)(m − 1)] / [C²(m − 1) + (n − 1)], where C = (S₁²/n) / (S₂²/m)
Key simulation results (Table 2): The separate variance t-test's Type I error rate stayed consistently close to .05 across all combinations of sample sizes and variance ratios tested. It is far more robust to unequal variances than the pooled test.
When to use which:
If sample sizes are equal and variances are equal, both tests give identical results, but the pooled test has higher power (larger df).
If sample sizes and variances are very unequal, the separate variance test is more reliable.
Many textbooks use df = min(n − 1, m − 1) as a conservative shortcut. This is not recommended because it substantially reduces power. For example, with n = 12, m = 15, S₁ = 6, S₂ = 10, the Welch df is 23.4, the pooled df is 25, but the conservative df is only 11.
⚠️ Always examine both means and standard deviations when populations differ. A difference in means alone is an inadequate summary if the variances also differ.
When population distributions are nonnormal but belong to the same location-scale family with equal scale parameters, the Wilcoxon rank sum test provides a distribution-free alternative.
Implementation:
Order all N = n + m observations from smallest to largest
Replace values with their ranks (1 to N). For ties, assign the average rank.
Compute W₁ = sum of ranks for sample 1 (or sample 2)
Compare W₁ to critical values from the Wilcoxon rank sum table or use R functions
Hypothesis tests:
Upper: H₀: θ₁ ≤ θ₂. Reject if W₁ ≥ W(α, n, m). R: wilcox.test(x, y, alternative="g", paired=F)
Two-sided: H₀: θ₁ = θ₂. Reject if W₁ is extreme in either direction. R: wilcox.test(x, y, alternative="t", paired=F)
Note: R's wilcox.test reports the Mann-Whitney U, not W₁ directly. The relationship is W₁ = U₁ + n(n+1)/2.
Normal approximation (large samples): When n and m are large, Z = (W₁ − µ) / σ is approximately N(0,1), where µ = n(n + m + 1)/2 and σ = √[nm(n + m + 1)/12].
A simulation by Randles and Wolfe (1979) compared the power of the t-test and Wilcoxon test across four distribution types (normal, double exponential, Cauchy, Weibull) at shifts of 0, 0.6σ, and 1.2σ.
Findings:
Normal distributions: the t-test has only slightly higher power than the Wilcoxon test.
Double exponential (symmetric, heavy-tailed): the Wilcoxon test has greater power.
Cauchy (extremely heavy-tailed): the t-test's actual Type I error rate deviates badly from .05, and its power is much lower than the Wilcoxon test.
Weibull (skewed): performance was similar for both tests.
The t-test is quite robust to moderate skewness, but not to heavy tails or extreme outliers.
⚠️ The Wilcoxon test maintains its stated Type I error rate for all distributions. The t-test does not, particularly with heavy tails or severe skewness.
Pairing is used to reduce variability when experimental units are highly heterogeneous. Units within a pair are more alike than units across pairs.
Types of pairing:
Common attribute: two treatments assigned within matched pairs (e.g., patients matched by disease severity)
Before/after: same unit measured twice (e.g., soil pH before and after mining)
Why pairing helps: For paired data, SE(Ȳ₁ − Ȳ₂) = √[(σ₁² + σ₂² − 2σ₁σ₂ρ) / n]. When ρ is positive (paired measurements are similar), this is smaller than the independent-sample standard error. However, the df also drops from 2(n − 1) to n − 1, so there is a trade-off between reduced variability and reduced degrees of freedom.
Paired t-test (normal differences):
Transform to differences Dᵢ = Xᵢ − Yᵢ, then apply the one-sample t-test:
T = √n (D̄ − µ_D₀) / S_D, with df = n − 1
This eliminates the need to estimate ρ.
Paired data, nonnormal differences:
If the distribution of Dᵢ is symmetric and continuous: use the Wilcoxon signed rank test on the Dᵢ values.
If the distribution of Dᵢ is continuous but not symmetric: use the sign test.
R: t.test(x, y, paired=T) or wilcox.test(x, y, paired=T)
Two independent samples (n = m, normal, equal variances):
n = 2 × [σ(z_α + z_β) / δ]²
For a two-sided test, replace z_α with z_(α/2). Here δ = µ₁ − µ₂ is the minimum detectable difference, and σ is a researcher-supplied estimate of the common standard deviation.
Unequal sample sizes (m = kn):
n = [(k + 1)/k] × [σ(z_α + z_β) / δ]² and m = kn
Paired samples:
n = [σ_D (z_α + z_β) / δ]²
where σ_D is the standard deviation of the differences.
R function: power.t.test(n=, delta=, sd=, sig.level=, power=, type=c("two.sample","paired"), alternative=c("one.sided","two.sided"))
Decision tree for two-sample location tests:
Normal data, σ₁ = σ₂ → Pooled t-test (df = n + m − 2)
Normal data, σ₁ ≠ σ₂ → Separate variance t-test (df = ν, Welch-Satterthwaite)
Nonnormal data, independent → Wilcoxon rank sum test
Paired data, normal differences → Paired t-test (df = n − 1)
Paired data, nonnormal symmetric differences → Wilcoxon signed rank test
Paired data, nonnormal asymmetric differences → Sign test
Pooled Sp formula:
Sp = √[(n − 1)S₁² + (m − 1)S₂²] / (n + m − 2)
When n = m, this simplifies to Sp² = (S₁² + S₂²) / 2.
Welch-Satterthwaite df:
ν = [(S₁²/n + S₂²/m)²] / [(S₁²/n)² / (n − 1) + (S₂²/m)² / (m − 1)]
(Equivalent form to the one in the handout.)
⚠️ The most common mistake is using the pooled t-test when variances are unequal and sample sizes differ. This inflates the Type I error rate dramatically.
⚠️ Equal sample sizes protect the pooled t-test against unequal variances. Assign equal sample sizes unless there is a strong reason not to.
⚠️ The conservative df = min(n − 1, m − 1) for the separate variance test is widely taught but substantially reduces power. Use the Welch-Satterthwaite df instead.
⚠️ For paired data, always work with the differences Dᵢ. Do not treat paired observations as independent samples.
⚠️ The Wilcoxon rank sum and Mann-Whitney U tests are mathematically equivalent (W₁ = U₁ + n(n+1)/2). R reports U, not W.
⚠️ Cluster effects and serial/spatial correlation violate the independence assumption. The pooled t-test will produce incorrect coverage probabilities and inflated Type I error rates with dependent data.
Q: You have two independent samples of sizes n = 15 and m = 30 from normal populations. The sample standard deviations are S₁ = 12 and S₂ = 4. Which test should you use and why?
A: The separate variance (Welch) t-test. The sample sizes are unequal and the ratio of standard deviations is 3:1. Table 1 from the handout shows that this combination produces heavily inflated Type I error rates with the pooled test.
Q: What three conditions must hold for the pooled t-test to be valid?
A: Both populations must be normally distributed, both must have equal variances (σ₁ = σ₂), and the two samples must be independent.
Q: In a Wilcoxon rank sum test with n = m = 10, you compute W₁ = 140. The normal approximation gives µ = 105 and σ = 13.23. What is the approximate p-value for a one-sided upper test?
A: Z = (140 − 105) / 13.23 = 2.646. p-value = P(Z > 2.646) = 1 − Φ(2.646) ≈ 0.004.
Q: A researcher wants an α = .05 test with 90% power to detect a difference of 1.5 units between two population means, with σ = 3. What sample sizes are needed (n = m)?
A: n = 2 × [3(1.645 + 1.282) / 1.5]² = 2 × 34.25 ≈ 69. Use n = m = 69 (or 70 per R's power.t.test).
Q: Why does pairing reduce the standard error of the difference in means?
A: Because the covariance term −2σ₁σ₂ρ enters the variance formula. When paired measurements are positively correlated (ρ > 0), this term is negative, shrinking the overall variance. The trade-off is fewer degrees of freedom (n − 1 instead of 2n − 2).
Q: When should you use the Wilcoxon rank sum test over the t-test?
A: When the Shapiro-Wilk test or normal probability plots indicate nonnormality, particularly heavy-tailed or outlier-prone distributions. The Wilcoxon test maintains its nominal Type I error rate regardless of the population distribution, while the t-test does not under nonnormality.
two-sample t-test, pooled t-test, equal variance t-test, separate variance t-test, Welch t-test, Satterthwaite approximation, Welch-Satterthwaite degrees of freedom, Wilcoxon rank sum test, Mann-Whitney U test, paired t-test, matched pairs, Wilcoxon signed rank test, sign test, noncentrality parameter, power function, sample size determination, Type I error, Type II error, robustness, homogeneity of variance, independence assumption, cluster effect, serial correlation, spatial correlation, distribution-free test, nonparametric test, rank-based test, location-scale family