Source: Tanhane/Dunlop, Section 6.3, Chapter 7, Sections 9.1, 14.1
Tags: Z-test, t-test, population mean, known sigma, unknown sigma, power function, sample size determination, robustness, skewness, heavy tails, non-normal, central limit theorem, noncentrality parameter
When testing hypotheses about a population mean μ, the choice of test statistic depends on whether σ is known and whether the population is normally distributed. The Z-test applies when σ is known; the t-test applies when σ is unknown. Both assume normality, but the t-test is reasonably robust to moderate departures for larger samples. For highly skewed or heavy-tailed populations with small samples, the t-test can give misleading results, and nonparametric alternatives are preferred.
Z-test statistic (σ known)
Z = √n(Ȳ − μ₀) / σ. Follows N(0, 1) when μ = μ₀. Used when the population is normal and σ is known.
t-test statistic (σ unknown)
T = √n(Ȳ − μ₀) / S. Follows a central t-distribution with df = n − 1 when μ = μ₀ and the population is normal.
Noncentral t-distribution
The distribution of T = √n(Ȳ − μ₀) / S when μ ≠ μ₀. Has noncentrality parameter Δ = √n(μ − μ₀) / σ and df = n − 1. Used for computing power of the t-test.
Noncentrality parameter (Δ)
Δ = √n(μ − μ₀) / σ. Measures how far the true mean is from the hypothesised value in standardised units. When μ = μ₀, Δ = 0 and the distribution reduces to the central t.
Sample size determination
Finding the minimum n so that a size α test achieves power ≥ 1 − β whenever |μ − μ₀| ≥ δ.
Suppose Y₁, ..., Yₙ are iid N(μ, σ²) with σ known. The test statistic is:
Z = √n(Ȳ − μ₀) / σ
The distribution of Z is N(√n(μ − μ₀)/σ, 1), which depends on the true value of μ.
Right-sided test: H₀: μ ≤ μ₀ vs H₁: μ > μ₀
Reject H₀ if Ȳ > μ₀ + z_α · σ/√n
Power: γ(μ) = 1 − Φ(z_α + √n(μ₀ − μ)/σ)
p-value = 1 − Φ(√n(ȳ − μ₀)/σ)
Left-sided test: H₀: μ ≥ μ₀ vs H₁: μ < μ₀
Reject H₀ if Ȳ < μ₀ − z_α · σ/√n
Power: γ(μ) = Φ(−z_α + √n(μ₀ − μ)/σ)
p-value = Φ(√n(ȳ − μ₀)/σ)
Two-sided test: H₀: μ = μ₀ vs H₁: μ ≠ μ₀
Reject H₀ if |√n(Ȳ − μ₀)/σ| > z_{α/2}
Power: γ(μ) = Φ(−z_{α/2} + √n(μ₀ − μ)/σ) + 1 − Φ(z_{α/2} + √n(μ₀ − μ)/σ)
p-value = 2(1 − Φ(|√n(ȳ − μ₀)/σ|))
Test H₀: μ ≤ 30 vs H₁: μ > 30 in N(μ, 81) with n = 25, α = .01.
Critical value: Ȳ ≥ 30 + (2.326)(9)/√25 = 34.19
At μ = 30: sampling distribution is N(30, 3.24), area beyond 34.19 is γ(30) = .01
At μ = 33: sampling distribution is N(33, 3.24), area beyond 34.19 is γ(33) = .2543
This illustrates that with n = 25, the power to detect a shift of 3 units from μ₀ = 30 is only about 25%.
Suppose Y₁, ..., Yₙ are iid N(μ, σ²) with σ unknown. The test statistic is:
T = √n(Ȳ − μ₀) / S
Under H₀ (when μ = μ₀), T follows a central t-distribution with df = n − 1. Under H₁, T follows a noncentral t-distribution with df = n − 1 and Δ = √n(μ − μ₀)/σ.
Right-sided: H₀: μ ≤ μ₀ vs H₁: μ > μ₀
Reject H₀ if T > t_{α, n−1}
p-value = 1 − pt(T_obs, n − 1)
Power: γ(μ) = 1 − pt(qt(1 − α, n − 1), n − 1, Δ) (using noncentral t cdf in R)
Left-sided: H₀: μ ≥ μ₀ vs H₁: μ < μ₀
Reject H₀ if T < −t_{α, n−1}
p-value = pt(T_obs, n − 1)
Power: γ(μ) = pt(−qt(1 − α, n − 1), n − 1, Δ)
Two-sided: H₀: μ = μ₀ vs H₁: μ ≠ μ₀
Reject H₀ if |T| > t_{α/2, n−1}
p-value = 2(1 − pt(|T_obs|, n − 1))
Test H₀: μ ≤ 84 vs H₁: μ > 84, n = 10, α = .05. From data: Ȳ = 86.3, S = 3.2.
T = √10(86.3 − 84)/3.2 = 2.27
Critical value: t_{.05, 9} = 1.833
Since 2.27 > 1.833, reject H₀
p-value = 1 − pt(2.27, 9) = .0247
95% lower bound on μ: 86.3 − 1.833(3.2/√10) = 84.45, which exceeds 84
Given specifications: significance level α, desired power 1 − β, minimum detectable difference δ from μ₀, and known σ.
One-sided alternative (H₁: μ > μ₀ or H₁: μ < μ₀):
n = ⌈σ(z_α + z_β) / δ⌉²
Two-sided alternative (H₁: μ ≠ μ₀), approximate:
n = ⌈σ(z_{α/2} + z_β) / δ⌉²
Test H₀: μ ≤ 20 vs H₁: μ > 20, σ ≈ 40, α = .05, want 90% power to detect μ = 35.
δ = 35 − 20 = 15
β = 0.1, so z_β = 1.282
z_α = z_{.05} = 1.645
n = [40(1.645 + 1.282)/15]² = 60.9, so n ≥ 61
More complex because df = n − 1 depends on n itself, requiring iterative solutions. In practice:
Use Table A11 (from the handout) with inputs: α, β, and ϕ = |μ − μ₀|/σ
Use R: power.t.test(delta=, sd=, sig.level=, power=, type="one.sample", alternative="one.sided")
Test H₀: μ ≤ 84 vs H₁: μ > 84, α = .05, power ≥ 80% to detect μ = 84.8, σ < 2.
ϕ = |84.8 − 84|/2 = 0.4
From Table A11 with α = .05, β = .2, ϕ = .4: n = 40
R confirmation: power.t.test(n=40, delta=.8, sd=2, sig.level=.05, type="one.sample", alternative="one.sided") gives power = 0.7997
The t-test assumes normality. Its behaviour under departures:
Heavy-tailed symmetric distributions:
True significance level tends to be smaller than nominal α for one-sided upper tests
Power drops considerably compared to the normal case
More Type II errors than expected
Skewed distributions (small samples, n < 20):
For H₁: μ > μ₀ with right-skewed data, the true level can be much less than .05 (e.g. .007 for heavily skewed data with n = 10), resulting in very low power
For H₁: μ < μ₀ with right-skewed data, the true level can be much greater than .05, inflating Type I error risk
Two-sided tests will also have inflated size
Practical guidance:
Always examine the data first via box plots or normal probability plots
If plots show no extreme skewness or many outliers, t-based procedures should be reasonably correct
If plots reveal severe skewness or heavy tails, the t-test is suspect and nonparametric methods should be used
The median becomes a more appropriate measure of centre than the mean for highly skewed data
For a nominal α = .05 test of H₁: μ > μ₀ (shifts measured in d = (μ − μ₀)/σ units):
Distribution | n = 10, d = 0 | n = 10, d = .6 | n = 20, d = 0 | n = 20, d = .6 |
|---|---|---|---|---|
Normal | .05 | .543 | .05 | .833 |
Heavy-tailed | .035 | .371 | .045 | .554 |
Light skew | .025 | .437 | .041 | .762 |
Heavy skew | .007 | .277 | .011 | .658 |
The heavy-skew row is most alarming: true level is far below .05, and power is severely reduced.
When observations are positively correlated (ρ > 0), two problems arise:
Var(X̄) > σ²/n, so S/√n underestimates the true standard error
Confidence intervals are too narrow (coverage < nominal)
The test statistic is inflated, so P[Type I error] > α
For equicorrelated data: Var(X̄) = (σ²/n)[1 + (n − 1)ρ]
For first-order autoregressive data: Var(X̄) ≈ (σ²/n)[(1 + ρ)/(1 − ρ)]
When correlation is detected, the standard error and critical values must be adjusted. Courses in time series (STAT 626), multivariate analysis (STAT 636), and spatial statistics (STAT 647) cover these adjustments.
Large n: The Central Limit Theorem ensures √n(X̄ − μ₀)/S is approximately t-distributed. How large n needs to be depends on the degree of skewness and heavy-tailedness.
Small n with non-normal data:
Box-Cox transformations can sometimes normalise the data, but identifying the correct transformed mean μ₀* is problematic and can lead to tests with size very different from α
Transformations in this situation are not generally recommended
Preferred alternatives: Sign Test or Wilcoxon Signed-Rank Test (covered in Part 3)
Z-test statistic: Z = √n(Ȳ − μ₀) / σ
t-test statistic: T = √n(Ȳ − μ₀) / S
Noncentrality parameter: Δ = √n(μ − μ₀) / σ
Power function (Z-test, right-sided): γ(μ) = 1 − Φ(z_α + √n(μ₀ − μ)/σ)
Sample size (one-sided): n = ⌈σ(z_α + z_β) / δ⌉²
Sample size (two-sided, approx.): n = ⌈σ(z_{α/2} + z_β) / δ⌉²
R functions for t-test power and sample size:
power.t.test(n=, delta=, sd=, sig.level=, power=, type="one.sample", alternative=)
Power via noncentral t: 1 - pt(qt(1-alpha, df), df, delta_ncp)
⚠️ The Z-test requires σ to be known, which is unrealistic in practice. The t-test is the standard for real-world applications.
⚠️ The t-test is not robust to severe skewness with small samples. Always check normality before applying it.
⚠️ For right-skewed data with a right-sided alternative (H₁: μ > μ₀), the t-test becomes overly conservative (true α much less than nominal), meaning you lose power and miss real effects.
⚠️ The p-value from the t-test is computed under the central t-distribution (Δ = 0). The noncentral t is used only for power calculations.
⚠️ Sample size formulas for the Z-test have a closed form; for the t-test they require iteration or tables, because df depends on n.
⚠️ Positive correlation in data inflates the t-statistic and increases the true Type I error rate above the nominal level.
Q: When σ is unknown and the data come from a normal population, what distribution does the test statistic T = √n(Ȳ − μ₀)/S follow under H₀?
A: A central t-distribution with df = n − 1.
Q: A researcher uses a t-test with n = 10 and α = .05 on data from a heavily right-skewed population. If the alternative is H₁: μ > μ₀, will the true Type I error rate be above or below .05?
A: Below .05. The right skew shifts the upper tail of the t-statistic's sampling distribution inward, making it harder to reject H₀ than the nominal α suggests.
Q: How do you compute the power of a t-test at a specific alternative μ₁ using R?
A: Use the noncentral t cdf: γ(μ₁) = 1 − pt(qt(1 − α, n − 1), n − 1, Δ), where Δ = √n(μ₁ − μ₀)/σ.
Q: Find the minimum sample size for a one-sided Z-test with α = .05, power = .80, σ = 10, δ = 5.
A: n = [10(1.645 + 0.842)/5]² = [10(2.487)/5]² = [4.974]² = 24.74, so n ≥ 25.
Q: Why can't you use the same closed-form sample size formula for the t-test as for the Z-test?
A: Because the critical value of the t-test depends on df = n − 1, which itself depends on the unknown n. This creates a circular dependency that requires iterative solution.
Q: If your data are positively correlated, what happens to the coverage probability of the standard confidence interval X̄ ± t_{α/2} · S/√n?
A: It drops below the nominal 100(1 − α)% level because S/√n underestimates the true standard error of X̄ when observations are positively correlated.
Z-test, t-test, one-sample t-test, population mean, known variance, unknown variance, test statistic, rejection region, power function, noncentral t-distribution, noncentrality parameter, sample size determination, power analysis, Type I error, Type II error, robustness, skewness, heavy tails, kurtosis, Central Limit Theorem, Box-Cox transformation, correlated data, autocorrelation, equicorrelation, autoregressive, standard error, confidence interval