Difficulty: Advanced | Prerequisites: Ch. 8–9 (confidence intervals, hypothesis testing, t-distribution)
These chapters extend inference from one sample to comparisons between groups and to relationships between variables. Ch. 10 compares two population means (independent or paired). Ch. 11 (ANOVA) generalises that comparison to k groups at once. Ch. 12 (regression) models the linear relationship between an explanatory variable X and a response variable Y. This is the capstone material for the course, and exam questions here tend to be longer and worth more. Everything from Ch. 8–9 (test statistics, p-values, CIs, t-distributions) is assumed knowledge.
Two-sample tests compare means of two groups (independent samples use a two-sample t-test; matched pairs use a paired t-test on the differences). ANOVA uses an F-test to check whether k group means are all equal, then Tukey’s method isolates which pairs differ. Simple linear regression fits a line ŷ = b₀ + b₁x, with r measuring correlation strength and r² measuring the fraction of variance explained.
Independent samples
Two samples where the selection of individuals in one has no effect on the selection in the other.
Paired data
Each observation in one sample is matched with a specific observation in the other. Analysis works on the differences d = x₁ – x₂ within each pair.
Pooling
Assuming two populations have equal variances and combining their sample variances into a single estimate. Used in the pooled two-sample t-test.
Satterthwaite approximation
A formula for the degrees of freedom in a two-sample t-test when equal variances are not assumed. Produces a non-integer df that is rounded down.
One-way ANOVA
A method for testing whether k population means are all equal using a single factor. The test compares between-group variation to within-group variation.
Factor / Level
The factor is the variable that differentiates the populations. Each distinct value of the factor is a level (group). k is the number of levels.
SSA (SS for factor / between groups)
The sum of squares measuring how far group means are from the overall mean. dfa = k – 1.
SSE (SS for error / within groups)
The sum of squares measuring variability within groups. dfe = n – k.
SST (SS total)
SSA + SSE. dft = n – 1.
F-test statistic
F = MSA / MSE. Large F means group means differ more than expected from random variation alone.
Tukey’s method
A multiple comparison procedure for identifying which pairs of group means differ significantly after ANOVA rejects H₀.
Simple linear regression
Models the relationship between one explanatory variable X and one response variable Y as Y = β₀ + β₁X + ε.
Least squares estimates
b₁ = S_XY / S_XX. b₀ = ȳ – b₁x̄. These minimise the sum of squared residuals.
Residual
eᵢ = yᵢ – ŷᵢ. The difference between what was observed and what the regression line predicts.
Correlation coefficient (r)
r = S_XY / √(S_XX · S_YY). Measures the strength and direction of the linear relationship. Ranges from –1 to 1.
Coefficient of determination (r²)
The fraction of the total variation in Y explained by the linear relationship with X. r² = SSR / SST.
Prediction interval
An interval for a single future observation y* at a given x*. Wider than the confidence interval for the mean response μ* because it includes individual variability.
Two samples are independent when selection in one group has no bearing on the other
E(x̄₁ – x̄₂) = μ₁ – μ₂. Var(x̄₁ – x̄₂) = σ²₁/n₁ + σ²₂/n₂
Two-sample z-test (σ₁, σ₂ known): z = [(x̄₁ – x̄₂) – Δ₀] / √(σ²₁/n₁ + σ²₂/n₂)
Two-sample t-test (σ₁, σ₂ unknown): t = [(x̄₁ – x̄₂) – Δ₀] / √(s²₁/n₁ + s²₂/n₂). Degrees of freedom via Satterthwaite approximation (round down)
CI for μ₁ – μ₂: (x̄₁ – x̄₂) ± t_{α/2, ν} · √(s²₁/n₁ + s²₂/n₂)
If variances are assumed equal, pool them. Otherwise use the Satterthwaite df
The t-procedure is very robust against non-normality
Compute the differences dᵢ = x₁ᵢ – x₂ᵢ within each pair, then run a one-sample t-test on the dᵢ values
df = n – 1 (number of pairs minus 1)
CI: d̄ ± t_{α/2, n–1} · s_D/√n
When to pair: use paired tests when there is high heterogeneity between units but high correlation within pairs. Use unpaired tests when units are relatively homogeneous and within-pair correlation is low
Tests H₀: μ₁ = μ₂ = … = μₖ against Hₐ: at least one pair of means differs
Assumptions: k independent SRSs, normal populations (check with QQ plots), equal variances (check s_max/s_min ≤ 2)
Model: Xᵢⱼ = μᵢ + εᵢⱼ where ε ~ N(0, σ²). DATA = FIT + RESIDUAL
The F-statistic compares between-group variation (MSA) to within-group variation (MSE): F = MSA / MSE
SSA = Σ nᵢ(x̄ᵢ – x̄)², dfa = k – 1
SSE = Σ(nᵢ – 1)s²ᵢ, dfe = n – k
SST = SSA + SSE, dft = n – 1
p-value = P(F ≥ F_ts) from an F distribution with df1 = k – 1 and df2 = n – k
Large F means group means are more spread out than expected from within-group noise. Small F is consistent with H₀
The F-test is always two-tailed in the sense that only large values of F lead to rejection
After ANOVA rejects H₀, use multiple comparisons to find which pairs differ
Tukey’s method: compute t** = Q_{α, k, n–k} / √2. Build CIs for each pair: (x̄ᵢ – x̄ⱼ) ± t** · √(MSE(1/nᵢ + 1/nⱼ))
Number of pairwise comparisons: k(k – 1)/2
A CI that contains 0 means those two means are not significantly different
Display results by ordering the means and drawing a line under groups that are not significantly different
Two variables are associated if knowing the value of one tells you something about the other. Association is not causation
Response variable Y is the outcome. Explanatory variable X explains or predicts Y. Y = g(X)
The regression model: Yᵢ = β₀ + β₁Xᵢ + εᵢ, where ε ~ N(0, σ²)
Assumptions: SRS with independent observations, linear relationship, response is normally distributed around the regression line, constant SD of the response
Least squares estimates: b₁ = S_XY / S_XX. b₀ = ȳ – b₁x̄
Residuals: eᵢ = yᵢ – ŷᵢ. Should look like a horizontal band around 0 with no pattern. s² = MSE
ANOVA decomposition for regression: SST = SSR + SSE. r² = SSR / SST
A horizontal regression line (b₁ = 0) implies no linear association
r = S_XY / √(S_XX · S_YY). Ranges from –1 to 1
r > 0: positive linear association. r < 0: negative. r near 0: weak or no linear association
r² = R² = fraction of variance in Y explained by the linear model
F-test for the regression: F = MSR / MSE, df1 = 1, df2 = n – 2. Tests H₀: no association (i.e. β₁ = 0)
t-test for the slope: SE(b₁) = √(MSE / S_XX). CI for β₁: b₁ ± t_{α/2, n–2} · SE(b₁)
For simple linear regression with one predictor, the F-test and the two-sided t-test for β₁ give the same p-value
Confidence interval for the mean response μ* at x = x*: μ̂* ± t_{α/2, n–2} · SE(μ̂*), where SE(μ̂*) = √[MSE(1/n + (x* – x̄)²/S_XX)]
Prediction interval for a single new observation y* at x = x*: ŷ* ± t_{α/2, n–2} · SE(ŷ*), where SE(ŷ*) = √[MSE(1 + 1/n + (x* – x̄)²/S_XX)]
The prediction interval is always wider than the confidence interval for the mean because it accounts for individual variation on top of estimation uncertainty
Regression diagnostics: check residual plots for patterns, non-constant variance, or outliers. The residuals should show no structure
t_{\text{two-sample}} = \frac{(\bar{x}_1 - \bar{x}_2) - \Delta_0}{\sqrt{\dfrac{s_1^2}{n_1} + \dfrac{s_2^2}{n_2}}}\text{Paired: } t = \frac{\bar{d} - \Delta_0}{s_D / \sqrt{n}}, \quad df = n - 1F = \frac{MSA}{MSE}, \quad MSA = \frac{SSA}{k-1}, \quad MSE = \frac{SSE}{n-k}b_1 = \frac{S_{XY}}{S_{XX}}, \quad b_0 = \bar{y} - b_1\bar{x}r = \frac{S_{XY}}{\sqrt{S_{XX} \cdot S_{YY}}}, \quad r^2 = \frac{SSR}{SST}SE_{\hat{\mu}^*} = \sqrt{MSE\left[\frac{1}{n} + \frac{(x^* - \bar{x})^2}{S_{XX}}\right]}SE_{\hat{y}^*} = \sqrt{MSE\left[1 + \frac{1}{n} + \frac{(x^* - \bar{x})^2}{S_{XX}}\right]}Students often confuse independent and paired tests. If the data are naturally paired (before/after on the same subject, matched siblings, left eye vs. right eye), use a paired test. If two separate groups are sampled independently, use a two-sample test
ANOVA’s F-test tells you whether at least one pair of means differs. It does not tell you which pairs differ. You need a follow-up procedure (Tukey) for that
r = 0 does not mean no relationship. It means no linear relationship. A strong curved relationship can have r near 0
A prediction interval is not the same as a confidence interval for the mean. The prediction interval is wider because it covers where a single new observation might fall, not just the average
Students sometimes compute r² and interpret it as r. An r² of 0.49 means r = 0.7, not 0.49
⚠️ Be able to decide: independent two-sample test vs. paired test. The exam will test this distinction.
⚠️ For ANOVA, know how to fill in an ANOVA table (SSA, SSE, SST, dfa, dfe, dft, MSA, MSE, F).
⚠️ Expect a Tukey follow-up: build CIs for all pairwise differences and determine which contain 0.
⚠️ For regression, be able to compute b₀, b₁, r, and r², and interpret each in context.
⚠️ Know the difference between the confidence interval for the mean response and the prediction interval.
True or False: In ANOVA, a large F-statistic supports the null hypothesis. (False – it suggests at least one mean differs)
Fill in the blank: The degrees of freedom for SSE in one-way ANOVA are ___. (n – k)
True or False: r² = 0.81 means the model explains 81% of the variation in Y. (True)
True or False: A prediction interval is narrower than a confidence interval for the mean. (False – it is wider)
Fill in the blank: The number of pairwise comparisons with k = 4 groups is ___. (6)
Q: Two independent samples: n₁ = 30, x̄₁ = 85, s₁ = 8; n₂ = 25, x̄₂ = 79, s₂ = 10. Test H₀: μ₁ – μ₂ = 0.
A: t = (85 – 79) / √(64/30 + 100/25) = 6 / √(2.133 + 4) = 6 / √6.133 = 6 / 2.477 ≈ 2.42. Use Satterthwaite df. This is significant at α = 0.05.
Q: An ANOVA has SSA = 120, SSE = 480, k = 4, n = 40. Compute F.
A: dfa = 3, dfe = 36. MSA = 120/3 = 40. MSE = 480/36 = 13.33. F = 40/13.33 = 3.0.
Q: A regression gives b₁ = 2.5, S_XX = 100, MSE = 9. Test whether β₁ = 0.
A: SE(b₁) = √(9/100) = 0.3. t = 2.5/0.3 = 8.33. This is highly significant; strong evidence of a linear relationship.
Q: r = –0.85. Interpret this.
A: There is a strong negative linear relationship between X and Y. As X increases, Y tends to decrease. r² = 0.7225, so about 72.25% of the variation in Y is explained by the linear model.
The two-sample t-test (Ch. 10) is the special case of ANOVA (Ch. 11) with k = 2 groups. The F-test in ANOVA is related to the F-test in regression (Ch. 12), both comparing explained variation to unexplained variation. Regression diagnostics (Ch. 12.4) rely on the normality checks from Ch. 6.3 and the residual concepts mirror the DATA = FIT + RESIDUAL idea from ANOVA.
Two-sample t-test, independent samples, paired t-test, matched pairs, pooled variance, Satterthwaite, one-way ANOVA, F-test, F-distribution, factor, level, SSA, SSE, SST, MSA, MSE, ANOVA table, Tukey method, multiple comparisons, simple linear regression, least squares, slope, intercept, residual, correlation, r, r-squared, coefficient of determination, scatterplot, confidence interval for mean response, prediction interval, regression diagnostics, STAT 35000, Purdue statistics