Source: Principles of Statistics I (Texas A&M), Tamhane/Dunlop Ch. 10–11
Tags: ANOVA for regression, R-squared, coefficient of determination, SSReg, SSE, SSTOT, F-test, residual analysis, normal probability plot, residual plot, confidence interval for slope, prediction interval, confidence interval for mean response, lack of fit test, SSPE, SSLOF, hypothesis testing regression
Once a regression line is fitted, you need to check whether the model conditions hold (via residual plots and normality checks) and then quantify how well the line explains the data. R² measures the proportion of variance explained, the ANOVA F-test checks whether the slope is significantly different from zero, and confidence/prediction intervals let you make inferences about mean responses and individual future observations. A formal lack-of-fit test can reveal whether a straight line is even the right shape.
Residual plot
A scatterplot of residuals (eᵢ) against fitted values (ŷᵢ). Used to check constant variance and linearity. Ideally shows a random horizontal band centred at zero.
Normal probability plot (of residuals)
A plot that assesses whether residuals are approximately normally distributed. Points falling close to a straight line indicate normality is reasonable.
SSTOT (total sum of squares)
SSTOT = SSyy = Σ(yᵢ − ȳ)². The total variability in the response variable before fitting any model.
SSReg (regression sum of squares, model sum of squares)
SSReg = Σ(ŷᵢ − ȳ)² = (SSxy)² / SSxx. The variability in Y that the regression line explains.
SSE (error sum of squares, residual sum of squares)
SSE = Σ(yᵢ − ŷᵢ)² = SSTOT − SSReg. The leftover variability the model does not explain.
Coefficient of determination (R²)
R² = SSReg / SSTOT = 1 − SSE / SSTOT. The proportion of total variation in Y explained by the linear relationship with X. Ranges from 0 to 1.
F-test for regression
Tests H₀: β₁ = 0 vs Hₐ: β₁ ≠ 0 using F = MSReg / MSE. Equivalent to the two-sided t-test on β₁.
Confidence interval for the mean response
An interval estimating μ(y|x₀), the average Y at a particular X = x₀. Narrower than a prediction interval because it targets the mean, not a single observation.
Prediction interval
An interval estimating an individual future Y at X = x₀. Wider than the confidence interval because it includes both estimation uncertainty and the natural variation of individual observations.
Lack-of-fit test
A formal F-test comparing the residual error from the straight-line model to the pure error estimated from replicate observations. A significant result means the straight-line shape is inadequate.
Pure error (SSPE)
SSPE = ΣΣ(yᵢⱼ − ȳᵢ)². The variation within groups of replicated X values. This is the irreducible noise, unrelated to model form.
Lack-of-fit sum of squares (SSLOF)
SSLOF = SSE − SSPE. The extra error attributable to the model having the wrong functional form.
After fitting the line, you check conditions C1–C4 using residuals eᵢ = yᵢ − ŷᵢ.
When the model is correct, residuals are approximately normally distributed with E[eᵢ] = 0 and approximately constant variance (exactly constant for large n).
C1, normality: construct a normal probability plot of the residuals. If the points track a straight line and a normality test (such as Ryan-Joiner) gives a large p-value, normality is not rejected. In the dose-strength example, the p-value was 0.169, so normality holds.
C2, constant variance: plot residuals against fitted values. The vertical spread should stay roughly the same across the horizontal axis. A funnel or fan shape signals non-constant variance.
C3, linearity of the mean: also assessed from the residual-vs-fitted plot. If the straight-line model is appropriate, the residuals should scatter randomly around zero with no discernible curve. A systematic pattern (e.g. a U-shape) suggests a non-linear relationship may be needed.
The total variability in Y decomposes into two pieces:
SSTOT = SSReg + SSE
The ANOVA table for SLR:
Source | df | SS | MS | F |
|---|---|---|---|---|
Model (Regression) | 1 | SSReg | SSReg / 1 | MSReg / MSE |
Error (Residual) | n − 2 | SSE | SSE / (n − 2) | |
Total | n − 1 | SSTOT |
The F-statistic tests H₀: β₁ = 0 (no linear relationship) vs Hₐ: β₁ ≠ 0.
For the dose-strength data: SSReg = 590.92, SSE = 173.48, F = 590.92 / 13.34 = 44.28, p-value = 0.000016. Very strong evidence of a linear relationship.
R² = SSReg / SSTOT = 1 − SSE / SSTOT
For the dose-strength data: R² = 1 − 173.48 / 764.40 = 0.773. The straight line explains 77.3% of the variation in strength.
Boundary values:
R² = 0: the model explains nothing. SSE = SSTOT, and β₁ = 0
R² = 1: the model explains everything. SSE = 0, and all points lie exactly on the line
What R² does not tell you:
It does not indicate whether the slope is positive or negative (both can produce the same R²)
It does not confirm a straight line is the best model. You can have R² = 0 when a strong non-linear relationship exists, or R² close to 1 when a curve would fit even better
The remaining 22.7% unexplained variation could come from a more complex model being needed, other variables affecting strength (laboratory conditions, virus load, measurement error), or both.
A 100(1 − α)% confidence interval:
β̂₁ ± t(α/2, n−2) · σ̂ / √SSxx
For the dose-strength data at 95%: 0.575 ± 2.160 · √13.34 / √1785.6 = 0.575 ± 0.187, giving (0.45, 0.70).
β̂₀ ± t(α/2, n−2) · σ̂ · √(Σxᵢ² / (n · SSxx))
For the dose-strength data at 95%: 8.67 ± 3.08, giving (5.59, 11.75).
To test H₀: β₁ ≤ C vs Hₐ: β₁ > C:
T.S. = (β̂₁ − C) / (σ̂ / √SSxx)
Reject H₀ if T.S. ≥ t(α, n−2).
Example: testing H₀: β₁ ≤ 0.5 vs Hₐ: β₁ > 0.5 gives T.S. = (0.575 − 0.5) / (√13.34 / √1785.6) = 0.87. Since 0.87 < t(0.05, 13) = 1.771, we fail to reject. There is insufficient evidence the slope exceeds 0.5.
The two-sided test H₀: β₁ = 0 vs Hₐ: β₁ ≠ 0 using the t-statistic (with C = 0) is equivalent to the F-test from the ANOVA table, since (T.S.)² = F.
Point estimate: μ̂(y|x₀) = β̂₀ + β̂₁x₀
Interval:
μ̂(y|x₀) ± t(α/2, n−2) · σ̂ · √(1/n + (x₀ − x̄)² / SSxx)
Example at x₀ = 5: μ̂ = 8.67 + 0.575(5) = 11.545. The 95% CI is 11.545 ± 2.461, giving (9.08, 14.01).
Point estimate: ŷ(x₀) = β̂₀ + β̂₁x₀ (same point estimate as above)
Interval:
ŷ(x₀) ± t(α/2, n−2) · σ̂ · √(1 + 1/n + (x₀ − x̄)² / SSxx)
Note the extra "1 +" under the square root compared to the confidence interval.
Example at x₀ = 5: 11.545 ± 8.264, giving (3.28, 19.81).
The prediction interval is much wider because it accounts for the natural scatter of individual observations, not just uncertainty in the mean.
Both intervals are narrowest when x₀ = x̄ (the centre of the data) and widen as you move away from x̄ in either direction.
To test H₀: μ(y|x₀) ≤ C vs Hₐ: μ(y|x₀) > C:
T.S. = (μ̂(y|x₀) − C) / (σ̂ · √(1/n + (x₀ − x̄)² / SSxx))
Example: testing whether mean strength > 15 when dose = 20. μ̂ = 8.67 + 0.575(20) = 20.17. T.S. = (20.17 − 15) / (√13.34 · √(1/15 + (20 − 12.4)² / 1785.6)) = 4.498 > 1.771. Reject H₀ with p-value = 0.0003.
When you have replicate observations at the same X values, you can split SSE into two parts:
SSPE (pure error): variation within groups sharing the same X
SSLOF (lack of fit): SSE − SSPE, the extra error from the model having the wrong shape
F = (SSLOF / (m − 2)) / (SSPE / (n − m))
where m = number of distinct X values.
For the dose-strength data: SSE = 173.45, SSPE = 43.3, SSLOF = 130.15. F = (130.15 / 3) / (43.3 / 10) = 10.02, p-value = 0.0023.
This strongly rejects the straight-line model. Fitting a cubic polynomial (ŷ = −1.175 + 3.820x − 0.220x² + 0.00395x³) improved R² from 0.773 to 0.935, with all polynomial terms significant.
Quantity | Formula |
|---|---|
SSTOT | SSyy = Σ(yᵢ − ȳ)² |
SSReg | (SSxy)² / SSxx |
SSE | SSTOT − SSReg |
R² | SSReg / SSTOT = 1 − SSE / SSTOT |
F-statistic | (SSReg / 1) / (SSE / (n − 2)) |
CI for β₁ | β̂₁ ± t(α/2, n−2) · σ̂ / √SSxx |
CI for mean response | μ̂ ± t · σ̂ · √(1/n + (x₀ − x̄)² / SSxx) |
PI for individual Y | ŷ ± t · σ̂ · √(1 + 1/n + (x₀ − x̄)² / SSxx) |
Lack-of-fit F | (SSLOF / (m − 2)) / (SSPE / (n − m)) |
⚠️ The prediction interval formula has "1 +" under the root that the confidence interval does not. This is the single most common mix-up on exams.
⚠️ Both intervals are narrowest at x₀ = x̄. The further you move from the mean of X, the wider they get.
⚠️ R² = 0.773 does not mean the straight-line model is appropriate. The lack-of-fit test can reject the linear model even when R² is reasonably high.
⚠️ The F-test from the ANOVA table and the two-sided t-test for β₁ = 0 are equivalent: F = t².
⚠️ The lack-of-fit test requires replicate observations at the same X values. Without replicates, you cannot separate pure error from lack of fit.
⚠️ When R² is close to 1, do not automatically conclude the straight line is the correct model. A non-linear pattern can still produce a high R².
Q: What is the difference between a confidence interval and a prediction interval at the same x₀?
A: The confidence interval estimates the mean of Y at x₀ (a population parameter). The prediction interval estimates a single future observation of Y at x₀. The prediction interval is always wider because it includes individual-observation variability on top of estimation uncertainty.
Q: In the dose-strength example, what percentage of variation in strength is unexplained by the linear model?
A: 22.7%. R² = 77.3%, so 100% − 77.3% = 22.7% is unexplained.
Q: If the F-test in the ANOVA table yields a p-value of 0.000016, what do you conclude?
A: There is very strong evidence that β₁ ≠ 0, meaning a statistically significant linear relationship exists between dose level and strength.
Q: The lack-of-fit test for the dose-strength data gives F = 10.02 with p-value = 0.0023. What does this tell you?
A: The straight-line model is not an adequate fit. The relationship between dose and strength is likely non-linear. A polynomial or other curved model should be considered.
Q: At what value of x₀ is the confidence interval for the mean response narrowest, and why?
A: At x₀ = x̄. The term (x₀ − x̄)² / SSxx equals zero there, minimising the standard error. This is the centre of the observed X range, where we have the most information.
ANOVA table for regression, regression sum of squares, residual sum of squares, total sum of squares, SSReg, SSE, SSTOT, MSE, mean square error, F-test regression, R-squared, coefficient of determination, proportion of variance explained, residual analysis, residual plot, residuals versus fitted, normal probability plot of residuals, confidence interval for slope, confidence interval for intercept, confidence interval for mean response, prediction interval, lack of fit test, pure error, SSPE, SSLOF, polynomial regression, cubic fit, model adequacy, hypothesis test for beta-one