Difficulty: Intermediate | Prerequisites: Part 1 of these notes (SLR model and least squares estimation)
Source: Lecture Notes, Chapter 1 (Part A) – Sharmistha Guha, Texas A&M University
Tags: residuals, predicted values, fitted values, SSE, SSR, SST, R-squared, coefficient of determination, ANOVA decomposition, error variance, sigma-hat, model checking, residual plots, homoskedasticity, normality, STAT 212
Once you have a fitted regression line, you need to assess how well it describes the data. This part of the chapter introduces residuals (the gaps between observed and predicted values), the estimation of error variance σ², and the R² statistic that summarises how much of the variation in Y the linear model explains. These tools are the backbone of every regression diagnostic you will encounter later in the course.
Residuals measure how far each data point falls from the fitted line. Squaring and summing them gives the error sum of squares (SSE), which is used to estimate σ². The total variation in Y decomposes neatly into SSR (explained by the model) plus SSE (unexplained), and R² = SSR/SST tells you the fraction of variation the line captures.
Predicted value (ŷᵢ)
The value of y at xᵢ according to the fitted line: ŷᵢ = β̂₀ + β̂₁xᵢ. In simple terms, it is the model's best guess for the ith observation.
Residual (eᵢ)
The difference between the observed response and its predicted value: eᵢ = yᵢ − ŷᵢ. Think of it as the bit the model failed to explain. Residuals serve as stand-ins for the unobservable true errors εᵢ.
Error sum of squares (SSE)
The sum of the squared residuals: SSE = Σ eᵢ². It measures the total amount of variation in Y that the linear model does not capture.
Error variance estimate (σ̂²)
SSE divided by n − 2. Dividing by n − 2 (rather than n) makes σ̂² an unbiased estimator of σ². The denominator uses n − 2 because two parameters (β₀ and β₁) were estimated from the data. This parallels dividing by n − 1 when estimating a population variance from a sample.
Total sum of squares (SST)
SST = Σ(yᵢ − ȳ)². It measures all the variation in the yᵢ values, regardless of source.
Regression sum of squares (SSR)
SSR = Σ(ŷᵢ − ȳ)². It captures the variation in Y that is systematic and explainable by the linear relationship between Y and x.
Coefficient of determination (R²)
R² = SSR / SST = 1 − SSE / SST. It is the fraction of the total variation in Y that is explained by the straight-line relationship with x. Always between 0 and 1.
After fitting the line ŷ = β̂₀ + β̂₁x, you compute a predicted value for every observation in the dataset: ŷᵢ = β̂₀ + β̂₁xᵢ. The residual for observation i is eᵢ = yᵢ − ŷᵢ.
Residuals are observable (you can compute them from the data), whereas the true errors εᵢ are not. If the model is correct, residuals should behave like a random sample from a normal distribution.
The error variance σ² controls how tightly data cluster around the true regression line. If σ² = 0, the relationship is deterministic and every point sits exactly on the line.
To estimate σ², begin with the natural idea of averaging the squared residuals. However, a straight average (dividing by n) underestimates σ² because the least squares line was chosen specifically to make the residuals small. Correcting for the two estimated parameters gives:
σ̂² = SSE / (n − 2)
This is an unbiased estimator: E(σ̂²) = σ².
A key algebraic identity links the three sums of squares:
SST = SSR + SSE
The proof adds and subtracts ŷᵢ inside the squared term (yᵢ − ȳ)², expands, and shows that the cross-product term Σ(yᵢ − ŷᵢ)(ŷᵢ − ȳ) equals zero. The zero cross-product follows because ŷᵢ − ȳ = β̂₁(xᵢ − x̄), so the cross-product simplifies to β̂₁ Σ(xᵢ − x̄)(yᵢ − ȳ) − β̂₁² Σ(xᵢ − x̄)², which cancels by the definition of β̂₁.
Interpretation:
SSR is the variation in Y that is systematic, explainable by x.
SSE is the variation in Y that is unsystematic, purely random noise unrelated to x.
R² = SSR / SST gives the proportion of the total variability in Y accounted for by the linear model.
R² near 1 means the line fits the data closely; most variation is explained.
R² near 0 means x tells you almost nothing about Y (at least linearly).
For the Galton father–son data, R² = 0.251, meaning about 25% of the variation in sons' heights is explained by the linear relationship with fathers' heights. The remaining 75% comes from genetics beyond the father, environment, nutrition, measurement variability, and so on.
R² is a descriptive measure, not a test. A low R² does not mean the relationship is unimportant, and a high R² does not guarantee the model is correct.
If the model assumptions hold, the residuals e₁, …, eₙ should look like a random sample from a normal distribution with mean zero and constant spread. Three diagnostic plots are standard:
Residuals versus x: checks whether the true regression curve is nonlinear. A systematic curved pattern (e.g. a U-shape) suggests a straight line is insufficient.
Residuals versus ŷ: checks the equal-variance (homoskedasticity) assumption. A "fan" or "trumpet" shape, where the spread of residuals grows with ŷ, signals heteroskedasticity.
Histogram, density plot, or Q–Q plot of residuals: checks the normality assumption. The Q–Q plot should follow the y = x line closely.
fit <- lm(y ~ x)
resid <- fit$residuals
predict <- fit$fitted.values
plot(x, resid, xlab = "x", ylab = "Residuals")
plot(predict, resid, xlab = "yhat", ylab = "Residuals")
hist(resid)
plot(density(resid))
qqnorm(resid)
Quantity | Formula |
|---|---|
Predicted value | ŷᵢ = β̂₀ + β̂₁xᵢ |
Residual | eᵢ = yᵢ − ŷᵢ |
SSE | Σ eᵢ² |
SST | Σ(yᵢ − ȳ)² |
SSR | Σ(ŷᵢ − ȳ)² |
ANOVA identity | SST = SSR + SSE |
Error variance estimate | σ̂² = SSE / (n − 2) |
R² | SSR / SST = 1 − SSE / SST |
R² is routinely reported in published research to communicate how well a model fits. In quality engineering, a high R² between a process setting and product output justifies tighter process control. Residual plots are the standard first step in any regression analysis: before trusting confidence intervals or p-values, an analyst checks residuals for patterns, because violations of the assumptions can invalidate all subsequent inference.
Students often divide SSE by n instead of n − 2. The correction matters, especially with small samples: n − 2 accounts for the two degrees of freedom used in estimating β₀ and β₁.
A high R² does not prove the model is correct. A nonlinear relationship can yield a high R² for a straight line if the curvature is mild, yet the model may still be wrong. Always check residual plots.
R² = 0 does not mean X and Y are unrelated. It only means there is no linear relationship. A perfect parabolic relationship could produce R² close to zero.
Students sometimes confuse residuals (eᵢ, observable) with errors (εᵢ, unobservable). Residuals are estimates of the errors, not the errors themselves.
⚠️ Know the ANOVA decomposition SST = SSR + SSE and be prepared to prove the cross-product term is zero.
⚠️ Be able to explain why σ̂² divides by n − 2 (degrees of freedom lost to estimating β₀ and β₁).
⚠️ Interpret R² in context: "X% of the variation in Y is explained by the linear relationship with x."
⚠️ Know which residual plot checks which assumption: eᵢ vs. xᵢ for linearity, eᵢ vs. ŷᵢ for constant variance, Q–Q plot for normality.
True or False: SSE + SSR = SST.
Fill in the blank: σ̂² = SSE / ________.
True or False: If R² = 0.80, then 80% of the variation in x is explained by y.
Fill in the blank: A "fan shape" in the plot of residuals versus fitted values suggests a violation of the ________ assumption.
True or False: Residuals and true error terms are the same thing.
Answers: 1. True. 2. n − 2. 3. False (80% of the variation in Y is explained by the linear relationship with x, not the other way around). 4. Homoskedasticity (equal/constant variance). 5. False (residuals are observable proxies for the unobservable errors).
Q: State the ANOVA decomposition for simple linear regression and define each term.
A: SST = SSR + SSE. SST = Σ(yᵢ − ȳ)² is the total variation in the response. SSR = Σ(ŷᵢ − ȳ)² is the variation explained by the linear model. SSE = Σeᵢ² is the unexplained (residual) variation.
Q: Why do we divide SSE by n − 2, not n, when estimating σ²?
A: Dividing by n − 2 corrects for the two degrees of freedom consumed by estimating β₀ and β₁. This makes σ̂² an unbiased estimator of σ², analogous to dividing by n − 1 when estimating a population variance from a single sample.
Q: For the Galton data, R² = 0.251. Interpret this value.
A: About 25.1% of the total variation in sons' heights is accounted for by the linear relationship with fathers' heights. The remaining roughly 75% is due to other factors (genetics beyond the father, environment, etc.) and random variability.
Q: Describe what you would look for in a plot of residuals versus fitted values and what a problem looks like.
A: You want a shapeless cloud of points scattered randomly around the horizontal line at zero, with roughly constant vertical spread. If the spread increases (or decreases) systematically as ŷ grows, that indicates heteroskedasticity, meaning the equal-variance assumption is violated.
The ANOVA decomposition introduced here generalises to multiple regression (Chapter 2+), where SSR has contributions from several predictors. The F-test for overall model significance is built directly on the SSR/SSE ratio. Residual diagnostics also carry over unchanged: checking linearity, constant variance, and normality is the same procedure regardless of how many predictors the model contains.
residuals, predicted values, fitted values, error sum of squares, SSE, regression sum of squares, SSR, total sum of squares, SST, ANOVA decomposition, R-squared, coefficient of determination, error variance, sigma-hat squared, model diagnostics, residual plots, Q-Q plot, homoskedasticity, heteroskedasticity, normality check, goodness of fit, STAT 212