Difficulty: Intermediate | Prerequisites: Chapter 13a (regression line, ANOVA table), Chapter 8 (normality tests, histograms, normal probability plots), Part 1 of these notes (correlation).
Before you run any inference on a regression (hypothesis tests, confidence intervals), you need to verify that the assumptions behind linear regression hold. This section teaches you which graphs check which assumptions, what violations look like, and how to spot outliers and influential points. If you have been checking normality with histograms and QQ plots all semester, most of this will feel familiar; the new wrinkle is that regression adds linearity and constant variance to the checklist, and those are checked with different plots.
Linear regression assumes SRS, linearity, constant standard deviation of residuals, and normality of residuals. Use the scatterplot and residual plot to check linearity and constant variance. Use the histogram and normal probability plot of the residuals to check normality. A residual plot is a transformed scatterplot that makes patterns easier to see. Always plot first; the numbers alone will not tell you whether the model is appropriate.
Residual
The difference between the observed y value and the predicted y-hat value for a given x. Formally: e_i = y_i - y-hat_i.
In simple terms, it is how far a data point sits above or below the regression line.
Residual plot
A scatterplot of the residuals (vertical axis) against the explanatory variable x (horizontal axis). The regression line maps to a horizontal line at zero.
Think of it as the scatterplot "flattened out" so you can see patterns more clearly.
Constant variance (homoscedasticity)
The assumption that the spread of the residuals stays the same across the entire range of x. If it changes, the variance is non-constant (heteroscedastic).
Megaphone shape
The most common pattern of non-constant variance: the residuals fan out as x increases, resembling a megaphone or trumpet. Often appears when the explanatory variable is age or time.
Y-outlier
A point with an unusually large residual, far above or below the regression line. On a residual plot, the zero line will appear off-centre because the software rescales to fit the extreme point.
X-outlier (leverage point)
A point whose x value is far from the rest of the data. It may not have a large residual, but it can pull the regression line toward it.
Influential point
An observation that, if removed, would substantially change the fitted regression line. X-outliers are the main candidates for being influential.
Normal probability plot (QQ plot)
A graph that plots the observed residuals against the quantiles expected from a normal distribution. If the residuals are approximately normal, the points fall close to a straight diagonal line.
SRS and independence. The pairs (x_i, y_i) are a simple random sample, and observations are independent of each other. There is no graph to check this; you rely on study design.
Linearity. The relationship between x and y in the population is a straight line.
Constant standard deviation of residuals. The spread of y around the line is the same for all values of x. This is also called homoscedasticity.
Normality of residuals. The residuals (and therefore the response variable around the line) follow a normal distribution.
The scatterplot (with the regression line overlaid) lets you check two assumptions: linearity and constant standard deviation.
Linearity: does the line pass through the data? Are points distributed evenly above and below?
Constant variance: slide your fingers (or imagine a fixed-width band) along the line. Does the vertical spread stay roughly the same?
You cannot determine normality from the scatterplot. That requires different plots.
A residual plot graphs e_i (residual) on the y-axis against x on the x-axis. The regression line is now a horizontal line at zero.
Two advantages over the scatterplot:
Comparing points against a horizontal reference is easier than against a slanted line.
The vertical scale is expanded because you no longer need room for the full range of y.
The residual plot checks the same two assumptions (linearity and constant variance) and can also reveal outliers more clearly.
What "no violation" looks like: points scatter randomly above and below the zero line with roughly constant spread. No pattern.
Non-constant variance (megaphone): the residuals fan out as x increases, wider on one side than the other. Common when x is age or time.
Non-linearity (parabola or curve): the residuals form a U-shape or some other systematic pattern rather than scattering randomly. If there is any pattern at all, linearity is violated. Note that a curved residual plot can still have constant variance.
Y-outlier: look for the zero line being off-centre. Software auto-scales the plot to fit all points. If zero is pushed away from the middle, a point with a very large residual is pulling the axis.
X-outlier (possible influential point): look for a point far to the right or left with few neighbours. These points often do not have large residuals (they sit near the line) because they pulled the line toward themselves. That is precisely what makes them dangerous: they change the equation of the line. Check whether removing the point would substantially alter the slope.
Y-outliers matter mainly if they affect normality. X-outliers matter because they may be influential and distort the entire model.
Normality of residuals is checked with the same tools used all semester:
Histogram of residuals with a normal curve overlay. Look for approximate bell shape.
Normal probability (QQ) plot of residuals. If the points follow the diagonal line closely, normality is reasonable.
These are different plots from the scatterplot and residual plot. Do not confuse them.
Assumption | Diagnostic plots |
|---|---|
SRS / independence | None (study design) |
Linearity | Scatterplot, residual plot |
Constant standard deviation | Scatterplot, residual plot |
Normality of residuals | Histogram of residuals, normal probability plot of residuals |
If an assumption fails on either the scatterplot or the residual plot, it fails for the data. You should cite both plots when asked which diagnostics are used.
In quality control, residual plots are used to check whether a prediction model for product strength (say, tensile strength vs. temperature) is valid before it gets built into production monitoring. If the residual plot shows a megaphone, the model's predictions become unreliable at the extremes, exactly where you need them most.
Students often say "the normal probability plot shows a line, therefore the data is linear." The QQ plot checks normality of residuals, not linearity of the x-y relationship. These are completely different assumptions checked by completely different plots.
Students sometimes claim "there are no outliers on the scatterplot, so the residuals are normal." Outliers and normality are separate issues. A dataset can have no outliers and still have non-normal residuals (e.g., uniformly distributed).
Students frequently forget to mention the SRS assumption. Even though no graph checks it, you still need to state it when listing the assumptions.
Students sometimes check only one of the two plots (scatterplot or residual plot) and conclude an assumption holds. If either plot shows a violation, the assumption fails. When asked what plots are used for linearity or constant variance, the answer is both.
Expect a matching question: "which plots check which assumptions?" The summary table is worth memorising.
The distinction between what the scatterplot/residual plot checks (linearity, constant variance) and what the histogram/QQ plot checks (normality) is a high-frequency exam item.
You may be shown a residual plot and asked to identify the violation: random scatter = no violation, megaphone = non-constant variance, parabola = non-linearity, off-centre zero line = y-outlier, isolated far point = possible influential x-outlier.
Graphical assessment is always a judgment call, not a definitive yes-or-no answer. State what you see and draw a conclusion.
True or False: You can determine normality from the scatterplot. ___
Fill in the blank: A residual plot that fans out in a widening pattern is called a ___ shape.
True or False: The residual plot provides a definitive yes-or-no answer about whether assumptions hold. ___
Name the two advantages of a residual plot over a scatterplot for diagnostics.
True or False: If an assumption passes on the scatterplot but fails on the residual plot, it still counts as valid. ___
Answers: 1. False (normality requires histogram and QQ plot of residuals). 2. Megaphone. 3. False (it is a judgment call). 4. Easier to compare against a horizontal line; larger scale. 5. False (if either plot shows a violation, the assumption fails).
Q: A residual plot shows residuals scattered randomly above and below zero with no visible pattern, but the spread narrows noticeably at higher x values. Which assumption is violated?
A: Constant standard deviation (constant variance). The narrowing spread means the standard deviation of the residuals is not the same across all x values.
Q: You examine a residual plot and see a clear U-shaped pattern. The variance appears constant. What is the problem?
A: The linearity assumption is violated. The U-shape (parabola) indicates the true relationship is curved, not linear. The data needs a different model (perhaps a quadratic term or a transformation).
Q: On a residual plot, the zero line sits near the top of the graph, and one point is far below. What does this indicate?
A: There is a y-outlier. The software rescaled the plot to accommodate the extreme negative residual, pushing zero toward the top.
Q: A point sits far to the right of all other data on the residual plot, but its residual is near zero. Should you be concerned?
A: Yes. This is an x-outlier and a potential influential point. Even though its residual is small, it may be pulling the regression line toward itself. Removing it could change the slope substantially.
Q: Which plots would you use to check normality of the residuals? Can you use the scatterplot?
A: Histogram of residuals and the normal probability (QQ) plot of residuals. The scatterplot cannot be used to check normality.
This connects directly to Chapter 8, where you first learned to check normality with histograms and QQ plots. The same tools apply here, but now to the residuals rather than to raw data. It also builds toward Part 3 of these notes (Inference), because every confidence interval and hypothesis test in Sections 13b.4 and 13b.5 requires that these assumptions hold. If the diagnostics fail, the inference results are not trustworthy.
regression diagnostics, residual plot, scatterplot, assumptions for linear regression, linearity assumption, constant variance, homoscedasticity, heteroscedasticity, megaphone pattern, normality of residuals, normal probability plot, QQ plot, histogram of residuals, y-outlier, x-outlier, influential point, leverage point, Anscombe's quartet, model checking, residual analysis