Source: Principles of Statistics I (Texas A&M), Tamhane/Dunlop Ch. 2 & 4
Tags: Pearson correlation, Spearman rank correlation, autocorrelation, autocorrelation function, ACF, sample correlation coefficient, lag, time series, AR(1) model, monotone relationship, influential observations, stationarity
The Pearson correlation coefficient measures linear association between two variables, while the Spearman rank correlation captures monotone (not necessarily linear) relationships and is resistant to influential observations. For data collected over time, the autocorrelation function measures how strongly observations at different lags are related, which matters for determining whether your data is independent or correlated.
Pearson correlation coefficient (population), ρ
ρ = E[(Y − μ_Y)(W − μ_W)] / (σ_Y × σ_W). Measures the linear relationship between two random variables. Ranges from −1 to +1.
Sample Pearson correlation, r
r = Σ(Y_i − Ȳ)(W_i − W̄) / [√Σ(Y_i − Ȳ)² × √Σ(W_i − W̄)²]. Computed from n independent pairs (Y_i, W_i).
Spearman rank correlation, r_sp
The Pearson correlation applied to the ranks of the data rather than the raw values. Measures monotone association. More robust to outliers and nonlinearity than Pearson.
Autocorrelation of order k, ρ_k
ρ_k = E[(X_t − μ)(X_{t−k} − μ)] / σ². Measures the linear relationship between observations separated by k time steps in a stationary series.
Sample autocorrelation of lag k, ρ̂_k
ρ̂_k = [Σ_{t=1}^{n−k} (X_t − X̄)(X_{t+k} − X̄)] / [Σ_{t=1}^{n} (X_t − X̄)²].
AR(1) model (first-order autoregressive)
X_t = μ + ρ × X_{t−1} + e_t, where the e_t are iid with mean 0 and variance σ². Under this model, ρ_k = ρ^k, so autocorrelations decay geometrically.
Stationarity
A time series property where the mean and variance remain constant over time, and the autocorrelation between observations depends only on the lag (distance in time), not on when the observations were taken.
Influential observation
A data point that has a disproportionate effect on a statistical measure (e.g. an extreme x-value that inflates the Pearson correlation).
Monotone relationship
A relationship between X and Y where Y consistently increases (or consistently decreases) as X increases, though not necessarily at a constant rate. Captured by Spearman but potentially missed by Pearson.
The sample Pearson correlation for n pairs (Y_i, W_i):
r = [Σ(Y_i − Ȳ)(W_i − W̄)] / [√Σ(Y_i − Ȳ)² × √Σ(W_i − W̄)²]
Properties:
Unit-free measure of linear association
−1 ≤ r ≤ 1
r = ±1 implies an exact linear relationship: Y_i = β₀ + β₁ × W_i, with sign of β₁ matching sign of r
r = 0 does not mean "no relationship," only "no linear relationship." A strong curved or nonlinear relationship can exist with r near zero.
Pearson r only considers the relationship between two variables at a time; it cannot detect multivariate relationships
The epilepsy data example in the handout shows a correlation matrix for seizure counts (Y1 through Y4), baseline count, and age. Correlations among the four seizure count periods range from 0.738 to 0.895, indicating strong positive linear association. Age has near-zero correlation with most other variables, suggesting it is not linearly related to seizure counts.
When the relationship between X and Y is monotone but not linear, Pearson r can understate the strength of the association. When there are extreme values in the x-direction (influential points), Pearson r can be inflated.
The Spearman correlation addresses both problems by working with ranks instead of raw values.
Computation:
Rank the X values among themselves (R_i = rank of X_i)
Rank the Y values among themselves (S_i = rank of Y_i)
Compute the Pearson correlation of the (R_i, S_i) pairs
If there are no ties:
r_sp = 1 − [6 × Σ d_i²] / [n(n² − 1)], where d_i = R_i − S_i
Two key examples from the handout:
Nonlinear monotone relationship (Figure 1): Pearson r = 0.79, Spearman r_sp = 0.94. Pearson understates the strength because the relationship is curved. Spearman correctly identifies the strong monotone pattern.
One influential x-value (Figure 2): Pearson r = 0.82, Spearman r_sp = 0.29. A single extreme observation in the x-direction inflates the Pearson correlation. Spearman, by working with ranks, is not distorted and correctly reflects the absence of association in the bulk of the data.
When to use which:
Pearson: when the relationship is expected to be linear and the data has no extreme outliers
Spearman: when the relationship may be monotone but nonlinear, or when outliers or influential observations are a concern
When data is collected over time (or space), successive observations may be correlated. The autocorrelation function quantifies this.
Population autocorrelation:
ρ_k = E[(X_t − μ)(X_{t−k} − μ)] / σ²
This measures the degree of linear association between observations k steps apart in a stationary series.
Sample autocorrelation:
ρ̂_k = [Σ_{t=1}^{n−k} (X_t − X̄)(X_{t+k} − X̄)] / [Σ_{t=1}^{n} (X_t − X̄)²]
Always: ρ̂₀ = 1 (every series is perfectly correlated with itself at lag 0).
A simple and widely used model for time-correlated data:
X_t = μ + ρ × X_{t−1} + e_t
where the e_t are iid with mean 0 and variance σ², independent of the X_t values, and |ρ| < 1.
Under this model, the autocorrelation at lag k is: ρ_k = ρ^k
This means autocorrelations decay geometrically. If ρ₁ is large and positive (say 0.9), adjacent observations are strongly correlated, and the correlation fades slowly.
The Stamford and Yonkers ozone time series (daily maximum ozone concentrations, May through September 1974) appear relatively stationary, with no apparent trend over the five-month period.
Key autocorrelation results:
Lag | Yonkers ρ̂_k | Stamford ρ̂_k |
|---|---|---|
1 | 0.434 | 0.334 |
2 | 0.135 | 0.136 |
3 | 0.081 | 0.077 |
4 | 0.183 | 0.087 |
5 | 0.062 | 0.030 |
Both sites show moderate positive lag-1 autocorrelation (adjacent days tend to have similar ozone levels), with rapid decay thereafter.
The monthly average AAA bond yields (1989-1993, n = 60) display a clear downward trend. The raw autocorrelations are very high (ρ̂₁ = 0.939) and decay slowly, which is characteristic of non-stationary data with a trend.
After fitting and removing a cubic trend:
Z_i = Y_i − (9.467 − 0.01388 × M_i + 0.00005455 × M_i² − 0.00001018 × M_i³)
the detrended series still shows strong lag-1 autocorrelation (ρ̂₁ = 0.792), but the pattern changes. The autocorrelations eventually become negative (around lag 7-11), suggesting a quasi-periodic behaviour in the residuals.
The key lesson: when a trend is present, you must remove it before interpreting the autocorrelation structure. Failing to detrend produces misleadingly high autocorrelations.
Pearson sample correlation:
r = Σ(Y_i − Ȳ)(W_i − W̄) / [√Σ(Y_i − Ȳ)² × √Σ(W_i − W̄)²]
Spearman rank correlation (no ties):
r_sp = 1 − [6 × Σ d_i²] / [n(n² − 1)], where d_i = R_i − S_i
Sample autocorrelation at lag k:
ρ̂_k = [Σ_{t=1}^{n−k} (X_t − X̄)(X_{t+k} − X̄)] / [Σ_{t=1}^{n} (X_t − X̄)²]
AR(1) model:
X_t = μ + ρ × X_{t−1} + e_t, with ρ_k = ρ^k
⚠️ r = 0 does not mean "no relationship." It means "no linear relationship." Always consider scatter plots.
⚠️ Pearson r measures only pairwise linear relationships between two variables. It cannot detect multivariate or higher-order associations.
⚠️ A single extreme observation can inflate Pearson r dramatically (Figure 2 example: r = 0.82 vs r_sp = 0.29).
⚠️ For monotone but nonlinear relationships, Spearman r_sp is more appropriate than Pearson r.
⚠️ In time series data with a trend, raw autocorrelations will be misleadingly high. Always detrend first.
⚠️ Under an AR(1) model, ρ_k = ρ₁^k. If you see autocorrelations that decay geometrically, this is the model to suspect.
⚠️ The R function acf(y) plots the sample autocorrelation function with dashed lines indicating approximate 95% significance bounds at ±2/√n.
⚠️ Tied observations in rank-based methods: when there are ties in X or Y values, average ranks are assigned. The simplified Spearman formula (using d_i²) works only when all ranks are distinct integers.
Q: What does Pearson r = 0 tell you about the relationship between two variables?
A: It tells you there is no linear relationship. There may still be a strong nonlinear or curved relationship.
Q: In the handout's Figure 1, why is Spearman r_sp = 0.94 much higher than Pearson r = 0.79?
A: The relationship between X and Y is monotone (consistently increasing) but nonlinear (curved). Pearson only captures the linear component, so it understates the association. Spearman works with ranks and captures any monotone pattern.
Q: In Figure 2, one extreme x-value produces Pearson r = 0.82 despite no real relationship. How does Spearman handle this?
A: By converting to ranks, the extreme x-value becomes just the highest rank, which has no more influence than any other rank. Spearman correctly gives r_sp = 0.29, reflecting the lack of association in the bulk of the data.
Q: What is the interpretation of ρ̂₁ = 0.434 for the Yonkers ozone data?
A: Adjacent days have a moderate positive linear association in ozone concentration. If today has high ozone, tomorrow is somewhat likely to also have high ozone.
Q: Why did the AAA bond yield data show ρ̂₁ = 0.939 before detrending?
A: The strong downward trend over time creates an artificial correlation: successive months have similar values because they are both on the same declining trend. After removing the trend, ρ̂₁ drops to 0.792, which reflects the genuine short-term autocorrelation in the detrended residuals.
Q: Under an AR(1) model with ρ₁ = 0.8, what is ρ₃?
A: ρ₃ = ρ₁³ = 0.8³ = 0.512.
Pearson correlation coefficient, Spearman rank correlation, sample correlation, autocorrelation, autocorrelation function, ACF, lag, time series, AR(1), autoregressive model, stationarity, detrending, influential observation, monotone relationship, linear association, rank correlation, ozone concentration data, AAA bond yields, scatter plot, draftsman plot, matrix plot, STAT principles of statistics, Texas A&M