Source: Principles of Statistics I, Texas A&M / Tamhane-Dunlop Ch. 15.1
Tags: Box-Cox transformation, power transformation, normality, skewed data, log transformation, square root transformation, MLE, likelihood profile, confidence interval for theta, Yeo-Johnson
When data are positive but skewed, many standard statistical procedures (which assume normality) break down. The Box-Cox transformation applies a power function y^(θ) that can pull skewed data toward a normal shape. The optimal θ is found by maximum likelihood, and a confidence interval tells you which simple powers (log, square root, reciprocal, etc.) are defensible. When comparing groups, use the same θ for all groups so the transformed measurements remain on a common scale.
Box-Cox transformation
A family of power transformations for positive data: y^(θ) = (y^θ − 1)/θ when θ ≠ 0, and y^(θ) = log(y) when θ = 0. Designed to make the transformed data approximately normal.
Power parameter (θ)
Controls the shape of the transformation. Common special cases: θ = 1 (no transformation), θ = 0.5 (square root), θ = 0 (log), θ = −1 (reciprocal).
Profile log-likelihood, l*_max(θ)
The maximised log-likelihood as a function of θ alone (μ and σ have been profiled out). Plotting this against θ reveals the optimal θ and its uncertainty.
Approximate confidence interval for θ
All θ values satisfying l_max(θ-hat) − l_max(θ) ≤ ½ χ²(1−α), where χ²(1−α) is the upper percentile of the chi-square distribution with df = 1. Graphically, draw a horizontal line at l_max(θ-hat) − ½ χ²(1−α) and read off where it intersects the profile curve.
Yeo-Johnson transformation
A modification of Box-Cox that handles data with both positive and negative values (Yeo and Johnson, 2000).
Many statistical methods (t-tests, ANOVA, regression) assume the data come from a normal distribution.
Real data are often right-skewed, especially measurements like concentrations, times, and counts.
A well-chosen power transformation can make skewed data approximately symmetric and normal, allowing standard methods to be applied validly.
For positive y and θ ≠ 0:
y^(θ) = (y^θ − 1) / θ
For θ = 0:
y^(θ) = log(y)
The division by θ and subtraction of 1 make the family continuous in θ, so that as θ approaches 0 the transformation smoothly becomes the log.
Requirement: all data values must be strictly positive. For data that include zero or negative values, use the Yeo-Johnson extension.
Method 1: Visual (trial and error)
Pick several candidate values of θ (e.g. −1, −0.5, 0, 0.5, 1).
For each, compute the transformed data and make a normal Q-Q plot.
Select the θ whose plot is closest to a straight line.
Method 2: Maximum likelihood
Assume y^(θ)_1, ..., y^(θ)_n are iid N(μ, σ²).
The log-likelihood of the original data y_1, ..., y_n is:
l(μ, σ², θ) = (θ−1) Σ log(y_i) − (n/2) log(2πσ²) − (1/2σ²) Σ (y^(θ)_i − μ)²
For each fixed θ, the MLEs of μ and σ² are just the sample mean and variance of the transformed values:
μ-hat = y-bar^(θ), σ-hat²(θ) = (1/n) Σ (y^(θ)_i − y-bar^(θ))²
The profile log-likelihood becomes:
l*_max(θ) = (θ−1) Σ log(y_i) − (n/2)[log(2π σ-hat²(θ)) + 1]
Maximise l*_max(θ) over θ. In practice, evaluate on a fine grid (e.g. θ from −3 to 3 in steps of 0.001) and pick the peak.
The MLE θ-hat is usually not a clean fraction. Standard practice is to round to a convenient value such as −2, −1, −0.5, 0, 1/3, 0.5, 1, 1.5, 2, provided that value falls within the confidence interval.
Conventional favourites: θ = 0 (log), θ = 0.5 (square root), θ = −1 (reciprocal).
An approximate 100(1−α)% CI consists of all θ where:
l_max(θ-hat) − l_max(θ) ≤ ½ χ²_{1}(1−α)
For a 95% CI: ½ χ²_{1}(0.95) = ½ × 3.841 = 1.92.
On the profile likelihood plot, draw a horizontal line 1.92 units below the peak. The θ values where the curve crosses that line are the CI endpoints.
Stamford ozone
MLE: θ-hat = 0.23, 95% CI = (−0.022, 0.485).
The interval contains θ = 0 (log) and nearly contains θ = 0.5 (square root).
Shapiro-Wilk results: raw data SW = 0.929 (p ≈ 0), y^0.23 SW = 0.987 (p = 0.238), log SW = 0.981 (p = 0.050), sqrt SW = 0.979 (p = 0.050).
The MLE transformation θ = 0.23 gives the best normal fit.
Yonkers ozone
MLE: θ-hat = 0.32, 95% CI = (0.047, 0.60).
Using θ = 0.23 (Stamford's MLE) for Yonkers still gives SW = 0.988, p = 0.206, which is a good fit.
If you need to compare two or more groups (e.g. Stamford vs Yonkers ozone), you must use the same θ for all groups. Otherwise the measurements are on different scales and comparisons of means, medians, or any other parameter are invalid.
Choose a common θ that produces acceptable fits for all groups. In the ozone example, θ = 0.23 works for both cities.
For reference, here is when to use each approach:
Discrete, all parameters known: Chi-square GOF, df = k − 1
Discrete, some parameters estimated: Chi-square GOF, df = k − 1 − w
Continuous, all parameters known: KS, CvM, or AD (prefer AD)
Continuous, normal with μ and σ unknown: Shapiro-Wilk test
Continuous, exponential/Weibull/other with unknown parameters: Modified AD with distribution-specific tables (Stephens and D'Agostino)
Continuous, distribution not covered by special tables: Estimate parameters via MLE, use KS or AD as if fully specified (approximate; works best for large n)
Censored data: Modified AD procedure, or estimate parameters from full likelihood (censored + uncensored), then use AD on uncensored values only with m as sample size
Always supplement with graphical methods (Q-Q plots, reference distribution plots)
Box-Cox transformation
y^(θ) = (y^θ − 1)/θ for θ ≠ 0 y^(θ) = log(y) for θ = 0
Profile log-likelihood
l*_max(θ) = (θ − 1) Σ log(y_i) − (n/2)[log(2π σ-hat²(θ)) + 1]
where σ-hat²(θ) = (1/n) Σ (y^(θ)_i − y-bar^(θ))²
95% CI for θ
{θ : l_max(θ-hat) − l_max(θ) ≤ 1.92}
Common special cases of θ
θ | Transformation |
|---|---|
−1 | Reciprocal |
−0.5 | 1/√y |
0 | log(y) |
0.5 | √y |
1 | y (identity) |
⚠️ The Box-Cox transformation only works for strictly positive data. If your data include zeros or negatives, you need Yeo-Johnson or a shift before applying Box-Cox.
⚠️ After transforming, verify normality of the transformed data using a GOF test (e.g. Shapiro-Wilk) or a normal Q-Q plot. The transformation is not guaranteed to produce normality.
⚠️ When comparing multiple groups, always apply the same θ to every group. Picking different transformations for different groups makes comparisons meaningless.
⚠️ A confidence interval for θ is essential. If it is wide, the data do not strongly prefer one power over another. If a convenient value (0, 0.5, etc.) falls inside the CI, use it for interpretability.
⚠️ Exam questions may give you the profile likelihood plot and ask you to read off θ-hat and the CI endpoints, or ask you to compute σ-hat²(θ) for a given θ and decide whether it improves fit.
Q: What is the Box-Cox transformation and when is it used?
A: It is a family of power transformations y^(θ) = (y^θ − 1)/θ (with log(y) at θ = 0) applied to positive, skewed data to make them approximately normal so that standard statistical procedures can be used.
Q: How is the optimal θ selected via maximum likelihood?
A: For each candidate θ, compute the transformed data, calculate the sample variance σ-hat²(θ), and evaluate the profile log-likelihood. The θ that maximises the profile log-likelihood is the MLE.
Q: A 95% CI for θ is (−0.02, 0.49). A researcher wants to use a log transformation (θ = 0). Is this justified?
A: Yes. θ = 0 falls within the 95% CI, so the log transformation is a statistically defensible choice, and it has the advantage of easy interpretation.
Q: Why is it important to use the same θ when comparing two groups?
A: Different θ values produce measurements on different scales. Using the same transformation ensures that summary statistics (means, variances) are directly comparable across groups.
Q: What should you do if your data contain some zero values and you want to apply Box-Cox?
A: Standard Box-Cox requires strictly positive data. Options include adding a small constant to all values before transforming (though this is ad hoc), or using the Yeo-Johnson transformation, which accommodates zero and negative values.
Q: After applying a Box-Cox transformation with θ = 0.23, you get a Shapiro-Wilk p-value of 0.24. What do you conclude?
A: The transformed data are consistent with a normal distribution (p = 0.24 is well above conventional thresholds). The transformation has successfully brought the data close to normality.
Box-Cox transformation, power transformation, theta, log transformation, square root transformation, reciprocal transformation, profile likelihood, MLE for theta, confidence interval for theta, Yeo-Johnson, normality assumption, skewed data, Shapiro-Wilk after transformation, normal Q-Q plot, comparing transformed groups, common transformation, ozone data example, Stamford, Yonkers