Source: Principles of Statistics I, Texas A&M / Tamhane-Dunlop Ch. 5, 6.1, 14.6.2, 15.1
Tags: goodness of fit, continuous distributions, Kolmogorov-Smirnov, Cramer-von Mises, Anderson-Darling, empirical CDF, distribution-free, probability integral transform, normal GOF, exponential GOF, Weibull GOF, Shapiro-Wilk, censored data
For continuous data, the chi-square GOF test is not recommended because binning the data loses sensitivity (especially in the tails). Instead, three EDF-based statistics compare the empirical CDF directly to the proposed model CDF: Kolmogorov-Smirnov (max gap), Cramer-von Mises (average squared gap), and Anderson-Darling (weighted average that emphasises the tails). When all parameters are known the statistics are distribution-free thanks to the probability integral transform; when parameters must be estimated, special modification tables are needed for each distribution family.
Empirical CDF, F_n(y)
The proportion of sample values at or below y: F_n(y) = (1/n) Σ I(Y_i ≤ y). It is a step function that estimates the true CDF.
Kolmogorov-Smirnov (KS) statistic, D_n
The maximum absolute vertical distance between the empirical CDF and the proposed model CDF over all y. D_n = max(D_n⁻, D_n⁺).
D_n⁺ and D_n⁻
The one-sided KS components. D_n⁺ = max_i [i/n − F_o(Y_(i))] captures where the empirical CDF exceeds the model; D_n⁻ = max_i [F_o(Y_(i)) − (i−1)/n] captures where the model exceeds the empirical CDF.
Cramer-von Mises (CvM) statistic, W²_n
An integral (sum) of squared differences between F_n and F_o. Sensitive to overall discrepancy rather than just the single worst point.
Anderson-Darling (AD) statistic, A²_n
A weighted version of the CvM statistic that places extra weight on the tails, making it more powerful for detecting tail departures.
Probability integral transform
If Y has continuous CDF F, then U = F(Y) follows a Uniform(0,1) distribution. This is the key result that makes KS, CvM, and AD statistics distribution-free when F is fully specified.
Distribution-free statistic
A test statistic whose null distribution does not depend on which continuous CDF F_o was proposed, provided F_o is correct. All three EDF statistics (KS, CvM, AD) have this property when parameters are known.
Shapiro-Wilk (SW) statistic, W
A normality-specific GOF measure based on the ratio of the BLUE of σ² to the MLE of σ². Values near 1 indicate normality; computed in R via shapiro.test(x).
Correlation coefficient GOF test (R)
Measures how closely the points on a normal probability plot fall on a straight line. Uses Blom plotting positions u_i = (i − 0.375)/(n + 0.25). Requires special critical-value tables (Looney and Gulledge, 1985).
Order statistics, Y_(1) ≤ Y_(2) ≤ ... ≤ Y_(n)
The sample values sorted from smallest to largest. All EDF-based GOF computations work with order statistics.
Modified statistic
A small-sample correction applied to D_n, W²_n, or A²_n so that a single set of percentile tables can serve all sample sizes.
Discretising a continuous distribution into bins is arbitrary (how many bins? where to place boundaries?).
Sensitivity to tail departures is poor, and many statistical procedures depend on accurate tail behaviour (confidence intervals, hypothesis test power, percentile calculations).
The same weaknesses that plague histograms as density estimators apply here.
The three EDF-based measures all start from the same setup:
Compute U_(i) = F_o(Y_(i)) for each order statistic. If the model is correct, these are order statistics from Uniform(0,1).
Calculate the statistic, then look up the modified statistic in Table 1 (percentile table for completely specified distributions).
D_n⁺ = max over i of [i/n − U_(i)]
D_n⁻ = max over i of [U_(i) − (i−1)/n]
D_n = max(D_n⁺, D_n⁻)
Modified: D_n * (√n + 0.12 + 0.11/√n)
Measures only the single largest gap, so it can miss widespread moderate departures and is weak in the tails.
W²_n = Σ [U_(i) − (2i−1)/(2n)]² + 1/(12n)
Modified: (W²_n − 0.4/n + 0.6/n²) * (1 + 1/n)
Captures overall discrepancy (integrates squared differences).
Still not particularly sensitive to tail departures.
Uses weight function Ψ(y) = [F_o(y)(1 − F_o(y))]⁻¹, which inflates discrepancies in the tails where Var[F_n(y) − F(y)] is small.
A²_n = −n − (1/n) Σ [(2i−1) ln(U_(i)) + (2n+1−2i) ln(1 − U_(i))]
No further modification needed for n ≥ 5; use Table 2 (AD CDF table) directly.
Most powerful of the three for detecting tail departures; generally the recommended choice.
Data: 20 weights of 21-day-old Leghorn Chickens.
Proposed model: N(200, 35²).
Results: KS modified = 0.790 (p > 0.25), CvM modified = 0.177 (p > 0.25), AD = 1.017 (p ≈ 0.35).
All three statistics confirm the normal model fits well.
R shortcut: ks.test(x, "pnorm", 200, 35) gives D = 0.1712, p = 0.60.
When parameters are unknown, estimate them via MLE, compute U_(i) = F_o(Y_(i); θ-hat), and apply distribution-specific modification formulas and tables. The completely-specified tables (Tables 1 and 2) are no longer valid.
Does not require specifying μ or σ.
W = [Σ a_{n−i+1} (X_(n−i+1) − X_(i))]² / Σ (X_i − X-bar)², where a_i coefficients come from Table A28.
W close to 1 implies good normal fit.
In R: shapiro.test(x). For the chicken data, W = 0.9757, p = 0.8667 (excellent fit).
One of the most powerful tests for normality.
Compute Blom plotting positions: u_i = (i − 0.375)/(n + 0.25).
Plot Q_Z(u_i) vs Y_(i) (normal Q-Q plot).
R = Pearson correlation of those n points.
R close to 1 means the points lie near a straight line, supporting normality.
Standard correlation tables are invalid here; use Looney-Gulledge tables.
For the chicken data: R = 0.991, p ≈ 0.90.
KS modified: D_n(√n − 0.01 + 0.85/√n). Reject at 0.05 if modified > 0.895.
CvM modified: W²_n(1 + 0.5/n). Reject at 0.05 if modified > 0.126.
AD modified: A²_n(1 + 0.75/n + 2.25/n²). Reject at 0.05 if modified > 0.752.
R package "nortest" provides ad.test(x), cvm.test(x), lillie.test(x).
MLE: β-hat = Y-bar.
Replace F_o(Y_(i)) with 1 − exp(−Y_(i)/Y-bar).
KS: (D_n − 0.2/n)(√n + 0.26 + 0.5/√n). Reject at 0.05 if > 1.094.
CvM: W²_n(1 + 0.16/n). Reject at 0.05 if > 0.222.
AD: A²_n(1 + 0.6/n). Reject at 0.05 if > 1.321.
Air conditioner example (n = 15): KS mod = 1.122, CvM mod = 0.221, AD mod = 1.210. Exponential fit is marginal (p around 0.05).
Caution: ks.test(x, "pexp", 1/beta_hat) in R does not adjust for the estimated parameter and will overstate the p-value.
Weibull CDF: G_o(y) = 1 − exp(−(y/α)^γ).
Transform Y_i = −log(X_i) to convert to an extreme-value distribution.
Use fitdistr(x, "weibull") from MASS package in R to get MLE of shape (γ) and scale (α).
Convert: θ = 1/γ, φ = −log(α).
Compute U_(i) = exp(−exp(−(Y_(i) − φ-hat)/θ-hat)), then get A²_n.
AD modified: A²_n(1 + 0.2/√n). Reject at 0.05 if > 0.757.
Ball bearing example (n = 23): AD = 0.328, modified = 0.341, p > 0.25. Weibull fits excellently.
Ozone data: Stamford AD = 0.698 (0.05 < p < 0.10), Yonkers AD = 0.572 (0.10 < p < 0.25).
For right-censored data with a completely specified model, a modified AD formula accounts for censoring proportion R.
Type I censoring: R = F_o(t) where t is the censoring time.
Type II censoring: R = F_o(Y_(m)) where m observations are uncensored.
p-values come from a special table indexed by R.
For random censoring: compute A²_m using only uncensored observations with m as the sample size.
Graphical approaches using the Kaplan-Meier estimator S-hat(t) can also assess fit:
Plot log(S-hat(t)) vs t for exponential
Plot log(−log(S-hat(t))) vs log(t) for Weibull
Plot S-hat(t) vs t against the lognormal CDF for lognormal
Empirical CDF
F_n(y) = (1/n) Σ I(Y_i ≤ y)
KS statistic
D_n = max(D_n⁺, D_n⁻) D_n⁺ = max_i [i/n − U_(i)] D_n⁻ = max_i [U_(i) − (i−1)/n]
CvM statistic
W²_n = Σ [U_(i) − (2i−1)/(2n)]² + 1/(12n)
AD statistic
A²_n = −n − (1/n) Σ [(2i−1) ln(U_(i)) + (2n+1−2i) ln(1−U_(i))]
Probability integral transform
If Y ~ F (continuous), then U = F(Y) ~ Uniform(0,1).
Shapiro-Wilk
W = [Σ a_{n−i+1}(X_(n−i+1) − X_(i))]² / Σ(X_i − X-bar)²
Key R functions
ks.test(x, "pnorm", mu, sigma) – KS test, fully specified
shapiro.test(x) – Shapiro-Wilk normality test
ad.test(x), cvm.test(x), lillie.test(x) – from "nortest" package, parameters estimated
fitdistr(x, "weibull") – from MASS package, Weibull MLEs
⚠️ The AD test is generally the most powerful of the three EDF tests because its weighting function emphasises the tails. When you have a choice, prefer AD.
⚠️ The KS test is distribution-free only when the model is completely specified. If you estimate parameters and still use the standard KS tables (e.g. ks.test in R with plug-in estimates), the p-value will be too large and you may incorrectly conclude the model fits.
⚠️ For normality testing with unknown μ and σ, the Shapiro-Wilk test is one of the most powerful options. Use it as your default normality check.
⚠️ Different distribution families need different modification formulas and tables (Table 3 for Normal, Table 4 for Exponential, Table 5 for Weibull via Extreme Value). Do not mix them up.
⚠️ The probability integral transform (U = F(Y) ~ Uniform(0,1)) is the theoretical backbone of all EDF-based GOF tests. It is commonly tested conceptually.
⚠️ When reporting GOF results, state which statistic you used, whether parameters were specified or estimated, the test statistic value, and the p-value (or at minimum, the range from the table).
Q: Why is the chi-square GOF test not recommended for continuous data?
A: Discretising the data into bins is arbitrary and loses sensitivity, particularly in the tails where accurate probability calculations matter most for inference.
Q: What does the probability integral transform guarantee, and why does it matter for GOF testing?
A: It guarantees that if Y has continuous CDF F, then F(Y) ~ Uniform(0,1). This means the null distributions of KS, CvM, and AD do not depend on F_o, so a single set of tables works for any fully specified continuous model.
Q: You compute D_n = 0.17 for n = 20 and the modified KS statistic is 0.79. Using Table 1, what can you say about the p-value?
A: The 0.25 upper percentile for the modified KS is 1.019. Since 0.79 < 1.019, the p-value exceeds 0.25, indicating an excellent fit.
Q: A colleague uses ks.test(x, "pexp", 1/mean(x)) in R and reports p = 0.20, concluding the exponential fits well. What is wrong with this approach?
A: The ks.test function assumes the distribution is completely specified. Because 1/mean(x) is an estimate of the rate parameter, the reported p-value is inflated. The correct approach uses the modification from Table 4, which would likely yield a smaller p-value.
Q: What advantage does the Anderson-Darling test have over the Kolmogorov-Smirnov test?
A: The AD test uses a weight function that emphasises the tails of the distribution, giving it greater power to detect departures where KS (which only captures the single largest gap) is weakest.
Q: In the Shapiro-Wilk test, what does W close to 1 indicate?
A: It indicates the normal reference plot is nearly a straight line, meaning the data are consistent with a normal distribution.
EDF-based GOF, empirical distribution function, Kolmogorov-Smirnov test, KS test, Cramer-von Mises, CvM, Anderson-Darling, AD test, probability integral transform, distribution-free, Shapiro-Wilk, W statistic, normality test, normal probability plot, correlation coefficient test, Blom plotting positions, modified statistic, percentile tables, exponential GOF, Weibull GOF, extreme value distribution, censored data GOF, right censoring, Kaplan-Meier, nortest R package, ks.test, shapiro.test