Source: Principles of Statistics I, Texas A&M / Tamhane-Dunlop Ch. 5, 6.1
Tags: goodness of fit, GOF, chi-square test, discrete distributions, binomial fit, Poisson fit, expected frequency, observed frequency, p-value, model adequacy
Goodness-of-fit (GOF) testing asks whether a proposed probability model plausibly generated the data you observed. For discrete distributions the workhorse is the chi-square statistic, which compares observed cell counts to the counts the model predicts. When model parameters are unknown you estimate them first, then pay a degrees-of-freedom penalty.
Goodness of fit (GOF)
A quantitative assessment of how well a proposed probability distribution matches observed data, going beyond visual tools like reference-distribution plots.
Chi-square GOF statistic (Q)
The sum of squared differences between observed and expected counts, each divided by its expected count. Measures overall discrepancy between data and model.
Observed count (O_i)
The number of sample observations that fall into category i.
Expected count (E_i)
The number of observations category i would receive if the proposed model were correct: E_i = n * p_i.
Completely specified distribution
A proposed model in which every parameter value is known before looking at the data (e.g. Binomial(5, 0.25)).
Incompletely specified distribution
A proposed model where one or more parameters must be estimated from the data before testing fit.
Maximum likelihood estimator (MLE)
The parameter estimate that maximises the likelihood of the observed data; used to fill in unknown parameters before computing Q.
Degrees of freedom (df)
For the chi-square GOF test: df = k − 1 when all parameters are specified, or bounded by k − 1 − w ≤ df ≤ k − 1 when w parameters are estimated. Using df = k − 1 − w is preferred (stricter).
Yates's correction for continuity
An adjustment applied when df = 1 and expected frequencies are below 10. Reduces each |O_i − E_i| by 0.5 before squaring.
Let Y_1, Y_2, ..., Y_n be iid from a discrete distribution with known pmf f_o(y; θ).
Group the possible outcomes into k categories with probabilities p_1, p_2, ..., p_k (the last category collects all remaining probability: p_k = 1 − sum of the rest).
Compute observed counts O_i and expected counts E_i = n * p_i for each category.
The chi-square GOF statistic is:
Q = Σ (O_i − E_i)² / E_i, summed from i = 1 to k
Division by E_i standardises the contribution so that large-n datasets do not automatically produce large Q values.
Under the null hypothesis (the model is correct), Q follows approximately a chi-square distribution with df = k − 1 for large n.
p-value = P[χ²(k−1) ≥ Q]. In R: 1 - pchisq(Q, k-1).
Every E_i must exceed 1.0.
No more than 20% of the E_i values may be below 5.0.
When these are violated (usually because n is small), combine adjacent cells until conditions are met, then recompute Q with the reduced k.
A rough minimum sample size: n = 5 / min(p_1, ..., p_k).
With only 1 degree of freedom, expected frequencies should be at least 10; otherwise apply Yates's correction:
Q = Σ (|O_i − E_i| − 0.5)² / E_i
p-value > 0.25: Excellent fit
0.15 ≤ p-value < 0.25: Good fit
0.05 ≤ p-value < 0.15: Moderately good fit
0.01 ≤ p-value < 0.05: Poor fit
p-value < 0.01: Unacceptable fit
These thresholds are conventional and somewhat arbitrary, but widely used.
100 couples, each with 5 children; proposed model B(5, 0.25) for number of diseased children per family.
Initial Q = 13.46 with df = 5, giving p-value ≈ 0.019 (poor fit).
Cells for Y = 4 and Y = 5 have E_i values of 1.465 and 0.0977, violating the validity rules.
After combining those two cells into "4 or 5": Q_new = 8.30, df = 4, p-value ≈ 0.08 (moderately good fit).
The binomial assumptions (independence of trials) may not hold perfectly here, which could explain the imperfect fit.
When parameter θ (or part of it) is unknown, estimate it via MLE (denoted θ-hat), then compute Q as before using the estimated probabilities.
The df for the approximating chi-square are reduced: use df = k − 1 − w, where w = number of estimated parameters.
Using the reduced df gives a more conservative test (harder to declare the model adequate), so it is the recommended approach.
The p-value is bounded: P[χ²(k−1−w) ≥ Q] ≤ true p-value ≤ P[χ²(k−1) ≥ Q].
150 flashlights, 4 batteries each; D = number of defectives per flashlight.
MLE of defective rate: θ-hat = 233/600 = 0.3883.
Q-hat = 15.87, df = 5 − 1 − 1 = 3, p-value = 0.0012: unacceptable fit.
The assumption that all 4 batteries in one flashlight share the same defective probability independently may be wrong (batteries within the same flashlight could be correlated).
300 chromosomes; count of exchanges ranges 0 to 9.
MLE: λ-hat = X-bar = 1155/300 = 3.85.
Estimated probabilities via dpois(i, 3.85) in R; last cell groups counts ≥ 9.
Q-hat = 9.12, df = 10 − 1 − 1 = 8, p-value = 0.332: excellent fit.
Chi-square GOF statistic
Q = Σ_{i=1}^{k} (O_i − E_i)² / E_i
Expected count
E_i = n * p_i
Yates's correction (df = 1 only)
Q = Σ (|O_i − E_i| − 0.5)² / E_i
Degrees of freedom
All parameters known: df = k − 1
w parameters estimated: df = k − 1 − w (preferred, conservative)
R functions
1 - pchisq(Q, df) for p-value
dpois(x, lambda) for Poisson pmf
dbinom(x, n, p) for binomial pmf
⚠️ Always check the validity conditions (E_i > 1, at most 20% below 5) before trusting the chi-square p-value. Combine cells if needed, and remember that combining cells changes both Q and df.
⚠️ When parameters are estimated from the data, use the reduced df = k − 1 − w. Using df = k − 1 instead overstates the p-value and may falsely suggest a good fit.
⚠️ A large p-value does not prove the model is correct; it means the data are consistent with the model. A small p-value means the model does not fit well, but examine which cells contribute most to Q to understand why.
⚠️ The chi-square GOF test is not recommended for continuous data (that topic is covered in Part 2) because discretising continuous data loses sensitivity, especially in the tails.
Q: What does the chi-square GOF statistic measure?
A: It measures the overall discrepancy between observed cell counts (O_i) and the counts expected under the proposed model (E_i), standardised by E_i so that the statistic is comparable across different sample sizes.
Q: Why do we divide by E_i in the formula for Q?
A: To modulate for sample size. Without dividing, larger datasets would produce larger Q values even when the model fits well, because raw differences grow with n.
Q: A researcher computes Q = 15.87 for a Binomial(4, θ) fit with 5 cells and one estimated parameter. What are the degrees of freedom and how is the p-value obtained?
A: df = 5 − 1 − 1 = 3. The p-value is P[χ²(3) ≥ 15.87] = 1 − pchisq(15.87, 3) ≈ 0.0012, indicating an unacceptable fit.
Q: You have 6 cells and two of them have E_i below 1. What should you do before computing the p-value?
A: Combine cells with low expected counts with adjacent cells until all E_i exceed 1 and at most 20% are below 5. Then recompute Q with the new, reduced number of cells k and df = k − 1 (or k − 1 − w if parameters were estimated).
Q: Why is df = k − 1 − w preferred over df = k − 1 when parameters are estimated?
A: Using the reduced df makes the test stricter (i.e. it requires a better fit to declare the model adequate). This compensates for the fact that estimating parameters from the same data used for the GOF test inflates the apparent agreement between model and data.
chi-square goodness of fit, GOF test, discrete GOF, observed vs expected, cell combining, Yates correction, binomial fit test, Poisson fit test, model adequacy, completely specified distribution, incompletely specified distribution, estimated parameters, degrees of freedom reduction, p-value interpretation, fit assessment guidelines, chi-square approximation validity, minimum expected frequency