Source: Handout #13, Tamhane/Dunlop Ch. 8, Ch. 9, Section 14.2
Tags: Hartley Fmax test, Brown-Forsythe-Levene test, BFL test, homogeneity of variance, equal variance test, von Neumann test, autocorrelation, runs test, serial correlation, Tukey HSD, pairwise comparisons, studentized range, experimentwise error rate, multiple comparisons
Comparing population variances uses Hartley's Fmax test (simple but requires normality and equal sample sizes) or the Brown-Forsythe-Levene test (robust to nonnormality). For checking whether observations are correlated over time, use the von Neumann test (normal data) or the runs test (distribution-free). When comparing more than two population means, the Tukey HSD procedure controls the experimentwise error rate while making all pairwise comparisons.
Hartley's Fmax statistic
Fmax = s²_max / s²_min, the ratio of the largest to smallest sample variance. Requires normal populations and equal sample sizes.
Brown-Forsythe-Levene (BFL) test
A robust test of equal variances based on the absolute deviations of observations from their group medians. Does not require normality or equal sample sizes.
von Neumann ratio (Q)
Q = [Σ(Xₜ − Xₜ₋₁)² / (n−1)] / [Σ(Xₜ − X̄)² / n]. Tests for first-order autocorrelation in normally distributed data. Small Q indicates positive correlation; large Q indicates negative correlation.
First-order autocorrelation (ρ)
The correlation between consecutive observations in a time series: Xₜ = θ + ρXₜ₋₁ + eₜ. When ρ = 0, the observations are independent.
Runs test
A distribution-free test for serial correlation. A run is a consecutive sequence of values all above (or all below) the mean. Too few runs indicates positive correlation; too many indicates negative correlation.
Tukey HSD (Honest Significant Difference)
A multiple comparison procedure based on the studentized range distribution. Controls the experimentwise Type I error rate exactly (not just a bound) when sample sizes are equal.
Studentized range statistic
q(t, ν) = (ȳ_max − ȳ_min) / (σ̂_p √(1/r)). The distribution of the range of t sample means, standardised by the pooled standard error.
Experimentwise error rate (α_Exp)
The probability of making at least one Type I error across all pairwise comparisons. The Tukey HSD controls this at exactly α₀ when sample sizes are equal.
Conditions:
Independent random samples of the same size n from t normally distributed populations
This is a strict requirement: Fmax is highly sensitive to nonnormality
Hypotheses:
H₀: σ₁² = σ₂² = ... = σₜ² Hₐ: not all σᵢ² are equal
Test statistic:
Fmax = s²_max / s²_min
Rejection rule: Reject H₀ if Fmax ≥ Fmax,α,t,n−1, using the Fmax percentile table (not the standard F-table).
When t = 2: Fmax has an exact F-distribution with df₁ = n − 1, df₂ = m − 1.
Sensitivity to nonnormality: A simulation with α = .05 shows:
Normal: actual error rate stays close to .05
Uniform: error rate drops to .004 to .01 (far too conservative)
t(df=5): error rate inflates to .12 to .15
Gamma(shape=1): error rate inflates to .22 to .26
Gamma(shape=0.1): error rate inflates to .67 to .69
The Fmax test detects the nonnormality, not the variance differences. When distributions are nonnormal, use the BFL test instead.
The BFL test works by transforming the data into absolute deviations from group medians, then running a standard ANOVA F-test on these transformed values.
Steps:
Compute the median ỹᵢ of each sample
Transform each observation: zᵢⱼ = |yᵢⱼ − ỹᵢ|
Compute the BFL statistic as an F-ratio on the zᵢⱼ values
Test statistic:
L = [Σᵢ nᵢ(z̄ᵢ. − z̄..)² / (t − 1)] / [ΣΣ(zᵢⱼ − z̄ᵢ.)² / (N − t)]
This is just a one-way ANOVA F-test on the absolute deviations.
Rejection rule: Reject H₀ if L ≥ F(α, t−1, N−t).
Hajek-Sidak modification: When a sample size is odd, exactly one zᵢⱼ will be zero (the observation equal to the median). This artificially dampens the variance. Replace the single zero with the minimum non-zero value within that group. If more than one zᵢⱼ is zero, keep them as zeros.
Advantages over Hartley:
Does not require normality
Does not require equal sample sizes
Recommended by Conover, Johnson, and Johnson (1981) as one of the best procedures
Trade-off: When populations are truly normal, Hartley's test has more power than BFL.
R: leveneTest(y, grp) from the car package, or compute manually via aov(z ~ grp) where z contains the absolute deviations from group medians.
SAS: means group / hovtest=bf in PROC GLM.
Tests whether observations collected over time are first-order autocorrelated.
Model: Xₜ = θ + ρXₜ₋₁ + eₜ, where eₜ is independent normal noise.
Hypotheses: H₀: ρ = 0 (observations are independent) vs Hₐ: ρ ≠ 0
Test statistic:
Q = [Σₜ₌₂ⁿ (Xₜ − Xₜ₋₁)² / (n − 1)] / [Σₜ₌₁ⁿ (Xₜ − X̄)² / n]
Interpretation:
Small Q indicates positive correlation (consecutive observations tend to be similar)
Large Q indicates negative correlation (consecutive observations tend to alternate)
Critical values:
For n ≤ 60, use the von Neumann table
For n > 60, use the normal approximation: Q_P,α ≈ 2n/(n−1) − z_α √(2n) and Q_N,α ≈ 2n/(n−1) + z_α √(2n)
Rejection rules:
Hₐ: ρ > 0 → Reject if Q < Q_P,α
Hₐ: ρ < 0 → Reject if Q > Q_N,α
Consequences of ignoring positive correlation:
When data is positively correlated but treated as independent:
Var(X̄) = σ²/n + Δ, where Δ > 0. The actual variance of the mean is larger than the formula σ²/n suggests.
Confidence intervals are too narrow, so coverage probability drops below 1 − α.
Tests of hypotheses have inflated Type I error rates.
When normality cannot be assumed, the runs test provides a nonparametric alternative to the von Neumann test.
Steps:
Centre the observations: Yₜ = Xₜ − X̄
Count the number of runs (R), where a run is a consecutive sequence of all positive or all negative values
Count n₁ (number of positive Yₜ) and n₂ (number of negative Yₜ)
Small samples (n₁ ≤ 20 and n₂ ≤ 20): Use the exact runs test table.
Large samples: Use the normal approximation:
Z = (|R − µ| − 0.5) / σ
where µ = 1 + 2n₁n₂/(n₁ + n₂) and σ² = 2n₁n₂(2n₁n₂ − n₁ − n₂) / [(n₁ + n₂)²(n₁ + n₂ − 1)]
Reject H₀ (data is independent) if Z > z_(α/2).
Autocorrelation function (ACF): The R function acf(y) plots lag-k correlations with 95% confidence bands. Values outside the bands at lags 1, 2, 3, ... indicate significant autocorrelation at those lags.
When comparing t > 2 population means, the Tukey HSD controls the experimentwise Type I error rate while testing all C(t,2) pairwise differences.
Conditions: Equal sample sizes (r = n₁ = ... = nₜ), equal variances (σ₁ = ... = σₜ), and normal distributions.
Based on the studentized range distribution:
q(t, ν) = (ȳ_max − ȳ_min) / (σ̂_p √(1/r))
HSD threshold:
HSD(t, ν) = q(α₀, t, ν) × σ̂_p × √(1/r)
where ν = t(r − 1), and q(α₀, t, ν) is the upper α₀ quantile of the studentized range distribution.
Decision rule: Declare µᵢ and µⱼ significantly different if |ȳᵢ − ȳⱼ| ≥ HSD(t, ν).
Simultaneous CIs: |ȳᵢ − ȳⱼ| ± HSD(t, ν) for all pairs simultaneously.
Tukey-Kramer modification (unequal sample sizes):
Replace √(1/r) with √(1/2 × (1/nᵢ + 1/nⱼ)), and use ν = N − t where N = Σnᵢ.
With unequal sample sizes, the experimentwise error rate is bounded: α_Exp ≤ α₀ (no longer exact).
R:
model <- aov(y ~ group)
TukeyHSD(model, "group", ordered=FALSE, conf.level=0.95)
plot(TukeyHSD(model, "group"))
The R function qtukey(1 - alpha, t, nu) provides the critical value from the studentized range distribution.
Comparison with Bonferroni: The Tukey HSD and Scheffé's procedure have exact experimentwise error rates. The Bonferroni procedure provides only a bound (α_Exp ≤ α₀). For pairwise comparisons specifically, Tukey HSD is generally more powerful than Bonferroni.
Hartley Fmax:
Fmax = s²_max / s²_min, reject if ≥ Fmax,α,t,n−1
BFL test statistic:
L = [Σ nᵢ(z̄ᵢ. − z̄..)² / (t − 1)] / [ΣΣ(zᵢⱼ − z̄ᵢ.)² / (N − t)], where zᵢⱼ = |yᵢⱼ − median(yᵢ)|
von Neumann Q:
Q = [Σ(Xₜ − Xₜ₋₁)²/(n−1)] / [Σ(Xₜ − X̄)²/n]
Runs test (large sample):
Z = (|R − µ| − 0.5) / σ, where µ = 1 + 2n₁n₂/(n₁+n₂)
Tukey HSD:
HSD = q(α₀, t, ν) × σ̂_p × √(1/r), ν = t(r−1)
Tukey-Kramer (unequal n):
HSD_ij = q(α₀, t, ν) × σ̂_p × √(1/2 × (1/nᵢ + 1/nⱼ)), ν = N − t
⚠️ Hartley's Fmax is extremely sensitive to nonnormality. With a Gamma(shape=0.1) distribution, the actual Type I error rate was .67 to .69 instead of .05. Always check normality before using Fmax.
⚠️ The BFL test is the recommended default for testing equal variances. It is robust to nonnormality and does not require equal sample sizes.
⚠️ In the BFL test, use absolute deviations from the group median, not the mean. The median-based version is more robust to outliers.
⚠️ Positive autocorrelation causes standard CIs to be too narrow and standard tests to reject too often. Always check for correlation in time-ordered data before applying standard methods.
⚠️ The Tukey HSD requires equal sample sizes for the experimentwise error rate to be exact. With unequal sizes, use the Tukey-Kramer modification, which provides a bound α_Exp ≤ α₀.
⚠️ For Tukey HSD: the degrees of freedom ν = t(r − 1), not n − 1 or n + m − 2. This is the pooled df across all t groups.
⚠️ If a CI from Tukey HSD contains zero, there is not significant evidence of a difference for that pair. These two statements (HSD comparison and CI) always agree.
Q: Three samples of size n = 9 from normal populations yield s₁² = 8.69, s₂² = 6.89, s₃² = 80.22. Using Hartley's Fmax test at α = .05, do you reject H₀: σ₁² = σ₂² = σ₃²?
A: Fmax = 80.22/6.89 = 11.64. The critical value Fmax,.05,3,8 = 6.00. Since 11.64 > 6.00, reject H₀. The variances are significantly different.
Q: The same data in the previous question produced non-normal Shapiro-Wilk results for two of the three samples. Is the Hartley test still valid?
A: No. Hartley's Fmax is highly sensitive to nonnormality and would likely be detecting the nonnormality rather than true variance differences. Use the BFL test instead.
Q: In a BFL test with t = 3 groups and N = 30 total observations, you compute L = 1.827. The critical value F(.05, 2, 27) = 3.35. What is your conclusion?
A: Since 1.827 < 3.35, fail to reject H₀. There is insufficient evidence of a difference in population variances.
Q: A time series of 150 observations yields Q = 0.441 from the von Neumann test. The estimated first-order autocorrelation is ρ̂ = 0.775. What does this indicate?
A: The small value of Q and large positive ρ̂ indicate strong positive serial correlation. Consecutive observations tend to be similar. Standard confidence intervals and tests assuming independence would be unreliable for this data.
Q: In a runs test, 150 observations produce R = 30 runs, n₁ = 70, n₂ = 80. What are µ, σ, and the Z-statistic?
A: µ = 1 + 2(70)(80)/150 = 75.67. σ² = 2(70)(80)(2(70)(80) − 70 − 80) / (150²)(149) = 36.92. Z = (|30 − 75.67| − 0.5) / √36.92 = 7.43. p-value ≈ 1.05 × 10⁻¹³. Overwhelmingly significant evidence of correlation.
Q: Using Tukey-Kramer with t = 4 groups, ν = 58, and σ̂_p = 3.247, what is HSD for comparing groups with n₁ = 18 and n₄ = 14?
A: qtukey(.95, 4, 58) = 3.741. HSD = 3.741 × 3.247 × √(1/2 × (1/18 + 1/14)) = 3.741 × 3.247 × √(.0635) = 3.741 × 3.247 × 0.252 ≈ 3.06.
Q: What is the key advantage of Tukey HSD over Bonferroni for pairwise comparisons?
A: Tukey HSD achieves the exact experimentwise error rate α_Exp = α₀ (with equal sample sizes), whereas Bonferroni only ensures α_Exp ≤ α₀. Because the Bonferroni bound is conservative, Tukey HSD typically produces narrower confidence intervals and more powerful tests.
Hartley Fmax test, Brown-Forsythe-Levene test, BFL test, Levene test, homogeneity of variance, equal variance assumption, variance ratio test, F-test for variances, von Neumann test, autocorrelation, first-order autocorrelation, serial correlation, AR(1), lag plot, time series, runs test, distribution-free test, Tukey HSD, Tukey-Kramer, honest significant difference, studentized range, pairwise comparisons, multiple comparisons, experimentwise error rate, family-wise error rate, Bonferroni correction, Scheffé procedure, simultaneous confidence intervals, ANOVA, one-way ANOVA, pooled variance, ACF, autocorrelation function