Source: Tamhane/Dunlop Ch. 2, Sections 4.1, 4.2, 4.3.4
Tags: empirical distribution function, EDF, sample CDF, smoothed EDF, order statistics, step function, piecewise linear CDF, cumulative distribution function estimator
The empirical distribution function (EDF) estimates the population CDF by counting how many data values fall at or below each point. The raw EDF is a step function, which is fine for discrete data but jagged for continuous data. Smoothed versions connect the steps with straight lines, producing a continuous, piecewise linear estimate.
Empirical distribution function (EDF)
The sample estimator of the population CDF, denoted F-hat(y). It gives the proportion of observed data values less than or equal to y.
Order statistics
The sample values sorted from smallest to largest: Y_(1) ≤ Y_(2) ≤ ... ≤ Y_(n). These form the backbone of every nonparametric estimator in this handout.
Indicator function I(Yi ≤ y)
A function that equals 1 when Yi ≤ y and 0 otherwise. Used to count how many observations fall at or below y.
Raw EDF (step function EDF)
The unmodified empirical distribution function, which jumps by 1/n at each distinct order statistic. It is right-continuous and piecewise constant.
Smoothed EDF
A continuous version of the EDF obtained by connecting points on the step function with straight lines, yielding a piecewise linear curve that better represents a continuous population CDF.
Tied values
Multiple observations sharing the same value. When ties occur, the jump height at that value is k/n, where k is the number of tied observations.
The population CDF is defined as F(y) = P[Y ≤ y], which can be read as the proportion of the population that is less than or equal to y.
The natural sample-based estimator replaces the population proportion with the sample proportion:
F-hat(y) = (number of Yi's ≤ y) / n
Equivalently: F-hat(y) = (1/n) × Σ I(Yi ≤ y), summing from i = 1 to n
This is the same as the sample proportion p-hat, where p = P[Y ≤ y]
It is a step function (piecewise constant)
Jumps occur at each order statistic Y_(1), Y_(2), ..., Y_(n)
Each jump has height 1/n when there are no ties
When k observations share the same value, the jump at that value has height k/n
The function is right-continuous, matching the convention for population CDFs
For discrete data, a step function is a natural and appropriate shape for the CDF
For continuous data, the true CDF is a smooth, continuous curve, so the step function is a rough approximation that can be improved by smoothing
A simple improvement: connect the midpoints of the flat segments in the raw EDF with straight lines.
If there are no ties, this produces a strictly increasing, continuous, piecewise linear function
This version better represents the shape you would expect from a continuous population CDF
An alternative version connects the endpoints of the steps rather than the midpoints. The formula for Y_(i) ≤ y < Y_(i+1) and i = 1, ..., n - 1 is:
F-hat_C(y) = F-hat(Y_(i)) × [(Y_(i+1) - y) / (Y_(i+1) - Y_(i))] + F-hat(Y_(i+1)) × [(y - Y_(i)) / (Y_(i+1) - Y_(i))]
This is a weighted average (linear interpolation) of the EDF values at the two neighbouring order statistics. At the order statistics themselves, it agrees with the raw EDF:
F-hat_C(Y_(k)) = F-hat(Y_(k)) = k/n for k = 1, ..., n
Boundary behaviour:
F-hat_C(y) = 0 for y < Y_(1)
F-hat_C(y) = 1 for y ≥ Y_(n)
Raw EDF formula
F-hat(y) = (1/n) × Σ I(Yi ≤ y) from i = 1 to n
Smoothed EDF (endpoint version) formula
For Y_(i) ≤ y < Y_(i+1):
F-hat_C(y) = (i/n) × [(Y_(i+1) - y) / (Y_(i+1) - Y_(i))] + ((i+1)/n) × [(y - Y_(i)) / (Y_(i+1) - Y_(i))]
Key graph features to remember
Raw EDF: staircase shape, open circles at left endpoints of steps, closed at right
Smoothed EDF: dotted diagonal lines connecting the step midpoints or endpoints through the staircase
⚠️ The EDF is the foundation for all the nonparametric estimators in this handout. Understanding how it works makes the quantile function and density estimators much easier to follow.
⚠️ Know the difference between the raw EDF (step function) and the two smoothed versions (midpoint vs. endpoint connections). Be able to say which is appropriate for discrete vs. continuous data.
⚠️ Tied values change the jump height from 1/n to k/n. This is a common exam detail.
⚠️ The smoothed EDF connecting endpoints uses linear interpolation, a concept that reappears in the continuous sample quantile function.
Q: What is the empirical distribution function, and what does F-hat(y) estimate?
A: The EDF is the proportion of sample values at or below y. It estimates the population CDF, F(y) = P[Y ≤ y].
Q: If you have n = 10 observations and 3 of them are less than or equal to 50, what is F-hat(50)?
A: F-hat(50) = 3/10 = 0.30.
Q: Why is the raw EDF inadequate for continuous data, and how is this fixed?
A: The raw EDF is a step function, while a continuous population CDF is smooth. Smoothing fixes this by connecting the midpoints or endpoints of the steps with straight lines, producing a piecewise linear, continuous function.
Q: In a sample of size n = 8 with no ties, what is the height of each jump in the raw EDF?
A: Each jump has height 1/8.
Q: If three observations in a sample of size 20 share the same value y_0, what is the jump height at y_0?
A: The jump height is 3/20 = 0.15.
empirical distribution function, EDF, ecdf, sample CDF, cumulative distribution function estimator, step function CDF, smoothed EDF, piecewise linear CDF, order statistics, indicator function, F-hat, right-continuous function, linear interpolation CDF, Principles of Statistics I, Texas A&M, STAT, Tamhane Dunlop