Source: Principles of Statistics I, Texas A&M University
Tags: Kaplan-Meier estimator, product limit estimator, distribution-free estimation, nonparametric survival, survival function estimation, Greenwood formula, standard error survival, censored data quantiles, comparing survival curves, KM estimator
When the underlying distribution family is unknown, we can still estimate the survival function S(t) using the Kaplan-Meier (product limit) estimator. This nonparametric approach handles censored data by adjusting the risk set at each event time. The mean and quantiles can then be estimated from the KM curve. The Greenwood formula provides standard errors for the estimated survival probabilities.
Distribution-free (nonparametric) estimator
An estimator that does not assume the data comes from any particular parametric family. The goal is to estimate S(t) = P(T > t) and related quantities (mean, median, quantiles) directly from the data.
Kaplan-Meier (KM) product limit estimator
A nonparametric estimator of the survival function for censored data. At each observed event time, the survival estimate is multiplied by the fraction of at-risk units that survived past that time. Written as a cumulative product of conditional survival probabilities.
Risk set
The set of units still "at risk" of experiencing the event just before a given time t_j. Units that have already experienced the event or been censored before t_j are excluded from the risk set.
Greenwood formula
A formula for the estimated variance (and hence standard error) of the KM estimator at any time t. It accounts for the sequential nature of the product limit calculation.
Restricted mean
The mean survival time estimated from the KM curve, computed as the area under the Ŝ(t) curve. Called "restricted" because it can only be computed up to the largest observed event time when censoring is present.
When there is no censoring, estimating S(t) and related quantities is straightforward.
Estimating S(t): The empirical survival function is Ŝ(t) = (1/n) · Σ I(T_i > t), the proportion of observations greater than t.
Estimating the mean μ: Using the identity E[T] = ∫ S(t) dt (integrated from 0 to ∞), we get:
μ̂ = ∫ Ŝ(t) dt = (1/n) · Σ T_i = T̄
The integral of the empirical survival function reduces to the sample mean. This is a reassuring result: the nonparametric approach recovers the familiar estimator when there is no censoring.
Estimating the median M: The median satisfies S(M) = 0.5, so:
M̂ = inf{t : Ŝ(t) ≤ 0.5}
Estimating the p-th quantile Q(p):
Q̂(p) = inf{t : Ŝ(t) ≤ 1 − p}
When data is censored, the simple empirical survival function no longer works because censored observations distort the count of units "surviving" past each time point. The Kaplan-Meier estimator handles this correctly.
Setup and notation:
n homogeneous units placed on test
t_1 < t_2 < ... < t_k are the distinct times at which events (failures) occur
d_j = number of events at time t_j
m_j = number of units censored in the interval [t_j, t_(j+1))
n_j = number of units at risk just before time t_j (those that have neither experienced the event nor been censored before t_j)
Note that n_1 = n − m_0, and each subsequent risk set drops by d_j + m_j.
The KM estimator:
For t in the interval [t_i, t_(i+1)):
Ŝ(t) = ∏(j=1 to i) [(n_j − d_j) / n_j]
Each factor (n_j − d_j) / n_j is the conditional probability of surviving past t_j, given survival up to just before t_j. The product accumulates these conditional probabilities.
The estimator is undefined for t greater than the largest observed event time t_k.
Intuition, building it step by step:
For t < t_1 (before any events): Ŝ(t) = 1
For t in [t_1, t_2): Ŝ(t) = (n_1 − d_1) / n_1
For t in [t_2, t_3): Ŝ(t) = [(n_2 − d_2) / n_2] · [(n_1 − d_1) / n_1]
Each additional event time multiplies in one more conditional survival factor
Censored observations between event times reduce the risk set for subsequent event times but do not themselves cause a drop in Ŝ(t). This is how the KM estimator correctly handles censoring.
Mean (restricted):
μ̂ = ∫ Ŝ(t) dt = Σ(i=1 to k) [T_(i) − T_(i−1)] · Ŝ(T_(i−1))
where T_(0) = 0 and Ŝ(0) = 1. This is the area under the step function Ŝ(t), computed as the sum of rectangle areas.
Median:
M̂ = inf{t : Ŝ(t) ≤ 0.5}
Read directly from the KM step function: the first time the survival estimate drops to or below 0.5.
Without censoring:
SE[Ŝ(t)] = √[Ŝ(t) · (1 − Ŝ(t)) / n]
This is just the standard error of a proportion.
With censoring (Greenwood formula):
SE[Ŝ(t)] = Ŝ(t) · √[Σ(j: t_j ≤ t) d_j / (n_j · (n_j − d_j))]
Each event time contributes a term d_j / [n_j · (n_j − d_j)] to the variance. The Greenwood formula reflects the accumulation of uncertainty through the sequential product.
The same 48-cord dataset (41 uncensored, 7 right-censored) was analysed using the KM estimator via SAS (proc lifetest) and R (survfit).
Key results from the KM estimator:
Ŝ_KM(53.1) = 0.6605
μ̂_KM = 54.1824
Q̂_KM(0.5) = 54.8 (median)
For comparison, the Weibull MLE gave:
Ŝ_W(53.1) = 0.6508
μ̂_W = 54.4630
Q̂_W(0.5) = 55.0069
The KM and Weibull estimates are very close, which supports the adequacy of the Weibull model for this data. When the parametric model is correctly specified, both approaches should yield similar results.
Quartile estimates from the KM estimator:
25th percentile: 52.4 (95% CI: 50.8 to 53.9)
50th percentile (median): 54.8 (95% CI: 53.6 to 56.1)
75th percentile: 57.1 (95% CI: 55.9 to 58.9)
Two groups of rats were exposed to the carcinogen DMBA. The event of interest is time to vaginal cancer mortality. Group 1 had 19 rats (2 censored) and Group 2 had 21 rats (2 censored).
KM estimates were computed separately for each group:
Group 1:
Median survival: 216 days (95% CI: 192, 234)
Mean (restricted): 218.8 days
25th percentile: 190 days
75th percentile: 234 days
Group 2:
Median survival: 233 days (95% CI: 232, 280)
Mean (restricted): 240.8 days
25th percentile: 232 days
75th percentile: 280 days
Group 2 appears to have somewhat longer survival times than Group 1 across the distribution. The KM curves for the two groups can be plotted on the same axes for visual comparison. Formal testing of the difference between two survival curves (e.g. the log-rank test) is covered in more advanced courses such as STAT 645/646.
SAS – KM estimator: proc lifetest with time T*C(1) syntax, where 1 indicates the censoring code. Use outsurv= to output the survival estimates. Use strata for group comparisons.
R – KM estimator: Use the survival package. Surv(time, status) creates the survival object, survfit() computes the KM estimator, and summary() prints the table. plot() on the survfit object produces the survival curve.
SAS – parametric (Weibull) MLE: proc lifereg with model T*C(1) = /dist=weibull. SAS reports the Weibull Scale (α) and Weibull Shape (γ) directly.
R – parametric (Weibull) MLE: survreg(Surv(time, status) ~ 1, dist='weibull'). Note: R's output uses a different parameterisation. The shape is γ = 1/Scale, and the scale is α = e^(Intercept).
KM estimator: Ŝ(t) = ∏(j: t_j ≤ t) [(n_j − d_j) / n_j]
Mean from KM curve: μ̂ = Σ(i=1 to k) [T_(i) − T_(i−1)] · Ŝ(T_(i−1))
Median from KM curve: M̂ = inf{t : Ŝ(t) ≤ 0.5}
Greenwood formula (SE with censoring): SE[Ŝ(t)] = Ŝ(t) · √[Σ(j: t_j ≤ t) d_j / (n_j · (n_j − d_j))]
⚠️ The KM estimator is a product of conditional survival probabilities, one for each event time. Censored observations reduce the risk set but do not cause a step down in the survival curve.
⚠️ Know the notation: d_j (events at time t_j), m_j (censorings in the interval), n_j (at risk just before t_j). Be able to compute Ŝ(t) by hand for a small dataset.
⚠️ The KM estimator is undefined beyond the largest observed event time. The restricted mean is computed only up to that point.
⚠️ When the parametric model is correct, the KM estimator and the parametric MLE will give similar results. Agreement between the two supports the model choice.
⚠️ R's survreg parameterises the Weibull differently from the standard form. To recover γ and α: γ = 1/Scale, α = e^(Intercept). This is a common source of confusion.
⚠️ The Greenwood formula accumulates variance contributions from each event time. Earlier events (where n_j is large) contribute less variance than later events (where n_j is small).
Q: What is the Kaplan-Meier estimator, and how does it handle censored observations?
A: The KM estimator is a nonparametric estimator of S(t) that takes the cumulative product of conditional survival fractions at each event time. Censored observations are handled by removing them from the risk set at the time of censoring, so they reduce n_j for subsequent event times but do not themselves cause a drop in the estimated survival.
Q: At time t_j, there are 20 units at risk and 2 events occur. What is the conditional survival factor for this time point?
A: (20 − 2) / 20 = 18/20 = 0.90.
Q: How is the mean estimated from the KM curve?
A: The mean is estimated as the area under the KM step function: μ̂ = Σ [T_(i) − T_(i−1)] · Ŝ(T_(i−1)), summed over the uncensored event times, with T_(0) = 0 and Ŝ(0) = 1.
Q: In the braided cord example, how do the KM and Weibull estimates of S(53) compare?
A: Ŝ_KM(53.1) = 0.6605 and Ŝ_W(53.1) = 0.6508. The two are very close (within about 1 percentage point), supporting the adequacy of the Weibull model.
Q: How do you convert R's survreg output to the standard Weibull parameterisation F(t) = 1 − e^(−(t/α)^γ)?
A: The shape parameter is γ = 1/Scale (where Scale is reported by R), and the scale parameter is α = e^(Intercept).
Q: What does the Greenwood formula estimate?
A: It estimates the standard error of the KM survival estimate Ŝ(t) at any time t. It accounts for the sequential product structure by summing variance contributions d_j / [n_j · (n_j − d_j)] over all event times up to t.
Kaplan-Meier estimator, product limit estimator, KM curve, nonparametric survival analysis, distribution-free estimation, risk set, Greenwood formula, standard error survival function, restricted mean, survival quantiles, median survival, comparing survival curves, log-rank test, proc lifetest SAS, survfit R, survival package R, survreg parameterisation, braided cord example, rat cancer example, STAT 645 646, STAT Handout 07