Source: Principles of Statistics I, Texas A&M University
Tags: censored data form, observed data CDF, likelihood function, MLE censored data, parametric estimation, exponential MLE, Weibull MLE, survival function, random censoring likelihood, Type I likelihood, Type II likelihood, Winsorized mean
When data is censored, the distribution of what we actually observe (Y_i) differs from the distribution of the underlying event times (T_i). The likelihood function must account for censoring: uncensored observations contribute the pdf, while censored observations contribute the survival function. Maximum likelihood estimation under censoring yields estimators that correctly account for incomplete data, unlike naive approaches that treat censored values as exact.
Survival function, S(t)
S(t) = P(T > t) = 1 − F(t), the probability that the event has not yet occurred by time t. Central to the analysis of censored data.
Observed data CDF, H
The cumulative distribution function of the data we actually observe (Y_i), as opposed to F, the CDF of the underlying event times (T_i). H depends on the type of censoring in the experiment.
Likelihood function for censored data
The product of contributions from each observation: uncensored units contribute f(T_i; θ), censored units contribute S(C_i; θ). Terms involving only the censoring distribution (not θ) factor out and do not affect the MLE.
Winsorized mean
An estimator where all extreme (censored) values are replaced with the largest observed (non-censored) value. Arises as the MLE of β under Type II censoring with an exponential model.
Naive estimator
An estimator that ignores censoring and treats all recorded values as exact observations. Systematically underestimates the true parameter because censored values are smaller than the true (unobserved) event times.
Let T_1, ..., T_n be the true event times, iid with CDF F. Let Y_1, ..., Y_n be what we actually observe, iid with CDF H.
No censoring: Y_i = T_i for all i, and H = F. Standard methods apply.
Type I censoring: The observed value is the event time if it occurs before t_c, otherwise we record t_c.
Y_i = T_i if T_i ≤ t_c, and Y_i = t_c if T_i > t_c
The CDF of Y_i, denoted H, has a jump (point mass) at t_c of height p = P(T_i > t_c) = 1 − F(t_c). This jump distinguishes the censored distribution from the original: H is a mixture of the continuous part of F (below t_c) and a discrete mass at t_c.
Type II censoring: Observe the first m ordered event times, then stop. For the remaining n − m units, record Y_i = T_(m).
Y_(i) = T_(i) for i = 1, ..., m, and Y_(i) = T_(m) for i = m+1, ..., n
The joint pdf uses order statistic properties:
f(y_1, ..., y_n) = C(n, m) · m! · f(y_1) · f(y_2) · ... · f(y_m) · [S(y_m)]^(n−m)
Random censoring: For each unit, we observe Y_i = min(T_i, C_i) together with the indicator δ_i = I(T_i ≤ C_i).
The joint pdf of (Y, δ) combines two cases:
If δ = 1 (uncensored): h(y, δ) = f(y) · [1 − G(y)]
If δ = 0 (censored): h(y, δ) = g(y) · S(y)
In a single expression: h(y, δ) = [f(y)]^δ · [1 − G(y)]^δ · [g(y)]^(1−δ) · [S(y)]^(1−δ)
Suppose F belongs to a known parametric family with parameter vector θ. To estimate θ using MLE, construct the likelihood and maximise it.
No censoring:
L(θ) = ∏ f(T_i; θ) over all i = 1, ..., n
Standard MLE.
Random censoring:
Each observation contributes either f (if uncensored) or S (if censored). Terms from the censoring distribution G that do not involve θ are collected into a factor K(Y_i, δ_i) and can be ignored for maximisation.
L(θ) = ∏(i ∈ U) f(T_i; θ) · ∏(i ∈ C) S(C_i; θ) · ∏ K(Y_i, δ_i)
where U is the set of uncensored indices and C is the set of censored indices.
Type I censoring:
A special case of random censoring with C_i = t_c for all i:
L(θ) = [S(t_c; θ)]^(n − n_U) · ∏(i ∈ U) f(T_i; θ) · ∏ K(Y_i, δ_i)
Type II censoring:
L(θ) = [n! / (n − m)!] · ∏(i=1 to m) f(T_(i); θ) · [S(T_(m); θ)]^(n − m) · ∏ K(Y_i, δ_i)
Let T_1, ..., T_n be iid exponential with CDF F(t; β) = 1 − e^(−t/β).
pdf: f(t; β) = (1/β) e^(−t/β)
Survival: S(t; β) = e^(−t/β)
Under random censoring with n_U uncensored observations, the log-likelihood (ignoring constant K terms) is:
l(β) = −n_U · log(β) − (1/β) · Σ Y_i
Setting the derivative to zero:
dl/dβ = −n_U / β + (1/β²) · Σ Y_i = 0
Solving gives:
β̂ = (1/n_U) · Σ Y_i = (Total time all n units were operating) / (Number of units that experienced the event)
This is a meaningful result. The numerator sums all observed times (both event times and censoring times), while the denominator counts only the units whose event was actually observed.
The same estimator applies to Type I censoring. For Type II censoring, the estimator becomes:
β̂ = (1/n_U) · [Σ(i=1 to n_U) T_(i) + (n − n_U) · T_(n_U)]
This is the Winsorized mean: replace each of the n − n_U unobserved values with the largest observed event time T_(n_U).
The naive estimator β̂* = (1/n) · Σ Y_i treats all n values as though they are exact event times. Because censored values satisfy C_i ≤ T_i, this estimator systematically underestimates β:
E[β̂*] ≤ E[β̂] = β
The correct MLE β̂ divides by n_U (the number of actual events) rather than n, which compensates for the fact that censored values are lower bounds, not true event times.
Let T_1, ..., T_n be iid Weibull with CDF F(t; γ, α) = 1 − e^(−(t/α)^γ).
pdf: f(t; γ, α) = (γ/α) · (t/α)^(γ−1) · e^(−(t/α)^γ)
Survival: S(t; γ, α) = e^(−(t/α)^γ)
Under Type I censoring with m uncensored observations, the log-likelihood is:
l(γ, α) = m·log(γ) − m·γ·log(α) + (γ−1)·Σ(i∈U) log(Y_i) − (1/α^γ)·Σ(i=1 to n) Y_i^γ + const
Setting partial derivatives to zero yields two equations:
α̂ = [(1/m) · Σ Y_i^γ̂]^(1/γ̂)
0 = 1/γ̂ + (1/m)·Σ(i∈U) log(Y_i) − [Σ Y_i^γ̂ · log(Y_i)] / [Σ Y_i^γ̂]
The second equation involves only γ̂ and must be solved numerically. Once γ̂ is obtained, α̂ follows from the first equation.
48 pieces of weathered braided cord were tested for strength. Seven cords were damaged during the study, producing right-censored values (the true strengths of those cords were higher than recorded). The goal was to estimate P(T > 53), the probability a cord exceeds strength 53.
A Weibull model was fit using SAS (proc lifereg) and R (survreg):
γ̂ (Weibull shape) = 16.2591
α̂ (Weibull scale) = 56.0223
Ŝ(53) = e^(−(53/56.0223)^16.2591) ≈ 0.667
So roughly 67% of cords are estimated to have strength of at least 53. This aligns well with the raw data: 28 of the 41 uncensored values exceed 53, giving 68.3%.
If censoring had been ignored (treating the 7 damaged cord values as exact), the MLEs would be γ̂ = 9.42, α̂ = 54.51, and Ŝ(53) = 0.464. The estimate drops from 67% to 46%, a substantial and misleading difference.
Maximum daily rainfall (mm) was recorded over 47 years in Sydney. Values above 2000 mm were considered unreliable due to instrument problems, creating right-censored data (7 values exceeded 2000).
Three analyses were compared:
Original data (no censoring adjustment): shape = 2.11, scale = 1537
Censored data (values > 2000 set to 2000 and flagged): shape = 2.60, scale = 1473
Deleted censored values (values > 2000 removed): shape = 3.04, scale = 1314
The impact on estimated quantiles was large, especially in the tails. For example, Q(0.99) ranged from 3170 (original) to 2652 (properly censored) to 2171 (deleted). Deleting the large values entirely compressed the right tail, underestimating extreme rainfall. Properly accounting for censoring preserved information about the tail behaviour.
Exponential MLE (random or Type I censoring): β̂ = Σ Y_i / n_U
Exponential MLE (Type II censoring, Winsorized mean): β̂ = [Σ(i=1 to m) T_(i) + (n − m) · T_(m)] / m
Weibull MLE (α, given γ̂): α̂ = [(1/m) · Σ Y_i^γ̂]^(1/γ̂)
Weibull survival function: S(t) = e^(−(t/α)^γ)
⚠️ In the likelihood for censored data, uncensored observations contribute the pdf f(·; θ) and censored observations contribute the survival function S(·; θ). This is the central principle.
⚠️ Terms involving only the censoring distribution G (not the parameter of interest θ) factor out and do not affect the MLE. They can be ignored when maximising.
⚠️ The naive estimator (dividing total time by n instead of n_U) underestimates the mean for an exponential model. Know why: censored values are lower bounds, not true event times.
⚠️ For the Weibull, the MLE equations require numerical solution. The equation for γ̂ is solved first (it involves only γ), and then α̂ is computed from a closed-form expression given γ̂.
⚠️ The braided cord example shows that ignoring censoring can severely distort estimates (67% vs 46% for Ŝ(53)). Be prepared to explain why.
Q: In the likelihood function for randomly censored data, what does an uncensored observation contribute, and what does a censored observation contribute?
A: An uncensored observation contributes f(T_i; θ), the pdf evaluated at the observed event time. A censored observation contributes S(C_i; θ), the survival function evaluated at the censoring time.
Q: For iid exponential data with random censoring, what is the MLE of the mean β?
A: β̂ = (Σ Y_i) / n_U, where Y_i = min(T_i, C_i) for all i, and n_U is the number of uncensored observations. In words: total operating time divided by the number of observed events.
Q: Why does the naive estimator β̂ = (1/n) · Σ Y_i underestimate β?*
A: Because censored values satisfy Y_i = C_i < T_i, so they are smaller than the true event times. Averaging all Y_i over n pulls the estimate below the true mean. The correct MLE compensates by dividing by n_U (the count of actual events) instead of n.
Q: What is the Winsorized mean in the context of Type II censored exponential data?
A: For Type II censoring, the MLE replaces each of the n − m unobserved event times with T_(m), the largest observed event time. The estimator is β̂ = [Σ(i=1 to m) T_(i) + (n − m) · T_(m)] / m.
Q: In the braided cord example, why did ignoring censoring change Ŝ(53) from approximately 0.67 to 0.46?
A: The 7 damaged cords had recorded strengths well below 53 (ranging from 26.8 to 42.5). Treating these as exact values dragged the estimated distribution to the left, reducing the estimated probability of exceeding 53. Properly accounting for censoring recognises that the true strengths of those cords were higher than recorded.
parametric estimation, MLE censored data, likelihood function censoring, survival function, exponential MLE, Weibull MLE, right censoring, random censoring likelihood, Type I censoring likelihood, Type II censoring likelihood, Winsorized mean, naive estimator bias, censored data CDF, proc lifereg SAS, survreg R, braided cord example, STAT Handout 07