Source: Principles of Statistics I, Texas A&M University / Tamhane-Dunlop Ch. 2
Tags: continuous distribution, normal, exponential, gamma, Weibull, beta, chi-squared, t-distribution, F-distribution, lognormal, Cauchy, logistic, Laplace, double exponential, Pareto, uniform, mixture distribution
The major continuous distributions (normal, exponential, gamma, Weibull, beta, chi-squared, t, F, lognormal, Cauchy, logistic, Laplace) each model different real-world phenomena and are connected through mathematical relationships. Mixture distributions combine multiple component distributions (discrete and/or continuous) to model populations that no single distribution can capture alone.
Uniform(a, b)
Constant density on the interval (a, b). PDF: f(x) = 1/(b − a). Mean = (a + b)/2. Variance = (b − a)²/12. Symmetric. Used in simulations.
Beta(α, β)
Generalisation of the Uniform on [0, 1]. PDF involves x^(α−1)(1−x)^(β−1). Can be symmetric, left-skewed, or right-skewed depending on α and β. Uniform when α = β = 1.
Normal(μ, σ²)
The Gaussian distribution. Symmetric, bell-shaped. PDF: (1/√(2πσ²)) exp(−(x−μ)²/(2σ²)). Mean = μ, Variance = σ². Central to statistics because of the Central Limit Theorem.
Lognormal(μ, σ²)
If Y ~ N(μ, σ²), then X = e^Y has a lognormal distribution. Right-skewed. Used for growth models, failure times, tumour sizes.
Exponential(β)
PDF: (1/β) e^(−x/β) for x ≥ 0. Mean = β, Variance = β². Has the memoryless property. Models constant failure rate processes. Special case of Gamma with α = 1.
Gamma(α, β)
PDF involves x^(α−1) e^(−x/β). Mean = αβ, Variance = αβ². Special cases: Exponential (α = 1), Chi-squared (α = ν/2, β = 2). Sum of α independent Exponential(β) RVs has Gamma(α, β) distribution (Erlang distribution).
Weibull(γ, β)
Generalisation of the Exponential to allow non-constant failure rates. If Y ~ Exp(β), then X = Y^(1/γ) ~ Weibull(γ, β). Exponential is the special case γ = 1. Belongs to the extreme value distribution family.
Chi-squared(ν)
Special case of Gamma with α = ν/2, β = 2. Right-skewed, becomes more symmetric as ν grows. If Z₁, …, Zₖ are i.i.d. N(0, 1), then Z₁² + … + Zₖ² ~ χ²(k). Parameter ν is called degrees of freedom.
Student's t(ν)
Symmetric with heavier tails than the normal. Converges to N(0, 1) as ν → ∞. If Z ~ N(0, 1) and W ~ χ²(ν) independently, then Z/√(W/ν) ~ t(ν). Used for financial data and inference with small samples.
F(ν₁, ν₂)
Right-skewed. If W₁ ~ χ²(ν₁) and W₂ ~ χ²(ν₂) independently, then (W₁/ν₁)/(W₂/ν₂) ~ F(ν₁, ν₂). Used in ANOVA and regression. If T ~ t(ν), then T² ~ F(1, ν).
Cauchy(θ₁, θ₂)
Symmetric with extremely heavy tails. Mean and variance do not exist. Standard Cauchy = t(1). If Z₁, Z₂ are independent N(0, 1), then Z₁/Z₂ ~ Cauchy(0, 1).
Double exponential / Laplace(θ₁, θ₂)
Symmetric with a sharp peak at the centre. Tails heavier than normal but lighter than Cauchy. PDF: (1/(2θ₂)) exp(−|x − θ₁|/θ₂). Variance = 2θ₂².
Logistic(μ, β)
Symmetric. Tail weight falls between the double exponential and the Cauchy. CDF has a closed form: F(x) = 1/(1 + e^(−(x−μ)/β)).
Pareto(α, β)
Heavy-tailed, right-skewed. Mean exists only for β > 1; variance only for β > 2. Used for income, city sizes, insurance claims.
Mixture distribution
A weighted combination of k component distributions: f(x) = Σ pᵢ fᵢ(x), where Σ pᵢ = 1. Models populations drawn from multiple sub-populations or processes.
Four symmetric distributions, ordered by tail weight from lightest to heaviest:
Double exponential (Laplace): sharpest peak, tails heavier than normal
Normal: moderate tails, the baseline reference
Logistic: tails between Laplace and Cauchy
Cauchy: heaviest tails, mean and variance undefined
The practical implication: if your data has more extreme values than the normal would predict, consider the t, logistic, or Cauchy families depending on how heavy the tails are.
Exponential
Constant hazard/failure rate: h(t) = 1/β for all t
Memoryless property: P[T ≥ t + y | T ≥ y] = P[T ≥ t]
Most real devices do not have constant failure rates over their entire lifetime, but may over a limited range
Connected to Poisson: if events occur at rate λ, the time between events is Exp(1/λ)
Gamma
Time to the k-th event in a Poisson process with rate λ: Gamma(k, 1/λ)
Sum of k independent Exp(β) RVs is Gamma(k, β) (the Erlang distribution)
Shape parameter α controls skewness; becomes more symmetric as α increases
Scale parameter β stretches the distribution
Weibull
Generalises the Exponential by allowing the failure rate to change over time
Failure rate is proportional to t^(γ−1): increasing if γ > 1 (wear-out), constant if γ = 1 (Exponential), decreasing if γ < 1 (infant mortality/burn-in)
Alternative parameterisation: f(y | γ, α) = (γ/α)(y/α)^(γ−1) e^(−(y/α)^γ), where β = α^γ. R and SAS use this form.
Part of the extreme value distribution family (modelling minimums or maximums in samples)
Lognormal
If log(X) is normal, then X is lognormal
Right-skewed; the degree of skew increases with σ
Used for growth rates (plants, tumours), failure times, financial returns
Chi-squared
Sum of squared standard normals
Right-skewed for small ν, approaching symmetry as ν grows
Fundamental to hypothesis testing (goodness-of-fit, independence tests)
Uniform(a, b)
Flat density, all values in (a, b) equally likely
Foundation for simulation: generate Uniform(0, 1) values and transform them
Beta(α, β)
Flexible shape on [0, 1]: U-shaped (α, β < 1), uniform (α = β = 1), bell-shaped (α, β > 1), skewed (unequal α, β)
Model for proportions (e.g. proportion of defective items across plants)
To get a Beta on [a, b]: let Z ~ Beta(α, β) on [0, 1], then Y = (b − a)Z + a
These connections are frequently tested:
Exponential is Gamma(1, β) and Weibull(1, β)
Chi-squared(ν) is Gamma(ν/2, 2)
t(1) = Cauchy(0, 1)
t² with ν df = F(1, ν)
If Y ~ Exp(β), then Y^(1/γ) ~ Weibull(γ, β)
If Y ~ N(μ, σ²), then e^Y ~ Lognormal(μ, σ²)
Normal is the limit of t as ν → ∞
If Y ~ Gamma(α, β), then √(Y/β) ~ Maxwell distribution
Uniform(0, 1) is Beta(1, 1)
F(ν₁, ν₂) = (χ²(ν₁)/ν₁) / (χ²(ν₂)/ν₂)
When to use a mixture
When a single distribution cannot adequately model the data because the population is a blend of sub-populations or processes.
Continuous mixture (multiple sub-populations)
Example: a central warehouse receives tyres from 5 factories. Each factory's output follows N(μᵢ, σᵢ²). The overall distribution in the warehouse is the mixture:
f(x) = Σ pᵢ fᵢ(x), where pᵢ is the proportion from factory i
Discrete-continuous mixture (point mass plus continuous)
When some proportion p of observations take a fixed value (often zero) and the rest follow a continuous distribution F*:
F(y) = p · I(y ≥ 0) + (1 − p) · F*(y)
Example: CPUE for shrimp nets. 20% of nets catch zero fish (Y = 0), the remaining 80% have Y following a lognormal distribution. The CDF is:
F(y) = 0.2 · I(y ≥ 0) + 0.8 · F*(y)
Visual signature of mixtures
Two normal components with well-separated means produce a bimodal density
Two components with close means and different variances produce a unimodal but heavy-tailed density
Mixture proportions control the relative heights of modes
Normal PDF: f(x) = (1/√(2πσ²)) exp(−(x − μ)²/(2σ²))
Exponential PDF: f(x) = (1/β) e^(−x/β), x ≥ 0
Gamma PDF: f(x) = [1/(Γ(α)β^α)] x^(α−1) e^(−x/β), x ≥ 0
Weibull PDF (R/SAS form): f(y) = (γ/α)(y/α)^(γ−1) e^(−(y/α)^γ), y ≥ 0
Beta PDF: f(x) = [1/B(α, β)] x^(α−1)(1−x)^(β−1), 0 ≤ x ≤ 1
Chi-squared PDF: f(x) = [1/(Γ(p/2) 2^(p/2))] x^(p/2 − 1) e^(−x/2), x ≥ 0
Mixture CDF: F(x) = Σ pᵢ Fᵢ(x), with Σ pᵢ = 1
Mixture with point mass at zero: F(y) = p · I(y ≥ 0) + (1 − p) · F*(y)
⚠️ Know the special-case chains: Exponential → Gamma → Chi-squared; Exponential → Weibull; Normal → Lognormal; Normal → Chi-squared → t → F.
⚠️ The Cauchy distribution has no mean and no variance. This is a classic exam fact.
⚠️ Weibull parameterisation differs between textbooks and software. R and SAS use (shape = γ, scale = α) where β = α^γ. Always check which form is being used.
⚠️ Memoryless property: among continuous distributions, only the Exponential has it. Among discrete distributions, only the Geometric.
⚠️ Mixture distributions: know how to write the CDF and PDF for both continuous-continuous mixtures and discrete-continuous mixtures (point mass plus continuous).
⚠️ For the Weibull, γ > 1 means increasing failure rate (wear-out), γ < 1 means decreasing failure rate (burn-in), γ = 1 is the Exponential (constant rate).
Q: If Z₁, Z₂, Z₃ are independent N(0, 1) random variables, what is the distribution of Z₁² + Z₂² + Z₃²?
A: Chi-squared with 3 degrees of freedom, i.e. χ²(3).
Q: What is the relationship between the t-distribution with 1 degree of freedom and the Cauchy distribution?
A: They are the same distribution. t(1) is the standard Cauchy, with location 0 and scale 1.
Q: A Weibull distribution has shape parameter γ = 2. Is the failure rate increasing, decreasing, or constant?
A: Increasing, because γ > 1. This models wear-out failure.
Q: A warehouse receives items from 3 factories with proportions p₁ = 0.5, p₂ = 0.3, p₃ = 0.2 and each factory's output follows a different normal distribution. Write the mixture PDF.
A: f(x) = 0.5 f₁(x) + 0.3 f₂(x) + 0.2 f₃(x), where fᵢ is the normal PDF for factory i.
Q: Which continuous distribution has the memoryless property?
A: The Exponential distribution. It is the only continuous distribution with this property.
Q: T has a t-distribution with ν degrees of freedom. What is the distribution of T²?
A: F(1, ν), the F-distribution with 1 and ν degrees of freedom.
normal distribution, Gaussian, exponential distribution, memoryless, gamma distribution, Erlang, Weibull distribution, extreme value, beta distribution, chi-squared, degrees of freedom, Student t, F-distribution, ANOVA, lognormal, Cauchy, heavy tails, double exponential, Laplace, logistic distribution, Pareto, uniform distribution, mixture distribution, bimodal, point mass, discrete-continuous mixture, failure rate, wear-out, burn-in, Central Limit Theorem, STAT 301, principles of statistics