Source: Principles of Statistics I, Texas A&M University / Tamhane-Dunlop Ch. 2
Tags: population distribution, random variable, sample space, CDF, PMF, PDF, quantile function, discrete RV, continuous RV, mixture RV, cumulative distribution function, probability mass function, probability density function
A statistical population is the full collection of values we could observe under given conditions. A random variable is a function that maps outcomes from a random experiment to real numbers, and it can be discrete, continuous, or a mixture of both. Three functions fully describe any random variable: the CDF, the PMF or PDF, and the quantile function.
Statistical population
The collection of all possible items or units sharing one or more characteristics under specified experimental or observational conditions.
Process
A repeatable series of actions that produces an observable characteristic or measurement.
Hypothetical population
A population of values that does not exist at the start of an experiment and may never fully exist. Common in industrial and laboratory settings.
Random experiment
A procedure or operation whose outcome is uncertain and cannot be predicted in advance.
Sample space (S)
The collection of all possible outcomes of a random experiment.
Random variable (RV)
A function Y that maps every element of the sample space to a real number. Formally, Y: S → (−∞, ∞). Each outcome s in S gets a unique numerical value Y(s).
Discrete random variable
A random variable whose set of possible values is at most finite or countably infinite.
Continuous random variable
A random variable whose set of possible values is one or more intervals on the real line. The probability it takes any single specific value is 0.
Discrete-continuous mixture
A random variable whose possible values include both one or more intervals on the real line and a set of distinct point masses.
Cumulative distribution function (CDF)
F(y) = P[Y ≤ y] for all y. Maps (−∞, ∞) into [0, 1]. Gives the proportion of the population with values less than or equal to y.
Probability mass function (PMF)
For a discrete RV: f(y) = P[Y = y], the proportion of the population taking the value y exactly. Related to the CDF by F(y) = Σ f(t) for all t ≤ y.
Probability density function (PDF)
For a continuous RV: the function f such that f(y) ≥ 0 and F(y) = ∫ f(t) dt from −∞ to y. The PDF is the derivative of the CDF. It is not a probability and can exceed 1.
Quantile function (Q(u))
The inverse of the CDF. For u in (0, 1), Q(u) = inf{y : F(y) ≥ u}, i.e. the smallest y for which F(y) ≥ u. Also written Q(u) = F⁻¹(u) when F is strictly increasing and continuous.
A statistical population can be real (all light bulbs in a warehouse) or hypothetical (all workers who will ever attend a future training programme).
Industrial and lab experiments typically involve hypothetical populations because the full set of values does not yet exist.
A random variable assigns a number to each outcome in the sample space.
Example: expose 3 rats to a toxic chemical, observe survival. The sample space has 2³ = 8 elements. Defining N = number surviving gives N: S → {0, 1, 2, 3}.
Multiple RVs can be defined on the same experiment:
Time to failure of a light bulb, T: S → [0, 10000] (continuous)
Whether a bulb is defective, D: S → {0, 1} (discrete)
Quality grade of a bulb, Q: S → {0, 1, 2} (discrete)
Discrete RV
Possible values form a finite or countably infinite set. Examples: number of defective components, number of deer with ticks.
Continuous RV
Possible values span one or more intervals. Examples: water temperature, shell thickness of turtle eggs, survival time of fish.
Discrete-continuous mixture
Has both point masses and continuous intervals. Examples:
CPUE (catch per unit effort): 40% of nets catch zero fish (Y = 0), remaining 60% have Y in (0, 500)
Health insurance spending: 20% of employees have no insurance (X = 0), rest have X in ($1200, $5000)
Plant growth after stimulant: 43% show no growth (G = 0), rest have G in (0, 28) cm
Yearly insurance payout: 46% have no payout (P = 0), rest have P in (0, $1,000,000)
1. Cumulative distribution function (CDF)
F(y) = P[Y ≤ y], defined for all y on the real line
Always non-decreasing, right-continuous
Ranges from 0 to 1
2. PMF (discrete) or PDF (continuous)
For discrete RVs, f(y) = P[Y = y]. You recover the CDF by summing: F(y) = Σ f(t) for t ≤ y.
For continuous RVs, f(y) = dF(y)/dy. The PDF is the rate of increase in F at y. Important: f(y) is not a probability. For example, the exponential PDF with λ = 5 gives f(0.04) = 5e^(−0.2) ≈ 4.09, which exceeds 1.
For small Δ, the approximation Δ · f(y) ≈ P[Y ∈ (y, y + Δ)] holds.
You recover a discrete PMF from the CDF by subtraction: f(dⱼ) = F(dⱼ) − F(dⱼ₋₁).
3. Quantile function
For continuous, strictly increasing CDFs: Q(u) = yᵤ if and only if F(yᵤ) = u. This is simply the inverse of F.
For discrete or mixture RVs, the standard inverse may not be a valid function (flat regions in the CDF mean multiple y values map to the same F, and jumps in the CDF mean some u values have no y). The general definition resolves this: Q(u) = inf{y : F(y) ≥ u}.
Special boundary cases: if Y ≥ a, then Q(0) = a. If a ≤ Y ≤ b, then Q(0) = a and Q(1) = b.
Graphical relationship: jumps in the CDF become flat regions in Q, and flat regions in the CDF become jumps in Q. The quantile plot is a rotation of the mirror image of the CDF plot.
For Y with CDF F(y) = 1 − e^(−y) for y ≥ 0:
Set u = 1 − e^(−yᵤ) and solve: Q(u) = −log(1 − u)
Because Y ≥ 0, set Q(0) = 0
For u in (0, 1), the value yᵤ = Q(u) satisfies:
At least 100u% of population values are ≤ yᵤ
At least 100(1 − u)% of population values are ≥ yᵤ
CDF (continuous): F(y) = ∫ from −∞ to y of f(t) dt
PDF from CDF: f(y) = dF(y)/dy
CDF (discrete): F(y) = Σ f(t) for all t ≤ y
PMF from CDF (discrete): f(dⱼ) = F(dⱼ) − F(dⱼ₋₁)
Quantile (general): Q(u) = inf{y : F(y) ≥ u}
Small-interval approximation: P[Y ∈ (y, y + Δ)] ≈ Δ · f(y)
⚠️ The PDF is not a probability. It can exceed 1. This is a common exam trap.
⚠️ For discrete RVs, you recover the PMF from the CDF by subtraction, not differentiation.
⚠️ The quantile function for discrete/mixture RVs uses the infimum definition, not a simple algebraic inverse. Know both the simple inverse (continuous case) and the infimum definition (general case).
⚠️ Hypothetical populations come up in experiment design questions. If the population "doesn't exist yet," it is hypothetical.
⚠️ Recognise which type of RV a scenario describes (discrete, continuous, or mixture). Mixture examples often involve a point mass at zero plus a continuous interval.
Q: A PDF f(y) evaluates to 2.5 at a particular point. Is this valid?
A: Yes. The PDF is not a probability; it is a density. It can take values greater than 1 as long as the total area under the curve equals 1.
Q: For a discrete RV with CDF values F(2) = 0.54 and F(3) = 0.79, what is f(3)?
A: f(3) = F(3) − F(2) = 0.79 − 0.54 = 0.25.
Q: What is the quantile function Q(u) for an Exponential(1) distribution?
A: Q(u) = −log(1 − u) for u in (0, 1), with Q(0) = 0.
Q: A study examines workers who will attend a future training programme. Is the population real or hypothetical?
A: Hypothetical, because the full population of trained workers does not yet exist and may never fully exist.
Q: In a discrete-continuous mixture, 40% of observations are exactly zero and the rest follow a continuous distribution on (0, 500). What does the CDF look like at y = 0?
A: The CDF has a jump of size 0.40 at y = 0, then increases continuously for y > 0 up towards 1.
population, process, hypothetical population, random experiment, sample space, random variable, RV, discrete, continuous, mixture, CDF, cumulative distribution function, PMF, probability mass function, PDF, probability density function, quantile function, inverse CDF, percentile, infimum definition, point mass, density not probability, STAT 301, principles of statistics, Tamhane-Dunlop Chapter 2