Source: Walpole Ch. 3–4, Lecture Weeks 4–6
Tags: random variable, PMF, PDF, CDF, expectation, variance, standard deviation, Chebyshev theorem, joint distribution, STAT 3021, UMN
Difficulty: Intermediate | Prerequisites: Counting and Probability Fundamentals, Conditional Probability (Ch. 1-2 notes), basic integration
Random variables bridge the gap between abstract probability and the numerical calculations you need for statistics. Instead of talking about events like "the coin lands heads," you assign a number to each outcome and work with that number's distribution. This chapter introduces the machinery (PMFs, PDFs, CDFs, expected values, variance) that every later topic in the course relies on. Expectation and variance are the two most important summary measures of a distribution, and Chebyshev's theorem gives you a universal bound that works regardless of the distribution's shape. You will need basic integration for continuous distributions.
A random variable assigns a number to each outcome of an experiment. Its distribution (PMF for discrete, PDF for continuous) tells you how probability is spread across possible values. Expected value is the long-run average. Variance measures how spread out the values are. Chebyshev's theorem guarantees that at least 1 - 1/k² of the distribution lies within k standard deviations of the mean.
Random variable (RV)
A function that assigns a numerical value to each outcome in a sample space. Think of it as a rule that converts outcomes into numbers you can do maths with.
Discrete random variable
An RV that takes a finite or countably infinite number of values (e.g. 0, 1, 2, ...). In simple terms, you can list all possible values.
Continuous random variable
An RV that can take any value in an interval (e.g. all real numbers between 0 and 1). In simple terms, there are infinitely many possible values and you cannot list them.
Probability mass function (PMF)
For a discrete RV, f(x) = P(X = x). It gives the probability of each specific value. The PMF must satisfy f(x) ≥ 0 for all x, and the sum of all f(x) values equals 1.
Probability density function (PDF)
For a continuous RV, f(x) is a function such that P(a ≤ X ≤ b) = ∫ₐᵇ f(x) dx. The PDF itself is not a probability; the area under the curve between two points is. Must satisfy f(x) ≥ 0 and the total area under the curve equals 1.
Cumulative distribution function (CDF)
F(x) = P(X ≤ x). Works for both discrete and continuous RVs. It is non-decreasing, starts at 0, and approaches 1. Think of it as the running total of probability up to x.
Expected value (mean), E(X) or μ
The long-run average value of the random variable if you repeated the experiment indefinitely. For discrete: E(X) = Σ x × f(x). For continuous: E(X) = ∫ x × f(x) dx.
Variance, Var(X) or σ²
A measure of how spread out the distribution is around the mean. Var(X) = E[(X - μ)²] = E(X²) - [E(X)]². Larger variance means more spread.
Standard deviation, σ
The square root of the variance. It is in the same units as X, which makes it more interpretable than variance.
Chebyshev's theorem
For any distribution with mean μ and standard deviation σ, the probability that X falls within k standard deviations of the mean is at least 1 - 1/k². In simple terms, this is a universal guarantee that works regardless of the distribution's shape.
Joint distribution
The probability distribution of two or more random variables considered together. For discrete RVs, f(x, y) = P(X = x and Y = y). For continuous, it involves a joint PDF integrated over a region.
Defined by listing all possible values and their probabilities (the PMF)
PMF rules: every probability is non-negative, and all probabilities sum to 1
CDF for discrete RVs: F(x) = P(X ≤ x) = sum of f(t) for all t ≤ x; it is a step function
To find P(a < X ≤ b): F(b) - F(a)
Defined by a PDF f(x) where P(a ≤ X ≤ b) = ∫ₐᵇ f(x) dx
For any specific value c, P(X = c) = 0 (a single point has no area)
This means P(X ≤ c) = P(X < c) for continuous RVs
CDF: F(x) = ∫₋∞ˣ f(t) dt; it is a smooth, non-decreasing function
The PDF is the derivative of the CDF: f(x) = F'(x)
For two discrete RVs: f(x, y) = P(X = x, Y = y)
Marginal distribution of X: f_X(x) = Σ_y f(x, y) (sum over all y values)
For continuous: f_X(x) = ∫ f(x, y) dy
X and Y are independent if f(x, y) = f_X(x) × f_Y(y) for all x, y
Conditional distribution: f(y|x) = f(x, y) / f_X(x)
Discrete: E(X) = Σ x × f(x)
Continuous: E(X) = ∫ x × f(x) dx
E(X) is a weighted average of the possible values, weighted by their probabilities
Linearity: E(aX + b) = aE(X) + b
For a function g(X): E[g(X)] = Σ g(x) × f(x) or ∫ g(x) × f(x) dx
E(X + Y) = E(X) + E(Y), always (no independence required)
Var(X) = E[(X - μ)²] = E(X²) - [E(X)]²
The shortcut formula E(X²) - [E(X)]² is almost always easier to compute
Var(aX + b) = a² × Var(X) (the constant b drops out, and a gets squared)
If X and Y are independent: Var(X + Y) = Var(X) + Var(Y)
Standard deviation: σ = √Var(X)
P(|X - μ| < kσ) ≥ 1 - 1/k² for any k > 1
k = 2: at least 75% of observations lie within 2 standard deviations of the mean
k = 3: at least 88.9% lie within 3 standard deviations
This is a conservative bound; the actual percentage is often much higher for common distributions (e.g. for a normal distribution, 95% is within 2σ)
Useful precisely because it makes no assumptions about the shape of the distribution
E(X) = \sum_x x \cdot f(x) \quad \text{(discrete)} \qquad E(X) = \int_{-\infty}^{\infty} x \cdot f(x)\,dx \quad \text{(continuous)}\text{Var}(X) = E(X^2) - [E(X)]^2\text{Var}(aX + b) = a^2 \cdot \text{Var}(X)P(|X - \mu| < k\sigma) \geq 1 - \frac{1}{k^2} \quad \text{(Chebyshev's theorem, } k > 1\text{)}Expected value is the basis for decision-making under uncertainty. Insurance companies set premiums by computing E(payout). Variance tells an investor how risky a stock is: two stocks with the same expected return but different variances carry different levels of risk. Chebyshev's theorem is used in quality control when the exact distribution of a manufacturing measurement is unknown.
Students confuse PDF values with probabilities. For a continuous RV, f(x) can exceed 1; it is a density, not a probability. Only the area under the curve between two points is a probability.
Forgetting that P(X = c) = 0 for continuous distributions. This means it does not matter whether you write < or ≤ in continuous probability statements.
Using the variance formula Var(X) = E(X²) - [E(X)]² but forgetting to square E(X). A common arithmetic error.
Assuming Var(X + Y) = Var(X) + Var(Y) without checking independence. The additive rule for variance requires independence (or at least zero covariance).
⚠️ The shortcut variance formula E(X²) - [E(X)]² appears in nearly every calculation. Memorise and practise it.
⚠️ Chebyshev's theorem problems are a common exam question. Know how to plug in k and interpret the result.
⚠️ Expect a problem where you must compute E(X) and Var(X) from a given PMF or PDF table. Show every step.
True or False: For a continuous RV, P(X = 3) could be 0.5. (False. P(X = c) = 0 for any continuous RV.)
Fill in the blank: Var(X) = E(X²) - ___. ([E(X)]²)
True or False: If X has mean 10 and standard deviation 2, Chebyshev's theorem guarantees at least 75% of the distribution is between 6 and 14. (True. k = 2, and 1 - 1/4 = 0.75.)
Fill in the blank: E(3X + 5) = 3E(X) + ___. (5)
Q: A discrete RV X has the following PMF: P(X=1) = 0.2, P(X=2) = 0.3, P(X=3) = 0.5. Find E(X) and Var(X).
A: E(X) = 1(0.2) + 2(0.3) + 3(0.5) = 0.2 + 0.6 + 1.5 = 2.3. E(X²) = 1(0.2) + 4(0.3) + 9(0.5) = 0.2 + 1.2 + 4.5 = 5.9. Var(X) = 5.9 - (2.3)² = 5.9 - 5.29 = 0.61.
Q: A continuous RV has PDF f(x) = 2x for 0 < x < 1 and 0 otherwise. Find P(0.25 < X < 0.75).
A: P(0.25 < X < 0.75) = ∫₀.₂₅⁰·⁷⁵ 2x dx = [x²]₀.₂₅⁰·⁷⁵ = 0.75² - 0.25² = 0.5625 - 0.0625 = 0.5.
Q: A distribution has mean 50 and standard deviation 5. Using Chebyshev's theorem, find the minimum probability that a randomly selected value falls between 35 and 65.
A: The interval 35 to 65 is 50 ± 15, so k = 15/5 = 3. P(|X - 50| < 15) ≥ 1 - 1/9 ≈ 0.889, or about 88.9%.
The expected value and variance formulas you learn here are used to derive the mean and variance of specific named distributions (binomial, normal, Poisson) in Chapters 5 and 6. The CDF concept reappears centrally in sampling distributions and hypothesis testing. Joint distributions and independence of random variables are the foundation for understanding how sample statistics behave.
random variable, discrete, continuous, PMF, probability mass function, PDF, probability density function, CDF, cumulative distribution function, expected value, mean, variance, standard deviation, Chebyshev inequality, Chebyshev theorem, joint distribution, marginal distribution, conditional distribution, independence of RVs, E(X), Var(X), STAT 3021, Walpole Chapter 3, Walpole Chapter 4