Difficulty: Intermediate | Prerequisites: Probability axioms, counting rules, conditional probability (see Part 1 notes)
Random variables turn outcomes into numbers, and distribution functions (PDFs and CDFs) describe how probability is spread across those numbers. This part of the course covers discrete and continuous random variables, how to move between PDFs and CDFs, joint and conditional distributions, and three methods for finding the distribution of a transformed variable. The binomial distribution is the key named distribution to know cold.
Random variable (X)
A function that maps each outcome in the sample space S to a real number. It gives a numerical label to each possible result of an experiment.
Think of it as: the experiment produces an outcome s; the random variable tells you "that outcome is worth this number."
Probability density function (PDF), f_X(x)
For a discrete random variable: f_X(x) = P(X = x), the probability that X takes the specific value x.
For a continuous random variable: f_X(x) is a density, and P(X ∈ A) = ∫_A f_X(t) dt. The density itself is not a probability; areas under it are.
Cumulative distribution function (CDF), F_X(x)
F_X(x) = P(X ≤ x). The probability that the random variable takes a value at or below x.
In simple terms, it is the running total of probability from the left.
Binomial distribution
The distribution of the number of successes X in N independent trials, each with success probability p. The PDF is f_X(x) = C(N, x) p^x (1-p)^(N-x).
Think of it as: flip a biased coin N times; X counts the heads.
Joint PDF, f_{XY}(x, y)
The probability density for two random variables considered together. For discrete variables, f_{XY}(x, y) = P(X = x and Y = y).
Marginal distribution
The distribution of one variable alone, obtained by summing (discrete) or integrating (continuous) the joint PDF over all values of the other variable. f_X(x) = ∫ f_{XY}(x, y) dy.
Conditional distribution
The distribution of Y given that X = x: f_{Y|X}(y|x) = f_{XY}(x, y) / f_X(x). The analogue of conditional probability, but for random variables.
Convolution formula
If X and Y are independent, the PDF of Z = X + Y is f_Z(z) = ∫ f_Y(z - w) f_X(w) dw. This is how you find the distribution of a sum of independent random variables.
Order statistics
Given i.i.d. random variables X₁, …, Xₙ, the kth smallest value Y_k is the kth order statistic. Its distribution depends on both the original CDF and PDF.
Change of variables formula
If Y = u(X) where X is continuous and u is one-to-one with inverse s, then f_Y(y) = f_X(s(y)) |ds/dy|. A shortcut that avoids the two-step CDF method.
A random variable X is a function from S to the real numbers ℝ. The probability function on S induces a probability distribution on ℝ through X:
P(X ∈ A) = P({s ∈ S : X(s) ∈ A})
This is the bridge between the abstract probability space and concrete numerical calculations.
The PDF captures where the probability mass sits.
Discrete case: f_X(x) = P(X = x). The PDF gives the probability of each specific value.
Continuous case: f_X(x) is a density. P(X ∈ A) = ∫_A f_X(t) dt. For continuous variables, the probability of any single point is zero; probability only comes from intervals.
F_X(x) = P(X ≤ x). The CDF tells you the total probability accumulated up to and including x.
This is the most important named discrete distribution for the midterm. It models the number of successes X in N independent trials, each with probability p.
f_X(x) = C(N, x) · p^x · (1 - p)^(N-x)
The C(N, x) term counts the number of distinct orderings of x successes among N trials. You should know this formula for the exam.
Discrete: sum up the PDF values. F_X(x) = Σ_{x_i ≤ x} f_X(x_i)
Continuous: integrate the PDF. F_X(x) = ∫_{-∞}^{x} f_X(t) dt
Discrete: the PDF at a point is the size of the jump in the CDF at that point. f_X(x) = F_X(x⁺) - F_X(x⁻)
Continuous: differentiate the CDF. f_X(x) = dF_X(x)/dx
Every CDF, regardless of whether X is discrete or continuous, satisfies these:
0 ≤ F_X(x) ≤ 1 for all x ∈ ℝ
F_X(x) is non-decreasing: if a < b, then F_X(a) ≤ F_X(b)
F_X(x) is right-continuous (continuous from the right)
F_X(x) is continuous everywhere if and only if X is a continuous random variable
These properties are useful for checking your work. If a CDF you have derived ever decreases or exceeds 1, something has gone wrong.
When you have two random variables X and Y, their joint PDF f_{XY}(x, y) describes the probability structure of both variables together.
Discrete: f_{XY}(x, y) = P(X = x and Y = y)
Continuous: P((X, Y) ∈ A) = ∫∫A f{XY}(x, y) dx dy
To get the distribution of X alone from the joint distribution, sum or integrate out Y:
f_X(x) = ∫{-∞}^{∞} f{XY}(x, y) dy
This collapses the two-dimensional distribution down to one dimension.
X and Y are independent if and only if their joint PDF factors into the product of the marginals:
f_{XY}(x, y) = f_X(x) · f_Y(y) for all (x, y) ∈ ℝ²
This is the random-variable analogue of the event-level independence condition P(A ∩ B) = P(A) P(B).
The conditional distribution of Y given X = x is:
f_{Y|X}(y|x) = f_{XY}(x, y) / f_X(x)
This is the random-variable analogue of conditional probability. You are slicing the joint distribution at a fixed value of X and renormalising.
Given a random variable X with known PDF f_X(x) and a function Y = u(X), the goal is to find f_Y(y). There are three methods, and which to use depends on whether X is discrete or continuous and whether u is one-to-one.
f_Y(y) = Σ_{x : u(x) = y} f_X(x)
You simply collect all the x-values that map to the same y and add up their probabilities.
Step 1: find the CDF of Y.
F_Y(y) = P(u(X) ≤ y) = ∫_{x : u(x) ≤ y} f_X(x) dx
Step 2: differentiate to get the PDF.
f_Y(y) = dF_Y(y)/dy
This method always works for continuous X, whether or not u is one-to-one.
If u is one-to-one with inverse function s (so x = s(y)), then:
f_Y(y) = f_X(s(y)) · |ds/dy|
The absolute value of the derivative of the inverse function acts as a scaling factor. This is a shortcut that avoids computing the CDF explicitly.
Convolution formula. If X and Y are independent and Z = X + Y, then f_Z(z) = ∫_{-∞}^{∞} f_Y(z - w) f_X(w) dw. Be careful with integration limits when the densities are zero outside certain ranges.
Integral transformation (probability integral transform). If X is continuous, then Y = F_X(X) is uniformly distributed on [0, 1]. This is a foundational result for simulation: to generate a draw from any continuous distribution, you start with a uniform random variable and apply the inverse CDF.
Order statistics. If X₁, …, Xₙ are i.i.d., the kth smallest value Y_k has PDF: f_{Y_k}(y) = k · C(n, k) · [F_X(y)]^(k-1) · [1 - F_X(y)]^(n-k) · f_X(y). This counts the ways to have exactly (k-1) values below y and (n-k) values above y, with one value landing exactly at y.
Binomial PDF
f_X(x) = \binom{N}{x} p^x (1-p)^{N-x}CDF from PDF (continuous)
F_X(x) = \int_{-\infty}^{x} f_X(t)\,dtPDF from CDF (continuous)
f_X(x) = \frac{d}{dx} F_X(x)Marginal PDF
f_X(x) = \int_{-\infty}^{\infty} f_{XY}(x,y)\,dyConditional PDF
f_{Y|X}(y|x) = \frac{f_{XY}(x,y)}{f_X(x)}Independence condition
f_{XY}(x,y) = f_X(x)\,f_Y(y) \quad \text{for all } (x,y)Change of variables (u one-to-one, inverse s)
f_Y(y) = f_X(s(y))\left|\frac{d}{dy}s(y)\right|Convolution
f_Z(z) = \int_{-\infty}^{\infty} f_Y(z-w)\,f_X(w)\,dwOrder statistics (kth smallest of n i.i.d.)
f_{Y_k}(y) = k\binom{n}{k} [F_X(y)]^{k-1}[1-F_X(y)]^{n-k} f_X(y)Treating the continuous PDF as a probability. For continuous X, f_X(x) is a density, not a probability. It can exceed 1. Probabilities come only from integrating the density over an interval. P(X = x) = 0 for any specific value.
Confusing CDF and PDF directions. Students sometimes differentiate when they should integrate, or vice versa. Remember: CDF is the integral of the PDF (PDF → CDF: integrate). PDF is the derivative of the CDF (CDF → PDF: differentiate).
Forgetting the Jacobian in change of variables. When applying f_Y(y) = f_X(s(y)) |ds/dy|, students often drop the |ds/dy| factor. Without it, f_Y does not integrate to 1 and is not a valid density.
Sloppy integration limits in convolution. When X and Y have densities that are zero outside bounded intervals (e.g. uniform on [0,1]), the limits of the convolution integral must reflect where both f_X(w) and f_Y(z - w) are simultaneously nonzero. Using -∞ to ∞ mechanically is fine only if you track where the product is zero.
⚠️ Know the binomial distribution PDF by heart and be able to explain where each term comes from.
⚠️ Be comfortable moving between PDFs and CDFs in both directions, for both discrete and continuous cases.
⚠️ For functions of random variables, expect to be asked to derive a new PDF. Identify the right method (discrete sum, two-step CDF, or change of variables) before computing.
⚠️ Joint, marginal, and conditional distributions are frequently tested. Be ready to extract a marginal from a joint PDF and compute conditional distributions.
⚠️ The integral transformation result (F_X(X) ~ Uniform) is a neat theoretical result worth understanding; it shows up in simulation contexts.
True or false: For a continuous random variable, P(X = 5) = 0.
Fill in the blank: To go from PDF to CDF for a continuous variable, you ______.
True or false: If f_{XY}(x, y) = f_X(x) · f_Y(y) for all (x, y), then X and Y are independent.
True or false: The change of variables formula works even when u is not one-to-one.
Fill in the blank: The CDF is always a ______ function of x.
Q: Let X ~ Binomial(n = 5, p = 0.3). What is P(X = 2)?
A: f_X(2) = C(5, 2) · (0.3)² · (0.7)³ = 10 · 0.09 · 0.343 = 0.3087.
Q: Suppose X is continuous with PDF f_X(x) = 2x for 0 ≤ x ≤ 1, and 0 otherwise. What is F_X(x) for 0 ≤ x ≤ 1?
A: F_X(x) = ∫₀ˣ 2t dt = t² evaluated from 0 to x = x². So F_X(x) = x² on [0, 1].
Q: X and Y have joint PDF f_{XY}(x, y) = 6(1 - y) for 0 ≤ x ≤ y ≤ 1, and 0 otherwise. Find the marginal PDF of Y.
A: f_Y(y) = ∫₀ʸ 6(1 - y) dx = 6(1 - y) · y for 0 ≤ y ≤ 1. The limits on x run from 0 to y because the joint density is defined on 0 ≤ x ≤ y.
Q: Let X be uniform on [0, 1] and Y = X². Find the PDF of Y using the two-step method.
A: Step 1. F_Y(y) = P(X² ≤ y) = P(X ≤ √y) = √y for 0 ≤ y ≤ 1 (since F_X(x) = x on [0, 1]). Step 2. f_Y(y) = d/dy (√y) = 1/(2√y) for 0 < y ≤ 1.
Q: X₁, X₂, X₃ are i.i.d. Uniform[0, 1]. What is the PDF of the maximum Y₃?
A: Using the order statistics formula with k = n = 3, F_X(y) = y, f_X(y) = 1: f_{Y₃}(y) = 3 · C(3, 3) · y² · 1 · 1 = 3y² for 0 ≤ y ≤ 1.
Random variables and their distributions are the language for everything that follows in the course. Expected values, variance, and moment-generating functions (likely covered after the midterm) are all computed from the PDF or CDF you learn to work with here.
The joint and conditional distribution machinery is the foundation for regression analysis: the conditional expectation E(Y|X) is the core object in any regression model.
Order statistics connect to topics in econometrics and auction theory (the distribution of the highest bid, or the second-highest bid, in an auction).
Related Terms / Search Tags
random variable, probability density function, PDF, probability mass function, PMF, cumulative distribution function, CDF, binomial distribution, discrete distribution, continuous distribution, joint PDF, marginal distribution, conditional distribution, independence of random variables, function of a random variable, change of variables, Jacobian, convolution, integral transformation, probability integral transform, order statistics, ECON-UA 20, analytical statistics, midterm 1 review