Difficulty: Intermediate | Prerequisites: Basic probability theory, p.d.f.s and c.d.f.s for discrete and continuous random variables, calculus (differentiation and integration).
This topic sits at the heart of Analytical Statistics. Once you know how a single random variable behaves, the next question is: what happens when you apply a function to it? Transformations of random variables let you derive the distribution of Y = u(X) from a known distribution of X. The techniques here (the two-step method, the change-of-variables formula) reappear constantly in later work on moment-generating functions, order statistics, and Bayesian inference. You need to be comfortable differentiating and integrating densities before diving in.
TL;DR
If you know the distribution of X and you define Y = u(X), you can find the distribution of Y either by working through the c.d.f. (the two-step method) or, for one-to-one transformations, by applying the change-of-variables formula directly. The discrete case sums probabilities; the continuous case integrates densities and differentiates. The same logic extends to functions of multiple random variables using Jacobian determinants.
Random variable (X)
A mapping from the sample space S into the real numbers. In simple terms, it assigns a number to every possible outcome of an experiment.
Transformation / function of a random variable (Y = u(X))
A new random variable created by applying a function u to X. The composition Y = u ∘ X maps S → ℝ → ℝ. Think of it as: you take whatever X gives you and run it through a formula to get Y.
Probability density function (p.d.f.), fₓ(x)
For a continuous random variable, the function whose integral over an interval gives the probability that X falls in that interval. For a discrete variable, often called the probability mass function (p.m.f.).
Cumulative distribution function (c.d.f.), Fₓ(x)
The probability that X takes a value less than or equal to x: Fₓ(x) = P(X ≤ x). The c.d.f. always exists, whether X is discrete or continuous.
Two-step method
A procedure for finding the distribution of Y = u(X): first derive the c.d.f. of Y by summing or integrating, then (if Y is continuous) differentiate to get the p.d.f. Think of it as: find P(Y ≤ y) in terms of X, then take the derivative.
Change of variables formula
A direct formula for the p.d.f. of Y when u is one-to-one and differentiable. It avoids going through the c.d.f. step by step. In simple terms, you plug in the inverse function and multiply by the absolute value of its derivative.
Inverse function, s(y) = u⁻¹(y)
The function that reverses u: if y = u(x), then x = s(y). Required for the change-of-variables formula to work directly.
Jacobian determinant, det(J)
In the multivariate case, the determinant of the matrix of partial derivatives of the inverse transformation. It plays the same role as |ds/dy| in the single-variable formula, scaling the density to account for how the transformation stretches or compresses space.
Integral transformation (probability integral transform)
The result that if X is a continuous random variable with c.d.f. Fₓ, then Fₓ(X) ~ Uniform(0, 1). Think of it as: feeding a random variable through its own c.d.f. always gives you a uniform distribution.
X is a random variable: a mapping from the sample space S to ℝ.
u is a function from ℝ to ℝ.
Y = u(X) = u ∘ X is the transformed random variable, mapping S → ℝ.
The goal: given the known distribution of X, find the distribution of Y.
The idea is always the same: express the c.d.f. of Y in terms of the distribution of X, then differentiate if Y is continuous.
When X is discrete with p.m.f. fₓ(x), Y = u(X) is also discrete. You find its p.m.f. by collecting all x values that map to the same y:
fᵧ(y) = P(Y = y) = P(u(X) = y) = Σ fₓ(x), summed over all x such that u(x) = y.
Worked example (discrete): Suppose fₓ(x) = 1/5 for x ∈ {−2, −1, 0, 1, 2} and 0 otherwise. Let Y = g(X) = |X|.
fᵧ(0) = fₓ(0) = 1/5
fᵧ(1) = fₓ(−1) + fₓ(1) = 2/5
fᵧ(2) = fₓ(−2) + fₓ(2) = 2/5
fᵧ(y) = 0 for all other y.
Notice that two x-values collapse onto each positive y-value, so the probabilities double there.
When X is continuous with p.d.f. fₓ(x) and Y = u(X), work through the c.d.f.:
Step 1: Find the c.d.f. of Y: Fᵧ(y) = P(Y ≤ y) = P(u(X) ≤ y) = ∫ fₓ(x) dx, integrated over all x such that u(x) ≤ y.
Step 2: Differentiate to get the p.d.f.: fᵧ(y) = d/dy Fᵧ(y).
Worked example (continuous): Suppose fₓ(x) = 1/2 for −1 ≤ x ≤ 1, and 0 otherwise. Let Y = X².
The range of X is [−1, 1], so Y ∈ [0, 1].
For y ∈ [0, 1]: Fᵧ(y) = P(X² ≤ y) = P(−√y ≤ X ≤ √y) = ∫ from −√y to √y of (1/2) dx = √y.
Differentiate: fᵧ(y) = d/dy (√y) = 1/(2√y) for y ∈ (0, 1], and 0 otherwise.
The density blows up near zero because X² compresses values near zero into a small range, piling up probability there.
X is a continuous random variable with known density fₓ(x), supported on an interval [a, b] where P(a ≤ X ≤ b) = 1. The function u(·) is strictly increasing (or strictly decreasing) and differentiable on [a, b], so it is one-to-one. Its inverse is s(y) = u⁻¹(y).
The p.d.f. of Y = u(X) is:
fᵧ(y) = fₓ(s(y)) · |ds/dy| for u(a) ≤ y ≤ u(b), and 0 otherwise.
The absolute value of ds/dy handles both increasing and decreasing transformations. This is the single-variable change-of-variables formula.
Consider the simple case u(x) = ax for a > 0:
Y = aX, so Fᵧ(y) = P(aX ≤ y) = P(X ≤ y/a) = Fₓ(y/a).
Differentiate using the chain rule: fᵧ(y) = (1/a) · fₓ(y/a).
The 1/a factor is exactly |ds/dy| where s(y) = y/a. The chain rule is doing the work, and the general formula extends this to any one-to-one differentiable u.
Let X ~ Uniform(0, 1), so fₓ(x) = 1 for 0 ≤ x ≤ 1.
u(x) = x² is strictly increasing on [0, 1], so the formula applies.
Inverse: s(y) = √y.
Derivative: ds/dy = 1/(2√y).
fᵧ(y) = fₓ(√y) · |1/(2√y)| = 1 · 1/(2√y) = 1/(2√y) for 0 < y ≤ 1, and 0 otherwise.
This matches the two-step method result, confirming both approaches.
Let X be continuous with c.d.f. Fₓ. Then Y = Fₓ(X) ~ Uniform(0, 1).
This is a powerful result. It says that feeding any continuous random variable through its own c.d.f. always produces a uniform distribution. The converse is equally useful: if U ~ Uniform(0, 1), then X = Fₓ⁻¹(U) has c.d.f. Fₓ. This is the basis for generating random samples from any distribution using a uniform random number generator.
Discrete two-step method: fᵧ(y) = Σ fₓ(x) where the sum runs over all x such that u(x) = y.
Continuous two-step method: Fᵧ(y) = P(Y ≤ y) = ∫ fₓ(x) dx over {x : u(x) ≤ y}, then fᵧ(y) = d/dy Fᵧ(y).
Change of variables (one-to-one, single variable): fᵧ(y) = fₓ(s(y)) · |ds/dy| where s = u⁻¹, valid on the range of Y.
Linear transformation Y = aX (a > 0): fᵧ(y) = (1/a) · fₓ(y/a).
Probability integral transform: If X is continuous with c.d.f. Fₓ, then Fₓ(X) ~ Uniform(0, 1).
Multivariate change of variables (preview for Part 2): fᵧ₁,...,ᵧₙ(y₁,...,yₙ) = fₓ₁,...,ₓₙ(s₁(y),...,sₙ(y)) · |det(J)| where J is the Jacobian matrix of the inverse transformation.
Linear transformation Y = AX (matrix form): fᵧ(y) = (1/|det(A)|) · fₓ(A⁻¹y).
Students often assume that if X is continuous, Y = u(X) must also be continuous. It does not have to be. A many-to-one function like Y = ⌊X⌋ (the floor function) can turn a continuous X into a discrete Y.
Students frequently forget the |ds/dy| factor in the change-of-variables formula. Without it, the resulting density will not integrate to 1. The Jacobian (or its single-variable equivalent) is not optional; it corrects for how the transformation stretches or compresses the probability.
A common error is applying the change-of-variables formula when u is not one-to-one. The formula requires a unique inverse. If u is many-to-one (e.g. Y = X²), you must either use the two-step method or split the domain of X into regions where u is one-to-one and sum the contributions.
Students sometimes confuse the p.d.f. and c.d.f. when writing down the two-step method. Remember: step 1 gives the c.d.f. Fᵧ(y), and you must differentiate to get fᵧ(y). Writing fᵧ(y) = P(Y ≤ y) is a notational error that will lose marks.
⚠️ The two-step method is the universal fallback. Even when the change-of-variables formula does not apply (non-monotone u, piecewise functions), the two-step method always works. Exam problems often test whether you can recognise which approach to use.
⚠️ The probability integral transform is a favourite exam topic. You should be able to prove that Fₓ(X) ~ Uniform(0, 1) and explain how this is used in simulation (inverse transform sampling).
⚠️ Deriving the density of Y = X² from a symmetric distribution (like Uniform(−1, 1)) is a classic problem. Make sure you can handle the fact that X² is not one-to-one on (−1, 1) by splitting the integral or using the two-step method carefully.
⚠️ Always check the support of Y. A common source of lost marks is writing the correct formula but forgetting to state the range of y for which the density is non-zero.
Real-world applications: Transformations of random variables are how statisticians derive the distributions of test statistics (e.g. chi-squared from squared normals). Engineers use them to model non-linear sensor outputs, and the probability integral transform underpins essentially every Monte Carlo simulation.
True or False: If X is continuous and Y = u(X), then Y is always continuous.
Answer: False. Y can be discrete if u is a many-to-one function (e.g. the floor function).
Fill in the blank: In the change-of-variables formula, fᵧ(y) = fₓ(s(y)) · ______.
Answer: |ds/dy| (the absolute value of the derivative of the inverse function).
True or False: The change-of-variables formula works for any transformation u, whether or not u is one-to-one.
Answer: False. It requires u to be one-to-one (and differentiable) on the support of X.
Fill in the blank: If X is continuous with c.d.f. Fₓ, then Fₓ(X) follows a ______ distribution.
Answer: Uniform(0, 1).
True or False: In the two-step method for the discrete case, you sum fₓ(x) over all x where u(x) ≤ y.
Answer: False. For the discrete p.m.f. of Y, you sum over all x where u(x) = y (not ≤ y). The ≤ y version gives the c.d.f., not the p.m.f.
Q: Let X have p.d.f. fₓ(x) = 2x for 0 ≤ x ≤ 1, and 0 otherwise. Find the p.d.f. of Y = X².
A: Since u(x) = x² is strictly increasing on [0, 1], use the change-of-variables formula. The inverse is s(y) = √y, and ds/dy = 1/(2√y). So fᵧ(y) = fₓ(√y) · 1/(2√y) = 2√y · 1/(2√y) = 1 for 0 ≤ y ≤ 1. Y is Uniform(0, 1).
Q: Suppose X ~ Uniform(0, 1). Use the two-step method to find the p.d.f. of Y = -ln(X).
A: For y > 0: Fᵧ(y) = P(−ln(X) ≤ y) = P(ln(X) ≥ −y) = P(X ≥ e⁻ʸ) = 1 − e⁻ʸ. Differentiate: fᵧ(y) = e⁻ʸ for y > 0. This is an Exponential(1) distribution.
Q: Explain why the probability integral transform guarantees that Fₓ(X) ~ Uniform(0, 1) for any continuous X.
A: Let Y = Fₓ(X). For 0 ≤ y ≤ 1: P(Y ≤ y) = P(Fₓ(X) ≤ y) = P(X ≤ Fₓ⁻¹(y)) = Fₓ(Fₓ⁻¹(y)) = y. This is the c.d.f. of Uniform(0, 1), so Y ~ Uniform(0, 1).
Q: Let X have p.m.f. fₓ(x) = 1/4 for x ∈ {1, 2, 3, 4}. Find the p.m.f. of Y = (X − 2)².
A: Compute Y for each x: Y(1) = 1, Y(2) = 0, Y(3) = 1, Y(4) = 4. So fᵧ(0) = 1/4, fᵧ(1) = fₓ(1) + fₓ(3) = 1/2, fᵧ(4) = 1/4.
This material connects directly to moment-generating functions, because if you know how to transform X into Y = e^(tX), you can derive the m.g.f. from the density of X. The techniques here also underpin the derivation of the chi-squared, t, and F distributions from normal random variables, which you will meet in inferential statistics. The probability integral transform connects to simulation methods and to copula theory in multivariate statistics.
Transformation of random variables, change of variables, Jacobian, probability integral transform, inverse transform sampling, c.d.f. method, two-step method, functions of random variables, derived distributions, monotone transformation, one-to-one transformation, p.d.f. of a function of a random variable, NYU Analytical Statistics, random variable mapping, density transformation, support of a transformed variable.