Source: Analytical Statistics (T), NYU
When you apply a function to a random variable X to get a new variable Y = u(X), you need the distribution of Y. There are two main routes: the two-step method (find the c.d.f. of Y first, then differentiate) and the change-of-variables formula (a shortcut for one-to-one transformations). The probability integral transform is a special case that turns any continuous random variable into a uniform one.
Difficulty: Intermediate | Prerequisites: single-variable p.d.f.s and c.d.f.s, differentiation (chain rule), basic integration, joint distributions (Lecture 6).
In the previous lecture you learned to describe pairs of random variables with joint, marginal, and conditional distributions. This lecture answers a different question: if you already know the distribution of X, what happens when you transform it through a function u to get Y = u(X)? This is not a theoretical curiosity. Every time you square a variable, take a logarithm, or standardise a score, you are performing a transformation and need to know the resulting distribution. The tools here (the two-step method, the change-of-variables formula, the integral transformation) give you systematic ways to derive it rather than starting from scratch each time.
Transformation of a random variable, Y = u(X)
A new random variable created by applying a function u to an existing random variable X. Since X maps the sample space S to R, and u maps R to R, the composition Y = u(X) maps S to R. Think of it as: you run X through a machine (the function u) and out comes Y.
Two-step method
A general technique for finding the distribution of Y = u(X). Step 1: derive the c.d.f. F_Y(y) = P(Y ≤ y) = P(u(X) ≤ y) by working out which values of X satisfy u(X) ≤ y. Step 2: differentiate to get the p.d.f. In simple terms, find the probability that Y is below y, then take the derivative.
Change of variables formula
A shortcut for continuous, one-to-one transformations. If u is strictly increasing and differentiable on [a, b] with inverse s(y) = u⁻¹(y), the p.d.f. of Y is f_Y(y) = f_X(s(y)) · |ds/dy|. Think of it as: evaluate the old density at the pre-image, then correct for how the transformation stretches or compresses the axis.
Integral transformation (probability integral transform)
If X is a continuous random variable with c.d.f. F_X, then the random variable F_X(X) is uniformly distributed on [0, 1]. In simple terms, plugging a continuous random variable into its own c.d.f. always produces a Uniform(0, 1) variable. This is the basis of simulation methods.
Functions of two or more random variables
Extending transformations to the multivariate case. If X₁, ..., Xₙ are discrete with a known joint p.d.f., and Y₁, ..., Yₘ are defined by m functions of those X's, the joint p.d.f. of the Y's is found by summing the joint p.d.f. of the X's over all (x₁, ..., xₙ) that map to the given (y₁, ..., yₘ).
Setup: X is a random variable (maps sample space S → R). A function u: R → R produces a new random variable Y = u(X): S → R.
The two-step method works for any transformation, whether or not u is one-to-one.
Discrete case:
f_Y(y) = P(Y = y) = P(u(X) = y) = Σ over all x where u(x) = y of f_X(x).
If X is discrete and u is any function, Y is also discrete.
Example: If f_X(x) = 1/5 for x ∈ {−2, −1, 0, 1, 2} and Y = |X|, then:
f_Y(0) = f_X(0) = 1/5
f_Y(1) = f_X(−1) + f_X(1) = 2/5
f_Y(2) = f_X(−2) + f_X(2) = 2/5
Continuous case:
Step 1: Find the c.d.f. F_Y(y) = P(Y ≤ y) = P(u(X) ≤ y). Rewrite P(u(X) ≤ y) in terms of X and integrate using f_X.
Step 2: If Y is also continuous, differentiate: f_Y(y) = dF_Y(y)/dy.
Example: X has p.d.f. f_X(x) = 1/2 for −1 ≤ x ≤ 1. Let Y = X².
Y takes values in [0, 1].
F_Y(y) = P(X² ≤ y) = P(−√y ≤ X ≤ √y) = ∫ from −√y to √y (1/2) dx = √y.
f_Y(y) = d/dy (√y) = 1/(2√y) for y ∈ (0, 1].
Applies when u is strictly increasing (or strictly decreasing) and differentiable on the support of X.
Let s(y) = u⁻¹(y) be the inverse function.
The p.d.f. of Y is: f_Y(y) = f_X(s(y)) · |ds(y)/dy|, for y in the range of u.
The |ds/dy| factor is the Jacobian in one dimension. It accounts for how the transformation stretches or compresses probability.
Linear scaling example:
If Y = aX for some constant a > 0, then s(y) = y/a and ds/dy = 1/a.
F_Y(y) = P(aX ≤ y) = P(X ≤ y/a) = F_X(y/a).
f_Y(y) = (1/a) · f_X(y/a).
This confirms the intuition: scaling X by a stretches the distribution and reduces the height by 1/a.
Square-root example:
Let X ~ Uniform(0, 1), so f_X(x) = 1 for 0 ≤ x ≤ 1. Let Y = X².
u(x) = x² is strictly increasing on [0, 1]. Inverse: s(y) = √y. ds/dy = 1/(2√y).
f_Y(y) = f_X(√y) · 1/(2√y) = 1/(2√y) for 0 < y ≤ 1.
The key mechanical point: always express the derivative of the inverse, not the derivative of u itself.
Let X be a continuous random variable with c.d.f. F_X. Define Y = F_X(X).
Then Y ~ Uniform(0, 1). This holds regardless of the distribution of X.
Why it works: F_X is a non-decreasing function from R to [0, 1], and for a continuous X it is strictly increasing on the support. So F_X is a valid one-to-one transformation, and the two-step method (or change of variables) yields a uniform distribution.
This result is the theoretical foundation for simulation: to generate a draw from any continuous distribution, generate U ~ Uniform(0, 1) and apply the inverse c.d.f. F_X⁻¹(U). The result has the distribution of X.
Suppose X₁, ..., Xₙ are discrete with joint p.d.f. f_{X₁,...,Xₙ}(x₁, ..., xₙ).
Define m new variables through m functions: Y₁ = u₁(X₁, ..., Xₙ), ..., Yₘ = uₘ(X₁, ..., Xₙ).
Define the pre-image set A_y = {(x₁, ..., xₙ) : u₁(x₁, ..., xₙ) = y₁, ..., uₘ(x₁, ..., xₙ) = yₘ}.
The joint p.d.f. of Y₁, ..., Yₘ is: f_{Y₁,...,Yₘ}(y₁, ..., yₘ) = Σ over (x₁,...,xₙ) ∈ A_y of f_{X₁,...,Xₙ}(x₁, ..., xₙ).
For continuous variables with one-to-one transformations, the multivariate change-of-variables formula involves the Jacobian determinant (covered in a later lecture).
Two-step method (continuous): F_Y(y) = P(u(X) ≤ y), then f_Y(y) = dF_Y(y)/dy.
Two-step method (discrete): f_Y(y) = Σ over {x : u(x) = y} of f_X(x).
Change of variables (one-to-one, continuous): f_Y(y) = f_X(s(y)) · |ds(y)/dy|, where s = u⁻¹.
Linear transformation Y = aX: f_Y(y) = (1/a) · f_X(y/a).
Probability integral transform: If X is continuous with c.d.f. F_X, then F_X(X) ~ Uniform(0, 1).
Inverse transform sampling: If U ~ Uniform(0, 1), then X = F_X⁻¹(U) has c.d.f. F_X.
Multivariate (discrete): f_{Y₁,...,Yₘ}(y₁,...,yₘ) = Σ_{(x₁,...,xₙ) ∈ A_y} f_{X₁,...,Xₙ}(x₁,...,xₙ).
Students often apply the change-of-variables formula when the transformation is not one-to-one (e.g. Y = X²) without restricting to a monotone region. For non-injective transformations, use the two-step method or split the domain into monotone pieces.
A frequent error is writing f_Y(y) = f_X(u⁻¹(y)) and forgetting the |ds/dy| correction factor. The Jacobian term is not optional; without it the density does not integrate to 1.
Students sometimes confuse the derivative of u with the derivative of the inverse s. The formula uses ds/dy (the derivative of the inverse), not du/dx.
The probability integral transform only works for continuous random variables. For discrete variables, F_X(X) is not uniformly distributed.
⚠️ The two-step method for a continuous transformation (find F_Y, then differentiate) is a near-certainty on the exam. Practise both the c.d.f. step and the differentiation step.
⚠️ The change-of-variables formula for one-to-one functions is tested directly. Be ready to identify the inverse s(y), compute ds/dy, and plug into the formula.
⚠️ The probability integral transform is a favourite "show that" question: prove that F_X(X) ~ Uniform(0, 1).
⚠️ Expect at least one problem with a non-monotone transformation (like Y = X²) where you must use the two-step method rather than the change-of-variables shortcut.
True or False: The change-of-variables formula requires the transformation to be one-to-one.
Fill in the blank: In the change-of-variables formula, the correction factor is the absolute value of the derivative of the ______ function.
True or False: If X is continuous with c.d.f. F_X, then F_X(X) follows a Uniform(0, 1) distribution.
Fill in the blank: For a discrete X with Y = u(X), the p.d.f. of Y at y is obtained by ______ f_X(x) over all x such that u(x) = y.
True or False: If Y = aX where a > 0, then f_Y(y) = a · f_X(ay).
Answers: 1. True. 2. Inverse. 3. True. 4. Summing. 5. False; f_Y(y) = (1/a) · f_X(y/a).
Q: Let X ~ Uniform(0, 1). Find the p.d.f. of Y = −2 ln(X).
A: u(x) = −2 ln(x) is strictly decreasing on (0, 1). Inverse: s(y) = e^{−y/2}. ds/dy = −(1/2)e^{−y/2}. So f_Y(y) = f_X(e^{−y/2}) · (1/2)e^{−y/2} = (1/2)e^{−y/2} for y > 0. This is an Exponential(1/2) distribution.
Q: X has p.d.f. f_X(x) = 2x for 0 < x < 1. Use the two-step method to find the p.d.f. of Y = X².
A: F_Y(y) = P(X² ≤ y) = P(X ≤ √y) = ∫ from 0 to √y of 2x dx = y, for 0 < y < 1. Then f_Y(y) = d/dy(y) = 1 for 0 < y < 1. So Y ~ Uniform(0, 1).
Q: Prove that if X is continuous with c.d.f. F_X, then Y = F_X(X) ~ Uniform(0, 1).
A: F_Y(y) = P(F_X(X) ≤ y) = P(X ≤ F_X⁻¹(y)) = F_X(F_X⁻¹(y)) = y for 0 < y < 1. This is the c.d.f. of a Uniform(0, 1) distribution.
Q: X is discrete with f_X(x) = 1/3 for x ∈ {1, 2, 3}. Let Y = (X − 2)². Find f_Y(y).
A: u(1) = 1, u(2) = 0, u(3) = 1. So f_Y(0) = f_X(2) = 1/3, f_Y(1) = f_X(1) + f_X(3) = 2/3.
Q: When should you use the two-step method instead of the change-of-variables formula?
A: Use the two-step method whenever the transformation is not one-to-one on the support of X (e.g. Y = X², Y = |X|). The change-of-variables formula only applies to strictly monotone, differentiable transformations.
This topic builds directly on the joint distributions from Lecture 6: functions of two or more random variables require the joint p.d.f. as the starting point. The probability integral transform connects to simulation and computational statistics, where inverse-transform sampling is one of the standard tools. The multivariate change-of-variables formula (Jacobian method) extends these ideas and appears in later lectures. These transformation techniques also underpin the derivation of important named distributions (chi-squared from normal, F-distribution from chi-squared ratios, etc.).
Transformation of random variables, functions of random variables, Y = u(X), two-step method, CDF method, change of variables formula, Jacobian, inverse function, probability integral transform, inverse transform sampling, uniform distribution, one-to-one transformation, monotone transformation, functions of two random variables, multivariate transformation, pre-image set, analytical statistics, MATH 301, NYU