Difficulty: Intermediate | Prerequisites: single-variable PDF and PMF basics (Chapter 3 notes)
Joint distributions describe the probability behaviour of two (or more) random variables together. If you have worked with single-variable PDFs and PMFs, this is the natural next step: instead of asking "what values does X take?", you ask "what values do X and Y take at the same time?" Everything that follows, including marginals, conditionals, and independence, builds directly on that idea. This material sits at the heart of multivariate probability and comes back repeatedly in regression, Bayesian inference, and statistical modelling.
A joint distribution tells you the probability of X and Y taking particular values simultaneously. Marginal distributions recover the behaviour of one variable by integrating (continuous) or summing (discrete) the other out. Two variables are independent precisely when their joint distribution factors into the product of their marginals, and a conditional distribution describes one variable given a fixed value of the other.
Joint probability density function (joint PDF)
A function f(x, y) that gives the probability density for continuous random variables X and Y over a region in the xy-plane. The total volume under f(x, y) over the entire support equals 1. Think of it as the continuous-variable version of a joint probability table.
Joint probability mass function (joint PMF)
A function p(x, y) = P(X = x, Y = y) that gives the probability that discrete random variables X and Y take specific values simultaneously. All entries in the table must be non-negative and sum to 1. In simple terms, this is the probability table you read off directly.
Marginal density function (marginal PDF)
The PDF of a single variable obtained by integrating the joint PDF over all values of the other variable. For X: f_X(x) = integral of f(x, y) dy over the support of Y. Think of it as "collapsing" the joint distribution down to one variable.
Marginal distribution (discrete)
The PMF of a single variable obtained by summing the joint PMF over all values of the other variable. For X: p_X(x) = sum of p(x, y) over all y. In simple terms, you add across a row or down a column of the joint probability table.
Conditional density function
The density of Y given that X takes a specific value x, defined as f(y | x) = f(x, y) / f_X(x), provided f_X(x) > 0. Think of it as slicing the joint distribution at a fixed x and renormalising so the slice integrates to 1.
Independence of random variables
X and Y are independent if and only if f(x, y) = f_X(x) * f_Y(y) for all x and y (continuous case), or p(x, y) = p_X(x) * p_Y(y) for all x and y (discrete case). In simple terms, knowing the value of one variable tells you nothing new about the other.
Given a joint PDF f(x, y) defined over a region, you find the marginal PDF of one variable by integrating the joint PDF over the full range of the other variable.
Marginal of X: integrate f(x, y) with respect to y, over y's support
Marginal of Y: integrate f(x, y) with respect to x, over x's support
Worked example: f(x, y) = (3x - y)/9 for 1 < x < 3, 1 < y < 2.
f_X(x) = integral from 1 to 2 of (3x - y)/9 dy = (1/9)[3xy - y^2/2] evaluated from y = 1 to y = 2 = (1/9)(3x - 3/2) = (6x - 3)/18 = (2x - 1)/6, for 1 < x < 3.
f_Y(y) = integral from 1 to 3 of (3x - y)/9 dx = (1/9)[3x^2/2 - xy] evaluated from x = 1 to x = 3 = (1/9)(12 - 2y) = (12 - 2y)/9, for 1 < y < 2.
Given a joint probability table, you find the marginal distribution of one variable by summing across the appropriate dimension.
Marginal of X: for each value of x, sum p(x, y) over all values of y (sum down each column)
Marginal of Y: for each value of y, sum p(x, y) over all values of x (sum across each row)
Worked example: using the joint table with X in {1, 2, 3} and Y in {1, 3, 5}.
p_X(1) = 0.05 + 0.05 + 0 = 0.10
p_X(2) = 0.05 + 0.10 + 0.20 = 0.35
p_X(3) = 0.10 + 0.35 + 0.10 = 0.55
p_Y(1) = 0.05 + 0.05 + 0.10 = 0.20
p_Y(3) = 0.05 + 0.10 + 0.35 = 0.50
p_Y(5) = 0 + 0.20 + 0.10 = 0.30
To check whether X and Y are independent, verify whether f(x, y) = f_X(x) * f_Y(y) holds for every point in the support (continuous), or p(x, y) = p_X(x) * p_Y(y) for every cell (discrete). A single counterexample is enough to prove dependence.
Continuous example: f(x, y) = (3x - y)/9 cannot be written as a product g(x) * h(y) because the numerator 3x - y mixes x and y in a way that does not factor. Therefore X and Y are not independent.
Verification: f_X(1) * f_Y(1) = (1/6) * (10/9) = 10/54, but f(1, 1) = (3 - 1)/9 = 2/9 = 12/54. These are not equal, confirming dependence.
Discrete example: check whether p(1, 1) = p_X(1) * p_Y(1). We have p(1, 1) = 0.05, and p_X(1) * p_Y(1) = 0.10 * 0.20 = 0.02. Since 0.05 is not 0.02, X and Y are not independent.
The conditional density of Y given X = x is the joint density divided by the marginal density of X at that point.
f(y | x) = f(x, y) / f_X(x), provided f_X(x) > 0
Continuous example: f(y | x) = [(3x - y)/9] / [(2x - 1)/6] = 2(3x - y) / [3(2x - 1)], for 1 < y < 2.
This is a valid density in y for each fixed x in (1, 3). You can verify by integrating f(y | x) over y from 1 to 2 and confirming it equals 1.
Discrete example: P(Y = 5 | X = 1) = p(1, 5) / p_X(1) = 0 / 0.10 = 0. Given X = 1, there is zero probability of Y equalling 5.
f_X(x) = \int_{-\infty}^{\infty} f(x, y) \, dyf_Y(y) = \int_{-\infty}^{\infty} f(x, y) \, dxp_X(x) = \sum_{\text{all } y} p(x, y)p_Y(y) = \sum_{\text{all } x} p(x, y)f(y \mid x) = \frac{f(x, y)}{f_X(x)}, \quad f_X(x) > 0\text{Independence: } f(x, y) = f_X(x) \cdot f_Y(y) \text{ for all } x, y"The joint PDF looks separable, so X and Y must be independent." Be careful. A function like (3x - y)/9 has both x and y in a single term that does not factor. You must attempt the factorisation explicitly or check a specific point. Visual inspection is unreliable.
"If X and Y are not independent, the conditional density must look completely different from the marginal." The conditional density can still resemble the marginal in shape. Dependence means there is some difference, but it may be subtle, particularly when the coupling between variables is weak.
"I can check independence at one point and conclude." One matching point proves nothing. Independence requires the product rule to hold at every point. However, one failing point is enough to prove dependence.
"Marginal distributions sum/integrate to something other than 1." If your marginal does not integrate (or sum) to 1, you have an algebra error. This is a reliable sanity check.
⚠️ Computing marginals is almost always the first step in any joint distribution problem. Get comfortable with the mechanics: set up the integral (or sum), identify the correct limits, and simplify.
⚠️ Independence questions appear frequently. The fastest disproof is a single counterexample: pick the easiest point in the support, compute both sides, and compare.
⚠️ Conditional density questions often follow directly from the marginal. If you have already computed f_X(x), the conditional is just the joint divided by that marginal. Keep your earlier work tidy.
⚠️ Always verify your marginals integrate/sum to 1 before moving on. This catches sign errors and limit mistakes early.
Joint and conditional distributions underpin any situation where you model two related quantities at once. In engineering, the joint distribution of temperature and pressure in a system determines failure probabilities. In finance, the joint distribution of two asset returns is the foundation of portfolio theory and risk management. Conditional distributions are the engine behind Bayesian updating: given observed data, what do you now believe about a parameter?
True or false: to find the marginal PDF of X, you integrate the joint PDF over all values of X. (False, you integrate over Y.)
Fill in the blank: X and Y are independent if and only if the joint distribution equals the ______ of the marginals. (product)
True or false: if one cell in a joint probability table fails the independence product rule, X and Y are dependent. (True.)
Fill in the blank: the conditional density f(y | x) equals the joint density divided by the ______ of X. (marginal density)
True or false: a marginal PMF can sum to a value other than 1 if some joint probabilities are zero. (False, it always sums to 1.)
Q: Given f(x, y) = (3x - y)/9 for 1 < x < 3, 1 < y < 2, find f_X(x).
A: Integrate over y from 1 to 2. f_X(x) = (1/9) * [3xy - y^2/2] from 1 to 2 = (1/9)(3x - 3/2) = (2x - 1)/6 for 1 < x < 3.
Q: Using the same joint PDF, are X and Y independent? Justify your answer.
A: No. The joint PDF (3x - y)/9 cannot be factored into a function of x alone times a function of y alone. Alternatively, checking at (1, 1): f(1, 1) = 2/9, but f_X(1) * f_Y(1) = (1/6)(10/9) = 10/54, and 2/9 = 12/54 does not equal 10/54.
Q: Find the conditional density f(y | x) for the joint PDF above.
A: f(y | x) = f(x, y) / f_X(x) = [(3x - y)/9] / [(2x - 1)/6] = 2(3x - y) / [3(2x - 1)] for 1 < y < 2.
Q: From the discrete joint table with X in {1, 2, 3} and Y in {1, 3, 5}, compute P(Y = 5 | X = 1).
A: P(Y = 5 | X = 1) = p(1, 5) / p_X(1) = 0 / 0.10 = 0.
Q: Using the same discrete table, find the marginal distribution of Y.
A: p_Y(1) = 0.20, p_Y(3) = 0.50, p_Y(5) = 0.30.
Q: Are X and Y independent in the discrete case? Show a specific counterexample.
A: No. p(1, 1) = 0.05, but p_X(1) * p_Y(1) = 0.10 * 0.20 = 0.02. Since 0.05 does not equal 0.02, they are not independent.
This material connects directly to expected value and variance of joint distributions (Chapter 5), where you use marginals and conditionals to compute E[X], E[Y], E[XY], and Cov(X, Y). Independence makes covariance zero, which simplifies variance calculations for sums of random variables. Conditional distributions are the foundation of Bayesian inference: posterior = likelihood times prior divided by marginal, which is exactly the conditional density formula in a different context.
Joint PDF, joint PMF, joint probability density function, joint probability mass function, marginal density, marginal distribution, marginal PDF, marginal PMF, conditional density, conditional distribution, conditional PDF, independence of random variables, joint distribution table, bivariate distribution, multivariate probability, f(x y), f(y given x), STAT 3021, probability and statistics, University of Minnesota