Source: Analytical Statistics (T), NYU
When you have two random variables at once, the joint distribution tells you the probability of every possible pair of outcomes. From that joint distribution you can recover each variable's individual (marginal) distribution by summing or integrating out the other variable. Conditioning one variable on the other, and testing whether the two are independent, both come down to comparing the joint distribution with those marginals.
Difficulty: Intermediate | Prerequisites: single-variable probability distributions (p.d.f., c.d.f.), basic integration, summation notation.
Up to this point in the course, you have been working with one random variable at a time. In practice, you almost always care about several quantities simultaneously: a person's height and weight, a stock's price and trading volume, two exam scores for the same student. This lecture extends the machinery of probability distributions to pairs of random variables. You will need your existing comfort with p.d.f.s, c.d.f.s, and basic integration, because every concept here (joint, marginal, conditional, independence) is built by combining those single-variable tools in a structured way.
Joint probability density function (joint p.d.f.), f_XY(x, y)
The function that gives the probability of X and Y taking a particular pair of values simultaneously. For discrete variables, f_XY(x, y) = P(X = x, Y = y). Think of it as the full two-dimensional picture: every combination of X and Y, with its probability.
Joint cumulative distribution function (joint c.d.f.), F_XY(x, y)
The probability that X is at most x and Y is at most y: F_XY(x, y) = P(X ≤ x, Y ≤ y). In simple terms, it accumulates probability from the bottom-left corner of the (x, y) plane up to the point (x, y).
Marginal distribution (marginal p.d.f.), f_X(x) or f_Y(y)
The distribution of one variable on its own, obtained by summing (discrete) or integrating (continuous) the joint p.d.f. over all values of the other variable. Think of it as collapsing the two-dimensional table down to one dimension.
Conditional distribution (conditional p.d.f.), f_{Y|X}(y | x)
The distribution of Y given that X takes a specific value x. Computed as the joint p.d.f. divided by the marginal of X: f_{Y|X}(y | x) = f_XY(x, y) / f_X(x). In simple terms, this is what Y looks like once you know what X is.
Independence of random variables
X and Y are independent if knowing the value of one tells you nothing about the other. Formally, f_XY(x, y) = f_X(x) · f_Y(y) for all (x, y). Think of it as: the joint distribution factors neatly into the product of the two marginals.
The joint p.d.f. f_XY(x, y) assigns a probability to every pair (x, y) in R².
All probabilities are non-negative, and the total across every pair sums (or integrates) to 1.
For any region A ⊆ R², the probability that (X, Y) falls in A is the sum or integral of f_XY over A.
The joint c.d.f. F_XY(x, y) = P(X ≤ x, Y ≤ y) accumulates probability from the lower-left.
You can recover rectangle probabilities from the joint c.d.f. using the inclusion-exclusion formula: P(a ≤ X ≤ b, c ≤ Y ≤ d) = F(b, d) − F(a, d) − F(b, c) + F(a, c).
Continuous case: F_XY(x, y) = ∫ from −∞ to x ∫ from −∞ to y f_XY(u, v) dv du.
Recovering the p.d.f. from the c.d.f.: f_XY(x, y) = ∂²F_XY / ∂x∂y.
Discrete case: F_XY(x, y) = Σ over u ≤ x, v ≤ y of f_XY(u, v).
The marginal p.d.f. of X is obtained by removing Y from the joint distribution.
Discrete: f_X(x) = Σ over all y of f_XY(x, y).
Continuous: f_X(x) = ∫ from −∞ to ∞ f_XY(x, y) dy.
Similarly for Y:
Discrete: f_Y(y) = Σ over all x of f_XY(x, y).
Continuous: f_Y(y) = ∫ from −∞ to ∞ f_XY(x, y) dx.
The marginals lose information about the relationship between X and Y. Two different joint distributions can share the same marginals.
X and Y are independent if, for any regions A, B ⊆ R: P(X ∈ A, Y ∈ B) = P(X ∈ A) · P(Y ∈ B).
An equivalent (and usually easier) test: check whether the joint p.d.f. factors into the product of the marginals everywhere: f_XY(x, y) = f_X(x) · f_Y(y) for all (x, y).
If independence holds, knowing X gives you no information about Y, and vice versa.
If even one pair (x, y) violates the factorisation, the variables are not independent.
The conditional p.d.f. of Y given X = x is: f_{Y|X}(y | x) = f_XY(x, y) / f_X(x), provided f_X(x) > 0.
For discrete variables: f_{Y|X}(y | x) = P(Y = y and X = x) / P(X = x).
The conditional p.d.f. behaves like any other p.d.f.: it is non-negative and integrates (or sums) to 1 over y.
Verification: ∫ f_{Y|X}(y | x) dy = ∫ f_XY(x, y) / f_X(x) dy = f_X(x) / f_X(x) = 1.
When X and Y are independent, the conditional distribution f_{Y|X}(y | x) reduces to the marginal f_Y(y) for every value of x. Independence means conditioning on X changes nothing about Y.
Joint p.d.f. (discrete): f_XY(x, y) = P(X = x, Y = y), with Σ_i f_XY(x_i, y_i) = 1.
Joint c.d.f.: F_XY(x, y) = P(X ≤ x, Y ≤ y).
Rectangle probability from the c.d.f.: P(a ≤ X ≤ b, c ≤ Y ≤ d) = F(b,d) − F(a,d) − F(b,c) + F(a,c).
CDF to PDF (continuous): f_XY(x, y) = ∂²F_XY(x, y) / ∂x∂y.
PDF to CDF (continuous): F_XY(x, y) = ∫{−∞}^{x} ∫{−∞}^{y} f_XY(u,v) dv du.
Marginal of X (continuous): f_X(x) = ∫_{−∞}^{∞} f_XY(x, y) dy.
Marginal of Y (continuous): f_Y(y) = ∫_{−∞}^{∞} f_XY(x, y) dx.
Conditional p.d.f.: f_{Y|X}(y | x) = f_XY(x, y) / f_X(x).
Independence condition: f_XY(x, y) = f_X(x) · f_Y(y) for all (x, y).
Joint distributions are the foundation for modelling any situation where two or more quantities interact. In finance, the joint distribution of asset returns determines portfolio risk. In engineering, the joint distribution of load and material strength drives reliability analysis. Marginals and conditionals appear whenever you want to isolate one factor (e.g. "what is the distribution of exam score given that the student studied more than 10 hours?"). Independence testing is critical in experimental design: if your treatment and control groups are not independent of some confound, your results are suspect.
Students often think that the marginals fully determine the joint distribution. They do not. Many different joint distributions can share the same marginals. The joint carries strictly more information.
Students sometimes confuse "uncorrelated" with "independent." Zero correlation means no linear relationship, but the variables can still be dependent in non-linear ways. Independence is a stronger condition.
A common error is dividing by f_Y(y) instead of f_X(x) when computing f_{Y|X}(y | x). The denominator is always the marginal of the variable you are conditioning on (the variable whose value is known).
Students occasionally forget the inclusion-exclusion signs in the rectangle-probability formula and drop the + F(a, c) term. Sketch the regions on a number line to see why all four terms are needed.
⚠️ Expect to be asked to compute a marginal from a given joint p.d.f., both in the discrete (summation) and continuous (integration) cases.
⚠️ The rectangle-probability formula using the joint c.d.f. is a classic exam question. Know the signs.
⚠️ Verifying or disproving independence by checking whether f_XY(x, y) = f_X(x) · f_Y(y) appears frequently. One counterexample is enough to disprove independence.
⚠️ Computing a conditional p.d.f. and then verifying it integrates to 1 is a standard "show that" problem.
True or False: If f_XY(x, y) = f_X(x) · f_Y(y) for all (x, y), then X and Y are independent.
Fill in the blank: To obtain the marginal p.d.f. f_X(x) from the joint p.d.f., you ______ the joint p.d.f. over all values of ______.
True or False: The conditional p.d.f. f_{Y|X}(y | x) can be negative for some values of y.
Fill in the blank: P(a ≤ X ≤ b, c ≤ Y ≤ d) = F(b,d) − F(a,d) − ______ + ______.
True or False: Two different joint distributions can have the same marginal distributions.
Answers: 1. True. 2. Integrate (or sum); Y. 3. False, it is always non-negative. 4. F(b,c); F(a,c). 5. True.
Q: Suppose X and Y are discrete with joint p.d.f. given by a table. How do you find f_X(2)?
A: Sum f_XY(2, y) over every value of y in the table. The result is the marginal probability that X = 2.
Q: Given F_XY(x, y), compute P(1 ≤ X ≤ 3, 2 ≤ Y ≤ 5).
A: Apply the rectangle formula: F(3, 5) − F(1, 5) − F(3, 2) + F(1, 2).
Q: You are given f_XY(x, y) = 6x for 0 < x < y < 1, and 0 otherwise. Find f_X(x).
A: Integrate over the valid range of y: f_X(x) = ∫ from x to 1 of 6x dy = 6x(1 − x) for 0 < x < 1.
Q: With the same joint p.d.f., find f_{Y|X}(y | x).
A: f_{Y|X}(y | x) = f_XY(x, y) / f_X(x) = 6x / [6x(1 − x)] = 1 / (1 − x), for x < y < 1. This is uniform on (x, 1).
Q: How would you check whether X and Y are independent given their joint p.d.f.?
A: Compute both marginals f_X(x) and f_Y(y). Then check whether f_XY(x, y) equals f_X(x) · f_Y(y) for all (x, y). If it fails at any single point, they are not independent.
This material connects directly to transformations of random variables (next lecture), where you will apply these joint-distribution tools to derive the distribution of a new variable defined as a function of X and Y. It also underpins expected value of functions of two variables, covariance, and correlation, which are the next steps in building the theory of how random variables relate to each other. In applied work, joint and conditional distributions are the backbone of Bayesian inference, regression, and multivariate statistical modelling.
Joint distribution, joint probability density function, joint p.d.f., joint PDF, joint CDF, joint cumulative distribution function, marginal distribution, marginal p.d.f., marginal PDF, conditional distribution, conditional p.d.f., conditional PDF, f_XY, f_{Y|X}, independence of random variables, bivariate distribution, multivariate distribution, two random variables, summing out, integrating out, product of marginals, rectangle probability formula, inclusion-exclusion CDF, analytical statistics, MATH 301, NYU