Introduction to Probability and Statistics, STAT 3021 | Source: Mathematical Expectation and Variance guide
Difficulty: Intermediate | Prerequisites: Basic probability distributions, summation notation, integral calculus fundamentals
Expected value and variance are the two most fundamental summaries of any probability distribution. Expected value tells you where a distribution sits on average; variance tells you how spread out it is. Together they underpin almost every method you will meet later in the course, from confidence intervals to hypothesis testing to regression. You need comfortable fluency with discrete and continuous probability distributions, summation notation, and basic integration before this material will land properly.
The expected value (mean) of a random variable is the probability-weighted average of its possible outcomes. Variance measures how far those outcomes typically stray from the mean, and standard deviation is just the square root of variance, put back into the original units. Both are calculated differently for discrete vs continuous variables, but the logic is the same: weight each outcome by how likely it is.
Expected value (mean), E(X) or μ_X
The probability-weighted average of all possible values a random variable can take. For a discrete variable, multiply each outcome by its probability and sum the results. For a continuous variable, integrate x times the density function over all x.
In simple terms, this is the long-run average you would see if you repeated the experiment forever.
Variance, Var(X) or σ²_X
The expected value of the squared deviation from the mean: E[(X – μ_X)²]. It measures how spread out the distribution is around its centre.
Think of it as a single number capturing how much "scatter" there is. Larger variance means outcomes are more dispersed.
Standard deviation, σ_X
The positive square root of the variance. It brings the spread measure back into the same units as X itself.
In simple terms, if your variable is measured in pounds, the standard deviation is also in pounds, whereas variance is in pounds squared.
Probability mass function (PMF)
P(X = x) for a discrete random variable. Assigns a probability to each possible value of X.
Think of it as a lookup table: plug in a value, get back the chance of that value occurring.
Probability density function (PDF), f(x)
The continuous counterpart of the PMF. The area under f(x) over an interval gives the probability that X falls in that interval.
In simple terms, the height of the curve at a point is not a probability by itself, but the area under it is.
Discrete case
Multiply each possible value x by its probability P(X = x), then sum across all x.
Formula: E(X) = Σ x · P(X = x)
Continuous case
Integrate x times the density function f(x) over the entire real line.
Formula: E(X) = ∫ x · f(x) dx, limits from –∞ to +∞
Linearity of expectation
For constants a and b: E(aX + b) = a · E(X) + b
This works whether or not the variables involved are independent. It is one of the most useful properties in the course.
Interpretation
E(X) is the "balance point" of the distribution. If you placed weights on a number line according to the PMF or PDF, E(X) is where the line would balance.
It represents the long-run average: repeat the experiment many times and the sample mean converges to E(X).
Worked example (discrete)
Cars passing through a car wash between 4:00 and 5:00 PM on a sunny Friday:
x | 4 | 5 | 6 | 7 | 8 | 9 |
|---|---|---|---|---|---|---|
P(X = x) | 1/12 | 1/12 | 1/4 | 1/4 | 1/6 | 1/6 |
E(X) = 4(1/12) + 5(1/12) + 6(1/4) + 7(1/4) + 8(1/6) + 9(1/6) = 6.83
So on average, about 6.83 cars pass through in that hour.
Definition of variance
Var(X) = E[(X – μ_X)²]
Equivalent computational formula: Var(X) = E(X²) – [E(X)]²
The computational formula is often easier in practice: find E(X²), find E(X), square E(X), subtract.
Properties of variance
Var(X) ≥ 0, always. A variance of zero means X is a constant.
Var(aX + b) = a² · Var(X). Adding a constant shifts the distribution but does not change the spread. Multiplying by a constant scales the spread by the square of that constant.
For independent X and Y: Var(X + Y) = Var(X) + Var(Y). Independence is required here, unlike linearity of expectation.
Standard deviation
σ_X = √Var(X)
Same units as X, which makes it more interpretable than variance.
Worked example
X has the following distribution:
x | 3 | 4 | 5 |
|---|---|---|---|
P(X = x) | 0.3 | 0.4 | 0.3 |
E(X) = 3(0.3) + 4(0.4) + 5(0.3) = 4
E(X²) = 9(0.3) + 16(0.4) + 25(0.3) = 16.6
Var(X) = 16.6 – 16 = 0.6
σ_X = √0.6 ≈ 0.775
Formula | Discrete | Continuous |
|---|---|---|
Expected value | E(X) = Σ x · P(X = x) | E(X) = ∫ x · f(x) dx |
Variance | Var(X) = Σ (x – μ)² · P(X = x) | Var(X) = ∫ (x – μ)² · f(x) dx |
Variance (shortcut) | Var(X) = E(X²) – [E(X)]² | Same |
Standard deviation | σ_X = √Var(X) | Same |
Linearity of E | E(aX + b) = aE(X) + b | Same |
Variance scaling | Var(aX + b) = a² Var(X) | Same |
Expected value is the basis for any decision that involves weighing outcomes by their likelihood. Insurance companies use it to set premiums: they estimate the expected payout per policy and price above it. In finance, the expected return of a portfolio is the probability-weighted average of possible returns.
Variance (and standard deviation) is how risk is quantified. A stock with high variance has returns that swing widely; a stock with low variance is more predictable. Portfolio theory builds on this: you choose combinations of assets that keep expected return high while bringing variance down.
Students often confuse the expected value with the most likely outcome (the mode). E(X) is the weighted average, not the peak of the distribution. A distribution can have an expected value that is not even a possible outcome.
Variance can never be negative. If your calculation gives a negative number, there is an arithmetic error. Go back and check.
Students sometimes apply Var(X + Y) = Var(X) + Var(Y) without verifying independence. When X and Y are not independent, a covariance term is needed: Var(X + Y) = Var(X) + Var(Y) + 2 Cov(X, Y).
Confusing variance and standard deviation in word problems is common. When a problem says "the spread is 3," check whether that refers to σ or σ². Getting these mixed up will throw your answer off by a square root.
⚠️ Be ready to compute E(X) and Var(X) from a probability table. This is the most common calculation question on this topic.
⚠️ Know both the definition formula and the shortcut formula for variance. Exams often give one and ask you to derive or use the other.
⚠️ Linearity of expectation shows up in multi-part problems where you need E(aX + b). Forgetting to distribute the constant a through, or dropping the additive constant b, is a frequent mark-loser.
⚠️ The scaling rule Var(aX + b) = a² Var(X) is tested in problems where the variable is transformed (unit conversions, for example). Notice that b disappears entirely.
True or False: E(X) must be one of the possible values that X can take.
False. E(X) is a weighted average and can fall between possible values.
Fill in the blank: Var(3X + 5) = ___ Var(X).
The additive constant drops out; the multiplicative constant is squared.
True or False: If Var(X) = 0, then X takes only one value with probability 1.
True. Zero variance means no spread at all.
Fill in the blank: The shortcut formula for variance is Var(X) = E(X²) – ___.
[E(X)]²
True or False: Standard deviation can be negative.
False. It is defined as the positive square root of variance.
Q: A random variable X has the distribution P(X = 1) = 0.2, P(X = 2) = 0.5, P(X = 3) = 0.3. What is E(X)?
A: E(X) = 1(0.2) + 2(0.5) + 3(0.3) = 0.2 + 1.0 + 0.9 = 2.1
Q: Using the same distribution, what is Var(X)?
A: E(X²) = 1(0.2) + 4(0.5) + 9(0.3) = 0.2 + 2.0 + 2.7 = 4.9. Var(X) = 4.9 – (2.1)² = 4.9 – 4.41 = 0.49
Q: If Y = 2X + 7, where X has E(X) = 5 and Var(X) = 3, find E(Y) and Var(Y).
A: E(Y) = 2(5) + 7 = 17. Var(Y) = 4(3) = 12. The additive constant 7 does not affect the variance.
Q: A continuous random variable X has density f(x) = 2(1 – x) for 0 < x < 1 and 0 elsewhere. What is E(X)?
A: E(X) = ∫ from 0 to 1 of x · 2(1 – x) dx = ∫ (2x – 2x²) dx = [x² – (2/3)x³] from 0 to 1 = 1 – 2/3 = 1/3
Q: Why does Var(X + 5) = Var(X)?
A: Adding a constant shifts every value of X by the same amount, so all deviations from the mean remain unchanged. The spread does not change.
Expected value feeds directly into the Law of Large Numbers, which formalises the idea that sample averages converge to E(X) as the sample grows. Variance is the building block of the Central Limit Theorem: the CLT describes how the distribution of sample means becomes normal with variance σ²/n.
These concepts also connect to covariance and correlation (covered in the companion notes), which extend the idea of variance to pairs of random variables. In regression analysis later in the course, variance and expected value are used to derive the least-squares estimates.
Expected value, mathematical expectation, mean of a random variable, E(X), mu, population mean, weighted average, variance, Var(X), sigma squared, dispersion, spread, standard deviation, sigma, discrete expected value, continuous expected value, linearity of expectation, variance shortcut formula, probability-weighted average, balance point of a distribution, long-run average