Difficulty: Intermediate | Prerequisites: Ch. 0–3 (descriptive statistics, basic set notation)
This block introduces the mathematical framework for quantifying uncertainty. Probability rules let you compute the chance of events; random variables and their distributions (binomial, Poisson) let you model outcomes numerically. Everything from sampling distributions (Ch. 7) through hypothesis testing (Ch. 9) and regression (Ch. 12) depends on these foundations. If you cannot set up a probability problem, compute a conditional probability, or recognise when to use a binomial model, the later inference chapters will not make sense.
Probability measures how likely an event is on a 0-to-1 scale. Conditional probability and Bayes' Rule let you update beliefs given new information. Discrete random variables assign numbers to outcomes; the binomial distribution models the count of successes in fixed trials, and the Poisson distribution models the count of events in a fixed interval.
Experiment
An activity with at least two possible outcomes whose result cannot be predicted with absolute certainty. A single run of the experiment is called a trial.
Sample space (S or Ω)
The listing of all possible outcomes of an experiment.
Event
Any collection of outcomes from the sample space. A simple event contains exactly one outcome.
Complement (A′)
Everything in the sample space that is not in A. P(A′) = 1 – P(A).
Union (A ∪ B)
A or B or both.
Intersection (A ∩ B)
Both A and B.
Disjoint (mutually exclusive)
Two events with nothing in common: A ∩ B = {}. If A and B are disjoint, P(A ∪ B) = P(A) + P(B).
Conditional probability
P(A | B) = P(A ∩ B) / P(B). The probability of A given that B has occurred.
Independence
Two events A and B are independent if and only if P(A | B) = P(A). Equivalently, P(A ∩ B) = P(A)P(B). In simple terms, knowing that one occurred does not change the probability of the other.
Bayes' Rule
A formula for reversing conditional probabilities: P(A | B) = P(B | A)P(A) / P(B). The generalised version partitions the sample space into k disjoint events.
Random variable
A function that assigns a unique numerical value to each outcome in a sample space.
Probability mass function (pmf)
For a discrete random variable, p(x) = P(X = x). Gives the probability for each specific value.
Expected value (mean) of a discrete RV
E(X) = μ = Σ x · p(x). The long-run average value if you repeated the experiment many times. Think of it as the "balance point" of the distribution.
Variance of a discrete RV
Var(X) = σ² = Σ(x – μ)² · p(x) = E(X²) – [E(X)]².
Binomial distribution
Models the number of successes in n fixed, independent trials, each with the same probability p of success. X ~ B(n, p).
Poisson distribution
Models the count of events occurring in a fixed interval of time or space, where events happen independently at a constant average rate λ. X ~ Poisson(λ).
P(A) is always between 0 and 1. P(S) = 1. P({}) = 0
Three types of probability:
Subjective: a personal judgement, no calculation
Empirical: P(A) = (number of times A occurs) / (total number of trials)
Theoretical (equally likely): P(A) = (number of outcomes in A) / (number of outcomes in S)
Complement rule: P(A′) = 1 – P(A). Often the fastest route to a probability
Addition rule: P(A ∪ B) = P(A) + P(B) – P(A ∩ B). For disjoint events, the intersection term drops out
Conditional probability: P(A | B) = P(A ∩ B) / P(B)
Independence test: A and B are independent if and only if P(A | B) = P(A)
If A and B are independent, then so are all combinations of these events and their complements (A and B′, A′ and B, A′ and B′)
General multiplication rule: P(A ∩ B) = P(A) · P(B | A). For three events: P(A ∩ B ∩ C) = P(A) · P(B | A) · P(C | A ∩ B)
Bayes' Rule (two variables): P(A | B) = P(B | A)P(A) / [P(B | A)P(A) + P(B | A′)P(A′)]
Generalised Bayes' Rule: partition the sample space into k disjoint events A₁, …, Aₖ. Then P(Aⱼ | B) = P(B | Aⱼ)P(Aⱼ) / Σ P(B | Aᵢ)P(Aᵢ)
A random variable assigns a number to each outcome in S
The probability distribution lists every possible value and its probability
A valid pmf satisfies: all pᵢ are between 0 and 1, and they sum to 1
Expected value: E(X) = Σ x · p(x). This is the theoretical long-run mean
Rules for means:
μ(a + bX) = a + bμX
μ(X ± Y) = μX ± μY
E(g(X)) = Σ g(xᵢ)pᵢ
Variance: Var(X) = E(X²) – [E(X)]²
Rules for variance:
σ²(a + bX) = b²σ²X
If X and Y are independent: σ²(X ± Y) = σ²X + σ²Y (variances always add, even for subtraction)
If X and Y have correlation ρ: σ²(X ± Y) = σ²X + σ²Y ± 2ρσXσY
Binomial requirements (BInS): only two outcomes per trial, trials are independent, n fixed identical trials, same probability p on each trial
X ~ B(n, p). P(X = x) = C(n,x) · pˣ · (1 – p)ⁿ⁻ˣ, for x = 0, 1, 2, …, n
Mean: μ = np. Variance: σ² = np(1 – p). SD: σ = √[np(1 – p)]
Cumulative distribution function: P(X ≤ x) = P(X = 0) + P(X = 1) + … + P(X = x)
Poisson requirements: events in non-overlapping intervals are independent, the probability of an event in a small interval is proportional to the interval size, the probability of more than one event in a very small interval is negligible
X ~ Poisson(λ). P(X = x) = e⁻ᵞλˣ / x!, for x = 0, 1, 2, …
Mean = Variance = λ. SD = √λ
If the rate changes proportionally (e.g. doubling the interval), the new parameter is kλ
P(A') = 1 - P(A)P(A \cup B) = P(A) + P(B) - P(A \cap B)P(A \mid B) = \frac{P(A \cap B)}{P(B)}P(A \mid B) = \frac{P(B \mid A)\,P(A)}{P(B \mid A)\,P(A) + P(B \mid A')\,P(A')}E(X) = \sum_{\text{all } x} x \cdot p(x)\text{Var}(X) = E(X^2) - [E(X)]^2P(X = x) = \binom{n}{x} p^x (1-p)^{n-x}, \quad \binom{n}{x} = \frac{n!}{x!(n-x)!}\mu = np, \quad \sigma^2 = np(1-p)P(X = x) = \frac{e^{-\lambda}\lambda^x}{x!}, \quad \mu = \sigma^2 = \lambdaStudents often confuse "mutually exclusive" with "independent." They are different concepts. If two events are mutually exclusive (disjoint), they cannot both occur, which means they are dependent (unless one has probability zero)
Variances add for both X + Y and X – Y when the variables are independent. Students frequently subtract variances for X – Y; this is wrong
The binomial formula C(n,x) counts combinations, not permutations. Order does not matter
Poisson is not just "binomial with a large n." Poisson models counts in a continuous interval with a constant rate; binomial models counts in a fixed number of discrete trials
⚠️ Bayes' Rule problems are a near certainty on the final. Practise setting up the denominator as P(B | A)P(A) + P(B | A′)P(A′).
⚠️ Know how to verify whether events are independent by checking P(A | B) = P(A), not by intuition.
⚠️ For binomial problems, confirm all four BInS conditions before applying the formula.
⚠️ Be comfortable computing E(X) and Var(X) from a pmf table.
True or False: If P(A) = 0.3 and P(B) = 0.4 and A, B are independent, then P(A ∩ B) = 0.12. (True)
Fill in the blank: For X ~ B(10, 0.5), E(X) = ___. (5)
True or False: For a Poisson distribution, the mean equals the variance. (True)
True or False: P(A ∪ B) = P(A) + P(B) always. (False – only when A and B are disjoint)
Fill in the blank: If X ~ Poisson(3), then σ = ___. (√3 ≈ 1.73)
Q: A medical test has a 95% detection rate (sensitivity) and a 3% false positive rate. The disease prevalence is 1%. If a person tests positive, what is the probability they actually have the disease?
A: Using Bayes' Rule: P(D | +) = (0.95 × 0.01) / [(0.95 × 0.01) + (0.03 × 0.99)] = 0.0095 / (0.0095 + 0.0297) = 0.0095 / 0.0392 ≈ 0.242. About 24.2%.
Q: X ~ B(8, 0.3). Find P(X = 2).
A: P(X = 2) = C(8,2) × 0.3² × 0.7⁶ = 28 × 0.09 × 0.1176 ≈ 0.2965.
Q: Are the events "drawing a heart" and "drawing a face card" from a standard deck independent?
A: P(heart) = 13/52 = 1/4. P(face card) = 12/52 = 3/13. P(heart ∩ face card) = 3/52. Check: (1/4)(3/13) = 3/52. Yes, they are independent.
Q: If X and Y are independent with Var(X) = 4 and Var(Y) = 9, what is Var(X – Y)?
A: Var(X – Y) = Var(X) + Var(Y) = 4 + 9 = 13. Variances add even for subtraction.
Conditional probability and Bayes' Rule underpin the logic of hypothesis testing in Ch. 9 (you are essentially asking: how likely is this data given the null hypothesis?). The binomial distribution reappears in Ch. 6.5 with the normal approximation, and again in Ch. 8–9 for proportion inference. The Poisson distribution is used in modelling rare events and connects to the exponential distribution (Ch. 6.4) through the waiting-time interpretation.
Probability, sample space, event, complement, union, intersection, disjoint, mutually exclusive, conditional probability, independence, multiplication rule, Bayes' Rule, Bayes' theorem, random variable, pmf, probability mass function, expected value, variance, standard deviation, binomial distribution, BInS, Poisson distribution, CDF, cumulative distribution function, STAT 35000, Purdue statistics