Source: Lecture notes / study guide
Difficulty: Intermediate | Prerequisites: Variables, Measurement, and Sampling (Doc 1); familiarity with basic algebra and factorials.
Probability is the bridge between descriptive statistics and inferential statistics. Once you can describe data, the next question is: how likely is a particular outcome? Probability gives you the language and the rules to answer that. This unit covers the foundational concepts (sample spaces, events, rules for combining probabilities), counting techniques (permutations and combinations), and the major discrete probability distributions (binomial, hypergeometric, multinomial) that reappear throughout hypothesis testing, confidence intervals, and beyond. If you are comfortable with this material, the normal distribution and inferential methods that follow will make considerably more sense.
Probability measures how likely an event is, on a scale from 0 (impossible) to 1 (certain). Three interpretations exist: classical (equally likely outcomes), empirical (based on observed frequencies), and subjective (educated guess). The addition rule handles "or" questions; the multiplication rule handles "and" questions. Counting rules (fundamental counting principle, permutations, combinations) tell you how many outcomes are possible. The binomial distribution models a fixed number of independent trials with two outcomes; the hypergeometric covers sampling without replacement; the multinomial extends to more than two outcomes.
Probability
The chance of an event occurring, expressed as a number between 0 and 1 (or equivalently, 0% and 100%).
Probability experiment
A chance process that generates a set of well-defined results called outcomes. Think of it as any process whose result you cannot predict with certainty beforehand.
Outcome
The result of a single trial of a probability experiment (e.g. getting heads on one coin flip).
Sample space (S)
The set of all possible outcomes of an experiment. For a single coin flip: S = {heads, tails}.
Event (E)
A subset of outcomes from the sample space. Think of it as the specific result or group of results you care about.
Independent events
Two events where the occurrence of one does not affect the probability of the other. Coin flips are independent: the first flip has no effect on the second.
Dependent events
Two events where the occurrence of one changes the probability of the other. Drawing cards without replacement: each draw changes what is left in the deck.
Complementary event (E-bar)
The set of all outcomes in the sample space that are not in event E. P(E) + P(E-bar) = 1.
Mutually exclusive events
Events that cannot occur at the same time. If A happens, B cannot happen, and vice versa.
Conditional probability
The probability that event B occurs given that event A has already occurred, written P(B|A). In simple terms, it is the probability of B in a world where A is known to have happened.
Permutation
An arrangement of objects in a specific order. Order matters. Think of it as how many ways you can line things up.
Combination
A selection of objects without regard to order. Order does not matter. Think of it as how many ways you can choose a group.
Binomial distribution
The discrete probability distribution that counts the number of successes in n independent trials, each with the same probability of success p.
Hypergeometric distribution
A discrete distribution describing the number of successes in n draws, without replacement, from a finite population containing K successes.
Multinomial distribution
An extension of the binomial for experiments with more than two possible outcomes per trial.
A probability experiment is any process whose outcome is uncertain. Each run of the experiment produces an outcome; the set of all possible outcomes is the sample space (S). A tree diagram is a useful tool for listing every outcome systematically, especially when the experiment has multiple stages (e.g. flipping two coins: HH, HT, TH, TT).
An event (E) is any subset of the sample space. Events are classified as:
Independent: the first event does not affect the probability of the second. Example: successive rolls of a die.
Dependent: the probability of the second event changes depending on the first. Example: drawing two cards from a deck without replacement.
Complementary: event E-bar contains everything in S that is not in E. By definition, P(E) + P(E-bar) = 1, so if you know one you can find the other.
Mutually exclusive: events that cannot happen simultaneously. If you roll a single die, "rolling a 2" and "rolling a 5" are mutually exclusive.
Three interpretations:
Classical probability: all outcomes are equally likely. P(E) = number of outcomes in E / total number of outcomes in S. No experiment needs to be run; you reason from the structure of the problem (e.g. a fair die has 6 equally likely faces).
Empirical probability: based on observed frequencies. P(E) = frequency of the class / total frequencies. You must actually collect data.
Subjective probability: an educated guess based on judgement, experience, or incomplete information. No formal experiment required.
Four basic probability rules:
Rule 1: 0 ≤ P(E) ≤ 1 for any event E.
Rule 2: if E cannot occur, P(E) = 0.
Rule 3: if E is certain, P(E) = 1.
Rule 4: the sum of the probabilities of all outcomes in S equals 1.
Practical interpretation: near 0 means highly unlikely; near 0.5 means a coin-flip; near 1 means highly likely. If you know P(E), you can always find P(E-bar) = 1 − P(E).
Addition rule ("or" questions):
Mutually exclusive events: P(A or B) = P(A) + P(B). Because A and B cannot overlap, there is nothing to subtract.
Non-mutually exclusive events: P(A or B) = P(A) + P(B) − P(A and B). You subtract the overlap to avoid counting it twice.
Multiplication rule ("and" questions):
Independent events: P(A and B) = P(A) × P(B). Neither event affects the other.
Dependent events: P(A and B) = P(A) × P(B|A). The probability of B is conditional on A having occurred.
Conditional probability: P(B|A) = P(A and B) / P(A). This gives you the probability of B in the restricted sample space where A has already happened.
A quick way to remember: "or" means add (and subtract any overlap); "and" means multiply (and condition if dependent).
Counting rules tell you how many outcomes are possible, which feeds directly into classical probability calculations.
Fundamental counting rule: if event 1 can occur in k₁ ways, event 2 in k₂ ways, and so on, the total number of ways all events can occur in sequence is k₁ × k₂ × ... × kₙ. Example: a restaurant offers 3 starters and 5 mains, so there are 3 × 5 = 15 possible two-course meals.
Permutation (order matters): the number of ways to arrange r objects selected from n distinct objects.
Formula: nPr = n! / (n − r)!
n! ("n factorial") = n × (n − 1) × (n − 2) × ... × 1. By convention, 0! = 1.
Use permutations when the arrangement or sequence matters (e.g. finishing positions in a race, digits of a combination lock).
Combination (order does not matter): the number of ways to choose r objects from n distinct objects, ignoring order.
Formula: nCr = n! / ((n − r)! × r!)
Use combinations when you are selecting a group and the order within the group is irrelevant (e.g. choosing 3 committee members from 10 candidates).
The key decision on any exam question: does the order of selection matter? If yes, use permutations. If no, use combinations. A useful check: nCr is always smaller than or equal to nPr for the same n and r, because combinations collapse all the rearrangements of the same group into one.
Probability distribution: a listing of every possible value of a random variable together with its probability. For a valid discrete probability distribution, two conditions must hold: every individual probability is between 0 and 1 inclusive, and the sum of all probabilities equals 1.
Random variables come in two types:
Discrete: a finite or countable number of values (0, 1, 2, 3 ...).
Continuous: infinitely many values on a continuous scale with no gaps (5, 5.1, 5.01 ...).
Mean of a discrete probability distribution (expected value): μ = Σ[x × P(x)]. This is the long-run average outcome.
Variance of a discrete probability distribution: σ² = Σ[(x − μ)² × P(x)].
Binomial distribution
Models the number of successes in n independent trials, each with exactly two outcomes (success/failure) and a constant probability of success p.
Four properties:
Fixed number of trials (n).
Each trial has exactly two outcomes.
Trials are independent.
The probability of success (p) stays the same across trials.
General formula: P(X = r) = nCr × p^r × q^(n−r), where q = 1 − p.
Mean: μ = np. Variance: σ² = npq.
The mode of a binomial B(n, p) distribution is the floor of (n + 1)p when that value is not an integer. When (n + 1)p is an integer in {1, ..., n}, both (n + 1)p and (n + 1)p − 1 are modes. There is no simple closed-form formula for the median.
Hypergeometric distribution
Models the number of successes in n draws, without replacement, from a finite population of size N containing exactly K successes.
Conditions:
Each draw is classified as success or failure.
The probability of success changes on each draw because the population shrinks (sampling without replacement from a finite population).
Probability mass function: P(X = k) = [C(K, k) × C(N − K, n − k)] / C(N, n).
Use the hypergeometric instead of the binomial when sampling is done without replacement from a relatively small population.
Multinomial distribution
Extends the binomial to experiments with more than two possible outcomes per trial.
Conditions:
A statistical experiment with k possible outcomes.
The experiment is repeated independently n times.
Formula: P = [n! / (n₁! × n₂! × ... × nₖ!)] × p₁^n₁ × p₂^n₂ × ... × pₖ^nₖ, where nᵢ is the number of times outcome i occurs and pᵢ is the probability of outcome i.
Classical probability: P(E) = n(E) / n(S)
Empirical probability: P(E) = f / n (frequency of the class / total frequencies)
Complement rule: P(E-bar) = 1 − P(E)
Addition rule (mutually exclusive): P(A or B) = P(A) + P(B)
Addition rule (non-mutually exclusive): P(A or B) = P(A) + P(B) − P(A and B)
Multiplication rule (independent): P(A and B) = P(A) × P(B)
Multiplication rule (dependent): P(A and B) = P(A) × P(B|A)
Conditional probability: P(B|A) = P(A and B) / P(A)
Fundamental counting rule: total outcomes = k₁ × k₂ × ... × kₙ
Permutation: nPr = n! / (n − r)!
Combination: nCr = n! / ((n − r)! × r!)
Binomial probability: P(X = r) = nCr × p^r × q^(n−r), where q = 1 − p
Binomial mean: μ = np
Binomial variance: σ² = npq
Mean of a discrete distribution: μ = Σ[x × P(x)]
Variance of a discrete distribution: σ² = Σ[(x − μ)² × P(x)]
Hypergeometric PMF: P(X = k) = [C(K, k) × C(N − K, n − k)] / C(N, n)
Multinomial: P = [n! / (n₁! × n₂! × ... × nₖ!)] × p₁^n₁ × p₂^n₂ × ... × pₖ^nₖ
The binomial distribution is used in quality control to model the number of defective items in a batch when each item independently passes or fails inspection. Insurance companies use probability rules to calculate the likelihood of multiple claims occurring in the same period. Hypergeometric distributions arise in ecological studies: researchers capture, tag, and release animals, then use the distribution to estimate population sizes from recapture data. Combinations underpin lottery mathematics: the number of ways to choose 6 numbers from 49 determines the odds of winning.
Students often confuse permutations and combinations. The test is simple: does the order of selection matter? If rearranging the same items gives a different outcome (e.g. a password), use permutations. If not (e.g. a committee), use combinations.
A common error with the addition rule is forgetting to subtract P(A and B) when events are not mutually exclusive. If you add P(A) + P(B) without removing the overlap, you double-count the outcomes in both events.
Students sometimes apply the binomial formula to sampling-without-replacement problems. If the draws are from a finite population without replacement, the trials are not independent, and the hypergeometric distribution is the correct model. (The binomial is a reasonable approximation only when the sample is very small relative to the population.)
"Independent" and "mutually exclusive" are not the same thing. Mutually exclusive events cannot happen together; independent events can happen together, but one occurring does not change the probability of the other.
⚠️ Be able to identify whether a problem calls for the addition rule or the multiplication rule. "Or" = addition; "and" = multiplication.
⚠️ Know how to decide between permutations and combinations. Exam questions often describe a scenario and ask for the number of ways something can happen.
⚠️ Expect questions that give you n, p, and r and ask you to compute a binomial probability by hand. Practise the formula until it is mechanical.
⚠️ Understand when to use the hypergeometric distribution instead of the binomial. The trigger: sampling without replacement from a finite, known population.
⚠️ Be ready to verify that a given table is a valid probability distribution (all probabilities between 0 and 1, sum to 1) and to compute its mean and variance.
⚠️ Conditional probability questions are a favourite. P(B|A) = P(A and B) / P(A). Make sure you can rearrange this to find P(A and B) when given P(B|A) and P(A).
True or False: P(E) + P(E-bar) always equals 1. (True)
Fill in the blank: If two events are mutually exclusive, P(A and B) = ______. (0)
True or False: 5P3 is larger than 5C3. (True, because permutations count order and combinations do not)
Fill in the blank: The four properties of a binomial experiment are fixed number of trials, two outcomes per trial, ______, and constant probability of success. (independence of trials)
True or False: The hypergeometric distribution assumes sampling with replacement. (False, it assumes sampling without replacement)
Q: A box contains 5 red and 3 blue marbles. If one marble is drawn at random, what is the probability it is red?
A: P(red) = 5 / 8 = 0.625. This is a classical probability calculation: 5 favourable outcomes out of 8 total.
Q: Events A and B are not mutually exclusive. P(A) = 0.4, P(B) = 0.3, P(A and B) = 0.1. Find P(A or B).
A: P(A or B) = 0.4 + 0.3 − 0.1 = 0.6.
Q: How many ways can a president, vice president, and treasurer be chosen from a club of 12 members?
A: Order matters (different positions), so use permutations. 12P3 = 12! / 9! = 12 × 11 × 10 = 1,320.
Q: A multiple-choice test has 10 questions, each with 4 options. A student guesses randomly on every question. What is the probability of getting exactly 3 correct?
A: This is binomial with n = 10, p = 0.25, r = 3. P(X = 3) = 10C3 × (0.25)³ × (0.75)⁷ = 120 × 0.015625 × 0.133484 ≈ 0.2503.
Q: A shipment of 20 tablets contains 4 defective ones. If 5 tablets are selected at random without replacement, what distribution models the number of defectives found?
A: The hypergeometric distribution, because we are sampling without replacement from a finite population (N = 20, K = 4, n = 5).
Q: Explain the difference between independent events and mutually exclusive events.
A: Independent events can both occur, but the occurrence of one does not change the probability of the other (e.g. two separate coin flips). Mutually exclusive events cannot occur at the same time (e.g. rolling a 2 and rolling a 5 on the same die throw). Mutually exclusive events with non-zero probabilities are always dependent, because if one occurs the other's probability drops to zero.
This connects to the normal distribution, which is the continuous counterpart of the binomial: as n grows large and p is not too extreme, the binomial approximates a normal curve. Hypothesis testing relies on probability to determine whether an observed result is statistically significant or could have occurred by chance. The counting rules reappear in combinatorics problems throughout mathematics and computer science. Conditional probability is the foundation of Bayes' theorem, which you may encounter in more advanced courses.
probability, sample space, outcome, event, independent events, dependent events, complementary event, mutually exclusive, classical probability, empirical probability, subjective probability, addition rule, multiplication rule, conditional probability, P(B|A), fundamental counting rule, permutation, combination, factorial, nPr, nCr, tree diagram, probability distribution, discrete random variable, continuous random variable, expected value, mean of a probability distribution, variance of a probability distribution, binomial distribution, binomial formula, hypergeometric distribution, multinomial distribution, sampling without replacement, probability mass function, PMF, Bayes' theorem, ASU statistics