Source: Purdue University, Introduction to Statistics
Tags: sample space, events, disjoint, mutually exclusive, frequentist probability, complement rule, addition rule, conditional probability, multiplication rule, independence, Bayes rule, Bayes theorem, tree diagram, Venn diagram
Difficulty: Intermediate Prerequisites: Chapters 1–2 (variable types, population vs. sample). Comfort with basic set notation (union, intersection, complement) is helpful but not strictly required.
Probability is the mathematical language that connects populations to samples. It gives you a way to quantify uncertainty: how likely is this outcome? Chapter 4 builds from the basics (what is a sample space, what does a probability mean) through the rules you need for combining and conditioning events. These rules are not just theoretical exercises; they form the machinery behind every inference method you will meet in later chapters. If you can handle conditional probability and the multiplication rule, the rest of the course leans on that skill repeatedly.
Probability measures how likely an event is, on a scale from 0 to 1. You combine events with the complement, addition, and multiplication rules. Conditional probability lets you update your belief about one event given information about another, and independence means that one event tells you nothing new about the other.
Sample space (S)
The set of all possible outcomes of an experiment.
In simple terms, it is a complete list of everything that could happen.
Event
A subset of the sample space. An event occurs if the outcome of the experiment falls within that subset.
Disjoint events (mutually exclusive)
Two events A and B that cannot both occur at the same time: A ∩ B = ∅.
Think of it as "if one happens, the other cannot."
Frequentist interpretation of probability
The probability of an event E is the long-run relative frequency: as the number of trials n → ∞, n(E)/n → P(E).
In simple terms, if you repeat the experiment many, many times, the proportion of times E occurs settles down to P(E).
Complement (A′ or Aᶜ)
Everything in the sample space that is not in A. P(A′) = 1 − P(A).
Union (A ∪ B)
The event that A or B (or both) occurs.
Intersection (A ∩ B)
The event that both A and B occur simultaneously.
Conditional probability
The probability of A given that B has occurred: P(A | B) = P(A ∩ B) / P(B).
Think of it as narrowing the sample space to only those outcomes where B happened, then asking how likely A is within that restricted world.
Independent events
Two events A and B are independent if knowing one occurred does not change the probability of the other. Equivalent conditions: P(A | B) = P(A), or P(B | A) = P(B), or P(A ∩ B) = P(A) · P(B).
Bayes' Rule (Bayes' Theorem)
A formula for "reversing" a conditional probability: P(A | B) = P(B | A) P(A) / [P(B | A) P(A) + P(B | Aᶜ) P(Aᶜ)]
In simple terms, it lets you flip the direction of conditioning when you know the probabilities in the other direction.
Write out the sample space S for a given experiment by listing every possible outcome.
An event is any collection of outcomes from S. It can contain one outcome, several, or none (the empty set).
0 ≤ P(A) ≤ 1 for any event A.
P(A) = Σ ωᵢ, where the ωᵢ are the probabilities of the individual outcomes in A.
P(S) = 1 (something always happens).
P(∅) = 0 (the impossible event has probability zero).
Equally likely outcomes (classical/theoretical): P(A) = (number of outcomes in A) / (total number of outcomes in S)
Empirical (from a table): use observed relative frequencies or a probability distribution table.
Venn diagrams are a visual tool for organising the probabilities of overlapping events. Fill in the intersection first, then work outwards.
Complement rule: P(A′) = 1 − P(A). Useful when "at least one" problems are easier to solve by finding the probability of "none."
General addition rule: P(A ∪ B) = P(A) + P(B) − P(A ∩ B). You subtract the intersection because it gets counted twice.
Addition rule for disjoint events: If A and B are disjoint, P(A ∪ B) = P(A) + P(B), since P(A ∩ B) = 0.
P(A | B) = P(A ∩ B) / P(B). This is only defined when P(B) > 0.
Reading it aloud: "the probability of A, given B."
Conditioning effectively shrinks the sample space to B.
General multiplication rule: P(A ∩ B) = P(A) · P(B | A) = P(B) · P(A | B).
Extended to three events: P(A ∩ B ∩ C) = P(A) · P(B | A) · P(C | A and B).
For independent events: P(A ∩ B) = P(A) · P(B). This extends to any number of independent events.
A tree diagram lays out sequential events with branches. Multiply along branches to get joint probabilities; add across final branches to get marginal probabilities.
Bayes' Rule is the algebraic version of "working backwards through the tree": P(A | B) = P(B | A) P(A) / [P(B | A) P(A) + P(B | Aᶜ) P(Aᶜ)]
Useful when you know P(B | A) but need P(A | B).
Independent events: knowing one occurred does not change the probability of the other. They can (and usually do) occur together.
Disjoint events: they cannot occur together, so knowing one occurred tells you the other did not. Disjoint events with non-zero probabilities are never independent.
This distinction is heavily tested. Do not conflate the two.
Complement: P(A′) = 1 − P(A)
General addition: P(A ∪ B) = P(A) + P(B) − P(A ∩ B)
Conditional probability: P(A | B) = P(A ∩ B) / P(B)
General multiplication: P(A ∩ B) = P(A) · P(B | A) = P(B) · P(A | B)
Three-event multiplication: P(A ∩ B ∩ C) = P(A) · P(B | A) · P(C | A ∩ B)
Independence test (any one is sufficient): P(A | B) = P(A), or P(B | A) = P(B), or P(A ∩ B) = P(A) · P(B)
Bayes' Rule: P(A | B) = P(B | A) P(A) / [P(B | A) P(A) + P(B | Aᶜ) P(Aᶜ)]
Medical screening is a classic Bayes' Rule problem. A test for a disease might have a 99% detection rate (sensitivity), but if the disease is rare, most positive results could still be false positives. Bayes' Rule lets you calculate the probability that a patient who tests positive truly has the disease. This kind of reasoning is used every day in diagnostics, spam filtering, and quality control.
Students frequently confuse "independent" with "disjoint." Two disjoint events with non-zero probabilities are dependent, not independent, because if one happens, the other definitely did not.
Students sometimes forget to subtract P(A ∩ B) in the general addition rule, leading to double-counting.
When using Bayes' Rule, students often mix up which conditional probability goes in the numerator. Draw the tree diagram first; it makes the structure visible.
Students sometimes assume events are independent without checking. Independence must be given in the problem or verified with one of the three equivalent conditions.
⚠️ Writing out the sample space correctly is often the first part of a multi-part problem. If you get S wrong, subsequent calculations will also be wrong.
⚠️ The distinction between independent and disjoint events is tested directly. Expect a question that asks, "Are these events independent, disjoint, both, or neither?"
⚠️ Conditional probability and the multiplication rule appear in nearly every probability exam question. Drill these until they feel automatic.
⚠️ Bayes' Rule or tree-diagram problems are common. Practise setting up the tree with the given information before computing.
True or False: If P(A) = 0.3, then P(A′) = 0.7.
Fill in the blank: Two events that cannot both happen are called __________ events.
True or False: If A and B are disjoint and both have positive probability, they are independent.
Fill in the blank: P(A | B) = P(A ∩ B) / __________.
True or False: P(S) = 0.
Answers: 1. True. 2. Disjoint (or mutually exclusive). 3. False. 4. P(B). 5. False (P(S) = 1).
Q: Suppose P(A) = 0.4, P(B) = 0.5, and P(A ∩ B) = 0.2. Find P(A ∪ B).
A: P(A ∪ B) = 0.4 + 0.5 − 0.2 = 0.7.
Q: Using the same values, find P(A | B).
A: P(A | B) = P(A ∩ B) / P(B) = 0.2 / 0.5 = 0.4.
Q: Are A and B independent in the example above?
A: Check: P(A | B) = 0.4, and P(A) = 0.4. Since P(A | B) = P(A), yes, A and B are independent.
Q: 1% of a population has a disease. A test detects the disease 95% of the time (sensitivity) and correctly identifies healthy people 90% of the time (specificity). If a person tests positive, what is the probability they have the disease?
A: Let D = disease, T⁺ = positive test. P(D) = 0.01, P(T⁺ | D) = 0.95, P(T⁺ | D′) = 0.10. By Bayes' Rule: P(D | T⁺) = (0.95 × 0.01) / (0.95 × 0.01 + 0.10 × 0.99) = 0.0095 / (0.0095 + 0.099) = 0.0095 / 0.1085 ≈ 0.0876, or about 8.8%.
Q: Are disjoint events independent? Explain briefly.
A: No. If A and B are disjoint and both have positive probability, then P(A ∩ B) = 0, but P(A) · P(B) > 0, so P(A ∩ B) ≠ P(A) · P(B). They are dependent.
Conditional probability and multiplication rules are the foundation of the binomial and Poisson distributions in Chapter 5, where you compute probabilities of sequences of independent events.
Bayes' Rule connects forward to inferential statistics, where you update beliefs about a population given observed data.
The complement rule is used constantly in later probability work, especially for "at least one" and CDF calculations.
sample space, event, outcome, disjoint, mutually exclusive, complement, union, intersection, probability rules, addition rule, multiplication rule, conditional probability, independence, independent events, Bayes rule, Bayes theorem, tree diagram, Venn diagram, frequentist probability, classical probability, empirical probability, STAT 101, intro to statistics, Purdue