Difficulty: Intermediate | Prerequisites: Parts 1 and 2 of these notes (sample space, events, set operations, complement rule, addition rule).
Most real probability questions are not about isolated events. They are about events in context: "given that this has happened, what is the chance of that?" Conditional probability formalises this idea. From it come the multiplication rule (for computing joint probabilities), the concept of independence (when conditioning changes nothing), and Bayes' theorem (for flipping a conditional probability around). Bayes' theorem in particular is one of the most practically important results in the entire course, showing up in medical testing, spam filtering, and any scenario where you update a belief with new evidence.
Conditional probability is the probability of one event given that another has occurred. The multiplication rule converts a conditional probability into a joint probability. Two events are independent when knowing one tells you nothing about the other. Bayes' theorem lets you reverse a conditional: if you know P(B|A), it gives you P(A|B).
Conditional probability P(A|B)
The probability of event A occurring given that event B has already occurred. In simple terms, you are zooming into a smaller universe (just the outcomes where B happened) and asking how likely A is within that universe.
Multiplication rule
P(A ∩ B) = P(A) · P(B|A), or equivalently P(A ∩ B) = P(B) · P(A|B). It lets you find the probability that both events happen. Think of it as: the chance of both = the chance of the first, times the chance of the second given the first already happened.
Independence
Two events A and B are independent if the occurrence of one does not affect the probability of the other. Formally: P(B|A) = P(B) and P(A|B) = P(A). In simple terms, knowing that A happened gives you zero new information about B.
Bayes' theorem
A formula that reverses the direction of a conditional probability. Given P(B|A), P(A), and P(B), it computes P(A|B). Think of it as the "update your belief" formula: you start with a prior, see some evidence, and calculate a posterior.
Prior probability
The initial probability of an event before new evidence is considered. In the Bayes' theorem context, this is P(A) before observing B.
Posterior probability
The updated probability of an event after incorporating new evidence. In the Bayes' theorem context, this is P(A|B).
P(A|B) = P(A ∩ B) / P(B)
This formula assumes P(B) > 0 (you cannot condition on an event with zero probability).
The numerator P(A ∩ B) is the probability that both A and B occur.
The denominator P(B) restricts the universe to outcomes where B has happened.
Worked example:
A table gives probabilities for students who read regularly from newspapers A, B, and C:
A | B | C | A ∩ B | A ∩ C | B ∩ C | |
|---|---|---|---|---|---|---|
P | 0.14 | 0.23 | 0.37 | 0.08 | 0.09 | 0.13 |
a) P(A|B) = P(A ∩ B) / P(B) = 0.08 / 0.23 = 0.348.
Given that a student reads B, there is roughly a 34.8% chance they also read A.
b) P(B|A) = P(A ∩ B) / P(A) = 0.08 / 0.14 = 0.571.
Given that a student reads A, there is roughly a 57.1% chance they also read B. Notice P(A|B) ≠ P(B|A) in general.
Rearranging the conditional probability formula:
P(A ∩ B) = P(A) · P(B|A)
Or equivalently:
P(A ∩ B) = P(B) · P(A|B)
For three events, the chain extends:
P(A ∩ B ∩ C) = P(A) · P(B|A) · P(C|A ∩ B)
Worked example (defective fuses):
A box has 8 good fuses and 2 defective fuses (10 total). Two fuses are drawn without replacement.
a) P(D₁ ∩ D₂) = P(D₁) · P(D₂|D₁) = (2/10) · (1/9) = 2/90 ≈ 0.022.
After drawing one defective fuse, only 1 defective remains among 9 fuses.
b) For three draws: P(G₁ ∩ D₂ ∩ D₃) = P(G₁) · P(D₂|G₁) · P(D₃|G₁ ∩ D₂) = (8/10) · (2/9) · (1/8) = 1/45 ≈ 0.022.
Two events A and B are independent if and only if:
P(B|A) = P(B) and P(A|B) = P(A)
An equivalent test: A and B are independent if and only if:
P(A ∩ B) = P(A) · P(B)
This last form is often the easiest to check. Multiply the individual probabilities; if the product equals the joint probability, the events are independent.
When to suspect independence:
Drawing with replacement (the first draw does not change the composition for the second).
Events from separate, unrelated processes (a coin flip and a die roll).
When events are dependent:
Drawing without replacement (the pool changes after each draw).
Events connected by a causal or structural link.
Key distinction from disjoint events:
Disjoint events are visualised with Venn diagrams (no overlap).
Conditional events are best handled with tree diagrams (branching probabilities).
Independent events are identified from context (does knowing one change the other?).
Bayes' theorem lets you reverse a conditional probability. The two-event form:
P(A|B) = P(B|A) · P(A) / P(B)
The denominator P(B) is often expanded using the law of total probability:
P(B) = P(B|A) · P(A) + P(B|A') · P(A')
Substituting, the full form becomes:
P(A|B) = P(B|A) · P(A) / [P(B|A) · P(A) + P(B|A') · P(A')]
The general form for multiple partitions A₁, A₂, ..., Aₖ:
P(Aⱼ|B) = P(B|Aⱼ) · P(Aⱼ) / Σᵢ P(B|Aᵢ) · P(Aᵢ)
This is the classic application. A disease test has:
Sensitivity: P(+|D) = 0.98 (probability of a positive test given the person has the disease).
Specificity: P(−|D') = 0.95 (probability of a negative test given the person is healthy).
Prevalence: P(D) = 0.01 (1% of the population has the disease).
Question: If a person tests positive, what is the probability they actually have the disease?
Step 1: set up the tree diagram.
Two branches from the root: Disease (0.01) and No Disease (0.99).
From Disease: Positive (0.98) and Negative (0.02). From No Disease: Positive (0.05, since specificity is 0.95) and Negative (0.95).
Step 2: compute joint probabilities.
P(D ∩ +) = P(D) · P(+|D) = 0.01 × 0.98 = 0.0098.
P(D' ∩ +) = P(D') · P(+|D') = 0.99 × 0.05 = 0.0495.
P(D ∩ −) = P(D) · P(−|D) = 0.01 × 0.02 = 0.0002. This is a check value: not needed for the answer but confirms consistency.
P(D' ∩ −) = P(D') · P(−|D') = 0.99 × 0.95 = 0.9405.
Step 3: apply Bayes' theorem.
P(D|+) = P(D ∩ +) / [P(D ∩ +) + P(D' ∩ +)] = 0.0098 / (0.0098 + 0.0495) = 0.0098 / 0.0593 ≈ 0.165.
Interpretation: Even with a positive test, there is only about a 16.5% chance the person has the disease. The low prevalence (1%) means most positive results are false positives. This result surprises most students and is a common exam question.
Conditional probability: P(A|B) = P(A ∩ B) / P(B)
Multiplication rule (two events): P(A ∩ B) = P(A) · P(B|A)
Multiplication rule (three events): P(A ∩ B ∩ C) = P(A) · P(B|A) · P(C|A ∩ B)
Independence test: P(A ∩ B) = P(A) · P(B)
Bayes' theorem: P(A|B) = P(B|A) · P(A) / [P(B|A) · P(A) + P(B|A') · P(A')]
General Bayes': P(Aⱼ|B) = P(B|Aⱼ) · P(Aⱼ) / Σᵢ P(B|Aᵢ) · P(Aᵢ)
Medical screening is the textbook application of Bayes' theorem: the low prevalence of many diseases means positive screening results often have a surprisingly low positive predictive value. Spam filters use Bayes' theorem to classify emails (the "Naive Bayes" classifier). In criminal forensics, DNA match probabilities are interpreted using Bayes' theorem to avoid the "prosecutor's fallacy," which confuses P(evidence|innocent) with P(innocent|evidence). The multiplication rule for dependent events is central to reliability engineering, where component failures cascade.
Confusing P(A|B) with P(B|A). These are almost never equal. The medical testing example above is the clearest illustration: P(+|D) = 0.98 but P(D|+) = 0.165.
Assuming independence when it has not been established. Unless the problem states independence or the setup guarantees it (replacement, separate processes), you must use the conditional version of the multiplication rule.
Thinking disjoint events are independent. If A and B are disjoint and both have positive probability, they are dependent: if A occurs, B cannot.
Forgetting to expand P(B) using the law of total probability when applying Bayes' theorem. The denominator accounts for all paths to B, not just the one through A.
⚠️ Conditional probability is tested in nearly every probability exam. Be comfortable with the formula and with reading joint probability tables.
⚠️ The medical testing / Bayes' theorem scenario is a favourite. Know how to set up the tree diagram, compute joint probabilities, and apply the formula.
⚠️ "Without replacement" in a problem signals dependent events and requires the multiplication rule with conditional probabilities, not the simplified independent version.
⚠️ Checking for independence using P(A ∩ B) = P(A) · P(B) is a common short-answer or true/false question.
⚠️ Be able to distinguish between disjoint, conditional, and independent events. The chapter explicitly maps these: disjoint events use Venn diagrams, conditional events use tree diagrams, independent events use context.
True or False: P(A|B) = P(B|A) always.
Fill in the blank: Two events are independent if P(A ∩ B) = ______.
True or False: In the medical testing example, a positive test means the person almost certainly has the disease.
Fill in the blank: The multiplication rule states P(A ∩ B) = P(A) · ______.
True or False: Bayes' theorem requires knowing (or being able to calculate) P(B).
Answers: 1. False. 2. P(A) · P(B). 3. False (only about 16.5% in the example). 4. P(B|A). 5. True.
Q: P(A) = 0.4, P(B) = 0.5, and P(A ∩ B) = 0.2. Are A and B independent?
A: Check: P(A) · P(B) = 0.4 × 0.5 = 0.2 = P(A ∩ B). Yes, A and B are independent.
Q: A box contains 5 red and 3 blue balls. Two are drawn without replacement. What is the probability both are red?
A: P(R₁ ∩ R₂) = P(R₁) · P(R₂|R₁) = (5/8) · (4/7) = 20/56 = 5/14 ≈ 0.357.
Q: In the medical testing example, what is P(D|−), the probability of having the disease given a negative test?
A: P(D ∩ −) / P(−) = 0.0002 / (0.0002 + 0.9405) = 0.0002 / 0.9407 ≈ 0.0002. Essentially negligible, which is reassuring: a negative test very reliably rules out the disease.
Q: A fair coin and a fair die are tossed. Let A = "heads" and B = "roll a 6." Are A and B independent? Compute P(A ∩ B).
A: Yes, A and B are independent because the coin and die are separate processes. P(A ∩ B) = P(A) · P(B) = (1/2) · (1/6) = 1/12 ≈ 0.083.
Q: State Bayes' theorem in words, without the formula.
A: The probability of a hypothesis given observed evidence equals the probability of the evidence given the hypothesis, times the prior probability of the hypothesis, divided by the overall probability of the evidence.
Conditional probability and independence are prerequisites for understanding discrete and continuous probability distributions (Chapters 5 and 6), where you will compute probabilities of outcomes from distribution functions. Bayes' theorem is the foundation of Bayesian inference, which is an entire statistical framework covered in advanced courses. The multiplication rule for dependent events appears again in counting problems (permutations and combinations) and in the context of joint distributions for multiple random variables.
conditional probability, given that, multiplication rule, joint probability, independence, independent events, dependent events, Bayes' theorem, Bayes' rule, prior probability, posterior probability, sensitivity, specificity, prevalence, false positive, positive predictive value, law of total probability, tree diagram, without replacement, STAT, PIV 203, Chapter 4, introduction to statistics, Purdue