Source: Walpole Ch. 2, Lecture Week 3
Tags: conditional probability, independence, Bayes theorem, law of total probability, multiplicative rule, STAT 3021, UMN
Difficulty: Introductory to Intermediate | Prerequisites: Counting and Probability Fundamentals (Ch. 1-2 notes)
Conditional probability answers the question: "Given that something has already happened, how does that change the probability of something else?" This is the single most important concept in the course, because it connects basic probability to Bayes' theorem, which is the engine behind medical testing, spam filters, and statistical inference. Independence is the special case where knowing one event tells you nothing about another. You need the counting and basic probability tools from Ch. 1-2 before tackling this material.
Conditional probability updates your probability estimate when you receive new information. The law of total probability lets you break a hard probability into easier conditional pieces. Bayes' theorem flips a conditional probability around: if you know P(B|A), it gives you P(A|B).
Conditional probability, P(A|B)
The probability of event A occurring given that event B has already occurred. Defined as P(A|B) = P(A ∩ B) / P(B), provided P(B) > 0. Think of it as shrinking your sample space to only the outcomes where B happened, then asking how many of those also contain A.
Independent events
Two events A and B are independent if knowing that one occurred does not change the probability of the other. Formally, P(A|B) = P(A), or equivalently P(A ∩ B) = P(A) × P(B). In simple terms, the two events have no influence on each other.
Multiplicative rule (general)
P(A ∩ B) = P(A|B) × P(B), or equivalently P(B|A) × P(A). This gives you the probability of both events happening. Think of it as a two-step process: B happens first, then A happens given B.
Law of total probability
If B₁, B₂, ..., Bₖ are mutually exclusive and exhaustive events (they partition the sample space), then P(A) = Σ P(A|Bᵢ) × P(Bᵢ). In simple terms, you break a complex probability into cases, compute each case separately, and add them up.
Bayes' theorem
A formula that reverses the direction of conditioning: P(Bⱼ|A) = P(A|Bⱼ) × P(Bⱼ) / Σ P(A|Bᵢ) × P(Bᵢ). Think of it as updating your belief about which "cause" (Bⱼ) is responsible, after observing an "effect" (A).
Prior probability
The initial probability P(Bⱼ) before observing any new evidence. In simple terms, your starting belief.
Posterior probability
The updated probability P(Bⱼ|A) after taking new evidence into account. In simple terms, your revised belief after seeing data.
P(A|B) = P(A ∩ B) / P(B), where P(B) > 0
Read as: "the probability of A given B"
The vertical bar "|" means "given that" or "conditional on"
Visually: you restrict the sample space to B, then look at how much of B is also in A
Example: 60% of students pass the final. Of those who attended every lecture, 85% pass. P(pass | attended all lectures) = 0.85
Rearranging the definition gives: P(A ∩ B) = P(A|B) × P(B)
Equivalently: P(A ∩ B) = P(B|A) × P(A)
Extends to three or more events: P(A ∩ B ∩ C) = P(A) × P(B|A) × P(C|A ∩ B)
This is how you compute "and" probabilities when events are not independent
A and B are independent if and only if P(A ∩ B) = P(A) × P(B)
Equivalent conditions: P(A|B) = P(A), or P(B|A) = P(B)
Independence simplifies the multiplicative rule: the joint probability is just the product of the marginals
Independence must be verified mathematically or stated in the problem; you cannot assume it from context
Multiple events: A₁, A₂, ..., Aₙ are mutually independent if every subset satisfies the product rule (not just pairwise)
If {B₁, B₂, ..., Bₖ} partition S (mutually exclusive and exhaustive), then:
P(A) = P(A|B₁)P(B₁) + P(A|B₂)P(B₂) + ... + P(A|Bₖ)P(Bₖ)
Useful when A is hard to compute directly but easy to compute within each "slice" Bᵢ
Common partition: B and B' (just two pieces), giving P(A) = P(A|B)P(B) + P(A|B')P(B')
Example: a factory has 3 machines producing 30%, 45%, and 25% of output. Each has a different defect rate. The total defect rate uses total probability across the three machines.
P(Bⱼ|A) = P(A|Bⱼ) × P(Bⱼ) / P(A), where P(A) is computed via the law of total probability
The numerator is the joint probability P(A ∩ Bⱼ) written as P(A|Bⱼ)P(Bⱼ)
The denominator normalises so the posterior probabilities sum to 1
Tree diagrams are the most reliable way to organise Bayes' theorem calculations: draw branches for each Bᵢ, then sub-branches for A and A' under each
Read the tree left to right to get joint probabilities (multiply along branches), then use the fraction to reverse direction
P(A|B) = \frac{P(A \cap B)}{P(B)}P(A \cap B) = P(A|B) \cdot P(B) = P(B|A) \cdot P(A)P(A) = \sum_{i=1}^{k} P(A|B_i) \cdot P(B_i)P(B_j|A) = \frac{P(A|B_j) \cdot P(B_j)}{\sum_{i=1}^{k} P(A|B_i) \cdot P(B_i)}Independence test:
P(A \cap B) = P(A) \cdot P(B)Bayes' theorem is the mathematics behind medical diagnostic testing. If a disease has a 1% prevalence rate and the test is 95% accurate, Bayes' theorem tells you the probability that a person who tests positive actually has the disease (often surprisingly low). The same logic drives spam filters, fraud detection, and search-engine ranking.
Students frequently confuse "independent" with "mutually exclusive." If two events are mutually exclusive and both have positive probability, they are dependent (knowing one happened tells you the other did not).
Confusing P(A|B) with P(B|A) is extremely common. "The probability of rain given clouds" is not the same as "the probability of clouds given rain." Bayes' theorem exists precisely to convert one into the other.
Students sometimes test independence by checking whether P(A|B) = P(B|A). That is not the test. The correct check is whether P(A|B) = P(A).
When building tree diagrams, students sometimes place the wrong probabilities on the branches. The first branches are the priors P(Bᵢ); the second branches are the likelihoods P(A|Bᵢ).
⚠️ Bayes' theorem problems appear on nearly every STAT 3021 exam. Practise setting up tree diagrams until the process is automatic.
⚠️ Expect a problem asking you to determine whether two events are independent by checking P(A ∩ B) = P(A)P(B). Show your working.
⚠️ The law of total probability is usually embedded within Bayes' theorem problems as the denominator. Recognise it as the "total" step.
True or False: If A and B are mutually exclusive, then A and B are independent. (False. Mutually exclusive events with P(A) > 0 and P(B) > 0 are always dependent.)
Fill in the blank: P(A ∩ B) = P(A|B) × ___. (P(B))
True or False: P(A|B) is always equal to P(B|A). (False. They are equal only in special cases.)
Fill in the blank: In Bayes' theorem, the denominator is computed using the law of ___. (Total probability)
Q: A factory has two machines. Machine 1 produces 60% of items and has a 3% defect rate. Machine 2 produces 40% of items and has a 5% defect rate. If a randomly selected item is defective, what is the probability it came from Machine 1?
A: Using Bayes' theorem. P(defective) = 0.60 × 0.03 + 0.40 × 0.05 = 0.018 + 0.020 = 0.038. P(Machine 1 | defective) = (0.60 × 0.03) / 0.038 = 0.018 / 0.038 ≈ 0.474.
Q: If P(A) = 0.3, P(B) = 0.4, and P(A ∩ B) = 0.12, are A and B independent?
A: Check: P(A) × P(B) = 0.3 × 0.4 = 0.12 = P(A ∩ B). Yes, they are independent.
Q: A medical test has a 98% sensitivity (P(positive | disease) = 0.98) and 95% specificity (P(negative | no disease) = 0.95). The disease prevalence is 1%. What is P(disease | positive test)?
A: P(positive) = 0.98 × 0.01 + 0.05 × 0.99 = 0.0098 + 0.0495 = 0.0593. P(disease | positive) = 0.0098 / 0.0593 ≈ 0.165, or about 16.5%. Even with an accurate test, the low prevalence keeps the posterior probability modest.
Conditional probability is the basis for understanding conditional distributions in Chapter 3. Bayes' theorem foreshadows Bayesian inference, which appears in advanced statistics courses. Independence is a core assumption of many distributions (binomial trials must be independent) and carries through to sampling and hypothesis testing.
conditional probability, given that, independence, dependent events, mutually exclusive vs independent, Bayes rule, Bayes theorem, law of total probability, partition, prior, posterior, likelihood, tree diagram, multiplicative rule, joint probability, marginal probability, STAT 3021, Walpole Chapter 2