Difficulty: Introductory to Intermediate | Prerequisites: Basic probability (sample spaces, events, P(A), union and intersection of events)
Conditional probability answers the question "how does knowing one thing change the likelihood of another?" It is the foundation of statistical reasoning, medical testing, spam filtering, and courtroom evidence analysis. This topic assumes you are comfortable with basic probability notation, can compute P(A), P(A and B), and P(A or B), and understand what a sample space is. If those ideas are not solid, revisit your introductory probability notes first.
Conditional probability P(A | B) measures the probability of A once you know B has happened. Bayes' theorem flips the conditioning: it lets you compute P(A | B) when you know P(B | A) and the base rates P(A) and P(B). Together, these tools let you update beliefs with new evidence.
Conditional probability
The probability of event A occurring given that event B has occurred, written P(A | B). Defined as P(A | B) = P(A and B) / P(B), provided P(B) > 0. Think of it as narrowing the sample space down to only the outcomes where B happened, then asking how many of those also include A.
Joint probability
P(A and B), the probability that both A and B occur simultaneously. In simple terms, this is the overlap between the two events.
Marginal probability
The probability of a single event on its own, such as P(A) or P(B), without conditioning on anything else. Think of it as the "baseline" or "overall" probability before you learn any new information.
Prior probability
In a Bayesian context, the probability of a hypothesis before observing new evidence, typically written P(A). In simple terms, it is what you believed before the data came in.
Posterior probability
The updated probability of a hypothesis after observing evidence, typically P(A | B). Think of it as your revised belief once you have taken the new information into account.
Likelihood
P(B | A), the probability of observing the evidence B given that hypothesis A is true. In simple terms, it measures how well the hypothesis explains the observed data.
Bayes' theorem
The formula P(A | B) = P(B | A) · P(A) / P(B), which relates the posterior to the prior, the likelihood, and the marginal probability of the evidence. Think of it as the mathematically precise way to update a belief when new evidence arrives.
P(A | B) = P(A and B) / P(B), defined only when P(B) > 0
The vertical bar "|" reads as "given"
This formula restricts attention to the subset of outcomes where B occurs, then checks the proportion that also satisfy A
Rearranging gives the multiplication rule: P(A and B) = P(A | B) · P(B)
Equally valid: P(A and B) = P(B | A) · P(A)
Two events A and B are independent if and only if P(A | B) = P(A)
Equivalently, P(A and B) = P(A) · P(B)
Independence means learning B gives you no new information about A
In most real problems, events are not independent, which is exactly why conditional probability is useful
The theorem states: P(A | B) = P(B | A) · P(A) / P(B)
This is useful when you know the "reverse" conditional probability P(B | A) but need P(A | B)
Each component has a name:
P(A) = prior (your belief before evidence)
P(B | A) = likelihood (how probable the evidence is if A is true)
P(B) = marginal likelihood or evidence (total probability of observing B)
P(A | B) = posterior (your updated belief after evidence)
When you need P(B) and it is not given directly, expand it:
P(B) = P(B | A) · P(A) + P(B | not A) · P(not A)
This partitions all ways B can happen into the cases where A is true and where A is false
For more than two hypotheses A1, A2, ..., An that partition the sample space:
P(B) = Σ P(B | Ai) · P(Ai)
Conditional probability: P(A | B) = P(A and B) / P(B)
Multiplication rule: P(A and B) = P(A | B) · P(B) = P(B | A) · P(A)
Bayes' theorem: P(A | B) = P(B | A) · P(A) / P(B)
Law of total probability (two-event partition): P(B) = P(B | A) · P(A) + P(B | not A) · P(not A)
Medical screening relies on Bayes' theorem constantly. If a disease test has a known sensitivity P(positive | disease) and you know the prevalence P(disease), Bayes' theorem tells you the probability a patient with a positive result actually has the disease, which is often much lower than people expect. The same logic powers spam filters, fraud detection, and courtroom DNA evidence interpretation.
Students frequently confuse P(A | B) with P(B | A). These are different quantities. The probability of having a disease given a positive test is not the same as the probability of testing positive given the disease.
A common error is to assume that a high likelihood P(B | A) means a high posterior P(A | B). If the prior P(A) is very small (a rare disease, for instance), the posterior can still be low even with a very accurate test.
Students sometimes forget that conditional probability is undefined when P(B) = 0. You cannot condition on an event that has zero probability of occurring.
Independence is often assumed without justification. If the problem does not state independence, do not assume P(A and B) = P(A) · P(B).
⚠️ Bayes' theorem questions appear in nearly every probability exam. The typical setup gives you sensitivity, specificity, and prevalence, then asks for the positive predictive value. Practise this pattern until it is routine.
⚠️ Expect questions that require the law of total probability to compute P(B) before applying Bayes' theorem.
⚠️ Watch for trick questions that test whether you confuse P(A | B) with P(B | A). Read the question carefully to identify which direction the conditioning goes.
Fill in the blank: P(A | B) = P(A and B) / ___.
True or false: P(A | B) is always equal to P(B | A).
True or false: If A and B are independent, then P(A | B) = P(A).
Fill in the blank: In Bayes' theorem, P(A) is called the ___ probability.
True or false: Bayes' theorem can be applied when P(B) = 0.
Answers: 1. P(B). 2. False. 3. True. 4. Prior. 5. False (P(B) must be greater than 0).
Q: A disease affects 1% of a population. A test for the disease is 95% sensitive (P(positive | disease) = 0.95) and 90% specific (P(negative | no disease) = 0.90). What is the probability that a person who tests positive actually has the disease?
A: Using Bayes' theorem with the law of total probability: P(disease | positive) = P(positive | disease) · P(disease) / P(positive). P(positive) = (0.95)(0.01) + (0.10)(0.99) = 0.0095 + 0.099 = 0.1085. P(disease | positive) = (0.95)(0.01) / 0.1085 ≈ 0.0876, or about 8.8%.
Q: If P(A) = 0.3, P(B) = 0.5, and P(A and B) = 0.15, are A and B independent?
A: Check whether P(A and B) = P(A) · P(B). We have 0.15 = 0.3 × 0.5 = 0.15. Yes, A and B are independent.
Q: Events C and D satisfy P(C | D) = 0.4 and P(D) = 0.25. What is P(C and D)?
A: By the multiplication rule: P(C and D) = P(C | D) · P(D) = 0.4 × 0.25 = 0.10.
Q: Explain in plain language why a positive result on a very accurate test can still mean a low probability of disease.
A: When the disease is rare, the vast majority of people tested are healthy. Even a small false-positive rate applied to that large healthy group produces many false positives, which outnumber the true positives from the small number of genuinely ill people. Bayes' theorem captures this: a low prior pulls the posterior down, even when the test is accurate.
Conditional probability is the gateway to Bayesian statistics, which applies these ideas to parameter estimation and machine learning. It also connects to decision theory (expected utility under uncertainty) and to Markov chains, where the probability of the next state depends only on the current state, a direct application of conditional probability.
conditional probability, Bayes' theorem, Bayes' rule, posterior probability, prior probability, likelihood, marginal probability, joint probability, law of total probability, independence, sensitivity, specificity, positive predictive value, false positive, base rate fallacy, updating beliefs, probability theory, MATH 101