Difficulty: Beginner to Intermediate | Prerequisites: Descriptive statistics study notes (variables, measures of centre and spread).
Probability is the language statistics uses to quantify uncertainty. Every inference method you will meet later in STAT 101, from confidence intervals to hypothesis tests, rests on the rules covered here. This material bridges the descriptive tools you already know (how to summarise data you can see) with inferential tools (how to draw conclusions about data you cannot see). You should be comfortable with basic set notation and the idea of a variable before starting.
Probability measures how likely events are, on a scale from 0 (impossible) to 1 (certain). The addition rule handles "or" questions for mutually exclusive events; the multiplication rule handles "and" questions for independent events. Conditional probability and Bayes' rule let you update probabilities when you learn new information.
Experiment (random experiment)
A procedure that produces an uncertain outcome. Examples: rolling a die, flipping a coin, measuring the time until the next customer arrives. In simple terms, any process where you cannot predict the exact result in advance.
Sample space (S)
The set of all possible outcomes of an experiment. For a coin flip: {Heads, Tails}. For a six-sided die: {1, 2, 3, 4, 5, 6}. Think of it as the complete list of everything that could happen.
Event
A subset of the sample space, representing one or more outcomes of interest. "Rolling an even number" = {2, 4, 6}. An event can be a single outcome or a collection of outcomes.
Mutually exclusive events (disjoint events)
Two events that cannot occur at the same time. P(A and B) = 0. Example: rolling a 3 and rolling a 5 on a single die throw. In simple terms, if one happens, the other cannot.
Independent events
Two events where the occurrence of one does not change the probability of the other. Mathematically, P(A and B) = P(A) x P(B). Example: flipping heads on a coin and rolling a 6 on a die.
Conditional probability
The probability of event A occurring given that event B has already occurred. Written P(A|B). In simple terms, "what is the probability of A, now that I know B happened?"
Bayes' rule (Bayes' theorem)
A formula for reversing conditional probabilities: P(A|B) = [P(B|A) x P(A)] / P(B). Lets you update your belief about A after observing B. Think of it as flipping the direction of a conditional probability.
Complement of an event
Everything in the sample space that is not in the event. Written A' or A-complement. P(A') = 1 - P(A). In simple terms, "the probability of A not happening."
Union (A or B)
The event that A occurs, or B occurs, or both. Written A U B.
Intersection (A and B)
The event that both A and B occur. Written A ∩ B.
Tree diagram
A branching diagram that maps out all possible outcomes of a multi-stage experiment, with probabilities on each branch. Useful for visualising conditional probabilities and sequential events.
Every probability problem starts with three ingredients.
Experiment: the process you are observing. It must have more than one possible outcome, and you cannot know in advance which outcome will occur.
Sample space (S): the full list of possible outcomes. For flipping two coins: {HH, HT, TH, TT}. Getting the sample space right is the foundation; if you miss an outcome or double-count, every probability you calculate from it will be wrong.
Events: subsets of the sample space you care about. "At least one head" = {HH, HT, TH}.
Set theory provides the language for combining events. The union (A or B) collects outcomes in either event. The intersection (A and B) collects outcomes in both. The complement (A') collects everything not in A.
Tree diagrams are particularly helpful for multi-stage experiments (e.g. drawing two cards in sequence). Each branch represents a possible outcome at that stage, and the probability of a path is the product of the probabilities along its branches.
There are two main ways to think about what probability means.
Frequentist perspective: probability is the long-run relative frequency of an event. If you flip a fair coin thousands of times, the proportion of heads approaches 0.5. This is the interpretation used in most of STAT 101.
Bayesian perspective: probability is a measure of belief or certainty, which gets updated as new evidence arrives. This interpretation drives Bayes' rule.
Regardless of interpretation, three properties always hold.
Every probability is between 0 and 1 inclusive.
The probability of the entire sample space is 1: P(S) = 1.
For mutually exclusive events, probabilities are additive: P(A or B) = P(A) + P(B).
These are the two workhorses for combining probabilities.
Addition rule ("or" questions)
For mutually exclusive events: P(A or B) = P(A) + P(B).
For events that are not mutually exclusive (they can overlap): P(A or B) = P(A) + P(B) - P(A and B). You subtract the intersection to avoid counting the overlap twice.
Multiplication rule ("and" questions)
For independent events: P(A and B) = P(A) x P(B).
For events that are not independent (the general case): P(A and B) = P(A|B) x P(B). This is the general multiplication rule, and it works whether or not the events are independent.
Conditional probability answers the question: "what is the probability of A, given that B has already happened?"
P(A|B) = P(A and B) / P(B)
The denominator P(B) acts as the new "universe." Once you know B occurred, you are no longer working with the full sample space; you are working only with the outcomes where B is true.
Conditional probability is central to medical testing, quality control and risk assessment. If a drug test is positive, what is the probability the person took the drug? That is a conditional probability question, and answering it correctly requires Bayes' rule.
This distinction trips up more students than almost any other concept in introductory probability.
Independent events: knowing one occurred tells you nothing about the other. P(A and B) = P(A) x P(B). Example: flipping a coin and rolling a die.
Disjoint (mutually exclusive) events: they cannot both happen. P(A and B) = 0. Example: drawing a heart and drawing a spade from the same single card.
The critical insight: disjoint events with non-zero probabilities are never independent. If you know A happened and A and B are disjoint, then B definitely did not happen, so knowing A changed the probability of B from P(B) to 0. That is the opposite of independence.
Bayes' rule lets you reverse a conditional probability. You know P(B|A) and want P(A|B).
P(A|B) = [P(B|A) x P(A)] / P(B)
The components have names. P(A) is the prior (your belief about A before seeing B). P(B|A) is the likelihood (how probable B is if A is true). P(A|B) is the posterior (your updated belief about A after observing B).
P(B) in the denominator is often expanded using the law of total probability: P(B) = P(B|A) x P(A) + P(B|A') x P(A').
Bayes' rule is used whenever you need to "go backwards" from an observed outcome to its cause. Classic example: a medical test comes back positive. You know the test's sensitivity (true positive rate) and specificity (true negative rate), and the prevalence of the disease. Bayes' rule tells you the probability the patient has the disease given the positive result.
Complement rule: P(A') = 1 - P(A)
Addition rule (general): P(A or B) = P(A) + P(B) - P(A and B)
Addition rule (mutually exclusive): P(A or B) = P(A) + P(B)
Multiplication rule (independent): P(A and B) = P(A) x P(B)
General multiplication rule: P(A and B) = P(A|B) x P(B)
Conditional probability: P(A|B) = P(A and B) / P(B)
P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}Law of total probability: P(B) = P(B|A) x P(A) + P(B|A') x P(A')
Spam filters use Bayes' rule: given that an email contains certain words, what is the probability it is spam? Medical screening relies on conditional probability to interpret test results, especially when the disease is rare (a positive test does not always mean you have the disease). Insurance companies use probability rules to price policies: the probability of multiple independent claims occurring together is the product of their individual probabilities.
Students often assume disjoint events are independent. They are not. If two events cannot happen together, knowing one happened tells you the other definitely did not, which is the opposite of independence.
A common error is forgetting to subtract the intersection when applying the addition rule to non-mutually-exclusive events. This double-counts the overlap.
Students frequently confuse P(A|B) with P(B|A). These are different quantities. "The probability of rain given clouds" is not the same as "the probability of clouds given rain."
Many students think a probability of 0.05 means an event is impossible. It means the event happens about 1 in 20 times, which is uncommon but far from impossible.
Expect at least one question requiring you to determine whether events are independent, disjoint, both or neither. Know the definitions and the mathematical tests.
Bayes' rule problems are common. You will typically be given P(B|A), P(A) and enough to compute P(B), then asked for P(A|B).
Be comfortable drawing and reading tree diagrams for two-stage experiments. The branches multiply; the final outcomes at the tips of the tree should sum to 1.
Know when to use the addition rule vs the multiplication rule. "Or" signals addition; "and" signals multiplication.
True or false: if P(A) = 0.3 and P(B) = 0.4, and A and B are independent, then P(A and B) = 0.12. (True)
Fill in the blank: the complement of an event with probability 0.7 has probability ____. (0.3)
True or false: disjoint events with non-zero probabilities are independent. (False)
Fill in the blank: P(A or B) for mutually exclusive events equals P(A) ____ P(B). (plus / +)
True or false: P(A|B) is always equal to P(B|A). (False)
Q: A bag contains 5 red and 3 blue marbles. You draw one marble. What is the probability of drawing red or blue?
A: These are mutually exclusive (a marble cannot be both red and blue). P(red or blue) = 5/8 + 3/8 = 1. This makes sense: every marble is either red or blue.
Q: A disease affects 1% of the population. A test has a 95% true positive rate and a 10% false positive rate. If a person tests positive, what is the probability they have the disease?
A: Using Bayes' rule. P(Disease) = 0.01. P(Positive|Disease) = 0.95. P(Positive|No Disease) = 0.10. P(Positive) = (0.95)(0.01) + (0.10)(0.99) = 0.0095 + 0.099 = 0.1085. P(Disease|Positive) = (0.95)(0.01) / 0.1085 = 0.0876, or about 8.8%. Even with a positive test, there is only about a 9% chance of having the disease, because the disease is rare.
Q: Events A and B are independent with P(A) = 0.6 and P(B) = 0.5. Find P(A or B).
A: P(A and B) = P(A) x P(B) = 0.6 x 0.5 = 0.3. P(A or B) = P(A) + P(B) - P(A and B) = 0.6 + 0.5 - 0.3 = 0.8.
Q: Are the events "rolling a 2" and "rolling an even number" on a single die mutually exclusive? Are they independent?
A: They are not mutually exclusive, because rolling a 2 satisfies both events. For independence: P(2) = 1/6. P(even) = 3/6 = 1/2. P(2 and even) = P(2) = 1/6. P(2) x P(even) = (1/6)(1/2) = 1/12. Since 1/6 is not equal to 1/12, they are not independent.
Conditional probability and independence are the backbone of random variables and probability distributions (covered in the next set of notes). The multiplication rule for independent events underpins the binomial distribution formula. Bayes' rule reappears in statistical inference when you study how to update beliefs with data. Tree diagrams connect to decision analysis and expected value calculations.
probability, sample space, event, experiment, mutually exclusive, disjoint, independent events, conditional probability, Bayes' rule, Bayes' theorem, addition rule, multiplication rule, complement, union, intersection, tree diagram, prior probability, posterior probability, likelihood, law of total probability, frequentist, Bayesian, STAT 101, introduction to statistics, Purdue