Source: Comprehensive Guide to Probability Theory and Applications
Tags: probability, sample space, events, inclusion-exclusion, complement rule, probability axioms, random experiment, outcomes, set theory in probability
Difficulty: Introductory Prerequisites: Basic set theory (unions, intersections, complements). Comfort with summation notation helps but is not essential.
Probability theory is the mathematical language for reasoning about uncertainty. It underpins statistics, machine learning, finance, engineering, and any discipline where outcomes are not fully determined in advance. This first set of notes covers the absolute foundations: what a sample space is, how probability is assigned to events, and the core rules (inclusion-exclusion, complement) you will use in every subsequent topic. If you are joining the course late, start here before touching distributions or Bayes' Theorem.
Every random experiment has a sample space (the full set of possible outcomes). An event is any subset of that space, and its probability is the sum of the probabilities of its outcomes. Two key rules, the inclusion-exclusion principle and the complement rule, let you combine and manipulate event probabilities without re-counting from scratch.
Sample space (S)
The complete set of all possible outcomes of a random experiment. For a single coin toss, S = {H, T}. For a six-sided die roll, S = {1, 2, 3, 4, 5, 6}. Think of it as the menu of everything that could possibly happen in one run of the experiment.
Event
Any subset of the sample space. An event can be a single outcome or a collection of outcomes. "Rolling an even number" is the event {2, 4, 6}. In simple terms, an event is a question you can answer with "did it happen or not?" after running the experiment.
Probability of an outcome, p(x)
A number between 0 and 1 assigned to each outcome x in the sample space such that the probabilities of all outcomes sum to 1. Think of it as the "weight" each outcome carries in the experiment.
Probability of an event, P(A)
The sum of the probabilities of every outcome inside event A. In simple terms, you add up the weights of all the outcomes that count as A happening.
Inclusion-exclusion principle
A rule for finding the probability of the union of two events without double-counting their overlap. Think of it as: "add both, then subtract what you counted twice."
Complement rule
The probability that an event does not happen equals 1 minus the probability that it does: P(Aᶜ) = 1 − P(A). In simple terms, if there is a 30% chance of rain, there is a 70% chance of no rain.
Every probability problem begins by defining the sample space S, the full list of things that can happen.
Each outcome x in S gets a probability p(x) ≥ 0.
The probabilities of all outcomes must sum to 1.
This is one of the Kolmogorov axioms, the bedrock of probability theory.
An event A is a subset of S.
P(A) = Σ (for all x in A) p(x).
You simply add up the individual outcome probabilities for every outcome in A.
If the sample space is finite and all outcomes are equally likely (uniform distribution), this simplifies to P(A) = |A| / |S|, the number of favourable outcomes divided by the total.
For two events A and B:
P(A ∪ B) = P(A) + P(B) − P(A ∩ B)
Why the subtraction? Outcomes that belong to both A and B get counted once in P(A) and once in P(B), so you remove the duplicate.
This extends to three or more events, alternating additions and subtractions, though the two-event form is what exams tend to test first.
P(Aᶜ) = 1 − P(A)
Useful when computing P(A) directly is awkward but computing "everything except A" is straightforward.
Classic use: "probability of at least one success" is easier as 1 − P(zero successes).
Name | Formula |
|---|---|
Probability of an event | P(A) = Σ p(x) for x ∈ A |
Inclusion-exclusion (two events) | P(A ∪ B) = P(A) + P(B) − P(A ∩ B) |
Complement rule | P(Aᶜ) = 1 − P(A) |
Probability foundations appear every time you assess risk or make decisions under uncertainty. Insurance companies use these rules to price policies. Software engineers apply them when estimating error rates in systems with multiple independent failure points. Even a simple question like "what is the chance that at least one server in a cluster goes down tonight?" is a complement-rule problem.
Students often assume "equally likely outcomes" is always the default. It is only true for uniform distributions (fair dice, fair coins). Most real problems have non-uniform sample spaces.
Confusing "or" (union, ∪) with "and" (intersection, ∩) is a persistent exam error. "A or B" means at least one happens. "A and B" means both happen.
Forgetting to subtract the intersection when using inclusion-exclusion. If you just add P(A) + P(B), you overcount whenever A and B overlap.
Applying the complement rule but forgetting to define the complement correctly. "Not rolling a 6" on a single die is {1, 2, 3, 4, 5}, not {6}.
⚠️ The inclusion-exclusion formula is tested frequently, often with three events. Make sure you can extend it beyond two.
⚠️ Complement-rule questions are a favourite because they test whether you can spot the shortcut. If a question says "at least one," think complement.
⚠️ Know your sample spaces cold. Getting S wrong at the start cascades through the entire problem.
True or false: The probabilities of all outcomes in a sample space must sum to 1.
Fill in the blank: P(A ∪ B) = P(A) + P(B) − ______.
True or false: P(Aᶜ) = 1 + P(A).
A fair die is rolled. What is P(rolling a number greater than 4)?
True or false: If A and B are mutually exclusive, P(A ∩ B) = 0.
Answers: 1. True. 2. P(A ∩ B). 3. False, it is 1 − P(A). 4. 2/6 = 1/3. 5. True.
Q: An urn contains 4 blue balls and 5 red balls. What is the probability of drawing a blue ball at random?
A: P(blue) = 4/9. There are 4 favourable outcomes out of 9 equally likely outcomes.
Q: Events A and B have P(A) = 0.4, P(B) = 0.5, and P(A ∩ B) = 0.2. What is P(A ∪ B)?
A: P(A ∪ B) = 0.4 + 0.5 − 0.2 = 0.7.
Q: The probability of rain tomorrow is 0.35. What is the probability it does not rain?
A: P(no rain) = 1 − 0.35 = 0.65.
Q: A fair six-sided die is rolled. What is the sample space, and what is the probability of rolling an odd number?
A: S = {1, 2, 3, 4, 5, 6}. The odd outcomes are {1, 3, 5}, so P(odd) = 3/6 = 1/2.
Q: Why does the inclusion-exclusion formula subtract P(A ∩ B)?
A: Because outcomes in both A and B are counted once in P(A) and once in P(B). Without subtracting the intersection, those outcomes are double-counted.
These rules are the prerequisite for conditional probability and Bayes' Theorem (next set of notes). You cannot compute P(A|B) without first understanding P(A ∩ B).
The complement rule becomes essential when working with distributions, particularly the binomial distribution, where "at least one" problems are standard.
Set operations (union, intersection, complement) connect directly to Boolean logic in computer science, so this material also reinforces topics in discrete maths.
probability basics, sample space definition, event probability, Kolmogorov axioms, inclusion-exclusion formula, complement rule probability, union of events, intersection of events, mutually exclusive events, random experiment, equally likely outcomes, uniform probability, set theory probability, STAT 101, CS foundations Purdue, probability theory introduction