Source: STAT 3021 Lecture Notes, Ch. 1–2
Difficulty: Introductory | Prerequisites: Basic calculus (double integrals) Tags: probability, statistics, sample space, events, sets, Venn diagrams, conditional probability, independence, STAT 3021, University of Minnesota
Big picture: This material covers the opening foundations of a university-level probability and statistics course. It establishes what probability and statistics each do, how they relate to one another, and then introduces the formal language of probability: sets, sample spaces, events, and the rules that govern them. If you are joining the course late or revising from scratch, start here. Everything that follows in the course (random variables, distributions, inference) builds on these definitions.
Probability measures how likely something is to happen, on a scale from 0 to 1. Statistics works the other way round: it takes a sample from a larger population and uses that sample to estimate what is true of the whole. The formal language underneath both is set theory, which gives you sample spaces, events, and the rules (like conditional probability and independence) for combining them.
Probability
The likelihood of something occurring, expressed as a value from 0 to 1. A probability of 0 means the event cannot happen; a probability of 1 means it is certain.
In simple terms, it is a number that tells you how likely something is.
Statistics
The discipline of collecting, analysing, and interpreting data from a sample in order to draw conclusions about a larger population.
Think of it as the reverse of probability: instead of knowing the population and predicting the sample, you observe the sample and infer properties of the population.
Population
The entire collection of individuals, items, or measurements you are interested in studying.
In simple terms, this is the "whole picture" you want to understand but usually cannot measure directly.
Sample
A subset of the population that you actually observe and collect data from.
Think of it as a representative slice. If you want to know the average weight of all fish in a lake, you catch 1,000 and weigh those. Those 1,000 fish are your sample.
Sample mean
The arithmetic average of values in your sample. It estimates the true population mean but will almost never match it exactly, because of random variation.
Set
A collection of distinct objects, written with curly braces. For example, A = {apple, orange, banana}.
In simple terms, a set is just a group of things.
Empty set (null set), denoted ∅
A set that contains no elements at all.
Experiment
Any process that generates a set of data. Rolling a die, drawing cards, measuring fish weights: all experiments.
Sample space (S)
The set of all possible outcomes of an experiment. If you roll a standard die, S = {1, 2, 3, 4, 5, 6}.
Think of it as the full menu of things that could happen.
Event
A subset of the sample space. An event is any collection of outcomes you are interested in.
Conditional probability
The probability of an event A given that another event B has already occurred, written P(A | B).
In simple terms, this answers the question: "Now that I know B happened, how likely is A?"
Independence
Two events A and B are independent if knowing that one occurred does not change the probability of the other. Formally, P(A | B) = P(A).
Think of it as: the two events have nothing to do with each other.
What statistics does
You have a large population (e.g. every fish in Lake Superior) and you want to know something about it (e.g. the average weight).
Measuring the entire population is usually impractical or impossible.
Instead, you draw a sample (e.g. 1,000 fish) and compute a statistic from that sample (e.g. the sample mean).
The sample mean will almost never equal the true population mean exactly. One researcher might get 3.53; the actual value might be 3.43. Another researcher sampling independently could get a different number entirely.
This variation is not a flaw. It is the inherent randomness of sampling. Statistics provides the tools to quantify and account for that randomness.
How probability and statistics relate
Probability works "forward": given a known population, it models what samples are likely to look like. This is probability modelling.
Statistics works "backward": given an observed sample, it infers properties of the unknown population.
The two directions are complementary. You need probability theory to understand sampling variability before you can do statistical inference.
Calculus prerequisites
This course requires comfort with basic calculus, including double integrals.
If you are rusty, work through a calculus review before the material on continuous distributions arrives.
Sets
A set is a collection of distinct objects, enclosed in curly braces: A = {apple, orange, banana}.
The empty set (null set) is a set with no elements, written ∅.
Sets are the foundation of probability. Every probability statement is ultimately a statement about sets.
Experiments and sample spaces
An experiment is any process that generates data. Rolling a die, flipping a coin, and drawing balls from an urn are all experiments.
The sample space S is the set of every possible outcome of the experiment.
Coin flip: S = {H, T}
Rolling a die: S = {1, 2, 3, 4, 5, 6}
Testing three electrical components for defects (D = defective, N = non-defective): S = {NNN, NND, NDN, DNN, NDD, DND, DDN, DDD}
Events
An event is any subset of the sample space.
Example: "exactly two defective components" corresponds to the event E = {DDN, DND, NDD}.
Events can be combined using set operations (union, intersection, complement). Venn diagrams are the standard visual tool for reasoning about these operations.
Set operations to know
Union (A ∪ B): all outcomes in A or B or both.
Intersection (A ∩ B): outcomes in both A and B.
Complement (A'): outcomes in the sample space that are not in A.
These operations follow standard rules (commutative, associative, distributive, De Morgan's laws). Practise drawing Venn diagrams until you can read these fluently.
Conditional probability
P(A | B) is the probability of event A occurring, given that event B has occurred.
The formula is: P(A | B) = P(A ∩ B) / P(B), provided P(B) > 0.
Conditioning narrows the sample space. Once you know B happened, you restrict attention to outcomes inside B and ask how many of those also fall in A.
Independence
Events A and B are independent when knowing one tells you nothing about the other.
Formally: P(A ∩ B) = P(A) × P(B), or equivalently P(A | B) = P(A).
Independence is a property you check or assume. It is not something you can see by looking at two events in isolation.
Example: successive coin flips are independent. The result of the first flip has no bearing on the second.
P(A \mid B) = \frac{P(A \cap B)}{P(B)}, \quad P(B) > 0\text{Independence: } P(A \cap B) = P(A) \cdot P(B)Venn diagram rules to remember:
Two overlapping circles inside a rectangle (the sample space S).
The overlap region = A ∩ B.
Everything in either circle = A ∪ B.
Everything outside a circle but inside the rectangle = the complement of that event.
De Morgan's laws: (A ∪ B)' = A' ∩ B', and (A ∩ B)' = A' ∪ B'. Practise verifying these on a diagram.
Sampling and estimation are how quality control works in manufacturing: rather than testing every unit off the line, you test a sample and infer the defect rate for the whole batch. Conditional probability underpins medical testing, where the question is not "how accurate is the test?" but "given a positive result, how likely is it that the patient actually has the condition?" (This is Bayes' theorem in action, which builds directly on conditional probability.)
Students often think the sample mean should equal the population mean. It will not, except by coincidence. The whole point of statistics is to quantify how far off it might be.
Students sometimes treat "unlikely" as "impossible." A probability of 0.01 is small, but over many trials it will happen. Rare events are not zero-probability events.
Confusing independent events with mutually exclusive events. Mutually exclusive events cannot happen at the same time (P(A ∩ B) = 0). Independent events can happen at the same time; it is just that one does not affect the other's probability. These are very different properties.
Assuming conditional probability is symmetric. P(A | B) is generally not the same as P(B | A). Getting these confused is one of the most common errors on exams.
⚠️ You will be expected to list sample spaces correctly. Practise writing them out for small experiments (coin flips, dice rolls, defective components). A missing or duplicated outcome loses marks.
⚠️ Conditional probability and independence are tested heavily. Know the formulas, and know how to verify independence by checking whether P(A ∩ B) = P(A) × P(B).
⚠️ Set operations (union, intersection, complement, De Morgan's laws) appear throughout the course. If you cannot manipulate them fluently now, every later topic will be harder.
⚠️ The distinction between "probability" (population to sample) and "statistics" (sample to population) is a conceptual question that appears early in exams.
True or false: The sample mean is always equal to the population mean. (False. It varies from sample to sample.)
Fill in the blank: Statistics goes from ______ to ______, while probability goes from ______ to ______. (sample to population; population to sample)
True or false: If two events are mutually exclusive, they are independent. (False. Mutually exclusive events with non-zero probabilities are never independent.)
Fill in the blank: The set of all possible outcomes of an experiment is called the ______. (sample space)
True or false: P(A | B) is always equal to P(B | A). (False.)
Q: You roll a fair six-sided die. Write out the sample space. What is the probability of rolling a number greater than 4?
A: S = {1, 2, 3, 4, 5, 6}. The event "greater than 4" = {5, 6}. P = 2/6 = 1/3.
Q: A bag contains 3 red and 2 blue marbles. You draw one marble. Define the sample space and find P(red).
A: S = {R1, R2, R3, B1, B2} (or equivalently, 5 equally likely outcomes). P(red) = 3/5.
Q: Events A and B satisfy P(A) = 0.4, P(B) = 0.5, and P(A ∩ B) = 0.2. Are A and B independent?
A: Check whether P(A ∩ B) = P(A) × P(B). We have 0.2 = 0.4 × 0.5 = 0.2. Yes, A and B are independent.
Q: Using the same events, find P(A | B).
A: P(A | B) = P(A ∩ B) / P(B) = 0.2 / 0.5 = 0.4. (Notice this equals P(A), consistent with independence.)
Q: Explain, in one sentence, why "probability" and "statistics" are described as working in opposite directions.
A: Probability starts with a known population and predicts what samples will look like; statistics starts with an observed sample and infers properties of the unknown population.
Conditional probability leads directly into Bayes' theorem, which is the foundation of Bayesian inference and appears in nearly every applied statistics course. The set-theory language introduced here (sample spaces, events, set operations) is the same language used to define random variables in Chapter 3. If these definitions are not solid, the move to probability distributions will feel abrupt.
probability, statistics, sample, population, sample mean, population mean, sampling variability, set, empty set, null set, sample space, event, experiment, outcome, union, intersection, complement, Venn diagram, De Morgan's laws, conditional probability, independence, P(A given B), P(A|B), Bayes, STAT 3021, intro to probability and statistics, University of Minnesota