Difficulty: Beginner to Intermediate | Prerequisites: Foundations of Statistics notes, Variables and Measurement Scales notes
Probability is the bridge between descriptive and inferential statistics. Descriptive statistics tells you what happened in your data; probability gives you a framework for predicting what is likely to happen next, or how confident you can be in generalising from a sample. This chapter introduces the language of probability (outcomes, events, independence), the main approaches to assigning probabilities, and the concept of continuous random variables, which sets the stage for the normal distribution later in the course.
A probability is a number between 0 and 1 that expresses how likely an outcome is. You can arrive at probabilities subjectively (personal belief), theoretically (equally likely outcomes), or empirically (observed frequencies). Events can be independent, dependent, or mutually exclusive, and you will need conditional probability to handle situations where prior information changes the odds. Continuous random variables are described by density curves rather than lists of individual probabilities.
Probability
A number between 0 and 1 that reflects the likelihood of occurrence of some outcome.
In simple terms, 0 means impossible, 1 means certain, and everything in between is "how likely."
Outcome
Any one of the possible results of an experiment.
Event
A subset of the possible outcomes of an experiment.
Think of it as one or more outcomes grouped together. "Rolling an even number" on a die is the event {2, 4, 6}.
Trial
A single repetition or instance of the experiment.
Mutually exclusive events
Events A and B are mutually exclusive if both cannot occur at the same time.
In simple terms, if one happens, the other is ruled out. Example: a single coin flip cannot be both heads and tails.
Independent events
Two events are independent if the occurrence of event A has no effect on the probability of event B.
Think of it as "knowing A happened tells you nothing new about B." Example: two separate coin flips.
Dependent events
Events where the occurrence of one changes the probability of the other.
Conditional probability
The probability of some event occurring given information about what has already occurred, written P(A | B).
In simple terms, "what is the chance of A, now that I know B happened?"
Union (A ∪ B)
The set of outcomes that belong to A, or B, or both.
Intersection (A ∩ B)
The set of outcomes that belong to both A and B simultaneously.
Subjective probability
A probability interpreted as a personal measure of the strength of belief that an outcome will occur.
In simple terms, your informed guess based on experience or intuition, not on formal calculation.
Theoretical (classical) probability
Probability calculated by assuming all outcomes are equally likely. P(event) = number of favourable outcomes / total number of outcomes.
Example: for a fair coin, P(Heads) = 1/2.
Continuous random variable
A variable that can take any value within an interval (not just whole numbers). Its probability distribution is described by a density curve.
Probability density function (PDF)
The function f(x) that describes the density curve for a continuous random variable. The area under the curve over an interval gives the probability that the variable falls in that interval.
Density curve
A smooth curve representing the probability distribution of a continuous random variable. The total area under the curve equals 1.
Subjective approach: based on personal judgement or experience. Useful when no data or theoretical model exists, but inherently informal.
Theoretical (classical) approach: relies on equally likely outcomes. Works well for simple, symmetric settings (coins, dice, cards).
Empirical (relative frequency) approach: probability is estimated from observed data. P(event) ≈ (number of times event occurred) / (total number of trials). As the number of trials grows, the estimate converges on the true probability.
Mutually exclusive vs independent: these are different ideas. Two events can be mutually exclusive without being independent, and vice versa. In fact, if two events with non-zero probability are mutually exclusive, they cannot be independent (knowing one occurred tells you the other did not).
Union and intersection are the building blocks of probability rules such as the addition rule and the multiplication rule, which you will use extensively.
Conditional probability reduces the sample space. When you condition on B, you are restricting attention to only those outcomes where B has occurred, then asking how likely A is within that restricted space.
Discrete: takes on a countable number of values. Example: number of heads in 10 coin flips.
Continuous: can take any value in a range. Example: the exact distance you drive to work.
The distinction can be subtle. The number of miles you drive to work (counted as whole miles on the odometer) is discrete. The actual distance (measured with arbitrary precision) is continuous.
For a continuous variable, the probability of any single exact value is 0. Probabilities are defined over intervals.
The probability that X falls between a and b equals the area under f(x) from a to b.
The total area under any valid density curve is exactly 1.
f(x) ≥ 0 for all x (the curve never dips below the horizontal axis).
Theoretical probability (equally likely outcomes):
P(A) = (number of outcomes favourable to A) / (total number of outcomes in the sample space)
Complement rule:
P(A') = 1 – P(A)
Where A' is the complement of event A (everything that is not A).
Area under the density curve:
P(a ≤ X ≤ b) = ∫ from a to b of f(x) dx
(You will not usually need to compute integrals by hand in an introductory course; the idea is that probability = area.)
Insurance companies use probability models to set premiums: the likelihood of a claim determines how much a policyholder pays. Weather forecasts express precipitation chance as a probability (e.g. "30% chance of rain" is a blend of empirical and model-based probability). Continuous random variables and density functions are the mathematical backbone of quality control, where engineers model measurement tolerances as distributions.
Students often confuse mutually exclusive with independent. They are distinct concepts, and if two events (each with non-zero probability) are mutually exclusive, they are necessarily dependent.
A common error is thinking P(A | B) is the same as P(B | A). They are generally different. "The probability of rain given clouds" is not the same as "the probability of clouds given rain."
Students sometimes believe a continuous random variable can have a non-zero probability at a single point. It cannot; probabilities for continuous variables are always over intervals.
The subjective approach is sometimes dismissed as "just guessing." It is more structured than that: it incorporates experience and available evidence, and it is formally used in Bayesian statistics.
⚠️ Be able to identify whether two events are mutually exclusive, independent, both, or neither, given a scenario.
⚠️ Know how to compute simple theoretical probabilities (favourable outcomes / total outcomes).
⚠️ Understand conditional probability notation: P(A | B) means "probability of A given B."
⚠️ Know the difference between discrete and continuous random variables, and that continuous probabilities correspond to areas under a curve.
⚠️ The complement rule (P(A') = 1 – P(A)) appears in many exam problems, often as the quickest route to the answer.
True or False: Mutually exclusive events are always independent.
Fill in the blank: P(A | B) denotes the probability of A ______ B has occurred.
True or False: For a continuous random variable, P(X = 3.5) = 0.
Fill in the blank: The total area under a valid probability density curve equals ______.
True or False: The subjective approach assigns probabilities based on equally likely outcomes.
Answers: 1. False (they are dependent if both have non-zero probability). 2. Given that. 3. True. 4. 1. 5. False (that is the theoretical/classical approach; subjective is based on personal belief).
Q: A bag contains 4 red marbles and 6 blue marbles. You draw one marble at random. What is P(red)?
A: P(red) = 4/10 = 0.4.
Q: Events A and B are mutually exclusive. P(A) = 0.3 and P(B) = 0.5. Are A and B independent? Explain.
A: No. If A and B were independent, P(A and B) would equal 0.3 × 0.5 = 0.15. But because they are mutually exclusive, P(A and B) = 0, which is not 0.15. So they are dependent.
Q: What is the complement rule, and why is it useful?
A: P(A') = 1 – P(A). It is useful when computing P(A) directly is hard but computing P(A') (the probability of everything except A) is straightforward.
Q: Explain why, for a continuous random variable, probabilities are defined over intervals rather than at individual points.
A: Because the variable can take infinitely many values over any range, the probability assigned to any single exact value is 0. Meaningful probabilities come from the area under the density curve across an interval.
Q: A weather service says there is a 70% chance of rain tomorrow. Is this an example of subjective, theoretical, or empirical probability?
A: It is primarily empirical (based on historical data and model output), though it may incorporate subjective judgement from the forecaster.
Probability is the engine behind inferential statistics. Confidence intervals and hypothesis tests (covered later) rely on probability distributions to quantify uncertainty. The concept of the density curve introduced here leads directly to the normal distribution, the most important distribution in introductory statistics. Conditional probability also connects to Bayes' theorem, which appears in more advanced coursework and has wide applications in machine learning and medical testing.
probability, outcome, event, trial, sample space, mutually exclusive, independent events, dependent events, conditional probability, union, intersection, complement rule, subjective probability, theoretical probability, classical probability, empirical probability, relative frequency, continuous random variable, discrete random variable, probability density function, PDF, density curve, area under the curve, P(A|B), STA 2023, Chapter 3, Principles of Statistics, UCF