Difficulty: Beginner | Prerequisites: Basic algebra
Big picture: Probability is the mathematical framework for reasoning about uncertainty, and it underpins every statistical method you will meet in this course. Random variables give us a way to attach numbers to the outcomes of random experiments so we can compute with them. If you are comfortable with fractions, basic set language (union, intersection), and the idea that an experiment can have multiple outcomes, you have what you need to start here.
Probability measures how likely an event is to occur, either by counting equally likely outcomes or by observing long-run frequencies. Random variables translate experiment outcomes into numbers, and they come in two flavours: discrete (countable outcomes) and continuous (any value in a range). Understanding these building blocks is essential before tackling any specific probability distribution.
Probability (frequentist interpretation)
The proportion of times an event occurs when a random experiment is repeated independently a large number of times. In simple terms, it is the long-run relative frequency of an outcome.
Classical probability
The probability of an event when every individual outcome is equally likely: P(event) = (number of ways the event can happen) / (total number of outcomes). Think of it as counting favourable outcomes and dividing by the total.
Mutually exclusive events
Two events that cannot both occur at the same time. If A and B are mutually exclusive, P(A or B) = P(A) + P(B). Think of it as: if one happens, the other is automatically ruled out.
Independent events
Two events where the occurrence of one has no effect on the probability of the other. If C and D are independent, P(C and D) = P(C) x P(D). In simple terms, knowing one happened tells you nothing about the other.
Discrete random variable
A random variable whose possible values form a countable set (e.g. 0, 1, 2, 3, ...). Think of it as outcomes you can list and count, like the number of heads in a series of coin flips.
Continuous random variable
A random variable that can take any real number within some interval. In simple terms, the possible values are not separated by gaps, like height or temperature.
Probability mass function (PMF)
A function that gives the probability of each possible value of a discrete random variable: f(x) = P(X = x). Think of it as a lookup table mapping each outcome to its probability.
Cumulative distribution function (CDF)
A function giving the probability that the random variable takes a value less than or equal to x: F(x) = P(X <= x). Works for both discrete and continuous variables. In simple terms, it answers "what is the probability of getting this value or lower?".
The frequentist view defines probability through repetition: flip a coin thousands of times and the proportion of heads converges on the true probability.
Classical probability applies when all outcomes are equally likely. You count the favourable outcomes and divide by the total.
Addition rule (mutually exclusive): When two events cannot co-occur, add their individual probabilities. P(A or B) = P(A) + P(B).
Multiplication rule (independent): When one event does not affect the other, multiply. P(C and D) = P(C) x P(D).
Discrete: outcomes you can list (number of defective items in a batch, number of customers arriving per hour). Probabilities are assigned by a PMF.
Continuous: outcomes measured on a scale with no gaps (weight, time, distance). Probabilities come from the area under a density curve, not from individual points. P(X = any single value) = 0 for continuous variables.
PMF maps each discrete outcome to its probability. All values must be between 0 and 1, and the full set must sum to 1.
CDF accumulates probability from the left: F(x) = P(X <= x). It is a non-decreasing function that starts at 0 and ends at 1. For discrete variables, it is a step function. For continuous variables, it is a smooth curve.
P(\text{event}) = \frac{\text{Number of favourable outcomes}}{\text{Total number of outcomes}}P(A \text{ or } B) = P(A) + P(B) \quad \text{(mutually exclusive events)}P(C \text{ and } D) = P(C) \times P(D) \quad \text{(independent events)}f(x) = P(X = x) \quad \text{(PMF, discrete variables)}F(x) = P(X \leq x) \quad \text{(CDF, discrete and continuous)}Classical probability sits behind every fair game, lottery calculation, and quality-control sampling plan. The addition and multiplication rules are the foundation of risk assessment: insurers combine independent event probabilities to price policies, and engineers multiply component-failure probabilities to estimate system reliability.
Students often confuse "mutually exclusive" with "independent." They are different concepts and, in fact, two events with non-zero probability cannot be both mutually exclusive and independent.
A common error is to add probabilities for independent events instead of multiplying. Addition applies to mutually exclusive events; multiplication applies to independent events.
Students sometimes assume P(X = x) gives useful probabilities for continuous random variables. For continuous variables, only intervals have non-zero probability; the probability at a single exact point is always zero.
Confusing the PMF with the CDF is another frequent mistake. The PMF gives the probability of each individual value; the CDF gives the accumulated probability up to and including that value.
⚠️ Expect questions that ask you to identify whether events are mutually exclusive, independent, or neither, and then to choose the correct rule.
⚠️ You will almost certainly be asked to compute a classical probability by counting outcomes.
⚠️ Know the difference between PMF and CDF cold. Exam problems may give you one and ask you to derive the other.
⚠️ Be prepared to classify a random variable as discrete or continuous from a word problem description.
True or false: If two events are mutually exclusive, they are also independent.
False. Mutually exclusive events with non-zero probability are never independent.
Fill in the blank: For a continuous random variable, P(X = 3) = ______.
True or false: The CDF is a non-decreasing function.
True.
Fill in the blank: The PMF values for all possible outcomes of a discrete random variable must sum to ______.
True or false: Classical probability requires that all outcomes are equally likely.
True.
Q: A bag contains 5 red and 3 blue marbles. You draw one at random. What is P(red)?
A: 5/8. There are 5 favourable outcomes out of 8 equally likely outcomes.
Q: Events A and B are mutually exclusive with P(A) = 0.3 and P(B) = 0.5. What is P(A or B)?
A: 0.8. Because the events are mutually exclusive, P(A or B) = P(A) + P(B) = 0.3 + 0.5.
Q: You roll a fair die and flip a fair coin independently. What is the probability of getting a 6 and heads?
A: 1/12. P(6) = 1/6, P(heads) = 1/2, and the events are independent, so P(6 and heads) = 1/6 x 1/2 = 1/12.
Q: A discrete random variable X has PMF: P(X=1) = 0.2, P(X=2) = 0.5, P(X=3) = 0.3. What is F(2)?
A: 0.7. The CDF at 2 is P(X <= 2) = P(X=1) + P(X=2) = 0.2 + 0.5.
Q: Is the temperature in a room a discrete or continuous random variable?
A: Continuous. Temperature can take any value within a range and is not restricted to countable outcomes.
These probability fundamentals feed directly into every named distribution you will study next (binomial, Poisson, normal, and so on). The rules for combining probabilities reappear in hypothesis testing and confidence intervals. The PMF and CDF concepts extend to continuous distributions through the probability density function (PDF) and its integral.
Related Terms / Search Tags
probability, frequentist probability, classical probability, equally likely outcomes, mutually exclusive, independent events, addition rule, multiplication rule, random variable, discrete random variable, continuous random variable, PMF, probability mass function, CDF, cumulative distribution function, intro to statistics, Purdue STAT