Source: Fundamental Statistics Concepts for Data Analysis
Difficulty: Introductory to Intermediate | Prerequisites: Descriptive Statistics and Data Foundations notes (the companion document to this one).
Tags: independent events, binomial experiment, probability distribution function, PDF, standard normal distribution, z-score, inferential statistics, central limit theorem, CLT, hypothesis testing, confidence interval, STAT 101
This material covers how probability works and how you use it to draw conclusions about populations from samples. It starts with the rules governing independent events and binomial experiments, moves through probability distribution functions and the standard normal distribution, and finishes with inferential statistics and the Central Limit Theorem. Together, these concepts form the reasoning chain that lets you go from "here is my sample" to "here is what I can say about the population." You need the descriptive statistics foundations (mean, standard deviation, variance) before this material will make full sense.
Probability provides the mathematical framework for quantifying uncertainty. Inferential statistics uses that framework, along with the Central Limit Theorem, to draw conclusions about a population based on sample data. The standard normal distribution is the bridge between raw data and probability calculations.
Independent events
Two events where the occurrence of one has no effect on the probability of the other. Think of it as: knowing the outcome of event A tells you nothing new about event B.
Binomial experiment
An experiment with a fixed number of independent trials, each having exactly two outcomes (success or failure), with the same probability of success on every trial. Think of it as the framework for any "how many times out of n" question.
Probability distribution function (PDF)
A mathematical description that assigns a probability to each possible outcome of a discrete random variable. In simple terms, it is the complete list of "what can happen" and "how likely is each outcome."
Standard normal distribution (Z-distribution)
A normal distribution with a mean of 0 and a standard deviation of 1. Think of it as the universal reference curve: you convert any normal variable into Z-scores so you can look up probabilities in a single table.
Z-score (standard score)
The number of standard deviations a data point sits above or below the mean. Calculated as z = (x − μ) / σ. Think of it as a way of expressing "how unusual is this value?"
Inferential statistics
Methods that use sample data to make predictions or draw conclusions about a larger population. In simple terms, this is how you go from "what my sample shows" to "what is probably true about the whole group."
Central Limit Theorem (CLT)
As the sample size increases, the sampling distribution of the sample mean approaches a normal distribution, regardless of the shape of the population distribution. Think of it as the reason the normal distribution appears everywhere in statistics.
Confidence interval
A range of values, constructed from sample data, that is likely to contain the true population parameter. Think of it as a margin of error around your best estimate.
Hypothesis testing
A formal procedure for deciding whether sample data provides enough evidence to reject a claim (the null hypothesis) about a population. Think of it as the statistical version of "innocent until proven guilty."
Two events A and B are independent if the outcome of one does not change the probability of the other.
Multiplication rule for independent events: P(A and B) = P(A) × P(B). This only holds when the events are independent.
Classic example: rolling a die and flipping a coin. The number on the die has no bearing on whether the coin lands heads. P(rolling a 3 and getting heads) = 1/6 × 1/2 = 1/12.
How to test for independence: if P(A | B) = P(A), the events are independent. In words: learning that B happened does not change what you think about A.
Dependent events, by contrast: drawing two cards from a deck without replacement. After the first draw, the deck has changed, so the probability of the second draw depends on the first.
A binomial experiment is a specific type of probability experiment defined by four conditions.
Fixed number of trials (n): you decide in advance how many times you will repeat the experiment.
Two outcomes per trial: each trial results in either "success" (probability p) or "failure" (probability q = 1 − p).
Constant probability: p does not change from one trial to the next.
Independence: the outcome of one trial does not affect any other trial.
When all four conditions hold, you can calculate the probability of getting exactly x successes in n trials using the binomial formula.
Everyday examples: the number of heads in 10 coin flips, the number of defective items in a batch of 50 when each item has a 2% defect rate, the number of patients who respond to a treatment out of 20.
A PDF gives every possible outcome of a random variable and the probability associated with it.
For a discrete random variable: the PDF is a table or formula listing each outcome and its probability. Every probability must be between 0 and 1, and the sum of all probabilities must equal 1.
For a continuous random variable: the PDF is a curve (a density function), and the probability of the variable falling within a range is the area under the curve over that range. The total area under the curve equals 1.
Key properties:
P(X = x) ≥ 0 for all x.
The sum (discrete) or integral (continuous) of all probabilities = 1.
Why it matters: the PDF is how you move from "what could happen" to "how likely is each outcome," which is the foundation for every probability calculation in the course.
The standard normal distribution is a special case of the normal distribution with mean μ = 0 and standard deviation σ = 1. It is denoted Z ~ N(0, 1).
Z-scores: any normally distributed variable X can be converted to Z using z = (x − μ) / σ. This lets you use a single Z-table to find probabilities for any normal distribution.
The 68-95-99.7 rule (empirical rule):
About 68% of values fall within 1 standard deviation of the mean.
About 95% fall within 2 standard deviations.
About 99.7% fall within 3 standard deviations.
Symmetry: the standard normal curve is perfectly symmetric about 0. P(Z < −a) = P(Z > a) for any positive a.
Why it is central to the course: hypothesis tests, confidence intervals, and probability calculations for normally distributed data all run through the Z-distribution. Once you can standardise a value and read a Z-table, you can answer most probability questions in the course.
Inferential statistics is the branch of statistics that uses sample data to make conclusions about a larger population.
Core techniques:
Hypothesis testing: start with a null hypothesis (no effect or no difference), collect data, and determine whether the data provides enough evidence to reject that hypothesis.
Confidence intervals: calculate a range around a sample statistic that, at a chosen confidence level (commonly 95%), is expected to contain the true population parameter.
Probability: underpins both of the above. Every test statistic and every interval relies on probability theory.
How it differs from descriptive statistics: descriptive statistics tells you what your data looks like. Inferential statistics tells you what your data means about the population it was drawn from.
The logic: you cannot measure an entire population, so you take a sample, calculate a statistic (like the sample mean), and use probability to estimate how close that statistic is to the true population parameter.
The CLT is one of the most important results in statistics. It states that as the sample size n increases, the sampling distribution of the sample mean approaches a normal distribution, regardless of the shape of the population distribution.
What "sampling distribution" means: if you took many samples of size n from the same population and plotted all the sample means, that distribution of means is the sampling distribution.
Key conditions:
The samples must be independent.
The sample size must be "large enough." A common rule of thumb is n ≥ 30, though this depends on how skewed the population is.
What the CLT gives you:
The mean of the sampling distribution equals the population mean (μ).
The standard deviation of the sampling distribution (called the standard error) equals σ / √n.
The shape of the sampling distribution is approximately normal for large n.
Why it matters so much: even if your population data is heavily skewed, lumpy, or otherwise non-normal, the CLT guarantees that sample means will be approximately normally distributed for large samples. This is what justifies using Z-tables and normal-distribution methods for hypothesis tests and confidence intervals in practice.
\text{Multiplication rule (independent events): } P(A \cap B) = P(A) \times P(B)\text{Binomial probability: } P(X = x) = \binom{n}{x} p^x (1-p)^{n-x}\text{Binomial mean: } \mu = np \quad \text{Binomial standard deviation: } \sigma = \sqrt{np(1-p)}\text{Z-score: } z = \frac{x - \mu}{\sigma}\text{Standard error of the mean: } SE = \frac{\sigma}{\sqrt{n}}\text{Confidence interval for the mean: } \bar{x} \pm z^* \cdot \frac{\sigma}{\sqrt{n}}where z* is the critical Z-value for the chosen confidence level (1.96 for 95%).
Quality control (binomial): a factory tests 100 items off the line. If the known defect rate is 3%, the binomial distribution tells you the probability of finding, say, 5 or more defectives in that batch.
Polling and elections (CLT and confidence intervals): political polls report a margin of error (e.g. ± 3%). That margin comes from a confidence interval, which relies on the CLT to justify treating the sampling distribution of the poll's percentage as approximately normal.
Medical trials (hypothesis testing): a pharmaceutical company tests whether a new drug lowers blood pressure more than a placebo. The hypothesis test formalises whether the observed difference is large enough to be unlikely under random chance alone.
"Independent events means the events cannot happen at the same time." That describes mutually exclusive events, which is a completely different concept. Independent events can and often do occur together; the key is that one does not change the probability of the other.
"The CLT says the population becomes normal as you take larger samples." The population distribution does not change at all. The CLT says the distribution of the sample means becomes normal, not the underlying population.
"A 95% confidence interval means there is a 95% probability the true parameter is inside this specific interval." Once the interval is calculated, the parameter is either in it or not. The 95% refers to the long-run success rate of the method: if you repeated the sampling and interval construction many times, about 95% of those intervals would contain the true parameter.
"You need a normal population to use Z-scores." For individual data points, yes, you need the population to be normal (or approximately so). But for sample means, the CLT lets you use Z-scores even when the population is non-normal, provided the sample is large enough.
⚠️ Be able to verify whether an experiment meets the four binomial conditions. Exams often present a scenario and ask whether it qualifies.
⚠️ Know how to calculate and interpret a Z-score. "A Z-score of 2.0 means the value is 2 standard deviations above the mean" is the kind of sentence you need to be able to produce.
⚠️ Understand the CLT well enough to state it in your own words and explain why sample size matters.
⚠️ Know the difference between independent and mutually exclusive events. This is one of the most commonly tested distinctions in introductory statistics.
⚠️ Be comfortable reading a Z-table: given a Z-score, find the area to the left, to the right, or between two Z-values.
True or False: If two events are independent, they cannot be mutually exclusive (assuming both have non-zero probability).
Answer: True. If A and B are mutually exclusive, P(A and B) = 0, but if independent, P(A and B) = P(A) × P(B) > 0.
Fill in the blank: In a binomial experiment, q = ___.
Answer: 1 − p (the probability of failure).
True or False: The Central Limit Theorem requires the population to be normally distributed.
Answer: False. The CLT works regardless of the population distribution, given a large enough sample.
Fill in the blank: The standard normal distribution has a mean of ___ and a standard deviation of ___.
Answer: 0; 1.
True or False: A Z-score of −1.5 means the observation is 1.5 standard deviations below the mean.
Answer: True.
Q: You flip a fair coin 8 times. What is the probability of getting exactly 5 heads?
A: This is a binomial problem with n = 8, x = 5, p = 0.5. P(X = 5) = C(8,5) × (0.5)^5 × (0.5)^3 = 56 × (1/256) = 56/256 ≈ 0.2188 (about 21.9%).
Q: A population has mean μ = 100 and standard deviation σ = 15. What is the Z-score for an observation of 130?
A: z = (130 − 100) / 15 = 30 / 15 = 2.0. The observation is 2 standard deviations above the mean.
Q: Explain, in one or two sentences, what the Central Limit Theorem guarantees.
A: The CLT guarantees that the sampling distribution of the sample mean will be approximately normal for sufficiently large sample sizes, no matter what the population distribution looks like. The mean of that sampling distribution equals the population mean, and its standard deviation (the standard error) equals σ / √n.
Q: Are the following events independent? Drawing a red card from a standard deck, then, without replacing it, drawing a second red card.
A: No. After the first draw the deck has changed (25 red out of 51 remaining, rather than 26 out of 52), so the probability of the second draw depends on the outcome of the first. These are dependent events.
Q: A 95% confidence interval for a population mean is (48.2, 51.8). What does this tell you?
A: We are 95% confident that the true population mean falls between 48.2 and 51.8. If we repeated the sampling and interval construction many times, about 95% of the resulting intervals would contain the true mean.
Independent events and binomial experiments rely on the descriptive foundations: you need to understand mean and standard deviation to work with the binomial mean (np) and standard deviation (√(np(1−p))).
The standard normal distribution is the bridge between descriptive and inferential statistics. You use descriptive tools (mean, standard deviation) to compute a Z-score, then use the Z-distribution to find probabilities and make inferences.
The Central Limit Theorem connects directly to confidence intervals and hypothesis testing. Without the CLT, there would be no justification for using normal-distribution methods on sample means from non-normal populations.
Independent events, dependent events, mutually exclusive, multiplication rule, binomial distribution, binomial experiment, binomial probability, n choose x, combinations, probability distribution function, PDF, CDF, discrete random variable, continuous random variable, standard normal distribution, Z-distribution, Z-score, standard score, bell curve, empirical rule, 68-95-99.7 rule, inferential statistics, hypothesis testing, null hypothesis, confidence interval, margin of error, Central Limit Theorem, CLT, sampling distribution, standard error, sample mean, population mean, STAT 101, intro probability