Source: Moore & Notz, Statistics: Concepts and Controversies (10th ed.), Chs. 14–22
Tags: inferential statistics, hypothesis testing, null hypothesis, alternative hypothesis, p-value, significance level, confidence interval, sampling distribution, Central Limit Theorem, CLT, Type I error, Type II error, statistical significance, STAT 101, intro statistics
Difficulty: Intermediate | Prerequisites: Descriptive Statistics (Document 1) and Normal Distribution / Z-Scores (Document 2). You need the mean, standard deviation, and z-score material before tackling this.
Descriptive statistics tells you what happened in your sample. Inferential statistics tells you what that sample suggests about the larger population you care about. The two main tools are confidence intervals (estimating a population parameter within a range) and hypothesis tests (deciding whether the data provide enough evidence against a claim). This is the conceptual heart of STAT 101 and where the course shifts from "calculate a number" to "make a decision with that number." Nearly half of exam content in most sections comes from this material.
Inferential statistics uses sample data to draw conclusions about a population. Confidence intervals give a plausible range for a population parameter. Hypothesis tests ask whether the data are surprising enough, under a stated assumption, to reject that assumption. Both rely on the idea that sample statistics vary predictably from sample to sample.
Population
The entire group you want to draw conclusions about. In simple terms, it is everybody (or everything) you are interested in studying, whether or not you can observe them all.
Sample
A subset of the population that you actually collect data from. Think of it as the slice of reality you can measure directly.
Parameter
A number that describes a characteristic of the population (e.g. population mean μ, population proportion p). Parameters are usually unknown and are what you are trying to estimate.
Statistic
A number computed from a sample (e.g. sample mean x̄, sample proportion p̂). Statistics are used to estimate parameters.
Sampling distribution
The distribution of a statistic over all possible samples of a given size. In simple terms, if you could repeat your study thousands of times, the sampling distribution shows how much your statistic would bounce around.
Central Limit Theorem (CLT)
For large enough sample sizes, the sampling distribution of the sample mean is approximately normal, regardless of the shape of the original population. This is the reason the normal distribution is so central to inference. Think of it as the theorem that makes all of inferential statistics work.
Standard error (SE)
The standard deviation of a sampling distribution. It measures how much a statistic typically varies from sample to sample. In simple terms, the standard error tells you how precise your estimate is: smaller SE means more precision.
Confidence interval
A range of plausible values for a population parameter, computed from sample data plus a margin of error. A 95% confidence interval means the method used will capture the true parameter in about 95% of all possible samples. Think of it as a net you cast: 95% of the time, the fish (true parameter) is inside.
Confidence level
The percentage of all possible samples for which the confidence interval procedure would capture the true parameter. Common levels: 90%, 95%, 99%. It is not the probability that the true parameter lies in your particular interval (a subtle but important distinction).
Margin of error
The "plus-or-minus" part of a confidence interval. It equals the critical value multiplied by the standard error. Larger samples produce smaller margins of error.
Null hypothesis (H₀)
The default claim about a population parameter, typically a statement of "no effect" or "no difference." In simple terms, the null hypothesis says nothing interesting is happening.
Alternative hypothesis (Hₐ or H₁)
The claim you are trying to find evidence for, typically a statement that there is an effect or a difference. Think of it as the research question expressed as a statistical statement.
Test statistic
A standardized value that measures how far the sample result is from what the null hypothesis predicts. For means, the test statistic is usually a z-score or t-score. In simple terms, it answers "how many standard errors away from the null value is my sample result?"
P-value
The probability of obtaining a test statistic as extreme as, or more extreme than, the one observed, assuming the null hypothesis is true. A small p-value means the data are unlikely under the null, which counts as evidence against it. Think of it as a measure of surprise: the smaller the p-value, the more surprising the result would be if the null were true.
Significance level (α, alpha)
The threshold you set in advance for rejecting the null hypothesis. The most common choice is α = 0.05. If the p-value is less than α, you reject H₀.
Type I error (false positive)
Rejecting the null hypothesis when it is actually true. The probability of a Type I error equals α. In simple terms, you concluded something is going on when it is not.
Type II error (false negative)
Failing to reject the null hypothesis when it is actually false. In simple terms, you missed a real effect.
Statistical significance
A result is statistically significant when the p-value falls below the chosen significance level. This does not necessarily mean the result is practically important or large in magnitude.
You observe a sample because you cannot (or do not want to) measure the entire population.
The sample statistic (e.g. x̄) is your best single guess for the population parameter (e.g. μ), but you know it will not be exactly right.
The sampling distribution tells you how much error to expect. The Central Limit Theorem guarantees that, for large samples, this distribution is approximately normal, which makes the maths tractable.
Structure: point estimate ± margin of error.
For a population mean with known σ: CI = x̄ ± z* × (σ / √n), where z* is the critical value from the standard normal distribution (e.g. z* = 1.96 for 95% confidence).
Increasing the sample size n makes the interval narrower (more precise).
Increasing the confidence level makes the interval wider (more certain but less precise).
The trade-off between confidence and precision is fundamental: you cannot have a very narrow interval and very high confidence at the same time without a large sample.
Correct: "We are 95% confident that the population mean falls between 42 and 58." This means the method produces intervals that capture the true mean 95% of the time across many samples.
Incorrect: "There is a 95% probability that the population mean is between 42 and 58." The population mean is a fixed (unknown) number, not a random variable. It is either in the interval or it is not.
State the hypotheses. Write H₀ and Hₐ in terms of the population parameter.
Choose the significance level. Typically α = 0.05, unless the problem specifies otherwise.
Compute the test statistic. For a test about a population mean: z = (x̄ − μ₀) / (σ / √n), where μ₀ is the value stated in H₀.
Find the p-value. Use the standard normal table (or calculator) to find the probability associated with the test statistic. One-tailed tests use one tail; two-tailed tests double the one-tail area.
Make a decision. If p-value ≤ α, reject H₀. If p-value > α, fail to reject H₀. State the conclusion in context.
Two-tailed (Hₐ: μ ≠ μ₀): you are looking for a difference in either direction.
Right-tailed (Hₐ: μ > μ₀): you are looking for evidence that the parameter is larger than the null value.
Left-tailed (Hₐ: μ < μ₀): you are looking for evidence that the parameter is smaller.
The choice depends on the research question, not on the data. You decide before looking at results.
Type I (rejecting a true null) is controlled by α. Setting α = 0.05 means you accept a 5% risk of this error.
Type II (failing to reject a false null) depends on the true parameter value, the sample size, and α. It is harder to control directly.
Reducing α makes Type I errors less likely but Type II errors more likely, and vice versa. The only way to reduce both is to increase sample size.
A large sample can produce a statistically significant result for a tiny, meaningless effect. A drug that lowers blood pressure by 0.1 mmHg might achieve p < 0.05 with 100,000 participants, but that reduction has no clinical value.
Always ask: is the observed effect large enough to matter in the real world? Report effect sizes or confidence intervals alongside p-values.
Standard error of the sample mean
SE = σ / √n
If σ is unknown (the usual case), substitute the sample standard deviation s.
Confidence interval for a population mean (σ known)
CI = x̄ ± z* × (σ / √n)
Common z* values: 1.645 for 90%, 1.96 for 95%, 2.576 for 99%.
Z test statistic for a population mean
z = (x̄ − μ₀) / (σ / √n)
Margin of error
ME = z* × (σ / √n)
Minimum sample size for a desired margin of error
n = (z* × σ / ME)²
Round up to the next whole number.
Clinical trials use hypothesis testing to decide whether a new treatment is more effective than a placebo. Political polls report confidence intervals to indicate how accurate their estimates are. Quality-control engineers test whether a production line's mean output meets a specification. In each case, the logic is the same: sample data, quantify uncertainty, make a decision or an estimate about the broader reality.
Students often interpret the p-value as the probability that the null hypothesis is true. It is not. The p-value is the probability of seeing data this extreme assuming the null is true, which is a different statement entirely.
Students sometimes think "fail to reject H₀" means "H₀ is proven true." It does not. It means the data did not provide strong enough evidence against H₀. Absence of evidence is not evidence of absence.
Students confuse statistical significance with importance. A statistically significant result can be trivially small. The p-value tells you about evidence, not about the size of the effect.
Students forget that the confidence level describes the method, not any single interval. Saying "there is a 95% chance the true mean is in this interval" is a common error on exams.
⚠️ Correct interpretation of p-values and confidence intervals is tested heavily and is where most marks are lost. Practise writing interpretations in full sentences.
⚠️ Know the five-step hypothesis testing framework and be ready to apply it to a scenario you have not seen before.
⚠️ Type I and Type II errors are a favourite for conceptual multiple-choice questions. Be able to identify each in a real-world scenario (e.g. "a jury convicts an innocent person" is a Type I error analogy).
⚠️ Expect at least one question on the relationship between sample size, confidence level, and margin of error.
⚠️ The Central Limit Theorem is the justification for using normal-based methods. Know what it says and when it applies (large n, any population shape).
True or false: A p-value of 0.03 means there is a 3% chance the null hypothesis is true.
Fill in the blank: The Central Limit Theorem states that the sampling distribution of the sample mean becomes approximately _______ as the sample size increases.
True or false: Increasing the confidence level from 95% to 99% makes the confidence interval narrower.
Fill in the blank: A Type I error occurs when you reject the null hypothesis and it is actually _______.
True or false: "Fail to reject H₀" is the same as "accept H₀."
Answers: 1. False (the p-value is the probability of the observed data, or more extreme, assuming H₀ is true). 2. Normal. 3. False (it makes it wider). 4. True. 5. False (failing to reject is not the same as proving H₀ true).
Q: A factory claims its light bulbs last an average of 1,000 hours. You test a sample of 50 bulbs and find a mean of 985 hours with a known population standard deviation of 40 hours. At the 0.05 significance level, is there evidence that the true mean is less than 1,000?
A: H₀: μ = 1,000. Hₐ: μ < 1,000 (left-tailed). z = (985 − 1,000) / (40 / √50) = −15 / 5.66 ≈ −2.65. The p-value for z = −2.65 is about 0.004. Since 0.004 < 0.05, reject H₀. There is sufficient evidence that the mean lifetime is less than 1,000 hours.
Q: A 95% confidence interval for the average commute time is (24.3, 31.7) minutes. A colleague says "there is a 95% probability that the true mean commute time is between 24.3 and 31.7 minutes." Is this correct?
A: No. The correct interpretation is: we are 95% confident that the interval (24.3, 31.7) contains the true population mean. The method captures the true mean in 95% of all samples, but for this particular interval, the true mean is either inside it or not. Probability does not apply to a fixed (unknown) quantity.
Q: Explain the difference between a Type I error and a Type II error in the context of a criminal trial.
A: A Type I error is convicting an innocent person (rejecting a true null hypothesis of "not guilty"). A Type II error is acquitting a guilty person (failing to reject a false null hypothesis). The justice system sets a high bar for conviction (beyond reasonable doubt), which keeps Type I error low but allows a higher Type II error rate.
Q: You want a 95% confidence interval with a margin of error no larger than 2 units. The population standard deviation is 10. What is the minimum sample size?
A: n = (z* × σ / ME)² = (1.96 × 10 / 2)² = (9.8)² = 96.04. Round up to n = 97.
Q: A study reports a statistically significant weight loss of 0.2 kg (p = 0.04) in a sample of 10,000 participants. Should you consider this result practically meaningful?
A: Statistical significance here is driven by the enormous sample size. A weight loss of 0.2 kg is negligible in practical terms. Statistical significance tells you the effect is unlikely to be zero; it does not tell you the effect is large enough to matter. Always evaluate the size of the effect alongside the p-value.
Inferential statistics rests on the descriptive tools from Document 1 (you need the sample mean and SD to compute a test statistic) and the normal distribution from Document 2 (the sampling distribution is approximately normal by the CLT). Beyond STAT 101, these ideas extend into regression analysis, ANOVA, and every branch of applied research. If you go on to take a second statistics course, nearly everything will be a variation on the confidence interval and hypothesis testing logic covered here.
inferential statistics, statistical inference, hypothesis testing, null hypothesis, alternative hypothesis, H0, Ha, p-value, significance level, alpha, confidence interval, confidence level, margin of error, sampling distribution, Central Limit Theorem, CLT, standard error, test statistic, z-test, Type I error, Type II error, false positive, false negative, statistical significance, practical significance, effect size, point estimate, sample size, power, STAT 101, intro to statistics, principles of statistics