Difficulty: Intermediate | Prerequisites: Basic understanding of sampling distributions, standard deviation, and the normal distribution.
Hypothesis testing is one of the central tools in inferential statistics: you collect sample data and ask whether it provides enough evidence to draw a conclusion about a population. This set of notes covers the choice between t-tests and z-tests, the assumptions that must hold for each, and the concept of degrees of freedom. If you are comfortable with the normal distribution and the idea of a sampling distribution, you are ready for this material. If those are fuzzy, revisit them first, as everything here builds on them directly.
T-tests are used when the population standard deviation is unknown (which is most of the time), while z-tests require it to be known. The t-distribution accounts for extra uncertainty from estimating the standard deviation, but with large samples it converges on the z-distribution. Degrees of freedom control the shape of the t-distribution and reflect how many values are free to vary once you fix a sample statistic.
T-test
A hypothesis test that uses the t-distribution to evaluate whether a sample mean (or difference between means) differs significantly from a hypothesised value. Used when the population standard deviation (sigma) is unknown and must be estimated from the sample.
Think of it as: the realistic version of the z-test, designed for the messy situations where you do not already know sigma.
Z-test
A hypothesis test that uses the standard normal (z) distribution. Requires that the population standard deviation is known.
Think of it as: the textbook-ideal test you would use if you had perfect population information, which in practice you almost never do.
Sampling distribution
The probability distribution of a statistic (such as the sample mean) computed over all possible samples of a given size from the population.
In simple terms, this means: if you could draw thousands of samples and compute the mean of each, the pattern those means form is the sampling distribution.
Central Limit Theorem (CLT)
States that the sampling distribution of the sample mean approaches normality as the sample size increases, regardless of the shape of the population distribution.
Think of it as: the mathematical guarantee that large enough samples will produce a roughly normal sampling distribution, even if the raw data is skewed or lumpy.
Degrees of freedom (df)
The number of independent values in a data set that are free to vary once a statistic (such as the mean) has been fixed.
In simple terms, this means: if you know the mean and all but one of the data points, you can work out the last one, so it is not "free." That constraint is why df = n − 1 for a one-sample t-test.
Random assignment
The process of allocating participants to experimental conditions using chance (coin flip, random number generator), so that each participant has an equal probability of ending up in any condition.
Think of it as: the mechanism that keeps pre-existing differences (age, ability, motivation) from clustering in one group and confounding your results.
Homogeneity of variance
The assumption that the populations being compared have equal (or at least similar) variances.
In simple terms, this means: the spread of scores in one group should be roughly the same as the spread in the other group.
Pooled variance
A weighted average of two sample variances used in the independent-samples t-test, valid when the homogeneity of variance assumption holds.
The z-test requires the population standard deviation (sigma) to be known. In almost all real research, sigma is unknown and must be estimated from the sample using s.
When you substitute s for sigma, you introduce additional sampling error. The t-distribution is wider than the normal distribution to account for this.
As sample size grows, s becomes a better estimate of sigma, and the t-distribution narrows toward the z-distribution.
At very large sample sizes (roughly n > 1000), the two distributions are nearly identical, so the choice of test makes negligible practical difference.
When sigma truly is known, the z-test is more statistically powerful: its critical values are smaller (the distribution is narrower), making it easier to detect a real effect.
One-sample t-test: compares a single sample mean to a known or hypothesised population mean. Used when you have one group and want to know if it differs from a benchmark. Example: does this class's mean AP score differ from the district mean of 3.71?
Independent-samples t-test: compares the means of two separate, unrelated groups. Used when different participants are in each condition. Example: nature group vs city group memory scores.
Dependent-samples (paired) t-test: compares means from the same participants measured under two conditions. Used when each person serves as their own control. Example: each participant tested at 1g and 2g marijuana dosage.
The t-test requires the sampling distribution of the mean (or the difference between means) to be approximately normal.
The population itself does not need to be normal if the sample size is large enough for the CLT to apply. A common guideline is n ≥ 30, though this depends on how non-normal the population is.
Sam's scenario (n = 100, non-normal population): the CLT ensures the sampling distribution will be approximately normal at that sample size, so the t-test is perfectly valid. Sam is confusing the shape of the population distribution with the shape of the sampling distribution.
Random assignment distributes individual differences (age, background, ability) roughly evenly across conditions, so any difference in outcomes can be attributed to the experimental manipulation rather than pre-existing group differences.
When random assignment is not used, systematic differences can appear. If one condition has participants all about the same age and the other has a wide range of ages, the variances of the two groups are likely unequal.
This violates the homogeneity of variance assumption required by the independent-samples t-test.
Think of a jigsaw puzzle with n pieces. Once you place n − 1 pieces, the position of the last piece is determined. It is not free.
For a one-sample t-test, df = n − 1. You lose one degree of freedom because the sample mean constrains the final data point.
For an independent-samples t-test, df = n₁ + n₂ − 2. Each group loses one degree of freedom for estimating its own mean.
Advantage of dependent (paired) samples: each participant acts as their own control, which removes between-subject variability from the error term. This typically makes the test more sensitive (more statistical power) for detecting a real effect.
Disadvantage: practical concerns such as order effects, carry-over effects, and fatigue. If the first condition changes the participant in a way that affects the second condition, results can be confounded.
With n = 1000, the t-distribution with 999 degrees of freedom is virtually indistinguishable from the standard normal distribution.
The critical values and resulting p-values will be nearly identical regardless of which test is chosen.
In practice, it does not matter. However, if sigma is still unknown (which it usually is), the t-test remains the formally correct choice.
Researchers almost never know sigma for a real population, so the t-test is the default hypothesis test in psychology, medicine, education, and most social sciences. The z-test turns up mainly in industrial quality control, where long-run process parameters (sigma) are established from historical data. The CLT is why polling organisations can draw conclusions from samples of 1,000 people about populations of millions.
Students often think the population itself must be normally distributed for a t-test to work. It does not. The CLT ensures the sampling distribution is approximately normal for sufficiently large samples.
Students sometimes believe a z-test is "better" than a t-test in all cases. The z-test is only more powerful when sigma is genuinely known; using it with an estimated sigma gives falsely narrow confidence.
A common confusion is thinking degrees of freedom is simply the sample size. It is n − 1 (one sample) or n₁ + n₂ − 2 (independent samples) because estimating means from the data costs a degree of freedom each.
Students may think random assignment guarantees equal groups. It does not guarantee it; it makes systematic bias unlikely over many replications.
⚠️ Know when to use each type of t-test (one-sample, independent, dependent) based on the research design described.
⚠️ Be able to explain why the t-test is valid even when the population is not normally distributed, provided the sample is large enough.
⚠️ Random assignment and homogeneity of variance are frequently tested conceptual questions.
⚠️ The degrees of freedom formula differs by test type. Know both formulas and be able to explain why each loses degrees of freedom.
⚠️ At very large n, z and t converge, so the choice is practically irrelevant, but the reasoning matters for exam answers.
True or False: A t-test can only be used when the population distribution is normal.
Fill in the blank: The degrees of freedom for an independent-samples t-test are __________.
True or False: Random assignment guarantees that all groups will have identical characteristics.
Fill in the blank: The main reason t-tests are more common than z-tests is that the population __________ is usually unknown.
True or False: With a sample size of 1,000, the t-distribution and the z-distribution produce virtually identical results.
Answers: 1. False. 2. n₁ + n₂ − 2. 3. False. 4. standard deviation (sigma). 5. True.
Q: Why would you use a t-test instead of a z-test?
A: Because the population standard deviation (sigma) is unknown and must be estimated from the sample. The t-distribution accounts for the extra uncertainty this introduces.
Q: A researcher has a non-normal population distribution and a sample size of 100. Can they use a t-test? Why or why not?
A: Yes. The Central Limit Theorem ensures that the sampling distribution of the mean will be approximately normal with n = 100, regardless of the population's shape.
Q: What does random assignment achieve, and what assumption is violated when it is not used?
A: Random assignment distributes participant characteristics evenly across conditions, supporting causal inference. Without it, groups may differ systematically, violating the homogeneity of variance assumption in an independent-samples t-test.
Q: Explain the degrees of freedom for a one-sample t-test using the puzzle analogy.
A: If you have n data points and you know their mean, only n − 1 of them are free to vary. The last one is determined, like the final piece of a jigsaw puzzle. So df = n − 1.
Q: What is one advantage and one disadvantage of a dependent-samples t-test compared to an independent-samples t-test?
A: Advantage: greater statistical power because between-subject variability is removed. Disadvantage: susceptible to order effects and carry-over effects.
Q: With a sample size of 1,000, does it matter whether you use a z-test or t-test?
A: Practically, no. The t-distribution with 999 df is nearly identical to the standard normal distribution. However, if sigma is unknown, the t-test is still the technically correct choice.
This material connects directly to confidence intervals, which also rely on the t-distribution when sigma is unknown. The concept of degrees of freedom returns in ANOVA (where df are partitioned among multiple sources of variance) and in chi-square tests. The assumptions covered here (normality of the sampling distribution, homogeneity of variance) also underpin more advanced methods like regression analysis.
t-test, z-test, t-distribution, standard normal distribution, population standard deviation, sigma, Central Limit Theorem, CLT, sampling distribution, degrees of freedom, df, random assignment, homogeneity of variance, equal variances assumption, one-sample t-test, independent-samples t-test, paired-samples t-test, dependent-samples t-test, pooled variance, hypothesis testing, statistical inference, UCF mathematical statistics