Difficulty: Intermediate | Prerequisites: Introductory probability, normal distribution basics, sample statistics (mean, variance, standard deviation)
Hypothesis testing is the formal framework statisticians use to decide whether observed data provides enough evidence to reject a claim about a population. It sits at the centre of inferential statistics and builds directly on your knowledge of sampling distributions, the normal distribution, and the idea that sample statistics vary from sample to sample. If you are comfortable computing a sample mean and a sample standard deviation, and you understand what a normal or t-distribution looks like, you have the prerequisites.
This set of notes covers one-sample tests: situations where you have a single sample and want to test a claim about the population mean or the population variance. You will see three test families here, the Z-test, the t-test, and the chi-squared test on variance, each suited to different information about the population.
You pick a test statistic whose distribution you know under the null hypothesis, compute it from your data, and check whether it falls in a region extreme enough to reject the null. One-sample tests let you test whether a population mean equals some value (Z-test when the variance is known, t-test when it is not) or whether a population variance equals some value (chi-squared test). The p-value tells you the smallest significance level at which you would still reject.
Null hypothesis (H₀)
The default claim about a population parameter, assumed true until evidence says otherwise. Think of it as the "nothing interesting is happening" baseline.
Alternative hypothesis (H₁ or Hₐ)
The claim you are trying to find evidence for. It can be two-sided (the parameter differs from the null value), or one-sided (greater than, or less than).
Test statistic
A number computed from the sample data that measures how far the observed result sits from what the null hypothesis predicts. In simple terms, it converts your data into a single score you can compare against a known distribution.
Significance level (α)
The probability threshold you set before testing. If the chance of seeing your data (or something more extreme) under the null is below α, you reject the null. Common choices are 0.05 and 0.01.
Critical value
The boundary on the test statistic's distribution that marks the start of the rejection region. If your test statistic crosses it, you reject H₀.
p-value
The smallest level of significance that would lead to rejection of H₀ with the given data. In simple terms, it answers: "If the null were true, how surprising is what I observed?"
Power
The probability of rejecting H₀ when the alternative hypothesis is true. Think of it as the test's ability to detect a real effect. Higher power means fewer missed effects.
Type I error (false positive)
Rejecting H₀ when it is in fact true. The significance level α is exactly the probability you allow for this mistake.
Type II error (false negative)
Failing to reject H₀ when H₁ is in fact true. Its probability is denoted β. Power equals 1 minus β.
Rejection region / critical region
The set of test statistic values for which you reject H₀. It can sit in one tail or both, depending on your alternative hypothesis.
Every hypothesis test follows the same four steps, regardless of which specific test you use.
Determine an appropriate test statistic. Choose a formula that summarises how far your sample result is from what the null hypothesis claims. The choice depends on what you are testing (mean vs. variance) and what you know (is the population variance known?).
Identify the distribution of the test statistic under the null hypothesis. If H₀ is true, the test statistic follows a known distribution: standard normal, t, chi-squared, or F.
Decide on whether to reject in the left tail, right tail, or both tails. This follows directly from your alternative hypothesis. H₁: μ ≠ μ₀ is two-tailed; H₁: μ > μ₀ is right-tailed; H₁: μ < μ₀ is left-tailed.
Determine the necessary quantiles and critical values. Look up the cutoff(s) from the relevant distribution at your chosen α. Compare the test statistic to the critical value(s), or compute the p-value and compare it to α.
When to use: You have one sample from a normal population and you know the true population variance σ². You want to test whether the population mean μ equals a hypothesised value μ₀.
Hypotheses
Two-sided: H₀: μ = μ₀ vs. H₁: μ ≠ μ₀
Right-sided: H₀: μ = μ₀ vs. H₁: μ > μ₀
Left-sided: H₀: μ = μ₀ vs. H₁: μ < μ₀
Test statistic
Z₀ = (X̄ − μ₀) / (σ / √n), which follows N(0,1) under H₀.
The numerator measures how far the sample mean sits from the hypothesised mean. The denominator is the standard error of the mean.
Rejection rules and p-values
Two-sided: reject if |z₀| > z(α/2). p-value = 2[1 − Φ(|z₀|)].
Right-sided: reject if z₀ > z(1−α). p-value = 1 − Φ(z₀).
Left-sided: reject if z₀ < z(α). p-value = Φ(z₀).
Here Φ denotes the standard normal CDF, and z(α/2) is shorthand for the α/2 upper quantile of the standard normal.
When to use: You have one sample from a normal population, but the population variance is unknown. You estimate it with the sample variance S² and want to test a claim about the population mean.
Hypotheses
Same structure as the Z-test: two-sided (μ ≠ μ₀), right-sided (μ > μ₀), or left-sided (μ < μ₀).
Test statistic
T₀ = (X̄ − μ₀) / (S / √n), which follows a t-distribution with n − 1 degrees of freedom under H₀.
The only difference from the Z-test is that σ is replaced by S, the sample standard deviation. This substitution introduces extra uncertainty, which is why the distribution has heavier tails than the normal.
Rejection rules and p-values
Two-sided: reject if |t₀| > t(n−1, α/2). p-value = 2[1 − F_t(|t₀|)], where F_t is the t CDF with n − 1 df.
Right-sided: reject if t₀ > t(n−1, 1−α). p-value = 1 − F_t(t₀).
Left-sided: reject if t₀ < t(n−1, α). p-value = F_t(t₀).
Key point: As n grows large, the t-distribution converges to the standard normal, so Z-test and t-test give nearly the same result for large samples. With small samples, the t-test is more conservative (wider confidence intervals, harder to reject).
When to use: You have one sample from a normal population and you want to test whether the population variance σ² equals a hypothesised value σ₀².
Hypotheses
Two-sided: H₀: σ² = σ₀² vs. H₁: σ² ≠ σ₀²
Right-sided: H₀: σ² = σ₀² vs. H₁: σ² > σ₀²
Left-sided: H₀: σ² = σ₀² vs. H₁: σ² < σ₀²
Test statistic
χ₀² = (n − 1)S² / σ₀², which follows a chi-squared distribution with n − 1 degrees of freedom under H₀.
The chi-squared distribution is not symmetric, so the two-sided rejection region uses different quantiles in each tail.
Rejection rules and p-values
Two-sided: reject if χ₀² > χ²(n−1, 1−α/2) or χ₀² < χ²(n−1, α/2). p-value = 2 × min[F_χ(χ₀²), 1 − F_χ(χ₀²)].
Right-sided: reject if χ₀² > χ²(n−1, 1−α). p-value = 1 − F_χ(χ₀²).
Left-sided: reject if χ₀² < χ²(n−1, α). p-value = F_χ(χ₀²).
Important: This test is sensitive to departures from normality. If the underlying population is not normal, the chi-squared test on variance can be unreliable even with moderate sample sizes.
Test | Test Statistic | Null Distribution | Degrees of Freedom |
|---|---|---|---|
Z-test (σ² known) | Z₀ = (X̄ − μ₀) / (σ / √n) | N(0,1) | n/a |
t-test (σ² unknown) | T₀ = (X̄ − μ₀) / (S / √n) | t(n−1) | n − 1 |
Chi-squared (variance) | χ₀² = (n−1)S² / σ₀² | χ²(n−1) | n − 1 |
Useful distributional facts
If X₁, ..., Xₙ are i.i.d. N(μ, σ²), then X̄ and S² are independent.
X̄ ~ N(μ, σ²/n).
(n−1)S² / σ² ~ χ²(n−1).
√n(X̄ − μ) / S ~ t(n−1).
Students often think a p-value is the probability that H₀ is true. It is not. The p-value is the probability of observing data as extreme as (or more extreme than) what you got, assuming H₀ is true.
Students often confuse "failing to reject H₀" with "proving H₀ is true." Failing to reject means the data did not provide enough evidence against H₀. That is a weaker statement than acceptance.
Students sometimes use the Z-test formula when the population variance is unknown and plug in S for σ. This is incorrect for small samples: when σ is unknown, use the t-test.
The chi-squared test on variance is sometimes applied without checking normality. Unlike the t-test, which is fairly robust to mild non-normality, the chi-squared variance test is quite sensitive to it.
⚠️ Know which test to use based on what information is given. If the problem states σ², use the Z-test. If it gives S² or says "variance unknown," use the t-test.
⚠️ Be able to set up hypotheses correctly. Read the problem for direction words: "is there evidence that the mean exceeds..." means a right-sided test.
⚠️ You will likely be asked to compute a test statistic, find a critical value from a table, and state your conclusion. Practise all three.
⚠️ p-value interpretation is a favourite exam question. Be precise: it is the probability of the observed (or more extreme) data under H₀, not the probability of H₀.
⚠️ The chi-squared test on variance uses (n − 1) in the numerator, not n. This is easy to mix up under time pressure.
True or false: A p-value of 0.03 means there is a 3% chance the null hypothesis is true.
False. The p-value is the probability of observing data this extreme (or more) if H₀ were true.
Fill in the blank: When the population variance is unknown and the sample is small, we use the ___ test instead of the Z-test.
t-test (with n − 1 degrees of freedom).
True or false: In a two-sided test at α = 0.05, you split the rejection probability equally between both tails, putting 0.025 in each.
True.
Fill in the blank: The test statistic for the one-sample variance test follows a ___ distribution with ___ degrees of freedom.
Chi-squared; n − 1.
True or false: Failing to reject H₀ proves H₀ is true.
False. It means the data did not provide sufficient evidence against H₀.
Q: A manufacturer claims the mean weight of their product is 500g. You take a sample of 36 items and find X̄ = 505g. The population standard deviation is known to be σ = 12g. At α = 0.05, is there evidence that the mean weight differs from 500g?
A: Z₀ = (505 − 500) / (12 / √36) = 5 / 2 = 2.5. The critical value for a two-sided test at α = 0.05 is z(0.025) = 1.96. Since 2.5 > 1.96, reject H₀. There is evidence the mean weight differs from 500g.
Q: A random sample of 16 observations from a normal population yields X̄ = 74 and S = 8. Test H₀: μ = 70 vs. H₁: μ > 70 at α = 0.05.
A: T₀ = (74 − 70) / (8 / √16) = 4 / 2 = 2.0. Degrees of freedom = 15. The critical value t(15, 0.95) ≈ 1.753. Since 2.0 > 1.753, reject H₀. There is evidence the mean exceeds 70.
Q: Explain the difference between the Z-test and t-test for a population mean.
A: The Z-test uses the known population standard deviation σ in the denominator and follows N(0,1). The t-test uses the sample standard deviation S and follows a t-distribution with n − 1 degrees of freedom. As n increases, the t-distribution approaches the standard normal.
Q: A sample of 25 observations from a normal population has S² = 18. Test H₀: σ² = 12 vs. H₁: σ² > 12 at α = 0.05.
A: χ₀² = (25 − 1)(18) / 12 = 24 × 1.5 = 36. Degrees of freedom = 24. The critical value χ²(24, 0.95) ≈ 36.415. Since 36 < 36.415, fail to reject H₀ at the 5% level. The evidence is not quite strong enough to conclude the variance exceeds 12.
This material connects directly to confidence intervals: a two-sided hypothesis test at level α rejects H₀ exactly when the corresponding (1 − α) confidence interval does not contain μ₀. If you understand one, you understand the other.
The distributional facts here (how chi-squared, t, and F distributions arise) are the foundation for two-sample tests covered in Part 2 of these notes, and for ANOVA and regression analysis later in the course.
Power analysis, touched on briefly here, becomes central when you study sample-size determination: choosing n large enough to detect a meaningful difference with high probability.
hypothesis testing, null hypothesis, alternative hypothesis, Z-test, t-test, chi-squared test, one-sample test, test statistic, p-value, significance level, alpha, critical value, rejection region, Type I error, Type II error, power of a test, normal distribution, t-distribution, chi-squared distribution, degrees of freedom, sample mean, sample variance, population variance known, population variance unknown, NYU analytical statistics, statistical inference, one-tailed test, two-tailed test