Difficulty: Intermediate | Prerequisites: Sampling distributions, Central Limit Theorem, standard normal (Z) table
Hypothesis testing lets you decide whether observed data provide enough evidence to reject a claim about a population parameter. You state a null hypothesis (the default claim), compute a test statistic and its p-value, then compare to a pre-set significance level. This material covers z-tests for population means and proportions, one-sided versus two-sided tests, p-value interpretation, and critical-value calculation.
Null hypothesis (H₀)
The default claim about a population parameter, assumed true unless the data provide strong enough evidence against it. Think of it as the "nothing special is happening" position.
Alternative hypothesis (H₁ or Hₐ)
The claim you are trying to find evidence for. It is the opposite of H₀ and represents the research question. In simple terms, this is what you suspect is really going on.
Significance level (α)
The probability threshold you set before collecting data. If the p-value falls below α, you reject H₀. Typical values are 0.05 or 0.01. Think of it as how much risk of a false alarm you are willing to tolerate.
P-value
The probability of observing a test statistic at least as extreme as the one you got, assuming H₀ is true. A small p-value means the data would be surprising under the null. In simple terms, it answers: "If the default claim were true, how weird is my sample?"
Z-test statistic
A standardised score that measures how far a sample statistic is from the null-hypothesis value, in units of its standard error. The formula for a mean is Z = (X̄ − μ₀) / (σ / √n). Think of it as converting your result into a "how many standard errors away" number.
Standard error (SE)
The standard deviation of the sampling distribution of a statistic. For a sample mean with known σ: SE = σ / √n. For a sample proportion: SE = √[p₀(1 − p₀) / n]. In simple terms, it tells you how much your statistic would bounce around from sample to sample.
One-sided (one-tailed) test
A test where H₁ specifies a direction: either μ > μ₀ or μ < μ₀. The entire rejection region sits in one tail of the distribution.
Two-sided (two-tailed) test
A test where H₁ says the parameter differs from μ₀ in either direction: μ ≠ μ₀. The rejection region is split across both tails, so the p-value is doubled relative to the one-sided version.
Critical value
The boundary value of the test statistic (or sample statistic) beyond which you reject H₀. For α = 0.05 one-sided, the critical z-value is 1.645; for α = 0.05 two-sided, it is ±1.96. Think of it as the line in the sand: cross it and you reject.
Proportion test (z-test for p)
A z-test applied to a sample proportion p̂ rather than a sample mean. The test statistic is Z = (p̂ − p₀) / √[p₀(1 − p₀) / n]. Same logic, different formula for the standard error.
Use this when you have a known population standard deviation σ, a reasonably large sample (or normal population), and your research question has a direction: you suspect the true mean is greater than (or less than) some claimed value.
H₀: μ = μ₀ (the claimed value)
H₁: μ > μ₀ (right-tailed) or H₁: μ < μ₀ (left-tailed)
Z = (X̄ − μ₀) / (σ / √n)
This converts the distance between your sample mean and the null value into standard-error units.
For a right-tailed test: p-value = P(Z ≥ z_observed). Look up z in the standard normal table and subtract from 1.
If p-value ≤ α, reject H₀. Otherwise, fail to reject.
A food cart claims the average wait is 8 minutes. You suspect it is longer. σ = 3, n = 36, X̄ = 9.1.
H₀: μ = 8
H₁: μ > 8
SE = 3 / √36 = 0.5
Z = (9.1 − 8) / 0.5 = 2.2
P-value = P(Z ≥ 2.2) ≈ 0.0139
At α = 0.05, 0.0139 < 0.05, so reject H₀. There is sufficient evidence that the average wait exceeds 8 minutes.
NYU tests whether a tutoring programme raises GPAs above the current average of 3.0. σ = 0.5, n = 25, X̄ = 3.25.
H₀: μ = 3.0
H₁: μ > 3.0
SE = 0.5 / √25 = 0.1
Z = (3.25 − 3.0) / 0.1 = 2.5
P-value = P(Z ≥ 2.5) ≈ 0.0062
At α = 0.05, reject H₀. The data support the claim that the programme increases GPA.
Use this when you want to detect a difference in either direction. The sample mean could be higher or lower than the claimed value, and either outcome would be interesting.
H₀: μ = μ₀
H₁: μ ≠ μ₀
The p-value doubles. You compute the one-tail probability and multiply by 2, because extreme results in both tails count as evidence against H₀.
Reject H₀ if p-value ≤ α, just as before, but the p-value is now 2 × P(Z ≥ |z|).
Same food cart setup: μ₀ = 8, σ = 3, n = 36, X̄ = 9.1.
H₀: μ = 8
H₁: μ ≠ 8
Z = 2.2 (same calculation as before)
Two-sided p-value = 2 × P(Z ≥ 2.2) = 2 × 0.0139 ≈ 0.0278
At α = 0.05, 0.0278 < 0.05, so reject H₀. The average wait time is significantly different from 8 minutes.
Notice the two-sided p-value (0.0278) is larger than the one-sided p-value (0.0139). Both lead to rejection here, but that will not always be the case. A result that is significant one-sided may fail to reach significance two-sided.
A critical value is the cutoff for the test statistic (or, equivalently, the sample statistic) that separates the rejection region from the non-rejection region. Instead of computing a p-value every time, you can compare your statistic directly to the critical value.
One-sided, α = 0.05: critical z = 1.645 (right-tailed) or −1.645 (left-tailed)
Two-sided, α = 0.05: critical z = ±1.96
To find the smallest X̄ that leads to rejection (right-tailed):
X̄_crit = μ₀ + z_crit × (σ / √n)
This is useful when you want to state the decision rule in the original units rather than in z-scores.
One-sided test, α = 0.05, μ₀ = 8, σ = 3, n = 36.
z_crit = 1.645
X̄_crit = 8 + 1.645 × (3 / √36) = 8 + 1.645 × 0.5 = 8.8225
Any sample mean above 8.8225 leads to rejection of H₀. The observed X̄ = 9.1 exceeds this, consistent with the p-value result.
When the parameter of interest is a proportion (e.g. the fraction of trains arriving on time) rather than a mean, the logic is identical but the standard error formula changes.
H₀: p = p₀
H₁: p ≠ p₀
Z = (p̂ − p₀) / √[p₀(1 − p₀) / n]
where p̂ = observed sample proportion and p₀ = the claimed proportion under H₀.
For a two-sided test at α = 0.05, the critical z-values are ±1.96. Convert to p̂ boundaries:
p̂_lower = p₀ − 1.96 × √[p₀(1 − p₀) / n]
p̂_upper = p₀ + 1.96 × √[p₀(1 − p₀) / n]
Reject H₀ if p̂ falls below p̂_lower or above p̂_upper.
The MTA claims 70% of trains arrive on time. Two-sided test, α = 0.05, n = 200.
H₀: p = 0.7, H₁: p ≠ 0.7
SE = √[0.7 × 0.3 / 200] = √[0.00105] ≈ 0.0324
p̂_lower = 0.7 − 1.96 × 0.0324 ≈ 0.6365
p̂_upper = 0.7 + 1.96 × 0.0324 ≈ 0.7635
Reject H₀ if the observed proportion is below 0.6365 or above 0.7635.
Z-test for a mean (σ known)
Z = (X̄ − μ₀) / (σ / √n)
Standard error of the mean
SE = σ / √n
Z-test for a proportion
Z = (p̂ − p₀) / √[p₀(1 − p₀) / n]
Standard error of a proportion (under H₀)
SE = √[p₀(1 − p₀) / n]
Critical value on the X̄ scale (right-tailed)
X̄_crit = μ₀ + z_α × (σ / √n)
Critical values on the p̂ scale (two-sided)
p̂_lower = p₀ − z_{α/2} × √[p₀(1 − p₀) / n]
p̂_upper = p₀ + z_{α/2} × √[p₀(1 − p₀) / n]
Common z critical values
α = 0.10 one-sided: z = 1.28
α = 0.05 one-sided: z = 1.645
α = 0.01 one-sided: z = 2.33
α = 0.10 two-sided: z = ±1.645
α = 0.05 two-sided: z = ±1.96
α = 0.01 two-sided: z = ±2.576
Students often think a p-value is the probability that H₀ is true. It is not. The p-value is the probability of seeing data this extreme given that H₀ is true. The probability that H₀ is true, given the data, requires Bayes' theorem and a prior.
Students confuse "fail to reject H₀" with "accept H₀." Failing to reject means the evidence was not strong enough; it does not prove the null is correct.
Students sometimes forget to double the one-tail probability for a two-sided test. If your one-sided p-value is 0.04 and you are running a two-sided test, the correct p-value is 0.08, and you would not reject at α = 0.05.
Students mix up σ (population standard deviation) and σ / √n (standard error). The test statistic uses the standard error, not σ by itself. Dividing by √n accounts for the fact that sample means are less variable than individual observations.
⚠️ You will almost certainly be asked to set up hypotheses, compute a z-statistic, find the p-value, and state a conclusion. Practise the full chain, not just the formula.
⚠️ Know when to use a one-sided versus two-sided test. The problem wording tells you: "is it longer" or "does it increase" signals one-sided; "is it different" signals two-sided.
⚠️ Be ready to convert between the z critical value and the X̄ (or p̂) critical value. Q1c and Q3a both test this.
⚠️ State conclusions in context. Do not just say "reject H₀." Say what that means for the specific problem (e.g. "There is sufficient evidence at the 5% level that the average wait time exceeds 8 minutes.").
True or false: A p-value of 0.03 means there is a 3% chance the null hypothesis is true.
False. The p-value is the probability of data this extreme under H₀, not the probability that H₀ is true.
Fill in the blank: For a two-sided test, the p-value equals ______ times the one-tail probability.
Two.
True or false: If you reject H₀ with a one-sided test at α = 0.05, you will always reject with a two-sided test at the same α.
False. The two-sided p-value is larger, so a borderline one-sided rejection (say p = 0.04) becomes p = 0.08 two-sided, which does not reject.
Fill in the blank: The standard error of the sample mean is σ divided by ______.
√n (the square root of the sample size).
True or false: Increasing the sample size decreases the standard error.
True. SE = σ / √n, so as n grows, SE shrinks.
Q: A manufacturer claims the mean weight of its cereal boxes is 500 g. You believe it is less. σ = 15 g, n = 49, X̄ = 496 g. State the hypotheses, compute the z-statistic, find the p-value, and decide at α = 0.05.
A: H₀: μ = 500, H₁: μ < 500. SE = 15/√49 = 15/7 ≈ 2.143. Z = (496 − 500)/2.143 ≈ −1.867. P-value = P(Z ≤ −1.867) ≈ 0.031. Since 0.031 < 0.05, reject H₀. There is evidence the boxes weigh less than 500 g on average.
Q: For the same cereal problem, what is the critical value of X̄ for the left-tailed test at α = 0.05?
A: X̄_crit = 500 − 1.645 × (15/7) = 500 − 3.525 ≈ 496.475. Any sample mean below 496.475 leads to rejection. The observed 496 is below this threshold, consistent with the p-value conclusion.
Q: The MTA claims 70% of trains are on time. You observe 126 on-time arrivals in a sample of 200. Is this significantly different from 70% at α = 0.05 (two-sided)?
A: p̂ = 126/200 = 0.63. SE = √(0.7 × 0.3 / 200) ≈ 0.0324. Z = (0.63 − 0.70)/0.0324 ≈ −2.16. Two-sided p-value = 2 × P(Z ≤ −2.16) ≈ 2 × 0.0154 = 0.0308. Since 0.0308 < 0.05, reject H₀. The on-time rate differs significantly from 70%.
Q: Explain why a two-sided test at α = 0.05 uses z = ±1.96 while a one-sided test uses z = 1.645.
A: In a two-sided test, the 5% rejection region is split between both tails (2.5% each), so the critical z puts 2.5% in each tail, giving 1.96. In a one-sided test, the full 5% is in one tail, so the critical z only needs to put 5% beyond it, giving 1.645.
This material builds directly on sampling distributions and the Central Limit Theorem: without the CLT, you could not assume X̄ is approximately normal for moderate n. The z-test framework here extends naturally to t-tests (when σ is unknown and must be estimated from the sample), which you will encounter next. Power analysis, Type II errors, and sample-size determination (covered in the companion study notes) are the other side of the same coin: once you understand how to reject H₀, you need to understand how often you will catch a real effect.
z-test, hypothesis test, null hypothesis, alternative hypothesis, one-tailed test, two-tailed test, one-sided test, two-sided test, p-value, significance level, alpha, critical value, rejection region, standard error, test statistic, population mean, sample mean, population proportion, sample proportion, z-score, normal distribution, hypothesis testing examples, NYU Analytical Statistics, ASTATS, PS5, Timothy Roeper, food cart wait time, MTA on-time trains