Difficulty: Intermediate | Prerequisites: Sampling distributions, normal distribution, z-scores
Hypothesis testing is a framework for deciding whether data provides enough evidence to reject a default assumption (the null hypothesis). Two things can go wrong: rejecting the null when it is true (Type I error, controlled by α) or failing to reject when it is false (Type II error, related to power). Increasing sample size reduces Type II error and increases power without changing the Type I error rate.
Null hypothesis (H₀)
The default claim you are testing against. It typically states "no effect" or "no difference." In simple terms, it is the boring explanation you are trying to disprove.
Alternative hypothesis (H₁ or Hₐ)
The claim you are trying to find evidence for. It can be one-sided (greater than or less than) or two-sided (not equal to).
Significance level (α)
The maximum probability of a Type I error you are willing to tolerate. Common values are 0.01, 0.05, and 0.10. You choose α before looking at the data.
Type I error (false positive)
Rejecting H₀ when it is true. The probability of this equals α by design. Think of it as a false alarm.
Type II error (false negative, β)
Failing to reject H₀ when it is false. The probability is denoted β. Think of it as missing something real.
Power
The probability of correctly rejecting H₀ when it is false. Power = 1 − β. Higher power means you are more likely to detect a real effect.
One-sided vs two-sided test
A one-sided test places the entire rejection region in one tail (you only care about deviations in one direction). A two-sided test splits α across both tails. Use one-sided when the alternative specifies a direction; use two-sided when you care about deviations in either direction.
Rejection region / critical value
The set of test statistic values that lead you to reject H₀. For a one-sided z-test at α = 0.05, the critical value is z = 1.645 (upper tail). For α = 0.01, it is z = 2.326.
p-value
The probability, under H₀, of observing a test statistic as extreme as (or more extreme than) the one you got. Reject H₀ if p-value ≤ α.
State H₀ and H₁ clearly. H₀ contains the "=" (e.g. p = 0.0615). H₁ contains the inequality.
Choose one-sided or two-sided based on the research question. If you only care whether the effect is positive (not negative), use one-sided.
Choose α before looking at the data.
Under H₀, the sample proportion p̂ has approximately: E[p̂] = p₀ and SE(p̂) = √(p₀(1 − p₀)/n).
The test statistic is z = (p̂ − p₀) / SE(p̂).
For a one-sided test (H₁: p > p₀) at α = 0.01, reject when z > 2.326, which means reject when p̂ > p₀ + 2.326 × SE(p̂).
Problem 2(a): The cutoff fraction is 0.0615 + 2.326 × √(0.0615 × 0.9385 / n).
Under H₀, X̄ has approximately: E[X̄] = μ₀ and SE(X̄) = σ/√n (known σ) or s/√n (unknown σ, use t-distribution).
Problem 8(b): μ₀ = 200, sample mean = 220, s = 36, n = 16. Test statistic = (220 − 200) / (36/√16) = 20/9 ≈ 2.22. At α = 0.05 one-sided, critical value = 1.645. Since 2.22 > 1.645, reject H₀.
Problem 12: Testing whether a programming course raises wages. H₀: μ_cw = 0, H₁: μ_cw > 0. One-sided because you are testing whether the course helps (not whether it changes wages in either direction).
Assume H₁ is true (i.e. the real parameter is some specific value under the alternative).
Find the rejection region under H₀.
Compute the probability of NOT falling in the rejection region, using the true distribution under H₁.
Problem 2(b): True p = 0.07, n = 400. The cutoff from part (a) is 0.0615 + 2.326 × √(0.0615 × 0.9385/400) ≈ 0.0615 + 2.326 × 0.01201 ≈ 0.0894. Under the true distribution, p̂ ~ N(0.07, √(0.07 × 0.93/400)). Standardise and find P(p̂ < 0.0894) under the true distribution. This probability is β.
Power = 1 − β. It depends on: the true parameter value, α, and n.
Increasing n increases power (the standard error shrinks, making it easier to detect the true effect).
Changing α does not affect power directly in a clean way: lowering α (being stricter) reduces power.
To find minimum n for a target power: set up the equation where the rejection boundary under H₀ equals the point that gives you (1 − target power) probability under H₁, then solve for n.
Problem 2(c): Want power = 0.95 with true p = 0.07, α = 0.01. Set z_α × SE₀ = (p₁ − p₀) − z_β × SE₁ and solve for n. The z-values: z_α = 2.326 (for α = 0.01 one-sided), z_β = 1.645 (for power 0.95). Approximate formula: n ≈ [(z_α √(p₀q₀) + z_β √(p₁q₁)) / (p₁ − p₀)]².
When σ is unknown and estimated by the sample standard deviation s, the test statistic follows a t-distribution with n − 1 degrees of freedom.
Problem 8(b) uses s = 36 with n = 16, so df = 15. At α = 0.05 one-sided, the critical value from the t-table is approximately 1.753 (slightly larger than 1.645 for z). The test statistic of 2.22 still exceeds this, so H₀ is still rejected.
Problem 8(a): Want power = 50% to detect a true mean of 210 vs H₀: μ = 200, σ = 50, α = 0.05.
Power of 50% means the rejection boundary under H₀ equals the true mean under H₁.
Rejection boundary: 200 + 1.645 × (50/√n) = 210. Solve: 1.645 × 50/√n = 10, so √n = 8.225, n ≈ 68 days.
Problem 12(b): The Type I error rate equals α = 0.05 regardless of sample size. That is the whole point of choosing α. Increasing n from 100 to 400 does not change the Type I error rate.
z = \frac{\hat{p} - p_0}{\sqrt{p_0(1-p_0)/n}} \quad \text{(proportion test statistic)}z = \frac{\bar{X} - \mu_0}{\sigma / \sqrt{n}} \quad \text{(mean test statistic, known } \sigma \text{)}t = \frac{\bar{X} - \mu_0}{s / \sqrt{n}} \quad \text{(mean test statistic, unknown } \sigma \text{)}\text{Reject } H_0 \text{ (one-sided upper) when } z > z_\alpha\beta = P(\text{fail to reject } H_0 \mid H_1 \text{ is true})\text{Power} = 1 - \betan \approx \left( \frac{z_\alpha \sqrt{p_0 q_0} + z_\beta \sqrt{p_1 q_1}}{p_1 - p_0} \right)^2 \quad \text{(minimum sample size for proportions)}Common critical z-values: α = 0.01 one-sided: 2.326. α = 0.05 one-sided: 1.645. α = 0.05 two-sided: 1.96. α = 0.01 two-sided: 2.576.
Students often think increasing sample size reduces the Type I error rate. It does not. α is fixed by your choice before the test. Larger n reduces Type II error (increases power).
Confusing one-sided and two-sided rejection regions. A two-sided test at α = 0.05 uses z = ±1.96, not ±1.645. A one-sided test at α = 0.05 uses 1.645 in one tail only.
Students sometimes compute power without specifying a particular alternative value. Power is always calculated against a specific assumed true parameter value under H₁.
Thinking that failing to reject H₀ means H₀ is true. It means the data did not provide enough evidence to reject it. The distinction matters on exams.
⚠️ You will be asked to derive rejection regions in terms of n. This requires plugging the standard error formula into the critical value inequality and solving for p̂ or X̄.
⚠️ Type II error calculations are a near-certainty on this exam. The procedure: find the rejection boundary under H₀, then compute the probability of falling outside that boundary under the true H₁ distribution.
⚠️ Minimum sample size for a target power is a multi-step problem that combines both the H₀ and H₁ distributions. Do not skip intermediate steps.
⚠️ Know when to use z vs t. If σ is given, use z. If σ is estimated from the sample (s), use t with n − 1 degrees of freedom.
True or false: A Type I error rate of 5% means you will incorrectly reject H₀ in 5% of tests where H₀ is true. (True)
Fill in the blank: Power = 1 − ______. (β, the Type II error probability)
True or false: Doubling the sample size cuts the Type I error rate in half. (False. α is chosen, not determined by n.)
Fill in the blank: For a one-sided upper test at α = 0.05, the critical z-value is ______. (1.645)
True or false: If you fail to reject H₀, you have proved H₀ is true. (False. You have only failed to find sufficient evidence against it.)
Q: H₀: p = 0.0615, H₁: p > 0.0615, α = 0.01, sample size n. What fraction of positive tests is needed to reject H₀?
A: Reject when p̂ > 0.0615 + 2.326 × √(0.0615 × 0.9385 / n). The critical z-value for α = 0.01 one-sided is 2.326.
Q: True p = 0.07, n = 400, α = 0.01, one-sided test against H₀: p = 0.0615. What is P(Type II error)?
A: Cutoff ≈ 0.0615 + 2.326 × √(0.0615 × 0.9385/400) ≈ 0.0615 + 0.0279 ≈ 0.0894. Under truth: SE = √(0.07 × 0.93/400) ≈ 0.01275. z = (0.0894 − 0.07)/0.01275 ≈ 1.52. β = P(Z < 1.52) ≈ 0.9357. So about a 93.6% chance of Type II error (very low power at this sample size).
Q: You sell falafel with daily revenue N(200, 50²). A new location has true mean 210 and σ = 50. How many days to achieve 50% power at α = 0.05?
A: Power = 50% means the cutoff under H₀ equals the true mean. Cutoff = 200 + 1.645 × 50/√n = 210. Solve: 82.25/√n = 10, √n = 8.225, n ≈ 68 days.
Q: You test whether a programming course raises wages. H₀: μ_cw = 0, H₁: μ_cw > 0, σ = 6000, α = 0.05. True effect = $1,000. What is the Type II error rate with n = 100?
A: Cutoff: 0 + 1.645 × 6000/√100 = 0 + 987 = 987. Under truth, X̄ ~ N(1000, 600²). z = (987 − 1000)/600 = −0.0217. β = P(Z < −0.0217) ≈ 0.4913. Power ≈ 50.9%. With n = 400: cutoff = 1.645 × 6000/20 = 493.5. z = (493.5 − 1000)/300 = −1.688. β = P(Z < −1.688) ≈ 0.0457. Power ≈ 95.4%.
Q: What sample size gives 99% power when the true effect is $1,000, σ = $6,000, and α = 0.05 (one-sided)?
A: Need: z_α × σ/√n + z_β × σ/√n ≤ effect. (1.645 + 2.326) × 6000/√n ≤ 1000. 3.971 × 6000/√n ≤ 1000. √n ≥ 23.826. n ≥ 568.
Q: With n = 400 and sample mean $3,000 for the programming course (σ = $6,000), form a 99% confidence interval.
A: SE = 6000/√400 = 300. CI = 3000 ± 2.576 × 300 = 3000 ± 772.8 = ($2,227.20, $3,772.80).
Hypothesis testing builds directly on the sampling distribution concepts from the estimators notes. The test statistic is just the estimator, standardised. Confidence intervals and hypothesis tests are two views of the same idea: a 95% CI contains all parameter values you would fail to reject at α = 0.05. Power calculations connect to sample size planning, which appears throughout applied statistics.
Related Terms / Search Tags
Hypothesis test, null hypothesis, alternative hypothesis, Type I error, Type II error, significance level, alpha, power, beta, rejection region, critical value, z-test, t-test, one-sided test, two-sided test, p-value, sample size determination, power analysis, ASTATS NYU, analytical statistics midterm 2, COVID test positivity, falafel cart, wage change test