Difficulty: Intermediate | Prerequisites: Basic probability, normal distribution, sample mean and variance concepts
This material sits at the heart of inferential statistics: you have a sample, and you want to say something about the population it came from. The t-distribution lets you do that when you do not know the population variance and must estimate it from the data. Confidence intervals give you a range of plausible values for a parameter, and the law of total probability with Bayes' rule lets you update beliefs when extra information (like which sub-population a draw came from) enters the picture. If you are comfortable with z-scores for known-variance problems, this extends that toolkit to the realistic case where variance is unknown, and adds the machinery for combining conditional probabilities.
When the population standard deviation is unknown, use the t-distribution (not the z) to compute probabilities and build confidence intervals. The law of total probability lets you handle situations where samples come from a mixture of distributions, and Bayes' rule flips a conditional probability to update which distribution was likely responsible for an observed outcome.
t-distribution (Student's t)
A probability distribution that arises when estimating the mean of a normally distributed population where the population standard deviation is unknown. It is wider and has heavier tails than the standard normal. Think of it as: the normal distribution's cautious sibling, accounting for the extra uncertainty of estimating sigma from the data.
Degrees of freedom (df)
The number of independent pieces of information that go into estimating a parameter. For a single-sample t-statistic with n observations, df = n - 1. In simple terms, this means: you lose one degree of freedom because you used the data to estimate the mean before computing the variance.
t-statistic
The test statistic (X-bar - mu) / sqrt(s^2 / n), which follows a t-distribution with n - 1 degrees of freedom when sampling from a normal population. Think of it as: a z-score, but using the sample standard deviation instead of the population one.
Confidence interval (CI)
A range of values, constructed from the sample, that contains the true parameter with a specified probability (the confidence level). In simple terms, this means: if you repeated the sampling many times and built an interval each time, that percentage of intervals would capture the true value.
Law of total probability
A rule for computing the overall probability of an event by summing over all the mutually exclusive ways it can happen: P(A) = sum of P(A | B_i) * P(B_i). Think of it as: breaking an event into cases, computing the probability in each case, and adding them up weighted by how likely each case is.
Bayes' theorem
A formula that reverses a conditional probability: P(B | A) = P(A | B) * P(B) / P(A). In simple terms, this means: given that an event happened, Bayes' rule tells you how to update your belief about which underlying condition produced it.
z-score vs t-score
A z-score standardises using the known population sigma; a t-score standardises using the sample standard deviation s. When sigma is known, use z. When sigma is unknown and estimated from the sample, use t. This distinction is tested heavily.
When sampling n observations from a normal distribution with unknown variance, the quantity (X-bar - mu) / sqrt(s^2 / n) follows a t-distribution with df = n - 1.
With n = 4, that gives df = 3. The t-table gives tail probabilities for specific quantiles at each df.
Example: P(t_3 > 0.978) = 0.2, read directly off the t-table.
To find the probability that a value constructed from X-bar and s falls below some threshold, rearrange the inequality so that the t-statistic is isolated on one side.
Example: P(X-bar - 0.978 * sqrt(s^2) / sqrt(4) < 0) rearranges to P(t_3 < 0.978) = 1 - P(t_3 > 0.978) = 1 - 0.2 = 0.8.
For interval coverage (probability that 0 falls between a lower and upper bound), set up a double inequality, rearrange both sides, and read off two tail probabilities.
P(-0.978 < t_3 < 0.978) = P(t_3 < 0.978) - P(t_3 < -0.978) = 0.8 - 0.2 = 0.6.
When the underlying distribution can take different forms with known probabilities (e.g., sigma = 1 with probability 0.75 and sigma = 2 with probability 0.25), the unconditional probability of an event is:
P(A) = P(A | sigma = 1) * P(sigma = 1) + P(A | sigma = 2) * P(sigma = 2)
Each conditional probability is computed separately, then combined.
Critical detail: when sigma is known in each case, you use a z-score (not a t-score) for that conditional calculation, because the variance is no longer estimated from the sample.
P(X-bar > 1 | sigma = 1) = P(z > (1 - 0) / (1 / sqrt(4))) = P(z > 2) = 0.0228
P(X-bar > 1 | sigma = 2) = P(z > (1 - 0) / (2 / sqrt(4))) = P(z > 1) = 0.1587
P(X-bar > 1) = 0.0228 * 0.75 + 0.1587 * 0.25 = 0.05678
After observing an outcome, Bayes' rule updates the probability that a particular condition was responsible.
P(sigma = 2 | X-bar > 1) = P(X-bar > 1 | sigma = 2) * P(sigma = 2) / P(X-bar > 1)
= (0.1587 * 0.25) / 0.05678 = 0.6987
Interpretation: even though sigma = 2 had only a 25% prior chance, after observing X-bar > 1 the updated probability jumps to about 70%. Large sample means are more plausible under a higher-variance distribution.
t = \frac{\bar{X} - \mu}{\sqrt{s^2 / n}} \sim t_{n-1}\text{CI bounds} = \bar{X} \pm t_{\alpha/2,\, df} \cdot \sqrt{s^2 / n}P(A) = \sum_i P(A \mid B_i)\, P(B_i)P(B_j \mid A) = \frac{P(A \mid B_j)\, P(B_j)}{P(A)}The t-distribution is the workhorse of small-sample inference in clinical trials, A/B tests with limited data, and quality control. When a pharmaceutical company tests a new drug on 20 patients and needs to estimate the mean treatment effect, the t-distribution is what makes that inference valid. Bayes' rule underpins diagnostic testing: if a medical test comes back positive, Bayes' theorem tells you the probability that the patient truly has the condition, given the test's sensitivity and the base rate in the population.
Students often think you always use a z-score when the population is normal. You do not. If sigma is unknown and estimated by s, you must use the t-distribution, even if the population itself is normal.
Students frequently forget to adjust degrees of freedom. With n = 4 observations, df = 3, not 4.
A common error in law-of-total-probability problems is forgetting to weight each conditional probability by the prior probability of that case. You cannot simply average the conditional probabilities unless the priors are equal.
When applying Bayes' rule, students often confuse P(A | B) with P(B | A). These are different quantities. Bayes' rule is precisely the tool for going from one to the other.
⚠️ Know when to switch from t to z. If sigma is given (known), use z. If sigma is estimated from the sample, use t. This distinction appeared explicitly in the midterm.
⚠️ The t-table is read differently from the z-table. Make sure you know how to look up tail probabilities for a given df and quantile.
⚠️ Law of total probability + Bayes' rule questions are multi-step. Partial credit requires showing each step clearly: set up the total probability first, compute each conditional, combine, then apply Bayes.
⚠️ Interval coverage probability ("probability that the interval contains 0") requires setting up a double inequality and reading two values from the t-table.
True or False: If you know the population variance, you should use the t-distribution.
False. Use the z-distribution when sigma is known.
Fill in the blank: With a sample of size 4, the t-statistic has ___ degrees of freedom.
3
True or False: P(t_3 > 0.978) = 0.2 means that P(t_3 < 0.978) = 0.2.
False. P(t_3 < 0.978) = 1 - 0.2 = 0.8.
Fill in the blank: In the law of total probability, each conditional probability is weighted by ___.
The prior probability of that condition (case).
True or False: P(A | B) and P(B | A) are the same thing.
False. Bayes' theorem relates one to the other.
Q: You draw 4 observations from a normal distribution with mu = 0 and unknown sigma. You compute X-bar and s^2, then form the quantity X-bar - 0.978 * sqrt(s^2 / 4). What is the probability this quantity is negative?
A: Rearrange to get P(t_3 < 0.978). From the t-table, P(t_3 > 0.978) = 0.2, so the answer is 1 - 0.2 = 0.8.
Q: Using the same setup, what is the probability that the interval [X-bar - 0.978 * sqrt(s^2/4), X-bar + 0.978 * sqrt(s^2/4)] contains 0?
A: This is P(-0.978 < t_3 < 0.978) = 0.8 - 0.2 = 0.6.
Q: Suppose X-bar comes from a normal with mu = 0 and n = 4, but sigma is either 1 (with probability 0.75) or 2 (with probability 0.25). What is P(X-bar > 1)?
A: Use the law of total probability. P(X-bar > 1 | sigma=1) = P(z > 2) = 0.0228. P(X-bar > 1 | sigma=2) = P(z > 1) = 0.1587. So P(X-bar > 1) = 0.0228 * 0.75 + 0.1587 * 0.25 = 0.05678. Note: z-scores are used because sigma is known in each case.
Q: Given that X-bar > 1 in the previous problem, what is the probability that sigma = 2?
A: Apply Bayes' rule. P(sigma=2 | X-bar > 1) = (0.1587 * 0.25) / 0.05678 = 0.6987.
This material connects directly to confidence interval construction (covered in the next set of notes on two-sample inference and pooled variance). The t-distribution reappears whenever you build a CI or run a hypothesis test with unknown variance. Bayes' rule and the law of total probability also appear in hypothesis testing contexts, particularly when computing the probability that the null is true given that you failed to reject it, which is covered in the notes on hypothesis testing and statistical power.
Student's t-distribution, t-test, t-table, t-score, degrees of freedom, df, confidence interval, CI, confidence level, coverage probability, law of total probability, total probability rule, Bayes' theorem, Bayes' rule, posterior probability, prior probability, conditional probability, z-score vs t-score, known vs unknown variance, sampling distribution of the mean, standard error