Difficulty: Introductory | Prerequisites: None (this is foundational material for the rest of the course).
This topic covers the mechanics of how we collect data from populations, and why the method of collection matters enormously for the conclusions we can draw. It sits at the very start of the course because every later chapter on confidence intervals, hypothesis testing, and regression assumes the data were gathered properly. If you are coming in cold, start here: everything in inferential statistics depends on understanding sampling.
Statistics uses samples to draw conclusions about populations, because measuring an entire population is rarely feasible. The way you select your sample determines whether your conclusions are trustworthy. Biased sampling methods (convenience samples, voluntary response samples) produce statistics that systematically miss the true population value, while probability-based methods (simple random samples, stratified random samples) give every individual a known chance of selection and support valid inference.
Parameter
A numerical characteristic of an entire population (e.g. the population mean, standard deviation, or proportion). Denoted by Greek letters: μ for the population mean, σ for the population standard deviation. In simple terms, a parameter is the "true answer" you would get if you could measure every single member of the population. You almost never know it exactly.
Statistic
A numerical value computed from sample data. Denoted by Roman/alphabet letters: x̄ for the sample mean, s for the sample standard deviation. Think of it as your best estimate of the parameter, calculated from the subset of the population you managed to observe.
Statistical inference
The process of using sample data (a statistic) to draw conclusions about a population (a parameter). In simple terms, this is the core logic of the entire course: measure a part, make a claim about the whole.
Sampling bias
A systematic tendency for a sampling method to produce statistics that consistently differ from the true population parameter. Think of it as a built-in tilt in your data collection, one that no amount of additional data from the same flawed method will fix.
Convenience sampling
A non-probability method where the researcher simply surveys whoever is easiest to reach. In simple terms, you ask whoever happens to be around. Cheap and fast, but the people you can easily reach are rarely representative of the whole population.
Voluntary response sampling
A non-probability method where respondents choose themselves whether to participate. Think of it as an open invitation where people with strong feelings (especially negative ones) are the most likely to show up. Online polls and phone-in surveys are classic examples.
Probability sample
Any sample where each unit in the population is selected by a known chance mechanism. No group is systematically over- or under-represented. In simple terms, randomness does the choosing, which protects against the researcher's own blind spots.
Simple random sample (SRS)
A probability sample in which every individual in the population has the same probability of being selected. Obtained by numbering all N units and randomly choosing n of them. Think of it as drawing names from a perfectly mixed hat.
Stratified random sample
A probability sample formed by dividing the population into homogeneous subgroups (strata) and then taking a separate SRS from each stratum. In simple terms, you split the population into meaningful categories first (e.g. male/female, urban/suburban/rural), then randomly sample within each category to ensure every group is represented.
Strata
The homogeneous subgroups into which a population is divided before drawing a stratified random sample. Each stratum shares a characteristic thought to affect the variable of interest. Think of it as the "bins" you sort people into before sampling from each bin.
A parameter describes the population; a statistic describes a sample.
Parameters use Greek letters (μ, σ); statistics use Roman letters (x̄, s).
A statistic estimates a parameter, but they are not equal. Never write x̄ = μ or s = σ.
The value of a statistic changes from sample to sample. This variability is called sampling variability and is covered in detail in Chapter 3.
In practice, the true value of a parameter is almost always unknown.
Example: pine seedlings. A researcher measures the heights of 40 one-year-old red pine seedlings and finds a sample mean of 1.72 cm. The number 1.72 is a statistic (computed from a sample). The corresponding parameter is the mean height of all one-year-old pine seedlings, which is unknown. The statistic 1.72 estimates that parameter.
Example: unemployment. 500 randomly selected people are interviewed; 40 are unemployed, giving a sample proportion of 0.08. This is a statistic. The parameter is the true proportion of unemployment in the entire county. The statistic 0.08 estimates it, but with some error because only part of the population was observed.
We want to know something about a population (parameter), but we cannot measure the whole thing.
So we take a sample, compute a statistic, and use it to infer the parameter.
The quality of that inference depends entirely on the quality of the sample.
A good sample is a good representative of the population, and getting one requires a good sampling technique.
A biased sampling method produces statistics that systematically differ from the population parameter.
When sampling bias is present, you cannot trust generalisations from sample to population.
Bias is a property of the method, not of a single sample. Even a large biased sample stays biased.
Non-probability methods (biased)
Convenience sampling: survey whoever is around.
Example: "man on the street" interviews. The location and time of day determine who you get.
Bias: opinions are limited to whichever individuals happen to be present. Asking about gun control in Berkeley, CA vs a small town in Arizona would yield very different results.
Voluntary response sampling: let respondents choose themselves.
Often called "public opinion polls" (phone-in surveys, online polls).
Not considered valid or scientific.
Bias: people with strong opinions, especially negative ones, are most likely to respond.
Example: a hospital emails all nurses about a new EHR system. Only nurses who feel strongly (very frustrated or very pleased) reply. The moderate middle is under-represented.
Probability methods (designed to reduce bias)
Particular units are chosen by chance. No group should be over- or under-represented.
Failure to use probability sampling often leads to bias (systematic errors).
Simple random sample (SRS)
Every individual has the same probability of selection.
How to draw one: (1) assign a number from 1 to N to each unit in the population, (2) use a random number generator or random digits table to select n of those numbers.
Example: from a list of 30 minority managers, select 6 at random to interview about performance appraisal.
Stratified random sample
Divide the population into homogeneous subgroups (strata) on a characteristic suspected to affect the response.
Draw a separate SRS from each stratum.
The combined SRSs form the stratified random sample.
The SRS from each stratum need not be the same size (e.g. 100 males and 150 females, or proportional to ethnic group sizes in a state).
Example: divide election districts into urban, suburban, and rural strata, then draw an SRS from each.
Other probability methods mentioned (not covered in depth here): cluster sample, systematic sample, multi-stage sample.
A truly random sample is ideal but often not feasible (no complete list of the population exists, or contacting every member is impossible).
Sometimes you must alter the target population to something you can sample from. Generalisations then apply only to the population you sampled, not necessarily to the one you originally wanted.
Always think critically about potential sources of sampling bias and try to minimise them.
Even with a perfectly random sample, data collected from humans can still be biased.
Question wording: subtle changes in phrasing dramatically shift responses.
"Should the US allow public speeches against democracy?" produced 21% yes.
"Should the US not forbid public speeches against democracy?" produced 39% yes.
Same underlying question, very different results.
Adding specific details also matters: "tax cut vs new government programmes" got 60% for the cut, but "tax cut vs programmes for education, environment, health care, crime-fighting, military defence" got only 22% for the cut.
Context: material presented alongside the question influences answers.
Ann Landers asked readers "Would you have children again?" after printing a letter listing reasons not to have kids: 30% said yes.
Newsday asked the same question of a random sample with no leading material: 91% said yes.
Inaccurate responses: people do not always tell the truth.
93% of US students claimed to be in the top half for driving skill, which is mathematically impossible.
Self-reported drug use (22.7%) is likely an undercount because respondents under-report sensitive behaviour.
Students often say x̄ = μ. It does not. The sample mean estimates the population mean, but they are almost never exactly equal. The same goes for s and σ.
Students confuse "large sample" with "unbiased sample." A large convenience sample is still biased. Size does not fix a flawed sampling method.
Students assume any survey counts as a simple random sample. A survey where respondents opt in (voluntary response) is fundamentally different from one where a random mechanism selects them.
Students mix up the population and the sample. The population is everyone you want to generalise to; the sample is the subset you measured. A common exam trap is asking you to identify which is which in a scenario.
⚠️ You will be asked to identify whether a given number is a parameter or a statistic. The giveaway: if it was calculated from a sample, it is a statistic; if it describes the entire population, it is a parameter.
⚠️ You will be asked to identify the sampling method in a scenario. Key clues: did people choose to respond (voluntary response)? Was whoever was available asked (convenience)? Was a random mechanism used (SRS or stratified)? Were subgroups formed first (stratified)?
⚠️ "Identify the population and the sample" is a very common exam question. Read the scenario carefully: the population is the group the study wants to generalise to; the sample is the group that was measured.
⚠️ Question-wording bias and context bias are tested with examples where two slightly different phrasings produce wildly different results. Be ready to explain why the results differ.
⚠️ Stratified random sampling vs simple random sampling: know that stratified sampling divides first, then samples within each stratum, while SRS draws from the whole population at once.
True or false: the sample mean x̄ is equal to the population mean μ. (False. x̄ estimates μ but is almost never exactly equal to it.)
True or false: a large convenience sample eliminates sampling bias. (False. Bias is a property of the method, not the sample size.)
Fill in the blank: parameters are denoted with ______ letters; statistics are denoted with ______ letters. (Greek; Roman/alphabet.)
True or false: in a voluntary response sample, respondents with moderate opinions are well represented. (False. People with strong opinions, especially negative ones, are most likely to respond.)
Fill in the blank: in a stratified random sample, you first divide the population into homogeneous subgroups called ______. (Strata.)
Q: A marketing firm wants to know if adult men in Flagstaff would shop at a new clothing store. It mails a questionnaire to a simple random sample of 100 residential addresses. What is the population of interest? What is the sample?
A: The population of interest is all adult men in Flagstaff. The sample is the 100 addresses to which the survey was mailed.
Q: A television station asks viewers to phone in and say whether they support federal funding for a policy. Of 2,241 callers, 70.24% are opposed. What kind of sample is this, and is it likely to be reliable?
A: This is a voluntary response sample. It is probably biased, because people with strong opinions (especially opposition) are most likely to call in. The 70.24% figure likely overstates the true level of opposition in the population.
Q: To sample 90 members of a gym, you divide members into men and women, then take an SRS of 45 men and a separate SRS of 45 women. What sampling method is this?
A: This is a stratified random sample. The strata are men and women, and an SRS is drawn from each stratum.
Q: A researcher records the heights of 40 one-year-old red pine seedlings and calculates a mean of 1.72 cm. Is 1.72 a parameter or a statistic? What is the corresponding parameter?
A: 1.72 is a statistic, because it was computed from a sample of 40 seedlings. The corresponding parameter is the mean height of all one-year-old red pine seedlings.
Q: Two versions of a survey question about tax policy produce very different results (60% vs 22% favouring a tax cut). What source of bias explains this?
A: Question wording bias. The version that named specific programmes (education, health care, environment, etc.) made the alternative to a tax cut more concrete and appealing, shifting responses.
This connects to sampling distributions (Chapter 3) because the variability of a statistic from sample to sample is the foundation for confidence intervals and hypothesis tests. If you understand why x̄ changes every time you draw a new sample, the rest of inferential statistics follows from that idea.
It also connects to experimental design (Chapter 1.3 and beyond). Sampling is about how you observe a population; experimental design is about how you assign treatments. Both are concerned with eliminating bias and supporting valid conclusions.
More broadly, the concept of bias reappears throughout the course whenever results might be systematically off-target, whether because of how data were collected, how questions were worded, or how subjects were assigned to groups.
sampling from a population, parameter vs statistic, Greek letters vs Roman letters, μ vs x-bar, σ vs s, population parameter, sample statistic, statistical inference, sampling bias, biased sample, convenience sample, man on the street survey, voluntary response sample, self-selected sample, public opinion poll, probability sample, random sample, simple random sample, SRS, random digits table, stratified random sample, strata, homogeneous subgroups, cluster sample, systematic sample, multi-stage sample, question wording bias, response bias, inaccurate responses, context bias, survey design, STA 270, introductory statistics, sampling methods, sampling techniques