Difficulty: Introductory | Prerequisites: Understand the difference between qualitative and quantitative data (see Data Types study notes).
Statistics exists because we almost never get to measure every single member of a population. Instead, we take a sample and use it to draw conclusions about the whole group. How you choose that sample determines whether your conclusions are trustworthy or rubbish. This section covers the standard sampling methods (simple random, stratified, cluster, systematic) and the ways a sample can go wrong (voluntary response, convenience, various forms of bias).
You also need to know the difference between a parameter (a number describing the whole population) and a statistic (a number describing a sample), because exam questions test this distinction repeatedly.
If you skipped the data types notes, go back. You need to know what kind of data you are collecting before you worry about how to collect it.
A parameter describes a population; a statistic describes a sample. Good sampling methods (simple random, stratified, cluster, systematic) give every member a known chance of selection. Bad ones (voluntary response, convenience) introduce bias. Bias also comes from how questions are asked (leading questions), who chooses to respond (voluntary response, nonresponse), and what people are willing to admit (social acceptability).
Population
The entire group of individuals or items that you want to study or draw conclusions about.
Think of it as: everyone or everything you care about, even if you cannot reach them all.
Sample
A subset of the population that is actually observed or measured.
In simple terms, the slice of the population you actually collect data from.
Parameter
A number that describes a characteristic of the entire population.
Think of it as: the "true" value. "84% of all residents aged 16+ in a small town own a car" is a parameter because it covers everyone in the town.
Statistic
A number that describes a characteristic of a sample.
In simple terms, it is the estimate you compute from the data you collected. "23% of 1,000 surveyed teenagers use tobacco" is a statistic because it comes from a sample, not the entire population of teenagers.
Simple random sample (SRS)
A sample chosen so that every possible sample of the same size has an equal chance of being selected.
Think of it as: names in a jar, draw at random. The professor who writes 126 names on slips and draws 14 is using an SRS.
Stratified sample
The population is divided into subgroups (strata) that share a characteristic, and a random sample is drawn from each stratum.
In simple terms, you split the group by a relevant feature (party affiliation, age bracket) and then sample randomly within each split. The pollster sampling 145 Democrats, 154 Republicans, and 19 Independents is stratified.
Cluster sample
The population is divided into groups (clusters), some clusters are randomly selected, and every member in those clusters is included.
Think of it as: pick some neighbourhoods at random, then survey everyone in each chosen neighbourhood. The municipality surveying all households in 149 randomly selected neighbourhoods is cluster sampling.
Systematic sample
Members are selected at regular intervals from an ordered list or production line.
In simple terms, pick every kth item. The manufacturer testing every 50th cell phone is using a systematic sample.
Sample of convenience
A sample chosen simply because members were easy to reach, with no randomisation.
Think of it as: you took whoever was there. A student handing leaflets to adults at a school function is using a convenience sample.
Voluntary response sample
A sample in which people decide for themselves whether to participate.
In simple terms, the respondents self-select. A radio show inviting callers, or a survey people choose to return. These should always be regarded as unreliable.
Sampling bias
Bias arising from a method of selecting the sample that tends to favour certain outcomes.
Think of it as: the net you cast caught fish from only one part of the pond. Asking only your friends who they will vote for is sampling bias.
Leading question bias
Bias introduced by wording a question in a way that pushes the respondent toward a particular answer.
In simple terms, the question itself steers the answer. "Do you favour a higher standard of living, even though it produces unclean air and water?" leads the respondent.
Social acceptability bias
Bias that occurs when respondents give answers they believe are socially acceptable rather than truthful.
Think of it as: people telling you what sounds good rather than what is true. Reluctance to admit behaviour that reflects negatively on oneself.
Nonresponse bias
Bias that occurs when a large fraction of those sampled do not respond, and the non-responders differ systematically from those who do.
In simple terms, the people who ignore your survey may think differently from those who fill it in. Only 12 out of 250 questionnaires returned is a textbook nonresponse problem.
Self-interest bias
Bias that occurs when the party reporting the data has a stake in the result.
Think of it as: the grocery store claiming 92% of their own customers love their produce. The store benefits from a favourable number.
Voluntary response bias
Bias that arises when a sample is composed entirely of volunteers, who tend to have stronger opinions than the general population.
In simple terms, only the people who feel strongly enough bother to respond, skewing the results.
A parameter is a fixed (usually unknown) number describing the entire population. A statistic is calculated from a sample and used to estimate the parameter.
"84% of residents in a small town own a car" covers every resident, so it is a parameter.
"23% of 1,000 surveyed teenagers use tobacco" comes from a sample, so it is a statistic.
"42% of televisions in an electronics store cost less than $500" covers every TV in the store (the store's inventory is the population), so it is a parameter.
"64% of respondents in a poll support stricter gun laws" comes from a poll (a sample of the public), so it is a statistic.
Simple random sample (SRS): Every member has an equal chance. Gold standard. Example: names drawn from a jar.
Stratified sample: Divide the population into homogeneous strata, then randomly sample within each. Ensures every subgroup is represented. Example: sampling by political party registration.
Cluster sample: Divide the population into clusters (often geographic), randomly select some clusters, and include all members of those clusters. Example: randomly selecting neighbourhoods and surveying every household in each.
Systematic sample: Select every kth member from an ordered list. Example: testing every 50th phone on an assembly line.
Voluntary response sample: Respondents self-select. Always unreliable because people with strong opinions are overrepresented. Radio call-in polls, mail-back leaflets.
Sample of convenience: You take whoever is easiest to reach. No randomisation, so the sample may not represent the population. Handing out leaflets at a school event.
Sampling bias: The selection method itself favours certain outcomes. Asking only your friends who they plan to vote for.
Leading question bias: The question's wording pushes respondents toward a particular answer. "Do you favour a higher standard of living, even though it produces unclean air and water?"
Social acceptability bias: Respondents give answers that make them look good. People reluctant to admit socially undesirable behaviour.
Nonresponse bias: A large portion of the sample does not respond, and non-responders may differ from responders. Only 12 of 250 questionnaires returned.
Self-interest bias: The party collecting or reporting the data benefits from a particular result. A grocery store surveying its own customers about its own produce quality.
Voluntary response bias: Only those who feel strongly respond. A radio show inviting callers to state opinions on a controversial topic.
Students confuse stratified and cluster sampling. In stratified sampling, you sample from every stratum. In cluster sampling, you randomly select some clusters and take everyone in those clusters. The key difference: stratified touches every group; cluster touches every member of a few groups.
Students assume that if a number describes a subset of people (e.g. residents aged 16+), it must be a statistic. It is not. If the number covers all members of a defined population (all residents aged 16+ in the town), it is still a parameter.
Students mix up nonresponse bias and voluntary response bias. Nonresponse is when people in a properly selected sample fail to respond. Voluntary response is when the sample itself is made up of volunteers from the start.
Students forget that convenience samples and voluntary response samples are always considered unreliable, even if the results happen to look reasonable.
⚠️ Parameter vs statistic is tested repeatedly. Ask yourself: does the number describe every member of the defined group, or just a sample? A survey of 1,000 teenagers yields a statistic. All residents of a town yields a parameter. All TVs in a store yields a parameter (the store is the population).
⚠️ You will be given a scenario and asked to identify the sampling method. Look for these cues: "randomly selected neighbourhoods, surveyed everyone" = cluster. "Sampled from each party" = stratified. "Every 50th item" = systematic. "Names drawn from a jar" = SRS.
⚠️ Voluntary response samples should always be regarded as unreliable. This is stated directly in the textbook and is a common exam question.
⚠️ Bias questions often hinge on a single detail: who is doing the asking (self-interest), how the question is worded (leading), whether people choose to respond (voluntary response / nonresponse), or whether people answer truthfully (social acceptability).
True or false: A voluntary response sample is a reliable sampling method. (False. Always unreliable.)
Fill in the blank: A number that describes a population is called a ______. (Parameter.)
True or false: A pollster who randomly samples Democrats, Republicans, and Independents separately is using cluster sampling. (False. That is stratified sampling.)
Fill in the blank: When only 12 of 250 mailed questionnaires are returned, the primary concern is ______ bias. (Nonresponse.)
True or false: If 42% of all televisions in a store cost under $500, this is a statistic. (False. It describes the entire store inventory, so it is a parameter.)
Q: A municipality surveys all households in 149 randomly selected neighbourhoods. What type of sample is this?
A: Cluster sample. Whole neighbourhoods (clusters) were randomly selected, and everyone in each was surveyed.
Q: A pollster randomly samples 145 Democrats, 154 Republicans, and 19 Independents. What type of sample is this?
A: Stratified sample. The population was divided into strata (party affiliation) and sampled within each.
Q: A manufacturer tests every 50th cell phone. What type of sample is this?
A: Systematic sample. Items are selected at a fixed interval from the production line.
Q: You ask your friends who they plan to vote for and conclude your candidate will win. What type of bias is this?
A: Sampling bias. Your friends are not a representative sample of all voters.
Q: A radio talk show invites people to call in with their opinions. What type of bias is involved?
A: Voluntary response bias. Callers self-select, so those with stronger opinions are overrepresented.
Q: A brewery sends 250 questionnaires and gets only 12 back. What type of bias is this?
A: Nonresponse bias. Most of the sample did not respond, and non-responders may hold different views.
Q: A grocery store claims 92% of its customers think it has the freshest produce. What type of bias is involved?
A: Self-interest bias. The store has a financial interest in reporting a favourable result.
Q: In a survey of 1,000 teenagers, 23% said they use tobacco products. Is this a statistic or a parameter?
A: Statistic. The figure comes from a sample of teenagers, not the entire population.
Q: In a small town, 84% of residents aged 16+ own a car. Is this a statistic or a parameter?
A: Parameter. It describes the entire defined population (all residents aged 16+ in the town).
This connects to experimental design because proper sampling is the foundation of both observational studies and experiments. If your sample is biased, no amount of clever analysis will fix the conclusions.
This also connects to confidence intervals and hypothesis testing later in the course. Those methods assume your sample was collected properly. If it was not, the interval or test result is meaningless regardless of the maths.
Sampling methods, simple random sample, SRS, stratified sample, cluster sample, systematic sample, convenience sample, voluntary response sample, parameter vs statistic, sampling bias, leading question bias, social acceptability bias, nonresponse bias, self-interest bias, voluntary response bias, survey methods, Navidi Chapter 1, elementary statistics sampling