Difficulty: Introductory | Prerequisites: Familiarity with population vs sample (see Data Types and Measurement Levels notes).
Knowing what kind of data you have is only half the story. The other half is how that data was collected and what kind of study produced it. Sampling methods determine whether your results can be generalised to a wider population, and study design determines whether you can claim cause and effect. These two ideas underpin every research paper, poll, and clinical trial you will encounter in this course and beyond. If you already understand population vs sample, this is the natural next step.
Sampling is how you choose a subset of a population to study, and the method you pick affects whether your conclusions hold up. Observational studies record what is happening without interfering; experimental studies deliberately manipulate a variable to test cause and effect. Knowing which is which tells you what kind of conclusions the study can support.
Convenience sampling
A non-random sampling method that uses whichever members of the population are readily available. Think of it as surveying whoever happens to be nearby, which is easy but often biased.
Quota sampling
A type of convenience sampling where the sample is selected to match the population on specific characteristics (e.g. age, gender). In simple terms, you fill quotas for each subgroup rather than selecting randomly.
Cluster sampling
Dividing the population into groups (clusters) and then randomly selecting entire clusters to study. Think of it as picking whole groups at random rather than picking individuals.
Systematic sampling
Randomly selecting a starting point in a list and then selecting every nth individual from that point. In simple terms, you pick a number (say every 10th person on a list) and stick with that interval.
Stratified sampling
Dividing the population into subgroups (strata) and then drawing a proportionate random sample from each subgroup. Think of it as making sure every important subgroup is fairly represented in your sample.
Observational study
A study where the researcher records data without attempting to control or manipulate any variable. In simple terms, you watch and measure but do not interfere.
Experimental study
A study where the researcher deliberately manipulates one or more variables and randomly assigns subjects to groups. Think of it as the classic lab experiment: change one thing, hold everything else constant, measure the outcome.
Bias
A systematic error in sampling or study design that makes results unrepresentative of the population. In simple terms, something about how you collected data is consistently pushing results in one direction.
Treatment
A specific value or condition of the explanatory variable applied to a group in an experiment. Example: in a drug trial, one group receives the drug (treatment) and one receives a placebo (control).
The overarching rule: a good sample is unbiased and representative of the population. The method you use determines how close you get to that ideal.
Convenience sampling
Not random. You use whoever is available.
Fast and cheap, but prone to bias because the available group may not reflect the wider population.
Related method: quota sampling, where you try to match the population's characteristics within a convenience approach.
Cluster sampling
Divide the population into groups (clusters), then randomly select entire clusters.
Useful when the population is spread over a large area. Instead of sampling individuals from everywhere, you sample whole locations.
Example: randomly selecting 10 schools from a district and surveying every student at those 10 schools.
Systematic sampling
Pick a random starting point on a list, then take every nth individual.
Straightforward to carry out with an ordered list.
The course notes describe systematic sampling as a special subset of cluster sampling.
Stratified sampling
Divide the population into meaningful subgroups (strata), then take a proportionate random sample from each.
Ensures that important subgroups are represented. If 60% of your population is female and 40% male, your sample reflects that split.
Example: a university survey that samples proportionally from each year group (freshers, second years, third years, postgraduates).
The key distinction between cluster and stratified sampling often appears on exams. In cluster sampling, you use all members of the selected clusters. In stratified sampling, you take a proportionate sample from every stratum.
This is one of the most important distinctions in introductory statistics, because it determines what kind of conclusions you can draw.
Observational study
The researcher does not attempt to control any aspect of the individuals in the sample.
Variables of interest are simply recorded.
Used when it would be difficult, unethical, or impossible to control factors in the sample.
Can identify associations between variables, but cannot establish causation on its own.
Examples: a study on the relationship between smoking and lung cancer risk, a survey of residents' opinions on a proposed school addition, a study on premature birth and early reading skills.
Experimental study
The researcher deliberately controls one or more aspects of the study.
Individuals are randomly divided into groups, and each group receives a different treatment.
The controlled, randomised design allows the study to establish cause-and-effect relationships.
Example: an agricultural agency divides 100 plots into two groups. One group receives no pesticides, the other receives pesticide treatments. Crop yields are measured at the end of the season.
The crucial difference: only an experiment, with its random assignment and controlled conditions, can support a causal claim. An observational study can show that two things are related, but a lurking variable might be the true cause.
Stratified sampling is used by polling organisations to ensure their surveys reflect the demographics of the voting population, which is why well-run political polls break their sample down by age, region, and education level.
The observational vs experimental distinction shows up in medicine constantly. Ethical constraints mean researchers cannot randomly assign people to smoke, so the link between smoking and lung cancer was established through decades of observational data rather than a single experiment.
Students often confuse cluster sampling with stratified sampling. The difference: in cluster sampling you randomly select whole groups and study everyone in them. In stratified sampling you sample proportionally from every group. Both start by dividing the population into groups, but what happens next is different.
Convenience sampling is sometimes mistaken for a valid random method. It is not random at all, and results from convenience samples should be treated with caution.
A common error is believing that observational studies can prove causation. They can show correlation and suggest possible causal links, but they cannot rule out lurking variables the way a controlled experiment can.
Students sometimes assume that larger samples automatically eliminate bias. A large but non-representative sample (say, surveying 10,000 people who all live in one city) can be more misleading than a smaller, well-designed random sample.
⚠️ Expect scenario-based questions that describe a sampling method and ask you to name it. Pay attention to whether whole groups are selected (cluster) or whether individuals from every group are selected (stratified).
⚠️ "Is this an observational study or an experiment?" is a staple exam question. The giveaway: if the researcher manipulates a variable and randomly assigns subjects to groups, it is an experiment.
⚠️ Follow-up questions often ask: "Can this study establish causation?" The answer for observational studies is always no, only association. For experiments with proper randomisation and controls, yes.
⚠️ Systematic sampling's relationship to cluster sampling (described in the course notes as a special subset) may come up as a detail question.
True or false: Convenience sampling is a type of random sampling. Answer: False. It is non-random and uses whoever is readily available.
Fill in the blank: In __________ sampling, you divide the population into groups and then randomly select entire groups. Answer: Cluster.
True or false: An observational study can establish a cause-and-effect relationship. Answer: False. It can show association, not causation.
Fill in the blank: In an experimental study, individuals are randomly divided into groups and each group receives a different __________. Answer: Treatment.
True or false: Stratified sampling takes a proportionate sample from each subgroup of the population. Answer: True.
Q: A university wants to survey student satisfaction. They email a survey link to every registered student. 400 out of 20,000 respond. What type of sampling is this, and what is a potential problem?
A: This is effectively convenience sampling (also called voluntary response sampling), because only those who chose to respond are included. The potential problem is non-response bias: the 400 who replied may feel more strongly (positively or negatively) than the 19,600 who did not.
Q: A company has 5,000 employees across 50 offices. A researcher randomly selects 8 offices and surveys every employee in those offices. Name the sampling method.
A: Cluster sampling. The offices are the clusters, and entire clusters are selected at random.
Q: A researcher wants to know whether a new teaching method improves exam scores. She randomly assigns 60 students to two groups: one uses the new method, the other uses the traditional method. Both groups take the same exam. Is this an observational study or an experiment?
A: An experiment. The researcher manipulates the teaching method (the explanatory variable) and randomly assigns students to groups.
Q: A study finds that people who drink more coffee tend to sleep fewer hours. Can we conclude that coffee causes less sleep based on this study alone?
A: Not from this information alone. This describes an observational study (no manipulation, no random assignment). There may be lurking variables (e.g. work stress) that drive both higher coffee consumption and shorter sleep. Only a controlled experiment could establish causation.
Q: A pollster divides voters into age brackets (18 to 29, 30 to 44, 45 to 64, 65+) and randomly samples from each bracket in proportion to its share of the electorate. Name the sampling method.
A: Stratified sampling. The age brackets are the strata, and a proportionate sample is drawn from each.
Sampling methods connect directly to probability: later in the course, the theory behind confidence intervals and hypothesis tests assumes that the sample was collected randomly. If it was not, the mathematical guarantees do not hold.
The observational vs experimental distinction feeds into the discussion of correlation vs causation, which recurs throughout regression analysis. When your regression shows that X predicts Y, the study design tells you whether "predicts" means "causes" or merely "is associated with".
Bias introduced by poor sampling also connects to the study of sampling distributions and the Central Limit Theorem, where the assumption of a representative sample is baked in.
sampling methods, convenience sampling, quota sampling, cluster sampling, systematic sampling, stratified sampling, random sampling, sampling bias, non-response bias, observational study, experimental study, experiment vs observation, causation vs correlation, treatment group, control group, random assignment, study design, MATH 399N, intro statistics, ASU statistics, Week 1