Difficulty: Intermediate
Prerequisites: Basic understanding of experimental design (independent/dependent variables, control vs. experimental groups, random assignment). Chapter material on research methods is helpful.
Internal validity is the backbone of any experiment. If your study lacks internal validity, you cannot confidently say that your independent variable caused the change in your dependent variable, because something else may have done it instead. This topic sits squarely in the research methods portion of PSY 301 and builds on everything you have learned about experimental design, random assignment, and controlling for confounds.
The MR SMITH ID mnemonic gives you a checklist of nine specific threats (plus "No Threat") that can undermine an experiment's internal validity. Being able to spot these threats in a described study is a core exam skill, and the scenarios in this unit train you to do exactly that.
Internal validity threats are alternative explanations for why your results turned out the way they did, beyond the effect of your independent variable. The MR SMITH ID mnemonic covers nine threats: Maturation, Regression to the mean, Selection, Mortality, Instrumentation, Testing, History, Interactions, and Diffusion. Learn to identify each one in a study scenario and you will handle the exam questions confidently.
Internal validity
The degree to which an experiment demonstrates that the independent variable, and only the independent variable, caused the observed change in the dependent variable. In simple terms, it answers: "Can I trust that my treatment actually produced the result?"
MR SMITH ID
A mnemonic for the nine threats to internal validity: Maturation, Regression to the mean, Selection, Mortality, Instrumentation, Testing, History, Interactions, Diffusion. Think of it as a checklist you run through when evaluating any experiment.
Maturation (M)
Changes in participants over time that are due to natural development or growing familiarity, not the treatment itself. Think of it as: participants getting better (or worse) simply because time passed, they practised, or they became more comfortable with the task.
Regression to the mean (R)
The statistical tendency for extreme scores on a first measurement to move closer to the average on a second measurement, regardless of any treatment. In simple terms, if you select people because they scored very high or very low, their next scores will likely drift toward the middle on their own.
Selection (S)
A threat that arises when participants are not randomly assigned to conditions, creating systematic differences between groups before the study even begins. Think of it as: the groups were unequal from the start, so any difference at the end may reflect who was in each group rather than what the treatment did.
Mortality (M)
Also called attrition or experimental mortality. This occurs when participants drop out of a study at different rates or for different reasons across conditions, changing the composition of the groups. In simple terms, if one group loses more participants (or a particular type of participant), the remaining sample is no longer comparable.
Instrumentation (I)
A change in the measurement tool or procedure between the pre-test and post-test, or between conditions, that could account for observed differences. Think of it as: if the ruler changed between measurements, you cannot be sure whether the object grew or the ruler shrank.
Testing (T)
The effect of taking a pre-test on participants' performance on a subsequent post-test. Participants may remember their earlier answers and respond consistently, or they may become sensitised to the topic. In simple terms, the act of being measured the first time changes how they respond the second time.
History (H)
An external event that occurs during the study, outside the researcher's control, that affects the outcome variable for all or some participants. Think of it as: something happened in the world (a news story, a new programme, a policy change) that influenced results independently of the treatment.
Interactions (I)
Also called selection-by-history or selection interactions. This threat occurs when an outside event (history) affects one group differently from another, creating a group-specific confound. In simple terms, the external event hit one condition harder than the other.
Diffusion (D)
Also called contamination or treatment diffusion. This occurs when participants in different conditions communicate with one another and share information about the treatment, blurring the line between the groups. Think of it as: the control group got a taste of the experimental treatment (or vice versa), so the groups are no longer truly separate.
Each letter in MR SMITH ID maps to one threat. When you read a study scenario on an exam, work through the letters one by one and ask whether the scenario describes that threat. Most scenarios contain one dominant threat; your job is to pick the best fit.
M - Maturation: Did participants change simply because time passed or they grew more comfortable?
R - Regression to the mean: Were participants selected because of extreme scores that would naturally drift back toward average?
S - Selection: Were groups formed in a non-random way that made them systematically different before the study started?
M - Mortality: Did participants drop out unevenly across conditions?
I - Instrumentation: Did the measurement tool or delivery platform change between pre-test and post-test, or differ between conditions?
T - Testing: Did taking the pre-test influence how participants responded on the post-test?
H - History: Did an outside event occur during the study that could explain the results?
I - Interactions (selection x history): Did an outside event affect one group differently from the other?
D - Diffusion: Did participants in different conditions share information about the treatment with each other?
History vs. Interactions: History is an external event that affects everyone roughly equally (washing out differences between groups). Interactions is when that external event hits one group more than the other (creating or masking a group difference). If the scenario says the event affected both groups the same and no group difference was found, that leans toward History. If the event affected one subgroup more (e.g. only the women, only the older participants), that is Interactions.
Selection vs. Regression to the mean: Both involve non-random group composition, but they differ in mechanism. Selection is about assigning systematically different people to different groups. Regression to the mean is about selecting people for extreme scores and then watching those scores drift back to average. The giveaway for regression is that participants were chosen because they were at one extreme (e.g. morbidly obese, the worst performers).
Instrumentation vs. Testing: Instrumentation is about the tool or platform changing. Testing is about the participant being affected by having taken the same test before. If the questions changed between pre and post, that is instrumentation. If the questions stayed the same and participants remembered their earlier answers, that is testing.
Mortality vs. Selection: Mortality is about who leaves the study after it starts. Selection is about who was put into each group in the first place.
One scenario on your assignment will have no internal validity threat. This is typically a scenario where random assignment was used, groups were kept separate, measurements were consistent, and no external events intruded. Read carefully before defaulting to "No Threat" as it only applies once.
The assignment uses one base study throughout all ten scenarios. Knowing this setup cold saves you time on the exam.
Topic: Mindfulness-based dieting, which teaches people to notice what, when, and why they eat rather than simply restricting food.
Three steps of mindfulness eating: (1) Increase awareness of triggers (emotions, social context, food availability). (2) Think about the outcomes of eating behaviour ("Will these cookies actually make me feel better?"). (3) Learn to sit with cravings rather than suppress them.
Experimental group: Uses a smartphone app with short modules (5 to 10 minutes each) covering mindfulness and diet.
Control group: Logs into a computer website to review the same module content.
Pre-study questionnaire: Collects data on cravings, emotional eating, binge eating, starting weight, and related measures.
Post-study questionnaire: Collects the same measures again after the intervention.
Prediction: Dieters in the smartphone group will benefit more (lose more weight, become more mindful) than those in the website group.
The design is a pre-test/post-test with two conditions. Each of the ten scenarios introduces a twist to this base setup, and your task is to identify which MR SMITH ID threat that twist represents.
Students often confuse History with Interactions. If an outside event affected both groups equally and no group difference was found, that is History. If the event affected one subgroup differently (e.g. only women watched the TV show), that is Interactions. The key question is: did the external event hit the groups symmetrically or asymmetrically?
Students frequently mix up Selection and Regression to the mean. Both involve non-random assignment, but regression specifically involves selecting participants for extreme scores. If the scenario says "we put the most extreme cases in one group," think regression. If it says "we put people with a particular characteristic in one group," think selection.
Testing and Instrumentation are often confused. Testing is about the participant remembering the pre-test and adjusting their post-test answers accordingly. Instrumentation is about the measurement device or platform itself changing or functioning differently. Ask: did the person change because of the test, or did the test itself change?
Students sometimes label a scenario as Mortality when no one actually dropped out. Mortality requires differential attrition. If participants simply engaged less (lower login rates), that is closer to an instrumentation or dosage problem, depending on the scenario's emphasis.
⚠️ Each threat is used only once across the ten scenarios. If you have already used "Selection" for one question, it will not be the answer for another. Use process of elimination.
⚠️ One scenario will have "No Threat" as the correct answer. Do not force a threat where none exists.
⚠️ The question order and answer order on Canvas may differ from the assignment sheet. Focus on the content of each scenario, not its number.
⚠️ When threats seem to overlap, choose the BEST (most specific) threat. The assignment instructions say so explicitly.
⚠️ Pay close attention to whether the study found a difference between groups or no difference. The pattern of results is a clue. External events that wash out group differences suggest History or Diffusion. Non-random assignment that inflates group differences suggests Selection or Regression to the mean.
True or False: Regression to the mean occurs whenever participants are randomly assigned to groups. ___
Fill in the blank: The threat called ___ occurs when participants in the control group learn about and access the experimental treatment.
True or False: If an external event affects both groups equally and eliminates group differences, the threat is Interactions. ___
Fill in the blank: When participants remember the pre-test questions and base their post-test answers on those memories, the threat is ___.
True or False: Mortality refers to participants dying during the study. ___
Answers:
False. Regression to the mean occurs when participants are selected for extreme scores, not when they are randomly assigned.
Diffusion.
False. That describes History. Interactions is when the event affects one group differently from the other.
Testing.
False. "Mortality" in this context means participants dropping out of the study (attrition), not literal death.
This material connects directly to external validity (whether results generalise beyond the study) because a study that fails on internal validity cannot meaningfully address external validity either. It also ties into the broader research methods discussion of confounding variables: each MR SMITH ID threat is essentially a specific type of confound.
If your course covers quasi-experimental designs, you will see these threats come up again, since quasi-experiments lack full random assignment and are therefore more vulnerable to Selection, Regression to the mean, and Interactions.
Internal validity, threats to internal validity, MR SMITH ID, maturation threat, regression to the mean, selection bias, experimental mortality, attrition, instrumentation threat, testing effect, history threat, selection-history interaction, treatment diffusion, contamination, confounding variables, pre-test post-test design, random assignment, quasi-experimental design, PSY 301, research methods, mindfulness dieting, smartphone intervention study