Difficulty: Intermediate
Prerequisites: Read the companion doc (Internal Validity Threats and MR SMITH ID, PSY 301 - Study Notes) for definitions and the mnemonic before working through these scenarios.
This document walks through each of the ten scenarios from Assignment #9 (Smartphone-Delivered Mindful Eating Intervention). For every scenario, you get a short summary of what happened, the correct MR SMITH ID threat, and a clear explanation of why that threat fits. Use this as a self-test: read the summary, try to identify the threat yourself, then check the reasoning.
Ten scenarios, ten answers (nine threats plus one "No Threat"). Each scenario introduces one specific twist to the base study. This doc breaks down each twist, names the threat, and explains the logic so you can replicate it on exam day.
What happened: Dieters were randomly assigned. The pre-study and post-study questionnaires used the exact same questions. Some dieters commented that they recognised the post-study questions from the pre-study and based their post-study answers on what they had said before. No difference was found between the two groups.
Why Testing: The critical clue is that participants remembered the pre-test questions and anchored their post-test responses to those earlier answers. The act of taking the pre-test changed how they responded on the post-test. This is the textbook definition of a testing threat. It is not instrumentation, because the measurement tool did not change; it stayed the same, which is precisely what allowed participants to recognise the questions.
What happened: The researcher noticed that some dieters were morbidly obese and, out of medical concern, deliberately assigned those high-risk (extreme-scoring) participants to the smartphone intervention. Lower-weight dieters were randomly assigned to either condition. The smartphone group lost significantly more weight.
Why Regression to the Mean: Morbidly obese participants were selected specifically because of their extreme scores (very high weight). Extreme scores naturally tend to move closer to the average on a second measurement, regardless of treatment. By concentrating the most extreme cases in the smartphone group, the researcher guaranteed that group would show the largest pre-to-post drop, even without a genuine treatment effect. This looks like selection at first glance, but the distinguishing feature is the selection of extreme scorers, which makes regression the better fit.
What happened: Researchers tracked login rates. Smartphone users logged in more frequently (90% daily) than website users (60% daily). The smartphone group showed better outcomes.
Why No Threat: This scenario describes a difference in engagement that is a natural consequence of the delivery method being tested. The smartphone app is more convenient and accessible, so higher login rates are part of the treatment effect the study is designed to measure. No confound has been introduced; the groups were presumably randomly assigned, and no external event, measurement change, or attrition issue is described. The difference in login frequency is the mechanism by which the independent variable works, not a threat to validity.
What happened: The researcher asked about prior dieting experiences and deliberately placed dieters who had failed at least five previous diets into the smartphone condition. The smartphone group benefited more.
Why Selection: Participants were assigned to conditions based on a pre-existing characteristic (number of past diet failures), not randomly. This means the two groups differed systematically before the study began. The smartphone group may have been more motivated, more experienced with dieting, or simply different in ways that affected the outcome. Any observed group difference could be due to who was in each group rather than which platform they used. This is not regression to the mean because the selection criterion was not an extreme score on the dependent variable; it was a count of past failures.
What happened: The website intervention was riddled with technical glitches: slow loading, videos failing, participants needing to log out and back in. The smartphone app ran smoothly. The smartphone group benefited more.
Why Instrumentation: The delivery mechanism (the "instrument") functioned differently across the two conditions. The website platform was unreliable, while the smartphone platform worked as intended. This means the two groups did not receive the same quality of intervention delivery. The measurement instrument, broadly conceived, changed between conditions. The difference in outcomes could be due to the technical problems rather than any genuine advantage of smartphone delivery.
What happened: A new TV programme about mindful eating started airing mid-study. Many dieters watched it. The show appealed mostly to women in the study. Female dieters showed no difference between smartphone and website conditions, but male dieters showed a smartphone advantage.
Why Interactions: An external event (the TV programme) occurred during the study, which would normally suggest History. However, the event affected the groups asymmetrically: it primarily influenced women, washing out any group difference among female participants while leaving male participants unaffected by the show. This differential impact of an external event across a participant characteristic (sex) is the hallmark of an Interactions threat (also called selection x history). The TV show acted as a confound for one subgroup but not the other.
What happened: Dieters were randomly assigned. They did not socialise with each other. The pre-study and post-study questionnaires covered the same content areas (including mindfulness), but the actual questions on the post-study questionnaire were different from those on the pre-study questionnaire. The smartphone group benefited more.
Why No Threat: Random assignment was used, groups were isolated, and the fact that the post-test questions differed from the pre-test questions actually eliminates the testing threat (participants could not simply recall and repeat their earlier answers). The content area was consistent, so measurement validity was maintained. No external event, attrition, or non-random assignment is described. This scenario controls for the major threats.
Note: Since "No Threat" can only be used once across the ten items and Scenario 3 is also a strong candidate, your professor's intended mapping may place No Threat on either 3 or 7. Read both carefully and use the one that best fits your professor's emphasis. If Scenario 3 is No Threat, then Scenario 7 might be considered Instrumentation (the questions changed between pre and post). Check your course materials for guidance.
What happened: A new TV programme about mindful eating started airing mid-study. Many dieters in both conditions watched it. No difference was found between the smartphone and website groups.
Why History: An external event (the TV programme) occurred during the study and influenced participants in both conditions. Because it affected both groups similarly, it washed out any potential difference between the groups. The result, no group difference, is consistent with History: the outside event provided the same benefit (or distraction) to both conditions, making it impossible to detect a treatment effect. This is distinct from Interactions (Scenario 6) because the event did not differentially affect one subgroup.
What happened: At the start of the study, participants in both conditions struggled with the technology: accessing modules, answering questionnaires electronically. Over the course of the month, they became more familiar and comfortable with the platforms. No differences were found between the two groups.
Why Maturation: Participants changed over time, not because of the treatment, but because they naturally grew more comfortable with the technology through repeated exposure and the passage of time. This improvement occurred equally in both conditions and is unrelated to the mindfulness content. The change is due to participants maturing (in the broad sense of developing, adapting, or growing accustomed to the study environment), which is the definition of the maturation threat.
What happened: Many participants knew each other from a weight-loss support group. They continued meeting outside the study and compared notes about their experiences. Website-group participants learned about the smartphone app and started using it. Ultimately, participants in both conditions had access to the smartphone app content. No difference was found between the groups.
Why Diffusion: Participants in the control condition (website) gained access to the experimental treatment (smartphone app) through social contact with participants in the other condition. This contamination blurred the distinction between the two groups, making it impossible to detect a treatment effect. The groups were no longer truly separate, which is the definition of treatment diffusion. The giveaway is that participants across conditions shared information and accessed each other's materials.
Scenario | Threat | Key Clue |
|---|---|---|
1 | Testing | Participants remembered pre-test questions and anchored post-test answers to them |
2 | Regression to the mean | Morbidly obese (extreme scorers) deliberately placed in the smartphone group |
3 | No Threat | Login rate difference is a natural feature of the treatment, not a confound |
4 | Selection | Dieters with 5+ past failures assigned to smartphone; non-random assignment |
5 | Instrumentation | Website platform had glitches; smartphone ran smoothly; delivery tool differed |
6 | Interactions | TV show affected women in both groups equally but not men; asymmetric external event |
7 | No Threat | Random assignment, isolated groups, different post-test questions (eliminates testing) |
8 | History | TV show affected both groups equally; washed out group differences |
9 | Maturation | Participants grew comfortable with technology over time; natural adaptation |
10 | Diffusion | Participants shared treatment info across conditions; control group used the app |
Note: Scenarios 3 and 7 are both strong candidates for "No Threat." Since the assignment says each threat is used only once, one of these two must carry "No Threat" and the other must be assigned a different label. If your professor treats Scenario 7 as Instrumentation (because the post-test questions changed), then Scenario 3 becomes No Threat. Check your course answer key.
Read the base study once, thoroughly. Every scenario is a variation on the same experiment. Knowing the base design cold means you only need to spot what changed.
Work through MR SMITH ID letter by letter. For each scenario, ask: does this describe Maturation? Regression? Selection? And so on. Most scenarios make one threat obviously stronger than the rest.
Use process of elimination. Each threat is used once. If you are confident about eight answers, the remaining two are constrained.
Check the result pattern. Did the study find a group difference, or no difference? External events that wash out differences suggest History or Diffusion. Non-random assignment that inflates differences suggests Selection or Regression.
Look for the distinguishing detail. Many scenarios share surface similarities. The one sentence that sets a scenario apart (e.g. "participants remembered the questions" vs. "the questions changed") is usually the sentence that points to the correct threat.
Q: A researcher selects the 20 students with the lowest exam scores and enrols them in a tutoring programme. After tutoring, their scores improve. Which MR SMITH ID threat is most likely?
A: Regression to the mean. The students were selected for extreme low scores, which would naturally move closer to the average on retesting regardless of the tutoring.
Q: In a drug trial, the new-drug group loses 30% of participants due to side effects, while the placebo group retains nearly all participants. The remaining new-drug participants show better outcomes. Which threat?
A: Mortality. Differential attrition changed the composition of the new-drug group: only participants who tolerated the drug (likely healthier or more responsive) remained, skewing the results.
Q: A study uses trained observers to rate classroom behaviour at the start and end of the semester. By the end, the observers have become more lenient in their ratings. Which threat?
A: Instrumentation. The measurement tool (the observers' rating standards) changed between pre-test and post-test.
Q: Participants in a control group learn about the experimental group's treatment from social media and begin practising the techniques on their own. No group difference is found. Which threat?
A: Diffusion. The control group gained access to the treatment, contaminating the comparison.
Q: A researcher randomly assigns participants to two conditions, uses different but content-equivalent pre and post measures, keeps groups separate, and finds that the experimental group outperforms the control group. No unusual events occur during the study. Which threat?
A: No Threat. The study controlled for the major threats: random assignment (eliminates selection), different test forms (eliminates testing), group isolation (eliminates diffusion), and no external events (eliminates history).
Internal validity threats applied, MR SMITH ID scenarios, testing threat example, regression to the mean example, selection threat, experimental mortality, instrumentation, history threat, interactions selection-history, treatment diffusion, contamination, mindfulness diet study, smartphone intervention, PSY 301, research methods practice questions, exam prep internal validity