Difficulty: Intermediate | Prerequisites: Introductory research methods, basic statistics (ANOVA, chi-square), familiarity with experimental design terminology.
This study sits within advanced research methods in psychology, specifically the design and analysis of between-subjects experiments. It investigates how the tone of someone's response on social media (rude versus polite) shapes other people's impressions of that person, particularly when the exchange involves correcting misinformation. If you have covered basic experimental design, independent and dependent variables, and the logic of random assignment, you have the groundwork. The study also touches on social perception, online incivility, and the psychology of misinformation, all of which are increasingly central to applied social psychology.
Researchers showed 187 participants a fake Facebook page where someone shared a debunked news story. The key manipulation was how that person responded to commenters who pointed out the story was fake: rudely, politely but in disagreement, or politely agreeing. Participants rated the rude responder as significantly more rude and anger-inducing than either polite condition, and the manipulation check confirmed people noticed the tone differences as intended.
Independent variable (IV)
The factor the researcher deliberately manipulates to observe its effect. In this study, the IV is the response condition: Rude Disagree, Polite Disagree, or Polite Agree.
Think of it as the thing the experimenters changed on purpose to see what would happen.
Dependent variable (DV)
The outcome the researcher measures to see whether the IV had an effect. Here, the DVs are participant ratings of "Corey seems like a rude person" and "Reading Corey's post made me feel angry."
In simple terms, this is what gets measured after the manipulation.
Between-subjects design
An experimental setup where each participant is exposed to only one condition. Each person saw only one version of Corey's response.
Think of it as: different people in different groups, rather than the same people seeing every version.
Manipulation check
A question included in a study to verify that participants noticed and understood the experimental manipulation. Here, participants were asked to recall how Corey responded to the comments.
In simple terms, it confirms the manipulation worked the way you planned.
One-way ANOVA
A statistical test used to compare the means of three or more groups on a single continuous DV. Used here to test whether rudeness ratings differed across the three conditions.
Think of it as a way to ask: are these three group averages different enough that it probably was not just chance?
Tukey HSD (post hoc test)
A follow-up test run after a significant ANOVA to identify which specific pairs of groups differ from each other. The ANOVA tells you "something differs"; Tukey tells you "which ones."
In simple terms, it pins down exactly where the differences lie.
Chi-square test of independence
A test that checks whether two categorical variables are related. Used here for the manipulation check (condition vs. what participants recalled).
Think of it as: are these categories distributed the way you would expect by chance, or is the pattern too strong for that?
Cramer's V
An effect-size measure for chi-square tests, ranging from 0 to 1. A higher value means a stronger association between the two categorical variables. The study reported a strong Cramer's V (.631).
In simple terms, it tells you how big the relationship is, not just whether it exists.
Likert-type scale
A rating scale (here 1 to 6) where participants indicate their level of agreement with a statement. Commonly used in survey research to measure attitudes and perceptions.
Think of it as the "strongly disagree to strongly agree" slider you have seen on countless surveys.
Random assignment
The process of placing participants into experimental conditions by chance, so that pre-existing differences between people are spread roughly evenly across groups. This is what allows causal claims.
In simple terms, randomisation is what separates an experiment from a survey.
Three-condition between-subjects experiment (N = 187, recruited at Florida International University)
Participants randomly assigned to one of three conditions, all viewing the same fake Facebook profile ("Corey") sharing a debunked news story ("FCC Forces CNN to change its accreditation from 'news' to 'entertainment'")
Two commenters (Peyton and Riley) point out the story is fake in all conditions
The only thing that changes across conditions is Corey's reply:
Rude Disagree (RD): Corey calls commenters "liberal idiots" and "morons," insists the story is real
Polite Disagree (PD): Corey disagrees but tells commenters to do their own research
Polite Agree (PA): Corey admits they were wrong and agrees the story is fake
Profile name "Corey" chosen to be gender-neutral
Profile picture deliberately ambiguous (a man and woman hugging, unclear which is Corey)
Intro, Friends, and Life Events sections left intentionally vague or blank
This controls for confounds: participants form impressions based on the post interaction, not extraneous profile details
Part 1: Participants study the Facebook page and all visible content
Part 2: Eight Likert items (1 to 6 scale) rating impressions of Corey, including reasonableness, politeness, open-mindedness, civility, rudeness, tolerance, willingness to mock, and likeability
Part 3: Seven Likert items rating impressions of the conversation, including prior suspicion the story was fake, post-reading belief, willingness to comment, perceived fairness of Peyton and Riley, anger, hope, and willingness to engage with Corey
Part 4: Demographics (gender, age, race/ethnicity, first language, FIU student status, political affiliation)
Part 5 (manipulation check): Participants select which statement best describes how Corey responded (rudely disagreed, politely disagreed, or politely agreed)
Not all survey items were analysed. The researchers focused on:
DV 1: "Corey seems like a rude person" (Part 2)
DV 2: "Reading Corey's last post made me feel angry" (Part 3)
Manipulation check: Part 5 recall question
Manipulation check passed: Chi-square significant, X²(4) = 149.11, p < .05, Cramer's V = .631. Most participants correctly recalled the tone of Corey's response in their assigned condition (RD: 69.7%, PD: 66.7%, PA: 80.0%)
Rudeness ratings (ANOVA): F(2,184) = 13.797, p < .05. Mean rudeness rating highest in RD (M = 3.38), lower in PD (M = 2.98), lowest in PA (M = 2.71). Tukey HSD: RD significantly different from PA (p = .001), but RD vs. PD only approached significance (p = .060), and PD vs. PA was not significant (p = .280)
Anger ratings (ANOVA): F(2,184) = 26.238, p < .05. Mean anger highest in RD (M = 3.09), lower in PA (M = 2.45), lowest in PD (M = 2.23). Tukey HSD: RD significantly different from both PD (p = .000) and PA (p = .001), but PD vs. PA was not significant (p = .365)
Results supported the hypothesis: participants in the RD condition rated Corey as ruder and reported more anger than those in the PA condition
The PD condition did not differ significantly from either RD or PA on rudeness, suggesting the boundary between "polite" and "rude" disagreement is subjective and may vary by individual
The researchers note this raises the question of what constitutes a rude versus polite response, since polite disagreement was not reliably distinguished from rude disagreement in perception
Students often think the ANOVA alone tells you which groups differ. It does not. The ANOVA tells you that at least one group mean is different from the others. You need a post hoc test (here, Tukey HSD) to identify which specific pairs are significantly different.
Students sometimes confuse the manipulation check with the main analysis. The chi-square here was used only to confirm participants noticed the manipulation. The ANOVA was the test of the actual hypothesis.
A common mistake is assuming that because the overall ANOVA was significant, every pair of conditions must differ. In this study, Rude Disagree versus Polite Disagree did not reach significance on the rudeness measure (p = .060), even though the overall F-test was significant.
Students often assume a 6-point Likert scale is interval data by default. The researchers treated it as interval (running ANOVAs), which is common practice, but some statisticians argue ordinal alternatives would be more appropriate. Be prepared to discuss this assumption if your exam covers measurement levels.
⚠️ Know the difference between a manipulation check (confirms the IV worked) and the main analysis (tests the hypothesis). Exams frequently ask you to identify which statistical test served which purpose.
⚠️ Be able to interpret Tukey HSD output: a significant overall F does not mean all pairwise comparisons are significant. You may be asked to read a multiple-comparisons table and identify which pairs reached significance.
⚠️ Understand why the researchers used a one-way ANOVA rather than a t-test. With three groups, running multiple t-tests inflates the family-wise error rate. The ANOVA controls for this.
⚠️ The gender-neutral stimulus design is an example of controlling for confounding variables. Expect questions about why researchers make these design choices and what confound they are preventing.
⚠️ Cramer's V as an effect-size measure for chi-square is a commonly tested concept. Know what values indicate weak, moderate, and strong associations (roughly: .1 small, .3 medium, .5+ large for a 3x3 table).
True or False: The independent variable in this study was the participant's rating of Corey's rudeness.
False. The IV was the experimental condition (RD, PD, PA). The rudeness rating was a dependent variable.
Fill in the blank: The statistical test used to confirm the manipulation worked was a ______ test.
Chi-square test of independence.
True or False: All three conditions produced significantly different rudeness ratings from each other.
False. The RD vs. PD comparison only approached significance (p = .060), and PD vs. PA was not significant.
Fill in the blank: Cramer's V of .631 indicates a ______ association between condition and recall of Corey's response.
Strong.
True or False: Each participant saw all three versions of Corey's response.
False. This was a between-subjects design; each participant saw only one condition.
Q: What was the main hypothesis of this study?
A: That participants who read a rude response to fact-checking comments would perceive the poster as less likeable, less open-minded, more uncivil, and ruder than participants who read a polite response.
Q: Why did the researchers use a gender-neutral name and ambiguous profile photo for the Facebook stimulus?
A: To control for gender-based confounds. If participants could identify Corey's gender, their pre-existing gender expectations might influence their ratings of politeness and rudeness, introducing a confounding variable.
Q: The overall ANOVA for rudeness ratings was significant at F(2,184) = 13.797, p < .05. Does this mean the Rude Disagree condition produced significantly higher rudeness ratings than the Polite Disagree condition?
A: Not necessarily. The Tukey HSD post hoc test showed the RD vs. PD difference only approached significance (p = .060). The significant overall F indicates at least one pair differs, but you must check the post hoc comparisons to see which specific pairs.
Q: What role did the chi-square test play in this study, and how is that different from the ANOVA?
A: The chi-square was used for the manipulation check, testing whether participants' recall of Corey's response was associated with the condition they were assigned to. The ANOVA tested the main hypothesis about whether condition affected rudeness and anger ratings. One confirms the manipulation worked; the other tests the research question.
Q: Why might the Polite Disagree condition not have produced significantly different rudeness ratings from the Rude Disagree condition?
A: The researchers suggest that participants may have personal, subjective thresholds for what counts as rude. Politely disagreeing with someone who is factually correct might still be perceived as somewhat rude by some participants. The boundary between polite disagreement and rude disagreement may be less clear-cut than the boundary between rudeness and polite agreement.
Q: If you were to replicate this study, name one design improvement you would recommend and explain why.
A: Possible answers include: using a larger sample to increase statistical power (particularly for the RD vs. PD comparison that approached but did not reach significance), adding a no-response control condition, using a within-subjects design with counterbalancing to reduce individual-difference noise, or pre-testing the stimuli to ensure the "polite disagree" condition is perceived as clearly polite.
This study connects directly to social cognition and impression formation: the core question is how people form judgements about someone based on limited online behaviour. If your course covers attribution theory or the fundamental attribution error, consider how participants may have been making dispositional attributions about Corey ("Corey is a rude person") based on a single interaction.
The misinformation angle ties to the psychology of persuasion and belief persistence. Even after being told a story is fake, the original poster's willingness to accept correction (or not) shapes how others perceive them, which has implications for how misinformation corrections succeed or fail in real social media environments.
The methodology connects to your broader research methods training: between-subjects design, manipulation checks, ANOVA with post hoc tests, and chi-square for categorical data are all foundational techniques you will encounter repeatedly across experimental psychology.
online incivility, social media rudeness, impression formation, fake news perception, misinformation correction, between-subjects experiment, one-way ANOVA, Tukey HSD, chi-square test of independence, Cramer's V, manipulation check, Likert scale, Facebook experiment, social perception, politeness norms, random assignment, post hoc comparisons, response tone, civility online, PSY 501, FIU psychology, advanced research methods