Difficulty: Intermediate | Prerequisites: Basic understanding of ANOVA, chi-square tests, and experimental design.
This study sits within PSY 3215 (Advanced Research Methods / Data Analysis) at Florida International University and examines how people react to rude versus polite responses on social media when someone's shared news story is called "fake news." It connects social psychology (impression formation, groupthink, incivility) with quantitative methods (one-way ANOVA, chi-square, Tukey post hoc tests). You should already be comfortable with the logic of independent and dependent variables, random assignment, and basic hypothesis testing. If you are coming in cold, review your notes on between-subjects experimental designs and the ANOVA family first.
Researchers showed participants a fabricated Facebook page where a user named "Corey" responded either rudely, politely disagreeing, or politely agreeing to comments calling a shared news story fake. Participants who read the rude response rated Corey as significantly ruder and felt significantly angrier than those in the two polite conditions. The polite conditions did not differ meaningfully from each other, supporting the hypothesis that tone of disagreement, not the act of disagreeing itself, drives negative impressions.
Independent variable (IV)
The variable the experimenter manipulates. In Study One, the IV is the condition participants were assigned to: Rude Disagree (RD), Polite Disagree (PD), or Polite Agree (PA). Think of it as the thing the researchers changed on purpose to see what would happen.
Dependent variable (DV)
The outcome being measured. Here, two DVs were analysed: (1) agreement with "Corey seems like a rude person" and (2) agreement with "Reading Corey's last post made me feel angry," both on a 1 to 6 Likert scale. In simple terms, this is what the researchers measured to see whether the manipulation worked.
Between-subjects design
An experimental setup where each participant experiences only one condition. Every participant in Study One saw only one version of Corey's response. Think of it as each person getting a single version of the story, with no crossover.
Manipulation check
A question verifying that participants noticed the intended experimental difference. Participants were asked to recall whether Corey rudely disagreed, politely disagreed, or politely agreed. In simple terms, this confirms the experiment did what it was supposed to do.
One-way ANOVA
A statistical test comparing the means of three or more groups on a single continuous outcome. Used here to test whether "Corey seems like a rude person" ratings differed across RD, PD, and PA. Think of it as asking: are these three group averages different enough that it is unlikely to be chance?
Tukey HSD (post hoc test)
A follow-up test after a significant ANOVA that identifies which specific pairs of groups differ. In simple terms, ANOVA tells you "something differs"; Tukey tells you "which ones."
Chi-square test (χ²)
A test for association between two categorical variables. Used here in the manipulation check to see whether participants correctly recalled the condition they were in. Think of it as checking whether the pattern of answers across groups is too lopsided to be random.
Cramer's V
An effect-size measure for chi-square tests, ranging from 0 (no association) to 1 (perfect association). In Study One, Cramer's V was .631 for the manipulation check, indicating a strong effect.
Likert scale
A rating scale (here 1 to 6) capturing the degree of agreement with a statement, from "strongly disagree" to "strongly agree." In simple terms, it turns opinions into numbers so you can run statistics on them.
"Fake News" phenomenon
The tendency for people to label information that conflicts with their existing beliefs as fabricated or unreliable. In this study, commenters called a news story Corey shared "fake," and the researchers measured how Corey's response to that label shaped participant impressions.
Groupthink
The tendency for people within a like-minded group to reinforce each other's views and dismiss outsiders. Relevant here because social media creates echo chambers where rude dismissal of opposing views is common.
Online disinhibition
The loosening of social restraints people normally follow in face-to-face interaction, often resulting in ruder or more aggressive behaviour online. In simple terms, people say things behind a screen they would never say in person.
Social media has become central to how people consume news and interact with others globally (Ismail, Shanmuganathan, & Shaari, 2020)
The "fake news" label is often used to dismiss information that conflicts with one's own views, regardless of its accuracy
False rumours spread faster, farther, and more broadly than true news because they provoke stronger emotional responses (Barfar, 2019)
Hateful speech on social media is linked to groupthink: people cluster with those who share their beliefs, reinforcing their own views and attacking out-group members (Kim & Kim, 2019)
Malicious online comments can cause serious psychological harm, including distress and suicidal ideation (Kim, Bock & Kim, 2019)
Since the 2016 US presidential election, platforms have tried to combat misinformation through "disputed" or "rated false" tags (Clayton et al., 2019)
Main prediction: Participants reading a rude response to "fake news" feedback would rate the user less favourably than those reading either polite condition
Specific predictions: The rude responder would be seen as less reasonable, less polite, less open-minded, less likeable, less civil, less tolerant, more rude, and more willing to mock others
Emotional predictions: Rude-condition participants would feel angrier, less hopeful, less likely to engage with the rude user, and less likely to post their own comment
No difference expected between the two polite conditions (PD and PA)
Design: Between-subjects experiment with three conditions: Rude Disagree (RD), Polite Disagree (PD), Polite Agree (PA)
Participants: 187 total (53.5% male, 46.5% female; mean age 25.35, SD = 9.109; 47.6% Hispanic, 25.1% Caucasian, 16.6% African American, 4.3% Asian American, 1.6% Native American, 4.8% Other)
Stimulus: A fabricated Facebook page belonging to "Corey" (kept gender-neutral). Corey shares a story claiming the FCC forced CNN to change its accreditation from "news" to "entertainment." Two commenters (Peyton Halliburton, Riley Anderson) call the story fake
Manipulation: Only Corey's reply to those comments varied across conditions:
RD: Corey calls commenters "liberal idiots" and morons, insists the story is real
PD: Corey politely disagrees, suggests commenters do their own research
PA: Corey admits the story is fake after doing more research
Measures (five-part survey):
Part 1: Read and absorb the Facebook page
Part 2: Rate impressions of Corey (1 to 6 scale). Key item: "Corey seems like a rude person"
Part 3: Rate impressions of the comments. Key item: "Reading Corey's last post made me feel angry"
Part 4: Demographics (gender, age, race/ethnicity, first language, political affiliation, FIU student status)
Part 5: Manipulation check, recall of Corey's response type
Manipulation check (chi-square):
χ²(4) = 149.11, p < .05 (significant)
69.7% of RD participants correctly recalled Corey rudely disagreeing
66.7% of PD participants correctly recalled Corey politely disagreeing
80.0% of PA participants correctly recalled Corey politely agreeing
Cramer's V = .631 (strong effect), confirming participants noticed the manipulation
DV 1: "Corey seems like a rude person" (one-way ANOVA):
F(2, 184) = 13.797, p < .05 (significant)
Means by condition:
RD: M = 3.38, SD = 1.187
PD: M = 2.98, SD = 0.920
PA: M = 2.71, SD = 0.786
Tukey HSD: Significant difference between RD and PA (mean difference = 0.670, p = .001). The difference between RD and PD approached but did not reach significance (p = .060). PD and PA did not differ significantly (p = .280)
DV 2: "Corey's post made me feel angry" (one-way ANOVA):
F(2, 184) = 26.238, p < .05 (significant)
Means by condition:
RD: M = 3.09, SD = 1.160
PD: M = 2.23, SD = 0.760
PA: M = 2.45, SD = 0.741
Tukey HSD: Significant differences between RD and PD (mean difference = 0.864, p < .001), and between RD and PA (mean difference = 0.636, p = .001). PD and PA did not differ significantly (p = .365)
The hypothesis was largely supported: participants in the RD condition rated Corey as significantly ruder and felt significantly angrier than those in the PA condition
The PD condition fell between the other two on the "rude person" measure and did not differ significantly from either, suggesting that polite disagreement occupies a grey area in people's perceptions
For the anger measure, PD and PA were both clearly distinguishable from RD, confirming that rudeness (not mere disagreement) drives emotional reactions
The researchers noted that the PD condition's ambiguous results may reflect individual differences in what people personally consider rude
This led to the research question motivating Study Two: does a person's age influence whether their online rudeness is perceived differently?
Test | Purpose in This Study | Key Output |
|---|---|---|
Chi-square (χ²) | Manipulation check: did participants recall the correct condition? | χ²(4) = 149.11, p < .05 |
Cramer's V | Effect size for the chi-square (3 × 3 table) | V = .631 (strong) |
One-way ANOVA | Compare means of "rude person" and "angry" ratings across 3 conditions | F(2, 184) = 13.797 and 26.238, both p < .05 |
Tukey HSD | Post hoc pairwise comparisons after significant ANOVA | RD vs PA significant on both DVs; PD vs PA not significant on either |
This research is directly relevant to platform moderation policy. Social media companies deciding how to flag or penalise uncivil responses can draw on the finding that rudeness, rather than mere disagreement, is what damages a user's perceived character and provokes anger in readers. It also applies to digital literacy education: teaching people that how they disagree online matters more than whether they disagree.
Students often think that disagreeing with someone online automatically makes you seem rude. This study shows the opposite: polite disagreement (PD) was not rated as significantly different from polite agreement (PA) on either measure. The tone is what matters, not the position
Students sometimes assume a manipulation check is optional or decorative. Here, the chi-square manipulation check was essential for confirming participants noticed the IV. Without it, any ANOVA result would be uninterpretable
A common error is treating a non-significant Tukey comparison as evidence that two groups are "the same." The PD condition did not differ significantly from RD on the rudeness measure, but that does not mean participants saw them as equivalent; the sample may simply have been underpowered for that comparison
Students sometimes confuse Cramer's V with Cohen's d. Cramer's V is for categorical data (chi-square), while Cohen's d is for comparing two means. They measure different things
⚠️ Know when to use a one-way ANOVA versus a chi-square. ANOVA compares means on a continuous DV across groups; chi-square tests association between categorical variables. This study uses both, so be ready to explain why each was chosen for its specific analysis.
⚠️ Be able to interpret Tukey HSD output. You may be asked to read a multiple-comparisons table and identify which pairs are significant.
⚠️ Understand the role of the manipulation check. An exam question might ask: "What would it mean if the manipulation check had failed?" (Answer: the IV was not effectively delivered, so any results on the DVs would be meaningless.)
⚠️ The distinction between the two polite conditions (PD vs PA) not being significant is a testable point. It demonstrates that polite tone, regardless of whether the person agrees or disagrees, produces similar impressions.
True or False: The study used a within-subjects design. (False, it was between-subjects.)
Fill in the blank: The manipulation check used a ______ test. (Chi-square.)
True or False: Participants in the Polite Disagree condition rated Corey as significantly ruder than those in the Polite Agree condition. (False, PD and PA did not differ significantly.)
Fill in the blank: The post hoc test used after the significant ANOVA was ______. (Tukey HSD.)
True or False: Cramer's V of .631 indicates a weak effect. (False, it indicates a strong effect.)
Q: Why was a one-way ANOVA used instead of a t-test for the main analysis?
A: Because there were three conditions (RD, PD, PA). A t-test only compares two groups; ANOVA handles three or more while controlling the family-wise error rate.
Q: The ANOVA for "Corey seems like a rude person" was significant at F(2, 184) = 13.797. What do the numbers 2 and 184 represent?
A: 2 is the degrees of freedom between groups (number of groups minus 1: 3 – 1 = 2). 184 is the degrees of freedom within groups (total participants minus number of groups: 187 – 3 = 184).
Q: Participants in the PD condition did not significantly differ from those in either the RD or PA condition on the "rude person" DV. What does this suggest about polite disagreement?
A: Polite disagreement occupies a middle ground. Some participants may still perceive any disagreement as mildly rude, while others do not. The condition did not produce a clear enough shift in either direction to reach statistical significance against either comparison group.
Q: Why was Corey's Facebook page kept gender-neutral?
A: To prevent gender stereotypes from confounding the results. If participants knew Corey's gender, their impressions of rudeness might be influenced by gendered expectations rather than the experimental manipulation alone.
Q: If the manipulation check had yielded a non-significant chi-square, what would the researchers conclude?
A: That participants did not reliably distinguish between the conditions, meaning the experimental manipulation failed. Any differences found on the DVs could not be attributed to the IV with confidence.
This connects to social cognition and impression formation: how people form rapid judgements about others based on limited cues (here, a single Facebook exchange). It also ties into the broader research methods curriculum, since it demonstrates how to choose between ANOVA and chi-square depending on data type. Study Two extends these findings by adding a second IV (age), moving the design from a one-way ANOVA to a two-way (factorial) ANOVA, which is a natural next step in the methods sequence.
Facebook rudeness, online incivility, fake news perception, social media impression formation, between-subjects ANOVA, chi-square manipulation check, Tukey HSD post hoc, Cramer's V, Likert scale, groupthink, echo chambers, online disinhibition, political polarisation social media, PSY 3215, FIU psychology, advanced research methods