Difficulty: Introductory | Prerequisites: Parts 1 and 2 of these notes
Source: Social Psychology: The Science of Everyday Life, 3e, Greenberg et al. | Chapter 1
Tags: internal validity, external validity, construct validity, operational definition, confound, replication, direct replication, conceptual replication, demand characteristics, experimenter bias, debriefing, informed consent, IRB, ethics, deception, confederate
This final section of Chapter 1 covers how researchers evaluate the quality of their studies (validity), common threats to good research (confounds, demand characteristics, experimenter bias), and the ethical rules that govern research with human participants. These concepts are tested heavily because they require you to think critically about study design, not just memorise definitions.
A good study needs validity: internal (the IV caused the change in the DV), external (the findings generalise), and construct (you measured what you meant to measure). Confounds, demand characteristics, and experimenter bias are the main threats to internal validity. Replication, both direct and conceptual, strengthens confidence in findings. Research with human participants must follow ethical rules: IRB approval, informed consent, confidentiality, and debriefing. Science itself has limitations, including questions it simply cannot answer.
Internal validity
The judgement that for a particular experiment, it is possible to conclude that the independent variable caused the change in the dependent variable. In simple terms, did the thing you manipulated cause the thing you measured to change?
External validity
The confidence that findings would generalise across different settings, different groups of people, and different times. Replicating results in other countries increases external validity.
Construct validity
The confidence that the independent variable is a true manipulation of what was intended, and the dependent variable is a true measure of what was intended. In simple terms, did you measure and manipulate what you think you did?
Operational definition
A specific, concrete method of measuring or manipulating a conceptual variable. Measuring anxiety by recording heart rate is an operational definition of anxiety.
Confound
A variable other than the variable intended to be manipulated that may be responsible for an observed effect. If one group sees pictures while another reads text, the format difference is a confound.
Direct replication
Repeating a study using the same methods and procedures as the original. Dr. R repeating Steele's exact stereotype threat procedure is direct replication.
Conceptual replication
Repeating a study with the major conceptual variables operationalised differently, but yielding similar results. Substituting one stressful task for a different stressful task is a conceptual replication.
Demand characteristics
Aspects of a study that give away its purpose or hint at how the participant is expected to behave. A study titled "Being Good Citizens" might tip participants off about the expected outcome.
Experimenter bias
The tendency for researchers to unconsciously influence participants to behave in ways consistent with the hypothesis. Keeping researchers blind to condition is the primary remedy.
Cover story
A false explanation of a study's purpose given to participants to reduce demand characteristics. Telling participants a study is about "validation of personality measures" when it is really about self-esteem threats.
Confederate
An actor working for the study organisers who poses as a real participant. In Milgram's obedience study, the "learner" receiving shocks was a confederate.
Informed consent
The process by which participants are told about procedures and potential risks, and are given the opportunity to ask questions and withdraw at any time.
Debriefing
A procedure at the end of a study during which the true nature of the study is explained to participants, counteracting any negative effects. This is the major ethical safeguard against negative feelings induced by deception.
Institutional review board (IRB)
A committee that must approve all research with human subjects, judging whether the benefits outweigh the ethical costs.
Reverse causality
NOT a form of validity. It is a problem with correlational data (covered in Part 2). Exam questions sometimes include it as a trick option alongside construct, external, and internal validity.
Internal validity: can you conclude the IV caused the change in the DV? Threatened by confounds. Requires random assignment, control of extraneous variables, and good operational definitions.
External validity: do the findings generalise to other people, settings, and times? Threatened when studies rely on convenience samples of Western college students. Replicating in other countries strengthens it.
Construct validity: did you measure and manipulate what you intended? If a study claims to measure "intelligence" using GPA and "agreeableness" using social media followers, construct validity is weak.
Reverse causality is NOT a form of validity. It is a problem with correlational data. Exam questions include it as a distractor.
An abstract conceptual variable (e.g. anxiety) needs a concrete operational definition (e.g. heart rate) to be studied.
Anxiety is the abstract conceptual variable; rapid heart rate is the operational definition.
There is no single "right" operational definition. Using multiple different operational definitions across studies (conceptual replication) strengthens confidence.
Construct validity is the extent to which an operational definition has been successful.
A confound is any variable other than the IV that could explain the observed effect.
Example: one group sees pictures of clowns, the other reads text about a forest. The format (pictures vs text) is a confound, separate from the clowns-vs-forest content.
Tactics to eliminate confounds: (a) random assignment to conditions, (b) anticipating alternative explanations and including control conditions, (c) conceptual replication with different operationalisations.
Getting a random sample from the population does NOT help eliminate confounds (it helps with external validity instead).
Direct replication: repeating the exact same methods and procedures. Confirms that the original finding is reliable.
Conceptual replication: testing the same conceptual hypothesis using different operational definitions. Confirms the finding is not an artefact of one specific procedure.
Both types strengthen confidence in a finding, but in different ways.
Demand characteristics are cues in the study that signal how participants should behave. A study titled "Being Good Citizens" might prime participants to act generously.
Experimenter bias occurs when the researcher unconsciously influences participant behaviour. The remedy is keeping researchers blind to condition.
Both threaten internal validity.
Used to reduce demand characteristics, to create conditions necessary to test a hypothesis, and to create psychologically involving situations.
It is NOT used "to determine if participants can guess how they are being deceived."
When deception is used, participants must receive a thorough debriefing afterwards.
Debriefing is the major ethical safeguard against negative feelings from deception.
IRB approval is required for all research with human subjects.
Informed consent must be obtained, and participants must be free to withdraw at any time.
Confidentiality and anonymity of data must be protected.
The full purpose of a study does NOT need to be explained before participants begin (this is why cover stories and deception are permitted). This is a commonly tested "NOT" question.
Scientists have NOT always abided by ethical principles. The statement that they have is false.
Some aspects of reality are beyond human perception (our senses are limited).
Human values influence what questions scientists choose to ask.
Some questions involve value judgements that science cannot answer (e.g. "Which is the correct religion?").
Scientists are human and can apply methods in biased ways.
The fact that competition between scientists leads to hypotheses being disproven and theories discarded is NOT a limitation; it is a strength. This is a commonly tested trick option.
"Internal validity and external validity are the same thing." They are independent dimensions. A well-controlled lab experiment may have high internal validity but low external validity if it only uses college students.
"The full purpose of a study must be explained to participants beforehand." This is NOT an ethical requirement. Cover stories and deception are permitted precisely because full disclosure would introduce demand characteristics.
"Random sampling and random assignment are the same thing." Random sampling means drawing participants from a population so everyone has an equal chance of being selected (helps external validity). Random assignment means allocating those participants to conditions so everyone has an equal chance of being in any group (helps internal validity).
"Conceptual replication means getting the same result in a different country." That is closer to a test of external validity. Conceptual replication means operationalising the variables differently while testing the same underlying hypothesis.
Be able to distinguish internal, external, and construct validity in a scenario. Expect questions that describe a study and ask which type of validity is weak.
Know that reverse causality is NOT a form of validity. It appears as a distractor option alongside construct, external, and internal.
Confounds: expect scenario-based questions where you identify the confound (e.g. pictures vs text format, not clowns vs forest content).
Know the three ethical safeguards: IRB approval, informed consent, and confidentiality. Know that full disclosure of the study's purpose is NOT required.
Know the difference between direct and conceptual replication and be able to identify which one a described study represents.
The "Being Good Citizens" study title and the Milgram obedience study confederate are frequently tested examples. Know what a demand characteristic is and what a confederate is.
Experimenter bias is addressed by keeping researchers blind to condition. This is a common exam question.
Know which statement about scientific ethics is FALSE: "Scientists have always abided by ethical principles" is the false statement.
Fill in the blank: A variable other than the IV that may be responsible for an observed effect is called a _____. Answer: confound.
True or False: The full purpose of a study must be explained to participants before they begin. Answer: False. Cover stories and deception are permitted; debriefing comes afterwards.
Fill in the blank: Repeating a study with the same methods and procedures is called _____ replication. Answer: direct.
True or False: Keeping researchers blind to condition addresses the problem of experimenter bias. Answer: True.
Fill in the blank: The form of validity concerned with whether findings generalise to other settings and populations is _____ validity. Answer: external.
Q: A researcher measures "intelligence" using GPA and "agreeableness" using social media followers. What type of validity is weak?
A: Construct validity. GPA and social media followers are poor operational definitions of intelligence and agreeableness.
Q: A researcher replicates her results in three other countries using the same operational definitions. What has she primarily increased?
A: External validity.
Q: A study uses pictures for one condition and text for another. What is the confound?
A: The format difference (pictures vs text) is a confound, because it varies alongside the intended manipulation.
Q: Which of the following is NOT an official ethical requirement: (a) informed consent, (b) IRB approval, (c) full disclosure of the study's purpose before participation, (d) confidentiality?
A: (c). Full disclosure is not required; that is why cover stories and deception are permitted.
Q: Dr. R repeats Steele's exact stereotype threat study using the same methods. What type of replication is this?
A: Direct replication.
Q: A study substitutes one stressful task for a different stressful task while keeping the same hypothesis. What type of replication is this?
A: Conceptual replication.
Q: In the Milgram obedience study, the "learner" who pretended to receive shocks was a what?
A: A confederate.
Q: Which of these is NOT a limitation of science: (a) human values influence research, (b) some questions cannot be answered scientifically, (c) competition between scientists leads to disproven hypotheses, (d) our senses are limited?
A: (c). Competition leading to disproven hypotheses is a strength of science, not a limitation.
Validity concepts come back in every research-methods discussion throughout the course. When you read about a study on attitudes or group behaviour, you will be expected to evaluate its internal and external validity. Ethical issues (deception, debriefing, confederates) appear in discussions of classic studies like Milgram's obedience experiments and Zimbardo's Stanford Prison Experiment. The concept of operational definitions connects to measurement in personality psychology and clinical assessment.
Internal validity definition, external validity definition, construct validity examples, operational definition psychology, confound definition, direct replication vs conceptual replication, demand characteristics examples, experimenter bias blind study, cover story psychology, confederate psychology, informed consent research, debriefing psychology, IRB institutional review board, deception in research ethics, limitations of science psychology, Milgram obedience confederate, Greenberg social psychology 3e Chapter 1