Source: Qi & Yu, "Participatory Design in Human-Computer Interaction: Cases, Characteristics, and Lessons"
Tags: PD in HCI, content analysis, PRISMA, application areas, PD functions, research method, design approach, design space exploration, healthcare PD, education PD, civic challenges
Difficulty: Intermediate | Prerequisites: Part 1 of these notes (PD history and core dimensions).
This section covers how the researchers actually conducted their study and what they found about where and why HCI scholars use PD. Knowing the method matters because it shapes how far you can generalise the findings. Knowing the application areas and functions matters because it reveals PD's practical role in the field and shows where the community's habits diverge from PD's original principles. If you are studying research methods in HCI or preparing for a design-methods exam, this is where the empirical detail lives.
The researchers sampled 185 self-claimed PD papers from 26 SIGCHI conferences (2014 to 2024), coded them using content analysis, and achieved data saturation by the 68th paper. PD in HCI is most commonly applied in healthcare, education, and communication/social media. Its three primary functions are: exploring design spaces (most common), developing new solutions, and studying PD itself as a research context.
PRISMA Protocol
A standardised reporting guideline for systematic reviews and meta-analyses, used here to structure the paper search and screening process. In simple terms, it is a checklist that makes sure a literature search is transparent and reproducible.
Data Saturation
The point in qualitative analysis where no new themes or codes emerge from additional data. The researchers confirmed saturation at paper 34 for application features and paper 68 for PD features.
Cohen's Kappa
A statistical measure of inter-rater reliability that accounts for chance agreement. The study achieved a score of 0.83, rated "Almost Perfect," indicating the two primary coders and a third independent coder agreed consistently.
Design Space Exploration
Using PD primarily as a research method to understand user needs, define design directions, or develop guidelines, without necessarily building a finished product. This was the most common function of PD in the sampled HCI papers.
Evolutionary Prototyping
A design approach where participants iterate on existing technologies or early prototypes created by designers, refining and improving them through feedback cycles, rather than designing from scratch.
MAXQDA
A qualitative data analysis software package used for coding and tagging textual data. The researchers used it to tag relevant information across all 185 papers.
The researchers searched the ACM Digital Library for the exact term "Participatory Design" in paper abstracts, limited to SIGCHI conference publications from January 2014 to May 2024.
Initial search returned 622 results. A loose title-and-abstract screening reduced this to 519. Full-text screening for explicit PD claims narrowed the set to 185 included papers.
Exclusion criteria removed papers that did not self-claim conducting PD, non-empirical papers (argument papers, literature reviews), non-full papers (posters, workshop papers), and papers from non-SIGCHI venues.
Two researchers (A1 and A2) performed iterative coding using MAXQDA and Excel. A1 coded all 185 papers initially; A1 and A2 refined themes collaboratively through regular meetings.
A third independent coder validated reliability on a random sample of 10 papers, achieving a Cohen's Kappa of 0.83.
Two analytical lenses were applied to each paper: application features of PD (areas and purposes) and PD features (the four core dimensions from Part 1).
No new application-feature themes emerged after the 34th paper; no new PD-feature themes after the 68th.
Eight additional papers from four non-SIGCHI HCI journals were coded as a generalisability check. No new themes appeared, supporting the claim that findings reflect the broader HCI landscape.
Both authors are design researchers with direct teaching experience in PD. They adopt a strict stance on PD's boundaries, especially its democratic empowerment principle.
They consider practices that merely use participants as testers or informants, without granting co-decision power, as misconceptions of PD.
Of the 185 papers, 48.65% were published at CHI, followed by DIS (12.43%), CSCW (11.89%), and IDC (11.89%).
Peak publication years were 2021 and 2023, each at 14.05%.
Geographically, 45.41% of studies were conducted in the USA, 13.51% in the UK, and 7.03% in Germany. Only 33 countries or regions were represented in total, highlighting a Western-centric bias.
The nine application areas identified, ranked by frequency:
Healthcare and well-being (N=63, 34.05%): The largest area by a wide margin. Examples include designing assistive robots for elderly people and exploring technology for managing menstrual hygiene.
Education and learning (N=29, 15.68%): Includes VR-based educational practices and culturally relevant computer science curricula for young learners.
Communication and social media (N=25, 13.51%): Examples include improving mixed-group video conference communication and reimagining social media with autistic adults.
Technology and interaction (N=20, 10.81%): Focused on creating new interfaces or interaction mechanisms, such as foldable smartwatches.
Civic challenges (N=16, 8.65%): Addresses issues like racism, discrimination, and the experiences of transgender and non-binary people.
Transportation and travelling (N=11, 5.95%): Designing for vehicle interfaces and shared automated vehicles.
Creativity support (N=10, 5.41%): Tools that help designers express and communicate creative concepts, such as digital mood boards.
Privacy and security (N=8, 4.32%): Designing safety apps for women and enhancing privacy in system design.
Sustainable development (N=3, 1.62%): The smallest category, using PD fiction to promote sustainable energy consumption.
A key pattern: 84.32% of papers addressed human-centred matters (empowering elderly people, supporting refugees), while only 15.68% focused on technical-oriented topics (wearable devices, gesture-based interaction).
This was the dominant use. Researchers employed PD to:
Understand user experiences and perspectives through ethnographic approaches combined with design activities (interviews, focus groups, sketching, prototyping).
Explore and define design directions by engaging participants in identifying challenges and conceptualising solutions.
Develop design principles and guidelines by involving participants in identifying critical considerations for specific user groups (e.g., children with ADHD).
Two sub-types were observed:
Developing new designs from scratch: Participants involved from understanding needs through to ideation, prototyping, and evaluation. In some cases, participants contributed mainly during ideation, with designers handling later prototyping.
Evolutionary prototyping: Participants iterated on existing technologies or early prototypes, providing feedback to refine designs. This often extended into user testing and field deployment.
These papers investigated PD itself:
Philosophical investigations examined PD's concepts and values, such as challenging privileged assumptions in traditional PD workshops.
Methodological investigations developed new PD methods and tools, such as Situated Participatory Design for engaging elderly people.
These three functions sometimes coexisted within a single paper. For example, a study might develop a new solution through PD and then reflect on design guidelines in the discussion section.
No mathematical formulas are involved, but the PRISMA flow diagram is important to understand:
622 records identified → 519 after loose screening → 185 after full-text screening (334 excluded for: not claiming PD [48], not full papers [72], no empirical data [17], non-SIGCHI venues [197]).
The dominance of healthcare and education as PD application areas reflects where participatory approaches have the most obvious ethical justification: designing systems for vulnerable populations (elderly people, children, patients) where getting the design wrong has real consequences. In industry, PD principles show up in service design for the NHS, community-centred technology projects, and accessibility design.
Students sometimes think PD can only be used to build finished products. In HCI, PD is more often used as a research method to explore design spaces (over half of all cases), producing guidelines and directions rather than deployable systems.
The fact that a paper mentions "participatory design workshops" does not necessarily mean it conducted PD. The researchers excluded papers that used the term loosely without actually involving participants in design decisions.
PD is not limited to in-person workshops. About 12% of the studies were conducted entirely online, and another 10% used mixed (online and in-person) settings.
A Western-centric bias in the data does not mean PD is only relevant in Western contexts. It means the current HCI literature under-represents non-Western PD practices.
⚠️ The three functions of PD in HCI (research method, design approach, research context) are a core finding. Be ready to define each and give an example.
⚠️ Healthcare and well-being is the dominant application area by far (34%). This is a useful data point for exam questions about where PD is most commonly applied.
⚠️ The "design space exploration" function (52.43%) is significantly more common than "developing new solutions" (26.49%). This asymmetry is important for the discussion of whether HCI scholars are under-delivering on PD's empowerment promise.
⚠️ The Western-centric geographic distribution is a stated limitation. If an exam asks about limitations or biases, this is the primary one.
True or false: The researchers analysed every HCI paper ever published on Participatory Design. False – they sampled 185 papers from SIGCHI conferences over a ten-year period.
Fill in the blank: The most common application area for PD in HCI was ________. Healthcare and well-being.
True or false: PD in HCI is most commonly used as a design approach to build finished products. False – it is most commonly used as a research method to explore design spaces (52.43%).
Fill in the blank: Data saturation for PD features was reached at the ________th paper. 68th.
True or false: The study found PD research is geographically balanced across all continents. False – 45.41% of studies came from the USA alone, indicating a strong Western-centric bias.
Q: What were the three primary functions of PD identified in HCI research, and which was most common?
A: (1) PD as a research method to explore design spaces (52.43%, most common), (2) PD as a design approach to develop new solutions (26.49%), and (3) PD as a research context to investigate PD's own philosophy and methodology (21.08%).
Q: Why did the researchers use the PRISMA protocol, and what were the main exclusion criteria?
A: PRISMA ensured transparency and reproducibility in the literature search. Papers were excluded if they did not self-claim conducting PD, were non-empirical (argument papers, reviews), were not full papers (posters, workshop papers), or were published at non-SIGCHI venues.
Q: What does the dominance of "design space exploration" over "developing new solutions" suggest about PD practice in HCI?
A: It suggests that HCI scholars tend to use PD more as an investigative tool for understanding user needs and generating guidelines than as a method for building and deploying tangible, functional technologies. This aligns with the criticism that contemporary PD often focuses on "here-and-now" issues without progressing to real-world impact.
Q: How did the researchers ensure their coding was reliable?
A: Two primary researchers coded iteratively with regular meetings to refine themes. A third independent coder performed deductive coding on a random sample of 10 papers, yielding a Cohen's Kappa of 0.83 ("Almost Perfect"), confirming inter-rater reliability.
This material connects to research methods courses, particularly qualitative content analysis, data saturation, and inter-rater reliability. It also links to systematic review methodology (PRISMA) and to broader debates in HCI about whether design research produces enough real-world impact. If your course covers design thinking or the double diamond model, PD's "design space exploration" function maps roughly onto the "discover" and "define" phases.
PRISMA protocol, content analysis, data saturation, Cohen's Kappa, inter-rater reliability, MAXQDA, SIGCHI conferences, CHI, CSCW, DIS, IDC, design space exploration, evolutionary prototyping, healthcare HCI, education HCI, PD application areas, PD functions in HCI, Western-centric bias, research method vs design method, civic challenges in HCI