Research Methods in

Human-Computer Interaction
Second Edition

Chapter 1:
Introduction to HCI
research

Lazar, Feng, and Hochheiser

Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
A Brief History of HCI
• 1980-The first book on HCI (Software Psychology)
• 1982-The first conference on HCI (which later became the
CHI conference
• Other similar work was going on in the late 1970s, often
under “office automation” or “human factors”
• The first computer mouse was publicly demoed in 1968
• Computers were becoming smaller and being used in
homes, schools, workplaces, and by non-technical people
Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
Types of HCI Research Contributions
• Empirical contributions
• Artifact contributions
• Methodological contributions
• Theoretical contributions
• Dataset contributions
• Survey contributions
• Opinion contributions
• Most published research is empirical and/or artifact (Wobbrock
and Kientz, 2016)

Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
Changes in topics of HCI research over time
• 1980s-office automation software, basic interaction
research, basic GUIs
• 1990s-advanced GUIs, UCD methods, Internet/Web,
computer-mediated communication
• 2000s- user-generated content, user diversity, mobile
computing, multimedia
• 2010s-collaboration, mobile/embedded computing,
crowdsourcing, emotional and persuasive computing,
natural user interfaces, sustainability, big data, accessibility

Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
Changes in HCI research methods over time

• New tools or tools where the costs have dropped

dramatically
• Eye-tracking, sensors, drones, facial EMG, EEG,
Mechanical Turk
• New approaches
• Social networking, big data, crowdsourcing, personal
health tracking, citation analysis, text parsing

Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
How HCI research differs from other fields
• Many disciplines collect national data sets using
established methodological controls
• National data on income, employment, family
• Statistics Canada, U.S. Census Bureau, EuroStat
• Large sets are publicly available which researchers can analyze
• Typically, HCI researchers must collect their own data
• Leads to smaller size data sets
• Or a combination of automated data collection (e.g. big data
and/or text parsing) and smaller, more in-depth studies
Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
How HCI research differs from other fields
• Other disciplines such as sociology and medicine often
track longitudinal data, over decades
• HCI researchers rarely collect longitudinal data
• Technological change may make it hard to compare devices
over time
• But other data points, such as time usage, communication
patterns, or psychological well-being, are appropriate for
longitudinal study
• The lack of longitudinal data limits the impact of HCI
research on other fields and populations
Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
How HCI research differs from other fields
• For a majority of HCI research, you must have
representative participants, not college students. ☺
• Representative in terms of:
• Age
• Educational experience
• Technical experience
• Domain knowledge/job experience
• If you are evaluating something simple like motor
performance, not usage patterns, then college
students may be appropriate
Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
Understanding differences in methods
and measurement
• HCI research requires both rigorous methods and
relevance
• There has been a historic focus on improving the
quality of life
• Depending on the disciplinary roots of your
collaborators, they may have different expectations for
which methods and metrics are most appropriate
• The earliest metrics were based on human factors and
psychology research (e.g. task and time performance)
Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
Understanding differences in methods
and measurement
• Difference between micro-HCI and macro-HCI
research (Shneiderman, 2011)
• Micro-HCI research
• Task performance, time performance, error rate, time to
learn, retention over time, user satisfaction
• Frequently can be studied in a lab setting
• Macro-HCI research
• Motivation, collaboration, social participation, trust,
empathy, and other societal-level impacts
• Often cannot be studied in a lab setting
Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
The nature of interdisciplinary research in
HCI
• Is HCI interdisciplinary? Multidisciplinary? It’s own
discipline?
• There are often challenges when HCI research gets
evaluated through a single-discipline lens
• Journals vs. conference proceedings
• Single-author publications vs. multi-author publications
• Openness and transparency of research vs. secrecy
• Does reflective research about a field count as research?
• Is grant money considered good or bad?
• Which stage of the research process must be perfect?
Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
Who is the audience for your research?
• Most HCI researchers, without realizing it, target their
research to other researchers
• Publish their research in outlets primarily read by other
researchers, not practitioners
• Do not take steps to inform other target audiences, or
other disciplines, about their research
• Citation analysis, and metrics such as h-index, measure
the impact on other researchers

Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
Targeting your research to developers
• The goal is to influence actual system and interface
design
• Working in corporate/industrial research labs
• Or a university partnering with a company and working on
tech transfer
• There may be issues about secrecy, disclosure, and
intellectual property ownership
• Types of controls may differ, so that research is focused on
the specific configurations that a company is interested in
• Must publish/present in practitioner-friendly venues
Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
Targeting your research to policymakers
• Policymakers need to have data from the HCI community to
inform their decisions related to laws, regulations, legal
cases, and treaties
• The HCI community does not yet have an established
outreach to public policy communities
• Individual HCI researchers are encouraged to establish long-
term relationships with policymakers
• Meet with policymakers face-to-face
• Be specific about relevant laws and number of people impacted
• Understand the relevant policy timelines and deadlines
• Provide appropriate summaries of research, don’t just send an
academic paper
Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
Understanding one research project in the
context of related research
• There is no perfection data collection method or
effort, all have flaws
• One data collection effort does not lead to a
definitive answer
• Multiple teams should examine the same research
questions over time, using different methods
• Replication is an important part of validating
research, although it is rare in HCI
Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
The “research lifecycle” for one project

• 1. Designing research
• 2. Running data collection
• 3. Reporting research (Hornbaek, 2011)
• This approach focuses on one specific research
project, with the assumption that there is a base of
existing research literature on the topic, with an
understanding of challenges and biases present

Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
The “research lifecycle” for a new area of
research
• If no previous research exists on a topic area, you should start
with an exploratory research approach
• Surveys, interviews, focus groups, and ethnography are
appropriate for exploratory research
• It’s hard to start structured research in a new research area,
because:
• You often don’t know what data needs to be collected
• You don’t know what structure to put in place, what controls you
need
• You often don’t know what biases might be present
• Start with observation, before doing intervention or
experimentation (Shneiderman, 2016)
Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
Controlled laboratory studies vs. “in
the wild” field studies
• Either type of research can come first, but both types of
research should be done if possible
• Findings may differ inside the lab and outside
• Field studies may be better for mobile device research
• Field studies help understand the context of users and their
environments, e.g. outside noise and distractions, as well as
network latencies
• Field studies may allow for more diverse users to participate
• Informed consent may also be harder in field settings
Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
Is there a clear consensus over time?
• Over a 35-year period of research, there is a
consensus that broad, shallow tree structures in
menu design are superior to narrow, deep structures
• Many other questions are still hotly debated, e.g.,
what is the optimal number of participants required
for usability testing?
• Note that user habits and preferences are fluid and
change over time, and technological change (and
change in technology infrastructure) may also have
an impact
Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License
Inherent Trade-Offs in HCI
• HCI is not about optimization
• Socio-technical systems rarely can be reduced to two or
three measurements
• Great HCI research helps us better understand all of the
factors present in socio-technical systems
• We seek “better” solutions rather than “optimal” ones
• There are usually multiple stakeholders with different goals
• Example: the trade-off between greatly improved interfaces
and consistent interfaces (e.g. why we still use QWERTY
keyboard layouts)
• Example: the environmental impacts of continuous
upgrades in hardware
Slides ©2017 Lazar, Feng, and Hochheiser, Creative Commons, Attribution-NonCommercial-ShareAlike 4.0 License