INF3720 - Human Computer Interaction II

Summary 2014

Table of Contents
Ref

Chap

Title

Page

1.

What is Interaction Design ?

2.

Understanding and Conceptualizing Interaction

3.

Social Interaction

15

4.

Emotional Interaction

17

5.

Data Gathering

24

6.

The Process of Interaction Design

34

7.

10

Establishing Requirements

41

8.

12

Introducing Evaluation

49

9.

14

Evaluation Studies: From Controlled to Natural Settings

55

10.

15

Evaluation: Inspections, Analytics, and Models

59

(Not examinable)

Interaction Design: Beyond Human-Computer Interaction

3rd Edition - Preece, J., Rogers, Y., & Sharp, H.

v1.00 April 2014

Ron Barnard

Chap 1 - What is Interaction Design ?

Chapter 1 - What is Interaction Design ?

Objectives

Explain the difference between good and poor interaction design;

Describe what interaction design is, and how it relates to human-computer interaction;

Explain the relationship between the user experience and usability;

Describe what and who is involved in the process of interaction design;

Outline the different forms of guidance used in interaction design;

Enable you to evaluate an interactive product and explain what is good and bad about it in terms of
the goals and principles of interaction design.

Content
1.1 Introduction
Some interactive products are a joy to use, others can be very frustrating. Why is there a difference ?
Many products that require users to interact with them, such as smartphones and social networking sites,
have been designed primarily with the user in mind. They are generally easy and enjoyable to use. Others,
such as switching from viewing a DVD to viewing TV, or setting the alarm on a digital clock, have not
necessarily been designed with the users in mind, but have been engineered primarily as systems to perform
set functions.
One of the main aims of interaction design is to reduce the negative aspects (eg: frustration, annoyance) of
the user experience while enhancing the positive ones (eg: enjoyment, engagement). It is about developing
interactive products that are easy, effective, and pleasurable to use - from a users perspective.
1.2 Good and Poor Design
Central concern of interaction design - develop interactive products that are USABLE o
o
o

Easy to learn;
Effective to use;
Provide an enjoyable user experience.

Examples of good and bad design i)

Voice Mail System

Bad design Infuriating;

confusing;
inefficient (carry out a number of steps for basic tasks);
Page 2 of 66

Chap 1 - What is Interaction Design ?

difficult to use, no proper feedback (if any messages have been left, or how many);
not obvious what to do (instructions partially from the system and partially written).

Good design Familiar physical objects that indicate visually how many messages have been left;
Aesthetically pleasing and enjoyable to use;
Only requires one step actions to perform core tasks;
Simple but elegant design;

ii) Remote Control Device

Bad design Dizzying array of small, multicoloured, and double-labeled buttons that seem arbitrarily
positioned;
Difficult to locate correct buttons, even for the simplest of tasks;
Can be frustrating for those who have to put on glasses to read the buttons.

Good design Buttons are large, clearly labelled, logically positioned;

Designed to fit into the palm of ther hand;
Colourful buttons that are distinctive, easy to identify.

1.2.1 What to Design

Who will use it ?

How will it be used ?
Where are they going to be used ?
What kinds of activities will it support ?

Key question - How do you optimize the users' interactions with a system, environment or product, so that
they support and extend the users' activities in effective, useful, and usable ways ?
Decide what choices to make by basing them on an understanding of the users

Take into account what people are good and bad at;
Consider what might help people with the way they currently do things;
Thinking through what might provide quality user experiences;
Listening to what people might want and getting them involved in the design;
Using tried and tested user-based techniques during the design process

1.3 What is Interaction Design ?

"Designing interactive products to support the way people communicate and interact in their everyday and
working lives". - Preece

Page 3 of 66

Chap 1 - What is Interaction Design ?

It is about creating user experiences that enhance and augment the way people work, communicate, and
interact.
"Designing spaces for human communication and interaction" - Winograd.
"The why as well as the how of our daily interactions using computers" - Thackara.
"The art of facilitating interactions between humans through products and services" - Saffer.

1.3.1 The Components of Interaction Design

Interaction Design comes from a multidisciplinary background, extends and enhances the way people work,
communicate and interact. It is fundamental to all disciplines, fields, and approaches that are concerned with
researching and designing computer-based systems for people.

1.3.2 Who is involved in Interaction Design ?

Interaction Design is mostly carried out by multidisciplinary teams, where the skill sets of engineers,
programmers, psychologists, anthropologists, sociologists, artists, toy makers, and others are drawn upon. It
is rarely the case that a design team would have all of these professionals working together. Who to include
would depend on a number of factors, including a companys' design philosophy, its size, purpose, and
product line.

1.3.3 Interaction Design Consultants

Interaction Design is now widespread in product development. The presence or absence of good interaction
design can make or break a company.
There are many interaction design consultancies, who design products, services, and environments for other
companies.

1.4 The User Experience

The user experience is central to interaction design - every product used by someone has a user experience.
It is about how people feel about a product and their pleasure and satisfaction when using it, looking at it,
holding it, and opening or closing it. It includes their overall impression of how good it is to use. An important
aspect is the quality of the experience someone has.
There are many aspects of the user experience that can be considered, and ways of taking them into
account when designing interactive products. Important - the usability, the functionality, the aesthetics, the
content, the look and feel, and the sensual and emotional appeal.

1.5 The Process of Interaction Design

The process of Interaction Design involves four basic activities 1.
2.

Establishing requirements;
Designing alternatives;
Page 4 of 66

Chap 1 - What is Interaction Design ?

3.
Prototyping;
4.
Evaluating.
These activities are repeated throughout the product life-cycle;

Evaluating what has been built is the heart of Interaction Design, focus is on ensuring that the
product is appropriate. Addressed through a user-centered approach to design;

Also important - understanding what people do, and how they act and interact with one another, with
information, and with various technologies.

Accessibility
Accessibility - refers to the degree to which an interactive product is accessible by as many people as
possible. A focus is on people with disabilities.
What does it mean to be disabled ? Definitions vary, but these are the main points

Mental or physical impairment;

The impairment has an adverse effect on their ability to carry out normal day-to-day activities;
The adverse effect is substantial and long term (lasted, or likely to last for more than 12 months);

Whether or not a person is considered to be disabled changes over time with age, or as recovery from an
accident progresses. In addition, the severity and impact of an impairment can vary over time. A person can
have more than one disability, including

Colour-blindness - The inability to distinguish between two colours affects approx. 1 in 10 men,
and 1 in 200 women. This has an impact on the use of colour for highlighting, or distinguishing
interface elements.

Dyslexia - Although usually associated with difficulties in reading and writing, there are many
different forms of dyslexia, some of which affect the way in which people comprehend the totality of
concepts. A relatively simple interaction design decision that can cause difficulties for people with
dyslexia is the contrast between foreground and background text or images.

Physical impairments - range from conditions such as tremors or shaking, weakness, pain,
reduced control of limbs, inability to sit upright, to short or missing limbs.

Accessibility is often considered as making sure there aren't any barriers to access for assistive
technologies, but without regard to usability, while usability usually targets everyone who uses a site or
product, without considering people who have disabilities. The challenge is to create a good user experience
for people with disabilities that is both accessible and usable. (Quesenbery).

1.6 Interaction Design and the User Experience

1.6.1 Usability Goals
Usability - refers to ensuring that interactive products are easy to learn, effective to use, and enjoyable
from the users' perspective. It involves optimizing the interactions people have with interactive products to
enable them to carry out their activities at work, school, and in their everyday lives.
Page 5 of 66

Chap 1 - What is Interaction Design ?

Usability Goals

Effectiveness - How good a product is at doing what it is supposed to do;

Efficiency The way a product supports users in carrying out their tasks. eg: Completing
common tasks with a single key press, instead of having to re-enter information;

Safety Protecting users from dangerous conditions / undesirable situations. eg: Prevent
users from making serious errors - by not putting Delete / Quit menu option next to Save option. Use
of confirmation dialogue boxes, Undo facilities.

Utility Extent to which the product provides the right kind of functionality so that users
can do what they need or want to do;

Learnability -

Memorability - How easy a product is to remember how to use, once learned.

How easy a system is to learn to use;

1.6.2 User Experience Goals

User Experience Goals: User experience is what the interaction with the system feels like to the users
(subjectively). User Experience Goals differ from the more objective usability goals in that they are
concerned with how users experience an interactive product from their own perspective, rather than
assessing how useful or productive a system is from its own perspective.
Desirable Aspects
Satisfying

Enjoyable

Engaging

Pleasurable

Exciting

Entertaining

Helpful

Motivating

Challenging

Enhancing sociability

Supporting creativity

Cognitively stimulating

Fun

Provocative

Surprising

Rewarding

Emotionally fulfilling
Undesirable Aspects
Boring

Frustrating

Making one feel guilty

Annoying

Childish

Unpleasant

Patronizing

Making one feel stupid

Cutesy

Gimmicky

Not all usability and user experience goals will be relevant to the design and evaluation of an interactive
product being developed. Recognizing and understanding the nature of the relationship between usability
and other user experience goals is central to interactive design. It enables designers to become aware of the
consequences of pursuing different combinations when designing products and highlight potential trade-offs
and conflicts.

Page 6 of 66

Chap 1 - What is Interaction Design ?

1.6.3 Design Principles
Design principles are used by interaction designers to aid their thinking when designing for the user
experience. These are generalized abstractions intended to orient designers towards thinking about different
aspects of their designs. A number of design principles have been promoted, which are are concerned with
how to determine what users should see and do when carrying out their tasks using an interactive product.
Norman's Design Principles

Visibility - The more visible functions are, the more likely users will know what to do next. The
relationship between the way controls have been positioned and what they do makes it easy for a
user to find the appropriate control for a task.

Feedback - involves sending back information about what action has been done and what has
been accomplished. Various kinds of feedback are available - audio, tactile, verbal, visual and
combinations of these.

Constraints - refers to determining ways of restricting the kinds of user interactions that can take
place at a given moment. Examples are - deactivating certain menu options, thereby restricting users
only to permissible options at that stage of the activity, or the physical design of a device which
allows only a particular cable / card / plug to be inserted into specific slots / sockets.

Consistency - refers to designing interfaces to have similar operations / use similar elements for
achieving similar tasks. One of the benefits of consistent interfaces is that they are easier to learn
and use.

Affordance - refers to an attribute of an object that allows people to know how to use it. To afford
means "to give a clue". Eg: a mouse button invites pushing, a door handle affords pulling. Interfaces
should make it obvious what can be done with them.

One of the problems of applying more than one of the design principles in interaction design is that trade-offs
can arise between them. Eg: The more you constrain an interface, the less visible it becomes. Consistency
can be a problematic - trying to design an interface to be consistent with something can make it inconsistent
with something else.
Usability Principles / Heuristics (heuristics are design principles used in practice - more prescriptive usability
principles that are used as a basis for evaluating a system / prototype)

Nielsen's 10 Usability Principles o

o
o
o
o
o
o
o
o
o

Visibility of System Status

Match between system and real world
User control and freedom
Consistency and standards
Help users recognize, diagnose and recover from errors
Error prevention
Recognition rather than recall
Flexibility and efficiency of use
Aesthetic and minimalist design
Help and documentation

Page 7 of 66

Chap 1 - What is Interaction Design ?

Summary
In this chapter we have looked at what interaction design is and its importance when developing apps,
products, services, and systems. To begin, a number of good and bad designs were presented to illustrate
how interaction design can make a difference. We described who and what is involved in interaction design,
and the core set of design processes that need to be followed. We explained in detail what usability and user
experience are and how they have been characterized, and how to operationalize them in order to assess
the quality of a user experience resulting from interacting with an interactive product. The increasing
emphasis on designing for the user experience and not just products that are usable was stressed. A number
of core design principles were also introduced that provide guidance for helping to inform the interaction
design process.

Key Points

Interaction design is concerned with designing interactive products to support the way people
communicate and interact in their everyday and working lives;

Interaction design is multidisciplinary, involving many inputs from wide-ranging disciplines and fields;

The notion of the user experience is central to interaction design;

Optimizing the interaction between users and interactive products requires taking into account a
number of independent factors, including context of use, types of activity, accessibility, cultural
differences, and user groups;

Identifying and specifying relevant usability and user experience goals can help lead to the design of
good interactive products;

Design principles, such as feedback and simplicity, are useful heuristics for analyzing and evaluating
aspects of an interactive product.
--ooOoo--

Page 8 of 66

Chap 2 - Understanding and Conceptualizing Interaction

Chapter 2 - Understanding and Conceptualizing Interaction

Objectives

Explain what is meant by the problem space;

Explain how to conceptualize interaction;

Describe what a conceptual model is and how to begin to formulate one;

Discuss the use of interface metaphors as part of a conceptual model;

Outline the core interaction types for informing the development of a conceptual model;

Introduce paradigms, theories, models, and frameworks informing interaction design.

Content
2.1 Introduction
Imagine you have been asked to design an application to enable people to share their photos, movies,
music, chats, documents, and so on in an efficient, safe, and enjoyable way. What would you do ? How
would you start ? Would you begin by sketching out how the interface might look, work out how the system
architecture should be structured, or simply start coding ? Or, would you start by asking users about their
current experiences of sharing files and look at existing tools, e.g. Dropbox, and, based on this, begin
thinking about why and how you were going to design the application ?
Interaction designers would begin by doing the latter. It is important to realize that having a clear
understanding of why and how you are going to design something, before writing any code, can save
enormous amounts of time, effort, and money later on in the design process. Ill thought-out ideas,
incompatible and unusable designs can be refined while it is relatively easy and painless to do so. Once
ideas are committed to code they become much harder to throw away. Such preliminary thinking through of
ideas about the user experience and what kinds of designs might be appropriate is, however, a skill that
needs to be learned. It is not something that can be done overnight by following a checklist, but requires
practice in learning to identify, understand, and examine the issues - just like learning to write an essay or to
program. In this chapter we describe the steps involved. In particular, we focus on what it takes to
understand and conceptualize interaction.
2.2 Understanding the Problem Space and Conceptualizing Design

Identifying the usability and user experience goals is a pre-requisite to undestanding the problem
space.

Another important consideration is to make explicit underlying assumptions and claims.

The problem of articulating the problem space is typically done as a team effort. Explicating peoples
assumptions and claims about why they think something might be a good idea (or not) enables the
Page 9 of 66

Chap 2 - Understanding and Conceptualizing Interaction

design team as a whole to view multiple perspectives on the problem space and, in so doing, reveal
conflicting and problematic ones.

Framework for making your implicit assumptions explicit

Are there problems with an existing product or user experience ? If so, what are they ?

Why do you think there are problems ?

How do you think your proposed design ideas might overcome these ?

If you have not identified any problems and instead are designing for a new user experience, how do
you think your proposed design ideas support, change, or extend current ways of doing things ?

Having a good understanding of the problem space greatly helps design teams to be able to conceptualize
the design space. Primarily this involves articulating the proposed system and the user experience. The
benefits of conceptualizing the design space early on are

Orientation - enabling the design team to ask specific kinds of questions abut how the conceptual
model will be understood by the targeted users.

Open-mindedness - preventing the design team from becoming narrowly focused early on.

Common ground - allowing the design team to establish a set of common terms that all can
understand and agree on, reducing the chance of misunderstandings and confusion arising later on.

2.3 Conceptual Models

A conceptual model provides a working strategy and a framework of general concepts and their
interrelations. The core components are

Metaphors and analogies that convey to people how to understand what a product is for, and how to
use it for an activity (eg: browsing, bookmarking);

The concepts that people are exposed to through the product, including the task-domain objects they
create and manipulate, their attributes, and the operations that can be performed on them (eg:
saving, revisiting, organizing);

The relationships between those concepts (eg: whether one object contains another, the relative
importance of actions to others, and whether an object is part of another);

The mappings between the concepts and the user experience the product is designed to support or
invoke (eg: one can revisit through looking at a list of visited sites, most frequently visited, or saved
websites).

How the various metaphors, concepts, and their relationships are organized determines the user experience.
The best conceptual models are those that appear obvious, the operations they support being intuitive to
use.

Page 10 of 66

Chap 2 - Understanding and Conceptualizing Interaction

Most interface applications are actually based on well-established conceptual models - eg: most on-line
shopping websites are based on the core aspects of the customer experience when at a shopping mall.
It is rare for completely new conceptual models to emerge that transform the way we carry out our everyday
and work activities when using a computer. The classics include - the desktop (Xerox Star interface), the
spreadsheet, and the web.

2.4 Interface Metaphors

Interface metaphors - are considered to be a central component of a conceptual model. They provide a
structure that is similar in some ways to aspects of a familiar entity/s, but that also have their own behaviours
and properties. It is instantiated in some way as part of the user interface eg: the desktop metaphor, the
search engine.
Interface metaphors are intended to provide familiar entities that enable people to readily understand the
underlying conceptual model and know what to do at an interface.

2.5 Interaction Types

Another way of conceptualizing the design space is in terms of the interaction types that will underlie the
user experience. These are the ways a person interacts with a product or application.
Note distinction between interaction types and interface types (Chap 6).
There are four main interaction types

Instructing - where users issue instructions to a system. This can be done in a number of ways typing in commands, selecting options from a menu, speaking commands aloud, gesturing, pressing
buttons, or using a combination of function keys.
One of the main benefits of designing an interaction based on issuing instructions is that the
interaction is quick and efficient.

Conversing - where users have a dialogue with a system. Users can speak via an interface or type
in questions to which the system replies via text or speech output. It differs from the activity of
instructing insofar as it encompasses a two-way communication process with the system acting like
a partner rather than a machine that obeys orders.
A main benefit of developing a conceptual model that uses a conversational style of interaction is
that it allows people, especially novices, to interact with a system in a way that is familiar to them.

Manipulating - where users interact with objects in a virtual or physical space by manipulating
them (eg: opening, holding, closing, placing). Users can use their familiar knowledge of how to
interact with objects.

Direct manipulation - digital objects designed at the interface so that they can be interacted with in
ways that are analogous to how physical objects in the physical world are manipulated. Enables
users to feel that they are directly controlling the digital objects represented by the computer.

Page 11 of 66

Chap 2 - Understanding and Conceptualizing Interaction

The benefits of direct manipulation include

helping beginners to learn basic functionality rapidly;

enabling experienced users to work rapidly on a wide range of tasks;
allowing infrequent users to remember how to carry out operations over time;
preventing the need for error messages, except very rarely;
showing users immediately how their actions are furthering their goals;
reducing users' experiences of anxiety;
helping users gain confidence and mastery and feel in control.

Exploring - where users move through a virtual environment or a physical space. Virtual
environments include 3D worlds, and augmented and virtual reality systems. They enable users to
use their familiar knowledge of physically moving around.

2.6 Paradigms, Theories, Models, and Frameworks

Other sources of inspiration and knowledge that are used to inform design and guide research are

A Paradigm - refers to a general approach that has been adopted by a community of researchers
and designers for carrying out their work, in terms of shared assumptions, concepts, values, and
practices;

A Theory - is a well-substantiated explanation of some aspects of a phenomenon, eg: the theory of

information processing that explains how the mind, or some aspect of it, is assumed to work;

A Model - is a simplification of some aspect of human-computer interaction intended to make it

easier for designers to predict and evaluate alternative designs;

A Framework - is a set of inter-related concepts and / or a set of specific questions that is intended
to inform a particular domain area, on-line communities, or an analytic method.

2.6.1 Paradigms
To follow a particular paradigm means adopting a set of practices that a community has agreed upon. These
include

the questions to be asked and how they should be framed;

the phenomena to be observed;
the way in which findings from studies are to be analyzed and interpreted.

In the 1980s the prevailing paradigm was how to design user-centered applications for the desktop
computer. The acronym WIMP (Windows, Icons, Menus, and Pointer) was used as a way of characterizing
the core features of an interface for a single user. This was later superseded by the GUI (graphical user
interface).
A big influence in the more recent paradigmatic changes was Weisers vision of ubiquitous technology. He
proposed that computers would become part of the environment, embedded in a variety of everyday objects,
devices, and displays.

Page 12 of 66

Chap 2 - Understanding and Conceptualizing Interaction

Since the late 1990s, many researchers have been concerned with how to embed and augment the
environment with various computational resources to provide information and services, when and where
desired.

2.6.2 Theories
Over the past 30 years, numerous theories have been imported into human-computer interaction, providing a
means of analyzing and predicting the performance of users carrying out tasks for specific kinds of computer
interfaces and systems.
One of the main benefits of applying such theories in interaction design is to help identify factors (cognitive,
social, and affective) relevant to the design and evaluation of interactive products.
2.6.3 Models
Models are typically abstracted from a theory coming from a contributing discipline, like psychology, that can
be directly applied to interaction design. For example, Norman developed a number of models of user
interaction based on theories of cognitive processing, arising out of cognitive science, that were intended to
explain the way users interacted with interactive technologies.
2.6.4 Frameworks
Numerous frameworks have been introduced in interactive design to help designers constrain and scope the
user experience for which they are designing. In contrast to a model - which is a simplification of a
phenomenon - a framework offers advice to designers as to what to design or look for. Frameworks, like
models, have traditionally been based on theories of human behaviour, but they are increasingly being
developed from the experiences of actual design practice and the findings arising from user studies.
Paradigms, theories, models, and frameworks are not mutually exclusive but overlap in their way of
conceptualizing the problem and design space, varying in their level of rigour, abstraction, and purpose.

Page 13 of 66

Chap 2 - Understanding and Conceptualizing Interaction

Summary
This chapter has explained the importance of understanding and conceptualizing the problem and design
space before trying to build anything. It has stressed throughout the need to be explicit about the claims and
assumptions behind design decisions that are suggested. It described an approach to formulating a
conceptual model and described the evolution of interface metaphors that have been designed as part of the
conceptual model. Finally, it considered other ways of conceptualizing interaction, in terms of interaction
types, paradigms, theories, models, and frameworks.

Key Points

It is important to have a good understanding of the problem space, specifying what it is you are
doing, why, and how it will support users in the way intended.

A fundamental aspect of interaction design is to develop a conceptual model.

A conceptual model is a high-level description of a product in terms of what users can do with it and
the concepts they need to understand how to interact with it.

Decisions about conceptual design should be made before commencing physical design (eg:
choosing menus, icons, dialogue boxes).

Interface metaphors are commonly used as part of a conceptual model.

Interaction types (eg: conversing, instructing) provide a way of thinking about how best to support
the activities users will be doing when using a product or service.

Paradigms, theories, models, and frameworks provide different ways of framing, and informing
design and research.
--ooOoo--

Page 14 of 66

Chap 4 - Social Interaction

Chapter 4 - Social Interaction

(Not examinable)

Objectives

Explain what is meant by social interaction;

Describe the social mechanisms that are used by people when communicating and collaborating;

Discuss how social media have changed the ways in which we keep in touch, make contact, and
manage our social and work lives;

Explain what is meant by telepresence;

Give an overview of shareable technologies and some of the studies showing how they can facilitate
collaboration and group participation;

Describe some of the new forms of social behaviour that have emerged as a result of using new social
media and communication technologies.

Content
4.1 Introduction
Imagine not having access to your cell phone or the Internet for a week. How would you cope? Would you get
bored, start twitching, or even go stir crazy ? Would you feel isolated and be constantly wondering what is
happening in your online social network ? Many people now cannot go for very long without checking the latest
postings, emails, etc. on their Black- Berries, iPhones, or computers - even when on vacation. It has become a
daily routine and an integral part of their social lives. This is not surprising given that humans are inherently
social: they live together, work together, learn together, play together, interact and talk with each other, and
socialize.
There are many kinds of sociality and many ways of studying it. In this chapter our focus is on how people
communicate and collaborate in their social, work, and everyday lives. We examine how the emergence of a
diversity of communication technologies has changed the way people live - the way they keep in touch, make
friends, and coordinate their social networks. We look at the conversation mechanisms that have conventionally
been used in face-to-face interactions and examine how these have changed for the various kinds of computerbased conversations that take place at a distance. We describe the idea of telepresence, where novel
technologies have been designed to allow a person to feel as if they are present or to give the appearance of
being present at another location. We also outline some technologies that have been developed to allow new
forms of interaction, focusing on how shareable technologies can facilitate and support collocated collaboration.
Finally, we describe some of the social phenomena that have emerged as a result of the use and appropriation
of social media.

Page 15 of 66

Chap 4 - Social Interaction

Summary
Key Points

Social interaction is central to our everyday life.

Social mechanisms have evolved in face-to-face and remote contexts to facilitate conversation,
coordination, and awareness.

Talk and the way it is managed are integral to coordinating social interaction.

Many kinds of computer mediated communication systems have been developed to enable people to
communicate with one another when in physically different locations.

Keeping aware of what others are doing and letting others know what you are doing are important
aspects of collaboration and socializing.

Social media have brought about significant changes in the way people keep in touch.
--ooOoo--

Page 16 of 66

Chap 5 - Emotional Interaction

Chapter 5 - Emotional Interaction

Objectives

Cover how emotions relate to the user experience;

Provide examples of interfaces that are both pleasurable and usable;

Explain what expressive interfaces are and the effects they can have on people;

Describe how technologies can be designed to change people's attitudes and behaviour;

Give an overview on how anthropomorphism has been applied in interaction design;

Present well-known models and frameworks of emotion and pleasure;

Enable you to critique the persuasive impact of an online agent on customers.

Content
5.1 Introduction
An overarching goal of interaction design is to develop products that elicit positive responses from users,
such as feeling at ease, being comfortable, and enjoying the experience of using them - be it a washing
machine or a flight deck. Designers are also concerned with how to create interactive products that elicit
specific kinds of emotional responses in users, such as motivating them to learn, play, or be creative or
social. There has also been much interest in designing websites that people can trust, and that make them
feel comfortable about divulging personal information when making a purchase.

Taken together, we refer to this emerging area as emotional interaction. In this chapter we look at how and
why the design of interactive products may cause certain kinds of emotional responses in people. We begin
by looking in general at expressive interfaces, examining the role of an interface's appearance to users and
how it affects usability.

We then examine how interactive products elicit positive effects, e.g. pleasure, and negative responses, e.g.
frustration. How technologies are being designed and used to persuade people to change their behaviour
and attitudes is then covered. We look, in particular, at ubiquitous technology interventions that are being
designed to improve health and well-being and reduce domestic energy and water consumption. Following
this, we show how anthropomorphism has been used in interaction design and the implications of designing
applications that have human-like qualities. A number of virtual characters and robot pets are described that
have been developed to motivate people to learn, buy, and listen and we consider how useful and
appropriate they are. Finally, we present three models that are well known in interaction design that
conceptualize the user experience in terms of emotion, pleasure, and user experience

(i) Norman's (2004) emotional design model;

(ii) Jordan's (2000) pleasure model for product design; and
(iii) McCarthy and Wright's (2004) technology as experience framework.

Page 17 of 66

Chap 5 - Emotional Interaction

5.2 Emotions and the User Experience
Emotional interaction is concerned with how we feel and react when interacting with technologies. It covers
different aspects of the user experience, from how we feel when first finding out about a new product to
getting rid of it. It also looks at why people become emotionally attached to certain products (eg: virtual pets),
how social robots might help reduce loneliness, and how to change human behaviour through the use of
emotive feedback.

Emotional interaction is about considering what makes us happy, sad, annoyed, anxious, frustrated,
motivated, delirious, and so on, and translating this knowledge into different aspects of the user experience,
from when we first want something, to when we no longer interact with it or need to replace it. However it is
not straightforward to achieve as peoples' moods and feelings are constantly changing.

In addition to creating user experiences that elicit, avoid, or encourage certain kinds of emotional reactions,
another approach, called affective computing, has attempted to develop computer-based systems that
recognize and express emotions in the same way humans do.

5.3 Expressive Interfaces

Expressive forms like emoticons, sounds, icons, and virtual agents have been used at the interface to convey emotional states; and / or
elicit certain kinds of emotional responses in users, such as feeling at ease, comfort, and happiness.
Icons and animations have been used to indicate the current state of a computer or a cell phone, notably
when it is waking up or being rebooted. The icon conveys a sense of friendliness, inviting the user to feel at
ease. The appearance of the icon on the screen is also very reassuring to users, indicating that their
computer is working correctly.

Other ways of conveying the status of a system are through the use of

Dynamic icons (eg: recycle bin expanding when a file is placed in it);
Animations (eg: whirling beach ball when the computer is busy);
Spoken messages, using various kinds of voices, telling the user what needs to be done (eg: GPS);
Various sonifications indicating actions and events (eg: for window closing, new e-mail);
Vibrotactile feedback (eg: Cell phone buzzes for certain types of calls).

One of the benefits of using these kinds of expressive embellishments is that they provide reassuring
feedback to the user that can be both informative and fun. They can sometimes have the opposite effect on
people who find them intrusive, causing them at times to get annoyed or even angry.
Users have also expressed their emotions at the computer interface - by using emoticons, combinations of
keyboard symbols. Nowadays people can select from sets of ready made emoticons which are colourful and
cute.

Page 18 of 66

Chap 5 - Emotional Interaction

The style of an interface, in terms of the shapes, fonts, colours, balance, white space, and graphical
elements that are used, and the way they are combined, can also influence its emotional impact. When the
look and feel of an interface is pleasing, users are likely to be more tolerant, eg: may be prepared to wait a
few more seconds for a website to download.
Good looking interfaces are often more satisfying and pleasurable to use. A key concern is therefore to strike
a balance between designing aesthetic and usable interfaces.

5.4 Frustrating Interfaces

In many situations, computer interfaces may inadvertently elicit negative emotional responses, such as anger
and disgust. This typically happens when something that should be simple to use, or set, turns out to be
complex. The most common examples are remote controls, printers, digital alarm clocks, and digital TV
systems.
Several methods have been devised to help the novice user get set up and become familiarized with a
technology. However, these have sometimes backfired, since the design solution itself has become a source
of annoyance and frustration. eg: Friendly agents which many users found to be too cute and childish.
Interfaces, if designed poorly, can make people look stupid, or feel insulted or threatened. The effect can be
to make them annoyed to the point of losing their temper. There are many reasons why such emotional
responses occur

When an application doesn't work properly or crashes;

When a system doesn't do what a user wants it to do;
When a users' expectations are not met;
When a system does not provide sufficient information to let the user know what to do;
When error messages pop up that are vague or unhelpful;
When the appearance of an interface is too noisy, garish, gimmicky, or patronizing;
When a system requires users to carry out too many steps to perform a task, only to discover a
mistake was made somewhere along the line and they need to start all over again.

Often user frustration is a result of bad design, no design, inadvertent design, or ill-thought-out design. It is
rarely caused deliberately. However, the impact of poor design on users can be quite drastic and make them
abandon the application or tool.
Gimmicks
Frustration can happen when clicking on a link to a website only to discover that it is still under construction.
It can be more annoying when the website displays an "Under Construction" icon, which merely increases
the users frustration, having made the effort to visit the website.

Error Messages
Error messages are notorious for their incomprehensibility. It would be more helpful to provide information
about how to resolve / overcome the problem. Threatening error messages can also cause users to become
anxious. Use of bold red letters and exclamation marks can be counter-productive.
Ideally, error messages should be treated as "how to fix it" messages. They should state the cause of the
problem, and what the user needs to do to fix it.
Page 19 of 66

Chap 5 - Emotional Interaction

Waiting
Websites or software apps that take a long time to download can be frustrating. Links that hang and
eventually do not load can also be very annoying.

Upgrading
Another common frustration is upgrading software. More often than not it is time-consuming, and requires a
range of things such as resetting preferences, checking configurations, and learning new ways of doing
things.

Appearance
People are often annoyed by

Websites that are overloaded by text and graphics, making it difficult to find and access information;

Flashing animations, pop ups, banner ads, that can cover what the user is looking for, and which
require them to click on check boxes to close them;

The over-use of sound effects and music;

Featuritis - an excessive number of buttons and options, such as on remote controls;

Childish designs that keep popping up, such as certain types of helper agents;

Poorly laid out keyboards, pads, control panels, and other input devices that cause users to press
the wrong keys or buttons.

5.5 Persuasive Technologies and Behavioural Change

Persuasive technology - interactive computing systems that are deliberately designed to change peoples'
attitudes and behaviours. eg: pop-up ads, warning messages, reminders, prompts, personalized messages &
recommendations.
Interactive techniques are being used on the web to entice, cajole, and persuade people to do something
they might not otherwise have done. eg: Amazons one-click mechanism that makes it easy and tempting to
buy something at their on-line store; recommender systems that suggest specific books, hotels, restaurants
etc.
Deceptive technology - being used to deceive people into parting with their personal details that would allow
fraudsters to access their bank accounts. "Phising" scams, e-mails supposedly from banks, PayPal etc
asking for personal information.

5.6 Anthropomorphism and Zoomorphism

Anthropomorphism - attributing human qualities to animals and objects.
Zoomorphism - shaping of an object or design in animal form.

Page 20 of 66

Chap 5 - Emotional Interaction

The finding that people, especially children, have a propensity to accept and enjoy objects that have been
given human-like qualities has led many designers to capitalize on it, most notably in the design of humancomputer dialogues modelled on how humans talk to each other.
Anthropomorphism has also been exploited in the development of computer-based cuddle toys that are
embedded with various sensors. eg: Interactive dolls have been designed to talk, sense, and understand the
world around them using sensor-based technologies and speech recognition.
Furnishing technologies with personalities and other human-like attributes makes them more enjoyable and
fun to interact with. They can also motivate people to carry out various activities, such as learning.
The anthropomorphic approach has generally had a positive response from users, and had a positive impact
on them particularly when the computer system is designed to flatter and praise the user.
Virtual agents and toys can also be patronizing and annoying in certain contexts. They also do not have the
range of emotional intelligence to respond in the ways humans do with each other.

5.7 Models of Emotion

A goal is to help designers understand how people react and respond when in different contexts, and to help
them know how to design for, or to try to reduce certain emotions.
i) Emotional Design Model (Norman)
Ortony et als' model of emotion and behaviour is couched in terms of different levels of the brain

Visceral level (Lowest level) - Parts of the brain are pre-wired to automatically respond to events
happening in the physical world. Responds rapidly, making judgements about what is good or bad,
safe or dangerous, pleasurable or abhorrent. It also triggers the emotional responses to stimuli (eg:
fear, joy, anger, and sadness) that are expressed through a combination of physiological and
behavioural responses.

Behavioural level - The brain processes that control our everyday behaviour, where most human
activities occur, eg: talking, typing, and driving.

Reflective level (Highest level) - Brain processes that contemplate - entails conscious thought
where people generalize across events, or step back from the routine and immediate.

The model explains how the human brain and body switch gear to respond appropriately to different events.
But how can it be used in interactive design ? Can products be designed to make people happy, and if so be
more creative ?
According to Norman, when people are happy they are more likely to overlook and cope with minor problems
they are experiencing with a device. In contrast, when someone is anxious or angry they are more likely to
be less tolerant. He states - "Things intended to be used under stressful situations require a lot more care,
and with much more attention to detail".

Page 21 of 66

Chap 5 - Emotional Interaction

ii) Pleasure Model (Jordan)
Jordans' model focuses on the pleasurable aspects of our interactions with products. It considers all of the
potential benefits that a product can deliver. It proposes four conceptually distinct types of pleasure

Physio-pleasure - refers to bodily pleasures connected to sensory experiences eg: touch, taste,
and smell. An example is the tactile pleasure of holding a sleek cell phone.

Socio-pleasure - refers to the enjoyment of being in the company of others, such as loved ones,
friends, and colleagues. An example is showing one another photos on a digital camera.

Psycho-pleasure - refers to peoples' emotional and cognitive reactions to a product. An example is

the emotionally satisfying experience of shopping on the web using an on-line site that is both
pleasing and easy to use. (Similar to Normans' behavioural level).

Ideo-pleasure (cognitive) - refers to peoples' values, entails the aesthetics of a product and the
cultural and personal values a person attributes to it. eg: Using a "green" product. (Similar to
Normans' reflective level).

The pleasure model does not attempt to explain how pleasures happen at a biological or behavioural level,
but is intended as a means of framing a designers' thinking about pleasure, highlighting that there are
different kinds.

iii) Technology as Experience Framework

Accounts for the user experience largely in terms of how it is felt by the user. Tries to capture the essence of
human experience by describing it in both holistic and metaphorical terms.
There are four core threads that make up our holistic experiences

Sensual thread - Concerned with our sensory engagement with a situation, eg: thrill, fear, pain similar to Normans' visceral level.

Emotional thread - Emotions are intertwined with with the situation in which they arise, eg: sorrow,
anger, joy, and happiness. Example: a person becomes angry with a computer because it does not
work properly.

Compositional thread - Concerned with the narrative part of an experience, as it unfolds, and the
way a person makes sense of them. The internal thinking a person does during their experiences.

Spatio-temporal thread - Refers to the space and time in which our experiences take place and
their effect on those experiences. Eg: we talk of time speeding up, standing still, slowing down, and
space as public / personal space, and needing ones' own space.

The threads are meant as ideas to help designers think and talk more clearly and concretely about the
relationship between technology and experience. By describing an experience in terms of its interconnected
aspects, the framework can aid thinking about the whole experience of a technology rather than as
fragmented aspects eg: its usability, its marketability, or utility.

Page 22 of 66

Chap 5 - Emotional Interaction

Summary
This chapter has described the different ways interactive products can be designed (both deliberately and
inadvertently) to make people respond in certain ways. The extent to which users will learn, buy a product
on-line, quit a bad habit, or chat with others depends on how convincing the interface is, how comfortable
they feel when using a product, or how much they can trust it. If the interactive product is frustrating to use,
annoying, or patronizing, users will easily become angry and despondent, and often stop using it. If, on the
other hand, the product is pleasurable, enjoyable to use, and makes people feel comfortable and at ease,
then they will continue to use it, make a purchase, return to the website, or continue to learn. This chapter
has described various interaction mechanisms that can be used to elicit positive emotional responses in
users and ways of avoiding negative ones.

Key Points

Emotional aspects of interaction design are concerned with how to facilitate certain states (eg:
pleasure) or avoid certain reactions (eg: frustration) in user experiences.

Well designed interfaces can elicit good feelings in people.

Aesthetically pleasing interfaces can be a pleasure to use.

Expressive interfaces can provide reassuring feedback to users as well as be informative and fun.

Badly designed interfaces can often make people frustrated, annoyed, or angry.

Technologies can be designed to persuade people to change their behaviours or attitudes.

Anthropomorphism is is the attribution of human qualities to objects.

Virtual agents and robot pets have been developed to make people feel motivated, reassured, and in
a good mood.

Models of emotion provide frameworks for thinking about how to conceptualize and take into account
emotional and pleasurable aspects of the user experience.
--ooOoo--

Page 23 of 66

Chap 7 - Data Gathering

Chapter 7 - Data Gathering

Objectives

Discuss how to plan and run a successful data gathering system;

Enable you to plan and run an interview;

Enable you to design a simple questionnaire;

Enable you to plan and execute an observation.

Content
7.1 Introduction
This chapter presents some techniques for data gathering which are commonly used in interaction design
activities. In particular, data gathering is a central part of establishing requirements, and of evaluation. Within
the requirements activity, the purpose of data gathering is to collect sufficient, accurate, and relevant data so
that a set of stable requirements can be produced; within evaluation, data gathering is needed in order to
capture users' reactions and performance with a system or prototype.
In this chapter we introduce three main techniques for gathering data: interviews, questionnaires, and
observation. In the next chapter we discuss how to analyze and interpret the data collected. Interviews
involve an interviewer asking one or more interviewees a set of questions which may be highly structured or
unstructured; interviews are usually synchronous and are often face-to-face, but they don't have to be.
Questionnaires are a series of questions designed to be answered asynchronously, i.e. without the presence
of the investigator; these may be on paper, or online. Observation may be direct or indirect. Direct
observation involves spending time with individuals observing activity as it happens. Indirect observation
involves making a record of the user's activity as it happens to be studied at a later date. All three techniques
may be used to collect qualitative or quantitative data.
Although this is a small set of basic techniques, they are flexible and can be combined and extended in many
ways. Indeed it is important not to focus on just one data gathering technique but to use them flexibly and in
combination so as to avoid biases which are inherent in any one approach. The way in which each technique
is used varies depending on the interaction design activity being undertaken. More detailed descriptions of
how they are used and additional techniques relevant only to specific activities of the lifecycle are given in
later chapters (Chapter 10 for requirements, and Chapters 12-15 for evaluation).

7.2 Five Key Issues

Data gathering sessions need to be planned and executed carefully. There are five key issues that require
attention for any data gathering session to be successful -

Page 24 of 66

Chap 7 - Data Gathering

7.2.1 Setting Goals
There are many different reasons for gathering data, and before beginning it is important to identify specific
goals for the study. The goals that are set will influence the nature of the data gathering sessions, the data
gathering techniques to be used, and also the analysis to be performed. Once the goals have been set, then
you can concentrate on what data to look for and what to do with it once it is gathered.
7.2.2 Identifying Participants
The goals you develop for your data gathering session will indicate the kind of people you want to gather
data from.
Population - Those people who fit the indicated profile.
In some cases, the people you need to gather data from may be clearly identifiable - maybe because there is
a small group of users and you have access to each one.
Sampling - More likely that you will need to choose the participants to include in your data gathering.
Saturation sampling - Situation where you have access to all members of your target population, quite
rare.
Assuming that you will be choosing to involve a proportion of your population in data gathering, you have two
options -

Probability Sampling - Most commonly used approaches are -

Simple random sampling - can be achieved by using a random number generator, or by

choosing every nth person in a list.

Stratified sampling - relies on being able to divide the population into groups (eg: classes in a
secondary school), and then applying random sampling.

Non-probability Sampling - Rely less on you choosing the participants and more on participants
being prepared to take part. Most common approaches are -

Convenience sampling - describes a situation where the sample includes those who were
available rather than those specifically selected.

Volunteer panels

The crucial difference between probability and non-probability methods is that in the former you can
apply statistical tests and generalize to the whole population, while in the latter such generalizations
are not robust.

7.2.3 Relationship with Participants

One significant aspect of any data gathering is the relationship between the person/s doing the gathering and
the person/s providing the data. Making sure this relationship is clear and professional will help to clarify the
nature of the study.
One way in which this can be achieved is to ask participants to sign an informed consent form. The informed
consent form is intended to protect the interests of both the data gatherer and the data provider.

Page 25 of 66

Chap 7 - Data Gathering

7.2.4 Triangulation
Triangulation - is a term used to refer to the investigation of a phenomenon from (at least) two different
perspectives.
Four types of triangulation have been identified

Triangulation of data - means that data is drawn from different sources at different times, in different
places, or from different people.

Investigator triangulation - means that different researchers (observers, interviewers etc) have been
used to collect and interpret the data.

Triangulation of theories - means the use of different theoretical frameworks through which to view
the data or findings.

Methodological triangulation - means to employ different data gathering techniques. This is the
most common form of triangulation.

Validation through triangulation is difficult to achieve. Different data gathering methods result in different
kinds of data which may or may not be compatible. Using different theoretical frameworks may or may not
result in complementary findings. Using more than one data gathering technique, and more than one data
analysis approach is good practice, but achieving true triangulation is rare.
7.2.5 Pilot Studies
Pilot study - is a small trial run of the main study. The aim is to make sure that the proposed method is
viable before embarking on the real study.
Data gathering participants can be very unpredictable, even when a lot of time and effort has been spent
carefully planning the data gathering session. Plans should be tested by doing a pilot study before launching
into the main study.
Eg: Printing and distributing 500 questionnaires, and then finding that two of the questions are confusing
wastes time, annoys participants, and results in wasted costs.
If it is difficult to find people to participate, colleagues or peers can be asked to comment. It is important that
nobody involved in a pilot study can be involved in the main study - they know more about the study and can
distort the results.
7.3 Data Recording
Capturing data is necessary so that the results of a data gathering session may be taken away and
analyzed. Some forms of data gathering such as questionnaires, diaries, interaction logging, and collecting
work artifacts are self-documenting and no further recording is necessary, but for other techniques there is a
choice of recording approaches.
Which data recording approaches are used will depend on the context, time available, and the sensitivity of
the situation; the choice of data recording approach will affect the level of detail collected, and how intrusive
the data gathering will be,
There are three common data recording approaches -

Page 26 of 66

Chap 7 - Data Gathering

7.3.1 Notes Plus Still Camera
Taking notes is the most flexible way of recording data.
Advantage - of handwritten notes include that pen and paper are much less intrusive than a keyboard, and
they are extremely flexible.
Disadvantage - can be difficult and tiring to write and listen or observe at the same time, easy to lose
concentration. Handwriting can be difficult to decipher, and speed is limited. Working with another person
can solve some of the problems.
If appropriate, still photographs can be used to supplement notes and hand-drawn sketches.
7.3.2 Audio Plus Still Camera
Audio recording can be a useful alternative to note taking and is less intrusive than video. It allows observers
to focus on activity and the interviewee, rather than the spoken word and trying to take notes as well as
listen.
Transcribing a lot of audio can be time-consuming, but it isn't always necessary to transcribe all of it - often
only sections are needed.
Audio recording can also be supplemented with still photographs.
7.3.3 Video
Video has the advantage of capturing both visual and audio data, but has some additional planning issues,
and can be intrusive.

Deciding whether to fix the cameras' position, or to use a roving recorder.

Deciding where to point the camera in order to capture what is required.

Understanding the impact of recoding on participants. It is often assumed that video recording may
have an impact on participants and their behaviour. Heath - suggests taking an empirical approach
and examining the data to see whether there is any evidence of behaviour orienting to the camera.

7.4 Interviews
Interviews can be thought of as a "conversation with a purpose". There are four main types of interview - the
most appropriate approach depends on the purpose of the interview, the questions to be addressed, and the
stage in the lifecycle.
7.4.1 Unstructured Interviews
Open-ended or unstructured interviews are at one end of a spectrum of how much control the interviewer
has over the interview process. They are exploratory and are more like conversations around a particular
topic.
It is always advisable to have a plan of the main topics to be be covered. Going into an interview without an
agenda should not be confused with being open to new information and ideas.
A benefit of unstructured interviews is that they generate rich data that is often interrelated and complex, ie:
data that gives a deep understanding of the topic. In addition, interviewees may mention issues that the
interviewer has not considered.
Page 27 of 66

Chap 7 - Data Gathering

A lot of unstructured data is generated and the interviews may not be consistent across participants.
Unstructured interviews can therefore be time-consuming to analyze.
7.4.2 Structured Interviews
In structured interviews the interviewer asks predetermined questions similar to those in a questionnaire, and
the same questions are used with each participant so the study is standardized.
The questions are typically closed questions, which means they require an answer from a predetermined set
of alternatives. Structured interviews are only really useful when the goals are clearly understood and
specific questions can be identified.
7.4.3 Semi-structured Interviews
Semi-structured interviews combine features of structured and unstructured interviews and use both closed
and open questions. The interviewer has a basic script for guidance, so that the same topics are covered
with each interviewee.
The interviewer starts with pre-planned questions and then probes the interviewee to say more until no new
relevant information is forthcoming. Semi-structured interviews are intended to be broadly replicable, so
probing and prompting should aim to help the interview along without introducing bias.
7.4.4 Focus Groups
Interviews are often conducted with one interviewer and one interviewee, but it is also common to interview
people in groups. A focus group is a form of group interview, normally three to ten people are involved, and
discussion is led by a trained facilitator. Participants are selected to provide a representative sample of the
target population.
The benefit of a focus group is that it allows diverse or sensitive issues to be raised that might otherwise be
missed. Focus groups aim to enable people to put forward their own opinions in a supportive environment.
A preset agenda is developed to guide the discussion, but there is sufficient flexibility for the facilitator to
follow unanticipated issues as they are raised. The facilitator guides and prompts discussion and skilfully
encourages quiet people to participate, and stops verbose ones from dominating the discussion. The
discussion is usually recorded for later analysis and participants may be invited to explain their comments
more fully at a later date.
7.4.5 Planning and Conducting an Interview
Planning an interview involves developing the set of questions or topics to be covered, collating any
documentation to give to the interviewee, checking that recording equipment works and know how to use it,
and organizing a suitable time and place.

Developing Interview Questions - Questions or an interview may be open or closed. Open

questions are best suited where the goal of the session is exploratory; closed questions can only be
used where the possible answers are known in advance. An unstructured interview will usually
consist entirely of open questions, while a structured interview will usually consist of closed
questions. A semi-structured interview may use a combination of both types.

Running the Interview - Before starting, make sure that the aims of the interview have been
communicated to and understood by the interviewees, and that they feel comfortable. During the
interview, it is better to listen more than to talk, to respond with sympathy but without bias, and to
enjoy the interview if possible.

Page 28 of 66

Chap 7 - Data Gathering

7.4.6 Other Forms of Interview
Telephone and on-line interviews are a good way of interviewing people with whom you cannot meet. You
cannot see the interviewees' body language or facial expressions, which may be an advantage for interviews
that involve sensitive issues, since interviewees may prefer to remain anonymous.
Video conferencing systems can be used if face-to-face meetings are desirable, but not possible.
7.4.7 Enriching the Interview Experience
Interviews often take place in a neutral environment, and the interview situation provides an artificial context.
In these circumstances it can be difficult for interviewees to give full answers to the questions posed. To help
combat this, interviews can be enriched by using props such as prototypes or work artifacts that the
interviewee or interviewer brings along. These props can be used to provide context for the interviewees.
7.5 Questionnaires
Questionnaires are a well established technique for collecting demographic data and users' opinions. They
are similar to interviews in that they can have closed or open questions.
Questionnaire questions and structured interview questions are similar, so how do you know when to use
which technique ? Essentially, the difference lies in the motivation of the respondent to answer the
questions. If you think that this motivation is high enough to complete a questionnaire without anyone else
present, then a questionnaire will be appropriate. On the other hand, if the respondents need some
persuasion to answer the questions, it would be better to use an interview format and ask the question faceto-face through a structured interview.
7.5.1 Questionnaire Structure
Checklist of general advice for designing a questionnaire

Think about the ordering of questions. The impact of a question can be influenced by question order;

Consider whether you need different versions of the questionnaire for different populations;

Provide clear instructions on how to complete the questionnaire;

A balance must be struck between using white space and the need to keep the questionnaire as
compact as possible. Long questionnaires cost more and deter participation and completion.

7.5.2 Questionnaire and Response Format

Some commonly used formats

Check boxes and ranges - Can have fixed options eg: Male / Female, or a range, eg: Ages 15-19,
20-24, 25-39 etc.

Rating Scales - Useful for getting people to make judgements about things eg: how easy, how
usable etc. Two commonly used scales -

Page 29 of 66

Chap 7 - Data Gathering

Likert scales - used for measuring opinions, attitudes, and beliefs - widely used for evaluating
user satisfaction with products. Check boxes with rating 1,2,3,4,5 etc or "agree, OK, disagree"
etc.

Semantic differential scales - explore a range of bipolar attitudes about a particular item. Each
pair of attitudes is represented as a pair of adjectives. The participant is asked to mark one of a
number of positions between the two extremes to indicate agreement with the poles eg: Clear - Confusing, Helpful - Unhelpful, Poor - Well designed.

7.5.3 Administering Questionnaires

Two important issues when using questionnaires

Reaching a representative sample of participants - For large surveys, potential respondents

need to be selected using a sampling technique. However, interaction designers commonly use
small numbers of participants, often fewer than 20 users.

Ensuring a reasonable response rate - 100% completion rates are often achieved with small
samples, but with larger or more remote populations, ensuring that surveys are returned is a wellknown problem. 40% return is generally acceptable for many surveys, but lower rates are common.

7.5.4 On-line Questionnaires

On-line questionnaires are good for reaching a large number of people quickly, and if they are geographically
dispersed. There are two main types

Email - main advantage is that you can target specific users.

Web based can be interactive, and can include check boxes, radio buttons, pull-down and pop-up menus,
help screens etc;

it can also provide immediate validation;

main problem is the difficulty of obtaining a random sample of respondents.

7.6 Observation
Observation is a useful data gathering technique at any stage during product development. Early in design,
observation helps designers understand the users' context, tasks, and goals. Observation conducted later in
the development, eg: in evaluation, may be used to investigate how well the developing prototype supports
these tasks and goals.
7.6.1 Direct Observation in the Field
It can be difficult for people to explain what they do, or even to describe accurately how they achieve a task.
It is very unlikely that an interaction designer will get a full and true story by using interviews or
questionnaires; observation in the field can help fill in details that are not elicited from other forms of
investigation. Observation in the field can be complicated and can result in a lot of data that is not relevant if
it is not planned and carried out carefully.

Page 30 of 66

Chap 7 - Data Gathering

Structuring Frameworks for Observation in the field - Events can be complex and rapidly
changing during an observation. There is a lot for observers to think about, so many experts have a
framework to structure and focus their observations - a series of specific questions broken down into
various topics.

Degree of participation - The degree of participation within the study environment varies across a
spectrum, depending on the type of study, and can be characterized as insider at one end and
outsider at the other.
An observer who adopts an approach right at the outsider end of the spectrum is called a passive
observer, and will not take any part in the study environment. An observer who adopts an approach
at the insider end of this spectrum is called a participant observer, and attempts to become a full
member of the group he is studying.

Planning and Conducting an Observation in the Field - Choosing a framework is only one
aspect of planning an observation. Other decisions include

level of participation to adopt;

how to make a record of the data;
how to gain acceptance in the group to be studied;
how to handle sensitive issues such as cultural differences or access to private spaces; and
how to ensure that the study uses different perspectives (people, activities, job roles etc).

It is important to write up experiences and observations at the end of each day, otherwise valuable
information may be lost.

Ethnography - is a research method where the researcher observes from the point of view of the
subject of the study (a participant observer), and observation is done without imposing any structure
or framework - everything is viewed as "strange". Ethonography allows designers to obtain a detailed
understanding of peoples' behaviour and the use of technology that cannot be obtained by other
methods of data gathering.

7.6.2 Direct Observation in Controlled Environments

Observation in a controlled environment takes on a more formal character than observation in the field, and
the user is likely to feel apprehensive. The same basic data recording techniques are used for direct
observation in the laboratory and field studies, but the way in which these techniques are used is different. In
the laboratory the emphasis is on the details of what individuals do, while in the field the context is important
and the focus is on how people interact with each other, the technology, and their environment.

The Think-Aloud Technique - One of the problems with observation is that the observer doesn't
know what users are thinking, and can only guess from what they see. Observation in the field
should not be intrusive as this will disturb the very context you are trying to capture, so asking
questions of the participant should be limited. However, in a controlled environment, the observer
can afford to be a little more intrusive. The think-aloud technique is a useful way of understanding
what is going on in a persons' head.
The technique requires people to say out loud everything they are thinking and trying to do, so that
their thoughts are externalized.

Page 31 of 66

Chap 7 - Data Gathering

7.6.3 Indirect Observation: Tracking Users' Activities
Sometimes direct observation is not possible because it is obtrusive or observers cannot be present over the
duration of the study, and so activities are tracked indirectly. Diaries and interaction logs are two techniques
for doing this

Diaries - Participants are asked to write a diary of their activities on a regular basis, eg: what they
did, when they did it, what they found hard or easy, and what their reactions were to the situation.
Diaries are useful when participants are scattered and unreachable in person. Diaries have several
advantages they do not take up much resource,
require no special equipment or expertise, and
are suitable for long-term studies.

Interaction Logs and Web Analytics - Interaction logging involves instrumenting the software to
record users' activity in a log that can be examined later.
A key advantage of logging activity is that it is unobtrusive, provided system performance is not
affected, but it also raises ethical concerns about observing participants without their knowledge.
Another advantage is that large volumes of data can be logged automatically.
Web analytics is a form of interaction logging that has become popular. It involves collecting,
analyzing, and reporting data that tracks a users' behaviour when interacting with a website. Web
analytics can be used to assess whether users' goals are being met, to support usability studies, and
to inform future design.

7.7 Choosing and Combining Techniques

It is usual to combine data gathering techniques in any one data gathering program. Although it can be timeconsuming and costly, the benefit is that it provides multiple perspectives. Choosing which data gathering
techniques to use depends on a variety of factors, there is no right technique or combination of techniques,
but the decision should take all of the following factors into account

The Focus of the Study - The techniques used must be compatible with the goal of the study, ie:
they must be able to gather appropriate data.

The Participants Involved - The characteristics of the target user group for the product will affect
the kind of data gathering technique used. For example

techniques used for data gathering from young children may be very different from those used
with adults;
if participants are in a hurry to catch a plane, they will not be receptive to a long interview;
if their ob involves interacting with people, then they may be comfortable in a focus group.

The location and accessibility of participants also needs to be considered. It may be attractive to run
a focus group for a large set of stakeholders, but if they are spread across a wide geographical area,
a face-to-face meeting is unlikely to be practical.

The Nature of the Technique - Need to consider if the technique requires specialist equipment or
training, and whether available investigators have the appropriate knowledge and experience.

Available resources - The availability of resources, including time, will influence the choice of data
gathering techniques.

Page 32 of 66

Chap 7 - Data Gathering

Summary
This chapter has presented three main data gathering methods that are commonly used in interaction
design: interviews, questionnaires, and observation. It has described in detail the planning and execution of
each. In addition, five key issues of data gathering were presented, and how to record the data gathered was
discussed.

Key Points

All data gathering sessions should have clear goals;

Each planned data gathering session should be tested by running a pilot study;

Triangulation involves investigating a phenomenon from different perspectives;

Data may be recorded using handwritten notes, audio or video recording, a camera, or any
combination of these;

There are three styles of interviews: structured, semi-structured, and unstructured;

Questionnaires may be paper-based, email, or web-based;

Questions for an interview or questionnaire can be open or closed. Closed questions require the
interviewee to select from a limited range of options. Open questions accept a free-range response;

Observation may be direct or indirect;

In direct observation, the observer may adopt different levels of participation ranging from insider
(participant observer) to outsider (passive observer);

Choosing appropriate data gathering techniques depends on the focus of the study, the participants
involved, the nature of the techniques, and the resources available.
--ooOoo--

Page 33 of 66

Chap 9 - The Process of Interaction Design

Chapter 9 - The Process of Interaction Design

Objectives

Consider what doing interaction design involves;

Explain some advantages of involving users in development;

Explain the main principles of a user-centered approach;

Present a simple lifecycle model of interaction design;

Ask and provide answers for some important questions about the interaction design process;

Consider how interaction design activities can be integrated into the wider product development
lifecycle.

Content
9.1 Introduction
Design is a practical and creative activity with the aim of developing a product that helps its users achieve
their goals. In previous chapters, we looked at different kinds of interactive products, issues that need to be
taken into account when doing interaction design, some of the theoretical basis for the field, and techniques
for gathering and analyzing data to understand users' goals. In this chapter we start to explore how we can
design and build interactive products.
Chapter 1 defined interaction design as being concerned with 'designing interactive products to support the
way people communicate and interact in their everyday and working lives.' But how do you go about doing
this ? Developing a product must begin with gaining some understanding of what is required of it, but where
do these requirements come from ? Whom do you ask about them ? Underlying good interaction design is
the philosophy of user-centered design, i.e. involving users throughout development, but who are the users ?
Will they know what they want or need even if we can find them to ask ? For an innovative product, users
are unlikely to be able to envision what is possible, so where do these ideas come from ?
In this chapter, we raise and answer these kinds of questions, discuss user-centered design, and revisit the
four basic activities of the interaction design process that were introduced in Chapter 1. We also introduce a
lifecycle model of interaction design that captures these activities.
9.2 What is Involved in Interaction Design ?
9.2.1 The Importance of Involving Users
The best way to ensure that development takes users' activities into account is to involve real users
throughout development. In this way, developers can gain a better understanding of users' goals, leading to a
more appropriate, more usable product.
However, two other aspects that have nothing to do with functionality are equally important if the product is to
be usable and used Page 34 of 66

Chap 9 - The Process of Interaction Design

Expectation management - is the process of making sure that the users' expectations of the new
product are realistic. The purpose of expectation management is to ensure that there are no
surprises for users when the product arrives. If users feel that they have been cheated by promises
that have not been fulfilled, then this will cause resistance and even rejection.
Involving users throughout development helps with expectation management because they can see
from an early stage what the products' capabilities are. They will also understand better how it will
affect their jobs and lives, and why the features are designed that way.

Ownership - Users who are involved and feel that they have contributed to a products'
development are more likely to feel a sense of ownership towards it, and support its use.

9.2.2 Degrees of User Involvement

Different degrees of user involvement may be implemented in order to manage expectations and to create a
feeling of ownership.
Users may be co-opted to the design team so that they are major contributors. For any one user this may be
on a full-time basis, or a part-time basis, and it may be for the duration of the project or a limited time only.
There are advantages and disadvantages - if a user is co-opted full-time for the entire project, their input will
be consistent and they will become very familiar with the product. However, if the project takes many years
they may lose touch with the rest of the user group, making their input less valuable.
If a user is co-opted part-time for the whole project, they will offer consistent input to development while
remaining in touch with other users. However, this may be stressful as the user needs to learn to handle
unfamiliar material while still fulfilling the demands of their original job.
At the other end of the spectrum, users may simply be kept informed through regular newsletters or other
channels of communication. Provided they are able to influence the development process through
workshops or similar events, this can be an effective approach to expectation management and ownership.
In situations with large numbers of users, this may be the only viable option.
Once a product has been released, and the focus of development moves to future versions, a different kind
of user involvement is possible - one that captures data about the real use of the product. This may be
obtained through interaction between users and customer service agents, or through automated error
reporting systems.
How actively users should be involved is a matter for debate. Some studies have shown that too much user
involvement can lead to problems - users are unpredictable, users become more sophisticated as project
progresses and want late changes to project, can generate unnecessary conflicts and increased reworking.

9.2.3 What is a User-Centered Approach ?

The real users and their goals, not just technology, are the driving force behind product development. A welldesigned system will make the most of human skill and judgement, will be directly relevant to the activity in
hand, and will support rather than constrain the user.

Three principles are the basis for a user-centered approach -

Page 35 of 66

Chap 9 - The Process of Interaction Design

Early Focus on Users and Tasks - This means first understanding who the users will be by
studying their cognitive, behavioural, anthropomorphic, and attitudinal characteristics. Requires
observing users doing their normal tasks, studying the nature of those tasks, and then involving
users in the design process.
Expanding on the above -

Users' tasks and goals are the driving force behind the development, not technology.

Users' behaviour and context of use are studied and the system is designed to support them.
How people perform their tasks is also significant, not just the actual tasks that they perform.

Users' characteristics are captured and designed for. Humans are prone to making errors, and
have certain limitations, both cognitive and physical. Products designed to support humans
should take these limitations into account.

Users are consulted throughout development from earliest phases to the latest, and their input is
seriously taken into account. It is important that users are respected by designers.

All design decisions are taken within the context of the users, their work, and their environment.
This does not necessarily mean that users are actively involved in design decisions, but
designers must remain aware of the users when making decisions.

Empirical Measurement - The reactions and performance of intended users to printed scenarios,
manuals etc. is observed and measured. Later on, users interactions with simulations and prototypes
and their performance and reactions are observed, recorded and analyzed.
Specific usability and user experience goals should be identified, documented, and agreed upon at
the beginning of the project. Identifying specific goals up front means that the product can be
empirically evaluated at regular stages as it is developed.

Iterative Design - Design and development is iterative, with cycles of design / test / measure/
redesign being repeated. Iteration allows designs to be refined based on feedback. Iteration is
inevitable because designers are unlikely to get the entire solution correct the first time.

9.2.4 Four Basic Activities of Interaction Design

Brief introduction to the four basic activities of interaction design, to be expanded in later chapters

Establishing Requirements - In order to design something to support people, we must know who
our target users are, and what kind of support an interactive product could provide. This activity is
fundamental to a user-centered approach, and is very important in interaction design. Achieved
through data gathering and analysis.

Designing Alternatives - is the core activity of designing - actually producing ideas for meeting the
requirements. Can be broken up into two sub-activities: i) Conceptual design, and ii) Physical design.

Prototyping - The most sensible way for users to evaluate designs is to interact with them.
Achieved through prototyping - different techniques: i) Paper-based prototypes, quick and cheap to
build, and ii) Role-playing - users can get a real sense of what it is like to interact with a product.

Page 36 of 66

Chap 9 - The Process of Interaction Design

Evaluating - is the process of determining the usability and acceptability of a product or design.
Measured in terms of a variety of criteria, including - the number of errors users make using it; how
appealing it is; and how well it matches the requirements. Evaluation does not replace testing and
quality assurance, but complements and enhances them.

9.2.5 A Simple Lifecycle Model for Interaction Design

Understanding what activities are involved in interaction design is the first step to being able to do it, but it is
also important to consider how the activities are related to one another, so that the full development process
can be seen. The term lifecycle model is used to represent a model that captures a set of activities and how
they are related.

A simple interaction design lifecycle model.

9.3 Some Practical Issues
9.3.1 Who Are the Users ?
Users - Those who interact directly with a product to achieve a task.
There are others who can also be thought of as users - those who manage direct users, those who receive
products from the system, those who test the system, those who make purchasing decisions, and those who
use competitive products.
There are a surprisingly wide collection of people who all have a stake in the development of a successful
product. These people are called stakeholders.
Stakeholders - are people or organizations who will be affected by the system and who have a direct or
indirect influence on the system requirements. The group of stakeholders for a particular product will be
larger than the group of people normally thought of as users, although it will of course include users. The
group of stakeholders includes the development team itself, as well as its managers, the direct users and
their managers, recipients of the products' output, people who may lose their jobs because of the the
introduction of the new product, and so on.
Identifying the stakeholders for your project means that you can make an informed decision about who
should be involved and to what degree.

9.3.2 What Do We Mean by 'Needs' ?

People do not know what is possible. Therefore, users cannot tell us what they "need" to do achieve their
Page 37 of 66

Chap 9 - The Process of Interaction Design

goals. Approach it by understanding the characteristics and capabilities of the users, what they are trying to
achieve, how they achieve it currently, and whether they would achieve their goals more effectively and have
a more enjoyable experience if they were supported differently.
It may be tempting for designers to simply design what they would like to use themselves, but their ideas
would not necessarily coincide with those of the target user group. It is imperative that representative users
from the real target group be consulted.
Whether the product is a new invention or not, it is always useful to start by understanding similar behaviour
that is already established. eg: Standard telephones were available before cell phones, and so
understanding the tasks performed with them was a good place to start.
Focusing on peoples' goals and on usability and user experience goals is a more promising approach to
interaction design than focusing on peoples' needs and expecting them to be able to tell us the requirements
for a product.

9.3.3 How Do You generate Alternative Designs ?

Humans tend to stick to what they know works - a better solution may exist, but it is easier to accept the
current one because we know it works - it is 'good enough'.
Innovations arise through cross-fertilization of ideas from different perspectives, individuals, and applications;
the evolution of an existing product through use and observation; or straight-forward copying of other, similar
products.
Often browsing a collection of designs will inspire designers to consider alternative perspectives and
solutions. Alternatives come from seeking different perspectives and looking at other designs. Deliberately
seeking out suitable sources of inspiration is a valuable step in any design process.

9.3.4 How do you choose among alternative designs ?

Choosing among alternatives is about making design decisions. Broadly speaking the decisions fall into two
categories: those that are externally visible and measurable features, and those that are about internal
characteristics that cannot be observed or measured without dissecting it.
The way in which users interact with the product is considered the driving force behind the design, and so we
concentrate on the externally visible and measurable behaviour.
One way to choose between alternative designs is by letting users and stakeholders interact with them, and
by discussing their experiences, preferences, and suggestions for improvement. This means that the designs
must be available in a form that can reasonably be evaluated with users. This is fundamental to a usercentered approach.
Documentation eg: description of how something will work or a diagram of its components, has traditionally
been for communicating a design - very limited. Prototypes can be used to overcome potential client
misunderstandings and to test the technical feasibility of a suggested design and its production.
Quality is another basis on which to choose between alternatives, but it requires a clear understanding of
what quality is.
Usability engineering - The process of writing down formal, verifiable - and hence measurable - usability
criteria, and then assessing the product against them.

Page 38 of 66

Chap 9 - The Process of Interaction Design

Some suggestions for measurable characteristics

Effectiveness: Appropriate support ?

Efficiency: Response time ?
Safety: How safe ? How often does it crash / loose data ?
Utility: Which functions are superfluous, or not used often ?
Learnability: How long does a novice take to learn to do tasks ?
Memorability: How long does it take to remember how to perform common tasks ?

9.3.5 How Do You Integrate Interaction Design Activities with Other Lifecycle Models ?
Attempts at integrating user-centered design and software engineering have centered on agile software
development. There are various methods, but they all stress the importance of iteration, early and repeated
user feedback, being able to handle emergent requirements, and striking a good balance between flexibility
and structure. They also all emphasize collaboration, face-to-face communication, streamlined processes to
avoid unnecessary activities, and the importance of practice over process.
One of the main proponents for integrating user-centered design and agile development in practice, Jeff
Patton, has articulated 13 patterns of common behaviours for successful agile product development,
including the importance of designers being part of the team, using parallel tracks with user research, doing
just enough user research, modelling and design upfront, buying design time when developers are working
on complex engineering stories, and cultivating a user group for continuous user validation.

Page 39 of 66

Chap 9 - The Process of Interaction Design

Summary
In this chapter we have looked at user-centered design and the process of interaction design, ie: what is
user-centered design, what activities are required in order to design an interactive product, and how these
activities are related. A simple interaction design lifecycle model consisting of four activities was introduced
and issues surrounding the involvement and identification of users, generating alternative designs,
evaluating designs, and integrating user-centered concerns with other lifecycles were discussed.

Key Points

The interaction design process consists of four basic activities establishing requirements;
designing alternatives that meet those requirements;
prototyping the designs so that they can be communicated and assessed; and
evaluating them.

User centered design rests on three principles early focus on users and tasks;
empirical measurements; and
iterative design.
These principles are also key for interaction design.

Involving users in the design process helps with expectation management and feelings of ownership,
but how and when to involve users is a matter of dispute.

Before you can begin to establish requirements, you must understand who the users are and what
their goals are in using the product.

Looking at others' designs and involving other people in design provides useful inspiration and
encourages designers to consider alternative design solutions, which is key to effective design.

Usability criteria, technical feasibility, and users' feedback on prototypes can all be used to choose
among alternatives.

Prototyping is a useful technique for facilitating user feedback on designs at all stages.

Integrating interaction design activities with other lifecycle models requires careful planning.

--ooOoo--

Page 40 of 66

Chap 10 - Establishing Requirements

Chapter 10 - Establishing Requirements

Objectives

Describe different kinds of requirements;

Enable you to identify different kinds of requirements from a simple description;

Explain how different data gathering techniques (those introduced in Chapter 7 and others) may be
used during the requirements activity;

Enable you to develop a scenario, a use case, and an essential use case from a simple description;

Enable you to perform hierarchical task analysis on a simple description.

Content
10.1 Introduction
An interaction design project may aim to replace or update an established system, or it may aim to develop a
totally innovative product with no obvious precedent. There may be an initial set of requirements, or the
project may have to begin by producing a set of requirements from scratch. Whatever the initial situation and
whatever the aim of the project, the users' needs, requirements, aspirations, and expectations have to be
discussed, refined, clarified, and probably re-scoped. This requires an understanding of, among other things,
the users and their capabilities, their current tasks and goals, the conditions under which the product will be
used, and constraints on the product's performance.
Establishing requirements is also not simply writing a wish list of features. Given the iterative nature of
interaction design, isolating requirements activities from design activities and from evaluation activities is a
little artificial, since in practice they are all intertwined: some design will take place while requirements are
being established, and the design will evolve through a series of evaluation-redesign cycles. However, each
of these activities can be distinguished by its own emphasis and its own techniques.
This chapter provides a more detailed overview of establishing requirements. We introduce different kinds of
requirements and explain some useful techniques.

10.2 What, How, and Why ?

10.2.1 What Are We Trying to Achieve in the Requirements Activity ?
There are two aims Understand as much as possible about users, their activities, and the context of that activity, so that
the system under under development can support them in achieving their goals;

Produce a set of stable requirements that form a sound basis to start designing.
Page 41 of 66

Chap 10 - Establishing Requirements

10.2.2 How Can We Achieve This ?
Broadly speaking, these activities progress in a sequential manner

Data gathering activities;

Data analysis activities;
Then extract some 'requirements' from the data.
All of this is iterative - the sub-activities inform and refine one another. In practice, requirements
evolve and develop as the stakeholders interact with designs and see what is possible.

10.2.3 Why Bother ? The importance of Getting it Right

The cost of fixing errors late in the software development cycle, rather than early during the requirements
activity, is significant. Insufficient user communication, poor specifications, and insufficient analysis are major
contributors to poor cost estimation. Finding and fixing a software problem after delivery is often 100 times
more expensive than finding and fixing it during the requirements and design phase.

10.2.4 Why 'Establish' Requirements ?

The activity of understanding what a product should do has been given various labels. The term
requirements analysis is normally used to describe the activity of investigating and analyzing an initial set of
requirements that have been gathered, elicited, or captured. Analyzing the information gathered is an
important step, since it is this interpretation of the facts, rather than the facts themselves, that inspires the
design.
Requirements engineering is a better term than the others because it recognizes that developing a set of
requirements is an iterative process of evolution and negotiation and one that needs to be carefully managed
and controlled.
We use the term establishing requirements to represent the fact that requirements have been established
from a sound understanding of the users' needs, and that they can be justified by and related back to the
data collected.

10.3 What Are Requirements ?

10.3.1 Different Kinds of Requirements
Requirement - is a statement about an intended product that specifies what it should do, or how it should
perform.
One of the aims of the requirements activity is to make the requirements as specific, unambiguous, and clear
as possible.

In software engineering, two different kinds of requirements have traditionally been identified - functional
requirements, which say what a system should do; and non-functional requirements, which say what
constraints there are on the system and its development.
Interaction design involves understanding the functionality required and the constraints under which the
product must operate or be developed, hence we we are concerned with a wide range of requirements Page 42 of 66

Chap 10 - Establishing Requirements

Functional requirements - what the product should do. Understanding the functional requirements
for an interactive product is fundamental.

Data requirements - the type, volatility, size / amount, persistence, accuracy, and value of the
required data. All interactive products have to handle some data.

Environmental requirements (or context of use) - refer to the circumstances in which the
interactive product will operate. Four aspects of the environment must be considered

Physical environment - how much lighting, noise, movement, and dust is expected in the
operational environment ?

Social environment - collaboration, coordination. Will data need to be shared ?

Organizational environment - availability of user support, communication infrastructure, training

facilities or resources. How hierarchical is the management structure ?

Technical environment - what technologies will the product run on, or need to be compatible
with, and what technological limitations might be relevant ?

User characteristics - capture the key attributes of the intended user group. Attributes that may
affect the design are users' abilities and skills;
users' nationality;
educational background;
mental or physical disabilities;
novice, expert, casual, frequent user.

Usability goals and user experience goals - these are another kind of requirement, and should
be captured together with appropriate measures.

10.4 Data Gathering Requirements

The overall purpose of data gathering in the requirements activity is to collect sufficient, relevant, and
appropriate data so that a set of stable requirements can be produced. Even if a set of initial requirements
exists, data gathering will be required to expand, clarify, and confirm those initial requirements.

Interviews
good at getting people to explore issues;
can be structured, unstructured, or semi-structured;
often used early to elicit scenarios;
important for development team members to meet stakeholders, and for users to feel involved.

Focus Groups / Workshops

good at gaining a consensus view and highlighting areas of conflict and disagreement;
helps for stakeholders to meet designers and each other, and to express their views in public;
Workshops have grown in popularity - carefully planned, attendees carefully chosen, and
specific deliverables are produced.

Page 43 of 66

Chap 10 - Establishing Requirements

Questionnaires
used for getting initial responses that can then be analyzed to choose people to interview, or to
get a wider perspective on particular issues that have arisen elsewhere;

Direct Observation
observation of participants in their natural setting is used to understand the nature of the tasks
and the context in which they are performed;
sometimes carried out by trained observers who report back to the design team;
sometimes carried out by, or with a member of the design team;
ethnography is one form of this.

Indirect Observation
Diaries and interaction logging are used less often where a new product is under development;
if a product is evolving, such indirect observation is very valuable;
interaction logging together with web analytics are particularly useful for improving websites.

Studying Documentation
Manuals and other documentation are a good source of data about the steps involved in an
activity, and any regulations governing a task;
Documentation should not be used as the only source;
good for understanding legislation and getting background information on the work;
no stakeholder time, which is a limiting factor on the other techniques.

Researching Similar Products

look at similar products in order to generate alternative designs;
helps to prompt requirements.

The choice of data gathering techniques for the requirements activity is influenced by several factors
including the nature of the task;
the participants;
the analyst; and
the resources available.
It is usual to for more than one data gathering technique to be used in order to provide different perspectives.

10.4.1 Contextual Inquiry

Contextual Inquiry - is an approach that emerged from the ethnographic approach to data gathering. It is
tailored to gather data that can be used in design and it follows an apprenticeship model - the designer works
as an apprentice to the user. The most typical format for contextual inquiry is a contextual interview, which is
a combination of observation, discussion, and reconstruction of past events.
There are four main principles to contextual inquiry

Context principle
happens.

Page 44 of 66

emphasizes the importance of going to the workplace and seeing what

Chap 10 - Establishing Requirements

Partnership principle - states that the developer and the user should collaborate in undestanding
the work - the understanding is developed through cooperation.

Interpretation principle - states that the observation must be interpreted in order to be used in
design, and this interpretation should also be developed in cooperation between the user and the
developer.

Focus principle - keep the data gathering focused on your goals.

10.4.2 Data Gathering Guidelines for Requirements

Focus on identifying the stakeholders' needs;

Involve all the stakeholder groups. make sure you get the views of all the right people;
Involve more than one stakeholder from each group, so that you do not get a narrow view;
Support the data gathering sessions with suitable props, such as task descriptions and prototypes.

10.5 Data Analysis, Interpretation, and Presentation

Omit

10.6 Task Description

In recent years, due to the emphasis on involving users earlier in the development lifecycle, and the large
number of new interactive products now being developed, task descriptions are used throughout
development, from early requirements activities, through prototyping, evaluation, and testing.

Three of the more common description types are 10.6.1 Scenarios

Scenario - is an "informal narrative description".
It describes human activities or tasks in a story that allows exploration and discussion of contexts, needs,
and requirements. It does not explicitly describe the use of software or other technological support to achieve
a task. Using the vocabulary and phrasing of users means that the scenarios can be understood by the
stakeholders, and they are able to participate fully in the development process.

10.6.2 Use Cases

Use cases - also focus on user goals, but the emphasis is on a user-system interaction rather than the
users' task itself.
Although their focus is specifically on the interaction between the user (called an actor) and a software
system, the stress is still very much on the users' perspective, not the systems'. A use case is associated
with an actor, and it is the actors' goal in using the system that the use case wants to capture.
Use cases may be described graphically -

Page 45 of 66

Chap 10 - Establishing Requirements

To identify a use case, first identify the actors, ie: the people or other systems that will be interacting with the
system under development. Then examine these actors and identify their goal, or goals, in using the system.
Each of these will be a use case.

10.6.3 Essential Use Cases

Essential use cases were developed to combat the limitations of scenarios and use cases. Scenarios are
concrete stories that concentrate on realistic and specific activities. They can therefore obscure broader
issues concerned with the wider organizational view. Traditional use cases contain certain assumptions,
including the fact that there is a piece of technology to interact with, and also assumptions about the user
interface and the kind of interaction to be designed.
Essential use cases (also referred to as task cases) - represent abstractions from scenarios, ie: they
represent a more general case than a scenario embodies, and try to avoid the assumptions of a traditional
use case.
An essential use case is a structured narrative consisting of three parts: a name that expresses the overall
user intention, a stepped description of user actions, and a stepped description of system responsibility. This
division between user and system responsibilities can be very helpful during conceptual design when
considering task allocation and system scope, ie: what the user is responsible for and what the system has
to do.
Essential use cases would normally be developed before the more detailed use case.

10.7 Task Analysis

Task analysis - is an umbrella term that covers techniques for investigating cognitive processes and
physical actions, at a high level of abstraction and in minute detail.
It is mainly used to investigate an existing situation, not to envision new products. It is used to analyze the
underlying rationale and purpose of what people are doing: what they are trying to achieve, why they are
trying to achieve it, and how they are going about it.
Page 46 of 66

Chap 10 - Establishing Requirements

The most widely used version is Hierarchical Task Analysis 10.7.1 Hierarchical Task Analysis
Hierarchical Task Analysis - involves breaking down a task into subtasks, then sub-sub-tasks and so on.
These are grouped as plans which specify how the tasks might be performed in an actual situation.

HTA focuses on physical and observable actions that are performed, and includes looking at actions
not related to software or an interactive product;
HTA starts with a user goal, which is examined, and the main tasks for achieving it are identified;
These tasks are then divided into sub-tasks where appropriate.

Example HTA

Use of HTA has been controversial - there are two main problems

Real tasks are very complex, and task analysis does not scale well. The notation soon becomes
unwieldy and difficult to follow.

Task analysis is limited in the kinds of tasks it can model - it cannot model tasks that are overlapping
or parallel, nor can it model interruptions.

Advantages of HTA

It lets you objectively compare alternative designs;

It provides a good understanding of the interaction at whichever level of abstraction is appropriate.
It supports design reuse.

Page 47 of 66

Chap 10 - Establishing Requirements

Summary
In this chapter, we have looked in more detail at the importance of the requirements activity, and how to
establish requirements for interaction design. The data gathering techniques introduced in Chapter 7 can be
used in various combinations to gather requirements data. In addition, contextual enquiry, studying
documentation, and researching similar products are commonly used techniques. Scenarios, use cases, and
essential use cases are helpful techniques for beginning to document the findings from the data gathering
sessions. Task analysis is a little more structured, but does not scale well.

Key Points

Getting the requirements right is crucial to the success of the interactive product;

There are different kinds of requirements: functional, data, environmental (context of use), user
characteristics, usability goals, and user experience goals. Every product will have requirements
under each of these headings;

The most commonly used data gathering techniques for this activity are: questionnaires, interviews,
focus groups, direct observation, indirect observation, studying documentation, researching similar
products, and contextual enquiry;

Descriptions of user tasks such as scenarios, use cases and essential use cases can help users to
articulate existing work practices. They also help to express envisioned use for new products.

Task analysis techniques help to investigate existing systems and current practices.
--ooOoo--

Page 48 of 66

Chap 12 - Introducing Evaluation

Chapter 12 - Introducing Evaluation

Objectives

Explain the key concepts and terms used in evaluation;

Introduce a range of different types of evaluation methods;

Show how different evaluation methods are used for different purposes at different stages of the
design process and in different contexts of use;

Show how evaluators mix and modify methods to meet the demands of novel systems;

Discuss some of the practical challenges that evaluators have to consider when doing evaluation;

Illustrate through short case studies how methods discussed in more depth in Chapters 7 and 8 are
used in evaluation and describe some methods that are specific to evaluation.

Content
12.1 Introduction
Imagine you have designed a website for teenagers to share music, gossip, and photos. You have
prototyped your first design and implemented the core functionality. How would you find out whether it would
appeal to them and if they will use it ? You would need to evaluate it - but how ? This chapter presents an
introduction to the main types of evaluation and the methods you can use.
Evaluation is integral to the design process. Evaluators collect information about users' or potential users'
experiences when interacting with a prototype, a computer system, a component of a computer system, an
application, or a design artifact such as a screen sketch. They do this in order to improve its design.
Evaluation focuses on both the usability of the system (e.g. how easy it is to learn and to use) and on the
users' experience when interacting with the system (e.g. how satisfying, enjoyable, or motivating the
interaction is).
Devices like iPods, iPads, and e-readers have heightened awareness about usability, but many designers
still assume that if they and their colleagues can use a product and find it attractive, others will, too. The
problem with this assumption is that designers may design only for themselves. Evaluation enables them to
check that their design is appropriate and acceptable for the wider user population.
There are many different evaluation methods. Which to use depends on the goals of the evaluation.
Evaluations can occur in a range of places such as laboratories, people's homes, outdoors, and work
settings. Evaluations usually involve observing participants and measuring their performance - in usability
testing, experiments, or field studies. There are other methods, however, that do not involve participants,
such as modelling user behaviour. These tend to be approximations of what users might do when interacting
with an interface, often done as a quick and cheap way of assessing different interface configurations. The
Page 49 of 66

Chap 12 - Introducing Evaluation

level of control on what is evaluated varies; sometimes there is none such as in field studies and in others
there is considerable control over what tasks are performed and the context, such as in experiments.
In this chapter we discuss why evaluation is important, what needs to be evaluated, where evaluation should
take place, and when in the product lifecycle evaluation is needed. The different types are then illustrated by
short case studies.
12.2 The Why, What, Where, and When of Evaluation
12.2.1 Why Evaluate ?
Nowadays users expect much more than just a usable system, they also look for a pleasing and engaging
experience. From a business and marketing perspective, well-designed products sell.
There are good reasons for companies investing in evaluation. Designers can focus on real problems and
the needs of different user groups, rather than debating what each other likes or dislikes. It also allows
problems to be fixed before the product goes on sale.

12.2.2 What to evaluate

What to evaluate ranges from low-tech prototypes to complete systems, basically any or all aspects of any
product.

12.2.3 Where to Evaluate

Where evaluation takes place depends on what is being evaluated.
Some characteristics, such as web accessibility, are generally evaluated in a laboratory, because it provides
the control necessary to systematically investigate whether all requirements are met. User experience
aspects can be evaluated more effectively in natural settings which are often referred to as in the wild
studies.
Remote studies of online behaviour can be conducted to evaluate natural interactions for a lot of participants
in their own homes, cheaply and quickly. Living laboratories have also been built which are somewhere in
between labs and in the wild settings; providing the setting of being in an environment, while also giving the
ability to control, measure, and record activities.

12.2.4 When to evaluate

At what stage in the product lifecycle evaluation takes place depends on the type of product, eg: the product
being developed could be a brand-new concept, or it could be an upgrade of an existing product.
If a product is new, then considerable time is usually invested in market research and establishing user
requirements. These are used to create initial sketches, storyboard, or prototype of the design ideas. These
are then evaluated to see if the designers have interpreted the users' requirements correctly.
Two main types of Evaluation based on when they are done -

Page 50 of 66

Chap 12 - Introducing Evaluation

Formative Evaluation - done during design to check that the product continues to meet users
needs. Cover a broad range of design processes, from the development of early sketches and
prototypes through to tweaking and perfecting an almost finished design.

Summative Evaluation - assess the success of a finished product.

12.3 Types of Evaluation

Evaluations are classified into three broad categories, depending on the setting, user involvement, and level
of control

Controlled settings involving users (Examples are laboratories and living labs) - users' activities
are controlled in order to test hypotheses and measure or observe certain behaviours. The main
methods are usability testing and experiments.

Natural settings involving users (Examples are online communities and public places) - there is
little or no control of users' activities in order to determine how the product would be used in the real
world. The main method is the use of field studies.

Any settings not involving users - consultants and researchers critique, predict, and model
aspects of the interface in order to identify the most obvious usability problems. The range of
methods includes inspections, heuristics, walkthroughs, models, and analytics.

There are pros and cons of each type. Lab-based studies are good at revealing usability problems but poor
at capturing context of use; field studies are good at demonstrating how people use technologies in their
intended setting, but are expensive and difficult to conduct; and modelling and predicting approaches are
cheap and quick to perform but can miss unpredictable usability problems and subtle aspects of the user
experience.
A key concern for deciding on which approach to use, is how much control is needed in order to find out how
an interface or device is used.

12.3.1 Controlled Settings Involving Users

Controlled settings enable evaluators to control what users do, when they do it, and for how long. It also
enables them to reduce outside influences and distractions, such as colleagues talking.
Usability Testing

involves collecting data using a combination of methods

questionnaires.

eg: experiments, observation,

generally done in laboratories, although increasingly being done remotely or in natural settings.

primary goal is to determine whether an interface is usable by the intended user population to carry
out the tasks for which it was designed.

as users perform tasks, they may be recorded on video, their interactions with the software may be
recorded, usually by logging software.

user satisfaction questionnaires and interviews can also be used to elicit users' opinions.
Page 51 of 66

Chap 12 - Introducing Evaluation

observing users' reactions to an interactive product has helped developers understand usability
issues that would be extremely difficult for them to glean simply through reading reports, or listening
to presentations.

12.3.2 Natural Settings Involving Users

The aim of field studies is to evaluate people in their natural settings. They are used primarily to help identify opportunities for new technology;
establish the requirements for a new design;
facilitate the introduction of new technology.

methods that are typically used observation;

interviews;
logging.

data takes the form of events and conversations that are recorded by the researchers as notes, or by
audio or visual recording, or by the participants as diaries and notes.

a goal is to be unobtrusive and not to affect what people do during the observation.

there has been a trend towards conducting in the wild studies. In moving into the wild, researchers
inevitably have to give up control of what is being evaluated in order to observe how people use
technologies in their everyday lives.

the downside of handing over control is that it makes it difficult to anticipate what is going to happen,
and to be present when it does happen. This is in contrast to usability testing where there is always
an investigator or camera at hand to record events.

12.3.3 Any Settings Not Involving Users

Evaluations that take place without involving users are conducted in settings where the researcher has to
imagine or model how an interface is likely to be used. Inspection methods are commonly used to predict
user behaviour, the contexts in which the system will be used, and the kinds of activities that users
undertake.
Techniques include

Heuristic evaluation - applies knowledge of typical users guided by rules of thumb. One of the
problems of using heuristics is that designers can sometimes be led astray by findings that are not
as accurate as they first appeared to be.

Cognitive walkthroughs - involve simulating a users' problem-solving process at each step in the
human-computer dialogue, and checking to see how users progress from step to step in these
interactions. A key feature of cognitive walkthroughs is that they focus on evaluating designs for
ease of learning.

Analytics - is a technique for logging data either at a customers' site, or remotely. Web analytics is
the measurement, collection, analysis, and reporting of Internet data in order to understand and
optimize web usage.

Page 52 of 66

Chap 12 - Introducing Evaluation

Models - have been used primarily for comparing the efficacy of different interfaces for the same
application. A well known approach is the keystroke level model (KLM) that provides numerical
predictions of user performance. Another uses Fitts' law to predict the time it takes to reach a target
using a pointing device.

12.3.4 Choosing and Combining Methods

The three broad categories identified above provide a general framework to guide the selection of evaluation
methods. Often combinations of methods are used across the categories to obtain a better understanding.

12.3.5 Opportunistic Evaluations

Evaluations may be detailed, planned studies, or opportunistic explorations. These are generally done early
in the design process to provide designers with feedback quickly about a design idea. Typically these early
evaluations are informal and do not require many resources.
Getting feedback early in design can help save time and money if an idea needs to be modified or
abandoned.

12.4 Evaluation Case Studies

n/a

Page 53 of 66

Chap 12 - Introducing Evaluation

Summary
The aim of this chapter was to introduce the main approaches to evaluation and the methods typically used.
These will be revisited in more detail in the next three chapters. The chapter stressed how evaluation is done
throughout design; collecting information about users' or potential users experiences when interacting with a
prototype, a computer system, a component of a computer system, or a design artifact (eg: a screen sketch)
in order to improve its design.
The pros and cons of running lab-based versus in the wild studies were outlined, in terms of cost, effort,
constraints, and the types of results that can be elicited. Choosing which approach to use will depend on the
aims of the evaluation, and the researchers' or evaluators' expectations and the resources available to them.
Finally, crowdsourcing was presented as a creative, cost-saving evaluation approach.

Key Points

Evaluation and design are very closely integrated;

Some of the same data gathering methods are used in evaluation as for establishing requirements
and identifying users' needs, eg: observation, interviews, and questionnaires;

Evaluations can be done in controlled settings, such as laboratories, less controlled field settings, or
where users are not present;

Usability testing and experiments enable the evaluator to have a high level of control over what gets
tested, whereas evaluators typically impose little or no control on participants in field studies;

Different methods are usually combined to provide different perspectives within a study.
--ooOoo--

Page 54 of 66

Chap 14 - Evaluation Studies: From Controlled to Natural Settings

Chapter 14 - Evaluation Studies: From Controlled to Natural Settings

Objectives

Explain how to do usability testing;

Outline the basics of experimental design;

Describe how to do field studies.

Content
14.1 Introduction
Imagine you have designed a new shared web space intended for advertising second-hand goods. How
would you find out whether householders would be able to use it to find what they wanted and whether it was
a reliable and effective service ? What evaluation methods would you employ ?
In this chapter we describe evaluation studies that take place in a spectrum of settings, from controlled
laboratories to natural settings. Within this spectrum we focus on usability testing which takes place in
usability labs; experiments which take place in research labs; and field studies which take place in natural
settings such as people's homes, work, and leisure environments.
14.2 Usability Testing
The usability of products has traditionally been tested in controlled laboratory settings. The goal is to test
whether the product being developed is usable by the intended user population to achieve the tasks for
which it was designed.

14.2.1 Methods, Tasks, and Users

Collecting data about users' performance on predefined tasks is a central component of usability testing.
A combination of methods is used to collect data - video recordings, keystrokes and mouse movements that
are logged, "thinking aloud", questionnaires, interviews, and data about how the product is used in the field.
The quantitative performance measures that are obtained during the tests include time to complete a task;
time to complete a task after a specified time away from the product;
number and type of errors per task;
number of errors per unit of time;
number of navigations to online help or manuals;
number of users making a particular error;
number of users completing a task successfully.
The results of the tests are then fed back into the design of the product.
A key concern is the number of users that should be involved in a usability study - five to twelve is
considered an acceptable number, but it is possible to use fewer.
Page 55 of 66

Chap 14 - Evaluation Studies: From Controlled to Natural Settings

14.2.2 Labs and Equipment
Many companies test their products in custom-built well equipped usability labs.
Usability labs can be very expensive and labour-intensive to run and maintain. A less expensive alternative is
the use of mobile usability testing equipment. Video cameras, laptops and other measuring equipment can
be temporarily set up in an office or other space. An advantage is that equipment can be taken into work
settings, enabling testing to be done on site.
Another possibility is to conduct remote usability testing, where users perform a set of tasks with a product in
their own setting, and their interactions are logged remotely.

14.2.3 An Example of Usability Testing: The iPad

Usability problems - links on pages too small, fonts difficult to read.
Problems identified classified according to well-known interaction design principles

mental models;
navigation;
quality of images;
problem of using a touchscreen with small target areas;
lack of affordances;
getting lost in the application;
effects of changing orientations;
working memory;
feedback received.

14.3 Conducting Experiments

In research contexts, specific hypotheses are tested that make a prediction about the way users will perform
with an interface. The benefits are more rigour and confidence that one interface feature is easier to
understand or faster than another. Hypotheses are often based on a theory, such as Fitts' Law, or previous
research findings. Specific measurements provide a way of testing the hypothesis.

Hypothesis Testing - Typically, a hypothesis involves examining a relationship between two things,
called variables - an independent variable and a dependent variable. Test the effect of the
independent variable on the dependent variable.

Experimental Design - A concern in experiment design is to determine which participants to use for
which conditions in an experiment. The experience of participating in one condition will affect the
performance of those participants if asked to participate in another condition.

Statistics: t-tests - The t-test uses a simple equation to test the significance of the difference
between the means for the two conditions. If they are significantly different from each other we can
reject the null hypothesis, and infer that the other hypothesis holds.

Page 56 of 66

Chap 14 - Evaluation Studies: From Controlled to Natural Settings

14.4 Field Studies
Increasingly, more evaluation studies are being done in natural settings with either little or no control
imposed on participants' activities. This change is largely a response to technologies being developed for
use outside of office settings.
Evaluations conducted in natural settings are very different from those in controlled environments, where
tasks are set and completed in an orderly way. In contrast, studies in natural settings tend to be messy in the
sense that activities often overlap and are constantly interrupted. By evaluating how people think about,
interact, and integrate products within the settings they will ultimately be used in, we can get a better sense
of how successful the products will be in the real world.
Field studies can range in time from just a few minutes to a period of several months, or even years. Data is
collected primarily by observing and interviewing people; collecting video, audio, and field notes to record
what happens in the chosen setting. Participants may be asked to fill out paper based or electronic diaries at
particular points during the day.

Page 57 of 66

Chap 14 - Evaluation Studies: From Controlled to Natural Settings

Summary
This chapter described evaluation studies in different settings. It focused on controlled laboratory studies,
experiments, and field settings in natural settings. A study of the iPad when it first came out was presented
as an example of usability testing. This testing was done in an office environment; this was controlled to
avoid outside influences affecting the testing. The participants were also asked to conduct predefined tasks
that the evaluators were interested in investigating. Experimental design was then discussed that involves
testing a hypothesis in a controlled research lab. The chapter ended with a discussion of field studies in
which participants use new technologies in natural settings. The UbiFit Garden example involved evaluating
how participants used a mobile fitness system designed to encourage people to do daily exercise. The goal
of the evaluation was to examine how participants used the system in their daily lives, what kinds of
problems they encountered, and whether they liked the system.
Key differences between usability testing, experiments, and field studies include the location of the study usability or makeshift usability lab, research lab, or natural environment - and how much control is imposed.
At one end of the spectrum is laboratory testing and at the other are in the wild studies. Most studies use a
combination of different methods and evaluators often have to adapt their methods to cope with unusual new
circumstances created by the new systems being developed.

Key Points

Usability testing takes place in usability labs or temporary makeshift labs in a room in a company
that is dedicated for the purpose. These labs enable evaluators to control the test setting.

Usability testing focuses on performance measures such as how long and how many errors are
made, when completing a set of predefined tasks. Observation (video and keystroke logging) is
conducted and supplemented by user satisfaction questionnaires and interviews.

Usability-in-a-box and remote testing systems have been developed that are more affordable than
usability labs and also more portable. Many contain mobile eye-tracking and other devices.

Experiments aim to test a hypothesis by manipulating certain variables while keeping others
constant.

The experimenter controls independent variable/s in order to measure dependent variables/s.

Field studies are evaluation studies that are carried out in natural settings; they aim to discover how
people interact with technology in the real world.

Field studies that involve the deployment of prototypes or technologies in natural settings may also
be referred to as 'in the wild'.

Sometimes the findings of a field study can be unexpected.

--ooOoo--

Page 58 of 66

Chap 15 - Evaluation: Inspections, Analytics, and Models

Chapter 15 - Evaluation: Inspections, Analytics, and Models

Objectives

Describe the key concepts associated with inspection methods;

Explain how to do heuristic evaluation and walkthroughs;

Explain the role of analytics in evaluation;

Describe how to perform two types of predictive methods: GOMS and Fitts' Law.

Content
15.1 Introduction
The evaluation methods described so far in this book have involved interaction with, or direct observation of,
users. In this chapter we introduce methods that are based on understanding users through knowledge
codified in heuristics, or data collected remotely, or models that predict users' performance. None of these
methods require users to be present during the evaluation. Inspection methods typically involve an expert
role-playing the users for whom the product is designed, analyzing aspects of an interface, and identifying
any potential usability problems by using a set of guidelines. The most well known are heuristic evaluation
and walkthroughs. Analytics involves user interaction logging, which is often done remotely. Predictive
models involve analyzing the various physical and mental operations that are needed to perform particular
tasks at the interface and operationalizing them as quantitative measures. Two of the most commonly used
predictive models are GOMS and Fitts' Law.

15.2 Inspections: Heuristic Evaluation and Walkthroughs

Sometimes users are not easily accessible, or involving them is too expensive or takes too long. In such
circumstances other people, usually referred to as experts, can provide feedback. Various inspection
methods have been developed as alternatives to usability testing, including heuristic evaluations, and
walkthroughs, in which experts examine the interface of an interactive product, often role-playing typical
users, and suggest problems users would likely have when interacting with it.
These methods can be used at any stage of a design project, and can be used to complement user testing.
15.2.1 Heuristic Evaluation
Heuristic evaluation - is a usability inspection method (developed by Nielson) in which experts, guided by
a set of usability principles known as heuristics, evaluate whether user-interface elements, such as dialogue
boxes, menus, navigation structure, online help etc conform to tried and tested principles.
Nielsen's heuristics

Visibility of system status - the system should always keep users informed about what is going on,
through appropriate feedback within a reasonable time;

Page 59 of 66

Chap 15 - Evaluation: Inspections, Analytics, and Models

Match between system and the real world - the system should speak the users' language, with
words, phrases, and concepts familiar to the user;

User control and freedom - users need a clearly marked emergency exit to leave an unwanted
state / selection. Support undo / redo;

Consistency and standards - users should not have to wonder whether different words, situations,
or actions mean the same thing;

Error prevention - even better than good error messages, is a careful design that prevents a
problem from occurring in the first place;

Recognition rather than recall - minimize the users' memory load by making objects, actions, and
options visible. Instructions for use of the system should be visible or easily retrievable whenever
appropriate;

Flexibility and efficiency of use - accelerators - unseen by the novice user - may often speed up the
interaction for the expert user such that the system can cater to both inexperienced and experienced
users;

Aesthetic and minimalist design - dialogues should not contain information that is irrelevant or
rarely needed;

Help users recognize, diagnose and recover from errors - error messages should be expressed in
plain language, precisely indicate the problem, and constructively suggest a solution.

Help and documentation - even though it is better if the system can be used without documentation,
it may be necessary to provide help and documentation.

Because users and special facilities are not needed for heuristic evaluation, and it is comparatively
inexpensive and quick, it is popular with developers and is often referred to as 'discount evaluation'.
For a quick evaluation of an early design, one or two experts can probably identify most potential usability
problems, but if a thorough evaluation of a fully working prototype is needed, then having a team of experts
conducting the evaluation and comparing their findings would be advisable.

Doing Heuristic Evaluation

Three stages of heuristic evaluation 1. The briefing session - in which the experts are told what to do.
2. The evaluation period - in which each expert typically spends 1-2 hours independently inspecting
the product, using the heuristics for guidance. The experts need to take at least two passes through
the interface. The first pass gives a feel for the flow of the interaction and the products' scope, and
the second pass allows the evaluator to focus on specific interface elements in the context of the
whole product, and to identify potential usability problems.
3. The debriefing session - in which the evaluators come together to discuss their findings and to
prioritize the problems they found, and to suggest solutions.

Page 60 of 66

Chap 15 - Evaluation: Inspections, Analytics, and Models

15.2.2 Walkthroughs
Walkthroughs are an alternative approach to heuristic evaluation for predicting users' problems without doing
user testing. They involve 'walking through' a task with the product and noting problematic usability features.
Most walkthrough methods do not involve users.

Cognitive Walkthroughs
Cognitive walkthroughs involve simulating a users' problem solving process at each step in the humancomputer dialogue, checking to see if the users' goals and memory for actions can be assumed to lead to the
next correct action.
The defining feature is that they focus on evaluating designs for ease of learning.
The steps involved in cognitive walkthroughs are 1. The characteristics of typical users are identified and documented and sample tasks are developed
that focus on the aspects of the design to be evaluated. A clear sequence of the actions needed for
the users to complete the task is produced.
2. A designer and one or more expert evaluators come together to do the analysis.
3. The evaluators walk through the action sequences for each task, and try to answer the following
questions a) Will the correct action be sufficiently evident to the user ?
b) Will the user notice that the correct action is available ?
c) Will the user associate and interpret the response from the action correctly ?
4. As the walkthrough is being done, a record of critical information is compiled.
5. The design is then revised to fix the problems presented.
Compared with heuristic evaluation, this technique focuses more closely on identifying specific user
problems at a high level of detail. It has a narrow focus that is useful for certain types of system but not
others. it is very time-consuming and laborious to do and evaluators need a good understanding of the
cognitive processes involved.

Pluralistic Walkthroughs
Pluralistic walkthroughs are another type of walkthrough in which users, developers and usability experts
work together to step through a scenario, discussing usability issues associated with dialogue elements
involved in the scenario steps.

Each evaluator is asked to assume the role of a typical user.

Scenarios of use, consisting of a few prototype screens, are given to each evaluator who writes
down the sequence of actions they would take to move from one screen to another, without
conferring with fellow panellists.
Then the panellists discuss the actions they each suggested, before moving on to the next round of
screens.
This process continues until all the scenarios have been evaluated.
Page 61 of 66

Chap 15 - Evaluation: Inspections, Analytics, and Models

The benefits of pluralistic walkthroughs include a strong focus on users' tasks at a detailed level. This level of
analysis can be invaluable for certain kinds of systems, such as safety-critical ones, where a usability
problem identified for a single step could be critical to its safety or efficiency.
Limitations include having to get all the experts together at once, and then proceed at the rate of the slowest.
Furthermore, only a limited number of scenarios, and hence paths through the interface, can usually be
explored because of time constraints.
15.3 Analytics
Analytics - is a method for evaluating user traffic through a system. When used to examine traffic on a
website, or part of a website, it is known as web analytics.
Web analytics can be collected locally or remotely across the Internet by logging user activity, counting and
analyzing the data in order to understand what parts of the website are being used and when.
Although analytics are a form of evaluation that is particularly useful for evaluating the usability of a website,
they are also valuable for business planning. Many companies use the services of other companies, such as
Google and VisiStat, that specialize in providing analytics and the analysis necessary to understand the data.
5.4 Predictive Models
Similar to inspection methods and analytics, predictive models evaluate a system without users being
present. Rather than involving expert evaluators role-playing users as in inspections, or tracking their
behaviour as in analytics, predictive models use formulas to derive various measures of user performance.
Predictive modelling provides estimates of the efficiency of different systems for various kinds of task.
GOMS (goals, operators, methods, selection rules) - is a well-known predictive modelling technique. This is
a generic term used to refer to a family of models that vary in their granularity concerning the aspects of the
users' performance they model and make predictions about. These include the time it takes to perform tasks
and the most effective strategies to use.

Two of the most well-known GOMS techniques are - GOMS Model, and the keystroke level model (KLM) -

15.4.1 The GOMS Model

GOMS aims to model the knowledge and cognitive processes used when users interact with a system.

Goals - refer to a particular state the user wants to achieve (eg: find a website);

Operators - refer to the cognitive processes and physical actions that need to be performed in
order to attain those goals (eg: decide which search engine to use, think up and then enter a
keyword into a search engine). The difference between a goal and an operator is that a goal is
obtained, and an operator is executed.

Methods - are learned procedures for accomplishing the goals. They consist of the exact sequence
of steps required.

Selection rules - are used to determine which method to select when there is more than one
available for a given stage of a task.

Page 62 of 66

Chap 15 - Evaluation: Inspections, Analytics, and Models

15.4.2 The Keystroke Level Model (KLM)
The KLM differs from GOMS in that it provides numerical predictions of user performance. Tasks can be
compared in terms of the time it takes to perform them when using different strategies. The main benefit of
making this kind of quantitative prediction is that different features of systems and applications can be easily
compared to see which might be the most effective for performing specific kinds of tasks.
The developers of KLM analyzed the findings of many empirical studies of user performance in order to
derive a standard set of approximate times for the main kinds of operators used during a task. They were
able to come with the average time it takes to carry ut common physical actions (eg: press a key, click a
mouse button), together with other aspects of user-computer interaction (eg: the time it takes to decide what
to do, and the system response rate).
This model allows for predictions to be made about how long it takes an expert user to perform a task. The
predicted time is computed by describing the sequence of actions involved in the task and summing their
approximate times (looked up from empirical data) T(execute) = Tk + Tp + Th + Td + Tm + Tr
Operators K (keystroke); P (pointing); H (homing); D (drawing); M (mental preparation); R (system response time).

15.4.3 Benefits and Limitations of GOMS

Benefits allows for comparative analysis to be performed for different interfaces, prototypes, or specifications
relatively easily;
KLM can be adapted for use with advanced cell phones.
Limitations not often used for evaluation purposes because of its highly limited scope - it can only really model
computer-based tasks that involve a small set of highly routine data-entry type tasks;
it is intended to be used only for predicting expert performance, and does not allow for errors to be
modelled.

A challenge with predictive models is that they can only make predictions about predictable behaviour. Given
that most people are unpredictable, it makes it difficult to use them as a way of evaluating how systems will
be used in the real world. They are only useful for comparing the efficiency of different methods in completing
short, clearly defined tasks.

15.4.4 Fitts' Law

Fitts' Law predicts the time it takes to reach a target using a pointing device.

Used to describe the time it takes to point at a target, based on the size of the object, and the
distance to the object;

Specifically, it is used to model the time it takes to use a mouse and other input devices to click on
objects on a screen;

Page 63 of 66

Chap 15 - Evaluation: Inspections, Analytics, and Models

One of its main benefits is that it can help designers decide where to locate buttons, what size they
should be, and how close together they should be on a screen display;

T = k log2 (D/S + 1.0)

where: T
D
S
k

= time to move the pointer to a target

= distance between the pointer and the target
= size of the target
is a constant of approx 200ms/bit.

In short, the bigger the target, the easier and quicker it is to reach it;

Fitt's law predicts that the most quickly accessed targets on any computer display are the four
corners of the screen;

Fitt's law is useful for evaluating systems for which the time to locate an object is critical - such as
handheld devices like mobile phones.

Page 64 of 66

Chap 15 - Evaluation: Inspections, Analytics, and Models

Summary
This chapter presented inspection evaluation methods, focusing on heuristic evaluation and walkthroughs
which are usually done by specialists (usually referred to as experts), who role-play users' interactions with
designs, prototypes, and specifications and then offer their opinions. Heuristic evaluation and walkthroughs
offer the evaluator a structure to guide the evaluation process.
Analytics, in which user interaction is logged, is often performed remotely and without users being aware that
their interactions are being tracked. Very large volumes of data are collected, anonymized, and statistically
analyzed using specially developed software services. The analysis provides information about how a
system is used, eg: how different versions of a website or prototype perform, or which parts of a website are
seldom used - possibly due to poor usability design or lack of appeal. Data are often presented visually so
that it is easier to see trends and interpret the results.
The GOMS and KLM models, and Fitts' Law, can be used to predict user performance. These methods can
be useful for determining whether a proposed interface, system, or keypad layout will be optimal. Typically
they are used to compare different designs for a small sequence of tasks. These methods are labourintensive and so do not scale well for large systems.
Evaluators frequently find that they have to tailor these methods so that they can use them with the wide
range of products that have come onto the market since the methods were originally developed.

Key Points

Inspections can be used for evaluating a range of representations including requirements, mock-ups,
functional prototypes, or systems.

User testing and heuristic evaluation often reveal different usability problems.

Other types of inspections used in interaction design include pluralistic and cognitive walkthroughs.

Walkthroughs are very focused and so are suitable for evaluating small parts of a product.

Analytics involves collecting data about the interactions of users in order to identify which parts of a
website or prototype are underused.

When applied to websites, analytics are often referred to as 'web analytics'.

The GOMS and KLM models and Fitts' Law can be used to predict expert, error-free performance for
certain kinds of tasks.

Predictive models require neither users nor usability experts to be present, but the evaluators must
be skilled in applying the models.

Page 65 of 66

Chap 15 - Evaluation: Inspections, Analytics, and Models

Predictive models are used to evaluate systems with limited, clearly defined, functionality such as
data entry applications, and key-press sequences for cell phones and other handheld devices.

eof()

Page 66 of 66