Cognitive aspects

Overview
• What is cognition?
• What are users good and bad at?
• Describe how cognition has been applied to
interaction design
• Mental Models
• Internals classic theories of cognition
• More recent external theories of cognition
Why do we need
to understand
users?
Interacting with technology
is cognitive
 Need to take into account
cognitive processes involved
and cognitive limitations
of users
 Provides knowledge about
what users can and cannot be
expected to do
 Identifies and explains the
nature and causes of problems
users encounter
 Supply theories, modelling
tools, guidance and methods
that can lead to the design of
better interactive products
Cognitive processes
Attention
Perception and
recognition
Memory
Reading, speaking
and listening
 Problem-solving,
planning, reasoning
and decision-making
Attention
Attention

Selecting things to concentrate

on at a point in time from
the mass of stimuli around us
Attention

Allows us to to focus on
information that is relevant
to what we are doing
Attention

Involves audio and/or visual

senses
Attention
Focused and divided attention
enables us to be selective in terms
of the mass of competing stimuli
but limits our ability to keep track
of all events
Attention
Information at the interface should be
structured to capture users’ attention,
e.g. use perceptual boundaries
(windows), colour, reverse video,
sound and flashing lights
Activity: Find the price
of a double room at the
Holiday Inn in Bradley
Activity: Find the price
for a double room
at the Quality Inn
in Columbia
 Activity
o Tullis (1987) found that the two screens produced quite different results
 1st screen - took an average of 5.5 seconds to search
 2nd screen - took 3.2 seconds to search
o Why, since both displays have the same density of information (31%)?
o Spacing
 In the 1st screen the information is bunched up together, making it
hard to search
 In the 2nd screen the characters are grouped into vertical categories of
information making it easier
Multitasking
and attention
 Is it possible to perform
multiple tasks without one
or more of them being
detrimentally affected?
Multitasking and attention

Ophir et al (2009) compared

heavy vs light multi-taskers
Multitasking and attention

heavy were more prone to

being distracted than those
who infrequently multitask
Multitasking and attention

heavy multi-taskers are easily

distracted and find it difficult to
filter irrelevant information
 Design
implications for
attention
Design implications for attention

Make information salient

when it needs attending to
Design implications for attention

Use techniques that make

things stand out like color,
ordering, spacing, underlining,
sequencing and animation
Design implications for attention

Avoid using too much because

the software allows it
Perception
Perception
How information is
acquired from the world
and transformed into
experiences
Perception

Obvious implication is
to design representations that
are readily perceivable, e.g.
Perception

- Text should be legible

- Icons should be easy to distinguish
and read
 Is color contrast good? Find Italian

37
 Are borders and white space better? Find French

38
Activity
Perception
Weller (2004) found
people took less time
to locate items for
information that was
grouped
Perception

- using a border
(2nd screen) compared
with using color contrast
(1st screen)
42
Perception

Some argue that too much

white space on web pages
is detrimental to search
Perception

- Makes it hard to find

information
Perception

Do you agree?
Which is easiest to read and why?
What is the time? What is the time?

What is the time? What is the time?

What is the time?

 Design implications
○ Icons should enable users to readily distinguish their meaning
○ Bordering and spacing are effective visual ways of grouping information
○ Sounds should be audible and distinguishable
○ Speech output should enable users to distinguish between the set of spoken words
○ Text should be legible and distinguishable from the background
○ Tactile feedback should allow users to recognize and distinguish different
meanings
Memory
Memory
Involves first encoding
and then retrieving
knowledge
Memory
We don’t remember
everything - involves
filtering and processing
what is attended to
Memory
Context is important
in affecting our memory
(i.e. where, when)
Memory
We recognize things
much better than being
able to recall things
Processing
in memory
Encoding is first stage of memory

determines which
information is attended
to in the environment
and how it is interpreted
The more attention paid
to something…
 The more it is processed
in terms of thinking about it
and comparing it with other
knowledge…
The more likely it is to be remembered

e.g. when learning about HCI,

it is much better to reflect upon it, carry
out exercises, have discussions with
others about it, and write notes than just
passively read a book, listen to a lecture
or watch a video about it
Context is
important
 Context

Context affects the extent

to which information
can be subsequently
retrieved
 Context
• Sometimes it can be difficult for people to recall information
that was encoded in a different context:
“You are on a train and someone comes up to you and says
hello. You don’t recognize him for a few moments but then
realize it is one of your neighbours. You are only used to
seeing your neighbour in the hallway of your apartment block
and seeing him out of context makes him difficult to recognize
initially”

60
Activity
 Try to remember
the dates of your
grandparents’ birthday
 Try to remember
the cover of the last two
movies you watched
Which was easiest?
Why?
 People are very good
at remembering visual
cues about things
e.g. the color of items,
the location of objects
and marks on an object
 They find it more
difficult to learn and
remember arbitrary
material
e.g. birthdays and phone
numbers
Recognition
versus recall
Recognition versus recall
Command-based
interfaces require users
to recall from memory
a name from a possible
set of 100s
Recognition versus recall

GUIs provide visually-based

options that users need only
browse through until
they recognize one
Recognition versus recall
Web browsers, MP3 players,
etc., provide lists of visited
URLs, song titles etc.,
that support recognition
memory
Recognition versus recall

The problem with

the classic ‘7±2’
Recognition versus recall
George Miller’s (1956)
theory of how much
information people can
remember
Recognition versus recall

People’s immediate
memory capacity is
very limited
Recognition versus recall

Many designers think

this is useful finding
for interaction design
Recognition versus recall

But…
Recognition versus recall
What some
designers get up
to…
 • Present only 7 options on a menu
• Display only 7 icons on a tool bar
• Have no more than 7 bullets in a list
• Place only 7 items on a pull down menu
• Place only 7 tabs on the top of a website page
–But this is wrong? Why?

79
Recognition versus recall

Why?
Recognition versus recall

Inappropriate application of
the theory
Recognition versus recall

People can scan lists of

bullets, tabs, menu items
for the one they want
Recognition versus recall
They don’t have to recall
them from memory having
only briefly heard
or seen them
Recognition versus recall

Sometimes a small number

of items is good
Recognition versus recall

But depends on task and

available screen estate
 Personal
information
management
Personal information management

Personal information
management is a growing
problem for many users
Personal information management

vast numbers of documents,

images, music files, video
clips, emails, attachments,
bookmarks, etc.,
Personal information management

where and how to save them all,

then remembering what they
were called and where to find
them again
Personal information management

naming most common means

of encoding them
Personal information management

but can be difficult to

remember, especially
when have 1000s and 1000s
Personal information management

How might such a process be

facilitated taking into account
people’s memory abilities?
Memory involves 2 processes

recall-directed and
recognition-based scanning
 File management systems
should be designed to optimize
both kinds of memory
processes
e.g. Search box and
history list
Help users encode files in richer
ways
Provide them with ways
of saving files using colour,
flagging, image, flexible text,
time stamping, etc
 Is Apple’s
Spotlight search
tool any good?

97
Memory aids
Memory aids

SenseCam developed by
Microsoft Research Labs
Memory aids
a wearable device that
intermittently takes photos
without any user
intervention while worn
Memory aids
digital images taken are
stored and revisited
using special software
Memory aids
Has been found to
improve people’s
memory, suffering from
Alzheimers
 SenseCam

103
 Design
implications
Memory aids
Don’t overload users’
memories with
complicated procedures
for carrying out tasks
Memory aids
Design interfaces that
promote recognition
rather than recall
Memory aids
Provide users with
various ways of
encoding information to
help them remember
Memory aids

e.g. categories, color,

flagging, time stamping
 Reading,
speaking, and
listening
Reading, speaking, and listening

The ease with which

people can read, listen,
or speak differs
Reading, speaking, and listening

Many prefer listening to

reading
Reading, speaking, and listening

Reading can be quicker

than speaking
or listening
Reading, speaking, and listening

Listening requires less

cognitive effort than
reading or speaking
Reading, speaking, and listening

Dyslexics have
difficulties understanding
and recognizing written
words
Applications
Reading, speaking, and listening
Speech-recognition
systems allow users to
interact with them by
using spoken commands
Reading, speaking, and listening

e.g. Google Voice

Siri
Reading, speaking, and listening

Speech-output systems
use artificially generated
speech
Reading, speaking, and listening

e.g. written-text-to-speech
systems for the blind
Reading, speaking, and listening
Natural-language
systems enable users to
type in questions and
give text-based
responses
Reading, speaking, and listening

e.g. Google search

engine
 Design
implications
Design interfaces that
encourage exploration
Design interfaces that
constrain and guide
learners
 Dynamically linking
concepts and representations
can facilitate the learning
of complex material
 Problem-solving,
planning, reasoning
and decision-making
All involves reflective cognition

e.g. thinking about what to do,

what the options are, and
the consequences
Often involves conscious
processes, discussion
with others (or oneself),
and the use of artifacts
e.g. maps, books, pen
and paper
 May involve working
through different
scenarios and deciding
which is best option
 Design
implications
 Provide additional
information/functions for users
who wish to understand more
about how to carry out
an activity more effectively
 Use simple computational aids to
support rapid decision-making and
planning for users on the move
 Mental models
o Users develop an understanding of a system through learning
about and using it
o Knowledge is sometimes described as a mental model:

 How to use the system (what to do next)

 What to do with unfamiliar systems or unexpected situations

(how the system works)
o People make inferences using mental models of how to carry out
tasks
Mental models
Craik (1943) described mental models
as:
internal constructions
of some aspect
of the external world
enabling predictions
to be made
Involves unconscious and conscious processes

images and analogies

are activated
Deep versus shallow models

e.g. how to drive a car

and how it works
 Everyday reasoning
and mental models
(a) You arrive home on a hot summer’s day to a warm house.
How do you get the house to cool down as quickly as possible?
Set the thermostat to be at its lowest or to the desired
temperature?

(b) You arrive home starving hungry. You look in the fridge and
find all that is left is an uncooked pizza. You have an electric
oven. Do you warm it up to 375 degrees first and then put it in
(as specified by the instructions) or turn the oven up higher to
try to warm it up quicker?
Cooling down a room or
heating up oven that is
thermostat-controlled
 Many people have
erroneous mental models
(Kempton, 1996)
 Why?
General valve theory,
where ‘more is more’
principle is generalised to
different settings (e.g. gas
pedal, gas cooker, tap,
radio volume)
Why?

Thermostats based
on model of on-off
switch model
 Heating up a room or oven that is
thermostat-controlled

o Same is often true for understanding how interactive

devices and computers work:
 poor, often incomplete, easily confusable, based on
inappropriate analogies and superstition (Norman, 1983)
 e.g. elevators and pedestrian crossings - lot of people hit
the button at least twice
 Why? Think it will make the lights change faster or ensure
144 the elevator arrives!
Exercise: ATMs
Write down how an ATM works

How much money are

you allowed to take out?
Write down how an ATM works

What denominations?
Write down how an ATM works

If you went to another

machine and tried
the same what would
happen?
Write down how an ATM works

What information is on
the strip on your card?
How is this used?
Write down how an ATM works

What happens if you

enter the wrong number?
Write down how an ATM works
Why are there pauses
between the steps of
a transaction? What
happens if you try to type
during them?
Write down how an ATM works

Why does the card stay

inside the machine?
Write down how an ATM works

Do you count the money?

Why?
 How did you fare?
o Your mental model

 How accurate?

 How similar?

 How shallow?
o Payne (1991) did a similar study and found that people
frequently resort to analogies to explain how they work
o People’s accounts greatly varied and were often ad hoc
154
External
cognition
External cognition

Concerned with explaining

how we interact with external
representations (e.g. maps,
notes, diagrams)
External cognition

What are the cognitive

benefits and what processes
involved
External cognition

How they extend our

cognition
External cognition

What computer-based
representations can we
develop to help even more?
External cognition

Externalizing to
reduce memory load
 External cognition
• Diaries, reminders, calendars, notes, shopping lists, to-do lists
– written to remind us of what to do
• Post-its, piles, marked emails
– where placed indicates priority of what to do
• External representations:
– Remind us that we need to do something (e.g. to buy something for
mother’s day)
– Remind us of what to do (e.g. buy a card)
– Remind us when to do something (e.g. send a card by a certain date)
Computational
offloading
Computational offloading
When a tool is used
in conjunction with an
external representation to
carry out a computation
(e.g. pen and paper)
Computational offloading

Try doing the two sums

below (a) in your head, (b)
on a piece of paper and (c)
with a calculator.
Computational offloading

- 234 x 456 =??

- CCXXXIIII x CCCCXXXXXVI = ???
Computational offloading

Which is easiest and why?

Both are identical sums
Annotation and
cognitive tracing
Annotation and cognitive tracing

Annotation involves
modifying existing
representations through
making marks
Annotation and cognitive tracing

e.g. crossing off, ticking,

underlining
Annotation and cognitive tracing

Cognitive tracing
involves externally
manipulating items into
different orders
or structures
Annotation and cognitive tracing

e.g. playing Scrabble,

playing cards
Annotation and cognitive tracing

Design
implication
Annotation and cognitive tracing

Provide external
representations at the interface
that reduce memory load and
facilitate computational
offloading
e.g. Information
visualizations have
been designed to
allow people to make
sense and rapid
decisions about
masses of data
Distributed
cognition
Distributed cognition

Concerned with the nature of

cognitive phenomena across
individuals, artifacts, and internal
and external representations
(Hutchins, 1995)
Distributed cognition

Describes these in terms

of propagation across
representational state
Distributed cognition

Information is transformed
through different media
(computers, displays, paper,
heads)
How it differs from information processing
 Distributed cognition

180
Distributed cognition

What’s involved
Distributed cognition

The distributed problem-

solving that takes place
Distributed cognition

The role of verbal and

non-verbal behavior
Distributed cognition

The various coordinating

mechanisms that are used
(e.g. rules, procedures)
Distributed cognition

The communication
that takes place as
the collaborative
activity progresses
Distributed cognition

How knowledge is shared

and accessed
Summary
Summary

Cognition involves several

processes including attention,
memory, perception and
learning
Summary

The way an interface is designed

can greatly affect how well users
can perceive, attend, learn and
remember how to do their tasks
Summary

Theoretical frameworks, such as

mental models and external cognition,
provide ways of understanding how
and why people interact with products
Summary

This can lead to thinking

about how to design better
products