Design of interactive software systems Adriana Guran

HCI BASICS
Lecture 1

Agenda
The Human
The Computer
The Interaction

THE HUMAN
HCI Basics

The Human
Information i/o
visual, auditory, haptic, movement
Information stored in memory
sensory, short-term, long-term

Information processed and applied

reasoning, problem solving, skill, error
Emotion influences human capabilities
Each person is different

Model Human Processor

Vision
Two stages in vision:
physical reception of stimulus
processing and interpretation of stimulus

Designing with blue

Optical center

The Eye - physical reception

mechanism for receiving light and transforming it into

electrical energy
light reflects from objects
images are focused upside-down on retina
retina contains rods for low light vision and cones for

colour vision
ganglion cells (brain!) detect pattern and movement

Vision

Vision - compensation

Vision

Optical Illusions

the Ponzo illusion

the Muller Lyer illusion

Optical Illusions

Optical Illusions

Optical Illusions

Colors
Major impact in UI design

Beyond pure aesthetics, color has associated meanings (Cultural

differences) and elicits emotional responses.

To prevent confusion in meaning, color must be used consistently
An UI must be designed without colors in the beginning
Magic number: 5+2 colors in an interface

For different concepts use different colors

Accessibility
The primary colors as seen with
normal color vision.
The primary colors as seen with
Protanopia (1% of male population).
The primary colors as seen with
Deuteranopia (6% of male population).
The primary colors as seen with
Tritanopia (1% of male population).

Designing with blue

There are special usability issues to be considered in using blue and

yellow in graphics.
Legibility, temporal response, spatial localization and perception of

geometrical shapes are all somewhat compromised in patterns in

which the only differences are in the short-wave-sensitive (SWS,
"blue") photoreceptors.
In graphics terms this mostly applies to color pairs that differ only in

the blue primary.

recommendation: Pure blue should not be used for fine detail or

background

Legibility

Difficult to read in any font size

Text differ from their backgrounds only in the blue

primary:

Solutions

Blue can be used in most contexts if care is taken to achieve

adequate luminance contrast. T

Reading
Several stages:
visual pattern perceived
decoded using internal representation of language
interpreted using knowledge of syntax, semantics, pragmatics

Reading involves saccades and fixations

Perception occurs during fixations
Word shape is important to recognition
Negative contrast improves reading from computer screen

Reading

Can you read this?

I cnduo't bvleiee taht I culod aulaclty uesdtannrd

waht I was rdnaieg. Unisg the icndeblire pweor of

the hmuan mnid, aocdcrnig to rseecrah at
Cmabrigde Uinervtisy, it dseno't mttaer in waht
oderr the lterets in a wrod are, the olny irpoamtnt
tihng is taht the frsit and lsat ltteer be in the rhgit
pclae. The rset can be a taotl mses and you can
sitll raed it whoutit a pboerlm. Tihs is bucseae the
huamn mnid deos not raed ervey ltteer by istlef,
but the wrod as a wlohe. Aaznmig, huh?

Correct paragraph
I couldn't believe that I could actually understand

what I was reading. Using the incredible power of

the human brain, according to research at
Cambridge University, it doesn't matter in what
order the letters in a word are, the only important
thing is that the first and last letter be in the right
place. The rest can be a total, mess and you can
read it without a problem. This is because the
human mind does not read every letter by itself,
but the word as a whole. Amazing, huh?

Hearing
Provides information about environment:
distances, directions, objects etc.
Physical apparatus:
outer ear
protects inner and amplifies sound
middle ear
transmits sound waves as
inner ear

Sound
pitch
loudness
timbre

vibrations to inner ear

chemical transmitters are released
and cause impulses in auditory nerve

sound frequency
amplitude
type or quality

Hearing
Humans can hear frequencies from 20Hz to 15kHz
less accurate distinguishing high frequencies than low.
Auditory system filters sounds
can attend to sounds over background noise.
for example, the cocktail party phenomenon - when one may
immediately detect words of importance originating from
unattended stimuli, for instance hearing one's name in another
conversation

Touch
Provides important feedback about environment.

May be key sense for someone who is visually impaired.

Stimulus received via receptors in the skin:
thermoreceptors

nociceptors
mechanoreceptors

heat and cold

pain
pressure
(some instant, some continuous)

Some areas more sensitive than others e.g. fingers.

Kinethesis - awareness of body position

affects comfort and performance.

Movement
Time taken to respond to stimulus:

reaction time + movement time

Movement time dependent on age, fitness etc.
Reaction time - dependent on stimulus type:
visual
~ 200ms
auditory
~ 150 ms
pain
~ 700ms
Increasing reaction time decreases accuracy in the

unskilled operator but not in the skilled operator.

Movement
Fitts' Law describes the time taken to hit a screen

target:
Mt = a + b log2(D/S + 1)
where:

a and b are empirically determined constants

Mt is movement time
D is Distance
S is Size of target

targets as large as possible

distances as small as possible

Memory
There are three types of memory function:
Sensory memories
Short-term memory or working memory

Long-term memory
Selection of stimuli governed by level of arousal.

sensory memory
Buffers for stimuli received through senses
iconic memory: visual stimuli
echoic memory: aural stimuli
haptic memory: tactile stimuli

Examples
sparkler trail
stereo sound
Continuously overwritten

Short-term memory (STM)

Scratch-pad for temporary recall
rapid access ~ 70ms
rapid decay ~ 200ms
limited capacity - 7 2 chunks

STM
Try to memorize as much as possible from the following sequence

265397620853

Try to memorize as much as possible from the following sequence

0040 732 215 754

Examples
212348278493202
0121 414 2626
HEC ATR ANU PTH ETR EET

Long-term memory (LTM)

Repository for all our knowledge
slow access ~ 1/10 second
slow decay, if any
huge or unlimited capacity
Two types
episodic
semantic

serial memory of events

structured memory of facts, concepts, skills

semantic LTM derived from episodic LTM

Long-term memory
Semantic memory structure
provides access to information
represents relationships between bits of information
supports inference
Model: semantic network
inheritance child nodes inherit properties of parent nodes
relationships between bits of information explicit
supports inference through inheritance

LTM - Forgetting
decay
information is lost gradually but very slowly

interference
new information replaces old: retroactive interference

old may interfere with new: proactive inhibition

so may not forget at all memory is selective

affected by emotion can subconsciously `choose' to forget

LTM - retrieval
recall
information reproduced from memory can be assisted by cues, e.g.

categories, imagery

recognition
information gives knowledge that it has been seen before
less complex than recall - information is cue

THINKING
Reasoning
deduction, induction, abduction
Problem solving

Deductive Reasoning
Deduction:

derive logically necessary conclusion from given premises.

e.g. If it is Friday then she will go to work
It is Friday
Therefore she will go to work.

Logical conclusion not necessarily true:

e.g.

If it is raining then the ground is dry

It is raining
Therefore the ground is dry

Deduction
When truth and logical validity clash
e.g. Some people are babies
Some babies cry
Inference - Some people cry

Correct?
People bring world knowledge to bear

Inductive Reasoning
Induction:
generalize from cases seen to cases unseen
e.g.
all elephants we have seen have trunks
therefore all elephants have trunks.

Unreliable:
can only prove false not true

but useful!
Humans not good at using negative evidence
e.g. Wason's cards.

Wason's cards

7 E 4 K
If a card has a vowel on one side it has an even number on the other
Is this true?
How many cards do you need to turn over to find out?

. and which cards?

Abductive reasoning
reasoning from event to cause
e.g.
Sam drives fast when drunk.
If I see Sam driving fast, assume drunk.

Unreliable:
can lead to false explanations

Reasoning

Problem solving
Process of finding solution to unfamiliar task using

knowledge.
Several theories.

Gestalt
problem solving both productive and reproductive
productive draws on insight and restructuring of problem
attractive but not enough evidence to explain `insight' etc.
move away from behaviourism and led towards information processing

theories

Problem solving
Problem space theory
problem space comprises problem states
problem solving involves generating states using legal operators
heuristics may be employed to select operators

e.g. means-ends analysis

operates within human information processing system
e.g. STM limits etc.
largely applied to problem solving in well-defined areas
e.g. puzzles rather than knowledge intensive areas

Problem solving
Analogy
analogical mapping:
novel problems in new domain?
use knowledge of similar problem from similar domain
analogical mapping difficult if domains are semantically different

Skill acquisition
skilled activity characterized by chunking
lot of information is chunked to optimize STM
conceptual rather than superficial grouping of problems
information is structured more effectively

Errors and mental models

Types of error
slips

right intention, but failed to do it right

causes: poor physical skill,inattention etc.
change to aspect of skilled behaviour can cause slip

mistakes

wrong intention

cause: incorrect understanding

humans create mental models to explain behaviour.
if wrong (different from actual system) errors can occur

Mental models
"The image of the world around us, which we carry in our

head, is just a model. Nobody in his head imagines all the

world, government or country. He has only selected
concepts, and relationships between them, and uses
those to represent the real system. (Jay Wright Forrester
MIT professor)

Everyday Life and Mental Models

(a) You arrive home on a cold winters night to a cold

house. How do you get the house to warm up as

quickly as possible? Set the thermostat to be at its
highest or to the desired temperature?
(b) You arrive home starving hungry. You look in the fridge
and find all that is left is an uncooked pizza. You have
an electric oven. Do you warm it up to 375 degrees first
and then put it in (as specified by the instructions) or
turn the oven up higher to try to warm it up quicker?

Exercise: ATMs
How an ATM works
How much money are you allowed to take out?
What denominations?
If you went to another machine and tried the same what would

happen?

What information is on the strip on your card? How is this used?

What happens if you enter the wrong number?
Why are there pauses between the steps of a transaction? What

happens if you try to type during them?

Why does the card stay inside the machine?

Do you count the money? Why?

How did you fare?

Your mental model
How accurate?
How similar?
How shallow?
Payne (1991) did a similar study and found

that people frequently resort to analogies to

explain how they work
Peoples accounts greatly varied and were

often ad hoc

Computers
Same is often true for understanding how

interactive devices and computers work:

Poor, often incomplete, easily confusable, based on

inappropriate analogies and superstition (Norman,

1983)
e.g. frozen cursor/screen - most people will bash all
manner of keys

External cognition
Concerned with explaining how we interact with

external representations (e.g. maps, notes,

diagrams)
What are the cognitive benefits and what

processes involved
How they extend our cognition
What computer-based representations can we

develop to help even more?

Externalizing to reduce memory load

Diaries, reminders,calendars, notes, shopping lists, to-do

lists - written to remind us of what to do

Post-its, piles, marked emails - where placed indicates

priority of what to do
External representations:
Remind us that we need to do something (e.g. to buy something for
mothers day)
Remind us of what to do (e.g. buy a card)
Remind us when to do something (e.g. send a card by a certain
date)

Computational offloading
When a tool is used in conjunction with an external

representation to carry out a computation (e.g. pen

and paper)
Try doing the two sums below (a) in your head, (b) on

a piece of paper and c) with a calculator.

234 x 456 =??

CCXXXIIII x CCCCXXXXXVI = ???

Which is easiest and why? Both are identical sums

Annotation and cognitive tracing

Annotation involves modifying existing

representations through making marks

e.g. crossing off, ticking, underlining

Cognitive tracing involves externally manipulating

items into different orders or structures

e.g. playing scrabble, playing cards

Design implication
Provide external representations at the

interface that reduce memory load and

facilitate computational offloading
e.g. Information
visualizations have
been designed to
allow people to make
sense and rapid
decisions about
masses of data

Mental models & system

design
Notion of mental models has been used as a basis for

conceptual models
Assumption is that if you can understand how people

develop mental models then can help them develop more

appropriate mental models of system functionality
For example, a design principle is to try to make systems

transparent so people can understand them better and

know what to do

The design principle of transparency

useful feedback
easy to understand
intuitive to use
clear & easy to follow instructions
appropriate online help
context sensitive guidance of
how to proceed when stuck

Emotions
Various theories of how emotion works
James-Lange: emotion is our interpretation of a physiological
response to a stimuli
Cannon: emotion is a psychological response to a stimuli
Schacter-Singer: emotion is the result of our evaluation of our

physiological responses, in the light of the whole situation we

are in

Emotion clearly involves both cognitive and physical

responses to stimuli

Emotions
The biological response to physical stimuli is called

affect
Affect influences how we respond to situations
positive creative problem solving
negative narrow thinking
Negative affect can make it harder to do even easy tasks;
positive affect can make it easier to do difficult tasks
(Donald Norman)

Emotion
Implications for interface design
stress will increase the difficulty of problem solving

relaxed users will be more forgiving of shortcomings in design

aesthetically pleasing and rewarding interfaces will increase

positive affect (Attractive things work better?)

Individual differences
long term

sex, physical and intellectual abilities

short term
effect of stress or fatigue
changing
age
Ask yourself:
will design decision exclude section of user population?

Psychology and the Design of Interactive

System
Some direct applications
e.g.

blue acuity is poor

blue should not be used for important detail

However, correct application generally requires understanding of

context in psychology, and an understanding of particular

experimental conditions
A lot of knowledge has been distilled in
guidelines
cognitive models
experimental and analytic evaluation techniques

THE COMPUTER
HCI Basics

The Computer
a computer system is made up of various elements
each of these elements affects the interaction
input devices text entry and pointing
output devices screen (small&large), digital paper
virtual reality special interaction and display devices
physical interaction e.g. sound, haptic, bio-sensing

paper as output (print) and input (scan)

memory RAM & permanent media, capacity & access

processing speed of processing, networks

Interacting with computers

to understand humancomputer interaction
need to understand computers!
what goes in and out
devices, paper,
sensors, etc.

what can it do?

memory, processing,
networks

A typical computer system

screen, or monitor, on which there are windows

keyboard
mouse/trackpad
window 1

variations
desktop
laptop
PDA

window 2

12-37pm

the devices dictate the styles of interaction that the system supports
If we use different devices, then the interface will support a different style of
interaction

How many
computers in your house?
hands up,
none, 1, 2 , 3, more!!

computers in your pockets?

are you thinking

PC, laptop, PDA ??

How many computers

in your house?

in your pockets?

PC

PDA

TV, VCR, DVD, HiFi,

phone, camera

cable/satellite TV
microwave, cooker, washing
machine
central heating
security system

smart card, card with

magnetic strip?
electronic car key
USB memory
try your pockets and bags

can you think of more?

Interactivity?
Long ago in a galaxy far away batch processing
punched card stacks or large data files prepared
long wait .
line printer output

and if it is not right

Now most computing is interactive

rapid feedback
the user in control (most of the time)

doing rather than thinking

Is faster always better?

Richer interaction

sensors
and devices
everywhere

TEXT ENTRY DEVICES

keyboards (QWERTY et al.)
chord keyboards, phone pads
handwriting, speech

Keyboards
Most common text input device
Allows rapid entry of text by experienced users
Keypress closes connection, causing a character code to

be sent
Usually connected by cable, but can be wireless

layout QWERTY
Standardised layout

but
non-alphanumeric keys are placed differently
accented symbols needed for different scripts
minor differences between UK and USA keyboards

QWERTY arrangement not optimal for typing

layout to prevent typewriters jamming!

Alternative designs allow faster typing but large social base of
QWERTY typists produces reluctance to change.

QWERTY

1
Q

3
W

A
Z

5
R

D
X

7
U

SPACE

9
I

J
M

0
O

L
,

alternative keyboard layouts

Alphabetic
keys arranged in alphabetic order
not faster for trained typists
not faster for beginners either!

Dvorak
common letters under dominant fingers
biased towards right hand
common combinations of letters alternate between hands

10-15% improvement in speed and reduction in fatigue

But - large social base of QWERTY typists produce market pressures not

to change

special keyboards
designs to reduce fatigue for RSI
for one handed use
e.g. the Maltron left-handed keyboard

Chord keyboards
only a few keys - four or 5
letters typed as combination of keypresses
compact size
ideal for portable applications
short learning time
keypresses reflect letter shape
fast
once you have trained

BUT - social resistance, plus fatigue after extended use

NEW niche market for some wearables

phone pad and T9 entry

use numeric keys with

multiple presses
2abc
3-def
4-ghi
5-jkl

6-mno
7-pqrs
8-tuv
9-wxyz
hello = 4433555[pause]555666

surprisingly fast!
T9 predictive entry
type as if single key for each letter
use dictionary to guess the right word
hello = 43556
but 26 -> menu am or an

Handwriting recognition
Text can be input into the computer, using a pen and a

digesting tablet
natural interaction

Technical problems:
capturing all useful information - stroke path, pressure, etc. in a
natural manner
segmenting joined up writing into individual letters
interpreting individual letters
coping with different styles of handwriting
Used in PDAs, and tablet computers

leave the keyboard on the desk!

Speech recognition
Improving rapidly
Most successful when:
single user initial training and learns peculiarities
limited vocabulary systems
Problems with
external noise interfering
imprecision of pronunciation
large vocabularies
different speakers

Numeric keypads
for entering numbers quickly:
calculator, PC keyboard
for telephones

not the same!!

ATM like phone

telephone

calculator

POSITIONING, POINTING
AND DRAWING
mouse, touchpad
trackballs, joysticks etc.
touch screens, tablets
eyegaze, cursors

the Mouse
Handheld pointing device
very common
easy to use
Two characteristics
planar movement
buttons
(usually from 1 to 3 buttons on top, used for making a selection,
indicating an option, or to initiate drawing etc.)

the mouse
Mouse located on desktop
requires physical space
no arm fatigue

Relative movement only is detectable.

Movement of mouse moves screen cursor
Screen cursor oriented in (x, y) plane,
mouse movement in (x, z) plane
an indirect manipulation device.
device itself doesnt obscure screen, is accurate and fast.
hand-eye coordination problems for novice users

How does it work?

Two methods for detecting motion
Mechanical
Ball on underside of mouse turns as mouse is moved
Rotates orthogonal potentiometers
Can be used on almost any flat surface

Optical
light emitting diode on underside of mouse
may use special grid-like pad or just on desk

less susceptible to dust and dirt

detects fluctuating alterations in reflected light intensity to calculate relative

motion in (x, z) plane

Even by foot
some experiments with the footmouse
controlling mouse movement with feet
not very common :-)

but foot controls are common elsewhere:

car pedals
sewing machine speed control
organ and piano pedals

Touchpad
small touch sensitive tablets
stroke to move mouse pointer
used mainly in laptop computers

good acceleration settings important

fast stroke
lots of pixels per inch moved
initial movement to the target

slow stroke
less pixels per inch
for accurate positioning

Trackball and thumbwheels

Trackball
ball is rotated inside static housing
like an upsdie down mouse!
relative motion moves cursor
indirect device, fairly accurate

separate buttons for picking

very fast for gaming
used in some portable and notebook computers.

Thumbwheels
for accurate CAD two dials for X-Y cursor position
for fast scrolling single dial on mouse

Joystick and keyboard nipple

Joystick
indirect

pressure of stick = velocity of movement

buttons for selection
on top or on front like a trigger
often used for computer games
aircraft controls and 3D navigation

Keyboard nipple
for laptop computers
miniature joystick in the middle of the keyboard

Touch-sensitive screen
Detect the presence of finger or stylus on the screen.
works by interrupting matrix of light beams, capacitance changes or ultrasonic
reflections
direct pointing device
Advantages:
fast, and requires no specialised pointer
good for menu selection
suitable for use in hostile environment: clean and safe from damage.
Disadvantages:
finger can mark screen
imprecise (finger is a fairly blunt instrument!)
difficult to select small regions or perform accurate drawing

lifting arm can be tiring

Stylus and light pen

Stylus
small pen-like pointer to draw directly on screen
may use touch sensitive surface or magnetic detection
used in PDA, tablets PCs and drawing tables

Light Pen
now rarely used
uses light from screen to detect location

BOTH
very direct and obvious to use
but can obscure screen

Digitizing tablet
Mouse like-device with cross hairs
used on special surface

- rather like stylus

very accurate

- used for digitizing maps

Eyegaze
control interface by eye gaze direction
e.g. look at a menu item to select it
uses laser beam reflected off retina
a very low power laser!
mainly used for evaluation
potential for hands-free control
high accuracy requires headset
cheaper and lower accuracy devices available

sit under the screen like a small webcam

Cursor keys
Four keys (up, down, left, right) on keyboard.

Very, very cheap, but slow.

Useful for not much more than basic motion for text-editing tasks.
No standardised layout, but inverted T, most common

Discrete positioning controls

in phones, TV controls etc.
cursor pads or mini-joysticks
discrete left-right, up-down
mainly for menu selection

DISPLAY DEVICES
bitmap screens (CRT & LCD)
large & situated displays
digital paper

bitmap displays
screen is vast number of coloured dots

resolution and colour depth

Resolution used (inconsistently) for
number of pixels on screen (width x height)
e.g. SVGA 1024 x 768, PDA perhaps 240x400
density of pixels (in pixels or dots per inch - dpi)
typically between 72 and 96 dpi

Aspect ratio
ration between width and height
4:3 for most screens, 16:9 for wide-screen TV

Colour depth:
how many different colours for each pixel?
black/white or greys only
256 from a pallete
8 bits each for red/green/blue = millions of colours

Digital paper
what?
thin flexible sheets
updated electronically
but retain display

how?
small spheres turned
or channels with coloured liquid
and contrasting spheres
rapidly developing area

appearance

cross
section

VIRTUAL REALITY AND

3D INTERACTION
positioning in 3D space
moving and grasping
seeing 3D (helmets and caves)

positioning in 3D space
cockpit and virtual controls
steering wheels, knobs and dials just like real!
the 3D mouse
six-degrees of movement: x, y, z + roll, pitch, yaw

data glove
fibre optics used to detect finger position
VR helmets
detect head motion and possibly eye gaze

whole body tracking

accelerometers strapped to limbs or reflective dots and video
processing

PAPER: PRINTING AND

SCANNING
print technology
fonts, page description, WYSIWYG
scanning, OCR

Printing
image made from small dots
allows any character set or graphic to be printed,
critical features:
resolution
size and spacing of the dots
measured in dots per inch (dpi)

speed
usually measured in pages per minute
cost!!

Types of dot-based printers

dot-matrix printers
use inked ribbon (like a typewriter
line of pins that can strike the ribbon, dotting the paper.
typical resolution 80-120 dpi

ink-jet and bubble-jet printers

tiny blobs of ink sent from print head to paper
typically 300 dpi or better .

laser printer
like photocopier: dots of electrostatic charge deposited on drum, which

picks up toner (black powder form of ink) rolled onto paper which is then
fixed with heat
typically 600 dpi or better.

Printing in the workplace

shop tills
dot matrix
same print head used for several paper rolls
may also print cheques
thermal printers
special heat-sensitive paper
paper heated by pins makes a dot
poor quality, but simple & low maintenance
used in some fax machines

Fonts

Font the particular style of text

Courier font
Helvetica font
Palatino font
Times Roman font
(special symbol)

Size of a font measured in points (1 pt about 1/72)

(vaguely) related to its height
This is ten point Helvetica

This is twelve point

This is fourteen point

This is eighteen point

and this is twenty-four point

Fonts
Pitch
fixed-pitch every character has the same width

e.g. Courier
variable-pitched some characters wider

e.g. Times Roman compare the i and the m

Serif or Sans-serif
sans-serif square-ended strokes

e.g. Helvetica
serif with splayed ends (such as)

e.g. Times Roman or Palatino

Readability of text
lowercase
easy to read shape of words
UPPERCASE
better for individual letters and non-words
e.g. flight numbers: BA793 vs. ba793
serif fonts

helps your eye on long lines of printed text

but sans serif often better on screen

Scanners
Take paper and convert it into a bitmap
Two sorts of scanner
flat-bed: paper placed on a glass plate, whole page converted into bitmap
hand-held: scanner passed over paper, digitising strip typically 3-4 wide

Shines light at paper and note intensity of reflection

colour or greyscale

Typical resolutions from 6002400 dpi

Scanners
Used in

desktop publishing for incorporating photographs and other

images

document storage and retrieval systems, doing away with

paper storage

special scanners for slides and photographic negatives

MEMORY
short term and long term
speed, capacity, compression
formats, access

Short-term Memory - RAM

Random access memory (RAM)
on silicon chips
100 nano-second access time
usually volatile (lose information if power turned off)
data transferred at around 100 Mbytes/sec
Some non-volatile RAM used to store basic set-up

information
Typical desktop computers:

64 to 256 Mbytes RAM

Long-term Memory - disks

magnetic disks
floppy disks store around 1.4 Mbytes
hard disks typically 40 Gbytes to 100s of Gbytes
access time ~10ms, transfer rate 100kbytes/s

optical disks
use lasers to read and sometimes write
more robust that magnetic media
CD-ROM
- same technology as home audio, ~ 600 Gbytes
DVD - for AV applications, or very large files