The Computer

a computer system is made up of various elements

each of these elements affects the interaction

0 input devices – text entry and pointing
0 output devices – screen (small&large), digital paper
0 virtual reality – special interaction and display devices
0 physical interaction – e.g. sound, touch screen
0 paper – as output (print) and input (scan)
0 memory – RAM & permanent media, capacity & access
0 processing – speed of processing, networks
 Interacting with computers
to understand human–computer interaction
… need to understand computers!

what goes in and out

devices, paper,
sensors, etc.

what can it do?

memory, processing,
networks
 A ‘typical’ computer system
? 0 screen, or monitor, on which there are windows
0 keyboard window 1
0 mouse/trackpad
window 2
0 variations
0 desktop
0 laptop
0 PDA
12-37pm

the devices dictate the styles of interaction that the system supports
If we use different devices, then the interface will support a different
style of interaction
 Interactivity?
Now most computing is interactive
0 quick feedback
0 the user in control (most of the time)
0 doing rather than thinking …

Is faster always better?

Richer interaction

sensors
and devices
everywhere
 How many …
0 computers in your house?
0 hands up, …
… none, 1, 2 , 3, more!!

0 computers in your pockets?

are you thinking …

… PC, laptop, PDA ??
 How many computers …
in your house? in your pockets?
0 PC 0 PDA
0 TV, VCR, DVD, 0 phone, camera
cable/satellite TV 0 smart card, card with
0 microwave, cooker, magnetic strip?
washing machine 0 electronic car key
0 central heating 0 USB memory
0 security system

try your pockets and bags

can you think of more?
 text entry devices
keyboards (QWERTY et al.)
chord keyboards, phone pads
Handwriting recognition, speech recognition

Keyboards
• Most common text input device
• Allows rapid entry of text by experienced users
• Key press closes connection, causing a character code to be sent
• Usually connected by cable, but can be wireless
 layout – QWERTY
0 Standardised layout
but …
0 non-alphanumeric keys are placed differently
0 highlighted symbols needed for different scripts
0 minor differences between UK and USA keyboards

0 QWERTY arrangement not optimal for typing

– layout to prevent typewriters jamming!
0 Alternative designs allow faster typing but large social base of
QWERTY typists produces reluctance to change.

1 2 3 4 5 6 7 8 9 0
Q W E R T Y U I O P
A S D F G H J K L
Z X C V B N M , .
SPACE
 alternative keyboard layouts
Alphabetic
0 keys arranged in alphabetic order
0 not faster for trained typists
0 not faster for beginners either!

Dvorak
0 common letters under dominant fingers
0 biased towards right hand
0 common combinations of letters alternate
between hands
0 10-15% improvement in speed and
reduction in fatigue
0 But - large social base of QWERTY typists
produce market pressures not to change
 Chord keyboards
0 only a few keys - four or 5 letters typed as combination of keypresses
0 compact size
– ideal for portable applications
0 short learning time
– keypresses reflect letter shape
0 fast
– once you have trained
0 BUT - social resistance is still high
0 Used where one-handed operation is possible
 Numeric keypads
0 for entering numbers quickly:
0 calculator, PC keyboard
0 for telephones
0 not the same!! ATM like phone
 Phone Pad and T9 Entry
0 use numeric keys with
multiple presses
2–abc 6-mno
3-def 7-pqrs
4-ghi 8-tuv
5-jkl 9-wxyz
hello = 4433555[pause]555666
surprisingly fast!
0 T9 predictive entry
0 type as if single key for each letter
0 use dictionary to ‘guess’ the right word
0 hello = 43556 …
0 but 26 -> menu ‘am’ or ‘an’
 Handwriting Recognition
0 Text can be input into the computer, using a pen and a
digesting tablet
0 natural interaction

0 Technical problems:
0 capturing all useful information - stroke path, pressure, etc. in
a natural manner
0 segmenting joined up writing into individual letters
0 interpreting individual letters
0 coping with different styles of handwriting

0 Used in PDAs, and tablet computers …

… leave the keyboard on the desk!
 Speech recognition
0 Improving rapidly

0 Most successful when:

0 single user – initial training and learns peculiarities
0 limited vocabulary systems

0 Problems with
0 external noise interfering
0 imprecision of pronunciation
0 large vocabularies
0 different speakers
positioning, pointing and
drawing
 the Mouse
0 Handheld pointing device
0 very common
0 easy to use

0 Two characteristics
0 planar movement
0 buttons
(usually from 1 to 3 buttons on top, used for making a
selection, indicating an option, or to initiate drawing etc.)
 the mouse (ctd)
Mouse located on desktop
0 requires physical space
0 no arm fatigue

Relative movement only is detectable.

Movement of mouse moves screen cursor
Screen cursor oriented in (x, y) plane,
mouse movement in (x, z) plane …
… an indirect manipulation device.
0 device itself doesn’t obscure screen, is accurate and fast.
0 hand-eye coordination problems for novice users
 How does it work?
Two methods for detecting motion

0 Mechanical
0 Ball on underside of mouse turns as mouse is moved
0 Rotates orthogonal potentiometers
0 Can be used on almost any flat surface

0 Optical
0 light emitting diode on underside of mouse
0 may use special grid-like pad or just on desk
0 less susceptible to dust and dirt
0 detects fluctuating alterations in reflected light intensity to calculate
relative motion in (x, z) plane
 Even by foot …
0 some experiments with the footmouse
0 controlling mouse movement with feet …
0 not very common :-)

0 but foot controls are common elsewhere:

0 car pedals
0 sewing machine speed control
0 organ and piano pedals
 Touchpad
0 small touch sensitive tablets
0 ‘stroke’ to move mouse pointer
0 used mainly in laptop computers

0 good ‘acceleration’ settings important

0 fast stroke
0 lots of pixels per inch moved
0 initial movement to the target
0 slow stroke
0 less pixels per inch
0 for accurate positioning
Trackball and thumbwheels
Trackball
0 ball is rotated inside static housing
0 like an upsdie down mouse!
0 relative motion moves cursor
0 indirect device, fairly accurate
0 separate buttons for picking
0 very fast for gaming
0 used in some portable and notebook computers.

Thumbwheels …
0 for accurate CAD – two dials for X-Y cursor position
0 for fast scrolling – single dial on mouse
Joystick and keyboard nipple
Joystick
0 indirect
pressure of stick = velocity of movement
0 buttons for selection
on top or on front like a trigger
0 often used for computer games
aircraft controls and 3D navigation

Keyboard nipple
0 for laptop computers
0 miniature joystick in the middle of the keyboard
 Touch-sensitive screen
0 Detect the presence of finger or stylus on the screen.
0 works by interrupting matrix of light beams, capacitance changes or
ultrasonic reflections
0 direct pointing device

0 Advantages:
0 fast, and requires no specialised pointer
0 good for menu selection
0 suitable for use in hostile environment: clean and safe from damage.

0 Disadvantages:
0 finger can mark screen
0 imprecise (finger is a fairly blunt instrument!)
0 difficult to select small regions or perform accurate drawing
0 lifting arm can be tiring
 Stylus and light pen
Stylus
0 small pen-like pointer to draw directly on screen
0 may use touch sensitive surface or magnetic detection
0 used in PDA, tablets PCs and drawing tables

Light Pen
0 now rarely used
0 uses light from screen to detect location

BOTH …
0 very direct and obvious to use
0 but can obscure screen
 Digitizing tablet
0 Mouse like-device with cross hairs

0 used on special surface

- rather like stylus

0 very accurate
- used for digitizing maps
 Eyegaze
0 control interface by eye gaze direction
0 e.g. look at a menu item to select it
0 uses laser beam reflected off retina
0 … a very low power laser!
0 mainly used for evaluation (ch x)
0 potential for hands-free control
0 high accuracy requires headset
0 cheaper and lower accuracy devices available
sit under the screen like a small webcam
 Cursor keys
0 Four keys (up, down, left, right) on keyboard.
0 Very, very cheap, but slow.
0 Useful for not much more than basic motion for text-editing
tasks.
0 No standardised layout, but inverted “T”, most common
Discrete positioning controls
0 in phones, TV controls etc.
0 cursor pads or mini-joysticks
0 discrete left-right, up-down
0 mainly for menu selection
display devices
 bitmap displays
0 screen is vast number of coloured dots
resolution and colour depth
0 Resolution … used (inconsistently) for
0 number of pixels on screen (width x height)
0 e.g. SVGA 1024 x 768, PDA perhaps 240x400
0 density of pixels (in pixels or dots per inch - dpi)
0 typically between 72 and 96 dpi
0 Aspect ratio
0 ration between width and height
0 4:3 for most screens, 16:9 for wide-screen TV
0 Colour depth:
0 how many different colours for each pixel?
0 black/white or greys only
0 256 from a pallete
0 8 bits each for red/green/blue = millions of colours
 Health hints …
0 do not sit too close to the screen
0 do not use very small fonts
0 do not look at the screen for long periods without a
break
0 do not place the screen directly in front of a bright
window
0 work in well-lit surroundings

Take extra care if pregnant.

but also posture, ergonomics, stress
 Liquid crystal displays
0 Smaller, lighter, and … no radiation problems.

0 Found on PDAs, portables and notebooks,

… and increasingly on desktop and even for home TV

0 also used in dedicted displays:

digital watches, and mobile phones
 large displays
0 used for meetings, lectures, etc.
0 technology
plasma – usually wide screen
video walls – lots of small screens together
projected – RGB lights or LCD projector
– hand/body obscures screen
– may be solved by 2 projectors + clever software
back-projected
– frosted glass + projector behind
virtual reality and 3D
interaction
 positioning in 3D space
0 cockpit and virtual controls
0 steering wheels, knobs and dials … just like real!
0 the 3D mouse
0 six-degrees of movement: x, y, z + roll, pitch, yaw
0 data glove
0 fibre optics used to detect finger position
0 VR helmets
0 detect head motion and possibly eye gaze
0 whole body tracking
0 accelerometers strapped to limbs or reflective dots and video
processing
 pitch, yaw and roll
yaw

roll
pitch
 3D displays
0 desktop VR
0 ordinary screen, mouse or keyboard control
0 perspective and motion give 3D effect
0 seeing in 3D
0 use stereoscopic vision
0 VR helmets
0 screen plus shuttered specs, etc.

also see extra slides on 3D vision

 VR headsets
0 small TV screen for each eye
0 slightly different angles
0 3D effect
 VR motion sickness
0 time delay
0 move head … lag … display moves
0 conflict: head movement vs. eyes
0 depth perception
0 headset gives different stereo distance
0 but all focused in same plane
0 conflict: eye angle vs. focus
0 conflicting cues => sickness
0 helps motivate improvements in technology
physical controls, sensors
etc.
 dedicated displays
0 analogue representations:
0 dials, gauges, lights, etc.

0 digital displays:
0 small LCD screens, LED lights, etc.

0 head-up displays
0 found in aircraft cockpits
0 show most important controls
… depending on context
 Sounds
0 beeps, bongs, clonks, whistles and whirrs

0 used for error indications

0 confirmation of actions e.g. keyclick

also see chapter 10

 Touch, feel, smell
0 touch and feeling important
0 in games … vibration, force feedback
0 in simulation … feel of surgical instruments
0 called haptic devices

0 texture, smell, taste

0 current technology very limited
 BMW iDrive
0 for controlling menus
0 feel small ‘bumps’ for each item
0 makes it easier to select options by feel
0 uses haptic technology from Immersion Corp.
 physical controls
0 specialist controls needed …
0 industrial controls, consumer products, etc.

easy-clean
smooth buttons

multi-function
control
large buttons
clear dials

tiny buttons
Environment and bio-sensing
0 sensors all around us
0 car courtesy light – small switch on door
0 ultrasound detectors – security, washbasins
0 RFID security tags in shops
0 temperature, weight, location

0 … and even our own bodies …

0 iris scanners, body temperature, heart rate, galvanic
skin response, blink rate
paper: printing and scanning
 Printing
0 image made from small dots
0 allows any character set or graphic to be printed,
0 critical features:
0 resolution
0 size and spacing of the dots
0 measured in dots per inch (dpi)
0 speed
0 usually measured in pages per minute
0 cost!!
Types of dot-based printers
0 dot-matrix printers
0 use inked ribbon (like a typewriter
0 line of pins that can strike the ribbon, dotting the paper.
0 typical resolution 80-120 dpi
0 ink-jet and bubble-jet printers
0 tiny blobs of ink sent from print head to paper
0 typically 300 dpi or better .
0 laser printer
0 like photocopier: dots of electrostatic charge deposited on drum,
which picks up toner (black powder form of ink) rolled onto paper
which is then fixed with heat
0 typically 600 dpi or better.
 Printing in the workplace
0 shop tills
0 dot matrix
0 same print head used for several paper rolls
0 may also print cheques

0 thermal printers
0 special heat-sensitive paper
0 paper heated by pins makes a dot
0 poor quality, but simple & low maintenance
0 used in some fax machines
 Fonts
0 Font – the particular style of text
Courier font
Helvetica font
Palatino font
Times Roman font
0 §´  (special symbol)

0 Size of a font measured in points (1 pt about 1/72”)

(vaguely) related to its height
This is ten point Helvetica
This is twelve point
This is fourteen point
This is eighteen point
and this is twenty-four point
 Fonts (ctd)
Pitch
0 fixed-pitch – every character has the same width
e.g. Courier
0 variable-pitched – some characters wider
e.g. Times Roman – compare the ‘i’ and the “m”
Serif or Sans-serif
0 sans-serif – square-ended strokes
e.g. Helvetica
0 serif – with splayed ends (such as)
e.g. Times Roman or Palatino
 Readability of text
0 lowercase
0 easy to read shape of words
0 UPPERCASE
0 better for individual letters and non-words
e.g. flight numbers: BA793 vs. ba793

0 serif fonts
0 helps your eye on long lines of printed text
0 but sans serif often better on screen
Page Description Languages
0 Pages very complex
0 different fonts, bitmaps, lines, digitised photos, etc.

0 Can convert it all into a bitmap and send to the printer

… but often huge !

0 Alternatively Use a page description language

0 sends a description of the page can be sent,
0 instructions for curves, lines, text in different styles, etc.
0 like a programming language for printing!

0 PostScript is the most common

 Screen and page
0 WYSIWYG
0 what you see is what you get
0 aim of word processing, etc.
0 but …
0 screen: 72 dpi, landscape image
0 print: 600+ dpi, portrait
0 can try to make them similar
but never quite the same
0 so … need different designs, graphics etc, for screen
and print
 Scanners
0 Take paper and convert it into a bitmap

0 Two sorts of scanner

0 flat-bed: paper placed on a glass plate, whole page converted into
bitmap
0 hand-held: scanner passed over paper, digitising strip typically 3-4”
wide

0 Shines light at paper and note intensity of reflection

0 colour or greyscale

0 Typical resolutions from 600–2400 dpi

 Scanners (ctd)
Used in
0 desktop publishing for incorporating photographs
and other images

0 document storage and retrieval systems, doing away

with paper storage

+ special scanners for slides and photographic

negatives
Optical character recognition
0 OCR converts bitmap back into text
0 different fonts
0 create problems for simple “template matching”
algorithms
0 more complex systems segment text, decompose it into
lines and arcs, and decipher characters that way
0 page format
0 columns, pictures, headers and footers
 Paper-based interaction
0 paper usually regarded as output only

0 can be input too – OCR, scanning, etc.

0 Xerox PaperWorks
0 glyphs – small patterns of /\\//\\\
0 used to identify forms etc.
0 used with scanner and fax to control applications

0 more recently
0 papers micro printed - like wattermarks
0 identify which sheet and where you are
0 special ‘pen’ can read locations
0 know where they are writing
memory
 Short-term Memory - RAM
0 Random access memory (RAM)
0 on silicon chips
0 100 nano-second access time
0 usually volatile (lose information if power turned off)
0 data transferred at around 100 Mbytes/sec
 Long-term Memory - disks
0 magnetic disks
0 floppy disks store around 1.4 Mbytes

0 optical disks
0 use lasers to read and sometimes write
0 more robust that magnetic media
0 CD-ROM
0 DVD - for AV applications, or very large files
 Blurring boundaries
0 PDAs
0 often use RAM for their main memory

0 Flash-Memory
0 used in PDAs, cameras etc.
0 silicon based but persistent
0 plug-in USB devices for data transfer
 virtual memory
0 Problem:
0 running lots of programs + each program large
0 not enough RAM

0 Solution - Virtual memory :

0 store some programs temporarily on disk
0 makes RAM appear bigger
 Compression
0 reduce amount of storage required
0 lossless
0 recover exact text or image – e.g. GIF, ZIP
0 look for commonalities:
0 text: AAAAAAAAAABBBBBCCCCCCCC 10A5B8C
0 video: compare successive frames and store change
0 lossy
0 recover something like original – e.g. JPEG, MP3
0 exploit perception
0 JPEG: lose rapid changes and some colour
0 MP3: reduce accuracy of drowned out notes
 Storage formats - text
0 ASCII - 7-bit binary code for to each letter and character
0 UTF-8 - 8-bit encoding of 16 bit character set
0 RTF (rich text format)
- text plus formatting and layout information
0 SGML (standardized generalised markup language)
- documents regarded as structured objects
0 XML (extended markup language)
- simpler version of SGML for web applications
 Storage formats - media
0 Images:
0 many storage formats :
(PostScript, GIFF, JPEG, TIFF, PICT, etc.)
0 plus different compression techniques
(to reduce their storage requirements)

0 Audio/Video
0 again lots of formats :
(QuickTime, MPEG, WAV, etc.)
0 compression even more important
0 also ‘streaming’ formats for network delivery
processing and networks
 Finite processing speed
0 Designers tend to assume fast processors, and make interfaces
more and more complicated

0 But problems occur, because processing cannot keep up with all

the tasks it needs to do
0 cursor overshooting because system has buffered keypresses
0 icon wars - user clicks on icon, nothing happens, clicks on another,
then system responds and windows fly everywhere

0 Also problems if system is too fast - e.g. help screens may scroll
through text much too rapidly to be read
 Moore’s law
0 computers get faster and faster!

/e3/online/moores-law/
 Networked computing
Networks allow access to …
0 large memory and processing
0 other people (groupware, email)
0 shared resources – esp. the web

Issues
0 network delays – slow feedback
0 conflicts - many people update data
0 unpredictability
 The internet
0 history …
0 1969: DARPANET US DoD, 4 sites
0 1971: 23; 1984: 1000; 1989: 10000
0 common language (protocols):
0 TCP – Transmission Control protocol
0 lower level, packets (like letters) between machines
0 IP – Internet Protocol
0 reliable channel (like phone call) between programs on machines
0 email, HTTP, all build on top of these