Interaction Styles

Dr. Mazhar Sadiq

 Interaction Styles
 There are different ways a user can communicate
with a computer system and a computer system
can communicate with a user. These are called
interaction styles.
 An interaction style is a collection of user interface
controls and their associated behavior. The
interaction style provides both the look
(appearance) and feel (behavior) of the user
interface components, indicating the way a user
will communicate with the system.

2
 Interaction Styles
 There are several interaction styles
from which a designer can choose. The
most common are as follow:
• Command line
• Direct manipulation
• Gestural Interaction
• WIMP

3
 Command Line
 The command line interface was the first
interactive dialog style to be commonly used. It
provides a means of directly instructing the
system, using function keys on a keyboard (F1, F8,
etc.), single characters, abbreviations, or whole-
word commands. When the user types characters
onto the screen, they appear as a line across the
screen, hence the term “command line.

4
 Command Line

Dr 5
 Command Line
 One limitation of command line UIs is that the
commands need to be remembered. Also, users
have to remember command sequences.
 Users’ frustration with command line interfaces
often occurs because of the memory and typing
demands and the low tolerance of errors shown by
this type of interface - a confusing error message is
often the consequence of a single typing mistake
in a long command.

6
 Command Line
 Command line interfaces are better for expert users
than for novices. For expert users, command
languages provide a sense of being in control. Users
learn the syntax and can express complex possibilities
rapidly, without having to read distracting prompts.
However, error rates are typically high, and training is
necessary. Error messages and online assistance are
hard to provide because of the interface (command)
concepts and syntax.

7
 Direct Manipulation
 Direct manipulation (DM) interfaces allow users
to interact directly with the UI
objects — for example, dragging a file from one
folder and dropping it into another
in Microsoft Explorer, as illustrated in Figure.

Dr 8
Direct Manipulation

9
 Direct Manipulation
 In Direct Manipulation interfaces, the keyboard
entry of commands or menu choices is replaced by
manipulating a visible set of objects and actions.
This is usually achieved by using a continuous input
device, such as a mouse, pen, or joystick or a finger
on the touch screen.
 DM interfaces exist in many application areas,
including word processing, desktop publishing,
computer-aided design (CAD), flight simulation,
virtual reality systems, and video games.

Dr 10
 Direct Manipulation
 DM interfaces are based on the idea that users
should be allowed to manipulate UIs in a way that
is analogous to the way they interact with task
objects in everyday life. In this way, they represent
a more natural and familiar
mode of interacting with the representation of the
task objects.

Dr 11
 Benefits of Direct Manipulation
 Novices can learn basic functionality quickly,
usually through a demonstration by a more
experienced user.
• Experts can work rapidly to carry out a wide range of
tasks, even defining new functions and features.
• Error messages are rarely needed.
• Users can immediately see if their actions are
furthering their goals, and, if not, they can simply
change the direction of their activity.

Dr 12
 Gestural Interaction
 Gestures and bodily postures are a natural part of
social communication and interaction between
humans.

Dr 13
 Gestural Interaction
 Gestures are movement of
– Finger(s)
– Hand
– Arm
– Foot, Head, Body
 Two basic variations:
– Touch gestures (done on a touch-sensitive surface)
– Air-based gestures (without a touch-sensitive
surface)

Dr 14
 Air-based gestures
• Uses camera recognition, sensor and computer vision
techniques
– can recognize people’s body, arm and hand gestures in a
room
– systems include Kinect

• Movements are mapped onto a variety of gaming

motions, such as swinging, bowling, hitting and
punching
• Players represented on the screen as avatars doing
same actions
www.id-book.com 15
 Gestural Interaction

Dr 16
Home entertainment

www.id-book.com 17
 WIMP
• Xerox Star first WIMP -> rise to GUIs
• Windows
– could be scrolled, stretched, overlapped, opened, closed, and moved
around the screen using the mouse

• Icons
– represented applications, objects, commands, and tools that were
opened when clicked on

• Menus
– offering lists of options that could be scrolled through and selected

• Pointing device
– a mouse controlling the cursor as a point of entry to the windows,
menus, and icons on the screen
www.id-book.com 18
 Windows
• Windows were invented to overcome physical
constraints of a computer display
– enable more information to be viewed and tasks to be
performed

• Scroll bars within windows also enable more

information to be viewed
• Multiple windows can make it difficult to find
desired one
– listing, iconising, shrinking are techniques that help

www.id-book.com 19
www.id-book.com 20
Apple’s shrinking windows

www.id-book.com 21
Safari panorama window view

www.id-book.com 22
Selecting a country from a
scrolling window

www.id-book.com 23
Is this method any better?

www.id-book.com 24
 Menu based Interaction
 Menu based interaction avoids many of the
problems associated with command line
interfaces. A menu is a set of options from which
the user must choose. Typically, the interface
displays the options as menu items or icons and
the user indicates a choice with a pointing device
or keystroke, receiving feedback that indicates
which option he or she has chosen, and the
outcome of the command being executed.

25
 Menu based Interaction
 Menu selection does not have to be visual. When
you telephone a bank, for example, it is
common to have speech interfaces, which require
you to choose between several options.
 Menus are effective because they offer cues for
user recognition rather than forcing the users to
recall the syntax of a command from memory.
 If the items are meaningful to the users, then
menu selection can be rapid and accurate.

26
 Menu based Interaction
 If the items are hard to understand or appear
similar to each other, users can become confused
and make errors. This means that if menus are to
be effective, their names or icons should be self-
explanatory.
 Menu based interaction is particularly effective
when users have little training, or are unfamiliar
with the terminology, or need help to structure
their decision-making process.

27
 Menu based Interaction
 On the other hand, the same decomposition
process can be too rigid for some users, and it may
slow the knowledgeable frequent user. With
the careful design of complex menus and
techniques such as shortcuts, menu based
interaction can become appealing even for expert
users.

28
Cascading menu

www.id-book.com 29
 Contextual menus
• Provide access to often-used commands that
make sense in the context of a current task
• Appear when the user presses the Control key
while clicking on an interface element
– e.g., clicking on a photo in a website together with holding down
the Control key results in options ‘open it in a new window,’ ‘save
it,’ or ‘copy it’

• Helps overcome some of the navigation

problems associated with cascading menus

www.id-book.com 30
Pie Menu

31
 Guidelines for designing Menu based Interaction

 Use flow of tasks and interaction to organize

menus.
 Give menu items titles that reflect their functions.
 Group items meaningfully.
 Avoid lengthy menus.
 Order menu items meaningfully.
 Use short names for menu items.
 Use consistent grammar, layout, and terminology.
 Consider the screen’s size when deciding the
number of menu items.

32
 Virtual reality
• Computer-generated graphical simulations
providing:
– “the illusion of participation in a synthetic environment
rather than external observation of such an
environment” (Gigante, 1993)

• Provide new kinds of experience, enabling users

to interact with objects and navigate in 3D space
• Create highly engaging user experiences

www.id-book.com 33
 Pros and cons
• Induces a sense of presence where someone is totally
engrossed by the experience
– “a state of consciousness, the (psychological) sense of being in
the virtual environment” (Slater and Wilbur, 1999)

• Provides different viewpoints: 1st and 3rd person

• Head-mounted displays are uncomfortable to wear, and
can cause motion sickness and disorientation

www.id-book.com 34
 Mobile
• Handheld devices intended to be used while on the
move
• Have become pervasive, increasingly used in all
aspects of everyday and working life
• Apps running on mobiles have greatly expanded, e.g.
– used in restaurants to take orders
– car rentals to check in car returns
– supermarkets for checking stock
– in the streets for multi-user gaming
– in education to support life-long learning

www.id-book.com 35
 The advent of the iPhone app
• A whole new user experience that was
designed primarily for people to enjoy
– many apps not designed for any need, want or use
but purely for idle moments to have some fun
– e.g. iBeer developed by magician Steve Sheraton
– ingenious use of the accelerometer that is inside the
phone

www.id-book.com 36
iBeer app

www.id-book.com 37
QR codes and cell phones

www.id-book.com 38
 Mobile challenges
• Smaller screens, small number of physical keys
and restricted number of controls

www.id-book.com 39
 Speech
• Where a person talks with a system that has a
spoken language application, e.g. timetable,
travel planner
• Used most for inquiring about very specific
information, e.g. flight times or to perform a
transaction, e.g. buy a ticket
• Also used by people with disabilities
– e.g. speech recognition word processors, page
scanners, web readers, home control systems

www.id-book.com 40
Have speech interfaces come of age?

www.id-book.com 41
 Get me a human operator!
• Most popular use of speech interfaces currently
is for call routing
• Caller-led speech where users state their needs
in their own words
– e.g. “I’m having problems with my voice mail”
• Idea is they are automatically forwarded to the
appropriate service
• What is your experience of speech systems?

www.id-book.com 42
 Haptic
• Tactile feedback
– applying vibration and forces to a person’s body,
using actuators that are embedded in their clothing or
a device they are carrying, such as a smartphone

• Can enrich user experience or nudge them to

correct error
• Can also be used to simulate the sense of touch
between remote people who want to
communicate
www.id-book.com 43
 Realtime vibrotactile feedback
• Provides nudges when
playing incorrectly
• Uses motion capture
• Nudges are vibrations
on arms and hands

www.id-book.com 44
 Shareable
• Shareable interfaces are designed for more than
one person to use
– provide multiple inputs and sometimes allow
simultaneous input by co-located groups
– large wall displays where people use their own pens
or gestures
– interactive tabletops where small groups interact with
information using their fingertips
– e.g. DiamondTouch, Smart Table and Surface

www.id-book.com 45
A smartboard

www.id-book.com 46
DiamondTouch Tabletop

www.id-book.com 47
 Advantages
• Provide a large interactional space that can
support flexible group working
• Can be used by multiple users
– Can point to and touch information being displayed
– Simultaneously view the interactions and have same
shared point of reference as others

• Can support more equitable participation

compared with groups using single PC

www.id-book.com 48
 Augmented and mixed reality
• Augmented reality - virtual representations are
superimposed on physical devices and objects
• Mixed reality - views of the real world are
combined with views of a virtual environment
• Many applications including medicine, games,
flying, and everyday exploring

www.id-book.com 49
 Examples
• In medicine
– virtual objects, e.g. X-rays and scans, are overlaid on part of
a patient’s body
– aid the physician’s understanding of what is being examined
or operated

• In air traffic control

– dynamic information about aircraft overlaid on a video
screen showing the real planes, etc. landing, taking off, and
taxiing
– Helps identify planes difficult to make out
www.id-book.com 50
Top Gear James May in AR
• Appears as a 3D character to act as personal tour
guide at Science Museum

www.id-book.com 51
 Robots and drones
• Four types of robot
– remote robots used in hazardous settings
– domestic robots helping around the house
– pet robots as human companions
– sociable robots that work collaboratively with humans,
and communicate and socialize with them – as if they
were our peers

www.id-book.com 52
 Advantages
• Pet robots are assumed to have therapeutic qualities,
helping to reduce stress and loneliness
• Remote robots can be controlled to investigate bombs
and other dangerous materials

www.id-book.com 53
 Drones
• Unmanned aircraft that are controlled remotely and used in a
number of contexts

– e.g. entertainment, such as carrying drinks and food to people at

festivals and parties;
– agricultural applications, such as flying them over crops and fields to
collect data that is useful to farmers
– helping to track poachers in wildlife parks in Africa

• Can fly low and and stream photos to a ground station, where
images can be stitched together into maps

• Can be used to determine the health of a crop or when it is the

best time to harvest the crop
www.id-book.com 54
Drone in work

www.id-book.com 55
 Brain-computer interfaces
• Brain–computer interfaces (BCI) provide a communication
pathway between a person’s brain waves and an external
device, such as a cursor on a screen
• Person is trained to concentrate on the task, e.g. moving
the cursor
• BCIs work through detecting changes in the neural
functioning in the brain

• BCIs apps:
– Games
– enable people who are paralysed to control robots

www.id-book.com 56
Brainball game

www.id-book.com 57