Human Computer Interaction

Other Evaluation Types

 Common Evaluation Methods
• Usability testing: involves recording typical
users' performance on typical tasks in
controlled settings
• Field studies: done in natural settings, the
aim is to understand how users naturally
interact with the technology and how it
affects them
• Analytical evaluation: inspections,
theoretically based models, typically using
consultants and experts
Analytical Evaluation
 Analytical Evaluation
• An evaluation approach where users are not directly involved.
Include various inspection methods

• Inspection methods typically involve an expert role-playing

the users for whom the product is designed for.
– The experts analyze aspects of an interface and identify any potential
usability problems using a set of guidelines
– The most well known are heuristic evaluations and cognitive
walkthroughs
INSPECTION METHODS EXAMPLE 1

Heuristic Evaluation
 Heuristic Evaluation
• Developed by Jacob Nielsen in the early 1990s.
• Based on heuristics distilled from an empirical
analysis of 249 usability problems.
• These heuristics have been revised for current
technology by Nielsen and others for:
– mobile devices,
– wearables,
– virtual worlds, etc.

• Design guidelines form a basis for developing

heuristics.
 Revised version (2014) of Nielsen’s
original heuristics
• Visibility of system status.
• Match between system and real world.
• User control and freedom.
• Consistency and standards.
• Error prevention.
• Recognition rather than recall.
• Flexibility and efficiency of use.
• Aesthetic and minimalist design.
• Help users recognize, diagnose, recover from errors.
• Help and documentation.
www.id-book.com 7
 Revised version (2014) of Nielsen’s
original heuristics
• Visibility of system status
– the system should always keep users informed about what
is going on, through appropriate feedback

• Match between system and real world

– the concepts, conventions, and terminology used in
the system should match the concepts,
conventions, and terminology that users have

www.id-book.com 8
 Revised version (2014) of Nielsen’s
original heuristics
• User control and freedom
– Let user leave unwanted state
– Support undo and redo
– The dialog below is a major violation of this
principle; it gives all of the power to the
computer:

www.id-book.com 9
 Revised version (2014) of Nielsen’s
original heuristics
• Consistency and standards.
– the idea that designs should minimize how many
new concepts users have to learn to successfully
use the interface.
– Users should not have to wonder whether
different words, situations, or actions mean the
same thing
– Follow platform conventions

www.id-book.com 10
 Revised version (2014) of Nielsen’s
original heuristics
• Error prevention.
– is the idea that user interfaces, when they can, should
always prevent errors rather than giving feedback that
they occurred

• Recognition versus Recall.

– Minimize the users memory load by making objects,
actions and options visible
– The classic example of this is a command line, which
forces you to remember everything you could possibly
type, compared to a menu, which displays all of the
commands you can invoke.

www.id-book.com 11
 Revised version (2014) of Nielsen’s
original heuristics
• Flexibility and efficiency of use.
– is the idea that common tasks should be fast to
do and possible to do in many ways. Will users be
saving a lot? Add a keyboard shortcut or support
auto-save

• Aesthetic and minimalist design.

– should not contain information that is irrelevant
or rarely needed

www.id-book.com 12
 Revised version (2014) of Nielsen’s
original heuristics
• Help users recognize, diagnose, recover from
errors.
– precisely indicate the problem (using plain
language) and constructively suggest a solution

• Offer help and documentation

– any such information should be easy to search,
focused on user’s task, list concrete steps to be
carried out, and not be too large

www.id-book.com 13
 Heuristic Evaluation with
examples – Activity 1
• 10 Nielsen Heuristic Evaluation examples
– https://www.youtube.com/watch?v=sa5lswRd7Q8
• 10 Heuristic examples in web design (with
pictures)
– https://blog.prototypr.io/10-usability-heuristics-
with-examples-4a81ada920c
 Heuristics for UI Design based on
Gestalt Principles
• Simplicity
• Proximity
• Enclosure
• Continuation
• Figure & Ground

* See slides on Topic 6 for detailed descriptions

www.id-book.com 15
Extract from the Heuristics Developed by Budd (2007)
that Emphasize Web Design Issues
• Clarity - Make the system as clear, concise, and meaningful as possible for
the intended audience.
– Write clear and meaningful labels
– Use meaningful icons.
– Only use technical language for a technical audience

• Minimize unnecessary complexity and cognitive load - Make the system as

simple as possible for users to accomplish their tasks.
– Remove unnecessary functionality, process steps, and visual clutter
– Use progressive disclosure to hide advanced features
– Break down complicated processes into multiple steps
– Prioritize using size, shape, color, alignment, and proximity.

www.id-book.com 16
Extract from the Heuristics Developed by Budd (2007)
that Emphasize Web Design Issues
• Provide users with context - Interfaces should provide users with a sense
of context in time and space.
– Provide a clear site name and purpose
– Highlight the current section in the navigation
– Provide a breadcrumb trail
– Use appropriate feedback messages
– Show number of steps in a process
– Reduce perception of latency by providing visual cues (e.g. progress indicator) or by
allowing users to complete other tasks while waiting.
• Promote a pleasurable and positive user experience - The user should be
treated with respect and the design should be aesthetically pleasing and
promote a pleasurable and rewarding experience.
– Create a pleasurable and attractive design
– Provide easily attainable goals
– Provide rewards for usage and progression.

www.id-book.com 17
 Using heuristics to evaluate ambient
displays
• Full paper
https://faculty.washingt
on.edu/garyhs/docs/ma
nkoff-CHI2003-
heuristics.pdf
• The study looked at
Nielsen’s heuristics and
modified them to suit
ambient displays

www.id-book.com 18
 3 stages for doing heuristic
evaluation
• Briefing session to tell experts what to do.

• Evaluation period of 1-2 hours in which:

– Each expert works separately;
– Take one pass to get a feel for the product;
– Take a second pass to focus on specific features.

• Debriefing session in which experts work together

to prioritize problems.

www.id-book.com 19
 Severity Ratings in Heuristic
Evaluation
• Severity ratings can be used to allocate the most
resources to fix the most serious problems and can also
provide a rough estimate of the need for additional
usability efforts.
• If the severity ratings indicate that several disastrous
usability problems remain in an interface, it will
probably be unadvisable to release it.
• But one might decide to go ahead with the release of a
system with several usability problems if they are all
judged as being cosmetic in nature.
Read more: https://www.nngroup.com/articles/how-to-rate-the-severity-of-usability-
problems/
MECQ1203: Advanced Software
2019 20
Engineering
 Problem prioritization
• To effectively understand the impact of the
usability problems, we estimate both its severity
in terms of usability principles and the ease to
solve the problem.
• Problem severity ratings were impacted by the
frequency with which the problem occurred, the
impact of the problem on the users and the
persistence of the problem—whether it is a one-
time problem that users can overcome once they
know about it or will users repeatedly be
bothered by the problem.

MECQ1203: Advanced Software

2019 21
Engineering
 Example of Severity Rating
(low to high)
1. Cosmetic issue – affects the appearance and
should be fixed only if time permits
2. Minor issue – hinders the user’s ability to
navigate and should be fixed when possible
3. Major issue – Frustrates or confuses users
and requires repair as soon possible
4. Catastrophic issue – prohibits users from
performing their given task and requires an
immediate modification
www.id-book.com 22
Example of
Nielsen
Heuristic
Evaluation
with severity
rating for
Kuching
tourism app
 Advantages and problems
• Few ethical & practical issues to consider because
users not involved.
• Can be difficult & expensive to find experts.
• Should select best experts that have knowledge of
application domain & users.

www.id-book.com 25
Heuristic Evaluation with Severity
Rating – Activity 2
• Heuristic evaluation with severity rating on
Expedia vs Travelocity
– https://medium.com/@WendyBravo/heuristic-
evaluation-of-two-travel-websites-13f830cf0111
 INSPECTION METHODS EXAMPLE 2

COGNITIVE WALKTHROUGH
 Cognitive Walkthroughs
• The fundamental idea of a walkthrough is
to think as the user would, evaluating every
step of a task in an interface for usability
problems.
– Designer presents an aspect of the design & usage
scenarios.
– Expert is told the assumptions about user population,
context of use, task details.
– One or more experts walk through the design prototype
with the scenario.
 Performing a walkthrough..
1) Select a task to evaluate (probably a frequently performed
important task that is central to the user interface's value).
– Identify every individual action a user must perform to accomplish
the task with the interface.
2) Obtain a prototype of all of the states necessary to perform
the task, showing each change.
– This could be anything from a low-fidelity paper prototype showing
each change along a series of actions, or it might be a fully-
functioning implementation.
3) Develop or obtain persona of representative users of the
system.
– You'll use these to help speculate about user knowledge and behavior.
 Performing a walkthrough..
4) For each step in the task you devised, answer the following
four questions:
– Will the user try to achieve the right effect? In other words, would the user even know
that this is the goal they should have? If not, there's a design flaw.
– Will the user notice that the correct action is available? If they wouldn't notice, you have
a design flaw.
– Will the user associate the correct action with the effect that the user is trying to
achieve? Even if they notice that the action is available, they may not know it has the
effect they want.
– If the correct action is performed, will the user see that progress is being made toward
the solution of the task? In other words, is there feedback that confirms the desired
effect has occurred? If not, they won't know they've made progress. This is a design
flaw.

• By the end of this simple procedure, you'll have found some

number of missing goals, missing affordances, gulfs of
execution, and gulfs of evaluation.
 Performing a walkthrough –
Activity 3
• Now using the 4 questions from the previous
slide, here is an example of a cognitive
walkthrough in action
– https://youtu.be/Edqjao4mmxM