Testing at every

stage of the
design process
Testing at every stage of development

Understand user expectations and

impressions at every stage of the
design process
One of the most important tools in a designer’s toolbox is feedback. Design teams make
critiquing work and gathering input from colleauges and peers a regular part of their creative
process. Most designers wouldn’t dream of launching a design without first getting feedback
from their peers. In fact, according to UserTesting’s annual CX Industry Report, 77% of
designers report that they should get user feedback before launching any new or updated
design. Yet when asked how often various experiences were tested with users, designers
reported that only a few were being tested on a regular basis. This leads to less confident
design decisions, not to mention running the risk of launching designs that don’t resonate with
users. User feedback—at every stage of the design process—is a designer’s best strategy to
create with confidence.

In this guide, we’ll share real life examples of how teams rely on user feedback to guide their
decisions, and tips on how to use two common methods of remote user feedback—self-
guided tests, and user interviews—so you can start gathering insights on your next design
project.

This guide is organized with tips and examples so that you understand
how and why to test during:

Early stage exploration and discovery

Wireframing and prototyping

Development and pre-launch

Post-launch

2 © UserTesting, Inc. All rights reserved

Testing at every stage of development

Exploration and discovery

During the exploration and discovery phase, your goal is to better understand your users. This
can entail identifying and getting to know different personas, understanding high-level pain
points, or even evaluating what your competition is doing to see where your designs can fill
an unmet need or established best practice. At this stage, you don’t have a product idea or an
endpoint in mind, instead, you’re in search of clarity on areas of opportunity. For example, you
can get feedback on initial concept ideas before moving forward with any designs.

Self-guided test User interview

While it’s better to do a live interview Conduct an open-ended interview on any

for the discovery phase (it offers the topic, including:
opportunity to ask follow-up questions
or pursue new areas and topics as the • Who are you, what’s your background,
conversation evolves), if you’re short and what are your needs?
on time you can also gather high-level
information using self-guided tests. • What are the challenges you encounter
[doing activity X, in your job, etc.]?
Here’s an example of a test plan you
could create: • How do you accomplish [activity X]?

Task: Go to the URL for a brand that you

• What brands do you turn to to complete
would visit to complete [activity X].
[activity X]? What do you like about
Task: Explain why you selected that each?
brand.

Task: Tell us about the last time you Download our tips guide for
completed [activity X]. more interview tips

Task: Now show us how you did that.

3 © UserTesting, Inc. All rights reserved

Testing at every stage of development

The Autotrader team wanted to better understand the car buying experience. Notes Bradley
Miller, Senior User Experience Researcher at Autotrader.com, “There was an institutional
perspective about the kind of influence third-party automotive classifieds sites like Autotrader
have on car shoppers, and therefore, we developed products and experiences around that
perspective.”

After conducting customer interviews, the team discovered that the process of buying a car
online resembled standard digital behavior: it all starts with a search engine. “Buying a car isn’t a
new or different circumstance. It’s just another problem people have. People have questions, so
they turn to Google or another search engine to get an answer.

“Having real-time conversations enabled us to gain deeper insights that confirmed hypotheses,
dispelled others, and uncovered new truths about the consumer shopping process.”

Adds Miller, “I learned that we need to meet consumers where they are. They don’t necessarily
seek out a site like Autotrader; rather, they’re more likely to stumble into us from a search result.

“Realizing that they are starting with a search engine and could be dropped onto any page of
our site challenges the notion that car buying follows a set sequential process. This is important
because when we are designing pages, we have to ensure that each part of the experience
draws them further into the website. We can’t assume they’ve been on other webpages and have
greater context.”

‘
Read the full case study

I learned we need to meet the customer

where they are.”

- Bradley Miller, Senior User Experience

Researcher at Autotrader.com

4 © UserTesting, Inc. All rights reserved

Testing at every stage of development

Wireframing and prototyping

When you have some early sketches or designs, it’s important to get feedback to validate them
before investing resources to build them out. Does the design make sense? Is it clear and intuitive
to the user? And if you find out that something in your early designs is really problematic, you can
change it and then test again to see if it’s a better experience before proceeding to development
or production.

User feedback is also a great way to settle internal design disputes. If you find that your team
can’t agree on a specific design the fastest, easiest way to choose confidently, is to ask your
users what they think. You can even have test participants evaluate multiple designs in a single
session, enabling you to continue to iterate on your design direction until you have something
everyone can agree on—especially your users.

You can test wireframes using either remote feedback method, however, there are some factors
to consider that will help you choose the best method. Before conducting your test, ask yourself:

• Do I need to maintain control over the wireframe or prototype, or are there security con-
cerns that would prevent me from sending this to a test participant?
• Will a test participant be able to interact with and navigate the wireframe or prototype
without much guidance, even if it’s a complicated design, or lower-fidelity?

Self-guided tests are best if you know users will be able to interact with your design without much
guideance or explanation, and if you have an easy way to share your design.

Live interviews are best if you want to ensure your designs are only shared with the test
participant during the duration of the test. They’re also more effecitve f you think a test
participant may get confused about a prototype or design (you can still do this via a self-guided
test, just be sure to be very clear in your instructions and anticipate any questions or areas
needing additional explanation.

5 © UserTesting, Inc. All rights reserved

Testing at every stage of development

Self-guided test User interview

Provide a link to your design (such as Share prototypes during the live interview
from a third party design tool or hosting session to maintain control over your
solution) to get feedback about early designs while getting feedback.
design ideas.
As with interviews conducted during the
It’s common for test participants to exploration and discovery phase, the
mistake prototypes (especially high flexible and dynamic interaction gives you
fidelity ones) for live sites or apps. Make unrestricted opportunity to gain insights.
it clear in the task directions that this This can help you reimagine and redesign
is a prototype with elements that the your designs before any development
test participant can’t select. Remind begins.
them that hey should speak out loud as
they explain what they’d expect if they This is also a good option if you have more
encountered the webpage or app live. complex designs that might be confusing
without additional explanation or context.

Read our blog post for

inspiration on testing questions
during the prototyping phase

6 © UserTesting, Inc. All rights reserved

Testing at every stage of development

The largest pediatric health system in a major US metropolis wanted to update its website
and apps—hubs of critical information that helped to attract prospective patients and kept
current patients healthy and informed.

Given the great scale of the project, and inversely the size of the lean team, they
sought to rebuild the website template by template. Using this approach, they could create
optimized designs that could then be applied across all relevant webpages—for rapid change at
scale as well as consistency throughout the digital experience.

Starting with the template used on department and program pages of the website, the team
gathered feedback to figure out what was working and what wasn’t , what people like and what
needed to change. The team asked questions like, “Do we have the right information there? And
of the information that’s there, is it in the right order?’”

Using these insights, the team created a wireframe of a potential new design. They then sought
feedback on the revised wireframe as well. The digital marketing lead shares, “We cut the
wireframe up and then had people change the order of the parts according to their preferences—
sort of like having them build their optimal page. We then created a design based on the most
popular order and again asked people to assess whether this new version provided all of the
information that they needed.”

Once the winning template design was finalized, the team ran a pilot using the updated design on
two department pages. After one month, one department page saw a 27.2% increase in requests
for an appointment (compared to the next highest month in the last 12 months). And another
department page saw an astonishing 39.8% increase in appointment requests. The year-over-
year increases were even more impressive: a 58.5% and 56.8% increase respectively.

Read the full case study

7
‘ Do we have the right information
there? And of the information that is
there, is it in the right order?”
– Digital Marketing team
© UserTesting, Inc. All rights reserved
Testing at every stage of development

Development or prior to launch

Get feedback as you reach each milestone to ensure that you end up where you planned. And
if feedback indicates there’s a problem or something that requires attention, you have the
opportunity to fix these issues before proceeding to the next milestone. Studies have shown that
problems that are discovered and fixed after a product release have 100x the cost compared to
when issues are identified and addressed while in development.

It’s also worth noting that many things are happening in tandem as a product or experience is
readied for launch. This is around the time that marketing and other GTM teams are preparing
their campaigns and other promotional assets. These GTM projects also benefit from customer
feedback to ensure that the messaging and promotional activites line up product capabilities with
customer needs and preferences.

Self-guided test User interview

Share a link (plus any necessary login Conduct a live interview session and share
information) for test participants to your desktop or mobile screen to show
access your staging environment so you the new design in a staged environment.
can see how they attempt to accomplish Bonus: Give the test participant mouse and
tasks using the fully-functional design at keyboard control to give them more “free
that point. reign” as they interact with the design.

You can also share promotional assets in

. any state (from sketches to fully-designed
content or createives) to iterate on
messaging up to the time of launch.

8 © UserTesting, Inc. All rights reserved

Testing at every stage of development

While developing ClientComm, a web app that allows probation officers (POs) to exchange text
messages with their clients from their computer, tablet or mobile device, the Code for America
team ran unmoderated tests to understand the needs of the product’s two user types.

“We have users with a wide range of digital literacy and we wanted to be sure we covered
everyone’s needs,” notes Rachel Edelman, Product Designer at Code for America. “Our
customers (POs) operate in a world where fax machines, phone calls and personal meetings are
the main modes of communication. Many have legitimate fears about their ability to adopt new
technology. Being able to do usability testing and ensure we were upholding our mandate to
provide intuitive, helpful solutions was very important.”

Adds Edelman, “We just wanted to make sure that people can complete the tasks they need to do
in order to do their jobs.”

“There’s only so much we can travel. It’s amazing that I can sit in my office and set up a study and
get all the results back within 30 minutes,” notes Edelman. “That’s about the amount of time it
would take me to take a cab to the airport. It’s an amazing time saver.”

Read the full case study

9
‘ ...It’s amazing that I can sit in my office and set
up a study and get all the results back within
30 minutes...That’s about the amount of time it
would take me to take a cab to the airport. It’s
an amazing time saver.”

– Rachel Edelman, Product Designer

at Code
© UserTesting, Inc. for America
All rights reserved
Testing at every stage of development

Post-launch
Even after a new design has launched, it’s important to continue monitoring to address
problems or to keep evolving the experience to keep up with experience best practices and user
expectations. For example, your team may wonder, “Why are people dropping off at this stage in
the workflow?” You can understand why this is happening by gathering qualitative insights and
using that feedback to continuously inform your design updates.

Qualitative insights are great when paired with analytics or quantitative metrics you may be
tracking already. Together, they provide a fuller understanding of the reasons (the ‘why’) behind
user behaviors.

Self-guided test User interview

Identify areas where you want a fuller Identify areas where you want to get
understanding of how users are more insight and have a dialogue with
interacting with or perceiving products, users about those sections in particular.
experiences, or messaging—anywhere
your designs are deployed. They can Understand how well the new release
include: is meeting user needs and uncover
new wants and needs. For exmaple,
• Areas with low user adoption have users show how they’re actually
• Unexpected user flows using the design that was released and
• Areas with high user drop-off or low compare this to what you learned during
conversion Exploration and discovery.
• High performance areas, so that you
can replicate winning strategies

10 © UserTesting, Inc. All rights reserved

Testing at every stage of development

In its efforts to continually improve upon Photoshop, the perennial graphics editing software
favored by design experts and novices alike, the Adobe team set out to launch an image
extraction feature frequently requested by users.

However, usage data following the launch as well as posts on user forums revealed that
customers were confused and struggling to use the new feature. The excitement of releasing the
high priority feature wore off as the team realized they had to make some major revisions.

The Photoshop team turned to UserTesting, using the on-demand panel to connect with new
Photoshop users representing a wide variety of skills and levels of expertise. Based on the
feedback, they made a number of updates, including giving the feature a clearer name, better
interface, and menu navigation. Doing so helped them launch an improved experience that is
helpful and easy-to-use for the wider spectrum of users.

Read the full case study

11 © UserTesting, Inc. All rights reserved

Testing at every stage of development

Let user feedback be your

design inspiration
Finally, it’s worth mentioning that while the design process is often presented as
linear, in reality inspiration and challenges can crop up anywhere and at any stage—
it’s rare for a team to follow the process from start to finish.

If, in the course of the design process, you get feedback that creates the
opportunity to pivot and change direction or creates an offshoot project for another
team to pursue, these are moments to regroup and decide how to proceed.

The point is: let feedback from your users be your inspiration—your proverbial
North Star—to ensure that all the work, time, and other resources you put into
creating newer and better experiences line up with what your users want and need.

12 © UserTesting, Inc. All rights reserved

 About UserTesting
UserTesting is an on-demand human insight platform that quickly
gives companies a first-person understanding of how their target
audience behaves throughout any experience and why. To learn
more, visit www.usertesting.com.

HEADQUARTERS
14 Townsend St.
San Francisco, CA 9410
—
UserTesting.o
c m
1-8 7 1 82

© UserTesting, In.c All rights reserved