The

Science & Art

of Metrics
CHAPTER 2
 Learning Objectives

After going through this lesson, you are expected to:

1. Quantify and qualify data;
2. Identify different types of data;
3. Decide on correct sample size; and
4. Know what to measure.
 Customer analytics largely involves turning customer
actions and attitudes into data.
Not all data is the same, and knowing what type of data
you’re dealing with guides you through what you can do with
it. When you can interpret the data that’s available to you, you
can start making better decisions about product features and
service experiences.
In this chapter, you will understand the different types of
data, and how to work with quantitative and qualitative data.
Including best practices for identifying the best metrics to
manage.
 You’ll have to put on your analytical thinking cap (the
science of metrics) and your creative thinking cap (the art of
metrics). If you aren’t a numbers person and decide to hire
someone who is, that’s okay. But this is still a useful chapter
because you need to know how to talk to your analytics
person. And you still need to know how to interpret the
numbers so you can make sound decisions
about how to improve your business.
 Quantitative Data

Quantitative data is information that’s broken down by

concrete numbers — for example, how many products a
customer places in the shopping cart (3) or how much revenue
you earn from a specific customer ($2,000).
Quantitative data falls into two categories:
✓ Discrete (countable items)
✓ Continuous (measurements)
 Adding Up Quantitative Data

You encounter a lot of numbers when quantifying

customer experience with products and services. Knowing
whether the data is discrete or continuous dictates the
method you use in your analysis and reporting.
 Discrete and Continuous data

Discrete data has finite values, or buckets. You can count

them. For example, the number of questions correct would be
discrete: There are a finite and countable number of
questions.
Other examples of discrete data are:
✓ Number of products in your catalog
✓ Number of employees you have
✓ Number of customer reviews for a specific product
 Discrete and Continuous data

Continuous data technically has an infinite number of

steps, which form a continuum. The time to find a product on
a website is continuous because it could take 31.627543
seconds. Time forms an interval from 0 to infinity.
 Discrete and Continuous data

Other examples of continuous data are:

✓ Dimensions of a specific product
✓ Miles to your retail store from a customer’s location
✓ Time for a customer to find the information he or she is looking for on
your website
✓ Days until a product ships to a customer
You can usually tell the difference between discrete and
continuous data because discrete data can’t be broken into smaller
meaningful units. You can’t have half a customer, but you can have half a
minute.
 Levels of data

Another way customer analytics data gets divided is by

the four levels of measurement. They’re levels because they
start with data that’s more limiting in the type of analysis you
can perform to the least limiting:
 Levels of data

✓ Nominal: This includes discrete data such as the name of

your company, type of car you drive, or name of a product.
Nominal means essentially “in name only”; if you have a name,
it belongs in this category. Nominal data is qualitative data.
✓ Ordinal: This includes data that has a natural ordering. The
ranking of customers by oldest to newest, the order of callers
in a queue for a call center, the order of runners finishing a
race, or more often, the choice on a rating scale, such as from
1 to 5.
 Levels of data

With ordinal data, you cannot know with certainty

whether the intervals between each value are equal. In
measuring customers’ attitudes toward their experience with
products and services, you have to rely heavily on
questionnaire data that uses rating scales. For example, on an
11-point rating scale, the difference between a 9 and a 10 is
not necessarily the same as the difference between a 6 and a
7.
 Levels of data

✓ Interval: This is data that has equally split intervals between

each value. The most common example is temperature in
degrees Fahrenheit. The difference between 29 degrees and
30 degrees is the same magnitude as the difference between
78 degrees and 79 degrees (I much prefer the latter of these
two examples).
 Levels of data

✓ Ratio: This is interval data with a natural zero point. For

example, time to find a product on a website is ratio, because
0 time is meaningful. Degrees Kelvin has a 0 point (absolute
zero). The steps in both these scales have the same degree of
magnitude.
Whenever you can establish that data is ratio, you can
make reasonable deductions, such as “customers are twice as
satisfied using a new product version compared to an old
version.”
 Levels of data

Just because customers’ average rating on one product is

a 4 and the rating on another product is a 2 doesn’t mean
customers are twice as satisfied. The first rating is definitely
twice as high, but unless the scale is both ratio, and calibrated
so the numbers correspond to customer behavior, making
such claims is risky. It’s best to simply say the rating was twice
as high.
Levels of data
 Republic of the Philippines
CAMARINES SUR POLYTECHNIC COLLEGES

VARIABLES
 Variables

A variable is a characteristic of a product or service that

varies, which can often be manipulated. For example, price,
delivery time, and color are product variables. Customer
variables can include gender, income, geography, new
customer versus existing customer, and type of industry, to
name a few. When you look at product and customer
variables, you can understand how different product
attributes attract more or less sales and how different
customers respond to different products and feature
combinations.
 Two Types of Variables

✓ Dependent variables are usually the things you care about

but can’t affect directly, such as profitability, customer
satisfaction, and customer loyalty. You can influence
dependent variables by changing the independent variables.
An example of this relationship is shown in Figure 2-2.
 Two Types of Variables

✓ Independent variables can be directly controlled or

manipulated. For example, independent variables include
price, features, advertising, and usability.
 Two Types of Variables

Often, independent variables correlate with dependent

variables, but the correlation doesn’t equal causation. Other
variables that you’re not measuring can “mediate” or be
responsible for the relationship. For example, higher sales
(dependent variable) might be attributed to a new marketing
campaign (independent variable) but the increase is actually
just due to a growing economy that’s helped all businesses
(mediator variable).
 Two Types of Variables

You can think of independent variables as the ingredients

you use to cook a stew. The soup is the dependent variable
(what you care about), but adjusting the ingredients and their
combinations is what you can control.
 Two Types of Variables
Variables often come in the form of words instead of
numbers — for example, new customer or existing customer,
male or female, high income or low income. To make analysis
of these qualitative values easier, you can code them into
dummy variables by assigning them a number (for example,
new customers get coded a 1 and existing customers get
coded a 0; men get coded as 1 and women coded as 0 [or vice
versa]).
With your variables coded as 1s and 0s, you can compute
the percentage of customers with each variable.
Two Types of Variables
 Qualitative Data

Qualitative data is data that is not easily reduced to

numbers. Qualitative data tends to answer questions
about the 'what', 'how' and 'why' of a phenomenon, rather
than questions of 'how many' or 'how much.
 Qualitative Data

Qualitative data is often helpful by itself to explain the

“why” behind low satisfaction rates, higher sales, or high
customer turnover rates.
For example, if you see customers complaining that
they don’t know what the total price of their order is in
local currency or how to change the currency in a website
shopping cart, you know what you can fix
 Republic of the Philippines
CAMARINES SUR POLYTECHNIC COLLEGES

WHAT
DATA
TO COLLECT
 What Data to Collect

✓ Descriptive: The descriptive data becomes the template for

whom you measure. It includes demographic data like gender,
age, geography, and income. It also includes self-described
attitudes and preferences toward products, categories, and
technology. From this data, you can create meaningful
segments

You can collect this data from purchases, registrations,

surveys, interviews, and contextual inquiries.
 What Data to Collect

✓ Behavioral: The behavioral data becomes the framework

for testing experiences. It is the general pattern customers
exhibit when using your products and services. It includes
making purchases, registering, browsing, and using different
devices.

For example, customers of certain product categories,

like consumer electronics or home furniture, tend to browse
products on their tablet at night and make purchases on their
desktop during the day.
 What Data to Collect

✓ Interaction: The interaction data becomes the task

scenarios that you simulate and measure during a usability
test. It includes the clicks, navigation paths, and browsing
activities found on websites and software.
The classic usability test typically focuses on this level of
granularity by simulating real interactions. You can use real-
time data from A/B testing, Google Analytics, and lab-based or
unmoderated testing to collect data for this grouping.
 What Data to Collect

✓ Attitudinal: Preference data, opinions, desirability, branding, and

sentiments are usually captured in surveys, focus groups, and
usability tests. This is where questionnaires like the SUPR-Q
(www.suprq.com), System Usability Scale (SUS;
www.measuringu.com/sus.php), or the Net Promoter Score
(www.netpromoter.com) quantifies how interactions and behaviors
affect attitudes. These attitudes will then affect some self described
descriptive attributes quantified in the descriptive grouping.
Improvements you make that affect attitudinal data, like
increased trust and loyalty, drive further buying behavior.
 Republic of the Philippines
CAMARINES SUR POLYTECHNIC COLLEGES

MANAGING
THE RIGHT
MEASURE
 Managing the Right Measure

If you can’t measure the customer experience, you

can’t manage it. Improving the customer experience starts
with measuring. But you must be sure you’re getting the
right measure (or usually measures) to manage. The right
measure(s) will:
✓ Identify problem areas
✓ Track improvements over time
✓ Be meaningful to the customer
 Managing the Right Measure

The wrong measure(s) can:

✓ Identify wrong areas of focus
✓ Miss problems all together
✓ Lead to unintended consequences
✓ Alienate customers
 Measuring Experiences

✓ Conversion rate versus number of conversions:

Conversion rates are the central metric for testing
better designs, ads, and campaign effectiveness. The
ratio of total users who purchase, register, or click
(convert) to all users who viewed the page is an
effective ratio because you can compare low traffic and
high traffic pages.
 Measuring Experiences

✓ Number of clicks versus time to destination: When

you’re trying to make a more efficient experience,
reducing the number of clicks to accomplish a goal
seems like a good way to measure. Putting all
functionality and content on one page would certainly
reduce the number of clicks, but that probably is not
what your customers have in mind.
 Measuring Experiences

✓ Call time or call satisfactorily resolved: Wonder why

those often scorned customer service agents you call to
complain to speak so quickly? If you want to reduce call time
in a customer support center, you can instruct agents to get
off the phone faster, but have you really increased service or
quality if customers have to call back?
Often a simple follow-up question sent via email can solve
this problem.
 Measuring Experiences

✓ Customer satisfaction as a bonus motivator: Many

companies pay bonuses based on achieving and
exceeding certain customer satisfaction goals.
Unfortunately and not surprisingly, this can lead
employees to improve their chances for getting the
bonuses in ways that make measures less meaningful.
 Measuring Experiences

✓ Likelihood to recommend or likelihood to repurchase:

With the popularity of the Net Promoter Score, it may seem
like word of mouth is the only measure you should care
about. But if everyone already knows about and owns your
product or visits your website, likelihood to purchase again
might be a better measure of growth.
For measuring customer loyalty, I recommend using
both repurchase rates and likelihood to recommend. This
provides a mix of both behavior and attitude.
 Measuring Experiences

✓ On-time arrival versus on-time departure: Have you ever

been on a plane that pulled away from the Jetway only for it
to sit on the tarmac waiting for mechanical issues or other
delays? You then arrive at your destination late? It’s likely
that the flight segment still counted as an on-time
departure. You can’t argue with the measure: The plane did
pull away from the Jetway on time and that does mean
something. However, that action just doesn’t mean that
much to the customer sitting in the idled plane.
Measuring is good. Knowing what to measure is
better. Finding the right measure
means taking multiple measures and seeing
which one best tracks customer
sentiments and revenue.
 THANK YOU MY
DEAR STUDENTS!
SEE YOU ON OUR
NEXT DISCUSSION on
Customer Analytics.