White Paper

Measuring and Tracking

Customer Satisfaction
You know why. We show you how.

By Michael Conklin, Chief Methodologist

MarketTools, Inc.
White Paper Measuring and Tracking Customer Satisfaction

About the Author

Michael Conklin is MarketTools’ Chief Methodologist. He is a marketing research veteran and analytical
methodology pioneer with 24 years experience. He came to MarketTools after serving for 10 years as a
Senior Vice President and Team Leader of Analytic Services for GfK Custom Research Inc. in Minneapolis.
He is a frequent speaker at the American Marketing Association's Annual Advanced Research Techniques
Forum as well as at the American Statistical Association's Joint Statistical Meetings conference . In
addition, Conklin has been published in key marketing publications, such as Marketing News and
Marketing Research Magazine, as well as numerous statistical journals. He received his education at The
Wharton School.

About Zoomerang
Zoomerang ( www.zoomerang.com ) offers an easy way for businesses to get fast answers to
important questions. With more than 100 million surveys sent and counting, Zoomerang has grown
rapidly and has been adopted worldwide. The number of subscribers using Zoomerang surveys
has doubled every year since 1999, and Zoomerang has been featured in Fortune magazine, on
NBC News, and on the NBC Today show. Zoomerang is part of the MarketTools family, a leading
provider of technology and advisory services for conducting Web-based market research.

For Subscribers only

This White Paper is provided to our Zoomerang zPro subscribers as an added benefit to the zPro survey
service. Zoomerang provides best practices and research methodology that our subscribers find useful to
increase both the quality and impact of their surveys.

2
White Paper Measuring and Tracking Customer Satisfaction

Best Practices in
Customer Satisfaction Research
How satisfied are your customers? Are they more or less pleased with your company than they
were last year? How does your customer satisfaction level compare with that of your competitors?
What impact does it have on your company’s profits?

How do you know?

Guesswork and hunches won’t cut it. You need hard data. You need to measure and monitor
customer satisfaction—consistently and regularly. You need a customer satisfaction research
strategy and program.

You can do it yourself, but there are a few things you need to consider. This paper gives you the
benefit of the experience and lessons learned by industry experts to help you design, distribute,
and analyze your customer satisfaction research. We want you to act on insight, not instinct.

Why Measure Customer Satisfaction?

Customer satisfaction is tied directly to profitability. If your customers are happy, they tend to be
loyal. And if they’re loyal they not only buy more, they refer other customers. Well-established
research by Bain & Company found that, for many companies, an increase of 5% in customer
retention can increase profits by 25% to 95%. The same study found that it costs six to seven
times more to gain a new customer than to keep an existing one.

$80

$70 • "It's six to seven times

more expensive to gain a
$60
Cost Per Customer

new customer than it is to

$50 retain an existing customer."

$40 • "A 5% increase in customer

retention can increase profits
$30 by 25% to 95%."
$20
(Source: Bain & Company study in
$10 Harvard Business Review, 2001)

$0
Existing Customer New Customer

Customer Type

Moreover, one bad experience can outweigh a whole lot of good experiences. Because of e-mail
and instant messaging, that bad experience can quickly be broadcast to dozens, hundreds, or
thousands of other customers, magnifying its impact. So if your business is doing something that
frustrates customers, you need to know right away.

It is critical to give customers the opportunity to provide feedback about their overall satisfaction
level and specific likes and dislikes. It is equally important to consistently measure and monitor that
input. Without an effective customer satisfaction research program in place, your company will be
losing business, missing opportunities, and putting itself at a competitive disadvantage.

3
White Paper Measuring and Tracking Customer Satisfaction

Customer Satisfaction Surveys:

Tips and Practical Advice
The first steps in establishing a customer satisfaction program are determining when to distribute
your survey, how many customers will be invited to respond, and how to deliver the survey to them.
We’ll take a closer look at how to formulate your survey questions in the next section.

1. When to Conduct Customer Satisfaction Research

Many managers wonder how frequently they should conduct customer satisfaction surveys. The
answer depends on the size of the customer base and the purpose of the research. There are two
key types of surveys, and they serve very different purposes:

> Transactional surveys solicit feedback directly from the product or service user about that
particular encounter. They are conducted immediately after each customer transaction.
For example, a survey may be administered after a call center experience.

> Relationship surveys collect input from people who have an ongoing relationship with the
company and have had multiple transactions. They are regularly scheduled surveys-
often quarterly. The respondents typically are responsible for deciding whether to continue
the working relationship.

Transactional surveys are sent out at the discretion of whoever has contact with customers—but
the data should be collected immediately after the interaction while the experience is still fresh in
the customer’s mind.

Relationship surveys should be spread out over the course of a year. Here are a couple of points to
keep in mind:

> If there is only one data point for each year, a single event could have a large impact on
results.
> Research in the hotel industry reveals that satisfaction ratings are consistently higher in
the spring than in the fall.
> For companies with a sufficiently large customer base—at least 10,000—relationship
surveys can be conducted on a continuous basis but reported on a monthly basis to
address the effects of seasonality or single events.

In many cases conducting both transactional and relationship surveys may be appropriate. For
example, a company may conduct transactional research for customer service purposes and
relationship research for routine checkups on the health of the business.

2. How Many Customers to Survey

For companies with a small customer base—for example,

2,000 or fewer—interview as many customers as possible. If you have 2,000 or
If your budget allows it, offer an incentive such as fewer customers,
company merchandise or a small gift certificate to
survey as many as
compensate respondents for their time.
your budget will
If your company has more than 2,000 customers and it is permit; if you have
not economically feasible to survey all of them, you can more than 2,000
survey a subset. In this case you’ll need to consider the customers, interview
following concepts relating to sample selection and a random sample.
accuracy of results:

4
White Paper Measuring and Tracking Customer Satisfaction

> Random selection. First, it is essential that a random selection of customers be contacted
to avoid introducing bias into how the customers were selected—and to help ensure that
the sample of customers is representative of the entire customer base. For example, if a
company surveyed only customers who contacted customer service, those individuals
may be very different from customers who have not contacted customer service.

> Margin of error. Next you need to establish the level of precision you want to have in the
accuracy of the results. The margin of error (also known as a confidence interval) is an
indicator of sample accuracy for random samples. It is the plus-or-minus figure that is
commonly reported with news polls, such as “plus or minus four points.” In a customer
satisfaction example, if the reported result is 80% satisfaction with a 4-point margin of
error, that means the true answer is somewhere between 76% (80 – 4) and 84% (80 + 4)
assuming the whole population of customers had been asked. The larger the sample size,
the more accurate the results (or the smaller the confidence interval).

> Confidence level. Then you need to determine how certain you want to be that the survey
results are within the margin of error. The confidence level indicates the level of certainty
that the survey results are within the confidence interval for random samples. Typically,
researchers use the 95% confidence level.

Together, the confidence level and margin of error together describe the certainty you have in the
precision of the data. For example, for a reported result of 80% satisfaction at the 95% confidence
level with a 4-point margin of error, you can say that you are 95% certain that percentage of
satisfied customers is between 76% and 84%.

The table below outlines the sample sizes needed for different customer bases at varying levels of
accuracy at the 95% confidence level.

Sample Size Needed

Size of Customer Base (95% Confidence Level)
+/-3% +/-5% +/-10
2,000 696 323 92
3,000 788 341 94
5,000 880 357 95
10,000 965 370 96
20,000 1,014 377 96
50,000 1,045 382 96
100,000 1,058 383 96

Another important consideration in determining the sample size is how much analysis will need to
be done on groups of customers. To analyze customers in particular industries or regions, the
sample size should be adjusted to at least 75 people in each group.

A number of online sample-size calculators are available that will provide the sample size neces-
sary for a particular confidence interval, or the margin of error for a particular sample size. Just
search “sample size calculator” with your search engine of choice on the Internet.

3. How to Distribute Your Survey

The best means of distributing a survey depends on whether it is a transactional or relationship
survey. A transactional survey is conducted at the point of customer contact. Depending on the
nature of the customer contact, an in-person, telephone, or online survey may be appropriate.
Relationship surveys, on the other hand, are most cost-effectively conducted online.

Online data collection offers significant advantages over other modes of interviewing customers,
and you should use it whenever possible. The advantages include:

5
White Paper Measuring and Tracking Customer Satisfaction

> Speed. The Internet offers instantaneous distribution of survey and real-time accumulation
and tabulation of results. This allows for immediate data analysis, even while the survey is
still in progress. Because customer satisfaction results are used to identify problems and
fix them, the faster responses arrive, the faster they can be addressed. In contrast, mail
surveys suffer from long lag times and low response rates, as low as 5%. Telephone
surveys take longer because of declining response rates. Refusal rates for phone
interviews have reached 60% (AC Nielsen, 2004). With the ease of answering online
surveys, they can be completed faster and a broader segment of the customer base can
be reached. Depending on a number of variables—the relationship with the survey
recipients, the length of the survey, whether a reminder is sent, and whether an incentive
is offered—response rates for online surveys can be upward of 35%. Even for online
surveys in which there is no prior relationship with recipients, response rates can be 23%
to 31% (Quirk’s Marketing Research Review, 2005).

> Candor. People are more honest when their answers are not filtered through someone on
the phone. This is essential for research on sensitive subject matter where studies indicate
people are more likely to answer questions on the Web than they are on the phone or in
personal interviews. (Quirk’s Marketing Research Review, 2003). The removal of
interviewer bias and the elimination of the wait time for an interviewer to record results
also yields more candid and complete responses to open-ended questions. This is
particularly important when customers volunteer additional information to explain their
satisfaction ratings. Such responses provide insight into what a company is doing well
and frequently provide warning signs about the health of the business relationship.

> Cost. The Internet eliminates many of the costs associated with traditional marketing
research. Online surveys avoid postage and telephone costs as well as basic materials like
paper, staples, envelopes, and printing. Because it is self-directed, there is no interviewer
cost. Finally, it’s more convenient so the cost of offering incentives can be reduced.

Online and offline methods of data collection can also be combined. If offline methods are neces-
sary for part of the customer base, the data for customers who can be reached only via mail, in
person, or by telephone can be input to an online survey tool. That way all survey results can be
captured, reviewed, and analyzed together.

How to Design an Effective

Customer Satisfaction Survey
Most businesses and organizations embrace a philosophy of continuous improvement. A well-
designed customer satisfaction survey will help measure your progress toward that goal. A quick
way to get started and ensure a successful survey design is to use Zoomerang’s Customer
Satisfaction Survey templates. If the survey will be designed from scratch, follow Zoomerang’s
recommended design principles. For an example survey demonstrating these principles, click here.

1. Ask Overall Satisfaction Early in the Survey

Ask the general satisfaction question at the start of the survey to avoid bias. This will allow
measurement of customers’ overall impressions of a company or an organization prior to prompt-
ing them to think of specific aspects of the relationship.

Consider the example of a bank’s customer satisfaction survey. The first question should be
“Overall, how satisfied are you with Bank X?” Then ask for specifics: their teller experiences, the
availability of ATMs, etc. Otherwise, the customers’ answers to the overall satisfaction question will
be influenced by their satisfaction with specific attributes of that relationship that they may not
have thought about until they were asked.

6
White Paper Measuring and Tracking Customer Satisfaction

Ask overall satisfaction first.

Then ask about satisfaction with

specific aspects of the relationship.

2. Use a 5-point Satisfaction Scale

Question scales should have descriptive labels associated with the numbers, and the top end of
the scale should mean that customers are truly “wowed.” If the ends of the scale are simply
“satisfied” and “dissatisfied,” it will not provide a sense of the intensity of customers’ happiness
with a product or service. A customer who is simply “satisfied” just hasn’t found a better deal yet.

The 5-point Satisfaction Scale

An insightful alternative to the 5-point satisfaction scale for customer service is a 5-point expecta-
tions scale. It provides clear direction and allows customers a polite way to suggest that a
company has not done a great job.

The 5-point Expectations Scale for Service

7
White Paper Measuring and Tracking Customer Satisfaction

3. Be Consistent in Your Questions

Consistency is critical in customer satisfaction research. Question scales should
be consistent within a particular questionnaire as well as over time. The key Question scales
measures of customer satisfaction, including the overall satisfaction question and
should be consistent
those asking about specific aspects of the relationship, should all use the same
scale. That way an apples-to-apples comparison can be made and it is readily within a particular
apparent where a particular part of the business may need improvement. For questionnaire as well
example, if a 5-point satisfaction scale is used for the overall satisfaction question as over time.
and 7-point scales are used for other questions about timeliness of service or
product quality, it will be challenging to quickly uncover weaknesses in the
business. If the survey reveals that 80% give a top rating in overall satisfaction on
a 5-point scale but only 62% do so for a particular attribute on a 7-point scale, how much of that
difference is a result of the difficulty of meeting a higher threshold on a larger scale?

Similarly, question content and scales should remain consistent over time. Customer satisfaction
scores are typically evaluated on a quarterly basis or at regular intervals; and if the questions and
the scales change, evaluating performance over time is compromised.

Note: Scales can have a dramatic effect on satisfaction ratings, so be wary of writing questions to
drive up the numbers—especially if results are being used for compensation purposes. Be
cautious about interpreting competitors’ claims of satisfaction scores. Know how the question was
asked before jumping to any conclusions about the health of the business.

4. Keep the Survey Short and Focused

Focus on getting a read on your customers’ relationship with you. Avoid the temptation to ask
everything you’ve ever wanted to know. The more ground you try to cover, the more likely it is that
respondents will abandon the survey. This can also result in inflated satisfaction scores because
those customers willing to spend that much time answering your survey are probably the ones with
positive feelings toward you.

An expedient way to develop a short, focused survey instrument is to rely on Zoomerang’s

Customer Satisfaction Survey templates.

If you build your own survey be sure to include all the key questions:

> Overall satisfaction

> An open-ended probing into the reason for the satisfaction rating Click here for
> Likelihood of recommendation the Customer
> An open-ended probing into the reason for the likelihood of recommendation Satisfaction
> Likelihood of repeat purchase
Survey example.
> Satisfaction with specific attributes of the product or service
> An opportunity to provide additional feedback

For an example survey demonstrating these questions, click here.

Think of the customer satisfaction survey as part of an ongoing conversation with your customers.
Rather than cramming every question you have into a single survey, do multiple surveys throughout
the year on different topics.

5. Ask Demographic or “Firmographic” Questions

Inquiring about demographics or firmographics (company or industry type and size) enables you to
analyze the data by different subgroups—such as new customers or regional customers. Consider
hypotheses about customer profiles that may affect satisfaction and include questions that capture
that data.

8
White Paper Measuring and Tracking Customer Satisfaction

Interpreting
Survey Results
When you’re analyzing customer satisfaction survey results, the most important goals are
minimizing the low scores and improving the top scores. It is important to monitor the “top
two-box” satisfaction number, which is the combined percentage of those saying they are very or
somewhat satisfied. It is essential, however, to call out to management the proportion of customers
who are dissatisfied and to reduce those percentages. Insights into how to do that are found by
learning what those who provide high ratings have to say and reviewing the results of those who
are dissatisfied with performance.

1. Trends in Satisfaction Score

Observing the top-two box percentage over time will reveal where a company is improving and
provide warning signs of what needs improvement. Percentages are a useful means of
communicating results in customer satisfaction research because they are readily understood by
most people. Moreover, they are more aligned with business objectives (increasing the percentage
of people who are satisfied and minimizing those who are dissatisfied).

Example of Trend Report in Overall Satisfaction

100%

90%
25% Top two-box = 78%
30%
35% 38%
80%

70%
Very Satisfied

60%
Somewhat Satisfied
39%
35%
50%
Neither Satisfied nor
40%
40% Dissatisfied
40%
Somewhat Dissatisfied
5%
30%
14% Very Dissatisfied
18% 8%
20% 7%
10%
9%
10% 9%
13% Bottom two-box = 15%
11% 8% 6%
0%
Q1 Q2 Q3 Q4

A succinct way to measure how well a company is succeeding

in minimizing low scores and improving top scores is to Conclusion: This company is
calculate the satisfaction differential by subtracting the bottom successfully minimizing low
two-box score—the combined percentage of those saying satisfaction scores and improving
they are very or somewhat dissatisfied—from the top two-box top satisfaction scores; the
score. A company that is successful on this metric will see this
satisfaction differential is increasing
number increasing over time. The table below illustrates that
score for the example trend report. each quarter.

Q1 Q2 Q3 Q4
Satisfaction Differential 33 44 58 63
(Top two-box – Bottom two-box)

9
White Paper Measuring and Tracking Customer Satisfaction

Competitive benchmarks for customer satisfaction can be difficult to come by because this is
typically proprietary information. Moreover, competitors’ results cannot be accurately interpreted
without knowing the specifics of question wording and the scales used. The satisfaction ratings for
the recipients of the Malcolm Baldrige Quality Award given by the U.S. Department of Commerce,
however, provide some insight into those companies that have been recognized for achievements in
quality and performance.

Malcolm Baldrige Quality Award Recipients’ Satisfaction Ratings

Company Name Category Satisfaction Rating

(Year of Award)
DynMcDermott Petroleum
Service 85%
Operations (2005)
Caterpillar Financial Services
Service 93%
Corporation (2003)
Park Place Lexus (2005) Small business 98%

Pal’s Sudden Service (2001) Small business 95.8%

Bronson Methodist Hospital (2005) Healthcare 97%

Robert Wood Johnson University
Healthcare 90%
Hospital Hamilton (2004)
Motorola Commercial, Government,
Manufacturing >88%
and Industrial Solutions Sector (2002)

2. Find Sources of Dissatisfaction

To increase the top two-box score next quarter, it is critical to understand what is driving the bottom
two-box scores. There are two easy ways to do that:

> Review the open-ended comments. Reading the volunteered comments following up on
the rationale for the satisfaction rating is essential. The remarks among the most satisfied
customers will shed light on what a company is doing well. They may also provide early
warning signs if satisfied customers voice concerns with a service, product, or customer
support experience. Dissatisfied customers will be vocal about the rationale for their
satisfaction rating and will provide actionable suggestions for how to improve.

> Cross-tabulate by satisfaction rating. A key analysis includes looking at the people who
are really happy and those who are not happy. Cross-tabulation features make compari-
sons between satisfied and unsatisfied customers fast and easy. The tendency in analyz-
ing customer satisfaction results is to focus on those company, product, or service
attributes that elicit the greatest dissatisfaction overall—price, for example. Most custom-
ers will want a lower price regardless of their satisfaction level. It is more revealing to
compare how satisfied and dissatisfied customers feel about various aspects of the
product or service relationship. The gap in satisfaction ratings for each attribute will help
prioritize areas for improvement—the areas with the greatest difference should have the
highest priority. In the example cross-tabulation table below, price has the lowest top
two-box satisfaction score among all customers. The gap between satisfied and dissatis-
fied customers, however, is biggest for customer service, so that is the area with the most
impact on overall satisfaction.

Example Cross Tabulation by Satisfaction Rating

Conclusion:
Top Two-Box Scores Improving
Attribute All Overall Satisfied Overall Dissatisfied customer service
Customers Customers Customers
should be a
Price 76% 80% 67%
Customer service 85% 95% 30% priority to
Product quality 90% 96% 64% improve overall
satisfaction.

10
White Paper Measuring and Tracking Customer Satisfaction

3. Present Findings and Action Items

Collecting customer satisfaction data is useful only if there is a process established to deliver
recommendations, implement action plans, assign plan owners, and monitor plan execution. Once
the results have been compiled and analyzed, they should be presented to management with
recommendations for resolving the identified weaknesses. The recommendations should be
actionable – reduce response times to four hours for example – and the person responsible and the
resources allocated to meet that goal should be determined. There should be periodic meetings to
evaluate progress on the action plan and the next customer satisfaction survey should evaluate
progress on that goal.

4. Contact Customers

If possible, dissatisfied customers should be personally contacted to see if there is something that
can be done to improve their perception of the business. This is important not simply to increase
the odds of keeping that customer but also to prevent negative word of mouth. The personal
contact itself may succeed in doing that. Additionally, a discount or free product should be
considered. If policies or products are changed based on customers’ feedback, those customers
should be contacted to let them know about those changes and that their feedback was taken
seriously.

Start a Customer Satisfaction

Research Program Now
Put an end to guesswork as you know it. Start a customer satisfaction research program today. Visit
http://info.zoomerang.com/zpro.htm to learn how Zoomerang zPro can help.

Additional Resources
Zoomerang recommends the following resources to learn more about customer satisfaction
research:

> Lester, Tom. “The Cost of Not Caring for Your Customers,” Financial Times,
January 30, 2006.

> McGregor, Jena. “Would You Recommend Us?” Business Week, January 30, 2006.

> Reichheld, Frederick F. Loyalty Rules! How Today’s Leaders Build Lasting Relationships
(Harvard Business School Press, 2001).

> For more information about online survey research, please visit the MarketTools Resource
Library.

11
MarketTools, Inc.
Zoomerang
150 Spear Street, Suite 600
San Francisco, CA 94105
U.S.A.
800-316-0662
sales@zoomerang.com

Copyright 2006 by MarketTools, Inc. This white paper can not be copied or distributed without
MarketTools permission. This whitepaper may be used by Zoomerang Subscribers only.