Designing Customer Surveys That Work
by Richard E. Zimmerman, Linda Steinmann and Vince Schueler
By completing 12 basic steps of survey design,
you can boost the odds of gathering useful,
timely and accurate customer information.
One of the first principles of quality management is listening to your customers.
Customer surveying is an essential listening tool that can yield information
about customer expectations, customer satisfaction and strategies for improvement.
Sound survey design is mostly common sense-it is a basic job skill you can
learn. By completing 12 basic steps of survey design, you can boost the
odds of gathering useful, timely and accurate customer information from
your surveys, whether you conduct them yourself or work with a survey research
Quality surveys are built on two types of expertise: Your business process
knowledge plus a survey specialist's technical knowledge. As a business
process participant, you are responsible for several of the steps that profoundly
affect the scope, type and quality of any survey effort. As Figure 1 illustrates,
the survey expert adds the highest value in later design stages.
This article provides step-by-step guidelines to assist you and your organization
in developing and implementing an effective survey. You need to make 12
key decisions in three areas: establishing goals and objectives, constructing
the survey instrument and finalizing survey logistics (see Figure 2).
Establishing goals and objectives
First things first-while many novice survey designers begin by writing questions
or determining sample size, these activities should occur downstream. The
most important, and often the most challenging, part of survey design is
clarifying survey intent and scope. This is where your business knowledge
is essential. Survey specialists can't read your mind. Only you know what
is most important to your business or program.
Step 1: Determine survey purpose. The most critical part of survey design
is a clearly defined statement of purpose and a well-structured view of
what you will do with your newly acquired information. Surveys are decision-making
tools. They have little value if you are not clear on the decisions your
survey will support. Completing the following phrases formalizes your intent.
Why I want to do a survey because
I intend to use the information I am seeking
The information to be gathered will enable
me to decide
I am prepared to implement change as a result
of this survey because
Your responses provide a compass, like the mission statement for your work
unit, that will keep the project focused. A clear purpose guides the remaining
survey design elements and emphasizes action based on results. One of the
worst outcomes is collecting information that is never acted upon.
Step 2: Identify who will use the results and how they communicate. Surveys
are communication tools. It is important to understand who will use the
survey results and what type of information they respond to. Avoid the common
pitfall of assuming others share your tastes in data. Although marketing
staff may be comfortable with focus group results, a technical audience
may want statistics. Leaders often want both numbers and human-interest
perspectives when making decisions. Thus, a quantitative survey paired with
selected case studies can be the most compelling for this group.
Completing the following statements helps you define the survey audience.
The users of survey results include ...
The group(s) I will provide with information
They want the information in order to ...
The way(s) they like to receive information
Step 3: Identify what specific information is needed and when. This next
level of refinement identifies and double-checks the information needed
from the survey process. Completing the following statements helps you prioritize
your information needs.
The information I really need is ...
I need this information because ...
My top-priority information needs are ...
List the criteria that will be used in making the decisions identified above.
Identify the information that will enable you to make decisions and take
Be specific. Identify every element of data that will be collected and its
added value. Avoid the tendency to gather information that is merely interesting
or might be good to know.
Develop specific research questions that need to be answered. For example,
if you are looking for information on a proposed product, identify the new
product's elements and the concerns you have about each of them. Also, consider
how your choice of survey information will protect the participants' privacy.
Once you have prioritized your information needs, determine when surveying
should begin and how often it should occur. Answering the following questions
will help you decide.
How soon will I need information from this
How often does significant change occur in
my process, my customers or their lives or in the environment in which this
How often do I need updated information to
manage my process or program?
How often do survey users need updated information?
The survey start date may be driven by important decisions you need to make,
or by how much time you are willing to devote to planning a long-term survey
Determine how often to survey by weighing the potential benefit derived
from resurveying vs. the investment in resources-both yours and your customers'.
As a rule, the time period between surveys should be short enough to give
you reliable information, yet long enough that your customers will not feel
bothered. Several factors drive survey frequency, including survey length
and complexity, changes in your customer base, changes in your service or
product delivery process, and seasonality of services or products.
Your survey planning should include approaches for retaining the data from
each survey and tracking the results to show trends over time. This emphasizes
the need for excellent data-requirements definition and database design
because comparisons over time can be tricky. Some questions to consider
include the following:
How will we track performance over time?
How will we provide staff access to survey
data while protecting confidential information?
What if our criteria requirements change over
Step 4: Determine resource requirements. Ideally, resources devoted to your
survey will be driven by the importance of the decisions you will make based
on the data. Consider the following questions when scoping your resource
What is the value of the information I am
What are the potential consequences of the
decisions I will make based on this information?
What is the cost of not having survey data?
What staff and other resources are currently
What staff and other resources do I need?
Then determine the resource requirements by identifying the staff and financial
resources that can be devoted to the project. It is important to agree about
resource issues to ensure you have the capability to deliver a survey that
meets your overall expectations. You may have internal resources, such as
trained interviewers or data analysts. If not, consider whether you are
willing to contract out for these services. Limitations on resource requirements
will shape the entire survey design process.
Step 5. Determine who should conduct the survey. Even if you have the in-house
capability to conduct a survey, sometimes it is important to use a neutral
third party. The main benefits are credibility and candor. In some cases,
survey users or respondents may believe that your organization lacks objectivity
in collecting or reporting survey results. Survey users may find a study
conducted by an independent third party more credible than one you conduct
yourself. In other cases, survey respondents may withhold information if
they are speaking to a seemingly "interested party" about sensitive
Even if you select an outside consultant, be sure to complete the survey
design steps that follow. Your answers to the remaining questions will help
the firm you select complete the work with excellence.
Constructing the survey instrument
Now you have entered the stages where it is important to tap into internal
or external survey expertise. Depending on the size and complexity of your
survey, you may need to involve a survey expert throughout the remaining
steps of the survey planning process.
Step 6: Determine who has the information you need. Once you have identified
information and presentation needs, you should consider whom to survey.
This is called selecting the "sample frame." In most cases, you
will gather data from one or more subsets of the entire universe of people
you can contact. By making an intelligent choice of whom to survey, your
results will be accurate and your survey costs will be lower. Consider the
Who has the information I need?
Is my customer base made up of two or more
discrete groups, or are all my customers fairly similar?
Who decides whether to use my products or
Do I have current addresses or phone numbers?
The key group(s) I need information from include
Shall I randomly sample from my entire customer
base or survey only certain types of customers?
It is important to understand who your sample represents in order to draw
appropriate conclusions from the data. If you wish to generalize from the
sample to the population it represents, the sample must be randomly selected.
Random selection tends to assure the most representative sample (and will
probably generate the highest credibility among users), but is not necessary
or desirable in every case. Depending on your overall customer base or the
decisions you are facing, you may want to concentrate your survey on a particular
segment rather than a random selection of your entire customer base. This
is especially true if your customer base is characterized by distinct segments
rather than homogeneity.
Step 7: Select an appropriate survey type. There are several types of survey
formats you could use to obtain the information needed:
Face-to-face interviews are often conducted
to explore issues that are not easily understood and when information cannot
be gathered from a written or telephone survey. Face-to-face interviews
also allow the interviewer to assess nonverbal responses and ask follow-up
questions. These interviews are often used to gain insight into customer
needs and requirements as well as to determine which services are most important
Telephone surveys consist of a series of questions
and answers presented in strict sequence by a trained interviewer. Some
survey organizations use computer-aided telephone interviewing software.
The strength of telephone surveys lies in their interactive nature-respondents
can ask for clarification about the questions, and surveyors can quickly
spot any problems with survey administration.
Written surveys are the most commonly used
survey type because of their lower cost and labor requirements. A written
survey allows the customer more time to respond, but there is no opportunity
to clarify intent. Because no interviewer is available to coach the respondent
through the questions, it is very important for questions to be ordered
properly and for clear directions to be given.
Focus groups are small-group conversations
moderated by a neutral facilitator. They can provide valuable in-depth,
detailed reactions to new concepts and ideas. Questions tend to be open-ended.
The results are largely qualitative and cannot be generalized to the entire
population unless extreme care is taken when recruiting respondents. One
use of focus groups is to provide a quick and fairly inexpensive way to
identify the issues to be addressed in a subsequent mail-out or telephone
Key informant interviews are face-to-face
or telephone interviews with leaders in a field whose responses can be assumed
to apply generally to others in that field. Open-ended or quantitative questions
can be used.
The survey method you select depends largely on the time available, customer
characteristics, the information you wish to obtain and the costs you are
willing to incur. Consider the following questions when choosing a survey
Survey type decision:
What format will get the needed information
at the lowest cost and effort, and within the time frame I need?
How will limitations of survey types impact
my ability to make key decisions?
See Figure 3 for a comparison of data-collection methods.
Mail surveys can suffer from low response rates. Ways to boost survey returns
Reducing respondent barriers to survey return
through such means as short surveys (faster to fill out) and postage-paid,
preaddressed mailers (easier to mail).
Using postcard or telephone reminders to follow
up with respondents who have not returned their surveys.
Providing incentives, such as gifts or chances
to win prizes, for returning surveys.
In most cases, surveys personally and courteously handed to customers during
the service-delivery process have higher response rates. Surveys available
for pick-up at the service-delivery location are often only completed by
customers who have had a very favorable or a very unfavorable experience.
This, of course, creates a different type of bias.
Telephone surveys provide the fastest turnaround and allow you to troubleshoot
problems not detected in the pretest. Telephone surveys are best used when
all segments of the survey population are known to have telephones and should
be conducted during the time of day when respondents will be available to
talk, which may be after 5:00 p.m.
Step 8: Design the survey questions. Survey results will be meaningless
unless you ask the right questions. Ideally they are based on the steps
outlined above and on the survey and analysis approaches employed. Consider
the following when designing questions:
How is this question relevant to the purpose
of the survey?
Is the question clearly worded?
Will the question be easily understood by
The question must contain enough specifics so the respondent can give a
meaningful answer. Compare, for example, the following examples: How would
you rate our service? and How would you rate the timeliness of our computer-repair
services? Answers to the former will be difficult to act upon, while answers
to the latter tell you clearly what, if anything, needs to change.
Types of questions: Two types of questions are generally asked on written
surveys-open questions and closed questions. Open questions allow the customer
to respond to a question in his or her own words. They can be a rich source
of information, especially when you may not know all the possible answers
that respondents might choose. However, you must consider how you will analyze
the results of open questions, for you will need a coding system to quantify
Closed questions offer the customer a choice of specific responses from
which to select. Multiple choice, rating scale and yes/no questions are
examples of closed questions.
The questions you ask your customers must be properly worded in order to
achieve good end results. Try to begin all of your questions with how, what,
when, where, why or do. Avoid the following:
Leading questions-They inject interviewer
Compound questions-They may generate a partial
Judging questions-They can lead to guarded
or partial responses.
Ambiguous or vague questions-They produce
Acronyms and jargon-These may be unknown to
Double negatives-They may create misunderstanding.
Long surveys-They discourage respondent participation.
Step 9: Design the data analysis and reporting format. As a final check
on the instrument, build the tables and charts, and fill them with test
data. Ask the question, "If these tables and charts are filled in,
will I be able to make the decisions I've planned?" Other useful questions
to address include:
What type of analysis will I need to perform?
How simple or sophisticated should it be?
Who will analyze the data? How long will it
For automated analysis, what format must the
survey results use?
You may need to iterate between steps 8 and 9 to ensure you will have the
data needed to meet your survey needs.
Finalize survey logistics
Step 10: Determine sample size and selection. Sample size is driven by many
factors, including: the survey type you have chosen, the complexity and
relative homogeneity of your target audience, and the margin of error you
and your users are willing to tolerate in this survey. The selection of
the sample size grows in importance as the audience size and diversity increase,
and as the importance of the decisions made from this information increases.
Sometimes users of the survey information will want you to assure them that
you have sampled adequately from certain groups they care about. Completing
the following phrases will clarify your sample size.
How many completed surveys do I need?
How many people must be contacted in order
to get the needed number of completed surveys?
How will we reach them?
Large samples are not necessary for many surveys. A sample of 40 to 50 surveys
is often quite adequate, and 100 suffices in most cases. At this point in
the process you should consult a survey expert to assure that your approach
is well-grounded. You may need to iterate between steps 6 and 10 to ensure
you are reaching the right number of desired respondents.
Step 11: Determine data-entry methods. Once survey data are collected, ensure
that the data are accurately recorded so they are ready for analysis. Questions
to ask include the following:
Who will perform the data entry?
How much time will it take?
Will we use a spreadsheet program, a database
program or a survey-analysis software program?
How will we assure data quality and accuracy?
Specific design tools are available to increase the ease and accuracy of
data entry. Consult a survey specialist for ideas.
Step 12: Pretesting the survey. Before distributing a survey to your customers,
always pretest the survey. No matter how confident you are in the survey
design, pretesting will help you:
Determine whether your instructions are understood.
Identify questions that may be misunderstood
or that are poorly worded.
Determine whether rating scales are understood.
Determine how long it will take your customers
to complete the survey.
Determine the customers' overall level of
interest in completing the survey.
One method to pretest your survey is through a pilot group process. Pilot
group participants should be representative members of your survey's target
audience. The pretest should be administered in person, with participants
understanding their role.
You will want to look for questions not answered, several responses given
for the same question and comments written in the margins. All are signals
that the question may not have been understood and may need revision.
You should conduct follow-up group discussions with your survey team to
gain additional insight into the strengths and weaknesses of your survey
product and to identify suggested improvements.
Do your homework
Survey planning can be completed quickly and without a lot of formality.
Doing your homework will ensure the most useful survey results at the lowest
cost. Survey goals and objectives are the key to the entire effort. Writing
the questions is a very important and visible element in your survey.
To have a successful survey, you should concentrate on the planning and
design, and let the steps we've offered lead you to the necessary questions
and format. You can bring in and collaborate with experts as you need them
to help you produce a quality product.
Peter Rosik of Research Services Associates in Gig Harbor, Washington.
About the authors
Richard E. Zimmerman is director, management consulting, in the Manufacturing
Services Group of the Pulp, Paper and Packaging sector of Weyerhaeuser.
For 1995­p;1996, he was director of the Washington Performance Partnership,
an agency-level function targeted at improving management practices.
Linda Steinmann is a performance budgeting specialist at the Washington
State Office of Financial Management. She has 11 years' experience in the
legislative and executive branches of Washington state government.
Vince Schueler is an independent consultant specializing in customer research,
program evaluation and performance measurement in Olympia, Washington. He
has more than 15 years' experience conducting and using mail, telephone
and key informant surveys in the public and private sector.