Review

Reviewed Work(s): Improving Survey Questions: Design and Evaluation by Floyd J. Fowler,;
Mail Surveys: Improving the Quality by Thomas W. Mangione
Review by: John P. Dickson
Source: Journal of Marketing Research, Vol. 34, No. 2 (May, 1997), pp. 296-298
Published by: American Marketing Association
Stable URL: https://www.jstor.org/stable/3151868
Accessed: 01-08-2018 18:44 UTC

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected].

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
https://about.jstor.org/terms

American Marketing Association is collaborating with JSTOR to digitize, preserve and

extend access to Journal of Marketing Research

This content downloaded from 80.32.237.5 on Wed, 01 Aug 2018 18:44:13 UTC
All use subject to https://about.jstor.org/terms
296 JOURNAL OF MARKETING RESEARCH, MAY 1997

researchers working in the area now and in the future.

future. II give
give more
more experienced
experiencedresearcher
researcherwould
wouldbenefit
benefit
to some
to some
degree
degree
the book my highest recommendation. by
by reading
readingthe
themyriad
myriad ofof
examples
examples
of of
howhow
to correct
to correct
bad bad
JOHN L. GRAHAM questions
questions and
andbybyreviewing
reviewing the
the
thorough
thoroughdiscussion
discussion
of how
of how
University of California, Irvine to
to pretest
pretestaaquestionnaire
questionnaire
effectively.
effectively.

REFERENCES Format

Alpert, Frank, Michael A. Kamins, and John L. Graham This book should prove easy to read for both experienced
(1992),
"An Examination of Reseller Buyer Attitudes Toward and
Order of
inexperienced survey researchers. For the most part, the
Brand Entry," Journal of Marketing, 56, (July), 25-37.writing style is clear and concise, which is in keeping with
Gilly, Mary C. and John L. Graham (1988), "A Macroeconomic
the goal of writing good survey questions. There are no ar-
Study of Promotion and the Consumption of Infant Formula in
eas of question design discussed that would require the read-
Developing Countries," Journal of Macromarketing, 8 (Spring),
21-32.
er to have more than a modest background in the concepts
Griffin, Abbie and John R. Hauser (1992), "The Voice of the Con-
associated with the design of questionnaires. The author
sumer," Report No. 92-106. Cambridge, MA: Marketing Sci- goes to great lengths to give many examples of bad and good
ence Institute (March). questions as well as to explain why the bad questions are
Levitt, Theodore (1986), The Marketing Imagination. New York: likely to elicit response errors. At times, the reader is almost
Simon and Schuster. overwhelmed by the sheer volume of examples, and there
Pechman, C. and S. Ratneshwar (1994), "The Effects of Anti- could be the tendency to lose sight of the main objectives be-
Smoking and Cigarette Advertising on Young Adolescents' Per- cause of the level of detail presented. Visually, the book's
ceptions of Peers Who Smoke," Journal of Consumer Research,
layout is modest. The author uses few charts, tables, or il-
21 (September), 236-251.
lustrations, which might have helped reinforce his ideas or
Petty, Richard E. and John T. Cacioppo (1996), "Addressing Dis-
turbing and Disturbed Consumer Behavior: Is It Necessary to break up the narrative. Virtually all the references cited are
Change the Way We Conduct Behavioral Science?" Journal of from the social sciences and medicine; only modest mention
Marketing Research, 33 (February), 1-8. is made of the work done in the field of business. Although
the sample questions and examples discussed are excellent,
they do not portray situations in a business context; this
IMPROVING SURVEY QUESTIONS: DESIGN AND might lessen the appeal to business students or practitioners.
EVALUATION, Floyd J. Fowler, Jr, Thousand Oaks, CA:
Chapter Review
Sage Publications, Inc., 1995, 191 pages, $37 (cloth),
$16.95 (paper). In the first chapter, the author briefly describes what he
considers to be the standards for writing reliable and valid
MAIL SURVEYS: IMPROVING THE QUALITY, Thomas
questions. His main points include the following: (1) All re-
W. Mangione, Thousand Oaks, CA: Sage Publications,
spondents must understand the questions in a consistent
Inc., 1995, 129 pages, $37 (cloth), $16.95 (paper).
way; (2) Consistent administration of questions is of great
Floyd J. Fowler, Jr. has been associated with the Center importance; (3) Respondents must understand what is being
for Survey Research, University of Massachusetts-Boston asked; (4) Respondents must have access to the information;
as a Senior Research Fellow and as a research associate of and (5) Respondents must be willing to answer the ques-
the Dartmouth Medical School. Thomas W. Mangione has tions. The remainder of the book is designed to discuss the
been associated with the JSI Research and Training Institute scientific basis for minimizing the errors associated with
at Boston as a Senior Research Scientist. Both authors re- each of the aforementioned points.
ceived their doctorates at the University of Michigan in the Chapter 2 focuses on writing questions designed to col
early 1970s in. the social sciences. Public health issues arelecta objective, factual data. The author discusses issues such
frequent subject of their individual and joint research stud-
as how to define the objective(s) of a survey as well as of in-
ies. These two books are part of a 40-volume (as of 1995) dividual questions, the difference between what respondents
series on survey research methodology published by Sage know versus remember, the importance of respondents hav-
Publications, Inc. Collectively, they have authored 4 (1 ing a shared meaning of a question, and how to deal with
jointly) of the volumes. "socially acceptable answers." The author provides many
good examples of how to reword questions to improve their
IMPROVING SURVEY QUESTIONS:
DESIGN AND EVALUATION
validity and reliability. However, some of the chapter reiter-
ates points made in chapter 1.
Purpose In chapter 3 he discusses how questions should be written
As Fowler points out, survey research error could to collect
be re-subjective information such as the feelings and per-
duced substantially by applying scientific method ceptions to the of de-respondents. The central focus of this chapter is
sign of survey questions. In his book he attempts to on delve
how toin-write questions that deal with scaled or ranked in-
to this fertile methodological area of study to enable formation.
the The author describes a few of the common rating
reader to avoid many common mistakes made by re- scales such as the Likert scale, but not the semantic differen-
searchers in designing questions for inclusion on survey tial scale. Less commonly used scales such as the Staple scale
questionnaires. To do this, he discusses topics such as "what are not mentioned at all. He discusses issues such as number
is a good question?" "wording questions," "pretesting ques- of scale points, labels, and order bias, but makes almost no
tions," and "validating your results." A novice writer of sur-mention of how to deal with "do not know" and "no opinion."
vey research questions would find this book valuable, and aHe makes the point that some of the scales are nominal, some

This content downloaded from 80.32.237.5 on Wed, 01 Aug 2018 18:44:13 UTC
All use subject to https://about.jstor.org/terms
New Books in Review 297

ordinal, and some interval but does not discuss the analytical
not as comprehensive as most marketing research textbooks
(statistical) implications of these differences. are (see Churchill 1995; Zikmund 1994), the rest of the
chapters cover most topics the reader would expect. For
In chapter 4, the author presents several rules for design-
ing a good survey instrument. There are many principles moreand experienced researchers, the book is possibly worth
secondary principles discussed that, because of the number-purchasing just to read the chapter on pretesting questions.
ing system and layout used, become a bit confusing. ThisMore is illustrations, better graphics, and a more comprehen-
a ironic, as one of the points the author is attempting to sive index would help improve the readability and useful-
make is that the layout of a questionnaire should be clearness
to of this book.
both the interviewer and respondent. There is an excellent
MAIL SURVEYS: IMPROVING THE QUALITY
summary of the principles of good question, design made at
the end of the chapter. However, one area the author neglects
Purpose
to discuss in this chapter is the problem of writing questions
This book focuses specifically on the various aspects in-
that have overlapping and/or nonexhaustive answer cate-
volved in conceiving and designing a mail survey. In partic-
gories. I have seen all too many examples of surveys that do
ular, the author discusses question design, sampling, nonre-
so. By accident, the author includes an example of a nonex-
sponse error, and questionnaire layout. What is the purpose
haustive answer (p. 41).
of such a narrowly focused book? As the author points out,
As the author states at the outset of the book, the presur-
researchers always will have a need for well-designed mail
vey evaluation of questions is the most important task a re-
surveys to complement other data collection methods such
searcher must carry out in attempting to design good ques-
as by telephone, door-to-door, or fax. Mangione set out to
tions. The ideas on this topic presented in chapter 5 are ex-
write a thorough book on when and how to do mail surveys
cellent. This is the most thorough description I have read re-
as well as how to do them well. A survey researcher who
garding how to use focus groups, intensive individual inter-
wants a quick reference for a fairly detailed discussion of
views, and field pretesting to improve the design of ques-
mail surveys would find this book valuable.
tionnaires. The author gives many excellent examples as to
how to use each of the aforementioned methods to reduce Format
the potential errors that can occur in developing question-
To a great extent, the author is correct when he says the
naires. The author includes sample "behavior coding" and
book is written as if it is a class lecture. The writing style in
"interviewer rating" forms, which can be used to evaluate
the book is clear and concise; yet, at times, the narrative
the questions being used on the questionnaire. The former
reads like class notes that have been transcribed and then
rates each question from the respondent's point of view, the
published. There are a lot of sections that are organized like
latter from the interviewer's perspective; both forms are very
an outline with several levels (e.g., 1. and then points a., b.,
interesting. This chapter is worth reading by most, if not all,
and c.). There are few charts, tables, or illustrations. This
survey researchers. However, the extensive nature of the
leads to a clear and brief, but somewhat bland, format. There
pretesting suggested by Fowler does leave the reader to
are ten short chapters in the book and an extensive list of ref-
wonder what the cost of such testing is in terms of time and
money.
erences. The references cited are selected evenly from busi-
Chapter 6 is a brief but good description of the impor-ness journals such as the Journal of Marketing Research and
other social science journals such as Public Opinion Quar-
tance of evaluating the validity of survey questions. The au-
thor discusses most of the normal approaches used to assessterly. However, almost all examples are from public sector
validity, including construct, predictive, and discriminantresearch such as studies involving health issues.
techniques. He also briefly discusses reliability issues. This
Chapter Review
chapter is short and clear and a good reminder of the need to
address the question, "Am I measuring what I intend to mea- Chapter 1 contains a discussion of how to determine when
sure?" Chapter 7 is a short summary of the main points pre- it is appropriate to conduct a mail survey as compared with
sented in the book; however, given the brevity of the book, Iother forms of surveys (e.g., telephone). The author cites
wonder if such a chapter is necessary. The book includes an nine examples of when to use mail surveys; however, he
appendix that consists of four sections. Each section pro-makes no attempt to explain or clarify why these same situ-
vides examples of how to deal with some of the most fre- ations could not be explored by telephone or some other
quently asked questions or problems, such as the "frequen- form of survey format. The author uses two classic cases of
cy with which one does something," "demographic" vari- how not to do mail surveys to point out the value of doing
ables, "open-ended" questions, and "achieving standardizedsurveys correctly. He presents eight advantages mail surveys
interviews." Because many of these issues are discussed inhave over other methods but makes no mention of what are
some of the disadvantages of mail surveys; the reader is left
the body of the text, it is not clear to me why the material
covered is not just included in the appropriate chapter. to wonder if there are any disadvantages. The chapter con-
cludes with an overview of the mail survey process.
Evaluation and Comment
Chapters 2 and 3 describe how to design good questions.
Improving Survey Questions will serve as a good refer- A relatively comprehensive yet brief discussion is given of
the types of questions frequently used in surveys. Topics
ence book for people who are interested in writing clear, ef-
range from how to design open-ended questions to how to
fective questions for use in survey research. The author pro-
vides many good examples of how questions should select be the appropriate number of response categories for rat-
worded so as to increase the reliability and validity of ques-
ing scales. The section on "guidelines to follow in con-
tionnaires. Although the chapter on measurement scales isstructing questions" is particularly well written and presents

This content downloaded from 80.32.237.5 on Wed, 01 Aug 2018 18:44:13 UTC
All use subject to https://about.jstor.org/terms
298 JOURNAL OF MARKETING RESEARCH, MAY 1997

many good
good examples
examples of of how
how to to correct
correctcommon
commonmistakes.
mistakes. ducing the coverage of topics such as sampling, which are
The discussion
discussion onon how
how toto create
create questions
questionsisistoo
toobrief
brieftotobebe not unique to mail surveys, the author could have spent more
of much
much value.
value. The
The author
author clearly
clearlystates
statesthe
thevalue
valueofofpretest-
pretest- time discussing issues such as when mail surveys are inap-
ing a questionnaire
questionnaire and and presents
presents many manygood
goodideas
ideastotoim-
im- propriate and what problems are faced when doing interna-
prove this
this aspect
aspect of
of the
the process.
process. The Thesecond
secondofofthese
thesetwotwo tional surveys by mail. The reader also would benefit from a
chapters
chapters deals
deals with
with some
some of of the
the more
moreproblematic
problematicissues
issues chapter that thoroughly discusses the various costs associat-
facing survey
survey researchers,
researchers, such
such as asitem
itemnonresponse,
nonresponse,skip skip ed with doing mail surveys. Finally, the book would benefit
instructions,
instructions, reliability
reliability problems,
problems,and andhow
howtotodesign
designques-
ques- from using more charts, graphs, and other illustrations.
tions that
that have
have multiple-answer
multiple-answer categories.
categories.This
Thischapter
chapter JOHN P. DICKSON
could be
be improved
improved by by providing
providing aafew fewmoremorespecific
specificexam-
exam- The University of Puget Sound
ples of the
the issues
issues being
being described.
described.
Chapters
Chapters 44 and and 55 present
present aa rudimentary
rudimentarydiscussion
discussionof of REFERENCES
sampling
sampling "basics"
"basics" andand "pitfalls."
"pitfalls." For Forthethemost
mostpart,
part,the
thedis-
dis- Alreck, Pamela L. and Robert B. Settle (1995), The Survey R
cussion is
is so
so limited
limited as as to
to be
be ofof little
littlereal
realvalue
valueexcept
exceptasasanan search Handbook, 2d ed. Homewood, IL: Richard D. Irwin.
overview
overview of of why
why aa researcher
researcher does doessampling,
sampling,what whatsome
someofof Churchill, Gilbert A., Jr. (1995), Marketing Research: Method-
the types
types of of random
random samples
samples are, are,howhowto todetermine
determinehow how ological Foundations, 6th ed. Fort Worth, TX: The Dryden
large a sample
sample should
should be,be, and
and howhowto todeal
dealwith
withframe
frameprob-
prob- Press.

lems. The
The clearest
clearest part
part of of these
these twotwochapters
chaptersdescribes
describesthethe Dillman, Don A. (1978), Mail and Telephone Surveys: The Total
differences
differences among
among the the types
types of of random
randomsamples
samplessuch
suchasasthe
the Design Method. Somerset, NJ: John Wiley & Sons.
Henry, Gary T. (1990 ), Practical Sampling. Thousand Oaks, CA:
stratified
stratified andand systematic
systematic sampling
samplingtechniques.
techniques.The Theauthor
author
Sage Publications, Inc.
presents
presents essentially
essentially no no mathematical
mathematicalformulae formulaeor orconcepts,
concepts,
Zikmund, William G. (1994), Exploring Marketing Research, 5th
and therefore,
therefore, the the section
section on on sample
samplesize sizedetermination
determinationisis ed. Fort Worth, TX: The Dryden Press.
minimally
minimally useful.
useful. TheThe author
author would
wouldhave havebeen
beenwise
wisetotocon-
con-
sider condensing
condensing the the two
two chapters
chaptersinto intooneoneand
andthen
thenrefer-
refer-
ring the
the reader
reader to to another
another source
sourcesuch suchasasthe
the21st
21stvolume,
volume,
THE PSYCHOLOGY OF ATTITUDES, Alice H. Eagly and
Practical
Practical Sampling
Sampling (Henry
(Henry 1990),
1990), in inthe
theapplied
appliedresearch
research Shelly Chaiken, Fort Worth, TX: Harcourt Brace
series of
of which
which thisthis book
book isis aa part.
part.
Jovanovich College Publishers, 1993, 794 pages, $70.
Chapters
Chapters 66 andand 77 discuss
discuss waysways to toavoid
avoidnonresponse
nonresponseer- er-
rors. This
This subject
subject appears
appears to to be
be one
oneof ofthe
theauthor's The impatient reader who succumbs to the temptation of
author'sstrengths.
strengths.
Topics include,
include, butbut are
are not
not limited
limitedto, to,cover skipping the preface of this book will miss the warning that
coverletters,
letters,return
return
postage,
postage, confidentiality,
confidentiality, prenotification,
prenotification,and "this long book covers a considerable range of topics, some
andincentives.
incentives.The
The
section on when to send reminder notices and second and of which encompass quite complex material." This reader
third waves of questionnaires is particularly clear. The con- also might not pick up readily on how the book is organized.
cepts and examples are described in great detail and with lu- As Eagly and Chaiken (hereinafter E&C) explain, their dis-
cidity. References cited are current and comprehensive. cussion of the psychology of attitudes follows a sequence of
Chapter 8 does an equally good job of describing the im- broad topics-the attitude concept, attitude measurement,
portance and characteristics of an aesthetically pleasing and attitude structure, the impact of attitudes on behavior, atti-
well-presented questionnaire. tude change, resistance to change, and social influence. But
Chapter 9 briefly describes some of the administrativethe format of each chapter varies in order to reflect how the
problems in editing and coding returned questionnaires. work In developed in the area-sometimes around theoretical
several places, the author refers to "data entry software" frameworks and at other times focused on specific phenom-
without giving much background on its nature, cost, and ena. This hybrid organization is an important strength of the
availability. The author does a good job of describing how book,to because it produces not only a strong conceptual un-
create scales that combine the respondents' answers to two derstanding of the attitude construct, but also a rich tempo-
or more questions. Chapter 10 is a summary of the book and ral sense of how the thinking developed in this literature.
adds little new material, with one interesting exception: The In chapter 1, E&C start traditionally with a definition.
author presents six pages of sample "time lines" for com- They posit (p. 1) that "attitude is a psychological tendency
pleting various segments of the survey research process that is expressed by evaluating a particular entity with some
without explanation. The time lines would help a researcher degree of favor or disfavor." They establish the generality of
budget time for various tasks, but they need further explana- this definition by contrasting it with conventional analyses.
tion to be clear. Attitudes are "tendencies"; not necessarily enduring dispo-
sitions, they can be learned or "unlearned" (i.e., have a bio-
Evaluation and Comment
logical basis), and they imply an evaluation that imputes
This book would prove valuable to novice survey re- some degree of goodness or badness to a focal object. E&C
searchers or researchers who are not familiar with designing maintain that the definition matches commonsense perspec-
and conducting mail surveys. The sections on reducing non- tives and has greater enduring value relative to treatments of
response error are particularly well designed and articulated. attitudes as outcomes of specific latent processes (e.g., a
However, in general, the author does not go much beyond memory-based association between an attitude object and an
what can be found in most marketing research textbooks orevaluation). They accept a tripartite framework: Attitudes
general survey research books such as The Survey Research form on the basis of antecedent cognitive, affective, and be-
Handbook (Alreck
(Alreck and
and Settle
Settle 1995)
1995) and
and Mail
Mail and
and Telephone
Telephone havioral processes and are manifested in these three cate-
Surveys: The Total Design Method (Dillman 1978). By re- gories of evaluative responses. However, they caution

This content downloaded from 80.32.237.5 on Wed, 01 Aug 2018 18:44:13 UTC
All use subject to https://about.jstor.org/terms