SURVEY

RESEARCH

Survey Research

a.k.a. paper-pencil measures or

self-report measures
represents the dominant paradigm
for social science research in the last
30 years
advantages:
expedience in gathering data
yields interval level data

Do subjects know their own

minds?

problems:

problem of mindfulness, verbal

reports of mental processes are
suspect
problem of non-attitudes, people
tend to venture an opinion whether
they have one or not
unreliability of self-reports data:
even factual information can be
misreported

Will subjects tell you if they

know?

intentional misreporting of
information.
social desirability bias
reasons for non-disclosure
embarrassment
fear of negative sanctions
shame, public humiliation
privacy
sometimes helpful to include a
lie scale

Questionnaire Design
and Construction

phrasing of questions is
critical
avoid vague, nebulous
questions
Bad example: What do
you think about abortion?
Bad example: How many
children do you have?
questions must be clear,
unambiguous
Good example: Which of
the following statements
best represents your
attitude toward elective
abortions in the first
trimester?
Conduct a readability
analysis

avoid lengthy questions;

keep questions short,
succinct
Bad example: Sara Palin
would be the first female
Vice President Secretary
of the United States. Do
you think her gender
wuold be a problem in
negotiating with the Arab
world, which is a maledominated culture?
Good example: Would
Hillary Clinton make a
good Supreme Court
justice?

questionnaire design and

construction--continued

avoid double-barreled
questions (compound
questions)
Bad example: Do you
favor stricter hand gun
controls and mandatory
minimum sentences for
carjackers?
avoid loaded language
(push polling)
Bad example: Dont you
think that...? Isnt it
true that...?
Bad example: Emotionally
charged words: gang
member, welfare
mother, extremist
groups, spin doctor, etc.

avoid slang, jargon,

abbreviations and
acronyms
Bad example: Should
states regulate PETA and
the ALF?
Bad example: Do you
think hip hop is wack?
avoid or minimize negative
wording
Bad example: Is the Bush
administration right in not
establishing a firm
deadline for withdrawing
from Iraq?

open-ended versus closeended questions

Schuman, Ludwig, & Krosnick (1986): 60% of

respondents selected one of four options in a
close-ended format, but only 2.4% mentioned any
of the same four responses in an open-ended
format.

open-ended: allows subjects more leeway,

flexibility

What is your primary ethnic/cultural

background?
requires a content analysis of responses

close-ended or forced-choice ties respondents

hands somewhat

easy to code the data

always include an other_____________
category

phrasing of questions is
critical

even slight variations in wording can alter

respondents answers.
occupied territory versus contested
territory
assisted suicide versus mercy killing
fetus versus unborn child
make questions concrete, come down the
ladder of abstraction
Use negatively worded questions or
statements sparingly
sometimes necessary to include
reverse-valenced items to identify a
response set

more on phrasing questions

Avoid evaluative language

Phrasing of questions should not imply approval,

disapproval
Follow-ups should not suggest surprise, liking,
disliking, etc.
Bad example: Do you think the Democratic
dominated Congress should lift the harsh
restrictions on stem cell research?
Bad example: Despite its poor track record in
crisis intervention, do you think the United
Nations should intervene in Darfur, Sudan?

respondents abilities

Respondents must be capable and willing

to answer the questions

May be unaware of topic/issue

May be unwilling to disclose
May lie, exaggerate, under-report or over-report
example: If you ask people what nonverbal cues
they leak when they lie, people may not know.
example: if you ask people what they would do in
a hostage crisis, how would they know?

additional considerations

order effects: sequencing of questions can be a

factor
halo effect: carry-over from one item to the
next
use multiple versions of questionnaire
response set: checking the same response
all the way down a questionnaire
use reverse scaling to detect this

additional considerations

gender of researcher/respondent can influence

responses
ethnicity of researcher/respondent can influence
responses
medium can influence responses
face-to-face interview, telephone survey, mail survey
anonymous versus non-anonymous questionnaire
fatigue: minimize length of surveys

Characteristics of types
of surveys
Type of survey

Obtaining
sample

Cooperation
rate

Cost per
respondent

advantages

disadvantages

in person

difficult

medium

high

interviewer
rapport,
nonverbal cues

respondent
apprehension,
expensive

telephone

easy

high

medium

Fast

Limited time,
nonrandom
sample

mall intercept

easy

medium

medium

fast, in-person

Nonrandom
sample

mail

easy

low

low

expensive

Nonrandom
sample,
respondent
errors

computerassisted

easy

high

low

automatic data
entry

Requires
computer
literacy