Questionnaire Design

A QUESTIONNAIRE
IS ONLY AS GOOD
AS THE QUESTIONS
IT ASKS

Purposes of the Questionnaire

A questionnaire ensures standardization and
comparability of the data across interviews,
increases speed and accuracy of recording
Facilitates data processing
Also allows the researcher to collect the
relevant information necessary to address the
management decision problem

Logical Steps to Develop a Good

Questionnaire
Recall the research objective, the research questions and
hypotheses
Identify the variables to be measured
Formulate questions (items in the questionnaire)
Order and wording of questions and the layout of the
questionnaire
Test for omissions and ambiguity
Correct the problems (pretest again, if necessary)

What Should Be Asked?

Questionnaire Relevancy
A questionnaire is relevant to the extent that all
information collected addresses a research question that
will help the decision maker address the current
business problem.

Questionnaire Accuracy
Accuracy means that the information is reliable and valid.

Validity and reliability

Validity:
question measure what you claim it
measures problem with self-report
Reliability:
results are reproducible or consistent with
similar groups of respondents, over time
and when other people administer the
questionnaire

Type of information
Knowledge - what people know
Opinions, attitudes, beliefs, values what people think about an issue
Behaviour - what people do
Attributes - what are peoples
characteristics
Remember - based on self-report

Knowledge
What is the recommended interval
between eye checks for patients with
uncomplicated diabetes?
6 months
1 year
2 years
Not sure

Attributes
When did you graduate from university?
_______________________

Opinions
What do you think are the major issues
affecting general practice in Australia at
the moment?
________________________________
Behaviour
Have you developed a care plan for any of
your patients?
Yes
No

Types of Questions
1. Open-Ended question:
People look for different things in
a job. What would you prefer most
in a job?

2. Closed-Ended question:
People look for different things in
a job. What would you prefer most
in a job?

Work
Work
Work
Work
off.

that pays well

that gives a sense of accomplishment
where you make most decisions by yourself
that is steady with little chance of being laid

Open Ended Questions

Advantages

Gain insight into the problem

Too many options to list
When verbatim responses are desired to give the flavor of the problem
When behavior to be measured is sensitive or disapproved
Interviewer / questionnaire structure influence can be minimal

Disadvantages
Inarticulate respondents
Interviewers skill in recording quickly and summarizing accurately
Time consuming, subjective judgments while tabulating, adds to cost

Closed-response Questions
Two Basic Formats for Closed Ended or
Structured Questions
Choice from a list of responses
Appropriate single-choice rating on a
scale

Closed-response Questions
What type of fast-food restaurant do you
visit most often?

Burger

Chicken
Seafood
Dont know

Mexican
Pizza
Chinese
Other (please specify)

Closed-response Questions
What is your overall satisfaction with McDonalds
burgers?
Very satisfied

Very satisfied
satisfied

Quite Satisfied
Somewhat satisfied

Not at all satisfied

Not at all

Closed-Ended Questions
Advantages

Easy to understand, quick responses possible

Micro-differences in responses can be captured
Easier tabulation and analysis
Answers are directly comparable from respondent to respondent

Disadvantages
Neutral category may attract more responses than warranted
Information between categories may be lost (extreme case
dichotomous categories)

Issues in Questionnaire Design

Respondent uncertainty
Should respondents be provided with a
Dont know or No opinion option?
When it is important to differentiate between
ambivalence and ignorance, both options
should be provided

Contents of the Questionnaire

Question Wording Vocabulary
- Depends on the nature of variable tappedsubjective feeling or objective

facts

Simple, easy to understand, commonly used

language
Avoid technical words and jargon (unless
sample is technically qualified)

Filter questions
Filter questions useful to ensure
respondents only answer relevant parts of
questions
Unfiltered:
If you use a medical software program,
which one do you use?
Filtered:
Do you use a medical software program?
No - jump to next question
Yes - which one?

. Do you believe that private citizens have the right to

own firearms to defend themselves, their families, and
property from violent criminal attack?
Yes
No
Undecided
-----------------------------------------------------------------------------. Do you believe that a ban on the private ownership
of firearms would be significantly reduce the number
of
murders and robberies in your community?
Yes

No

Undecided

Issues in Questionnaire Design

Question Wording
vocabulary
Questions need to be clear, simple and
precise
Poorly written questions lead to ambiguity
and misunderstandings and can be
wasteful

 Avoid Complexity: use simple, conversational

language
Avoid leading and loaded questions
Avoid ambiguity: be as specific as possible
Avoid double-barreled items
Avoid making assumptions
Avoid burdensome questions

Double-barrelled questions
Two concepts in one question
Have you had a neck ache or a back ache
since your last visit?
Since your last visit, have you had any of the
following symptoms (tick as many that apply):
Neck ache
Back ache
Headache

Leading and Loaded questions

A leading question suggests or implies certain
answers
Do accounting graduates who attended state universities, such
as Washington State University, make better auditors?

A loaded question suggests a socially

desirable answer or is emotionally charged

Do you believe that private citizens have the right to

own firearms to defend themselves, their families, and
property from violent criminal attack?
Yes

No

Undecided

Do you believe that a ban on the private ownership

of firearms would be significantly reduce the number
of
murders and robberies in your community?
Yes

No

Undecided

Vague questions
Avoid vague questions
Taken altogether, how happy are you with
your stay in hospital?
Overall, how would you describe the care
you received in hospital?
Avoid Ambiguity: Words such as often, occasionally,
usually, regularly, frequently, many, should be
used with caution. If these words have to be used, their
meaning should be explained properly.

Avoid Making Assumptions

Is the question applicable to all
respondents?
Why do you like fast-food?
Assumes respondents like fast foods
Better strategy would be to ask a filter
question first.

Burdensome Questions
Avoid Burdensome Questions That May
Tax the Respondents Memory telephone
A survey conducted during the 24-hour period following
the airing of the Super Bowl might establish whether the
respondent watched the Super Bowl and then ask, Do
you recall any commercials on that program?

Question Wording
Question length
Should be short
Longer questions confuse and fatigue respondents

Sensitive questions
Questions on information perceived to be
embarrassing, like personal income, criminal
activities, alcoholism, smoking, drugs habits, social
desirability issues, etc.
Creativity rules (assurances of confidentiality,
anonymity, slipping it in sideways, open-ended
questions, asking in third person, etc.)

Sequence And Layout

Decisions
Opening questions easy and non-threatening
Flow smooth and logical avoid jumps
Broad to specific
Critical questions placed in the middle
Appealing and interesting
Order bias the possibility that subsequent responses
are influenced by preceding responses e.g. fewer people
will say that their taxes are too high after being asked
whether govt. spending should be increased in certain
areas.
Demographic questions first or last?

Layout

Just as important as wording

Aim for a professional look
Tips:
cover letter/introductory page giving study title,

organisation, aims of the survey

enough space for open-ended questions
font large enough to read without strain
consistent and clear instructions
dont split questions or answers across pages
enough white space

Distinguishing question and

responses:
LAYOUT A:
Do you agree, disagree or have no opinion that this company
has:
A good vacation policy - disagree/not sure/agree.
Good management feedback - disagree/not sure/agree.
Good medical insurance - disagree/not sure/agree.
LAYOUT B:
Does this company have:

Disagree Not Sure Agree

A good vacation policy
1
2
3
Good feedback
1
Good medical insurance

2
1

3
2

Pretesting and Correcting

Problems
Purpose

of

pretest:

To ensure that the

questionnaire meets the expectations in terms of
the information that will be obtained
Pretesting Specific Questions For
Meaning
Task difficulty
Respondent interest and attention

Pretesting the overall Questionnaire

Flow of the questionnaire
Skip patterns
Length

Put yourself in the respondents shoes and answer

the questionnaire.