Welcome to Powerpoint slides

for
Chapter 4
Questionnaire Design:
A Customer-centric
Approach
Marketing Research
Text and Cases
by
Rajendra Nargundkar
1

Questionnaire design, to be effective, should be done with

the respondent in mind.
Language
The first and foremost question we have to ask ourselves
as a researcher is
What language is the respondent going to understand and
respond in?
The questionnaire must be designed such that it can be
used in the language concerned.
This does not
necessarily mean it has to be printed in each language in
which it has to be administered.
2

For instance, a questionnaire printed in English

could be administered to the respondent in the
local language he speaks, by a trained interviewer
who could translate each question on-line. The
answers can be recorded in the given English
language form if the interviewer is fluent in both
languages. This makes it easier to tabulate.
Alternatively, the numerical codes for the answers
can be in usual numbers, and the questionnaire
could be translated into any language required for
the respondent to understand. But the translation
must be as consistent as possible with the original.
3

Difficulty Level
Avoid marketing jargon or difficult words unless the
respondent is a postgraduate or an experienced
executive. In other words, keep the language as simple
and straightforward as possible.
Fatigue
Avoid unnecessary questions. The golden rule is to keep
the questionnaire as short as possible, and the ideal
maximum interview time is probably about 20 minutes
per interview.
4

Cooperation with Researcher

Encourage the respondent to respond.
In personal interviews, introduce the
subject of the research and the agency
represented, before starting the interview.

In questionnaires which are filled by

respondents themselves, there must be a twothree line introduction and request for
respondents cooperation at the top of the
questionnaire.
In mailed questionnaires, a covering letter
detailing the purpose of the study and
explaining what use its results will be put to,
along with a return pre-paid/stamped
envelope, is likely to increase manifold the
response rate.
6

Social Desirability Bias

There is a tendency on the part of respondents to
give wrong, but socially acceptable answers to
even the most ordinary, innocuous questions. For
example, the socially desirable answer to the
question Do you read the daily newspaper? is
yes. It is as likely to be wrong as right.

There are many ways to verify the accuracy

of responses and to deal with them. Some of
the techniques are
Repeating the same or similar question
in the questionnaire at different places.
Asking indirect questions
Asking follow up questions to probe if
the respondent is really truthful.

For example, we could ask the respondent to

state one important headline, or describe one
important story he remembers, if he states that
he reads the daily newspaper. This could be
from the same days or previous days,
newspaper.
.Deliberately
introducing
non-existent
periodicals, or advertisements, and asking the
respondent if he/she has seen them.
9

Ease of Recording
A questionnaire, that it has to be carried on the field, and
data may be recorded on it while standing in awkward
postures. The questionnaire design should ensure it is
easy to carry, visible in different kinds of light, and the
distance between different answer categories should be
sufficient so that there is no confusion or mistake while
placing a tick over the actual response for a given
question.

10

Coding
If the questionnaire is coded before doing the field work
(as most questionnaires are these days), it must be
ensured that the field staff knows where to mark the
answers on the code or on the actual answer choice.
This should be done during the briefing and mock
interview.
Instructions for Navigation
Frequently, a questionnaire contains printed instructions
for the interviewer. This includes Go To statements,
such as If respondent is a non-user of Brand X, then
Go To Q.5. If not, Go To Q.9.
11

Sequencing of Questions
Questions in a questionnaire should appear in a sequence
starting from non-threatening or ice-breaking or
introductory questions, and then proceed to the main body
of questions.
Generally, the age, income, occupation, education and
similar demographic questions should appear at the end of
a questionnaire, after an interviewer has established a
rapport or familiarity with the respondent. If these are
asked in the beginning, there is a high likelihood of
suspicion and non-cooperation resulting in a wasted effort
in many cases.
As far as possible, questions should follow a logical
sequence, and must be phrased appropriately.
12

Biased and Leading Questions

The questions should be carefully worded to
avoid bias. It is not a good practice to ask
questions such as Dont you think
liberalisation is a good idea? You could be
better off getting an unbiased reply asking a
question like Some people think
liberalisation is a good thing, and some think
it is bad. What do you think?
13

Monotony
One indicator that a questionnaire is monotonous
for the respondent is if he answers Agree to every
question or Disagree to every question, for four to
five questions in a row.
If this happens, the researcher must find a way to
overcome the potential problem, by re-sequencing
the questions which force the respondent to think
before he answers, or by changing the scale, or by
some other method.
14

Analysis Required
A questionnaire design is dependent on the
analysis required from it. But the most
important effect of the analysis required is in
the scale of measurement that must be used.
So we will deal with this topic the scale of
measurement next.

15

Scales of Measurement Used in Marketing

Research
Marketing research uses the following four major
types of scales Nominal, Ordinal, Interval and Ratio.

16

Nominal Scale
A nominal scale uses numbers as labels, with
no numerical sanctity. For example, if we
want to categorise male and female
respondents, we could use a nominal scale of 1
for male and 2 for female.
But 1 and 2 in this case do not represent any
order or distance. They are simply used as
labels. For instance, we could easily label
females as 1 and males as 2, and it could
still be a valid nominal scale.
17

We can use the nominal scale to indicate

categories of any variable which is not to be
given a numerical significance. For example,
demographic variables such as religion,
education level, languages spoken, and other
variables like magazines read, T.V. shows
watched, user or non-user of a brand, brands
bought, etc. can be nominally scaled.

18

Nominally scaled variables cannot be used to

perform many of the statistical computations such
as mean, standard deviation etc., because such
statistics do not have any meaning when used with
nominal scale variables.

19

However, counting of number of responses in

each category and computation of percentages
after division by the sample size is allowed.
Also, nominal scale variables can be used to do
cross tabulations, one of the most popular
methods of routine analysis. The chi-squared
test can be performed on a cross tabulation of
nominal scale data.
To repeat, simple tabulations (also called
frequency tables) and cross tabulations can be
done with nominal scale variables.
20

Ordinal Scale
Ordinal scale variables are ones which have a
meaningful order to them. A typical marketing
variable is ranks given to brands by respondents.
These ranks are not interchangeable, as nominal
scale labels are. This is because rank 1 means it is
ranked higher than rank 2. Similarly, rank 2 is
higher than rank 3, and so on.

21

Instead of 1, 2 and 3, however, we could use

any other numbers which preserve the same
order. For example, 3, 10, 15 could denote the
same ranking order instead of 1, 2 and 3. This is
because we do not know for sure what the
distance between 1 and 2 is, or what the distance
between 2 and 3 is. Ranking simply denotes that
1 is higher than 2, and 2 higher than 3, but
higher by how much is unknown. For one
respondent, 1 and 2 may be close together; for
another, they could be far from each other.

22

The statistics which can be used with the

ordinal scale are the median, various
percentiles such as the quartile, and the
(Spearman) Rank Correlation. This is in
addition to the frequency tables and cross
tabulations, which can also be used.
Arithmetic mean (or average) should not be
used on the ordinal scale variables. For
example, the average rank of a set of rankings
does not have any meaning. Even though
weighted indexes are calculated in practice
from rank order data, it is, strictly speaking,
not allowed.
23

Interval Scale
An interval scale variable can be used to compute the
commonly used statistical measures such as the average
(arithmetic mean), standard deviation, and the Pearson
Correlation coefficient. Many other advanced statistical
tests and techniques also require interval-scaled or ratioscaled data.
Most of the behavioural measurement scales used to
measure attitudes of respondents on a scale of 1 to 5 or 1
to 7 or 1 to 10 can be treated as interval scales. These
types of scales, also known as Rating Scales, are very
commonly used in marketing research.
24

If a consumer is asked for his satisfaction level

with a product or service or any other attribute
related to it, on a scale of 1 to 10, it is an intervalscaled rating. We could use it to compute the
average rating given by all respondents in the
sample. Standard deviation can also be computed.
The difference between interval scale and ordinal
scale variables is that the distance between 1 and 2
is the same as the distance between 2 and 3, and 3
and 4 in an interval scale. That is, the difference
between two successive numerical measures is
fixed, whereas in rank-ordered data, it is not fixed.
25

Ratio Scale
All arithmetic operations are possible on a ratio-scaled
variable. These include computation of geometric
mean, harmonic mean, and all other statistics like the
average, standard deviation and Person Correlation,
and also the tests such as the t test and the F test.
In a ratio type scale, there is a
beginning point. An interval scale
unique zero (It is an arbitrary zero).
two values of the scale corresponds
among the measured values.

unique zero or
does not have a
Also, the ratio of
to the same ratio
26

For example, distance is a ratio scaled variable.

It has a zero which is unique. 2 metres is to 1
metre as 2 kilometres is to 1 kilometre. Also, 4
metres to 1 metre, and 30 metres to 7.5 metres.
The ratios can be measured at any two points,
and they would correctly denote the
relationship.
Not many ratio-scaled variables exist in
marketing. Some of them are length, height,
weight, age (in years) and income (measured in
rupees, not as an income category).
27

Structured and Unstructured Questionnaires

Structured questionnaires are those where the
questions to be asked are standardised, and no
variation is permitted in terms of the wording
of the questions between different interviewers.
Standardisation in a structured questionnaire
usually extends to the answers also. In effect,
then, we can standardise either (1) questions
only, or (2) both questions and answers.

28

Structured Questions
Structured questions improve the reliability of
the study, by ensuring that every respondent is
asked the same question, word for word.
For example, the question " Do you live in
Delhi?" may be construed differently from the
question " Are you a resident of Delhi?" by
some respondents, even though it appears that
both questions are asking for the same
information.
29

A person who is normally not resident in Delhi

but is living there at present on a short visit
may answer "yes" to the first question but
"no" to the second one. It is best to keep the
question exactly the same (either version 1 or
version 2), when asked by different
interviewers.

30

Structured Answers
Structuring or standardising answers which a
respondent can choose from in a questionnaire
also
achieves
consistency
of
form.
Additionally, it makes the interpretation of
answers, analysis and tabulation, easier than in
the case of unstructured answers.
Unstructured answers become difficult to
categorise after the study, and different
analysts may interpret them differently - so
they may lend themselves to subjective
interpretations.

31

Subjectivity by itself is not bad, but it becomes

difficult to defend it if the sponsors (clients) of
the study are quantitatively oriented. Most large
scale studies in marketing research therefore,
choose the less risky, and easier to manage,
structured-answer approach.

32

Open ended and Closed ended Questions

Questions which permit any answer from the respondent
in his own words are called open-ended questions.
Questions which structure the possible answers
beforehand are known as closed-ended questions.
An example of an open-ended question is " What do you
like about Surf
detergent?"____________________________________

33

The respondent can say whatever he wants to, in

response to this question.
On the other hand, a closed-ended question which elicits
similar information could be "What do you like about
Surf detergent?"
Its cleaning power
Its Price
Its fragrance
That it dissolves easily
Its stain-removing ability
Any other, (please
specify)_________________________________
Here, options "a" to "e" are pre-determined, but "f"
provides for anything else the respondent wants to add.
34

Disguised Versus Undisguised Questions

Sometimes questions that are disguised (rather
than direct) can elicit more accurate replies. For
example, we may ask a person if he/she is a good
parent. This is a direct question.
Or, we may ask for the respondent's opinion on
the deficiencies they have observed in how others
bring up their children- say, their neighbours,
relatives or friends. This is an indirect question,
and a qualified analyst can interpret the answers
to gauge how good a parent the respondent might
be, from the responses given.
35

The problem with the direct question in this case

is that most people will not admit to being a bad
parent. But they may come out freely with other
people's deficiencies, some of which could
reflect their own shortcomings.
There are other reasons why disguised questions
are sometimes needed. It is often found that
respondents are biased when they know who is
the sponsor of the study. To get true, unbiased
opinions regarding attitudes towards brands,
researchers sometimes do not let on the name of
the sponsor.
36

For example, a well known multinational

company making electrical switches for industrial
application once did an anonymous survey in
Mumbai among its customers (a study done by the
author) and found many deficiencies in its
products and service which they otherwise may
not have found out. If it results in more accurate
data without doing any harm to the respondent, it
may be a legitimate way to do the study.

37

Completely disguised or indirect questions probing

into the psyche of a person are usually used for
qualitative research, as part of
projective
techniques, etc.
To summarise, market researchers usually ask
structured, undisguised questions in a typical study
done on a large sample. Most studies also tend to
be of the "quantitative" type, where numbers
(frequencies), percentages, averages or similar
summary statistics are computed. These types of
analyses are easier to do with structured formats for
answers.
38

Even if a study is primarily based on structured responses,

a couple of open-ended questions may still be included in
it if they are the best suited for the task on hand. One such
category of questions is called "Probing" questions in
marketing research terminology. These are used as a
follow up after a structured response question. An
example of this use of open-ended question following a
structured question is
Which brand of mosquito mats do you use?
Good Knight
Mortein
Jet
Why do you use this particular brand?
_________________________
In this question, the second part is open-ended, while the
first part is closed-ended.
39

Types of Questions
The six major types of questions that most
questionnaires would generally use areOpen-ended
Dichotomous (2 choices)
Multiple Choice
Ratings or Rankings
Paired Comparisons
Semantic Differential, or other special types of
scales.
40

An open-ended question is one which

leaves it to the respondent to answer it as
he chooses. An example is What do you
think of the taste of Brand X of Cola?
No alternatives are suggested.
The
answer can be in the respondents own
words.

41

Dichotomous questions
These are those which ask the respondent to choose
between two given alternatives.
The most common example of this is the yes or no
type of questions Are you a user of Brand X toilet
soap? Yes or No are the alternatives given.
A third choice is sometimes added to dichotomous
questions such as Do you like Brand X of potato
chips? The choices given are Yes, no, and
neither like nor dislike.
Sometimes, any other, please specify ______ is
used instead of neither like nor dislike.
42

Multiple choice questions

These are extensions of dichotomous questions,
except that the alternatives listed number more
than two. A common example is as follows
Please tick against the factors which made you
buy this brand of car :
. Reasonable Price
. Great Looks (Appearance)
. Fuel Economy
. Easy Availability of Service
. Any Other, please specify.
43

In the above question, more than one category can

be chosen. In some multiple choice questions,
only one category is to be chosen. For example,
look at the question belowPlease specify your age group. Below 15
. 16-25
. 26-40
. Above 40
Only one of the above is to be chosen. It must be
clear to the respondent and the interviewer
whether only one choice is allowed, or more than
one are allowed for a multiple choice question. 44

Ratings or Rankings : This is a question of the type,

Please rate the following detergent brands on a scale of
1 to 7 in their ability to clean clothes.
Brand A
Brand B
Brand X

1
1
1

2
2
2

3
3
3

4
4
4

5
5
5

6
6
6

7
7
7

This is an example of rating. Ranking would have

looked as follows :
Please rank (1=Best, 2=next best, etc.) the following
detergent brands on their ability to clean clothes.
Brand A
----Brand B
----Brand X
-----

45

Paired Comparisons
A special type of question is the paired
comparison.
This requires the respondent to choose between
pairs of choices at a time. For example, there
could be six brands of colour TVs, Brands A, B,
C, D, E, F. A respondent may be asked to do a
paired comparison to say which Brand is better,
but for only two Brands at a time.

46

He is given a table or a card with two brands

written on it, and has to choose the better brand,
each time. This process has to repeat for as many
pairs as exist in the given set of objects or brands.
Some
special
techniques
such
as
Multidimensional Scaling need data from paired
comparisons. This technique is explained later in
Part II of this book.

47

Semantic Differential
Another scale commonly used by marketing researchers is
called the semantic differential. This type of question is
similar to the rating scale. The only additional
feature is that a set of two adjectives forms the two extreme
points of the scale. For example, a product is
Easy to Use |----------------------|
Difficult to Use
Expensive |----------------------|
Inexpensive
Easily Available|----------------------| Not Easily
Available
Convenient |-----|-----|-----|-----|
Inconvenient
48

There may be several intermediate points

between the two extreme values of the scale.
These could be coded 1 to 5 or 1 to 7 or
whatever the number of points is. A commonly
used 5 point scale is from Completely Agree to
Completely Disagree.
There may be questions based on other scales
which are standard or specially constructed.
Some scales like the Likert Scale or Thurston
Scale are named after people who invented them.
49

How to Choose a Scale and Question Type

The researcher must decide on the scale and type
of question based on the following factors.
Information Need
Output format desired
Ease of tabulation
Ease of interpretation
Ease of statistical analysis
Reduction of various errors in understanding
or use by respondents and field workers

50

Transforming Information Needs Into A Questionnaire

We will now illustrate by developing a complete
questionnaire for a given set of information needs.
Example of Information Needs : A soft drink
concentrate manufacturer (such as Rasnas
manufacturer, for example) wants to know the
following :
Demographic profile of users versus non-users of
soft drink concentrates.
51

Among users
the preference for liquid concentrate versus powder.
preference for powder with sugar added, versus
powder with no added sugar.
occasions of use by self
whether served to guests
rating on convenience, taste, price and availability
brand preferred among soft drink concentrates.
Among non-users
Reasons for not using soft drink concentrate
Substitute product usage, if any, and reasons for using
or consuming them
Let us attempt to develop a questionnaire for the
above information needs. A possible questionnaire is
shown in the next slide
52

Questionnaire for Soft Drink Concentrate Study

Q. No. _______

Date ----------

Centre _______
Dear Sir / Madam,
We are doing a brief survey to find out more about
consumer preferences regarding soft drink concentrate.
We would be grateful if you could spare a few minutes
to participate in it. Thank you for your cooperation.
53

Do you use soft drink concentrate to make your

own soft drinks at home ?
Yes
No
If yes, continue with Q.2. If No, Go To Q.9.
Do you use liquid or powdered concentrate ?
(Tick only one)
Liquid
Powder
Both
54

(Questionnaire, contd.)
Which type of concentrate do you prefer out of the
following ?
Concentrate with sugar added
Concentrate without sugar added
What are the occasions when you use soft drink
concentrate to make soft drinks ? (Tick only one)
Regularly, all year round
Regularly, only in summer
Occasionally, all year round
Occasionally, only in summer
55

Do you serve it to guests ?

Yes

No

Depends on the guest

Which brand do you use ?

Rasna

Brand X

Brand Y

Any other (please specify) _______________

_____________________________________
56

(Questionnaire, contd.)
Please rate the brand you use on the following
attributes, on a scale of 1 to 7 (7=Very Good,
1=Very poor).
1 2
3
4
5
6 7
Availability |-------|------|------|------|-------|-----|
Taste
|------|------|------|------|-------|-----|
Convenience |-------|------|------|------|-------|-----|
Price
|-------|------|------|------|-------|-----

57

Any other comments on the brand you

use ?
________________________________
_______________________
________________________________
_________________________
________________________________
_______________________
After Q. 8, Go To Demographics Q.11.

58

(Questionnaire, contd.)
NON USERS
Do you consume any of the following regularly ? (You
may tick more than one)
Yes
No
Fruit Juice
Squash
Bottled Soft Drinks

59

Yes

No

Tea
Coffee
Nimbu Pani
Buttermilk

60

(Questionnaire, contd.)
DEMOGRAPHICS
Please let us know a little more about yourself.
Your age group
Less than 25
26 40
41 50
Over 50
61

Your monthly household income

Less than 5000 Rupees/Month
5001 to 10,000 Rupees/Month
10,001 to 15,000 Rupees/Month
Over 15,000 Rupees/Month
Address :
________________________________________
________________________________________
________________________________________

62

Critically examine the questionnaire above to suggest

improvements in any of the questions or the scales or
the choices given in the multiple choice questions.
Some hints for discussing the merits and demerits of the
above questionnaire
Are the income and age categories adequate for
analysis of the data? (Questions 11 and 12)
Is the 7 point scale used in Question 7 easy to
understand? Is it appropriate? Adequate? respondents
regarding price? Are we getting the answer?
63

Should there be an open-ended question

number 8? Why?
Have we left out anything? Such as who
decides on the brand to buy (for users)? Who
decides to buy/use substitutes (for non-users)?
Should we also ask which family members
drink the soft drink (for users) made from
concentrate?
Should we ask the convenience and price
questions
separately
(Question7)
and
differently? What exactly do we want to know
from res
64

Double-Barrelled Questions
Inexperienced questionnaire designers have a
tendency to combine two questions into a single
question, such as
Are you happy with the price and quality of
Brand Y ?
Yes

No
65

This is not a good question to ask, because

the answer will be ambiguous, whether it is
yes or no. It would not be clear whether the
respondent has said yes for price alone,
quality alone, or for both. The same problem
exists for a no answer.
It is better to rephrase the question and
provide for different answer categories for
each attribute, or ask two separate questions,
one for price and one about quality.
66

Good Questionnaires and Bad Questionnaires

In general, a questionnaire is good if it measures
what it set out to measure (ie., it is VALID) and does
it in an efficient manner.
Usually, a questionnaire goes through various stages
before it is used in the field.
Listing of information needs
Conversion into questions with suitable scales of
measurement
Sequencing of questions into a logical order
67

 Trying it out in a pre-test on a handful of

respondents in a
convenience sample or a
field
sample.
Modifications in the wording, scale or sequence
as a result of the pre-test,
and then
Preparation of the final draft for the actual
study, are the usual steps involved. Most faults in
a questionnaire would be ironed out in this
process if followed meticulously.

68

Blank Output Formats/Tables

Problems in a typical study stem from a lack of
sufficient thought given to the analysis required in
advance.
The solution for this is to prepare blank output
formats for each question on the questionnaire,
before doing the field work.
In many cases, the value of the research increases
manifold by slightly modifying the scale or wording
of the questions asked. Remember, it is cheaper to
modify the questionnaire in advance than think about
what could have been done after the study is over. 69

Reliability and Validity of a

Questionnaire
Reliability is the property by which consistent
results are achieved when we repeat the
measurement of something.
A questionnaire used on a similar population which
produces similar results can be termed as reliable.
Consistency of form and manner of asking
questions (their exact wording, the amount of
structuring, etc.) generally ensures reliability.
Proper training given to interviewers in a study also
improves reliability, by reducing variation in the
way they ask questions and record answers.
70

Validity is the property by which a

questionnaire measures what it is
supposed to measure.
If we want to measure attitudes towards
brands of washing machines in terms of
service and product features, then that is
what the critical questions in the
questionnaire should measure.

71

The validity of questions on a questionnaire

can be checked by comparing it with
previously used items (questions) measuring
the same thing, and also trying out different
questions to find out which one seems to
measure what we intended to measure.
A certain amount of judgement which comes
with experience is of great help in framing
"valid" questions. It is also possible to consult
experts in research methodology, or the
subject on hand to check that a given set of
questions is "valid".
72

Summary
Questionnaire design is an art, but there are
certain common sense rules that can help, as we
have discussed throughout this chapter.
Scales to be used should be decided on by the
researcher in consultation with the study
sponsor, keeping in mind the kind of output
formats or tables required for decision-making.
73

Validity and reliability issues are of

particular importance if the subject of the
study is new or the researcher is
inexperienced.
Practice with designing questionnaires is
the best way to perfect the art.
Please do test the questionnaire on a small
sample, and modify it if necessary, before
going full steam ahead.
74