Introduction to Questionnaire Design

Dr Christine Thomas Dr Rachel Slater U501 Workshop, 28th April 2009

Workshop objectives
By the end of this session you will be able to: Understand why questionnaires are used and when to use them Understand the process of constructing a questionnaire Acknowledge the key features of good question design

Questionnaire design in the context of the survey process

Research aim and research questions Identify the population and sample Decide how to collect replies Design your questionnaire Run a pilot survey Carry out main survey Analyse the data Report findings and dissemination

Questionnaire design in the context of the survey process

Research aim and research questions
Identify the population and sample Decide how to collect replies

Design your questionnaire

Run a pilot survey Carry out main survey

Analyse the data

Write up findings and dissemination

What is a questionnaire
A research tool for data collection Its function is measurement (Oppenheim, 1992) The term questionnaire used in different ways: often refers to self-administered and postal questionnaires (mail surveys) some authors also use the term to describe interview schedules (telephone or face-to-face)

Why would you use a questionnaire?

Why use a questionnaire?

Target large amount of people Use to describe, compare or explain Can cover activities and behaviour, knowledge, attitudes, preferences Specific objectives, standardised and highly structured questions Used to collect quantitative data information that can be counted or measured

Strengths and limitations

Strengths
Can target large number of people Reach respondents in widely dispersed locations Can be relatively low cost in time and money Relatively easy to get information from people quickly Standardised questions Analysis can be straight-forward and responses pre-coded Low pressure for respondents Lack of interviewer bias (possibility of ghost interviewer effect)

Limitations
Low response rate and consequent bias and confidence in results Unsuitable for some people e.g. poor literacy, visually impaired, young children Question wording can have major effect on answers Misunderstandings cannot be corrected

Limitations
No opportunities to probe and develop answers No control over the context and order questions are answered No check on incomplete responses Seeks information only by asking, can we trust what people say? e.g. issues with over-reporting

Maximising the response rate

If you were sending out a questionnaire, what would you do to maximise the response rate?
In groups of 3 or 4, 5 minutes

Techniques for minimising nonresponse

Good design Thoughtful layout, easy to follow, simple questions, appearance, length, degree of interest and importance, thank people for taking part Pre-notification Explanation of selection Sponsorship, e.g. letter of introduction / recommendation Cover letter

Techniques for minimising nonresponse

Incentives Small future incentives, e.g. prize draw Understanding why their input is important Reminders Confidentiality Anonymity Pre-paid return envelopes

Clear specification

Political questionnaire exercise

In groups of 3 or 4, spend 15 minutes What research question(s) do you think the questionnaire is trying to answer? What are you reactions to: The question wording and structure? The answer options? Which are open questions and which are closed questions? How could the questions be improved?

Question wording things to avoid

Abbreviations Alternative meanings (tea, cool, dinner) Ambiguity and vague wording (fairly, generally, you the respondent, household, family?) Doubled barrelled do you speak English or French? Double negatives Inappropriate categories

Question wording things to avoid

Leading questions Memory issues Social desirability Question complexity

Question wording other things to think about

Missing categories include other, dont know and not applicable Sensitive questions Simple language not technical or slang Question ordering Open or closed questions? Closed question choice of alternative replies Open question written text (or spoken answers)

Open and closed questions

(from Oppenheim, 1992)
Strength OPEN Freedom & spontaneity of answer Opportunity to probe Useful for testing hypothesis about ideas or awareness CLOSED Requires little time No extended writing Low costs Limitation Time-consuming Coding more problematic More effort from respondents Loss of spontaneous responses Bias in answer categories Sometimes too crude

Easy to process
Make group comparisons easy Useful for testing specific hypothesis

May irritate respondents

Hypotheses and variables

A hypothesis is a proposition to be tested or a tentative statement of a relationship between two variables (Neumann, 2000) a tentative answer to a research question Attendance at research training workshop (independent variable) influences students probation marks (dependant variable) Directional (one-tailed) and non-directional (two-tailed) The dependant variable alters as a consequence of the independent variable its value is dependant on this

Create a questionnaire
work in groups of 3 or 4
Research Question Why do people recycle? Develop 10 hypotheses (take no more than 10 minutes) and state the independent and dependant variables, e.g.
Recycling behaviour (dep var) is affected by age (indep var) People with higher incomes (indep var) recycle more (dep var)

Using the hypotheses you have devised, write questions for a questionnaire. You should normally have more than one question for each hypothesis (20 minutes) Feedback (10 minutes)

Readings
Oppenheim, A.N. (1992) Questionnaire Design, Interviewing and Attitude Measurement. Pinter Publishers, London. Moser, C. and Kalton, G. (2001) Survey Methods in Social Investigation. Ashgate, Aldershot. Foddy, W. (1994) Constructing Questions for Interviews and Questionnaires. Cambridge University Press, Cambridge. De Vaus, D.A. (1990) Surveys in Social Research. Allen and Unwin, London. Hoinville, G and Jowell, R. (1982) Survey Research Practice. Heinemann, London. Fink, A. (Ed.)(1995) The Survey Kit. Sage, London. Fowler, Floyd J. (2002) Survey Research Methods. Sage, London

Other resources
Doing Political Research DVD (OU) ESRC offer courses in questionnaire design (and statistical analysis) through CASS (Courses in Applied Social Surveys). See: http://www.esrcsocietytoday.ac.uk/ESRCInfoCentre/rese arch/resources/CASS.aspx http://www.s3ri.soton.ac.uk/cass/programme.php The Open universitys OpenLearn survey research http://labspace.open.ac.uk/course/view.php?id=4197&to pic=all AACS (OU) run SPSS training courses