Using Questionnaires in Education Research

Search RESINED
powered by FreeFind

Beginning Research | Action Research | Case Study | Interviews | Observation Techniques | Education
Research in the Postmodern
Evaluation Research in Education | Narrative| Presentations | Qualitative Research | Quantitative
Methods | Questionnaires | Writing up Research

Prepared by Professor Andrew Hannan
Now led by Dr. Julie Anderson
A Hannan, Faculty of Education, University of Plymouth, 2007
CONTENTS
A. INTRODUCTION
B. QUESTIONNAIRE DESIGN - How to do it
C. QUESTIONNAIRE DESIGN - Examples
D. QUESTIONNAIRE ANALYSIS - How to do it
E. TASKS
F. FURTHER READING

A. INTRODUCTION
1) You will no doubt have had numerous experiences of having to fill in a
questionnaire, everything from the Census itself to forms to get your motor
insurance or library card.
2) Such questionnaires wouldnt be so popular if they werent in some measure
successful in getting the information required in a form it can be usefully analysed.
3) I want to begin by asking you the following questions.
What are questionnaires good at doing - what are their advantages?
What are questionnaires not good at doing - what are their
Using Questionnaires in Education Research http://www.edu.plymouth.ac.uk/resined/QUESTS/index.htm
1 sur 10 01/03/2013 11:35
disadvantages?
Jot down your answers before reading on. Use the lists you have created to check
against the points made below.
4) Notice how I used a form of open-ended question to try to get you state your
views, having decided that this was better than listing potential advantages and
disadvantages and asking you to tick to indicate the ones with which you agreed.
The issue of how best to pose questions is one to which we shall return.
5) Straightforward written questions requiring an answer by ticking the appropriate
box are very efficient ways of collecting facts.
6) Questionnaires are also employed as devices to gather information about peoples
opinions, often asking respondents to indicate how strongly they agree or disagree
with a statement given, but sometimes merely posing a question and giving
respondents space in which to formulate their own replies.
7) One of the obvious advantages of questionnaires is that they provide data
amenable to quantification, either through the simple counting of boxes or through
the content analysis of written responses.
8) Problems arise, however, when the facts themselves are difficult to establish,
when the question posed contains ambiguity or bias or when the range of available
questions or answers does not allow the respondent the opportunity to state what he
or she wishes. The agenda is normally set by the researcher with the respondent
being somewhat constrained so as to follow planned pathways; there is little room
for the unexpected. The picture presented is often static, with facts and views given
as more concrete and fixed than they may be in the dynamic flow of personal
formation and social interaction.
9) Let us have a look at how questionnaires are put together.
Back to CONTENTS list
B. QUESTIONNAIRE DESIGN - How to do it
1) Ask yourself, why should I use a questionnaire? It is worth being self
reflective when beginning to construct your own questionnaire, by writing down your
reasons for choosing such a research instrument rather than another (say interviews
or observation), for inventing your own rather than using one already available in the
literature, and for posing the sorts of questions you want to use. Such notes may be
useful when you come to write the methods chapter/section of your research
report.
2) The fundamental question that must then be asked is, what are you trying to
find out? Every questionnaire must have a purpose, ie it must draw from some
underlying hypotheses about what are the important facts or opinions and even
make some predictions about which facts may be relevant in explaining the opinions
expressed.
3) Write your own rationale, in terms of statements like, I need to know whether
or not senior members of staff are more likely to support the moves to introduce
appraisal and what reasons they have for the positions they express. I need to find
2 sur 10 01/03/2013 11:35
out why junior staff seem opposed, ie are they misinformed about the nature of the
reforms or are they protecting weaker colleagues from what is seen as scapegoating
in an under-funded profession of whom too much is demanded with too little
support?
4) This can be developed so as to produce a justification for every question
used, eg I asked this so as to probe the extent to which those of various positions
in the hierarchy valued staff consensus and the feeling of shared purpose, with the
intention of seeing whether those who were strongly committed to such views were
also more or less opposed to staff appraisal. If you cant come up with a good
rationale, drop the question.
5) Many questions can be closed-ended, ie the respondent has simply to tick the
appropriate box, although these are most suitable for the gathering of unproblematic
facts. Such a device can be employed to ascertain the viewpoints of respondents but
there are more problems involved in both posing the questions and offering a range
of possible answers.
6) The best descriptions I have come across of the issues involved are those given in
Munn & Drever (1999), which I strongly recommend you should read.
7) You need to decide how to pose your questions and the form of coding you might
use, to ensure that your survey produces data that you can analyse. You need to
avoid ambiguity and bias, and to refrain from leading your respondents. For further
guidance consult the types of questions; decisions about question content;
decisions about question wording; decisions about response format; and, question
placement and sequence pages in the The Research Methods Knowledge Base.
8) I would personally like to recommend the use of open-ended questions that
allow respondents to state their opinions in ways not pre-selected by the researcher.
These give the possibility of discovering things that were unsuspected and enable
some respondents to challenge the sort of assumptions that may have been made.
The disadvantage of such questions is that computation is very difficult and can only
follow a process of categorisation, which in any case has to be undertaken by the
researcher.
9) However, a combination of closed-ended and open-ended questions has its
advantages in that it preserves the possibility of easy computation whilst providing
respondents with the space to develop their own ideas, eg
To what extent are you satisfied with the current proposals for staff appraisal?
Very satisfied Satisfied Neutral Not satisfied Very dissatisfied
1 2 3 4 5
Please circle as appropriate and explain your response in the space below: ....
The Likert scale used here (1-5) also serves as a self-coding for any explanation
given.
10) Let me identify some other important points: -
Make sure you show a full draft to someone else, preferably a tutor, before
trying it out.
1.
3 sur 10 01/03/2013 11:35
Make use of a pilot trial run first if at all possible - it is amazing just how flawed
the product of hours of solitary effort can be once it is put into practice. Use a
parallel smaller sample or a sub-sample of the target population concerned. If
all else fails get 3 or 4 friends to pretend they fit the respondent categories. Ask
those in your pilot sample to feed back to you their views about the
questionnaire itself, eg how long it took to complete, which questions they
found ambiguous or leading or biased, etc. Dont forget to analyse the results to
see if you can make sense of the data that you have collected. Its a lot better
to find out youve made a mistake at this stage than to do so when its too late!
2.
Bear in mind how you propose to use the data so collected - it is better to build
in a coding device for closed-ended responses from the very beginning and to
check in advance that you have sufficient information to undertake a statistical
analysis (see Munn and Drever [1999] on analysing the results, although dont
necessarily take their advice on avoiding computers).
3.
Attempt to obtain as big a response as possible, the whole population would be
best (!) but otherwise you will need to seek a random or structured sample
(see Sampling in The Research Methods Knowledge Base), not forgetting that
you will need a minimum of 30 respondents to do statistical analysis of anything
more than a very low level kind. If, for example, you decided that you wanted to
know what primary school teachers thought about the National Numeracy
Strategy, you would be best to ask all of them. As this is probably beyond your
scope, for reasons of the costs and time involved, you may feel it best to
confine yourself to a sample, eg all primary school teachers in Devon. Even this
is probably too many to cope with, so you may decide on all those in, say, East
Devon, or all those in Exmouth, or, even, all those in one particular school.
Alternatively, you might give every primary school in the country (or Devon) a
number and choose a 10% random sample, using a table of random numbers,
and survey all the teachers in those schools. You might try and put together a
stratified sample of schools typical of the different varieties that you know to
exist to include in your survey. You might deliberately seek unusual schools
where you know teachers have taken a particular stance to the teaching of
mathematics in order to find out more about the extremes or about vanguard
cases where ideas were being tried in ways which others were likely to follow.
Whatever choice of sample you make you need to justify it, ie to make a case to
the reader who examines your results that he/she has good grounds for taking
your findings seriously in terms of their representativeness.
4.
Ensure that you pose your questions in a manner that makes them easy to
answer and that your whole questionnaire is short enough to mean that most
people will complete it. Beware of using survey methods that make it likely that
a significant proportion of your target population wont have the chance to
respond, eg using email when not all have access to it or forms of written
presentation beyond the literacy level of some of your respondents.
5.
You need to be careful about obtaining the highest possible response rate
otherwise the answers you get may not be representative of the overall
population or of the sample you chose. Presentation and politeness are
important here, remember the respondents are doing you a favour and be sure
to thank them! Postal surveys normally obtain very low response rates, even
when pre-paid envelopes are supplied. It helps to make it as simple as possible
for respondents to return their forms. You need to convince members of your
target group that it is worth their while to complete and return your form - tell
them how much it matters, how it will have real consequences, how they can
find out the results. Dont forget to reassure them about confidentiality. If you
6.
4 sur 10 01/03/2013 11:35
are asking them to identify themselves (not normally a good idea), you must
explain to them how you will use and present the information you gather, in
accordance with your ethics protocol (which you may wish to incorporate in an
abbreviated form in the introduction to the questionnaire itself). If you get a low
response rate you need to ask yourself if those who replied are likely to differ
substantially from those who didnt, ie are the non-respondents likely to be from
particular groups or to take different views of the issues involved? A response
rate of over 60% is fairly respectable. One of less than 50% is a concern and
you need some strong arguments to justify taking the results seriously.
C. QUESTIONNAIRE DESIGN - examples
1) The first example given is the text of a questionnaire I distributed to all Devon
primary school headteachers in the summer of 1993. Please click here and have a
look at it with reference to the design recommendations made above and make a list
of its strengths and weaknesses- you can use this as a self-corrective guide when
designing your own questionnaire!
2) The second example is from the Families and Children Study conducted by the
National Centre for Social Research. Please click here to have a look at the
self-completion questionnaire issued to a sample of school children aged 11-15 in
2006. Note how the designers have done their best to produce a child-friendly form,
complete with smiley faces!
D. QUESTIONNAIRE ANALYSIS - How to do it
In general
1) When you get the forms back you will need to number each completed form (the
ID for each respondent).
2) You should then sort out the easy stuff first, ie the answers to closed-ended
questions which you can simply count in order to give you frequencies. Question 1 in
section A of the headteachers questionnaire given earlier is one of these, as are each
of the Likert scale options which set out the various government proposals for
questions 1-7 in section B. The responses to the closed-ended items should be
entered onto a manual or computer generated spreadsheet, with the respondents
numbers (ID) given for each row and the answers coded by number or letter for
each column. The spreadsheet below has the first ten entries from the actual survey:
ID A1 A2 A3 B1 B2 B3 B4 B5 B6 B7
01 6 220 9.05 5 5 3 2 3 2 3
02 5 105 4.7 5 3 2 1 4 2 2
03 5 186 - 5 4 4 4 5 2 2
5 sur 10 01/03/2013 11:35
04 6 350 12.8 5 - 4 2 4 2 2
05 5 47 2.5 5 4 4 2 5 2 2
06 5 35 2.2 5 5 5 5 5 4 4
07 5 420 16 5 2 3 2 5 2 2
08 5 65 3.3 5 3 3 1 3 1 1
09 5 40 2.2 5 5 5 2 5 3 3
10 5 60 2.7 5 2 1 1 5 2 -
Note how a hyphen has been entered where there is a missing or unclassifiable
response. Items A2 (number of pupils on roll) and A3 (number of FTE teachers)
would require further categorisation before analysis, eg enter 1 for schools with less
than 50 pupils, 2 for those 50-99, 3 100-199, 4 200-299, 5 300-399, 6
400-499.
If you have designed your questionnaire correctly the respondents will have done the
coding for you by ringing the appropriate number which you can then enter into the
spreadsheet. Its not difficult to find out how many respondents gave each sort of
answer. These numbers can easily be turned into percentages (showing what
percentage of the respondents gave each sort of response). These figures are
already very useful as analytical tools.
3) Very often questionnaires have a background characteristics section like the first
part of the headteachers survey. These frequently give information on the age or sex
of respondents. The next stage in statistical analysis is to take whichever of these
variables interests you and to look at how the patterns of response vary, eg
comparing males and females in terms of the answers they gave to all the other
questions, or headteachers of infant and junior schools in the same manner. This is
known as crosstabulation and is easiest done by computer programme (especially
SPSS). You can then ask such analytical questions of the data as, Are there gender
differences at work? or, in terms of the example given, Are infant school heads
more in favour of moving teacher training into schools and out of universities than
junior school heads?
4) Of course, far more sophisticated statistical analysis can also take place, with
measures of significance and correlation, etc. But these are not covered here. If you
are interested please consult the Quantitative Methods in Education Research
component. You may also wish to consult the Analysis page from the The Research
Methods Knowledge Base.
5) The headteachers questionnaire was part of a survey that also involved slightly
different questionnaires issued to parents of pupils at seven Devon primary schools
and teacher training students and tutors at what was then Rolle Faculty of Education.
The results were first analysed in the sort of fashion described above (using Excel
spreadsheets and SPSS), with comparisons being made between the different
categories of respondent. See my paper The Initial Training of Primary School
Teachers: Response to the DFE, an Interim Report. Note in particular the elements of
the research report, viz the description of the research instrument, samples and
6 sur 10 01/03/2013 11:35
response rates, and the manner in which the question asked is presented alongside
the information collected in response.
6) The comments provided by the headteachers, students, tutors and parents in
explanation of their Likert scale responses and in answer to the open-ended question
posed in section C were entered on computer into a qualitative data software
package called HyperQual. The Likert responses were then used to code that data,
enabling me to gather together information to answer analytical questions such as,
What were the comments made by headteachers who strongly opposed the Mums
Army idea? and, What were their general views of government policies?.
7) To see how an analysis of this more qualitative data was undertaken see my
article The Case for School-led Primary Teacher Training, Journal of Education for
Teaching, 21, 1, 25 -35, 1995.
8) In this example, HyperQual was used to sort answers into analytical categories,
but content analysis of the different points made was then carried out manually. In
practice this meant that every statement was analysed for content and placed under
an appropriate heading, along with any others which were sufficiently similar. These
were then grouped under more general umbrella headings to produce the description
of points made with reference to their nature, range and frequency.

Using IT
1) It helps to make sure when you design your questionnaire that it is amenable to
computer analysis.
2) There are software packages that facilitate content analysis of responses to
open-ended questions, the most popular one being Survey Monkey which provides a
limited degree of functonality for free. Weft QDA is a free qualitative analysis
software application - 'an easy-to-use tool to assist in the analysis of textual data',
which can be downloaded from http://www.pressure.to/qda/. NVivo is a very
common analysis tool for qualitative data and is provided over the University of
Plymouth server to all networked PCs or as a use at home package.

3) For quantifiable data, MS Works and Excel spreadsheets will all produce a wide
range of computations and forms of presentation such as graphs and charts as well
as carrying out some statistical operations.

4) However, to undertake sophisticated statistical analysis to produce tables of
results as well as figures and charts there is nothing to compare with SPSS, which is
provided over the University of Plymouth server to all networked PCs or as a use at
home software package.
5) It is a fairly straightforward matter to enter the data on computer yourself. There
are considerable advantages in doing so rather than following the advice of Munn and
Drever (1999) and doing it all manually, in that once the data are entered the
computer packages are wonderfully quick and flexible tools of analysis.
7 sur 10 01/03/2013 11:35
6) SPSS is capable of performing any number of t tests, Chi- squares, etc but it is
not beneath undertaking relatively simple tasks such as crosstabulation (producing
tables separating out various categories of respondent for each set of answers) and
producing row, column and cumulative percentages.
7) You will need to use the in-built 'tutorials' and the 'help' menu to teach yourself
how to use SPSS.
Not Using Computers
1) Get a copy of Munn and Drever (1999). This book takes you through a pencil-and
paper method ideal for computer-phobes.
E. TASKS
(NB: only for those University of Plymouth students undertaking the
Research in Education module as part of the preparation for the
submission of a MA dissertation proposal)
Tasks, once completed, should be sent to resined@plymouth.ac.uk, making clear:
which component it is from;
which task it is (B or C);
the name of your dissertation supervisor.
It will then be passed on to the component leader (and copied to your supervisor).
The component leader will get back to you with comments and advice which we hope
will be educative and which will help you in preparing your dissertation proposal once
you are ready. (Remember that these tasks are formative and that it is the proposal
which forms the summative assessment for the MERS501 (resined) module.) This
email address is checked daily so please use it for all correspondence about RESINED
other than that directed to particular individuals for specific reasons.

TASK B (DATA COLLECTION)
Design your own questionnaire on a topic of your choice, describing the target
group and providing an ethics protocol.
NB For a QUESTIONNAIRE survey it is often preferable to incorporate the ethical
provisions in the introduction to the questionnaire form itself, ie telling informants
what the project is about (informed consent), giving them the choice not to respond
to individual items or the form as a whole (right to withdraw), setting out how
feedback may be obtained (debriefing), describing the provisions for confidentiality
(particularly if the form is not anonymous), etc. This section of the questionnaire is
in effect the 'ethics protocol' and can be submitted for approval.

TASK C (DATA ANALYSIS)
When you have received feedback from me, pilot your questionnaire with a
8 sur 10 01/03/2013 11:35
group of at least ten respondents (manufacture your own responses to make
the numbers up to ten if necessary).
Enter the quantifiable data on a manual or computer spreadsheet and analyse
the results.
Then, undertake a content analysis of the qualitative data (with or without the
help of a computer).


F. FURTHER READING

CD-ROM
Barrett, Elizabeth; Lally, Vic; Purcell, S & Thresh, Robert (1999) Signposts
for Educational Research CD-ROM: A Multimedia Resource for the
Beginning Researcher. Sage Publications, London. (This CD-ROM includes
a section called 'Travelogues' that gives advice on three commonly used
methods of data collection - Interviews, Observation and Questionnaire
surveys.)

WEBSITES
Trochim, William M. The Research Methods Knowledge Base, 2nd Edition.
Internet WWW page, at URL: http://www.socialresearchmethods.net/kb/
(version current as of July 01, 2008). This is the excellent site referred
[and linked] to several times in the sections presented above.
The Survey Question Bank has many examples of lists of questions used in
structured interviews as well as what it calls 'self-completion
questionnaires'. It's a good place to look for examples, which you can
normally use without worrying about copyright (although you will need to
acknowledge the sources in the normal fashion).

BOOKS
Munn, Pamela & Drever, Eric (1999) Using Questionnaires in Small-Scale
Research: A Teachers Guide Scottish Council for Research in Education,
Edinburgh.
Robson, C. (2002) Real world research : A resource for social scientists
and practitioner-researchers, Oxford, Blackwell [has an excellent chapter
on surveys/questionnaires]
For more on statistical analysis, from basic averages, standard deviation
9 sur 10 01/03/2013 11:35
etc. to co-variant analysis, try...
Wright, D. & London, K. (2009) First and Second Steps in Statistics, (2nd
edn.) London, SAGE.


Beginning Research | Action Research | Case Study | Interviews | Observation Techniques |
Education Research in the Postmodern
Evaluation Research in Education | Narrative| Presentations | Qualitative Research |
Quantitative Methods | Questionnaires | Writing up Research

A Hannan, Faculty of Education, University of Plymouth, 2007

10 sur 10 01/03/2013 11:35

Using Questionnaires in Education Research

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Using Questionnaires in Education Research

Hochgeladen von

Copyright:

Verfügbare Formate

Search RESINED

Das könnte Ihnen auch gefallen