Research Skills One

Oh, no!
Please, no!
Not statistics!!
Aims of the two first year Research Skills
courses:

Basic grounding in research skills -

designing questionnaires and experiments

data analysis using statistics, Excel and SPSS

writing up studies using standard presentation conventions
Comparison of RM1 exam marks for 2004-2008
0
5
10
15
20
25
30
35
40
0-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 91-100
exam mark
f
r
e
q
u
e
n
c
y

(
%
)
2004 (N = 249)
2005 (N = 246)
2006 (N = 243)
2007 (N = 214)
2008 (N = 234)
Outline of this course (Research Skills One):

Issues in designing questionnaires and experiments.

Descriptive statistics - effective summarising of data.

Statistical tests (correlation, Chi-Square, nonparametric tests for
comparing groups or conditions).

Using SPSS ("Statistical Package for the Social Sciences") for data
analysis and Excel for producing graphs and tables.

Research Skills Two:

More statistical theory.

More statistical tests.

Practice at running experiments and writing them up.

Ethics.



Organisation for this course:

One lecture and one practical session every week.


(Lectures: Thursdays at 1pm in Chichester Lecture Theatre
until week 12 December 6th).

Practical sessions - details on Sussex Direct (you are split into
lots of smaller groups).


Course requirements:






Assessment is by a mixture of coursework and
exam:
Coursework
SONA research participation (4 hours this term): 10%

Lab-report 1 10%
Lab-report 2 20%


Exam -
Unseen exam (in winter assessment period): 60%

Overall pass mark (exam and coursework combined): 40%

This term's requirements:






1. Write a lab-report based on a questionnaire from the STARS fast
food project (10%).

2. Write a lab-report based on a questionnaire on maths learning
experiences (20%).
3. Tackle the weekly statistics problem sheets (not marked, but
essential practice).
4. Four hours of research participation (10%).

5. Unseen 2-hour exam at the end of the course
statistical concepts (multiple choice questions);
deciding which statistical test is most appropriate;
interpreting statistical output from SPSS.
Recommended textbooks:






Field and Hole (2003). Sage publications.
Palgrave, 2006
Sage, 2006
BUY THIS ONE, I NEED THE MONEY !!!!!
Permitted calculators:


Casio FX82, 83, 85, 115, 570 or 991, all with any suffix.

No other type of calculator is acceptable.








Course documentation, handouts, lecture slides:

Go to Sussex Direct - this course is

Research Skills One (C8511)

OR

Type Graham Hole into Google
Click on
this link
Everything
you need to
know for this
course
Who to contact:

In the first instance, the course tutors running your practical
session.


Graham Hole (grahamh@sussex.ac.uk)

Questionnaire design:
Perhaps now you'll fill in
my questionnaire?


How does the way you collect data affect the data
collected?

How does question wording affect peoples answers?

Stages in questionnaire design:
5. Administer the questionnaire, after it has been revised
in the light of (4).
1. Formulate the research question(s) clearly.
2. Identify the population and sample.
3. Design the questionnaire: think about question
wording; questionnaire formatting; mode of
administration; data analysis.
4. Pre-test the questionnaire.
Samples and populations:
Sample: a subset from a
population (e.g. first-year
psychology students).

Population: a complete set
of things (e.g. all of
humanity).

For valid inferences to be made about a population's
characteristics, a sample must be representative of its
parent population (e.g. similar in age, SES, IQ, etc.)
Methods of obtaining questionnaire data:
Postal
questionnaire
Personal
interview
Phone
interview
Internet
questionnaire
Cost Low High Moderate Low
Data quality:
Response rate
Respondent
motivation
Interviewer bias

Low
Low

None

High
High

Moderate

Moderate
High

Low

Low
Low

None
Sample quality: Low, unless
high response
rate
High Moderate to
high,
especially
with random
digit dialling
Moderate, but
improving as
Internet access
widens
Postal
Questionnaire
Personal
interview
Phone
interview
Internet
questionnaire
Possible interview
length:
Short Very long Long Short
Ability to clarify
and probe:
None High High None
Anonymity: High Low Low Low
Dependence on
respondents
literacy:
High None None High
Control of context
and question
order:
None High High Depends
Goals of Questionnaire design:

1. To obtain facts about a person.

2. To obtain information about their attitudes and beliefs.

3. To find out what a person has done (behaviours).
Questionnaire wording:
1. Should be exact.
2. Should be simple.
3. Avoid biased or emotive words.
Schuman and Presser (1981): subtle changes of wording
may influence responses.

e.g. Should the Government allow public speeches by a
Communist? produced 25% fewer pro-free-speech
responses when allow was replaced with forbid.
4. Make all alternatives clear.
e.g. Payne (1951): "Do you think most manufacturing
companies that lay off workers during slack periods could
arrange things to avoid layoffs and give steady work
throughout the year?"
63% - companies could avoid layoffs.
22% - couldnt avoid layoffs.
15% - no opinion.

Same question plus phrase "or do you think layoffs are
unavoidable?"
35% - companies could avoid layoffs.
41% - couldnt avoid layoffs.
24% - no opinion.
5. Avoid the format: "Some people say x: do you agree or
disagree?"

6. Avoid unwarranted assumptions. e.g. "What is your
occupation?" assumes person has a job.

7. Avoid double-barrelled questions. e.g. "Should immigrants
be repatriated and their possessions confiscated?" is two
questions.

8. Avoid double negatives. e.g. "Are you against a ban on
smoking?"
9. Consider the relative merits of open-ended and
closed-ended questions.

Open-ended: allow unconstrained responses.
e.g. "How do you travel to the University?".
May produce richly detailed responses, but hard and tedious
to score.

Closed-ended: require choice from a limited range of
alternatives.
e.g. "Do you travel to the University by
(a) bus, (b) car, or (c) unicycle (tick one)".
Easy to code, but prone to bias.
Closed-ended questions must have
(a) a balanced response scale;
(b) mutually exclusive categories;
(c) facilities for handling "don't know" and "other"
responses.

Rating scales:
The Likert Scale:
"Criminals should be flogged".
Strongly agree Agree Neither agree
nor disagree
Disagree Strongly disagree
1 2 3 4 5
Can be 5-. 7- or 9-point scale (doesn't make much
difference).

Visual Analogue Scale:
Strongly agree Strongly disagree
Problems with questionnaires about
attitudes:

1. May not have an attitude - "doorstep opinions".
2. Attitudes may be complex and multi-dimensional.
3. Attitudes vary in intensity.
4. Expressed attitudes may depend on question wording,
sequence and interviewer effects.
Problems with questionnaires about
behaviour:
1. Memory limitations -
e.g. Chapman and Underwood's (2000) study of drivers' memory for accidents
and near-misses.
Can be counteracted by
(a) asking specific questions;
(b) asking for birth date rather than age;
(c) using a chronological format;
(d) re-interviewing.
2. Response biases due to social desirability or suspicion,
especially for illegal or anti-social activities.
Can be counteracted by ensuring anonymity.
5. Do your children prefer to eat in KFC or Macdonalds?
1. Do you visit fast food emporia regularly?
Over-complex wording. Exactly what does "regularly" mean?
2. How many burgers do you eat per month?
Assumes you eat burgers.
3. Some people suggest that fast food is leading to increased tooth decay and an
increase in obesity amongst teenagers in many parts of the U.K. Do you agree?
Over-long. Includes two separate questions. Implies you should agree with the views
expressed.
4. Which of the following methods do you use to travel to your fast-
food outlet? (a) Bus (b) Car (c) Bicycle
Does the questioner want you to choose only one option, or can you
choose more? No option for responding "other" .
What are the problems with these questions?
Assumes you have children.
Conclusions:

It's hard to design questionnaires properly!
Always be sceptical of survey results -ask yourself
Who were they collected by?
Who were they collected from?
How were the questions worded, exactly?
Remember - 8 out of 10 cats prefer Whiskas" has
become "In tests, 8 out of 10 cats who expressed a
preference, preferred Whiskas".
Useful references:
Burgess, T.F. (2001). A general introduction to the design of questionnaires
for survey research.
http://iss.leeds.ac.uk/info/312/surveys/217/guide_to_the_design_of_questionnaires

Taylor-Powell, E. (1998). Questionnaire design: asking questions with a
purpose.
http://learningstore.uwex.edu/pdf/G3658-02.pdf