Educational Research:

Descriptive Research

EDU 8603
Educational Research
Richard M. Jacobs, OSA, Ph.D.

Research...
The

systematic application of a
family of methods employed to
provide trustworthy information
about problems
an ongoing process based on many accumulated
understandings and explanations that, when taken
together, lead to generalizations about problems
and the development of theories

Quantitative research methods...

collect and analyze numerical data
obtained from formal instruments

Quantitative methods...
descriptive

research (survey research)

correlational

research

causal-comparative

research (ex post facto research)

experimental

research

 descriptive research

(survey research)

collects data in order to answer

questions about the current status of
the subject or topic of study
uses formal instruments to study
preferences, attitudes, practices,
concerns, or interests of a sample

The basic steps of descriptive

research...
An orderly scientific and disciplined
process, involving:
recognizing and identifying a topic to
be studied
selecting an appropriate sample of
participants
collecting valid and reliable data
reporting conclusions

Classifications of descriptive
research studies...
cross-sectional survey
survey
involves collecting data from
selected individuals in a single
time period however long it
takes to collect data from
participants

longitudinal survey
survey
involves collecting data at two or more
instances in order to measure changes
over time

self-report survey
survey
requires individuals to respond to a
series of statements or questions
about themselves

observation study
study
the researcher obtains data by
watching participants in a situation

Types of longitudinal surveys...

trend
cohort

survey

survey
panel survey
follow-up survey

 trend survey

a study where a sample is taken from

the general population in order to
collect data over time

involves different groups and

different samples over time

 cohort survey

a study where a specific population is

examined by sampling different groups
within the population over time

involves the same group but different

samples from that group over time

 panel survey

a study where the same group and the

same sample are examined over time

 follow-up survey

a study undertaken after a panel survey

and seeks to examine subsequent
development or change

Conducting a questionnaire study

1. state the problem
2. select participants
3. construct the questionnaire
4. prepare cover letter
5. pretest questionnaire
6. follow up activities
7. analyze/report results

1. state the problem

topic must be of sufficient significance to
motivate potential respondents and justify
the research effort in the first place

define topic in terms of specific

objectives indicating the kind of
information needed

2. select participants
use an appropriate sampling
technique
participants must be able to provide the
desired information sought and willing
to provide it to the researcher

3. construct the instrument

should be attractive, brief, and easy
to fill out
there are a variety of data collection methods
that can be used, including: questionnaires,
interviews, observations, and telephone calls

 types

of self-report items
scaled
ranked
checklist
free response

Designing an instrument
include a brief statement describing the
study and its purpose at the top of the
instrument

provide standardized directions

organize items leaving sufficient
white space on instrument

place similar items together

ask general, non-threatening items
first, moving into more specific, selfrevealing items

 some

dos and donts

construct items according to a set of
predetermined guidelines
include only items relating directly
to the purpose of the study
avoid jargon or any term or concept
that might mean different things to
different people

each item should deal with a single

concept and be worded as clearly as
possible
items should indicate a point of
reference beyond the self in order to
judge the self

avoid ambiguous terms like several

or usually

avoid touchy matters in items which

respondents might not respond to
honestly or at all
avoid leading questions which suggest
that one response may be more
appropriate than another

each item must be able to stand on

its own

dont jam items together

dont put the most important
questions at the end of the instrument

4. prepare the cover letter

a brief, neat, explanation of the
significance of the study and what is
being asked of the respondent and why

addressed to the specific, potential

respondent

an endorsement adds credibility

guarantee anonymity or
confidentiality
include a specific deadline date by
which the completed instrument
(questionnaire) is to be returned

individually sign each letter

include a self-addressed, stamped,
return envelope

5. pretest the instrument

conduct a pilot study to gather
information about deficiencies and
suggestions for improving the instrument

provides greater content validity

6. follow-up activities
because first mailing results tend to be low
(30% - 50% response rate), researchers need
strategies to increase the response rate

 initial

follow-up strategies to increase response rate up to 20%

send out reminder postcard

mail a second packet with a new,
positively worded cover letter and another
self-addressed, stamped, return envelope

 secondary

follow-up strategies to increase response rate up to 10%

telephone nonrespondents to
encourage them to respond

 dealing

with nonrespondents

study whether nonrespondents differ from

respondents in some systematic way by
selecting a sample from the nonrespondents

telephone sample, aggregate data,

include in report

 dealing

with nonresponses
study the items to determine the
problem with the item
include description of the
nonresponses and the determination of
the reason in the report

7. analyze/report results
discriminant item analysis includes giving the
response rate for each item as well as the total sample
size and the overall percentage of returns, since not all
respondents will answer questions

group items into clusters that address the same

issue and develop total scores across an item
cluster in order to avoid information overload

Conducting an interview study

1. state the problem
2. select participants
3. construct the interview guide
4. communicate professionally and record responses accurately

5. pretest interview procedure

6. analyze/report results

1. state the problem

topic must be of sufficient significance to
motivate individuals to participate and
justify the research effort in the first place

define topic in terms of specific

objectives indicating the kind of
information needed

2. select participants
use an appropriate sampling
technique
participants must be able to provide the
desired information sought and willing
to provide it to the researcher

3. construct the interview instrument (instrument guide)

guide

indicates the question to be asked, the

order, and how much additional
prompting or probing is permitted

the goal is to obtain standardized,

comparable data from each interviewee

4. communicate professionally and record responses accurately

effective interviewing requires training

and experience to avoid interviewer bias
and interviewer error

record responses manually on the

interview guide
(requires interviewee permission) use a
tape recorder or VCR to verify accuracy
of responses

5. pretest interview procedure

use a small group from the same population or a
similar population to the one being studied to validate
the interview instrument and the interviewers
communication and recording skills

6. analyze/report results
item analysis includes giving the response rate for
each item, as well as the total sample size and the
overall percentage of returns, since not every
respondent will answer all questions

also include in-depth data to open-ended

interview items to portray responses in
more accurate and honest terms

Conducting an observational study

1. state the problem
2. select participants
3. define the observational variables
4. record observations
5. assess observer reliability
6. reduce observer bias
7. analyze/report results

1. state the problem

topic must be of sufficient significance to
motivate potential respondents and justify
the research effort in the first place

define topic in terms of specific

objectives indicating the kind of
information needed

2. select participants
use an appropriate sampling
technique
participants must be able to provide the
desired information sought and willing
to provide it to the researcher

3. define the observational variables

operationalize the variables to be
observed in terms of specific behaviors
that can be quantified
ensure that the observations can be quantified in
a way so that all observers will count the
behavioral activities in exactly the same way

simplify procedure for recording

observations by developing a coding
procedure

4. record observations
record behavior at the time it occurs
alternate observation periods and
recording periods

5. assess observer reliability

use at least two observers who
make independent observations
compute interobserver reliability

6. reduce observer bias

train observers until a satisfactory level
of agreement is achieved (at least 80%)

monitor observers

 types

of observer bias
response set
halo effect
knowledge of participants

 response set
the tendency of an observer to rate the majority of
observees as above average, average, or below
average regardless of the observees actual behavior

 halo effect

the tendency of an observers initial positive

or negative impressions of the observee to
affect subsequent observations

 self-fulfilling prophecy

the tendency of an observers knowledge

of the observees or the purposes of the
study to affect the observation(s)

Meta-Analysis...
a

statistical approach to
summarizing the results of many
studies that have investigated
basically the same problem
provides a numerical coefficient
expressing the average result
of the studies

requiring the researcher to find, describe, classify,

and code the research studies to be included metaanalytic review, and for measuring and analyzing
the study findings

each studys results are translated into an effect

size (ES) which is a numerical expression of the
strength or magnitude of a reported relationship,
be it causal or not

ES =

X e Xc
SDc

Where

Xe = the mean score of the experimental group

Xc = the mean score of the control group
SDc = the standard deviation of the control group

Strength or magnitude of an effect

size ()

-.80

0.00

+.80

both groups strong positive

performed
similarly
experimental
control group
group performed
performed better
better than
than experimental
control group
group
strong negative

Mini-Quiz
True

or false

in a descriptive research study, the researcher

predetermines what variables will be surveyed
before selecting or observing the research
participants

True

one of the most difficult activities on the part of

questionnaire researchers is to write or select
questions that are clear and unambiguous

True

longitudinal surveys are useful for

studying the dynamics of a topic or
issue over time
True

one of the problems with longitudinal

studies is that the samples tend to
shrink as time goes by
True

one of the problems with cross-sectional

studies is selecting samples that truly represent
the population at a particular level or ability

True

an external review of an instrument

provides the researcher greater
assurance of it its content validity
True

if the responses from respondents and nonrespondents

are essentially the same, the researcher may assume
that the response group is representative of the whole
sample and that the survey results are generalizable

True

if the responses from respondents and nonrespondents

are different, the generalizability across both the
respondent and nonrespondent groups is not present
and must be discussed in the report

True

analyzing clusters of instrument

items related to the same issue
improves the reliability of the scores
True

bias can affect the validity of the

interpretations in observational
studies
True

although there are no hard and fast rules, it is generally

agreed than an effect size in the twenties indicates a
treatment that produces a relatively small effect,
whereas an effect size in the eighties indicates a
powerful treatment

True

 Fill

in the blank

a study requiring individuals to

respond to a series of statements or
questions about themselves
self-report study

 Fill

in the blank

a study in which individuals are not directly

asked for information but data is obtained as
the researcher watches participants engage in
a situation
observation study

 Fill

in the blank

items on a survey that are answered

by circling a letter, checking a list, or
numbering preferences
close-ended items

 Fill

in the blank

the researcher guarantees that no

one, including the researcher, knows
who completed each questionnaire
anonymity

 Fill

in the blank

the researcher guarantees that no one, except

the researcher, knows who completed each
questionnaire and promises not to divulge that
information
confidentiality

 Fill

in the blank

the oral, in person administration of

an instrument to each member of a
sample
interview

 Fill

in the blank

determining the current status of a

phenomenon not through an instrument but as
the researcher watches the participants
engage in the activity being studied

observational study

 Fill

in the blank

an observation in which the

researcher watches behavior as it
normally unfolds
naturalistic observation

 Fill

in the blank

an observation in which the researcher

creates a situation to be observed and tells
participants what activities to engage in

simulation observation

 Fill

in the blank

a coefficient found by dividing the total

number of agreed observations by the total
number of agreed and disagreed observations

inter-observer reliability

 Fill

in the blank

a means by which observers record

observations at the time it occurs by
simplifying the recording process
coding

 Fill

in the blank

a statistical approach that summarizes

the results of many studies having
investigate the same problem
meta-analysis

This module has focused on...

descriptive research
which involves collecting data in
order to test hypotheses or to
answer questions about the
opinions of people about some
topic or issue

The next module will focus on...

correlational research
...which involves collecting data in
order to determine whether, and to
what degree, a relationship exists
between two or more quantifiable
variables