Instrumentation

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Instrumentation
Chapter Seven
Instructions: Circle the choice that indicates your opinion.
1. Teachers unions should be abolished.
Strongly
agree
(5)

Agree
(4)

Undecided
(3)

Disagree
(2)

Strongly
disagree
(1)

2. School administrators should be required by law to teach at least one class in a

public school classroom every year.
Strongly
agree
(5)

Agree
(4)

Undecided
(3)

Disagree
(2)

Strongly
disagree
(1)

3. Classroom teachers should be able to choose the administrators in their schools.

Strongly
agree
(5)

McGraw-Hill

Agree
(4)

Undecided
(3)

Disagree
(2)

Strongly
disagree
(1)

2006 The McGraw-Hill Companies, Inc. All rights

What are Data?

Data refers to the information researchers

obtain on the subjects of their research.
Demographic information or scores from a
test are examples of data collected.
The researcher has to determine what kind
of data they need to collect.
The device the researcher uses to collect
data is called an instrument.

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Key Questions

The instruments and procedures used in collecting data

is called instrumentation.
Questions arise regarding the procedures and
conditions under which the instruments will be
administered:

Where will the data be collected?

When will the data be collected?
How often are the data to be collected?
Who is to collect the data?

The most highly regarded types of instruments can

provide useless data if administered incorrectly, by
someone disliked by respondents, under noisy,
inhospitable conditions, or when subjects are
exhausted.

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Validity, Reliability, and Objectivity

Validity is an important consideration in the choice of an

instrument to be used in a research investigation

Reliability is another important consideration, since

researchers want consistent results from
instrumentation

It should measure what it is supposed to measure

Researchers want instruments that will allow them to make
warranted conclusions about the characteristics of the subjects
they study

Consistency gives researchers confidence that the results

actually represent the achievement of the individuals involved

Objectivity refers to the absence of subjective

judgments

This should be eliminated with regards to achievement of the

subjects, even if its difficult to do so

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Usability

An important consideration for any researcher in choosing

or designing an instrument is how easy the instrument will
actually be to use.
Some of the questions asked which assess usability are:

How long will it take to administer?

Are the directions clear?
How easy is it to score?
Do equivalent forms exist?
Have any problems been reported by others who used it?

Getting satisfactory answers can save a researcher a lot of

time and energy.

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Ways to Classify Instruments

Who Provides the Information?

Themselves: Self-report data
Directly or indirectly: from the subjects of
the study
From informants (people who are
knowledgeable about the subjects and
provide this information)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Types of Researcher-completed
Instruments

Rating scales
Interview
schedules
Tally sheets
Flowcharts

McGraw-Hill

Performance
checklists
Anecdotal records
Time-and-motion
logs
Observation forms

2006 The McGraw-Hill Companies, Inc. All rights

Excerpt from a Behavior Rating Scale for Teachers

(Figure 7.4)
Instructions: For each of the behaviors listed
below, circle the appropriate number, using
the following key: 5 = Excellent, 4 = Above
Average, 3 = Average, 2 = Below Average,
1 = Poor.
A. Explains course material clearly.
1

B. Establishes rapport with students.

1

C. Asks high-level questions.

1

D. Varies class activities.

1

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Excerpt from a Graphic Rating Scale

(Figure 7.5)
Instructions: Indicate the quality of the students participation
in the following class activities by placing an X anywhere along
each line.
AlwaysFrequently Occasionally

Seldom

Never

1. Listens to teachers instructions.

Always

Frequently

Occasionally

Seldom

Never

Seldom

Never

2. Listens to the opinions of other students.

Always

Frequently

Occasionally

3. Offers own opinions in class discussions.

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Example of a Product Rating Scale (Figure 7.6)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Interview Schedule (Figure 7.7)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Sample Observation Form

(Figure 7.8)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Discussion Analysis Tally Sheet (Figure 7.9)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Participation Flowchart (Figure 7.10)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Performance Checklist Noting Student Actions (Figure 7.11)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Time-and-Motion Log (Figure 7.12)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Types of Subject-completed
Instruments

Questionnaires
Self-checklists
Attitude scales
Personality
inventories

McGraw-Hill

Achievement/aptitu
de tests
Performance tests
Projective devices
Sociometric
devices

2006 The McGraw-Hill Companies, Inc. All rights

Example of a Self-Checklist (Figure 7.13)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Example of Items from a Likert Scale

(Figure 7.14)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Example of the Semantic Differential

(Figure 7.15)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Pictorial Attitude Scale for Use with

Young Children (Figure 7.16)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Sample Items from a Personality

Inventory (Figure 7.17)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Sample Items from an Achievement Test

(Figure 7.18)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Sample Item from an Aptitude Test

(Figure 7.19)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Sample Items from an Intelligence Test

(Figure 7.20)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Example from the Blum Sewing Machine Test

(Figure 7.21)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Sample Items from the Picture

Situation Inventory (Figure 7.22)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Example of a Sociogram (Figure 7.23)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Example of a Group Play (Figure 7.24)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Item Formats

Questions used in a subject-completed

instrument can take many forms but are
classified as either selection or supply items.
Examples of selection items are:

True-false items
Matching items
Multiple choice items
Interpretive exercises

Examples of supply items are:

McGraw-Hill

Short answer items

Essay questions

2006 The McGraw-Hill Companies, Inc. All rights

Unobtrusive Measures

Many instruments require the cooperation of the

respondent in one way or another.
An intrusion into an ongoing activity could be involved
which causes a form of negativity within the respondent.
To eliminate this, researchers use unobtrusive measures,
data collection procedure that involve no intrusion into
the naturally occurring course of events.
In most cases, no instrument is used, however, good record
keeping is necessary.
They are valuable as supplements to the use of interviews
and questionnaires, often providing a useful way to
corroborate what more traditional data sources reveal.

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Types of Scores

Quantitative data is reported in the form of scores

Scores are reported as either raw or derived scores

Raw score is the initial score obtained

Taken by itself, a raw score is difficult to interpret, since it has little

meaning

Derived score are scores that have been taken from raw scores
and standardized

They enable researchers to say how well the individual performed

compared to others taking the same test
Examples include:

McGraw-Hill

Age and Grade-level Equivalents

Percentile Ranks

Standard scores are mathematically derived scores having

comparable meaning on different instruments

2006 The McGraw-Hill Companies, Inc. All rights

Examples of Raw Scores and

Percentile Ranks (Table 7.1)
Raw
Score
95
93
88
85
79
75
70
65
62
58
54
50

Cumulative
Frequency
1
1
2
3
1
4
6
2
1
1
2
1

Percentile
Frequency
25
24
23
21
18
17
13
7
5
4
3
1

Rank
100
96
92
84
72
68
52
28
20
16
12
4

N = 25
McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Norm-Referenced vs. CriterionReferenced Instruments

All derived scores give meaning to individual scores by

comparing them to the scores of a group.
The group used to determine derived scores is called the
norm group and the instruments that provide such scores
are referred to as norm-referenced instruments.
An alternative to the use of achievement or performance
instruments is to use a criterion-referenced test.
This is based on a specific goal or target (criterion) for
each learner to achieve.
The difference between the two tests is that the criterion
referenced tests focus more directly on instruction.

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Measurement Scales

There are four types of measurement

scales

McGraw-Hill

Nominal Scales
Ordinal Scales
Interval Scales
Ratio Scales

2006 The McGraw-Hill Companies, Inc. All rights

Four Types of Measurement Scales

(Figure 7.25)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Four Types of Measurement Scales

(Table 7.2)
Measurement
Scale

Characteristics

Nominal

Groups and labels data only;

reports frequencies or percentages.

Ordinal

Ranks data; uses numbers only to

indicate ranking.

Interval

Assumes that equal differences between

scores really mean equal differences in
the variable used.

Ratio

All of the above, plus true zero point.

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Nominal Scales

Considered the simplest form of

measurement.
Researchers assign numbers to different
categories.
An example would be to assign a 1 to
men, a 2 to women.
The advantage to assigning numbers to
the categories is to facilitate computer
analysis.
Involves groups and labels data only.

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

A Nominal Scale of Measurement (Figure 7.26)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Ordinal Scales

Involves the use of numbers to rank or order

scores from high to low.
An example would be the ranking of high to low
scores on an examination.
Differences in rankings would not necessarily be
the same with certain scores.
Ordinal scales indicate relative standing among
individuals.

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

An Ordinal Scale: The Winner of a

Horse Race (Figure 7.27)

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Interval Scales

Involves the use of numbers to represent

equal intervals in different segments on a
continuum.
Very similar to an ordinal scale with the
exception of the equal intervals of points.
Assumes that equal differences between
scores really mean equal differences in the
variable measured.

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights

Ratio Scales

A ratio scale involves the use of

numbers to represent equal distances
from a known zero point.
A scale designed to measure height
would be a ratio scale since the zero
point represents the absence of height.
Ratio scales are almost never
encountered in Educational Research.

McGraw-Hill

2006 The McGraw-Hill Companies, Inc. All rights