Tests and tools construction in

Educational Research
Course Instructor
Dr. Maksal Minaz
 Objectives
• After this session students will be able to
• Know about test
• Define test
• Understand instruments/tools
• To illustrate types of tests and instrument to
collect data
• To understand what type of instrument used to
collect data
• Steps involve in constructing an instrument
 Tests
• There are three major ways to collect data
• Administer a standardized instrument (developed by
experts and well constructed)
• Administer a self developed instrument
• Record naturally available data (factual information)
• A test is…..
• A means of measuring the knowledge, Skills, Feelings,
intelligence, or aptitude of an individual or group.
• Tests produce numerical scores that can be used to
identify , classify, or evaluate test takers.
• An instrument
• is a tool for measuring, observing, or documenting
quantitative data. Identify before the researchers
collect data, the instrument may be a test,
questionnaire, tally sheet, observational checklist,
inventory, or assessment instrument.
• Researchers use instruments to measure achievement,
assess individual ability, observe behavior, develop a
psychological profile of an individual, or interview a
person. In quantitative research, four major types of
information are gathered, as shown in Table
Types of Data Types of Tests, Definition of the Example of the
Instruments, or Type of Test, Specific Tests,
Documents to Collect Instruments, or Instrument, or Source
Data Document of Information
Measures of Achievement test: norm A test where the individual’s Iowa Test of Basic
Individual referenced grade is a measure of how well Skills
Performance tests he did in comparison with a
large group of test takers (Vogt,
2005)
Criterion referenced tests test where the individual’s General Educational
grade is a measure of how well Development
he or she did in comparison to a or GED Test
criterion or score
Intelligence test that measures an Wechsler intelligence
individual’s intellectual ability Scale for Children
Aptitude test to measure a person’s Cognitive ability:
ability to estimate how they will Binet–Simon
perform at some time in the Scale to identify a
future or in a different situation child’s mental level
Interest inventory that provides information Strong Interest
about an individual’s interests Inventory
and helps them make career
choices
Personality Assessment helps a person identify and Minnesota Multiphasic
measure human characteristics Personality
that help predict or explain inventory, sixteen
behavior over time and across personality
situations questionnaire
 Types of Tests

Measures of Affective scale An instrument that Attitudes toward

Individual measures Self-Esteem
Attitude positive or negative Scale
effect for or
against a topic
Observation Behavioral checklist An instrument used Flanders’
of Individual to record Interaction Analysis
Behavior observations about Behavioral Checklist
individual in Reading
behavior Vineland Adaptive
Behavior Scale
Factual Public documents Information from Census data
Information or public sources School grade
school records that provides data reports
about a sample School attendance
or population reports
 Types of Tests
• Achievement tests
• Measures the current status of individuals with respect to
proficiency in given area of knowledge or skill.
• Standardize achievement tests
• For individual curriculum areas like reading and mathematics,
and also in the form of comprehensive batteries that measure
achievement in several different areas
• Personality
• Designed to measure characteristics of individuals along a
number of dimensions and to assess feelings and attitudes
towards self, others and a variety of activities , institutions
and situations. The most used tests in educational research
• Non projective tests: self report measures such tests
ask individuals to respond to a set of questions or
statements. (frequently used in descriptive studies like
ratio of high school dropouts).
• Personality inventories
• Presents lists of questions or statements describing
behaviors characteristics of certain personality traits
and the respondents is ask to indicate “yes” “no”
undecided whether the statement describe him or her.
• Reading is one of my favorite pastimes.
• Attitude scales
• Attempt to determine what an individual believes, perceives or feel.
• 1. Likert scale ,
• The Likert scale is designed to examine how strongly subjects agree or
disagree with statements on a 5-point scale with the following anchors:
• Strongly Agree (SA), Agree (A) Undecided (U) Disagree (DA) Strongly
disagree (SD)
• 2. Semantic differential scale
• The semantic differential scale is used to assess respondents’ attitudes
toward a particular brand, advertisement, object, or individual. The
responses can be plotted to obtain a good idea of their perceptions.
• Responsive — — — — — — — Unresponsive
• Beautiful — — — — — — — Ugly
• Courageous — — — — — — — Timid
• 3.Thurstone scale: asks and individual to select
from a list of statements that represent different
point of view from those with which he or she is
in agreement.
• Each item has an associated point value between
1 and 11
• 4 Guttman scale: measures how much of a
positive or negative attitude a person has
towards a particular topic.
• (an individual is agree with a given statement also
agree with all related preceding statements)
• Creativity test
• To measure personality characteristics that are
related to creative behavior.(Thorrance tests
of creativity include graphic, or pictorial, item
as well as verbal items)
• Interests inventories; to indicates personal
likes and dislikes, like activities respondent like
or engage in.(vocational interest inventory)
• Aptitude: to predict how will someone is likely to perform in
a future situation. If an aptitude or ability test administered to
employees at the time of recruitment is to differentiate
individuals on the basis of their future job performance, then
those who score low on the test should be poor performers
and those with high scores good performers.
• General Aptitude: verbal and non verbal tests to measure the
ability to apply knowledge and solve problems.to measure
current ability ,
• Specific Attitude: attempt to predicted level of performance
that can be expected of an individual following future
instruction or training in a specific area.
• Such as algebra , music and mechanical ability.
• Factual Information
• Quantitative, numeric data is also available in
public educational records. Factual
information or personal documents consist of
numeric, individual data available in public
records. Examples of these types of data
include grade reports, school attendance
records, student demographic data, and
census information.
How to Decide What Types to Choose
• To select your data sources, ask yourself the following
questions:
• What am I trying to learn about participants from my
research questions and hypotheses? If you are trying to learn
about individual behaviors of parents at a student–parent
conference meeting, then you could use a behavioral checklist
and record observations. If you are trying to measure the
attitudes of teachers toward a bond issue, attitudinal
questions or an attitudinal instrument will be required.
• What information can you realistically collect? Some types
of data may not be collectible in a study because individuals
are unwilling to supply it. For example, precise data on the
frequency of substance abuse in middle schools may be
difficult to collect; identifying the number of student
suspensions for substance abuse is much more realistic.
 WHAT INSTRUMENT WILL YOU USE
TO COLLECT DATA
• It is more difficult to develop an instrument than to locate
one and modify it for use in a study. Modifying an
instrument means locating an existing instrument,
obtaining permission to change it, and making changes in it
to fi t your requirements.
• An instrument to measure the variables in your study may
not be available in the literature or commercially.
• Developing an instrument consists of several steps, such as
identifying the purpose of the instrument, reviewing the
literature, writing the questions, and testing the questions
with individuals similar to those you plan to study. The four
phases of development, recommended by Benson and
Clark (1983)
 Criteria for Choosing a Good
Instrument
• Several criteria can be used to assess whether it
is a good instrument to use. Ask yourself:
• Have authors developed the instrument
recently, and can you obtain the most recent
version? With knowledge expanding in
educational research, instruments over 5 years
old might be outdated.
• Is the instrument widely cited by other authors?
Frequent use by other researchers will provide
some indication of its endorsement by others.
 Criteria for Choosing a Good
Instrument
• Are reviews available for the instrument? Look for
published reviews about the instrument in the MMY or in
journals such as Measurement and Evaluation in
Counseling and Development. If reviews exist, it means that
other researchers have taken the instrument seriously and
seek to document its worth
• Is there information about the reliability and validity of
scores from past uses of the Instrument
• Does the procedure for recording data fi t the research
questions/hypotheses in
• your study?
• Does the instrument contain accepted scales of
measurement?
 Steps in Developing or Constructing
an Instrument
• Phase I: Planning
• State purpose of test and target groups
• Identify and define domain of test
• Review literature on construct or variable of
interest
• Give open-ended questions to target group
• Interpret open-ended comments
• Write objectives
• Select item format
 Phase II: Construction
• Develop table of specifications
• Hire and train item writers
• Write pool items
• Validate content
• Have judges complete qualitative evaluation
• Develop new or revise items
 Phase III: Quantitative Evaluation

• Prepare instrument for first pilot test

• Administer first pilot test
• Debrief subjects
• Calculate reliability
• Run item analysis
• Revise instrument
• Prepare for second pilot test
 Phase IV: Validation/Reliability

• Administer second pilot test

• Run item analysis
• Reliability
• Repeat steps of revision, pilot administration,
and item analysis
• Begin validation
• Administer for validation data
• Continue validation and reliability