National Seminar-Workshop on

Assessing Student Learning

Current Trends on Classroom
Application
of Test and Measurement
February 15-16, 2008
Venue: Institute for Small Scale Industries
(ISSI)
Virata Hall, E. Jacinto Street
UP Diliman, Quezon City
Prepared by:
Anunciacion V. Lara, Ph.D.
University of the Philippines
Diliman, Quezon City

Day Two

Session One
Topic: Determining the Quality of a Test through
Validity
I. Learning Targets
At the end of this session the participants
should be able to:
a. define the term validity
b. describe the types of validity
c. identify types of criterion-related validation
d. identify types of evidence used in
construct validation

I. Learning Targets
At the end of this session the participants
should be able to:
e. describe the methods used in the
validity of the interpretations from
the test scores
e. identify construct validation; and
f. appreciate the validity of test.

II. Session Notes

A. Definition and other features of
validity
Validity refers to the appropriateness of the
interpretation of the results of a test or
evaluation instrument for a given group of
individuals, and not to the instrument itself.
It is more correct to speak of the validity of
the interpretation to be made of the

results.

 Validity is a matter of degree.

Best considered in terms of categories that
specify degree, such as high validity, moderate
validity and low validity.
Is always specific to some particular use
interpretation.
Is as unitary concept based on various kinds of
evidence.

B. Approaches to Test Validation

Content-related evidence
Criterion-related evidence
Construct-related evidence
C. Types of criterion-related
validation (based on time difference
only)

D. Some types of evidence in construct

validation
Math Reasoning Test
Compare the test tasks to the task domain
(content-related evidence).
Have test takers Think Aloud as they answer
(to check on mental process).
Compare scores of math majors with scores of
non-majors.
Correlate scores of other math reasoning tests.
Correlate scores with scores of reading tests
(to rule out reading ability).
Correlate scores with math grades
(criterion-related evidence).

E. Methods used in Construct validation

Defining the domain of tasks to be measured.
Analyzing the mental process required for the
test items.
Comparing the scores of known groups.
Comparing scores before and after some
particular treatment.
Correlating the scores with other
measures.

F. Factors that lower the validity of the

interpretations from the test scores
Unclear direction
Reading vocabulary and sentence structure too
difficult
Appropriate level of difficulty of the test items
Poorly constructed test items
Ambiguity

F. Factors that lower the validity of the

interpretations from the test scores
Test items inappropriate for the outcomes
being measured
Inadequate time limits
Tests too short
Improper arrangement of items
Identical pattern of answers

Session Two
Topic: Determining the Quality of a Test through
Reliability
I. Learning Targets
At the end of this session the participants should
be able to:
a. define the term reliability;
b. define basic terminology: coefficient, validity
coefficient and reliability coefficient;
c. explain the procedure for determining reliability by
correlation methods;
d. discuss the methods of estimating reliability;
e. explain reliability demands and nature of the
decision; and
f.
appreciate the reliability of tests.

II. Session Notes

Definition and other features of reliability

Reliability refers to the results obtained with

an assessment instrument and not to the
instrument itself.
An estimate of reliability always refers to a
particular type of consistency.

 Test scores are reliable (or generalizable) over

different periods of time
different scores of questions
different raters
Tests scores are not reliable in general
The appropriate type of consistency in a
particular case is dictated by the use to be
made of the results.

B. Determining reliability by correlation

methods (refer to Attachment #1 Reliability)
C. Methods of estimating reliability
(refer to Attachment #2 Reliability)

D. Some basic terminology

Correlation coefficient
Validity Coefficient
Reliability coefficient
(refer to Attachment # 3- Reliability)

E. Reliability demands and nature of the

decision
High reliability is demanded when the
Decision is important.
Decision is final.
Decision concerns individuals.
Example:
Select or reject college applicant.

 Low reliability is tolerable when the

Decision is minor importance
Decisions concerns groups
Decision making is in early stage
Decision is reversible
Example:
Whether to review a classroom lesson.

Session Three
Workshop II: Exercises on Validity and
Reliability
I. Learning Targets
At the end of the session the participants
should be able to:
a. answer exercises on validity and reliability;

b.

calculate the Pearson

using both the
deviation score formula and the raw score
formula;

c.

interpret the Pearson

correlation coefficient.

appreciate

II.

Workshop II

1.

Discussion of exercise on reliability and

validity.
Discussion of the Pearson product-moment
correlation and the two formulae.

2.

r.

3.

Calculation of Pearson

4.

Interpretation of Pearson r.

Thank you.