Beruflich Dokumente
Kultur Dokumente
BUILDING VALIDITY IN
EDUCATIONAL AND SOCIAL RESEARCH
MUSTAFA AKIR
Introduction
664
tell us different things about the population impossible, but rather that there is no metaof the universe and about that population's linguistic formula for problem solving that
behavior. What a man sees depends both might be employed to compare standards
upon what he looks at and also upon what and values in all possible contexts (Bernhis previous conceptual experience. Incom- stein, 1991).
mensurability comes in three basic
"We must avoid the fallacy of thinkvarieties:
ing that since there are no fixed,
Incommensurability of Standards or
determinate rules for distinguishing
Cognitive Values
better from worse interpretations,
Incommensurability of Language
there is consequently no rational way
Incommensurability of Experience
of making and warranting such pracMeaning and understanding are
tical comparative judgments"
fundamental concepts in Gadamer's philo(Bernstein, 1991).
sophical hermeneutics. All understanding
Gadamer's hermeneutics will reveal that
involves interpretation and all interpreta- what is called the "incommensurability of
tion involves understanding. We are always paradigms" has nothing to do with relaunderstanding and interpreting in the light tivism. Instead of saying that individuals
of our anticipatory prejudgments and prej- are "locked" within socially constructed
udices, which are themselves changing in points of view, incommensurability charthe course of history (Bernstein, 1991). acterizes the "openness" of one's
Meaning comes to realization only in and perspective. It renders dialogue with the
through the "happening" of understand- "other" not only possible, but never ending.
ing. A horizon is limited but it is open. We
"Understanding must be conceived
are not trapped in our horizon, which is
as a part of process of the coming
changing and fluid, instead it functions as
into being of meaning, in which the
a window.
significance of all statements is
formed and made complete" (BernTheories of IVuth and Validity Concerns
stein, 1991).
As a word "validity" has been used in
This also can be misinterpreted as a so many difl'erent meanings in social scitype of relativism. Gadamer's reflections ences. One reason could be we invent new
on language, horizons, and historicity con- words for new kinds of validity concerns
tribute to undermining relativism. and redefine or use old words in new ways.
Understanding is limited but not closed; it Validity is not clearly dened in logic
is open to appropriating what is alien. No either. Usually it is defined by an example,
theoretical algorithm can provide "a point such as; a valid deductive conclusion is
by point comparison between theories or always true when the premises are true
cultures". In essence, what is meant by (Enerstvedt, 1989). From this example it
"incommensurability," then, is not that is very important to be aware of that a valid
rational comparison between paradigms is conclusion does not need to be true rather
a knowledge statement to its practical consequences. It asserts that what is tme can
be verified in practice and can be shown
to be useful for certain purposes. To pragmatists, tmth is whatever assists us to take
actions that produce the desired results
(Kvale, 1992).
The correspondence theory of tmth is
a realisfic theory as it makes the distinction between a theory and the facts, which
the theory describes; and it makes it possible to say that a theory is tme, or false.
It allows us to speak of a reality different
from the theory (Messick, 1989). Although
we have no criterion of tmth, and no means
of being even quite sure of the falsity of a
theory, it is easier to find out that a theory
is false than to find out that it is tme. We
have even good reasons to think that most
of our theories-even our best theories are,
strictly speaking, false, for they oversimplify or idealize the facts (Popper, 1999).
Popper accepts the correspondence theory of tmth; believe in absolute objective
truth. The aim of science, for him, is to
find theories, which get nearer to the tmth.
On the other hand, Kuhn rejects the notion
that tmth, in the sense of correspondence
to nature, is central to the debate between
competing paradigms. Kuhn's objection
was not just that no theory might be closer to tmth than another. It was that there is
no basis on which to judge that theories
are closer to the tmth (Kuhn, 1996).
As a consequence of giving up a correspondence theory of truth, validation
becomes investigation, providing arguments, questioning and theoretically
interpreting research findings.
In an introduction of his book. Against
(1946) stated some instruments are "obviously valid" and need no further study,
some researchers used the term "face validity" for this quality. Mosier (1947)
identified three distinct implication typically attributed to the term "face validity":
1) validity by assumption, 2) validity by
definition, and 3) appearance of validity.
He dismissed this "type" of validity as a
"pernicious fallacy." In 1950, Cruton
defined validity as the correlation of
observed scores on the test with true scores
in criterion. One major problem with validity coefficients was demonstrating the
relevance of the chosen criterion to the purpose of the testing (Suen, 2003). Another,
even more serious problem was demonstrating the validity of the criterion itself.
In 1951, American Psychological Association (APA) commissioned a panel to
offer test standards to be used in the construction, use, and interpretation of
psychological tests. The first product from
the Committee was the Technical Recommendations for Psychological Tests and
Diagnostic Techniques: A Preliminary
Proposal (APA, 1952). This publication
suggested four categories of validity: predictive validity, status validity, content
validity, and congruent validity. In 1954,
a joint committee of the APA, American
Educational Research Association
(AERA), and the National Council on
Measurements Used in Education (NCME)
was formed, and published the Technical
Recommendations for Psychological Tests
and Diagnostic Techniques (APA, 1954).
There were several modifications to the
1952 proposed recommendations. For
example, "Congruent validity" was
the social constmction of knowledge. Cronbach, who introduced the concept, has later
argued for an extended concept of constmct validity: it is an open process where
to validate is to investigate - "...validation
is more than corroboration; it is a process
for developing sounder interpretations of
observations". In an article where he argues
that value free standards for validity is a
contradiction in terms, Cronbach concludes
with a discursive concept of validity resting upon public discussion:
In 1974, AERA, APA, and NCME
revised the 1966 Standards, and published
the Standards for Educational and Psychological Tests. This revision retained the
notion of three unique "aspects" of validity. After publication of the 1974 Standards,
two schools of thought emerged regarding
validation theory: one school promoting
the idea that validity consisted of three
"separate but equal" aspects; the other
school advocating a unitary conceptualization centered on construct validity.
Standards for Educational and Psychological Testing (1985) emphasized a
unitary conceptualization of validity. In
this version, the "aspects" of validity denoted in the 1971 Standards were described
as "categories" of validation. All validity
is of one kind, namely constmct validity.
Other so-called separate types of validity
- whether labeled content validity, criterion-related validity, consequential validity,
or whatever - cannot stand alone in validity arguments. Rather, these so-called
validity types refer to complementary
forms of evidence to be integrated into an
overall judgment of constmct validity.
Evidential basis
Test interpretation
Test use
Construct validity
Construct validity
+ Relevance/utility
Consequential basis
Value implications
Social consequences
Note. From "Validity" by S. Messick, 1989b, In: R.L. Linn (Ed) Educational measurement, p.2O.
and the critical realist (or critical rationalist). The realist view is that theories can be
tested against facts that exist independent
of theories about them. The constmctivist
view does not deny the existence of a separate reality, but simply denies that there
are independent facts against which theories can be tested. The third view, critical
realism, holds that theories can be tested
against corresponding facts, despite the
fact that they are not fully independent of
the theory. For the realist, theoretical terms
are direct conjectures about unobserved
attributes that exist independent of scienfific theory. The realist accepts that such
conjectures cannot be verified, and instead
submits them to critical tests. The constmctivist judge tmth by the standard of
consistency between statements and the
usefulness of the resulting coherent set of
beliefs. We do test conjectures against
observafions, but not in the unproblematic way that the realist envisioned. Rather,
the process of testing is itself nested in a
specific set of pre-existing practices. This
is the constmctivist's contribution to the
synthesis. The hope is that although the
procedures for testing conjectures are relative to our specific practices, they will
still lead to rational decisions about the
conjectures. The function of test validation is to collect evidence and arguments
in support of, or counter to, proposed interpretations and uses of test scores (Messick,
1989). Messick's Singerian approach to
inquiry is a powerful means of maintaining the openness of validity theory.
Evidence derived from any of the research
traditions is important for validity. It serves
both a critical and a generative function, by
Copyright of Education is the property of Project Innovation, Inc. and its content may not be copied or emailed
to multiple sites or posted to a listserv without the copyright holder's express written permission. However,
users may print, download, or email articles for individual use.