How to Make Theories Untestable: A Guide to Theorists

Author(s): Clifford D. Shearing

Source: The American Sociologist, Vol. 8, No. 1 (Feb., 1973), pp. 33-37
Published by: American Sociological Association
Stable URL: http://www.jstor.org/stable/27702070
Accessed: 08-05-2017 15:58 UTC

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted
digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about
JSTOR, please contact support@jstor.org.

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
http://about.jstor.org/terms

American Sociological Association is collaborating with JSTOR to digitize, preserve and extend access to
The American Sociologist

This content downloaded from 148.202.10.146 on Mon, 08 May 2017 15:58:56 UTC
All use subject to http://about.jstor.org/terms
 HOW TO MAKE THEORIES UNTESTABLE:
A GUIDE TO THEORISTS*
CLIFFORD
University of Toronto
The American Sociologist 1973, Vol. 8, (Feb.) : 33-37
A cursory glance at the development of so-
ciological theory should indicate to any in-
dependent observer that any theorist who
seeks sociological fame must ensure that his
theories are essentially untestable. Despite
this, to my knowledge, all writers who have
addressed themselves to the question of theory
building-and this is, surprisingly, particu-
larly true in explicitly pedagogical texts-
have inexplicably concerned themselves exclu-
sively with the question of how to construct
testable theories. It is hoped that this paper
may held redirect attention to what is clearly
the more sensible question : how does one con-
struct untestable theories?
I propose here two methods for making a
theory untestable. To provide a rationale for
these two methods, it is necessary to discuss
briefly the nature of theory generally and sci-
entific theory in particular. 1 The term theory,
Kaplan (1964 :294) writes, "stands for the
symbolic dimension of experience, as opposed
to the apprehension of brute fact." However,
as Cohen (1968 :102) notes, not only do
theories go beyond fact but they do so by
virtue of their use of universal categories.
Scientific theories are a special type of theory
-Cohen (1968 :2) distinguishes three other
types-defined in terms of a set of normative
criteria to which scientists seek to conform
and in terms of which their theories are eval-
uated. Central among these is the principle
that scientific theories should be empirical.
That is, they should have an empirical refer-
*I would like to thank Robin Gerrish, John Howe,
Peter MacnaughtonSmith, Mike Petrunik, Judy
Posner, Jim Turk, Irvin Waller and Kurt Weis for
their comments on this paper.
1 Regretably, because of the total lack of literature
concerning how theories are to be made untestable,
in this paper I have been forced, albeit reluctantly,
in developing the argument, to refer to authors wlio
are concerned with constructing testable theories.
This content downloaded from 148.202.10.146 on Mon, 08 May 2017 15:58:56 UTC
All use subject to http://about.jstor.org/terms
D. SHEARING
ent-refer to what we call "the world"-and
be ''capable of being tested by experience''
(Popper, 1959 :40). Popper (1959 :40-42) has
argued that in view of the fact that scientific
theories consist of universal statements relat-
ing at least two variables-a further criterion
of a scientific theory-testability means, in ef-
fect, not that scientific theories must be veri-
fiable but rather that they must be '' falsifi-
able.'' A ''theory is empirical if it can, in
principle, be refuted by empirical observa-
tion. If it is irrefutable by observation, then it
is not strictly empirical" (Cohen, 1968 :3).
This discussion provides for the general prin-
ciple that, if a theory is to be untestable, it
must be non-empirical.
Now, because scientific statements are uni-
versal, and empirical observations are of par-
ticular events, if a theory is to be refutable
there must be some arrangement whereby
theoretical statements can be linked to ''or-
dinary propositions'' about empirical parti-
culars (Zetterberg, 1965 :79-86). In short, we
must be able to ''deduce statements of par-
ticular events from scientific theories, and to
check these by observation . . . '' (Cohen,
1968 :3). This does not mean that all state-
ments must be capable of being tested in this
manner but simply that one must be able
to deduce some empirically testable proposi-
tions.2
We are now in a position to state our first
method for making sociological theory un-
testable; that is, for making it unscientific:
make certain that no empirically refutable
statements can be deduced from the theory.
In other words, ensure that no statement about
the relationship between particular events
can be deduced from the theory (Bridgman,
2 The position taken here is explicitly rejected by
Popper who argues that every scientific statement
"Must be capable of being tested . " (Popper,
1959 :47) ; but is supported by Nagel (1961: 101-102).
33
34 THE AMERICAN
1927: Chapter 1; Nagel, 1961 :90) .3 There are
several ways in which this can be done and
we will discuss a few of them :
(i) Provide no operational definitions for
any of the terms in the theory. This means, in
effect, refusing to specify either the domain of
a concept (the "object" to which it applies)
or the content of the concept in empirical
terms (just what aspect of the "object" is
being referred to) or both. Zetterberg
(1965 :80) provides an excellent example of
what not to do in his discussion of how the
empirical content of a theoretical proposition
is specified by the operational definition of its
concepts. He illustrates the process by means
of the theoretical proposition, "Persons tend
to engage in actions that maintain the evalua-
tions they receive from their associates.''
Zetterberg demonstrates, by the specification
of some of the concepts involved, how one
moves from theoretical to ordinary proposi-
tions and hence towards testability. For ex-
ample, he suggests that one of the indices of
the concept ''evaluations'' is ''rank,'' where
" 'rank' is defined as an evaluation of a posi-
tion in a social structure'' and where there is
some means of operationally defining, by way
of particular measurement procedures, this
more specific concept ( 1965 :81). On the basis
of this analysis it is possible to move towards
a more testable proposition by rewriting the
above proposition as, ''Persons tend to engage
in actions that maintain the rank they enjoy
in their social structure'' ( 1965 :81) . See how
dangerous a seemingly innocuous move to-
wards operational definition can be! If this
process of operational definition of terms is
unwittingly continued the theorist will soon
find himself in the disastrous position of hav-
ing derived from his theory an empirically re-
futable proposition (1965 :82).
a Note that failure to provide concepts of a theory
with an empirical referrent does not mean, as for
example Bridgman has maintained (1927:Chapter 1),
that these concepts have no meaning, or even that
they do not have a valid scientific meaning. For,
besides operational definitions, there are two other
sources of meaning for scientific concepts: ''an ab-
stract calculus that is the logical skeleton of the
explanatory system, and that 'implicitly defines' the
basic notions of the system; ... and an interpretation
or model for the abstract calculus which supplies
some flesh for the skeletal structure in te-rms of more
or less familiar conceptual or visualizable materials"
(Nagel, 1961:90).
This content downloaded from 148.202.10.146 on Mon, 08 May 2017 15:58:56 UTC
All use subject to http://about.jstor.org/terms
SociOLOGIST
There are, of course, in the corpus of so-
ciological theory many examples of building
untestable theory that the novice can emulate:
examples where it is not possible to provide
central concepts with empirical content and
where consequently, testing the theory be-
comes difficult and perhaps impossible. An
area ripe with such example is structural-
functionalism. Philosophers of science, nota-
bly Hempel (in Gross, 1959 :271-307) and
Nagel (1961: Chapter 12), have analyzed care-
fully the logic of functional analysis and have
specified criteria which must be met if func-
tional analysis is to qualify as scienitfic. Two
of these are particularly relevant for this
paper: (a) one must specify the system in
question, that is, the domain to which the
analysis refers and (b) the specific goal state
in question. Zetterberg, in considering these
criteria refers to the following statement by
Merton, "Functions are those observed con-
sequences which make for the adaptation or
adjustment of a given system; and dysfunc-
tions those observed consequences which lessen
the adaptation of the system" (quoted in
Zetterberg, 1965 :74-75). Zetterberg (follow-
ing and referring to Galtung's unpublished
manuscript, no date) notes that the crucial
words in this statement are "adaptation" and
"adjustment" and the crucial question is
''adaptation'' and ''adjustment'' to what?
In terms of the criteria of scientific functional
analysis just given, this question can be con-
sidered in two parts: (a) what is the system
or sub-system in question? and (b) how would
"adaptation" or "adjustment" be recog-
nized? (That is, what criteria would we use
to identify them?) Merton, as a wise theorist
-whether this wisdom is manifest or latent
is not at issue here-does not provide a clearly
specified answer to either of these questions.
In the context of these two criteria of valid
functional analysis Talcott Parsons seems, at
first glance, to have blundered in developing
his theory of action. For, in this theory he
appears to have given considerable attention
to specifying systems and sub-systems of ac-
tion and to identifying the goal states of all
systems of action. These goal states are con-
ceptualized as four system problems that must
be solved if the system is to continue to main-
tain its boundaries vis-a-vis the environment
(Parsons, 1961 :38-41). However, once Par-
sons' theory is examined in detail it soon be-
 How TO MAKE THEORIES UNTESTABLE
comes appart>nt that it is not at all clear how
one would know whether these problems were
being solved independently of the mainte-
nance of boundaries (assuming for the mo-
ment that one could know when a system's
boundaries were being maintained), so that
it is difficult to see how one would test propo-
sitions relating these two concepts. In his
discussion of Parsons' four goal states, Zetter-
berg notes that in order to test Parsons' theory
it would be necessary to specify operationally
the nature of each goal state so that this state
could be reeognized. In order to allow his
readtrs to conceive of what such a state of
affairs would be like, Zetterberg invites them
to imagine that we ''can measure how well
each rfunctional problem l is resolved in a
social system by reading four master gauges,
and that all subsystems have similar gauges,
and that we also can rate the subsystems ac-
cording to the input and output they make
in these four quantities in the larger system"
( 1965 :77). Zetterberg 's point is that in order
to rigorously test this aspect of Parsons'
theory one would have to be able to do some-
thing like this and one simply can't imagine
how one would do this.
(ii) Prom:de no propositions in the theory.
Without propositions there clearly can be no
possibility of testing one's theory. This is a
particularly good method for one can create
the illusion of testability by introducing con-
cepts which can be clearly operationally de-
fined. However, the theorist should be warned
that it is difficult in practice to develop a
theoretical scheme which is no more than a
tnxonomy. Thus, I do not advise using this
method alone, despite its intrinsic attractions.
Notwithstanding its difficulty, however, many
socologicnl theorists have used this method
skillfully and have developed theories with
many more concepts than propositions. For
example, Romans (1967), Turk (1967) and
Zetterberg (1965) have pointed to the fact
that Parsons' theory is fundamentally a tax-
onomy rathPr than a set of propositions.
(iii) If one finds one has included proposi-
tions in a theory, a "second best" way to
makP the theory untcstable is to ensure that
the relationship between terms remains as un-
clear as possible. This method, of course,
should be used in conjunction with ( i) above
for best results. I suggest that an excellent
exmnple of an nttpmpt to usp just this method
is Parsons' comment (in Black, 1961 :311-363)
This content downloaded from 148.202.10.146 on Mon, 08 May 2017 15:58:56 UTC
All use subject to http://about.jstor.org/terms
35
on Black's paper ''Some Questions About
Parsons' Theories" (Black 1961 :268-288).
Black, in his paper, attempts the difficult task
of formulating, in propositional form, the
basic assumptions underlying Parsons' theory.
In responding to Black's formulation, Par-
sons sets himsPlf the task of providing ''a new
succinct statemt>nt'' of ''the frame of refer-
ence'' for action theory which the reader can
compare with Black's. Parsons, in his response
to Black, despite statements to the contrary,
successfully foils Black's attempt to produce
testable proposition by thoroughly and sys-
tematically muddying any clarity that Black
managed to produce. Parsons, thereby, deftly
defends his theory by negating Black's at-
tempt to move it towards testability.
George C. Homans is very adept at using
this method for producing untestable theory.
His particular forte lies in developing propo-
sitions which, on first glance, appear the very
model of clarity and precision but which, as
M:nris (1970) has recently shown, reveal a
subtle ambiguity when an attempt is made
to derive testable theories from them. One
strategy used by Romans is the useful tech-
nique of leaving vague the conditions under
which a relationship holds. For example, in
Homans' second postulate he introduces the
temporal qualifiPr "within a given period of
time'' and in his fourth postulnte he uses the
qualifier ''in the recent past.'' 4 Both qunli-
fiers help to preserve an essential element of
vagueness and ambiguity in seemingly very
precise propositions (Maris, 1970 :1071-1072).
Another strategy used by Homans is the
''more than-less than'' technique. By in-
troducino- the terms more and less into one's
propositi~ns ( eg. the more X, the less Y), one
gives theoretical statements an undeniable
propositional form. However, by not further
qualifying such propositions and stating how
much more or how much less, one makE's re-
futation more difficult (unfortunately, not
impossible and this is a clear defect in Ho-
4 The postulates in question are:
Postulate 2:
''The more often 'dthin a given period of time
a man's aetivity rewards the activity of any ot11er,
the more oftrn the other will emit the activity."
Postulate 4:
''The more often a man has in the recent past
received a rewarding activity from another, the less
valuable any further unit of that activity becomes to
him." (Maris, 1970 :1071)
36 THE AMERICAN
mans' theory). Further, so long as no quali-
fying assumptions specifying the. degree of
association between the variables are made,
logical deductions from propositions becomes
problematic (see Maris, 1970:1076; Duncan,
1963 :452) unless the stringent requirements
that one's propositions are stated in asymme-
tric causal form and logical closure can be
assumed (Costner and Leik, 1964) .
1 turn now from giving examples of the
first method of making theories untestable to
a statement of a second method for accom-
plishing this end. In discussing the first
method I have talked about testing theories
by testing propositions which are deduced
from the theory. This method of theory testing
rests on the assumption that the theory in
question is a logically closed system. In such
a theory all the assumptions necessary to de-
rive the empirical propositions are explicitly
stated and, further, each proposition has iden-
tifiable implications for all the other proposi-
tions in the theory. (For a discussion of the
concept of a closed theoretical system see Par-
sons, 1968 :10.) Only under these conditions,
as Popper (1959 :71-72) has indicated, is it
possible, by testing specific empirical proposi-
tions, to provide a severe test of the theory
as a whole. It is because of the importance
of this criterion of scientific theory that sim-
plicity and axiomatic formulation are stressed
by those naive theorists who stress the ip:l-
portance of testability. The importance of
axiomatic formulation follows from the fact
that only under these conditions is it possible
to assess whether or not a theory is closed,
and, if it is not closed, to determine where
the logical gaps are. This criterion for scien-
tific theory, that is, testable theory, is often
referred to as the criterion of internal con-
sistency. From this it follows that a second
method for making theory untestable is to
ensure that the theory is internally incon-
sistent. That is, take steps to ensure that one's
theory does not form a deductive system where
all the propositions of the theory can be de-
duced from a set of axioms. Under these con-
ditions, a test of any proposition, even if it
fulfills all the criteria of refutability (and
herein lies the beauty of this method), would
simply be a test of that particular proposition
and not the theory as a whole. In terms of this
criterion (and this is evidence of the wide-
spread historical use of this method) prac-
tically all major sociological theories remain
This content downloaded from 148.202.10.146 on Mon, 08 May 2017 15:58:56 UTC
All use subject to http://about.jstor.org/terms
SociOLOGIST
untestable. Romans' theory probably comes
as close as any to achieving internal consis-
tency or logical closure but even here, as
Maris (1970) has shown, in its present form,
assumptions still have to be added to Romans'
theory if it is to be stated in axiomatic rather
than verbal form. Towards the other end of
the continuum is Parsons' theory of action,
although Parsons, more than most other theor-
ists, works seriously at integrating the various
parts of his theory. For an example, see his
Working Papers in the Theory of Action
(1953) and "Pattern Variables Revisited: A
Response to Dubin" ( 1960). Even further
towards the inconsistency-end of the scale
are symbolic interactionists like Herbert
Blumer, who, in his use of the -Meadian con-
cept of the "I," provides for a built-in inde-
terminancy which prohibits the development
of a systematic set of propositions in deduc-
tive form.
In concluding this paper a w'ord of caution
is necessary about the use of these methods.
The methods just described assume a partic-
ular frame of reference : the positivistic ontol-
ogy, in which an external and independent
reality is postulated, and in which this postu-
late is used as the basis for the theory of evi-
dence which underlies the above discussion.
However, as Lewis Carroll and others have re-
peatedly pointed out in both fairy tales and
philosophy, these common sense assumptions
about the nature of reality can be suspended.
Once this is done and one steps ''through the
looking glass'' a new world is revealed : a
world with different questions, different en-
terprises and different rules of evidence. So,
before using these methods to evaluate theory,
be it one's own or others', be sure you know
on which side of the "looking glass" the
theorist is working; for these methods, either
as standards or guides to action, are applic-
able only on one side.
REFERENCES
Black, M. (ed.)
1961 The Social Theories of Talcott Parsons.
Englewood Cliffs, N.J.: Prentice-Hall.
Blau, P.
1964 Exchange and Power in Social Life. New
York: John Wiley and Sons.
Blumer, H.
1969 Symbolic Interactionism: Perspective and
Method. Englewood Cliffs, N.J.: Prentice-
Hall.
 COMPUTER SIMULATION IN TEACHING 37
Bridgman, P. theory. '' American Sociological Review 35
1927 The Logic of Modern Physics. New York: (August): 1069-1081.
Macmillan. Nagel, E.
Cohen, P. (ed.) 1961 The Structure of Science. New York: Har-
1968 Modern Social Theory. London: Heinne court, Brace and World.
mann. Parsons, T ., et a!.
Costner, H. L. and R. K. Leik 1953 Working Papers in the Theory of Action.
1964 ''Deductions from 'axiomatic theory.' '' Glencoe, Ill.: Free Press.
American Sociological Review 29 (Decem- Parsons, T.
ber) :819-835. 1960 ''Pattern variables revisited: a response to
Duncan, 0. D. Dubin.'' American Sociological Review 25
1963 ''Axioms or correlations~'' American So- (August) :467-483.
ciological Review 28 (June) :452. 1961 Theories of Society. T. Parsons, et a!., eds.
Gross, L. ( ed.) New York: Free Press of Glencoe.
1959 Symposium on Sociological Theory. New 1968 The Structure of Social Action. New York:
York: Harper and Row. Free Press.
Romans, G. Popper, K.
1961 Social Behavior: Its Elementary Forms. New 1959 The Logic of Scientific Discovery. London:
York: Harcourt, Brace and World. Hutchinson.
1967 The Nature of Social Science. New York: Turk, A.
Harcourt, Brace and World. 1967 ''On the Parsonian approach to theory
Kaplan, A. construction.'' Sociological Quarterly 8
1964 The Conduct of Inquiry. San Francisco: (Winter) :37-50.
Chandler. Zetterberg, H. L.
Maris, R. 1965 On Theory and Verification in Sociology.
1970 ''The logical adequacy of Romans' social Totowa, N.J.: Bedminster Press.

MANUSCRIPTS FOR THE

ASA ROSE SOCIOLOGY SERIES

Fellows and Active and Student members of the Association

may submit manuscripts of I00 to 300 typed pages for
publication in the ASA Arnold and Caroline Rose Mono-
graph Series in Sociology to the Series Editor, Sheldon
Stryker, Department of Sociology, Indiana University,
Bloomington, Indiana 47401.

This content downloaded from 148.202.10.146 on Mon, 08 May 2017 15:58:56 UTC
All use subject to http://about.jstor.org/terms