Perception, 1983, volume 12, pages 269-279

R L Gregory and others: the wrong picture of the picture

theory of perception

Stuart Katz
Department of Psychology, University of Georgia, Athens, GA, USA
Received 6 July 1982, in revised form 24 November 1982

Abstract. It is the position of R L Gregory and other cognitive theorists that perceptual knowledge
conceived as an inner picture leads to an infinite regress, but that perceptual knowledge conceived
as an abstract or coded representation does not. It is argued here that this view is mistaken.
All inner representations, whether pictorial or abstract, lead to the regress because all representations,
inner or outer, require interpretation, and hence an interpreter. The problem will not disappear,
furthermore, by formalizing the representation because rule-following is not equivalent to
interpretation. The regress can only be avoided if the whole organism is made the interpreter, and
representations are given their appropriate place: in the external world, not inside heads.
What one wishes to say is: "Every sign is
capable of interpretation; but the meaning
mustn't be capable of interpretation. It is
the last interpretation".
Wittgenstein in The Blue Book

On the very first page of the third edition of R L Gregory's acclaimed Eye and Brain
(1978) we are introduced to more philosophical sophistication than we often get in
the writings of psychologists. On that page Gregory makes an interesting, though not
new, observation about the 'picture theory' of perception. He says this:
"The task of eye and brain is quite different from either a photographic or a
television camera converting objects merely into images. There is a temptation,
which must be avoided, to say that the eyes produce pictures in the brain. A
picture in the brain suggests the need of some kind of internal eye to see it—but
this would need a further eye to see its picture ... and so on in an endless regress
of eyes and pictures. This is absurd", (page 9)
The picture theory, and the obvious regress to which it invariably leads, is, as I
have mentioned, not new. Descartes made a similar observation three centuries
ago, (1) and one can find many references to the regress in the contemporary
philosophical literature, especially in the penetrating accounts of Wittgenstein (1953)
and Malcolm (1977). (For a general discussion of the infinite regress, and its
limitations, see Passmore 1969.) But that does not mean that the insight cannot still
be a useful reminder to us. For, while casual recognition is sometimes given to the
work of the aforementioned school of philosophy, it remains nonetheless true that all
manner of logical oddities, including the infinite regress, can still be found in the
cognitive theories of perception, memory, and thought. Indeed, the theory of
knowledge known as 'representative realism' (see Hirst 1959 for a general account),
which embraces nearly all contemporary psychological theory, is a rich source of
these difficulties.

W Gregory points out in his recent book Mind in Science (1981) that one can go back even further
to Theophrastus (c 372-286 BC) who faults Empedocles for an auditory picture theory of hearing.
The basis of his criticism is, surprisingly, the infinite regress. For a brief but informative discussion
of early accounts of this issue see pages 359-362 of this book.
270 S Katz

My reason for bringing Gregory's remarks to the attention of the reader, however,
is not simply as a reminder of the regress, but to ask whether Gregory has, in fact,
avoided it. Since the rest of Eye and Brain is devoted to his own version of the
representative realist epistemology, we might suspect that he has not. His widely
known theory that in perception the brain encodes sensory data, generates hypotheses,
and then draws conclusions that give rise to perception, is vintage representative realism,
and as such is subject to many of the faults familiar to philosophical students of
perception. But, as a matter of fact, we do not need to read the rest of Gregory's
book to answer the question. On the very same page where he introduces us to the
regress of pictures, precisely one paragraph earlier, he says the following:
"We are so familiar with seeing, that it takes a leap of imagination to realize that
there are problems to be solved. But consider it. We are given tiny distorted
upside-down images in the eyes, and we see separate solid objects in surrounding
space. From the patterns of stimulation on the retinas we perceive the world of
objects, and this is nothing short of a miracle", (page 9)
By "tiny distorted upside-down images" Gregory, of course, means the retinal
images or, in other words, retinal pictures. And when he says "We are given ..."
such pictures, he means they are presented to each of us, the perceivers, as the first
visual pieces of information of the world with which we make contact. But if each
of us as a perceiver is presented with two upside-down little pictures in the back of
our eyes, then as Gregory himself tells us in the succeeding paragraph, from which
we first quoted him above, a picture suggests the need for some further eye to see it.
There would, therefore, have to be an inner eye or eyes to look at the retinal pictures,
and " ... this would need a further eye to see its picture ... and so on in an endless
regress of eyes and pictures". Thus it might seem that Gregory commits the very
same error of which he accuses those who postulate pictures in the brain. He has
simply relocated the pictures from the central to the peripheral regions of the nervous
system.
Now the reader may protest that Gregory, like other cognitive theorists, does not
fall victim to the regress because he does not, even unwittingly, postulate another
pair of eyes behind the retinal pictures. And that, it turns out, is exactly right/ 2)
Instead, Gregory says, the brain encodes an abstract representation of these images.
To quote him once again:
"What the eyes do is to feed the brain with information coded into neural activity—
chains of electrical impulses—which by their code and the patterns of brain
activity, represent objects No internal picture is involved", (page 9)
Using an information-processing model, therefore, he has replaced the logically
objectionable picture in the brain with the logically acceptable abstract neural
representation.
But none of this will do. If it is reasonable to terminate the regress by postulating
a coded representation behind the picture on the retina, then it is also reasonable to
terminate the regress one or more steps later by postulating a coded representation
'behind' the picture in the head. The fact that we can observe a picture on the
retina, but have as yet found none in the brain, in no way precludes the latter as a
possibility, especially since the form in which the picture might be found has never
been elucidated. We may wish, as nearly everyone does, to reject the old Gestalt
brain field isomorphism, but there are still other possibilities, subject only to the
limits of the geometric imagination.

(2)
See footnote 1; also page 592 of Gregory (1981).
The picture theory of perception 271

The real problem with Gregory's argument, however, does not reside in chosing a
location for a pictorial terminus ad quern and an information-processing terminus a
quo. The regress of eyes and pictures has, at its basis, nothing to do with any
peculiar or unique property of eyes nor of 'visual' pictures, but rather with a
characteristic of all representations in whatever form they may be found. That
characteristic is this: Every presentation, and every representation, of the external
world requires an interpretation; no representation alone ever 'shows its sense' as
Wittgenstein (1922) once asserted and later rejected. Every representation admits of
more than one interpretation, depending on the uses to which the interpreter puts it.
Consider, for example, a railroad timetable. It represents, usually in the form of a
list of numbers, the arrivals and departures of trains at one or more locations over a
given time period. The timetable is usually in printed form, and is therefore read
by the traveller, but it need not be; it could be announced over a loudspeaker system
(and, in fact, it usually is) in bits and pieces, over a stretch of time, as trains actually
arrive and depart. It contains information that is independent of any one sensory or
perceptual system, and it stands for, or represents, states of affairs. Now let us ask
what might at first seem an odd question: Could the timetable be interpreted
differently? Let us assume, for example, that our traveller has missed his train, and
the next is not scheduled to leave the station for several hours. He has brought no
reading materials with him, and the newsstand has closed for the evening. The
station, moreover, is practically deserted, leaving no opportunity for conversation
with others. How might our traveller occupy himself while waiting? Well, he does
have the timetable in hand. What can he do with it? He might, for example, make
a game of counting the vowels, or consonants, or numbers on it to see which occur
most frequently, which least frequently. It may seem silly to resort to such a
triviality, but we ought never underestimate the pain of boredom, nor our reliance on
trivia (soon, themselves, to go the way of boredom) to relieve it. For our purposes,
however, what is important is that we have discovered a second (if insignificant) use
for the timetable. Nor does that by any means exhaust the number of possible
further uses, but I leave it to the reader to imagine others.
Our next query is: What determines which interpretation is given in a particular
situation? Surely there is nothing about the timetable itself that determines this, for
it is always the same piece of paper with the selfsame markings.(3) The answer, of
course, is that the circumstances determine what the interpretation will be. If the
traveller must know when his train leaves, he may use the paper with the markings
on it to get that sort of information. If, however, he is stuck in an empty train
station for some period of time, and with little means of occupying himself, he may
use the very same paper-with-markings to relieve the boredom, by counting the
characters on it.
And that is the point of this example; for as it is with a railroad timetable, so it
is with anything that represents: A representation is inherently ambiguous, whether
it is in the brain, on the retina, or on the walls of the Louvre; whether it is
embodied as neurons, magnetic tapes, or carvings on stone; whether it is about
timetables, musical scores, or landscapes. If this were realized by information
processing theorists, then they would also realize that abstract brain codes would be
as fully objectionable as brain pictures. For a neural code, like a picture in the brain,
like the retinal image, like a timetable, is a representation, and every representation
permits more than one interpretation, and where there is more than one interpretation,
there is need for an interpreter. And if this is so, then an abstract neural representation
(3) Indeed, to call it that, namely a piece of paper with the selfsame markings, is yet another
interpretation of the timetable, one that might be used in the sort of argument found in the
present paper.
272 S Katz

will itself require an interpreter, which will need a further interpreter to see its
interpretation "... and so on in an endless regress ...". Gregory therefore cannot
escape the regress, even if he decides to change, at some convenient place in the
information processing sequence, from visual representations (the retinal images) to
neural representations.
These difficulties notwithstanding, Gregory is supported by arguments expressed in
many places in the literature of cognitive psychology. Two especially influential
essays are due to Fodor (1975) and Pylyshyn (1973), respectively, and it would be
useful to consider briefly what these writers have to say about the matter. Perhaps
they can offer something further which will strengthen Gregory's position. (For
more extensive analyses of the Fodor essay see Dennett 1978; Heil 1981.)
In both essays difficulties are once again acknowledged for inner pictures, but
denied for noniconic, abstract representations. Citing Wittgenstein, for example,
Fodor argues that the very same picture which depicts, say, John as fat, may also
depict John as tall, or John as standing up, or John as pregnant, or John as having
an indefinite number of other possible properties, some of which happen to be true
in some particular world, and others false. Pictures, in other words, are by their very
nature ambiguous; there are any number of propositions about them which may be
true or false. Fodor, however, asserts that linguistic signs are not ambiguous:
"... symbols (as opposed to icons) are exempt from these worries; that's one of the
respects in which symbols really are abstract. A picture of fat John is also a
picture of tall John. But the sentence 'John is fat' abstracts from all of John's
properties but one: It is true if he's fat and only if he is", (page 181)
Fodor is correct about pictures, but not correct about linguistic signs. A picture
is always potentially ambiguous, but so is a sentence. The sentence "John is fat"
may mean that John is obese, but it may also mean that John is prosperous, or lazy,
or—if the writer intends irony—even skinny. A sentence does not differ from a
picture in this respect. And what is true of linguistic signs is true of other symbols
as well.
Pylyshyn, on the other hand, agrees that both pictures and sentences require
interpretation. But, he argues, there is something that is unambiguous, to wit, the
proposition. A picture or a sentence may be equivocal; but a proposition itself,
which a sentence or picture is intended to express, is unequivocal (1973, pages 4-8).
And it is proposition-like mental representations that are stored and computationally
manipulated inside our heads. Is this assertion correct? Have we found the context-
free self-interpreting symbol at last, the representation that 'shows its sense'?
Let us assume, with Pylyshyn, that knowledge is comprised of proposition-like
mental representations. These representations will, of course, have to be embodied
in some way as brain activity, whether as ensembles of neuronal firings, as holograms, as
large molecules, or in some other guessed or as yet unguessed manner. But how, then, is
this embodiment to be of use to its possessor? Will it not have to be first interpreted?
If one cannot discover the propositional content in a sentence without interpretation,
how can one discover the propositional content in the brain embodiment without
interpretation? Neural activity no more transparently reveals what it represents than
graphite characters on paper transparently reveal what they represent. A proposition
may not itself require interpretation (because, one might say, it is the interpretation)
but the proposition is not a picture, nor a sentence, nor'the physical states of a
computer (see Heil 1981; Searle 1980) nor the physical states of a brain. Propositional
content may be embodied in any of these ways, and in countless others, but the
embodiment will always require an interpretation. And this takes us to the heart of
the matter. For if the embodiment does take the form of brain states, then, since
The picture theory of perception 273

brain states are inside our heads, we shall have to ask who will interpret what is
represented therein, and then who will interpret the representations of that
interpreter "... and so on in an endless regress ...". It seems that Fodor and
Pylyshyn have not helped us to solve Gregory's problem. The regress persists.
But cognitivists will not let the matter end there. It is true that the problem of
the infinite regress has always been a difficult one, hovering ominously, like a dark
cloud, over the information-processing enterprise. Dennett (1978) has called it
'Hume's problem', for Hume himself, more than two centuries ago, saw that inner
representations cannot stand alone, but require a user or interpreter. And if that is
the case, it seemed, then the regress follows. But modern cognitivists are not longer
intimidated by the regress because—they claim—a major technological innovation has
shown how to avoid it. That innovation is ... the digital computer. Computers, the
argument goes, have inner representations, and yet they function perfectly well.
They constitute, in other words, an 'existence proof that the regress can be avoided.
If something exists, after all, then what it does is actual, hence possible. Since
computers exist, it is therefore possible for there to be cognition, with its centerpiece,
the inner representation.
But what, exactly, is it about representations in computers that enables them to
dodge the regress? There is a certain important phrase in the argot of cognitivism
which is supposed to be the answer. The phrase is "self-understanding representation"
(Dennett 1978). Here is what it means:
"... as many writers in AI [Artificial Intelligence] have observed, it has gradually
emerged from the tinkerings of AI that there is a trade-off between sophistication
in the representation and sophistication in the user. The more raw and uninterpreted
the representation—eg, the mosaic of retinal stimulation at an instant—the more
sophisticated the interpreter or user of the representation. The more interpreted
a representation—the more procedural information is embodied in it, for instance—
the less fancy the interpreter need be. It is this fact that permits one to get away with
lesser homunculi at high levels, by getting their earlier or lower brethren to do some
of the work", (page 124)
In other words, by specifying in advance the procedures, or rules, which characterize
the formation and the transformation of a representation, homunculi or interpreters can
be eliminated or, as Dennett says, "discharged". The representation now stands on its
own; it is 'self-understanding'. Dennett has thus elevated an important programming
dictum (if you want a program to work you must specify unambiguously what it is you
want it to do) to a law about the nature of understanding, and he has graced the law
with the poetic phrase "self-understanding". Dennett has, of course, correctly described
digital computers. But he has not got the character of homunculi or interpreters right.
A program, as he says, is a set of rules devised by a programmer that "discharges fancy
homunculi...". A computer, specifically its physical embodiment, acts in accordance
with those rules. We might speak metaphorically, as we shall do for present purposes
(though this is not literally correct), and say that the computer 'follows' the rules.
Now if computers follow rules, is that what persons do when they understand? Is
understanding or interpreting, in other words, the same as rule-following? The simple
and elegant thought experiment posed by Searle (1980) gives us the answer. Searle
conceives a situation where a person plays the role of a computer. This person is
confined to a room where he receives from the outside, and gives back, funny squiggles
on paper that are meaningless to him. He is able, however, to use the squiggles he
receives to find the squiggles he gives back because he is also given a set of rules in his
native tongue that tell him precisely how to do this. In computer terminology, the
squiggles he receives are the 'input', the squiggles he gives back are the 'output', and the
274 S Katz

rules he follows are the 'program'. Now it so happens, in this thought experiment,
that the squiggles are authentic Chinese characters, and the rules he follows are
ingeniously constructed so as to make the input and output appear, from the
outside, and to someone who understands Chinese, to be perfectly good conversation.
Now we ask: Does the person inside the room understand Chinese? We can find
out simply by bringing him out of the room, and then trying to continue the
conversation in writing. Our protagonist, we discover, will not be able to carry it on.
And when we ask him (in his native tongue), why, he will give the right answer: He
has no idea what the squiggles mean. Understanding, therefore, is not rule-folio wing,
and Dennett's reduction of the former to the latter is therefore incorrect.
The cognitivists have fooled us, and themselves, by an ill-conceived identity—
understanding is rule-following—and have concluded that if procedures governing
the formation and transformation of representations are completely specified in the
program, the representation is "self-understanding". But as we have seen, reduction
to rules is not sufficient for understanding, for one can follow a set of rules and
have no understanding whatsoever/ 4) We may therefore assert, as we asserted
earlier, that representations are not interpretable without an interpreter. And since
a representation requires an interpreter, an inner representation will require an inner
interpreter "... and so on in an endless regress ...". The regress will not go away.
What then is the appropriate use of the concept of representation? Is it fine
metaphor, but nonsense under philosophical scrutiny? Consider, once again, the
kind of representation with which we began this discussion, namely the visual
picture. When we refer to such pictures in the ordinary sense, say when we refer to
pictures on the walls of the Louvre, do we get into the same difficulties as when
we speak about the other 'pictures' we have mentioned? Can we talk about
pictures at all, or must we remain in fearful silence, lest we fall helplessly into
the regress? There is, I believe, a straightforward answer to these questions.
The objects that in ordinary discourse we call pictures, for example those
objects found on the walls of museums, can be thus called without leading to a
regress, while the extraordinary objects called pictures by perceptual theorists
cannot. Why? Pictures on walls do have eyes to look at them, or to put it
more precisely, pictures on walls have perceivers who see them with their eyes
located in their heads situated atop their torsos (Gibson 1966, 1979). There is no
regress, provided that pictures are assigned their rightful places: on walls, or in books,
or on television or movie screens, or in atlases, but not, after all, in brains or on retinas.
There is no mind's eye; there are just ordinary eyes which are used for seeing objects
and events in the world. More importantly, this is true not only for pictures but for
all representations. In general, there is no regress provided that representations, of
whatever form, are assigned their rightful places: in the world, but not in brains or on
retinas. There is, for every perceiver, but one interpreter of a representation: the
perceiver himself; and the interpretation he makes "... is the last interpretation".
There are no inner interpreters of inner representations.
Ryle (1949) and Wittgenstein (1953, 1958) were, it seems, right after all. The
basic flaw in Gregory's account, and in others like it, is that they require a duplicate,
a shadow of the world in the mind of the perceiver. This duplicate is called variously
an inner representation, perceptual hypothesis, cognitive map, isomorphism,
homomorphism, schema or inner picture. It must be given up, or the regress can
never be made to disappear. But to do so is to give up one of the core assumptions
of representative realism, a point of view, as I mentioned earlier, that encompasses
nearly the whole of modern psychology. Now one might think that would be asking
^And, in any case, the rules themselves will require interpretation. That is why computers do not
literally follow rules, though they conform to rules according to an interpretation by a human being.
The picture theory of perception 275

altogether too much for the sake of one small logical puzzlement. Perhaps it is.
But I would ask the reader to think of the regress as a loose thread attached to a
theoretical suit of clothes. Keep pulling the thread, and you might find that the suit
will eventually disappear.

Postscript
Until recently, it would have been fair, I believe, to judge Gregory's theoretical and
empirical work as a single elegant argument in support of one of a class of theories of
knowledge called representative realism. That is why I so labelled his position at the
outset of this paper. Gregory has since, however, published a quite remarkable book
entitled Mind in Science (1981), a work which in retrospect I think requires a
reconsideration of that attribution. It also, in my opinion, requires a reconsideration,
on his own part, of claims he has made about the nature of perception generally, and
the perceptual theory of J J Gibson particularly, about which he believes his own
theory to be the antithesis. It is these matters I wish to briefly consider here.
Mind in Science takes us through a virtually exhaustive history of the concept of
mind, from the earliest myths, to classical Greek thought, to early modern science, to
the present. Though several themes can be traced through this fascinating and erudite
work, there is a central theme, one which has appeared in less developed form in
Gregory's earlier work (eg Gregory 1980). The theme is this: There is a parallel, or
put more strongly, a "deep identity", between the conscious verification of hypotheses
in the conduct of science, and the unconscious verification of hypotheses in perception.
There is, in other words, a single mode of acquiring knowledge, and that is the
testing of hypotheses. In a certain way, therefore, my judgment of Gregory's theory
of perception, as I have already expressed it, would seem to be justified. The
perceptual hypothesis functions in a manner similar to the representation, and
representations are indirect means for knowing the external world. Toward the end
of the book, however, it becomes apparent that Gregory can no longer be so easily
called a representative realist. What he asserts there about the theory of knowledge
is, instead, a form of relativism. "But having rejected Direct Realism", he says on
page 537, "it seems that we cannot avoid a radical and extreme Relativism for
knowledge". And shortly after that, on page 556, he says, "We have come to accept
in this book an extreme form of relativity of belief". What precisely Gregory means
by 'relativism', and what has led him to it, cannot be made entirely clear until we
take a step backward, and look at some definitions.
Representative realism, as the name tells us on its face, is a form of realism. By
realism, I mean a theory of knowledge that implies a certain sort of dualzsm, namely
the complete independence of an external world from a knowing subject. However
one may characterize the real, whether as a Platonic world of forms, or as a material
world, or in some other way, the real is entirely distinct from the subject, and is
asserted to exist whether or not the subject exists. The real is thus the unchangeable
foundation of knowledge, the standard against which all perception and belief are to
be measured. Representative realism, known also as indirect or causal realism, assumes
that the knowing subject acquires knowledge about the real indirectly by representing
the real, more or less adequately, but generally with verisimilitude, in the form of
inner structures, pictures, or schemata.
There are many reasons why inner representations themselves are believed to exist.
The familiar phenomena found in the perception laboratory demonstrate that the
perceiver is occasionally in error (illusion), perceives the same object in more than
one way (ambiguity), and makes adjustments to proximal stimuli, eg retinal images,
to perceive distal stimuli accurately (constancy). These results, it is thought, could
not be explained without positing inner simulacra of the world, interposed between
276 S Katz

observer and the world, guiding the former to know the latter. Inner representations
also reveal the discrepancy between thought, imagination, or dreams, on the one hand,
and the real itself, on the other. For surely imagination is only a play on the world,
a transformation of it, and representations, unlike the world, are easily transformed—
they are protean derivations of the real.
Philosophers and psychologists have discerned several other forms, of realism of
which, for present purposes, I need name only one: direct (sometimes naively called
naive) realism. According to this theory, every act of perception is direct, and not
mediated by inner representations. Because perception is asserted to be direct or
unmediated, it follows that it is always veridical, despite the apparent conflict with
the evidence of illusion, ambiguity, and constancy, and the apparently obvious
difference between imagination and reality. As we see in the quote of Gregory just
above, he explicitly rejects direct realism, as do most perceptual theorists, for the
reasons just mentioned.
From a modern perspective, the advantage of realism in any form is that by
positing an external standard there is something towards which organisms, or species,
or forms of life generally, can progress, and against which success (ultimately biological
success) can be measured. Life must conform to this standard, the real, or it cannot
survive. Sense can thus be made of learning and development (ontogenetic change),
and ultimately of evolution (phylogenetic change). If science and perception are
fundamentally the same, as Gregory has asserted, then from a realist perspective it is
because both seek knowledge of the real. There are, however, serious conceptual
difficulties with realism. Representative realism, as a case in point, runs straight into
the following paradox (first made explicit, to this writer's knowledge, in Plato's
Meno): If the subject is to adjust or modify its inner representations in conformity
to the external world, then it must, in one way or another, make a match between
representation and world, the latter functioning as the standard. But if a match is
to be made, the subject must have both within its field of observation; one cannot
make a comparison if one does not have in view what is to be compared. But
apprehension of the standard (to make the comparison) vitiates a basic assumption of
representative realism, namely that knowledge can only be acquired, indirectly,
through representations. For in order to modify representations, which are the
mediate or indirect sources of knowledge, the subject must have direct knowledge of
the standard, and this is plainly paradoxical (see, eg, Hirst 1959; Katz and Frost
1979; Wilcox and Katz in press).
Direct realism, on the other hand, because it postulates no intermediary, is immune
to the paradox. But because it assumes a standard, it still must find a place for error,
as, for example, in the case of illusion. If every act of perception is direct, then it is
a puzzle to explain how something seen as it actually is can be seen in more than one
way. Conflicting perceptions cannot both be verisimilar, and yet that is what direct
realism demands, for it assumes direct apprehension of the real. There is a way out
of this difficulty, as there is with the paradox of representative realism, but the
solution is the strongest medicine possible: the rejection of realism altogether, and
the adoption of relativism.
Unlike realism, relativism is not concerned with the correspondence between what
is perceived and a standard, for there is no immutable standard. Truth is relative, not
absolute. What, the realist may rhetorically ask, is it relative to? That, to speak
relativistically, depends. The answer I give here, and the one to which I believe
Gregory subscribes, is that truth is determined by a duality (Turvey et al 1978), as
opposed to a dualism of subject and world. A dualism, we have seen, separates
subject and world, and makes them independent of one another. That is why we
could speak, in connection with realism, of the immutable standard, for what the
The picture theory of perception 277

subject is, what the subject does, what the subject believes, carries no implication for
what the world is; the world is independent of any possible knower. A duality of
subject and world, however, is a mutually defining relation; what the subject is or
does or believes bears significantly on what the world is, and vice versa. Subject and
world are together a system. Relativism, in other words, takes the observer into
account as a part of the definition of what is in the world, and conversely.
The reasons for relativism in Mind in Science which appear to me to have most
impressed Gregory are found in chapter 19, "The nurture of physics". There he gives
numerous examples, many of them familiar to us, of physical theories that seem to
require just such a duality of observer and world. For example, Gregory invokes
Einstein's theory of relativity to demonstrate a rejection of an absolute frame of
reference for describing motion. And in another example he points to the uncertainty
principle of Heisenberg to show that, in the very act of measuring, the observer
disturbs what is measured, so that a line of causality enabling us to predict future
events (at least in the subatomic world) is broken with every mensural intrusion.
These, and still other examples, correctly point to the increased awareness of the role
of the observer in understanding the physical world. One may say indeed that that
realization is an important mark of twentieth-century physics. Nonetheless,
unresolved disputes surrounding the philosophical import of the theory of relativity,
or of quantum theory, make it clear that such theories can still be seen from a
realist point of view. Einstein, after all, maintained a thoroughgoing realism (we
might even consider him an old-fashioned deist). And, as Newman (1956) points out,
it is one thing to call the uncertainty principle a principle of physics; it is quite
another to call it a principle of philosophy. Relativism writ large, in any case,
subscribes to no particular physical theory, even to one which is taken as evidence
for relativism. Its point, rather, is that every theory, every concept, and every
percept is valid with respect to a set of circumstances, whether those circumstances
are physical, historical, or biological.
Relativism, of course, is not new, and it is a way of thinking that has many faces.
It can be found in the pragmatism of Peirce or William James, in the functionalism of
Dewey, and in various theories in the philosophy of science. It is contained, at least
implicitly, in phenomenology, and in the later writings of Wittgenstein. It is
becoming today more and more of a force in macrobiology and in the environmental
sciences, where it is called 'ecology', and in the behavioral sciences, where it is called
'ecological psychology'. As I have mentioned, Gregory has criticized this last school,
specifically the theory of its founder, J J Gibson. He, and many other cognitive
theorists, have ironically construed Gibson's theory, not as an example of relativism,
but of its opposite, direct realism. In my opinion, this is in error, and I should like,
therefore, to conclude with a reply to the criticism.
In Mind and Science, and in other places, Gregory characterizes Gibson's theory of
perception as a type of direct realism, that is to say, as a theory of perception in
which the external world is held independent of the perceiver, and in which the mode
of apprehension is direct, or unmediated by inner representations. Let us consider
these in turn.
(i) Gibson and realism. Nearly everyone, detractors and supporters alike, has treated
Gibson as a realist. In the past, I have myself thought of him in this way. But how
justified is this? An analysis of the words 'real', 'realism', 'external world', and the
like in Gibson's books will not be of much help. Gibson does not use them
frequently, and when he does use them, he employs them in a variety of contexts, to
deal with a variety of issues. The crucial question, however, regardless of terminology,
is this: Does Gibson commit himself to a theory in which the external world is
conceived as an immutable standard that is independent of the perceiver? I do not
278 S Katz

pretend here to offer an exegesis of Gibson's entire body of work. I can, however,
say this much. In connection with his earlier work, I have found it difficult to
answer this question one way or the other. But in his most recent (and sadly, last)
book, the answer is, I think, in the negative. Here are two especially revealing
examples of what he does say there (Gibson 1979):
"The fact is worth remembering because it is often neglected that the words
animal and environment make an inseparable pair. Each term implies the other.
No animal could exist without an environment surrounding it. Equally, although
not so obvious, an environment implies an animal (or at least an organism) to be
surrounded", (page 8)
And later, in connection with the discussion of what Gibson calls affordances,
he says:
"The affordances of the environment are what it offers the animal, what it
provides or furnishes, either for good or ill. The verb to afford is found in the
dictionary, but the noun affor dance is not. I have made it up. I mean by it
something that refers to both the environment and the animal in a way that no
existing term does. It implies the complementarity of the animal and the
environment", (page 127)
These are not, I would suggest, the words of a realist. They are, of course, but
two brief quotes, extracted for my purposes, from a lengthy thesis. But I am
convinced that they represent its quintessence, and I believe a full reading of the
thesis will support my conviction. Gibson, to put it plainly, is proposing something
very different from any form of realism. It makes little sense in his ecological
psychology to describe properties either of organism or world that are independent of
one another. Realism is simply the wrong concept, or more accurately the antithetical
concept, to apply to Gibson.
(ii) Gibson and direct perception. Having peremptorily, but I hope not unfairly,
dismissed the thesis that Gibson is a realist, I want to assert just as peremptorily that
his theory is a theory of direct perception. Since opinion here is, once again,
unanimous, I would not expect anyone to deny this assertion, though many, of
course, would disagree with the thesis asserted. What I should like to argue here,
however, is that a theory of direct perception is perfectly consistent with a relativist
theory of knowledge, and that a theory of indirect perception is not. The theory of
indirect perception, the reader will recall, implies that knowledge of the external
world is mediated by inner representations, perceptual hypotheses (to use Gregory's
words), or other cognate mechanisms, whose job it is to reconstruct the external
world with limited means—impoverished sensory data. Perceivers, to use the
metaphor of the mystery thriller, are not witnesses to external events, but detectives,
who arrive on the scene after the fact, gather some clues, draw inferences, and then
point the finger. The theory implies an external world, an immutable standard, but
never permits the perceiver direct access to it. I have already recounted the reasons
why this sort of theory is accepted by cognitive theorists, and Gregory himself
develops those reasons in great depth, and with much elegance, in all of his work.
The essential point I wish to make here is that mediation in perception only makes
sense in the context of a commitment to a standard, that is, a commitment to
realism. The term 'indirect' carries no import, serves no purpose, unless there is a
framework against which perceptual hypotheses can be judged. Sherlock Holmes may
surmise correctly or incorrectly, but it is against a background of actual events from
which he is forever debarred (he is the detective, not the witness), yet which he
firmly believes to have happened, and to have happened in a way that admits of but
The picture theory of perception 279

one interpretation, of but one truth. The only problem with our metaphor, of
course, is that in perception, according to the indirect theory, there can be no
witness, for by definition a witness sees—does not infer—events, and the indirect
theory will not allow this.
For relativism, by contrast, perception cannot be indirect, since relativism
recognizes no implicit standard against which acts of perception must be compared.
Hence it recognizes no mediate entity between the knower and the known or at
least no basis for such an entity. Organism and environment, perceiver and world,
subject and object, knower and known, are, as we have seen, mutually defining,
complementary, relational. There is no correct or incorrect, right or wrong, except in
changed circumstances, or in a changed context, or from a different point of view.
But this is simply another way of expressing the idea that perception is direct.
Relativism defines a two-part relation between subject and object, not a three-part
relation that includes subject, mediating entity, and object.
In sum, I believe that Gibson's theory of direct perception is consistent with the
sort of relativism propounded by Gregory in Mind in Science and, moreover, is
required by it. Gregory is certainly correct to show that perception cannot be direct
within the framework of realism, but I believe he is wrong to assert the same within
the framework of relativism. He is also inconsistent, I believe, to assert the existence
of mediate entitites, representations, in a relativistic context.

References
Dennett D, 1978 Brainstorms (Montgomery, VT: Bradford Books)
Fodor J, 1975 The Language of Thought (New York: Thomas Y Crowell)
Gibson J J, 1966 The Senses Considered as Perceptual Systems (Boston, MA: Houghton Mifflin)
Gibson J J, 1979 The Ecological Approach to Visual Perception (Boston, MA: Houghton Mifflin)
Gregory R L, 1978 Eye and Brain third edition (New York: McGraw-Hill)
Gregory R L, 1980 "Perceptions as hypotheses" Philosophical Transactions Royal Society of
London, Series B 290 181-197
Gregory R L, 1981 Mind in Science (Cambridge: Cambridge University Press)
Heil J, 1981 "Does cognitive psychology rest on a mistake?" Mind XC 321-342
Hirst R L, 1959 The Problems of Perception (London: George Allen and Unwin)
Katz S, Frost G, 1979 "The origins of knowledge in two theories of brain: The cognitive paradox
revealed" Behaviorism 7 35-44
Malcolm N, 1977 Memory and Mind (Ithaca, NY: Cornell University Press)
Newman J, 1956 The World of Mathematics volume 2 (New York: Simon and Schuster)
Passmore J, 1969 Philosophical Reasoning (New York: Basic Books)
Pylyshyn Z, 1973 "What the mind's eye tells the mind's brain: A critique of mental imagery"
Psychological Bulletin 80 1-24
Ryle G, 1949 The Concept of Mind (New York: Barnes and Noble)
Searle J, 1980 "Minds, brains, and programs" The Behavioral and Brain Sciences 3 417-457
Turvey M T, Shaw R E, Mace W, 1978 "Issues in the theory of action: Degrees of freedom,
coordinative structures, and coalitions" in Attention and Performance volume VII Ed. J Requin
(Hillsdale, NJ: Lawrence Erlbaum Associates)
Wilcox S, Katz S (in press) "Can indirect realism be demonstrated in the psychological laboratory?"
Philosophy of the Social Sciences
Wittgenstein L, 1922 Tractatus Logico-Philosophicus (London: Routledge & Kegan Paul)
Wittgenstein L, 1953 Philosophical Investigations (Oxford: Basil Blackwell)
Wittgenstein L, 1958 The Blue and Brown Books (New York: Harper and Row)