15/02/2018 Seeing and Imagining: Clues to the Workings Of the Mind's Eye - The New York Times

This copy is for your personal, noncommercial use only. You can order presentation-ready copies for distribution
to your colleagues, clients or customers, please click here or use the "Reprints" tool that appears next to any
article. Visit www.nytreprints.com for samples and additional information. Order a reprint of this article now. »

August 31, 1993

Seeing and Imagining: Clues to the

Workings Of the Mind's Eye
By SANDRA BLAKESLEE

CLOSE your eyes and consider these questions: What shape are a
German shepherd's ears? Which is darker green: a frozen pea or a
Christmas tree? If you rotate the letter "N" 90 degrees to the right, is a
new letter formed?

In seeking answers to such questions, scientists say, most people will

conjure up an image in their mind's eye, mentally "look" at it, add
details one at a time and describe what they see. They seem to have a
definite picture in their heads.

But where in the brain are these images formed? And how are they
generated? Without hands in the brain, how do people "move things
around" in their imaginations?

Using clues from brain-damaged patients and advanced brain imaging

techniques, neuroscientists have now found that the brain uses virtually
identical pathways for seeing objects and for imagining them -- only it
uses these pathways in reverse.

In the process of human vision, a stimulus in the outside world is passed

from the retina to the primary visual cortex and then to higher centers
until an object or event is recognized. In mental imaging, a stimulus
originates in higher centers and is passed down to the primary visual
cortex, where it is recognized.
http://www.nytimes.com/1993/08/31/science/seeing-and-imagining-clues-to-the-workings-of-the-mind-s-eye.html?pagewanted=1&pagewanted=all… 1/8
15/02/2018 Seeing and Imagining: Clues to the Workings Of the Mind's Eye - The New York Times

The implications are beguiling. Scientists say that for the first time they
are glimpsing the biological basis for abilities that make some people
better at math or art or flying fighter aircraft. They can now explain why
imagining oneself shooting baskets like Michael Jordan can indeed
improve one's athletic performance. And, in a finding that raises
troubling questions about the validity of eyewitness testimony, they can
show that an imagined object is, to the observer's brain at least, every bit
as real as one that is seen.

"People have always wondered if there are pictures in the brain," said
Dr. Martha Farah, a psychology professor at the University of
Pennsylvania. More recently, she said, the debate centered on a specific
query: as a form of thought, is mental imagery rooted in the abstract
symbols of language or in the biology of the visual system?

The biological arguments are winning converts every day, Dr. Farah
said. The new findings are based on the notion that mental capacities
like memory, perception, mental imagery, language and thought are
rooted in complex underlying structures in the brain. Thus an image
held in the mind's eye has physical rather than ethereal properties.

Mental imagery research has developed apace with research on the

human visual system, said Dr. Stephen Kosslyn, a psychologist at
Harvard University who is a pioneer in both fields. Each provides clues
to the other, he said, helping to work out the details of a highly complex
system.

Vision is not a single process but rather a linking together of subsystems

that process specific aspects of vision. To understand how this works,
Dr. Kosslyn said, consider looking at an apple on a picnic table 10 feet
away. Light reflects off the apple, hits the retina and is sent through
nerve fibers to an early visual way station that Dr. Kosslyn calls the
visual buffer. Here the apple image is literally mapped onto the surface
of brain tissue as it appears in space, with high resolution.

http://www.nytimes.com/1993/08/31/science/seeing-and-imagining-clues-to-the-workings-of-the-mind-s-eye.html?pagewanted=1&pagewanted=all… 2/8
15/02/2018 Seeing and Imagining: Clues to the Workings Of the Mind's Eye - The New York Times

"You can think of the visual buffer as a screen," Dr. Kosslyn said. "A
picture can be displayed on the screen from the camera, which is your
eyes, or from a videotape recorder, which is your memory."

In this case, he said, the image of the apple is held on the screen as the
visual buffer carries out a preliminary analysis of the scene. Edges,
contour, color, depth of field and a variety of other features are
examined separately, he said. But the brain does not yet know it is
seeing an apple.

Next, distinct features of the apple are sent to two higher subsystems for
further analysis, Dr. Kosslyn said. They are often referred to as the
"what" system and the "where" system. The brain needs to match the
primitive apple pattern with memories and knowledge about apples, he
said, and so it seeks knowledge from visual memories that are held like
videotapes in the brain. 'What' and 'Where' Systems

The "what" system, in the temporal lobe, contains cells that are tuned
for specific shapes and colors of objects, Dr. Kosslyn said. Some respond
to red, round objects in an infinite variety of positions, ignoring local
space. Thus the apple could be on a distant tree, on the picnic table or in
front of your nose, he said, and it would still stimulate cells tuned for red
round objects, which might be apples, beach balls or tomatoes.

The "where" system, in the parietal lobe, contains cells that are tuned to
fire when objects are in different locations. If the apple is far away, one
set of cells is activated, while another set fires if the apple is close up.
Thus the brain has a way of knowing where objects are in space so the
body can navigate accordingly.

When cells in the "what" and "where" systems are stimulated, they may
combine their signals in yet a higher subsystem where associative
memories are stored, Dr. Kosslyn said. This system is like a card file
where visual memories, as if held on videotapes, can be looked up and
activated, he said.

http://www.nytimes.com/1993/08/31/science/seeing-and-imagining-clues-to-the-workings-of-the-mind-s-eye.html?pagewanted=1&pagewanted=all… 3/8
15/02/2018 Seeing and Imagining: Clues to the Workings Of the Mind's Eye - The New York Times

If the signals from the "what" and "where" system find a good match in
associative memory, Dr. Kosslyn said, you know the object is an apple.
You also know what it tastes and smells like, that is has seeds, that it can
be made into your favorite pie and everything else stored in your brain
about apples.

But sometimes, he said, recognition does not occur at the level of

associative memory. Because it is far away, the red object on the picnic
table could be a tomato or an apple. You are not sure of its identity, he
said. So another level of analysis kicks in. Decision-Making Site

This highest level, in the frontal lobes, is where decisions are made, Dr.
Kosslyn said. To use the same analogy, it is like a catalogue for the
videotapes in the brain. You look up features about the image to help
you identify it, he said. A tomato has a pointed leaf, while an apple has a
slender stem. When the apple stem is found at this higher level, he said,
the brain decides that it has an apple in its visual field.

Signals are then fired back down through the system to the visual buffer
and the apple is recognized. Significantly, Dr. Kosslyn said, every visual
area that sends information upstream through nerve fibers also receives
information back from that area. Information flows richly in both
directions at all times.

Mental imagery is the result of this duality. Instead of a visual stimulus,

a mental stimulus activates the system, Dr. Kosslyn said. The stimulus
can be anything, including a memory, odor, face, reverie, song or
question.

For example, Dr. Kosslyn said, "I ask you to imagine a cat." Images are
based on previously encoded representations of shape, he said, "so you
look up the videotape in associative memory for cat."

When that subsystem is activated, he said, a general image of a cat is

mapped out on the screen, or visual buffer, in the primary visual cortex.
It is a stripped-down version of a cat and everyone's version is different.
http://www.nytimes.com/1993/08/31/science/seeing-and-imagining-clues-to-the-workings-of-the-mind-s-eye.html?pagewanted=1&pagewanted=all… 4/8
15/02/2018 Seeing and Imagining: Clues to the Workings Of the Mind's Eye - The New York Times

"Now I ask you another question," Dr. Kosslyn said. "Does the cat have
curved claws?" To find out, the mind's eye shifts attention and goes back
to higher subsystems where detailed features are stored.

"You activate the curved claws tape," he said, "and then zoom back down
to the front paws of the cat and you add them to the cat. Thus each
image is built up, a part at a time." Mimic of Real World

The more complex the image, the more time it takes to conjure it in the
visual buffer, Dr. Kosslyn said. On the basis of brain scans with the
technique known as positron emission tomography, he estimates that it
requires 75- to 100-thousandths of a second to add each new part.

The visual system maps imagined objects and scenes precisely,

mimicking the real world, Dr. Kosslyn said. "You scan it and study it as
if it were there."

This can be demonstrated when people are asked to imagine objects at

different sizes. "Imagine a tiny honeybee," Dr. Kosslyn said. "What color
is its head?" To do this, people have to take time to zoom in on the bee's
head before they can answer, he said.

Conversely, objects can be imagined so that they overflow the visual

field. "Imagine walking toward a car," Dr. Kosslyn said. "It looms larger
as you get closer to it. There comes a point where you cannot see the car
at once. It seems to overflow the screen in your mind's eye."

People with brain damage often demonstrate that the visual system is
doing double duty, Dr. Farah said. For example, stroke patients who lose
the ability to see colors also cannot imagine colors.

An epilepsy patient experienced a striking change in her ability to

imagine objects after her right occipital lobe was removed to reduce
seizures, Dr. Farah said. Before surgery, the woman estimated she
would stand, in her mind's eye, about 14 feet from a horse before it

http://www.nytimes.com/1993/08/31/science/seeing-and-imagining-clues-to-the-workings-of-the-mind-s-eye.html?pagewanted=1&pagewanted=all… 5/8
15/02/2018 Seeing and Imagining: Clues to the Workings Of the Mind's Eye - The New York Times

overflowed her visual field. After surgery, she estimated the overflow at
34 feet. Her field of mental imagery was reduced by half, Dr. Farah said.

Another patient suffered damage to his "what" system while his "where"
system was intact. "If you ask him to imagine what color is the inside of
a watermelon, he does not know," Dr. Farah said. "If you press him, he
might guess blue. But if you ask him, is New Jersey closer to Oklahoma
or North Carolina, he answers correctly instantly."

Imaging studies of healthy brains produce similar findings, Dr. Farah

said. When a person is asked to look at and then to imagine an object,
she said, the same brain areas are activated. When people add details to
images, they use the same circuits used in vision. Interestingly, people
who say they are vivid imagers show stronger activation of the relevant
areas in the brain, she said.

People use imagery in their everyday lives to call up information in

memory, to reason and to learn new skills, the scientists said. Einstein's
Use of Imagery

It can lead to creativity. Albert Einstein apparently got his first insight
into relativity when he imagined chasing after and matching the speed
of a beam of light.

It can improve athletic skills. "When you see a gifted athlete move in a
particular way, you note how he or she moves," Dr. Kosslyn said, "and
you can use that information to program your own muscles." Basically,
the brain uses the same representations in the "where" system to help
direct actual movements and imagined movements, he said. Thus,
refining these representations in imagery will transfer to actual
movements, provided the movements are physically practiced.

Humans exhibit vast individual differences in various components of

mental imaging, which may help explain certain talents and
predilections, Dr. Kosslyn said. Fighter pilots, for example, can imagine

http://www.nytimes.com/1993/08/31/science/seeing-and-imagining-clues-to-the-workings-of-the-mind-s-eye.html?pagewanted=1&pagewanted=all… 6/8
15/02/2018 Seeing and Imagining: Clues to the Workings Of the Mind's Eye - The New York Times

the rotation of complex objects in a flash, but most people need time to
imagine such tasks.

In a study in progress, Dr. Kosslyn and colleagues are examining the

brains of mathematicians and artists with a new imaging machine that
reveals individual differences in the way brains are biologically wired
up. They are looking to see if people who are good at geometry have
different circuitry from that of people who are good at algebra.

In a philosophical conundrum arising from the new research, it seems

that people can confuse what is real and what is imagined, raising
questions about witnesses' testimony and memory itself.

"In visual perception," Dr. Kosslyn said, "you prime yourself to see an
object when you only have part of the picture. If you expect to see an
apple, its various fragments can drive the system into producing the
image of an apple in your visual buffer." In other words, he said, you
prime yourself so much that you actually play the apple tape from your
memory banks.

Thus, people can be fooled by their mind's eye, Dr. Kosslyn said.
Imagine seeing a man standing before a frightened store clerk and you
assume a robbery is under way. It is dark and his hand is in the
shadows. Because you expect to see a gun, your thresholds are lowered
and you may actually run the tape for a gun, even though it is not there.
As far as your brain is concerned, it saw a gun, Dr. Kosslyn said. Yet it
may not have been real.

Luckily, he said, inputs from the eye tend to be much stronger than
inputs from imagination. But on a dark night, under certain
circumstances, it is easy to be fooled by one's own brain.

It is amazing that imagination and reality are not confused more often,
said Dr. Marcia Johnson, a Princeton psychologist who in her laboratory
can make people swear that they saw or heard things that never
happened. In general, she said, images are fuzzier and less coherent
http://www.nytimes.com/1993/08/31/science/seeing-and-imagining-clues-to-the-workings-of-the-mind-s-eye.html?pagewanted=1&pagewanted=all… 7/8
15/02/2018 Seeing and Imagining: Clues to the Workings Of the Mind's Eye - The New York Times

than real memories, and humans are able to differentiate them by how
plausible they seem.

Diagram: "Pathways for Imagery in the Brain" The processing of a visual

perception in the cortex of the brain begins in the occipital lobe (1) and
proceeds simultaneously to the parietal (2a) and temporal (2b) lobes.
These brain areas process the location and identity of the perceived
object. In mental imagery, these same areas specify thelocation and
appearance of the imagined object, which is "displayed" in the occipital
lobe. A new theory suggests that associative memory areas (3) and
decision-making areas (4) of the brain also contribute to perception and
that the process of imagining proceeds in the reverse order from that of
visual perception. (Source: Dr. Martha Farah/University of
Pennsylvania) (pg. C7) Drawing (pg. C1)

Copyright 2018 The New York Times Company Home Privacy Policy Search
Corrections XML Help Contact Us Back to Top

http://www.nytimes.com/1993/08/31/science/seeing-and-imagining-clues-to-the-workings-of-the-mind-s-eye.html?pagewanted=1&pagewanted=all… 8/8