Beruflich Dokumente
Kultur Dokumente
Background
Synesthesia (sin-əs-THEE-sha) is a neurological condition in which one
sensory function can simultaneously and inadvertently trigger other sensory
functions. A synesthete, or a person who has synesthetic experiences on a regular
basis, can experience taste from sounds, color from words and letters, and physical
feeling from visual stimuli, among other possible combinations. This report on my
research will discuss some background on synesthesia before focusing on the
artistic implications of synesthesia, and will end on my own composition, Chroma
Improv, that was inspired by synesthetic works.
Despite historic skepticism, synesthesia is a real, diagnosable condition in
one in about two thousand adults.1 It is much more common in adult females than
adult males, by about a six to one ratio.2 Many neurologists who have studied
synesthesia believe that all, or many, children are born synesthetic and gradually
lose it as they get older.3 Formal research studies on synesthesia began with
Richard Cytowic in the 1980s, and modern research using brain imaging
technology gives little doubt that the condition is very real for synesthetes.4
1 Oliver, Sacks. Musicophilia: Tales of Music and the Brain. Knopf, 2007.
2Oliver, Sacks. Musicophilia: Tales of Music and the Brain. Knopf, 2007.
B. B. “When Brains Wring Colors from Words.” Science News, 2002. Vol. 161, No. 12. pp. 189.
3Oliver, Sacks. Musicophilia: Tales of Music and the Brain. Knopf, 2007.
B. B. “When Brains Wring Colors from Words.” Science News, 2002. Vol. 161, No. 12. pp. 189.
4Van Campen, Cretien. “Artistic and Psychological Experiments with Synesthesia.” Leonardo, 1999. Vol. 32, No. 1.
pp 9-14.
Oliver, Sacks. Musicophilia: Tales of Music and the Brain. Knopf, 2007.
Heis, Synesthesia, Art and Media 2
2) the synesthete realizes that the experiences are not happening in reality,
3) the experiences must be long-term, consistent, and simple in correlation,
4) the synesthete must be able to remember and recall the experiences, and
5) the experiences must be emotional.5
many times as the experience occurs, and the experience does not need to be either
memorable or emotional.
Perhaps one in two thousand sounds like a very rare condition not worthy of
research, but in reality, even if a person is not a diagnosable synesthete, it does not
And, a 2006 study reported in Discover magazine found that given a choice
between animations made by synesthetes and non-synesthetes, most people found
the animations created by synesthetes were better synchronized. This means that
even regular people can link different senses unknowingly, a view also held by
Cytowic.7
5Cytowic, Richard E. The Man Who Tasted Shapes. New York: G. P. Putnam's Sons, 1993.
Cytowic, Richard E. Synesthesia: A Union of the Senses. New York: Springer-Verlag, 1989.
6Oliver, Sacks. Musicophilia: Tales of Music and the Brain. Knopf, 2007.
B. B. “When Brains Wring Colors from Words.” Science News, 2002. Vol. 161, No. 12. pp. 189.
synesthetes. Researchers are looking into several avenues for inducing or learning
synesthesia. Examples of this are through odor-taste synesthesia, which seems to
be the most natural form of it,8 drugs like cannabis and opiates, and in particular
LSD,9 certain forms of mediation,10 and more extreme methods including sensory
deprivation, where having one sense shut off incurs synesthetic experiences,
epilepsy, and electrical stimulation of the brain.11 All of these forms, to varying
There are three traditional, and four more modern, theories about
understanding how synesthesia works. The three traditional theories are miswiring,
leakage, and wrong feedback; the more modern views are via the limbic system,
neonatal pruning, which means everyone has it and loses it as we get older, brain
plasticity, which uses the argument that the fact the brain can replicate lost
functions may explain synesthesia, and disinhibition.13 Richard Cytowic, one of the
8Calvert, Gemma, Charles Spence, and Barry E. Stein, eds. The Handbook of Multisensory Processes. Cambridge:
MIT Press, 2004. Chapter 5.
9 Van Campen, Cretien. The Hidden Sense: Synesthesia in Art and Science. Cambridge: MIT Press, 2008. Chapter 7
10 Van Campen, Cretien. The Hidden Sense: Synesthesia in Art and Science. Cambridge: MIT Press, 2008. Chapter 7
11 Cytowic, Richard E. The Man Who Tasted Shapes. New York: G. P. Putnam's Sons, 1993. Chapter 16
12
Calvert, Gemma, Charles Spence, and Barry E. Stein, eds. The Handbook of Multisensory Processes. Cambridge:
MIT Press, 2004. Chapter 2.
13Van Campen, Cretien. The Hidden Sense: Synesthesia in Art and Science. Cambridge: MIT Press, 2008. Chapters
8 and 9
Heis, Synesthesia, Art and Media 4
population.15
The focus of my research is not on the neurological or psychological
14 Cytowic, Richard E. The Man Who Tasted Shapes. New York: G. P. Putnam's Sons, 1993. Chapter 20.
15 Cytowic, Richard E. The Man Who Tasted Shapes. New York: G. P. Putnam's Sons, 1993. Chapter 20.
16Oliver, Sacks. Musicophilia: Tales of Music and the Brain. Knopf, 2007. pp. 172.
Van Campen, Cretien. The Hidden Sense: Synesthesia in Art and Science. Cambridge: MIT Press, 2008. Chapters 8
and 9
Steen, Carol. “Visions Shared: A Firsthand Look into Synesthesia and Art.” Leonardo, 2001. Vol. 34, No. 3. pp
203-208.
Heis, Synesthesia, Art and Media 5
creative people.17 Many synesthetes are closeted, or were at some time, and find
Internal influences are perhaps even more significant however. Research has
found that, for most synesthetes, color is what is triggered by other senses, making
it the dominate synesthetic response. Taste and sound as a response are also not
uncommon.19 Research has also found that when synesthetes experience conflict
between their own internal correspondences and what they literally experiencing, it
causes distress in the synesthete.20 Unpleasant color combinations can trigger
experience is real.23
17 Cytowic, Richard E. Synesthesia: A Union of the Senses. New York: Springer-Verlag, 1989. Chapter 8
18Steen, Carol. “Visions Shared: A Firsthand Look into Synesthesia and Art.” Leonardo, 2001. Vol. 34, No. 3. pp
203-208.
19Van Campen, Cretien. The Hidden Sense: Synesthesia in Art and Science. Cambridge: MIT Press, 2008. Chapters
8 and 9
20Haack, Paul A. and Rudolf E. Radocy. “A Case Study of a Chromesthetic.” Journal of Research in Music
Education, 1981. Vol. 29, No. 2. pp. 85-90.
21Steen, Carol. “Visions Shared: A Firsthand Look into Synesthesia and Art.” Leonardo, 2001. Vol. 34, No. 3. pp
203-208.
22Hubbard, Timothy L. “Synesthesia-like Mappings of Lightness, Pitch, and Melodic Interval.” The American
Journal of Psychology, 1996. Vol. 109, No. 2. pp. 219-238.
23Steen, Carol. “Visions Shared: A Firsthand Look into Synesthesia and Art.” Leonardo, 2001. Vol. 34, No. 3. pp
203-208.
Heis, Synesthesia, Art and Media 6
course these are all generalizations. Synesthetes tend to prefer soft to well defined
shapes,24 and prefer to keep their work simple as opposed to complex, using black
influence synesthetic perception as well, for example the phrase ‘red hot.’26
Again, all synesthetes are unique in their correspondences between senses.
However, there are some things that are common in association. Lighter colors are
associated with higher pitch, darker colors are associated with lower pitch.27 The
vowels ‘u’ and ‘o’ are darker, ‘a’ in the middle, and ‘e’ and ‘i’ are typically lighter
in color.28 Another way of associating pitch with color: colors tend to start at red on
the note ‘C’ and move through the color wheel until green at the note ‘G’, and are
shades of pink and purple thereafter.29 Matching qualities are bright, smooth, hard,
cold, and sharp; another set is dull, rough, soft, blunt, warm, and heavy.30
24Steen, Carol. “Visions Shared: A Firsthand Look into Synesthesia and Art.” Leonardo, 2001. Vol. 34, No. 3. pp
203-208.
25Hubbard, Timothy L. “Synesthesia-like Mappings of Lightness, Pitch, and Melodic Interval.” The American
Journal of Psychology, 1996. Vol. 109, No. 2. pp. 219-238.
26Hubbard, Timothy L. “Synesthesia-like Mappings of Lightness, Pitch, and Melodic Interval.” The American
Journal of Psychology, 1996. Vol. 109, No. 2. pp. 219-238.
27Hubbard, Timothy L. “Synesthesia-like Mappings of Lightness, Pitch, and Melodic Interval.” The American
Journal of Psychology, 1996. Vol. 109, No. 2. pp. 219-238.
28Van Campen, Cretien. The Hidden Sense: Synesthesia in Art and Science. Cambridge: MIT Press, 2008. Chapters
8 and 9
29
Brougher, Kerry, Jeremy Strick, Ari Wiseman, and Judith Zilczer, eds. Visual Music. Los Angeles: Thames &
Hudson, 2005. pp 213
30Van Campen, Cretien. The Hidden Sense: Synesthesia in Art and Science. Cambridge: MIT Press, 2008. Chapters
8 and 9
Heis, Synesthesia, Art and Media 7
use of ratios in media and art. From Cytowic’s earlier book: design principles, like
the divine proportion and thirds, as well as musical temperaments, were not forced
combinations are also seen in renaissance artwork. Ratios and numbers are
associated with particular shapes, like multiples of three for triangles and the
pattern 2, 4, 16... for squares. These ratios across perceptual boundaries have
actually been found to be similar in correspondence for synesthetes.31
With that, let’s move on to synesthetic artists and the artwork that has been
make in the name of synesthesia.
Artistic Synesthetes and Synesthetic Art
Prometheus: Poem of Fire (1910), a symphonic poem by Alexander
Scriabin, is one of the most famous synesthetic works. Prometheus is about twenty
minutes in length, and is the story of an early man stealing fire from the gods. It
makes use of a color organ which outputs both music and colored light in specified
shapes.32 The instrument did not exist at the time the work was composed.33
Scriabin articulated in the score how the colors and shapes corresponded with the
music. The colors and shapes correspond with musical keys and not by note,
31 Cytowic, Richard E. Synesthesia: A Union of the Senses. New York: Springer-Verlag, 1989. Chapter 8
32 Van Campen, Cretien. “Artistic and Psychological Experiments with Synesthesia.” Leonardo, 1999. Vol. 32, No.
1. pp 9-14.
Berman, Greta. “Synesthesia and the Arts.” Leonardo, 1999. Vol. 32, No. 1. pp. 15-22.
33 Cytowic, Richard E. Synesthesia: A Union of the Senses. New York: Springer-Verlag, 1989. Chapter 8
Heis, Synesthesia, Art and Media 8
moving around the circle of fifths,34 leading some to call Scriabin not a true
so forth. Perhaps the most significant output of the symphonic poem is the mystic
chord, sometimes called the Prometheus chord, that Scriabin used as a foundational
mistakes. Schoenberg uses colors of lights, costumes, and set pieces to synchronize
with the onstage personalities and the plot, using conventions as opposed to
34 A concept in tonal music theory, using an intervallic relationship to map musical keys.
35Van Campen, Cretien. The Hidden Sense: Synesthesia in Art and Science. Cambridge: MIT Press, 2008.
Cook, Nicholas. Analysing Musical Multimedia. Oxford: Clarendon Press, 1998
Galeyev, B. M. and I. L. Vanechkina. “Was Scriabin a Synesthete?” Leonardo, 2001. Vol. 34, No. 4. pp 357-361.
36Brougher, Kerry, Jeremy Strick, Ari Wiseman, and Judith Zilczer, eds. Visual Music. Los Angeles: Thames &
Hudson, 2005.
Heis, Synesthesia, Art and Media 9
Rider), which had at least one truly synesthetic member. Wassily Kandinsky saw
deep rooted connections between all the senses and was inspired a variety of
sensory information, which he translated into paint. His most famous synesthetic
work is call Der gelbe Klang or The Yellow Sound (1912), a theater work involving
(composed 1929 to 1974) that had synesthetic influences. The works are not mixed
media, they are only music. Messiaen’s sound-color synesthesia is not as well
synesthetic experiences from it is complex.40 His music has been found to link
chord spacing and absolute pitch with color.41 The very form of the three works
38 Van Campen, Cretien. The Hidden Sense: Synesthesia in Art and Science. Cambridge: MIT Press, 2008.
39
Bernard, Jonathan W. “Messiaen’s Synaesthesia: The Correspondence between Color and Sound Structure in His
Music.” Music Perception, 1986. Vol. 4, No. 1. pp. 41-68.
40 Cytowic, Richard E. Synesthesia: A Union of the Senses. New York: Springer-Verlag, 1989. Chapter 8
listed above is based on color.42 Messiaen did not use lighting or other techniques
to demonstrate his perception of the generated colors, but they are still useful to
look at as they influence his music. While Messiaen’s sound-color correspondences
have been found to not be one hundred percent consistent,43 they are still a
valuable tool for analyzing his music.
the context of their key signature.44 His most famous synesthetic composition is
Ecstatic Orange, and his other synesthesia inspired works include Green, Purple,
Ash, and Bright Blue Music. All of these works have dance choreographed to them
as well.45 Torke interprets these works as dreams in particular colors, exploring the
Rez. In Rez, as the player interacts with the very abstract and colorful game
environment by shooting enemies and objects, different sounds are generated along
with various colors and shapes. The designers and artists behind Rez cite
Kandinsky’s paintings as inspiration for creating the game.47
43
Bernard, Jonathan W. “Messiaen’s Synaesthesia: The Correspondence between Color and Sound Structure in His
Music.” Music Perception, 1986. Vol. 4, No. 1. pp. 41-68.
44 Oliver, Sacks. Musicophilia: Tales of Music and the Brain. Knopf, 2007.
45 Berman, Greta. “Synesthesia and the Arts.” Leonardo, 1999. Vol. 32, No. 1. pp. 15-22.
46Berman, Greta. “Synesthesia and the Arts.” Leonardo, 1999. Vol. 32, No. 1. pp. 15-22.
Oliver, Sacks. Musicophilia: Tales of Music and the Brain. Knopf, 2007.
47Hawkins, Matthew. “Go To Synesthesia... Jake Kazdal’s Journey Though The Heart of Rez.” Gamasutra, May 6,
2005.
Heis, Synesthesia, Art and Media 11
There are many other examples of synesthesia in art and media, including
also an entire movement in video art called Visual Music, which I cannot even
begin to list the number of works created as part of it.
Creating Synesthetic Experiences
All of this leads to the question: how would one create a synesthetic piece of
art or media? The most obvious is to use the diagnosis of synesthesia as a guide
and from what I have written about influences and similarities:
• keeping it simple in correlation,
• keeping it consistent in correlation,
• emphasizing colors and words,
• combining multiple senses,
• limiting source material,
• blurring extremities,
• and using ratios to create the correlations between senses
would all make sense. But perhaps there is more we can do, more we can use from
synesthesia in art and in media.
The academic journal Leonardo had a series of articles about synesthesia
and art in the late eighties and early nineties. One article described the
requirements of the ultimate synesthetic medium: it does a wide variety of
mediums, it is spatial oriented, it has an organizational system to keep everything
Heis, Synesthesia, Art and Media 12
connected, and it takes full advantage of ability to “capture, generate, process and
or is, but what it does to the other mediums involved. This implies making
connections between mediums, and to hide the synchronization between mediums,
using things like movement, color, shape, and emotion. One medium can
significantly alter the time perception of others; particularly in the case of sound,
48Mallary, Robert. “Spatial-Synesthetic Art through 3-D Projection: The Requirements of a Computer-Based
Supermedium.” Leonardo, 1990. Vol. 23, No. 1. pp. 3-16.
improvisation functions similar to the way a true synesthete would perceive color
experience is forced onto the audience rather than being internal. The video part is
an attempt to create what a synesthete might perceive visually when hearing the
The setup of the work involves an electric bass player, standing in front of a
projected screen, and a camera positioned to view the the front of the bass player.
The score is set up in a six by six grid of twelve preselected pitches, with register
preselected, and the bass player improvises by moving across the grid in various
data for both sound and video. For the sound part, the computer transposes the
incoming signal three to six discreet times. It then pulses out the transposed signals
to outputting the dry bass signal for reference. Perhaps a future version of this
work will also use sound filtering to change the timbre of the sound output on the
hundred fifty-five have been preselected for each one of the twelve possible
Heis, Synesthesia, Art and Media 14
pitches, and on the basis of which pitch is currently being played the computer
filters out the color of the video signal from the camera on the bass player and
outputs the video signal onto the projector.
While this seemingly simple concept that synchronizes sonority, pulse rates,
and video color may seem too basic to be an interesting composition, it is true to
the concept of the work. The end result is more than the sum of its parts, which
only aids to prove that synesthesia is something we all experience at varying
degrees.
The future, conclusions
Synesthesia will grow as an area of interest as our world accelerates towards
combining senses for entertainment, for education, and for information. Things like
motion sensors, virtual reality, and feedback systems are all on the rise in many
different mediums, like films, games, and interactive works. It is possible that
because of this, the number of cases of diagnosable synesthesia will increase. The
more we understand this unique combination of the senses, the better we can craft
mixed media messages and artwork to generate the maximum response in the
audience.
Heis, Synesthesia, Art and Media 15
Sources
B. B. “When Brains Wring Colors from Words.” Science News, 2002. Vol. 161,
No. 12. pp. 189.
Berman, Greta. “Synesthesia and the Arts.” Leonardo, 1999. Vol. 32, No. 1. pp.
15-22.
Bernard, Jonathan W. “Messiaen’s Synaesthesia: The Correspondence between
Color and Sound Structure in His Music.” Music Perception, 1986. Vol. 4,
No. 1. pp. 41-68.
Brougher, Kerry, Jeremy Strick, Ari Wiseman, and Judith Zilczer, eds. Visual
Music. Los Angeles: Thames & Hudson, 2005.
Calvert, Gemma, Charles Spence, and Barry E. Stein, eds. The Handbook of
Multisensory Processes. Cambridge: MIT Press, 2004.
Cook, Nicholas. Analysing Musical Multimedia. Oxford: Clarendon Press, 1998.
Crawford, John C. “Die gluckliche Hand”: Shoenberg’s “Gesmatkunstwerk”. The
Musical Quarterly, Vol. 60 No. 4., Oct. 1974. pp. 583-601.
Cytowic, Richard E. Synesthesia: A Union of the Senses. New York: Springer-
Verlag, 1989.
Cytowic, Richard E. The Man Who Tasted Shapes. New York: G. P. Putnam's
Sons, 1993.
Galeyev, B. M. and I. L. Vanechkina. “Was Scriabin a Synesthete?” Leonardo,
2001. Vol. 34, No. 4. pp 357-361.
Garfield, Kathryn. “Are We All Synesthetes?” Discover, Dec. 15, 2006.
Haack, Paul A. and Rudolf E. Radocy. “A Case Study of a Chromesthetic.” Journal
of Research in Music Education, 1981. Vol. 29, No. 2. pp. 85-90.
Heis, Synesthesia, Art and Media 16
Interval.” The American Journal of Psychology, 1996. Vol. 109, No. 2. pp.
219-238.
Steen, Carol. “Visions Shared: A Firsthand Look into Synesthesia and Art.”
Leonardo, 2001. Vol. 34, No. 3. pp 203-208.
Van Campen, Cretien. The Hidden Sense: Synesthesia in Art and Science.
Cambridge: MIT Press, 2008.