Beruflich Dokumente
Kultur Dokumente
:
i
Figure 11. Rafael Lozano-Hemmer, Body Movies (2001). Photograph by Antimodular Research.
which can measure from two to twenty-five meters high, depending
on how far people are from the light sources placed on the floor of the
square. The essential element of the grammar of interaction is to give pres-
ence to those who are absent. In media ontological terms, one could say
that the images exist only if they exist to somebody; that is, the image is
physically created in the process of perception rather than production.
The interactor's shadow also manifests a split between self and other. The
interactor is symbolically deprived of her shadow, or rather of her self,
because by functioning as a means of projection, her shadow represents
another person. One can see an analogy with tattoos in that the body is
being used as a canvas. Here, however, the image is revealed not on the
skin but on the shadow, and is ephemeral. In addition, the person has no
choice as to what is displayed on her body. Body Movies thus carries out
an emblematic occupation: the body of those present is overwritten by
absent others.
Lozano-Hemmer's intent that interactors embody a portrait by align-
ing scale and position of their shadow was only partly transmitted into
practice. Because the actual embodiment "is dull and boring," as Graham
128
INTE RACTI VE IN STALLATIONS
Coulter-Smith (2006, 275) puts it, interactors showed more interest in
playing with their own shadows than in engaging the portraits. While the
disclosure of the portraits occurred in "deathly silence," the creative
infringement of the grammar of interaction-that is, a gigantic shadow
woman stomping a tiny shadow man to the ground-was "followed by
peals oflaughter" (275). Lozano-Hemmer's surprise about the audience's
demonstrates that the artist expects interactors to act in a spe-
Cific way, as opposed to just providing the interactive environment for
self-discovery.
14
The interactors' misuse of the apparatus signifies a resis-
tance to the artist's expectation, a rejection of the prescribed inscription
and threatening deprivation, or simply of the dullness and boredom of
the original grammar of interaction-and a creative, playful occupation
of the work. Coulter-Smith wonders "whether Lozano-Hemmer possesses
the proverbial gift of genius because even when he fails he produces some-
thing quite extraordinary" (275). Given the title of this work, one may also
wonder whether the artist in fact (unconsciously) wanted his audience to
act exactly in this way because whereas the display of the hidden por-
traits generates stills, the shadow games can indeed be seen as producing
little body movies. 1s
One conclusion of the discussion above is that an example of interac-
tive art can-and should-be perceived as both event and object. The event
is object insofar as it is a deliberately composed structure of different sig-
nifiers (the falling letters and the real poem behind them in Text Rain)
and possible actions (collecting the letters with the option to elevate but
not to rearrange them) building up to a specific meaning. This notion
seems plausible for works such as Still Standing, whose grammar ofinter-
action (and title!) clearly points to a particular interaction: standing still.
Similarly, in Screen, the action expected from the interactor is obvious.
Even if the user refuses to perform the expected action by not standing
still, by not pushing back the dropping words, it would only be in response
to the grammar of interaction and would reinforce the message behind it.
However, in cases where the interactors' actions are much less restricted
or predesigned, it seems to be impossible to read the interactive setting
as an object with a certain meaning. Two examples are Rokeby's Very Ner-
vous System and Utterback's Untitled 5, both of which give interactors
complete freedom in how to engage with the work. Both do not allow inter-
actors to completely understand the grammar of interaction such that
they could deliberately use their body as an instrument or tool.ts
However, if the grammar of interaction is obscure, it does not necessarily
INTERACTI VE IN STALLATION S 129
have to be without meaning. Behind the very fact of its indistinctness can
be a philosophical and pedagogical agenda, as Rokeby's Very Nervous
System proves. Rokeby deplores the general fetishization of control and
considers it dangerous "if we weed out of our lives those things that are
uncertain, unpredictable and ambiguous." This is why he likes to create
"systems of inexact control" (Rokeby 2003). The computer, Rokeby states,
sets up the illusion and fantasy of total control, which is not a useful par-
adigm for real-world encounters. Rokeby's system tangibly reflects the
interactors' input but does not allow them to gain complete control. It is
a learning experience intended not least to improve our relationships
with our neighbors, as Rokeby suggests: "The fact that my neighbour does
not always reply the same way when I say 'hello' does not mean that this
is a disempowering experience."
It is helpful to illustrate the different approaches to an interactive artwork
with a particular example. Picking a work that has been reviewed by an
art critic and that reveals how little attention even experts pay to the pos-
sible meanings of the symbolic structure of a work and its grammar of
interaction will help to show us what is at stake for the treatment of inter-
active art. In Bill Seaman's interactive video-sound installation Exchange
Fields (2000), the viewer can interact with thirteen furniture-like objects,
each designed and labeled in relationship to a particular part of the body,
and trigger certain dance sequences on one of the three video screens in
front of the objects (Figure l2)Y If, for example, the interactor puts her
arm into the appropriate box and does not move it, after a while she will
trigger the projection of a dance sequence, choreographed and danced
by Regina van Berkel, focusing on the arm. If the interactor puts her leg in
the appropriate box, a moving leg will appear on the screen. The installa-
tion is able to project four overlapping dance sequences, thus allowing
four viewers to interact with four furniture sculptures at the same time. In
addition to the video sequences, which are almost the only source of light
in the gallery, one hears meditative sound playing in the background,
blending with the chanting of Seaman's lyrical texts reflecting on human-
machine interrelations. The grammar of interaction is easily understood;
the hypnotic composition of visuals, sound, and material environment
provides the audience with an intense experience. However, there are at
least three different ways to engage with this work.
A reaction that might be termed atmospheric perception will be based
on the fascination by the hypnotic composition of visuals, sound, and
130 INTERACTIVE INSTALLATIONS
Figure 12. Exchange Fields (2000). Drawing for arm sculpture (top). Video still for arm
object (bottom). Copyright 2000 Bill Seaman and Regina van Berkel, drawing by Seaman,
rendering by Casino Container, detail from the installation Exchange Fields.
material environment in this Gesamtkunstwerk: the clever space design,
the striking dance scenes, Seaman's meditative chanting about the human-
machine relationship. Others will be less impressed. They have seen similar
settings and heard similar lyrical refashioning of philosophical language
and will be more interested in testing the interface. They will enjoy trick-
ing the system by putting their hand in the box assigned to the foot and
their chest onto the spot for the back. This form of rebellious interaction
is supported by Tilman Baumgartel, a pioneer among the German critics
of digital art, who considers Seaman's work to be a manipulation of the
audience because its grammar of interaction requires the interactors to
constrain their body parts in order to see a ballerina on the screen. Ex-
change Field, Baumgartel (2000) concludes, degrades its audience to "inter-
active guinea pigs," and he calls the observed attempt of interactors to
crash the system the "revenge of art lovers." Such revenge is unfair to the
implicit pact between the artist and the audience in interactive art: the
acceptance of the proffered interface, regardless of its practical limits and
flaws. Such revenge is also unwise because it entraps the so-called art lovers,
as well as the art critic, within an ignorance regarding the work's possible
meaning, rather than opening out to a creative engagement with the work.
Both consequences may reinforce each other insofar as the search for tech-
nical flaws reflects a focus on craft that may (although it does not have to)
reveal a lack of interest in any conceptual aspects of the work.
A third approach-beyond both naive enthusiasm and "been there-
done that" boredom-will consider the specific grammar of interaction
and wonder what the artist wishes to communicate. Analysis will reveal
that one must immobilize one's limbs to make the corresponding limb of
another person appear and move on the screen. One will also note that
this other limb disappears as soon as the interactor moves again. One pos-
sible way to understand this grammar of interaction is the notion that the
reality of the interactor's body is traded for the projection of a foreign
body on the screen that appears in dramatic lighting and camera moves,
and is imbued with eroticism. The corporeal (thick, fleshy, old, washed-
out) body is exchanged for the ideal body. In a double reverse of the Pyg-
malion myth, the viewer is converted into the sculpture and brings her
object of desire alive if she ceases to move. It is easy to imagine that the
reception of the work and the interaction with it would change if the
immobilization of the interactor's body triggered the appearance not of a
young, idealized body but an old, exhausted one. The interpretation will
also suppose that the screen in Exchange Fields represents more than the
132
INTERACTIVE INSTALLATIONS
screen of the installation-namely television, movie screens, and adver-
tising billboards, which confront people every day with beautiful bodies
and hence inform their personal body ideal. This information overwrites
our native conception of the body and thus, in a way, results in what Sea-
man's work requires its audience to do. What is more or less consciously
our day-to-day experience is literally reenacted within the artwork. Ex-
change Fields urges us to discover this ongoing substitution and to reflect
on our complicity with it.
The aesthetics of Exchange Fields could also be discussed with respect
to recent developments in neuroscience offering a scientific explanation
for the intuitive understanding of how others feel and the empathetic re-
sponse to an artwork and engagement in aesthetic experiences. The the-
ory of mirror neurons states that a type of brain cell is activated both by
performing an action and by watching the same action performed by oth-
ers. The discovery of mirroring mechanisms and embodied simulation no
doubt has tremendous implications, not only for the empathetic responses
to images and works of visual art (Freed berg and Gallese 2008), but also
for interactive installations and immersive environments, where the audi-
ence, motivated by mirror neurons, is able to react directly to what it sees.
In this context, one may consider Exchange Fields as a play with the mir-
ror drive deliberately, and in contrast to the common logic of interactive
installations, avoiding any visual signs of the possible emphatic response
through the required immobilization of the interactor.
In this context, we should also note that although the discovery of mir-
ror neurons challenges the primacy of cognition in responses to art, it
does not necessarily question the role of cognition in the discussion of
art. Aesthetic experiences based on noncognitive operations can still be
interpreted-and should, in order to move the reflection of an art work
beyond sheer description. If David Freed berg and Vittorio Gallese state,
"historical, cultural and other contextual factors do not preclude the im-
portance of considering the neural processes that arise in the empathetic
understanding of visual artworks" (2008, 198), one may add that the re-
versed perspective is also true: The audience's noncognitive response to
an artwork does not preclude considering the importance of historical,
cultural, and other contextual factors to the artist's aesthetic decisions and
the critic's discursive approach.
For the immediate argument, it is not important how convincing the sug-
gested understanding of Seaman's work may be, or how many other ways
INTERACTIVE INSTALLATIONS 13 3
to read it can be found. What is crucial is the implied recurrence of the
Cartesian focus on cognitive activity. This recurrence does not dismiss
the role of the body in interactive art, as is usually the case within the
Cartesian paradigm. However, it claims that physical interaction within
the responsive environment should not exclude the reflection of the expe-
rience. One may argue that in some cases, the discussion of a work does
not require a live experience of the work and that one can also talk about
the grammar of interaction by observing others physically engaging with
the work or even only seeing a documentary or reading a description of
it. Yet even if a work such as Still Standing can be understood without
experiencing it, one will not understand it with the body if one does not
actually stand still in front of the screen to read the text. Without really
experiencing those minutes of immobilization-in Still Standing as well
as in Exchange Fields-one cannot feel the meaning of the grammar of
interaction. The physical experience is imprinted in the body, with lasting
consequences. An hour of the continuous, direct feedback in Rokeby's
Very Nervous System, for example, strongly reinforces a sense of connec-
tion with the surrounding environment. The body remembers having been
an "instrument" and wants to play again. "Walking down the street after-
wards," Rokeby reports, "I feel connected to all things. The sound of a
passing car splashing through a puddle seems to be directly related to my
movements. I feel implicated in every action around me. On the other
hand, if I put on a CD, I quickly feel cheated that the music does not
change with my actions" (1997, 30).
To be sure, the experience of an interactive environment and one's own
(re)action within it does not necessarily lead to a reflection of the body's
experience within the interactive environment. In many cases, the audi-
ence only tests the grammar of interaction, assuming it has understood
what the work means once it has understood how it functions. To use
Ascott's terminology to contradict his own statement above (2003, 110), it
may be said that interactive art can be perceived as both event and fact.
It provides a space for experiences, but also a message to be understood.
The fact that an interactive artwork needs its audience's participation to
be completed does not mean that it merely receives its meaning from this
participation. On the contrary, by generating a specific grammar of inter-
action, the artist infuses her work with meaning and obtains a certain
control over the message that the artwork offers. This becomes clear with
respect to Rokeby, who declares, in line with declarations by Ascott, Bour-
riaud, and many others, "I'm an interactive artist. I construct experiences,"
134
INTERACTIVE INSTALLATIONS
but also announces a deeper meaning behind the experience of inexact
control he intends to construct: "If culture, in the context of interactive
media, becomes something we 'do,' it's the interface that defines how we
do it and how the 'doing' feels" (1997, 28). According to Rokeby, interfaces
using the term interface for what Fujihata calls the grammar
of the specific mode of interaction made possible by the artist.
There IS no reason not to make this content the subject of interpretation.
A condition for such an endeavor is, as in any hermeneutic task, to
undertake a close reading, paying attention to all details of the work and
discussing the results. In what follows, I want to illustrate what a close
of the grammar of interaction and the discussion of the symbolic
mean with respect to various examples from different genres
of mteract1ve art that have different degrees of interaction. We will then
resume the general discussion of the role of interpretation in interactive art.
Understanding the Grammar of Interaction
JC]-Junkman is an interactive work by Ken Feingold from 1995. Here a
puppet head, which Feingold had used previously in Jimmy Charlie Jimmy
(1992), appears on the screen surrounded by a whirling storm of ever-
changing images that appear and disappear. If one manages to click on
one of the images, the puppet speaks a piece of arbitrary language, such
as "Hasta lavista, baby,'' "Put it down,'' "May you go and dig this,'' "Mes-
sage," "Kambiariota," "Das war's." There is no beginning and no end in this
game of chasing the links, no other levels, no score. "It is 'interactivity'
reduced to a zero-degree, as thousands of narrative fragments displace
each other, fueled by a raw desire to get something," says Feingold. "The
emotion produced is that any choice will do."
18
Erkki Huhtamo identifies
in this situation of pseudochoices an obligation to rethink our relation-
ship to the medium. For him, the meaning of the work is the loss of con-
trol over the flow of data trash that we have already internalized: "The
junkmen we have become do not collect the garbage to sort it or recycle it
in the ecological sense of the word. Although we may think otherwise, we
merely reiterate the cycle of junk that connects our minds with the media
reality" (1996, 48).
Another work from the early days of digital media, not online but in a
museum, is Der Zerseher (literally, "dis-viewer," also known as Iconoclast)
by Joachim Sauter and Dirk Liisebrink (1992). In this installation, the view-
ers deconstruct a painting by looking at it. The painting-Boy with a
INTERACTIVE INSTALLATIONS 135
Child-Drawing in His Hand (ca. 1525) by Giovan Francesco Caroto-is a
framed rear projection onto canvas behind which, invisible to the viewer,
an eye tracker is installed (camera, PC, video-tracking software). The cam-
era sends the information about the viewer's eye movement to the com-
puter, which locates the center of the iris and the point where an infrared
light reflects from the viewer's eye. With the resulting data, it can precisely
identify the part of the painting the viewer is looking at. This part of the
painting is consequently distorted. The observer who encounters this work
in a museum environment, not expecting a painting to dissolve under her
gaze, realizes that the more she looks at the painting to discover its mean-
ing, the more she destroys the object of her analysis. Because this comes
as a surprise, it may contribute to confusion, but it will certainly also rein-
force a reflection concerning the difference between this kind of experi-
ence of a painting and the more traditional way. It is part of the irony of
the work that the painting can be restored only by disinterest in it. If more
than thirty seconds elapse without the picture being viewed, it reverts to
its original condition.
This installation fundamentally changes the perception of a visual
object. As the artists state, "In the past an old master might leave an im-
pression in the mind of the passive onlooker -now the onlooker can leave
an impression on the old