Beruflich Dokumente
Kultur Dokumente
≥
Applications of Cognitive Linguistics
11
Editors
Gitte Kristiansen
Michel Achard
René Dirven
Francisco J. Ruiz de Mendoza Ibáñez
Mouton de Gruyter
Berlin · New York
Multimodal Metaphor
Edited by
Charles J. Forceville
Eduardo Urios-Aparisi
Mouton de Gruyter
Berlin · New York
Mouton de Gruyter (formerly Mouton, The Hague)
is a Division of Walter de Gruyter GmbH & Co. KG, Berlin
앪
앝 Printed on acid-free paper
which falls within
the guidelines of the ANSI
to ensure permanence and durability.
ISBN 978-3-11-020515-2
ISSN 1861-4078
쑔 Copyright 2009 by Walter de Gruyter GmbH & Co. KG, D-10785 Berlin
All rights reserved, including those of translation into foreign languages. No part of this book
may be reproduced or transmitted in any form or by any means, electronic or mechanical,
including photocopy, recording, or any information storage and retrieval system, without
permission in writing from the publisher.
Printed in Germany
Table of contents
Preface ..............................................................................................................xiii
Chapter 1
Introduction
Charles Forceville and Eduardo Urios-Aparisi...................................................... 3
Chapter 2
Non-verbal and multimodal metaphor in a cognitivist framework:
Agendas for research
Charles Forceville .............................................................................................. 19
Chapter 3
Brand images: Multimodal metaphor in corporate branding
messages
Veronika Koller ................................................................................................. 45
Chapter 4
Cutting across the senses: Imagery in winespeak and
audiovisual promotion
Rosario Caballero............................................................................................... 73
Chapter 5
Interaction of multimodal metaphor and metonymy in TV
commercials: Four case studies
Eduardo Urios-Aparisi ...................................................................................... 95
Chapter 6
Nonverbal and multimodal manifestation of metaphors
and metonymies: A case study
Ning Yu ........................................................................................................... 119
vi Table of contents
Chapter 7
Visual metaphor versus verbal metaphor: A unified account
Francisco Yus................................................................................................... 147
Chapter 8
Metaphor in political cartoons: Exploring audience responses
Elizabeth El Refaie........................................................................................... 173
Chapter 9
Image alignment in multimodal metaphor
Norman Y. Teng .............................................................................................. 197
Chapter 10
Visual metaphoric conceptualization in editorial cartoons
Joost Schilperoord and Alfons Maes ................................................................. 213
Chapter 11
Anger in Asterix: The metaphorical representation of anger
in comics and animated films
Bart Eerden ...................................................................................................... 243
Chapter 12
Pictorial metaphors of emotion in Japanese comics
Kazuko Shinohara and Yoshihiro Matsunaka ................................................... 265
Chapter 13
Words, gestures, and beyond: Forms of multimodal metaphor in the
use of spoken language
Cornelia Müller and Alan Cienki ..................................................................... 297
Chapter 14
Metonymy first, metaphor second: A cognitive-semiotic approach to
multimodal figures of thought in co-speech gesture
Irene Mittelberg and Linda R. Waugh ............................................................. 329
Table of contents vii
Chapter 15
Music, language, and multimodal metaphor
Lawrence M. Zbikowski ................................................................................... 359
Chapter 16
The role of non-verbal sound and music in multimodal metaphor
Charles Forceville ............................................................................................ 383
Chapter 17
Multimodal metaphor in classical film theory from the 1920s
to the 1950s
Mats Rohdin..................................................................................................... 403
Chapter 18
Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor
in horror films
Gunnar Theodór Eggertsson and Charles Forceville ......................................... 429
Rosario Caballero
Department of Modern Philology
University of Castilla-La Mancha
Ciudad Real, Spain
Alan Cienki
Department of Language and Communication
Vrije Universiteit Amsterdam
Amsterdam, The Netherlands
Bart Eerden
Research group Visual Rhetoric
Avans Academy, AKV|St. Joost
Breda, The Netherlands
Lisa El Refaie
Centre for Language and Communication Research
Cardiff University
Cardiff, United Kingdom
Charles Forceville
Department of Media Studies
University of Amsterdam
Amsterdam, The Netherlands
Veronika Koller
Department of Linguistics and English Language
Lancaster University
Lancaster, United Kingdom
Fons Maes
Department of Communication and Information Studies
University of Tilburg
Tilburg, The Netherlands
x List of contributors
Yoshihiro Matsunaka
Faculty of Arts
Tokyo Polytechnic University
Tokyo, Japan
Irene Mittelberg
Human Technology Centre
RWTH Aachen University
Aachen, Germany
Cornelia Müller
Department of Cultural Studies
European University Viadrina
Frankfurt Oder, Germany
Mats Rohdin
Department of Cinema Studies
Stockholm University
Stockholm, Sweden
Joost Schilperoord
Department of Communication and Information Studies
University of Tilburg
Tilburg, The Netherlands
Kazuko Shinohara
Institute of Symbiotic Science and Technology
Tokyo University of Agriculture and Technology
Tokyo, Japan
Norman Y. Teng
Institute of European and American Studies
Academia Sinica
Taipei, Taiwan
Eduardo Urios-Aparisi,
Department of Modern and Classical Languages
University of Connecticut
Storrs, CT, USA
List of contributors xi
Linda R. Waugh
Department of French and Italian
University of Arizona
Tucson, AZ, USA
Ning Yu
Department of Modern Languages, Literatures, and Linguistics
University of Oklahoma
Norman, OK, USA
Francisco Yus
Department of English Studies
University of Alicante
Alicante, Spain
Lawrence Zbikowski
Department of Music
University of Chicago
Chicago, IL, USA
Preface
The editors would like to thank at least a few among the many people who
made this book possible. First of all, although one of us had been toying
with the idea for this volume for a long time, it was the chance to organize a
conference panel on multimodal discourse at the 9th conference of the
International Pragmatics Association (Riva del Garda 2005), that really got
things going. In Jef Verschueren and Ann Verhaert we want to thank IPrA
for this opportunity. Anke Beck, at Mouton, was enthusiastic about the
book project well before it deserved that name: she actually invited us to
submit a book proposal when we had scarcely even sent out the call for
papers for the conference panel. We thank her for her trust, and Birgit
Sievert and Monika Wendland for guiding us through many practicalities at
Mouton during later stages. We are also indebted to Gitte Kristiansen,
managing editor of the Applications of Cognitive Linguistics series, and to
Francisco Ruiz de Mendoza, a staunch promoter of the book from its earli-
est beginning. Most of all we are grateful, of course, to our authors, who
graciously responded to our critical comments on chapter drafts, and our
requests for further revisions and fine-tuning.
Charles Forceville: I furthermore want to acknowledge how much I
benefited from the interactions with the many students who, over the past
ten years, followed my course on pictorial and multimodal metaphor in the
Media Studies department at the Universiteit van Amsterdam, and from the
fine papers they wrote. It is a source of pleasure and pride for me to have
two of these former students as authors in this volume. I also want to ex-
press my appreciation for the academic associations that, despite their
strong orientation toward verbal discourse, provided me with a platform to
talk about metaphor in advertising, comics, and film: the International
Cognitivist Linguistics Association (ICLA), the Researching and Applying
Metaphor (RaAM) association, and the Poetics and Linguistics Association
(PALA). It is one of the privileges of being a scholar that one has the op-
portunity to attend international conferences, where shared professional
interests often lead to warm personal contacts. I have good memories of
many such conferences. Among the numerous colleagues that have inspired
me, I will mention one person by name. Ray Gibbs has always been excep-
tionally generous to me with his time, encouragement, and expertise.
In 2008 I spent six months at the Centre for Advanced Studies of the
Royal Flemish Academy of Belgium for Science and the Arts (VLAC), in
xiv Preface
Brussels, where I had the pleasure of collaborating with Kurt Feyaerts and
Tony Veale on the research project The Agile Mind: Creativity in Models and
Multimodal Discourse. The former Royal stables at Hertogstraat 1 provided a
stimulating environment for carrying out the final editing rounds of this
volume. And last but not least, I want to say how lucky I have always been
with the sound and commonsense advice of Kuif’s agency.
Eduardo Urios-Aparisi: I would like to thank my colleagues and the de-
partment of Modern and Classical Languages of the University of Con-
necticut, Storrs. I want to express my deepest gratitude to all those who
have helped me through all these years with patience, encouragement and
love.
Charles Forceville
Eduardo Urios-Aparisi
Introduction
We believe that the book you have in your hands is pertinent to scholars
in both metaphorology and multimodality. Clearly, metaphorists considering
themselves adherents of the Conceptual Metaphor Theory (CMT) initiated
by Lakoff and Johnson need to take seriously at least one crucial conse-
quence of the tenet that “metaphor [is] not a figure of speech, but a mode of
thought” (Lakoff 1993: 210): that metaphor can occur in other modes than
language alone. Indeed they must do so, for if researching non-verbal and
not-purely-verbal metaphor does not yield robust findings, this jeopardizes
the Lakoff-and-Johnsonian presupposition that we think metaphorically.
After all, in that case the supposedly metaphorical nature of human thinking
would turn out to be a misconception: what has been presented as the CON-
CEPTUAL level of metaphor would then simply be verbal metaphor under a
different name, disguised in SMALL CAPITALS. Mark Johnson appears to
agree, arguing that lurking behind an exclusive focus on language is the
prejudice that meaning is only to be found in words. He emphasizes that “the
processes of embodied meaning in the arts are the very same ones that make
linguistic meaning possible” (2007: 209). Of course work to correct the one-
sided emphasis on verbal manifestations has already been done, notably on
gesture and pictures, both by authors represented in this book and by others.
What is new in this book is that it focuses not so much on non-verbal meta-
phor per se, but on multimodal metaphor, that is, on metaphors whose target
and source are rendered exclusively or predominantly in two different
modes/modalities (the terms “mode” and “modality” are currently both in
use; it is unclear at present which one will catch on) – and in many cases the
verbal is one of these. The definition of a mode is an extremely thorny one
(for more discussion, see Forceville 2006/this volume). For present pur-
poses, the modes to be taken into account are two or more of the following:
(1) written language; (2) spoken language; (3) static and moving images; (4)
music; (5) non-verbal sound; (6) gestures. Since what can be conveyed in
terms of facts, emotions, and aesthetic pleasure differs from one mode to
another, the choices for (one) particular mode(s) over (an)other(s) that the
producer of a multimodal metaphor has to make is/are bound to affect its
overall meaning. One mode’s potential to render “meaning” can never be
completely “translated” into that of another mode – and sometimes transla-
tion is downright impossible. For this reason alone, a healthy theory of (cog-
nitive) metaphor must systematically study non-verbal and multimodal
metaphor. It may well be – indeed it is very probable – that the excessive
emphasis on the verbal manifestations of metaphorical thought has blinded
researchers to dimensions of the latter that quite simply cannot be cued by
the verbal mode.
Introduction 5
vents (cf. Haser 2005: 50). In addition, each chapter is thereby expected to
spawn ideas how the proposed procedure can be deployed to analyze other
multimodal representations than those examined there. Contributors were
also encouraged to present (some of) their conclusions in a form that allows
for empirical testing. Most of those we approached responded positively, and
of the latter, the majority of the delivered chapters displayed the quality we
had in mind. Early drafts of the chapters were extensively commented upon
both by the editors and by one other contributing author.
The guiding principle running through the chapters is a consideration of
which modes play a role in the identification and interpretation of the meta-
phors studied. Almost invariably, this entails taking into account the genre to
which the discourse featuring a multimodal metaphor belongs: advertise-
ments, political cartoons, comics, animation, musical compositions, oral
conversations and lectures, feature films. A third recurring dimension is the
extent to which a metaphor is not only embodied but also governed by the
cultural or professional community in which it functions. We will now
briefly introduce each of the chapters in the book.
Chapter 2 is a slightly updated version of the position paper on pictorial
and multimodal metaphor by Forceville (2006). This paper provides and
discusses the definition of multimodal metaphor that contributors to the cur-
rent volume were asked to use – or else explain why they opted for an alter-
native definition.
The first cluster of chapters pertains to multimodal metaphor in advertis-
ing. It makes sense to begin with this topic, since advertising has been the
subject of a number of studies pertaining to pictorial metaphor – the variety
of non-verbal metaphor that hitherto has attracted most scholarly attention.
This is not surprising, for advertising constitutes a body of texts and prac-
tices that is persuasive par excellence. It allows bringing into play the modes
of language, visuals, and sound/music. The first contribution in this cluster,
“Brand images: Verbal and visual metaphor in corporate branding mes-
sages,” by Veronika Koller (chapter 3), charts how the logos, visuals, and
layouts that are used to create companies’ corporate identities often require
or invite the construal of metaphors. Tying in with the pervasive BRANDS
ARE LIVING ORGANISMS metaphor, visual elements often subtly encourage
the inference of positive corporate qualities that are not necessarily verbal-
ized. Identifying the metaphorical mechanisms deployed to achieve this goal
points the way to how the inevitably biased nature of companies’ self-
portraits can be critically examined.
Chapter 4 is Rosario Caballero’s “Cutting across the senses: Imagery in
winespeak and audiovisual promotion.” The chapter is part of an ongoing
Introduction 7
this volume. Rohdin thus is the only contributor to present a diachronic per-
spective on the issue of multimodal metaphor. Moreover, he draws attention
to the fact that cinematic metaphors may acquire extra meanings because
through visual styling they can create intertextual references to other films
and phenomena familiar from everyday life. Finally, Rohdin finds that, con-
trary to expectation, the silent cinema was particularly rich in multimodal
metaphors of the verbo-pictorial variety, due to the creative use of intertitles.
The final chapter, co-authored by Gunnar Eggertsson and Charles Force-
ville, is titled “Multimodal expressions of the HUMAN VICTIM IS ANIMAL
metaphor in horror films” (chapter 18). Its key argument is that human vic-
tims in extreme horror films are typically abused as if they were animals.
The findings shed light on metaphor theory, the genre of horror films, but
they also encourage reflection on the issue of animal rights for, in the spirit
of Kövecses (2005) we can adapt a famous dictum and say: “show me your
metaphors and I will tell you who you are.”
The division in clusters and chapters chosen – loosely on the basis of gen-
res and modes – could have been made in different ways, since many other
thematic patterns can be detected across the chapters of the book. Without
elaborate discussion, we will briefly list some of these patterns, presenting
them as something with a status that hovers between hypothesis and research
program. Some of the issues have been discussed in relation with verbal
metaphors, but often their importance has been underestimated in that realm;
others appear to reveal themselves precisely thanks to the multimodal nature
of the metaphors that are the specific focus of attention here.
Many metaphors are mini-narratives. The paradigmatic NOUN A IS
NOUN B formula disguises the dynamic nature of metaphor. Human beings
move literally through space and figuratively through time, and it is within
these parameters that they need to make sense of their lives. This sense-
making happens through real or imagined metaphor actions; it would per-
haps be better to conceive of metaphor as A-ING IS B-ING, since metaphor is
always metaphor in action. The A IS B format – which maybe became popu-
lar also because CMT long discussed only decontextualized metaphors that
already came in a ready-made verbal “A is B” form – is no more than a con-
venient short-hand for what Andreas Musolff calls a “metaphor scenario”
(Musolff 2006). And of course we should not forget that Paul Ricoeur
(1977) already strongly emphasized the discursive character of metaphor.
Though not always explicitly, all chapters in the volume tie in with this no-
tion of a scenario or a narrative.
Target and source in multimodal metaphor may both be concrete enti-
ties. Classic CMT has always stressed that human beings can only come to
12 Charles Forceville and Eduardo Urios-Aparisi
grips (sic) with the abstract by metaphorically coupling it with the concrete –
i.e., with that which is perceptible. But the chapters in this volume are re-
minders that only a target that is concrete is, for instance, depictable, which
is important in advertising a product, satirizing a politician in a cartoon, or
conveying information about a character in a film. The focus on verbal
manifestations of conceptual metaphors, that is, has had as an unfortunate
side effect that for instance the stylistic dimensions of metaphors and other
tropes have been somewhat ignored by cognitivist scholars (but cf. Semino
and Culpeper 2002). Many illuminating (aesthetic as well as persuasive)
multimodal metaphors convey something about this specifically styled target
in terms of this specifically styled source. Moreover, while the “embodied”
nature of conceptual metaphors is one of the basic tenets of CMT, Caballero
(this volume) correctly points out that the embodied domains of smell and
taste need rather than provide metaphorical sources. The strong focus on a
bottom-up approach (from attested “textual” manifestations to formulations
of the conceptual metaphors which supposedly underlie them rather than the
other way round) may also be the reason why in several of the chapters there
is some interference of the terminology associated with Max Black’s (1979)
interaction theory. Black – whose early contributions to cognitive theories of
metaphor have insufficiently been acknowledged by most CMT theorists –
anticipated that metaphor could be a matter of thought rather than language,
but discussed specific, creative metaphors in terms of “features” that were
projected or transferred from source to target. CMT favors referring to this
process as the partial mapping of entities and knowledge structures from
source to target, resulting in a (temporary) understanding of the target in
terms of the source – but the occasional lapse into Black’s terminology is a
healthy reminder that sometimes no more than a single aspect (“feature”) of
the source is mapped.
It is impossible to study metaphor without addressing metonymy. Me-
tonymy has over the past decade begun to receive sustained attention from
cognitive linguists (Barcelona 2000; Dirven and Pörings 2002; Kristiansen
et al. 2006). Clearly, each property or feature that is mapped from a source
to a target must first have been metonymically related to that source. Of
course, a metonym can be an ad hoc one, created by a particular context or
shared by a specific community of users (cf. Yus, this volume). In addition,
a metonym may have a strong emotional or evaluative relation to its source –
and it may well be this latter that is the rationale for the metaphor in the first
place. Secondly, a given phenomenon may double as the source domain in a
metaphor and as metonymically related to the target. If this is the case, the
consequence may be that a construal of the relation between two things as
Introduction 13
metaphor is invited rather than forced; after all there may be a realistic,
metonymic motivation for the source’s presence on the grounds of expected
contiguity in the domain of the target. The interaction between metaphor and
metonymy is explicitly addressed in the chapters by Urios-Aparisi, Yu, and
Mittelberg and Waugh.
Non-verbal and multimodal metaphors may make salient certain as-
pects of conceptual metaphors that are not, or not as clearly, expressible in
their verbal manifestations. The role of for instance size and spatial dimen-
sions in source domains (e.g., in POWERFUL IS BIG, HONEST IS STRAIGHT) is
more noticeable in visual discourses than in verbal ones. Music, in turn,
affords for example scalarity and loudness in ways that can be made produc-
tive in source-to-target mappings, and the same holds for a voice’s timbre or
an intonational pattern. Arm-and-hand gestures, both in face-to-face interac-
tion and in the stylized varieties characterizing protagonists’ behaviors in
comics, manga, and animation are embodied actions whose metaphorical
exploitation communicates perspectives and emotions not (readily) available
in verbal metaphors. A consequence of this is that any “translation” of these
non-verbal and multimodal metaphors into verbal ones – necessary for in-
stance to enable scholarly discussion as in this book – inevitably is an ap-
proximation at best. Metaphor scholars should be acutely aware of this, and
reflect on what the choice for one verbalization of a multimodal metaphor
over another may entail. The verbal “short-hands” of multimodal metaphors
suggest an explicitness and precision that may well be absent in their origi-
nally non-verbal or multimodal, forms. Aspects of this issue are addressed in
the chapters by Eerden, Shinohara and Matsunaka, Yu, Yus, El Refaie,
Mittelberg and Waugh, Müller and Cienki, Teng, Rohdin, Zbikowski, and
Forceville.
Personification is a crucial variety of multimodal metaphor no less than
of verbal metaphor. Living organisms and animals are attractive choices as
source domains both for human target domains and for phenomena such as
organizations and cars. This makes sense for a variety of reasons: as hu-
mans, we find fellow humans as well as animals provide rich opportunities
for the mapping both of idiosyncratic features (snails are typically slow,
peacocks proud and beautiful) and for what Black called “implicative com-
plexes” (Black 1979) and Gentner and Loewenstein (2002) “aligned struc-
tures.” To a considerable extent, the place of humans and animals in the
medieval hierarchy of the Great Chain of Being (see Tillyard 1976 [1943],
Lakoff and Turner 1989) is still pertinent today, but creatures’ status can
also be strongly influenced by cultural myths (think of the connotations of
14 Charles Forceville and Eduardo Urios-Aparisi
References
Musolff, Andreas
2006 Metaphor scenarios in public discourse. Metaphor and Symbol 21:
23–38.
O’Halloran, Kay L. (ed.)
2004 Multimodal Discourse Analysis. London/New York: Continuum.
Ortony, Andrew (ed.)
1979 Metaphor and Thought. Cambridge: Cambridge University Press.
1993 Metaphor and Thought. 2nd ed. Cambridge: Cambridge University
Press.
Pragglejaz Group
2007 MIP: A method for identifying metaphorically used words in dis-
course. Metaphor and Symbol 22: 1–39.
Ricoeur, Paul
1977 The Rule of Metaphor: Multi-disciplinary Studies of the Creation of
Meaning in Language. Trans. by R. Czerny, Kathleen McLaughlin,
and John Costello. Toronto: University of Toronto Press.
Ruiz de Mendoza, Francisco José, and Olga Isabel Díez Velasco
2002 Patterns of conceptual interaction. In Metaphor and Metonymy in
Comparison and Contrast, René Dirven and Ralph Pörings (eds.),
489–532. Berlin/New York: Mouton de Gruyter.
Semino, Elena, and Jonathan Culpeper (eds.)
2002 Cognitive Stylistics: Language and Cognition in Text Analysis. Am-
sterdam/Philadelphia: Benjamins.
Sperber, Dan, and Deirdre Wilson
1995 Relevance: Communication and Cognition. 2nd ed. Oxford: Black-
well.
Tillyard, E.M.W.
1976 [1943] The Elizabethan World Picture. Harmondsworth: Penguin.
Ventola, Eija, Cassily Charles, and Martin Kaltenbacher (eds.)
2004 Perspectives on Multimodality. Amsterdam/Philadelphia: Benja-
mins.
Chapter 2
Charles Forceville
Abstract
Conceptual metaphor theory (CMT) has over the past 25 years amply sought to
underpin the claim that humans’ pervasive use of verbal metaphor reflects the fact
that they think largely metaphorically. If this tenet of CMT is correct, metaphor
should manifest itself not just in language but also via other modes of communica-
tion, such as pictures, music, sounds, and gestures. However, non-verbal and mul-
timodal metaphor have been far less extensively studied than their verbal sisters.
The present chapter provides a review of work done in this area, focusing on a
number of issues that require further research. These issues include the proposal
to distinguish between monomodal and multimodal metaphor; reflections on the
difference between structural and creative metaphor; the question of how verbali-
zations of non-verbal or conceptual metaphors may affect their possible interpreta-
tion; thoughts as to how similarity between target and source is created; and sug-
gestions about the importance of genre for the construal and interpretation of
metaphor.
1. Introduction
Andrew Ortony’s edited volume Metaphor and Thought (1979) and Lakoff
and Johnson’s monograph Metaphors We Live By (1980) were milestone
publications in the sense that they marked the switch from research into
metaphor as a primarily verbal to a predominantly conceptual phenomenon.
The “conceptual metaphor theory” (CMT), as the Lakoffian-Johnsonian
20 Charles Forceville
model is habitually referred to, has been a very productive one (e.g., Gibbs
1994; Johnson 1987, 1993, 2007; Kövecses 1986, 2000, 2002; Lakoff 1987,
1993; Lakoff and Johnson 1999, 2003; Lakoff and Turner 1989; Sweetser
1990; Turner 1996). A key notion in this theory is that “the mind is inher-
ently embodied, reason is shaped by the body” (Lakoff and Johnson 1999: 5;
Chapter 3). Very briefly, what this means is the following. Human beings
find phenomena they can see, hear, feel, taste and/or smell easier to under-
stand and categorize than phenomena they cannot. It is perceptibility that
makes the former phenomena concrete, and the lack of it that makes the lat-
ter abstract. In order to master abstract concepts, humans systematically
comprehend them in terms of concrete concepts. Thus abstract concepts such
as LIFE, TIME, and EMOTIONS are systematically understood in terms of con-
crete phenomena. LIFE is understood as A JOURNEY (He’s without direction
in his life; I’m at a crossroads in my life) – but also, for instance, as A
STORY (Tell me the story of your life; Life’s … a tale told by an idiot ...).
TIME is comprehended in terms of SPATIAL MOTION (The time for action has
arrived; Time is flying by; He passed the time happily). Emotions are
typically represented by drawing on the domain of FORCES. (I was over-
whelmed; I was swept off my feet; examples from Kövecses 2000; Lakoff
1993; Lakoff and Johnson 1980). Conceptualizations of many phenomena,
CMT proposes, have deeply entrenched metaphorical forms, in which the
metaphor’s target (topic, tenor) is abstract and its source (vehicle, base) is
concrete. A metaphor’s interpretation boils down to the “mapping” of perti-
nent features from the source to the target; a mapping that in the case of
entrenched metaphors such as the above occurs automatically. Since “con-
creteness” is apprehended perceptually, metaphorical source domains are
strongly rooted in the functioning of the human body. Metaphorical reason-
ing is thus governed by the “arch” metaphor MIND IS BODY (Lakoff and
Johnson 1999: 249). A more recent development rooted in CMT is “blending
theory” (Fauconnier and Turner 2002). Rather than postulating a target and
a source domain, it presents two (or more) “input spaces.” The input spaces
have both shared and unique characteristics, and it is this combination that
allows for the construal of a so-called “blended space.” Blending theory is a,
mainly descriptive, model claiming to be superior to metaphor theory in be-
ing able to account for ad hoc linguistic creativity, metaphorical and other-
wise. Hitherto it cannot quite convince (for a critical review, see Forceville
2004a), but new work, taking into account pragmatic rhetorical factors, is
promising (see Coulson and Pascual 2006; Terkourafi and Petrakis, forth-
coming).
Metaphor in a cognitivist framework 21
cannot be based on its verbal manifestations alone, since this may result in a
biased view of what constitutes metaphor.
In this chapter I will sketch how adopting the view that metaphors can
assume non-verbal and multimodal appearances can and should guide the
research of a new generation of metaphor scholars. I will do so partly by
bringing to bear multimodal perspectives on issues already familiar from
research by language-oriented metaphor scholars, and partly by discussing
issues that have either been neglected by such researchers or are simply not
pertinent to purely verbal metaphors. The chapter should be seen as a map
of mostly uncharted territory, with only a few details inked in, much of it
reporting theory-driven analyses and informed speculation awaiting empiri-
cal testing. Multimodal metaphor researchers have a vast amount of work to
look forward to.
ple: imagine somebody wants to cue, for whatever reason, the metaphor CAT
IS ELEPHANT pictorially in an animation film. She could do this for instance
by depicting the cat with a trunk-like snout and large flapping ears; by show-
ing the cat with a canopy on its back in which a typical Indian elephant rider
is seated; by juxtaposing cat and elephant in the same salient pose; or by
letting the cat behave (for instance: move) in an elephant-like manner. These
variants would constitute monomodal metaphors of the pictorial kind, featur-
ing hybrid, contextual, simile, and integrated subtypes respectively (see For-
ceville 1996, 2002b, 2005a) – and of course these subtypes could be com-
bined. Now imagine the producer wishes to cue the same metaphor
multimodally. She could for instance have the cat make a trumpeting sound
or have another cat shout “elephant!” to the first one (note that this is not a
case of synaesthesia, since there is no conflation of the two domains). In
these cases the source domain ELEPHANT would be triggered in two modes
(sound and language, respectively) that are different from the target
(visuals). By this token, the metaphor would be truly multimodal. But, as in
the case of the visual mode alone, the producer would of course not have to
choose between any of these modes: she could depict the cat with a trunk-
like snout and large ears and have it trumpet, and have another cat shout
“elephant!” In this case, the source is cued in three modes simultaneously,
only one of these (namely: the visual) exemplifying the same mode as the
target. In such a case I also propose to label the metaphor multimodal. Of
course the metonymy cueing the source domain in itself is often chosen for
its specific connotations. Both tusks and a trunk trigger ELEPHANT, but the
former connotes, among other things, aggressiveness, whiteness, costliness
and the latter among other things flexibility, sensitivity, and “instrument-to-
spray-water-or-sand-with.” For examples, as well as more discussion, of
multimodal metaphors involving (moving) images, see Forceville (1999a,
2003, 2004b, 2005b, 2007, 2008). There is also a growing literature on
multimodal metaphors involving language and gestures (Cienki 1998;
McNeill 1992; Müller 2004), in which the gesture-modality cues the source
rather than the target domain (McNeill 2005: 45).
Lakoff and Turner (1989) have argued that not only metaphors occurring in
everyday verbal communication can be traced back to conceptual metaphors,
but also those in artistic texts, specifically poetry. Particularly when poems
thematize abstract concepts such as life and death, they cannot but draw on
26 Charles Forceville
monomodal metaphors of the pictorial variety, both target and source are
depicted. In advertising, metaphorical targets usually coincide with promoted
products and, unsurprisingly, are depicted – and hence are necessarily con-
crete: a beer brand is depicted as a wine; an elegant watch as a butterfly, a
close-fitting bathing suit as a dolphin’s tight and supple skin (examples from
Forceville 1996). The same holds for metaphors in feature films (Forceville
2005b; Whittock 1990). In short, to what extent monomodal metaphors of
the non-verbal variety and multimodal metaphors are amenable to the corre-
lation metaphors that are the center of attention in CMT is an empirical
question. Some of them no doubt do; for instance, the personification of
commodities is a very familiar marketing strategy, and ties in with CMT
views (Lakoff and Turner 1989: 72). But many pictorial and multimodal
metaphors are of the OBJECT A IS OBJECT B type. Traditional CMT has not
much to say about these. Even Lakoff and Turner’s (1989) invocation of the
Great Chain metaphor is only of limited use here, since it depends on typo-
logical hierarchies that may be subverted, or simply irrelevant, in creative
metaphors, many of which function in contexts creating highly specific, ad
hoc metaphorical resemblances (see Black 1979).
There is a third aspect in which CMT has a one-sided emphasis. As dis-
cussed above, the typical source domain’s concreteness has in CMT been
traditionally connected to the notion of “embodiment.” The embodied nature
of source domains emphasizes their physical nature: it is human physical
interaction with the world that familiarizes humans with it to such an extent
that the resulting knowledge structures can in turn be mapped onto abstract
concepts. Knowledge about source domains is not simply a matter of em-
bodiment, however, but also of cultural connotations, as Lakoff and Turner
(1989: 66) acknowledge. More recent studies have demonstrated in a variety
of ways how the structure of source domains – and the salient (and hence:
easily mappable) elements in it – is influenced by culture (Gibbs and Steen
1999; Kövecses 2005; Shore 1996; Yu 1998). Indeed, the cultural connota-
tions that are metonymically related to a source domain are often more im-
portant for potential mappings to a target than its embodied aspects. In a
Dutch commercial promoting a Gazelle bicycle in terms of a dressage horse
the embodied mapping of “riding a horse” to “riding a bicycle” is less impor-
tant for the interpretation of the metaphor than the mapping of the cultural
connotations from the dressage horse’s owner, champion Anky van Gruns-
ven, to the prospective buyer and user of the bike. Similarly, while advertis-
ing a high-tech Senseo coffee machine in terms of a motorbike certainly has
Metaphor in a cognitivist framework 29
We have seen that within the CMT paradigm, most surface metaphors
should be amenable to a pre-existing conceptual A IS B format. Inevitably, in
order to discuss the metaphor, this A and B must be named, i.e., rendered in
language. It is by no means a foregone conclusion, of course, that the “lan-
guage of thought” is actually a verbal language. The convention to verbalize
the image-schematic structures underlying surface metaphors by using
SMALL CAPITALS – useful inasmuch as this facilitates analyzing them – may
disguise a number of consequences that seem to me more problematic in the
discussion of discourses that are not (exclusively) verbal ones than of purely
verbal ones. One of these consequences is that it is the analyst’s responsibil-
ity to find an adequate or acceptable verbal rendering of the metaphor’s un-
derlying image-schematic level, but such a verbalization, even though used
as a convenient shorthand, is never neutral. The design of the Senseo coffee
machine suggests the posture of somebody bending over and modestly offer-
ing something (i.e., a cup) on a plate. But should this awareness result in the
verbalization COFFEE MACHINE IS SERVANT, or is COFFEE MACHINE IS BUT-
LER more appropriate? Although “servant” and “butler” share many fea-
tures, they also differ: “butler” is more specific, and may in some people
(but not in others) evoke connotations of Britishness and standards of service
that “servant” does not. As a result, the mappings suggested by the two ver-
balizations may differ. Bartsch might conclude that this very inability to
agree on a single verbalization of the source domain that is shared within a
community shows that the source has no conceptual status, and reflects a
“quasi-concept” at best (Bartsch 2002: 50). However, to the extent that there
is a community that recognizes the source as cueing a serving person, the
source admits predicates understood as “true” in the community (such as “is
there to serve the user,” “obeys your requests,” and “is almost always avail-
able”). In a visually literate society, a vast number of endlessly repeated and
recycled images (such as famous paintings, photographs, film shots, flags,
logos, animation characters) evoke specific phenomena and events in a cli-
chéd, shorthand manner widely shared within a community, and hence ar-
guably aspire to conceptual status. But this speculation leads us far beyond
the concerns of the present chapter and deserves in-depth reflection else-
where.
Another consequence is that verbalization of a non-verbal metaphor is
necessarily a conscious action, and a fairly unusual one at that. It is only the
scholar writing an academic paper who, to be able to discuss a multimodal
Metaphor in a cognitivist framework 31
for discussion). For this reason, it is important to study how genre has an
impact on the production and interpretation of metaphors (monomodal and
multimodal alike). In advertising, for instance, the targets of metaphors often
coincide with the product promoted (Forceville 1996). This is to be ex-
pected: an advertisement or commercial predicates something about a prod-
uct, brand, or service, and this neatly and naturally fits the metaphor’s TAR-
GET IS SOURCE format. Moreover, the features mapped from source to target
are positive ones (unless the metaphor is used to disqualify a competitor’s
brand, in which case the mapped features are typically negative). But in
feature films, there is no phenomenon that in a similar, “natural,” way quali-
fies as a metaphorical target. Metaphors in artistic narratives pertain to phe-
nomena that, for whatever reason, are deemed salient by their producers.
These phenomena can be protagonists, but also objects, or even events. The
mapped features will often be less clear-cut, and may have a richer “aligned
structure” (Gentner and Loewenstein 2002), than those in advertising.
Metaphors in artistic representations may also differ in other respects
from those in commercial messages. For instance, while in commercials
there will seldom be a question what is target and what is source in a meta-
phor, an artistic narrative may give rise to two different construals of a
metaphor: both A IS B and B IS A are appropriate. (While Carroll 1994, 1996
calls such metaphors “reversible,” I prefer to say that, in the given context,
both the metaphors A IS B and B IS A are pertinent, in order to retain the no-
tion that target and source in a metaphor are, in principle, irreversible.)
Commensurate with this, metaphors in artistic contexts presumably allow
for greater freedom of interpretation than do metaphors in commercials (cf.
also Shen 1995).
Another parameter that deserves further research is whether any of the
subtypes of pictorial metaphor or of the manifold varieties of multimodal
metaphor can be systematically related to certain text genres. For instance, it
seems that commercial advertising seldom makes use of the hybrid variety of
pictorial metaphors (in Forceville 1996 these were called MP2s). Again, this
makes sense: if metaphorical targets typically coincide with products, adver-
tisers would want their product portrayed in their entirety, and not in a man-
ner that might evoke connotations of incompleteness or mutilation. Hybridiz-
ing it with a metaphorical source domain would not fit this goal. By contrast,
in animation films, or science fiction films, no such problem arises.
Finally, it would be worthwhile to investigate whether in the case of mul-
timodal metaphors there are any systematic correlations between textual
genres and the modes in which a target and source are represented. In adver-
tisements, the visual mode is typically used for representing the target – and
34 Charles Forceville
this may well be true for different genres as well. But perhaps alternative
patterns in the choice of mode for the source domain are detectable in differ-
ent types of texts, while this may also change over time within a genre.
6. Concluding remarks
Acknowledgments
I am indebted to Francisco Ruiz de Mendoza and René Dirven for comments and
suggestions on earlier drafts of this paper.
Metaphor in a cognitivist framework 35
Notes
References
Altman, Rick
1999 Film/Genre. London: British Film Institute.
Bartsch, Renate
2002 Generating polysemy: Metaphor and metonymy. In Metaphor and
Metonymy in Comparison and Contrast, René Dirven, and Ralf
Pörings (eds.), 49–74. Berlin/New York: Mouton de Gruyter.
Black, Max
1979 More about metaphor. In Metaphor and Thought, Andrew Ortony
(ed.), 19–43. Cambridge: Cambridge University Press.
Bolter, Jay David, and Richard Grusin
1999 Remediation: Understanding New Media. Cambridge MA: MIT
Press.
Brooke-Rose, Christine
1958 A Grammar of Metaphor. London: Secker and Warburg.
Carroll, Noel
1994 Visual metaphor. In Aspects of Metaphor, Jaakko Hintikka (ed.),
189–218. Dordrecht: Kluwer.
Carroll, Noel
1996 A note on film metaphor. In Theorizing the Moving Image, 212–
223. Cambridge: Cambridge University Press.
Charteris-Black, Jonathan
2004 Corpus Approaches to Critical Metaphor Analysis. London: Pal-
grave-MacMillan.
36 Charles Forceville
Cienki, Alan
1998 Metaphoric gestures and some of their relations to verbal meta-
phoric expressions. In Discourse and Cognition: Bridging the Gap,
Jean-Pierre Koenig (ed.), 189–204. Stanford, CA: Center for the
Study of Language and Information.
Cook, Nicholas
1998 Analyzing Musical Multimedia. Oxford: Clarendon.
Coulson, Seana, and Esther Pascual
2006 For the sake of argument: Mourning the unborn and reviving the
dead through conceptual blending. Annual Review of Cognitive Lin-
guistics 4: 153–181.
Crisp, Peter
2003 Conceptual metaphor and its expression. In Cognitive Poetics in
Practice, Joanna Gavins and Gerard Steen (eds.), 99–113. Lon-
don/New York: Routledge.
Cupchik, Gerry D.
2003 The “interanimation” of worlds: Creative metaphors in art and de-
sign. The Design Journal 6 (2): 14–28.
Dirven, René, and Ralf Pörings (eds.)
2002 Metaphor and Metonymy in Comparison and Contrast. Berlin/New
York: Mouton de Gruyter.
Durand, Jacques
1987 Rhetorical figures in the advertising image. In Marketing and Semi-
otics: New Directions in the Study of Signs for Sale, Jean Umiker-
Sebeok (ed.), 295–318. Berlin/New York: Mouton de Gruyter.
Eerden, Bart
this vol. Anger in Asterix: The metaphorical representation of anger in com-
ics and animated films.
El Refaie, Elisabeth
2003 Understanding visual metaphors: The example of newspaper car-
toons. Visual Communication 2 (1): 75–95.
Fauconnier, Gilles, and Mark Turner
2002 The Way We Think: Conceptual Blending and the Mind’s Hidden
Complexities. New York: Basic Books.
Forceville, Charles
1988 The case for pictorial metaphor: René Magritte and other Surreal-
ists. In Vestnik IMS 9, Aleš Erjavec (ed.), 150–160. Ljubljana: Inšti-
tut za Marksistične Študije [Slovenia].
1994 Pictorial metaphor in advertisements. Metaphor and Symbolic Ac-
tivity 9: 1–29.
1995 (A)symmetry in metaphor: The importance of extended context.
Poetics Today 16: 677–708.
1996 Pictorial Metaphor in Advertising. London/New York: Routledge.
1999a The metaphor COLIN IS A CHILD in Ian McEwan’s, Harold Pinter’s,
and Paul Schrader’s The Comfort of Strangers. Metaphor and
Symbol 14: 179–98.
Metaphor in a cognitivist framework 37
Indurkhya, Bipin
1991 Modes of metaphor. Metaphor and Symbolic Activity 6: 1–27.
1992 Metaphor and Cognition: An Interactionist Approach. Dordrecht:
Kluwer.
Johnson, Mark
1987 The Body in the Mind: The Bodily Basis of Meaning, Imagination
and Reason. Chicago: University of Chicago Press.
1993 Moral Imagination: Implications of Cognitive Science for Ethics.
Chicago: University of Chicago Press.
2007 The Meaning of the Body: Aesthetics of Human Understanding.
Chicago and London: University of Chicago Press.
Johnson, Mark, and Steve Larson
2003 “Something in the way she moves” – metaphors of musical motion.
Metaphor and Symbol 18: 63–84.
Kaplan, Stuart J.
1990 Visual metaphors in the representation of communication technol-
ogy. Critical Studies in Mass Communication 7: 37–47.
1992 A conceptual analysis of form and content in visual metaphors.
Communication 13: 197–209.
Kennedy, John M.
1982 Metaphor in pictures. Perception 11: 589–605.
1993 Drawing and the Blind: Pictures to Touch. New Haven/London:
Yale University Press.
Knop, Sabine de, René Dirven, Ning Yu, and Birgit Smieja (eds.)
2005 Bibliography of Metaphor and Metonymy. Amsterdam/ Philadel-
phia: John Benjamins. Available electronically only at: http://www.
benjamins.com/online/met.
Kövecses, Zoltán
1986 Metaphors of Anger, Pride and Love. Amsterdam/Philadelphia:
Benjamins.
2000 Metaphor and Emotion: Language, Culture, and Body in Human
Feeling. Cambridge: Cambridge University Press.
2002 Metaphor: A Practical Introduction. Oxford: Oxford University
Press.
2005 Metaphor in Culture: Universality and Variation. Cambridge: Cam-
bridge University Press.
Kress, Gunther, and Theo van Leeuwen
1996 Reading Images: The Grammar of Visual Design (2nd ed. 2006).
London/New York: Routledge.
2001 Multimodal Discourse: The Modes and Media of Contemporary
Communication. London: Arnold.
Lakoff, George
1987 Women, Fire and Dangerous Things: What Categories Reveal about
the Mind. Chicago: University of Chicago Press.
1993 The contemporary theory of metaphor. In Metaphor and Thought,
2nd ed., Andrew Ortony (ed.), 202–251. Cambridge: Cambridge
University Press.
40 Charles Forceville
Veronika Koller
Abstract
1. Introduction
In Cognitive Metaphor Theory (see e.g., Barcelona 2000; Lakoff and John-
son 1999, 2003) metaphor, rather than being a merely decorative literary
device, is regarded as an essentially cognitive phenomenon structuring much
of human thought. In particular, metaphor is the means by which the human
mind conceives of one usually abstract entity in terms of another, usually a
more concrete one. The processes by which these metaphorically structured
mental models are brought about have been theorized as mappings from a
source to a target domain, or blends of two or more input spaces (Faucon-
nier and Turner 2002). In any case, it is only in a second step that these
metaphoric models are realized as metaphoric expressions at the surface
level of language, or of any other semiotic mode.
Metaphoric expressions have been most exhaustively studied in the ver-
bal mode, i.e., as surface-level linguistic expressions of metaphorically
structured mental models. This body of work was followed by research into
visual, or pictorial, metaphor (Carroll 1994; Forceville 1994, 1996). How-
ever, most of those studies still addressed mono-modal metaphor, in that
both source and target domains (or input spaces) were provided in the visual
mode and only reinforced, rather than co-constructed, by the verbal co-text.
Multimodal metaphor, on the other hand, is constituted by a mapping, or
blending, of domains from different modes, e.g., visual and verbal, or visual
and acoustic. With a view to the examples presented in this chapter, it seems
Brand images 47
This chapter will focus on the interaction of pictorial and written signs, as
they combine into multimodal metaphoric expressions realizing the concep-
tual metaphor BRANDS ARE LIVING ORGANISMS and specifically, BRANDS
ARE PEOPLE.
The duality of conceptual metaphors as both operating on the cognitive
level and being realized in surface level metaphoric expressions leads to
inductive reasoning on the part of the researcher, in that the analysis of
metaphoric expressions leads to inferences about underlying metaphoric
models. In the case of verbal metaphoric expressions, verbalizations of in-
48 Veronika Koller
ferred conceptual metaphors are to some extent motivated by the verbal for-
mulation of the observed expression. For example, the phrase we channeled
investments in parallel with our corporate DNA, said by a company execu-
tive about their financial strategy (Schwartz 2006), allows for a metaphor
COMPANIES ARE LIVING ORGANISMS to be inferred. Obviously, any verbali-
zation reflects the researcher’s interpretation of the observed expressions,
and will in turn steer the analysis of similar phrases. This bias is aggravated
in the case of multimodal metaphor, especially when it is the source domain
that is expressed in non-verbal form: For example, a company’s business
principles might feature pictures of a boat race (e.g. ABN Amro bank,
http://www.group.abnamro.com/about/business_principles.cfm) with no
possible literal interpretation.3 Should the inferred conceptual metaphor be
worded as COMPANY X IS A RACING TEAM, or, in more general terms, BUSI-
NESS IS A RACE? In the case of a relay race, the metaphor’s significance
changes again, foregrounding aspects of teamwork rather than competition.
The ABN Amro example is further complicated by the verbal co-text realiz-
ing the JOURNEY metaphor, referring to the business principles as a compass
to guide us on our journey. In view of this complexity, it could be argued
that the traditional A IS B type metaphor as such is limiting, and that inferred
metaphors should be seen as part of a metaphor scenario (Musolff 2006)
with different levels of abstraction as well as cause-effect relations. In our
example, this could mean that both COMPANY X IS A RUNNER and BUSINESS
IS A RACE are viable. In addition, the metaphoric scenario triggered by the
multimodal metaphor could include semi-metaphoric components like “com-
pany X delivers faster results and outruns the competition.” I will return to
this question when discussing the examples from corporate discourse below.
In a given discourse,4 only particular models will be relevant, i.e., one
form of structured knowledge of a particular notion or entity along with its
associations, evaluations and affective components will be preferred to oth-
ers. Subsequently, the metaphoric expressions deriving from metaphoric
models and their evaluations become typical of that discourse. Repeated
exposure to the same set of metaphoric expressions, extended and elaborated
as they may be in various modes, can be assumed to activate the metaphoric
mental model underlying them and thereby reinforce it. By the same token,
positive evaluation of particular notions would also be reinforced. Discursive
reinforcements of mental models, including evaluations, have been referred
to as “moralized activities,” i.e., “activities represented by means of abstract
terms that distil from them a quality that triggers reference to positive or
negative values” (van Leeuwen and Wodak 1999: 105). An example would
be the metaphoric reference to companies as “corporate citizens,” which has
Brand images 49
It has been noted that “inasmuch as … pictures are more easily recognized
transnationally than … languages, pictorial and multimodal metaphors allow
for greater cross-cultural access than verbal ones” (Forceville 2008). This
makes visual elements, including the visual components of multimodal meta-
phor, particularly attractive as a communicative strategy for multinational
corporations. Multimodal metaphor in particular can be used as a tool to
meet the persuasive function of most corporate communication, because it
requires the text’s recipient to construct a meaningful reading by processing
verbal and visual elements together. The necessary cognitive effort poten-
tially reinforces a particular conceptualization of the company in the reader’s
mind. Indeed, much research into visual metaphor to date has investigated
the phenomenon in contexts such as advertising, often with a view to proc-
esses of reception and interpretation (Phillips 2003; Phillips and McQuarrie
2004), or international business magazines (Koller 2005).
Supplementing those studies, this chapter looks at corporate branding as
the interface between marketing and strategic organizational management.
As the “outward oriented (i.e., projected) organizational identity” (Kapferer
50 Veronika Koller
the point where brands become metaphorical people interacting with stake-
holders.6 In a final twist, the BRANDS ARE PEOPLE metaphor implies that a
brand is supposed to incorporate the ideal characteristics the consumer
wishes to be seen as having. In an ultimately narcissistic relationship with
the BRANDS ARE PEOPLE metaphor, the brand’s characteristics are theorized
to be transferred to the consumer, who thus engages in “symbolic consump-
tion” (Harquail 2006: 174–175), using brands “as symbolic resources for
the construction and maintenance of identity” (Elliott and Wattanasuwan
1998: 132).
A central role in the projection of corporate brand image is played by ar-
tifacts, or multimodal “texts,” e.g., logos, architecture, office design, call
centre music etc. (Cappetta and Gioia 2006: 212). Such brand artifacts are
seen as the “vehicles for the transfer of meaning from the brand to the con-
sumer,” bringing with them “emotional and self-expressive benefits” for the
consumer (Schultz, Hatch, and Ciccolella 2006: 150). Forceville (2006: 388)
notes that “the personification of commodities is a very familiar marketing
strategy,” and it is certainly one that can be extended to (corporate) brand
artifacts. Metaphors as a reflection of allegedly shared organizational dis-
course (Cunliffe and Shotter 2006: 135) are used in a range of modes, most
notably visual and verbal, which, often in concert, construct the BRANDS
ARE PEOPLE metaphor and endow it with particular characteristics. Such
multimodal metaphors tend to take particular forms and to cluster in particu-
lar genres.
reality of the company” means that ideally, “if you analyse a company’s
design you can also trace its identity [and] personality” (Johansson and
Holm 2006: 140). Although not discussed in this chapter, illustrations, logos
and layout, and other low-sensuality artifacts (Pratt and Rafaeli 2006: 281)
should be understood as acting in concert with other corporate artifacts in
communicating brand concepts. Further, while the present analysis focuses
on multimodal metaphors in which verbal components anchor a metaphoric
interpretation of visual elements, discursive construction and cognitive con-
ceptualization of brands can equally be realized in different modes, e.g.,
sounds and music, and the study can therefore be taken as a starting point
for empirically testing its claims against further data involving different mo-
dalities.
In the data, multimodal metaphors with visual and verbal components fea-
ture in three ways: Firstly, illustrations on corporate websites support con-
ventional metaphors such as BRANDS ARE LIVING ORGANISMS. Secondly,
logos and logo elements tend to visualize particular characteristics ascribed
to the BRANDS ARE PEOPLE metaphor as a special case of the LIVING ORGAN-
ISMS metaphor. Such characteristics are often expressed using spatial con-
cepts as source domains (e.g., illustrating dynamism, balance, or openness).
Thirdly, layout can also visualize source domains of metaphoric expressions
that are anchored by verbal elements, e.g., when connecting design elements
in the corporate color are used in the brochure of a company that claims to
have a “networking personality.”
5.1 Illustration
(3) Since its founding in 1910, Hitachi has acted from a corporate philoso-
phy of contributing to society through technology. In the intervening
years, the world and society have changed greatly, but we have never lost
our pioneering spirit, based on the principles of harmony and sincerity
(my emphasis, VK).
By setting up a dichotomy of technology and harmony, the latter is concep-
tually linked to what one would normally expect as the opposite of technol-
ogy, namely nature. Not only does the size of the tree then suggest an age
equaling that of the company, but it may also be used to visualize the notion
of harmony, as linked to nature. In mission statements there is a clear, literal
link between harmony and nature, as evidenced by the following examples:
(4) We … will shoulder the responsibility to contribute to a sustainable soci-
ety in harmony with nature. (Asahi)
(5) Stay in harmony with nature; blend in with local societies; and put our
hearts into creating a more vibrant, richer culture. (NYK Line)
(6) To conduct fair and open business operations while acknowledging our
social responsibilities and aspiring for harmony with our global envi-
ronment. (Toppan)
If the picture of the tree is indeed meant to metaphorically represent natural
growth, with the verbal elements further cueing a link between growth and
harmony, then we are dealing with a typical case of partial mappings in
promotional genres, in which all negatively connoted knowledge about the
source – such as the disarray and imbalance brought about by growth – are
“muted” (Ungerer 2000). Understood as such, we would be confronted with
a multimodal metaphor in which the source domain (NATURAL GROWTH) is
represented visually, while the verbal co-text relates another concept (HAR-
MONY) to the source domain, linking both to the target domain of the brand.
The relation between visuals and language here makes it possible, if not
necessary, to construe a metaphor FINANCIAL SUCCESS IS NATURAL
GROWTH. In that sense, multimodal metaphors that are based on the inter-
play between verbal and visual elements are not unlike pictorial similes
(Forceville 1996). This “unforced” metaphorical interpretation may well
serve to make instances like these powerful precisely because the metaphor
is neither explicit nor “strongly signaled” (Forceville 1999).
56 Veronika Koller
(8) “We all know that these are highly competitive markets, but I firmly
believe that the stellar attributes which are in the DNA of the brands and
Brand images 57
(9) “[The corporate culture] really has gotten into the DNA,” says Ms Con-
sidine, who will stay on as executive chairman. “I don’t say that lightly –
I used to be a biochemist.” (Beales 2006)
In the last example, the metaphoric expression is given particular weight and
credibility by linking it to its source domain via the speaker’s professional
background. The genetic metaphor has not escaped the attention of brand
theorists, who have raised the question whether such an essentialist notion is
still viable given the fragmented consumer identities found in postmodern
markets (Csaba and Bengtsson 2006: 124–125). On the other hand, essen-
tialist brand identity may be reinforced in corporate discourse to provide
orientation and the idea of reliability – another key word in the mission
statement corpus – for consumers.
Illustrations on corporate websites suggest that brand values are essen-
tialized as stable and inherent in a brand that is metaphorically conceptual-
ized as a living organism. Corporate logos specify the metaphor to BRANDS
ARE PEOPLE, revealing what allegedly inalterable traits are ascribed to such
a metaphorical human being.
5.2 Logo
Figure 4a and 4b. Curved elements in logos. The HVB logo is reproduced with
permission. The Vanteon logo is owned by Vanteon and is used with
permission from Vanteon.
Be they for banks (HVB), sportswear companies (Nike),9 software consult-
ants (Vanteon) or even not-for-profit organizations such as the British Home
Office,10 curved logo elements (“swooshes”) seem ubiquitous, up to the point
of cliché (Lindsay 2000). Perceived as conveying “global reach [and] impact
… full spectrums” (Brannon Cashion, senior vice president at Addison
Whitney, quoted in Lindsay 2000: 204), curved logo elements visually repre-
sent one of the key words in the mission statement corpus, “global.” While
this design feature is not in itself metaphoric – representing as it does a styl-
ized version of the globe – other possible interpretation such as “drive” and
“impetus” are better candidates. Given that “dynamic,” “proactive,” and
“agile” are again key words in the mission statements corpus, curved logo
elements may be interpreted as contributing to a multimodal metaphor of
dynamic speed.
This reading hinges on whether the elements are understood as indexical
or symbolic. “Swooshes” could be seen as varieties of what Kennedy (1982)
has called “runes,” i.e., the action lines used in comics to indicate, among
other things, movement. In a later paper (Kennedy, Green, and Vervaeke
1993), such action lines are compared to “a trail, as if left by lights on the
moving body, like time-lapse photography” (p. 247). Understood as such,
action lines would be indexical in that they point to a moving object. In com-
posite logos, however, curved elements include the brand name instead of the
moving object, making the whole logo a multimodal metaphor that includes
the target domain in the verbal and the source domain in the visual mode.11
60 Veronika Koller
It has been noted that “the logo [has] a significant role in … triggering
the cognitive frame” that a company wishes to be perceived in (Baruch
2006: 182). In corporate discourse, particular frames will be prioritized over
others. Thus, characteristics such as “dynamic” and “active” are overrepre-
sented at the expense of “contemplative” and “receptive,” both verbally as
lexical items and visually as logo elements. Given the ubiquity not only of
certain logos but also of pervasively used logo elements such as the
“swoosh,” such “endlessly repeated and recycled images … arguably aspire
to conceptual status” (Forceville 2006: 390). The mental models of “activ-
ity” and “continuous movement” have achieved a currency that makes these
models and their linguistic and visual expressions almost defining notions of
the discourses and other practices of corporations. Due to the prestige and
influence in late capitalism of all things corporate, such models become posi-
tively connoted in other areas as well and are likely to be desired as identity
traits by many consumers. Hence, metaphorically constructing brands as
people is intended to appeal to stakeholders and incite them to interact with
the brand and co-produce its identity (Csaba and Bengtsson 2006: 124), in
the hope of acquiring some of its characteristics by doing so. In how far this
strategy can be successful in the face of widespread consumer indifference
and even cynicism is an open question.
5.3 Layout
We have seen how the Hitachi website (figure 1) uses visual elements to
represent the source domain of the BRANDS ARE LIVING ORGANISMS meta-
phor. Moreover, the brand name was accompanied by visual elements signi-
fying the brand’s main characteristics. This device points to a third area of
use of multimodal metaphor in corporate branding discourse, namely layout.
Not just nameable objects, such as buildings and trees, but also layout can
assist in construing a metaphor, even though this strategy may require a
visually literate audience in order to be successful.
To take but one example, in various samples of the brand communication
of banking group HSBC (brochures, advertisements, posters, websites; see
Koller 2007) red, one of its corporate colors, acts as a structuring device:
Red sub-headings, red frames and red bullet points all serve layout functions
in arranging pictorial elements. In one brochure, red lines connect para-
graphs about the various places where the company does business. The lines
also frame details of easily recognizable, even clichéd local symbols such as
a Chinese dragon or the Eiffel Tower. As such, the layout of the brochure is
Brand images 61
This leaves us with the question why corporate brands should be personified
in the first place. What are the specific affordances of BRANDS ARE PEOPLE
that make this metaphor appropriate for the persuasive genres of corporate
branding communication? As Lakoff and Turner (1989: 72) note with regard
62 Veronika Koller
when listing the attributes of their cars. This narcissistic relationship be-
tween the consumer and the brand is made possible by first ascertaining
social and psychological aspects of the target audience through market re-
search and subsequently constructing the brand “personality” as mirroring
that of the ideal consumer. This conceptualization is effected through mar-
keting, particularly branding messages and their typical features, such as
multimodal metaphor. Unsurprisingly, given the persuasive thrust of market-
ing, the practice is highly manipulative in that it addresses consumers’ self-
image and exploits potential lack of self-esteem. On a more positive note,
sociological research into brand communities (Muniz and O’Guinn 2001)
has made a case for brands fostering social cohesion among individuals iden-
tifying with a brand, rather than fragmenting societies into narcissistic indi-
viduals engaging in compensatory brand consumption. Given the nature of
logos as condensed branding messages, such symbols obviously play a cen-
tral role as textual resources for such brand communities (Muniz and
O’Guinn 2001: 423).
The double-edged nature of branding as a corporate discourse practice is
captured in its description “as a (global) ideoscape [that] provides the ideo-
logical basis for the establishment of new meaning systems, new practices
and new identity forms for the members of the consumer culture” (Aske-
gaard 2006: 98). Although vocal criticism of branding (Klein 2000) has
ironically become part of branding itself, companies still have a vital interest
in not having their brands demystified. One strategy to mystify brands is
naturalization: Verbally and visually, texts promoting corporate brands draw
on LIVING ORGANISM metaphors to recast social practices as natural kinds
and thus make them less vulnerable to criticism. Rothbart and Taylor’s
(1992) observation that social categories tend to be viewed in an essentialist
fashion as homogeneous and inalterable natural kinds is crucial in this con-
text. The question has been raised whether essentialist notions of brands are
suitable for postmodern consumer identities (Csaba and Bengtsson 2006:
130–131). However, given the persuasive thrust of branding communication,
it can equally be argued that brands’ illusions of stability and inalterability
not only respond to public mistrust but actually attract stakeholders who
lack orientation in highly fragmented societies, for better or worse.
Yet, with promotional texts, genre is key: The designing institution in
each case is a company whose ulterior motive in producing and distributing
texts is promoting itself as a brand and, ultimately, maximize profits. Audi-
ences can be expected to be aware of, and even cynical towards, the profit
64 Veronika Koller
Notes
1. I would like to thank the two editors as well as Rosario Caballero for their
helpful comments on a draft version of this chapter.
2. The embodiment approach to metaphor (Lakoff and Johnson 1999) sees
kinesthetics as the foundation of basic, “primary” metaphoric models that are
Brand images 65
acquired early in life, e.g. GOOD IS UP (Grady 1997). However, note the dif-
ference between kinesthetically founded conceptual metaphor as a cognitive
phenomenon and gestural metaphor as a surface-level expression of such a
metaphoric model.
3. The image of the race could be literal as an illustration of, say, a marathon
sponsored by the company in question. A case could be made for sponsoring
practices literalizing aspects of the brand personality the company constructs
for itself, e.g., that of fastness and speed.
4. Discourse is here defined as the total of texts produced, distributed and re-
ceived between members of a particular social field, e.g., the business com-
munity.
5. In branding and organizational theory, an internally communicated corporate
brand image is sometimes referred to as brand identity (Hatch and Schultz
2000: 22).
6. Indeed, the metaphor BRANDS ARE PEOPLE has become so entrenched in cor-
porate discourse that it has been used as a methodological tool. Thus, Davies
and Chun (2002) developed a Corporate Personality Scale to measure the dif-
ference between internal and external images of organizations.
7. http://www.total.com/identite/portail/en/index.htm, accessed 12 February
2009.
8. http://www.mazda.com/profile/vision/, accessed 12 February 2009.
9. http://www.nike.com/nikeos/p/nike/en_GB/, accessed 12 February 2009.
10. http://www.homeoffice.gov.uk/, accessed 20 August 2007. In its combination
of traditional (coat of arms) and modern (curve) elements, the Home Office
logo is a good example of the transition from time-honored heraldic symbols
to contemporary abstract logos (see Baruch 2006, Frick 1996, Mautner forth-
coming, Pimentel and Heckler 2003). The curved element can also be read as
reinforcing the word “home” over which it arches, with both drawing on no-
tions of safety and protection.
11. The metaphoric nature of such logos becomes even more obvious when the
curve is interpreted as a movement heavenwards.
12. In Kress and van Leeuwen’s theory of visual grammar, vectors “are formed
by depicted elements that form an oblique line,” typically a diagonal one that
indicates an actor and the goal acted upon. In the HSBC example, “connect-
ing lines without an indicator of directionality … mean something like ‘is
connected to,’ ‘is conjoined to,’ ‘is related to’” (Kress and van Leeuwen
2006: 59).
References
Askegaard, Søren
2006 Brands as a global ideoscape. In Brand Culture, Jonathan E.
Schroeder and Miriam Salzer-Mörling (eds.), 91–102. London/New
York: Routledge.
66 Veronika Koller
Text=DNA&y=0&javascriptEnabled=true&id=041012000957&x=5
Accessed 19 September 2006.
Grady, Joseph
1997 Foundations of Meaning: Primary Metaphors and Primary Scenes.
Ph.D. dissertation. University of California Berkeley.
Harquail, Celia V.
2006 Employees as animate artifacts: Employee branding by “wearing the
brand.” In Artifacts and Organizations: Beyond Mere Symbolism,
Anat Rafaeli and Michael G. Pratt (eds.), 161–180. Mahwah, NJ:
Lawrence Erlbaum.
Hatch, Mary Jo, and Majken Schultz
2000 Scaling the Tower of Babel: Relational differences between identity,
image, and culture in organizations. In The Expressive
Organization: Linking Identity, Reputation, and the Corporate
Brand, Majken Schultz, Mary Jo Hatch, and Mogens H. Larsen
(eds.), 11–35. Oxford: Oxford University Press.
Holt, Douglas B.
2002 Why do brands cause trouble? A dialectical theory of consumer
culture and branding. Journal of Consumer Research 29: 70–90.
Johansson, Ulla, and Lisbeth S. Holm
2006 Brand management and design management. In Brand Culture,
Jonathan E. Schroeder and Miriam Salzer-Mörling (eds.), 136–152.
London/New York: Routledge.
Kapferer, Jean-Noël
2002 Corporate brand and organizational identity. In Corporate and
Organizational Identities: Integrating Strategy, Marketing,
Communication and Organizational Perspectives, Bertrand
Moingeon and Guillaume Soenen (eds.), 175–193. London/New
York: Routledge.
Kennedy, John M.
1982 Metaphor in pictures. Perception 11: 589–605.
Kennedy, John M., Christopher D. Green, and John Vervaeke
1993 Metaphoric thought and devices in pictures. Metaphor and Symbolic
Activity 8: 243–255.
Killgren, Lucy, and Eoin Callan
2004 HMV names new chief. Financial Times 18 July. Available from
http://search.ft.com/searchArticle?page=4&queryText=DNA&y=0&
javascriptEnabled=true&id=06071Mautner8003693&x=5. Accessed
19 September 2006.
Klein, Naomi
2000 No Logo: Taking Aim at the Brand Bullies. New York: Picador.
Brand images 69
Koller, Veronika
2005 Designing cognition: Visual metaphor as a design feature in
business magazines. Information Design Journal and Document
Design 13: 136–150.
2007 “The world’s local bank”: Glocalization as a strategy in corporate
branding discourse. Social Semiotics 17: 111–130.
2008 “Our customers embrace us as an essential partner”: Corporate
brands as socio-cognitive representations. In Cognitive
Sociolinguistics: Language Variation, Cultural Models, Social
Systems, Gitte Kristiansen and René Dirven (eds.), 389–418. Berlin/
New York: Mouton de Gruyter.
Kress, Gunther, and Theo van Leeuwen
2006 Reading Images: The Grammar of Visual Design. 2nd ed.
London/New York: Routledge.
Kristiansen, Gitte
2008 Idealized cultural models: The group as a variable in the develop-
ment of cognitive schemata. In Body, Language and Mind. Volume
2: Sociocultural Situatedness (Cognitive Linguistics Research 35.2),
Roslyn M. Frank, René Dirven, Tom Ziemke, and Enrique
Bernárdez (eds.), 409–432. Berlin/New York: Mouton de Gruyter.
Kunda, Ziva
1999 Social Cognition: Making Sense of People. Cambridge, MA: MIT
Press.
Lakoff, George, and Mark Johnson
1999 Philosophy in the Flesh: The Embodied Mind and Its Challenge to
Western Thought. New York: Basic Books.
2003 Metaphors We Live By. 2nd ed. Chicago: University of Chicago
Press.
Lakoff, George, and Mark Turner
1989 More Than Cool Reason: A Field Guide to Poetic Metaphor.
Chicago: University of Chicago Press.
Lassen, Inger, Strunck, Jeanne, and Torben Vestergaard (eds.)
2006 Mediating Ideology in Text and Image. Amsterdam/Philadelphia:
Benjamins.
Laudenbach, Peter
2005 Wagen ohne Volk. [Volkswagen: a car without people] brand eins
8: 84–90.
Lindsay, Greg
2000 I want a unique logo – just like theirs. Fortune 24 July: 204–205.
Mautner, Gerlinde
forthc. The spread of corporate discourse to other social domains. In
Language and the Market, Helen Kelly-Holmes and Gerlinde
Mautner (eds.). Basingstoke: Palgrave Macmillan.
70 Veronika Koller
Messaris, Paul
1997 Visual Persuasion: The Role of Images in Advertising. London/
Thousand Oaks, CA: Sage.
Morgan, Gareth
1986 Images of Organization. London/Thousand Oaks, CA: Sage.
Muniz, Albert M., and Thomas C. O’Guinn
2001 Brand community. Journal of Consumer Research 27: 412–432.
Musolff, Andreas
2006 Metaphor scenarios in public discourse. Metaphor and Symbol 21:
23–38.
Phillips, Barbara J.
2003 Understanding visual metaphor in advertising. In Persuasive
Imagery: A Consumer Response Perspective, Linda M. Scott and
Rajeev Batra (eds.), 297–310. Mahwah, NJ: Lawrence Erlbaum.
Phillips, Barbara J., and Edward F. McQuarrie
2004 Beyond visual metaphor: A new typology of visual rhetoric in
advertising. Marketing Theory 4: 113–136.
Pimentel, Ronald W., and Susan E. Heckler
2003 Changes in logo design: Chasing the elusive butterfly curve. In
Persuasive Imagery: A Consumer Response Perspective, Linda M.
Scott and Rajeev Batra (eds.), 105–127. Mahwah, NJ: Lawrence
Erlbaum.
Pratt, Michael G., and Anat Rafaeli
2006 Artifacts and organizations: Understanding our “objective” reality.
In Artifacts and Organizations: Beyond Mere Symbolism, Anat
Rafaeli and Michael G. Pratt (eds.), 279–288. Mahwah, NJ:
Lawrence Erlbaum.
Rayson, Paul
2008 Wmatrix: A web-based corpus processing environment. Computing
Department, Lancaster University. Available from http://ucrel.lancs.
ac.uk/wmatrix/.
Rothbart, Myron, and Marjorie Taylor
1992 Category labels and social reality: Do we view social categories as
natural kinds? In Language, Interaction and Social Cognition, Gün
R. Semin and Klaus Fiedler (eds.), 11–36. London/Thousand Oaks,
CA: Sage.
Rumbo, Joseph
2002 Consumer resistance in a world of advertising clutter: The case of
ad busters. Psychology & Marketing 19 (2): 127–148.
Schultz, Majken, Hatch, Mary Jo, and Francesco Ciccolella
2006 Brand life in symbols and artifacts: The LEGO company. In
Artifacts and Organizations: Beyond Mere Symbolism, Anat Rafaeli
Brand images 71
Abstract
The present chapter surveys some of the metaphors recurrently used in wine pro-
motion. The starting point is the tasting note, a genre that basically deals with
translating a sensorial and highly subjective experience into comprehensible and,
above all, shareable terms. This endeavor implies the abundant use of figurative
language. If we take into account that tasting notes are the verbal translations of
organoleptic experiences (i.e., perceived by a sense organ), the issue here is
whether the metaphors used are monomodal or multimodal according to the defi-
nition of multimodal metaphor in the present volume.
Some of the metaphorical frames in winespeak are also exploited to promote
wine in commercials and print advertisements – particularly in the latter. Both
genres show that imagery may encompass modes other than language but, at the
same time, here the difficulties of communicating smell and taste are particularly
conspicuous. Indeed, should we have to choose one mode of expression for this
particular endeavor, wine advertising points to language as the best – albeit also
limited – tool at our disposal. In this regard, my second aim in this chapter is to
discuss the problems derived from attempting to “translate” verbal metaphors into
images in wine advertising – the metaphors in this case being unquestionably
multimodal – and the weight of culture in using and, above all, interpreting figu-
rative phenomena rendered via images.
Keywords: Winespeak, sensory perception, culture, tasting note, advertising gen-
res, cross-modal metaphor
1. Introduction
In the present discussion, major insights have been drawn from an ongo-
ing research project devoted to exploring metaphor in a corpus of 12,000
TNs and 100 print advertisements retrieved from American and British wine
publications. However, for this chapter I have also used a small corpus
comprising texts and adverts in Spanish and French.1 The main reason for
including these lies in the fact that adverts need not necessarily be translated
into the language of the specialized magazine inserting them (French being a
case in point given its former status as the language every oenologist and
wine lover should master). Moreover, although the metaphors and metony-
mies underlying winespeak largely cut across languages and cultures
(Nedilko 2006), this is not always the case when images are concerned, or
when images and words combine in wine adverts. In this regard, comparing
how wines from different countries are advertised may bring to the fore the
weight of culture in using and, above all, interpreting figurative phenomena
rendered via images. A clarification concerning my use of the term culture in
this chapter is in order at this point. Thus, culture covers both the shared
beliefs, knowledge and world view(s) characterizing national groups as well
as the more specific beliefs, knowledge and view(s) of the community articu-
lated around wine – in the latter case, culture cutting across national or re-
gional differences and foregrounding the importance of topic and shared
interests in building up discourse communities or cultures within cultures.
The chapter is organized as follows. It starts with a survey of the figura-
tive language used by wine critics in TNs. This is followed by a discussion
of the cross-modal or multimodal quality of such language and its possible
relationships with other figurative phenomena. Then I discuss the use of
imagery in advertising genres, pointing to their heavy dependence on the
verbal mode and the difficulties arising from this fact.
The tasting note is one of the most representative and popular genres in wine
discourse, as well as a key instrument in the process of wine acculturation.
TNs are short texts (from 20 to 200 words) devoted to describing and evalu-
ating wine, and are prototypically organized in three distinct sections that
capture the three canonical steps in any wine tasting procedure, namely, the
assessment of wines’ (a) color, (b) smell – metonymically referred to as the
wine’s nose/nariz/nez in English, Spanish, and French respectively, and (c)
mouth-feel – which subsumes smell, taste, and touch, and is metonymically
referred to as the wine’s palate/boca/bouche.
76 Rosario Caballero
(3) Los taninos [del vino] … son puro terciopelo. [The tannins are pure
velvet]
(4) Très belle robe rouge pourpre … équilibre dû aux tannins fins et soyeux.
[Very beautiful red purple color/robe … balance due to fine and silky
tannins]
(5) En bouche, il se déploie comme un tapis de velours sur lequel dansent les
arômes. [In the mouth, it unfurls like a tapestry on which aromas dance]
Cutting across the senses 77
(7) The fruit [in this wine] barely peeks through the wall of chewy tannins
on the finish.
(8) Bien armado, con cuerpo y taninos firmes … [Well assembled, with
body and firm tannins]
(9) La bouche a … des tanins matures, bien fondus mais qui donnent une
architecture bien définie. [The mouth has ripe, well integrated tannins
which provide a well defined architecture]
All in all, however, the most salient metaphorical frame in the three lan-
guages at issue is WINES ARE LIVING ORGANISMS. This is far from surpris-
ing since wine is a mutable entity resulting from an organic process: the juice
extracted from grapes changes considerably along its life inside both oak
casks and bottles – a process referred to as breeding or ageing (Sp. “cri-
anza,” Fr. “élevage”) – as well as during the very act of drinking it. Thus,
among the many terms used to describe wine’s evolutionary state we find
forceful (Sp. “vigoroso,” Fr. “vigoreux”), weak (Sp. “débil,” Fr. “faible”),
youthful (Sp. “joven” or “juvenil,” Fr. “jeune”), tired (Sp. “agotado,” Fr.
78 Rosario Caballero
(11) [This wine] has a nicely buried backbone of acidity and tannin. … ulti-
mate impression of both muscle and flesh.
(12) Like its older sibling, [this wine] will be delicious in its first 3–4 years of
life.
(13) Un gran tinto de la Ribera, con casta y finura. [A great red from Ribera,
with breed and finesse]
Cutting across the senses 79
(14) [Este vino] es una bestia encerrada de gran complejidad. … Nariz pode-
rosa muy compleja. En boca es musculoso, tiene cuerpo, estructura, po-
tencia … carnosidad … todavía es muy joven … Posee raza y fuerza. [a
caged beast of great complexity … powerful and very complex nose. In
mouth it is brawny, with body, structure, power, flesh … it is still very
young … has breed and strength]
(15) Nez expressif, déjà bien ouvert … Joli nez, assez complexe. … Attaque
franche et volumineuse. … Joli potentiel de vieillissement. [Expressive
nose, already well open … Beautiful nose, quite complex. Straightfor-
ward and voluminous … Beautiful potential for aging]
(16) Jambes colorées. Très beau nez … Très belle saveur fruitée. ... Charnu.
Du corps. ... Très beau vin. [Colored legs. Very beautiful nose … Very
beautiful fruity flavor … Fleshy. Full-bodied. Very beautiful wine]
ties. This does not mean that some of the metaphors cannot – through re-
peated use in textual interaction – be further extended and, thus, conform to
the metaphorical frames discussed earlier. Yet, regarding such frames as
conceptually rather than cross-sensually concerned would betray the nature
of the experience metaphorically portrayed and communicated.
Indeed, the complexity – and interest – of the figurative expressions dis-
cussed so far arises from the fact that their underlying mappings seem to
involve different modes. Thus, whereas many of the targets of the metaphors
are smell(s), taste(s) and mouth-feel (i.e., physical, perceptual experiences
via sense organs), the sources concern both concrete (three-dimensional arte-
facts or the human body) and abstract (kinship) entities and domains – an
idiosyncrasy worth investigating, yet which falls outside the general aims of
the present volume. Nevertheless, since both the source and target in the
figurative expressions are rendered verbally, hence in the same communica-
tive mode, these belong to the monomodal type of metaphor in Forceville’s
(2006) sense. Having said this, let us have a look at wine advertisements,
and assess whether these contain truly multimodal metaphors, that is, meta-
phors whose target and source are (primarily) rendered in different modes
(e.g., via images and language).
verbal metaphors in the accompanying body text or, what is worse, show the
“literal” referent of the entity denoted by a given figurative expression or
term irrespective of its relevance and/or meaning in the wine realm (e.g.,
adverts exploiting anthropomorphic metaphors, as discussed in section 3.1.).
This, in turn, suggests that understanding the pictorial metaphors in wine
adverts is no easy matter, and nearly always requires acquaintance with wine
discourse in general and, of course, with the figurative language used by the
community of wine lovers.
As already pointed out, the approach to multimodal metaphor followed in
this chapter is the one provided by Forceville (2005, 2006, 2007). Two im-
portant steps in Forceville’s scheme are (a) to establish the domains involved
in the metaphors – which, in turn, implies determining whether the meta-
phors favor certain source domains, and (b) to see which types of multimo-
dal metaphor are illustrated in the data under analysis. These questions are
addressed in the following sections.
Given the fierce competition in the sector and the overuse of certain proce-
dures to enlarge production, most wine adverts are particularly concerned
with explaining the winemaking process followed by the winery so that their
products achieve the sought-after sense of terroir (i.e., how the wine exhibits
the characteristics of the vineyard site). Unsurprisingly, the adverts thus
oriented favor the anthropomorphic frame outlined earlier, playing up the
kinship and physiological aspects of the wine at issue in order to highlight its
“pedigree” or “lineage.” When the metaphor is used for this purpose, it is
mostly instantiated through language. A nice example of this is a Spanish
commercial designed for Navarra wines (one of the Spanish guarantees of
origin and quality of wines), where the audience is shown a middle-aged man
who appears to be talking to his departing son in the following terms:
You’re leaving tomorrow. The world awaits you and I’m sure that when
they know you they’ll love you. We’re proud of you at home. We know that
you’ll become one among the great. Some won’t appreciate your worth:
pay no attention. Mmm … Others will see a winner in you: don’t boast.
Don’t be afraid of time: it will make you mature and, when you grow old,
do it with dignity. Always remember the land where you were born. Son,
I’ve given you everything I know. I’ll miss you. OK, enough. You have a
long trip tomorrow. NAVARRA REDS, SONS OF OUR LAND. [Trans-
lated from Spanish, RC] Bassat. Reproduced with permission.
Cutting across the senses 83
Only after the last sentence we are shown the “son” thus addressed: a bottle
of wine resting on a table. The advert is true to its purpose, namely, advertis-
ing a wine-producing region rather than a particularly winery. Accordingly,
the bottle bears no label and the camera is solely concerned with the father,
who metaphorically stands for the region. Of course, the language used is
unmistakably figurative, yet we are aware of this only after we are shown
the bottle-son at the very end of the commercial – this “delay” being a com-
mon advertising strategy that seeks to surprise audiences. The commercial is
a true example of multimodal metaphor in that the target (Rioja wines) is
rendered visually and the source (“son”) is rendered verbally.
“Being the latest genera-
tion in a line of truly
great wines means that
you have evolved from
very solid roots. You
have also grown up shel-
tered by the knowledge
of one family. You are
loyal to one name only
… Muga, whose tradi-
tion and character allow
you to grow even further
and develop new forms
of art, hence Torre
Muga. Torre Muga
comes from only the best
vintages and is pampered
throughout its traditional
winemaking process.
[…]” [translation from
Spanish by RC].
Figure 1. Print ad for the Spanish Torre Muga winery. WINES ARE PEOPLE
Personification is also exploited in the following three examples, all of which
promote specific brands and attempt to render a human view of wines via
language and images. Figure 1 shows several products of a well-known
Spanish winery, yet is particularly concerned with promoting the latest
product – as explicitly acknowledged throughout the text – namely, Torre
Muga. Following the aforementioned prototypical technique, the excellence
84 Rosario Caballero
literal translation is “We show you our latest nose,” the term “nez” may also
be understood as “born” should we read the slogan aloud (nez and né being
homophones in French).
Figure 4. Advertisement for Gran Feudo Chivite wine: SWIRLING WINE IS SWIRLING
BALLET DANCER.
Cutting across the senses 87
In Figure 4 we also find language and image side by side; yet, in this par-
ticular case, the image does not need the help of language to be understood.
Thus, the advertisers have used well-known ballet dancer Tamara Rojo to
visually convey the swirling of a wine within a glass – i.e., the necessary
first step to release its aromas and assess its nose. Moreover, swirling also
helps tasters grasp the first impression of wines’ texture (remember the
aforementioned use of legs or tears in this regard) before this is finally as-
sessed inside their mouths. In other words, the dancer’s swirling is used to
represent both the wine’s swirling and its texture. Given the characteristics
of the dancer’s dress (which, as rosés, is pink), this texture appears to be
assessed as fresh, light, smooth and soft or supple. All such traits are char-
acteristic of rosé wines like the one here advertised, draw upon tactile ex-
periences typically felt outside rather than inside the mouth (i.e., through
skin contact) and, therefore, may be seen as cases of synesthesia particular
to the wine domain, and usually co-occur with one of the most conventional
figurative terms used in texture assessment: silky. Of course, the image may
be interpreted as also illustrating the metaphor A WINE IS A BALLERINA, yet
all in all it is the releasing of aromas and the wine’s texture that are at stake
in the advert. As it is, equating a wine with a ballerina may, indeed, reinforce
the aforementioned traits of the wine at issue. In sum, the advert not only is
congruent with some of the figurative terms used by expert tasters in TNs,
but, according to an informal discussion held with some wine connoisseurs,
also appears to successfully convey them in visual form.
Together with illustrating some of the problems derived from attempting
to represent the metaphors in winespeak in visual form, the adverts shown so
far may be used to discuss the pertinence of culture in metaphor interpreta-
tion – whether metaphor is represented verbally, visually, or both.
images are good exponents of the weight of culture in both producing and
interpreting messages – whether this is done through metaphors or otherwise.
Thus, the same image may mean totally different things to people immersed
in Western culture, a national culture within this broader frame, or a more
specific community built upon shared – professional or ludic – interests and,
more often than not, cutting across national differences.
This is also postulated by Forceville, who acknowledges that “the old ad-
age that a picture tells more than a thousand words should not blind us to the
fact that pictures and other multimodal representations seldom communicate
automatically or self-evidently. As in verbal metaphors, it is connotations
rather than denotations of source domains that get mapped in metaphors, and
these may substantially differ from one (sub)cultural group to another”
(2006: 389). Although I fully agree with Forceville’s views, I think that
knowledge of the target domain and, above all, of the conventions, beliefs,
needs, etc. of the community whose metaphorization of the world is scruti-
nized should not be underestimated either. Before taking this point further,
let us consider two other wine adverts.
“easy,” and “comfortable” are used to qualify both jeans and wines; on the
other, the verb “try on” reinforces the wines-jeans equation. As to the prop-
erties involved in the comparison, “soft” and “comfortable” point to texture
(i.e., tactile sensations), whereas “easy” suggests the casual, everyday, un-
complicated quality of both jeans and the wines thus qualified – which en-
compasses more things than just sensory experience like, for instance, that
both can be worn/drunk with nearly everything and at any occasion. Indeed,
part of the success of this advert lies in the fact that the aforementioned ad-
jectives are frequently used in wine commentary focused on wine’s texture.
However uncontroversial the advert may seem at first sight, for people in
their middle age it may seem odd to qualify jeans as “soft” unless they are
really old and worn out. Of course, soft – albeit new – jeans are the result of
modern cloth manufacturing procedures such as stonewashing (which may
well be seen as a pun on the winery's name, “Ironstone”) as well as current
jeans fashion. However, for people in their forties, the connotations of jeans
– whether visually or verbally cued – cover anything but softness. Age and
fashion considerations apart, the advert could be interpreted as highlighting
the casual, young, uncomplicated, everyday qualities of the wines at issue –
a view that fits the North American lifestyle also promoted by other con-
sumer goods such as cosmetics or clothes. Moreover, although softness is
usually a result of ageing, this advert tells us explicitly that Ironstone wines
are ready to be consumed – i.e., do not need to be kept inside the bottle for a
while to acquire such a property.
However desirable – and sought-after – such qualities may be in modern
winemaking and in certain wine markets, the question remains as to whether
wine lovers from other countries would find the wines thus advertised worth
buying. Indeed, one of the assumptions in countries such as France or Spain
as well as the main feature of some famous vintages is, precisely, that great
wines stand out among others for their sense of terroir (which rules out the
connotations derived from adjectives such as “easy” and “comfortable”) or
the winemaking and ageing process used (which usually takes time). More-
over, softness may be a desirable quality in a wine (meaning that it is round,
fruity and lacks aggressive tannins) or, rather, be regarded as negative in the
sense that it indicates that the wine at issue provides little impact on the pal-
ate – i.e., is somewhat watery.
The advert of a Spanish wine in figure 6 offers a dramatically different
picture, the slogan reading “Osborne arrives at Malpica.” At first sight,
there is nothing figurative here: what we have is a close up of a grapevine
and a slogan that tells us that a well-known Spanish brand (Osborne) has
been recently established in Malpica (a location in Spain). Yet, the advert
90 Rosario Caballero
also includes the brand’s trademark, a bull, which was originally created to
advertise one of the most popular Spanish brandies (Veterano) in billboards,
and has become a distinctive landmark of the Spanish landscape – as well as
a national identity symbol if we heed the numerous reproductions of Os-
borne’s bull in the T-shirts, tea mugs, etc. sold to tourists. Finally, the sur-
name Osborne in Spain is loaded with connotations that bring to mind wines
as well as bull breeding – which may be missed by those people not familiar
with Spanish culture.
In my view, the advert nicely illustrates a pictorial metonymy, that is, an
image that provides access to the entity “bull” via one of its most prototypi-
cal traits: its head and, more specifically, its horns. Moreover, since I am
acquainted with Spanish TNs, I could not help but interpret the metonymy as
cueing one of the best compliments a wine may be paid, namely, that of be-
ing thoroughbred. For whereas this English term is associated with horses or
dogs, its Spanish equivalent casta is prototypically used to qualify bulls.
This characteristic shows both in the animal’s behavior and in its appear-
ance, and horns are largely responsible for the latter. My specific claim here
is that any Spaniard could easily interpret the advert as equating wines with
bulls regardless of the presence of the trademark or the connotations verbally
cued by the term Osborne, yet would be seen as merely showing a grapevine
by many audiences outside Spain. A different issue is whether the trait
metonymically mapped (lineage, breed) would be understood by Spanish
people outside the wine realm.
Several – related – questions are relevant concerning this last point, and
are applicable to any advert. The first one is: would non-expert audiences
understand what is communicated in wine adverts at all? Of course, any
reader acquainted with advertising discourse will know that something posi-
tive is being promoted via language, images, or both. S/he will presumably
make use of her/his background knowledge to interpret the adverts – knowl-
edge which may include different kinds of information and, most impor-
tantly, will be culturally biased.
In turn, bringing in cultural knowledge in advert and/or metaphor under-
standing also prompts two more questions: What do we mean when we use
the term culture? How do the different ways in which the term may be ap-
plied affect the understanding of metaphors in adverts? For if culture is un-
derstood as the shared beliefs, knowledge and world view(s) characterizing
national groups, the adverts may of course be understood, yet the interpreta-
tion of the images and, above all, of the metaphors thus rendered may very
well be more open given the heterogeneous nature of the audience (due to
their diverse backgrounds, and hence diverse concerns and expectations). In
Cutting across the senses 91
4. Concluding remarks
The discussion in this chapter has been mainly concerned with the experien-
tial dimension of metaphor, that is, with how it is used by wine critics and
advertisers to articulate the sensory experience(s) afforded by wines in dif-
ferent media and communicative modes. Thus, I have provided a brief over-
view of the metaphors found in two promotional genres within wine dis-
course: TNs and wine adverts. Concerning the former genre, I have
explained wine critics’ abundant use of figurative language as responding to
their need to overcome the difficulties inherent in communicating the various
organoleptic experiences conflating in wine tasting. Concerning the latter
genre, the adverts here shown illustrate a penchant for anthropomorphic
metaphors, and largely fall into Forceville’s verbo-pictorial variety of mul-
timodal metaphor. Nevertheless, the foregoing discussion has also revealed
the problems derived from representing anthropomorphic metaphors in both
verbal and visual form, particularly those concerned with the sensory prop-
erties of wines (the exploitation of the term body being a case in point of the
contradictory, even wrong views often rendered by the image-language com-
bination). In this respect, however attractive the image-language combina-
tion may be to sell the product, it also poses many problems for the meta-
92 Rosario Caballero
phor researcher since it not always captures the previously discussed cross-
sensory or multimodal dimension of most metaphors in winespeak. A point
also worth noting is the fact that many of the metaphors rendered in pictorial
form are actually the “translation” of metaphors “acquired” or learnt – con-
sciously or unconsciously – via such a cultural manifestation as language
rather than being embodied in the cognitive sense of the term.
A final aim of the chapter has been to draw attention to the role of culture
in understanding the metaphors used to advertise wine: both the broad na-
tional culture of the audience at which the adverts are aimed as well as the
specific culture articulated around wine. For, indeed, as Forceville (2006:
389) acknowledges, the connotations from the source domains in the meta-
phors are important in their interpretation, yet these also leave more room
for interpreting them. In my view, this endeavor also requires knowledge of
the target domain (in the present case, wine) as well as of the schemas under-
lying the particular worldview of the wine community.
Acknowledgments
Notes
References
Amoraritei, Loredana
2002 La métaphore en Oenologie. Metaphorik.de 3: 1–12. Available from
www.metaphorik.de/03/amoraritei.htm (last accessed 23 March
2003).
Bruce, Nigel
2000 Classification and hierarchy in the discourse of wine: Emile
Peynaud’s The Taste of Wine. English for Special Purposes Journal
23–26: 149–164.
Caballero, Rosario
2006 Re-Viewing Space: Figurative Language in Architects’ Assessment
of Built Space. Berlin/New York: Mouton de Gruyter.
Caballero, Rosario, and Ernesto Suarez-Toste
forthc. A genre approach to imagery in winespeak: Issues and prospects. In
Researching and Applying Metaphor in the Real World, Alice
94 Rosario Caballero
Abstract
on gestures). The interaction of metonymy and metaphor shows that they are
not two opposite poles, but two parts of a continuum from literalness to
metaphor, as Radden (2002: 409) suggests.
Metonymy is understood here as an internal mapping of a subdomain
within the same experiential domain (Lakoff and Johnson 1980, Radden
2002; for the topic of multimodal metonymy per se, see Yu this volume).
While a metaphorical mapping bridges the distance between entities that are
experienced as belonging to two different domains, in metonymy a mapping
is connected to the mental highlighting or activation of one (sub)domain over
another (Barcelona 2002, Croft 1993). The target and source domains in a
metaphor establish symmetrical correspondences between different concepts
in a way that does not happen in metonymy. For instance, in the metaphor of
LOVE IS A JOURNEY the concepts in the target (lovers, love relationship, etc)
correspond to the concepts in the source (travelers, vehicle, etc) as Lakoff
(1993: 208) has shown. As Barcelona (2002) says, in metonymy this corre-
spondence is asymmetrical: “the metonymic source projects its conceptual
structure onto that of a target, not by means of a systematic matching of
counterparts, but by conceptually foregrounding the source and by back-
grounding the target (cf. Barcelona 2002: 226, italics in original). For in-
stance, in case 4, the front of the car stands for the whole car or even the
whole car company. The foregrounding of the source (the car front: bumper,
spoiler with company logo, head and turning lights) highlights features of the
car which the advertiser intends to underscore. In contrast with metaphor,
which can be either referential or predicative (e.g., Warren 2006), metonymy
has been considered to have mostly a referential function (Lakoff and Turner
1989: 103). Other functions can be “meaning extension” (cf. Taylor 2002:
325) or pragmatic inferencing (cf. Panther and Thornburg 2003).
The distinctions between metaphor and metonymy are fuzzy. An instance
of how metonymy and metaphorical mappings can overlap is found in a
discussion in Forceville (1996). In a printed advert a beer bottle is pictured
in a wine cooler, thus expressing the metaphor BEER IS CHAMPAGNE. This
metaphor is developed from the metonymy which connects both target and
source to a single domain: [alcoholic] drinks.2 Research on metonymy-
metaphor interaction has led to different typologies. Goossens (1990) was
the first to analyze their interaction in linguistic action expressions and cre-
ated the term “metaphtonymy.” This term included four types of combina-
tion in cases of meaning extension: “metaphor from metonymy,” “metonymy
within metaphor,” “demetonymisation inside a metaphor” and “metaphor
within metonymy.”
Interaction of multimodal metaphor and metonymy 99
3. Case studies
2 3
4 5
6 7
Ahora más
versiones
Now more
versions
8 9
…Renault 11
10 11
Figure 1. Fasa Renault 1986 01 Salto Renault 11 “Salto” ‘Jump.’
102 Eduardo Urios-Aparisi
both domains is the power of the car’s engine (and metonymically the whole
car), and the power of the athlete who can jump very far. Just before the
jump, the car appears completely and the male voice-over says: “Now more
versions…” The verbal mode explicates the target of the commercial: the
new properties and more powerful motor of a car model.
The attributes by which both domains are cued constrain the metaphori-
cal mappings by highlighting those features which are relevant for the target
domain and the intention of the advertiser. Although both source and target
of the metaphor are identified by the montage of images of the athlete and
the car, the voice-over further anchors the commercial’s message by clarify-
ing the target domain of the metaphor.
The montage of the images of the athlete and the car identify both target
and source of the metaphor, and verbal anchorage further clarifies the target
of the metaphor. The potential mappings in the metaphor CAR IS PERSON are
limited to the ones the advertiser is interested in activating by creating meto-
nymical visual correspondences between the domains and by the verbal mo-
dality naming those conceptual features.
Submode 2: Soft
colors and low
modality.
SOFT COLOR IS
WARMTH
The visual mode could be divided into several submodes such as color or
movement. As mentioned above, the submodes are building blocks of each
mode (cf. Stöckl 2004: 14). The movement of the tea tag acts out the way a
104 Eduardo Urios-Aparisi
hypnotizer’s watch moves. The soft colors in a gold and brown hue and the
slightly out of focus image represent the view of the person who is falling
under the spell of the watch. The hue and colors are stereotypically warm
and thus represent the feeling of wellness that the product is supposed to
give and associate to the metaphor SOFT COLOR IS WARMTH and, thus, to the
metaphor AFFECTION IS WARMTH which can be found in the context of emo-
tional relationships (cf. Kövecses 2000: 93) and, if that is the case, the color
submode would integrate experiences of friendship and emotion with the
product. The visual mode also activates the sound mode with the speaker’s
hushed tonality which is similar to the color and hues of the image. The
modes and their association with different components in the metonymy and
metaphor in this commercial are listed in table 2.
In conclusion, the metaphor is elaborated through a complex interaction
of modal techniques. Each mode and submode associates the product with
the main metaphor TEA TAG IS A HYPNOTIZER’S WATCH and relates it to
sensations of warmth, relaxation and wellness.
This commercial’s point is not to advertise any new model, but it is a public
announcement and praise for the safety fixtures and reliability of all the cars
of that brand. Therefore, it is a kind of corporate advertising with a public
service tone, most likely aired around a time of high traffic and traveling, as
the voice-over clearly implies: “En estos días mucha gente saldrá a la carret-
era…” (“These days a lot of people will go on the highway…”). The com-
mercial’s design is very simple: the camera focuses on the front grill of a
Volvo car from the 240 series which was developed in the 1970s.3
of the car is a complex image. In it we can see the frame of the car with its
bulky design, the big bumpers and lights, the license plate with the word
Volvo in blue capitals with a white background (the official typeface of the
brand; Egyptian according to http://www.volvoclub.org.uk /history/volvo_
logo.shtml) and the logo which is located on top of the radiator. These two
elements are symbolic metonymies of the product and they have metonymi-
cal (LOGO FOR PRODUCT) as well as other meanings (on logos as multimodal
metaphors, see Koller this volume).4 The image represents the identity of the
corporation in three aspects: the product, the corporate signs and symbols,
and the main features they want to associate themselves with: reliability,
strength and safety.
Table 3. Metonymies and metaphor in Case 3
Figure Components Visual Words Sound
Metonymy 1 TARGET Car
PART FOR SOURCE Front of the car
WHOLE
Front of the car
with logo and
official typeface
Metonymy 2 TARGET Car’s reliability
PART FOR and safety
WHOLE SOURCE Front of the car Changing
Changing weather
weather condi- conditions
tions and sounds
associated
to moving
car
Metaphor TARGET Car
SOURCE Person:
Advice to a
driver by the
voice-over
Metonymy 3 TARGET Company
PART FOR SOURCE “Intelligent” car,
WHOLE Logo and official
typeface
106 Eduardo Urios-Aparisi
After the first image of the car, the camera pulls back and the “front of the
car” moves through all kinds of weather and driving conditions smoothly and
surely, and this driving is accompanied by metonymical sounds and images
which can be associated with these driving conditions and to passengers and
other persons: the sound of children getting in the car, the door closing, the
noise of the car engine, the rain, thunder and snow (also visible in the im-
ages), the turn signal noise and image, the noise and the image of a ball sud-
denly bouncing in front of the car, the car horn and brakes.
On the one hand, the images are metonymies of the car as a whole, per-
haps referentially highlighting the car’s strength, sturdiness and immutability
towards the changes in the road conditions. Also, as mentioned above, the
color submodality would also contribute to these meanings. On the other
hand, the language used does not refer to the product at all. The male voice-
over addresses the consumer directly by giving advice about how to drive
when there is high traffic during vacation time (the images suggest that it is
winter). The words insist that the driver be sensitive, drive safely and pru-
dently, and focus on the importance of the family (referenced metonymically
as “carga” or “load”) and on arriving safely (which implies that it is better to
arrive safely than fast).
As we can observe, this commercial features good examples of meto-
nymic references in various modes. Metonymy, in this case, is mostly refer-
ential with respect to the car, its passengers, and the weather and driving
conditions, but this metonymy can create further implicit meanings by the
audience’s knowledge of car-safety and brands. Besides these components,
there is one obvious absence in the visual representations: that of the driver
who is directly addressed in the words. Whereas the commercial features
various metonymies, the voice-over’s address predicates of the car some of
the properties which generally are associated with a person, the driver of the
car: sensitivity and prudence. Therefore, as part of the creation of the corpo-
rate image, the car seems to be personified (CAR IS PERSON). A personifica-
tion is a kind of ontological metaphor in which the target is understood in
human terms (Lakoff and Johnson 1980). It can be explained along the lines
of Lakoff and Turner’s (1989: 195) discussion of metaphors like PERSONS
ARE ANIMALS (Achilles is a lion) within the cultural model of the GREAT
CHAIN METAPHOR. According to this metaphor, attributes and behaviors are
associated with animate creatures within a hierarchical scale: “the Great
Chain is a scale of forms of being – human, animal, plant, inanimate object
– and consequently a scale of the properties that characterize forms of being
– reason, instinctual behavior, biological function, physical attributes, and so
on” (Lakoff and Turner 1989: 167). In this commercial, we understand the
Interaction of multimodal metaphor and metonymy 107
perspective on the bootie changes and the camera shows a close-up of the
thread of the bootie slowing down. The source has been identified by the
visual mode while the target is in the verbal mode throughout the commercial
until the final sentence: “no vivas pendiente de un hilo” (“don’t live hanging
by a thread”) which is made literal in the image by showing the last thread of
the bootie. The final expression is a conventional metaphor in Spanish and
returns to some extent to the first metaphor: MAKING A DECISION IS PULLING
A THREAD. The commercial leaves the opportunity to reach other conclu-
sions to the audience.
10. Planifícate
Make plans
Figure 4. “Patuco” (“baby bootie”) (1988) Ministerio de Sanidad (Health Depart-
ment, Spanish Government). Advertising company: Vitruvio.
Interaction of multimodal metaphor and metonymy 109
In table 4, I have summarized the metonymy and the metaphors of this com-
mercial. It is an emotionally charged commercial. The bootie, out of all the
possible pieces of baby clothing, is readily associated with a baby (older
children do not wear booties); also, it shows with more immediacy the shape
of the body, and, finally, it has immediate connections to life and movement:
crawling and walking. The metonymy motivates the metaphor, but also maps
other meanings to the target of the metonymy (the baby). This process to-
gether with the music (the so called Brahms’ Lullaby) creates another emo-
tional layer to the commercial by reliving the decision making process in the
enactment of undoing a bootie.
Table 4. Metonymy and metaphor in case 4.
SOURCE UNDOING A
BOOTIE
SOURCE UNKITTING A
BABY-BOOTIE
4. Conclusions
the colors can be associated not only with the difficult driving conditions but
also with the strength and endurance of iron or steel. While in case 4, the
choice of yellow in the color of the baby bootie intentionally avoids colors
like blue or pink, which are gender-specific. The diffused light of the back-
ground highlights the baby bootie and its dramatic undoing. In the case of
metaphor, they include meanings that aim at supporting the visual consis-
tency and the identification with those features which are stereotypical of the
domain.
In brief, I would conclude:
1. Television commercials are dynamic texts in which all modes can con-
tribute to multimodal metaphors either in the source domain or the target
domain.
2. In order to understand how metaphor creates meaning, metaphor needs
to be studied within its embeddedness in the context of the commercial and
the persuasive functions of advertising.
3. The interactions of metaphor and metonymy show that layering of rhe-
torical figures is not random, but follows clear cognitive patterns which re-
strict and define their design and persuasion. As seen in case 2, the meton-
ymy has a double function: represent the target for the metaphor in a way
that can be realistic for the metaphorical representation, and motivate the
message of the commercial. Once metonymical correspondences are mapped,
the commercial can create additional metaphorical mappings. Also, meton-
ymy can identify those entities which are to be transferred from the two do-
mains, as in case 1.
4. As shown, a metaphor expands the meaning by associating new do-
mains with the original metaphor or metonymy. It creates further imagery
that can trigger more emotional or intellectual associations with the product.
5. The grounding of the meaning of a commercial in the viewer’s knowl-
edge and experience can be accomplished by various means. One of them is
metonymy, and this is consonant with general views on this figure. Meton-
ymy is considered closer to literalness in the literalness-metaphor continuum
(Radden 2002 and Dirven 2002), and is frequently used in realist art (Jakob-
son 1971 [1956]). In narrative it can highlight conventional belief, structure
episode development and, thus, help interpretation (cf. Pankhurst 1997). In
the commercials, the metonymy activates or highlights an aspect of the real-
ity of the product. This feature can be recognized by the audience most eas-
ily or can be productive to provoke implicit positive meanings. As shown, it
also can constrain the amount of possible correspondences in a metaphorical
mapping.
112 Eduardo Urios-Aparisi
Acknowledgments
I would like to thank Charles Forceville, Brian Patrick, John Bardem and Cristin
Siebert for their insightful comments on, and thorough revisions of, earlier ver-
sions of this chapter.
Interaction of multimodal metaphor and metonymy 113
Notes
References
Barcelona, Antonio
2000 Introduction: The cognitive theory of metaphor and metonymy. In
Metaphor and Metonymy at the Crossroads: A Cognitive Perspec-
tive, Antonio Barcelona (ed.), 1–28. Berlin/New York: Mouton de
Gruyter.
2002 Clarifying and applying the notions of metaphor and metonymy-
within cognitive linguistics: An update. In Metaphor and Metonymy
in Comparison and Contrast, René Dirven and Ralf Pörings (eds.),
202–277. Berlin/New York: Mouton de Gruyter.
Barthes, Roland
1988 [1964] The Semiotic Challenge. New York: Hill and Wang.
Block, Bruce
2008 The Visual Story. 2nd edition. Amsterdam, etc.: Focal Press.
114 Eduardo Urios-Aparisi
Caballero, Rosario
2006 Re-viewing Space: Figurative Language in Architects’ Assessment
of Built Space. Berlin/New York: Mouton de Gruyter.
this vol. Cutting across the senses: Imagery in winespeak and audiovisual
promotion.
Cameron, Lynne
2003 Metaphors in Educational Discourse. London/New York: Contin-
uum.
Cook, Guy
2001 The Discourse of Advertising. London/New York: Routledge.
Croft, William
1993 The role of domains in the interpretation of metaphors and me-
tonymies. Cognitive Linguistics 4: 335–370.
Dirven, René
2002 Metonymy and metaphor: Different mental strategies of conceptu-
alization. In Metaphor and Metonymy in Comparison and Contrast,
René Dirven and Ralph Pörings (eds), 75–112 Berlin/New York:
Mouton de Gruyter.
Dirven, René, and Ralph Pörings (eds.)
2002 Metaphor and Metonymy in Comparison and Contrast. Berlin/New
York: Mouton de Gruyter.
Dirven, René, Frank Polzenhagen, and Hans-Georg Wolf
2007 Cognitive Linguistics, Ideology and Critical Discourse Analysis. In
The Oxford Handbook of Cognitive Linguistics, Dirk Geeraerts and
Hubert Cuyckens (eds.), 1222–1240. Oxford: Oxford University
Press.
El Refaie, Elisabeth
this vol. Metaphor in political cartoons: Exploring audience responses.
Forceville, Charles
1996 Pictorial Metaphor in Advertising. London/New York: Routledge.
2002 The identification of target and source in pictorial metaphors. Jour-
nal of Pragmatics 34: 1–14.
2006/this vol. Non-verbal and multimodal metaphor in a cognitivist frame-
work: Agendas for research. In Current Applications and Future
Perspectives, Gitte Kristiansen, Michel Achard, René Dirven, and
Francisco Ruiz de Mendoza (eds.), 379–402. Berlin/New York:
Mouton de Gruyter.
2007 Multimodal metaphor in ten Dutch TV commercials. Public Journal
of Semiotics 1: 19–51, available at http://semiotics.ca/.
2008 Pictorial and multimodal metaphor in commercials. In Go Figure!
New Directions in Advertising Rhetoric, Edward F. McQuarrie and
Barbara J. Phillips (eds.), 272–310. Armonk, NY: ME Sharpe.
this vol. The role of non-verbal sound and music in multimodal metaphor.
Interaction of multimodal metaphor and metonymy 115
Goossens, Louis
1990 Metaphtonymy: The interaction of metaphor and metonymy in figu-
rative expressions for linguistic action. In By Word of Mouth: Meta-
phor, Metonymy and Linguistic Action in a Cognitive Perspective,
Louis Goosens, Paul Pauwels, Brygida Rudzka-Ostyn, Anne-Marie
Simon-Vandenbergen, and Johan Vanparys (eds.), 159–174. Am-
sterdam/Philadelphia: Benjamins.
Jakobson, Roman
1971 [1956] Two aspects of language and two types of aphasic disturbances, in
Selected Writings II: Word and Language. The Hague: Mouton.
Koller, Veronika
this vol. Brand images: Multimodal metaphor in corporate branding mes-
sages.
Kövecses, Zoltan
2000 Metaphor and Emotion: Language, Culture and Body in Human
Feeling. Cambridge: Cambridge University Press.
2005 Metaphor in Culture: Universality and Variation. Cambridge:
Cambridge University Press.
Kövecses, Zoltan, and Günter Radden
1998 Metonymy: Developing a cognitive linguistic view. Cognitive Lin-
guistics 9: 37–77.
Kress, Gunther, and Theo van Leeuwen
1996 Reading Images: The Grammar of Visual Design. London/New
York: Routledge.
Lakoff, George
1987 Women, Fire, and Dangerous Things: What Categories Reveal
about the Mind. Chicago: University of Chicago Press.
Lakoff, George, and Mark Johnson
1980 Metaphors We Live By. Chicago: University of Chicago Press.
Lakoff, George, and Mark Turner
1989 More than Cool Reason: A Field Guide to Poetic Metaphor. Chi-
cago: University of Chicago Press.
Messaris, Paul
1997 Visual Persuasion: The Role of Images in Advertising. Thousand
Oaks, CA/London/New Delhi: SAGE.
McQuarrie, Edward F., and Barbara J. Phillips
2005 Indirect persuasion in advertising: How consumers process meta-
phors in pictures and words. Journal of Advertising 34: 7–21.
Mittelberg, Irene, and Linda Waugh
this vol. Metonymy first, metaphor second: A cognitive-semiotic approach to
multimodal figures of thought in co-speech gesture
116 Eduardo Urios-Aparisi
Pankhurst, Anne
1997 Interpreting unknown worlds: Functions of metonymic conceptuali-
zation in William Golding’s The Sea Trilogy. Language and Litera-
ture 6: 121–131.
Panther, Klaus-Uwe, and Günther Radden
1999 Metonymy in Language and Thought. Amsterdam/Philadelphia:
Benjamins.
Panther, Klaus-Uwe, and Linda L. Thornburg, (eds.)
2003 Metonymy and Pragmatic Inferencing. Amsterdam/Philadelphia:
Benjamins.
Phillips, Barbara J., and Edward F. McQuarrie
2002 The development, change, and transformation of rhetorical style in
magazine advertisements 1954–1999. Journal of Advertising 31 (4):
1–13.
2004 Beyond visual metaphor: A new typology of visual rhetoric in adver-
tising. Marketing Theory 4 (1–2): 113–136.
Radden, Günther
2000 How metonymic are metaphors? In Metaphor and Metonymy at the
Crossroads: A Cognitive Perspective, Antonio Barcelona (ed.), 93–
108. Berlin/New York: Mouton de Gruyter.
Ruiz de Mendoza, Francisco José
1999 Introducción a la teoría cognitiva de la metonimia. Granada:
Método ediciones.
2000 The role of mappings and domains in understanding metonymy. In
Metaphor and Metonymy at the Crossroads: A Cognitive Perspec-
tive, Antonio Barcelona (ed.), 109–132. Berlin/New York: Mouton
de Gruyter.
Ruiz de Mendoza, Francisco José, and Olga Isabel Díez Velasco
2002 Patterns of conceptual interaction. In Metaphor and Metonymy in
Comparison and Contrast, René Dirven and Ralph Pörings (eds),
489–532. Berlin/New York: Mouton de Gruyter.
Sperber, Dan, and Deirdre Wilson
1995 Relevance: Communication and Cognition. 2nd ed. Oxford: Black-
well.
Scott, Linda, and Rajeev Batra (eds.)
2003 Persuasive Imagery: A Consumer Response Perspective Mahwah,
NJ/London: Lawrence Erlbaum.
Stöckl, Hartmut
2004 In between modes. In Perspectives on Multimodality, Elija Ventola,
Cassily Charles, and Martin Kaltenbacher (eds.), 9–30. Amsterdam/
Philadelphia: Benjamins.
Interaction of multimodal metaphor and metonymy 117
Tanaka, Keiko
1994 Advertising Language: A Pragmatic Approach to Advertisements in
Britain and Japan. London/New York: Routledge.
Taylor, John
2002 Category extension by metonymy and metaphor. In Metaphor and
Metonymy in Comparison and Contrast, René Dirven and Ralph
Pörings (eds.), 323–347. Berlin/New York: Mouton de Gruyter.
Teng, Norman Y., and Sewen Sun
2002 Grouping, simile, and oxymoron in pictures: A design-based cogni-
tive approach. Metaphor and Symbol 17: 295–316.
Urios-Aparisi, Eduardo
2004 Quarrelling about metaphors on love: A pragmatic approach. In
Current Trends in the Pragmatics of Spanish, Maria Elena Placen-
cia and Rosina Márquez Reiter (eds.), 283–310. Amsterdam/ Phila-
delphia: Benjamins.
Warren, Beatrice
2006 Referential Metonymy. Lund: Almqvist and Wiksell.
Yu, Ning
this vol. Nonverbal and multimodal manifestations of metaphors and me-
tonymies: A case study.
Yus, Francisco
this vol. Visual metaphor versus verbal metaphor: A unified account.
Chapter 6
Nonverbal and multimodal manifestations of
metaphors and metonymies: A case study
Ning Yu
Abstract
This paper intends to analyze, within the cognitive linguistic paradigm, the non-
verbal and multimodal manifestations of metaphors and metonymies in an educa-
tional commercial screened on China Central Television (CCTV). Specifically, it
shows how two major conceptual metaphors, LIFE IS A JOURNEY and LIFE IS A
STAGE, are manifested in dynamic visual and aural, as well as verbal, discourse.
The various visual, aural and verbal elements are interactive with and dependent
upon each other when they combine into a “conceptual blend” with “input spaces”
in visual, aural and verbal modes. This blend contains conspicuous juxtapositions
of various kinds, simultaneous or sequential, which cast in relief the unity and
contrast between the Chinese and the Western, between thought and action, be-
tween primitivity and modernity, and between tradition and innovation. They all
contribute to the central theme of the commercial that China, thanks to a motiva-
tion for change that originates in her “heart,” has been undergoing the process of
modernization and globalization while retaining her cultural identity.
1. Introduction
During the past two decades, cognitive science has seriously challenged the
fundamental assumption that most of our thinking about the world is literal,
directly corresponding to external reality (see e.g., Gibbs 2006; Lakoff and
Johnson 1999). The results of cognitive linguistic studies show that human
minds are embodied in the cultural world, and thinking and reasoning are
largely metaphorical and imaginative, shaped by embodied and acculturated
experiences (e.g., Gibbs 1994, 2006; Johnson 1987; Lakoff 1987; Lakoff
and Johnson 1980, 1999). It is argued that “all cognition is embodied in
cultural situations” (Gibbs 1999: 156).
According to the conceptual metaphor theory of cognitive linguistics,
metaphor is not merely a figure of speech, but also a figure of thought, giv-
ing rise to understanding one conceptual domain in terms of another concep-
tual domain.1 Conceptual metaphors in people’s conceptual systems influ-
ence to a considerable extent how they think, understand, reason, and
imagine in everyday life, and “many concepts, especially abstract ones, are
structured and mentally represented in terms of metaphor” (Gibbs 1999:
145).
It is worth stressing that the experiential basis of conceptual metaphors is
both bodily and cultural. Cognitive linguistics maintains that our minds are
embodied in such a way that our conceptual systems draw largely upon the
peculiarities of our bodies and the specifics of our physical and cultural
environments (e.g., Gibbs 1994, 2006; Lakoff and Johnson 1980, 1999).
According to the cognitive linguistic view, conceptual metaphors emerge
from the interaction between body and culture: they are grounded in bodily
experience, but shaped by cultural understanding.
In order to answer the question why some metaphors are widespread or
even universal and others are culture-specific, the newer version of Concep-
tual Metaphor Theory puts forth a “decomposition” account based on the
distinction between two kinds of conceptual metaphors: primary metaphors
and complex metaphors (see Grady 1997a, 1997b, 2005; Kövecses 2002,
2005; Lakoff and Johnson 1999, 2003). In short, primary metaphors are
derived directly from experiential correlations, or “conflations in everyday
experience” that “pair subjective experience and judgment with sensorimotor
experience” (Lakoff and Johnson 1999: 49), whereas complex metaphors are
combinations of primary metaphors and cultural beliefs and assumptions
and, for that reason, tend to be culture-specific. For instance, Lakoff and
Johnson (1999: 60–61) suggest that the complex metaphor A PURPOSEFUL
122 Ning Yu
Figure 9. Dancing past the wall Figure 10. Leading the way
Figure 11. Leading and supporting Figure 12. “Size of heart and stage”
Figure 13. Dancing alone again Figure 14. Gazing afar standing
4. Analysis
In this section, I analyze the TV commercial to show that its didactic and
aesthetic effects are achieved through, among other things, nonverbal and
multimodal manifestations of two common conceptual metaphors: LIFE IS A
JOURNEY and LIFE IS A STAGE. While the former is realized almost exclu-
126 Ning Yu
In this group, (1a) and (1b) are two propositions that reflect the Chinese
cultural conceptualization of the “heart” whereas (1c) and (1d) are meta-
phors that are rooted in the cultural beliefs of the “heart” as the central fac-
Nonverbal and multimodal metaphors and metonymies 127
JOURNEY has a metonymic basis when JOURNEY is mapped onto LIFE, char-
acterized by a conceptual metonymy PART FOR WHOLE.
In the TV commercial under discussion, the traveler is the country girl.
For her, traveling is dancing a Western ballroom dance that she really en-
joys, even though she does it initially without a partner. For her, the path of
the journey runs from the cold of the snow-covered countryside to the
warmth of the sun-bathed modern metropolis, and from the country field to
the top of a skyscraper in a big city. More abstractly, this is a path of going
upward in spatial conceptualization of success in life.
The correspondences listed in (3) and (4) are some of the entailments of
the mappings given in (2) (see also Lakoff 1993):
(3) EXPERIENCE IN LIFE IS TRAVEL ON JOURNEY
a. DIFFICULT TRAVEL → BAD EXPERIENCE
b. EASY TRAVEL → GOOD EXPERIENCE
c. FAST MOTION → FAST PROGRESS
d. SLOW MOTION → SLOW PROGRESS
Here, (5a) and (5b) present two propositions as the cultural beliefs or as-
sumptions upheld by people who subscribe to the complex metaphor LIFE IS
A JOURNEY; (5c–g) are primary metaphors of the so-called Event Structure
Nonverbal and multimodal metaphors and metonymies 129
The list in (7) below gives some of the conceptual mappings entailed by (6b)
from the concrete domain of space to the abstract domain of success.
130 Ning Yu
(7) SUCCESSFUL IS UP
a. UP → SUCCESSFUL
b. HEIGHT OF LOCATION → DEGREE OF SUCCESS IN LIFE
c. HIGH LOCATION → SUCCESSFUL STATUS IN LIFE
d. HIGHER → MORE SUCCESSFUL
e. LOWER → LESS SUCCESSFUL
Here (8c) is the key metaphorical component. Since ACTION IN LIFE IS AC-
TION ON STAGE, after the identical item, ACTION, on both sides of the equa-
tion is eliminated, we have LIFE IS A STAGE. But what is the experiential
motivation for (8c)? I argue that this metaphor is motivated by a more fun-
damental figurative relationship, a metonymy, ACTING ON STAGE STANDS
FOR ACTING IN LIFE, which is a specific instantiation of the more general
conceptual metonymy PART STANDS FOR WHOLE. That is, acting on the
stage is only part of the whole, acting in life, and here we use a part to stand
for the whole. In the above list, (8d) and (8e) represent two primary meta-
Nonverbal and multimodal metaphors and metonymies 131
phors in the Event Structure Metaphor System on which the complex meta-
phor LIFE IS A STAGE is based. Life is a series of states whereas a stage is a
special kind of location. The actions that people take in life, whether con-
crete or abstract, are generally understood as self-propelled motions through
space. In this particular case, actions taken in life are metaphorically con-
ceptualized as artistic moves of ballroom dancing. Besides, I assume that the
cultures that subscribe to the LIFE IS A STAGE metaphor also hold the propo-
sitions in (8a) and (8b), in combination with (8c). Thus, the conceptual par-
allel is perceived as follows. People act to achieve success in life, just as
performers act to achieve success on the stage; their actions in life are evalu-
ated by others, just as actors and actresses’ performances are evaluated by
their audience.
The LIFE IS A STAGE metaphor establishes, for instance, the correspon-
dences between the following elements in the two conceptual domains.
(9) LIFE IS A STAGE
SOURCE TARGET
a. STAGE → LIFE
b. PERFORMANCE ON STAGE → ACTIVITY IN LIFE
c. ROLES ON STAGE → PEOPLE IN LIFE
In the TV commercial, the country girl dances all the way from a small vil-
lage to a big city, so the “stage” of her life is indeed very big. Her perform-
ance can be divided into four phases. In the first phase, she is alone and
starts dancing ballroom dance. Her moves, though graceful, are repetitive,
metaphorically representing, I suggest, her persistence and perseverance in
pursuit of her goal. The fact that the Chinese country girl dances a Western
ballroom dance, rather than a Chinese folk dance, is unexpected and for that
matter really significant. In China, the Western ballroom dance is often re-
ferred to as the “international standard dance.” During the past 15 years or
so, a goal that China has been trying to achieve to implement the “reform
and open-door” policy is “to be connected with the international track/rail”
(与国际接轨), i.e., to meet the international standard. Thus, the country
girl’s persistence in dancing the “international standard dance” can be seen
as a visual metaphor of China’s effort to meet the international standard.
In the second phase, the country girl is joined by a male dancing partner
wearing the standard ballroom dance apparel (i.e., a black swallow-tailed
tuxedo and black leather shoes), as in sharp contrast with her Chinese peas-
ant-style clothing. Together, they make all kinds of beautiful moves and
poses, accompanied by Western ballroom dance music. Their fast-tempo
132 Ning Yu
As shown in (10) and (11), the entailments all contribute to the systematic
mappings from the source to the target domain, activated by the LIFE IS A
STAGE metaphor. In the TV commercial, the country girl is indeed very suc-
cessful in life since, later in the show, she is playing a leading role in the
foreground of the “stage.”
Nonverbal and multimodal metaphors and metonymies 133
Given in (13) below are some of the conceptual mappings entailed by (12b)
from the concrete domain of spatial dimensions to the abstract domain of
success.
(13) SUCCESSFUL IS BIG
a. BIG → SUCCESSFUL
b. SIZE OF OBJECT → DEGREE OF SUCCESS IN LIFE
c. BIG OBJECT → SUCCESSFUL STATUS IN LIFE
d. BIGGER → MORE SUCCESSFUL
e. SMALLER → LESS SUCCESSFUL
That is, the physical size of an object is mapped on to the degree of success.5
In (12) the OBJECT is specified as a “stage.” Thus, (14) lists some of the
conceptual mappings entailed by (12):
(14) A SUCCESSFUL LIFE IS A BIG STAGE
a. BIG → SUCCESSFUL
b. SIZE OF STAGE → DEGREE OF SUCCESS IN LIFE
c. BIG STAGE → SUCCESSFUL STATUS IN LIFE
That is to say, the size of one’s stage is metaphorically correlated with the
degree of success in one’s life: DEGREE OF SUCCESS IN LIFE IS SIZE OF
STAGE. The bigger one’s stage is (of course metaphorically), the more suc-
cessful one is in life. It is worth noting that the top of the skyscraper, where
the girl, her dancing partner, and 24 other pairs are dancing, looks very
much like a big stage. Thus, the metaphor A SUCCESSFUL LIFE IS A BIG
STAGE is manifested visually as well as verbally.
In summary, the LIFE IS A STAGE metaphor is manifested visually through
moving images, accompanied by musical sounds, as well as linguistically
through the verbal message appearing on the TV screen. The fact that the
country girl has made remarkable progress in life is revealed via visual
metaphors. First, the change in physical location of her dancing from the
134 Ning Yu
That is, as the first step, STAGE is mapped metonymically onto the PER-
FORMANCE ON STAGE and, as the second step, PERFORMANCE ON STAGE is
mapped metaphorically onto ACTIVITY IN LIFE.
Apart from the two major conceptual metaphors discussed above, the TV
commercial has also deployed a number of metonymies to achieve its didac-
tic purpose and artistic effect, as has already been touched upon above. For
instance, in the verbal message “In everyone’s heart there is a big stage;
however big one’s heart is, that is how big the stage is,” we can say that,
initially, the reference to the “stage” is a metonymy for the “performance on
the stage,” i.e., STAGE FOR PERFORMANCE ON STAGE, or more generally
LOCATION OF ACTIVITY FOR ACTIVITY. In this case, the figurative mapping
takes place from one thing to another within the same conceptual domain. It
is through further mapping across the domains that the metaphor ACTIVITY
IN LIFE IS PERFORMANCE ON STAGE is constructed.
In this section, I discuss several other metonymies in the visual and aural
modes. In effect, these metonymies under analysis are all integrated into the
complex of conceptual metaphors. The first visual metonymy is STYLE OF
CLOTHING STANDS FOR CULTURE. The girl wears typical peasant-style cloth-
ing, which is metonymically associated with traditionally agricultural Chi-
nese culture. It fits well into the rural setting at the beginning of the commer-
Nonverbal and multimodal metaphors and metonymies 135
In sum, the metonymies discussed in this section all fall into one general
pattern, where PROTOTYPICAL ITEMS OF A CULTURE STAND FOR THAT CUL-
TURE or SALIENT FEATURES OF A THING STAND FOR THAT THING. As we
have seen, this conceptual metonymy can be manifested visually and aurally
as well as verbally.
5. Conclusion
her “heart,” has been undergoing the process of modernization and global-
ization while retaining her “Chinese characteristics.”
Table 1. Some possible multimodal mappings of the metaphors LIFE IS A JOURNEY
and LIFE IS A STAGE.
Metaphors Visual Aural Verbal
Source: (1) the country girl danc- (1) starting with
JOURNEY ing from the field of the slow-tempo music;
countryside to the top of a (2) switching to
skyscraper in the me- fast-tempo music;
tropolis; (2) enjoying the (3) winding down
global view at height and with slow-tempo
looking afar and ahead music again
Target: (a) the person/China (1) starting at a
LIFE undergoing the process of self-propelled pace;
modernization and glob- (2) switching to a
alization; (2) gaining a fast speed; (3) slow-
deep understanding upon ing down to a stop
success and looking into
the future
Source: the girl in peasant cloth- (1) starting with “In everyone’s
STAGE ing (1) dancing Western Chinese music; heart there is a
ballroom dance alone; (2) (2) switching to big stage;
dancing in pair with a Western music; however big
male partner in Western one’s heart is;
(3) ending with
tuxedo; (3) dancing as that is how big
Chinese music
the leading couple; (4) the stage is.”
again
still wearing the same
peasant clothing in a
changed environment
Target: The person/China, with (1) Chinese culture; Success in life
LIFE her cultural identity, (1) (2) international- originates
making self-propelled ization; internally in
effort toward moderniza- one’s mind;
(3) retention of
tion and globalization; only when one
cultural identity
(2) working in coopera- “thinks big”
tion with the West; (3) can one “act
advancing as a leader in big” and
development; (4) retain- achieve big
ing cultural identity in success in life.
modernization
Nonverbal and multimodal metaphors and metonymies 139
Acknowledgments
An earlier version of this paper, titled “Cultural identity and globalization: Mul-
timodal metaphors in a Chinese educational advertisement,” was presented at the
12th Annual Conference of the International Association of Intercultural Commu-
nication Studies held in San Antonio in August 2006, and appeared in China
Media Research 3(2): 25–32, 2007. I am greatly appreciative of Charles Force-
ville and Mats Rohdin for their insightful comments on the earlier versions of this
chapter, but I am solely responsible for any errors that may remain.
Notes
References
Francisco Yus
Abstract
Multimodal metaphors are those “whose target and source are each represented
exclusively or predominantly in different modes” (Forceville 2006: 384/this vol-
ume), mainly with a verbal-visual interface of source and/or target. When multi-
modality is analyzed in metaphors, the verbal and visual inputs are wrongly
treated as different phenomena demanding different interpretive strategies when
searching for a metaphoric interpretation. In this chapter, on the contrary, it is
claimed that the comprehension of verbal, visual and multimodal metaphors in-
volves similar mental procedures. Although the perception of images differs from
linguistic decoding, reaching an interpretation of metaphors entails similar ad-
justments of conceptual information of texts and images and multimodal combina-
tions, regardless of the modal quality of the input.
1. Introduction
Sperber and Wilson’s (1995 [1986]) relevance theory (henceforth RT) pre-
dicts that human comprehension follows two stages:
“jam” have to do with food, but faced with the incongruity between “food”
and the advertised “transport company,” the reader will continue testing
interpretive hypotheses concluding, at a second stage, that bread is a collo-
quial word for “money” and jam refers to “traffic jams.” The reader will
now be satisfied at this interpretation and stop processing here.
Specifically, for RT, comprehension does not normally start in a commu-
nicative vacuum, but takes place against a context of previous utterances
whose interpretation (stored in the short-term memory) works as a back-
ground against which new information is processed. Cognitive linguistics has
also drawn attention to the role of context for the right comprehension of
metaphors. For example, for Conceptual Metaphor Theory, metaphors may
be activated as part of the hearer’s understanding of context, and this will
make the interpretation of metaphors easier at subsequent stages in dis-
course. Similarly, conceptual blending theory stresses the role of context in
metaphor comprehension: “because cognitive activity mediates the relation-
ship between words and the world, the study of meaning is the study of how
words arise in the context of human activity, and how they are used to evoke
mental representations” (Coulson, quoted in Tendahl and Gibbs 2008:
1843).
This “cumulative” background context of previous utterances in the con-
versation is normally absent in the processing of visual metaphors, which are
inserted in media discourses such as newspapers, billboards or magazines,
and hence the viewers1 have to interpret them from scratch, without this
readily available “short-term memory store” of information. This does not
mean that visual metaphors do not require a great deal of background
knowledge for their satisfactory interpretation. The metaphors used in car-
toons, such as the ones analyzed in this chapter, are often related to recent
news-worthy events whose knowledge is essential to get the right extent of
the metaphoric mappings (cf. Peñamarín 1996; El Refaie this volume; Schil-
peroord and Maes this volume; Forceville 2005).
RT predicts two clear-cut phases during interpretation: one of decoding
and one of inference. The first one is in charge of the language module of
the mind (Fodor 1983), which apprehends a linguistic sequence and yields a
de-contextualized but grammatical “logical form” which has to be enriched
in order to be meaningful. By contrast, cognitive linguistics disregards
modularity in favor of what is called the embodied-mind hypothesis, accord-
ing to which “the same neural mechanisms used in perception and bodily
movement play a role in all forms of conceptualization, including the crea-
tion of lexical fields and abstract reasoning” (Ruiz de Mendoza 2005: 36).
150 Francisco Yus
For RT, language does not encode thoughts, but only clues that help the
hearer access the speaker’s thoughts, which are often more complex than the
literal meaning encoded by the utterances. For example, the (b) versions of
the following utterances are closer to the thoughts that the speaker intended
to communicate with them than the schematic (and communicatively useless)
(a) versions, the ones actually uttered:
The hearer of (1a-c) is expected to adjust the encoded concept “soft” into a
more appropriate and contextualized type of softness that specifically ap-
plies to sofas, skins and cats respectively, that is, adjust into more relevant
ad hoc concepts SOFT*, SOFT**, and SOFT***.2
Visual metaphor versus verbal metaphor 151
(4) a. The fish attacked some people near here (FISH* = dangerous fish, e.g.,
sharks).
b. The boy has a temperature (TEMPERATURE* = higher temperature than
normal).
c. It will take some time to fix the car (SOME TIME* = longer than it usu-
ally takes).
(5) a. The steak is raw (RAW* = undercooked).
b. She is a genius! (GENIUS* = not literally a genius, but having some of
his/her qualities).
c. It was quiet in the street last night (QUIET* = with very little noise).
(6) A. Why does your boyfriend want you to go with him everywhere?
B. Because he is a baby.
(BABY* denotes a person who cannot be independent, cannot look after
himself, can’t do things alone, etc. These are qualities applicable to all
babies (as prototypical referents) and also to some adults such as the
speaker’s boyfriend).
152 Francisco Yus
(b) Ad hoc concepts which contain qualities which are applicable to some
of the prototypical referents of the encoded concepts and also to a range of
other referents, as in (7) below:
(7) Being the only boy, Dave has always been the prince of the house.
(PRINCE* denotes a subset of princes who are spoilt and do as they
please, as well as a set of young boys who are not princes but are spoilt
and do as they please).
(c) Ad hoc concepts which contain qualities which are applicable to none
of the prototypical referents of the encoded concepts but are applied to other
referents, as in the utterance quoted in (8):
(8) I tried to persuade him to change the essay topic but there was no way.
He is an iron bar.
(IRON BAR* denotes people who are difficult to convince, persuade, etc.,
qualities which are not found in iron bars as prototypical referents).
In my opinion, these three cases are not only inherent to verbal metaphor
comprehension, but are also found in the processing of visual metaphors. In
this sense, case (c) is interesting because it gives rise to the so-called emer-
gent features or emergent properties which apparently do not belong to the
target domain of the metaphor but seem to emerge during comprehension
(Gineste, Indurkhya, and Scart 2000; Wilson and Carston 2006). These
emergent properties might appear to be found only in the interpretation of
verbal metaphors but, as will be argued below, they are also frequent in
visual metaphor comprehension (cf. Yus 2003a) and in any multimodal
combination of text and image.
Many explanations have been suggested for the creation of these emer-
gent properties.4 I will follow an interesting proposal by Vega-Moreno
(2004) within a relevance-theoretic point of view, and I will argue that this
proposal is applicable, in a similar way and with the necessary adjustments,
to the processing of emergent properties in visual or multimodal metaphors.
More generally, I will show to what extent conceptual assessment is involved
in visual metaphor comprehension, basically through what will be called
stable versus innovative conceptual upload. The analysis will be divided
into several steps that the reader is expected to go through during the inter-
pretation of a visual metaphor. Comparisons with verbal metaphor compre-
hension will be made where necessary, and there is an inherent claim in this
proposal: that combinations of text and image in multimodal metaphors de-
Visual metaphor versus verbal metaphor 153
physical object or picture to which the referent is associated helps the reader
to update the prototypical referent that was created when the reader first
perceived it.
Upon detecting the ad hoc pointer, the reader of the image(s) enters another
stage in interpretation, which I will call visual-conceptual interface, in-
between a sub-attentive perception of the images and a fully inferential ex-
traction of a relevant connotative (i.e., metaphoric) interpretation of the im-
age. At this stage, the reader aiming at an optimally relevant interpretation
has to raise a number of preliminary hypotheses concerning the intended
relationship that holds between the depicted images and the encyclopedic
(conceptual) information stored about the referents of these images, mostly
of a stereotypical quality. In short, the readers would ask themselves ques-
tions such as the following:
1. Which are the two images related metaphorically? Are both present in
the picture? Visual metaphor involves a mapping of information transferred
from one image to another, which we have called source image and target
image respectively. Often both images are present in the picture (either fused
together or separated) but sometimes one of them – normally the source
image – is absent. Therefore, there are different degrees of mental effort
involved in processing visual metaphors depending on whether both the
source image and the target image are depicted in a metaphoric visual con-
figuration, or one of the images is absent and is only accessible through an
inferential operation regarding the encyclopedic information on its prototypi-
cal referent. At the same time, some mental effort has to be devoted to identi-
fying the source and target images in the first place, which are not always
clearly distinguishable, even when both images are present.
2. What kind of visual arrangement is there between the images? The
reader is also expected to infer what relationship holds between the previ-
ously identified source image and target image. Is the target image sup-
posed to be like the source image? Is it opposed to the source image? (cf.
Phillips and McQuarrie 2004.)
3. Are the prototypical encyclopedic referents of the images themselves
the ones that are going to undergo inferential adjustment in order to obtain a
metaphoric interpretation or do the images stand for a different encyclopedic
referent? I believe that visual metaphor comprehension, in a similar way to
verbal metaphor comprehension, also involves an access to and adjustment
of conceptual information stored in or attached to the encyclopedic proto-
typical referents of the image or images depicted. This implies that it is of
utmost importance to determine whether the author intends the most accessi-
ble referents of the images to undergo metaphoric processing or whether the
intended sources of metaphor have to be found elsewhere. This is the case of
Visual metaphor versus verbal metaphor 157
images that stand metonymically for other referents. For instance, in the
visual metaphor described in (9), it is the prototypical referent of the image
“buildings” that works as target domain for metaphoric mapping, whereas in
(10), the visual referent of “book page” is not expected as target domain,
since it stands for “culture in general,” which is the intended target domain.
required, in the same way as are emergent properties that arise in case 3 of
conceptual adjustment during verbal metaphor comprehension.
For example, another cartoon by El Roto (El País, 9 June 2002) depicts a
goal keeper clearing a book (instead of the expected ball) with his fists. In
the metaphor THE BOOK IS A BALL, none of the features of the prototypical
referent “ball” seems to be applicable to the referent “book” of the target
image. Whatever metaphoric interpretation the reader chooses (e.g., football
makes people reject books” or “football stops people from getting real cul-
ture” – if “book” stands metonymically for “culture”) will emerge during the
processing of the referents of the images, and will involve more inferential
effort than a simple broadening of one or several features of the prototypical
encyclopedic referent of the source image in order to obtain an ad hoc con-
cept BALL*. More inferential activity will have to be devoted to adjusting
some feature(s) that belong to the prototypical encyclopedic referent.
Vega-Moreno (2004: 318f) exemplifies this possible explanation of
emergent properties with the verbal metaphor communicated in (11) below:
seeking procedure); (c) the encyclopedic referent EARTH has qualities of the
encyclopedic referent SAUCEPAN; and (d) these are the prototypical referents
intended by the author; the images of the earth and the saucepan do not stand
metonymically for other referents.
5. The reader starts computing assumptions in order of accessibility fol-
lowing a relevance-guided procedure, beginning with the ones which belong
to the prototypical encyclopedic referent depicted in the source image:
SAUCEPAN.
6. One of the encyclopedic features of the prototypical encyclopedic ref-
erent SAUCEPAN seems to be directly applicable to the earth: “heats up
gradually,” since the earth, due to global warming and the so-called “green-
house effect” is also heating up gradually. This involves the creation of an
ad hoc concept SAUCEPAN* whose quality “heats up gradually” remains
relatively stable in the metaphoric process (i.e., undergoes a minimal ad-
justment via broadening). This new concept is applicable to all saucepans
and also metaphorically to the new encyclopedic referent EARTH. In this
sense, this could perhaps be a case of ontological metaphor in Lakoff and
Johnson’s (1980) terminology, since in this case an inherent quality of
saucepans is attributed to the target image. It would also fit Ruiz de Men-
doza’s (1998) one-correspondence metaphor, since in this case only one
correspondence between the source and the target is exploited.
This visual metaphor would fit case 1 of ad hoc concept formation, in
which the intended conceptual feature is found in all the prototypical refer-
ents depicted by the image plus a number of other entities included through
broadening.
7. The presence of SAUCEPAN* in what can be called the explicit content
of the referents attached to the items depicted in the cartoon warrants the
derivation of a number of possible implicated conclusions. Cartoons are a
good example of a medium in which current news-worthy events play a part
in the generation of implicated conclusions. In this case, if the reader knows
about the fact that, at the time the cartoon was published, there was a debate
on the Kyoto protocol and whether Japan and Australia would sign it (i.e., if
this information is manifest to him/her, in RT terms), this information will
influence both the accessibility to the visual metaphor and the mental effort
devoted to its processing.
(b) THE BALLOT BOX IS A DICE (figure 4)
Steps 1. and 2. as above.
3. The reader enters a visual-conceptual interface, in which a number of
hypotheses are made concerning the encyclopedic referents of the images,
once the prototypical visual referents have been perceived, again preparing
164 Francisco Yus
the ground for a fully inferential stage. Some conclusions should be derived:
(a) the dice is the source image; (b) the dice stands metonymically for “gam-
bling with dice” and more generally for “all types of gambling” (and hence
the encyclopedic referent intended by the author to undergo metaphoric
transference is not DICE, but GAMBLING); (c) the ballot box is the target
image (facilitated by the reader’s background knowledge about the fact that
the cartoon was published in a time of political elections); (d) the ballot box
is in a metonymic relationship to “political elections” (and hence the referent
intended by the author to undergo metaphoric transference is not BALLOT
BOX, but POLITICAL ELECTIONS in general or more specifically the ones tak-
ing place in the near future); (e) the political elections have qualities of gam-
bling.
From the examples analyzed in the previous section, we can conclude that
interpreting visual metaphors also involves a great deal of conceptual upload
and adjustment following a criterion guided by a search for relevance. In this
sense, it is worth commenting that many visual metaphors are original in the
way they create a metaphoric link through an anomalous visual arrangement,
while others seem to include an anchorage of previously used verbal meta-
phors which are simply transferred to a visual medium and were probably
stored previously as conventionalized metaphors. In these cases, the meta-
phor-seeking conceptual assessment can indeed be speeded up by the fact
that a particular feature of the prototypical encyclopedic referent of the
source image has been made prominent by previous use through verbal
means, or even facilitated by the fact that the visual metaphor only exists
because there is an underlying verbal one. In a way, this is the counterpart of
Visual metaphor versus verbal metaphor 167
the well-studied fact that verbal metaphors also draw on a conceptual reper-
toire of visual sensory schemas which aid in the metaphoric attribution, to
the extent that these images often end up becoming conventionalized in the
language and deprived of their sensory metaphoric power.11
Therefore, it comes as no surprise that so many metaphors involve the
mediation of visual sensory information contained in image schemas (see
Lakoff 1987), or involve what can be called re-visualization of convention-
alized verbal metaphors, which is extensively used by cartoonists. Several
steps are involved in the comprehension of the cartoons that fit this quality:
(a) initially, an image is much more effective (i.e., vivid) than the range of
coded options available to communicate a thought. (b) A metaphor is created
that contains a schema as a referent. This schema contains visual sensory
information.12 (c) Repeated use of the metaphor makes it lose its sensory
vividness and it ends up becoming conventionalized and hence people stop
regarding it as a metaphor. (d) The cartoonist takes this conventionalized
metaphor and re-visualizes it, as it were, forcing the reader to re-incorporate
into its processing all the sensory vividness that the metaphor had already
lost.
An example is a cartoon drawn by El Roto (El País, 15 April 1996) de-
picting the earth split into two parts, and with a big gap between the North
and the South hemispheres. There are people trying to jump from the South-
ern hemisphere onto the Northern one but they inevitably fall into the huge
gap. This metaphor, which can be described as “there is an abyss between
the North and the South,” reproduces the aforementioned steps: (a) the sen-
sory information of an abyss is more vivid than other coded options to com-
municate depth and distance between A and B; (b) a metaphor THE DIFFER-
ENCE BETWEEN NORTH AND SOUTH IS AN ABYSS is created containing visual
sensory information; (c) the metaphor ends up becoming conventionalized
and loses its sensory power (people stop seeing an abyss when uttering the
metaphor); (d) the cartoonist re-visualizes the information contained in the
metaphor, forcing the reader to see the sensory qualities of the image schema
that had been lost due to conventionalization.
6. Concluding remarks
Interpreting visual metaphors does not differ substantially from verbal meta-
phor comprehension. Both kinds of metaphor are “decoded” by a specialized
mental module (Fodor 1983) which delivers schematic information that has
to be enriched inferentially in order to obtain the intended interpretation (an
168 Francisco Yus
Notes
1. Since all the visual metaphors analyzed or referred to in this article are taken
from newspapers, from now on the viewer of the visual metaphor will be re-
ferred to as “reader”.
2. I will use the convention of adding asterisks to stress the fact that the accom-
panying word is an ad hoc concept and not an encoded concept.
3. In fact, context plays an important role aiding the addressee in determining
the kind of CONCEPT* that the speaker or the author intends. Normally, in the
course of a conversation, previous utterances and background knowledge
about the speaker work as an important “short-term-memory storage” of in-
formation against which new utterances are interpreted. Vega-Moreno (2004:
317) explains this with the metaphor “my boss is a shark.” If it is clear from
previous turns of the conversation or from general encyclopedic information
about the speaker that he is happy with his boss, the concept AGGRESSIVE
may be adjusted to denote a kind of (positive) aggressiveness that involves
energy and assertiveness (represented as AGGRESSIVE*). However, processing
the metaphor on the assumption that the speaker is afraid of his boss’s tactics
Visual metaphor versus verbal metaphor 169
References
Barthes, Roland
1977 Image, Music, Text. London: Fontana.
Carston, Robyn
2002 Thoughts and Utterances: The Pragmatics of Explicit Communica-
tion. Oxford: Blackwell.
Cavanagh, Patrick
1998 Top-down processing in vision. In Encyclopedia of Cognitive Sci-
ence (MITECS), 839–840. Cambridge, MA: MIT Press.
El Refaie, Elisabeth
2003 Understanding visual metaphor: The example of newspaper car-
toons. Visual Communication 2 (1): 75–95.
this vol. Metaphor in political cartoons: Exploring audience responses.
Fauconnier, Gilles, and Mark Turner
1998 Conceptual integration networks. Cognitive Science 22: 133–187.
Fodor, Jerry
1983 The Modularity of Mind. Cambridge, MA: The MIT Press.
Forceville, Charles
1996 Pictorial Metaphor in Advertising. London/New York: Routledge.
2005 Addressing an audience: Time, place, and genre in Peter van
Straaten’s calendar cartoons. Humor 18: 247–278.
2006/this vol. Non-verbal and multimodal metaphor in a cognitivist frame-
work: Agendas for research. In Current Applications and Future
Perspectives, Gitte Kristiansen, Michel Achard, René Dirven, and
Francisco Ruiz de Mendoza (eds.), 379–402. Berlin/New York:
Mouton de Gruyter.
Visual metaphor versus verbal metaphor 171
Elisabeth El Refaie
Abstract
Using data from a study of young people’s responses to British newspaper car-
toons, this chapter considers the ways in which readers interpret multimodal
metaphors of the verbo-visual variety. One of the central tenets of Conceptual
Metaphor Theory is that many metaphors derive from our bodily experience and
are thus likely to be understood in similar ways by all human beings. But in fact
there is increasing evidence that the interpretation of metaphors is partly depend-
ent upon people’s socio-cultural background, as well as on the contexts in which
the metaphors are used. The results of our study suggest that some metaphorical
mappings in cartoons, such as those between size and power/status, and between
movement through space and the passing of time, might be understood more gen-
erally and at a more intuitive level than more elaborate structural metaphors,
which tend to be interpreted in different ways by different individuals.
1. Introduction
toons combine visual and verbal codes. As I argue in section 2, using two
cartoons from our study as examples, newspaper cartoons typically act as a
bridge between fact and fiction, combining actual current events with an
imaginary, make-believe world created by the cartoonist (Edwards 1997: 8).
While this metaphorical process of transferring meaning from the make-
believe to the real world tends to be conveyed predominantly in the visual
mode, most cartoon metaphors also rely to some extent on verbal cues.
Sometimes either the target or the source is represented exclusively through
language, but more frequently verbal labels in cartoons are used as a means
of specifying important aspects of a primarily visual metaphor. In the case
of political cartoons, Forceville’s (2006: 384/this volume) definition of mul-
timodal metaphors as “metaphors whose target and source are each repre-
sented exclusively or predominantly in different modes” thus needs to be
interpreted in a way that also embraces such asymmetrical verbal-visual
relationships.
Section 3 explores the issue of universality versus individuality in the un-
derstanding of metaphors in cartoons. Conceptual Metaphor Theory is based
on the proposition that metaphor derives from our bodily experience and is
thus an essential part of our everyday patterns of thinking. This suggests that
most instances of metaphor will be understood in similar ways by all mem-
bers of a language community. But in fact it is becoming increasingly clear
that the choice and interpretation of metaphors is partly dependent upon the
participants’ social and cultural background (Kövecses 2005; Proctor, Proc-
tor, and Papasolomou 2005), as well as on the specific contexts in which the
metaphors are encountered (Ritchie 2004).
In section 4, I describe the data and methods used in our study of young
people’s readings of political cartoons, including the measures we took to
ensure that we did not pre-empt their responses. I also explain our decision
to consult the makers of the two cartoons used in the study, Nicholas Gar-
land and Peter Schrank, about their intentions.2 Although we do not consider
the artists to be the ultimate arbiters of the meaning of their work, it proved
revealing to compare the intended meanings with the analysis of the cartoons
by the author of this chapter and with the interpretations generated by the
young people participating in the study.
The fifth section of this chapter discusses some of the results of our
study. The main focus is on the way readers recognize and interpret multi-
modal metaphors, but this cannot always be separated from more general
considerations of how people read visual meaning and how their world
knowledge influences the interpretation process. The discussion of the data
therefore sometimes goes beyond the issue of multimodal metaphor in the
Metaphor in political cartoons 175
narrow sense and includes some reflections on wider issues of cartoon inter-
pretation.
In the final section I reconsider the results and propose ideas for further
research.
If, as most researchers now accept, metaphors operate at the level of thought
rather than being merely linguistic, then any form of communication can be
seen as an instance of metaphor if it is able to “induce a metaphoric thought
or concept” (Kennedy, Green, and Vervaeke 1993: 244). Since the early 90s,
researchers have been discovering manifestations of metaphor in various
non-verbal modes, thereby providing additional evidence for the existence of
metaphorical thought patterns (Seitz 1998). It has also been shown that
metaphors can be cued in more than one mode simultaneously (Forceville
2004; 2006).
However, the search for commonalities must not distract from potential
variations in meaning arising from the genre in which a metaphor occurs. As
Sol Worth ([1974] 1981: 161) pointed out in an early discussion of visual
metaphor, “[i]t is the fact that we learn the agreed-upon rules for the inten-
tional creation of meaning within specific contexts that makes metaphor
possible.” Therefore, genre is likely to have an important influence on the
choice of metaphors by producers, the form these metaphors take, and the
ways in which they are recognized and interpreted by audiences.
Most of the research on visual and multimodal metaphor has so far fo-
cused upon its use in advertising (Forceville 1996; Kaplan 1992; Messaris
1997; Phillips 2003; Scott 1994), where the communicative purpose is obvi-
ous: to attract the attention of potential customers and create (implicit) cog-
nitive links between the product and some desirable abstract quality. Be-
cause of this, visual metaphors in advertising are often highly creative and
unusual. Researchers have tended to focus on this level of explicit meta-
phorical meaning and to disregard the issue of whether adverts also contain
more basic orientational and ontological metaphors, which are thought to
structure human perception and experience at a very fundamental level (La-
koff and Johnson 1980; Kövecses 2002).
Clearly, in the case of political cartoons we are dealing with a completely
different genre, with its own distinctive styles, conventions, and communica-
tive purposes. A political cartoon is an illustration, usually in a single panel,
published on the editorial or comments pages of a newspaper. Generally, the
176 Elisabeth El Refaie
intention was thus to use exaggerated size to signify the abuse of power and
force, rather than simply to increase the salience of this visual element.
relies on every reader’s ability to complete in his or her head what is sug-
gested by an image, including the actions that precede and follow the de-
picted moment (Edwards 1997: 53).
Since static images are also unable to express chronology directly, car-
toonists are bound to use spatial relations to indicate the passing of time.
According to Lakoff and Johnson (1980: 41–44), we are in any case used to
thinking about time in terms of space, with the future in front of us, the pre-
sent right by us, and the past behind. We imagine the passing of time in two
different ways: in the TIME IS A MOVING OBJECT metaphor, we are facing
toward the future, which creeps up on us and which we have to meet head
on. However, time can also be conceptualized as a stationary landscape
through which we move in the direction of the future (we’re approaching the
end of the year). Although these two metaphors are not consistent, they are
nevertheless coherent by virtue of being special cases of the same underlying
metaphor TIME PASSING IS MOTION (Lakoff 1993: 217).
In the two cartoons shown above, action is implied through frozen
movement and strong diagonal vectors, while chronology is suggested
through the relative position of the main active participants, who, in accor-
dance with the MOVING OBSERVER metaphor, are shown traveling through
time from the past to the future. In figure 1, the past is in the foreground and
the future in the background, so that the viewer is facing in the same direc-
tion as the main participants and, like them, is confronted with the prospect
of an uncertain future. The image of someone walking along and, reaching a
fork in the road, having to decide which way to go, is an instantiation of the
LIFE IS A JOURNEY metaphor, which has been described as one of the most
conventional ways of conceptualizing the link between space and time in
Western cultures (Lakoff 1993). Perhaps partly because of the conventional-
ity of this metaphor, which is firmly rooted in the embodied “source-path-
goal” schema (Johnson 1987), visual information alone may suffice to cue
the idea of two destinations representing future decisions.
However, a more specific reading of this cartoon, which could perhaps be
verbalized as FUTURE US FOREIGN POLICY IS BOOT CRUSHING OTHER COUN-
TRIES, requires more detailed information about the various slots in this
schema; this additional information is at least partially provided by verbal
tags. The slot of “moving observer,” for instance, is further specified
through a doubly coded metonym: Since the American President is fre-
quently referred to as “W” in order to distinguish him from his father, the
use of this letter in the cartoon is likely to support the reading of the cowboy
boot as a visual metonym for Bush, a reading principally triggered by his
Texan background and his famous predilection for Western-style casual
180 Elisabeth El Refaie
dress. The detailed drawings of the wall and nuclear power station, meto-
nyms for Iran’s controversial nuclear program and the long-running territo-
rial conflict between Israelis and Palestinians, provide additional information
about the precise “goals” of the boot, as do the visual symbols of the vulture
and the dove, representing war and peace respectively. The verbal elements
on the signs pointing to “Israel + Palestine” and “Iran + North Korea” pro-
vide important support in the identification of these visual elements in the
background, which may otherwise be difficult to recognize. This example
clearly demonstrates that reading political cartoons often requires readers to
draw on several different kinds of interpretive strategies, including the ability
to recognize visual symbols and the targets of metonyms.
Similarly, in figure 2 the basic interpretation of a conceptual link between
space and time involves only the visual mode and is not dependent upon
verbal information. Again, the image of the toddler crawling towards some-
thing cues the MOVING OBSERVER metaphor, although here the orientation is
reversed, so that the background of the image represents the past and the
foreground the future. This places the viewer in a “knowing” position where,
in contrast to Bush-the-toddler, he or she is already able to see only too
clearly what the next few years are likely to bring.
This general orientational schema provides the basis for a more complex
structural metaphor, which we might verbalize as ACHIEVING RE-ELECTION
IS GETTING HOLD OF A BOX OF MATCHES. In this case, only the source is
visually present, while the target must be gleaned from the pictorial context,
any verbal clues, or more generally the “world knowledge” of the viewers
(Forceville 1996: 109). The inscription on the box of matches is instrumental
in pointing the viewer towards a more exact interpretation of the general
SPACE-IN-FRONT IS FUTURE metaphor, since the precise concept of SPACE-
IN-FRONT IS FOUR MORE YEARS IN OFFICE could not have been rendered
exclusively in the visual mode and needed to be supplied partially as a verbal
message.3 In this context, the box of matches thus denotes Bush’s bid to be
re-elected and the opportunity this would offer him to cause even more
havoc, while the flames are clearly meant to represent some of the Presi-
dent’s foreign policies, such as the invasion of Iraq and the launch of his so-
called “war on terror,” which by the time of the cartoon’s publication were
already being seen by many as disastrous.
In the case of the complex structural metaphors FUTURE US FOREIGN
POLICY IS BOOT CRUSHING OTHER COUNTRIES and ACHIEVING RE-ELECTION
IS GETTING HOLD OF A BOX OF MATCHES I believe we are dealing with ex-
amples of true multimodal metaphors, even though, strictly speaking, they
are not covered by Forceville’s (2006: 384) definition, which stipulates that
Metaphor in political cartoons 181
This chapter is based on data collected in phase one of our study into the
ways in which young people understand cartoon images, including the mul-
timodal metaphors they often contain. For this purpose, we conducted one-
to-one interviews with 25 young people between the ages of sixteen and nine-
teen in Bradford, a city with a large British Asian population. Of the 13 men
and 12 women, 13 were Muslims from a Pakistani or Bangladeshi back-
ground, while two each were Hindus and Sikhs from Indian families. The
family of one young man had come to Britain from Afghanistan, and there
were also one British-Indian, one black, and six white British-born young-
sters who described themselves either as Christian or Atheist. Most of our
respondents were doing A-levels, whereas six of the participants were taking
vocational courses. The majority of the interviews were conducted in July
2005 and a small number in November of the same year. We used five car-
toons on the US Presidential elections published several months previously,
in October or November 2004.
Since the participants had volunteered to take part in the study, we ex-
pected them to be on average a little more interested in political issues and
more likely to follow the news than others in their peer group. Although
more than half of our interviewees initially described themselves as not very
or not at all interested in politics, during the interviews the Muslim students
in particular actually turned out to hold quite passionate views about those
geopolitical issues that they felt concerned them directly, such as the inva-
sion of Iraq and the conflict between Israel and Palestine (Hörschelmann
2008). If they read any newspapers at all, the respondents tended to read the
local paper or, more rarely, a tabloid or the occasional broadsheet newspa-
per. In the UK, local newspapers and tabloids tend not to carry many car-
toons of an explicitly political nature.4 Not surprisingly, many of the young
participants were thus not very familiar with the political cartoon genre.
The semi-structured one-to-one interviews, which lasted between 30 and
45 minutes each, took place at the Technology College and local Further
Education College where the young people were studying. After explaining
that the cartoons were all about the recent Presidential elections in the USA,
the cartoons were discussed one by one, using the entire newspaper page in
order to ensure that at least some of the context was preserved. We encour-
aged respondents to read any of the headlines that they would normally read
in conjunction with the cartoon before describing the drawing and then at-
tempting an interpretation. In fact, while the articles below the cartoons did
generally comment on the US Presidential elections, none of them related
184 Elisabeth El Refaie
directly to the content of the cartoons. This meant that, in the few cases that
the respondents did avail themselves of the chance to read the headlines, this
did not necessarily assist them in their interpretation of the cartoons.
In order to ensure that the interview data were comparable, we used the
same basic interview schedule, consisting of a small number of open and
non-leading questions such as: “Can you recognize any of the depicted char-
acters?”; “How would you describe their mood/feelings?”; “What do you
think the cartoonist wanted to say with this cartoon?”; “What are your own
thoughts and feelings when you look at this cartoon?”
While taking care not to prejudice the participants in their responses, we
occasionally used additional prompts in order to elicit more details or to
clarify their answers. For instance, if respondents seemed to be struggling to
describe the mood of the figures in the drawing, we might ask them to de-
scribe the relationship between the depicted characters and to imagine what
they might be feeling. If they overlooked certain important details, such as
the inscription “four more years” on the matchbox in figure 2, we would
generally draw their attention to these and ask what they might be referring
to. In order to preserve the anonymity of the interviewees, responses were
coded with a letter from A to Y. “ER” indicates that I conducted the inter-
view, and “KH” stands for my collaborator, Kathrin Hörschelmann. The
symbol (.) in the transcriptions indicates a hesitation or pause in the respon-
dent’s delivery.
We also sent short questionnaires to the creators of the cartoons used in
the study, asking them to describe the intended meanings of their drawings.
In addition to completing the questionnaire, most of the artists agreed to a
telephone interview, during which they talked about their general working
conditions and practices, and explained in more detail what had given them
the inspiration for the cartoons used in the study, why they had chosen par-
ticular symbols and metaphors, and what they were trying to express
through their work.
Clearly, the meanings intended by the cartoonist are not the only ones
that a particular cartoon – or, indeed, a particular multimodal metaphor – is
able to generate, nor are his or her intentions to be equated with its “correct”
or most valid interpretation. By its very nature, the cartoon genre is ambigu-
ous and open to multiple readings, and the process of making sense of a
cartoon is likely to be strongly dependent upon the individual interpreter’s
background and experiences. In fact, our own detailed analysis of the car-
toons corresponded very closely with the artists’ intended meanings. This
may be linked to the fact that we probably represent the sort of politically
minded regular newspaper readers that the cartoonists apparently had in
Metaphor in political cartoons 185
mind when creating their work. In contrast to this, the young people’s read-
ings of the cartoons and the multimodal metaphors they contained reflected
their very different interests and preoccupations, as well as perhaps a degree
of unfamiliarity with cartoon conventions.
As I showed in section 2, two of the cartoons used in our study contain sev-
eral layers of metaphorical meaning, from the basic connection between
movement through space and the passing of time to the more complex asso-
ciations between a make-believe mini narrative and events in the real world.
In the case of the cartoon by Peter Schrank, published in the Independent on
Sunday (figure 1), another basic metaphor that seemed to evoke a common
response was that of the darkened sky, which for the four participants who
mentioned this aspect indicated a sense of danger and impending disaster.
The orientational TIME IS MOTION THROUGH SPACE metaphor also seemed to
be understood in a similar way by all the respondents, since nearly every-
body equated the concept of the boot walking towards Iran and North Korea
with future actions, with the two signposts pointing in opposite directions
representing alternative decisions.
One exception to this general rule was respondent I, who described the
boot as coming from Palestine and other Muslim countries, where it has
caused a lot of destruction. For this young man, the figure stuck to the sole
of the boot embodied the “innocent people in Afghanistan who have suffered
George Bush’s wrath or so-called wrath, people that suffered in Iraq.” Simi-
larly, respondent J, a young Muslim woman who was clearly very concerned
with the plight of Palestinian civilians, thought the small figure represented a
Palestinian:
J I think he is trying to show all (.) all this war going on (.) on in Pales-
tine (.) that’s been going on for the past year or so (.) it’s trying to (.)
show that (.) it’s George Bush’s fault and you know he is just crush-
ing people for nothing without realizing it and people are yelling at
him look what you are doing to our country but he’s not taking no no-
tice and he is just crushing people under his feet (.) and now he’s
been to Palestine then he’s going to Iran and so he’s going across the
world doing it to all the countries
In fact, eleven respondents did not recognize Tony Blair, seeing the drawing
instead as symbolizing an ineffective politician or, more generally, the weak
186 Elisabeth El Refaie
and powerless: “So there is a big authority leader going towards the way he
wants and there are smaller people getting stuck” (F). Many of the partici-
pants said it was the “W” on the boot that allowed them to make the connec-
tion to George Bush, while a few seemed to rely more on the general context
or the Texan connotations of the cowboy boot to draw this conclusion. Even
among the eight respondents who did not make the metonymic connection
between the boot and George Bush there was still unanimous agreement
about the fact that the person symbolized by the boot was powerful and ruth-
less. Participants talked about him having the ability to “go where he likes”
or “do as he likes” and his actions were described as “standing,” “stepping,”
“crushing,” or “trampling on” someone less powerful: “sort of don’t care
who he steps on” (M). Thus, the metaphorical link between size and power,
status, or force was understood in a similar way by everybody, although the
degree to which the boot was seen as ferocious and sadistic seemed to de-
pend on whether or not Blair and Bush were recognized and the degree to
which the respondent identified with the figure squashed under the boot.
Clearly, only the eleven respondents who recognized the references to
both politicians were able to read the cartoon in the way it was intended by
the cartoonist, namely as a comment on the relationship between the two
men and their respective countries:
R Tony Blair is basically just (.) he’s following George Bush round like
a lost sheep (.) even though that Tony Blair does have his own (.) he
wants to do his own things (.) he’s like scared that (.) he won’t let go
‘cause he’s scared of losing America (.) and that’s why I think that
Tony Blair he just isn’t a suitable leader
This excerpt also demonstrates that, when discussing the meaning of a par-
ticular metaphor, respondents often introduced new metaphors or similes to
express their thoughts. The description of Tony Blair following Bush around
“like a lost sheep” evokes a very different image from the one represented in
the cartoon and shifts the focus from the actions of the US President to those
of the British Prime Minister.
The young people struggled to recognize other aspects of the cartoon im-
age as well. For instance, most of our interviewees read out the place names
on the signposts but were unable to make sense of the little drawings in the
background. The wall behind the signpost pointing to the left was described
by six respondents as a wall or fence, while four students realized that the
drawing of a power station on the other side had something to do with nu-
clear energy or nuclear weapons. The latter drawing reminded two inter-
viewees of the oil industry, while others described it as “a crane, buildings,
Metaphor in political cartoons 187
satellites and communication” (B), “a building with a big ball on top of it”
(N) and “a door of a mosque and buildings” (L). Only two of the respon-
dents (K and U) were sufficiently well informed about the related geopoliti-
cal issues to be able to refer to these in their interpretation of the cartoon. In
Ritchie’s (2004) terms, most of the young people did not have these facts
stored in their long-term memory, or at least they were not able to access
them when cued by the cartoon image. Unsurprisingly, this influenced their
readings of the two metonyms WALL IS ISRAEL-PALESTINE CONFLICT and
POWER STATION IS NUCLEAR ISSUE.
21 of the participants were also not able to name either the vulture or the
dove (or both), referring to them instead as “a big bird” (T), a “turkey” (J),
an “eagle” (K), and as “a normal (smaller) bird” (J, T) or “pigeon” (P) re-
spectively. With some respondents, it was simply that they could not think of
the correct term, but in many other instances the students were apparently
not aware of the conventional meanings associated with these symbols.
However, for these respondents the size and other physical features of the
birds nevertheless seemed to act as powerful metaphors for particular atti-
tudes and characteristics. Respondent S, for instance, described the bird on
the left as “homeless” and the one on the right as “selfish, greedy.”
As pointed out above, this lack of political background knowledge and
familiarity with common cultural symbols did not prevent the vast majority
of our respondents from understanding that the fork in the road represented a
choice of future actions according to the “source-path-goal” schema. This
would seem to support a tentative conclusion that basic orientational meta-
phors are generally quite widely and easily understood, particularly if they
are represented through a very conventional visual image, like the one in
figure 1. By contrast, the interpretation of metaphors of a more elaborate
structural nature (e.g. FUTURE US FOREIGN POLICY IS BOOT CRUSHING
OTHER COUNTRIES) requires readers to use any verbal tags, pictorial sym-
bols and/or metonyms to assemble more precise information about target and
source and to apply the general schema to a specific social or political sce-
nario.
The cartoon by Nicholas Garland (figure 2) was again recognized by al-
most all the participants to be a narrative image, and spatial relations were
used to comprehend the sequence of events. However, in this case an under-
standing of the nature of future action was more tightly connected to grasp-
ing the implications of the inscription (“four more years”) on the matchbox.
23 out of 25 respondents understood that the matches had something to
do with the future, but this interpretation was generally not forthcoming until
after the inscription had been noted and read out in the course of the inter-
188 Elisabeth El Refaie
views. Only respondent X seemed not to understand that the “four more
years” referred to the future: “the matchbox (.) four years (.) is that how
long he has been president?” Another young person was unsure about how to
read the sequence of events: “it looks like he is going for the matches but
there’s already a fire lit (.) no idea what that’s supposed to mean” (H). Five
of the participants explicitly made the connection between this inscription
and the length of a Presidential term in office in the US, with one young man
drawing attention to the number of matches lying on the ground: “there are
four matches out representing four more years” (U). The great majority (18)
seemed to grasp from the context that the inscription must refer to George
Bush’s re-election, even though they did not explicitly refer to the length of
the US Presidential term and may not have been aware of this fact.
As I discussed in section 1, the more elaborate structural metaphor BUSH
IS TODDLER is a monomodal metaphor of the visual variety, which is com-
pletely independent of the verbal mode. However, it does require the reader
to identify the caricature of the American President. Six people did not rec-
ognize George Bush as the target of this metaphor, in which case they tended
to read the cartoon in a rather literal way, as a depiction of the threatening
nature of fire. One young woman, for instance, was clearly worried by the
picture of a baby in extreme danger and speculated that the cartoon was
“trying to convey a message ‘do not leave matches out for little babies’” (E).
In this context, the interviewee was puzzled by the inscription on the box,
speculating that it might refer to the age of the child or to the longevity of a
particular brand of quality matches:
KH have a look at the box of matches (.) what do you think four more
years means?
E never actually noticed that before (.) four more years four more years
(.) it could have a double meaning four more years (.) unless a child is
four years old or could live for few more years rather than putting the
child in danger (.) or it could be branding matches could last a long
time
In addition to the respondents who did not recognize the US President, 12 of
the participants did not seem to see the figure in the drawing as childlike at
all, which meant that the source of this metaphor was also not perceived in
the same way by everyone. In fact, one participant described the figure as
“an old woman” (F) and another as “a gentleman”:
B okay there’s an image of a gentleman crawling towards a box of
matches which says ‘four more years’ on them (.) so I’m guessing it’s
to do with the election (.) there’s smoke and fire in the background
Metaphor in political cartoons 189
While most people saw the fire as a metaphor for general problems, con-
flict, war and destruction, or more specifically the invasion of Iraq or 9/11,
respondent A interpreted the fire in the background as a reference to the
difficulties George Bush had to face in order to be re-elected: “he has been
through hell to get four more years.” Participant S, by contrast, thought the
figure crawling towards the match box looked like John Kerry and saw the
matches as representing his being under stress: “he looks so pressurized
cause he’s got a flame up the back and he’s trying to reach to the matches
cause that shows that he’s got tension (.) and he’s got a lot of stress due to
the voting four more years to work (.) he has to organize it present it every-
thing so it all goes well.” Although most of the interviewees did make a con-
nection between the matches, fire and danger of some sort, only four explic-
itly mentioned the idea that children should not be allowed to play with
matches and that “if you play with matches you get burnt” (U).
Notes
has prepared” (Gombrich 1971: 128; cf. El Refaie 2009). None of the par-
ticipants in our study referred to this expression.
4. The other three cartoons used in the study were taken from the Guardian, the
Independent and the Daily Mail respectively.
References
1999 Philosophy in the Flesh: The Embodied Mind and its Challenge to
Western Thought. New York: Basic Books.
Messaris, Paul
1994 Visual Literacy: Image, Mind, and Reality. Boulder, CO/Oxford:
Westview.
1997 Visual Persuasion: The Role of Images in Advertising. Thousand
Oaks, CA: Sage.
Morrison, Susan S.
1992 The feminization of the German Democratic Republic in political
cartoons 1989–90. The Journal of Popular Culture 25 (4): 35–52.
Mumford, Alan
2001 Stabbed in the Front: Post-War General Elections Through Politi-
cal Cartoons. Canterbury: Centre for the Study of Cartoons and
Caricature.
Philippe, Robert
1982 Political Graphics: Art as Weapon. Oxford: Phaidon Press.
Phillips, Barbara J.
2003 Understanding visual metaphor in advertising. In Persuasive Im-
agery: A Consumer Response Perspective, Linda M. Scott and Ra-
jeev Batra (eds.), 297–310. Mahwah, NJ/London: Lawrence Erl-
baum.
Proctor, Tony, Stella Proctor, and Ioanna Papasolomou
2005 Visualizing the metaphor. Journal of Marketing Communications
11 (1): 55–72.
Ritchie, David
2004 Metaphors in conversational context: Toward a connectivity theory
of metaphor interpretation. Metaphor and Symbol 19: 265–287.
Schirato, Tony, and Jen Webb
2004 Understanding the Visual. London/Thousand Oaks/New Delhi:
Sage.
Scott, Linda M.
1994 Images in advertising: The need for a theory of visual rhetoric.
Journal of Consumer Research 21 (2): 252–274.
Seitz, Jay A.
1998 Nonverbal metaphor: A review of theories and evidence. Genetic,
Social and General Psychology Monographs 124 (1): 95–119.
Talebinejad, M. Reza, and H. Vahid Dastjerdi
2005 A cross-cultural study of animal metaphors: When owls are not
wise! Metaphor and Symbol 20: 133–150.
Templin, Charlotte
1999 Hillary Clinton as threat to gender norms: Cartoon images of the
First Lady. Journal of Communication Inquiry 23 (1): 20–36.
196 Elisabeth El Refaie
Turner, Mark
1996 The Literary Mind. Oxford: Oxford University Press.
Worth, Sol
1981 Studying Visual Communication. Philadelphia: University of Penn-
sylvania Press.
Zakia, Richard D.
2002 Perception and Imaging. Boston/Oxford: Focal Press.
Chapter 9
Image alignment in multimodal metaphor
Norman Y. Teng
Abstract
This chapter focuses on how image alignment as a design strategy figures in the
construction of multimodal metaphors. Six editorial cartoons from The Christian
Science Monitor are used as illustrative examples. Image alignment can take
many forms. It can be linear, curvilinear, or exhibit a two-dimensional pattern. It
works by making some constituent components of the alignment salient, surpris-
ing, evocative, or otherwise noticeable, or by making the shape of the overall
alignment conspicuous and unexpected. Sometimes it is only implicitly involved
in a design choice. How non-pictorial elements in a multimodal metaphor interact
with the aligned pictorial components is explained by concrete examples. As to
the conceptual basis for image alignment as a design strategy, a tentative thesis is
put forward for future research: image alignment renders the abstract concept
SIMILARITY visible on the basis of the experiential correlation that motivates the
primary metaphor SIMILARITY IS ALIGNMENT.
1. Introduction
Figure 1. The Horror Show, by Clay Bennett, The Christian Science Monitor, May
13, 2004, page 8. © 2004 The Christian Science Monitor
(www.csmonitor.com). All rights reserved. Reprinted with permission.
Multimodal metaphor is a newly formed research topic proposed and ex-
plored by Forceville (2006/this volume, 2008). This newly defined research
topic incorporates Lakoff and Johnson’s (1980a, b) insight that the occur-
rence of metaphors is not restricted to language. It directs attention to the
Image alignment in multimodal metaphor 199
format, this metaphor can be labeled PEACE IN THE MIDDLE EAST IS AN EN-
DANGERED SPECIES.
Figure 4. Call to Prayer, by Clay Bennett, The Christian Science Monitor, Sep-
tember 24, 2001, page 8. © 2002 The Christian Science Monitor
(www.csmonitor.com). All rights reserved. Reprinted with permission.
Figure 5 exhibits alignment in a curvilinear form. In figure 5, two army offi-
cers are pondering possible scenarios and what action the military should
take. The American flag and the US badge on their upper arms indicate that
they are US army officers. The words “war on terrorism” on the door win-
dow depicted in the upper left corner of the picture indicate that it is military
strategies for war on terrorism that they are pondering. They seem to come
Image alignment in multimodal metaphor 203
worth noting here that alignment is defined in terms of iconic features such
as size, orientation, and distance, and, as a result, the symbolic representa-
tion in figure 3 is anchored to the alignment via the iconic function of the
pictorial components. In figure 4, the worshipper at the center of the align-
ment is in stark contrast to the rest of worshippers with respect to their
body-orientation. This is a design choice, too. All these choices make some
constituent components of the alignment salient, surprising, evocative, or
otherwise noticeable. In figure 5, the model tanks line up into a shape remi-
niscent of the mathematical symbol of infinity. This is a design choice that
makes the shape of the overall alignment conspicuous, and, probably, unex-
pected in that context. In contrast to figure 3, the symbolic representation in
figure 5 is achieved via the overall shape of the alignment, rather than a
particular aligned component. Design choices of the sort just described point
to the directions in which the readers should take their interpretations, but
they do not determine their interpretations. The above discussion of figures
2–5 shows how one may proceed to work out pertinent and well-balanced
interpretations.
each other within this spatial frame. The business person’s position defines a
viewpoint from which he sees a growing trend in the economy. The laborer’s
position defines another viewpoint from which he sees a downward trend in
the economy. Notice that the business person and the laborer in figure 6
would stand next to each other if they were aligned in a normal, spatial
frame. It is against this implicit understanding of the normal alignment that
the unconventional spatial frame and the consequent opposing viewpoints are
made possible and salient. This design gives a succinct, metaphorical ac-
count of the economic situation. The target is the economic trend; it is repre-
sented by the arrow and the word “economy.” The source is the direction
that the arrow is supposed to point in. It is represented by the arrowhead and
its spatial relations to the positions of the business person and the laborer.
Connecting the target to the source yields a metaphor that cannot but be
formulated in somewhat laborious terms: THE DIRECTION OF THE ECONOMIC
TREND IS THE DIRECTION OF THE ARROW AS IT IS VIEWED FROM EITHER THE
BUSINESS PERSON’S POSITION OR THE LABORER’S POSITION IN THE UNCON-
VENTIONAL SPATIAL FRAME. This metaphor expresses a critical stance on
the economy by reminding people of the existent alternative perspectives,
and using the unconventional spatial frame, instead of a normal spatial
frame, metaphorically suggests that the conventional assumption that a
growing economy will eventually benefit all people is not, or no longer,
valid.
Alignment implies an orderly arrangement of pictorial components, and
the examples we have encountered up to this point are effective in exposing
how an orderly arrangement of pictorial components may figure in the con-
struction of multimodal metaphors. Nonetheless, a disorderly array of picto-
rial components may also participate in the construction of multimodal meta-
phors, especially if such an array is intended for a particular effect against a
backdrop of some understood, orderly alignment. Figure 7 is a case in point.
In this cartoon, five sheets of paper are arranged in a rather untidy way.
Moreover, the four sheets in the background have been damaged – some
letters were cut out from them. The sheet in the foreground is intact, and the
letters cut out from the background sheets have been pasted on it. The scis-
sors, the glue, and the scattered shreds give further evidence of the clipping
and pasting. The symbol “SeCuRitY” thus created on the foreground sheet is
a jumble of both lower case and upper case letters, and the letters are not
lined up in an orderly way. One can tell from the context that the symbols
printed on the background sheets were “Liberty,” “Justice,” “Equality,” and
“Freedom,” but, because of the clipping and pasting, they are now in bad
shape. This design suggests the following interrelated interpretations: (a) the
206 Norman Y. Teng
cuttings metaphorically mean that liberty, justice, equality, and freedom have
been severely and dangerously compromised; (b) the clipping and pasting
metaphorically mean that liberty, justice, equality, and freedom have been
curtailed in the service of security; and (c) the jumbled form of “SeCuRitY”
metaphorically mean that security measures have been badly managed. On
top of all this, the fact that the five sheets of paper are not well arranged
suggests the metaphorical reading that issues concerning liberty, justice,
freedom, equality and security have not been handled carefully. It is worth
emphasizing that disorder need not be a poorly thought-out design choice. It
can be carefully crafted so as to suggest that an action has been performed.
The disarray in figure 7 testifies to such a design choice. It is the implied
actions, rather than the things portrayed, that figure importantly in this con-
struction of multimodal metaphor.
people probably will have no clue how to read the garbage images if the
verbal representations are removed from the picture. It is also likely that
many people will have no idea what figure 3 is intended to mean if “endan-
gered species,” “Southern Africa,” “Western China,” and “Middle East” are
deleted from it. By contrast, if the target audiences are already familiar with
the political cartoon genre, “terrorism” in figure 4 is probably dispensable
because of the prototypical depiction of a terrorist. “Economy” in figure 6 is
also likely dispensable because of the typical portrayal of a boss and a la-
borer. As to the mathematical symbol of infinity in figure 5, if the model
tanks were lined up into a shape other than the mathematical symbol for
infinity, the metaphorical meaning would be lost. Finally, the lower and up-
per case letters in figure 7 are definitely verbal elements, but they are also
pictorial components by design. In addition to the texts guiding and framing
readers’ interpretation, the garbage in figure 2, the dove-with-olive branch in
figure 3, the terrorist in figure 4, the American flag in figure 5, and the boss
and the laborer in figures 6 are all pictorial components that carry symbolic
or cultural meanings familiar in the Western world. When the target audi-
ences are conversant with the symbolic and cultural meanings, those picto-
rial components can be good choices for communicative purposes. (For an
empirical study of audience responses to political cartoons, see El Refaie,
this volume)
It is worth noting that the cartoon’s metaphors are mainly pictorial in
their mode of representation, and the verbalizations of the metaphorical
meanings can sometimes be a laborious task. Figure 6, for example, is a case
in point (see Forceville 2006: 390–392 about the implications of this phe-
nomenon for metaphor research). From a design perspective this should not
be a surprise, since words and pictures belong to different modes of repre-
sentation and are suitable for different communicative purposes. The idea of
image alignment as a design strategy discussed above offers a perspective
from which different modes of representation (here pictures and language)
can be deftly combined and coherently understood.
It is also worth noting that the patterns of image alignment discussed
above should not be taken to be instances governed by the invariance princi-
ple, one version of which runs as follows: “Metaphorical mappings preserve
the cognitive topology (that is, the image-schema structure) of the source
domain, in a way consistent with the inherent structure of the target domain”
(Lakoff 1993: 215; for a slightly different version, see Turner 1991: 172–
182; for a critical discussion, see Brugman 1990, Lakoff 1990, Turner
1990; for a discussion of how this principle fares against blending theory,
see Turner 1996: 108–109; for a discussion of how this principle should be
208 Norman Y. Teng
further revised, or abandoned, see Lakoff and Johnson 2003: 253–254). The
patterns of image alignment, despite their image-schematic and topological
nature, are not something to be preserved or overridden in the metaphorical
mappings. Or to put it differently, the idea of target domain overrides is just
not appropriate to frame the issues concerning the role of image alignment in
the construction of multimodal metaphors. Instead, it is more suitable to
consider them as constructional schemas, and the design choices described in
the previous section as elaborations of the schemas. (For a recent discussion
of constructional schemas in cognitive grammar, see Langacker 2008: 167–
174; for a discussion of elaboration, see Langacker 2008: 198–205.) This
line of thinking gives us a way to apply cognitive grammar and its usage-
based approach to design research. (I thank Francisco Ruiz de Mendoza and
an anonymous reviewer for prompting this clarification.) It also suggests the
hypothesis that primary metaphors may well serve as a common conceptual
basis for multimodal constructions. A metaphor is primary in the sense that
the association of the target with the source is directly based on an experien-
tial correlation between them (Grady 1997a: 47–48; see also Grady 1997b,
1999, 2005, Grady and Johnson 2000, Johnson 2007: 178–179, Lakoff and
Johnson 1999: 45–73). Moreover, the source and target concepts refer to
basic dimensions of experience, the shared structure of which coincides
largely with parameters relevant to the characterization of basic grammatical
categories in cognitive grammar, and the metaphorical mappings appear not
to be governed by the invariance principle. (Grady 2005: 1606–1607).
One entry in Grady’s (1997a: 281–299) list of primary metaphors is
SIMILARITY IS ALIGNMENT. The linguistic examples and the experiential
motivation for this metaphor are given below:
Motivation:
Objects may be oriented in the same way because they serve similar func-
tions, are involved in similar processes or acted on by similar forces.
And/or, orientation is a basic parameter for perceptual/cognitive classifi-
cation.
Examples:
Her new dress is very much in line with those worn by her co-workers.
There are stunning parallels between these two novels (Grady 1997a:
283).
Let us suppose that Grady is correct about this metaphor (and I think he is).
One may ask what is the relationship between this metaphor and the image
alignment discussed in this chapter. My guess is that this metaphor provides
Image alignment in multimodal metaphor 209
Acknowledgments
References
Barthes, Roland
1985 [1964] Rhetoric of the image. [Trans. by Richard Howard] In The Re-
sponsibility of Forms: Critical Essays on Music, Art, and Represen-
tation, 21–40. New York: Hill and Wang.
Brugman, Claudia
1990 What is the invariance hypothesis? Cognitive Linguistics 1: 257–
266.
210 Norman Y. Teng
Forceville, Charles
1996 Pictorial Metaphor in Advertising. London/New York: Routledge.
2006/this vol. Non-verbal and multimodal metaphor in a cognitivist frame-
work: Agendas for research. In Cognitive Linguistics: Current Ap-
plications and Future Perspectives, Gitte Kristiansen, Michel
Achard, René Dirven, and Francisco Ruiz de Mendoza Ibáñez
(eds.), 379–402. Berlin/New York: Mouton de Gruyter.
2008 Metaphor in pictures and multimodal representations. In The Cam-
bridge Handbook of Metaphor and Thought, Raymond W. Gibbs, Jr.
(ed.), 462–482. Cambridge: Cambridge University Press.
Grady, Joseph E.
1997a Foundations of meaning: Primary metaphors and primary scenes.
Ph.D. diss., University of California, Berkeley.
1997b THEORIES ARE BUILDINGS revisited. Cognitive Linguistics 8: 267–
290.
1999 A typology of motivation for conceptual metaphor: Correlation vs.
resemblance. In Metaphor in Cognitive Linguistics, Gerard J. Steen
and Raymond W. Gibbs, Jr. (eds.), 79–100. Amsterdam/
Philadelphia: Benjamins.
2005 Primary metaphors as inputs to conceptual integration. Journal of
Pragmatics 37: 1595–1614.
Grady, Joseph E., and Christopher Johnson
2000 Converging evidence for the notions of subscene and primary
scene. In Metaphor and Metonymy in Comparison and Contrast,
René Dirven and Ralf Pörings (eds.), 533–554. Berlin/New York:
Mouton de Gruyter.
Johnson, Mark
2007 The Meaning of the Body: Aesthetics of Human Understanding.
Chicago: University of Chicago Press.
Lakoff, George
1990 The invariance hypothesis: Is abstract reason based on image-
schemas? Cognitive Linguistics 1: 39–74.
1993 The contemporary theory of metaphor. In Metaphor and Thought
2nd ed., Andrew Ortony (ed.), 202–251. Cambridge: Cambridge
University Press.
Lakoff, George, and Mark Johnson
1980a Metaphors We Live By. Chicago: University of Chicago Press.
1980b Conceptual metaphor in everyday language. Journal of Philosophy
77: 453–486.
1999 Philosophy in the Flesh: The Embodied Mind and Its Challenge to
Western Thought. New York: Basic Books.
2003 Metaphors We Live By (with a new Afterword). 2nd ed., Chicago:
University of Chicago Press.
Image alignment in multimodal metaphor 211
Langacker, Ronald W.
2008 Cognitive Grammar: A Basic Introduction. Oxford: Oxford
University Press.
Teng, Norman Y.
2006 Metaphor and coupling: An embodied, action-oriented perspective.
Metaphor and Symbol 21: 67–85.
Teng, Norman Y., and Sewen Sun
2002 Grouping, simile, and oxymoron in pictures: A design-based cogni-
tive approach. Metaphor and Symbol 17: 295–316.
Turner, Mark
1990 Aspects of the invariance hypothesis. Cognitive Linguistics 1: 247–
255.
1991 Reading Minds: The Study of English in the Age of Cognitive
Science. Princeton: Princeton University Press.
1996 The Literary Mind: The Origins of Thought and Language. Oxford:
Oxford University Press.
Chapter 10
Visual metaphoric conceptualization in editorial
cartoons
Abstract
Typical of metaphors in editorial cartoons is that they not only require somehow
the mapping of features from one object or domain to another, as all metaphors
do, but their interpretation also includes a critical stance towards a particular
socio-political situation, event or person. We will argue that the “full” interpreta-
tion of editorial cartoon metaphors can best be accounted for on the basis of the
combination of two cognitive interpretation strategies, i.e., schematic vs. taxo-
nomic reasoning, following the two types of source domains distinguished by
Shen (1999). In the chapter, we argue that schema-based reasoning tends to trig-
ger the rich variety of features to be mapped from source to target domain, and
that taxonomic reasoning is often the crucial trigger in interpreting the critical
stance expressed in editorial cartoons.
1. Introduction
rich communicative area. For example, Wikipedia observes that “most [edi-
torial cartoons] use visual metaphors and caricatures to explain complicated
political situations, and thus sum up a current event with a humorous pic-
ture.”1 The political cartoonist for the Daily Telegraph, Nicholas Garland
(cited in Plumb 2004: 432) tellingly portrays the force of an editorial cartoon
as being derived “from the vehicle itself which, besides caricature, requires
some or all of a mixture of caricature, metaphor, distortion, surrealism, de-
liberate misunderstanding and mockery.”
The communicative functions of editorial cartoons and advertisements
differ considerably, which makes it interesting to investigate whether and
how these differences are reflected in the metaphoric conceptualizations
found. In this chapter, we will discuss a number of building blocks which we
consider crucial in constructing a cognitively plausible interpretation of
metaphors in editorial cartoons. Section 2 briefly characterizes editorial
cartoons. In section 3, the method of selecting and defining metaphors in
editorial cartoons will be discussed. Section 4 discusses four cases of edito-
rial cartoons, representing different structural types of visual metaphor. In
section 5, we will present two interpretative ingredients which we consider
crucial in metaphoric cartoons and which enable us to account not only for
their metaphoric conceptualization, but also for their critical stance. On the
one hand, metaphoric cartoons tend to evoke scenarios and thus require a
schematic interpretation of the source domain, in which clusters of (rather
than individual) attributes or relations are mapped from source to target
domain. On the other hand, they almost exclusively propagate a critical
stance towards a certain topic, which – as we will argue – requires a cate-
gorical interpretation of the source domain. We conclude the chapter with a
discussion on the multimodal nature of editorial cartoons.
or the cold war, e.g., Becker 1996; Edwards 2001), particular countries
(such as the US, Japan or Austria, e.g., El Refaie 2003; Feldman 1995) or
particular societal issues (such as the role of women in society, e.g., Gilmar-
tin and Brunn 1998; Morrison 1992), or they study the work of a particular
cartoonist (e.g., Plumb 2004).
Although the role of metaphor is often acknowledged in these studies, and
some of them extensively explain metaphors within the pragmatic (political)
context in which they thrive (e.g., Edwards 2001; El Refaie 2003), to our
knowledge there has been no attempt to account for the generic structure of
metaphoric conceptualization as it takes shape in editorial cartoons.
As already indicated, editorial cartoons differ considerably from com-
mercial advertisements. Cartoons aim at affecting states of minds, beliefs,
points of view, and perspectives on socio-political affairs, rather than at
changing or influencing behavior. In addition, the processing and interpreta-
tion of cartoons requires a complex mix of political, cultural, historical, and
contextual knowledge. Most importantly, however, they almost invariably
express a particular critical, if not radically negative stance towards the
topic. Whereas commercial ads intend to evoke positive attitudes and feel-
ings with respect to the target, the reverse often holds for editorial cartoons.
One goal of this paper is to demonstrate the necessity of making this point of
view explicit in order to properly analyze the way metaphors operate in car-
toons (cf. Groarke 2002).
us that this entity should be identified as the target. The ad thus expresses
the message that the product is the “life-blood of one’s car” (Forceville
1996: 159). The viewer/reader is invited to map positively oriented elements
like professional care and expert treatment from source to target. The con-
text of medical care is attributed to the maintenance of one’s car: MOBIL OIL
IS TO A CAR WHAT AN INTRAVENOUS DRIP IS TO THE HUMAN BODY.
Figure 2. The Mobil oil cartoon (Bernhardt Willem Holtrop, De Nieuwe Linie,
1977)
Now look at figure 2: an editorial cartoon drawn by the Dutch artist Bern-
hardt Willem Holtrop (“Willem”), which was published in the magazine De
Nieuwe Linie in 1977. The cartoon shows us Ian Smith, leader of the former
Rhodesian apartheid regime (presently Zimbabwe). Smith is portrayed in
quite a deplorable physical condition, sitting in a wheelchair and connected
to several bottles of intravenous drips. On these bottles the names of various
Dutch and multinational companies are shown, amongst others Mobil. We
can paraphrase the metaphor in this cartoon as INTERNATIONAL SUPPORT OF
SMITH IS INTRAVENOUS DRIP, or, specified to Mobil as one of the support-
ers: MOBIL IS INTRAVENOUS DRIP, i.e., Mobil’s support of the Smith-regime
is to that regime what an intravenous drip is to the human body. Now sup-
pose a viewer, presented with this image, interprets it by mapping the same
elements to the target object as intended by the Mobil oil ad. That is, s/he
infers that Mobil oil is like a doctor to the Smith regime, providing profes-
sional care and expert treatment. S/he would then clearly misunderstand the
essence of this cartoon. The central tenet of the message seems to be that the
218 Joost Schilperoord and Alfons Maes
Rhodesian apartheid regime can only survive by (an overkill of) external
treatment, as it is no longer able to stand on its own, and to take care of its
existence itself. In other words, whereas the Mobil advertisement in figure 1
aims at cross domain mappings that should lead to a favorable attitude to-
wards Mobil, figure 2 aims at precisely the opposite. In our view, this short
analysis suggests a systematic interpretative difference between editorial
cartoons and commercial advertisements and brings to light that the same
visual metaphor serves widely distinct purposes in these two communicative
genres.
We used two main sources to collect the sample of editorial cartoons that we
discuss in this chapter. Most of the cartoons were taken from collections of
cartoons published as separate books (containing work) by various famous
Dutch cartoon artists, such as Fritz Behrendt, Opland, Willem, Tom
Janssen, Jos Collignon and Peter van Straaten. Our decision to focus mainly
on Dutch cartoons is practical as it offers the best guarantee that we are able
to understand as precisely as possible the public events and persons the car-
toons refer to. In addition, we consulted a number of websites with collec-
tions of (international) editorial cartoons.
In collecting cartoons, we did not systematically control cartoons per art-
ist, time period, publication medium or type of public event satirized, as we
are not interested in (representative analyses of) particular periods, artists or
event types. Rather, we intend to characterize the dominant generic structure
of metaphoric conceptualizations in editorial cartoons and to analyze the
way visual metaphors are employed to make a point or express a viewpoint.
Therefore, our main criterion in selecting cartoons is that they should contain
a visual metaphor, or a combination of visual and verbal metaphor.2 That is,
the image is to refer to (at least) two domains X and Y that have a meta-
phorical relationship (rather than a literal analogy) X IS Y, where one domain
(X) serves as target and the other (Y) as vehicle.3 This way, we sampled a
corpus of 117 Dutch and 27 international cartoons, large enough to make
observations and cautious hypotheses about representative characteristic
features and generalizations. Many of the selected cartoons obviously con-
tain more than just a metaphor. For example, many also contain caricatures
(Bush-as-elephant, Russia-as-bear) or other rhetorical figures, such as hy-
perboles. However, our analyses will primarily focus on metaphorical con-
ceptualizations of target domains.
Visual metaphoric conceptualization in editorial cartoons 219
– We determined the way in which the two domains are realized. Fol-
lowing Forceville (1996), Philips and McQuarrie (2004) and Teng
and Sun (2002), we tried to distinguish cases in which only one do-
main is expressed visually (so called replacements), cases in which
the two domains are expressed separately (i.e., juxtapositions), and
cases in which the two domains are visually integrated (i.e., fu-
sions).
– Following the practice of Groarke (2002) in analyzing the argumen-
tative structure of editorial cartoons, we reconstructed as objectively
as possible the point of view expressed.
In this section we will illustrate these analytical steps by discussing four
cartoons. We will show that the three structural classes are relevant in de-
tecting relations and attributes to be mapped and argue that cartoons need an
additional interpretation step in which the critical point of view is extracted.
we repeat last year’s summer broadcasting programs). Note that the two
objects DOG and TV-SET are placed next to each other, and this way of
grouping them invites readers to solve the cognitive dissonance caused by
the apparent differences between the objects and to look for relevant connec-
tions between the objects (cf. Teng and Sun 2002). Distilling the critical
comment in this cartoon requires some contextual knowledge about the two
objects. First, summer broadcasting in the Netherlands (and elsewhere) is
often criticized for mainly repeating old programs. Second, many people
have the deplorable habit of going on holiday unwilling to take proper care
of their pets. Pets are often left on their own in just the way this cartoon
depicts. This knowledge enables us to identify the metaphorical objects as
SUMMER TV PROGRAMMING and LEFT-BEHIND PET, and to determine the
crucial element which connects the two objects: they represent comparable
examples of contemptuous human behavior. Although the phenomena are
equally deplorable, only the image of the dog refers literary to a real-world
state of affairs. We may conclude that this should be the source object, and
that the metaphorical structure can be rephrased as SUMMER TV PROGRAM-
MING IS (LIKE) LEFT BEHIND PET. By metaphorically grouping these objects,
the artist thus expresses the idea that Dutch broadcast organizations who are
responsible for summer TV programming relate to their audience the way
dog owners who leave behind their dogs relate to their pets.
phor as CUBA IS LITTLE BAD BOY, whereas the stars on the sleeve of the
pointing figure tell us the second metaphor is AMERICA IS ANGRY TEACHER.
What is special about this cartoon is that the two metaphors together trigger
a scenario or scheme, as is in fact the case with a large number, if not all of
the cartoons in our corpus (see section 5). In this case the cartoon not only
maps attributes of bad boys and angry teachers onto the target domains
Cuba and America respectively, but it also maps certain typical roles. As
such the cartoon expresses the idea that Cuba relates to America like a little
bad boy relates to his angry teacher.
The scenario contains several elements: persons (staff, patients), roles (doc-
tor, patient), relations (doctors and medical instruments) and objects (in-
struments, a drip), and attributes (being sick). Hence, with respect to the
topic of this cartoon the scenario serves as a supplier of various metaphori-
cal relations:
WALESA IS DOCTOR
JARUZELSKI IS DOCTOR
MARX IS DOCTOR
POLAND IS PATIENT
DAS KAPITAL IS MEDICINE
“SOLIDARNOSC” IS MEDICINE
POLITICAL DOMAIN/PUBLIC AREA IS HOSPITAL ROOM
Not only do scenario elements serve as source objects, but the scenario also
imposes an additional conceptual structure onto the image by virtue of the
conceptual relations that hold between the scenario elements. According to
Lakoff and Johnson (1980), scenario metaphors can be seen as Idealized
Conceptual Models, i.e., cognitive networks with causal, temporal and other
sorts of relationships between persons, roles, locations and attributes which
all are more or less fixed, and conventionally known by all members of the
cultural community in which they appear.
A more fine-grained account of this type of metaphor can be found in
Shen’s (1999) notion of schematic source domain. With this notion, Shen
provides an explanation why, in processing metaphorical messages, certain
features are mapped from source to targets in preference to others. For in-
stance, as also noted by Gentner et al. (2001), in comprehending a metaphor
like plant stems are drinking straws, people tend not to map properties like
tubular or hollow from source to target. Rather, their understanding of the
metaphor is likely to involve the notion of a drinking device, and the relation
between the person/object that drinks and the liquid being drunk. Shen ac-
counts for this preference by proposing that the source domain of the drink-
ing straw evokes a schema that not only contains the object itself, but also
entities such as the person using the straw, the liquid being drunk, the recep-
tacle, and more in general the affordances, experiences, and functions asso-
ciated with these objects grouped together in this particular constellation.
Besides sources-as-schemas, he also distinguishes sources-as-taxonomic
categories. We return to this type of source domains in the next section
where we attempt to account for the critical stance of editorial cartoons.
226 Joost Schilperoord and Alfons Maes
MINISTER IS COOK
POLICY IS DISH
MEASURE IS INGREDIENT
MEASURE IS COOKING EQUIPMENT
Visual metaphoric conceptualization in editorial cartoons 227
Once the schema is triggered, it is not difficult to infer other metaphors such
as PEOPLE ARE CONSUMERS OF DISH.
Other frequently encountered metaphorical scenarios in our corpus are
marriage, funerals, and (boxing) games. The marriage scenario is often em-
ployed to portray coalitions or political treaties. The sports game scenario is
well suited to metaphorically depict political opponents or election cam-
paigns. An example is figure 9, published during the 2004 US presidential
election campaign, depicting George Walker Bush and John Kerry as boxers
in a boxing ring.
main is the snail, often employed to express the opinion that some measure
or action has been undertaken too late, or comes too slowly.
ing medical scheme: by using Mobil oil a car owner takes care of his car like
a doctor takes care of his patient. The medical schema evokes entities like
doctor, medicine and patient, and the interpretation of the metaphor is based
on the standard relations of contiguity between these entities in the medical
scheme: actor, patient, instrument, and so on. Obviously, a drip may also be
seen as member of the category of things used to treat sick patients, and even
as a member with a high diagnostic value, but this adds little to what the
viewer already knows, namely that he is to form positive attitudes towards
the advertised product. Now consider figure 2. This cartoon triggers exactly
the same medical schema. But in order for the viewer to distill the critical
tenet of the cartoon, additional interpretation is required, otherwise, s/he
would simply feel invited to understand the international support of the
Smith regime in terms of medical care. In our view, this additional interpre-
tation is critically dependent on categorical reasoning about the medical
source domain. In particular, the medical care domain has to be viewed as a
member of the set of external forces or assistance from outside. With re-
spect to the target domain (political leaders or regimes), this categorical
interpretation has a rich diagnostic potential. Political leaders or statesmen
are good to the extent that they are supported by internal forces and assis-
tance from inside, for example their own governments and people. Con-
versely, political leaders who can only stay in power due to external support
are considered bad. Hence, the diagnostic potential of the categorical inter-
pretation of medical care as a member of the set of external assistance offers
the crucial trigger for the critical point the cartoon attempts to make (i.e.,
international support for the Smith regime is “bad”). Note that pictorial
details are in line with this interpretation (e.g., the facial expression and
posture of Smith, the relative large size of the drips, and the visual perspec-
tive of the dangling wheelchair).
A similar categorical interpretation can explain the critical position ex-
pressed in the Poland cartoon (figure 7). A strictly schematic interpretation
(as provided in section 5.1) actually does little more than signifying the state
of affairs in Poland in 1981: Walesa, Jaruzelski, and Marx relate to Poland
like a doctor relates to a patient. The critical tenet of the cartoon, however,
rests on a multiple categorical interpretation of the source domain element
doctor. Two opposite diagnostic properties of the class of doctors (or the
superset of caregivers) are crucial. A doctor can either be a member of the
set of caregivers who help and cure other people or he can be a member of
the set of caregivers who mistreat patients and destroy their lives. So who
belongs to which category in this cartoon? Our knowledge of the political
circumstances in Poland by the time, and the political points of view of the
230 Joost Schilperoord and Alfons Maes
cartoonist will lead us to infer that Walesa is the good doctor. But this same
conclusion is also warranted by pictorial elements, especially by the different
facial expressions of the three doctors. So, the intended critical interpreta-
tion, we believe, is carried by the diagnostic potential of the categorical in-
terpretation of the source domain element doctor. Furthermore, metaphorical
entailment allows us to make the following inference: Because good doctors
have good medicines, and bad doctors have bad medicines, Solidarnosc is a
good medicine, and Das Kapital a bad one.
Similar observations can be made with regard to the cooking source do-
main. The cartoon in figure 8 rests on a cooking schema to the extent that it
clarifies how this minister relates to his policy and the people affected by it.
However, only a categorical interpretation evokes the diagnostic property
which reveals the critical stance of the cartoon: the cook is to be seen as a
member of what can be termed the set of bad cooks, i.e., cooks who are not
up to the task of organizing their work in the kitchen and preparing a proper
meal. This categorical interpretation is supported by a number of visual
details, such as the messy kitchen, or the black smoke, and reveals the rele-
vant properties to be mapped to the target domain.
In sum, it is evident that factual pragmatic knowledge is an essential con-
dition to come up with these fine grained interpretations of cartoons. But the
point that we want to pursue is that a generic cognitive mechanism (i.e., set-
member classification) is at work which accounts for the evaluative aspect of
visual metaphor in cartoons as it is triggered by pictorial details. We are
only beginning to understand the kind of pictorial devices that can be inter-
preted in terms of point of view. Bodily expressions and the scaling (i.e., the
relative size) of objects are good candidates. For example, the bad cook idea
in figure 8 seems to rely on the relative size of the cook and his cooking pot,
in that the small cook cannot be supposed to master his much too large cook-
ing pot. Figure 9 is another example. At first sight, the cartoon just seems to
portray the presidential election as a boxing game with Bush and Kerry as
boxers in the ring, and thus to require only a schematic interpretation. But a
closer look reveals a bigger glove on Kerry’s right hand with the word “Iraq”
projected on it. This enlarged glove suggests a categorical interpretation, i.e.,
that it is a member of the set of decisive weapons, which results in the inter-
pretation that Kerry has the better weapon; he can attack Bush with the war
in Iraq and therefore he is likely to win the elections (or at least, he himself
believes this to be so). So, again, the gist of the message, and the critical
position of the author is accounted for by a categorical aspect of the source
domain, highlighted by pictorial details.
Visual metaphoric conceptualization in editorial cartoons 231
6. Conclusion
In this chapter we have argued that most schematic source domains of meta-
phors in editorial cartoons require an additional categorical source domain
interpretation, which results in the detection of the critical stance which is
typical of editorial cartoons as a genre. The ultimate message a cartoon
communicates is criticism, or a particular political stance towards a certain
public topic, and this is, we believe, best accounted for by the categorical
aspect of the source domain
dogs), the cartoon expresses the message that in this case the cure is at least
as harmful as the pain. In addition, by depicting the owner of the anti-
terrorism dog as much smaller than his dog, Donner is presented as someone
who is not capable of mastering his own dog (let alone the other one). Cer-
tainly, there is a message involved in this cartoon (“with his new antiterror-
ism policy, the minister is playing with fire”), but much more important here
is the persuasive impact of the message form (sameness, big versus small)
and the conclusions we have to draw from it. These are determined by the
rhetorical choices, especially the strategic choice of the source domain, and
the visual details providing us with the conceptualization of the problem and
the critical position towards it. That way, we think that the interpretation of
metaphors in editorial cartoons nicely illustrates the pervasive force of mes-
sage form in establishing meaning.
In the preceding, we did not explicitly address the issue whether metaphors
in editorial cartoons are a multimodal phenomenon. El Refaie (this volume)
does address this issue. She contends that in the cartoons she has analyzed,
the viewer should probably succeed in identifying the target objects from the
caricatured features of the politicians appearing in her cartoons. Moreover,
viewers will also draw on “broader contextual information” in doing so. In
both cartoons, she states, the verbal messages also play a role in pointing the
viewer towards the precise targets. She has in mind the verbal signs that
often appear on persons or objects depicted in a cartoon, as the names of the
multinationals in figure 2, the surname Thom and the noun new voting sys-
tem in figure 8, etc. So, if indeed these verbal signs point the reader towards
the target entities, this would suggest the following metaphors: MULTINA-
TIONALS ARE DRIPS, THOM DE GRAAF IS CHILD, NEW VOTING SYSTEM IS
COOKING POT.
To address the question whether these metaphorical relations are real
multimodal metaphors, we should first ask ourselves what multimodal meta-
phors are. Forceville (2006) states that multimodal metaphors are metaphors
whose target and source are each represented exclusively or predominantly
in different modes. His example is the cueing of the metaphor CAT IS ELE-
PHANT in an animation film. The designer could do this by depicting a cat
and have it make a trumpeting sound. In this case, the target would be trig-
gered visually, and the source by means of sound. And because vision and
234 Joost Schilperoord and Alfons Maes
sound are different modes, the resulting metaphor would be truly multimo-
dal.
If we apply this view to the cartoons in this chapter one may simply con-
clude that the above-mentioned cartoons with verbal cues are multimodal
metaphors, because visual and verbal codes are different representational
modes. But this is probably too simple. Look at the little cook in figure 8. If
the viewer misses the necessary factual knowledge and does not know who
secretary De Graaff is, the surname label (Thom) on the trousers will not be
helpful, and will not make his metaphoric conceptualization multimodal. So,
we should read and apply Forceville’s characterization in such a manner that
metaphors are multimodal only if the identification of a metaphorical term
really depends on the presence of that label. This is exactly the case in the
CAT IS ELEPHANT example. The removal of one of the modes (either the im-
age or the sound) would simply cause the metaphor to evaporate. All that
would remain is a picture of a cat or the sound of an elephant. If, on the
other hand, the proper name Thom is removed from the trousers of the cook-
ing child, this would not destroy the metaphor DE GRAAFF IS CHILD. So, it
would seem that the visual metaphors with proper noun label are not auto-
matically multimodal. All the proper name does, or is intended to do, is to
provide the viewer with an additional aid to make sure that the person is
recognized.
Not all verbal labels have the same function. Take the MEASURE/POLICY
IS COOKING POT metaphor in figure 8. At first sight, this metaphor appears
to be truly multimodal (albeit not a very exciting one), as there seems to be
only one way of identifying the source (the visual sign) and the target (the
verbal label sign on the cooking pot). Nonetheless, we believe there are at
least two reasons to doubt this characterization. The first has to do with the
metaphorical scenario itself. As soon as the viewer identifies the image as
metaphorical (an aspect of interpretation that is evidently triggered by prag-
matic considerations and knowledge of the genre) s/he would have little
trouble identifying the cooking pot as “some policy,” e.g., on the basis of a
straightforward entailment: “if this secretary is portrayed as a cook, then the
cooking pot and the stuff he is preparing must be his policy.”
The exact identification will, of course, depend on various external
sources of knowledge, such as the time of publication and broader contextual
information, for example the context of the ongoing public discussion in
news media about the new voting system. This pragmatic knowledge should
direct the viewer at the precise target (the new voting system), but not at the
type of target (policy making), since the latter is to be figured out on the
basis of the scenario evoked. And since that immediate context is evoked
Visual metaphoric conceptualization in editorial cartoons 235
Notes
References
Becker, Jonathan A.
1996 A disappearing enemy: The image of the United States in Soviet
political cartoons. Journalism and Mass Communication Quarterly
73: 609–619.
Bowdle, Brian F., and Dedre Gentner
2005 The career of metaphor. Psychological Review 112: 193–215.
Cienki, Alan, and Cornelia Müller (eds.)
2008 Metaphor and Gesture. Amsterdam/Philadelphia: Benjamins.
Coulson, Seana
in press What’s so funny: Conceptual blending in humorous examples. In
The poetics of cognition: Studies of Cognitive Linguistics and the
Verbal Arts, Vimala Herman, (ed.). Cambridge: Cambridge Univer-
sity Press.
Visual metaphoric conceptualization in editorial cartoons 237
2003b Visual and verbal rhetorical figures under directed processing ver-
sus incidental exposure to advertising. Journal of Consumer Re-
search 29: 579–587.
McQuarrie, Edward F., and Barbara J. Phillips
2005 Indirect persuasion in advertising. Journal of Advertising 34: 7–20.
McQuarrie, Edward F., and Barbara J. Phillips (eds.)
2008 Go figure! New Directions in Advertising Rhetoric. Armonk, NY:
ME Sharpe.
Morrison, Susan S.
1992 The feminization of the German Democratic Republic in political
cartoons 1989–90. Journal of Popular Culture 25: 35.
Mothersbaugh, David L., Bruce A. Huhmann, and George R. Franke
2002 Combinatory and separative effects of rhetorical figures on consum-
ers’ effort and focus in ad processing. Journal of Consumer Re-
search 28: 589–602.
Ortony, Andrew
1979 Beyond literal similarity. Psychological Review 86:161–180.
Phillips, Barbara J.
2000 The impact of verbal anchoring on consumer response to image ads.
Journal of Advertising 29: 15–24.
2003 Understanding visual metaphor in advertising. In Persuasive Im-
agery. A Consumer Perspective, Linda M. Scott and Rajeev Batra
(eds.), 297–310. Mahwah, NJ/London: Lawrence Erlbaum.
Phillips, Barbara J., and Edward F. McQuarrie
2002 The development, change, and transformation of rhetorical style in
magazine advertisements 1954–1999. Journal of Advertising 31: 1–
14.
2004 Beyond visual metaphor: a new typology of visual rhetoric in adver-
tising. Marketing Theory 4 (1–2): 113–136.
Plumb, Steve
2004 Politicians as superheroes: The subversion of political authority
using a pop cultural icon in the cartoons of Steve Bell. Media, Cul-
ture and Society 26: 432–439.
Scott, Linda M.
1994 Images in advertising: The need for a theory of visual rhetoric.
Journal of Consumer Research 21: 252–273.
Scott, Linda M., and Rajeev Batra (eds.)
2003 Persuasive Imagery: A Consumer Response Perspective. Mahwah,
NJ/London: Lawrence Erlbaum.
Shen, Yeshayahu
1999 Principles of metaphor interpretation and the notion of “domain”: A
proposal for a hybrid model. Journal of Pragmatics 31: 1631–1653.
240 Joost Schilperoord and Alfons Maes
Teng, Norman Y.
2006 Metaphor and coupling: An embodied, action-oriented perspective.
Metaphor and Symbol 21: 67–86.
Teng, Norman Y., and Sewen Sun
2002 Grouping, simile, and oxymoron in pictures: A design-based cogni-
tive approach. Metaphor and Symbol 17: 295–316.
Tom, Gail, and Anmarie Eves
1999 The use of rhetorical devices in advertising. Journal of Advertising
Research 39: 39–43.
Van Mulken, Margot
2003 Analyzing rhetorical devices in print advertisements. Document
Design 4: 114–128.
Van Mulken, Margot, Renske Van Enschot-van Dijk, and Hans Hoeken
2004 Puns, relevance and appreciation in advertisements. Journal of
Pragmatics 37: 707–721.
Zbikowski, Lawrence M.
2002 Conceptualizing Music: Cognitive Structure, Theory, and Analysis.
Oxford: Oxford University Press.
IV
Bart Eerden
Abstract
This chapter analyses anger in two Asterix comics albums and two Asterix
animated films. This will on the one hand yield, enrich, and qualify Forceville’s
(2005a) earlier findings on the visual representation of the Idealized Cognitive
Model of anger in the Asterix album La Zizanie and on the other hand enable
insights into medium-specific representations of the ICM of anger. In
investigating the structural part of the emotion metaphors, handbooks for comics
and animated films will be taken into account as well, thus allowing for
comparisons between theories developed in metaphor scholarship and the
practices of animation artists. The methodological framework constructed for the
analysis in this paper is applicable to (a) other emotions and (b) other pictorial
manifestations of emotions in static and moving images.
1. Introduction
A CAPTIVE ANIMAL (Kövecses 2000: 21). Over the past decade the research
of visual and multimodal metaphors within the cognitivist paradigm has
taken shape in the work of Carroll (1996), Forceville (1996, 2002, 2005b,
2006/this volume, 2008), Kennedy (1993), Khordoc (2001), Whittock
(1990), but most of this work focuses on creative rather than on structural
metaphors. Kövecses’ model, however, provides a good starting point for the
investigation of structural emotion metaphors in non-verbal and multimodal
representations.
Elaborating on findings by Kennedy (1982, 1993), Forceville (2005) ex-
amines visual representations of anger in an Asterix comic. Comics provide
good source material for research, because of their rich use of pictorial
metaphors to convey a vast array of emotions (see also Fein and Kasher
1996; Khordoc 2001). Forceville (2005) introduces various pictorial signs
such as “red face,” “spirals” and “bulging eyes” that are frequently used in
the Asterix comic to depict anger. The nature and use of these signs appear
to confirm that these are not just creative metaphors in the sense of Black
(1979) and Forceville (1996) but indeed manifestations of structural meta-
phors (like the “he spat fire” example) in the sense of Lakoff and Johnson
(1980, 1999).
Whereas Forceville confined himself to anger in one comics album, this
chapter, building on Eerden (2004), will analyze anger in two other Asterix
comics albums and two Asterix animation films. This will on the one hand
enrich and qualify Forceville’s earlier findings and on the other hand enable
insights into how yet another medium (animated film) can represent the ICM
of anger. In investigating the structural part of the emotion metaphors, hand-
books for comics and animated films will be taken into account as well, thus
allowing for checking theories from the realm of metaphor scholarship
against the practices of comics and animation artists. The methodological
framework constructed for the analysis in this paper is intended to be subse-
quently applicable to (a) other emotions and (b) other pictorial manifesta-
tions of emotions in static and moving images.
2. Non-verbal metaphor
3. Anger in comics
Before presenting the results of the analysis of anger in two Asterix comics,
it is important to look at the results of Forceville’s (2005a) earlier analysis.
Forceville’s analysis of anger in La Zizanie will be the reference point for
the current analysis. La Zizanie contains 103 angry characters, and anger is
expressed through twelve different signs (Forceville 2005a: 75–77). The
twelve “angry signs” can be further divided in two categories: pictorial runes
(ex-mouth,1 spirals around the head, smoke above a head, bold face and
jagged text-balloon lines) and indexical signs (bulging eyes, tightly closed
246 Bart Eerden
eyes, wide mouth, tightly closed mouth, red face, arm/hand position and
shaking.
Table 1. Signs of anger in percentages of total number of angry characters per
album. (Percentages in the table are rounded to whole percents. For ex-
ample, “bulging eyes” occur 47 times in La Zizanie on a total of 103 an-
gry characters. This means that in 46 (45,63) percent of all angry char-
acters in La Zizanie “bulging eyes” is present.)
Kövecses (2000, 2005) and Forceville (2006) as well as Lakoff and Turner
(1989) and Gibbs and Steen (1999) emphasize the influence of culture and
context on (the representation of) ICMs and the interpretation of source do-
mains, which is an important issue for the current analysis. After all, only
three comics albums by the same artist have been examined in this chapter.
Previous research (Kövecses 2005) shows evidence of quite fundamental
differences between for example the verbal manifestations of the Chinese
and Western container metaphor for anger. Yu (1998) describes how the
Chinese container metaphor, contrary to its Western equivalent, does not
involve a hot fluid but a gas which produces pressure on the body container
and which is not associated with heat. Shinohara and Matsunaka (2003)
demonstrate that in Japanese the metaphor ANGER IS GASTRIC CONTENTS IN
THE DIGESTIVE SYSTEM is saliently present. But within Western culture,
differences between verbal (and visual) metaphors in different languages
seem minor, suggesting that the most important conceptual metaphors of an
ICM transcend at least national/language borders.
Handbooks for comics and animated films in western culture can prove to
be helpful in further investigating the structural part of the anger representa-
tions. The results of a short survey of handbooks (Blair 1994; Maestri 1999,
2002; Thomas and Johnston 1981; Williams 2001) not only show a great
similarity between the advice proffered by the different handbooks among
themselves, but also between the handbooks and the findings on anger in the
three Asterix comics investigated. The use of, and emphasis on, the same
signs to represent anger in different examples suggests the existence of em-
bodied concepts that are, to a certain extent, cross-culturally represented
through the same signs in comics and animation.
Blair (1999) describes a number of prototypical characters and facial ex-
pressions pertaining to standard emotions. His description of the standard
anger face strikingly resembles Forceville’s description of the visualization
of anger in La Zizanie. According to Blair, a prototypical angry character
has fisted hands, clamped teeth, V-shaped heavy brows and small, black
Anger in Asterix 249
pupils located in the corner of the eyes (Blair 1999: 24, 52, 53). All these
elements appear in Forceville’s analysis as well as in the current analysis of
two new Asterix albums (“arm/hand (i),” “tight mouth” and “bulging eyes”).
My earlier claim (Eerden 2004) about the existence of “strong” signs
which prototypically represent a certain emotion, and “weak” signs which
can only represent a certain emotion in combination with other signs, is con-
sistent with Thomas and Johnston’s (1981) views. They also emphasize that
emotions more often than not are represented by a certain combination of
signs. While a combination of signs visually represents an emotion, say an-
ger, an isolated sign from such a combination, or such a sign in combination
with other signs, might represent a headache (“spirals”) or a person who has
eaten spicy food (“red face”) rather than anger. According to Thomas and
Johnston, emotions are most importantly expressed via the eyes, and again
the similarities with angry eyes in the analyzed Asterix comics are evident.
As a final point, Thomas and Johnston describe how Disney Studios initially
tried to copy facial expressions from real actors. This method proved unsuc-
cessful and Disney’s designers turned back to their drawing boards to ex-
periment with and try out other ways to represent emotions visually. This
seems to confirm a metonymically motivated connection between sign and
emotion instead of a sign iconically depicting an emotion.
In digital animation, to conclude this brief survey of handbooks, the same
few pictorial signs of anger can be found. Williams (2001) and Maestri
(1999, 2002) yet again emphasize the eyes (with brows) and the mouth as
the main sites for the expression of emotions. These are among the most
differentiated signs in the Asterix albums.
Although the fact that the handbooks corroborate the theoretical findings
should not lead to sweeping conclusions, it does present data in support of
the idea that pictorial anger signs metonymically instantiate conceptual
metaphors, specifically the ICM of anger. Moreover, the handbooks present
“eyes” as the most important part of the face for expressing an emotion,
which ties in with “eyes” being the largest and most differentiated category
in all three Asterix albums under scrutiny here. Interestingly, Kövecses
(1990) gives numerous examples of eyes as the source domain of emotion
metaphors in English and Hungarian. The verbal expressions found by
Kövecses, however, do not correspond with the signs found in the Asterix
albums. Whereas verbal expressions seem to focus on the presence and kind
of emotions (“love showed in his eyes,” “I could see the fear in his eyes”),
consistent with the idea of eyes as “mirror of the soul” (Kövecses 1990:
173), visual signs, at least of anger, appear to focus on representing the in-
tensity of an emotion in, and also around, the eyes (pouches, brows, lines).
250 Bart Eerden
5. Animated films
The new indexical sign “stretched forward” returns in the expression of an-
ger in the two animated films. This sign typically appears in combination
252 Bart Eerden
with the other newly introduced sign “upright.” These signs usually appear
one after the other, expressing the build-up, followed by the release of anger.
The upright position is a manifestation of the building up of pressure in the
container, usually in combination with arms close to the body and/or
clenched fists, both associated with keeping the pressure in the container
(Forceville 2005a: 81). The “stretched forward” sign expresses the release of
pressure from the body container. “Stretched forward” usually appears in
combination with stretched arms and pointing or fisted hands. The outward
pointing arms also suggest the release of pressure. For examples of both
signs, see figures 1 and 2.5
As with “bulging eyes” the “wide mouth” sign in animated films also ap-
pears to differ from the same sign in comics because of the multimodality
(especially the presence of the sonic modality) and the motion aspect of the
animated film. Because animated film is not static, it is even better able to
build up a sign through different stages. In the Asterix comics, “a mouth
counts as ‘wide’ if at least two of the following are visibly present: (i) the
tongue; (ii) teeth; (iii) (a) line(s) running over the cheek from the nose to the
corners of the mouth” (Forceville 2005a: 76). In animated films, by contrast,
the “wide mouth” requires the presence of only one of these three features to
Anger in Asterix 253
The two animated films show a wide variety of “bulging eyes.” All signs,
however, include a V-shaped brow with pouches and frowns as already de-
scribed by Forceville in the Asterix comics album (Forceville 2005a: 75).
The medium animated film and its specific features (moving images) possi-
bly plays an important role in the variation in pictorial signs of anger. The
animated films show a greater range in the depiction of bulging eyes. The
two animated films for example have “bulging eyes” wide open but also with
one eye closed and the other wide open or all the traits of bulging eyes with
254 Bart Eerden
the eyelids still slightly closed. Bulging eyes are a manifestation of the (re-
lease of the) internal pressure of the body-container.
6.3 Shaking
This sign expresses anger in the same way as does the “shaking” sign in
comics which consists of (i) multiple superimposition of a character and/or
(ii) a non-moving character depicted as “loose from the ground” (Forceville
2005a: 76). There is, however, a difference in form. “Shaking” can appear in
animated film without the superimposition described for the comics version
of “shaking,” and in other cases the head of the character is the only shaking
part. The absence of superimposition or extra lines inevitably means that
“shaking” is only recognized in a sequence of frames, but not in a single
(freeze)frame.
“Shaking” is a manifestation of the (immense) internal pressure of the
container. The “shaking” sign appears especially at moments where sup-
pressed anger turns into an outburst or when an outburst ends. An earlier
study of pictorial signs representing love in animated films suggested that
“shaking” is also used as a sign when love “strikes” a person or when a per-
son instantly falls out of love (Eerden 2004: 53). This particular example of
the “shaking” sign is limited to the head only.
The shaking of the head can be traced back to the folk theory of emo-
tions. Kövecses (1990, 2000, 2005) describes the folk theory according to
which an emotion can be characterized as a five-stage scenario. In this cog-
nitive model the emotion affects the self as a force that causes a change of
state. As a result the self loses control and at a final stage responds to the
emotion with emotional behavior (Kövecses 2000: 58–59). The new sub-
category of “shaking,” which focuses specifically around the head, marks the
sudden “entering” of one of the final stages of an emotion. The character
instantly loses control.
6.4 Ex-mouth
In comics this sign has “straight lines emitting from the mouth” (Forceville
2005a: 77). The lines express something forcefully coming out of the mouth.
This could be explained as the release of pressure, which makes “ex-mouth”
a pictorial rune. On the other hand, the straight lines might simply represent
Anger in Asterix 255
Figure 4. Ex-mouth and the waving movement of clothes in the Roman soldier’s
cape (hand-traced still from Asterix et la Surprise de César).
6.5 Upright
As an indexical sign for anger, “upright” has an erect position of the body
with the back of the head usually pressed against the neck and the nose
pointing upwards. The upright position of the body signals the build-up of
internal pressure in the body-container.
256 Bart Eerden
QUANTITY and INTENSITY IS SPEED (Kövecses 2005: 27). Although the pri-
mary metaphors seem universal, combinations of primary metaphors vary
between different cultures, thus resulting in different complex metaphors,
which accounts for cultural variation in verbal expressions of anger. This
same phenomenon seems to be relevant for different modes of communica-
tion. I suggest that different modes of communication can account for varia-
tions in conceptual metaphors in much the same way as cultural context
does.
7. Results
Many of the signs found in the analyzed comics and films are commensurate
with the results of previous research (Forceville 2005a, Eerden 2004). The
analysis of comics and animated films reported in this chapter has also re-
sulted in the identification of some new signs of anger. Some of these oc-
curred in the two Asterix comics albums, but were not reported by Forceville
258 Bart Eerden
(2005a); others appear to be specific for the medium of animation. The ques-
tion here is whether different signs of anger downplay or highlight different
parts of the ICM of anger in verbal, comics, and animated form, respec-
tively. Further analysis is necessary to chart the conceptual metaphors which
are expressed through the newly found animated signs.
The “eyes” and “mouth” signs, followed by “arm/hand” are omnipresent
in both comics and animated films. The analysis of handbooks confirms the
important role of the three largest categories of signs and the rich variations
existing within each category (Blair 1999; Maestri 1999, 2002; Thomas and
Johnston 1981; Williams 2001). These three categories are the most used
and most differentiated signs in the comics, animated films, and handbooks
examined. But however important these signs are in visual representations,
verbal equivalents seem hard to find.
An example of the differences between verbal and visual representations
can be found in the source domains for “eyes.” The verbal expressions
largely pertain to EYES AS CONTAINER FOR EMOTIONS with the emotion
being visible in the eyes (“I could see the fear in his eyes,” “his eyes were
filled with anger,” and “love showed in his eyes”). Comics and animated
films, by contrast, are able to express not only the presence of a certain emo-
tion, but also the intensity of an emotion like anger, as well as its stage. This
again confirms the idea that complex conceptual metaphors might be con-
structed and expressed differently in different media and modes of communi-
cation.
The representation of anger in animated films results in the identification
of at least one new sign (“low angle”) which does not seem to fit in with the
central conceptual metaphor of anger (ANGER IS A HOT FLUID IN A PRESSUR-
IZED CONTAINER) as derived from verbal expressions. While all the other
signs can be explained as referring to embodied metaphors, “low angle” has
no relation to the body of the angry person, which explains why it is not
compatible with the container metaphor. Kövecses also presents verbal ex-
pressions that do not refer to the container metaphor but to other conceptual
metaphors, such as ANGER IS A DANGEROUS ANIMAL or ANGER IS AN OPPO-
NENT. However none of these examples seem to be related to the “low angle”
sign. The “low angle” sign originates from the unique possibilities (such as
framing and motion) of the animated medium. Also “low angle” seems to
focus more on the perspective of a “victim” experiencing or witnessing the
anger expressed by another character. When a “victim”-character is not
present in the story, it is the viewer who experiences the anger through a
virtual point-of-view-shot.
Anger in Asterix 259
Animated films and comics in general contain yet other examples of signs
conveying emotions that are not related to the body. One might think of a
light-bulb or rain cloud above someone’s head. Certainly this area of picto-
rial metaphor needs more research. Shinohara and Matsunaka (this volume)
give examples of such external signs as thunder, wind, or flowers often oc-
curring in Japanese Manga. They also give examples of background scenes
of panels which are used to express the emotional state of the character in
the panel. A similar version sometimes appears in the Asterix comics. This is
not included in the current analysis, as the signs do not seem to appear struc-
turally, but further research is important here. One such sign is “green text
balloons,” which appears in La Zizanie. Forceville labels this sign as arbi-
trary and thus excludes it from his analysis, using the sign as an independent
indicator of anger, since the green text balloons appear in over 50 percent of
the anger panels (Forceville 2005a: 75). The green text balloons seem even
more arbitrary because they do not appear in the next two albums. However,
a closer look at the emotion anger in Asterix et Latraviata and Asterix et la
Surprise de César shows many unrealistically colored backgrounds in pan-
els with angry characters. These background and text balloon signs might be
arbitrary, but in light of Shinohara and Matsunaka’s research it is possible
that these signs actually express certain conceptual metaphors.
The results in table 1 present “hand/arm” and “eyes” as the largest two
categories of signs, followed by “mouth” signs. On average, “eyes” appears
in 85 percent of anger panels, for “hand/arm” the average is 63 percent, and
the category “mouth” can be found in 51 percent of the anger panels. The
three categories are not only the largest in numbers but also the most differ-
entiated signs, both in comics and animated films. The three signs are at the
very least commensurate with the conceptual metaphor ANGER IS A HOT
FLUID IN A PRESSURIZED CONTAINER. I would venture the stronger claim
that the container metaphor is at the center of the representation of anger in
the analyzed comics as well as the animated films.
Kövecses emphasizes the role of the container concept as central to the
concept of anger. It seems warranted to conclude that certain complex and
central embodied concepts such as CONTAINER and FORCE play a central
role in metaphors, irrespective of medium or mode. However, based on ver-
bal evidence, the concept of anger consists of a number of other important
metaphors such as ANGER IS A BURDEN, ANGER IS AN OPPONENT IN A
STRUGGLE and ANGER IS A CAPTIVE ANIMAL (Kövecses 2000: 21). These
metaphors are good examples of concepts that are not primarily based on
embodiment but seem more related to behavioral aspects. It seems that vis-
ual representations of anger focus almost entirely on embodied container
260 Bart Eerden
8. Further research
Notes
References
Black, Max
1979 More about metaphor. In Metaphor and Thought, Andrew Ortony
(ed.), 19–43. Cambridge: Cambridge University Press.
Blair, Preston
1994 Cartoon Animation. Laguna Hills: Walter Foster.
Bordwell, David, and Kristin Thompson
2004 Film Art: An Introduction. 7th ed. Boston: McGraw-Hill.
Carroll, Noel
1996 A note on film metaphor. In Theorizing the Moving Image, 212–
223. Cambridge: Cambridge University Press.
Cienki, Alan
1998 Metaphoric gestures and some of their relations to verbal meta-
phoric expressions. In Discourse and Cognition: Bridging the Gap,
Jean-Pierre Koenig (ed.), 189–204. Stanford: Center for the Study of
Language and Information.
Eerden, Bart
2004 Liefde en woede: De metaforische verbeelding van emoties in
Asterix. [Love and anger: The metaphorical visualization of emo-
tions in Asterix.] MA thesis, Department of Media Studies, Univer-
sity of Amsterdam, The Netherlands.
Fein, Ofer, and Asha Kasher
1996 How to do things with words and gestures in comics. Journal of
Pragmatics 26: 793–808.
Forceville, Charles
1996 Pictorial Metaphor in Advertising. London/New York: Routledge.
1999 The metaphor COLIN IS A CHILD in Ian McEwan’s, Harold Pinter’s,
and Paul Schrader’s The Comfort of Strangers. Metaphor and Sym-
bol 14: 179–98.
2002 The identification of target and source in pictorial metaphors. Jour-
nal of Pragmatics 34: 1–14.
Anger in Asterix 263
1999 Philosophy in the Flesh: The Embodied Mind and its Challenge to
Western Thought. New York: Basic Books.
Lakoff, George, and Mark Turner
1989 More than Cool Reason: A Field Guide to Poetic Metaphor. Chi-
cago: University of Chicago Press.
Maestri, George
1999 Digital Character Animation: Volume 1 Essential Techniques. Indi-
ana: New Riders.
2002 Digital Character Animation: Volume 2 Advanced Techniques.
Indiana: New Riders.
McCloud, Scott
1994 Understanding Comics: The Invisible Art. New York: HarperPeren-
nial.
Shinohara, Kazuko, and Yoshihiro Matsunaka
2003 An analysis of Japanese emotion metaphors. Kotoba to Ningen:
Journal of Yokohama Linguistic Circle 4: 1–18.
Thomas, Frank, and Ollie Johnston
1981 Disney Animation: The Illusion of Light. New York: Abberville
Press.
Uderzo, Albert
2001 Astérix et Latraviata. Paris: Les Editions Albert René.
Whittock, Trevor
1990 Metaphor and Film. Cambridge: Cambridge University Press.
Williams, Richard
2001 The Animator’s Survival Kit. London: Faber and Faber.
Yu, Ning
1998 The Contemporary Theory of Metaphor: A Perspective from Chi-
nese. Amsterdam/Philadelphia: Benjamins.
Chapter 12
Pictorial metaphors of emotion in Japanese comics
Abstract
1. Introduction
language, CMT claims that “metaphor is pervasive in everyday life, not just
in language but in thought and action,” and that “[o]ur ordinary conceptual
system, in terms of which we both think and act, is fundamentally meta-
phorical in nature” (Lakoff and Johnson 1980: 3). In CMT, a conceptual
metaphor is defined as the systematic mapping between two different con-
ceptual domains, and it is assumed that surface linguistic expressions of
such a metaphor emerge from that conceptual mapping. The central tenet of
CMT is that metaphor, by its very nature, not only affects surface linguistic
expressions but also characterizes cognitive/conceptual structure.
This tenet of CMT, however, has not been fully explored, although some
researchers have started to apply this idea to the study of multimodal meta-
phors. Forceville (1994) suggests that metaphors are not limited to linguistic
ones, and he maintains that “to further validate the idea that metaphors are
expressed by language, as opposed to the idea that they are necessarily
linguistic in nature, it is necessary to demonstrate that, and how, they can
occur non-verbally and multimodally as well as purely verbally” (2006:
381/this volume, emphasis in original). Thus, in recent developments of the
study of metaphor, researchers’ attention is being drawn to the new field of
multimodal metaphors. Metaphors expressed through the verbal modality
and those expressed through other modalities appear to share the same fun-
damental motivation, and thus they cannot be regarded as totally separate
phenomena. Although, of course, multimodal metaphors that are based on
nonverbal sources have unique properties that can be traced back to the na-
ture of the medium’s modalities (Forceville 1994, 1996, 1999, 2005; Carroll
1994), they share the underlying mapping between conceptual domains with
verbal metaphors. Thus, previous studies on multimodal metaphors strongly
support the argument that metaphor is a matter of concept and cognition not
limited to language.
This chapter builds upon these previous studies, especially the one by
Forceville (2005), which deals with pictorial metaphors of anger in a French
comics album. Following Forceville’s view, we try to demonstrate that at
least some manifestations of anger are found in visual no less than in verbal
metaphors. We also try to extend Forceville’s view to other types of emo-
tions such as happiness, love, anxiety, surprise, or disappointment. To attain
this goal, we examine pictorial metaphors of emotion in Japanese comics.
The source of this type of metaphor is pictorial or visual, where meanings
are conveyed via pictorial or visual representations. The target is emotion,
which belongs to a more abstract domain of psychological experience. In this
metaphor, what is expressed as a picture can be interpreted as representing
some emotion. Data are taken from some of the present day popular Japa-
Pictorial metaphors of emotion in Japanese comics 267
As these examples show, emotions such as sorrow, joy and hatred are
thought of as contents in the container (the chest). They either exert a kind of
force upon the container, which makes it tear apart or become full, or they
stay in the container in an unstable state, having a certain impact on the
container. Matsunaka and Shinohara (2001b) call this kind of force the in-
ner force, since the force affects the container from inside. In addition to
this, Matsunaka and Shinohara demonstrate that there are at least some ex-
amples of metaphors of emotions in Japanese that imply an outer force
rather than an inner force.
(4) a. kanashimi-ni uchi-nomes-areru.
sorrow-by beat-flat-Passive
“(Self) is beaten by sorrow.”
b. kanashimi-ni mune-ga shime-tsuker-areru.
sorrow-by chest-Nom. screw-put-Passive
“Chest is screwed up by sorrow.”
Pictorial metaphors of emotion in Japanese comics 269
These examples show that the metaphorical force of emotion can exist out-
side the self or the part of the body that is regarded as the container of emo-
tion. The force of emotion affects the container from the outside, making its
shape change, or even breaking it down. Though previously it was assumed
that only very intense emotions are conceptualized as outer forces such as
natural forces (Kövecses 2000: 72), it has been found that weak or mild
emotions can also be conceptualized in terms of natural or meteorological
phenomena in Japanese. The following Japanese examples illustrate that
both intense and mild emotions can be expressed in terms of natural or mete-
orological phenomena (Matsunaka and Shinohara 2001b, Shinohara and
Matsunaka 2001).
(5) a. kokoro-ni honokana hikari-ga sasu.
heart-Loc. faint light-Nom. shine
“A weak light brightens heart. (One feels relieved a little
bit.)”
b. kumo-yuki-ga ayashii.
cloud-go-Nom. strange
“The weather is getting squally. (A person is getting bad-
tempered.)”
c. kuro-kumo-ga mune-ni ooi-kabusaru.
black-cloud-Nom. chest-Loc. cover-lap
“A black cloud (=anxiety) covers chest.”
d. kimochi-ga harebaresuru.
feeling-Nom. become.fine.weather
“My heart clears up. (I feel very happy.)”
e. kokoro-wa doshaburi da.
heart-Topic heavy.rain be
“It rains heavily in (my) heart. (I am very sad.)”
270 Kazuko Shinohara and Yoshihiro Matsunaka
These wakas (Japanese old-style poems) date back to the 11th or 12th cen-
tury. Since that time, our hearts have been thought of as experiencing night,
seasons, or landscapes such as rivers. This cultural tradition of Japan may
be one of the background factors that motivate EMOTION IS EXTERNAL ME-
TEOROLOGICAL/NATURAL PHENOMENON THAT SURROUNDS THE SELF in
Japanese. If this emotion metaphor is culture-specific, then it may be due to
this type of tradition in Japanese society. However, the mappings between
natural/meteorological phenomena and emotions do not seem to be totally
arbitrary or unmotivated, but rather, they seem to be supported by the bio-
meteorological tendencies mentioned above. This tendency may not be so
universal as the rise of blood pressure and temperature during the experience
of anger and thus its physiological motivation for the conceptual metaphor of
emotion may perhaps be rather weak. Consequently, metaphors based on this
tendency may be culture-specific.
To sum up this section, we have argued that (1) the general schema of the
anger metaphor ANGER IS A HOT FLUID IN A CONTAINER is applicable to
Japanese emotion metaphors; (2) there is an elaborated distinction between
inner and outer force in Japanese emotion metaphors; (3) the source domain
of natural force is highly elaborated in Japanese emotion metaphors and
displays various levels of intensity; and (4) some Japanese emotion meta-
phors have mappings from natural or meteorological phenomena to emo-
tions, which are basically experientially motivated but allow for the possibil-
ity of culture-specificity. As the discussion proceeds, we will examine
whether these findings from a cognitive linguistic viewpoint also occur in
pictorial/visual metaphors of emotion in Japanese manga.
ward indexical signs) includes “bulging eyes,” “tightly closed eyes,” wide
mouth,” “tightly closed mouth,” “pink/red face,” “arm/hand position,” and
“shaking.” These are indexical signs since “we recognize them as symptoms
accompanying anger from our everyday experience” (Forceville 2005: 77).
Category II (pictorial runes, a term coined by Kennedy 1982) includes “spi-
rals,” “ex-mouth,” “smoke,” “bold face,” and “jagged line.” These “are not
perceptible in real life, and their indexicality is therefore less evident” than
those in Category I (2005: 77). Through the analysis of the collected exam-
ples of expressions of anger in La Zizanie, Forceville concludes, “the picto-
rial runes signaling anger appear indeed to be Peircean indices rather than
Peircean symbols, since they are motivated rather than arbitrary signs”
(2005: 82).
Here, motivatedness is an important notion. In CMT, it has been repeat-
edly argued and demonstrated that the most basic linguistic metaphors are
motivated in such a way that the motivation can be traced back to our per-
ceptual or bodily experience. One of the theses of cognitive linguistics is that
language is embodied and therefore our mind is fundamentally embodied.
The above-mentioned study of pictorial metaphors by Forceville and other
researchers along this line provide strong support for this view of mind. If
we can find additional evidence that demonstrates parallel structures between
linguistic metaphors and pictorial and multimodal metaphors, this will
strengthen the claim that metaphorical mappings are not merely a matter of
language but reside in a deeper layer in our mind, thus enabling manifesta-
tions in multimodality. Eerden (this volume) demonstrates that Forceville’s
arguments also apply to animated films. This constitutes evidence to support
this line of ideas. In our study we aim at the same goal, but use a different
kind of data. We will add evidence taken from works of Japanese comics
written by several authors.
(i) Kashimashi: Girl Meets Girl Vol. 1, by Akahori [story] and Katsura [art],
2005.
(ii) Azumanga Daiou, Vol.1, by Azuma, 2000.
(iii) Azumanga Daiou, Vol. 4, by Azuma, 2002.
(iv) Yotsubato, Vol. 5, by Azuma, 2006.
(v) Ichigo Mashimaro, Vol. 1, by Barasui, 2003.
(vi) Ichigo Mashimaro, Vol. 4, by Barasui, 2005.
(vii) Black Jack, Vol. 3, by Tezuka, 1993.
(viii) Crayon Shinchan, Vol. 1, by Usui, 1992a.
(ix) Crayon Shinchan, Vol. 3, by Usui, 1992b.
(x) Crayon Shinchan, Vol. 16, by Usui, 1997.
The first three authors (Akahori, Azuma, and Barasui) have been popular
among young people in recent years, and though originally written as comics
for girls, their readership extends beyond gender differences. Most of their
works selected for this study are about young heroines’ school and family
lives. The last two authors, Tezuka and Usui, and their pieces used in this
study, are well known long-selling comic artists for broader age groups in
Japan. Usui’s works, which describe the daily life of a kindergartener and his
parents, are more vulgar and comical than Tezuka’s serious medical drama.
274 Kazuko Shinohara and Yoshihiro Matsunaka
Figure 1. Slanted open eyes representing anger (Yotsubato vol. 5: 80, © Kiyohiko
Azuma).
A typical pictorial expression of an angry person in Japanese manga is
“slanted eyes” (figure 1) or “slanted eyebrows” (figure 2). These eyes can be
either wide open or narrow/closed. They correspond to Forceville’s “bulging
eyes” and “tightly closed eyes” respectively, but the angle (slant) seems more
noticeable than size, since half-open slanted eyes can also express anger. The
slanted shapes of eyes and eyebrows are so expressive that, as figure 1
shows, the detailed parts of eyes like the pupils are sometimes omitted but
still the reader can recognize that the person is angry. Slanted eyes and eye-
brows are sometimes accompanied by wrinkles near their inner edge or be-
Pictorial metaphors of emotion in Japanese comics 275
Figure 2. Slanted eyebrows, tightly closed eyes, and wide mouth representing
anger (Yotsubato vol. 5: 85, © Kiyohiko Azuma).
Shrunken or round pupils are also typical pictorial expressions of anger in
Japanese comics. In figure 3, shrunken round pupils are used to express the
person’s anger. A tightly closed mouth and shaking body also appear in this
picture, like the ones that Forceville (2005) found in La Zizanie. (The girl in
figure 3 has also Y-shaped signs on her forehead and cheek, which will be
analyzed in 3.2 as pictorial runes.)
Figure 3. Shrunken pupils, slanted eyes, tightly closed mouth, and shaking body
representing anger (Yotsubato vol. 5: 82, © Kiyohiko Azuma).
“Arm/hand position,” as suggested by Forceville, is also expressive of anger.
In figure 4, the girl on the left side raises her arm and clenches her fist. Even
276 Kazuko Shinohara and Yoshihiro Matsunaka
though her face is not drawn, readers can easily understand that she is angry.
This may be because the final stage of the scenario of the folk theory about
anger is that an angry person loses control of herself and vents her anger in
the act of retribution (Lakoff 1987: 397–398), and the raised hand and fist
represent this act, showing that the angry girl is about to hit, or at least
threatening to hit, the other girl.
Smoke (in other comics also often “fire”) is clearly an effect of the heating
up of the fluid or gas in the body-container….
Bold face and larger fonts [i.e., in the letters in the text balloons] can be
seen as … equivalent to saying “he spoke very, very loudly.” The large
fonts and bold face, then, cue loudness via a more generic metaphor; and
loudness is metonymically associated with (expressed) anger.
The angularity of the “jagged line connection” [i.e., linking the text bal-
loon to the speaking character] is a less obvious cue for anger, but if we
characterize it as “non-smooth” as opposed to the rounded and hence
smooth way of connecting balloon to character that is the default, we may
hypothesize that it fits in with a whole category of “tense” behaviors.
In the Japanese manga we have investigated, “spirals” were not found. In-
stead, radial straight lines are often drawn as emanating from the front of the
entire angry person, as seen in figure 5.
Figure 5. Radial straight lines representing anger (Azumanga Daiou vol. 1: 30, ©
Kiyohiko Azuma).
These straight lines may have the same function as Forceville’s “ex-mouth,”
that is, signaling loud voice or the release of pent-up pressure in the body-
278 Kazuko Shinohara and Yoshihiro Matsunaka
Figure 7. Steam representing anger (Black Jack vol. 3: 98, © Osamu Tezuka).
The small cloud-like shape and the line(s) under it represent that the angry
person is blowing off steam from the head. It seems to be “steam” rather
than “smoke,” since the same shape can be used to express steam coming out
of a boiling pan. Indeed, we have a verbal metaphor of anger using the word
for “steam” (example (2a) in Section 2.1, repeated here).
Figure 9. Fire representing anger (Crayon Shinchan vol. 1: 98, © Yoshito Usui).
We have verbal metaphorical expressions corresponding to the concept of
“fire” in Japanese as in (7). Again, the motivation for the use of “fire” is
provided by the folk theory about anger.
Being based on the physically occurring states, this sign might be classified
as an indexical signal of anger. However, the shape of the sign, especially
the cross-shape, does not coincide with the shape of the veins in the temple.
Moreover, this sign is indiscriminately used even for young girls or children,
whose temple veins are not likely to stand out. Thus, it seems reasonable to
classify this sign as a pictorial rune.
The sign of pressurized veins can be seen in most of the above figures of
manga. However, it may be noticed that the location of the sign varies. This
displacement is a very interesting phenomenon that deserves discussion. We
will examine it more closely in the next section.
As mentioned in the previous section, one of the most typical and productive
pictorial runes of anger in Japanese manga is a sign of pressurized veins.
This pictorial rune can be detached from its original (physically occurring
and perceivable) place, that is, one’s temples, and can be drawn in places
that seem to be physically impossible. For example, in figures 3 and 6, it is
drawn on the forehead and cheek. In figure 11, the signs are on the hands of
the angry mother who is pinching the boy’s face.5 In figure 12, it appears on
the neck (or the lower part of the cheek). In figure 13, four of the people
waiting in the queue have this sign on their head but three of them have no
detailed face. So it is not certain where the pressurized vein is, but we can
easily conclude that they are all angry from the narrative and the utterances.
Figure 11. Displaced sign of anger (Crayon Shinchan vol. 3: 16, © Yoshito Usui).
282 Kazuko Shinohara and Yoshihiro Matsunaka
Figure 12. Displaced sign of anger (Crayon Shinchan vol. 3: 16, © Yoshito Usui).
Figure 13. Displaced sign of anger (Crayon Shinchan vol. 1: 34, © Yoshito Usui).
In all the above cases, the signs of the pressurized veins appear on a per-
son’s face, except in the case of hands. However, this sign can also be wit-
nessed in unexpected places. In figures 4 and 8, it is on the back of the head,
or on the hair. It is definitely impossible to see pressurized veins standing out
on one’s hair. At this stage, the sign seems to have been freed from its fea-
ture of appearing on a person’s skin. In figure 5, it is in the air, just above
the head. This indicates that the sign can even become detached from the
human body. Finally, in figures 9 and 11, it is in the balloon, beside the
words, as if indicating the intonation or tone of voice with which the words
are uttered.
Thus, even in places other than temples, the sign of pressurized veins can
express anger. Though this sign can be traced back to its original status as a
metonymy, it seems that it has become a sort of independent sign represent-
ing anger and thus can enjoy free displacement and deviation of a kind that is
not seen in verbal metaphors. In verbal metaphors, example (8) implies hem-
orrhage in the brain and nowhere else in the body. To verbally express the
location of the sign such as on hands, on the back of the head, in the air, and
Pictorial metaphors of emotion in Japanese comics 283
so forth, would make no sense. It is only in the visual mode that this sign can
be displaced. It seems that the free deviation of this pictorial rune is made
possible by the visual properties of the medium in which it is used. This
suggests that pictorial metaphor is not a mere substitution for or equivalent
of verbal metaphor, though these two obviously share experiential motiva-
tions and the same cross-conceptual mappings. As Forceville (2006: 281)
maintains, “…an exclusive or predominant concentration on verbal manifes-
tations of metaphor runs the risk of blinding researchers to aspects of meta-
phor that may typically occur in multimodal representations only.” Dis-
placement of the pictorial rune of pressurized veins may be an instantiation
of the aspects that Forceville argues. It is a “visual” manifestation of a con-
ceptual metaphor of anger and its meaning cannot be fully covered by the
corresponding verbal metaphor.
So far, we have seen that there are the same or similar kinds of pictorial
runes in Japanese comics as Forceville finds in the French album La Zizanie.
They are all compatible with the verbal expression of the ANGER IS A HOT
FLUID IN A CONTAINER metaphor or its background folk theory and the sce-
nario of anger events. The pictorial runes are not straightforward expres-
sions of physically perceivable states of an angry person, so they are not
straightforward indexical signs in Peircean semiotic theory. However, as
Forceville (2005: 82) argues, they are not symbols in Peircean sense either,
since they are not arbitrary but are motivated. Considering the importance of
the notion of motivation in the study of metaphor, we agree with Forceville
in maintaining that pictorial runes should be classified as indexical signs.
For this discussion, Forceville provides examples from a French comics
album. The present study provides further supports for this claim by giving
evidence from Japanese comics. Motivated pictorial runes are not confined
to French, but are realized in at least two language communities.
Figure 14. Cloud or fog representing anger (Yotsubato vol. 5:79, © Kiyohiko
Azuma).
Figure 16. Thunder, wind, and rain representing anger (Crayon Shinchan vol. 1:
100, © Yoshito Usui).
In the background scene of figure 14, a cloud or fog-like substance is de-
scribed. Since the setting of this scene is in the living room at sunny lunch-
time, this cloud or fog cannot be a physical occurrence. It expresses the
girl’s anger, which is getting more and more intense. In figures 15 and 16,
thunder appears, which expresses that the angry person is releasing his/her
anger, and in typical cases, shouting out. Thunder may be associated with
emission of intense energy, perhaps with a loud noise, which may be meta-
phorically mapped onto the loudness of the person’s shouting voice.
Figure 16 has wind and rain besides thunder. The slant lines and the
onomatopoeia written on the left side (which reads “Gooooo”) represent
strong wind. The lines may also represent rain, together with the drops be-
tween them. They all combine to represent a “storm.” Since there is no ac-
tual storm in this scene, the storm is here a metaphorical expression of the
person’s emotion. The image of storm may be associated with intense energy
and destructive power that will affect the person nearby. It may be noticed
that this “storm” is drawn only around the mother, not around the boy. This
clearly means that the mother is very angry but the boy does not care at all.
The effect of the storm does not reach the boy. In this way, visual represen-
tation of meteorological phenomena can be used very effectively and expres-
sively in manga.
So far, we have observed meteorological or natural phenomena in picto-
rial metaphors that express the emotion of anger. These kinds of pictorial
metaphors are, however, not limited to anger. Other types of emotions like
love, happiness, surprise, disappointment, or anxiety can also be expressed
by meteorological or natural phenomena that are not physically present. The
following are some examples.
286 Kazuko Shinohara and Yoshihiro Matsunaka
Figure 17. Lightning representing surprise (Ichigo Mashimaro vol. 4: 58, © Bara-
sui).
Figure 21. Flowers representing love (Kashimashi vol.1: 152, © Yuukimaru Ka-
tsura).
Figure 22. Flowers representing love (Crayon Shinchan vol. 16: 12, © Yoshito
Usui).
288 Kazuko Shinohara and Yoshihiro Matsunaka
metaphor is subtler in this case than the case of thunder, whose presence is
very unlikely. Since birds can fly in the air, the conceptual mapping HAPPY
IS UP may motivate this use of birds.
Figure 20 represents a girl drinking beer after taking a bath. She is very
relaxed and happy. This state of emotion is expressed by surrounding flow-
ers. The relationship between happiness and flowers is obvious, since it is a
typical reaction for us to feel relaxed and happy when we see flowers. Thus,
this use of flowers is experientially motivated.
In figures 21 and 22, flowers represent love. This is also observed in ver-
bal metaphor in Japanese, in which a rich vocabulary about flowers is asso-
ciated with love. This may hold in other languages as well.
Figure 23 has a more complex structure. This pictorial metaphor consists
of two successive panels. In the first panel, the flower behind the girl has a
complete shape. However, after she hears what other persons say, which is
shown in the two balloons, one petal of the flower drops, as in the second
panel. This change of state of the flower represents the girl’s emotional state,
that is, how she became disappointed when she heard those utterances. The
petal dropping indicates incompleteness of the flower, deficiency, or lack of
something. Since flower represents a happy state, this deficiency means “de-
stroyed happiness.” It is, however, not a severely destructive experience
because the flower is still a flower even when it has lost one of the petals.
Thus, this pictorial metaphor is highly expressive of such dynamic state of
one’s emotion, and it illustrates that dynamic or complex use of visual signs
is possible in this kind of media.
To sum up, meteorological or natural phenomena used in pictorial emo-
tion metaphors we have examined are experientially motivated. Flowers can
represent happiness or love, but they do not represent anxiety or shock.
What makes pictorial metaphors of this kind comprehensible and interpret-
able resides in the motivation or grounding that supports their mappings.
Considering these properties, pictorial metaphors in Japanese manga should
be classified as Peircean indices rather than Peircean symbols.
However, it should be noted that the use of meteorological or natural
phenomena to express emotional states may be, at least to some extent, cul-
ture-specific. As already mentioned, to regard natural scenes as reflections
of one’s internal or emotional states may be an aspect of Japanese socio-
cultural tradition which may not be shared by other cultural groups. We
have argued elsewhere that experiential motivation does not imply universal-
ity. Even when a conceptual mapping has physical motivation, it could still
be culture-specific. Pictorial runes of meteorological or natural phenomena
290 Kazuko Shinohara and Yoshihiro Matsunaka
4. Conclusion
Notes
References
1999 Philosophy in the Flesh: The Embodied Mind and its Challenge to
Western Thought. New York: Basic Books.
Matsuki, Keiko
1995 Metaphors of anger in Japanese. In Language and the Cognitive
Construal of the World, John Taylor and Robert MacLaury (eds.),
137–151. London: Methuen.
Matsunaka, Yoshihiro, and Kazuko Shinohara
2001a ANGER IS GASTRIC CONTENTS: Japanese anger metaphor revisited.
Paper presented at 4th International Conference on Researching and
Applying Metaphor, Tunis, Tunisia.
2001b Emotion and outer force schema: A perspective from Japanese.
Paper presented at The First Seoul International Conference on Dis-
course and Cognitive Linguistics, Seoul, Korea.
2003 Clouds and sunshine in mind: Meteorology-based Japanese emotion
metaphors. Paper presented at 5th International Conference on Re-
searching and Applying Metaphor, Paris, France.
McCloud, Scott
2006 Making Comics: Storytelling Secrets of Comics, Manga and
Graphic Novels. New York: HarperCollins Publishers.
Nihon-Seikishoogakkai [Japanese Society of Biometeorology]
1992 Seikishougaku no Jiten [Encyclopedia of Biometeorology]. Tokyo:
Asakura Shoten.
Shinohara, Kazuko, and Yoshihiro Matsunaka
2001 Is emotion really force? Japanese metaphor of sorrow. Paper pre-
sented at The Seventh International Cognitive Linguistics Confer-
ence, Santa Barbara CA, USA.
2003 An analysis of Japanese emotion metaphors. Kotoba to Ningen
[language and human being]: Journal of Yokohama Linguistic Cir-
cle 4: 1–18.
Tezuka, Osamu
1993 Black Jack. Vol. 3. Tokyo: Akita Shoten.
Usui, Yoshito
1992a Crayon Shinchan. Vol. 1. Tokyo: Futabasha.
1992b Crayon Shinchan. Vol. 3. Tokyo: Futabasha.
1997 Crayon Shinchan. Vol. 16. Tokyo: Futabasha.
Yamanaka, Keiichi
2003 Waka no Shigaku [The Poetics of Classical Japanese Poetry]. Tokyo:
Taishuukan Shoten.
Yu, Ning
1998 The Contemporary Theory of Metaphor: A Perspective from Chi-
nese. Amsterdam/Philadelphia: Benjamins.
V
Abstract
This chapter offers a systematic account of the forms that mono- and multimodal
metaphors may take in face-to-face communication. The account is based on the
relation of source and target domains expressed either in one modality only (thus
forming a monomodal metaphor) or in two modalities (forming a multimodal
metaphor). We will then illustrate the inherent dynamic nature of metaphors when
used in spoken interaction, pointing out more specifically how metaphors are
being elaborated within and across modalities. We will focus particularly on
metaphors that are realized in speech and/or gesture, but point out the relevance
of studying metaphors in other articulatory forms such as stress and intonation.
The different forms of multimodal metaphors are systematically based on different
relations between metaphoric and gestural expressions. Finally, implications for
metaphor theory and for the dynamic aspects of “thinking for speaking” are dis-
cussed, suggesting that multimodal metaphors in spoken language are products of
the process of creating metaphoricity (by a speaker/gesturer and ideally also by a
listener/perceiver), which is essentially independent of modality and expressive
form.
1. Introduction
The situation which has been most influential for the form that spoken lan-
guages have is arguably the face-to-face encounter. We take it as a scenario
that has been described by sociologists, social psychologists, anthropologists
and linguists, and which rather unanimously has been characterized as a
communicative situation that is inherently multimodal.
298 Cornelia Müller and Alan Cienki
and speech are visible and audible actions that form one single utterance
(e.g., Kendon 2004) or proposing that gesture and speech are dynamically
based in different forms of thought but constitute one integrated system (e.g.,
McNeill 1992, 2005). Given that spoken language involves multiple modali-
ties, it makes sense that metaphor should have the potential for multimodal-
ity when used in this form of communication; and indeed over the past years
quite a substantial body of research on metaphor in gesture, speech, and sign
language has been carried out (cf. Bouvet 1997, 2001; Calbris 1998, 2000,
2003; Cienki 1998, 2005b; Cienki and Müller 2008a, 2008b; McNeill 1992;
Mittelberg 2006; Mittelberg and Waugh, this volume; Müller 1998, 2008;
Núñez 2004; Núñez and Sweetser 2006; Webb 1996; Wilcox 2000, 2004).
The topic we want to explore here in particular is the different forms that
multimodal metaphors may take in face-to-face communication. We will
specifically concentrate on the kinds of relations between metaphors that are
realized in speech and/or gesture. It is not by accident that the study of
metaphor is increasingly taking data from gesture studies into account (e.g.,
Cienki and Müller 2008a, 2008b; Müller 2008) and this chapter offers a
systematic account of the forms of metaphors that occur either in speech or
in gesture or in both modalities at the same time. However, we would also
like to point out that beyond gesture, there are additional properties of spo-
ken communication which have received much less or no attention in terms
of their implications for the expression of metaphor, including prosodic fea-
tures, such as stress and intonation, and the time course in which all of these
expressive forms are used during acts of speaking (for the latter point see
Müller 2007, 2008).
In order to clarify what we are discussing, we will restrict the term “mo-
dality” to two dimensions of face-to-face communication: one will refer to
what is expressed orally and perceived primarily aurally as sound (the
oral/aural modality), and the other will refer to bodily forms and movements
in space which are primarily perceived visually (the spatial/visual modality).
In this sense, we will see that gesture/word combinations can constitute mul-
timodal metaphors. Within each modality, there are various forms which can
be used for expressive purposes. In the oral/aural modality, intonation and
stress can be discussed separately from each other and separately from the
words being articulated. We will refer to these as different articulatory forms
within this modality. Similarly within the spatial/visual modality, eye gaze,
body shifts, manual gestures, etc., can all be considered different expressive
forms. Our understanding of articulatory form partially overlaps with Force-
ville’s use of the term mode. In his critical stance towards giving “a satisfac-
tory definition of ‘mode’” or of compiling “an exhaustive list of modes,”
300 Cornelia Müller and Alan Cienki
Forceville argues that this fundamental difficulty “is no obstacle for postu-
lating that there are different modes and that these include, at least, the fol-
lowing: (1) pictorial signs; (2) written signs; (3) spoken signs; (4) gestures;
(5) sounds; (6) music; (7) smells; (8) tastes; (9) touch” (Forceville 2006:
382–3/this volume). In short, spoken words and gestures are articulatory
forms or modes, which are realized in an aural/oral or spatial/visual modal-
ity.
After presenting an overview of what appear to be the most common
ways in which the use of metaphor can play out in the oral/aural and spa-
tial/visual modalities and articulatory forms, we will point out the inherent
dynamic nature of metaphors when used in spoken interaction. Eventually
we will suggest that these observations indicate that multimodal metaphors
are products of the process of creating metaphoricity (by a speaker/gesturer
and ideally also by a listener/perceiver), which is essentially independent of
modality and articulatory form, if metaphoricity is a matter of understand-
ing one idea (or domain) in terms of another. However, we will also argue
that the different modalities and forms that are involved in spoken interaction
afford the use of different expressions for metaphors. What one can express
via a given modality and expressive form will have an effect on what one
will express using that modality. We will conclude by considering the impli-
cations this has for how we can think with metaphors while we are speaking,
or attending to someone who is speaking.
First, we can confirm that we often find the use of metaphoric verbal expres-
sions without co-occurring metaphoric gestures. For example, one American
Words, gestures, and beyond 303
student talks about how people may verbalize certain beliefs about honesty,
but sometimes might not behave in accordance with them. At this point she
says (Example 1):
Example 1:
Just because of the pressure,
the peer pressure,
The word “pressure” was coded as metaphoric in this context because “peer
pressure” normally involves behaviors other than physical pressing – the
more basic physical meaning of the word. Even if peer pressure involved
physical contact, the word still can be understood with the abstract sense of
coercive behavior (it has potential metaphoricity). Although the word was
coded as metaphorically used, the speaker made no gestures while saying it
either time, keeping her hands resting on her leg as she was sitting. We might
refer to such a use of metaphor purely on the verbal level as monomodal
metaphor or as verbal metaphoric expression.
We also find the converse monomodal pattern of metaphor use: meta-
phors expressed in gestures without metaphors in the co-occurring speech,
that is, gestural metaphoric expressions that are used concurrently with
speech. Example 2 comes from one of the Russian students talking about
how they take exams at their university. An English translation is provided
below the transliterated transcript of the Russian. The student is trying to
characterize the Russian concept of “chestnost’,” which may be translated
as “honesty.”
Example 2 (from Russian):
Dlia menia chestnost’ eto nekaia absoliutnaia kategoriia.
For me chestnost’ is a kind of absolute category.
G1 preparation
bh raised in front of torso, flat in vertical
plane, fingers pointing out
Kogda vot iest’ situatsiia,
When there’s this situation,
G1 stroke
bh move straight
downward slightly
seichas postupit’ chestno tak.
then [you need] to act honestly like this.
The student begins seated, hands at rest in his lap, and starting when he says
“situatsiia” [= situation], he lifts his two hands in front of his torso, the right
304 Cornelia Müller and Alan Cienki
hand somewhat higher than the left, palms and fingers flat in the vertical
plane, fingers pointing forward. He holds his hands there until he gets to the
word “tak” [= “like this”], at which point he moves them both slightly
downward in unison, maintaining their position relative to each other and flat
in the vertical plane. Here the speaker explicitly uses the gesture to make
reference – if you don’t see the gesture, you don’t know what he thinks it
means to act honestly (“chestno”). The speaker uses a verbal deictic particle
to direct the attention of his co-participant to the gesture: “like this” points to
the gesture, and his hands move in temporal coordination with the verbal
deictic. Thus verbally he makes clear that the gesture contains relevant in-
formation, but there is no verbal mention of a metaphoric source. In this
instance, manner of behavior (honest) is expressed gesturally as a physical
form (flat/straight) with a certain motion (brief and straight). (See Cienki
1999 for further discussion of this and related examples.)
Note that this is a particularly interesting case, since not only are the ges-
ture’s source and target independent from any verbal metaphoric expression
(in fact there is none in the concurrent speech), but the gestural metaphoric
expression is used in place of words. Thus we might speculate whether this
is an instance of a multimodal utterance consisting of a monomodal gestural
metaphoric expression which is being inserted into a verbal utterance.
Another type of gestural metaphoric expression that is very common
among different cultures involves gestures which perform a speech-act or
more generally a communicative activity. These are gestures that recur in
form and function over a large amount of contexts and therefore we term
them recurrent gestures (cf. Bressem and Ladewig in prep.; Ladewig in
prep.; Teßendorf in prep. a, b). Examples are the palm-up-open-hand gesture
(cf. Kendon 2004; Müller 2004; Streeck 1994), the ring gesture (Fatfouta in
prep.; Kendon 2004; Morris 1977; Neumann 2004), or the brushing aside
gesture (Müller and Speckmann 2002; Speckmann 1999; Teßendorf in prep.
a, b). These gestures all share a common origin, in that they are all meto-
nymic derivations of everyday actions (cf. Mittelberg 2006; Mittelberg and
Waugh, this volume; Mittelberg and Müller in prep.; Müller 1998, in prep.
a, b; Müller and Haferland 1997; Streeck 1994): presenting, offering or
receiving something (the palm-up-open-hand gesture); picking up small ob-
jects with the index finger and thumb (the ring gesture); or brushing aside
small objects. What we observe in these gestures is a two-step semiotic
process as identified and described by Mittelberg and Waugh (this volume),
in which the metonymic target of the sign-formation process turns into the
source of the metaphoric gesture (see also Mittelberg 2007). The targets of
the metonymic process in our cases are the modulated actions: i.e., part of
Words, gestures, and beyond 305
the action stands for the action as a whole, thus consituting a “classical”
instance of a synecdochic relation (Müller in prep. a, b). This modulated
action is used now as a metaphoric source for symbolizing abstract issues
such as presenting a discourse object on the palm-up-open-hand, indicating
the preciseness of arguments, or brushing aside unpleasant topics. For in-
stance the brushing aside gesture is widely used to express negative assess-
ments, and this is what we will see in the next example. Example 3 comes
from free conversations recorded in Cuba (Müller and Speckmann 2002;
Speckmann 1999). (For a detailed analysis of the brushing aside gesture
used by speakers of the Iberian Peninsula, see Teßendorf in prep. a, b).
In Example 3 the speaker thinks out loud about the possible conse-
quences of what it would be like to have four instead of two TV-channels in
Cuba. He is convinced that this would disturb family life by causing endless
discussions about which program to watch. He describes the big arguments
this would raise in a very lively way, and in doing this becomes himself part
of such an imaginary situation: he imagines himself standing in the living
room, and he indicates three different places, each of them relating to a dif-
ferent person voting emphatically for another program: “Yo quiero vel
aquello yo quiero ver lo otro yo quiero ver esto” (“I want to see this one, I
want to see that one, I want to see the other one”). The more programs to
choose from, the more arguments you have in your family – this is the moral
of the speaker’s imagined scenario. It is clearly not desirable to have four
channels on Cuban TV and correspondingly he concludes his discussion with
a negative assessment performed gesturally as a brushing aside gesture.
Example 3 (from Cuban Spanish):
G1
rh point to the right
yo quiero vel aquello,
I want to see that one
G2
lh points straight
yo quiero ver lo otro,
I want to see the other
G3 G4
rh points straight rh brushes aside
yo quiero ver esto (.)
I want to see this one (.)
306 Cornelia Müller and Alan Cienki
The first three gestures the speaker uses relate to the propositional content of
the utterance; G1, G2, and G3 point to three different places in the imagined
apartment, localizing three different persons with three different wishes. The
brushing aside gesture (G4) is located in a micro-pause at the end of this
utterance – and clearly assesses this imagined situation as an undesirable
one. There is no verbalization of a negative assessment; the gesture takes
over the entire communicative burden. It is noteworthy that the gesture is
placed at the end of the phrasal unit, precisely where a verbal evaluative
particle could have been placed. Instead the speaker pauses and produces a
gesture with similar content. It seems as if the brushing aside gesture would
do the “same job” as a verbal particle would (which is why Müller and
Speckmann, suggested the term “gestural particle”). It gives a negative
evaluation of a situation being described, and the gestural meaning is derived
from the negative connotation of the practical action. (For a detailed cogni-
tive semiotic analysis of this process, see Teßendorf in prep. a, b.)
What the brushing aside gesture shares with the other recurrent gestures
mentioned before is that it has a performative or (more general pragmatic)
function rather than a referential one, and it is obvious that metaphor plays a
different role here than in example 2. In the second example the communica-
tive function is metaphorical reference, whereas in example 3 the gestures’
function is the performance of a communicative action. Hence the first use
of the metaphoric gestures belongs to the realm of semantics while the other
one belongs to pragmatics. The difference is not a simple matter, but for the
sake of brevity it might be characterized as a difference between gestures
contributing information to the propositional content of the utterance and
gestures contributing meta-communicative information. While in the second
example the metaphoric gestures expressed aspects of the propositional con-
tent (honesty as a physical form and movement) in the third example the
metaphoric gestures are used for meta-communicative purposes (they qualify
the propositional content), telling us how the propositional content (the
choice of various programs) is being assessed by the speaker. Thus while
metaphor as well as metonymy are clearly involved in these gestures, they
come in at the level of the semiotic process of sign formation rather than on
the level of communicative function.
We may conclude that monomodal metaphors are frequent in words but
they also can be found in gestures. As for gestural metaphoric expressions
we have found two different kinds: on the one hand there seem to be gestures
that are more likely to be created on the spot (such as example 2, “honesty”
with a “straight” gesture), and others that appear to recur with a relatively
stable form and function (the “brushing aside” case, example 3). These two
Words, gestures, and beyond 307
kinds of gestural metaphors furthermore seem to fall into two different func-
tional groups: one of them expressing parts of the propositional content, the
other one performing meta-communicative acts in the widest sense. It seems
that the latter ones show a tendency for conventionalization, which is why
we are able to put together repertoires of those forms but not of the sponta-
neous referential gestures, created ad hoc.
Example 4:
G1
open palms touching each other repeatedly
also da hab ich schon gemerkt naja’
So there I had already realized, well,
G1 continued
des is ganz schön klebrig.
this is pretty clingy,
308 Cornelia Müller and Alan Cienki
G1 continued
(..) oder heftig.
(..) or intense.
using this idiom for the first time, he does not gesture. Gesturing begins with
his elaboration and illustration of this metaphoric argument.
Example 5:
nein es is nich so,
no this is not the case,
aber es stellt natürlich Weichen.
but it obviously sets tracks.
das is das Problem.
this is the problem.
The speaker begins to develop his alternative viewpoint with a very common
rhetorical pattern in German conversations, the “nein aber” (“no but”) pat-
tern, in which a preceding suggestion is first confirmed and then challenged.
The confirmation in our example is verbalized in the first line: “no, this is
not the case,” hereby confirming his interlocutor's point of view, which is
then followed by the counterargument in line two, beginning with “but”: “but
it obviously sets tracks.” He verbally formulates his alternative viewpoint,
and he does this metaphorically: “it obviously sets the tracks.” No gesture is
produced along with this first formulation of his counterargument; he only
begins to gesture with his first reformulation of the verbal metaphoric ex-
pression. Having had no ratifying reaction from his co-participant he begins
to elaborate his argument. And with this elaboration he performs a pointing
gesture towards his left. Note that the pointing gesture is one in which the
extended palms, held vertically, are used to indicate a certain direction. Note
that there is a systematic variation of form and function in pointing gestures.
Kendon and Versante (2003) show that in Neapolitan conversations speakers
use the index finger to point out objects, whereas the flat hand is used to
indicate directions. In our case the vertical open palms of the two hands are
joined to indicate one direction of a future career. In short, we see here an-
other example of a source being expressed in words and in gestures; the goal
of a track is to lead the train into a certain direction, and the gesture visual-
izes and spatializes this aspect of directionality of the source: gesturally the
future career is located to the left hand side of the speaker. That this pointing
gesture is a metaphoric one only becomes clear when considering the words
with which it is co-expressive, and these entail a verbal metaphoric expres-
310 Cornelia Müller and Alan Cienki
sion “Weiche” (“tracks”). Words and gestures share source and target of a
metaphoric expression, and these cases are what Müller (2008) terms
“verbo-gestural metaphors” (cf. Forceville’s 1996, 2002 concept of verbo-
pictorial metaphors).
As soon as the speaker says “far” she lifts her two hands up and places them
next to each other with the palms of her hands basically facing herself, and
turned slightly towards each other. Both hands are cupped, with the fingers
tense and curled inward halfway towards the palms. The shape is as if her
hands were surrounding a medium-sized ball that she were squeezing. Given
that she holds her hands in this position for the entire phrase “abstract
thought of honesty is” (making rhythmic beats on the syllables “far,” “-
stract,” and ‘hon-”) we argue that this is a way in which she physically char-
acterizes this “abstract thought” in gesture. We therefore find the metaphoric
target domain in her words and the source domain (a solid form like that of a
round object) in the gesture. Note that there is no metaphoric expression on
the verbal level.
We sometimes see verbal and gestural metaphoric expressions being ut-
tered at the same time, each using a different source to express the same
target. In example 7 there is a color metaphor expressed verbally with a
Words, gestures, and beyond 311
spatial metaphor expressed gesturally (see also the discussion of this exam-
ple in Cienki 2008, and Cienki and Müller 2008b). Here the speaker from
Example 6 above continues the thought which was begun there describing
honesty as something that does not have “gradations”: instead it is character-
ized by clear oppositions: right or wrong, black or white.
Example 7:
I mean--
y'know,
…
y'know,
G1
bh in front of chest, palms facing self, fingers curled
a- as far as an abstract thought of honesty is,
y'know,
G2
bh palms together, flat in horizontal plane, lh palm up,
rh sweeps left to right across palm of lh
ther- there is no gradations.
G3
lh flat and palm up, rh flat,
outer edge taps palm of lh (‘v’ marks tap),
alternating slightly to the left (L) and to the right (R)
vL vR vL vR
Either you're right you're wrong y'r black 'r white y'know.
While verbally describing these oppositions (G3), she moves her left hand
out in front of her, palm up and open. She holds her right hand above it, flat,
with the palm held vertically, and taps the right edge of her palm against her
open left hand in time with the speech as she says each of the words “right,”
“wrong,” “black,” and “white.” Her right hand taps the left hand first
slightly on the left side of her palm (while saying “right”), then slightly to the
right side (while saying “wrong”), and repeats these left and right taps when
saying “black” and “white,” respectively. In one sense the gesture appears to
be the dividing line, separating the space on the palm of her left hand into
two parts (left and right spaces); but at the same time it indicates those very
spaces, the left and right sides of the palm of her hand, by tapping them.
Whereas the gesture indicates each member of the two sets of opposing cate-
gories as two spaces, the words invoke an opposition between black and
white. While colors (or the lack of them) would be difficult to represent in an
312 Cornelia Müller and Alan Cienki
iconic way with gestures, spatial concepts are easily rendered, and conse-
quently the metaphor used in gesture (different spaces) is different than the
one used in words (different colors). We see here how the specific character-
istics of the expressive modality may inform the type of metaphors ex-
pressed, leading in this case to multimodal metaphoric expressions that have
different sources but share the same metaphoric target (different categories
of behavior). These expressions might be tentatively termed “verbo-gestural
metaphoric compounds.” They differ from verbo-gestural metaphors (source
and target are shared) in that they work together in expressing the same tar-
get metaphorically but do so with different means, i.e., by using different
sources.
4.4 Discussion
G1
2 joined flat hand point towards left
es is schon ne Weiche-—
it does set tracks
G2
flat hands point forward
es is wieder ne Weiche--
it sets another track
G3
flat hands point upward
wenn de sachst ich studiere Medizin--
when you say I will study medicine
G4
2 flat hands point towards left
oder Germanistik--
or German studies
Words, gestures, and beyond 315
G5
2 flat hands point towards right
oder Landwirtschaft--
or agriculture
G6
2 flat hands point upwards and clap during pause
oder (..) werde Tennislehrer.
or become a tennis coach
des is schon ne Weichenstellung.
this is a kind of setting the tracks
G7
1 flat hand points forward twice, held through pause
nachm Studium mußte dir wirklich überlegen welche--(...)
after graduating [from university] you really have to think carefully
which...
In the first part of his response he expresses his alternative viewpoint with a
verbal metaphoric expression: “it does set tracks.” We see no gesture going
with this first formulation; rather this counterargument is highlighted ver-
bally through a meta-comment: “this is the problem.” These first verbal
moves set the stage for a sequence of verbal illustrations and gestural enact-
ments of the verbal metaphoric expression. Subsequently the verbally ex-
pressed metaphoric concept of “setting the tracks” is illustrated by listing
three job alternatives – medicine, agriculture, tennis – each one being gestur-
ally situated in a different direction: medicine is the path to the left, agricul-
ture to the right, and tennis is located in the upward direction. The gestures
visualize the source of the metaphoric expression “setting the tracks,” they
embody directionality, and they locate the different future career paths in
three alternative directions in the gesture space (left, right, up). But this is
not the end of the speaker’s argument. After verbalizing three alternatives
and enacting three different directions for three different jobs, he summarizes
and comes back to the verbal metaphoric expression that he had used ini-
tially: “this is a kind of setting the tracks”; once again this verbal metaphoric
is not accompanied by a gesture. By returning to his initial expression he
retrospectively frames his verbo-gestural elaborations as examples for the
metaphoric expression he had used to challenge his co-participant’s argu-
ment while at the same time preparing his last and now fully explicit re-
formulation of his counter-argument: “after graduating you really have to
think carefully which-- (…).” This last re-formulation ends with a gestural
expression of the idiom that replaces the words and is inserted into the
316 Cornelia Müller and Alan Cienki
speech-pause. He uses yet another pointing gesture, but this one is per-
formed with one hand only and it is directed forward – a direction which has
not yet been “occupied” by any of his preceding examples. Moreover the
gesture is highly articulated in shape: it is supported by the left hand, di-
rected towards the recipient, repeated twice, and held through the speech
pause at the end of the turn; and with this gesture the counter-argument and
the turn end.
To sum up, in this segment of talk we find a verbal metaphoric expres-
sion at the onset which is further verbally illustrated with concrete examples,
enacted and elaborated in gestural metaphoric expressions, and completed
with a final gestural metaphoric expression at the end of this counterargu-
ment. This example nicely illustrates that metaphoricity is a dynamic feature
which may trigger metaphoric elaborations in multiple modalities succes-
sively in time, and which may provide grounds for the ad hoc creation of
new metaphoric gestures, doing “different jobs.” We may argue that when
the verbal metaphoric expression was uttered first, metaphoricity was not in
the foreground of the speaker’s attention; we find no indication that meta-
phoricity was particularly active for the speaker at that point in time. Put
differently, at this moment the metaphor was sleeping; only as the speaker is
moving on is he building his elaborations in words and gestures on this
sleeping metaphoric expression, thus using it as source. Doing this makes
clear that metaphoricity becomes successively more active, as he moves
along with his argument, such that we may now speak of waking metaphors.
Formulated in McNeill’s terms, what we may find here is a metaphorical
growth point that structures a whole unit of discourse (McNeill 1992;
McNeill and Duncan 2000).4 For conceptual metaphor theory this raises
questions about how to account for metaphoricity as a dynamic property,
which can be more or less highlighted (Müller 2008). Again, the formula of
TARGET IS SOURCE problematically reifies the two domains as static entities.
image schema names, but this second group only read the phrases and did
not hear the recordings of the speakers’ voices uttering them. The experi-
ments were actually conducted as controls for another experimental setting
in which the participants saw and heard the video clips in which the speakers
uttered these phrases and made co-verbal gestures (Cienki 2005a). Since the
utterances were chosen because they were ones which occurred with gestures
of various kinds, the words and phrases themselves were rather random,
ranging from more substantive ones, such as “their tests are difficult” and
“it’s like you’re performing,” to comments and interjections, such as “no,
not really” and “like.”
After completing the categorization task, the participants in the first
group were asked to write a sentence or two explaining how they used the
image schemas to categorize the phrases they heard. The results revealed
that they sometimes categorized some of the phrases according to their
acoustic properties, rather than referring to the meanings of the lexical items.
Consider the following response as an example: “a phrase where the tone
rose and fell back again seemed cyclical, whereas when the tone steadily rose
it seemed like a path.” We see how metaphor may play a role in interpreting
how an utterance was spoken. As a side note, this could be important for
metaphor researchers in terms of setting up stimuli for experiments on meta-
phor interpretation. The findings underscore the importance of considering
the mode of presentation of experimental stimuli (in oral versus written
form) because of the effect it may have on the interpretation of the “same”
linguistic expressions.
It is worth noting with these examples of metaphor in intonation that we
are not dealing with verbal semantics, but with metaphor on the pragmatic
level – what the speaker meant with the use of a given intonation contour.
Interestingly, we find a parallel phenomenon of metaphor on the pragmatic
level in gesture. The primary function of some gestures appears to be to
highlight interactive or interpersonal relations, to parse the discourse, or to
accomplish a performative act (Kendon 2004: ch. 9). Müller, referring to
unpublished observations by Jürgen Streeck, discusses the pragmatic func-
tions of the palm-up open-hand (PUOH) gesture, which can serve to “pre-
sent an abstract, discursive object as a concrete, manipulable entity” (2004:
233). The gesture can indicate that what the speaker is saying is to be inter-
preted as an idea to be discussed, a proposal, or a question (Kendon 2004:
159). In terms of conceptual metaphor theory, we might say that this gesture
uses the pragmatic metaphor of INTRODUCING AN IDEA IS PRESENTING AN
OBJECT. Here as in the other recurrent gestures discussed above, the meta-
phor does not simply work on the level of what the speaker’s words express
Words, gestures, and beyond 319
A major conclusion we can draw from the fact that metaphors can be real-
ized in multiple modalities is that metaphoricity is modality-independent. It
documents that the establishment and creation of metaphoricity is a cognitive
process with products in various modalities, thus offering strong support for
Lakoff and Johnson’s initial idea of moving metaphor(icity) out of the realm
of literary discourse into the mundane world of everyday thought (Müller
2003, 2007, 2008). However, this also has critical implications for metaphor
theory in that it calls for refined empirical methodology as well as for a new
theoretical understanding of the different forms of multimodal metaphors and
their constitutive semantic relations. It also directs our attention to the neces-
sity of including a cognitive-semiotic analysis of metaphoric, as well as of
metonymic, processes (see Mittelberg 2006, 2007; Mittelberg and Müller in
prep. a; Mittelberg and Waugh this volume). A major implication of the
insights gained through the analysis of multimodal metaphors in the use of
spoken language is the fact that as spoken language is inherently dynamic, so
is multimodal metaphor.
As already indicated above, the study of metaphors as expressed in the
dynamic processes of speaking presents us with metaphoric source domains
which are themselves contingent on time for their realization. This raises a
problem, given the traditional means of conceptual metaphor analysis,
namely that it involves the static verbal formula of TARGET IS SOURCE (an
issue raised long ago by the anthropologist Bradd Shore, personal communi-
cation). Various authors in recent research have suggested alternatives to try
to overcome the limitations of this analytic device.
For some types of source domains, one solution is to characterize them
by using schematic images. An example described in Cienki (2005b) is that
when Al Gore was a candidate for U.S. president in 2000 he used the same
gesture at several points during the televised debates: a gesture with one or
both hands palm up and cupped slightly with the fingers slightly curved, as if
he were holding a small ball. This gesture occurred with phrases such as
“enable us to project the power for good,” “shepherds that economic
strength,” “the power of example is America’s [greatest power] in the
world” (with square brackets indicating the timing of the gesture in the last
320 Cornelia Müller and Alan Cienki
example). We would argue that in the examples from Gore, the gesture
serves basically the same purpose as the PUOH gesture discussed by Müller
(2004), but that there is an added element here indicated by the cupped shape
of the hand. In a physical situation, such a hand shape would be used not
only to support a small object in the hand, but also to prevent it from falling
off the extended hand, thus protecting it in a way. Thus the gesture not only
suggests that the speaker is treating AN IDEA AS AN OBJECT, which he is
presenting to the addressee (the moderator of the debate and, by extension,
the television audience), but that he is also showing something about his
attitude toward the idea he is presenting, perhaps that it is something good
which he wants to support (all three utterances expressed positive ideas
which Gore espoused). In light of the meaning added by the cupped hand
shape, the manner of presenting is significant, and (as argued in Cienki
2005b) could be indicated by a diagram or schematic image – see the ones
Efron (1972) used in his analysis of the linguistic properties of the gestures
used by Italians as compared to Eastern-European Jewish immigrants to
New York City. See also Calbris’ (2003) schematizations of gesture hand
shapes and motions in diagrammatic form. Finally, the increasing use of
digital publishing (online or on CDs or DVDs) allows for video characteriza-
tion of source domains which are dynamic in nature, in that they can be pre-
sented as moving schematic images, for example as small animations.
Slobin (1987, 1996) argues that there is a special form of thought which is
mobilized in the process of talking, which he calls thinking-for-speaking. As
he describes it, “‘Thinking for speaking’ involves picking those characteris-
tics [of a perceived event, CM and AC] that (a) fit some conceptualization of
the event, and (b) are readily encodable in the language” which the speaker is
using at the moment (Slobin 1987: 435). Thus the lexical and grammatical
means of expression available in a language are used by speakers already as
they are anticipating how to utter what they want to utter. McNeill and Dun-
can (2000) suggest that gesture needs to be taken into account in this process
as well. They discuss how the idea units which we are continually develop-
ing and unraveling for expression while we talk, what McNeill (1992) has
called growth points, combine both imagery and linguistic-categorical con-
tent. In the process of thinking while speaking, which McNeill and Duncan
(2000: 157) note is perhaps a more accurate way to refer to the phenomenon,
the imagistic content receives partial expression in the gestures that the
Words, gestures, and beyond 321
9. Conclusion
process which takes into consideration the nature and the expressive poten-
tial of the respective modalities (colors do not lend themselves to expression
in gesture, but for spatial relations the opposite holds true). Compare for
instance what is known about metaphoric expression in another use of the
manual modality, namely in sign language. In both cases, gesture and sign
language, the iconic nature of visual/manual expression affords different
potentials than aural/oral expression does (Müller in prep.; Taub 2001),
although gestures with speech are normally co-verbal, as opposed to consti-
tuting linguistic signs in and of themselves. However, in the process of
communication – or to put it in Wallace Chafe’s (1994) terms, in the flow of
discourse – these modality-specific properties can be exploited to varying
extents in any given event of speaking.
A dynamic approach to linguistic theory (such as that proposed by
McNeill 2005) or to metaphor theory (as in Müller 2008) which can ac-
commodate the multimodal potential of language production and reception
can provide a more complete picture of the complexity of this form of human
behavior than the static views of language, metaphor, and thought which
currently dominate the field of cognitive linguistics and beyond. In conclu-
sion, for researchers of spoken language, moving beyond the level of the
words can uncover many facets of metaphoricity that had previously lain
hidden.
Acknowledgements
We are grateful to Charles Forceville, Irene Mittelberg, and Linda Waugh for
insightful comments on an early draft of this chapter.
Notes
References
Bouvet, Danielle
1997 Le corps et la métaphore dans les langues gestuelles: A la recher-
che des modes de production des signes. Paris: L’Harmattan.
2001 La dimension corporelle de la parole: Les marques posturo-mimo-
gestuelles de la parole, leurs aspects métonymiques et métaphori-
ques, et leur rôle au cours d’un récit. Paris: Peeters.
Bressem, Jana, and Silva Ladewig (eds.)
in prep. Handmade patterns: Recurrent Forms and Functions of Gestures.
Amsterdam/Philadelphia: Benjamins.
Calbris, Geneviève
1990 The Semiotics of French Gestures. Bloomington, IN: Indiana Uni-
versity Press.
1998 Geste et images. Semiotica 118 (1-2): 105–120.
2003 From cutting an object to a clear cut analysis: Gesture as the repre-
sentation of a preconceptual schema linking concrete actions to ab-
stract notions. Gesture 3: 19–46.
Cameron, Lynne, and Graham Low (eds.)
1999 Researching and Applying Metaphor. Cambridge: Cambridge
University Press
Chafe, Wallace
1994 Discourse, Consciousness, and Time: The Flow and Displacement
of Conscious Experience in Speaking and Writing. Chicago: Uni-
versity of Chicago Press.
Cienki, Alan
1998 Metaphoric gestures and some of their relations to verbal meta-
phorical expressions. In Discourse and Cognition: Bridging the
324 Cornelia Müller and Alan Cienki
Mittelberg, Irene
2002 The visual memory of grammar: Iconographical and metaphorical
insights. Metaphorik.de 2/2002: 69–89.
2006 Metaphor and Metonymy in Language and Gesture: Discourse Evi-
dence for Multimodal Models of Grammar. Ph.D. diss., Cornell
University.
2007 Internal and external metonymy in co-verbal gesture: Jakobsonian
and cognitivist views on contiguity relations within and across mo-
dalities. Paper presented at the 10th ICLC, Krakow, July 2007.
Mittelberg, Irene, and Linda Waugh
this vol. Metonymy first, metaphor second: A cognitive-semiotic approach to
multimodal figures of thought in co-speech gesture.
Mittelberg, Irene, and Cornelia Müller (eds.)
in prep. Metaphor and Metonymy in Gesture and Sign Languages. Special
Issue. CogniTextes.
Morris, Desmond
1977 Manwatching. London: Jonathan Cape.
Müller, Cornelia
1998 Redebegleitende Gesten. Kulturgeschichte – Theorie – Sprachver-
gleich. Berlin: Berlin Verlag Arno Spitz.
2003 Gestik als Lebenszeichen “toter Metaphern.” Tote, schlafende und
wache Metaphern. Zeitschrift für Semiotik 1-2: 61–72.
2004 Forms and uses of the Palm Up Open Hand: A case of a gesture
family? In The Semantics and Pragmatics of Everyday Gestures,
Cornelia Müller and Roland Posner (eds.), 233–56. Berlin: Weidler.
2007 A dynamic view of metaphor, gesture and thought. In Gesture and
the Dynamic Dimension of Language, Susan Duncan, Justine Cas-
sell, and Elena Levy (eds.), 109–116. Amsterdam/Philadelphia:
Benjamins.
2008 Metaphors – Dead and Alive, Sleeping and Waking: A Dynamic
View. Chicago: University of Chicago Press.
in prep. a Creative devices in gestures and sign languages. In Cornelia Müller,
Ellen Fricke, Silva Ladewig, Irene Mittelberg, Sedinha Teßendorf
(in prep.) Gestural Modes of Representation as Mimetic Devices –
Revisited. Gesture.
in prep. b How hand movements turn into gesture: Gestural modes of repre-
sentation as metonymic resources of gesture creation. In Metaphor
and Metonymy in Gesture and Sign Languages. Special Issue of
CogniTextes, Irene Mittelberg and Cornelia Müller (eds.).
Müller, Cornelia, and Ulrike Bohle
2007 Das Fundament der Interaktion: Zur Vorbereitung und Herstellung
von Interaktionsräumen durch körperliche Koordination. In
Koordination: Analysen zur multimodalen Interaktion, Reinhold
Schmitt (ed.), 129–165. Tübingen: Narr.
Words, gestures, and beyond 327
Steen, Gerard J.
2006 Metaphor in applied linguistics: Four cognitive approaches. Docu-
mentação de Estudos em Lingüística Teórica e Aplicada –
D.E.L.T.A. 22 (1): 21–44.
2007 Finding Metaphor in Grammar and Usage: A Methodological
Analysis of Theory and Research. Amsterdam/Philadelphia: Benja-
mins.
Steen, Gerard J. and Raymond W. Gibbs, Jr.
1999 Introduction. In Metaphor in Cognitive Linguistics, Raymond W.
Gibbs, Jr. and Gerard J. Steen (eds.), 1–8. Amsterdam/Philadelphia:
Benjamins.
Streeck, Jürgen
1994 “Speech-handling”: The metaphorical representation of speech in
gestures. A cross-cultural study. Ms. University of Texas at Austin.
Sweetser, Eve
1998 Regular metaphoricity in gesture: Bodily-based models of speech
interaction. Actes du 16e Congrès International des Linguistes
(CDRom). Amsterdam: Elsevier.
Teßendorf, Sedinha
in prep. a From everyday action to gestural performance: Metonymic motiva-
tions of a pragmatic gesture. In Metaphor and Metonymy in Gesture
and Sign Languages. Special Issue of CogniTextes, Irene Mittelberg
and Cornelia Müller (eds.).
in prep. b Pragmatic and recurrent: The case of the brushing aside gesture. In
Recurrence in Coverbal Gestures – Investigating Form and Func-
tion, Jana Bressem and Silva Ladewig (eds.). Amsterdam/ Philadel-
phia: Benjamins.
Taub, Sarah
2001 Language from the Body: Iconicity and Metaphor in American Sign
Language. Cambridge: Cambridge University Press.
Webb, Rebecca
1996 Linguistic Features of Metaphoric Gestures. Ph.D. diss., University
of Rochester, NY.
Wilcox, Phyllis P.
2000 Metaphor in American Sign Language. Washington, DC: Gallaudet
University Press.
2004 A cognitive key: Metonymic and metaphorical mappings in ASL.
Cognitive Linguistics 15: 119–147.
Chapter 14
Metonymy first, metaphor second: A cognitive-
semiotic approach to multimodal figures
of thought in co-speech gesture
Abstract
Based on spoken academic discourse and its accompanying gestures, this chapter
presents a cognitive-semiotic approach to multimodal communication that assigns
equal importance to metaphor and metonymy. Combining traditional semiotics
with contemporary cognitivist theories, we demonstrate how these two figures of
thought jointly structure multimodal representations of grammatical concepts and
structures. We discuss Jakobson’s view of metaphor and metonymy, and particu-
larly his distinction between internal and external metonymy, thus discerning
various principles of sign constitution and indirect reference within metaphoric
gestures (whether or not the concurrent speech is metaphorical). We then intro-
duce a dynamic two-step interpretative model suggesting that metonymy leads the
way into metaphor: in order to infer the imaginary objects or traces that gesturing
hands seem to hold or draw in the air, a metonymic mapping between hand
(source) and imaginary object (target) is a prerequisite for the metaphorical map-
ping between that very object (source) and the abstract idea (target) it represents.
1. Introduction
Work done by scholars in many disciplines has shown that metaphor and
metonymy rely on general cognitive processes of conceptualization and asso-
ciation that may materialize in modalities other than spoken and written
words, e.g., in gesture. While the chapters in this volume contribute to a
unified approach to the role of metaphor in multimodal representations, we
will show here that it is both metaphor and metonymy that, by working to-
gether in multimodal communication, function to convey complex meanings,
330 Irene Mittelberg and Linda R. Waugh
1990; Waugh et al. 2004). Thirdly, both Jakobson and Peirce stress the
point of view of the interpreter of a given message: in similar fashion, our
aim will be to explicate how gestures are interpreted by the viewer (e.g., the
student, the gesture analyst) through both metaphor and metonymy.
Connecting the body, language, cognition, society and culture, gestures can
provide a window into thought processes and their pragmatic and ecological
anchorage (Goodwin 2003; Ochs et al. 1996; Streeck 2002). From a cogni-
tivist viewpoint, i.e., taking the perspective of the speaker, research has
shown how hands (and arms) may reveal, consciously or unconsciously,
thoughts and attitudes that speech might conceal (McNeill 1992: 246). We,
on the other hand, are trying to see how the gestures help the viewer to un-
derstand the conceptualization of abstract ideas that the speaker/gesturer is
communicating.
The spontaneous gestures we will be analyzing here are not part of an
elaborated sign system but are created by the speaker as he/she speaks, and
thus gesture and speech can produce very different effects, including junc-
ture or disjuncture, redundancy, complementation, or mismatch (Goldin-
Meadow 2003; McNeill 2000, 2005). A gesture may disambiguate linguistic
information and thus make meaning more precise (for instance, by pointing
at a concrete referent that is linguistically only referred to via an unspecified
pronoun), or it may add components of meaning not expressed in the speech
it accompanies (Kendon 2000).1 Often, however, spontaneous gestural signs
tend to be polysemous and need a contextual support to be correctly inter-
preted; thus, discourse-pragmatic factors and concurrent speech help to dis-
ambiguate them (Calbris 1990; Kendon 2004; McNeill 2005; Müller 1998).
As Jakobson noted, a pointing gesture at a package of cigarettes could be
interpreted to mean “this package in particular, or a package in general, one
cigarette or many, a certain brand or cigarettes in general, or, still more gen-
erally, something to smoke.” The viewer does not know if the pointer is
“simply showing, giving, selling, or prohibiting the cigarettes.” The only
way to know is through the accompanying speech (Jakobson 1953: 567). A
single gesture could also fulfill several functions at once: e.g., from represen-
tational to deictic, or from accentuating the rhythm of the speech to attract-
ing attention and managing interaction between the interlocutors. Gestures
are thus visuo-spatial “motor signs” (Jakobson 1987a: 474) that derive their
Metonymy first, metaphor second 333
locally-situated meaning from the very human body that articulates them, the
speech they accompany, and the socio-cultural and material environment the
person interacts with. Consequently, a gestural sign does not exist, and can-
not be analyzed, detached from either the human body or the here and now of
the speech event (the origo in Bühler’s terms, see Fricke 2007). This means
that in order to understand the gestures under discussion here, we first need
to characterize the speech they accompany in terms of its genre and func-
tions.
As indicated above, our data come from one specific spoken genre:
metalinguistic academic discourse in lecture format, from a corpus of such
lectures by four professors (three women and one man), all native speakers
of American English, while they were teaching introductory linguistics to
undergraduate and graduate students at two major American universities.
The lectures were videotaped in a naturalistic setting, that is, regularly
scheduled classes where neither the teacher nor the students knew about the
purpose of the taping (in particular, they did not know that the analysis was
to focus on gesture). Thus, the assumption is that the gestures used by the
professors were not affected by the videotaping (for a detailed description of
methods of collecting, editing and transcribing the data, including the coding
and annotation systems used, see Mittelberg 2006, 2007). Now, in the typi-
cal classroom setting there are other visual modalities: e.g.,
black/green/white boards with writing and other visuals on them, handouts,
slides and power point projections. However important these are for the
communication of information in the classroom, what is unique to gestures is
that they are conveyed by the body of the lecturer and correlated with the
speech that is emanating from that same body.2
The speech that is at issue here is highly complex. It has multiple func-
tions: it conveys information about language that reflects the beliefs of the
speaker and is directed at the audience (the students in the class) with the
aim that the students will gain at least an understanding of, and perhaps also
a belief in, the concepts being discussed. The gestures have the same com-
plex multifunctionality as they contribute to the communication and under-
standing of the lectures. Our focus will be on what Müller (1998: 110–113)
calls “referential gestures,” that is, gestures that depict objects, attributes of
objects and people, actions, or behaviors, whether concrete or abstract
(Müller 1998; see also Cienki 2005). More specifically, the gestures ana-
lyzed here are all attempts at making fairly abstract grammatical concepts
and aspects of the syntactic structure of sentences more understandable for
the listener/viewer, by turning them into (partial) visuo-spatial and embodied
manifestations of these concepts.
334 Irene Mittelberg and Linda R. Waugh
an entity related in some semantic way (e.g., cause and effect; instrument;
source)” (Wales 2001: 252), e.g., the term ”the White House” when refer-
ring to the President of the U.S. (place for person); and (2) “internal meton-
ymy”(synecdoche), in which “the name of the referent is replaced strictly by
the name of an actual part of it” (Wales 2001: 252) or by the name of the
whole of which it is a part; e.g., part stands for whole and whole for part
(e.g., “all hands on deck,” in which “hands” stands for the whole body). That
is, Jakobson integrated synecdoche as an important sub-type of metonymy,
and as we will see, these two types of metonymy are crucial to the study of
gesture. Most importantly, Jakobson insisted that similarity and contiguity –
and metaphor and metonymy – are not mutually exclusive: just as signs can
exhibit both similarity and contiguity in differing hierarchies (Jakobson
1966: 411), so the nature of a given sign is dependent on the preponderance
of one of the two modes over the other (see Jakobson 1956: 130).
According to Jakobson (1956: 117, see also Waugh and Monville-
Burston 1990), the similarity/contiguity relations between signs are different
from the basic types of operations by which any linguistic utterance is con-
structed by the speaker. Any act of utterance formation involves the selec-
tion of certain linguistic entities from the code (e.g., words) and their combi-
nation into linguistic units of a higher degree of complexity (e.g., phrases
and sentences). Understanding by the addressee implies the reverse order of
operations: the combination of units of greater complexity has to be dis-
solved into the individual linguistic entities selected. Both “modes of ar-
rangement” (Jakobson 1956: 119) reflect the structural reality of language:
selection relies on the organization of the linguistic system, while combina-
tion is evidenced in the fact that every sign is made up of constituent signs
(sentences, words, morphemes, phonemes, features) and serves as the con-
text for other signs. Jakobson (1956: 119) referred to this kind of semiotic
contextualization as “contexture,” e.g., the process by which “any linguistic
unit at one and the same time serves as a context for simpler units and/or
finds its own context in a more complex linguistic unit. [… C]ombination
and contexture are two faces of the same operation.” In the case of multimo-
dal messages, signs from more than one mode are selected and combined to
constitute the contexture for one another: for example, gesture combined
with speech. Such combinations may be concurrent and/or sequential: so, a
given gesture is concurrent with the simultaneously occurring words, and the
way in which gestures unfold in time (with or without speech) is an example
of sequential combination.
336 Irene Mittelberg and Linda R. Waugh
one and the same gestural form may potentially refer to either a concrete or
an abstract entity. Depicting via metonymy contextually pertinent features of
objects or actions, referential gestures may either portray predominantly
iconic sign-object relationships (representing concrete objects or move-
ments), or they may rely on metaphorical sign-object relationships (involving
abstract entities) and thus call forth a metaphorical interpretation. For ex-
ample, a gesture with two hands may trace the frame of a painting or the
frame of a theory. In both interpretations, the gesture is synecdochic since it
provides only some aspects of the frame by rendering the parts that are
pragmatically salient in the given discourse context. When used non-
metaphorically, the synecdochic gesture can be interpreted as referring to a
spatial, physical structure (e.g., the essential panels of the frame itself, not
the other elements that hold the painting in place). In the case of a meta-
phorical interpretation, the synecdochic gesture further represents, in
Peirce’s (1960: 157) terms, some sort of parallelism – or similarity – be-
tween the form and function of a physical frame and the form and function
of an abstract frame structure (Köller 1975). Adopting a cognitivist perspec-
tive, we can say that the gesture is interpreted with respect to the metaphori-
cal concepts IDEAS ARE OBJECTS and CONCEPTUAL STRUCTURE IS GEOMET-
RIC PHYSICAL STRUCTURE (Lakoff and Johnson 1980; Sweetser 1998).
Stated in McNeill’s (1992) terms, such images of abstract ideas (originally
called metaphorics) represent both source and target domain information (in
this example, the gesture would be regarded as representing both the concep-
tual frame and the physical frame). However, as we will see, the metaphori-
cal gestures we will be discussing here are not directly iconic of the concrete
source domain they involve. In fact, what is common to all of the metaphori-
cal interpretations is that they rest on a first interpretation of the gesture
through metonymy: e.g., the traces in the air have to be interpreted as mean-
ing a frame of some sort; only then can the metaphorically-motivated object
be accessed.
Let us look at some examples from the data to determine how metaphor
and metonymy are manifested in gesture and how source and target domains
play out in the two modalities. The gesture represented in figure 1 is an ex-
ample of a frequently occurring form that has several potential interpreta-
tions. Looking simply at the morphology of the gesture, we see that it con-
sists of two, relatively relaxed, open hands held fairly far apart with palms
facing each other (the right hand is partly closed because it contains a piece
of chalk). If the speaker was referring linguistically to the length of a physi-
cal object such as a large box, the gesture would receive a concrete interpre-
tation through metonymy, as if the speaker were holding an elongated object
338 Irene Mittelberg and Linda R. Waugh
like a box between his hands. However, in this case, the speaker is referring
to a sentence and represents the sentence metonymically by the hands, which
are assumed to be marking the beginning and end of its projection in space.
The sentence is conceptualized metaphorically as bounded space or a large,
elongated object. Thus, the gesture may be said to reflect some basic meta-
phorical concepts proposed in the cognitive linguistic literature: e.g., IDEAS
ARE OBJECTS; CONTENTS ARE CONTAINERS; CATEGORIES ARE CONTAINERS;
CONCEPTUAL STUCTURE IS GEOMETRIC PHYSICAL STRUCTURE (Lakoff and
Johnson 1980; Sweetser 1998). To get to the idea of a sentence, however,
the viewer first has to take a metonymical path from the hands to the space
or the imaginary object. This is a case of external metonymy, because the
speaker is holding the imaginary object between his hands, which are exter-
nal (i.e., adjacent) to the object. And then to get from the object (or the space
extending between the hands) to the sentence, the viewer has to take a meta-
phorical path from the imaginary concrete entity (or space) to the abstract
entity (the sentence).4
(1) (sentences)
... Sentences, \
G1
pvoh-bh far apart
(..) [while they’re made up of words, _
G1 being held
(...) aren’t made up of words, \]
Figure 1. A sentence as an elongated object held (or space extending) between two
hands
It should be noted here that while the term “sentence” is non-metaphorical,
its gestural portrayal is first metonymical in nature and then interpreted
metaphorically. In other words, there are two interpretative moves needed to
get to the imaginary object: (1) the hands represent, via (external) meton-
Metonymy first, metaphor second 339
ymy, the object held between them; (2) the object is a metaphorical represen-
tation of a “sentence” (which is a non-metaphorical linguistic expression).
The imaginary object being held is metonymically inferred through the ges-
ture itself (ACTION FOR OBJECT INVOLVED IN ACTION, Panther and
Thornburg 2004). But the underlying metaphorical mapping, involving the
target domains CONCEPTUAL STRUCTURE, IDEA, or CATEGORY and the
source domains PHYSICAL STRUCTURE, OBJECT, or CONTAINER respec-
tively, can only be inferred by a metaphorical interpretation of the meto-
nymically conveyed object. Forceville suggests that “[b]y contrast to mono-
modal metaphors, multimodal metaphors are metaphors whose target and
source are each represented exclusively or predominantly in different modes”
(Forceville 2006: 384). For this example, this definition holds: the target
domain (“sentence”) is expressed linguistically and the source domain (ob-
ject) is conveyed manually.
The gesture above recurs in the data in slight variations referring to lin-
guistic units of different degrees of complexity (words, phrases, constituents,
sentences, etc.). By contrast, single words, units below the word level, and
grammatical categories such as noun and verb are often represented by a
single hand, for example by an open hand with the flat palm turned upward,
thus forming a surface on which to present something to the addressee (see
Müller 2004 for a detailed discussion of this gesture type). According to our
analysis, the gesture is interpreted metonymically to mean that there is an
object on the hand, and then, through metaphor, that object is interpreted as
a word, a morpheme, a noun, or a verb. In example 2 (figure 2), this open
hand is combined with a closed fist. The speaker, who is talking about mor-
phological structure, illustrates the fact that the English noun “teacher” con-
sists of two morphemes by forming two closed fists held next to each other.
His left fist seemingly contains the lexical morpheme “teach-“ and his right
fist, which opens up into a relaxed palm-up open hand during the demonstra-
tion, contains the grammatical morpheme “-er.” Although the interpretation
of the linguistic expressions relies on neither metaphor nor metonymy, the
two figures of thought again are involved in a two-step process in this semi-
otically complex instance of indirect gestural reference. Here, however, there
is no direct similarity (i.e., image iconicity) between the form of the gesture
and the objects it refers to (as in the frame example discussed above). In-
stead, the enclosed fist is interpreted metonymically as containing, and the
open hand as holding, small physical objects, e.g., LOCATION FOR OB-
JECT; ACTION FOR OBJECT INVOLVED IN ACTION; REPRESENTATION
FOR REPRESENTED (Panther and Thornburg 2004; Wilcox 2004). Thus,
the left hand serves as a CONTAINER and the right hand as a SUPPORT
340 Irene Mittelberg and Linda R. Waugh
structure for the imagined objects; they evoke, independently of the speech
content, these two basic image schemas (cf. Johnson 1987; Mandler 1996).5
And in both cases, these imagined objects are metaphorically construed as
being the two morphemes (IDEAS ARE OBJECTS, Lakoff and Johnson
1980).
1.7
[not]
Without going into the theoretical views regarding the differences between
metaphor and metonymy currently debated in the cognitive linguistics litera-
ture, it should be noted that one of the received understandings holds that
whereas metaphor is based on cross-domain mappings, metonymy consists
of mappings within the same experiential domain (cf. Barcelona 2000a; see
also Croft 1993; Radden 2000).6 In light of this domain-based definition of
metonymy, we can say that both manual actions constitute common experi-
ential domains of holding objects, and thus the gesturer can expect the view-
ers to easily relate to the action from their own experience and to build the
basis for accessing the metaphorically construed objects. According to the
two interpretative moves we introduced above, metonymy again comes first:
the gestural vehicles (e.g., the hand configurations) serve as visible meto-
nymic sources, that is, “reference points” (Langacker 1993); they point to
the invisible target concepts (“teach-” and “-er,” sitting in/on the hands) that
are mentioned in the concurrent discourse. These are instances of external
metonymy, since the imagined objects are adjacent to (contained in or sitting
on), but external to, the hands. The gestural form embodies the source, thus
making it perceivable and present in the immediate context and pointing to
the unperceivable target. So while the associative relation between visible
source and associated invisible target is based on conceptual contiguity, the
abstract notions are metaphorically construed as imaginary objects. Meton-
Metonymy first, metaphor second 341
ymy is also based on the fact that the two hands positioned somewhat near
each other hold associated objects – the metaphorically inferred lexical and
grammatical morphemes referred to in the speech – that together make up a
word (PART FOR WHOLE, PART FOR PART, Jakobson 1956, 1963).
Having discussed some of the most basic gestural forms that recur in the
data, we will now turn to more complex multimodal representations of syn-
tactic structure that are based on a specific model of linguistic structure,
namely generative grammar. In these cases, there are ready-made metaphori-
cal visualizations provided by the theory (tree structure diagrams to depict
syntactic structure) that can then be referred to by the gestures. For example,
when explaining dependent clauses in English, a speaker employing this
framework used the right hand to sketch a branch of a tree structure diagram
extending toward the lower right of her body. Figure 3 shows such a diago-
nally descending movement that is meant to represent an embedded clause.
The speaker illustrates the idea of subordination (G1) by repeatedly moving
her right hand first up to eye-level and then downward to her right side,
thereby making a wave-like movement by tilting the hand from side to side.
This can be assumed to roughly sketch out, through synecdoche, an elabo-
rated tree structure, which is a diagrammatic metaphor used in generative
grammar for the structure of complex sentences with subordinate clauses.
More importantly, such tree structures are used in linguistic textbooks for
Metonymy first, metaphor second 343
learners and in research articles by scholars – and in the case discussed here,
there is a tree diagram behind the speaker on the white board. The fact that
the speaker (who is left-handed) is talking about English, a right-branching
language, may motivate the use of her right hand for the gesture even fur-
ther.
(G1 repeated) G2
rh extended arm and index finger
point toward ground
coming in verb phrases] [all the way down].
and its canonical metaphors and diagrams, the gestures could not be inter-
preted correctly. This is different from the more intuitive examples in section
4.1, in which the speech was non-metaphorical (sentence, morphemes) and
the gesture rendered a metaphorical understanding of abstract entities as
objects or chunks of space without any ready-made visualization to fall back
on.
Since linguistic theories are often built on many specific metaphors, in-
teractions of more than one metaphorical understanding can also be observed
in the data. The subordination gesture (G1) in example (3) represents, as we
just saw, the notion of “embedded sentence” (mentioned in the concurrent
speech) as a wavy line descending in a diagonal toward the floor. Subordi-
nated (embedded) entities are thought of in generative grammar as below the
ones that dominate them. This indicates that the theory the speaker has in
mind when talking about sentence structure motivates the form the gestures
take. Moreover, the theory of syntactic structure proposed within generative
grammar rests on a combination of spatial metaphors (i.e., the tree diagram)
and power relations (i.e., dominance, control, etc.). The question that arises
is whether, and if so, how, these two different source domains are made
manifest in the verbal and/or manual modalities.
In the example above, relations of dominance are not alluded to linguisti-
cally, but let us look at another sequence where the same speaker makes
reference to the idea of dominance in the speech modality. Just as in figure 3,
the gesture derives its meaning from the movement and the virtual traces left
in the air. As shown in figure 4, the speaker draws a tree chunk in the form
of a triangle in the air, with both hands starting out at the center top (the
node) and then tracing diagonals outward and downward to either side of the
body. The gesture is a synecdochic depiction that is metaphorically inter-
preted as meaning several technical terms (nodes alpha and delta, domina-
tion, and branching). Outside of this theoretical model, these terms do not
necessarily entail spatial relationships, or if they do, then they might not be
represented in exactly the same way (e.g., “branching” does not necessarily
have a downward orientation). An interesting moment occurs when the
speaker realizes that she was talking about a node dominating elements
without actually having introduced the idea of “dominance.” In the speech
modality, she quickly changes from the hierarchical understanding of domi-
nance back to the spatial tree metaphor involving a node being “on top of
two things.” Thus, the speech here is metaphorical in two compatible ways;
compatible because spatial and social hierarchies both draw on spatial rela-
tions such as UP and DOWN, with certain values attached to each location in
the corresponding system (e.g., POWER IS UP, see Lakoff and Johnson
Metonymy first, metaphor second 345
1980). Although in the speech modality there is, for a moment, a slight hesi-
tation about which metaphor to use, the gesture modality consistently and
repeatedly represents the spatial features of the tree model and thus is moti-
vated by the corresponding spatial metaphor, which is a conventional part of
the theory.
(..) it branches. \
Figure 4 represents (1) the very beginning of the branching gesture (hands
are joined at the top, the node, of the triangle) and (2) the repeated down-
ward movement that reinforces the idea of an active branching process.
As for the underlying conceptual metaphors, these gestural diagrams all
reflect the metaphor SYNTACTIC STRUCTURES ARE GEOMETRIC PHYSICAL
STRUCTURES, based on CONCEPTUAL STRUCTURE IS GEOMETRIC PHYSICAL
STRUCTURE (Sweetser 1998) discussed above. Given its specific semiotic
affordances, gesture, a semiotic system exploiting space, provides a spatial
projection of compatible metaphors stemming from the domains of physical
structures and social hierarchies with a built in up-down orientation
(POWER/HIGH STATUS IS UP; HAVING CONTROL IS UP; BEING SUBJECT TO
CONTROL IS DOWN; LOW STATUS IS DOWN; Lakoff and Johnson 1980;
Sweetser 1998). Following canonical tree diagrams, these gestures depict
346 Irene Mittelberg and Linda R. Waugh
What all the examples in this paper show is that whether the metaphorical
interpretation of the metonymic gesture is simple and easily accessible (e.g.,
a sentence, morphemes) or complex and only understandable in the context
Metonymy first, metaphor second 347
speech are given by the theory of generative grammar, which also provides
conventional ways of diagramming syntactic structure in the form of inverse
tree diagrams. Accordingly, the gestures that depict aspects of “embedded
sentences” or “dominance” are more or less sketchy (i.e., synecdochic) rendi-
tions of those ready-made visualizations.
In the light of the importance that those who work on multimodal mani-
festations of figurative thought place on the specific materiality and logic of
each modality (e.g., Cienki and Müller 2008; Müller and Cienki, this vol-
ume; Forceville 2006; Kress and van Leeuwen 2006; Kress et al. 2001; Mit-
telberg 2002, 2006), it is interesting to realize that making sense out of what
a speaker-gesturer is trying to convey involves our imaginative abilities as
much as our visual and auditory senses. Interpreting gestures entails combin-
ing perceivable visual and verbal materialized information; but the manual
configurations and movements also appeal to our capacity during the process
of interpretation to assign meaning to empty space and to fill in missing in-
formation, for example, when inferring objects and actions from gestures
involving closed fists, open hands, or lines drawn in the air.
In the multimodal manifestations of metaphor and metonymy examined
above, source and target meanings are not always neatly distributed across
the two modalities (see Forceville 2006), and gesture may be the only modal-
ity in which the metaphor is expressed (especially when it is spatial meta-
phor). Of course, there are also instances in the data in which the speech is
metaphorical but there is no gesture. As we saw, source and target domains
of a mapping are not necessarily co-present in a given instance of multimo-
dal representation: they may need to be inferred by interpretative hypotheses
(Peirce 1991, 1992, 1998) from the discourse and/or physical context
(neighborhood/contexture; Jakobson 1956), or the knowledge of the linguis-
tic theory talked about.
Since gesture is a largely unconscious, spontaneous means of expression,
the multimodal metaphors discussed here can hardly be compared with
elaborated and consciously chosen metaphorical messages in cartoons or
advertisements (see Forceville 1996, 2002, 2005; El Refaie 2003, this vol-
ume; Yus, this volume; Schilperoord and Maes, this volume). And the ques-
tion of whether the linguistic explanations and the linguistically expressed
metaphors could be recognized and understood by the audience without the
gestural support is not answerable on the basis of our data. However, the
interplay between metaphor and metonymy deserves, as has been shown
already (e.g., Bouvet 2001; Forceville 2005; Gibbs 1994; Whittock 1990), a
more detailed scrutiny in other forms of multimodal communication. The
Jakobsonian (and Peircean) notions, combined with contemporary cognitivist
Metonymy first, metaphor second 349
approaches, are a way to account for not only the materialized dimensions of
figures of thought motivating multimodal discourse, but also for their cogni-
tive and imaginative dimensions.
Acknowledgements
We wish to thank the editors of this volume, Charles Forceville and Eduardo
Urios-Aparisi, as well as Alan Cienki and Cornelia Müller for insightful com-
ments on an earlier version of this chapter. We are also grateful to Allegra Gio-
vine, Joel Ossher, and Daniel Steinberg for their help with data coding and to
Yoriko Dixon for providing the artwork.
Notes
References
Barcelona, Antonio
2000a The cognitive theory of metaphor and metonymy. In Metaphor and
Metonymy at the Crossroads: A Cognitive Perspective, Antonio
Barcelona (ed.), 1–28. Berlin/New York: Mouton de Gruyter.
Barcelona, Antonio (ed.)
2000b Metaphor and Metonymy at the Crossroads: A Cognitive Perspec-
tive. Berlin/New York: Mouton de Gruyter.
Bradford, Richard
1994 Roman Jakobson: Life, Language, Art. London/New York:
Routledge.
Bouvet, Danielle
2001 La Dimension Corporelle de la Parole. Les Marques Posturo-
Mimo-Gestuelles de la Parole, leurs Aspects Métonymiques et Mé-
taphoriques, et leur Rôle au Cours d’un Récit. Paris: Peeters.
Calbris, Genevieve
1990 The Semiotics of French Gestures. Bloomington: University of Indi-
ana Press.
2003 From cutting an object to a clear cut analysis: Gesture as the repre-
sentation of a preconceptual schema linking concrete actions to ab-
stract notions. Gesture 3 (1): 19–46.
Carroll, Noel
1994 Visual metaphor. In Aspects of Metaphor, J. Hintikka (ed.), 89–218.
Dordrecht: Kluwers.
Cienki, Alan
1998 Metaphoric gestures and some of their relations to verbal meta-
phoric expressions. In Discourse and Cognition: Bridging the Gap,
Jean-Pierre Koenig (ed.), 189–204. Stanford: CSLI Publications.
2005 Image schemas and gesture. In From Perception to Meaning: Image
Schemas in Cognitive Linguistics, Beate Hampe (ed.), in coopera-
tion with Joseph Grady, 421–442. Berlin/New York: Mouton de
Gruyter.
Cienki, Alan, and Cornelia Müller (eds.)
2008 Metaphor and Gesture. Amsterdam/Philadelphia: Benjamins.
Croft, William
1993 The role of domains in the interpretation of metaphor and meton-
ymy. Cognitive Linguistics 4: 335–370.
Danaher, David
1998 Peirce’s semiotic and Cognitive Metaphor Theory. Semiotica 119
(1–2): 171–207.
Metonymy first, metaphor second 351
Dirven, René
2002 Metonymy and metaphor: Different mental strategies of conceptu-
alisation. In Metaphor and Metonymy in Comparison and Contrast,
René Dirven and Ralf Pörings (eds.), 75–111. Berlin/New York:
Mouton de Gruyter,
Dirven, René, and Ralf Pörings (eds.)
2002 Metaphor and Metonymy in Comparison and Contrast. Berlin/New
York: Mouton de Gruyter.
Forceville, Charles
1996 Pictorial Metaphor in Advertising. London/New York: Routledge.
2002 The identification of target and source in pictorial metaphors.
Journal of Pragmatics 34: 1–14.
2003 Bildliche und multimodale Metaphern in Werbespots. Zeitschrift für
Semiotik 25 (1–2): 39–60.
2005 Visual representations of the Idealized Cognitive Model of anger in
the Asterix album La Zizanie. Journal of Pragmatics 37: 69–88.
2006/this vol. Non-verbal and multimodal metaphor in a cognitivist frame-
work: Agendas for research. In Cognitive Linguistics: Current Ap-
plications and Future Perspectives, Gitte Kristiansen, Michel
Achard, René Dirven, and Francisco Ruiz de Mendoza Ibáñez
(eds.), 379–402. Berlin/New York: Mouton de Gruyter.
Fricke, Ellen
2007 Origo, Geste und Raum – Lokaldeixis im Deutschen. Berlin/New
York: Mouton de Gruyter.
Furuyama, Nobuhiro
2001 De-syntacticizing the theories of reference maintenance from the
viewpoint of poetic function of language and gesture: A case of
Japanese discourse. Ph.D. diss., University of Chicago.
Geeraerts, Dirk
2002 The interplay of metaphor and metonymy in composite expressions.
In Metaphor and Metonymy in Comparison and Contrast, René
Dirven and Ralf Pörings (eds.), 435–465. Berlin/New York: Mouton
de Gruyter.
Gibbs, Raymond W., Jr.
1994 The Poetics of Mind: Figurative Thought, Language, and Under-
standing. Cambridge: Cambridge University Press.
1999 Speaking and thinking with metonymy. In Metonymy in Language
and Thought, Karl-Uwe Panther and Günter Radden (eds.), 61–76.
Amsterdam/Philadelphia: Benjamins.
2003 Embodied experience and linguistic meaning. Brain and Language
84: 1–15.
2006 Embodiment and Cognitive Science. Cambridge: Cambridge Uni-
versity Press.
352 Irene Mittelberg and Linda R. Waugh
Goldin-Meadow, Susan
2003 Hearing Gesture: How Hands Help Us Think. Cambridge,
MA/London: The Belknap Press of Harvard University Press.
Goodwin, Charles
2003 The semiotic body in its environment. In Discourses of the Body,
Justine Coupland and Richard Gwyn (eds.), 19–42. Basing-
stoke/New York: Palgrave-Macmillan.
Goossens, Louis
1995 Metaphtonymy: The interaction of metaphor and metonymy in figu-
rative expressions for linguistic action. In By Word of Mouth: Meta-
phor, Metonymy and Linguistic Action in a Cognitive Perspective,
Goossens, Louis, Paul Pauwels, Barbara Rudzka-Ostyn, A.-S.
Simon-Vandenbergen, and Jan Vanparys (eds.), 159–174. Amster-
dam/Philadelphia: Benjamins.
Hiraga, Masako
1994 Diagrams and metaphors: Iconic aspects in language. Journal of
Pragmatics 22: 5–21.
2005 Metaphor and Iconicity: A Cognitive Approach to Analysing Texts.
Basingstoke/New York: Palgrave-MacMillan.
Jakobson, Roman
1953 Results of a joint conference of anthropologists and linguists. In
Selected Writings II: Word and Language, 554–567.
1956 Two aspects of language and two types of aphasic disturbances. In
On Language, Linda Waugh, and Monique Monville-Burston (eds.),
115–133.
1960 Linguistics and poetics. In Language in Literature, Krystyna Po-
morska, and Stephen Rudy (eds.), 62–94.
1963 Parts and wholes in language. In On Language, Linda Waugh and
Monique Monville-Burston (eds.), 110–114.
1966 Quest for the essence of language. In On Language, Linda Waugh,
and Monique Monville-Burston (eds.), 407–421.
1971 Selected Writings II: Word and Language. The Hague: Mouton.
1987a On the relation between auditory and visual signs. In Language in
Literature, Krystyna Pomorska and Stephen Rudy (eds.), 467–473.
1987b Language in Literature, Krystyna Pomorska and Stephen Rudy
(eds.). Cambridge, MA: Harvard University Press.
Jakobson, Roman, and Krystyna Pomorska
1983 Dialogues. Cambridge, MA: MIT Press.
Johnson, Mark
1987 The Body in the Mind: The Bodily Basis of Meaning, Imagination,
and Reason. Chicago: University of Chicago Press.
Metonymy first, metaphor second 353
Kendon, Adam
2000 Language and gesture: unity or duality? In Language and Gesture,
David McNeill (ed.), 47–63. Cambridge: Cambridge University
Press.
2004 Gesture: Visible Action as Utterance. Cambridge: Cambridge Uni-
versity Press.
Kennedy, John M.
1982 Metaphor in pictures. Perception 11: 589–605.
Köller, Wilhelm
1975 Semiotik und Metapher. Untersuchungen zur grammatischen
Struktur und kommunikativen Funktion von Metaphern. Stuttgart:
Metzler.
Kövecses, Zoltán
2002 Metaphor: A Practical Introduction. Oxford: Oxford University
Press.
Kress, Gunter, and van Leeuwen, Theo
2006 Reading Images: The Grammar of Visual Design. 2nd ed. Lon-
don/New York: Routledge.
Kress, Gunter, Carey Jewitt, Jon Ogborn, and Charalampos Tsatsarelis
2001 Multimodal Teaching and Learning: The Rhetorics of the Science
Classroom. London/New York: Continuum.
Lakoff, George
1987 Women, Fire, and Dangerous Things: What Categories Reveal
About the Mind. Chicago: University of Chicago Press.
1993 The contemporary theory of metaphor. In Metaphor and Thought,
2nd ed., Andrew Ortony (ed.), 202–251. Cambridge: Cambridge
University Press.
Langacker, Ronald W.
1993 Reference-point constructions. Cognitive Linguistics 4: 1–38.
Lodge, David
1977 Two Modes of Modern Writing: Metaphor, Metonymy, and the Ty-
pology of Modern Literature. Ithaca, NY: Cornell University Press.
Mandler, Jean
1996 Preverbal representation and language. In Language and Space,
Paul Bloom, Mary A. Peterson, Lynn Nadel, and Merrill F. Garrett
(eds.), 365–384. Cambridge, MA: MIT Press.
McNeill, David
1992 Hand and Mind: What Gestures Reveal about Thought. Chicago:
Chicago University Press.
2005 Gesture and Thought. Chicago: Chicago University Press.
McNeill, David (ed.)
2000 Language and Gesture. Cambridge: Cambridge University Press.
354 Irene Mittelberg and Linda R. Waugh
Streeck, Jürgen
2002 A body and its gestures. Gesture 2 (1): 19–44.
Taub, Sarah
2001 Language from the Body: Iconicity and Metaphor in American Sign
Language. Cambridge: Cambridge University Press.
Turner, Mark, and Gilles Fauconnier
2002 Metaphor, metonymy, and binding. In Metaphor and Metonymy in
Comparison and Contrast, René Dirven and Ralf Pörings (eds.),
469–487. Berlin/New York: Mouton de Gruyter.
Wales, Katie
2001 A Dictionary of Stylistics, 2nd ed. London: Longman.
Waugh, Linda R.
1998 Semiotics and language: The work of Roman Jakobson. In Hi Fives:
A Trip to Semiotics, Roberta Kevelson (ed.), 85–102. New York: Pe-
ter Lang.
2000 Jakobson, Roman (1896–1982). In Concise Encyclopedia of Prag-
matics, Jacob Mey (ed.) Oxford: Pergamon.
Waugh, Linda R., and Monique Monville-Burston
1990 Roman Jakobson: His life, work and influence. In Jakobson on
Language, Linda R. Waugh and Monique Monville-Burston (eds.),
1–45. Cambridge, MA: Harvard University Press. Updated version
published as 2002, Introduction to Roman Jakobson: Selected Writ-
ings, I, third edition, v-xcviii.
Waugh, Linda R., Norma Barletta, Susan Smith, Elizabeth Specker, Shawn
Steinhart, and Jue Wang
2004 Peircean theory, diagrammatic iconicity and academic texts: Global
structure, abstracts, and the role of narrative. Logos and Language.
Journal of General Linguistics and Language Theory 6: 39–62
(special issue on Aspects of Iconicity in Contemporary Linguistics).
Webb, Rebecca
1996 Linguistic Features of Metaphoric Gestures. Ph.D. Diss., University
of Rochester.
Whittock, Trevor
1990 Metaphor and Film. Cambridge: Cambridge University Press.
Wilcox, Phyllis P.
2004 A cognitive key: Metonymic and metaphorical mappings in ASL.
Cognitive Linguistics 15 (2): 197–222.
Wilcox, Sherman, and Jill Morford
2007 Empirical methods in signed language research. In Methods in Cog-
nitive Linguistics, Monica Gonzalez-Marquez, Irene Mittelberg, Se-
ana Coulson and Michael Spivey (eds.), 171–200. Amsterdam/
Philadelphia: Benjamins.
VI
Lawrence M. Zbikowski
Abstract
This chapter considers the topic of multimodal metaphor from the perspective of
cross-domain mappings between the musical and the linguistic domains. Begin-
ning with an example of what musicans call “text painting” (in which music is
used to “paint” an image related to the text of a vocal work), I explore the differ-
ent ways music and language structure thought. Examples of musical passages
from Palestrina, Biber, Bach, Schubert, and Jerome Kern are used to demonstrate
how music contributes to meaning construction and thus may serve as a source
domain for a multimodal metaphor. I conclude with a brief discussion of how
conceptual blending theory can be used for the analysis of text-music relations,
and the multimodal metaphors that may result.
1. Introduction
Giving voice to an idea that took a number of forms in his later work,
Ludwig Wittgenstein, near the end of his Philosophical Investigations,
wrote, “Understanding a sentence is much more akin to understanding a
theme in music than one may think” (Wittgenstein 2001: frag. 527). The
close relationship between language and music suggested by Wittgenstein’s
observation is borne out by similarities between the two: both are unique to
the human species, both unfold over time, both have syntactic properties,
and both make use of sound. There are also, of course, notable differences:
musical meaning is on the whole much less precise than linguistic meaning;
music often involves simultaneous events, where language does not; and
there is more of a sense of play in ordinary music than there is in ordinary
360 Lawrence M. Zbikowski
language. On the one hand, these differences suggest that language and mu-
sic belong to two different conceptual domains. On the other hand, the simi-
larities between the two suggest that language and music may recruit some
of the same cognitive resources, and that structure from one domain may be
readily mapped to the other to create meaning. Understanding a sentence is
like understanding a musical theme because both language and music offer
possibilities for constructing meaning, possibilities that can be exploited
through multimodal metaphors.
As an example, consider the passage from the Credo of Giovanni Pier-
luigi da Palestrina’s Pope Marcellus Mass (printed 1567) given in figure 1.1
The text Palestrina sets here is “Qui propter nos homines et propter nostram
salutem descendit de cælis” (“Who for us men, and for our salvation, came
down from heaven”). With the first statement of the word “descendit,” each
voice begins a scalar descent. Christ’s descent from heaven is thus repre-
sented with a cascading fall through musical space, a series of overlapping
movements “down” the musical scale. This representation exploits the com-
mon construal of musical pitches as situated in vertical space, a construal
that follows from the characterization of pitches as “high” or “low” with
respect to one another.
As was noted above, there are numerous similarities between language and
music, and there are any number of cultural practices that blur the bounda-
ries between the two modes of communication (Boiles 1967; Fornäs 2003).
It is still the case, however, that we do not live our lives inside the equivalent
of a grand opera or Broadway musical, with every utterance sung and every
action accompanied by an orchestra. So why is it that human cultures have
developed both language and music? Although there is any number of ways
to approach this question, I take the position that language and music have
different functions within human culture. In doing so I am influenced by the
work of the developmental psychologist Michael Tomasello, who situates the
emergence of language in our species within the broader development of
human culture. In Tomasello’s view, the primary function of language is to
direct the attention of another person to objects or concepts within a shared
referential frame (Tomasello 1999: chap. 5). I would argue that music is
similarly part of a cultural framework unique to our species, but one whose
primary function is to represent through patterned sound various dynamic
processes that are common in human experience. Chief among these dy-
namic processes are those associated with the emotions (which, following
recent work by Antonio Damasio, can be construed as sequences of physio-
logical and psychological events that subtend feelings [Damasio 1999,
2003]) and the movements of bodies – including our own – through space.
The difference in function between these two modes of communication is
matched by a difference in the forms through which the functions are real-
ized. In the case of language, this is accomplished through symbolic units
that correlate with functions such as those represented by nouns and verbs,
as well as with the many other parts of speech recognized by grammarians
(Croft 2001: chap. 2). Through the use of these symbolic units we can direct
the attention of another person to objects or concepts within a shared refer-
ential frame.
In the case of music, the basic formal unit is what I call a sonic analog,
which represents through patterned sound the central features of some dy-
namic process.6 “Descent,” for instance, is one such dynamic process: Pal-
estrina, in the example shown above, provides one sonic analog for this
364 Lawrence M. Zbikowski
process; Biber, in the Credo from his mass, provides another. I would hasten
to add that neither musical passage stands for descent in a direct or un-
equivocal fashion. Instead, the phenomenal properties of these passages are
such that they can serve as analogs for the process of descent.7
Given this perspective, it should be clear that mappings between the do-
mains of language and music will involve structures that are fundamentally
different in kind. Language tends to focus on objects (whether real or imag-
ined) and relationships between objects. Language can direct our attention to
a process (that is, the noun “descent” picks out a dynamic process that in-
volves a traversal of space), but it is less common for language to embody
such a process. When it does – when, for instance, we imitate the sound of a
horse’s step with the words “clip-clop, clip-clop” – language starts to be-
come more like music. Music, for its part, does not tend to be involved with
the rich symbolic systems typical of language.8 In those cases where music
does exploit this sort of symbolic system – as, for instance, through musical
topics of the type employed by Mozart and Haydn (Ratner 1980; Allanbrook
1983; Agawu 1991, 1999) – its dynamic aspect tends to recede in impor-
tance. In sum, then, mappings from language to music will tend to focus on
static aspects of the musical domain; mappings from music to language will
draw out the dynamic aspects of the domain of language.
Although any attempt to determine a crisp boundary for what counts as
language or what counts as music may be an endeavor destined to generate
more heat than light, it seems clear that, at least in their characteristic usage
in the contemporary world, language and music have different functions.
While the range of language functions is broad, primary among these is the
use of symbolic tokens to direct the attention of another person to objects
and relations within a shared referential frame. Music, by contrast, provides
sonic analogs for a wide range of dynamic processes that are marked in hu-
man experience, especially those associated with the regulation of emotions.
Multimodal metaphors that involve language and music draw on both of
these resources, as the analyses in the next section shall make clear.
3. Text Painting
the target domain). This mapping structures our understanding of the musi-
cal domain: we hear the sounds as “descending.” But as I suggested in my
introductory comments, the sonic analog provided by the music also shapes
our understanding of the text, for the music gives the delivery of the words a
specific contour and duration.9 Again, given our tendency to give priority to
linguistic domains, the notion that the seemingly indefinite and nonconcep-
tual domain of music could be used to structure thought may seem at best
little more than a passing curiosity, and at worst downright nonsense. But
consider three situations: “descendit” spoken; “descendit” sung by Pal-
estrina’s singers; and “descendit” sung by Biber’s singers. If there are any
differences between these three utterances, they come from the structure
music can impose on language.
Two further examples of text painting can help to elaborate this point.
The first of these comes from the fourth movement of Johann Sebastian
Bach’s Advent cantata “Nun komm der Heiden Heiland” (BWV 61). The
text and music for each of the three preceding movements of the cantata have
all focused on the coming of Christ (as befits the Advent theme). In the
fourth movement Christ is suddenly before us, speaking words from the third
chapter of Revelation: “Behold, I stand at the door, and knock. If any man
hear my voice, and open the door, I will come in to him, and will sup with
him, and he with me.” Bach sets this text as an accompanied recitative for
baritone, with the strings playing pizzicato throughout; a shorthand version
of the string parts and solo bass melody for the first four measures is given
in figure 3.
syllable is repeated. Second, he uses staccato marks on the three notes of the
melisma, which place silences between these notes; these silences are similar
to those that fall between knocks on a door. Third, he sets the words with a
broken chord (or arpeggio). This places a kind of distance between each
successive note but also allows us to hear all as belonging to a single con-
nected gesture.
Again, as an out-and-out imitation of the act of knocking Bach’s setting
of “und klopfe an” leaves something to be desired. Knocks are usually un-
pitched, for instance, but Bach gives us different pitches for each blow.
Knocking is not usually accompanied, but here we have pizzicato strings
pulsing in the background.10 The reason we hear these musical events as
knocking (to the extent that we do) is that our understanding of them is
structured by mapping concepts from language onto music. Again, once we
hear these musical events as the sound of knocking our understanding of the
text starts to change, for the music creates a rather specific analog for the
dynamic process of knocking on a door. This sonic analog relies on the
rhythmic and pitch resources of music: the delivery of “und klopfe an” is at
first halting and then (when it is repeated) hurried; the rhythmic structure of
the passage is coordinated with a notable expansion of pitch space (from the
span of B3 to D#3 in measures 1-2 to the span of C4 to B2 in the first part
of measure 3) which is then compressed by the final F#3-B3-F#3-G3. These
resources in turn shape our understanding of the text. We not only know that
someone is knocking at the door, but how they are knocking: first tentatively,
and then with more urgency.
My final example of text painting is from Franz Schubert’s song
“Gretchen am Spinnrade,” which takes as its text a scene from Goethe’s
Faust. In the scene we overhear Gretchen as, alone in her room, she de-
scribes her love for – or perhaps enchantment by – Faust. The song begins,
as shown in figure 4, with the briefest of introductions by the piano, a swirl-
ing sixteenth-note figure that circles around a D minor chord, a few sparse
and repetitive bass notes sounding beneath. This pattern, or some version of
it, continues throughout the entire 120 measures of the song, with but one
interruption (to which we shall return in a moment). At first glance, the text
(the first lines of which translate as “My peace is gone, / My heart is heavy,
/ I will find it never / and never more”) may seem to have little to do with
this monotonous accompaniment. The link is provided by the title: this is
Gretchen at the spinning wheel. Schubert’s accompaniment is, of course,
meant to evoke the sound of the wheel in action, with the swirling sixteenth
notes summoning the wheel itself and the repetitive, off-beat accents in the
middle voice representing the clack of the bobbin, but for modern listeners
368 Lawrence M. Zbikowski
this sonic image will not typically resolve itself until we have mapped con-
ceptual structure from the domain of the text onto the domain of the music.
The text thus serves a function similar to what Roland Barthes has called
“anchoring,” rendering transparent an aural image that might otherwise re-
main opaque.
focus here will not be on one moment, or even one basic image, but on con-
cepts that are developed over the course of an entire song.
Figure 5. Lead sheet for Jerome Kern and Dorothy Field’s “The Way You Look
Tonight” © Polygram International Publishing, Inc., and Aldi Music.
Jerome Kern and Dorothy Field’s “The Way You Look Tonight” is typical
of many of the tunes produced during what some have called the Golden Age
of American song. As can be seen from the lead sheet given in figure 5, Kern
chose an AABA form for the tune. That is, the music for the first sixteen
measures (the A section) is re-used with little alteration for the second six-
teen measures. Contrasting music (the B section, often called the bridge) is
introduced in the third sixteen-measure unit (measures 33-48), and this is
then followed in measure 49 by a return of the A section (in a slightly modi-
fied form, with a “tag” added at the end). Kern’s musical form meshes with
that of Field’s lyrics, in which the third stanza contrasts with the others in
both the length of its lines and its rhyme scheme. In the following, I would
like to concentrate on the first verse and explore the mental space set up by
372 Lawrence M. Zbikowski
the lyrics, the mental space set up by the music, and then the blended space
that results from their combination.
Fields’s lyrics open with a characteristic space builder: “Someday.”13
Here, the space builder establishes a mental space focused on a future state
of affairs rather than on the present. The followings lines fill out the picture:
the speaker, beset with rather dire circumstances (“When I’m awfully low /
And the world is cold”) will be comforted by the remembrance of the object
of his affections and, more specifically, by the way she looks on this particu-
lar night (“I will feel a glow just thinking of you / And the way you look
tonight”). As is hinted at by the transformative effect of the appearance of
the beloved (an effect confirmed by the second verse’s “There is nothing for
me but to love you”), what is involved here is not simply a kind of passive
looking, with one person gazing on another, but an intimacy of association
that has both power and depth. The mental space established by the first
verse thus develops into a scene in which what is of moment is not some
future opportunity to look back to the present as a golden past but the cen-
trality of “the way you look tonight” to a highly charged romantic relation-
ship. The “Someday” space builder is thus somewhat misleading: what is
important is not the future but the present.
The melody for the song also begins with a space builder, but in this case
the falling fifth A5-D4. Although these pitches could be understood in a
variety of ways, the simplest interpretation (and one supported by the open-
ing D major chord) is as the fifth and first notes in a D scale. In the music
that follows the registral space between D4 and A5 is filled in by a sequence
of arch-like figures that flesh out the musical topography with notes from the
key of D major (rather than, for instance, D minor). These figures ultimately
move past A5 (in measure 7) and arrive on the high note of the melody (D5)
of measure 9. This arrival coincides with a return to the rhythm of the open-
ing gesture, but with the A5-D4 falling fifth replaced by a D5-D4 descend-
ing octave. It is worth noting that while the D4 of measure 2 and the D4 of
measure 10 are the same pitch, their context is quite different: the registral
space above the D4 of measure 10 has been expanded through the sequential
figures of measures 3-8, and the temporal space between the two whole
notes has been filled in by the moving quarter-notes in these same measures.
Measure 10 is followed by a final passage that owes something to the com-
positional strategies that filled out the musical space in measures 3-8, but
which now lead directly to the D4 that concludes the melody of the A section
in measure 13. From a musical perspective, then, the song opens with a
space builder roughly equivalent to the “Someday” of the words, after which
Music, language, and multimodal metaphor 373
the space is filled out – with notes instead of words – to solidify and stabilize
the conceptual realm prompted by the space builder.
The melodic process that leads to the arrival on D4 in measure 13 is rein-
forced by rhythmic and harmonic processes that are to some extent inde-
pendent of the melody.14 Over the course of measures 1-8 the number of
shorter-duration notes increases until it reaches its maximum density in
measures 7-8. This is followed (as has already been noted) by the whole
notes of measures 9-10. The contrast between shorter- and longer-duration
notes is then revisited in a more orderly fashion, with the quarter notes of
measure 11, half notes of measure 12, and whole note of measure 13 (tied
over into the quarter note of measure 14). While the harmonies used by Kern
– a four-measure pattern repeated (with slight variation) four times – are
quite typical in American popular music (what musicians would call a I-VI-
II-V pattern), two details contribute to the overall dynamic shape of the A
section. First, although the D major harmony of measure 5 represents a re-
turn to the opening harmony, the music is kept moving by the melodic se-
quence that accompanies this harmony. Second, although D major returns
once more in measure 9 (and supports long notes that recall the long notes of
the opening measures), Kern destabilizes the chord by turning it into a domi-
nant seventh. The overall effect of all of these processes – melodic, rhyth-
mic, and harmonic – is to make measures 11-13 a goal for the A section, a
goal whose culmination is the D4 of measure 13, the note which correlates
with the arrival on the last syllable of “The way you look tonight.”
Essential features of the mental spaces set up by the lyrics and the music
for the first verse of “The Way You Look Tonight” are given in the concep-
tual integration network diagrammed in figure 6, with each mental space
represented by a circle. Where the mental space set up by the text is con-
cerned primarily with objects and relations (namely, the appearance of the
beloved and its importance for “romance”) the mental space set up by the
music is concerned with a set of coordinated processes that lead to the final
phrase in the section. When these two spaces are correlated with one another
(as they are in the song) aspects of their structure are projected into the
blended space to yield a dynamic representation of the development of an
intimacy of association. Guiding the integration of these concepts is the ge-
neric space (represented in the top circle of the diagram), which defines the
core cross-space mapping and basic topography for the network. The generic
space, which reflects the insights captured in the invariance principle (Lakoff
1990; Turner 1990), is centered on the idea that focused attention is a form
of intimacy. This sort of attention is behind the fixed gaze of the lover, and it
374 Lawrence M. Zbikowski
is also behind our discomfort when confronted with the unbending stare of a
stranger.
Figure 6. Conceptual integration network for the first verse of “The Way You
Look Tonight.”
The development of this sort of intimacy was at the heart of the scene from
the 1935 movie Swing Time (starring Fred Astaire and Ginger Rogers)
where “The Way You Look Tonight” first appeared. In this scene Astaire is
trying to win Rogers over: he has managed to get in to her hotel suite, and,
even though she has repeatedly spurned him, gives it his best effort by sing-
ing this song (“accompanying” himself at the piano). Rogers, for her part,
has locked herself in her room and has started shampooing her hair. As the
song unfolds, however, she emerges from her room in a bathrobe with hair
lathered, first smiling on Astaire and then soundlessly walking over to stand
behind him at the piano, where she rests her hand on his shoulder. This ges-
ture coincides with Astaire’s arrival at the music of measure 61 (the penul-
timate statement of “The way you look tonight”), and in response to it he
sings the words one more time, turning to look into Rogers’ eyes. At the
conclusion of the song his gaze becomes quizzical as he notices the lather on
her head and she, observing this change, turns to a mirror and discovers her
appearance. This comedic moment is, of course, a play on “the way you look
Music, language, and multimodal metaphor 375
tonight,” but Astaire and Rogers have just shared an encounter typical of a
much more intimate relationship than they have enjoyed thus far. This inti-
macy is one that is worked out not only in the first verse of Kern and Field’s
song, but over its entire course. The second verse moves from generalities of
appearance to specifics (“With your smile so warm / And your cheek so
soft”), the bridge (with its music momentarily suspending the process en-
acted by the first two A sections) adds the detail of “that laugh that wrinkles
your nose,” and the final verse finds the singer speaking directly to the be-
loved: “Never, never change.”
From the perspective of conceptual blending theory, then, in some cases
words and music will prompt the construction of two independent but corre-
lated mental spaces. Both of these spaces contribute structure to a third men-
tal space, in which concepts drawn from each of these two input spaces are
blended. This new space typically serves as a site for the imagination. For
the conceptual blend created by the words and music for “The Way You
Look Tonight,” we might well imagine that the intimacy established between
the lovers is one that would lead them to dance together, or to exchange lov-
ing words, or perhaps just to stare into each other’s eyes. None of these pos-
sibilities concludes the scene from Swing Time – Rogers, aghast at her ap-
pearance, rushes back to her boudoir – but subsequent scenes do make clear
that Rogers’ and Astaire’s characters are now a pair.
The mappings associated with a conceptual blend of this sort are different
from those associated with a metaphor in two important ways. First, blend-
ing typically involves highly fluid and thoroughly pragmatic mental spaces
rather than established domains. Metaphorical mappings often yield systems
of metaphor; while blends may exploit such systems, they may also destabi-
lize them by extending the system in novel ways, and thus push against the
boundaries of the domain. Second, rather than one domain (the source) pro-
viding structure for the other (the target) – a mapping that gives rise to the
directionality of metaphor noted above – in a blend correlated spaces each
contribute to structure that is mapped onto the blend. But whether metaphor
or conceptual blending is involved, I hope I have demonstrated the resources
for meaning construction provided by these two different modalities. Most
readers will find mapping from language to music to be simple enough, not
the least because language is the primary means most humans use to struc-
ture their understanding of the world. Mapping from music to language
might seem a stranger alternative, but I believe it is a real possibility, espe-
cially when language is for some reason ambiguous (as it is in the opening of
Schubert’s “Gretchen am Spinnrade”). Additional evidence that music can
serve as a proper conceptual domain, and thus have at least the potential to
376 Lawrence M. Zbikowski
5. Conclusion
language and music, for such metaphors give us compelling insights into the
rich and varied world of meaning within which humans dwell.
Notes
12. For an overview of the theory see Fauconnier and Turner (1998, 2002). For a
review of the latter, see Forceville (2004).
13. Fauconnier describes a space builder as follows: “Linguistic expressions will
typically establish new spaces, elements within them, and relations holding
between the elements. I shall call space-builders expressions that may estab-
lish a new space or refer back to one already introduced in the discourse”
(1994: 17).
14. Music scholars often analyze music in terms of three primary parameters:
melody, harmony, and rhythm. In the same way that the very notion of “mu-
sic” varies broadly across cultural practices, the manifestation of these pa-
rameters is not always obvious or unequivocal. In “The Way You Look To-
night,” however, both the harmony (indicated by the chord symbols above the
staff) and the rhythmic frame (indicated by the notated durations of pitches
and by barlines) are relatively clear-cut.
References
Damasio, Antonio R.
1999 The Feeling of What Happens: Body and Emotion in the Making of
Consciousness. New York: Harcourt Brace.
2003 Looking for Spinoza: Joy, Sorrow, and the Feeling Brain. Orlando,
FL: Harcourt.
Deacon, Terrence W.
1997 The Symbolic Species: The Co-Evolution of Language and the
Brain. New York: W.W. Norton.
2003 Universal grammar and semiotic constraints. In Language Evolu-
tion, Morten H. Christiansen, and Simon Kirby (eds.), 111–139.
Oxford: Oxford University Press.
Fauconnier, Gilles
1994 Mental Spaces: Aspects of Meaning Construction in Natural Lan-
guage. 2nd ed. With a foreword by George Lakoff and Eve Sweet-
ser. Cambridge: Cambridge University Press.
Fauconnier, Gilles, and Mark Turner
1998 Conceptual integration networks. Cognitive Science 22: 133–87.
2002 The Way We Think: Conceptual Blending and the Mind’s Hidden
Complexities. New York: Basic Books.
Forceville, Charles
2004 Review of Gilles Fauconnier and Mark Turner, The Way We Think:
Conceptual Blending and the Mind’s Hidden Complexities. Meta-
phor and Symbol 19: 83–89.
Fornäs, Johan
2003 The words of music. Popular Music and Society 26: 37–51.
Goldin-Meadow, Susan
2003a Hearing Gesture: How Our Hands Help Us to Think. Cambridge,
MA: Harvard University Press.
2003b The Resilience of Language: What Gesture Creation in Deaf Chil-
dren Can Tell Us About How All Children Learn Language. New
York: Psychology Press.
Holyoak, Keith J., and Paul Thagard
1995 Mental Leaps: Analogy in Creative Thought. Cambridge, MA: MIT
Press.
Keller, Hans
1994 The musical analysis of music. In Essays on Music, Christopher
Wintle (ed.), 126–28. Cambridge: Cambridge University Press.
Kendon, Adam
2004 Gesture: Visible Action as Utterance. Cambridge: Cambridge Uni-
versity Press.
Lakoff, George
1990 The invariance hypothesis: Is abstract reason based on image-
schemas? Cognitive Linguistics 1: 39–74.
380 Lawrence M. Zbikowski
Charles Forceville
Abstract
1. Introduction
music can play a role in the construal and interpretation of multimodal meta-
phor. Its aim is to chart parameters that need to be taken into account in a
full-fledged theory of multimodal metaphor by discussing ten case studies.
Specifically, I will focus on (a) the role of sound and music to (help) identify
the metaphor’s source domain; (b) the role of sound and music to (help)
identify features that can be mapped from source to target. Five examples
originate in advertising, with its clearly specifiable genre-convention of at-
tempting to persuade an audience of positive qualities adhering to a certain
product, brand, or service; and five are fragments from fiction films, a genre
which is supposed – let us say with Horace – to delight, instruct, and move.
The analyses are guided by the three questions that need at the very least be
capable of being answered for something to count as a metaphor:
(1) What are its two domains?
(2) What is its target domain, and what its source domain?
(3) Which feature or (structured) cluster of features can/must be mapped
from source to target? (see Forceville 1996: 108)
2. Case studies
The fragment (composed by Sander Baas and Iwan den Boestert) sounds like
a hit song of the kind popular in Holland at the time of writing, but was in
fact specifically composed for the commercial (see http://www. megamedia-
magazine.nl/mvtr.php [accessed 9 July 2008] under KPN Telecom “Volsla-
gen Mobiel”).
The role of non-verbal sound and music in multimodal metaphor 385
Two, three, right! … And up! …. And relax”). This text evokes the domain
of doing fitness exercises, but so does the rhythmic beat accompanying the
voice-over. Thus the target domain is cued visually alone. If the sound is
switched off, and all the verbal cues are (mentally) eliminated, the personifi-
cation of the cookie might still be inferred by some viewers: after all it seems
clear that the cookie jumps to and fro entirely of its own accord. However,
the specification of the source as not simply a person but as a person doing
fitness exercises is cued by the voice-over text as well as by the rhythmic
beat. Each of these would suffice alone for this specification, although no
doubt their combination facilitates and probably quickens comprehension of
the metaphor COOKIE IS PERSON DOING FITNESS EXERCISES.
Figure 2. A Tuc cookie jumps up and down like a fitnesser. Text: “But Tuc is most
of all a lot more tastier” (still from Tuc Commercial, The Netherlands).
Example 3 (Shell): CARS ARE FISH. The images in the commercial are those
of beautifully colored, animated fish apparently swimming just above the
bottom of the sea (reminiscent of the animation film Finding Nemo, USA
2003). We see a school of fish all stopping in mid-swim (figure 3), then
moving on, and a small fish darting away just in time from a swordfish try-
ing to stab it. The sounds we hear are traffic sounds – revving motors,
screeching tires, claxons, a siren. The voice-over reinforces the traffic do-
main:
What is the advantage, in everyday traffic, of a gasoline that has been de-
veloped in collaboration with the people at Ferrari’s? V-Power is a new
gasoline that ensures better performance. Thanks to a better combustion.
So that your car can respond faster … when necessary. And where do you
find a gasoline that guarantees such a good performance as V-Power?
Shell. Where you stop, we go further.
The role of non-verbal sound and music in multimodal metaphor 387
Figure 3. Fish halt suddenly as if cars stopping before traffic lights, suggested by
street sounds (still from Shell commercial, The Netherlands).
The traffic domain is thus cued by both non-verbal sound and spoken lan-
guage. Once the audience has accessed this domain, some movements of the
fish, for instance their completely synchronized stopping and moving, can be
interpreted as signaling the traffic domain (here: stopping before a traffic
light), but these visual cues alone would not suffice to ensure its recognition.
Moreover, the sound adds liveliness and precision to the images: for in-
stance, the little fish’s escape from the swordfish is emphasized not just
visually by the quick movement, and verbally by the brief pause before
“when necessary,” but also by the revving sound. Since speed embodies a
quality claimed to be facilitated by the product advertised (V-Power), this is
meaningful.
In view of the claim that in principle metaphors, irrespective of the mo-
dalities they draw on, do not allow for reversal of target and source (Force-
ville 1995, 2002), it is interesting that in this metaphor its distribution is not
immediately clear. In the first instance, before the voice-over is audible, I
suspect viewers hypothesize that the fish are, or belong to the domain of, the
product promoted. That is, they might at this stage speculate, for instance,
that this is a commercial for a zoo, or perhaps an amusement park with an
aquarium. At the moment they hear the words “traffic” and “gasoline,”
words that strongly connote the realm of cars and driving, the assumption
that the fish are to be taken literally is probably discarded. Indeed, it is not
until this moment that viewers will reinterpret the fish as the source domain
of what is to be construed as a metaphor. I submit that just as inferences
made on the basis of the visual track tend to prevail over those made on the
basis of the non-verbal soundtrack, inferences on the basis of verbal lan-
388 Charles Forceville
guage override those originating in the visual track. It is thus the verbal
track, supported by the traffic sounds, that makes viewers postulate the
metaphor CARS ARE FISH. The target-status of cars is further reinforced once
we hear the name of the brand advertised: Shell – and see one of a series of
shells on the sea bottom transform into the Shell logo.
Example 4 (Iglo): CORNCOB AND FRENCH BEAN ARE BRIDE AND
GROOM. An Iglo instant-meal commercial features a mini-corncob and a
French bean, walking together toward an Iglo package (figure 4). Their
movements alone already suggest anthropomorphizing, but it is the tune of
the Wedding March which metaphorizes the two into bride and bridegroom
(and makes us realize that the Iglo package actually resembles a church). A
voice-over tells us:
In our newest Iglo dish there are mini-corncobs and French beans – a com-
bination of young and crispy. And what should of course never be missing
on such a joyful occasion? … Exactly! … And together with chicken filet
and soy sauce these make up the delicious new Iglo dish.
Figure 4. A min-corncob and a French bean walk as a married couple, the Wed-
ding March being audible on the sound track (still from commercial for
Iglo meals).
At the moment the male voice-over says “exactly” we hear a faint voice say
“yes, now!”, followed by a kissing sound, and we see, under cheers and ap-
plause, rice showered upon the couple. The metaphor CORNCOB AND
FRENCH BEAN ARE BRIDE AND GROOM is cued mainly by the Wedding
March tune, since the anthropomorphizing of the two does not suffice to turn
them into a wedding couple. If the sound were turned off, most viewers
would not access the source domain on the basis of visual cues until the rice
The role of non-verbal sound and music in multimodal metaphor 389
shower, if then. Note that despite the reference to a “joyful occasion,” the
verbal text alone, too, would not be sufficient to cue WEDDING at all. That
is, there is no unambiguous verbal reference to the source domain. Both
target and source are cued pictorially, but it is the Wedding March tune that
turns the vegetables not just into humans, but into bride and groom – even
before we see the rice-shower.
Example 5 (Senseo): COFFEE MAKER IS MOTORCYCLE. A Philips coffee
maker, Senseo, shows the metallic machine first in a number of extreme
close-ups defying recognition. We hear a throbbing motorcycle engine and a
fragment of Steppenwolf’s Born to be Wild featuring the line “Get the motor
running, hit it on the highway” – made famous by the opening sequence of
Easy Rider (Dennis Hopper, USA 1969). The metaphor COFFEE MAKER IS
MOTORCYCLE is further emphasized by concurrently showing a finger push-
ing the coffee maker’s on/off button and sounding a kick-starting motorcy-
cle. The shiny black covering of the coffee machine, moreover, looks like the
surface of a motorcycle and the dripping coffee resembles a drop of oil (fig-
ure 5). The following phrases are superimposed, one after the other, on the
images: “designed with a vision … developed with passion … makes of each
moment a sensation … that delights all senses.” The metaphor in this com-
mercial is cued at least as much aurally as visually. One mappable feature is
clearly the high-tech design, but more importantly the music evokes connota-
tions such as living life in the fast lane, freedom, unconventionality, youth,
sixties’ counterculture – a whole range of qualities nostalgically associated
with Easy Rider motorbiking that are potentially mapped to making your
coffee with a Senseo machine. That these connotations are considered impor-
tant by the commercial’s maker is confirmed by the written pay-off line at
the end of the commercial: “three years old, and already a legend – at least in
the kitchen.” Switching off the sound of the commercial presumably elimi-
nates the most important cue for the source domain, Easy Rider motorbike.
This commercial, therefore, is best classified as a multimodal metaphor. (I
owe this example and part of the analysis to Victor 2004.)
Watching commercials is governed by the strong genre-expectation that
some product, service, or brand name is promoted, which enormously facili-
tates and constrains the preferred interpretation of everything visible and
audible in them, including the metaphors. In most metaphors in commercials,
the product is the target and the source is something else, which means that
it is positive features that are mapped from source to target (Forceville 1996:
104). However, it is important not to theorize multimodal metaphor exclu-
sively on the basis of case-studies exemplifying a single genre, since this
might lead to a mistaken conception of pictorial and multimodal metaphor’s
390 Charles Forceville
Figure 5. A drop of what looks like oil – an impression reinforced by the accom-
panying “Born to be Wild” song – is in fact a drop of coffee (still from a
commercial for Senseo, The Netherlands).
Example 6 (The General): REAL CANNON IS CIRCUS CANNON. Buster Kea-
ton’s brilliant film The General (1927), set during the American Civil War,
features Keaton as the train driver Johnnie Gray, whose locomotive is stolen
by enemy Northern soldiers. Johnnie gets hold of another locomotive, hitches
on to the loc a wagon with a cannon, and single-handedly goes in pursuit
along the same railway track. When he has the Northerners in sight, he loads
the cannon (figure 6), ready to fire over the cabin of his own loc. Just before
Johnnie fires the cannon, we hear the soft-and-quick drum on the soundtrack
that we recognize as a cliché device to create tensive expectation, while the
firing of the canon ball itself is accompanied by a very unrealistic popping
sound (the music and sound of the version discussed here is by Konrad Elf-
ers). As such, there is nothing much metaphorical about this, but the conven-
The role of non-verbal sound and music in multimodal metaphor 391
tional drums anticipating the firing will no doubt remind many viewers of a
similar situation in circus acts, where a “human cannon ball” is fired. Hence
construal of the metaphor REAL CANNON IS CIRCUS CANNON is invited. The
mapped features in this case are the connotations evoked by a circus context,
such as pleasurable excitement, risk-taking, the idea of watching a perform-
ance – and of course the mapping onto real-cannon firing turns Johnnie’s
action into slapstick.
Incidentally, the fact that this film predates the sound era (Al Johnson’s
The Jazz Singer, 1927, is conventionally credited with being the first sound
film), has an interesting consequence from the point of view of metaphors
involving music and sound. In the pre-sound era, “the typical program had
musical accompaniment. In the more modest presentations, a pianist might
play; in vaudeville theatres, the house orchestra provided music” (Thompson
and Bordwell 1994: 13). This, then, gave ad-libbing musicians opportunities
to create multimodal metaphors involving sound where these may not have
been envisaged by the films’ makers.
Figure 6. Johnnie (Buster Keaton) loads a cannon, while tensive “circus-act mu-
sic” is audible (still from Buster Keaton, The General, USA 1927).
Example 7 (If …): PUBLIC SCHOOL IS ARMY. Lindsay Anderson’s If .... (UK
1968) is a satire on the perverse aspects of British public school life. The
verbal and visual references to battle and war, combined with the violence
and the expectation of blind discipline in a hierarchical system gives rise to
the metaphor PUBLIC SCHOOL IS ARMY – a conceptual metaphor that is rein-
forced throughout the film. Starting out realistically, the film gradually be-
gins to show bizarre and surrealistic events that, commensurate with the
counterfactuality suggested by the title, destabilize the status of the “real.”
Indeed, the film increasingly literalizes the metaphor, resulting in a climactic
392 Charles Forceville
Figure 7. The massacre in the college quadrangle (still from If …, Lindsay Ander-
son, UK 1968).
Example 8. (The Godfather I): MENTAL STATE IS FAST TRAIN. Michael (Al
Pacino) in The Godfather I (Francis Ford Coppola, USA 1972) prepares to
take revenge on his family’s enemies, Sollozzo and McCluskey. He has
planned for a gun to be hidden in the toilet in the restaurant where they will
be having dinner. Since he is not an experienced killer at this stage of his
life, this is an emotional moment for Michael. When he returns from the
toilet, with the gun, the sound track features a rumbling sound, which be-
The role of non-verbal sound and music in multimodal metaphor 393
comes identifiable as that of a riding train. The sound is in this scene non-
diegetic, that is, it does not emanate from the actual events portrayed at this
moment. That this is so is made clear by the fact that the sound is not con-
tinuous: it is audible in the shots when we see Michael in the toilet, but not in
the shots of Sollozzo and McCluskey left behind at the restaurant table. The
sound is thus used to convey Michael’s mental state, suggesting the meta-
phor MICHAEL’S MENTAL STATE IS FAST TRAIN. Possible mappings are such
a train’s unstoppability, the inexorable rhythm of its progress, the circum-
stance that its noise drowns out other sounds – which in the target domain
translate as, say, Michael’s determination, or his refusal to reconsider his
plan to kill. Lena Chatzigrigoriou (to whom I owe the example and part of
the analysis) interprets the scene as follows: “The sound of the shrieking
train breaks gradually, overpowering Sollozzo’s voice. This sound tells us
that Michael is not listening anymore, he is ready for action” (Chatzigri-
groriou 2006: 13). That the sound is a train’s is confirmed in the DVD
commentary track by Francis Ford Coppola, who observes that there was an
elevator train in the vicinity of the restaurant where the scene was shot, and
that it was, in fact, the sound of this train that provided the idea for its usage
in the scene. This comment also suggests that we indeed already heard the
same train sound earlier, but then diegetically, namely during the car trip to
the restaurant. As in the If … scene discussed in example 7, then, the sound
used as source domain in the metaphor had been cued realistically in an ear-
lier scene.
Even if the analysis is accepted, it is nonetheless clear that presumably
few spectators will consciously construe the concept TRAIN as a metaphori-
cal source domain in this highly suspenseful scene. They may, instead, con-
strue the sound more generically as “swelling rumble” or something like that.
This leaves intact the claim that the sonic source domain brings to the fore to
the audience what Michael thinks – which is not directly made visible (al-
though Al Pacino’s facial expression arguably also helps the audience inter-
pret what he is thinking). In Indurkhya’s (1991) terms, this would be an
example of a “projective metaphor” (Indurkhya 1991: 16), a type in which a
source domain structures a largely unstructured target domain. But the ex-
ample points up an issue that is particularly pertinent to multimodal meta-
phors whose source, for whatever reason, is difficult to verbalize unambigu-
ously. Should we say that members of the audience who failed to cue TRAIN,
but who did cue, say, SWELLING RUMBLE, and somehow mapped associa-
tions adhering to that source to the target MICHAEL’S MENTAL STATE have
or that they have not processed the metaphor? Or have they processed differ-
394 Charles Forceville
ent metaphors? (See Bartsch 2002 and some responses to Bartsch in Force-
ville 2006.) This issue deserves sustained discussion.
Figure 8. Adam as dog, biting the hand of his creator (still from Adam, Peter
Lord, UK 1991).
Example 9 (Adam and Remember to Keep Holy the Sabbath Day): HUMAN
BEING IS DOG. Animation films are rich in metaphors in which sound plays a
role. A simple example is the juxtaposition of a human(oid) creature and a
recognizable animal sound. Adam (Peter Lord, UK 1991) is a short clayma-
tion which playfully refers to the fact that the eponymous hero derives its
name from Hebrew “Adamah,” meaning reddish clay. Its Godlike animator
keeps ordering Adam about on his miniature planet earth. At one moment
Adam, confused what behavior his creator expects of him, goes down on all
fours, barks, and snaps at his creator (figure 8), so that for a moment the
viewer is invited to entertain the metaphor ADAM IS DOG. A similar situation
occurs in one of Phil Mulloy’s bleak animations, “Remember to Keep Holy
the Sabbath Day” in the Ten Commandments series: Ezekiel Mittenbender
kneels down and begins to bark. Note, incidentally, that the metaphor is used
for different narrative purposes: in the first case, the metaphor is deployed to
indicate the hero’s temporary confusion; in the second, to convey his insan-
ity. Furthermore, while redundant to cue the source domain “animal” (the
fact that both Adam and Ezekiel go down on all fours provides sufficient
visual cues for that), the barking narrows down the animality to that of a dog
– thus potentially activating in the audience a whole range of connotations
adhering to DOG – and hence contributes information not available in the
image track.
Example 10 (Robin Hood): TENT IS TRAIN. In the animation film Robin
Hood (Wolfgang Reitherman, USA 1973) at one moment a group of crea-
tures in a tent scurries over a plain, only their feet being visible. The phe-
The role of non-verbal sound and music in multimodal metaphor 395
scene it was diegetically motivated music, which in the latter scene has be-
come non-diegetic (but recalls its diegetic use). The same holds true for the
fragment from The Godfather, in which the elevator train had been diegeti-
cally audible in an earlier scene. The effect, I propose is similar to the effect
in the commercials: it reduces the sense of artificiality necessarily associated
with the presentation of a source domain that has no realistic motivation
whatsoever.
Bordwell and Thompson point out that “sound can achieve very strong
effects and yet remain quite unnoticeable” (1997: 315). Inasmuch as sound
is less consciously registered than images, metaphors with sonic source do-
mains may exercise their persuasive or narrative influence more subtly than,
for instance, metaphors whose terms are both presented in pictorial terms.
Experimental research on multimodal metaphors with a sound dimension in
which sound tracks are suppressed or altered (see Chatzigrigoriou 2007) is
imperative to gain more insight into the working of sonic metaphors.
Not all examples presented here are metaphorical with a high degree of
explicitness. As always, the signals used to cue the source domain must be
comprehensible by the envisaged audience for a metaphorical construal to be
possible in the first place. In fact, one could venture that, because they never
have a verbal “is” to explicitize them, multimodal metaphors tend to be less
explicit than purely verbal ones (for more discussion on implicitly versus
explicitly signaled metaphors, see Forceville 1999b).
Finally, the genre to which a representation belongs steers the possible or
most plausible interpretations of any element in it, including metaphor (see
Charteris-Black 2004 for corpus analysis of metaphors in genres such as
sports news, political manifestos, and religious texts in the Bible and the
Koran; Caballero 2006 for an in-depth examination of metaphor use in ar-
chitectural building reviews; and Caballero, this volume, for their use in
wine-tasting notes). Systematic investigation of sonic metaphors in different
genres (e.g., commercials, art films, horror films, video clips, computer
games), or sequences within films (dreams, hallucinations, flash-backs) may
reveal patterns specific for such genres or sequences.
Notes
1. This chapter is a revised and expanded version of a paper that appeared ear-
lier of Forceville (2004). I am indebted to Eduardo Urios-Aparisi for com-
ments on an earlier draft of this new version.
398 Charles Forceville
2. The melody of the song is that associated with J. Threlfall’s “Hosanna, Loud
Hosanna,” but the words sung, so far as they can be deciphered, do not fit
that hymn’s text. Lindsay Anderson himself wrote about the melody that he
“had originally asked Marc Wilkinson to write some music for the final on-
slaught, where Mick alone on the roof tries to hold at bay the attacking forces
of Establishment, but we found when we played with the sound tracks in the
cutting room that a simple organ version of the College song, which fortu-
nately I had recorded when we were on location, fitted the sequence much
better” (Anderson 1975). Apparently, then, the hymn’s melody was used for
a college song, whose text I have not been able to locate. I am indebted to
Thomas Elsaesser, Erik Hedling, and Andrew Webber for help in my hunt
for clues on the song.
References
Anderson, Lindsay
1975 Using music. [Typed draft, Library collection University of Stirling,
UK] http://www.is.stir.ac.uk/libraries/collections/anderson/music.
php (last accessed 8 July 2008).
Bartsch, Renate
2002 Generating polysemy: Metaphor and metonymy. In Metaphor and
Metonymy in Comparison and Contrast, René Dirven and Ralf
Pörings (eds.), 49–74. Berlin/New York: Mouton de Gruyter.
Bensdorp, Thomas, and Arjan Vergeer
2004 Geluid en de Populaire Animatiefilm: Een Cognitivistisch
Onderzoek naar Betekenisvorming. [Sound and the popular anima-
tion film: A cognitivist approach to meaning formation.] MA diss.,
Department of Media Studies, University of Amsterdam, The Neth-
erlands.
Bordwell, David, and Kirstin Thompson
1997 Film Art: An Introduction. 5th ed. New York: McGraw-Hill.
Caballero, Rosario
2006 Re-Viewing Space: Architects’ Assessment of Built Space. Berlin/
New York: Mouton de Gruyter.
this vol. Cutting across the senses: Imagery in winespeak and audiovisual
promotion.
Carroll, Noel
1994 Visual metaphor. In Aspects of metaphor, Jaakko Hintikka (ed.),
189–218. Dordrecht: Kluwer.
1996 A note on film metaphor. In Theorizing the moving image, 212–
223. Cambridge: Cambridge University Press.
The role of non-verbal sound and music in multimodal metaphor 399
Charteris-Black, Jonathan
2004 Corpus Approaches to Critical Metaphor Analysis. London: Pal-
grave-MacMillan.
Chatzigrigoriou, Eleni
2006 Take-home exam pictorial and multimodal metaphor. Manuscript.
Department of Media Studies, University of Amsterdam, The Neth-
erlands.
2007 Trans-Diegetic Sound: In-Between Spaces in Post-Classical Cin-
ema. MA diss., Department of Media Studies, University of Am-
sterdam.
Eggertsson, Gunnar Theodór
2006 Animal Horror: An Investigation into Animal Rights, Horror Cin-
ema and the Double Standards of Violent Human Behaviour. MA
diss., Department of Media Studies, Universiteit van Amsterdam.
Eggertsson, Gunnar Theodór, and Charles Forceville
this vol. The HUMAN VICTIM IS ANIMAL metaphor in extreme horror films.
El Refaie, Elizabeth
this vol. Metaphor in political cartoons: Exploring audience responses.
Fokkema, Douwe, and Elrud Ibsch
2000 Knowledge and Commitment: A Problem-Oriented Approach to
Literary Studies. Amsterdam/Philadelphia: Benjamins.
Forceville, Charles
1995 (A)symmetry in metaphor: The importance of extended context.
Poetics Today 16: 677–708.
1996 Pictorial Metaphor in Advertising. London/New York: Routledge.
1999a Art or ad? the influence of genre-attribution on the interpretation of
images. SPIEL 18: 279–300.
1999b The metaphor COLIN IS A CHILD in Ian McEwan’s, Harold Pinter’s,
and Paul Schrader’s The Comfort of Strangers. Metaphor and Sym-
bol 14: 179–98.
2002 The identification of target and source in pictorial metaphors.
Journal of Pragmatics 34: 1–14.
2003 Bildliche und multimodale Metaphern in Werbespots [Tr. from
English by Dagmar Schmauks] Zeitschrift für Semiotik 25: 39–60.
2004 The role of non-verbal sound and music in multimodal metaphor. In
Words in their Places: A Festschrift for J. Lachlan Mackenzie,
Aertsen, Henk, Mike Hannay, and Rod Lyall (eds.), 65–78. Amster-
dam: Faculty of Arts, VU Amsterdam.
2005 Cognitive linguistics and multimodal metaphor. In Bildwissen-
schaft: Zwischen Reflektion und Anwendung, Klaus Sachs-Hombach
(ed.), 264–284. Cologne: Von Halem.
2006/this vol. Non-verbal and multimodal metaphor in a cognitivist frame-
work: Agendas for research. In Cognitive Linguistics: Current Ap-
400 Charles Forceville
Mats Rohdin
Abstract
1. Introduction
Arthur Danto (1992: 74) has suggested that a possible explanation why art
criticism never developed a concept for pictorial metaphor in ancient Greece
was that scholars were too much concerned with mimesis. Danto’s opinion is
no less pertinent to film scholars during the first decades of the twentieth
century, when the new medium was regarded as the culmination of the old
mimetic tradition, almost as “reality representing itself” (Hake 1993: 277).
The close connection between film and perceptual reality is probably one
important reason why there is not any systematic account of metaphor in
film theory until the late 1920s. Another reason is, of course, that many
404 Mats Rohdin
In silent cinema during the 1920s experiments were done in different coun-
tries in order to find filmic equivalents of verbal metaphor. Soviet montage
cinema, and especially Sergei Eisenstein, is particularly renowned in this
respect. Eisenstein (1996: 222) even claimed that experiments dealing with
montage tropes, including film metaphor, was Soviet montage cinema’s most
important contribution to film history.
As a matter of fact, many of the canonized examples of film metaphor
from the silent film period are multimodal, since they involve not only mov-
ing images (visual signs) but also intertitles (shots inserted between live-
action shots, providing explanatory text or dialogue). The breakthrough of
sound film in the late 1920s further enhanced the possibility of multimodal
metaphors since three more modes were added to the standardized film prod-
uct: speech, sound, and music.
When it comes to the question of how to identify a verbal metaphor, early
theories often assumed that metaphor was a thing or an object in the text,
which could be identified thanks to some sort of deviation or anomaly at the
surface level, for example, semantic anomaly or logical absurdity when
taken literally. In film theory the equivalent has often been a disruption of or
deviation from a mimetic form of representation (for example by a superim-
position, that is when two or more images are “combined into one over the
whole area of the frame by double printing or double exposure in the cam-
era,” Salt [1992: 329]) or against narrative codes (for example by using a
nondiegetic insert in a mainstream fiction film, that is a “shot or series of
shots cut into a sequence, showing objects that are represented as being out-
side the world of the narrative” [Bordwell and Thompson 2007: 480]). How-
ever, modern verbal and non-verbal metaphor theories often claim that there
does not have to be any deviation or incoherence at all on the surface level of
the representation in order to identify a metaphor (Black 1993: 34; Cohen
1997: 223–225, 1999: 399; El Refaie 2003: 79–80).
In the writings of classical film theory between the 1920s and 1950s, a
preliminary classification of different formal characteristics which are said
to cue the spectator in identifying the film metaphor would, according to my
analysis, look like this:
- Superimposition (e.g., Arnheim 1933; Carroll 1996a)
- Verbal image (e.g., Balázs 1930, 1952; Ejxenbaum 1981; Eisenstein
1987a, 1987b, 1996)
- Montage (primarily the nondiegetic insert) (e.g., Arnheim 1997; Balázs
1930, 1952; Bazin 1967; Eisenstein 1974, 1996)
406 Mats Rohdin
2.1 Superimposition
sters in general, thereby losing specific connotations that are attached to the
Moloch figure. Thus the pictorial signs and the intertitle, I would suggest,
qualify the metaphor as multimodal. Mappable features from the source
domain are, for example, how innocent people in ancient times were sacri-
ficed into the fire-burning opening of Moloch, which can be compared to
how workers in the film slave to death at the machines, and so on. Moloch is
literally “eating” people, while modern industrial machines are metaphori-
cally “eating” human beings, and thereby killing them.
pet theatre, which were very popular at that time because of illiteracy
amongst the majority of the people. Sergei Eisenstein often used these sorts
of clichés when portraying capitalists, kulaks, priests, policemen, etcetera.
Thus the film viewer could use experiences from other media and art forms,
which shared codes and conventions with film. As a result, the spectator’s
processing of the film narrative was facilitated considerably, which was
important due to the typically fast editing of Soviet cinema at that time.
Sergei Eisenstein’s film debut Strike includes many scenes built upon the
visual expression of verbal metaphors and clichés. The famous slaughter-
house sequence at the end of the film, for example, is preceded by a scene in
which the police chief gets furious and knocks a bottle of ink over a map
showing the streets of the striking workers’ district, thus literalizing the
metaphor “the streets running with blood” (Bordwell 1993: 58–59) (figure
5). The verbal expression is target, while the ink on the map is source. The
map metonymically stands for the streets, while the spilled ink metaphori-
cally stands for the workers’ blood, which gives the metaphor STREETS
RUNNING WITH BLOOD ARE MAP SPILLED WITH INK.
Figure 5. When the police chief in Sergei Eisenstein’s Strike rages and knocks a
bottle of ink over a map, the verbal image “streets running with blood”
is visualized.
A mappable feature from the source domain is the manner in which the
spilled ink overflows the streets of the map, which correlates with how the
next scene shows the massacre of the workers, as “the streets running with
blood.” In order to understand the metaphor the interpreter has to construct
the implicit target domain, STREETS RUNNING WITH BLOOD, which requires
awareness of the metaphoric expression, either in spoken or written form.
410 Mats Rohdin
Thus the source is presented before the implicit target, but the target is then
visualized in the following scene, a common method used by Eisenstein when
dealing with verbal images.
The Russian formalists, who also worked as film critics and scenario
writers, were fond of using verbal images, which may not be surprising con-
sidering their literary background. Boris Ejxenbaum (1981: 79), for exam-
ple, claimed that verbal images, or “internal speech” as he called it, were the
only way to express metaphors in film. One advantage of visualizing verbal
images, according to some critics, was that they revitalized conventional and
worn out metaphors and clichés (Balázs 1952: 111–112, 127), a method
used in different media of today as well, for example in political cartoons
(e.g., El Refaie 2003: 89). A change in the medium will also, of course, in
one way or another, affect the content, which is always important to remem-
ber. Nevertheless, many critics and filmmakers (e.g., Clair 1972: 48) con-
demned this way of metaphorical filmmaking as too “literary,” a common
accusation at a time when proponents of film claimed the medium to be an
independent art form. Quite soon, indeed, filmmakers realized that meta-
phors in the form of verbal images would forever subordinate film to lan-
guage. The result of this would be that novel metaphors could not be created
in film since they had to exist verbally in order for the spectator to be able to
identify them at all. For this reason, filmmakers began to seek other ways to
express metaphors.
Filmmakers in the early years soon realized how important montage was to
achieve filmic signification. Some critics suggested that the film metaphor
could be created within the story world (diegesis) by juxtaposing different
objects or things by montage. For example, Balázs (1952) paid attention to
this type of “metaphorical montage,” as he called it, which during silent
cinema often was visual and monomodal, since the two domains belonged to
the diegetic world. He exemplifies this subtype using a sequence from Eisen-
stein’s Battleship Potemkin (Bronenosets Potemkin, 1925), where close ups
of the faces of the sailors are crosscut with the ship’s engines: “Such re-
peated juxtaposition compels comparison. A visual parallel inevitably con-
jures up a parallel in the mind. The angry, resolute faces of the sailors trans-
fer their own expression to the wheels and cranks. Yes, they are fighting side
by side in a common struggle” (Balázs 1952: 126). This suggests the con-
ventional metaphor MACHINES ARE PEOPLE (Lakoff and Turner 1989: 132)
Multimodal metaphor in classical film theory 411
“throwing the coat ‘like’ a football, a writer would say” (cited in Arnheim
1933: 264–265). This, I would argue, qualifies as a multimodal metaphor:
MEN CHASING COAT ARE MEN PLAYING RUGBY. Perhaps it would be possi-
ble to argue that this is a monomodal metaphor because the metaphor is
possible to understand without the nondiegetic sound. Although the source
domain “men playing rugby” is cued by both visual and aural elements, I
believe that many viewers would miss this metaphor if the obtrusive non-
diegetic sound were missing.
But the metaphor, I propose, may also draw on the subtitles that precede the
nondiegetic insert and the accompanying intense and aggressive music. The
intertitles “‘Modern Times.’ A story of industry, of individual enterprise –
Multimodal metaphor in classical film theory 415
2.4 Cinematography
set-up makes the film image look “real” in the diegetic story world, but how
it also can express metaphoric meanings: “In such metaphoric sequences the
objects photographed are real … the set-up merely gives them a deeper
meaning, a second, symbolical significance, without depriving them of their
own, real, normal meaning. The shot would be comprehensible, as the detail
of an ordinary film scene, even to those who failed to grasp this second
meaning” (1952: 113).
According to Balázs such metaphors show the skillfulness of the artist,
the ability to take advantage of the film image’s capacity to embrace two
different meanings at the same time. This type of metaphor could be based
upon unusual camera angle and framing, as when Vsevolod Pudovkin in The
End of St. Petersburg (Konets Sankt-Peterburga, 1927) shows the rulers of
Russia with their heads cut off by the frame (figure 11). Balázs’ commen-
tary is, as usual, concise, and not very informative: “A pictorial metaphor
whose meaning is obvious” (Balázs 1930: 36, my translation, MR). How-
ever, the intra-textual context includes a lot of intertitles, which describe
how “Mother Russia calls” the people to enter World War I with enthusi-
asm. Probably “Mother Russia” (the rulers) is target and the image of the
rulers with their heads excluded is source, thus probably some sort of visu-
alization of a verbal image: MOTHER RUSSIA IS HEADLESS PEOPLE. This
ought to be labeled multimodal, since the intertitles are important to aid the
spectators to process the rapidly changing images showing cheering people,
marching soldiers, people sitting on chairs with their heads cropped by the
frame, etcetera. The source domain’s image of the rulers of that time, with
their heads cut off by the frame, indicates their headless politics when Rus-
sia entered Word War I, features which are mapped to the target domain
“Mother Russia.”
Another example of this type of metaphor cued by cinematography comes
from Eisenstein’s Old and New (1929), were the director uses a close-up
with wide-angle lens and low camera angle to criticize the Soviet bureauc-
racy. The intra-textual context is recounted with various shots and intertitles:
the farmers are applying for a loan to buy a tractor in order to be able to
bring in the harvest. But the local authorities deny them this, because the
rules state that the farmers can only be given the loan after they have com-
pleted the harvest. Eisenstein wanted to disparage the system of bureauc-
racy, and he did this in different shots by showing objects (writing machine,
pencil sharpener, law book) in gigantic close-ups made with a wide-angle
lens, which dwarfs the clerks in the background (figure 12). In the shot in
question a man is barely visible behind a gigantic book, perhaps a literal
visualization of the verbal image, “to hide behind bureaucracy.” Thus the
Multimodal metaphor in classical film theory 417
This type of film metaphor challenges the old demand for a deviation on the
surface level since there are no formal characteristics that cue the spectator
to perform a metaphoric interpretation. The following example is taken from
the writings of the French film critic André Bazin in the 1950s. Among film
scholars he is frequently regarded as one of the most important proponents
of a realistic film style in film history. Nevertheless, Bazin did speak favora-
bly about cinema and its “plastic reproduction of reality” as “an art of po-
tential metaphor” (1981: 151). A good example of his metaphoric interpreta-
tions occurs in an article when he writes about a scene at the end of Federico
Fellini’s The Swindle (Il bidone, 1955). The story is about an old, tired
crook, who suddenly decides to become a better man and start anew. After
the last swindle, he tries to cheat his accomplices out of a large sum of
money. They find out his plans, however, beat him unconscious, and leave
him to die on a mountain slope. The last thing he sees before he dies is a
group of women and children passing by on the road, carrying bundles of
sticks on their backs. Bazin interprets this as “angels pass,” which he sees as
a typical element of Fellini’s preoccupation with the angel metaphor (1971:
89) (figures 13–14).
The two domains are the PEASANT WOMEN (target) and ANGELS (source)
respectively: WOMEN ARE ANGELS. Thus the source domain is absent, but is
cued by the similarity between the women’s bundles of sticks on their backs
and angels’ wings. But it is also possible that Bazin’s religious interpretation
rests upon his knowledge of the works of Dante, for example Vita Nuova
and La Divina Commedia. The former deals with a man’s wish to start
anew, as is the case in Fellini’s film, and so does the latter, which begins
with the following words: “In the middle of life’s road / I found myself in a
dark wood.” La Divina Commedia has a geographical composition (hell,
purgatory, and paradise), which has a correspondence in the film’s journey
from the hectic city nightlife to the top of the mountain near heaven. This
can be compared to well-known conceptual metaphors like LIFE IS A JOUR-
NEY, DEATH IS DEPARTURE, and DIVINE IS UP (Lakoff and Turner 1989: 9–
11, 150–151). Dante is guided into paradise by Beatrice in the same way as
Multimodal metaphor in classical film theory 419
the group of women lead the crook to heaven after his last words: “Wait, I’m
coming.” In early Christian belief, angels are regarded as messengers be-
tween humans and God, especially after death when every human is guided
to heaven by angels (Nationalencyklopedin [The National Encyclopedia]
1996: 408–409). The crook is ready to meet the Lord. Moreover, when he
dies, non-verbal sounds of dry leaves scattering in the wind are heard, thus
connoting his death. But what is most important in this religious interpreta-
tion is the sound of the wind, which activates ideas of God. In the Bible, God
sometimes appears in disguise as the wind, for example in Genesis (8:1). To
summarize, I would categorize this scene from The Swindle as a multimodal
metaphor since speech and non-verbal sounds play such an important role in
identifying the absent source domain, angels.
Figure 13. Federico Fellini’s The Figure 14. … showing no formal char-
Swindle ends, according to acteristics on the surface level
Bazin, with a metaphor … cueing a metaphoric interpre-
tation.
stock, moving camera, zoom lens, color film, cinemascope, stereo sound,
etcetera), which gives each period and each film a particular appearance.
Bazin’s way of using the concept of style is restricted and pertains mainly
to formal characteristics, that is, the “repeated and salient uses of film tech-
niques characteristic of a single film or a group of films” (Bordwell and
Thompson 2007: 481). Bazin’s opinion about style is subtle, and can be
compared to Arthur Danto’s view on style attribution in The Transfigura-
tion of the Commonplace: A Philosophy of Art: “The structure of a style is
like the structure of a personality. And learning to recognize a style is not a
simple taxonomic exercise. Learning to recognize a style is like learning to
recognize a person’s touch or his character” (1981: 207, emphasis mine,
MR).
One of the most interesting findings in this chapter, I propose, is the impor-
tant role of intertitles in the canonic examples of film metaphors in the silent
cinema. Consequently, many of these examples qualify as multimodal meta-
phors (of the verbo-pictorial variety). This may be surprising considering
that the film medium during this period often was characterized as “moving
images” or “visual Esperanto” (e.g., Lindsay 1916: 177). Evidently, many
film scholars seem to have proceeded from the conviction that film meta-
phors of silent cinema had to be monomodal, that is, both target and source
were visuals. The reason for this is, of course, that the predominance of the
visual mode in film was taken for granted, which made the written text auxil-
iary by definition. But many of the metaphors taken from the silent era listed
above are to a high degree dependant on intertitles when spectators identify
and/or interpret these metaphors.
A possible explanation why early film theorists put so much emphasis on
the visual mode is that, in order to prove that film could be accepted as an
independent art, intertitles had to be downplayed. When early film theorists
wrote about metaphors they often presented them out of context, thereby
ignoring the intertitles, which often were of vital importance for identifica-
tion and/or interpretation of the metaphor. At the same time it should not
come as any surprise that, for example, metaphor experiments in Soviet
montage cinema were dependent upon intertitles, since the spectator needed
help to understand the logic of the story when suddenly superimpositions,
nondiegetic inserts, and unfamiliar camera set-ups were invading the screen.
Multimodal metaphor in classical film theory 421
With the breakthrough of sound film in the late 1920s it was possible to
include new modes in metaphorical compositions. Sound films like The Mil-
lion, Fury, Modern Times, and The Swindle show that speech, sound, and
music played an important role in both the identification and interpretation
of multimodal metaphors.
Often different modes interacted in order to make it easier for the specta-
tor to quickly identify and interpret the metaphor, which of course was im-
portant considering the irreversibility of the spectacle (the spectator can not
stop the film during projection in the theatre to think over or re-examine
obscure passages). Sound could also produce meanings that were difficult to
represent visually, for example in The Swindle, where the notion of God in
the barren mountain landscape is conveyed by the sound of the wind. Never-
theless, elements such as speech, sound, and music are often neglected in
early film theory when metaphors are discussed, as though the visuals alone
are sufficient to make up the metaphor in question.
Another pertinent question in this chapter is how target and source have
been presented in these examples. In most of them, both target and source
are visually present. An exception is the example from Metropolis, CINEMA
IS MOLOCH, where the target is supplied by extra-textual information. Symp-
tomatically, this is an interpretation from our time. My opinion is that this is
quite uncommon in classical film theory texts, but much more accepted to-
day, when reader-response theories have influenced criticism and interpreta-
tions in, for example, film studies at universities. In fact, much current
metaphor interpretation deals with how to find a target to match a source in
order to create a metaphor (Johnson and Malgady 1980: 266). This is also in
line with the view of metaphor/figurative processing as a reader strategy in
contexts which stress novelty and originality, for example, in academic writ-
ing (Gibbs 1994: 448–449; see also Steen 1994).
The only metaphor example discussed in this chapter that draws on an
absent source, which is rare, is Fellini’s WOMEN ARE ANGELS, where the
angels are implied by extra-textual interpretation. An alternative interpreta-
tion could be that the metaphor should be reversed, that is, “angels are
women.” This would correspond to the view that the source in linguistic
metaphors “is always supplied by the description (the text)” (Indurkhya
1992: 18–19). Another suggestion for how to decide on target and source is
to choose as target the domain with the strongest denotation or which be-
longs to the story (Whittock 1990: 31–32), that is, in the Fellini film, the
women. However, both Indurkhya and Whittock put forth principles that are
more prescriptive than descriptive, which blind them to the rich variety of
manifestations of metaphor where target and source are decided on different
422 Mats Rohdin
writers like Aesop and La Fontaine have always been popular in authoritar-
ian societies like Russia and later the Soviet Union.
In the introduction of this chapter I posed three different questions,
which, according to Forceville (1996: 65–66; 2002: 2–3; 2005: 266), have
to be answered: (1) Which are the two terms of the multimodal metaphor?
(2) Which is the metaphor’s target domain and which is the metaphor’s
source domain? (3) Which features are mapped from the source domain to
the target domain? Although my classification in this chapter mostly deals
with formal characteristics (for example, superimpositions, and nondiegetic
inserts), it has also shown that the question of how to identify the metaphor,
to quote Danto about style, “is not a simple taxonomic exercise.” Although
some sort of deviation or surface anomaly frequently holds, especially in the
early examples, it is not a necessary condition. Therefore, I agree with the
philosopher Ted Cohen when he asserts: “If we have learned anything since
the explosion of interest in metaphor that began about twenty-five years ago,
it is that the classical and standard definitions of ‘metaphor’ are unreliable,
and so every sensible person, I think, must suppose that we are dealing with
formally undefined phenomena” (Cohen 1999: 399, emphasis mine, MR).
In contemporary metaphor theory, writers seem to agree that any linguis-
tic material can be used to make up a metaphor (e.g., Kittay 1989: 103;
Entzenberg 1998: xviii–xxvii). In Rohdin (2003) I have demonstrated that
the same holds true for any filmic material. This is in accordance, I argue,
with Lakoff and Johnson’s well known definition of metaphor: “The essence
of metaphor is understanding and experiencing one kind of thing in terms
of another” (Lakoff and Johnson 1980: 5, emphasis in original). However,
this should not prevent us from examining which filmic expressions have
been used in metaphorical interpretations throughout film history. Therefore
I think that corpus-based studies of metaphors in different genres, move-
ments, national cinemas, and among individual directors, would yield inter-
esting results. Worth remembering when dealing with questions like these is
Forceville’s critique of Noël Carroll’s restricted metaphor classification:
“Categorizing and name-giving should follow analysis of phenomena, and
not the other way round” (2002: 9).
Finally, when the history of non-verbal metaphors is outlined, references
often start with I. A. Richards’ The Philosophy of Rhetoric: “Thought is
metaphoric, and proceeds by comparison, and the metaphors of language
derive therefrom” (1965: 94, emphasis in original). But it is interesting to
note that the term “metaphor” already had been used for several decades in
film theory since the 1910s (for example, Lindsay 1916: 172; Epstein 1974
vol. II, 68). It is important to remember that with film we have a new me-
424 Mats Rohdin
dium and art form whose development we have been able to monitor from its
birth, in the late 1890s, which makes it exceptional compared to other, older
art forms like theatre, literature or painting. This not only gives us important
knowledge about the film medium itself, and how it has changed over time,
but also demonstrates that metaphor, including what we today call multimo-
dal metaphor, had been introduced, discussed, questioned, and, lastly, estab-
lished in a non-verbal art such as film, from its earliest beginnings.
Acknowledgment
References
Arnheim, Rudolf
1933 Film [1932]. L. M. Sieveking and Ian F. D. Morrow (transl.), Lon-
don: Faber and Faber.
1997 Sound film by force [1931]. In Film Essays and Criticism [1977],
39–42. Brenda Benthien (transl.), Madison/London: The University
of Wisconsin Press.
Balázs, Béla
1930 Der Geist des Films. Halle: Verlag Wilhelm Knapp.
1952 Theory of the Film: Character and Growth of A New Art [1945],
Edith Bone (transl.), London: Dennis Dobson.
Bazin, André
1967 The evolution of the language of the cinema [1950–55]. In What is
Cinema? Vol. I [1958–59], Hugh Gray (ed. and transl.), 23–40.
Berkeley/Los Angeles/London: University of California Press.
1971 Cabiria. The voyage to the end of neorealism [1957]. In What is
Cinema? Vol. II [1961–62], Hugh Gray (ed. and transl.), 83–92.
Berkeley/Los Angeles/London: University of California Press.
1973 The French Renoir. In Jean Renoir [1971], François Truffaut (ed.),
W. W. Halsey II and William H. Simon (transl.), 74–91. New York:
Simon and Schuster.
1981 On L’Espoir, or style in the cinema [1945]. In French Cinema of
the Occupation and Resistance: The Birth of a Critical Esthetic
[1975], François Truffaut (ed.), Stanley Hochman (transl.), 145–
157. New York: Frederick Ungar.
Multimodal metaphor in classical film theory 425
Black, Max
1993 More about metaphor. In Metaphor and Thought. 2nd ed. Andrew
Ortony (ed.), 19–41. Cambridge: Cambridge University Press.
Bordwell, David
1993 The Cinema of Eisenstein. Cambridge, MA/London: Harvard Uni-
versity Press.
Bordwell, David, and Kristin Thompson
2007 Film Art: An Introduction. 8th ed. Boston: McGraw Hill.
Buckland, Warren
2000 The Cognitive Semiotics of Film. Cambridge: Cambridge University
Press.
Carroll, Noël
1996a A note on film metaphor. In Theorizing the Moving Image, 212–
223. Cambridge: Cambridge University Press.
1996b Language and cinema: Preliminary notes for a theory of verbal im-
ages [1980-1981]. In Theorizing the Moving Image, 187–211. Cam-
bridge: Cambridge University Press.
Clair, René
1972 In the beginning was the image. In Cinema Yesterday and Today, R.
C. Dale (ed.), Stanley Appelbaum (transl.), 40–80. New York: Do-
ver.
Cohen, Ted
1997 Metaphor, feeling, and narrative. Philosophy and Literature 21 (2):
223–244.
1999 Identifying with metaphor: Metaphors of personal identification.
Journal of Aesthetics and Art Criticism 57 (4): 399–409.
Danto, Arthur C.
1981 The Transfiguration of the Commonplace: A Philosophy of Art.
Cambridge, MA/London: Harvard University Press.
1992 Metaphor and cognition. In Beyond the Brillo Box: The Visual Arts
in Posthistorical Perspective, 73–87. Berkeley/Los Ange-
les/London: University of California Press.
Ejxenbaum, Boris
1981 Problems of cinema stylistics [1927]. In Russian Formalist Film
Theory, Herbert Eagle (ed.), 55–80. Ann Arbor: University of
Michigan.
Eisenstein, Sergei
1974 Zur Komposition des “Streik”-Finale [1933]. In Schriften 1: Streik,
Hans-Joachim Schlegel (ed.), 274–276. München: Carl Hanser.
1987a Nonindifferent nature [1945]. In Nonindifferent Nature, Herbert
Marshall (transl.), 216–396. Cambridge: Cambridge University
Press.
426 Mats Rohdin
Jakobson, Roman
1987 Is the film in decline? [1933]. In Language in Literature, Krystyna
Pomorska and Stephen Rudy (eds.), 458–465. Cambridge
MA/London: Harvard University Press.
Johnson, Michael G., and Robert C. Malgady
1980 Toward a perceptual theory of metaphoric comprehension. In Cog-
nition and Figurative Language, Richard P. Honeck, and Robert R.
Hoffman (eds.), 259–282. Hillsdale, NJ: Lawrence Erlbaum.
Kittay, Eva Feder
1989 Metaphor: Its Cognitive Force and Linguistic Structure. Oxford:
Clarendon Press.
Kövecses, Zoltán
2002 Metaphor: A Practical Introduction. New York: Oxford University
Press.
Lakoff, George, and Mark Johnson
1980 Metaphors We Live By. Chicago: University of Chicago Press.
Lakoff, George, and Mark Turner
1989 More than Cool Reason: A Field Guide to Poetic Metaphor. Chi-
cago/London: University of Chicago Press.
Lindsay, Vachel
1916 The Art of the Moving Picture. New York: MacMillan.
Nationalencyklopedin
1996 Nationalencyklopedin [The National Encyclopedia] VITRY–ÖÄ.
Höganäs: Bokförlaget Bra Böcker.
Richards, I(vor) A(rmstrong)
1965 The Philosophy of Rhetoric. Oxford: Oxford University Press.
Rohdin, Mats
2003 Vildsvinet i filmens trädgråd. Metaforbegreppet inom filmteorin
[The wild boar in the garden of film: The concept of metaphor in
film theory] Summary in English 318–329. Ph.D. diss., Stockholm:
Edita Norstedts Tryckeri AB.
Salt, Barry
1992 Film Style and Technology: History and Analysis. 2nd ed. Starword:
London.
Stanford, W(illiam) B(edell)
1936 Greek Metaphor: Studies in Theory and Practice. Oxford: Basil
Blackwell.
Steen, Gerard
1994 Understanding Metaphor in Literature: An Empirical Approach.
London/New York: Longman.
428 Mats Rohdin
Tynjanov, Jurij
1981 On the foundations of cinema [1927]. In Russian Formalist Film
Theory. Herbert Eagle (ed.), 81–100. Ann Arbor: University of
Michigan.
Ward, Janet
2001 Weimar Surfaces: Urban Visual Culture in 1920s Germany. Berke-
ley/Los Angeles: University of California Press.
Whittock, Trevor
1990 Metaphor and Film. Cambridge: Cambridge University Press.
Chapter 18
Abstract
1. Introduction
done by focusing on specific genres, since the meanings that can or must be
gleaned from representations are strongly steered and constrained by the
audience’s awareness of the genre to which a discourse belongs (Forceville
1996, 1999b, 2006b). The current chapter aims (a) to highlight the impor-
tance of the structural metaphor HUMAN VICTIM IS ANIMAL in the modern
horror film; (b) to demonstrate how this metaphor is expressed multimo-
dally, and to formulate some tentative generalizations about the surface
manifestations of the metaphor.
Before addressing the central metaphor itself, we want to clarify briefly our
use of the labels “multimodal” and “monomodal” metaphor. According to
Forceville (2006a), monomodal metaphors present target and source do-
mains both in the same mode. In film, the list of modes includes at least:
visuals, written language, spoken language, sound, and music. Although this
is not an exhaustive list (Mittelberg and Waugh this volume and Müller and
Cienki this volume extensively discuss the gestural mode), we will in the
present chapter restrict ourselves to these. The best-known variant of mono-
modal metaphor is verbal metaphor, which until Lakoff and Johnson (1980)
was considered the only type of metaphor anyway. Another variant of
monomodal metaphor is the pictorial one, which cues both target and source
visually (Forceville 1996). By contrast, multimodal metaphors are “meta-
phors whose target and source are each represented exclusively or predomi-
nantly in different modes” (Forceville 2006a: 384). A “pure” multimodal
metaphor thus presents a target in one mode/modality, and one mode only,
and the source in another mode/modality only. But such pure multimodal
metaphors are distinguished for analytical purposes only: there is nothing
inherently better about “pure” than about “impure” multimodal metaphors.
Indeed, as acknowledged by Forceville (2006a: 385), probably by far the
majority of multimodal metaphors in moving images cue target and/or
source in more than one mode simultaneously (we could label these multi-
modal metaphors in the broad sense, as opposed to multimodal metaphors in
the strict or narrow sense). The broad definition of multimodal metaphor is
particularly pertinent if a metaphor is not a creative metaphor that surfaces
only once in a multimodal “text,” but is an expression of a conceptual meta-
phor that keeps appearing throughout a narrative or argument. In Musolff’s
(2006) terminology, HUMAN VICTIM IS ANIMAL would be a “scenario.”1 In
such a situation, the same conceptual metaphor can well be expressed in a
Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor 431
The year is 1957 and in the small deer hunting community of Plainfield,
Wisconsin, something truly horrifying is waiting to be discovered. Investiga-
tors will enter the house of one Edward Theodore Gein to find the remains of
numerous female bodies, including the woman they were looking for, freshly
killed Bernice Worden. The house will quickly be dubbed the “Gein House
of Horrors” by the press as reports of strange artistic experiments and dis-
turbing uses of the human body become apparent. Bowls made of human
skulls, human skin suits, furniture upholstered by body parts and a necklace
made of nipples are a few examples of the horrid imagery presented to those
willing to know about such things. The case of Ed Gein becomes infamous
in its time and a subject of infinite conversation and shock among the public.
Atrocious crimes are no longer the subject of far-away places or past reali-
ties. The realization that danger can lurk in the neighborhood and that not
everyone can be trusted becomes the new reality. Ed Gein constitutes the
prototypical serial-killer-next-door and goes on to directly inspire such hor-
ror film legends as Norman Bates, Leatherface, and Buffalo Bill. In fact, the
story becomes a source and influence for at least three generations of horror
cinema, ranging most notably from Psycho (Hitchcock, USA 1960) to The
432 Gunnar Theodór Eggertsson and Charles Forceville
Texas Chain Saw Massacre (Hooper, USA 1974) and The Silence of the
Lambs (Demme, USA 1991). Other horror films influenced by the Gein
story include Three on a Meathook (Girdler, USA 1972), Deranged (Gillen
and Ormsby, Canada/USA 1974), Maniac (Lustig, USA 1980), Ed Gein
(Parello, USA 2000), and House of 1000 Corpses (Zombie, USA 2003).
Horror scholars generally see the release of Hitchcock’s Psycho as marking
the shift from classic to modern horror film. The monsters populating classic
horror films were always foreign in a way – mummies, vampires, aliens –
but after checking in at Psycho’s Bates Motel the home became the new
territory for horror. Reynald Humphries claims about modern horror that
“the element that needs to be stressed is that we are dealing with the modern,
everyday world, not with one situated in the past. Gone are the mad scien-
tists, the remote islands and settings, the dangers of invasion and radiation”
(Humphries 2002: 85). This element can be directly traced to the films that
sprang from the obsession with the case of Ed Gein, where horror found a
new home – right next door in Anytown.
But there is more to the Gein influence than meets the eye. By the time of
his death in 1984, Gein had already become a legend in horror folklore, an
inspiration to decades of horror cinema and “revered by horror buffs as the
prototype of every knife-, axe-, and cleaver-wielding maniac who has stalked
America’s movie screens for the past thirty years” (Internet Zombie 2006,
http://www.houseofhorrors.com/gein.htm last accessed 11.05.07). On the
surface, this seems very obvious. Not only did he bring terror into suburban
reality, and cast a shadow of doubt and paranoia over even the most lovable
neighbor; his violent crimes and methods have been recreated and imitated in
countless cheap horror flicks since the 1960s. But there is a more deep-
rooted argument to be made regarding the extent of Gein’s crimes – one that
will be inspected more closely in this chapter. Sure enough, the shock of
such atrocious and horrific crimes taking place in the peaceful rural commu-
nity had an extreme and long-lasting effect not only on the evolution of hor-
ror art but on the American psyche in general. This shock derives from a
very particular aspect of his violent crimes – something that provided the
base for the public’s extremely horrific reactions to his actions, and that was
responsible for a crucial element in horror cinema ever since: Ed Gein
treated human beings in precisely the same way human beings treat their
non-human counterparts. For the purpose of this chapter, this statement can
be rephrased as: Ed Gein created a metaphor that has been embraced and
recycled by horror cinema ever since investigators came upon the gutted,
decapitated body of Mrs. Worden hanging upside down in Gein’s House of
Horrors back in 1957. This metaphor went unnoticed at the time but after
Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor 433
fifty years of roaming wild and pervading the genre it is time to finally ana-
lyze it: HUMAN BEING IS ANIMAL,2 or to be more descriptive: HUMAN VICTIM
IN HORROR FILM IS ANIMAL USED AS RESOURCE.
4. Metaphorical vulgarity
tures from the Alien (Scott, UK 1979) cycle, who need to kill other life
forms in order to procreate. When human beings attack other human beings,
the actions are more difficult to justify.
In many cases hardly even an attempt is made in the films themselves to
provide proper justification. The genre of extreme horror is one famous for
poorly constructed narratives, usually existing solely in order to string to-
gether different scenes of gratuitous violence where the pleasure of the killer
is shared by the audience rooting for progressively elaborate effect sequences
of blood and gore. This has led to the claim that it is in fact the special ef-
fects crew that ranks first, above director, screenwriter, producer, or actors,
in such films. The subgenre of extreme horror is furthermore an interesting
field for metaphor studies, as it is generally considered to be of lesser aes-
thetic quality than the more “conservative” horror movie – that style of hor-
ror which actually aims at scaring the audience, keeping up a reasonable
storyline, and generally leaning towards some sort of intellectual process in
the viewing experience. The extreme horror movie, on the other hand, fo-
cuses purely on the bodily experience: the more body-harassing details are
provided, the more clearly does the film belong to the genre. This constitutes
a great part of why these movies are generally frowned upon by the advo-
cates of “proper” cinema – which tends not to show everything. It is always
more in the style of fine arts to metaphorize instead of vulgarly presenting
the subject. But in the genre of extreme horror, metaphors abound – be they
intentionally created or not.
ing their conversation Mick describes different ways of hunting and killing
animals and when one of the girls asks him if he really kills kangaroos he
replies that he is in fact doing people a favor, making the following explicit
analogy: “they’re everywhere out here now – like tourists!” The HUMAN IS
ANIMAL metaphor in Wolf Creek comes from different verbal references to
kangaroos, Mick’s evening speech about hunting, and from visual cues in the
latter half of the film, which mainly focuses on Mick hunting down two of
the escaped backpackers.
Figure 1. The victims of Wolf Creek are shown driving, through a bullet hole in a
kangaroo warning sign, as part of a running reference to the characters
as kangaroos to be hunted.
The most obvious way the metaphor HUMAN IS ANIMAL surfaces is through
the contextual information of Mick’s hunting background, combined with his
weapons of choice (including a hunting knife and rifle) and his method of
trapping and caging his prey: he tampers with people’s cars and then offers
them help, thereby luring them to his home where he subsequently drugs his
victims. In a telling scene, one of the victim girls stumbles upon a room full
of hunting memorabilia, including video footage of other people having
fallen prey to Mick in precisely the same way. After trapping and caging the
backpackers, he proceeds to torture and kill them as he sees fit to appease
his sadistic desires. And even though he does not intend for his victims to
escape, he is well prepared for that possibility, thus turning the game into a
proper hunt. When the two girls manage to flee he goes on the road to pursue
and kill them. In a similar manner to most cinematic serial killers of his
genre, Mick gets his first victim up close and personal (Clover 1992: 31–32)
when he cuts her up with a big hunting knife, mutilating her hand and sever-
ing her spinal chord in order to disable her. But he steps away from his genre
colleagues when he pulls out his hunting rifle – a weapon made to use at a
distance. The second girl manages to flee out to the highway, where she is
436 Gunnar Theodór Eggertsson and Charles Forceville
picked up by an old man whom Mick shoots from far away. The girl takes
the car and a chase ensues. After having been thrown off the road, Mick
shoots her tires with the rifle. Even as he arrives at the crash site and finds
her still alive, he does not kill her up close but ultimately shoots her from a
few feet away.
Figures 2a and 2b. Kangaroos jump around the outback. The last survivor is
cross-cut with the kangaroos to further emphasize his role as a hunted,
wounded animal (Wolf Creek).
There are a few other hints to bring out the animal metaphor, given through-
out the film as part of a running reference to the characters as kangaroos to
be hunted. Very early on, the group of victims is filmed through a bullet-hole
in a kangaroo warning sign (figure 1). A close-up of the road sign fills the
screen, with the focus on the victims’ car in the background, seen through
the hole, driving down the road. The focus then quickly shifts to the fore-
ground, showing viewers not only that the sign portrays a shaded image of a
kangaroo, but that the hole in question has been made by a bullet. Through a
juxtaposition of the car, the bullet hole, and the kangaroo image, the specific
metaphor HUMAN VICTIMS ARE KANGAROOS is first introduced. In addition
Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor 437
Figures 3a-c. From top to bottom: Marc ensnared in a trap in the woods; Boris
treats the trapped Marc as if he were his lost dog; Marc is hunted in the
final chase through the woods (Calvaire).
Another manifestation surfaces relatively early in the film, with the added
sexual implication of Marc as an object of desire, when Marc takes a walk
in the woods and comes across a farming area where he hears animal noises
coming from a nearby stable. Silently he sneaks towards the sounds and
spies on a family of male villagers – the same ones that will raid the inn later
on and attempt to rape Marc – standing around one family member who is
getting his genitals licked by a calf. There is also a pig present and a cow
looking on. A few small dead animals hang from the stable rafts behind the
men, who are all enthusiastically watching their relative lying on the stable
floor getting a bestial blowjob to the sounds of their squealing pet pig. It is a
clear case of an innocent animal being used for sexual pleasure in the same
way as Bartel – and these same villagers – will use Marc later on. In a way,
as Marc peeps through the wooden boards of the stable, he is looking into
his own future near-fate: to be sexually exploited by a bunch of crazy men
(figures 4a, b). The calf is the source domain of a premonitory metaphor –
MARC IS CALF – derived from the larger animal metaphor and created from
440 Gunnar Theodór Eggertsson and Charles Forceville
the contextual information of what lies ahead for Marc (Bartel taking him to
bed and dressing him up, the villagers’ rape attempt), the visual juxtaposi-
tion of Marc and the calf at the stable, as well as the sounds of the farm
animals during the stable scene, most notably the pig squealing.
Figures 4a-c. From top to bottom: Marc peeks into the barn and into his own
future; the villagers within take sexual pleasure from one of the farm
animals; Marc is “shorn” by Bartel.
The final noteworthy sub-metaphor is that of the trap and the hunt. During
his failed escape attempt, Marc is caught in an animal trap, as previously
mentioned, and subsequently brought back to his captor, Bartel, and returned
to his “cage,” where he is locked up, constantly watched over, and subjected
to cruel and degrading treatment. When Boris drives Marc back to Bartel
after freeing him from the trap in the woods (with Marc lying on the back of
the tractor like a hunted animal), they pass two villagers who are in the
process of gutting a boar they have hunted, creating a graphic hunting juxta-
position. Furthermore, during the second escape attempt – after the villagers
have killed both Boris and Bartel – the viewer is again reminded of a hunt,
where the villagers bear hunting rifles and use their pigs as a tracker or
bloodhound. In the end, Marc is left alone in the woods, completely lost and
Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor 441
out of place in the world, much like the last survivor of Wolf Creek. These
scenes all hint at the metaphor MARC IS HUNTED ANIMAL.
In each manifestation of these sub-metaphors Marc is presented in terms
of animals that are in human captivity – thus creating the metaphor MARC IS
CAPTIVE ANIMAL. By presenting the protagonist in such a way, certain ele-
ments become highlighted for his character portrayal. He is defenseless,
suffering, subjugated, and stripped of freedom, but most importantly his life
and well-being depend solely on the will and desire of his captor. All of these
features belong to the “social status” of nonhuman animals in contemporary
society but they do not usually apply to human beings – at least not in a
society that bases its legislation on “universally declared” human rights.
Few horror films deal as explicitly with slaughter or are as full of references
to meat as the original The Texas Chain Saw Massacre (Hooper, USA
1974). Fictionally represented as being based on a true story, it is famously
inspired, at least in part, by the Gein murders. The theme of slaughtering
human beings like animals and putting their bodies to use (e.g., as food,
clothing, and decoration) resonates throughout the film as the metaphor HU-
MAN VICTIM IS ANIMAL RESOURCE comes to life in numerous scenes. The
storyline has become horror movie stock material. Five friends head into the
countryside to visit an old family house that is currently uninhabited. Upon
arrival, one by one they fall prey to the cannibalistic family next door, until
the last surviving girl, Sally, manages to escape into the open road and gets
picked up by a truck driver and wheeled away to safety. The cannibal family
consists of two brothers, their father, and their grandfather, whose business
used to be slaughtering cattle. However, similar to Wolf Creek’s killer Mick,
they have lost their jobs due to new farming technologies. The father works
as a gas station attendant and the two boys seem to be employed only to rob
graves and hunt people and animals for the family unit. Grampa is long re-
tired, but retains the claim to fame as one of the greatest butchers this rural
community ever had.
The first shot to greet viewers after the opening credits is that of an ar-
madillo road kill in the street. Driving past the dead armadillo the youngsters
are filmed heading to their imminent doom on the open road. The juxtaposi-
tion clearly mirrors that of the kangaroo warning scene in Wolf Creek and
serves as part of the running HUMAN VICTIM IS ANIMAL RESOURCE metaphor
– or in this case, HUMAN VICTIMS IN CAR ARE DEAD ARMADILLO IN STREET.
442 Gunnar Theodór Eggertsson and Charles Forceville
As they head down the road they pass the old slaughterhouse, at which point
Franklin, whose relatives used to own the house they are driving towards,
reminisces about how his grandfather used to sell his cattle to that place. He
describes old-fashioned cattle slaughter methods in vivid detail, involving
bashing in heads with a large sledgehammer. Although the audience does not
yet know it at this stage – unless they are alert to extreme horror’s genre
conventions – here the abused-animal story helps build up the metaphor’s
source domain, since the head-bashing will be acted out on some of the hu-
man victims as the film progresses. This lends support to the specific meta-
phor HUMAN VICTIMS ARE CATTLE, which is emphasized by crosscutting the
youngsters’ discussion with images of cows in small stalls, huddled together
on their way to slaughter, as the car drives past (figure 5a-f). Thus the meta-
phor is monomodally presented through the use of dialogue and visual juxta-
position, respectively, and multimodally through verbal and visual references
to killings with sledgehammers. The scene at the film’s end, when Sally is
held captive with her head over a basin, and the legendary butcher grandfa-
ther makes vain attempts to bash her head in, is particularly reminiscent of
old-fashioned cattle slaughter, precisely reproducing common procedures as
described by Franklin in the car.
Figures 5a-f. Discussions in the car about slaughtering, cross-cut with cattle (The
Texas Chain Saw Massacre).
Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor 443
Another gruesome scene involves the first two murders of the friends, shortly
after they arrive at the country house. A couple – Kirk and Pam – go knock-
ing on the cannibal house next door looking for gasoline. Pam stays outside
in a swing as Kirk enters the house, after getting no answers from their
knocking. Inside, a wall is decorated with animal remains and on the sound-
track the constant noise of a squealing pig can be heard. When Kirk enters
the opposite room, Leatherface, one of the brothers, pounds his head with a
sledgehammer. Kirk’s body twitches and writhes like a freshly slaughtered
animal as Leatherface hits him repeatedly and drags him into the room,
which closes with a heavy steel door, reminiscent of an industrial slaughter-
house. When Pam follows to investigate she goes into a different room where
she falls into a pile of animal bones. The whole room is decorated with ani-
mal remains (human and nonhuman alike, presumably) and to further em-
phasize the metaphor there is a live chicken hanging in a small cage, cluck-
ing as it anxiously tries to move around its confined space. The room is filled
with macabre artworks, such as a couch made of bones and fetishized human
skulls, and shots of animal remains are crosscut with the nervous chicken
and the shocked Pam, who subsequently becomes nauseous. As she stumbles
back out of the room Leatherface emerges to pull her into the slaughter
room, where he sticks her up on a large meat hook hanging from the ceiling.
He lets her hang there, screaming and convulsing, as he begins to cut up her
boyfriend’s body with a chain saw. Finally, when their friend Jerry comes
looking for them he follows a strange sound into the slaughter room, resonat-
ing from within a big icebox. Upon opening it he finds Pam still twitching,
her body being kept in cool storage. Suddenly Leatherface reappears and
kills Jerry with the sledgehammer.
The way the violence is presented – from the telling descriptions of
slaughter by sledgehammer to the storing of meat in an icebox – is all done
in terms of animal treatment. Furthermore, it remains obvious through
crosscutting (the dead armadillo, the pent-up cattle, and the caged chicken)
that the victims are symbolized to be at the same level as subjugated ani-
mals, powerless to act against their impending doom as weak prey for a
stronger attacker. The metaphor resounds mostly through the contextual
nature of cattle slaughter as well as repeated hints of juxtaposing. The can-
nibal family hunt their victims and subsequently kill them. Even though in
the case described above the victims all wandered in the house of their own
free will, it is clearly put forth in the film that the brothers’ job is to catch
people out on the road. The father sells barbecue equipment at his local gas
station and it can be read between the lines that the meat he is selling is most
likely human meat (possibly mixed with that of animals). The family mem-
444 Gunnar Theodór Eggertsson and Charles Forceville
bers eat other types of meat as well, but make no distinction between eating
humans or animals. Furthermore, they put their human victims to more use
than only for food. Leatherface makes clothes out of human skin, symbolized
by his trademark mask, and the house is filled with monstrous artworks
made from human bones and/or skin, similar to the objects found in the Gein
house of horrors. Unlike the sports hunter of Wolf Creek these human hunt-
ers are interested in more than just the thrill of the kill, and unlike the love-
seeking innkeeper of Calvaire their prey is not merely an object of sexual
desire. To them, it is one of their main sources of income and sustenance,
much like for the people of Plainfield.
The symbolism of people-as-meat is a well known interpretation for The
Texas Chain Saw Massacre, but the traditional interpretation does not in-
clude references to animal treatment; rather it is volunteered in the context of
the Vietnam War as being the great machine that, at the time, was eating
America’s youth (an interpretation presented in Ursula MacFarlane and
David Richardson’s Clive Barker’s A-Z of Horror, UK 1997). The animal
metaphor around the cannibal house is vividly based on the work of Ed Gein
and serves as a constant reminder of the role our victims play throughout the
film. In the end Sally, the sole survivor of the unfortunate five, has come to
realize what it is like to be treated as an animal for the slaughter and may
bring that realization with her back into society as she rides away on the
truck in the finale scene.
Although by and large the mappable features apply across the range of ani-
mal species, in a given instantiation the species may play a role: Wolf Creek
mostly discusses kangaroos, which are typically hunted in Australia, but
makes references to horses, pigs and buffalos as well. Calvaire presents
farmyard animals (pig, cow, calf) and includes a trap that is obviously made
to catch the leg of an animal (probably a rabbit or a fox). The Texas Chain
Saw Massacre mainly presents cattle as the metaphorical source, although it
also refers to a pig squealing as well as to a caged chicken. It is to be noted
that the precise way in which an animal can be abused varies per species –
depending on such factors as whether it is wild or domestic, big or small,
edible or non-edible, furred or feathered. Such considerations are important,
because they help counterbalance “classic” CMT’s sometimes one-sided
emphasis on the conceptual level. Yes, the central metaphor in these films is
HUMAN VICTIM IS ANIMAL – but the precise mappings vary in many small
and not-so-small ways depending on the precise narrative context in which
the metaphor surfaces. This view ties in with more recent work in CMT.
Thus Kövecses’ observation that “the mappings of the same metaphor may
be different across any two languages or [language] varieties” (2005: 123)
needs to be even further refined to accommodate the specificities of ad hoc
context.
The recurrence of the metaphor HUMAN VICTIM IS ANIMAL can be con-
sidered an important indication to characterize a film as a horror film.
Hence the films discussed illuminate that structural metaphors can contrib-
ute to genre theory (Altman 1999; Neale 2000, 2002). Note that this claim
only holds if the metaphor is verbalized in this particular way. The more all-
encompassing HUMAN BEING IS ANIMAL may contribute to the characteriza-
tion of other genres such as, arguably, the fable (although here the thorny
question remains whether the metaphor is HUMAN BEING IS ANIMAL or ANI-
MAL IS HUMAN BEING, perhaps calling for a blending theory approach, see
Fauconnier and Turner 2002). The centrality of the HUMAN VICTIM IS ANI-
MAL metaphor in the horror genre could be further investigated by focusing
on promotion materials such as trailers and posters.
Finally, the consistency with which the metaphor under scrutiny is de-
ployed in horror film inevitably focuses attention on issues of morality, spe-
cifically those pertaining to animal rights. As Lakoff and Johnson point out,
“the most fundamental values in a culture will be coherent with the metaphorical
structure of the most fundamental concepts in the culture” (1980: 22). In the
cases discussed in this chapter, the horrific mistreatment of humans that is cen-
tral to the horror genre’s conventions can apparently be no more hauntingly
presented than through the metaphor HUMAN VICTIM IS ANIMAL, in which
Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor 447
Acknowledgment
The authors are indebted to Mats Rohdin for his thoughtful comments on an ear-
lier draft of this chapter, but of course remain entirely responsible for all views
expressed.
Notes
References
Altman, Rick
1999 Film/Genre. London: British Film Institute.
Black Max
1979 More about metaphor. In Metaphor and Thought, Andrew Ortony
(ed.), 19–43. Cambridge: Cambridge University Press.
Carroll, Noel
1996 A note on film metaphor. In Theorizing the Moving Image, 212–
223. Cambridge: Cambridge University Press.
448 Gunnar Theodór Eggertsson and Charles Forceville
Clover, Carol J.
1992 Men, Women and Chain Saws: Gender in the Modern Horror Film.
London: British Film Institute.
Eggertsson, Gunnar Theodór
2006 Animal Horror: An Investigation into Animal Rights, Horror Cin-
ema and the Double Standards of Violent Human Behaviour. Re-
search MA diss., Media Studies Department, Universiteit van Am-
sterdam.
Fauconnier, Gilles, and Mark Turner
2002 The Way we Think: Conceptual Blending and the Mind’s Hidden
Complexities. New York: Basic Books.
Forceville, Charles
1996 Pictorial Metaphor in Advertising. London/New York: Routledge.
1999a The metaphor COLIN IS A CHILD in Ian McEwan’s, Harold Pinter’s,
and Paul Schrader’s The Comfort of Strangers. Metaphor and Sym-
bol 14: 179–98.
1999b Art or ad? The effect of genre-attribution on the interpretation of
images. SPIEL 18: 279–300.
2003 Bildliche und multimodale Metaphern in Werbespots [Trans. from
English by Dagmar Schmauks] Zeitschrift für Semiotik 25: 39–60.
2005 Cognitive linguistics and multimodal metaphor. In
Bildwissenschaft: Zwischen Reflektion und Anwendung, Klaus
Sachs-Hombach (ed.), 264–284. Cologne: Von Halem.
2006a/this vol. Non-verbal and multimodal metaphor in a cognitivist frame-
work: agendas for research. In Cognitive Linguistics: Current Ap-
plications and Future Perspectives, Gitte Kristiansen, Michel
Achard, René Dirven, and Francisco Ruiz de Mendoza Ibáñez
(eds.), 379–402. Berlin/New York: Mouton de Gruyter.
2006b The Source-Path-Goal schema in the autobiographical journey
documentary: McElwee, Van der Keuken, Cole. New Review of Film
and Television Studies 4: 241–261.
2007a Multimodal metaphor in ten Dutch TV commercials. Public Journal
of Semiotics 1: 19–51. http://semiotics.ca/
this vol. The role of non-verbal sound and music in multimodal metaphor.
Forceville, Charles, and Marloes Jeulink
2007 The Source-Path-Goal schema in animation film. Paper presented at
10th International Cognitivist Linguistics conference (ICLC, July) at
Jagiellonian University, Krakow (Poland).
Grant, Barry Keith (ed.)
1996 The Dread of Difference: Gender and the Horror Film. Austin, TX:
University of Texas Press.
Multimodal expressions of the HUMAN VICTIM IS ANIMAL metaphor 449
Humphries, Reynald
2002 The American Horror Film: An Introduction. Edinburgh: Edinburgh
University Press.
Lakoff, George, and Mark Johnson
1980 Metaphors We Live By. Chicago: University of Chicago Press.
Johnson, Mark
1993 Moral Imagination: Implications of Cognitive Science for Ethics.
Chicago: University of Chicago Press.
2007 The Meaning of the Body: Aesthetics of Human Understanding.
Chicago/ London: University of Chicago Press.
Kövecses, Zoltán
2005 Metaphor in Culture: Universality and Variation. Cambridge:
Cambridge University Press.
Mittelberg, Irene, and Linda Waugh
this vol. Metonymy first, metaphor second: A cognitive-semiotic approach to
multimodal figures of thought in co-speech gesture.
Müller, Cornelia, and Alan Cienki
this vol. Words, gestures and beyond: Forms of multimodal metaphor in the
use of spoken language.
Musolff, Andreas
2006 Metaphor scenarios in public discourse. Metaphor and Symbol 21:
23–38.
Neale, Steve
2000 Genre and Hollywood. London/New York: Routledge.
Neale, Steve (ed.)
2002 Genre and Contemporary Hollywood. London: British Film Insti-
tute.
Rohdin, Mats
2003 Summary in English. In: Vildsvinet I Filmens Trädgård: Metafor-
begreppet inom Filmteorin [The wild boar in the garden of film:
The concept of metaphor in film theory], 318–329. Ph.D. diss.,
Stockholm: Edita Norstedts Tryckeri AB.
this vol. Multimodal metaphor in classical film theory from the 1920s to the
1950s.
Singer, Peter
2001 Animal Liberation. New York: Harper Perennial.
Whittock, Trevor
1990 Metaphor and Film. Cambridge: Cambridge University Press.
Subject index
439, 442, See also target and horror ~, 11, 397, 430–446
source superimposition in ~, 254, 255,
conceptual ~, 7, 9, 121, 127, 405, 406–408, 412, 415, 421,
131, 134, 225, 235, 236, 265, 423
266 ~ montage, 100, 102, 112, 395,
schematic source ~, 27, 224–226, 405, 408, 410, 411, 421
230–232 silent ~ (cinema), 11, 403, 405,
categorical source ~, 213, 215, 408, 410, 420
221, 228–232, 236 Fury [Fritz Lang], 413, 421, 422
embodied knowledge, 9, 27, 28 genre, 5–11, 19, 32–34, 45, 46, 51,
embodiment (embodied meaning), 52, 55, 61–64, 73–76, 80, 85,
4, 6, 7, 12, 20, 21, 27, 28, 64, 87, 91, 95, 96, 112, 175, 183,
92, 121, 135, 149, 179, 181, 184, 189, 191, 192, 198, 207,
185, 248, 258–260, 272, 315, 218, 219, 232, 234, 333, 383,
321, 330, 333, 336, 340, 342, 384, 389, 390, 397, 408, 422,
364, 377, 423, 445 423, 429, 430, 433–435, 442,
The End of St. Petersburg [Vsevolod 446
Pudovkin], 416, 417, 422 gesture(s), 3–5, 9, 10, 13, 19, 22,
23, 25, 34, 47, 80, 97, 98,
features, 12, 13, 14, 20, 24, 29, 30, 214, 251, 261, 297–322, 323,
33, 46, 89, 98, 100, 102–105, 329–349, 364, 365, 376
107, 110, 111, 113, 157–161, co-speech ~, 10, 321, 330, 336
163, 166, 213, 225, 384, 385, deictic ~, 332, 343
389, 391, 392, 396, 404, 407– functions of~, 309, 318, 332, 333
409, 413, 415, 416, 422, 423, manual ~, 299, 313, 322, 330,
441, 446 336, 339, 340, 341, 342, 344,
emergent ~ (properties), 8, 152, 346, 347, 348
160, 161, 166, 169 metaphoric ~, 297, 300–302,
encyclopedic ~, 155, 159, 163, 304–306, 309, 310, 316, 329,
164 336, 337
mappable ~, 14, 225, 385, 389, metonymic ~, 346, 347
392, 396, 407, 409, 413, 415, spontaneous ~, 298, 307, 332,
422, 446 336, 342, 348
film, 3, 5, 6, 10–12, 14, 21, 24, 28, referential ~, 298, 301, 306, 307,
32–34, 140, 313, 314, 383, 333, 336, 337
384, 390–392, 396, 397, 403– synecdochic ~, 305, 337, 343,
424, 429, 430, 436, 437, 446 344, 348
animated ~ (also ~ animation), ~ and words, 316–319
9, 25, 233, 243, 244, 248, grammar, 22, 321, 330, 336
250–261, 272, 386, 394, 395 generative ~, 342, 344, 348
classical ~ theory, 403–405, 421, musical ~, 359
422, 424
454 Subject index
Great Chain of Being, 13, 27, 28, manga, 9, 13, 259, 261, 265, 266,
106, 112 271–274, 276–281, 284, 285,
289, 290
happiness, 23, 265, 266, 283, 285– manifestation, 8, 9, 29, 126, 321,
287, 289 334, 430, 431, 445
nonverbal and multimodal ~ of
indirect reference, 329, 331, 339 metaphor, 5, 7, 13, 34, 92,
image alignment, 8, 61, 197–209 119, 120, 123, 125, 129, 130,
image schema, 8, 30, 139, 167, 179, 136, 137, 139, 140, 175, 235,
187, 207, 208, 224, 317, 320, 243, 244, 246, 250, 252, 254,
330, 331, 340, 342, 349 255, 267, 272, 333, 348, 422,
SOURCE-PATH-GOAL ~, 8, 29, 429, 431, 439
127, 140, 179, 187 verbal ~ of metaphor, 4, 12, 13,
interpretation, 6, 8–10, 19, 20, 26– 21, 22, 34, 123, 248, 283,
28, 32, 33, 47–49, 53, 55, 58, 431
59, 62, 64, 87, 90–93, 110– mapping(s), 10, 13, 20, 24, 28–30,
112, 147–150, 152–158, 160– 46, 61, 79, 81, 95, 96, 100,
162, 165, 167–169, 173–175, 120, 122, 127, 128, 133, 135,
180–184, 187, 190–192, 201– 138, 139, 169, 181, 213, 214,
207, 213–216, 219–221, 224, 217–219, 226, 228, 265, 266,
228–232, 234, 248, 250, 317, 270, 271, 273, 283, 284, 289,
318, 331, 336, 337, 339, 341, 290, 336, 340, 341, 348, 359,
343, 346, 348, 372, 384, 385, 361, 364–367, 369, 370, 373,
389, 395, 397, 403, 404, 408, 375, 376, 384, 391, 393, 407,
411, 418–421, 423, 444 408, 414, 417, 422, 434, 437,
inference(s), 29, 47, 97, 128, 149, 441, 445, 446
151, 169, 230, 387, 445 double ~, 129, 346, 347
integration, conceptual, 122, 236, metaphoric(al) ~, 10, 13, 20, 24,
373, 374, See also blending 31, 97, 102, 104, 107, 110,
intertextuality, 11, 47, 56, 415 111, 112, 132, 149, 155–158,
invariance principle, 207, 208, 373 173, 190, 207, 208, 272, 313,
317, 339, 341, 375
layout, 6, 45, 46, 52, 53, 60–61, 64, metonymic(al) ~, 7, 98, 99, 110,
201 112, 134, 329, 341, 347
logo (see also brand), 6, 30, 45–47, partial ~, 12, 55
51–53, 57–65, 97, 98, 103, medium, 9, 21, 32, 157, 163, 166,
105, 113, 385 (audiologo), 214, 218, 243, 244, 250, 253,
388, 396 255, 258–261, 266, 283, 403,
love, 98, 249, 254, 258, 265, 266, 410, 412, 418, 420, 424, 430,
285, 287, 289, 300, 302, 307, 444
367, 372, 373, 396, 444 metaphor
articulatory forms of ~, 299–300,
307
Subject index 455
artistic (also metaphor in artis tic gestural ~, 65, 298, 300, 303,
texts / discourses, metaphor, 304, 306–308, 310, 311, 316,
creative) ~, 396 347, See also metaphor in
complex ~, 119, 120–122, 127– gesture
131, 133–135, 137, 180, 181, ~ in advertising, 6, 7, 9, 28, 33,
191, 235, 256–260 34, 48, 74, 81, 82, 86, 87,
conceptual ~, 4, 7–10, 12–14, 19, 90–92, 96, 111, 112, 175,
21, 25, 26, 30, 47, 48, 65, 80, 176, 214, 215, 218, 383, 390
96, 112, 119–123, 125–130, ~ in animation, 6, 9, 13, 25, 233,
134, 136, 137, 139, 149, 161, 243, 258, 259, 260, 394, 395
173, 213–216, 218, 219, 234, ~ in artistic texts/discourses, 14,
235, 246, 248–250, 252, 256– 25, 33, 396
261, 265–267, 271, 273, 280, ~ in film, 6, 10–12, 14, 21, 24,
283, 284, 290, 300, 301, 307, 28, 33, 34, 140, 383, 384,
313, 316, 318, 319, 321, 331, 390–392, 396, 397, 403–406,
336, 345, 361, 391, 403, 419, 410–413, 415, 418–420, 429–
422, 429, 430, 444, 445, See 431, 436, 438, 441, 442, 444–
also Cognitive / Conceptual 446
Metaphor Theory ~ in gesture(s), 9, 10, 299, 302–
construal of ~, 6, 10, 12, 14, 19, 309, 312, 316, 318, 321, 323,
20, 31, 33, 64, 336, 384, 385, 329, 336, 337, 346, See also
391, 397 gestural metaphor
conventional (conventionalized) ~ in intonation, 9, 13, 297, 299,
~, 26, 27, 29, 34, 53, 108, 302, 316–318
113, 166, 167, 170, 179, 308, ~ in speech, 10, 297, 303, 304,
342, 406, 407, 410, 422 317, 321, 330–332, 336, 340,
creative ~, 8, 9, 12, 19, 25, 28, 341
34, 175, 244, 422, 429, 430 interaction theory of ~, 12, 24
diagrammatic ~, 331, 342 interaction of ~ with metonymy,
diegetic/nondiegetic ~, 393, 396, 7, 13, 95–100, 104, 109–112,
410, 411, 413, 415 334, 344
directionality in ~, 10, 309, 369, monomodal ~, 9, 19, 22–25, 28,
370, 375, See also invariance 31, 33, 74, 79, 81, 177, 188,
principle 190, 297, 300, 302, 303, 304,
dual encoding in ~, 45, 47, 49, 306, 313, 321, 334, 339, 385,
54, 62 403, 410–414, 420, 429–431,
dynamic ~, 11, 95, 297, 300, 437, 442, 445
313, 316, 319, 320, 321 multimodal ~, 3–14, 19, 21–22,
elaboration of ~, 104, 122, 170, 24, 25, 28–31, 33–34, 45–49,
182, 268, 298, 313–316, 321 51, 53, 55–57, 59, 60, 62–64,
emotion ~, 243, 244, 265–268, 73–75, 79–83, 88, 91, 92, 96–
270–272, 283, 284, 290 99, 105, 111, 112, 119, 120,
122, 123, 125, 130, 137, 139,
456 Subject index
140, 147, 148, 150, 152, 158, 178, 180, 181, 187, 188, 189,
161, 173–175, 180, 181, 183– 191, 199, 200, 202, 205, 207,
185, 190–192, 197–200, 202, 208, 213–215, 217–222, 224–
203, 205, 206, 208, 215, 233– 234, 235, 236, 244, 248, 249,
235, 244, 251, 252, 260, 266, 258, 266, 267, 271, 297, 300,
271, 272, 283, 290, 297–300, 302, 304, 305, 307–310, 312,
302, 304, 307, 308, 310, 312– 313, 315, 316, 319–321, 329,
314, 316, 319, 321, 329–331, 331, 336, 337, 339–341, 344,
334, 336, 339, 341, 342, 347, 346–349, 359, 365, 369, 370,
348, 359, 360–365, 369, 370, 375, 383–389, 392, 393–397,
376, 383, 384, 389–393, 395– 404, 406–423, 430, 431, 439,
397, 403–405, 407, 412, 413, 442, 444, 446
416, 419–424, 429–431, 442, spatial ~, 128, 311, 312, 344–
445 346, 348
naturalization of ~, 63, 396 systems of ~, 62, 121, 128, 129,
ontological ~, 106, 163, 166, 131, 132, 176, 266,
175, 181, 182, 313 transparent ~, 308
orientational ~, 175, 176, 180– verbal ~, 4, 6–8, 10–14, 19, 21–
182, 185, 187, 190 24, 26, 29, 31, 34, 46, 47, 49,
pictorial ~, 3, 6, 8, 9, 19, 22–25, 73, 74, 81, 82, 84, 87, 88, 90,
28, 33, 46, 49, 55, 81, 82, 92, 123, 137, 140, 147, 148, 150–
148, 177, 187, 197, 199, 202, 153, 156, 158–161, 166–168,
243–246, 250, 259, 265–267, 218, 243, 245, 248–250, 256,
271–273, 276, 283, 285, 288– 257, 260, 265–267, 271, 273,
290, 330, 385, 389, 390, 395, 278–280, 282–284, 288–290,
403, 416, 430, 431, 445 297, 300–304, 308–310, 313–
poly-interpretability of ~, 390 317, 383, 396, 397, 405, 408,
primary ~, 119, 121, 122, 128– 409, 422, 430, 431, 435, 445
130, 133, 137, 140, 181, 197, verbalization of conceptual ~, 13,
208, 209, 256, 257, 260 19, 30, 31, 47, 48, 74, 192,
products and processes in ~, 297, 207, 306, 404
300, 319, 321 verbo-gestural ~, 297, 300, 308,
projective ~, 393 310, 312, 315, 321
prosody in ~, 316, 321 verbo-pictorial ~, 11, 86, 91,
sleeping ~ (versus waking ~), 199, 310, 383, 420
316 verbo-visual ~, 173
source and target in ~, 4, 7, 9– visual ~, 6, 8, 23, 24, 31, 46, 49,
14, 19–21, 23–25, 27–31, 33– 54, 107, 131, 135, 147–150,
35, 45–48, 53–57, 59–62, 74, 152, 153, 155–157, 159, 161–
79, 81–83, 86, 88, 92, 97– 163, 165–168, 174, 175, 178,
102, 104, 107, 108, 110, 111, 192, 213–215, 218, 223, 224,
120, 122, 127, 128, 132, 147, 228, 230, 234–236, 244, 245,
148, 155–166, 169, 174, 177,
Subject index 457
248, 250, 271, 290, 330, 390, 100, 102, 103, 106, 108, 110–
413 112, 119, 120, 137, 139, 147,
metaphoricity, 297, 300, 303, 308, 148, 168, 169, 174, 175, 177,
316, 319, 323, 180, 181, 188, 190–192, 199,
activation of ~, 307, 308, 321 207, 213, 233, 234, 251–253,
~ and attention, 308 257–261, 265, 266, 283, 290,
~ as cognitive process, 300, 316, 297, 299, 300, 302, 307, 310,
319 312, 313, 316, 318, 319, 321,
creation of ~, 297, 300 322, 329, 330, 331, 333–337,
foregrounding of ~, 316 339, 341, 342, 344–346, 348,
modality independent ~, 300, 349, 361, 363, 375, 376, 383,
319, 321 385, 387, 395, 404, 405, 413,
metonymy, 7, 10, 12, 13, 23–25, 34, 420–422, 430, 431, 444
61, 90, 95–112, 119–120, sub ~, 95, 100, 103, 104, 106,
128, 130, 134–137, 139, 158, 110, 112
282, 306, 329–343, 346–348, types of ~, 5, 6, 22, 23, 46–48,
349 53, 80, 97, 300, 404, 405
~ and metaphor, 7, 10, 12, 13, models
24, 25, 95–100, 104, 107, mental ~, 46, 48, 60, 112
109–111, 120, 139, 306, 329– abstract ~, 50
332, 334–337, 339–341, 347– Idealized Conceptual Models
348 (ICM, folk models), 225, 243,
external ~, 329, 334, 338, 340, 244, 246, 248–250, 256, 258,
346, 347 260, 261
functions of ~, 98, 111, 139, 347 Modern Times [Charlie Chaplin],
highlighting in ~, 96, 98, 100, 413–415, 421, 422
102, 103, 106, 110, 111 module, 100, 148, 149, 153, 167
internal ~, 329, 334, 335, 343, language ~, 149, 153, 169
347, See also synecdoche perceptual ~, 154, 162
~ mapping, 7, 96, 98, 100, 110, specialized ~, 148, 167
112, 120, 129, 135, 329, 340, motivation, 13, 95, 102, 134, 151,
341, 346, 347 170, 265, 266, 272, 278, 279,
multimodal ~, 119, 120, 134– 288, 289, 396, 397, 419
137, 139, 334, 348 experiential ~, 127, 130, 208,
visual ~ (pictorial ~), 90, 105, 283, 289
134–137 physiological ~, 271, 289
Metropolis [Fritz Lang], 407, 415, music, 4, 5, 6, 10, 13, 19, 21–23,
421, 422 35, 47, 51, 53, 57, 123, 126,
The Million [René Clair], 411, 412, 131, 133, 136, 139, 261, 300,
413, 421 331, 359–378, 383–385, 389–
mode/modality, 2, 4–11, 21–25, 29, 392, 395–398, 403–405, 411–
31, 33, 34, 45–47, 51–53, 62, 413, 415, 421, 430, 431
73–75, 80, 81, 91, 95, 96, text painting in ~, 359, 364–370
458 Subject index
diegetic ~, 393, 396, 397 representation, 23, 24, 29, 54, 100,
nondiegetic ~, 393, 396, 413 102, 149, 178, 199, 204, 207,
multimodal metaphor 248, 256, 259, 301, 346, 360,
definition of ~, 4, 5, 46, 74, 82, 376, 377, 397, 405, 412, 430
97, 120, 148, 174, 191, 198– artistic ~, 33
199, 233, 235, 266, 297, 299, cinematic / film ~, 404, 411, 415
336, 339, 383, 404, 429, 430, dynamic ~, 341, 347, 373
See also metaphor, multimo- gestural ~, 330, 336
dal medium specific ~, 260
explicitness of ~, 13, 55, 56, 100, metaphorical ~, 9, 111, 243, 245,
397 260, 261, 339
types of ~, 6, 9, 25, 28, 33, 47, multimodal ~, 6, 21, 29, 34, 88,
53, 91, 404, 405, 430 244, 283, 329, 330, 331, 341,
~ versus monomodal metaphor, 342, 347, 348
9, 22–25, 300, 302, 313, 413, surface ~, 97
431 visual ~ (pictorial ~), 31, 54, 56,
105, 106, 109, 154, 198, 207,
narrative, 11, 33, 111, 178, 185, 245, 250, 251, 256, 258, 259,
281, 290, 394, 397, 405, 415, 261, 266, 285
430, 434, 446 resemblance, perceptual, 31
~ meaning, 178 rhythm, 100, 310, 332, 367, 372,
~ and action/events, 11, 178 373, 378, 386, 393, 395
~ and time/chronology, 185, 190, runes, pictorial, 23, 59, 245, 246,
36 247, 250, 254, 255, 272, 275,
~ image, 187 276, 277, 280, 281, 283, 289
~ styles, 198
~ context, 290, 446 scenario, 11, 48, 154, 176, 187, 202,
film ~, 409 215, 223–228, 234, 235, 254,
276, 280, 283, 297, 305, 429,
October [Sergei Eisenstein], 411, 430, 445, 447
422 Semiotic(s), 5, 10, 329, 334
Old and New [Sergei Eisenstein], Jakobsonian ~, 331, 332, 334,
411, 416, 417, 422 335, 348
Peircean ~, 283, 331, 348
personification (see also anthropo- ~ process, 304, 306
morphism), 13, 28, 51, 62, sensory
80, 83, 84, 100, 106, 112, ~ domain, 80
113, 120, 224, 386 ~ experience, 74, 79, 80, 89, 91
pragmatic adjustment, 151 ~ motor abilities, 342
~ perception, 73, 80
Relevance Theory, 8, 24, 147, 148– ~ schema, 167
166, 168, 199 ~ information, 167
Subject index 459
simile, 24, 25, 55, 74, 88, 186, 197, universality vs. individuality, 174,
198, 214, 411 181, 190, 289
sign
iconic ~, 162, 256, 337 verbal image, 137, 405, 408–410,
indexical ~, 59, 245, 246, 251, 416, 417
255, 256, 260, 265, 272, 274, visual-conceptual interface, 147,
276, 281, 283, 291 156–158, 162, 163, 165
sonic analogs, 10, 363–367, 369,
370, 376, 377 winespeak, 6, 73–74, 79–80, 85, 86,
song, 123, 136, 359, 367, 369–375, 92, 93
384, 385, 390, 396, 398 wine tasting note, 7, 74, 75, 91
sound, 3–6, 10, 19, 21, 22, 23, 25, Wolf Creek [Greg McLean], 434–
31, 47, 53, 57, 80, 96, 97, 436, 441, 444–446
103–106, 214, 233, 234, 250,
251, 255, 261, 298, 299, 321,
359–368, 377, 383–397, 404,
405, 412, 413, 419–421, 430
diegetic versus nondiegetic ~,
393, 396, 397, 405, 412, 413
Strike [Sergei Eisenstein], 406, 407,
409, 411, 422
Surrealist art, 24
The Swindle [Federico Fellini], 418,
419, 421, 422
synecdoche, 24, 331, 335, 336, 342,
347, See also internal meton-
ymy
syntactic structure, visual, 333, 342
Danto, Arthur C., 403, 420, 423 120, 122, 123, 137, 140, 147–
Dastjerdi, H. Vahid, 181 149, 155, 174, 175, 177, 180,
Davies, Gray, 65 181, 191, 197–199, 202, 207,
Day, Sean, 80 214, 216, 217, 220, 233–235,
de Chernatony, Leslie, 46 243, 244–249, 251–254, 257,
Deacon, Terrence W., 377 259, 260, 261, 262, 265, 266,
Deutscher, Guy, 214 271, 272, 274–277, 279, 280,
Díez Velasco, Olga Isabel, 7, 95, 97, 283, 288, 291, 299, 300, 310,
99, 110, 158 313, 314, 330, 336, 339, 348,
Dines-Levy, Gail, 176 349, 378, 383, 384, 387, 389,
Dirven, René 12, 24, 95, 111, 112, 390, 394, 395, 397, 404, 413,
120, 139, 330, 334 422, 423, 429, 430, 431, 444
Duncan, Susan, 316, 320 Fornäs, Johan, 363
Durand, Jacques, 23 Fox, John, 52
Fox, Renata, 52
Edwards, Janis L., 173, 174, 176, Franke, George R., 214
179, 216 Frick, Wolfgang, 65
Eerden, Bart, 9, 13, 29, 244, 249, Fricke, Ellen, 331, 333, 343
254, 257, 260, 272 Friedman, Vanessa, 56
Efron, David, 298, 320 Furuyama, Nobuhiro, 349
Eggertsson, Gunnar, 11, 14, 395,
444 Geeraerts, Dirk, 291, 341
Eisenstein, Sergei, 405–411, 416– Gentner, Dedre, 13, 33, 214, 225,
418 226, 236
Ejxenbaum, Boris, 405, 408, 410 Gevaert, Caroline, 27, 291
El Refaie, Elizabeth, 8, 13, 34, 110, Gibbs, Raymond, 14, 20, 21, 28, 34,
149, 150, 155, 176, 177, 192, 121, 139, 149, 151, 170, 214,
193, 207, 216, 219, 233, 348, 235, 248, 261, 321, 330, 331,
390, 405, 410 336, 348, 395, 421, 445
Elliot, Richard, 51 Gilmartin, Patricia, 216
Entzenberg, Claes, 423 Gilot, Françoise, 313
Epstein, Jean, 424 Gineste, Marie-Dominique, 8, 24,
Eves, Anmarie, 214 152
Gioia, Dennis, 51
Fauconnier, Gilles, 7, 8, 10, 20, 46, Gluck, Malcolm, 74
122, 169, 235, 330, 370, 378, Glucksberg, Sam, 214, 228
446 Goatly, Andrew, 31
Fein, Ofer, 244 Goldin-Meadow, Susan, 332, 364
Feldman, Ofer, 216 Gombrich, Ernst H., 193
Fischer, Roger A., 182 Goodwin, Charles, 332
Fishbein, Martin, 232 Goossens, Louis, 98, 330, 341
Fodor, Jerry, 8, 149, 153, 167 Goscinny, René, 245, 247, 250, 261
Forceville, Charles, 4–11, 13, 14, Grady, Joseph, 27, 65, 121, 122,
20, 24, 25, 28, 29, 32–35, 46, 208, 256, 265
47, 49, 51, 55, 60, 61, 74, 81, Green, Christopher D., 59, 194
82, 86, 88, 91, 92, 95–98, 112, Groarke, Leo, 216, 220
Author index 463
Langacker, Ronald W., 208, 340 323, 330–333, 336, 339, 348,
Larson, Steve, 35 364, 430
Lassen, Inger, 49 Mumford, Alan, 182
Laudenbach, Peter, 62 Muniz, Albert M., 63
Lehrer, Adrienne, 74, 77 Musolff, Andreas, 11, 48, 429, 430,
Lidov, David, 365 447
Lindsay, Greg, 59
Lindsay, Vachel, 420, 424 Neale, Steve, 446
Lodge, David, 331, 341 Neumann, Ragnhild, 304
Loewenstein, Jerry, 13, 33 Nusbaum, Howard C., 365
Low, Graham, 321
O’Guinn, Thomas C., 63
Maalej, Zouhair, 29 O’Halloran, Kay, 5
Maes, Alfons, 8, 14, 149, 231, 348, Oakley, Todd, 122, 235
390 Ochs, Eleanor, 332, 349
Maestri, George, 248, 249, 258 Okrent, Arika, 365
Mandler, Jean, 340 Ortony, Andrew, 3, 19, 228
Matsuki, Keiko, 268
Matsunaka, Yoshihiro, 9, 13, 29, Pankhurst, Anne, 111
248, 259, 261, 268–270, 273, Panther, Klaus-Uwe, 95, 98, 120,
284, 291 139, 330, 339, 342
Mautner, Gerlinde, 65 Papasolomou, Ioanna, 174
McCarthy, Michael S., 214 Pascual, Esther, 20
McCloud, Scott, 245, 291 Peirce, Charles Sanders, 256, 262,
McLuhan, Marshall, 21 272, 283, 289, 331, 332, 334,
McMahon, Jennifer A., 154 337, 343, 346, 348, 377
McNeill, David, 9, 25, 34, 299, 302, Peñamarín, Cristina, 149
316, 320, 322, 330–332, 336, Perlman, Marcus, 14
337, 364 Petrakis, Stefanos, 20
McQuarrie, Edward, 34, 49, 96, 97, Peynaud, Émile, 74, 77
113, 156, 165, 214, 220 Philippe, Robert, 173
Messaris, Paul, 64, 100, 175, 177 Phillips, Barbara, 34, 49, 96, 97,
Mick, David, Glen, 34, 214 113, 156, 165, 175, 214
Miller, Christine M., 34 Pierrehumbert, Janet, 316
Mínguez, Norberto, 154 Pilkington, Adrian, 150
Mittelberg, Irene, 10, 13, 95, 98, Pimentel, Ronald W., 57, 58, 65
299, 304, 313, 319, 330, 331, Plumb, Steve, 215, 216,
333, 336, 346, 348, 349, 430 Polzenhagen, Frank, 112
Moingeon, Bertrand, 50 Pörings, Ralf, 12, 24, 95, 120, 330,
Morford, Jill, 331 334
Morris, Desmond, 304 Porras, Jerry I., 51
Morrison, Susan, 173, 216 Pragglejaz Group, 14, 301
Mothersbaugh, David L., 214 Pratt, Michael G., 53
Müller, Cornelia, 9, 13, 25, 34, 214, Proctor, Stella, 174
299, 301, 302, 304–306, 308, Proctor, Tony, 174
310, 311, 313, 314, 316, 318– Pudovkin, Vsevolod, 411, 416, 417
Author index 465
Radden, Günther, 95, 98, 99, 111, Speckmann, Gerald, 304, 305, 306
120, 139, 330, 340, 341 Sperber, Dan, 8, 24, 97, 147, 148
Rafaeli, Anat, 53 Spitzer, Michael, 35
Ramachandran, Vilayanur, 80 Stanford, W(illiam) B(edell), 404
Ratner, Leonard G., 364 Steen, Gerard J., 28, 32, 139, 248,
Rayson, Paul, 52 261, 321, 421, 445
Richards, Ivor Armstrong, 23, 423 Stöckl, Hartmut, 100, 103
Ricoeur, Paul, 11 Stockwell, Peter, 27
Ritchie, David, 174, 181, 182, 187, Streeck, Jürgen, 304, 318, 332
191, 192 Strunck, Jeanne, 49
Rogers, Priscilla, 52 Sun, Sewen, 34, 61, 96, 197, 214,
Rohdin, Mats, 10, 11, 13, 14, 34, 220, 222
404, 411, 423, 429 Swales, John, 52
Rompay, Thomas, van, 34 Sweetser, Eve E., 20, 129, 139, 299,
Rothbart, Myron, 63 330, 331, 336, 337, 338, 345
Rozik, Eli, 24
Ruiz de Mendoza, Francisco, José, Talebinejad, M. Reza, 181
7, 95, 97, 99, 110, 149, 151, Tanaka, Keiko, 96, 148
158, 163, 170, 208 Taub, Sarah, 322, 331, 336
Taylor, John, 98
Salt, Barry, 405 Taylor, Marjorie, 63
Scart, Véronique, 152 Templin, Charlotte, 173
Schilperoord, Joost, 8, 14, 149, 231, Tendahl, Markus, 149, 151, 170
348, 390 Teng, Norman, 8, 13, 34, 61, 96,
Schirato, Tony, 178 197, 198, 214, 220, 222
Schultz, Majken, 50, 51, 64, 65 Terkourafi, Marina, 20
Schwartz, Jonathan, 48 Teßendorf, Sedinha, 304, 305, 306,
Scott, Linda, 96, 175, 214 323
Seitz, Jay A., 175 Thagard, Paul, 369
Semino, Elena, 12 Thibault, Paul J., 5, 176
Shapiro, Michael, 334 Thomas, Frank, 248, 249, 258
Shen, Yeshayahu, 33, 213, 225, Thompson, Kristin, 256, 391, 397,
226, 228, 235, 236 405, 406, 415, 420
Shibles, Warren, 23 Thorau, Christian, 35
Shinohara, Kazuko, 9, 13, 29, 248, Thornburg, Linda L., 95, 98, 120,
259, 261, 268–270, 273, 284, 139, 330, 339, 342
291 Tillyard, E.M.W., 13, 27
Shintel, Hadas, 365, 377 Tom, Gail, 214
Shore, Bradd, 28, 319 Tomasello, Michael, 363
Shotter, John, 51 Turner, Mark, 7, 8, 10, 13, 20, 25–
Simons, Jan, 29 28, 35, 46, 61, 98, 106, 122,
Singer, Peter, 447 139, 169, 181, 207, 235, 248,
Slobin, Dan I., 320 330, 373, 377, 378, 407, 410,
Smith, Gregory, W. H., 176 419, 446
Smith, Ken, 23 Tversky, Barbara, 23
Soenen, Guillaume, 50 Tynjanov, Jurij, 406, 408
466 Author index
Uderzo, Albert, 245, 247, 250, 261 Yu, Ning, 7, 8, 13, 28, 80, 96, 98,
Ungerer, Fiedrich, 55 126, 129, 132, 139, 248, 291
Urios-Aparisi, Eduardo, 7, 13, 14, Yus, Francisco, 8, 12, 13, 100, 152,
112 155, 169, 348, 390