Emotion Review

http://emr.sagepub.com/

Universality Revisited
Nicole L. Nelson and James A. Russell
Emotion Review 2013 5: 8
DOI: 10.1177/1754073912457227
The online version of this article can be found at:
http://emr.sagepub.com/content/5/1/8

Published by:
http://www.sagepublications.com

On behalf of:

International Society for Research on Emotion

Additional services and information for Emotion Review can be found at:
Email Alerts: http://emr.sagepub.com/cgi/alerts
Subscriptions: http://emr.sagepub.com/subscriptions
Reprints: http://www.sagepub.com/journalsReprints.nav
Permissions: http://www.sagepub.com/journalsPermissions.nav

>> Version of Record - Jan 2, 2013

What is This?

Downloaded from emr.sagepub.com by ancuta anca on October 25, 2014

457227

EMR5110.1177/1754073912457227Emotion ReviewNelson & Russell Universality Revisited

2013

Emotion Review
Vol. 5, No. 1 (January 2013) 815
The Author(s) 2013
ISSN 1754-0739
DOI: 10.1177/1754073912457227
er.sagepub.com

Universality Revisited
Nicole L. Nelson

Department of Psychology, Brock University, Canada

James A. Russell

Department of Psychology, Boston College, USA

Abstract
Evidence does not support the claim that observers universally recognize basic emotions from signals on the face. The percentage of
observers who matched the face with the predicted emotion (matching score) is not universal, but varies with culture and language.
Matching scores are also inflated by the commonly used methods: within-subject design; posed, exaggerated facial expressions
(devoid of context); multiple examples of each type of expression; and a response format that funnels a variety of interpretations
into one word specified by the experimenter. Without these methodological aids, matching scores are modest and subject to various
explanations.

Keywords
culture, emotion, facial expression, universality

Let us define the universality thesis as the claim that certain

human facial expressions are signals of specific basic emotionssuch as happiness, surprise, fear, anger, disgust, and sadnesssignals universally recognized by human beings whatever
their cultural background or spoken language. Reviewers of
data relevant to this claim have gone back and forth. Early
reviewers were critical, Izard (1971) and Ekman, Friesen, and
Ellsworth (1972) supportive, Russell (1994) critical. We examine
evidence published since Russells (1994) review.
Both proponents and critics of the universality thesis agree
that human beings derive information relevant to emotion from
the faces of others. The question is: What evidence would support the universality thesis over alternative accounts of what
information the face conveys? To support the universality thesis
specifically, matching scores (the percentage of observers
matching the face to the specific predicted emotion) must be
(a) high and (b) universal, that is, reasonably invariant across
culture, language, method, and so on. As proponents Haidt and
Keltner (1999, p. 238) wrote, If the universality thesis is correct then those expressions that are pancultural should elicit
very high recognition rates, generally in the 7090% range . . .
even when methodological constraints are relaxed.
We searched PsychInfo for all peer-reviewed journal articles
published between 1992 and 2010 that included the words

emotion* and cultur* in the title, abstract, or keywords. From

the resulting 6,566 entries, we selected all studies that were not
reviewed by Russell (1994), that included a cross-cultural comparison, that had nonclinical adult samples, and that examined
recognition of adult facial expressions that met criteria specified
by Ekman and Friesen (1978). (This last criterion excluded
studies on facial dialects in which individuals posed culturalspecific expressions; see Elfenbein, 2013.) The result was 21
studies that provided 57 sets of data.

Cross-Cultural Judgment Studies

Of the 57 data sets that met our criteria, 39 had a similar enough
method to allow statistical comparisons and are listed in the
Appendix. The remaining 18 sets are discussed in later sections.
The 39 sets used what we call the standard method: at least four
of the hypothesized basic emotions, posed facial expressions, a
forced-choice response format, and a within-subjects design.
Table 1 compares the newer data sets seen in the Appendix
with the older ones reviewed by Russell (1994). Newer studies
yielded similar but slightly lower matching scores. This change
is difficult to explain. Newer studies were carried out by new
experimenters with new photographs of perhaps less intense

Author note: This research was supported by two grants from the National Science Foundation: No. 1025563 awarded to James A. Russell and No. 1064757 awarded to Nicole
L. Nelson. We thank Erin Heitzman, Alan Fridlund, Rachael Jack, Mary Kayyal, Joe Pochedley, Alyssa McCarthy, and Sherri Widen for comments on a draft of this article.
Corresponding author: Nicole L. Nelson, Department of Psychology, Brock University, 500 Glenridge Ave., St. Catharines, Ontario L2S 3A1, Canada. Email: nnelson@brocku.ca

Downloaded from emr.sagepub.com by ancuta anca on October 25, 2014

Nelson & Russell Universality Revisited 9

Table 1. Average matching scores for Western literate, non-Western literate, and isolated illiterate samples based on the standard method
Facial expression

Happiness

Sadness

Pre-1994
19922010

Pre-1994
19922010

Pre-1994
19922010

95.1
88.6

86.5
74.2

87.1
90.7

75.1
67.1

92.0
84.0

53.5
51.0

Overall Mean

89.6

67.9

Anger

Fear

Literate, Western cultures

79.1
77.3
75.2
68.9
Literate, non-Western cultures
61.8
63.2
60.8
46.3
Illiterate, isolated cultures
56.0
46.0
33.0
30.0
61.0

55.3

Surprise

Disgust

Mean

87.4
81.3

81.1
64.7

77.7
86.0

69.0
51.6

36.0*
58.0

29.0*
44.0

84.4
75.5

72.3
67.1

52.1
50.0

71.1

56.6

Note: Matching score is the percentage of observers who selected the predicted label. Pre-1994 refers to articles reviewed by Russell (1994); 19922010 refers to articles
summarized in the Appendix. Average is a mean, except matching scores for the pre-1994 illiterate isolated samples are sometimes medians (marked with an *) because some
sample values were not provided in the primary source.

Figure 1. Matching scores are the percentage of observers who selected the predicted label of studies in the Appendix (i.e., studies published between
1992 and 2010) for six facial expressions. Error bars represent standard error, except for the illiterate isolated sample, for which there is but one sample.

expressions (e.g., Beaupr & Hess, 2005). Or perhaps the

greater amount of data now simply provides a better estimate of
matching scores.
In the newer studies (those listed in the Appendix), matching scores varied with culture: For each emotion, Figure 1
shows matching scores for Western literate, non-Western literate, and isolated illiterate observers. Reliability of the large
differences seen in Figure 1 was verified in several ways. The
nine studies from the Appendix that included both Western and
non-Western literate observers are listed in Table 2. In all nine
studies, mean matching score was higher in the Western than
non-Western literate sample. Happiness and surprise
expressions showed no reliable cultural difference (with a
two-tailed sign test, all ps > .18), but for sadness, anger,
disgust, and fear expressions, non-Western matching
scores were reliably lower (all ps < .01).

We next updated Russells (1994) statistical comparison of

Western literate versus non-Western literate samples. Matching
scores from 54 samples (those reviewed by Russell together
with those listed in the Appendix) were analyzed with a mixeddesign two-factor analysis of variance. Each sample was
treated as a single case (n = 31 for Western cultures and n = 23
for non-Western cultures). Culture (Western or non-Western)
was a between-subjects factor. Type of expression (happiness, surprise, sadness, anger, fear, or disgust) was
a within-subject factor. Greenhouse-Geisser corrections for
lack of sphericity were used where needed. The effect of culture was significant, F(1, 52) = 14.90, p < .001, as was the
effect of type of expression, F(5, 245) = 46.33, p < .001. The
Culture Expression interaction was also significant, F(5,
245) = 8.01, p < .001. A simple effects analysis pursued the
interaction effect. When experiment-wise alpha level was

Downloaded from emr.sagepub.com by ancuta anca on October 25, 2014

10 Emotion Review Vol. 5 No. 1

Table 2. Literate Western (bold) versus literate non-Western comparisons of matching scores
Study
Society/Culture

Facial expression

N
Happiness

Beaupr and Hess (2005)

Canadian
20
67.0
20
71.0
Chinese
20
76.0
Sub-Saharan African
Biehl, et al. (1997)
98.0
Polish
75
Hungarian
45
98.0
32
99.0
Sumatran
34
99.0
Vietnamese
Elfenbein, Beaupr, Lvesque, and Hess (2007)
Canadian
20
88.0
20
80.0
Gabonese
Elfenbein, Mandal, Ambady, Harizuka, and Kumar (2004)
American
50
93.5
Indian
50
99.0
50
98.5
Japanese
Jack, Blais, Scheepers, Schyns, and Caldara (2009)
99.0
European Caucasian
13
13
99.0
East Asian
Matsumoto et al. (2002)
American
138
93.3
137
89.4
Japanese
Tcherkasoff and de Suremain (2005)
97.0
French
96
96
94.0
Burkinabe
Wickline, Bailey, and Nowicki (2009)
American
72
93.0
American
102
89.0
84.0
European
30
30
79.0
African
Yik and Russell (1999)
Canadian
30
100.0
30
100.0
Chinese
30
100.0
Japanese
Mean Western
Mean non-Western

92.2
90.4

Sadness

Anger

Fear

Surprise

Disgust

Mean

56.0
38.0
29.0

50.0
45.0
49.0

33.0
28.0
27.0

56.0
49.0
46.0

52.4
46.2
45.4

88.0
83.0
81.0
80.0

85.0
87.0
79.0
81.0

69.0
74.0
57.0
67.0

89.0
94.0
89.0
92.0

83.0
84.0
76.0
58.0

85.3
86.7
80.2
79.5

71.0
41.0

72.0
63.0

68.0
50.0

62.0
32.0

72.2
53.2

54.0
39.5
68.5

79.0
65.0
51.5

77.0
75.0
49.5

82.0
78.0
92.5

34.5
53.5
20.0

70.0
68.3
63.4

89.0
90.0

89.0
80.0

88.0
64.0

84.0
91.0

88.0
69.0

65.7
47.8

52.3
54.7

78.7
88.6

89.5
82.2

72.5
70.1

83.0
62.0

88.0
53.0

85.0
65.0

77.0
82.0
85.0
70.0

65.0
80.0
71.0
50.0

74.0
78.0
70.8
76.0

86.7
76.7
80.0

73.3
46.7
56.7

66.7
23.3
53.3

76.7
83.3
93.3

56.7
56.7
30.0

88.3
68.5

77.3
82.3
77.7
68.8

76.7
64.5
68.9

73.8
63.7

73.6
59.1

71.5
50.9

83.2
90.0

64.7
48.5

76.9
65.9

Note: Matching score is the percentage of observers who selected the predicted label.

determined by Bonferronis method, there were no significant

effects of culture for happiness, surprise, and sadness
expressions, but Western observers matching scores were
higher than were non-Western observers scores for anger,
F(1, 52) = 25.05, p < .001, fear, F(1, 52) = 28.39, p < .001,
and disgust, F(1, 52) = 12.02, p = .001. In short, the pattern
reported by Russell (1994) remained intact with the addition
of new data: Matching scores varied reliably with culture for
three of six expressions.
Because our search found but one study of isolated, illiterate observers, that sample was not included in the statistical

comparisons reported above. Still, as emphasized by Ekman

(1980), such studies are a necessary test of the universality
thesis. Table 3 provides matching scores from the three studies
in the earlier review plus those from the Tracy and Robins
(2008) study, in which high matching scores for the happiness expression (84%) showed that these observers understood the task presented to them. Matching scores for all other
expressions were lower: M = 43.2%.
Matching scores also varied with language. Of the studies
listed in the Appendix in which we could determine the
language used for testing, there were 29 samples tested in an

Downloaded from emr.sagepub.com by ancuta anca on October 25, 2014

Nelson & Russell Universality Revisited 11

Table 3. Matching scores from isolated illiterate observers, based on the standard method
Society/Culture

Language tested in

Facial expression
Happiness

Sadness

Anger

Fear

Burkinabea

Foreb
Foreb
Sadongb

Surprise

Disgust

Median

58.0

44.0

38.0
<19.0c
36.0

29.0
44.0
<23.0c

47.5

50.0
46.0
47.5

<37.0

<36.5

Appendix
39

Dioula

84.0

51.0

33.0

18
14
15

Neo-Melanesian Pidgin
Fore
Bidayuh

99.0
82.0
92.0

55.0
N/A
52.0

56.0
50.0
64.0

88.0

52.0

53.0

Median

30.0
Pre-1994
46.0
54.0
40.0
43.0

Note: Matching score is the percentage of observers who selected the predicted label.
aTracy and Robins (2008); bEkman, Sorenson, and Friesen (1969); cModal choice was another emotion, and no matching score was given, although it was less than the figure
presented here.

Table 4. Differences due to language: Mean matching score of within-study comparisons of language
Society/Culture
Language
Indiana
English
Hindi
Mexicanb
English
Spanish
Burkinabec
French
Moore
Palestiniand
English
Arabic
Mean English
Mean non-English

Facial expression
Happiness

Sadness

Anger

Fear

Surprise

Disgust

Mean

99.0
99.0

94.0
90.0

90.0
89.0

81.0
74.0

90.0
90.0

90.0
78.0

68.0
62.0

58.0
58.0

65.0
56.0

87.0
77.0

57.0
41.0

92.0
96.0

75.0
52.0

60.0
54.0

65.0
67.0

62.5
66.5

85.0
65.0

73.0
53.0

62.0
44.0

75.0
88.0

90.8
88.4

70.8
62.0

73.0
67.3

71.5
63.3

83.8
86.3

82.3
68.8

73.7
62.8

73.0
65.5

79.7
70.3

66.0
64.5

76.4
69.7

Note: Matching score is the percentage of observers who selected the predicted label. Kayyal and Russell (2012) allowed observers to attribute more than one emotion to a
single expression. The data presented here represent the percentage of observers who selected the target category as one of the emotions conveyed by the expression.
aMatsumoto and Assar (1992); bMatsumoto, Angus-Wong, and Martinez (2008); cTcherkasoff and de Suremain (2005); dKayyal and Russell (2012).

Indo-European language (all 20 Western samples and nine of

the 19 non-Western ones) and eight samples in a non-IndoEuropean language. Treating each expression as a separate case
resulted in 204 data points. Matching scores were higher with an
Indo-European language (158 data points; M = 73.2%) than
without (46 data points; M = 63.4%), t(202) = 3.03, p = .002.
Table 4 gives a more specific comparison of studies that tested
observers from the same culture in more than one language
with the same result.

are inflated by method factors. Each of several aspects of the

standard method pushes scores higher by some amount.
Combined, these method factors can produce highly exaggerated scores. The rebuttal that each individual method factor produces only a small effect is factually incorrect, and it misses the
point that, even if each effect were small, their cumulative
impact can be large. The magnitude of the matching scores as
they change with method is thus a key issue. Here we consider
recent evidence on three of those factors.

The Standard Method Inflates Matching Scores

Posed Facial Expressions

The matching scores discussed so far cannot be taken at face

value, according to Russells (1994) argument that such scores

The universality thesis is about emotions and facial expressions

that occur spontaneously. However, the standard method uses

Downloaded from emr.sagepub.com by ancuta anca on October 25, 2014

12 Emotion Review Vol. 5 No. 1

expressions posed with the purpose of conveying a single emotion as clearly as possible and then selected from a larger set as
the best at doing so. Our search turned up two studies of spontaneous expressions. Shown photos of spontaneous expressions
predicted to convey a single basic emotion, British, American,
and Japanese observers yielded matching scores ranging from
31% to 35% (Matsumoto, Olide, Schug, Willingham, & Callan,
2009). Similarly, shown photos of spontaneous expressions
provided by Ekman (1980) to illustrate the universality thesis,
American observers yielded a mean matching score of 35%
(Naab & Russell, 2007). Both studies produced matching scores
noticeably lower than the overall mean (70.9%) from studies in
the Appendix, which used posed expressions. Use of posed
expressions inflates matching scores relative to spontaneous
expressions.

Presentation of Multiple Expressions

In the standard method, observers are shown more than one
facial expression (an average of 47 for studies in the Appendix)
through previewing, practice trials, or a within-subjects
design. Principles of perception predict that observers judge
one facial expression relative to others (Russell, 1991).
Especially with a forced-choice response format, observers
may come to realize which types of expressions are being presented and can then use a process of elimination to help make
their judgments. Empirically, the commonly used withinsubjects design results in higher matching scores than does a
between-subjects design (Russell, 1994). Yik, Widen, and
Russell (2012) found that high matching scores (M for three
samples = 67.5%) for the disgust expression depended on
having recently seen an anger expression; otherwise, the
mean dropped to 35%. With the 20 data sets in the Appendix
that included all six emotions, matching scores were higher for
observers who had viewed more faces (30 or more; n = 10;
M = 78%) than those who had viewed fewer (624 faces;
n = 10; M = 67%), F(1, 18) = 9.026, p = .007.

Response Format
The standard method uses a forced-choice response format in
which observers select one emotion from a short list provided by
the experimenter. This format creates many problems. Here, we
highlight three. First, the observers spontaneous interpretation
of a facial expression is not limited to emotions. Observers reliably report social messages (Yik & Russell, 1999), incipient
actions (Tcherkassof & de Suremain, 2005), and the antecedent
situation (Widen, 2013) or its appraisal (Scherer & Grandjean,
2008). The standard forced-choice format channels all interpretations into an emotional one.
Second, when observers were allowed to attribute more than
one emotion to an expression, they did so. In one study, they
selected for each face, on average, four of the eight available
emotions (Kayyal & Russell, 2012). Quantitative ratings similarly show that observers see multiple basic emotions in the
same face, albeit to different degrees (Beaupr & Hess, 2005;

Mandal, Bryden, & Bulman-Fleming, 1996; Tcherkassof & de

Suremain, 2005). Observers reliably rate faces on broad bipolar
dimensions such as pleasuredispleasure and arousalsleepiness
(Gao, Maurer, & Nishimura, 2010). When observers were
allowed to provide any emotion label they wanted for a given
face, they generated many (Haidt & Keltner, 1999; Rosenberg
& Ekman, 1995). Thus, the forced-choice format shoehorns
each observers full attribution of various emotions into one of
a few preselected categories.
Third, with a forced-choice format, a majority of observers
attribute qualitatively different basic emotions to the same
face, depending on the options available (Russell, 1993). Frank
and Stennett (2001) showed that with the word surprise omitted from the list in the traditional format, the surprise expression was called happy by 61% of observers. They concluded
that the standard forced-choice format can demand agreement (2001, p. 83).
Russell (1994) noted that the traditional forced-choice format does not allow observers to select no emotion, an emotion
not on the list, or more than one emotion. Frank and Stennett
(2001) agreed that a none option was needed and found that
7.8% of observers chose none in response to posed expressions of hypothesized basic emotionsevidence that indeed
some observers do not find any of the available options applicable. In the 39 sets of data in the Appendix, only seven provided
a none option. Although adding none is useful, it is no panacea: The full range of each observers interpretation is still funneled into one category, although now including none as an
additional category. The observer still cannot provide their own
spontaneous interpretation of the expression, nor can they indicate that the expression conveys multiple emotions, an emotion
not on The list, or nonemotional information. Thus, even with a
none option, the forced-choice format funnels a range of
responses into one option, thereby oversimplifying the observers
response and inflating the matching score.

Do Humans Universally Recognize Basic

Emotions from Facial Expressions?
Like the evidence before it (Russell, 1994), the evidence
reported between 1992 and 2010 (listed in the Appendix) shows
that for only one emotionhappinessdid a large percentage
of observers (89.6%) match it to the predicted facial expression
across cultures and languages. For negative emotions (sadness,
anger, fear, and disgust), Western observers gave the highest
matching scores (M = 71%). Non-Western but still literate
observers gave lower scores (57%), and illiterate observers isolated from the West lower still (39%). Both non-Western groups
failed to reach Haidt and Keltners (1999) 7090% criterion.
Furthermore, these observed matching scores, low as they are,
were inflated by the method used to gather them.
Shariff and Tracy (2011, p. 407) defended the universality
thesis, writing of easily recognizable signals even in geographically and culturally isolated populations. Against the objection
that matching scores are low, vary with culture and language,
and are inflated by method, they pointed to the statistical

Downloaded from emr.sagepub.com by ancuta anca on October 25, 2014

Nelson & Russell Universality Revisited 13

significance of matching scores: we believe these findings

should be held to the same accepted standard of statistical significance as other empirical results. This statistical-significance
defense rests on an elementary error of statistical inference.
That the data reach statistical significance rules out the null
hypothesis that observers matched expressions with emotions
randomly, but does not rule out various substantive alternative
hypotheses or the possibility of method artifacts. Ruling out the
null hypothesis does not rule in any one interpretation of the
data. There are many alternative interpretations of nonrandom
responding; Russell (1994) listed eight, and there are undoubtedly others. These alternatives all predict that matching scores
are statistically significant, and therefore statistical significance
cannot decide among them.
Evidence does not support the claim that facial expressions
are preinterpreted signals for specific basic emotions universally recognized by human beings. But neither are humans clueless when interpreting a face. When asked what emotion a face
expresses, humans do not answer randomly; they figure out an
answer. Faces might provide information on core affect, social
messages, incipient actions, situations, or appraisals; and from
that information, observers might figure out the expressers
emotion. We see no use for further studies using the standard
method to discover that humans are not random when matching
expressions with emotions. Rather, we see the need for hypotheses and evidence on the specific processes involved when an
observer interprets anothers facial expression.

References
Beaupr, M. G., & Hess, H. (2005). Cross-cultural emotion recognition
among Canadian ethnic groups. Journal of Cross-Cultural Psychology,
36, 355370.
Biehl, M., Matsumoto, D., Ekman, P., Hearn, V., Heider, K., Kudoh,
T., & Ton, V. (1997). Matsumoto and Ekmans Japanese and Caucasian facial expressions of emotion (JACFEE): Reliability data
and cross-national differences. Journal of Nonverbal Behavior, 21,
321.
Ekman, P. (1980). The face of man. New York, NY: Garland Publishing.
Ekman, P., & Friesen, W. (1978). The facial action coding system. Palo
Alto, CA: Consulting Psychologists Press.
Ekman, P., Friesen, W. V., & Ellsworth, P. (1972). Emotion in the human
face: Guidelines for research and an integration of findings. New
York, NY: Pergamon Press.
Ekman, P., Sorenson, E. R., & Friesen, W. V. (1969). Pan-cultural elements
in the facial displays of emotions. Science, 164, 8688.
Elfenbein, H. A. (2006). Learning in emotion judgments: Training and
the cross-cultural understanding of facial expressions. Journal of
Nonverbal Behavior, 30, 2136.
Elfenbein, H. A. (2013). Nonverbal dialects and accents in facial expressions of emotion. Emotion Review, 5, 9096.
Elfenbein, H. A., & Ambady, N. (2003). When familiarity breeds accuracy:
Cultural exposure and facial emotion recognition. Journal of Personality and Social Psychology, 85, 276290.
Elfenbein, H. A., Beaupr, M., Lvesque, M., & Hess, U. (2007). Toward a
dialect theory: Cultural differences in the expression and recognition of
posed facial expressions. Emotion, 7, 131146.
Elfenbein, H. A., Mandal, M. K., Ambady, N., Harizuka, S., & Kumar, S.
(2004). Hemifacial differences in the in-group advantage in emotion
recognition. Cognition & Emotion, 18, 613629.

Frank, M. G., & Stennett, J. (2001). The forced-choice paradigm and the
perception of facial expressions of emotion. Journal of Personality and
Social Psychology, 80, 7585.
Gao, X., Maurer, D., & Nishimura, M. (2010). Similarities and differences
in the perceptual structure of facial expressions in children and adults.
Journal of Experimental Child Psychology, 105, 98115.
Haidt, J., & Keltner, D. (1999). Culture and facial expression: Open-ended
methods find more expressions and a gradient of recognition. Cognition
& Emotion, 13, 225266.
Huang, Y., Tang, S., Helmeste, D., Shioiri, T., & Someya, T. (2001).
Differential judgment of static facial expressions of emotions in three
cultures. Psychiatry and Clinical Neurosciences, 55, 479483.
Izard, C. (1971). The face of emotion. New York, NY: Appleton-CenturyCrofts.
Jack, R. E., Blais, C., Scheepers, C., Schyns, P. G., & Caldara, R. (2009).
Cultural confusions show that facial expressions are not universal.
Current Biology, 19, 15431548.
Kayyal, M., & Russell, J. A. (2012). Americans and Palestinians judge spontaneous facial expressions of emotion. Manuscript submitted for publication.
Mandal, M. K., Bryden, M. P., & Bulman-Fleming, B. (1996). Similarities
and variations in facial expressions of emotions: Cross-cultural evidence.
International Journal of Psychology, 31, 4158.
Matsumoto, D., Angus-Wong, A. M., & Martinez, E. (2008). Priming
effects of language on emotion judgments in SpanishEnglish bilinguals. Journal of Cross-Cultural Psychology, 39, 335342.
Matsumoto, D., & Assar, M. (1992). The effects of language on judgments of universal facial expressions of emotion. Journal of Nonverbal
Behavior, 16, 8599.
Matsumoto, D., Consolacion, T., Yamada, H., Suzuki, R., Franklin, B.,
Paul, S., & Uchida, H. (2002). AmericanJapanese cultural differences in judgments of emotional expressions of different intensities.
Cognition & Emotion, 16, 721747.
Matsumoto, D., Olide, A., Schug, J., Willingham, B., & Callan, M. (2009).
Cross-cultural judgments of spontaneous facial expressions of emotion.
Journal of Nonverbal Behavior, 33, 213238.
Naab, P. J., & Russell, J. A. (2007). Judgments of emotion from spontaneous facial expressions of New Guineans. Emotion, 7, 736744.
Rosenberg, E., & Ekman, P. (1995). Conceptual and methodological issues
in the judgment of facial expressions of emotion. Motivation & Emotion,
19, 111138.
Russell, J. A. (1991). The contempt expression and the relativity thesis.
Motivation and Emotion, 15, 149168.
Russell, J. A. (1993). Forced-choice response format in the study of facial
expression. Motivation and Emotion, 17, 4151.
Russell, J. A. (1994). Is there universal recognition of emotion from facial
expression? A review of the cross-cultural studies. Psychological
Bulletin, 115, 102141.
Scherer, K. R., & Grandjean, D. (2008). Facial expressions allow inference of both emotions and their components. Cognition & Emotion,
22, 789801.
Shariff, A. F., & Tracy, J. L. (2011). Emotion expressions: On signals, symbols, and spandrels A response to Barrett (2011). Psychological Science, 20, 407408.
Shioiri, T., Someya, T., Helmeste, D., & Tang, S. W. (1999). Misinterpretation of facial expression: A cross-cultural study. Psychiatry and
Clinical Neurosciences, 53, 4550.
Tcherkassof, A., & de Suremain, F. (2005). Burkina Faso and France:
A cross-cultural study of the judgment of action readiness in facial
expressions of emotion. Psychologia, 48, 317334.
Tracy, J. L., & Robins, R. W. (2008). The nonverbal expression of pride:
Evidence for cross-cultural recognition. Journal of Personality and
Social Psychology, 94, 516530.
Wickline, V. B., Bailey, W., & Nowicki, S. (2009). Cultural in-group
advantage: Emotion recognition in African American and European

Downloaded from emr.sagepub.com by ancuta anca on October 25, 2014

14 Emotion Review Vol. 5 No. 1

American faces and voices. Journal of Genetic Psychology, 170,

529.
Widen, S. C. (2013). Childrens interpretation of facial expressions: The
long path from valence-based to specific discrete categories. Emotion
Review, 5, 7277.

Yik, M., & Russell, J. A. (1999). Interpretation of faces: A cross-cultural

study of a prediction from Fridlunds theory. Cognition & Emotion,
13, 93104.
Yik, M., Widen, S. C., & Russell, J. A. (2012). The within-subject design in
the study of facial expressions. Manuscript submitted for publication.

Appendix
Matching scores from cross-cultural judgment studies, 1992 to 2010
Society/Culture

Canadiana
Canadianb
Canadianc
Frenchd
Hungariane
Indianf
Indiang
Indianh
Indianh
Mexicani
Mexicani
Polishe
Americanj
Americanf
Americang
Americank
Americanl
Americanl
Europeanm
Europeanl

Burkinabed
Burkinabed
Chinesen
Chinesea
Chineseo
Chinesej
Gaboneseb
Japanesef
Japanesek
Japanesep
Japanesec
Hong Kongc
Sumatrane
Tibetann

No. of
expression sets

20
20
30
96
45
50
40
100
100
274
274
75
37
50
40
138
72
102
13
30

12
8
1
4
8
4
1
8
8
8
8
8
16
4
1
4
11
11
8
11

48
48
32
20
237
38
20
50
137
123
30
30
32
11

4
4
4
12
7
16
8
4
4
8
1
1
8
4

Language

Facial expression
Happiness

Sadness

Anger

Literate, Western cultures (20 samples)

67.0
56.0
50.0
88.0
71.0
72.0
100.0
86.7
73.3
97.0
83.0
88.0
98.0
83.0
87.0
99.0
39.5
65.0
45.0
42.5
80.0
99.0
94.0
90.0
99.0
90.0
89.0
90.0
68.0
58.0
78.0
62.0
58.0
98.0
88.0
85.0
80.0
89.0
93.5
54.0
79.0
73.0
88.0
83.0
93.3
65.7
52.3
93.0
77.0
65.0
89.0
82.0
80.0
99.0
89.0
89.0
84.0
85.0
71.0
Literate, non-Western cultures (18 samples)
French
92.0
75.0
60.0
Moore
96.0
52.0
54.0
Mandarin
97.0
72.0
57.0
French
71.0
38.0
45.0
Chinese
92.0
77.0
66.0
English
81.0
75.3
French
80.0
41.0
63.0
Japanese
98.5
68.5
51.5
English
89.4
47.8
54.7
English
94.8
66.6
56.3
Japanese
100.0
80.0
56.7
Cantonese
100.0
76.7
46.7
?
99.0
81.0
79.0
Mandarin
86.0
75.0
61.0
French
French
English
French
Hungarian
English
Oriya
English
Hindi
English
Spanish
Polish
English
English
English
English
English
English
English
English

Fear

Surprise

Disgust

Mean

33.0
68.0
66.7
85.0
74.0
75.0
55.0
81.0
74.0
65.0
56.0
69.0
63.8
77.0
55.0
74.0
78.0
88.0
70.8

76.7
94.0
78.0
42.5
90.0
90.0
87.0
77.0
89.0
89.9
82.0
80.0
78.7
84.0
-

56.0
62.0
56.7
84.0
53.5
82.5
57.0
41.0
83.0
34.5
78.0
88.0
-

52.4
72.2
76.7
88.3
86.7
68.3
57.9
90.8
88.4
70.8
62.0
85.3
80.7
70.0
76.2
72.5
77.3
82.3
89.5
77.7

65.0
67.0
21.0
28.0
56.0
61.2
50.0
49.5
44.5
53.3
23.3
57.0
16.0

74.0
76.0
90.0
92.5
88.6
89.2
93.3
83.3
89.0
73.0

54.0
49.0
50.0
32.0
20.0
63.7
30.0
56.7
76.0
66.0

73.0
67.3
63.0
46.2
70.0
77.0
53.0
63.0
70.0
69.0
69.0
64.0
80.0
63.0

(Continued)

Downloaded from emr.sagepub.com by ancuta anca on October 25, 2014

Nelson & Russell Universality Revisited 15

Appendix (Continued)
Society/Culture

Vietnamesee
Africanl
Sub-Saharan
Africana
East Asianm

Burkinabeq
Overall mean

No. of
expression sets

Language

34
30
20

8
11
12

?
English
French

13

English

39

Dioula

Facial expression
Happiness

Sadness

Anger

Fear

Surprise

Disgust

Mean

99.0
79.0
76.0

80.0
70.0
29.0

81.0
50.0
49.0

67.0
76.0
27.0

92.0
-

58.0
46.0

80.0
69.0
45.4

99.0
90.0
80.0
Illiterate cultures (1 sample)
84.0
51.0
33.0

64.0

91.0

69.0

82.0

30.0

58.0

44.0

50.0

58.5

82.5

57.3

70.9

89.6

70.2

67.3

Note: Matching score is the percentage of observers who selected the predicted label.
aBeaupr and Hess (2005); bElfenbein et al. (2007); cYik and Russell (1999); dTcherkasoff and de Suremain (2005); eBiehl et al. (1997); fElfenbein et al. (2004); gHaidt and
Keltner (1999); hMatsumoto and Assar (1992); iMatsumoto et al. (2008); jElfenbein (2006); kMatsumoto et al. (2002); lWickline et al. (2009); mJack et al. (2009); nElfenbein
and Ambady (2003); oHuang, Tang, Helmeste, Shioiri, and Someya (2001); pShioiri, Someya, Helmeste, and Tang (1999); qTracy and Robins (2008).

Downloaded from emr.sagepub.com by ancuta anca on October 25, 2014