Beruflich Dokumente
Kultur Dokumente
Jonathan Nickles
Northeastern University
FACIAL EMOTION RECOGNITION IN AUTISM: DETAILS, DIS !2
Autism spectrum disorder (ASD) is an exceedingly prevalent disease affecting over 3.5 million
people in the United States alone (Autism society, 2015). Because of this staggering number, nearly every
American knows, or has interacted with, someone who has been diagnosed. Understanding more about
ASD is marked by abnormal social interaction, communication skills and repeated behaviors
(Grossman, Klin, Carter, & Volkmar, 2000). One of the primary results of ASD is the severe reduction in
drive to interact with those around them (Baron-Cohen, 1995). This reduction may be produced by an
impairment in the ability to perceive other’s emotional expressions, specifically emotion of the face
(Hobson 1993). Some emotion theories suggest that there are distinct facial expressions that map onto
basic emotions like happiness, sadness, fear, anger and disgust. Because nonverbal communication (e.g,
reading facial features) facilitates and maintains social interaction, a deficit in these skills can produce a
Facial emotion recognition (FER) has been a very popular area of study in the last three decades.
Despite the amount of research on this topic, the results are convoluted. Generally, research points to-
wards varying levels of impairment in FER for individuals with ASD depending on the specific task (e.g.
Baron-Cohen, 1991; Langdell, 1978; Teunisse & de Gelder, 1994). However, a number of studies have
failed to find any impairment of FER in those with ASD (e.g. Ogai et al., 2003; Ozonoff, Pennington, &
Rogers, 1990; Teunisse and Gelder, 1994). Despite these mixed results, the data that finds varying levels
All emotions are not equal when it comes to recognizing them through facial features. Emotions
can be broken down into internal and external emotions (Grossman, Klin, Carter, & Volkmar, 2000). In-
ternal emotions are those related to personal cognition which require mental schema to grasp. Internal
emotions include surprise and embarrassment. External emotions like happiness and sadness are seen in
more superficial situations that only require an understanding of contextual information. Simply put, the
FACIAL EMOTION RECOGNITION IN AUTISM: DETAILS, DIS !3
difference between the two types of emotions are their underlying cause, external being the situation at
hand and internal being a belief about the world. Autistic subjects in studies of emotion cognition show a
marked inability to understand internal emotions (Baron-Cohen, 1991) which leads to impairment in fa-
In recent studies researchers have looked at the ability of autistic subjects to recognize a number
of basic emotions. Their findings showed that there was notable impairment of all emotions, but an even
more severe deficit for negative emotions like fear, anger and disgust (Ashwin, Chapman, Colle, &
Baron-Cohen, 2007). The inability to recognize negative emotions is supported by the amygdala theory,
one of many theories that describes the cause, or partial cause of ASD. The theory links amygdala dys-
function to the impairment in recognition of negative emotions. Support for this theory comes from stud-
ies that that show an aversion to pursuing and maintaining direct eye contact (Grice et al., 2005), an abili-
ty of the amygdala. Furthermore, fMRI scans have shown decreased size as well as decreased activation
of the amygdala during different task demands (Baron-Cohen et al., 1999; Pierce, Müller, Ambrose,
Allen, & Courchesne, 2001). On a cellular level, autopsies performed on those with ASD have found
cell abnormalities within the amygdala (Bauman & Kemper, 1994). Lastly, in a study where the amygdala
of monkeys were lesioned, significant behavioral abnormalities mimicking the expressions of ASD were
found (Bachevalier, 1994). These studies support the amygdala theory, a likely cause as to why subjects
with ASD recognize negative emotions with significantly lower accuracy than positive emotions.
In authentic human interactions, emotions are not displayed to their fullest degree every single
time. Also, the majority of facial expressions that are displayed are not purely one emotion. For example
happiness and surprise can be seen when a deployed soldier returns home to his/her family. Anger and
disgust are often expressed in combination with one another when a partner is found to be unfaithful.
Consequently, researchers in 2010 combined differing intensities of emotion ranging from 20% to 100%
instead of only 100% like previous research. The task presented adolescents with ASD with a video of a
neutral expression morphing into an emotion with variable intensities. The researchers again found that
negative emotions were less accurately recognized than positive but more specifically, that anger, disgust,
and surprise were debilitated at different intensities. Specifically, surprise was only impaired at low levels
FACIAL EMOTION RECOGNITION IN AUTISM: DETAILS, DIS !4
of intensity (20-40%), anger at low and medium (20-70%) and disgust at low medium and high
(20-100%) (6). The increased difficulty with disgust has been previously mentioned in studies (Ashwin,
Chapman, Colle, & Baron-Cohen, 2007) and may be a cause of insufficient eye gaze. As noted earlier,
individuals with ASD generally tend to avert their sight from others’ eyes. This can be especially prob-
lematic in emotion recognition because many of the clues the brain uses to identify emotion comes from
the eye region (Kussiko et al., 2009). Recognizing disgust, as opposed to the other basic emotions like
happiness, relies most heavily upon the eyes which is why there is impairment at all intensities.
In the same study, researchers combined the stimuli they presented by using a blended emotion
scale (BES). The results showed that ASD subjects interpreted ambiguous stimuli (an emotion and a neu-
tral expression blended) as negative as well as choosing negative emotions incorrectly (e.g. anger instead
of disgust) (Kuusikko et al., 2009). This may raise the question of why those on the spectrum are more
prone to attenuate towards negative emotions rather than positive, a bias which is also seen in other so-
Happiness, on the other side of the spectrum, was found to be the emotion that was the easiest to
recognize. Control groups correctly picked it almost one hundred percent of the time and the ASD group
correctly chose it at above chance levels (Feldman, McGee, Mann, & Strain, 1993). This was the only
basic emotion in the test group to be chosen at above chance levels which is attributed to the fact that this
emotion is the first to be expressed, discriminated and recognized during development (Wellman, Harris,
Banerjee, & Sinclair. 2008). In a meta analysis of FER in ASD, the 95% confidence intervals for happi-
ness all spanned zero showing almost no difficulty across the studies. The only impairments that autistic
individuals faced were subtle differences in reaction time (Uljarevic & Hamilton, 2013) Nearly normal
levels of expression and recognition in happiness lends credit to the amygdala theory which affects only
negative emotions.
An explanation for the rate of recognition in specific emotions may be found in diagnosed indi-
viduals’ abnormal eye gaze. Not only do people with ASD avert their attention away from the eyes, but
they also attenuate to particular regions of the face instead of the face as a whole (Gross, 2004). Specifi-
cally, they are inclined to fixate their attention on the lower portion of the face more than the eyes and the
upper regions. While negative emotions rely upon the eyes and the detail of the eyebrows and forehead
FACIAL EMOTION RECOGNITION IN AUTISM: DETAILS, DIS !5
for discrimination, positive emotions rely most heavily on the lower half of the face like the mouth and
cheeks. These gaze patterns would support the difficulty in recognition of negative emotion and apparent
To test these findings researchers presented stimuli in the form of partial face regions. A study
conducted in 1978, the very first study of FER in autism, took 10 autistic individuals and ten mentally
retarded non autistic individuals and looked at their ability to sort happy and sad faces. In the study, sub-
jects could discriminate both complete faces and just the lower half at similarly proficient levels. Howev-
er, when just the upper portion of the face was presented, there was a sharp decrease in recognition of the
test group (Langdell, 1978). These findings support the theory that autistic individuals avoid the upper
As researchers began to get a grasp of facial recognition and the basic emotions, they realized that
people with autism develop deep emotional schema that is specific to the human face (Gross, 2004). More
importantly was the fact that as the stimuli varied from the prototypical human face, the ability to recog-
nize emotion severely decreased. In fact, there was even a significant decrease between a random human
face and that of a family member, as well as peers versus a person outside of the subject’s age group
(Humphreys, Minshew, Leonard, & Behrmann, 2006; Langdell 1978). It must be noted that in these tasks
where the stimuli were manipulated, control groups also had a reduction in recognition ability but were
R.P. Hobson, a researcher well known for his work with autistic groups, conducted a number of
studies involving the recognition of emotion. One such study (1986), had test subjects sort a group of
photographs according to the emotion in the picture. Many of the faces in the picture had extraneous
stimuli in the form of different hats. Some hats were smaller and less intrusive while others were large
and overbearing. Not surprisingly, the participants paid little attention to the emotions on the faces and
began to categorize the photographs by the type of hat being worn (Hobson, 1986).
In another study conducted by Hobson and colleagues, the presentation of facial features was dis-
torted even further. Instead of the typical upright orientation, faces were presented upside down. Interest-
ingly enough, the test group was superior at identifying emotions on the upside down face compared to
FACIAL EMOTION RECOGNITION IN AUTISM: DETAILS, DIS !6
the controls (Hobson, 1986). These findings may occur due to a permanence in the strategies used by con-
trol groups. During development, strategies for recognizing identity and emotion are refined; however,
when the face is rotated, the strategies no longer work. Autistic individuals lack complex strategies like
mental rotation and simply evaluate the face as-is. The reason why ASD subjects outperform the control
is because they have no regard for the meaning attached to the facial features. By sorting the meaningless
parts of the face they are just matching abstract patterns (Langdell, 1978). Although the same strategies
are used in both the upright and inverted tasks, the inverted faces are recognized at a lower rate because
they deviate from the prototypical face that autistic people score the highest on.
In the same study both groups were then presented with complete upright faces. As the study
continued, specific features of the face were blanked out. Starting with a blanked out mouth, test groups
maintained a high level of accuracy in recognition (this finding goes against Langell’s 1978 study that the
mouth is the critical feature for FER), but as subsequent features were removed the performance showed
further decrease (Hobson, 1986). The study ultimately reveals interesting advances on the strategies used
by those with ASD in FER. Specifically, the hierarchy of importance in features that are visibly available.
The final study that deviates from the typical human expression is one that looks at the difference
in recognition between human stimuli and nonhuman stimuli. In the study, 28 children with autism were
trained on a recognition task that presented the subjects with a picture of an emotional facial expression
(Gross, 2004). Subjects then had to match the expression with one of four options given (happy, sad, an-
gry, surprised. All also in photograph form). The stimuli in the study was either a human face, an orang-
utan face or a canine face. All the animal expressions were consistent with the description of primates and
canines as described in previous literature (e.g., Chevalier-Skolnikoff, 1973; Hall & Sharp, 1978). Find-
ings for this study paralleled many of the patterns already noted: children in the autistic group compared
to other control groups showed broad recognition deficiencies. Errors were higher when matching orang-
utan and canine faces especially from incorrectly choosing negative emotions when the correct answer
was either positive or neutral. Happiness was again the only emotion in which recognition rates were
above chance, this finding held true for canine faces. All other emotions, excluding sadness, were per-
ceived at a higher rate in human stimuli than animal; sadness was recognized at similarly low rates for
each species. The findings once again show that as the stimuli move further away from the typical human
FACIAL EMOTION RECOGNITION IN AUTISM: DETAILS, DIS !7
expression, recognition of that emotion becomes worse. Although this holds true throughout many of the
studies, it is important to remember that there is a large amount of variability between each emotion.
Despite over 100 published studies on emotion recognition in autism, the results remain mixed.
The findings of these studies vary for a number of reasons and through a recent meta analysis researchers
First, participant sizes range from an N of 5 all the way to 97 (Uljarevic & Hamilton, 2013).
However only 15 of the 48 studies analyzed had groups with 20 participants or more. By conducting a
straightforward power analysis with a power of .95 and two independent groups (effect size =.08), the
studies require at least 35 participants in each group. Interestingly, the two studies with the largest sample
sizes both found no group differences. A trend in recent literature is toward larger sample sizes. Future
A further problem is controlling for the actual participants and the tasks they do. The term ASD is
extremely broad and can be categorized into a number of different subgroups including: classical autism
(CA), high functioning (HF) autism and Asperger’s syndrome. Each of these categories has a different
level of mental functioning as well as FER ability. Many studies fail to categorize their subjects which can
lead to problems when comparing the data. A group of HF may show absolutely no deficit in FER while
on the same task someone who is a CA may show severe problems. The inverse dilemma may also occur
where the stimuli presented are either far too exaggerated (the ceiling effect) (Law Smith, Montagne, Per-
rett, Gill, & Gallagher, 2010), or far too subtle for any relevant effects to be shown (Humphreys, Min-
shew, Leonard, & Behrmann, 2006). One way a few studies have alleviated this problem was by control-
ling for IQ or verbal ability. By controlling for these details, the variability within the group decreases as
well as the mental ability between the test group and the control group. Future studies would benefit from
making note of the type of autistic participant they recruited as well as the range of IQ or verbal ability
Additionally, a wide variety of tasks have been implemented over the three decades this subject
has been studied. The two most popular are labelling and matching. In labelling participants view emo-
FACIAL EMOTION RECOGNITION IN AUTISM: DETAILS, DIS !8
tional imagery and then either choose the correct answer from a pool of options or come up with their
own response. Matching asks the participant to choose the same emotion from a pool of options as the
emotion presented (Uljarevic & Hamilton, 2013). While the meta analysis did not find any significant
performance differences between the two tasks, there are a number of important differences that may play
smaller roles. Matching tasks may be able to be completed with some accuracy using contextual clues and
other compensatory strategies for recognition. Labeling tasks on the other hand present a number of op-
tions to choose from. Through simple elimination the correct answer may be guessed easily. Depending
on the individual participant, FER may vary between the two tasks.
In order for future studies to provide valid insight into this topic they must increase their sample
size and control for the participants they recruit (Uljarevic & Hamilton, 2013). If possible, it would be
beneficial for studies to include both matching and labelling tasks in order to account for individual dif-
ferences. Furthermore, a number of studies only used stimuli from a single person’s face. In order to de-
crease effects caused by familiarity and proximity to the prototype, it would be advisable to diversify and
regularize the stimuli used. Since the majority of the naturally occurring facial stimuli that is experienced
is moving, using dynamic stimuli in future studies may produce more applicable results. With dynamic
stimuli researchers would not only be able to look at the rate of recognition but the speed as well.
Studies of FER in the autistic community have resulted in wide range of results. The discrepan-
cies in these findings can be attributed to a variability in the studies (number of participants, type of task,
difficulty of the stimuli) as well as a variability in the disorder itself. In order to increase internal reliabili-
ty, researchers must increase their sample size and specify which type of autism they are studying.
Methodologically, studies should control for IQ and verbal ability, present a varying degree of difficulty
in the stimuli and include multiple tasks. Adherence to these suggestions in future studies will results in a
References
Ashwin, C., Chapman, E., Colle, L., Baron-Cohen, S. (2007). Impaired recognition of negative
basic emotions in autism: A test of the amygdala theory. Social Neuroscience, 1(3-4),
349-363
FACIAL EMOTION RECOGNITION IN AUTISM: DETAILS, DIS ! 10
Autism Society. (August, 2015). Facts and statistics. Retrieved from autism-society.org
Bachevalier, J. (1994). Medial temporal lobe structures in autism: A review of clinical and
Bal, E., Harden, E., Lamb, D., Vaughan Van Hecke, A., Denver, J.W., Porges, S. W. (2010).
Emotion recognition in children with autism spectrum disorders: Relations to eye gaze
and autonomic state. Journal of Autism and Developmental Disorders, 40(3), 358-370
Baron-Cohen, Simon. (1991). Do people with autism understand what causes emotion? Child
Baron-Cohen, S. (1995). Are children with autism blind to mentalistic significance of the eyes?
Baron-Cohen, S., Ring, H. A., Wheelwright, S., Bullmore, E. T., Brammer, M. J., Simmons, A.,
Williams, S. C. R. (1999). Social intelligence in the normal and autistic brain: An fMRI
Bauman, M., Kemper, T. (1994). The neurobiology of autism, Baltimore, Maryland: Johns
Bormann-Kischkel, C., Vilsmeier, M., Baude, B. (1995). The development of emotional con-
Feldman, R. S. (1982). Development of nonverbal behavior in children. New York, New York:
Springer-Verlag Publishing
Feldman, R. S., McGee, G. G., Mann, L., Strain, P. (1993). Nonverbal affective decoding ability
in children with autism and in typical preschoolers. Journal of Early Intervention, 17(4),
341-350
FACIAL EMOTION RECOGNITION IN AUTISM: DETAILS, DIS !11
Grice, S. J., Halit, H., Farroni, T., Baron-Cohen, S., Bolton, P., Johnson, M. H. (2005). Neural
correlates of eye-gaze detection in young children with autism. Cortex, 41(3), 342-353
Gross, T. F. (2004). The perception of four basic emotions in human and nonhuman faces by
children with autism and other developmental disabilities. Journal of Abnormal Child
Grossman, J. B., Klin, A., Carter, A.S., Volkmar, F. R. (2000). Verbal bias in recognition of
facial emotions in children with asperger syndrome. Journal of Child Psychology and
Hobson, P. R., Baron-Cohen, S., Flusberg, H. T., Cohen, D. J. (1993). Understanding persons:
The role of affect in understanding other minds: Perspectives from autism. Journal of
Humphreys, K., Minshew, N., Leonard, G. L., Behrmann, M. (2007). A fine-grained analysis of
45(4), 685-695
Kuusikko, S., Haapsamo, H., Verkasalo-Jansson, E., Hurtig, T., Mattila, M. L., Ebeling, H.,
Law Smith, M. J., Montagne, B., Perrett, D. I., Gill, M., Gallagher, L. (2010). Detecting subtle
facial emotion recognition deficits in high functioning autism using dynamic stimuli of
Langdell, T. (1978). Recognition of faces: An approach to the study of autism. The Journal of
Ogai, M., Matsumoto, H., Suzuki, K., Ozawa, F., Fukuda, R., Uchiyama, I...Takei, N. (2003).
Ozonoff, S., Pennington, B. F., Rogers, S.J. (1990). Are there emotion perception deficits in
young autistic children? Journal of Child Psychology and Psychiatry, 31(5), 343-361
Pierce, K., Müller, R.A., Ambrose, J., Allen, G., Courchesne, E. (2001). Face processing occurs
outside the fusiform ‘face area’ in autism: Evidence from functional MRI. Brain: A
Teunisse, J.P., de Gelder, B. (1994). Do autistic children have a generalized face processing
Wellman, H. M., Harris, P. L., Banerjee, M., Sinclair, A. (2008). Early understanding of emotion: