ClassicThesis PDF

fiammetta ghedini T H E I L L U S I O N O F A M B I G U I T Y: F R O M B I S TA B L E P E R C E P T I O N T O ANTHROPOMORPHISM
T H E I L L U S I O N O F A M B I G U I T Y: F R O M B I S TA B L E PERCEPTION TO ANTHROPOMORPHISM fiammetta ghedini
A dissertation submitted in partial fullment of the requirements for the degree of Doctor in Philosophy in Innovative Technologies of Information and Communication Engineering and Robotics 30 May 2011
Fiammetta Ghedini: The Illusion of Ambiguity: from Bistable Perception to Anthropomorphism, A dissertation submitted in partial fullment of the requirements for the degree of Doctor in Philosophy in Innovative Technologies of Information and Communication Engineering and Robotics, 30 May 2011
No lesson of psychology is perhaps more important for the historian to absorb than this multiplicity of layers, the peaceful coexistence in man of incompatible attitudes. Sir Ernest Gombrich There is an universal tendency among mankind to conceive all beings like themselves, and to transfer to every object, those qualities, with which they are familiarly acquainted, and of which they are intimately conscious. David Hume
ABSTRACT
This thesis is a multidisciplinary work at the merging of neuroscience, art and human interaction with technologies. Its objectives are: 1. Review the literature and further investigate, by means of a brain imaging study, which are the mechanisms allowing the illusion of ambiguity in the brain, by proposing a framework of analysis based on different levels of ambiguity; 2. Discuss the concept of the illusion of life, dened as a perceptual phenomenon included into the wider category of ambiguity and caused by intentionality and animacy being hard-wired in the brain (part of that previous knowledge necessary to successfully process sensory inputs); 3. Explore features in behaviour and form which are most likely to trigger anthropomorphism, drawing insights from art, technological applications and cognitive sciences, and focusing on human interaction with articial creatures.
vii
P U B L I C AT I O N S
Some ideas and gures have appeared previously in the following publications: Perception of Ambiguous Figures: an fMRI study, F. Ghedini & M.Bergamasco, VRR-IJCAI 2011, 18-24 July, Barcelona, Spain [Ghedini and Bergamasco, 2011b] Life evocation in art: from representation to behaviour. F. Ghedini & M.Bergamasco, ISEA 2011, 12-16 September, Instanbul, Turkey [Ghedini and Bergamasco, 2011a] Robotic Art: Perceiving and inventing reality. F. Ghedini & M.Bergamasco, Art and Science: exploring the limits of human perception, Conference Proceedings, Benasque, Spain, on July 12-16, 2009 [Ghedini and Bergamasco, 2009] "Passages: An Artistic 3D Interface for Childrens Rehabilitation and Special Needs" F. Ghedini, H. Faste, M. Carrozzino, M. Bergamasco ICDVRAT International Conference Series on Disability, Virtual Reality, and Associated Technologies, Portugal, 09-08-2008 [Ghedini et al., 2008] Robotic expression. Developing an applied framework for the integration of artistic approaches and technological competences, 2008, PhD Application for the Scuola Superiore SantAnna [Ghedini, 2008] A conversation with Bill Vorn (on robotic art creatures and other believable living machines) Scuola Superiore SantAnna, Pisa, Italy, 09-17-2007[Vorn, 2007] "Enactive Network of Excellence, Digest 2006: Multimodal Interfaces," Massimo Bergamasco, Fiammetta Ghedini, Haakon Faste, Editors Enactive Consortium Press, project IST-2004-002114ENACTIVE, 02-15-2007 Haptic interaction with virtual sculptures, 2007 Massimo Bergamasco, Marcello Carrozzino, Fiammetta Ghedini "Enaction and Inactive Interfaces: A Handbook of Terms," Enactive Systems Books, 2007 "Le interfacce aptiche per i Beni Culturali" M. Bergamasco, C.A. Avizzano, F. Ghedini, M. Carrozzino Proceedings of LUBEC 2007, "Valorizzazione dei Beni Culturali e Innovazione", Lucca, Italy, 2008 ISBN 978-88-89766-10-1 [Bergamasco et al., 2007]
ix
ACKNOWLEDGMENTS
This dissertation would not have been possible without the support and encouragement of Professor Massimo Bergamasco, my Ph.D. supervisor at Perceptual Robotics Laboratory of Scuola Superiore SantAnna. Throughout my thesis he provided guidance, advices and inspiration. I would like to thanks all the colleagues with whom I had the pleasure of collaborating: Carlo Alberto Avizzano, Franco Tecchia, Antonio Frisoli, Chiara Evangelista, Elisabetta Sani, Federico Vanni, Walter Aprile, Andrea Bizdideanu, Vittorio Spina, Davide Vercelli, Rosario Leonardi, Paolo Tripicchio, Paolo Gasparello, Emanuele Giorgi; Vittorio Lippi and Emanuele Ruffaldi for their precious help with LateX and those other colleagues whom I have neglected to mention. A special thanks to Haakon Faste for his inspiring enthusiasm and eclectic skills, Marcello Carrozzino for providing useful feedback and to Francesca Farinelli and Alessandra Scucces for being not only helpful colleagues but also very dear friends, and for supporting me during my thesis-writing period. My gratefulness goes also to all the people working in the administration of the Scuola SantAnna and especially Laura Bevacqua. For enlightening conversations, encouragement, feedback and inspiration I would like to thank Israel Roseneld. I owe a lot to him and to his perspectives on neuroscience. I also would like to thank Simon Penny and Bill Vorn for visiting our Laboratory and inspiring me with their artworks on articial life. My deepest gratefulness goes to Professor Semir Zeki who made possible the fMRI experiment described in this thesis. Thank you for your guidance and for working with me for over one year at the Wellcome Lab of Neurobiology at UCL, London. I am indebted to all the people working in Professor Zekis lab: Barbara Nordhjem for her continuous support, John Romaya for his experience, Jon Stutters for his skills, Shelley Tootell for her helpfulness and Tomohiro Ishizu and Sam Cheadle for being always supportive, through difcult times too. Thanks for you friendship and kindness. Thanks to all the people from the academic world who in conferences, seminars and workshops provided new inspiring ideas, feedback on my research and good company: Mel Slater, Mavi Sanchez, Doron Friedman, Peggy Weil, Nonny de la Pena, Marcelo Wanderlay, Elena Pasquinelli, Benoit Bardy and many
xi
others. A very special thanks goes to Tommaso Andreussi: you will print your next book on Luoyang paper! Finally, "merci" to Franois Pachet for his encouragement, for being a model of what a researcher should be, and most importantly for coming into my life. And "grazie" to my parents, Anna Casanova and Fabio Ghedini, because they taught me the importance of knowledge: to them I dedicate my thesis.
xii
CONTENTS
i introduction 1 introduction 1.1 Perception: Making Sense of the Senses . . . 1.1.1 Watching without seeing . . . . . . . . 1.1.2 The Brain: an Abstraction Machine . . 1.1.3 The Case of Colour Vision . . . . . . . 1.2 Concepts and Categories in the Brain . . . . 1.3 Seeing through Illusions . . . . . . . . . . . . 1.3.1 Two Different Categories of Illusions . 1.3.2 Ambiguity and Ambiguities . . . . . .
11 13 . . . 14 . . . 15 . . . 17 . . . 19 . . . 22 . . . 24 . . . 24 . . . 27 31 33 33 36 41 42 42 46 48 48 50 53
ii first and second level ambiguity 2 perception of bistable ambiguity 2.1 What is multistable ambiguity . . . . . . . . . . . . 2.1.1 Neural processes underlying multistable phenomena . . . . . . . . . . . . . . . . . . 2.1.2 Attention and perception of bistable gures 2.1.3 Levels of ambiguity . . . . . . . . . . . . . . 2.2 An Experiment on Two-Levels Bistable Ambiguity 2.2.1 General fMRI Analysis . . . . . . . . . . . . 2.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 Behavioural results . . . . . . . . . . . . . . 2.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . 2.4.1 Conclusion . . . . . . . . . . . . . . . . . . . iii third level ambiguity 3 third level ambiguity : the illusion of life 3.1 Perceptual knowledge of life . . . . . . . . . . . . 3.2 The illusion of intentionality . . . . . . . . . . . . 3.3 The illusion of life in the brain . . . . . . . . . . 3.3.1 Emotional clues in the illusion of life . .
61 63 . 63 . 66 . 69 . 72 75 77 77 77 78 78
iv forth level ambiguity 4 aesthetics of anthropomorphism 4.0.2 Anthropomorphism as a forth level ambiguity . . . . . . . . . . . . . . . . . . . . . . 4.0.3 Variability in anthropomorphisation . . . . 4.1 Life evocation in the arts . . . . . . . . . . . . . . . 4.1.1 Dening life away . . . . . . . . . . . . . . .
xiii
xiv
contents
4 .2
4 .3
4.1.2 Art and the illusion of life . . . . . . . . . . 4.1.3 From art to technology . . . . . . . . . . . . 4.1.4 Uncanniness as a result of life evocation . . The illusion of life in articial creatures: features of believability . . . . . . . . . . . . . . . . . . . . . 4.2.1 Form: is realism a necessity? . . . . . . . . Conclusions . . . . . . . . . . . . . . . . . . . . . .
79 83 84 85 86 90 95
bibliography
LIST OF FIGURES
Figure 1.1 Figure 1.2 Figure 1.3
Figure 1.4 Figure 1.5 Figure 2.1 Figure 2.2 Figure 2.3 Figure 2.4 Figure 2.5 Figure 2.6 Figure 2.7
Figure 2.8
Figure 2.9
Figure 2.10
The response of an orientation selective cell The response of an orientation selective cell in V5 . . . . . . . . . . . . . . . . . . . . A rotation of a mask: the forth image shows the inside of the mask, appearing convex even if it is hollow . . . . . . . . . . . . . . Model of perception, based on (Gregory 2009) . . . . . . . . . . . . . . . . . . . . . . Different kinds of ambiguities . . . . . . . . a) Necker Cube - b) Rubin Vase . . . . . . . Binocular rivalry . . . . . . . . . . . . . . . Auditory streaming is a case of multistable perception in the auditory modality . . . . Experiment stimuli: bistable intra-categorical images + stabilized versions . . . . . . . . . Experiment stimuli: bistable inter-categorical images + stabilized versions . . . . . . . . . Bistable Figures > Stable Figures . . . . . . Global 3D view of activations for the contrast internal change > external change for a random effects analysis with 16 subjects: selected activations superimposed on to averaged anatomical sections . . . . . . . . T statistic for Bistable > Stable in conjunction with Intra-categorical: Bistable > Stable switches (left) and Inter-categorical: Bistable > Stable switches (right) . . . . . . Bistable activations conjoined with Intercategorical and intra-categorical bistable activations . . . . . . . . . . . . . . . . . . . Bistable faces > All. FFA activation projected onto averaged structural scans (left) and main HRF and TD plotted for FFA [38 -58 -14] (right) . . . . . . . . . . . . . . . .
18 19
25 27 29 34 35 36 54 55 56
57
57
58
59
xv
xvi
List of Figures
Figure 4.1
The Uncanny Valley graph following Mori
85
Part I INTRODUCTION
INTRODUCTION
1
From bistable perception to anthropomorphism
This thesis is a multidisciplinary work, whose goal is to merge insights and research from elds as different as art, neuroscience and technology. My original question was: how do we attribute features of animacy (such as emotions and intentions) to objects, in spite of the certain knowledge of their non-animacy? This question brought me to consider, in the rst place, some basic phenomena of perception, and in particular, illusions. Illusions make evident that perception is not a passive mapping of inputs coming from the environment, but an active interpretation, involving a mechanism of inference. As illustrated in Chapter I, studies on optical illusions may be very useful in order to reveal the active role of the brain in the organisation of perceptual processes. Among all different kinds of illusion, ambiguity is one of the most interesting, since it claries how our brain disambiguates the information received. Ambiguity is traditionally dened as an alternation in time of two mutually exclusive interpretations of the same stimulus [Zeki, 2004]. As I propose in the framework of this thesis, this denition ts to a specic kind of ambiguity, namely the "intra-categorical" one, taking place when the two possible interpretations of the same stimulus belong to the same semantic category, for example the two recessional planes of the Necker cube. This kind of ambiguity is the basic model of a mechanism that probably happens endlessly in our brain, since our everyday environment contains ambiguities and conicts which we do not usually notice because our brain successfully - and continuously - disambiguates them. It can be hypothesised that such an evaluation process is occurring all the time, but becomes evident when ambiguities are maximised, as in the case of ambiguous stimuli. Indeed, also multistable ambiguity involves a continuous and frequent evaluation of sensory inputs, but it results in puzzling ips (or reversals) in perception, since ambiguous percepts do not provide enough clues for "deciding" which is the "good" interpretation. But, in partial contrast with the above quoted traditional denition, I do not consider all possible interpretations of an ambiguous image as mutually excluding. In Chapter II, on the basis of brain imaging data, I will propose the notion of different levels
13
14
introduction
of ambiguity, including in such a framework the concepts of the illusion of life and of life evocation as "higher level" ambiguities. The illusion of life is a phenomenon that can be considered as the perceptual basis of anthropomorphising, or attributing animacy, emotions, intentionality, personality and other traits common to living agents to non-animated stimuli. The illusion of life takes place when geometrical stimuli showing certain features of motion and behavior automatically elicit attribution of animacy and intention, as investigated in the seminal studies of [Michotte, 1946] and [Heider and Simmel, 1944]. While the illusion of life is automatically and universally perceived, with the only exception of persons with very specic brain damages, anthropomorphising is a psychological tendency implying an individual variability and depending on factors such as social isolation and need of mastering the environment. In Chapter III I will focus on the features and neural events underlying the illusion of life, outlining why it can be considered as an illusion of ambiguity. As illustrated in Chapter IV, perception of animacy of non-animated objects has always been exploited and explored by artists, who have been featuring in their works the aesthetics of life evocation. Finally, life evocation triggering anthropomorphising has a great potential of application in todays society, where humans interact more and more with technological objects such as robots and avatars, designed to embody believable creatures. In Chapter IV I will discuss some features of animacy perception drawing inspiration from arts and insights from the cognitive sciences, with the objective of outlining believability issues for the design of technological applications. 1 .1
Perception as an active process
perception : making sense of the senses
Philosophy and science have traditionally separated intelligence from perception, vision being interpreted as a passive windows on the world. It was German fellow Hemann von Helmholtz (1821-1894) who rst proposed the principle that visual perceptions are unconscious inferences [von Helmholtz, 1962]. Von Helmholtz thought that human perception is only indirectly related to objects, being inferred from fragmentary data. There have been researchers who maintained a "direct" theory of perception, notable American psychologist J. Gibson [Gibson, 1950] who outlined the theory of affordances, proposing a model in which our senses "pick up" information form the environment giving signicance to pattern of stimulation without recurring
1.1 perception : making sense of the senses
15
to further processing. But the majority of studies today agree on the hypothesis that perception is not a passive reception of the information coming from the surrounding environment, rather afrming that the brain is a story-teller who actively builds the reality surrounding us, as illustrated by [Roseneld, 1988]. Researchers point out that our senses are faced by a chaotic, persistently changing world without labels. What our brain actively does is organising external reality, allowing us to generalise and thus use the information we need from the environment, imbuing it with meaning. Since meaning is not in things but is in the brain, information, if not interpreted by the brain, is empty of meaning. 1.1.1 Watching without seeing
The born-blind issue
Researchers supporting the thesis of "active perception" often refer to the argument of born-blind people regaining sight during adulthood. Indeed, born-blid individuals who recuperated their sight after many years, even if visually perceiving objects, do not "see" them (they do not understand what the objects are), or learn to do it after a long and difcult training. In this brain condition, generally labelled as agnosia, sensations of light, colour, movement and shape can reach the brain but are meaningless; therefore for individuals suffering from this condition, objects are seen as "meaningless" items. The rst debate about this topic can be traced back to a correspondence between English philosopher John Locke and William Molyneaux [Gregory, 1987]. In 1688 the Irish scientist and politician William Molyneux (1656 1698) sent a letter to John Locke in which he asked whether a man who has been born blind and, during the course of his life, has learnt to distinguish and name a cube and a sphere by touch, would be able to distinguish and name these objects simply by sight, once he had been enabled to see. The so-called "Molyneaux problem" has been solved forty years later, when the English surgeon William Chelsden operated a 13 years old from a cataract, allowing the boy to see for the rst time in his life. The rst impression of the boy was objects were "disproportioned". Moreover he did not have any sense of distance, and of the relation between size and distance: a little object very close to his eyes was equivalent to a big house seen from far away. He was not able to distinguish a cat from a dog, unless he could touch them.
16
introduction
a case of sight recovery When he rst saw, Cheselden writes, he was so far from making any judgement of distances, that he thought all object whatever touched his eyes (as he expressed it) as what he felt did his skin, and thought no object so agreeable as those which were smooth and regular, though he could form no judgement of their shape, or guess what it was in any object that was pleasing to him: he knew not the shape of anything, nor any one thing from another, however different in shape or magnitude; but upon being told what things were, whose form he knew before from feeling, he would carefully observe, that he might know them again; and (as he said) at rst learned to know, and again forgot a thousand things in a day. One particular only, though it might appear triing, I will relate: Having often forgot which was the cat, and which the dog, he was ashamed to ask; but catching the cat, which he knew by feeling, he was observed to look at her steadfastly, and then, setting her down, said, So, puss, I shall know you another time. He was very much surprised, that those things which he had liked best, did not appear most agreeable to his eyes, expecting those persons would appear most beautiful that he loved most, and such things to be most agreeable to his sight, that were so to his taste. We thought he soon knew what pictures represented, which were shewed to him, but we found afterwards we were mistaken; for about two months after he was couched, he discovered at once they represented solid bodies, when to that time he considered them only as party-coloured planes, or surfaces diversied with variety of paint; but even then he was no less surprised, expecting the pictures would feel like the things they represented, and was amazed when he found those parts, which by their light and shadow appeared now round and uneven, felt only at like the rest, and asked which was the lying sense, feeling or seeing? [Cheselden, 1683-1775] The boy thus had a troublesome learning path, in which nothing came natural or spontaneous; he was conscious of the effort and he was trying to learn and record things everyday. For instance, he knew that he had a cat and a dog, and that he had to distinguish them by seeing them; this task was probably as difcult as for a normal-sighted person learn to distinguish at a glance two very similar specimen of the same breed. But, tridimensionality appeared to Cheseldens patient as some kind of
17
illusion. Even after he acquired the capacity to correctly interpret bidimensional images, he was still "amazed" to perceive the body represented by paintings as at. The phenomenon thanks to which he could see a bidimensional object as tridimensional was so obscure to him that he was convinced that one of his sensory channels must have being lying. Several similar operations have been conducted afterwards, and always with the same results. In particular, bidimensional representations are always troubling for born-blind individuals recovering their sight, because they see such representations in three dimensions, and they cannot understand how this can be, when they touch the image nding it at. "Normal" perception of size, distance and tridimensionality is a phenomenon that we take for granted but is actually the result of an operation of abstraction produced by the brain. 1.1.2 The Brain: an Abstraction Machine
Distance, size and tridimensionality are abstractions produced by the brain among all the mechanisms allowing us to recognise objects. Acquiring the capacity to create the abstraction of "representation on two dimensions" is probably something that we learn to do very early in time, and blind born individuals do not have the possibility to develop at "due time". Studies afrm that the common capacity of the cerebral cortex - whether visual, auditory, somato-sensory ot otherwise- is namely the one of abstraction: "the capacity to abstract seems to accompany, and to be a corollary of, every specicity" [Zeki, 2009]. What it is meant for abstraction, in this specic context, is the capacity to grasp a general property instead of the particular one. An example of this process at a single cell level is orientation selectivity [Hubel and Wiesel, 1977]. Orientation selectivity is the property of some cells in the visual brain, which respond to objects (lines) of a specic orientation and do not respond to lines oriented orthogonally at their "preferred" orientation: there are cells which will respond only to vertical lines, and other to horizontal ones. For instance, a vertical-selective cell will respond when the stimulus is vertical, without being concerned with what is vertical, whether a tree or a vertical line or a tower: this means that the cell has abstracted the property of verticality from different stimuli discarding individual specicities. In Figure 1.1, we can see the reponse of an orientation selective cell. Such a response can be studied by inserting an electrode
Abstraction at cell level
18
introduction
Figure 1.1: The response of an orientation selective cell
into the visual cortex (a) and showing bars of light of different orientation into the Receptive Field (the small rectangle on the left, RF), that is to say that part of the global visual eld (the bigger rectangle) which will trigger an electrical discharge in the studied cell. (b) shows the representation of the cells selectivity to orientation: the cell responds positively to the oblique line moved in two opposite directions (rst three records) while it is unresponsive to the orthogonal orientation. In the same way, cells of the brain area specialised for processing motion, V5, are directionally selective. This means that they respond to motion in a specic direction and not in the opposite one, as illustrated in Figure 1.2. This kind of abstraction does not concern exclusively the visual brain, but also other sensory modalities. Cells "designed" for perceiving pressure in the somatosensory cortex will respond to pressure independently from what causes it; the same thing is true for a cell responding to temperature, pain, and for the auditory system.
19
Figure 1.2: The response of an orientation selective cell in V5
1.1.3
The Case of Colour Vision
Abstraction is the strategy used by the brain in order to create a coherent environment out of a chaotic and ever-changing world. As in the largely discussed example of colour vision, we cannot help but perceive and organise constant features; we cannot avoid seeing a leaf as green at dawn and dusk, or a rose as red both in a sunny and cloudy day, taking for granted that "colours" are features intrinsic to objects. Since the world we see is always changing and the retina receives a constant ow of different kinds of visual information, the brain must be able to select visual properties of objects and surfaces in order to give them meaning. In acquiring this ability, the brain has developed specialized functions for the analysis of different properties, such as colour, shape, and movement. For example, contrary to our visual experience, there are no colours in the world, only electromagnetic waves of many frequencies. Our retinal receptors for colour are divided in three categories, responding optimally to long (red), middle (green), and short (blue) wavelengths. The brain compares the amount of light reected in the wavelengths, and from these comparisons creates the colours we see. The amount of light reected by a particular surface - a table, for example - depends on the frequency and the intensity of the
What are colours?
20
introduction
The Land hypothesis
light hitting the surface; some surfaces reect more short-wave frequencies, others more long-wave frequencies; but the particular sensation produced by a specic colour is created by our brain. Indeed, if our perception of an object as red were to change with every change in the wavelength of light reected from it, the object would constantly change in appearance. If we were aware of our real visual worlds we would see constantly changing images of dirty grey - constantly changing images that would be very confusing, often making it impossible for us to see forms. While, thanks to this mechanism, the brain is able to attribute a constant colour making itself largely independent of the amount of light of the waveband reected from objects surfaces. This mechanism is a way of our brain of "inventing" an attribute allowing us to identify objects by means of a constant and meaningful feature. How can this happen in the brain? Von Helmholtz [von Helmholtz, 1962] based this phenomenon on learning: he thought that since we know that a leaf is green, we operate a mechanism, which he names "unconscious inference", allowing to "attribute" a colour to the leaf even if the illuminant (the wavelength composition of the light reected in different lighting conditions) always changes. Another German psychophysicist, Ewald Hering, thought that memory played a central role [Hering, 1964 (originally pub. 1877]. Higher cognitive functions have therefore always been addressed when trying to explain mechanisms underlying colour vision, until Edwin Land (the inventor of the Polaroid) nally proposed another point of view [Land, 1974]. He hypothesised that the assignment of constant colours to objects is the result of a simple computational process of the brain, due to some kind of innate capacity of organisation of visual signals. Following Land, colours are the result of the comparison between the amount of light of different wavebands reected from a surface and from its surrounds. This comparison has as a result a ratio, which is constant (differently from the amount of reected light). the inherited concept of ratio - taking : In the Land experiment, a green surface is surrounded by yellow, red and blue surfaces. If the surface is lighten by a projector projecting blue light, the surface will reect lets say 60 units (milliwatts) of green light. The surrounds will be reecting less green light. The same surface is seen under a red light: the green surface will now reect 30 units of green light. But we will still see it as green! This is because the surrounding surfaces will reect even less units
21
of green light than before. Indeed, the ratio (relationship between the amount of green light reected by the green surface and the one reected by non-green surfaces) is always identical. Colour perception is the most powerful example of how our experience of the world is based on the brains capacity to perform operations related to inherited concepts rather then mere physical reality. The crucial feature of the colour system in the brain is the ability to ascribe a constant colour to an object despite of wide-ranging changes in the wavelengths-energy composition of the light reected by such an object. If the colour of the surface would change along with every change in its lighting environment, objects would have ever-changing surfaces (e.g., leafs would be green in the morning and red in the evening). This mean that colours would not mean much and, rather than provide us with information about things, would actually confuse us, being a non-constant overow of useless information. The importance of understanding colour as a construction of the brain is that colour is an example of mechanism that the brain has developed through evolution, and that concurs to instil meaning and thus gain additional knowledge from the environment: our visual worlds are stabilized because the brain, through colour perception, simplies the environment by comparing the amounts of lightness and darkness in the different frequencies from one moment to the other. In the same way, an object maintains its identity independently from the point of view and distance we look at it. The capability to organize constantly changing stimuli in a stable and meaningful way is an impressive feature of perception and is the way our brain allows us to cope with reality. And just as colours are different in kind from the images projected on our retinas, space and depth are different in kind from the individual images projected the retinas of our eyes. In creating three-dimensionality and depth the brain in actually invents subjective experience. The three-dimensionality we normally see is not part of the "real" world; all of our perceptions are from a particular point of view and that individual point of view is our subjectivity; it is created by the brain as it makes sense of the physical world that surrounds us.
22
introduction
1 .2
concepts and categories in the brain
Categorising behaviour
We saw how the brain can be independent from the continual change of the surrounding information, allowing us to recognize essential and non-changing characteristics of objects and situations, optimising knowledge acquisition by abstraction. Colour perception is an example of categorisation intrinsic to the brains design. The capacity to see colours can be lost, as happened to the most famous case of cerebral achromatopsia, the one of the colour-blind painter Johnathan I. [Sacks, 1995]. But, even if the capacity to see colours stays intact, the brain can lose the capacity to abstract; and in this case, even if colours are "viewed", they are not "understood", becoming useless. Gelb and Goldstein [Gelb and Goldstein, 1925] have reported the case of a patient who was incapable to dene the colours of objects, even if, apparently, he could perceive them. The patient could not indicate the name of the colour of an object mentioned to him, neither he was able to point out a colour corresponding to a colour-name. Presented with colour samples, he uttered some names, giving the impression that words had no meaning. When he was asked to choose a red object among a sample of different objects, he would choose by chance. But, if he was requested to chose a colour sample tting to a coloured object, he always succeed: he never chose a wrong colour. On the contrary, he was never satised: if the colour specimen did not match perfectly with the colour of the object, he continued looking for a more tting specimen. This proved that he could "perceive" colours. But, when the patient had to assort different specimen of colours, the patient was incapable to nd a unifying category in order to group them: the factor of hue, brightness, or other factors may prevail. Gelb and Goldstein explained the patients behaviour (as well as other patients symptoms) as effects of the reduction from the level of "categorical" behaviour to the level of "concrete" behaviour. Confronted with the same task (grouping colour specimina according to hue) the normal person immediately assorts the same specimina, e.g. a very light and a very dark red, by categorizing them as "red". In doing so, she is not unaware of the difference between them; she is performing an operation of abstraction (abstracting the "redness") and categorisation (by discarding "individual" differences among specimina and unifying under the category "red"). Gelb and Goldsteins patient, even seeing colours, was unable to establish abstract relations between the category (red, blue, green, etc.) and the
1.2 concepts and categories in the brain
23
concrete colour. When he was asked to dene the colours, he always had to make a comparison (e. g. "like an orange, like a cherry") and he was meticulously grouping colours but without backing up on a categorical sense. The patient was conned to the content of perception as it is actually experienced (he was seeing the "real" world, in a way); in other words he could not go through his perceptual experience under the perspective of a principle external to the perceptual experience itself, being unable to refer his actual experience to any conceptual order. In the framework of this regression to a "concrete" behaviour, there is no hierarchy or differentiation between the experiential features and their signicance. All features are equally important; each features is of paramount importance because all of them are encountered "at the same level" in actual perception. The patient was overwhelmed by the actual experience, and he could not emancipate from it by imposing a structure able to create meaning in the world. What Goldstein refers to as "categorising" behaviour is parallel to Zekis theory of inherited and acquired concepts: "The inherited concepts organize the signals coming into the brain so as to instil meaning into them and thus make sense of them", [Zeki, 2009] while acquired concepts are generated throughout our existence, with the goal of simplifying perceiving and recognizing things and situations. Inherited brain concepts are identiable by three features, in Zekis view: absence of free will, immutability and autonomy. Therefore, in this perspective colour perception may be classied as an inherited concept, since we can not choose what colours we perceive, nor we can choose of seeing the "real" colours of things instead that the colours "invented" by our brain; secondly, our system of perception for colours is immutable, does not change over years - in normal conditions - and thirdly, the generation of colours is dependent on a specialised cortical system that differs from other systems designed for processing other kinds of visual features. While acquired concepts are a way to organise knowledge throughout our life (for instance, we can learn to dene a specic set of paintings as "still life" and we will be able to insert in such a category all still life that we will see afterwards, even if they will be composed of different objects, lighting conditions, colours, etc.). Still, what acquired and inherited concepts have in common is the capacity of the brain to abstract and to generalize.
Inherited and acquired concepts
24
introduction
1 .3
seeing through illusions
The way in which our brain organises knowledge can be more efciently investigated in exceptional conditions: those situations in which things do not work as usual. This means analysing those circumstances in which the brains capacity of stabilisation and abstraction is invalidated, temporary or permanently discarded, as in the cases of brain damages and illusions [Gregory, 2009]. It is hard to give a satisfactory denition of the term "illusion". The OED (Oxford English Dictionary) denes it as "an instance of a wrong or misinterpreted perception of a sensory experience"; here I will refer more precisely, to those phenomena implying a discrepancy measurements and inner perception. Such a discrepancy can happen when the knowledge (inherited or acquired) that we "superimpose" to perception is inappropriate or misapplied; in other words, where the "assumptions" we are doing are wrong. Indeed, perception is - following this perspective - based on inferences that we deduce from sensory signals; being an inference, it requires a previous knowledge. This is why illusions can clarify many mechanisms behind perception, indicating where and how this "previous knowledge" (abstraction, categories and things we learn throughout our life) play a role. As Chelsdens blind born boy asked himself whether was the sight or the touch which was lying: he thought that perceived tridimensionality was an illusion. And in a way it is, because behind our capacity to view tridimensional images in pictures there is a level of abstraction: illusions can provide evidence of working rules of perception. 1.3.1
Physical illusions
Two Different Categories of Illusions
Gregory identies two different kinds of illusion, the cognitive and the physical ones. While the latter ones have a physical cause, cognitive illusions are due following Gregory to "misapplication of knowledge" employed by the brain in interpreting sensory signals [Gregory, 2009]. Illusions due to the disturbance of light, between objects and the eyes, are different from illusions due to the disturbance of sensory signals of eye, though both might be classied as physical. Among these, we can list with Gregory mist (loss of information increasing uncertainty), mirage (refraction of light between the object and the eyes displacing objects or parts of objects, rainbow. Discussing this
1.3 seeing through illusions
25
taxonomy is nevertheless outside the scope of this thesis, and for further reading on this subject we remand to [Gregory, 2009]. Cognitive illusions are further devided by Gregory into "specic knowledge" of objects from "general knowledge". An example of specic knowledge cognitive illusion is the "hollow face" illusion.
Cognitive illusions
Figure 1.3: A rotation of a mask: the forth image shows the inside of the mask, appearing convex even if it is hollow
As illustrated in Figure 1.3, when seeing a hollow mask we are strongly biased in seeing it as convex. This bias of seeing masks as convex is so strong it competes with monocular depth cues, such as shading and shadows, and also very considerable unambiguous information from the two eyes signalling stereoscopically that the perceived object is hollow. Also, a texture imitating wood could look like wood even if it is plastic, or if it is painted; this is because we apply our specic knowledge to the texture and make an inference about the material on the basis of a specic and repeated experience (wood is grainy, brownish, etc.). An example of general knowledge leading to cognitive illusions is misleading rules of Gestalt (Wertheimer, 1923/1938) applied to tricky objects (designed to trigger the illusion) e. g. Kanitsa triangle, due to postulating a nearer occluding surface to
26
introduction
A model of perception
Automaticity of illusions
explain "surprising" gaps [Gregory, 1972]. General knowledge, in this case, is the rules of Gestalt (closure, proximity, continuity, common fate) singled out by Wertheimer and inuencing our vision of objects (independently from the objects themselves). In order to trigger illusion related to misapplication of general knowledge it is necessary to investigate the perception of exceptional and atypical objects. For Gregory [Gregory, 1972] this is the evidence of top-down knowledge applied to perception: this means that since we know, due to our experience, that faces and therefore masks - are generally convex, we perceive it as such. The top-down knowledge applied to this experience is very strong, and even if we know that the mask is hollow, we see it as convex. Interestingly, patients affected by schizophrenia do not perceive this illusion [Schneider et al., 1996]. This may mean that they are more inclined to top-down inuences since they are affected by a lack of connectivity: Dima [Dima et al., 2009] demonstrated that schizophrenic patients show an increased activity of bottom-up processes if compared to healthy controls, who do perceive the hollow face illusion; schizophrenic subjects in this sense see the world "as it really is" relying on stimulusdriven processes. On the basis of this perspective on illusions as a model of perception is illustrated in Figure 1.4: perception is an hypothesis, or an inference in Von Helmotzs detion [von Helmholtz, 1962] coming from a continuous dialogue of signal processing, perceptual and conceptual knowledge. Signals from the senses (bottom-up) are not only inuenced (top-down) by pre-existing knowledge (e.g.: faces are convex) but also by perceptual "rules", which constitute the "language of the brain" (e. g. Gestalt rules, colour view); this continuous loop gives rise to qualia, sensations about what we perceive. What is typical of several illusions is that they are experienced perceptually though the observer knows conceptually that they are illusions. This phenomenon can tell us something interesting about the nature of knowledge that we superimpose to perception. It is not "conceptual" knowledge, but it is a separated kind of knowledge, that could be dened as "perceptual knowledge". This perceptual knowledge is probably linked to evolution allowing perceptual mechanisms to work very fast and efciently, this being useful in case of danger. Perceptual knowledge is the assumptions that our brain takes, and can coexist with conceptual knowledge even when in contradiction, as in the case of several illusions. Also, if perceptual knowledge would not be different from conceptual knowledge, our beliefs would deter-
27
Figure 1.4: Model of perception, based on (Gregory 2009)
mine perception making us blind to the new or strange, which would be dangerous in unusual situations, and would limit perceptual learning. What is this previous knowledge? For von Helmoltz, signal processing interpretation is resolved thanks to previous knowledge by unconscious inductive inference. As we saw above, Zeki [Zeki, 2009] declines this denition in inherited and acquired concepts, granting the brain a grasp on the world (but also demanding a "prize to pay" when misapplied). Gregory [Gregory, 2009] organises "previous knowledge" in general knowledge and specic knowledge, proposing that perceptions are hypothesis predicting "unsensed characteristics" of the objects. I will use the term "conceptual knowledge" for indicating top down higher cognitive knowledge and "perceptual knowledge" for indicating bottom up impressions that can, as investigated in the next chapters, overcome and/or coexist with conceptual knowledge. 1.3.2 Ambiguity and Ambiguities
Among different illusions, it is especially the phenomenon of ambiguity that make us think of perception as actively creative: indeed, it makes clear that some kind of interpretation and
28
introduction
A general denition
Four levels of ambiguity
previous knowledge is necessary. Ambiguity, understood as it is dened in the OED ("ambiguous: uncertain, open to more than one interpretation, of doubtful position") is a feature of the environment that our brain faces every day, since it is confronted with situations or views that are open to more than one interpretations. Retinal images are inherently ambiguous (for example for size, shape and distance of objects), but also language, situations, sounds. If, on the basis of the above model of perception, interpretations are hypothesis, thus, ambiguity means openness to two or more alternative hypothesis. Such hypothesis can be mutually excluding in the instant but coexisting over time, or just coexisting. This thesis will explore the hypothesis of different levels of ambiguity, on the basis of the possibilities of different interpretation to be mutually exclusive, partially or totally coexist. In certain cases, as the above discussed case of colour vision, the "choice" among interpretation is relatively simple, since it is limited by the design of the brain. It becomes more complex in other cases, when no single solution is more likely than other possible solutions. Each interpretation becomes as valid as the other interpretations, and there is no correct interpretation. As illustrated in Figure 1.5, I will propose a framework of analysis identifying four levels of ambiguity, whose rst two levels are illustrated in Chapter II as results of a brain imaging study on neural bistability. In Chapter III I will introduce the concept of "illusion of life", as a third level ambiguity, in which two possible interpretations are coexisting in the conscious stage of the qualia but not mutually excluding; the illusion of life always coexist with the awareness of the non-animacy of the stimuli; nevertheless, such an illusion is automatically and universally perceived (except for patients with specic brain damages). Going further, we could identify a phenomenon of ambiguity in anthropomorphism, in which the concept of individual variability has to be introduced (cfr. Chapter IV).
29
Figure 1.5: Different kinds of ambiguities
Part II FIRST AND SECOND LEVEL AMBIGUITY
P E R C E P T I O N O F B I S TA B L E A M B I G U I T Y
This chapter will discuss literature about how and where ambiguous perception takes place in the brain, and why ambiguity is a crucial issue for understanding how the mind works and how do we gather information from the surrounding environment. It will also discuss the results of an fMRI experiment carried out in the general framework of ambiguity with a focus on visual bistability, proposing a framework of analysis based on multiple levels of ambiguity, which takes into account that different interpretations of the stimulus content may not only exclude each other, but also coexist, depending on different levels of ambiguity. 2 .1 what is multistable ambiguity
Bistable ambiguous images are puzzling because they can be spontaneously experienced as two equally valid percepts. Following the model of perception illustrated in Chapter I, when the brain is confronted with an ambiguous stimulus perceptual knowledge is oscillating between two different interpretation, while conceptual knowledge is aware that the percept is not actually changing. Following the literature, during the so-called bistability illusion, the brain is confronted with one ambiguous stimulus that can be stably interpreted in only one way at any given moment, but will present two possible interpretations over time; in other words, a stimulus is ambiguous when it is consistent with two or more mutually exclusive interpretations [Zeki, 2004]. Repeated viewing of ambiguous stimuli lead to spontaneous perceptual switches, or "ips", where the brain alternates between two or more stable perceptual interpretations every few seconds. Multistable ambiguity is an especially interesting illusion because it can help us understand the mechanisms generating a meaningful and coherent experience of the world, even though the the information we have is fragmentary and ambiguous. Visual illusions of ambiguity are actually a sort or quintessential and simplied model of the choices that our brain is confronted with everyday, and in every modality. Some
33
34
perception of bistable ambiguity
Figure 2.1: a) Necker Cube - b) Rubin Vase
Binocular rivalry
Auditory multistability
Verbal transformation effect
of the most well-know ambiguous gures are the Necker cube and the Rubin vase (Figure 2.1). The perceptual interpretation of the Necker cube oscillates between two different recessional planes, while the interpretation of the Rubin vase alternates between one vase and two proles. Ambiguity is present also in other perceptual phenomena, namely bistable apparent motion and binocular rivalry (Figure 2.2). Binocular rivalry results from presentation of different images to each eye, thanks to a dispositive separating the view of one eye to the other one. The result is bistable alternation between the two images: the subject is conscious of one stimulus at the time, and the perception is incessantly uctuating between the two stimuli. For instance, the subject represented in Figure 2.2, looking at the vertical lines with his left eye and looking at the horizontal lines with his right eye will cyclically perceive only vertical lines alternating with only horizontal lines. Multistable perception (when the possible interpretations are more than two) can characterise the auditory modality as well, in the form of auditory stream segregation [Bregman, 1990] and verbal transformation effect [Warren and Gregory, 1958]. Auditory stream segregation happens when two tones of a different frequency are presented alternately in a repeating temporal pattern, as illustrated in Figure 2.3. Subjects perception alternates between interpreting the sequence either as one stream with uctuating tones or as two segregated streams (in the picture, indicated by the grey thick line). Verbal transformation effect occurs when a word is cycled in continuous repetition [Warren and Gregory, 1958] like for
2.1 what is multistable ambiguity
35
Figure 2.2: Binocular rivalry
instance the word "life". Initially, a percept corresponding to the original form prevails, but after a certain period another percept takes over and it stably alternates with the original one: in the case of the repeated word "life", at a certain point the subject will perceive the word "y" alternating with "life". Ambiguity is also a feature of many sentences, which can mean two very different things, being in a way "bistable" and interpretable only by means of the context in which they are used. There is a distinction between vagueness and real ambiguity: truly "bistable" sentences are the ones whose "meaning" can ip and can be stabilized only by context, for instance. "People like us" [Gregory, 2000] or "She cant bear children". Bistability exist also in the modality of touch, even if it has to be triggered articially. A tactile illusion has been developed by Carter et al [Carter et al., 2008]. By means of a device designed to provide the subject with vibrotactile stimuli, the researchers could lead participants to report switches between the perception of motion directed either up and down or left and right across their ngertip, while the sensory input due to the vibrotactile device stayed unvaried. Nevertheless, visual ambiguity is undoubtedly the most studied phenomenon among all these. Indeed, ambiguous gures have become an experimental tool to study interpretive cognitive processes related to conscious awareness because the perceptual experience alternates over time without any external changes of the stimuli [Leopold and Logothetis, 1999]; [Sterzer et al., 2009].
Ambiguity in language
Tactile ambiguity
36
Figure 2.3: Auditory streaming is a case of multistable perception in the auditory modality
In particular, interest in this eld has been growing thanks to the advent of non-invasive brain imaging techniques such as fMRI (functional magnetic resonance imaging), which is the technique used for the experiment described further on. 2.1.1 Neural processes underlying multistable phenomena
Top-down vs bottom-up
Even tough perception of ambiguous gures has been extensively studied, there are many contradicting ndings and controversies. Traditionally, two main theories explain why ambiguous gures reverse; the main distinction is between supporters of the central role of bottom-up versus top-down processes [Leopold and Logothetis, 1999]. The bottom-up theory proposes that neurons maintaining one percept fatigue over time and give rise to neurons supporting the other percept; the switch should thus be independent from high-level (cognitive) control. According to the top-down theory, ambiguous gures do not switch spontaneously: the preconditions for seeing reversals are that the viewer knows the gure is bistable, knows the possible interpretations of the gure, and switches are initiated by intention. There are experimental ndings supporting both theories, thus
37
more recent studies have adopted an hybrid model explicating bistability as the result of a continuous dialogue between bottom-up and top-down processes [Toppino and Long, 2005] [Mitroff et al., 2006] Concerning the localisation in the brain where bistable ambiguity takes place, several studies show that changes in early visual activity precedes conscious changes of perception. Thanks to fMRI studies, it has been consistently demonstrated that binocular rivarly strongly effects V1, the earliest visual processing area [Polonsky et al., 2000], [Tong and Engel, 2001] [Lee, 2005]. A MEG study [Parkkonen et al., 2008] shows that early visual brain areas (V1 and V2) reect how an ambiguous gure is perceived, both for binocular rivalry and for Rubins vase. Also activity in primary visual cortex recorded with EEG could predict what perspective is perceived even before a geometrical ambiguous gure was presented [Kornmeier and Bach, 2005]. This study demonstrated that perceived reversal of perspective was preceded by 160 ms with negativity in primary visual cortex. A similar negative component was found 50 ms earlier doing the experiment with a stabilized version of the gure. fMRI studies have also shown that activity not only in V1 but also in another post-retinal processing area, the LGN (lateral geniculate nucleus) reects the perceptual outcome of binocular rivalry [Tong et al., 2006]. These experiments alone could support the bottom-up thesis, that is to say the hypothesis that perception of ambiguous gures is resolved in a feed-forward manner by primary sensory areas without the involvement of higher cognitive areas; but, since there is evidence also for top-down processes, this data can be interpreted as a proof that early visual processing stages including V1 and LGN are the prerequisite for conscious interpretation of percepts. In any case, the role of this early visual areas is that of processing information but also inuence interpretation ("am I going to see a vase or a face?") either via local interactions or through modulation by feedback signals from higher cognitive areas [Tong, 2003]. Evidence for the latter hypothesis comes from fMRI experiments on bistable apparent motion. Indeed, whenever the perception of apparent motion is inconsistent with added clues, early visual activity is suppressed [Sterzer and Kleinschmidt, 2005]. Most studies agree on the role of the extrastriate cortex in perceiving multistable ambiguities (extrastriate visual cortex includes those areas lying beyond V1). Many fMRI studies revealed correlations between subjective perception and activity
Subcortical and early cortical visual processing
Extrastriate visual cortex
38
Neural correlates of ips
in the functionally specialised extrastriate cortex, that is to say, areas specialised to process a certain type of information, reecting the "content" of the stimulus itself. fMRI studies (see [Sterzer et al., 2009] for a review) demonstrated that during binocular rivalry, uctuations in signals in extrastriate cortex are similar to those during actual alternation of two different stimuli, suggesting that the interpretation of ambiguous stimuli are fully resolved at the stage of processing, without maintaining a representation of the temporally suppressed stimulus. Actually, perception of binocular rivalry is inuenced by information linked to the suppressed stimulus as well [Andrews and Blakemore, 1999] indicating that the suppressed stimulus is somehow "present" and processed in the brain. For instance, the emotional content of a suppressed stimulus (e. g. a fearful face) is still processed in the amygdala [Jiang and He, 2006], [Williams, 2004], where activity is expected when the stimulus is consciously perceived. A more recent and high resolution fMRI study has also found activities corresponding to responses to different object category in "houses versus faces" binocular rivalry, namely activations in the fusiform face area (FFA) and in the parahippocampal place area (PPA) even during binocular suppression of one of the two categories of stimuli [Sterzer et al., 2008]; in the same way, face-specic responses are reduced but still present in EEG during a face-suppression period [Sterzer et al., 2009]. In parallel with these ndings on binocular rivalry, studies on ambiguous motion and bistable images conrm that activity in the extrastriate visual areas correlates with conscious perception, and also that those areas are involved in the cyclical resolution of ambiguities. Following [Andrews et al., 2002] and [Hasson et al., 2001], signals from FFA are greater during the perception of two faces in the Rubin vase-face illusion; and following the same principle, a bistable illusion whose elements can be perceived either as a coherent shape or as a random structure, will correlate with lateral occipital complex (LOC, which preferably activates processing objects) activations when the stimulus is perceived as coherent. In the case of ambiguous apparent motion versus icker, the conscious perception of motion will correlates with activations in V5 (area which processes motion) while the conscious perception of icker will not. Another line of research has been focusing not on the perceptual states between one conscious perception and the other one, but on the ips themselves, in other words neural events correlating with perceptual reversals. Flips-related activity is
39
generally observed in extrastriate visual areas and it is associated with activations tuned to the features of the percept that is perceived. For instance, perceptual reversals from faces to object correlates with activations in the ventral stream [Leopold and Logothetis, 1999] while apparent changes in motion direction correlates with activation in V5, motion sensitive area. [Sterzer and Kleinschmidt, 2007] suggested that prefrontal areas set off changes in perception of ambiguous motion. In an fMRI study they used both ambiguous and unambiguous moving dots; activation in right inferior frontal cortex appeared earlier than V5 activation in both conditions. Moreover, the frontal activation happened earlier for spontaneous ips when looking at ambiguous stimuli, compared to when a ip was stimulus driven. By using EEG, [Britz et al., 2009] were able to predict stimulus perception in a similar manner to Kornmeier and Bach (2004), they found signicant increase of activity in right inferior parietal cortex 50 ms before a ip in perception occurred. It has also been demonstrated that activity in the FFA can be used to predict whether faces or a vase is perceived during presentation of the Rubin gure [Hesselmann et al., 2008]. Indeed, activity in the FFA is higher when subjects subsequently report perceiving two faces instead of a vase, suggesting that pre-stimulus neural activity precede subsequent perceptual inference. Such activations suggest that ongoing brain activity inuences the resolution of ambiguity before stimulus driven processes, and a key role of functionally specialised areas in processing and interpreting conscious visual perception. fMRI activations correlated with perceptual reverals or ips are also assessed in the parietal and frontal areas [Lumer et al., 1998]. While extrastriate areas are equally activated both by non-ambiguous and by ambiguous stimuli, parietal and prefrontal regions show higher levels of activity during ambiguity illusions [Lumer et al., 1998]. Such special activations in frontal and prefrontal cortex could suggest top-down mechanisms triggering a re-organisation of activity in the primary sensory areas during ips as in [Leopold and Logothetis, 1999]. Or, from the bottom-up perspective, they could reect the feed-forward communication of events from the earlier visual cortex to higher cognitive areas, a sort of initiating gateway for further processing. For example, changes in apparent motion perception show that activations of the prefrontal cortex precede that of V5 for bistable motion perception [Sterzer and Kleinschmidt, 2007]. This "initiating" role is corroborated by other ndings,
Parietal, frontal and prefrontal cortex
40
Bottom up versus top down: conclusions
drawn from a study suggesting that ips during observation of a Necker cube are preceded by activations in the right parietal cortex [Britz et al., 2009]. In such a framework, it is noteworth that many people are able to control ips of bistable gures, which suggest that perception is also inuenced by top-down factors. For instance, participants were able to voluntarily increase or suppress the frequency of switches for the Necker cube [Tracy et al., 2005]. It is demonstrated that higher cognitive areas have an effect on voluntary switches. Indeed, the ability to voluntarily increase switches is diminished in frontal cortexdamaged patients; but, the same patients did not experience a signicantly different rate of reversals when ips where happening spontaneously [Windmann et al., 2006], while research of effects of meditation consistently demonstrate that meditators can alter the normal uctuations in conscious state induced by binocular rivalry [Carter et al., 2005]. The role of frontal cortex in active switches suggests that top down inuence may not be necessary, but higher cognitive areas can play a role voluntarily initiation changes, while alternation during passive viewing is less dependent on prefrontal cortex. Activities in frontal and parietal cortex is not only associated with ips, but also in percept stabilisation. Indeed, the tendency of an individual to stabilize a percept during, for instance, periods in which the stimulus has been removed, is correlated with activations in the frontal and parietals areas [Raemaekers, 2009]. In conclusion, experimental ndings neither support a pure top-down or bottom-up account. There is both support for reversals driven by primary visual areas, processing specic areas such as the FFA and parietal and frontal regions. Even though activations in typical higher cognitive areas appear to be related to perceptual switches [Kleinschmidt et al., 1998]; [Lumer et al., 1998]; [Sterzer et al., 2002] their involvement is still debatable. [Zeki, 2004] argues that the frontoparietal network is involved when there is a change in perception, without being involved in the actual visual percept. One solution to the contrasting experimental ndings is that there might be interaction between higher cognitive and primary sensory regions. Indeed, recent studies do not point to either a purely top-down or bottom-up model. For instance, [Britz et al., 2009] found that activation in right inferior parietal cortex precedes the ips, while [Kornmeier and Bach, 2005] were able to predict perception from activation in primary visual cortex. Also behavioural ndings seem to contrast a purely cognitive top-down account. It is true
41
that subjects who are not informed that a picture is ambiguous do not always experience reversals, but evidence supports that perceptual bistability be spontaneously experienced by children as young as 5 years [Mitroff et al., 2006]. 2.1.2 Attention and perception of bistable gures
The overlap between areas involved in spatial attention and perception of bistable gures have lead to speculation about the role of attention processes, suggesting that frontoparietal regions could activate when re-directing attention to sensory input and re-initiate an evaluation of the current interpretation, leading either to maintain stable the current interpretation or change it. [Slotnick et al., 2003] found common neural activation for both voluntary shifts of attention and voluntary perceptual reversals. Since perceptual reversals usually occur spontaneously there has been some debate about the exact role of the frontoparietal network [Sterzer et al., 2009]. Areas involved in voluntary attention may serve several functions, such as being engaged in feedback to the sensory areas and perform an ongoing re-evaluation or the visual experience [Leopold and Logothetis, 1999]. Another suggestion is that ambiguous information is detected in early sensory areas activates the frontoparietal network, which shifts attention between the possible versions over time. According to the so-called focal-feature hypothesis, local areas within an ambiguous gure favour different global interpretations. Necker himself (quoted in [Toppino, 2003] proposed that reversals are driven by eye movements. Perception of the Necker cube can be biased by moving the point of xation during viewing [Peterson and Gibson, 1991]; [Toppino, 2003], and similar effect has been found other bistable gures as well [Tsal and Kolbet, 1985]. Also free viewing conditions support that eye gaze and perception of the Necker cube is closely linked [Einhauser et al., 2004]. After a switch, the eye position also shifts. The authors suggest that the changes of eye position serves as a negative feedback signal to suppress the previous percept. [Leopold and Logothetis, 1999] propose that the same motor processes underlie both selective attention and bistable perception, and that there in most cases is a close coupling between saccades and percept switches. Changing eye gaze and perceptual reversals could both reect the way we actively explore and constantly reinterpret the stimuli. It may however be possible to alternate bistable gures without eye movements.
The attention issue
42
To sum up, while previous studies opposed top-down [Leopold and Logothetis, 1999] to bottom-up [Attneave, 1971], [Blake, 1989] models, now multistable illusion is considered as a continuous and frequent dialogue between low-level (sensorial) and high-level (parietal and frontal) areas, aiming to verify in time interpretation of stimuli (and initiating changes in perception). There may be a relation between selective attention and perception of bistable gures, but this still needs to be claried. 2.1.3 Levels of ambiguity
In his essay "The Neurology of Ambiguity" [Zeki, 2004] distinguishes between different levels of ambiguity, introducing a new component in a discussion traditionally limited to the bottom-up/top-down approach. Zeki proposes that ambiguity has different layers, or levels, which correlate with the neural activity in one or more processing regions with a conscious correlate. Following Zeki, gures like the Necker cube belong to the most simple form of ambiguity because brain activity remains within the same area over time, with the same neural correlates for both states it can be experienced. At a higher level of ambiguity such as the Rubin gure, several processing areas are involved and the current perceptual state becomes conscious by uctuation of neural activity between these areas. The Necker cube is always seen as a cube, while some images uctuate between image categories such as the Rubin gure. On a much more sophisticated level, artwork such as Vermeers paintings are ambiguous in terms of narrative interpretation. Zekis differentiation between levels of ambiguities is rooted in his theory of microconsciousness, where consciousness is seen as distributed over functionally specialized processing sites in the brain, which give rise to consciousness without further higher interpretation. The level of ambiguity can be dened at uctuations within one or between several microconscious states. 2 .2 an experiment on two - levels bistable ambiguity
In the work reported here, to investigate if there are general patterns of neural activations when looking at bistable gures, we compared perception of bistable images, where the physical stimulus remains the same but perception alternates between two interpretations, with perception of two externally alternat-
2.2 an experiment on two - levels bistable ambiguity
43
ing stable images. Subjects were instructed to repetitively report their conscious experience of the visual stimuli by key-presses, reporting the occurrence of perceptual endogenous reversals and the occurrence of actual reversals of stable percepts during a "replay condition". Based on previous research we hypothesized increased activation in the ventral occipital cortex, in parietal areas, as well as in frontal areas during bistable stimuli compared with stable stimuli [Ilg et al., 2008]; [Kleinschmidt et al., 1998]; [Lumer et al., 1998]; [Sterzer et al., 2002]. Further we wanted to test the theory of levels of ambiguity, verifying if perception of gures with different levels of ambiguity engage different brain areas. To compare levels of ambiguity we used two types of bistable images. We dene those images where perceptual ip takes place within the same category as "intra-categorical". This is the case for the Necker cube (Figure 2.1 a). Here, what appears to be in the front can occupy a different recessional plane with prolonged viewing, but nevertheless it conceptually remains the same gure. This contrasts with what we refer to as "inter-categorical", namely the two images created by a single picture belong to different categories, as in the face- vase bistable image (Figure 2.1 b). We dene as "intracategorical" those images in which the perceptual ip takes place within the same category. In the present study, inter-categorical stimuli alternates between bodies and faces, since studies have demonstrated the selectivity of different brain regions to the visual representation of faces [Kanwisher and Yovel, 2006] and bodies [Peelen and Downing, 2007]. We compared how neural activity during perception of bistable intra-categorical images and bistable inter-categorical images contributed to the overall activations during perception of all bistable gures. To our knowledge, this is the rst imaging study comparing two levels of ambiguity. In line with [Zeki, 2004] above quoted theory we expected more involvement of higher cognitive areas for the ambiguous inter-categorical images. As a third approach, we also addressed how brain activity uctuates over time, and if transient patterns of activation are related to the type of images perceived. We investigated if separate neural mechanisms are involved in the transition from one percept to another. The goal was to investigate if two percepts belonging to the same attribute or category, for example the two recessional planes of the Necker cube, evoke different areas of activation compared to when the transition is from one category to another, as in the Rubin face-vase bistable image. We looked
Levels of ambiguity
Content of the percept
44
Experimental design
at neural activation during alternating percepts for both intraand inter-categorical gures. We hypothesized that same brain area would be active when the translation would take place within images belonging to the same category, while areas of activation would change when the two percepts belong to two different categories. Based on previous ndings, we further hypothesized that during reported face perception the FFA would respond stronger [Andrews et al., 2002]; [Hasson et al., 2001]; [Tong et al., 1998], while the same areas would be active when the translation would take place within images belonging to the same category. Design: We used bistable images to separate, and therefore compare, perceptual from stimulus-driven changes. In our study, subjects were requested to repetitively report by key-presses their conscious experience of ips during the observation of bistable gures. Their responses were recorded and the subjective occurrence of perceptual reversals was replayed by alternating the two stabilized versions of the same percepts. We used two sets of bistable images: intra-categorical and intercategorical. Subjects were instructed, when looking at "geometrical gures" (intracategorical stimuli) to alternate key presses when spontaneously perceiving a ip. For the intercategorical ambiguous gures, subjects were requested to press a specic key indicating body perception and another key indicating face perception. During the replay condition, subjects were presented with stabilised versions of the ambiguous gures. The onsets of the alternating stabilised pictures were the same as when subjects indicated seeing a ip in the ambiguous condition. Subjects were also requested to perform key presses during the replay condition, in order to control for motor responses. Subjects: 16 healthy subjects (with normal or corrected to normal vision; 8 females) were recruited through advertisements requesting volunteers for a study about optical illusions. Their age varied from 21 to 40 years (mean 29,8 years). Two subjects were left handed. Informed written consent was obtained from all participants and the study was covered by the Minimum Risk Ethics (Minimum risk magnetic resonance imaging studies of healthy human cognition, UCL Ethics Project ID number: 1825/003 / Data protection ref: Z6364106/2010/03/04). During a rst visit to the laboratory, prior to scanning, each subject was requested to do a pre-test in order to qualify for the fMRI experiment, by performing the same task to be carried out in the scanner. X participants were excluded because they were not
45
able to perform the task correctly. During each scanning session subjects heart-rate and respiration were continuously recorded, providing physiological measurements to be subsequently used as regressors-of-no-interest in the rst-level SPM analysis for each subject. Stimuli: Stimuli were generated using Cogent 2000 and Cogent Graphics (http://www.vislab.ucl.ac.uk/cogent.php). Four intra-categorical images were chosen from the images in The Psychophysics of Form: Reversible-Perspective Drawings of Spatial Objects, Hochberg and Brooks, The American Journal of Psychology, Vol. 73, No. 3 (Sep., 1960), pp. 337-354, University of Illinois Press) and 1 was created manually. Images were selected following a pre-test in which four subjects viewed 8 images and chose the ones that most easily ipped between two states for all subjects. All ve intra-categorical ambiguous images were generated using Adobe InDesign CS3. Five intercategorical images were chosen from existing ambiguous gures. The chosen intercategorical images were those displaying two mutually exclusive interpretation, a body OR a face. Subsequently, each ambiguous image was modied to create two stable versions, which could be shown successively to the subjects using Photoshop CS3, and two stable versions of each image were created (see Figures 2.4 and 2.5 ). Each subject was exposed to two runs displaying the same images in the same order. Each run began with a neutral background, lasting 26 s, during which the rst six brain volumes were discarded to allow T1 equilibration effects to subside. The stimulus sequence then began. During each session the 10 ambiguous images (5 intracategorical + 5 intercategorical) were displayed; subjects were instructed to alternate key presses when perceiving a ip from one percept to the other. With intercategorical ambiguous images, subjects were instructed to press a button to indicate whether it was a body or a face that they perceived. During each session, ambiguous images were mixed with a following replay condition of the subjects perception, displaying the two alternating stabilised versions of each ambiguous image. The replay condition was implemented using the recorded button presses relative to each ambiguous image so that the time sequences remained the same. Consequently, the onsets of the alternating stabilised pictures were the same as the alternating perceptual ips indicated by the subjects. In order to control for motion correction, subjects were also required to press buttons during the replay condition, each
46
time that the image changed. Each epoch lasted 16 s with an interstimulus interval varying in duration between 3 and 5 s between stimuli where a blank grey screen was presented. Stimuli were presented in a pseudo-random sequence, ensuring that each ambiguous image was presented before its stabilized versions. Scanning details: Scans were acquired using a 1.5T Siemens Magneton Sonata MRI scanner tted with a head volume coil (Siemens, Erlangen, Germany) to which an angled mirror was attached, allowing subjects to view a screen onto which stimuli were projected using an LCD projector. An echo-planar imaging (EPI) sequence was applied for functional scans, measuring BOLD signals (echo time TE =50 ms, repeat time TR= 90 ms, volume time 4.32 s). Each brain image was acquired in a descending sequence comprising 48 axial slices covering the whole brain. The experiment consisted of 2 runs; 100 volumes were acquired per run. After functional scanning had been completed, a T1* weighted anatomical scan was acquired in the sagittal plane to obtain a high resolution structural image (176 slices per volume). 2.2.1 General fMRI Analysis
Analysis: Data were analysed using SPM8 (http://www.l.ion.ucl.ac.uk/SPM). The time series of functional brain volume images for each subject was realigned and normalized into MNI space (voxel size 3 x 3 x 3 mm) and then smoothed using a Gaussian smoothing kernel of 9 mm. The stimulus for each subject was modelled as a set of regressors in the SPM8 general linear model (GLM) (rstlevel) analysis. The stimulus was a block design merged with an event-related design; boxcar functions were used to dene regressors which modelled the onsets and durations of each stimulus, as indicated by each subject by means of keypresses. Consequently, the regressors were: faces, bodies, geometrical state 1 and geometrical state 2. Regressors were further subdivided in stables versus unstable; stable onsets corresponded to actual changes, while unstable onsets corresponded to perceptual changes, as indicated by key-presses. Key-presses were modelled as delta functions in an additional regressor. Head-movement parameters calculated from the realignment pre-processing step and physiological data acquired during the scan (heart-rate and respiration) were included as regressors of no interest. Regressors were convolved with the default SPM8
47
canonical hemodynamic response function (HRF), its temporal derivative (TD) and dispersion derivatives (DD). The resultant parameter estimates for each regressor (at each voxel) were compared using ttests to establish the signicance of differences in activation between conditions. We have investigated ve main effects: Unstable Figures vs Stable Figures; in respect to inter categorical stimuli, Unstable Faces vs baseline and Unstable Bodies vs. Baseline; in respect to inter-categorical stimuli, Geometrical State 1 vs. Baseline and Geometrical State 2 vs Baseline. Contrast images for these effects for each subject were entered into randomeffect analyses at the second level. A conjunction analysis [Friston et al., 1999] was performed to asses how the two types of bistable images, intra-categorical and inter-categorical, each contributed to the areas of activation found when contrasting all bistable with all stabilised gures. Separate contrasts were made for each for both types of gures: Bistable intra-categorical vs. Stable intra-categorical, and Bistable inter-categorical vs. Stable inter-categorical. To make the conjunctions, the contrast Bistable Figures vs. Stable Figures was paired separately with the contrasts Bistable intra-categorical vs. Stable intra-categorical, and Bistable inter-categorical vs. Stable inter-categorical. For the even-related part of the analysis we made contrasts for inter-categorical stimuli: Bistable Faces vs. All and Bistable Bodies vs. All, and in respect to inter-categorical stimuli: Bistable state 1 vs. All and Bistable state 2 vs. All. Regressors were convolved with the default SPM8 canonical hemodynamic response function (HRF) and a rst-order Taylor approximation in terms of the temporal derivative (TD) was added [Friston et al., 1998]. Whole brain t-maps for main effects of interest and for temporal and dispersion derivatives were created for each subject. Contrast images for the main HRF and its TD were computed separately for each effect investigated. Then, second level random-effects models were created for each contrast, using the t-maps from the rst-level xed effects analysis. The onsets of internally and externally driven changes were modelled based on the recorded key presses and set to a xed duration of one second. Perception of faces and bodies as well as the two states of the intra-categorical gures were also modelled separately based on the recorded key presses. A more complex model for analysing the event-related results has been chosen; adding the TD to the canonical HRF gave us the possibility to model BOLD signals with deviations in onsets. The Henson et al. (2002) have
Conjunction analysis
Event-related analysis
48
demonstrated that it can be useful to model differential latencies of the HRF as a to investigate if BOLD signal may occur earlier or later then what the canonical parameter estimates. 2 .3 2.3.1 results Behavioural results
Blocked fMRI analysis: Neural specicity of bistable images
Conjunction analysis: comparing levels of ambiguity
All subjects reported that they were able to see the stimuli during scanning and alternate their key-presses according to the instructions. Overall, subjects perceived each type of state for a similar period of time during presentation of bistable images; bistable faces mean 2.5, 2.35 s., bistable bodies mean 2.1, 1.47 s., bistable geometrical 1 mean 2.3, 1.64 s., and bistable geometrical 2 mean 2.51, 2.29 s. Subjects indicated percept durations ranging from .02 to 15.12 s. The periods between ips found here are shorter than what subjects indicated in Kleinschmidt et al. (1998) who had inter-reversal times of 9.0 2.6 s and 8.1 1.9 s, but in line with studies of ambiguous motion where similar durations of alternating percepts were observed ([Ilg et al., 2008]). We compared spontaneous and stimulus-driven perceptual switches. We rst contrasted perception of Bistable Figures with perception of Stable Figures with blocked design analysis. Spontaneous perceptual changes were correlated with increased activations in right inferior and superior parietal lobules, and in bilateral inferior frontal, middle frontal, and insular cortex. Increased activity was also observed in regions of the anterior cingulate cortex, supplementary motor area, and left primary motor and somatosensory cortex. Selective activation during perceptual transitions were also found in the right extrastriate visual cortex and the cerebellum, putamen and thalamus (see Figures 2.6 and 2.7). A conjunction analysis was performed to provide more insight into how each type of gures contributed to the overall activation found during perception of bistable stimuli. Several similar areas of activation were identied for overall bistable perception in conjunction with both inter- and intra-categorical bistable perceptions respectively (Figures 2.8 and 2.9 ). There were clearly also differences between the two conjunctions: the intra-categorical gures evoked bilateral activation of superior parietal lobule while inter-categorical gures only showed signicant right activation of this region. The signicant regions
2.3 results
49
were also noticeably larger for the intra-categorical conjunction, both in frontal and parietal areas. We assessed if face perception was correlated with increased BOLD activation in the FFA, known as a region specically involved in face processing. We predicted that we would nd percept specic activation during face perception, which has also been demonstrated, in previous studies [Andrews et al., 2002]; [Hasson et al., 2001]. As hypothesized, our results showed clear activation in the fusiform gyrus when subjects indicated perceived faces during inter-categorical stimulus presentation, an activation pattern we did not nd in any of the other contrasts with bistable stimuli. To ensure we did not miss any activation for the contrast Bistable bodies > All, we did a small volume correction for the contrast Bistable bodies > All with a 16 mm sphere at [38 -58 -14] which was identied as the extrstriate body area (EBA) [Downing et al., 2001]; no signicant voxels were revealed. For the intra-categorical stimuli we did expect to have activations in the same areas for both Bistable state1 > All and Bistable state2 > All. Our results showed signicant activation only for Bistable state2 in right middle occipital gyrus. We did a small volume correction to test if there was possible activation in the same area for State1 with a 16 mm sphere at [36 -85 7]. The small volume search revealed a highly signicant cluster (x, y, z = 36, -82, 7, Z = 4.79). The central interest of the eventrelated part of this study was to investigate if activation would change between the two perceptual states for both inter- and intra-categorical bistable images. For the inter-categorical images there was a correlation between face perception and activation in the fusiform face area, which was not found during perception of bodies. For both states of the intra-categorical gures we identied signicant activation in the same area, the middle occipital gyrus. The middle occipital gyrus has previously been associated with spatial attention [Noesselt et al., 2002]. All eventrelated contrasts showed deactivations in primary visual regions in line with previous ndings by [Kleinschmidt et al., 1998], where deactivations in occipital areas also were found during bistable perception. The contrast estimates showed that the main canonical HRF accounted for very little of the activation (see gure 2.10 for a representative example), while the TD is much larger and positive which shows that activation actually takes place earlier than the events [Friston et al., 1998]. For the deactivations in the occipital regions the TD was negative, indicating that suppres-
Event-related results: contrasting interpretations
Latency
50
sion might take place just before upcoming perceptual switches. Also for the deactivations, the main signal modelled with the canonical HRF was much lower compared to the estimate of the TD. This could indicate that the deactivations are more transient, and may occur in the transition between perceptual states rather than in the period when the percept is temporarily stable. 2 .4
Neural specicity of bistable images
discussion
The aim of the rst part of our study was to compare spontaneous perceptual reversals with externally driven changes in terms of neural activity. In accordance with previous imaging studies using bistable percepts [Kleinschmidt et al., 1998]; [Lumer et al., 1998], we observed activations in several frontal and parietal areas. As expected we did not nd any notable activations in the primary visual system because the image was held constant. It is unlikely that our results reect the motor task, because subjects were doing the same key presses during both bistable and stabilised stimuli presentation. The frontoparietal activations could reect a continuous loop of communication from the visual cortex to higher-order areas. One interpretation is that these areas are involved in a feedback to early visual areas, re-evaluating the multistable percept over: this would support a top-down explanation. Alternatively, changes are driven by signals from lower perceptual areas due to destabilization of the current percept, in line with the satiation / bottom-up hypothesis. The role of the frontoparietal attention network would then be to detect changes or to momentarily stabilise the current interpretation. The frontal and parietal regions identied in our study are similar to the areas found during both voluntary reversals of bistable images and changes in visual spatial attention [Slotnick et al., 2003]. In the current study, it is not likely that frontoparietal regions reect voluntary changes in attention because we specically instructed the subjects not to try to voluntary elicit the ips, but to signal report reversals. The involvement of these regions in this context suggests a coupling between spatial visual attention and dynamic changes of visual perception. It is still debatable if these activations reect initiation or detection of changes in interpretation. We speculate that the brain is constantly engaged in interpretation and revaluation of perceptual input, and during this process attention is shifted to different features of the gure. These changes of attended local features could initiate switches. Our expla-
2.4 discussion
51
nation corresponds to the theoretical framework suggested by [Leopold and Logothetis, 1999], and is supported by behavioral results showing that attending to different local features of an ambiguous gure bias perception [Peterson and Gibson, 1991]; [Tsal and Kolbet, 1985]. The pre-central gyrus has found to be involved in selective processing of relevant visual targets rather than directing attention towards a cued target [Hopnger et al., 2000]. The area also contains the frontal eye elds, an area also found in the study by [Kleinschmidt et al., 1998]. In our study a xation cross was not used, so activation could be due to reversal-related changes of gaze or covert shifts of attention. Several areas in the cerebellum were identied when contrasting whole periods of bistable perception with stabilised replays. The cerebellum is traditionally associated with movement at speech function, but recent studies also show involvement in mental activities such as attention, error detection (see [Ito, 2008] for a review). It has also been demonstrated that the posterior part of cerebellum (lobule VII, crus I) supplies temporal information to frontoparietal spatial attention network involved in visual attention [OReilly et al., 2008]. Involvement of the posterior cerebellum was more active when subjects had to predict the trajectory of an occluded moving object. In our study, it may be possible that the cerebellum predicts the upcoming dominant visual percept. In the conjunction analysis, larger parts of the overall activations during bistable perception were accounted for by activation during viewing of intra-categorical gures. There may be several ways to interpret this nding. Our initial hypothesis, in line with [Zeki, 2004], was that reversing gures represent a very simple form of ambiguity and are not affected by top-down factors, while face/body gures should trigger an activation in frontal areas (top-down). Zekis hypothesis seems to contrast the fact that we found larger areas of frontal and parietal activation during perception of intra-categorical gures. However, there may also be other explanations. If, indeed, we are less able to control alternations of intracategorical gures, and the frontoparietal network reects detection of changes in perception, then increased activity could be due to less anticipation of changes. In our study, several subjects spontaneously reported that they were able to see both gures at the same time for the inter-categorical gures, while this was not the case for the intra-categorical ones. This implies that inter-categorical are not "purely" bistable, then they are
Levels of ambiguity
52
Contrasting interpretations
less effective as stimuli and that could also account for why we found less ambiguity-specic activation. In order to check this hypothesis, after the experiment we asked subjects whether they had the impression they could see both images at the same time in geometrical stimuli and in face/body stimuli. while only 12% of the subjects said they could perceive geometrical (intracategorical) two interpretations at the same time, 56 % said they had the impression they could see both face and body in the intra-categorical images. Therefore in terms of ambiguities level, we afrm that intra-categorical images belong to second level ambiguity. For the event-related analysis we hypothesized that activation would take place within the same brain area for both states of the intra-categorical stimuli, while activation would change back and forth between two areas during perception of inter-categorical stimuli. As expected, we did nd increased activation in the same area for both states of the intra-categorical stimuli. For the inter-categorical stimuli, we did nd activation in FFA during face perception in line with previous studies [Andrews et al., 2002]; [Hasson et al., 2001]; [Tong et al., 1998]. This supports the general idea that the conscious experience of a gure is correlated with activation in its processing specic area. A cube will remain a cube even if it is seen from a different perspective, and therefore processing takes place within the same area. Activation for one state was only found using a small volume correction. A weakness of this study is that it was difcult for subjects to report which was the dominant interpretation of intra-categorical stimuli, while it was much easier to assign one button to perception of faces and one to perception of bodies for the inter-categorical gures. In further studies, our ndings could be supplemented with a similar experiment using ambiguous motion in the future because direction of movement is easier to report than perspective. The TD of the HRF modelling perception of faces indicates that activation in FFA actually occur before the reversal is consciously perceived. This nding is in line with the study by [Andrews et al., 2002] showing that FFA activation can be detected before subjects report seeing a face when looking at the Rubin gure. Also transient deactivations were found in occipital areas, most consistently in the lingual gyrus. This area has been shown to be sensitive to direction of spatial attention [Giesbrecht et al., 2003], and similar patterns of deactivation have been found during inhibition of unattended visual stimuli [Hopnger et al., 2000]; [Slotnick et al., 2003].
2.4 discussion
53
2.4.1
Conclusion
We addressed the general question of ambiguous perception in three different ways. We compared endogenous with exogenous changes by contrasting perception of bistable gures with a replay condition using stable gures. With our results, we support the hypothesis that frontoparietal processes constantly re-evaluate the current interpretation of the sensory stimulus input causing changes in subjective perception. The neural processes underlying spatial attention and perception of ambiguous stimuli appear to be similar, but the exact role of attention needs to be further investigated. In general, endogenously driven perceptual changes seem to involve widely distributed brain areas. Further, we investigated neural correlates of different levels of ambiguity. Surprisingly, intra-categorical stimuli evoked a larger magnitude frontal and parietal activation than the intercategorical gures. The implication of this nding is that it may be useful to distinguish between different types and levels of ambiguity. We propose that in rst level ambiguity frontoparietal activations are more signicant and this fact correlates with a subjective experience of a more "pure" perception of bistability, implying the mutual exclusiveness of the stimulus interpretation. Second level ambiguity, when activations take place in two or more different brain areas, may correlate with the coexistence of interpretations and with a diminished activity in the fronto-parietal network, which appears to be an ambiguity-specic neural activity. Our ndings could further be expanded to involve cognitive multistability, such as more open-ended ambiguities, referring to gender or racial ambiguities [Chiu et al., 2011], situation-related ambiguity [Zeki, 2004] or, as we are going to discuss in the next chapter, the illusion of life. We also found support for the hypothesis that for rst level ambiguity, activation takes places within one brain area. During perception of ambiguous gures changing between different categories, neural activity uctuates between two or more processing areas. Our study conrms that activation changes in processing areas are correlated with the each stable experienced percept over time.
54
Figure 2.4: Experiment stimuli: bistable intra-categorical images + stabilized versions
2.4 discussion
55
Figure 2.5: Experiment stimuli: bistable inter-categorical images + stabilized versions
56
Figure 2.6: Bistable Figures > Stable Figures
2.4 discussion
57
Figure 2.7: Global 3D view of activations for the contrast internal change > external change for a random effects analysis with 16 subjects: selected activations superimposed on to averaged anatomical sections
Figure 2.8: T statistic for Bistable > Stable in conjunction with Intracategorical: Bistable > Stable switches (left) and Intercategorical: Bistable > Stable switches (right)
58
Figure 2.9: Bistable activations conjoined with Inter-categorical and intra-categorical bistable activations
2.4 discussion
59
Figure 2.10: Bistable faces > All. FFA activation projected onto averaged structural scans (left) and main HRF and TD plotted for FFA [38 -58 -14] (right)
Part III THIRD LEVEL AMBIGUITY
T H I R D L E V E L A M B I G U I T Y: T H E I L L U S I O N O F LIFE
Chapter II has illustrated how and where in the brain the illusion of ambiguity takes place. Introducing the concept of levels of ambiguity correlating with fronto-parietal activations and coexistence, at the conscious level, of different possible interpretations. In this chapter I propose the concept of life perception as an illusion, the illusion of life. This specic illusion results in the qualia of attributing aliveness, and animacy attributes such as intentions and emotions to objects, in spite of the certain knowledge that such objects are not alive. This illusion could descend by the specic knowledge of behaviour and appearance of alive entities; but it is so strong an so compelling that suggests that animacy perception is due to a perceptual knowledge, or an inherited concept, that is present in our mind and organises our perceptions even when we face blatantly non-animated stimuli, like geometrical gures. This illusion may represent a higher (third) level of ambiguity, in line with our results of Chapter II. Such results demonstrate that ambiguity-related activations are more likely to correlate with intra-categorical images, that are also the most decidedly "mutually exclusive" interpretations. Inter-categorical images are more likely to be less exclusive. As the illusion of life represents a higher level ambiguity, the observer will attribute aliveness-related features to objects while continuously maintaining the awareness of their non-aliveness. The illusion of life is so strong and compelling that many studies reported the impossibility of preventing the subjects of anthropomorphizing stimuli [Schultz et al., 2004]. [Castelli et al., 2000] pointed out that several subjects were anthropomorphising also random-moving stimuli. 3 .1 perceptual knowledge of life
Automaticity of illusions
Attributes and features of animacy can elicit in us strong automatic reactions that may be classed under that label of "perceptual knowledge" which persists in qualia and can contradict conceptual knowledge in illusions. The ability of perceiving animacy, being obviously useful and especially robust, allows
63
64
third level ambiguity : the illusion of life
The importance of life perception
us not only to immediately spot all "living beings" surrounding us, but also to attribute intentions, emotions, thoughts, and a theory of mind to them. The mechanism of "life perception" is so powerful that it invariably leads us to assess some "false positives", hence the illusion of life: to perceive inanimate objects as animate, to ascribe them life-related attributes (e.g. intentions, awareness, perceptions), and to experience towards them feelings such as empathy and affection. The false positives caused by the efciency of "life perceptions mechanisms" are always experienced as ambiguous, since they conate cognitive awareness of "non-aliveness" with partial perception (qualia) of "aliveness" (e.g. attribution of personality, feeling of sympathy). The nature of the life perception as a crucial specic knowledge causes its manifestation as illusion of life in ambiguity-generating situations. The evolutionary necessity of spotting alive creatures is mirrored in our perceptual ability to identify biological movements, which is amazingly effective. This was rst demonstrated in classical experiments by [Johansson, 1973], who showed that a few dots of light placed strategically on a moving human or animal body are instantaneously organized into the coherent percept of a living creature. [Baron-Cohen, 1995] denes a partial aspect of life perception as the ID: the Intentionality Detector. ID is a "perceptual device that interprets motion stimuli in terms of the primitive volitional mental states of goal and desire", thus attributing intentionality to anything with (apparently) self-propelled motion. ID is supposed to be a very basic function, working through sensory modalities. It is based on [Premack, 1990] argument that infants distinguish between two kinds of objects, those that are and those that are not self-propelled. Premack follows research lines by [Leslie and Keeble, 1987] that describe the "appropriate stimulation" (the percept needed to elicit the concept of causality) as temporal and spatial contiguity between appropriate events. Applying and extending this argument, Premack afrms that the brain deduces basic assumptions from motion features, inferring the principles of causality, intention and reciprocation. Premack follows the hypothesis that the perception of intention and causality are "hard-wired perception based not on repeated experience but on appropriate stimulation". It is not motion itself that is critical but change: a switch from rest to motion (or vice versa), and from one speed/direction to another; "just as causality is the infants principal hard-wired perception for
3.1 perceptual knowledge of life
65
non self-propelled objects, so intention is its principal hardwired perception for self-propelled objects". [Mandler, 1992] also attempts to investigate what distinguishes the "animate" from the "inanimate", arguing that this is one of earliest concepts formed in early development stage, and attributing its perceptual "gateway" to the analysis of different kinds of motion. Causal perception through motion is a perceptual mechanism rst studied by [Michotte, 1946]. Causal perception is what allows us to "attribute" cause-and-effect relations to facts; this concept is so strong that we attribute causality and intentions even to bi-dimensional moving shapes moving schematically, by describing their motions with intentionality verbs. All these results demonstrate that hard-wired brain functions related to "life perception" would lead to "false positives" provoking attribution of intentions and goals to inanimate objects or even shapes. The classical demonstration of this mechanism is a study by [Heider and Simmel, 1944], where subjects were asked to describe a lm in which geometric shapes moved. The subjects tended to ascribe intentionality and causality to the shapes themselves or to construct narratives where squares and triangles act as human characters. From the neurobiological point of view, Warrington and Shallice [Warrington and Shallice, 1984] led a seminal study on four patients struck by herpes simplex encephalitis. In all patients, researchers identied a specic difculty in identifying visually and verbally - living things more than inanimate objects. This kind impairment has been dened as "category-specic semantic decit". Since then, many studies have been carried out on patients investigating the animate / inanimate categories as issues concerning the organisation of conceptual knowledge in the brain. For a complete review of the approaches to this debate, see [Caramazza and Mahon, 2006] where it is suggested that the domains in which conceptual knowledge regarding this issue are organised are living animate, living inanimate, conspecics and tools. Inside each of these domains, the authors suggest that there would be different mechanisms for the analysis of visual form, visual motion, and conceptual knowledge, as well the attribution of intentional content. From the biological perspective, a "perceptual life detector" has been proposed in relation with biological motion perception in animals [Johnson, 2006]. Experiments with scrambled point light displays gave results that the researchers interpreted as evidence of a visual lter "tuned" to motion of the limbs of an animal in locomotion,
Animacy perception as a category
66
thus functioning as a general detection system for articulated terrestrial animals. 3 .2 the illusion of intentionality
Intentionality in the social brain
Seminal studies on the illusion of life
As discovered by Johansson, biological motion perception is exceptionally robust in humans (and animals); but, a feature allowing of the illusion of life is that some movements are perceived as animate even if what is moving is a blob, dots or geometrical objects. Motion perception is central to life perception; indeed, motion-related information can not only provide the observers with the feeling of animacy but also cause attribution of roles, goals, intention, personality and emotions. As a common characteristic of illusion, (see Chapter I, about the hollow face perception), also the illusion of life triggered by motion information is perfectly compatible with the awareness of its deceptive nature. When viewers make "inferences" ascribing animacy to moving geometric shape, they do it being fully aware that what they are seeing are geometric shapes. Nevertheless, the illusion is so compelling that it is impossible not to ascribe animacy to these shapes. Such an automaticity, once again, typical of illusions, is a nding that may be interpreted against a pure top-down framework of analysis. Activations correlating with animacy attribution of non-alive stimuli are surprisingly coincident with neural activations relative to perception of other individuals: the temporal parietal junction (TPJ), the FFA and both ventral and dorsal medial prefrontal cortex (VMPC and DMPC). This areas constitutes what is known as the "social brain" [Brothers, 1990], [Skuse et al., 2003]; [Adolphs, 2003] because it is generally activated during social tasks. [Michotte, 1946] and [Heider and Simmel, 1944] where the rst to investigate how do motion features could lead to animacy attribution, working in the same years but separately. Michotte with his experiment demonstrated that different features of motion can be interpreted as physically caused while other as psychologically caused. For instance, if the surface of moving objects A comes to contact with the surface of object B, and B immediately starts moving, the event is interpreted by all the viewers as a transfer of physical motion. But if B moves not immediately but after a pause, the attribution of animacy becomes compelling and the motion is interpreted as psychologically-caused: object B moves because object A told it to move or it tries to escape from A etc. Heider and Simmel (1946) have been working on a 2.5 minutes
3.2 the illusion of intentionality
67
movie. The video casts three geometrical shapes, two triangles (a small one and a big one) and a circle. The three shapes were moving around, in and out a rectangular perimeter, displaying different motion features. During the experiment, subjects were requested to describe what happened in the video; with very few exceptions, the subjects were reporting a narrative including attribution of intentions, goals, social relations, gender. For example, the circle is almost always interpreted as an aggressive man and the two triangles as a couple trying to avoid him or escape him, while the perimeter is interpreted as a building. Other studies [for a review see (Scholl and Tremoulet, 2000)] have been following this rst experiments, trying to identify motion features of the Michotte tradition and theory of mind issues for Heider and Simmel. In the Michotte tradition, studies on a single moving object have singled out some features linking motion to animacy attribution demonstrated that self-propelled motion is crucial [Stewart, 1982]; [Tremoulet and Feldman, 2000]; also, environmental features are important, since animacy can be seen as emerging from an interaction with the context, as well as changes in velocity and directions. Studies containing two or more moving object seems to be more compelling in eliciting the illusion of life, probably because objects constitute environment and create interaction for each others. [Bassili, 1976] demonstrated that timing is crucial for perception of animacy, since temporal events suggest whether the objects are interacting (and therefore animated) or not; and time contingencies would inuence the psychological interpretation. When motion paths and timing coincided with each other an object was interpreted as "leader" and another one as the "follower", and especially the following object was eliciting intentionality attribution, but when motion paths were random, the impression of agency was lessened [Dittrich and Lea, 1994] Nevertheless, intentionality does not correspond to animacy, which is a propriety on its own, but contributes to create the illusion of animacy. This is demonstrated by an experiment by [Opfer, 2002] which designed two sets of blobs, identical in shape - but one was autonomously moving whilst the other one was "chasing" objects; the rates of animacy were higher for the intentionality provided blob. [Blythe et al., 1999] studied the interaction possibilities implied by two objects moving in pair with each other; objects were simple computer cursors "moved" by participants to the experiment whose had roles to interpret (chase, court, follow). An algorithm based on a very few motion parameters could successfully cate-
68
The role of motion
The obligatory nature of illusions
gorise the trials, suggesting that not only intentionality but also psychological content can be delivered with very few motion clues. Affective content can be delivered also by motion clues, as demonstrated by [Rime et al., 1985] who found a very high consensus between US and European subjects while dening the emotions conveyed by geometrical shapes such as the ones used by Michotte. What is interesting here is that the consensus was remarkably lower when the shapes were substituted by human silhouettes. Minimal information seems to be more adequate to elicit the illusion of life, and especially robust in the case of stimuli like point light displays, and even just limbs [Troscianko et al., 1996]. Summing up, literature demonstrates that autonomous motion, is quite a good way to elicit animacy illusion, but less then goal directed motion showing an interaction with the environment or between two objects. The illusion of life is so compelling since based on strong perceptual clues that this mechanisms are valid not only for human adults but also for children and babies. Infant under 1 year can categorise goal-directed shapes. Furthermore, motion as a clue of intentionality and animacy is stronger when "pure" and not associated with other information like form (even if the form openly leads to anthropomorphism). This suggest that intention attribution or agency may be hardwired in the brain [Scholl and Tremoulet, 2000]; [Tremoulet and Feldman, 2000]. Slightly more complex movement can go furher and convey not only intention but emotions and mental content, as in te study of [Blythe et al., 1999]. Another clue indicating that form is not useful if not counter-productive is that attribution of intentions and psychological content to Heider-like objects is due to movements features and changes of location over time, but not to the shapes themselves. Other version of the Heider movie has been produced, in which movement has been disrupted, causing the absence of illusion in the description of the scenes; whilst in a version in which the forms where changed the description remained the same as with the original movie (Berry et al, 1992). The most striking feature of the illusion of life is its compulsory nature, as we said above, typical of illusions. Despite knowing that we are watching geometrical shapes, the urge to attribute them animacy features is compelling. This "qualia" of "obligatory" perception has been studied by [Hashimoto, 1966] who has been working with a group of observers looking to a Heider-like movie. He told the subjects that they had to describe
3.3 the illusion of life in the brain
69
the shape moving by keeping in mind that they were just looking at geometrical objects and thus avoiding any reference to animacy. The subjects were nevertheless describing the movie with anthropomorphic terms "leaking" out of the descriptions, and this has been the rst demonstration of the obligatory nature of the illusion of life. Following Hashimotos study, other researchers have been investigating such a compelling impression. Other methods used have been counting in the descriptions the words depicting social roles, affective content, and a specic analysis tool - the Linguistic Inquiry and Word Count [Heberlein, 2008], organising in 74 categories the references to animacy in the description of the stimuli. Using this software, Heberlein and colleagues found that, even explicitly forbidding subjects to describe in anthropomorphic terms a Heider-like movie, the percept of intentionality is powerful enough to leak through the verbal descriptions: all the subjects failed in describing the shapes without using animacy-related terms. They also found that the rate of speech in subjects forbidden to describe the movie in anthropomorphic terms was signicantly slower then the control group who describing the scene without any instruction; these data demonstrate the effort in overriding the percepts of intentionality, supporting the hypothesis that intentionality and animacy are hard-wired being inherited concepts in the brain. 3 .3 the illusion of life in the brain
fMRI studies have been investigating the neural events correlating with the attribution of intentions and perception of animacy. As anticipated at the beginning of the Chapter, many structures that appear active when perceiving stimuli eliciting the illusion of life seems to be also involved in perceiving social facts, and indeed the active circuit has been labelled "social brain". These brain areas involve the posterior sperior temporal sulcus (pSTS), the amygdala, the fusiform gyrus, the ventral and medial prefrontal cortices, the tempo-parietal junction. The social brain also include this areas, as proposed by [Brothers, 1990] who dened as a circuit specialised in making inferences about behaviour (mental states and predictions of actions based on them). fMRI studies have been obtaining such a brain map by scanning subjects while they were looking or describing Heidder-like or Michotte-like movies. Generally, this movies were depicting shapes animated by intentions, but animacy, as specied in the
70
Superior temporal sulcus
Other activations
previous section, does not overlap with animacy (but it is a very good cue for it). Animacy correlates have been isolated with a contrast analysis from perception of intentions attribution by showing movies of goal directed motions versus physically random [Schultz et al., 2004], physically but not psychologically caused [Blakemore et al., 2003] or geometrical motions [Castelli et al., 2000]. [Schultz et al., 2005] have been designing an experiment isolating perception of movement from animacy perception. The stimuli used were animations of two autonomously propelled discs, in which, periodically, one "chases" or "follows" the other one. By continuously varying the disc movements, alternating between interactive and non-interactive, the analysis could grasp the perception of interaction isolating it from the perception of movement. In the moments in which the movement was interactive (when one disc was chasing the other one) the stimuli were perceived as more animated. When subjects were indeed observing interactive motions, bilateral regions of posterior STS/STG were found to be activated in contrast with the non-interactive (non correlated) motions. This area was active even when the task given to subjects was meant to distract their attention from the perceived animacy of the objects, for instance, when the subjects were told to judge velocity or other features irrelevant to animacy. This means that this brain area processes animacy even in absence of conscious attention devoted to it; nevertheless, when attention is elicited towards animacy (when subjects are instructed to observe the interaction between the shapes, or to try to "interpret" the casing strategy of one object) activity in the posterior STS/STG is even increased [Blakemore et al., 2003]; [Schultz et al., 2004]. This suggests that the pSTS/STG is the area specialised for perceiving animacy, as the same area is active is animated in correlation with the event of spotting biological motion while viewing a point light display stimulus [Zacks et al., 2006]. Researchers have been trying to identify neural correlates of animacy perception. [Castelli et al., 2000] compared activations of subjects viewing three kinds of Heider-like stimuli: random movements, goal-directed movements and movements triggering psychological content (e. g. seducing, bullying, etc). Four regions, pSTS regions, medial prefrontal cortex, fusiform gyrus and extrastriate regions of the lateral occipital cortex were more active looking at goal-oriented movements (chasing, following) and in the high-psychological content movements if compared
71
to random moving stimuli. [Martin and Weisberg, 2003] have found a consistent map of activations viewing animated actions compared to objects. In a study showing thre geometrical objects interacting, subjects were asked to judge whether the objects were friends (thus imposing a priori an attribution of animacy) [Schultz et al., 2004] and they showed activations, if compared to non animated stimuli of which subjects had to evaluate the weight, in the fusiform gyrus, amygdala temporal pole, medial PFC and STS. Particular activations as been found in the FFA, after a localization. This activation of FFA in stimuli without faces might mean that either the FFA is included in social judgements even whether stimuli have no faces, or that the FFA might by potentiated by inputs from the amygdala which as we saw may be implicated in processing social stimuli. This latter interpretation is reinforced by a damage study [Heberlein and Adolphs, 2004] on a subject with a bilateral amygdalal damage, who was describing in totally non-anthropomorphic terms the original movie by Heider and Simmel. The amygdala is known for processing emotional content, in particular processing fear and rating trustworthiness. The amygdala may be relevant for directing attention towards socially relevant stimuli, since it projects to frontal and temporal regions; thus the amygdala may project signals to pSTS, prefrontal regions and FFA. pSTS has been considered as related to animacy perception and among that area TPJ has been isolated among other regions for inference about others people mind and intentions in contrast with others representations [Saxe and Kanwischer, 2003]. Also mPFC has been found to be active during representations of mind [Amodio and Frith, 2006], but bilateral damage does not affect intention attribution [Bird et al., 2004]. Studies did not give any denitive answer, but mPFC seems to be important for processing people-related information. [Wheatley and Martin, 2009] have been addressing the question whether the illusion of life (which they refer to as anthropomorphising) is due to a top-down or bottom-up mechanism. The same set of moving geometrical objects was duplicated and associated with two different context, one encouraging and the other one discouraging animacy attribution. Brain regions active in the animacy-stabilised versions of the stimuli were once again overlapping with the social brain: fusifom gyrus, STS, amygdala, insula, mPFC, thus overlapping with the regions active during attribution of animacy to simple geometrical gures without
The role of amygdala and pSTS
Top-Down versus Bottom-up
72
biasing for anthropomorphising, and also when subjects were asked to imagine anthropomorphic attributions. So apparently top down information inuences the activations of these regions, and not just perceptual information, in a hybrid mechanism as for rst and second level ambiguity. 3.3.1 Emotional clues in the illusion of life
Brain damage preventing the illusion of life
Negative bias
Following neuroscience, then, what are the features allowing the illusion of life, and thus the activation of the social brains circuit? Goal-directedness alone is not enough, as suggested by [Blakemore et al., 2003] and [Schultz et al., 2004] and [Schultz et al., 2005]. Contextual clues also are not crucial, since a very simple and contextual-free stimulus as the Heider and Simmel movie is undoubtedly apt to trigger the illusion. On the contrary, the original movie features several interactions between the shapes (ghting with, escaping and hiding from, protecting and chasing each other). The fact that a bilateral brain damage at the amygdala prevents from anthropomorphizing the Heider and Simmel movie [Heberlein and Adolphs, 2004] is an important clue of what can triggers anthropomorphism. The Heider and Simmel movie contains emotionally-charged interactions, since it is interpreted as a story of bullying or threatening. It is very interesting to observe that the emotions observed in the movie are negatives ones. In the movie, the two triangles are bullied, or persecuted, or chased, by the bigger shape; there is an "happy end", since they gently touch each other in a behaviour described as "kissing", "celebrating", "high-ving" by the subjects. But, this last "event" is seldom included in the description of the movie, while the most negative events are always explicitly described, as the "destruction of the house" (the rectangle). Negative events, and thus negative emotions, seems to be more important and drive more attention, and they are the one which trigger the social interpretation of the stimulus, thus the illusion of life. In this sense, the amygdala damage preventing anthropomorphism is very meaningful since as specied above the amygdala processes emotional information and especially fear and threat-related clues. The negative bias in the illusion of life has been suggested also by [Morewedge, 2009] who showed that a negative intention or outcome is more easily interpreted as human. During an ultimatum game, subjects were asked to judge wheter on the "other side" there was a human or a computer. If confronted with negative behaviours, such a
73
selsh ultimatum, the subjects would generally rate the agent as human, while if confronted with a generous offer, the agent would have been rated as a computer. Negativity bias seems to be linked with the evolutionary necessity of dealing with unpredictable and potentially harmful living entities, other humans or animals; thus, we appear to be more inclined to attribute social attribution to negative events. It is a necessity which gives rise to false-positives, that from an evolutionist point of view are less harmful than negatives.
Part IV F O RT H L E V E L A M B I G U I T Y
AESTHETICS OF ANTHROPOMORPHISM
4.0.2
Anthropomorphism as a forth level ambiguity
The illusion of life is probably the perceptual basis of those common and inescapable phenomena of anthropomorphism, that all of us experienced. Our capacity to invent living agents and superimpose this belief on objects that are clearly non animated is impressive: we name our cars, curse computers and love gadgets. I consider the illusion of life as the perceptual knowledge leading to inference of aliveness. In this Chapter this psychological phenomenon is declined in life evocation in the arts and anthropomorphism in human-machine interaction. Anthropomorphism includes the illusion of life, since it represents a process of inference allowing people imbue the real or imagined behaviour of other agents with human-like characteristics, motivations, intentions, or underlying mental states [Epley et al., 2007]. While the illusion of life is universally perceived, with the only exception of brain damaged patients, life evocation and anthropomorphism are variable depending on bias. Indeed, some agents are more easily anthropomorphised than others, some cultures are more likely to anthropomorphise than others [Asquith, 1986], children anthropomorphise more than adults [Carey, 1985] and some situations and needs lead to anthropomorphisation [Epley et al., 2008]. On the basis of such an individual variability, anthropomorphism can be dened as a forth level ambiguity, being a conation of the awareness of non-animacy and qualia of animacy, but which is not always and automatically perceived, as in the case of the illusions we explored in the previous chapters. 4.0.3 Variability in anthropomorphisation
Research has outlined that different people react in a different way to anthropomorphisation; research has just begun to explain and predict the variability of this phenomenon [Waytz et al., 2010a]. [Epley et al., 2007] identied three primary determinants explaining anthropomorphism. The rst is motivation for social connection (in a situation of loneliness); experimentally induced
77
78
aesthetics of anthropomorphism
The need for anthropomorphism
Ambiguity and art
isolation increased tendency to anthropomoprhisation [Epley et al., 2008]. The second is the need to master our environment. Anthropomorphising a stimulus makes it seem more predictable and understandable, demonstrating that anthropomorphism is increased by effectance motivation. Effectance is the possibility to attribute predictability to "behaving" things allow us to have the impression to understand them easily, and to familiarise, having thus the perception of increased mastery, with the nal goal of making sense of an otherwise uncertain environment. Research on experimental psychology has demonstrated that the experience of unpredictability stimulates attempts to gain mastery [Berlyne, 1962]; [Whitson, 2008]. A study by [Waytz et al., 2010b] demonstrated that unpredictable and unexpected behaviour triggers the motivation to understand and explain it. Life evocation has long been exploited by artists, and anthropomorphism has a great potential of application in todays society, where we interact more and more with technological objects such as robots and avatars, implicitly designed for embodying believable creatures. Identifying features of believability, and drawing examples and inspiration from arts and technology could lead to insights useful both in terms of design and in cognitive studies. Yantis identied an analogy between ambiguity and artistic phenomena [Yevin, 2000] proposing a modelisation of perceptual ambiguity (proposing the model of saturation of attention). Ambiguities have been exploited in artworks such as paintings by Dali. The most famous example of artistic ambiguity is the Mona Lisa, in the words of Gombrich: "Even in photographs of the picture we experience this strange effect, but in front of the original in the Paris Louvre it is almost uncanny. Sometimes she seems to mock at us, and then again we seem to catch something like sadness in her smile" [Gombrich, 1995]. In the same perspective, Zeki makes reference to Vermeers painting as featuring a higher level ambiguity [Zeki, 2004], where ambiguity displayed here refers to ambiguity of different situations, that can can be interpreted in many different ways. 4 .1 4.1.1 life evocation in the arts Dening life away
The attribution of intention to shapes, motion features and emotional clues are all involved in the illusion of life. But, what is "life"? Even if the answer seems to be intuitive and straightfor-
4.1 life evocation in the arts
79
ward, researchers propose that we lack of an adequate theoretical framework [Cleland and Chyba, 2002]. As Cleland and Chyba argument, the possibility of dene life is now facing the same difculties hindering the denition of "water" before the invention of molecular theory, since we still lack a "theory of biology that allows us to attain a deep understanding of the nature of life and formulate a precise theoretical identity for life comparable to the statement water is H20. The distinction between life and non-life has also been a central issue discussed by Articial Life researcher. The so-called "strong thesis of Articial Life" has also argued that it would be possible to create "life" in some other medium, abstracting the essence of life from the details of its implementation in any particular model, trying to build models that are so life-like that they cease to become models of life and become examples of life themselves. Yet, despite the difculties in agreeing upon a scientic denition of life, everybody knows what "being alive" means in the "ordinary sense"; the difculties lie in "dening it away", to say reducing it to its "independently intelligible properties". Even if there is no consensus about how to "dene life away", our perception of something as alive can be elicited by pressing some Darwinian button able to evoke in us a visceral reaction. Life evocation arises when we are confronted with perceptions that we know are deceptive, but we feel as true and real. 4.1.2 Art and the illusion of life
What is life?
One of the main objectives of art has always been to represent and evoke life. Many myths illustrate this idea, beginning from the one about the origin of the rst painting, as narrated by Pliny: the rst painting was actually a silhouette draw at candlelight by a "Corinthian Maid". She traced the features of her sleeping lover onto a wall. He was about to leave for a far away country, and she traced this image in order to keep his presence with her. Another meaningful story is the one of Giotto. Vasari recounts that when Giotto was only a boy in the studio of his master Cimabue, "he once painted a y on the nose of a face that Cimabue had drawn, so naturally that the master, returning to his work, tried more than once to drive it away with his hand, thinking it was real"[De Vere, 1912-1915]. Realism and accuracy were intended as the tool for not only representing life, but to attribute to the work of art the ideal feature of communicating the feeling of life-like presence, or life evocation. Several other
Art imitating life
80
stories establish a relation between the realism of life representation creating the "illusion" of life with aesthetic appreciation and artistic value. In literature, many histories portraying artists and focusing on creativity and inspiration stage the ambiguity between the inanimate and the animate. Typically, the topoi of art-related literature include works of art awakening to life, the will of creating life by means of art, the uncertain nature of a character (animate or inanimate, statue or human being). The most celebrated example of the rst category is the Portrait of Dorian Gray, in which the inanimate object (the portrait) absorbs some specic characteristic of its animate counterpart (Dorian), to say moral traits, embodying them through formal features. The second category probably stages a common metaphor for artistic inspiration and creation, ranging from the carpenter who carved Pinocchio to Pygmalion giving life to his loved statue. The third case, more specically focused on ambiguity, is the narrative expedient often used for arousing the feeling of uncanniness, thus used in fantastic and horror ction, leaving the reader or the main character in uncertainty as to whether a work of art is animate or not. Examples of this mechanism are Colomba by Merime (1840), or A Mystery of the Campagna, (1887) by Crawford, or the most famous Sandman (1816) by ETA Hoffman. Another art exposing the ambiguity of life evocation is puppetry, a literal enactment of the "animate versus inanimate" contradiction. The main characteristic of puppetry is the "inescapable tension" that Steve Tillis identies as being between the material object itself and the object as "signier of life" [Tillis, 1996], setting up a conict between the puppet as object and the puppet as life. Indeed, a marionette elicits a double point of view on the spectator: it is an object but one onto which the viewer projects her own emotions and a theory of mind. When attending a puppet show, the spectator is drawn, little by little, towards increasing her suspension of disbelief, nally granting the puppet the status of an actor. Puppetry imagery in contemporary art takes, as historic point of departure, Alfred Jarrys 1896 puppet play Ubu Roi. Later, other puppets were featured in works from international well-established artists (such as Kiki Smith, Pierre Huyghe, Christian Jankowski, Kara Walker, Laurie Simmons). [Cohen, 2006] identies a rst wave of puppet imagery appearing in avant-garde art coinciding with the Western appropriation of masks and other artefacts from exotic culture and folk art. He specically refers to the Bauhaus and
81
Futurist artists Jan Toorop and Paul Klee who created abstract puppet spectacles using geometric gures. In this period Picasso, Cocteau, and Calder created mobile sculptures and puppets; and Joan Miro designed an experimental puppet show, Death to the Bogeyman (Mori el Merma, 1978), with monstrous painted body puppets drawn from Ubu Roi. From where does this artistic interest in puppetry stem? Puppets challenge the audiences understanding of object and life, and question a complex relation with acting, non-living beings. After the advent of photography, representation has not being anymore the central objective of art practices. Assuming an auto-reexive attitude, art has been exploring other domains, inventing a newer aesthetics including the spectator, aware of him-or-herself existing in the same space as the work, and being aware of the perceptual relationships that are established. Cultural trends began to steer art theory and practice towards concepts of interaction and perception at the end of the 50s. In 1957 Marcel Duchamp delivered a key lecture, The Creative Act, in which he argued that "the work of art is not performed by the artist alone", since "ce sont les regardeurs qui font les tableaux" [Sanouillet, 1973]. It was the dismissal of the Modernist conception of the art objects internal self-sufciency in favour of a sense of its dependence on contingent, external factors such as context and audience participation. How could artists translate the traditional objective of live evocation in this new challenging and changing phase of art development? Many critics rejected the idea that art should mimic life, and many artistic trends dismissed this ambition, exploring other visual languages, such as abstraction. Especially in modernist aesthetics, the repudiation of anthropomorphism was radical. It was in favour of medium specicity, autonomy of the artwork from its environment, and rejected narrative. Nevertheless, life evocation is such a powerful tool - both in terms of human perception and artistic expression - that, even if abandoned in theory, it reemerged very soon in practice. Art evocation re-emerged in contemporary art practices after Modernism, beginning with Minimalism and continuing with technological and in particular robotic art. These artistic explorations continued the trajectory of anthropomorphism in Western art, but switching their focus from representation of formal features to simulation of behaviour as a form of life evocation in parallel with a greater attention to elicited perceptual processes. Minimalists have delivered anthropomorphic and mimetic content in their works, not by means of representing
The return of anthropomorphism in modern art
82
life in a human form, but rather by simulating the feeling of presence and interaction. This peculiarity of minimalism has been paradoxically pointed out by a modernist critic, M. Fried, who analysed minimalist works specically attacking their tendency of being "a kind of statue; a surrogate person". The basic switch of sensibility identied by Fried lays on the fact that minimalist art produces "objects in situation" which "[occupy] a position in the world" [Fried, 1967]. These works interlace relations with the spectators, losing the frame and separation from the surrounding world (as modernist aesthetics would), embracing objecthood and acknowledging audience, being concerned with "the actual circumstances in which the beholder encounters the work" and, eventually, concealing at the core of its theory and practice "hidden naturalism, indeed anthropomorphism". Robert Morris, a minimalist artist, conrms these features (of course from a positive perspective) in "Notes on Sculpture" [Morris, 1969-1969], in which he argues that the art objects should be designed for triggering physical participation by the visitor. In this way, the spectator is directly led towards her own perceptual activity which is, by the same token, revealed and disclosed. The "economy of tools" (to which is due the label "minimalism") is precisely intended to reveal the artwork as a relational system by means of a reductionist approach: the work is conceived starting from elementary parameters (light, shape, colour, size) susceptible to entertain continuous changing relations with the spectator, whose "task" is the physical exploration of the artwork. In Frieds words, the work "depends on the beholder, is incomplete without him, it has been waiting for him. And once he is in the room the work refuses, obstinately, to let him alone - which is to say, it refuses to stop confronting him, distancing him, isolating him". This use of intentional verbs is very interesting: indeed, being confronted with a minimalist object "is not entirely unlike [being] distanced, or crowded, by the silent presence of another person". But, how it is possible that minimalist works - utterly simple, geometric 3D objects, such as cubes or parallelepipeds - would elicit in the spectator such an inescapable life evocation perception? Writings by minimalist artist Tony Smiths may explain this perceptual phenomenon. In minimalism, an important feature for eliciting the impression of being confronted by "something alive" probably lies in the size of the work: "(not too big, or it would become a monument) nor too small (it would become an object)" as Smiths claims about his six-foot cube, "Die". Minimalists have also been designing works
83
that are "unitary and wholistic shapes", having the perceptual property of suggesting an inside: two openly anthropomorphic features. These formal characteristics (an appropriate size, a unitary shape) have always being part of Western traditional sculpture; but only with Minimalism, which reduced life evocation to minimal terms expressive tools, could they be singled out as tactics for eliciting the specic ambiguous perception of live evocation in the brain of the spectator. 4.1.3 From art to technology
Art addresses questions on how we view, perceive and interact with our surroundings, changing and evolving in a continuous dialogue with the development of culture and technology. After the advent of photography and thanks to the advances of technology in elds such as robotics and computer graphics, one of the objectives of art practices has become not only to represent life, but also to simulate it by creating articial creatures crossing category boundaries, and questioning from the cognitive point of view which features can evoke in us life perception. These artworks suggest that there is no such a thing as a clear and discrete gap in our perception between animate creatures and inanimate objects, but rather a continuous category. In their paradoxical status of quasi-living entities, these articial creatures are agents of cognitive dissonance, addressing the ambiguities of our perceptions and confronting us with stimuli that we know as deceptive or ctional yet accept as "true" or "real", operating a "suspension of disbelief", feeling empathy or attributing them intentions and self-awareness. These works are paving the way to a time where we will have to interact more and more with articial creatures, in a technological world in which emotion and its affect will have an increasing prominent role in technological artefacts. In the future we will probably interact more and more with objects crossing category boundaries, as the borders between articial and organic will be less and less clear. Both in terms of theory and practices, interaction between art, science and technology can help stimulate a new way of feeling and thinking not achievable through purely rational inquiry. The issue of life evocation in articial artefacts is crucial to our technology-oriented society. Indeed, studies demonstrate that our relationships with machines are both natural and social, since our brain mechanisms evoke empathy, trust, uncanniness, etc. towards an assembly of circuits or mechanical pieces. A
Life evocation in behaviour
84
technological object can suspend our disbelief, just as would a literary character or a puppet; moreover, we project our feelings and attachment more and more on virtual worlds, and we become operators or "interactors" with many on-line puppets (the avatars) that represent ourselves or others in the digital realm. Robots, like puppets, are an example of the ontological paradox that can take place in our technology-saturated environment, as entities simultaneously occluding and exposing their articiality. 4.1.4 Uncanniness as a result of life evocation
The Uncanny Valley
The concept of the "uncanny" can be explored as a rst example of concept concerning life evocation leaked from aesthetics to technology. Ambiguity related to life evocation has been singled out and discussed in theory of literature and psychology, where it elicits, following Jentsch (Jentsch 1909) and then Freud (Freud 1919), specic feeling of "uncanniness", which the two authors dene as the result of the process of familiarity (heimlich) turned unfamiliar (unheimilich, translated in English as "uncanny"). Among the example proposed by Jentsch and then taken up by Freud, there are wax gures, which have the "ability to retain their unpleasantness after the individual has taken a decision as to whether it is animate or not". Jentsch suggests that this feeling is caused by "secondary doubts which are repeatedly and automatically aroused anew when one looks again and perceives ner details; or perhaps it is also a mere matter of the lively recollection of the rst awkward impression lingering in ones mind". Jentsch intuition has been translated into robotics in the 70ies with the so-called "Uncanny Valley" [Mori, 1970], promoted by robotic pioneer Mori and illustrated in Figure 4.1. The Uncanny Valley (a translation from the Japanese "Bukimi No Tani") theory states that as the level of human-likeness increases up to a certain point the sense of familiarity with the robot increases as well, but only up to a certain point; beyond this it drops off suddenly as robots become too human-like and thus appearing uncanny. In his diagram showing this negative peak, Mori introduces movement as a factor reinforcing the perception of uncanniness or familiarity. The rst part of Moris model describes the increasing of the sense of familiarity up to the maximum peak, after which the uncanny valley opens up. From the other side of the valley another positive peak grows in correspondence with creatures reproducing human attributes exactly and nally with healthy human subjects.
4.2 the illusion of life in artificial creatures : features of believability
85
Figure 4.1: The Uncanny Valley graph following Mori
4 .2
the illusion of life in artificial creatures : fea tures of believability
In the eld of Human/Robot Interaction (HRI) and Computer Graphics (CG), the notion of life evocation is translated into the concept of believability. Denitions of believability are numerous and diversied: the issues emerging from studies concern traditional agents properties (agents can be robots or avatars, or any articial artefact that acts) concern social functions, behaviour, and a relationship with the environment. At a basic level, believability is a specic property of articial agents aiming the users suspension of disbelief and "illusion" of life, triggering ones "instinctive" reaction. Researchers attempting to create engaging articial creatures both in the HRI and in the CG elds may nd important insights in the work of artists who have explored life evocation. Researchers in articial intelligence and robotics have built various types of social robots which can express emotions. Research pursues such a goal mainly through speech, facial expressions and hand gestures. Indeed, it is generally a common hypothesis in research that if robots have salient human-like attributes, then peoples mental models of robotic assistants will become more anthropomorphic as they interact with them. It is
Believability
86
a common hypothesis in the scientic community that in order to achieve a higher degree of empathy robots should have a human-or-pet-like shape, or a very realistic anthropomorphic shape [Kiesler and Goetz, 2002]. 4.2.1 Form: is realism a necessity?
Why looking at the arts?
While the efforts of robotic design research on emotions is generally focused on developing robots with anthropomorphic features such as hand and facial expressions [Roccella et al., 2007]. A small but still focused part of qualitative research has directly faced the question whether anthropomorphic appearance affects believability, generally giving a positive answer, stating that the more the articial creature is realist and human-like the more can trigger empathy and emotions. Other researchers tried to get inspiration from the arts, especially in the eld of CG, stating "that artists tell us are needed to present the convincing, persistent, illusion of life" [Bates, 1994]. But, artistic practices with the goal of believability and arousal of empathy follow the opposite path: in order to create a believable creature, realism is not a necessity, and for evidence we can look to artists creation of believable characters. For instance, an analysis of robotic art practice may suggest that non-humanoid creatures trigger empathy and believability as effectively as humanoid ones. Bill Vorn is an artist whose work is focused on life evocation. His installations involve robotics, motion control, sound, lighting, video, and cybernetic processes. The aim of his robotic art projects is to induce empathy from the viewers towards characters which are nothing more than simple articulated metal structures. The strength and richness of Vorns mechanical machines relies on human perception of basic reactive behaviours in robots (essentially based on sensors) and by an appropriate immersive audiovisual context that sets an environment for the works. But, when confronted with Vorns machines (which in terms of shape are totally non-humanoid), an inevitable reex of empathic projection arises. In Vorns case, the whole realistic appearance is dismissed in favour of a few repetitive movements and simple interactions. Artistic creatures are often a distillate of a few traits, abstracted from reality, and allowing enough space for the spectator to project feelings and interpretations. Caricaturing is a means for capturing the essence, as in the words of two of the most famous Disney animators, Thomas and Johnston, who described their techniques they used in order to create believ-
87
able characters: "The more an animator goes toward caricaturing the animal, the more he seems to be capturing the essence of that animal. If we had drawn real deer in Bambi there would have been so little acting potential that no one would have believed the deer really existed as characters. But because we drew what people imagine a deer looks like, with a personality to match, the audience completely real" [Thomas and Johnston, 1981, reprint 1997]. Believability does not require human form, as suggested by the arts. Drawing another example from Disney animations, we can quote the Fliying Carpet starred in the lm Aladdin (1992): it has no eyes, limbs nor even a head. It is only a carpet that can move. And yet, it has a denite personality with its own goals, motivations and emotions, thus is able to evoke life. "Petit Mal", an artwork by Australian artist Simon Penny, explicitly attempts to explore autonomous behaviour as a probe of emotion, expression, and believability. "Petit Mal" (an epileptic condition, a lapse of consciousness), displays an unpredictable behaviour and is not only an artistic exploration of a mediums potential but also an act of humour on the typical conventional idea of control in robotics: the device is "anti-optimized" to induce the maximum level of personality. Thanks to its sensors, Pennys "Petit Mal" senses and explores architectural space, reacting to people in its environment. Its form and behaviour are neither anthropomorphic nor zoomorphic, but it is clearly perceived as a living creature by observers interacting with it, since they never gain complete control over the system (as in traditional man/machine interaction). This "reactive" model is a communication scheme which is closer to the relationship between living organisms and their environment if compared to the common interactive model where the system is waiting for an input from the user in order to react. In a reactive context proper to autonomous systems, the objects react on their own "will", by themselves, and without the required presence of viewers, communicating a personality through motion [Penny, 1997], and eventually evoking life by other means that form. The central feature of this robot is unpredictability: it never reacts in the same way in the same way to people surrounding it, therefore, is elicits anthropomorphising. [Waytz et al., 2010a] demonstrated that increasing the perceived unpredictability of a non-human agent increases the motivation for mastery and thus anthropomorphism. Anthropomorphising a stimulus makes it seem more predictable and understandable, demonstrating that
Unpredictability
88
Negative bias
anthropomorphism is increased by effectance motivation. Effectance is the possibility to attribute predictability to "behaving" things allow us to have the impression to understand them easily, and to familiarise (giving names to hurricanes) having thus the perception of increased mastery, with the nal goal of making sense of an otherwise uncertain environment. For example [Berlyne, 1950] identies uncertainty of the environment the primary motivation for babies to master their environment. From this seminal publication on, research on experimental psychology has demonstrated that the experience of unpredictability stimulates attempts to gain mastery [Berlyne, 1962]; [Whitson, 2008]. A study by [Waytz et al., 2010b] carried out a study where participants were confronted with malfunctioning computers and unpredictable gadgets, following surprising patterns of behaviour. The more frequently computers malfunctioned, or the more the gadget was rated as unpredictable, the higher was the anthropomorphisation tendency. For instance, in order to trigger anthropomorphism in the the AIBO (a dog robot) Sony researchers have been working on programming this robot in order to make it preserve its "free-will". Systems of behaviour of the AIBO consisted of a base (the "instinct" of the robot"); the robot can learn from the user and be trained, but its instincts will never be completely cancelled. Also, the AIBO has been programmed with "moods" inuencing the reaction in time to training and to enhance perception of unpredictability [Kaplan, 2001]. As we saw in Chapter II, negative bias can inuence anthropomorphism. Anyone had the feeling of attributing a mind to a computer after a crash, with a feeling of frustration; 790/00 of people scold and 73 0/00 curse their computer when they do not "behave" following their intentions [Luczak et al., 2003]. Looking back at the arts, it is quite surprising that any robots designed by artists exhibit a negative behaviour, such as suffering. The Suffering Machine by Ricardo Nascimiento, without being anthropomorphic in shape, is a clear example of how artistic intuition can come to the same results than neuroscience. The Suffering Machine is a robot consisting of three arms or legs all connected at the same joint. All of the robots limbs are controlled by a separate motor, although one of the motors is purposely weak in an attempt to invoke a type of articial handicap. This design poses difculties for the machine to move around its immediate environment and thus attempts to provoke an emotional connection between the audience and the
89
object, triggering individuals empathy with the difculties of movement of the robot. If someone attempts to reach down to help it, the robot plays "dead", thus not accepting any outside aid. Even an everyday object can trigger empathy, if its movements and behaviour are consistently designed: the animated artwork survivor, designed by artist Laura Morelli, is a robotic chair that walks and react to the surrounding environment, in the attempt to emulate and suggest some feeling and behaviours that are typical of survivors of land-mines blasts learning to use crutches. This has the aim of sensitizing viewers and usually provokes very strong empathy reactions in the public. Another common way of evoking life in robotic artworks is simulating specic kinds of animal behaviours such as ocking. Indeed, robotic art has been linked with Articial Life, the study of articial systems which exhibit behaviour characteristic of natural living systems with a bottom-up approach. This is in contrast with the classical Articial Intelligence top-down approach. Some robotic artworks are A-Life sculptures, such as the works by Yves Klein [Klein, 1998]. The process of creating a "living sculpture" involves developing technologies for gesture, locomotion, sensory input, and behaviour to achieve a unied creature. For instance, Octofungi is an eight-sided polyurethane sculpture that uses a neural network to integrate current events via multiple sensors and shape-metal alloy for silent, non-linear motion. Motion provided to the robotic creatures is specic to biological forms, chain reactions, propagation and aggregation behaviour, herds and swarms. The same interest for biologically specic motion is to be found in Flock by Ken Rinaldo. The Flock consists of an assemblage of hanging robotic arms that interact with one another and with viewers, to manifest a "ocking" behaviour that develops from an awareness of each other and the environment. The artworks behaviour is analogous to the ocking found in natural groups such as birds, schooling sh, or ying bats. Flocking behaviours demonstrate characteristics of supra-organization, of a series of animals or articial life forms that act as one. They are complex interdependent interactions which require individual members to be aware of their position in relation to others. Another important factor eliciting believability by means of emotion attribution and empathy can be attributed to infantilelike form. For Konrad Lorenz [Lorenz, 1970] juvenile traits automatically trigger a reaction of empathy and tenderness. Features identied by Lorenz are "a relatively large head, a disproportion-
Animal behaviour
Feeding and empathy
90
Personality
ately large forehead, large eyes placed underneath, prominent curved cheeks, short thick limbs a rm elasticity, and awkward movements". These are the same principles used by Cynthia Breazeal in order to design the robot Kismet [Breazeal, 2000]. Kismet is a robotic head which interact with his environment in a non-verbal modality. The robot is very expressive, designed with big eyes eliciting empathy and encouraging interaction. Kismet can express emotions only by varying the position of his head, neck, ears and eyes. Lorenzs shapes triggering empathy are largely independent from realism, as it appears in other robotic creatures designed for interaction, as the AIBO, Furby or Paro. On the contrary, if the above quoted theory of the Uncanny Valley is true, a resemblance which is too close with an existing living being may be counter-productive in terms of empathy. Paradoxically in this forth level of ambiguity it is probably necessary that the creature is perceived as articial to fully elicit anthropomorphism. Movements, just as in the case of the illusion of life, is a crucial feature for eliciting empathy. Contrary to the Kismet, the AIBO does not have juvenile features in form, but in movement: his movement being awkward, he elicit the idea of a puppy; when it is switch off, many users can nd him repulsive and cold [Kaplan, 2001]. [Thomas and Johnston, 1981, reprint 1997] refer to personality as the most important requirement for creating believable characters, dening personality as "all the particular details especially details of behaviour, thought and emotions - that together dene the individual. The second requirement for believable agents is that they appear to have emotional reactions and to show those emotions in some way. This is such a fundamental requirement in the traditional character-based arts that often artists refer to expressing the emotions of their characters as what brings them to life. 4 .3 conclusions
An inspiring function of both art and technology is to go beyond representational "likeness" for evoking the feeling of life itself, triggering those qualia emerging when we are confronted with an animated creature. In two domains as different as these, such a function has different objectives and strategy of accomplishment, but a common perceptual feature: the possibility to convey the impression that and object is alive despite the exact awareness that it is not. It is undoubtedly not too difcult to press our
4.3 conclusions
91
"Darwinian buttons" since, probably for evolutionary reason, we are biased to perceive life: "there is a universal tendency among mankind to conceive all beings like themselves, and to transfer to every object, those qualities, with which they are familiarly acquainted, and of which they are intimately conscious". It is known from cognitive sciences that the human brain organises external reality abstracting it, in order to make it meaningful; life, therefore, could be an inborn perceptual category that we spontaneously apply even at the price of cognitive dissonance. Thus life perception could be thought of as an optical illusion, not so different from seeing two lines diverging even knowing they are parallel, and in general similar to those phenomena causing a friction between what we know (conceptual knowledge) and what we feel (perceptual knowledge): a temporary overriding of bottom-up impressions on top-down awareness. In particular, life perception of non-animated objects could be assimilated to the illusion of ambiguity, traditionally dened in literature as the possibility for a stimulus to be stably interpreted in only one way at a given moment, but in two or more different ways over time. In this thesis I investigated, by means of an fMRI study, how the illusion of ambiguity relates to perception of categories, by using stimuli whose interpretable content is known form previous studies to be processed in different brain areas, namely geometrical gures, faces and bodies. In line with the initial hypothesis, in the case of geometrical ambiguous images, for both possible interpretations activations where found in the same area, the middle occipital gyrus. In the case of face-body ambiguous images, there was a correlation between face perception as indicated by the subjects and the FFA, known as the area processing faces. Previous studies on the illusion of ambiguity have consistently shown a network of frontal-parietal activations in correlation with the perception of ambiguous images when contrasted with their stable counterparts. In line with literature, the contrast between stable and bistable images revealed increased bilateral activations in the superior frontal gyrus and in the superior parietal lobule. Surprisingly, the activation of this network was signicantly larger for geometrical ambiguous images (intra-categorical) then for face-body ones (inter-categorical), as if the latter were less effective stimuli in terms of ambiguity, being not purely bistable. A qualitative questionnaire outlined how 56 % of subjects had the impression they could see both images at the same time in the case of inter-categorical images. The implication of this
92
ndings is that it may be useful to distinguish between different levels of ambiguity, in which the rst one correlates with a more "pure" perception of bistability implying the mutual exclusiveness of the two possible interpretations. Second level ambiguity may correlate with the coexistence of the two interpretations and a diminished activity in the fronto-parietal network. In this framework, we identied the illusion of life as a third level ambiguity. The illusion of life is the attribution of aliveness features (desires, goals, intentions, strategies) to simple geometrical moving gures. In this case, activations are surprisingly coincident with neural activations relative to perception of other individuals, corresponding to those activations constituting the "social brain"; in particular, the posterior and superior temporal sulcus and gyrus activates in correlation with animacy, as they do in correlation with spotting biological motion while viewing a point-light display stimulus, and the amygdala, known for processing emotion-related information. While a frontal brain damage can prevent subjects to see ips of ambiguous images, an amygdala damage can prevent the illusion of life, and mental attributions are often inappropriate in autistic children. Except for this cases, the illusion of life is universally perceived, while variability in what I identied as a forth level ambiguity, anthropomorphism, is much higher, depending on culture, age and needs. In this last level of ambiguity I believe that merging between art and technology could be especially fruitful. Indeed, anthropomorphism as life evocation as long been exploited by art, and it has a great potential of application in todays society, where we interact more and more with articial creatures such as robots and avatars. As a future work. it could be interesting to investigate in depth which are the insights that art could suggest both to cognitive sciences and technological applications. Artworks suggest that there is no such a thing as a clear and discrete gap in our perception between animate creatures and inanimate objects, but rather a continuum. In their paradoxical status of quasi-living entities, these articial creatures are agents of cognitive dissonance, addressing the ambiguities of our perceptions and confronting us with stimuli that we know as deceptive or ctional yet accept as "true" or "real", operating a suspension of disbelief, feeling empathy or attributing them intentions and self-awareness. These works are paving the way to a time where we will have to interact more and more with articial creatures, in a technological world in which emotion and its affect will have an increasing prominent role in technologi-
4.3 conclusions
93
cal artefacts. In the future we will probably interact more and more with objects crossing category boundaries, as the borders between articial and organic will be less and less clear. Both in terms of theory and practices, interaction between art, science and technology can help stimulate a new way of feeling and thinking not achievable through purely rational inquiry.
BIBLIOGRAPHY
R. Adolphs. Recognizing emotions from facial expressions: psychological and neurological mechanisms. Behavioural and Cognitive Neuroscience Reviews, pages 2161, 2003. D. Amodio and C. Frith. Meeting of minds: the medial frontal cortex and social cognition. National Review of Neuroscience, 7 (4):268277, 2006. T.J. Andrews and C. Blakemore. Form and motion have independent access to consiousness. Nat. Neurosci, 2:406462, 1999. T.J. Andrews et al. Activity in the fusiform gyrus predicts conscious perception of rubins vase-face illusion. Neuroimage, 17:890901, 2002. P.J. Asquith. Primateontogeny, cognition, and social behavior, chapter Anthropomorphism and the Japanese and Western traditions in primatology. Cambridge University Press, 1986. F. Attneave. Multistability in perception. Sci. Am, 225:6371, 1971. B. r. r, 2002. S. Baron-Cohen. Mindblindness: an essay on autism and theory of mind. MIT Press/Bradford Books, 1995. J.N. Bassili. Temporal and spatial contingencies in the perception of social events. Journal of Personality and Social Psychology, 33 (6):680685, 1976. J. Bates. The role of emotion in believable agents. Technical report, Carnegie Mellon University, 1994. M. Bergamasco, C.A. Avizzano, F. Ghedini, and M. Carrozzino. Le interfacce aptiche per i beni culturali. Proceedings of LUBEC 2007, "Valorizzazione dei Beni Culturali e Innovazione", 2007. D.E. Berlyne. Novelty and curiosity as determinants of exploratory behavior. British Journal of Psychology, 41:6880, 1950.
95
96
bibliography
D.E. Berlyne. Uncertainty and epistemic curiosity. British Journal of Psychology, 53:27 34, 1962. C.M. Bird, F. Castelli, et al. The impact of extensive medial frontal lobe damage om "theory of mind" and cognition. Brain, 127:914928, 2004. R. Blake. A neural theory of binocular rivalry. Psychol. Rev., 96: 145167, 1989. S.J. Blakemore et al. The detection of contingency and animacy from simple animations in the human brain. Cerebral Cortex, 13(8), 2003. P.W. Blythe, P.M. Todd, and G.F. Miller. Simple heuristic that make us smart, chapter How motion reveals intention: categorising social interactions. Oxford University Press, 1999. C. Breazeal. Proto-conversations with an anthropomorphic robot. Proceedings of the Ninth IEEE International Workshop on Robot and Human Interactive Communication (Ro-Man), pages 328333, 2000. A.S. Bregman. Auditory Scene Analysis. MIT Press, 1990. J. Britz et al. Right parietal brain activity precedes perceptual alternation of bistable stimuli. Cereb. Cortex, 19:5565, 2009. L. Brothers. The social brain: a project for integrating primate behaviour and neurophysiology in a new domain. Concepts in Neuroscience, pages 2752, 1990. A. Caramazza and B. Mahon. The organisation of conceptual knowledge in the brain: the futures past and some future directions. Cognitive Neuropsychology, 23(1):1338, 2006. S. Carey. Conceptual change in childhood. Cambridge, MIT Press, 1985. O. Carter, D. Presti, et al. Meditation alters perceptual rivalry in tibetan buddhist monks. Current biology, 2005. O. Carter et al. Tactile rivalry demonstrated with an ambiguous apparent-motion quartet. Current Biology, 19:10501054, 2008. F. Castelli et al. Movement and mind: a functional imaging study of perception and interpretation of complex intentional movement patterns. NeuroImage, 12:314325, 2000.
bibliography
97
W. Cheselden. An account of some observations made by a young gentleman, who was born blind, or lost his sight so early, that he had no remembrance of ever having seen, and was couchd between 13 and 14 years of age. Philosophical Transactions, 35:447450, 1683-1775. Y.C. Chiu, M. Esterman, et al. Decoding task-based attentional modulation during face categorization. Journal of Cognitive Neuroscience, 2011. C.E. Cleland and C.F. Chyba. Dening "life". Origins of Life and Evolution of the Biosphere, 32:387393, 2002. M. Cohen. The art of puppetry. Animations Online, 18, 2006. G. De Vere, editor. Lives of the Most Eminent Painters, Sculptors and Architects, volume 1. Macmillan and the Medici Society, London, 1912-1915. D. Dima et al. Understanding why patients with schizophrenia do not perceive the hollow-mask illusion using dynamic causal modelling. NeuroImage, 46(4):11801186, 2009. W.H. Dittrich and S. E. G. Lea. Visual perception of intentional motion. Perception, 23:253268, 1994. P.E. Downing et al. A cortical area selective for visual processing of the human body. Science, 293:24702473, 2001. W. Einhauser, K.A.C. Martin, and G. Konig. Are switches in perception of the necker cube related to eye position? European Journal of Neuroscience, 20:28112818, 2004. N. Epley, A. Waytz, and J.T. Cacioppo. On seeing human: A three-factor theory of anthropomorphism. Psychological Review, 114:864886, 2007. N. Epley, A. Waytz, et al. When i need a human: Motivational determinants of anthropomorphism. Social Cognition, 26:143 155, 2008. M. Fried. Art and objecthood. Art Forum, 1967. K. Friston, P. Fletcher, et al. Event-related fmri: characterizing differential responses. Neuroimage,, 7(1):3040, 1999. K.J. Friston, P. Fletcher, et al. Event-related fmri: characterizing differential responses. Neuroimage, 7:3040, 1998.
98
bibliography
A. Gelb and K. Goldstein. Uber farbennamenamenesie. Psychologische Forschung, 1925. F. Ghedini and M. Bergamasco. Robotic art: Perceiving and inventing reality. Art and Science: exploring the limits of human perception, Conference Proceedings, 2009. F. Ghedini and M. Bergamasco. Life evocation in art: from representation to behaviour. ISEA 2011, 2011a. F. Ghedini and M. Bergamasco. Perception of ambiguous gures: an fmri study. VRR-IJCAI 2011, 2011b. F. Ghedini, H. Faste, M. Carrozzino, and Bergamasco M. Passages: An artistic 3d interface for childrens rehabilitation and special needs. ICDVRAT, 2008. Fiammetta Ghedini. Developing an applied framework for the integration of artistic approaches and technological competences. PhD Application for the Scuola Superiore SantAnna, 2008. J.J. Gibson. Perception of the Visual World. Houghton Mifin, Boston, 1950. B. Giesbrecht, M.G. Woldorff, et al. Neural mechanisms of topdown control during spatial and feature attention. Neuroimage, 19:496512, 2003. E. Gombrich. The Story of Art. Phaidon, New York, 1995. R.L. Gregory. Cognitive contours. Nature, 238:5152, 1972. R.L. Gregory. The Oxford Companion to the Mind. Oxford University Press, 1987. R.L. Gregory. Ambiguity of ambiguity. Perception, 29:11391142, 2000. R.L. Gregory. Seeing Through Illusions. Oxford University Press, 2009. H. Hashimoto. A phenomenal analysis of social perception. Journal of Child Development, pages 326, 1966. U. Hasson et al. Vase or face? a neural correlate of shapeselective grouping processes in the human brain. J. Cogn. Neurosci., 13:744753, 2001.
bibliography
99
A. S. Heberlein. How Humans See, Represent, and Act on Events, chapter Animacy and intention in the brain: neuroscience of social event perception. Oxford University Press, 2008. A.S. Heberlein and R. Adolphs. Impaired spontaneous anthropomorphizating despite intact perception and social knowledge. Proceedings of the National Academy of Sciences of the United States of America, 101(19):74877491, 2004. F. Heider and M. Simmel. An experimental study of apparent behavior. American Journal of Psychology, 57:243259, 1944. E. Hering. Outlines of a theory of the Light Sense. Harvard University Press, transl. by L.M. Hurvich and D. Jameson, 1964 (originally pub. 1877). G. Hesselmann et al. Spontaneous local variations in ongoing neural activity bias perceptual decisions. Proc. Natl. Acad. Sci. U. S. A., 105:1098410989, 2008. J.B. Hopnger, M.H. Buonocore, and G.R. Mangun. The neural mechanisms of top-down attentional control. Nature Neuroscience, pages 284291, 2000. D.H. Hubel and T.N. Wiesel. The ferrier lecure. funcitonal achitecture of macaque monkey visual cortex. Proceedings Royal Society London, 198:159, 1977. R. Ilg et al. Neural correlates of spontaneous percept switches in ambiguous stimuli: an event-related functional magnetic resonance imaging study. European Journal of Neuroscience, 28 (11):23252332, 2008. M. Ito. Control of mental activities by internal models in the cerebellum. Nature Review Neuroscience, 9:304 313, 2008. Y. Jiang and S. He. Cortical responses to invisible faces: dissociating subsystems for facial-information processing. Curr. Biol., 16:231258, 2006. G. Johansson. Visual perception of biological motion and a model for its analysis. Percept. Psychophys., 195(204), 1973. M.H. Johnson. Biological motion: a perceptual life detector? Curr Biol., 23(16), 2006.
100
bibliography
N. Kanwisher and G. Yovel. The fusiform face area: A cortical region specialized for the perception of faces. Philosophical Transactions of the Royal Society of London, 361:21092128, 2006. F. Kaplan. LArt, la pensee, les emotions, chapter Un robot peut-il etre notre ami? Grame, Paris, 2001. S. Kiesler and L. Goetz. Mental models and cooperation with robotic assistants. Proceedings of Conference on Human Factors in Computing Systems (CHI), 2002. Y. Klein. Living sculpture: The art and science of creating robotic life. Leonardo, 31(5), 1998. A. Kleinschmidt et al. Human brain activity during spontaneously reversing perception of ambiguous gures. Proc. R. Soc, pages 2427 2433, 1998. J. Kornmeier and M. Bach. The necker cube - an ambiguous gure disambiguated in early visual processing. Vision Research, 45, 2005. E. Land. The retinex theory of color vision. Proc R Inst Gt Br, 47: 2358, 1974. S. H. Lee. Traveling waves of activity in primary visual cortex during binocular rivalry. Nat. Neurosci., 8:2223, 2005. L. Leopold and G. Logothetis. Multistable phenomena: changing views in perception. Trends Cogn. Sci., pages 254264, 1999. A.M. Leslie and S. Keeble. Do six-month-old infants perceive causality? Cognition, 25:265288, 1987. K. Lorenz. Essais sur le comportement animal et humain. Le Seuil, Paris, 1970. H. Luczak, M. Roetting, and L. Schmidt. Lets talk: Anthropomorphization as a means to cope with stress of interacting with technical devices. Ergonomics, 46:13611374, 2003. E.D. Lumer et al. Neural correlates of perceptual rivalry in the human brain. Science, 280:19301933, 1998. J.M. Mandler. How to build a baby ii: Conceptual primitives. Psychological Review, 99:587604, 1992.
bibliography
101
A. Martin and J. Weisberg. Neural foundations for undestandinf social and mechanical concepts. Cognitive Neuropsychology, 30 (3-6):575558, 2003. A. Michotte. The Perception of Causality. trans. T. R. and E. Miles in 1963, 1946. S.R. Mitroff, D.M. Sobel, and A. Gopnik. Reversing how to think about ambiguous gure reversals: Spontaneous alternating by uninformed observers. Perception, pages 709 715, 2006. C.K. Morewedge. A mind of its own: negativity bias in the perception of intentional agency. Journal of Experimental Psychology, 138(4):535545, 2009. M. Mori. The uncanny valley. Energy, 7:3335, 1970. R. Morris. Notes on sculpture. Artforum, 1969-1969. T. Noesselt et al. Delayed striate cortical activation during spatial attention. Neuron, 5:575587, 2002. J.E. Opfer. Identifying living and sentient kinds from dynamic infomation: the case of goal-directed versus aimless autonomous movement in conceptual change. Cognition, 86(2):97122, 2002. J.X. OReilly, M.M. Mesulam, and A.C. Nobre. The cerebellum predicts the timing of perceptual events. Journal of Cognitive Neuroscience, pages 22522260, 2008. L. Parkkonen et al. Early visual brain areas reect the percept of an ambiguous scene. Proc. Natl. Acad. Sci. U. S. A., 105: 2050020504, 2008. M.V. Peelen and P.E. Downing. Selectivity for the human body in the fusiform gyrus. Journal of Neurophysiology, 93:6038, 2007. S. Penny. Embodied cultural agents: at the intersection of art, robotics, and cognitive science. In Socially Intelligent Agents Symposium, MIT, 1997. M.A. Peterson and B.S. Gibson. Directing spatial attention within an object. Journal of Experimental Psychology, 17:170 82, 1991. A. Polonsky et al. Neuronal activity in human primary visual cortex correlates with perception during binocular rivalry. Nat. Neurosci., pages 11531159, 2000.
102
bibliography
D. Premack. Do infants have a theory of self-propelled objects? Cognition, 1990. M. Raemaekers. Widespread fmri activity differences between perceptual states in visual rivalry are correlated with differences in observer biases. Brain Res, 1252:161171, 2009. B. Rime, B. Boulanger, et al. The perception of interpersonal emotions originated by patterns of movement. Motivation and Emotion, 9(3):241260, 1985. S. Roccella et al. Design and development of ve-ngered hands for a humanoid emotion expression robot. International Journal of Humanoid Robotics, 4(1), 2007. I. Roseneld. The Invention of Memory. Basic Books, New York, 1988. O. Sacks. The case of the colorblind painter. In An Anthropologist on Mars, pages 341. Random House, 1995. E. Peterson Sanouillet, editor. The Creative Act. Salt Seller, New York, 1973. R. Saxe and N. Kanwischer. People thinking about people. the role of the temporal-parietal junction in "theory of mind". Neuroimage, 19(4):18351842, 2003. U. Schneider, F.M. Leweke, U. Sternemann, M. Weber, and H. Emrich. Visual 3d illusion: a systems-theoretical approach to psychosis. Eur. Arch. Psychiatry Clin. Neurosci, pages 256 260, 1996. B.J. Scholl and P.D. Tremoulet. Perceptual causality and animacy. Trends in Cognitive Sciences, 4(8):299309, 2000. J. Schultz et al. Activation of the human superior temporal gyrus during observation of goal attribution by intentional objects. Journal of Cognitive Neuroscience, 16(4):16951705, 2004. J. Schultz et al. Activation in posterior superior temporal sulcus parlallels parameter inducing the percept of animacy. Neuron, 45(4):625635, 2005. D. Skuse, J. Morris, and K. Lawrence. The amygdala and development of the social brain. Annals of the New York Academy of Sciences, 1008:91101, 2003.
bibliography
103
S.D. Slotnick, J. Schwarzbach, and S. Yantis. Attentional inhibition of visual processing in human striate and extrastriate cortex. Neuroimage, 19:16021611, 2003. P. Sterzer and A. Kleinschmidt. A neural signature of colour and luminance correspondence in bistable apparent motion. Eur. J. Neurosci., 21:30973106, 2005. P. Sterzer and A. Kleinschmidt. A neural basis for inference in perceptual ambiguity. Proc Natl Acad Sci U S A, 104:323328, 2007. P. Sterzer et al. Neural correlates of spontaneous direction reversals in ambiguous apparent visual motion. Neuroimage, 15:908916, 2002. P. Sterzer et al. Fine-scale activity patterns in high-level visual areas encode the category of invisible objects. Curr. Opin. Neurobiol, 13:433439, 2008. P. Sterzer et al. Electromagnetic responses to invisible face stimuli during binocular suppression. Neuroimage, 46:803808, 2009. J.A. Stewart. Perception of animacy. PhD thesis, University of Pennsylvania, 1982. F. Thomas and O. Johnston. The Illusion of Life: Disney Animation. Hyperion, 1981, reprint 1997. S. Tillis. The actor occluded: Puppet theatre and acting. Theatre Topics, 6(2):109119, 1996. F. Tong. Primary visual cortex and visual awareness. Nat. Rev. Neurosci., pages 219229, 2003. F. Tong and S. A. Engel. Interocular rivalry revealed in the human cortical blind-spot representation. Nature, pages 195 199, 2001. F. Tong, K. Nakayama, et al. Binocular rivalry and visual awareness in human extrastriate cortex. Neuron, 21:753759, 1998. F. Tong et al. Neural bases of binocular rivalry. Trends Cogn. Sci., pages 502511, 2006. T.C. Toppino. Reversible-gure perception: Mechanisms of intentional control. Perception and Psychophysics, pages 12851295, 2003.
104
bibliography
T.C. Toppino and G.M. Long. Dynamic cognitive processes, chapter Top-down and bottom-up processes in the perception of reversible gures: Toward a hybrid model, pages 3758. Tokyo: Springer-Verlag, 2005. J. Tracy et al. The brain topography associated with active reversal and suppression of an ambiguous gure. European Journal of Cognitive Psychology, 17:267288, 2005. P.D. Tremoulet and J. Feldman. Perception of animacy from the motion of a single object. Perception, 29:943951, 2000. T. Troscianko, S.E. Lea, and D. Morgan. Perception of emotion from dynamic point-light displays represented in dance. Perception, 25(6):727738, 1996. Y. Tsal and L. Kolbet. Disambiguating ambiguous gures by selective attention. Quarterly Journal of Psychology, pages 25 37, 1985. H. von Helmholtz. Handbuch der Physiologischen Optik, English translation, J.P.C. Southall, Treatise on Physiological Optics. Dover, New York, 1962. B. Vorn. A conversation with bill vorn (on robotic art creatures and other believable living machines). Interview by Fiammetta Ghedini, Walter Aprile and Haakon Faste, 2007. R.M. Warren and R.L. Gregory. An auditory analogue of the visual reversible gure. Am. J. Psychol, 71:612613, 1958. E. Warrington and T. Shallice. Category specic impairments. Brain, 107:829854, 1984. A. Waytz, J. Cacioppo, and N. Epley. Who sees human? : The stability and importance of individual differences in anthropomorphism. Perspectives on Psychological Science, 5(219), 2010a. A. Waytz, C.K. Morewedge, et al. Making sense by making sentient: Effectance motivation increases anthropomorphism. Journal of Personality and Social Psychology, 2010b. B. Weiner. Spontaneous causal thinking. Psychological Bulletin, 97:7484, 1985. Milleville Wheatley and Martin. Understanding animate agents: distinct roles for social networks and mirror system. Psychological Science, 18(6):469474, 2009.
bibliography
105
J.A. Whitson. Lacking control increases illusory pattern perception. Science, 322:115117, 2008. M.A. Williams. Amygdala responses to fearful and happy facial expressions under conditions of binocular suppression. J. Neurosci., pages 28982904, 2004. S. Windmann, M. Wehrmann, et al. Role of the prefrontal cortex in attentional control over bistable vision. Journal of Cognitive Neuroscience, 18:456471, 2006. I. Yevin. Ambiguity and art. URL Publication on VisMath, 2000. URL www.vismath1.tripod.com/igor/. J.M. Zacks, K.M. Vettel, and J.M. Avoy. Visual motion and the neural correlates of event perception. Brain Research, 1076(1): 150162, 2006. S. Zeki. The neurology of ambiguity. Consciousness and Cognition, 13:173196, 2004. S. Zeki. Splendors and miseries of the brain. Blackwell, London, 2009.
colophon
AT X 2 using Hermann Zapfs This thesis was typeset with L E Palatino and Euler type faces (Type 1 PostScript fonts URW Palladio L and FPL were used). The listings are typeset in Bera Mono, originally developed by Bitstream, Inc. as Bitstream Vera. (Type 1 PostScript fonts were made available by Malte Rosenau and Ulrich Dirr.) The typographic style was inspired by Bringhursts genius as presented in The Elements of Typographic Style [B, 2002]. It is AT X via CTAN as classicthesis. available for L E
note : The custom size of the textblock was calculated using the directions given by Mr. Bringhurst (pages 2629 and 175/176). 10 pt Palatino needs 133.21 pt for the string abcdefghijklmnopqrstuvwxyz. This yields a good line length between 2426 pc (288312 pt). Using a double square textblock with a 1:2 ratio this results in a textblock of 312:624 pt (which includes the headline in this design). A good alternative would be the golden section textblock with a ratio of 1:1.62, here 312:505.44 pt. For comparison, DIV9 of the typearea package results in a line length of 389 pt (32.4 pc), which is by far too long. However, this information will only be of interest for hardcore pseudotypographers like me. To make your own calculations, use the following commands and look up the corresponding lengths in the book:
\settowidth{\abcd}{abcdefghijklmnopqrstuvwxyz} \the\abcd\ % prints the value of the length
Please see the le classicthesis.sty for some precalculated values for Palatino and Minion. 159.85193pt Final Version as of May 24, 2011 at 16:35.
D E C L A R AT I O N
This thesis is a presentation of my original research work. Wherever contributions of others are involved, every effort is made to indicate this clearly, with due reference to the literature, and acknowledgement of collaborative research and discussions. The work was done under the guidance of Professor Massimo Bergamasco, at the Perceptual Robotics Laboratory, Scuola Superiore SantAnna, Pisa, Italy. Pisa, 30 May 2011
Fiammetta Ghedini

ClassicThesis PDF

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

ClassicThesis PDF

Hochgeladen von

Copyright:

Verfügbare Formate

fiammetta ghedini T H E I L L U S I O N O F A M B I G U I T Y: F R O M B I S TA B L E P E R C E P T I O N T O ANTHROPOMORPHISM

T H E I L L U S I O N O F A M B I G U I T Y: F R O M B I S TA B L E PERCEPTION TO ANTHROPOMORPHISM fiammetta ghedini

Figure 1.1 Figure 1.2 Figure 1.3

The Uncanny Valley graph following Mori

perception : making sense of the senses

1.1 perception : making sense of the senses

1.1 perception : making sense of the senses

Abstraction at cell level

Figure 1.1: The response of an orientation selective cell

1.1 perception : making sense of the senses

Figure 1.2: The response of an orientation selective cell in V5

The Case of Colour Vision

What are colours?

The Land hypothesis

1.1 perception : making sense of the senses

concepts and categories in the brain

1.2 concepts and categories in the brain

Inherited and acquired concepts

seeing through illusions

Two Different Categories of Illusions

1.3 seeing through illusions

1.3 seeing through illusions

Figure 1.4: Model of perception, based on (Gregory 2009)

Four levels of ambiguity

1.3 seeing through illusions

Figure 1.5: Different kinds of ambiguities

Part II FIRST AND SECOND LEVEL AMBIGUITY

perception of bistable ambiguity

Figure 2.1: a) Necker Cube - b) Rubin Vase

Verbal transformation effect

2.1 what is multistable ambiguity

Figure 2.2: Binocular rivalry

perception of bistable ambiguity

2.1 what is multistable ambiguity

Subcortical and early cortical visual processing

Extrastriate visual cortex

perception of bistable ambiguity

Neural correlates of ips

2.1 what is multistable ambiguity

Parietal, frontal and prefrontal cortex

perception of bistable ambiguity

Bottom up versus top down: conclusions

2.1 what is multistable ambiguity

The attention issue

perception of bistable ambiguity

2.2 an experiment on two - levels bistable ambiguity

Content of the percept

perception of bistable ambiguity

2.2 an experiment on two - levels bistable ambiguity

perception of bistable ambiguity

2.2 an experiment on two - levels bistable ambiguity

perception of bistable ambiguity

Blocked fMRI analysis: Neural specicity of bistable images

Conjunction analysis: comparing levels of ambiguity

Event-related results: contrasting interpretations

perception of bistable ambiguity

perception of bistable ambiguity

perception of bistable ambiguity

Figure 2.4: Experiment stimuli: bistable intra-categorical images + stabilized versions

Figure 2.5: Experiment stimuli: bistable inter-categorical images + stabilized versions

perception of bistable ambiguity

Figure 2.6: Bistable Figures > Stable Figures