Programming the post-human: Computer science redefines life

Ellen Ullman, a former software engineer, is the author of Close to the Machine: Technophilia and Its Discontents and the novels The Bug and By Blood.

There are times when you feel you are witnessing the precise moment when science fiction crosses over into science. I dont mean the advent of gadgetry like voice recognition software or flat panel computer monitors! or even the coming of space tourism! "ut a moment when one of the great! enduring conundrums of our speculative literature suddenly materiali#es as an actual human situation. $erhaps the greatest of these is the pro"lem of distinguishing humans from ro"ots%what uni&ue element! if any! separates us from machines'%a theme that received its classic treatment "y Isaac (simov in his )*+, story -.vidence./ In the story! a future society has for"idden the use of humanoid ro"ots! since it is feared that the intelligent machines! identical in appearance to humans "ut with superior powers! will take over the world. ( man named 0tephen Byerley! who was suspected of "eing a ro"ot! eventually does 1ust that! "ecoming the first -2orld Coordinator./ Many years later! however! his humanity is still in dou"t: I stared at her with a sort of horror! -Is that true'/ -(ll of it!/ she said. -(nd the great Byerley was simply a ro"ot./ -3h! theres no way of ever finding out. I think he was. But when he decided to die! he had himself atomi#ed! so that there will never "e any legal proof. Besides! what difference would it make'/ In 4anuary 5665! fifty si7 years after the pu"lication of (simovs story! a group of computer scientists! cryptographers! mathematicians! and cognitive scientists gathered at -the first workshop on human interactive proofs!/ where their goal was the creation of a C($TC8(! a -Completely (utomated $ro"a"ilistic $u"lic Turing Test to Tell Computers and 8umans (part./ In (simovs story! distinguishing ro"ots from humans was a matter of world historical importance! a &uestion of human dignity and worth9 the pro"lem for the scientists at the workshop was the development of automated methods to prevent software ro"ots! or -"ots!/ from invading chat rooms and "arraging email systems with unwanted -spam/ messages. Thus fantasy leaks into everyday life! a grand vision of the future reduced to a pressing! if mundane! commercial pro"lem: how to tell human from machine. 2hat is interesting a"out this pro"lem is that its one we humans have "rought on ourselves. It is not a contest of human versus machine! though it is often presented that way9 it is instead an outgrowth of what is most deeply human a"out us as Homo faber, the toolmaker. 2e have imagined the e7istence of ro"ots! and having dreamed them up we feel compelled to "uild them! and to endow them with as much intelligence as we possi"ly can. 2e cant help it! it seems9 its in our nature as fashioners of helpful :and dangerous; o"1ects. 2e cant resist taking up the dare: Can we create tools that are smarter than we are! tools that cease in crucial ways to "e -ours/' <nderlying that dare is a philosophical shift in the scientific view of humanitys role in the great pro1ect of life. =esearchers in ro"otics and artificial life :also known as -(life!/ the "ranch of computer science that concerns itself with the creation of software e7hi"iting the properties of life; openly &uestion the -specialness/ of human life. 0ome call life as we know it on .arth

merely one of many -possi"le "iologies!/ and see our reverence for humanity as something of a pre1udice :-human chauvinism/;. $ersonhood has "een defined as -a status granted to one another "y society! not innately tied to "eing a car"on "ased life form./ (ccording to =odney Brooks! director of MITs artificial intelligence la"! evolution spelled the end of our uni&ueness in relation to other living creatures "y defining us as evolved animals9 and ro"otics! in its &uest to create a sentient machine! looks forward to ending the idea of our uni&ueness in relation to the inanimate world. In what may reflect supreme humility :we are no "etter than the rocks or the apes; or astounding hu"ris :we can create life without the participation of either >od or the natural forces of evolution;! computer science has initiated a de"ate over the coming of the -posthuman/: a non"iological! sentient entity. (ccording to this idea! the post humans thoughts would not "e limited "y the slow speed of our own nervous systems. <nhampered "y the messy wet chemistry of car"on "ased life! loosed from the random walk of evolution! the post human can "e designed! consciously! to e7ceed our capa"ilities. Its memory can "e practically limitless. It can have physical strength without "ounds. (nd! freed from the senescence of the cells! it might live forever. If this sounds like 0uperman :-with powers and a"ilities far "eyond those of mortal man/;! consider another of those moments when science fiction passes over into science: The date was (pril )! 5666. The place was a lecture hall on the campus of 0tanford <niversity. Douglas 8ofstadter! the computer scientist perhaps "est known for his "ook Gdel, Escher, Bach, assem"led a panel of ro"oticists! engineers! computer scientists! and technologists! and asked them to address the &uestion: -2ill spiritual ro"ots replace humanity "y 5)66'/ Despite the date! it was not an (pril ?ools 1oke. 8ofstadter "egan "y saying! crankily! that he had -decided to eliminate naysayers/ from the panel! making his point with a cartoon of a fish that thinks it is ridiculous that life could e7ist on dry land :-gri""it! gri""it!/ went the sound of a frog;. -It is more ama#ing!/ he said! -that life could come from inert matter than from a change of su"strate/%more ama#ing that life could arise from a soup of dead molecules than change its "ase from car"on to something else9 silicon! for e7ample. 8ofstadter looked into the future and said! without nostalgia or regret: -I really wonder whether there will "e human "eings./ The room was filled to fire marshal alarming proportions. $eople 1ammed the doors! stood against the walls! sat in the aisles! on the steps in the steep "alcony of the lecture hall! leaned dangerously against the "alcony rails. The audience! young and old! students and -gray"eards/ of the computing community of 0ilicon @alley! sat still and &uiet! leaning forward! putting up with the crowding and the heat and Doug 8ofstadters grouchy refusal to use a microphone. 0itting! as I was! high up in the "alcony! the scene reminded me of nothing so much as those paintings of early medical dissections! crowds of men peering down to where the cadaver lay slashed open in the operating theater "elow. That day at 0tanford there was the same sense that some threshold! previously ta"oo to science! had "een crossed. Computer science! which heretofore had served humanity "y creating its tools! was now considering another o"1ective altogether: the creation of a non"iological! -spiritual/ "eing%sentient! intelligent! alive%who could surpass and! perhaps! control us.

This was not the first time computer science thought it was on the verge of creating a successor race of machines. 2hen I was a young programmer in the late )*A6s and early )*B6s! a "ranch of computer science then called -artificial intelligence/ "elieved that it was close to creating an intelligent computer. (lthough (I would fail spectacularly in fulfilling its grand e7pectations! the de"ate surrounding the field was alluring. Cike many at the time! I saw in (I the opportunity to e7plore &uestions that had previously "een in the province of the humanities. 2hat are we' 2hat makes a human intelligent' 2hat is consciousness! knowledge! learning' 8ow can these things "e represented to a machine! and what would we learn a"out ourselves in the formation of that representation' It was clear that as mem"ers of a secular society that has given up on the idea of >od we would "e looking elsewhere for the source of what animates us! and that -elsewhere/ would "e the study of cy"ernetic intelligence! the engine of postmodern philosophical speculation. It is for this reason that the &uestion of the post human is worth e7ploring. 2hether or not we can "uild a -spiritual ro"ot/ "y 5)66! in asking what is -post/ human! we must first ask what is human. The ensuing de"ate inherits the &uestions that once "elonged almost e7clusively to philosophy and religion%and it inherits the same ancient! deep seated confusions. 3ver the years! as I listened to the engineering give and take over the &uestion of artificial life forms! I kept coming up against something o"durate inside myself! some stu""orn resistance to the definition of -life/ that was "eing promulgated. It seemed to me too reductive of what we are! too mechanistic. .ven if I could not &uite get myself to "elieve in >od or the soul or the Tao or some other metaphor for the ineffa"le spark of life! still! as I sat there high in the "alcony of the 0tanford lecture hall! listening to the cy"erneticists claims to "e on the path toward the creation of a sentient "eing! I found myself muttering! Do! thats not right! were not 1ust mechanisms! youre missing something! theres something else! something more. But then I had to ask myself: 2hat else could there "e'

3ver the last half century! in addressing the &uestion -2hat are we humans'/ cy"ernetics has come up with three answers. 2e are! in order of their occurrence in the de"ate! :); computers! :5; ants! and :E; accidents. The first! the co identification of human sentience and the computer! appeared almost simultaneously with the appearance of computers. In )*F6! only four years after the construction of .DI(C! generally regarded as the first digital computer! the mathematician (lan Turing famously proposed that digital machines could think. (nd "y the time computers had come into general use! in the )*,6s! the view of the human "rain as an information processor was already firmly installed. It is an odd view! if you consider it. .DI(C was conceived as a giant calculator: it was designed to compute the tra1ectory of artillery shells. That is! its role was understood as human complement! doing well what we do poorly :tedious computation! precise recall of lists of num"ers and letters; and doing "adly what we do well :intuitive thinking! acute perception! reactions involving the comple7 interplay of mental! physical! and emotional states;. Get "y )*,*! when computers were still room si#ed! heat generating "ehemoths with "lock letter character screens! the computer scientist and Do"el Caureate in economics 8er"ert 0imon did not seem inclined to e7plain his premise when he wrote: -The computer is a mem"er of an

important family of artifacts called sym"ol systems. ... (nother important mem"er of the family :some of us think! anthropomorphically! it is the most important; is the human mind and "rain./ 0imon could "egin a thought "y saying! -If computers are organi#ed somewhat in the image of man/ without going on to &uestion that -if./ In e7istence "arely twenty five years! the machine that was designed to "e our other%the not human! accurate in a world where to "e human is to err%had "ecome the very analogue of human intelligence! the image of man. 0imon! along with his colleague (llen Dewell! was a pioneer in the field of artificial intelligence! and it is worthwhile now to look "ack at 0imons important "ook The Sciences of the Artificial, for here one can see the origins of the curious reasoning where"y the computer "ecomes a model for humanity. 0imon "egins "y discussing what on the surface might seem o"vious: the difference "etween the natural and the artificial worlds. Datural o"1ects have the authority of e7istence! he says9 the -laws/ of nature determine what must "e. The artificial! in contrast! is designed or composed in light of what might and ought to "e. But then 0imons reasoning takes an odd turn. 8e goes on to define artifacts as -an Hinterface in todays terms%"etween an Hinner environment! the su"stance and organi#ation of the artifact itself! and an Houter environment! the surroundings in which it operates/: that is! as "odiless processes mediating "etween inner and outer environments. Then: -Dotice that this way of viewing artifacts applies e&ually well to many things that are not man made%to all things in fact that can "e regarded as adapted to some situation9 and in particular it applies to the living systems that have evolved through the forces of organic evolution./ By the si7th page of his "ook! where this statement appears! human "eings have "een removed from the realm of the -natural./ @iewed as adapta"le products of evolution! we have "ecome hollow artifacts! interfaces to our environment! engineered -systems./ It is a startling turna"out: what is "eing proposed here is not the possi"ility of creating artificial life "ut the redefinition of life itself as artificial. 3nce you accept the definition of human life as artificial%designed! engineered%it is then an easy matter to say that the proper study of man is not man "ut some other engineered o"1ect! the machine. (nd this is indeed what 0imon advocates: making the computer itself the o"1ect of study! as a phenomenon of a living system. -0ince there are now many such devices in the world IcomputersJ! and since the properties that descri"e them also appear to "e shared "y the human central nervous system! nothing prevents us from developing a natural history of them. 2e can study them as we would ra""its or chipmunks and discover how they "ehave under different patterns of environmental stimulation./ 0tanding ama#ed "efore this human created machine! the computer scientist declares it to "e our very identity9 conse&uently! to learn who and what we are! he advises that we study ... the machine. This circular idea%the mind is like a computer9 study the computer to learn a"out the mind has infected decades of thinking in computer and cognitive science. 2e find it in the work of Marvin Minsky! the influential figure in artificial intelligence who! when asked if machines could think! famously answered: 3f course machines can think9 we can think and we are -meat machines./ (nd in the writings of cognitive scientist Daniel Dennett! whose "ook Consciousness E !lained is suffused with conflations "etween human sentience and computers: -2hat counts as the Hprogram when we talk of a virtual machine running on the "rains parallel hardware' ... 8ow

do these programs of millions of neural connection strengths get installed on the "rains computer'/ (nd in an e7treme version in the predictions of =ay Kur#weil! the inventor of the Kur#weil music synthesi#er and of reading systems for the "lind! who sees the coming of -spiritual machines/ almost entirely in the language of computer programming. Kur#weil calls memory the -mind file!/ and looks forward to the day when we can scan and -download/ the mind into a silicon su"strate! analy#ing it for its "asic -algorithms!/ there"y creating a -"ackup copy/ of the original human "eing! all without the aid of natural evolution! which he calls -a very inefficient programmer./ The limitations of this model of human intelligence should have "ecome clear with the demise of (I! that first naive "lush of optimism a"out the creation of sentient machines. In selecting the computer as the model of human thinking! (I researchers were addressing only one small portion of the mind: rational thought. They were! in essence! attempting a simulation of the neocorte7%rule "ased! conscious thinking%declaring that to "e the essence of intelligence. (nd (I did prove successful in creating programs that relied upon such rule "ased thinking! creating so called e7pert systems! which codified narrow! specific! e7pert domains! such as oil e7ploration and chess playing. But the results of early (I were generally disappointing! )1. Some observers of the chess match between Garr" #as!arov and $B%&s 'ee! Blue have used 'ee! Blue as an e am!le of an A$ !rogram that achieved something of the !resence we associate with sentience( )#as!arov re!orted signs of mind in the machine,* wrote Hans %oravec, the noted roboticist( $ believe that there was indeed one game in which, !robabl" due to some accidental combinatorial e !losion, 'ee! Blue did not !la" li+e a machine, as #as!arov re!orted at the time( #as!arov then ad,usted his !la", loo+ing for the strategies of that )mind,* which failed to rea!!ear( This !ut #as!arov off his game, and he !la"ed rather badl" -for him.( The !oint is that the !rogram had not attained sentience/ the human had !ro,ected sentience onto the machine, and became flustered( as the philosopher 8u"ert Dreyfus pointed out. They were systems devoid of presence and awareness! a -distur"ing failure to produce even the hint of a system with the fle7i"ility of a si7 month old child./ Get the idea of human "eing as computer lives on. Most trou"ling! even after "ecoming controversial in computer science! it has taken up residence in the natural sciences. The notion of human as computational machine is "ehind the stu""orn view of DD( as -code!/ and it endures in the idea of the "ody as mechanism. (s =odney Brooks put it in an article in 0ature1 -The current scientific view of living things is that they are machines whose components are "iomolecules./ The psychologist 0teven $inker! in the first pages of his classic "ook How the %ind 2or+s, writes that the pro"lems of understanding humans -are "oth design specs for a ro"ot and the su"1ect matter of psychology./ This view shows up in surprising places! for instance in an email I received from Cucia 4aco"s! a professor of psychology at Berkeley who studies s&uirrel "ehavior: -I am an ethologist and know virtually nothing a"out computers! simulations! programming! mathematical concepts or logic!/ she wrote. -But the research is pulling me right into the middle of it./ 8er"ert 0imons views have come full circle9 it is now standard scientific practice to study machine simulations as if they were indeed chipmunks! or s&uirrels. -2hat seems to "e crystalli#ing! in short!/ wrote 4aco"s of her work with ro"ots! -is a powerful outlook on spatial navigation! from ro"ots to human reasoning. This is wonderful./ $sychology and cognitive science%and indeed "iology%are thus poised to "ecome! in essence! "ranches of cy"ernetics.

?ailing to produce intelligence "y modeling the -higher functions/ of the corte7! cy"ernetics ne7t turned to a model creature without any corte7 at all: the ant. This seems like an odd place to look for human intelligence. (nts are not generally thought of as "eing particularly smart. But as a model they have one enormous advantage over human "rains: an e7planation of how apparent comple7ity can arise without an overseeing designer. ( group of dum" ants produces the comple7ity of the ant colony%an e7ample of organi#ational intelligence without recourse to the perennial difficulties of religion or philosophy. (gain! the source for this key idea seems to "e 8er"ert 0imon. The third chapter of The Sciences of the Artificial opens "y descri"ing an ant making its way across a "each: 2e watch an ant make his la"orious way across a wind and wave molded "each. 8e moves ahead! angles to the right to ease his clim" up a steep dune let! detours around a pe""le! stops for a moment to e7change information with a compatriot. Thus he makes his weaving! halting way "ack to his home. 0o as not to anthropomorphi#e a"out his purposes! I sketch the path on a piece of paper. It is a se&uence of irregular! angular segments%not &uite a random walk! for it has an underlying sense of direction! of aiming toward a goal. I show the unla"eled sketch to a friend. 2hose path is it' (n e7pert skier! perhaps! slaloming down a steep and somewhat rocky slope. 3r a sloop! "eating upwind in a channel dotted with islands or shoals. $erhaps it is a path in a more a"stract space: the course of a search of a student seeking the proof of a theorem in geometry. The ant leaves "ehind it a comple7 geometric pattern. 8ow' The ant has not designed this geometry. 0imons revolutionary idea was to locate the source of the comple7ity not in the ant! which is &uite simply -viewed as a "ehaving system!/ "ut in the ants interaction with its environment: in the "yplay of the ants simple! unaware reactions to complications of pe""le and sand. 0imon then goes on to pronounce what will turn out to "e inspirational words in the history of cy"ernetics: -In this chapter! I should like to e7plore this hypothesis "ut with the word Hhuman "eing su"stituted for Hant./ 2ith that! 0imon introduces an idea that will rever"erate for decades across the literature of ro"otics and artificial life. 3ne can hardly read a"out the su"1ect! or talk to a researcher! without coming upon the e7ample of the ant%or the "ee! or termite! or swarm! or some other such reference to the insect world. Its not at all clear that the later adopters of -the ant idea/ completely grasp the implications of 0imons view9 they seem to have dropped the difficulties of interaction with the environment! its vast comple7ity and varia"ility! concentrating instead on low level ant to ant communications. Get what was distilled out of 0imons utterance was a powerful model: (n ant colony is an intricate society! "ut the comple7ity comes into "eing without a ruling -god/ or mind to plan or direct it. The complicated order of the colony arises not from a"ove! not from a plan! "ut from "elow! as a result of many -dum"!/ one to one interactions "etween individual creatures. This phenomenon! known as -emergence!/ produces outcomes that cannot "e predicted from looking only at the underlying simple interactions. This is the key idea in fields known! variously! as -comple7ity theory!/ -chaos theory!/ and -cellular automata./ It is also a

foundational concept in ro"otics and (life. -.mergence/ lets researchers attempt to create intelligence from the "ottom up! as it were! starting not from any theory of the "rain as a whole "ut from the lowest level elementary processes. The idea seems to "e that if you construct a sufficient num"er of low level! atomic interactions :-automata/; what will eventually emerge is intelligence%an ant colony in the mind. 0entience is not a thing! according to this view! "ut a property that arises from the organi#ation of matter itself. 8ans Moravec writes: (ncient thinkers theori#ed that the animating principle that separated the living from the dead was a special kind of su"stance! a spirit. In the last century "iology! mathematics! and related sciences have gathered powerful evidence that the animating principle is not a su"stance! "ut a very particular! very comple7 organi#ation. 0uch organi#ation was once found only in "iological matter! "ut is now slowly appearing in our most comple7 machines. In short! given enough computing power! which gets easier every year as the computational a"ilities of chips increase e7ponentially! its possi"le to "uild a ro"otic creature that crosses some critical threshold in the num"er of low level! organi#ational interactions it is a"le to sustain. 3ut of which will emerge%like @enus surfacing from the sea on a half shell%sentience. There is a large flaw in this reasoning! however. Machines are indeed getting more and more powerful! as predicted "y former Intel chairman >ordon Moore in what is known as Moores Caw. But computers are not 1ust chips9 they also need the instructions that tell the chips what to do9 that is! the software. (nd there is no Moores Caw for software. Luite the contrary: as systems increase in comple7ity! it generally "ecomes harder to write relia"le code. The thinking of todays ro"oticists! like that of their predecessors in early (I efforts! is infected "y their vision of the computer itself! the machine as model human. (gain! they mistake the tool for its "uilder. In particular! the error comes from mistaking the current methods of software writing as a paradigm for human mental organi#ation. In the )*A6s a computer program was a centrali#ed! monolithic thing! a small world unto itself! a set of instructions operating upon a set of data. It should "e no surprise! then! that researchers at the time saw human intelligence as a ... centrali#ed! monolithic! logical mind operating upon the data in a -knowledge "ase./ By the )**6s that monolithic paradigm of programming had "een replaced "y something called -o"1ect oriented/ methods! in which code was written in discrete! atomic chunks that could "e com"ined in a variety of ways. (nd%what do you know'%human sentience is now seen as something emerging from the comple7 interaction of ... discrete! atomic chunks. Is cognitive science driving the science of computing! or is it the other way around' (nd there is a more fundamental pro"lem in using todays software methods as a paradigm for the emergence of human sentience: 0oftware presupposes the e7istence of a designing mind! whereas the scientific view is that human intelligence arose! through evolution! without a conscious plan. ( -little man!/ a homunculus! lives inside software. To write code! even using -o"1ect oriented/ methods that seek to work from the "ottom up! someone must have an overall conception of what is going on. (t some level there is an overriding theory! a plan! a predisposition! a container! a goal. To use the computer as a model! then%to "elieve that life

arises like the workings of a well programmed computer%is to posit! somewhere! the e7istence of a god. ( cy"ernetic re"uttal to this idea of a god would "e that the -program/ in the natural world%the organi#ational intelligence%is supplied "y Darwinian selection. That is! the human programmer takes over the work of -inefficient programmer/%evolution. But I think you cant have it "oth ways: Gou cant simultaneously say that you can program a -sentient/ ro"ot! freed from the pressures of survival and reproduction! and that the e&uivalent of programming in the natural world is natural selection! which is predicated on the pressures of reproduction and survival. Gou may say you are "uilding something%a mechanical o"1ect that simulates some aspects of human sentience! for instance%"ut you cannot say that the organi#ational principles of that mechanical o"1ect illuminate the real "ases of human sentience. The processes of engineering! particularly of programming! are not analogous to the processes of the natural world. ?or e7ample! the computer storage mechanisms that we call -memory/ do not illuminate the workings of human memory. (ccording to current research! the contents of human long term memory are dynamic. .ach time we recall something! it seems! we reevaluate it and reformulate it in light of everything relevant that has happened since we last thought a"out it. 2hat we then -remem"er!/ it turns out! is not the original event itself "ut some endless variant! ever changing in the light of e7perience. If a computers memory functioned that way! we would call it -"roken./ 2e rely upon machine memory not to change9 it is useful "ecause it is not like us. There are signs that even the cy"erneticists who promoted the concept of the "ottom up emergence as a paradigm for human sentience are sensing its limits. Christopher Cangton! a key figure in (life research! has admitted that there is the pro"lem of -finding the automata/9 that is! in deciding what indeed constitutes the lowest level interactions that must "e simulated in order to create life. 8ow deep must one go' To interactions "etween cells' molecules' atoms' elementary particles of matter' Talking with me at a cafe in Cin#! (ustria! Cangton looked up from a scri""led note"ook and said with sincere worry: -2heres the "ottom of physics'/ Meanwhile! the ro"oticist =odney Brooks was wondering a"out the -top/ of the pro"lem! the higher level cognitive functions that the theory of emergence seeks to portray as an effect of an organisms organi#ation. 0entience! after all! entails conscious action! intention! what we call -free will./ (fter years spent creating ro"ots that were like insects! Brooks recogni#ed that something else is involved in the grand pro1ect of intelligence. 8e is now revisiting the pro"lem that stumped (I researchers in the )*A6s: finding a way to give the cy"ernetic creature some internal representation of the world. -2ere trying!/ said Brooks! -to introduce a theory of mind./ The "ottom of physics! a theory of mind. 8ere we go again. Back we are drawn into the metaphysical thickets from which engineering empiricism hoped permanently to flee. The hope was to turn sentience into a pro"lem not of philosophy! or even of science! "ut of engineering. -Gou dont have to understand thought to make a mind!/ said Douglas 8ofstadter! wishfully! while introducing the spiritual ro"ot panel at 0tanford. -The definition of life is hard!/ =odney Brooks said to me. -Gou could spend five hundred years thinking a"out it or spend a few years doing it./ (nd here is the underlying motive of ro"otics: an anti intellectualism in search of the intellect! a flight from introspection! the desire to "anish the horrid muddle of this -thinking

a"out it!/ thousands of years of philosophical speculation a"out what animates us without nota"le progress. -Gou can understand humans either "y reverse engineering or through "uilding!/ says Cynthia Brea#eal! a young ro"oticist at MIT. In other words! dont think a"out it! "uild it9 e&uate programming with knowledge. Get still we circle "ack to the old confusions! for conceptuali#ation is as deep in our human nature as tool "uilding! Homo faber wrestling with Homo sa!iens(

3ne way to get around the difficulties of human sentience is to declare humans all "ut irrelevant to the definition of life. This is the approach taken "y (life researchers! who see human "eings! indeed all life on .arth! as -accidents!/ part of the -highly accidental set of entities that nature happened to leave around for us to study./ (s Christopher Cangton writes in his introduction to Artificial 3ife1 An 4verview, -The set of "iological entities provided to us "y nature! "road and diverse as it is! is dominated "y accident and historical contingency. ... 2e sense that the evolutionary tra1ectory that did in fact occur on earth is 1ust one out of a vast ensem"le of possi"le evolutionary tra1ectories./ Based on the same foundations as modern ro"otics%emergence theory%(lifes goal is the creation of software programs that e7hi"it the properties of "eing alive! what is called -synthetic "iology!/ the idea "eing that researchers can learn more a"out life -in principle/ if they free themselves of the specific conditions that gave rise to it on .arth. (life research says farewell to the entire natural world! the what must "e! in 8er"ert 0imons formulation! with "arely a "ackward glance :e7cept to occasionally cite the e7ample of ants;. -Cife/ in the conte7t of (life is defined very simply and a"stractly. 8ere is one typical approach: -My private list Iof the properties of lifeJ contains only two items: self replication and open ended evolution./ (nd another: -Cife must have something to do with functional properties ... we call ada!tive, even though we dont yet know what those are./ Bruce Blum"erg! an MIT researcher who creates ro"otic dogs in software animations! descri"es the stance of (life this way: -2ork has "een done without reference to the world. Its hard to get students to look at phenomena. Its artificial life! "ut people arent looking at life./ 2hat (life researchers create are computer programs%not ro"ots! not machines! only software. The cy"ernetic creatures in these programs :-agents/ or -automata/; go on to -reproduce/ and -adapt!/ and are therefore considered in principle to "e as alive as we are. 0o does the image of the computer as human paradigm! "egun in the )*F6s! come to its logical e7treme: pure software! unsullied "y e7igencies of car"on atoms! "odies! fuel! gravity! heat! or any other messy concern of either soft tissued or metal "odied creatures. (gain the image of the computer is conflated with the idea of "eing alive! until only the computer remains: life that e7ists only in the machine.

2hat these views of human sentience have in common! and why they fail to descri"e us! is their disdain for the "ody: the utter lack of a "ody in early (I and in later formulations like Kur#weils :the lonely corte7! scanned and downloaded! a "rain in a 1ar;9 and the disregard for this "ody! this mammalian flesh! in ro"otics and (life. .arly researchers were straightforward a"out discarding the flesh. Marvin Minsky pronounced us to "e -meat machines./ -Instead of trying to consider

the Hwhole person! fully e&uipped with glands and viscera!/ said 8er"ert 0imon! -I should like to limit the discussion to 8omo sapiens! Hthinking person. Meat and glands and viscera%you can sense the corruption implied here! the "ody as "utchery fodder! polluting the discussion of intelligence. This suspicion of the flesh! this &uest for a disem"odied intelligence! persists today. =ay Kur#weil "rushes aside the physical life as irrelevant to the pro1ect of "uilding -spiritual/ "eings: -Mammalian neurons are marvelous creations! "ut we wouldnt "uild them the same way. Much of their comple7ity is devoted to supporting their own life processes! not to their information handling a"ilities./ In his view! -life/ and -information handling/ are not synonymous9 indeed -life/ gets in the way. 8e sees evolution as -inefficient!/ -a sloppy programmer!/ producing DD( that is mostly -useless./ (nd (life researchers! seeing -life/ in their computer programs! pay no attention at all to the "ody! imagining that the properties of life can somehow! like tissue specimens! "e cut away from the dross of living: 2hether we consider a system living "ecause it e7hi"its some property that is uni&ue to life amounts to a semantic issue. 2hat is more important is that we recogni#e that it is possi"le to create disem"odied "ut genuine instances of specific properties of life in artificial systems. This capa"ility is a powerful research tool. By separating the property of life that we choose to study from the many other comple7ities of natural living systems! we make it easier to manipulate and o"serve the property of interest. 3ne might think that ro"otics! having as it does the imperative of creating some sort of physical container for intelligence! would have more regard for the human "ody. But the entire pro1ect of ro"otics%the engineering of intelligent machines%is predicated on the "elief that sentience is separa"le from its original su"strate. I had a talk with Cynthia Brea#eal! who was a student of =odney Brooks and is now on the faculty of the MIT Media Ca". Brea#eal is a thoughtful researcher. 8er work involves the creation of ro"ots that respond to human "eings with simulated emotional reactions! and she shows a sincere regard for the emotional life. Get even she revealed an underlying disgust for the "ody. >rowing impatient with me as I pressed her for a definition of -alive!/ she said: -Do you have to go to the "athroom and eat to "e alive'/ The &uestion stayed with me%do you have to go to the "athroom and eat to "e alive'%"ecause Brea#eals o"vious intent was to pick what she considered the most "ase part of life! to make it seem ridiculous! humiliating even. But after a while I came to the conclusion: may"e yes. >iven the amount of time living creatures devote to food and its attendant states%foodM the stuff that sustains us%I decided that! yes! there might "e something crucial a"out the necessities of eating and eliminating that defines us. 8ow much of our state of "eing is dependent upon "eing hungry! eating! having eaten! "eing full! shitting. 8ungerM 3ur word for everything from nourishment to passionate desire. 0atisfiedM Meaning everything from well fed to se7ually fulfilled to mentally soothed. 0hitM 3ur word for human waste and our e7pletive of impatience. The more I thought a"out it! the more I decided that there are huge swaths of e7istence that would "e impenetra"le indescri"a"le! unprogramma"le! utterly una"le to "e represented%to a creature that did not eat or shit.

In this sense! artificial life researchers are as "ody loathing as any medieval theologian. They seek to separate the -principles/ of life and sentience%the spirit%from the dirty muck it sprang from. (s Brea#eal puts it! they envision a -set of animate &ualities that have nothing to do with reproduction and going to the "athroom!/ as if these messy e7periences of alimentation and "irth! these deepest "iological imperatives%stay alive! eat! create others who will stay alive%were not the foundation! indeed the source! of intelligence9 as if intelligence were not simply one of the many strategies that evolved to serve the striving for life. If sentience doesnt come from the "odys desire to live :and not 1ust from any physical "ody! from this "odys striving! this particular one;! where else would it come from' To "elieve that sentience can arise from anywhere else%machines! software! things with no fear of death%is to "elieve! ipso facto! in the separa"ility of mind and matter! flesh and spirit! "ody and soul.

8ere is what I think: 0entience is the crest of the "ody! not its crown. It is integral to the su"strate from which it arose! not something that can "e taken off and placed elsewhere. 2e drag along inside us the "rains of reptiles! the tails of tadpoles! the DD( of fungi and mice9 our cells are permuted paramecia9 our salty "lood is whats left of our "irth in the sea. >enetically! we are "arely more than roundworms. .volution! that sloppy programmer! has seen fit to create us as a wild amalgam of everything that came "efore us: the whole history of life on .arth lives on! written in our "odies. (nd who is to say which piece of this history can "e e7cised! separated! deemed -useless/ as an essential part of our nature and "eing' The "ody is even the source of a"stract reasoning! usually thought of as the very opposite of the flesh! according to the linguists >eorge Cakoff and Mark 4ohnson. -This is not 1ust the innocuous and o"vious claim that we need a "ody to reason!/ they write! -rather! it is the striking claim that the very structure of reason itself comes from the details of our em"odiment./ If we were made out of some other -details/%say! wire and silicon instead of sinew and "one%we might indeed have something called logic! "ut we would not necessarily recogni#e it as anything intelligent. It is this "ody! this particular fleshly form! that gave "irth to the thing we call intelligence. (nd what I mean "y this particular form is not 1ust that of human "eings or even primates. It is our e7istence as mammals. 3ddly! in all the views of human intelligence promulgated "y cy"ernetics! this is the one rarely heralded: what we call sentience is a product of mammalian life. Mammalian life is social and relational. 2hat defines the mammalian class! physiologically! is not dependence on the female mammary gland or egg laying "ut the possession of a portion of the "rain known as the lim"ic system! which allows us to do what other animals cannot: read the interior states of others of our kind. To survive! we need to know our own inner state and those of others! &uickly! at a glance! deeply. This is the -something/ we see in the eyes of another mammalian creature: the a"ility to look at the other and know that he or she has feelings! states! desires! that are different from our own9 the a"ility to see the other creature looking "ack at us! "oth of us knowing we are separate "eings who nonetheless communicate. This is what people mean when they say they communicate with their dogs or cats! horses or "unnies: mammals reading each other. 2e dont go looking for this in ants or fish or reptiles9 indeed! when we want to say that someone lacks that essential spark of life! we call him -reptilian./ 2hat we mean "y

this is that he lacks emotions! the a"ility to relay and read the emotions of others9 that he is! in short! ro"otic. If sentience is a mammalian trait! and what distinguishes mammals is the capacity for social life! then sentience must have its root in the capacity for rich social and emotional interchange. That is! sentience "egins with social life! with the a"ility of two creatures to transact their inner states %needs! desires! motivations! fears! threats! contentment! suffering! what we call -the emotions./ Moreover! the more avenues a creature has for understanding and e7pressing its emotional states! the more intelligent we say it is. (nts were not a good place to look for rich social interchange9 the logical inference engines of early (I were a particularly poor choice of model9 computer software running in the astringent purity of a machine wont find it. To get at the heart of intelligence! we should have started "y looking at the part of human life ordinarily considered -irrational!/ the opposite of -logical!/ that perennial pro"lem for computers: emotions.

0ome ro"otics researchers are "eginning an investigation into the ways that a mechanical o"1ect can have! or appear to have! an emotional and social e7istence. -Most ro"oticists couldnt care less a"out emotions!/ says Cynthia Brea#eal! one of the few researchers who does care a"out emotions. 8er -Kismet/ ro"ot is a very cute device with the face and ears of a ?ur"y. It has simulated emotional states :e7pressed adora"ly! floppy ears drooping pitiously when its sad;9 it is designed to interact with and learn from humans as would a human child. -0ocial intelligence uses the whole "rain!/ she says. -It is not devoid of motivation! not devoid of emotion. 2ere not cold inference machines. .motions are critical to our rational thinking./ =odney Brooks speaks of adding -an emotional model!/ of giving his new ro"ot -an understanding of other people./ Cynthia Brea#eals Kismet is designed to suffer if it doesnt get human attention! and to care a"out its own well "eing. Bruce Blum"erg! who -does dogs!/ as he puts it! understands that -you cant say youre modeling dogs without social "ehavior./ 3f the three! however! only Blum"erg seems to grasp the si#e of the pro"lem theyre undertaking! to "e willing to admit that there is something ineffa"le a"out a living "eings social and emotional e7istence. -My approach is to "uild computer devices to catch a spark of whats really there in the creature!/ he says! -to understand what makes dogs%and us%have a sort of a magical &uality./ Then! perhaps em"arrassed at this recourse to magic! he adds! -Dinety nine percent of computer scientists would say youre no computer scientist if you were talking in terms of Hmagical &ualities./ Indeed! his colleague Brea#eal has a pragmatic! even cynical! view of the emotions. =o"ots will need to have something like emotions! she says! "ecause corporations are now investing heavily in ro"otic research and -emotions "ecome critical for people to interact with ro"ots%or you wont sell many of them./ The point seems to "e to fool humans. ("out her ro"ot Kismet she says! -2ere trying to play the same game that human infants are playing. They learn "ecause they solicit reactions from adults./ But a human infants need for attention is not simply a -game./ There is a true! internal reality that precedes the childs interchange with an adult! an actual inner state that is "eing

communicated. (n infants need for a mothers care is dire! a physical imperative! a &uestion of life or death. It goes "eyond the re&uirement for food9 an infant must learn from adults to survive in the world. But without a "ody at risk! in a creature who cannot die! are the programming routines Brea#eal has given Kismet even analogous to human emotions' Can a creature whose flesh cant hurt feel fear' Can it -suffer/' .ven if we leave aside the &uestion of em"odiment! even if we agree to sail away from the philosophical shoals of what it means to really have an emotion as opposed to 1ust appearing to have one! the &uestion remains: 8ow close are these researchers to constructing even a rich simulation of mammalian emotional and social life' ?urther away than they reali#e! I think. The more the MIT researchers talk a"out their work! the longer grows the list of thorny &uestions they know they will have to address. -Is social "ehavior simply an ela"oration of the individual'/ asks Blum"erg. -2hat does the personality really mean'/ -2e need a model of motivation and desires./ -8ow much of life is like that%pro1ection'/ ?rom Brea#eal: -8ow do you "uild a system that "uilds its own mind through e7perience'/ (nd this great conundrum: -( creature needs a self for social intelligence%what the hell is that'/ In turning to the emotions and social life! they have hit right up against what Brea#eal calls the -limiting factor: "ig ideas./ Theories of learning! "rain development! the personality! social interaction! motivations! desires! the self%essentially the whole of neurology! physiology! psychology! sociology! anthropology! and 1ust a "it of philosophy. 3h! 1ust that. It all reminded me of the sweet engineering naNvetO of Marvin Minsky! "ack in the early days of (I! when he offhandedly suggested that the field would need to learn something a"out the nature of common sense. -2e need a serious epistemological research effort in this area!/ he said! "elieving it would "e accomplished shortly.

3f course the "iggest of the -"ig ideas/ is that old "uga"oo: consciousness. Difficult! fu##y! and unwilling to yield up its secrets despite thousands of years devoted to studying it! consciousness is something ro"otics researchers would rather not discuss. -In our group! we call it the C word!/ says =odney Brooks. Consciousness! of course! is a pro"lem for ro"ots. Besides "eing hard to simulate! the very idea of consciousness implies something unfathoma"ly uni&ue a"out each individual! a self! that -magical &uality/ Bruce Blum"erg is daring enough to mention. Brookss impulse! like that of his former student Cynthia Brea#eal! is to view the interior life cynically! as a game! a "unch of foolery designed to elicit a response. Brooks is an ur"ane and charming man. 8e speaks with a soft (ustralian accent and seems genuinely interested in e7changing thoughts a"out arcane matters of human e7istence. 8e sat with me at a small conference ta"le in his office at MIT! where photographs of his insectlike ro"ots hung on the walls! and piled in the corner among some "ooks was the ro"otic doll called -My =eal Ba"y/ he had made for the 8as"ro toy company. I mentioned Brea#eals Kismet! told him I thought it was designed to play on human emotions. Then I asked him: -(re we 1ust a set of tricks'/ 8e answered immediately. -I think so. I think youre a "unch of tricks and Im 1ust a "unch of tricks./

Trickery is deeply em"edded in the fa"ric of computer science. The test of machine intelligence that (lan Turing proposed in )*F6! now known as -the Turing Test!/ was all a"out fooling the human. The idea was this: 8ave a human "eing interact with what might "e either another human or a computer. $lace that first human "ehind some metaphorical curtain! a"le to see the te7t of the responses "ut una"le to see who or what -said/ them. If that human "eing cannot then tell if the responses came from a person or a machine! then the machine could "e 1udged to "e intelligent. ( circus stunt! if you will. ( 2i#ard of 3# game. ( trick. To think otherwise! to think there was something more to intelligence than 1ust the perception of a fooled human "eing! would "e to "elieve there was some essence! a -something else!/ in there. 4ust then! as I sat in Brookss office! I didnt much feel like a "unch of tricks. I didnt want to think of myself as what he had descri"ed as -1ust molecules! positions! velocity! physics! properties%and nothing else./ 8e would say this was my reluctance to give up my -specialness/9 he would remind me that it was hard at first for humans to accept that they descended from apes. But I was aware of something else in me protesting this idea of the empty person. It was the same sensation Id had while at the spiritual ro"ot symposium hosted "y Douglas 8ofstadter! an internal round and round hum that went! Do! no! no! thats not it! youre missing something. I asked Brooks a"out the purpose of consciousness. -I dont know!/ he answered. -Do you know what consciousness is good for'/ 2ithout hesitation! I told him that! yes! I did know what consciousness is good for. I told him we are "orn helpless and defenseless. 3ur only hope to survive is to make contact with other humans. 2e must learn to tell one individual from another! make alliances! immediately see on the face of another human "eing whether this is friend or foe! kin or stranger. I told him that I think human e7istence as a species is predicated on this we" of social interactions! and for this we must learn to identify individuals. (nd out of that! the recognition of the identity of others! comes our own identity! the sense that we e7ist! ourselves! our self. .verything we call consciousness unwinds from that. -Its not mystical!/ I told him. -Its an evolutionary imperative! a matter of life and death./ Brooks put his chin on his hand and stared at me for a moment. Then he said: -8uh. Done of our ro"ots can recogni#e their own kind./

It took me a while! "ut after thinking a"out =odney Brookss remark a"out ro"ots and their own kind! my round and round humming an7iety%that voice in me that kept protesting! Do! no! youre missing something%finally stopped. ?or there it was! the answer I was looking for! the missing something else: recognition of our own kind. This is the -magical &uality/%mutual recognition! the moment when two creatures recogni#e each other from among all others. This is what we call -presence/ in another creature: the fact that it knows us! and knows we know it in turn. If that other "eing were 1ust a trick! 1ust the product of a set of mechanisms! you would think that snakes could make this recognition! or paramecia! or li#ards! or fish. Their "odies are full of marvelous mechanisms! refle7es! sensors! to give them an awareness of the world around them. (nt pheromones should work. =o"ots with transponders "eaming out their serial num"ers should do the 1o". But we are! as Cynthia

Brea#eal said! creatures whose "rains are formed "y learning9 that is! through e7perience and social interaction. 2e dont merely send out signals to identify ourselves9 we create one anothers identity. It is true that the idea of the human "eing as a unity is not an entirely accurate concept. Most of our intelligence is unconscious! not availa"le for introspection! having an independent e7istence! so to speak. (nd the "ody itself is not a unity! "eing instead a complicated colony of cells and sym"iotic creatures. 2e cant live without "acteria in our gut9 tiny creatures live on our skin and eyelids9 viruses have incorporated themselves into our cells. 2ere walking #oos. Get somehow! for our own survival :and pleasure; it is critical that we attain a unified view of ourselves as uni&ue selves. But I dont think this idea of "eing a uni&ue self is 1ust some chauvinistic sense of specialness! some ego pro"lem we have to let go of. Dature has gone to a great deal of trou"le to make her creatures distinct from one another. The chromosomes purposely mi7 themselves up in the reproductive cells. Through the wonder of natural DD( recom"ination! nearly every human "eing on .arth is distinct from every other. This recom"ining of the genetic material is usually thought of as creating diversity! "ut the corollary effect is the creation of uni&ueness. Twins fascinate us for this reason: "ecause they are rare! the only humans on .arth without their own faces. 2ere "orn distinct and! as our "rains develop in the light of e7perience! we grow ever more different from one another. Mammalian life takes advantage of this fact! "asing our survival on our a"ility to tell one from another! on forming societies "ased on those mutual recognitions. <ni&ueness! individuality! specialness! is inherent to our strategy for living. Its not 1ust a trick: there really is someone different in there. (I researchers who are looking at social life are certainly on the right path in the search to understand sentience. But until they grasp the centrality of identity! I dont think theyll find what theyre looking for. (nd then! of course! even supposing they grant that there is something called an identity! a uni&ue constellation of "ody and e7perience that somehow makes a creature a someone! a self%even then! theyll still have to find a way to program it. Their task in simulating a self identifying sentient creature will "e a little like trying to simulate a hurricane. Think a"out how weather simulations work. <na"le to take into account all of the comple7ity that goes into the production of weather :the whole world! essentially;! simulations use some su"set of that comple7ity and are a"le to do a fairly good 1o" of predicting what will happen in the ne7t hours or days. But as you move out in time! or at the e7tremes of weather! the model "reaks down. (fter three days! the predictions "egin to fail9 after ten! the simulation no longer works at all. The fiercer the storm! the less useful the simulation. 8urricanes are not something you predict9 theyre something you watch. (nd that is what human sentience is: a hurricane%too comple7 to understand fully "y rational means! something we o"serve! marvel at! fear. In the end we give up and call it an -act of >od./