Neural Networks 22 (2009) 190–196

Contents lists available at ScienceDirect

Neural Networks

journal homepage: www.elsevier.com/locate/neunet

2009 Special Issue

Embodying meaning: Insights from primates, autism, and BrentanoJuan Carlos GómezSchool of Psychology, University of St. Andrews, United Kingdom

a r t i c l e i n f o

Keywords:MeaningTheory of mindIntentionalityGaze followingPrimates

a b s t r a c t

In this paper I offer an analysis of social meaning in terms of coding intentional relations between agentsand objects. This analysis is amenable to embodied interpretations of intentionality and Theory of mindand may prove of help for the task of simulating the attribution of intentionality with neural networks.Work with non-human primates suggests that evolution first dealt with the problem domain of socialmeaning by selecting specific adaptations for coding behaviour as intentional in the sense of Brentano,i.e., as directed to or being about objects (for example, gaze following behaviours). Work with autismsuggests that alterations in these basic mechanisms of intentional attribution may explain some of themost prevalent problems with social meaning in this condition. Brentanian mechanisms for encodingintentional relations between agents and objects are more basic and evolutionarily more primitivethan representational theories of mind. However, some of the key components of representationaltheories of mind may be already present in the intentional abilities of primates, suggesting that thedecisive evolutionary step may have consisted of a re-organization of existing abilities rather than thecreation of new skills. Neural network models and models of the evolution and development of primitivemechanisms for intentional encoding may benefit from mutual cross-fertilization.

© 2009 Elsevier Ltd. All rights reserved.

1. Introduction: Meanings of meaning

Meaning is a complex and multifaceted notion, ranging fromsimple instances of the type ‘‘clouds mean rain’’, based on therecognition of natural physical connections (what Grice (1957)called Natural Meaning), to more complex cases such as what Imean when I point to the clouds outside my window and say‘‘it’s going to rain’’ implying that we should cancel our picnicplans. The latter is an instance of what Grice called Non-NaturalMeaning, where meaning is achieved through an interplay ofcomplex cognitive intentions. In Grice’s own words, in Non-Natural meaning‘‘A (an agent) means something by x (an utterance or gesture) if

and only if A intends the utterance or gesture x to produce some effectin an audience by means of the recognition of this intention’’ (Grice,1957, p. 386).In between these extreme examples of simple physicalmeaning

and complex cognitive meaning there are intermediate casesof evolved communicative signals such as animals’ alarm calls(e.g., Vervet monkeys alerting others of the presence of a snakeor a leopard with distinct alarm vocalizations) or facial emotionalexpressions, which, while being natural, do not necessarily reflectphysical or mechanical causation, but at the same time neednot involve complex exercises of embedded attribution of mentalstates.

E-mail address: Jg5@st-andews.ac.uk.

0893-6080/$ – see front matter© 2009 Elsevier Ltd. All rights reserved.doi:10.1016/j.neunet.2009.01.010

However, all these examples of meaning seem to havesomething in common. In all of them we go from one event X toa different event Y: from clouds to rain, from words and gesturesto the idea of canceling a picnic, from alarm vocalizations to thepresence of a predator, from facial configurations to emotions.Meaning could therefore be simply defined as ‘‘A perceived relationbetween two phenomena, such that one points to the other’’. Thispointing relation is not a physical mechanism in itself. Even ininstances of Natural Meaning based upon physical systems suchas clouds/rain; it is an observer who thinks of the rain beforeit actually occurs. Meaning is therefore a cognitive or mentalinterpretation of the physical world (be it non-living things likeclouds or the behaviour of living organisms). In all cases it isminds(human or animal) that give meaning to things by making themental journey from X to Y.This is why meaning becomes especially intricate when it is

social meaning, i.e., when the X in ‘‘X >> Y’’ is another organismwith its own mind. For example, in Grice’s Non-natural meaningone and the same behaviour (pointing to the clouds while saying‘‘it’s going to rain’’) may have different meanings. It could imply asuggestion to cancel the picnic, but I could also be just remindingeveryone to take their umbrellas, or making the point that thelaundry should be taken inside the house, or that I was rightwhen I warned that it was going to rain, and so on. One and thesame physical cue may be attributed different meanings. In thisview, social meaning is essentially a ‘‘disembodied’’ phenomenon,where we must understand the unobservable mental intentionswith which observable behaviours are produced. How can we dealwith those hidden intentions that give meaning to behaviour?

J.C. Gómez / Neural Networks 22 (2009) 190–196 191

1.1. Theories of mind

A recent solution to the problem of how, despite theirunobservability, we manage to deal with intentions and othermental states in everyday interaction has been the assumptionthat we possess Theories of mind—a specialized ability to readobservable behaviour in terms of unobservable mental states.This ability comes to us either innately or as part of a moregeneral ability to think of the world in theoretical terms (i.e., interms of hypothesized unobservables that explain the empiricalobservables) (see, for example, Carruthers and Smith (1996)).With Theories of mind we imagine the hidden mental states thatguide other people’s behaviour (for example, the communicativeintention behindmyutterance ‘‘it is raining’’ or behindmypointingto the window). In this view, meaning is hidden in the head of theother and we must figure it out by imagining what is controllinghis/her behaviour from inside.Indeed in Autism, severe problems with communication and

appreciating socialmeaning in general go hand in handwith severeproblems in Theory of mind (see, for example, Frith (2003)). Evenhigh-functioning individuals with autism (who understand wellthe literal meaning of words [‘‘it is raining’’]) may have problemsunderstanding the implied, un-utteredmeaningwithwhichwordsare so frequently used (‘‘let’s cancel the picnic’’). The case ofAutism, with their problems going beyond the literal surface ofbehaviour into the unobservable realms of mental meaning, lendssupport to the Theory of mind view of social meaning.

1.2. Non-human meaning

However, the attempt to explain socialmeaning throughTheoryof mind creates a problem for our understanding of simplerinstances of communication, especially in non-humans. On theone hand, contemporary animal research reveals a complex typeof social meaning in animal communication: referential signals,i.e., signals that draw the attention of the receiver upon specificobjects in the environment (Zuberbuhler, 2003). For example, avariety of old-world monkeys produce alarm calls associated withdifferent types of predators (one type for terrestrial predators suchas leopards; another type for aerial predators such as eagles). Incarefully controlled playback experiments the monkeys behaveas if they are indeed inferring the appropriate referent from thecalls. For example, in response to playback calls, they not onlyproduce an appropriate escape response, but also visually scaneither the sky or the ground in search of the appropriate referentpredator. The monkeys seem to make a meaningful interpretationof the calls, in the sense that their minds go from the callto a representation or expectation of the appropriate predator.Similarly captive chimpanzees and apes in general use whole-hand pointing gestures to request things or direct people to targetsin ways that closely resemble some referential uses of humanpointing (Gomez, 2007; Leavens & Hopkins, 1999; Zimmermannet al., 2009).But this evidence for referential meaning in monkeys and apes

co-exists with controversial evidence about their Theory of mindskills. Cheney and Seyfarth (1995) bluntly concluded that theirmonkeys were not using anything similar to Gricean meaning,because they did not seem to take into account others’ knowledgeor intentions. Experimental results suggest that even our closestrelatives, chimpanzees, either completely lack ‘‘anything remotelyresembling’’ Theories of mind (Penn & Povinelli, 2007) or, lessradically, understand only a limited range of simple mental states,not including communicative intentions (Tomasello et al., 2003).To make things more complicated, there is a sub-group of

high-functioning people with autism who, despite a seeminglyintact ability to attribute mental states to others (as evidenced bytheir passing laboratory Theory of mind tests), still have problemswith non-literal communication, or even more interestingly, show

problems with the spontaneous understanding and production ofsimple instances of referential communication such as pointinggestures or gaze (Leekam et al., 1997). Their Theory of mindcompetence does not seem to translate into an ability tounderstand simple instances of social meaning.How can we reconcile the evidence for referential communica-

tion in non-humans with the evidence for lack of the type of The-ory of mind associated with social meaning in humans? How canwe reconcile the evidence for Theory of mind skills in some high-functioning individuals with Autism with their problems under-standing the meaning of simple social signals?1

2. Brentano and the intentionality of mind

The Theory ofmind approach to socialmeaning is deeply rootedin a Cartesian view of cognition, where mental phenomena areseen as radically different frombehaviour. Descartes proposed thatthe uniqueness of minds was explained by their being made of analtogether different type of substance (res cogitans) in contrastwiththe res extensa of which the physical world was made. Of course,modern cognitive science assumes thatmental phenomena are justfunctions of res extensa (the neural networks of the brain), not aseparate type of matter. However, it still keeps a sharp conceptualdistinction between behaviour and mental phenomena, withthe latter being private, unobservable, detachable from theirphysical manifestations, but ultimately the cause and explanationof observable behaviour. This Cartesian distinction has remainedespecially influential in the notion of Theory of mind, wheremany authors insist that knowing behaviour is different toknowing minds, and different mechanisms are required for oneand the other (Heyes, 1998; Penn & Povinelli, 2007). In thisview, social meaning is a disembodied phenomenon, because itdepends on how the mind intends and represents the physicalworld. Because we cannot directly observe others’ intentions andrepresentations, to understand meaning we must, so to speak,‘‘overcome’’ behaviour.There is, however, a different view about what constitutes the

essence of mind. In the late XIX century the German/Austrianphilosopher Franz Brentano proposed that the mark of the mentalwas the property of intentionality—being about or pointing tothings other than itself (e.g., Crane (1998)). Mental states arealways directed at some target. We think of something, remembersomething, desire something—our mental exertions always referto some (real or imaginary) object. Note that intentional heremeans ‘‘pointed to or directed at something’’. This is different tobeing intentional in the sense of being purposeful or ‘‘intended’’,which is the more common meaning associated with thisword. In Brentano’s sense of ‘‘intentional’’ even our involuntarymental states (e.g., unwittingly being reminded of something) areintentional, in that they refer to something.This perspective is in principle compatible with Cartesian

dualism: one can argue that intentionality is achieved throughprivate, unobservable representations. Indeed the objects ofmental states that Brentano discussed, rather than real objectsin the world, were ‘‘objects in the mind’’. Moreover, one of thephilosophical outgrowths of Brentanowas phenomenology, whichinmany respects takes Cartesian dualism to the extreme. However,I propose that Brentano’s intuition of intentionality as the mark ofthe mental can be developed in a different direction—one in whichdualism becomes irrelevant and intentionality is conceptualized asan ‘‘embodied’’ relation with ‘‘potentially real’’ objects (see Fig. 1).

1 It is unclear if High-functioning people with autism who pass Theory ofmind tests do so with the same mechanisms as typical people or using somecompensatory strategy (Frith, 2003).

192 J.C. Gómez / Neural Networks 22 (2009) 190–196

Fig. 1. How do organisms give meaning to behaviour? Two views are contrasted here. (A) In a Cartesian view organisms must imagine that others acquire representationsor images of objects in their heads (the thought bubble metaphor) and these representations guide their behaviour; organisms must possess some sort of representationalTheory ofmind to perceive themeaning of gaze andpointing in the example. (B) In the Brentanian view sketchedhere, organismsdetect that others form intentional or directedrelations with objects. This can be understood as an external configurational relation (the arrowmetaphor) without attributing intermediate, unobservable representations.See text for more details. (Base figure taken from Descartes (1662), with which he illustrated how the immaterial soul had to mediate between sensation and action in thepineal gland: Thought bubble, arrows, and objects added.)

The emphasis upon a relation of directed pointing betweena subject and an object as the key property of the mental hasimportant consequences for our understanding of social meaningin an evolutionary perspective. It may also have consequencesfor the problem of how to simulate social meaning with neuralnetwork models. An evolutionary analysis of how evolution foundways of dealing with social meaning may help informing efforts toconstruct networks capable of simulating social meaning.The challenge for a Cartesian organism dealing with social

meaning is to figure out something utterly unobservable – others’representations of the world – by computing entities that existwith independence of the sets of observable cues (behaviours)the organism has to react to. Being independent, the observedcues frequently do not correlate with their unobservable causes(e.g., the different meanings that one and the same utterance mayhave). In contrast, the challenge for a Brentanian organism is todetect relations between potentially observable entities (agentsand their objects of action and attention) on the basis of cuesthat usually correlate with their outcomes (e.g., directed graspingand gaze direction; see below). This challenge is more tractablefor evolution and presumably for simulation. Indeed my thesisis that evolution has produced a variety of mechanisms to dealwith social meaning that can be interpreted as direct adaptationsto the intentional relations (in Brentano’ sense) that guide animalbehaviour without relying upon the computation of unobservablemental states as intermediate variables. Let me illustrate thisapproach with a paradigmatic case—gaze following.

3. A prototypical case: Gaze and attention

Vision is one of the primary ways of relating to the world inmany animal species, especially so in primates, that have evolvedsophisticated colour and stereoscopic vision and expanded brainareas specialized in the processing of visual stimuli (Gomez, 2004).Vision is not a passive affair, but one in which animals activelylook at the surrounding world. This involves an orientation of theeyes in particular directions that normally requires a concurrentorientation of the head or even the body, especially so in animalslike primates that have their eyes frontally located in the head

and with considerable overlap of their visual fields. This variablebehavioural constellation is what we call gaze.Gaze is intentionally (in Brentano’s sense) oriented to particular

objects in the environment and this orientation is physically ad-vertised throughmany bodily cues (not only the eyes), that almostinevitably correlate with the neural processing of visual stimuli.Evolutionary processes appear to have capitalized on this fact byselecting the so-called ‘‘gaze following’’ behaviours—behaviouralreactions of looking in the same direction as other animals in re-sponse to a variety of gaze cues. Virtually all primates experimen-tally tested so far show gaze following reactions in response tocon-specifics or humans looking in particular directions (Gomez,2005). The cues they are preferentially sensitive to may vary. Forexample, great apes may be maximally sensitive to eye/head ori-entations and less sensitive to eye direction on its own when thisconflicts with head direction (Tomasello et al., 2007). Lemurs – as-sumed to reflect the earliest stages of primate evolution – are atleast sensitive to head direction and can distinguish this from bodyorientation when both cues conflict (Ruiz, Gómez, & Byrne, sub-mitted for publication). Humans, in turn, are very sensitive to eyedirection evenwhen it conflicts with head direction. It is suggestedthat this sensitivity may correlate with the physical trait of whitesclera (an exclusively human trait among primates), that wouldmaximize the detectability of eye direction independently of headdirection (Kobayashi & Kohshima, 1997).Physiological studies suggest that neurons in the STS of

macaque monkeys may code for the different cues of gaze (body,head and eyes orientation) in a hierarchical way. Depending uponthe visibility of the eyes or the head, these neural systems extractan invariant that is best characterised as ‘‘direction of gaze’’ or‘‘attention’’ (Perret & Emery, 1994, #860). This invariant may workas a Gestalt, i.e., ‘‘a configurational quality in addition to, or insteadof, the individual elements composing the pattern of stimulatingenergies’’ (Hochberg, 1987, p. 288). This Gestalt invariant canbe encoded across a variety of physical configurations. In thissense, perceiving gaze direction in the Brentanian sense goesbeyond the physical information given, but not by postulatingunderlying unobservable processes (as in Cartesian Theories ofmind), but by recognizing a higher-order perceptual invariant in

J.C. Gómez / Neural Networks 22 (2009) 190–196 193

Fig. 2. Left: Examples of Gestalt configurations creating the impression of a square that does not really exist. Right: Examples of intentional Gestalten: direction of gazecreates an impression of connectedness between the gazer and one of the targets in the figure.

the physical cues, much in the same way as Gestalt configurationsare recognized in physical stimuli, such as dots aligned in the shapeof a square vs. a circle (Hochberg, 1987) (see Fig. 2).To sum up, given the inevitability of giving away abundant

cues about the direction of gaze for engaging in effectivevisual information processing in the environment, evolutionselected gaze following adaptations in primates (and other animalspecies (Emery & Clayton, 2009; Gomez, 2005)). Gaze followingappears to be a prototypical case of adaptation to a behaviouralmanifestation of Brentanian intentionality that correlates almostperfectly with its behavioural counterpart—gaze is directed to theobject of attention of the agent. The basis for this adaptationis not the computation of an unobservable state of attentionor vision assumed to occur inside the observer’s head, butrather the perception of a configural relation between agent andobject. Evolutionarily this is a more basic and preliminary stepfor understanding social meaning than computing unobservableentities such as Cartesian mental states.

4. Is gaze really coded intentionally?

Do primates really code a relation between agents and objects,or do they just turn in the direction of gaze in a reflex-likeway? There is some evidence that primates’ gaze following isgenuinely intentional in Brentano’s sense (i.e., involves the codingof a directed relation). For example, chimpanzees do not simplyturn in the direction of a model’s gaze: they actively look for anobject in the line of gaze of the model. Thus, if there is a screen inthe way of the model, they look at the model’s side of the screen,expecting to find the target there, not beyond the screen throughwhich the model’s gaze cannot penetrate. Moreover, chimpanzeesare reported to manually search places looked at by others if theycannot gain visual access to the area, and look back and forthbetween gazer and target location if there is no obvious objectthere, as if checking the relation between gazer and target (Call &Tomasello, 2005). Scerif et al. (2004) report experimental evidenceof increased looking at the model in Diana monkeys when a targetis in the wrong position in relation to the direction of gaze. Finally,Lorincz, Perrett, and Gómez (in preparation) have analysed thetopography of gaze following in humans and rhesus monkeys witheye tracking recordings finding a similar pattern of going backand forth between the face and the target in both species, insteadof simply going from face to target when watching pictures ofmodels looking sideways. This suggests that gaze is processed asa relation between an agent and an object, which fits the notion ofintentionality.

5. Beyond gaze: Coding intentional action

Is gaze a special case, or is the intentional coding of behavioura more general feature of primate social cognition? Mountingbehavioural and physiological evidence suggests that intentionalcoding is the norm. For example, Perret (1999) and Rizzolatti andCraighero (2004) report the existence of neurons in the rhesusmonkey brain that fire specifically to the sight of hands graspingobjects, but not to the sight of a hand grasping in vacuo, or thesight of an object (e.g., a sponge) being deformed in the sameway it would if grasped by a hand. It is the specific interactionbetween hand and object that triggers the firing of the ‘‘grasping’’neurons. Moreover, Rizzolatti and Craighero (2004) report thatmirror neurons for ‘‘grasping’’ can fire to the sight of a handinitiating a grasping movement and disappearing behind a screenif the monkey has previously been shown that behind the screenthere is an object. However, the same neuron fails to fire ifthe monkey has previously been shown that there is no objectbehind the screen. These neurons seem to be coding for theintentional directionality of actions even when the target of theaction is not immediately visible. Recent behavioural evidencesupports these conclusions with rhesus monkeys (Wood et al.,2008). There is therefore an emerging picture that non-humanprimates code action in intentional terms – as being about targets –and importantly the targets need not be concurrently visible withthe action. This can form the basis for the understanding of socialmeaning in primates.

6. Intentionality and meaning

We have thus an evolutionary scenario in which primatesroutinely code others’ behaviour in relation to potential targetson the basis of an intentional (in Brentano’s sense) reading ofbehavioural cues such as gaze, reaching, etc. They give meaningto the actions of others by relating them to specific targets.This is a powerful breeding ground for the evolution of morecomplex forms of social meaning, especially those that involve theselection of behaviours specialized in conveying social meaning—communicative signals, such as the alarm call systems of Vervet orDianamonkeys or the rich repertoire of facial expressions thatmayoccur concurrentlywith gaze direction in primates. Indeed primatefaces offer a unique combination of advertised gaze direction andadvertised emotional reactions in the form of facial expressions(supported by muscles specifically selected for this function) thatgive further information about how the agent relates to the target(fear, anger, surprise, etc.). Primate faces are uniquely suited for thefunction of conveying meaning in a Brentanian way.In addition, the Brentanian perspective provides a framework

for understanding the complexity and flexibility of primate vocal

194 J.C. Gómez / Neural Networks 22 (2009) 190–196

referential signals in the absence of representational theories ofmind. These signals seem to be mounted upon this wider systemof intentional interpretation of behaviour as directed at something.Thus Vervet monkeys not only engage in the correct escaperesponse upon hearing one of their three possible predator alarmcalls, but literally look for the appropriate predator (scanning thesky if it is the eagle alarm; scanning the ground if it is the snakealarm). This is consistent with a referential reading of the acousticsignals in an intentional way (in Brentano’s sense)—they must beabout something, just as the gaze of a conspecific must be aboutsomething.This can occur in the absence of any evidence for representa-

tional theories of mind in monkeys, as suggested by their failure totake into account whether others know or not about the predatorwhen producing their calls (Cheney & Seyfarth, 1990). However,they do take into account if others are present or not: in the ab-sence of an audience, monkeys usually do not call; as receivers,they also take into account the identity of the caller. Thus, whenalarms are given by infants, they check if there is a real predatorbefore echoing the alarm or engaging in an escape response. In ex-periments where they have ceased to respond to the pre-recordedcall produced by a particular individual, they respond again if thesame call is played back from a different individual (Cheney & Sey-farth, 1990). This is consistent with the coding of individual inten-tional relations between agents and targets.Other primate species, especially our closest relatives the apes,

may be capable ofmore sophisticated forms of referentialmeaning.Chimpanzees take into account what other chimps have seen orfailed to see in the past: for example, they no longer avoid a pieceof food previously seen by a dominant individual if its locationwas changed while the dominant was not present (Hare et al.,2001). Orangutans and bonobos direct their human caretakers tothe tools they will need to give them food when the tools havebeen hidden unbeknownst to them (Gomez, 2004; Zimmermannet al., 2009). However, as argued by Gomez (2008), all thiscan be done with a Brentanian ability to attribute Intentionalavailability, where records of the intentional relations betweenspecific agents and specific objects are kept and updated, oreven actively manipulated. All these complex things can be donewithout representational theories of mind.The Brentanian intentionality framework that I propose allows

therefore to reconcile the complexity and flexibility of socialmeaning reported in non-human primates with the possibleabsence of representational forms of Theory of mind involving thecomputation of unobservable mental states.

7. Autism: Meaning blindness?

This perspective may also help to understand some paradoxicalfindings in autism research. There is a sub-group of high-funct-ioning people with autism spectrum disorders – i.e., individualswith general intelligence and language skills within the normalrange – who can pass traditional (‘‘Cartesian’’) Theory of mindtests, such as false-belief tests (Frith, 2003). These people seemto have some notion of private, unobservable representationsand how they can affect the behaviour of others. However, intheir everyday lives they still have serious problems in socialinteraction and communication. It is as if their theories of mindare precisely that—theories with limited practical application.Moreover, when tested with other simple social cognitive teststhat do not require explicit reasoning about mental states, thesepersons typically tend to paradoxical shortcomings. Interestingly,they may show abnormalities in what I have argued is one of themost basic systems of perceived intentionality—gaze following.Many individuals with autism typically fail to spontaneouslyfollow the gaze of others or their pointing gestures even if, when

verbally prompted, they show evidence of possessing the requisitegeometrical mechanisms for calculating the targets of others’attention (Leekam et al., 1997).I suggest that what these personswith autismmay have altered

is their ability to perceive the social world intentionally in thesense of Brentano (rather than representationally in the Cartesiansense). They may suffer from subtle alterations in some of thephylogenetically oldmechanisms for referentialmeaning, whereasother, phylogenetically more recent and arguably more complexmechanisms (such as language or representational Theories ofmind), may be largely preserved (albeit probably impaired inpractical terms because they might not work properly in theabsence of the supporting Brentanian mechanisms with whichthey interact). To put it in other words, Autism may start as aproblem with embodied meaning, rather than with disembodiedmeaning (see also Hobson (1993) and Klin et al. (2002) for otherviews defending the importance of embodiment perspectives inAutism).

8. Embodied origins of disembodied meaning

In addition to Brentanian intentionality, humans possessrepresentational Theories of mind in the Cartesian style. We doattribute ‘‘disembodied’’ mental states detachable from behaviour,and this allows us to engage in very complex forms of socialmeaning and communication. Where do representational theoriesof mind come from? Are they a radical evolutionary departurefrom the Brentanian mechanisms that make intentional sense ofbehaviour?The key difference between Brentanian and Representational

Theories of mind (ToM) is that whereas in Brentanian ToMs agentA codes the relation of agent B to reality as represented by agent Aitself, in Representational ToMs agent A codes the relation of agentB to agent B’s own representation of reality, whichmay differ fromagent A’s representation (e.g., a false-belief situation where agentB incorrectly represents object X as being in a location Z, when itreally is in location Y, and agent A knows it). Hence the need to havesome sort of explicit notion of representation as detachable from‘‘reality’’ (note that ‘‘reality’’ means here agent A’s representationof reality), instead of using one’s own representations as theanchoring point for the intentional relations attributed to others.A new type of intentional relation appears with representationalToMs—relations of Intentional Non-existence (Gomez, 2008), andwith it disembodied forms of meaning. The question is: does thisreally entail a radical departure from the embodied ToMs of otheranimals, and if so what is the mechanism responsible for thisevolutionary innovation?The coding of relations of intentional non-existence might be

enabled by the appearance of a new mechanism in evolution,for example Leslie’s ToMM (Leslie, 1987), which allows for theextraction of representational fragments of reality and theirattribution to agents in an inverted commas relation (i.e., onethat suspends the truth/reality status of such fragments). Leslieproposes that the earliest manifestation of this innovative skill inhuman ontogeny is ‘‘pretend play’’, the ability to act as if somethingwere the case when it actually is not, for example, pretending thatone has a glass ofwater in the hand and acting as if drinking from it.In this case, one has to ‘‘imagine’’ an object that is not there, and todo this, according to Leslie, one needs the ability tometa-representor attribute decoupled representations to agents.However, if Leslie is right and pretend play is a genuine early

manifestation of this new human ability that later in developmentis used to understand false beliefs, we may have a cue as to theevolutionary origins of this skill, that may be less innovative andradically different than it looks.

J.C. Gómez / Neural Networks 22 (2009) 190–196 195

The key ingredient of pretend play is the ability to ‘‘imagine’’objects or properties; for example, to imagine the glass and thewater in the child’s empty hand. Typically the child’s hand willadopt a glass-graspingmovement, whereas the glass and thewaterare left to the imagination of the observer. However, this veryability to infer the existence of objects that are not present fromfragments of action appears to exist in non-human primates.As mentioned earlier, macaque monkeys need not see the hand

actually grasping the object to code an action as grasping. Oneand the same action – reaching behind a screen with the hand ingrasping pose – is coded as grasping if the monkey knows thereis a hidden object, or not grasping if the monkey knows thereis nothing behind the screen. One and the same action is givena different meaning according to what the monkey ‘‘imagines’’is behind the screen. However, because monkeys were shown inadvance the object behind the screen, it could be argued thatrather than imagination they just use short-termmemory. It is notnecessary that children have just seen a glass to be able to pretendthat they are holding one.Preliminary results from a study with Capuchin monkeys

(Gómez, Kersken, & Rowlings, in preparation) suggest that thesemonkeys might be able to imagine objects from actions withouthaving seen the objects before, or indeed without the objectsreally existing. When shown a grasping hand disappear behinda screen without having been shown if there is or not anythingbehind the screen, the monkeys were ‘‘surprised’’ (looked forlonger) if, upon lifting the screen, no objectwas found behind. Theymust be ‘‘imagining’’ an object that does not really exist from thegrasping action, which appears to be exactly what is required forengaging in pretend action. Why then do not monkeys and apespretend (Gomez, 2008)?The difference with human children lies, not in the ability

to imagine objects from actions, but in the ability not to blockor inhibit this inference when reality reveals that there is noobject. Human children activate and keep their imagination ofthe absent object counterfactually. Monkeys appear to discardit immediately. It certainly makes evolutionary sense to havereality override erroneous representations, and there must bein place quick and powerful inhibitory systems to cancel wrongrepresentations (Leslie, 1987).A hypothesis is therefore that what emerged in human

evolution was not a new mechanism for generating imaginedintentional relations, but a new way to handle the imaginedintentional relations that turn out to be wrong or out-of-date—anability to keep and generate them counterfactually, and, as Lesliesuggests, to keep them separate from ‘‘true’’ representations. Thisis of course no small business, but this bold evolutionary step mayhave built upon existing primate adaptations, including the abilityto imagine objects from actions. Paradoxically, this step towardsdisembodied cognition seems to be built upon pure embodiment:the use of bodily actions to suggest and infer things that do notexist, as human infants routinely do in their pretend play.

9. Simulating perceived intentionality

A challenge for the Brentanian intentionality framework isto specify the mechanisms whereby agent–object intentionalrelations are encoded and how they interact in evolution anddevelopmentwith othermechanismsof social cognition, especiallyrepresentational theories of mind. Neural simulation could helpclarify many aspects of this challenge. For example: How isthe illusion of gaze direction acquired? Is it the consequenceof simple associative learning of the correspondence betweencertain gaze configurations and finding objects of interest, ormust some intrinsic directionality be built into the model?Does intentional perception (coding of directional agent–object

relations) come about simply through associative learning of thetype ‘‘gaze > interesting object’’, or does it require somethingadditional, for example, an interaction between gaze and actionperception derived from the frequent correlation between lookingand grasping, and looking and manipulating objects? Where doesthe ability to imagine unperceived objects from perceived actionscome from? Is it a simple extension of the coding of agent–objectrelations or something else is necessary? And finally, what isnecessary for the emergence of counterfactual imagination ofobjects?Wouldmodel capable of coding intentionally non-existingobjects (as in pretend play) be also capable of coding false belief ormisrepresentations?The Brentanian framework may in turn have implications

for the simulation with neural networks of mechanisms capableof detecting social meaning. Just as evolution appears to havedealt first with the challenge of dealing with the detection andadvertisement of intentional relations, the simulation of socialmeaning may have to start with mechanisms capable of detectingand processing intentional relations in the Brentanian sense. Inrelation to what I identified earlier as a paradigmatic case ofBrentanian intentionality – gaze following – a variety of attemptshave been carried out to simulate the acquisition of the skillof finding rewarding objects in the line of gaze of others (forexample, Hoffman et al. (2006) and Triesch et al. (2006)). However,it is unclear whether these attempts have succeeded at simulatingthe very property of intentionality in gaze following – the encodingof a line of gaze that is oriented in particular directions connectingagents and their targets and that cannot be identified with anyparticular pattern of physical stimuli but seems to act as aconfigurational invariant. This is somehow pre-built in the model,rather than being the target of the simulation itself. In addition, it isunclear if the existing models can simulate the coding of a relationbetween gazer and target, and not just the one-way re-orientationfrom gaze to target. As discussed earlier, behavioural and eye-tracking findings suggest that gaze followers do not simply turn inthe direction of gaze: they expect and object, and they frequentlyreturn their gaze to the gazer (especially but not exclusively whenno object of interest is to be found) after looking at the target.Finally, it is also unclear to what extent existing simulations ofgaze following can accommodate for the ability to actively searchfor an object or assume the existence of an object that is notimmediately visible, as some non-human primates do in responseto gaze behind barriers. Addressing the issue of how to simulatenot just the reaction of visual co-orientation, but the coding of thedirectional relation between gaze and target would throw lightboth on the evolution of basic social meaning mechanisms, and ontheir alteration in conditions like autism.May be the bigger challenge both for simulation and evolu-

tionary/developmental analyses of intentionality attribution is un-derstanding the transition from Intentional availability to Inten-tional Non-existence, to use the labels coined by Gomez (2008). Insimulation, the challenge is to build not only neural networkscapable of ‘‘imagining’’ objects from actions, but keeping the imag-inary objects counterfactually, that is, with a simultaneous encod-ing of states of reality that differ from the inferred reality. To myknowledge there have been no attempts at simulating pretendplay or other forms of misrepresentation with neural networks.The interplay between simulation and an evolutionary approachto the origins of this ability may be especially fruitful in suggestingcandidate mechanisms for rewiring a primate brain capable ofcalculating intentional availability into a primate brain in additioncapable of calculating intentional non-existence (Gomez, 2008).

196 J.C. Gómez / Neural Networks 22 (2009) 190–196

10. Concluding remarks

Although the analysis of social meaning in humans seems tolead to cognitive accounts based on the attribution of disembodiedintentions via representational Theories of mind á la Descartes, anevolutionary perspective based upon a fresh development of theBrentanian notion of intentionality suggests a different picture.Social meaning may have emerged in evolution through thecoding of embodied intentions—relations between agents and theirtargets as advertised by their behaviour towards objects in theworld. These perceived relations cannot be reduced to the directperception of physical parameters: they involve a configurationalorganization of stimulus patterns that goes beyond the informationgiven in a Gestalt fashion.This coding of intentional relations is not reduced to the here

and now. At least some non-human primates are capable ofkeeping records of past intentional relations (who witnessed ordid not witness what) and adapt their behaviour accordingly. Thisability to flexibly track and manipulate the intentional availabilityof objects to agents can account for the complexity reported in thesocial and communicative interactions of non-human primates,as well as for a substantial and essential part of human socialmeaning.Humans have indeed evolved disembodied or Cartesian The-

ories of mind in addition to what I have called ‘‘Brentanian in-tentional mechanisms’’. But Cartesian ToMs can only work prop-erly in collaborative integrationwith the Brentanian systems. Theirimportance is clearly demonstrated by the paradoxical patternsof impairment shown by some high-functioning individuals withautism, who although reasonably competent in Cartesian ToMtasks, may fail to spontaneously respond in intentional terms tovery simple cues, such as gaze or pointing.In addition, the evolution of human Cartesian mentalism

may be substantially continuous with Brentanian systems. Someessential components of Cartesian ToMs were already presentin Brentanian systems, notably the ability to imagine objects ofaction or attention in the absence of direct perceptual or memoryinformation about them. This points to a process of re-organizationof existing abilities in the evolution of representational ToMs,rather than the emergence of a completely new system.Brentanian intentionalitywas the basic challenge confronted by

evolution to generate mechanisms capable of meaning. It may alsobe the basic challenge for simulationists aiming to understand howmeaningful action and interaction are coded—the embodiment anddisembodiment of meaning.

Acknowledgements

This paper was written under Project REFCOM (NEST-PATHFINDER INITIATIVE Project 12787). My gratitude to the participantsin this project for useful discussions on the topic of referenceand communication, particularly to Tim Crane, K. Zuberbühler,D. Byrne, and V. Janik. My thanks also to Toni Gomila for his veryhelpful comments on Brentano and intentionality.

References

Call, J., & Tomasello, M. (2005). What chimpanzees know about seeing revisited:An explanation of the third kind. In N. Eilan, et al., (Eds.), Joint attention:Communication and other minds (pp. 45–64). Oxford University Press.

Carruthers, P., & Smith, P. K. (Eds.) (1996). Theories of theories of mind. CambridgeUniversity Press.

Cheney, D. L., & Seyfarth, R. M. (1990). How monkeys see the world. ChicagoUniversity Press.

Cheney, D. L., & Seyfarth, R. M. (1995). Intention and function in primatecommunication. In R. Dunbar et al. (Eds.), Evolution of primate social systems.Royal Society Meetings.

Crane, T. (1998). Intentionality as the mark of the mental. In A. O’Hear (Ed.), Currentissues in the philosophy of mind (pp. 229–251). Cambridge University Press.

Descartes, R. (1662). Tractatus de homine, et de formatione foetus. Amsterdam:Elzevirium.

Emery, N., & Clayton, N. S. (2009). Comparative social cognition. Annual Review ofPsychology, 60, 87–113.

Frith, U. (2003). Autism: Explaining the enigma (2nd ed.). Blackwell.Gomez, J. C. (2004). Apes, monkeys, children and the growth of mind. HarvardUniversity Press.

Gomez, J. C. (2005). Species comparative studies and cognitive development. Trendsin Cognitive Sciences, 9(3), 91–158.

Gomez, J. C. (2007). Pointing behaviors in apes and human infants: A balancedinterpretation. Child Development , 78, 729–734.

Gomez, J. C. (2008). The evolution of pretense: From intentional availability tointentional non-existence.Mind and Language, 23(5), 586–606.

Gómez, Kersken, & Rowlings, The perception of intentional inexistence in Capuchinmonkeys (in preparation).

Grice, H. P. (1957). Meaning. Philosophical Review, 66, 377–388.Hare, B., et al. (2001). Do chimpanzees know what conspecifics know? AnimalBehaviour , 61(2), 139–151.

Heyes, C. M. (1998). Theory of mind in nonhuman primates. Behavioral and BrainSciences, 21(1), 101–148.

Hobson, R. P. (1993). Understanding persons: The rTMle of affect. In S. Baron-Cohen,et al., (Eds.), Understanding other minds: Perspectives from the theory of mindhypothesis of autism (pp. 204–227). Oxford University Press.

Hochberg, J. (1987). Gestalt theory. In R. L. Gregory (Ed.), The Oxford companion tothe mind (pp. 288–291). Oxford University Press.

Hoffman, M., et al. (2006). A probabilistic model of gaze imitation and sharedattention. Neural Networks, 19, 299–310.

Klin, A., et al. (2002). Defining and quantifying the social phenotype in autism.American Journal of Psychiatry, 159, 895–908.

Kobayashi, H., & Kohshima, S. (1997). Uniquemorphology of the human eye.Nature,387, 767–768.

Leavens, D., &Hopkins,W. (1999). Thewhole handpoint: The structure and functionof pointing from a comparative perspective. Journal of Comparative Psychology,113, 417–425.

Leekam, S., et al. (1997). Eye-direction detection: A dissociation between geometricand joint attention skills in autism. British Journal of Developmental Psychology,15(1), 77–95.

Leslie, A. (1987). Pretense and representation: The origins of ‘‘theory of mind’’.Psychological Review, 94, 412–426.

Lorincz, E., Perrett, D., & Gómez, J.-C. Spontaneous Attention Following: AComparative Study of Adult Rhesus Macaques and Humans (in preparation).

Penn, D. C., & Povinelli, D. J. (2007). On the lack of evidence that non-human animalspossess anything remotely resembling a Ôtheory of mindÕ. PhilosophicalTransactions of the Royal Society of London B, 362, 731–744.

Perret, D. (1999). A cellular basis for reading minds from faces and actions.InM. Hauser, &M. Konishi (Eds.), The design of animal communication. MIT Press.

Perret, D., & Emery, N. (1994). Understanding the intentions of others from visualsignals: neurophysiological evidence. Current Psychology of Cognition, 13(5),683–694.

Rizzolatti, G., & Craighero, L. (2004). The mirror neuron system. Annual Review ofNeuroscience, 27, 169–192.

Ruiz, A., Gómez, J.-C., & Byrne, R. Lemurs prioritise head orientation over bodyorientation when following conspecific gaze (submitted for publication).

Scerif, G., et al. (2004). What do Diana monkeys know about the focus of attentionof a conspecific? Animal Behaviour , 68, 1239–1247.

Tomasello, M., et al. (2003). Chimpanzees understand psychological states — Thequestion is which ones and to what extent. Trends in Cognitive Sciences, 7(4),153–160.

Tomasello, M., et al. (2007). Reliance on head versus eyes in the gaze following ofgreat apes andhuman infants: The cooperative eyehypothesis. Journal of HumanEvolution, 52, 314–320.

Triesch, J., et al. (2006). Gaze following: Why (not) learn it? Developmental Science,9, 125–147.

Wood, J. N., et al. (2008). Rhesusmonkeys’ understanding of actions and goals. SocialNeuroscience, 3(1), 60–68.

Zimmermann, F., et al. (2009). Orangutans (Pongo pygmaeus) and bonobos (Panpaniscus) point to inform a human about the location of a tool. Animal Cognition,12, 347–358.

Zuberbuhler, K. (2003). Referential signalling in non-human primates: Cognitiveprecursors and limitations for the evolution of language. Advances in the Studyof Behaviour , 33, 265–307.