Integrating Autopoiesis and Behavior: An exploration in computationalchemo-ethology

Matthew D. Egbert, Ezequiel Di PaoloCCNR, University of Sussex, Brighton, BN1 9QJ

Correspondence to:Matthew Egbert,

CCNR, University of Sussex, Brighton, BN1 9QJ, UKE-mail:mde@matthewegbert.comTelephone:+44 (0)1273 872948

It has been argued that the difference between an autonomousentity and an agent is in theability of the latter to perform behaviors supplemental to processes of self-maintenance, e.g.autopoiesis. Theories have been proposed concerning how such behaviors might relate to au-topoiesis, but so far, computational models of autopoiesishave paid little attention to theserelations. In this paper we present a new model designed to explore the relationship betweenmechanisms of autopoiesis and behavior. We report on three clarifications of the theory pro-vided by the model: (a) mechanisms of behavior can be relatedto mechanisms of autopoiesiswhile remaining operationally distinct, (b) the organization of an operationally closed systemcan change over time while remaining operationally closed,and (c) behavior modulation basedupon autopoietic efficacy has limitations that can be avoided through the use of a partiallydecoupled behavioral system. Finally, we discuss questions that have surfaced during exami-nation of the model.

Keywords: artificial life, autopoiesis, agency, behavior, operational closure, chemo-ethology,artificial chemistry

Introduction: Behavior

Fundamental to the concept of agency is the notion of theintrinsic origin of behavior. Agentsact. Moreover, natu-ral agents are self-produced entities, as in the case of livingsystems. Following this observation, it has been proposedthat self-production implies a form of autonomy, which is afundamental requirement for agency (Di Paolo, 2005; Ruiz-Mirazo & Moreno, 2004). Theory surrounding the idea ofautopoiesis (Maturana & Varela, 1980), the process of self-production of a distinct entity, has furthered our understand-ing of autonomy, but ‘bare’ autopoiesis fails to fully ad-dress behavior and explains only direct self-maintenance (DiPaolo, 2005, 2009). The relation between autopoiesis and be-havior is a subtle one. For instance, it has been proposed thatthe mechanisms underlying the production of behavior (reg-ulated coupling with the environment) have acquired com-plexity in the history of life through a succession of ‘decou-plings’ from underlying metabolic levels (Moreno & Etxe-berria, 2005; Barandiaran & Moreno, 2008). These are com-plex theoretical proposals that can be difficult to grasp in theabsence of a simple model of the ideas at work. The modelpresented in this paper has been developed to serve as sucha demonstration. We hope that by defining a specific systemthat has the organizational features described in the autopoi-etic literature, we shall facilitate the study of autopoiesis and

its relationship with behavior and relevance to agency.Most computational models of autopoiesis demonstrate

only self-maintenance but no behavior. Generally the simu-lated autopoietic entities exist in an environment which re-quires no organism-scale action to continue to exist (e.g.Varela, Maturana, & Uribe, 1974; McMullin, 2004). A fewmore recent models have demonstrated agents performing aslightly extended autopoiesis; extensions such as incorporat-ing a simple behavior such as osmotic crisis avoidance (Ruiz-Mirazo & Mavelli, 2007) or chemotaxis (Suzuki & Ikegami,2009). In these cases, the added behaviors are actuallyexten-sions of the mechanisms of autopoiesis– they are inseparablefrom the autopoiesis. To stop the mechanism of behavior isto stop the mechanism of autopoiesis. However, this is notthe case for the majority of behaviors observed in nature thatstop and start while autopoiesis continues.

This raises the question of how mechanisms of behaviorcan be related to, but somewhat independent of the mecha-nism of autopoiesis. How can behavior be integrated withand yet ‘decoupled’ (Moreno & Etxeberria, 2005; Barandi-aran & Moreno, 2008) from mechanisms of autopoiesis?

In this paper, we describe two tools of analysis that weuse to study this question. The first is the notion of ‘opera-tional closure’ which we describe in detail in the next section.The second is the computational model of autopoiesis that wehave developed to study the relationship between autopoiesis

2 MATTHEW D. EGBERT, EZEQUIEL DI PAOLO

Figure 1. A schematic of an operationally closed system and itsenvironment. Filled circles represent processes and arrows repre-sent dependencies. The processes within the dashed circle fit thecriteria for operational closure and are thus part of an operationallyclosed system.

and behavior. This model does not demonstrate new behav-ior, rather it demonstrates a different organization of behaviorrelative to autopoiesis.

Operational Closure

Operational closure allows us to identify distinct sets ofinterdependent processes. Operationally closed sets are notcompletely independent of other processes, but rather eachmember process both depends upon and enables other pro-cesses within the set1. The aim of this definition is to pro-vide a formal framework for the notion of a self-sustainingsystem. Other expressions for this concept can be found in(Varela, 1979; Thompson, 2007; Di Paolo, 2009) and else-where.

Given a collection of processesC , a subsetP of those pro-cesses is operationally closed if, for every constituent processP∈ P , the following holds:

1. Another processP′∈ P is conditioned by processP

2. ProcessP is conditioned by another processP′′∈ P .

(P′ andP′′ can be one and the same process.)In a weak sense, a process A is conditioned by an-

other process B if alterations to B result in changes in A.In a stronger sense, the relation can be understood as onewhereby B enables A, in this case, for A to sustain itself asa process, B must be present. In both situations the relationof conditioning captures the idea of a process influencing an-other in a weaker, but also less problematic sense, than thatof causality.

With these criteria, given a set of processes and their rela-

tionships of condition, it is possible to identify operationallyclosed networks of processes. As an example, consider thehypothetical system depicted in Figure 1. Each of processesa,b,c, and d depend upon another process within the setandalso enable another process within the set. There is also asecond, nested operationally closed network, consisting ofnodes b, c and d.

So, given a collection of processes and their interdepen-dencies, it is possible to identify operationally closed sets.However, the situation becomes less well defined when werecognize the dynamic nature of interdependencies. ProcessA might depend upon process Bnow, but not later (or viceversa). The point we wish to underline here is that system in-terdependencies can change, not only because the processesare dynamic but also because relationships of contingencychange. It is already established that the realization of anoperationally closed system can also change over time but,in addition to these structural2 changes, the very relationsbetween the constituent processes of an operationally closedsystem may also vary in important ways. As we shall show inthis paper, an operationally closed system can recruit behav-ioral processes, while conserving operational closure. Per-haps understanding this dynamic aspect of operational clo-sure can help us understand the relationship between au-topoiesis and behavior.

This concept of operational closure is related to but shouldnot be confused with the concept of closure as the propertyof mathematical groups nor with the similar idea of closureof a set of molecular reactions (Dittrich & di Fenizio, 2007).The latter specifies that given a set of molecular species thatreact with each other, there will be no new product out ofthese reactions that does not already belong to this closedset. Operational closure in the current context relates to theorganization of processes which may include physical, chem-ical, mechanical and/or behavioral aspects. What matters isthat there is a relation of dependence between these processessuch that they form a closed network and each process is pre-carious in the sense that it cannot be sustained on its own andrequires the presence of the closed network.

Computational chemo-ethology

Our model lies between two established approaches: ar-tificial chemistries (Dittrich, Ziegler, & Banzhaf, 2001) andcomputational neuroethology (Beer & Chiel, 2008). We de-fine a set of abstract artificial chemicals (reactants) and a set

1 Developing the formal definitions of ‘process’, ‘dependence’and ‘enabling’ is outside of the scope of this paper. For the presentdiscussion, it suffices to resort to intuitive understanding of theterms. This is an area in need of attention (see Virgo, Egbert, &Froese, 2009).

2 In the autopoietic literature organization and structure have sig-nificantly different connotations. Theorganizationof a system isthe way the different parts of the system relate to each otherandthestructureis what the different parts of the system are made outof. The structure of a membrane (the molecules out of which itismade) changes over time, but the organization (e.g. the membranesability to maintain high concentrations of reactants within the cell)can not change without the system failing.

INTEGRATING AUTOPOIESIS AND BEHAVIOR: AN EXPLORATION IN COMPUTATIONAL CHEMO-ETHOLOGY 3

of rules of interaction and then observe the resulting dynam-ics.

Among the dynamics that take place within our model area set of processes that maintain an autopoietic system. Thisis similar to previous models of autopoiesis in which an au-topoietic unity emerges and maintains itself. However, pre-vious models served only to demonstrate autopoiesis. Theseearlier models neither required nor performed any system-level behavior. In contrast, our model has been developed toexplore the relationship between autopoiesis and behavior.We have therefore constructed the reactants and rules of in-teraction to produce an environment in which the autopoiesisof the agent depends upon a behavior that it performs.

The ‘Computational Neuroethology’ approach, proposedby Dave Cliff in (Cliff, 1991) advocated the study of cog-nition through the study of behavior of embedded, embod-ied agents. We adopt the dynamical analysis and behavioralanalysis aspects of this approach, studying not only the low-level chemical reactions but also the agent-level behavior.

The computational neuroethology approach has beenlargely associated with the evolutionary robotics (ER) ap-proach (Harvey, Di Paolo, Wood, Quinn, & Tuci, 2005),where parameters of agents are optimized by a genetic al-gorithm and the behavior and dynamics of the agent are ana-lyzed. However, it is very difficult to study autopoiesis basedtheories of autonomy and agency using typical ER methodol-ogy because the agents are not modelled as autopoietic; theyneither degrade nor self-maintain. In contrast, in our model,the behavior generating mechanism and the autopoiesis aremodelled in the same (artificial chemical) domain. This al-lows us to study not only the low-level and agent-level pro-cesses, but how behavioral and autopoietic processes inter-act.

Model

To explore the relationship between autopoiesis and be-havior, we have developed a model inspired by biologicalcells. A primary motivation in the construction of this modelwas to produce a simulation that models behavior and au-topoiesis in a single, unified manner. The autopoiesis and thechemotaxis of the agent are both the result of the same type ofdynamics, namely the interactions between enzymes, a mem-brane and a ‘high-energy’ resource. This makes it possiblefor us to study the relationship between the two phenomenaas there is no ‘gap’ between them in the model.

The model consists of a set of particulate enzymes sur-rounded by a flexible membrane. These components aresimulated in a two-dimensional arena and interact in such away as to produce a mobile, autopoietic cell-like entity: ‘theagent’ (see figure 2). Before explaining the details of themodel, we outline here how the agent is both anautopoieticand abehavingentity.

To show that the agent is autopoietic we must demonstratethat it is composed of a set of reactions that are operationallyclosed. Each process must be precarious in the sense that itdepends upon some other process in the network to continue.Each process must also make possible at least one other pro-

cess in the network. This is most easily understood when weconsider the two processes ofproducing the membraneandthe autocatalysis of the enzymes.

First, let us consider the precariousness of these processes– their tendency to stop in the absence of each other. Thisprecariousnessis included in our model by causing both themembrane and the enzymes to degrade over time. The mem-brane continually shrinks in size and the enzymes have a sig-nificant chance of degrading into substrate particles each iter-ation. These processes of degradation can be countered in ahealthy agent by the production of newly catalyzed enzymes.But, if the membrane becomes too small, the random Brown-ian motion of the enzymes will cause them to leave the agentat a rate greater than their production and their populationwill fall to zero. Thus, the membrane must be kept abovea certain size if new enzymes are to be produced. The agenthas a problem however in that as we have already mentioned,the membrane steadily shrinks. Fortunately for the agent,certain enzymes (enzymeM andW) interact with the mem-brane in a way that causes it to grow. Here we have a cycleof dependence – the rate of enzyme production depends uponthe membrane being sufficiently large, and the maintenanceof a large membrane depends upon the ongoing productionof enzymes. This cyclical, reflexive dependence means thatthe agent is operationally closed. As a boundary maintain-ing, precarious, operationally closed system, we considertheagent to be autopoietic.

The agent performs the simple behavior of moving to-wards high concentrations of reactants (chemotaxis), thusmodulating its coupling with the environment. The partic-ular reactant that the agent seeks out is required for its self-maintaining reactions. As such it can be conceptualized asa food or resource, upon which the agent depends. This re-actant is named ‘resource’. In each trial, a finite amount ofit is placed in an arena, and the behavior of a single agent isobserved as it consumes the resource.

We shall now describe in more detail the various reactantsand reactions within our model.

Resource

The agent exists in a 2D, infinite environment initializedwith a patch of ‘resource’ (R). Reactions 1a and 2a in Table1 show the use ofR in the production ofM, an enzyme thatis fundamentally important in the maintenance of the mem-brane. Thus, the agent requires access toR if it is to be able toproduce sufficientM to counteract its degradation. The finitequantity ofR in the environment implies a finite maximumpossible life-span of the agent.

R does not diffuse. Its initial distribution is a square ofconcentration 1000R/unit2, with dimensions 1200 x 1200units (about 3 times the diameter of an agent) with a borderof lower concentration (500) of width 200. This distributionof resource was selected to be large enough to observe theagents behavior in a healthy environment, but limited enoughto determine how the agent responds as it has consumesRand finds itself in an increasinglyR impoverished environ-ment. As the agent consumesR in the production ofM, the

4 MATTHEW D. EGBERT, EZEQUIEL DI PAOLO

Figure 2. A time series of a healthy agent. To improve readability, only 10% of the enzymes are drawn. See text for a detailed descriptionof the processes involved. A) Soon after the start of a trial,stochastic processes have already produced some small, random motion in partsof the membrane. B) The agent has begun moving upwards. Note the asymmetry in concentrations ofM andW. Also note the increased sizeof the agent due to the high local levels ofR. C) After having visited most parts of the arena, the agent returns to the center where theR isnow high again relative to the agent’s previous location. Note the agent has shrunk due to the decrease in availability ofR. D) The end of atypical run for a healthy agent.

distribution ofR changes. This can be seen in the backgroundvalues of Figure 2 which indicate higher concentrations ofRin darker colors. NoR is added after the initialization of thesimulation.

To be clear, the resource (R) should not be confused withthe substrate particles (S1 andS2) discussed below. Substrateparticles are transformed into enzymes in a process that con-sumesR. R can be thought of as a ‘high-energy’ molecule,similar to ATP, that contributes energy to what would other-wise be an endergonic reaction. Substrate particles are themolecules that are transformed into the product.

Enzyme properties and chemical reactions

There are four enzymes that are simulated as particles inBrownian motion: M, W, T, andN. The membrane is per-meable to these enzymes. However, the enzymes degradeimmediately if outside of the membrane. This is a simpleway for us to capture a dependence of the reactions upon themembrane that they maintain, and it can be conceptualizedas the membrane preventing an enzyme toxin from enteringthe agent.

The Brownian motion of these enzymes is simulated bymodifying each enzyme’s two spatial coordinates by values

INTEGRATING AUTOPOIESIS AND BEHAVIOR: AN EXPLORATION IN COMPUTATIONAL CHEMO-ETHOLOGY 5

Rate Reactants Products

2×10−4R M+ S1 + R =⇒ M+ M (1a)0.15 M+ M =⇒ M+ S1 (1b)2×10−4R W+ S1 + R =⇒ W+ M (2a)0.15 W+ W =⇒ W+ S1 (2b)0.4 T+ S2 =⇒ T + T (3a)0.15 T+ T =⇒ T + S2 (3b)0.4 N+ S2 =⇒ N+ N (4a)0.15 N+ N =⇒ N+ S2 (4b)f (R,V) T + M =⇒ T + W (5)

Table 1Metabolic reactions. In the rate column, r represents theconcentration of the resource (R). See main text for furtherdetails.

selected from a Gaussian distribution (mean = 0, std = 20).This occurs every iteration.

In addition to the aforementioned enzymes, the model in-cludes substrate particlesS1 andS2. These are not enzymesas they do not catalyze reactions, but rather are reactants thatare transformed into products by autocatalytic reactions (re-actions 1a, 2a, 3a, and 4a in Table 1). Note that we usedtwo types of substrate.S1 can be transformed intoM andS2 can be transformed intoT andN. Our motivation for uti-lizing two different types of substrate was to simplify theanalysis of the system. Specifically, to explore the relation-ships between behavior and autopoieisis we damaged partsof the behavioral mechanisms by removingT andN from thesimulation (explained below in the Section on lesion stud-ies). We wanted to explore the behavioral ramifications ofthese lesions experiments without the additional complexityof indirect metabolic competition between the autocatalysisof the ‘behavioral’ enzymes (T andN) and the ‘autopoietic’enzymes (M andW). Using different substrates made it pos-sible for us to keep separate the behavioral and metabolicramifications of the lesions.

Note that each of the autocatalytic reactions can run inreverse (reactions 1b, 2b, 3b, and 4b). The relative rates ofreaction were selected to produce an equilibrium for eachreaction pair such that typically a small number of substratewould be available for the autocatalytic reactions at any givenlocation within the agent.

To simulate the enzyme-enzyme reactions described inTable 1, every iteration, the arena is divided up into a gridof ‘pockets’, each containing the enzymes in that area. Toavoid boundary effects, the offset of the grid is varied ran-domly each iteration. For each enzyme within each pocket arandomly selected reaction in which it takes part is selected.If the relevant reactants are present, there is a chance (indi-cated in the ‘Rate’ column of Table 1) that the reaction willoccur, i.e. the reactants will be replaced by the products.

It should be noted that this method of simulating enzyme-enzyme reactions means that, as in real chemistry, the reac-tant concentration affects the chance of a reaction occurringduring any iteration. For example, if an area contains 1000T

Enzyme Role

M Adds phospholipids to the membrane. Acti-vates cilia to produce a local outward accelera-tion

W Adds phospholipids to the membrane. Acti-vates cilia to produce a local inward acceler-ation

N ModifiesV according to local concentration ofR

V Stores a representation of the mean quantity ofresource experienced by the agent as a whole

T TransformsM into W when the local concentra-tion of R is lower than the average (representedby the concentration ofV)

S1 A substrate particle that can be transformedinto M

S2 A substrate particle that can be transformedinto T or N

Table 2Summary of enzyme roles.

(and sufficientS2) the chance of autocatalysis ofT (reaction3a) occurring somewhere in that area is more likely than ifit only contained 10T. This aspect, in conjunction with thebidirectionality of the autocatalytic reactions, means that ifone of the autocatalytic enzymes reaches a high concentra-tion, the ‘backward’ reaction (breakdown of the autocatalystinto substrate) becomes more likely than the ‘forward’ reac-tion. In this way, the system does not transform all substrateto enzymes, but instead finds a balance consisting of a smallpopulation of both types of substrate.

There is one last enzyme which we must introduce.V isdifferent from the other enzymes in that it is assumed to havea very high rate of diffusion (relative to its production andconsumption). On this basis it is modelled as becoming in-stantaneously distributed (i.e., of equal concentration)acrossthe cell within each time step. Computationally, this is mostefficiently realized by representingV with a single, cell wide,continuously valued, concentration variable, rather thanbymultiple discrete particles with specific positions which haveto be updated each iteration.

The concentration ofV is increased byN when N is ina higher concentration ofR thanV. Similarly, N decreasesthe concentration ofV when the concentration ofR is lowerthan the concentration ofV. Thus the concentration ofV isupdated by eachN, each iteration, according to equation 6wherec is the rate constant (1× 10−3) andGn is the localconcentration ofR at the location of the nthN enzyme. Weassume that this is the only reaction that affects the concen-tration ofV.

dV

dt= ∑

nc(Gn−V) (6)

In addition to the autocatalytic reactions in Table 1, thereare two non-autocatalytic reactions (2a and 5). Both of these

6 MATTHEW D. EGBERT, EZEQUIEL DI PAOLO

reactions involveW. This enzyme can be thought of as analternative form ofM in that it participates in the same re-actions asM and degrades to the same substrate. We shallsee below that the only actual difference between these twoenzymes is in how they interact with the membrane;M tendsto cause a local motion of the cell away from the center andW tends to cause the opposite motion. As we shall see, theseenzymes are fundamental to the chemotaxis behavior.W isnot autocatalytic, but instead catalyzes the production ofM.This is represented in reaction 2a.W is only produced by thetransformation ofM into W by T. The rate of this reaction isdetermined by the relative concentrations ofR andV accord-ing to the functionf (R,V) = 0.9·H(V−R) whereH() is theunit step function.

The rate of reaction 5 can be thought of as a compari-son mechanism. Where the local concentration ofR is lowerthan the concentration ofV, there is a high probability ofTtransformingM into W. This mechanism is inspired by andsimilar to the type of mechanism hypothesized by (Macnab& Koshland, 1972) to occur in chemotacticSalmonella ty-phimurium(see Falke, Bass, Butler, Chervitz, & Danielson,1997; Blair, 1995 for discussion of the chemotactic biochem-ical pathways found in bacteria). The biochemical mecha-nism is slightly different from that in our simulation, in thatit compares a current concentration to information about aprevious concentration rather than comparing the concentra-tion of two currently present chemicals. However, as can beseen in the following description given by Blair, the principalconcept of a differential reaction based upon a comparison ofone chemical concentration to another is clearly part of thebiological system.

“The cell measures the concentration encoun-tered during the past second and compares itwith that encountered during the previous threeor four seconds, basing decisions to run or tum-ble on the difference. Bacterial chemotaxisthus involves a simple, very short-term mem-ory...[that] allows the cell to make the tempo-ral comparisons that guide its choices to run ortumble.” (Blair, 1995)

The reactions betweenV, R, T, M, W and N produce acomparison mechanism. Speaking anthropomorphically, theagent keeps track of the average concentration in its localarea ofR by varying the concentration ofV. It uses this valueto transformM into W when they are in areas of low concen-tration ofR relative to the average. This produces an asym-metrical distribution ofM andW that induces chemotaxis.

All non-substrate enzymes (excludingV) degrade intosubstrate. Every iteration, there is ap = 0.001 chance thatan enzyme will degrade into its substrate (M degrade intoS1and the other non-substrate enzymes degrade intoS2). Asmentioned above, if an enzyme leaves the confines of themembrane, it is immediately transformed into substrate (Mor W into S1 and the other non-substrate enzymes intoS2).

Membrane dynamics

The flexible membrane is modelled as a circle of mass-points connected with linear and rotational springs3. Eachmass-point with its associated linear and rotational springrepresent one ‘membrane-section’. The rest-length of the lin-ear springs and the mass of the membrane-section is propor-tional to the number of phospholipids (P) in the membrane-section. The linear springs apply a forceF = −kx to themass-point, where k is the spring constant and x is the dis-tance that the spring has been displaced from its rest-length.In this way,P influences the size and mechanical propertiesof the membrane.

Figure 3. Schematic of a membrane-section.

The rotational spring applies torqueτ = −κθ (whereκ isthe spring constant andθ is the rotational displacement ofthe spring) to the associated mass-point and its two neigh-bors. These forces can be thought of as a crude simulation ofmembrane rigidity. The rest-angle for each of the rotationalsprings isπ (a straight line with the mass point in the center).

The simulated membrane consists of 32 such points con-nected in a circle. This number was selected because it is rel-atively low, reducing computational load, while high enoughto produce a relatively smooth membrane shape.

Each membrane-section degrades over time. This degra-dation takes the form of a steady exponential decrease inthe number of phospholipids at each membrane section overtime. The particular rate of degradation was selected toproduce a system that degrades rapidly in the absence ofthe system’s self-production, but is stable when the agent ishealthily self-producing. Left unchecked, the degradation ofthe membrane causes the agent to eventually shrink to a sizetoo small to maintain the populations of enzymes. This is theequivalent of death within our model.

It is interesting that quantifying this size is non-trivial.The viability (likelihood of survival) of the agent is not onlya matter of the size of the membrane, but is the result of manyinteracting factors including the number and location of vari-ous enzymes, the amount of resource available, etc. We havebegun investigating how the viability of autopoietic systemscould be measured (Egbert, Di Paolo, & Barandiaran, 2009).

3 The model we have used for the membrane is more complicatedthan is strictly necessary. We developed this mass-spring modelwith future extensions in mind.

INTEGRATING AUTOPOIESIS AND BEHAVIOR: AN EXPLORATION IN COMPUTATIONAL CHEMO-ETHOLOGY 7

Diffusion of P occurs between neighboring mass pointsaccording to Equation 7 wherek is the rate of diffusion (0.01)andPx represents the number of P at mass pointx. This rateof diffusion was selected as it tends to maintain a roughlycircular membrane.

dPx

dt= −P2

x ×10−6+k(

(Px−1−Px)+ (Px+1−Px))

(7)

Membrane-sections have inertia and are subjected to adrag force proportional to the square of their velocity. Thisforce represents the drag that would be present for a cell in aviscous medium.

Fdrag = −0.5v2 (8)

Membrane-enzyme reactions

Membrane particles do not prevent the motion of en-zymes. However, upon contact with the membrane (consid-ered to occur if the Brownian motion of an enzyme causes itto cross a membrane section),M andW contribute 185 phos-pholipids (P) to the membrane. This number was found byexperimentation to be sufficient to counter the degradationof the membrane, but not so high as to make the cell growto an inappropriately large size. In addition to contributingphospholipids to the membrane,M particles briefly activatescilia in the local region where they collide with the mem-brane. This is modelled by applying a small force upon themembrane point perpendicular to the tangent of the surfaceof the membrane.M andW activate cilia in different mannersuch that forM, the resulting force is outwards (away fromthe center of the agent) and forW the force is towards thecenter of the agent. These interactions between enzymes andmembrane cause the deformations of the membrane as wellas the motion of the agent.

Initial conditions

We initialized the model described above with a circularmembrane of radius 300, placed in the center of a squarearena, with 10 particles each ofT, N, andM placed insidethe membrane. Also,S1 andS2 particles were randomly dis-tributed around the arena at a density of 0.001 particles perunit square for each type. As mentioned at the beginning ofthis section, the arena is initialized with a square ofR, ofconcentration 1.0, and with dimensions 1200 x 1200 units.This square ofR has a border of lower concentration (0.5)Rof width 200. Outside of this border, the concentration ofRis zero.

Analysis

Initially the agent remains momentarily stationary (seeFigure 2A). Soon however, the autocatalysis ofM begins tosignificantly decrease the local concentration ofR. Brown-ian motion of the enzymes and their stochastic and thereforeslightly asymmetric reactions with the membrane result in

part of the agent extending into an area higher in concen-tration ofR than those areas that have been depleted by theagent. N particles in this area increase the concentration ofV. Then,T transformsM into W in the parts of the agent thatare in theR-depleted area. The asymmetrical distribution ofM andW particles, their Brownian motion, and their differenteffects upon collision with the membrane result in a motionof the agent away from the area low in concentration ofR.Figure 2B shows an agent after it has begun to move in thisway. The motion in this case is upwards but varies randomlyas can be seen in Figure 4.

As the agent moves, it continues to reduce the local con-centration ofR. Areas that the agent has occupied for a longerperiod of time are more depleted ofR. This causes areas to-wards ‘the rear’ of the agent to be lower inR which causesfurther production ofW which propagates the motion of theagent. In this way, the agent moves around the arena in arelatively directed manner. That is to say, once it starts mov-ing in a particular direction, it tends to continue in the samedirection.

This directed motion is a good demonstration of how lo-cal molecular interactions can acquire a global coherence,creating a spatial asymmetry that is the enabling factor forthe agent’s motion. Although the underlying mechanisms arequite different, local to global bi-directional interactions havebeen found in other minimal systems capable of generatingself-movement in a homogeneous spatial situation, e.g. self-moving oil droplets studied by Hanczyc, Toyota, Ikegami,Packard, & Sugawara, 2007. In their physical system andin our model, the conditions that cause movement are main-tained by the movement itself.

The lower right plot in Figure 4 shows the motion of thecenter of a healthy agent during a typical run. It movesaround the arena, turning when it approaches the end of thehigh-R concentration square, occasionally doubling back andoften returning to areas it has been before. As can be seen inthe left plot in Figure 5, a healthy agent consumes all of thehigh-R-concentration areas at least briefly at different timesduring a typical run.

Lesion studies

To demonstrate the interactions between the mechanismsof behavior and of autopoiesis, we performed three ‘lesionstudies’ of our agent. By mechanism, we mean a set of pro-cesses. These studies consisted of the complete removal ofeitherN or T or both. The removal was performed after aninitial settling period of 1000 iterations which was identified,by observing non-lesioned trials, as sufficient time for thesystem to ‘relax’ from initial conditions. BothN andT areonly produced by autocatalysis. Thus, when lesioned theirpopulation remains zero for the duration of the trial.

Lesion studies are typically used to analyze the dynamicsof complex systems. It is no different with their use here.Though we designed the system, it is complex enough thatthis kind of analysis is useful to understand how differentaspects of the system are interacting.

The lesion experiments are identified by a two-letter

8 MATTHEW D. EGBERT, EZEQUIEL DI PAOLO

-1000

-500

0

500

1000

-1000 -500 0 500 1000

-1000

-500

0

500

1000

-1000 -500 0 500 1000

0

100

200

300

400

500

0 10000 30000 50000

-1000

-500

0

500

1000

-1000 -500 0 500 1000

-1000

-500

0

500

1000

-1000 -500 0 500 1000

0

100

200

300

400

500

0 10000 30000 50000

-1000

-500

0

500

1000

-1000 -500 0 500 1000

-1000

-500

0

500

1000

-1000 -500 0 500 1000

0

100

200

300

400

500

0 10000 30000 50000

x,y (single trial) x,y (10 trials) Health (10 trials)

-/-

-/T

N/T

Figure 4. Paths and health of the healthy (N/T) and lesioned agents. Paths are indicated for a single trial(left column) and for 10 differenttrials (center column). The circles represent the locationof the agent at the end of the trial and does not represent agent size. Note thedoubling back in the healthy agent and the avoidance of previously visited areas by the -/T agent. The rightmost column indicates agent‘health’ (number ofM particles) for the same 10 trials.

acronyms that indicate which of these particles were nottransformed into substrate; ‘N/T’ ≡ “all enzymes intact” or‘-/T’ ≡ “T are intact, butN is transformed intoS2 at itera-tion 1000” ‘-/-’ ≡ “T andN are both transformed intoS2 atiteration 1000”.

Behavior determined directly by autopoiesis efficacy islimited. Let us first examine the lesioned agent -/-. In thisagent, the main behavior producing enzymes have been de-stroyed. The only reactions that remain active in these agentsare the autocatalysis ofM, and the interactions betweenMand the membrane.

The rightmost column of Figure 4 shows the number ofM particles in the simulation plotted against time, each plotshowing 10 trials. We use this number as an approximationof health of the agent asM particles must be produced for themembrane to maintain sufficient size to enclose enzymes. It

is clear when comparing -/- to N/T that the lesioned entitiestend to have a shorter life-span. The longest surviving -/-agent dies shortly after iteration 35000 whereas almost allofthe N/T agents survive past the end of the trial (past 50000iterations). The mean survival-time of -/- agents was 22787iterations (std. 8486.1). As can be seen in Figure 4 the -/-agent moves around the arena, but in a less directed, morerandom walk than the N/T agent.

The motion of the -/- agent is caused by the agent de-pleting the localR concentration to levels where further pro-duction ofM becomes extremely unlikely. In areas whereRhas been so reduced,M levels fall, causing an asymmetricaldistribution ofM within the agent, causing an asymmetry inthe activation of cilia which produces motion. This mecha-nism is reminiscent of one of the extensions of the originalmodel of autopoiesis made by Suzuki and Ikegami (Suzuki

INTEGRATING AUTOPOIESIS AND BEHAVIOR: AN EXPLORATION IN COMPUTATIONAL CHEMO-ETHOLOGY 9

1000

800

200

400

600

00 10000 600003000020000 5000040000

IterationsIterations

R C

oncentr

ati

on

N/T1000

800

200

400

600

00 5000 300001500010000 2500020000

IterationsIterations

R C

oncentr

ati

on

-/-

Figure 5. R density plotted against time for healthy and lesioned (-/-)agents. Brighter values indicate that a larger portion of the arena hasthe indicatedR-concentration.

& Ikegami, 2009), in that lesioned agents only begin to movewhen part of them slows its contribution to autopoiesis. Thatis to say, the agents only move after they have come close to‘death’, i.e. close to their viability boundary. When in thisfragile state, the random variation in e.g. the particle trajec-tories can be sufficient to inhibit motion for long enough asto cause death.

Unlike the non-lesioned agents, the chemotaxis of -/-agents is modulated quite directly by the efficacy of the au-topoietic system. This direct relationship between behav-ior and autopoietic viability is a fragile organization in twoways. First, in a system such as -/-, response to a danger-ous phenomena requires a change (decrease) in efficacy ofthe autopoietic system. This is, by definition, a decrease inviability which correlates with an increase in fragility oftheagent. This increase in fragility makes the agent less capableof surviving various scenarios e.g. turning corners or endur-ing unlucky stochastic dynamics such as a brief unlucky mo-tion downtheR-gradient. Second, direct response to changein the efficacy of the autopoietic system can trap the agentin a local maximum. This local maximum is surroundedby more damaging situations, so the direct response wouldkeep the agent in the local maximum. However, under cer-tain circumstances, just on the other side of the more danger-ous situations there might be more advantageous situations.An agent whose behavior is determined directly by the effi-cacy of the autopoietic system would never find these supe-rior conditions, but an agent whose behavioral mechanismsare more decoupled from the autopoiesis might be capableof forging out into a less safe environment in search of asuperior environment. It is interesting to consider this dy-namic while examining the paths taken by the agents (Figure4). The N/T agents (with their partially decoupled behav-ioral mechanism) clearly make better use of resource, tend-ing to feed on the higher resource areas before consumingareas down to dangerous levels (see figure 5).

What we wish to highlight here is that a decoupled behav-ioral system, a system that is not as directly determined bythe efficacy of the autopoietic system, is not subject to thesame limitations. A behavioral mechanism could, for exam-ple, continually move the agent from one precarious situationto the next, and in such a motion, produce an environmentfor the autopoiesis that is actually less precarious than anyof the situations are on their own. This is a capability of adecoupled behavioral mechanism that as far as we can see isabsent in bare autopoietic systems.

Mechanisms that are related but distinct. Let us now ex-amine the -/T agents. As a result of lesioning theN enzymes,the concentration ofV becomes fixed. Speaking anthropo-morphically, these lesioned agents are incapable of changingtheir standards for what is good and what is bad. Without thisability, they move around the arena once, consuming someRfrom each area and then lie still. At this stage inappropriatelyhigh value ofV (compared to a healthy agent) will causeT tochangeall M to W preventing any asymmetry.

In the long term, these agents have a lower average healthand life-span compared to the non-lesioned agents (see rightcolumn in Figure 4), but in the short term the autopoiesiscontinues. The same can be said for the -/- agents. Further-more, it is easy to imagine a small change in the environment(e.g. if theR diffused through the environment) that makesthe motion of the agent unnecessary for the autopoiesis tocontinue. What we are driving at is that the behavioral mech-anism in all of these agents is related to autopoiesisbut alsodistinct. We have removed a component of the behavioralmechanism without removing a component of the autopoi-etic mechanism. The behavior affects the autopoiesis indi-rectly rather than being directly part of the autopoietic pro-cess. Because the behavioral mechanisms are somewhat de-coupled from the mechanisms of autopoiesis, it is possiblefor the entity to continue to existwithoutperforming the be-havior. This is, as far as we know, unlike previous models

10 MATTHEW D. EGBERT, EZEQUIEL DI PAOLO

Figure 6. Interdependencies within our model. Arrows representinterdependencies as in figure 1.

of autopoiesis (Suzuki & Ikegami, 2009) which if they wereself-maintaining were always performing their behavior (e.g.chemotaxis).

This relationship between the mechanisms of autopoiesisand behavior is asymmetrical. Without autopoiesis there isno agent to perform the behavior but the autopoiesis is notdependent (in the short term) upon the behavior. In the caseof our model, the chemotactic mechanism depends upon theautopoietic mechanism but the autopoietic mechanism canoperate in the absence of the chemotactic mechanism. Thisis not to say that the behavior is irrelevant to the autopoiesis.The health plots in Figure 4 clearly shows the behavior tendsto have an effect on the duration of autopoiesis, but the rela-tionship is more indirect and more subject to change than thedependence of the behavior upon autopoiesis.

To understand the relationships between these mecha-nisms it is helpful to view the system from a perspectiveof operational closure. This can be most easily realizedthrough study of Figure 6 which depicts the interdependen-cies present within our model4. The clearest example ofoperational closure in our model is the autopoietic system,which is highlighted by the grey filled circle. For theM par-ticles to be produced, the membrane must be large enoughand for the membrane to be large enough,M particles mustbe made. A membrane (physical boundary) is maintained bythese processes and therefore this collection of processesisnot only operationally closed but also autopoietic.

Just as the autocatalysis ofM relies upon the membraneencircling sufficient substrate particles, so does the autocatal-ysis ofT andN. These relationships are indicated by the ar-rows from ‘Membrane Size’ to ‘Production ofT’ and ‘Pro-duction ofN’. The T andN are integral to the reactions thatcause chemotaxis. But what is less clear is the relevance ofchemotaxis to the production ofM. This depends upon theenvironment in which the agent exists. For example, we canimagine that in an environment in which the consumption ofR is insignificant compared to the replenishment ofR, thenno motion is necessary for the agent. It can remain more orless stationary, existing indefinitely. If however, the agent

depletesR in its local environment, if it is to survive it mustmove to another area to find moreR. Because this depen-dence changes depending upon the environment in which theagent exists, we have depicted this dependence as a dashedarrow5. We wish to draw attention to these organizationalrelationships in order to highlight the non-intuitive notionthat the operationally closed system can change organizationover time. In this case, depending upon the environmentalconditions, the operationally closed system can include onlythe autopoietic cycle (the filled circle) or it can include boththe autopoietic cycle and the behavioral mechanisms (dashedcircle).

Finally, it is worth briefly considering a hypothetical alter-native system organization. What if the production ofT andN depended upon the chemotaxis? This would produce twonew operationally closed loops (one consisting of ‘Produc-tion of T’ and ‘Chemotaxis’, the other consisting of ‘Produc-tion of N’ and ‘Chemotaxis’). These loops have their own op-timal operating conditions and it would be possible for suchbehavioral loops to have norms different or even in conflictwith the norms of the primary autopoietic system (some dis-cussion of this can be found in Di Paolo, 2009). There isnot space to discuss this further here, but we intend to returnto this idea ofbehavior in conflict with autopoiesisin futureexplorations of this model.

Conclusions

This paper reports on a new computational model that en-ables us to explore how the relationships between mecha-nisms of behavior and autopoiesis. The model, which in-corporates aspects of computational artificial chemistry andneuroethology has helped us examine the idea of decou-pling in the context of mechanisms of autopoiesis and behav-ior. Specifically, three clarifications have been made throughanalysis of the model.

First, behavior modulation based upon autopoietic effi-cacy has limitations that can be avoided through the use ofa partially decoupled behavioral system. An agent whosebehavioral mechanism is based on its autopoietic efficacymust approach its ‘viability boundary’, becoming more frag-ile before it can respond. Furthermore, direct sensitivitytoautopoietic rates can trap an agent in local viability maximathat are not optimal solutions in the long term. These limita-tions can be avoided by having a behavioral mechanism thatis not as directly modulated by autopoietic processes.

4 It is worth mentioning that this Figure could be drawn at dif-ferent scales. That is to say we can describe the relationships ofdependence at different levels of detail (e.g. we chose not to includethe intermediate process of the creation ofW by T as it is not im-portant to our argument. It suffices to describe chemotaxis as beingdependent upon the production ofT).

5 It is also possible to imagine contrived scenarios in which theother interdependencies cease to exist. In this sense the dependenceof continued enzyme production upon chemotaxis is not unique.However, we have chosen to highlight the changeability of thisdependence as an example of how certain natural organizationallyclosed systems can be formed and cease to exist, as differentnaturalenvironmental conditions are encountered.

INTEGRATING AUTOPOIESIS AND BEHAVIOR: AN EXPLORATION IN COMPUTATIONAL CHEMO-ETHOLOGY 11

Second, mechanisms of behavior can be related to mecha-nisms of autopoiesis while remaining operationally distinct.We have been able to make more explicit the idea of decou-pling as a form of changing dependence and independencewithin operationally closed networks. This relationship ofrelated but independentis built upon the idea that opera-tionally closed systems change over time.

Third, the organization of an operationally closed systemcan change over time while remaining operationally closed.In this context of ‘dynamic operational closure’, behavioralmechanisms are those mechanisms that are not always inoperation and can, depending upon environmental condi-tions, affect viability. In contrast, autopoietic mechanismsare those mechanisms that are always required, extenuatingcircumstances aside, to occur if the agent is to continue toexist.

The simplicity afforded by our model (compared with bi-ological systems) has enabled us to see more clearly how be-havioral and autopoietic processes can operate over the samephysical components and have relations of dependence thatcan change depending upon the situation.

This relation between behavior and self-production re-veals an interesting possibility for operationally closedsys-tems, one that has rarely been made explicit in theoreticalterms. Namely, an identity can be maintained while the net-work of interdependencies changes over time. While themetabolic loop remains always active and unaltered in itsclosure, the dependence of some of its processes and innerrelations on the efficacy of behavior is an organizational, notmerely a structural, change to the system. We suspect thatthis possibility of entering temporarily different modes ofsystem organization can play an essential role in developingthe theory of autopoiesis towards an account of biologicaltransformations of organization (e.g. from embryo to adult,or the appearance of novelty across generations), which is atthe moment one of its blind-spots.

To summarize, this model has allowed us to see the re-lationship between self-production and behavior-productionin a more concrete manner. We intend to continue to fur-ther explore how simultaneous mechanisms, operating overthe same physical space can interact to produce the cognitiveabilities that we associate with the notion of agency. Fu-ture study may include increasing the variety of environmen-tal conditions within the model. For example, one area ofthe arena might contain non-diffusingR, while another areacontains diffusingR. In these different environments, the de-pendence of the autopoiesis upon the chemotactic behaviordepends upon the location of the agent. This would allow usto study the changing dependency and operational closure inone continuous system. In the more distant future it wouldbe interesting to implement an evolutionary process in thisenvironment to see what the long term trends would be interms of the relationship between behavioral and autopoieticmechanisms.

Acknowledgements

Thanks to Xabier Barandiaran, Tom Froese, NathanielVirgo, Lucas Wilkins, Marieke Rohde and the three anony-mous reviewers for their helpful comments. Also, thank youto the participants and organizers of the Kyoto Agency Work-shop for the interesting conversation that inspired this work.

12 MATTHEW D. EGBERT, EZEQUIEL DI PAOLO

About the authors

Matthew D. Egbert

Matthew Egbert is a PhD student, studying theoreti-cal biology at the Centre for Computational Neuroscienceand Robotics (CCNR) at the University of Sussex. Hisresearch interests include metabolism, the origin of life,self-production, self-organization and embodied cognition.Matthew completed a BSc in Computer Science from theUniversity of St. Andrews in 2001 and an MSc with distinc-tion in Evolutionary and Adaptive Systems from the Univer-sity of Sussex in 2007.

Ezequiel Di Paolo

Ezequiel Di Paolo is a Reader in Evolutionary and Adap-tive Systems at the University of Sussex and co-director ofthe Evolutionary and Adaptive Systems (EASy) MSc pro-gramme. He is a member of the Centre for ComputationalNeuroscience and Robotics (CCNR) and the Centre for Re-search in Cognitive Science (COGS). He is the author ofover 100 peer-reviewed publications and his interests in-clude: adaptive behaviour in natural and artificial systems,biological modelling, evolutionary robotics, embodied cog-nition, philosophy of mind and philosophy of biology.

References

Barandiaran, X., & Moreno, A. (2008, October). Adaptivity:Frommetabolism to behavior.Adaptive Behavior, 16(5), 325–344.

Beer, R. D., & Chiel, H. J. (2008). Computational neuroethology.Scholarpedia, 3(3), 5307.

Blair, D. (1995). How Bacteria Sense And Swim [Review].AnnualReview Of Microbiology, 49, 489-522.

Cliff, D. T. (1991). Computational neuroethology: A provisionalmanifesto. In J. A. Meyer & S. W. Wilson (Eds.),In from an-imals to animats: Proceedings of the first international confer-ence on simulation of adaptive behavior.Cambridge, Mass.:MIT Press.

Di Paolo, E. A. (2005). Autopoiesis, adaptivity, teleology, agency.Phenomenology and the Cognitive Sciences, 4(4), 429-452.

Di Paolo, E. A. (2009). Extended life.Topoi, 28, 9-21.Dittrich, P., & di Fenizio, P. S. (2007). Chemical organization the-

ory: towards a theory of constructive dynamical systems.Bul-letin of Mathematical Biology, 69, 1199.

Dittrich, P., Ziegler, J., & Banzhaf, W. (2001). Artificialchemistries—a review.Artif. Life, 7(3), 225–275.

Egbert, M. D., Di Paolo, E. A., & Barandiaran, X. E. (2009).Adaptive behavior of a motile protocell: Sensor-less sensitiv-ity to emergent viability conditions. InAdvances in ArtificialLife, Proceedings of the Ninth European Conference on Artifi-cial Life, ECAL (forthcoming).Springer Verlag.

Falke, J. J., Bass, R. B., Butler, S. L., Chervitz, S. A., & Danielson,M. A. (1997). The two-component signaling pathway of bac-terial chemotaxis: a molecular view of signal transductionbyreceptors, kinases, and adaptation enzymes.Annual Review ofCell and Developmental Biology, 13, 457–512.

Hanczyc, M., Toyota, T., Ikegami, T., Packard, N., & Sugawara, T.(2007). Chemistry at the oil-water interface: Self-propelled oildroplets.J. Am. Chem. Soc., 129(30), 9386 - 9391.

Harvey, I., Di Paolo, E., Wood, R., Quinn, M., & Tuci, E. (2005).Evolutionary robotics: A new scientific tool for studying cogni-tion. Artificial Life, 11(1-2), 79–98.

Macnab, R. M., & Koshland, D. E. (1972, Sep). The gradient-sensing mechanism in bacterial chemotaxis.Proceedings of theNational Academy of Sciences USA, 69, 2509–2512.

Maturana, H. R., & Varela, F. J. (1980).Autopoiesis and cogni-tion : the realization of the living[Book]. D. Reidel Pub. Co.,Dordrecht, Holland ; Boston :.

McMullin, B. (2004). Thirty years of computational autopoiesis: Areview. InArtificial life 10 (p. 277-295). MIT Press.

Moreno, A., & Etxeberria, A. (2005). Agency in natural and artifi-cial systems.Artificial Life, 11(1-2), 161–176.

Ruiz-Mirazo, K., & Mavelli, F. (2007). Simulation model forfunc-tionalized vesicles: Lipid-peptide integration in minimal proto-cells. In F. A. e Costa, L. M. Rocha, E. Costa, I. Harvey, &A. Coutinho (Eds.),Ecal (Vol. 4648, p. 32-41). Springer.

Ruiz-Mirazo, K., & Moreno, A. (2004). Basic autonomy as a fun-damental step in the synthesis of life.Artificial Life, 10(3), 235–259.

Suzuki, K., & Ikegami, T. (2009). Shapes and self-movement inprotocell systems.Artificial Life, 15(1), 59-70.

Thompson, E. (2007).Mind in life: Biology, phenomenology, andthe sciences of mind(1 ed.). Belknap Press.

Varela, F. J. (1979).Principles of biological autonomy. North-Holland, New York.

Varela, F. J., Maturana, H. R., & Uribe, R.(1974). Autopoiesis: Theorganisation of living systems, its characterization and amodel.In Biosystems(Vol. 5, p. 187-196).

Virgo, N., Egbert, M. D., & Froese, T.(2009). The role of the spatialboundary in autopoiesis. InAdvances in Artificial Life, Proceed-ings of the Ninth European Conference on Artificial Life, ECAL(forthcoming).Springer Verlag.