M

Da

b

a

ARR2AA

KENM

1

aimvtttnspadgC

aoiLssmT

L

0d

Ecological Modelling 221 (2010) 400–404

Contents lists available at ScienceDirect

Ecological Modelling

journa l homepage: www.e lsev ier .com/ locate /eco lmodel

utual information in the Tangled Nature model

ominic Jonesa,b, Henrik Jeldtoft Jensena,b,∗, Paolo Sibania,b

Imperial College London, South Kensington Campus, London SW7 2AZ, United KingdomInstitut for Fysik og Kemi, SDU, DK5230 Odense M, Denmark

r t i c l e i n f o

rticle history:eceived 15 June 2009eceived in revised form

a b s t r a c t

We consider the concept of mutual information in ecological networks, and use this idea to analysethe Tangled Nature model of co-evolution. We show that this measure of correlation has two distinctbehaviours depending on how we define the network in question: if we consider only the network

9 September 2009ccepted 25 October 2009vailable online 1 December 2009

eywords:volutionary ecology

of viable species this measure increases, whereas for the whole system it decreases. It is suggested thatthese are complimentary behaviours that show how ecosystems can become both more stable and betteradapted.

© 2009 Elsevier B.V. All rights reserved.

etwork structureutual information

. Motivation

Identifying universal features of ecosystem dynamics has beenlong-standing goal in ecology. These attempts have usually

nvolved identifying system variables that are potentially opti-ised during the evolution of an ecosystem. Many such candidate

ariables have been identified. Increasingly the focus has been onhe network properties of the ecosystem, or more precisely therophic net defined by the mass flows between the species consti-uting the ecosystem. However empirical evidence at the resolutioneeded to verify any particular claim remains out of reach for mosttudies. For ecologists these quantities are both of theoretical andractical interest. From a theoretical point of view it would be nice,s already noted, to find some governing principle of ecologicalynamics, while practically speaking there is a need to establish aood measure of ecosystem health and maturity (Ulanowicz, 2002;hristensen, 1995).

In this paper we propose to study this issue in the context ofwell established evolutionary model. The Tangled Nature modelf co-evolution (Christensen et al., 2002) has already been stud-ed in several contexts (Hall et al., 2002; Sibani and Jensen, 2005;awson and Jensen, 2006) and is ideal for this work as it is designed

pecifically to study long time behaviour in ecological networks. Itsimplicity along with the rich complexity of its resulting behaviourakes it a paradigmatic model for testing co-evolutionary ideas.

he model retains the binary string genotype geometry found in

∗ Corresponding author at: Imperial College London, South Kensington Campus,ondon SW7 2AZ, United Kingdom.

E-mail address: h.jensen@imperial.ac.uk (H.J. Jensen).

304-3800/$ – see front matter © 2009 Elsevier B.V. All rights reserved.oi:10.1016/j.ecolmodel.2009.10.036

previous approaches (for example the quasispecies model (Eigenand Schuster, 1977)or the NK model (Kauffman, 1990), but replacestheir ‘ad hoc’ static fitness landscapes with a set of populationdependent interactions between extant species, similar to the ‘tan-gled’ interactions of an eco-system. From a ‘random’ initial state,the network of extant and interacting population changes overtime, slowly, but radically, enabling the system to support an evergrowing number of individuals.

Despite its simplicity, the model is able to reproduce the longtime decrease reported in the overall macroscopic extinction rate,the observed intermittent nature of macro-evolution, denotedpunctuated equilibrium by Gould and Eldredge, the log-normalshape often observed for the Species Abundance Distributions, thepower law relation often seen between area and the number of dif-ferent species number, the framework of the model is also ableto reproduce often reported exponential degree distributions ofthe network of species as well as the decreasing connectance withincreasing species diversity that has attracted much observationaland theoretical interest.

The details of the model are described in greater detail below,but the key aspect of its behaviour is that it moves through a seriesof different network configurations. In this paper we analyse thesedynamic networks using tools developed in ecology. In particu-lar, we are able to shed light on the tension between robustnessand efficiency in ecological networks highlighted by Jorgensen etal. (2007). Increased correlation lead to greater brittleness in the

case of perturbations, but greater robustness leads to an appar-ent squandering of resources. We suggest how this conflict can beresolved using evidence from Tangled Nature, where it is possibleto divide the system into two interacting parts—a viable networkof keystone species, and a periphery of unviable mutants. Seen

Modelling 221 (2010) 400–404 401

fvnr

2

2

tvastipt

N

imi

H

wdtS

ttmn〈ibo�btaitoiht

2

sr

P

oAi

Fig. 1. An example of the configuration of the Tangled Nature system in a meta-stable state. This is a four dimensional model for expository purposes only, the modelin this paper has 20 dimensions. The vertices of the hypercube represent the 16possible types in the model. The dotted lines represent nearest neighbour links intype space, and the solid lines represent non-zero interaction terms with blue =

interactions between species, as evidenced for example by the log-arithmically increasing population (Fig. 3). It is this non-stationaryaspect of the model that this paper tries to explain, albeit onlypartially.

Fig. 2. Overview of a typical run of TaNa. The y-axis is simply a species label, ranging

D. Jones et al. / Ecological

rom this perspective the apparent paradox is resolved, as theiable network becomes increasingly correlated, while the totaletwork (including many species in potentia) develops greateredundancy.

. Review of the basic behaviour of the model

.1. Type space and the interaction matrix

A type is represented by a vector S of L elements belonging tohe set [0, 1]. Thus there are 2L possible types, corresponding to theertices of a unit hypercube in L-dimensions. S may be interpreteds a genome, or a set of characteristics—either way it is directlyusceptible to mutations and defines the type completely (that ishere is no phenotype level in this model). Each type, which we canndex by a number i in the range 1 to 2L to simplify notation, has aopulation of ni(t) identical individuals, so the total population ishe sum over all the 2L possible types

(t) =2L∑

i=1

ni(t) (2.1)

The ability of an individual to reproduce is determined by howt interacts with the other types present at a given time. This is for-

alised in the reproduction weight function (which is then turnednto a probability of reproducing—see below)

i(t) = c

N(t)

2L∑i=1

J(Si, S)n(Si, t) − �N (2.2)

here the sum is over all other types, C is a control parameter thatetermines the level of inhomogeneity in the population, N(t) ishe total population at time t, and n(S, t) is the population of type.

Two types Si and Sj are coupled via the interaction matrix J(Si, Sj)hat can be either positive negative or zero. This number is intendedo be the sum of all the influences of i upon j. This interaction

atrix is unrelated to the type space outlined above so there areo correlations in the interactions between different types—that isJ(Si, Sj)J(Sk, Sj)〉 = 0 even if the average is restricted to neighboursn type space. This interaction is not necessarily material in natureut may represent any influence that one type has on another. Theverall connectivity of the interaction matrix is set by a parameter

which for this paper has a value of 0.2 (that is 0.2 of all possi-le connections between types actually exist). The distribution ofhe nonzero values of the function J(Si, Sj) are irrelevant as longs they are distributed in some reasonable, continuous way. Thenteraction matrix is constructed such that if J(Si, Sj) is nonzerohen J(Sj, Si) is also nonzero. This means there are three typesf interaction—mutualistic, antagonistic and predator–prey. Fig. 1llustrates the key components of the tangled nature model—theypercubic type space, varying type occupancies, and the differentypes of possible interaction between types.

.2. Reproduction, mutations and death

The model is simulated stochastically, with a time-step con-isting of the following: one individual is selected at random, andeproduces asexually according to the probability

r(Si, t) = 11 + exp[H(S , t)]

∈ [0, 1] (2.3)

i

If successful the individual is replaced with two copies. In eachf these copies there is a probability of mutation per ‘gene’, pm.nother individual is picked at random and is killed with probabil-

ty pk.

black solid, red = dashed in front, green = dashed behind (For interpretation of thereferences to colour in this figure legend, the reader is referred to the web versionof the article.).

2.3. General behaviour of the model

We start a run with N(0) = 1000 individuals on one randomlychosen site. Initially there is no reproduction, since there canbe no interactions between species, so H is very negative andthe probability of reproduction is zero. Then as the resourcelimitation term diminishes, reproduction becomes possible, andconsequently some new types are generated by mutations. Onceinteractions between these new types begin, the interaction termin the reproduction probability becomes significant. After some re-organisation, a set of species that interact in a stable way emerges,and persists for some time (see Fig. 2). This period of stabilityis ended by another chaotic reorganisation, from which anothermeta-stable state emerges.

The bulk properties of these meta-stable states turn out todepend on the age of the system—the system slowly optimises the

from 1 to 2L , and the x-axis is time in generations. If a position is occupied at agiven time, a dot is placed at the corresponding number for that time step. Theplot clearly shows the alternating stable and unstable periods. The stable periodsare characterised by a steady population and constant set of species, whereas thetransitions have a constantly changing set of species (e.g. between 100,000 and150,000 generations). Figure from Christensen et al. (2002).

402 D. Jones et al. / Ecological Model

Fl

3

tmri

0dsiid

3

taawshddeTt

3

oaiawfmul

over the ensemble P(J, t)) measures the degree to which the pres-ence of some link value J1 influences the presence of some othervalue J2.

To consider the probability of a link value appearing at time t, wefirst introduce a new variable which will simplify the following. We

Fig. 4. A typical stable ecological network seen in the Tangled Nature model. Eachcircle represents a species with the size proportional to the current population.

ig. 3. The mean population (averaged over an ensemble of 1000 runs) increasesogarithmically in time.

. Results

We ran 1500 simulations of the model with an initial popula-ion confined to one randomly chosen site. The random interaction

atrix was regenerated each time. The parameters used for all theuns were the same, and were chosen to robustly generate thentermittent regime for a population of a manageable size.

We use the following parameter values: � = 0.14, pmut =.03, pkill = 0.2, c = 10. Detailed discussion of the various regimesefined by these parameters can be found elsewhere; for now weimply note that the behaviour generated by this set is character-stic of a significant area of parameter space. The one major changes seen when pmut goes above the error threshold, which results iniffusion dominated behaviour.

.1. The core and the periphery

The network realised at any given time can be divided intowo classes—those nodes that are viable (loosely, those that havebirthrate approximately equal to the death rate) and those that

re not. This second group are the mutants from the viable core,ho in the current configuration are not able to reproduce. Fig. 4

chematically depicts this arrangement, with each viable speciesaving a flower of unviable mutants surrounding it. These mutantso not, in general play an active role (even as a stabilising factor)uring a stable period, but they are in the end responsible for theventual collapse of one metastable state and creation of another.he following results are obtained for both the whole system, andhe viable core.

.2. Mutual information

Ideas from information theory have been used in ecology forver 50 years (MacArthur, 1955; Wilhelm and Hollunder, 2007),nd Routledge et al. (1976) introduced the idea of using the mutualnformation of networks as a measure of their stability. This wasll somewhat unnoticed by those working more recently on net-orks in graph theory and complexity. This is principally due to the

act that ecologists must work with weighted networks, whereasost recent work on network characterisation has focussed on

nweighted networks, for which there exist a large arsenal of ana-ytical tools.

ling 221 (2010) 400–404

First we define what the mutual information is for a generalrandom process, then we will define how we use this measure inthis paper. The information of a realisation x of a random variableX is defined via its probability distribution P(x), as

I(x) = P(x)log P(x) (3.1)

For two random variables, we can define the mutual informa-tion, which is defined as the reduction in the uncertainty of X givenknowledge of Y . The mutual information is defined on two randomvariables X and Y as

I(X, Y) =∑x,y

P(x, y) log(

P(x, y)P1(x)P2(y)

)(3.2)

where P1 and P2 are the marginal distributions of X and Y respec-tively, and P the joint probability distribution. Equally we can thinkof the mutual information as the constraint imposed on X by Y .

The Tangled Nature model is a model of network evolution. Asthe structure of the network changes, we ask the question: howdoes the current network structure constrain its evolution? Thenetwork we consider is the interaction network J weighted by theoccupancy of the species, so that we only consider connectionsbetween extant species. When this condition is met, we considerthere to be ninj copies of link Jij . Consider the ensemble link valuedistribution at time t, P(J, t). This gives the probability of a linkvalue J for an ensemble of realisations. However if we consider aparticular realisation, we can expect that this distribution, P(J, t, r)(where r indexes specific realisations) will in general differ from theensemble average. We can measure this difference by looking at thejoint probability distribution P(J1, J2, t, r). The degree to which thisquantity differs from the product of the marginal distributions forJ1 and J2 (which in our case are identical, equal to the distribution

Lines represent interaction links (|J| > 0) between species. The mutation networkhas been suppressed to clarify the figure but the line indices give the hammingdistance (a measure of evolutionary separation) between each species. The core isthe network of large nodes, and the whole system includes the smaller unlabellednodes. Figure from Christensen et al. (2002).

Modelling 221 (2010) 400–404 403

watra

P

wD

ps

P

Wt

I

od

ofod

woiamvai

estsp

rt4

4

stsgdtro

e(b

Fig. 5. Contrasting trends in the mutual information for different subsets. (a) Themutual information of the whole network as a function of time. We see a slow butsignificant decrease signifying a decorrelation of the component parts of the net-work. (b) The mutual information of the core as a function of time. For this subset

D. Jones et al. / Ecological

ill consider a single index k that runs over all links in a realisation,nd each link is waited by dk, the product of the occupancy of thewo nodes at either end: dk = d(Jij) = ninj . Explicitly we define theelevant quantities as follows: the probability that the link value Jppears at time t is

(J, t) = 1DR

∑k,r

dkı(J − Jk) (3.3)

here D is the number of links counted between all extant species,= ∑

i,jninj and R is the number of realisations, whereas the jointrobability distribution for two link values to appear in one reali-ation is

(J1, J2, t, r) = 1D

∑k,l

(dk + dl)ı(J1 − Jk)ı(J2 − Jl) (3.4)

ith these quantities defined, we may define the mutual informa-ion on these distributions as

(J, J′, t) =∑r,i,j

P(Ji, Jj, t, ) log

(P(Ji, Jj, t, r)

P(Ji, t)P(Jj, t)

)(3.5)

Since the link distribution fluctuates due to the stochastic naturef the system, this distribution is calculated over a small time win-ow ıt where (ıt/tmax << 1).

Fig. 5(a) and (b) shows the evolution of the mutual informationver time for two different subsets of the system. Fig. 5(a) is the MIor the whole system, where we see a declining trend. The subsetf vertices linking nodes with more than 5 individuals by contrastisplays an increase in the MI over time (Fig. 5(b)).

We note that in general the mutual information is quite low,hich is expected. We are measuring the influence of the presence

f link values on the presence of other link values; this influences highly constrained by the quenched randomness of the networknd the stochastic dynamics, so in general we do not expect theutual information to be high. Nevertheless we have compared the

alues obtained to simulated random networks of equivalent sizend connectance, and found the mutual information to be approx-mately three orders of magnitude smaller.

The data is significantly noisy despite being the result of a largensemble average. Nevertheless it is clear, especially for the wholeystem, that the curves are approximately linear in logarithmicime. This corresponds to the behaviour of other measures of theystem, and can possibly ultimately be related back to some recordrocess.

Averaging over more realisations increased the clarity of theesults, but at the cost of computing time. To decrease the fluctua-ions by an order of magnitude would have required approximately00 weeks more computing time.

. Discussion

The question of how the structure of an ecosystem, or anyystem of interacting, evolving agents, changes over time is a con-roversial one, and to some extent depends on the details of theystem under consideration. In this paper we have considered aeneric evolutionary model with the aim of elucidating ecologicalynamics in the general case. The apparent competition betweenwo requirements of a viable ecosystem – that they maximiseesource us on the one hand, and remain robust to perturbations

n the other – poses the question: what in fact happens?

The obvious way to answer this question would be to do anxperiment. However, ecological experiments of the type requiredboth in terms of detail and time resolution) are not currently possi-le. Indeed, ecological data recorded over evolutionarily significant

the mutual information increases over time, indicating greater correlation and effi-ciency. (c) The mutual information of a random set of networks using the sameaverage diversity and population as the simulation results. The mutual informationhas no trend and is approximately 2 orders of magnitude smaller.

4 Model

tiesTopf

lsswprs

rflycrtitrruawgpni(pd

mfdTistsssTepiem

Sibani, P., Jensen, H.J., 2005. Intermittency, aging and extremal fluctuations. EPL(Europhysics Letters) 69 (4), 563–569.

04 D. Jones et al. / Ecological

imescales is practically unattainable for any but the fastest evolv-ng systems, such as microbial populations (see, for example, Blountt al., 2008). However, even for such experimentally manipulableystems it may be hard to infer interaction networks accurately.he practical difficulties of experiments in evolutionary ecology isne of the key reasons why we believe theoretical work such is thatresented is important, since it can act both as spur and guide foruture experimental work.

We have found that while the ecosystem as a whole becomesess correlated over time, the correlation of the network of its corepecies increases. While we have not shown it here, it seem plau-ible that this is two sides of the same coin—decorrelation of thehole system implies that the system explores a greater range ofossible networks, from which it chooses more and more well cor-elated subsets. This fits with other results we have obtained thathow the model increases its population over time.

When considering ecological networks, most work has natu-ally focussed on trophic networks that is networks of materialow through an ecosystem. This has yielded a natural way to anal-se these networks, since the dynamics is conservative, one canonsider the probability of any two species being involved in mate-ial exchange. The Tangled Nature model explicitly models morehan simply mass flow in ecosystems: it attempts to quantify thenfluence that one species has on another. While this has the advan-age of allowing one to consider more than simply predator–preyelationships (for example mutualistic behaviour arises very natu-ally in the model), it means that one cannot simply take over toolssed on trophic nets wholesale. In this paper we have adapted thepproach used in ecology and elsewhere to this interaction viewith the caveat that our results are not directly comparable to those

leaned from analysis of food webs; we did also attempt to inter-ret the model as a flow model but found that this approach yieldedo clear information about the network structure. One possibility

n this direction is to adopt the approach in Demetrius and Ziehe2007) where once a network has evolved one imagines some sim-le Markovian dynamics entirely independent of the actual modelynamics in order to determine the relevant network measures.

We have not used any of the more simple information theoreticeasures available (for example the entropy). This is because we

ound it necessary to consider a quantity that characterised theifference of a specific realisation from an ensemble of realisations.he entropy of the system as a whole increases over time, but theres no corresponding decrease in the core population. It is easy toee why: the entropy over an ensemble of realisations is simplyhe sum of individual realisations and so one would only expect toee a decrease in entropy if every realisation converged on a smallet of link values. This by no means has to be the case, since theystem can adjust species populations to a wide range of networks.he mutual information, on the other hand, measures how the

xistence of certain links within one realisation determines theresence of other links within that same realisation and so does

ncrease over time. It remains to be seen whether there is somentropic measure in Tangled Nature (or indeed in reality) which isaximised through evolution.

ling 221 (2010) 400–404

One might naively think that the result for the core is simplydue to the increasing stability of the system observed in other con-texts. Taken by itself this is reasonable, since it is possible that thesystem stabilises over time, and that this stabilisation would posi-tively contribute to mutual information of the core. However, if itwas purely an artefact of the system spending more time in a sta-ble configuration then we would expect the whole system (that isboth the viable core and the surrounding mutants in Fig. 4) to dis-play a similar positive trend, which is clearly not the case. Thereforewe conclude that the increasing correlation of the core, along withthe increasing decorrelation of the periphery of the system, playsa causal role in the stabilisation of the system as a whole. We pos-tulate that these two phenomena are linked - the system exploresa greater number of possible links which allows it to find betteradapted sets of links for the core, which in turn leads to a biggerpopulation and an even larger set of links to select from. Whilewe do not claim to have proved that this is the case, the data isstrong evidence that some adaptive behaviour of this type is occur-ring. In future papers we hope to probe the nature of this adaptivedynamics further.

Acknowledgement

HJJ grateful acknowledge support from the EPSRC under con-tract EP/D051223.

References

Blount, Z.D., Borland, C.Z., Lenski, R.E., 2008. Historical contingency and the evolutionof a key innovation in an experimental population of Escherichia coli. PNAS 105(23), 7899–7906.

Christensen, V., 1995. Ecosystem maturity—towards quantification. Ecological Mod-elling 77 (1), 3–32.

Christensen, K., Di Collobiano, S.A., Hall, M., Jensen, H.J., 2002. Tangled Nature:a model of evolutionary ecology. Journal of Theoretical Biology 216 (1),73–84.

Demetrius, L., Ziehe, M., 2007. Darwinian fitness. Theoretical Population Biology 72(3), 323–345.

Eigen, M., Schuster, P.K., 1977. A principle of natural self-organisation. Naturwis-senschaften 64, 541–565.

Hall, M., Christensen, K., di Collobiano, S.A., Jensen, H.J., 2002. Time-dependentextinction rate and species abundance in a tangled-nature model of biologicalevolution. Physics Review E 66 (1), 011904, Jul.

Jorgensen, S.E., Bastianoni, S., Fath, B.D., Muller, F., Marcques, J.C., Nielsen, S.N., Pat-ten, B.C., Tiezzi, E., Ulanowicz, R.E., 2007. A New Ecology: Systems Perspective.Elsevier.

Kauffman, S.A., 1990. The Origins of Order: Self Organisation and Selection in Evo-lution. Oxford University Press.

Lawson, D., Jensen, H.J., 2006. The species–area relationship and evolution. Journalof Theoretical Biology 241 (3), 590–600.

MacArthur, R., 1955. Fluctuations of animal populations and a measure of commu-nity stability. Ecology 36 (3), 533–536.

Routledge, R.W., Basore, B.L., Mulholland, R.J., 1976. Ecological stability: an informa-tion theory viewpoint. Journal of Theoretical Biology 57, 355–371.

Ulanowicz, R.E., 2002. The balance between adaptability and adaptation. Biosystems64 (1–3), 13–22.

Wilhelm, T., Hollunder, J., 2007. Information theoretic description of networks. Phys-ica A: Statistical Mechanics and its Applications 385 (1), 385–396.