Multi-Agent Systems

Andrew ArcherRichard Hesketh

Jonathan MallinsonLia Marzan

March 8, 2011

Contents

1 Introduction 2

2 Cellular Automata 32.1 Advantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 Elements and Variation . . . . . . . . . . . . . . . . . . . . . . . 32.3 Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.4.1 Case Study: The Wolfram Approach . . . . . . . . . . . . 72.4.2 Case study:Reversible Cellular Automata . . . . . . . . . 8

2.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 Crowd Simulations 93.1 General Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.2 Potential Challenges and Limitations . . . . . . . . . . . . . . . . 93.3 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.4 Tools, Methods and Variants . . . . . . . . . . . . . . . . . . . . 103.5 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.5.1 Case Study: Massive Software . . . . . . . . . . . . . . . . 12

4 Particle Swarm Optimisation 144.1 Basic Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 144.2 Performance of Particle Swarm Optimisation . . . . . . . . . . . 154.3 Variations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

5 Stochastic Diffusion Search 165.1 Basic algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175.3 Variations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185.4 Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185.5 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

6 Ant Colony Optimisation 186.1 Ant Behaviour . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196.2 Ant System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196.3 The ACO Metaheuristic and Variations . . . . . . . . . . . . . . 216.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

6.4.1 Total Weighted Tardiness Problem . . . . . . . . . . . . . 236.4.2 AntNet . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

6.5 Advantages and Disadvantages . . . . . . . . . . . . . . . . . . . 25

7 Conclusion 26

1

1 Introduction

There are countless examples that can be found in nature of complex behaviourarising from the simple interactions of multiple individuals, such as the flockingof birds, shoaling of fish, or co-operation of ants. These animals are capable ofimpressive feats despite only following simple rules. Such examples in naturehave served to inspire a large number of algorithms that take advantage ofthis sort of behaviour to realistically simulate the world or to solve practicalproblems.

Such multi-agent systems are powerful and varied, and all share some im-portant properties:

• Consist of several partially independent agents

• Agents do not have global knowledge of the simulation (or no practicalmeans of applying the knowledge)

• No centralised controller

The result is a group of agents following simple rules that combine to producesome sort of desired behaviour, whether that is modelling how a panicked crowdbehaves in a building layout or finding a good solution to a computationallyintensive problem.

One of the most significant uses of multi-agent systems is to solve difficultoptimisation problems. In such a problem, the task is to find values for a setof parameters which satisfy some constraints and maximise or minimise an as-sociated evaluation function. For example, given a mathematical equation, thesystem may be tasked with finding values for the variables in the equation thatproduce the smallest possible value when the equation is evaluated. Generally,the types of problems which multi-agent systems are used to produce solutionsfor are what are considered NP-hard problems - problems for which there areno known ways of solving precisely, or those that are known are far too compu-tationally intensive to be practically solvable. Rather than trying to calculatethe perfect solution, the system produces approximate solutions.

Using rules to produce an approximate solution is a heuristic approach toproblem solving. The optimisation techniques described in this report are knownas metaheuristics - they provide a general framework for applying a specific com-putational technique to solving problems, which are then modified and tunedto the specific problem being solved[33]. As multi-agent systems, they typicallywork by having individual agents construct solutions, evaluating them to findout how good they are, and then inform the other agents about their results insome manner to help direct the agents to produce better solutions.

Different types of metaheuristic are useful for different classes of problem.As we will see, a technique which is a valid approach to solving one problemmay not work well or even be applicable at all to a different one.

2

2 Cellular Automata

A cellular automaton is a multi-agent system that has the ability to performa wide array of computations [2]. There are many different types of cellularautomatons, but almost all of them have cells arranged on an n-dimensionalplane, capable of holding a distinct state. Thcese states change depending ontransition rules, which use the states from neighbouring cells. These rules areapplied simultaneously to each cell at distinct time steps [15].

2.1 Advantages

Cellular automata are a very powerful technique. They are flexible enough to doa variety of tasks from simulations to cryptology, and have the ability to mimicother nature inspired designs such as particle swarms and crowd simulations,although this is usually done only as an academic exercise [16] [3]. They areTuring complete, which means that anything computable can be done usinga cellular automaton, although, again, this is has very little real world use.There are, however, a series of hobbyists who use a rule set called Wireworld toimplement their own computers using only Cellular automata [14] [12].

The main strength and practical use of cellular automata is within modellingspatial natural events, as a lot of natural events depend on locality as wellhomogeneity (particle diffusion, simulating the group behaviour of swarms offish, etc.).

2.2 Elements and Variation

There are multiple types of cellular automata, each designed to perform a varietyof different tasks.

Probably the most important factor in the design of a cellular automaton isthe set of states that each cell can be set to. Normally, all cell in an automatonhave the same set of possible states. Still, this is not always the case: poly-geneous cellular automaton use multiple types of cells - however, this type ofcellular automaton has received very little attention. Not many uses have beenfound for it, but theoretical studies have nonetheless been made [15].

Another variation involves having not only a cell state, but also a set ofparameters for each state. An example of this is a movable cellular automaton(MCA), used to simulate behaviours of material (such as fracturing). In thissystem, each cell holds a parameter, which indicates the amount of overlap it haswith each neighbour [13]. A standard cellular automaton would not allow this,as each cell would need to have multiple parameters(one from each neighbour).Additionally, normal cellular automata only use integers, whereas the valuesused in the above simulation would be real values. Movable cellular automatatherefore allow for better modelling of stress in materials, as the direction ofcompression can be taken into account.

Another important aspect in cellular automata design is geometry - thelayout of the grid containing the cells. Choices in geometry design include

3

whether or not the grid is finite and whether it wraps around itself (i.e. oneedge is considered to be neighbouring the opposite edge) [15]. Again, the optiondepends on the simulation. For example, finite grids are simpler to simulate, butmay be less accurate. The shape of the cell can also vary - while usually square,it is possible have other shapes as well (e.g. hexagons) [15]. There are alsocellular automata where the geometry changes with each time step. Althoughthis type would be still considered static if the neighbours dont change, if theydo, the cellular automaton would be dynamic. Dynamic cellular automaton isused in biology, for things such as modelling diffusion enzyme rate processes andmetabolism [7]. For this type of problem, the cellular automaton needs to bedynamic: it involves a particle movement in a random directions, which couldntbe done in a static CA.

Neighbourhoods (the set of cells which can influence each other) can alsovary. Normally, cells are considered to be neighbours if they are touching eachother. In the case of square cells, this would normally sum up to either 4 (vonNeumann neighbourhood) or 8 (Moore neighbourhood) neighbours [15]. Lesscommonly used are cells that take an area around them as neighbours, similarlyto random spatial networks [15] - demonstrationg how cellular automata can,with a few adjustments, model other nature inspired designs. Whether this isuseful or not is debatable: it would be useful to be able to use one model foreverything, but other models are more specialised and (in cases such as randomspatial networks) give more options.

Finally, rules are used to change cell states from one to another at eachtime step. While these rules are usually deterministic, this does not have tobe the case. There is also the possibility of each cell having its own rules - ahybrid cellular automaton, for instance, was used to study forest fire spread,with different types of trees having different rules for catching fire.

2.3 Algorithms

Most algorithms used to compute cellular automata will be very similar, withthe bulk of the algorithm being used to update the cellular automaton at eachtime step. The standard way of doing this is to create a new array, of thesame size as the current array of cells, then calculate the state of each cellstarting at the top left, putting the new states in the new array. The originalarray remains unaltered, and the automaton is only updated once all the newcell states have been calculated. This is a sequential method that can do theupdate with complexity O(number of cells) [4].

The sequential approach has the advantage of being commonly used (andtherefore well understood), as well as being simple to implement. There arealso slight variations which can offer a significant increase of speed. The Hash-life method involves saving the states and outcomes of an array of a cellularautomata. Then, when a pattern has been previously encountered, it simplyretrieves the result from memory, without having to calculate it again [4]. Thisis very useful for computing problems that have repeating patterns, but worse

4

than the sequential approach if there is no repetition, as time is wasted calcu-lating hash values [4].

If faster computation is required, then multiple CPUs will need to be takenadvantage of, resulting in parallel computation. This can be problematic, asupdating needs to be done simultaneously between all cells. There are twomain ways of doing this: process per cell, where every cell has a process, andprocess per block, where a collection of neighbouring cells has a process [4].

The simplest method is for each process to represent a cell. Then, at eachtime step, every cell transmits its state to its neighbours, resulting in everycell simultaneously updating its state. However, this procedure can be wastefulcompared to the sequential approach. The majority of the resourses are put intocommunicating values, which wastes time if all the neighbours for one cell havenot changed state (i.e. the cell wont change state). There should therefore beno communication unless the cell state has changed. As updates wouldn’t occurevery time period, there would need to be a clock to keep track of time. Thisapproach would only work on individual cell - however, it could be modified towork on groups of cells, using a sequential approach within each cell group andthen the procces per cell aproach for communication between groups [4].

2.4 Applications

Modelling nature desn’t just help us understand the natural world, it can alsohelp develop new technology. For instance, modelling fish behaviour mightproduce techniques which could help build aquatic robots.

The most well-known example of cellular automata is probably John Con-way’s Game of Life. It is meant to be a representation of the cycle of life, withevery cell having two states: alive or dead. These states change at every timestep, according to the following rules [1]:

• Any live cell with less than two neighbours dies due to under-population.

• Any live cell with two or three live neighbours continues to live.

• Any live cell with more than three live neighbours dies, because of over-crowding.

• Any dead cell with three neighbours comes to life, because of reproduction.

Although this simulation doesnt seem to model anything in particular, itshows the power of emergent behaviour. Cellular automata can produce com-plex repeating patterns, which makes them an area of interest for multiple fields,particularly those interested in how life evolved from simple mollecular struc-tures [9] [10].

Because cellular automata work well on multiple CPUs, they are often con-sidered for use within encryption, which tends to be CPU intensive. Althoughunconventional, cellular automata have been successful in the field of cryptologyfor a while now. This stems from the fact that cellular automata can produce

5

Figure 1: Possible scenarios in The Game of Life.

complex patterns, which results in pseudo-randomness. There are several waysto use a cellular automaton for encryption systems, with advantages and disad-vantages. Example methods include the transform approach, the Wolfram ap-proach (discussed in section 2.4.1) and reversible cellular automaton approach(see section 2.4.2).

There are two types of encryption: symmetric and asymmetric. The dif-ference is that symmetric encription requires a secret key to be exchanged be-forehand, whereas asymmetric encryption does not. Cellular automata-basedencryption is symmetric, as few algorithms can be used for asymetric encrip-tion. The general algorithm of symmetric encryption is as follows:

Figure 2: Symmetric key encription scheme. [11]

6

2.4.1 Case Study: The Wolfram Approach

The Wolfram approach to encryption uses a 1D array to represent the messageand another 1D array to represent the encryption sequence. The encryptionsequence(b) and the plain text(a) would are then combined using XOR to pro-duce the cypher text(c). To get the plain text back, the cypher text and theencryption sequence simply passed combined using XOR again [11].

Figure 3: The Wolfram encryption system: a:The message; b:The encryptionkey; c:The encrypted message. [11]

The problem with having a predefined encryption sequence is that the se-quence needs to be transfered securely every time a message needs to be sent.Since the encryption sequence has to be as big as the message, the messageitself might be exchange just as easily instead. The way around this is to usea cellular automaton to generate an encryption sequence. This way, only rulesand parameters would need to be exchanged (and even those only once). To dothis securely, the automaton rules need to achieve a high level of randomnessand non-repeatability. Without these properties, the message could easily bedecrypted [11].

To come in the aid of this problem, a suggestion has been made to use cellularprograming, an evolutionary method of generating rules for cellular automata,with the fitness function to be the randomly generated [5]. This is a goodapproach, as cellular automata allow fast computation, meaning that evolutioncan take place quickly, with good results. The other advantage of this approachis that it does not necessarily need to be done by a computer - the simplicityof the rules means a person could encrypt text securely by hand. However,being an unusual encryption technique, this could make cryptologists hesitantto depend on it.

7

2.4.2 Case study:Reversible Cellular Automata

Instead of generating an encryption sequence in a cellular automaton, the re-versible method encodes the message in the cellular automaton itself, then ap-plies a certain number of time steps to encode it [6].

Figure 4: Encryption scheme using reversible cellular automata. [11]

To decode the message, the cypher text is inserted in the Cellular automaton,then the reverse of the original rules is applied. However, for this to be possible,the cellular automaton needs to be circular. If the cellular automaton is notcircular, the information would be lost, as information would fall off the bottom[6].

One desirable property of encryption is that a small change in the encryptionkey should make a large change to the cipher text. An important factor toachieve this is the number of cells used in the simulation, with more cells makingbigger changes. However, this is done at the cost of computation. [6] found 15cells to be optimal, with any more offering only small improvements at greatcosts to speed.

This method is flexible enough that it can be built into a box algorithm,widely used within used encryption systems such as AES. This shows that cel-lular automata can be used to implement other encryption algorithms. Becauseof the high speed of cellular automata, there is a good chance that in the futureit will be used for this.

2.5 Conclusions

Cellular automata can be used for a wide range of tasks and there are manyreasons they should be.They are simple to code, as well as being well suited formulti-CPU computers, allowing them to produce fast results. They also sharecommon features with other nature inspired design methods, such as collectionsof ages with local interactions.This, however, does not mean that cellular au-tomaata should be used for every occasion - other specialised models have moreflexibility and more options in their field.

8

3 Crowd Simulations

Crowd simulations focus on replicating certain aspects of behaviour patternsexhibited by large groups of individuals. They generally tend to represent eachmember of the group as a distinct individual, but some simulations only focuson the flow of the group as a whole [17].

3.1 General Design

A crowd simulation consists of a set of agents, placed in a virtual environmentand made to interact with it, eachother and possibly human users.

Normally, the agents possess some form of body (2D or 3D, depending ontheir given environment), capable of performing a given set of actions: move-ment, gestures, expressions, etc., with a corresponding set of animations. Com-binations of these low-level functions can then be combined, in order to performmore complicated, high-level ones.

The agents also possess a certain behaviour pattern, varied to a degree foreach individual, which can include emotions like fear or surprise or simulatetraits like bravery. This set of behaviours (normally forming a finite-state ma-chine) is used to determine what combination of actions is appropriate for agiven situation.

The physical representation and the behaviour tend to be implemented asseparate features, allowing, for example, a change of model (such as switchingto a lower resolution representation, in order to allow the use of more agents),while maintaining the same behaviour. Similarly, if necessary, simmulations canbe done with no graphical representation at all [17]. Figure 5 shows a possiblestructure for a crowd simulation program, as described by Magnenat-Thalmannet al. [17].

3.2 Potential Challenges and Limitations

Crowd simulations pose different challenges to, for example, single agent pro-grams. While a single agent might be convincing on its own, in order to make acrowd of agents convincing, there needs to be variety in the behaviour of eachagent. There are also resource issues: according to Magnenat-Thalmann et al.[17], while, in some cases (such as agent-environment interactions), the demandfor resources grows linearly with the number of agents, in many cases (suchas agent-agent interactions), the growth is quadratic. The challenge, therefore,consists in designing a system which allows for variety, while not exceeding thecomputational capacity of its designated platform.

The separation of the virtual representation from the underlying simulation(as described in section 3.1) helps with this issue. Due to the fact that thegraphics and the actual simulation do not rely on eachother, the quality ofone can be altered without affecting the other. Furthermore, by decreasing thequality of one, the quality and complexity of the other can be increased morethat would have otherwise been possible with similar resources.

9

Figure 5: A possible structure for a crowd simulation program.

3.3 Objectives

Using only real people(be they volunteers, actors, etc.), the cost of producinglarge-scale projects would be way beyond the budget of most companies, therisk of injuries would increase significantly, and precisely coordinating actions inorder to achieve a certain result would be near-impossible. Crowd simulationsare meant to come in aid of these problems. As such, an important objectivefor them is to drastically reduce the cost, duration and danger of reproducingcertain real-life scenarios [19].

In order to be effective, simulations should merely be a simplification ofreal-life scenarios, leaving out irrelevant information and keeping only what isessential to the task at hand. According to Bouvier et al. [19], propreties of agood simulation model are as follows:

• the simulation maintains the essential propreties of the original system;

• any simplifications made do not alter the accuracy of results;

• the parameters are easy to understand and control;

• the results produced by the simulation are reliable;

• the simulation can adapt to system modifications.

3.4 Tools, Methods and Variants

Based on the level of abstraction used, simulation models can be either phys-ically based or abstract. While abstract models can also be effective, physical

10

representations tend to be more similar to the original system, and hence easierto interpret. The advantage of the abstract model, on the other hand, is that itallows for a reasonably accurate result, while avoiding and overly complicatedsolution; it also permits the re-use of existing solutions for different problemswith minimum adaptation, reducing production costs significantly. Dependingon the context it is used in, a system can potentially qualify as both a physicalsystem or an abstract one: fluid dynamics can be used to represent both themovement of a liquid under certain conditions (physical model) and the flow ofcars in traffic (abstract model) [19] .

The method in which agent behaviour and movement is implemented canvary as well. The agents can make use of combinations of entirely pre-programmedbehaviours and motions, potentially with some parameter variation. This pro-vides the programmer with complete control of the agents’ behaviour, but canbe difficult to implement and poses the risk of not being sufficiently realistic.Evolutionary algorythms can also be used, allowing for a more realistic finalresult, with less effort from the programmer. The end result, however, will bemore difficult to predict, and not always the desired one.

Another available tool is footage of actual people. Traditionally used toperform motion capture, it can significantly reduce the cost of creating realisticagent animations - instead of having to implement each individual movementof the agent, the animator can use motions captured from a real person withconsiderably less effort, for more realistic gestures. A different approach, coveredin a study by Kang Hoon Lee et al. [20], involves using learning algorithms toteach agents realistic crowd movement from videos of crowds. To do this, KangHoon Lee et al. recorded crowds (primarily human, but footage of ants was alsoused) from an aerial view, extracted 2D trajectories for each person in the crowdand used these to learn the behaviour of the agents. The main advantage of thismethod is that it allows for a level of realism and complexity which can not beeasily achieved using more traditional methods, especially useful in situationswhich involve a large amount of agent interaction. However, computationalcosts can become an issue.

An effective crowd simmulation is likely to make use of at least a few ofthese techniques, using the strenghts of one to compensate for the weaknessedof another. Which techniques are used can vary widely, depending on what thegoals of the simulation are - section 3.5 describes a few different potential goalsand their requirements.

3.5 Applications

One area that makes extensive use of crowd simulation is the entertainmentindustry, with an increasing number films, games and commercials employingthe technology. Well-known titles, such as The Lord of The Rings, Avatar, KingKong, I, Robot or Chronicles of Narnia, all relied on crowd simulation programsto depict scenarios ranging from large, epic battles, to busy city streets [18].While a reasonable amount of accuracy is still required, more importantly, thesesimulations need to be capable of producing convincing, high quality imagery.

11

Figure 6: A queueing simulation.

As such, they normally employ high resolution 3D models, capable of performinga large array of gestures and sometimes (such as in the case of the film TheAnt Bully, an animation which used crowd simulation based programs for mostbackground character reactions) even detailed and varied facial expressions [18].

The technology is also used to help with real-world problems: architects andpsychologists, as well as policemen, firemen and the miltary make use of humancrowd simulations in order to determine what reactions large masses of peopleare likely to have in a given situation. This usually focuses on crowds in a stateof fear and panic, but other emotional can prove to be of interest as well. A fewuses for this type of simulation include identifying potential hazards in the eventof an emergency, designing buildings in a manner which facilitates crowd flow orcrowd behaviour and control training. As the main focus of these programs isto accurately reproduce real-life behaviour, the focus is more on the behaviourof the agents, and less on their looks: agents are often modelled as simplified,single colour, humanoid shapes (see figure 6).

On a smaller scale, crowd simulations can also be used to reproduce particlemovement: fluid dynamics or movement of air currents, for example, can bereproduced in this way.

3.5.1 Case Study: Massive Software

Massive is an award-winning crowd and autonomous agent simulation software,best known due to its use in a large number of well known films (the titles

12

Figure 7: A scene produced using the Massive software.

mentioned in section 3.5 all used Massive for crowd generation), television pro-ductions and commercials. However, adapted versions of the software are alsoused in education, engineering and robotics.

Due to being originally designed as commercial use software for the enter-tainment industry, Massive focuses heavily on aesthetics, adaptability and easeof use. The virtual models and the agent behaviour are implemented separately,allowing users to import their own animations and models.

Building an agent using this system involves four steps:

1. Building the motion tree. The motion tree is a graph consisting of motionsavailable to the agent. Note that at this point no actual functionality isimplemented for said motions: the array of motions available is simplydefined, to be linked with corresponding animations later.

2. Producing/choosing animations for each previously-defined motion. Ani-mations are made either by using motion capture or from scratch, usingkey framing. They can be either pre-defined (purchased as a separatepackage), or designed specifically for the production in question.

3. Building the “brain” of the agent - a set of low-level and high-level be-haviours linked to available motions.

Interaction with the world is then done using a combination of physicalsimulation (for gravity, momentum, etc.) and pre-defined motions (see above).

The software uses an Artificial Life approach, based on nature-inspired tech-niques: agents use auditory, visual and tactile cues, similar to the way livingbeings interact with their environment, increasing the realism of agent reactions.Another factor which contributes to the “natural” feel of the agents’ movementis the bottom-up design: complex, high-level reactions are all a result of simple,low-level ones.

Though primarily used in entertainment, Massive has a number of otherprojects in development. A real world simulator is currently in development,

13

with potential uses in architectural design, evacuation simulations and otherfields. A robotics project, currently still in its prototype phase, is also usingan adapted version of the software. “Zeno” is a robot that interracts withthe environment in a similar way to the above-mentioned simulation agents,navigating through the world, as well as gathering information and learningfrom it. Currently, the robot is capable of recognising and rememnbering voicesand faces, tracking people and making eye contact with them, displaying facialexpressions, as well as performing a with variety of motions (lying down, gettingup, gesturing, etc.) [18]. While not strictly speaking crowd-simulation related,the Zeno project shows how crowd simulation technology can be adapted foruse in innovative ways in other fields.

4 Particle Swarm Optimisation

Particle swarm optimisation is an iterative technique that attempts to optimisea function through a population of roaming agents known as ’particles’. As withmost optimisation methods, it can be applied to solve a vast range of problemsbut some areas of particular potential include simulation, system design, classi-fication problems, pattern recognition and modelling biological systems.

The model was inspired by Reynolds’ research on simulating flocking be-haviour, where agents are following 3 rules[35]:

• Separation - agents will move away from each other when they are tooclose

• Alignment - agents attempt to move in the average direction of their neigh-bours

• Cohesion - agents attempt to stay close to their neighbours

4.1 Basic Algorithm

In particle swarm optimisation techniques, agents communicate a “best posi-tion” which other agents will move towards. Agents will eventually convergearound the best solutions that have been located.

The basic algorithm firstly uniformly distributes the particles over the samplespace, after which it will:

1. Check against objective function

2. If better than the particle’s previous best, update it

3. Determine best particle from previous positions

4. Update particle velocities with update rule

5. Move particles to new positions

14

6. Repeat from step 1 until some halting criteria are met

[34]The update rule is perhaps the most important part of the algorithm.

[21]The first part of the rule defines the inertia of each particle, affecting the

“distance” they move at each time step. The second part can be seen as theparticle’s own influence on its velocity - the part of the calculation based on itsown “personal best” position. The third part of the rule is the “global” influence- based on the best position that any particle has found. [34]

4.2 Performance of Particle Swarm Optimisation

In general, there are several factors to consider when deciding whether to useparticle swarm optimisation. Much research has been conducted into the per-formance of metaheuristic algorithms, including their efficiency, ability to lo-cate better solutions, convergence and resource efficiency. Wolpert’s “No FreeLunch” theorems are of particular interest as they prove that for any optimi-sation algorithm, any good performance on a particular class of problems willbe offset by poor performance on another class of problems [32]. Multi-agentoptimisation algorithms certainly have an advantage in being extremely robust- they can be used to solve many different problems, and work well even deal-ing with “noisy” or missing data [33]. However, Wolpert’s research shows usthat optimisation algorithms must be chosen carefully based on the data theyare working on, and therefore that multi-agent optimisation will often not bethe best class of algorithms for a particular problem. At a very general level,PSO simulations are computationally expensive, and the complex interactionsinvolved also make it extremely difficult to choose the optimal algorithm andparameters for a given problem. Therefore even if PSO (or any multi-agentalgorithm) could provide an efficient method of optimisation, finding the op-timal parameters may prove extremely difficult. PSO may be seen as havinga small advantage over some other algorithms in having few parameters thatneed to be optimised, and also the relative ease with which it can be adaptedto parallel/distributed computing.

4.3 Variations

There are several variations of the original algorithm that can provide betteranswers to certain types of function. One of the most common variations giveseach particle a “neighbourhood” [34]. Each agent obtains the “best position”from all the agents within a certain distance of itself. This encourages morelocal exploration of the search space by each particle.

15

One idea is to add an “inertia weight” to the update function to affect thevelocity of the particles [30]. The inertia weight can be decreased over time,which allows for wider exploration of the search space at the start of the sim-ulation, and more local exploration towards the end. This method has becomeone of the most prevalent variations as it has provided significant performanceand accuracy increases in a number of applications. Exploring a larger area ofthe search space at the start provides a very general idea of where peaks andtroughs may lie, whereas increased local exploration of the optimal areas meansparticles are more likely to locate the optimal value.

Research by Clerc led to “canonical” particle swarm optimisation [31]. Canon-ical PSO adds a constraint to the entire update function, and typically it willincrease the speed of convergence. It works in a similar way to the inertia weight,encouraging convergence of particles by using real number between 0 and 1 as a“constriction factor” on the entire rule to reduce the velocity of particles. Overtime, this causes particles to gradually move together by reducing the amountof movement and heading change by each particle every time it updates.

Fully-informed PSO is another variation where each particle is attracted byevery other particle in its neighbourhood [29]. This removes the emphasis fromthe “best particle” in the neighbourhood, allowing a more general movementtowards better positions. This avoids situations where perhaps the second orthird-best neighbours have located better areas, but these are not exploreddue to a single better value in a less promising region overall. In the initialresearch proposing fully-informed PSO, it outperformed canonical PSO in everymeasure (performance, iterations to meet the halting criteria, and number oftrials reaching the halting criteria).

4.4 Applications

An interesting application of particle swarm optimisation was by an Americancorporation seeking to optimise the mixture of ingredients used to grow strainsof micro-orgranisms that are used to manufacture substances of interest [27].The resultant product mix proved more than twice as effective as the previous,more traditional optimisation methods used. The solution was also outsideof an area that would have been explored by other methods, illustrating howcounterintuitive solutions can be located by general optimisation algorithms.

Particle swarm algorithms have proved to be a highly effective solution tomany classes of optimisation problems, however other multi-agent optimisationalgorithms can be far better suited to other problems.

5 Stochastic Diffusion Search

Stochastic diffusion search is another multi-agent swarm intelligence search andoptimisation algorithm. The basic method involves a population of agents, eachof which is partially evaluating an objective function (looking for a characteristic

16

of their goal), and sharing hypotheses with each other when their evaluation issuccessful. Over time, the agents will begin to cluster around better solutions.

5.1 Basic algorithm

In the first stage of the algorithm, agents are initialised with their hypotheses.They can be initialised in various ways, but often their hypotheses are uniformlydistributed across the search space. In cases where more information aboutprobable solutions is available, the initialisation can include a “bias” to avoidimprobable solutions from being explored unnecessarily. [26]

Agents then evaluate their various hypotheses by selecting some features ofthe objective at random, after which they are divided into “active” or “inactive”agents. Active agents have successfully managed to locate the selected featuresof their objective in the search space. In the “diffusion phase”, inactive agentsthen randomly pick an agent to communicate with. If the selected agent isactive, the inactive agent copies the other’s hypothesis. If the selected agent isinactive, it will randomly pick a new hypothesis. This is a synchronous process- all agents are updated during the same time interval. [26]

By moving repeatedly through evaluating and “diffusing” stages, agents canexplore all possible solutions. Gradually, agents start to converge around thebest solutions as successful agents attract others to “good” areas of the searchspace. These iterations continue until certain halting criteria are met.

5.2 Example

DeMeyer provided an excellent example which may help illustrate how the al-gorithm works [26]. If a system’s objective is the locate the Queen of Heartsfrom five cards, five agents would be initialised with hypotheses from 1-5 (onefor each of the cards). Each agent would then randomly evaluate a part of theobjective function for their hypothesis - some might be looking for a Queen,others might be looking for a heart card. If the agents succeed in finding thosefeatures on their hypothesised card they will be set to active, whereas otherswill be set to inactive. The inactive agents will then randomly select anotheragent. If that agent is active, they will assume their hypothesis; otherwise, theywill randomly select another card as their hypothesis. Eventually, all agents willconverge onto the Queen of Hearts.

Halting criteria are an area that varies greatly. One system simply involveswaiting until a certain number of agents have converged onto a single hypothesisand remain with it for a specified period of time. As with other SDS parameters,an optimal time interval can be difficult to estimate. Once the halting criteriahave been met, the best solutions can then be identified as the hypotheses withthe largest clusters of agents surrounding them.

17

5.3 Variations

In addition to varying initialisation and halting criteria, many variations onthe algorithm itself have been proposed. Grech-Cini proposed altering the wayhypotheses are communicated by introducing agents that do not always discardtheir original hypothesis, or agents that do not always communicate a goodhypothesis to other inactive agents [25]. Another proposal was “focused SDS”,in which the general region of the optimal region is sought, and then subdividedfor further, more “focused” exploration [24]. Asynchronous models have alsobeen proposed.

5.4 Performance

SDS can be modelled mathematically to provide an accurate measure of com-plexity, resources required, robustness and other general behaviour of the algo-rithm (e.g. convergence behaviour) [26]. As with particle swarm optimisationand other multi-agent optimisation algorithms, SDS is expensive to computeand the optimal algorithm/parameters can be very hard to predict. They arealso likely to vary massively given different types of problem. It does carrya similar advantage however, in that it is highly robust and easy to adapt toparallel/distributed computing scenarios.

5.5 Applications

As an optimisation technique, the application areas of SDS are extremely var-ied. The first example of SDS by Bishop was for a text search problem [22].One of the first applications by Bishop and Torr was attempting to find theeyes in images of human faces as part of a hybrid system with a neural net-work to classify patterns in the image [23]. This hybrid model achieved somesuccess; a 60% success rate on unseen data, and all limitations were put downto the characteristics of the network, rather than the stochastic diffusion com-ponents. Grech-Cini argues that SDS is particularly useful in situations wherelittle information is available in the objective function to guide the search, andgave a method for showing if an objective function for a matching problem isparticularly suitable to be solved by SDS [25].

Nature-inspired algorithms are not always as abstract as the ones previouslydiscussed. Others take a far more obvious inspiration from nature in theirbehaviour, an excellent example being ant colony optimisation algorithms.

6 Ant Colony Optimisation

One of the earlier nature-inspired agent-based set of algorithms is what is nowknown as Ant Colony Optimisation, a class of metaheuristics which are - as thename suggests - based on the collective behaviour displayed by some speciesof ants in the wild. Ants are obviously incapable of high-level communication,yet they still manage the impressive feats of co-ordination required to construct

18

nests and, most relevantly for this algorithm, find the shortest path from theirnests to a food source to retrieve the food most efficiently. It was observedthat the way that ants manage to find these shortest paths could be modelledcomputationally and applied to discrete combinatorial optimisation problems,such as the Travelling Salesman Problem.

6.1 Ant Behaviour

Ants cannot communicate directly, but instead manage to convey information toeach other via stigmergy - communication mediated by the environment. In thecase of ants, they modify their environment by releasing pheromones that otherants can detect, and they are capable of effectively laying a trail of pheromoneto define a path. When ants forage for food, they initially explore randomly, butonce an ant has found food it returns to the nest, laying a trail of pheromoneas it does.[36]

This pheromone attracts other ants and those who encounter the trail arehighly likely to follow it (with probability proportional to the trail’s intensity).Ants walking to and from the food source release additional pheromones, makingother ants even more likely to follow the trail, in a positive feedback loop. In thisway the ants manage to co-ordinate and converge on a food source discoveredby a single ant.

The mechanism also encourages ants to find the shortest routes to thesesources - if there are multiple possible paths to such a source, discovered byvarious ants, any ants that choose the shortest path will complete the journeyin a shorter time and hence more frequently, resulting in increased reinforcementof that pheromone trail. In turn, other ants are now more likely to take theshorter route and the longer routes will become less frequently travelled and,as the pheromone dissipates, eventually become ignored in favour of the shorterpath.

An important consideration is that although the ants are highly likely tofollow any given trail, it is not guaranteed, and so these ants which randomlydecide to follow unexplored routes are effectively continuing to search for bettersolutions. They may even follow the trail to some extent and then diverge fromit, possibly finding a shortcut which improves the previously known path. Inthis way as time passes the ants will tend to find the optimal, or at least a verygood, path between the nest and the food.

6.2 Ant System

Ant behaviour is not precisely replicated by most Ant Colony Optimisationalgorithms but they are heavily inspired by the pheromone laying, trail-followingactivities of such ants. The first ACO system, called Ant System[39][37], wasdesigned as an algorithm to solve examples of the Travelling Salesman Problem(TSP). In the TSP, given a completely connected weighted graph of cities, theobjective is to find the shortest tour which visits every city exactly once.

19

Figure 8: Example of a simple Travelling Salesman Problem (weight of edgesrelated to Eucledian distance between the linked nodes).

In the Ant System’s ant-cycle algorithm, as applied to the TSP, a populationof ant agents construct tours of the graph’s nodes, subject to the constraints ofthe TSP. Every edge in the graph has a distance weighting, and a pheromonevalue, set to some initial value at the start of the algorithm’s run. The algorithmthen iterates as follows:

Figure 9: Example of weighting and pheromone values for edges at a node ofa TSP graph (weighting in black, pheromone value in red). Here the left pathwill be prioritised by the distance heuristic, but the central path prioritised bythe pheromone heuristic. The right path is least favourable to both heuristicsbut still has a chance of being selected by an ant at this node.

• Each ant constructs a solution by starting at a given node and traversingthe graph node by node, selecting an edge to follow at each node as follows:

1. Determine which edges are valid paths to follow by comparing theirdestination to the list of nodes it has already visited

20

2. For each valid edge, assign a probability of selection by consideringtwo heuristics: the greedy construction heuristic that prioritises lowdistance weighting, and the pheromone heuristic that prioritises highpheromone values

3. Randomly choose an edge according to the probabilities assignedEach ant chooses edges in turn, simultaneously constructing tours

• Once all ants have constructed solutions, update the pheromone values asfollows:

1. Reduce all pheromone values by some proportion to represent dissipa-tion of the pheromone (helps ensure the ants choose varied solutions)

2. For each ant, increase the pheromone values of the edges used in itstour by a value inversely proportional to the length of the overalltour

• Repeat until termination criteria are satisfied - the best tour found so faris the algorithm’s solution to the problem

In tests the Ant System proved to be very effective at finding good solutionsto hard examples of the TSP, finding the best-known or nearly best solutions incomparable time to other general-purpose algorithms like Simulated Annealing,although it performed somewhat worse than algorithms designed specifically tofind solutions to the TSP[39]. However, the Ant System was also designed to beeasily applied to any sort of discrete combinatorial optimisation problem, suchas scheduling problems. The ants do not necessarily require an actual graph towalk on, only a representation which informs them of valid choices when theyreach a decision point and an evaluation function which can assign a positivecost to the solution they generate. Later ants, if they find themselves at thesame decision point as ants from previous iterations, have the pheromone valuesto inform them of which choices the other ants have made. It is simply easiestto imagine this as a graph.

6.3 The ACO Metaheuristic and Variations

The fundamental mechanics of the Ant System serve as the foundation for mostnewer Ant Colony Optimisation methods. The Ant System is customisable tosome extent - the number of ants, the importance of the heuristics in pathselection, the initial distrubtion of ants among the nodes, and other parameterscan all be varied - but this foundation has been developed into a much moregeneral metaheuristic which can be applied to a wider variety of problems.

Given a problem which can be represented as a set of discrete decision vari-ables, each with a domain of possible values that can be assigned to it, a set ofconstraints that define which solutions are actually acceptable, and an evalua-tion function that assigns some positive cost value to a constructed solution, itis fairly simple to create an ACO system that will work on such a problem.

21

Every possible individual binding of a variable to a value is given an as-sociated pheromone value. When the system is first initialised this will besome preset value, but obviously the pheromone value for any given bindingwill change as the system iterates through generations of ants. The ants inthe system then construct solutions by iterating through the decision pointsand stochastically selecting values to bind to each variable - ideally informedby both the pheromone values and a greedy construction heuristic, but in caseswhere the latter is not available, the system can still find solutions using only thepheromone values. Depending on the precise nature of the constraints, solutionsthat violate these constraints may be discarded or prevented from being selectedin the first place, for hard constraints, or penalised in the fitness evaluation insoft cases.

Once the ants have all constructed their solutions, it is typical for imple-mentations to also use a local search algorithm on their solutions to see if itcan be improved[37]. Local search algorithms are often effective at making im-provements on a given solution, but there is a limit to how well they can do -they typically trap themselves in local minimums. By using a local search atthis stage the solutions found can be improved somewhat before the pheromoneupdate is applied. This is an optional step that is not necessary, but can helpthe system find good solutions faster.

As in the original Ant System, the pheromone values for each possible bind-ing are then updated according to some rules, but the precise rules and methodof doing so vary greatly from implementation to implementation, and are by farthe most variable aspect of ACO systems. Many researchers made extensionsto the original Ant System which were primarly focused on different ways ofdetermining how to update the pheromone values of the system[37].

One improvement proposed in the original Ant System was an ”elitist ants”variation, in which the best solution found received some amount of additionalreinforcement - effectively, whichever ant found the best solution was consideredto be several ants by the pheromone update. The number of so-called elitistants had to carefully balanced - too few and their impact was negligiable, buttoo many and the system rapdily converged on poor solutions as the exces-sive reinforcement of the initial solution discovered discouraged further randomexploration[39].

Rank-based Ant System introduces the concept of using the best solutionso far to update pheromone values (even if that solution was not constructedby ants in the current generation), as well as a somewhat elitist system whichonly allows some number of the best solutions found in this generation to updatepheromones (to a degree related to their ranking, based on solution quality)[37].

In some systems, such as the MAXMIN Ant System, only a single ant isallowed to update pheromones - either the best in the generation or the bestoverall solution so far found is reinforced. The MMAS also applies upper andlower bounds to pheromone values (hence the name), which ensures that nopossible solution becomes overly dominant because of repeated reinforcementor completely ignored because of disuse[36].

Ant Colony System reintroduces a concept that was discarded in the ini-

22

tial Ant System - local pheromone updates, where the ants change the valueof pheromones as they traverse the graph (or in the case of the more generalmetaheuristic, make bindings). The Ant System’s ant-density and ant-quantityalgorithms reinforced edges as they travelled, making ants in the same genera-tion that reached nodes over ants had already visited more likely to select thesame edges, similar to the behaviour of real ants. ACS, however, uses the localupdate the opposite way - when an ant traverses an edge (or makes a binding)it actually decreases the pheromone values in order to encourage other ants toproduce different solutions, resulting in a faster, more varied exploration of thesolution search space[37].

6.4 Applications

ACO can be applied to a wide variety of problems and systems. One of the mostcommonly used benchmark problems is the aforementioned Travelling SalesmanProblem, as it was the first problem used to test the original Ant System andis still an NP-hard problem which is prohibitively expensive to solve perfectly(hence the use of heuristic systems like Ant System and later variants to findapproximate solutions). Although the original Ant-System was outclassed byother, state-of-the-art algorithms designed specifically to solve the TSP, in thepresent day ACO based systems are amongst the best performing in severalproblem categories.[36]

6.4.1 Total Weighted Tardiness Problem

Scheduling problems are one such category, a specific example of which is theSingle Machine Total Weighted Tardiness Scheduling Problem. In such a prob-lem, a given machine must complete a set of tasks, each of which has an asso-ciated time cost, due time, and importance weighting. The machine can onlyperform one task at a time. The objective is to minimise the weighted total ofdelays (where a task has been completed after the due time).

We represent this problem for our metaheuristic by considering that thetasks must be performed in a given order, and so we have a number of decisionvariables equal to the number of tasks to be performed. For each, the domainof possible bindings is that of the tasks that must be solved, with the constraintthat no one task be bound twice (as that would represent repeating the task).The pheromone values associated with potential bindings thus indicate the pref-erence of scheduling a given task at that position in the queue. Given a valid setof bindings it is trivial to evaluate them to determine how late any given taskwould be and calculate the weighted tardiness of the whole set. There are alsolocal heuristics that can be used to guide solution construction; prioritising thetask due soonest, for example, or the task which would add the least tardinesswhen performed then (such heuristics are necessarily based on the previouslybound variables)[37].

Tests performed using such an ACO system on benchmark examples of thisproblem in [40] found that the system had excellent performance, always finding

23

the best solutions for the problems given and usually in a very short space oftime. It strongly outperformed most algorithms (such as Tabu search) and wasmore likely to find the optimal solutions than even the best opposing algorithms(such as iterated dynasearch), although it required slightly more computationaltime.

6.4.2 AntNet

Routing problems, such as the TSP, are another category where ACO systemsare considered very high performance, especially in complex examples such asthe asymmetric TSP (where weighting depends on direction of travel) and dy-namic systems where the cost of taking a given path is variable - for example,packet routing on Internet-like networks, where volume of traffic greatly variesthe cost of a path and links are not necessarily reliable.

When considering routing on the internet, there are many problems thatcan arise. The volume of traffic is variable and any given link may sometimesperform well, but sometimes be overly congested and perform poorly. Faultsare also commonplace, with individual links or entire nodes dropping out of thenetwork due to unforeseeable circumstance such as hardware failure. There aremany measures of performance it is desirable to optimise - overall latency, forexample (attempting to ensure the shortest possible delay in the arrival of apacket after being sent), or throughput (the actual amount of data successfullydelivered in a certain time period), and reliability (how well the system performsin the event of a fault). Constraints demand that all are optimised - users ofthe internet want high bandwidth, low latency, and their traffic to reach itsdestination reliably, especially if they have established a connection to someservice over the internet.[41]

In the AntNet system[41], the standard ACO system is altered somewhat toallow it to solve internet routing problems. Ant agents are tasked with findingroutes through the network from one node to another, calculating the cost oftheir route by considering the total delay between being sent and arriving atthe destination node. Each node maintains a routing table of pheromone values,with the values being associated with the specific combination of edge and finaldestination. When an ant has completed their path, they retrace their routeback through the network at a high priority, updating the routing tables of thenodes they used on the way, such that an ant that used a very poor path willdecrement the pheromone values of the route it took, while ants that found goodpaths increase them. The individual nodes then use these pheromone values toinform the routing of ordinary traffic. Nodes in the network frequently sendants to random destinations during normal operation in order to update therouting tables, discovering better routes if older ones have become poor due tochanging network conditions, such as faults in the network and changing load.

Under tests run in [41], the AntNet routing system was found to performexceptionally well, achieving superior results to commercial routing algorithmsunder a wide variety of potential network behaviours.

24

6.5 Advantages and Disadvantages

It is easy to see that ACO systems have a great many things to recommendthem. A well-designed ACO system, using good local heuristics and a well-performing local search algorithm, will frequently find optimal or best-knownsolutions to even extremely complex problems, as mentioned in the examplesabove, and it will find these solutions comparatively efficiently and reliably. Itachieves such good performance by directing its searches so they are mainlywithin the neighbourhood of good solutions, as determined by previous itera-tions of the algorithm, but ensuring that there is enough random deviation thatbetter solutions can be found and inform the search of later generations[37].

ACO-type systems also lend themselves very well to multicore processing, asthe individual ant agents can run in parallel while constructing their solutions(although this works less well in ACS, as the ants update the pheromones asthey construct their solutions, so order of execution becomes important andconcurrency issues can arise). They lend themselves less well to distributedcomputing because of the considerable communications overhead involved inthe pheromone update stage of the algorithm[36], although a performance im-provement could potentially still be observed if the problem necessitates a suf-ficiently huge population of ants (such that the processing time of constructingsolutions outweighs communications overhead), but in this case the ACS variantis especially unsuitable.

However, the specific representation type required for the ACO metaheuris-tic and even modified versions of it means this technique is certainly not suitablefor all problems. Such a system is not capable of solving continuous optimisationproblems, for example, a problem for which techniques such as Particle SwarmOptimisation are designed. The representation and operations of an ACO sys-tem would have to be considerably changed in order to apply it to a continuousproblem.

Even on the type of problems for which the ACO metaheuristic is suited,caution must be taken to ensure the system is configured properly to ensuredesirable behaviour. If it is not, the system can rapidly converge on suboptimalsolutions or search seemingly aimlessly without ever converging on a good solu-tion. The pheromone update method, weighting of heuristics in the stochasticselection, number of ants in a generation, and other parameters must all bechosen appropriately for the problem at hand, and good choices for these pa-rameters are not immediately obvious. Some problems exist where the systemwill fail to find the best solutions due to competition bias - good solution com-ponents end up ignored because they receive very little reinforcement, probablydue to very low priority from the greedy heuristic, whereas suboptimal solutioncomponents are repeatedly selected and reinforced.

As such, the metaheuristic presented is very general and non-specific, andsome work is required to make a practically useful system. As it is stands isalso only applicable to problems which can be represented in the specific wayrequired by the metaheuristic. The ACO system can be modified to enable itsapplication to more complicated problems, as seen in the AntNet example, but

25

this is nontrivial and requires some creative effort on the part of the designer.

7 Conclusion

Multi-agent systems is an extremely broad topic. We have attempted to covera variety of different areas, but we have not been able to cover many topics inthis report. Not only are there a great number of nature-inspired multi-agenttechniques, there are also a vast amount of variations on them, each adapted tosuit a particular class of problem. One point that has been particularly clearthroughout this research is the importance of selecting the correct algorithm fora particular problem, and adapting it appropriately.

Multi-agent systems is an extremely active field of research, with variationsupon models constantly being developed to further adapt them to new problems.Well-designed multi-agent systems provide good performance, and with ongoingresearch this performance is only likely to improve.

26

References

[1] J. Claycomb, J. Tran, Introductory biophysics:perspectives on the livingstate, Jones & Bartlett Learning, 2010.

[2] A. Garcıa Software engineering for large-scale multi-agent systems: re-search issues and practical applications, Springer-Verlag, Berlin, 2003.

[3] T. Hamagami, H. Hirata, Method of crowd simulation by using multiagenton cellular automata, IAT ’03 Proceedings of the IEEE/WIC InternationalConference on Intelligent Agent Technology, Washington, DC, USA: IEEEComputer Society, 2003.

[4] A.SAMPSON, P.WELCH, F. BARNES Lazy Cellular Automata with Com-municating Processes Communicating process architectures 2005:WoTUG-28 : proceedings of the 28th WoTUG Technical Meeting, 18-21 September2005, Technische Universiteit Eindhoven, The Netherlands.

[5] F. Seredynski, P Bouvry c, A. Zomaya Cellular automata computations andsecret key cryptography, Parallel Computing - Special issue: Parallel andnature-inspired computational paradigms and applications archive, Volume30, Issue 5-6, May 2004.

[6] S.MARCIN, B. PASCAL Block Cipher Based on Reversible Cellular Au-tomata, New generation computing, 2005, vol. 23, no3, pp. 245-258.

[7] D. WISHART, R. YANG , D. ARNDT, P. TANG, J. CRUZ Dynamiccellular automata : An alternative approach to cellular simulation In silicobiology,2005, vol. 5, no2, pp. 139-161.

[8] J. Hlinsley http://www.jhlinsley.net/wp-content/uploads/2008/11/life.png.

[9] R. Beer Autopoiesis and Cognition in the Game of Life Summer 2004, Vol.10, No. 3, Pages 309-326.

[10] P.bak, K. chen, M. creutz letters to nature october 1989,http://whiteboard.olin.edu/cm/handouts/bak89life.pdf.

[11] A. Badar an alterntaive cellular automata cryptogram studies in informa-tion and control, vol 11, no.4, december 2002.

[12] M. Owen, The Wireworld Computer, http://www.quinapalus.com/wi-index.html, 2010.

[13] S. Psakhie, Movable cellular automata method for simulating materials withmesostructure, Theoretical and Applied Fracture Mechanics, 2001.

[14] P. Rendell, Collision-based computing, Springer-Verlag, London 2001.

[15] P. Sarkar, A brief history of cellular automata, ACM Computing Surveys,2000.

27

[16] Y. Shi, H. Liu, L.Gao, G. Zhang, Cellular particle swarm optimization,ScienceDirect, www.sciencedirect.com, doi:10.1016/j.ins.2010.05.025, 2010.

[17] N. Magnenat-Thalmann, D. Thalmann, Computer Animation and Simula-tion, Springer-Verlag, Wien, 2001.

[18] , Weta Digital, Massive Software Website, http://massivesoftware.com/.

[19] E. Bouvier, E. Cohen, L. Najman, From crowd simulation to airbag deploy-ment: particle systems, a new paradigm of simulation, Journal of ElectronicImaging, Vol. 6, Issue 1, 1997.

[20] Kang Hoon Lee, Myung Geol Choi, Qyoun Hong and Jejee Lee, Group Be-haviour from Video: A Data-Driven Approach to Crowd Simulation, Euro-graphics/ ACM SIGGRAPH Symposium on Computer Animation, 2007.

[21] J. F. Schutte, The Particle Swarm Algorithm,http://www.mae.ufl.edu/haftka/stropt/Lectures/PSO introduction.pdf,2005.

[22] J. Bishop Stochastic Searching Networks, Artificial Neural Networks, 1989.

[23] J. Bishop, P. Torr, The Stochastic Search Network, Neural Networks forImages, Speech and Natural Language, 1992.

[24] P. Beattie, The Design and Implementation of a Focused Stochastic Dif-fusion Network to Solve the Self-Localisation Problem on an AutonomousWheelchair, PhD Thesis, University of Reading, 2000.

[25] H. Grech-Cini, Locating Facial Features, PhD Thesis, University of Read-ing, 1995.

[26] K. De Meyer, Foundations of Stochastic Diffusion Search, PhD Thesis,Department of Cybernetics, University of Reading, 2004.

[27] R. Eberhart, Y. Shi, Particle Swarm Optimization: Developments, Appli-cations and Resources, Proceedings of the 2001 Congress on EvolutionaryComputation, 2001.

[28] R. Mendes, J. Kennedy, J. Neves, The Fully Informed Particle Swarm:Simpler, Maybe Better, IEEE Transactions On Evolutionary Computation,Volume 8, Issue 3, 2004.

[29] R. Mendes, J. Kennedy, J. Neves, The Fully Informed Particle Swarm:Simpler, Maybe Better, IEEE Transactions On Evolutionary Computation,Volume 8, Issue 3, 2004.

[30] Y. Shi, R. Eberhart, A modified particle swarm optimizer, EvolutionaryComputation Proceedings, 1998.

28

[31] M. Clerc, J. Kennedy, The particle swarm explosion, stability, and conver-gence in a multidimensional complex space, IEEE Transactions on Evolu-tionary Computation, Issue 6.

[32] D. H. Wolpert, W. G. Macready, No Free Lunch Theorems for Optimiza-tion, IEEE Transactions on Evolutionary Computation, Volume 1, Issue 1,1997.

[33] J. Kennedy, R. C. Eberhart, Particle Swarm Optimization, Academic Press,San Diego CA, 2001.

[34] M. A. Montes de Oca, Particle Swarm Optimization, IRIDIA-CoDE,http://iridia.ulb.ac.be/ mmontes/slidesCIL/slides.pdf, 2007.

[35] C. W. Reynolds, Flocks, herds, and schools: A distributed behavioral model,ACM Computer Graphics, Volume 21, Issue 4, 1987.

[36] M. Dorigo, M. Birattari, T. Stutzle, Ant Colony Optimisation: ArtificialAnts as a Computational Intelligence Technique, IEEE Computational In-telligence Magazine, Vol. 1, Issue 4, 2006.

[37] M. Dorigo, T. Stutzle, Ant Colony Optimisation: Overview and RecentAdvances, International Series in Operations Research & Management Sci-ence, Vol. 146, 2010.

[38] M. Dorigo, C. Blum, Ant colony optimization theory: A survey, TheoreticalComputer Science, Vol. 344, Issues 2-3, 2005.

[39] M. Dorigo, V. Maniezzo, A. Colorni, Ant System: Optimization by a Colonyof Cooperating Agents, IEEE Transactions on Systems, Man, and Cyber-neticsPart B, Vol. 26, No. 1, 1996.

[40] M. den Besten, T. Stutzle, M. Dorigo, Ant Colony Optimisation for theTotal Weighted Tardiness Problem, Ant Colony Optimisation for the TotalWeighted Tardiness Problem, Lecture Notes in Computer Science, Vol.1917, 2000.

[41] G. di Caro, M. Dorigo, AntNet: Distributed Stigmergetic Control for Com-munications Networks, Journal of Artificial Intelligence Research, 9, 1998.

29