Hybrid Control for Large Swarms of Aquatic Drones

Miguel Duarte, Sancho Moura Oliveira and Anders Lyhne Christensen

Instituto de Telecomunicacoes &Instituto Universitario de Lisboa (ISCTE-IUL), Lisboa, Portugal

{miguel duarte,sancho.oliveira,anders.christensen}@iscte.pt

Abstract

Maritime tasks, such as surveillance and patrolling, aqua-culture inspection, and wildlife monitoring, typically requirelarge operational crews and expensive equipment. Only re-cently have unmanned vehicles started to be used for suchmissions. These vehicles, however, tend to be expensive andhave limited coverage, which prevents large-scale deploy-ment. In this paper, we propose a scalable robotics systembased on swarms of small and inexpensive aquatic drones.We take advantage of bio-inspired artificial evolution tech-niques in order to synthesize scalable and robust collectivebehaviors for the drones. The behaviors are then combinedhierarchically with preprogrammed control in an engineered-centric approach, allowing the overall behavior for a particu-lar mission to be quickly configured and tested in simulationbefore the aquatic drones are deployed. We demonstrate thescalability of our hybrid approach by successfully deployingup to 1,000 simulated drones to patrol a 20 km long strip for24 hours.

IntroductionCoastal countries have faced an increased spending over theyears in order to carry out maritime tasks. In Italy, for in-stance, the problem of illegal immigration (Monzini, 2007;Carling, 2007) and organized crime (Lutterbeck, 2006) hascontributed for the growth of the Guardia di Finanza’s bud-get from $1.10B to $3.21B, during the 1990s, with an in-crease both in personnel (28%) and equipment, which, in1999, counted 582 boats (78% increase), 96 helicopters(41% increase), and a total of 14 airplanes (Lutterbeck,2004). In Spain, immigrants crossing the Gibraltar Straitthrough Morocco have lead the Spanish government toimplement the Sistema Integrado de Vigilancia Exterior(SIVE), which is composed of military technology such asfixed and mobile radars, infrared sensors, and traditionalaquatic and aerial vehicles (Lutterbeck, 2006).

Since current state-of-the-art systems require a large hu-man crew in order to successfully execute maritime tasks, ef-forts have been made to adapt unmanned vehicle technology,such as unmanned aerial vehicles (UAV) and unmanned sur-face vehicles (USV) for these tasks. However, current UAV

and USV systems tend to be expensive to acquire and oper-ate, and therefore tend to be comprised of only a small num-ber of units (Schwing, 2007). Examples are the military-grade General Atomics MQ-9 Reaper UAV, and the Protec-tor USV (Yan et al., 2010). Only recently have technologicaladvances made it possible (and affordable) for researchers tobegin to study the use of medium or large-sized swarms ofinexpensive and unmanned autonomous vehicles (Manley,2008).

In this paper, we propose a system composed of aswarm of relatively simple aquatic drones with decentral-ized control for maritime missions. Our proposed systemis composed of potentially hundreds or thousands of smallautonomous aquatic drones. Such decentralized systemspresent numerous potential benefits compared with mono-lithic robotic systems (Bayindir and Sahin, 2007). On theone hand, the use of multiple drones introduces redundancyin the system, which reduces the impact of hardware failuresand has the potential to improve operational efficiency byallowing a larger area to be covered simultaneously. On theother hand, automating such missions considerably reducesthe maintenance cost (smaller human crew) while increasingavailability and scalability.

We employ our novel hybrid control approach (Duarteet al., 2014b) that combines evolutionary robotics (ER)techniques and preprogrammed logic to facilitate config-urable, decentralized control. We evolve several simple self-organized behaviors, which would be challenging to manu-ally program. The use of ER techniques enables the syn-thesis of robust and scalable controllers for simple tasks,which are then combined using high-level preprogrammedlogic for the complete task. In this way, our approach ad-dresses fundamental challenges in the synthesis and use ofself-organized control for real-world swarm systems. By di-viding control into individual components, (i) it becomescomputationally feasible to evolve self-organized control forlarge-scale systems, (ii) we diminish bootstrapping issuessince solutions to a set of simpler tasks are sought instead ofsolutions to the more complex, global task, and (iii) controlfor new missions can be composed based on existing behav-

ALIFE 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems

ioral components.Our study is novel in three respects: (i) we apply evolu-

tionary techniques to the domain of large swarms of aquaticdrones, a combination which to the best of our knowledge isunexplored, (ii) we demonstrate our hybrid control approachapplied to a multirobot system with up to 1,000 drones,where it has previously been applied only to single-robotsystems (Duarte et al., 2014b) and to a small team of threerobots (Duarte et al., 2014a), and (iii) we show that combin-ing evolved and preprogrammed control allows system de-signers to automatically synthesize self-organized behaviorsand then compose control for large-scale maritime missionsbased on these behaviors.

BackgroundMultirobot systems are well suited to tasks that require dis-tributed sensing and/or action. Furthermore, the degree ofrobustness and reliability of the system is potentially highgiven the inherent redundancy at the unit-level, and hard-ware failures often have only limited impact on performanceof the system as a whole (Farinelli et al., 2004). Manual de-sign of decentralized control for large-scale multirobot sys-tems has, however, proven difficult, since a set of micro-scopic behavioral rules that give rise to the desired macro-scopic, self-organized behavior cannot be derived in the gen-eral case (Dorigo et al., 2004).

ER has been used as an alternative to manual program-ming of decentralized robotic control, since it allows for theautomatic synthesis of control and for the self-organizationof behavior. An evolutionary algorithm (Goldberg, 1989)optimizes a controller, which is copied to every robot inthe system. The algorithm fine-tunes the microscopic rulesguiding the individual robots based on the resulting macro-scopic behavior, and thus removes the need for manual spec-ification of low-level control (Floreano and Keller, 2010).Artificial neural networks (ANN) are widely (but not exclu-sively) used in ER as robotic controllers. ANNs provideevolution with a relatively smooth search space (Nolfi andFloreano, 2000), are able to represent general and adaptivesolutions (Floreano and Mondada, 1994), and are able to tol-erate noise, which is inherent to many real-world sensorsand actuators (Kam-Chuen et al., 1996).

In the domain of multirobot systems, evolutionary tech-niques have been used in a variety of contexts, fromrobotic soccer (Uchibe and Asada, 2006), to collective trans-port (Groß and Dorigo, 2009), and to establish and maintaindata links with swarms of flying robots (Hauert et al., 2009).

Notable examples of projects in the domain of swarms ofaquatic robots are CoCoRo (Schmickl et al., 2011), whichfocuses on synthesis of control for underwater robots, andASSISIbf (Halloy et al., 2013), which aims to develop com-munication channels between robots and animals (fish andhoneybees, in particular). While these projects rely on bio-inspired control systems, they are only tangentially related

with evolved ANN-based control, such as the one presentedin this paper. Although the use of ER in aquatic robots hasnot yet been widely explored, a few studies have approachedthe subject. Some examples include the evolution of loco-motion behaviors (Ijspeert et al., 2007; Meyer et al., 2003),station keeping for an underwater robot (Moore et al., 2013),and a neuroaugmenting approach to the evolution of central-ized control for a small team of UAVs (Praczyk, 2014).

Over the years, researchers have identified certain chal-lenges associated with the application of ER. One of themost prevalent challenges concerns bootstrapping the evo-lutionary process in complex tasks. If controllers for a rela-tively complex task are sought, evolution may be unable tofind a fitness gradient that leads to adequate solutions (Nel-son et al., 2009). Another challenge is the use of evolvedcontrol in real robotic hardware. Except for a few cases inwhich evolution is conducted onboard real hardware (see,for instance (Watson et al., 1999)), the evolution of roboticcontrollers is conducted offline, in simulation, due to thelarge number of evaluation necessary to obtain capable con-trollers. Simulation-specific features, which are not presentin the real world, may be exploited by evolution. As a con-sequence, the process of transferring evolved controllers toreal robotic hardware, known as crossing the reality gap,typically fails to preserve the level of performance achievedin simulation (Jakobi, 1997).

Unless very simple tasks are considered, it is difficult toforesee which evolutionary setup might be suitable for solv-ing a particular task (Christensen and Dorigo, 2006). Be-tween the controller, fitness function and evolutionary algo-rithm, many different combinations of settings are possible.It then becomes necessary to run the computationally inten-sive evolutionary process, often multiple times with differ-ent initial conditions, to assess if a particular setup producesuseful solutions. This leads to a time-consuming trial-and-error process. While a few studies have been conducted inwhich evolution was applied in a more engineered-orientedmanner, such attempts have, so far, been ad-hoc (see Großet al. (2006) for an example). Techniques such as incre-mental evolution (Mouret and Doncieux, 2008) and task-decomposition (Lee, 1999; Whiteson et al., 2005) have beenproposed, but such approaches do not address the semi-automatic synthesis of behavior in a systematic way.

Our hybrid approach to the synthesis of behavioral controladdresses these issues by applying systematic sub-divisionof complex tasks into simpler sub-tasks. By evolving be-haviors for simpler sub-tasks, the main challenges of ERcan be overcome. On the one hand, if a task is too com-plex for solutions to be evolved, the sub-division still allowsus to apply ER techniques to sub-tasks where manually pro-gramming solutions is difficult. On the other hand, the sub-division enables the combined use of preprogrammed andevolved control in an engineering-centric approach (Silvaet al., 2014), giving more control over the final solutions

ALIFE 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems

to human designers than with traditional evolutionary tech-niques. Preprogrammed control can also be used when cer-tain robot-environment interactions are too difficult to accu-rately model in simulation, as in the case of tasks that requirefine sensorimotor coordination (Duarte et al., 2014b), whichcan prevent controllers from successfully crossing the realitygap. Moreover, manually programmed behaviors can easilybe scaled to long-term operation, while evolved controllersare not typically evolved for long-duration tasks carried outin large physical spaces, due to the computational cost asso-ciated with evolution of controllers in such conditions.

Proposed SolutionIn this section, we describe the hardware model and soft-ware platform necessary to realize the robotics system pro-posed in this paper. The conceptual aquatic drones used inour system are relatively small (length: 1 m), inexpensive(< $1000) and equipped with a number of sensors, suchas a camera, GPS, and short-range communication equip-ment. The aquatic drones are able to move at speeds upto 10 km/h and are equipped with batteries that operate be-tween two and ten hours depending on motor usage. De-velopment and testing of a hardware platform that meetsthese requirements is being carried out as part of our on-going work. The local communication system will add thecapability to exchange data between drones, such as theirGPS location and mission-related information. In this way,communication enables the drones to adjust their behaviorbased on the actions of other drones, and to coordinate theircollective behavior. The capacity to sense nearby robots al-lows, for instance, drones to maintain coverage of a patrolzone, even when some of the drones experience hardwarefailure or leave to execute other tasks.

In our approach, behavior primitives are evolved for par-ticular tasks that the drones need to execute during a mis-sion, such as navigation, patrolling, and intruder detection.In this way, a collection of evolved behaviors is built. Theindividual controllers are then combined hierarchically us-ing behavior arbitrators, which are decision nodes that del-egate control to their sub-controllers (Duarte et al., 2014b).The controllers can have multiple hierarchical layers of bothevolved and preprogrammed nodes, allowing for detailedcontrol over behavior. By combining evolved behaviors withtop-level preprogrammed control, we rely on evolution tofind behaviors based on self-organization, while letting thesystem designer define the conditions that trigger the execu-tion of the different behaviors.

By using such an engineering-centric approach, behaviorscan be added or removed based on the mission’s characteris-tics without the need for additional computational-intensiveprocesses: if a particular mission does not require, for in-stance, intruder detection, that behavior can be easily re-moved from the top-level preprogrammed control, and theexisting manual rules can be modified and tested in various

simulated scenarios before real drones are deployed.

Simulation and Experimental SetupWe use JBotEvolver (Duarte et al., 2014c), an open-source,multirobot simulation platform, and neuroevolution frame-work, for evolution of behavioral control. Each neural-network-based controller in our experiments was evolvedusing a simple generational evolutionary algorithm with pa-rameter values that have led to the synthesis of good solu-tions in previous studies (Duarte et al., 2014b). We use apopulation of 100 genomes, and the fitness score assigned toeach genome is the mean score obtained in 10 simulationswith different initial conditions. The five highest scoringgenomes are copied directly to the next generation. Another19 copies of each genome are made and mutation is appliedto each gene with a probability of 10%. A Gaussian offsetwith a mean of 0 and a standard deviation of 1 is appliedwhen a gene undergoes mutation. Ten evolutionary runswere conducted for each behavior, and each run lasted 100generations. Genomes consist of floating-point alleles thatencode the parameters of a continuous-time recurrent neu-ral network with one hidden layer of fully-connected neu-rons (Beer and Gallagher, 1992). All controllers evolvedfor the experiments presented in this paper have five hiddenneurons.

Each drone was equipped with a number of different sen-sors: (i) four drone sensors with a range of 200 m, (ii) a way-point sensor, which indicates the orientation and distance ofa selected GPS waypoint, (iii) four patrol boundary sensorswith a range of 50 m, (iv) a Boolean boundary sensor whichindicates whether the drone is inside the patrol zone or not,and (v) an intruder sensor, which detects intruders up to arange of 100 m in front of the drone using the camera. Allsensors have an opening angle of 90◦, and each sensor feedsthe ANN with a value in the interval [0, 1], depending on thedistance of the closest object inside its field-of-view. Sincethe camera does not collect depth data, it feeds the ANNwith (i) an input that indicates whether an intruder is beingdetected or not, and (ii) an input that indicates the relativedirection to the intruder, in case an intruder is detected.

The drone sensors are implemented by combining localcommunication with GPS position, while the boundary sen-sors are implemented based on GPS position and a set ofGPS coordinates. The four sensors of each type providedrones with omnidirectional drone sensing and omnidirec-tional boundary sensing up to the sensors’ respective ranges.The ANN controller’s output layer has three output neurons:two outputs control the speed of both motors, while a thirdstop motors output turns off the motors if its activation isover a threshold of 0.5.

Experiments and ResultsTo test the proposed approach, we chose a maritime pa-trolling and intruder detection task where a swarm of aquatic

ALIFE 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems

Figure 1: Map of the island of Lampedusa in the Mediterranean Sea, with a 20 km by 0.5 km patrol zone. Drones are deployedfrom two base stations to a random location inside the patrol zone.

Top Level

Behavior Arbitrator

Behavior Primitive

Patrol

Behavior Primitive Behavior Primitive

Go To Waypoint Pursue Intruder

Figure 2: Representation of the hierarchical controller, withone preprogrammed behavior arbitrator and three evolvedbehavior primitives.

drones must remain within a previously designated patrolzone and pursue intruders that try to cross it. The patrol zoneis defined by a polygon, in which the vertices are GPS co-ordinates. Drones are configured with the mission-specificGPS coordinates prior to deployment. The drones are ini-tially located on one of two base stations, to which theymust eventually return in order to recharge their batteries.For the complete mission, we used a patrol zone with a sizeof 20,000 m by 500 m. Such a patrol zone would allow thecoverage of the south coast of the Italian island of Lampe-dusa (see Figure 1), a major hub for illegal migration fromTunisia and Libya to Italy (Coppens, 2013).

For the complete task, we use three evolved behaviorprimitives: “Go To Waypoint”, “Patrol”, and “Pursue In-truder”. The behavior primitives were then combined in amission controller using a top-level preprogrammed behav-ior arbitrator (see Figure 2). The evolution of the behaviorprimitives is discussed below.

Evolved behaviorsGo to waypoint: In many maritime missions, it is neces-sary for drones to move to a specific location, indicated by

a GPS waypoint, without colliding with obstacles or nearbydrones. The waypoints can be configured before the droneis deployed, or they can be discovered autonomously duringa mission.

Controllers for the “Go To Waypoint” behavior wereevolved using a single drone in an environment with anadditional 50 static drones that posed as obstacles. Thestatic drones were randomly placed between the starting po-sition and the destination waypoint in a band with a widthof 50 m, and the evolving drone had to reach a waypointwithin 10,000 control cycles (equivalent to 1,000 seconds).The waypoint was placed from 500 m to 1,000 m from thedrone’s starting position at a random orientation, with re-spect to the drone. The evaluation function rewarded con-trollers for (i) getting close to the waypoint, and (ii) stoppingwithin 3 m of the waypoint. The evaluation function also pe-nalized controllers for colliding with the static drones:

f = −10−2 · P +

D − dD

far from waypoint

2− C − sC

near waypoint,

where P is the number of control cycles that the drone col-lided with static drones, D is the initial distance from thedrone to the waypoint, d is the current distance from thedrone to the waypoint, C is the total number of control cy-cles, and s is the number of control cycles where the drone isstopped within 3 m of the waypoint. Seven of the ten evolu-tionary runs conducted for the “Go To Waypoint” behaviorevolved solutions that shut down the motors after reachingthe waypoint. In all seven successful solutions, the dronemoves at full speed toward the waypoint while evading staticdrones. After arriving at the waypoint, the drone then shutsdown the motors using the stop motors output of the ANN.

ALIFE 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems

t = 20 s t = 100 s t = 500 s

Figure 3: Evolved “Patrol” behavior with 50 drones in a patrol zone with a size of 300 m × 800 m at different times. In thefigure, the size of the drones has been increased relative to the size of the patrol zone for clarity.

Patrol: Once inside the patrol zone, the drones should dis-perse to cover the zone evenly while remaining inside itspre-defined boundaries. Controllers were evolved for the“Patrol” behavior by placing 50 drones near the center of arectangular zone with sizes varying from 250 m to 1,500 mfrom sample to sample. Controllers were evolved for 5,000control cycles (equivalent to 500 seconds), and each dronehad an individual copy of the evaluated ANN controller. Thecontrollers were evaluated at each control cycle based ontheir distance to other drones and patrol zone boundaries,and on the energy spent:

gi =

R∑r=1

23 · (1−max(αir)) +

13 · (1− pir)

R,

where αir is the set of activations of drone r’s boundary sen-sors and drone sensors at control cycle i, pir is the percent-age of power applied to the motors, and R is the number ofdrones. If any of the drones is outside the patrol zone at con-trol cycle i, the value of gi is set to 0. The fitness obtained ina sample was the mean of gi during the sample. The evolvedcontrollers move outward in a circle until they stop sensingnearby drones. When the patrol zone is small and drones areable to reach its boundaries before the sample terminates,they stop moving outward and instead optimize the spacingbetween one another. An example of this behavior can beseen in Figure 3.

Pursue intruder: When an intruder is detected by adrone’s camera, the drone should attempt to follow it with-out crossing the boundaries of the patrol zone. Since thedrones’ cameras are unable to collect depth data, a minimumof two drones should pursue an intruder in order to triangu-late its position. For the evolution of the “Pursue Intruder”

behavior, an intruder attempted to cross a patrol zone. Thesize of the patrol zone varied in each sample from 250 mto 1,000 m. The 50 drones, each with a copy of the con-troller being evaluated, started evenly dispersed inside thezone. The intruder moved at a fixed speed of 10 km/h. Thecontrollers were evaluated according to the number of con-trol cycles in which the intruder was detected. Furthermore,gi was reused in order to reward dispersion of the drones,optimize power usage, and to obtain solutions in which thedrones remain inside the patrol zone:

hi = gi +min(Di, 2),

where Di is the number of drones detecting an intruder incontrol cycle i. The fitness obtained in a sample was themean of hi during the sample. The evolved controllers pur-sue the intruder as soon as it is detected, and remain staticotherwise. If several drones are pursuing the intruder, someof them cease the pursuit, typically leaving only two. Assoon as the intruder exits the patrol zone, the drones stop thepursuit and adjust their positions with respect to neighboringdrones.

Preprogrammed arbitratorAfter the behavior primitives had been evolved, we com-bined them using a simple state-based preprogrammed arbi-trator (see Figure 4). The arbitrator determines which behav-ior should be active at any given time, depending on the cur-rent state and the drone’s sensory inputs. When an intruderis detected, the drone alerts other drones within a 200 m ra-dius. The alerted drones automatically activate the “PursueIntruder” behavior. If a drone is in the “Pursue Intruder” be-havior and has not seen an intruder for the past five minutes,it reverts to the “Patrol” behavior.

The amount of energy necessary to return to the base sta-tion is estimated based on the drone’s distance to the base

ALIFE 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems

PatrolGo To

RechargePursue

Intruder

batterycharged

within 10 metersof waypoint

intruderdetected/alert

no intruderfor 5 min

batterylow

batterylow

Waypoint

Figure 4: A state machine representing the preprogrammedmission controller.

station. If the battery reaches the estimated limit, the dronereturns to the base station in order to recharge its battery.

Scalability testsWe conducted scalability tests with the final controller ina large-scale patrolling and intruder detection task. In thebeginning of the experiment, a drone is deployed from eachof the two base stations to a random position inside the patrolzone every ten seconds. Each base station deploys drones toone half of the patrol zone. The drones have a battery life often hours of stand-by use, or two hours of full throttle. Whenback at the base station, the drones recharge their batteriesfor a period of 30 minutes. Two hours into the experiments,intruders begin to cross the patrol zone at random locationsevery 30 minutes (a total of 44 crossings). The simulation isrun for a total of 24 simulated hours.

We ran experiments in ten different scenarios with a num-ber of drones varying from 100 to 1,000 at increments of 100drones. The simulation was run in a patrol zone with a sizeof 20,000 m by 500 m, as shown in Figure 1.

The performance observed for the different swarm sizescan be seen in Figure 5. By increasing the number of drones,the number of detected intruders increases until it reaches amaximum of 100% detection rate with 500 drones, whichcorresponds to a density of 50 drones/km2. Another indica-tor of performance is the amount of time that the intrudersare being pursued by either (i) one drone, or (ii) two or moredrones. Ideally, an intruder should be pursued by at leasttwo drones in order for its position to be correctly triangu-lated. Although the detection rate peaked at a swarm size of500 drones, we can observe a slight improvement in the pro-portion of time that two or more drones pursue an intruderas we deploy more drones. The mean amount of time untilan intruder has been detected after entering the patrol zonedecreases as more drones are deployed. With 100 drones the

0

0.2

0.4

0.6

0.8

1

100 200 300 400 500 600 700 800 900 1000

Pro

port

ion o

f ti

me

Number of drones in swarm

One drone pursuingTwo or more drones pursuing

Intruders detected

Figure 5: The figure shows both the percentage of detectedintruders as points, and the percentage of time that the in-truders were pursued by (i) one drone, and (ii) two or moredrones, as histograms for the different simulated scenarios.

mean amount of time was 62 s, with 500 drones the meanamount of time was 19 s, and with 1,000 drones the averagetime was 16 s.

In the scenario with 500 drones, the hybrid controllerssuccessfully completed the task by detecting 44 out of the44 intruders. In total, intruders were present in the patrolzone for 132 minutes, of which they were detected by a sin-gle drone for 28 minutes, and by two or more drones for92 minutes (21% and 70% of the total time, respectively).In the final ten hours, the system reached an equilibrium inwhich a mean of 79% of the drones were patrolling, 14%were returning to the base or recharging, 6% were goingfrom the base to the patrol zone, and 1% were pursuing anintruder (see Figure 6). We conducted the same analysis forthe setup with 1,000 drones and observed similar results.

Conclusions and Future WorkIn this paper, we applied a hybrid approach, based ona combination of evolved and preprogrammed control, tothe synthesis of controllers for large swarms of aquaticdrones. Controllers for three different behavior primitiveswere evolved: “Go To Waypoint”, “Patrol”, and “Pursue In-truder”. The controllers were then combined using a state-based preprogrammed behavior arbitrator in order to executea patrolling and intruder detection task.

The use of ER techniques for the synthesis of self-organized control is particularly relevant for large-scale mul-tirobot systems based on decentralized control, since it isoften challenging to manually design behavioral rules forthe individual robots. Evolutionary techniques, however, aredifficult to apply in complex tasks, provide little control overthe solutions evolved, and require a considerable amount oftime to evolve controllers for large robotic swarms. By di-viding the task into simpler sub-tasks, it is possible to evolverobust and scalable behaviors that solve different parts of the

ALIFE 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems

0

100

200

300

400

500

0 5 10 15 20

Num

ber o

f dro

nes

Time (hours)

P

RGI

Patrol (P)Recharge (R)

Go To Waypoint (G)Pursue Intruder (I)

Figure 6: Plot of the states of the drones’ preprogrammedbehavior arbitrators in the scenario with 500 deployeddrones over a period of 24 hours of simulation.

task. The controllers evolved for the sub-tasks can then becombined using a preprogrammed behavior arbitrator thatcontrols when each behavior primitive should be executed.Furthermore, evolved behavior primitives can potentially bereused in various types of missions.

In our ongoing work, we are exploring ways in whichinter-drone communication can be further exploited in mar-itime scenarios. Moreover, we are preparing our hardwaretestbed for proof-of-concept missions based on the approachproposed in this paper. Our long-term goal is to have a soft-ware and hardware platform for swarm-based maritime mis-sions.

AcknowledgementsThis work was partly supported by FCT – Foun-dation of Science and Technology under grantsSFRH/BD/76438/2011, PEst-OE/EEI/LA0008/2013and EXPL/EEI-AUT/0329/2013.

ReferencesBayindir, L. and Sahin, E. (2007). A review of studies in

swarm robotics. Turkish Journal of Electrical Engi-neering & Computer Sciences, 15(2):115–147.

Beer, R. D. and Gallagher, J. C. (1992). Evolving dynam-ical neural networks for adaptive behavior. AdaptiveBehavior, 1(1):91–122.

Carling, J. (2007). Migration control and migrant fatalities atthe Spanish-African borders. International MigrationReview, 41(2):316–343.

Christensen, A. L. and Dorigo, M. (2006). Evolving an in-tegrated phototaxis and hole-avoidance behavior for aswarm-bot. In Proceedings of the 10th InternationalConference on the Simulation & Synthesis of Living

Systems (ALIFE), pages 248–254. MIT Press, Cam-bridge, MA.

Coppens, J. (2013). The Lampedusa disaster: How to pre-vent further loss of life at sea? TransNav, the Interna-tional Journal on Marine Navigation and Safety of SeaTransportation, 7(4):589–598.

Dorigo, M., Trianni, V., Sahin, E., Groß, R., Labella, T., Bal-dassarre, G., Nolfi, S., Deneubourg, J., Mondada, F.,Floreano, D., and Gambardella, L. M. (2004). Evolv-ing self-organizing behaviors for a swarm-bot. Au-tonomous Robots, 17(2):223–245.

Duarte, M., Oliveira, S. M., and Christensen, A. L. (2014a).Evolution of hierarchical controllers for multirobot sys-tems. In Proceedings of the 14th International Confer-ence on the Synthesis & Simulation of Living Systems(ALIFE). MIT Press, Cambridge, MA. In press.

Duarte, M., Oliveira, S. M., and Christensen, A. L. (2014b).Evolution of hybrid robotic controllers for complextasks. Journal of Intelligent and Robotic Systems. Inpress.

Duarte, M., Silva, F., Rodrigues, T., Oliveira, S. M., andChristensen, A. L. (2014c). JBotEvolver: A versa-tile simulation platform for evolutionary robotics. InProceedings of the 14th International Conference onthe Synthesis & Simulation of Living Systems (ALIFE).MIT Press, Cambridge, MA. In press.

Farinelli, A., Iocchi, L., and Nardi, D. (2004). Multi-robot systems: a classification focused on coordination.IEEE Transactions on Systems, Man, and Cybernetics,Part B: Cybernetics, 34(5):2015–2028.

Floreano, D. and Keller, L. (2010). Evolution of adaptivebehaviour in robots by means of Darwinian selection.PLoS Biology, 8(1):1–8.

Floreano, D. and Mondada, F. (1994). Automatic creationof an autonomous agent: Genetic evolution of a neural-network driven robot. In From Animals to Animats3: Proceedings of the 3rd International Conferenceon Simulation of Adaptive Behavior (SAB), pages 421–430, MIT Press, Cambridge, MA.

Goldberg, D. E. (1989). Genetic Algorithms in Search, Op-timization and Machine Learning. Addison-Wesley,Boston, MA.

Groß, R., Bonani, M., Mondada, F., and Dorigo, M.(2006). Autonomous self-assembly in swarm-bots.IEEE Transactions on Robotics, 22(6):1115–1130.

Groß, R. and Dorigo, M. (2009). Towards group transportby swarms of robots. International Journal of Bio-Inspired Computing, 1(1–2):1–13.

ALIFE 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems

Halloy, J., Mondada, F., Kernbach, S., and Schmickl, T.(2013). Towards bio-hybrid systems made of socialanimals and robots. In Proceedings of the 2nd Inter-national Conference on Biomimetic and Biohybrid Sys-tems (LM), pages 384–386. Springer, Berlin, Germany.

Hauert, S., Zufferey, J.-C., and Floreano, D. (2009). Evolvedswarming without positioning information: an appli-cation in aerial communication relay. AutonomousRobots, 26(1):21–32.

Ijspeert, A. J., Crespi, A., Ryczko, D., and Cabelguen, J.-M. (2007). From swimming to walking with a sala-mander robot driven by a spinal cord model. Science,315(5817):1416–1420.

Jakobi, N. (1997). Evolutionary robotics and the radi-cal envelope-of-noise hypothesis. Adaptive Behavior,6(2):325–368.

Kam-Chuen, J., Giles, C., and Horne, B. (1996). An analy-sis of noise in recurrent neural networks: convergenceand generalization. IEEE Transactions on Neural Net-works, 7:1424–1438.

Lee, W.-P. (1999). Evolving complex robot behaviors. In-formation Sciences, 121(1-2):1–25.

Lutterbeck, D. (2004). Between police and military the newsecurity agenda and the rise of gendarmeries. Cooper-ation and Conflict, 39(1):45–68.

Lutterbeck, D. (2006). Policing migration in the mediter-ranean. Mediterranean Politics, 11(1):59–82.

Manley, J. E. (2008). Unmanned surface vehicles, 15 yearsof development. In Proceedings of OCEANS 2008,pages 1–4. IEEE Press, Piscataway, NJ.

Meyer, J.-A., Doncieux, S., Filliat, D., and Guillot, A.(2003). Evolutionary approaches to neural controlof rolling, walking, swimming and flying animats orrobots. In Duro, R., Santos, J., and Grana, M., ed-itors, Biologically Inspired Robot Behavior Engineer-ing, pages 1–43. Springer, Berlin, Germany.

Monzini, P. (2007). Sea-border crossings: The organizationof irregular migration to italy. Mediterranean Politics,12(2):163–184.

Moore, J. M., Clark, A. J., and McKinley, P. K. (2013).Evolution of station keeping as a response to flowsin an aquatic robot. In Proceeding of the 15th An-nual Conference on Genetic and Evolutionary Com-putation Conference (GECCO), pages 239–246. ACMPress, New York, NY.

Mouret, J.-B. and Doncieux, S. (2008). Incremental evo-lution of animats’ behaviors as a multi-objective op-timization. In Proceedings of the 10th InternationalConference on the Simulation of Adaptive Behaviour(SAB), pages 210–219. Springer, Berlin, Germany.

Nelson, A. L., Barlow, G. J., and Doitsidis, L. (2009).Fitness functions in evolutionary robotics: A surveyand analysis. Robotics and Autonomous Systems,57(4):345–370.

Nolfi, S. and Floreano, D. (2000). Evolutionary robotics:The biology, intelligence, and technology of self-organizing machines. MIT Press, Cambridge, MA.

Praczyk, T. (2014). Using augmenting modular neural net-works to evolve neuro-controllers for a team of under-water vehicles. Soft Computing, pages 1–16.

Schmickl, T., Thenius, R., Moslinger, C., Timmis, J.,Tyrrell, A., Read, M., Hilder, J., Halloy, J., Campo, A.,Stefanini, C., et al. (2011). CoCoRo–The Self-AwareUnderwater Swarm. In Proceedings of the 5th IEEEConference on Self-Adaptive and Self-Organizing Sys-tems Workshops (SASO), pages 120–126. IEEE Press,Piscataway, NJ.

Schwing, R. P. (2007). Unmanned aerial vehicles-revolutionary tools in war and peace. Technical report,U.S. Army War College, Carlisle Barracks, Carlisle,PA.

Silva, F., Duarte, M., Oliveira, S. M., Correia, L., and Chris-tensen, A. L. (2014). The case for engineering the evo-lution of robot controllers. In Proceedings of the 14thInternational Conference on the Synthesis & Simula-tion of Living Systems (ALIFE). MIT Press, Cambridge,MA. In press.

Uchibe, E. and Asada, M. (2006). Incremental coevolutionwith competitive and cooperative tasks in a multirobotenvironment. Proceedings of the IEEE, 94(7):1412–1424.

Watson, R., Ficici, S., and Pollack, J. (1999). Embodiedevolution: Embodying an evolutionary algorithm in apopulation of robots. In Proceedings of the Congresson Evolutionary Computation (CEC), pages 335–342.IEEE Press, Piscataway, NJ.

Whiteson, S., Kohl, N., Miikkulainen, R., and Stone, P.(2005). Evolving keepaway soccer players through taskdecomposition. Machine Learning, 59(1):5–30.

Yan, R.-j., Pang, S., Sun, H.-b., and Pang, Y.-j. (2010). De-velopment and missions of unmanned surface vehicle.Journal of Marine Science and Application, 9(4):451–457.

ALIFE 14: Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems