J Intell Robot SystDOI 10.1007/s10846-012-9783-5

A Decentralized Architecture for Multi-Robot SystemsBased on the Null-Space-Behavioral Controlwith Application to Multi-Robot Border Patrolling

Alessandro Marino · Lynne E. Parker ·Gianluca Antonelli · Fabrizio Caccavale

Received: 20 April 2012 / Accepted: 11 September 2012© Springer Science+Business Media Dordrecht 2012

Abstract This paper presents a control architec-ture for multi-robot systems. The proposed ar-chitecture has been developed in the frameworkof the Null-Space-based-Behavioral (NSB) con-trol, a competitive-collaborative behavior-basedcontrol approach. The standard NSB staticallydetermines a set of suitably defined elementary

The manuscript is based on three conference papers ofthe same authors, namely, Marino et al. [27–29].

A. Marino (B)Dipartimento di Ingegneria Elettronica e IngegneriaInformatica, Università degli Studi di Salerno,Via Ponte Don Melillo, 84084, Fisciano, SA, Italye-mail: almarino@unisa.it

L. E. ParkerDepartment of Electrical Engineering and ComputerScience, The University of Tennessee,1122 Volunteer Blvd, Knoxville, TN 37996-3450, USAe-mail: leparker@utk.edu

G. AntonelliDIEI - Dipartimento di Ingegneria Elettricae dell’Informazione, Università di Cassinoe del Lazio Meridionale, Via G. Di Biasio 43,03043, Cassino, FR, Italye-mail: antonelli@unicas.it

F. CaccavaleScuola di Ingegneria, Università degli Studi dellaBasilicata, Viale dell’Ateneo Lucano 10,85100, Potenza, Italye-mail: fabrizio.caccavale@unibas.it

tasks (behaviors) and their priorities, i.e., theycannot be dynamically changed according to mis-sion requirements and environmental constraints.In this paper, a three layer architecture has beendesigned in order to avoid such a drawback. Thesingle robotic unit (agent) performing the mis-sion is placed on the lower layer. In the middlelayer, suitably defined elementary behaviors aredefined; these elementary behaviors are then com-bined, via the NSB approach, in more complexactions. The upper layer is a Supervisor in chargeof dynamically selecting the proper action to beexecuted. As further contribution, the architec-ture has been applied to the multi-robot borderpatrolling mission to generate a decentralized, de-terministic and non-communicative solution thatis robust to faults, and prevents collisions, even inthe case of high robot density. Finally, the simula-tions on a team composed by a large number of ro-bots, and experiments on a real setup, composedby three Pioneer-3DX robots, are provided.

Keywords Behavioral control · Null-space basedbehavioral approach · Multi-robot systems ·Swarm robotics · Border patrol

1 Introduction

Collaborative multi-robot teams have great po-tential to add capabilities and minimize risks,

J Intell Robot Syst

especially in the security domain. In fact, robotsare increasingly used in performing patrolling,surveillance and military tasks. Today, such robotsare not able to carry out a mission on their own,due to limited autonomy; hence, most of them areremotely controlled by human operators. This isdue also to the risks connected to robots’ failuresin highly dynamic scenarios.

Of course, multi-robot systems require a co-ordination mechanism to accomplish the mission.Although several paradigms have been developedin the last decades, behavior-based robotics hasbeen proven to be a successful technique [9].

Arbitration mechanisms are usually adoptedfor selection of active behaviors on the basis ofmission and system requirements. An example isthe subsumption architecture, in which a priorityis assigned to each behavior, and behaviors withhigher priorities are allowed to override the out-put of behaviors with lower priority [9]. However,multiple non-conflicting behaviors cannot be acti-vated simultaneously.

Command fusion mechanisms try to establisha sort of cooperation between behaviors. For ex-ample, the motor schema allows the output ofmultiple behaviors to be combined [7]. However,it suffers from the problem of local minima andcommand averaging without explicit handling ofconflicting behaviors.

The traditional NSB approach requires thedefinition of a set of elementary behaviors [4].The mission achievement is obtained by properlycombining the elementary behaviors via the null-space projection mechanism.

Thus, the NSB can be defined as a competitive-cooperative approach, that overcomes the dis-advantages of the above approaches, allows theactivation of several behaviors at once and prop-erly handles the conflicts among them. A compari-son of the NSB approach with the other paradigmscited above can be found in Antonelli et al. [5].However, the behaviors and their priorities arestatically determined, i.e., they cannot be changeddynamically according to mission requirementsand environment constraints. Hence, the standardNSB approach can be effectively applied onlyto relatively simple missions. Thus, it is worth

further extending this mechanism to handle dy-namic changes of behaviors and their priorities.

In this paper, the NSB approach for multi-robot systems has been extended with respectto previous published papers. Namely, a missionwhose complexity cannot be effectively handledby the standard NSB approach is first decom-posed into several sub-missions (or sub-goals).Each sub-mission requires the activation of mul-tiple behaviors, each characterized by its pri-ority, and their combination according to theNull-Space-Behavioral approach. Such a combi-nation of behaviors is called action. Finally, ahigher level (i.e., the Supervisor) is needed todynamically select the action to be activated. Inaddition, the classic NSB has a centralized struc-ture, i.e., a central computing unit or a leaderagent is in charge of collecting the system’s stateand calculating the agents’ motion commands.Here, a different perspective is used. Namely, acompletely decentralized architecture is proposedto overcome this limitation.

The proposed architecture has been appliedto the multi-robot patrolling mission. The over-all patrolling architecture in the NSB frameworkis able to perform the mission in very criticalconditions and under strong requirements (seeSection 3.1). In detail, a suitable set of elemen-tary behaviors is defined, on the basis of typicalpatrolling mission requirements. Then, these be-haviors are composed, via NSB, into more mean-ingful and complex actions, which represent givensub-goals. Once the set of actions is defined, theSupervisor selects the action to be performed byeach robot in order to achieve mission goals.

The proposed control architecture can beadopted by multi-robot systems to perform dif-ferent classes of missions, by properly formulatingthe set of behaviors and actions. For the sakeof clarity, in this paper the designed procedurehas been presented by direct application to thepatrolling mission that is a challenging scenariothat helps to understand the main features ofthe approach.

The paper has been organized as follows. InSection 2, the contributions of the paper arestrengthened and summarized, while in Section 3,

J Intell Robot Syst

an overview of the patrolling mission is carriedout and the main assumptions and requirementsare determined. A description of the designedarchitecture is given in Section 4. A short reviewof the NSB approach together with the definitionof the elementary behaviors and actions is givenin Section 5. The supervisor in charge of select-ing the proper action to execute is described inSection 6. Simulations and experimental resultsare presented in Sections 8 and 9. Finally, conclu-sions are drawn in Section 10.

2 Main Contributions and Novelty

The contribution of the paper aims to be twofold.First, a framework has been designed that allowshandling complex missions by means of a three-layered architecture. This objective is achievedthanks to the use of the NSB in the middle layer,that allows building a set of well-structured be-haviors and actions, whose output is fully pre-dictable. Moreover, the NSB framework allowsthe designer to focus on the design of the toplevel rather than on the low level effects. Second,the feasibility of the approach has been demon-strated by showing a practical example of how theproposed architecture can be used to accomplishthe patrolling task. In addition, experiments havebeen run on a real setup composed by three fullyautonomous vehicles.

3 The Patrolling Mission: Main Requirementsand Assumptions

As stated above, the mission to accomplish isthe surveillance of a given area, i.e., a border orany military and civil facility. An early exampleof a robotic surveillance system is the MobileDetection Assessment and Response System-Exterior (MDARS-E) [19] whose goal is toprovide multiple mobile platforms that performrandom patrols within assigned areas of ware-houses. Sandia National Laboratories has de-veloped the Surveillance And ReconnaissanceGround Equipment (SARGE) robotic vehicle for

the US Department of Defense [35]. SARGE isa teleoperated robot for battlefield surveillanceapplications without computing power, aimedat supporting autonomous navigation or visionprocessing.

Recently, commercial border patrol applica-tions have been proposed as well, such asGuardium [21] and the unmanned autonomousspeed boat Protector manufactured by Ltd [36].

Perimeter surveillance algorithms form the ba-sis for effective execution of monitoring tasks ina number of applications fields, e.g., monitoringof oil spills [15], contaminant clouds, algae bloom[8], forest fires [12] and border security [23].

In Machado et al. [26], an analysis of the mainpatrolling task issues and some multi-agent-basedsolutions are presented. Several features (agenttype, agent communication, coordination scheme,agent perception and decision-making) are evalu-ated by using different criteria.

In Kingston et al. [25], a decentralized solutionto the multi-robot perimeter surveillance is pre-sented, where changes of both the shape of theperimeter and the number of robots are takeninto account; moreover, critical coordination in-formation is exchanged to optimize some perfor-mance index. In Agmon et al. [1], the authorsanalyze non-deterministic paths for a group ofhomogeneous mobile robots patrolling a frontier,under the assumption of a hostile agent trying toenter the area, where the latter has full knowledgeof the algorithm.

In this paper, a linear border to be patrolledwas considered. The border may be any imaginary(like in the case of aerial vehicles as in [25]) orreal line surrounding strategical facilities or evencountries’ borders. In a real mission scenario, thefirst task the vehicles have to be able to performis to get to the border itself from, for example, abase station. After the border has been reached,the vehicles have to move along the border whilekeeping staying on the border. In addition, sta-tic or dynamic obstacles need to be considered.Obstacles can be represented by other teammatesor natural obstacles close to the border. Finally,the case of a friend vehicles that try to enterthe border can be of interest. A friend vehicle

J Intell Robot Syst

is any agent that is allowed entering the area tobe patrolled and that has not to be stopped bypatrolling vehicles.

In the following, the main requirements and as-sumptions related to the border patrolling missionare listed.

3.1 Requirements

A patrolling mission requires a high degree ofautonomy and robustness. Several aspects andconstraints might affect the mission achievements,e.g., occurrence of robot faults, large number ofpatrolling robots, presence of friends or intrudersinterfering with patrolling robots, limited commu-nication range and computational capabilities. Forthese reasons, the following requirements need tobe met.

Requirement 1: Decentralization The control ar-chitecture needs to be fully decentralized, i.e.,there is not a central unit in charge of computingthe patrolling robots’ motion commands, by gath-ering information from them.

Requirement 2: Communication Explicit com-munication between patrolling robots is forbid-den. This requirement is devoted to supply morerobustness to the control architecture as well asto enhance security, since, in certain applications,encryption of communications would be needed,at the expense of an increased software complex-ity. Moreover, in some scenarios (e.g., underwaterenvironments and/or wide areas to be patrolled)reliable communications could be not available.Moreover, even in the case that explicit commu-nications can be assumed, the gathered data mayrequire significant time to transmit (e.g., completevideo footage). Of course, this requirement is notindependent from the previous one, since, withoutany form of communication, a centralized archi-tecture cannot be implemented.

Requirement 3: Collision avoidance It is obviousto require that each patrolling robot must avoidcollisions with other teammates, friends or intrud-ers, even in the case of a large number of robots.In addition, in the case of a large number ofrobots, interference phenomena must be avoided,

i.e., the adoption of a large number of patrollingrobots must not lead to mission failure or otherunexpected and undesired events.

Requirement 4: Robustness Patrolling missionsare usually performed in challenging environ-ments. Hence, it is mandatory to deal with robotfaults, by providing a suitable degree of toleranceto faults of the single agents in the patrolling team.More generally, sudden changes in the number ofpatrolling robots (due, e.g., to an increase of theavailable robots or failures of single robots) mustbe taken into account.

Requirement 5: Computational burden Usually,in multi-robot systems, relatively simple robotsare employed to achieve a given mission. Thus, thecomputational burden of the control algorithmsversus the available computational power of thesingle robotic units must be taken into account.

3.2 Assumptions

The following assumptions are adopted to de-velop the proposed solution to the multi-robotborder patrolling problem, according to the abovedefined requirements.

Assumption 1: Border geometry The generalperimeter surveillance problem is reduced to thelinear surveillance problem, by assuming that theperimeter to be monitored is homeomorphic to aline, and thus it can be represented as a single pathbetween two points. In addition, closed perimetersare allowed.

Assumption 2: Visibility range It is assumed thateach robot has a limited visibility range, whereit recognizes the presence of other agents anddetects the border. For the sake of simplicity,the visibility area is modeled as a circle aroundthe robot. This is not an unrealistic assumption,since it well approximates the case of pan-tilt-zoom camera sensors, laser range finders or omni-directional cameras.

Assumption 3: Localization capabilities It is as-sumed that each robot is able to localize it-self in the environment and knows a local (i.e.,

J Intell Robot Syst

limited to its visibility range) geometric descrip-tion of the border (or is able to locally estimatethe geometry of the border). Localization and, ingeneral, Simultaneous Localization and Mapping(SLAM) [38] are well-established problems in theliterature and will be not addressed in this paper.Indeed, each robot needs to estimate only its posi-tion with respect to the border and other agentsthat are in its visibility range. How this goal isachieved highly depends on the available sensorsand the practical goals to be faced. A notableexample is given in Bruemmer et al. [10], wherea swarm is able to locate and surround a waterspill using social potential fields implemented viaIR, chirps and light sensing; moreover, in Clarkaand Fierro [15], cameras are used to locate theperimeter of the area to patrol. Finally, it assumedthat each robot is able to follow the border [40].

Assumption 4: Safety area Each robot must becharacterized by its own safety area, in whichother agents are not allowed to enter. The safetyarea is a circle around the robot (smaller than thevisibility area). Such an assumption is necessary tomeet the collision avoidance requirement, partic-ularly in the case of teams composed by a largenumber of robots, where conflicts and interfer-ences may arise.

Assumptions 5: Awareness Each robot does notknow the exact number of patrolling robots. How-ever, it is aware of the existence of other agents(other patrolling robots, friends or intruders). Asdescribed in the following, the awareness assump-tion will be implicitly taken into account by prop-erly defining the elementary behaviors.

4 Control Architecture

4.1 Paradigm

By taking into account the requirements definedin Section 3.1, an approach belonging to theswarm robotics class has been adopted. In swarmrobotics (see Dorigo and Sahin [16] for a sur-vey and Dudek et al. [17] for a taxonomy),the achievement of the mission is the result ofthe cooperation of independent agents forming

the swarm. The swarm intelligence aims at de-veloping robust task solving by minimizing thecomplexity of the individual units; its central ideais to distribute the control over a group of numer-ous minimalist robots rather than gathering andredistributing information with the help of a cen-tral unit. There are two main advantages of thisapproach: first, scalability from a few to thousandsof units, second, increased system robustness, notonly through unit redundancy but also through theunit simple design. In this approach, the overallbehavior is generally emergent, i.e., each unit doesnot have a global cognition of the assigned missionand reacts to external stimuli via simple behaviors,where, in general, the stimuli-behaviors couplesare organized in such a way that the cooperationimplicitly arises from the interaction among ro-bots and with the environment.

Moreover, this approach is clearly modular,since new emergent behaviors at the swarm levelcan be obtained and, in general, more com-plex missions could be accomplished by properlydefining new behaviors at the level of the singleunit. There are, of course, some drawbacks con-nected to this approach. Namely, formal methodsare still not available that can generate (startingfrom a suitable description of the overall mis-sion) a set of stimuli-behaviors couples, whoseinteraction leads to task accomplishment. Thisis true especially in the presence of dynamicallychanging environments.

4.2 Outline of the Control Architecture

Since the approach is completely decentralizedand disallows any form of communication, thecontrol architecture described in the followingmust refer to the single robot in the patrol-ling team.

As it can be seen in Fig. 1, the control ar-chitecture is composed by three layers. Startingfrom the top layer, they are the Supervisor Level,the Action Level and the Robot Level. The firsttwo levels are abstract levels. In particular, theAction Level, described in detail in Section 5,defines the set of actions the robot can undertake.Clearly, the set of actions depends on the missionto be achieved, involving different skills. As al-ready stated, this means that the architecture does

J Intell Robot Syst

Fig. 1 Sketch of the control architecture

not depend on the particular mission, but can beapplied to different missions by properly definingthe action set. Once the actions have been defined,each robot chooses the proper action to perform,according to its internal state and environmentalinformation. To this aim, a Supervisor needs to bedesigned. Finally, the Robot Level comprises thecomputing hardware/software and the mechanicalfeatures of the single robot; the structure of thislayer depends on the adopted robotic platformand is primarily concerned with the available sen-sors and actuators.

A description of the platform used to test theproposed approach is given in Section 9. In thefollowing, the architecture will be detailed withreference to the patrolling mission

5 Behaviors and Actions

In this section, the set of behaviors required bythe patrolling mission is defined. Then, thesebehaviors are combined into more complex andmeaningful actions by means of the Null-Space-based-Behavioral (NSB) approach.

5.1 Review of the NSB Approach

A complex mission for a robotic system can beencoded via different basic tasks (behaviors) to beproperly combined to achieve mission goals. Asalready stated, the NSB approach can be definedas a competitive-cooperative approach, trying toovercome the disadvantages of arbitration andcommand fusion approaches; it is based on the

task-priority control approach for robotic manip-ulators [32]; it has been experimentally comparedto the layered-control system and motor-schemacontrol in the case of a wheeled mobile robot [5],and successfully applied to control of a platoon ofground vehicles [6].

Let us consider the task function σ ∈ �m whichis the mathematical representation of a generictask to be achieved by a robotic system. It isassumed that

σ = σ ( p) ∈ �m, (1)

where p ∈ �n is a vector describing the systemconfiguration (e.g., in case of a team of robots,p is the vector containing the positions of all thevehicles) and m is the dimension of the task space.Assuming that σ is a differentiable function, it isuseful to consider the relationship between thefirst derivatives of σ and p

σ = ∂σ ( p)

∂ pp = J( p)v, (2)

where J ∈ �m×n is the task Jacobian matrix.Equations 1 and 2 represent the so-called di-rect task model of the system, i.e., the equationsneeded to compute the value of the task func-tion, σ , given the system configuration, p. Theinverse task model problem is defined as the de-termination of the desired system’s configuration,pd, corresponding to a desired task function, σ d.It can be easily recognized that the inverse taskmodel problem is of the utmost importance, since,by means of the desired task function, the de-sired high level aggregate behavior of the sys-tem is specified, while the corresponding desiredconfiguration represents the set of commands tobe delivered to the actuators. A possible solutionto the inverse task model problem can be soughtat the differential level, by inverting the (locallylinear) mapping (Eq. 2); this solution has beenwidely studied in robotics (see, e.g., [37] for a tuto-rial). A typical requirement is to pursue minimum-norm velocity in a closed loop version, leading to

vd = J†(σ d + �σ), (3)

where J† = JT (J JT)−1

is the pseudo-inverse ofthe Jacobian matrix, � is a suitable positive-

J Intell Robot Syst

definite matrix gain and σ = σ d − σ is thetask error.

It can be easily shown that Eq. 3 leads to thefollowing asymptotically stable error system

˙σ + �σ = 0. (4)

When dealing with complex missions, severaltasks need to be handled. If N tasks (encoded byfunctions σ d,1, . . . , σ d,N) are considered, the NSBsolution to the task combination can be formu-lated in a recursive way, by computing the seriesof velocity vectors as follows

⎧⎨

⎩vd,i = J†

i

(σ d,i + �σ i

)

v(i) = vd,i + Ns,iv(i+1) ,

i = 1, 2, · · · N , (5)

where Ji is the i-th task Jacobian, v(N+1) = 0, vd =v(1) and the matrix

Ns,i = I − J†s,i J

Ts,i

is a projector onto the null-space of Js,i (i.e., itscolumns form a basis of the null space of Js,i). Thematrix Js,i is obtained by stacking the matrices Jk

with k = 1, 2, · · · , i. In the case of two tasks σ 1 ∈�m1 and σ 2 ∈ �m2 , as will be the case in this paper,Eq. 5 becomes

⎧⎪⎪⎨

⎪⎪⎩

vd,1 = J†1

(σ d,1 + �σ1

)

vd,2 = J†2

(σ d,2 + �σ2

)

vd = vd,1 + N1vd,2.

(6)

In sum, the commanded velocities correspondingto a lower priority task are projected onto thenull-space of the immediately higher priority task;then, eventually conflicting velocity componentsare cut off before being added to the higher pri-ority task velocity components. In this way, lowerpriority behaviors are executed only in their com-ponents not affecting higher priority behaviors.Convergence to zero of task errors can be guaran-teed for properly defined tasks [3]. On the otherhand, a differentiable analytic expression of thedefined behaviors is required, so as to computethe required Jacobian matrices.

5.2 Elementary Behaviors and Actions

In the case of the border patrolling of a linearborder, a set of elementary behaviors is defined:

• Reach Frontier• Patrol Frontier Clockwise• Patrol Frontier Counter-Clockwise• Teammate Avoidance• Friend Avoidance

whose semantics and analytical expressions aregiven in Appendix A.1.

As motivated above, it is appropriate to com-pose the elementary behaviors into more com-plex behaviors; the latter are sometimes definedas behavior sets in the literature. Similar to theconcept of behavior set, here a higher abstractionlayer is introduced: the action. An action is givenby the proper composition, achieved via NSB,of several elementary behaviors and represents amacroscopic attitude of the robotic system. Onlyone single action can be active at once.

For the specific case of the border pa-trolling task, the following set of actions is ob-tained by combining the elementary behaviorsdefined above:

• Action Reach Frontier• Action Keep Going• Action Patrol Clockwise• Action Patrol Counter-Clockwise• Action Teammate Avoidance• Action Friend Avoidance

According to the definitions in Appendix A.2,each action is given by elementary behaviorsarranged in priority; e.g., the Reach Frontieraction properly combines the elementarybehaviors Reach Frontier and TeammateAvoidance, depending on the sensed presenceof other patrolling robots in the visibility rangeand the distance from the border. It is worthnoticing that such actions require that each robotis able to recognize other agents and their nature(friends or teammates).

5.3 Behaviors Compatibility

In the NSB framework it is possible to defineconditions for behavior compatibility [3], allowing

J Intell Robot Syst

the robot to properly handle conflicts amongdifferent behaviors (algorithmic singularities). Is-sues related to singularities are deeply discussedin Chiaverini [14], where a singularity-robust task-priority strategy, in the case of two tasks, is de-rived. According to this technique, a secondarytask is fulfilled if it does not conflict with ahigher level task, while it is released in the caseit is conflicting. The more general case of com-patibility of three or more tasks handled in theNSB framework has been discussed in Antonelli[3], by resorting to a Lyapunov-based analysis.A straightforward application of the results ofthe above mentioned papers guarantees properdefinition of all the actions used in this paper; inparticular, collisions between robots and betweenrobots and obstacles are avoided. Compatibility ofthe behaviors combined in actions is discussed inthe Appendix.

6 The Supervisor

The set of actions defines the robot’s skills neededto react to situations encountered in the envi-ronment. Hence, each robot has to be equippedwith a Supervisor that is in charge of selecting theaction to be executed, according to the robot’sinternal state and/or external stimuli. Accordingto the assumptions and requirements defined inSection 3, the Supervisor of each robot decidesthe next action to be performed, based only onlocally available information. Although severalparadigms might be used, two appealing tools forthe design of the Supervisor are represented byFinite State Automata and Fuzzy Logic Engines.The Finite State Automata Supervisor [27] willbe described in the following, while details on theFuzzy Logic Supervisor can be found in Marinoet al. [29].

A comprehensive description of hierarchicalstate machines and of their properties is car-ried out in Alur and Yannakakis [2]. Finitestate machine action selection mechanisms as-sume that [11]:

• there are only a limited number of salientsituations the agent can find itself in,

• these situations are mutually exclusive,

• actions to be performed can be easily mappedto situations.

To build a Finite State Automata, two sets mustbe determined:

• the sets of the states the agent can be in,• the sets of causes forcing the agent to change

its state.

The structure of the supervisor that selects theproper action is shown in Fig. 2.

As it can be seen, the supervisory layer isarranged in a hierarchical way, by defining threelevels. At the top level two scenarios have beenidentified. The first one corresponds to the situ-ation where a friend agent is in the safety area

Fig. 2 Sketch of the supervisor

J Intell Robot Syst

of the patrolling robot. In the second scenariono friend agents are in the visibility area of pa-trolling agents. The second level defines fourmacro-states: MS0, for the first scenario, MS1, MS2and MS3 for the second scenario. At each timeinstant the robot can be in one of the macro-states.In macro-state MS0, Action Avoid Friend isactive. On the other hand, if no friend agents arepresent (second scenario), one of the states MS1,MS2 or MS3 is active. In the macro-state MS1, therobot tries to reach the border (and, at the sametime, avoid the teammates). In the macro-stateMS2 the robot behaves as if it is patrolling theborder (while avoiding teammates); however, inthis state, the robot still cannot be considered asperforming the patrolling mission. In the macro-state MS3, the robot performs the patrolling mis-sion and does not allow robots approaching theborder to influence its motion.

The reason for the distinction between macro-states MS2 and MS3 is that, in the case of alarge number of patrolling robots, interferencebetween robots can be effectively counteracted.In fact, when a robot is in the macro-stateMS3, it cannot be influenced by other team-mates that are not in the same macro-state; theavoidance of other teammates that are in themacro-state MS3 is achieved by activating ActionPatrol Clockwise and/or Action PatrolCounter-Clockwise.

In detail, the supervisor acts as described inthe following.

Scenario 1 One or more friend agents are in thesafety area of the robot

• Macro-State MS0:

– Action Avoid Friend is active

Scenario 2 No friend agents are in the safety areaof the robot

• Macro-State MS1: The distance between therobot and the border is larger than theVisibility Range

– if no teammate is in the safety area thenAction Reach Frontier is active,

– if one or more teammates are in the safetyarea then Action Avoid Teammateis active.

• Macro-State MS2: The distance between therobot and the border is smaller than the Vis-ibility Range and larger than a given thres-hold, ζFS

– if no teammate is in the safety area thenAction Keep Going is active,

– if one or more teammates are in the safetyarea then Action Avoid Teammateis active.

• Macro-State MS3: The distance between therobot and the border is smaller than a giventhreshold, ζFS

– if no teammate is in the visibility rangethen Action Keep Going is active,

– if there is a teammate on the left thenAction Patrol Clockwise is active,

– if there is a teammate on the right thenAction Patrol Counter-Clockwiseis active.

Finally, it is worth noticing that, when the sys-tem has enough degrees of mobility, a more ad-vanced Supervisor could be designed to furthercombine, via NSB, the actions’ outputs in orderto achieve more sub-goals at once. For instance,when a Fuzzy Logic Supervisor is used, severalactions can fire at once with different degrees ofactivation. These can be used as priority of thefired actions in the NSB approach.

7 Analysis

The FSA has been designed according to guide-lines described in Murphy [31, pp. 163–209]. Themain properties that have been considered andchecked are the consistency and completeness [33].Namely, a FSA is said to be consistent if only a sin-gle transition rule can be enabled at the same timefor all the states. By considering the structure ofthe supervisor, it can be verified that this propertyholds.

Proposition 7.1 The FSA shown in Fig. 2 isconsistent.

J Intell Robot Syst

Proof Since the supervisor is arranged in a hierar-chal way with three levels (Fig. 2), with scenariosat the top layer, macro tasks in the middle layerand actions at the lowest layer, it is required toprove that in each time instant:

1. only one scenario can be active,2. for each scenario, only one macro task can

be active,3. for each macro-task, only one action can

be active.

1) Consistency of the Scenario LayerWith regards to the scenarios, the followingtransition has been defined:

• C0: there is a friend in the safety area ofthe considered robot;

this condition, essentially, identifies two situ-ations depending on whether a friend vehicleis close or not to the given patrolling vehicle.Since the two conditions are mutually exclu-sive, it is obvious that only one scenario canbe active at a time.

2) Consistency of the Macro-Task LayerWith regards to the macro-tasks, there is onlya macro-task in the first scenario (MS0), thus,no conflict occurs in this scenario. With re-gards to the second scenario, the followingtransitions for the macro-tasks (MS1, MS2,MS3) have been defined:

• C1: the distance from the border is smallerthan the visibility range,

• C2: the distance from the border issmaller than the threshold ζFS defined inSection 6.

From each of the considered macro-tasks, onlyone output transition is defined, thus, conflictscannot occur.

3) Consistency of the Actions LayerIn the case of the actions in each macro-state, the following situations have tobe considered:

• the macro-state MS0 is active. In this case,only the action Action Avoid Friendcan be active;

• the macro-state MS1 is active. Two ac-tions might be active (Action ReachFrontier and Action Avoid Team-mate). Only one transition rule, namelyc1, is present; this means that conflicts donot arise;

• the macro-state MS2 is active. Two ac-tions might be active (Action KeepGoing and Action Avoid Teammate).Also in this case, only one transition,namely c1, is present; thus, any conflictis avoided;

• the macro-state MS3 is active. Three ac-tions might be active (Action KeepGoing, Action Patrol Clockwiseand Action Patrol Counter-Clock-wise). Three transition rules are present,namely c2, c3, c4. Because c2 and c3cannot both be true at same time, thefollowing combinations are admitted:

c2 c3 c4=!c2 ∧ !c31 0 00 1 00 0 1

this implies that only one transition rule isactive at a time. In addition, each actionis characterized by only one output transi-tion. Thus the consistency is proved. ��

Moreover, a FSA is said to be complete [33]if it operates correctly for all possible input/state sequences.

Proposition 7.2 The FSA shown in Fig. 2, com-bined with the behavior-action level and the prop-erties of the NSB approach, is complete.

Proof Completeness can be inferred from themain NSB properties. In the environmental modelthat has been considered, in each time instantthe inputs a vehicle takes into account are thedistance from the border, the distance from thefriend vehicles and the distance from teammatesor other obstacles. Based on the macro stateMSi (i=0,1,2,3), any sequence of the consid-ered inputs is admissible for the designed FSA.

J Intell Robot Syst

In fact, any sequence of the following situationsis admissible:

• a friend is in the visibility area. In this case, themacro state MS0 is active and then the ActionAvoid Friend. As stated in Section A.2.5,this action guarantees that no collision bet-ween friends occurs;

• no friend and no obstacle is in the vis-ibility area. In this case, either the vehi-cle is in the macro states MS1 or MS2.Thanks to the Action Reach Frontier(see Section A.2.1) it is ensured that thevehicle gets to the border and starts thepatrolling mission;

• an obstacle is in the visibility area. In thiscase, the robot is either in the macro stateMS1 or MS2. The Action Avoid Teammatein Section A.2.4 guarantees that no collisionoccurs, since the collision avoidance has thehighest priority task in the NSB sense. Thuscompleteness is proven. ��

In sum, in the case of one robot, the FSAensures the robot reaches the border and startsthe patrolling mission (in macro-state MS3). Infact, whatever is the starting macro-state, sinceReach Frontier is always the primary behav-ior (in the absence of obstacles or other team-mates), the robot is forced to reach the border(i.e., the FSA is in the macro-state MS3). In thecase of multiple robots (but in the absence offriends), the transition towards MS3 occurs if thereis enough space along the border to let the robotstay on it. The behavior of robots left out of theborder (whose supervisor is in macro-states MS1or MS2), thanks to the distinction between macro-state MS2 and MS3, do not affect the patrollingrobots. In the presence of friend vehicles and inde-pendently from the other conditions, the macro-state MS0 becomes active. In this case, the ReachFrontier Behavior is not the primary behav-ior; hence, the robot reaches the border only afterthe friend vehicles have crossed it (the behav-iors composing Action Avoid Friend are notcompatible [3]).

7.1 Advantage of the use of NSB

The considered task priority algorithm guaranteesthat the higher priority tasks are not affectedby the lower ones. For the generic task, in fact,it has been demonstrated that its error con-verges to zero within the null-space of the higherones and it is not affected by the lower ones[5]. The actions used in this paper are prop-erly defined as discussed in the Appendix; inaddition, tuning of the feedback gains is ex-tremely simplified with respect to cooperativebehavioral approaches. In fact, the NSB decou-ples the tasks spaces, meaning that the parametertuning can be done for each task independentlyfrom the remaining ones and independently fromits priority. This property is a significant ad-vantage with respect to any cooperative behav-ior approach. In fact, the coupling among thetasks/behavior is done by selecting the priority,thus at a higher level, and not by selecting thegains.

8 Simulations

Several simulations, performed in the presence ofboth closed and open borders with different sizesand shapes, have been carried out by using bothMatlab [30] and Player/Stage [39] environments.A video of the simulation can be downloaded atthe footnote link.1

8.1 Simulations in the Presence of a LargeNumber of Robots

In order to test the approach in the presenceof a large number of robots in the team, afree environment with a closed border has beenconsidered in a numerical simulation. The teamis composed by 60 robots, with visibility andsafety areas equal to 20 m. The matrix gains inEqs. 10–12 of Appendix A.1 have been chosen

1A video of the simulation is available at: http://webuser.unicas.it/lai/robotica/video/SimPatrolling.avi.

J Intell Robot Syst

Fig. 3 Sample frames atdifferent time instants ofrobots performing apatrolling mission withoutfriend agents

[m]

[m]

[m]

[m]

[m]

[m]

[m]

[m]

time = 0.4s time = 4s

time = 7s time = 25s

-200

-200

-200

-200

-200

-200

-200

-200

-100

-100

-100

-100

-100

-100

-100

-100

0

0

0

0

0

0

0

0

100

100

100

100

100

100

100

100

200

200

200

200

200

200

200

200

as λr f = 5I, λcw = 8I, λccw = 8I, λta = 10I andλ f a = 10I. The gains have been selected by set-ting the desired velocities to be assumed whenapproaching and patrolling the border, as well aswhen escaping from teammate and friend agents.Friend vehicles are not present in this simulation.

Figure 3 shows the robot positions at fourdifferent time instants. Robots are represented bygreen points surrounded by their visibility ranges;the continuous blue line represents the closedborder to be patrolled. As it can be seen, thetask is executed in four different phases (eachcorresponding to a frame in Fig. 3):

• in the first phase, all robots are trying to reachthe border, while avoiding teammates;

• in the second phase, some robots reach theborder and start patrolling the border;

• in the third phase, most of the robots arepatrolling the border,

• in the fourth phase, the robots reach the max-imum density along the border, and the pa-trolling robots are not affected by other agentstrying to reach the border.

It is useful to remark that a swarm approachshould also prevent robot collisions in critical con-ditions like this one. To this aim, in Fig. 4 (top), the

time history of the minimum value of the distancesamong all possible robot pairs is depicted, i.e.:

dr,min(t) = min∀i �= j i, j∈Nr

‖ pi,r(t) − pj,r(t) ‖, (7)

18

Minimum distance between robots

Coverage Index

[m]

[s]

[s]

6

8

12

14

16

22

5

5

10

10

10

15

15

20

20

20

25

25

30

30

0.2

0.4

0.6

0.8

1

[]

Fig. 4 Top: time history of minimum of distances betweenall possible robot pairs. The dashed line represents the ra-dius of the safety area. Bottom: time history of the coverageindex over time

J Intell Robot Syst

Fig. 5 Sample frames atdifferent time instants ofrobots performing apatrolling mission withfriend agents. Dashedvehicles are the friendagents

−200 −150 −100 −50 0 50 100 150 200

time = 1

m

−200 −150 −100 −50 0 50 100 150 200

−200

−150

−100

−50

0

50

100

150

200time = 20

m

m

−200 −150 −100 −50 0 50 100 150 200

time = 56.2

m

−200 −150 −100 −50 0 50 100 150 200

−200

−150

−100

−50

0

50

100

150

200time = 63.5

m

m

−200

−150

−100

−50

0

50

100

150

200

m

−200

−150

−100

−50

0

50

100

150

200

m

where pi,r(t) is the position of the i-th robot atinstant t, and Nr is the set of patrolling robots.

As it can be seen in Fig. 4 (top), even in the caseof a large number of robots, collisions are avoided.Moreover, the structure of the supervisor avoidsinterference between robots. In fact, the fourthframe in Fig. 3 shows that robots still approachingthe border do not affect the motion of robotsalready performing the patrolling mission (i.e.,robots in the macro-state MS3). In Fig. 4 (bottom),the time history of a coverage index of the border(defined as the ratio between the portion of bor-der that is in the visibility area of the patrollingrobots and the overall length of the border) isshown. As it can be seen, it monotonically ap-proaches the maximum value 1.

8.2 Simulations in the Presence of FriendVehicles

Simulations showing the approach performance inthe presence of 10 friend agents (trying to enterthe border) are carried out. The team is composedby 60 robots with visibility and safety areas equalto 20 m. The gains are the same as in the previoussimulations. Figure 5 shows the robot positions atdifferent time instants. As in the previous case, the

different phases (each corresponding to a frame inthe figure) are the following:

• in the first phase, robots are randomly dis-tributed and try to reach the border, whileavoiding other teammates;

0 5 10 15 20 25 300

5

10

15

20

25

[s]

[m]

MinimumDist

0 5 10 15 20 25 300

10

20

30

40

50

60

70

[s]

[m]

MinimumDistFriend

Fig. 6 Top: time history of minimum of distances betweenall possible robot pairs. The dashed line represents theradius of the safety area. Bottom: time history of the mini-mum of distances between all robot-friend pairs

J Intell Robot Syst

0 5 10 15 20 25 300

0.2

0.4

0.6

0.8

1

[s]

[]

CoverageIndex

Fig. 7 Time history of the coverage index over time in thepresence of several friend vehicles

• in the second phase, some robots havereached the border and start to perform thepatrolling mission, activating Action KeepGoing and Action Patrol Clockwise(or Patrol Counterclockwise);

• in the third phase, friend agents, representedby cross markers surrounded by their safetyareas (dash-dot circles), start to approach theborder and cross it, without any collision withpatrolling robots;

• in the fourth phase, friends gain the cen-ter of the bordered zone; in this situation,patrolling robots are no longer affected byfriends’ motion.

In Fig. 6 (top), the time history of the minimumvalue of the distances among all the possible robotpairs, computed according to Eq. 7, is depicted.

In Fig. 6 (bottom), the minimum distance overtime of all the possible robot-friend pairs isshown, i.e.:

d f,min(t) = min∀i∈Nr, j∈N f

‖ pi,r(t) − pj, f (t) ‖, (8)

where pj, f (t) is the position of j-th friend agentat the time instant t and N f is the set of friend

0 5 10 15 20 25 300

0.2

0.4

0.6

0.8

1

[s]

[]

CoverageIndex

Fig. 9 Time history of the coverage index over time in thecase of several faults

agents. Friend agents cross the border between15 s and 24 s; even during border crossing thesafety distance (dashed line) is not violated. Thus,the proposed architecture is capable of avoidingcollisions, and the mission is safely accomplished.

In Fig. 7, the time history of the coverage indexis shown. Again, since interference situations donot occur, the coverage index monotonically ap-proaches 1; the index decreases only when friendscross the border.

8.3 Simulations in the Presence of FriendVehicles and Multiple Sudden Faults

In this simulation case study, the initial conditionsand the parameter values are the same as in theprevious one. In addition, robot failures occur.Namely, 50 % of the initial patrolling robots sud-denly fail. As in the previous simulations, at thebeginning, robots are randomly distributed andreach a final configuration corresponding to thefourth frames of Figs. 3 and 5. Starting from thisconfiguration, Fig. 8 (left) shows the configurationof robots immediately after the occurrence ofthe faults. The second frame shows that the

Fig. 8 In the left frame,several robots fail. In theright frame, the remainingrobots automaticallyredistribute aroundthe border

−200 −150 −100 −50 0 50 100 150 200

−200

−150

−100

−50

0

50

100

150

200 time = 21.2

m

m

−200 −150 −100 −50 0 50 100 150 200

−200

−150

−100

−50

0

50

100

150

200 time = 26.4

m

m

J Intell Robot Syst

Fig. 10 The experimentalsetup at the DistributedIntelligence Laboratoryof University ofTennessee. Left: aPioneer-3DX mobilerobot. Right: the teamused in the experiments

surviving robots redistribute along the border af-ter the failures, thus keeping the mission objec-tive. In Fig. 9, the time history of the coverageindex is shown. Initially, the index almost reachesthe value 1; at 15.5 s, when 50 % of the robotssuddenly fail, the remaining robots redistribute tomaximize the coverage index.

8.4 Discussion

The numerical simulations have been designedin order to stress the algorithm in severe tests.Dozens of different case studies with differentsimulation parameters such as, e.g., the number ofrobots, the visibility range, the robot speed, etc.,have been run. The results show that, even in thepresence of a large number of robots, collisionsand interference between robots are avoided. Theresults confirm that the approach is capable ofachieving nearly optimal values of the adoptedperformance indexes. The presence of a genericnumber of friend robots in a cloud configuration isalso handled in a proper way, while ensuring satis-factory performance. Finally, simultaneous faultsof several robots affect significantly the perfor-mance only during a transient phase; the expectedvalues for the given number of working robots arethen promptly reached.

9 Experiments

The proposed control architecture has been im-plemented on an experimental setup composedof three Pioneer-3DX mobile robots (Fig. 10),

available at the Distributed Intelligence Labora-tory of University of Tennessee. Videos about theexperiments can be downloaded at the footnotelinks.2

The Pioneer-3DX robot has a 0.44 m ×0.38 m × 0.22 m aluminum body with 0.165 mdiameter. This differential drive platform is non-holonomic and can rotate in place by moving bothwheels, or it can swing around a stationary wheelon a circle of 0.32 m radius. A rear castor balancesthe robot. In addition to motor encoders, therobot base includes eight ultrasonic transducers(range-finding sonar) arranged to provide a 180-degree forward coverage at a sampling rate of25 Hz. The computational board includes a 32-bitRISC-based controller running Linux. Additionalhardware includes ethernet communication, aWi-Fi radio turret for remote control, a Pan-Tilt-Zoom color camera and a laser rangefinder. Sincethe orientation θ of the robot is not of interest andour aim is not to solve the control problem fornon-holonomic systems, the following direct taskmodel has been considered

pb =[

xb

yb

]=

[cos(θ) −b sin(θ)

sin(θ) +b cos(θ)

] [v

ω

], (9)

where pb is a point whose distance from thewheels’ axis is equal to b . Taking into accountthe velocities generated by the control algorithm,the dimensions and the maximum linear and an-gular velocities of the Pioneer-3DX robots, b has

2The videos of the experiments on an open and closed bor-der are available at: http://webuser.unicas.it/lai/robotica/video/ExpOpenBorder.avi and http://webuser.unicas.it/lai/robotica/video/ExpClosedBorder.avi.

J Intell Robot Syst

been chosen equal to 0.1 m. This value preventssaturation of the actuators and ensures a controlpoint located inside the robot chassis.

The Player/Stage environment [39] has beenused to implement the proposed architecture.Player/Stage is based on a client-server paradigm;it provides a set of tools to simulate the sys-tem (Stage), as well as the software primitivesfor communicating with the robotics hardware(Player). In Fig. 11 (left) a portion of the patrolledarea is shown, together with its representation inPlayer/Stage (right); in addition, an open (top)and closed (bottom) borders have been consid-ered. The blue line represents the border, the red,green and cyan polygons represent the robots,together with their visibility ranges (assumed tobe equal to the corresponding safety areas). Theopen border (top) is 10 m long; the closed border(bottom), composed by segments joined by arcs,has an overall length of 51 m. The results in thefollowing refer to the closed border; nevertheless,similar results have been obtained for the openborder. The robots know the exact description ofthe border and they approach it at a speed of0.3 m/s (λr f = 0.3), patrol at a speed of 0.35 m/s(λcw = λccw = 0.35) and escape other teammates

at a speed of 0.3 m/s (λta = 0.3). Localization inthe environment is achieved via a pre-built mapand a localization driver based on an adaptiveparticle filter [20], available within the Playersoftware. The visibility range and safety area areequal to 2.5 m, the threshold value ζFS for thetransition between macro-states MS2 and MS3 is0.6 m. Moreover, if a robot is in the patrollingstate, every 30 s it can decide to invert its motiondirection or to keep going in the same one, accord-ing to a random variable.

Figure 12 (top) shows the time history ofthe distance from the border for the three ro-bots. In normal operating conditions (i.e., in theabsence of faults), the mutual distances are within0.1 m, this value is acceptable for the experimentalconditions and assumed requirements. Moreover,peaks are reached during rotation movements dueto sensor noise and, above all, to the neglectedrobot dynamics in the control law.

When a fault occurs to robot number 2, after28 min from the mission start, the robot is manu-ally driven far from the border and is reactivatedat minute 41. Two faults occur to robot number 3,after 15 and 33 min from the mission start. As canbe noticed in Fig. 12, thanks to the decentralized

Fig. 11 Left: a portion ofthe environment. Right:environmentrepresentation inPlayer/Stage, the pathand the threepatrolling robots

J Intell Robot Syst

0 2 4 6 8 10 12 14 160

5

10

15

20

25

30

35

40

45

50

Distance between consectuvise robot

t [min]

m

Desired valueRob.1 and Rob.2Rob.2 and Rob.3Rob.1 and Rob.3Width of Safety area

0 10 20 30 40−0.2

00.20.4

t [min]

mTracking error of the border

Robot1

0 10 20 30 40−0.2

00.20.4

t [min]

m

Robot2

0 10 20 30 40−0.2

00.20.4

t [min]

m

Robot3

Fig. 12 Top: distance from the border. Bottom: distancebetween two consecutive robots

structure of the control law, faults do not affectthe overall behavior of the swarm. For the sakeof clarity, it is worth showing the time historyof the distance between two consecutive robots(Fig. 12, (bottom)).

9.1 Discussion

In addition to intensive tests based on numericalsimulation, the approach has been also validatedby performing experiments with 3 robots. Theexperiments, even if involving a limited numberof robots, have been conducted without particularshortcuts (e.g., an ad-hoc localization method).The faults have been emulated by turning on/offthe robots during the patrolling of the remainingones. The experiments’ duration has been limited

only by battery life. The achieved results clearlyshow that the proposed approach is also practi-cally viable.

According to Chevaleyre [13], at each time in-stant, the distance between two consecutive ro-bots should be one third of the overall borderlength; moreover, all robots should move in thesame direction. On the contrary, according toAgmon et al. [1], robots should move synchro-nously but in a nondeterministic way, so as tomaximize the probability of intercepting an in-truder. Both cases require a centralized supervisoror information exchanges between robots, so itis straightforward to imagine that the proposedapproach can be characterized by better perfor-mance only by increasing the number of vehiclesor by removing some constraints.

10 Conclusion

A higher layer above the traditional Null-Space-Behavioral (NSB) approach has been designed.At the basis of the developed architecture, thereis the concept of action, obtained by combin-ing elementary behaviors in the NSB framework.Once the set of actions has been defined, a Su-pervisor can be designed that chooses the actionto be executed. The developed architecture hasbeen applied to the challenging border patrollingmission. To this aim, a set of constraints and re-quirements have been formulated and the missionachieved thanks to the designed architecture. Thealgorithm is fully scalable; the computational load,in fact, is simply independent of the number of ro-bots as no communication occurs between robotsand the control strategy of each robot does notdepend on the number of robots. The robustness,in a wide sense, has been designed by avoiding theneed for a central computational unit or dead-lockcommunication among the robots. This intrinsicrobustness of the approach has been confirmedby simulation and experimental results. Futurework will be devoted to the application of thedeveloped architecture to different missions, e.g.,coverage and flocking, by properly redefining thebehavior and action sets. Moreover, it would be ofutmost importance to test the approach by adopt-ing different classes of vehicles (e.g., unmanned

J Intell Robot Syst

underwater vehicles and/or unmanned aerial ve-hicles) and in the case of heterogeneous teamsof robots.

Comparison of control strategies for unstruc-tured robotics is an open issue in the robot-ics community. The European network EURONhas a specific committee [18] and the AmericanNIST (National Institute of Standards and tech-nologies) has a specific annual conference on it(PERMIS, Performance Metrics for IntelligentSystems Workshop: Permis [34]). Hence, futurework might be devoted at investigating compari-son metrics in this compelling case study.

Appendix

A.1 Elementary Behaviors

In the following the elementary behaviors for thepatrolling mission are defined and described in theframework of the NSB. Definition of behaviorsdepends, of course, on the mission to be exe-cuted. In this paper, therefore, they will be deter-mined by identifying the elementary componentsrequired by patrolling missions.

A.1.1 Reach Frontier

This behavior pushes the robot towards the bor-der to be patrolled (Fig. 13).

Then, given the robot position pr ∈ �2 and theborder B, pB ∈ �2 is the closest point to pr be-longing to B. The behavior is then encoded by thefunction σr f

⎧⎪⎪⎨

⎪⎪⎩

σr f = ‖ pr − pB‖, σr f,d = 0,

Jr f = rTr f , J†

r f = rr f , Nr f = I2 − rr f rTr f ,

vr f = −λr f rr f σr f ,

(10)

where σr f,d is the desired value of the task func-tion, rr f = (

pr − pB

)/‖ pr − pB‖, Jr f is the task

Jacobian, I2 is the (2 × 2) identity matrix, Nr f is

Fig. 13 Graphicalrepresentation ofthe ReachFrontier Behavior

the null-space projection matrix, λr f is a positivescalar gain and vr f is the commanded velocity.

It is worth noticing that the computation ofthe point on the border closest to the robot, pB,is needed. Hence, a discretization of the borderand/or a proper analytical approximation of itsshape [1] are required.

Of course, boundary detection and boundarytracking capabilities are required to accomplishthis behavior. However, such problems are behindthe scope of this paper and were widely discussedin other works (e.g., [22, 24, 40]).

A.1.2 Patrol Frontier Clockwise(Counter-Clockwise)

Once the robot is close to the border, it shouldmove according to the border shape. To this aim,given the border B and a point pB belongingto B, rcw is the unit vector tangent to the bor-der in pB and oriented in the clockwise direc-tion of the border. The behavior is, then, directlydefined as:⎧⎪⎪⎨

⎪⎪⎩

vcw = λcwrcw,

Jcw = rTcw

Ncw = I2 − rcwrTcw,

(11)

where vcw is the velocity vector encoding the be-havior, rT

cw plays the role of the task Jacobian,Ncw is the null-space projection matrix and λcw

is a positive scalar gain. It is worth noticing thatthe behavior simply commands the robot to movealong the direction determined by the border tan-gent (Fig. 14), no matter what the robot posi-tion is (along the border or out of the border).If the vector tangent to the border is orientedin the counter–clockwise direction we can definePatrol Frontier Counter-Clockwise be-havior (by replacing λcw and rcw with λccw and rccw

in Eq. 11, vccw and Nccw are easily obtained).

Fig. 14 Graphicalrepresentation ofthe PatrolClockwise Behavior

J Intell Robot Syst

A.1.3 Teammate Avoidance

In order to avoid collisions between robots, asuitable behavior has to be defined. Let pr, bethe robot position, pt the position of the closestteammate to the robot and dt the safety distance,i.e., the radius of the circular safety area. Thebehavior is, then, defined as follows:⎧⎪⎪⎨

⎪⎪⎩

σta = ‖ pr − pt‖, σta,d = dt,

Jta = rTta, J†

ta = rta, N ta = I2 − rtarTta,

vta = λtarta (dt − σta) ,

(12)

where σta,d denotes the desired value of the be-havior function, rta = (

pr − pt

)/‖ pr − pt‖, Jta is

the task Jacobian, N ta is the null-space projectionmatrix, λta is a positive scalar gain and vta is thecommanded robot velocity.

Hence, this behavior allows the robot to keepthe other teammates out of the border of its ownsafety area (Fig. 15).

A.1.4 Friend Avoidance

A friend is an agent that moves independentlyfrom patrolling robots and that is allowed to crossthe border. Therefore, when a friend tries to crossthe border, patrolling robots should keep a de-sired distance from it, without affecting its mo-tion. This behavior is similar to the TeammateAvoidance behavior; then, given the robot po-sition, pr, the friend position, p f , and a safetydistance d f , the behavior is defined as follows

⎧⎪⎪⎨

⎪⎪⎩

σ f a = ‖ pr − p f ‖, σ f a,d = d f ,

J f a = rTf a, J†

f a = r f a, N f a = I2 − r f arTf a,

v f a = λ f ar f a(d f − σ f a

),

(13)

Fig. 15 Graphicalrepresentation of theTeammate AvoidanceBehavior. The dashedline represents the borderof the safety area

where σ f a,d denotes the desired value of the be-havior function, λ f a is a positive scalar gain, N f a

is the null-space projector matrix and v f a is thecommanded robot velocity. Similarly to the previ-ous case, this behavior, keeping the robot far fromthe friend agent, allows it to move without beingaffected by patrolling agents.

A.2 Actions

The above defined behaviors are not yet suitableto accomplish the patrolling mission, since theyonly encode simple atomic tasks. For example,the Reach Frontier behavior allows the ro-bot to reach the border, while the two PatrolFrontier behaviors only allow the robot tomove in a direction tangent to the border, butnot necessarily on the border. Then, once the setof actions is defined, it is necessary to combinethem to produce a set of more complex and mean-ingful actions. This combination is obtained viaNSB, in order to obtain a predictable output; thus,priorities among the elementary behaviors are tobe assigned.

A.2.1 Action Reach Frontier

This action allows the robot to reach the bor-der, e.g., when it is far from it. In this case, thedefinition of the action simply coincides with theelementary behavior Reach Frontier:

vAr f = vr f . (14)

This action is useful when the robot is particularlyfar from the border. In such a situation, the onlyobjective of the robot is to reach the frontier be-fore starting other behaviors; hence, no secondarytasks are present.

A.2.2 Actions Patrol Clockwise(Counter-Clockwise)

This action allows the robot to stay on the border,while covering it in the clockwise direction. To thisaim, two behaviors are needed. The primary be-havior is Reach Frontier, while the secondary

J Intell Robot Syst

one is Patrol Frontier Clockwise, leadingto

vApcw = vr f + Nr f vcw. (15)

By activating this action, the robot is able to reachor keep on the border while patrolling it. As willbe shown in Section 5.3, since the two elementarybehaviors composing the action are compatible inthe NSB sense, the priority of behaviors can beexchanged. In Fig. 16, a typical robot motion un-der the effect of Action Patrol Clockwiseis shown.

While, in the case of Action PatrolCounter-Clockwise the action is obtainedconsidering the Patrol Frontier Counter-Clockwise behavior as secondary behavior.With regards the compatibility in the NSB senseof the defined behaviors, it easy to show that thecompatibility condition [3] Jr f J†

cw = 0 ( Jr f J†ccw =

0) globally holds.

A.2.3 Action Keep Going

This action allows the robot to stay on the border,while covering it in the clockwise or counter–clockwise direction. This action is obtainedby combining the Reach Frontier and thePatrol Frontier Clockwise (or PatrolFrontier Counter-Clockwise) behaviors inthe NSB sense, i.e.,

vAkg = vr f + Nr f v p, (16)

where v p is the vector tangent to the border at theclosest point belonging to the border. The valueof the velocity vector v p is decided according tosome criteria. For example, it can be varied, de-pending on the value of a random variable, everyT seconds and set equal to vcw or vccw. This can beuseful for conferring some unpredictability to the

Fig. 16 Typical robotmotion under the effectof Action PatrolClockwise

mission. The compatibility of the two behaviorscan be demonstrated via the same arguments usedfor the previous action.

A.2.4 Action Teammate Avoidance

A teammate entering the safety area of a robotmust be avoided, while the robot tries to stay onthe border or to reach it; in this way, it can restartthe patrol mission once the teammate vehicle isfar enough. This action can be obtained by com-bining the behaviors Teammate Avoidance andReach Frontier in the NSB sense, i.e.,

vAta = vta + N tavr f . (17)

In Fig. 17, a typical path generated by this actionis shown.

Differently from the previous action, it is notpossible to ensure task compatibility in all situ-ations. In fact, it is not possible to ensure thatthe compatibility condition Jr f J†

ta = 0 does nothold in general. Only the velocity components ofthe secondary behavior that do not conflict withthe primary behavior will be executed, leadingto the robot moving on the border of the team-mate safety area (Fig. 17).

A.2.5 Action Friend Avoidance

This action is similar to Action TeammateAvoidance. In the case a friend vehicle entersthe safety area of a robot, the friend must beavoided, while trying to stay on the border, so thatit can restart the patrol mission once the friendis far enough. This action can be obtained by

Fig. 17 Typical robot motion under the effect of ActionTeammate Avoidance; pr is the position of the robot, ptis the position of the teammate closest to the robot, and pBis the point on the border closest to the robot

J Intell Robot Syst

combining the Friend Avoidance and ReachFrontier behaviors

vAfa = v f a + N f avr f . (18)

Since Reach Frontier is the secondary be-havior, only its velocity components that do notconflict with the primary task will be executed.Also in this case, as in the previous one, it is notpossible to ensure that Jr f J†

f a = 0 holds in anycondition; nevertheless, it is guaranteed that nocollision happens being the Friend Avoidancethe highest priority behavior.

References

1. Agmon, N., Kraus, S., Kaminka, G.A.: Multi-robotperimeter patrol in adversarial settings. In: Proceedings2008 IEEE International Conference on Robotics andAutomation. Pasadena, CA, pp. 2339–2345 (2008)

2. Alur, R., Yannakakis, M.: Model checking of hierarchi-cal state machines. ACM Trans. Program. Lang. Syst.23(3), 273–303 (2001)

3. Antonelli, G.: Stability analysis for prioritized closed-loop inverse kinematic algorithms for redundant ro-botic systems. IEEE Trans. Robot. 25(5), 985–994(2009)

4. Antonelli, G., Chiaverini, S.: Kinematic control of pla-toons of autonomous vehicles. IEEE Trans. Robot.22(6), 1285–1292 (2006)

5. Antonelli, G., Arrichiello, F., Chiaverini, S.: The null-space-based behavioral control for autonomous roboticsystems. J. Intell. Serv. Robot. 1(1), 27–39 (2008)

6. Antonelli, G., Arrichiello, F., Chiaverini, S.: Exper-iments of formation control with multirobot sys-tems using the null-space-based behavioral control.IEEE Trans. Control Syst. Technol. 17(5), 1173–1182(2009)

7. Arkin, R.: Motor schema based mobile robot naviga-tion. Int. J. Rob. Res. 8(4), 92–112 (1989)

8. Bertozzi, A.L., Kemp, M., Marthaler, D.: Determiningenvironmental boundaries: asynchronous communica-tion and physical scales. In: Proceedings of the BlockIsland Workshop on Cooperative Control, vol. 309,pp. 25–42 (2004)

9. Brooks, R.: A robust layered control system for amobile robot. IEEE J. Robot. Autom. 2(1), 14–23(1986)

10. Bruemmer, D.J., Dudenhoeffer, D., Anderson, M.O.,McKay, M.D.: A robotic swarm for spill finding andperimeter formation. In: Spectrum 2002: InternationalConference on Nuclear and Hazardous Waste Manage-ment, Reno, Nevada, USA (2002)

11. Bryson, J.J.: Action selection and individuation inagent based modelling. In: Proceedings of Agent 2003:Challenges of Social Simulation, pp. 317–330 (2003)

12. Casbeer, D.W., Kingston, D.B., Beard, A.W., Mclain,T.W., Li, S., Mehra, R.: Cooperative forest fire sur-veillance using a team of small unmanned air vehicles.Int. J. Syst. Sci. 37, 360 (2006)

13. Chevaleyre, Y.: Theoretical analysis of the multi-agent patrolling problem. In: Procedings of theIEEE/WIC/ACM International Conference on Intelli-gent Agent Technology, Beijing, China, pp. 302–308,20–24 Sept 2004

14. Chiaverini, S.: Singularity-robust task-priority redun-dancy resolution for real-time kinematic control of ro-bot manipulators. IEEE Trans. Robot. Autom. 13(3),398–410 (1997)

15. Clarka, J., Fierro, R.: Mobile robotic sensors forperimeter detection and tracking. ISA Trans. 28, 3–13(2007)

16. Dorigo, M., Sahin, E.: Swarm robotics—special issueeditorial. Auton. Robots 17, 115–147 (2004)

17. Dudek, G., Jenkin, E., Wilkes, D.: A taxonomy forswarm robots. In: In Proceedings of 1993 IEEE In-ternational Conference on Intelligent Robots and Sys-tems, pp. 441–447 (2003)

18. Euron: Special interest group in good experimentalmethodology and benchmarking (2008). http://www.euron.org/activities/benchmarks/index. Accessed 6May 2009

19. Everett, H.R., Laird, R.T., Gilbreath, G., Heath-Pastore, T.A., Inderieden, R.S., Grant, K., Jaffee,D.M.: Multiple resource host architecture for the mo-bile detection assessment and response system. In:Technical Document 3026, Space and Naval WarfareSystems (1998)

20. Fox, D., Burgard, W., Dellaert, F., Thrun, S.: Montecarlo localization: efficient position estimation for mo-bile robots. In: Proceedings of the National Conferenceon Artificial Intelligence, pp. 343–349 (1999)

21. GNIUS: Autonomous unmanned ground vehicles(2005). http://www.defense-update.com/products/g/guardium. Accessed 6 Dec 2009

22. Hsieh, M.A., Loizou, S.G., Kumar, V.: Stabilization ofmultiple robots on stable orbits via local sensing. In:ICRA, Rome, Italy, pp. 2312–2317 (2007)

23. Inderieden, R., Everett, H., Heath-Pastore, T., Smurlo,R.: Overview of the mobile detection assessment andresponse system. In: DND/CSA Robotics and KBSWorkshop, St. Hubert, Quebec (1995)

24. Kalantar, S., Zimmer, U.R.: Distributed shape controlof homogeneous swarms of autonomous underwatervehicles. Auton. Robots 22, 37–53 (2007)

25. Kingston, D., Beard, R., Holt, R.S.: Decentralizedperimeter surveillance using a team of UAVs. IEEETrans. Robot. 24(6), 1394–1404 (2008)

26. Machado, A., Ramalho, G., Zucker, J., Drogoul, A.:Multi-agent patrolling: an empirical analysis of alterna-tive architectures. In: Multi-Agent Based Simulation,pp. 155–170 (2002)

27. Marino, A., Parker, L., Antonelli, G., Caccavale, F.:Behavioral control for multi-robot perimeter patrol:a finite state automata approach. In: Proceedings2009 IEEE International Conference on Robotics andAutomation. Kobe, J. (2009)

J Intell Robot Syst

28. Marino, A., Parker, L., Antonelli, G., Caccavale, F.: Afault-tolerant modular control approach to multi-robotperimeter patrol. In: IEEE International Conferenceon Robotics and Biomimetics (ROBIO 2009), Guilin,China (2009)

29. Marino, A., Parker, L., Antonelli, G., Caccavale,F.: A fuzzy-based multiple robots border patrol. In:Proceedings 2009 17th Mediterranean Conference onControl and Automation, Thessaloniki, Greece (2009)

30. Matlab: http://www.mathworks.com (2010). Accessed 11Aug 2010

31. Murphy, R.R.: Introduction to AI Robotics, 1st edn.MIT Press, Cambridge (2000)

32. Nakamura, Y., Hanafusa, H., Yoshikawa, T.: Task-priority based redundancy control of robot manipula-tors. Int. J. Rob. Res. 6(2), 3–15 (1987)

33. Ouimet, M., Lundqvist, K.: Automated verification ofcompleteness and consistency of abstract state machinespecifications using a SAT solver. In: Electronic Notesin Theoretical Computer Science. Proceedings of theThird Workshop on Model Based Testing (MBT 2007),vol. 190(2), pp. 85–97 (2007)

34. Permis: Performance metrics for intelligent systemsworkshop (2004) www.isd.mel.nist.gov/research_

areas/research_engineering/Performance_Metrics/past_wkshp.html. Accessed 8 July 2009

35. Pletta, J.B., Sackos, J.: An advanced unmanned vehiclefor remote applications. Sandia National LaboratoryReport (1998)

36. Rafael Armament Development Authority Ltd:http://www.rafael.co.il (2006). Accessed 3 Feb2009

37. Siciliano, B.: Kinematic control of redundant robot ma-nipulators: a tutorial. J. Intell. Rob. Syst. 3(3), 201–212(1990)

38. Thrun, S., Leonard, J.J.: Springer handbook of ro-botics. In: Siciliano, B., Khatib, O. (eds.), Simultane-ous Localization and Mapping, Ch. 37, pp. 871–889Springer, Heidelberg, Germany (2008)

39. Vaughan, R.T., Gerkey, B., Howard, A.: The player/stage project: tools for multi-robot and distributed sen-sor systems. In: Proceedings of the 11th InternationalConference on Advanced Robotics, (ICAR), Coimbra,Portugal, pp. 317–323 (2003)

40. Zhang, F., Haq, S.: Boundary following by robot for-mations without gps. In: Proceedings of 2008 Interna-tional Conf. on Robotics and Automation, Pasedena,CA, pp. 152–157 (2008)