TRANSFORMING HIGH LEVEL TASKS TO LOW LEVEL

CONTROLLERS

Hadas Kress-Gazit

A DISSERTATION

in

Electrical and Systems Engineering

Presented to the Faculties of the University of Pennsylvania

in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy

2008

Supervisor of Dissertation

Graduate Group Chair

To my family

ii

Acknowledgements

I like reading acknowledgments. I like the personal aspect of it, the humor and private

jokes that usually filter in and most of all the feelings of accomplishment and gratitude

that are inherent in the situation. And now it is my turn to write one.

I’m done! five years that took forever yet passed by in a heart beat. Five years of

tremendous professional and personal growth, of discovering “what I want to be when I

grow up”.

I could not have done it without the support of George Pappas, my advisor. His en-

abling attitude of “sure, go ahead and do that”, his open door “yeah, of course I have

time to talk” and his invaluable professional advice on so many aspects of academia and

science shaped the researcher I have become. Not to mention the numerous restaurant

recommendations, around the world...

I’d like to thank my committee members, Dan Koditschek, Vijay Kumar and Lydia

Kavraki for their insightful comments during my proposal, which eventually led to a

stronger job talk and thesis. I thank them for their comments regarding this thesis and

defense and for their support and enthusiasm towards the problems addressed in this work.

I have been fortunate to collaborate academically with some of my friends. Working,

as well as hanging out, with Georgios Fainekos has been a lot of fun, I think our strengths

complement each other. David Conner, my long time cross state collaborator from CMU,

our personalities definitely complement each other. It has been good for me to be a little

iii

more deep and thorough and it has been a fruitful and very enjoyable collaboration. I’d also

like to thank Howie Choset and Al Rizzi, David’s advisors, for supporting and commenting

on this work. The last paper to make it into this thesis was written with Nora Ayanian.

Apart from being a good friend, her ability to think in an insane number of dimensions led

to some fun, funny and interesting work. LTLMop was written with Cameron Finucane,

the undercover Japanese Hide N’ Seeker. It is always fun working with someone who has

a very unique view of life and is a bit intimidated by you...

The Israeli connection (or rather synthesis connection?); Many many thanks to Nir

Piterman, Yaniv Sa’ar and Amir Pnueli for their help, synthesis code, time and just good

spirits. LTL synthesis and robotics? why not...

Many thanks also to Richard Murray and Tichakorn Wongpiromsarn (AKA Nok) from

Caltech for Urban challenge discussions and a very memorable ride in Alice.

Having a good or bad grad school experience is tied with the people that surround

you. I have enjoyed spending my time in the GRASP lab with many friends from the lab

and from the department. Special thanks go (in no particular order) to Michael, Georgios,

Brian, Ani, Dave, Nora, Ben, Adam and Agung. My work life (and lunches) would have

been really lonely without you.

Last, and by no means least, I would like to thank my family. I cannot sum up in a few

sentences the love, support and advice they have given me over the years so I will mention

some other things I am so grateful for. A child needs the approval of her parents. Mine

have continuously made me feel that they are proud of me; while this remark is a bit self

centered, it speaks loudly of the environment in which I was (and in a sense still am) raised.

A poem my father, Moshe, wrote me for my high school graduation can still bring tears to

my eyes. My mother, Nurit, showed me in actions rather than words what being a strong

woman means. Anyone annoyed by my strong opinions regarding gender can blame her.

iv

My sisters, Naama and Noa, each so talented in her field, are a great example of sisterly

bonds and the list goes on (rather large, happy, noisy, loving, slightly crazy family).

Mika was born in the middle of my PhD program. I cannot express the love, joy,

amazement, change of perspective and fear that comes with having a daughter. Now, a

little over three years old, I can see there is such a thing as karma. We are raising a very

opinionated child. I guess I deserve it1.

Finally, Liran, my partner in life. Mua.

1I really wouldn’t want it any other way.

v

ABSTRACT

TRANSFORMING HIGH LEVEL TASKS TO LOW LEVEL CONTROLLERS

Hadas Kress-Gazit

Supervisor: George J. Pappas

A major goal in robotics is to develop machines that perform useful tasks with minimal su-

pervision. Instead of requiring each small detail to be specified, one would like to describe

the task at a high level, and have the system autonomously execute it correctly.

The work presented in this thesis addresses some of the challenges of creating au-

tonomous systems that perform complex tasks while providing guarantees for correct be-

havior. Here, tasks defined using the formalism of temporal logic are automatically trans-

lated into controllers that drive a robot or group of robots and activate the different robot

actions such that the task is completed, if it is feasible. The approach presented is cor-

rect by construction, it is complete in the sense that if under specific assumptions the task

can be done, a controller will be created and it is sound, that is, if the task cannot be

accomplished the algorithms return that it is infeasible and no controller is created.

The approach to generating such controllers comprises of three steps. First, the task

itself is described; This description captures the required behavior as well as relevant in-

formation about the robot capabilities such as actions and sensors, and the environment

in which the robot is operating. This step transforms a generally continuous problem into

a discrete one by creating abstractions of the different task components. The next step is

to find a discrete solution for the problem, a discrete plan, that if followed the robot is

guaranteed to achieve the discrete behavior required of it. The final step is to return to the

original continuous problem by transforming the discrete plan to a hybrid controller that

switches between a set of atomic controllers or control primitives that provide continuous

local guarantees for the behavior of the robot.

vi

Contents

1 Introduction 1

1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.3 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

1.4 Thesis Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 Non reactive tasks 11

2.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.1.1 Linear Temporal Logic . . . . . . . . . . . . . . . . . . . . . . . 11

2.1.2 Model Checking . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.1.3 Bisimulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.2 Problem formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.2.1 Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.2.2 Workspace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.2.3 Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

2.3 Generating non reactive controllers . . . . . . . . . . . . . . . . . . . . . 16

2.3.1 Discrete Abstraction of Robot Motion and Action . . . . . . . . . 16

2.3.2 Temporal Logic Planning using Model Checking . . . . . . . . . 22

2.3.3 Continuous Implementation of Discrete Trajectory . . . . . . . . 23

vii

2.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

2.5.1 Guarantees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

2.5.2 Optimality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

2.5.3 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

2.5.4 Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

2.5.5 Choosing the right partition . . . . . . . . . . . . . . . . . . . . 35

3 Reactive tasks 36

3.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.1.1 Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.2 Problem formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.2.1 Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3.2.2 Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3.2.3 Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3.3 Generating reactive controllers . . . . . . . . . . . . . . . . . . . . . . . 41

3.3.1 Automaton synthesis . . . . . . . . . . . . . . . . . . . . . . . . 41

3.3.2 Hybrid Controller . . . . . . . . . . . . . . . . . . . . . . . . . . 44

3.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

3.4.1 Nursery scenario . . . . . . . . . . . . . . . . . . . . . . . . . . 47

3.4.2 Animal Herding . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.4.3 Visiting Regions in Sequence . . . . . . . . . . . . . . . . . . . 52

3.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

3.5.1 Guarantees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

3.5.2 Optimality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

3.5.3 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

viii

3.5.4 Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

4 Complex Dynamics 59

4.1 Discrete Abstraction of Motion . . . . . . . . . . . . . . . . . . . . . . . 59

4.1.1 Controllers for complex dynamics . . . . . . . . . . . . . . . . . 59

4.1.2 Robust temporal logic . . . . . . . . . . . . . . . . . . . . . . . 60

4.2 Example - Parking Car . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

4.2.1 Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

4.2.2 Atomic Controllers . . . . . . . . . . . . . . . . . . . . . . . . . 64

4.2.3 Discrete abstraction of motion . . . . . . . . . . . . . . . . . . . 65

4.2.4 Parking formula . . . . . . . . . . . . . . . . . . . . . . . . . . 67

4.2.5 Continuous execution of discrete automata . . . . . . . . . . . . 68

4.2.6 Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

5 Language 71

5.1 Grammar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

5.2 Initial conditions - EnvInit, RobotInit . . . . . . . . . . . . . . . . . . . . 73

5.3 Liveness - EnvLiveness, RobotLiveness, RobotGoStay . . . . . . . . . . . 74

5.4 Safety - EnvSafety, RobotSafety . . . . . . . . . . . . . . . . . . . . . . . 74

5.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

5.5.1 No Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

5.5.2 Sensors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

6 Multi robot 81

6.1 Non-reactive Model-checking Approach . . . . . . . . . . . . . . . . . . 81

6.2 Decentralized . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

6.2.1 Adhering to traffic laws . . . . . . . . . . . . . . . . . . . . . . . 84

ix

6.2.2 Parking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

6.2.3 Leaving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

6.2.4 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

6.3 Centralized . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

6.3.1 Centralized Atomic Controller . . . . . . . . . . . . . . . . . . . 90

6.3.2 Recycling examples . . . . . . . . . . . . . . . . . . . . . . . . 91

6.3.3 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

6.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

6.4.1 Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

6.4.2 Global guarantees - liveness of the system . . . . . . . . . . . . . 100

7 Case study: DARPA’s Urban Challenge 102

7.1 The DARPA Urban Challenge . . . . . . . . . . . . . . . . . . . . . . . 103

7.1.1 The Challenge . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

7.1.2 The general solutions . . . . . . . . . . . . . . . . . . . . . . . . 104

7.1.3 Synthesizing the planner and controller . . . . . . . . . . . . . . 104

7.2 The synthesized system . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

7.2.1 Subsystem I - Traffic Behavior . . . . . . . . . . . . . . . . . . . 107

7.2.2 Subsystem II - Driving control . . . . . . . . . . . . . . . . . . . 112

7.3 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

8 LTLMop 119

8.1 Region Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

8.2 Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

8.3 Compilation and Automaton Visualization . . . . . . . . . . . . . . . . . 126

8.4 Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

x

9 Conclusions and future work 134

9.1 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

Appendices 137

A NuSmv input and output 137

B Context Free Grammar 149

C LTLMop files 151

Bibliography 156

xi

List of Figures

1.1 OmniTread . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Robotic Space Mission classes2, as of 2005 . . . . . . . . . . . . . . . . 3

1.3 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1 Workspace for Example 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.2 Partitioned workspace for Example 1 . . . . . . . . . . . . . . . . . . . . 16

2.3 Discrete system representing the robot motion . . . . . . . . . . . . . . . 19

2.4 Discrete systems representing the robot action . . . . . . . . . . . . . . . 20

2.5 Discrete robot model D . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.6 Close up of Discrete robot model D . . . . . . . . . . . . . . . . . . . . 21

2.7 Velocity vector fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.8 Simulation of Example 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.9 Graphical User Interface . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.10 Example 2: Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.11 Example 3: Path generated by NUSMV . . . . . . . . . . . . . . . . . . 29

2.12 Example 3: Path generated by SPIN . . . . . . . . . . . . . . . . . . . . 30

2.13 Example 4: Complex environment - Visit the two black areas . . . . . . . 31

2.14 Example 5: go to Room 3 (avoid 1,2), then go to Room 4 (avoid 1,2),then

go to Room 1 (avoid 2) and finally go to Room 2 . . . . . . . . . . . . . 32

xii

3.1 The workspace of Example 6. . . . . . . . . . . . . . . . . . . . . . . . 39

3.2 The synthesized automaton of Example 6 . . . . . . . . . . . . . . . . . 43

3.3 A possible run of the automaton of Example 6 . . . . . . . . . . . . . . 46

3.4 Nursery Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.5 Animal herding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

3.6 Visiting regions in sequence . . . . . . . . . . . . . . . . . . . . . . . . 53

4.1 Example 10 - robust LTL . . . . . . . . . . . . . . . . . . . . . . . . . . 62

4.2 Workspace for Section 4.2 . . . . . . . . . . . . . . . . . . . . . . . . . 63

4.3 Car-like system with Ackermann Steering. . . . . . . . . . . . . . . . . . 64

4.4 Control policy based on [11] . . . . . . . . . . . . . . . . . . . . . . . . 65

4.5 Parking behavior induced by the composition of atomic controllers. . . . 66

4.6 Four executions of the basic scenario . . . . . . . . . . . . . . . . . . . . 70

5.1 Simulation for the Visit and Beep example . . . . . . . . . . . . . . . . . 77

5.2 Simulation for the Search and Rescue I example . . . . . . . . . . . . . . 79

5.3 Simulation for the Search and Rescue II example . . . . . . . . . . . . . 80

6.1 Workspace for decentralized multi robot example . . . . . . . . . . . . . 83

6.2 A snapshot of the simulation. . . . . . . . . . . . . . . . . . . . . . . . . 87

6.3 Close up looks at different behaviors seen throughout the simulation . . . 89

6.4 Workspace for recycling examples . . . . . . . . . . . . . . . . . . . . . 91

6.5 Part of the automaton that satisfies Example 15 . . . . . . . . . . . . . . 95

6.6 Simulation of the automaton segment of Figure 6.5 . . . . . . . . . . . . 96

6.7 Simulation of Example 15 . . . . . . . . . . . . . . . . . . . . . . . . . 98

7.1 Site of the NQE and final challenge . . . . . . . . . . . . . . . . . . . . 103

xiii

7.2 A General high level description of the planning and control architecture

of various DARPA Challenge teams . . . . . . . . . . . . . . . . . . . . 105

7.3 Synthesized planning and control system . . . . . . . . . . . . . . . . . . 106

7.4 Synthesized automaton for the Estop example . . . . . . . . . . . . . . . 108

7.5 Close up of areas A and C of the NQE . . . . . . . . . . . . . . . . . . . 115

7.6 Behavior while encountering an obstacle . . . . . . . . . . . . . . . . . . 116

7.7 Encountering a temporarily blocked road . . . . . . . . . . . . . . . . . . 117

7.8 Estop issued . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

7.9 4-way stop behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

8.1 Specification Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

8.2 Region Editor displaying a workspace and its partition . . . . . . . . . . 122

8.3 Region Editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

8.4 Writing specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

8.5 Compilation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

8.6 Compilation Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

8.7 Automaton is visualized using dotty [1] . . . . . . . . . . . . . . . . . . 129

8.8 Simulation Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 131

8.9 Simulation using Stage . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

8.10 Simulation using Gazebo . . . . . . . . . . . . . . . . . . . . . . . . . . 133

A.1 Input to NuSmv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

A.2 Output of NuSmv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

C.1 Robot file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

C.2 Specification file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154

xiv

Chapter 1

Introduction

1.1 OverviewRobots these days, and mechanical systems in general, can accomplish difficult and me-chanically challenging feats such as traversing rough terrain [101], climbing up betweentwo walls [105], running [54] and much more. However, in terms of autonomy and con-trol, the robotics world is far from the vision of C-3PO. These days, most robots are eitherprogrammed to execute a specific task or remote controlled by an expert. For example,the OmniTread [22] shown in Figure 1.1, can climb up pipes and fit through small holes,however to accomplish these tasks it is remote controlled by three trained people simulta-neously.

For programmed robots, if a different task is needed then the robot must be repro-grammed, usually by an engineer that designed the system. This process can be timeconsuming, expensive and error prone, furthermore it requires a great deal of effort to beinvested in the verification or validation of the system. As robots become an integral partof the workplace and the home, some of the challenges facing robotics researchers is howto make these machines more autonomous, how to design control paradigms that are flexi-ble thus allowing tasks to be easily changed and how to provide guarantees of correctnessfor the system’s behavior.

The Defense Advanced Research Projects Agency (DARPA) 2007 Urban Challengeinvited researchers interested in autonomous robots to design an autonomous vehicle fora complex urban environment - “this event required teams to build an autonomous vehiclecapable of driving in traffic, performing complex maneuvers such as merging, passing,parking and negotiating intersections. This event was truly groundbreaking as the firsttime autonomous vehicles have interacted with both manned and unmanned vehicle trafficin an urban environment.1” Most of the teams participating in this challenge had complex,modular and partially hand written subsystems that were in charge of the different tasksrelating to this challenge, such as sensing, estimation, planning and control. The lackof automated tools for designing systems that guarantee high level behaviors led teams

1http://www.darpa.mil/grandchallenge/

1

Figure 1.1: OmniTreadOne of the operators can be seen in the back.

to spend a lot of time testing, redesigning and testing again to make sure the changesdid not ruin the previously checked behaviors. Furthermore, these systems did not haveany guarantees in terms of correct behavior. Team Caltech for example, were disqualifiedpartially due to unsafe intersection behavior. The situation leading to that behavior was notanticipated by the system designers and therefore was not simulated. This challenge whichonly six teams were able to complete, was a good example for the need for verification andvalidation techniques for complex autonomous systems as well as automated tools that cancreate correct by construction systems from high level descriptions.

Space exploration is a natural setting for the use of robots due to the distance and theharsh conditions that such missions entail but even the sophisticated robots sent to Marslacked autonomy that could have saved communication and valuable mission time. Theearlier Mars rovers spent most of their time waiting to be told what to do - “The Sojournerrover spent between 40% and 75% of its time awaiting instructions. Moving to a nearbytarget and taking a reading required an average of 2.4 Mars days.2.” While these days theMars rovers employ advanced navigation and obstacle avoidance algorithms, their missionis still to travel between two given locations, “When the rovers are navigating themselves,they get a command telling them where to end up...”3

The table presented in Figure 1.2 published by NASA2 in 2005, while somewhat out-dated, states that robotic space missions that involve intensive and global exploration withlimited and intermittent communications cannot be done using current technology.

2http://ti.arc.nasa.gov/is/intro.html3http://marsrovers.nasa.gov/technology/is autonomous mobility.html

2

Figure 1.2: Robotic Space Mission classes2, as of 2005

The unique conditions of space exploration further emphasize the need for creatingflexible systems that can switch tasks in a provably correct and robust manner. Such tasks,due to communication constraints should be defined at a high level and the robot shouldhave the ability of determining what it must do in order to achieve the task.

The examples and challenges described above lead to the following question:How does one describe symbolic, high level tasks and transform them automaticallyinto sensing and control while obtaining formal guarantees of correctness?

Answering this question requires one to address many issues, ranging from user in-terface to high level planning to low level control to dealing with complexity and muchmore. What is the right language and formalism to capture tasks? What is considered ahigh level task? What assumptions must be made in order to guarantee correctness? Whatis the complexity of the problem? Is it tractable? What robot models can be addressed?How to deal with uncertainty? are only some of the questions that need to be considered.

A number of communities have been addressing these questions using different ap-proaches. The Artificial Intelligence, and more specifically, the planning community havebeen studying this problem using a top-down view. There, the system has the ability toperform different, usually discrete actions and the goal is to compose such actions intocomplex goals [74, 85, 98, 110] in a correct and computationally efficient way.

The dynamics and control world has traditionally focused on the bottom - up view

3

where complex continuous systems are defined and analyzed and the challenges are todrive systems to some form of equilibrium while insuring stability, optimality and safety.In the robot motion-planning community [29], the focus is on driving a robot or a setof robots from an initial configuration to a final one while taking into account complexdynamics, body size, different motion constraints and obstacles and dealing with issuessuch as complexity [26] and optimality.

The emerging fields of symbolic control [13] and hybrid systems [6, 102] are makingprogress towards combining ideas and results from computer science, such as languages,automata theory and logic with basic control concepts such as stability, optimality andconvergence. In a nutshell, a hybrid system is a system that has both discrete evolutionand continuous dynamics which affect each other. Such systems arise naturally whenconsidering complex physical systems with different behaviors and possible dynamics,such as robots, and they provide a suitable platform for combining discrete planning withcontinuous control. Several results that share some of the same ideas and techniques ofthis thesis are further discussed in Section 1.2.

The work presented in this thesis addresses some of the challenges of creating au-tonomous systems that perform complex tasks while providing guarantees for correct be-havior. Here, tasks defined using the formalism of temporal logic [42] are automaticallytranslated into controllers that drive a robot or group of robots and activate the differentrobot actions such that the task is completed, if it is feasible. The approach presented iscorrect by construction, it is complete in the sense that if under specific assumptions thetask can be done, a controller will be created and it is sound, that is, if the task cannotbe accomplished the algorithms return that it is impossible to complete the task and nocontroller is created.

Figure 1.3 depicts the three steps that are taken when causing a robot to perform ahigh level task. First, the task itself is described; This description captures the requiredbehavior as well as relevant information about the robot capabilities such as actions andsensors, and the environment in which the robot is operating. This step transforms agenerally continuous problem into a discrete one by creating abstractions of the differenttask components. The next step is to find a discrete solution for the problem, a discreteplan, that if followed the robot is guaranteed to achieve the discrete behavior requiredof it. The final step is to return to the original continuous problem by transforming thediscrete plan to a hybrid controller that switches between a set of atomic controllers orcontrol primitives that provide continuous local guarantees for the behavior of the robot.

This thesis addresses two types of tasks; nonreactive and reactive. Here reactivitydoes not refer to handling noise and disturbances in the low level control but rather towhether the task depends on information that is obtained during the task execution. A nonreactive or “open loop” task requires the robot to preform a certain behavior no matterwhat is happening in the robot’s environment for example follow motion patterns in theworkspace. These tasks must deal with noise and disturbances inherent in a physicalsystem but the robot does not need to make any decisions regarding what to do next.

Reactive or “closed loop” tasks, on the other hand, are tasks in which the behavior of

4

Figure 1.3: Overview

the robot does depend on what is happening around it. For example, a robot patrolling aperimeter that must sound an alarm and follow the intruder if one is detected. The behaviorof the robot will be different if there is no intruder, if one is seen at the start of the patroland if the intruder appears in the middle of the perimeter. As in the case of non reactivetasks, the robot control must deal with noise and low level disturbance but here it mustalso deal with high level decisions the robot must make during the execution of the task.

The term “high level” which shows up frequently throughout this thesis, is used toindicate that task descriptions do not specify every aspect of how to perform the task butrather describe the intended behavior that the robot must achieve. For example, if the robotmust go to Room x on the other side of the workspace, and it may choose any route there,the high level task will be “go to Room x” instead of “first leave this room with velocityy, then travel the hall until you reach an intersection...” and so on. The use of “low level”in this thesis usually refers to the continuous control inputs that are sent to the robot’sactuators such as velocity and steering commands.

5

1.2 Related workAt the heart of the work presented here, lays the idea of creating basic or atomic entitiesthat are capable of simple behaviors and composing them to obtain complex behaviorswhile providing guarantees of correctness. This idea is a recurring theme in the area ofsymbolic control [13].

The work in [25, 97] relies on the idea of sequential composition. There, to achievecomplex motions, control policies which are defined over local domains are composedwhile providing guarantees that the robot will reach its goal. Later work expanding on thisidea [35, 38] describe automated methods for both developing such control policies anddetermining the order in which they must be executed to achieve the goal (in this case,driving a robot in a complex workspace to a predefined goal). The controllers developedby Conner at. el. [35, 38] are used throughout this thesis. Similar ideas, namely creating ahybrid controller that drives a robot from an initial position to a final one while visiting aset of locations along the way, were explored in [15]. There the workspace is decomposedinto triangles and the controllers are generated automatically by solving linear program-ming instances.

The idea of composing several controllers to achieve a complex task, such as control-ling an inverted pendulum, also appears in [95]. There first the qualitative behavior of thebasic controllers are defined and then these behaviors are refined to specific control laws.Prior to that, [106] describes how to verify that continuous systems satisfy CTL* [42]formulas by using the same notion of qualitative simulation as in [95].

In [19] the control input is quantized and represented as a finite word that can betransmitted to the robot which in turn decodes it and activates the appropriate control. Theauthors show that by choosing a suitable input encoding, the quantized control can drive awide variety of dynamical systems arbitrarily close to a desired equalibrium.

In [49] the authors create a regular language where each symbol is a motion primitive(and not a control input) that belongs to a finite library of basic motions. This languageis used to solve the motion planning problem of controlling the system such that it goesfrom an initial configuration to a final one under some constraints (such as operationalenvelope). Each string in this language corresponds to a motion behavior of the systemand furthermore, if a behavior can be captured by a string, it is dynamically feasible and itis safe with respect to the operational envelope constraints. In [41] the primitives, in thiscase control programs, are derived from empirical data which is input output pairs of anobserved system.

The use of temporal logic for control of systems was advocated as far back as [86, 87]in the context of Discrete Event Systems (DES) [27, 93, 94]. In the theory of DES, thesystem or plant that is to be controlled is discrete by nature (discrete state space, piecewiseconstant trajectories) and the evolution or behavior of the system is governed by discreteevents. These events are divided up into two sets; controllable and uncontrollable. Thecontrol of DES is concerned with restricting the controllable event such that the system isguaranteed to achieves a goal. In [86] the DES is modeled as a timed transition systemand the author develops a non automated procedure for generating controllers that satisfy

6

safety specifications given in real-time temporal logic.Expanding on the work in DES, the idea of controller synthesis that satisfies certain

classes of specifications was explored in [7] for timed systems. Later, tools for controllersynthesis for timed systems were introduced [3].

Lately, advanced and computationally efficient methods from the computer aided ver-ification community such as Binary Decision Diagrams (BDD [23]) and symbolic modelchecking [34] are being used for the design of controllers for DES that satisfy temporallogic specifications [51]. The work described in this thesis differs from the DES resultsboth in terms of the systems being considered, continuous vs. discrete, and in the notionof control. While in DES the system is given and must be restricted to provide desiredbehaviors here the objective is to build the system such that it satisfies a desired task byconstruction.

In the context of hybrid systems [6, 102], temporal logic was used as specification asearly as [5]. There, the authors employ the techniques of DES control, namely removingcontrollable events, together with temporal logic and model-checking to control a walkingmachine which is hybrid in nature (continuous dynamics of the legs coupled with theirdiscrete modes).

Decidability issues for linear temporal logic (LTL) were addressed in [107,108]. There,it was shown that the existence of controllers that satisfy linear temporal logic specifica-tions for discrete time controllable linear systems is decidable and furthermore, a methodfor generating such hybrid controllers is described. The notion of bisimulation, describedin Section 2.1.3 used in that work is the basis for the guarantees provided in this thesis.The work in [63] uses model-checkers to find discrete paths satisfying a temporal logicspecification in an abstraction of the workspace. It follows the work described in [45, 46]and in Chapter 2 by adapting it to handle linear systems in higher dimension that may havedrift.

In the AI planning literature [85, 98], the planning problem is usually formulated asa set of actions and a final goal that should be reached using the available actions. Eachaction defines a precondition which is some logical condition that must be satisfied tobe able to perform the action. An action also defines a postcondition which describesthe results of executing that action. The term universal plan, introduced in [104], refers toplans that cause the system to reach its final goal in nondeterministic domains. The actionsof the system depends on the behavior of the environment at execution time, much like thereactive tasks addressed in this thesis. The notion of temporally extended goals, goals thatrepresent sequences rather than final states, has been gaining interest from the planningcommunity in the past several years. Temporal logic was first suggested as the formalismto define such goals in [9]. This work was later implemented in TLplan [10], a plannerthat uses control knowledge expressed in a first order linear temporal logic.

The Model Based Planner (MBP) [17,18,31,32,72] is a state of the art planner for non-deterministic domains which is based on symbolic model checking techniques, making itcomputationally efficient. It produces universal or “strong” plans when such plans exists,it can generate “strong cyclic” plans which are plans that may contain loops but guarantee

7

the system eventually reaches the goals and it can produce “weak” plans that when exe-cuted, the system has a chance of succeeding [31]. Unlike the majority of implementedplanners, this tool deals with temporally extended goals [72] and issues related to partialobservability [18].

Planning using model checking techniques has gained interest along the years [59],whether with the use of symbolic model-checking as in MBP, or explicit state [28, 73].Recently, SAT solvers which are often used for bounded model checking were used toplan for temporally extended goals expressed as LTL formulas [83].

The main difference between the planning literature that deals with extended goals andthe planning aspects of the work presented here is a rather philosophical one. In the AIplanning world one must define actions and the system cannot make any move that wasnot specifically defined as a legal action. In contrast here, in the work presented in Chapter3 for reactive tasks (which can be viewed as universal plans) the system can do anythingunless restricted by the temporal logic specifications.

Chapter 6 discusses multi robot tasks. While there is a vast amount of work concerningmulti robot missions and control, the work mentioned here shares some common themeswith the work described throughout this thesis. The Universal Multi-agent OBDD-basedPlanner (UMOP) [55] expands the ideas developed for MBP [17] to multi robot nondeter-ministic domains. Furthermore, it adds the notion of optimistic planning where optimalityguarantees are relaxed if an optimal plan that solves the problem in the nondeterministicdomain cannot be found. This planner utilizes the efficient BDD representation that is usedby MBP and by the work in this thesis however it plans for final goals and not temporallyextended goals.

In [81] a hybrid controller was created to regulate vehicles in an automated highwaysystem. This controller was generated using game theoretic techniques where the vehicles“play” against the disturbances on the road (other vehicles) and the controller is modifiedbased on the performance that can be guaranteed. This method guarantees safety of thesystem by design. The idea of the robot “playing” against its environment appears inChapter 3.

Driving a group of robots from an initial configuration to a final one while maintaininginter robot proximity constraints (the robots should not collide but should maintain a pre-defined communication range) is addressed in [8]. The ideas presented in that paper arerevisited in Section 6.3 where the multi robot controllers form the basic controllers thatare later composed.

Linear Temporal Logic specifications are used in [80] together with multi robot navi-gation functions to create controllers for multi robot missions such as switching betweenformations while driving to a final configuration. The use of timed automata [4] was pro-posed in [92] where multi robot goals are represented by CTL formulas. There, the robotsare moving using discrete steps in a planar grid and there are no continuous dynamicsinvolved. The paths or motion plans satisfying the requirements are generated using theUPPAAL tool [12].

Multi robot nonreactive missions such as UAV scheduling are addressed in [61] where

8

by combining optimization techniques such as Mixed Integer Linear Programming withLTL specifications, complex multi robot goals are achieved while providing optimality.

Complex multi robot behaviors are encoded as graph grammars in [62]. There, in adecentralized manner, each robot acts according to the grammar and global tasks, such asself assembly of robots, are achieved.

1.3 ContributionsThe contribution of the work presented in this thesis is three fold. First, a unified frame-work is developed that translates tasks given as high level instructions in a subset of En-glish to low level control inputs, such as velocity commands, for a robot or a group ofrobots in an automatic and provable correct way. This approach represents a bridge thatlinks planning and AI to traditional control, two communities that usually do not overlapbut that must be connected to enable the creation of systems capable of complex behaviorsthat can be easily changed.

The second is the novelty of accommodating high level reactive tasks. The ideas pre-sented here are in the spirit of symbolic control where basic actions are developed andcomposed to create complex behaviors while obtaining certain guarantees, however, thework described in Section 1.2 addresses complex yet nonreactive goals. Allowing reactivetasks enriches the set of behaviors that might be required of a robot. Furthermore, by au-tomatically translating such tasks to controllers that are guaranteed to be correct, systemdesigners are relieved of the need to add extra logic that must be validated to their system.

The third is the development of a Python based toolbox that enables one to define arobot and an environment and then give the robot instructions in Structured English thatare then automatically translated into a controller, if such a controller exists. The modulardesign of this toolbox allows researchers to experiment with different controller design(by including code for different control laws) and different robot models as explained inChapter 8.

1.4 Thesis StructureThis thesis is structured as follows. Chapter 2 describes how nonreactive high level tasks,specified using temporal logic, can be handled using model-checking as done in [45, 46].Chapter 3, following [68], moves to reactive high level tasks in which the behavior of therobot may depend on the state of the environment as sensed during execution. While theprevious two chapters focus on a simple robot model (fully actuated point robot), Chapter4 describes how more complex robot models can be addressed as well, as explained in[36, 44, 47].

The high level tasks addressed in Chapter 3 are captured using a specific fragmentof Linear Temporal Logic (discussed in Section 3.2.3). Chapter 5 describes the workin [67,69] in which a restricted set of English sentences are automatically translated to the

9

required logic formulas thus creating a interface that is more natural to use than writinglogic directly.

Multi robot tasks can be captured using the methods of the previous chapters as de-scribed in Chapter 6. When dealing with such tasks, the planning and control approachcan be either decentralized as in [66] or centralized as in [65]. Chapter 6 discusses theadvantages and limitation of these two approaches.

The Urban Challenge mentioned above is used in Chapter 7 (following [70]) as a casestudy for the work presented in the previous chapters. Chapter 8 describes a Python basedtool that implements many of the ideas in this thesis and that can be used to simulate a robotin the Player/Stage/Gazebo framework [2]. Conclusions and future research directions areaddressed in Chapter 9.

10

Chapter 2

Non reactive tasks

In this work a non reactive task for a robot is defined as a task where the robot does notneed to make any high level decisions regarding its behavior while it is operating. Onecan consider such tasks as “open loop” behaviors since the robot does not use any highlevel information from the environment, other than location, as feedback to its controller.Such tasks may include complex motion patterns or action sequences as demonstrated inSection 2.4.

This chapter addresses the challenge of generating robot controllers that are guaranteedto control a robot such that it satisfies a high level nonreactive task.

2.1 Preliminaries

2.1.1 Linear Temporal LogicLoosely speaking, Linear Temporal Logic (LTL) [42] consists of the standard proposi-tional logic with some temporal operators that allow us to express requirements on se-quences of propositions.

Syntax: Let AP be a set of atomic propositions. Linear Temporal Logic formulas areconstructed from atomic propositions π ∈ AP according to the following grammar:

ϕ ::= π | ¬ϕ | ϕ ∨ ϕ | © ϕ | ϕU ϕ

As usual, the Boolean constants True and False are defined as True = ϕ ∨ ¬ϕ andFalse = ¬True respectively. Given negation (“not”,¬) and disjunction (“or”,∨), one candefine

• Conjunction (“and”) ϕ ∧ ϕ = ¬(¬ϕ ∨ ¬ϕ)

• Implication (“if”) ϕ1 ⇒ ϕ2 = ¬ϕ1 ∨ ϕ2

• Equivalence (“iff”) ϕ1 ⇔ ϕ2 = (ϕ1 ⇒ ϕ2) ∧ (ϕ2 ⇒ ϕ1)

11

Given the “next” (©) and “until” (U) temporal operators, additional temporal operatorscan be derived such as

• “Eventually” 3ϕ = True U ϕ

• “Always” 2ϕ = ¬3¬ϕ

Semantics: The semantics of an LTL formula ϕ is defined on an infinite sequence σof truth assignments to the atomic propositions π ∈ AP . The set σ(i) denotes the setof atomic propositions that are True at position i. The recursive definition of whethersequence σ satisfies LTL formula ϕ at position i (denoted σ, i |= ϕ) is:

• σ, i |= π iff π ∈ σ(i)

• σ, i |= ¬ϕ iff σ, i 6|= ϕ

• σ, i |= ϕ1 ∨ ϕ2 iff σ, i |= ϕ1 or σ, i |= ϕ2

• σ, i |=©ϕ iff σ, i+ 1 |= ϕ

• σ, i |= ϕ1 U ϕ2 iff there exists k ≥ i such that σ, k |= ϕ2, and for all i ≤ j < k wehave σ, j |= ϕ1

Intuitively, the formula©ϕ expresses that ϕ is True in the next “step” (the next posi-tion in the sequence) and the formula ϕ1 U ϕ2 expresses the property that ϕ1 is True untilϕ2 becomes True. The sequence σ satisfies formula ϕ if σ, 0 |= ϕ. The sequence σ sat-isfies formula 2ϕ if ϕ is True in every position of the sequence, and satisfies the formula3ϕ if ϕ is True at some position of the sequence. Sequence σ satisfies the formula 23ϕif ϕ is True infinitely often.

2.1.2 Model CheckingGiven a model of a system and a set of requirements the system should satisfy, modelchecking [34] is the algorithmic procedure used to check whether the system indeed sat-isfies the requirements or not. The model that represents the system that is being verifiedis usually given in the form of a finite state machine or a directed graph. The specificationformula is usually given in some form of temporal logic such as LTL.

Model checking algorithms are sound and complete. This means that if the algorithmreturns that the requirement is satisfied then every execution of the system model is guar-anteed to satisfy the requirement. On the other hand, if there is even a single execution ofthe model that does not satisfy the requirement, the algorithm will return that the require-ment is False. Furthermore, usually1 the model checker will provide a counter examplewhich is an execution trace of the model that does not satisfy the requirement.

1For branching time logics, such as CTL, the counter example for a requirement of the form “it is possiblethat ψ is True” must show that on all executions ψ is not True. Such a counter example cannot be given sinceit requires all infinite executions of the system to be specified.

12

The interested reader if referred to [34] for a description of model checking algorithmsfor different logics, to [103] for a survey of the complexity results and to [109] for ahistorical perspective. In the following, model checking tools are treated as black boxes.

2.1.3 BisimulationAn unlabeled transition system T is defined as

T = (Q, I,→) (2.1)

where Q is a set of states, I ⊆ Q is the set of initial states and→⊆ Q×Q is the transitionrelation defining the possible transition between the states.

Given two transition systems T1 = (Q1, I1,→1), T2 = (Q2, I2,→2) the binary relation∼= ⊆ Q1 ×Q2 is a bisimulation relation iff

∀(q1, q2) ∈ ∼=If q1 →1 q

′1 then ∃q′2 s.t. q′1 ∼= q′2 and q2 →2 q

′2

If q2 →2 q′2 then ∃q′1 s.t. q′1 ∼= q′2 and q1 →1 q

′1

(2.2)

and the two systems are bisimilar if

∀q ∈ I1 ∃q′ ∈ I2 s.t. q ∼= q′

∀q′ ∈ I2 ∃q ∈ I1 s.t. q ∼= q′(2.3)

Intuitively, if two systems T1, T2 are bisimilar, their observed behavior is the same2.Another way of putting it is that T1 can match any move that T2 makes and vice versa.

2.2 Problem formulationThe problem being considered in this chapter combines three components; A robot, theworkspace in which it is operating and the task it needs to satisfy.

2.2.1 RobotThe robot model considered in this chapter is a fully actuated, planar robot moving in apolygonal workspace P such that

p(t) = u(t) p(t) ∈ P ⊆ IR2 u(t) ∈ U ⊆ IR2 (2.4)

where p(t) is the position of the robot at time t, and u(t) is the control input.

2Usually, the bisimulation relation is defined on the observations of the systems. Here, the states act asthe observations.

13

The robot may have a set of actions it can perform such as activating a video cameraor turning on a siren. These actions are denoted by the set Action = {a1, a2, . . . , al} andthe activation and deactivation of these actions is governed by an action activation policyact : t→ 2Action where

∀i : ai ∈ act(t) → ai is ON at time t∀i : ai 6∈ act(t) → ai is OFF at time t (2.5)

The robot is assumed to have perfect information regarding its state, i.e., perfect lo-calization and full control of its actions. Furthermore, the actions it can perform have notiming constraints, that is, they can be turned on and off at any time.

2.2.2 WorkspaceWorkspace P is assumed to be partitioned using a finite number of cells P1, . . . , Pn, where

P = ∪ni=1PiPi ∩ Pj = ∅ ∀i 6= j

(2.6)

Section 2.3.1 elaborates on how one might choose a partition but in general it correspondsto both the areas of interest in the workspace and to the motion capabilities of the robot.In the following each of these cells is assumed to be a convex polygon of the form

Pi = {p ∈ IR2 |∧

1≤k≤m

aTk p+ bk ∼ 0, ak ∈ IR2, bk ∈ IR} (2.7)

where ∼∈ {<,≤}. One can use many efficient cell decomposition methods for polygonalenvironments [29, 40].

The partition naturally creates a set of Boolean propositions R = {r1, r2 . . . , rn}which are True if and only if the robot is located in Pi,

ri =

{True if p ∈ PiFalse if p 6∈ Pi

(2.8)

Since {Pi} is a partition of P , exactly one ri can be True at any time.

2.2.3 TaskThe desired tasks are represented as LTL formulas. Here, the atomic propositions are thesetsR and Action3 and the desired task is define by their composition.

Some LTL examples that express interesting properties include:

3With a slight abuse of notation, the set Action is used to denote both the actions and the propositionsrelating to these actions. The truth value of the proposition indicates whether the action is activated or notwith True = ON and False = OFF.

14

• Reachability: The formula 3ri means that the path of the robot should eventuallyreach a state where ri is observed, i.e the robot reaches cell Pi of the workspace P .

• Reach goal while avoiding some regions: The formula ¬(ri ∨ rj ∨ rk)Urgoal ex-presses the desired property that eventually the robot will enter cell Pgoal and untilthen, it must avoid regions labeled Pi, Pj and Pk.

• Sequencing: The requirement that the robot must visit P1, P2, and P3 in this orderis naturally captured by the formula 3(r1 ∧3(r2 ∧3r3)).

• Coverage: The formula 3r1 ∧ 3r2 ∧ · · · ∧ 3rm reads as the robot will eventuallyreach P1 and eventually P2 and ... eventually Pm, requiring the robot to eventuallyvisit all regions of interest. Note that this formula does not specify the order in whichthe robot must visit these rooms.

• Action - location dependency: The requirement that the robot activate its siren ifand only if it is in cells P1 or P3 is captured by the formula 2((r1 ∨ r3)⇔ asiren)

More complicated specifications can be composed from basic specifications using thelogic operators.

For such temporal logic formulas, this chapter provides a computational solution ofthe following problem

Problem 1. [Non reactive task] Given a robot model (2.4), a robot action set Action, anenvironment decomposition (2.6), initial position p(0), initial value for the actions act(0),and a LTL formula ϕ, construct a control input u(t) and an action activation policy act(t)so that the resulting robot behavior satisfies the formula.

Example 1. In order to better explain the different steps, the following example is consid-ered throughout this chapter. Consider a robot capable of two actions, activating a sirenand a video camera, that is moving in a square environment with five areas of interestdenoted by r1, . . . , r5. Initially, the robot is placed somewhere in the region labeled r1 (seeFigure 2.1, robot initial position denoted by an X). The desired specification for the robotgiven in natural language is: ”Visit area r2 then area r5 then area r3 and, finally, go toregion r4 while avoiding areas r1, r2 and r5. Only when you are in r1 turn your siren onand the camera should be on whenever you are in areas r2, r3, r4 and r5”.

The partition of the workspace is given in Figure 2.2 and the temporal formula describ-ing this example is given by

ϕ = r1 ∧ as ∧ ¬ac∧3(r2 ∧3(r5 ∧3(r3 ∧ (¬r1 ∧ ¬r2 ∧ ¬r5)Ur4)))∧2(r1 ⇔ as) ∧2((r2 ∨ r3 ∨ r4 ∨ r5)⇒ ac)

(2.9)

where ri represent the cells, ac is the activation of the camera and as is the activation ofthe siren.

15

Figure 2.1: Workspace for Example 1

Figure 2.2: Partitioned workspace for Example 1

2.3 Generating non reactive controllersThe solution to generating continuous robot trajectories satisfying LTL formulas ϕ consistsof the following three steps:

1. Discrete Model of the Robot: Using the workspace decomposition {Pi}, the robotdynamics (2.4) and action set Action construct a discrete robot model that corre-sponds to the possible robot motion and action combinations.

2. Discrete Solution: Construct plans for the discrete robot model satisfying the desiredspecifications using model checkers.

3. Continuous Implementation of Discrete Plan: Implement the discrete plan at thecontinuous level while preserving the satisfaction of the temporal formula.

2.3.1 Discrete Abstraction of Robot Motion and ActionThe first step towards solving Problem 1 is to transform it into a discrete problem. Thetask is already given in discrete terms as a combination of propositions relating to the

16

location of the robot and to the on/off state of its actions as are the actions themselves.What remains to be discussed is the discretization of the robot’s motion capabilities andthe creation of the discrete robot model.

Robot Motion

Workspace P is partitioned into a set of convex cells {Pi} which are labeled by the setR = {r1, . . . , rn}. These cells typically correspond to areas with specific meaning in theworkspace, such as rooms, hallways and obstacles.

Let T : P −→ R denote the partition map that sends each position p ∈ P to the finiteset R such that T (p) = ri iff4 p ∈ Pi. In other words, T−1(ri) contains all positionsp ∈ Pi.

Let CMotion be a transition system describing the continuous motion of the robot

CMotion = (P, p(0),→CM) (2.10)

where P is the continuous state space, p(0) ∈ P is the initial position of the robot and→CM

⊆ P × P is the transition relation between the cells. Formally, there is a transitionp →CM

p′ if p and p′ belong to adjacent cells, and it is possible to construct a controllaw u(t) that generates a trajectory p(t) for 0 ≤ t ≤ τ with p(0) = p and p(τ) = p′, and,furthermore, for all 0 ≤ t ≤ τ , p(t) ∈ (T−1(T (p))∪T−1(T (p′))). Informally, p→CM

p′ ifthe robot can be steered from p to p′ without visiting any cell other than the cell containingp or the neighboring cell containing p′.

Let the discrete representation of the motion of the robot DMotion be a finite statemachine whose states are the labels of the cells {Pi}

DMotion = (R, r(0),→DM) (2.11)

where R is the finite set of states representing the convex cells5 and r(0) ∈ R is the cellcontaining the initial robot state p(0) ∈ P , that is r(0) = T (p(0)).

The discrete dynamics of the robot are captured by the transition relation→DM⊆ R×

R. If ri, rj are connected (ri →DMrj) this means that the robot can move in the discrete

sense from cell Pi to cell Pj without going through any other cell. Note that in this workself loops, that is transitions from a cell to itself, are not considered.

The object of this step is to create a discrete model of the robot that behaves the sameas the continuous one. The discrete model captures the full continuous dynamics if forevery transition p→CM

p′ in CMotion there exists a discrete transition T (p)→DMT (p′) in

DMotion. The continuous model CMotion can implement any discrete transition r →DMr′

if for every p ∈ T−1(r) there is some p′ ∈ T−1(r′)such that p →CMp′. More formally,

4Since it is a partition, every p ∈ P is mapped to exactly one ri. When dealing with complex dynamicsthe partition requirement is lifted as explained in Chapter 4.

5Since P is partitioned, there is a one to one correspondence between states in DMotion and the labelsR. In order to keep the notation concise the set R is used as the states as well.

17

for the discrete system DMotion and the continuous system CMotion to behave the samethey must be bisimilar. Viewing T as a binary relation T ⊆ P × R where (p, T (p)) ∈ T ,DMotion and CMotion are bisimilar iff

∀(p, r) ∈ TIf p→CM

p′ then ∃r′ s.t. T (p′) = r′ and r →DMr′

If r →DMr′ then ∃p′ s.t. T−1(r′) = p′ and p→CM

p′(2.12)

Clearly, for a specific partition T , the choice of control law and the underlying ro-bot dynamics determine whether DMotion and CMotion are bisimilar. This issue is furtheraddressed in Sections 2.3.3 and 2.5.

Robot Action

Similarly to DMotion, each action the robot can perform is represented by a state machine

DActioni= (Ai, ai(0),→Ai

) (2.13)

where Ai = {∅, ai} contains two states corresponding to whether the action is off or on,ai(0) ∈ Ai is the initial state ofDActioni

and→Ai= Ai×Ai allowing any possible transition

between the states.

Discrete Robot Model

The discrete robot model is created by composing the discrete motion DMotion with all ofthe action representations DActioni

. Let Q = R × A1 × . . . × Al be the set of composedstates such that q = (r, a1, . . . , al), r ∈ R, ai ∈ {∅, ai}. Let

π : Q −→ Rπ(q) = r

(2.14)

be the projection map which extracts the cell label r from the composed state q and let

α : Q −→ 2Action

α(q) = a1 ∪ · · · ∪ al(2.15)

be the set representing the actions that are ON in the composed state q.The discrete robot model is defined as

D = (Q, q(0),→D) (2.16)

where Q is the set of states, q(0) = (r(0), a1(0), . . . , al(0)) ∈ Q is the initial state of therobot which includes the initial cell and the initial state of the actions and→D⊆ Q×Q isthe transition relation which is defined as

qi →D qj iff π(qi)→DMπ(qj) (2.17)

18

Note that since there are no restrictions on the action transitions, the only constraints onthe transition of the full robot model are due to the motion constraints. Furthermore, sincethe relation→DM

does not contain self loops, the discrete robot model D does not containtransitions of the form (ri, ∅, . . . )→D (ri, a1, . . . ) where the cell does not change but theset of active actions does. While such transitions can be easily added, their continuousmeaning is that an action can be switched on or off while the robot is in a specific cell.Since timing constraints cannot be enforced, usually there is no need for such a behavior6

Having defined transitions →D for the finite state machine D, one can define trajec-tories σ of D as sequences of the form σ = σ(0), σ(1), σ(2), . . . , where σ(i) ∈ Q and∀i ≥ 0 σ(i)→D σ(i+ 1).

Example 1 continued: The discrete model D for Example 1 is constructed from thediscrete system DMotion and the two discrete systems DAs and DAc . The motion repre-sentation,depicted in Figure 2.3, consists of 17 states corresponding to the 17 cells in thepartition of workspace P . The action representations, each containing two states are shownin Figure 2.4. The discrete model of the robot, containing 68 states, is the composition ofthe motion and action systems. Figure 2.5 shows the complete system D where for clarity,multiple transitions are represented by thick arrows. Each such arrow is the collection oftransitions between every state in one box to every state in the other box, with each boxrepresenting all the states that share the same ri but differ on the set of active actions. Aclose up of such a transition is depicted in Figure 2.6. Note that since there are no transi-tions between two states that share the same ri, there are no transitions between states thatare in the same box.

6It is possible to have a specification that requires an action to be on or off for a certain number of steps,by using the ‘next’ operator, however such specifications are not considered here.

Figure 2.3: Discrete system representing the robot motion

19

Figure 2.4: Discrete systems representing the robot action

Figure 2.5: Discrete robot model D

20

Figure 2.6: Close up of Discrete robot model D

21

2.3.2 Temporal Logic Planning using Model CheckingTo solve the discrete problem, this step involves looking for computation paths σ that sat-isfy the temporal formula σ, 0 |= ϕ. In the model checking community, this is knownas the generation of witnesses. Unfortunately, the current versions of the model checkingsoftware tools do not support the construction of witnesses as they are mainly analysistools. Hence, the algorithms that solve the dual problem, i.e. the generation of counterex-amples, are employed. In this case, when the model checker determines that a formula ϕis False, it constructs a finite trace σ which demonstrates that the negation of ϕ is True, i.e.σ, 0 |= ¬ϕ.

Let ϕ be the formula that the system should satisfy. Assume now that what is given asinput to the model checking algorithm is the LTL formula ¬ϕ, representing the negationof the desired behavior. If the formula is False in the discrete model of the robot D, thenthe model checker will return a finite trace σ that satisfies the formula ¬(¬ϕ) ≡ ϕ and,thus, a finite discrete path that satisfies the original LTL formula ϕ is found and can beexecuted by the robot.

Out of the variety of model checking tools that have been developed over the years,the most dominant ones are NUSMV [30] which is based on symbolic model checkingtechniques and is mainly targeted for CTL (but it can also handle LTL) model checkingproblems, and SPIN [53] which uses an automaton approach to the model checking prob-lem and accepts only LTL formulas. Both toolboxes support hierarchy and composition,multiple agents, generation of counterexamples in case the temporal formula is invalidatedand nondeterministic environments. Of course, there are also several differences betweenthe two toolboxes mainly concerning the way they deal with the model checking problem,the user interface and the expressive power of the underlying logic. SPIN only supportsasynchronous communication among agents, but it supports the option for the generationof traces that are optimal in the sense of minimum number of transitions (trace length).

The conversion of the discrete transition system of Section 2.3.1 to the input languageof NUSMV or to the input language of SPIN is straightforward and it is automated.

Example 1 continued: The discrete robot model, containing 68 states, is automaticallytranslated into the input format required by NUSMV as shown in Appendix A.1. The resultof the model checker, containing the discrete sequence σ that satisfies the LTL formula ϕis shown in Appendix A.2. The sequence σ satisfying the desired behavior of Example 1is

(r1, ∅, as), (r11, ∅, ∅), (r2, ac, ∅), (r11, ∅, ∅), (r3, ac, ∅), (r14, ∅, ∅),(r5, ac, ∅), (r15, ∅, ∅), (r12, ac, ∅), (r1, ∅, as), (r11, ∅, ∅), (r3, ac, ∅),(r13, ac, ∅), (r11, ∅, ∅), (r8, ac, ∅), (r6, ∅, ∅), (r9, ac, ∅), (r4, ac, ∅), (r7, ∅, ∅)

Following this sequence, one can see that the desired behavior is satisfied. However,since the model checker gives an arbitrary counter example this path causes the robot tobehave in correct yet unexpected ways. In this example, the robot moves from cell r5

to cell cell r3 by traversing cell 15,12,1 and 11 although it is faster to go through 14.Furthermore, the camera is indeed on in cells r2, r3, r4 and r5 but it is also on in other cells

22

such as r8 even though it was not required to be on there. The issues of optimality andunexpected behavior are revisited in Section 2.5.

2.3.3 Continuous Implementation of Discrete TrajectoryThe final step is the conversion of the discrete trajectory σ to both the control input u(t)and the action activation policy act(t) for t ≥ 0 thus generating a continuous behavior thatsatisfies the task.

This desired goal is achieved by simulating (or implementing) at the continuous leveleach discrete transition of σ = σ(0), σ(1), . . . . Let the time sequence t0, t1, t2, . . . denotethe times when the robot moves from discrete state σ(i) ∈ Q to σ(i+ 1) ∈ Q such that

∀ti ≤ t < ti+1, p(t) ∈ T−1(π(σ(i))), act(t) = α(σ(i))and

p(ti+1) ∈ T−1(π(σ(i+ 1))), act(ti+1) = α(σ(i+ 1))(2.18)

Control law u(t)

Let σM = σM(0), σM(1), σM(2), . . . be the sequence representing the robot motion suchthat

σM = π(σ(0)), π(σ(1)), π(σ(2)), . . . (2.19)

where σ is the solution to the discrete problem as determined by the model checker.The control law u(t) is generated by continuously implementing the discrete sequence

σM or more specifically, the transitions σM(i)→DMσM(i+ 1).

Proposition 1. Let ϕ be an LTL formula, and let T : P −→ R be a bisimulation. If σ, 0 |=ϕ, then for every p(0) ∈ T−1(r(0)) there exists a trajectory p(t) satisfying p(0) |= ϕ.

Proof. Let σM = r0 →DMr1 →DM

r2 →DM. . . be a trajectory of transition system

DMotion starting at r0 = r(0) and satisfying formula ϕ. A sequence of continuous statesp0, p1, p2, . . . is constructed such that pi = T−1(ri) and p0 →CM

p1 →CMp2 →CM

. . . .Consider the first discrete transition r0 →DM

r1. Consider any p(0) ∈ T−1(r(0)).Since T is a bisimulation, for all p(0) ∈ T−1(r(0)) there exists a p1 with T (p1) = r1

with p0 →CMp1. Next, the transition r1 →DM

r2 is implemented. Once more, since Tis a bisimulation, for all p ∈ T−1(r1) (and thus for p1) there exists a p2 with T (p2) = r2

with p1 →CMp2. Continuing this process, the desired trajectory p(t) is constructed which

causes the robot to visit the cells in exactly the same order as the discrete trajectory σM ,hence satisfying the exact same LTL formula.

There are several recent approaches for generating controllers that satisfy the bisim-ulation requirement for different robot models, such as [37, 76–78] and [14]. Figure 2.7illustrates the velocity vector fields induced by such controllers, either driving the robotthrough a specific face or keeping it in the cell forever7.

7When the robot reaches the last cell in the sequence, it must stay there.

23

(a) (b)

Figure 2.7: Velocity vector fields

When considering fully actuated, kinematic, point robots one can use feedback con-trollers as simple as a controller that follows the line connecting the current point and thecenter of the facet shared by the current cell and the adjacent cell. In the following twodifferent controllers are used which are a little more complex but create smoother motionby guaranteeing a smooth transition between adjacent cells.

Note that by satisfying the bisimulation property using feedback controllers, the tem-poral logic formula is robustly satisfied not only by the initial state p(0), but also by allother states in the same cell T (p(0)).

Activation policy act(t)

For each time t, the action activation policy turns the actions on or off depending on therequired state of the robot.

act(t) = α(σ(i)) ∈ 2Action , ∀ti ≤ t < ti+1, ∀i = 0, 1, . . . (2.20)

The continuous execution procedure is summarized in Algorithm 1.Example 1 continued: The continuous behavior of the robot, satisfying the task speci-

fied in Example 1 and simulated using MATLABTMis shown in Figure 2.8. The controllersused here are the ones developed in [37] for two-dimensional convex polygons. In this ap-proach, a convex polygon is mapped to the unit disk, then Laplace’s equation is solved(in closed form) on the disk, obtaining the potential function and finally, the solution ismapped back to the polygon. This approach resembles the navigation functions introducedin [96].

In this simulation the behavior of the robot is shown using six snapshots and the actionsare represented by the different markers. When the robot is activating the siren it is shownas an ‘x’, when it is activating the camera it is shown as a star and when both actions are

24

Algorithm 1 Continuous Execution of the discrete sequence σ

procedure EXECUTE(σ, p0, act(0))CurrCell← σM(0)CurrOutput← act(0)ActivateDeactivateOutputs(CurrOutput, Action)p← p0

for i← 1, length(σ)− 1 do . For the length of σif i ≡ length(σ)− 1 then . Last cell

while | p− pcenter |≥ ε do . Moving towards the center of the cellp← ApplyLastController(CurrCell, p)

end whilereturn . Arrived close to the center of the last cell, Task completed

end ifNxtCell← σM(i)while p ∈ T−1(CurrCell) do . Moving towards next cell

p← ApplyController(CurrCell, NxtCell, p)end whileif p ∈ T−1(NxtCell) then . Crossed facet to next cell

CurrCell← NxtCellCurrOutput← α(σ(i))ActivateDeactivateOutputs(CurrOutput, Action)i+ +

elseERROR - A catastrophic disturbance occurred, ended up in wrong cell

end ifend for

end procedure

25

off it is represented by dot. In this simulation it is never the case that both actions are on.The robot starts in the cell labeled r1 with the siren on. It first goes to Cell 2, turning thesiren off once it reaches Cell 11 (a), and then goes to Cell 5 while passing through 3. Asrequired, the camera is activated in Cells 2,3 and 5 (b). Although not required, the camerais activated in Cell 12 (c). This is considered correct since the requirement only statesthe camera must be on in 2,3,4 and 5 but does not require the camera to be off everywhereelse. After reaching Cell 3, the robot continues to 4 while avoiding 1,2 and 5 (d,e). Finally,the robot stays in Cell 7 which was not required but was included in the model checkergenerated counter example.

2.4 ExamplesThis section describes examples of complex motion requirements in different workspacesand using different model checkers. In all of the following the workspace P and the areasof interest are partitioned into triangles. There are several algorithms, each with its ownproperties, that allow an automatic triangulation of a polygonal environment [40]. Herethe workspace is triangulated using software developed in [84].

The controllers used here to drive the robot are the ones developed in [14] for simplices.There, an affine vector field is created in each triangle (2D simplex), that drives the robotto the desired adjacent triangle, while taking into consideration any velocity bounds therobot might have.

The first step of creating each example consists of specifying the workspace. Theworkspace is described as a set of vertices which define the outer contour, inner holes andinner regions of interest (such as rooms). These vertices are specified either by using aMATLABTMbased graphical user interface which allows the user to select points on a grid(see Figure 2.9) or by writing MATLABTMfunctions that create vertices in a desired pat-tern. Next a C program is called which triangulates the polygon using software developedin [84], and creates an input to the model checker being used. After writing the temporallogic formula a path is generated using a counter example in a model checker. The finalstep is creating the control law u(t) and simulating the path. This step is performed inMATLABTMand the control law is generated according to the method developed in [14],using linear programming.

Example 2. Figure 2.10 is an example of a trajectory satisfying a coverage requirement.In this example the desired behavior was to visit each of the rooms (shaded areas) in noparticular order. The LTL formula that was satisfied was [3r1∧3r2∧3r3∧3r4∧3r5∧3r6].

26

(a) Visiting r2 (b) Visiting r5

(c) Going to r3 through r1 (d) Visiting r3

(e) Going to r4 while avoiding r1, r2 and r5 (f) Task completed

Figure 2.8: Simulation of Example 1

27

Figure 2.9: Graphical User Interface

Figure 2.10: Example 2: Coverage

28

Example 3. This is an example of a trajectory satisfying a more complex requirement.In this example the desired behavior was ”visit Room r2, then Room r1 and then coverRooms r3,r4,r5 - all this while avoiding obstacles o1,o2,o3”. Figure 2.11 depicts the pathgenerated by NUSMV [30] while Figure 2.12 displays the path generated by SPIN [53].Although the paths are different, they both satisfy the specification formula.

Figure 2.11: Example 3: Path generated by NUSMV

29

Figure 2.12: Example 3: Path generated by SPIN

30

Example 4. Figure 2.13 is an example of a very large environment. This environmentincludes 1156 areas of interest (rooms) and its discrete abstraction consists of 9250 trian-gles. The specification for this example was ”start in the white room and go to both blackrooms”.

Even though this environment is very large, the computation time was a few secondsfor the triangulation, about 55 seconds for the path generation in NUSMV and around 90seconds for the controller synthesis of a path of 145 triangles in MATLABTM.

Figure 2.13: Example 4: Complex environment - Visit the two black areas

31

Example 5. Figure 2.14 is another example of a path satisfying a complex requirement.In this example the desired behavior was ”first go to Room 3 while avoiding Rooms 1,2 ,then go to Room 4 while avoiding Rooms 1,2, next go to Room 1 while avoiding Room 2and finally go to Room 2”

Figure 2.14: Example 5: go to Room 3 (avoid 1,2), then go to Room 4 (avoid 1,2),then goto Room 1 (avoid 2) and finally go to Room 2

32

2.5 DiscussionThis chapter describes an approach based on model checking tools for solving the problemof non reactive motion and task planning. This problem begins at a high level with com-plex behavior specification, expressed in temporal logic, and ends in creating continuouscontrol inputs for the robot, that satisfy those requirements. This section discusses someaspects of this method such as the guarantees it provides and its lack of optimality.

2.5.1 GuaranteesGiven a partition and a set of control laws that satisfy the bisimulation requirement, ifthe desired behavior can be achieved by the discrete system D then a discrete sequencesatisfying the specification will be found and the robot is guaranteed to continuously exe-cute the behavior such that it satisfies the desired task. Furthermore, if there is no discretesequence that satisfies the desired task, that means that the task cannot be be achieved bythe robot in the current partition.

When considering fully actuated point robots, there are several methods for generatingcontrol laws that ensure the partition is a bisimulation and thus the approach is sound andcomplete. However, for more realistic robots finding such controllers may be more chal-lenging. If the continuous motion system CMotion has a transition that is not representedin the discrete model DMotion, meaning the robot can continuously move between two ad-jacent cells but the discrete system cannot, then if a discrete solution exists (σ was found)the robot is guaranteed to behave correctly but if a discrete solution is not found, the taskmay still be done by the robot.

If the discrete system DMotion has a transition ri →DMrj that cannot be guaranteed

by the continuous system, meaning that there exists p ∈ T−1(ri) such that ∀p′ ∈ T−1(rj),p 6→CM

p′, there are no longer any guarantees for the behavior of the robot.

2.5.2 OptimalityOptimality of the behaviors generated can be measured in the continuous world as totalpath length, minimum number of action turned on, power consumption, etc. One can evendefine optimality measures for the discrete model by either optimizing the number of stepstaken to satisfy the task or adding costs to the transition and optimizing on that. In thisframework, since the discrete sequence σ is generated by way of counter example, thereare no optimality guarantees, continuous or discrete, which results in behaviors that areclearly non optimal as shown in Example 1.

One way of shortening the path is by using the notion of Bounded Model Checking(BMC) [20, 21]. When set for BMC, the model checker looks for a counter example ofincreasing length, up to a preset bound. In a nutshell, the model checker instantiates thetemporal formula using a large number of Boolean propositions and then solve a satis-fiability problem using powerful SAT solvers (for example [43]). While such a methodwill produce the shortest possible discrete sequence it is computationally very expensive

33

(doubly exponential in the number of state variables and the size of the formula [33]) andeven if found, such optimality is not easily converted to continuous optimality guarantees.

2.5.3 ScalabilityModel checkers are used regularly these days in the hardware design industry [16, 60]and they can handle very large system models [24]. This work utilized off the shelf, free,research systems that were not specially constructed for this work and all the examples,except for Example 4 were solved in a few seconds on a regular laptop.

While scalability does not seem to be a problem, one should keep in mind that thesystem model grows exponentially with the number of actions the robot can preform. Away of dealing with this is to encode some of the LTL formula in the system model, thusreducing the state space. For example if part of the LTL formula is 2(r1 ⇔ a1) reading“action a1 is active if and only if the robot is in r1” then all the states of the form (r1, ∅, . . . )or of the form (ri, a1, . . . ), i 6= 1 can be removed from the model D. However, such aprocedure may be hard to automate while the current exponential model is automaticallygenerated from a partition and a list of actions.

2.5.4 BehaviorThere are several factors that effect the continuous robot behavior. While the behavioris guaranteed to be correct, it may be non optimal, as discussed above, and it may besomewhat unexpected.

In Example 1 the robot turns on the camera in the cells it should (r2, r3, r4 and r5) butit also turns it on in other cells (r8, r9, r12 and r13). While correct, according to the speci-fication, it is probably not the intended meaning of the instruction “Turn on the camera inCells r2, r3, r4 and r5.” In this method, since there is no notion of a good solution, if a cer-tain behavior is required (such as conserving power and not turning actions on arbitrarily)it must be encoded explicitly in the LTL formula.

In the same example, the robot first visits Cell r2 however it visits it in a very non robustsense, meaning that the first time the robot’s position is in Polygon P2 the requirement ofvisiting that cell is satisfied. Thus if there is some localization error, the robot might missthe cell completely. This can be solved by using a robust version of the LTL formula [47]or by using controllers that make sure the robot enters deeper into a cell before determiningthat the cell was visited.

The choice of controllers decides how the motion of the robot will look. Some con-trollers lend themselves to smoother overall paths by matching the velocity vectors on thefacets of the polygonal cells. Some controllers tend to drive the robot along the edges ofthe cells [14] while others drive the robots closer to the center of the cell and cause thevelocity vector on the edge connecting the cells to be perpendicular to that edge [37].

34

2.5.5 Choosing the right partitionAs hinted before, choosing the “right” partition is very difficult yet important for thismethod. It must take into account regions of interest, such as rooms or hallways, and itmust correspond to a set of bisimilar controllers. Choosing a partition that is too fine resultsin a large discrete model and it may lead to continuous paths that are not smooth or makemany seemingly unnecessary twists and turns. Choosing it to be too coarse on the otherhand limits the expressive power since there are less cells that describe the workspace.Furthermore, if there are not enough cells, the task might be deemed impossible by thediscrete system even though with a different partition the robot could have achieved thedesired behavior. For example, if a hall connecting two rooms is considered one celland part of it is unsafe, the specification will require the robot to not enter that cell, thuspossibly preventing the robot from entering one of the rooms. If the hall was built up oftwo regions, one containing the unsafe area but the other still connecting the rooms, therobot would not have a problem.

35

Chapter 3

Reactive tasks

A reactive task is one in which the behavior of the robot depends on the high level infor-mation it gathers while preforming the task. It can be viewed as “closed-loop” behaviorsas opposed to the “open-loop” tasks described in Chapter 2 since here the behavior of therobot changes based on events occurring in its environment.

This chapter addresses the challenge of generating robot controllers that are guaran-teed to control a robot such that it satisfies a high level reactive task in “well behaved”environments.

3.1 Preliminaries

3.1.1 SynthesisIn general, synthesis is the problem of automatically generating a system from a set ofrequirements or specifications. Such a system is created from the requirements and thusis “correct by construction” and does not need to be verified. This can be seen as thedual problem to model checking because while model checking takes a system someonedesigned and verifies that it satisfies the requirements, the synthesis algorithm’s startingpoint is the requirements and as such, it is a much harder problem.

In the context of this thesis, given an LTL formula, the realization or synthesis prob-lem consists of constructing an automaton whose behaviors satisfy the formula if such anautomaton exists. In general, creating such an automaton is proven to be doubly expo-nential in the size of the formula [89]. However, by restricting to a special class of LTLformulas, an efficient algorithm recently introduced in [88] can be used. This algorithm ispolynomial O(n3) time, where n is the size of the state space.

3.2 Problem formulationSimilar to Chapter 2, the problem being considered in this chapter combines three compo-nents; a robot, an environment and a high level task.

36

3.2.1 RobotAs in chapter 2, a fully actuated, planar robot is moving in a polygonal workspace P withthe dynamics defined in 2.4, with an action set Action = {a1, a2, . . . , al} and with anaction activation policy act (2.5).

As before, the robot is assumed to have perfect localization and full control of itsactions. While no continuous timing constraints are allowed on the actions, they can havediscrete step constraints such as the camera has to be turned off for at least two steps1.

3.2.2 EnvironmentThe environment the robot is operating in consists of the physical workspace P of Section2. The partition of this workspace 2.6 is captured by the set of Boolean propositionsR = {r1, r2 . . . , rn} as before. In addition, the environment can be dynamic, i.e. itmay have things “happening” in it such as people moving around, items that can be seen,obstacles popping up, fires or floods. The robot of Section 3.2.1 is assumed to have a set ofsensors that can determine high-level or “semantic” knowledge about the environment itis working in. Thus it is assumed that a vision sensor can decide whether Nemo is seen, orwhether it is Marlin, a temperature sensor can decide whether a fire is burning, etc. Notethat the sensor behavior and estimation process is abstracted in this work as discussedfurther in Section 3.5.

3.2.3 TaskLet X be the set of propositions corresponding to the environment state, or more specifi-cally, to the information the robot gathers from the environment using its sensors. Let Ybe the set of propositions corresponding to the robot state, i.e. its position and actions.

X = {x1, . . . , xm} (3.1)Y = {R, Action} = {r1, . . . , rn, a1, . . . , al} (3.2)

These two sets of propositions are used to encode reactive tasks as LTL formulas.Following [88], a special class of linear temporal logic formulas is considered. These

special formulas are of the form

ϕ = (ϕe ⇒ ϕs) (3.3)

The subformula ϕe is an assumption about the sensor propositions, and thus aboutthe behavior of the environment. An environment is considered admissible if it alwayssatisfies the assumptions made about it encoded in ϕe. The formula ϕs represents thedesired behavior of the system.

1a step can be defined by the computation cycle of the system.

37

The formula ϕ is True if ϕs is True, i.e., the desired robot behavior is satisfied, orϕe is False, i.e. the environment did not behave as expected. This means that when theenvironment does not satisfy ϕe and is thus not admissible, there is no guarantee about thebehavior of the system, a point that will be reiterated in the following.

Both ϕe and ϕs have the following structure

ϕe = ϕei ∧ ϕet ∧ ϕeg (3.4)ϕs = ϕsi ∧ ϕst ∧ ϕsg (3.5)

where

• ϕei , ϕsi - Non-temporal Boolean formulas constraining (if at all) the initial value(s)for the sensor propositions X and system propositions Y respectively.

• ϕet - Represents the possible evolution of the state of the environment. It consistsof a conjunction of formulas of the form 2Bi where each Bi is a Boolean formulaconstructed from subformulas in X ∪ Y ∪©X , where©X = {©x1, . . . ,©xm}.Intuitively, ϕet constrains the next sensor values©X based on the current sensor Xand system Y values.

• ϕst - Represents the possible evolution of the state of the system. It consists ofa conjunction of formulas of the form 2Bi where each Bi is a Boolean formula inX∪Y∪©X©Y . This formula constraints the next values of the system propositionsbased on the current values of both the sensor and system propositions and the nextvalue of the sensor propositions2.

• ϕeg, ϕsg - Represent goal assumptions for the environment and desired goal specifica-tions for the system. Both formulas consist of a conjunction of formulas of the form23Bi where each Bi is a Boolean formula.

The formulas ϕet , ϕst represent safety assumptions and requirements, i.e. constraints that

must always hold, for example “you can never see the sun at night” or “never enter thehaunted room”. The formulas ϕeg, ϕ

sg on the other hand, represent liveness assumptions

and requirements that must become True sometimes (or eventually), for example “youwill eventually see Waldo” or “If you see Waldo eventually report back to the base station”.Note that the translation to English of the LTL formulas 23Bi states that Bi must becomeTrue infinitely often. The formula φ = (ϕeg ⇒ ϕsg) which is discussed in Section 3.3.1,is called a Generalized Reactivity(1) (GR(1)) formula.

For such temporal logic formulas this chapter provides a computational solution of thefollowing problem

2As explained in Section 3.3.1, the environment acts first therefore the behavior of the system can dependon the next environment move

38

Figure 3.1: The workspace of Example 6.

Problem 2. [Reactive task] Given a robot model (2.4), a robot action set Action, anenvironment decomposition (2.6), and a LTL formula ϕ (3.3), construct a control inputu(t) and an action activation policy act(t) so that the resulting robot behavior satisfiesthe formula in any admissible environment.

The following example is used throughout this chapter for illustration

Example 6. Consider a robot that is moving in the workspace shown in Fig. 3.1 consistingof 12 regions labeled 1, ..., 12 (which define the robot propositions {r1, . . . , r12}). Initially,the robot is placed either in Region 1, Region 2 or Region 3. In natural language, thedesired specification for the robot is: “Nemo can only be in Regions 1, 3, 5 and 8. Lookfor Nemo and if you find him, turn your video camera on and stay where you are. If hedisappears again, turn the camera off and resume the search.”

Nemo is part of the environment, therefore the set of sensor propositions contains oneproposition X = {sNemo} which becomes True if the robot sensor has detected Nemo.The assumptions about Nemo are captured by ϕe = ϕei ∧ϕet ∧ϕeg. The robot initially doesnot see Nemo, thus

ϕei = (¬sNemo) (3.6)

Since Nemo can only be sensed in Regions 1, 3, 5 and 8, the requirement that in otherregions the value of sNemo cannot change is encoded. This requirement is captured by theformula:

ϕet = 2((¬r1 ∧ ¬r3 ∧ ¬r5 ∧ ¬r8) ⇒ (©sNemo ⇔ sNemo)) (3.7)

There are no further assumptions on the environment propositions, which means that

ϕeg = 23(True) (3.8)

completing the modeling of the environment assumptions. Notice that the environment isadmissible whether Nemo is there or not.

Now for the modeling of the robot and the desired specification, captured by ϕs =ϕsi ∧ ϕst ∧ ϕsg. There are 13 robot propositions Y = {r1, · · · , r12, aCameraOn}. Initially, the

39

robot starts somewhere in Regions 1, 2 or 3 with the camera off, hence

ϕsi =

(r1 ∧i∈{2,··· ,12} ¬ri ∧ ¬aCameraOn)∨(r2 ∧i∈{1,3,··· ,12} ¬ri ∧ ¬aCameraOn)∨(r3 ∧i∈{1,2,4,··· ,12} ¬ri ∧ ¬aCameraOn)

(3.9)

The formula ϕst models the possible changes in the robot state. The first block ofsubformulas represent the possible transitions between regions; for example, from Region1 the robot can move to adjacent Region 9 or stay in 1. The next subformula capturesthe mutual exclusion constraint: that is, at any step, exactly one of r1,...,r12 is True. For agiven decomposition of workspace P , the generation of these formulas is easily automated.The final block of subformulas is part of the desired specification and states that if the robotsees Nemo it should remain in the region it is in at the next step3 and the camera should beon. It also states that if the robot does not see Nemo, the camera should be off.

ϕst =

∧

2(r1 ⇒ (©r1 ∨©r9))∧2(r2 ⇒ (©r2 ∨©r12))

...∧2(r12 ⇒ (©r2 ∨©r9 ∨©r11 ∨©r12))∧

2( (©r1 ∧i 6=1 ¬© ri)∨(©r2 ∧i 6=2 ¬© ri)...∨(©r12 ∧i 6=12 ¬© ri)∧

2(©sNemo ⇒(∧i∈{1,··· ,12}© ri ⇔ ri) ∧©aCameraOn)∧

2( ¬© sNemo ⇒ ¬© aCameraOn )

(3.10)

Finally, the requirement that the robot keeps looking in Regions 1, 3, 5 and 8 unless ithas found Nemo is captured by

ϕsg =

23(r1 ∨ sNemo)∧23(r3 ∨ sNemo)∧23(r5 ∨ sNemo)∧23(r8 ∨ sNemo)

(3.11)

This completes the modeling of the robot specification as well. Combining everythingtogether, the required formula ϕ = (ϕe ⇒ ϕs) is complete.

3The truth value of the region propositions does not change.

40

3.3 Generating reactive controllersThe solution to generating continuous robot behaviors satisfying complex high-level reac-tive tasks consists of the following three steps:

1. Creating ϕ: Using the workspace decomposition {Pi}, the robot dynamics (2.4),the sensor propositions X and robot propositions Y , construct the LTL formula ϕ =ϕe ⇒ ϕs that captures the assumptions about the environment as well as the desiredrobot behavior.

2. Discrete Solution: Generate a discrete automaton satisfying the desired specifica-tions using synthesis tools.

3. Continuous Implementation of Discrete Plan: Implement the discrete plan at thecontinuous level while taking into account the information gathered.

3.3.1 Automaton synthesisThe synthesis process is viewed as a game played between the system (robot) and theenvironment (as the adversary). Starting from some initial state, both the robot and theenvironment make transition to the state of the game. The winning condition for the gameis given as a GR(1) formula φ. The way the game is played is that at each step, first theenvironment makes a transition according to its transition relation and then the systemmakes its own transition. If the system can satisfy φ no matter what the environment does,then the system is winning and an automaton can be extracted for the robot. However, ifthe environment can falsify φ then the environment is winning and the desired behavior isunrealizable.

Relating the formulas of section 3.2.3 to the game mentioned above, the initial states ofthe players are given by ϕei and ϕsi . The possible transitions the players can make are givenby ϕet and ϕst , and the winning condition is given by the GR(1) formula φ = (ϕeg ⇒ ϕsg).Note that the system is winning, i.e. φ is satisfied if ϕsg is True, which means that thedesired robot behavior is satisfied, or ϕeg is False, which means that the environment didnot reach its goals (either because the environment was faulty or the system prevented itfrom reaching its goals). This implies that when the environment does not satisfy ϕeg thereis no guarantee about the behavior of the system. Furthermore, if the environment doesnot “play fair”, i.e. violates its assumed behavior ϕei ∧ϕet , the automaton is no longer valid.

The synthesis algorithm [88] takes the formula ϕ and first checks whether it is real-izable. If it is, the algorithm extracts a possible (but not necessarily unique) automatonwhich implements a strategy that the robot should follow in order to satisfy the desiredbehavior. The automaton that is generated by the algorithm can be modeled as a tupleA = (X ,Y , Q, q0, δ, γ) where:

• X is the set of input (environment) propositions

• Y is the set of output (system) propositions

41

• Q ⊂ N is the set of states

• Q0 ⊂ Q is the set of initial states

• δ : Q× 2X → 2Q is the transition relation, i.e. δ(q,X) = Q′ ⊆ Q where q ∈ Q isa state and X ⊆ X is the subset of sensor propositions that are True.

• γ : Q→ 2Y is the state labeling function where γ(q) = Y and Y ∈ 2Y is the set ofsystem propositions that are True in state q.

Note that this automaton can be nondeterministic4. An admissible input sequence is a se-quence X1, X2, ... ,Xj ∈ 2X that satisfies ϕe. A run of this automaton under an admissibleinput sequence is a sequence of states σ = q0, q1, .... This sequence starts at an initial stateq0 ∈ Q0 and follows the transition relation δ and the truth values of the input propositions,i.e. for all j ≥ 0, qj+1 ∈ δ(qj, Xj). An interpretation of a run σ is a sequence Y0, Y1, ...where Yi = γ(qi) is the label of the ith state in the run. This sequence of labels is usedto construct the discrete path the robot must follow as well as the activation policy act(t).As mentioned before, when given a non-admissible input sequence, i.e. an input sequencethat violates any part of ϕe, the automaton is no longer relevant and it may be impossibleto construct a correct path for the robot.

Example 6 continued: Figure 3.2 represents the synthesized automaton that realizes thedesired behavior. The circles represent the states of the automaton and the propositionsthat are written inside each circle are the state’s label, i.e. the output propositions that areTrue in that state. The possible initial states are shown as filled circles. The edges arelabeled with the sensor propositions that must be True in order for the transition to bemade. Edges with no labels are thus labeled with ¬sNemo.

As can be seen, the automaton causes the robot to stay where it is and turn the cameraon if it senses Nemo, otherwise the robot will keep searching for Nemo, with the cameraoff, forever. Note that this automaton is not unique and is nondeterministic. Furthermore,this automaton can only be executed if the environment (Nemo) is behaving according tothe assumptions. Thus, if the robot suddenly senses Nemo in Region 9 the automaton willnot have a suitable transition.

Caveat on completeness

For the synthesis algorithm to be complete, it must produce an automaton whenever theLTL formula ϕ = ϕe ⇒ ϕs can be satisfied. The algorithm [88], as pointed out by MarcoRoveri and Oded Maler, is not complete because if the system can cause the environmentto violate any of its assumptions ϕe at the price of violating its own safety constraints(possible moves ϕst ), the LTL formula is satisfied5 but the algorithm will return that the

4By making a small change in the algorithm, the automaton may become deterministic, i.e. for everyinput there will be a unique next state.

5The formula ϕe is False and so the implication ϕe ⇒ ϕs is True.

42

Figure 3.2: The synthesized automaton of Example 6

43

specification is unrealizable. An example for such a case is if the environment has as-sumptions that state “Waldo is never sensed” and “Infinitely often you sense Waldo if andonly if you are in Room 1” and the system is required to go to Room 1 if and only if itsees Waldo and furthermore, eventually go to Room 1. In this example, the environmentassumptions are True if the robot never goes to Room 1 (the implication is True) which iswhat it must do since Waldo is not present. However, if the robot does not go to Room 1 itdoes not satisfy its requirements and thus the specification is unrealizable. If, on the otherhand, the robot disregards the requirement that it go to Room 1 if and only if it sees Waldoand chooses to stay in Room 1 forever, the environment assumptions are falsified and thefull LTL formula is True.

While this incompleteness has theoretical importance, in this work it does not pose aproblem since the purpose of the plan is to cause the robot to behave in a certain way ina specific environment and not satisfy a logic formula. Even if the robot could satisfy theformula by violating its constraints and thus falsifying the environment assumptions, sucha plan would be unacceptable to the user since the robot does not do as it is “told”.

3.3.2 Hybrid ControllerIn order to continuously implement the discrete automaton of the previous section, a hybridcontroller is constructed that takes a set of atomic bisimilar controllers and composes themsequentially according to the discrete execution of the automaton, similar to Chapter 2.The difference between this execution and that of Chapter 2 is that the choice of whichatomic controller to invoke depends on the sensor information in addition to the currentstate.

The algorithm for executing the discrete automaton is shown in Algorithm 2. Initially,the robot is placed at position p0 in Region i such that ri ∈ γ(q0) where q0 is an initialautomaton state satisfying q0 ∈ Q0. Furthermore, based on all other propositions γ(q0),the appropriate robot actions are activated. At each execution cycle or step, the robotsenses its environment and determines the values of the binary inputs X . Based on theseinputs and its current state it chooses a successor state NxtState and extracts the nextregion NxtReg it should go to from γ(NxtState). Then it invokes an atomic controllerthat drives it towards the next region. If the robot enters the next region in this step thenthe execution changes the current automaton state, extracts the appropriate actions andactivates/deactivates them. If the robot is neither in the current region nor in the nextregion, which could only happen if the environment violated its assumptions or the robotwas subject to a catastrophic disturbance6, the execution is stopped with an error.

Note that in the type of automaton that is extracted here, i.e. one that always makesprogress towards the goals, an action might not be turned on/off simultaneously with thesensor input change. That is, the change in the action might only occur when the robotenters a new region. This may be avoided by extracting an automaton that does not have

6The atomic controllers are feedback controllers and as such are robust to small disturbances. Howeverif the robot is picked up and carried to a different region or is hit by a bigger robot, it may not recover.

44

to make progress at each step,

Algorithm 2 Continuous Execution of the discrete automatonprocedure EXECUTE(A, q0, p0)

CurrState← q0

CurrReg ← ri ∈ γ(q0)CurrOutput← {a1, a2, · · · } ∈ γ(q0)ActivateDeactivateOutputs(CurrOutput,Y)p← p0

while True do . Infinite executionInputV al← Sense()NxtState←

GetNxtState(A,CurrState, InputV al)NxtReg ← ri ∈ γ(NxtState)p← ApplyController(CurrReg,NxtReg, p)if p ∈ PNxtReg then . Automaton transition

CurrState← NxtStateCurrReg ← NxtRegCurrOutput← {a1, a2, · · · } ∈ γ(CurrState)ActivateDeactivateOutputs(CurrOutput,Y)

else if p ∈ PCurrReg then . No transitionContinue

elseERROR - No legal automaton state

end ifend while

end procedure

This kind of execution allows the robot to react in real time to changing environmentconditions by assessing at each step the current state of the environment and deciding whatto do next. Another way to execute the automaton is to gather information only when adiscrete transition was taken in the automaton, i.e. when the robot enters a different cell.This approach was taken in [68] and while it does not allow fast reactions to changingenvironments, if the sensor processing is expensive and there are no events that the robotmust immediately respond to, it can be chosen.

Example 6 continued: Figure 3.3 depicts a possible execution of the automaton syn-thesized in Example 6. Here the robot starts in Region 3 and searches for Nemo. In (a), therobot finds Nemo in Region 5 and therefore it stays there and turns on its camera, indicatedby the magenta squares. Then, as depicted in (b), Nemo disappears and the robot resumesthe search. It then continues to search all the regions of interest as seen in (c).

45

(a) Nemo is sensed in Region 5 (b) Nemo disappeared therefore the robot contin-ues the search

(c) The robot continues to search Regions 1, 3, 5and 8

Figure 3.3: A possible run of the automaton of Example 6

46

3.4 ExamplesThis section presents several examples of different behaviors, the automata that implementthem and the trajectories which they induce. The automata presented in here were createdusing an implementation of the synthesis algorithm written by Piterman, Pnueli and Sa’ar[88] on top of the TLV system [90]. All simulations were done using MATLABTM.

3.4.1 Nursery scenarioExample 7. The robot is moving around a workspace (nursery) with ten regions. It cansense whether a baby is crying and it can recognize adult care givers. The desired behavioris: “Starting in Region 1, keep checking whether a baby is crying in Regions 2 or 4. If youfind a crying baby, go look for an adult in Regions 6, 7 and 8. Keep looking until you findhim or her. After finding the adult, go back to monitoring the babies and so on...”

One can define two environment (sensor) propositions here, one indicating a cryingbaby was sensed and another indicating an adult was found. In order to reduce the numberof variables, the computation time and the size of the automaton, one environment propo-sition, CkBby, is used to indicate whether the robot should check on the babies (whenthe proposition is True) or go look for an adult (when the proposition is False). InitiallyCkBby is True. It is assumed that the proposition becomes False in Regions 2 and 4 if therobot senses a baby crying and once it becomes False it stays False as long as it is in 2 or4 (a baby does not stop crying on her own and she cannot be ignored). Furthermore, it isassumed that CkBby becomes True in Regions 6, 7 and 8 only if the robot sensed an adult.Once it becomes True it stays True in these regions (once the adult was found, the robotmust return to check on the babies). In all other regions, the truth value of the propositionmay not change.

Following these assumptions, ϕe is constructed:

ϕe =

CkBby∧2(((r2 ∨ r4) ∧ ¬CkBby)→ (¬© CkBby)))∧2(((r6 ∨ r7 ∨ r8) ∧ CkBby)→ (©CkBby)))∧2(¬(r2 ∨ r4 ∨ r6 ∨ r7 ∨ r8)→ (©CkBby ↔ CkBby))∧

23True

(3.12)

As for the robot, there are ten system propositions r1, ..., r10, one for each region.Constructing ϕs:

ϕs =

r1 ∧i=2,...,10 ¬ri∧

Transitions∧

Mutual Exclusion∧i∈{2,4}23(ri ∨ ¬CkBby)∧i∈{6,7,8}23(ri ∨ CkBby)

(3.13)

47

The first and second lines encode the initial condition, possible transitions and mutualexclusion requirement as in Example 6. The rest of the formula describes the desiredbehavior, for example, the third line requires the robot to infinitely often either visit regioni, i ∈ {2, 4} or look for an adult.

Running this example through the synthesis algorithm, the computation time was 2seconds and the automaton that realizes this specification has 41 states. Sample simula-tions are shown in Figure 3.4.

(a) Babies are not crying (b) A baby in region 4 cries and the adult is inregion 8

Figure 3.4: Nursery Example

3.4.2 Animal HerdingOne of the strengths of this approach is the ability to generate an automaton from a largeset of simple instructions or rules. The automatically generated automaton is guaranteedto obey all given rules, if they are consistent, as opposed to a hand coded automaton thatmight contain a hidden and esoteric bug. Furthermore, if the set of rules is inconsistent,the algorithm will stop without creating the automaton, indicating that the desired rule setcannot be implemented.

Example 8. A robot is moving in a workspace that has 35 connected regions as shown inFigure 3.5. The robot’s goal is to herd dogs, cats and mice to their respective meetingpoints, denoted on the figure by Dogs, Cats, Mice. In order to do that, the robot can showa bone to the dogs, trail a piece of yarn for the cats and play a flute for the mice. Theanimals can appear anywhere in the workspace other than the meeting places and whilean animal is being herded it might choose not to follow the robot, and thus it may no longerbe sensed by the robot. Further restrictions are imposed on the behavior of the robot:

1. The robot cannot herd a dog and a cat at the same time.

2. The robot cannot herd a cat and a mouse at the same time.

48

3. Unless there are no animals, the robot must herd at least one animal at any giventime.

4. The robot cannot herd a cat through the “NoCats” regions.

5. The robot cannot herd a mouse through the “NoMice” region.

6. The robot must start herding a dog when it senses one, unless it is already herdinga cat.

7. When sensing a dog and a cat, the dog is to be herded.

8. When sensing a cat and a mouse, the cat is to be herded.

9. The robot must start herding a cat when it senses one, unless it is already herding amouse or it senses a dog as well.

10. The robot should continue herding an animal until it reaches its meeting region or itdisappears, and not switch between animals randomly.

In the original set of rules, Restriction 9 stated that “the robot must start herding a catunless it is already herding a mouse” without any mention of a dog that may be spottedat the same time as the cat. This is inconsistent with Requirements 1 and 6 therefore noautomaton was generated.

In order to encode this behavior, the set of sensor propositions was X = {sdog, scat,smouse} and the set of robot propositions was Y = {r1, · · · , r35, abone, ayarn, aflute}7. Someof the requirements are shown below.

All the above restriction are encoded as part of ϕst , including Restrictions 1, 3, 5 and6:

...∧2(¬(©abone ∧©ayarn))

...∧2( (©sdog ∨©scat ∨©smouse)⇒ (©abone ∨©ayarn ∨©aflute) )

...∧2(©aflute ⇒ ¬© r20))∧

2( (©sdog ∧ (¬(ayarn ∧©scat))) ⇒ ©abone)...

(3.14)

where Region 20 is the “NoMice” region.

7When coding this example and the next one, the regions are encoded as binary vectors instead of creatinga separate proposition for each region.

49

The robot must search the space for animals, and once it encounters one it must herd itto its meeting region. This requirement is encoded as part of ϕsg, where the search of eachregion is encoded as

23(ri ∨ sdog ∨ scat ∨ smouse) (3.15)

meaning Region i should be searched unless an animal is sensed, and the herding is en-coded as:

23(r1 ∨ ¬abone)∧23(r32 ∨ ¬ayarn)∧23(r36 ∨ ¬aflute)

(3.16)

meaning the robot should go to the dogs’ (cats’, mice’s) meeting region if the robot isshowing a bone (trailing yarn, playing the flute), i.e. herding a dog (cat, mouse).

Figure 3.5 shows a sample run of the generated automaton. The robot starts by search-ing the space. It encounters a cat (a) and proceeds to trail yarn, indicated by the bluedots. It herds the cat to its meeting region (b) while ignoring dogs that are present alongthe way. Furthermore, the robot causes the cat to reach its meeting region without goingthrough the “NoCats” regions. Immediately after leaving the cats’ meeting region, therobot encounters a dog and proceeds to herd it to its meeting region by showing it a bone,indicated by red dots. Along the way it finds a mouse (c) and starts playing the flute aswell, indicated by the red stars. The robot, now herding both a dog and a mouse, first goesto the dogs’ meeting region (d), drops the dog there and continues, with the mouse, to themice’s meeting region. Along the way it sees another dog (e) and after dropping the mouseoff it takes the dog to its meeting place (f).

50

(a) A cat is found. (b) The robot herds the cat to its meeting region,without going through the NoCats regions, bytrailing yarn. It ignores dogs along the way.

(c) A dog is sensed immediately after leavingthe Cats meeting region. It is then herded to theDogs region. A mouse is sensed along the way.

(d) The robot herds both the dog and the mouseto the Dogs meeting region.

(e) The mouse is herded to its meeting region. Adog is sensed.

(f) The mouse stays at its meeting region and thedog is being herded.

Figure 3.5: Animal herding

51

3.4.3 Visiting Regions in SequenceAnother strength of this approach is the ability to satisfy a sequence of goals.

Example 9. In this example, depicted in Figure 3.6, the robot is to drive to seven regionsin a specific order without going through any gray region more than once. Furthermore,the robot cannot go through any of the goal regions until it has gone through all previousgoal regions and the robot can start at any of four initial regions, denoted by red ‘I’s inthe figure.

Note that this behavior is non reactive, i.e. there is no sensor input that may change thebehavior of the robot. This type of task can be solved by applying Dijkstra’s algorithm;however, the requirement that some regions may be traversed at most once would requiremultiple runs of the algorithm with different graphs and extra logic. The model checkingapproach of Chapter 2 can be used for a specific initial condition however that algorithmwould have to be run once for every initial condition as opposed to the single automatongenerated here.

As mentioned above, this task is non reactive therefore there is no sensor input. Tokeep the structure of the LTL formula a dummy sensor proposition is created and it is set toalways be False. The robot propositions for this example include the region propositions{r1, · · · , r168}. Two additional types of robot propositions, WIi, i ∈{ Regions to betraversed at most once} and Vi, i ∈{ Goal regions} are defined.

The propositions WIi and Vi are used to “remember” whether the robot has alreadydriven through Region i. Such propositions are initially set to False and once the robotpasses through region i, the corresponding WIi or Vi is set to True and remains Truethereafter. These two propositions are distinguished for clarity as the way they affect therobot behavior is different.

To prevent the robot from going twice through Region i that can be driven through atmost once the following formula is added to ϕst :

2(WIi ⇒ ¬© ri) (3.17)

The following formula enforces the order of the goal regions to be visited :

2( (¬Vg1 ∨ · · · ∨ ¬Vgi) ⇒ ¬© rgi+1

) (3.18)

Which states that the i+1 goal region to be visited cannot be entered if the previous i goalregions were not visited.

Figure 3.6 shows a run of the generated automaton from one initial state. As can beseen, the gray regions are driven through at most once and the regions are visited in order.

52

(a) (b)

(c) (d)

Figure 3.6: Visiting regions in sequence

53

3.5 DiscussionThis chapter describes an approach based on synthesis of specifications into systems forsolving the problem of reactive motion and task planning. This problem begins at a highlevel with complex behavior specification, expressed as a linear temporal logic formula ofspecific structure, and ends in creating continuous control inputs for the robot, that satisfythose requirements, if the environment “behaves” as expected, i.e. it is admissible. Thissection discusses some aspects of this method such as the guarantees it provides and thedifferent trade-offs. Some of the aspects discussed in Chapter 2 are relevant here as well,such as choosing the right partition (see Section 2.5.5) and what it means if the partition isnot a bisimulation (see Section 2.5.1).

3.5.1 GuaranteesThere are several guarantees of correctness for this approach, starting from the high levelspecifications and going down to the low level controls. First, given the high level specifi-cation encoded as an LTL formula, the synthesis algorithm reports whether the specifica-tion is realizable or not. If not, then the algorithm returns that there exists no automatonthat can satisfy the desired behavior. A specification is unrealizable if it contains an incon-sistency, such as, “always keep moving and if you see a stop light stop,” if a specificationrequires an infeasible move in the discrete abstraction of the workspace, such as “alwaysavoid the hall outside the room and eventually leave the room,” or if the environment canprevent the robot from achieving its task, as mentioned below.

Second, given a realizable specification, the algorithm is guaranteed to produce an au-tomaton such that all its executions satisfy the desired behavior if the environment behavesas assumed. The construction of the automaton is done using ϕet which encodes admissibleenvironment behaviors; if the environment violates these assumptions, the automaton is nolonger correct. The continuous motion of the robot which is controlled by the activationof the atomic feedback controllers is guaranteed to be correct unless the robot is subjectto a catastrophic disturbance (e.g., an out of control truck). Modulo a disconnect betweenϕet and the environment, or a catastrophic disturbance to the continuous dynamics, this ap-proach leads to a correct continuous execution of the automaton that satisfies the originalhigh level desired behavior.

Realizability

Note that the synthesis algorithm takes the “worst case” approach in which the environ-ment in the adversary and as such is doing everything in its power to make sure the robotdoes not satisfy its desired behavior. Only if the environment cannot prevent the robot fromreaching its goals the specification is considered realizable and an automaton is extracted.

This conservative approach has the strength of being able to guarantee that what everthe environment does, as long as it admissible, the robot can still achieve its task. However,in order to synthesize automata, sometimes the user must supply additional assumptions

54

about the fairness or infinite behavior of the environment, as will be shown in Chapters6 and 7. These assumptions can be either common sense assumptions such as things donot move by themselves (as in Section 6.3.2) or made up sensor propositions to indicateevents that must become True at some unspecified time in the future (as in Section 7.2).

3.5.2 OptimalityCurrently, the synthesis algorithm produces an automaton that either achieves the goalsspecified in ϕsg cyclically in the order they are written or prevents the environment fromreaching its goals, specified in ϕeg. The automaton is constructed such that when satisfyingthe robot goals, it reaches these goals in a minimal number of steps.

Since the goals are satisfied in the order they were written the robot behaviors areoften suboptimal. Furthermore, an optimal order of goals for one environment might stillbe non optimal for another. For example, if the goal is to go through three rooms in order(assuming all rooms are connected) or stop, written as

ϕsg = 23(r1 ∨ stop) ∧23(r2 ∨ stop) ∧23(r3 ∨ stop)

then a possible execution would be to first go to Room 1, then stop (something happenedthat made the robot stop based on another part of the specification) then go to Rooms3,1,2. Room 2 is eventually reached but only after Room 1 is visited again. This behavioris correct but not optimal in the sense of visiting the rooms8.

Even if the automaton is optimal in terms of discrete steps, it might exhibit non optimalcontinuous behaviors. If the region sizes are not uniform or if there are tight corners totraverse a longer discrete path may result in a better continuous one. Looking at how to addcost to the automaton extraction and how to define what optimality means in this contextis a future research direction.

3.5.3 ScalabilityIn general, as noted above, the time complexity of synthesizing an LTL formula is doubleexponential in the length of the formula [89]. While there is an implementation of such analgorithm (Lily [56,57]) it is limited to about ten propositions and fifteen conjunctions. Forthe specific fragment considered in this work (Section 3.2.3) the synthesis is polynomialin the state space. The state space is, at the worst case, exponential in the number ofpropositions since a state is defined by a legal combination (one that does not violate anyof the specifications) of truth values to the sensor and robot propositions9. This synthesisalgorithm is usually O(n3) where n is the size of the state space but if the environmenthas no goals, i.e. ϕeg = 23(True) then the algorithm’s complexity is O(n2). Using animplementation of the synthesis algorithm on the JTLV system [100], an automaton with

8This specific problem can be solved with a more complex specification but other examples might not.9The automata defined in Section 3.3.1 are labeled with only the robot propositions but a state can be

viewed, as in [88], as the label of the state together with the label of the transition leading to that state.

55

50,000 states was created and by making further changes to the implementation, largerautomata can be generated.

Trade-off: expressivity

While the fragment of LTL considered here is more restrictive than the full LTL, theredoes not seem to be a significant loss of expressivity in terms of the tasks that can bespecified. Anything that can be encoded using a deterministic Buchi automaton10 can bewritten using this structure.

An example for a specification that cannot be captured is

32φ

read as “eventually always φ”. This formula states that from some unknown time in thefuture, φ must always be True. Instead, one can encode

2((s ∨ φ)⇒©φ)

read as “always, if s or φ are True then φ is True in the next state”. This formula, ifrealizable, will cause φ to become True once s is True and it will stay True from then on.Here s can be a random sensor input that will eventually become True and so a similar yetnot identical behavior is achieved.

Trade-off: assumptions Vs. size

As mentioned multiple times throughout this chapter, the assumptions made about theenvironment or sensor behavior must be satisfied by the environment during runtime oth-erwise the automaton will no longer be valid. However, these assumptions can be verygeneral and in fact they can allow the environment to exhibit any behavior by setting

ϕe = True ∧2True ∧23True

The trade-off the user must face between the assumptions and computation time andsize is the more restrictive the assumptions, the smaller the state space (less legal combi-nations of sensor propositions), computation time and generated automaton. On the otherhand, the more general the assumptions the less likely it is the environment will falsifythem which would cause the robot to terminate its execution.

10A deterministic Buchi automaton is a deterministic automaton with accepting states that accepts aninfinite input sequence if this input sequence causes the automaton to go through some accepting statesinfinitely often. In the context of finite robot tasks, if the task can be captured by a deterministic automaton,it can be encoded using this LTL fragment.

56

Trade-off: sensor logic Vs. specification

There is a rather philosophical question regarding where the information gathering logicshould be, in the sensor modules or in the synthesized task automaton. If all of the logic isin the robot’s task automaton, the composed behavior of the sensor and the robot is morelikely to work well. This comes at the price of a larger automaton and a more complex setof specifications. On the other hand, putting the logic governing the high level informationgathered from the environment in the sensor would require an extra step of verification tomake sure the logic and the assumptions encoded in the robot’s automaton are consistent.This trade-off is a direction for future research.

3.5.4 BehaviorThe automata that are generated using the synthesis algorithm of [88] are such that achievethe goals in ϕsg in order. For example if the goals are

ϕsg = 23r1 ∧23r5 ∧23r3

then the automaton will drive the robot first to Cell P1 then to Cell P5 and finally to Cell P3.If care is not taken when writing out these goals the robot may display behavior that is farfrom optimal (if there is no required ordering of the goals) as mentioned in Section 3.5.2.Furthermore, there are several ways to extract automata using the synthesis algorithm. Itcan be deterministic or it can be nondeterministic. In the nondeterministic case, it caneither always try to achieve the next goal with the minimum number of discrete transitionsor it can just guarantee that some progress is made towards the goal. All of the examplesdescribed in this thesis have nondeterministic automata that achieve the next goal withthe minimum number of transitions but if the other type of nondeterministic automata iscreated, additional unexpected behaviors can be seen.

The automaton can be constructed such that one of the strategies it employs for therobot is to cause the environment to falsify its assumptions ϕeg. For example if the as-sumption states “ Waldo is never in Room 1 and infinitely often you sense Waldo” thenthe robot by staying only in Room 1 makes the assumptions False, thus satisfying the for-mula ϕe ⇒ ϕs. This behavior, while correct in terms of the logic, is usually not whatthe user intended. A simple modification to the synthesis algorithm would remove such astrategy. Note that in most of the Examples throughout this thesis the formula ϕeg is set toTrue therefore there is no strategy to falsify the environment and such behaviors are notpresent.

Sensors, or more specifically, the binary inputs used by the automaton, are of great im-portance in this framework. First, as mentioned above, they must satisfy the assumptionsmade about them in the LTL formula, otherwise the automaton will not be correct. Sec-ond, even if they do satisfy these assumptions, they may still cause correct yet unintendedbehavior such as deadlocks (getting stuck in one state forever) or livelocks (changing statebut making no progress towards the goal). The issue of deadlocks and livelocks is dis-

57

cussed further in Section 6.4. Verifying sensor models and making sure they will notcause deadlocks or livelocks is a hard problem and is a future research direction.

As noted in Section 2.5.4, the choice of controllers determines the smoothness of thecontinuous robot motion.

58

Chapter 4

Complex Dynamics

Chapters 2 and 3 describe methods for creating robot controllers that are guaranteed todrive a simplified robot model (fully-actuated point) such that it achieves a high levelnonreactive/reactive task. While the simple robot model may have velocity bounds, it ismainly adequate for simulations but is harder to translate to real physical robots. ThisChapter discusses how richer and more complex robot models can be handled using thesame tools of the previous chapters.

4.1 Discrete Abstraction of MotionThe only difference between dealing with a fully-actuated point robot, as in Chapters 2 and3, and a robot that may have complex dynamics, nonholonomic constraints and a body (nota point robot) is in how the motion is abstracted. Recall that for both the nonreactive taskswhere a discrete robot model is first created and the reactive task where the discrete motionconstraints are captured in the specification, the partition of the workspace served as abisimulation and as such the adjacency graph of the cells served as the discrete abstractionof the motion.

Moving to more complex robot models, there are two approaches that can be taken. Inthe first, described in Section 4.1.1 and 4.2, atomic controllers that take into account dy-namics and body size are created. These controllers cover the workspace or configurationspace of the robot and their connectivity defines the possible discrete transition of the sys-tem. In the second, briefly explained in Section 4.1.2, the task controllers are created forthe simple model but by using error bounds on the complex system the original guaranteesstill hold.

4.1.1 Controllers for complex dynamicsThe methods of Chapters 2 and 3 are essentially robot model independent as long as thediscrete abstraction of the motion and the decomposition of the workspace are bisimilar,meaning that the continuous system can match any discrete transition and vice versa. In the

59

previous chapters the workspace P was partitioned because it is easier to specify tasks interms of disjoint regions of space. Also, there are many approaches for creating controllersthat cause the partition and the discrete abstraction of the motion to be bisimilar for asimple kinematic point robot [14, 37, 77].

If, for a more complex robot model, there are controllers that allow the workspace tobe partitioned and still retain the bisimulation property, the methods of Chapters 2 and3 remain unchanged. For example [78] describes controllers that drives a point robotwhile allowing path curvature constraints. However, if a bisimular partition cannot becreated, one can still create a bisimilar discrete abstraction by defining a set of cells andcorresponding controllers that satisfy:

i) Cells lie completely in the free configuration space of the robot

ii) Under influence of a given controller the robot trajectory must not depart the cell exceptvia a specified goal set

iii) The system must reach the designated goal set in finite time

iv) The controller must have efficient tests for domain inclusion given a known configu-ration [35]

The discrete graph representing the motion is created by checking for every two cellswhether the goal of one is a subset of the domain of the other. For two cells Pi and Pj , ifthe goal(Pi) ⊂ Pj then the discrete robot motion graph has a transition from ri to rj .

If the goal set of one cell only intersects the domain of another cell (but is not a subset,meaning some of the goal set is in PC

j = P\Pj , the complement of Pj), i.e. goal(Pi) ∩PCj 6= ∅, a transition is not created since there are some configurations that cannot be

guaranteed to reach the next cell. One could create a nondeterministic discrete abstractionto handle such cases however this is a direction for future reseach and is not in the scopeof this thesis.

When dealing with cells that overlap, it is likely that regions of interest belong toseveral cells therefore care needs to be taken when writing the specification. For exampleif a certain region must be avoided, all cells that intersect that region must be avoided aswell.

4.1.2 Robust temporal logicAnother approach, as developed in [44,47], is to first design a control law that enables thecomplex model to track a simpler kinematic model with a globally bounded error, guaran-teeing that the trajectory of the complex model is always at most δ away from the kinematictrajectory. Then the cells of the original workspace partition are either contracted by δ (ifthe cells must be visited by the robot) or expanded by δ (if the robot must always avoidthe cell) thus ensuring that if the simple model goes through a cell, so will the complex

60

model since it is always at most δ away1. These augmented cells are used to define thediscrete motion of the robot (their adjacency information). Finally a controller is createdusing the discrete abstraction of the motion, as in Chapters 2 and 3, that drives the simplemodel such that it achieves the task, if possible. The controller for the complex model isthe composition of the tracker of the first step and the controller of the final step. Notethat, although not addressed in [44, 47], this method can also handle robots with bodies(not point).

The following is an example taken from [47]:

Example 10. Consider a robot with dynamics defined by x = u moving in the workspaceshown in Figure 4.1(a). The specification is ”Starting in Cell π1, visit Area π2, then Areaπ3, then Area π4 and, finally, return to and stay in Region π1 while avoiding Areas π2 andπ3.

Figure 4.1(b) shows the cells after being adjusted to δ = 1. The yellow (light gray)cells are the cells that need to be visited therefore they were contracted by δ. The red (darkgray) represent the cells that are expanded by δ to indicate areas that should be avoided.Figures 4.1(c,d) depict the trajectories of the simple kinematic model and the dynamicmodel.

If δ is too big, meaning the complex systems does not track the kinematic model well,it may be the case that the achievement of the task cannot be guaranteed, as shown inFigure 4.1(e). There, the robot cannot reach its goal regions while avoiding the areas.

4.2 Example - Parking CarAs an example of the use of controllers designed for a complex robot model a conven-tional Ackermann steered vehicle operating in an urban environment is considered. Figure4.22 shows the environment, and the results of one simulation run. This run is a continu-ous execution of an automaton that satisfies the high-level specification “drive around theenvironment until you find a free parking space, and then park.”

4.2.1 DynamicsThis example focuses on the control of a rear-wheel drive car-like vehicle with Ackermannsteering. It is shown schematically in 4.3 and it is sized based on a standard “mini-van.”The vehicle has four wheels in contact with the ground. The two rear wheels provide themotive force via traction with the ground; the two front wheels provide steering.

The vehicle pose, g, is represented as

g = {x, y, θ} (4.1)

1Thus, robust LTL.2All figures in this chapter were created by David C. Conner for [36], as were the simulations.

61

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

x1

x 2

0

1

2

3

4

(a) Workspace for Example 10

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

z1

z 2

ξ¬ π

0

ξπ

1

ξπ

2

ξπ

3

ξπ

4ξπ

0

ξπ

1

ξπ

2

ξπ

3

ξπ

4

ξ¬ π

1

ξ¬ π

2

ξ¬ π

3

ξ¬ π

4

Z

b

c

d

a’

b’ c’

d’

Z0

a

(b) Contracting and expanding the cells

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

z1

z 2

(c) Kinematic model trajectory

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

x1

x 2

(d) Original model trajectory

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

z1

z 2

(e) δ is too big, task cannot be achieved

Figure 4.1: Example 10 - robust LTL

62

Figure 4.2: Workspace for Section 4.2The environment has 40 parking spaces arranged around the middle city block. The

high-level specification encodes “drive around until you find a free parking space, andthen park.” This example shows the path taken from the time the vehicle enters the area

until it parks in the first open parking space it encounters. There are four remaining openspaces.

63

Figure 4.3: Car-like system with Ackermann Steering.The inputs are forward velocity and steering angle velocity.

is the where (x, y) is the location of the midpoint of the rear axle with respect to a globalcoordinate frame, and θ is the orientation of the body with respect to the global x-axis.The angle of the steering wheel is φ ∈ (−φmax, φmax), a bounded interval.

The nonholonomic constraints inherent in the rolling contacts uniquely specify theequations of motion via a non-linear relationship between the input velocities and the bodypose velocity. Let the system inputs be u = {v, ω} ∈ U , where U is a bounded subsetof IR2, v is the forward velocity, and ω is the rate of steering. The complete equations ofmotion are

xy

θ

φ

=

cos θ 0sin θ 0

1L

tanφ 00 1

[vω]

(4.2)

The system evolution is also subject to configuration constraints. The body pose isconstrained by the interaction of body shape with obstacles in the environment. The poseis further constrained by local conventions of the road, such as driving in the right lane.The system inputs are constrained based on speed limits in the environment and systemcapabilities.

4.2.2 Atomic ControllersThe navigation tasks are defined by vehicle poses that must be reached or avoided; there-fore, the cells are defined in the vehicle pose space. Each cell has a designated region ofpose space that serves as the goal set. Over each cell, a scalar field is defined that specifiesthe desired steering angle such that steering as specified induces motion that leads to thegoal set. Taking the steering angle derivative with respect to body pose gives a reference

64

(a) (b)

Figure 4.4: Control policy based on [11]a) workspace path with local frame defined, b) the cell boundary forms a “tube” around

the curve in pose space. The sliding surface is shown in the cell interior.

steering vector field over the cell. The resulting controllers satisfy the four requirementsgiven in Section 4.1.1.

The approach to defining the cell boundary and desired steering angle is based on avariable structure control approach [11]. The cells are parameterized by a path segmentin the workspace, as shown in 4.4-a. The path is lifted to a curve in body pose space byconsidering the path tangent vector orientation as the desired orientation. One end of thecurve serves as the goal set center.

The controller defines a boundary in the local frames along the path. 4.4-b shows thecell boundary defined by the local frame boundaries along the path; the interior of this“tube” defines the cell. The size of the tube can be specified subject to constraints inducedby the path radius of curvature and the vehicle steering bounds. The cell can be tested forcollision with an obstacle using the technique outlined in [35].

For a detailed description of the control policy derivation, the reader is referred to[35, 36, 66].

4.2.3 Discrete abstraction of motionTo set up the basic scenario, the urban parking environment shown in 4.2 is defined basedon a “green practices” guideline for narrower streets3. The regularity of the environmentallows an automated approach to controller deployment.

First, a cache of atomic controllers is specified based on the different path segmentsthat can be used. The cache uses a total of 16 controllers: one normal traffic flow, fourassociated with left and right turns at the intersections, six associated with parking, andfive associated with leaving a parking space. Ten of the controllers move the vehicle

3http://www.nahbrc.org/greenguidelines/userguide site innovative.html

65

Figure 4.5: Parking behavior induced by the composition of atomic controllers.The feedback controllers guarantee the safety of the maneuver.

forward, and six move the vehicle in reverse. Each controller in the cache is definedrelative to a common reference point. At this point, the specification of the free parametersfor each controller in the cache is a trial and error process that requires knowledge of theenvironment, the desired behaviors, and some engineering intuition. During specificationof the controllers, it is verified that the convergence and invariance properties are satisfied,and that the cells are free of obstacles based on the road layout.

Controllers from the cache are then instantiated at grid points defined throughout theroadways. This is done offline based on knowledge of the local roadways. The instantia-tion process selects a subset of the controllers in the cache based on the grid point location.Given the cache and specified grid points, the instantiation process is automated. Nor-mally, the test for obstacle collision would be conducted as the controllers are deployed,but the regularity of the roadway renders this unnecessary. For intersections, the four turn-ing controllers are deployed for each travel direction, along with the basic traffic flow one.For straight traffic lanes, the grid points lie in the middle of the traffic lanes aligned withthe front of the parking space markers; orientation is defined by the traffic flow.

If a potential parking space is adjacent to the grid point, a special parking controlleris instantiated. Although considered a single controller by the automaton, each parkingcontroller is actually composed of several atomic controllers from the cache. The park-ing component controllers are only instantiated if the parking behavior is invoked by theglobal parking automaton; at this point execution control switches to a local parking con-troller encoded as a partial order of the parking controllers. Figure 4.5 shows an exampleparking maneuver induced by the composition of the local feedback controllers. For theregion defined in 4.2, there are a total of 306 instantiated controllers, including 40 parkingcontrollers associated with the 40 possible parking spaces.

66

As part of the instantiation process, cells are pairwise tested for goal set inclusion. Ifthe goal set of one cell is contained in the domain of a second, the first is said to preparethe second [25]. This pairwise test defines the prepares graph, which encodes the discretetransition relation between cells. The atomic controllers in the cache are specially definedso that cells instantiated at neighboring grid points prepare one another appropriately.

4.2.4 Parking formulaIn the basic scenario, a vehicle is searching for an empty parking space, and parks once itfinds one; therefore there is one input, park, which becomes True when an empty parkingspace is found. The controller, Φi, to be activated is an output4.

Assumptions on the environment

Initially there is no parking near the vehicle therefore

ϕei = ¬park (4.3)

The vehicle can only determine whether there is a free parking space if it is in a cellnext to it. This means that park cannot become True if the vehicle is not next to a parkingspace or in one. Also, for implementation reasons, an assumption is made that the inputpark remains True after parking.

ϕet =

2( [ (¬(∨i∈ParkCellΦi)) ∧

(¬(∨j∈PreparesParkCellΦj))]⇒ ¬© park )∧

2( (park ∧ (∨i∈ParkCellΦi)) ⇒ ©park )

(4.4)

There are no assumptions on the infinite behavior of the environment (it is not assumedthat there is an empty parking spot), therefore the goal component is set to True,

ϕeg = 23(True) (4.5)

Constraints on the behavior of the vehicle (system)

Initially the vehicle must be in the domain of an initial cell,

ϕsi = ∨i∈InitialCellΦi (4.6)

4For ease of reading, a different output is defined for each controller/cell. In the actual implementationthe cell numbers are encoded as binary vectors.

67

The allowable transitions are encoded as

ϕst =

∧i 2(Φi ⇒ (©Φi ∨j∈SuccessorsOfCelli ©Φj))∧i∈ParkCell 2(¬© park ⇒ ¬© Φi)∧2(©park ⇒ (∨i∈ParkCell© Φi) )

(4.7)

The first line encodes the transitions of the prepares graph from 4.2.3. The vehicle cannotpark if there is no parking space available, as indicated by the park input on the secondline. The third line states that if there is an empty parking space, the vehicle must park;removing this line may allow the vehicle to pass an open spot before parking. Additionaloutputs can be added to the transition relation. For example, controllers that cause left orright turns can trigger appropriate signals. The initial signal status should also be set inϕsi .

Finally for the goal, a list of cells the vehicle must visit infinitely often if it has notparked yet is added, thus

ϕsg = ∧i∈V isitCell23(Φi ∨ park) (4.8)

These cells define the area in which the vehicle will look for an available parking space.Note that the goal condition is True if either the vehicle visits these cells infinitely often(when there is no parking space available) or it has parked.

4.2.5 Continuous execution of discrete automataThe synthesis algorithm described in Chapter 3 generates an automaton that governs theexecution of the atomic controllers; however, the continuous evolution of the system in-duced by the controllers governs the state transitions within the automaton. The continuousexecution follows the steps of Algorithm 2 but now the cells may overlap (this does notchange the execution).

4.2.6 SimulationThe approach is verified in a simulation executed using MATLABTM. First, the workspaceis laid out, and a cache of cells/controllers is specified. Second, the cells are automaticallyinstantiated in the configuration space of the vehicle, and the prepares graph is defined.Next, based on the desired scenario, an LTL formula is written. The LTL formula is thengiven to the automatic synthesis algorithm implemented by Piterman, Pnueli and Sa’ar [88]on top of the TLV system [90]. At this point, the resulting automaton is used to govern theexecution of the local controllers, based on the local behavior of the environment.

In the following examples, the workspace is the one shown in Figure 4.2, with the 306cells instantiated as described in 4.2.3. In the LTL formulas, the visit cells correspond tothe 8 lanes around the parking spaces (4 going clockwise and 4 going counter clockwise),and the initial cells correspond to the 10 entry points to the workspace. Initially, 35 of the40 parking spaces were randomly specified as occupied.

68

Basic parking scenario

The basic parking scenario corresponds to the LTL formula described in 4.2.4.For each run, a new vehicle was introduced at a random entrance, while the parking

spaces were filled according to the previous run. As the automaton executes, if a parkingcell is a successor to the current state, the empty/occupied status is checked via a localsensor. This work does not address the required sensor, but assumes a binary signal.Transition to the parking cell is enabled if the associated space is empty. If the transitionis enabled, other transitions are disabled until the vehicle pose enters the domain of theparking cell, at which point the control shifts to the local parking controller.

Six runs were simulated using the global parking automaton; Figure 4.6 shows theresults for four of these runs. In Runs #3-5, the vehicle parks in the first available parkingspace. In Run #6, there are no parking spaces available; therefore, the vehicle continues tocircle past every possible parking space, waiting on another vehicle to leave.

New behaviors are easily implemented, either by removing a part of the basic formulaor adding new inputs and outputs, as mentioned below.

Nondeterministic choice whether to park

In the basic scenario, the vehicle must park once it finds an empty space. Mimicking amore human behavior, one can give the vehicle a nondeterministic choice whether to parkor keep looking for a different available space. This is done simply be removing the thirdline of ϕst .

Additional Outputs

Other outputs can be easily added to the specifications, thereby increasing the expressive-ness of the system. For example, given knowledge of which controllers induce left/rightturns, the outputs SigRight and SigLeft that control the vehicle signal light can be cre-ated. This is shown in Section 6.2.1.

69

Run #3 Run #4

Run #5 Run #6Figure 4.6: Four executions of the basic scenario

The initial conditions for each run are circled. The last run continues to loop as noparking spaces are available.

70

Chapter 5

Language

The logic (LTL) used in the previous chapters is expressive and can be used to capturemany different types of tasks. However, being a formal mathematical model, it takes priorexperience and knowledge (especially for the fragment of LTL considered for reactivetasks) to be able to write even the simplest tasks. This chapter describes steps taken to-wards a more natural and intuitive way of describing tasks, by using a restricted form oflanguage, specifically Engish.

The problem of translating natural language to logic or any other precise mathematicalformalism is a very hard problem. Appendix B briefly describes a step towards having anatural language interface (with a limited lexicon) to the logic. It shows a very limitedContext Free Grammar (CFG) [58, 82] that when used to parse an acceptable sentencereturns the LTL formula that captures the task.

The following described a Structured English interface that allows the user to describethe task using a set of predefined sentences in English that are then automatically translatedto the fragment of LTL considered in Chapter 3. While Structured English does not soundlike natural language it is easily understood and it acts as a mediator between the logicalformalism that the robots accept as input and the natural language to which humans areaccustomed. Also, restricting to a controlled language minimizes the problems that areintroduced in the system due to ambiguities inherent in natural language [91]. This can beof paramount importance in safety-critical applications.

Here, similar to [48, 64], first a simple grammar (Table I) is given that produces thesentences in the controlled language and then the semantics of some of the sentences inthe language are given with respect to the LTL formulas described in Section 3.2.3.

71

5.1 GrammarFirst a set of boolean formulas is defined:

φ ::= x ∈ X | y ∈ Y | not φ | φ or φ | φ and φ | φ implies φ | φ iff φφenv ::= x ∈ X | not φenv | φenv or φenv | φenv and φenv |

| φenv implies φenv | φenv iff φenvφrobot ::= y ∈ Y | not φrobot | φrobot or φrobot | φrobot and φrobot |

| φrobot implies φrobot | φrobot iff φrobotφregion ::= r ∈ R | not φregion | φregion or φregion | φregion and φregion |

| φregion implies φregion | φregion iff φregionφaction ::= a ∈ Act | not φaction | φaction or φaction | φaction and φaction |

| φaction implies φaction | φaction iff φaction

Intuitively, φ captures a logical connection between propositions belonging to both theenvironment and the robot while φenv restricts to propositions relating to the environment,φrobot restricts to propositions relating to the robot, φregion to the robot’s region (cell)propositions and φaction to the robot’s action propositions.

The grammar in Table 5.1 contains different types of sentences. In each of these, ex-actly one of the terms written inside of parenthesis is required while terms written insidesquare brackets are optional. Past and present tenses in Condition are treated differentlyonly when combined with EnvSafety or RobotSafety or Stay as explained in Section 5.4.In all other cases, they are treated the same. The user specification may include any com-bination of sentences and there is no minimal set of instructions that must be written. Ifsome sentences, such as initial conditions, are omitted, their corresponding formulas arereplaced by default values as defined in the next sections.

Two forms of behaviors are captured; Safety and Liveness. Safety includes all behav-iors that the environment or the robot must always satisfy, such as not sensing Mika ina certain region or avoiding regions. These behaviors are encoded in ϕet and ϕst and areof the form 2(formula). The other behavior, liveness, includes things the environmentor the robot should always eventually satisfy, such as sensing Mika (if we assume she issomewhere in the environment and not constantly avoiding the robot) or reaching rooms.These behaviors are encoded in ϕeg and ϕsg and are of the form 23(formula).

A point that should be made is that the grammar is designed so as the user can writespecifications for only one robot. Any inter-robot interaction comes into play through thesensor propositions. For example one can add a sensor proposition Robot2in4, which isTrue whenever the other robot is in Room 4, and then refer to that proposition: “If you aresensing Robot2in4 then go to room1”.

The different sentences are converted to a single temporal logic formula by takingconjunctions of the respective temporal subformulas.

72

EnvInit ::= “Environment starts with (φenv | False | True) ”RobotInit ::= “Robot starts [in φregion]

[with φaction | with False | with True] ”EnvSafety ::= “Always φenv”RobotSafety ::= “(Always | Always do | Do) φrobot”EnvLiveness ::= “Infinitely often φenv”RobotLiveness ::= “(Go to | Visit | Infinitely often do) φrobot”RobotGoStay ::= “Go to φregion and stay [there]”Conditional ::= “If Condition then Requirement ” |

| “Requirement unless Condition ” || “Requirement if and only if Condition ”

Condition ::= “Condition and Condition” | “Condition or Condition” || “you (were | are) [not] in φregion” || “you sensed φenv” || “you did not sense φenv” || “you are sensing φenv” || “you are not sensing φenv” || “you activated φaction” || “you did not activate φaction”| “you are activating φaction” || “you are not activatingφaction”

Requirement ::= EnvSafety | RobotSafety | EnvLiveness || RobotLiveness | “stay [there]”

Table 5.1: The basic grammar rules for the motion planning problem.

5.2 Initial conditions - EnvInit, RobotInit

The formula capturing the assumed initial condition of the environment ϕei is a booleancombination of propositions in X while the initial condition for the robot ϕsi is a booleancombination of propositions in Y . Therefore, the different options in EnvInit are translatedto ϕei = φenv, ϕei =

∧x∈X ¬x and ϕei =

∧x∈X x respectively. RobotInit is translated in the

same way.If EnvInit or RobotInit are not specified, then the corresponding formula is set to True

(ϕei = True for the environment and the same for the robot) thus allowing the initial valuesof the propositions to be unknown, meaning the robot can start anywhere and can senseanything upon waking up. Furthermore, if φenv or φregion or φaction do not contain all therelevant propositions, the set of valid initial conditions contain all possible combinationsof truth values for the omitted propositions.

73

5.3 Liveness - EnvLiveness, RobotLiveness, RobotGoStay

The subformulas ϕeg and ϕsg capture the assumed liveness properties of the environmentand the desired goals of the robot. The basic liveness specifications, EnvLiveness andRobotLiveness translate to ϕeg = 23(φenv) and ϕsg = 23(φrobot) respectively. Theseformulas specify that infinitely often φenv and φrobot must hold.

The “go to” specification does not make the robot stay in room r once it arrives there.If the desire is to specify “go to r and stay there”, a safety behavior is added that requiresthe robot to stay in room r once it arrives there. The specification is translated to

ϕstgGoStay(r)= 23r ∧2(r →©r) (5.1)

This formula states that if the robot is in room r, in the next step it must be in room r aswell.

Note that the simple grammar in Table 5.1 allows one to specify both “go to r” and“go to q and stay there” in a single task. This is an infeasible specification since “go to”implies visiting a region infinitely often, and the synthesis algorithm will inform the userthat it is unrealizable.

The general from of a liveness conditional is translated into

ϕαt = 23(Condition→ Requirement) −If statementϕαt = 23(Condition ∨Requirement) −Unless statementϕαt = 23(Condition⇔ Requirement) −If and only if statement

(5.2)

Where α ∈ {e, s}, the condition is captured by a combination of φenv, φregion andφaction and the requirement is captured by (φenv) or (φrobot) .

If there are no liveness requirements for the environment or for the robot then thecorresponding formula is set to 23(True).

5.4 Safety - EnvSafety, RobotSafety

The subformulas ϕet and ϕst capture how the sensor and robot propositions are allowed tochange from the current state in the synthesized automaton to the next. Due to the way thealgorithm works (see Chapter 3 and [68, 88]), the next sensor values can only depend onthe current sensor and robot values and the next sensor values while the next robot valuescan depend on the current as well as the next sensor and robot values 1.

The basic safety specifications, EnvSafety and RobotSafety translate to the formulasϕet = 2(©(φenv)) and ϕst = 2(©(φrobot)) respectively. These formulas specify thatalways, at the next state, φenv and φrobot must hold.

1The sensing is done before acting, therefore when choosing the next motion and action for the robot,the most recent sensor information can be used.

74

The general from of a safety conditional is translated into

ϕαt = 2(Condition→ Requirement) −If statementϕαt = 2(Condition ∨Requirement) −Unless statementϕαt = 2(Condition⇔ Requirement) −If and only if statement

(5.3)

Where α ∈ {e, s}. The requirement is captured by ©(φenv) or ©(φrobot) or ‘Stay[there]’ which is translated to the formula

∧r∈Reg(r ⇔©r) that encodes the requirement

that the robot not change the region it is in (if ri is True (False) then it is also True (False)in the next state).

As mentioned before, for safety conditions, the past tense relates to the value of thepropositions in the current state while the present tense relates to the value of the proposi-tions in the next state. The atomic conditions are translated into

φ - Positive past¬φ - Negative past©φ - Positive present¬© φ - Negative present

where φ is either φenv or φregion or φaction, depending on the sentence. A generalcondition can be built up by combining several atomic conditions using “and” (∧) and“or” (∨).

One should keep in mind that the past tense only relates to situations that are Truein the current state. Such behaviors may include “If you were in r1 and youare in r2 then Beep” meaning that if the robot moved from Region 1 to Region 2 itshould beep. This sentence does not capture a behavior in which if the robot was sometimein the past in Region 1 and now it is in Region 2 it should beep. If the desired behaviorneeds to depend on events that took place in the past, an auxiliary proposition should becreated to ‘remember’ that the event took place.

In addition to the Structured English sentences, the parser automatically encodes mo-tion constraints of the robot that are induced by the workspace decomposition. Theseconstraints are that the robot can only move, at each discrete step, from one region to anadjacent region and it can not be in two regions at the same time (mutual exclusion). Atransition is encoded as

ϕstTransition(i)= 2(ri → (©ri ∨r∈N ©r)) (5.4)

where N is the set of all the regions that are connected to ri in the discrete abstraction ofthe workspace. All transitions are encoded as

ϕstTransitions= ∧i=1,...,n ϕ

stTransition(i)

(5.5)

75

The mutual exclusion is encoded as

ϕstMutualExclusion= 2(∨1≤i≤n (©ri ∧1≤j≤n,i 6=j ¬© rj)) (5.6)

If there are no safety requirements for the environment then ϕet = 2(True) thus al-lowing the sensor propositions to become True or False at any time. If there are nosafety requirements for the robot then ϕst = ϕstTransitions

∧ ϕstMutualExclusion, encoding only

the workspace decomposition.

5.5 ExamplesIn the following, the workspace of the robot contains 24 rooms (Figures 5.1, 5.2). Giventhis workspace ϕstTransitions

and ϕstMutualExclusionare automatically generated

ϕstTransitions=

2(r1 ⇒ (©r1 ∨©r8)) ∧∧2(r2 ⇒ (©r2 ∨©r10)) ∧

...∧2(r24 ⇒ (©r24 ∨©r5 ∨©r23))

(5.7)

ϕstMutualExclusion= 2

( ∨1≤i≤24

(ri ∧

∧1≤j≤24,i 6=j

¬rj

))(5.8)

=

2( (r1 ∧ ¬r2 ∧ · · · ∧ ¬r24)∨∨(¬r1 ∧ r2 ∧ · · · ∧ ¬r24)∨

...∨(¬r1 ∧ ¬r2 ∧ · · · ∧ r24) )

(5.9)

5.5.1 No SensorsThe behavior of the robot in these examples does not depend on its sensor inputs. However,for the completeness of the LTL formula, at least one sensor input must be specified.Therefore a dummy sensor input X = {Dummy} is created which is define to be constantin order to reduce the size of the automaton. It is arbitrarily chosen to be False.

Visit and Beep

Example 11. Here the robot can move and beep, therefore Y = {r1, . . . , r24, Beep}. Thedesired behavior of the robot includes visiting Rooms 1, 3, 5 and 7 infinitely often andbeeping whenever it is in Corridors 9, 12, 17 and 23, but only there. It is assumed that therobot starts in Room 1 and initially does not beep.

The user specification is:

• “ Environment starts with False”

76

0 50 100 150 200 250 300 3500

50

100

150

200

250

300

1

23

4

5

67

8

9

10

11

12

13 14

15

16

17

1819

20

21 22

2324

Figure 5.1: Simulation for the Visit and Beep example

• “ Always not Dummy”

• “ Robot starts in r1 with False”

• “ Activate Beep if and only if you are in r9 or r12 or r17

or r23”

• “ Go to r1”

• “ Go to r3”

• “ Go to r5”

• “ Go to r7”

The behavior of the above example is first automatically translated into the formula:

ϕe = ¬Dummy ∧2¬Dummy ∧23True (5.10)

ϕs =

r1 ∧i=2,...,24 ¬ri ∧ ¬Beep∧ϕstTransitions

∧ ϕstMutualExclusion

∧2((©r9 ∨©r12 ∨©r17 ∨©r23)⇔©Beep)∧23(r1) ∧23(r3) ∧23(r5) ∧23(r7)

(5.11)

Then an automaton is synthesized and a hybrid controller is constructed. The computationtime of the synthesis algorithm is less than 2 seconds and the resulting automaton has 25states. Sample simulations are shown in Figure 5.1 where beeping is indicated by lightercolored stars.

Visit while Avoiding - Impossible specification

Example 12. In this example it is assumed that the robot can only move.

77

The user specification is:

• “ Environment starts with False”

• “ Always not Dummy”

• “ Robot starts in r1 with False”

• “ Always not r8 and not r22”

• “ Go to r2”

• “ Go to r5”

Viewing the workspace of the robot one can see that this user specification is impossibleto implement since the robot cannot leave Room 1 if it avoids 8, and thus it cannot reachthe other rooms. In this case, the synthesis algorithm terminates with the message that thespecification is not realizable.

5.5.2 SensorsSearch and Rescue I

Example 13. Let us assume that the robot has two sensors, a vision system that candetect an injured person and another sensor that can detect a gas leak, therefore X ={Person, Gas}. Here, other than moving, the robot can communicate to the base stationa request for either a medic or a fireman. It is assumed that the base station can trackthe robot therefore it does not need to transmit its location. The robot propositions areY = {r1, . . . , r24,Medic,Fireman}.

The user specification is:

• “ Environment starts with False”

• “ Robot starts in r1 with False”

• “ Do Medic if and only if you are sensing Person”

• “ Do Fireman if and only if you are sensing Gas”

• “ Go to r1” ...

• ... “ Go to r24”

The computation time for this example is 36 seconds and the resulting automaton has 300states. A sample simulation is shown in Figure 5.2. Here, a person was detected in Region10 resulting in a call for a Medic (light cross). A gas leak was detected in Region 24resulting in a call for a Fireman (light squares). In Region 12, both a person and a gas leakwere detected resulting in a call for both a Medic and a Fireman (dark circles).

78

0 50 100 150 200 250 300 3500

50

100

150

200

250

300

1

23

4

5

67

8

9

10

11

12

13 14

15

16

17

1819

20

21 22

2324

Figure 5.2: Simulation for the Search and Rescue I example

Search and Rescue II

Example 14. In this scenario, the robot must monitor Rooms 1, 3, 5 and 7 for injuredpeople (the assumption is that injured people can only appear in these rooms). If it sensesan injured person, the robot must pick him or her up and take them to a medic who is ineither Room 2 or Room 4. Once the robot finds the medic, it leaves the injured personthere and continues to monitor the rooms.

Here X = {Person, Medic} and Y = {r1, . . . , r24, carryPerson}. The specificationis:

• “Environment starts with False”

• “If you were not in r1 or r3 or r5 or r7 then alwaysnot Person”

• “If you were not in r2 or r4 then always not Medic”

• “Robot starts in r1 with False”

• “If you are sensing Person then do carryPerson”

• “If you are not sensing Person and you did not activatecarryPerson then do not carryPerson”

• “If you are not sensing Medic and you activatedcarryPerson then do carryPerson”

• “If you are sensing Medic and you activated carryPersonthen do not carryPerson”

• “Go to r1 unless you are activating carryPerson”

79

0 50 100 150 200 250 300 3500

50

100

150

200

250

300

1

23

4

5

67

8

9

10

11

12

13 14

15

16

17

1819

20

21 22

2324

(a) The robot search the rooms and found an in-jured person in Room 7

0 50 100 150 200 250 300 3500

50

100

150

200

250

300

1

23

4

5

67

8

9

10

11

12

13 14

15

16

17

1819

20

21 22

2324

(b) The medic was found in Room 4

Figure 5.3: Simulation for the Search and Rescue II example

• “Go to r3 unless you are activating carryPerson”

• “Go to r5 unless you are activating carryPerson”

• “Go to r7 unless you are activating carryPerson”

• “If you are activating carryPerson then go to r2 ”

• “If you are activating carryPerson then go to r4 ”

A sample simulation is shown in Figure 5.3. In the simulation, the robot begins by search-ing Rooms 1 then 3 then 5 and finally 7 for an injured person (Figure 5.3(a)). It finds onein Region 7 and goes looking for a medic (indicated by the light crosses) first in Room 2and then in Room 4. It finds the medic in Room 4 and starts to head back through Region17 towards Region 1 to continue its task of monitoring (Figure 5.3(b)).

80

Chapter 6

Multi robot

In the previous chapters, different types of high level tasks, robot models and specificationformats were address, all in the context of a single robot. While the single robot caseis difficult enough, moving to a multi-robot behavior adds another layer of challenges.Having every robot achieve its specific goals while contributing to a global coordinatedtask requires each robot to react to information about other robots, for example to avoidcollisions. Furthermore, each robot must incorporate new information into its decisionframework in order to react to environmental changes induced by other robots, since thisknowledge may effect its behavior.

This chapter examines how and whether the methods of the previous chapters can beextended to handle tasks of multiple robots, either in a centralized or decentralized manner,each with its advantages and limitations.

6.1 Non-reactive Model-checking ApproachAt a first glance, the approach using model-checking for nonreactive tasks can be adjustedto the multi robot case. One can easily define propositions that relate each robot to itslocation. For example, R1r1 is True if Robot 1 is in Cell 1 and false otherwise. Thediscrete abstraction of the motion, if each robot has its own controller, is simply the productof the discrete abstractions of each robot, thus there may be a state in the product system(R1r1, R2r1, R3r5) indicating both Robot 1 and Robot 2 are in Cell 1 while Robot 3 is inCell 5. Then, an LTL formula, capturing the multi robot task is first negated, then givento the model-checker together with the combined discrete system and a counterexample isfound, just as described in Chapter 2.

However, this method of dealing with multi robot tasks has several flaws. First, thecomplexity of the problem grows exponentially with the number of robots even if theircontrol is decentralized, since the discrete abstraction of the motion is a product of theindividual robot motions. Since model-checkers can handle very large state spaces, formoderately sized workspaces and a small number of robots this is not a real issue butwhen looking at large swarms moving around this method may not work.

81

The biggest drawback of this approach, when the controllers are decentralized, is the“open loop” nature of the solution. The model-checker returns a sequence, or a list ofinstructions to be executed one after another. This sequence can require the system tomove for example from (R1r1, R2r1) to (R1r2, R2r2). The continuous execution, in thiscase, must make sure both robot move from Cell 1 to Cell 2 at the exact same time sinceotherwise the continuous system will go through a non specified discrete step (such as(R1r2, R2r1) or (R1r1, R2r2)) which may violate the desired task. This type of synchro-nization can be enforced artificially by stopping the robots right before they cross to thenext cell however that induces segmented motion. Furthermore, if each robot has its owncontroller, the problem of collision avoidance must be addressed using other means. Hav-ing said that, when considering a centralized controller that drives all robots together, asin Section 6.3, the “open loop” nature can be used.

The work described in the Section 6.2, based on the synthesis approach, is better suitedfor a decentralized approach. For the centralized approach, both the model-checking (witha suitable construction of the combined discrete motion) and synthesis approaches workwell, as explained in Section 6.3.

6.2 DecentralizedIn a decentralized approach to multi robot tasks, using the synthesis method of Chapter 3,each robot has its own LTL formula and its own automaton. The information about theother robots it needs to complete its task is part of the sensor inputs. That is, all otherrobots are part of the environment and their relevant actions are given to the robot throughthe propositions in X . For example, if the robot must patrol the nursery if Robot 2 ran outof battery, it will have a proposition R2dead and the specification will state “ If R2deadthen visit Nursery.”

The multi robot decentralized approach is demonstrated using an example similar toto the one in Section 4.2 only here, several cars are operating in the same workspace.The vehicles in this example execute one of two automata. The first automaton satisfiesthe high-level specification “drive around the environment, while obeying traffic rules,until you find a free parking space, and then park”. The second automaton satisfies thespecification “Leave the block, while obeying traffic rules, throughExiti” where i is givenas input. Figure 6.11 shows a simulation snapshot with eight currently active vehicleseither searching for a parking space or exiting the block.

The specifications for the aforementioned task are described in the following. Section6.2.1 first describes an LTL formula that encodes appropriate behavior in traffic, i.e. thereaction to hazardous conditions and the activation of the turn signals. This LTL formulacaptures the multi-robot aspect of the behavior. Sections 6.2.2 and 6.2.3 then add the morespecialized behavior for the parking and leaving tasks, respectively.

1All of the figures in Section 6.2 were generated by David C. Conner for [66].

82

Figure 6.1: Workspace for decentralized multi robot exampleFor any vehicle, the high-level specification encodes either “drive around until you find a

free parking space and then park” or “leave your parking space and exit the block”.

83

6.2.1 Adhering to traffic lawsSocially acceptable driving behavior includes stopping at stop lights, driving in the desig-nated lane, keeping a safe distance from vehicles ahead and using the left and right turnsignals. In order to encode such behavior an input, hazard, is defined which becomesTrue whenever the car must stop. Such an input may be the result of a proximity sensor inthe case of keeping a safe distance from another vehicle or of a vision system recognizinga red light at the intersection, or another vehicle signaling that it is about to make a turn.The input hazard is also used to cause a vehicle intending to park to wait on a vehicle thatis ready to leave an occupied parking space. While in some cases the more natural reactionto such conditions is to slow down rather than stop, here the more conservative approach istaken for simplicity. The local feedback controller ID’s serve as outputs. Additional outputpropositions are signalL and signalR which indicate whether the left (right) turn signalshould be activated and the proposition stop which indicates whether the vehicle shouldstop. These outputs are detectable by other robots. The formula encoding this behavior is:

Assumptions on the environment

Initially there is no need to stop therefore ϕei = ¬hazard. There are no further restrictionson the behavior of the hazard input thus it can become True or false at any time. In orderto keep the structure of the formula both ϕet and ϕeg are encoded as the trivial formula Truewhich means that these formulas are always satisfied.

ϕet ∧ ϕeg = 2(True) ∧23(True) (6.1)

Constraints on the behavior of the vehicle (system)

Initially the vehicle must be in the domain of an initial cell, no turn signal is on (thevehicle is assumed to start by driving straight, this is changed in 6.2.3) and the vehicle isnot required to stop.

ϕsi = ∨i∈InitialCellΦi ∧ ¬SignalL ∧ ¬SignalR ∧ ¬stop (6.2)

The vehicle can only transition from one cell to the next based on the prepares graph from4.2.3 (first line of ϕst below). It must turn the left turn signal only if it is turning left andthe same for the right turn signal (second and third line). It must stop if and only if thehazard signal is True (last line).

ϕst =

∧i 2(Φi ⇒ (©Φi ∨j∈SuccessorsOfCelli ©Φj))∧ 2( (∨j∈LeftTurnCells© Φj) ⇔ ©signalL)∧ 2( (∨j∈RightTurnCells© Φj) ⇔ ©signalR)∧ 2(©hazard ⇔ ©stop)

(6.3)

84

Finally, since this formula is only concerned with obeying traffic laws and does not requirethe vehicle to go anywhere, the goal is simply

ϕsg = 23(True) (6.4)

6.2.2 ParkingIn this scenario, a vehicle is searching for an empty parking space, and parks once it findsone. This behavior is almost identical to the one of Section 4.2.4. For ease of reading theformulas are written here as well but for the description please refer to Section 4.2.4.

Starting from the formula in 6.2.1 another input, park, is defined which becomes Truewhen an empty parking space is found.

The following subformulas are added to ϕe of 6.2.1:

¬park ∧2( [ (¬(∨i∈ParkCellΦi)) ∧

(¬(∨j∈PreparesParkCellΦj))]⇒ ¬© park )

∧2( (park ∧ (∨i∈ParkCellΦi)) ⇒ ©park )

(6.5)

There are no assumptions on the infinite behavior of the environment (there may or maynot be an empty parking spot), therefore the goal component remains set to True.

The following subformulas are added to ϕs of 6.2.1:{∧i∈ParkCell 2(¬© park ⇒ ¬© Φi)∧ 2(©park ⇒ (∨i∈ParkCell© Φi) )

(6.6)

The only difference between the goal in Section 4.2.4 and this goal is that here thevehicle can stop due to a hazard (an accident ahead of it or a broken stop light), thereforeϕsg in the multi robot case is

ϕsg = ∧i∈V isitCell23(Φi ∨ park ∨ stop) (6.7)

6.2.3 LeavingIn this scenario, a vehicle is leaving its parking space and exiting the block via somespecified exit. As before, starting from the formula in 6.2.1 additional inputs ExitΦi

2 areadded for i ∈ ExitCells. These are inputs that are constant and define which exit thevehicle should use (the proposition that is True), thus two vehicle leaving may use thesame generated automaton with different inputs.

2As before, in the actual implementation these are encoded as binary vectors.

85

Assumptions on the environment

These subformulas are added to ϕe of 6.2.1: Initially only one ExitΦi is True. This isadded to ϕei

∨i∈ExitCells (ExitΦi ∧j∈ExitCells,j 6=i ¬ExitΦj) (6.8)

The inputs must be constant, meaning they cannot change. Therefore the following isadded to ϕet

∨i∈ExitCells (ExitΦi ⇔ ©ExitΦi) (6.9)

There are no assumptions on the infinite behavior of the environment, therefore thegoal component remains set to True.

Constraints on the behavior of the vehicle (system)

Initially, the car is leaving a parking space, hence it must turn on the left turn signal.Formula ϕsi of 6.2.1 is modified to:

ϕsi = ∨i∈InitialCellΦi ∧ SignalL ∧ ¬SignalR ∧ ¬stop (6.10)

No further subformulas are added to ϕst of 6.2.1. As for ϕsg, it is replaced with therequirement that the vehicle must go to the designated exit cell if it hasn’t stopped.

ϕsg = ∧i∈ExitCells23( (Φi ⇔ ExitΦi) ∨ stop) (6.11)

6.2.4 SimulationsThe simulation follows the one described in Section 4.2.6.

In this simulation, the sensors that govern the behavior of the park and hazard inputsmust be simulated. The park input is set to True whenever there is a free parking spacenear by3. The hazard input which enables the traffic law abiding behavior and thus themulti-robot task should be set to True whenever the car must stop. Here, a proximitysensor is simulated with added logic that sets hazard to True whenever the car is tooclose to a car ahead of it (keeping safe distance), whenever a car ahead is backing up topark (being polite), whenever the car is leaving a parking space and another car passesby and whenever another car is leaving a parking space which the car will park in next.Furthermore, a vision system is simulated that sets hazard whenever a stoplight is red.

In the following example, the workspace is the one shown in 6.1, with 306 cells instan-tiated as described in 4.2.3. In the parking LTL formula, the visit cells correspond to the 8lanes around the parking spaces (4 going clockwise and 4 going counter clockwise), andthe initial cells correspond to the 10 entry points to the workspace. Likewise, in the leavingautomaton, the 40 parking spaces are the possible initial cells and the 10 exit points are thepossible goals. Initially, 35 of the 40 parking spaces were randomly specified as occupied.

3park will become True only in cells that prepare the appropriate parking cell for the free space

86

Figure 6.2: A snapshot of the simulation.Cars surrounded by red ellipses are cars that are stopping due to the hazard input, in this

case based on a stoplight

In this simulation, eight cars enter the block at different times and from different entrypoints, looking for a parking space. During the execution, an additional three cars leavetheir parking spaces and exit the workspace. Figure 6.2 shows a general snapshot of thesimulation. At this point in time, seven cars are moving in the workspace. Cars that aremarked with red ellipses are the cars who’s hazard input is True, therefore they havestopped. All stopped cars in this figure are obeying stoplights.

Figure 6.3 shows several close up looks at different traffic behaviors encountered dur-ing the simulation. In (a), the blue car which is leaving the parking space has stopped,indicated by a red ellipse, to let the brown car drive by. This hazard was invoked basedon a “proximity sensor”. In (b), the red car is parking while the blue car waits for it tofinish before passing. In (c), the orange car is stopping to allow the gray car to complete aleft turn. The white car on the left is leaving the parking space that later will be occupiedby the brown car. Figure 6.3(d) shows two cars stopping before a stoplight. While the

87

white car stopped based on the stoplight, the black car behind stopped based on the prox-imity to the car ahead of it. Figures 6.3(e) and (f) are two snapshots of two cars parkingsimultaneously in opposite lanes. The car that started the parking maneuver later (bottomlane) pauses to allow the other car to park safely.

88

(a) Blue car leaving (b) Red car parking

(c) Yielding to turn in progress (d) Two cars at stoplight

(e) Two cars parking (f) Two cars parking

Figure 6.3: Close up looks at different behaviors seen throughout the simulation

89

6.3 CentralizedIn a centralized approach, all robots are controlled using one global controller. Thus, thecontrol input to one robot depends on the position and control of all other robots. Whilesuch an approach scales exponentially with the number of robots, its strength is the factthat collision avoidance and other proximity constraints are naturally handled. Further-more, deadlock and livelock problems are easier to avoid compared to the decentralizedapproach, as discussed in Section 6.4.

6.3.1 Centralized Atomic ControllerHere, as opposed to the previous section, the atomic controller is such that generates acontrol input for all the robots u = (u1, u2, . . . ). Each controller drives robots from anyinitial position in one set of locations x y z to another set l m nwithout going through anyother combination. Furthermore, they maintain prescribed inter-agent constraints, such asavoiding collisions and maintaining communication.

The problem of creating such controllers was first addressed in [8]. Here, a summeryof the algorithm is given and for further details the reader is referred to [8, 65].

The algorithm for synthesizing a set of centralized, atomic, multi robot controllersinvolves the following four steps:

Algorithm 3 Creating multi robot centralized controllers

1. Construct task configuration space CT

2. On each polytope P i, create a set of feedback controllers that drive the system fromany initial state to any facet.

3. On each polytope P i, create a feedback controller that drives the system to a goalconfiguration

4. Find paths onGP for each possible transition from one room combination to anotherand combine the controllers which correspond to that path.

The task configuration space CT is the Cartesian product of the partitioned free work-space of each robot. For example, for three robots moving in a two dimensional workspace,CT is build of six dimensional polytopes. Each room combination for the robots is repre-sented by a group of polytopes in the task space. Furthermore, CT is created such that allof its polytopes causes the robots to maintain proximity constraints thus ensuring collisionavoidance and communication range.

The graph GP represents the adjacency of the polytopes. Two polytopes are connectedif they share a facet. Note that this graph is much larger than the discrete abstraction of

90

the combined robot motion since several polytopes can be associated with the same roomcombination.

To create an atomic controller driving the set of robots from one cell combination toanother, first a set of feedback controllers are created such that each drives the system fromsome polytope to an adjacent one (step 2). In the following, the controller design followsthe work in [52]. Furthermore, if there are specific goal configuration of interest withinthe polytope, a feedback controller is created for driving the system to that goal.

For an atomic controller driving the robots from x y z to l m n, once all the controllersare created the graph GP is searched for paths originating from polytopes associated withx y z to the closest polytope associated with l m n. The atomic controller is the composi-tion of the feedback controllers belonging to these polytopes. If the system must reach agoal configuration in some room, the controllers of step 3 are added.

The discrete abstraction of motion used later on is a graph where the nodes are theroom combinations and there are edges between “legal” cell changes. Due to the atomiccontrollers construction, only one robot will cross a room threshold at any time thereforethere are edges between two nodes if exactly one robot changes its cell to an adjacent cell.

6.3.2 Recycling examplesHere a multi robot sorting task is used as an illustration of the centralized approach. Figure

Figure 6.4: Workspace for recycling examples

6.4 depicts the setup of the following examples. It consists of three robots, named a1, a2

and a3, moving in a workspace P ⊂ R2 with ten rooms and it is assumed that initiallya1 (blue square) is in Room 1, a2 (green triangle) is in Room 3 and a3 (magenta circle)is in Room 5. The high level task requires the robots to pick up different items from

91

predesignated pick up locations and drop them, according to the material they are madeof, in the right spot while avoiding collisions and deadlocks. Example 15 is used as thebasic scenario to explain the different steps of the approach and Section 6.3.3 illustrateshow easily other behaviors and constraints can be handled.

Example 15. Here, the robots pickup items from Rooms 6 and 7. These items can be ofthree types: metal, glass and paper. Whenever an item is ready for pickup, one of therobots must deposit it in its destination: Room 8 for metal, Room 9 for glass and Room10 for paper. If no item is present, the robots must wait in Rooms 1, 3, and 5 (it doesn’tmatter which robot is in which room). An additional constraint is imposed that there be atmost one robot in Rooms 6 and 7 at a time, for the recyclers peace of mind.

First the task ϕ is specified using the following propositions. The sensor propositionsX are

• pu6, pu7 - propositions that are True when there is an available item in Rooms 6 and7 respectively. These represent information that is transmitted to all robots.

• m1, g1, p1,m2, g2, p2,m3, g3, p3 - propositions indicating what type of material theitem robot ai picked up is made of (metal, glass and paper respectively)4

The system propositions include propositions relating to the different robot actions Act:

• a1PU , a2PU , a3PU - propositions that are True when the robot picks up an item

• a1Carry, a2Carry, a3Carry - propositions that indicate the robot is carrying anitem

• a1D, a2D, a3D - propositions that are True when the robot drops the item it hasbeen carrying

As well as propositions relating to the motion (location) of the robots. These propositionscorrespond to all combinations of rooms5, for example proposition 1 3 5 is True when a1

is in Room 1, a2 is in Room 3 and a3 is in Room 5. The workspace contains ten rooms andthree robots therefore there are 1000 possible combinations in general. In the transitionformula relating to the motion, the system state is allowed to change at most one robot’sregion at a time, thus 1 2 5 is allowed to change to 1 3 5 but not to 1 3 4 as explained inSection 6.3.1.

Once the propositions are defined, the task must be specified using Structured English.The following sentences relate to Example 15. Also, the sentences refer to a1 but the fullspecification contains the same sentences for a2 and a3 as well.

Assumptions about the behavior of the sensor propositions are captured in Items 1through 5:

4In the actual implementation these three propositions are encoded as a two bit binary vector thus re-ducing the number of propositions and insuring these propositions are mutually exclusive, that is, an itemcannot be both paper and glass.

5As before, we encode these propositions as a binary vector instead of creating a proposition for eachroom combination.

92

1. “Environment starts with false” - All sensor propositions are initiallyfalse. It is assumed that at system start up, there is no known item to pick up.

2. “If you did not activate a1Carry then always not m1and not g1 and not p1” - These assumptions say that if a1 is not carrying anitem, it has no knowledge about material type. This statement is not needed for thetask however it is a reasonable assumption to make and it reduces the size of theresulting automaton.

3. “If you activated a1Carry and you sensed m1 then alwaysm1” - same for g and p. These assumptions say that once the material type of acarried item is determined, it does not change.

4. “If you activated a1Carry then always m1 or g1 or p1”- These assumptions state that the sensors will tell the robot what type of material ithas picked up.

5. “If you sensed pu6 and you did not activate a1PU and6 X X and you did not activate a2PU and X 6 X and youdid not activate a3PU and X X 6 then always pu6” -where 6 X X corresponds to all combinations of room in which a1 is in Room 6.The same assumption is also written for pu7. These complex sentences basicallysay that if an item appears in Room 6 (7) and no robot picked up an item in Room 6(7), then the item is still there. Without this assumption the environment can preventthe robots from satisfying the task, as explained in Item 13.

Desired system behavior is captured in Items 6 through 14 together with the LTL formulagenerated from the discrete abstraction of the environment.

6. “System starts in 1 3 5 with false” - Initially the robots are not car-rying, picking up or dropping anything.

7. “Activate a1PU if and only if you did not activatea1Carry and (you are in 6 X X and you are sensing pu6or you are in 7 X X and you are sensing pu7)” - If the robotis not carrying an item and it is in a room with an available item, it should pick it up.

8. “Activate a1D if and only if you activated a1Carryand you are in 8 X X and you are sensing m1 or youactivated a1Carry and you are in 9 X X and you aresensing g1 or you activated a1Carry and you are in 10 X Xand you are sensing p1” - The robot should drop the item it is carrying ifit is in the correct room.

9. “If you activated a1PU or you activated a1Carryand did not activate a1D then do a1Carry” - This together withItems 10 and 11 define that a1Carry should be True between pick up and drop.

93

10. “If you did not activate a1PU and did not activatea1Carry then do not a1Carry”

11. “If you activated a1D and activated a1Carry thendo not a1Carry”

12. “If you are not activating pu6 and you are notactivating pu7 and you are not activating a1Carry andyou are not activating a2Carry and you are notactivating a3Carry then go to WaitRegions” - whereWaitRegions is all possible permutations of Rooms 1,3 and 5. This requirementstates that if there are no items in the workspace, the system must drive the robots tothe waiting rooms.

13. “If you are not activating a1Carry then if you aresensing pu6 then go to 6 X X and if you are sensing pu7then go to 7 X X” - this sentence is conjuncted with the corresponding sen-tences for a2 and a3 resulting in a requirement that says that if there is an itemlessrobot and there is an item, the robot must go pick it up. In order to satisfy this goal,the assumption in Item 5 must be made otherwise the environment can prevent thesystem from reaching this goal by switching pu6 and pu7 on and off in a way thatthe robots will cycle between these rooms without picking up an item.

14. “If you are activating a1Carry and you are sensing m1then go to 8 X X” - this sentence is conjuncted with the corresponding sen-tences for a2, a3, 9 X X, 10 X X, g and p resulting in a requirement that says that ifa robot is carrying an item, it will go to the correct drop off location.

The requirement that there be at most one robot at a time in Rooms 6 and 7 can beeasily captured in the Structured English description (for example “Always not 6 6 X”).However, to reduce the size of the problem, such combinations are referred to as illegaland they are omitted from the discrete graph that represents the locations of the robots andthus never reached. This constraint reduces the number of combinations from the general1000 to 944.

The synthesized automaton for this example contains 12,585 states. While such atask can potentially be encoded by hand in a much smaller automaton, this was createdautomatically and is guaranteed to be correct. A small part of this automaton is shown inFigure 6.5. The circles represent the automaton states and the propositions that are writteninside each circle are the robot propositions’ that are True in that state. The edges arelabeled with all of the sensor propositions that are True when that transition is enabled.Starting from the top most state, in which the robots are in Rooms 6,3 and 7, a1 is pickingup an item and a3 is carrying an item, if there are no more items to pick up (left and rightbranches, in both pu6 and pu7 are false) the robots proceed to the drop off location (in theright branch, a3 drops the paper item in Room 10, as required), otherwise if there are more

94

Figure 6.5: Part of the automaton that satisfies Example 15

items (middle branch, pu7 is True) a2 moves until it picks up the item in the correct room(Room 7).

An important thing to note about automata and hybrid controllers created using thismethod, is that the goals are satisfied in a cyclic manner, that is, the first goal writtenis reached and then the second and finally after the last goal is achieved the automatonsatisfies the first goal again and so on. In Example 15 this results in the robots first pickingup available items until either all robots are carrying something or there are no more items,and only then all the robots go to the appropriate drop off room.

Figure 6.6 depicts part of a simulation run that corresponds to Example 15 and illus-trates both the continuous execution of the automaton segment shown in Figure 6.5 and thefact that using the same automaton, the behavior of the system varies significantly basedon what is happening in the environment.

6.3.3 SimulationsThis section presents simulations of Example 15 and demonstrates how different tasks caneasily be accommodated using the same atomic controllers but a different automaton.

95

(a) The left branch (b) The middle branch

(c) The right branch - first snapshot (d) The right branch - second snapshot

Figure 6.6: Simulation of the automaton segment of Figure 6.5

96

The atomic controllers were designed in MATLABTMusing the Multi-Parametric Tool-box for polytope computations [71] and all of the automata were synthesized using a pro-totype of the JTLV system [100] on which the synthesis algorithm was implemented. Thesimulations were run in MATLABTM.

Example 15

Figure 6.7 depicts a sample simulation of Example 15. In this scenario, there is alwayssomething to pick up (denoted as a purple X) in both pick up locations. Robots a1, a2 anda3 represented as a blue square, green triangle and magenta circle, start in Regions 1,3 and5 respectively. At first a3 goes to Room 7 and picks up an object (a), then a2 picks up alsoin Room 7 (b) and then a1 picks up in Room 6. Note that, as required, there is at most onerobot in Rooms 6 and 7 at any given time. Also, as discussed in Section 6.3.1, for everydiscrete transition in the automaton only one robot is changing the region it is in.

Once all the robots have identified the object they are carrying (b,c) they continue todrop them off in the appropriate location. Robot a3 drops the paper object (d), a1 and a2

drop off a glass and a metal object (e). Since there are more objects to pick up, the robotsmove towards the pickup rooms beginning with a3 picking up another paper in Room 7(f).

Adding motion constraints

Adding motion constraints on the robots can be done in two ways, as hinted before. Oneway is to explicitly state such constraints in the user specifications and the other is toremove nodes and transitions from the discrete graph that represents the abstraction of theteam configuration space.

Example 16. A “baby sister” constraint is added to Example 15 that requires robot a2 toalways follow robot a1 around, meaning that they must always be either in the same roomor in adjacent rooms.

Adding this constraint to Example 15 reduces the number of atomic controllers from944 to 256 and the resulting automaton contains 6,874 states.

Adding inputs

Sensor inputs as well as system outputs can be added in a very flexible way as long as theadded specification does not create a logical contradiction with the previously specifiedtask or results in an infeasible motion request.

Example 17. For safety reasons a3 should not enter Room 6 when there are childrenpresent. To encode that a sensor input kidIn6 is added to Example 16 that when Trueindicates that a child is in Room 6 and therefore a3 may not enter there (if it is alreadythere it will exit). Then the a3 equivalent of item 13 of the specification is changed to “if

97

(a) a3 picks up in Room 7 (b) a2 picks up in Room 7

(c) a1 picks up in Room 6 (d) a3 drops off a paper object

(e) a1 and a2 drop off a glass and a metal objectrespectively

(f) a3 picks up in Room 7 again

Figure 6.7: Simulation of Example 15

98

you are not activating a3Carry then if you are sensing pu6 and not kidIn6 then go toX X 6 and if you are sensing pu7 then go to X X 7” and the constraint “if you are sensingkidIn6 then always not X X 6” is added.

The resulting automaton contains 16,724 states.

99

6.4 DiscussionWhen choosing between a decentralized and a centralized approach, the strengths of oneare the limitation of the other. A decentralized approach can scale better while the central-ized one can provide global guarantees for the behavior of the whole system.

6.4.1 ScalabilityThe centralized approach encodes the full state of the system in the specification and theautomaton therefore it scales exponentially with the number of robots. Furthermore, itrequires the design of a much larger set of atomic controllers compared to the decentralizedapproach. The large number of controllers is less of an issue since once they are designedfor a workspace, number of robots and the robot dynamics, they can be reused for manytasks, however the exponential blowup of the automaton is harder to deal with.

In the decentralized approach, if the robot needs to know the full state of all the otherrobots, the problem again scales exponentially with the number of robots. However, ifthe robot needs less information (such as knowledge of specific events in the environmentand not where every robot is at every time) the decentralized approach results in smallerautomata.

6.4.2 Global guarantees - liveness of the systemThe approach to reactive tasks described in this thesis provides guarantees of correctnesshowever it does not prevent deadlock (the system is stuck in a state and thus makes noprogress toward its goals) or livelock (the system cycles through a set of states but makesno progress toward its goals). Usually, if the specification encodes deadlock avoidance(such as never stay in the same cell) but there are no assumptions about the environmentthat prevent deadlock situations, the specification will be unrealizable. As for livelock,such specifications are much harder to encode.

In a centralized approach to multi robot tasks, the controller can eliminate many ofthese issues. Since the system has full information it can steer the robots through tightspaces that would otherwise cause problems. Furthermore, the system can allocate differ-ent aspects of a task to the robots thus avoiding “tight” situations from the beginning.

While collision avoidance can be handled in the decentralized approach, for exampleby using the hazard input, a careless design of the sensors that set hazard can easilylead to deadlock that cannot be detected a priori. For example, for the task in Section 6.2,if the proximity sensor set the hazard input to True whenever another vehicle was in acertain radius, even if that vehicle was behind in a forward driving lane, both vehicles mayget deadlocked, i.e. both would stop forever. While this behavior satisfies the originalspecification it does not follow the spirit of finding a parking space. On the other hand,both cars stopping might be a desired behavior when an accident occurred therefore wewould not want to forbid it in the specifications. Such unintended behavior would notbe present in a centralized approach where the controller has full knowledge and not just

100

local information as is the case here. An interesting direction for future research is how tomaintain global liveness guarantees in a decentralized framework.

101

Chapter 7

Case study: DARPA’s Urban Challenge

The Defense Advanced Research Projects Agency (DARPA) 2007 Urban Challenge wasthe second of two challenges initiated by DARPA that put to the test the ability of currenttechnology to create truly autonomous systems that can drive by themselves in changingand complex environment without causing harm to themselves or their surroundings. Thechallenges required engineers to combine advanced control methods together with sens-ing capabilities and the ability to reason and make decisions about system state and thestate of the world. Its predecessor, the 2005 Grand Challenge required robotic vehicles tocomplete a 132 mile desert course in up to 10 hours, while dealing with rough terrain andstatic obstacles but with practically no interaction with other vehicles. The 2007 UrbanChallenge extended the previous challenge by moving to an urban environment where therobot had to complete a multi-part mission while negotiating hazards, overcoming blockedroads, avoiding both static and dynamic obstacles and obeying the California traffic laws.Figure 7.1 shows the site of the final Urban Challenge event together with the road seg-ments defined for the National Qualification Event (NQE).

In this chapter the DARPA 2007 Urban Challenge, specifically the planning and controlaspects, is used as a case study for the approach presented in Chapters 3 and 5 that tacklesthe control and decision making challenges of autonomous vehicles.

As discussed in Section 7.1, the teams that participated in the challenge had handwritten logic that was responsible for the high level behavior of their car (passing, makinga U-turn, etc.). The design of this logic was time and effort consuming and since in mostparts it was only tested, not verified, it led to vehicles exhibiting strange behaviors. Thestrengths of the approaches described in this thesis are that they allow the system designerto reason at a high level, have guarantees of correctness and easily and quickly createcomplex systems thus replacing error prone hand coded design.

102

Figure 7.1: Site of the NQE and final challenge

7.1 The DARPA Urban Challenge

7.1.1 The ChallengeFor DARPA’s 2007 Urban Challenge, a robot vehicle had to be able to complete a missionwhich comprises of going through a sequence of checkpoints in a 60 mile course. It hadto obey the California traffic laws while encountering traffic circles and intersections. Fur-thermore, it had to operate in an environment with other moving vehicles, static obstaclessuch as parked cars, and blocked roads.

Each Urban Challenge team received two text files which describe the challenge. TheRoute Network Definition File (RNDF) contained the description of the course. It spec-ified road segments by giving information about lanes, waypoints and locations of stopsigns and checkpoints. It also included descriptions of zones which are polygonal areasin which the vehicle can move freely (while avoiding obstacles) and parking spots thatare within these zones. The second file, which was given to the team five minutes be-

103

fore the start of the mission, was the Mission Data File (MDF) containing the sequence ofcheckpoints to be traversed together with speed limits for the different road segments.

As a safety precaution, all teams had to include the emergency stop (Estop) systemprovided by DARPA. This system was used to send start and stop commands remotelyto the vehicles. It consisted of a radio receiver which was installed on the vehicle and aremote transmitter that had two switches. One switch sends a Enable/Disable commandand the other switch a Run/Pause command. In Disable mode the vehicle had to, irrespec-tive of the other switch, first stop and then disable the main propulsion unit and as manyvehicle systems as necessary (e.g. fuel pump and electrical bus) to shut the vehicle down.In Enable mode the behavior depended on the Run/Pause switch; in Run mode the vehiclehad to carry out its mission and in Pause mode it had to come to a complete stop.

7.1.2 The general solutionsA requirement of all team participating in the semifinal event, the National QualificationEvent (NQE), was to write a technical paper describing, among other things, the designapproach and system architecture [39]. Looking at these reports revels that many teamschose common design concepts such as creating a modular architecture where a varietyof interconnected modules were used for various tasks. These tasks can be roughly splitinto two categories, one dealing with gathering the sensor information and estimating thestate of the vehicle and the world while the other deals with planning the mission andcontrolling the vehicle.

Here, the approach of Chapter 3 is demonstrated by focusing on the planning andcontrol aspects of the Urban Challenge. Figure 7.2 shows a high level generalization ofthe approach many semifinalists took when designing their planning and control systems.Starting from the highest level, the mission planner takes the RNDF and the MDF andfinds a sequence of waypoints that need to be traversed in order to complete the mission.It then sends to the Finite State Machine (FSM) the next waypoint that should be reached.The FSM, based on the state of the vehicle and the world that it receives from the sensormodules, searches for a suitable behavior that will cause the robot to reach the given way-point (driving, changing lanes, making a U-turn). Following that, the FSM either sendsthe controller a reference trajectory to track or sends a message to the planner that thewaypoint cannot be reached, for example if the road is blocked. If a reference trajectory isfound, the controller then sends steering and velocity commands to the vehicle that ensurethe vehicle moves according to the required path.

7.1.3 Synthesizing the planner and controllerThe mission planner can employ different graph searching techniques such as Dijkstra’salgorithm and A* search. The controller can use a wide range of well studied controlmethods such as sampling based or feedback policies [29]. However, the FSM is stillwritten and tested by hand in a tedious and error prone manner. While some teams haveused formal verification to validate small parts of their design (team Caltech verified pieces

104

Figure 7.2: A General high level description of the planning and control architecture ofvarious DARPA Challenge teams

of Alice), the full system was tested in a limited way and simulated without any formalguarantees that it actually does what the system designer intended it to do.

In this chapter it is shown how this whole complex subsystem, containing the missionplanner, the hand coded behavior FSM and the controller, can be automatically synthesizedfrom a high level description of the problem. This synthesis approach allows the systemdesigner to either decompose the problem into several interacting pieces or treat it as onelarge system. It naturally supports giving many short instructions, which are easier tospecify than long statements with many clauses and it is guaranteed to either producea system that always satisfies those instructions by construction or give the designer anindication that the required behavior cannot be satisfied.

105

7.2 The synthesized systemThis section elaborates on the specifications used to generate the Urban Challenge au-tomata. While the whole planning and control subsystem can be created as one very largehybrid controller, here two subsystems are created as depicted in Figure 7.3.

Figure 7.3: Synthesized planning and control system

The first subsystem is a discrete automaton (much like the behavior FSM) that capturesthe road behavior combined with sensor information that is gathered by sensor processingmodules which are not addressed in this work. It is used to detect hazardous conditionsthat require the vehicle to stop, to recognize blocked roads and to determine behavior atintersections. The second subsystem is a hybrid controller that drives the vehicle such thatit satisfies its mission while taking into account obstacles and blockages.

106

7.2.1 Subsystem I - Traffic BehaviorThis subsystem is designed to alert the driving control to situations in which it must stopthe car and to situations in which the road or lane is blocked. In order to keep it small andeasier to specify, three separate interconnected automata are defined that capture all thedesired behaviors. These automata include the Estop automaton, an automaton that dealswith intersection behavior, or “right of way” and finally one that deals with obstacles,determines blockages and sends inputs to driving control described in Section 7.2.2. In thefollowing, the input and output propositions are defined together with required behavior forthe Estop, intersection and the obstacle automata. Note that these three automata producebinary (discrete) outputs therefore in this section the automata do not require a continuousexecution.

Estop Automaton:

The Estop task leads to the definition of four discrete propositions. The sensor, or in thiscase transmitted, propositions are { Enable, Run }. Since Disable/Enable and Run/Pauseare binary switches only one proposition is required for each switch. Thus Disable =¬Enable and Pause = ¬Run where ¬ is the logical connective “not”. The automaton’soutputs to be used by the robot’s main controller are { Stop, ShutDown }.

The Structured English specifications for this task are:

• “Environment starts with True”

• “Robot starts with False”

The initial state of the world is assumed to be such that the Estop is Enable and Runtherefore the robot’s propositions Stop and ShutDown are both False. It should be notedthat this assumption is not necessary and these two sentences could be omitted .

• “Do ShutDown if and only if you are not sensing Enable” -Always and only when the Estop is in Disable mode, the robot must be shut down.

• “Do Stop if and only if you are sensing Enable and youare not sensing Run or you are not sensing Enable” - The ro-bot must stop whenever Estop is in Enable and Pause state or it is in Disable. Notethat in this specification the automaton issues a Stop command together with theShutDown when the Estop is in Disable mode.

The automaton for the Estop example is shown in Figure 7.4. The circles representthe automaton states and the propositions that are written inside each circle are the robotpropositions’ truth value in that state. The edges are labeled with the sensor propositions’values that enable that transition, for example a transition labeled with Enable,¬Run canbe taken only if the Estop switches are in Enable and Pause modes. A run of this automatonwould start at the top most state. In this state the robot should not stop and not shutdown.

107

Figure 7.4: Synthesized automaton for the Estop example

At every time step, depending on Enable and Run, the values of Stop and ShutDownmay change.

An automaton implementing the desired behavior for this simple example can be eas-ily written by hand. Furthermore, such an automaton would probably be smaller, havingonly three states (combining the left most and right most states in Figure 7.4 into one).However, as the number of propositions grows and as the specification becomes morecomplex writing such automata becomes cumbersome, time consuming and error prone.The strength of the method described here is that it automatically generates a correct au-tomaton whenever possible and alerts the user whenever a desired behavior cannot beachieved.

Intersection Automaton:

The sensor information is given through these binary input propositions

• intersection - The robot is at an intersection

• leftOcc, rightOcc, frontOcc - Lanes from Left/Right/Front in the intersection areoccupied. This is used to determine the whether the car should stop at an intersec-tion.

• leftMoved, rightMoved, frontMoved - Vehicle coming from Left/Right/Fronthas moved into and cleared the intersection

These output propositions describe the robot state as controlled by this automaton

• interOcc - The intersection is not free, the robot does not have right of way.

108

• leftClear, rightClear, frontClear - Lanes from Left/Right/Front in the intersec-tion are clear and the vehicle can move. The specifications regarding these proposi-tions will determine the vehicle’s behavior in an intersection.

The behavior of these outputs together with the assumptions made about the environ-ment, i.e. the input signals, are captured in the structured English specification below.Items 1 and 4 specify what happens initially, when the automaton starts executing. It isassumed that if the robot is not at an intersection the lanes cannot be occupied and that if alane was not occupied in the previous time step, a vehicle cannot move in that direction inthe current step (items 2,3). These assumptions are not necessary for the correct behaviorof the robot, they are used here to demonstrate the possibility of such assumptions and toreduce the size of the generated automaton. Note that while they are not necessary, oncespecified the inputs must obey these assumptions in order for the automaton to behavecorrectly (Section 3.5).

Items 4 to 9 specify how the outputs should behave. The output interOcc is set when-ever a direction is not considered clear (Item 5). Items 6 to 10 specify under what condi-tions a direction is considered clear. All directions are clear if not at an intersection (Item6), if a vehicle has moved (Item 7) or if when arriving at an intersection the direction isclear (Item 10). A direction is not clear if it wasn’t clear before and no vehicle moved inthat direction (Item 8) or if arriving at an intersection and a vehicle is already there (Item9). This behavior assumes all intersections are all-way stops and that the right of way isgiven only to the first vehicle in each direction that arrived at the intersection before therobot did.

1. “Environment starts with False”

2. “If you did not sense intersection then always not leftOcc” -same for right and front

3. “If you did not sense leftOcc then always not leftMoved” -same for right and front

4. “Robot starts with not interOcc and leftClear and rightClearand frontClear”

5. “Do not interOcc if and only if you are activatingleftClear and you are activating rightClearand you are activating frontClear”

6. “If you are not sensing intersection then do leftClearand rightClear and frontClear”

7. “If you are sensing leftMoved then do leftClear” - same for rightand front

109

8. “If you are sensing intersection and you did not activateleftClear and you are not sensing leftMoved then do notleftClear” - same for right and front

9. “If you did not sense intersection and you are sensingintersection and you are sensing leftOcc then do notleftClear” - same for right and front

10. “If you did not sense intersection and you are sensingintersection and you are not sensing leftOcc then doleftClear” - same for right and front

An automaton implementing the specification takes a few seconds to compute and has134 states.

Using this approach, it is very easy and fast to adjust the desired behavior of the robot,add inputs and change the outputs as opposed to hand coded automata. If, for example,the task is to implement a right-timid driver that only enters an intersection when its histurn and there are no cars waiting on his right, then the input proposition rightMoved andthe robot propositions rightClear are omitted, Items 3,6-9 for rightMoved are removed,4 is adjusted and item 5 is changed to have ¬rightOcc instead of rightClear.

One can easily encode more complex behaviors. Such as requiring the robot to alwaysgive right of way to two cars coming from the right and never to cars coming from the left,or always wait a few seconds before moving by adding a timer (as shown in Section 7.2.1).Other traffic behaviors might require extra inputs such as intersection type (all-way stopor merging into a highway), other vehicle signaling lights or more detailed outputs such asslow down, speed up (if already inside the intersection and a another car is approaching)or honk the horn. Such changes can come up during system development and this methodcan facilitate and expedite the process of generating the controller.

Obstacle Automaton:

This subsystem is designed to alert the driving control to situations in which it must stopthe car and to situations in which the road or lane is blocked. The input propositions forthis automaton capture both sensor information and the state determined by the Estop andintersection automata.

• stop - The vehicle received a stop command from the Estop system. This input isthe Stop output of the automaton described in section 7.2.1

• obstacle - There is an obstacle in the way

• interOcc - The vehicle is at an intersection and doesn’t have right of way. Thisinput is the interOcc output of the intersection automaton of section 7.2.1

• timerUp - A timer has timed up. This is used to decide when the lane is blockedrather than temporarily obstructed.

110

The output propositions are

• hazard - The vehicle should take into consideration that there is a hazard, either anobstacle in the lane, someone else’s turn at an intersection or an Estop was issued.The driving control will respond to this signal by stopping the car, as explained inSection 7.2.2

• blocked - The current lane is blocked.

• startT imer, resetT imer - Set and reset a timer to determine when an obstacle isconsidered a lane block.

The specification for this automaton is shown below. Initially all input and outputpropositions are False (Items 1,3). It is assumed that the timer indicates it has elapsed ina single time step (Item 2). This assumption is also not necessary for the behavior of therobot.

Items 3 to 14 specify how the outputs should behave. The output hazard is set when-ever stop is detected or there is no obstacle but the intersection is occupied (Items 4,5).The specification deals with obstacles by setting a timer whenever an obstacle is encoun-tered for the first time (Item 13) and resetting it if the obstacle has been cleared or stopreceived (Item 14). While an obstacle is being encountered, if the timer has not elapsedthen hazard is set (Items 7,8) otherwise blocked is set (Items 9-11). In any case, hazardand blocked are never True at the same time (Item 6). Whenever everything is free, i.e. noEstop, obstacle or occupied intersection, both hazard and blocked should be False (Item12).

1. “Environment starts with False”

2. “If you sensed timerUp then always not timerUp”

3. “Robot starts with False”

4. “If you are sensing stop then do hazard”

5. “If you are sensing interOcc and you are not sensingobstacle then do hazard”

6. “Always not (hazard and blocked)”

7. “If you are sensing obstacle and you did not activateblocked and did not activate hazard then do hazard”

8. “If you are sensing obstacle and you are not activatingblocked then do hazard”

9. “If you are sensing obstacle and you are not sensing stopand you activated block then do blocked”

111

10. “If you are sensing obstacle and you are not sensing stopand you activated hazard and you are sensing timerUpthen do blocked”

11. “If you are sensing obstacle and you activated hazardand you are not sensing timerUp then do not blocked”

12. “If you are not sensing obstacle and you are not sensingstop and you are not sensing interOcc then do not blockedand not hazard”

13. “Do startT imer if and only if you are sensing obstacleand you are not sensing stop and(you did not activate hazard or you sensed stop)”

14. “Do resetT imer if and only if you are sensing stop oryou are not activating hazard”

An automaton implementing the specification takes less than a second to compute andhas 25 states.

7.2.2 Subsystem II - Driving controlThe second subsystem takes as input the hazard and blocked outputs of the first subsys-tem and data structures that were created by parsing the RNDF and the MDF files andoutputs continuous velocity, steering and signaling commands. This subsystem is a hybridcontroller that executes a discrete automaton by composing atomic controllers based onthe inputs it receives.

Automaton

The input propositions for the discrete automaton are

• hazard

• blocked

The outputs of the automaton are

• signalL, signalR - The left or right signal lights need to be turned on

• stop, stopSign - The controller must cause the car to stop, either until a hazard hascleared or momentarily at a stop sign

• waypoint - A binary vector encoding the next waypoint that needs to be reached.

112

For this automaton the goal is to reach a sequence of checkpoints while negotiatinghazards and blockages. No assumptions are made on the behavior of hazard thereforthe goal is stated as “reach a checkpoint or stop” (see Item 11 below). However, if noassumptions are made on blocked the environment can prevent the robot from reachingits goal simply by blocking all lanes leading to the checkpoint. This may cause the robotto move in a loop while never reaching its goals and therefore the synthesis algorithmwill terminate with a message that the specification cannot be satisfied and an automatonwill not be created. To avoid this problem it is assumed that at some unspecified point intime, there will be no more blocks. To encode that a dummy input endBlocked is definedthat when set to True guarantees that no more blocks will occur. It is also assumed thateventually this new input will become True (Items 2-4). These assumptions prohibit theenvironment from always blocking the way to the checkpoints which is an assumptionTrue for the real challenge as well since a mission cannot be completed if any checkpointis not accessible.

The synthesis algorithm generates an infinitely executing automaton meaning that itsatisfies its desired behavior infinitely many times. In this specifications the goals of therobot are to reach a set of checkpoints; if the first checkpoint can be reached from the last(creating a possibly infinite loop of execution) the automaton is created. However, if thelast point is a one way dead end road, the algorithm will report that the desired behaviorcannot be satisfied. In the NQE there were two types of missions, repeating and non-repeating. In zone A the mission was to loop as many times as possible in a set time framebetween two checkpoints. For this mission an automaton was created that causes the robotto reach these two points repeatedly. To create an automaton for non-repeating missions anextra robot output missionCounter was added that keeps track of how many checkpointswere reached and after all were visited, the automaton issues a stop command.

The LTL formula ϕ that represents the desired behavior is created using StructuredEnglish (Items 1-6) and an automatic tool that transforms the RNDF and MDF informationto the required logical statements (Items 7-11).

1. “Environment starts with False” - it is assumed that initially hazardand blocked are False

2. “If endBlocked then always endBlocked” - it is assumed that once the ro-bot is told that there are no more blocks that information will not change (this as-sumption is for technical reasons, in the actual execution of the automaton this inputwill be set to False)

3. “If endBlocked then always not blocked” - it is assumed that if the robotis told that there are no more blocks, there will indeed be no more blocks

4. “Infinitely often endBlocked” - it is assumed that at some point we willknow that there are no more blocks

5. “Robot starts in initialWaypoint with False” - The vehicle’sstarting position is captured by initialWaypoint

113

6. “Do stop if and only if you are sensing hazard”

7. Topology: automatic encoding of the adjacency relation of the waypoints basedon the parsing and preprocessing of the RNDF file. There are two types of linksconnecting pairs of waypoints, regular links and “escape” links needed to avoid ablock (these links either change a lane or perform a U-turn). The desired behavioris to use the regular lanes whenever possible and only use the escape links whena block is detected. Therefore the statements added to ϕ are of the form: “If youwere in waypointi then always waypointi or (regularSuccessor and not blocked)or (escapeSuccessor and blocked)”

8. Signal lights: First, the set of links that connect two waypoints using a turn is de-termined. Then, for both signalL and signalR, based on the direction of the turns,the statement added is: “Do signalL if and only if (you were in waypointsource1and you are in waypointtarget1) or (you were in waypointsource2 and you are inwaypointtarget2) or ... ” for all source/target waypoints of the relevant turn links.

9. Stop signs: Waypoints that are specified in the RNDF as locations with a stop signare found and the corresponding statement is “Do stopSign if and only if you are in(stopWaypoint1 or stopWaypoint2 or ...)”

10. Mission counter: This binary vector is only added in non-repeating missions. Therobot initialization sets this vector to zero, meaning that no checkpoint has beenreached. The statements added to increment this counter and to stop the robot afterthe last checkpoint is reached are

• “If you are in lastCheckpoint then do stop”

• “If you are in checkPointi and you activated counteri−1

then do counteri”

• “If you are not in checkPointi and you activatedcounteri−1 then do counteri−1”

• “If you are in lastCheckpoint and you activatedcounterlast then do counterlast” - This is just to stop the counter fromchanging once the robot completed the mission

11. MDF Checkpoints: for every waypoint that corresponds to a checkpoint in theMDF the sentence: “Go to waypoint or stop” is added. If the stop proposition wasnot added, the synthesis algorithm would have returned that the desired task is notrealizable since there are no assumptions about the behavior of the hazard input,which causes the robot to stop, thus it could be always True preventing the robotfrom moving to its goals. Furthermore, for non-repeating missions, after reachingthe final checkpoint the robot is stopped.

114

This automaton uses the fine grained waypoints to define the current and next locationof the vehicle. If a smaller automaton is desired, the lanes and zones can be definedinstead for the vehicles location. For the NQE example, the RNDF included 53 lanes in41 segments and 3 zones while defining 405 waypoints.

The automatically created automaton that satisfies the above specifications can be largein size, however it contains all contingency plans. If any road is blocked or any laneoccupied, there is no need to replan online for a different route since all these routes arealready encoded as an execution path in the automaton and can be chosen based on theinputs received in that situation.

The continuous execution of this automaton follows Algorithm 2 of Chapter 3. Theatomic controllers can be either feedback controllers [35,79] or sampling based controllers[29, 75] where the next waypoint serves as the goal and the road as the constraints.

7.3 SimulationsThe approach described here is demonstrated by simulations in MATLABTMof differenttraffic scenarios using the actual RNDF and MDFs of the NQE. For this event, the RNDFcontained 405 waypoints defining three areas, each of which had at least one MDF asso-ciated with it. Figure 7.5 depicts areas A and C which are the areas used for the followingsimulations.

(a) Area A (b) Area C

Figure 7.5: Close up of areas A and C of the NQE

Since the focus here is the planning and hybrid control rather than the atomic con-trollers driving the robot from one waypoint to the next, the robot is simulated as a fullyactuated point robot that perfectly tracks the reference trajectory connecting two adjacentwaypoints. More complex dynamics can be handled as shown in Chapter 4 and in [36,79]

115

All behaviors seen in the following have been generated using the exact same behavior au-tomaton and the driving controller relating to the MDF. The different behaviors exhibitedare due to the different sensor information gathered in the different scenarios.

In the following figures, the robot is represented as a blue square whenever it is movingand as a red triangle when it stops. Waypoints are marked with a blue ‘+’ while waypointsthat have a stop sign are marked with a blue ‘*’. If present, Obstacles are represented bya magenta ‘X’, other vehicles by different colored stars and Estop command by a magentacircle around the robot.

Figures 7.6,7.7 and 7.8 depict area A where the mission is to repeatedly reach thecheckpoints marked with a red diamond. Note that the leftmost waypoint in the middlehorizontal road has a stop sign associated with it. Figure 7.6 shows snapshots of thebehavior induced by encountering an obstacle, Figure 7.7 shows the behavior when a roadblock is detected and Figure 7.8 shows how the robot reacts to an Estop (Enabled andPause) command.

(a) moving towards checkpoint 2 (b) stopping due to obstacle

(c) moving after obstacle cleared (d) stopping at stop sign

Figure 7.6: Behavior while encountering an obstacle

Figure 7.9 depicts area C where there are two 4-way stops. In this area, two othervehicles are driving. Initially the robot enters the area (a), then it arrives at an intersection

116

(a) stopping due to obstacle (b) timer timed out

(c) taking a different route (d) block cleared

Figure 7.7: Encountering a temporarily blocked road

(a) Estop issued (b) resuming motion

Figure 7.8: Estop issued

117

shortly after the green car stopped (b). The robot then stops until the green car has clearedthe intersection and then resumes its course (c,d).

(a) initial locations of cars (b) both the robot and the green car stop

(c) the green car moves (d) the robot moves

Figure 7.9: 4-way stop behavior

118

Chapter 8

LTLMop

Most of the work presented in this thesis has been implemented in a tool called LTLMop(Linear Temporal Logic MOtion Planning) written together with Cameron Finucane. Thistool is written in Python and it calls TLV [90] for the synthesis algorithm and Dotty whichis part of the Graphviz package [1] for the automaton visualization. It also interfaces tothe Player/Stage/Gazebo [2, 50] project which allows the user to simulate a robot in a twodimensional simulation (Stage) a three dimensional simulation that has a physics engine(Gazebo) or run a robot using a Player client. The Python, TLV and Dotty portions ofthe tool can be run on either Linux or Windows operating systems; however installingPlayer/Stage on a Windows machine is far from trivial therefore this tool was tested onLinux machines only. The following describes the different features of this tool and whatcan be done with it.

Figure 8.1 shows the main interface, the specification editor SpecEditor. The boxes onthe right display the set of propositions, the regions (top most), sensor inputs (second fromtop), actions (bottom) and custom propositions (below actions, seen in Figure 8.4b). Theregion propositions relate to the partition of the workspace, as discussed in Section 8.1and the sensor and action propositions relate to a robot model as explained in Section 8.2.In SpecEditor the user can also define task specific propositions (for example, to serve asmemory propositions). The box on the left is where the Structured English specification iswritten as discussed in Section 8.2. The box on the bottom displays in one tab the results(errors or success) of parsing the specification, creating the automaton and running thesimulation. The other tab displays the LTL formula which is automatically created fromthe Structured English instructions.

With SpecEditor the user can:

• Create, save and load specification files (Section 8.2)

• Load robot descriptions (Section 8.2)

• Create, save and load workspaces and their partition by calling RegionEditor (Sec-tion 8.1)

• Add propositions (Section 8.2)

119

• Choose which of the sensor and action propositions are relevant to the task (Section8.2)

• Write Structured English instructions (Section 8.2)

• Compile the specifications to LTL, if the syntax is correct, and then to an automatonif realizable (Section 8.3)

• Be notified whether an error occurred - wrong syntax or unrealizable specification(Section 8.3)

• Visualize the synthesized automaton (Section 8.3)

• Configure a simulation - initial conditions, values of sensor propositions, simulationtype (Section 8.4)

• Run simulations in Stage and Gazebo (Section 8.4)

• Run experiments with a real robot

Figure 8.1: Specification Editor

120

8.1 Region EditorThe program that facilitates the partition of a workspace, RegionEditor can be called frominside SpecEditor by clicking on the ‘edit regions’ button, or as a stand alone programfrom the terminal.

This program has the following functionality:

• The user can create, save and load region files

• The user can load images to serve as a background (as seen in Figure 8.2)

• The user can define new regions - by using either rectangles or defining the verticesof polygons (Figure 8.3a)

• The user can adjust regions by

– Moving, adding or deleting vertices (Figure 8.3d)

– Deleting regions

– Renaming regions

• The program automatically detects common edges, used later on for adjacency in-formation (Figure 8.3b)

• The program automatically detects nonconvex region, and marks them with slantedlines (Figure 8.3c)

• The region file created includes region names (to be used as the propositions inSpecEditor), location and size and the adjacency graph (to be used as part of thespecification in SpecEditor)

121

Figure 8.2: Region Editor displaying a workspace and its partition

122

(a) Defining new regions (b) Common edge detected automatically

(c) New region is not convex (noted by lines) (d) Vertex moved to make the region convex

Figure 8.3: Region Editor

123

8.2 SpecificationA robot task is described using three components, as discussed in Chapter 3; a robot, aworkspace and a set of specifications. In LTLMop, these components are captured usingthree files:

1. RobotName.robot - This file, written by hand, defines a robot model which includesinformation about the sensors, actions, controllers and simulation model. The sensorinformation is captured using a list of allowable sensor propositions and the same forthe actions. The file defines which controller code to use as the atomic controllers,thus allowing flexibility in the controller choice and it points to a more complexrobot model needed by the Gazebo simulator. A sample file is shown in Figure C.1.

2. WorkspaceName.regions - This file defines a workspace; The regions, their geometryand location, their name, the adjacency information, and the background image. Thefile is automatically written by RegionEditor every time a workspace is saved.

3. TaskName.spec - This is the main file which is written by SpecEditor. It defineswhich robot and regions file are part of this task, which robot and region propositionsare used as well as new custom propositions and it contains the Structured Englishinstructions. Furthermore, the simulation parameters of the last saved simulation arepart of this file as well. A sample file is shown in Figure C.2.

The first step of describing a new task is to import a robot file and a workspace fileusing the File menu. Once imported, the region propositions from the workspace aswell as the sensor and action propositions from the robot definition appear in the box onthe right side of SpecEditor as shown in Figure 8.4a. The propositions relating to thesensor inputs and the actions have check boxes next to them. Initially all the boxes arechecked, indicating that all propositions will be used for the task. If some propositionis not needed, it may be unchecked and then it will not be included in the LTL formula.Scrolling down the box on the right (Figure 8.4b), the user can define task specific custompropositions which can be used for the Structured English instructions and will be addedto the formula.

Once the required set of propositions is defined, the user writes out the task usingthe Structured English grammar described in Chapter 5, Table 5.1. Double clicking on aproposition name inserts it into the text of the specification. When writing, lines startingwith # are considered comments and are ignored during the translation to LTL.

124

(a)

(b)

Figure 8.4: Writing specifications

125

8.3 Compilation and Automaton VisualizationTranslating the Structured English to LTL and synthesizing an automaton is done usingthe Compile button in the Run menu. Once pressed, the specification together with thepropositions and the workspace adjacency information are translated into an LTL formulaas shown in Figure 8.5a-b. After the input files to the synthesis algorithm are written, theautomaton is synthesized using the TLV system [90, 99] . If the specification is realizablea success message is displayed as in Figure 8.5c.

If there were errors or warnings along the way they are displayed, as shown in Figure8.6. Errors include wrong syntax of the Structured English (a) and the specification beingunrealizable (c). A warning is issued if a proposition is selected (there is a check signnext to it in the proposition box) however it does not appear in the Structured Englishinstructions (Figure 8.6(b)). This warning is used to draw the attention of the user thatsome information is not used and since it is not constrained, each such unused propositiondoubles the size of the resulting automaton.

The resulting automaton can be displayed visually using Dotty [1]. Pressing the ViewAutomaton under the Run menu creates a PostScript file named TaskName.ps2 in thesame directory as the TaskName.spec file. An example for such an automaton is shown inFigure 8.7. This example has 120 states and can be visualized, however trying to visualizelarger automata could prove to be problematic.

126

(a) Compiling

(b) LTL formula

(c) Automaton successfully synthesized

Figure 8.5: Compilation

127

(a) Wrong Syntax

(b) Some selected propositions are not used

(c) Specification is unrealizable

Figure 8.6: Compilation Errors

128

Figure 8.7: Automaton is visualized using dotty [1]

129

8.4 SimulationThe Run menu has two buttons relating to the simulation; Configure Simulationand Simulate. The desired simulation is configured using the windows shown in Figure8.8. The initial state of the robot actions and of the custom propositions are set in theEnvironment Settings tab as well as the values of the sensor propositions in thedifferent regions (a). A checked box indicates that the sensor proposition written at thetop of the column is True in the region written next to that line. The environment in thesimulations is static in the sense that the user cannot change the simulated values of thedifferent propositions based on time but only based on the region of the workspace. Thus,as in 8.8(a), a fly is always sensed when the robot is in bacon.

The other tab, Simulation Settings displayed in Figure 8.8(b,c), allows theuser to toggle between the Stage and Gazebo simulators [2]. The regions defined on abackground image must be transformed to the coordinate system of the simulator; thereare four boxes for the X and Y scale and offset values. For a Stage simulation these valuescan be automatically computed by pressing the calibrate button and dragging the robot tothe top leftmost point in the workspace and then to bottom rightmost point. Note that thesevalues are different for Stage and Gazebo.

In the same tab the robot’s simulated initial region is selected from the list of regionsand the user may choose to overlay the regions on the background image during the simu-lation.

After the configuration is done, pressing the Simulate button starts the simulation.Figure 8.9 shows a Stage simulation and Figure 8.10 shows a Gazebo simulation. Inboth cases, the terminal from which SpecEditor was called displays a message every timethe robot changes its automaton state (for example when it reaches a new region). Thismessage includes all action and custom propositions that are True in that state as well aswhich region the robot is heading to next. These messages are seen in both Figure 8.9 and8.10 on the top left window.

130

(a) Setting initial conditions and inputs

(b) Stage (c) Gazebo

Figure 8.8: Simulation Configuration

131

Figure 8.9: Simulation using Stage

132

Figure 8.10: Simulation using Gazebo

133

Chapter 9

Conclusions and future work

The work presented in this thesis describes a framework for creating low level hybrid con-trollers that guarantee that a robot achieves a high level task, under certain assumptions.A high level task can be specified using Structured English and is formally captured usingLinear Temporal Logic. Using tools and ideas from the formal methods community suchas model-checking and program synthesis, discrete controllers satisfying the LTL formu-las are created. These discrete plans are then turned into low level hybrid controllers thatactivate a set of atomic or basic low level control laws based on the information the robotgathers at runtime.

This framework naturally accommodates different robot models, assuming low levelcontrollers with certain properties (bisimilar to the discrete abstraction of the workspace)can be created, and it can handle multi robot tasks in both a centralized and decentralizedmanner.

This work provides a natural connection between high level mission and motion plan-ning and low level control, while providing formal guarantees of correctness. Creatingsuch connections is crucial for developing robots and mechanical systems that exhibitcomplex and interesting behaviors while providing safety, predictability and flexibility.

9.1 Future workFuture directions of research follow the issues discussed in Sections 2.5 and 3.5:

• Optimality - The work discussed in this thesis guarantees that the behavior therobots exhibit is correct with respect to the task definition, however it does not guar-antee that this behavior is optimal. A natural extension of the work described hereis first to define appropriate optimality measures for nonreactive and reactive tasks.These measures are likely to be continuous measures such as path length, total en-ergy spent, time to completion, etc. Once defined, the challenge is to investigatethe correspondence between the continuous optimality and the discrete plans andfurthermore to develop algorithms that provide some form of optimal guarantees.

134

• Scalability - For reactive tasks the synthesis approach used in this work has bettercomplexity than the general synthesis problem however it still scales at the worstcase exponentially with the number of inputs. As the amount of information thetask requires increases, the size and computation time of the automaton increase.Looking at on-the-fly automaton extraction and exploring hierarchical approachescan make the synthesis approach suitable for larger sized problems.

• Uncertainty - Sensor inputs and localization are assumed to be prefect in this frame-work. These assumptions do not hold in any physical system operating in a realisticsetting therefore looking at the effects of noise and uncertainty on the results of thiswork must follow. In an uncertain framework, one can imagine providing proba-bilistic guarantees for tasks completion.

The physical workspace of the robots is also assumed to be know a priori, as is thediscrete abstraction. An interesting future direction is to extend the methods in thisthesis to handle unknown or partially known workspaces by combining SLAM typeexploration with planning and examining what guarantees can be provided regardingthe robots’ behavior.

• User Interface - Here Structured English sentences are used to define the task.While it is “friendlier” than writing logic formulas it is still too restrictive. Inves-tigating more intuitive interfaces such as a subset of natural language will add tothe usability of the proposed methods. Also, currently if the specification is unreal-izable, the algorithm simply returns that the specification cannot be satisfied but itdoes not point to the problem. Another direction for the future is to use the formalmethods theory to be able to give better feedback to the user as to what went wrongor why the specification is unrealizable.

Extension of the work presented in this thesis seems suitable to tackle many interestingapplications, from mission and motion planning in robotics, to manufacturing and assem-bly to computer graphics. These should provide plenty of new issues that will have to beinvestigated.

135

Appendices

136

Appendix A

NuSmv input and output

The following are the input file for the model checker NuSMV [30] that captured Example1 as well as the file created by the model checker that shows the counterexample thatfalsifies ¬ϕ.

137

-- NuSMV file: robot navigation using LTL formula-- number of robots: 1-- Example 1 of thesis-- June 16, 2008

MODULE robot(init_cond)VAR

node : 0 .. 68;ASSIGN

init(node) := init_cond;next(node) := case

node=1 : {9,11,12,14,26,28,29,31,43,45,46,48,60,62,63,65};

node=2 : {6,8,9,11,23,25,26,28,40,42,43,45,57,59,60,62};

node=3 : {11,13,14,16,28,30,31,33,45,47,48,50,62,64,65,67};

node=4 : {7,9,10,12,24,26,27,29,41,43,44,46,58,60,61,63};

node=5 : {12,14,15,17,29,31,32,34,46,48,49,51,63,65,66,68};

node=6 : {2,8,9,19,25,26,36,42,43,53,59,60};node=7 : {4,9,10,21,26,27,38,43,44,55,60,61};node=8 : {2,6,11,19,23,28,36,40,45,53,57,62};node=9 : {1,2,4,6,7,18,19,21,23,24,35,36,38,

40,41,52,53,55,57,58};node=10 : {4,7,12,21,24,29,38,41,46,55,58,63};node=11 : {1,2,3,8,13,18,19,20,25,30,35,36,37,

42,47,52,53,54,59,64};node=12 : {1,4,5,10,15,18,21,22,27,32,35,38,39,

44,49,52,55,56,61,66};node=13 : {3,11,16,20,28,33,37,45,50,54,62,67};node=14 : {1,3,5,16,17,18,20,22,33,34,35,37,39,

50,51,52,54,56,67,68};node=15 : {5,12,17,22,29,34,39,46,51,56,63,68};node=16 : {3,13,14,20,30,31,37,47,48,54,64,65};node=17 : {5,14,15,22,31,32,39,48,49,56,65,66};

Figure A.1: Input to NuSmv

138

node=18 : {9,11,12,14,26,28,29,31,43,45,46,48,60,62,63,65};

node=19 : {6,8,9,11,23,25,26,28,40,42,43,45,57,59,60,62};

node=20 : {11,13,14,16,28,30,31,33,45,47,48,50,62,64,65,67};

node=21 : {7,9,10,12,24,26,27,29,41,43,44,46,58,60,61,63};

node=22 : {12,14,15,17,29,31,32,34,46,48,49,51,63,65,66,68};

node=23 : {2,8,9,19,25,26,36,42,43,53,59,60};node=24 : {4,9,10,21,26,27,38,43,44,55,60,61};node=25 : {2,6,11,19,23,28,36,40,45,53,57,62};node=26 : {1,2,4,6,7,18,19,21,23,24,35,36,38,

40,41,52,53,55,57,58};node=27 : {4,7,12,21,24,29,38,41,46,55,58,63};node=28 : {1,2,3,8,13,18,19,20,25,30,35,36,37,

42,47,52,53,54,59,64};node=29 : {1,4,5,10,15,18,21,22,27,32,35,38,39,

44,49,52,55,56,61,66};node=30 : {3,11,16,20,28,33,37,45,50,54,62,67};node=31 : {1,3,5,16,17,18,20,22,33,34,35,37,39,

50,51,52,54,56,67,68};node=32 : {5,12,17,22,29,34,39,46,51,56,63,68};node=33 : {3,13,14,20,30,31,37,47,48,54,64,65};node=34 : {5,14,15,22,31,32,39,48,49,56,65,66};node=35 : {9,11,12,14,26,28,29,31,43,45,46,48,

60,62,63,65};node=36 : {6,8,9,11,23,25,26,28,40,42,43,45,57,

59,60,62};node=37 : {11,13,14,16,28,30,31,33,45,47,48,50,

62,64,65,67};node=38 : {7,9,10,12,24,26,27,29,41,43,44,46,58,

60,61,63};node=39 : {12,14,15,17,29,31,32,34,46,48,49,51,

63,65,66,68};node=40 : {2,8,9,19,25,26,36,42,43,53,59,60};

Figure A.1 (continued)

139

node=41 : {4,9,10,21,26,27,38,43,44,55,60,61};node=42 : {2,6,11,19,23,28,36,40,45,53,57,62};node=43 : {1,2,4,6,7,18,19,21,23,24,35,36,38,40,

41,52,53,55,57,58};node=44 : {4,7,12,21,24,29,38,41,46,55,58,63};node=45 : {1,2,3,8,13,18,19,20,25,30,35,36,37,42,

47,52,53,54,59,64};node=46 : {1,4,5,10,15,18,21,22,27,32,35,38,39,44,

49,52,55,56,61,66};node=47 : {3,11,16,20,28,33,37,45,50,54,62,67};node=48 : {1,3,5,16,17,18,20,22,33,34,35,37,39,50,

51,52,54,56,67,68};node=49 : {5,12,17,22,29,34,39,46,51,56,63,68};node=50 : {3,13,14,20,30,31,37,47,48,54,64,65};node=51 : {5,14,15,22,31,32,39,48,49,56,65,66};node=52 : {9,11,12,14,26,28,29,31,43,45,46,48,60,

62,63,65};node=53 : {6,8,9,11,23,25,26,28,40,42,43,45,57,59,

60,62};node=54 : {11,13,14,16,28,30,31,33,45,47,48,50,62,

64,65,67};node=55 : {7,9,10,12,24,26,27,29,41,43,44,46,58,

60,61,63};node=56 : {12,14,15,17,29,31,32,34,46,48,49,51,

63,65,66,68};node=57 : {2,8,9,19,25,26,36,42,43,53,59,60};node=58 : {4,9,10,21,26,27,38,43,44,55,60,61};node=59 : {2,6,11,19,23,28,36,40,45,53,57,62};node=60 : {1,2,4,6,7,18,19,21,23,24,35,36,38,40,

41,52,53,55,57,58};node=61 : {4,7,12,21,24,29,38,41,46,55,58,63};node=62 : {1,2,3,8,13,18,19,20,25,30,35,36,37,42,

47,52,53,54,59,64};node=63 : {1,4,5,10,15,18,21,22,27,32,35,38,39,44,

49,52,55,56,61,66};node=64 : {3,11,16,20,28,33,37,45,50,54,62,67};

Figure A.1 (continued)

140

node=65 : {1,3,5,16,17,18,20,22,33,34,35,37,39,50,51,52,54,56,67,68};

node=66 : {5,12,17,22,29,34,39,46,51,56,63,68};node=67 : {3,13,14,20,30,31,37,47,48,54,64,65};node=68 : {5,14,15,22,31,32,39,48,49,56,65,66};

esac;

MODULE mainVAR

robot_1 : process robot(18);LTLSPEC!( (F (r2 & F (r5 & F (r3 & (!r1 & !r2 & !r5)U r4))))

& G(r1 <-> a_s) & G((r2 | r3 | r4 | r5) -> a_c))

DEFINEr1:= robot_1.node=1 | robot_1.node=18

| robot_1.node=35 | robot_1.node= 52 ;r2 := robot_1.node=2 | robot_1.node=19

| robot_1.node=36 | robot_1.node= 53 ;r3 := robot_1.node=3 | robot_1.node=20

| robot_1.node=37 | robot_1.node= 54 ;r4 := robot_1.node=4 | robot_1.node=21

| robot_1.node=38 | robot_1.node= 55 ;r5 := robot_1.node=5 | robot_1.node=22

| robot_1.node=39 | robot_1.node= 56 ;r6 := robot_1.node=6 | robot_1.node=23

| robot_1.node=40 | robot_1.node= 57 ;r7 := robot_1.node=7 | robot_1.node=24

| robot_1.node=41 | robot_1.node= 58 ;r8 := robot_1.node=8 | robot_1.node=25

| robot_1.node=42 | robot_1.node= 59 ;r9 := robot_1.node=9 | robot_1.node=26

| robot_1.node=43 | robot_1.node= 60 ;r10 := robot_1.node=10 | robot_1.node=27

| robot_1.node=44 | robot_1.node= 61 ;

Figure A.1 (continued)

141

r11 := robot_1.node=11 | robot_1.node=28| robot_1.node=45 | robot_1.node= 62 ;

r12 := robot_1.node=12 | robot_1.node=29| robot_1.node=46 | robot_1.node= 63 ;

r13 := robot_1.node=13 | robot_1.node=30| robot_1.node=47 | robot_1.node= 64 ;

r14 := robot_1.node=14 | robot_1.node=31| robot_1.node=48 | robot_1.node= 65 ;

r15 := robot_1.node=15 | robot_1.node=32| robot_1.node=49 | robot_1.node= 66 ;

r16 := robot_1.node=16 | robot_1.node=33| robot_1.node=50 | robot_1.node= 67 ;

r17 := robot_1.node=17 | robot_1.node=34| robot_1.node=51 | robot_1.node= 68 ;

a_s := robot_1.node=18 | robot_1.node=19| robot_1.node=20 | robot_1.node= 21| robot_1.node=22 | robot_1.node=23| robot_1.node=24 | robot_1.node= 25| robot_1.node=26 | robot_1.node=27| robot_1.node=28 | robot_1.node= 29| robot_1.node=30 | robot_1.node=31| robot_1.node=32 | robot_1.node= 33| robot_1.node=34 | robot_1.node=52| robot_1.node=53 | robot_1.node= 54| robot_1.node=55 | robot_1.node=56| robot_1.node=57 | robot_1.node= 58|robot_1.node=59 | robot_1.node=60| robot_1.node=61 | robot_1.node= 62| robot_1.node=63 | robot_1.node=64| robot_1.node=65 | robot_1.node= 66| robot_1.node=67 | robot_1.node=68 ;

Figure A.1 (continued)

142

a_c := robot_1.node=35 | robot_1.node=36| robot_1.node=37 | robot_1.node= 38| robot_1.node=39 | robot_1.node=40| robot_1.node=41 | robot_1.node= 42| robot_1.node=43 | robot_1.node=44| robot_1.node=45 | robot_1.node= 46| robot_1.node=47 | robot_1.node=48| robot_1.node=49 | robot_1.node= 50| robot_1.node=51 | robot_1.node=52| robot_1.node=53 | robot_1.node= 54| robot_1.node=55 | robot_1.node=56| robot_1.node=57 | robot_1.node= 58| robot_1.node=59 | robot_1.node=60| robot_1.node=61 | robot_1.node= 62| robot_1.node=63 | robot_1.node=64| robot_1.node=65 | robot_1.node= 66| robot_1.node=67 | robot_1.node=68 ;

Figure A.1 (continued)

143

*** This is NuSMV 2.3.1 zchaff

*** For more information on NuSMV see

*** <http://nusmv.irst.itc.it>

*** or email to <nusmv-users@irst.itc.it>.

*** Please report bugs to <nusmv@irst.itc.it>.

-- specification !(( F (r2 & F (r5 & F (r3 &-- (((!r1 & !r2) & !r5) U r4)))) & G (r1 <-> a_s))-- & G ((((r2 | r3) | r4) | r5) -> a_c)) is false-- as demonstrated by the following execution sequenceTrace Description: LTL CounterexampleTrace Type: Counterexample-> State: 1.1 <-

robot_1.node = 18a_c = 0a_s = 1r17 = 0r16 = 0r15 = 0r14 = 0r13 = 0r12 = 0r11 = 0r10 = 0r9 = 0r8 = 0r7 = 0r6 = 0r5 = 0r4 = 0r3 = 0r2 = 0r1 = 1

Figure A.2: Output of NuSmv

144

-> Input: 1.2 <-_process_selector_ = robot_1

-> State: 1.2 <-robot_1.node = 11a_s = 0r11 = 1r1 = 0

-> Input: 1.3 <-_process_selector_ = robot_1

-> State: 1.3 <-robot_1.node = 36a_c = 1r11 = 0r2 = 1

-> Input: 1.4 <-_process_selector_ = robot_1

-> State: 1.4 <-robot_1.node = 11a_c = 0r11 = 1r2 = 0

-> Input: 1.5 <-_process_selector_ = robot_1

-> State: 1.5 <-robot_1.node = 37a_c = 1r11 = 0r3 = 1

-> Input: 1.6 <-_process_selector_ = robot_1

-> State: 1.6 <-robot_1.node = 14a_c = 0r14 = 1r3 = 0

Figure A.2 (continued)

145

-> Input: 1.7 <-_process_selector_ = robot_1

-> State: 1.7 <-robot_1.node = 39a_c = 1r14 = 0r5 = 1

-> Input: 1.8 <-_process_selector_ = robot_1

-> State: 1.8 <-robot_1.node = 15a_c = 0r15 = 1r5 = 0

-> Input: 1.9 <-_process_selector_ = robot_1

-> State: 1.9 <-robot_1.node = 46a_c = 1r15 = 0r12 = 1

-> Input: 1.10 <-_process_selector_ = robot_1

-> State: 1.10 <-robot_1.node = 18a_c = 0a_s = 1r12 = 0r1 = 1

-> Input: 1.11 <-_process_selector_ = robot_1

-> State: 1.11 <-robot_1.node = 11a_s = 0r11 = 1r1 = 0

Figure A.2 (continued)

146

-> Input: 1.12 <-_process_selector_ = robot_1

-> State: 1.12 <-robot_1.node = 37a_c = 1r11 = 0r3 = 1

-> Input: 1.13 <-_process_selector_ = robot_1

-> State: 1.13 <-robot_1.node = 47r13 = 1r3 = 0

-> Input: 1.14 <-_process_selector_ = robot_1

-> State: 1.14 <-robot_1.node = 11a_c = 0r13 = 0r11 = 1

-> Input: 1.15 <-_process_selector_ = robot_1

-> State: 1.15 <-robot_1.node = 42a_c = 1r11 = 0r8 = 1

-> Input: 1.16 <-_process_selector_ = robot_1

Figure A.2 (continued)

147

-> State: 1.16 <-robot_1.node = 6a_c = 0r8 = 0r6 = 1

-> Input: 1.17 <-_process_selector_ = robot_1

-> State: 1.17 <-robot_1.node = 43a_c = 1r9 = 1r6 = 0

-> Input: 1.18 <-_process_selector_ = robot_1

-> State: 1.18 <-robot_1.node = 38r9 = 0r4 = 1

-> Input: 1.19 <-_process_selector_ = robot_1

-- Loop starts here-> State: 1.19 <-

robot_1.node = 7a_c = 0r7 = 1r4 = 0

-> Input: 1.20 <-_process_selector_ = main

-- Loop starts here-> State: 1.20 <--> Input: 1.21 <-

_process_selector_ = main-> State: 1.21 <-

Figure A.2 (continued)

148

Appendix B

Context Free Grammar

The Context Free Grammar (CFG) that can translate simple natural language sentences tothe LTL formulas of Chapter 3 is shown in Tables B.1 and B.2. The semantics associatedwith this grammar are lambda calculus expressions that when reduced, result in the desiredLTL formula. Note that the described CFG does not allow negation which proved to bedifficult to handle in this framework.

Grammar RulesS[sem =<app(?vp,?np)>] → NP/NP[spec=?sp,sem=?np] VP[spec=?sp,sem=?vp]S[sem =<app(?cond,?np)>] → NP/NP[spec=?sp,sem=?np] CondP[spec=?sp,sem=?cond]NP[sem=<app(?det,?nom)> ] → Det[sem=?det] Nom[sem=?nom]NP[sem=?np] → PropN[sem=?np]NP[sem=<app(?nom,?con)> ] → PropN[sem=?nom] ConjA[sem=?con]NP[sem=<app(?nom,?con)> ] → PropN[sem=?nom] ConjO[sem=?con]ConjA[sem=<app(?nom,?con)> ] → PropN[sem=?nom] ConjA[sem=?con]ConjA[sem=<app(?nom,?a)> ] → And[sem=?a] PropN[sem=?nom]ConjMA[sem=<app(?a,?np)> ] → AndM[sem=?a] NP[sem=?np]NP[sem=<app(?nom,?con)> ] → PropN[sem=?nom] ConjMA[sem=?con]ConjO[sem=<app(?nom,?con)> ] → PropN[sem=?nom] ConjO[sem=?con]ConjO[sem=<app(?nom,?o)> ] → Or[sem=?o] PropN[sem=?nom]ConjO[sem=<app(?np,?o)> ] → Or[sem=?o] NP[sem=?np]CondP[spec=?sp,sem =<app(?ifp,?thenp)>] → IfP[sem=?ifp] ThenP[spec=?sp,sem=?thenp]IfP[sem =<app(?if,?cond)>] → IF[sem=?if] Cond[sem=?cond]ThenP[spec=?sp,sem =<app(?then,?cont)>] → THEN[sem=?then] Content[spec=?sp,sem=?cont]Cond[sem =<app(?vp,?np)>] → NP[sem=?np] VP[sem=?vp]Cond[sem =<app(?cond,?condconj)>] → Cond[sem=?cond] CondConj[sem=?condconj]CondConj[sem =<app(?cond,?and)>] → And[sem=?and] Cond[sem=?cond]CondConj[sem =<app(?cond,?or)>] → Or[sem=?or] Cond[sem=?cond]Content[spec=?sp,sem=?vp] → VP[spec=?sp,sem=?vp]Nom[sem=?nom] → N[sem=?nom]VP[spec=?sp,sem=<app(?v,?obj)>] → TV[spec=?sp,sem=?v] NP[sem=?obj]VP[spec=?sp,sem=<app(?v,?pp)>] → TV[spec=?sp,sem=?v] PP[sem=?pp]VP[sem=<app(?v,?p)>] → V[sem=?v] P[sem=?p]PP[sem=<app(?p,?np)>] → P[sem=?p] NP[sem=?np]NP/NP[spec=liveness, sem=< \P. (GF P))>] →NP/NP[spec=safety, sem=< \P. (G P)>] →NP/NP[spec=init, sem=< \x. x>] →

Table B.1: CFG - Grammar rules

149

Lexical RulesPropN[sem=< \P. (P room1) >] → ’room1’PropN[sem=< \P. (P room2) >] → ’room2’PropN[sem=< \P. (P room3) >] → ’room3’Det[sem=< \P Q. (Q all x.(P x))>] → ’every’Det[sem=< \P Q. (Q some x.(P x))>] → ’some’Det[sem=< \P Q. (Q some x.(P x))>] → ’a’Det[sem=< \x. x>] → ’the’And[sem=< \P Q R. ((R Q) ND (R P))>] → ’and’AndM[sem=< \Z Q W. ((W Q) ND (Z W))>] → ’and’ | ’,’Or[sem=< \P Q R. (R (Q O P))>] → ’or’N[sem=< \P. (P room)>] → ’room’N[sem=< \P. (P lightOn)>] → ’light’TV[spec=liveness,sem=< \x. x>] → ’visit’TV[spec=liveness,sem=< \x. x>] → ’go’TV[spec=safety,sem=< \P Q. (P(Q NNx))>] → ’avoid’TV[spec=init,sem=< \x. x>] → ’start’V[sem=< \x. x>] → ’is’P[sem=< \x. x>] → ’to’P[sem=< \x. x>] → ’in’P[sem=< \x. x>] → ’on’P[sem=< \P Q. (Q(P N))>] → ’off’IF[sem=< \P Q R. (Q(R(P \x.x)))>] → ’if’THEN[sem=< \P Q. (P(Q imp))>] → ’then’THEN[sem=< \P Q. (P(Q imp))>] →

Table B.2: CFG - Lexical rules

150

Appendix C

LTLMop files

The following are sample files for LTLMop. Figure C.1 shows the structure of a robotdefinition file. Figure C.2 depicts the specification file containing all the information forthe task; Robot, workspace, specification and simulation parameters. Both are describedin Section 8.2.

151

Name:Frog

Sensors:FlyBirdSnakeHuman

Actions:PickUpDropJumpPonder

Controller:controller.py

Model:frog.gazebo

Figure C.1: Robot file

152

RegionFile:breakfast.regions

RobotFile:frog.robot

Sensors:Fly 1Bird 0Snake 0Human 0

Actions:PickUp 1Drop 1Jump 0Ponder 0

Customs:CarryingFly

InitialTruths:

InitialRegion:2

SensorRegions:Fly 1 4

XYTransform:0.023438 -12.000000 -0.023438 9.000000

SimType:Gazebo

Figure C.2 (continued)

153

BackgroundOverlay:1

VertexMarkers:1

Spec:# Initial ConditionsEnv starts with falseRobot starts with false

# TransitionsDo PickUp if and only if you are sensing Fly and you arenot activating CarryingFlyDo Drop if and only if you are activating CarryingFlyand you were in mysteryIf you are activating PickUp or you activated PickUpthen stay thereIf you are activating Drop or you activated Dropthen stay there

If you activated PickUp then do CarryingFlyIf you activated Drop then do not CarryingFly

If you activated CarryingFly and you did notactivate Drop then do CarryingFlyIf you did not activate CarryingFly and you did notactivate PickUp then do not CarryingFly

If you were in mystery then do not FlyIf you did not activate CarryingFly then always not mystery

Figure C.2: Specification file

154

# GoalsIf you are not activating CarryingFly then visit hash_brownIf you are not activating CarryingFly then visit baconIf you are not activating CarryingFly then visit eggyIf you are not activating CarryingFly then visit mushroomsIf you are not activating CarryingFly then visit toastIf you are not activating CarryingFly then visit beansIf you are not activating CarryingFly then visit tomatoIf you are activating CarryingFly then visit mystery

Figure C.2 (continued)

155

Bibliography

[1] Graphviz - graph visualization software, http://www.graphviz.org/.

[2] The player project, http://playerstage.sourceforge.net/.

[3] K. Altisen and S. Tripakis. Tools for controller synthesis of timed systems. In 2ndWorkshop on Real-Time Tools (RT-TOOLS’2002), july 2002.

[4] Rajeev Alur and David L. Dill. A theory of timed automata. Theoretical ComputerScience, 126(2):183–235, 1994.

[5] M. Antoniotti and B. Mishra. Discrete event models + temporal logic = supervi-sory controller: Automatic synthesis of locomotion controllers. IEEE InternationalConference on Robotics and Automation, 1995.

[6] P.J. Antsaklis. A brief introduction to the theory and applications of hybrid systems.Proc. IEEE, Special Issue on Hybrid Systems: Theory and Applications, 88(7):879–886, July 2000.

[7] E. Asarin, O. Maler, A. Pnueli, and J. Sifakis. Controller synthesis for timed au-tomata. In Proc. System Structure and Control. Elsevier, 1998.

[8] Nora Ayanian and Vijay Kumar. Decentralized feedback controllers for multi-agent teams in environments with obstacles. In IEEE International Conferenceon Robotics and Automation, Pasadena, CA, May 2008.

[9] Fahiem Bacchus and Froduald Kabanza. Planning for temporally extended goals. InProceedings of the Thirteenth National Conference on Artificial Intelligence (AAAI-96), pages 1215–1222, Portland, Oregon, USA, 1996. AAAI Press / The MIT Press.

[10] Fahiem Bacchus and Froduald Kabanza. Using temporal logics to express searchcontrol knowledge for planning. Artif. Intell., 116(1-2):123–191, 2000.

[11] A. Balluchi, A. Bicchi, A. Balestrino, and G. Casalino. Path tracking control forDubin’s car. In IEEE International Conference on Robotics and Automation, pages3123–3128, Minneapolis, MN, 1996.

156

[12] Gerd Behrmann, Alexandre David, Kim G. Larsen, Oliver Mller, Paul Pettersson,and Wang Yi. UPPAAL - present and future. In Proc. of 40th IEEE Conference onDecision and Control. IEEE Computer Society Press, 2001.

[13] C. Belta, A. Bicchi, M. Egerstedt, E. Frazzoli, E. Klavins, and G. J. Pappas. Sym-bolic planning and control of robot motion: State of the art and grand challenges.Robotics and Automation Magazine, 14(1):61–70, 2007.

[14] Calin Belta and L.C.G.J.M. Habets. Constructing decidable hybrid systems withvelocity bounds. In IEEE Conference on Decision and Control, Bahamas, 2004.

[15] Calin Belta, Volkan Isler, and George J. Pappas. Discrete abstractions for robotplanning and control in polygonal environments. IEEE Transactions on Robotics,21(5):864–874, October 2005.

[16] Shoham Ben-David, Cindy Eisner, Daniel Geist, and Yaron Wolfsthal. Modelchecking at ibm. Form. Methods Syst. Des., 22(2):101–108, 2003.

[17] P. Bertoli, A. Cimatti, M. Pistore, M. Roveri, and P. Traverso. Mbp: a model basedplanner. In In Proc. of the IJCAI’01 Workshop on Planning under Uncertainty andIncomplete Information, pages 93–97, August 2001.

[18] Piergiorgio Bertoli, Alessandro Cimatti, Marco Roveri, and Paolo Traverso. Strongplanning under partial observability. Artif. Intell., 170(4):337–384, 2006.

[19] A. Bicchi, A. Marigo, and B. Piccoli. Feedback encoding for efficient symbolic con-trol of dynamical systems. IEEE Trans. on Automatic Control, 51(6):1–16, 2006.

[20] Armin Biere, Alessandro Cimatti, Edmund M. Clarke, and Yunshan Zhu. Sym-bolic model checking without bdds. In TACAS ’99: Proceedings of the 5th In-ternational Conference on Tools and Algorithms for Construction and Analysis ofSystems, pages 193–207, London, UK, 1999. Springer-Verlag.

[21] Armin Biere, Keijo Heljanko, Tommi Junttila, Timo Latvala, and Viktor Schuppan.Linear encodings of bounded LTL model checking. Logical Methods in ComputerScience, 2(5:5), 2006. (doi: 10.2168/LMCS-2(5:5)2006).

[22] Johann Borenstein, Malik Hansen, and Adam Borrell. The omnitread ot-4 serpen-tine robot—design and performance: Field reports. J. Field Robot., 24(7):601–621,2007.

[23] Randal E. Bryant. Graph-based algorithms for boolean function manipulation.IEEE Transactions on Computers, 35(8):677–691, 1986.

[24] J. R. Burch, E. M. Clarke, K. L. Mcmillan, D. L. Dill, and L. J. Hwang. Symbolicmodel checking: 10¡sup¿20¡/sup¿ states and beyond. In Logic in Computer Science,1990. LICS ’90, Proceedings., Fifth Annual IEEE Symposium on e, pages 428–439,1990.

157

[25] R. R. Burridge, A. A. Rizzi, and D. E. Koditschek. Sequential composition ofdynamically dexterous robot behaviors. International Journal of Robotics Research,18(6):534–555, 1999.

[26] J. F. Canny. The Complexity of Robot Motion Planning. PhD thesis, MIT, Depart-ment of Electrical Engineering and Computer Science, Cambridge, MA, 1987.

[27] Christos G. Cassandras and Stephane Lafortune. Introduction to Discrete EventSystems. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2006.

[28] Serenella Cerrito and Marta Cialdea Mayer. Using linear temporal logic to modeland solve planning problems. Lecture Notes in Computer Science, 1480:141–153,1998.

[29] H. Choset, K. M. Lynch, L. Kavraki, W. Burgard, S. A. Hutchinson, G. Kantor, andS. Thrun. Principles of Robot Motion: Theory, Algorithms, and Implementations.MIT Press, Boston, USA, 2005.

[30] A. Cimatti, E. M. Clarke, E. Giunchiglia, F. Giunchiglia, M. Pistore, M. Roveri,R. Sebastiani, and A. Tacchella. NuSMV 2: An opensource tool for symbolic modelchecking. In In Proceeding of International Conference on Computer-Aided Verifi-cation (CAV 2002), 2002.

[31] A. Cimatti, M. Pistore, M. Roveri, and P. Traverso. Weak, strong, and strong cyclicplanning via symbolic model checking. Artif. Intell., 147(1-2):35–84, 2003.

[32] A. Cimatti, M. Roveri, and P. Bertoli. Conformant planning via symbolic modelchecking and heuristic search. Artif. Intell., 159(1-2):127–206, 2004.

[33] Edmund M. Clarke, Daniel Kroening, Joel Ouaknine, and Ofer Strichman. Com-pleteness and complexity of bounded model checking. In VMCAI, pages 85–96,Venice, Italy, Jenuary 2004.

[34] Edmund. M. Clarke and O. Grumberg D.A. Peled. Model Checking. MIT Press,Cambridge, Massachusetts, 1999.

[35] D. Conner, H. Choset, and A. Rizzi. Integrated planning and control for convex-bodied nonholonomic systems using local feedback control policies. In Proceedingsof Robotics: Science and Systems, Cambridge, USA, June 2006.

[36] David C. Conner, Hadas Kress-Gazit, Howie Choset, Alfred A. Rizzi, and George J.Pappas. Valet parking without a valet. In IEEE/RSJ Int’l. Conf. on Intelligent Robotsand Systems, pages 572–577, San Diego, CA, October 2007.

[37] David C. Conner, Alfred A. Rizzi, and Howie Choset. Composition of Local Poten-tial Functions for Global Robot Control and Navigation. In IEEE/RSJ Int’l. Conf.on Intelligent Robots and Systems, pages 3546 – 3551, Las Vegas, NV, October2003.

158

[38] David C. Conner, Alfred A. Rizzi, and Howie Choset. Construction and automateddeployment of local potential functions for global robot control and navigation.Technical Report CMU-RI-TR-03-22, Carnegie Mellon University, Robotics Insti-tute, Pittsburgh, Pennsylvania, USA, 2003.

[39] DARPA. Urban challenge documentation. available online athttp://www.darpa.mil/grandchallenge/rules.asp.

[40] Mark de Berg, Marc van Kreveld, Mark Overmars, and Otfried Schwarzkopf. Com-putational geometry: algorithms and applications. Springer-Verlag New York, Inc.,third edition, 2008.

[41] F. Delmotte, T.R. Mehta, and M. Egerstedt. Modebox a software tool for obtaininghybrid control strategies from data. Robotics and Automation Magazine, 15(1):87–95, 2008.

[42] E. Allen Emerson. Temporal and modal logic. In Handbook of theoretical com-puter science (vol. B): formal models and semantics, pages 995–1072. MIT Press,Cambridge, MA, USA, 1990.

[43] Niklas En and Niklas Srensson. An extensible sat-solver. In Enrico Giunchigliaand Armando Tacchella, editors, SAT, volume 2919 of Lecture Notes in ComputerScience, pages 502–518, Santa Margherita Ligure, Italy, 2003. Springer.

[44] Georgios E. Fainekos, Antoine Girard, Hadas Kress-Gazit, and George J. Pappas.Temporal logic motion planning for dynamic robots. Technical Report MS-CIS-07-02, Dept. of CIS, Univ. of Pennsylvania, January 2007.

[45] Georgios E. Fainekos, Hadas Kress-Gazit, and George J. Pappas. Hybrid controllersfor path planning: A temporal logic approach. In Proceedings of the 44th IEEEConference on Decision and Control, pages 4885 – 4890, December 2005.

[46] Georgios E. Fainekos, Hadas Kress-Gazit, and George J. Pappas. Temporal logicmotion planning for mobile robots. In IEEE International Conference on Roboticsand Automation, pages 2020–2025, Barcelona, Spain, 2005.

[47] Gerogios E. Fainekos, Antoine Girard, Hadas Kress-Gazit, and George J. Pappas.Temporal logic motion planning for dynamic mobile robots. Automatica. To appear.

[48] Stephan Flake, Wolfgang Muller, and J. Ruf. Structured english for model checkingspecification. In GI–Workshop Methoden und Beschreibungssprachen zur Model-lierung und Verifikation von Schaltungen und Systemen in Frankfurt, Berlin, 2000.VDE Verlag.

[49] E. Frazzoli, M. A. Dahleh, and E. Feron. Maneuver-based motion planning fornonlinear systems with symmetries. IEEE Trans. on Robotics, 21(6):1077–1091,December 2005.

159

[50] Brian Gerkey, Richard T. Vaughan, and Andrew Howard. The player/stage project:Tools for multi-robot and distributed sensor systems. In 11th International Confer-ence on Advanced Robotics ICAR, pages 317–323, Coimbra, Portugal, June 2003.

[51] A. Gromyko, M. Pistore, and P. Traverso. A tool for controller synthesis via sym-bolic model checking. In 8th International Workshop on Discrete Event Systems,pages 475–476, July 2006.

[52] L.C.G.J.M. Habets and J.H. van Schuppen. A control problem for affine dynamicalsystems on a full-dimensional polytope. Automatica, 40:21–35, 2004.

[53] G.J. Holzmann. The Spin Model Checker Primer and Reference Manual. Addison-Wesley, 2004.

[54] Jonathan W Hurst, Joel Chestnutt, and Alfred Rizzi. Design and philosophy of thebimasc, a highly dynamic biped. In IEEE Conference on Robotics and Automation,April 2007.

[55] R.M. Jensen and M. M. Veloso. OBDD-based universal planning for synchronizedagents in non-deterministic domains. Journal of Artificial Intelligence Research,13:189–226, 2000.

[56] Barbara Jobstmann and Roderick Bloem. Linear logic synthesizer (lily),http://www.ist.tugraz.at/staff/jobstmann/lily/ 2006.

[57] Barbara Jobstmann and Roderick Bloem. Optimizations for ltl synthesis. In FM-CAD ’06: Proceedings of the Formal Methods in Computer Aided Design, pages117–124, Washington, DC, USA, 2006. IEEE Computer Society.

[58] Daniel Jurafsky and James H. Martin. Speech and Language Processing (2nd Edi-tion). Prentice-Hall, Inc., Upper Saddle River, NJ, USA, 2006.

[59] Froduald Kabanza and Sylvie Thibaux, editors. AIPS 2002 Workshop on Planningvia Model Checking, Toulouse, France, April 2002.

[60] Roope Kaivola and Katherine R. Kohatsu. Proof engineering in the large: For-mal verification of pentium R©4 floating-point divider. In CHARME ’01: Proceed-ings of the 11th IFIP WG 10.5 Advanced Research Working Conference on CorrectHardware Design and Verification Methods, pages 196–211, London, UK, 2001.Springer-Verlag.

[61] S. Karaman and E. Frazzoli. Complex mission optimization for multiple-uavs usinglinear temporal logic. In American Control Conference, Seattle, Washington, 2008.

[62] Eric Klavins, Robert Ghrist, and David Lipsky. A grammatical approach to self-organizing robotic systems. IEEE Transactions on Automatic Control, 51(6):949–962, June 2006.

160

[63] M. Kloetzer and C. Belta. A fully automated framework for control of linear sys-tems from ltl specifications. In 9th International Workshop on Hybrid Systems:Computation and Control, Santa Barbara, California, 2006.

[64] Sascha Konrad and Betty H. C. Cheng. Facilitating the construction of specificationpattern-based properties. In Proceedings of the IEEE International RequirementsEngineering Conference, Paris, France, August 2005.

[65] Hadas Kress-Gazit, Nora Ayanian, George J. Pappas, and Vijay Kumar. Recyclingcontrollers. In IEEE Conference on Automation Science and Engineering, Wash-ington D.C., USA, 2008. To appear.

[66] Hadas Kress-Gazit, David C. Conner, Howie Choset, Alfred A. Rizzi, and George J.Pappas. Courteous cars: Decentralized multi-agent traffic coordination. Roboticsand Automation Magazine, 15(1):30–38, 2008.

[67] Hadas Kress-Gazit, Georgios E. Fainekos, and George J. Pappas. From structuredenglish to robot motion. In Proceedings of the IEEE/RSJ International Conferenceon Intelligent Robots and Systems, pages 2717–2722, Oct 2007.

[68] Hadas Kress-Gazit, Georgios E. Fainekos, and George J. Pappas. Where’s waldo?sensor-based temporal logic motion planning. In IEEE International Conference onRobotics and Automation, pages 3116–3121, Rome, Italy, 2007.

[69] Hadas Kress-Gazit, Gerogios E. Fainekos, and George J. Pappas. Translating struc-tured english to robot controllers. Advanced Robotics Special Issue on SelectedPapers from IROS 2007. To appear.

[70] Hadas Kress-Gazit and George J. Pappas. Automatically synthesizing a planningand control subsystem for the darpa urban challenge. In IEEE Conference on Au-tomation Science and Engineering, Washington D.C., USA, 2008. To appear.

[71] M. Kvasnica, P. Grieder, and M. Baoti. Multi-parametric toolbox (mpt),http://control.ee.ethz.ch/ mpt/ 2004.

[72] Ugo Dal Lago, Marco Pistore, and Paolo Traverso. Planning with a language forextended goals. In Eighteenth national conference on Artificial intelligence, pages447–454, Menlo Park, CA, USA, 2002. American Association for Artificial Intelli-gence.

[73] Khaled Ben Lamine and Froduald Kabanza. Reasoning about robot actions: Amodel checking approach. In Revised Papers from the International Seminar onAdvances in Plan-Based Control of Robotic Agents,, pages 123–139, London, UK,2002. Springer-Verlag.

[74] S. M. LaValle. Planning Algorithms. Cambridge University Press, Cambridge,U.K., 2006.

161

[75] S. M. LaValle. Planning Algorithms, chapter 5. Cambridge University Press, Cam-bridge, U.K., 2006. Chapter 5: Sampling-Based Motion Planning.

[76] S. Lindemann and S. LaValle. Computing smooth feedback plans over cylindri-cal algebraic decompositions. In Proceedings of Robotics: Science and Systems,Cambridge, USA, June 2006.

[77] S. R. Lindemann and S. M. LaValle. Smoothly blending vector fields for globalrobot navigation. In IEEE Conference on Decision and Control, Seville, Spain,2005.

[78] S. R. Lindemann and S. M. LaValle. Smooth feedback for car-like vehicles inpolygonal environments. In IEEE Conference on Robotics and Automation, 2007.

[79] Stephen. R. Lindemann, Islam I. Hussein, and Steven M. LaValle. Realtime feed-back control for nonholonomic mobile robots with obstacles. In IEEE Conferenceon Decision and Control, San Diego, CA, 2006.

[80] S. Loizou and K. Kyriakopoulos. Automatic synthesis of multi-agent motion tasksbased on ltl specifications. In IEEE Conference on Decision and Control, Bahamas,December 2004.

[81] John Lygeros, Datta N. Godbole, and Shankar Sastry. Verified hybrid controllersfor automated vehicles, April 1998.

[82] Christopher D. Manning and Hinrich Schtze. Foundations of Statistical NaturalLanguage Processing. The MIT Press, June 1999.

[83] Robert Mattmuller and Jussi Rintanen. Planning for temporally extended goalsas propositional satisfiability. In Manuela M. Veloso, editor, IJCAI, pages 1966–,2007.

[84] A. Narkhede and D. Manocha. Fast polygon triangulation based on seidel’s algo-rithm. available online at http://www.cs.unc.edu/ dm/CODE/GEM/chapter.html.

[85] Dana Nau, Malik Ghallab, and Paolo Traverso. Automated Planning: Theory &Practice. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2004.

[86] J.S. Ostroff. Synthesis of controllers for real-time discrete event systems. In IEEEConference on Decision and Control, Tampa, FL, 1989.

[87] J.S. Ostroff and W.M. Wonham. State machines, temporal logic and control: aframework for discrete event systems. In IEEE Conference on Decision and Con-trol, Los Angeles, CA, 1987.

[88] Nir Piterman, Amir Pnueli, and Yaniv Sa’ar. Synthesis of Reactive(1) Designs. InVMCAI, pages 364–380, Charleston, SC, Jenuary 2006.

162

[89] A. Pnueli and R. Rosner. On the synthesis of a reactive module. In POPL ’89:Proceedings of the 16th ACM SIGPLAN-SIGACT symposium on Principles of pro-gramming languages, pages 179–190. ACM Press, 1989.

[90] A. Pnueli and E. Shahar. The TLV system and its applications, 1996.

[91] Stephen Pulman. Controlled language for knowledge representation. In Proceed-ings of the 1st International Workshop on Controlled Language Applications, 1996.

[92] Michael Melholt Quottrup, Thomas Bak, and Roozbeh Izadi-Zamanabadi. Multi-robot planning: a timed automata approach. In IEEE International Conference onRobotics and Automation, pages 4417–4422, New Orleans, LA, 2004. IEEE.

[93] P. J. Ramadge and W. M. Wonham. Supervisory control of a class of discrete eventprocesses. SIAM J. Control Optim., 25(1):206–230, 1987.

[94] P. J. Ramadge and W. M. Wonham. The control of discrete event systems. Proceed-ings of the IEEE, 77(1):81–98, 1989.

[95] S. Ramamoorthy and B. Kuipers. Controller synthesis using qualitative modelsand simulation. In J. de Kleer and K. Forbus, editors, International Workshop onQualitative Reasoning (QR-2004), 2004.

[96] Elon Rimon and Daniel E. Kodischek. Exact robot navigation using artificial po-tential functions. IEEE Transactions on Robotics and Automation, 8(5):501–518,October 1992.

[97] Alfred A. Rizzi. Hybrid control as a method for robot motion programming. InIEEE International Conference on Robotics and Automation, volume 1, pages 832– 837, May 1998.

[98] S. Russell and P. Norvig. Artificial Intelligence, A Modern Approach. Prentice Hall,second edition, 2003.

[99] Yaniv Sa’ar. Synthesis, http://www.wisdom.weizmann.ac.il/ saar/synthesis/ 2007.

[100] Yaniv Sa’ar. Java temporal logic verifier (jtlv), http://jtlv.sourceforge.net 2008.

[101] Uluc Saranli, Martin Buehler, and D. E. Koditschek. Rhex: A simple and highlymobile hexapod robot. International Journal of Robotics Research, 20(7):616 –631, July 2001.

[102] A.J. van der Schaft and J.M. Schumacher. An Introduction to Hybrid Dynami-cal Systems, volume 251 of Lecture Notes in Control and Information Sciences.Springer, London, 2000.

163

[103] Philippe Schnoebelen. The complexity of temporal logic model checking. InPhilippe Balbiani, Nobu-Yuki Suzuki, Frank Wolter, and Michael Zakharyaschev,editors, Selected Papers from the 4th Workshop on Advances in Modal Logics(AiML’02), pages 393–436, Toulouse, France, 2003. King’s College Publication.Invited paper.

[104] M. J. Schoppers. Universal plans for reactive robots in unpredictable environments.In John McDermott, editor, Proceedings of the Tenth International Joint Conferenceon Artificial Intelligence (IJCAI-87), pages 1039–1046, Milan, Italy, 1987. MorganKaufmann publishers Inc.: San Mateo, CA, USA.

[105] Amir Shapiro, Aaron Greenfield, and Howie Choset. Frictional compliance modeldevelopment and experiments for snake robot climbing. In ICRA, pages 574–579.IEEE, 2007.

[106] B. Shults and B. Kuipers. Qualitative simulation and temporal logic: proving prop-erties of continuous systems. Technical Report TR AI96–244, University of Texasat Austin, Dept. of Computer Sciences, 1996.

[107] Paulo Tabuada and George J. Pappas. Model checking LTL over controllable linearsystems is decidable. In Oded Maler and Amir Pnueli, editors, HSCC, volume 2623of Lecture Notes in Computer Science, pages 498–513. Springer, 2003.

[108] Paulo Tabuada and George J. Pappas. Linear time logic control of discrete-timelinear systems. IEEE Transactions on Automatic Control, 51(12):1862–1877, De-cember 2006.

[109] Moshe Y. Vardi. From Church and Prior to PSL. In Proceedings of Workshop on25 Years of Model Checking, August 2006.

[110] Daniel S. Weld. Recent advances in AI planning. AI Magazine, 20(2):93–123,1999.

164