Autonomous Mobile RobotsCPE 470/670

Lecture 11

Instructor: Monica Nicolescu

CPE 470/670 - Lecture 11 2

Review

• Expression of behaviors

– Stimulus Response

– Finite State Acceptor

– Situated Automata

• Behavioral encoding

– Discrete: rule-based systems

– Continuous: potential fields, motor schemas

CPE 470/670 - Lecture 11 3

Examples of Schemas

• Obstacle avoid and stay on corridor schemas

Rd

SdRRS

dSSd

Vmagnitude

0

2/2/

2/

WdGW

dWdP

Vmagnitude

CPE 470/670 - Lecture 11 4

The Role of Gains in Schemas

• Low gain • High gain

CPE 470/670 - Lecture 11 5

Schema-Based Robots

• At Georgia Tech (Ron Arkin)• Exploration

• Hall following

• Wall following

• Impatient waiting

• Navigation

• Docking

• Escape

• Forage

CPE 470/670 - Lecture 11 6

Behavior Coordination

• Behavior-based systems require consistent

coordination between the component behaviors for

conflict resolution

• Coordination of behaviors can be:

– Competitive: one behavior’s output is selected from

multiple candidates

– Cooperative: blend the output of multiple behaviors

– Combination of the above two

CPE 470/670 - Lecture 11 7

Competitive Coordination

• Arbitration: winner-take-all strategy only one response chosen

• Behavioral prioritization

– Subsumption Architecture

• Action selection/activation spreading (Pattie Maes)

– Behaviors actively compete with each other

– Each behavior has an activation level driven by the robot’s

goals and sensory information

• Voting strategies (DAMN architecture, Rosenblatt)

– Behaviors cast votes on potential responses

CPE 470/670 - Lecture 11 8

Cooperative Coordination

• Fusion: concurrently use the output of multiple behaviors

• Major difficulty in finding a uniform command

representation amenable to fusion

• Fuzzy methods

• Formal methods

– Potential fields

– Motor schemas

– Dynamical systems

CPE 470/670 - Lecture 11 9

Emergent Behavior

The resulting robot behavior may sometimes be

surprising or unexpected

emergent behavior

CPE 470/670 - Lecture 11 10

Wall Following

• A simple wall following controller:

– If too close on left-back, turn left

– If too close on left-front, turn right

– Similarly for right

– Otherwise, keep straight

• If the robot is placed close to a wall it will follow

• Is this emergent?

– The robot has no explicit representations of walls

– The controller does not specify anything explicit about

following

CPE 470/670 - Lecture 11 11

Emergence

• A “holistic” property, where the behavior of the robot

is greater than the sum of its parts

• A property of a collection of interacting components

• Often occurs in reactive and behavior-based

systems (BBS)

• Typically exploited in reactive and BBS design

CPE 470/670 - Lecture 11 12

Flocking

• How would you design a flocking behavior for a

group of robots?

• Each robot can be programmed with the same

behaviors:

– Don’t get too close to other robots

– Don’t get too far from other robots

– Keep moving if you can

• When run in parallel these rules will result in the

group of robots flocking

CPE 470/670 - Lecture 11 13

Emergent Behavior

• Emergent behavior is structured behavior that

is apparent at one level of the system (the

observer’s point of view) and not apparent at another (the controller’s point of view)

• The robot generates interesting and useful behavior

without explicitly being programmed to do so!!

• E.g.: Wall following can emerge from the interaction

of the avoidance rules and the structure of the

environment

CPE 470/670 - Lecture 11 14

Components of Emergence

• The notion of emergence depends on two

components

– The existence of an external observer, to observe the

emergent behavior and describe it

– Access to the internals of the controller, to verify that the

behavior is not explicitly specified in the system

• The combination of the two is, by many researchers,

the definition of emergent behavior

CPE 470/670 - Lecture 11 15

Unexpected & Emergent Behavior

• Some argue that the description above is not

emergent behavior and that it is only a particular

style of robot programming

– Use of the environment and side-effects leads to the novel

behavior

• Their view is that emergent behavior must be

truly unexpected, and must come to a surprise to

the external observer

CPE 470/670 - Lecture 11 16

Expectation and Emergence

• The problem with unexpected surprise as property

of behavior is that:

– it entirely depends on the expectations of the

observer which are completely subjective

– it depends on the observer’s knowledge of the

system (informed vs. naïve observer)

– once observed, the behavior is no longer

unexpected

CPE 470/670 - Lecture 11 17

Emergent Behavior and Execution

• Emergent behavior cannot always be designed in advance and

is indeed unexpected

• This happens as the system runs, and only at run-time can

emergent behavior manifest itself

• The exact behavior of the system cannot be predicted

– The real world is filled with uncertainty and dynamic properties

– Perception is affected by noise

– Would have to consider all possible sequences and combinations

of actions in all possible environments

• If we could sense the world perfectly, accurate predictions could

be made and emergence would not exist!

CPE 470/670 - Lecture 11 18

Desirable/Undesirable Emergent Behavior

• New, unexpected behaviors will always occur in any

complex systems interacting with the real world

• Not all behaviors (patterns, or structures) that

emerge from the system's dynamics are desirable!

• Example: a robot with simple obstacle avoidance

rules can oscillate and get stuck in a corner

• This is also emergent behavior, but regarded as a

bug rather than a feature

CPE 470/670 - Lecture 11 19

Sequential and Parallel Execution

• Emergent behavior can arise from interactions of the

robot and the environment over time and/or over space

• Time-extended execution of behaviors and interaction with the environment (wall following)

• Parallel execution of multiple behaviors (flocking)

• Given the necessary structure in the environment and

enough space and time, numerous emergent

behaviors can arise

CPE 470/670 - Lecture 11 20

Architectures and Emergence

• Different architectures have different methods for dealing with emergent behaviors: modularity directly affects emergence

• Reactive systems and behavior-based systems exploit emergent behavior by design– Use parallel rules and behaviors which interact with each

other and the environment

• Deliberative systems and hybrid systems aim to minimize emergence– Sequential, no interactions between components, attempt

to produce a uniform output of the system

CPE 470/670 - Lecture 11 21

Deliberative Systems

• Deliberative control refers to systems that take a lot

of thinking to decide what actions to perform

• Deliberative control grew out of the field of AI

• AI, deliberative systems were used in non-physical

domains, such as playing chess

• This type of reasoning was considered similar to

human intelligence, and thus deliberative control

was applied to robotics as well

CPE 470/670 - Lecture 11 22

Shakey (1960)

• Early AI-based robots used computer vision

techniques to process visual information from

cameras

• Interpreting the structure of the environment

from visual input involved complex processing

and required a lot of deliberation

• Shakey used state-of-the-art computer vision

techniques to provide input to a planner and

decide what to do next (how to move)

CPE 470/670 - Lecture 11 23

Planning

Planning:

– Looking ahead at the outcomes of possible actions, searching for a sequence that would reach the goal

• The world is represented as a set of states

• A path is searched that takes the robot from the

current state to the goal state

• Searching can go from the goal backwards, or

from the current state to the goal, or both ways

• To select an optimal path we have to consider all

possible paths and choose the best one

CPE 470/670 - Lecture 11 24

SPA Architectures

• Deliberative, planner-based architectures involve

the sequential execution of three functional steps:

– Sensing (S)

– Planning (P)

– Acting (A), executing the plan

• SPA has serious drawbacks for robotics

CPE 470/670 - Lecture 11 25

Drawback 1: Time-Scale

• It takes a very long time to search in large state spaces

• The combined inputs from a robot’s sensors:

– Digital sensors: switches, IRs

– Complex sensors: cameras, sonars, lasers

– Analog sensors: encoders, gauges

+ representations constitutes a large state space

• Potential solutions:

– Plan as rarely as possible

– Use hierarchies of states

CPE 470/670 - Lecture 11 26

Drawback 2: Space

• It may take a large amount of memory to represent and manipulate the robot’s state space representation

• The representation must be as complete as possible

to ensure a correct plan:

– Distances, angles, landmarks, etc.

– How do you know when to stop collecting information?

• Generating a plan that uses this amount of

information requires additional memory

• Space is a lesser problem than time

CPE 470/670 - Lecture 11 27

Drawback 3: Information

• The planner assumes that the representation

of the state space is accurate and up-to-date

• The representation must be updated and checked

continuously

• The more information, the better

• Updating the world model also requires time

CPE 470/670 - Lecture 11 28

Drawback 4: Use of Plans

Any plan is useful only if:

• The representation on which the plan was based is

accurate

• The environment does not change during the

execution of the plan in a way that affects the plan

• The robot’s effectors are accurate enough to

perfectly execute the plan, in order to make the next

step possible

CPE 470/670 - Lecture 11 29

Departure from SPA

• Alternatives were proposed in the early 1980 as a

reaction to these drawbacks: reactive, hybrid,

behavior-based control

• What happened to purely deliberative systems?

– No longer used for physical mobile robots, because the

combination of real-world sensors, effectors and time-

scales makes them impractical

– Still used effectively for problems where the environment is

static, there is plenty of time to plan and the plan remains

accurate: robot surgery, chess

CPE 470/670 - Lecture 11 30

SPA in Robotics

• SPA has not been completely abandoned in

robotics, but it was expanded

• The following improvements can be made:

– Search/planning is slow

saved/cache important and/or urgent decisions

– Open-loop execution is bad

use closed-loop feedback and be ready to re-plan when

the plan fails

CPE 470/670 - Lecture 11 31

Summary of Deliberative Control

• Decompose control into functional modules: sense-

world, generate-plan, translate-plan-to-actions

• Modules are executed sequentially

• Require extensive and slow reasoning computation

• Encourage open-loop execution of generated plans

CPE 470/670 - Lecture 11 32

Hybrid Control

• Idea: get the best of both worlds

• Combine the speed of reactive control and the brains of deliberative control

• Fundamentally different controllers must be made to

work together

– Time scales: short (reactive), long (deliberative)

– Representations: none (reactive), elaborate world models

(deliberative)

• This combination is what makes these systems

hybrid

CPE 470/670 - Lecture 11 33

Biological Evidence

• Psychological experiments indicate the existence of two

modes of behavior: willed and automatic

• Norman and Shallice (1986) have designed a system

consisting of two such modules:

– Automatic behavior: action execution without awareness or

attention, multiple independent parallel activity threads

– Willed behavior: an interface between deliberate conscious

control and the automatic system

• Willed behavior:

– Planning or decision making, troubleshooting, novel or poorly

learned actions, dangerous/difficult actions, overcoming habit or

temptation

CPE 470/670 - Lecture 11 34

Hybrid System Components

• Typically, a hybrid system is organized in three

layers:

– A reactive layer

– A planner

– A layer that puts the two together

• They are also called three-layer architectures or

three-layer systems

CPE 470/670 - Lecture 11 35

The Middle Layer

The middle layer has a difficult job:

• compensate for the limitations of both the planner and

the reactive system

• reconcile their different time-scales

• deal with their different representations

• reconcile any contradictory commands between the

two

• The main challenge of hybrid systems is to achieve

the right compromise between the two layers

CPE 470/670 - Lecture 11 36

An Example

• A robot that has to deliver medication to a patient in a

hospital

• Requirements:

– Reactive: avoid unexpected obstacles, people, objects

– Deliberative: use a map and plan short paths to

destination

• What happens if:

– The robot needs to deliver medication to a patient, but does

not have a plan to his room?

– The shortest path to its destination becomes blocked?

– The patient was moved to another room?

– The robot always goes to the same room?

CPE 470/670 - Lecture 11 37

Bottom-up Communication

Dynamic Re-Planning

• If the reactive layer cannot do its job

It can inform the deliberative layer

• The information about the world is updated

• The deliberative layer will generate a new plan

• The deliberative layer cannot continuously generate

new plans and update world information

the input from the reactive layer is a good

indication of when to perform such an update

CPE 470/670 - Lecture 11 38

Top-Down Communication

• The deliberative layer provides information to the

reactive layer

– Path to the goal

– Directions to follow, turns to take

• The deliberative layer may interrupt the reactive

layer if better plans have been discovered

• Partial plans can also be used when there is no time

to wait for the complete solution

– Go roughly in the correct direction, plan for the details

when getting close to destination

CPE 470/670 - Lecture 11 39

Reusing Plans

• Frequently planned decisions could be reused to

avoid re-planning

• These can be stored in an intermediate layer and

can be looked up when needed

• Useful when fast reaction is needed

• These mini-plans can be stored as contingency

tables

– intermediate-level actions

– macro operators: plans compiled into more general

operators for future use

CPE 470/670 - Lecture 11 40

Universal Plans

• Assume that we could pre-plan in advance for all

possible situations that might come up

• Thus, we could generate and store all possible

plans ahead of time

• For each situation a robot will have a pre-existing

optimal plan, and will react optimally

• It has a universal plan:

– A set of all possible plans for all initial states and all goals

within the robot’s state space

• The system is a reactive controller!!

CPE 470/670 - Lecture 11 41

Domain Knowledge

• A key advantage of pre-compiled systems

– domain knowledge (i.e., information that the designer has

about the environment, the robot, and the task), can be

embedded into the system in a principled way

• The system is compiled into a reactive controller

the knowledge does not have to be reasoned about

(or planned with) explicitly, in real-time

CPE 470/670 - Lecture 11 42

Applicability of Universal Plans

• Examples have been developed as situated automata

• Universal plans are not useful for the majority of real-world domains because:– The state space is too large for most realistic problems

– The world must not change

– The goals must not change

• Disadvantages of pre-compiled systems– Are not flexible in the presence of changing environments,

tasks or goals

– It is prohibitively large to enumerate the state space of a real robot, and thus pre-compiling generally does not scale up to complex systems

CPE 470/670 - Lecture 11 43

Reaction – Deliberation Coordination

• Selection:

Planning is viewed as configuration

• Advising:

Planning is viewed as advice giving

• Adaptation:

Planning is viewed as adaptation

• Postponing:

Planning is viewed as a least commitment process

CPE 470/670 - Lecture 11 44

Readings

• M. Matarić: Chapters 17, 18