Reactive Paradigm – Overview

Subsumption Architecture

By Ian Jonkers

159.734 Studies in Machine Learning: Intelligent Robotics

Reactive Paradigm Overview

• Two Representative Reactive Architectures:1) Subsumption2) Potential Fields Summation

• Reactive Paradigm emerged late 1980’s

• Reactive Paradigm still important for two reasons:1) Robotic systems in limited task domain still being constructed2) Forms the basis for the Hybrid Reactive-Deliberative

Paradigm

• Reactive Paradigm grew out of the dissatisfaction of the hierarchical paradigm and with the influx of ideas from biological intelligence.

Horizontal Decomposition of Hierarchical Model

SENSE PLAN ACT

Sensors ExtractFeatures

CombineFeatures

into Model

PlanTasks

TaskExecution

MotorControl

Actuators

Vertical Decomposition

• Instead, examination of ethological literature suggests that intelligence is layered in a vertical decomposition:

1. Agent starts with primitive survival behaviours.

2. Evolve new layers of behaviour which either:

a) Reuse the lower, older behaviour

b) Inhibit older behaviour

c) Create parallel tracts of more advanced behaviours

3. Parallel tracks can be thought of as vertically stacked layers

4. Each layer has access to sensors and actuators independently of any other layer.

5. If anything happens to the more advanced layer, lower levels should still operate. i.e. human brain (breathing) continue independently of higher order functions (counting, face recognition).

Vertical Decomposition

SENSE ACT

SENSE ACT

SENSE ACT

SENSE ACT

Sensors Actuators

Avoid Collisions

wander

explore

Build maps

Vertical decomposition of tasks into a S-A Organisation, associated with the Reactive Paradigm

Attributes of Reactive Paradigm• The fundamental attribute of reactive paradigm is all actions are

accomplished through behaviours

• As in ethological systems, behaviours are a direct mapping of sensory inputs to a pattern of motor actions that are then used to achieve a task.

Behaviour

SENSE ACTBehaviour

SENSE ACTBehaviour

SENSE ACTBehaviour

SENSE ACT

S-A organisation of the Reactive Paradigm into multiple, concurrent behaviours

Attributes of Reactive Paradigm (continued)

• From mathematical perspective, behaviours are simply a transfer function.

• The Reactive Paradigm essential threw away the PLAN component.

• The SENSE & ACT are tightly coupled into behaviours & sequential or concurrent robotic activities emerge.

• Sensing is local to each behaviour, but sensors may be shared and is immediately available to the behaviour's perceptual schema which can be computationally inexpensive.

Characteristics of Reactive Behaviours

• Reactive robotic systems execute rapidly (tight coupling of senses permits real-time operation)

• Behaviours can be implemented directly in hardware circuits or low computational complexity algorithms (O(n)).

• Have no memory (limiting behaviours to stimulus-response reflexes)

• Main point “Behaviours controlled by what is happening in the world, duplicating the spirit of the innate releasing mechanisms, rather than the program storing & remembering what the robot last did”

• Five Characteristics of reactive Paradigm are:1) Situated Agent (integrated part of world)

Robots are situated agents operating in an ecological niche.i.e. when a robot acts, it changes the world, and receives immediate feedback about the world through sensing.

2) Emergent BehavioursBehaviours serve as the basic building blocks for robotic action, and the overall behaviour of the robot is emergent.Behaviours are independent computational entities and operate concurrently, Hence there is no explicit “controller” module which determines what will be done, or which function call other functions.

3) Eco-centricOnly local, behaviour specific sensing is permitted.i.e. does not matter that an obstacle is in the world at coordinates (x,y,z), only where it is relative to the robot.

4) Modular BehavioursThese systems inherently follow good software design principles.The modularity of behaviours supports the decomposition of a task into component behaviours.

5) Biological MotivationAnimal models of behaviour are often cited as a basis for these systems or a particular behaviour (unlike earlier AI days where a conscious effort not to mimic biological intelligence was made)

Characteristics of Reactive Behaviours (continued)

Advantages of Programming by Behaviour

• Constructing a robotic system under Reactive Paradigm often referred to as programming by behaviour.

• Good Software Engineering since Behaviours are Modular.

• Robot becomes more intelligent by having more behaviours.

• Modules have Low Coupling can function independently of each other with minimal connections or interfaces promoting easy reuse.

• Modules have High Cohesion Data and operations contained by a module relate only to the purpose of that module.

Reactive Paradigm Representative Architectures

• The overall action of the robot emerges from the multiple concurrent behaviours and the architecture must provide mechanisms for:1) Triggering behaviours2) Conflict resolution when multiple behaviours are active at any one time.

• The two most well known Reactive Architectures are1) Potential Fields – Behaviours combined by summation of fields.2) Subsumption – Decomposition into layers of task achieving behaviours.

Subsumption Architecture

• Rodney Brooks Subsumption Architecture most influential of the purely Reactive Paradigms.

• Many look like shoe-box sized insects (6 legs and antennae)• Implementations quite often have the behaviours embedded directly

in the hardware or small micro processors (unheard pre mid 1980’s)• Robots could now walk, avoid collisions and climb over obstacles

without the move-think-move-think pauses of Shakey.

• A behaviour is a network of sensing and acting modules which accomplish a task.

• The modules are Augmented Finite State Machines (AFSM), or finite state machines which have registers, timers & other enhancements to permit them to be interfaced with other modules.

• Behaviours are released in a stimulus response way, without an external program explicitly coordinating and controlling them.

• Layers of Competence:The layers reflect a hierarchy of intelligence or competence. lower layers encapsulate basic survival functions (collisions) higher levels create more goal directed actions (mapping)Each of the layers can be viewed as an abstract behaviour for a particular task.

• Subsumption of lower layers:- Modules in a higher layer can override or subsume the output from behaviours in the next lower layers.- Behaviour layers operate concurrently and independently and hence need mechanism to handle potential conflicts winner always takes all (always the highest layer)

Subsumption Architecture (continued)

• No World Model (Internal State)

- No persistence representation of the world model since information comes directly from the world.

- Dangerous to depend on internal state since may diverge from reality.

- Some internal state needed for releasing behaviours (i.e. scared, hungry), but good design minimises this.

• Taskable

- Tasks are accomplished by activating the appropriate layer, which then activate the lower layers.

- In practice, Subsumption style systems are not easily taskable, that is, they can’t be ordered to do another task without being reprogrammed.

Subsumption Architecture (continued)

Obstacle Avoidance Example

• Sensor: A SONAR module that gives the distance to the objects in polar coordinates.

• Internal Modules– COLLIDE detects if front obstacle is too close (i.e. halt)– FEELFORCE sensor reading acts as repulsive force field– RUNAWAY provides direction to move

• Actuators– TURN provides motor output to turn robot– FORWARD switches forward motion on or off

Sonar Module

“robo-centric” view of range readings

0

1

2

3

4

7

6

50 1 3 4 5 6 72

Unrolled into a plot

Sonar module reads the sonar ranges. Polar plot range readings in polar coordinates (r,) surrounding

the robot.

Level 0 Architecture: obstacle avoidance

Level 1 Architecture: wander

How does Turn know which module to take heading from?

Wander - Can’t pass directly onto Turn since will sacrifice obstacle avoidance

Suppression – replaces all other inputs to the module with input coming from the “suppressing” module.

Inhibition – blocks output from the specified module for the defined time interval.

Level 2 Architecture: follow corridorIntegrate - estimates how far robot has travelled off course Supplies dangerous internal state

Evaluating Subsumption Architecture• Modularity:

behaviours are modular, but generally favour a hardware implementation

• Niche targetability: high due to direct perception.

• Portability: limited to tasks which can be accomplished with reflexive behaviours generally can’t be transferred where planning is needed

• Robustness: offers graceful degradation if anything should disable a higher level behaviour, then the lower level can be left intact.

Summary

• Layers of abstract behaviours: achieved by grouping schemas like modules into layers.

• Suppression and inhibition of lower layers: Higher layers may subsume & inhibit behaviours in lower layers, but the behaviours in lower layers never rewritten or replaced. Mimics biological evolution (i.e. frogs with large objects)

• Difficult to design: More of a art form than a science.

• Behaviour Release Mechanisms Behaviours are released by the presence of stimulus in the environment.

Summary (continued)

• Solution to the Frame Problem:

Solves the frame problem by eliminating the need to model the world.

Behaviours don’t remember the past.

• Direct Perception and Affordance

The release for a behaviour is almost always the percept for guiding the motor schema.

• Perception is Eco-centric & Distributed

i.e. sensing objects relative to the robot and the sensors can be shared among modules.

References

• Robin R. Murphy - Introduction to AI robotics - Cambridge, Mass. : MIT Press, 2000

• Jonathan SIMPSON, Christian L. JACOBSEN and Matthew C. JADUD - Mobile Robot Control - The Subsumption Architecture and occam-pi - Communicating Process Architectures 2006

http://www.jonsimpson.co.uk/weblog/2006-09-24/mobile-robot-control-the-subsumption-architecture-and-occam-pi.html