ONR July 21, 1998

Exploration of Hybrid and Intelligent Control Architectures in Conjunction with Probabilistic Verification

S. Shankar Sastry

July 21, 1998

Electronics Research Laboratory

University of California, Berkeley

ONR UCAV Project Overview

ONR July 21, 1998

Problem: Design of Intelligent Control Architectures for Distributed Multi-Agent Systems

An architecture design problem for a distributed system begins with specified safety and efficiency objectives for each of the system missions (surveillance, reconnaissance, combat, transport) and aims to characterize control, observation and communication.– Mission decomposition among different agents– Task decomposition for each agent– Inter-agent and agent—mother ship coordination– Continuous control and mode switching logic for

each agent– Fault management

This research attempts to develop fundamental techniques, theoretical understanding and software tools for distributed intelligent control architectures with UCAV as an example.

ONR July 21, 1998

Fundamental Issues for Multi-Agent Systems

Central control paradigm breaks down when dealing with distributed multi-agent systems– Complexity of communication, real-time performance– Risk of single point failure

Completely decentralized control – Has the potential to increase safety, reliability and speed of response – But lacks optimality and presents difficulty in mission and task

decomposition

Real-world environments– Complex, spatially extended, dynamic, stochastic and largely unknown

We propose a hierarchical perception and control architecture– Fusion of the central control paradigm with autonomous intelligent

systems– Hierarchical or modular design to manage complexity– Inter-agent and agent–ship coordination to achieve global performance– Robust, adaptive and fault tolerant hybrid control design and

verification– Vision-based control and navigation

ONR July 21, 1998

Autonomous Control of Uninhabited Combat Air Vehicles

UCAV missions– Surveillance, reconnaissance, combat, transport

Problem characteristics– Each UCAV must switch between different modes of

operation

• Take-off, landing, hover, terrain following, target tracking, etc.• Normal and faulted operation

– Individual UCAVs must coordinate with each other and with the mothership

• For safe and efficient execution of system-level tasks: surveillance, combat

• For fault identification and reconfiguration– Autonomous surveillance, navigation and target

tracking requires feedback coupling between hierarchies of observation and control

ONR July 21, 1998

Research Objectives: Design and Evaluation of Intelligent Control Architectures for Multi-agent Systems such as UCAV’s

Research Thrusts Intelligent control architectures for coordinating multi-agent systems

– Decentralization for safety, reliability and speed of response– Centralization for optimality– Minimal coordination design

Verification and design tools for intelligent control architectures– Hybrid system synthesis and verification (deterministic and

probabilistic)

Perception and action hierarchies for vision-based control and navigation– Hierarchical aggregation, wide-area surveillance, low-level

perception

Experimental Testbed Control of multiple coordinated semi-autonomous DV8 helicopters

ONR July 21, 1998

Methods

Formal Methods– Hybrid systems

(continuous and discrete event systems)• Modeling• Verification• Synthesis

– Probabilistic verification

– Vision-based control

Semi-Formal Methods– Architecture design

for distributed autonomous multi-agent systems

– Hybrid simulation– Structural and

parametric learning– Real-time code

generation– Modularity to

manage:• Complexity• Scalability• Expansion

Methods

ONR July 21, 1998

Thrust 1: Intelligent Control Architectures

Coordinated multi-agent system– Missions for the overall system: surveillance, combat,

transportation– Limited centralized control

• Individual agents implement individually optimal (linear, nonlinear, robust, adaptive) controllers and coordinate with others to obtain global information, execute global plan for surveillance/combat, and avoid conflicts

– Mobile communication and coordination systems

• Time-driven for dynamic positioning and stability• Event-driven for maneuverability and agility

Research issues– Intrinsic models– Supervisory control of discrete event systems– Hybrid system formalism

Research Thrust 1: Intelligent Control Architectures

ONR July 21, 1998

Given a strategic objective and local observation– What are the required information protocols with

Human-centered system and other autonomous agents to command tactical control?• Given a distributed control problem and the local observation at each site,

what is the inter-site communication (minimal) or coordination protocols required to solve this problem?

Given a cooperative mission– What is the strategic objective (possibly dynamic)

of each autonomous agent?• How to distribute among the available agents a specified centralized control

problem?

Decentralized Observation, Communication and Control for Multi-Agent Systems

ONR July 21, 1998

Agent Communication Channels

A1 A2

1m

A3

Plant (Lp)

2m 3m

1c 1op 2c 2op 2f 1f 3c 3op 3f

• The agents have partial observation but can exchange messages. • The plant has a set of unobservable distinguished events (failures). • OBJECTIVE: Design the inter-agent communication scheme required to detect and isolate the distinguished events

Decentralized Observation and Communication for Discrete Event Systems

ONR July 21, 1998

Theorem 1 (Lp, poi, fii) is decentrally diagnosable if there exists

n N such that for all f f, ufv Lp, vn, implies

(w Lp) ( i, Ppoi (w) = Ppoi(ufv(f w

If any two sufficiently long plant traces look the same to all the agents, then either they have no failures or have all the same failures.

Synthesis: The communicate all plant observations solution works.

General drawback: Redundant information is communicated.

L() may not be regular even though Lp is regular.

Current focus: Minimal communication, protocol synthesis, trace abstraction

Documentation: Draft paper available and sent to WODES’98

Synthesis of Inter-agent Communication for Decentralized Observation

ONR July 21, 1998

Each agent has a set of controllable events

Controllable events are a subset of the set of observable events

The next event is either an uncontrollable event from the plant, a controllable event enabled by an agent, or a message event scheduled by an agent

A control objective is specified by a language

Investigate the existence and synthesis of coordination protocols

Decentralized Control of Discrete Event SystemsProblem Formulation

ONR July 21, 1998

Advantages:– Will synthesize symbolic, event-driven, inter-agent

communication over a finite message set – Very simple models permitting logical or

combinatorial analysis and insights– AHS example: Worked for most coordinating

maneuvers other than stability properties for vehicle following

Limitation: No formal way to capture continuous dynamics– The semantics of an event is generally some

alignment or safety conditions in velocity, position, and euler angles with respect to targets or other agents

– SOLUTION: Distributed control of hybrid systems systems

Communication and Control Synthesis for DES models

ONR July 21, 1998UCAV Dynamics

Intelligent Control Architecture

Strategic Layer

Mission Control

Tactical Layer

Regulation Layer

Strategic Objective

Inter-UCAV Coordination

Trajectory Constraints

Sensor Info on Targets, UCAV’s

Environmental Sensors

Trajectory

Actuator Commands

Replan

Tracking errors

• Flight Mode Switching• Trajectory Planning

• Trajectory Tracking• Set Point Control

• Mission Planning• Resource Allocation

• Generating Trajectory Constraints• Fault Management

UCAV Control Architecture

ONR July 21, 1998

Preliminary Control Architecture for Coordinating UCAVs

Regulation Layer (fully autonomous)– Control of UCAV actuators in different modes: stabilization

and tracking

Tactical Layer (fully autonomous)– Safe and efficient trajectory generation, mode switching– Strategic Layer (semi-autonomous)– Generating trajectory constraints and influencing the tasks

of other agents using UCAV-UCAV and UCAV-ship coordination for efficient• Navigation, surveillance, conflict avoidance

– Fault management– Weapons configuration

Mission Control Layer (centralized)– Mission planning, resource allocation, mission optimization,

mission emergency response, pilot interface

ONR July 21, 1998

Thrust 2: Verification and Design Tools

The conceptual underpinning for intelligent multi-agent systems is the ability to verify sensory-motor hierarchies perform as expected

Difficulties with existing approaches:– Model checking approaches (algorithms) grow rapidly in

computational complexity– Deductive approaches are ad-hoc

We are developing hybrid control synthesis approaches that solve the problem of verification by deriving pre-verified hybrid system.

– These algorithms are based on game-theory, hence worst-case safety criterion

– We are in the process of relaxing them to probabilistic specifications.

Research Thrust 2: Verification and Design Tools

ONR July 21, 1998

Thrust 2: Verification and Design Tools

Approach– The heart of the approach is not to verify that every run of

the hybrid system satisfies certain safety or liveness parameters, rather to ensure critical properties are satisfied with a certain safety critical probability

Design Mode Verification (switching laws)– To avoid unstable or unsafe states caused by mode

switching (takeoff, hover, land, etc.)

Faulted Mode Verification (detection and handling)– To maintain integrity and safety, and ensure gradual

degraded performance

Probabilistic Verification (worst case vs. the mean behavior)– To soften the verification of hybrid systems by

rapprochement between Markov and Bayesian decision networks

Hybrid Control Synthesis and Verification

ONR July 21, 1998

Controller Synthesis for Hybrid Systems

The key problem in the design of multi-modal or multi-agent hybrid control systems is a synthesis procedure.

Our approach to controller synthesis is in the spirit of controller synthesis for automata as well as continuous robust controller controller synthesis. It is based on the notion of a game theoretic approach synthesis. It is based on the notion of a game theoretic approach to hybrid control design.to hybrid control design.

Synthesis procedure involves solution of Hamilton Jacobi Synthesis procedure involves solution of Hamilton Jacobi equations for computation of safe sets.equations for computation of safe sets.

The systems that we apply the procedure to may be proven to be The systems that we apply the procedure to may be proven to be at best semi-decidable, but approximation procedures apply. at best semi-decidable, but approximation procedures apply.

ONR July 21, 1998

Thrust 3: Perception and Action Hierarchies

Design a perception and action hierarchy centered around the vision sensor to support surveillance, observation, and control functions

Hierarchical vision for planning at different levels of control hierarchy– Strategic or situational 3D scene description, tactical

target recognition, tracking, and assessment, and guiding motor actions

Control around the vision sensor– Visual servoing and tracking, landing on moving

platforms

Research Thrust 3: Perception and Action Hierarchies

ONR July 21, 1998

What Vision Can Do for Control

Global situation scene description and assessment – Estimating the 3D geometry of the scene, object and

target locations, behavior of the objects• Allows looking ahead in planning, anticipation of future events• Provides additional information for multi-agent interaction

Tactical target recognition and tracking– Using model-based recognition to identify targets

and objects, estimating the motion of these objects• Allows greater flexibility and accuracy in tactical missions• Provides the focus of attention in situation planning

ONR July 21, 1998

Relation between Control and Vision

Higher level visual processing: precise, global information, computational intensive

Lower level visual processing: local information, fast, higher ambiguity

Task decomposition for each agent

Inter-agent, agent—mother ship coordination

Higher level

Lower level

The control architecture needs The vision system provides

Situation, 3D scene description

Target recognition

Continuous control Object tracking

Motion detection, and optical flow Guided motor action

ONR July 21, 1998

Key Issues in Vision and Control: Deliver the Right Information at the Right Time

How to coordinate the planning stage with sensing stage– The planner should adjust to the speed and

uncertainty of the vision system– The vision system should optimize its information

flow from the lower level to the higher level, given the need of the planner

How to adjust the focus of attention– Selecting attention of visual processing in terms of

the object locations, as well as level of abstraction – Fine tuning lower-level vision-motor control loop

A well-designed lower-level vision-motor control alleviates computation requirements of higher-level visual processing

ONR July 21, 1998

Approach for Hierarchical Vision Processing

Use grouping to extract a compact description of the scene from lower processing

– Reduces the computation complexity of higher-level reasoning, provides a basis for attention selection

– Information estimated from “big picture” of the scene is less likely to be affected by noise in the sensor

– Efficient computation algorithm which is able to capture the “big picture” of a scene has been developed• General results reported in CVPR’97, results on motion reported in ICCV’98

ONR July 21, 1998

Approach for Hierarchical Vision Processing

Applying higher-level reasoning on the groups extracted– Model-based object recognition

• Matching image groups to object models

– 3D scene geometry estimation • Based on the motion correspondence found

– Tracking and behavior analysis of objects

– Applying Bayesian theory in selecting the right level of visual processing

ONR July 21, 1998

Approach for Lower-level Vision—Motor Control

Vision-guided motor control– Use low-level image, motion flow information in

formulating motor control law• Tracking in the 3D coordinates

– Use optical flow equations to build a model of the scene in 3D space

– Look-ahead control law to allow for visual processing time

• Tracking in the image plane (2D)– Track objects (such as the landing pad) in image

frame– Relate image measurement (such as image location

of the pad, curvature of the lane marker) to motor control law

ONR July 21, 1998

Research Contributions

Fundamental Research Contributions– Design of hybrid control synthesis and verification tools

that can be used for a wide range of real-time embedded systems

– Design of vision and control hierarchies for surveillance and navigation • Hierarchical vision for planning at different levels of control hierarchy• Control around the vision sensor

Our multi-agent control architecture can be used for many applications– ONR applications

• UCAVs, simulated battlefield environment, distributed command and control, automatic target recognition, decision support aids for human-centered systems, intelligent telemedical system

– General engineering applications• Distributed communication systems, distributed power systems, air traffic

management systems, intelligent vehicle highway systems, automotive control

ONR July 21, 1998

Software Tools forSynthesis and Verification

Research Schedule

O N D J F M A M J J A SFY 98 FY 99 FY 00

IntelligentControl

Architectures

Public Tests

Perception

O N D J F M A M J J A S

Smart Aerobots3D Simulation

VerificationTools

O N D J F M A M J

Label RecognitionC++ QNX Real-Time

Visual Situation Assessment

Cal Day DemoApril 17

Terrain FollowingControl Scheme

Vision System forAutonomous Takeoff/Landing

Integrated System for Target Recognition and Terrain Followingfor Multiple UCAVs

Multi-AgentDecentralizedObservation System

Robotic Helicopter CompetitionAug 12-13, Richland, WA

Preliminary UCAVArchitecture

Hybrid ControlSynthesis Methods

Final UCAVArchitecture

ProbabilisticVerification Theory

Performance Evaluation of UCAV Architecture

UCAV ArchitectureDemo

Cal Day Demo

Robotic Helicopter Competition

ONR July 21, 1998

Deliverables

Task Duration Deliverables

Intelligent Control Architectures

Specification Tools 7/97 - 7/98 software, technical reports

Design Tools 7/97 - 9/99 software, technical reports

Architecture Evaluation Environment 7/97 - 7/00 software, technical reports

UCAV Application 7/97 - 7/00 experiments, technical reports

Verification Tools

Design Mode Verification 7/97 -12/98 software, technical reports

Faulted Mode Verification 7/97 - 9/99 software, technical reports

Probabilistic Verification 9/97 - 9/99 technical reports

Perception

Surveillance 7/97 - 9/99 software, experiments

Hierarchical Vision 7/97 - 7/00 software, technical reports

Visual Servoing 9/97 - 7/00 experiments, technical reports

UCAV Application 7/97 - 7/00 experiments, technical reports

ONR July 21, 1998

Measures of Program Success

FY97-98– Design of preliminary UCAV architeture– Design of hybrid control synthesis methods– Design of multi-agent decentralized observation system

FY98-99– Development of probabilistic verification theory– Final UCAV architecture design– Vision and control for terrain following, take-off and landing for

single UCAV

FY99-00– Performance evaluation of UCAV architecture– Integration of vision and control for multiple coordinating

UCAVs– Final version of the software tools for

• Hybrid control synthesis and verification, and • Decentralized observation and control

– Demonstration of UCAV architecture using the helicopter testbed

ONR July 21, 1998

FY97-98 Accomplishments

• Controller synthesis for hybrid systems. Developed algorithms and computational procedures

fordesigning verified hybrid controllers optimizing

multipleobjectives

• Multi-agent decentralized observation problem. Designed inter-agent communication scheme to

detect and isolate distinguished events in system dynamics

• SmartAerobots. 3D virtual environment simulation. Visualization tool for control schemes and vision

algorithms—built on top of a simulation based on mathematical

models of helicopter dynamics

• Label recognition: prototype in Matlab, then in C++ (QNX real-time)

ONR July 21, 1998

Berkeley Team

Name Role Tel E-mail

Shankar Sastry Principal (510) 642-7200 sastry@robotics.eecs.berkeley.edu

Investigator (510) 642-1857

(510) 643-2584

Jitendra Malik Co-Principal (510) 642-7597 malik@cs.berkeley.edu

Investigator

Datta Godbole Research (510) 643-5806godbole@robotics.eecs.berkeley.edu

Engineer (510) 231-9582

John LygerosPostdoc (510) 643-5795 lygeros@robotics.eecs.berkeley.edu

Jianbao Shi Postdoc (510) 642-9940 jshi@cs.berkeley.edu

Omid Shakernia Graduate Student (510) 643-2383 omids@robotics.eecs.berkeley.edu

ONR July 21, 1998

Teaming and Interdependency

Collaboration with Prof. Varaiya (Berekeley) in designing a hierarchical control architecture for coordinating UCAVs

Collaboration with Prof. Russell (Berkeley) in developing probabilistic design and analysis tools

Collaboration with Prof. Zadeh (Berkeley) on soft computing tools for control of UCAVs and mode transition methods for DV 8 developed using fuzzy control

Collaboration with Prof. Speyer (UCLA) on fault detection and handling methods

Collaboration with Prof. Morse (Yale) on vision-guided navigation Informal conversations with Prof. Anderson (ANU), Prof. Hyland

(Michigan) and visit to Naval Post Graduate School Pending: more formal collaborations with Profs. Narendra, Morse