Post on 15-Dec-2015
Approved for Public Release, Distribution Unlimited
Pervasive Self-Regeneration through Concurrent Model-Based
Execution
Brian Williams (PI)Paul Robertson
MITComputer Science and Artificial Intelligence Laboratory
7/20/04 2Approved for Public Release, Distribution Unlimited
Outline
• What we are trying to do.
• Demonstration scenario and test bed.
• Implications of successful results.
• Technical approach.
• What is new.
• Anticipated challenges.
• Technology transition.
• Looking forward: next steps.
7/20/04 3Approved for Public Release, Distribution Unlimited
What we are trying to do
• Why software fails:– Software assumptions about the environment
become invalid because of changes in the environment.
– Software changes introduce incompatibilities.– Software is attacked by a hostile agent.
• What can be done when software fails:– Recognize that a failure has occurred.– Diagnose what has failed – and why.– Find an alternative way of achieving the
intended behavior. Runtime
Models
7/20/04 4Approved for Public Release, Distribution Unlimited
Building upon a proven technology base.
• By extending RMPL to support software failure, we can extend robustness in the face of hardware failures to robustness in the face of software failures.
• Many of the same issues pertain:– Detection of faulty behavior– Diagnosis of the fault given faulty behavior– Reconfiguration of the software to achieve intended
behavior using different software components.– Select among alternatives to maximize utility.
7/20/04 5Approved for Public Release, Distribution Unlimited
Deliverables
• Model-based programming tools– A language for modeling (RMPL):
• The process in terms of desired state evolutions;• The components; and• The environment.
• Model-based executives that provide:– Safe optimal dispatch.– Redundant methods.– Continuous monitoring and diagnosis.– Regeneration and optimization.
7/20/04 6Approved for Public Release, Distribution Unlimited
Architectural overview
SPlant
Obs Cntrl
Model-basedEmbedded Programs
S
ContinuousReactive
Commanding
Continuous Mode/StateEstimation
Model
Desiderata: languages that are•Suspicious
•Monitor intentions and plans
•Self-Adaptive•Exploits and generates contingencies
•State and Fault Aware•Anticipatory
–“Model-predictive languages”–Plans and verifies into the future–Predicts future states
–Plans contingencies
7/20/04 7Approved for Public Release, Distribution Unlimited
Basic assumptions for our approach
• Objectives for the RMPL language– Low overhead
• The effort required of the programmers to achieve robustness should be small compared to the effort required to implement the base capability.
– Pervasive• The technology should be applied not only to major
components but to all components in the system.
– Incremental• Increased robustness can be achieved
incrementally by adding greater modeling.
7/20/04 8Approved for Public Release, Distribution Unlimited
Innovative claims
• Features of our approach:– Fault-aware processes.– Fault-adaptive.– Model-based.– Synthesizes a fault-adaptive process to achieve state evolutions.– Reasons from MODELS of correct and faulty behavior of
supporting service components.– Constructs novel recovery actions in the face of novel faults.
• Specific approaches:– Dynamic selection from redundant methods.– Self-optimization: select the optimal candidates.– Continuous monitoring.– Incremental addition of robustness by adding monitoring
procedures incrementally.– Synthesis of repair procedures from models.
7/20/04 9Approved for Public Release, Distribution Unlimited
Demonstration scenarioEnd to End Self-regeneration of Command & Control Systems:
• Robot must plan and execute motion to one or more targets.• Robot must perform designated tasks at selected destinations.• Robot utilizes various sensors and actuators in achieving its
task.• Robot utilizes various software components in interpreting its
sensor data and manipulating its actuators.• Changing environment will cause software components to fail.
Failed software components will be detected and diagnosed. Alternate configurations of the software will be found that can maintain mission objectives while maximizing utility—in real-time.
• Fault injection testing: We will inject faults into the test bed including (1) environment changes, (2) incompatibilities, (3) software attacks.
7/20/04 10Approved for Public Release, Distribution Unlimited
Test bed summary• Robot scenario involves:
– Path planning and execution.– Goal selection with risks and rewards.– Visual and other sensors and actuators utilized in
navigation and task execution.– Reacts to:
• Failures in the software.– Exploiting redundant methods.– By re-planning to maintain optimality.
• Discoveries in the environment.– Obstacles– Suitability of using selected sensors given terrain lighting etc.
• Attack
7/20/04 11Approved for Public Release, Distribution Unlimited
Rover test bed
Allows real-world testing of planning and execution software
Consists of a reconfigurable environment with one ATRV-2 and three ATRV-JRs.
7/20/04 12Approved for Public Release, Distribution Unlimited
Rover test bed setup
Differential drive
Laser range scannerSonar sensors
Wheel encoders(odometry)
Stereo camera
Inclinometer
GPS receiver
Compass
Antennas for wireless LAN
rFLEXcontroller
rFLEXscreen
Sonarcontrolboard
Sonar sensors
SICK LMS 200 laser scanner
Onboard PC
Firewire card
ttyR ports
Stereo camera
Serial port
Inclinometer
802.11a wireless network adapter
Ethernet card
Left motor
Right motor
• Sensors give information on motion and environment.
• Onboard PC allows for real-time computation and command processing.
7/20/04 13Approved for Public Release, Distribution Unlimited
Implications of successful results
• Robotic systems that can operate autonomously to achieve goals in a complex and changing environment.– Modeling environment
• Software that detects and works around “bugs” resulting from incompatible software changes.– Modeling software components
• Software that detects and recovers from software attacks.– Modeling attack scenarios
• Software that automatically improves as better software components and models are added.
• Higher level of command and control for robotic missions.
7/20/04 14Approved for Public Release, Distribution Unlimited
Military roles for autonomous robots
• Reconnaissance– Rover makes its way to designated places and
reports back—such as with pictures.
• Search and Rescue– Rover enters a dangerous area and reports back on
the presence of wounded people.
• Mapping– Rover explores (a building) producing a map
annotated with pictures in support of ground forces.
• Munitions Delivery– Rover goes to designated locations and delivers
munitions. (like predator UAV)
7/20/04 15Approved for Public Release, Distribution Unlimited
Technical approach
• We will extend the technology developed for execution, hardware fault detection, diagnosis and reconfiguration successfully used in Deep Space One to be applied to software fault awareness and reconfiguration.
• Software components look like hardware components but reconfiguration is less restricted.
• A greater emphasis on environment modeling is necessary because most software faults will be because of environmental changes.
7/20/04 16Approved for Public Release, Distribution Unlimited
Systems Should be Suspicious
Sensor Interpretation:
• Nominal:
• Command Confirmation
• Fault Detection
• Failure:
• Fault Isolation
• Fault Diagnosis
Traditional Commanding is Open Loop
Continuous Estimation of
Modes and State
Model
RealTime
Robust Systems Should be Fully State AwareAnd Should Use To Dynamically Monitor Specifications
7/20/04 17Approved for Public Release, Distribution Unlimited
Commanding InvolvesRepairing a Correct State
For long-lived systems, failure is the rule:
• Must navigate around failures.
• Must operate with varying resources and capabilities.
• Should select best means amongst multiple alternatives.
Nominal commanding is Traditionally pre-determined
Robust systems should select the best action
in context
ContinuousReactive
Commanding
Continuous Mode/StateEstimation
Model
7/20/04 18Approved for Public Release, Distribution Unlimited
How Do We Guide Self-Regenerative Systems? - By Interacting Directly with State
Embedded programs
interact with sensors and actuators.
Model-based programs
interact with state
Embedded Program
SServices
Obs Cntrl
Model-basedEmbedded Program
SServices
Programmers map between sensors, actuators to states.
Model-based executives map between sensors, actuators to states.
7/20/04 19Approved for Public Release, Distribution Unlimited
Model-based Programs Interact Directly with State
SPlant
Obs Cntrl
Model-basedEmbedded Programs
Model-basedExecutive
S
PlantModel
State estimates
ReactiveCommanding
ModeEstimation
State goals
s
RMPL State assertion State query Conditional Execution Preemption Iteration Concurrent execution
Fault OccursCurrent Belief State
S T
X0 X1 XN-1 XN
Reconfigure
S T
X0 X1 XN-1 XN
least cost reachable goal stateFirst Action
7/20/04 20Approved for Public Release, Distribution Unlimited
Real-Time Real-Time ExecutionExecution
Real-Time Real-Time ExecutionExecution
Flight Flight H/WH/W
Flight Flight H/WH/WFault Fault
MonitorsMonitorsFault Fault
MonitorsMonitorsPlanning Experts Planning Experts (incl. Navigation)(incl. Navigation)Planning Experts Planning Experts (incl. Navigation)(incl. Navigation)
Model-based Autonomy Architecture
Ground Ground SystemSystemGround Ground SystemSystem
RAX ManagerRAX Manager
Model-Model-based Faultbased FaultProtectionProtection
Mission Mission ManagerManager ExecutiveExecutive
Planner/Planner/SchedulerScheduler
7/20/04 21Approved for Public Release, Distribution Unlimited
Real-Time Real-Time ExecutionExecution
Real-Time Real-Time ExecutionExecution
Flight Flight H/WH/W
Flight Flight H/WH/WFault Fault
MonitorsMonitorsFault Fault
MonitorsMonitorsPlanning Experts Planning Experts (incl. Navigation)(incl. Navigation)Planning Experts Planning Experts (incl. Navigation)(incl. Navigation)
Ground Ground SystemSystemGround Ground SystemSystem
RAX ManagerRAX Manager
Model-based Model-based FaultFault
ProtectionProtection
Mission Mission ManagerManager
ExecutiveExecutive
Planner/Planner/SchedulerScheduler
Remote AgentRemote AgentGoal States
Model-based Autonomy Architecture
7/20/04 22Approved for Public Release, Distribution Unlimited
Real-Time Real-Time ExecutionExecution
Real-Time Real-Time ExecutionExecution
Flight Flight H/WH/W
Flight Flight H/WH/WFault Fault
MonitorsMonitorsFault Fault
MonitorsMonitorsPlanning Experts Planning Experts (incl. Navigation)(incl. Navigation)Planning Experts Planning Experts (incl. Navigation)(incl. Navigation)
Ground Ground SystemSystemGround Ground SystemSystem
RAX ManagerRAX Manager
Model-basedModel-basedFault Fault
ProtectionProtection
Mission Mission ManagerManager
ExecutiveExecutive
Planner/Planner/SchedulerScheduler
Remote AgentRemote Agent
Model-based Autonomy Architecture
7/20/04 23Approved for Public Release, Distribution Unlimited
Real-Time Real-Time ExecutionExecution
Real-Time Real-Time ExecutionExecution
Flight Flight H/WH/W
Flight Flight H/WH/WFault Fault
MonitorsMonitorsFault Fault
MonitorsMonitorsPlanning Experts Planning Experts (incl. Navigation)(incl. Navigation)Planning Experts Planning Experts (incl. Navigation)(incl. Navigation)
Ground Ground SystemSystemGround Ground SystemSystem
RAX ManagerRAX Manager
Model-Model-based Fault based Fault ProtectionProtection
Mission Mission ManagerManager
Procedural Procedural ExecutiveExecutive
Planner/Planner/SchedulerScheduler
Remote AgentRemote Agent
Low-Level Fault
Protection
Model-based Autonomy Architecture
7/20/04 24Approved for Public Release, Distribution Unlimited
Real-Time Real-Time ExecutionExecution
Real-Time Real-Time ExecutionExecution
Flight Flight H/WH/W
Flight Flight H/WH/WFault Fault
MonitorsMonitorsFault Fault
MonitorsMonitorsPlanning Experts Planning Experts (incl. Navigation)(incl. Navigation)Planning Experts Planning Experts (incl. Navigation)(incl. Navigation)
Ground Ground SystemSystemGround Ground SystemSystem
RAX ManagerRAX Manager
Model-Model-based based
ExecutiveExecutive
Mission Mission ManagerManager
Procedural Procedural ExecutiveExecutive
Planner/Planner/SchedulerScheduler
Remote AgentRemote Agent High-Level Fault
Protection
Model-based Autonomy Architecture
7/20/04 25Approved for Public Release, Distribution Unlimited
Remote Agent ExperimentMay, 1999
May 17-18th experiment: High-level Fault Protection• Generate plan for course correction and thrust • Diagnose camera as stuck on
– Power constraints violated, abort current plan and replan
• Perform optical navigation• Perform ion propulsion thrust
May 21th experiment: Low-level Fault Protection• Diagnose faulty device and
– Repair by issuing reset.• Diagnose switch sensor failure.
– Determine harmless, and continue plan. • Diagnose thruster stuck closed and
– Repair by switching to alternate method of thrusting. • Back to back planning
RA was a toolbox, want a seamless language
7/20/04 26Approved for Public Release, Distribution Unlimited
Technology transition
• The structure of our solution facilitates transition. – Implements self-regenerative system using
language+executive, as opposed to a set of regeneration utilities.
– Easily wraps self-regeneration around existing components, through a clean separation of the “executive” and “plant.”
– Has supported a long history of successful technology transition.
• Papers and other publications
7/20/04 27Approved for Public Release, Distribution Unlimited
Looking forward: next steps
• Reasoning about complex software models.• Coordination while preserving privacy of
subsystem information.• Extending the technology to work with complex
distributed self-regenerative systems.– Regeneration requires peer to peer coordination of
systems.– Subtle faults that are distributed across multiple
systems.– Faults that are detected in systems in which we do
not have direct control—and must negotiate fault resolution.
7/20/04 28Approved for Public Release, Distribution Unlimited
Appendix A
7/20/04 29Approved for Public Release, Distribution Unlimited
Model-based Execution Kernels as Stochastic Optimal Controllers
DeductiveController
Plant
modeestimation
mode reconfiguration
s’(t)
commands(t)
ffs (t)
gg
Observations(t)
ModelGoalStates
7/20/04 30Approved for Public Release, Distribution Unlimited
Operators and programmers reason through system-wide interactions to:
• isolate faultsisolate faults• diagnose causesdiagnose causes
Diagnosis
7/20/04 31Approved for Public Release, Distribution Unlimited
OPSAT
Conflict-directed A*
Conflict-directed A*
OptimalOptimalfeasiblefeasiblemodesmodes
ConflictConflict
CheckedCheckedkernelkernel
ISATISAT
Generate Most-likely Candidate Diagnoses: • conflict-directed A* (< 10 states visited)
Test Against Model and Observables: • Incremental Satisfiability (avg. < 10 % off ideal)• Learn from counter examples (conflicts) to guide generation.
7/20/04 32Approved for Public Release, Distribution Unlimited
Compare Most Likely Candidate to Observations
Helium tank
Fuel tankOxidizer tank
MainEngines
Flow1 = zeroPressure1 = nominal
Pressure2= nominal
Acceleration = zero
7/20/04 33Approved for Public Release, Distribution Unlimited
Isolate Conflicting Modes
Helium tank
Fuel tankOxidizer tank
MainEngines
Flow 1= zero
A conflict, C, is an assignment to a subset of the mode variables that is inconsistent with the model and observations.
7/20/04 34Approved for Public Release, Distribution Unlimited
Helium tank
Fuel tankOxidizer tank
MainEngines
Flow 1= zero
Generate Next Most Likely Candidate
Every consistent mode assignment must differ from the conflict for at least one mode variable
7/20/04 35Approved for Public Release, Distribution Unlimited
Generate Next Most Likely Candidate
Every consistent mode assignment must differ from the conflict for at least one mode variable
Helium tank
Fuel tankOxidizer tank
MainEngines
7/20/04 36Approved for Public Release, Distribution Unlimited
Testing Candidate Detects Another Conflict
Pressure1 = nominal Pressure2= nominal
Acceleration = zero
Helium tank
Fuel tankOxidizer tank
MainEngines
7/20/04 37Approved for Public Release, Distribution Unlimited
Pressure1 = nominal Pressure2= nominal
Acceleration = zero
Helium tank
Fuel tankOxidizer tank
MainEngines
Consistent mode assignment must differ from both conflicts
Generate Next Most Likely Candidate
7/20/04 38Approved for Public Release, Distribution Unlimited
Consistent: Most Likely Diagnosis
Helium tank
Fuel tankOxidizer tank
MainEngines
Pressure1 = nominalFlow1 = zero
Pressure2= nominalFlow2 = positive
Acceleration = zero
7/20/04 39Approved for Public Release, Distribution Unlimited
Operators and programmers reason through system-wide interactions to:
• isolate faultsisolate faults• diagnose causesdiagnose causes
Diagnosis
• repairrepair• reconfigurereconfigure
ConfigurationConfigurationManagementManagement
7/20/04 40Approved for Public Release, Distribution Unlimited
Conflicts Focus MRGoal: Achieve Thrust
Find least cost modes that entail the current goal.• A conflict, C, is an assignment to a subset of the mode variables
that entails the negation of the goal.
7/20/04 41Approved for Public Release, Distribution Unlimited
Goal: Achieve Thrust
Conflicts Focus MR
Find least cost modes that entail the current goal.• A conflict, C, is an assignment to a subset of the mode variables
that entails the negation of the goal.
7/20/04 43Approved for Public Release, Distribution Unlimited
Goal: Achieve Thrust
Conflicts Focus MR
Find least cost modes that entail the current goal.• A conflict, C, is an assignment to a subset of the mode variables
that entails the negation of the goal.
7/20/04 44Approved for Public Release, Distribution Unlimited
Closed
Open Stuckopen
Stuckclosed
OpenCloseCost 5
Prob .9
Models are Concurrent, Constraint Encodings of Partially Observable Markov Decision Process
Vlv = closed => Outflow = 0;
vlv=open => [Outflow]S = [Inflow]S and[Outflow]R = [Inflow]R
vlv=stuck open => [Outflow]S = [Inflow]S and[Outflow]R = [Inflow]R
vlv=stuck closed=> Outflow = 0;
Unknown
• One automaton for each device.• Communication through shared variables.
7/20/04 45Approved for Public Release, Distribution Unlimited
Operators and programmers reason through system-wide interactions to:
• isolate faultsisolate faults• diagnose causesdiagnose causes• monitormonitor• confirm commandsconfirm commands • track goalstrack goals
Estimating Modes
7/20/04 46Approved for Public Release, Distribution Unlimited
Mode Estimation
Left engine on Observe“no thrust”
Enumerated by decreasing probability.
Find most likely reachable states consistent with observations.
Every component transitions at each time step.
7/20/04 47Approved for Public Release, Distribution Unlimited
Operators and programmers reason through system-wide interactions to :
Controlling ModesControlling ModesEstimating Modes
• monitormonitor• track goalstrack goals• confirm commandsconfirm commands• isolat faultsisolat faults• diagnose faults diagnose faults
• repairrepair• reconfigurereconfigure• executeexecute• avoid failuresavoid failures• change control change control
policiespolicies
7/20/04 48Approved for Public Release, Distribution Unlimited
1. Find least cost state that entails the current goal and is reachable from the current state.
2. Find first action that moves towards this state.
Model-based Reactive Planning
ModeReconf.
ModeEst.
Command
Configuration goals
Model
goal statecurrent state
ReactivePlanning
Burton Model-based Executive IJCAI97
7/20/04 49Approved for Public Release, Distribution Unlimited
Architectural overview
SPlant
Obs Cntrl
Model-basedEmbedded Programs
S
ContinuousReactive
Commanding
Continuous Mode/StateEstimation
Model
Desiderata: languages that are•Suspicious
•Monitor intentions and plans
•Self-Adaptive•Exploits and generates contingencies
•State and Fault Aware•Anticipatory
–“Model-predictive languages”–Plans and verifies into the future–Predicts future states
–Plans contingencies
7/20/04 50Approved for Public Release, Distribution Unlimited
Anticipated Challenges
• Software is harder than hardware:– Hardware recovers by leveraging backups,
while software leverages alternative methods. – Models of software are more complex to
specify.– While hardware’s topology is static,
software’s topology dynamically changes.
• Performance– Software systems result in larger state
spaces, making real-time response a challenge.