Lifecycle Verification of the NASA Ames K9 Rover

ExecutiveDimitra GiannakopoulouMike LowryCorina Păsăreanu (pcorina@email.arc.nasa.gov)Rich Washington

Collaborators: Howard Cannon, Ray Garcia – ARA GroupSummer Students: Colin Blundell (University of Pennsylvania), Jamie Cobleigh (University of Massachusetts)

ObjectivesMotivation / NASA Relevance:

Verification is essential for autonomy insertion into missionsTraditional testing of autonomy software is hard due to high complexity and unpredictable environments Integration problems are difficult to detect and expensive to fix; verification should be addressed early in the software lifecycle

Objectives:Use a combination of formal analysis techniques (model checking) and testing to analyze autonomous systems throughout their lifecycleProvide support for compositional (“divide and conquer”) verification to address scalability Use design level artifacts to guide the implementation and to enable efficient source code verification

Requirements &Design Coding Deployment

Compositional Verification

Model checking/Testing

Properties,assumptions

cost of detecting / fixing bugs increases

integration issues handled early

Accomplishments/Contributions

Development of novel, fully automated compositional verification techniquesIntegrated lifecycle approach to verification of autonomy software

Analysis of K9 Rover Executive (35KLOC of C++)Design level modeling– Design models: describe concurrent architecture and autonomy features – Requirements: concurrency and plan execution properties

Design level analysis– Model checking used for exhaustive verification of design models– 10x improvement over monolithic (non-compositional) verification

Code level analysis– Model checking: 3x improvement over monolithic verification– Advanced testing (automated plan input generation and run time verification)

Several integration problems discovered Change in design as a result of our analysis– Simplified architecture with increased modularity

Outline

Introduction

The K9 Rover Executive

Compositional Techniques for Design- and Code- Level Verification

Analysis of the Rover Executive

Conclusions and Future Work

Ames K9 Rover Executive

Executes flexible plans for autonomy – branching on state / temporal conditions

Multi-threaded system– communication through shared variables

– synchronization through mutexes and condition variables

Architecture changed as a result of our analysis– Simplified inter-thread communication through event queue

Action Execution

Executive

Database

Plan Watcher

ExecCondChecker

Exec Timer

Internal

DBMonitor

Exec Timer

Executive

Action Execution

Database

Internal

DBMonitor

Plan Watcher

Event Queue

Alternate plan library

ExecCondChecker

Original architecture Updated architecture

Outline

Introduction

The K9 Rover Executive

Compositional Techniques for Design- and Code- Level Verification

Analysis of the Rover Executive

Conclusions and Future Work

Model Checking for Design Level Analysis

OKFinite-state model

Specification

Verification tool

or

Error trace

Line 5: …Line 12: …Line 15:…Line 21:…Line 25:…Line 27:… …Line 41:…Line 47:…

Check P on entire system: too many states Check one component at a time: need abstraction of other modulesM2 M1

Does system made up of M1 and M2 satisfy property P?

Our work: the first incremental and automated approach for assume-guarantee reasoning

Solution 1

(2002) algorithmic generation of assumption as a controller

knowledge of environment is not required

applied to Rover Executive (ASE’02)

(2003) extended to deal with deadlock

applied to DS-1 remote agent executive (Journal of ASE, 2003)

Solution 2 (2003)

incremental assumption computation based on learning and knowledge of environment

applied to Rover Executive (TACAS’03, Journal on STTT, submitted October 2003)

extended framework for symmetric assume-guarantee rules (SAVCBS’03)

Compositional Verification for Increased Scalability

finite state modelssafety properties

P holds in system

A Assume-guarantee reasoning: 1. check P on M1 with assumption A for M2

2. check that M2 satisfies A

Assumption generation is a manual process

Learning Framework for Automated Compositional Verification

P violated

Process is iterative

We use a learning algorithm to infer the smallest assumption at each stage

Assumptions are generated by querying the system, and are gradually refined

Queries are answered by model checking

Refinement is based on counterexamples obtained by model checking

Termination is guaranteed

Algorithm is “any-time”

Learning

Model Checking

Ai

real error?counterexample – refine Ai

counterexample – refine Ai

M2 satisfies Ai

M1 satisfies P, under Ai

P holdstrue

false

true

false

N Y

TECHNOLOGY

Extended LTSA verification tool to support automated assume guarantee reasoning at design level

PROBLEM Suppose we have a (finite) design model for a piece of

software and safety properties Can use our automated assume-guarantee reasoning on

the design model to verify the safety properties Can we use this knowledge to improve the performance of

verifying the properties on the actual implementation?

TECHNOLOGY Developed methodology for using design-level

assumptions for reasoning about source code, to improve scalability

For source code verification, we used Model checking

Extended Java PathFinder model checker to support assume-guarantee reasoning at the source code level; applied approach to K9 Rover (ICSE’04)

Advanced testingAutomated plan input generation based on plan

language grammarProperty monitoring (Eagle)

Design models

Component implementations

M1 M2A

C1 C2AP

P

SOLUTION Use assumption A generated at the design level to

check the implementation C1 || C2 in an assume-guarantee style

Assume-guarantee Verification of Code with Design-Level Assumptions

Testing Framework

Input plan &

property generation

event stream

K9 executive

instrumentation

Observer

(Eagle)

Input

plans

Behavioural

properties

reports

instrumentation

Exec Timer

Executive

Action Execution

Database

Internal

DBMonitor

Plan Watcher

Event Queue

Alternate plan library

ExecCondChecker

Outline

Introduction

The K9 Rover Executive

Compositional Techniques for Design- and Code- Level Verification

Analysis of the Rover Executive

Conclusions and Future Work

Analysis of the ExecutiveProperties

Concurrency: deadlocks, correct lock usage / data races, sequence of interactions between modules

Plan execution properties

Analyzed several versions

Original architecture: components (threads), locks, condition variables

New architecture: queuing and event handling, floating branch execution (execution of synchronous and asynchronous alternate plans)

Design level analysis

Several synchronization issues discovered (also present in actual implementation)

10x improvement over monolithic (non-compositional) verification

Code level analysis

Model checking: 3x improvement over monolithic verification

Testing– Automatic generation of hundreds of plans in a few seconds– Detection of integration problems early (during unit testing)– Better coverage (predict, by mathematical inference, properties about all

possible inter-leavings)

Analyzed PropertiesP1: If the last task in the plan terminates successfully, then the only possible outcome for the plan is successful termination.

P2: When a task fails, the continue-on-failure flag on the block will always be checked before any outcome is produced; moreover, if continue-on-failure is true, the outcome is success, otherwise it is failed.

P3: The Executive only receives ExecCondChecker events if it has registered for them.

P4: The ExecCondChecker only puts events in the queue if the Executive registered for them.

P5: When a task fails, it will always check its continue-on-failure flag; moreover, if the continue-on-failure flag is false, no subsequent task in the block will be started; new tasks can be started after the parent block reports the results (i.e. other block is expanded).

P6: If a task fails, then the parent block’s continue-on-failure flag will be checked: if it is true, then the block succeeds, otherwise it fails.

P7: If the Executive thread reads the value of the shared variable savedWakeupStruct, then the ExexcCondChecker thread should not read it until the Executive clears it first.

P8: (Race condition) All accesses to shared structure conditionSetChanged by the Executive and the ExecCondChecker threads will be protected by locks.

P9: (Race condition) All accesses to shared structure existConditions by the Executive and the ExecCondChecker threads will be protected by locks.

P10: Absence of local and global deadlocks.

P11: No irrelevant action execution events can happen.

P12: No irrelevant condition checker events can happen.

P13: Alternate and principal plans

Example property (P7)ExecT

imerC

heckerD

atabase(state of system

)A

ctionExecution

DB

Mon

itor

Intern

al

Execu

tive

ExecC

ond

Ch

eck

er

savedWakeupStruct

If the Executive thread reads the value of variable savedWakeupStruct, the ExecCondChecker thread should not read this value unless the Executive clears it first.

plan

Example Property (P7)

0

error

executive.savedWakeupStruct.read[0..1]

1execCondCh.savedWakeupStruct.read[0..1]

executive.savedWakeupStruct.assign[0]execCondCh.savedWakeupStruct.assign[0..1]

Generated Assumption

Assumption = Q0,

Q0 = ( executive.exec.lock -> Q2),

Q2 = (executive.exec.unlock -> Q0

| executive.savedWakeupStruct.read[1] -> Q3

| executive.savedWakeupStruct.assign[0] -> Q4

| executive.savedWakeupStruct.read[0] -> Q5),

Q3 = ( executive.savedWakeupStruct.read[1] -> Q3

| executive.savedWakeupStruct.assign[0] -> Q4),

Q4 = ( executive.exec.unlock -> Q0

| executive.savedWakeupStruct.assign[0] -> Q4

| executive.savedWakeupStruct.read[0] -> Q5),

Q5 = ( executive.savedWakeupStruct.assign[0] -> Q4

| executive.savedWakeupStruct.read[0] -> Q5).

not displaying sink state Q1

Analysis Results for P7

Analysis Tool LOC Monolithic model checking

Modular verification

Design level LTSA 700 FSP 4,672 states 541 states

Code level JPF 7.2K Java 183K states 53K states

Conclusions and Future Work

Lifecycle verification and validation of K9 executive– Modeled and checked key autonomy features at design level– Found error in design (before coding); error fixed by

developer– Influenced developer to improve/simplify design

Developed testing framework for code verification– Tool for automated input plan generation, property monitoring

Developed novel compositional verification techniques for increased scalability

Future plans– Verification and validation for future universal executive

(PLEXIL)– Automated verification of plans generated by Ames planner