MaCS: Monitoring, Checking and Steering

O. Sokolsky, S. Kannan, I. Lee, U. Sammapun,

J. Shin, M. Viswanathan

CIS, Penn

M. Kim

SECUi.com, Korea

Outline

Motivation Runtime Formal Analysis Our MaCS Prototype Examples Future Work

Motivation

Two traditional approaches to certify correctness of systems Testing cannot guarantee correctness of

application completely and lacks of formality. Formal verification lacks scalability and does not

apply to implementation directly gap between models and implementations

We need a new approach - run-time formal analysis

Run-time Formal Analysis Run-time formal analysis ensures the run-time

compliance of an execution of a system with its formal requirement.

The analysis validates properties on the current execution of application.

The analysis can detect incorrect execution of applications predict error and steer computation collect statistics of actual execution

Run-time Formal Analysis using MaCS

To detect incorrect execution, we need Monitor: monitors changes in an

application by instrumenting monitor code into an application. This will create an abstract view of the application.

Checker: uses monitor information and requirement spec to detect violation

To steer, we need The violation information from Checker and

the Feed-back to the application

System SpecSystem

Spec

RequirementSpec

RequirementSpec

Formal verification

Design

SystemImplementation

SystemImplementation

MonitoringData

MonitoringData

Implementation

Monitor +CheckerMonitor +Checker

SystemSystem FilterCommunication

Run-time Check

MaCS Methodology

Issues in Run-time Formal Analysis

Side effect of instrumentation to a target system

An expressive formal language describing correctness criteria

A proper granularity of monitoring

Automatic v.s. Manual instrumentation

Synchronous v.s. Asynchronous monitoring

Outline Motivation Runtime Formal

Analysis Our MaCS Prototype Examples Future Work

MaCS languages

PEDL: abstraction

Run-time state:•control locations•object state•local variables

Abstract state:•events•conditions•auxiliary variables

MEDL

PEDL

SADL

MEDL: abstract transformation SADL: feedback

Events and Conditions Information about the system

comes in two different forms: Conditions, which are true, false, or

undefined for a finite duration of time

Events, which are either present or absent at some instant of time

Examples of Events and Conditions

C3

C3. condition safeVolts = ( -10 < IP.volts ) && ( IP.volts < 10 );

IP.volts = 6

IP.volts = 15

E1 E2 E2

IP.init()

E1. event startProg = startM( IP.init() );E2. event voltsChange= update( IP.volts);

Logic for Events and Conditions

Let c be a primitive condition Let e be a primitive event Define Condition (C) and Event (E) as:

<C> ::= c | [ <E>, <E> ) | ! <C> | <C> && <C> | <C> || <C> | <C> => <C> <E> ::= e | start( <C> ) | end( <C>) | <E> && <E> | <E> || <E> | <E> when <C>

Conditions

<C> ::=

[ <E1>, <E2> )

! <C><C1> && <C2>

<C1> || <C2>

<C1> => <C2>

c |

|

|

|

|

Primitive condition

T from E1 until E2

T if C is F

T if C1 and C2 are T

T if C1 or C2 are T

T if C2 is T whenever C1 is T

Events

<E> ::=

start( <C> )

<E1> && <E2>

<E1> || <E2>

<E> when <C>

e |

|

|

|

|

Primitive event

An instant when C becomes T

An instant when C becomes F

An instant when both events happen

An instant when either events

happenAn instant when E

happens and C holds T

end( <C> )

The language that creates an abstract view of the application

It maps the low-level state information of the system to high-level events/conditions used in describing the requirements.

Primitive Event Definition Language (PEDL)

Primitive Event Definition Language (PEDL)

x=0,y=0

x=1,y=0

x=2,y=0

x=2,y=1

ProgramExecution

Abstract ViewMonitor Sees

x=3,y=1

x < 2

x =2

x> 2

x=3,y=2

InformationExtraction

Primitive Event Definition Language (PEDL)

Provides primitives to refer to values of variables and to certain points in the execution of the program. condition safeVolts =

( -10 < IP.volts ) && ( IP.volts < 10 ); event startProg = startM( IP.init() ); event voltsChange= update( IP.volts);

Provides primitive “time” to refer to time when events happen condition slowDoor =

( time(doorClose) - time(doorOpen) ) > 3000;

Meta Event Definition Language (MEDL)

Expresses requirements using the events and conditions, sent by a monitor. Defines Auxiliary Variables, which may be

used to store history. RaiseGateEvent -> { num_train_pass++; }

Defines more events and conditions using auxiliary variables and events/conditions sent by a monitor.

event safeStart = startProg when (safeVolts)

Meta Event Definition Language (MEDL) (Cont. )

Using auxiliary variables, events and conditions, we describes the safety requirements of the system, in terms of conditions that must always be true, and alarms (events) that must never be raised. safeprop safeIP = safeTrack && safeVolts; alarm violation = start (!safeIP);

Specifies updates to variables and steering invocations in response to events violation -> { controller = 0;

invoke change2SafetyController(); }

Property checking

A MEDL specification can also be seen as an automaton running on a stream of events provided by the monitor and the auxiliary variable

aux. variables

Steering Action Definition Language(SADL)

What: Identifies object used in steering// the target of steering is the object dm of type

// DecisionModule located in the class IP

DecisionModule IP:dm;

How: Defines steering actions// setSC() method of dm is invoked

steering action change2SC = { call (IP:dm).setSC(); }

Where: Specifies steering conditions locations in the code where the actions can be

executedbefore read DecisionModule:volts;

Steering process

system

checker

violation

action invoked

action executed

detection

steeringconditionsatisfied

actioninvocationreceived

event received

MaCS Toolset

Program (.class)(Java byte code)

Monitoring Script(PEDL)

Requirements(MEDL)

PEDLPEDLCompilerCompiler

MEDLMEDLCompilerCompiler

Instrumented Code

Filter Generator(JTREK)

InstrumentationInformation

Compiled PEDL

Compiled MEDL

Event Recognizer Checker

Steering Script(SADL)

SADLSADLCompilerCompiler

InstrumentationInformation

Injector class(Java byte code)

Filter (or an instrumented class)

Initializes the connection between application and monitor.

For each monitored object: for each variable update or start/end method,

send events/conditions to the monitor.

Filter (or an instrumented class)

IP.init()

Instrumented class

IP.volts = 6

IP.volts = 15

send to monitor

Implementation of monitor

EventReconizer class: Listener (i.e. server)

accepts events and conditions, specified in PEDL, from filter.

Notify (i.e. client) sends events and conditions to runtime checker

Implementation of Monitor

IP.init()

Instrumented class Monitor

IP.volts = 6

IP.volts = 15

startProg

safeVolts = T

voltsChange

voltsChange

safeVolts = F

send to checker

Implementation of checker

RuntimeChecker class: Listener (i.e. server)

accepts events and conditions, specified in MEDL, from monitor.

Check for any violation If violation occurs, check if any action needs to

be invoked by Steering Notify (i.e. client)

sends action to be invoked to steerer.

Implementation of Checker

IP.init()

Instrumented class Monitor

IP.volts = 6

IP.volts = 15

startProg

safeVolts = T

voltsChange

voltsChange

safeVolts = F

startProg

alarm violation = voltsChange when !

safeVolts

ALARM

OK

Checker

send to steering component

Implementation of steering

Injector class: Action bodies as methods Listener thread

accepts action invocations and sets invocation flags Calls to action bodies at fixed locations

defined in the steering script, added by instrumentation

Implementation of Checker

IP.init()

Instrumented class Monitor

IP.volts = 6

IP.volts = 15

startProg

safeVolts = T

voltsChange

voltsChange

safeVolts = F

startProg

alarm violation = voltsChange when !

safeVolts

ALARM

OK

Checker Injector

Action change2SC

IP.setSC()

Implementation Issues

How close should the application and the monitor be? Tight coupled

Pro: Decrease Steering Latency Con: Compete for resources

Implementation Issues

Conservativeness How to ensure that the application

does not suffer a significant performance degradation as long as steering is not invoked?

Implementation Issues

Effectiveness How to ensure that the steering

action cannot be ignored by the application?

How to ensure that after the action is invoked, the application will not “undo” the action?

Inverted Pendulum (courtesy L. Sha)

m

l

x

g

Muf

Example

Simplex Architecture (courtesy L. Sha)

Safety

Experimental

DecisionModule

PhysicalSystem

us

ue

xu

SC

EXx0

Equilibriumstate

Experimental controllers provide improved performance but uncertain stability properties Can be dynamically added or replaced

Safety controller has the largest stability region

ExperimentalController

ExperimentalController

Inverted Pendulum in MaCS

DeviceDrivers

angle,track

volts

Decision Module

ExperimentalController

Switching logic

SafetyController

JNI

monitor

steer

Case study results

It works! Without disturbance, there is slightly

more jitter partly due to instrumentation but mostly due to

Java

Switch to the safety controller takes 0.32s (16 control cycles) bad values compensated by the safety controller in general may not be good enough

Future Work

Synchronous vs. asynchronous steering currently: the application and may produce

erroneous outputs when checking is slow May be unacceptable in critical situations

alternative: before producing critical output, pause and wait for checker feedback

IP case study suggests it is possible in some cases Need predictable checker turnaround

Currently work in progress

Future Work

Static analysis to enhance steering currently, user is responsible for specifying

steering locations – error prone control flow analysis can identify locations by

code-independent criteria e.g. “steering location is visited once in each

main loop iteration” or even “if action changes variable x, do not

place before the system writes x” What are the right criteria?

Study other uses of steering