5/27/03MDES 20031 Supporting Model-Based Validation at Run-time Insup Lee and Oleg Sokolsky...

5/27/03 MDES 2003 1

Supporting Model-Based Validation at Run-time

Insup Lee and Oleg SokolskyDepartment of Computer and Information

ScienceUniversity of Pennsylvania

5/27/03 MDES 2003 2

SDRL & RTGUniversity of Pennsylvania

Software Development Process• Requirements capture and analysis

– Informal to formal– Consistency and completeness– Assumptions and interfaces between

system components– Application-specific properties

• Design specifications and analysis– Formal modeling notations– Analysis techniques (simulation, model

checking, equivalence checking, testing, etc.)

– Abstractions• Implementation

– Manual/automatic code generation (refinement)

– Validation • Testing• Model extraction and verification• Run-time monitoring and checking

Requirements

Designspecification

Implementation

5/27/03 MDES 2003 3


Model-based validation at run-time

• Run-time verification is to assure the compliance of an execution of a system with its formal requirements.

• This is achieved by monitoring the current execution and checking properties.

• The monitoring and checking (MaC) system can– detect incorrect execution of an application– predict error and steer computation– collect statistics of actual execution (e.g., QoS

validation)

5/27/03 MDES 2003 4


System SpecSystem

Spec

RequirementsSpec

RequirementsSpec

Formal verification

Design

SystemImplementation

SystemImplementation

MonitoringRefinementMonitoringRefinement

Implementation

Event Recognizer +

Checker

Event Recognizer +

Checker

SystemSystem FilterCommunication

Run-time Check

MaC Methodology

5/27/03 MDES 2003 5


The MaC Languages• Meta Event Definition Language(MEDL)

– Describes the safety requirements of the system, in terms of conditions that must always be true, and alarms (events) that must never be raised.

– Target program implementation independent.

• Primitive Event Definition Language (PEDL)– Maps the low-level state information of the system to high-level events.– Provides primitives to refer to values of variables and to certain points in

the execution of the program.– PEDL is defined so that events can be recognized in time linear to the

size of the PEDL specification– Depends on target program implementation

Run-time state:•control locations•object state•local variables

Abstract state:•events•conditions•auxiliary variables

MEDL

PEDL

5/27/03 MDES 2003 6


Design of the MaC Languages

• Must be able to reason about both time instants and information that holds for a duration of time in a program execution.– Events and conditions are a natural division, which is also

found in other formalisms such as SCR. – Conditions, which are true or false for a finite duration of

time (e.g., is variable x >5?)– Events, which are either present or absent at some instant of

time (e.g., is control right now at the end of method f?).• Need temporal operators combining events and conditions in

order to reason about traces.

start(position==100) end(position==100)

1:00:10 1:00:301:00:15

raiseGate

Time

position == 100

5/27/03 MDES 2003 7


Logical Foundation

• Conditions interpreted over 3 values: true, false and undefined.

• [., .) pairs a couple of events to define an interval.• start and end define the events corresponding to

the instant when conditions change their value.

212121 | | | ),[ | )(defined:: CCCCCEEC c |C

CE

EEEECCeE

when

| | | )(end | )(start | :: 2121

5/27/03 MDES 2003 8


Meta Event Definition Language (MEDL)

ReqSpec <spec_name>

/* Import section */ import event <e>; import condition <c>;

/*Auxiliary variable */ var int <aux_v>;

/*Event and condition */ event <e> = ...; condition <c>= ...;

/*Property and violation */ property <c> = ...; alarm <e> = ...;

/*Auxiliary variable update*/ <e> -> { <aux_v'> := ... ; }End

• Expresses requirements using the events and conditions, sent by event recognizer.

• Expresses the subset of safety properties.

• Describes the safety requirements of the system, in terms of conditions that must always be true, and alarms (events) that must never be raised.

– property safeRRC = IC -> GD;

– alarm violation = start (!safeRRC);

• Auxilliary variables may be used to store history.

– endIC-> { num_train_pass’ = num_train_pass + 1; }

5/27/03 MDES 2003 9


Property checking

• A MEDL specification can be seen as an automaton with auxiliary store running on a stream of events provided by the event recognizer

aux. variables

5/27/03 MDES 2003 10


PEDL for Java

• The language maps the low-level state information of the system to high-level events and conditions used in describing the requirements.

• Provides primitives to refer to – primitive variables – beginnings/endings of methods

• Primitive conditions are constructed from – boolean-valued expressions over the

monitored variables• Ex: condition IC = (position == 100);

• Primitive events are constructed from – update(x)– startM(f)/endM(f)

• Ex: event raiseGate= startM(Gate.gu());

MonScr <spec_name> /* Export section */ export event <e>; export condition <c>;

/* Overhead reduction */ [timestamp;] [valueabstract;] [deltaabstract;] [multithread;]

/* Monitored entities */ monobj <var>; monmeth <meth>;

/* Event and condition*/ event <e> = ...; condition <c>= ...;End

5/27/03 MDES 2003 11


Java-MaC Prototype

Requirement Specification

Program(Java source code)

Program(Java byte code)

Monitoring Script(PEDL)

Requirements(MEDL)

PEDLPEDLCompilerCompiler

MEDLMEDLCompilerCompiler

InstrumentedTargetprogram

Filter (Compiled MEDL)Run-time Checker

(Compiled PEDL)Event Recognizer

Filter Generator(JTREK)

InstrumentationInformation

Static Phase

Dynamic Phase

5/27/03 MDES 2003 13


Specifications for Stock ClientsMonScr StockClient export event startPgm, periodStart, conFail, queryResend, oldDataUsed;

monmeth void Client.main(String[]); monmeth void Client.run(); monmeth void Client.failConnection(ConnectTry); monmeth Object Client.retryGetData(int); monmeth Object Client.processOldData();

event startPgm = startM(Client.main(String[])); event periodStart = startM(Client.run()); event conFail = startM(Client.failConnection(ConnectTry)); event queryResend = startM(Client.retryGetData(int)); event oldDataUsed = startM(Client.processOldData());End

ReqSpec StockClient import event startPgm, periodStart, conFail, queryResend, oldDataUsed; var long periodTime; var long lastPeriodStart; var int numRetried; var int numConFail;

alarm violatedPeriod = end((perioidTime’ >= 900) && (periodTime’ <= 1100)); alarm wrongFT = oldDataUsed when ( (numRetries’ < 4)|| (numConFail’ < 3)); startPgm -> {periodTime’ = 1000; lastPeriodStart’ = time(startPgm) -1000; numRetries’ = 0; numConFail’ = 0;} periodStart ->{ numREtries’ = 0; numConFail’ = 0; periodTime’ =time(periodStart)-lastPeriodStart; lastPeriodStart’ = time(periodStart);}...End

5/27/03 MDES 2003 14


Extensions for Embedded Systems

• MaC based on hybrid system models• Distributed MaC• Steering-based adaptation• Model-based profiling

5/27/03 MDES 2003 15


Monitoring of hybrid systems

• Embedded systems often contain continuously evolving as well as discrete components

• Monitoring of continuous entities introduces additional challenges– Monitoring of discrete systems is event-driven, which is

not enough for continuous changes– “Continuous monitoring”, in practice, involves sampling– Sampling rates for each continuous entity can be

obtained from a hybrid model of the embedded system

• Deriving properties of embedded systems:– Use a hybrid model as an oracle, driven by the

observations

5/27/03 MDES 2003 16


Distributed MaC

• Run-time monitoring and checking of distributed systems – Local monitors and checkers– How to decompose a global property into locally

checkable properties, i.e., • Given the set of nodes, N1,…, Nm and a global

property Pg, find locally checkable properties, P1 ,…,

Pm such that if Ni detects the violation of Pi, then Pg is violated.

– Tradeoff between amount and frequency of communication and timely detection of property violation

5/27/03 MDES 2003 17


Steering-based Adaptation

5/27/03 MDES 2003 18


Steering-based adaptation

• Steering provides feedback from the monitor to the system

• When can a system be effectively steered?– the system is designed for run-time adjustments

• e.g., Simplex architecture

– the system naturally offers a degree of tolerance • e.g., routing protocols: flush buffers when

performance decreases

• Need timing guarantee over feedback loop– Real-time Java

5/27/03 MDES 2003 19


Model-based Profiling

• Collect statistics during monitored executions• Over multiple runs, such statistics will provide

dependability and performance measures• Possible uses:

– Collect coverage• Successful runs serve as tests• Enhance “synthetic” test suites

– Infer dynamic interfaces (e.g. interface automata)• Record input and output patterns of successful

executions• Helps in reusing the component in different contexts

5/27/03 MDES 2003 20


Summary

• The MaC architecture provides a lightweight formal methodology for assuring of the correct execution of a target program at run-time– Rigorous analysis, Flexible, Automatic

• Identified several possible extensions to MaC• Reduction of monitoring overhead • Port the MaC architecture to platforms other than

Java • http://www.cis.upenn.edu/~rtg/mac

5/27/03 MDES 2003 21

Q & A

5/27/03MDES 20031 Supporting Model-Based Validation at Run-time Insup Lee and Oleg Sokolsky...

Documents

Transcript of 5/27/03MDES 20031 Supporting Model-Based Validation at Run-time Insup Lee and Oleg Sokolsky...