The UML/MARTE Veri er

ETR course

The UML/MARTE VerifierA Property Driven toolchain for model checking real time systems

Marc Pantel (based on Ning Ge and Faiez Zalila work)

Universite de Toulouse, IRIT-CNRS, ACADIE

August 27, 2015

Work funded byFUI TOPCASED, ITEA OPEES, FUI Projet P, ITEA openETCS, IRT Saint Exupery

Marc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 1 / 59

Outline

1 Introduction

2 Method to integrate formal verification for DSMLs

3 Property-Driven Approach

4 Semantic Mapping from UML-MARTE to TPN

5 Real-Time Property Specification

6 Observer-Based Property Verification

7 Property Specific State Space Reduction

8 Feedback Analysis Proposal

9 Synthesis


Introduction

Outline

1 Introduction








9 Synthesis


Introduction

Safety Critical Real-Time Embedded Systems


Introduction

Real-Time Requirements

!"#$%&'(")!"*+',"("-./


Introduction


!"#$%&'(")!"*+',"("-./

012'3#$)4("),"*+',"("-./

567/'3#$)4("),"*+',"("-./


Introduction


!"#$%&'(")!"*+',"("-./

012'3#$)4("),"*+',"("-./

!"#$%&'($)*$'*+($!,(*-$-*"*./.+0$-*1$,-2%$(3*$4.52(❚

567/'3#$)4("),"*+',"("-./


Introduction


!"#$%&'(")!"*+',"("-./

012'3#$)4("),"*+',"("-./

567/'3#$)4("),"*+',"("-./

1 5 10 15 20

!"#$%&'($)*$'*+($!,(*-$-*"*./.+0$-*1$,-2%$(3*$4.52(❚


Introduction


!"#$%&'(")!"*+',"("-./

012'3#$)4("),"*+',"("-./

567/'3#$)4("),"*+',"("-./

1 5 10 15 20

!"#$%&'($)*$'*+($!,(*-$-*"*./.+0$-*1$,-2%$(3*$4.52(

!"#$%&'($)*$'*+($!,(*-$-*"*./.+0$-*1$,-2%$(3*$4.52($.+$6789$:8;%'$.+$*!"3$4*-.2<❚


Introduction


!"#$%&'(")!"*+',"("-./

012'3#$)4("),"*+',"("-./

567/'3#$)4("),"*+',"("-./

1 5 10 15 20

!"#$%&'($)*$'*+($!,(*-$-*"*./.+0$-*1$,-2%$(3*$4.52(

!"#$%&'($)*$'*+($!,(*-$-*"*./.+0$-*1$,-2%$(3*$4.52($.+$6789$:8;%'$.+$*!"3$4*-.2<

=3*$*+0.+*$#**4'$'(!55.+0$,2-$!($5*!'($>$'*"❚


Introduction


!"#$%&'(")!"*+',"("-./

012'3#$)4("),"*+',"("-./

567/'3#$)4("),"*+',"("-./

1 5 10 15 20

!"#$%&'($)*$'*+($!,(*-$-*"*./.+0$-*1$,-2%$(3*$4.52(

!"#$%&'($)*$'*+($!,(*-$-*"*./.+0$-*1$,-2%$(3*$4.52($.+$6789$:8;%'$.+$*!"3$4*-.2<=3*$*+0.+*$#**4'$'(!55.+0$,2-$!($5*!'($>$'*"❚

!"*+',"893'"-.)

:",';3#41-


Introduction

Model Driven Engineering & Formal Methods

!"#$%&'()*$+&,+-)+$$()+-

."(/0%&!$12"#3

4


Introduction


!"#$%&"'"()

*

!"#$%&"'"()*

+,+

-&./%)".)$&

"0

1"*%2(

-&./%)".)

$&"0

1"*%2(0

+,+1")3%4"

50

1"*%2(

1")3%4"50

1"*%2(0+,+

675"

8"("&3

97(

675"0

8"("&397

(

+,+

!"#$%&"'"()*

+,+

1")3%4"50

1"*%2(0

+,+

-&./%)".)$&

"0

1"*%2(0

+,+

-&./%)".)$&"0

1"*%2(0+,+

!"#$%&"'"()

*

+,+

!"#$%&"'"(

)*

+,+

:75"401&%;"(0<(2%(""&%(2

=7&'340:")/75*

,


Introduction


!"#$%&"'"()

*

!"#$%&"'"()*

+,+

-&./%)".)$&

"0

1"*%2(

-&./%)".)

$&"0

1"*%2(0

+,+1")3%4"

50

1"*%2(

1")3%4"50

1"*%2(0+,+

675"

8"("&3

97(

675"0

8"("&397

(

+,+

!"#$%&"'"()*

+,+

1")3%4"50

1"*%2(0

+,+

-&./%)".)$&

"0

1"*%2(0

+,+

-&./%)".)$&"0

1"*%2(0+,+

!"#$%&"'"()

*

+,+

!"#$%&"'"(

)*

+,+

M3_str

[0,60000]

M1_str

[0,50000]KU1_devitf MFD1_devitf

KU1_offset

[0,0]

MFD1_offset

[25000,25000]

FM1_offset

[0,0]

NDB_input

NDB_data

[0,0] NDB_execp

NDB_exectr

[0,20000]

FM1a_devitf

MFD1_hold

MFD1_data

[0,0] MFD1_execp

KU1_data

[0,0]KU1_execp

KU1_exectr

[0,25000]KU1_output

FM1_data

[0,0]

MFD1_input

SP_inittr

KU1_input SP_initp

KU1_holdMFD1_waitp

KU1_null

[0,0]KU1_waitp

KU1_waittr

[50000,50000]

MFD1_waittr

[50000,50000]

MFD1_null

[0,0]

FM1_holdFM1_waitp

FM1_null

[0,0]

FM1_waittr

[60000,60000]

FM1_inputFM1_execp

FM1_exectr

[0,30000]FM1_output

FM1a_offset

[0,0]

FM1_devitf

FM1a_hold

FM1a_input

M7_sp NDB_devitf

NDB_offset

[0,0] NDB_hold

NDB_null

[0,0] NDB_waitp

NDB_waittr

[100000,100000]

FM1a_MFD1_comm

[310,490]

KU1_FM1_comm

[298,444]

M3_sp

FM1_NDB_comm

[268,310]

M7_str

[0,100000]

NDB_output

M1_sp

NDB_FM1a_comm

[400,508] NDB_bag

NDB_FM1a_bag

[0,64000]

FM1a_exectr

[0,30000]

FM1a_data

[0,0]

FM1a_null

[0,0]

FM1a_waittr

[60000,60000]

FM1a_waitp

FM1a_execp FM1a_output

MFD1_output

MFD1_exectr

[0,25000]

:75"401&%;"(0<(2%(""&%(2

,

=7&'340:")/75*


Introduction

V & V in MDE

!"#$%&"'"()* !"

#$%&"'"()*

+,+

-&./%)".)$&"0

1"*%2( -

&./%)".)$&"0

1"*%2(0+,+

1")3%4"50

1"*%2( 1

")3%4"50

1"*%2(0+,+

675"

8"("&397(

675"08"("&397(

+,+

!"#$%&"'"()*

+,+

1")3%4"50

1"*%2(0+,+

-&./%)".)$&"0

1"*%2(0+,+

-&./%)".)$&"0

1"*%2(0+,+

!"#$%&"'"()*

+,+

!"#$%&"'"()*

+,+

:%'"0;%("

Note: from MeMVaTEx methodology


Proposed method

Outline

1 Introduction








9 Synthesis


Proposed method

Domain-Specific Modeling Languages (DSMLs)

model

model

model

represented by

represented by

represented by

conforms to

conforms to

conforms to

Model-Driven Engineering

editors

simulators

User

verifiers

generators

DSML

editors

simulators

User

verifiers

generators

DSML

editors

simulators

User

verifiers

generators

DSML


Proposed method

Domain-Specific Modeling Languages (DSMLs)

model

model

model

represented by

represented by

represented by

conforms to

conforms to

conforms to

Model-Driven EngineeringLanguage Engineering

editorsLanguage

expert

Domain expert

simulators

User

verifiers

generators

DSML

editors

simulators

User

verifiers

generators

DSML

editors

simulators

User

verifiers

generators

DSML

Language expert

Domain expert

Language expert

Domain expert


Proposed method

Verification and Validation (V&V) activities

model

model

model

represented by

represented by

represented by

conforms to

conforms to

conforms to

Model-Driven EngineeringLanguage Engineering

Formal verification

editorsLanguage

expert

Domain expert

simulators

User

verifiers

generators

DSML

editors

simulators

User

verifiers

generators

DSML

editors

simulators

User

verifiers

generators

DSML

Language expert

Domain expert

Language expert

Domain expert


Proposed method

Formal verification technique

User requirements:

Ease of useAutomationEfficiencySoundnessCompleteness

Candidate:

Automated theorem proving (SAT/SMT solvers) (logic based, userprovided dedicated abstractions)Abstract interpretation (state based, automated generic abstractions)Model checking (state based, user provided dedicated abstractions)


Proposed method

Model checking based formal verification architecture

model-checkingtools

DSMLmodel

Formalmodel

Formalproperties

Formalverification

results

DSMLverification

resultsDSML

end-user

defines

obtains

defines/uses

DSMLbehavioral properties

Formal verificationDSML Verifier


Proposed method

Model checking based formal verification architecture

model-checkingtools

DSMLmodel

Formalmodel

Formalproperties

Formalverification

results

DSMLverification

resultsDSML

end-user

defines

obtains

defines/uses


Formal verification

Interpretation approach (Operational semantics)

Translational approach (Translational semantics)

DSML Verifier


Proposed method

Translational approach

model-checkingtools

DSMLmodel

Formalmodel

Formalproperties

Formalverification

results

DSMLverification

resultsDSML

end-user

defines

obtains

defines/uses


Formal verification

Translational approach (Translational semantics)

DSML Verifier


Proposed method

DSML Verifier: Reuse formal tools

model-checkingtools

DSMLmodel

Formalmodel

Formalproperties

Formalverification

results

DSMLverification

resultsDSML

end-user

defines

obtains

defines/uses


model-checkingtools

Formalmodel

Formalproperties

Formalverification

results

DSML Verifier


Proposed method

Defining a translational semantics

model-checkingtools

DSMLmodel

Formalmodel

Formalproperties

Formalverification

results

DSMLverification

resultsDSML

end-user

defines

obtains

defines/uses


model-checkingtools

Formalmodel

Formalproperties

Formalverification

results

Translational semantics

Domain expert

Language expert

specifies implementsDSML Verifier


Proposed method

Completing the integration

model-checkingtools

DSMLmodel

Formalmodel

Formalproperties

Formalverification

results

DSMLverification

resultsDSML

end-user

defines

obtains

defines/uses


model-checkingtools

Formalmodel

Formalproperties

Formalverification

results

Translational semantics

Domain expert

Language expert

specifies implements

Properties generation

Feedbackverification

results

DSML Verifier


Proposed method

Use case driven method


Proposed method


Ad-hoc solutions

Analyse results

Suggest generic

solutions


Proposed method


Ad-hoc solutions

Analyse results

Suggest generic

solutions Capitalize know-how

and expertise

Apply on use-case

Validateproposed solutions


Proposed method


Ad-hoc solutions

Analyse results

Suggest generic


and expertise

Apply on use-case


Apply on use-case

Validate proposedsolutions

Capitalize know-how

and expertise


Proposed method


Ad-hoc solutions

Analyse results

Suggest generic


and expertise Package our contributions

Collect applications feedbacks

Synthesize our contributions

Apply on use-case


Apply on use-case

Validate proposedsolutions

Capitalize know-how

and expertise


Proposed method

Case Study: Flight Management System (FMS)

Rely on Integrated Modular Avionics (IMA) principles


Proposed method

FMS Architecture Model by Boniol and Lauer

Module1KU1

MFD1

Module2KU2

MFD2

Module3FM1

Module4FM2

Module5ADIRU1

Module6ADIRU2

Module7NDB

S1

S4 S5

S3S2

RDC1 RDC2

sensor1 sensor2

keyboard1 display2 keyboard2display1

!"#$%"&'()*+&,-'.#)$

!"/+.$0%'.#)$

123

functions, AFDX network


Proposed method


Module1KU1

MFD1

Module2KU2

MFD2

Module3FM1

Module4FM2

Module5ADIRU1

Module6ADIRU2

Module7NDB

S1

S4 S5

S3S2

RDC1 RDC2

sensor1 sensor2


!"#$%"&'()*+&,-'.#)$

!"/+.$0%'.#)$

123

%04



Proposed method


Module1KU1

MFD1

Module2KU2

MFD2

Module3FM1

Module4FM2

Module5ADIRU1

Module6ADIRU2

Module7NDB

S1

S4 S5

S3S2

RDC1 RDC2

sensor1 sensor2


!"#$%"&'()*+&,-'.#)$

!"/+.$0%'.#)$

123

4+1(5 4+1(6



Proposed method


Module1KU1

MFD1

Module2KU2

MFD2

Module3FM1

Module4FM2

Module5ADIRU1

Module6ADIRU2

Module7NDB

S1

S4 S5

S3S2

RDC1 RDC2

sensor1 sensor2


!"#$%"&'()*+&,-'.#)$

!"/+.$0%'.#)$

123

4.0%-5 4.0%-6



Proposed method


Module1KU1

MFD1

Module2KU2

MFD2

Module3FM1

Module4FM2

Module5ADIRU1

Module6ADIRU2

Module7NDB

S1

S4 S5

S3S2

RDC1 RDC2

sensor1 sensor2


!"#$%"&'()*+&,-'.#)$

!"/+.$0%'.#)$

123

,#*40%5 ,#*40%6



Proposed method


Module1KU1

MFD1

Module2KU2

MFD2

Module3FM1

Module4FM2

Module5ADIRU1

Module6ADIRU2

Module7NDB

S1

S4 S5

S3S2

RDC1 RDC2

sensor1 sensor2


!"#$%"&'()*+&,-'.#)$

!"/+.$0%'.#)$

123

4+1#5"6 4+1#5"7



Proposed method


Module1KU1

MFD1

Module2KU2

MFD2

Module3FM1

Module4FM2

Module5ADIRU1

Module6ADIRU2

Module7NDB

S1

S4 S5

S3S2

RDC1 RDC2

sensor1 sensor2


!"#$%"&'()*+&,-'.#)$

!"/+.$0%'.#)$

123

()*+4 ()*+5



Proposed method

Latency Real-Time Requirements

In the pilot request functional chain, the time between req1 and the firstoccurrence of disp1 depending on req1 must be in time range [bct,wct].

KU1 MFD1 KU1 MFD1 KU1 MFD1 KU1 MFD1 KU1 MFD1 KU1 MFD1

FM1 FM1 FM1 FM1 FM1

NDB NDB NDB

M1

M3

M7

req1[1]

0

15 45

25 50 225

75 195

25 45 125

disp1[5] disp1[6]

l10 240


Proposed method

Verification of FMS Case Study

Proposal of Boniol and Lauer

Abstraction based on trajectory approach for the AFDX network

Formal modeling using tagged signal model

Transformed in Integer Linear Programming (ILP) problems

Model Checking?

Modeling and Analysis using timed automata & UPPALL

State space combinatorial explosion issue

Further Study on Model Checking

Methods for minimizing verification semantics to reduce the state space.


Proposed method

Phase in the development process

!"#$%&"'"()* !"

#$%&"'"()*

+,+

-&./%)".)$&"0

1"*%2( -

&./%)".)$&"0

1"*%2(0+,+

1")3%4"50

1"*%2( 1

")3%4"50

1"*%2(0+,+

675"

8"("&397(

675"08"("&397(

+,+

!"#$%&"'"()*

+,+

1")3%4"50

1"*%2(0+,+

-&./%)".)$&"0

1"*%2(0+,+

-&./%)".)$&"0

1"*%2(0+,+

!"#$%&"'"()*

+,+

!"#$%&"'"()*

+,+

:%'"0;%("


Proposed method

Phase in the development process

!"#$%&"'"()* !"

#$%&"'"()*

+,+

-&./%)".)$&"0

1"*%2( -&

./%)".)$&"0

1"*%2(0+,+

1")3%4"50

1"*%2( 1

")3%4"50

1"*%2(0+,+

675"

8"("&397(

675"08"("&397(

+,+

!"#$%&"'"()*

+,+

1")3%4"50

1"*%2(0+,+

-&./%)".)$&"0

1"*%2(0+,+

-&./%)".)$&"0

1"*%2(0+,+

!"#$%&"'"()*

+,+

!"#$%&"'"()*

+,+

:%'"0;%("

<=;>=-!:?

:%'"0@")&%0A")*0B0:CA-


Property-Driven Approach

Outline

1 Introduction








9 Synthesis




Principle

The formal activities in the development process are based on the purpose ofproperty-verification-ease.

Experiments by B. Combemale

Verification of structural and temporal properties for Development Process models.

Requires more scalable methods to verify quantitative properties.

Proposed method

1 Characterize expected properties.

2 Characterize mandatory observable states and events to assess these properties.

3 Express real-time properties using elementary property patterns.

4 Define translational semantics to Time Petri Net (TPN) with observers and reachabilityassertions.

5 Reduce state space: property-specific reduction for TPN.

6 Validate model and feedback: automated failure analysis.



Time Petri Net

[0,0] [3,10]

2

[11,15]

[19,27]Pinit Tfork

Ptask1

Ptask2

Texe1

Texe2

Pjoin Texit Pexit

(10, ∞]

2

Trestart

TINA toolset

Analyze µ-calculus, LTL, CTL properties for TPN.

Integrate state space abstraction techniques (preserving different kinds of properties),on-the-fly model checking.

Data manipulation (tts): variables used in transition guards and actions.

Proposal

Rely on observers and reachability assertions.

Transform quantitative problem into reachability problem.

Minimize semantics for observation based on state space preserving markings.



Challenge & Property-Driven Verification Framework

TPN

Reduced Observer TPN

ReachabilityAssertions

TPN Model CheckingTag Property Pattern Result

Architecture/Behavior Mapping

Observer TPN Generation

IterationTag

Property Pattern Result

Real-Time PropertySpecification

Verification Result

Computation

Real-Time PropertyVerification Result

Feedback Generation

System ModelReal-Time

RequirementArchitecture

ModelBehavior

Model

UML Real-Time Software Model

Timing Property Pattern


Real-Time Property Patterns

1

5

3

2

3

3

Observer TPN

Tag Property Pattern Result Interpretation

3

TPN Reduction

4




TPN


Reachability Assertions




IterationTag



Verification Result

Computation


Feedback Generation



ModelBehavior

Model





1

5

3

2

3

3

Observer TPN


3

TPN Reduction

4

[0,0] [3,10]

2

[11,15]

[19,27]Pinit Tfork

Ptask1

Ptask2

Texe1

Texe2

Pjoin Texit Pexit

(10, ∞]

2

Trestart

TINA toolset

Assess µ-calculus, LTL, CTL requirements for TPN.

Integrate state space abstraction techniques (preserving different types of properties),on-the-fly model checking.

Provide data manipulation (tts): variables used in transition guards and actions.

Proposal

Rely on observers and reachability assertions.

Transform quantitative problems into reachability problems.

Minimize semantics for observation based on state space preserving markings.




TPN








Verification Result

Computation


Feedback Generation



ModelBehavior

Model





1

5

3

2

3

3

Observer TPN


3

TPN Reduction

4

IterationTag




TPN








Verification Result

Computation


Feedback Generation



ModelBehavior

Model





1

5

3

2

3

3

Observer TPN


3

IterationTag

TPN Reduction

4




TPN








Verification Result

Computation


Feedback Generation



ModelBehavior

Model





1

5

3

2

3

3

Observer TPN


3

TPN Reduction

4

IterationTag




TPN






IterationTag



Verification Result

Computation


Feedback Generation



ModelBehavior

Model





1

5

3

2

3

3

Observer TPN


3

TPN Reduction

4


Semantic Mapping from UML-MARTE to TPN

Outline

1 Introduction








9 Synthesis



Modeling Context

Real-Time Software Systems

Clocks: single & multiple clocks (rate, drift, offset)

Communication: synchronous & asynchronous

Object Value

Ignored in the architecture design

Cyclic execution

Event-trigger: activated by the data and control flow

Time-trigger: also activated by the rising edge of time cycle

MARTE

Simplification on the use of MARTE

Resource scheduling

A generic scheduling algorithm with preemption option is provided




Object Value

Ignored in the architecture design phase

Cyclic execution



MARTE


Resource scheduling




Modeling Context




Object Value

Ignored in the architecture design phase

Cyclic execution



MARTE


Resource scheduling




Modeling Context




Object Value

Ignored in the architecture design

Cyclic execution



MARTE


Resource scheduling




Defining Mapping Semantics from UML-MARTE to TPN

Semantic Mapping Objectives

1 Conforming to the semantics in UML Specification 2.4.1, explicit semantics forvariation points

2 Property specific semantic mapping, preserving minimal set of property-relevantsemantics as possible

3 Standardized mapping for some untimed UML elements

4 Verification-ease, guarantee efficiency of model checking

5 Facilitate the assembly of mapping results

UML-MARTE diagrams

Composite structure diagram

Activity diagram

State machine diagram

Covers a large scope of modeling elements



FMS: Modeling for Latency Requirement

Functional chain on IMA

FM1

FM2 NDB

KU1

FM2

MFD1FM1NDB

MFD2

req1

disp2

disp1

wpInfo2

wpInfo1

answer2

answer1

query2

query1wpId1

wpId2

Architecture

M1:KU_MFD_Module

<<Allocated>> req

<<Allocated>> wpId

<<Allocated>> disp

<<Allocated>> wpInfo

M3:FM_Module

<<Allocated>> wpId

<<Allocated>> query

<<Allocated>> wpInfo

<<Allocated>> anwser

M7:NDB_Module

<<Allocated>> query

<<Allocated>> anwser

<<CommunicationMedia>>




Behavior of FM module

<<RtSpecification>>occKind = PeriodicPattern

(period=[60000,60000]; phase=[0,60000]; occurrences=-1)

FM1

<<Allocated>> wpId1

<<TimeProcessing>>

<<Allocated>> query1

FM1a

<<Allocated>> answer1

<<TimeProcessing>>

<<Allocated>> wpInfo1



FMS: TPN Mapping Result

!"#




!"#$%&'$()*+,

!" $%&

-,.




!"#$%&'$()*+,

!" $%&




!"#"$%&'(




!"#$%&'()*




!"#$%&'$()*+,

!" $%&


Real-Time Property Specification

Outline

1 Introduction








9 Synthesis



Property Pattern Approach

Research Background

Qualitative patterns proposed by Dwyer cover 90% temporal requirements.

Extension to quantitative patterns by Konrad.

Speci&cation

Qualitative Quantitative

Occurrence Order Duration PeriodicQuantitative

Order

Absence Existence PrecedenceChain

Precedence

ResponseUniversalityBounded

Existence

Chain

Response

Maximum

Duration

Bounded

Recurrence

Bounded

Response

Minimum

DurationBounded

Invariance

Type

Catalog

PaCern

Classi&cation

by Dwyer




Problem

Specification-orientation, semantically not atomic.

Proposal

A set of verification-ease elementary time property patterns.

Works as a bridge between specification patterns and formalverification.

Transform Dwyer and Konrad specification patterns and mostMARTE CCSL (Clock Constraint Specification Language) constraints.




Atomic Pa*ern

Event Modi2erState

Scope Modi2erBasic PredicateOccurrence Modi2er

Real-Time Property

Composite Pa*ern

Real-Time

Property Pa0ern

Exist A After B Within [bct, wct]

Operator Occurrence Basic predicate Scope

Absent B global

or Exist A ∧ B between (B + bct) and (B + wct)



FMS: Latency Specification

Atomic Pa*ern

Event Modi2erState

Scope Modi2erBasic PredicateOccurrence Modi2er

Real-Time Property

Composite Pa*ern

Real-Time

Property Pa0ern

FMS latency property:time between pilot’s request and first disp depending on request must be in [bct,wct]

Operator Occurrence Basic predicate Scope

always T (req, disp) ≥ bct global

and always T (req, disp) ≤ wct global


Observer-Based Property Verification

Outline

1 Introduction








9 Synthesis



Verification of Real-Time Property

Proposal

Observer-based model checking approach.

Executed concurrently with the model under assessment.

Define a set of elementary observers for the property patterns.

TPN observers for event based property.tts observers for state based property.

Error feedback provides all failure scenarios (that invalide theobserver)



Design of Observers

Soundness Requirement

Time divergence

No side-effect on the system’s original behavior.

Ensured by construction (structure of the patterns).

TPN Structure TPN Structure

TPN Structure

Component A TPN Component B TPN

TPN Observer

[0,0] [0,0]

TA TB

ptester

Efficiency Requirement

State Abstraction: abstraction preserving markingsRelated work: Abid (PhD thesis, 2013), tts observers with priority arc, state abstraction

Relatively optimal (minimizes states and transition numbers – not proved)

Independent checking: allows parallel computation



Catalog of Observers

Event modifier observers:

E E'

TPN

Structure

Observer

Predicate observers:

EM

TPN

Structure

!!"#$%%&'()*+%

Observer

Scope modifier observers



Occurrence Modifier

Assume in the state class graph

P: set of states that match the predicate,

S : set of states that match the scope,

P ∧ S : set of states that match both the predicate and the scope.

Occurrence

Exist Predicate in Scope:

{P ∧ S 6= ∅ if S 6= ∅;True if S = ∅.

Absent Predicate in Scope: P ∧ S = ∅Always Predicate in Scope: P ∧ S = S

!"#$%&'%()"*+% !"#$%&'%()"*+%

!"#$%

&'%()"*+%

!"#$% &'$()% &*+,-$



Computing Bound Value of Property

Requirement

When performing model checking, an observer can give an answersuch as Yes or No for the satisfaction of the given property.

For quantitative properties, however, users usually expect to knowwhat is the bound [bct,wct] of that property instead of whether theproperty is bounded by [bct,wct]?

Solution

An iterative method that will gradually approach the bound value byintegrating the observers into a binary (k-ary) search engine.



FMS: Verification of Latency Property

[tmin,tmin] [0,0]

[tmax,tmax]

[0,0]

[0,25000]

MFD_exectrSP_ini6r

[0,0]

MFD_exectrSP_ini6r

[0,0] [0,25000]

BCT Observer WCT Observer

TPN System

... ...

TPN System

... ...

(a) Best Case (a) Worst Case

TesterA

[0,0] [0,0]

Over>owA Over>owB

TesterA

Over>ow2 2

Property Property Value (ms) State/Transition Number Execution Time (s)

LatencySystem N/A 9378/23250 N/A

wct 450.4 67105/145024 278.313bct 75.2 11162/28922 43.781

Same results as Boniol and LauerMarc Pantel (IRIT-ACADIE) Property-Driven Verification toolchain August 27, 2015 38 / 59


FMS: Verification of Latency Property

TPN








Verification Result

Computation


Feedback Generation



ModelBehavior

Model





1

5

3

2

3

3

Observer TPN


3

TPN Reduction

4

IterationTag


Property Specific State Space Reduction

Outline

1 Introduction








9 Synthesis



State Space Reduction for TPN

Minimizing verification semantics

Modeling abstraction

Mapping abstraction

State space abstraction provided by TINA

On-the-fly model checking provided by TINA

Existing reduction techniques in model checking

Focus on universal properties

Property specific reduction methods are needed

Solution

1 Remove property irrelevant semantics

2 Combine property relevant semantics by replacing original sub-nets by behavioralequivalent ones



Removal of Property-Irrelevant Semantics

Idea: analyze causality in the state class graph to remove transitions and statesirrelevant to the observed transitions and states.

Paradox: if the state class graph can be generated and analyzed, the reduction isnot needed.

Solution: use dependence analysis as an over-approximation.

Algorithm: search for and remove TPN places and transitions that the targetproperty does not depend on.

A

B

C

D

ETPN Model

F

Obs

Obs



Regular Real-Time Property Specific Behavior

A

Occurrence Time [tmini , tmax

i ] Time Diff [tmini − tmin

i−1, tmaxi − tmax

i−1 ]

0 [0, 0] -1 [5, 10] [5, 10]2 [22, 79] [17, 69]3 [39, 148] [17, 69]... ... ...n [5+17(n-1), 10+ 69(n-1)] [17, 69]




t4

[5,10] p1

t1

[17,69] p2p0

t5

[0,0]p5

A

Before Reduction

177 states /365 transitions

After Reduction

3 states / 3 transitions




Observation

Regular behaviors occur in property related elements.

What are real-time property related elements?

Firing occurrence times of the observed transitions.

The time range of each occurrence of the observed outgoing transitions.

Proposal

Identify potential regular behaviors.

Detect sub-nets that may exhibit these behaviors.

Construct simpler substitute sub-nets that exhibit the same behaviors.

Verify the behavioral equivalence between the original sub-net and the substitute.



Regular Real-Time Property Specific Behaviors

Principle

After replacing the target sub-net int the system, this one exhibits exactly the sameproperty specific behavior as before.

Regular behaviors

Occurrence times, firing time range of the outgoing transition

Finite firing occurrence : sequential sectionInfinite firing occurrence: (sequential section) + loop section

B'

A

C

TA

C

[t1,t2]

[t3,t4]

….

[tm,tn]

B

[ti,tj]

[tp,tq]

….

[tx,ty]

A'

TB

(a) (b)



Divide and Conquer Reduction Approach

System

A

A'

B C

B' C'

3 steps:

1 Identification: some reducible sub-nets like A, B, and C are identified.

One-way-out pattern: single portal outgoing transitionGeneric pattern: single portal incoming and outgoing transition.

2 Reduction: search for the regularity of real-time behavior, construct reducedsub-nets (A′,B ′, and C ′), relying on observers.

3 Refinement: verify the correctness (behavioral equivalence) of the reducedsub-nets, relying on observers by model checking.



What is the benefit of this method?

Benefit

make a trade-off between computation time and space

turns the combination problem of O(N ·M) into a divide-and-conquerproblem of O(n · N + M · δ), where

N is the state unfolding complexity of the target sub-net,M is the complexity of the other parts of the TPN,n is unfolding times of target sub-net by the reduction and refinement,δ is the complexity introduced by the substitute sub-net;it is expected (and often the case according the early test results) that1 ≤ δ � N.



FMS: Scalability Test (Boniol and Lauer)

The latency functional chain is enlarged by increasing the number of NDB. Each latencyfunctional chain traverses P NDB, i.e. 2P + 3 functions. P = 1...11.

L1 =req1−−−→ KU1

wpId1−−−−→ FM1query1−−−−→ NDB1

query2−−−−→ ...queryP−1−−−−−−→ NDBP−1

queryP−−−−→ NDBP

answerP−−−−−→ NDBP−1

answerP−1−−−−−−−→ ...answer2−−−−−→ NDB1

answer1−−−−−→ FM1wpInfo1−−−−−→ MFD1

disp1−−−→(1)

NDB/Fun.Prop. Val. (ms) S/T (after R.) Reduction Time

(s)Analysis Time (s) Solving Time (s)

wct bct wct bct wct bct wct bct

1/7 75.2 450.4 9/10 8/9 38.049 2.484 1.860 40.533 39,9092/8 125.2 750.4 9/10 8/9 57.876 2.656 1.883 60.532 59,7593/9 275.2 1050.4 9/10 6/5 79.813 2.812 2.079 82.625 81,892

4/10 375.2 1350.4 9/10 6/5 102.500 2.906 2.079 105.406 104,5795/11 425.2 1650.4 9/10 6/5 124.987 3.015 2.102 128.002 127,0896/12 575.2 1950.4 9/10 6/5 149.359 2.891 2.196 152.250 151,5557/13 675.2 2250.4 9/10 6/5 169.607 2.953 2.227 172.560 171,8348/14 725.2 2550.4 9/10 6/5 193.329 3.031 2.250 196.360 195,5799/15 875.2 2850.4 9/10 6/5 216.239 3.000 2.211 219.239 218,45

10/16 975.2 3150.4 9/10 6/5 239.953 3.047 2.195 243.000 242,14811/17 1025.2 3450.4 9/10 6/5 263.049 3.188 2.195 266.237 265,244



FMS: Scalability Test

0

50

100

150

200

250

300

1 2 3 4 5 6 7 8 9 10 11

Solving Time (s)

NDB Number

Latency for L1

WCT

BCT


Feedback Analysis Proposal

Outline

1 Introduction








9 Synthesis



Model Verification Feedback

State of the art

Counterexamples in state-class graph are difficult to analyze

Existing approach provide a set of suspicious component withoutparticular ranking factor

Or animate the error trace in the design model.

Abstraction Issue

Abstraction in design model at early phases.

Abstraction in the mapping from design model to verification model.

Abstraction in state class graph.

Proposal

Rank suspicious components using a suspiciousness factor, when asafety property is not satisfied



Fault Contribution & Error Trace

Definition (Fault Contribution)

Fault Contribution CF (t) is a suspiciousness factor to measure the suspicion level of a transitiont. It is used to rank the suspiciousness of transitions on the error traces.

Definition (Error Trace)

For all the states {si} on the path from an initial state s0 to a violation state sv in thereachability graph, all the outgoing transitions of si are considered as error trace π.

0 41 2 3Svt1

t0 t2t3S0

5

8

7

9

t2 t4

t5t4

6t1

π = {t0, t1, t2, t1, t5, t4, t2, t3, t4}



FMS: Failure Analysis for Latency Property

The bct for latency is 75.2 ms. If we want to check that it is 75.201 ms,the analysis gives the following results:

FunctionFaulty contribution Rank

Rank Var Rank Var %r0 r3 r5 r7 r0 r3 r5 r7

FM1 10,04 9,14 1,46 0,32 1 1 3 2 0,6875 0,0859375MFD1 5,64 5,00 4,91 1,13 2 3 1 1 0,6875 0,0859375KU1 4,98 5,00 4,06 0,16 4 2 2 3 0,6875 0,0859375NDB 5,45 0,58 0,25 0,16 3 6 5 3 1,6875 0,2109375

KU1 FM1 comm 1,03 0,99 0,05 0,03 5 5 6 5 0,1875 0,0234375NDB FM1a comm 1,03 0,12 0,05 0,03 6 7 6 5 0,5 0,0625FM1 MFD1 comm 1,00 1,00 0,99 0,03 8 4 4 5 2,6875 0,3359375FM1 NDB comm 1,01 0,12 0,05 0,03 7 7 6 5 0,6875 0,0859375


Synthesis

Outline

1 Introduction








9 Synthesis


Synthesis

Synthesis

Property-driven proposal

Minimizing verification semantics by

Semantic mapping from UML-MARTE to TPN.

Specification of real-time requirements by property patterns.

Verification and computation of real-time property by observers.

Property-specific reduction of state space.

Feedback analysis proposal

Ranking suspicious faulty elements based on data mining of failure scenarios.

Prototype toolset

Development of toolset prototype (30264 lines of Java code using Eclipse ModelingFramework).

Experiment

Application to FMS case study and test of scalability.


Synthesis

Synthesis

Property-driven proposal

Minimizing verification semantics by

Semantic mapping from UML-MARTE to TPN.

Specification of real-time requirements by property patterns.

Verification and computation of real-time property by observers.

Property-specific reduction of state space.

Feedback analysis proposal

Ranking suspicious faulty elements based on data mining of failure scenarios.

Toolset prototype

Development of toolset prototype (30264 lines of Java code using Eclipse ModelingFramework).

Experiment

Application to FMS case study and test of scalability.


Synthesis

Perspective: Applications

Short term activities

Specify verification-ease property pattern with MARTE CCSL.

Other industrial case studies should be experimented and used to furthervalidate our proposal.

The automated feedback approach can be further experimented andcompared with the existing approaches.

Application to other modeling language

Apply the property-driven and feedback approaches to other end-usermodeling language such as AADL, EAST-ADL or to intermediate languageslike FIACRE.

Redefine semantic mapping.


Synthesis

Perspective: Mapping Semantics

Resource scheduling semantics mapping

Specify scheduling policies such as Earliest Deadline First, FIFO, Fixed Priority, LeastLaxity First, Round Robin, Time Table Driven, etc.

Analysis of schedulability with specific policy.

Proof of correctness of a provided schedule.

Complete the toolset.

Verification of model transformation

A concern with the semantic mapping approach is whether the model transformation iscorrect.

Ideally, map to different formal models and verify if they converge into the same formalsemantics. Lost between semi-formal and formal semantics cannot be proved, onlyassessed using testing and human proof reading.

Verify some important intended behavioral properties conform to the execution semantics,such as RTC in state machine.


Synthesis

Thanks for your attention!


Semantic mapping

Semantic Mapping for State Machine Diagram

Event pool, vent processing and run-to-completion semantics

Flattening semantics:

Converts a nested state machine model to an unnested model to easethe mapping afterwards.Handles regions, states (composite state and submachine state),external transitions, nested pseudostates (entry/exit point,shallow/deep history, and fork/join).Target model only contains simple states, final states, transitions (localand internal), unnested pseudostates.

Mapping semantics

Maps unnested vertices to TPN


Semantic mapping

Semantic Mapping for Composite Structure Diagram

Composite Structure Diagram specifies the internal structure of a class, includingits interaction points to other parts of the system, and the architecture of allparts managed by this class. It is used to explore run-time instances ofinterconnected instances collaborating over communications links.

Coverage Library: UML-MARTE Composite Structure DiagramNode Group Node Type TPN Mapping Coverage

Object

Part√

RoleInterface

√

Port√

CollaborationUse

ConnectionsConnector

√

InterfaceRealizationRole Binding


Semantic mapping

Semantic Mapping for Activity Diagram

Activity modeling emphasizes the sequence and conditions for coordinatinglower-level behaviors.

Coverage Library: UML-MARTE Activity DiagramNode Group Node Type TPN Mapping Coverage

Common Activity Partition

Control

Initial Node√

Decision Node√

Merge Node√

Fork Node√

Join Node√

Activity Final√

Flow Final√

Expansion RegionStructured Activity NodeConditional NodeInterruptible Activity RegionLoop NodeSequence Node

Actions Action√

Object

Activity ParameterCentral Buffer

√

DataStore√

ExpansionInput Pin

√

Output Pin√

ConnectionsControl Flow

√

Object Flow√

Exception Handler


Semantic mapping

Semantic Mapping for State Machine Diagram

The State Machine package defines a set of concepts that can be used formodeling discrete behavior through finite state- transition systems.

Coverage Library: UML-MARTE State Machine DiagramNode Group Node Type TPN Mapping Coverage

Object

Region√

State√

Composite State√

Submachine State√

ConnectionPointReferenceFinalState

√

Pseudostates

Initial√

Deep History√

Shallow History√

Join√

Fork√

Junction√

Choice√

Entry Point√

Exit Point√

Terminate√

ConnectionsExternal Transition

√

Local Transition√


Property specification

Real-Time Property Patterns MetaModel



Verification of Real-Time Properties

[tmin,tmin] [0,0]

[tmax,tmax]

[0,0]

[0,25000]

MFD_exectrSP_ini6r

[0,0]

MFD_exectrSP_ini6r

[0,0] [0,25000]


TPN System

... ...

TPN System

... ...

(a) Best Case of Latency Property (a) Worst Case of Latency Property

TesterA

[0,0] [0,0]

Over>owA Over>owB

TesterA

Over>ow2 2

[tmin,tmin] [0,0]

[tmax,tmax]

[0,0]

[0,25000]

MFD_output_kp1_postRDC_take_k_pre

[0,0] [0,0] [100,25200]


TPN System

... ...

TPN System

... ...

(a) Best Case of Freshness Property (a) Worst Case of Freshness Property

TesterA

[0,0] [0,0]

Over>owA Over>owB

TesterA

Over>ow2 2

MFD_output_kp1_postRDC_take_k_pre

Property Property Value (ms)State/Transition Number Execution Time (s)

Before Reduc. After Reduc. Before Reduc. After Reduc.

LatencySystem N/A 9378/23250 N/A N/A N/AWCT 450.4 67105/145024 9/10 278.313 2.484BCT 75.2 11162/28922 8/9 43.781 3.719

FreshnessSystem N/A 53/85 N/A N/A N/AWCT 316429 259/446 34/44 7.578 3.688BCT 1012 125/202 54/79 7.360 2.125



Freshness Real-Time Requirements

Allows to ensure that a system variable depending on another variable is fresh enough.The time interval between an event at the end of a functional chain and the earliestprevious event of the dependent event at the beginning of the chain.

Example (Freshness Requirement Example)

On the functional chain:pres1−−−→ RDC1

pres1−−−→ ADIRU1speed1−−−−→ FM1

ETA1−−−→ MFD1disp1−−−→, the worst

case of displaying ETA on the screen by MFD must not be superior to 400 ms.

RDC1

M5

M3

pres1[1]

0 25.010

25

ADIRU1 ADIRU1 ADIRU1

FM1

40 100

FM1 FM1 FM1

160 220

5

295

ETA1[1] ETA1[3]

pres1[1]pres1[4]

speed1[2]speed1[3]

KU2 MFD2 KU2 MFD2 KU2 MFD2 KU2 MFD2 KU2 MFD2 KU2 MFD2

5 280M2

speed1[1]

5

ETA1[2]

disp1[4]disp1[1] disp1[2] disp1[3]

pres1[4]

f1

f2



Reduction by Equivalent of Sub-nets

Topology-implicit semantic equivalence (not specific)

Redundant zero-time pattern

Sequential patternIndirect initialization patternShorten cycle pattern

Sequential encapsulation pattern

Behavioral equivalence

Cannot anymore reduce using topology-implicit pattern

Necessary to propose reduction method based on property-specificbehavior



Property Specific Reduction

Algorithms:

Identification of one-way-out pattern, relying on dependency analysis

Identification of generic pattern, relying on dependency analysis

Reduction function, relying on infinity, WCET, BCET observers

Sequential sectionLoop section: search for the loop starting firing occurrence and theloop length

Refinement function, relying on time interval observers

assess the soundness for the sequential and loop section




Event modifier observers:

E E'

TPN

Structure

Observer

E i : i th Occurrence of E

E−k : kth Occurrence Delay of E

E /k : k Times Slower Sub-occurrence of E

I + t: Time Passed Since System Initialization

E + t: Time Passed Since E

SS&SE : Entering and Exiting Events of a State




Predicate observers:

EM

TPN

Structure

!!"#$%%&'()*+%

Observer

O(E i ) = true: E i has occurred

isFinite(E ) = True: Bounded Occurrence of E

Freq(EA) · NA = Freq(EB) · NB : Equivalent Occurrence between EA

and EB

T (EA,EB) > t: Minimum Time Interval between Events

T (EA,EB) < t: Maximum Time Interval between Events

D(s) ≥ t: Minimum Time Duration of State

D(s) < t: Maximum Time Duration of State




Scope modifier observers:

Global

All states, denoted as ABefore E i & After E i

Between EA and EB

Occurrence modifier: Assume that in the state class graph, N(P) is thenumber of states that match the predicate P, N(S) is the number ofstates that match the scope S , and N(P ∧ S) is the number of states thatmatch both the predicate and the scope.

Exist Predicate in Scope:

{N(P ∧ S) ≥ 1 if N(S) > 0;True if N(S) = 0.

Absent Predicate in Scope: N(P ∧ S) = 0

Always Predicate in Scope: N(P ∧ S) = N(S)



FMS: Scalability Test

Same parameters as the work of Boniol and Lauer

The depth of the case study is extended by increasing P.

KU1

MFD1

KUN

MFDN

ADIRU11

FM1

ADIRUNP

FMN

NDB1

X

X X

XX

RDC2

sensor2

keyboard1 displayN keyboardNdisplay1

X

RDC1

sensor1

ADIRU1P

NDBP...

...............

......

...........

...........

.........

ADIRUN1



KL-Divergence

Definition (KL-Divergence)

Kullback-Lerbler (KL) Divergence is a measure that quantifies in bits how close aprobability distribution P = {pi} is to a model (or candidate) distribution Q ={qi}. The KL-divergence of Q from P over a discrete random variable is defined as

DKL(P ‖ Q) =∑i

P(i) lnP(i)

Q(i)(2)

Example

A textual document d is a discrete distribution of |d | random variables, where |d |is the number of terms in the document. Let d1 and d2 be two documents whosesimilarity we want to compute. This is done using DKL(d1 ‖ d2) and DKL(d2 ‖ d1).



TF-IDF

Definition (TF-IDF)

TF-IDF (Term Frequency - Inverse Document Frequency): a numerical statisticwhich reflects how important a term is for a given document in a corpus(collection) of documents. It is often used as a weighting factor in informationretrieval and text mining.

Documents

Error Traces Violation States

Keyword SemanticsTerms

Transitions

Semantic

Contribution

Fault

Contribution

Definition (TC-ITC)

TC-ITC (Transition Contribution - Inverse Trace Contribution),CF (t) = TC (t) · ITC (t).



Testbed

For a given TPN system S(P,R,M)

P are the processes which run infinitely and need a resource beforethe next task (a task is represented by a transition);

R are resource which are shared by all the processes, but onlyaccessible in an exclusive way;

M is a matrix to decide whether process Pi will need to accessresource Rj .

Create deadlock:

Randomly let some processes during some tasks forget to release aused resource.

These tasks are then considered as the error source of system’sdeadlock.



Evaluation of Efficiency

Develop a testbed to randomly generate systems (process andresource 5-20) with deadlocks

Test thousands of case with different number of fault (1-8)

Efficiency:

System EvaluationFault Num. Test Num. Av. State/Transition Average Time (s)

1 400 4949 / 15440 2.90922 517 2428 / 7130 1.12443 500 9884 / 31237 3.35334 402 8811 / 26663 2.59985 303 6756 / 18247 1.21966 504 27094 / 75808 5.0647 757 104857 / 304741 15.00728 100 112306 / 283004 15.0289



Evaluation of Effectiveness

Effectiveness:

EXAM Score: the percentage of statements that have to be examineduntil the first statement containing the fault is reached.

FN.Best Cases Worst Cases Average

EXAM Var. Rank Rank V. EXAM Var Rank Rank V. EXAM Rank

1 0,13335 0,00134 3,25 1,79 0,18603 0,00244 4,33 1,63 0,15969 3,792 0,04229 0,00219 1,1 1,75 0,09574 0,00213 2,11 1,75 0,069015 1,6053 0,02108 0,00106 0,75 1,52 0,05892 0,0009 1,75 1,52 0,04 1,254 0,00722 0,0004 0,26 0,49 0,039 0,00042 1,26 0,49 0,02311 0, 765 0,02044 0,0017 0,83 2,95 0,0478 0,00162 1,83 2,95 0,03412 1,336 0,05369 0,00336 2,46 7,36 0,0766 0,0033 3,46 7,36 0,065145 2,967 0,08857 0,00372 4,61 10,9 0,10822 0,0037 5,61 10,9 0,098395 5,118 0,13091 0,00099 7,3 3,95 0,14905 0,001 8,3 3,95 0,13998 7, 8

Best case EXAM 1% - 13%

Worst case EXAM 4% - 18%

Average EXAM 3% - 16% (rank: 1 - 8)


The UML/MARTE Veri er

Documents

Transcript of The UML/MARTE Veri er