Efficient Model Partitioning for Distributed Model Transformations
Testing Model Transformations
Transcript of Testing Model Transformations
PRUEBAS EN EL DESARROLLO DE SOFTWARE DIRIGIDO POR
MODELOS
Juan de Lara
(trabajo conjunto con Esther Guerra)
modelling&software engineeringresearch grouphttp://www.miso.es
PRUEBAS DE TRANSFORMACIONES DE
MODELOS
Juan de Lara
(trabajo conjunto con Esther Guerra)
modelling&software engineeringresearch grouphttp://www.miso.es
Model-Driven Engineering (MDE).• Meta-Modelling and DSMLs.• Model Transformations.
Testing model-to-model transformations
Conclusions and Open lines
AGENDA
3
MODEL DRIVEN ENGINEERINGIncrease the level of abstraction in software development.
Models are actively used to:• Describe the problem…• Simulate/verify/test…• Generate code for the final application.
Move from the solution space (code-centric) to the problem space:
• Higher levels of productivity and quality.• Less accidental details.
4
MODEL DRIVEN ENGINEERINGFor specific domains:
• Avoid coding the same solutions over and over.
• Families of applications.• Domain-Specific Modelling
Languages.
CodeGenerator
• Modelling, validation and automatic generation of telephony services. 5
MODEL DRIVEN ENGINEERINGFor specific domains:
• Avoid coding the same solutions over and over.
• Families of applications.• Domain-Specific Modelling
Languages.
• Code generation from State-Machines for Railway Signaling Systems. 6
Code generator
MetaEdit+
DOMAIN SPECIFIC MODELLING LANGUAGES
Describe the structure of the domain• Relevant primitives and abstractions.• Relations, features.• Explicit expert knowledge.
7
DOMAIN SPECIFIC MODELLING LANGUAGESDSMLs need not be graphical…
xText 8
MODELS AND META-MODELSThe abstract syntax of DSMLs is defined through a meta-model
• Classes, • Attributes, • Relations.
9
Factory meta-model
Machine
Part
Conveyor
Generator Assembler
inps
outs*
*
* parts
Terminator
1..*
1..*
MODELS AND META-MODELSThe abstract syntax of DSMLs is defined through a meta-model
• Classes, • Attributes, • Relations.
10
«conforms to»
c1:Conveyor
g:Generator
a:Assemblerc2:Conveyor
c3:Conveyor t:Terminator
p2: Partp2: Partouts
outs
inps
inps
outs inps
Factory meta-model
Machine
Part
Conveyor
Generator Assembler
inps
outs*
*
* parts
Terminator
1..*
1..*
OCL CONSTRAINTS
11
Object Constraint Language
Well-formedness rules, which every model should satisfy.
Based on First-Order Logic.
g:Generator
«conforms to»
c:Conveyor
Factory meta-model
Machine
Part
Conveyor
Generator Assembler
inps
outs*
*
* parts
Terminator
1..*
1..*
Factory meta-model
Machine
Part
Conveyor
Generator Assembler
inps
outs*
*
* parts
Terminator
1..*
1..*
context Generator inv: self.inps->isEmpty() and self.outs->size()>0
context Generator inv: self.inps->isEmpty() and self.outs->size()>0context Terminator inv: self.outs->isEmpty() and self.inps->size()>0context Assembler inv: self.inps->size()>0 and self.outs.size()>0…
inps
MODELS AND META-MODELSModels are represented using concrete syntax
• Visual • Textual
No need for a 1-1 correspondence between abstract and concrete syntax elements.
12
asse
MODEL TRANSFORMATIONSModels need to be manipulated for:
• Simulation.• Optimization/refactoring.• Generating another model.• Generating code.
in-place transformations
13…
MODEL TRANSFORMATIONSModels need to be manipulated for:
• Simulation.• Optimization/refactoring.• Generating another model.• Generating code.
SourceTarget
14
model-to-modeltransformations
MODEL TRANSFORMATIONSModels need to be manipulated for:
• Simulation.• Optimization/refactoring.• Generating another model.• Generating code.
15
MMsrc MMtar
Msrc Mtar
Transformationdefinition
from to
«conforms to» «conforms to»
Transformationexecution
Transformationdeveloper
Final user
model-to-modeltransformations
MODEL TRANSFORMATIONSModels need to be manipulated for:
• Simulation.• Optimization/refactoring.• Generating another model.• Generating code.
16
Template languages
query andmodel navigation
17
SPECIFYING MODEL TRANSFORMATIONSModel transformations are not normally specified using general purpose languages
Specialized languages for transformation• Declarative/imperative• Visual/textual• Formal/semi-formal• …
LHS RHS
G H
work
18
SPECIFYING MODEL TRANSFORMATIONSModel transformations are not normally specified using general purpose languages
Specialized languages for transformation• Declarative/imperative• Visual/textual• Formal/semi-formal• …
L R
G H
K
D
l r
m k m*
f d
P.O. P.O.
SPECIFYING MODEL TRANSFORMATIONS
19
Model transformations are not normally specified using general purpose languages
Specialized languages for transformation• Declarative/imperative• Visual/textual• Formal/semi-formal• …
operation main() { // Epsilon Object Language …}operation Machine work(){ for (c in self.inps) { var p : Part := c.parts.random(); c.parts.remove(p); delete p; } for (c in self.outs) c.parts.add(new Part); }http://www.eclipse.org/epsilon/
rule Conveyor2Place { from c : FAC!Conveyor to p : PN!Place (tokens <- c.parts->size())}
Model transformations are not normally specified using general purpose languages
Specialized languages for transformation• Declarative/imperative• Visual/textual• Formal/semi-formal• …
SPECIFYING MODEL TRANSFORMATIONS
20
rule Conveyor2Place { from c : FAC!Conveyor to p : PN!Place (tokens <- c.parts->size())}
Model transformations are not normally specified using general purpose languages
Specialized languages for transformation• Declarative/imperative• Visual/textual• Formal/semi-formal• …
SPECIFYING MODEL TRANSFORMATIONS
21
SourceTarget
Model-Driven Engineering (MDE).
Testing model-to-model transformations• Challenges• PaMoMo: A specification language for MTs• Specification-based MT testing
Conclusions and open lines
AGENDA
22
TRANSFORMATION TESTINGCHALLENGES
23
Urgently needed• we have found 1000+ issues in a repository of ~100 ATL model
transformations (http://www.eclipse.org/atl/atlTransformations/)
Generation of input models• (very) costly if done manually• are we testing interesting cases?
Oracle function• is the result correct?
• syntactically (model conformant to target meta-model)• semantically (produces the intended model)
TRANSFORMATION TESTING APPROACHES
24
Black box• Meta-model based:
• Input models derived covering the input meta-model. • SAT solving/model finding
• Oracle? Random testing: just check for crashes/conformance• Do the models allow testing all properties of interest?
• Specification (contract) based:• Input models derived from a high level specification• The specification serves as oracle• Specifications for model transformations are still rarely built (OCL?)
White box• Look at the transformation logic to generate the tests• Oracle?
A SPECIFICATION LANGUAGE FOR MODEL TRANSFORMATIONSPaMoMo: A formal specification language based on algebraic graph transformation
• Pattern-based, declarative• Category of graphs and morphisms• Compilation into OCL for practical purposes
Pre-conditions: (+/-) conditions on input modelsPost-conditions: (+/-) conditions on output modelsInvariants: relations between input/output models
• Constructive (+)• Constraining (-)
25
CONSTRAINT GRAPHS (1)Encoding of (meta-)models as (typed, attributed) graphs
26
p0: Placename=“inp”tokens=2
t: Transition
in
p1: Placename=“outp”tokens=0
name=“tr”out
p0
t
p1
“inp”
2
“tr”
“outp”
0
in
out
tokens
name
name
name
tokens
Place Transitionin
out
intString
tokensname name
type
CONSTRAINT GRAPHS (2)Encoding of (meta-)models as (typed, attributed) graphs
• G= (V, EV, D, ED,
sV: EV V,
tV: EV V,
sD: ED V,
tD: ED D)• Plus algebras for data types
Typing as graph homomorphism• Four commuting functions
27
p0
t
p1
“inp”
2
“tr”
“outp”
0
in
out
tokens
name
name
name
tokens
Place Transitionin
out
intString
tokensname name
type
CONSTRAINT GRAPHS (3)To express conditions on input/output models, we need:
• Two graphs typed w.r.t source/target meta-models• Attributes of graphs are variables• A formula relating variables of both• C=Gs, Gt,
28
p0: Placename=Y
m: Machinename=X
X=Y
SATISFACTION OF CONSTRAINT GRAPHSSatisfaction of constraint graphs is checked by pattern matching
• Finding graph homomorphisms
29
p: Place
p0: Placename=“inp”tokens=2
t: Transitionin p1: Placename=“outp”tokens=0
name=“tr”
out
m1={(p, p0)} m2={(p, p1)}
SATISFACTION OF CONSTRAINT GRAPHSHow to relate constraint graphs and graphs?
30
p: Place
p0: Placename=“inp”tokens=2
t: Transitionin p1: Placename=“outp”tokens=0
name=“tr”
out
m1={(p, p0)}
tokens=XX>0
SATISFACTION OF CONSTRAINT GRAPHS
31
p: Place
p0: Placename=N1tokens=T1
t: Transitionin p1: Placename=N3tokens=T2
name=N2
out
m1={(p, p0)}
tokens=X
1 X>0
2 N1=“inp” T1=2 N2=“tr” N3=“outp” T2=0
A graph can be seen as a constraint graph• Constraint graph homomorphism = graph morphism + (reverse) formula implication (in the
example )2[X/N1]1
Invariants have a bidirectional nature• Forwards/Backwards satisfaction
32
SATISFACTION OF INVARIANTS
p0: Placename=Y
m: Machinename=X
X=Y
Invariants have a bidirectional nature• Forwards satisfaction
33
SATISFACTION OF INVARIANTS
m: Machinename=X
p0: Placename=Y
m: Machinename=X
X=Y
m: Machinename=“m1”
p0: Placename=“m1”
p1: Placename=“m2”
Invariants have a bidirectional nature• Backwards satisfaction
34
SATISFACTION OF INVARIANTS
p0: Placename=Y
m: Machinename=X
X=Y
m: Machinename=“m1”
p0: Placename=“m1”
p1: Placename=“m2”
p0: Placename=Y
EXAMPLES
35
EXAMPLES
36
EXAMPLES
37
EXAMPLES
38
WHAT CAN WE DO WITH AN SPECIFICATION?Generate “reference” implementations [OGLE09]
• Based on graph transformation• Terminating, correct, complete• …but not very efficient
Testing: • Generation of oracles based on QVT [GL13], OCL [GS15]• Generation of input models, combining constraints [GS15]
SAT-based refinement checking w.r.t:• Other specifications, implementations in ATL [BEGL13]
39
[BEGL13] Büttner, Egea, Guerra, de Lara. “Checking Model Transformation Refinement”. ICMT 2013: 158-173[GL13] Guerra, de Lara, et al. “Automated verification of model transformations based on visual contracts”. Autom. Softw. Eng. 20(1): 5-46 (2013)[GS15] Guerra, Soeken. “Specification-driven model transformation testing”. SoSyM 14(2): 623-644 (2015)[OGLE09] Orejas, Guerra, de Lara, Ehrig. “Correctness, Completeness and Termination of Pattern-Based Model-to-Model Transformation”. CALCO 2009: 383-397
40
SPECIFICATION-BASED TESTING
report
selectspecification
coverage criterion
mtunit script
oracle(assertions)
input test models
.xmi
SAT solver model transformation
transformationspecification
.pamomo
transformationimplementation
.etl, .atl…
refers totester mtunit
engine
12
3
designer
developer
Given a specification, generation of:•input models.•OCL assertions. •Test script
FROM BPMN TO PETRI NETS
41
OUR SPECIFICATION LANGUAGE
42
Features of Pamomo:
•Formal, pattern-based, declarative
•Positive and negative requirements
•Compilation into OCL
P(OneStartEvent)
e.outgoing.size()=1
e
Preconditions Postconditions Invariants(conditions on (conditions on (relations between the elementsinput models) output models) in the input and output models)
N(UnconnectedPlaces)
pl.inarcs.size()=0 andpl.outarcs.size()=0
pl
N
t2pl2
+
pl1
t1.name = pl1.name and t2.name = pl2.name
N(ParallelGateway3)
t1
g
t2
+g
43
1. Translation of properties into OCL
GENERATION OF INPUT MODELS
N
t2pl2
+
pl1
t1.name = pl1.name and t2.name = pl2.name
N(ParallelGateway3)
t1
g
t2
+g
Task::allInstances()->exists(t1 | Task::allInstances()->exists(t2 | ParallelGateway::allInstances()->exists(g | t1.outgoing->includes(g) and t1<>t2 and not t2.outgoing->includes(g) )))
pre-1 inv-1
1pos-1
ocli-1oclp-1
44
1. Translation of properties into OCL
2. Select coverage criteria (e.g. generate models that allow testing one property at a time, two properties, no property, etc.)
=> the selected coverage induces a different set of ocl expressions, e.g., property1 and property2, not property 1 and property2, etc.
GENERATION OF INPUT MODELS
pre-1 inv-1
coverage criteria
1
2
pos-1
ocli-1oclp-1
ocli-1 and ocl-j
45
1. Translation of properties into OCL
2. Select coverage criteria (e.g. generate models that allow testing one property at a time, two properties, no property, etc.)
3. Use constraint solver to generate models conforming to the meta-model and satisfying the OCL expressions
GENERATION OF INPUT MODELS
pre-1 inv-1
coverage criteria
1
2
pos-1
ocli-1oclp-1
ocli-1 and ocl-j
constraint solver3
model-1
sourcemeta-model
46
• A test suite is generated from the specification
• It contains a test case for each invariant and postcondition, defining:
• assertion for the invariant or postcondition• input models to test (all but those generated from
expressions negating the invariant, as they satisfy the invariant vacuously)
GENERATION OF ORACLE
pre-1 inv-1
coverage criteria
1
2
pos-1
ocli-1oclp-1
ocli-1 and ocl-j
constraint solver3
model-1
sourcemeta-model
mtUnittest script(+ oracle)
4
transformation
report
TOOL SUPPORT
47
generation
PAMOMO•EMF-based tool.•Textual/visual syntax for patterns.•Generation of input models and test scripts. •Model generation by using the USE solver.
mtUnit•EMF-based tool.•Automates transformation testing.
Model-Driven Engineering (MDE).
Testing model-to-model transformations
Conclusions
AGENDA
48
CONCLUSIONSMDE: increase the level of abstraction in software development by the use of (domain-specific) models.
V&V of the different MDE artefacts (meta-models, transformations, code generators)
Model transformation testing:• Generation of input models• Oracle function
A specification language like PaMoMo can be used for both
49
CURRENT LINESStatic analysis of (ATL) model transformations
• Type inference• Quick fixes
DSLs for model mutation• Primitives to generate mutants of models conformant to
arbitrary meta-models• Transformations are also models
Model transformation reuse• Transformations used with different meta-models they were
initially built
50