Combining Model-Based Testing and Machine...
Transcript of Combining Model-Based Testing and Machine...
![Page 1: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/1.jpg)
Combining Model-Based Testing and Machine Learning
Roland GROZ Université Grenoble Alpes (COMUE), LIG (Laboratoire d’Informatique de Grenoble) France
TAROT Summer School 2016
![Page 2: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/2.jpg)
In a nutshell:
2
Testing a system is somehow LEARNING the behaviour of a system
n when writing a model is not an option !
n Model-based testing…
Problem: test orderly to learn correct & “complete” behaviour
![Page 3: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/3.jpg)
Outline n Motivation: why learning ? n ML & Soft. Engineering n Seminal algorithm: L* (Angluin 87) n Enhancements for various issues
¨ Counter-example processing ¨ Tree-based (quotient algo) ¨ No Reset ¨ Integration ¨ EFSM
n Related work 3
![Page 4: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/4.jpg)
Soft. Engineering trends n MDE & MBT
¨ Growing trend in some industries (e.g. embedded) ¨ Derive design, code and tests (MBT) ¨ Models = 1st class citizens
n Non formal (e.g. Agile) ¨ Dominant & growing trend ¨ Absence of (formal) models ¨ Or pb maintaining spec <-> model ¨ Often Test Driven Dvt (TDD)
4
TAROT J
Rest of the world L
![Page 5: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/5.jpg)
MBT in software development
5
Models
Design
Implementation
Unit tests
System tests
Integration tests
Acceptance tests
The chore of writing test cases can be GREATLY relieved if n Formal specs and models are AVAILABLE n Test cases can be AUTOMATICALLY generated from
models (MBT)
Hum ! Matched by the chore of • Writing MODELS • Maintaining them
![Page 6: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/6.jpg)
6
Component Based Software Engineering
Components selection
Integrated System
Requirement Analysis
High Level System Design
n Rapid Development n Reuse Components n Reduce cost n Flexibility n Ease of integration
Components selection & Integration
![Page 7: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/7.jpg)
7 Integrated System
How do I perform system behavioural analysis? How do I identify integration problems ?
Typical Issues in System Dvt
Behaviours Interactions
Validation
Understanding a System of
Black Box/3rd party Components is a challenge
MODELS could help !
But what if NO model ?
![Page 8: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/8.jpg)
MDE & MBT in the reverse n MDE assumption
¨ Start from model, formal spec
¨ Models = 1st class citizens J
n Test Driven Development (XP, Agile…) ¨ Tests are spec: 1st class citizens
¨ Formal models ? No way ! L No time…
n Proposed approach ¨ Derive models from tests, & combine with MBT
n = LEARN models from tests
¨ CHALLENGE: Reconcile Test-Driven (or code-driven) dvt with Models
8
![Page 9: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/9.jpg)
Principle
9
Component Model Tests
Scenario 1
Scenario 2
Reverse engineer
Generate Requirements
Partial, incremental and approximate models
![Page 10: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/10.jpg)
10
Main Technical Goals
n Reverse Engineering ¨ Understanding the behaviours of the black box components
n by deriving the formal models of the components/system n Can also serve documentation purposes (tests for doc)
n System Validation ¨ Being able to derive new systematic tests ¨ Analyzing the system for anomalies
n by model checking (wrt properties) n by developing a framework for integration testing of the system of
black box components
![Page 11: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/11.jpg)
Objections Answers n Model is derived from
bugged components ¨ Derived tests will
consider bug=feature
n Incremental: stopping criterion ?
n Unit vs system ¨ Combining model-
checking & learning ¨ Integration testing will
reveal errors n Tunable approximated
model of system n Key notion: counter-
examples
11
![Page 12: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/12.jpg)
Outline n Motivation: why learning ? n ML & Soft. Engineering n Seminal algorithm: L* (Angluin 87) n Enhancements for various issues
¨ Counter-example processing ¨ Tree-based (quotient algo) ¨ No Reset ¨ Integration ¨ EFSM
n Related work 12
![Page 13: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/13.jpg)
Various types of Machine Learning n Artificial Intelligence (& datamining)
¨ Ability to infer rules, recognize patterns ¨ Learning from samples ¨ E.g. neural networks
n Two major techniques (among others) ¨ Statistical inference from collection of data ->
e.g. Weka tool in (data) testing
¨ Grammatical inference of language from theoretical computer science
13
![Page 14: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/14.jpg)
Pioneering inference in SoftEng
n [Peled 1999] Black Box Checking ¨ Using L* + Vasilievski’s W-method for Model
Checking BB components n [Steffen, Hagerer 2002] Model generation
by Regular extrapolation ¨ Applied to testing of telecom switch
n Picked up from 2003 by Dortmund, NASA, Uppsala, Grenoble, Nijmegen, KTH…
14
![Page 15: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/15.jpg)
15 15
Learning languages from samples "Learning from given positive/negative samples”
• Finding a minimum DFA (Deterministic Finite Automaton) is NP-HARD
– Complexity of automaton identification from given data. [E. Gold 78]
• Even a DFA with no. of states polynomially larger than the no. of states of the minimum is NP-Complete
– The minimum consistent DFA problem cannot be approximated within any polynomial. [Pitt & Warmuth 93]
• Probably Approximately Correct (PAC) – A theory of the learnable. [L.G. Valiant 84]
![Page 16: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/16.jpg)
16
Active learning (Query learning)
n Active Learning ¨ "Learning from Queries”: inference algorithm can query an
oracle of the language
¨ Angluin's Algorithm L* [Angluin 87] n Reference algorithm n Two types of queries: membership, equivalence n Learns Deterministic Finite Automaton (DFA)
in polynomial time
¨ Applied in formal Software Engineering n Black Box Checking [Peled 99] n Learning and Testing Telecom Systems [Steffen 02-03] n Protocol Testing [Shu & Lee 08] n …
Dana Angluin Yale University
![Page 17: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/17.jpg)
Outline n Motivation: why learning ? n ML & Soft. Engineering n Seminal algorithm: L* (Angluin 87) n Enhancements for various issues
¨ Counter-example processing ¨ Tree-based (quotient algo) ¨ No Reset ¨ Integration ¨ EFSM
n Related work 17
![Page 18: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/18.jpg)
18
Concept of the Regular Inference (Angluin's Algorithm L*)
The Algorithm L*
Input Alphabet Σ
Final Minimum DFA Conjecture
Oracle
Black Box Machine
Assumptions: n The input alphabet Σ is known n Machine can be reset
Complexity : O( |Σ| m n² ) n |Σ| : the size of the input alphabet n n : the number of states in the actual machine n m : the length of the longest counterexample
![Page 19: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/19.jpg)
19
Our Context of Inference (testing s/w)
The Algorithm L*
Input Alphabet Σ
Final DFA Conjecture
Oracle
Black Box Machine
System of Communicating
Black Box Components
n Components having I/O behaviors n I/O are structurally complex (parameters) n Formidable size of input sets
Enhanced State Machine Models Mealy Machines Parameterized Machines
More adequate for complex systems DFAs may result in transition blow up
Test Strategies and heuristics Learned Models can be used to generate tests to find discrepancies
![Page 20: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/20.jpg)
20
Preliminaries
n Mealy Machine: M = (Q, I, O, δ, λ, q0) ¨ Q : set of states ¨ I : set of input symbols ¨ O : set of output symbols ¨ δ : transition function ¨ λ : output function ¨ q0 : initial state
n Input Enabled ¨ dom(δ) = dom(λ) = Q × I n Q = {q0, q1, q2, q3}
n I = {a,b} n O = {x,y}
Running example
![Page 21: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/21.jpg)
21
The Algorithm LM*
Input set I
Final Mealy Conjecture
Oracle
Black Box Mealy Machine
Assumptions: n The input set I is known n Machine can be reset n For each input, the corresponding output is observable
Mealy Machine Inference Algorithm The Algorithm LM*
![Page 22: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/22.jpg)
22
I={a, b}
Observation Table
• ε is an empty string
Black Box Mealy Machine Component
Build queries row.col submit row.col -> record output <- for col
Basic principles of LM* algorithm
a ε
a b
b aa ab
x x
x y
x x
S (span seq for) States
x y
x x
Discriminating sequences
[a] [ε]
tr1: a/x
tr4: b/x tr2: b/x tr3: a/y
S • I lookahead tail state id
Conjecture: minimal FSM consistent with observations
![Page 23: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/23.jpg)
23
I={a, b}
Observation Table (SM,EM,TM)
• ε is an empty string
Output Queries: s•e, s ∈ (SM∪ SM • I), e ∈ EM
a ε
b
EM
SM
SM • I
Black Box Mealy Machine Component
x
Initialization n SM = ε n EM = I
a b ε
a
• = a / x
x
x y
x x
Mealy Machine Inference Algorithm LM* (1/6) Initialization
![Page 24: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/24.jpg)
24
SM
• ε is an empty string
a ε
EM
a b
b aa ab
x x
x y
x x
SM • I
SM
Concepts: n Closed : All the rows in SM • I must be equivalent to the rows in SM
¨ Same behaviour = known state n Consistency
Mealy Machine Inference Algorithm LM* (2/6) Concept: Closed
I={a, b}
Black Box Mealy Machine Component Observation Table (SM,EM,TM)
x y
x x
![Page 25: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/25.jpg)
25
[a] [ε]
tr1: a/x
tr4: b/x tr2: b/x tr3: a/y
I={a, b}
Black Box Mealy Machine Component
Mealy Machine Inference Algorithm LM* (3/6) Making Conjecture
a ε
a b
b aa ab
x x
x y
x x
SM • I
SM
Observation Table (SM,EM,TM)
x y
x x
tr1
tr3 tr2 tr4
EM
Counterexample: a b a b b a a
component's response: x x x x x x y conjecture's response: x x x x x x x
![Page 26: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/26.jpg)
26
Mealy Machine Inference Algorithm LM* (4/6) Processing Counterexamples
a ε
a b
b aa ab
x x
x y
x x
SM • I
SM
Observation Table (SM,EM,TM)
x y
x x
EM
Counterexample: a b a b b a a Method: Add all the prefixes of the counterexample to SM
a ε
a b
ab aba abab
x x
x y
x x
x x
x x ababb ababba ababbaa aa
x y
x y
x x
x x
b x x
abb x x
abaa x x
ababa y x ababbb x x
![Page 27: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/27.jpg)
27
Mealy Machine Inference Algorithm LM* (5/6) Concept: Consistency
a ε
a b
ab aba abab
x x
x y
x x
x x
x x ababb ababba ababbaa aa
x y
x y
x x
x x
b x x
abb x x
abaa x x
ababa y x ababbb x x
Concepts: n Closed n Consistency : All the successor
rows of the equivalent rows must also be equivalent
n First inconsistency ¨ ε and ab look similar…
but not ε.a and ab.a n Later inconsistency:
¨ ab and aba, but not aba and abaa n …
a
a xy
yy
xx
xx
xy
yy
yy
xx
xy
xx
xx xy
yy xy
Observation Table (SM,EM,TM)
a
a
![Page 28: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/28.jpg)
Complexity : O( |Σ| m n² ) n |Σ| : the size of the input alphabet n n : the number of states in the actual machine n m : the length of the longest counterexample Final Observation Table (SM,EM,TM)
after processing counterexample according to LM*
28
Mealy Machine Inference Algorithm LM* (6/6) Termination: Conjecture = Black Box
![Page 29: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/29.jpg)
Outline n Motivation: why learning ? n ML & Soft. Engineering n Seminal algorithm: L* (Angluin 87) n Enhancements for various issues
¨ Counter-example processing ¨ Tree-based (quotient algo) ¨ No Reset ¨ Integration ¨ EFSM
n Related work 29
![Page 30: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/30.jpg)
Other algorithms derived from L*
n Counter-example processing ¨ Rivest & Schapire (1993)
n Do not add prefixes (avoid compatibility check) n Dichotomic search for discriminating suffix
¨ Complexity falls to O(|Σ|n2 + n log m) ¨ But flawed (Balcazar 97)
n Corrected by Shahbaz, Irfan and Groz (2009): ¨ Suffix1by1
n Only membership queries ¨ Howar: Zulu competition at ICGI 2010
30
![Page 31: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/31.jpg)
31
Processing Counterexamples avoiding consistency checks
a
ε
a b
b aa ab
x x
x y
x x
x y
x x
Observation Table (SM,EM,TM) before processing counterexample
a b b a a Counterexample a b
Add all the suffixes to EM
a
ε
a b
b aa ab
x x
x y
x x
x y
x x
Observation Table (SM,EM,TM) after processing counterexample
baa aa bbaa abbaa xy xxx xxxy xxxxx
yy xxx xxxx yxxxx
xx xxy xxxx xxxxy
xx xxx xxxy xxxxx
yy xxx xxxx yxxxx
All rows remain inequivalent (inconsistency never occurs)
![Page 32: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/32.jpg)
32
Comparison of the two Methods
Final Observation Table (SM,EM,TM) after processing counterexample
according to LM+
Final Observation Table (SM,EM,TM) after processing counterexample
according to LM*
Total Output Queries in LM+ : 64 Total Output Queries in LM* : 86
![Page 33: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/33.jpg)
33
Total Output Queries in LM* : 86 Total Output Queries in LM+ : 64
Observation Table (SM,EM,TM) after processing counterexample
according to LM+
Observation Table (SM,EM,TM) after processing counterexample
according to LM*
Comparison of the two Methods
Complexity of LM*:
O( |I|² n m + |I| m n²)
Complexity of LM+:
O( |I|² n + |I| m n²)
¨ I : the size of the input set ¨ n : the number of states in the actual machine ¨ m : the length of the longest counterexample
![Page 34: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/34.jpg)
Outline n Motivation: why learning ? n ML & Soft. Engineering n Seminal algorithm: L* (Angluin 87) n Enhancements for various issues
¨ Counter-example processing ¨ Tree-based (quotient algo) ¨ No Reset ¨ Integration ¨ EFSM
n Related work 34
![Page 35: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/35.jpg)
Other active learning algorithms
n Other data structures: trees vs tables ¨ Kearns & Vazirani (1994): binary tree
n O(|Σ|n3 + nm)
¨ Z-quotient: tree & quotient automata n Petrenko, Li, Groz (HASE 2014)
¨ TTT n Isberner, Howar, Steffen ( RV 2014)
35
![Page 36: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/36.jpg)
36
Mealy Machine Quotients
n Let Φ be a set of strings from I then ¨ the states s1 and s2 are Φ-equivalent if they produce same
outputs for all the strings in Φ ¨ A quotient based upon Φ-equivalence is called Φ-quotient
Mealy Machine M Φ-quotient of M
Φ = {a, b, ab, ba, bb, bba} q0 and q2 are Φ-Equivalent q1 and q3 are Φ-Equivalent
![Page 37: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/37.jpg)
37
Relation between the Conjecture and the Black Box Machine
Closed (and Consistent) Observation Table (SM,EM,TM)
Conjecture from the Observation Table (SM,EM,TM)
EM
SM • I
SM
EM-Quotient
Black Box Mealy Machine
![Page 38: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/38.jpg)
38
Initial k-Quotient
Machine M
b/0
q0
q3 q2
q1
b/1
a/0 a/1
a/0
a/1b/1
q4
b/0
a/1
b/1
q0,q2 q1,q3
a/1,b/1
a/0,b/0
1-Quotient of M
b/0
q0,q2
q3
q1
b/1
a/0 a/1
a/1 b/1
2-Quotient of M
3-Quotient(M) ≡ M
q0 and q2 are still 2-Equivalent q1 and q3 are 2-Disting. a/1 a/?
q0 and q2 are 1-Equivalent: a/0,b/0 q1 and q3 are 1-Equivalent: a/1,b/1
![Page 39: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/39.jpg)
39
Inferring a k-quotient (example with k=1) a/0 b/0
a/1 b/1 a/1 b/1
a/0 b/0 a/0 b/0
n BFS exploration of traces of increasing length
n Pruning under node k-equiv to another one
n Final step: merging node when trace included, and redirecting transitions a/1,b/1
a/0,b/0
Groz,Li,Petrenko,Shahbaz TestCom 2008 Extended to arbitrary Σ-quotients Σ ⊆ I*
![Page 40: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/40.jpg)
Outline n Motivation: why learning ? n ML & Soft. Engineering n Seminal algorithm: L* (Angluin 87) n Enhancements for various issues
¨ Counter-example processing ¨ Tree-based (quotient algo) ¨ No Reset ¨ Integration ¨ EFSM
n Related work 40
![Page 41: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/41.jpg)
Motivational example
• Reverse-engineer models of Web applications to detect security vulnerabilities
• E-Health app provided by Siemens as a Virtual Machine
41
Learner • single I/O RTT over LAN: < 1 ms • reset=reboot VM: ~1 minute
• Timewise: reset is O(105) RTT in example • Many systems CANNOT be reset AT ALL.
![Page 42: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/42.jpg)
Key difficulties when no reset
n How can we know in which state seq is applied ?
n No backtrack possible to check other sequence
n Losing track: we no longer know from where we apply an input ¨ è localizer procedure
Can we infer a Black-Box machine without reset? 42
![Page 43: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/43.jpg)
Problem, assumptions, result
Groz, Simao et al 2015 n Known bound N on nb of
states: n ≤ N
n Known W-set for BB ¨ Card(W) = p
Algo: polynomial in N <<O(f Np+2) bound but mean O(f N1.9) for p=2
Rivest & Schapire 1993 n Oracle knows BB, can
answer yes or no n Oracle can provide CE
¨ |Largest CE| = m n Known Homing
Sequence for BB Algo: polynomial in n ~O(f m n3)
43
![Page 44: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/44.jpg)
Example: W = {a, b}, N=3
44
1
Localizer seq. L = a5b
a/0 2
a/0 2
b/0 1
a/0 2
a/0 2
q0 {a0,b0}
?
a/1 2
{a1,?}
q1={a1,b1}
b/1 3
L
a/0 2 …
b/0 1 …
q0
a/1 2
b/0 1
q0 q1
b/1 3
b/1 3
q2={a0,b1}
2
3
a/1
b/0
a/0 b/1
b/1
a/0
1
a/1 2
a/0 2 …
b/0 1 q1
L
![Page 45: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/45.jpg)
Example (end)
45
3 a/0
2 b/0
1 … 2 q0 q1
a/1 2
a/0 2
b/0 1
b/1 3
3 a/0
2 b/0
1 b/1
3 b/1
3
2
3
a/1
b/0
a/0 b/1
b/1
a/0
1
L
![Page 46: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/46.jpg)
It pays off to learn without reset !
46
Learner • single I/O RTT over LAN: <1ms • reset=reboot VM: ~1minute
Cost of single reset ~sequence of 105 inputs • If we know W of 2 elements, it is FASTER to learn
WITHOUT reset ! • If we know W of 3 elements, it may still pay off
depending on number and length of queries
finite state
![Page 47: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/47.jpg)
Outline
n Motivation: why learning ? n ML & Soft. Engineering n Seminal algorithm: L* (Angluin 87) n Enhancements for various issues
¨ Counter-example processing ¨ No Reset ¨ Integration ¨ EFSM
n Related work 47
![Page 48: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/48.jpg)
Integration testing
n Popular issues ¨ Architecture, testability ¨ Integration order, stubbing ¨ Interoperability testing
n Combining integration with Model Learning ¨ Unit learning (1st approach) ¨ Deriving integration tests from combined
learned models
48
No formal models L
![Page 49: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/49.jpg)
Integration exposes models
49
X / a
Component U: IU={X,y} Component V: IV={a,b}
y / b a / y b / y
U V
X a, b
y
C
Z
X / Z
b / C
y / a
a / C
Composed Model: X a y b y a y b y a y … Livelock !
![Page 50: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/50.jpg)
Analysing the problem
n Artefact ? ¨ Possibly: models are approximate
n Check sequence on real system 1. If Livelock confirmed: report error 2. If Real sequence differ: counter example
50
![Page 51: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/51.jpg)
Integration provides counter-examples
51
X / a
Component U: IU={X,y} Component V: IV={a,b}
y / b a / y b / y
U V
X a, b
y
C
Z
X / Z
b / C
y / a
a / C
Real : X a y b y a y b y b C
Composed Model: X a y b y a y b y a y … Livelock !
-> Refine U model with (projected) counter-example
![Page 52: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/52.jpg)
52
System architecture & assumptions
n System of communicating Mealy Machine Components n Components are deterministic and input-enabled n System has External and Internal i/o interfaces
¨ External interface is controllable ¨ External and Internal interfaces are observable
n Single Message in Transit and Slow Environment
![Page 53: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/53.jpg)
53
Overview (simplified)
Model Extraction
Reachability Analysis
Step 1 Step 2
Inference
Verification
Given: Modular System
Test generation
Step 3
Step 4 Goal: Error
detection
Pb on real
System?
A B c
C b
dD
y, w e
ax
Refinement
Components' Analysis & Testing
Tasks performed on models Tasks performed by testing real components
Legend:
Counter example
![Page 54: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/54.jpg)
54
I. T.
C.E.
Iterations
M1(1)
M2(1)
…
Mn(1)
U. T.
U. T.
U. T.
I. T.
M1(2)
M2(2)
…
Mn(2)
U. T.
U. T.
U. T.
C.E.
I. T.
M1(3)
M2(3)
…
Mn(3)
U. T.
U. T.
U. T.
U. T.: Unit testing I. T.: Integration
testing C.E.: Counter- example
![Page 55: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/55.jpg)
55
Learning & Testing Framework
Step 1: Learn Models
Step 2(a): Compose Models
Step 2(c): Confirm Problem on System
Step 3: Refine Models
Terminate
[compositional problem]
[discrepancy as counterexample]
[problem confirmed]
Step 2(b): Analyze Product
Step 4: Generate Tests from Product
Step 5: Resolve Discrepancy
(exception, crash, out of memory,...?)
Discrepancy trace
[error found]
Product
Product
[problem as counterexample]
No Compositional
Problems
[no discrepancy]
Learned Models
![Page 56: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/56.jpg)
Outline
n Motivation: why learning ? n ML & Soft. Engineering n Seminal algorithm: L* (Angluin 87) n Enhancements for various issues
¨ Counter-example processing ¨ No Reset ¨ Integration ¨ EFSM
n Related work 56
![Page 57: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/57.jpg)
Learning extended FSM n Dealing with boolean variables
¨ Th. Berg, B. Jonsson & H. Raffelt FASE 2006 n Parameterized inputs/outputs
¨ no var, arbitrary I/O functions: Shahbaz 2007 ¨ Var. with equality: Berg, Jonsson… 2008
n With variables ¨ Register automata: Howar et al VMCAI 2012 ¨ With Data Mining inference of guards and
output functions: Li, Hossen, Groz 57
![Page 58: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/58.jpg)
Combining state & data inference n Connecting to Daikon tool, for dynamic invariant
detection ¨ Shahbaz ISOLA 2007 Daikon: inductive inference of functions from samples
y=f(x) M. Ernst (U. Washington) n Weka & FSM inference
¨ Dury & Petrenko: security of Web interface ¨ Li & Groz: EFSM inference Weka: data mining toolset, clustering (U. Waikato)
58
![Page 59: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/59.jpg)
Inferring for security
n Input parameters critical (e.g. Cross site scripting...)
n Storing past values: cookies, session IDs n Non-deterministic values: nonces
n Model: Extended FSM with ND values, and storage
59
![Page 60: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/60.jpg)
Needham Schroeder authentication
60
s0
s1
m1(p1)/ p2 = p1, p3 = ndv,
m2(p2, p3), v3 := p3
s2 m3(p4), p4 = v3/ OK
m3(p4), p4 ≠ v3/ KO
m1(p1)/ p2 = p1, p3 = ndv,
m2(p2, p3), v3 := p3
Extended FSM model of NSPK Responder
[ε] m1(p1)/
p2 = p1, p3 = ndv, m2(p2, p3),
v3 := p3
[m1(5)]
m3(p4), p4 = v3/ OK
m3(p4), p4 ≠ v3/ KO
m3(p4)/ Ω
m1(p1)/ Ω
Inferred EFSM model
![Page 61: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/61.jpg)
State inference // data inference m1 m1
ε (5, m2) (ndv3, m2)
(10, Ω) (ndv3, Ω)
m1(5) (5, Ω), (ndv3, Ω)
(10, KO) (ndv3, OK)
m3(10) (5, m2) (ndv3, m2)
(10, Ω) (ndv3, Ω)
m1(5) m1(5)
(5, Ω), (ndv3, Ω)
(10, KO) (ndv3, OK)
m1(5)m3(10)
(5, Ω), (ndv3, Ω)
(10, KO) (ndv3, OK)
m1 m1 ε
(5, (0, 0, 0, 0) → (5, ndv)), (0, (0, 0, 0, 0) → (0, ndv))
(10, (0, 0, 0, 0) → ω),
(0, (0, 0, 0, 0) → ω)
m1(5)
m3(10) (5, (0, 0, 0, 10) → (5, 600)), (0, (0, 0, 0, 10) → (0, 800))
(10, (0, 0, 0, 10) → ω),
(0, (0, 0, 0, 10) → ω)
m1(5) m1(5)
(5, (5, 5, 900, 0) → ω),
(0, (5, 5, 110, 0) → ω)
(10, (5, 5, 120, 0) → ω), (130, (5, 5,
130, 0) → ω) m1(5)m3(
10) (5, (5, 5, 140, 10) → ω), (150, (5, 5,
150, 10) → ω),
(10, (5, 5, 150, 10) → ω), (160, (5, 5,
160, 10) → ω) 61
![Page 62: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/62.jpg)
Outline
n Motivation: why learning ? n ML & Soft. Engineering n Seminal algorithm: L* (Angluin 87) n Enhancements for various issues
¨ Counter-example processing ¨ No Reset ¨ Integration ¨ EFSM
n Related work 62
![Page 63: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/63.jpg)
Related work
n Active learning in Soft. Eng/Testing ¨ D. Peled (Bar-Ilan): Black Box Checking (1999) ¨ C. Pašareanu (NASA): Assume-Guarantee Proof(2008) ¨ B. Steffen, H. Raffelt (Dortmund): Dynamic Testing via
Automata Learning (2003-2007) ¨ D. Lee & G. Shu (Ohio 2007): Security protocol testing ¨ B. Jonsson, T. Berg (Uppsala): Register automata ¨ K. Meinke (KTH): Learning Based Testing (& model
checking), Congruence on Abstract Data Types ¨ F. Vaandrager, S. Verwer (Nijmegen): Smartcard
63
![Page 64: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/64.jpg)
Related work
n Many other approaches ¨ Specification mining, becoming popular
n May assume code available, often passive n Typical papers:
¨ Ammons (POPL 2002) coined the word ¨ Lorenzoli, Mariani, Pezze (ICSE 2008) ¨ Bertolino, Inverardi (FSE 2009)
n Use of (statistical) Machine Learning in testing ¨ E.g. for test data classification & partition
refinement (Briand 2008) 64
![Page 65: Combining Model-Based Testing and Machine Learningtarot2016.wp.imtbs-tsp.eu/files/2016/02/TAROT16_RGroz.pdf · n Artificial Intelligence (& datamining) ¨ Ability to infer rules,](https://reader033.fdocuments.in/reader033/viewer/2022050400/5f7e0d262f8a7069015b6bb1/html5/thumbnails/65.jpg)
Reference book on learning automata
• For machine learning in
general: • Many references, • e.g. A. Cornéjuols & L. Miclet
• No book as yet for Software Testing & machine learning • Planned April 2017
(Springer): outcome of Dagstuhl seminar 2016
65