Testing Generation at UPenn
Model-Based Test Generation
Temp. Prop. Translator
ControllerModel Checker----------------------
Witness generator
1 Æ ……Æ ni
i ² i
Concretizer
TS={1,……, n}
Specification Model
Implementation
Tester demand/Specification
Test Suite
Test Setting = temporal properties +
specification + implementation
Coverage Criteria
Testing Generation at UPenn
Model-Based Test Generation
Overview: Use witness-ready model checker to automate the
test generation. Flexible: properties can mimic the traditional coverage
criteria, or can be directly specified by tester. Efficient: Efficiency inherent from checker’s search
engine. Effective: fine-tune test by specifying extra constrains
f=G ( (begin ) F end) Æ (setValue ) F move)) Add: f1=F (setValue): Test fÆ f1
Current research Testing discrete systems.
Mimicing traditional coverage criteria Is in general LTL property “testable”?
Testing hybrid systems. Randomized simulation approach Reachability checker-assisted approach.
Testing Generation at UPenn
Testing discrete systemsGiven: Test setting = LTL/9 LTL + the
specification+ Blackbox implementation.Problem: Currently testing properties is limited
to 9LTL with eventuality only.Question: is there a test for “F( G( a ! Xb))”. Require to test all the possible executions May require a test with infinite length.
Testing Generation at UPenn
Property-Coverage Testing Synthesizing test suites for 9LTL property.
1. Is the property “E GF a” testable?1. No finite trace can be attest to this property.
2. If the number of states in blackbox is bounded by n,
1. A trace for 9 LTL + the specification is rational: ().
2. A infinite trace () can be cut to ()n
3. Buchi tree automaton-based model checker can be used to generate rational traces.
Synthesizing test suites for LTL property.1. LTL can be translated to a set of interesting 9 LTL
properties.1. E( GF( a) Æ F(G(a ) X b)) is an interesting property
for F(G(a ) X b)) 2. Each interesting 9 LTL property focuses on
testing a particular portion of LTL formula.
Testing Generation at UPenn
Testing Hybrid System: Phase I
Randomized test generator=Randomized Simulator+ Coverage Checker. 1. Local ramdomization, gobal strategy.
1. Stay or jump2. Where to jump3. How long to stay
2. Gobal ramdomization1. Aborting/Continuing on current trace.
Mode Adf/dt=1
a: True:f=0b: 1· f<3:m=1
c: 2 · f<4:m=2
Mode Bdw/dt=1
Mode C
d: 2 · f<5:m=4
Testing Generation at UPenn
Testing Hybrid System: Phase I
1. Heuristic search1. Uncovered neighbor first2. Syntax-based distance matrix (Shortest distance to uncovered state/location)3. Open question: Make local decision based global information/history.
1. deciding the weight for outgoing transitions based on the history (What should we learn from a failed search).
2. Deciding the duration to stay in a mode.2. Current status: a working version of randomized test generation is written on CHARON simulator.
Testing Generation at UPenn
Testing Hybrid System: Phase II
SystemModeling
CHARON(Model) Flatten hybrid model
Concretizer
Implementation
Test Suite
Set of predicates
Coverage criteria
Bad set
Reachability Checker
Yes w/ TraceSimulation/refinment
NO w/ more predicates
YES
No
Testing Generation at UPenn
Intelligent simulatorIntelligent simulator=simulator+ property checker
(monitor)1. Verification as the byproduct of simulation
1. LTL Property encoded as the monitor1. MEDL: A subset of LTL, has been applied to Java running-
time monitoring.
2. Monitor advances when the simulation proceeds.3. Open problem: LTL with eventuality only is easy, but
how about other formula requires circularity reasoning. 1. Need to remember the states traversed to sense the loop.
1. Difficult because the domain of continuous variables are dense.
2. The search is tailored by the property.1. A transition “measure” has the priority higher than
others if the property is G(measure => X (home)). 2. Most interesting simulation trace: Covering as many
parts of property as possible using less steps.
Top Related