Automatic Test Generation from here until the end (of my Phd.) University of Geneva Levi Lúcio SMV...

Automatic Test Generationfrom here until the end (of my Phd.)

University of Geneva

Levi Lúcio

SMV & LGL @ Les Diablerets

2SMV & LGL @ Les Diablerets

Presentation Plan

The overall picture; Ideas I would like to explore in my

thesis; Implementation issues; Conclusion.


Presentation Plan




The Overall Picture

We do Model-Based test generation : Given a model (CO-OPN / Fondue) of an SUT… … we want to generate meaningful tests using

that model.

TestCases

Model SUT


The Overall Picture (2)

Main challenges for a test set: Coverage of infinite execution paths (loops) Coverage of infinite input values

My predecessors and Didier developed a theory for coping with this problem: By stating hypotheses about the behavior of the

SUT in order to reduce the exhaustive test set; If the hypotheses hold on the SUT, the final test

set will be finite and valid.


Test Generation FlowDoes P satisfy SP?

P SP

Test Selection(Hypotheses H Application)

Execution of P using T

OracleP satisfies, or not, T!

P does not satisfy SP Undefined P satisfies SP!

P satisfies, or not, H

Test Set T

no inconclusive yes

no yes

Correction of P

TestProcedure

The validity of thetests is decided

by executing them against the model

prototype!


Modeling Language Integration

The idea is to integrate the test theory with the CO-OPN language;

CO-OPN allows building specifications based on three different kinds of modules: Algebraic Data Types (Adts) - types +

operations Class modules - state and coordination Context modules - coordination but no state


Presentation Plan




Test Language

The test language is a language for expressing constraints on the operation sequences and operation’s parameters that compose tests;

It includes predicates to: Limit the size and type of the operations’

sequences; Choose input values from their domain:

All values One value One value per equivalence class


Test Language

Mobile Phone Example

A (simple) mobile phone is composed by: A card containing (among other things) the personal identification

number (PIN) to activate the phone. The card gets blocked after 3 wrong attempts at inserting a PIN.

A phone controller holding (among other things) the state of the phone, which can be: Off Standby (waiting for the correct PIN) Idle (ready to receive or make calls)

In our specification the user can perform three actions: Turn on the phone; Insert a PIN number, Reset the card when it becomes blocked


Test Language

Constraints for the Phone SpecConstraintSet NatelBehaviors;

Interface

Constraints insertPins; block; reachOn;

Body

Constraints nWrongPins;

Use Boolean; Pin;

Axiom [] in nWrongPins; f in nWrongPins => f . HML(<insertPin(newPin(1 1 1 1)),pinResult(false)>) in nWrongPins; f in nWrongPins, depth(f) <= 3 => HML(<turnOn,null>) . f . HML(<m,g>) in insertPins; depth(f) <= 3 => HML(<turnOn,null>) . f . HML(<resetCard,null>) in block; depth(f) <= 3 => HML(<turnOn,null>) . f . HML(<insertPin(newPin(1 2 3 4)),phoneReady>) in reachOn;

Variables f : HML; m : method; g : gate;

End NatelBehaviors;

Exported constraints

Locally defined constraints

Imported Algebric Data Types

Variables used in constraint definition Constraints’Semantics


Integration and Unit Testing

We always perform Black-box testing… … but it can be done at different levels of

the model!

Coopn Model - outer context SUT interface

Class 1 Class 1

Class 2

Subsystem

Class 3

Class 3


Tests are created Top-Down

The engineer startsdefining tests for

the full system andends defining testsfor the individual

classes (refinement)

The SUT is testedbottom up by firstapplying the teststo the individualclasses or to the

smallest subsystems

Model SUT


Test Drivers and Oracles

Tools are available to execute tests on Web Pages or Graphic Gui;

Could we integrate with Junit (Unit Testing)? TTCN/3 is a solution for integration/system

testing… UML 2.0 test profile? A homogeneous solution doesn’t seem to

exist - I’m not sure I can tackle this…


Testing Fondue Specifications

The idea is to translate Fondue into CO-OPN so that the test facilities can be used (Luis’ presentation);

Can OCL be translated into CO-OPN behavior?

Will the translated model be “testable” with the constraints’ language?

Should we already express test intentions at the Fondue level? - are these intentions already present anyway?


More ideas…

We would like to manage behaviors of the SUT which cannot be calculated in test generation time: Example: an e-banking application proposes a challenge

as part of the authentication mechanism. How do we calculate the reply to the challenge in order to login successfully during the test?

Solution: Introduce the possibility to have “external” functions that will perform calculations in testing time.

Predefined “templates” for the constraint language. Example: reach at least one output for each operation in the module’s interface.


Presentation Plan




Presentation Plan


thesis; Implementation issues; Technical challenges; Conclusion.


Implementation Issues

CO-OPNSpec

Constraint Modules

Semanticexplorer:- Java- Prolog

Validated tests


Implementation Issues (2)

Unfortunately the tests cannot be fully calculated syntactically: Easy to calculate input values for each

operation… But how to calculate the outputs? Worse, how to calculate inputs in function of

outputs? The Java prototype is one solution, but only

works “forward”; Prolog works “forward” and “in reverse”…


Implementation Issues (3)

But… representing the state of a real model in Prolog may be too heavy.

Resolution may become too slow! Will it be possible to “reverse” all operations

without falling in Prolog resolution traps? Possible solutions:

Implement several resolutions for each of the tackled problems;

Be less ambitious in the constraint language; Use CLP technology to treat certain types.


Implementation Issues

Under Thought

Generation of object parameters for dynamic OO models;

Mechanisms for decomposing CO-OPN’s operations in classes of behavior: According to CO-OPN’s semantics!


Presentation Plan




Conclusion

There are many ideas, but how many of them are feasible in the remaining time?

Development is being done now, more experimenting with Prolog and CLP is necessary…

I am now focusing on the core of the test generation;

Remaining time for issues to be selected.

Automatic Test Generation from here until the end (of my Phd.) University of Geneva Levi Lúcio SMV...

Documents

Transcript of Automatic Test Generation from here until the end (of my Phd.) University of Geneva Levi Lúcio SMV...