Michael Ebner Institute for Informatics · University of Göttingen · Germany TTCN-3 Test Case...

Michael EbnerMichael Ebner

Institute for Informatics · University of Göttingen · GermanyInstitute for Informatics · University of Göttingen · Germany

TTCN-3 Test Case GenerationTTCN-3 Test Case Generation

from Message Sequence Chartsfrom Message Sequence Charts

ISSRE2004 - WITULISSRE2004 - WITUL

2TTCN-3 Test Case Generation from MSCs

Overview

Concept Mapping of MSC to TTCN-3 Related Concepts

Graphical Format of TTCN-3 (GFT)UML Testing Profile (UTP)Comparison

Summary


Customer

Supervisor

Salesperson

Establish Credit

Telephone Catalog

Shipping Clerk

Check Status

Place Order

Fill Orders

Use Cases

Product

Customer

Organization

Service

OrderHeader

LineItem

Account

*

order

item

1

ShoppingCart

1

1

productItem

product

serviceItem

service1

1

client

client

customerOrder

organizationOrder

cart

cartOrder

accountOrder

account

11..*

1..*1

1

0..1

0..*

1

{xor}

{xor}

Classes

Unified Modeling Language (UML)

selectAmount

enterAmount

ok

abort

otherAmount

amount

abort

ReadAmountSM

aborted

Statessd N

s[u]:B s[k]:B

m3()

m3()

Sequences

Testing

Unit Test

System Test

IntegrationTest


UML-based Test Specification

generate

map

TTCN-3Data

IDL

UML

TTCN-3Behaviour

MSC

map

Class Diagram

Sequence, Activity,and Interaction Overview

Diagram

map

use

WSDL

CORBA


Overview



Summary


MSC Test Purpose Description

SUT-P1 SUT-P2

PCOs(Points of Control and Observation)

SUT(System Under Test)

PCO_X PCO_Y PCO_Z


Message Descriptions

<message type>

<message value>

[<message type>]

<template reference>

[<message type>]

<derived template>

Matching mechanisms are only allowed for messages to be received from the SUT.


Mapping I

MSCBasic MSC

HMSC

instance axis

note, comment, text comment

chart name

instance kind

flow control

instance creation

TTCN-3Testcase(s)

Testcase calls (control part)

represents a port (PCO)

comment(ttcn-3,begin,position)

Testcase name

component name or SUT

call or getreply procedure

create component


Mapping II

MSCMessage

Flow control

(Synchronise) Condition

Timer

Action

Coregion

Alternative, Option,and Exception

Loop

MSC reference

TTCN-3Send or receive message

call or getreply procedure

Synchronising

Timer (start, stop, timeout)

TTCN-3 Statements

Interleave

Alternative (incl. sync.)

For or While loop (incl. sync.)

Function call


Example I – Mapping of MSC

testcase InresRTexample() runs on inres { var float sendTime1:=-1.0; var integer iterator1:=0; for (iterator1:=0; iterator1<100; iterator1:=iterator1+1) { if (sendTime1==-1.0) { sendTime1:=self.now+0.01; } else { resume(sendTime1); sendTime1:=sendTime1+0.01; } log(TimestampType:{self.now,"IDATreq1"}); ISAP.send(IDATreq:{"data"}); MSAP.receive(MDATind:{DT,number,"data"}); log(TimestampType:{self.now,"MDATind2"}); MSAP.send(MDATreq:{AK,number}); } setverdict(pass); stop;}


Example II – Mapping of MSCtestcase InlineAlt()

runs on syminres { ISAP1.send( ICONreq ) ISAP1.send( IDATreq ) ISAP2.receive( ICONind ) alt { [] ISAP2.receive( IDATind ){} [] ISAP1.receive( IDISind ) { ISAP1.send( ICONreq ) ISAP1.send( IDATreq ) ISAP2.receive( ICONind ) ISAP2.receive( IDATind ) } } ISAP2.send( IDISreq ) ISAP1.receive( IDISind )}

msc InlineAlt

ICONreq

ISAP1 syminres ISAP2

ICONind

altIDATind

IDISind

ICONreqICONind

IDATind

IDISreqIDISind

SUT

IDATreq

IDATreq


Overview



Summary


Overall Picture of TTCN-3

ASN.1 Types& Values

Other Types& Valuesn

TTCN-3CoreLanguage

TTCN-3User

TabularFormat

GraphicalFormat

Presentationformatn

IDL Interfaces& Types

XML Types

IDL Interfaces& Types


GFT I Provides a test specific profile for

MSC-2000 Defines generic extensions

Value returning GFT ReferencesHyper GFTHybrid GFT

Defines test specific extensions, e.g.Port instancesSpecial Messages for instance handlingType and Template definition for messagesTrigger and check messageVerdicts as special conditionsDefaults


GFT II

Does not use GatesGeneral OrderingTime ConceptsMSC Reference ExpressionInstance Decomposition

Assumed data types are TTCN-3 data types


testcase InresRTexample() runs on inres{ var float sendTime1:=-1.0; var integer iterator1:=0; for (iterator1:=0; iterator1<100; iterator1:=iterator1+1) { if (sendTime1==-1.0) { sendTime1:=self.now+0.01; } else { resume(sendTime1); sendTime1:=sendTime1+0.01; } log(TimestampType:{self.now,"IDATreq1"}); ISAP.send(IDATreq:{"data"}); MSAP.receive(MDATind:{DT,number,"data"}); log(TimestampType:{self.now,"MDATind2"}); MSAP.send(MDATreq:{AK,number}); } setverdict(pass); stop;}

TIMEDGFT-Example


Overview



Summary


The UML 2.0 Testing Profile (UTP)

Profile: UML extension mechanism

Initiated late 2001

Make UML applicable for SW testing by enhancing UML with test concepts

Enhances reuse of design model

Adopted by OMG in June 2003 and since April 2004 standardized

Allow automated test generation with UML


UTP ConceptsArchitecture

conceptsBehaviour concepts

Dataconcepts

TimeConcepts

SUT Test objective Wildcards Timer

Test components Test case Data partition Time zone

Test suite Defaults Coding rules

Test configuration Verdicts

Test control

Arbiter

Utility part

UML 2.0 based meta-model


: SlaveApplication

: SlaveBTRoaming

sd BT-Roaming

: LocationServer

<<testcomponent>>

sa: SlaveApplication

<<SUT>>sr: Slave

BTRoaming

<<testcomponent>>

co: Test-Coordinator

sd TestRoaming_noWarning():Verdict

Master1_BT: Master BT-HW

Master2_BT: MasterBT-HW

: SlaveBT-HW

<<testcomponent>>

hw: SlaveBT-HW

<<testcomponent>>

M1: Master

<<testcomponent>>

M2: Master

<<testcomponent>>

M3: Master

ref Connect_To_Master(M1)data

refGood_Link_Quality

con_request

con_requestcon_request

con_request

con_acceptcon_confirm

con_confirmcon_confirm(M3)

data

refBad_Link_Quality

refmakeList

ref

roamingList(RList,Slave)

roamingList(RList)

<<validationAction>>

pass

<<default>>Conf_Default

T(6s)

T

<<default>>

sd Conf_Default

self

alt

*

con_confirm(*)

<<validationAction>>fail

<<validationAction>>inconc


UTP Outlook

Upcoming tool support: IBM/Rational: Eclipse Project Hyades Telelogic: Tau G2 Microsoft: Visual Studio Daimler/FOKUS/Nokia/TestingTech:

ITEA-Project TT-Medal


Overview



Summary


Comparison I

all three concepts were based on MSC support of graphical test development differ in deepness of test features

MSCGFTUTP


Comparison II Test Case Generation from MSCs

centred on test purposes is focussed on manual, scenario testing abstracts from details no support of all TTCN-3 features direct mapping to test case templates requires less knowledge about TTCN-3 timedTTCN-3 support

GFT MSC based representation format for TTCN-3 has to present all TTCN-3 features timedTTCN-3 support not desired as graphical specification language


Comparison III

UTP representation and specification of all

aspects independent of TTCN-3 no complete mapping to TTCN-3enhanced testing conceptsbetter integration to UMLmeta-model for testingwide application area (mapping to JUnit and

TTCN-3)


Summary

MSC as powerful means for test purpose and test case specifications

MSC like presentation formats

Test case generation from MSC test purposes

Michael EbnerMichael Ebner

Institute for Informatics · University of Göttingen · GermanyInstitute for Informatics · University of Göttingen · Germany

TTCN-3 Test Case GenerationTTCN-3 Test Case Generation

from Message Sequence Chartsfrom Message Sequence Charts

WITUL 2004 (IEEE Conference ISSRE)WITUL 2004 (IEEE Conference ISSRE)


END


Backup


MSCTest Purpose

InteractiveSimulation

ManualSpecification

AutomaticComputation

Test PurposeSpecification

Test SuiteProduction

Test CaseGeneration

State SpaceExploration

DirectTranslation

DynamicBehaviour

Constraints

TTCN-2-Test Suite-Export

TTCN-2-Test Suite

Test Architecture

Autolink - Process


Example III – Mapping of MSCtestcase NamingServiceExample()

runs on Naming { Client1.call( bindingContext ) Client1.call( bind ) Client2.call( resolve ) alt { [] Client1.catch(InvalidName) { Client1.call( bind ) Client2.catch(NotFound) Client2.call( resolve ) } [else] {} }

Client1.call( unbind )}

msc NamingServiceExample

bindContext

Client1 Naming Client2

opt

resolve

InvalidName

bind

resolve

unbind

SUT

bind

NotFound


GFT Example – timedTTCN-3

Test behaviour

TTCN-3 TIMEDGFTmapping

TIMED

Core notation Presentation format

MSC

Test purpose

generate


Test Purpose Example


Generated Testcase



: SlaveApplication

: SlaveBTRoaming



: SlaveBT-HW

con_request

con_accept

con_confirmcon_confirm

data

data

get_link_quality

quality(good)

get_link_quality

quality(bad)

ref newMaster:=Search_NewMaster()

sd BT-Roaming

{0.5s..0.5s}

: LocationServer

ref RList:=makeList(Slave)

<<testcomponent>>

sa: SlaveApplication

<<SUT>>sr: Slave

BTRoaming

<<testcomponent>>

co: Test-Coordinator

sd TestRoaming_noWarning():Verdict



: SlaveBT-HW

<<testcomponent>>

hw: BT-HW

Master1_R: Master

BTRoaming

Master2_R: Master

BTRoaming

<<testcomponent>>

M1: Master

<<testcomponent>>

M2: Master

connected(Slave)

roamingList(RList,Slave)

roamingList(RList)

<<testcomponent>>

M1: Master

<<testcomponent>>

M2: Master

<<testcomponent>>

M3: Master

Connect_To_Master(M1)

Good_Link_Quality

Bad_Link_Quality


Use of SDL and MSC

SDL-2000MSC-2000

Interpreted as

C++ etc.

Product

used tocreate

used toproduce

SpecificationIdeas

Customer

modelled byDesign

Engineers

Design

Implementation Software

Engineers

modelled by

made as

executesas

Interpreted as

Tools

transformed by


ooTTCN-3

UML-based Test Specification I

generate

map

UML

TTCN-3

IDL MSC

map

use

WSDL

CORBA


Graphical Format for TTCN-3 (GFT)

Provides a test specific profile forMSC-2000Uses a subset of MSC-2000Defines test specific extensionsDefines general extensions

Formerly called Test Sequence Charts (TSC)

Supports different forms to represent test cases


UTP Methodology Wrap-Up Test preparations

Define a test package with a test suite. Determine SUT and import its implementation. Determine test objectives.

Test configuration Assign roles: SUT, test components, utility part. Attach stereotypes.

Test cases Group instances. Attach stereotypes. Define re-usable test fragments to functions. Assign verdicts. Specify timers. Specify default behaviours


Use of Inline Expressions and HMSCs

ProblemSeveral alternative test sequences referred to

in one diagram.

Two possibilitiesAll alternatives are put in one test case.For each alternative a separate test case is

generated.

StrategyAlternatives described in form of HMSCs are

translated into separate test cases.Alternatives described by inline expressions are

put into the same test case.


ToC der Dissertation

1. Introduction2.A General Metamodel For Testing3.Graphical Test Development (with case

study) With TTCN-3 With U2TP (UML Based Metamodel) Relationship and Mapping Between TTCN-3

and U2TP

4.Graphical Realtime Testing with TTCN-3 and U2TP (with case study)

5.Conclusion


UTP Outlook

FTF submission: April 2004 Upcoming tool support:

IBM/Rational: Eclipse Project Hyades Telelogic: Tau G2 Microsoft: Visual Studio Daimler/FOKUS/Nokia/TestingTech:

ITEA-Project TT-Medal Automated test generation





and U2TP


5.Conclusion


Customer

Supervisor

Salesperson

Establish Credit

Telephone Catalog

Shipping Clerk

Check Status

Place Order

Fill Orders

Anwendungsfälle

Product

Customer

Organization

Service

OrderHeader

LineItem

Account

*

order

item

1

ShoppingCart

1

1

productItem

product

serviceItem

service1

1

client

client

customerOrder

organizationOrder

cart

cartOrder

accountOrder

account

11..*

1..*1

1

0..1

0..*

1

{xor}

{xor}

Klassen

Unified Modeling Language (UML) …

selectAmount

enterAmount

ok

abort

otherAmount

amount

abort

ReadAmountSM

aborted

Zuständesd N

s[u]:B s[k]:B

m3()

m3()

Sequenzen

Entwickler

Integrator

System-integrator

Testen

Einheiten-test

Systemtest

Integrations-test


... und Testen

Entwickler

Integrator

System-integrator

Customer

Supervisor

Salesperson

Establish Credit

Telephone Catalog

Shipping Clerk

Check Status

Place Order

Fill Orders

Anwendungsfälle

Product

Customer

Organization

Service

OrderHeader

LineItem

Account

*

order

item

1

ShoppingCart

1

1

productItem

product

serviceItem

service1

1

client

client

customerOrder

organizationOrder

cart

cartOrder

accountOrder

account

11..*

1..*1

1

0..1

0..*

1

{xor}

{xor}

Klassen

selectAmount

enterAmount

ok

abort

otherAmount

amount

abort

ReadAmountSM

aborted

Zuständesd N

s[u]:B s[k]:B

m3()

m3()

Sequenzen

Testen

Einheiten-test

Systemtest

Integrations-test


TTCN-3 Übersicht

ASN.1 Typen& Werte

Andere Typen& Werten

TTCN-3Kern-sprache

TTCN-3Benutzer

TabellarischesFormat

GraphischesFormat

Präsentations-formatn

IDL Typen& Werte


MSC and TTCN (1)

Systemspecification

Assessment

Test results

Executabletest case

Abstracttest case

Test purpose

MSC

MSC-based PresentationFormat for TTCN-3

TTCN-3


MSC and TTCN (2)

TTCN-3

ATS

1. Generation

2. Presentation

Tracing

Validation

Compilation

MSCSimulator

Compilation

MSC- 2000

TTCN-3

ETS


Information in MSC Test Purposes

Descriptions of messages to be sent toand received from the SUT

Synchronization information required dueto the partial order semantics of MSC

Structure information


MSC Objectives

is a scenario language supports complete and incomplete

specifications is a graphical language is widely applicable can be used throughout the engineering

process supports structured design is often used in conjunction with other

methods and languages


The Tutorial Example

INRES - Initiator-Responder protocol an abridged version of the Abracadabra

protocol used for academic studies and illustrative purposes.

a connection-oriented, asymmetric communication protocol

a medium with uncorrupted data, loss of data, misordering


INRES Structure

Initiator Responder

Medium

User

ISAP1

MSAP2

ICONreq,IDATreq,IDISreq,ICONconf,IDISind

MDATreq,MDATind

SUT (System under Test) = Initiator and Medium


Objectives(3)

msc mi_synch1_conc1

MTC

MTCType port ISAP1type

ISAP1

?port MSAP2type

MSAP2


MSC Test Purpose Development View

Test System

System Under Test


Objektorientierung für TTCN-3

Kein Objekttyp und Portinstanzen können nicht übertragen werdenObjektreferenzen in IUT nicht

handhabbar Einführung von Klassen Erweiterung vorhandener Konzepte

Vererbung von Templates Hierarchie von Komponenten


Autolink - INAP CS-2 Testzweck

Schnitt-stellenSystem

Empfangs-Ereignis

Sende-Ereignis

Postambel

Präambel


• Testing and Test Control Notation 3 (TTCN-3)– Universelle Testbeschreibungssprache– Standardisierung 2001 durch ETSI – Entwicklung eines TTCN-3-Syntaxcheckers

• Verbesserte Modellierung der Testumgebung– Praxis: Standard-Werte und benutzerdefinierte Signale– Ineffektive Suche, Nachbearbeitung von Testfällen – Lösung: Symbolische Ausführung– Entwicklung eines Prototypen

Zusammenfassung und Ausblick

Ausblick


Ausblick

TTCN-3 Beispiel


Weitere Arbeiten

Testing and Test Control Notation 3

• Erweiterung der Anwendungsgebiete– Testarten: Interoperabilitätstest, Performanztests, ... – Architekturen: CORBA, APIs, ...

• Moderne, allgemeine Sprachkonzepte– C/C++-ähnliche Notation– keine OSI-spezifischen Elemente – Erweiterte Kommunikationskonzepte– Dynamische, verteilte Testarchitekturen

• Standardisierung 2001 durch ETSI – Entwicklung des ersten freien TTCN-3-Syntaxcheckers





and U2TP


5.Conclusion


MSC

SDL/MSC

TTCN

How to Test?

Systemimplementation

Systemspecification

Assessment

evaluate

Test results

execute

Executabletest case

implement

Abstracttest case

specify

Test purpose

identify


MSC and TTCN (1)

Systemspecification

Assessment

Test results

Executabletest case

Abstracttest case

Test purpose

MSC

MSC-based PresentationFormat for TTCN-3

TTCN-3


MSC and TTCN (2)

TTCN-3

ATS

1. Generation

2. Presentation

Tracing

Validation

Compilation

MSCSimulator

Compilation

MSC- 2000

TTCN-3

ETS


Basic MSC in a Nutshell

MSC diagram

MSC heading

Condition

Output event

Input event

Instance

Instance end

msc INRES_Example1

ICONreq

ICONconf

User

Idle

Connected

ISAP1

MDATind

MDATreq

Message to the environment


HMSCHMSC Start

MSC Reference

ConditionAlternative

Loop

Connection PointConnecting

msc INRES_Behaviour

Idle

Connected

DataTransfer

Disconnecting

Connected


New challenges for TTCN (1)

Corba and Internet TestingDynamic configurationsAddressing entities inside the SUTSynchronous communication

Real-time- and performance- TestingHandling of load generatorsPossibility to repeat test runs

New and improved design techniquesSmooth interworking with

SDL'2000, MSC'2000, ASN.1'99, UML and IDL

Adaptation to different application areas


New challenges for TTCN (2)

Module and Group concept „New “ data types Synchronous communication Less OSI concepts and terms Introduction of templates Textual base


System Under Test (SUT)

Real Test System Interface

Abstract Test System Interface

MTC PTC

Test system Connected PortsIN

IN

IN

IN

OUT

OUT

OUT

OUT

Mapped Ports

TTCN-3 Test Architecture (2)


TTCN-3 Examplefunction internetUser () runs on InternetType { timer Tvisit; var integer orderNr; var PizzaDeliveryType thePizza; activate(StandardDefault, InternetDefault); Tvisit.start(OverallDuration); P2.call(StandardINetOrder, maxConnectTime) { [] P2.getreply (INetOrder:* value *) -> value orderNr { verdict.set(pass) } [] P2.catch (INetOrder, reasonType : *) { verdict.set(fail)

} [] P2.catch (timeout) { verdict.set(inconc) } }


The Development of MSC

dialects of sequence chartsbefore1992

instances, messages, events, conditionsMSC-92

HMSC, references,general ordering, inline expressionsMSC-96

object orientation, data, time, method calls

MSC-2000


Object Orientation

MSC documents define instance kinds instance kinds are types suited for object orientation

Inheritance inheritance of instance kinds means inheriting all contained

instances and MSCs

Virtuality Virtual MSCs means that MSCs may be redefined in

specialized instance kinds

Close correspondance with object orientation in SDL and other object-oriented languages such as Java


Data in MSC-2000

MSC has no data language of its own! MSC can make use of data languages such that

fragments of your favorite (data) language can be used (C, C++, SDL, Java, TTCN-3, ... )

MSC can be parsed without knowing the details of the chosen data language

the data language strings have no unnecessary extra delimiters

Mappings to data languages have not yet been made up to any user could be standardized in the future


Use of Data

Initiator Responder

MDATreq(DT(isdu,seq_no)=:dt)

MDATreq(no:=ACK(rec_no))

msc Transmission (isdu: ISDUType; seq_no, try: Integer);

Check_Data(rec_dt)

seq_no:= 1-seq_no Static variables have MSCs as homes (cannot change binding within the MSC)

Binding Left-hand or right-hand in actions with messages with MSC references

Medium

MDATind(dt=: rec_dt)

MDATind(rec_no:=ACK(rec_no))

when (seq_no != rec_no) and (try <4)

Transmission(isdu,seq_no,try+1)


Method Calls

Method CallsBlocking (with reply)Non-Blocking

Method calls can be super-imposed

Used to describe the control flow between instances


Summary & Outlook (2)

MSC as powerful means for test purpose and test case specificationsin particular, extensions of MSC-2000 such

as data support and method calls are used Test case generation from MSC test

purposesdirect mapping to test case templates

MSC like presentation format for TTCN-3


Enjoy your lunch

and

have a nice trip home!


The UML 2.0 Testing Profile

Profile: UML extension mechanism

Initiated late 2001

Make UML applicable for SW testing

Adopted by OMG in June 2003

At time it is revised by the Finalization Task Force (FTF)


Outlook

FTF submission: April 2004 Upcoming tool support:

IBM/Rational: Eclipse Project Hyades Telelogic: Tau G2 Microsoft: Visual Studio Daimler/FOKUS/Nokia/TestingTech:

ITEA-Project TT-Medal Automated test generation


Outline & Introduction

Test behaviour

TTCN-3 GFTmapping

TIMED


Presented at Testcom 2002(Submitted for standardisation.)

MSC

Test purpose

generate TIMEDGFTmapping

1. Instrument functional testcases to generate timestamps,

2. Execute testcase,

3. Apply evaluation functions to the generated timestamps,

4. Assign a test verdict.

MSC

Test purpose



TIMEDTTCN-3=TTCN-3+Time Extensions

1. Local clock – read (self.now), wait (resume)

2. Logfile – log, sort, retrieve of structured data

3. Timezones – specification of clock synchronised test

components

4. Conf-Verdict – passconfinconcfail


Test Case Generation

Test behaviour

TTCN-3 TIMEDGFTmapping

TIMED

1. Using MSC as test purpose:

• MSC instances represent SUT and TTCN ports.

• MSC messages depict TTCN send/receive.

2. Generating corresponding TTCN test cases.


MSC

Test purpose

generate


Generating TIMEDTTCN-3 from MSC

Pure functional test cases: Proven approach and implemented by TTCN-2 tools, e.g.:Autolink (Telelogic)

ptk (Motorola)

New: generating TIMEDTTCN-3 test cases.

MSC allows to attach time constraints to events:Absolute time constraints,

Relative time constraints,

Time measurements.


Test Purpose Example


Generated Testcase



Graphical Test Case Specification

Test behaviour

TTCN-3 GFTmapping

TIMED

1. New GFT symbols.

2. One-to-one mapping between

TIMEDTTCN-3 core notation and TIMEDGFT.


MSC

Test purpose



TIMEDGFT-Example

testcase InresRTexample() runs on inres{ var float sendTime1:=-1.0; var integer iterator1:=0; for (iterator1:=0; iterator1<100; iterator1:=iterator1+1) { if (sendTime1==-1.0) { sendTime1:=self.now+0.01; } else { resume(sendTime1); sendTime1:=sendTime1+0.01; } log(TimestampType:{self.now,"IDATreq1"}); ISAP.send(IDATreq:{"data"}); MSAP.receive(MDATind:{DT,number,"data"}); log(TimestampType:{self.now,"MDATind2"}); MSAP.send(MDATreq:{AK,number}); } setverdict(pass); stop;}


TIMEDGFT

Overview: New GFT symbols and

mapping TIMEDTTCN-3 TIMEDGFT:

TIMEDTTCN-3 TIMEDGFT


TIMEDGFT (continued)

TIMEDTTCN-3 TIMEDGFT

Michael Ebner Institute for Informatics · University of Göttingen · Germany TTCN-3 Test Case...

Documents

Transcript of Michael Ebner Institute for Informatics · University of Göttingen · Germany TTCN-3 Test Case...