An overview on the Testing and Test Control Notation version 3 (TTCN-3) Helmut Neukirchen Háskóli...

An overview on the Testing and Test Control

Notation version 3 (TTCN-3)

Helmut Neukirchen

Háskóli Í[email protected]

http://www.hi.is/~helmut

Helmut Neukirchen: TTCN-3

2

Outline

1. About me

2. What is TTCN-3?

3. TTCN-3 example

4. Test automation with TTCN-3

5. Conclusions


3

Helmut (í. Helmút) Neukirchen: Born and educated in Germany: University of Aachen,

University of Lübeck, University of Göttingen. PhD in Computer Science.

Since 2008: Associate professor for Computer Science and

Software Engineering at University of Iceland. Working in the field of software quality and distributed

systems, in particular testing distributed systems and quality of tests.

Test methodology, test languages, test tools, test metrics, test “smells”, test refactoring, test patterns, standardisation of tests.

If you have interesting testing problems for students to work on as part of their BSc, MSc, PhD project: contact me!

Office in Tæknigarður, best contacted via e-mail: [email protected]

1. About me


4

2. What is TTCN-3? Testing and Test Control Notation version 3 Test specification and test implementation language.

Test cases specified at an abstract level. Hardware and operating system independent.

Can be turned into automated test cases by compiling into executable code, adding an adaptation layer.

Standardised technology: European Telecommunication Standards Institute (ETSI). International Telecommunication Union (ITU).

Strength in functional black-box testing of distributed systems. However, applicable to various other domains, levels, and types of

tests: Embedded systems, distributed systems, pure software systems. From unit test to acceptance test level. From functional to non-functional (real-time, load, performance) tests.


5

3. TTCN-3 example:Scenario Black-box test of a message-based weather service:

Purpose of a very simple example test case: Send weather request for Reykjavik to SUT. Check that weather response is received containing:

location “Reykjavik” any date, temperature within a reasonable range (-60..+60°C) wind speed within a reasonable range (0..100m/s).

System Under Test (SUT)Test System

Weather serviceTest case

1. Weather request(“Reykjavik”)

2. Weather response(“Reykjavik”, 25.11.2009, 3°C, 9m/s)


6

TTCN-3 example:Test architecture

Weather service example: SUT=Weather service, Simple example test case: only one test component needed.

However, for some load test, we could, e.g., run the same test case in parallel on multiple test components to generate some load.

System Under Test (SUT)

Real Test System Interface

Test

Syst

em

Abstract Test System Interface

Test component

InOut

Specified and implemented using TTCN-3 language

Test adapter (implemented using Java, C, C++ etc.)

Black-box

Ports

Test component

Test component

Test behaviour is

executed by a test

component.

TTCN-3 supports distributed, parallel test

components.


7

TTCN-3 example:Definition of data types and test data

module WeatherServiceTestSuite {

type record weatherRequest { charstring location } template weatherRequest ReykjavikWeatherRequest := { location := "Reykjavik" }

type record weatherResponse { charstring location, charstring date, integer temperature, integer windVelocity } template weatherResponse ReykjavikWeatherResponse := { location := "Reykjavik", date := *, temperature := (-60..60), windVelocity := (0..100) }

Definition of data types.

Templates=Definition of test

data.

Definition of test data: powerful matching

mechanism allows fuzziness (wildcards, ranges, regular

expressions).


8

TTCN-3 example:Definition of ports and interfaces

type port weatherPort message { in weatherResponse; out weatherRequest; }

type component TestComponentType { port weatherPort weatherOffice; }

Definition of a port used for communication: allowed messages (=data types).

Definition of a test component type: ports that may be used by test behaviour running on

an instance of that component type.


9

TTCN-3 example:Definition of test behaviour

testcase testWeather() runs on TestComponentType {

weatherOffice.send(ReykjavikWeatherRequest);

alt { [] weatherOffice.receive(ReykjavikWeatherResponse) { setverdict(pass) } [] weatherOffice.receive { setverdict(fail) } } } // End of testcase

control { execute(testWeather()) }

} // End of module

Test component type used to run test case

behaviour.Send message specified by template

via port.

Two expected alternatives:Either expected response (i.e.

matched by template) is received (PASS)

or any other response is receive (FAIL).

Specification of order (and, e.g., conditional execution) of

test cases.


10

TTCN-3:Further concepts Not presented in the previous example:

Procedure-based communication, Timer and catching timeouts, Distributed testing using parallel test components, Concepts from general-purpose programming

languages: Conditional statements (if-then-else, for, while, …), Local variables, Functions, Parametrisation.


11

4. Test automation with TTCN-3 TTCN-3 test cases are abstract:weatherOffice.send(ReykjavikWeatherRequest)makes no statement

how the actual communication shall be implemented: UDP datagram? SOAP over http? SMS over GSM? …?

how the data to be communicated shall be encoded: XML? Proprietary ASCII-based format? Proprietary binary format? …?


12

Test adaptation Automatically executable

TTCN-3 test suite = Compiled TTCN-3 test

cases + CoDecs (encode data to be

sent to SUT / decode datareceived from SUT)

+ SUT adapter

(communication with SUT) +

Platform adapter (operating system specific)

Test tool user interfaceTest tool user interface

SUT Adapter Platform Adapter

Test Control

System Under Test (SUT)

[compiled]TTCN-3 Executable

CodecsLogging

TTCN-3 Test System

Note: TTCN-3 execution tools come with default test control, logging, and platform adapter.


13

TTCN-3 & Re-usability Separation of concerns (abstract test

case description vs. concrete adaptors): Re-use TTCN-3 test cases for different SUTs:

Same, unmodified weather service test case can be used for SOAP/http-based system under test, for SMS/GSM-based system under test, etc.

Re-use adaptors for different test cases: A SOAP/http SUT adaptor can be used for testing

all kinds of Web services, an SMS/GSM SUT adaptor can be used for testing all kinds of mobile services, etc.


14

TTCN-3 tools Tools for, e.g., editing, compiling, deploying,

executing, debugging TTCN-3: Telelogic Tester (IBM/Rational/Telelogic, Sweden), TTworkbench (Testing Technologies, Germany), OpenTTCN Tester for TTCN-3 (OpenTTCN, Finland) TTCN-3 toolbox (Danet, Germany), Exhaustif/TTCN (Métodos y Tecnología, Spain), MessageMagic (ELVIOR, Estonia), TTCN-3 Express (Fraunhofer FIRST/Metarga, Germany). TRex TTCN-3 Refactoring and Metrics tool (University of

Göttingen/University of Iceland, Germany/Iceland) Standardised language and test-tool interfaces:

No vendor lock-in!


15

5. Conclusions TTCN-3 benefits:

Standardised test technology supporting automated tests. Well documented, many tools, no vendor lock-in.

Compatible with any test methodology and test management approach.

Fosters re-use. Mature technology used by industry and standardisation.

Nokia, Ericsson, Siemens, Motorola, Huawei, ETSI, WiMax forum, Open Mobile Alliance, Automotive Open System Architecture, …

TTCN-3 standard freely available (via http://www.ttcn-3.org): ETSI European Standard 201 873, ITU-T Recommendation Z.140

TTCN-3 risks: Heavyweight approach (may only pay off for big, distributed systems):

Compilers are expensive (>1000€). Another language and technology to learn. While Unit Testing is possible (and done) with TTCN-3,

I would in most cases recommend a xUnit framework for Unit Testing.


16

Thank you for your attention! Make sure to use the right tools!

An overview on the Testing and Test Control Notation version 3 (TTCN-3) Helmut Neukirchen Háskóli...

Documents

Transcript of An overview on the Testing and Test Control Notation version 3 (TTCN-3) Helmut Neukirchen Háskóli...