TTCN-3 Test Case Generation from arbitrary traces Capture & Replay Bogdan Stanca-Kaposta & Theofanis...

TTCN-3 Test Case Generation from arbitrary traces

Capture & Replay

Bogdan Stanca-Kaposta & Theofanis Vassiliou-Gioles (Testing Technologies)

Copyright Testing Technologies 2012. Confidential Information. All Rights Reserved. More Information at www.testingtech.com.©

Agenda

The Motivation

The Technical Challenges

Capture & Replay for TTworkbench

Demo

Q & A

2


Motivation(s)

TTCN-3 UsersWe have a TTCN-3

framework, where do the test cases come from?

IssuesManual test case creation tedious taskError proneDomain Knowledge required

Non-TTCN-3 UsersWe see a scenario and we

would like to replay it later

IssuesWhat means replay w.r.t. testing?What is the test system? What is the system under test?

3

The desire to capture observed communication in test scripts suitable for test automation


The Idea

4

Automatically generate

test case out of traces

Reproduce & Analyse

Situation from the Field (w/o

SUT)

Automate existing

manual tests

Fast IOP Trace Analysis

(IMS/TETRA/ITS)

Codec tests


What is a Trace?

5


What is a Trace?

The observed communication is captured (stored) in a trace (file)

Trace containsCommunication between two or more entitiesMessage exchangeOne or more protocols

6


A Generic Communication Setup

7

A1

A2

B1

B2

.

.

.

.

.

.


The SetupBut Who is Who?

A1 B1

A1 SUT

SUT B1

IOP1 IOP2

SIM1 SIM2


The Setup From Message to Templates

A1 B1

INVITE sip:[email protected] SIP/2.0Call-ID: [email protected]: <sip:[email protected]:5061;transport=UDP>Content-Disposition: sessionContent-Length: 150Content-Type: application/sdpCSeq: 2 INVITEFrom: "TESTER1" <sip:[email protected]>;tag=1169Max-Forwards: 70To: <sip:[email protected]>Via: SIP/2.0/UDP 127.0.0.1:5061;branch=z9hG4bK5704

...

template INVITE_Request T1 := { requestLine := { method := “INVITE”, requestUri := “sip:[email protected]”, sipVersion := “SIP/2.0”}, msgHeader := { callId := “[email protected]”, contact := …, cSeq := “2 INVITE”, fromField := "TESTER1 …" , toField := <sip:[email protected]>, via := …., maxForwards := { fieldName := MAX_FORWARDS_E,

forwards := 70 }, .... allow := omit, ...


The Technical Challenge

It is somehow easy to generate TTCN-3 templates from each message, but

the challenge is to obtain usable and maintainable test data and test scripts

Variable informationFrom run to run, perhaps unpredictableFrom message to message, perhaps constant perhaps dependend

Variable but constant informationShould be configurable before execution

White noiseLot of information that is potentially irrelevant for a particular test scenario but inevitable for the test execution

10


The Technical Challenge

It is somehow easy to generate TTCN-3 templates from each message, but

the challenge is to obtain usable and maintainable test data and test scripts

Variable informationFrom run to run, perhaps unpredictableFrom message to message, perhaps constant perhaps dependend

Variable but constant informationShould be configurable before execution

White noiseLot of information that is potentially irrelevant for a particular test scenario but inevitable for the test execution

11

Protocol and/or domain knowledgehas to be included!

The SolutionAutomated TTCN-3 Test Generation and Execution

12


Example Scenario

13

SIP

User A calls User B

User B accept call


Example Scenario

14

SIP

Use a common Monitor Tool to capture messages e.g. Wireshark

Trace File


Example Scenario

15

Trace File TTworkbench Execution

Put the Trace File into TTworkbench ….

Generate the captured Test Scenarios…

Replay your previously captured Test Scenarios full automatically


Example Scenario

16

Trace File TTworkbench Execution

Put the Trace File into TTworkbench ….

Generate the captured Test Scenarios…

Replay your previously captured Test Scenarios full automatically


Know-How Integration

Manual definitions in a configuration file, toExtract existing data and use the same values in the message flow (sequence numbers, call IDs, …)Define uninteresting fields for ignoring at matching time

Automatically extract similar data from templates to create “better” templates and structures

Reuse existingType systems and codecsTest adapters and component definitionsTest functions and Alt-Steps


DEMO!

18


Key Benefits

Speed up functional and interoperability testingBy reducing manual task

Automate testingGenerate your test, instead of implementing it manuallyExecute them full automatically Get your detailed test reports

Quality Optimization Test case analyzer reduces errors in test cases Fast reusability and easy extension of already defined tests

19


Supported Capturing Sources & Requirements

Wireshark The most popular network analysis and capturing tool

Jpcap Service The remote capturing service provides capturing on remote machines

Pcap Files Trace files created beforehand

TTworkbench logs (TLZ)Analyze the logs without an SUT

Capturing Console Input and Output

Capturing of data via serial connections (RS232, USB) SSH, TELNET Local Console

WindowsWinPCAP

Linux/UnixLibpcap at least version 1.0

Recommendation: Install Wireshark

Includes the installation of WinPCAP and LibPCAP

20


Further information

Testing Technologiestestingtech.comfacebook.com -> Testing-Technologies

Capture & Replaytestingtech.com/products/ttworkbench.phptestingtech.com/download/datasheets/CaptureAndReplay.pdf

21


Contact

22

Bogdan Stanca-Kaposta

Testing Technologies

[email protected]

Theo VassiliouTesting Technologies

[email protected]

TTCN-3 Test Case Generation from arbitrary traces Capture & Replay Bogdan Stanca-Kaposta & Theofanis...

Documents

Transcript of TTCN-3 Test Case Generation from arbitrary traces Capture & Replay Bogdan Stanca-Kaposta & Theofanis...